Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* remove un-used
* use utc time for slice sync
* use utc time
* refactor
* add regression test
* use utc time
* use utc time
* make utctime required parameter
* add utcTime in slice creation
* assert warm up complete
* check if algorithm is still warmingup
* use exchange tz
* extract binance brokerage
* remove binance tools from main toolbox app
* modify OrderTestParameters constructor
* fix BinanceBrokerage ref config.json
* update symbol properties
* move BinanceFeeModel tests to the right place
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* initial commit
* get history from data providers
* merge history from history providers
* merge slice
* Add merge function in slice
* Update test
* merge aux data
* add data points in _data
* append new data points in original list
* Add test suite for HistoryProviderManager
* reduce complexity
* setup once
* add fake history provider
* doesn't count aux data
* add tests for options
* style changes
* add custom data
* merge rawDataList
* use array of history providers
* optimize
* fix formatting error
* add comment
* add tests
* simplify
* use list
* use abstraction
* split tests
* add test
* use abstraction to create generic enumertor class
* refactor
* accept list of type T
* sync history slices
* use SubscriptionDataReaderHistoryProvider for live
* rename test file
* return empty
* add tests
* cleanup
* address reviews
* optimize
* Fix method definition
* use initial time for basedata
* refactor
* re-use collection
* inherit HistoryProviderBase
* always return HistoryProviderManager
* add tests
* update rawDataList
* reset composer
* consider null elements
* add tests for binary search method
* Follow lean coding style
* revert
* remove binary search method
* convert to field
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Use GC server mode for tests
* Fix daily auxiliary data points emission time
- Due to fillforwarding, in some cases with daily resolution symbol
change events (generically any auxiliary data) would arrive late.
Updating regression test to reproduce the issue. Adding unit test
- Some refactoring and logging improvements
* Address reviews
* Remove old xml docs param
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Add support for live price scaling
- Add support for live trading price scaling for continuous futures.
Adding unit tests
* Move live price scale application. Updating unit tests
- Live trading application of price scaling will happen before fill
forwarding and updating securities real time price. Updating unit
tests to reproduce issue
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Zip entries will be sourced from cache provider
- Option underlying will use SubscriptionDataSource to fetch it's data.
Fixing bug where it would let through an old data point, or miss
sending data through.
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Live subscription enumerator will clone the underlying data set when
live mapping is being done. To avoid issues where IDQH implementations
could reuse a data point with same configurations. Adjusting unit test
to reproduce issue
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Do not send internal SecurityChanges to Algorithm
- Will not send internal security changes to the Algorithm by default.
Following custom security changes filter pattern. Updating regression
algorithms to assert behavior.
- The universe member will know wether it was added with internal
configurations or not
* Address reviews use a separate collection for internals
* Refactor solution. Adding security changes constructor class
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Align universe security and configuration TZ
- Make sure universe security and configuration tz are aligned always.
Adding unit test reproducing issue
* Address review UserDefinedUniverse will use UTC TZ
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* CompositeDataQueueHandler handles duplicate configs
- CompositeDataQueueHandler will handle duplicate subscriptions sharing
the same config. Adding unit tests
* Refactor bug solution after review
- LiveSubscriptionEnumerator will not long perform any symbol mapping
but will just trigger a new subscription call when remapped.
- CompositeDHQ will handling symbol mapping and keep track of these
mapped configs to trigger unsubscribe accordengly. Adding unit tests
* Adjust solution after review
- To avoid breaking uniqueness of SubscriptionDataConfig will be adding
a private `mapped` property that will change equality of mapped
configs to avoid them clashing. Adding unit tests
* Rename IDQH subscription extension methods
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Minor performance improvements
* CompositeDataQueueHandler is UniverseProvider
- CompositeDataQueueHandler implements the UniverseProvider interface.
Adding unit tests
* initial commit
* Follow IDQH implementation
* Expect a list of data handlers from LiveNodePacket
* Return null if can not subscribe
* refctor to add check for subscription
* initialze null
* Add tests
* Check subscribe retuns null/not-null
* cleanup
* Read all required IDQH credentails to job
* Use CDQH to handle all IDQH instances
* constructor abstraction to call from setjob
* use flag
* remove redundant because derived will call initialize on it
* abstract and initialize from setjob
* handle null enumerators
* get creds from data handlers
* handle single data handler value from data-queue-handler
* Fix to support a json array
* Fix missed constructor call
* change access modifier to access from Tests files
* Add test to get brokerageFactory from dataQueueHandler
* Fix init flag to handle all conditions
* Add docs
* initialize from setjob
* Check if websocket open before using
* change defination of initialzie to include tradier
* clean up
* change defination
* fix wrong api key name
* return empty enumerator
* check websocket open before sending request
* check connection before subscribing
* fix to include more cases
* check websocket open before sending request
* Minor refactoring
* reafctor and use IsConnected
* remove unused
* clean up
* Fix test cases
* reverse change
* include config changes
* connect to websocket from setjob
* check websocket connection from setjob
* clean up
* include condition for IDQH that are not brokerage
* Address review
* Add market check condition before subscribe
* Remove deprecated
* Minor fix for deserializing data queue handler
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Continuous Futures Refactor. Live Mappings
- Adding support for live mappings. LiveTradingDataFeed will handle any
symbol mapping at the configuration layer and resubscribe through the
IDQH
- Refactoring continuous futures adding ContinuousFutureUniverse that
will select the currently mapped security
* Minor fixes
- Remove addition of configurations in UniverseSelection step, leave
resposability for universe.
- LiveTradingDF future unit test will only assert slice data for non
internal feeds.
- ContinuousContractUniverse will respect internal option interest
subscription
* Address review
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add BaseDataCollectionAggregatorReader
- Add enumerable for BaseDataCollection
- Add BaseDataCollectionAggregatorReader
- Clean up CollectionSubscriptionDataSourceReader
* Address reviews
- Add new FileFormat that will be handled by the new Reader
- Adding unit tests for the new BaseDataCollectionAggregatorReader
- Some DataSourceReader duplication cleaning up
* Minor adjustment after some live trading tests
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
As most brokers use separate exchange codes for CBOE and CFE, we added the new Market.CFE (previously Market.CBOE was used for both exchanges). This change also avoids hacky code in symbol mappers.
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adds support for ETF constituent universes
* Adds filtering for universe data if it doesn't match the
universe subscription type
* Update mapping for ALL underlying Symbols if
`Symbol.UpdateMappedSymbol(...)` is called. Required to support
constituent ETF universes that might have mapping events
* Delistings of composite constituent universe Symbol will result in
removal of universe securities.
* Added regression algorithms for ETF constituent mappings (C#/Python),
along with data required to run locally
* Refactor universe delistings in SubscriptionSynchronizer -
big thank you to @Martin-Molinero :)
* Address review: update regression algorithms and add explanatory comments
* Address review: add additional checks to delisting regression algorithms
* Adds new regression algorithm testing the addition of a universe
without calling AddEquity() and asserts same behavior
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates
* Regression algos updated and python algorithms moved to DataSource repos
* Fixes failing unit tests
* Add "LiveDataTypes" field to LiveNodePacket
* Adds Initialize() to IDataChannelProvider
* Adds new extension method to convert
HistoryRequest -> SubscriptionDataConfig
* Address review: Add protobuf definitions for Iconic data types
* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo
* Small adjustments to variable naming and documentation
* Move test files to respective DataSource repos
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data
* Updates Unit Tests
* Updates Regression Tests
In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.
* Updates Regression Tests 2
The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`
The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238
Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
- BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`
The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying
* Revert File Update for SPWR and SPWRA
* Fix Regression Tests
Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.
* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]
Uses `trust-constr` method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request
* key -> filePath rename and some cleanup
* Refactor
* Add Organizations Endpoints
* Add some organization api wrapper objects
* Address namespace issue
* Reorganize Api Test into seperate files using one ApiTestBase
* Add Organization tests
* Use capitalized "API" test namespace to reduce amount of file changes
* Add License to test base
* Update /data endpoint functions and response objects
* Update ApiDataProvider Logic
* Handle deserialization of organization products
* Simplify converter
* Only throw for equity requests when not subscribed to map/factor files
* Add missing header
* Make arguement exception
* Api adjustments
* Add Zip factor and map file providers
- Common project will now reference Compression project and not the other way
round.
- Adding Zip FactorFile and MapFile providers
* Refactor FactorFileProvider to use DataProvider to fetch files
* Use resulting MinimumDate in construction of FactorFile
* Nit FactorFile comments and arrangement
* Refactor MapFileProviders to use DataProvider for fetching files
* Refactor ZipFileProvider
* Clean up
* Refactor Backtesting Future/Option chain providers to use dataprovider
* Fixes for data/ endpoints and test adjustments
* Response objects adjustments/cleanups
* ApiDateProvider fixes and testing
* Add LocalZipFactorFileTests
* Update ApiDataProvider download test to verify stream is not null
* Implement posting of agreement summary and signed time
* Mark all Api related tests as explicit and document details on running
* Clarify default token on ApiTestBase
* Adjust summary
* Update Api responses for QCC, except org products which are sold in USD
* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests
* Fix multiple markets for ZipFactorFile provider
* Use Symbol as cache key
* Api.cs review
* Dispose of factorFileStream after reading
* Use zip.EntryFileNames
* Address a few reviews
* Few more fixes
* Address Api Review
* Add Job Org id to config
* Minor tweaks
* Compare with invariant culture
* Fixes Option Universe selection
* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider
* Fix research
* Fix null reference exception
* Make duplicate log debug
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Build & Test Lean / build (push) Has been cancelled
* Attempt to remove SecurityInitializer from UniverseSelection stack entirely
* Fix broken Python Universe Selection model
* Remove mistake change (meant for another branch)
Build & Test Lean / build (push) Has been cancelled
* FOPs margin as a curve
- Adjust FOPs margin requirement so that it behaves like an 'S' curve
for the contract strike, around the underlying price and it's margin requirement.
Based on IB behavior. Adding unit tests for different future contracts
- Improve a couple of unit test cleanup
* Add ES Short Call unit test
Build & Test Lean / build (push) Has been cancelled
* Fix FutureOptionMarginModel Margin Requirements
- Fixes FuturesOptionsMarginModel margin requirements being zero due to
underlying being null on model creation. Adding unit test
* Address reviews
- SecurityService will set underlying when creating a new security when
provided
Build & Test Lean / build (push) Has been cancelled
* Updates Fundamental Data
* Updates Regression Algorithms
- `CoarseFundamentalTop3Algorithm`
- New coarse has higher `DollarVolume` for `FB`
- `CoarseNoLookAheadBiasAlgorithm`
- Change from `SPY` update not included in #5493
- `SectorExposureRiskFrameworkAlgorithm`
- `HasFundamentalData` was false for `GOOG` on 20140401 and 20140402.
* Updates Unit Tests
Minor changes in expected values
* Updates Regression Tests
Should have been included in #5493:
- `OptionsExpiredContractRegression`
- `FuturesExpiredContractRegression`
* Adjusted Quantity By Lot Size in OrderTargetsByMarginImpact
`OrderTargetsByMarginImpact` didn't calculate the order value with the quantity adjusted by order size which less to values that did not reflect the actual order value.
It has a particular affect in `SectorExposureRiskFrameworkAlgorithm` where the Python and C# versions have the same orders but placed in a different sequence because of decimal/double precision.
* Reduce dictionary access x3 on OrderTargetsByMarginImpact
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add futures regression reproducing the issue
* Cleanup futures regression
* Add Options regression
* Add IRegressionAlgorithmDefinition
* Add IRegressionAlgorithmDefinition
* Adjust regressions to compare to expiration date; delisting time is not always correct
* Refactor subscription enumerator filtering of AuxData to top of the stack
* Update broken tests to reflect the split/dividends/delisting subscriptions
* Update regression statistics because of Delisting EOD instead of at time
* Let custom data configurations bypass filter
* Adjust expected count, now that we are letting aux data through to history requests
* another small adjustment
* Use ExpectedBarCount in error message
* Add filter test for both cases
* Add some clarifying comments
* Verify we did recieve data in the regressions
* Make _shouldEmitAuxiliaryData private readonly
* Filter out aux data for history requests
* Remove option to not include aux data in subscriptions
* Refactor filtering to be more explicit for each piece of data; fixes universe selection aux data
* Cleanup comments after removed var
* Refactor order of filtering for performance reasons.
* Set _lastEmit before emitting, otherwise _workingBar is always null
* Aggregate bars if the data endTime is past lastEmit
* Add unit test
* Address Review
* Clean up unit tests
* Refactor solution to set consistent _lastEmit behaviour
* Add another unit test
* Make fixture non-parallelizable
* Undo last change, and adjust breaking test directly
* Reconcile duplicated code
* Add License header
* CS0219 Fixes: Value assigned, but never used
* CA1507: Use nameof in place of string literals
* CS0108 : Hides Inherited Member; Use new keyword to overwrite formally
* CS0114: Hides inherited member; use override keyword
* CS0168: Variable is declared but never used
* Tests CS1062; using obsolete implicit Symbol -> String; fix via .ToString()
* CS0472: Non Nullable Obj getting Null Checked
* CS0067 Member not used; ignore all cases for future use
* CS00162 : Unreachable code; either removed or ignored for debugging and test cases
* CS0169 Remove non-used fields; ignore those that may be used in future
* CS0414; Field is assigned but never used.
* CS0618; Obsolete properties and members; Only fixes simple ones, rest will have to broken up
* CS0649; Field never assigned too
* CS0659 & CS0661 ; Overwrite operators and equals but not hashcode; I don't really override it but just call base
* Small comment fix
* Cleanup pragma statement
* Reduce live coarse selection interval check to 10 min
- Reduce live coarse selection interval check to 10 min. Adding unit
tests, reducing code duplication in tests.
* Adjust unit test after self review
* Add support for live trading delisting events
- Adding `LiveDelistingEventProviderEnumerator` that will emit delisting
events. Adding unit tests
- Remove unused `LiveAuxiliaryDataEnumerator`.
- Refactor the corporate event providers being used per security type
* Self review: compare date component
* Add new LiveDataBaseDelistingEventProvider for equities
- Add support for delisting events for equities. Adding unit tests
* Fix cleanup of delisting equity subscription
* Fix IB future chain being started at reset hours
- Fix IB future chain being started at reset hours. Adding unit test
* Refactor CanAdvanceTime into CanPerformSelection
* Add main exchange to map file
minimum minimorum test
* Add MainExchange to MapFilRow IEquatable implementation
Also write CSV line correctly
* Normalize exchanges
Also add small test form MapFileRow
* Exchange name normalization
- Exchange name normalization.
- Tick will protobuf `ExchangeCode` which will set existing `Exchange`
* Include single character exchange code
* Encode exchange as byte, Include exchange name as Enum
* Update MapFileRow.cs to accept a byte instead a string for primary exchange
* Fix issues in encoding exchanges map
* Fix broken MapFile tests
* Rename Main Exchange property in MapFileRow
* Use BATS instead of BATSZ
* Add primary exchange from map file
* Move MapFilePrimaryExchangeProvider to its own file
* Implement Char for primary exchange in map files
* Fix broken tests after implementing char exchange into map file
* Add a new constructor for MapFileRow
it accepts a PrimaryExxchange object as parameter
* Fix broken test
WIP
* Address review
WIP
* Handles unexpected encoded exchanges
* Keep addressing review
* Address review
* Fix broken tests
Make PrimaryExchange as default even if the IPrimaryExchangeProvider is not instantiated
* Revert breaking change.
* Add a check that resolved map file actually have content
The issue that leads to this change was caused by a map file resolved to APC in the CoarseFundamentalTop3Algorithm.
The resolver returns a not-null MapFile, but it is empty.
To my best understand, the MapFile Resolver should return only existing map files in the map_files folder. But the LocalMapFileProvider returned a map file with a permtick that doesn't exist in the file system as CSV.
* Set PrimaryExchange.UNKNOWN as default
* Address review couple minor bugs
- Use SID market instead of USA
- Default ticke exchange is UNKNOWN
* Update map files with latest format and latest info
This include:
- VXX, it was delisted, then VXXB appears and later it chnages from VXXB -> VXX
- AAA delisting and a new listing with ticker AAA
- FOXA ticker changue to TFCFA and later it was delisted
* Update test after updating latest map files
* Fix bug with factor files after updating map files
* Fix bug with factor files after updating map files
and revert statistics in regression tests
* Update OrderHash in regression test
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Fix delisted liquidation orders being cancelled
- Place delisted liquidation orders 10 min before market closes of 10
min before the end of the delisting warning date. Adding regression
test and unit tests. Updating existing tests.
* Fix failing python option unit tests
* Fix bug where positions would be open delisting liquidation
* Fix universe selection and delisting
- Delisting will happen ASAP for all types. Giving priority to close
positions on derivates first
- Fix bug in universe selection where OptionChain would remove
underlying even if holding a position in derivate.
- Updating regression tests statistics
* Add unit test, fix unit test expected stats
* Migrate to newer CSProj format; build succesful
Rebase onto master
* Maintain binaries directory
* Reinstate FSharp default condition
* Remove reference to packages.config
* Test project fixes
* Remove nuget.config and "Solution Items"
* Remove repetitive assembly info
* Reinstate default compiles and cleanup redundant settings
* Reinstate any specific settings that defaults are unclear for
* Reinstate Framework.nuspec
* Make test internal classes public for composer use; undo assembly change
Rebase onto master part 2
* Address review; restore some settings
* Restore RootNamespace and AssemblyName attributes
* Use NugetPackageRoot instead of NugetPackageFolders
* Reinstate "RestorePackages" setting
* Transfer compilied Python files to launcher and tests bin
* Remove conflicting settings
* Fix WebSocketSharpFork references
* Fix FSharp Core reference
* Upgrade TestPlatform.ObjectModel to 16.8.3, and redirect V11 binding
* Add Mono.Cecil package
* Address review
* fill forward data points until expiry date
* wip: test non Delisting
* handle Aux not-Delisting dp; keep order while ff'ing
* simplify test checks
* wip
* fix liquidation expected date
* change statistics
* check EndTime, because it's used for time synchronization
* clean up code; remove duplicated test
* change filename template. add fillforward resolution suffix
* replicate GH issue 5116 on master
it's easy to reproduce on FXCM market by using data resolution different from fillforward resolution.
in this cases daily vs hour/minute were added
* change priorities of Time & EndTime values.
it's necessary to calculate EndTime properly, and then we can align Time to it.
potential end time should be also calculated using ptoper TZ. Because of we store open hours without TZ movement TZ in market-hours it's necessary to reapply TZ
fix tests
* Miss 2AM bar on Sunday of DST; FF 2AM bar on Sunday ST
* use UTC TimeZone as baseline during comparison
Ronit Jain