Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Append out of memory diagnostics to the algorithm runtime error message
* Merge out of memory detection and diagnostics into a single method and shorten the message
* Only check the top level exception for out of memory diagnostics
Build & Test Lean / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Implement indicator-based option price model
This model uses IV and Greeks indicators to implement Lean's own option pricing model
* Minor fixes
* Address peer review
* Minor tests fixes
* Make the indicator based price model the default for options
* Address peer review
* Cleanup and minor changes
* Support indicators configuration for new pricing model
* Some cleanup
* Add QL option price model example algorithm
* Return lean models from static helpers
* Minor tests fixes
* Minor test fixes
* Address peer review
* Cleanup
* Fix unit tests
* Move QL models to OptionPriceModels.QuantLib.*
* Add forward tree helper method
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Make FOPs selection universe file-based for backtesting
* Make FOPs option chains universe file based
* Make Future universe selection file-based like option universe
* Make Future universe selection file-based like option universe
* Abstraction cleanup
* Add FuturesChains API to QC algorithm
Also refactor future chain provider to use the new FutureUniverse instead of zip file names
* Update regression algorithms stats
* Refactor QuantBook option and future history to use new universes
* Fix failing tests
* Fix failing tests
* Fix failing tests
* Minor future chains unit test improvement
* Add futures chains DataFrame property
Also, remove IDerivativeSecurity interface from Future
* Add DataFrame property to FuturesChains class
* Add regression algorithms
* Add regression algorithms
* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api
* Minor fixes
* Reduce number of universe files in repo
* Minor data fixes
* Regression algorithms updates
* Add implicit conversion from FuturesContract to Symbol
Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.
* Improve resolution handling for history requests
* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property
* Refactor data handling in BaseChain and TimeSliceFactory
- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method
* Remove specific constructors and indexers from Chain classes
Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.
* Remove chain cache logic from FuturesChainUniverse
* Refactor class and interface names for clarity
Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.
* Add base class for options and futures contracts
- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface
* Add minor fix for future options tickers parsing
Added tests
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Clean chain provider classes up
* Remove ZipEntryName other classes and unused code
Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.
* Minor changes and cleanup
* Trigger Build
* Trigger Build
* Refactor FuturesContract data handling
Forward price data from bars and ticks stored in private fields for improved memory usage
* Fix: use universe data for market data in FuturesContract
* Update regression algorithms stats after rebase
Added HSI futures universe files
* Sort configs by internal flag
Internals go first
* Throw from option universe data filters for future options
Future options IV, Open interest and greeks are not supported for future options
* Minor changes
* Improve some regression algorithms
* Minor fix for failing unit tests
* Update FOPs universe file header
Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.
* Minor unit test fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Add history provider as constructor argument for chain providers
* Update new regression algorithms data points count
* Minor fix for FakeDataQueue
* Add initialize method to chain providers classes
* Minor changes
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Minor fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Add logs to ProcessedDataProvider
* Removed test logs
* Minor fix
* Support downloading options and futures universe files from api data provider
* Daily data Time & EndTime Improvement
- Adjust daily data Time & EndTime to actually reflect the time of the
data used, for example US Equity from 9.30 to 4PM. Adding new unit and
regression tests
* Refactor solution to use enumerator
- Refactor daily strict end times solution to be through enumerator
usage, so it applies for history providers too
* Minor fixes
* Revert fill forward enumerator change
- Revert FillForward enumerator causing stats changing, enhancing unit
tests
* Some cleanup
* Improve handling of live trading FF enumerator
- Improve handling of live trading FF enumerator, by adding support for
bars to arrive with a delay so we can handle auction close/option
prices or data providers which might have some delay making the data
available. Adding new unit tests asserting the behavior
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Portfolio state chart
- Cleanup and renames
- Add series.IndexName. Uodating unit tests
- Add Chart.LegendDisabled. Adding new unit tests
- Add ChartPoint.Tooltip. Updating unit tests
- Minor compression tweak. Adding unit test
- Add ChartJsonConverter. Adding unit tests
* Minor chart serialization order tweak
* Refactor portfolio state sampling and storing
* Move PortfolioMargin into a lean side chart
* Handle/filter brokerage side orders
* Add unit tests
* Minor fix
* Add unrequested security for new brokerage-side orders
* Add unit test for algorithms overriding the brokerage message handler
* Add unit test for python algorithm overriding the brokerage message handler
* Minor change
* Address peer review
* Store traded securities subscription data configs to disk
* Delegate traded subscriptions storage to the data monitor
* Add unit tests
* Address peer review
* Minor changes
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Support sourcing custom data from ObjectStore
* Add Python regression algorithm
* Wire engine to pass ObjectStore down to the stream readers
* Minor changes
* Minor unit tests fixes
* Remove unused SetupHandlerParameters.ObjectStore
* Minor changes
* Minor changes
* Support single-file zipped data to be sourced from object store
* Assert object store custom data history requests in regression algorithms
* Add custom object store data live data feed unit test
* Add multi-file object store custom data regression algorithms
* Minor fix
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics
- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
algorithms
* Optimization backtest result json converter update
* Address reviews
- Remove IAlphaHandler, move insight storage responsability to IResultHandler
and centralizing insight collection on the QCAlgorithm.Insights to be
reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
sampling handling. Updating regression algorithms
* Add InsightCollection tests and minor fixes
* Adding more & improved tests
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* add state to backtesting result packets
* self review
* add state in live results
* set runtime error in algorithm
* set hostname in job packets
* Set state when algorithm is null
* address review
* address reviews
* set runtime error when algorithm is not available during runtime
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Data monitor (FIRST TEST)
* Calculate data request frequency in data monitor
* Detecting universe data usage in data monitor
* Json data monitor report generation
* Data monitor refactor
* Log and store data monitor report
* Data monitor report tests
* Clean up
* Clean up
* Address changes request
* Address changes request
* Wire data monitor only for backtest, not for live or research
* Minor tweaks
* Some minor tweaks
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Refactor chain providers
- ChainProviders will use quote, open interest and trade information,
which ever is first. Updating regression algorithm changing contract
being added.
- Add base BacktestingChainProvider for shared logic, reducing
duplication.
- ChainProviders will now use the given IDataCacheProvider and
IMapFileProvider
- If Providers are called for a date the exchange is closed they will
search for the previous available date where the exchange is open.
Adding unit tests.
* Address reviews. Logging improvements
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Warmup minor fixes
- Result handler fix
- Centralize and normalize status update during warmup
* Address reviews
* Minor adjustments
* Add console message during warmup period
* Fix warmup percentage update
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* initial commit
* get history from data providers
* merge history from history providers
* merge slice
* Add merge function in slice
* Update test
* merge aux data
* add data points in _data
* append new data points in original list
* Add test suite for HistoryProviderManager
* reduce complexity
* setup once
* add fake history provider
* doesn't count aux data
* add tests for options
* style changes
* add custom data
* merge rawDataList
* use array of history providers
* optimize
* fix formatting error
* add comment
* add tests
* simplify
* use list
* use abstraction
* split tests
* add test
* use abstraction to create generic enumertor class
* refactor
* accept list of type T
* sync history slices
* use SubscriptionDataReaderHistoryProvider for live
* rename test file
* return empty
* add tests
* cleanup
* address reviews
* optimize
* Fix method definition
* use initial time for basedata
* refactor
* re-use collection
* inherit HistoryProviderBase
* always return HistoryProviderManager
* add tests
* update rawDataList
* reset composer
* consider null elements
* add tests for binary search method
* Follow lean coding style
* revert
* remove binary search method
* convert to field
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing
- Improve python exception parsing adding support for line shift. Adding
unit tests
* PythonException revert change
* Centralized and normalize algorithm runtime handling
* Adding support for C# line and file exception report
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add IResultHandler.BrokerageMessage
- Result handlers can receive notifications for brokerage message events
* Make LiveTradingResultHandler.RuntimeError virtual
Build & Test Lean / build (push) Has been cancelled
- Ensure data is downloaded only once with the ApiDataProvider. Adding
unit test
- Log invalid securityType ApiDataProvider requests once
- Assert Factor file and map file providers are compatible with the
ApiDataProvider
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request
* key -> filePath rename and some cleanup
* Refactor
* Add Organizations Endpoints
* Add some organization api wrapper objects
* Address namespace issue
* Reorganize Api Test into seperate files using one ApiTestBase
* Add Organization tests
* Use capitalized "API" test namespace to reduce amount of file changes
* Add License to test base
* Update /data endpoint functions and response objects
* Update ApiDataProvider Logic
* Handle deserialization of organization products
* Simplify converter
* Only throw for equity requests when not subscribed to map/factor files
* Add missing header
* Make arguement exception
* Api adjustments
* Add Zip factor and map file providers
- Common project will now reference Compression project and not the other way
round.
- Adding Zip FactorFile and MapFile providers
* Refactor FactorFileProvider to use DataProvider to fetch files
* Use resulting MinimumDate in construction of FactorFile
* Nit FactorFile comments and arrangement
* Refactor MapFileProviders to use DataProvider for fetching files
* Refactor ZipFileProvider
* Clean up
* Refactor Backtesting Future/Option chain providers to use dataprovider
* Fixes for data/ endpoints and test adjustments
* Response objects adjustments/cleanups
* ApiDateProvider fixes and testing
* Add LocalZipFactorFileTests
* Update ApiDataProvider download test to verify stream is not null
* Implement posting of agreement summary and signed time
* Mark all Api related tests as explicit and document details on running
* Clarify default token on ApiTestBase
* Adjust summary
* Update Api responses for QCC, except org products which are sold in USD
* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests
* Fix multiple markets for ZipFactorFile provider
* Use Symbol as cache key
* Api.cs review
* Dispose of factorFileStream after reading
* Use zip.EntryFileNames
* Address a few reviews
* Few more fixes
* Address Api Review
* Add Job Org id to config
* Minor tweaks
* Compare with invariant culture
* Fixes Option Universe selection
* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider
* Fix research
* Fix null reference exception
* Make duplicate log debug
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Add main exchange to map file
minimum minimorum test
* Add MainExchange to MapFilRow IEquatable implementation
Also write CSV line correctly
* Normalize exchanges
Also add small test form MapFileRow
* Exchange name normalization
- Exchange name normalization.
- Tick will protobuf `ExchangeCode` which will set existing `Exchange`
* Include single character exchange code
* Encode exchange as byte, Include exchange name as Enum
* Update MapFileRow.cs to accept a byte instead a string for primary exchange
* Fix issues in encoding exchanges map
* Fix broken MapFile tests
* Rename Main Exchange property in MapFileRow
* Use BATS instead of BATSZ
* Add primary exchange from map file
* Move MapFilePrimaryExchangeProvider to its own file
* Implement Char for primary exchange in map files
* Fix broken tests after implementing char exchange into map file
* Add a new constructor for MapFileRow
it accepts a PrimaryExxchange object as parameter
* Fix broken test
WIP
* Address review
WIP
* Handles unexpected encoded exchanges
* Keep addressing review
* Address review
* Fix broken tests
Make PrimaryExchange as default even if the IPrimaryExchangeProvider is not instantiated
* Revert breaking change.
* Add a check that resolved map file actually have content
The issue that leads to this change was caused by a map file resolved to APC in the CoarseFundamentalTop3Algorithm.
The resolver returns a not-null MapFile, but it is empty.
To my best understand, the MapFile Resolver should return only existing map files in the map_files folder. But the LocalMapFileProvider returned a map file with a permtick that doesn't exist in the file system as CSV.
* Set PrimaryExchange.UNKNOWN as default
* Address review couple minor bugs
- Use SID market instead of USA
- Default ticke exchange is UNKNOWN
* Update map files with latest format and latest info
This include:
- VXX, it was delisted, then VXXB appears and later it chnages from VXXB -> VXX
- AAA delisting and a new listing with ticker AAA
- FOXA ticker changue to TFCFA and later it was delisted
* Update test after updating latest map files
* Fix bug with factor files after updating map files
* Fix bug with factor files after updating map files
and revert statistics in regression tests
* Update OrderHash in regression test
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Add internal subscription manager
- Add InternalSubscriptionManager that will handle internal
Subscription. Replaces the realtime updates
- Fix thread race condition in the TimeTriggeredUniverseSubscription, we
have one thread injecting data points, the main algorithm thread, and
the base exchange is pulling from it
- Fixes for FakeDataQueue
- Adding unit tests
* Address reviews and fixes
- Internal subscription will use extended market hours
- Only sample charts accordingly
- Get api-url once
- `UniverseSelection` and `Research` will always use
`AlgorithmHandler.DataProvider` instance instead of
`DefaultDataProvider`
- Remove Compression at test/app.config since the dll isn't required
- Add missing license header
* Live Coarse universe refactor
- Live trading will source Coarse and Fine fundamental data directly
from disk. Updating unit tests.
* Adds ILiveDataProvider interface
* Adds wrapper for IDataQueueHandler implementations
* Replaces IDataQueueHandler with ILiveDataProvider in
LiveTradingDataFeed
* Edits IDataQueueHandler documentation
* Maintains aggregation for current IDQH impls and skips for ILDF impls
* Note: No unit test was created for this method, go back and TODO
* Protobuf Market data
- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
unit tests.
* Adds unit tests for LiveDataAggregator changes
* Fixes bug where custom data was not handled as it was before
* Fixes race condition bug because of variable reuse in class
* Add protobuf extension serialization
* Fixes for protobuf serialization
* Refactor
* Fix OptionChainUniverse
* replace BaseDataExchange pumping ticks with consolidators
* AlpacaBrokerage
* BitfinexBrokerage
* GDAXBrokerage
* OandaBrokerage
* InteractiveBrokers
* TradierBrokerage
* FxcmBrokerage
* PaperBrokerage
* etc
* WIP fixes for existing LTDF unit tests
* Fixes more LTDF unit tests
* make IDataAggregator.Update recieving Generic BaseData rather than Tick
* Change IDataQueueHandler.Subscribe method
* Some fixes after adding new commits
* Adds protobuf (de)serialization support for Dividend and Split
* Serialize protobuf with length prefix
* Fix missing LTDF unit tests
* Adds TiingoNews protobuf definitions
* fix comments
* more fixes on IQFeedDataQueueHandler
* disallow putting ticks into enumerator directly
* ScannableEnumerator tests
* fix OandaBrokerage
* AggregationManager unit tests
* fix AlpacaBrokerage tests
* fix InteractiveBrokers
* fix FxcmBrokerage tests
* call AggregationManager.Remove method on unsubscribe
* fix GDAX existing tests
* Fixes, refactor adding more tests for AggregatorManager
* Adds BenzingaNews protobuf definitions and round trip unit test
* Adds missing TiingoNews unit test to Protobuf round trip tests
* Improve sleep sequence of LiveSynchronizer
* need start aggregating first, and then can subscribe
* More test fixes and refactor
- Refactoring AggregationManager and ScannableEnumerator so the last is
the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer
* Improve performance of LEquityDataSynchronizingEnu
* Add missing Set job packet method
* Minor performance improvements
* Improvements add test timeout
- Improvements adding test timeout to find blocking test in travis
* Improve aggregationManager performance
* Testing improvements for travis
* Remove test timeouts
* More test fixes
- Adding more missing dispose calls and improving determinism
* fix IEXDataQueueHandler and tests
* Final tweaks to LTDF tests
* more AggregationManager tests
* consume and log ticks
* fix test: couldn't subscribe to Forex tickers
* change Resolution for all bar configs
* Improve RealTimeScheduleEventServiceAccuracy
* refactoring: move common code to base class
* fixed bug; unsubscribe SubscriptionDataConfig
* Small performance improvement
* Minor fixes
* Avoid Symbol serialization
* Fixes coarse selection in live mode
* Fix for live coarse
* Adds protobuf (de)serialization support for Robintrack
* Adds round-trip unit test
* Minor performance improvements
* More minor performance improvements
* pass LiveNodePacket through to OandaBrokerage
* Fixes empty list becoming null value when deserializing with protobuf
* Reverts BZ live trading exception removal and fixes tests
* Refactor WorkQueue making it abstract
* Add try catch for composer
* Adds optional data batching period to LiveFillForwardEnumerator
* Override data-queue-handler with config
* Improve PeriodCountConsolidator.Scan performance
* Move batching delay to main Synchornizer thread
* Reverts addition of Robintrack protobuf definitions
* Give priority to config history provider if set
* Add Estimize protobuffing
- Add Estimize protobuffing support. Adding unit tests
* Always dispose of data queue handler
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
- Adding new DataPermissionManager that will own the existing
datachannelProvider. Will assert configurations before added to the
data feed. Adding unit tests
- Refactor ZipDataCacheProvider to avoid value types thread issues, it
will now use a timer. Add missing dispose calls.
- Synchronizer won't share the `SubscriptionFrontierTimeProvider`
instance since it's not thread safe and shouldn't be called
- Fix BacktestingResultHandler `_daysProcessed` thread safety
- Fixing `RealTimeScheduleEventService` thread safety
Jhonathan Abreu