17841
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Roman Yavnikov
|
d1337eae2c |
Feat: CanonicalDataDownloaderDecorator for chain support (#9299)
* feat: add CanonicalDataDownloaderDecorator for chain support Introduced CanonicalDataDownloaderDecorator to enable automatic resolution and parallel downloading of all contracts for canonical symbols (options and futures chains). Updated Program.cs and DownloaderDataProvider to use this decorator, ensuring seamless data retrieval for both canonical and non-canonical symbols. Refactored initialization logic to handle chain providers within the decorator and removed redundant code from Program.cs. * feat: improve canonical symbol error handling in data downloader * refactor: data downloader selection in DownloaderDataProvider * refactor: CanonicalDataDownloaderDecorator construction * feat: limit parallelism and improve contract data error handling * refactor: canonical data downloader and provider usage * remove: CanonicalNotSupportedException and refactor handling * refactor: data downloader selection with selector class Introduce DataDownloaderSelector to choose the correct IDataDownloader implementation based on data type, using CanonicalDataDownloaderDecorator only when needed. Update Program.cs and DownloaderDataProvider to use the selector, remove redundant initialization logic, and ensure proper resource disposal. This improves flexibility, correctness, and resource management for data downloads, especially for custom and canonical data types. * fix: wrong selector condition * refactor: change default log handler; add parallelism config to downloader - Set ConsoleLogHandler as the default log handler in code and config - Make contract download parallelism configurable via downloader-thread-count (default 4) - Track and log number of processed contracts in CanonicalDataDownloaderDecorator - Add error logging for missing universe data - Improve logging clarity and code readability * feat: prevent duplicate contract downloads in canonical chains Introduce ContractDownloadParameters to uniquely identify contract/tick type/resolution combinations and cache them in CanonicalDataDownloaderDecorator, avoiding redundant downloads when contracts are shared across canonical symbol chains. Add date range optimization for contract downloads and unit tests for the new class. * Revert "feat: prevent duplicate contract downloads in canonical chains" This reverts commit 44386c4c788904531c473c60c2a173e2354561ec. * feat: clamp contract date ranges, deduplicate downloads Added AdjustDateRangeForContract to CanonicalDataDownloaderDecorator to clamp start/end dates based on contract expiry and security type. Introduced _contractsCache to prevent duplicate downloads across canonical chains and _processedContracts for logging. Updated contract retrieval logic for deduplication. Added NUnit tests to verify date range adjustments for futures and options. * refactor: canonical contract date range adjustment logic * test:feat: add test for non-option/future contract date adjustment * feat: make look-back periods for canonical symbols configurable * refactor: remove Lazy from CanonicalDataDownloaderDecorator Revert "refactor: remove Lazy from CanonicalDataDownloaderDecorator" This reverts commit 909c1299406688a19352273f8d0b5b3ea99c0fb9. refactor: Inject IMapFileProvider into downloader selectors Revert "refactor: Inject IMapFileProvider into downloader selectors" This reverts commit 0add95269809680e425bfa6080e465bf16377e56. Reapply "refactor: remove Lazy from CanonicalDataDownloaderDecorator" This reverts commit 9e3c4bed078868a3c8dfedcab0e52d6dced1daad. * refactor: data downloader dependency injection * refactor: provider initialization in data downloader classes * refactor: DataDownloaderSelector dependencies, add tests - Require explicit IDataProvider in DataDownloaderSelector constructor, removing default and fallback logic - Change default log handler to ConsoleLogHandler in Program.cs - Pass DownloaderDataProvider as IDataProvider to DataDownloaderSelector - Add DataDownloaderSelector unit tests to verify correct downloader selection and decorator usage * feat: ensure factorFileProvider is initialized when defaulted * fix: missed dataProvider in DataDownloaderSelector ctor * refactor: improve IFactorFileProvider resolution in DataDownloaderSelector |
||
|
Roman Yavnikov
|
86fcc40c3e |
Feature: Generic brokerage downloader wrapper (#8235)
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* feat: new constructor of AlgorithmNodePacket * refactor: extract JobQueue configuration * remove: not used `using` in IDataDownloader * feat: create BrokerageDataDownloader * Revert "refactor: extract JobQueue configuration" This reverts commit 5778936b712befd6e367bee3decc4a3565ca69af. * Revert "feat: new constructor of AlgorithmNodePacket" This reverts commit d7a565ff761243ae1391bd45fd10c93221f0329e. * feat: new config `data-download-brokerage` in DataDownloadProvider * refactor: initialize in BrokerageDataDownloader * remove: not used `using` in Program's DataDownloadProvider * remove: not used ref on QuantConnect.Queue proj * refactor: use default market based on SecurityType * refactor: MarketName in DataDownloadConfig struct test:feat: validate MarketName * feat: support Canonical Symbols in BrokerageDataDownloader * remove: not used command arguments * feat: init CacheProvider of IOptionChainProvider in Downloader * feat: add brokerage message event in BrokerageDataDownloader |
||
|
Martin-Molinero
|
b527a3dd86 |
Minor fixes (#7980)
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Minor test improvement - Adjust DownloaderDataProvider to follow the pattern |