15 Commits

Author SHA1 Message Date
Jhonathan Abreu 16fdf61903 Track securities addition and removal notifications (#9097)
* Track security addition and removal sent notifications

* Minor regression algorithm fixes

* Fix regression algorithms data points count

* Add description to new regression algorithms

* Simplify tracking removal notifications

* Cleanup
2025-11-28 10:37:31 -04:00
Martin-Molinero 6916636e8e Remove Future expiry date from data & use Bank Holidays For Expirations (#8580)
Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Use Futures Bank Holidays For Expirations

* Minor improvements

- Futures will be stored by their contract month, not expiry

* Delete dairy future products

* Minor test fixes
2025-10-24 17:18:20 -03:00
Jhonathan Abreu e63217a953 New options/futures universe files format (#8848)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use correct sids when reading options/futures universe files

* Introduce new format for options and futures universe files

* Minor change

* Minor data fix

* Minor change

* Fix failing unit tests

* Add options and futures symbols cache

* [HACK] Force use test universe files

* Minor fix for universe symbols cache

* Performance improvements

* Miror change

* Cleanup

* Sort universe files

* Minor regression algorithm fix

AddAndRemoveOptionContractRegressionAlgorithm to not depend on universe file entries ordering
2025-07-11 16:38:14 -04:00
Jhonathan Abreu 69d2f5ae82 Futures and Future Options file-based universes (#8480)
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Make FOPs selection universe file-based for backtesting

* Make FOPs option chains universe file based

* Make Future universe selection file-based like option universe

* Make Future universe selection file-based like option universe

* Abstraction cleanup

* Add FuturesChains API to QC algorithm

Also refactor future chain provider to use the new FutureUniverse instead of zip file names

* Update regression algorithms stats

* Refactor QuantBook option and future history to use new universes

* Fix failing tests

* Fix failing tests

* Fix failing tests

* Minor future chains unit test improvement

* Add futures chains DataFrame property

Also, remove IDerivativeSecurity interface from Future

* Add DataFrame property to FuturesChains class

* Add regression algorithms

* Add regression algorithms

* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api

* Minor fixes

* Reduce number of universe files in repo

* Minor data fixes

* Regression algorithms updates

* Add implicit conversion from FuturesContract to Symbol

Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.

* Improve resolution handling for history requests

* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property

* Refactor data handling in BaseChain and TimeSliceFactory

- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method

* Remove specific constructors and indexers from Chain classes

Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.

* Remove chain cache logic from FuturesChainUniverse

* Refactor class and interface names for clarity

Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.

* Add base class for options and futures contracts

- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface

* Add minor fix for future options tickers parsing

Added tests

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Clean chain provider classes up

* Remove ZipEntryName other classes and unused code

Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.

* Minor changes and cleanup

* Trigger Build

* Trigger Build

* Refactor FuturesContract data handling

Forward price data from bars and ticks stored in private fields for improved memory usage

* Fix: use universe data for market data in FuturesContract

* Update regression algorithms stats after rebase

Added HSI futures universe files

* Sort configs by internal flag

Internals go first

* Throw from option universe data filters for future options

Future options IV, Open interest and greeks are not supported for future options

* Minor changes

* Improve some regression algorithms

* Minor fix for failing unit tests

* Update FOPs universe file header

Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.

* Minor unit test fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Add history provider as constructor argument for chain providers

* Update new regression algorithms data points count

* Minor fix for FakeDataQueue

* Add initialize method to chain providers classes

* Minor changes

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Minor fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Add logs to ProcessedDataProvider

* Removed test logs

* Minor fix

* Support downloading options and futures universe files from api data provider
2025-03-25 16:22:38 -04:00
Jhonathan Abreu f2100adb46 Fix price shifts around futures rollover date (#8371)
* Test

* Fix price shift on continuous future rollover

* Keep track of both data and exchange time in subscription data reader

* Fix for daily and hourly resolutions.

Add daily and hour algorithms

* Fixes for emitting new exchange dates at the beginning and end when data is not available

* Cleanup

* Minor fix

* Minor fix: detect mapping in data reader after advancing time keeper

* Minor fix

* More fixes

* More fixes

* Fixes for universe files

* More fixes for daily resolution data

* Separate DateChangeTimeKeeper to its own file

* Add DateChangeTimeKeeper unit tests

* Minor fix

* Add more DateChangeTimeKeeper unit tests

* Address peer review

* Address peer review

* Address peer review

* Detect mappings using the config NewSymbol event

* DateChangeTimeKeeper performance improvement for time zone conversions

* Clean up date change time keeper

* Cleanup

* Minor regression algo update

* Cleanup

* Cleanup

* Fix perform universe selection on sundays for continuous futures with extended market hours

* Cleanup

* Address peer review

* More cleanup

* More cleanup
2024-10-21 09:38:00 -03:00
Martin-Molinero 73c71c6913 Add missing GC future auxiliary data (#7312)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add missing GC future auxiliary data

- For improved testing add missing GC comex future auxiliary data.
  Updating regression tests

* Update regression algorithm expected data points
2023-06-09 19:43:34 -03:00
Martin-Molinero e316f12394 Refactor MarginRequirementEntry. Update margins (#6210)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix master

* Refactor MarginRequirementEntry to it's own file

* Update margin files

* Remove duplicate dates cases

* Update margins after fix
2022-02-15 20:24:28 -03:00
Colton Sellers 1358bd8115 Future and FutureOption low res support (#6069)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fixes for FutureOptions support in LeanData

* Add CreateCanonicalOption() utility function for Symbol.cs

* Add aggregated Futures/FuturesOptions data to Lean

* Add FutureOptions regressions for daily/hourly data

* Allow Futures to be added with low resolution

* Add Future regressions using hour/daily data

* Nit - Python Class names

* Add reviews

* Add alias into CreateCanonicalOption
2021-11-24 09:53:21 -03:00
Gerardo Salazar d5aa9e65f3 Adds FuturesOptionsUnderlyingMapper and FuturesListing (#5142)
* Adds FuturesOptionsUnderlyingMapper

  This adds a mapper for Futures options that have a different
  contract month between the FOP and the underlying future, as is
  observed in the contracts ZB, ZC, ZS, ZT, ZW, HG, SI, GC.
  An example of this is GC (e.g. OGH21 -> GCJ21). Other contracts follow
  different rules, which have been included as part of this commit.

  * Adds unit tests for FuturesOptionsUnderlyingMapper

  * Adds unit and regresssion tests for underlying Futures mapping for FOPs

* Addresses review and adds FuturesListings

  * Removes dependency on IFuturesChainProvider in
    FuturesOptionsUnderlyingMapper

  * Fixes bugs related to FuturesOptionsUnderlyingMapper and incorrect
    results

  * Modifies misc. code to handle new results correctly from FOPs
    underlying mapper

  * Makes FOPs underlying mapper static, and makes other methods private

  * Adds new tests for FuturesListings

* Addresses review: code cleanup

* Address review: Makes arrays to List in FuturesListing private methods
2021-01-12 06:12:00 -08:00
Gerardo Salazar 3baf8c3959 Address review: Removes changing percentage margins
* Only the first margin of a percentage margin will be
    used from now onwards
2020-06-15 16:00:38 -07:00
Gerardo Salazar 81f2c43d0a Updates margins using new contract selection strategy
* Fixes previously incorrect margins
2020-06-08 17:55:12 -07:00
Gerardo Salazar b0f787abd8 Address review - removes duplicates in margins and skip non-USD margins 2020-06-03 15:20:43 -07:00
JJD a3356904b0 Update futures margins 2020-06-02 17:39:12 -03:00
JJD 2ef5129e9d Move margins to its respective exchanges folders 2020-04-28 16:29:16 -03:00
JJD 4ab5d8fc34 Update future data sample 2020-04-28 16:29:15 -03:00