63 Commits

Author SHA1 Message Date
Martin-Molinero e568162bff Minor slice.AllData api change (#9130)
API Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2025-12-15 10:25:00 -03:00
Jhonathan Abreu 053c9a20c4 ExtendedDictionary improvements (#8723)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Make ExtendedDictionary extend IDictionary

* Revert "Make ExtendedDictionary extend IDictionary"

This reverts commit bbbe9c769be682767cb9e487b904480376b53596.

* Make ExtendedDictionary key a type parameter

* Make SecurityPositionGroupModel an ExtendedDictionary

* Add unit tests

* Minor change

* Add more unit tests and other minor changes

* Add Count and ContainsKey to BaseChain

* Update pythonnet version to 2.0.43

* Update some python regression algorithms

* Minor regression algorithms fix

* Make CashBook and ExtendedDictionary

* Minor changes

* Minor changes

* Update pythonnet version to 2.0.44
2025-05-02 11:02:06 -04:00
Martin-Molinero 4584feb7da Merge PythonSlice into Slice (#8707)
- To simplify stub generation, merge PythonSlice into Slice, following
  existing pattern too, see QCAlgorithm.Python.cs
- Minor sintax fixes in example python algorithms
2025-04-21 13:37:33 -03:00
Jhonathan Abreu e29bb2c5e0 File-based options universe (#8212)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Initial options universe with greeks implementation

* Options universe improvements

* Address peer review

* File based options universe fixes and improvements.

- Adjust OptionUniverse start-end times and period.
- Adapt unit tests and some algorithms to pass with new options universe selection.

* Updated options regression algorithms stats for new universe data

* Updated options regression algorithms stats for new universe data

* Updated options regression algorithms stats for new universe data

* Updated options regression algorithms stats for new universe data

* Updated options regression algorithms stats for new universe data

* Option chain provider with new options universe

* Allow canonical option history requests

* Address peer review

* Address peer review

* Fix symbols parsing in OptionUniverse

* Fix universe selection subscriptions start time to not include extended market hours

* Minor changes

* Minor changes

* Peer recommended changes and fixes

* Update regression algorithm stats

* Update regression algorithms stats and minor fixes

* Fix option chain provider history request

* Round option indicators values

* Added option universe csv header property

* Update regression algorithms stats

* Update regression algorithms stats

* Data fixes and regression algos stats update

* Unit test fixes

* Minor changes

* Option chain handling in live trading data feed

* Minor changes

* Added processed data provider

* Fix thread-safety violation in Slice class

* Minor change

* Update options filter universe API to use OptionUniverse data

Add new filter methods for greeks, IV and open interest

* Option filter universe api updates

* Add OptionUniverse history regression algorithms

* Add regression algorithms for new options filter universe api methods

* Added options greeks data and updated regression algorithms

* Address peer review

* Address peer review

* Add more assertions to new options filter api regression algorithms

* Minor performance improvement.

Reduce greeks binomial model steps to 140

* Minor tests updates

* Greeks numerical models performance improvements

* Greeks numerical models performance improvements

* Revert array pool change for option pricing numerical models

* Update default dividend yield provider depending on option type

* [TEST]

* Add helper method con calculate time till expiration

* Use double in price option numerical models

* Implied volatility calculation improvements

- Adjust root finding method accuracy as a factor of the option price
- Use BSM to get a first guess

* Cleanup

* Some regression algorithms and unit tests cleanup

* Regression tests updates after rebasing from master

* Add universe files

* Self review and cleanup

* Minor regression tests updates after rebase

* Fix: set data time zone to same as exchange tz for options universes

* Minor change

* Minor change

* Fix for live trading options universe selection

* Keep underlying when aggregating collections in BaseDataCollectionAggregatorEnumerator

* Update index options regression algorithms stats

* Minor change

* Address peer review

* Memory usage improvements

* Minor build fix

* Minor changes and test fixes

* Cache symbols in OptionUniverse

* Cleanup

* Fix index option creation in OptionUniverse

* Use cached underlying SID when parsing from string

* Abstract symbols cache to BaseDataCollection

* Return actual underlying symbol when mapping decomposing ICO ticker

* Address peer review

* Minor performance improvements reduce garbage

* Limit Symbols and SIDs cache size to help with memory usage

* Minor fix in symbols and sid cache cleanup

* Build fix

* Lazily parse greeks on individual access

* Cleanup and tests

* Address peer review

* Minor greeks fix

---------

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2024-09-09 12:39:31 -03:00
Martin-Molinero 3eedde9b6e Normalize ETF constituents universe (#7858)
- Normalize ETF constituents universe data. Adding new tests and
  assertions
2024-03-19 11:28:17 -03:00
Martin-Molinero 516befe0b5 Backwards compatibility research universe history (#7849)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Add backwards compatibility for research universe history api, adding
  unit tests
2024-03-13 11:20:02 -03:00
Martin-Molinero 7498d2e6e2 QuantBook Universe Selection (#7587)
* QuantBook Universe Selection

- QuantBook universe selection helper method. Adding new unit tests.
- Universe selection data sets improvements

* QuantBook API renames
2023-11-21 17:05:02 -03:00
Martin-Molinero ef01980b07 Python history improvements (#7283)
Benchmarks / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Pandas frame include all ticks

- Pandas data framde history response will include all ticks. Updating
  existing and adding new tests

* Python pandas converter performance improvement
2023-05-31 10:22:21 -03:00
Jhonathan Abreu fbf8ffd924 Handle price discontinuity on volatility models (#7058)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Apply splits and dividends to volatility models

* Apply splits and dividends to volatility models using history requests

* Add new ScaleRaw data normalization mode

Handling the new mode in the price scale enumerator.

* DataNormalizationMode.ScaledRaw history requests

* Minor changes

* Minor changes

* Disable new normalization mode in AddSecurity methods and other minor changes

* Peer review

* Minor changes

* Peer review

* Minor changes

* Peer review

* Peer review

* Peer review

* Add scaled raw history regression algorithm

* Add more regression algorithms

* Add more regression algorithms

* Add Slice.TryGet unit tests

* Peer review

* Peer review

* Peer review

* Peer review

* Peer review

* Update algorithms stats

* Peer review

* Peer review
2023-03-13 13:11:30 -03:00
Martin-Molinero f12e8fc08e CryptoFutures adjustments (#6836)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* CryptoFutures adjustments

- Address reviews of CryptoFutures:
- Add new Slice MarginInterestRates collection
- Add new regression tests asserting funding rate application behaves
  the same no matter the resolution
- Add auxiliary data by type into the security cache
- Revert BuyingPowerModel changes

* Make interest rate application time deterministic
2023-01-05 18:07:01 -03:00
Martin-Molinero ee683933a3 Feature Perpetual Crypto Futures (#6807)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* WIP

* Add base currency cash

* Symbol properties and data processing

* Add basic template algorithm

* Add hourly crypto future algorithm

* Minor fixes after live trading testing

* CoinApiDataQueueHandler CryptoFuture support

* Address reviews

* Fix regression algorithms after update
2022-12-30 13:58:11 -03:00
Ronit Jain f86926bf7a Fixes HistoryProviderManager slice merging by using algortime for all slices (#6226)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* remove un-used

* use utc time for slice sync

* use utc time

* refactor

* add regression test

* use utc time

* use utc time

* make utctime required parameter

* add utcTime in slice creation

* assert warm up complete

* check if algorithm is still warmingup

* use exchange tz
2022-02-18 20:43:33 -03:00
Ronit Jain 2d644d7879 Feature handle multiple history providers (#6187)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* initial commit

* get history from data providers

* merge history from history providers

* merge slice

* Add merge function in slice

* Update test

* merge aux data

* add data points in _data

* append new data points in original list

* Add test suite for HistoryProviderManager

* reduce complexity

* setup once

* add fake history provider

* doesn't count aux data

* add tests for options

* style changes

* add custom data

* merge rawDataList

* use array of history providers

* optimize

* fix formatting error

* add comment

* add tests

* simplify

* use list

* use abstraction

* split tests

* add test

* use abstraction to create generic enumertor class

* refactor

* accept list of type T

* sync history slices

* use SubscriptionDataReaderHistoryProvider for live

* rename test file

* return empty

* add tests

* cleanup

* address reviews

* optimize

* Fix method definition

* use initial time for basedata

* refactor

* re-use collection

* inherit HistoryProviderBase

* always return HistoryProviderManager

* add tests

* update rawDataList

* reset composer

* consider null elements

* add tests for binary search method

* Follow lean coding style

* revert

* remove binary search method

* convert to field
2022-02-14 11:58:00 -03:00
Martin-Molinero c8dc343c13 GetLastKnownPrices python data (#6191)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adding unit tests reproducing issue.

* Fix a couple of minor bugs

- IsMarketOpen will work correctly when used with daily and hourly
  resolution.
- slice.Get will work correctly with python custom data
- ExtendedDictionary will be able to dinamically access methods,
  required for python and private C# data types

* Refactor solution. Add more tests

* Remove unrequired import statement
2022-02-08 15:37:05 -03:00
Ricardo Andrés Marino Rojas 472f78cc53 Remove Quandl from LEAN (#6110)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Remove Quandl from LEAN

* Nit changes and CustomLiveDataFeedTests.cs

* Resolve conflicts

* Remove files related with Quandl

* Fix bug

* Fix QuantBookHistoryTests.cs

* Fix bug

* Fix bug

* Fix unit tests

* Try fix regression tests

* Nit changes

* Fix bug

* Some of the requested changes

* The missing changes

* Requested changes

* Nit changes

* Revert "Nit changes"

This reverts commit 9800bc5c34f3ac20e30bea7a92dd4a9867213bb5.

* Nit changes

* Fix bug

* Requested changes

* Missing file using Quandl to be removed

* Nit changes

* Not applied nit change

* Nit change

* Nit change

* Add nasdaq-auth-code parameter in config.json

* Remove 'quandl-auth-token' from config.json

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-01-12 12:10:20 -03:00
Martin-Molinero 4a0fb30df5 Fixes for OpenInterest storing (#4712)
- Fix Slice.Get OpenInterest type. Adding unit test
- Fix for SecurityCache that wasn't storing OpenInterest types
- Updateing regression tests to covere these usages
2020-09-11 15:28:27 -07:00
Martin Molinero 3e09d2f3c9 Adding new ReferenceWrapper
- Adding new ReferenceWrapper for structs, value types, to avoid thread
  race conditions while reading and writting. In C# reference type
  assignments are atomic, so it allows us avoid using locks.
2020-04-30 19:56:15 -03:00
Martin Molinero 7989d41aa7 Fix SecurityCache OHLC data type
- Fix for Equity SecurityCache OHLC default data type to be used. Will
  ignore QuoteBars
2020-04-08 19:31:37 -03:00
Martin Molinero 4eac5e4142 Add comment in Slice.cs
- Add a comment in Slice.cs about temporary patch until GH issue 4196 is
  fixed
2020-04-07 11:30:41 -03:00
Martin Molinero 5671a3293a Slice dynamic accessors ignore Quote for equities
- Slice dynamic data accessors will ignore QuoteBars and
Ticks TickType.QuoteBars for equities. This is to make slice dynamic
accessors deterministic and avoid breaking backwards compatibility when
adding QuoteBars for equities.
2020-04-06 10:33:02 -03:00
AlexCatarino 64b78df045 Addresses Peer-Review
- Fixes:
 - `Slice.GetValues`
    Not returning the `Value` but the `KeyValuePair`
 - `ExtendedDictionary.update(PyObject)`
    Paramenter type should be `PyObject` not `PyDict`
- Adds unit test for `update()`
- Fixes `OrderListHash` of `PythonDictionaryFeatureRegressionAlgorithm` test.
2020-04-01 16:15:05 +01:00
AlexCatarino 2c0d7e9ebe Implements Logic in ExtendedDictionary.this[string]
And removes it from chidren implementation.
2020-04-01 16:15:05 +01:00
AlexCatarino 27d31b7846 Renames BaseDictionary to ExtendedDictionary
Changes the return type of `keys()` and `values()` methods to `PyList`. Since `Keys` and `Values` return `ICollection` in some cases, the Python type was not consistently of `list` type.
2020-04-01 16:15:04 +01:00
AlexCatarino a59bd17fb9 Adds BaseDictionary That Implements Python Dict Methods
`BaseDictionary` is an abstract implementation of `IExtendedDictionary` keyed by `Symbol` that implements Python `dict` methods. `Slice`, `DataDictionary`, `SecurityManager`, and `SecurityPortfolioManager` derives from it in order to behave like Python `dict`.
2020-04-01 16:15:04 +01:00
Martin Molinero 07a4285baf Test approach 2020-04-01 16:15:04 +01:00
AlexCatarino c1e18f82bb WIP 2020-04-01 16:15:04 +01:00
Martin Molinero b28f9f501a Refactor and fix Slice bugs
- Fixing Python slice enumeration which would not happen when custom
data was present. Adding unit test
- Fix Slice constructor which would fail to create data collections due
to relying on non-deterministic 'slice._data'. Adding unit test
- Improving custom data retrieval for symbols which have more than 1
data type in a slice. Adding unit test
- First step refactoring slice internally, no behavior changed
2020-03-27 12:44:05 -03:00
Martin Molinero 2a2711d056 IndicatorWarmup fixes
- WarmupIndicator will be able to determine the correct type to use
- Fix bug in `History.GetMatchingSubscriptions()` which would use the
same TZ for exchange and data. Covered by regression algorithm.
- Consolidate will only infer `TickType` from `T` is not abstract
- Adding regression algorithm
- Slice will expose `Get(Type)` to get data by type, adding unit tests
2020-03-25 18:52:48 -03:00
Jared 57010f2eee Revert "Slice[Symbol] return type order" 2020-03-11 19:14:16 -07:00
Martin Molinero 05fc9a5006 Keep track of types and always return based on the same order
- SymbolData at Slice will keep track of the types it has and GetData
will always return in the same order, TradeBars, QuoteBars, Tick, Custom
- Fixing slice.Get(type) for custom type which relied on insertion order
- Updating regression test
2020-03-11 22:13:34 -03:00
Martin Molinero 0f595efa1f Performance improvements
- Only define `DEBUG` is `SelectedOptimization` is not defined
- `DynamicSecurityData` will Keep a cache of the generic types
- Avoid using so much linq at `Security.Update()`
- `Slice` will also keep a cache of the generic types
2019-10-09 11:46:21 -03:00
Martin Molinero 0b9862a7a7 Fix for PythonSlice 2019-10-02 23:22:52 -03:00
Martin Molinero 57a0934e78 Reduce dynamic usage 2019-10-02 20:03:44 -03:00
Martin Molinero ec203be632 Self review - improvement 2019-10-02 18:25:20 -03:00
Martin Molinero a349f619a0 Add Slice.Get(Type) for python
- Adding `Slice.Get(Type)` and `Slice.Get(Type, Symbol)` for python use case
- Adding unit tests
2019-10-02 18:19:44 -03:00
Michael Handschuh d709d1c4e0 Update Common to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to go through the new
StringExtensions methods that use CultureInfo.InvariantCulture

See #3045
2019-09-12 14:13:24 -04:00
Stefano Raggi 5d90a3ef87 Fix generic Slice.Get to handle TradeBar and QuoteBar for the same symbol 2019-01-23 11:45:06 +01:00
Martin Molinero ff5e888313 Fix for delisted subscription data being sent 2018-07-18 09:45:13 -03:00
Stefano Raggi 0c30fd84bc Generalize data type used in Slice.GetKeyValuePairEnumerable 2018-05-02 20:49:05 +02:00
Stefano Raggi 311703dba4 Fix RuntimeBinderException when enumerating Slice with tick data
Closes #1949
2018-05-02 17:34:40 +02:00
quant1729 3819561a82 Merge branch 'futures' into master-futures-merged
Conflicts:
	Common/QuantConnect.csproj
	Tests/RegressionTests.cs
2016-12-01 17:32:40 +01:00
Michael Handschuh 591f6b2127 Use IBaseData in type constraints
Refactors existing consolidators, indicators, and helper methods to depend on
IBaseData instead of BaseData. These updates also defines an IBaseDataBar to
act as an abstraction point between TradeBar and QuoteBar.
2016-11-09 09:20:16 -05:00
quant1729 1141cd21e6 Merge branch 'futures' into master
Conflicts:
	Algorithm.CSharp/QuantConnect.Algorithm.CSharp.csproj
	Algorithm/QCAlgorithm.cs
	Brokerages/InteractiveBrokers/InteractiveBrokersBrokerage.cs
	Common/QuantConnect.csproj
	Common/Securities/Option/Option.cs
	Data/equity/usa/factor_files/aapl.csv
	Data/symbol-properties/symbol-properties-database.csv
	Engine/DataFeeds/Enumerators/Factories/BaseDataSubscriptionEnumeratorFactory.cs
	Engine/DataFeeds/Enumerators/Factories/OptionChainUniverseSubscriptionEnumeratorFactory.cs
	Engine/DataFeeds/SubscriptionDataReader.cs
	ToolBox/QuantConnect.ToolBox.csproj
2016-11-07 16:18:11 +01:00
quant1729 80e20788b0 Added FutureChains property to Slice class 2016-10-03 19:55:28 +02:00
quant1729 f89ea2183f Added real-time support for futures. Tested with IQFeed, IB. 2016-09-28 19:10:03 +02:00
Michael Handschuh bf7f8bc72d Adds QuoteBars/OptionChains to Slice 2016-04-07 13:39:28 -04:00
jameschch@outlook.com 0ae38360b3 Fix for History<Tick> 2016-03-03 21:41:37 +00:00
snugs a5dd738a52 Fixes bug in Slice ctor HasData 2015-12-09 13:32:32 -05:00
snugs 4f73c7ecea Make Slice DataDictionary lazy eval 2015-12-07 12:02:16 -05:00
snugs ef4e3f5ab5 Clean up warnings from bad pragma comments 2015-11-19 19:46:36 -05:00