48 Commits

Author SHA1 Message Date
Martin-Molinero 14427129c7 Minor fix to handle all tick security updates (#9117) 2025-12-08 17:41:28 -03:00
JosueNina ff1638f653 Support CustomData in SecurityCache.GetData(PyObject type) method (#9036)
* Initial Solution

* Add regression algorithms

* Solve review comments

* Update CustomDataSecurityCacheGetDataRegressionAlgorithm.py

---------

Co-authored-by: Martin-Molinero <martin.molinero1@gmail.com>
2025-10-16 10:37:00 -03:00
Martin-Molinero ff2d749d61 Minor performance improvement for session tracking (#8990)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2025-09-25 18:58:41 -03:00
JosueNina f7ce35ae82 Track OHLCV for Current Trading Day (#8922)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Implement Session class

* Use MarketHourAwareConsolidator inside Session

* Delete SecurityCacheSessionConfig

* Refactor session class

* Solve review comments

* Fix regression algorithms

* Add null check to AvailableDataTypes

* Update session class

* Solve new review comments

* Add new test for futures and update regression test to PEP8

* Add new regression algorithm for Session with futures

* Resolve review comments

* Add local time keeper to SecurityCache

* Scan on consolidator updates and time updates

* Add new regression algorithms and unit tests

* Add unit tests and improve comments

* Fix syntax test

* Solve review comments

* Add regression algorithms

* Resolve review comments

* Make SessionConsolidator implement IDataConsolidator

* Add caching to avoid recreating SessionBar instances on every access

* Solve review comments

* Make SessionConsolidator inherit from MarketHourAwareConsolidator

* Add unit test to verify Session preserves previous day values

* Update regression algorithms

* Solve review comments

* Resolve review comments

* Solve new review comments

* Add unit test for Session class

* Refactor SessionBar properties and Update method

* Use WorkingDataInstance reference instead of _workingData.Clone()

* Use a delegate to get the workingBar instance

* Fix bug in MarketHourAwareConsolidator

* Solve review comments

* Handle OI values

* Add regression algorithms for future contracts and options

* Solve review comments

* Refactor SessionConsolidator to inherit PeriodCountConsolidatorBase

* Prevent overlapping higher resolution data

* Update tests

* Solve review comments

* Minor tweaks
2025-09-19 12:25:24 -05:00
JosueNina cc38a6410f Add python SecurityCache.GetData method (#8724)
* Initial solution

* Fix regression algorithms
2025-04-28 11:46:31 -03:00
Martin-Molinero d57f75ade2 Fix for division by zero in option assigment check (#8644)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Fix for division by zero in option assigment check. Adding regression
  test reproducing issue. Will now rely on normalized path in universe
  selection to clear the cache
2025-03-21 18:18:48 -03:00
Jhonathan Abreu 021e507ef0 Holdings price adjustment after split (#7985)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix: properly adjust prices after a split

* Minor fixes

* Minor: Adjust cached data on split

* Minor change

* Expand regression algorithms assertions
2024-05-01 17:42:43 -04:00
Martin-Molinero 3eedde9b6e Normalize ETF constituents universe (#7858)
- Normalize ETF constituents universe data. Adding new tests and
  assertions
2024-03-19 11:28:17 -03:00
Martin-Molinero f8b258d329 New Fundamental Data (#7490)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* New Fundamental Data

* Minor CIK lookup fix

* Handle live mode & delete unexisting properties

* Minor coarse fundamental adjustment

* Add fundamental history support

* Fix unit tests

* Performance improvements

* Fixes

* Minor regression algorithm fix

* Improvements. Add FundamentalUniverseSelectionModel

* Change default values

* Fix unit test

* Minor tweaks

* Fix unit test

* Minor error handling improvement

* Fix rebase
2023-10-25 15:47:42 -03:00
Jhonathan Abreu 956428b7bf Security cache dynamic collection for custom properties (#7327)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Dynamic security properties for quick objects storage

* Documentation and other minor changes

* Add Security indexer to access custom properties

* Address peer review

* Minor changes

* Improve regression algorithms

* Bumped pythonnet to version 2.0.19

* Bumped pythonnet to version 2.0.20

* Address peer review
2023-06-27 10:39:20 -04:00
Martin-Molinero a7516b80ea Daily future cash settlement (#7226)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Refactor settlement mode api

- Add new Scan method on ISettlementModel API. To be used by
  futures settlement model mark to market handling
- Use 'Parameters' classes for 'ISettlementModel'

* Address reviews

* Daily future settlement

- Implement daily future settlement. Adding unit & regression tests

* Address reviews

* Assert cash balance every day
2023-05-01 11:25:37 -03:00
Martin-Molinero f12e8fc08e CryptoFutures adjustments (#6836)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* CryptoFutures adjustments

- Address reviews of CryptoFutures:
- Add new Slice MarginInterestRates collection
- Add new regression tests asserting funding rate application behaves
  the same no matter the resolution
- Add auxiliary data by type into the security cache
- Revert BuyingPowerModel changes

* Make interest rate application time deterministic
2023-01-05 18:07:01 -03:00
Martin-Molinero 45e13bb35b Memory Related Performance improvements (#6700)
* Memory Related Performance improvements

- Make sure we cleanup & dipose of python related objects during pandas
  data generation.
- Disable memoizing enumerable use while creating pandas data frames,
  since we do not require it
- Reduce unrequired object creations
- Replace concurrentCollections for ordinary locks

* Decimal parsing typo fix
2022-10-21 15:11:18 -03:00
Martin-Molinero 575f17c6ec Order command improvements (#6584)
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Create SubmitOrderRequest method in algorithm API to reduce
  duplication of code and responsability.
- Add order command tests for different failure and success cases
- Fix null reference exception in equity fill model, when there is no
  data for an asset.
2022-08-25 15:53:11 -03:00
Martin-Molinero 06c05c4274 Fully reset security cache (#6277)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fully reset the security cache when it's finally removed from the algorithm.
   Adding regression algorithm reproducing issue
- Updating regression algorithms which would trade based on data
  previously available
2022-04-01 15:00:29 -03:00
Martin-Molinero 87db3fe379 Performance improvements (#6182)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- User server mode for GC
- Remove SecurityType cache type check
2022-02-02 19:52:45 -03:00
Martin-Molinero 4a0fb30df5 Fixes for OpenInterest storing (#4712)
- Fix Slice.Get OpenInterest type. Adding unit test
- Fix for SecurityCache that wasn't storing OpenInterest types
- Updateing regression tests to covere these usages
2020-09-11 15:28:27 -07:00
Alexandre Catarino a14f8f1f47 Refactors SecurityCache to Support Access to Tick of Different Type (#4621)
* Refactors SecurityCache to Support Access to Tick of Different Type

Adds two read-only lists of BaseData to save the last list of Tick of TickType.Trade and TickType.Quote. Changes methods accordingly to get and set these lists.

* Addresses Peer Review

The assignment of `_lastData` is moved to keep the current behavior of not beeing set with fill forward data.
`Reset` and `ShareTypeCacheInstance` implements the new class objects.

Adds unit tests for `Reset` and `ShareTypeCacheInstance`.

* Only Cache Non Fill-Forward Data

Undo changes that made AddData cache non fill-forward data.
2020-08-24 10:37:50 -03:00
Martin Molinero 6ad123ad8c Update regression algorithms stats
- Update regression algorithms stats after making SecurityCache ignore
  QuoteBars for equity for OHCL values and GetLastData(). They were
  affected since the `BenchmarkSecurity` used `.Price` which was QB for
  equities. Order list hashes changed because SubmissionLastPrice will
  now be TB instead of QB
2020-04-08 19:31:42 -03:00
Martin Molinero 7989d41aa7 Fix SecurityCache OHLC data type
- Fix for Equity SecurityCache OHLC default data type to be used. Will
  ignore QuoteBars
2020-04-08 19:31:37 -03:00
Martin Molinero b00a599e16 Address reviews
- Address reviews and rebase
2019-10-22 13:47:45 -03:00
Martin Molinero a5dc1774a7 Add SecurityCacheProvider
- Adding `SecurityCacheProvider` this class allows for two different
`Security` to share the same data type cache through different instance
of `SecurityCache`. This is used to directly access custom data types
through their underlying in a peformant maner
- Some small improvements
2019-10-22 13:46:57 -03:00
Martin Molinero b2706b427c SecurityCache performace improvements
- `DynamicSecurityData` will be a view into the `SecurityCache` instance
- Custom data which has an underlying will use the underlying
`SecurityCache` data type cache instance
- Refactors for `Security` and `SecurityCache` to avoid storing twice
the same data points in the data type cache
2019-10-22 13:46:56 -03:00
Martin Molinero 9ec1a41235 Performance improvements
- `PortfolioTargetCollection` avoid calling `Count` on
ConcurrentDictionary directly -> has to take all locks
- `SecurityChanges` change Union for Concat since constructor will call
HashSet
- Make `DynamicSecurityData` hold lazy data objects
- `RegisteredSecurityDataTypesProvider` avoid looping over all
registered types, adding `TryGetType`
- `Security.Update()` will no call group by on data since this data is
already grouped by type. Adding `ContainsFillForwardData` will allows to
be lazy and not re loop through the data unless necessary
- `DefaultAlphaHandler` will use the `static`
`Enumerable.Empty<Insight>` instance when possible
- `SubscriptionSynchronizer` will be lazy to construct the
`universeData` dictionary which is not used in most of the times. Will
use `Count` vs `Any` -> `Count` is known by the dictionary
- For python algorithms `JobQueue` will respect `AlgorithmLocation`, was
using unexisting `"algorithm-path-python"`
2019-10-17 10:26:37 -03:00
Martin Molinero 0f595efa1f Performance improvements
- Only define `DEBUG` is `SelectedOptimization` is not defined
- `DynamicSecurityData` will Keep a cache of the generic types
- Avoid using so much linq at `Security.Update()`
- `Slice` will also keep a cache of the generic types
2019-10-09 11:46:21 -03:00
Michael Handschuh 8c1867193b Add Security.Data => DynamicSecurityData
Provides dynamic access to cached security data keyed by the type's name.
For example, `security.Data.GetAll<Tick>()` would yield a list of ticks.
Likewise, using the dynamic accessors, `((dynamic)security.Data).Tick`
would return the same list. In C# you'll need to cast security.Data to
a dynamic. In python, all C# objects are viewed as dynamic, so python can
simply access `security.Data.Tick` directly.

See #3620
2019-09-30 13:31:56 -04:00
Michael Handschuh aa03f0739d Update SecurityCache to hold list and save custom derivatives
Custom derivative data is now being saved into the underlying security's
cache. This makes the custom derivative data available via the underlying's
security object via underlying.Cache.Get<T> where T is the custom data type.
2019-09-30 13:31:55 -04:00
AlexCatarino a96d6725aa Fixes SecurityCache tick update
Trade ticks are not allowed to update quote variables and vice-versa.
Explicitly define the `TickType` of the tick with the subscription type.
2018-10-05 21:59:54 +01:00
Michael Handschuh ddcbc66cf8 Add SecurityCache.StoreData and use to store universe data
We never want universe data to impact a security's price or other cache
properties. To that end, this change adds another way of adding data to
the cache that simply stores the data for retrieval and does NOT update
any of the security cache properties. The check for aux data was left in
to prevent aux data from setting security prices as well.
2018-04-26 14:18:34 -04:00
Michael Handschuh 58efd4ae6a Store universe data in security cache
Universe data is piped through the TimeSlice and saved in the
security cache in the algorithm manager for consumption by the
algorithm.

This change includes an update to SecurityCache.AddData to preclude
us from setting the security price using auxiliary data. Tests were
updated accordingly.
2018-04-24 18:23:01 -04:00
Stefano Raggi c75cab6001 Remove unnecessary cast to Tick in SecurityCache.AddData
Also cleaned up unit tests
2018-04-18 11:18:09 +02:00
Stefano Raggi c203ff4c93 Fix open interest tick handling in SecurityCache
Fixes #1873
2018-04-16 17:03:22 +02:00
Michael Handschuh df48212dad Fixes security.Volume for options/futures
This change aims to keep the quote bar preference for values in the cache
while also ensuring that the volume data is properly being updated. Tests
were added to provide coverage around expected property changes from different
starting configurations (last data type) and a data update.
2018-03-28 06:23:08 -04:00
Juan José D'Ambrosio 60daa65d3f Update SecurityCache.cs 2018-03-21 14:46:15 -03:00
Juan José D'Ambrosio af8fb9f20e SecurityCacheTests added. 2018-03-20 08:45:16 -03:00
Juan José D'Ambrosio e22ca5ae6a BasicTemplateCrytpoAlgoruthm changed. 2018-03-17 23:57:53 -03:00
Juan José D'Ambrosio 45834d87c5 Option tests pass 2018-03-17 20:46:28 -03:00
Juan José D'Ambrosio dc048e2c10 SecurityCache doesn't cache fill-forward data.
version 8


version 10


PR version
2018-03-17 20:46:28 -03:00
Andrew 01487bbf90 Add return statement to SecurityCache when tick arrives 2017-09-11 14:10:57 -04:00
Andrew e69022ce17 Update SecurityCache.Volume when Tick passed to SecurityCache.AddData() 2017-09-11 13:11:38 -04:00
Stefano Raggi 508e63af4d Update Bar types Volume and Bid/AskSize from long to decimal
This PR is another required step towards crypto-currency support.
2017-06-26 21:14:55 +02:00
quant1729 586c2b4006 Fixing issue with invalid data in portfolio holdings for futures 2016-12-09 14:40:53 +01:00
quant1729 1305d96edd Open interest support: backtesting, converters, live data handlers (IQFeed/IB), live trading (IB). First cut. Regression Test. 2016-11-15 21:33:33 +01:00
Michael Handschuh 5b4b5326d1 Updates ImmediateFillModel/SecurityCache to properly respect bid/ask in all cases 2016-04-07 07:04:01 -04:00
Michael Handschuh 4f7908019d Improve SecurityCache to hold different types of data 2016-03-31 20:00:48 -04:00
QuantConnect fde0cbed3a Strip out block quote section header comments 2015-06-13 16:04:27 -04:00
snugs 89b6e98af8 Fixes warnings mostly relating to xml comments
Also cleans up some block comments

Via PR#72 - thanks @ammachado!
2015-05-13 10:56:47 -04:00
Jared Broad b3de520915 Initial commit 2015-01-12 12:03:33 -03:00