* DataConsolidator Wrapper for Python Consolidators
* Regression Unit Test
* Refactor Regression test
* Bad test fix
* pre review
* self review
* Add RegisterIndicator for Python Consolidator
* Python base class for consolidators
* Modify regression algo to register indicator
* unit test - attach event
* Test fix
* Fix test python imports
* Add license header file and null check
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
- AlgorithmPythonWrapper will directly call base OnFrameworkData()
implementation skipping going through python and it's overhead
- Small performance improvement for adding Tick data points into a Ticks
collection
- For python always wrap slice with PythonSlice, so that slice.Get()
works even when no custom data is present, adding test.
- Setting SPY as the default security benchmark
- The security benchmark subscription will be added at `UniverseSelection`
as an internal subscription. Using its own dedicated Security instance
which doesn't live in the algorithms.Securities collection.
- Reducing algorithms exposure to internal subscriptions
- `TimeSliceFactory` will prioritize higher resolution bars, when same
symbol is present twice (for non-internal subscriptionst)
- Adding regression test `CustomUniverseWithBenchmarkRegressionAlgorithm`
Implements `DataNormalizationMode` field in `UniverseSettings` to enable the settings of a desired `DataNormalizationMode` to the securities that are chosen in Universe Selection.
Since subscriptions are enumerated in non-deterministic order, tick consolidators for multi-tick-type security types (such as Crypto, Future and Option) could end up being added to the wrong subscription, e.g. QuoteBarTickConsolidator added to a tick subscription with TickType.Trade. This was causing the Consolidated event handler to never be called.
The Resolution.Tick case is now handled properly, checking the subscription tick type.
- Adding new ISubscriptionDataConfigBuilder implemented by DataManager, exposed by
SubscriptionManager
- SecurityManager and SubscriptionManager will use new interface.
In a next PR, its intended for it to also be consumed by Universe.GetSubscriptionRequests().
- Moving LookupSubscriptionConfigDataTypes implementation into
DataManager
Refactoring previous commit so DataManager only keeps and receives a
reference to IAlgorithmSettigs, with the objective of reducing tight
coupling
Note: Investigate if IAlgorithmSettings.DataSubscriptionLimit should limit subscriptions or unique securities.
Today its using SubscriptionManagerSubscriptions.Where(x => !x.Symbol.IsCanonical()).DistinctBy(x => x.Symbol.Value).Count() @DataManager
All consolidators now clear the event handlers list when being disposed.
In addition, SubscriptionManager.RemoveConsolidator will now dispose of
the consolidator before returning. This ensures the consolidator and any
downstream indicators that were attached to it can be properly cleaned
by garbage collection.
This allows algorithms to remove consolidators at run time. This is especially
relevant with universe selection where we can add/remove securities at run time
and potentially be creating indicators as well. When the security is removed
from the universe we'll also want to remove any consolidators/indicators that
have been wired up as well.
Previously Tick resolution subscriptions only received ticks with TickType.Trade, now TickType.Quote and TickType.OpenInterest are received as well.
This PR replaces PR #1065
Futures and Options have multiple data types: QuoteBar, TradeBar and OpenInterest.
With these asset types only QuoteBarConsolidator could be used before, TradeBarConsolidator and OpenInterestConsolidator can now be used as well.
This flag will be used to speed python algorithms execution, since it avoids a wrapping operation that is only required when there is custom data in python algorithms.
Subscriptions can now be explicit as to whether or not a SubscriptionFilterEnumerator is to be applied.
Regular security price data subscriptions get filtered (user/market-hours)
Universe subscriptions don't get filtered (even if a subscription of equity price data, such as options underlying feeds)
Add dynamic data flag to config
Remove dynamic data flag from Security
Rename dynamically loaded flag to IsCustomData
Remove Security from SubscriptionDataReader ctor
This includes updating all usages of symbol as a security identifier to use the new type.
The type includes a unique field, SID, as well as the current ticker's value. This allows
for consistent addressability while also allowing the ticker to evolve over time with the
mapping changes.
Effort was made to maintain compile and runtime backwards compatibility.
Colton Sellers