- Adding `IRealTimeHandler.OnSecurityChanged()` will be used to update
the `OnEndOfDay` security related scheduled events
- Adding `BaseRealTimeHandler.cs` to reduce code duplication in the
`Backtesting` and `LiveTrading` `RealTimeHandlers`
- Adding CSharp and Python regression tests
- Deprecating `OnEndOfDay()` callback because of two reasons, mainly
because Python does not support two methods with the same name, but also
because different assets have different market close times.
- `ScheduledEvents` set at the same time will now be deterministic
- Adding new `StepTimeProvider` that will advance time based on a
desired custom evaluator.
- Live trading `Options`, `Futures` and `Coarse` data will use a
`FrontierAwareEnumerator` + `StepTimeProvider`. This will allow to hold
the selection data until its the desired time
- `Universe.CanRemoveMember` will round the members time in the universe
based on the `UniverseSettings.MinimumTimeInUniverse`
- C# `Timer` can sometimes be triggered before the expected due time.
Implementing `RealTimeScheduleEventService` that will guarantee a
scheduled event due time is respected and the event never happens before
time.
- `TextSubscriptionDataSourceReader` will now cache raw price mode data
points for each requested local source file, per data type.
Caching will be handled by the `MemoryCache` class.
- `GetLastKnownPrice()` will not `RoundDown` end time. This was causing
it to fail to fetch a data point in some cases.
- Adding new `PriceScaleFactorEnumerator` that will scale raw prices
based on a provided `SubscriptionDataConfig` and update the
`SubscriptionDataConfig.PriceScaleFactor`. Adding unit tests.
- `BaseData` factories (`TradeBar.cs`, `QuoteBar.cs`, `Tick.cs`) will
no longer scale factor prices, they will generate data points in raw
mode.
- `SubscriptionDataReader` will no longer update the
`SuscriptionDataConfig.PriceScaleFactor`
- Fix `BrokerageTransactionHandlerTests` unit test that was having a
race condition.
- `UniverseSelection` class will now keep track and remove internal
currency subscriptions through the usage of new class
`CurrencySubscriptionDataConfigManager`
- Modifying existing regression tests to assert bug 2773.
- Adding new regression test that fixes the case where a new cash is set
after initialization. This could happen during cash sync performed by
the `BrokerageTransactionHandler`.
- Adding new `CashAdded` event for the `CashBook` to solve the
issue.
- Moving `UniverseSelection.EnsureCurrencyDataFeeds` call into the
`IResultHandler` implementation through usage of the new `SetupHandlerHelper`
class, that will also set an initial conversion rate if none present.
- Adding regression test, that reproduces original issue
- Moving `Splits`, `Dividends`, `Mappings`, `Delistings`, from the
`SubscriptionDataReader` into there own enumerator, inheriting from
new `CorporateEventBaseEnumerator`
- Adding new helper `static class CorporateEventEnumeratorFactory` to
create the new enumerators
- Adding unit tests for new `CorporateEventBaseEnumerator`
- Adding some extra checks to `HourSplitRegressionAlgorithm` and
`DelistingEventsAlgorithm`
- Removing unneeded `CashBook` instance to create a new `TimeSlice`
- Adding new `TimeSliceFactory`, an instance base class that will
provide methods to create a new `TimeSlice`. Will own the `DateTimeZone`
property.
- Adding new `ISecurityPrice` and `IOptionPrice` that will provide a
reduced interface for accessing price properties and methods used when
creating a new `TimeSlice`
- The class `Subscription` will internally track each `Universe`
`SubscriptionRequest` added or removed
- Adding regression test in which two different `Universe` request the
same `SubscriptionDataConfig` and one of them removes/adds it in a
toggle fashion (fails on current master)
- `UniverseSelection` pending removals will also be tracked by
`Universe`
- `UniverseDecorator` will overwrite the `Universe` member of
`SubscriptionsRequests` at `GetSubscriptionRequests()`. This is due to
`this != this,Universe`
- Adding `Subscription` unit tests covering expected behavior
- Extracting pending removals logic from `UniverseSelection` class into
a new helper class `PendingRemovalsManager`. This new class will keep
track of the `universes` requesting to remove a security. Adding unit tests
- Creating new `Synchronizer` which will consolidate and combine `TimeSlice`
streaming for both live and backtests modes. Will live in front of the
`DataManager`
- `SubscriptionSynchronizer` will be the `FrontierTimeProvider` exposed,
and owned, by the new `Synchronizer`
- Updating existing tests
Provides a common abstraction for UserDefinedUniverse and the
yet-to-be-impemented ScheduledUniverse. Categorized by the fact
that they don't operate on data, but instead a fixed schedule and
any data required is fetched through external means, such as a
web request.
The term 'alpha' is used to describe the entire algorithm. Therefore, 'alpha'
produces insights. From this we have things like IAlphaModel, which is the model
defining how insights are produced. We have IAlphaHandler, which defines how the
insights from a single 'alpha' (the algorithm) are managed, analyzed, and stored.
Types closer to the individual prediction level, such as InsightDirection, or
InsightScore relate directly to exactly 1 insight. The distinction between the
two became more clear as we developed the insights API, and from that effort it
was decided to harmonize alpha/insight terminology across the various QC systems.
When inspecting regression differences, the first thing that should be looked at
are the fills. Some of the regression algorithms log this data making it possible
to inspect whie others do not. In addition, for algorithms with many securities the
log can quickly become fills with noise from scheduled event logging.
This change aims to make it very easy to compare any regressions in orders/fills
against the most recent successful run of the specified regression algorithm.
A large percentage of synchronization time is spent performing time zone conversions.
Previously, it was possible that we may compute the same tz conversion multiple times
on the same piece of data until it's emitted. This change moves the time zone conversion
logic onto the data feed thread right before it puts the data into enqueueable enumerator.
In addition, we're not performing the clone operation at the same moment. This removes
unnecessary computation while moving the necessary computation to the data feed thread to
lighten the load on the synchronization/algorithm thread.
Provides an abstraction point allowing us to easily substitute in a different
synchronization algorithm. This isn't 100% complete as I'm hopefully able to
remove the universe selection dependency from the synchronization implementation.
The IAlphaManagerExtension defines a type that needs to react to events produced
by the AlphaManager. The actual events were removed in favor of a interface to
handle the events. This removes the need to wire events and instead just pass the
extensions to the alph manager and it will handle invoking the extensions at the
appropriate time.
This change removes all charting and statistics aggregation logic from the alpha
handler and moves it into dedicated types, AlphaChartingManagerExtension and
AlphaStatisticsManagerExtension. The resulting types are highly decoupled from the
LEAN ecosystem allowing them to be easily unit tested, whereas before the logic
was embedded in a handler with many many dependencies which would be very hard to
properly unit test.
As part of this change (and in preparation for moving scoring to the alpha thread)
the resolution of SecurityValues was removed from the alpha manager. In this new
pattern, the alpha manager is pushed generated alphas and security values at each
time step.
- Move BacktestingFutureChainProvider provider to Lean.Engine.DataFeeds along with its options equivalent.
- EmptyFutureChainProvider: provider that returns an empty list of symbols
- CachingFutureChainProvider: implements caching by date
- BacktestingFutureChainProvider: provider that gets chain from local files
- LiveFutureChainProvider: provider that gets chain from external source (empty list of symbols for now)
Previously the number of symbols that could be selected was limited by estimates of RAM usage calculated using the resolution of subscription, now it will only be limited by the physical RAM available.
Defines an enumerator factory for live custom data. This is a refactoring
and bugfix over the previous implementation. This implementation now checks
the transport medium of each refresh of the stack to see if it's a file or
a rest server and provides the appropriate fast forward/rate limiting behavior.
Stefano Raggi