- `TimeSliceFactory` will avoid creating empty collections
- `ExecutionModels` will check target collection count before trying to
enumerate
- Reduce calls to .`TotalPortfolioValue`
- `SecurityValues` will only be created when required
- `TimeKeeper` will use TimeZone unique Id as dictionary key. The
TimeZone hash is expensive.
- `AlgorithmManager` will avoid calling `DateTime.UtcNow`,
`ConvertFromUtc()` and `RoundDownInTimeZone()`
- `UniverseSelection` class will now keep track and remove internal
currency subscriptions through the usage of new class
`CurrencySubscriptionDataConfigManager`
- Modifying existing regression tests to assert bug 2773.
- Adding new regression test that fixes the case where a new cash is set
after initialization. This could happen during cash sync performed by
the `BrokerageTransactionHandler`.
- Adding new `CashAdded` event for the `CashBook` to solve the
issue.
- Removing unneeded `CashBook` instance to create a new `TimeSlice`
- Adding new `TimeSliceFactory`, an instance base class that will
provide methods to create a new `TimeSlice`. Will own the `DateTimeZone`
property.
- Adding new `ISecurityPrice` and `IOptionPrice` that will provide a
reduced interface for accessing price properties and methods used when
creating a new `TimeSlice`
- The class `Subscription` will internally track each `Universe`
`SubscriptionRequest` added or removed
- Adding regression test in which two different `Universe` request the
same `SubscriptionDataConfig` and one of them removes/adds it in a
toggle fashion (fails on current master)
- `UniverseSelection` pending removals will also be tracked by
`Universe`
- `UniverseDecorator` will overwrite the `Universe` member of
`SubscriptionsRequests` at `GetSubscriptionRequests()`. This is due to
`this != this,Universe`
- Adding `Subscription` unit tests covering expected behavior
- Extracting pending removals logic from `UniverseSelection` class into
a new helper class `PendingRemovalsManager`. This new class will keep
track of the `universes` requesting to remove a security. Adding unit tests
- Creating new `Synchronizer` which will consolidate and combine `TimeSlice`
streaming for both live and backtests modes. Will live in front of the
`DataManager`
- `SubscriptionSynchronizer` will be the `FrontierTimeProvider` exposed,
and owned, by the new `Synchronizer`
- Updating existing tests
Adds the concept of universe disposal which is requested by an algorithm
through invocation of UniverseManager.Remove, which is invoked via
algorithm.RemoveSecurity. This instructs the data feed that the algorithm
has requested to completely remove the universe and any child subscriptions
from the feed. Security changes are fired for all removed securities.
With the enumerator in blocking mode (backtesting), MoveNext was returning true while Current was being set to null.
This was causing a NullReferenceException in SubscriptionSynchronizer.Sync.
If we pull data and on the same time step that security gets removed,
we can still get that data in OnData(Slice) even though it was removed.
This change filters out removed securities by tracking a reference to
the subscription's disposed flag. Another change was made to wait until
the end of the time step to dispose of subscriptions.
Add IDataFeed.GetSubscription(SubscriptionDataConfig)
The fine enumerator factor accepts a 'date'. This date is that day
that the data is produced, so the emit time on that data is midnight
the following day. When we go to read the fine data we can't use the
current time because that's the 'emit time' -- instead we need to back
the time up a full day to find the correct start time. In addition,
the fine data was being read using UTC time stamps which led to even
more confusion here -- this change resolves both issues.
In addition to the above, the security.Fundamentals property wasn't being
properly set for the first time step when a security is selected due to
a bug in the subscription synchronizer not clearing out the universeData
in preparation for another loop (in the event changes != None).
Fine universes are considered special due to their chaining behavior with
the coarse universe. UniverseSelection.ApplyUniverseSelection manually
reads the fine data for symbols that have passed the initial coarse filter.
There doesn't exist a mechanism for pipe this data back to the data feed
for inclusion in the TimeSlice object. As such, this hack relies on reference
semantics to updates the BaseDataCollection to include the merged coarse and
fine fundamental data.
Pipes universe data from the data feed, through TimeSlice.Create and adds
TimeSlice.UniverseData dictionary property for read access from the algo
manager, where the data will be placed onto the correct security object.
A large percentage of synchronization time is spent performing time zone conversions.
Previously, it was possible that we may compute the same tz conversion multiple times
on the same piece of data until it's emitted. This change moves the time zone conversion
logic onto the data feed thread right before it puts the data into enqueueable enumerator.
In addition, we're not performing the clone operation at the same moment. This removes
unnecessary computation while moving the necessary computation to the data feed thread to
lighten the load on the synchronization/algorithm thread.
The underlying price data was offset one time step in the future due to the
usage of `subscription.Current` after we've performed the synchronization loop.
At this time, `subscription.Current` is guaranteed to either be null
(`EndOfStream`) or be ahead of the frontier
Provides an abstraction point allowing us to easily substitute in a different
synchronization algorithm. This isn't 100% complete as I'm hopefully able to
remove the universe selection dependency from the synchronization implementation.
The concept of subscription frontier is only relevant to the subscription synchronizer.
This change removes the back and forth communication of the frontier data between the
data feed component and the synchronizer component
This caused a change in the CoarseUniverseTop5DollarVolumeAlgorithm. The changed values are
due to the subscriptions now correctly stopping before the turn of the new year. The exact
same number of data points are emitted and the same trades are made, but due to the subscription
stopping slightly sooner (correctly), we lose a benchmark sample which caused a slightly different
beta to be computed (along with dependent statistics).
This method was adding unnecessarily two types of DataFeedPacket objects.
The first type of packet contained the coarse data just used for universe selection, the other one contained only empty packets.
Since these packets would be filtered out later anyway, there is no point in passing them to TimeSlice.Create which needs to enumerate them.
This update reverts PR #664 and includes an improved filtering solution.
It prevents consolidators and indicators from receiving inputs at a resolution higher than the data subscription resolution, typically fill-forward bars.
Volume-based indicators and consolidators will now be calculated correctly with fill-forward data.
It also fixes the RegressionAlgorithm failing test.
Before now, universe data was provided as a single subscription.
Now, you may have many subscriptions feeding one universe, so we
aggregate the universe data and then invoke universe selection.
This change also removes the restriction of a single subscription
per symbol
Martin Molinero