- Custom data types will know whether or not Lean should use map files
- Updating regression test with sample custom data using map files,
which can run locally
- Adding unit tests for the `SubscriptionDataReaderHistoryProvider`,
checking it mappes equities and options correctly
- We will now check if python selection method returned `Universe.Unchanged`
- Removing `ToList()` call on fine and coarse data before sending it to
the python algorithm
- Adding regression algorithms
In order to provide full Lean Indicator functionality to python custom indicators, they need to inherit from a C# class. `PythonIndicator` will serve for this purpose.
Algorithms can use the former version (no inheritance).
In order to add support custom python indicators for `QCAlgorithm.PlotIndicator`, we created a `PythonIndicator` class that wraps the custom python indicator. In `QCAlgorithm`, the reference of the wrapper is saved into a dictionary keyed by the python indicator handle.
- Adding _some_ of the missing PyObject.Dispose calls. In the cases
where C# is calling the Python side.
- Note that Python calls to C# code is correctly handling the
disposure of resources.
Replaces Enum `CalendarType` for static class with the same name. This class defines two properties (`Weekly` and `Monthly`) that can be used to define the previous calendar date (Monday or 1st of current month) which will correspont to the `Time` of a `IBaseData` object.
Refactor `PeriodCountConsolidatorBase` to define use `GetRoundedBarTime` based on a period specification that depends on the constructor overload: `integer`, `TimeSpan` or `Func<DateTime, CalendarInfo>`. The last one can be set with the `CalendarType` properties.
- Adding new `HistoryRequestFactory` class. Will provide some methods to
facilitate the creation of new `HistoryRequests`. Moving
`CreateHistoryRequest` and `GetStartTimeAlgoTz` into the new
`HistoryRequestFactory`. And consolidating `GetStartTimeAlgoTz` and
`GetStartTimeAlgoTzForSecurity`.
By using the python object parant class, which is either `PythonQuandl` or `PythonData`, instead of `DynamicData`, the `AlgorithmManager.Stream` method can find a matching subcription data configuration used to create a data feed packet.
Closes#2694
- Adding new ISecurityService and its implementation SecurityService.
Expose by SecurityManager.
This class will expose a method for creating new securities. The
SecurityManager is exposing this new interface, calling _securityService
internally, so Future/OptionUniverseSelectionModel.cs can use it
- Replacing all usages of SecurityManager.CreateSecurity for new
ISecurityService
- Modifying `Cash.cs` and `CashBook.cs` `EnsureCurrencyDataFeeds()` to
return newly added `SubscriptionDataConfig` instead of `Security`. This
will avoid using `Security.Subscriptions` at call site.
- Moving old SecurityManager.CreateSecurity into new
SecurityServiceTests.cs
We didn't experience the expected performance improvements. Locally under
unit test there was aboout an order of magnitude throughput increase, but
when run against the history benchmark, this new approach was 60% slower.
We're reverting this for now to perform further analysis and better
understand the performance profiling of the python history stack.
- Do not throw, but log, when a entry in the zip file does not exist. It can happen in trade bar data for options/futures since a given contract may only exist as quote bar.
- Verify whether the minimum requires arguments ("periods", "span" and "start") are present in the dictionary used to pass the variables.
- Adds doctring to History.
- Fixes type check for custom data.
Since futures and options have multiple entries in the zip file, we need to include the entry name in the request.
Futures have multiple open market periods during on trading day, therefore we need to apply a more complex mask to pandas dataframe index.
GetSymbolsFromPyObject now returns IEnumerable<Symbol> and throws if its argument is not Symbol or Symbol[] object and if the Symbol has not been added to the Securities object.
When creating a CRL type in runtime to represent a python custom data class, we need define `DynamicData` as its parent class so that it passes the `IsAssignableFrom` condition in `SubscriptionManager.AddConsolidator`
The check for Lean indicators was testing for `Indicator`, `BarIndicator` and `TradeBarIndicator` types where it should check for `IndicatorBase<T>` which includes `WindowIndicator<T>`.
In order to access the custom data classes, the module containing them was added to the ObjectActivator. This was unnecessary if it wasn't a custom data algorithm.
Also, this operation would not be taken into account if the custom data class were defined after the algorithm was created: this is the case for QuantBook.
We refactor how custom data is handled: a new class was added to provide a instance creation factory that creates an instance of each python custom type.
For cases where implicit conversion to decimal from indicator was not properly handled by pythonnet, we expliticly get `Current.Value` and redirect to `Plot(string,decimal)` overload.
We enable Logging and Debug methods to accept python objects to avoid the need of calling the str method.
Those objects are safely converted into string objects.
It was not possible to add a security based on its historical data, since we needed to add the security before requesting its security data. Universe Selection algorithms are an example of such usage.
Add MarketHoursDatabase.SetEntry and SetEntryAlwaysOpen. This allows runtime modification of the
market hours database which is necessary for correct custom data time zone handling.
Extracts complicate ternary logic into its own method and make it human readable.
Set the market hours entry for custom data universe subscriptions defaulting to the security's time zone.
If we make a prediction for 1 day in the future, we actually mean 1 trading day.
This change updates the alpha analysis logic to take into account the security's
market hours.
In this PR we are disabling the default security seeding (automatically getting the last price for a security when added to the algorithm) for a couple reasons, both when using large universes:
- In live trading, these history requests are sent to a history server, potentially causing timeouts
- In backtesting, depending on the algorithm this could also cause slowdowns up to 30%
Unlike other methods used to add securities to the algorithm that return a Security or a derived object, AddData was returning void. This commit change it to return Security for consistency.
Michael Handschuh