Energy futures (CL, HO, RB, NG) expire in the month before the contract month.
To handle these properly, the following changes have been implemented:
- the AlgoSeekFuturesReader has been updated to create the future symbol with the correct expiry date
- the zip entry names in futures data files now contain the full expiry date (in addition to the contract month)
- the sample data files have been updated to use the new zip entry names
- new unit tests have been added and existing ones updated
- `FactorFile` will keep an ordered reversed list with the dates.
Calling `Reverse()` on the `SortedList` is expensive.
- `MapFiles` will keep first and last date, so we don't need to call
`First()` and `Last()` multiple times.
- `Liquidate` will go through all the algorithms securities only if
necessary
- `TradeBar` parsing will not call `new T` for pure `TradeBar` which is
expensive
- Removing `Lazy` hash code and security type for the
`SecurityIdentifier`, replacing for direct initialization. Accessing the
`Lazy` value adds an overhead.
- Replacing `Enum` to string for hardcoded switch statement. `Enum.ToString` is expensive.
- `DataManager` will be lazy for counting the subscriptions for
determining if its above the limit
- Adding `AlgorithmSecurityValuesProvider.GetAllValues()`, removes the
need to fetch all the security keys twice.
- During universe selection, will not try to re add already added symbol
This project can take trade as well as orderbook information from Kaiko
and convert the data into Lean compatible data at all resolutions in
modeled as both trades and quotes.
The Crypto security can be modeled with both trades and quotes easily, therefore it is necessary to make sure that the LeanData, LeanDataWriter and associated BaseData.Reader() methods can read/write both Trades and Quotes.
Because cryptocurrencies can be either Trades or Quotes, the Crypto security type needs it's own switch statement in LeanData.GenerateLine. This produces non-scaled TradeBars or QuoteBarrs.
- Open interest should use yyyyMMdd HH:mm in csv file when frequency is daily or hourly
- Filename should be [baseticker]_[quote|openinterest|trade]_yyyyMM.csv
Remove all trailing 0s in LeanData.GenerateLine(). This creates smaller data files and prevents large numbers of trailing zeros on data points. This change will apply to all data written with LeanData.GenerateLine.
This method takes a path and will return a Security and DateTime. This is useful in the ApiDataProvider that uses the api to retrieve files that are not present on disc. The IDataProvider only ha a path to the file. Using this method, the ApiDataProvider can extract the useful information from the path to communicate with the api.
This method is used by both Future and OptionsChainUniverse. This method is useful, espeically in the SecurityManager. It will eventually be used there.
The current implementation of generate line doesn't take into
account that base data instances are typically time stamped
with their exchange time zone, but must be written to
disk in their data time zone.
- Fixed issue with changing option file formats (column order)
- Added support for symbol constraints to filter the symbols we may need
- Temporarily added support for command line argument (date)
Futures converter:
- First version. Works with supplied data files.
Stefano Raggi