- Moving `MapFile` and `FactorFile` resolution to the data feed stack so
that they do not add a performance overhead to the algorithm thread.
- Create logging string messaged only if required.
- Calculate `FactorFileRow.PriceScaleFactor` the least amount of times
- Adding `LazyToUpper()` implementation, that will avoid the call to
`ToUpper` if the string is already upper.
- Reduce the timezone conversions at `Time.EachTradeableDayInTimeZone`
- `TotalPortfolioValue` will iterate over all securities once
- Adding new `SecurityIdentifier` cache, significant impact for
algorithms using coarse/fine data
- `FactorFile` will keep an ordered reversed list with the dates.
Calling `Reverse()` on the `SortedList` is expensive.
- `MapFiles` will keep first and last date, so we don't need to call
`First()` and `Last()` multiple times.
- `Liquidate` will go through all the algorithms securities only if
necessary
- `TradeBar` parsing will not call `new T` for pure `TradeBar` which is
expensive
- Removing `Lazy` hash code and security type for the
`SecurityIdentifier`, replacing for direct initialization. Accessing the
`Lazy` value adds an overhead.
- Replacing `Enum` to string for hardcoded switch statement. `Enum.ToString` is expensive.
- `DataManager` will be lazy for counting the subscriptions for
determining if its above the limit
- Adding `AlgorithmSecurityValuesProvider.GetAllValues()`, removes the
need to fetch all the security keys twice.
- During universe selection, will not try to re add already added symbol
Fixed bug where factor file reference price output had very high
precision
Fixed bug where map file had a useless entry before the final line if
the asset was not delisted
Fixed bug where program would crash because of NextPrice reaching its
maximum attempts. Fixed by removing call to NextPrice and using another
generation method.
Created random-seed argument for rdg in order to let the user get deterministic output
Update documentation in 'FactorFileRow.cs' to accurately reflect factor
file structure
Update CSV generation for FactorFile so that it uses FactorFileRow's CSV
generator
Add FinancialCalendar to make it easier to implement logic regarding
financial quarters
Add mapping events to RandomDataGenerator
Update MapFileRow ToCsv method to correctly emit the ticker as lowercase
Fix bug in FactorFile where we would get the same initial data point twice
when we converted it to CSV
Create new method to convert a MapFile to CSV
Create new method to write MapFile to disk as CSV
Add unit test to test for successful CSV generation in MapFile
Add new files to project
Add FinancialCalendar unit tests
Create new class to handle generation of dividends, splits, and maps
Adds rounding of price and scale factors when writing a factor file
row to csv. The precision provided is the same that is used by quant
quote in their raw files
The time check in FactorFileRow.Apply(Dividend) was not using the previous
trading date for comparison.
Added a 'source' parameter to FactorFileRow.ToCsv() for tracking each row's
data source.
The first ticker and first date are most commonly used to properly create
security identifiers. Currently resolution of the first ticker is strewn
about in various places and often times it doesn't take into account the
case of an empty map file, where the first (and last) ticker are assumed
to be the permtick after removing the '.<number>' specifier at the end.
It's important to also not remove '.<letter>' -- as some securities are
named as such.
Empty factor files (whether a single 2050 line or no lines), by definition don't
have any splits or dividends.
FactorFile.MostRecentFactorChange yields the maximum date that isn't the 2050 date
Adds reference price to dividend/split
Adds GetSplitsAndDividends to FactorFile
Adds Apply methods to FactorFile/FactorFileRow
Updates factor files to include reference prices
Some factor files contain duplicate rows causing a duplicate key exception when loaded. We now ignore these rows, allowing factors to be loaded and applied correctly.
Before this PR, backtesting some symbols fails with an error when parsing the factor file.
Some factor files have INF split values, indicating that the stock has so many splits that prices can't be calculated with correct numerical precision.
To allow backtesting these symbols, we need to move the starting date forward when reading the data.
Known symbols: GBSN, JUNI, NEWL
This allows one to define a subscription as the set of symbols obtained from parsing the zip entry names in a zip file. This will be used by option universes in order to determine what contracts are available and provide a data source for contract filtering.
Many places in the code used Log.Error(err.Message) or equivalent which
strips out all the really useful information, such as the stack trace
and inner exceptions. Using Log.Error(exception) is the correct way to
log an error as it will correctly write all the message details, also,
by passing the full Exception object we can improve the logging in this
one place and all call sites will automatically benefit from the improvements
Martin Molinero