Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
* Support sub-indicator discovery using IIndicator and exclude PandasIgnore attribute
* WIP: PandasInclude
* Add PandasInclude annotation
* Resolve review comments
* Add a list of ignored properties
* Clean up [PandasIgnore] in IndicatorBase
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Normalize universe data frames
Universe and (generically BaseDataCollection) data frames are not normalize and unpacked into a data frame, instead of just creating data frames with the universe lists within it
* Fix unit tests and algorithms to expecte new universe dataframe format
* Fixes
* Add PandasConverter.DataFrameGenerator class
* Pandas data frame generator class fixes
* Add comments
* Housekeeping
* Add attributes to mark classes and properties for pandas processing
* Improve pandas properties expanding
Allow and handle duplicate names
* Use PandasData generalization for Lean common data types
* Add points time as column when converting base data collections to data frames
* Cleanup and minor changes
* Minor change
* Pandas data to get type members on demand
* Move Pandas helper classes to their own files
* Minor changes
* Add flatten argument to python history api
This allows users to decide whether they want fully expanded dataframes for universe and other collection data types. Else, master behavior is kept
* Adding missing changes to last commit
* Update Pythonnet version to 2.0.40
* Add flattent argument to algorithm's OptionChain api
* Minor changes
* Housekeeping
* Minor changes
* Bug fix skipping data collection data points
* Add comment
* Set correct exchange time to OptionUniverse instances
* Address peer review and cleanup
* Cleanup
* Minor changes
* Fix pandas converter to handle list of data with different symbols
* Properly convert list of data into dataframe
Take into consideration data for multiple symbols in the same list
* Cleanup
* Index dataframes by symbol object instead of SID string
* Add symbol equality operator to compare against object
* Exclude "ID" from option chain dataframe
* Minor fix
* Add greeks columns directly in option chain dataframe.
Also add pass-through properties for greek values in OptionUniverse
* Some cleanup
* Minor fix
* Add new QCAlgorithm.OptionChains() method
- Use OptionChains as output
- Add DataFrame to OptionChain and OptionChains
- Rename Greeks classes
- Add ISymbolProvider for classes that have a symbol (IBaseData, OptionContract)
* Unify QCAlgorithmOptionChain API
Also refactor OptionContract to handle: (1) Actual market data and option price model data, and (2) OptionUniverse data
* Pass symbol properties to OptionUniverse option chain from algorithm
* Format OptionContract for dataframe
* Minor fix
* Add multiple option chains api regression algorithms and other minor changes
* Address peer review
Add NullGreeks class: keep ModeledGreeks as internal as possible
* Minor fix and add PandasConverter unit tests
* Peer review: Non-thread-safe Lazy for Python
* Handle Greeks unwrapping by PandasData
* PandasData cleanup
* Add data and other minor changes
* Unit test fix
* Update Pythonnet to 2.0.39
* Cleanup
* PandasData handling children class members
Address peer review
* Fix: indexing symbol conversion in pandas mapper
* Fix pandas mapper to convert string keys to symbol only when necessary
* Cleanup
* Cleanup
* Add PandasColumn python class to handle proper indexing
This allows propery hash and equality between Symbols, C# strings and Python strings
* Minor fixes
* Symbol cache improvements
* Minor fix for cache miss
* Revert PandasMapper reserved names and improvements
* Minor fix
* Revert reserved names
* Minor fix for Symbol equality operators
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Add indicator helper methods for base Algorithm
- Add indicator helper methods for base algorithm. Adding new unit tests
* Expand indicators helper methods
- Expand indicators helper methods, adding support for C# and custom
indicators too. Adding unit tests
- Expand indicators helper methods to support multiple symbols as input.
Adding unit tests
- Improve conversion of symbol enumerable from python to C# adding unit
tests
* Address reviews
- Keep old QB.Indicator methods for backwards compatibility
- Rename new API to IndicatorHistory, matching
FutureHistory/OptionHistory
- Add new regression algorithms
- Minor improvement to DynamicData so it supports snake name access
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add support for auxiliary data history request
- Add support for split/dividends/margin interest history requests.
Adding regression algorithms
* Expand auxiliary history regression tests
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Pandas frame include all ticks
- Pandas data framde history response will include all ticks. Updating
existing and adding new tests
* Python pandas converter performance improvement
* Memory Related Performance improvements
- Make sure we cleanup & dipose of python related objects during pandas
data generation.
- Disable memoizing enumerable use while creating pandas data frames,
since we do not require it
- Reduce unrequired object creations
- Replace concurrentCollections for ordinary locks
* Decimal parsing typo fix
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Get history of a given bar type in python
* Separate GetDataFrame in sub methods to avoid dynamic type resolution
* Housekeeping and unit tests update
* Use all requests in python type method history method
* Address changes request
* Reverted some requested changes
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Overload PandasConverter.GetIndicatorDataFrame to accept a Python dict
* Shared implementation code for PandasConverter.GetIndicatorDataFrame overloads
* Added documentation for private shared methods used by PandasConverter.GetIndicatorDataFrame
* Added unit tests for PandasConverter.GetIndicatorDataFrame
* Added unit tests for PandasConverter.GetIndicatorDataFrame Dictionary overload
* Address change requests
* Address change requests
In this implementation, we dynamically create new classes that wraps key functions and properties. The wrappers will map/convert any parameter that are convertible to the string representation of Symbol.ID before they are used by the original function/property.
- Add Pandas backwards compatibility shim
- Adding `MappingExtensions` which will remove data type from the
`Symbol.ID.Symbol` value to resolve the `MapFile`
- `SecurityIdentifier.TryParse()` will throw when given an invalid
`SecurityType`
From [pandas-docs](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.concat.html):
> The current default of sorting is deprecated and will change to not-sorting in a future version of pandas.
>
> Explicitly pass sort=True to silence the warning and sort. > Explicitly pass sort=False to silence the warning and not sort.
Since Lean/QuantConnect data from a symbol can be found in Slice.Ticks, Slice.Bars and Slice.QuoteBars, information for all of this members must be used in order present all information in the pandas.DataFrame.
- Moves PandasData class to its own file
- Refactors PandasData class to deal with list of tick
- Other minor changes requested by peer-review
- Adds unit test for PandasConverter's tick handling
In the previous implementation, each one of bars.Select statement causes an extra enumeration of the bars enumerable. To make matters worse, this is all invoked 4*N times, where N is the number of data points.
Another bottleneck was related to the way we concatenate pandas.DataFrames: we would add a new data frame to an existing data frame individually. Now, we collect all data frames and concatenate them in one operation.
With this new implementation, we have accomplshed a reduction of 90% of memory usage in a history request of 1000 symbols.
With simple commands, we can use Lean indicators in QuantBook. It fetchs the historical data from the symbol, calculates the indicator and saves the output in a pandas.DataFrame.
History requests should not return a dictionary with a dataframe, but a multi-index dataframe.
It is more common to work with multi-index dataframes rather than multi-column.
JosueNina