121 Commits

Author SHA1 Message Date
Alexandre Catarino 6efeee07dc Implements Equity Fill Model (#4913)
* Implements Equity Fill Model

This commit sets the base to create a new equity fill model and the `EquityFillModel` is just a copy of `FillModel`.

* Adds Summary to FillModelPythonWrapper.GetPricesInternal

Adds summary to FillModelPythonWrapper.GetPricesInternal with remarks that it's a temporarily method to help the refactoring of fill models.
2020-11-04 15:51:41 -03:00
Colton Sellers 34676f0c46 Bug 4668 python register indicator (#4670)
* Convert to timespan if possible

* Fix to support both timespans and custom consolidators

* Adjust tests for new wrapper requirements

* Cleanup
2020-08-29 02:53:32 -03:00
Colton Sellers b8674731a5 Feature 2456 custom Python consolidator support (#4637)
* DataConsolidator Wrapper for Python Consolidators

* Regression Unit Test

* Refactor Regression test

* Bad test fix

* pre review

* self review

* Add RegisterIndicator for Python Consolidator

* Python base class for consolidators

* Modify regression algo to register indicator

* unit test - attach event

* Test fix

* Fix test python imports

* Add license header file and null check

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-08-25 17:26:55 -03:00
Alexandre Catarino 75a5f267f1 Adds Support to List of Symbol and String (Ticker)
Adds case in the mapper method to handled list type.
2020-07-04 00:11:19 +01:00
Jared f9cad3dc0c Merge pull request #4451 from QuantConnect/bug-pandasdata-misaligned-series-indexes-crash
Fixes alignment of Series objects created in PandasData causing crash
2020-05-26 14:02:17 -07:00
Gerardo Salazar 833c0540e0 Address review - short circuit the PandasData field/property type check 2020-05-25 09:46:58 -07:00
AlexCatarino 7bb7d6487f Fixes PandasData Wrapper Support for Query Method
query/eval methods needs to look for a scope variable at a higher level since the wrapper classes are children of pandas classes.
2020-05-23 02:19:52 +01:00
Gerardo Salazar bb2947d941 Fixes alignment of Series objects created in PandasData causing crash
* When a BaseData instance has Nullable fields, the number of data
    points per Series is inconsistent, and results in a Series with
    a length different from the other Series we produce, resulting
    in an error "ValueError: cannot handle a non-unique multi-index!"
    when we were constructing the final DataFrame.
2020-05-22 15:57:16 -07:00
AlexCatarino 2ab7eefd63 PandasData Speed Improvement
Use the original `pandas.Series` to create `Series` objects before `DataFrame` (wrapper version) creation.
It improves the speed because it avoids unnecessary and expensive index wrapping operations of the `DataFrame` creation.
2020-05-21 15:55:00 +01:00
AlexCatarino 34c337e766 Addresses Peer-Review 2020-05-18 16:28:19 +01:00
AlexCatarino e9eba27e6e Implements New PandasData.Remapper
In this implementation, we dynamically create new classes that wraps key functions and properties. The wrappers will map/convert any parameter that are convertible to the string representation of Symbol.ID before they are used by the original function/property.
2020-05-15 23:30:19 +01:00
Martin-Molinero 85cc7b13dc Revert "Adds pandas Methods (Part I)" 2020-05-06 20:16:04 -03:00
Martin-Molinero 31ed6e349b Revert "Adds pandas Methods (Part II)" 2020-05-06 20:15:41 -03:00
Martin-Molinero 3d9a24ff5c Revert "Adds pandas Methods (Part III)" 2020-05-06 20:15:17 -03:00
Martin-Molinero 07ef201f01 Revert "Adds pandas Methods (Part IV)" 2020-05-06 20:14:57 -03:00
Martin-Molinero 07bf7f48e4 Revert "Adds pandas Methods (Part V)" 2020-05-06 20:14:17 -03:00
AlexCatarino c1baf7da1a Addresses Peer-Review
`first_valid_index´ has been removed for now as it doesn't have a satifactory init test. It will be added in a future commit.
2020-05-06 14:28:59 +01:00
AlexCatarino 1538d0f134 Adds pandas Methods (Part V)
Methods: 'eq', 'eval', 'explode', 'ffill', 'fillna', 'filter', 'first', 'floordiv', 'ftypes', 'ge'
2020-05-06 02:05:20 +01:00
AlexCatarino b7b54ffa0b Adds pandas Methods (Part IV)
Methods: 'diff', 'div', 'divide', 'drop', 'drop_duplicates', 'droplevel', 'dropna', 'dtypes', 'duplicated'

`BackwardsCompatibilityDataFrame_binary_operator` replaces `BackwardsCompatibilityDataFrame_add` to handle all operations (more to be added in future commits)
2020-05-05 21:43:12 +01:00
AlexCatarino 02d90c4fb0 Adds pandas Methods (Part III)
Methods: 'columns', 'combine', 'combine_first', 'compound', 'copy', 'corr', 'corrwith', 'count', 'cov', 'cummax', 'cummin', 'cumprod', 'cumsum'
2020-05-05 17:15:30 +01:00
AlexCatarino a14c41bb97 Addresses Peer-Review
Removes `bool` because it doesn't return a pandas object with an index.
2020-05-05 15:24:34 +01:00
AlexCatarino 321603926a Adds pandas Methods (Part II)
Methods: 'asfreq', 'asof', 'assign', 'astype', 'at', 'at_time', 'axes', 'between_time', 'bfill', 'bool', 'clip', 'clip_lower', 'clip_upper'.
2020-05-05 12:40:14 +01:00
AlexCatarino ff32b8e15e Adds pandas Methods (Part I)
Methods: 'abs', 'add', 'add_prefix', 'add_suffix', 'agg', 'aggregate', 'align', 'all', 'any', 'append', 'apply', 'applymap'.
2020-05-05 01:32:49 +01:00
AlexCatarino a892173710 Fixes Not Wrapped pandas.Dataframe
`Remapper.__getitem__`  was not returns a `Remapper` object when the result was `pandas.DataFrame`. It is needed for a sequence of `.loc.` calls.
Refactors `Remapper._self_mapper` to handle tuples where the key can be found in both first and second position.

Update unit tests that should test `Symbol` object as key, but were using `str(Symbol)`.
2020-05-01 00:25:46 +01:00
Martin Molinero 8e71e48c07 PythonSlice
- AlgorithmPythonWrapper will directly call base OnFrameworkData()
  implementation skipping going through python and it's overhead
- Small performance improvement for adding Tick data points into a Ticks
  collection
- For python always wrap slice with PythonSlice, so that slice.Get()
  works even when no custom data is present, adding test.
2020-04-27 19:03:03 -03:00
AlexCatarino c1e18f82bb WIP 2020-04-01 16:15:04 +01:00
AlexCatarino 5e37a8b259 Implements IExtendedDictionary
`PythonSlice` implements `IExtendedDictionary`
2020-04-01 16:15:04 +01:00
Martin Molinero b28f9f501a Refactor and fix Slice bugs
- Fixing Python slice enumeration which would not happen when custom
data was present. Adding unit test
- Fix Slice constructor which would fail to create data collections due
to relying on non-deterministic 'slice._data'. Adding unit test
- Improving custom data retrieval for symbols which have more than 1
data type in a slice. Adding unit test
- First step refactoring slice internally, no behavior changed
2020-03-27 12:44:05 -03:00
Martin Molinero 6eb08334cd Revert GetBuyingPower removal
- Fix bug where OptioMarginModel would return 0 initial margin required
2020-02-24 14:06:16 -03:00
Jared 7f941edeec Merge pull request #4087 from QuantConnect/bug-4080-fix-report-creator-macosx-pythonpath
Fixes issue where report creator could not be ran in Mac OSX
2020-02-11 16:56:58 -08:00
Gerardo Salazar d3a34dcd70 Fixes issue where report creator could not be ran in Mac OSX
* Adds `PythonInitializer.SetPythonPathEnvironmentVariable(...?)` method
2020-02-11 15:01:47 -08:00
AlexCatarino 0beaab873f Implements Wrappers for __str__ and __repr__ in PandasData.Remapper 2020-02-11 16:54:51 +00:00
Martin Molinero 85f5b68972 Address reviews
- Revert `OptionMarginModel` behavior change
- Renames
- Refactor future intraday margin modelling
2020-02-04 18:41:15 -03:00
Martin Molinero 6eb45baa59 Refactor GetMaximumOrderQuantityForTargetBuyingPower
- Refactoring `GetMaximumOrderQuantityForTargetValue` into
`GetMaximumOrderQuantityForTargetBuyingPower` which algorithm uses
margin
- Reducing code duplication
- Adding unit tests
- Fixing futures margin model. Adding intra day logic.
2020-02-04 18:41:15 -03:00
Martin Molinero 936af7df7b Refactor BuyingPowerModel
- Remove unrequired `GetBuyingPower`
- Making `BuyingPowerModel.GetMaintenanceMarginRequirement` protected
instead of public
- Adding `GetMaximumOrderQuantityForDeltaBuyingPower` to replace
public `GetMaintenanceMarginRequirement` and improve API experience for
consumers like the `DefaultMarginCallModel`
- Adding new unit tests
2020-02-04 18:41:15 -03:00
Martin Molinero cd9bf4cc77 Making WorkerThread static
- Making WorkerThread instance static
- Adding PythonInitialize.AddPythonPaths(), useful to add paths to
python after initialization
2020-01-14 11:31:58 -03:00
Martin Molinero 19edc94f0f Improve pandas Symbol key support
- Add support for Symbol key access for pandas ix and iloc results
- Wrapp pdf merge, join, concat method results
- Wrapp pandas.concat method result with `Remapper`
- Adding unit tests
2019-11-20 13:48:42 -03:00
Martin Molinero 68ad2f51b6 Address reviews
- Replacing `BaseData.AdjustResolution` for `DefaultResolution` and
`SupportedResolutions`
- Making `Resolution` nullable for `Algorithm.AddData` methods
- The `ISubscriptionDataConfigService` will set the default resolution
if none was provided and assert it is supported
- Fix bug with `PythonData` `IsSparseData` and `RequiresMapping`
resolution
2019-11-04 20:42:30 -03:00
Martin Molinero 1d43dcd601 Add BaseData.AdjustResolution
- Adding `BaseData.AdjustResolution()` that should return a valid
resolution for the given data and security type.
This allows us to set a limitation which is useful to avoid invalid data
requests or unnecessary fill forward situations. The user will be
notified through a console message.
- Adding unit and regression test
- Updating example algorithms custom data resolution
- Some performance improvements. Wont change console color if
`SelectedOptimization` is defined
2019-11-04 20:38:26 -03:00
AlexCatarino b97d11d37a Addresses Reviews
- Covers another level on inheritance of Market Data by using `Type.IsAssignableFrom`
- Caches the list of `MethodInfo` for custom data types to avoid redefining that list.
2019-10-22 15:14:04 +01:00
AlexCatarino 0cca6869f7 Fixes Support for Custom Data
When custom data classes inherited from market data classes such as `TradeBar`, it created duplicate entries. Therefore, we need to exclude the common properties in the private field `PandasData._members`.
2019-10-22 15:14:04 +01:00
Martin Molinero 0b9862a7a7 Fix for PythonSlice 2019-10-02 23:22:52 -03:00
Martin Molinero 1cb4856557 Add support for Symbol as key for Pandas dataframe
- Improve user experience allowing users to use `Symbol` instance as key
for pandas data frame
2019-10-01 18:43:38 -03:00
Martin Molinero 6fabe30317 Add backwards compatibility index.levels[0]
- Add backwards compatibility shim for the `df.index.levels[0]` use
case. Reverting broken use cases and adding unit tests.
2019-09-24 10:34:35 -03:00
Martin Molinero b6b7720a1e Add more backwards compatible cases
- Moving mapper from C# to Python since some cases did not work when
implemented in C#
- Small changes to `PandasDataFrameHistoryAlgorithm` which runs till the
end with no errors
- Adding more backwards compatible unit tests
2019-09-24 10:34:35 -03:00
Martin Molinero 6a2f5b09f2 Improvements
- Add new dependency to documentation, docker files, python setup readme
- Adding backwards compatibility pandas DataFrame unit tests
- Adding missing cases for backwards compatility shim
2019-09-24 10:34:35 -03:00
Martin Molinero 8dbbbb618f Pandas and CustomData mapping fixes
- Add Pandas backwards compatibility shim
- Adding `MappingExtensions` which will remove data type from the
`Symbol.ID.Symbol` value to resolve the `MapFile`
- `SecurityIdentifier.TryParse()` will throw when given an invalid
`SecurityType`
2019-09-24 10:34:35 -03:00
Michael Handschuh d709d1c4e0 Update Common to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to go through the new
StringExtensions methods that use CultureInfo.InvariantCulture

See #3045
2019-09-12 14:13:24 -04:00
Martin Molinero 24527e0cc2 Address reviews - Renaming 2019-08-21 16:30:03 -03:00
Martin Molinero 73d780daae Moving UsesMapFiles to DataType
- Custom data types will know whether or not Lean should use map files
- Updating regression test with sample custom data using map files,
which can run locally
- Adding unit tests for the `SubscriptionDataReaderHistoryProvider`,
checking it mappes equities and options correctly
2019-08-20 17:33:41 -03:00