1084 Commits

Author SHA1 Message Date
Martin-Molinero e37f8ae878 Fix for adding continuous future and future contract at the same time (#6091)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Fix for adding continuous future and manually adding contract at the
  same time. Continuous future was using the user defined universe
  symbol and caused clashes. Adding regression test reproducing the issue.
2021-12-02 18:13:52 -03:00
Martin-Molinero d1a35e6281 Continuous Futures Refactor. Live Mappings (#6076)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Continuous Futures Refactor. Live Mappings

- Adding support for live mappings. LiveTradingDataFeed will handle any
  symbol mapping at the configuration layer and resubscribe through the
  IDQH
- Refactoring continuous futures adding ContinuousFutureUniverse that
  will select the currently mapped security

* Minor fixes

- Remove addition of configurations in UniverseSelection step, leave
  resposability for universe.
- LiveTradingDF future unit test will only assert slice data for non
  internal feeds.
- ContinuousContractUniverse will respect internal option interest
  subscription

* Address review
2021-11-30 21:38:50 -03:00
Ricardo Andrés Marino Rojas df63b6f5d6 Renaming requested changes (#6072)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-11-24 09:58:37 -03:00
Colton Sellers 1358bd8115 Future and FutureOption low res support (#6069)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fixes for FutureOptions support in LeanData

* Add CreateCanonicalOption() utility function for Symbol.cs

* Add aggregated Futures/FuturesOptions data to Lean

* Add FutureOptions regressions for daily/hourly data

* Allow Futures to be added with low resolution

* Add Future regressions using hour/daily data

* Nit - Python Class names

* Add reviews

* Add alias into CreateCanonicalOption
2021-11-24 09:53:21 -03:00
Martin-Molinero a8c81cad2a Fixes for continuous futures history requests (#6067)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fixes for continuous futures history requests

- Fixes for continuous future history requests and warmup. Adding
  regression test

* Address reviews
2021-11-19 20:52:15 -03:00
Jovad Uribe 325e788728 Super Trend Indicator (#6013)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Super Trend Indicator #4653

* Updated test data

Previous test data was wrong

* Reduced if statement

* Updated tests

Replacing spy test data with dwac test data from trading view.

* Minor comment update

* Minor tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-11-17 12:19:26 -03:00
Ricardo Andrés Marino Rojas 5415fe6bc0 Beta indicator (#6042)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* First BetaIndicator prototype
- In construction

* Fist BetaIndicator version and unit tests

* More unit tests and regression test

* Nit change

* Requested changes

* Nit changes

* Requested changes

* Adjust beta formula slightly and nit changes

* Nit change
2021-11-15 15:57:24 -03:00
Martin-Molinero b8b0d18993 Continuous futures (#6034)
* Continuous Future Contracts

* Mapping approach

* Tweaks WIP

* Live mapping

* Live mapping

- Add support for live mapping, refreshing mapfiles
- Fix future expiration functions
- Adding unit tests

* Update moq test package

* Continuous futures price scaling

* Refactor price factors scaling

* Factor file related renames

* Address reviews
2021-11-15 14:44:30 -03:00
Ricardo Andrés Marino Rojas fc6ddc2120 Feature 5988 WarmUpIndicator() method for indicators written in Python (#6027)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Implement IIndicatorWarmUpPeriodProvider
- Implement IIndicatorWarmUpPeriodProvider in PythonIndicator.cs
- Make a unit test to check whether the WarmUpPeriod is working as expected
- Make a regression test to check the new feature at a system level

* Nit change

* Change Period parameter for WarmUpPeriod parameter
- Change regression test to check if the new parameter keep backwards compatibility with indicators that do not set WarmUpPeriod

* Documentation change

* Fix tests bugs
- In CommonIndicatorTests.cs before finish the test it checks the period.value with the number of samples but for default the period.value was set to -1

* Change names

* Change WarmUp and RegisterIndicator methods
- Lean WarmUp indicator skip custom python indicators that don't define WarmUpPeriod parameter

* Call WarmUpIndicator manually
- Add a new "bridge" method called WarmUpIndicator in QCAlgorithm.Python.cs to set up everything to call WarmUpIndicator in QCAlgorithm.Indicators.cs
- Change the regression algorithm to warm up the indicators manually

* Remove unnecessary code and add more tests

* Nit change

* Revert "Nit change"

This reverts commit da411f59c9e4295d75a11c6c581f615a938dd2c6.

* Fix bugs

* Try fix bugs

* Add C# regression test
- More nit changes
- Fix bugs

* Requested changes

* Remove unnecessary code

* Requested changes

* Nit changes
- Add new Python class to check a custom indicator, which doesn't inherits from PythonIndicator, warms up properly

* Reduce redundant code

* Fix bug and add more unit and regression tests

* - Add more unit tests

* Nit change

* Test cleanup

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-10-28 20:29:22 -03:00
Martin-Molinero 8c71dbc8ba Revert "Feature 5988 IIndicatorWarmUpPeriodProvider for custom PythonIndicator (#5992)" (#6022)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
This reverts commit 995f598999.
2021-10-26 22:07:22 -03:00
Ricardo Andrés Marino Rojas 995f598999 Feature 5988 IIndicatorWarmUpPeriodProvider for custom PythonIndicator (#5992)
* Implement IIndicatorWarmUpPeriodProvider
- Implement IIndicatorWarmUpPeriodProvider in PythonIndicator.cs
- Make a unit test to check whether the WarmUpPeriod is working as expected
- Make a regression test to check the new feature at a system level

* Nit change

* Change Period parameter for WarmUpPeriod parameter
- Change regression test to check if the new parameter keep backwards compatibility with indicators that do not set WarmUpPeriod

* Documentation change

* Fix tests bugs
- In CommonIndicatorTests.cs before finish the test it checks the period.value with the number of samples but for default the period.value was set to -1

* Change names

* Change WarmUp and RegisterIndicator methods
- Lean WarmUp indicator skip custom python indicators that don't define WarmUpPeriod parameter

* Call WarmUpIndicator manually
- Add a new "bridge" method called WarmUpIndicator in QCAlgorithm.Python.cs to set up everything to call WarmUpIndicator in QCAlgorithm.Indicators.cs
- Change the regression algorithm to warm up the indicators manually

* Remove unnecessary code and add more tests

* Nit change

* Revert "Nit change"

This reverts commit da411f59c9e4295d75a11c6c581f615a938dd2c6.

* Fix bugs

* Try fix bugs

* Add C# regression test
- More nit changes
- Fix bugs

* Requested changes

* Remove unnecessary code

* Requested changes

* Nit changes
- Add new Python class to check a custom indicator, which doesn't inherits from PythonIndicator, warms up properly

* Reduce redundant code
2021-10-25 18:08:57 -03:00
Martin-Molinero 58ccdee1b1 PythonNet version bump 2.0.10 (#6010)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-10-22 20:27:38 -03:00
Colton Sellers 6bd2859f64 Update to PythonNet 2.0.9 (#6004)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-10-21 20:06:38 -03:00
Colton Sellers cff3640f15 Update to PythonNet 2.0.8 (#5978)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-10-13 20:19:06 -03:00
Martin-Molinero 1d6fed8843 Update python to 2.0.7 (#5968)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-10-08 12:55:14 -03:00
Colton Sellers d2d99b1f10 Algorithm Sampling and Statistics Fixes (#5936)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement scheduled event sampling solution

* Use UTC time, only update daily portfolio value once a day

* For daily resolutions sample chart always

* Cleanup

* Drop resample daily all together

* Force final sample

* Regression updates

* FIx LiveResultHandler to update portfolio and benchmark values outside of sampling event

* Name the daily sampling event

* Address review pt 1

* Drop force and use reference wrapper

* Adjust tests

* Fix warning for Benchmark Timezone Misalignment and also add test

* Fix for daily resolution orders and test adjustments

* Also warn on universe settings with daily resolution

* Update missed regression

* Fix reference wrapper use

* Update regression after rebase

* Add values back in for Daylight Algo

* Have statistics builder skip day 1 performance

* Regression adjustments

* Test adjustments

* Update regression unit test

* Adjust some regressions starts to show performance values

* Add hourly algorithm for beta comparison

* Address missing Python regression changes

* Remove null comment
2021-10-05 19:31:25 -03:00
Martin-Molinero 9579c4263b Fix history requests so they ignore internal subscriptions (#5945)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Fix for history requests so that they ignore internal subscriptions
  accordengly. Adding regression test reproducing issue
2021-09-27 20:31:05 -03:00
Gerardo Salazar 024d416ddd Adds support for getting a Lean symbol based on FIGI, CUSIP, ISIN, SEDOL (#5922)
* Adds support for getting a Lean symbol based on FIGI, CUSIP, ISIN, SEDOL

* Address review: bug fixes for mapping, improve docs, cleanup & refactor

  * Make LocalZipMapFileProvider and LocalZipFactorFileProvider only
    initialize if they haven't been initialized yet.

* Address review: makes ISIN, SEDOL, CUSIP case-insensitive

  * Cleans up map file provider in SecurityDefinitionSymbolResolver
  * Modifies some test cases to test for case-insensitivity

* Add null check for SecurityDefinitions

* Fix unit tests

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-09-10 19:16:18 -07:00
quantify-cflynn 5d4f428906 Add ValueAreaVolumePercentage Parameter (#5908)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add ValueAreaVolumePercentage Parameter

Remove hard coding of parameter and set it to optional parameter for the user.

* Update TimeProfile.cs to include valueAreaVolumePercentage parameter

* Update TimeProfile definition in QCAlgorithm.Indicators.cs
2021-09-07 12:17:23 -03:00
Ricardo Andrés Marino Rojas da60daa60d Market profile indicator (#5891)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Create Market Profile indicator.

* - Requested changes in MarketProfile indicator made it.
- TimeProfile and VolumeProfile created and tested.
- tp_datatest.csv and vp_datatest.csv extracted from https://github.com/bfolkens/py-market-profile

* Final potential Market Profile Indicator.
All unit test made it and the indicator is passing all of them
Test cases made it with python library from GH issue

* Code styling request changes

* Minor suggestions

* Minor renaming tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-09-03 20:18:44 -03:00
Martin-Molinero b251cffc2f Alpha Streams improvements (#5887)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Few improvements

- Alpha Streams algorithm will add target securities right away
- If algorithm is warming up PortfolioTargetCollection will emit not
  insight
- Fix local disk factor file provider to use local cache instead of data
  folder
- EWAS PCM will not emit targets for securities which have not been
  added by the algorithm yet, to avoid runtime exception

* Fix GetLastKnownPrice default order adding more unit tests
2021-09-01 13:12:26 -03:00
Martin-Molinero 933e5ce6ca Alpha Streams improvements (#5876)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Alpha Streams improvements

- Warmup added securities so we can trigger an order right away
- Fix bug where AddSecurity returned a security which was not the one
  being used
- Fix bug where EWAS PCM would remove from a dictionary while iterating
  over it

* Fix unit tests

* Add QCAlgorithm.GetLastKnownPrices API

- Add new GetLastKnownPrices that will return the last known data point
  for all subscribed data types.
- Fix for MHDB GetDataTimeZone which was using an invalid entry key
  format for custom data.
- Fix for using GetLastKnownPrice/s to seed a security during creation,
  when it's not added in the Algorithm.Securities collection

* Adding more unit tests for GetLastKnownPrice

* Address reviews

- Refactor GetLastKnownPrice/s to perform a single history request for
  all data types
2021-08-27 13:50:45 -03:00
Martin-Molinero 9acf68ad19 Bump pythonNet to 2.0.6 (#5881)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-08-26 18:01:05 -03:00
Martin-Molinero 5ebf451fb3 Fix for python SetHoldings precision loss (#5879)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- After https://github.com/QuantConnect/Lean/pull/5872 trading API
  changes numpy float64 was not converted correctly by pythonNet and
  used an int. Reverting API changes and adding regression test. This
  should be fixed at pythonNet layer
2021-08-25 15:25:39 -03:00
Gerardo Salazar a4d49c05ca Adds ETF(...) to UniverseDefinitions (#5873)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adds ETF(...) to UniverseDefinitions

  * Adds ETF constituents universe framework regression algorithm
    for C#/Python

* Address review: adds test cases for ticker/Symbol ETF universe additions

  * Fixes bug where null Market would result in null dereference exception

* Address review: add missing Index tests

* Address review: don't hardcode market when creating constituent universe

  * Uses Brokerage Model's default markets collection to determine
    the market for the given security type

* Address review: restore QC500 and DollarVolume.Top(...)

  * Restores algorithms related to both helper universe
    definition methods

* Address review: remove copy to output directory for python algos

* Add example algorithms for ETF constituent universes using custom RSI alpha model

* Address review: adjust algorithm to use cache + algo RSI & clean up code

* Address review: make ETF Constituent RSI Alpha Model algo a regression test

* Address review: increase trade count and remove single trade logic
2021-08-25 11:22:31 -03:00
Martin-Molinero aaba566954 Alpha Streams Improvements. Python Imports (#5874)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Order handling improvements

- Execution model will only trigger market order if they are above the
  minimum order margin portfolio percetage value
- SecurityCache.Reset is complete

* Python Import fixes

- Add regression test for ImmediateExecutionModel minimum order margin
  check
2021-08-24 11:15:48 -03:00
Martin-Molinero bf28a1d313 Add basic template Atreyu algorithm (#5872)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add basic template Atreyu algorithm

- Add C# and Python basic template atreyu algorithm. Show casing how to
  specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use

* Lean Exchange improvements

- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value

* Regression tests fixes
2021-08-23 18:24:44 -03:00
Martin-Molinero 3410832ccf Alpha holdings state (#5770)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state

- Alpha result packet will optionally provide the algorithms portfolio
  state

* Rename

* Convert AlphaStreamsPortfolio to data source

* Improvements on AlphaStreams algorithm

* Fix regression tests

* Add unit tests for EW AS PCM and fixing bugs

* Protobuf AlphaStreamsPortfoliot staState

- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM

* Add alpha license to Organization response

* Improvements EW AS PCM respects free portfolio value

* Fixes

- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType

* Fixed and adding more regression tests

- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
  types
- Add support and regression test for algorithm alpha consumer with existing holdings

* Add AlphaStreamsOrderEvent data type
2021-08-19 13:56:18 -03:00
Martin-Molinero 0da9d626dc Remove security from all universes holdings it (#5798)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Calling RemoveSecurity() will remove the security from all the
  universes holdings it, there was a race condition here where it would
  just take the first universe and remove it from it.
  Adding regression test adding and removing an option contract and it's underlying
2021-07-28 18:03:09 -03:00
Jovad Uribe 9888081cc0 Relative Daily Volume Indicator (#5747)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Relative Volume Indicator

* Requested changes and renamed indicator

* Made changes, included Resolution

* Added requested changes

* Update

* Update RelativeDailyVolumeTests.cs

* Update RelativeDailyVolume.cs

* Fixed Test Case File

* Update RelativeDailyVolume.cs

* Fixed IsReady flag
2021-07-28 18:02:34 -03:00
Marco Grassi e823dfdfb7 fix a typo, and remove all unnecessary semicolumns in Python (#5795)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-27 16:05:13 -03:00
Martin-Molinero 4469d18eb6 Set timezone for DateRules (#5796)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- If algorithm timeZone is changed Update DateRules timezone. Adding
  unit test
2021-07-27 15:34:11 -03:00
Colton Sellers 8d11bbf0ba Update to PythonNet 2.0.5 (#5764)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-14 11:01:57 -03:00
Jovad Uribe e5f62fa4dd Feature Augen Price Spike Indicator (#5713)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Added Augen Price Spike Files

* AugenPriceSpike Update

Adjusted indicator calculations and added test data from Trading View

* Requested Changes

* Minor changes
2021-07-09 20:07:15 -03:00
Colton Sellers f47775338c Expand Lean Default Charts and Reduce Clutter (#5680)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update charts that are not empty and not default values

* Testing

* Filter final results

* Explicitly filter alpha charts if not needed

* Use filtered charts for Live results handler

* Refactor solution to manage alpha charts internally by creator

* nit comment fixes

* Implement drawdown as default chart

* Implement Capacity Estimate for backtest default chart

* Add default values to QC.Plotting

* Adjust drawdown calculation

* Only sample "Capacity" once a day

* Implement Volume chart

* Nit fixes

* Volume Chart adjustments and fixes

* Rename to "Assets Sales Volume"

* Implement Exposure Sampling

* Store algorithm currency symbol, round capacity to nearest 1k

* Add new plots to default charts

* nit change name

* Address reviews

* Reduce duplication and clean up exposure sampling

* Address reviews

* Improve sample exposure

* Address review

* Only enumerate holdings once

* nit - comments

* Post rebase fix

* Don't need to round anymore
2021-06-22 11:19:48 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
2021-06-15 19:06:06 -03:00
Martin-Molinero b7c18cbfe7 Update pythonNet to net5 (#5653)
Build & Test Lean / build (push) Has been cancelled
- Csproj clean up legacy build print logs
- Update pythonNet to net5
2021-06-11 13:51:09 -03:00
Martin-Molinero a231a7d47a Update python net to 2.0.3 (#5640)
Build & Test Lean / build (push) Has been cancelled
* Update pythonNet version to 2.0.3

* Fix outputPath causing issues with nuget
2021-06-07 21:37:26 -03:00
Colton Sellers 41f9d9da94 Remove SecurityInitializer from Universe Selection (#5418)
Build & Test Lean / build (push) Has been cancelled
* Attempt to remove SecurityInitializer from UniverseSelection stack entirely

* Fix broken Python Universe Selection model

* Remove mistake change (meant for another branch)
2021-05-28 14:53:13 -03:00
Martin-Molinero 0335b49edf Update pythonNet to 2.0.2 (#5611) 2021-05-28 12:10:55 -03:00
Martin-Molinero 9e96aeb440 Language update. Csproj clean up (#5590)
Build & Test Lean / build (push) Has been cancelled
- Update CSharp language version to latest 9
- Remove old csproj build configurations
2021-05-24 11:10:03 -03:00
IlshatGaripov c384fe0e2a Implements PivotPointsHighLow indicator + tests (#5352)
Build & Test Lean / build (push) Has been cancelled
* Implements PivotPointsHighLow indicator + tests

# Conflicts:
#	Tests/QuantConnect.Tests.csproj

* Some fixes

* Refactoring + Adds NewPivotPointFormed event

* Fixing tests

* Some fixes for the Get methods to not throw when points array is empty

* To address review + more fixes

* Fixing xml comments typos

* Implements separate rolling windows to calculate highs and low

* Makes the number of last stored indicator values as an input parameter

* Overrides Reset()

* Adds a helper method

* Change numerical return to Enum types

* Changes IsReady condition :

the indicator is ready and starts calculating the pivot point when any of the rollings is ready

* Address reviews

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-24 11:09:07 -03:00
IlshatGaripov dcb578cdfb Do not restrict unrequested securities by 1.0m leverage (#5375)
Build & Test Lean / build (push) Has been cancelled
* Do not restrict unrequested securities by 1.0m leverage, let security initializer do its job

* Use UniverseSettings for leverage, fillforward, and extendedmarkethours

* Update unrequested security message

* Use UniverseSettings ExtendedMarketHours for underlying on OptionContracts

* Add unit test to prove setting persistence

* Cleanup unit test

* Adjust crypto unit test assert

Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-19 12:59:42 -03:00
Martin-Molinero 9887a05d77 Fix FutureOptionMarginModel Margin Requirements (#5570)
Build & Test Lean / build (push) Has been cancelled
* Fix FutureOptionMarginModel Margin Requirements

- Fixes FuturesOptionsMarginModel margin requirements being zero due to
  underlying being null on model creation. Adding unit test

* Address reviews

- SecurityService will set underlying when creating a new security when
  provided
2021-05-18 14:04:30 -03:00
Colton Sellers d76bfaade4 Consolidate SetupHandlers (#5529)
Build & Test Lean / build (push) Has been cancelled
* Reconcile and consolidate SetupHandlers

* Centralize GetConfiguredDataFeeds()

* nit - extra space

* Check for null, also allow null to be passed back if no config value

* Fix breaking test

* Cleanup

* fix return var

* remove unneeded if/else

* Minor changes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-10 22:12:16 -03:00
Colton Sellers ed077da501 DataNormalizationMode Applies to AddSecurity (#5528)
Build & Test Lean / build (push) Has been cancelled
* Use UniverseSettings.DataNormalizationMode for securities added in Algorithm

* Stop SubscriptionUtils from forcing Adjusted mode

* Return behavior to original and add comments

* nit typo

* Add regression

* Add unit test that verifies DataNormalizationMode can be altered manually by security

* Cleanup and add Py version of regression
2021-05-10 20:58:50 -03:00
Martin-Molinero f3c386663b Feature .net 5 (#5505)
* Update projects to use .NET 5.0, the successor to .NET Core

* Fix ambiguous errors. Add IBAutomator net5

* Remove FXCM

* Upgrade IBAutomater to v1.0.51

ignored, and an empty message aborts the commit.

* Fix rebase

- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy

* Remove unrequired references

* Fixes

- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests

* Don't call ReadKey when input is redirected

* Fix ConsoleLeanOptimizer

* Research fixes

* Update comment

* Add vsdbg to Dockerfile

* Fixes

- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
   - Option assignment will be deterministic in the order
   - 'Rolling Averaged Population' is calculated using doubles, updating
     expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock

* Replace ICSharp with .NET Interactive

* Fixes after rebase

* CSharp research fixes

- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
2021-05-06 17:23:51 -03:00
Colton Sellers 7ddee34e86 Configurable MarketOnCloseOrder Buffer (#5516)
Build & Test Lean / build (push) Has been cancelled
* Create a configurable SubmissionTimeBuffer for MarketClose orders.

* Nit error spacing

* Add Py and C# regression
2021-05-03 18:08:27 -03:00
Martin-Molinero 83f9499b4a Option Margin Strategies (#5511)
* Refactor HasSufficientBuyingPowerForOrder implementations

Adds Sufficient and Insufficient helper methods to HashSufficientbuyingPowerForOrderParameters
enabling syntax like:

return paraeeters.Sufficient()
returnparameters.Insufficient(reason)

The next change will add the initial margin required which will simply require
updating both of these helper methods to accept the value.

* IBuyingPowerModel: Add margin functions Maintenance/Initial/ForOrder

These were originally hidden in an effort to only expose what's necessary
for the engine to perform its work. Additionally, we encapsulated all of
the method arguments into parameters classes to prevent having to break
anyone in the future. Not including these foundational methods turns out to
be an oversight. These methods are not required by the engine, but rather by
other models. Another possible solution here is to add an additional abstraction
and include these methods on this new abstraction. BuyingPowerModel would then
explicitly implement these methods and models that depend on them would require
two code paths, one for when the buying power model implements this interface
and another for when it doesn't.

Tests were additionally updated to remove test model implementations created for
the sole purpose of exposing these private methods.

* Add ConstantBuyingPowerModel

Provides an implementation of IBuyingPowerModel that returns the same
constant value

* Update BuyingPowerModelPythonWrapper to use reflection for method names

Having a bunch of hard-coded strings is a sure fire way for someone to
overlook when changing methods. This change ensures that noone needs to
remember that this code exists :)

Cleans up the syntax around verifying a python object implements a particular
C# interface via the ValidateImplementationOf<T> method by having it return a
value since the only use cases are in constructors when setting the models.

I was initially going to update ALL python wrappers to validate the passed
in models, but such a change could break many things that are 'working' right
now. Such an effort should be saved for its own dedicated PR.

* Add Parameters/Result types for new buying power model methods

* Support computing maintenance margin for arbitrary quantities

The existing GetMaintenanceMargin function assumes that we're only interested
in the maintenance margin for the entirety of the provided security's holdings.
This makes it impossible to perform what-if analysis or to even ask how much
maintenance margin is devoted to a particular subset of the security's holdings.
This change adds the quantity to the MaintenanceMarginParameters class. Futures
and Options models also depend on holdings cost and holdings value, so they have
also been added to the parameters type. Finally, static factory methods were
added to improve discernment of intent: ForCurrentHoldings provides the existing
behavior and then ForQuantityAtCurrentPrice to support what-if scenarios where
we're looking for the change in maintenance margin if we were to execute an order
for the securiy at the current time step. Obviously a constructor is provided to
set all of the values explicitly, using any price metric the caller desires.

* Address review

- Fix BPM xml documentation
- Fix python unit tests and PythonWrapper validate method

* Add SecurityHolding.QuantityChanged event

Adding event handlers will allow us to orchestrate complex
events from distant parts of the codebase through wiring
them up. If we continue down this path, it will move us away
from the current, very 'mechanical' data flows expressed in
LEAN and towards a more modern, event processing based system.
This is but a baby step in that direction and the initial use
case is using this QuantityChanged event to trigger resolution
of the algoritm's positions groups. This is part of an effort
to improve the fidelity of options margin modeling where we'll
model an OptionStrategy as an IPositionGroup. This will allow
us to compute the margin requirements of an OptionStrategy as
a unit instead of computing margin of each security individually
in isolation.

See #4065

* PortfolioManager: Group fields and remove unused field

This codebase generally places fields as the first members, but
this class had some fields at the top, then some properties, and
then some more fields. This change brings all the fields together
at the top of the file and also removes pointless comments placed
directly above some of the fields. Additionally, an unused field
was removed.

* Remove unused _currencyConverter from Security

Looks like at some point the only code using this member variable was removed
and the necessary clean up was overlooked.

* Add Parse.Enum functions

* Support disabling regression algorithms by language via config.json

Adds 'regression-test-languages' to config.json and filters regerssion algorithms to
run based on this value. When cycling on a particular feature, it's nice to be able
to run the entire regression set while ignoring the python algorithms. Once the C#
algorithms are all passing, one can then go back and run C# and Python in a final run,
since 99% of feature work doesn't impact python specifically.

* Implement IComparable in SecurityIdentitfier

This can be used to deterministically sort securities and symbols

* Add .editorconfig to enforce common formatting for json/sh files

* Fix typo in IBuyingPowerModel.GetBuyingPower xml docs

* Add ListEquals/GetListHashCode and OrderDirection.Closes(PositionSide)

ListEquals and GetListHashCode are designed to be used together as they
complement each other according to C#'s requirements for Equals and
GetHashCode functions.

PositionSide.ToOrderDirection() extension simply converts a PositionSide
to its logical equivalent OrderDirection. Long->Buy, Short->Sell, None->Hold

OrderDirection.Closes(PositionSide) determines if a particular OrderDirection
would have the effect of reducing a position's absolute size. This function
greatly improves the readability of buying power functions that must provide
adjustments when an order/contemplated trade reduces/closes an existing position.
OrderDirection.Buy.Closes(PositionSide.Short)
OrderDirection.Sell.Closes(PositionSide.Long)
All other combinations return false

Adds ToArray/ToImmutableArray convenience functions that combine a call
to Select followed by To(Immutable)Array all in one function call.

* Add decimal.DiscretelyRoundBy extension method

Supports rounding a decimal value by an arbitrarily chosen maximum precision,
or 'quanta'

* Update FutureMarginBuyingPowerModelTests to respect the security's lot size

* Add core position group classes and abstractions

* Add initial/maintenance margin support, buying power model consistency tests

* Add SufficientBuyingPower and GetReservedBuyingPower to position group model

Includes update to BrokerageTransactionHandler to use position group BPM for
sufficient buying power checks.

* Resolve position groups on each fill

We need to update the state of our position groups on each fill so that
we can properly handle multiple orders within the same time step. We
also limit the number of positions sent into the resolver by removing
securities without any holdings.

* fixup! Add SufficientBuyingPower and GetReservedBuyingPower to position group model

* Add GetMaximumLotsFor{Target|Delta}BuyingPower

Instead of computing order quantity, these functions compute the
maximum number of position group lots, which is the position group
quantity, and is guaranteed to be a whole number, for the provided
target/delta buying power parameters.

The SecurityPositionGroupBuyingPowerModel delegates to the security's
IBuyingPowerModel by applying a scaling factor equal to the security's
lot size.

This change also updates references to IBuyingPowerModel.GetMaximum...
to use the new position group model methods.

* Convert remaining IBuyingPowerModel call sites to position groups

* Rename PositionManasger.CreateDefaultGroup -> GetOrCreateDefaultGroup

Better describes its behavior

* Add Position Groups readme.md

* Add Option Strategy BuyingPowerModel

- Adding CompositePrositionGroupResolver and
  OptionStrategyPositionGroupResolver
- Adding OptionStrategyPositionGroupBuyingPowerModel handling option
  strategies based on IBs margin table. Adding regression algorithms
- Few changes so that option strategies executed by multiple orders are
  detected
- Adjust OptionStrategyDefinitionMatch to include equity legs in the
  matching result
- Minor tweaks fixing previous rebase
- Minor fixes for existing option strategies definitions, adding new
  missing strategies.
- Fixing minor bugs in option strategy matcher. Adding more unit tests

* Address self reviews

- Fixing bug in 'PositionGroupCollection'
- Few minor simplificaitons
- Adding BasicTemplateOptionEquityStrategyAlgorithm

* Address reviews

- Improve regression algorithms margin remaining and used assert logic to be exact. Taking into account spread and fees

Co-authored-by: Michael Handschuh <mhandschuh@gmail.com>
2021-04-30 18:45:27 -03:00
Colton Sellers fd4e1dc0a0 Address Security Normalization Changed to Raw Logging (#5509)
* Log security mode change once per universe addition process

* Limit max size of the warning queue to 10

* Only allow to emit once per backtest

* Set limit as var, if at limit suggest more

* Always suggest more warnings may exist because we opted to log only once
2021-04-29 10:54:46 -03:00