363 Commits

Author SHA1 Message Date
Jhonathan Abreu 5758b65099 Added configuration parameters to Python QCAlgorithm.History() method that takes custom data source type (#6448)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add new Python QCAlgorithm.History() method with all parameters and type

* Add regression algorithms

* Using all parameters in History()

* Use private methods to reuse History() code

* Use private methods to reuse History() code

* Add unit tests for QCAlgorithm.Python.History()

* Add unit tests for QCAlgorithm.Python.History()

* Add unit tests for QCAlgorithm.Python.History()

* Add unit tests for QCAlgorithm.Python.History()

* Add unit tests for QCAlgorithm.Python.History()

* Add unit tests for QCAlgorithm.Python.History()

* Asserting history count
2022-07-05 13:14:08 -03:00
Jhonathan Abreu e9409bf207 Add contractDepthOffset parameter to QCAlgorithm.History() (#6438)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add contract depth offset parameter to big History() methods

- Added unit tests
- Added little more ED daily data

* Add C# regression algorithm

* Add Python regression algorithm

* Update regression algorithms stats

* Address changes request
2022-06-27 10:04:33 -03:00
Jhonathan Abreu 9128ce1260 Add data normalization mode parameter to QCAlgorithm.History() (#6435)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add data normalization mode parameter to big History() methods

* Add C# regression algorithm

* Add Python regression algorithm
2022-06-23 17:02:50 -03:00
Jhonathan Abreu 505ef17565 Add data mapping mode parameter to QCAlgorithm.History() (#6415)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add dataMappingMode parameter to QCAlgorithm.History()

* Add C# regression algorithm

* Add Python regression algorithm

* Cleanup

* Add data mapping mode parameter only to big History() methods

* Fix regression algorithms and add required data

* Fix unit test

* Update regression algorithms stats
2022-06-22 19:25:30 -03:00
Martin-Molinero a42a53671f Add missing Py.Gil in base PCM (#6414)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add missing python Py.Gill in base PortfolioConstructionModel. Adding
  unit test reproducing issue, seg fault
2022-06-20 19:44:31 -03:00
Jhonathan Abreu e736b28568 Add data normalization mode parameter to AddEquity method (#6407)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add DataNormalizationMode parameter to QCAlgorithm.AddEquity method

* Add regression algorithm

* Add Python regression algorithm

* Style changes

* Fix test error
2022-06-17 18:27:06 -03:00
Martin-Molinero a8e7a8b27e Reconcile the Market Opening for Futures and Scheduled Events (#6397)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Changes taken from https://github.com/QuantConnect/Lean/pull/6227
- Rebased + a few more tweaks and added tests
2022-06-15 13:47:00 -03:00
Martin-Molinero 223066d6d9 Disable live trading warmup plotting (#6393)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Disable live trading warmup plotting during warmup
2022-06-09 17:27:55 -03:00
Martin-Molinero 3a00b6963c Avoid sending expired symbols to IDQH (#6378)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Avoid sending expired symbols to the IDQH. This can happen during
  warmup period. Adding unit tests
2022-06-06 19:30:04 -03:00
Martin-Molinero e0b9a2735f Warmup fixes (#6293)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add regression test reproducing issue

* WIP

* Update regression algorithms

* Improvements

* Cleanup and more fixes

- Clean up BaseDataExchange

* RealTimeHandler fixes

* Address reviews

* More comments, tests and minor tweaks

* Avoid false test failures

* Fixes

* Minor improvements

* Scheduled Event start time fix

* Add warmup option regression algorithm and fix

* Add WarmupFutureRegressionAlgorithm

* Normalize backtesting and live future selection
2022-05-31 10:03:56 -03:00
Jhonathan Abreu 69dd4fc638 Fixed bug for trailing stop model tracking original security price (#6356)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Reproduce TrailingStopRiskManagementModel bug with test case

* Fix TrailingStopRiskManagementModel to make it relative to max price

* Adapted old TrailingStopRiskManagementModel test to new implementation

* Fix TrailingStopRiskManagementModel Python version

* Fixed TrailingStopRiskFrameworkAlgorithm regression tests data

* Handling both long and short positions in TrailingStopRiskManagementModel

* Traking holdings value instead of unrealized profit in TrailingStopRiskManagementModel

* Checking for position side change in TrailingStopRiskManagementModel

* Handling immediate liquidation in TrailingStopRiskManagementModel
2022-05-26 19:50:54 -03:00
Ricardo Andrés Marino Rojas e78c3e1eb7 Solve bug when plotting Python indicators (#6347)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* First commit

* Add unit test

* Change implementation

* Add unit tests

* Nit change

* nit change

* Remove unnecessary methods

* Add more unit tests

* Revert "Add more unit tests"

This reverts commit 1ba2ab7454efafba7c38034a15df3f8e97bc7146.

* Add more unit tests

* Add more unit tests

* Improve Implementation

* Change unit tests

* Remove unit tests

Remove unit tests from AlgorithmRegisterIndicatorTests.cs
2022-05-26 13:03:42 -03:00
Martin-Molinero b9d3d99917 Bump pythonNet version 2.0.12 (#6310)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Updates after pythonNet rebase

* Bump pythonNet version 2.0.12

* Fix exception types being thrown
2022-05-02 14:38:50 -03:00
Ricardo Andrés Marino Rojas cde4743ca7 Warm up EmaCrossAlphaModel indicators (#6270)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Warm up EmaCrossAlphaModel

Warm Up EmaCrossAlphaModel indicators

* Fix regression test bug

When using the default `EmaCrossAlphaModel()` the period of both indicators to be ready is bigger than the difference between the start date and the end date of the algorithm. Then, as the algorithm didn't warm up the data both indicators of EmaCrossAlpha never were ready, but now as the model warms up the data both indicators are ready so we get different statistics

* Requested change

* Fix unit tests

As there wasn't items in `AddedSecurities`, when trying to remove the items in ´RemovedSecurities´ there was nothing to remove because there was never a security in `_symbolDataBySymbol`. That's why, in order to test, the behavior of `EmaCrossAlphaModel` when removing a security we need to first add one to then remove it.

* Requested changes in Python

- Requested changes in Python
- Nit changes

* Nit change

* Requested Changes

* Add RemoveConsolidators() method in Python version
2022-04-01 12:15:35 -03:00
Ronit Jain f86926bf7a Fixes HistoryProviderManager slice merging by using algortime for all slices (#6226)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* remove un-used

* use utc time for slice sync

* use utc time

* refactor

* add regression test

* use utc time

* use utc time

* make utctime required parameter

* add utcTime in slice creation

* assert warm up complete

* check if algorithm is still warmingup

* use exchange tz
2022-02-18 20:43:33 -03:00
Alexandre Catarino 3e52816f6e Fixes GetBuyingPowerModel Method of DefaultBrokerageModel (#6215)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`CashBuyingPowerModel`, which reflected on `AlphaStreamBrokerageModel`, a margin-only brokerage.

It also didn't consider the account type, so `InteractiveBrokersBrokerageModel` was using the margin model even if `AccountType.Cash` was selected.

Removes `GetBuyingPowerModel` method from other Brokerage Models when their cases are covered by `DefaultBrokerageModel`

Fixes some typoes in `TradierBrokerageModel`

Fixes unit and regression tests. For the regression tests, we have explicitly set the brokerage model.
2022-02-17 19:41:55 -03:00
Martin-Molinero c8dc343c13 GetLastKnownPrices python data (#6191)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adding unit tests reproducing issue.

* Fix a couple of minor bugs

- IsMarketOpen will work correctly when used with daily and hourly
  resolution.
- slice.Get will work correctly with python custom data
- ExtendedDictionary will be able to dinamically access methods,
  required for python and private C# data types

* Refactor solution. Add more tests

* Remove unrequired import statement
2022-02-08 15:37:05 -03:00
Martin-Molinero e1d1e28bb8 Fix for Tick subscription history requests (#6156)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Fix for history requests != Tick for existing subscriptions with Tick
  resolution. Adding unit test reproducing issue
2022-01-18 17:36:27 -03:00
Ricardo Andrés Marino Rojas 472f78cc53 Remove Quandl from LEAN (#6110)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Remove Quandl from LEAN

* Nit changes and CustomLiveDataFeedTests.cs

* Resolve conflicts

* Remove files related with Quandl

* Fix bug

* Fix QuantBookHistoryTests.cs

* Fix bug

* Fix bug

* Fix unit tests

* Try fix regression tests

* Nit changes

* Fix bug

* Some of the requested changes

* The missing changes

* Requested changes

* Nit changes

* Revert "Nit changes"

This reverts commit 9800bc5c34f3ac20e30bea7a92dd4a9867213bb5.

* Nit changes

* Fix bug

* Requested changes

* Missing file using Quandl to be removed

* Nit changes

* Not applied nit change

* Nit change

* Nit change

* Add nasdaq-auth-code parameter in config.json

* Remove 'quandl-auth-token' from config.json

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-01-12 12:10:20 -03:00
Martin-Molinero 5bdc60b137 Fix for warmup history requests when internal subscriptions present (#6146)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2022-01-10 11:10:44 -03:00
Martin-Molinero 8e2554b110 Add continuous futures MHDB always. Adding unit tests (#6139)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2022-01-04 20:31:31 -03:00
Martin-Molinero 68e2a9170a Do not send internal SecurityChanges to Algorithm (#6118)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Do not send internal SecurityChanges to Algorithm

- Will not send internal security changes to the Algorithm by default.
  Following custom security changes filter pattern. Updating regression
  algorithms to assert behavior.
- The universe member will know wether it was added with internal
  configurations or not

* Address reviews use a separate collection for internals

* Refactor solution. Adding security changes constructor class
2021-12-21 20:24:32 -03:00
Martin-Molinero 5f95a9ba77 Bug fix python history requests enumerable data (#6066)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix for python enumerable data history request

- Fix for python enumerable type data history request. Adding unit test.

* Add methods for adding data points into a baseDataCollection
2021-11-19 14:58:48 -03:00
Colton Sellers 33599b473d Refactor Delistings Processing (#6059)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Move processing of delistings to Brokerage

* Deal with case that exchange is not open on OptionSymbol.ID.Date

* Refactor solution to use DelistingNotification event

* Adjust some regression expected liquidation time

* Mark some todos on deprecated functions

* Update expected liqudation time for Py regressions

* Update regressions that have been validated

* Use HandlePositionAssigned for assignment orders

* Update regressions

* Update some missed unit tests; remove one that is already covered by regression

* Cleanup deprecated backend functions

* nit - small cleanup adjustment

* Post rebase fix

* Address review

* Minor tweak to py regression
2021-11-17 17:43:35 -03:00
Martin-Molinero b8b0d18993 Continuous futures (#6034)
* Continuous Future Contracts

* Mapping approach

* Tweaks WIP

* Live mapping

* Live mapping

- Add support for live mapping, refreshing mapfiles
- Fix future expiration functions
- Adding unit tests

* Update moq test package

* Continuous futures price scaling

* Refactor price factors scaling

* Factor file related renames

* Address reviews
2021-11-15 14:44:30 -03:00
Colton Sellers dd4da7ba95 Feature Daily/Hourly Options Support (#6017)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Create generic writing for LeanDataWriter, + notes on todos

* Make Options Daily/Hourly data store by year

* Refactor Generic Write

* Permit hour and daily resolutions for options

* Refactor writer to merge when needed with other files

* Cleanup redundancies, run write tasks in parallel

* Make needed classes/vars available

* Update tests to reflect new naming convention for daily hourly options data

* Add Byte[] overloads for ZipData functions in compression

* Implemented Store() for ZipDataCacheProvider

* Have LeanDataWriter use a DataCacheProvider

* ZipDataCacheProvider cleanup

* ZipDataCacheProvider tweaks, doesn't support storing non-zips

* Test adjustments

* Update LeanDataWriter to use Write instead of SaveDailyHourly/SaveMinuteSecond

* Implement tests to verify DownloadAndSave behavior

* Nit cleanup on DownloadAndSave tests

* Fix for options daily/hourly underlying equity subscription read

* Add daily/hourly options data and regressions

* Add missing open interest for hourly

* Fix writing of OpenInterest Daily/Hourly data

* Update data

* Fix Date typo in regression

* Use daily algorithm to test delisting

* Revisions part 1

* Expand test for DataCacheProviders; refactor DiskDataCacheProvider

* nit - test adjustments

* ZipDataCacheProvider test setup refactor

* Adjust multithreaded read/write test; fixes for ZipDataCacheProvider

* Move DiskDataCacheProvider to its own file and add write test

* Remove _appendToZips; always overwrite entry or create zip

* Add mapping regression for daily options

* nit - add license to regression

* Fix Tick write case where more than one data point for a DateTime

* Fix data issue

* Address review

* Tweaks for tests

* Stop Store() early if no entry name is given
2021-11-11 20:05:31 -03:00
Ricardo Andrés Marino Rojas 3c1ddb7b96 Enable Warm Up process in MacdAlphaModel (#6037)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Warm up MACD indicators
- When a security is added in the MACD alpha model, it's warm up at once

* Add unit tests

* Nit change

* Code style and nit changes
2021-11-04 12:15:32 -03:00
Ricardo Andrés Marino Rojas fc6ddc2120 Feature 5988 WarmUpIndicator() method for indicators written in Python (#6027)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Implement IIndicatorWarmUpPeriodProvider
- Implement IIndicatorWarmUpPeriodProvider in PythonIndicator.cs
- Make a unit test to check whether the WarmUpPeriod is working as expected
- Make a regression test to check the new feature at a system level

* Nit change

* Change Period parameter for WarmUpPeriod parameter
- Change regression test to check if the new parameter keep backwards compatibility with indicators that do not set WarmUpPeriod

* Documentation change

* Fix tests bugs
- In CommonIndicatorTests.cs before finish the test it checks the period.value with the number of samples but for default the period.value was set to -1

* Change names

* Change WarmUp and RegisterIndicator methods
- Lean WarmUp indicator skip custom python indicators that don't define WarmUpPeriod parameter

* Call WarmUpIndicator manually
- Add a new "bridge" method called WarmUpIndicator in QCAlgorithm.Python.cs to set up everything to call WarmUpIndicator in QCAlgorithm.Indicators.cs
- Change the regression algorithm to warm up the indicators manually

* Remove unnecessary code and add more tests

* Nit change

* Revert "Nit change"

This reverts commit da411f59c9e4295d75a11c6c581f615a938dd2c6.

* Fix bugs

* Try fix bugs

* Add C# regression test
- More nit changes
- Fix bugs

* Requested changes

* Remove unnecessary code

* Requested changes

* Nit changes
- Add new Python class to check a custom indicator, which doesn't inherits from PythonIndicator, warms up properly

* Reduce redundant code

* Fix bug and add more unit and regression tests

* - Add more unit tests

* Nit change

* Test cleanup

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-10-28 20:29:22 -03:00
Colton Sellers d2d99b1f10 Algorithm Sampling and Statistics Fixes (#5936)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement scheduled event sampling solution

* Use UTC time, only update daily portfolio value once a day

* For daily resolutions sample chart always

* Cleanup

* Drop resample daily all together

* Force final sample

* Regression updates

* FIx LiveResultHandler to update portfolio and benchmark values outside of sampling event

* Name the daily sampling event

* Address review pt 1

* Drop force and use reference wrapper

* Adjust tests

* Fix warning for Benchmark Timezone Misalignment and also add test

* Fix for daily resolution orders and test adjustments

* Also warn on universe settings with daily resolution

* Update missed regression

* Fix reference wrapper use

* Update regression after rebase

* Add values back in for Daylight Algo

* Have statistics builder skip day 1 performance

* Regression adjustments

* Test adjustments

* Update regression unit test

* Adjust some regressions starts to show performance values

* Add hourly algorithm for beta comparison

* Address missing Python regression changes

* Remove null comment
2021-10-05 19:31:25 -03:00
Stefano Raggi 6c42a266be IB brokerage - Update option positions at contract expiration (#5880)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* IB brokerage - Update option positions at contract expiration

* Add handling of live option expiration events

* Address review

- Nits

* Refactor option notification events [WIP]

* Add early exercise+assignment unit tests

* Fix sign bug in IB ExerciseOrder

* Address review

- log unexpected position
- add unit test cases for partial exercise/assignment
2021-09-30 13:48:41 -03:00
Colton Sellers 8358c394e1 Support all security types for benchmarks (#5919)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust benchmark subscription to dynamically determine best resolution

* Get types from SM

* Add red/green test set for security types
2021-09-10 16:58:31 -03:00
Martin-Molinero b251cffc2f Alpha Streams improvements (#5887)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Few improvements

- Alpha Streams algorithm will add target securities right away
- If algorithm is warming up PortfolioTargetCollection will emit not
  insight
- Fix local disk factor file provider to use local cache instead of data
  folder
- EWAS PCM will not emit targets for securities which have not been
  added by the algorithm yet, to avoid runtime exception

* Fix GetLastKnownPrice default order adding more unit tests
2021-09-01 13:12:26 -03:00
Martin-Molinero 933e5ce6ca Alpha Streams improvements (#5876)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Alpha Streams improvements

- Warmup added securities so we can trigger an order right away
- Fix bug where AddSecurity returned a security which was not the one
  being used
- Fix bug where EWAS PCM would remove from a dictionary while iterating
  over it

* Fix unit tests

* Add QCAlgorithm.GetLastKnownPrices API

- Add new GetLastKnownPrices that will return the last known data point
  for all subscribed data types.
- Fix for MHDB GetDataTimeZone which was using an invalid entry key
  format for custom data.
- Fix for using GetLastKnownPrice/s to seed a security during creation,
  when it's not added in the Algorithm.Securities collection

* Adding more unit tests for GetLastKnownPrice

* Address reviews

- Refactor GetLastKnownPrice/s to perform a single history request for
  all data types
2021-08-27 13:50:45 -03:00
Gerardo Salazar a4d49c05ca Adds ETF(...) to UniverseDefinitions (#5873)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adds ETF(...) to UniverseDefinitions

  * Adds ETF constituents universe framework regression algorithm
    for C#/Python

* Address review: adds test cases for ticker/Symbol ETF universe additions

  * Fixes bug where null Market would result in null dereference exception

* Address review: add missing Index tests

* Address review: don't hardcode market when creating constituent universe

  * Uses Brokerage Model's default markets collection to determine
    the market for the given security type

* Address review: restore QC500 and DollarVolume.Top(...)

  * Restores algorithms related to both helper universe
    definition methods

* Address review: remove copy to output directory for python algos

* Add example algorithms for ETF constituent universes using custom RSI alpha model

* Address review: adjust algorithm to use cache + algo RSI & clean up code

* Address review: make ETF Constituent RSI Alpha Model algo a regression test

* Address review: increase trade count and remove single trade logic
2021-08-25 11:22:31 -03:00
Martin-Molinero bf28a1d313 Add basic template Atreyu algorithm (#5872)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add basic template Atreyu algorithm

- Add C# and Python basic template atreyu algorithm. Show casing how to
  specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use

* Lean Exchange improvements

- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value

* Regression tests fixes
2021-08-23 18:24:44 -03:00
IlshatGaripov eccca2c029 Fix for Trailing Stop Risk Management Model (#5791)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* To fix the referenced issue

* Fixing up TrailingStopRiskManagementModel

* Fix-ups to address review

* Impl. tests + relevant improvements

* Minor logic improvement at first dictionary update

* Regression test small fix

* Adds explicit 'D' suffix for numbers in test case double arr

* Use integer values in test cases

* Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming

* Revert "Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming"

This reverts commit f9cd279f8cc5e9e1ede5b6f1eae5f4266b7dd295.

* Fix up for failing test cases
2021-08-20 15:14:36 -03:00
Martin-Molinero 3410832ccf Alpha holdings state (#5770)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state

- Alpha result packet will optionally provide the algorithms portfolio
  state

* Rename

* Convert AlphaStreamsPortfolio to data source

* Improvements on AlphaStreams algorithm

* Fix regression tests

* Add unit tests for EW AS PCM and fixing bugs

* Protobuf AlphaStreamsPortfoliot staState

- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM

* Add alpha license to Organization response

* Improvements EW AS PCM respects free portfolio value

* Fixes

- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType

* Fixed and adding more regression tests

- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
  types
- Add support and regression test for algorithm alpha consumer with existing holdings

* Add AlphaStreamsOrderEvent data type
2021-08-19 13:56:18 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
2021-08-02 19:24:45 -03:00
Colton Sellers 3430b46e6a BuyingPowerModel GetMaximumOrder Adjustments (#5756)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add reproducing unit test

* Apply fix to force adjustment if needed

* Adjust fee to correctly replicate original issue

* Adjust solution

* Update unit test to reflect the way this function is used

* nit - rename test

* Cleanup error message to improve reproducibility

* nit - Drop unneeded var

* Add in adjustment test cases as well, fixed
2021-07-09 19:37:48 -03:00
bmello4688 8383163aae fixed problems with testcasesource (#5722)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-06 12:04:04 -03:00
Colton Sellers 171f0f19d5 MaximumOrderQuantity Bug Fix (#5727)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement solution with lightest changes possible

* Update regressions v1

* Adjust AlgorithmTradingTests

* Adjust PatternDayTradingMarginBuyingPowerModel tests

* Drop need to loop twice

* Adjust last unit test, with calculations included

* nit - comment fix

* Break out adjustment calculation to static function; add unit test

* Update Py regression

* Upgrade adjustment calculation to be smart enough to get us to target always

* nit - cleanup GetAmountToOrder

* Add license to test

* nit - comment fix

* cleanup GetAmountToOrder further

* Add additional test cases
2021-07-02 18:52:15 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
2021-06-15 19:06:06 -03:00
Louis Szeto 9f15ca4c9c feature-<#5631>-<Create SpreadExecutionModel> (#5633)
Build & Test Lean / build (push) Has been cancelled
* Create SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Create SpreadExecutionModel.cs

* Create SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update QuantConnect.Algorithm.Framework.csproj

* Create SpreadExecutionModelRegressionAlgorithm.py

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.cs

* Address review

- Adding CSharp regression algorithm implementation
- Updating unit tests, minor format changes

* Address review. Add comment and Abs spread percent

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-06-09 14:55:00 -03:00
Colton Sellers ab6027723f Refactor Api (#5251)
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request

* key -> filePath rename and some cleanup

* Refactor

* Add Organizations Endpoints

* Add some organization api wrapper objects

* Address namespace issue

* Reorganize Api Test into seperate files using one ApiTestBase

* Add Organization tests

* Use capitalized "API" test namespace to reduce amount of file changes

* Add License to test base

* Update /data endpoint functions and response objects

* Update ApiDataProvider Logic

* Handle deserialization of organization products

* Simplify converter

* Only throw for equity requests when not subscribed to map/factor files

* Add missing header

* Make arguement exception

* Api adjustments

* Add Zip factor and map file providers

- Common project will now reference Compression project and not the other way
  round.
- Adding Zip FactorFile and MapFile providers

* Refactor FactorFileProvider to use DataProvider to fetch files

* Use resulting MinimumDate in construction of FactorFile

* Nit FactorFile comments and arrangement

* Refactor MapFileProviders to use DataProvider for fetching files

* Refactor ZipFileProvider

* Clean up

* Refactor Backtesting Future/Option chain providers to use dataprovider

* Fixes for data/ endpoints and test adjustments

* Response objects adjustments/cleanups

* ApiDateProvider fixes and testing

* Add LocalZipFactorFileTests

* Update ApiDataProvider download test to verify stream is not null

* Implement posting of agreement summary and signed time

* Mark all Api related tests as explicit and document details on running

* Clarify default token on ApiTestBase

* Adjust summary

* Update Api responses for QCC, except org products which are sold in USD

* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests

* Fix multiple markets for ZipFactorFile provider

* Use Symbol as cache key

* Api.cs review

* Dispose of factorFileStream after reading

* Use zip.EntryFileNames

* Address a few reviews

* Few more fixes

* Address Api Review

* Add Job Org id to config

* Minor tweaks

* Compare with invariant culture

* Fixes Option Universe selection

* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider

* Fix research

* Fix null reference exception

* Make duplicate log debug

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-06-07 11:37:44 -03:00
Colton Sellers d76bfaade4 Consolidate SetupHandlers (#5529)
Build & Test Lean / build (push) Has been cancelled
* Reconcile and consolidate SetupHandlers

* Centralize GetConfiguredDataFeeds()

* nit - extra space

* Check for null, also allow null to be passed back if no config value

* Fix breaking test

* Cleanup

* fix return var

* remove unneeded if/else

* Minor changes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-10 22:12:16 -03:00
Colton Sellers ed077da501 DataNormalizationMode Applies to AddSecurity (#5528)
Build & Test Lean / build (push) Has been cancelled
* Use UniverseSettings.DataNormalizationMode for securities added in Algorithm

* Stop SubscriptionUtils from forcing Adjusted mode

* Return behavior to original and add comments

* nit typo

* Add regression

* Add unit test that verifies DataNormalizationMode can be altered manually by security

* Cleanup and add Py version of regression
2021-05-10 20:58:50 -03:00
Martin-Molinero f3c386663b Feature .net 5 (#5505)
* Update projects to use .NET 5.0, the successor to .NET Core

* Fix ambiguous errors. Add IBAutomator net5

* Remove FXCM

* Upgrade IBAutomater to v1.0.51

ignored, and an empty message aborts the commit.

* Fix rebase

- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy

* Remove unrequired references

* Fixes

- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests

* Don't call ReadKey when input is redirected

* Fix ConsoleLeanOptimizer

* Research fixes

* Update comment

* Add vsdbg to Dockerfile

* Fixes

- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
   - Option assignment will be deterministic in the order
   - 'Rolling Averaged Population' is calculated using doubles, updating
     expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock

* Replace ICSharp with .NET Interactive

* Fixes after rebase

* CSharp research fixes

- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
2021-05-06 17:23:51 -03:00
Martin-Molinero 83f9499b4a Option Margin Strategies (#5511)
* Refactor HasSufficientBuyingPowerForOrder implementations

Adds Sufficient and Insufficient helper methods to HashSufficientbuyingPowerForOrderParameters
enabling syntax like:

return paraeeters.Sufficient()
returnparameters.Insufficient(reason)

The next change will add the initial margin required which will simply require
updating both of these helper methods to accept the value.

* IBuyingPowerModel: Add margin functions Maintenance/Initial/ForOrder

These were originally hidden in an effort to only expose what's necessary
for the engine to perform its work. Additionally, we encapsulated all of
the method arguments into parameters classes to prevent having to break
anyone in the future. Not including these foundational methods turns out to
be an oversight. These methods are not required by the engine, but rather by
other models. Another possible solution here is to add an additional abstraction
and include these methods on this new abstraction. BuyingPowerModel would then
explicitly implement these methods and models that depend on them would require
two code paths, one for when the buying power model implements this interface
and another for when it doesn't.

Tests were additionally updated to remove test model implementations created for
the sole purpose of exposing these private methods.

* Add ConstantBuyingPowerModel

Provides an implementation of IBuyingPowerModel that returns the same
constant value

* Update BuyingPowerModelPythonWrapper to use reflection for method names

Having a bunch of hard-coded strings is a sure fire way for someone to
overlook when changing methods. This change ensures that noone needs to
remember that this code exists :)

Cleans up the syntax around verifying a python object implements a particular
C# interface via the ValidateImplementationOf<T> method by having it return a
value since the only use cases are in constructors when setting the models.

I was initially going to update ALL python wrappers to validate the passed
in models, but such a change could break many things that are 'working' right
now. Such an effort should be saved for its own dedicated PR.

* Add Parameters/Result types for new buying power model methods

* Support computing maintenance margin for arbitrary quantities

The existing GetMaintenanceMargin function assumes that we're only interested
in the maintenance margin for the entirety of the provided security's holdings.
This makes it impossible to perform what-if analysis or to even ask how much
maintenance margin is devoted to a particular subset of the security's holdings.
This change adds the quantity to the MaintenanceMarginParameters class. Futures
and Options models also depend on holdings cost and holdings value, so they have
also been added to the parameters type. Finally, static factory methods were
added to improve discernment of intent: ForCurrentHoldings provides the existing
behavior and then ForQuantityAtCurrentPrice to support what-if scenarios where
we're looking for the change in maintenance margin if we were to execute an order
for the securiy at the current time step. Obviously a constructor is provided to
set all of the values explicitly, using any price metric the caller desires.

* Address review

- Fix BPM xml documentation
- Fix python unit tests and PythonWrapper validate method

* Add SecurityHolding.QuantityChanged event

Adding event handlers will allow us to orchestrate complex
events from distant parts of the codebase through wiring
them up. If we continue down this path, it will move us away
from the current, very 'mechanical' data flows expressed in
LEAN and towards a more modern, event processing based system.
This is but a baby step in that direction and the initial use
case is using this QuantityChanged event to trigger resolution
of the algoritm's positions groups. This is part of an effort
to improve the fidelity of options margin modeling where we'll
model an OptionStrategy as an IPositionGroup. This will allow
us to compute the margin requirements of an OptionStrategy as
a unit instead of computing margin of each security individually
in isolation.

See #4065

* PortfolioManager: Group fields and remove unused field

This codebase generally places fields as the first members, but
this class had some fields at the top, then some properties, and
then some more fields. This change brings all the fields together
at the top of the file and also removes pointless comments placed
directly above some of the fields. Additionally, an unused field
was removed.

* Remove unused _currencyConverter from Security

Looks like at some point the only code using this member variable was removed
and the necessary clean up was overlooked.

* Add Parse.Enum functions

* Support disabling regression algorithms by language via config.json

Adds 'regression-test-languages' to config.json and filters regerssion algorithms to
run based on this value. When cycling on a particular feature, it's nice to be able
to run the entire regression set while ignoring the python algorithms. Once the C#
algorithms are all passing, one can then go back and run C# and Python in a final run,
since 99% of feature work doesn't impact python specifically.

* Implement IComparable in SecurityIdentitfier

This can be used to deterministically sort securities and symbols

* Add .editorconfig to enforce common formatting for json/sh files

* Fix typo in IBuyingPowerModel.GetBuyingPower xml docs

* Add ListEquals/GetListHashCode and OrderDirection.Closes(PositionSide)

ListEquals and GetListHashCode are designed to be used together as they
complement each other according to C#'s requirements for Equals and
GetHashCode functions.

PositionSide.ToOrderDirection() extension simply converts a PositionSide
to its logical equivalent OrderDirection. Long->Buy, Short->Sell, None->Hold

OrderDirection.Closes(PositionSide) determines if a particular OrderDirection
would have the effect of reducing a position's absolute size. This function
greatly improves the readability of buying power functions that must provide
adjustments when an order/contemplated trade reduces/closes an existing position.
OrderDirection.Buy.Closes(PositionSide.Short)
OrderDirection.Sell.Closes(PositionSide.Long)
All other combinations return false

Adds ToArray/ToImmutableArray convenience functions that combine a call
to Select followed by To(Immutable)Array all in one function call.

* Add decimal.DiscretelyRoundBy extension method

Supports rounding a decimal value by an arbitrarily chosen maximum precision,
or 'quanta'

* Update FutureMarginBuyingPowerModelTests to respect the security's lot size

* Add core position group classes and abstractions

* Add initial/maintenance margin support, buying power model consistency tests

* Add SufficientBuyingPower and GetReservedBuyingPower to position group model

Includes update to BrokerageTransactionHandler to use position group BPM for
sufficient buying power checks.

* Resolve position groups on each fill

We need to update the state of our position groups on each fill so that
we can properly handle multiple orders within the same time step. We
also limit the number of positions sent into the resolver by removing
securities without any holdings.

* fixup! Add SufficientBuyingPower and GetReservedBuyingPower to position group model

* Add GetMaximumLotsFor{Target|Delta}BuyingPower

Instead of computing order quantity, these functions compute the
maximum number of position group lots, which is the position group
quantity, and is guaranteed to be a whole number, for the provided
target/delta buying power parameters.

The SecurityPositionGroupBuyingPowerModel delegates to the security's
IBuyingPowerModel by applying a scaling factor equal to the security's
lot size.

This change also updates references to IBuyingPowerModel.GetMaximum...
to use the new position group model methods.

* Convert remaining IBuyingPowerModel call sites to position groups

* Rename PositionManasger.CreateDefaultGroup -> GetOrCreateDefaultGroup

Better describes its behavior

* Add Position Groups readme.md

* Add Option Strategy BuyingPowerModel

- Adding CompositePrositionGroupResolver and
  OptionStrategyPositionGroupResolver
- Adding OptionStrategyPositionGroupBuyingPowerModel handling option
  strategies based on IBs margin table. Adding regression algorithms
- Few changes so that option strategies executed by multiple orders are
  detected
- Adjust OptionStrategyDefinitionMatch to include equity legs in the
  matching result
- Minor tweaks fixing previous rebase
- Minor fixes for existing option strategies definitions, adding new
  missing strategies.
- Fixing minor bugs in option strategy matcher. Adding more unit tests

* Address self reviews

- Fixing bug in 'PositionGroupCollection'
- Few minor simplificaitons
- Adding BasicTemplateOptionEquityStrategyAlgorithm

* Address reviews

- Improve regression algorithms margin remaining and used assert logic to be exact. Taking into account spread and fees

Co-authored-by: Michael Handschuh <mhandschuh@gmail.com>
2021-04-30 18:45:27 -03:00
Colton Sellers 1c3d849ad5 Fix Warnings V2 (#5436)
* Reconcile duplicated code

* Add License header

* CS0219 Fixes: Value assigned, but never used

* CA1507: Use nameof in place of string literals

* CS0108 : Hides Inherited Member; Use new keyword to overwrite formally

* CS0114: Hides inherited member; use override keyword

* CS0168: Variable is declared but never used

* Tests CS1062; using obsolete implicit Symbol -> String; fix via .ToString()

* CS0472: Non Nullable Obj getting Null Checked

* CS0067 Member not used; ignore all cases for future use

* CS00162 : Unreachable code; either removed or ignored for debugging and test cases

* CS0169 Remove non-used fields; ignore those that may be used in future

* CS0414; Field is assigned but never used.

* CS0618; Obsolete properties and members; Only fixes simple ones, rest will have to broken up

* CS0649; Field never assigned too

* CS0659 & CS0661 ; Overwrite operators and equals but not hashcode; I don't really override it but just call base

* Small comment fix

* Cleanup pragma statement
2021-04-02 11:20:01 -07:00
Colton Sellers e2a0873b7c Fix Lean Warnings V1 (#5408)
Cleanup all non-breaking warnings
2021-03-22 11:08:48 -07:00