127 Commits

Author SHA1 Message Date
Jhonathan Abreu 5436c72fad Handle margin call orders for position groups (#7137)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Handle position group margin calls

* Minor changes

* Update regression algorithms

* Minor changes

* Minor changes

* Added regression algorithm

* Peer review

* Peer review

* Peer review

* Peer review

* Minor changes

* Minor changes

* Add unit test

* Allow sufficient buying power when closing position group

* Add unit test

* Improve regression algorithms
2023-04-10 12:42:08 -03:00
Alexandre Catarino 497ac8497a Fixes Consolidator Removal from Subscription Manager in Python Algorithms (#7102)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Adds Unit Test

Asserts that we can remove a consolidator that was previously added.

* AddConsolidator Method Only Wraps Non-C# Consolidators

This behavior was implemented in `QCAlgorithm.RegisterIndicator` to avoid wrapping an object unnecessarily.
2023-03-21 11:08:49 -03:00
Martin-Molinero bbbab6d9a8 Refactor alpha statistics phase I (#7055)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics

- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
  algorithms

* Optimization backtest result json converter update

* Address reviews

- Remove IAlphaHandler, move insight storage responsability to IResultHandler
  and centralizing insight collection on the QCAlgorithm.Insights to be
  reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
  sampling handling. Updating regression algorithms

* Add InsightCollection tests and minor fixes

* Adding more & improved tests
2023-03-10 13:12:23 -03:00
Martin-Molinero 66c016b284 Feature foundation update (#7028)
* Foundation Update

* Adding more foundation packages

* Update existing libraries

* Further package updates

* Image cleanup

- Update base image to ubuntu 22.04
- Some clean up on dockerfile
- Remove ARM base image

* Test updates

- Minor docker file tweaks
- Update existing python environment tests

* Add more tests & fix Julia

* Remove jupyter-dash

* Remove R from foundation

* Remove Julia

* Remove Tensorforce
2023-03-01 16:30:46 -03:00
Jhonathan Abreu 41a730bc68 OTM option excerise orders improvements (#6767)
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Set fill price to zero in OTM exercise orders.

Improved the OTM exercise orders message.

* Update regression algorithms and unit tests

* Add IsInTheMoney property to OrderEvent

* Update SerializedOrderEvent

* Properly setting the option exercise order price to strike price or zero

* Minor changes

* Minor changes
2022-12-06 14:30:40 -03:00
Martin-Molinero fed1b6e780 Add pyvinecopulib to foundation (#6679)
Benchmarks / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Adding pyvinecopulib to foundation. Adding unit test
2022-10-11 16:30:57 -03:00
Martin-Molinero f528c39700 Foundation update (#6677)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add new python packages to foundation images. Adding tests
2022-10-10 20:03:33 -03:00
Alexandre Catarino abeb841a0c Adds Additional Condition to Parse SecurityIdentifier Properties (#6658)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Adds Additional Condition to Parse SecurityIdentifier Properties

A string can pass the previous conditions and create an invalid `SecurityIdentifier` (e.g. "Sharpe ration").  If it is invalid, the Market is not supported (e.g. "378").

* Adds Unit Test to Pandas Indexing

* Log Only Once if TryParseProperties Cannot Parse

Use cache to increase speed and remove redundant logging
2022-09-29 10:13:45 -03:00
Martin-Molinero ee685ea045 Add new python libraries to foundation (#6629)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-09-19 13:20:54 -03:00
Jhonathan Abreu f62f110b1b Handle non-unique multi-index error in history dataframe creation (#6617)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Handle non-unique multi-index error for Ticks in data frame creation

* Update unit tests

* Update unit tests and add comments

* Add regression algorithm
2022-09-08 19:40:57 -03:00
Alexandre Catarino f755b91158 Fixes PythonData.EndTime (#6602)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Fixes PythonData.EndTime

The `EndTime` property of `PythonData` didn't override the base implementation in `BaseData`. Therefore `EndTime` was always set as `Time`.

Fixes Regression Tests
- The C# version of `Bitcoin` class needs to implement `EndTime`-
- Fixes regression tests

* Minor tweaks and rebase

- Add support for python data types just setting EndTime. Adding unit
  tests for PythonData
- Fix for DynamicData EndTime property being fetched. Updating unit
  tests

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-09-07 14:53:05 -03:00
Martin-Molinero 19f3d8b305 Fix python memory leak (#6579)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix python memory leak

- Fix python memory leak. Adding unit test

* Update pythonnet to 2.0.17
2022-08-24 18:24:07 -03:00
Martin-Molinero 7fdedbe98f Rename python environments (#6572)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Rename python environments

* Rename to Foundation-Py/C#-Default
2022-08-23 19:06:19 -03:00
Ryan Russell 9cd38a388b chore(.NET 6): Update references from dotnet 5 to dotnet 6 #6560 (#6565)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* docs(dotnet6): `dotnet 5` -> `dotnet 6`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

* `.Net 5` -> `.Net 6`

Signed-off-by: Ryan Russell <git@ryanrussell.org>

Signed-off-by: Ryan Russell <git@ryanrussell.org>
2022-08-22 17:51:24 -03:00
Martin-Molinero 3f42101f6d Update to python38 and libraries (#6558)
* Update to python38

- Update to python38
- Update libraries

* Create latest virtual environment. Add CI unit tests

* Further update python packages

* Virtual env fixes

- Refactor python virtual env setup
- Adding CI unit tests for python packages and virtual envs

* Adding more python packages

- Adding NBeats, Ax-platform, Riskfolio-Lib

* Rename pomegranate venv
2022-08-22 13:54:02 -03:00
Martin-Molinero 6f3208eded Python Virtual Environments Support (#6556)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add support to use a python virtual environment if defined in the
  algorithm job. Adding tests
2022-08-17 16:40:52 -03:00
Martin-Molinero bb0c27fefd Period timespan consolidation improvements (#6408)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Period timespan consolidation improvements

- If user is trying to consolidate a period providing data of a bigger
  period we will now throw an exception. Adding tests
- If both consolidated and given data share the same period, gently
  adjust the consolidator into a data count of 1. Adding tests
- Fixing bug in QuoteBarConsolidator period double accounting. Adding unit tests

* Add Period and Count regression algorithm
2022-06-20 15:14:44 -03:00
Jhonathan Abreu df308cc2a7 Make PandasConverter.GetIndicatorDataFrame accept Python dictionary (#6365)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Overload PandasConverter.GetIndicatorDataFrame to accept a Python dict

* Shared implementation code for PandasConverter.GetIndicatorDataFrame overloads

* Added documentation for private shared methods used by PandasConverter.GetIndicatorDataFrame

* Added unit tests for PandasConverter.GetIndicatorDataFrame

* Added unit tests for PandasConverter.GetIndicatorDataFrame Dictionary overload

* Address change requests

* Address change requests
2022-06-01 19:57:51 -03:00
Martin-Molinero eb995b86ea Update to pythonNet 2.0.14 (#6322)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Update to pythonNet 2.0.14

* Add unit test reproducing issue
2022-05-06 17:29:55 -03:00
Martin-Molinero b9d3d99917 Bump pythonNet version 2.0.12 (#6310)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Updates after pythonNet rebase

* Bump pythonNet version 2.0.12

* Fix exception types being thrown
2022-05-02 14:38:50 -03:00
Ronit Jain f86926bf7a Fixes HistoryProviderManager slice merging by using algortime for all slices (#6226)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* remove un-used

* use utc time for slice sync

* use utc time

* refactor

* add regression test

* use utc time

* use utc time

* make utctime required parameter

* add utcTime in slice creation

* assert warm up complete

* check if algorithm is still warmingup

* use exchange tz
2022-02-18 20:43:33 -03:00
Ricardo Andrés Marino Rojas 472f78cc53 Remove Quandl from LEAN (#6110)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Remove Quandl from LEAN

* Nit changes and CustomLiveDataFeedTests.cs

* Resolve conflicts

* Remove files related with Quandl

* Fix bug

* Fix QuantBookHistoryTests.cs

* Fix bug

* Fix bug

* Fix unit tests

* Try fix regression tests

* Nit changes

* Fix bug

* Some of the requested changes

* The missing changes

* Requested changes

* Nit changes

* Revert "Nit changes"

This reverts commit 9800bc5c34f3ac20e30bea7a92dd4a9867213bb5.

* Nit changes

* Fix bug

* Requested changes

* Missing file using Quandl to be removed

* Nit changes

* Not applied nit change

* Nit change

* Nit change

* Add nasdaq-auth-code parameter in config.json

* Remove 'quandl-auth-token' from config.json

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-01-12 12:10:20 -03:00
Martin-Molinero 68e2a9170a Do not send internal SecurityChanges to Algorithm (#6118)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Do not send internal SecurityChanges to Algorithm

- Will not send internal security changes to the Algorithm by default.
  Following custom security changes filter pattern. Updating regression
  algorithms to assert behavior.
- The universe member will know wether it was added with internal
  configurations or not

* Address reviews use a separate collection for internals

* Refactor solution. Adding security changes constructor class
2021-12-21 20:24:32 -03:00
Martin-Molinero 5f95a9ba77 Bug fix python history requests enumerable data (#6066)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix for python enumerable data history request

- Fix for python enumerable type data history request. Adding unit test.

* Add methods for adding data points into a baseDataCollection
2021-11-19 14:58:48 -03:00
Colton Sellers 33599b473d Refactor Delistings Processing (#6059)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Move processing of delistings to Brokerage

* Deal with case that exchange is not open on OptionSymbol.ID.Date

* Refactor solution to use DelistingNotification event

* Adjust some regression expected liquidation time

* Mark some todos on deprecated functions

* Update expected liqudation time for Py regressions

* Update regressions that have been validated

* Use HandlePositionAssigned for assignment orders

* Update regressions

* Update some missed unit tests; remove one that is already covered by regression

* Cleanup deprecated backend functions

* nit - small cleanup adjustment

* Post rebase fix

* Address review

* Minor tweak to py regression
2021-11-17 17:43:35 -03:00
Colton Sellers 2109394060 Add subset test after dropping 'symbol' (#5966)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-10-06 18:21:33 -03:00
Colton Sellers d2d99b1f10 Algorithm Sampling and Statistics Fixes (#5936)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement scheduled event sampling solution

* Use UTC time, only update daily portfolio value once a day

* For daily resolutions sample chart always

* Cleanup

* Drop resample daily all together

* Force final sample

* Regression updates

* FIx LiveResultHandler to update portfolio and benchmark values outside of sampling event

* Name the daily sampling event

* Address review pt 1

* Drop force and use reference wrapper

* Adjust tests

* Fix warning for Benchmark Timezone Misalignment and also add test

* Fix for daily resolution orders and test adjustments

* Also warn on universe settings with daily resolution

* Update missed regression

* Fix reference wrapper use

* Update regression after rebase

* Add values back in for Daylight Algo

* Have statistics builder skip day 1 performance

* Regression adjustments

* Test adjustments

* Update regression unit test

* Adjust some regressions starts to show performance values

* Add hourly algorithm for beta comparison

* Address missing Python regression changes

* Remove null comment
2021-10-05 19:31:25 -03:00
Martin-Molinero 4e6e0c85c6 Lean exchange improvements (#5932)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Lean exchanges improvements

- Adding new Exchange class to avoid exchange code clash.
  Adding and updating unit test

* Add Market for MapFile API

* Self review
2021-09-20 13:50:15 -03:00
Colton Sellers e722e72aaa Pandas Mapper Recursion Bug Fix (#5848)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust wrapper

* Adjustments

* Add red -> green unit tests

* Address review
2021-08-10 20:32:09 -03:00
Colton Sellers e2ae173d99 Feature Python Library: Ray (#5824)
* Add Ray lib to both foundation images

* Update some version failing tests
2021-08-05 17:55:51 -03:00
Alexandre Catarino 9b7af08b3e Fixes Market On Open Fill of Equity Fill Model (#5679)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fixes Market On Open Fill of Equity Fill Model

Only use trade data (Tick with Trade type or TradeBar) to get the open price, since MOO is filled with the opening action price. Ensure that this method doesn't use trade data from before the market opens for high-resolution data case.

Fix unit tests to show that the new implementation only fills with trade data from the current open market.

Change regression tests to reflect the bug fix.
In the `ExtendedMarketHoursHistoryRegressionAlgorithm`, MOO was filled with extended market hours.

* Fix Bug for Tick Resolution Case

For tick susbcription, the tick with the open price information is the the first valid (non-zero) tick of trade type from an open market.
Addresses peer-review by moving the if-condition for data belonging to the open market where the subscriscribed types are checked.

* Fix Bug for Low Resolution Edge Case

For the edge case where the order is placed after the trade bar is open, for example, order places at 1 pm with daily-resolution data. The fill model will not use the open of the bar that will close at midnight, since this value is prior to the order.

Adds unit test.

Change regression tests to reflect the bug fix.
In the `RegressionAlgorithm`, MOO was filled with open prior to the order. The algorithm now has one order less, since the last MOO would need to wait another day to be filled.

* Implements SaleCondition and Exchange Check For Tick

- Adds additional unit tests for MOO

* Fixes Regression Test in DataConsolidatorPythonWrapperTests

* Addresses Peer-Review

- Adds new unit test cases.
2021-08-05 12:53:04 -03:00
Colton Sellers a9b634b096 Pandas Dataframe indexing w/ list (#5813)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Wrap df.__getitem__ to map all keys before indexing

* Add reproducing test case
2021-07-30 16:34:03 -03:00
Colton Sellers d8a86f182f Pandas Remapper Refactor (#5808)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Refactor pandas mapper to support newer versions

* Clean up remapper

* Finalize deprecated tests and adjust those that should apply to new pandas

* Move mapper to its own file

* Add supporting Py tests and setup instructions

* Add license
2021-07-29 20:10:00 -03:00
Martin-Molinero cb326788b3 Filter out small orders based on Setting (#5776)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Filter out small orders based on Setting

- BuyingPowerModel will filter out small orders based on algorithm
  setting, a % of PTV, instead of hard coded 1 share value. Addin unit
  and regression tests
- Updating regression algorithms to use new setting, reduce order trades

* Update regression algorithms
2021-07-19 13:17:51 -03:00
Colton Sellers 171f0f19d5 MaximumOrderQuantity Bug Fix (#5727)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement solution with lightest changes possible

* Update regressions v1

* Adjust AlgorithmTradingTests

* Adjust PatternDayTradingMarginBuyingPowerModel tests

* Drop need to loop twice

* Adjust last unit test, with calculations included

* nit - comment fix

* Break out adjustment calculation to static function; add unit test

* Update Py regression

* Upgrade adjustment calculation to be smart enough to get us to target always

* nit - cleanup GetAmountToOrder

* Add license to test

* nit - comment fix

* cleanup GetAmountToOrder further

* Add additional test cases
2021-07-02 18:52:15 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
2021-06-15 19:06:06 -03:00
Colton Sellers 459f812223 Duplicated Points in Custom DataFrame Fix (#5606)
Build & Test Lean / build (push) Has been cancelled
* Don't add "value" to series twice

* Reproducing unit test
2021-05-26 18:44:13 -03:00
Martin-Molinero 83f9499b4a Option Margin Strategies (#5511)
* Refactor HasSufficientBuyingPowerForOrder implementations

Adds Sufficient and Insufficient helper methods to HashSufficientbuyingPowerForOrderParameters
enabling syntax like:

return paraeeters.Sufficient()
returnparameters.Insufficient(reason)

The next change will add the initial margin required which will simply require
updating both of these helper methods to accept the value.

* IBuyingPowerModel: Add margin functions Maintenance/Initial/ForOrder

These were originally hidden in an effort to only expose what's necessary
for the engine to perform its work. Additionally, we encapsulated all of
the method arguments into parameters classes to prevent having to break
anyone in the future. Not including these foundational methods turns out to
be an oversight. These methods are not required by the engine, but rather by
other models. Another possible solution here is to add an additional abstraction
and include these methods on this new abstraction. BuyingPowerModel would then
explicitly implement these methods and models that depend on them would require
two code paths, one for when the buying power model implements this interface
and another for when it doesn't.

Tests were additionally updated to remove test model implementations created for
the sole purpose of exposing these private methods.

* Add ConstantBuyingPowerModel

Provides an implementation of IBuyingPowerModel that returns the same
constant value

* Update BuyingPowerModelPythonWrapper to use reflection for method names

Having a bunch of hard-coded strings is a sure fire way for someone to
overlook when changing methods. This change ensures that noone needs to
remember that this code exists :)

Cleans up the syntax around verifying a python object implements a particular
C# interface via the ValidateImplementationOf<T> method by having it return a
value since the only use cases are in constructors when setting the models.

I was initially going to update ALL python wrappers to validate the passed
in models, but such a change could break many things that are 'working' right
now. Such an effort should be saved for its own dedicated PR.

* Add Parameters/Result types for new buying power model methods

* Support computing maintenance margin for arbitrary quantities

The existing GetMaintenanceMargin function assumes that we're only interested
in the maintenance margin for the entirety of the provided security's holdings.
This makes it impossible to perform what-if analysis or to even ask how much
maintenance margin is devoted to a particular subset of the security's holdings.
This change adds the quantity to the MaintenanceMarginParameters class. Futures
and Options models also depend on holdings cost and holdings value, so they have
also been added to the parameters type. Finally, static factory methods were
added to improve discernment of intent: ForCurrentHoldings provides the existing
behavior and then ForQuantityAtCurrentPrice to support what-if scenarios where
we're looking for the change in maintenance margin if we were to execute an order
for the securiy at the current time step. Obviously a constructor is provided to
set all of the values explicitly, using any price metric the caller desires.

* Address review

- Fix BPM xml documentation
- Fix python unit tests and PythonWrapper validate method

* Add SecurityHolding.QuantityChanged event

Adding event handlers will allow us to orchestrate complex
events from distant parts of the codebase through wiring
them up. If we continue down this path, it will move us away
from the current, very 'mechanical' data flows expressed in
LEAN and towards a more modern, event processing based system.
This is but a baby step in that direction and the initial use
case is using this QuantityChanged event to trigger resolution
of the algoritm's positions groups. This is part of an effort
to improve the fidelity of options margin modeling where we'll
model an OptionStrategy as an IPositionGroup. This will allow
us to compute the margin requirements of an OptionStrategy as
a unit instead of computing margin of each security individually
in isolation.

See #4065

* PortfolioManager: Group fields and remove unused field

This codebase generally places fields as the first members, but
this class had some fields at the top, then some properties, and
then some more fields. This change brings all the fields together
at the top of the file and also removes pointless comments placed
directly above some of the fields. Additionally, an unused field
was removed.

* Remove unused _currencyConverter from Security

Looks like at some point the only code using this member variable was removed
and the necessary clean up was overlooked.

* Add Parse.Enum functions

* Support disabling regression algorithms by language via config.json

Adds 'regression-test-languages' to config.json and filters regerssion algorithms to
run based on this value. When cycling on a particular feature, it's nice to be able
to run the entire regression set while ignoring the python algorithms. Once the C#
algorithms are all passing, one can then go back and run C# and Python in a final run,
since 99% of feature work doesn't impact python specifically.

* Implement IComparable in SecurityIdentitfier

This can be used to deterministically sort securities and symbols

* Add .editorconfig to enforce common formatting for json/sh files

* Fix typo in IBuyingPowerModel.GetBuyingPower xml docs

* Add ListEquals/GetListHashCode and OrderDirection.Closes(PositionSide)

ListEquals and GetListHashCode are designed to be used together as they
complement each other according to C#'s requirements for Equals and
GetHashCode functions.

PositionSide.ToOrderDirection() extension simply converts a PositionSide
to its logical equivalent OrderDirection. Long->Buy, Short->Sell, None->Hold

OrderDirection.Closes(PositionSide) determines if a particular OrderDirection
would have the effect of reducing a position's absolute size. This function
greatly improves the readability of buying power functions that must provide
adjustments when an order/contemplated trade reduces/closes an existing position.
OrderDirection.Buy.Closes(PositionSide.Short)
OrderDirection.Sell.Closes(PositionSide.Long)
All other combinations return false

Adds ToArray/ToImmutableArray convenience functions that combine a call
to Select followed by To(Immutable)Array all in one function call.

* Add decimal.DiscretelyRoundBy extension method

Supports rounding a decimal value by an arbitrarily chosen maximum precision,
or 'quanta'

* Update FutureMarginBuyingPowerModelTests to respect the security's lot size

* Add core position group classes and abstractions

* Add initial/maintenance margin support, buying power model consistency tests

* Add SufficientBuyingPower and GetReservedBuyingPower to position group model

Includes update to BrokerageTransactionHandler to use position group BPM for
sufficient buying power checks.

* Resolve position groups on each fill

We need to update the state of our position groups on each fill so that
we can properly handle multiple orders within the same time step. We
also limit the number of positions sent into the resolver by removing
securities without any holdings.

* fixup! Add SufficientBuyingPower and GetReservedBuyingPower to position group model

* Add GetMaximumLotsFor{Target|Delta}BuyingPower

Instead of computing order quantity, these functions compute the
maximum number of position group lots, which is the position group
quantity, and is guaranteed to be a whole number, for the provided
target/delta buying power parameters.

The SecurityPositionGroupBuyingPowerModel delegates to the security's
IBuyingPowerModel by applying a scaling factor equal to the security's
lot size.

This change also updates references to IBuyingPowerModel.GetMaximum...
to use the new position group model methods.

* Convert remaining IBuyingPowerModel call sites to position groups

* Rename PositionManasger.CreateDefaultGroup -> GetOrCreateDefaultGroup

Better describes its behavior

* Add Position Groups readme.md

* Add Option Strategy BuyingPowerModel

- Adding CompositePrositionGroupResolver and
  OptionStrategyPositionGroupResolver
- Adding OptionStrategyPositionGroupBuyingPowerModel handling option
  strategies based on IBs margin table. Adding regression algorithms
- Few changes so that option strategies executed by multiple orders are
  detected
- Adjust OptionStrategyDefinitionMatch to include equity legs in the
  matching result
- Minor tweaks fixing previous rebase
- Minor fixes for existing option strategies definitions, adding new
  missing strategies.
- Fixing minor bugs in option strategy matcher. Adding more unit tests

* Address self reviews

- Fixing bug in 'PositionGroupCollection'
- Few minor simplificaitons
- Adding BasicTemplateOptionEquityStrategyAlgorithm

* Address reviews

- Improve regression algorithms margin remaining and used assert logic to be exact. Taking into account spread and fees

Co-authored-by: Michael Handschuh <mhandschuh@gmail.com>
2021-04-30 18:45:27 -03:00
Alexandre Catarino 2b0fd2e607 Updates SPY Market Data (#5493)
* Fixes Double to Decimal Cast in GetAnnualPerformance

`GetAnnualPerformance` raises an exception if the `AnnualPerformance` calculation returns a double that cannot be cast to decimal (smaller than `decimal.MinValue` or bigger than `decimal.MaxValue`).
See `ProbabilisticSharpeRatio` where the same solution was applied.

* Updates SPY Market Data

SPY is a key asset since it is the default benchmark, and any change can lead to different `Alpha` and `Beta`

* Updates Unit Tests to Reflect Data Update

* Updates Regression Tests to Reflect Data Update I

Most of the regression tests change because of updated data (market and factors) of SPY (default benchmark) while the total trade remain the same.

* Updates Regression Tests to Reflect Data Update II

The following regression tests were changed to adapt to adjusted prices and keep the total trades:
- `BacktestingBrokerageRegressionAlgorithm`
- `LimitIfTouchedRegressionAlgorithm`
- `PortfolioRebalanceOnCustomFuncRegressionAlgorithm`
- `SetAccountCurrencySecurityMarginModelRegressionAlgorithm`
- `StopLossOnOrderEventRegressionAlgorithm`
- `TimeInForceAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `FreePortfolioValueRegressionAlgorithm` 2 -> 3
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 291 -> 298
- `TrailingStopRiskFrameworkAlgorithm` 5 -> 7

Especial cases:
- `AutoRegressiveIntegratedMovingAverageRegressionAlgorithm` 65 -> 52
 - ARIMA model sensibility
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 19
 - BLM model sensibility
- `ExtendedMarketHoursHistoryRegressionAlgorithm` 20 -> 18
 - Less minute bars before market opens

* Addresses Peer-Review

Fix `BacktestingBrokerageRegressionAlgorithm` to use `CalculateOrderQuantity` and round down `quantity` to an even number to pass a value assertion and update the expected value from 50 to 52.
The quantity calculated by `CalculateOrderQuantity` has changed from 50 to 53 because of factor file update.
2021-04-19 13:31:01 -03:00
Colton Sellers 395c1123da Remove Obsolete QCAlgorithm.OnEndOfDay() (#5441)
* Remove and replace OnEndOfDay() ref

* Restore functionality of obsolete EOD, waiting for deprecation in August 2021

* Cleanup

* Adjustments to only post message when using obsolete EOD

* nit, extra space

* Address review

* Adjust test to reflect new behaviour

* Move GetPythonArgCount to an extension method

* Add unit test

* nit accidental import

* Refactor broken test

* Use Py.GIL() state for extension
2021-04-07 13:39:49 -03:00
Colton Sellers 87b42f6fb5 Named Args Unit Tests (#5381)
Add regression and unit test
2021-03-16 16:53:02 -07:00
Colton Sellers 834d4a4d58 QC PythonNet 2.0 (#5376)
* Update to new QC PythonNet

* Update readme

* Remove Python.Runtime config, replaced by env var

* Allow local packages testing in repo

* Address Review

* Add the LocalPackages readme

* Update Jupyter Image

* Update Research ReadMe
2021-03-09 18:25:31 -03:00
Gerardo Salazar e2964dd4b1 Make OrderListHash deterministic by using MD5 as its underlying hash function (#5276)
* Update OrderListHash to use MD5 as hash instead of hash code

* Update regression algorithm OrderListHash statistic

* Use full MD5 hash as OrderListHash, update regression statistic

* Fixes failing regression tests
2021-02-09 12:19:25 -03:00
Martin-Molinero 3ccf428498 Adjust delisting liquidation time (#5203)
* Adjust delisting liquidation time

- Adjust delisting liquidation time to 15 min before market closes.
  Adding unit tests. Updating existing.
- Handle `Statistics.CompoundingAnnualPerformance` invalid calculation
  to avoid exception.
- AlgorithmManager will not handle delisting events in live trading
- Fix bug where due to a split driven liquidation matching delisting
  date a position in the option would remain open. Reproduced by
  `BasicTemplateOptionsFrameworkAlgorithm`

* Address review

- Address review add documentation on delisting offset span
2021-01-22 14:41:29 -03:00
Martin-Molinero 592d037085 Fix delisted liquidation orders being cancelled (#5169)
* Fix delisted liquidation orders being cancelled

- Place delisted liquidation orders 10 min before market closes of 10
  min before the end of the delisting warning date. Adding regression
  test and unit tests. Updating existing tests.

* Fix failing python option unit tests

* Fix bug where positions would be open delisting liquidation

* Fix universe selection and delisting

- Delisting will happen ASAP for all types. Giving priority to close
  positions on derivates first
- Fix bug in universe selection where OptionChain would remove
  underlying even if holding a position in derivate.
- Updating regression tests statistics

* Add unit test, fix unit test expected stats
2021-01-18 22:02:40 -03:00
Colton Sellers fc81f606e4 Reserve Names for Pandas Mapper (#4978)
* Fix for #4886 and unit test

* Fix test function

* Address review

* Clarify comments
2020-12-11 16:39:13 -03:00
Michael Handschuh cf9b547e2e Refactor OptionExerciseOrder.Quantity to be consistent with other Order types (#4796)
* Improve information tracked in regression's {algorithm}.{lang}.details.log

The details.log file aims at providing a diff-able document that quickly and
easily provides actionable information. Since many regression algorithms use
the algorithm's debug/error messaging facilities to log various pieces of algo
state. This document also support a configuration option: regression-high-fidelity-logging'
that logs EVERY piece of data, again, with the aim of providing an easily diff-able
documenbt to quickly highlight actionable information. I may have missed omse key
pieces of information here, but now that the entire QC knows about this regression
tool, if additional information is required then hopefully it's easy enough at this
point to extend the RegressionResultHandler to suit our needs.

The RegressionResultHandler was initially implemented to provide a concise log of
all orders. This was achieved by simply using the Order.ToString method. While
testing/investigating OptionExerciseOrder behavior, it became evident that more
information was required to properly identify the source of potential failures or
differences between previous regression test runs. This change adds logging for
almost every IResultHandler method and additionally attempts to capture the
actual portfolio impact of every OrderEvent. This is accomplished by logging
the portfolio's TotalPortfolioValue, Cash properties and the security's
SecurityHolding.Quantity property.

This change also standardizes the timestamps used to folloow the ISO-8601 format.

When using the RegressionResultHandler, it is highly recommeded to also disable
'forward-console-message' configuration option to ensure algorithm Debug/Error
message logging is done synchronously to ensure correct ordering with respect to
log messages via Log.Debug/Trace/Error.

* Fix typo in options OrderTests test case name

* Update SymbolRepresentation.GenerationOptionTickerOSI to extension method

Far more convenient as an extension method

* Improve R# default code formatting rules

Many of these rule changes focus on improving the readability of code,
with a particular emphasis on multi-line constructs, chained method calls
and multi-line method invocations/declarations.

* Add braces, use string interpolation and limit long lines

* Refactor OptionExerciseOrder.Quantity to indicate change in #contracts

For all other order types, the Order.Quantity indicates the change in the algorithm's
holdings upon order execution for the order's symbol. For OptionExerciseOrder, this
convention was broken. It appears as though only exercise was initially implemented,
in which case only long positions were supported and a code comment indicated that
only positive values of quantity were acceptable, indicating the number of contracts
to exercise. At a later date, assignment simulation was added and utilized a negative
order quantity. This caused some major inconsistencies in how models view exercise
orders compared to all other order types. This change brings OptionExerciseOrder.Quantity
into alignment with the other order types by making it represent the change in holdings
quantity upon order execution.

This change was originally going to be much larger, but in order to minimize risks and to
make for an easier review experience, the additional changes will be committed separately
and pushed in their own PR. Some of the issues identified include:
* Manual Exercise (especially for OTM) is not covered
* Margin Calculations (in particular taking into account opposing contracts held)
* IBrokerage.OptionPositionAssigned is raised for exercise (later filtered by tx handler)

Fixes OptionPortfolioModelTests to use exercise model to properly model exercise of
non-account quote currency option contract.
2020-10-08 20:05:20 -03:00
Michael Handschuh c7a74306fb Bug 4731 Fix Option Expiration Order Tags and Order Event IsAssignment Flag (#4743)
* Add OrderRight.GetExerciseDirection(isShort) extension

Returns the OrderDirection resulting from exercise/assignment of a particular
option right

See: BUG #4731

* Fix option exercise/assignment order tags and order event messages

The algorithm manager was doing work to determine whether or not the option ended
in exercise or assignment at expiration. This decision should be left for the exercise
model to decide -- from the algorithm manager's perspective, all that matters is that
the option was expired. The DefaultExerciseModel was updated to properly track whether
the option expired with automatic assignment or exercise, dependending on whether or
not we wrote or bought the option (held liability or right, respectively). Updated unit
tests to check for order event counts and order event messages for option exercise cases.

Fixes: #4731

* Fix typo in algorithm documentation

* Update regression tests order hash

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-09-28 15:29:09 -03:00
aarjaneiro 9916a9069c Update PythonPackagesTests.cs (#4673)
* Update PythonPackagesTests.cs

Detecting issues related to https://stackoverflow.com/questions/56957512/pythonnet-missing-addreference-method, which has shown up for me when using lean (https://www.quantconnect.com/forum/discussion/9054/lean-vagrant-box/p1).

* Update PythonPackagesTests.cs

Named test

* Name change of MonoTest to SanityClrInstallation 

Name suggestion by @Martin-Molinero

* Fix minor typo

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2020-09-10 16:34:37 -03:00
Colton Sellers f135fb8060 Bug Backtesting Brokerage Clones (#4644)
* fix order updates

* Fix option exercise issue

* Regression changes

* Update regressions to reflect fixes

* Refactor handling of order to fully fix #2846

* Regression Algorithm for unit test

* Pre review

* Fix breaking tests

* OrderImmutability Regression Algo

* OrderImmutability Regression Algo Compile

* Address review

* Update regressions with new orderhash
2020-09-09 19:44:56 -03:00