Commit Graph

17 Commits

Author SHA1 Message Date
Martin-Molinero cb326788b3 Filter out small orders based on Setting (#5776)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Filter out small orders based on Setting

- BuyingPowerModel will filter out small orders based on algorithm
  setting, a % of PTV, instead of hard coded 1 share value. Addin unit
  and regression tests
- Updating regression algorithms to use new setting, reduce order trades

* Update regression algorithms
2021-07-19 13:17:51 -03:00
Alexandre Catarino 5361f87dd1 Updates Equity Market Data (#5576)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data

* Updates Unit Tests

* Updates Regression Tests

In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.

* Updates Regression Tests 2

The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238

Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
 - BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`

The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying

* Revert File Update for SPWR and SPWRA

* Fix Regression Tests

Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.

* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]

Uses `trust-constr`  method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
2021-06-17 14:04:51 -03:00
Colton Sellers 5bf72d2432 Adjust Volume for Splits (#5525)
Build & Test Lean / build (push) Has been cancelled
* Adjust TradeBar volume for splits

* Refactor solution to centralized function `Scale`

* Expand BaseData scale/adjust/normalize tests to assert volume behavior

* Add regression algorithm

* Update regression statistics

* Address review

* NOP to trigger cloud build

* adjust QuoteBar ask and bid size

* adjust broken regression

* Update tests to include QuoteBar adjustments

* nit cleanup

* Adjust Quote Tick bid and ask sizes

* Round volume and size values to nearest int in scale()

* nit

* Adjust regressions
2021-05-17 16:47:44 -03:00
Alexandre Catarino 58793acae8 Removes Assets that Are Not Invested and Not Tradable From Capacity Calculations (#5515)
Build & Test Lean / build (push) Has been cancelled
* Add property for capacity. Remove unused variable

* Move SymbolCapacity and CapacityEstimate to common, passed through Symbol to runtime statistics

* Add null checks

* Remove uninvested and untradable assets from capacty calculations

* Add SymbolCapacity influential period

* Updates Regression Tests

- DelistingEventsAlgorithm
  - Allows additional contributions from delisted AAA.1

- DelistingFutureOptionRegressionAlgorithm
  - Removes DC01H12 contributions one month later

- FutureOptionBuySellCallIntradayRegressionAlgorithm
  - Allows additional contributions from future after expiry replacing the contribution of the next contract option

- DelistedFutureLiquidateRegressionAlgorithm
- FutureOptionCallITMExpiryRegressionAlgorithm
- FutureOptionCallITMGreeksExpiryRegressionAlgorithm
- FutureOptionPutITMExpiryRegressionAlgorithm
- FutureOptionShortCallITMExpiryRegressionAlgorithm
- FutureOptionShortPutITMExpiryRegressionAlgorithm
- FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm
  - Allows additional contributions from future after expiry

- FutureOptionCallOTMExpiryRegressionAlgorithm
- FutureOptionPutOTMExpiryRegressionAlgorithm
- FutureOptionShortPutOTMExpiryRegressionAlgorithm
- IndexOptionCallITMGreeksExpiryRegressionAlgorithm
- IndexOptionCallOTMExpiryRegressionAlgorithm
- IndexOptionShortCallOTMExpiryRegressionAlgorithm
  - Allows additional contributions from option after expiry

- MACDTrendAlgorithm
  - Removes contribution when SPY is not invested for over one month

- UniverseSelectionRegressionAlgorithm
  - Allows additional contributions from delisted GOOAV replacing GOOG (new symbols)

* Adds Lowest Capacity Asset to Regression Tests

* Normalize expected value -0, because -0 is also written to file if updated

* Write Symbol.Value for lowestCapacitySymbol or  empty string for empty Symbol

* Update Regressions

* Update 'Lowest Capacity Asset' to Symbol.ID

Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
2021-05-13 20:25:19 -03:00
Martin-Molinero f3c386663b Feature .net 5 (#5505)
* Update projects to use .NET 5.0, the successor to .NET Core

* Fix ambiguous errors. Add IBAutomator net5

* Remove FXCM

* Upgrade IBAutomater to v1.0.51

ignored, and an empty message aborts the commit.

* Fix rebase

- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy

* Remove unrequired references

* Fixes

- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests

* Don't call ReadKey when input is redirected

* Fix ConsoleLeanOptimizer

* Research fixes

* Update comment

* Add vsdbg to Dockerfile

* Fixes

- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
   - Option assignment will be deterministic in the order
   - 'Rolling Averaged Population' is calculated using doubles, updating
     expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock

* Replace ICSharp with .NET Interactive

* Fixes after rebase

* CSharp research fixes

- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
2021-05-06 17:23:51 -03:00
Alexandre Catarino 2b0fd2e607 Updates SPY Market Data (#5493)
* Fixes Double to Decimal Cast in GetAnnualPerformance

`GetAnnualPerformance` raises an exception if the `AnnualPerformance` calculation returns a double that cannot be cast to decimal (smaller than `decimal.MinValue` or bigger than `decimal.MaxValue`).
See `ProbabilisticSharpeRatio` where the same solution was applied.

* Updates SPY Market Data

SPY is a key asset since it is the default benchmark, and any change can lead to different `Alpha` and `Beta`

* Updates Unit Tests to Reflect Data Update

* Updates Regression Tests to Reflect Data Update I

Most of the regression tests change because of updated data (market and factors) of SPY (default benchmark) while the total trade remain the same.

* Updates Regression Tests to Reflect Data Update II

The following regression tests were changed to adapt to adjusted prices and keep the total trades:
- `BacktestingBrokerageRegressionAlgorithm`
- `LimitIfTouchedRegressionAlgorithm`
- `PortfolioRebalanceOnCustomFuncRegressionAlgorithm`
- `SetAccountCurrencySecurityMarginModelRegressionAlgorithm`
- `StopLossOnOrderEventRegressionAlgorithm`
- `TimeInForceAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `FreePortfolioValueRegressionAlgorithm` 2 -> 3
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 291 -> 298
- `TrailingStopRiskFrameworkAlgorithm` 5 -> 7

Especial cases:
- `AutoRegressiveIntegratedMovingAverageRegressionAlgorithm` 65 -> 52
 - ARIMA model sensibility
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 19
 - BLM model sensibility
- `ExtendedMarketHoursHistoryRegressionAlgorithm` 20 -> 18
 - Less minute bars before market opens

* Addresses Peer-Review

Fix `BacktestingBrokerageRegressionAlgorithm` to use `CalculateOrderQuantity` and round down `quantity` to an even number to pass a value assertion and update the expected value from 50 to 52.
The quantity calculated by `CalculateOrderQuantity` has changed from 50 to 53 because of factor file update.
2021-04-19 13:31:01 -03:00
Gerardo Salazar f9dc38efab Use trades per period instead of days for capacity calculation (#5389)
Use trades per period instead of days for capacity calculation
Updates regression algorithms
2021-03-16 16:30:49 -07:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
2021-03-02 18:46:31 -03:00
Gerardo Salazar e2964dd4b1 Make OrderListHash deterministic by using MD5 as its underlying hash function (#5276)
* Update OrderListHash to use MD5 as hash instead of hash code

* Update regression algorithm OrderListHash statistic

* Use full MD5 hash as OrderListHash, update regression statistic

* Fixes failing regression tests
2021-02-09 12:19:25 -03:00
Martin Molinero 6ad123ad8c Update regression algorithms stats
- Update regression algorithms stats after making SecurityCache ignore
  QuoteBars for equity for OHCL values and GetLastData(). They were
  affected since the `BenchmarkSecurity` used `.Price` which was QB for
  equities. Order list hashes changed because SubmissionLastPrice will
  now be TB instead of QB
2020-04-08 19:31:42 -03:00
Martin Molinero 499248fe12 Revert "Revert "Adding Support for Equity L1 Quote Data""
This reverts commit 8cd8d206ca.
2020-04-06 10:32:59 -03:00
Martin Molinero 486b2e6641 Update regression stats
- Update expected OrderListHash in regression algorithms due to
serialization improvements
2020-03-31 12:21:13 -03:00
Jack Simonson c6767780c2 Update regression test statistics 2020-03-26 18:29:29 -07:00
Jared 8cd8d206ca Revert "Adding Support for Equity L1 Quote Data" 2020-03-11 19:16:05 -07:00
Juan José D'Ambrosio 951e89314a Update Regression tests 2020-03-11 14:34:16 -03:00
Martin Molinero 983d84bf01 Implement PortfolioBias for MVOPCM, BLOPCM, AIPCM
- Implement portfolio bias for
`MeanVarianceOptimizationPortfolioConstructionModel`,
`BlackLittermanOptimizationPortfolioConstructionModel`,
`AccumulativeInsightPortfolioConstructionModel`. Adding unit tests
2020-02-21 12:12:59 -03:00
AlexCatarino c2ff03caad Implements PortfolioBias in EWPCM and IWPCM
Implements `PortfolioBias` in EWPCM, CWPCM and IWPCM. With this new feature, these PCM will ignore insights that do not respect the desired bias. E.g. for `PortfolioBias.Long`, on Insights with `InsightDirection.Up` will be converted into `PortfolioTarget.Quantity` greater than zero and other `InsightDirection` will result in `PortfolioTarget.Quantity` of zero.
2020-02-20 19:29:12 -03:00