278 Commits

Author SHA1 Message Date
Ricardo Andrés Marino Rojas eeb512fac6 Fix RenkoConsolidator (#7317)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix bug and add unit tests

The bug was raised because rounding the first point inserted in the
RenkoConsolidator didn't get the closest y-multiple of the given number
x, where y is the bar-size. In order to compute the first y-multiple of
the given number x, we need to first compute x mod y := x - y*Floor(x/y),
once we get this number we divide it by y to know if we should round x
to minimum y-multiple greater than x, or should round x to the maximum
y-multiple smaller than x. If (x mod y)/y < 0.5 we round down, otherwise we round up (the amount (x mod y)/y is always less than one and higher than 0). See the Art of Computer Programming, Vol I. Donald E. Knuth.

- Add `GetClosestMultiple()` method in `RenkoConsolidator.cs`
- Add more unit tests

* Nit change

* Add XML docs

* Address required changes and add more unit tests
2023-06-16 18:36:56 -03:00
Ricardo Andrés Marino Rojas 3e86712c1a Implement ShortableProviderPythonWrapper.cs (#7209)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Implement ShortableProviderPythonWrapper.cs

- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done

* Implement ShortableProviderPythonWrapper.cs

- Modify AllShortableSymbolsCoarseSelectionRegressionAlgorithm.cs and ShortableProviderOrdersRejectedRegressionAlgorithm.cs to use ShortableProvider from Security and not from the Brokerage model
- Add SetShortableProvider() overload method in Security.cs to set a custom shortable provider from Python
- Remove AllShortableSymbols() method from LocalDiskShortableProvider.cs
- Remove DefaultShortableProvider class
- Add regresion algorithms in C# to cover the changes done

* Solve bugs and nit change

* Address review

---------

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2023-04-25 19:55:44 -03:00
Jhonathan Abreu fbf8ffd924 Handle price discontinuity on volatility models (#7058)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Apply splits and dividends to volatility models

* Apply splits and dividends to volatility models using history requests

* Add new ScaleRaw data normalization mode

Handling the new mode in the price scale enumerator.

* DataNormalizationMode.ScaledRaw history requests

* Minor changes

* Minor changes

* Disable new normalization mode in AddSecurity methods and other minor changes

* Peer review

* Minor changes

* Peer review

* Minor changes

* Peer review

* Peer review

* Peer review

* Add scaled raw history regression algorithm

* Add more regression algorithms

* Add more regression algorithms

* Add Slice.TryGet unit tests

* Peer review

* Peer review

* Peer review

* Peer review

* Peer review

* Update algorithms stats

* Peer review

* Peer review
2023-03-13 13:11:30 -03:00
Martin-Molinero bbbab6d9a8 Refactor alpha statistics phase I (#7055)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics

- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
  algorithms

* Optimization backtest result json converter update

* Address reviews

- Remove IAlphaHandler, move insight storage responsability to IResultHandler
  and centralizing insight collection on the QCAlgorithm.Insights to be
  reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
  sampling handling. Updating regression algorithms

* Add InsightCollection tests and minor fixes

* Adding more & improved tests
2023-03-10 13:12:23 -03:00
Martin-Molinero b9e6781c96 Add missing exchanges (#7011)
* Add missing exchanges

- Add missing exchange definitions

* Fix and add more tests
2023-02-24 16:31:51 -03:00
Louis Szeto 8730eeb8a9 Volume Renko Consolidator & Bar (#6888)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Volume Renko Consolidator & Bar

* address peer review

* Minor fix on tests

---------

Co-authored-by: LouisSzeto <hke0073@hotmail.com>
2023-02-09 10:46:02 -03:00
Martin-Molinero f12e8fc08e CryptoFutures adjustments (#6836)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* CryptoFutures adjustments

- Address reviews of CryptoFutures:
- Add new Slice MarginInterestRates collection
- Add new regression tests asserting funding rate application behaves
  the same no matter the resolution
- Add auxiliary data by type into the security cache
- Revert BuyingPowerModel changes

* Make interest rate application time deterministic
2023-01-05 18:07:01 -03:00
Martin-Molinero ee683933a3 Feature Perpetual Crypto Futures (#6807)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* WIP

* Add base currency cash

* Symbol properties and data processing

* Add basic template algorithm

* Add hourly crypto future algorithm

* Minor fixes after live trading testing

* CoinApiDataQueueHandler CryptoFuture support

* Address reviews

* Fix regression algorithms after update
2022-12-30 13:58:11 -03:00
Jhonathan Abreu 31f8561330 Allow optional selector in classic renko consolidator in Python and C# (#6755)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2022-11-18 17:14:03 -03:00
Louis Szeto b34cea4458 Calculate Option Greeks with Implied Volatility (#6720)
* Using IV to calculate Greeks, and remove single-step stochastic approximation

* Correct calculation for theta, vega, rho

* Add calculation from Black Calculator and peer review

* Address peer review and added unit test

* Update some tests and correct vega/eho

* Fix Unit Test and Improve Comments

Fixes `IndexOptionCallITMGreeksExpiryRegressionAlgorithm` since `Vega` was really non-zero.

* Fix regression test and add IV calculation

* refactor and bug fixing on peer review

* refactor and bug fixing on peer review

* for rerun test

* add warning on IV estimation not coveraged and edit speed unit test to not exceed 2s per 1000 iteration

* update logging

* update logging and description

* Add default option pricing models and unit tests

* address review

* Added Fed interest rate as risk-free rate with unit tests and set as default for option greeks calculation, added regression algorithms, addressed peer review

* refactor structure of interest rate

* Skip Saturday and Sunday

* regression test fix

* peer review

* Fixes Interest Rate Provider Logic

* Minor tweaks

* Fix start date

* Minor test tweak

* Update interest rates

* Fix unit tests

* Add minor log

Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-11-17 13:35:42 -03:00
Martin-Molinero fc6835cace Normalize time in universe behavior (#6693)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Normalize time in universe behavior

- Normalize Option & Future chain universe behavior regarding their
  assets time in universe. They will now respect the universe settings
  time in universe value. Adding new regression algorithms asserting the
  behavior

* Address reviews & cleanup
2022-10-14 15:13:59 -03:00
Martin-Molinero b899aa4767 Make UserDefinedUniverse thread safe (#6682)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Make UserDefinedUniverse thread safe. Adding regression test
2022-10-12 15:06:01 -03:00
Alexandre Catarino f755b91158 Fixes PythonData.EndTime (#6602)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Fixes PythonData.EndTime

The `EndTime` property of `PythonData` didn't override the base implementation in `BaseData`. Therefore `EndTime` was always set as `Time`.

Fixes Regression Tests
- The C# version of `Bitcoin` class needs to implement `EndTime`-
- Fixes regression tests

* Minor tweaks and rebase

- Add support for python data types just setting EndTime. Adding unit
  tests for PythonData
- Fix for DynamicData EndTime property being fetched. Updating unit
  tests

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-09-07 14:53:05 -03:00
Martin-Molinero c43f472503 Fill model will use internal configurations (#6494)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fill model will take into account internal configurations, this is
  specially relevant for continuous future mapped contracts. Updating
  regression algorithms and unit tests.
- Allowing internal configurations to update trade builder information
2022-07-19 19:00:35 -03:00
Martin-Molinero 5c6a779f4d Period consolidator adjusments (#6444)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Replace logic to convert TimeSpan based PeriodConsolidator into count
  based, by an override of the 'potentialStartTime' in the case we could
  be falling into a look ahead consolidated bar end time
2022-06-27 19:17:20 -03:00
Martin-Molinero 4c3afa6ff1 Add OpenInterestAnnual (#6436)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Adding OpenInterestAnnual where the contract maps when any of the back month
  contracts of the next year have a higher volume that the current front month.
- Add support for backwards compatible data mapping mode additions
- Updating ES map and factor files. Adding a little daily data too
2022-06-23 19:22:24 -03:00
Martin-Molinero bb0c27fefd Period timespan consolidation improvements (#6408)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Period timespan consolidation improvements

- If user is trying to consolidate a period providing data of a bigger
  period we will now throw an exception. Adding tests
- If both consolidated and given data share the same period, gently
  adjust the consolidator into a data count of 1. Adding tests
- Fixing bug in QuoteBarConsolidator period double accounting. Adding unit tests

* Add Period and Count regression algorithm
2022-06-20 15:14:44 -03:00
Martin-Molinero 740b40ff11 Fix QuoteBar Non Time Based Consolidator Period (#6409)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix QuoteBar NonTimeBased Consolidator Period

- Fix QuoteBar non timebased consolidator period, that was accounting
  twice for the initial bar period. Updating unit tests

* Improve QuoteBarConsolidator assertion

- Assert quote bar consolidated time and endtime in unit tests
2022-06-17 19:44:37 -03:00
Martin-Molinero b007d9f962 Fix intermittent ZipMapFile cache test failure (#6406)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix for intermittent ZipMapFile cache test failure and reducing test length
2022-06-16 15:30:20 -03:00
Martin-Molinero 4b86c78df0 Add unit test showcasing feature (#6339)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-05-13 20:21:08 -03:00
Martin-Molinero b9d3d99917 Bump pythonNet version 2.0.12 (#6310)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Updates after pythonNet rebase

* Bump pythonNet version 2.0.12

* Fix exception types being thrown
2022-05-02 14:38:50 -03:00
quantify-cflynn a374ce9b6e Update Tick.cs Time parsing (#6280)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Update Tick.cs Time parsing

* Adjust equity tick millisecond read

* Fix tick sub millisecond precision

- DateTime.AddMilliseconds will rount to the nearest integer. So instead
  use AddTicks

* Avoid sporadic test failure

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-04-04 17:36:57 -03:00
Ronit Jain f86926bf7a Fixes HistoryProviderManager slice merging by using algortime for all slices (#6226)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* remove un-used

* use utc time for slice sync

* use utc time

* refactor

* add regression test

* use utc time

* use utc time

* make utctime required parameter

* add utcTime in slice creation

* assert warm up complete

* check if algorithm is still warmingup

* use exchange tz
2022-02-18 20:43:33 -03:00
Ronit Jain 2d644d7879 Feature handle multiple history providers (#6187)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* initial commit

* get history from data providers

* merge history from history providers

* merge slice

* Add merge function in slice

* Update test

* merge aux data

* add data points in _data

* append new data points in original list

* Add test suite for HistoryProviderManager

* reduce complexity

* setup once

* add fake history provider

* doesn't count aux data

* add tests for options

* style changes

* add custom data

* merge rawDataList

* use array of history providers

* optimize

* fix formatting error

* add comment

* add tests

* simplify

* use list

* use abstraction

* split tests

* add test

* use abstraction to create generic enumertor class

* refactor

* accept list of type T

* sync history slices

* use SubscriptionDataReaderHistoryProvider for live

* rename test file

* return empty

* add tests

* cleanup

* address reviews

* optimize

* Fix method definition

* use initial time for basedata

* refactor

* re-use collection

* inherit HistoryProviderBase

* always return HistoryProviderManager

* add tests

* update rawDataList

* reset composer

* consider null elements

* add tests for binary search method

* Follow lean coding style

* revert

* remove binary search method

* convert to field
2022-02-14 11:58:00 -03:00
Martin-Molinero c8dc343c13 GetLastKnownPrices python data (#6191)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adding unit tests reproducing issue.

* Fix a couple of minor bugs

- IsMarketOpen will work correctly when used with daily and hourly
  resolution.
- slice.Get will work correctly with python custom data
- ExtendedDictionary will be able to dinamically access methods,
  required for python and private C# data types

* Refactor solution. Add more tests

* Remove unrequired import statement
2022-02-08 15:37:05 -03:00
Martin-Molinero 459f60603b Fees is base currency subtracted from quote currency (#6188)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP

* Fix fees in base currency not being subtracted

- Fix fees in base currency not being subtracted from the quote currency
  for crypto cash accounts. Updating regression tests to assert
  portfolio, cashbook state and holdings state.

* Fix unit test race condition
2022-02-07 14:57:40 -03:00
Ricardo Andrés Marino Rojas 472f78cc53 Remove Quandl from LEAN (#6110)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Remove Quandl from LEAN

* Nit changes and CustomLiveDataFeedTests.cs

* Resolve conflicts

* Remove files related with Quandl

* Fix bug

* Fix QuantBookHistoryTests.cs

* Fix bug

* Fix bug

* Fix unit tests

* Try fix regression tests

* Nit changes

* Fix bug

* Some of the requested changes

* The missing changes

* Requested changes

* Nit changes

* Revert "Nit changes"

This reverts commit 9800bc5c34f3ac20e30bea7a92dd4a9867213bb5.

* Nit changes

* Fix bug

* Requested changes

* Missing file using Quandl to be removed

* Nit changes

* Not applied nit change

* Nit change

* Nit change

* Add nasdaq-auth-code parameter in config.json

* Remove 'quandl-auth-token' from config.json

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-01-12 12:10:20 -03:00
Martin-Molinero 68e2a9170a Do not send internal SecurityChanges to Algorithm (#6118)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Do not send internal SecurityChanges to Algorithm

- Will not send internal security changes to the Algorithm by default.
  Following custom security changes filter pattern. Updating regression
  algorithms to assert behavior.
- The universe member will know wether it was added with internal
  configurations or not

* Address reviews use a separate collection for internals

* Refactor solution. Adding security changes constructor class
2021-12-21 20:24:32 -03:00
Martin-Molinero fed1fa929b Fix low resolution index TradeBar parsing (#6081)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Fix low resolution Index TradeBar parsing. Updating unit tests. Adding
  unit tests
2021-11-29 12:59:16 -03:00
Colton Sellers 33599b473d Refactor Delistings Processing (#6059)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Move processing of delistings to Brokerage

* Deal with case that exchange is not open on OptionSymbol.ID.Date

* Refactor solution to use DelistingNotification event

* Adjust some regression expected liquidation time

* Mark some todos on deprecated functions

* Update expected liqudation time for Py regressions

* Update regressions that have been validated

* Use HandlePositionAssigned for assignment orders

* Update regressions

* Update some missed unit tests; remove one that is already covered by regression

* Cleanup deprecated backend functions

* nit - small cleanup adjustment

* Post rebase fix

* Address review

* Minor tweak to py regression
2021-11-17 17:43:35 -03:00
Martin-Molinero b8b0d18993 Continuous futures (#6034)
* Continuous Future Contracts

* Mapping approach

* Tweaks WIP

* Live mapping

* Live mapping

- Add support for live mapping, refreshing mapfiles
- Fix future expiration functions
- Adding unit tests

* Update moq test package

* Continuous futures price scaling

* Refactor price factors scaling

* Factor file related renames

* Address reviews
2021-11-15 14:44:30 -03:00
Martin-Molinero 4e6e0c85c6 Lean exchange improvements (#5932)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Lean exchanges improvements

- Adding new Exchange class to avoid exchange code clash.
  Adding and updating unit test

* Add Market for MapFile API

* Self review
2021-09-20 13:50:15 -03:00
Colton Sellers a2ac956d53 Move WickedRenkoConsolidator to RenkoConsolidator as Default (#5931)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Rename Renko to ClassicRenko

* Rename WickedRenko to Renko

* Adjust tests and usages

* Backwards support WickedRenkoConsolidator

* Renames for consistency
2021-09-17 16:32:01 -03:00
Martin-Molinero bf28a1d313 Add basic template Atreyu algorithm (#5872)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add basic template Atreyu algorithm

- Add C# and Python basic template atreyu algorithm. Show casing how to
  specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use

* Lean Exchange improvements

- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value

* Regression tests fixes
2021-08-23 18:24:44 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
2021-08-02 19:24:45 -03:00
Alexandre Catarino 5361f87dd1 Updates Equity Market Data (#5576)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data

* Updates Unit Tests

* Updates Regression Tests

In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.

* Updates Regression Tests 2

The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238

Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
 - BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`

The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying

* Revert File Update for SPWR and SPWRA

* Fix Regression Tests

Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.

* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]

Uses `trust-constr`  method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
2021-06-17 14:04:51 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
2021-06-15 19:06:06 -03:00
Alexandre Catarino c4e6e7f8dc Changes VIXCentralContango.F9 to Nullable (#5662)
Build & Test Lean / build (push) Has been cancelled
According to the data, the contract 8 months away from the front-month contract, `F9`, can be null.
2021-06-14 16:23:28 -07:00
Gerardo Salazar 250ffa1fec Adds VIXCentralContango data, modifies cached alternative data sourcing in live mode (#5648)
Build & Test Lean / build (push) Has been cancelled
* Adds VIXCentral Contango data

  * Refactor cached alternative data in live mode to
    use web endpoint if the data is not available locally

* Remove fallback to cache bucket for CBOE/FRED/EIA/VIXCentralContango

* Address review: force VIX as ticker for all VIXCentralContango data
2021-06-09 20:44:53 -03:00
Colton Sellers ab6027723f Refactor Api (#5251)
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request

* key -> filePath rename and some cleanup

* Refactor

* Add Organizations Endpoints

* Add some organization api wrapper objects

* Address namespace issue

* Reorganize Api Test into seperate files using one ApiTestBase

* Add Organization tests

* Use capitalized "API" test namespace to reduce amount of file changes

* Add License to test base

* Update /data endpoint functions and response objects

* Update ApiDataProvider Logic

* Handle deserialization of organization products

* Simplify converter

* Only throw for equity requests when not subscribed to map/factor files

* Add missing header

* Make arguement exception

* Api adjustments

* Add Zip factor and map file providers

- Common project will now reference Compression project and not the other way
  round.
- Adding Zip FactorFile and MapFile providers

* Refactor FactorFileProvider to use DataProvider to fetch files

* Use resulting MinimumDate in construction of FactorFile

* Nit FactorFile comments and arrangement

* Refactor MapFileProviders to use DataProvider for fetching files

* Refactor ZipFileProvider

* Clean up

* Refactor Backtesting Future/Option chain providers to use dataprovider

* Fixes for data/ endpoints and test adjustments

* Response objects adjustments/cleanups

* ApiDateProvider fixes and testing

* Add LocalZipFactorFileTests

* Update ApiDataProvider download test to verify stream is not null

* Implement posting of agreement summary and signed time

* Mark all Api related tests as explicit and document details on running

* Clarify default token on ApiTestBase

* Adjust summary

* Update Api responses for QCC, except org products which are sold in USD

* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests

* Fix multiple markets for ZipFactorFile provider

* Use Symbol as cache key

* Api.cs review

* Dispose of factorFileStream after reading

* Use zip.EntryFileNames

* Address a few reviews

* Few more fixes

* Address Api Review

* Add Job Org id to config

* Minor tweaks

* Compare with invariant culture

* Fixes Option Universe selection

* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider

* Fix research

* Fix null reference exception

* Make duplicate log debug

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-06-07 11:37:44 -03:00
Colton Sellers 8ee9432bda Wicked Renko Consolidator Fix (#5589)
Build & Test Lean / build (push) Has been cancelled
* Round first value to Renko BarSize

* Add unit tests
2021-05-24 21:39:21 -03:00
Colton Sellers 865da14540 Scheduled Universe Timed Triggers Bug (#5521)
Build & Test Lean / build (push) Has been cancelled
* Add unit test

* Make test cover issue case

* Filter out times before startTimeUtc

* Refactor solution, fixes missing first date

* Fix case where none is expected

* Cleanup tests, add TriggerTimesNone

* Drop unused imports

* Add missing dipose call

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-05 11:23:38 -03:00
Colton Sellers 3579fecc58 Live Consolidator Bug Fix (#5429)
* Set _lastEmit before emitting, otherwise _workingBar is always null

* Aggregate bars if the data endTime is past lastEmit

* Add unit test

* Address Review

* Clean up unit tests

* Refactor solution to set consistent _lastEmit behaviour

* Add another unit test

* Make fixture non-parallelizable

* Undo last change, and adjust breaking test directly
2021-04-05 17:05:53 -07:00
Colton Sellers 1c3d849ad5 Fix Warnings V2 (#5436)
* Reconcile duplicated code

* Add License header

* CS0219 Fixes: Value assigned, but never used

* CA1507: Use nameof in place of string literals

* CS0108 : Hides Inherited Member; Use new keyword to overwrite formally

* CS0114: Hides inherited member; use override keyword

* CS0168: Variable is declared but never used

* Tests CS1062; using obsolete implicit Symbol -> String; fix via .ToString()

* CS0472: Non Nullable Obj getting Null Checked

* CS0067 Member not used; ignore all cases for future use

* CS00162 : Unreachable code; either removed or ignored for debugging and test cases

* CS0169 Remove non-used fields; ignore those that may be used in future

* CS0414; Field is assigned but never used.

* CS0618; Obsolete properties and members; Only fixes simple ones, rest will have to broken up

* CS0649; Field never assigned too

* CS0659 & CS0661 ; Overwrite operators and equals but not hashcode; I don't really override it but just call base

* Small comment fix

* Cleanup pragma statement
2021-04-02 11:20:01 -07:00
Gerardo Salazar bc4dbf5707 Removes extra day added to CBOE Time causing data to shift one day forward (#5266)
* Added test
2021-02-05 19:42:01 -03:00
Martin-Molinero 3ccf428498 Adjust delisting liquidation time (#5203)
* Adjust delisting liquidation time

- Adjust delisting liquidation time to 15 min before market closes.
  Adding unit tests. Updating existing.
- Handle `Statistics.CompoundingAnnualPerformance` invalid calculation
  to avoid exception.
- AlgorithmManager will not handle delisting events in live trading
- Fix bug where due to a split driven liquidation matching delisting
  date a position in the option would remain open. Reproduced by
  `BasicTemplateOptionsFrameworkAlgorithm`

* Address review

- Address review add documentation on delisting offset span
2021-01-22 14:41:29 -03:00
Juan José D'Ambrosio 765f2cab3b Add primary exchage to map files (#5194)
* Add main exchange to map file

minimum minimorum test

* Add MainExchange to MapFilRow IEquatable implementation
Also write CSV line correctly

* Normalize exchanges

Also add small test form MapFileRow

* Exchange name normalization

- Exchange name normalization.
- Tick will protobuf `ExchangeCode` which will set existing `Exchange`

* Include single character exchange code

* Encode exchange as byte, Include exchange name as Enum

* Update MapFileRow.cs to accept a byte instead a string for primary exchange

* Fix issues in encoding exchanges map

* Fix broken MapFile tests

* Rename Main Exchange property in MapFileRow

* Use BATS instead of BATSZ

* Add primary exchange from map file

* Move MapFilePrimaryExchangeProvider to its own file

* Implement Char for primary exchange in map files

* Fix broken tests after implementing char exchange into map file

* Add a new constructor for MapFileRow

it  accepts a PrimaryExxchange object as parameter

* Fix broken test 

WIP

* Address review 

WIP

* Handles unexpected encoded exchanges

* Keep addressing review

* Address review

* Fix broken tests

Make PrimaryExchange as default even if the IPrimaryExchangeProvider is not instantiated

* Revert breaking change.

* Add a check that resolved map file actually have content

The issue that leads to this change was caused by a map file resolved to APC in the CoarseFundamentalTop3Algorithm. 
The resolver returns a not-null MapFile, but it is empty.
To my best understand, the MapFile Resolver should return only existing map files in the map_files folder. But the LocalMapFileProvider returned a map file with a permtick that doesn't exist in the file system as CSV.

* Set PrimaryExchange.UNKNOWN as default

* Address review couple minor bugs

- Use SID market instead of USA
- Default ticke exchange is UNKNOWN

* Update map files with latest format and latest info

This include:
 - VXX, it was delisted, then VXXB appears and later it chnages from VXXB -> VXX
 - AAA delisting and a new listing with ticker AAA
 - FOXA ticker changue to TFCFA and later it was delisted

* Update test after updating latest map files

* Fix bug with factor files after updating map files

* Fix bug with factor files after updating map files

and revert statistics in regression tests

* Update OrderHash in regression test

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2021-01-21 18:46:12 -03:00
Aaron Janeiro Stone 7aefe07274 4553 - Refactor RenkoConsolidator (#5052)
* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Removes base class for brevity

* Delete BaseRenkoConsolidator.cs

* Fixes naming for classic Renko consolidator

* Addresses review concerns

* Fixes missing null checks and missing typesafe constructor

* Adds warning

* WickedRenkoConsolidator -> RenkoConsolidator calls

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Removes base class for brevity

* Fixes naming for classic Renko consolidator

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Delete BaseRenkoConsolidator.cs

* Addresses review concerns

* Fixes missing null checks and missing typesafe constructor

* Adds warning

* WickedRenkoConsolidator -> RenkoConsolidator calls

* Addresses review

* unneeded tag removed

* Minor tweaks address reviews

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2021-01-18 15:54:15 -03:00
Gerardo Salazar 3e7af17e02 Implements Easy To Borrow Reality Modeling (WIP) (#5159)
* Creates IShortableProvider interface + interface impls

  * This is the foundational work for the addition of the shortable stocks
    feature for backtesting and live trading. Note that the QCAlgorithm
    API and the backend transaction handling will be implemented
    separately.

* temp; work on preorder checks

* improve checks

* tmep

* Enforces ETB checks at BrokerageTransactionHandler

  * Adds ETB Shortable Provider to QCAlgorithm and IAlgorithm
  * Removes ETB check from PreOrderChecksImpl
  * Removes outdated test, new test to come soon for relevant class

* Work in progress commit, pushing for review.

  * Adds new regression algorithm (WIP)
  * Enhances ShortableProviderOrdersRejectedRegressionAlgorithm
  * Adds new methods to QCAlgorithm
  * Move IShortableProvider to have BrokerageModel own it
  * Comments updates
  * Adds new properties to Equity Security for shortable

  * Fixes bug where retrieving open order quantities would aggregate the
    submitted order quantity rather than the remaining order quantity for
    open orders.

  * Fixes bug where quantity of zero would result in a
    false positive in QCAlgorithm.Shortable(...)

  * Code refactoring and comments updates

Co-authored-by: Jared <jaredbroad@gmail.com>
2021-01-14 13:33:44 -08:00
Colton Sellers 4006ba01e4 Reduce Travis Log (#5039)
* Reduce Travis setup verbosity

* Introduce ConsoleErrorLogHandler

* Change Console.WriteLine to Log statements

* Quiet wget

* Route build stubs stdout to null

* Fix Quantbook history test

* Silence stub packages directly

* Use parameterized log-handler for testing

* Rename AssemblyInitialize Setup

* Fix AlgorithmRunner file logging

* Drop all overriden LogHandlers in tests

* Change to OneTimeSetup to maintain LogHandlers

* Permit any ILogHandler to be defined in params

* Fix for AlgorithmRunner Handlers V2
2020-12-17 12:59:13 -03:00