10977 Commits

Author SHA1 Message Date
Stefano Raggi 47fb4e64f2 Update IBAutomater to v2.0.37 (#5676)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12005 12007 12009
2021-06-17 16:14:38 -03:00
Alexandre Catarino 5361f87dd1 Updates Equity Market Data (#5576)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data

* Updates Unit Tests

* Updates Regression Tests

In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.

* Updates Regression Tests 2

The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238

Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
 - BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`

The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying

* Revert File Update for SPWR and SPWRA

* Fix Regression Tests

Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.

* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]

Uses `trust-constr`  method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
12003
2021-06-17 14:04:51 -03:00
Jasper van Merle 72fb362f96 Fix futures not working with IQFeed (#5675)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix futures not working with IQFeed

* Trim slash from value instead of using ID.Symbol
12000
2021-06-17 08:46:03 -07:00
Martin-Molinero 24e3a71581 Coinapi improvements (#5672)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Add centralized download extension helper method
- Update coinApi client
11994
2021-06-16 20:30:51 -03:00
Colton Sellers 15555f710d Python Loading and Debugging Refactor (#5670)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use Python algorithms directly in their location; also refactor pythonpath additions to one method

* Update VSCode Launch options

* Drop issue these changes fixes

* Use AddRange
11993
2021-06-16 20:30:33 -03:00
Martin-Molinero 3d13fb6935 Implement Atreyu fees (#5674)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Empty

* Implement Atreyu fees
11992
2021-06-16 19:43:27 -03:00
Colton Sellers 74f0c89499 Option Chain Provider Symbol Map Resolve (#5664)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use mapFileProvider to resolve symbol mapping

* Add reproducing unit test

* Only resolve mapping for types that require it

* address review
11991
2021-06-16 19:42:38 -03:00
Martin-Molinero 0991f089d0 Reduce disk usage in the cloud (#5673)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
11988
2021-06-16 18:43:55 -03:00
Colton Sellers 424fa36703 Organization Product ID and ApiDataProvider Fix (#5671)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add ID to Organization Products

* Use Product ID to verify subscription
11982
2021-06-16 15:01:53 -03:00
Colton Sellers afbd058dad Add Site-Package Issue to Readme (#5668)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add information to Python and Research readme

* Adjust doc
11977
2021-06-16 13:57:13 -03:00
Stefano Raggi 66d2f42d01 IBAutomater updates (#5669)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.36

* Trigger build
11976
2021-06-16 12:52:43 -03:00
Martin-Molinero 11f8dc2958 Improvements for CoinApi subscription (#5666)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
11971 11965 11968
2021-06-15 19:49:14 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
11963 11964
2021-06-15 19:06:06 -03:00
Stefano Raggi c367451c6d Update IBAutomater to v2.0.35 (#5663)
Build & Test Lean / build (push) Has been cancelled
11960 11961
2021-06-15 16:58:31 -03:00
Alexandre Catarino c4e6e7f8dc Changes VIXCentralContango.F9 to Nullable (#5662)
Build & Test Lean / build (push) Has been cancelled
According to the data, the contract 8 months away from the front-month contract, `F9`, can be null.
11953
2021-06-14 16:23:28 -07:00
Colton Sellers ef5f7fbc50 QuantBook Provider and Composer Fix (#5660)
Build & Test Lean / build (push) Has been cancelled
* Use Composer instance for QB so providers are the same across Lean; also initialize map and factorfiles

* Don't need map and file initialize
11951
2021-06-14 19:17:53 -03:00
Colton Sellers 7f9d35f666 Research BollingerBands Value Error (#5656)
Build & Test Lean / build (push) Has been cancelled
* Stop update when properties will fail to update

* Only enforce our case after the BB is ready

* Nit comment fixes

* Add unit test

* Use faster DateTime.UtcNow
11943
2021-06-14 14:23:05 -03:00
Gerardo Salazar be0fc9a879 Fixes case where crypto has short exposure in ReportGenerator in long-only strategies (#5655)
Build & Test Lean / build (push) Has been cancelled
11934 11940
2021-06-11 21:40:49 -03:00
Colton Sellers a882aadeef Research Fixes and Doc Updates (#5649)
Build & Test Lean / build (push) Has been cancelled
* Updates to default notebooks

* Update readme

* Notebook adjustments

* Drop launch script support

* Use our cwd if no QC dlls are in AppDomain directory

* Modify Intialize.csx to use parent directory only if QC files are detected there

* Clean up some details after testing

* Adjustments

* nit, NullOrWhiteSpace
11926
2021-06-11 16:06:47 -03:00
Stefano Raggi 02f7742878 Update IBAutomater to v2.0.34 (#5652)
Build & Test Lean / build (push) Has been cancelled
11925
2021-06-11 14:38:45 -03:00
Martin-Molinero b7c18cbfe7 Update pythonNet to net5 (#5653)
Build & Test Lean / build (push) Has been cancelled
- Csproj clean up legacy build print logs
- Update pythonNet to net5
11922
2021-06-11 13:51:09 -03:00
Martin-Molinero 6e6a1631dc Enable loading crypto live virtual position tracking (#5651)
Build & Test Lean / build (push) Has been cancelled
* Enable loading live virtual position tracking

- Enable GDAX paper trading configurations
- GDAX will optionally load existing holdings from live job package
- Bitfinex will try loading existing holdings from live job package for
  cash account type

* Binance live holdings loading
11911 11918
2021-06-10 20:05:23 -03:00
Gerardo Salazar 250ffa1fec Adds VIXCentralContango data, modifies cached alternative data sourcing in live mode (#5648)
Build & Test Lean / build (push) Has been cancelled
* Adds VIXCentral Contango data

  * Refactor cached alternative data in live mode to
    use web endpoint if the data is not available locally

* Remove fallback to cache bucket for CBOE/FRED/EIA/VIXCentralContango

* Address review: force VIX as ticker for all VIXCentralContango data
11908
2021-06-09 20:44:53 -03:00
Martin-Molinero 7e2678d0f6 Normalize live holdings initialization (#5647)
Build & Test Lean / build (push) Has been cancelled
* Normalize live holdings initialization

* PaperBrokerage config based live state recovery

- Increase algorithm status storing sampling period
- Fix bug where duplicate cash entries could happen
11905
2021-06-09 20:11:38 -03:00
Louis Szeto 9f15ca4c9c feature-<#5631>-<Create SpreadExecutionModel> (#5633)
Build & Test Lean / build (push) Has been cancelled
* Create SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Create SpreadExecutionModel.cs

* Create SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update QuantConnect.Algorithm.Framework.csproj

* Create SpreadExecutionModelRegressionAlgorithm.py

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.cs

* Address review

- Adding CSharp regression algorithm implementation
- Updating unit tests, minor format changes

* Address review. Add comment and Abs spread percent

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11897
2021-06-09 14:55:00 -03:00
IlshatGaripov d23ac25b67 Polygon IO limited aggregate bars per request. Fix. (#5363)
Build & Test Lean / build (push) Has been cancelled
* Applies a fix

* Fixing the tests to cover downloader func. Fixing the queue handler

* this has to be removed

* -- Fixes -- [WIP]

* Implements the way to upload historical bars in async fashion. [ WIP ]

* Finishes GetEquityTradeTicks() impl.

* Do not break iteration if result is null

* Uses DisposeSafely and fixes a typo

* To use IEnumerable for return type in GetHistoryAsync

* Fixes GetEquityQuoteTicks()

* Removes GetHistoryAsync() and reverts the associated with it changes

* SubscriptionManager -> revert the changes

* PolygonDataQueueHandler -> use SIP timestamp as it keeps the order

* PolygonDataQueueHandler -> use Log.Trace for the no data {date} message

* Fixes GetForexQuoteTicks()

* Introduce ResponseSizeLimitAggregateData

* Fixes up GetCryptoTradeTicks()

* Small fixes for tick processing methods

* Removes launchSettings.json

* Refactor download data logic

- Replace WebClient for HttpClient
- Remove duplicated download logic and parsing for a single static
  method

* Update PolygonDataQueueHandler.cs

Remove commented code

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11894
2021-06-09 10:43:34 -03:00
Colton Sellers 8f7f862563 Ignore Tests (#5646)
Build & Test Lean / build (push) Has been cancelled
11893
2021-06-09 10:25:07 -03:00
Colton Sellers 0f0eeaff82 DataProvider Updates & Refactoring (#5641)
* Use DataProvider for fetching Shortable and FutureMargins files

* Use data provider for AllShortableSymbols

* Refactor of NeedToDownload logic

* Adjustments

* nit - cleanup

* Expand test cases

* Create the directory if it doesn't exist
2021-06-08 22:11:21 -03:00
Martin-Molinero e8cf002238 Live trading holding improvements (#5644)
Build & Test Lean / build (push) Has been cancelled
- Store holdings at end of algorithm
- Warmup holding market price if 0
11891
2021-06-08 21:55:17 -03:00
Stefano Raggi e875b111d7 IB Brokerage updates (#5642)
Build & Test Lean / build (push) Has been cancelled
* Add immediate logging for IB brokerage unit tests

* Filter brokerage message event for IB warning

- 10197 - No market data during competing live session.

* Update IBAutomater to v2.0.32

* Update IBAutomater to v2.0.33
11887 11889
2021-06-08 11:29:29 -03:00
Martin-Molinero a231a7d47a Update python net to 2.0.3 (#5640)
Build & Test Lean / build (push) Has been cancelled
* Update pythonNet version to 2.0.3

* Fix outputPath causing issues with nuget
11884
2021-06-07 21:37:26 -03:00
Colton Sellers 37cce922d3 Api Data Request Path Format Issue (#5639)
Build & Test Lean / build (push) Has been cancelled
* Address some format issues for data request paths

* Add reproducing test

* Drop logging for failure, let Api take care of it
11880 11883
2021-06-07 20:59:39 -03:00
Colton Sellers 5f9e4a6bca Option Exercise Quantity Fix (#5638)
Build & Test Lean / build (push) Has been cancelled
* Use determined quantity for exercise order

* Add regression algorithm for issue

* Minor tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11876
2021-06-07 18:51:52 -03:00
Martin-Molinero 42c58d6e53 Ensure data is downloaded only once (#5635)
Build & Test Lean / build (push) Has been cancelled
- Ensure data is downloaded only once with the ApiDataProvider. Adding
  unit test
- Log invalid securityType ApiDataProvider requests once
- Assert Factor file and map file providers are compatible with the
  ApiDataProvider
11870 11872
2021-06-07 14:26:24 -03:00
Colton Sellers ab6027723f Refactor Api (#5251)
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request

* key -> filePath rename and some cleanup

* Refactor

* Add Organizations Endpoints

* Add some organization api wrapper objects

* Address namespace issue

* Reorganize Api Test into seperate files using one ApiTestBase

* Add Organization tests

* Use capitalized "API" test namespace to reduce amount of file changes

* Add License to test base

* Update /data endpoint functions and response objects

* Update ApiDataProvider Logic

* Handle deserialization of organization products

* Simplify converter

* Only throw for equity requests when not subscribed to map/factor files

* Add missing header

* Make arguement exception

* Api adjustments

* Add Zip factor and map file providers

- Common project will now reference Compression project and not the other way
  round.
- Adding Zip FactorFile and MapFile providers

* Refactor FactorFileProvider to use DataProvider to fetch files

* Use resulting MinimumDate in construction of FactorFile

* Nit FactorFile comments and arrangement

* Refactor MapFileProviders to use DataProvider for fetching files

* Refactor ZipFileProvider

* Clean up

* Refactor Backtesting Future/Option chain providers to use dataprovider

* Fixes for data/ endpoints and test adjustments

* Response objects adjustments/cleanups

* ApiDateProvider fixes and testing

* Add LocalZipFactorFileTests

* Update ApiDataProvider download test to verify stream is not null

* Implement posting of agreement summary and signed time

* Mark all Api related tests as explicit and document details on running

* Clarify default token on ApiTestBase

* Adjust summary

* Update Api responses for QCC, except org products which are sold in USD

* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests

* Fix multiple markets for ZipFactorFile provider

* Use Symbol as cache key

* Api.cs review

* Dispose of factorFileStream after reading

* Use zip.EntryFileNames

* Address a few reviews

* Few more fixes

* Address Api Review

* Add Job Org id to config

* Minor tweaks

* Compare with invariant culture

* Fixes Option Universe selection

* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider

* Fix research

* Fix null reference exception

* Make duplicate log debug

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11866
2021-06-07 11:37:44 -03:00
Alexandre Catarino 37473c4580 Fixes OverflowException in FuturesOptionsMarginModel (#5629)
Build & Test Lean / build (push) Has been cancelled
11861
2021-06-04 21:24:19 -03:00
Gerardo Salazar 677661766b Fixes filtering of CoinAPI raw data in CoinApiDataConverter (#5625)
Build & Test Lean / build (push) Has been cancelled
* Fixes bug where raw data was incorrectly be filtered when ID was duped

* Filter CoinAPI raw data by distinct Symbol created from the SymbolMapper

* Fixes bug where only quotes or trades would be processed, not both

* Trigger rebuild
11849 11856
2021-06-04 13:53:47 -03:00
Martin-Molinero c66a26e01d Fix build: Holding.Type is not setable (#5628)
Build & Test Lean / build (push) Has been cancelled
11844 11848
2021-06-03 21:03:18 -03:00
Martin-Molinero effa2da594 Avoid need for Holding Type update build status (#5623) 2021-06-03 20:21:18 -03:00
Stefano Raggi 572aa99743 IBAutomater updates (#5626)
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.27

* Update IBAutomater to v2.0.28

* Update IBAutomater to v2.0.29

* Update IBAutomater to v2.0.30

* Trigger build
11839 11841
2021-06-03 19:04:54 -03:00
Gerardo Salazar e77e202373 Adds support for LeanDataWriter minute/second/tick writing for files w/ more than 2GB uncompressed (#5620)
Build & Test Lean / build (push) Has been cancelled
11823
2021-06-02 18:29:50 -03:00
Martin-Molinero f9c50e28ef Add data libraries into image (#5619)
Build & Test Lean / build (push) Has been cancelled
11815 11819 11820 11821
2021-06-02 12:45:28 -03:00
Martin-Molinero 4c773c348c Update readme.md
Build & Test Lean / build (push) Has been cancelled
11800
2021-05-31 11:13:13 -03:00
Colton Sellers 41f9d9da94 Remove SecurityInitializer from Universe Selection (#5418)
Build & Test Lean / build (push) Has been cancelled
* Attempt to remove SecurityInitializer from UniverseSelection stack entirely

* Fix broken Python Universe Selection model

* Remove mistake change (meant for another branch)
11796
2021-05-28 14:53:13 -03:00
Martin-Molinero 5f6a40fd9c Update readme.md
Build & Test Lean / build (push) Has been cancelled
11795
2021-05-28 12:18:52 -03:00
Martin-Molinero 0335b49edf Update pythonNet to 2.0.2 (#5611) 2021-05-28 12:10:55 -03:00
Derek Melchin 1175c304e1 Fix live timestamp alignment in CustomDataBitcoinAlgorithm (#5603)
Build & Test Lean / build (push) Has been cancelled
* Fix python version

* Make C# version match

* Revert c# changes, make python version use exchange timezone
11787 11788 11793
2021-05-27 20:09:48 -03:00
Colton Sellers 6c42f58b68 VS Code Tasks and Python Debugging Fixes (#5601)
Build & Test Lean / build (push) Has been cancelled
* ~Fix Python debugging and update docs

* Cleanup tasks
11784
2021-05-27 16:34:04 -03:00
Alexandre Catarino 22a5378604 Reduce Leverage for Equity to 1 in Alpha Stream Brokerage Model (#5598)
Build & Test Lean / build (push) Has been cancelled
All security type use 1 as leverage but Forex and CFD that are allow to use 10.
11783
2021-05-27 15:31:34 -03:00
Martin-Molinero 2df6715301 Add hostname to initial log message (#5608)
Build & Test Lean / build (push) Has been cancelled
11773
2021-05-26 19:27:05 -03:00