10991 Commits

Author SHA1 Message Date
Antoine Dray 4dd2ad85a9 Fix KrakenDownloader error on invalid pair format (#5703) (#5704)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
The Kraken allow queries from pairs of format XBTUSD and XXBTZUSD but it will only return data with the XXBTZUSD causing errors when XBTUSD ticker is specified.
This commit adds a more meaningful error when KDL is called with XBTUSD format.
12149 12151
2021-06-24 19:42:13 -03:00
Martin-Molinero 10defd4928 Include IB CSharpAPI in brokerage nuget (#5706)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12147
2021-06-24 18:59:14 -03:00
Martin-Molinero 891aeb87d3 Update readme.md
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12114 12115 12118 12119 12106 12125 12126 12127 12128 12129 12130 12131 12132 12133 12134 12135 12137 12139 12140 12143 12124 12110 12121 12112
2021-06-23 18:17:25 -03:00
Stefano Raggi 2fa07561b1 Add missing IBGateway dependency library (#5696)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12102 12104
2021-06-23 17:12:50 -03:00
Stefano Raggi 148b9c8d45 IB Brokerage updates (#5695)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.39

* Downgrade IB logging level from Detail (5) to Information (4)

- The online documentation warns of performance overhead when setting logLevel=5
12099 12100 12101
2021-06-23 11:34:33 -03:00
Jared 65cecef030 Update readme.md
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12094 12095
2021-06-22 18:15:35 -07:00
Stefano Raggi 4fed304eb7 Update Lean Foundation docker files to IBGateway v984 (#5686)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12089 12093 12086
2021-06-22 17:20:47 -03:00
Colton Sellers a987e0231f Scheduled Events for Start/End Week/Month Fix (#5690)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Apply fix for both start and end month cases

* Add unit tests for both cases

* Fix weekstart/weekend for same issue; add unit tests

* nit - rename tests

* Adjust logic
12083
2021-06-22 14:20:00 -03:00
Colton Sellers f47775338c Expand Lean Default Charts and Reduce Clutter (#5680)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update charts that are not empty and not default values

* Testing

* Filter final results

* Explicitly filter alpha charts if not needed

* Use filtered charts for Live results handler

* Refactor solution to manage alpha charts internally by creator

* nit comment fixes

* Implement drawdown as default chart

* Implement Capacity Estimate for backtest default chart

* Add default values to QC.Plotting

* Adjust drawdown calculation

* Only sample "Capacity" once a day

* Implement Volume chart

* Nit fixes

* Volume Chart adjustments and fixes

* Rename to "Assets Sales Volume"

* Implement Exposure Sampling

* Store algorithm currency symbol, round capacity to nearest 1k

* Add new plots to default charts

* nit change name

* Address reviews

* Reduce duplication and clean up exposure sampling

* Address reviews

* Improve sample exposure

* Address review

* Only enumerate holdings once

* nit - comments

* Post rebase fix

* Don't need to round anymore
12062 12081 12078 12079
2021-06-22 11:19:48 -03:00
Colton Sellers fa1e97bc60 Round Capacity Estimate for Runtime Statistics (#5688)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Refactor solution to deal with rounding in CapacityEstimate

* Use DiscretelyRoundBy + adjust regressions
12060
2021-06-21 21:06:09 -03:00
Martin-Molinero 8fb649ceb4 SymbolJsonConverter improvements (#5683)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* SymbolJsonConverter improvements

- SymbolJsonConverter will be able to deserialize symbols expressed in
  user friendly format. Adding unit test
- Adding a couple of helper methods, removing duplicated code

* Fix FOPs expiration year parsing
12059
2021-06-21 21:02:46 -03:00
Stefano Raggi 7d8754f33c Update IBAutomater to v2.0.38 (#5682)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.38

* Trigger build
12046 12057
2021-06-21 12:23:55 -03:00
Stefano Raggi 080511fa5a Add IResultHandler.BrokerageMessage (#5681)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add IResultHandler.BrokerageMessage

- Result handlers can receive notifications for brokerage message events

* Make LiveTradingResultHandler.RuntimeError virtual
12047
2021-06-21 12:23:46 -03:00
Martin-Molinero 5146329c0f CoinApi: reduce thread contention by lock per symbol (#5677)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Reduce thread contention in coinApi DQH by taking a lock per symbol
  versus a global lock.
- Improve tick creating by reducing object being wasted on empty
  constructor
12018 12024 12034
2021-06-17 20:47:57 -03:00
Stefano Raggi 47fb4e64f2 Update IBAutomater to v2.0.37 (#5676)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12005 12007 12009
2021-06-17 16:14:38 -03:00
Alexandre Catarino 5361f87dd1 Updates Equity Market Data (#5576)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data

* Updates Unit Tests

* Updates Regression Tests

In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.

* Updates Regression Tests 2

The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238

Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
 - BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`

The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying

* Revert File Update for SPWR and SPWRA

* Fix Regression Tests

Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.

* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]

Uses `trust-constr`  method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
12003
2021-06-17 14:04:51 -03:00
Jasper van Merle 72fb362f96 Fix futures not working with IQFeed (#5675)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix futures not working with IQFeed

* Trim slash from value instead of using ID.Symbol
12000
2021-06-17 08:46:03 -07:00
Martin-Molinero 24e3a71581 Coinapi improvements (#5672)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Add centralized download extension helper method
- Update coinApi client
11994
2021-06-16 20:30:51 -03:00
Colton Sellers 15555f710d Python Loading and Debugging Refactor (#5670)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use Python algorithms directly in their location; also refactor pythonpath additions to one method

* Update VSCode Launch options

* Drop issue these changes fixes

* Use AddRange
11993
2021-06-16 20:30:33 -03:00
Martin-Molinero 3d13fb6935 Implement Atreyu fees (#5674)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Empty

* Implement Atreyu fees
11992
2021-06-16 19:43:27 -03:00
Colton Sellers 74f0c89499 Option Chain Provider Symbol Map Resolve (#5664)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use mapFileProvider to resolve symbol mapping

* Add reproducing unit test

* Only resolve mapping for types that require it

* address review
11991
2021-06-16 19:42:38 -03:00
Martin-Molinero 0991f089d0 Reduce disk usage in the cloud (#5673)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
11988
2021-06-16 18:43:55 -03:00
Colton Sellers 424fa36703 Organization Product ID and ApiDataProvider Fix (#5671)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add ID to Organization Products

* Use Product ID to verify subscription
11982
2021-06-16 15:01:53 -03:00
Colton Sellers afbd058dad Add Site-Package Issue to Readme (#5668)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add information to Python and Research readme

* Adjust doc
11977
2021-06-16 13:57:13 -03:00
Stefano Raggi 66d2f42d01 IBAutomater updates (#5669)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.36

* Trigger build
11976
2021-06-16 12:52:43 -03:00
Martin-Molinero 11f8dc2958 Improvements for CoinApi subscription (#5666)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
11971 11965 11968
2021-06-15 19:49:14 -03:00
Martin-Molinero 03f56481d4 Refactor python algorithm import (#5657)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements

- Improve start.py for research env
- Remove unrequired imports

* Centralize algorithm imports

* Add regression test GH action

* Unit test python import clean up

* Join research and main imports

* More python import clean up

* Fix failing skipped regression algorithm
11963 11964
2021-06-15 19:06:06 -03:00
Stefano Raggi c367451c6d Update IBAutomater to v2.0.35 (#5663)
Build & Test Lean / build (push) Has been cancelled
11960 11961
2021-06-15 16:58:31 -03:00
Alexandre Catarino c4e6e7f8dc Changes VIXCentralContango.F9 to Nullable (#5662)
Build & Test Lean / build (push) Has been cancelled
According to the data, the contract 8 months away from the front-month contract, `F9`, can be null.
11953
2021-06-14 16:23:28 -07:00
Colton Sellers ef5f7fbc50 QuantBook Provider and Composer Fix (#5660)
Build & Test Lean / build (push) Has been cancelled
* Use Composer instance for QB so providers are the same across Lean; also initialize map and factorfiles

* Don't need map and file initialize
11951
2021-06-14 19:17:53 -03:00
Colton Sellers 7f9d35f666 Research BollingerBands Value Error (#5656)
Build & Test Lean / build (push) Has been cancelled
* Stop update when properties will fail to update

* Only enforce our case after the BB is ready

* Nit comment fixes

* Add unit test

* Use faster DateTime.UtcNow
11943
2021-06-14 14:23:05 -03:00
Gerardo Salazar be0fc9a879 Fixes case where crypto has short exposure in ReportGenerator in long-only strategies (#5655)
Build & Test Lean / build (push) Has been cancelled
11934 11940
2021-06-11 21:40:49 -03:00
Colton Sellers a882aadeef Research Fixes and Doc Updates (#5649)
Build & Test Lean / build (push) Has been cancelled
* Updates to default notebooks

* Update readme

* Notebook adjustments

* Drop launch script support

* Use our cwd if no QC dlls are in AppDomain directory

* Modify Intialize.csx to use parent directory only if QC files are detected there

* Clean up some details after testing

* Adjustments

* nit, NullOrWhiteSpace
11926
2021-06-11 16:06:47 -03:00
Stefano Raggi 02f7742878 Update IBAutomater to v2.0.34 (#5652)
Build & Test Lean / build (push) Has been cancelled
11925
2021-06-11 14:38:45 -03:00
Martin-Molinero b7c18cbfe7 Update pythonNet to net5 (#5653)
Build & Test Lean / build (push) Has been cancelled
- Csproj clean up legacy build print logs
- Update pythonNet to net5
11922
2021-06-11 13:51:09 -03:00
Martin-Molinero 6e6a1631dc Enable loading crypto live virtual position tracking (#5651)
Build & Test Lean / build (push) Has been cancelled
* Enable loading live virtual position tracking

- Enable GDAX paper trading configurations
- GDAX will optionally load existing holdings from live job package
- Bitfinex will try loading existing holdings from live job package for
  cash account type

* Binance live holdings loading
11911 11918
2021-06-10 20:05:23 -03:00
Gerardo Salazar 250ffa1fec Adds VIXCentralContango data, modifies cached alternative data sourcing in live mode (#5648)
Build & Test Lean / build (push) Has been cancelled
* Adds VIXCentral Contango data

  * Refactor cached alternative data in live mode to
    use web endpoint if the data is not available locally

* Remove fallback to cache bucket for CBOE/FRED/EIA/VIXCentralContango

* Address review: force VIX as ticker for all VIXCentralContango data
11908
2021-06-09 20:44:53 -03:00
Martin-Molinero 7e2678d0f6 Normalize live holdings initialization (#5647)
Build & Test Lean / build (push) Has been cancelled
* Normalize live holdings initialization

* PaperBrokerage config based live state recovery

- Increase algorithm status storing sampling period
- Fix bug where duplicate cash entries could happen
11905
2021-06-09 20:11:38 -03:00
Louis Szeto 9f15ca4c9c feature-<#5631>-<Create SpreadExecutionModel> (#5633)
Build & Test Lean / build (push) Has been cancelled
* Create SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.py

* Create SpreadExecutionModel.cs

* Create SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModel.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.cs

* Update QuantConnect.Algorithm.Framework.csproj

* Create SpreadExecutionModelRegressionAlgorithm.py

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModelTests.cs

* Update SpreadExecutionModel.py

* Update SpreadExecutionModel.cs

* Address review

- Adding CSharp regression algorithm implementation
- Updating unit tests, minor format changes

* Address review. Add comment and Abs spread percent

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11897
2021-06-09 14:55:00 -03:00
IlshatGaripov d23ac25b67 Polygon IO limited aggregate bars per request. Fix. (#5363)
Build & Test Lean / build (push) Has been cancelled
* Applies a fix

* Fixing the tests to cover downloader func. Fixing the queue handler

* this has to be removed

* -- Fixes -- [WIP]

* Implements the way to upload historical bars in async fashion. [ WIP ]

* Finishes GetEquityTradeTicks() impl.

* Do not break iteration if result is null

* Uses DisposeSafely and fixes a typo

* To use IEnumerable for return type in GetHistoryAsync

* Fixes GetEquityQuoteTicks()

* Removes GetHistoryAsync() and reverts the associated with it changes

* SubscriptionManager -> revert the changes

* PolygonDataQueueHandler -> use SIP timestamp as it keeps the order

* PolygonDataQueueHandler -> use Log.Trace for the no data {date} message

* Fixes GetForexQuoteTicks()

* Introduce ResponseSizeLimitAggregateData

* Fixes up GetCryptoTradeTicks()

* Small fixes for tick processing methods

* Removes launchSettings.json

* Refactor download data logic

- Replace WebClient for HttpClient
- Remove duplicated download logic and parsing for a single static
  method

* Update PolygonDataQueueHandler.cs

Remove commented code

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11894
2021-06-09 10:43:34 -03:00
Colton Sellers 8f7f862563 Ignore Tests (#5646)
Build & Test Lean / build (push) Has been cancelled
11893
2021-06-09 10:25:07 -03:00
Colton Sellers 0f0eeaff82 DataProvider Updates & Refactoring (#5641)
* Use DataProvider for fetching Shortable and FutureMargins files

* Use data provider for AllShortableSymbols

* Refactor of NeedToDownload logic

* Adjustments

* nit - cleanup

* Expand test cases

* Create the directory if it doesn't exist
2021-06-08 22:11:21 -03:00
Martin-Molinero e8cf002238 Live trading holding improvements (#5644)
Build & Test Lean / build (push) Has been cancelled
- Store holdings at end of algorithm
- Warmup holding market price if 0
11891
2021-06-08 21:55:17 -03:00
Stefano Raggi e875b111d7 IB Brokerage updates (#5642)
Build & Test Lean / build (push) Has been cancelled
* Add immediate logging for IB brokerage unit tests

* Filter brokerage message event for IB warning

- 10197 - No market data during competing live session.

* Update IBAutomater to v2.0.32

* Update IBAutomater to v2.0.33
11887 11889
2021-06-08 11:29:29 -03:00
Martin-Molinero a231a7d47a Update python net to 2.0.3 (#5640)
Build & Test Lean / build (push) Has been cancelled
* Update pythonNet version to 2.0.3

* Fix outputPath causing issues with nuget
11884
2021-06-07 21:37:26 -03:00
Colton Sellers 37cce922d3 Api Data Request Path Format Issue (#5639)
Build & Test Lean / build (push) Has been cancelled
* Address some format issues for data request paths

* Add reproducing test

* Drop logging for failure, let Api take care of it
11880 11883
2021-06-07 20:59:39 -03:00
Colton Sellers 5f9e4a6bca Option Exercise Quantity Fix (#5638)
Build & Test Lean / build (push) Has been cancelled
* Use determined quantity for exercise order

* Add regression algorithm for issue

* Minor tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11876
2021-06-07 18:51:52 -03:00
Martin-Molinero 42c58d6e53 Ensure data is downloaded only once (#5635)
Build & Test Lean / build (push) Has been cancelled
- Ensure data is downloaded only once with the ApiDataProvider. Adding
  unit test
- Log invalid securityType ApiDataProvider requests once
- Assert Factor file and map file providers are compatible with the
  ApiDataProvider
11870 11872
2021-06-07 14:26:24 -03:00
Colton Sellers ab6027723f Refactor Api (#5251)
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request

* key -> filePath rename and some cleanup

* Refactor

* Add Organizations Endpoints

* Add some organization api wrapper objects

* Address namespace issue

* Reorganize Api Test into seperate files using one ApiTestBase

* Add Organization tests

* Use capitalized "API" test namespace to reduce amount of file changes

* Add License to test base

* Update /data endpoint functions and response objects

* Update ApiDataProvider Logic

* Handle deserialization of organization products

* Simplify converter

* Only throw for equity requests when not subscribed to map/factor files

* Add missing header

* Make arguement exception

* Api adjustments

* Add Zip factor and map file providers

- Common project will now reference Compression project and not the other way
  round.
- Adding Zip FactorFile and MapFile providers

* Refactor FactorFileProvider to use DataProvider to fetch files

* Use resulting MinimumDate in construction of FactorFile

* Nit FactorFile comments and arrangement

* Refactor MapFileProviders to use DataProvider for fetching files

* Refactor ZipFileProvider

* Clean up

* Refactor Backtesting Future/Option chain providers to use dataprovider

* Fixes for data/ endpoints and test adjustments

* Response objects adjustments/cleanups

* ApiDateProvider fixes and testing

* Add LocalZipFactorFileTests

* Update ApiDataProvider download test to verify stream is not null

* Implement posting of agreement summary and signed time

* Mark all Api related tests as explicit and document details on running

* Clarify default token on ApiTestBase

* Adjust summary

* Update Api responses for QCC, except org products which are sold in USD

* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests

* Fix multiple markets for ZipFactorFile provider

* Use Symbol as cache key

* Api.cs review

* Dispose of factorFileStream after reading

* Use zip.EntryFileNames

* Address a few reviews

* Few more fixes

* Address Api Review

* Add Job Org id to config

* Minor tweaks

* Compare with invariant culture

* Fixes Option Universe selection

* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider

* Fix research

* Fix null reference exception

* Make duplicate log debug

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
11866
2021-06-07 11:37:44 -03:00
Alexandre Catarino 37473c4580 Fixes OverflowException in FuturesOptionsMarginModel (#5629)
Build & Test Lean / build (push) Has been cancelled
11861
2021-06-04 21:24:19 -03:00