10792 Commits

Author SHA1 Message Date
Jasper van Merle 8076782ea6 Make random data generator save non-equity data (#5371) 11044 2021-03-08 09:49:17 -03:00
hsm207 e869765759 Fix typo (#5361) 11035 11036 11037 11023 11024 11026 11027 11029 11030 11033 11034 11028 11031 2021-03-04 15:28:21 -03:00
Martin-Molinero c7fb6165e2 Revert "Upgrade IBAutomater to v1.0.51 (#5359)" (#5362)
This reverts commit caedddddf0.
11021
2021-03-04 15:20:15 -03:00
IlshatGaripov d9d02bc2d0 Applies a fix (#5358) 11015 11016 11018 2021-03-04 13:13:41 -03:00
Stefano Raggi caedddddf0 Upgrade IBAutomater to v1.0.51 (#5359) 11013 2021-03-04 13:03:31 -03:00
Martin-Molinero 4e00324b57 Bug insight internal to procted internal (#5357)
* Update Insight.cs

* Insight protected to protected internal

Co-authored-by: Jared <jaredbroad@gmail.com>
11008 11011 11012
2021-03-03 16:14:36 -08:00
Martin-Molinero 1b6301d813 Update Insight.cs (#5356) 2021-03-03 16:02:57 -08:00
Jared d24695856d Update Insight.cs (#5355) 2021-03-03 15:17:25 -08:00
hsm207 17e13665eb Fix typo (#5353) 10993 10995 10996 10998 10999 10992 10997 2021-03-03 10:23:05 -03:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
10989 10990 10991
2021-03-02 18:46:31 -03:00
IlshatGaripov 83da0affc5 Implements Coin API HistoryProvider (#5298)
* CoinApi HistoryProvider impl.

* Small fix of ToTradeBar()

* Revert the change : CreateSliceEnumerableFromSubscriptions

* More tests and fixes.

* Removes redundant ToList()

* too much spacing

* Fixes to get data for an unlimited period of time, by means of pagination
10984 10985 10986
2021-03-02 11:08:14 -03:00
Jasper van Merle 60798df64c Make optimizer usable in Docker and improve its logging (#5344)
* Make optimizer usable in Docker and improve its logging

* Wait for reset event instead of polling state
10965 10966 10967 10968 10971 10974 10975 10976 10977 10978
2021-02-26 15:50:16 -03:00
Martin-Molinero bb20f9d9df IB will not load holdings internally if requested (#5342) 10954 10955 10956 10957 10958 10959 10960 10961 10962 10963 10964 2021-02-25 11:43:38 -08:00
Martin-Molinero 32686db739 NotificationJsonConverter is case insensitive (#5341)
- NotificationJsonConverter will be case insensitive. Adding unit test
10953
2021-02-25 11:42:42 -08:00
Stefano Raggi 7d97f05133 Tradier Brokerage updates (#5326)
* Add options support

- Added IDataQueueUniverseProvider implementation
- Fix subscribe ticker for options

* Tradier DataQueueHandler web sockets implementation

- Equities and Options (trades and quotes)

* Add error handling

* Add support for options

* Fix log messages

* Address review

- Updated Tick constructor usage
- Use $empty$ symbol for last unsubscribe message
- Added symbol mapper unit tests
10946 10947 10948 10944 10945
2021-02-25 10:41:47 -03:00
Colton Sellers 8b173f2306 Add dotnet 5 to Lean Foundation Image (#5307)
* Add dotnet 5 to foundation image

* Address review. Remove net5 package after install

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10942
2021-02-24 21:09:59 -03:00
Jasper van Merle 9cde4344fc Re-apply changes from #4870 to fix ThreadAbortException in ToolBox (#5339) 10941 2021-02-24 20:50:56 -03:00
Martin-Molinero 785c706d4c Live delisting will only ask mapfile for equities (#5336)
* Live delisting will only ask mapfile for equities

* Remove unrequired log error
10930 10937 10938 10940 10936
2021-02-24 14:14:50 -03:00
Gerardo Salazar 0dc521f787 Reduces log spam on long backtests when generating capacity (#5334) 10920 10923 10925 10926 2021-02-23 16:22:49 -08:00
Martin-Molinero 94d9996b21 Enforce USD account currency if allocation is limited (#5333)
* Enforce USD account currency if allocation is limited

- Enforce USD account currency if allocation is limited. Adding unit
  test

* Adjust currency log message
10909 10911 10912 10913 10917 10918 10916
2021-02-23 11:57:48 -08:00
Martin-Molinero 67b3873205 Reduce live coarse selection interval check to 10 min (#5332)
* Reduce live coarse selection interval check to 10 min

- Reduce live coarse selection interval check to 10 min. Adding unit
  tests, reducing code duplication in tests.

* Adjust unit test after self review
10908
2021-02-23 16:37:08 -03:00
Gerardo Salazar acd9ef4cfd Removes SevenZipSharp library from ToolBox (#5329)
* Removes SevenZipSharp library from ToolBox

  - Library removal required for update to .NET 5.0 since it's not
    compatible with it and Linux is unsupported.

  * Adds new extract 7z functionality to Compression project
  * Refactors AlgoSeekFuturesConverter 7z extract
  * Refactors DukascopyDataDownloader 7z extract

* Addresses review: code cleanup + exception type change on timeout

  * Removes not needed stream of data in Dukascopy downloader
  * Makes output directory non-optional
10901
2021-02-23 11:38:07 -03:00
Gerardo Salazar 7f934b2adf Remove unused dependency WebSocketSharpFork (#5331) 10900 2021-02-23 11:33:17 -03:00
Martin-Molinero 596e940ccd Update R library (#5328)
* Update R.Net libraries

* Update R installation version
10886 10898 10899
2021-02-22 19:24:05 -03:00
Gerardo Salazar f38ff0d1f2 Reduces StrategyCapacity memory load and boosts performance (#5327)
* Adds logging for progress tracking
  * Reads data one day at a time rather than loading all data at once
10869 10885
2021-02-22 11:53:12 -08:00
Gerardo Salazar 37d26a35ce Fixes various bugs in estimated capacity class (#5324)
* Fixes potential mapping issue, caused by not mapping Symbols before
    reading data off disk

  * Fixes issue where some results would evaluate to zero capacity

  * Loads of refactoring, mostly resulting in cleaner code
10864 10862
2021-02-19 18:00:08 -08:00
Martin-Molinero 834326760d Fix regression test History Request (#5321)
- After https://github.com/QuantConnect/Lean/pull/5207 history requests
  are selecting the correct data type and cause the regression test to
  fail because there is no QuoteBars for daily equity. Updating and
  fixing this.
- Centralizing the logic around getting the SubscriptionDataConfig to
  use
10859
2021-02-19 20:39:50 -03:00
Martin-Molinero d4ebd93586 Limit live cash allocation (#5320)
* Add brokerage data cash limit feature

* Expand unit test to asser orders are skipped

* Add algorithm debug messages

* Ignore account changed event
10855
2021-02-19 20:08:32 -03:00
Gerardo Salazar 0645513f5e Capacity estimation in report generator (#5318)
* Adds strategy capacity estimation in report generator

* Style and license fixes

* Adds comments, constants, and new method accessibility

* Removes unneeded imports
10840
2021-02-18 17:51:40 -08:00
Martin-Molinero ecf91546e6 Update Nlog Math and fasterflect (#5310)
* Update Nlog Math and fasterflect

* Remove NLog
10835
2021-02-18 20:18:07 -03:00
Martin-Molinero c834a1c902 Revert Lean.Launcher nuget package rename (#5317) 10834 2021-02-18 19:39:05 -03:00
Colton Sellers 43a540cbb1 OpenInterest Bug Fixes (#5207)
* Filter values that are before subscription start time; also adjust starttime for OpenInterest

* Use data EndTime for comparison

* Allow Auxiliary data through

* Fix OpenInterest DataReader Logic

* Add regression

* Address review

* Ignore open interest for time slice

- TimeSliceFactory will directly ignore open interest for determining if
  the slice has data or not. Open interest will still be available
  through the Tick collection. Reverting some of the previous commits
  changes since they are no longer required.
- HistoryRequests and SubscriptionRequest will use AlwaysOpen exchange
  for open interest requests. Adding unit test reproducing issue
- Adding `BaseDataRequest` to avoid duplication logic.

* Make OpenInterest an internal feed and ignored by default in history

- Adding unit tests

* Revert SubscriptionFilterEnumerator Start time addition

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10829 10830
2021-02-18 19:04:29 -03:00
Alexandre Catarino c5bb840bde Adds Static AlgorithmCreationTimeout Property to BaseSetupHandler (#5316)
* Adds Static AlgorithmCreationTimeout Property to BaseSetupHandler

This value will be used to set maximum time that the creation of an algorithm can take.

* Adds Additional Logging to AlgorithmPythonWrapper

It will informing the user how long it takes to import the module.

* Changes How AlgorithmCreationTimeout is Initialized

It will prevent callling Config.GetDouble on every call.
10828
2021-02-18 19:00:33 -03:00
Martin-Molinero 4632840da9 Add license file for project packages (#5314) 10826 2021-02-18 18:57:36 -03:00
Martin-Molinero ba4c45d729 Remove version checking (#5315) 10824 2021-02-18 18:32:07 -03:00
Martin-Molinero 9232427663 Remove all nuspec files (#5311) 10820 2021-02-18 17:13:11 -03:00
Martin-Molinero c30929f612 Add support for live trading delisting events (#5195)
* Add support for live trading delisting events

- Adding `LiveDelistingEventProviderEnumerator` that will emit delisting
  events. Adding unit tests
- Remove unused `LiveAuxiliaryDataEnumerator`.
- Refactor the corporate event providers being used per security type

* Self review: compare date component

* Add new LiveDataBaseDelistingEventProvider for equities

- Add support for delisting events for equities. Adding unit tests

* Fix cleanup of delisting equity subscription
10801 10806
2021-02-17 19:53:53 -03:00
Martin-Molinero df6e133833 Fixes for assembly information (#5306)
- Update version to 2.5
- Normalize version and assembly information
- Remove commented and unrequired code
10793
2021-02-17 16:42:23 -03:00
Juan José D'Ambrosio 2d99afc8c3 Refactor IPrimaryExchangeProvider.GetPrimaryExchange (#5302)
* Refactor IPrimaryExchangeProvider.GetPrimaryExchange 

now it returns a PrimaryExchange instead of a string

* Update MapFilePrimaryExchangeProvider.cs

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
10776 10777 10778 10780 10781 10783 10790 10791 10775 10779 10792
2021-02-17 12:43:41 -03:00
Gerardo Salazar b3835510d0 Changes calculation of rolling/series graphs to use EOD equity (#5303)
* To maintain consistency between calculations, we will use
    the end of day equity value to calculate the returns per day.
    This fixes a bug where daily equity series would zero out and result
    in an exception being thrown since no elements were being passed to
    the Sharpe calculation.
10766 10772 10773 10774
2021-02-16 13:43:27 -08:00
Stefano Raggi 239fa29bf3 Tradier Brokerage updates (#5265)
* Tradier brokerage updates

- Removed old authentication code (refresh tokens) and settings
- Added "tradier-use-sandbox" config setting

* Fix brokerage unit tests

* Update Tradier config.json settings

* Add sandbox check in Subscribe

* Bug fixes + unit test updates

* Trigger build

* Fix Tradier fee model

* Address review
10758
2021-02-16 13:37:46 -03:00
Juan José D'Ambrosio df4abd3c3a Bug algoseek taq writes tick exchanges correctly (#5301)
* Implements wrapper to get the exchange as a single letter representation

Also replace lower 'l' with capital 'L' in condition flags enums. it is a new warning on the compiler and it looks cool :)

* There are cases where the exchange is null or empty, in those cases return null

* Add tests for single character exchange representation

* Update GlobalTest.cs

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-02-16 12:51:08 -03:00
Gerardo Salazar 7d93f43d38 Calculate more report elements using LiveResult data in report generator (#5300)
* Calculate PSR for live algorithms in report generator

* Use live series for calculating of CAGR, Max DD, and PSR

  * Removes Kelly Estimate from report and template.html

* Update template.html

* Rolling Beta bug fix when using daily data

  * Max drawdown output as percent

* Make days live report element use equity curve last day instead of today

Co-authored-by: Jared <jaredbroad@gmail.com>
10753 10754
2021-02-15 19:56:52 -08:00
Gerardo Salazar c435caa7d1 Add check for new deployments to DaysLiveReportElement (#5299)
* This fixes a potential exception thrown when trying to access the
    first element of the Live equity series, when no equity series
    exists.
10749
2021-02-15 21:30:12 -03:00
Aaron Janeiro Stone 643e8754ed Feature 5162 - LimitIfTouched Orders (#5164)
* Adds LimitIfTouched order.

TODO:
-- Add tests.
-- Add into existing regression algorithms.
-- Refactors (?)

* Fixes

- Remove unrequired space changes
- Fix EquityFillModel min/max limit price fill
- Add TriggerPrice for UpdateOrderRequest.

Quote info used w.r.t. comparing against Limit prices for LIT

FillModel.cs implementation is fixed to use quotes when comparing against set limit prices.
Also changes test implementations to assert which of quotes/trade-bars are being used

Reviewer-suggested fixes
------------
Merge remote-tracking branch 'origin/limiftouched' into limiftouched
Styling
Adds missing null check for quotebar
Styling
Adds missing null check for quotebar
Merge remote-tracking branch 'origin/limiftouched' into limiftouched
High/Low w.r.t. trigger price for determining if TriggerTouched changed to Current price
https://github.com/QuantConnect/Lean/pull/5164/files/0462ad668a5fe65e771122165f36c72d56b92df6#r569832380
fill fixes:
FillModel.cs: Fills exactly at the limit
EquityFillModel.cs: https://www1.interactivebrokers.com/en/index.php?f=608
Equity fill now uses quotebars
Revert - use new constructor which emulates former SubmitOrderRequest
Style changes
Reverts order model to original by means if new constructor
High/Low w.r.t. trigger price for determining if TriggerTouched changed to Current price
https://github.com/QuantConnect/Lean/pull/5164/files/0462ad668a5fe65e771122165f36c72d56b92df6#r569832380
fill fixes:
FillModel.cs: Fills exactly at the limit
EquityFillModel.cs: https://www1.interactivebrokers.com/en/index.php?f=608
Equity fill now uses quotebars
Revert - use new constructor which emulates former SubmitOrderRequest
Style changes
Reverts order model to original by means if new constructor
Merge commit 'bf4c96d2a055ea808fa4293662528c11a89b72c7'

* Suggested style fixes

* Review fixes
-------------------
- Suggested style fixes
- Changes LIT regression to better incorporate order modifications
- TODO: orderlisthash must be fixed
Fixed LIT regression algo
-----------------------
- Includes asserts in OnOrderEvent

* Fix OrderListHash

OrderListHash -> -292689487

* Re-adds quote nullchecks

* EquityFillModelTests fixes asserts

* Reordering FillModel.cs

* Fixes quote logic, adds methods in FillModel.cs

* Refactoring + LIT regression fixes
-- revert unneeded changes

* Fixes list hash

* Rebase -- catch up upstream

* OrderListHash fix

* Various fixes by reviewer

* Final requested changes

* tagged time -> utcinvariant

* Fixes listorderhash

* Time changed to UtcTime.ToString(DateFormat.US, CultureInfo.InvariantCulture)

* Adds Python LimitIfTouchedRegressionAlgorithm

* adds LimitIfTouchedRegressionAlgorithm.py

* adds LimitIfTouchedRegressionAlgorithm.py

* Minor changes to LIT regression algorithms

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10746 10742 10744
2021-02-15 10:29:51 -03:00
IlshatGaripov d17fe34c74 Fixing up Coin API streaming (#5295)
* fixing up Coin API streaming

* Fixing file name

* Adding log showing CoinApi product plan

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10741
2021-02-15 09:55:00 -03:00
Stefano Raggi 512000d500 Update IBAutomater to v1.0.44 (#5290) 10732 2021-02-12 12:10:02 -03:00
Gerardo Salazar f5e730d776 Re-enable second page of reports generation (#5294) 10730 10731 2021-02-11 17:12:30 -08:00
Martin-Molinero a0f39ec6a4 Remove Microsoft.Net.Compilers (#5282)
* Remove Microsoft.Net.Compilers

* Remove deprecated analyzer packages

- Replace deprecated analyzer packages

* Travis will use dotnet for building
10723 10728 10729
2021-02-11 09:31:37 -03:00
IlshatGaripov 54bbdacd76 Just a small fix (#5287) 10711 10709 10720 2021-02-10 12:17:43 -03:00