10799 Commits

Author SHA1 Message Date
Gerardo Salazar 561aa3cf25 Adds backwards compatibility for MHDB and SPDB for new SecurityTypes (#5373)
* Adds backwards compatibility for MHDB and SPDB for new SecurityTypes

* Skip invalid SecurityTypes in MHDB and SPDB

* One time log invalid SecurityTypes in MHDB/SPDB

* Moves logging of SecurityType into new extension method

  * Adds TryParseSecurityType to Extensions
11068 11069 11070 11072 11079 11080 11081 11071
2021-03-09 10:52:14 -03:00
hsm207 e4af2ef856 Fix typo (#5374) 11066 2021-03-09 10:29:21 -03:00
Martin-Molinero 3f8158a5be Add live trading Cash enforcement flag (#5372)
- Add live trading cash enforcement flag usefull for brokerages like IB
  which allow trading with currecies you don't have. Updating unit test
11065
2021-03-08 22:09:44 -03:00
Jared 0c4a641590 Changes Protection Level of Constructor of Serialized Order Classes (#5366)
Makes `SerializedOrderEvent.Id` a virtual member.

Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
11064
2021-03-08 17:03:09 -08:00
Stefano Raggi 20986059a9 Add OutsideRth property to InteractiveBrokersOrderProperties (#5360)
* Add OutsideRth property to InteractiveBrokersOrderProperties

* Address review
2021-03-08 20:15:25 -03:00
Balamurali Pandranki c22a538bad Indian stock markets support & Zerodha brokerage implementation (#4873)
* Indian stock markets & Samco and Zerodha brokerage implementations

* Build fixes & Implement multi leg orders (BracketOrder & CoverOrder)

* Build fixes

* Clean & refactor

* Clean up & remove samco brokerage

* Clean up & remove samco brokerage

* Fix Nifty, BankNifty & Sensex Index futures expiry time functions

* Fix Nifty, BankNifty & Sensex Index futures expiry time functions

* Fix Futures Expiry Testcases

* Fix Futures Expiry Testcases

* Refactor Zerodha Symbol Mapper

* Refactor Zerodha Symbol Mapper

* Add Future symbols to symbol prop db csv

* Fix Symbol Mapper context

* Fix Market Hours Database

* Fix OrderJsonConverter

* Add Zerodha AccountBaseCurrency

* Add QuantConnect License headers to new source files

* cleanup config.json formatting & tick aggregator implementation

* cleanup config.json formatting & tick aggregator implementation

* Refactor ZerodhaBrokerageModel

* Fix

* Build Fixes

* Refactor

* Refactor Brokerage class & remove TextFieldParser

* Refactor Brokerage FeeModel

* Add ZerodhaOrderProperties

* Add Refactor HistoryProvider

* Refactor CanExecuteOrder

* Refactor SymbolMapper

* Refactor market names

* Refactor & remove Zerodha subscription manager

* Refactor & remove ZerodhaWebSocketChannels

* Refactor & remove ZerodhaWebSocketChannels

* Refactor ZerodhaBrokerage

* Refactor symbol prop db

* zerodha update downloader ticker data using symbol name

* 1 Zerodha mapper class update to get instruments based on Market. 2 Zerodha Getholdings data fetch fix.

* Update market-hours-database.json

* Implement basic testcases for ZerodhaBrokerage

* Update market-hours-database.json

* Update Market.cs

* 1 Fix for Zerodha subscriber and unsubscribe 2 User of CSVHelper to read instrument list

* Rename Zerodha DataQueueHandler class implementation

* Changes related to TradeBar and emit tick

* Fix DataQueueUniverseProvider & Handle Timezone conversion in WS EmitQuotes

* Emit Order Fixes

* DataQueueHandler refactoring and build fix

* Update config.json

* Removal of IHistoryProvider impplementation

* Reverting timezone logic as already taken care by BrokerImplementation using Unix time

* Place, modify and cancel order implementation

* fix zerodha test cases

* Updating get quotes and restricting getHoldings to MIS

* Testcase Fixes

* Testcase Fixes

* Testcase Fixes

* Updating emitFillOrder

* Add ProductType to ZerodhaOrderProperties

* Addition of ZerodhaProduct Type property and test update

* Fix for unit tests and minor changes for place and update order

* Unit test fix for Zerodha

* Addition of the product type and trading segment configs

* PR review fixes

* Addition of trading segment and product type configs

* Nuget Fixes

* Fix ZerodhaBrokerage DataQueueHandler

* Cleanup OrderTypes & rm global.json

* Update UpdateOrderRequest.cs

* Removal of custom csvreader

* Implementing additional method CanPerformSelection

* Add LiveOptionChainProvider

* Clean up and add QC license headers

* Quick cleanup

* Update ZerodhaBrokerage.cs

* Fix OrderEvent timezone conversions

* use item.Unrealised for Intraday position holdings

* Refactor Option chain get instruments calls

* Refactor Option chain get instruments calls

* Review fixes

* Remove whitespaces

* Updating options strike price

* Undo time stamp change

* Optional gethistory

* Addition of comments and minor changes

* Build Fixes

* Add comments for LogType enums

* Options fix

* Fix json name in CsvInstrument

* Update tick generation in DataQueueHandler

- Use Timestamp field for both trades and quotes
- Fix incorrect bid/ask sizes
- Avoid reading depth on each tick to find top quote
- Use new Tick constructors
- Remove unnecessary locking

* Add missing null check in Utils.StringToDate

* Order fixes

- Include TriggerPending orders in GetOpenOrders
- Remove unnecessary invalid check in PlaceOrder
- Fix GetOrderPrice and GetOrderTriggerPrice

* Zerodha account balance fetch

* Removal of Futures and Options

* Remove Futures & Options Support

* Update ZerodhaBrokerage.cs

* Update ZerodhaBrokerage.cs

* Add License headers

* Add License Headers

* Refactor

* historical TradeBar & security fee calculation fixes

* fix brokerage test case

* Update ZerodhaBrokerage.cs

* Update ZerodhaBrokerageHistoryProviderTests.cs

Co-authored-by: Aman Ray <ray.aman9876@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
11058
2021-03-08 18:51:54 -03:00
Martin-Molinero 5dcda56a73 Revert "Revert "Upgrade IBAutomater to v1.0.51 (#5359)" (#5362)" (#5365)
This reverts commit c7fb6165e2.
11045 11049 11050 11051 11052
2021-03-08 09:54:50 -03:00
Jasper van Merle 8076782ea6 Make random data generator save non-equity data (#5371) 11044 2021-03-08 09:49:17 -03:00
hsm207 e869765759 Fix typo (#5361) 11035 11036 11037 11023 11024 11026 11027 11029 11030 11033 11034 11028 11031 2021-03-04 15:28:21 -03:00
Martin-Molinero c7fb6165e2 Revert "Upgrade IBAutomater to v1.0.51 (#5359)" (#5362)
This reverts commit caedddddf0.
11021
2021-03-04 15:20:15 -03:00
IlshatGaripov d9d02bc2d0 Applies a fix (#5358) 11015 11016 11018 2021-03-04 13:13:41 -03:00
Stefano Raggi caedddddf0 Upgrade IBAutomater to v1.0.51 (#5359) 11013 2021-03-04 13:03:31 -03:00
Martin-Molinero 4e00324b57 Bug insight internal to procted internal (#5357)
* Update Insight.cs

* Insight protected to protected internal

Co-authored-by: Jared <jaredbroad@gmail.com>
11008 11011 11012
2021-03-03 16:14:36 -08:00
Martin-Molinero 1b6301d813 Update Insight.cs (#5356) 2021-03-03 16:02:57 -08:00
Jared d24695856d Update Insight.cs (#5355) 2021-03-03 15:17:25 -08:00
hsm207 17e13665eb Fix typo (#5353) 10993 10995 10996 10998 10999 10992 10997 2021-03-03 10:23:05 -03:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
10989 10990 10991
2021-03-02 18:46:31 -03:00
IlshatGaripov 83da0affc5 Implements Coin API HistoryProvider (#5298)
* CoinApi HistoryProvider impl.

* Small fix of ToTradeBar()

* Revert the change : CreateSliceEnumerableFromSubscriptions

* More tests and fixes.

* Removes redundant ToList()

* too much spacing

* Fixes to get data for an unlimited period of time, by means of pagination
10984 10985 10986
2021-03-02 11:08:14 -03:00
Jasper van Merle 60798df64c Make optimizer usable in Docker and improve its logging (#5344)
* Make optimizer usable in Docker and improve its logging

* Wait for reset event instead of polling state
10965 10966 10967 10968 10971 10974 10975 10976 10977 10978
2021-02-26 15:50:16 -03:00
Martin-Molinero bb20f9d9df IB will not load holdings internally if requested (#5342) 10954 10955 10956 10957 10958 10959 10960 10961 10962 10963 10964 2021-02-25 11:43:38 -08:00
Martin-Molinero 32686db739 NotificationJsonConverter is case insensitive (#5341)
- NotificationJsonConverter will be case insensitive. Adding unit test
10953
2021-02-25 11:42:42 -08:00
Stefano Raggi 7d97f05133 Tradier Brokerage updates (#5326)
* Add options support

- Added IDataQueueUniverseProvider implementation
- Fix subscribe ticker for options

* Tradier DataQueueHandler web sockets implementation

- Equities and Options (trades and quotes)

* Add error handling

* Add support for options

* Fix log messages

* Address review

- Updated Tick constructor usage
- Use $empty$ symbol for last unsubscribe message
- Added symbol mapper unit tests
10946 10947 10948 10944 10945
2021-02-25 10:41:47 -03:00
Colton Sellers 8b173f2306 Add dotnet 5 to Lean Foundation Image (#5307)
* Add dotnet 5 to foundation image

* Address review. Remove net5 package after install

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10942
2021-02-24 21:09:59 -03:00
Jasper van Merle 9cde4344fc Re-apply changes from #4870 to fix ThreadAbortException in ToolBox (#5339) 10941 2021-02-24 20:50:56 -03:00
Martin-Molinero 785c706d4c Live delisting will only ask mapfile for equities (#5336)
* Live delisting will only ask mapfile for equities

* Remove unrequired log error
10930 10937 10938 10940 10936
2021-02-24 14:14:50 -03:00
Gerardo Salazar 0dc521f787 Reduces log spam on long backtests when generating capacity (#5334) 10920 10923 10925 10926 2021-02-23 16:22:49 -08:00
Martin-Molinero 94d9996b21 Enforce USD account currency if allocation is limited (#5333)
* Enforce USD account currency if allocation is limited

- Enforce USD account currency if allocation is limited. Adding unit
  test

* Adjust currency log message
10909 10911 10912 10913 10917 10918 10916
2021-02-23 11:57:48 -08:00
Martin-Molinero 67b3873205 Reduce live coarse selection interval check to 10 min (#5332)
* Reduce live coarse selection interval check to 10 min

- Reduce live coarse selection interval check to 10 min. Adding unit
  tests, reducing code duplication in tests.

* Adjust unit test after self review
10908
2021-02-23 16:37:08 -03:00
Gerardo Salazar acd9ef4cfd Removes SevenZipSharp library from ToolBox (#5329)
* Removes SevenZipSharp library from ToolBox

  - Library removal required for update to .NET 5.0 since it's not
    compatible with it and Linux is unsupported.

  * Adds new extract 7z functionality to Compression project
  * Refactors AlgoSeekFuturesConverter 7z extract
  * Refactors DukascopyDataDownloader 7z extract

* Addresses review: code cleanup + exception type change on timeout

  * Removes not needed stream of data in Dukascopy downloader
  * Makes output directory non-optional
10901
2021-02-23 11:38:07 -03:00
Gerardo Salazar 7f934b2adf Remove unused dependency WebSocketSharpFork (#5331) 10900 2021-02-23 11:33:17 -03:00
Martin-Molinero 596e940ccd Update R library (#5328)
* Update R.Net libraries

* Update R installation version
10886 10898 10899
2021-02-22 19:24:05 -03:00
Gerardo Salazar f38ff0d1f2 Reduces StrategyCapacity memory load and boosts performance (#5327)
* Adds logging for progress tracking
  * Reads data one day at a time rather than loading all data at once
10869 10885
2021-02-22 11:53:12 -08:00
Gerardo Salazar 37d26a35ce Fixes various bugs in estimated capacity class (#5324)
* Fixes potential mapping issue, caused by not mapping Symbols before
    reading data off disk

  * Fixes issue where some results would evaluate to zero capacity

  * Loads of refactoring, mostly resulting in cleaner code
10864 10862
2021-02-19 18:00:08 -08:00
Martin-Molinero 834326760d Fix regression test History Request (#5321)
- After https://github.com/QuantConnect/Lean/pull/5207 history requests
  are selecting the correct data type and cause the regression test to
  fail because there is no QuoteBars for daily equity. Updating and
  fixing this.
- Centralizing the logic around getting the SubscriptionDataConfig to
  use
10859
2021-02-19 20:39:50 -03:00
Martin-Molinero d4ebd93586 Limit live cash allocation (#5320)
* Add brokerage data cash limit feature

* Expand unit test to asser orders are skipped

* Add algorithm debug messages

* Ignore account changed event
10855
2021-02-19 20:08:32 -03:00
Gerardo Salazar 0645513f5e Capacity estimation in report generator (#5318)
* Adds strategy capacity estimation in report generator

* Style and license fixes

* Adds comments, constants, and new method accessibility

* Removes unneeded imports
10840
2021-02-18 17:51:40 -08:00
Martin-Molinero ecf91546e6 Update Nlog Math and fasterflect (#5310)
* Update Nlog Math and fasterflect

* Remove NLog
10835
2021-02-18 20:18:07 -03:00
Martin-Molinero c834a1c902 Revert Lean.Launcher nuget package rename (#5317) 10834 2021-02-18 19:39:05 -03:00
Colton Sellers 43a540cbb1 OpenInterest Bug Fixes (#5207)
* Filter values that are before subscription start time; also adjust starttime for OpenInterest

* Use data EndTime for comparison

* Allow Auxiliary data through

* Fix OpenInterest DataReader Logic

* Add regression

* Address review

* Ignore open interest for time slice

- TimeSliceFactory will directly ignore open interest for determining if
  the slice has data or not. Open interest will still be available
  through the Tick collection. Reverting some of the previous commits
  changes since they are no longer required.
- HistoryRequests and SubscriptionRequest will use AlwaysOpen exchange
  for open interest requests. Adding unit test reproducing issue
- Adding `BaseDataRequest` to avoid duplication logic.

* Make OpenInterest an internal feed and ignored by default in history

- Adding unit tests

* Revert SubscriptionFilterEnumerator Start time addition

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10829 10830
2021-02-18 19:04:29 -03:00
Alexandre Catarino c5bb840bde Adds Static AlgorithmCreationTimeout Property to BaseSetupHandler (#5316)
* Adds Static AlgorithmCreationTimeout Property to BaseSetupHandler

This value will be used to set maximum time that the creation of an algorithm can take.

* Adds Additional Logging to AlgorithmPythonWrapper

It will informing the user how long it takes to import the module.

* Changes How AlgorithmCreationTimeout is Initialized

It will prevent callling Config.GetDouble on every call.
10828
2021-02-18 19:00:33 -03:00
Martin-Molinero 4632840da9 Add license file for project packages (#5314) 10826 2021-02-18 18:57:36 -03:00
Martin-Molinero ba4c45d729 Remove version checking (#5315) 10824 2021-02-18 18:32:07 -03:00
Martin-Molinero 9232427663 Remove all nuspec files (#5311) 10820 2021-02-18 17:13:11 -03:00
Martin-Molinero c30929f612 Add support for live trading delisting events (#5195)
* Add support for live trading delisting events

- Adding `LiveDelistingEventProviderEnumerator` that will emit delisting
  events. Adding unit tests
- Remove unused `LiveAuxiliaryDataEnumerator`.
- Refactor the corporate event providers being used per security type

* Self review: compare date component

* Add new LiveDataBaseDelistingEventProvider for equities

- Add support for delisting events for equities. Adding unit tests

* Fix cleanup of delisting equity subscription
10801 10806
2021-02-17 19:53:53 -03:00
Martin-Molinero df6e133833 Fixes for assembly information (#5306)
- Update version to 2.5
- Normalize version and assembly information
- Remove commented and unrequired code
10793
2021-02-17 16:42:23 -03:00
Juan José D'Ambrosio 2d99afc8c3 Refactor IPrimaryExchangeProvider.GetPrimaryExchange (#5302)
* Refactor IPrimaryExchangeProvider.GetPrimaryExchange 

now it returns a PrimaryExchange instead of a string

* Update MapFilePrimaryExchangeProvider.cs

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
10776 10777 10778 10780 10781 10783 10790 10791 10775 10779 10792
2021-02-17 12:43:41 -03:00
Gerardo Salazar b3835510d0 Changes calculation of rolling/series graphs to use EOD equity (#5303)
* To maintain consistency between calculations, we will use
    the end of day equity value to calculate the returns per day.
    This fixes a bug where daily equity series would zero out and result
    in an exception being thrown since no elements were being passed to
    the Sharpe calculation.
10766 10772 10773 10774
2021-02-16 13:43:27 -08:00
Stefano Raggi 239fa29bf3 Tradier Brokerage updates (#5265)
* Tradier brokerage updates

- Removed old authentication code (refresh tokens) and settings
- Added "tradier-use-sandbox" config setting

* Fix brokerage unit tests

* Update Tradier config.json settings

* Add sandbox check in Subscribe

* Bug fixes + unit test updates

* Trigger build

* Fix Tradier fee model

* Address review
10758
2021-02-16 13:37:46 -03:00
Juan José D'Ambrosio df4abd3c3a Bug algoseek taq writes tick exchanges correctly (#5301)
* Implements wrapper to get the exchange as a single letter representation

Also replace lower 'l' with capital 'L' in condition flags enums. it is a new warning on the compiler and it looks cool :)

* There are cases where the exchange is null or empty, in those cases return null

* Add tests for single character exchange representation

* Update GlobalTest.cs

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-02-16 12:51:08 -03:00
Gerardo Salazar 7d93f43d38 Calculate more report elements using LiveResult data in report generator (#5300)
* Calculate PSR for live algorithms in report generator

* Use live series for calculating of CAGR, Max DD, and PSR

  * Removes Kelly Estimate from report and template.html

* Update template.html

* Rolling Beta bug fix when using daily data

  * Max drawdown output as percent

* Make days live report element use equity curve last day instead of today

Co-authored-by: Jared <jaredbroad@gmail.com>
10753 10754
2021-02-15 19:56:52 -08:00