11089 Commits

Author SHA1 Message Date
Martin-Molinero aaba566954 Alpha Streams Improvements. Python Imports (#5874)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Order handling improvements

- Execution model will only trigger market order if they are above the
  minimum order margin portfolio percetage value
- SecurityCache.Reset is complete

* Python Import fixes

- Add regression test for ImmediateExecutionModel minimum order margin
  check
12745 12754
2021-08-24 11:15:48 -03:00
Martin-Molinero bf28a1d313 Add basic template Atreyu algorithm (#5872)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add basic template Atreyu algorithm

- Add C# and Python basic template atreyu algorithm. Show casing how to
  specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use

* Lean Exchange improvements

- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value

* Regression tests fixes
12732
2021-08-23 18:24:44 -03:00
Julio 85a4d3364b Add prefix "oanda-" to access-token (#5870)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Add prefix "oanda-" to account-id
12724
2021-08-23 10:55:31 -03:00
IlshatGaripov eccca2c029 Fix for Trailing Stop Risk Management Model (#5791)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* To fix the referenced issue

* Fixing up TrailingStopRiskManagementModel

* Fix-ups to address review

* Impl. tests + relevant improvements

* Minor logic improvement at first dictionary update

* Regression test small fix

* Adds explicit 'D' suffix for numbers in test case double arr

* Use integer values in test cases

* Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming

* Revert "Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming"

This reverts commit f9cd279f8cc5e9e1ede5b6f1eae5f4266b7dd295.

* Fix up for failing test cases
12717 12720
2021-08-20 15:14:36 -03:00
Martin-Molinero b0314e1c7c Fix for alpha license (#5868)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix organization read alpha license field

* Add import statement for python
12703 12704 12708 12710 12712 12705
2021-08-19 15:51:48 -03:00
Martin-Molinero 3410832ccf Alpha holdings state (#5770)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state

- Alpha result packet will optionally provide the algorithms portfolio
  state

* Rename

* Convert AlphaStreamsPortfolio to data source

* Improvements on AlphaStreams algorithm

* Fix regression tests

* Add unit tests for EW AS PCM and fixing bugs

* Protobuf AlphaStreamsPortfoliot staState

- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM

* Add alpha license to Organization response

* Improvements EW AS PCM respects free portfolio value

* Fixes

- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType

* Fixed and adding more regression tests

- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
  types
- Add support and regression test for algorithm alpha consumer with existing holdings

* Add AlphaStreamsOrderEvent data type
12694
2021-08-19 13:56:18 -03:00
Stefano Raggi d573a0f6c7 BrokerageTransactionHandler - handle OrderStatus.New in order update/cancel (#5867)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12689 12691 12692 12690
2021-08-19 12:01:38 -03:00
IlshatGaripov 7a3d8667c5 More logging for BinanceRestApiClient (#5846)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* More logging for BinanceRestApiClient

* Changing LogLevel to Debug for history providers : Binance, Polygon

* Minor fixes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
12669 12671 12673
2021-08-18 11:25:51 -03:00
Martin-Molinero c3543f844a Add paper trading for Binance (#5859)
- Adding paper trading support for binance
2021-08-18 10:57:45 -03:00
Martin-Molinero 1fdb36ee89 Improve object store logs (#5864)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12667
2021-08-18 10:33:34 -03:00
Gerardo Salazar cd9e451ead Adds Support for ETF Constituent Universes (#5862)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adds support for ETF constituent universes

  * Adds filtering for universe data if it doesn't match the
    universe subscription type

  * Update mapping for ALL underlying Symbols if
    `Symbol.UpdateMappedSymbol(...)` is called. Required to support
    constituent ETF universes that might have mapping events

  * Delistings of composite constituent universe Symbol will result in
    removal of universe securities.

  * Added regression algorithms for ETF constituent mappings (C#/Python),
    along with data required to run locally

  * Refactor universe delistings in SubscriptionSynchronizer -
    big thank you to @Martin-Molinero :)

* Address review: update regression algorithms and add explanatory comments

* Address review: add additional checks to delisting regression algorithms

  * Adds new regression algorithm testing the addition of a universe
    without calling AddEquity() and asserts same behavior
12661
2021-08-17 19:54:29 -03:00
Stefano Raggi b4f6b51ad1 Update IBAutomater to v2.0.52 (#5863)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12657
2021-08-17 17:07:33 -03:00
Stefano Raggi a5e395d298 Update IBAutomater to v2.0.50 (#5860)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update IBAutomater to v2.0.47

* Update IBAutomater to v2.0.48

* Update IBAutomater to v2.0.49

* Update IBAutomater to v2.0.50
12650
2021-08-16 21:08:22 -03:00
Stefano Raggi a40329c70c Update IBAutomater to v2.0.46 (#5852)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12623 12624
2021-08-12 11:32:14 -03:00
Alexandre Catarino 6546647e08 Get Best Effort Price For Fill Price (#5855)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`EquityFillModel` will use Trade Tick or TradeBar data if there is no L1 data available leading to filling to stale price.
12622
2021-08-12 10:16:20 -03:00
Martin-Molinero 46829f0f01 Dependency clean up (#5834)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12604 12620 12621
2021-08-11 12:39:00 -03:00
Colton Sellers e722e72aaa Pandas Mapper Recursion Bug Fix (#5848)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust wrapper

* Adjustments

* Add red -> green unit tests

* Address review
12599
2021-08-10 20:32:09 -03:00
Martin-Molinero e92230a7fe Fix SubscriptionDataReader auxiliary data out of order (#5843)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix SubscriptionDataReader auxiliary data out of order

- Fix for SubscriptionDataReader emitting auxiliary data out of order
  due to sparse that and enumerator refresh logic. Adding regression
  algorithm
- Minor tweaks for unit tests failing on and off

* Fix for SDR internal subscriptions

- Fix for SubscriptionDataReader enumerator refresh for internal
  subscriptions which were ignored. Adding regression test
- Updating custom data regression algorithms affected by the issue
	- DropboxBaseDataUniverse was emitting a custom data point being end time
	- UnlinkedTraderBarIconicType was emitting a single data point
	  of the underlying SPY minute data when if should of emitted
	  all data points
12598
2021-08-10 18:57:23 -03:00
Ronit Jain 7881aeb3a5 Websocket-refactor for websocket message data as text and binary (#5833)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* websocket-refactor for websocket message  as text and binary

Initial commit

websocket support for text and binary messageData

code style fixes

test fix

* delete un-used file

* zerodha websocket textMessage parse

* messageData class abscration

* Empty

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
12597
2021-08-10 17:40:07 -03:00
Stefano Raggi 32fdb51ab5 Update IBAutomater to v2.0.45 (#5847)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12594
2021-08-10 12:01:29 -03:00
IlshatGaripov 190fe012f1 Perform update (#5841)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12593
2021-08-10 11:18:23 -03:00
IlshatGaripov 893ca2b41d Makes sure to create directory for results-destination-folder (#5842)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12588
2021-08-09 17:39:57 -03:00
Stefano Raggi c30bd32fb0 Fix IB history requests (#5832)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix IB history requests

- Fix time zone bug causing incomplete results
- Use RegularTradingHours flag only for equities
- Second resolution data is now requested in 30 minute chunks (instead of 1 minute):
https://interactivebrokers.github.io/tws-api/historical_limitations.html

* Filter history result respecting market hours and IncludeExtendedMarketHours flag

* Add history unit tests
12586
2021-08-09 16:57:12 -03:00
Martin-Molinero e720147e50 Algorithm Manager will break when algorithm is deleted (#5840)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12585
2021-08-09 15:42:13 -03:00
Martin-Molinero b77f0122b2 Improve python exception parsing (#5831)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing

- Improve python exception parsing adding support for line shift. Adding
  unit tests

* PythonException revert change

* Centralized and normalize algorithm runtime handling

* Adding support for C# line and file exception report
12578
2021-08-06 20:11:54 -03:00
Colton Sellers 646adb9131 Remove Ray from ARM image (#5830)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12569
2021-08-05 19:06:44 -03:00
Colton Sellers e2ae173d99 Feature Python Library: Ray (#5824)
* Add Ray lib to both foundation images

* Update some version failing tests
2021-08-05 17:55:51 -03:00
Stefano Raggi 1cfc044034 Update Lean Foundation docker files to IBGateway v985.1j (#5828) 2021-08-05 17:55:30 -03:00
Alexandre Catarino 9b7af08b3e Fixes Market On Open Fill of Equity Fill Model (#5679)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fixes Market On Open Fill of Equity Fill Model

Only use trade data (Tick with Trade type or TradeBar) to get the open price, since MOO is filled with the opening action price. Ensure that this method doesn't use trade data from before the market opens for high-resolution data case.

Fix unit tests to show that the new implementation only fills with trade data from the current open market.

Change regression tests to reflect the bug fix.
In the `ExtendedMarketHoursHistoryRegressionAlgorithm`, MOO was filled with extended market hours.

* Fix Bug for Tick Resolution Case

For tick susbcription, the tick with the open price information is the the first valid (non-zero) tick of trade type from an open market.
Addresses peer-review by moving the if-condition for data belonging to the open market where the subscriscribed types are checked.

* Fix Bug for Low Resolution Edge Case

For the edge case where the order is placed after the trade bar is open, for example, order places at 1 pm with daily-resolution data. The fill model will not use the open of the bar that will close at midnight, since this value is prior to the order.

Adds unit test.

Change regression tests to reflect the bug fix.
In the `RegressionAlgorithm`, MOO was filled with open prior to the order. The algorithm now has one order less, since the last MOO would need to wait another day to be filled.

* Implements SaleCondition and Exchange Check For Tick

- Adds additional unit tests for MOO

* Fixes Regression Test in DataConsolidatorPythonWrapperTests

* Addresses Peer-Review

- Adds new unit test cases.
12557 12559
2021-08-05 12:53:04 -03:00
Stefano Raggi 1a02fbaae5 Add autorestart for Binance data connections (#5827)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12551 12552 12553
2021-08-05 10:54:50 -03:00
Alexandre Catarino ed835faf35 Change Binance Benchmark to BTCUSDC (#5823)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`BTCUSD` is not supported by Binance (see Symbol Properties Dabase)
12549
2021-08-04 21:45:03 -03:00
Gerardo Salazar 5ea70e2a89 Adds support for CoinAPI Binance processing (#5819)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12538 12539 12540 12541 12545 12548
2021-08-03 21:17:23 -03:00
Martin-Molinero 2603694263 C# research will pre load all QC dlls (#5820)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12537
2021-08-03 21:14:59 -03:00
Stefano Raggi 4a3a7a44f5 Update Lean Foundation docker files to IBGateway v985 (#5815)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update Lean Foundation docker files to IBGateway v985

* Add apt-get update for ARM foundation

* Update default value of IB gateway

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
12534
2021-08-03 19:13:38 -03:00
Martin-Molinero 76c930a035 Update readme.md
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12525
2021-08-03 11:47:27 -03:00
Martin-Molinero 12f70aab46 Add DataSource place holder (#5817) 2021-08-03 11:30:12 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
12518 12520 12524
2021-08-02 19:24:45 -03:00
Martin-Molinero ef4fea7592 Fix engine initialization (#5814)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12502 12508 12509
2021-07-30 17:33:03 -07:00
Martin-Molinero 3cf9cc7d41 Update RestSharp to 106.12.0t (#5810)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12495 12496 12497
2021-07-30 17:45:59 -03:00
Colton Sellers a9b634b096 Pandas Dataframe indexing w/ list (#5813)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Wrap df.__getitem__ to map all keys before indexing

* Add reproducing test case
12492 12494
2021-07-30 16:34:03 -03:00
Stefano Raggi 1793add749 Add support for IBGateway v985 (#5812)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add support for IBGateway v985

- Effective in TWS version 985 and later, for US stocks the bid, ask, and last size quotes are shown in shares (not in lots).

* Add missing error log in IB symbol mapper

* Trigger build
12490
2021-07-30 13:55:26 -03:00
Martin-Molinero 41636c94c3 Common engine initialization (#5809)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Create Lean common engine initialization class, which will be called
  by research and Lean launcher
- Fix backtest deserialization bug due to failing to handle decimal
  value
12477 12488 12489
2021-07-29 21:14:17 -03:00
Stefano Raggi 2624b8c6f5 Update IBAutomater to v2.0.43 (#5807)
* Update IBAutomater to v2.0.42

* Update IBAutomater to v2.0.43
2021-07-29 20:53:38 -03:00
Colton Sellers d8a86f182f Pandas Remapper Refactor (#5808)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Refactor pandas mapper to support newer versions

* Clean up remapper

* Finalize deprecated tests and adjust those that should apply to new pandas

* Move mapper to its own file

* Add supporting Py tests and setup instructions

* Add license
12474
2021-07-29 20:10:00 -03:00
Martin-Molinero 0da9d626dc Remove security from all universes holdings it (#5798)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Calling RemoveSecurity() will remove the security from all the
  universes holdings it, there was a race condition here where it would
  just take the first universe and remove it from it.
  Adding regression test adding and removing an option contract and it's underlying
12451
2021-07-28 18:03:09 -03:00
Jovad Uribe 9888081cc0 Relative Daily Volume Indicator (#5747)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Relative Volume Indicator

* Requested changes and renamed indicator

* Made changes, included Resolution

* Added requested changes

* Update

* Update RelativeDailyVolumeTests.cs

* Update RelativeDailyVolume.cs

* Fixed Test Case File

* Update RelativeDailyVolume.cs

* Fixed IsReady flag
12448
2021-07-28 18:02:34 -03:00
Martin-Molinero 611326b140 WebSocketClient improvements (#5799)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Take lock to refresh client to avoid null reference
  exception
- Reduce gate rate
- Use extension methods to await tasks
- Avoid copying the data multiple times
12447
2021-07-28 15:17:08 -03:00
Stefano Raggi 907a9f086c Update BinanceBrokerage to handle more than 512 symbols (#5773)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update BinanceBrokerage to handle more than 512 symbols

* Address review

- fetch symbol weights only if required
- remove code duplication

* Add rate limiting for new connections

* Update WebSocketMessage to include the websocket instance

* Handle resubscriptions on reconnect

* Address review

* Address review

* Remove unnecessary locking

* WebSocketClientWrapper updates

- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close

* Set WebSocketClientWrapper task to LongRunning

* Add missing check in GetHistory

* Fix exceptions with Binance downloader

- closes #5794
12444
2021-07-28 11:10:40 -03:00
Stefano Raggi 0c4e577885 BitfinexBrokerage updates (#5787)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update BinanceBrokerage to handle more than 512 symbols

* Address review

- fetch symbol weights only if required
- remove code duplication

* Add rate limiting for new connections

* Update WebSocketMessage to include the websocket instance

* Handle resubscriptions on reconnect

* Address review

* Address review

* Remove unnecessary locking

* WebSocketClientWrapper updates

- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close

* Set WebSocketClientWrapper task to LongRunning

* Add missing check in GetHistory

* Fix exceptions with Binance downloader

- closes #5794

* Update Bitfinex symbols in symbol properties database

* Update BitfinexBrokerage to use BrokerageMultiWebSocketSubscriptionManager

* Address review

* Remove unnecessary locking

* Remove old channels on resubscription
12443
2021-07-28 11:10:12 -03:00
Marco Grassi e823dfdfb7 fix a typo, and remove all unnecessary semicolumns in Python (#5795)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12434 12432 12431 12435 12436
2021-07-27 16:05:13 -03:00