11096 Commits

Author SHA1 Message Date
Colton Sellers d45ceea5db Apply packet type
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12633 12639 12640 12641 12643
2021-08-12 16:34:49 -07:00
C-SELLERS 705ebb009d Reduce Foundation Image for a DefaultVenv 2021-08-12 16:34:49 -07:00
C-SELLERS 1a03fa686f Address review 2021-08-12 16:34:49 -07:00
C-SELLERS 37da038f86 Force format of venv dir, allows /dir/ and /dir 2021-08-12 16:34:49 -07:00
C-SELLERS 558fd410b2 Post rebase fix 2021-08-12 16:34:49 -07:00
Colton Sellers 2f74672bd0 Only install VENV shell for library mount at runtime 2021-08-12 16:34:49 -07:00
C-SELLERS fda01be1bf nit - config comment 2021-08-12 16:34:49 -07:00
Colton Sellers 2963b3cbd8 Use Py Venv for testing if defined 2021-08-12 16:34:49 -07:00
Colton Sellers dea10585ac Fix libDir resolution 2021-08-12 16:34:49 -07:00
Colton Sellers 3572e6e720 Tweak dockerfile 2021-08-12 16:34:49 -07:00
Colton Sellers cb1992242c Cleanup PythonInitializer 2021-08-12 16:34:49 -07:00
Colton Sellers 57e4db5a9c Add venv kernel for research support 2021-08-12 16:34:49 -07:00
Colton Sellers 1a3b46eae0 Expand virtual environment 2021-08-12 16:34:49 -07:00
Colton Sellers 09ba158cdf Address review 2021-08-12 16:34:49 -07:00
Colton Sellers 2f7642be85 Fix foundation building 2021-08-12 16:34:49 -07:00
Colton Sellers 657e46afa5 Refactor solution to use JobPackets for Cloud usage, and directly manipulate PythonPath 2021-08-12 16:34:49 -07:00
Colton Sellers 7ef1a5ced5 Add value to config 2021-08-12 16:34:49 -07:00
Colton Sellers 86a4deeba9 Add virtual environment creation to Dockerfile 2021-08-12 16:34:49 -07:00
Colton Sellers 01b95ff24e Only modify path on the first initialize call 2021-08-12 16:34:49 -07:00
Colton Sellers 62e95d9383 Use "python-venv" config value to add additional or overriding libraries to PythonNet path 2021-08-12 16:34:49 -07:00
Stefano Raggi a40329c70c Update IBAutomater to v2.0.46 (#5852)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12623 12624
2021-08-12 11:32:14 -03:00
Alexandre Catarino 6546647e08 Get Best Effort Price For Fill Price (#5855)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`EquityFillModel` will use Trade Tick or TradeBar data if there is no L1 data available leading to filling to stale price.
12622
2021-08-12 10:16:20 -03:00
Martin-Molinero 46829f0f01 Dependency clean up (#5834)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12604 12620 12621
2021-08-11 12:39:00 -03:00
Colton Sellers e722e72aaa Pandas Mapper Recursion Bug Fix (#5848)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust wrapper

* Adjustments

* Add red -> green unit tests

* Address review
12599
2021-08-10 20:32:09 -03:00
Martin-Molinero e92230a7fe Fix SubscriptionDataReader auxiliary data out of order (#5843)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix SubscriptionDataReader auxiliary data out of order

- Fix for SubscriptionDataReader emitting auxiliary data out of order
  due to sparse that and enumerator refresh logic. Adding regression
  algorithm
- Minor tweaks for unit tests failing on and off

* Fix for SDR internal subscriptions

- Fix for SubscriptionDataReader enumerator refresh for internal
  subscriptions which were ignored. Adding regression test
- Updating custom data regression algorithms affected by the issue
	- DropboxBaseDataUniverse was emitting a custom data point being end time
	- UnlinkedTraderBarIconicType was emitting a single data point
	  of the underlying SPY minute data when if should of emitted
	  all data points
12598
2021-08-10 18:57:23 -03:00
Ronit Jain 7881aeb3a5 Websocket-refactor for websocket message data as text and binary (#5833)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* websocket-refactor for websocket message  as text and binary

Initial commit

websocket support for text and binary messageData

code style fixes

test fix

* delete un-used file

* zerodha websocket textMessage parse

* messageData class abscration

* Empty

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
12597
2021-08-10 17:40:07 -03:00
Stefano Raggi 32fdb51ab5 Update IBAutomater to v2.0.45 (#5847)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12594
2021-08-10 12:01:29 -03:00
IlshatGaripov 190fe012f1 Perform update (#5841)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12593
2021-08-10 11:18:23 -03:00
IlshatGaripov 893ca2b41d Makes sure to create directory for results-destination-folder (#5842)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12588
2021-08-09 17:39:57 -03:00
Stefano Raggi c30bd32fb0 Fix IB history requests (#5832)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix IB history requests

- Fix time zone bug causing incomplete results
- Use RegularTradingHours flag only for equities
- Second resolution data is now requested in 30 minute chunks (instead of 1 minute):
https://interactivebrokers.github.io/tws-api/historical_limitations.html

* Filter history result respecting market hours and IncludeExtendedMarketHours flag

* Add history unit tests
12586
2021-08-09 16:57:12 -03:00
Martin-Molinero e720147e50 Algorithm Manager will break when algorithm is deleted (#5840)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
12585
2021-08-09 15:42:13 -03:00
Martin-Molinero b77f0122b2 Improve python exception parsing (#5831)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing

- Improve python exception parsing adding support for line shift. Adding
  unit tests

* PythonException revert change

* Centralized and normalize algorithm runtime handling

* Adding support for C# line and file exception report
12578
2021-08-06 20:11:54 -03:00
Colton Sellers 646adb9131 Remove Ray from ARM image (#5830)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12569
2021-08-05 19:06:44 -03:00
Colton Sellers e2ae173d99 Feature Python Library: Ray (#5824)
* Add Ray lib to both foundation images

* Update some version failing tests
2021-08-05 17:55:51 -03:00
Stefano Raggi 1cfc044034 Update Lean Foundation docker files to IBGateway v985.1j (#5828) 2021-08-05 17:55:30 -03:00
Alexandre Catarino 9b7af08b3e Fixes Market On Open Fill of Equity Fill Model (#5679)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fixes Market On Open Fill of Equity Fill Model

Only use trade data (Tick with Trade type or TradeBar) to get the open price, since MOO is filled with the opening action price. Ensure that this method doesn't use trade data from before the market opens for high-resolution data case.

Fix unit tests to show that the new implementation only fills with trade data from the current open market.

Change regression tests to reflect the bug fix.
In the `ExtendedMarketHoursHistoryRegressionAlgorithm`, MOO was filled with extended market hours.

* Fix Bug for Tick Resolution Case

For tick susbcription, the tick with the open price information is the the first valid (non-zero) tick of trade type from an open market.
Addresses peer-review by moving the if-condition for data belonging to the open market where the subscriscribed types are checked.

* Fix Bug for Low Resolution Edge Case

For the edge case where the order is placed after the trade bar is open, for example, order places at 1 pm with daily-resolution data. The fill model will not use the open of the bar that will close at midnight, since this value is prior to the order.

Adds unit test.

Change regression tests to reflect the bug fix.
In the `RegressionAlgorithm`, MOO was filled with open prior to the order. The algorithm now has one order less, since the last MOO would need to wait another day to be filled.

* Implements SaleCondition and Exchange Check For Tick

- Adds additional unit tests for MOO

* Fixes Regression Test in DataConsolidatorPythonWrapperTests

* Addresses Peer-Review

- Adds new unit test cases.
12559 12557
2021-08-05 12:53:04 -03:00
Stefano Raggi 1a02fbaae5 Add autorestart for Binance data connections (#5827)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12551 12552 12553
2021-08-05 10:54:50 -03:00
Alexandre Catarino ed835faf35 Change Binance Benchmark to BTCUSDC (#5823)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`BTCUSD` is not supported by Binance (see Symbol Properties Dabase)
12549
2021-08-04 21:45:03 -03:00
Gerardo Salazar 5ea70e2a89 Adds support for CoinAPI Binance processing (#5819)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12538 12539 12540 12541 12545 12548
2021-08-03 21:17:23 -03:00
Martin-Molinero 2603694263 C# research will pre load all QC dlls (#5820)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12537
2021-08-03 21:14:59 -03:00
Stefano Raggi 4a3a7a44f5 Update Lean Foundation docker files to IBGateway v985 (#5815)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update Lean Foundation docker files to IBGateway v985

* Add apt-get update for ARM foundation

* Update default value of IB gateway

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
12534
2021-08-03 19:13:38 -03:00
Martin-Molinero 76c930a035 Update readme.md
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12525
2021-08-03 11:47:27 -03:00
Martin-Molinero 12f70aab46 Add DataSource place holder (#5817) 2021-08-03 11:30:12 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
12520 12524 12518
2021-08-02 19:24:45 -03:00
Martin-Molinero ef4fea7592 Fix engine initialization (#5814)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12502 12508 12509
2021-07-30 17:33:03 -07:00
Martin-Molinero 3cf9cc7d41 Update RestSharp to 106.12.0t (#5810)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
12495 12496 12497
2021-07-30 17:45:59 -03:00
Colton Sellers a9b634b096 Pandas Dataframe indexing w/ list (#5813)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Wrap df.__getitem__ to map all keys before indexing

* Add reproducing test case
12492 12494
2021-07-30 16:34:03 -03:00
Stefano Raggi 1793add749 Add support for IBGateway v985 (#5812)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add support for IBGateway v985

- Effective in TWS version 985 and later, for US stocks the bid, ask, and last size quotes are shown in shares (not in lots).

* Add missing error log in IB symbol mapper

* Trigger build
12490
2021-07-30 13:55:26 -03:00
Martin-Molinero 41636c94c3 Common engine initialization (#5809)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Create Lean common engine initialization class, which will be called
  by research and Lean launcher
- Fix backtest deserialization bug due to failing to handle decimal
  value
12477 12488 12489
2021-07-29 21:14:17 -03:00
Stefano Raggi 2624b8c6f5 Update IBAutomater to v2.0.43 (#5807)
* Update IBAutomater to v2.0.42

* Update IBAutomater to v2.0.43
2021-07-29 20:53:38 -03:00