11 Commits

Author SHA1 Message Date
Martin-Molinero 5c078cd2a9 Remove 'data-directory' usage (#7241)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2023-05-12 18:24:12 -03:00
Alexandre Catarino e8b95b80e1 Polygon Handles Data Subscriptions with One Security Type (#6855)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Handles Data Subscriptions with One Security Type

Polygon sells data subscriptions for security types separately, so we should not test whether the users have all three subscriptions, Equity, Forex, and Crypto. Users just need the type they will use.

Adds new crypto markets, and unit tests for them. Adapt unit test.

* Fixes ToolBox Downloader

Adds `--api-key` argument.
Closes #6687

* Address reviews

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2023-01-12 17:37:29 -03:00
Martin-Molinero b7df632294 Correctly handle Polygon authentication sequence (#6485)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Correctly handler Polygon data queue handler authentication sequence
2022-07-18 13:40:21 -03:00
Ronit Jain 264c3c8374 Composite IDQH - Support multiple live data feeds (#6047)
* initial commit

* Follow IDQH implementation

* Expect a list of data handlers from LiveNodePacket

* Return null if can not subscribe

* refctor to add check for subscription

* initialze null

* Add tests

* Check subscribe retuns null/not-null

* cleanup

* Read all required IDQH credentails to job

* Use CDQH to handle all IDQH instances

* constructor abstraction to call from setjob

* use flag

* remove redundant because derived will call initialize on it

* abstract and initialize from setjob

* handle null enumerators

* get creds from data handlers

* handle single data handler value from data-queue-handler

* Fix to support a json array

* Fix missed constructor call

* change access modifier to access from Tests files

* Add test to get brokerageFactory from dataQueueHandler

* Fix init flag to handle all conditions

* Add docs

* initialize from setjob

* Check if websocket open before using

* change defination of initialzie to include tradier

* clean up

* change defination

* fix wrong api key name

* return empty enumerator

* check websocket open before sending request

* check connection before subscribing

* fix to include more cases

* check websocket open before sending request

* Minor refactoring

* reafctor and use IsConnected

* remove unused

* clean up

* Fix test cases

* reverse change

* include config changes

* connect to websocket from setjob

* check websocket connection from setjob

* clean up

* include condition for IDQH that are not brokerage

* Address review

* Add market check condition before subscribe

* Remove deprecated

* Minor fix for deserializing data queue handler

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-12-03 16:18:41 -03:00
bmello4688 1d8243ecac Add tick type to data downloader get (price data) call in order for a… (#6057)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add tick type to data downloader get (price data) call in order for api's with rate limits to handle not making an api call at all.

* Added DTO object to pass into data downloader get

* Added missing header for new file

* Address review. Minor tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-11-22 12:00:16 -03:00
IlshatGaripov 7a3d8667c5 More logging for BinanceRestApiClient (#5846)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* More logging for BinanceRestApiClient

* Changing LogLevel to Debug for history providers : Binance, Polygon

* Minor fixes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-08-18 11:25:51 -03:00
Ronit Jain 7881aeb3a5 Websocket-refactor for websocket message data as text and binary (#5833)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* websocket-refactor for websocket message  as text and binary

Initial commit

websocket support for text and binary messageData

code style fixes

test fix

* delete un-used file

* zerodha websocket textMessage parse

* messageData class abscration

* Empty

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-08-10 17:40:07 -03:00
Martin-Molinero 24e3a71581 Coinapi improvements (#5672)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Add centralized download extension helper method
- Update coinApi client
2021-06-16 20:30:51 -03:00
IlshatGaripov d23ac25b67 Polygon IO limited aggregate bars per request. Fix. (#5363)
Build & Test Lean / build (push) Has been cancelled
* Applies a fix

* Fixing the tests to cover downloader func. Fixing the queue handler

* this has to be removed

* -- Fixes -- [WIP]

* Implements the way to upload historical bars in async fashion. [ WIP ]

* Finishes GetEquityTradeTicks() impl.

* Do not break iteration if result is null

* Uses DisposeSafely and fixes a typo

* To use IEnumerable for return type in GetHistoryAsync

* Fixes GetEquityQuoteTicks()

* Removes GetHistoryAsync() and reverts the associated with it changes

* SubscriptionManager -> revert the changes

* PolygonDataQueueHandler -> use SIP timestamp as it keeps the order

* PolygonDataQueueHandler -> use Log.Trace for the no data {date} message

* Fixes GetForexQuoteTicks()

* Introduce ResponseSizeLimitAggregateData

* Fixes up GetCryptoTradeTicks()

* Small fixes for tick processing methods

* Removes launchSettings.json

* Refactor download data logic

- Replace WebClient for HttpClient
- Remove duplicated download logic and parsing for a single static
  method

* Update PolygonDataQueueHandler.cs

Remove commented code

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-06-09 10:43:34 -03:00
Adalyat Nazirov 75eac27795 Bug 4600 brokerage unsubscribe symbol granular (#4640)
* bitfinex unsubscribe impl

* IB implementation

* Alpaca Brokerage

* Fxcm Brokerage

* Bitfinex remake

* GDAX Brokerage

* move & rename DataQueueHandler subscription manager to better place for using in ToolBox

* performance tuning

use unified Channel class everywhere
BaseWebsocketBrokerage.GetSubscribed method relies on DQHSubscriptionManager

* TradierBrokerage implementation

* OandaBrokerage implementation

* FakeDataQueue subcription manager

* Fake SubscriptionManager for testing

* CoinApi implementation

* IEXDataQueueHandler implementation

* IQFeed Implementation

* unit tests

* fix SubscribeSymbols params

* thread safe subscribed symbols

* don't need to lock Keys prop, because it's thread safe

* accurate PolygonDataQueueHandler subscribe/unsubscribe methods

* Alpaca isnt DataQueueHAndler anymore

* remove unused variable

* remove redundant hashset

* fix coin api subscribe method

* disclaimer

* prettify code

* race condition?

* fix symbol conversion

* log if unsubscribed; change Channel Id type

* requested changes

* pass GetChannelName func as required parameter

* centralized logs

* use single name for all idqh with no difference in tick type

* change Oanda resubscribe method

* CanSubscribe doesn't change instance state - can be marked as static.

* change Oanda Subscription tracking

* style changes

* bitfinex fix

* fix spelling
2020-09-14 20:11:44 -03:00
Stefano Raggi 375f192f07 Add PolygonDataQueueHandler (#4604)
* Add PolygonDataQueueHandler

* Add history provider and downloader for Polygon

* Add aggregator to PolygonDataQueueHandler

* Address review

- Removed duplication in message classes
- Added public Subscribe/Unsubscribe methods in PolygonWebSocketClientWrapper
- Added history requests for Forex and Crypto

* Address review

- Add security type and market arguments to downloader
- Fix time zone bug in downloader

* Remove unnecessary locks

* Add Polygon history for all resolutions

- Equity: trades and quotes
- Forex: quotes only
- Crypto: trades only
2020-08-27 12:09:20 -03:00