Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* websocket-refactor for websocket message as text and binary
Initial commit
websocket support for text and binary messageData
code style fixes
test fix
* delete un-used file
* zerodha websocket textMessage parse
* messageData class abscration
* Empty
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates
* Regression algos updated and python algorithms moved to DataSource repos
* Fixes failing unit tests
* Add "LiveDataTypes" field to LiveNodePacket
* Adds Initialize() to IDataChannelProvider
* Adds new extension method to convert
HistoryRequest -> SubscriptionDataConfig
* Address review: Add protobuf definitions for Iconic data types
* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo
* Small adjustments to variable naming and documentation
* Move test files to respective DataSource repos
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* logging and kite connection retry
(cherry picked from commit 9845a2d79e86c338093cadcae7f41b29d897c78c)
* changed thread.sleep time for kite connection retries
* websresponse retry on error - coding style change
* datetome parse to convert time in utc
* indentation code-style fix
* zerodhaDownloader support for dataTimeZone
* removed code redundancy for single conversion without hardcoded TZ
* removed hardcode dataTimeZone and code refactor
* returns exchange TZ to lean and data TZ to local disk from downloader
* Kite historical candle timestamp TZ conversion
* added unit test for kite historical candle
* access modifier correection
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* zerodhaDownloader support for India Market
(cherry picked from commit c36272ef9c4c7eddfff77413d900faab3a2a8654)
* indenation styling fix
* zerodha donwloader
* Market.India support from zerodhaSymbolMapper
* code styling changes
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix coarse universe generator on single processor
* Make sure MaxDegreeOfParallelism value is not less than 1
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* added symbolData class to store exchange and token values of tickers
* support for India Market
(cherry picked from commit 5f629dcb9e72426dc7b560e782cb1fae72861574)
* removed local vscode workspace file
* support for india market instead of nse and other exchanges
* changed hardcoded india values to Market.India
* changed zerodha getcash fromm available to net
* addded edge case handoing with expections
* support for market info (nse, bse) for india market
* exchange info for EmitQuoteTick
* basic testing with c#
* c# algo
* c# testing
* live orders during market open success
* support orderproperties for exchange value for order placement
* added exceptional handling cases
* fix xml descriptions for the files
* fix zerodha-product-type in DefaultOrderProperties
* add basic template for india market
* fix orderProperites for default values
* fix for local data, python implementation not available
* fix access modifiers of SymbolData
* fix variable name case
* fix single check for exchange, call to base
* fix styling, redundent code and access modifers
* fix non static not callable from static
* fix static
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust AlgoSeek Futures price multipliers to quote prices in USD
* Revert changes to AUP scaling
* After a second review, seems that this contract isn't scaled.
The contract multiplier differs from the NYMEX Aluminum Index's
contract multiplier, and so does the minimum price variation.
The minimum price fluctuation seems consistent with CME's
quoted prices, which also claims the contract is quoted in USD.
* Fix futures contract multipliers for some scaled contracts
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add new DownloaderDataProvider
- Moving LeanDataWrite and IDataDownloader to common project
- Add new BaseDataDownloaderDataProvider with concurrency helper method
- Add new DownloaderDataProvider which will use a IDataDownloader or
IBrokerage implementation as data source
* Add support for downloading symbol chains data
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
The Kraken allow queries from pairs of format XBTUSD and XXBTZUSD but it will only return data with the XXBTZUSD causing errors when XBTUSD ticker is specified.
This commit adds a more meaningful error when KDL is called with XBTUSD format.
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* SymbolJsonConverter improvements
- SymbolJsonConverter will be able to deserialize symbols expressed in
user friendly format. Adding unit test
- Adding a couple of helper methods, removing duplicated code
* Fix FOPs expiration year parsing
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Reduce thread contention in coinApi DQH by taking a lock per symbol
versus a global lock.
- Improve tick creating by reducing object being wasted on empty
constructor
Build & Test Lean / build (push) Has been cancelled
* Enable loading live virtual position tracking
- Enable GDAX paper trading configurations
- GDAX will optionally load existing holdings from live job package
- Bitfinex will try loading existing holdings from live job package for
cash account type
* Binance live holdings loading
Build & Test Lean / build (push) Has been cancelled
* Applies a fix
* Fixing the tests to cover downloader func. Fixing the queue handler
* this has to be removed
* -- Fixes -- [WIP]
* Implements the way to upload historical bars in async fashion. [ WIP ]
* Finishes GetEquityTradeTicks() impl.
* Do not break iteration if result is null
* Uses DisposeSafely and fixes a typo
* To use IEnumerable for return type in GetHistoryAsync
* Fixes GetEquityQuoteTicks()
* Removes GetHistoryAsync() and reverts the associated with it changes
* SubscriptionManager -> revert the changes
* PolygonDataQueueHandler -> use SIP timestamp as it keeps the order
* PolygonDataQueueHandler -> use Log.Trace for the no data {date} message
* Fixes GetForexQuoteTicks()
* Introduce ResponseSizeLimitAggregateData
* Fixes up GetCryptoTradeTicks()
* Small fixes for tick processing methods
* Removes launchSettings.json
* Refactor download data logic
- Replace WebClient for HttpClient
- Remove duplicated download logic and parsing for a single static
method
* Update PolygonDataQueueHandler.cs
Remove commented code
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request
* key -> filePath rename and some cleanup
* Refactor
* Add Organizations Endpoints
* Add some organization api wrapper objects
* Address namespace issue
* Reorganize Api Test into seperate files using one ApiTestBase
* Add Organization tests
* Use capitalized "API" test namespace to reduce amount of file changes
* Add License to test base
* Update /data endpoint functions and response objects
* Update ApiDataProvider Logic
* Handle deserialization of organization products
* Simplify converter
* Only throw for equity requests when not subscribed to map/factor files
* Add missing header
* Make arguement exception
* Api adjustments
* Add Zip factor and map file providers
- Common project will now reference Compression project and not the other way
round.
- Adding Zip FactorFile and MapFile providers
* Refactor FactorFileProvider to use DataProvider to fetch files
* Use resulting MinimumDate in construction of FactorFile
* Nit FactorFile comments and arrangement
* Refactor MapFileProviders to use DataProvider for fetching files
* Refactor ZipFileProvider
* Clean up
* Refactor Backtesting Future/Option chain providers to use dataprovider
* Fixes for data/ endpoints and test adjustments
* Response objects adjustments/cleanups
* ApiDateProvider fixes and testing
* Add LocalZipFactorFileTests
* Update ApiDataProvider download test to verify stream is not null
* Implement posting of agreement summary and signed time
* Mark all Api related tests as explicit and document details on running
* Clarify default token on ApiTestBase
* Adjust summary
* Update Api responses for QCC, except org products which are sold in USD
* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests
* Fix multiple markets for ZipFactorFile provider
* Use Symbol as cache key
* Api.cs review
* Dispose of factorFileStream after reading
* Use zip.EntryFileNames
* Address a few reviews
* Few more fixes
* Address Api Review
* Add Job Org id to config
* Minor tweaks
* Compare with invariant culture
* Fixes Option Universe selection
* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider
* Fix research
* Fix null reference exception
* Make duplicate log debug
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Build & Test Lean / build (push) Has been cancelled
* Fixes bug where raw data was incorrectly be filtered when ID was duped
* Filter CoinAPI raw data by distinct Symbol created from the SymbolMapper
* Fixes bug where only quotes or trades would be processed, not both
* Trigger rebuild
Build & Test Lean / build (push) Has been cancelled
* Adds filtering of negative ticks in CoinAPI converter
* Tick data will remain the same, only aggregated data
will have changes reflected in its output as a result
* Adds suspicious tick filtering to ticks
* Address review: Add Crypto Trade Tick suspicious test case
Build & Test Lean / build (push) Has been cancelled
* Implemented AlphaVantageDataDownloader
* Style and documentation changes requested
* Throw exception for unexpected response content
* Added rate limiting to AlphaVantageDataDownloader
* Minor tweaks and rebase
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Build & Test Lean / build (push) Has been cancelled
* Fixes SEC rate limiting issue by reducing clustering of requests
* We add a user-agent header to all outgoing requests for SEC
as they require it to identify the client, otherwise we're rate
limited.
* Some adjustments to how we wait for work to be done, and RateGate
rate of requests. We wait for all requests now/don't use tasks
anymore
* Address review - remove class HttpClient in favor of local instance
Build & Test Lean / build (push) Has been cancelled
* Adds processed data directory to read price data from
* Make coarse universe generator look at data directory before failing to find daily data
* Set coarse generator output of missing daily file to debug log
* Add CoarseUniverseGenerator logs
* Fixes 100 nanosecond increment lookahead bias when parsing large numbers
* Whenever we parse a number that is has precision greater than
DateTime ticks (sub-100 nanoseconds), if we have nanoseconds
between [0, 1000), excluding numbers divisible by 100,
we will have leftover nanoseconds between [0, 100) nanoseconds, but
they won't be factored in to the DateTime calculation, since casting
to `long` only takes the integer component of the number, so we lose
the extra nanoseconds that came with the decimal, and time is set to
the "floored" value without those nanoseconds.
Since .NET `DateTime` type has a limitation of only being able
to represent time in increments of 100 nanoseconds, by not
considering the sub-100 nanoseconds, we introduce a look-ahead
bias of at most 100 nanoseconds/1 tick
* Misc adjustment to make method use `decimal` instead of `double`
for increased precision when parsing large numbers
* Changes CoinAPI data converter to support processing raw files in original directory structure and file name
* Removes Market requirement from CoinAPI data converter
* Remove timeout on decompression of raw AlgoSeek futures data
* Updates SEC downloader to use HttpClient where requests were failing
* For some unknown reason, valid requests to a valid URL were
failing when using WebClient. Changing our requester to
HttpClient fixes the issue, and enables us to leverage
async capabilities where applicable.
* Added fault tolerance to index file downloads, including a
rate limit in case we've been rate limited
* Further refactoring; catches 429 errors, adds missing rategate calls
* Replace all usage of WebClient, force retry for all failures
* Adds optional config value for Benzinga News API key in downloader
* Modifies Estimize Downloader api config name and fixes directory not found bug
* Refactor Estimize to speed up processing time
* Adds ticker limits if desired
* Misc. bug fixes, performance improvements, code cleanup
* Remove debug log statements leftover from previous commit
* Add support for non-tick Index resolutions in LeanDataWriter
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Empty commit
* Lower requests/second for SEC downloader, add missing rategate call
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Update projects to use .NET 5.0, the successor to .NET Core
* Fix ambiguous errors. Add IBAutomator net5
* Remove FXCM
* Upgrade IBAutomater to v1.0.51
ignored, and an empty message aborts the commit.
* Fix rebase
- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy
* Remove unrequired references
* Fixes
- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests
* Don't call ReadKey when input is redirected
* Fix ConsoleLeanOptimizer
* Research fixes
* Update comment
* Add vsdbg to Dockerfile
* Fixes
- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
- Option assignment will be deterministic in the order
- 'Rolling Averaged Population' is calculated using doubles, updating
expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock
* Replace ICSharp with .NET Interactive
* Fixes after rebase
* CSharp research fixes
- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
* Reconcile duplicated code
* Add License header
* CS0219 Fixes: Value assigned, but never used
* CA1507: Use nameof in place of string literals
* CS0108 : Hides Inherited Member; Use new keyword to overwrite formally
* CS0114: Hides inherited member; use override keyword
* CS0168: Variable is declared but never used
* Tests CS1062; using obsolete implicit Symbol -> String; fix via .ToString()
* CS0472: Non Nullable Obj getting Null Checked
* CS0067 Member not used; ignore all cases for future use
* CS00162 : Unreachable code; either removed or ignored for debugging and test cases
* CS0169 Remove non-used fields; ignore those that may be used in future
* CS0414; Field is assigned but never used.
* CS0618; Obsolete properties and members; Only fixes simple ones, rest will have to broken up
* CS0649; Field never assigned too
* CS0659 & CS0661 ; Overwrite operators and equals but not hashcode; I don't really override it but just call base
* Small comment fix
* Cleanup pragma statement
* Add support for Index SecurityType 🚀 (#5364)
* Add Index SecurityType 🚀
* Extend SecurityIdentifier & Lean Data classes with Index support
* Add Index SecurityType 🚀
* Extend SecurityIdentifier & Lean Data classes with Index support
* Fixes
* Added index cross basic template demonstration
* WIP: Prototype index security type for LEAN as non tradable asset
* Re-adds Index entries to MHDB after rebase
* First steps to getting Index Options running
* Looks at any instance where we pattern match for an option type
and replaces it with a generic call to `.IsOption()` for easier
extensibility in the future for additional option security types
* Adds IndexOption security and misc. classes
* Misc. changes, mainly related to any sort of special casing of
equity options and made index options take the same path
* Enables index options data for backtesting
* Adds new index options market hours to MHDB
* Misc. bug fixes for index options
* WIP: add live support for index options and indexes
* Use OptionMarginModel for Index Options because they both use the
same calculation for margin requirements
* Fixes contract not found errors on SPX index options and SPX index in IB
* Turns out index options' last trading day is the day before expiry,
which IB was expecting the last trading day.
* Add index option test cases (temp)
* LiveOptionChainProvider fix, use Symbol vs. ticker
* Description updates to regression algorithms
* Fixes bug in live trading for indexes and index options
* Adds overridable minimum price variation symbol property
* Adds variable sized minimum price variation for index options
* Adjusts symbol properties for index options
* Misc. bug fixes
* Fixes option assignment simulation for European options
* Updates index options regression algorithms (WIP)
* Fixes bug where index option exercise would trade index underlying
* Fixes bugs where SecurityType.Index was getting flagged as tradable
* Regression algorithms updates and addresses review
* Misc. style fixes and refactoring + a few bug fixes
* Updates regression algorithms to run without runtime errors
* Adds data for regression algos
* Sets DefaultOptionStyle on Canonical and support index options
* Update regression algos statistics
* Removes bad line in regression algorithm causing build to fail
* Minor tweaks
* Address review add comment about quoteBar parse scale
Co-authored-by: Balamurali Pandranki <balamurali@live.com>
Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Update to new QC PythonNet
* Update readme
* Remove Python.Runtime config, replaced by env var
* Allow local packages testing in repo
* Address Review
* Add the LocalPackages readme
* Update Jupyter Image
* Update Research ReadMe
Martin-Molinero