Commit Graph

42 Commits

Author SHA1 Message Date
Martin-Molinero 9cdb4a91c5 Refactor live data feed (#4636)
* Live Coarse universe refactor

- Live trading will source Coarse and Fine fundamental data directly
  from disk. Updating unit tests.

* Adds ILiveDataProvider interface

  * Adds wrapper for IDataQueueHandler implementations

  * Replaces IDataQueueHandler with ILiveDataProvider in
    LiveTradingDataFeed

  * Edits IDataQueueHandler documentation

* Maintains aggregation for current IDQH impls and skips for ILDF impls

  * Note: No unit test was created for this method, go back and TODO

* Protobuf Market data

- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
  unit tests.

* Adds unit tests for LiveDataAggregator changes

  * Fixes bug where custom data was not handled as it was before
  * Fixes race condition bug because of variable reuse in class

* Add protobuf extension serialization

* Fixes for protobuf serialization

* Refactor

* Fix OptionChainUniverse

* replace BaseDataExchange pumping ticks with consolidators

* AlpacaBrokerage

* BitfinexBrokerage

* GDAXBrokerage

* OandaBrokerage

* InteractiveBrokers

* TradierBrokerage

* FxcmBrokerage

* PaperBrokerage

* etc

* WIP fixes for existing LTDF unit tests

* Fixes more LTDF unit tests

* make IDataAggregator.Update recieving Generic BaseData rather than Tick

* Change IDataQueueHandler.Subscribe method

* Some fixes after adding new commits

* Adds protobuf (de)serialization support for Dividend and Split

* Serialize protobuf with length prefix

* Fix missing LTDF unit tests

* Adds TiingoNews protobuf definitions

* fix comments

* more fixes on IQFeedDataQueueHandler

* disallow putting ticks into enumerator directly

* ScannableEnumerator tests

* fix OandaBrokerage

* AggregationManager unit tests

* fix AlpacaBrokerage tests

* fix InteractiveBrokers

* fix FxcmBrokerage tests

* call AggregationManager.Remove method on unsubscribe

* fix GDAX existing tests

* Fixes, refactor adding more tests for AggregatorManager

* Adds BenzingaNews protobuf definitions and round trip unit test

* Adds missing TiingoNews unit test to Protobuf round trip tests

* Improve sleep sequence of LiveSynchronizer

* need start aggregating first, and then can subscribe

* More test fixes and refactor

- Refactoring AggregationManager and ScannableEnumerator so the last is
  the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer

* Improve performance of LEquityDataSynchronizingEnu

* Add missing Set job packet method

* Minor performance improvements

* Improvements add test timeout

- Improvements adding test timeout to find blocking test in travis

* Improve aggregationManager performance

* Testing improvements for travis

* Remove test timeouts

* More test fixes

- Adding more missing dispose calls and improving determinism

* fix IEXDataQueueHandler and tests

* Final tweaks to LTDF tests

* more AggregationManager tests

* consume and log ticks

* fix test: couldn't subscribe to Forex tickers

* change Resolution for all bar configs

* Improve RealTimeScheduleEventServiceAccuracy

* refactoring: move common code to base class

* fixed bug; unsubscribe SubscriptionDataConfig

* Small performance improvement

* Minor fixes

* Avoid Symbol serialization

* Fixes coarse selection in live mode

* Fix for live coarse

* Adds protobuf (de)serialization support for Robintrack

  * Adds round-trip unit test

* Minor performance improvements

* More minor performance improvements

* pass LiveNodePacket through to OandaBrokerage

* Fixes empty list becoming null value when deserializing with protobuf

* Reverts BZ live trading exception removal and fixes tests

* Refactor WorkQueue making it abstract

* Add try catch for composer

* Adds optional data batching period to LiveFillForwardEnumerator

* Override data-queue-handler with config

* Improve PeriodCountConsolidator.Scan performance

* Move batching delay to main Synchornizer thread

* Reverts addition of Robintrack protobuf definitions

* Give priority to config history provider if set

* Add Estimize protobuffing

- Add Estimize protobuffing support. Adding unit tests

* Always dispose of data queue handler

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
2020-08-18 20:21:10 -03:00
Adam May 381105b1df Have Config.Write() refer to config being used. 2020-03-04 15:47:36 +11:00
Michael Handschuh f9ddb316e6 Update Configuration to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to use CultureInfo if
possible. This project can not reference QuantConnect.Common, so it's
unable to use the new StringExtensions.

See #3045
2019-09-12 15:30:44 -04:00
IlshatGaripov 176c2738e1 just a small fix. for LeanEngineSystemHandlers defaults to correspond to defaults settings in config.json file. 2019-03-13 11:06:30 +03:00
mblouin02 ef5ecf63cc Minor change: Moved Reset call within Config.cs
Adressing Martin Molinero's comment from this pull request:
https://github.com/QuantConnect/Lean/pull/2251#issuecomment-405359871
2018-07-16 16:01:08 -04:00
Stefano Raggi f5085b7ff6 Add Config.Reset for regression tests 2018-06-19 23:20:41 +02:00
Amir Mamaghani b4044c37d9 *fix* Removed settings logging 2018-05-17 13:20:30 +02:00
Amir Mamaghani 9d449800eb Feature - CLI arguments overrides configuration file + load config from CLI 2018-05-15 21:44:25 +02:00
Amir Mamaghani 4ac2c8cf27 QuantConnect.Configuration - Added ability to choose configuration file during runtime. 2018-05-15 15:40:30 +02:00
Anshul ac2f20de70 Fixes three warnings with code CS0168 2017-06-12 17:59:24 +05:30
mblouin02 8e9ef74b1c Null pointer check added
Added a null pointer check in Config.cs after encountering it.

This was the only unchecked pointer in the function, simply added it.
2016-03-27 13:51:45 -04:00
Michael Handschuh cf6b4aa57d Use BacktestingResultHandler instead of ConsoleResultHandler 2016-01-20 14:05:41 -05:00
Michael Handschuh b8909be3a1 Adds Config.Write 2016-01-19 18:01:28 -05:00
Michael Handschuh 1b8a752451 Provides Config supprt for random types T 2016-01-13 15:50:45 -05:00
snugs 02c18b3d2d Fixes Config.Flatten, was mutating input object 2015-12-31 12:15:55 -05:00
snugs 01a3ebfa11 Fixes ConfigTests.FlattenTest 2015-12-29 13:26:59 -05:00
snugs f2b5beb660 Adds Config.Flatten 2015-12-29 13:00:34 -05:00
snugs 9504955576 Adds Json deserialization support to ConvertTo<T>/GetValue<T> methods 2015-12-10 13:45:39 -05:00
snugs ad3b4a29e7 Change Config.Get to use ToString, more generic 2015-12-10 13:45:38 -05:00
snugs 3fa7642128 Improve Config.GetValue<T>, includes static Parse
The previous impl had a restriction on types that implement IConvertible
This restriction has been lifted and another step added, this time we
search for a static function of the form Type.Parse(string) and if
found invoked using the value from configuration
2015-10-30 15:04:39 -04:00
snugs 77e9e63960 Adds GetToken to config 2015-10-23 15:04:57 -04:00
snugs 941b0c5e51 Update test to be more config agnostic 2015-06-25 15:54:43 -04:00
snugs 6f73a23641 Move resolution of algo handlers after NextJob in engine main 2015-06-23 13:52:06 -04:00
snugs fbfb0515bb Update Config.Set to respect environments via dot pathing
For example, Config.Set("backtesting.live-mode", "false") will set the live-mode value on the backtesting environment
2015-06-23 12:14:22 -04:00
snugs c8b4e6bdba Remove config 'local' value
This change includes addition of the 'tradier-save-tokens' config value used to write tradier access/refresh tokens to disk
2015-06-22 19:05:04 -04:00
snugs 51aba52b6c Use JObject to store config in memory
Updated standard configuration to show usage of environments and preload some for tradier/interactive live trading
2015-06-17 22:34:06 -04:00
snugs 05d42175a4 Add environments to configuation, to make it easier to switch between live/backtesting modes 2015-06-17 22:34:00 -04:00
QuantConnect fde0cbed3a Strip out block quote section header comments 2015-06-13 16:04:27 -04:00
Stefano Raggi 360af61b24 Constants.DataFolder is now a get property that reads the path from config.json 2015-06-08 22:42:08 +02:00
snugs 19862cbe57 Use MEF for result handlers, default to Console impl 2015-06-08 11:46:47 -04:00
snugs 6165c354e7 Initial implementation of TradierBrokerage
Adds new base test class for brokerages: BrokerageTests
2015-05-20 16:37:33 -04:00
snugs 89b6e98af8 Fixes warnings mostly relating to xml comments
Also cleans up some block comments

Via PR#72 - thanks @ammachado!
2015-05-13 10:56:47 -04:00
snugs c908b72cd9 Adds algorithm test runner
Includes RegressionTests.BasicTemplateAlgorithm
Also includes some lines to help produce the dictionary needed for asserting algorithm statistics in ConsoleResultHandler
2015-04-24 15:58:31 -04:00
snugs 28e16d4c3c Wire up log handler based on configuration
Using "local":true for ConsoleLogHandler, false for FileLogHandler, we can later override this with an MEF implementation and be specified by type name like the other MEF plugins.

Improved FileLogHandler to be thread safe and have non-async flush operations
2015-03-31 16:29:35 -04:00
snugs 58e18e7d16 Merge remote-tracking branch 'origin/master' into ib 2015-01-25 10:38:00 -03:00
Jared Broad 87c35d8175 Live mode config and QCAlgorithm Json comment removal 2015-01-24 21:15:20 -03:00
snugs b399bd6654 Adds initial impl of Interactive Brokers Brokerage
The IBBrokerage is event driven, so some types were also added for Account/Portfolio updates. These can later be listened to for updating algorithm properties and such.
2015-01-23 15:59:39 -03:00
Jared Broad 90156a7a17 UTC time sample window bug fix 2015-01-21 12:55:59 -03:00
snugs 2d95be7b4a Updated Config.cs to have generic method/default values 2015-01-20 11:38:09 -03:00
snugs 9340d2f885 Adds GetDouble to Config.cs 2015-01-20 11:17:05 -03:00
snugs dda08d729f Renames IControls to IApi
Also fixes a few configuration bugs/mismatches
QuantConnect.Controls namespace -> QuantConnect.Api
2015-01-19 13:56:50 -03:00
Jared Broad b3de520915 Initial commit 2015-01-12 12:03:33 -03:00