* Live Coarse universe refactor
- Live trading will source Coarse and Fine fundamental data directly
from disk. Updating unit tests.
* Adds ILiveDataProvider interface
* Adds wrapper for IDataQueueHandler implementations
* Replaces IDataQueueHandler with ILiveDataProvider in
LiveTradingDataFeed
* Edits IDataQueueHandler documentation
* Maintains aggregation for current IDQH impls and skips for ILDF impls
* Note: No unit test was created for this method, go back and TODO
* Protobuf Market data
- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
unit tests.
* Adds unit tests for LiveDataAggregator changes
* Fixes bug where custom data was not handled as it was before
* Fixes race condition bug because of variable reuse in class
* Add protobuf extension serialization
* Fixes for protobuf serialization
* Refactor
* Fix OptionChainUniverse
* replace BaseDataExchange pumping ticks with consolidators
* AlpacaBrokerage
* BitfinexBrokerage
* GDAXBrokerage
* OandaBrokerage
* InteractiveBrokers
* TradierBrokerage
* FxcmBrokerage
* PaperBrokerage
* etc
* WIP fixes for existing LTDF unit tests
* Fixes more LTDF unit tests
* make IDataAggregator.Update recieving Generic BaseData rather than Tick
* Change IDataQueueHandler.Subscribe method
* Some fixes after adding new commits
* Adds protobuf (de)serialization support for Dividend and Split
* Serialize protobuf with length prefix
* Fix missing LTDF unit tests
* Adds TiingoNews protobuf definitions
* fix comments
* more fixes on IQFeedDataQueueHandler
* disallow putting ticks into enumerator directly
* ScannableEnumerator tests
* fix OandaBrokerage
* AggregationManager unit tests
* fix AlpacaBrokerage tests
* fix InteractiveBrokers
* fix FxcmBrokerage tests
* call AggregationManager.Remove method on unsubscribe
* fix GDAX existing tests
* Fixes, refactor adding more tests for AggregatorManager
* Adds BenzingaNews protobuf definitions and round trip unit test
* Adds missing TiingoNews unit test to Protobuf round trip tests
* Improve sleep sequence of LiveSynchronizer
* need start aggregating first, and then can subscribe
* More test fixes and refactor
- Refactoring AggregationManager and ScannableEnumerator so the last is
the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer
* Improve performance of LEquityDataSynchronizingEnu
* Add missing Set job packet method
* Minor performance improvements
* Improvements add test timeout
- Improvements adding test timeout to find blocking test in travis
* Improve aggregationManager performance
* Testing improvements for travis
* Remove test timeouts
* More test fixes
- Adding more missing dispose calls and improving determinism
* fix IEXDataQueueHandler and tests
* Final tweaks to LTDF tests
* more AggregationManager tests
* consume and log ticks
* fix test: couldn't subscribe to Forex tickers
* change Resolution for all bar configs
* Improve RealTimeScheduleEventServiceAccuracy
* refactoring: move common code to base class
* fixed bug; unsubscribe SubscriptionDataConfig
* Small performance improvement
* Minor fixes
* Avoid Symbol serialization
* Fixes coarse selection in live mode
* Fix for live coarse
* Adds protobuf (de)serialization support for Robintrack
* Adds round-trip unit test
* Minor performance improvements
* More minor performance improvements
* pass LiveNodePacket through to OandaBrokerage
* Fixes empty list becoming null value when deserializing with protobuf
* Reverts BZ live trading exception removal and fixes tests
* Refactor WorkQueue making it abstract
* Add try catch for composer
* Adds optional data batching period to LiveFillForwardEnumerator
* Override data-queue-handler with config
* Improve PeriodCountConsolidator.Scan performance
* Move batching delay to main Synchornizer thread
* Reverts addition of Robintrack protobuf definitions
* Give priority to config history provider if set
* Add Estimize protobuffing
- Add Estimize protobuffing support. Adding unit tests
* Always dispose of data queue handler
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
- Adding new DataPermissionManager that will own the existing
datachannelProvider. Will assert configurations before added to the
data feed. Adding unit tests
- Live and backtesting will send delta order events updates
- Live will store order events every 10 minutes per day
- Backtesting will store last 100 order events on every update and will
store all order events in the end of the backtest
- When BacktestNodePacket has the inital `CashAmount` set we will clear
all existing cash amounts and set the account currency
- Adding more unit tests
Errors raised during persistence aren't able to be handled by user code,
and in fact, are swallowed by the implementation after being logged. By
exposing these errors as events we allow the algorithm to be notified of
such an error and take any step necessary to handle the persistence error.
persistenceIntervalSeconds defines the number of seconds between
each save operation. For the local object store, this dictates
how often the contents of the object store is packaged and written
to disk. The PersistData virtual method is provided for subclasses
to provide a different implementation of how/where to persist the
data. The change to be in-memory aims at keeping the object store
performant with reasonable persistence guarantees.
The job packet is now able to specify a maximum file count as well
as a maximum storage size limit. These controls are enforced in the
LocalObjectStorage implementation.
- Remove chart subscription logic. Will stream all chart updates if any
(wont stream empty updates)
- Only serialize properties which are not null
- Adding Chart and Series `IsEmtpy()` extension. Adding unit tests
- Adding `IDataChannelProvider` that will be used to determine if
streaming should be used
- Adding `data-channel-provider` for `config.json`
- Adding unit tests
We restrict each algorithm time loop to a pre-determined amount of time.
Exceeding this limit will cause the algorithm to immediately terminate.
This quickly becomes an issue when considering users running trainable
models that have a long initialization period that exceeds the time loop
maximum.
This change provides a mechanism through which a long-running scheduled
event is permitted to keep running and is permitted to avoid the time loop
permitted by requesting additional time. Requests for additional time are
limited according to a leaky bucket implementation whose parameters are
set via the job's controls structure. The fundamental time unit for the
algorithm is a single minute.
Here's how it works. If a scheduled event takes longer than one full wall
clock second then a request is made to the leaky bucket for one more minute.
If the scheduled event continues to take more time, it will continue to
request additional minutes. Each requested minute will prevent the algorithm's
time loop check from terminating the algorithm. When the bucket is empty and
no more minutes are available to be requested, a TimeoutException is thrown
causing a cascade that ends in the algorithm's termination and status being
flipped to RuntimeError.
Additionally, this applies equally to ALL scheduled events. While some helpers
were added with the naming of Train and TrainNow to the ScheduleManager, these
methods don't do anything special and the infrastructure doesn't otherwise
flag them as different, so this feature becomes part of the core Scheduled
Event feature set.
Further, the live scheduled events were not touched and are still pending
further discussion regarding the value added by enforcing a time restriction
when simulation time and wall clock time are equivalent.
Fixes#3319
- Removing `using QCAlgorithmFramework = QuantConnect.Algorithm.QCAlgorithm`
- Removing `QCAlgorithmFrameworkBridge`
- Removing `IsFrameworkAlgorithm`
- Making `EmitInsightBasedOnFill` private. Adding new
`IOrderEventProvider` exposing an `event` to which `QCAlgorithm` will
subscribe.
- `AccountType.Cash` algorithms will be allowed to manually trade and
emight insights manually or with alpha model.
The term 'alpha' is used to describe the entire algorithm. Therefore, 'alpha'
produces insights. From this we have things like IAlphaModel, which is the model
defining how insights are produced. We have IAlphaHandler, which defines how the
insights from a single 'alpha' (the algorithm) are managed, analyzed, and stored.
Types closer to the individual prediction level, such as InsightDirection, or
InsightScore relate directly to exactly 1 insight. The distinction between the
two became more clear as we developed the insights API, and from that effort it
was decided to harmonize alpha/insight terminology across the various QC systems.
Martin-Molinero