Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Make FOPs selection universe file-based for backtesting
* Make FOPs option chains universe file based
* Make Future universe selection file-based like option universe
* Make Future universe selection file-based like option universe
* Abstraction cleanup
* Add FuturesChains API to QC algorithm
Also refactor future chain provider to use the new FutureUniverse instead of zip file names
* Update regression algorithms stats
* Refactor QuantBook option and future history to use new universes
* Fix failing tests
* Fix failing tests
* Fix failing tests
* Minor future chains unit test improvement
* Add futures chains DataFrame property
Also, remove IDerivativeSecurity interface from Future
* Add DataFrame property to FuturesChains class
* Add regression algorithms
* Add regression algorithms
* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api
* Minor fixes
* Reduce number of universe files in repo
* Minor data fixes
* Regression algorithms updates
* Add implicit conversion from FuturesContract to Symbol
Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.
* Improve resolution handling for history requests
* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property
* Refactor data handling in BaseChain and TimeSliceFactory
- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method
* Remove specific constructors and indexers from Chain classes
Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.
* Remove chain cache logic from FuturesChainUniverse
* Refactor class and interface names for clarity
Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.
* Add base class for options and futures contracts
- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface
* Add minor fix for future options tickers parsing
Added tests
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Clean chain provider classes up
* Remove ZipEntryName other classes and unused code
Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.
* Minor changes and cleanup
* Trigger Build
* Trigger Build
* Refactor FuturesContract data handling
Forward price data from bars and ticks stored in private fields for improved memory usage
* Fix: use universe data for market data in FuturesContract
* Update regression algorithms stats after rebase
Added HSI futures universe files
* Sort configs by internal flag
Internals go first
* Throw from option universe data filters for future options
Future options IV, Open interest and greeks are not supported for future options
* Minor changes
* Improve some regression algorithms
* Minor fix for failing unit tests
* Update FOPs universe file header
Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.
* Minor unit test fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Add history provider as constructor argument for chain providers
* Update new regression algorithms data points count
* Minor fix for FakeDataQueue
* Add initialize method to chain providers classes
* Minor changes
* Trigger Build
* Trigger Build
* Trigger Build
* Minor fix
* Minor fix
* Trigger Build
* Trigger Build
* Trigger Build
* Trigger Build
* Add logs to ProcessedDataProvider
* Removed test logs
* Minor fix
* Support downloading options and futures universe files from api data provider
* Daily data Time & EndTime Improvement
- Adjust daily data Time & EndTime to actually reflect the time of the
data used, for example US Equity from 9.30 to 4PM. Adding new unit and
regression tests
* Refactor solution to use enumerator
- Refactor daily strict end times solution to be through enumerator
usage, so it applies for history providers too
* Minor fixes
* Revert fill forward enumerator change
- Revert FillForward enumerator causing stats changing, enhancing unit
tests
* Some cleanup
* Improve handling of live trading FF enumerator
- Improve handling of live trading FF enumerator, by adding support for
bars to arrive with a delay so we can handle auction close/option
prices or data providers which might have some delay making the data
available. Adding new unit tests asserting the behavior
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Expand Index option support
- Adjust APIs so that the same underlying can be used
for different options, adding support for SPX weekly options. Adding
regression tests
* Fix IndexOption.IsStandard
* Add IndexOption test
* Feature combo orders
- Add support for combo orders
* Make fill model wait for all grouped orders to emit fills
* Add ComboFill to model multiple fills for combo orders
* Fill combo limit orders
Add some regression algorithms
* Add fill implementation for combo leg limit orders
* Add IFill as common interface for Fill and ComboFill
* Refactor combo orders removing IGroupOrder interface
Move the group order manager to the base Order class
* Update algorithms
* Handle combo order events atomically
* Refactor brokerage transaction event handler
* Refactor combo fill models
* Process fills in batch
* Combo orders fill model tests
* Combo leg limit orders algorithm
* Regression algorithms cleanup
* Fill and combo fill classes cleanup
* Housekeeping
* Refactor equity fill model to derive from base fill model
* Address review changes request
* Handling the new types of orders in the OrderJsonConverter
* Add regression algorithm to test combo orders update/cancel
* Add regression algorithm to test combo orders update/cancel
* Housekeeping
* Address review changes request
* Minor changes
* Security transaction handler method for setting order request id
* Extend public interface for placing combo orders
* Combo order tickets demo algorithm python version
* Tweaks and updates
* Minor fixes
* Minor changes
* Minor fixes
* Address reviews minor fixes
* Minor fixes
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
* Refactor chain providers
- ChainProviders will use quote, open interest and trade information,
which ever is first. Updating regression algorithm changing contract
being added.
- Add base BacktestingChainProvider for shared logic, reducing
duplication.
- ChainProviders will now use the given IDataCacheProvider and
IMapFileProvider
- If Providers are called for a date the exchange is closed they will
search for the previous available date where the exchange is open.
Adding unit tests.
* Address reviews. Logging improvements
* Add set market price during extended market hours
- Set market prices during extended market hours for live trading.
Adding unit test
* Add assert on internal data count
* Add internal subscription manager
- Add InternalSubscriptionManager that will handle internal
Subscription. Replaces the realtime updates
- Fix thread race condition in the TimeTriggeredUniverseSubscription, we
have one thread injecting data points, the main algorithm thread, and
the base exchange is pulling from it
- Fixes for FakeDataQueue
- Adding unit tests
* Address reviews and fixes
- Internal subscription will use extended market hours
- Only sample charts accordingly
- Get api-url once
* Live Coarse universe refactor
- Live trading will source Coarse and Fine fundamental data directly
from disk. Updating unit tests.
* Adds ILiveDataProvider interface
* Adds wrapper for IDataQueueHandler implementations
* Replaces IDataQueueHandler with ILiveDataProvider in
LiveTradingDataFeed
* Edits IDataQueueHandler documentation
* Maintains aggregation for current IDQH impls and skips for ILDF impls
* Note: No unit test was created for this method, go back and TODO
* Protobuf Market data
- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
unit tests.
* Adds unit tests for LiveDataAggregator changes
* Fixes bug where custom data was not handled as it was before
* Fixes race condition bug because of variable reuse in class
* Add protobuf extension serialization
* Fixes for protobuf serialization
* Refactor
* Fix OptionChainUniverse
* replace BaseDataExchange pumping ticks with consolidators
* AlpacaBrokerage
* BitfinexBrokerage
* GDAXBrokerage
* OandaBrokerage
* InteractiveBrokers
* TradierBrokerage
* FxcmBrokerage
* PaperBrokerage
* etc
* WIP fixes for existing LTDF unit tests
* Fixes more LTDF unit tests
* make IDataAggregator.Update recieving Generic BaseData rather than Tick
* Change IDataQueueHandler.Subscribe method
* Some fixes after adding new commits
* Adds protobuf (de)serialization support for Dividend and Split
* Serialize protobuf with length prefix
* Fix missing LTDF unit tests
* Adds TiingoNews protobuf definitions
* fix comments
* more fixes on IQFeedDataQueueHandler
* disallow putting ticks into enumerator directly
* ScannableEnumerator tests
* fix OandaBrokerage
* AggregationManager unit tests
* fix AlpacaBrokerage tests
* fix InteractiveBrokers
* fix FxcmBrokerage tests
* call AggregationManager.Remove method on unsubscribe
* fix GDAX existing tests
* Fixes, refactor adding more tests for AggregatorManager
* Adds BenzingaNews protobuf definitions and round trip unit test
* Adds missing TiingoNews unit test to Protobuf round trip tests
* Improve sleep sequence of LiveSynchronizer
* need start aggregating first, and then can subscribe
* More test fixes and refactor
- Refactoring AggregationManager and ScannableEnumerator so the last is
the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer
* Improve performance of LEquityDataSynchronizingEnu
* Add missing Set job packet method
* Minor performance improvements
* Improvements add test timeout
- Improvements adding test timeout to find blocking test in travis
* Improve aggregationManager performance
* Testing improvements for travis
* Remove test timeouts
* More test fixes
- Adding more missing dispose calls and improving determinism
* fix IEXDataQueueHandler and tests
* Final tweaks to LTDF tests
* more AggregationManager tests
* consume and log ticks
* fix test: couldn't subscribe to Forex tickers
* change Resolution for all bar configs
* Improve RealTimeScheduleEventServiceAccuracy
* refactoring: move common code to base class
* fixed bug; unsubscribe SubscriptionDataConfig
* Small performance improvement
* Minor fixes
* Avoid Symbol serialization
* Fixes coarse selection in live mode
* Fix for live coarse
* Adds protobuf (de)serialization support for Robintrack
* Adds round-trip unit test
* Minor performance improvements
* More minor performance improvements
* pass LiveNodePacket through to OandaBrokerage
* Fixes empty list becoming null value when deserializing with protobuf
* Reverts BZ live trading exception removal and fixes tests
* Refactor WorkQueue making it abstract
* Add try catch for composer
* Adds optional data batching period to LiveFillForwardEnumerator
* Override data-queue-handler with config
* Improve PeriodCountConsolidator.Scan performance
* Move batching delay to main Synchornizer thread
* Reverts addition of Robintrack protobuf definitions
* Give priority to config history provider if set
* Add Estimize protobuffing
- Add Estimize protobuffing support. Adding unit tests
* Always dispose of data queue handler
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
In order to tell to the caller whether the websocket connection is open or not, TryOpenSocketConnection now returns a bool. ApiDataQueueHandler also now operates on BaseData in order to handle all types of securities.
This includes updating all usages of symbol as a security identifier to use the new type.
The type includes a unique field, SID, as well as the current ticker's value. This allows
for consistent addressability while also allowing the ticker to evolve over time with the
mapping changes.
Effort was made to maintain compile and runtime backwards compatibility.
DataFeeds now expose a single bridge (BlockingCollection<TimeSlice>)
- IDataFeed now responsible for all time syncing, DataStream was removed
- BlockingCollection allows for easy throttling of data feed thread without sleeps (internally using semaphore slim)
- Slimmed down IDataFeed interface to not include members unused externally
DataFeeds now use raw IEnumerator<BaseData> instead of subscription data readers directly
- Simplifies fill forward logic (see FillForwardEnumerator.cs and FillForwardEnumeratorTest.cs)
- Allows for composition (raw reader -> fill forward, user/market filters)
- SubscriptionDataReader now handles refreshing source (continuous enumeration)
LiveTradingDataFeed cleaned up to work with time slice bridge
- RealTimeSynchronizedTimer handles pushing time slices into bridge every second, with or without data
- Subscriptions with tick resolution get their data pushed directly into the bridge immediately
Fill forward changes
- Implemented as an IEnumerator<BaseData>
- Fill forward lower res on higher res (daily gets filled forward on second/minute/hour bars)
- Second/minute/hour will not be filled forward on daily bars (at midnight) in line with current behavior
MISC:
- Performance boost in backtesting brokerage via not checking _pending.Count, instead use local bool to determine if there's pending orders
- Added a FakeDataQueue to aid in testing LiveTradingDataFeed
- FileSystemDataFeed now uses CancellationTokenSource to handle thread exits
- Added logging
Jhonathan Abreu