Commit Graph

21 Commits

Author SHA1 Message Date
Martin-Molinero 9cb2452025 Oanda default Forex Market (#4706)
* Oanda default forex Market

- Use Oanda as default forex Market since it has more pairs.
- Remove FXCM data add Oanda equivalente data.
- Update unit and regression tests

* Address reviews

- Revert FXCM data removal
- Remove unrequired commented code

* Fix rebase
2020-09-14 16:43:23 -03:00
Martin-Molinero 9cdb4a91c5 Refactor live data feed (#4636)
* Live Coarse universe refactor

- Live trading will source Coarse and Fine fundamental data directly
  from disk. Updating unit tests.

* Adds ILiveDataProvider interface

  * Adds wrapper for IDataQueueHandler implementations

  * Replaces IDataQueueHandler with ILiveDataProvider in
    LiveTradingDataFeed

  * Edits IDataQueueHandler documentation

* Maintains aggregation for current IDQH impls and skips for ILDF impls

  * Note: No unit test was created for this method, go back and TODO

* Protobuf Market data

- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
  unit tests.

* Adds unit tests for LiveDataAggregator changes

  * Fixes bug where custom data was not handled as it was before
  * Fixes race condition bug because of variable reuse in class

* Add protobuf extension serialization

* Fixes for protobuf serialization

* Refactor

* Fix OptionChainUniverse

* replace BaseDataExchange pumping ticks with consolidators

* AlpacaBrokerage

* BitfinexBrokerage

* GDAXBrokerage

* OandaBrokerage

* InteractiveBrokers

* TradierBrokerage

* FxcmBrokerage

* PaperBrokerage

* etc

* WIP fixes for existing LTDF unit tests

* Fixes more LTDF unit tests

* make IDataAggregator.Update recieving Generic BaseData rather than Tick

* Change IDataQueueHandler.Subscribe method

* Some fixes after adding new commits

* Adds protobuf (de)serialization support for Dividend and Split

* Serialize protobuf with length prefix

* Fix missing LTDF unit tests

* Adds TiingoNews protobuf definitions

* fix comments

* more fixes on IQFeedDataQueueHandler

* disallow putting ticks into enumerator directly

* ScannableEnumerator tests

* fix OandaBrokerage

* AggregationManager unit tests

* fix AlpacaBrokerage tests

* fix InteractiveBrokers

* fix FxcmBrokerage tests

* call AggregationManager.Remove method on unsubscribe

* fix GDAX existing tests

* Fixes, refactor adding more tests for AggregatorManager

* Adds BenzingaNews protobuf definitions and round trip unit test

* Adds missing TiingoNews unit test to Protobuf round trip tests

* Improve sleep sequence of LiveSynchronizer

* need start aggregating first, and then can subscribe

* More test fixes and refactor

- Refactoring AggregationManager and ScannableEnumerator so the last is
  the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer

* Improve performance of LEquityDataSynchronizingEnu

* Add missing Set job packet method

* Minor performance improvements

* Improvements add test timeout

- Improvements adding test timeout to find blocking test in travis

* Improve aggregationManager performance

* Testing improvements for travis

* Remove test timeouts

* More test fixes

- Adding more missing dispose calls and improving determinism

* fix IEXDataQueueHandler and tests

* Final tweaks to LTDF tests

* more AggregationManager tests

* consume and log ticks

* fix test: couldn't subscribe to Forex tickers

* change Resolution for all bar configs

* Improve RealTimeScheduleEventServiceAccuracy

* refactoring: move common code to base class

* fixed bug; unsubscribe SubscriptionDataConfig

* Small performance improvement

* Minor fixes

* Avoid Symbol serialization

* Fixes coarse selection in live mode

* Fix for live coarse

* Adds protobuf (de)serialization support for Robintrack

  * Adds round-trip unit test

* Minor performance improvements

* More minor performance improvements

* pass LiveNodePacket through to OandaBrokerage

* Fixes empty list becoming null value when deserializing with protobuf

* Reverts BZ live trading exception removal and fixes tests

* Refactor WorkQueue making it abstract

* Add try catch for composer

* Adds optional data batching period to LiveFillForwardEnumerator

* Override data-queue-handler with config

* Improve PeriodCountConsolidator.Scan performance

* Move batching delay to main Synchornizer thread

* Reverts addition of Robintrack protobuf definitions

* Give priority to config history provider if set

* Add Estimize protobuffing

- Add Estimize protobuffing support. Adding unit tests

* Always dispose of data queue handler

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
2020-08-18 20:21:10 -03:00
Martin Molinero bcc67d2457 DataPermissionManager is used by history provider
- DataPermissionManager will be used by the history provider
- Moving interfaces from Engine to Common
2020-06-12 12:51:14 -03:00
Martin Molinero 0672d2082d Add resolution deserialization unit test 2020-06-12 12:34:10 -03:00
Martin Molinero 452bf27bd9 Adding new DataPermissionManager
- Adding new DataPermissionManager that will own the existing
  datachannelProvider. Will assert configurations before added to the
  data feed. Adding unit tests
2020-06-12 12:34:10 -03:00
Martin Molinero e782ffa4c6 Reuse existing FileAccess permissions 2020-06-09 18:13:12 -03:00
Martin Molinero e73e927c0f Enable parallel unit testing 2020-04-27 17:45:53 -03:00
Martin Molinero 25ecb55732 Fix rebase 2020-04-06 10:57:06 -03:00
Martin Molinero 499248fe12 Revert "Revert "Adding Support for Equity L1 Quote Data""
This reverts commit 8cd8d206ca.
2020-04-06 10:32:59 -03:00
Jack Simonson c6767780c2 Update regression test statistics 2020-03-26 18:29:29 -07:00
Jared 8cd8d206ca Revert "Adding Support for Equity L1 Quote Data" 2020-03-11 19:16:05 -07:00
Martin Molinero d5d00f8472 Update tests
- Update regression tests
- Add missing data
- Revert Messaging.cs change
- Update unit tests
2020-03-11 14:34:16 -03:00
Gerardo Salazar 6f5e394d52 Address Martin's review - test modifications and cleanup
* Get rid of `previousTime` and use `time` instead in AlgorithmManager
* Refactor variable names in Backtesting and Live IResultHandler impls
* Moves shared variables to BaseResultHandler
* Modifies BacktestNodePacketTests statistics to get tests passing
* Adds new StatisticsBuilder tests
* Modifies BacktestingResultHandler tests to make them passing
    - Regarding these tests, the decision was made to get them
      passing so that if any behavior changes, we will know immediately.

Next commit will contain regression test changes for easy rollback.
2020-01-16 14:03:54 -08:00
Martin Molinero b4b2f90303 AddUniverse will be deferred to EndOfTimeStep
- `AddUniverse` call will add new Universe to the pending collection which
will be consumed at the `OnEndOfTimeStep` where it will be added to the
data feed, same as we do for the `UserDefinedUniverses`. This is
required since the start and end date, during initialize, is consumed by
these universe subscriptions.
-Adding unit test.
2020-01-09 20:39:00 -03:00
Martin Molinero 87bc03d71e Address reviews
- When BacktestNodePacket has the inital `CashAmount` set we will clear
all existing cash amounts and set the account currency
- Adding more unit tests
2019-12-31 13:25:58 -03:00
Prasad Somwanshi a1a2c494cb Modifed code for failing unit test 2019-12-31 13:13:05 -03:00
Prasad Somwanshi 9649da3ee6 Added optional property to BacktestNodePacket 2019-12-31 13:13:05 -03:00
Martin Molinero 417590ddc2 Add unit test for BacktestNodePacker 2019-11-08 17:51:33 -03:00
Martin Molinero 9afe74a618 Fix rebase 2019-11-08 17:51:33 -03:00
Martin Molinero 2aeed3595d Algorithm will respect job dates if present
- Console and Backtesting setup handler will use job dates if present
(not the brokerage setup handler)
- Adding unit test
2019-11-08 17:51:33 -03:00
Martin Molinero f887c42638 Enable python and CSharp debugging
- Adding `DebuggerHelper` class, handles debugging initialization
- Setting the "PYTHONPATH" will be handled by the `JobQueue`
2019-08-12 22:03:13 -03:00