Commit Graph

86 Commits

Author SHA1 Message Date
Colton Sellers 7174fcb9d7 RoundDown fix for PeriodCountConsolidatorBase (#4940)
* Limit rounding interval to 1 day, add SubtractRoundDown and AddRoundUp functions

* fix error message

* Only subtract round down if period is greater than a day

* Tests

* Comment clarification

* Pivot solution to simple fix

* fix error message

* Remove RoundDown/Up limitations; add remarks

* address review
2020-11-17 21:40:10 -03:00
Martin-Molinero 98d3a98656 Inline some methods for performance (#4696)
- Add AggressiveInlining for some methods
2020-09-04 18:13:46 -03:00
Colton Sellers b8674731a5 Feature 2456 custom Python consolidator support (#4637)
* DataConsolidator Wrapper for Python Consolidators

* Regression Unit Test

* Refactor Regression test

* Bad test fix

* pre review

* self review

* Add RegisterIndicator for Python Consolidator

* Python base class for consolidators

* Modify regression algo to register indicator

* unit test - attach event

* Test fix

* Fix test python imports

* Add license header file and null check

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-08-25 17:26:55 -03:00
Martin-Molinero 9cdb4a91c5 Refactor live data feed (#4636)
* Live Coarse universe refactor

- Live trading will source Coarse and Fine fundamental data directly
  from disk. Updating unit tests.

* Adds ILiveDataProvider interface

  * Adds wrapper for IDataQueueHandler implementations

  * Replaces IDataQueueHandler with ILiveDataProvider in
    LiveTradingDataFeed

  * Edits IDataQueueHandler documentation

* Maintains aggregation for current IDQH impls and skips for ILDF impls

  * Note: No unit test was created for this method, go back and TODO

* Protobuf Market data

- Adding protobuf support for Ticks, TradeBars and QuoteBars. Adding
  unit tests.

* Adds unit tests for LiveDataAggregator changes

  * Fixes bug where custom data was not handled as it was before
  * Fixes race condition bug because of variable reuse in class

* Add protobuf extension serialization

* Fixes for protobuf serialization

* Refactor

* Fix OptionChainUniverse

* replace BaseDataExchange pumping ticks with consolidators

* AlpacaBrokerage

* BitfinexBrokerage

* GDAXBrokerage

* OandaBrokerage

* InteractiveBrokers

* TradierBrokerage

* FxcmBrokerage

* PaperBrokerage

* etc

* WIP fixes for existing LTDF unit tests

* Fixes more LTDF unit tests

* make IDataAggregator.Update recieving Generic BaseData rather than Tick

* Change IDataQueueHandler.Subscribe method

* Some fixes after adding new commits

* Adds protobuf (de)serialization support for Dividend and Split

* Serialize protobuf with length prefix

* Fix missing LTDF unit tests

* Adds TiingoNews protobuf definitions

* fix comments

* more fixes on IQFeedDataQueueHandler

* disallow putting ticks into enumerator directly

* ScannableEnumerator tests

* fix OandaBrokerage

* AggregationManager unit tests

* fix AlpacaBrokerage tests

* fix InteractiveBrokers

* fix FxcmBrokerage tests

* call AggregationManager.Remove method on unsubscribe

* fix GDAX existing tests

* Fixes, refactor adding more tests for AggregatorManager

* Adds BenzingaNews protobuf definitions and round trip unit test

* Adds missing TiingoNews unit test to Protobuf round trip tests

* Improve sleep sequence of LiveSynchronizer

* need start aggregating first, and then can subscribe

* More test fixes and refactor

- Refactoring AggregationManager and ScannableEnumerator so the last is
  the one that owns the consolidator
- Adding pulse on the main LiveSynchronizer

* Improve performance of LEquityDataSynchronizingEnu

* Add missing Set job packet method

* Minor performance improvements

* Improvements add test timeout

- Improvements adding test timeout to find blocking test in travis

* Improve aggregationManager performance

* Testing improvements for travis

* Remove test timeouts

* More test fixes

- Adding more missing dispose calls and improving determinism

* fix IEXDataQueueHandler and tests

* Final tweaks to LTDF tests

* more AggregationManager tests

* consume and log ticks

* fix test: couldn't subscribe to Forex tickers

* change Resolution for all bar configs

* Improve RealTimeScheduleEventServiceAccuracy

* refactoring: move common code to base class

* fixed bug; unsubscribe SubscriptionDataConfig

* Small performance improvement

* Minor fixes

* Avoid Symbol serialization

* Fixes coarse selection in live mode

* Fix for live coarse

* Adds protobuf (de)serialization support for Robintrack

  * Adds round-trip unit test

* Minor performance improvements

* More minor performance improvements

* pass LiveNodePacket through to OandaBrokerage

* Fixes empty list becoming null value when deserializing with protobuf

* Reverts BZ live trading exception removal and fixes tests

* Refactor WorkQueue making it abstract

* Add try catch for composer

* Adds optional data batching period to LiveFillForwardEnumerator

* Override data-queue-handler with config

* Improve PeriodCountConsolidator.Scan performance

* Move batching delay to main Synchornizer thread

* Reverts addition of Robintrack protobuf definitions

* Give priority to config history provider if set

* Add Estimize protobuffing

- Add Estimize protobuffing support. Adding unit tests

* Always dispose of data queue handler

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
2020-08-18 20:21:10 -03:00
Martin Molinero c18b1af5ce Address reviews
- Add new comment expanding on the QuoteBar ask and bid fill forward behavior for new bars, which are not a fill forward bar.
2020-06-23 14:50:23 -03:00
Martin Molinero febc4b50ea Fix QuoteBarConsolidator
- TickQuoteBarConsolidator will use previous bar. Updating unit tests
- QuoteBarConsolidator open ask and bid will match previous bar close
  bid and ask. Adding unit tests
2020-06-23 14:42:56 -03:00
Anand Vishnu 79a05ebd27 Quote bar previous close and current open should be same 2020-06-23 14:42:56 -03:00
Aytaç Aşan d27513c4c4 Added Quarterly & Yearly Calendar Types
Calendar Support Expanded for Quarterly & Yearly Calendar Types.
2020-02-14 21:08:29 -03:00
Martin Molinero 8f50691927 Update documentation 2020-02-14 19:59:22 -03:00
Martin Molinero 4951dbdea4 Address reviews 2020-02-14 19:53:39 -03:00
AlexCatarino ceabf93e94 Renames CalendarType to Calendar
Keeps `CalendarType` for backward compatibility.
2020-02-14 19:47:48 -03:00
Martin Molinero 322aa9574d Fix Symbol Equals emtpy
- Fix Symbol Equals Empty implementation
- Improving performance
- Adding unit test
2020-01-16 12:50:30 -03:00
AlexCatarino f615a14155 Fix Consolidators Python timedelta
In `PeriodCountConsolidatorBase`, differentiate the type of the `PyObject` and create an `IPeriodSpecification` accordingly. If the `PyObject` is a C# `Func<DateTime, CalendarInfo>` or a convertable Python method, we create a `FuncPeriodSpecification` with it as a parameter. If the `PyObject` is a `datetime.timedelta`, we create a `TimeSpanPeriodSpecification` with it as a parameter after a conversion to `TimeSpan`.

- Adds example in `DataConsolidationAlgorithm`
- Adds unit test for `timedelta` overload.
2019-12-04 17:59:31 +00:00
Jared 1c4e9379bc Merge pull request #3816 from QuantConnect/feature-3780-consolidator-pyobject
Adds Constructor Overload to PeriodCountConsolidatorBase that Accepts PyObject
2019-12-03 13:44:38 -08:00
Rohan Talip 315c87e24b Minor typo and grammar fixes to comments in PeriodCountConsolidatorBase 2019-11-26 19:19:56 -08:00
Jared ca39929f4b Update PeriodCountConsolidatorBase.cs 2019-11-26 15:34:17 -08:00
Rohan Talip 704e96f0cd Throw and test for an InvalidOperationException instead of a generic Exception. 2019-11-26 04:41:20 -08:00
Rohan Talip dea85cd91d Ensure that consolidators derived from the PeriodCountConsolidatorBase class only consolidate for a single symbol. 2019-11-24 14:01:03 -08:00
AlexCatarino 95c9b9327f Fixes Typos in Consolidators 2019-11-08 23:47:53 +00:00
Prasad Somwanshi 41f456dfef Added unit test for new constructor, corrected access level in TickConsolidator 2019-11-08 22:06:16 +00:00
Prasad Somwanshi 41f9b17015 Added overloaded constructors to classes derived from PeriodCountConsolidatorBase 2019-11-08 22:06:16 +00:00
Prasad Somwanshi e431081337 Added constructors taking PyObject 2019-11-08 22:06:16 +00:00
Michael Handschuh d709d1c4e0 Update Common to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to go through the new
StringExtensions methods that use CultureInfo.InvariantCulture

See #3045
2019-09-12 14:13:24 -04:00
Michael Handschuh 141eaa1fee Add missing license banner 2019-08-16 18:07:30 -03:00
AlexCatarino 0202df35f4 Replaces Enum CalendarType for Static Class with the Same Name
Replaces Enum `CalendarType` for static class with the same name. This class defines two properties (`Weekly` and `Monthly`) that can be used to define the previous calendar date (Monday or 1st of current month) which will correspont to the `Time` of a `IBaseData` object.

Refactor `PeriodCountConsolidatorBase` to define use `GetRoundedBarTime` based on a period specification that depends on the constructor overload: `integer`, `TimeSpan` or `Func<DateTime, CalendarInfo>`. The last one can be set with the `CalendarType` properties.
2019-04-08 22:10:04 +01:00
AlexCatarino 5314e7426d Implements Calendar (Weekly/Monthly) Consolidator
Implements consolidators that group bars by a calendar unit (week or month).
2019-04-08 22:10:04 +01:00
Michael Handschuh 3922dbc886 #2824: Use TickAggregator, add FilteredIdentityDataConsolidator
The existing IdentityDataConsolidator consumes all ticks, completely ignoring
the tick type. I doubt this is ever the desired behavior, but given my
reluctance to break existing regression and unit tests as well as perhaps
user algorithms, I've added a layer on top to provide the proper filtering.
Removed the stub type which mirrored the TickAggregator and also expanded
the TickAggregator implementations to provide full coverage of the possible
ticktype/resolution cases: OpenInterestTickAggregator and IdentityTickAggregator
2019-01-15 14:50:45 -05:00
AlexCatarino feeaf99fb5 Adds check for Pyrhon -> Func conversion 2018-04-17 23:03:28 +01:00
AlexCatarino b0265ba7f5 Adds support to python selectors in Renko Consolidators 2018-04-16 14:39:25 +01:00
AlexCatarino 35e8e0891e Fixes RenkoConsolidator for python
- Missing `event` keyword prevented pythonnet to recognize `DataConsolidated` as a event handler.
- Adds python version of `RenkoConsolidatorAlgorithm`.
2018-04-16 13:39:16 +01:00
Michael Handschuh 99491a9ec7 Add IDataConsolidator.Dispose to remove event handlers
All consolidators now clear the event handlers list when being disposed.
In addition, SubscriptionManager.RemoveConsolidator will now dispose of
the consolidator before returning. This ensures the consolidator and any
downstream indicators that were attached to it can be properly cleaned
by garbage collection.
2018-04-03 14:59:10 -04:00
Stefano Raggi ac6fa2902c Update TickConsolidator to process only TickType.Trade ticks 2018-04-02 21:41:03 +02:00
Ian Worthington 454b3b0d32 Changed logic in QuoteBarConsolidator to match that of TradeBarConsolidator for the period.
Removed check for EndTime on the Count consolidator. Count consolidators need a separate review.
2018-02-20 11:56:53 +09:00
Ian Worthington f6a64a9355 More EndTime tests, fix for TickQuoteBarConsolidator 2018-02-14 17:02:15 +09:00
Ian Worthington a93b39625a Wrote a test to show QuoteBar not properly updating it's EndTime.
Fixed it.
2018-02-14 15:34:09 +09:00
Stefano Raggi f09826c510 Fix RenkoConsolidator not accepting Tick data 2017-08-08 13:21:12 +02:00
Stefano Raggi 508e63af4d Update Bar types Volume and Bid/AskSize from long to decimal
This PR is another required step towards crypto-currency support.
2017-06-26 21:14:55 +02:00
AnshulYADAV, Computer Artist (CoArsey) 350e58567f Fixes 18 warnings with code CS1572 (#908)
* Fixes 5 warnings with code CD1572

* Adds the Xml Comment parameter and implements the missing selector parameter

* Fixes rest of the 13 warnings with code CS1572
2017-05-12 09:28:46 -04:00
AlexCatarino a32a883c29 Tick consolidator minor fix
WorkingBar resolution was not set explicitly and that made it incoherent with the Consolidator, since the default is minute. It causes IDataConsolidator.Scan to emit a bar that was already emited
2017-02-14 13:57:38 -02:00
Andrew Hart 489875de4f Set value of QuoteBar in QuoteBarConsolidator 2017-01-17 15:05:21 -05:00
quant1729 41bfdbcc9d Updated code according to the discussion in PR 2016-12-30 18:13:07 +01:00
quant1729 2df14b7d0d Merge branch 'futures' into master-merge-new-updates
Conflicts:
	Engine/Engine.cs
	Launcher/Program.cs
2016-12-02 12:43:46 +01:00
quant1729 3819561a82 Merge branch 'futures' into master-futures-merged
Conflicts:
	Common/QuantConnect.csproj
	Tests/RegressionTests.cs
2016-12-01 17:32:40 +01:00
Stefano Raggi b0c9c41bba Fix time-based consolidator bug with fill-forward data 2016-11-30 09:04:57 +01:00
quant1729 1305d96edd Open interest support: backtesting, converters, live data handlers (IQFeed/IB), live trading (IB). First cut. Regression Test. 2016-11-15 21:33:33 +01:00
Michael Handschuh 591f6b2127 Use IBaseData in type constraints
Refactors existing consolidators, indicators, and helper methods to depend on
IBaseData instead of BaseData. These updates also defines an IBaseDataBar to
act as an abstraction point between TradeBar and QuoteBar.
2016-11-09 09:20:16 -05:00
Louis S. Berman 34611f07db Merged WickoConsolidator with RenkoConsolidator and related classes, such as tests; got rid of defunct classes 2016-09-25 21:24:15 -04:00
Louis S. Berman 08bd519aa1 Added back the underscores to the private variables from L36 to L44 in WickoConsolidator.cs; also Also restored original RenkoConsolidatorAlgorithm.cs 2016-08-29 10:47:13 -04:00
Louis S. Berman 6d77d3c493 More field naming fixups; hopefully this round finally matches QC conventions 2016-08-28 09:03:30 -04:00
Louis S. Berman 1bb74def66 Removed WickoConsolidator<space>.cs file from commit 2016-08-27 17:13:56 -04:00