Commit Graph

179 Commits

Author SHA1 Message Date
Martin Molinero 1581801f9b Address reviews 2019-06-11 12:01:57 -03:00
Martin Molinero ec1d410907 Performance improvements
- `SecurityIdentifier` will calculate `StrikePrice` just once. Replacing
`!=` for a direct call to `Equals()` preventing unnecessary checks.
- Slightly improving Linq queries at `OptionFilterUniverse`
- Replace `OpenReader().CopyTo` for `Extract(stream)` avoids copying the
data twice.
- Replace `DateTime.Now` for `DateTime.UtcNow`
2019-06-10 22:40:19 -03:00
Martin Molinero c39638668c Overall performance improvements
- `FactorFile` will keep an ordered reversed list with the dates.
Calling `Reverse()` on the `SortedList` is expensive.
- `MapFiles` will keep first and last date, so we don't need to call
`First()` and `Last()` multiple times.
- `Liquidate` will go through all the algorithms securities only if
necessary
- `TradeBar` parsing will not call `new T` for pure `TradeBar` which is
expensive
- Removing `Lazy` hash code and security type for the
`SecurityIdentifier`, replacing for direct initialization. Accessing the
`Lazy` value adds an overhead.
- Replacing `Enum` to string for hardcoded switch statement. `Enum.ToString` is expensive.
- `DataManager` will be lazy for counting the subscriptions for
determining if its above the limit
- Adding `AlgorithmSecurityValuesProvider.GetAllValues()`, removes the
need to fetch all the security keys twice.
- During universe selection, will not try to re add already added symbol
2019-04-22 10:47:27 -03:00
Martin Molinero b7930aff67 Performance improvements
- Using `Aggregate(lambda)` vs `Sum(lambda)` since the later is slower
due to performing an extra `Select`
- For `QCAlgorithm.Framework.OnFrameworkData()` will avoid calling
`ToArray()` on empty `Enumerables` due to its cost * the number of
calls. If the `Enumerable` is the empty instance, which is static,
will create a new empty array and return it instead.
- Replacing `SecurityIdentifier` `SecurityType` and `GetHashCode`
implementations for `Lazy` versions, that are performed just once, since
these values do not change and are used multiple times.
- For the different `DataDictionary<T>` implementations adding `this[
Symbol] get; set` since existing overload `this [string]` produces an
extra round operations `Symbol->string->Symbol` with a significant
impact.
- Adding `PortfolioTargetCollection.AddRange()` overload using an array
to avoid unnecessary convertions.
2019-04-11 19:01:27 -03:00
Martin Molinero ed5f7ab58c Moving price scale factoring to new enumerator
- Adding new `PriceScaleFactorEnumerator` that will scale raw prices
based on a provided `SubscriptionDataConfig` and update the
`SubscriptionDataConfig.PriceScaleFactor`. Adding unit tests.
- `BaseData` factories (`TradeBar.cs`, `QuoteBar.cs`, `Tick.cs`) will
no longer scale factor prices, they will generate data points in raw
mode.
- `SubscriptionDataReader` will no longer update the
`SuscriptionDataConfig.PriceScaleFactor`
- Fix `BrokerageTransactionHandlerTests` unit test that was having a
race condition.
2019-02-27 15:11:34 -03:00
Stefano Raggi 273d6e90da Fix parsing of crypto tick data with sub-millisecond timestamps
- also updated extension methods ToInt32 and ToInt64 to support input strings with decimals
2019-01-21 16:47:27 +01:00
AlexCatarino 4592f1a7f2 Adds remarks to explain the issue 2019-01-09 20:21:27 +00:00
AlexCatarino d846efdac6 Implements indexer in DataDictionary derived classes
Explictly implements the indexert `this[string]` to all classed that inherit from `DataDictionary` since pythonnet was not able to access the indexer from the parent class.

- Changes DividentAlgorithm.py to test the fix.
2019-01-09 17:48:31 +00:00
Stefano Raggi 57d50c69c4 Fix XML documentation compiler warnings
- Fixed all warnings except for missing XML comments (CS1591)
2018-11-09 11:44:03 +01:00
AlexCatarino 6ee4363942 Implements DataDictionary.GetValue
`DataDictionary.GetValue` is meant to be used as an alternative to `DataDictionary.TryGetValue`. It was created due to limitations to the python implementation.
2018-10-17 22:18:12 +01:00
Juan José D'Ambrosio 3fd2ea7675 Set Tick time stamp using ticks for CFD and Fx
This is required to support Oanda's sub-millisecond resolution data.
2018-10-01 11:32:31 -03:00
Juan José D'Ambrosio 249092d90d Read CFD and Forex tick timestamps as double intead int
This allows resolution higher than millisecond.
2018-10-01 11:32:31 -03:00
Stefano Raggi db942e0d2e Zero volume out in fill forward TradeBars 2018-09-25 22:29:53 +02:00
Michael Handschuh 49117137f0 Improve split/dividend application logging in live mode
It's important we see the before/after and the data used to make the change.
This will aid in debugging potential misapplications of split/dividend events.
2018-09-20 17:20:02 -04:00
Michael Handschuh 76234ee3b4 Make Dividend.ReferencePrice public set
This is to be consistent with the other properties in Dividend
2018-08-27 13:53:04 -04:00
Michael Handschuh 57affafdc2 Implement 'live' factor files
Adds reference price to dividend/split
Adds GetSplitsAndDividends to FactorFile
Adds Apply methods to FactorFile/FactorFileRow
Updates factor files to include reference prices
2018-08-14 17:58:44 -04:00
Michael Handschuh 1a01cfb0ec Add Dividend.ToString, fix Split.ToString doc 2018-07-20 19:32:03 -04:00
Stefano Raggi ba1f5ae673 Add Tiingo daily data 2018-07-19 21:22:18 +02:00
Worthy7 8812147809 Add IBaseDataBar
Allows it to be used with Bar related methods
2018-06-20 09:42:16 +09:00
Michael Handschuh bb21171943 Implements python Consolidate API
Collapse QuoteBar to TradeBar if trade bar handler specified
2018-06-12 13:17:49 -04:00
Stefano Raggi 885b7849b6 Trigger Travis build 2018-06-07 15:56:00 +02:00
Stefano Raggi 0c2276c800 Fix JSON deserialization for Split and Delisting 2018-06-07 12:05:44 +02:00
Michael Handschuh dd3076e992 Move IntraDayDividendSplit to FactorFileGenerator
This is a type used internally by the generator and is not intended to
be used anywhere, so moved it to a private nested class.
2018-04-26 13:13:50 -04:00
AlexCatarino d9a7e1dcd7 Adds Volume property to FutureContract and OptionContract
Minor change in `TimeSlice` to update `FutureContract.Volume` and `OptionContract.Volume` with `TradeBar.Volume` value.
2018-03-23 09:58:22 +00:00
Michael Handschuh 41ce1f1741 Add Delisting.ToString override
These overrides are super useful when debugging and logging various events.
2018-01-22 22:06:30 -05:00
Michael Handschuh ca747b3667 Add split warning events
This follows the pattern used by delisting events, where we send a warning event
before the start of trading on the trading day before the split will happen. This
change also adds a SplitType enum having a Warning and SplitOccurred values.
2018-01-22 22:06:30 -05:00
Andrew 076bb1627c Added KaikoDataConverter Toolbox project
This project can take trade as well as orderbook information from Kaiko
and convert the data into Lean compatible data at all resolutions in
modeled as both trades and quotes.
2017-10-27 15:50:47 -04:00
Andrew 8264cc4a9f LeanDataTests passing for crypto securities
The Crypto security can be modeled with both trades and quotes easily, therefore it is necessary to make sure that the LeanData, LeanDataWriter and associated BaseData.Reader() methods can read/write both Trades and Quotes.
2017-10-25 16:52:31 -04:00
jameschch@outlook.com 578efe843f Further changes for crypto type 2017-08-29 12:19:15 +01:00
12112 921ec8f983 merge from upstream 2017-08-22 16:56:35 +01:00
Stefano Raggi c0b1077dbc Update Tick quantity fields from integer to decimal
Tick.Quantity: int -> decimal
Tick.BidSize: long -> decimal
Tick.AskSize: long -> decimal
2017-08-16 20:30:14 +02:00
Stefano Raggi 8684ff6513 Fix price scaling in Futures data with Tick resolution 2017-08-11 15:15:14 +02:00
12112 83fdf9c386 crypto wip 2017-07-14 16:53:42 +01:00
Stefano Raggi ba1e462fc4 Fix open interest bug setting value to zero for contracts added intraday
When using a filter function creating a dynamic chain universe, contracts added during the day would be set to zero.
2017-06-30 18:31:32 +02:00
Jared 678b05bb4f Merge pull request #980 from smartquant/daily_futures_settings
Fix for using daily or hourly resolution market data for futures.
2017-06-30 11:09:48 -04:00
Stefano Raggi 91178b4ce1 Revert changes to FuturesContract and OptionContract 2017-06-26 22:27:12 +02:00
Stefano Raggi 508e63af4d Update Bar types Volume and Bid/AskSize from long to decimal
This PR is another required step towards crypto-currency support.
2017-06-26 21:14:55 +02:00
smartquant fb2cf7deaf Fix for using daily or hourly resolution market data for futures.
- Open interest should use yyyyMMdd HH:mm in csv file when frequency is daily or hourly
- Filename should be [baseticker]_[quote|openinterest|trade]_yyyyMM.csv
2017-06-19 11:15:12 +02:00
Anshul e7a3a2f3bd Fixes 5 warnings with code CS1574 2017-05-18 17:05:09 +05:30
AnshulYADAV, Computer Artist (CoArsey) 350e58567f Fixes 18 warnings with code CS1572 (#908)
* Fixes 5 warnings with code CD1572

* Adds the Xml Comment parameter and implements the missing selector parameter

* Fixes rest of the 13 warnings with code CS1572
2017-05-12 09:28:46 -04:00
Stefano Raggi 83150dbe62 Update Reader error log messages for QuoteBar and TradeBar 2017-05-07 21:01:45 +02:00
Stefano Raggi 607e392977 Add Resolution to QuoteBar.Reader error logging 2017-05-06 11:01:15 +02:00
Stefano Raggi b2216ba22d Add symbol to QuoteBar.Reader error logging 2017-05-06 10:30:37 +02:00
Andrew 4c82273eb4 Future quotebars do not have scaling applied when read 2017-04-25 16:17:22 -04:00
Andrew 01656c1a42 Future tradebars have no scaling applied when read 2017-04-25 16:17:21 -04:00
Andrew 22175bb7f0 QuoteBar.Reader does not apply scaling factor to forex and cfd 2017-03-27 12:01:24 -04:00
Jared 89254db694 Fixing issues from futures-options merge (#692)
* Swallow bug in QuoteBar.Reader() if an error is thrown during parsing

Following the pattern in TradeBar.Reader(), the QuoteBar.Reader() method will now swallow a parsing error, log the error message and line number and then proced to return a default instance of Quotebar.  The current implementation of QuoteBar.Reader() will stop the executation of the program if a parsing error is found.  Given the amount of data Lean may parse during the execuation of an algorithm, it is better to be resiliant to small errors in the data than to stop the execuation of the program because of a malformated line of csv.

* Added QuoteBar.Reader tests

* Minor changes to config.json; back to proper defaults

Remove QLLib.NET reference

Added date to QuoteBar.Reader() error message

* Incorrect timezone specified on consolidator filter
2017-01-15 16:41:09 -05:00
quant1729 65eef05d25 Fixed bug: missing bid or ask made quotebar produce corrupted OLHC prices (halved). Test. 2016-12-22 16:27:10 +01:00
quant1729 8dddb413e3 Fixed crash due to not initialized greeks in options backtests 2016-12-13 12:26:54 +01:00
quant1729 2d83515d99 Fixed trade bar to parse futures bars 2016-12-05 16:06:03 +01:00