Commit Graph

1922 Commits

Author SHA1 Message Date
Jared Broad 0f942149a0 Testing shuffling worker thread to algorithmSystemHandlers 2019-07-28 15:54:17 -07:00
Jared Broad 1e8fdba3d9 Revert to master 2019-07-28 14:32:04 -07:00
Jared Broad 66a3f2e1d2 Move worker thread creation to launcher before job creation 2019-07-28 12:54:17 -07:00
Martin Molinero 1d3d4b9bc8 Improvements - self review 2019-07-26 11:43:45 -03:00
Martin Molinero b6e9498b9e Improve FineFundamental backtesting performance
- Reduce the amount of `Path.Combine()` usages -> it has a peformance
overhead
- Improving `FineFundamentalSubscriptionFactory` GetSource algorithm,
now it will not check if each file exists while finding the appropriate,
since we already iterated the directory before
- `DefaultDataProvider` will not check if file exists since `new
FileStream` performance the same operation internally
2019-07-25 22:35:28 -03:00
Stefano Raggi e39791ac6c Upgrade RestSharp library from v105.2.3 to v106.6.10 2019-07-17 10:28:35 +02:00
Stefano Raggi e9739064e6 Fix LiveEquityDataSynchronizingEnumerator for input data after the current time
Previously the enumerator would get stuck and stop emitting data if one of the underlying enumerators returned a data point with the time greater than the current time. The existing unit test would only emit the first data point for the two underlying streams.

The enumerator has been updated to support data points in the future and the unit test has been extended to assert both data point counts and values.
2019-07-09 16:49:50 +02:00
Jared f89031d8ed Merge pull request #3366 from Martin-Molinero/bug-3363-zipdatacacheprovider-dispose-race-condition
Fix dispose race condition at ZipDataCacheProvider
2019-07-07 13:13:00 -07:00
Jared d45b6970d8 Merge pull request #3365 from QuantConnect/bug-3364-live-option-chain-provider-retry-api-failure
Add retry logic to LiveOptionChainProvider.GetOptionContractList
2019-07-05 11:45:54 -07:00
Martin Molinero 82da3b5521 Fix dispose race condition at ZipDataCacheProvider 2019-07-04 13:30:42 -03:00
Stefano Raggi a0c433ff02 Address review
- retry only on WebException
- throw after last failed attempt
2019-07-04 01:09:58 +02:00
Stefano Raggi 43a16c3580 Add retry logic to LiveOptionChainProvider.GetOptionContractList 2019-07-03 18:31:42 +02:00
Gerardo Salazar 2033c7873b Update outdated DotNetZip dependency in nuspec files 2019-07-02 18:02:23 -07:00
Martin Molinero a92bee5bf1 Fix OnEndOfDay not fired
- Adding `IRealTimeHandler.OnSecurityChanged()` will be used to update
the `OnEndOfDay` security related scheduled events
- Adding `BaseRealTimeHandler.cs` to reduce code duplication in the
`Backtesting` and `LiveTrading` `RealTimeHandlers`
- Adding CSharp and Python regression tests
- Deprecating `OnEndOfDay()` callback because of two reasons, mainly
because Python does not support two methods with the same name, but also
because different assets have different market close times.
- `ScheduledEvents` set at the same time will now be deterministic
2019-06-20 17:35:28 -03:00
Martin Molinero f6b5e92a96 Add conditional optimization flag 2019-06-19 16:11:15 -03:00
Martin Molinero e8660d9f52 Address reviews 2019-06-17 23:40:56 -03:00
Martin Molinero 760e32cf65 Add unit test for Coarse Scheduled Selection
- New unit test fails in master
- Adjust lower invalid time limit to 5 AM
2019-06-17 23:40:56 -03:00
Martin Molinero 65201cb808 Adding StepTimeProvider for live
- Adding new `StepTimeProvider` that will advance time based on a
desired custom evaluator.
- Live trading `Options`, `Futures` and `Coarse` data will use a
`FrontierAwareEnumerator` + `StepTimeProvider`. This will allow to hold
the selection data until its the desired time
- `Universe.CanRemoveMember` will round the members time in the universe
based on the `UniverseSettings.MinimumTimeInUniverse`
2019-06-17 23:40:55 -03:00
Jared 51008b4d3c Merge pull request #3311 from Martin-Molinero/performance-reduce-shutdown-time
Reduce engine shutdown time
2019-06-17 09:22:29 -07:00
Jared ff19bd266c Merge pull request #3312 from QuantConnect/bug-datamanager-remove-utcnow-usage
Remove usage of DateTime.UtcNow in DataManager
2019-06-17 09:15:23 -07:00
Jared 5b03d3a917 Merge pull request #3309 from QuantConnect/testing-live-synchronizer-add-timeout-property
Add a virtual property get in LiveSynchronizer
2019-06-17 09:13:42 -07:00
Stefano Raggi 5143645fe6 Fix LiveTradingDataFeed handling of Auxiliary data at Tick resolution 2019-06-14 23:18:17 +02:00
Martin Molinero c58b479578 Reduce shutdown time 2019-06-14 16:05:45 -03:00
Stefano Raggi e87eda1d78 Remove usage of DateTime.UtcNow in DataManager
This change is needed for live trading unit tests which use date ranges in the past.
2019-06-14 18:54:52 +02:00
Stefano Raggi 6e387d451c Add a virtual property get in LiveSynchronizer
This enables overriding the 500ms value in unit tests for faster run times.
2019-06-14 18:35:45 +02:00
Jared ac0de6b466 Merge pull request #3303 from Martin-Molinero/performance-3302-mapfile-factorfile-data-feed-stack-resolution
MapFile and FactorFile data feed stack resolution
2019-06-13 16:37:36 -07:00
Martin Molinero 1370ae666a Address reviews 2019-06-13 20:33:22 -03:00
Jared a515693835 Merge pull request #3299 from Martin-Molinero/performance-3298-insight-score-calculation
Calculate insight score once per time step
2019-06-13 16:16:25 -07:00
Jared 10de77e600 Merge pull request #3307 from QuantConnect/bug-3286-live-data-delay-v2
Fix live data delay
2019-06-13 16:13:08 -07:00
Martin Molinero 19701461cc Fix live data delay
- C# `Timer` can sometimes be triggered before the expected due time.
Implementing `RealTimeScheduleEventService` that will guarantee a
scheduled event due time is respected and the event never happens before
time.
2019-06-13 17:52:29 -03:00
Martin Molinero 7aafdef827 MapFile and FactorFile data feed resolution
- Moving `MapFile` and `FactorFile` resolution to the data feed stack so
that they do not add a performance overhead to the algorithm thread.
- Create logging string messaged only if required.
- Calculate `FactorFileRow.PriceScaleFactor` the least amount of times
2019-06-12 17:48:47 -03:00
Jared 641e8d8ec1 Merge pull request #3295 from gsalaz98/bug-3293-security-upgrade-dotnetzip-to-v1.13.3
Upgrade Dependency DotNetZip To v1.13.3 (Fixes CVE-2018-1002205)
2019-06-12 07:49:35 -07:00
Martin Molinero 1c92986994 Calculate insight score 1 per time step 2019-06-11 19:06:39 -03:00
Gerardo Salazar 340cc3a330 Upgrade dependency DotNetZip to version 1.13.3 to fix security issue 2019-06-11 11:03:37 -07:00
Martin Molinero 1581801f9b Address reviews 2019-06-11 12:01:57 -03:00
Martin Molinero ec1d410907 Performance improvements
- `SecurityIdentifier` will calculate `StrikePrice` just once. Replacing
`!=` for a direct call to `Equals()` preventing unnecessary checks.
- Slightly improving Linq queries at `OptionFilterUniverse`
- Replace `OpenReader().CopyTo` for `Extract(stream)` avoids copying the
data twice.
- Replace `DateTime.Now` for `DateTime.UtcNow`
2019-06-10 22:40:19 -03:00
Martin Molinero 2c1cf0fa93 Address reviews
- Adding new regression test algorithm
- `SecurityPortfolioManager` will subscribe to the `CashBook.Updated`
event to invalidate the `TotalPortfolioValue
2019-06-07 17:51:30 -03:00
Martin Molinero 084353f182 Smart TotalPortfolioValue Calculation
- `SecurityPortfolioManager` will keep track of the current
`TotalPortfolioValue` and only re calculated when flagged as invalid.
2019-06-07 16:13:57 -03:00
Martin Molinero 90a8f23d98 Address reviews 2019-06-07 16:02:53 -03:00
Martin Molinero 08b72c5907 Overall performance improvements v2
- Adding `LazyToUpper()` implementation, that will avoid the call to
`ToUpper` if the string is already upper.
- Reduce the timezone conversions at `Time.EachTradeableDayInTimeZone`
- `TotalPortfolioValue` will iterate over all securities once
- Adding new `SecurityIdentifier` cache, significant impact for
algorithms using coarse/fine data
2019-06-05 17:06:59 -03:00
Stefano Raggi 9dc4ad17fc Add missing null check in LiveTradingResultHandler.Exit 2019-06-05 12:19:28 +02:00
Jared 47a038c5e8 Merge pull request #3242 from QuantConnect/feature-3240-fitness-function
Add Fitness Score
2019-06-04 15:10:11 -07:00
Martin Molinero 14c779733b Address reviews 2019-06-04 17:16:52 -03:00
Martin Molinero 61db544817 Address reviews 2019-06-03 15:10:19 -03:00
Martin Molinero 61bab7439b Address reviews 2019-06-03 15:10:18 -03:00
Martin Molinero c015d4ca1f Add Fitness Score
- Adding new `FitnessScore`, score of the strategys performance, and
suitability for the Alpha Stream Market
2019-06-03 15:10:18 -03:00
Martin Molinero ee4f8fee82 Overall performance improvements
- `TimeSliceFactory` will avoid creating empty collections
- `ExecutionModels` will check target collection count before trying to
enumerate
- Reduce calls to .`TotalPortfolioValue`
- `SecurityValues` will only be created when required
- `TimeKeeper` will use TimeZone unique Id as dictionary key. The
TimeZone hash is expensive.
- `AlgorithmManager` will avoid calling `DateTime.UtcNow`,
`ConvertFromUtc()` and `RoundDownInTimeZone()`
2019-06-03 15:09:02 -03:00
Martin Molinero d360ec36ac Address reviews 2019-05-31 18:26:18 -03:00
Martin Molinero 57fbf588fe Download RemoteFile once for backtests
- For backtests download RemoteFile just once
- Live subscriptions and history requests will continue to download the
`RemoteFile` on each request.
- Adding unit tests
2019-05-29 18:38:07 -03:00
Jared 8a95a35d95 Merge pull request #3191 from StefanoRaggi/bug-3190-live-suspicious-tick-filtering
Add Suspicious Tick Filtering in LiveTradingDataFeed
2019-05-29 07:10:07 -07:00