Commit Graph

48 Commits

Author SHA1 Message Date
Martin Molinero 1c92986994 Calculate insight score 1 per time step 2019-06-11 19:06:39 -03:00
Jared 47a038c5e8 Merge pull request #3242 from QuantConnect/feature-3240-fitness-function
Add Fitness Score
2019-06-04 15:10:11 -07:00
Martin Molinero 61db544817 Address reviews 2019-06-03 15:10:19 -03:00
Martin Molinero 61bab7439b Address reviews 2019-06-03 15:10:18 -03:00
Martin Molinero c015d4ca1f Add Fitness Score
- Adding new `FitnessScore`, score of the strategys performance, and
suitability for the Alpha Stream Market
2019-06-03 15:10:18 -03:00
Martin Molinero ee4f8fee82 Overall performance improvements
- `TimeSliceFactory` will avoid creating empty collections
- `ExecutionModels` will check target collection count before trying to
enumerate
- Reduce calls to .`TotalPortfolioValue`
- `SecurityValues` will only be created when required
- `TimeKeeper` will use TimeZone unique Id as dictionary key. The
TimeZone hash is expensive.
- `AlgorithmManager` will avoid calling `DateTime.UtcNow`,
`ConvertFromUtc()` and `RoundDownInTimeZone()`
2019-06-03 15:09:02 -03:00
Martin Molinero 40267da95b Adding try catch for disposed Timer 2019-04-29 19:45:26 -03:00
Martin Molinero c39638668c Overall performance improvements
- `FactorFile` will keep an ordered reversed list with the dates.
Calling `Reverse()` on the `SortedList` is expensive.
- `MapFiles` will keep first and last date, so we don't need to call
`First()` and `Last()` multiple times.
- `Liquidate` will go through all the algorithms securities only if
necessary
- `TradeBar` parsing will not call `new T` for pure `TradeBar` which is
expensive
- Removing `Lazy` hash code and security type for the
`SecurityIdentifier`, replacing for direct initialization. Accessing the
`Lazy` value adds an overhead.
- Replacing `Enum` to string for hardcoded switch statement. `Enum.ToString` is expensive.
- `DataManager` will be lazy for counting the subscriptions for
determining if its above the limit
- Adding `AlgorithmSecurityValuesProvider.GetAllValues()`, removes the
need to fetch all the security keys twice.
- During universe selection, will not try to re add already added symbol
2019-04-22 10:47:27 -03:00
Martin Molinero d4d0ffa260 Address reviews
- Will use timers for live insight storing, with a 10 minute period
2019-03-27 23:42:23 -03:00
Martin Molinero 5b04cfe202 Emiting Insights based on Fills
- Classic Algorithms will emight insights based on order fills.
   - To be able to update generated insights closed time, we will not
   clone emitted insights.
   - `InsightAnalysisContext` will update `AnalysisEndTimeUtc` when the
   Insight period is closed and the period is `EndOfTimeTimeSpan`
- Adding new regression algorithm asserting on the new emitted insights
- Adding unit tests
- `LiveTradingResultHandler` will store `AlphaRunTimeStatistics`
- Making `DefaultAlphaHandler.ProcessAsynchronousEvents` virtual to
facilitate cloud changes
2019-03-26 16:09:22 -03:00
Martin Molinero 75ffa300f1 Adding new IAlgorithm.AccountCurrency
- Adding new `IAlgorithm.AccountCurrency { get; }` that will point to the
`Portfolio.CashBook.AccountCurrency`. Setter will be added in a
following PR.
- Base `Brokerage` class will now have a `AccountCurrency { get }`
pointing to the `IAccountCurrencyProvider`. Will be used by the different
brokerages implementations.
2018-12-10 11:59:02 -03:00
Michael Handschuh dd764ad8e9 Rename InsightCollection->GeneratedInsightsCollection
This type is just used as a container for generatd insights. Renaming in
preparation for a new InsightCollection to mirror the PortfolioTargetCollection
2018-04-10 19:24:36 -04:00
Michael Handschuh 11df0388ba Remove limit on scoring insights
This check was moved to directly prior to persistence.
NOTE: The default persistence implementation was not touched since
it writes to the local file system.
2018-03-23 12:30:15 -04:00
Michael Handschuh 708b0e91b8 Dispose of InsightManager extensions on exit
This isn't technically required, but it's best practice to explicitly
dispose of resources that require it.
2018-03-22 14:51:42 -04:00
Michael Handschuh fc269b502b Send finalized insights to messaging handler
The AlphaResultPacketSender is implemented as an IInsightManagerExtension.
It binds to the event that fires when an insight is done being scored.
These are enqueued for later pushing via the messaging handler. Nominally
the limits are set to emit a maximum of 50 finalized insights, with a
second between the completion of a send to the start of the next send
(timer disabled while sending).
2018-03-22 14:51:42 -04:00
Michael Handschuh 662ae5a65b Limit total insights processed
Nominally set to 10k, similar to order limits
2018-03-22 11:37:34 -04:00
Michael Handschuh bf4f57f8bd Completely remove messaging/persistence updates
Persistence is performed at the end of the backtest.
We'll address live mode separately
2018-03-21 08:26:51 -04:00
Michael Handschuh f592c25e57 Add try/catch around storing insights and pushing messaging updates 2018-03-21 08:26:51 -04:00
Michael Handschuh 4eabd230a0 Fix R# warning on potential null assignment
This warning could be ignored but was simply remedied throw a simple refactor
2018-03-21 08:26:51 -04:00
Michael Handschuh 668038b269 Make insight scoring synchronous 2018-03-21 08:26:51 -04:00
Michael Handschuh 6b239674e2 Renames Alpha -> Insight
The term 'alpha' is used to describe the entire algorithm. Therefore, 'alpha'
produces insights. From this we have things like IAlphaModel, which is the model
defining how insights are produced. We have IAlphaHandler, which defines how the
insights from a single 'alpha' (the algorithm) are managed, analyzed, and stored.
Types closer to the individual prediction level, such as InsightDirection, or
InsightScore relate directly to exactly 1 insight. The distinction between the
two became more clear as we developed the insights API, and from that effort it
was decided to harmonize alpha/insight terminology across the various QC systems.
2018-03-09 16:12:56 -05:00
Michael Handschuh 622446ffb0 Fix DefaultAlphaHandler.IsActive flag
Fixes the IsActive flag to be explicitly set and unset at the start
and stop of the Run method respectively. Previously if cancellation
was requested, then IsActive would return false, even if it was still
processing messages.
2018-02-21 10:59:40 -05:00
Michael Handschuh 69d3a10a93 Decompose DefaultAlphaHandler into IAlphaManagerExtension
The IAlphaManagerExtension defines a type that needs to react to events produced
by the AlphaManager. The actual events were removed in favor of a interface to
handle the events. This removes the need to wire events and instead just pass the
extensions to the alph manager and it will handle invoking the extensions at the
appropriate time.

This change removes all charting and statistics aggregation logic from the alpha
handler and moves it into dedicated types, AlphaChartingManagerExtension and
AlphaStatisticsManagerExtension. The resulting types are highly decoupled from the
LEAN ecosystem allowing them to be easily unit tested, whereas before the logic
was embedded in a handler with many many dependencies which would be very hard to
properly unit test.

As part of this change (and in preparation for moving scoring to the alpha thread)
the resolution of SecurityValues was removed from the alpha manager. In this new
pattern, the alpha manager is pushed generated alphas and security values at each
time step.
2018-01-12 12:08:40 -05:00
Michael Handschuh efef0fe2ce Default alpha runtime statistics to null
These statistics will be null unless it is a framework algorithm
2018-01-05 11:45:05 -05:00
Michael Handschuh d634cf20aa Replace inheritdoc tags with documentation from interface 2018-01-04 19:21:38 -05:00
Michael Handschuh 9174fd250d Add AlphaResultPacket.UserId
Very useful to have user information with the generated alphas
2018-01-04 19:21:38 -05:00
Michael Handschuh 363003815b Rename AlphaPacket -> AlphaResultPacket
Making room for the AlphaWorkPacket
2018-01-04 12:43:41 -05:00
Michael Handschuh d82b7b19aa Add logging for alpha handler thread exit signals 2017-12-28 14:47:25 -05:00
Michael Handschuh cfea54aecd Fixes average estimated value and delays avg score sampling 2017-12-28 12:24:59 -05:00
Michael Handschuh b07928bd58 Properly account for market hours in alpha prediction periods
If we make a prediction for 1 day in the future, we actually mean 1 trading day.
This change updates the alpha analysis logic to take into account the security's
market hours.
2017-12-28 11:27:41 -05:00
Michael Handschuh 9a4d1b2e83 Don't sample average scores before they've been assigned a value. 2017-12-28 11:27:40 -05:00
Michael Handschuh 0a50fea37c Fix bar chart daily sampling... again. Use date change as signal 2017-12-20 16:10:03 -05:00
Michael Handschuh 46fe7adf62 Remove asset breakdown chart
This chart is the cause of much consternation regarding the best way to display this data.
For now, we're removing it from the charts collection but keeping the computation logic in
place. We'll circle back after alphas launch when we figure out the best way to display this
data.
2017-12-20 09:35:11 -05:00
Michael Handschuh 2295e524f4 Add IAlphaRuntimeStatisticsGenerator and send to result handler
Provides estimates of alpha value as well as performs online computations of
alpha scores and other KPIs.

Sends alpha stats to result handler
Update live result with framework flag
2017-12-20 08:33:30 -05:00
Michael Handschuh 5337cb0b5e Fixes reporting of finalized alpha scores 2017-12-19 18:18:33 -05:00
Michael Handschuh d4c266ce45 Add 'Alpha Assets' pie chart containing total alphas/symbol
This change also includes various cleanup of the default alpha handler to make some things
easier to follow. A known issue is that by the time this daily chart data makes its way to
persistent storage and/or the browser, we seem to get a sample every hour instead of every
day. While debugging, however, placing a breakpoint at IResultHandler.SendFinalResult shows
the expected samples of 1 per day. Further investigation required here.
2017-12-19 14:46:23 -05:00
Michael Handschuh 8babf9e11a Fixes bug in first alpha count sampling
Also removes a full iteration of the assetbreakdowns just to get total predictions
2017-12-18 16:24:28 -05:00
Michael Handschuh bf36f44a1f Move asset breakdown to daily sampling 2017-12-18 16:21:38 -05:00
Michael Handschuh e546790104 Make alpha count a bar chart 2017-12-18 16:18:52 -05:00
Michael Handschuh 8b7051dd7e Sample prediction counts once daily at EOD 2017-12-18 16:18:52 -05:00
Michael Handschuh a0dfd0b857 Disable alpha handler for no-framwork algos 2017-12-18 14:51:26 -05:00
Michael Handschuh df9776127c Only chart average scores of finalized alphas 2017-12-18 14:24:11 -05:00
Michael fd59896edb Merge pull request #1407 from QuantConnect/bugfix-dont-throw-alpha-error
Remove throw statements from alpha handler try/catch
2017-12-17 20:14:20 -05:00
Michael 4ff4c9044c Merge pull request #1405 from QuantConnect/alpha-score-range-bounding
Alpha score range bounding
2017-12-17 17:05:33 -05:00
Michael Handschuh 2015a0a5c9 Remove throw statements from alpha handler try/catch
These were put in to prevent bubbling exceptions up. So far the experience
has been exceptions caused here are due to double->decimal out of bounds.
2017-12-17 17:03:22 -05:00
Michael Handschuh cbbdb481d1 Add try/catch blocks around alpha processing 2017-12-17 05:20:32 -05:00
Michael Handschuh f9192bd123 Verify score range is valid decimal before plotting
This is to avoid cast exceptions from really small scores. The checks on the upper
end are also added for completeness.
2017-12-17 05:19:38 -05:00
Michael Handschuh b4b3aae112 Rename Signal -> Alpha
Whoa... I think I got everything :)
2017-12-15 18:24:58 -05:00