Commit Graph

90 Commits

Author SHA1 Message Date
Martin Molinero 38e8621bb9 Serialized Insight has default values 2019-07-10 11:33:42 -03:00
Martin Molinero 64d43c1487 Add Insight.ReferenceValueFinal 2019-07-09 17:53:58 -03:00
Jared a2a63ae058 Merge pull request #3016 from QuantConnect/feature-3015-add-weight-to-insights
Add Insight optional Weight
2019-06-04 16:52:48 -07:00
Martin Molinero ee4f8fee82 Overall performance improvements
- `TimeSliceFactory` will avoid creating empty collections
- `ExecutionModels` will check target collection count before trying to
enumerate
- Reduce calls to .`TotalPortfolioValue`
- `SecurityValues` will only be created when required
- `TimeKeeper` will use TimeZone unique Id as dictionary key. The
TimeZone hash is expensive.
- `AlgorithmManager` will avoid calling `DateTime.UtcNow`,
`ConvertFromUtc()` and `RoundDownInTimeZone()`
2019-06-03 15:09:02 -03:00
Jared ffcee5bd73 Merge pull request #3211 from Martin-Molinero/bug-3155-insightmanager-thread-safety
Make InsightManager thread safe
2019-05-29 07:09:47 -07:00
Jared 167d5c348c Merge pull request #3230 from Martin-Molinero/bug-3154-overflow-exception-blacklitterman-meanvariance
Fix overflow exception
2019-05-28 08:17:38 -07:00
Stefano Raggi 9d3c1df5f3 Fix execution models to handle partially filled orders 2019-05-24 11:50:31 +02:00
Martin Molinero e294b3c3e2 Fix overflow exception
- Adding new `AlgorithmSettings` Min and Max absolute portfolio target
percentage
- Adding new `PortfolioConstructionModel.FilterInvalidInsightMagnitude()`
helper method that will be used by the `BlackLitterman` and
`MeanVariance` optiomization portfolio construction models to skip
insights with extreme magnitudes that will cause exceptions
- `PortfolioTarget.Percentage()` will now verify requested percent is
withing the settings values
2019-05-23 20:30:44 -03:00
Martin Molinero 1fc2ef11a3 Make InsightManager thread safe
- Adding new `lock` so that `InsightManager` `Step()`,
`AllInsights`, `ContextsOpenAt()`, `GetUpdatedContexts()`
and `OpenInsights` are thread safe.
2019-05-20 15:35:59 -03:00
Gerardo Salazar 4af1967087 Reduce CopyTo in InsightCollection to a one-liner
Add unit tests for CopyTo
2019-05-07 18:47:19 -05:00
Gerardo Salazar 14f95b832c Fix infinite loop in InsightsCollection.CopyTo 2019-05-05 17:16:26 -05:00
Martin Molinero 3d52343799 Optimize stored result json
- `Alpha Assets` chart will only store last data point
- Adding new `JsonRoundingConverter` that will round to 4 (number of
digits currently used for comparing alpha statistics) fractional
digits.
   - Will be used for `Insights` and `ChartPoint`
2019-04-29 19:17:00 -03:00
Martin Molinero 7af29e3697 Add Insight Portfolio Weight
- Adding optional portfolio Weight property for `Insights`
2019-04-24 10:58:45 -03:00
Martin Molinero c39638668c Overall performance improvements
- `FactorFile` will keep an ordered reversed list with the dates.
Calling `Reverse()` on the `SortedList` is expensive.
- `MapFiles` will keep first and last date, so we don't need to call
`First()` and `Last()` multiple times.
- `Liquidate` will go through all the algorithms securities only if
necessary
- `TradeBar` parsing will not call `new T` for pure `TradeBar` which is
expensive
- Removing `Lazy` hash code and security type for the
`SecurityIdentifier`, replacing for direct initialization. Accessing the
`Lazy` value adds an overhead.
- Replacing `Enum` to string for hardcoded switch statement. `Enum.ToString` is expensive.
- `DataManager` will be lazy for counting the subscriptions for
determining if its above the limit
- Adding `AlgorithmSecurityValuesProvider.GetAllValues()`, removes the
need to fetch all the security keys twice.
- During universe selection, will not try to re add already added symbol
2019-04-22 10:47:27 -03:00
Martin Molinero 938047de47 Will ignore flat insights
- `Flat` insights will be ignored both for scoring (`InsightManager`)
and for statistics (`StatisticsInsightManagerExtension`). Adding unit
tests
2019-04-16 20:39:55 -03:00
Martin Molinero b7930aff67 Performance improvements
- Using `Aggregate(lambda)` vs `Sum(lambda)` since the later is slower
due to performing an extra `Select`
- For `QCAlgorithm.Framework.OnFrameworkData()` will avoid calling
`ToArray()` on empty `Enumerables` due to its cost * the number of
calls. If the `Enumerable` is the empty instance, which is static,
will create a new empty array and return it instead.
- Replacing `SecurityIdentifier` `SecurityType` and `GetHashCode`
implementations for `Lazy` versions, that are performed just once, since
these values do not change and are used multiple times.
- For the different `DataDictionary<T>` implementations adding `this[
Symbol] get; set` since existing overload `this [string]` produces an
extra round operations `Symbol->string->Symbol` with a significant
impact.
- Adding `PortfolioTargetCollection.AddRange()` overload using an array
to avoid unnecessary convertions.
2019-04-11 19:01:27 -03:00
Martin Molinero 32ac3146b4 Merge Framework and Classic Algorithms
- Merging Framework and Bridge algorithms into classic QCAlgorithm
class.
- Removing Framework project, VS17 and VS15
2019-04-03 21:54:32 -03:00
AlexCatarino 5931a42d48 Implements Insight Expiry Helper
Implement a new overload to `Insight` constructor that accepts a  `Func<DateTime, DateTime>` that is used to compute the `CloseTimeUtc` and `Period` after the `Insight` object is emitted (`SetPeriodAndCloseTime` method).

Adds the static Expiry class with functions that can be used to compute a future date/time (expiry) given a date/time.

Closes #3038
2019-04-03 21:46:14 +01:00
Martin Molinero 9d5b1f5be6 Fix auto generated insight period
- Updating regression test to assert of period value
2019-03-28 18:43:09 -03:00
Jared 9bf537a9b3 Merge pull request #3030 from QuantConnect/feature-3029-order-based-insight-autogeneration
Emiting Insights based on Order Fills
2019-03-28 10:04:54 -07:00
Martin Molinero 5b04cfe202 Emiting Insights based on Fills
- Classic Algorithms will emight insights based on order fills.
   - To be able to update generated insights closed time, we will not
   clone emitted insights.
   - `InsightAnalysisContext` will update `AnalysisEndTimeUtc` when the
   Insight period is closed and the period is `EndOfTimeTimeSpan`
- Adding new regression algorithm asserting on the new emitted insights
- Adding unit tests
- `LiveTradingResultHandler` will store `AlphaRunTimeStatistics`
- Making `DefaultAlphaHandler.ProcessAsynchronousEvents` virtual to
facilitate cloud changes
2019-03-26 16:09:22 -03:00
Martin Molinero 7d1777d364 Ignore Flat Insights direction scoring
- We will ignore flat insights direction scoring
2019-03-21 14:43:48 -03:00
Martin Molinero 1b0bdd9b0b Adding SetAccountCurrency for backtesting
- Adding new `SetAccountCurrency()` for backtesting. Has to be called
before adding any `Security` or calling `SetCash()`, else will throw.
- Adding new Non account currency unit tests for `CashBuyingPower`,
`SecurityPortfolioModel`, `SecurityMarginModel`,
`SecurityPortfolioManager`, `Future/OptionMarginBuyingPowerModels`
- Adding new C# regression test using `SetAccountCurrency()`, one for
`CashBuyingPowerModel` and one for `SecurityMarginModel`
- Adding new Py and C# basic regression algorithms using
`SetAccountCurrency()`
- `Options` and `Futures` will use not use `AccountCurrency` as quote
Cash.
- `SecurityBenchmark` value will be in account currency
2019-01-25 14:54:43 -03:00
Stefano Raggi bd4c799d2b Rename parameter classes from Context to Parameters 2018-11-29 22:05:49 +01:00
Stefano Raggi 78f3b4b099 Refactor additional IBuyingPowerModel methods to use context objects 2018-11-27 22:50:15 +01:00
Martin Molinero 83897a7b88 Portfolio target percent respects FreePortfolioValuePercentage 2018-08-28 12:03:56 -03:00
Stefano Raggi f0e81155e8 Fix error message in PortfolioTarget.Percent 2018-08-21 18:10:51 +02:00
Stefano Raggi 4abc525a38 PortfolioTarget.Percent returns null if error 2018-08-21 18:10:51 +02:00
AlexCatarino 239a5a909e Fixes EqualWeightingPortfolioConstructionModel not flattening expired insights
- When there are or aren'tt new insights, the EqualWeightingPortfolioConstructionModel will creates a target to flatten delisted securities from the universe of expired insights.
- Helper methods were added to deal with removing expired insights and getting active ones and used in `EqualWeightingPortfolioConstructionModel`
- Adds unit test
- Updates framework algorithms
2018-08-13 19:45:25 +01:00
Michael Handschuh d5f88c6bb7 Add Insight.IsActive and Insight.IsExpired methods
Easily check if an insight is active or expired
2018-08-10 13:52:17 -04:00
Michael Handschuh 4fd16f6daf Fix resolution of insight close times, allow user defined close times
Fixes a bug where we were using the security's data resolution to compute
the insight's close time. This led a case such as insight.Period == 20days
to step 20days worth of tradable minutes (assuming minute data resolution),
yielding a close time that was very far in the future.

We also add different means of specifying an insight's period/close time:
1. Specify insight period as a TimeSpan and we compute close time
2. Specify insight period and a resolution and bar count and we compute close time
3. Specify insight close time local directly and we compute the insight period

The key here is maintaining consistency between the three different approaches
which is heavily validated with the corresponding unit tests.

Edits also made to trust the insight's close time as the analysis end time in
the case where the analysis period == insight period (extra analysis period = 0).
Given the current setup (extra analysis period == 0), this guarantees that close
and analysis end times are equivalent.

Regression statistics were updated and expectedly we get many more insights that
have completed analysis, and as such, average scores have also changed.
2018-08-07 11:21:11 -04:00
Martin Molinero 6da57a1e12 Improving GetMaximumOrderQuantityForTargetValue 2018-08-03 16:26:14 -03:00
Martin Molinero c73b7bb62f Adding smarter methods for PortfolioTargetCollection 2018-07-27 10:48:55 -03:00
Michael Handschuh d2c546e78e Make Insight.FromSerializedInsight public
This is needed elsewhere and seems less worse than exposing it
via InternalsVisibleTo
2018-07-19 13:46:11 -04:00
Stefano Raggi 7bc991a621 Update execution models to execute trades ordered by margin impact 2018-06-22 18:07:13 +02:00
AlexCatarino b15cfacff2 Insight.Group retuns the original Insight array with modified Insights
Use the new return type in PairsTradingAlphaModel.
2018-05-10 16:18:07 +01:00
AlexCatarino 659c64d5ce Adds single Insight overload to Insight.Group
pythonnet does not find the method overload when we only provide one `Insight` instance, that leads to a discrepancy from C# API.
2018-05-10 11:50:12 +01:00
AlexCatarino 62adc3a24b Changes Insight.Group return type to IEnumerable<Insight> from Guid 2018-05-09 14:49:00 +01:00
Michael Handschuh 2e0111b858 Handle null group-id in FromSerializedInsight
Also updates the insight serialization tests to exercse the null
group-id case to prevent future regressions.
2018-04-23 13:26:41 -04:00
Michael Handschuh b4902691ed Ignore null values of group-id and source-model
There's no need to transport these null values as it simply adds to
overhead and delays transmission of the insight.
2018-04-23 13:12:49 -04:00
Michael Handschuh 728868d46f Remove incorrect insight closing time approximation
The insight itself defines when it closes via the algorithm framework
and properly takes into account weekends and out of market hours. The
insight analysis was not respecting the insight's stated close time,
but instead was simply doing generated time + period, which doesn't
properly take into account market hours. This causes the number of
closed insights to decrease and due to the extra time for each insight,
the values of the insights have also increased.
2018-04-23 13:11:06 -04:00
Michael Handschuh 5f33d61220 Add Insight.GroupId and Insight.Group( insights )
Provides a means of grouping insights together.
This new value is serialized as 'group-id'.
2018-04-19 12:47:08 -04:00
Michael Handschuh c3c6a9aff8 Add Insight.SourceModel
This identifier is used to determine the alpha model that generated.
This is NOT ideal, since it requires users to specify the value, more
thought will be givent to how we can resolve this value automatically

Adds test for surviving roundtrip copy operation.
2018-04-17 16:36:47 -04:00
Michael Handschuh ef67abea7e Add InsightCollection
Provides a collection type for managing insights. Internally it uses
a dictionary Symbol->List<Insight> but does NOT implement the dictionary
interface due to potentially unexpected behavior when enumerating, i.e,
different behavior when enumerating if statically known as list vs statically
known as dictionary -- not sure how python would resposne to the ambiguity,
so best to leave well enough alone :)
2018-04-10 19:24:37 -04:00
Michael Handschuh dd764ad8e9 Rename InsightCollection->GeneratedInsightsCollection
This type is just used as a container for generatd insights. Renaming in
preparation for a new InsightCollection to mirror the PortfolioTargetCollection
2018-04-10 19:24:36 -04:00
AlexCatarino bf1aa7fa4c Implements Insight.Price helper method.
Implements `Insight.Price` method to make it easier to create new instances of `Insight` of `InsightType.Price`.
Standardize the parameter order to `Symbol`, `TimeSpan`, `InsightType`, `InsightDirection`, `Double`, `Double`.
2018-04-10 19:49:29 +01:00
Michael Handschuh af9202952e PortfolioTarget.ToString - <symbol>: <quantity>
IMO, this was backwards from the beginning. This ordering makes for
better, more intuitive logs.
2018-04-06 16:48:49 -04:00
Michael Handschuh 92c5e64024 Adds VWAP and STD execution models
VWAP will submit market orders while the current price is more favorable than VWAP.
STD will submit market orders while the current price is a configured number of
standard deviations away from the mean in the favorable direction.
2018-04-06 16:30:26 -04:00
Michael Handschuh f11acf1246 Remove Insight.Equals
This method is misleading at best and incorrect at worst.
Insight objects should use reference equality or compare ids to
perform equality checking. The only usage, in MacdAlphaModel,
was easily converted to not relying on this method.
2018-04-03 23:32:34 -04:00
Michael Handschuh b1e624dfdc Fixes bug in PortfolioTarget.Percent
Buying power model's GetMaximumOrderQuantityForTargetValue returns the
delta quantity needed to reach a particular position, so we need add
back in the existing quantity to get the total quantity required.
2018-04-02 11:30:13 -04:00