* Improves MinimumVariancePortfolioOptimizer Values Handling
The C# version of the `MinimumVariancePortfolioOptimizer` generated NaN resulting in unit tests failing.
If the solver returns NaN for an item, we set it to zero. if all items are NaN or Zero, we return the initial guess, since the sum cannot be zero.
* Fixes `RiskParityPortfolioConstructionModelTests`
We need to add insights to insight manager before we call `CreateTargets`.
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Asserts Number of Insights In One PCM Regression Test
If the `EqualWeightingPortfolioConstructionModel` interacts with the `QCAlgorithm.Insights`, the number of elements in the collection should not the sum of emitted insights.
* Refactor Portfolio Construction Models to Use Insight Manager
`PortfolioConstructionModel` will use `QCAlgorithm.Insights" instead of class property `InsightCollection` to manage the insights. It no longer adds insights to the collection, but it removes them if they expire or the securities are removed from the universe.
Updates PCMs that were affected by the change.
* Updates Unit Tests
We need to add the insights to the insight manager before we call `PortfolioConstruction.CreateTargets`
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Performance Improvements
- Remove Immutable collection from position groups to improve
performance. Adding more tests
- Remove ConcurrentDictionary from PortfolioTargetCollection &
DataManager. Adding more tests
- Add Securities enumerator keys & values cache
* Add test for PortfolioTargetCollection remove by reference
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* remove annualization
* Update python optimizer to best fit in convex problem
* rerun test
* test metric
* Add comment to explain using simple return
* Add unit test on PR
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* MRP
* Fix bug
* Rename
* Provide virtual `GetPriceRelative` function to override for any variants in future
* SymbolData class within model
* Address peer review
* fix bug
* revise unit test and fix bug in python model
* revise unit test and fix bug in model
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Revise regression statistics
* Address peer review
* Updated unit tests according to peer review
* Address peer review
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Add new CustomWeight PCM for alpha streams
- Add new AlphaStreams AlphaModule that will handle security additions
and removals, removing this logic from AlphaStreamsBasicTemplateAlgo
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Few improvements
- Alpha Streams algorithm will add target securities right away
- If algorithm is warming up PortfolioTargetCollection will emit not
insight
- Fix local disk factor file provider to use local cache instead of data
folder
- EWAS PCM will not emit targets for securities which have not been
added by the algorithm yet, to avoid runtime exception
* Fix GetLastKnownPrice default order adding more unit tests
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Alpha Streams improvements
- Warmup added securities so we can trigger an order right away
- Fix bug where AddSecurity returned a security which was not the one
being used
- Fix bug where EWAS PCM would remove from a dictionary while iterating
over it
* Fix unit tests
* Add QCAlgorithm.GetLastKnownPrices API
- Add new GetLastKnownPrices that will return the last known data point
for all subscribed data types.
- Fix for MHDB GetDataTimeZone which was using an invalid entry key
format for custom data.
- Fix for using GetLastKnownPrice/s to seed a security during creation,
when it's not added in the Algorithm.Securities collection
* Adding more unit tests for GetLastKnownPrice
* Address reviews
- Refactor GetLastKnownPrice/s to perform a single history request for
all data types
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Order handling improvements
- Execution model will only trigger market order if they are above the
minimum order margin portfolio percetage value
- SecurityCache.Reset is complete
* Python Import fixes
- Add regression test for ImmediateExecutionModel minimum order margin
check
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state
- Alpha result packet will optionally provide the algorithms portfolio
state
* Rename
* Convert AlphaStreamsPortfolio to data source
* Improvements on AlphaStreams algorithm
* Fix regression tests
* Add unit tests for EW AS PCM and fixing bugs
* Protobuf AlphaStreamsPortfoliot staState
- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM
* Add alpha license to Organization response
* Improvements EW AS PCM respects free portfolio value
* Fixes
- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType
* Fixed and adding more regression tests
- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
types
- Add support and regression test for algorithm alpha consumer with existing holdings
* Add AlphaStreamsOrderEvent data type
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data
* Updates Unit Tests
* Updates Regression Tests
In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.
* Updates Regression Tests 2
The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`
The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238
Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
- BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`
The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying
* Revert File Update for SPWR and SPWRA
* Fix Regression Tests
Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.
* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]
Uses `trust-constr` method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Python research import improvements
- Improve start.py for research env
- Remove unrequired imports
* Centralize algorithm imports
* Add regression test GH action
* Unit test python import clean up
* Join research and main imports
* More python import clean up
* Fix failing skipped regression algorithm
* Update projects to use .NET 5.0, the successor to .NET Core
* Fix ambiguous errors. Add IBAutomator net5
* Remove FXCM
* Upgrade IBAutomater to v1.0.51
ignored, and an empty message aborts the commit.
* Fix rebase
- Fix ambiguous Index
- Remove StrategyCapacity.cs
- Update System.Threading.Tasks.Extensionsy
* Remove unrequired references
* Fixes
- Travis will use dotnet, not nunit nor mono
- Remove mono from foundation image
- Fix python setup in research
- Fix unit tests
* Don't call ReadKey when input is redirected
* Fix ConsoleLeanOptimizer
* Research fixes
* Update comment
* Add vsdbg to Dockerfile
* Fixes
- Revert dockerfile FROM custom changes
- Adjust and fix regression algorithms
- Option assignment will be deterministic in the order
- 'Rolling Averaged Population' is calculated using doubles, updating
expected values.
- Update readme, removing references to mono
- Add missing Py.Gil lock
* Replace ICSharp with .NET Interactive
* Fixes after rebase
* CSharp research fixes
- Adding new Initialize.csx that pre loads all assemblies
- Adjusting template research file
- Moving steps in dockerfilejupyter
- Fix unit tests and regression tests after rebase
Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Stefano Raggi <stefano.raggi67@gmail.com>
Co-authored-by: Jasper van Merle <jaspervmerle@gmail.com>
- Fix out of range exception, if the returns array is smaller for some
of the Symbols it will use double.NaN, similar behavior to when a date
isn't found for a Symbol. Adding unit test
- Fix duplicate key exception at ReturnsSymbolData.Returns, this could
happen with FF in history requests. Adding unit test
When new securities are added to the universe, the `ReturnsSymbolData` is warmed up with historical data that may not have the same timestamp causing an index mismatch that leads to a rejection to several valid data. In this case, we will assume that there is a time correspondence similar to what is done in Python. Unit test was added.
`BlackLittermanOptimizationPortfolioConstructionModel` will consider a new view only if there is a new last active insight by updating the `ReturnsSymbolData` with the `Insight.GeneratedTimeUtc` instead of the `IAlgorithm.Time`. Consequently, the timestamp of the historical data is converted to UTC for consistency.
`BlackLittermanSymbolData` now rejects duplicate keys like its C# version: `ReturnsSymbolData`.
Finally, `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` statistics was updated because of the bug fixes.
- Reduce MinimumVariancePortfolioOptimizar precision goal so that both
CSharp and Py MeanVarianceOptimizationFrameworkAlgorithm return the same
results
- Limit factor file dates in factor file generator unit test
Implements `PortfolioBias` in EWPCM, CWPCM and IWPCM. With this new feature, these PCM will ignore insights that do not respect the desired bias. E.g. for `PortfolioBias.Long`, on Insights with `InsightDirection.Up` will be converted into `PortfolioTarget.Quantity` greater than zero and other `InsightDirection` will result in `PortfolioTarget.Quantity` of zero.
- Adding new unit test for python PCM implementations, asserting each
method is correctly called
- Reverting some unrequired changes in the
`MeanVarianceOptimizationFrameworkAlgorithm`
- Refactor shared logic from `EqualWeightingPortfolioConstructionModel`
into base `PortfolioConstructionModel` implementation
- `MeanVarianceOptimizationPortfolioConstructionModel` will respect
rebalancing period and will use all active inisights, not just the last
- Refactor AccumulativeInsightPortfolioConstructionModel to inherit from
the EWPCM, reducing code duplication and adding support for rebalancing
period
- Fixing bug where only 1 new insight per symbol was processed per loop
- Adding unit tests
- Adding new `Func<DateTime, DateTime?>` that allows PCM to return null
if the next rebalance time is null, in which case the function will be
called again in the next loop.
- Adjusting PCM next rebalance time check to perform rebalance once the
time is reached
- Adding new regression test. Updating existing
- Moving InsightCollection into base `PortfolioConstructionModel`
- Will call `InsightCollection.GetNextExpiryTime()` on each check, and
for performance `InsightCollection` will keep track of next insight
expiry time
- Removing need for PCM base classes having to call `RefreshRebalance`
- Some refactor clean up at base
PortfolioConstructionModel.IsRebalanceDue()
- Refactoring some PCM methods to be `protected` since they are not required
to be public
- Adding new `PortfolioConstructionModel.RebalanceOnInsightChanges`
flag, that will allow avoiding new insights or insight expirations to
trigger a rebalance
- Updating unit tests
- Fix for the MeanVarianceOptimizationPortfolioConstructionModel that
was skipping, in some cases, 0 magnitude insights
- Add missing PCM constructor methods for the different supported
rebalancing periods overloads
- Normalize rebalance behavior in the base `PortfolioConstructionModel`
- Adding new `PortfolioConstructionModel.RebalanceOnSecurityChanges`
that will allow disabling rebalance on security changes
- Adding unit tests
Alexandre Catarino