Commit Graph

20 Commits

Author SHA1 Message Date
JosueNina 7008d17714 Add MaxDrawdownRecovery metric (#8865)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Implement  a prototype of the maximum recovery time function.

* Add unit test skeletons.

* Add failing test

* Issue #4581: Implement MaxDrawdownRecoveryTime.

* Issue 4581: Add DTO for Drawdown Percentage, Drawdown Enddate, and High Value

* Issue 4581: Fix bgu for when lDrawdowns list is empty.

* Issue 4581: Change names of tests. Change name of file.

* Issue 4581: Make adjustements to flow of adding drawdowns to lDrawdowns.

* Issue 4581: Add multiple unit tests.

* Issue #4581: Change name of unit test

* Issue #4581: Add to PerformanceMetrics

* Issue #4581: Add Maximum Drawdown Recovery to PortolioStatistics class.

* Issue #4581: Add to portolfio statistics class.

* Issue #4581: Add to statistics builder.

* Issue #4581: Add report key.

* Case #4581: Convert to decimal.

* Issue #4581: Correct comment.

* Issue #4581: Correct performance metrics view model string.

* Case #4581: Correct statistics builder view model string..again.

* Issue #4581: Placed DradownDradownDateHighValueDTO at the end of the file for simpler diff.

* Issue #4581: Add 2 new tests.

* Issue #4581: Change algorithm so that when multiple maximum drawdowns occur, the longest of all recoveries is reported.

* Issue #4581: Add unit test.

* Issue #4581: Remove reportkey. Change dto name.

* Issue #4581: Change summary.

* Issue #4581: Change comment.

* Add max drawdown recovery calculation with unit tests

* Update regression algorithms with the new metric

* Solve review comments

* Update regression algorithms

* Add TryGet to safely get the key: MaximumDrawdownRecovery

* Ignore MaximumDrawdownRecovery metric in OptimizationBacktest Json

* Revert changes in Messaging

* Update regression algorithms

* Add test case: TakesLongestRecoveryAmongMultipleDrawdowns

* Use integer days for MaximumDrawdownRecovery

* Add MaximumDrawdownRecoveryReportElement

* Use more explicit names

* Rename files and variables for consistency

* Update regression algorithms

---------

Co-authored-by: Alain Schaerer <aschaerer@pcatg.com>
2025-07-17 16:32:23 -03:00
Jhonathan Abreu 69d2f5ae82 Futures and Future Options file-based universes (#8480)
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Make FOPs selection universe file-based for backtesting

* Make FOPs option chains universe file based

* Make Future universe selection file-based like option universe

* Make Future universe selection file-based like option universe

* Abstraction cleanup

* Add FuturesChains API to QC algorithm

Also refactor future chain provider to use the new FutureUniverse instead of zip file names

* Update regression algorithms stats

* Refactor QuantBook option and future history to use new universes

* Fix failing tests

* Fix failing tests

* Fix failing tests

* Minor future chains unit test improvement

* Add futures chains DataFrame property

Also, remove IDerivativeSecurity interface from Future

* Add DataFrame property to FuturesChains class

* Add regression algorithms

* Add regression algorithms

* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api

* Minor fixes

* Reduce number of universe files in repo

* Minor data fixes

* Regression algorithms updates

* Add implicit conversion from FuturesContract to Symbol

Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.

* Improve resolution handling for history requests

* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property

* Refactor data handling in BaseChain and TimeSliceFactory

- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method

* Remove specific constructors and indexers from Chain classes

Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.

* Remove chain cache logic from FuturesChainUniverse

* Refactor class and interface names for clarity

Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.

* Add base class for options and futures contracts

- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface

* Add minor fix for future options tickers parsing

Added tests

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Clean chain provider classes up

* Remove ZipEntryName other classes and unused code

Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.

* Minor changes and cleanup

* Trigger Build

* Trigger Build

* Refactor FuturesContract data handling

Forward price data from bars and ticks stored in private fields for improved memory usage

* Fix: use universe data for market data in FuturesContract

* Update regression algorithms stats after rebase

Added HSI futures universe files

* Sort configs by internal flag

Internals go first

* Throw from option universe data filters for future options

Future options IV, Open interest and greeks are not supported for future options

* Minor changes

* Improve some regression algorithms

* Minor fix for failing unit tests

* Update FOPs universe file header

Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.

* Minor unit test fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Add history provider as constructor argument for chain providers

* Update new regression algorithms data points count

* Minor fix for FakeDataQueue

* Add initialize method to chain providers classes

* Minor changes

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Minor fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Add logs to ProcessedDataProvider

* Removed test logs

* Minor fix

* Support downloading options and futures universe files from api data provider
2025-03-25 16:22:38 -04:00
Jhonathan Abreu ec428c5f6c Daily options contracts valid open interest values (#8450)
* Fix for daily options open interest data to be added to contracts

* Cleanup

* Update regression algorithms stats

* Minor fix and cleanup

* Minor fix

* Minor fix

* Update regression algorithms data point count

* Address peer review
2024-12-12 14:50:59 -03:00
Jhonathan Abreu 7eea365aa1 Let delistings through for internal subscriptions (#8398)
* Let delistings through for internal subscriptions

Without it, previously mapped continuous future contracts for which a position is open never received a delisting event for them to be liquidated and marked as non-tradable

* Update regression algorithm stats

* Revert/remove transaction handler initialization parameters DTO class

* Cleanup
2024-11-25 11:41:02 -03:00
Martin-Molinero 7879795207 Enable daily precise end time by default (#8254)
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Default daily precise end times

- Enable by default daily precise end times. Updating stats
- Minor fix for algorithm manager consolidator updates, adding new regression test
  asserting behavior and updating others
- Minor fix for SubscriptionData creator avoid round down on warmup if
  not appropiate
- Adjust consolidators to emit on daily strict end times if requested
  daily resolution and setting enabled
- Updating regression algorithms

* Skip daily data on extended market hours

* Some cleanup and self review

* Revert unrequired change
2024-08-14 12:49:56 -03:00
Martin-Molinero 6c30157fab Remove universe selection on extended market dates (#8160)
- Remove universe selection on dates with extended market hours only
- Updating regression algorithms
- Expand date & time rules API to support specifying whether extended
  market hours only dates are desired or not
2024-07-08 11:58:50 -03:00
Ricardo Andrés Marino Rojas 16a737ee26 Fix most of CA1725 warnings (#8154)
* Fix CA1725 warning

* Address requested changes
2024-07-05 16:19:56 -03:00
Ricardo Andrés Marino Rojas a2b420cb0a Fix warnings part 8 (#8113)
* Fix CA1819 and CA1002 warnings

Changed the type of Languages statistic in regression tests from
Language[] to List<Language>. By doing that, the warning CA1819 was
removed but then the warning CA1002 was raised. However, this warning
was expected to be excluded from QuantConnect.Algorithm.CSharp.

* Improve implementation

* Simplify code

* Fix bugs
2024-06-28 10:10:30 -03:00
Ricardo Andrés Marino Rojas b41df5b0f2 1/4 of the changes of RegressionTestException (#8100)
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-06-24 15:23:33 -03:00
Ricardo Andrés Marino Rojas 8c33536498 Add algorithm status statistic (#8095)
* First draft of the solution

* Fix bugs

* Fix bugs
2024-06-24 14:19:32 -03:00
Ricardo Andrés Marino Rojas ead2efe6b9 Add Starting and Ending KPI's (#7811)
* First draft of the solution

* Add missing changes

* Remove the new KPI's from report

* Fix bugs

* nit change

* Add improvements

* Fix regression tests

* Solve bugs in the regression algos

* Fix regression tests bugs

* Expand unit tests and add minor changes
2024-03-25 15:40:38 -03:00
Martin-Molinero feff802479 Standardize trade count statistic (#7827)
* Standarize trade count statistic

* Rename 'Total Trades' to 'Total Orders'
2024-03-06 14:52:34 -03:00
Jhonathan Abreu 09db45c91d Universes immediate selection (#7785)
* Make sure continuous futures universe selection is performed right away on the start of the algorithm.

This ensures the Future.Mapped is not null

* Minor changes

* Minor fixes

* Update regression algorithms stats

* Add DC futures map file

* Update DC futures map file

* Update regression algorithms stats

* Include ETFs for immediate universe selection

* Include Coarse Fundamentals for immediate universe selection

* Include Futures Chain Universe for immediate selection

* Add Option Chain Universe test

* Include custom universes for immediate selection

* Minor updates

* Include constituents universes for immediate selection

* Minor update

* Minor changes, docs and stats updates

* Minor changes

* Minor changes and clean up

* Minor regression algorithm stat update
2024-02-21 08:59:58 -04:00
Derek Melchin eefa74baaa Add Sortino ratio to statistics and report (#6698)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add Sortino ratio to statistics and report

* Adds Sortino Ratio to Report Key Statistics

* Addresses Peer-Review

Reuse `SharpeRatioReportElement` and change the template.

* Reuse Calculations Across Statistics and PortfolioStatistics

* Adds Sortino Ratio to Regression Algorithms

* Removes Sortino Ratio from Optimization Result Table

---------

Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
2023-12-12 21:06:13 -03:00
Ricardo Andrés Marino Rojas dcb5f8ee4e Merge branches 7501 and 7506 (#7605)
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Update Future-cme-[*] and Future-cme-ES

Acoording to `pandas_market_calendars` there were some dates in
Future-cme-[*] who wasn't early_closes, so they needed to be removed
from there. On the other hand, the early closes list of Future-cme-ES were
shifted by 1 hour according to CME webpage. Besides, there were some
missing dates.

* Update CME Future entries in MHDB

* Rebase

* nit change

* Fix unit tests

* Resume after early close/halts

* Add missing dates in MHDB and fix bugs in it

* Fix bug, add more unit tests and add docs

* fix regression algos

* address required changes

* Update failing regression test stats

After debugging the tests it was found they were failing due to the last
change on SecurityExchangeHours.IsOpen(). That method wasn't taking into
account that even if there is a late open after an early close if the
timespan is after the early close but before the late open, the market
is still close.

* enhance solution

* Update and fix bugs in MHDB

* Address required changes and update stats

* Update stats after rebase

* Nit change

* Missing update to regression test

* Use MHDB instead of USHoliday for Expiration Dates

VIX expiry function now relies completely on MHDB. However, it had to be
created an entry in MHDB for VIX since there wasn't one for it. CBOE
webpage only provided 2023 holidays so only those dates were considered
in the Holidays entry in MHDB. Therefore, some unit tests failed so it
was necessary to change also the VIX entry in FuturesExpiryFunctionsTestData.xml.

* Remove Global.cs/USHolidays class

* Use a lazy implementation

* First draft of the solution

* Use MHDB in FuturesExpiryFunctions.cs

* Remove unused class and fix indentation errors

* Fix indentation errors

* Nit changes

* Merge branches 7501 and 7506

* Merge changes in 7501 and 7506

In order to check compatibility between those branches, a new branch
was created out of branch 7501 and then it was merged with branch 7506.
2 regression tests and 8 unit tests failed, the regression tests failed on
the DataPoint stats. On the other hand, the unit tests failed since the
default parameter UseEquityHoliday was removed from
FuturesExpirtyUtilityFunctions.AddBusinessDays() and from other methods in
the same class too.

* Add missing changes

* Remove repeated good fridays

* Address minor review

---------

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2023-12-04 18:09:57 -03:00
Martin-Molinero c33ce2f8c3 MHDB Merge Common Holidays, LateOpens & EarlyCloses (#7278)
* MHDB will merge common entry

- The MHDB will merge the market and security common entry holidays,
  early closes and late opens

* Normalize & reuse future US holidays

- Normalize & reuse future US holidays

* Update existing unit tests expected stats
2023-05-30 11:54:28 -03:00
Jhonathan Abreu 0b81cf0218 Add dataMappingMode parameter to History methods (#7204)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add dataMappingMode parameter to every history api method overload

* Minor unit tests fixes

* Update regression algorithm stats

* Minor changes

* Minor changes
2023-04-21 17:24:14 -03:00
Martin-Molinero bbbab6d9a8 Refactor alpha statistics phase I (#7055)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics

- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
  algorithms

* Optimization backtest result json converter update

* Address reviews

- Remove IAlphaHandler, move insight storage responsability to IResultHandler
  and centralizing insight collection on the QCAlgorithm.Insights to be
  reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
  sampling handling. Updating regression algorithms

* Add InsightCollection tests and minor fixes

* Adding more & improved tests
2023-03-10 13:12:23 -03:00
Martin-Molinero c4433098c3 Refactor Framework Statistics (#7041)
* Refactor framework statistics

* Further insight chart cleanup
2023-03-03 19:40:29 -03:00
Martin-Molinero 3507c91640 Fix future daily price scale factor update (#6963)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix future daily price scale factor update

- Fix future daily price scale factor update which was happening 1 day
  late due to data and exchange time zone differences.
  Adding regression algorithm reproducing and asserting it.

* Address review
2023-02-14 20:09:59 -03:00