* Wait for first session bar before filling equity market orders at open
EquityFillModel.MarketFill could fill a market order placed right after
market open using data from the previous trading date, because the first
bar of the current session has not been emitted yet. ShouldWaitForFreshData
only covered hour/daily resolutions, so minute/second orders filled on stale
prices.
Add IsWithinFirstResolutionSpanAfterMarketOpen: when the order time is within
the lowest subscribed resolution span after the open and the price is stale,
wait for the first bar instead of filling on the previous date's price.
* Share opening-bar stale-fill wait across fill models
Move IsWithinFirstResolutionSpanAfterMarketOpen to the base FillModel and add
a ShouldWaitForFreshDataOnStale sibling helper that combines it with the
existing coarse-resolution ShouldWaitForFreshData check. The base FillModel,
FutureFillModel and EquityFillModel market fills now share this single wait
decision at their stale-data guards.
ShouldWaitForFreshData is intentionally left untouched at its GetMarketFillPrice
call site, which uses it to choose the bar open vs current price and is not
gated by staleness, so fill prices for finer resolutions are unchanged. The
opening-bar helper is guarded against always-open markets, which have no
session open to wait for.
* Add regression algorithm for stale fill at market open
Reproduces the opening-bar stale fill issue: a market order placed one second
after the open while subscribed to minute resolution. Without the fix the order
fills on the previous trading date's stale price; the algorithm asserts in
OnOrderEvent that a fill never happens within the first minute after the open,
so it errors without the fix and passes with it.
Uses SPY minute data over 2013-10-07 to 2013-10-11, which is available in the
repository Data folder.
* Add unit tests for stale fill wait at market open
Cover the opening-bar stale fill scenario directly at the fill model level:
a market order placed within the first bar after the session open, while only
the previous session's stale bar is available, must wait instead of filling on
the stale price, and fills once the first session bar arrives. EquityFillModel
also asserts the boundary (orders past the first bar still fill on stale data),
and FutureFillModel covers the shared base helper from the future path.
* Generalize stale market-order fill wait to any time of day
Replace the market-open-specific wait with a generic check: a market order
that would be filled on stale data waits for fresh data when the latest
available data is more than one subscribed resolution bar behind the current
time. This no longer considers the market open explicitly; it covers the
opening bar (the first session bar has not been emitted yet) and any intraday
data gap larger than the resolution.
ShouldWaitForFreshDataOnStale now takes the latest data end time and the
current time instead of the order time, and is shared by FillModel,
FutureFillModel and EquityFillModel. Coarse resolutions (hour/daily) still
always wait; tick never waits. Internal configurations are included when
sizing the resolution bar. EquityFillModel's best-effort price helpers now
report the stale data end time so the gap can be measured.
Tests: EquityFillModelTests and FutureFillModelTests cover the market-open and
mid-session stale cases (wait then fill on fresh data) plus the within-one-bar
boundary (fill on stale). The regression algorithm is generalized to assert no
fill happens on data staler than the resolution, with orders at the open and
mid-session. Pre-existing plumbing/data-selection tests that used degenerate
timestamps were given fresh timestamps so they still exercise their original
intent.
* Add sample data and adjust regression algorithms for stale-fill wait
Add minute/daily sample data so market orders that now wait for fresh data
can fill (ES futures gap days, TWX/GOOG equities and options, SPXW weeklies,
GC futures/options copy for 2020-01-06). Adjust a few regression algorithms
to the deferred-fill behavior: cap orders in the extended-market continuous
future test, ignore daily-resolution SPY in the automatic-seed data checks,
and refresh OptionAssignmentStatistics expected constants.
* Update regression expected statistics for stale-fill wait
Regenerate ExpectedStatistics, DataPoints and AlgorithmHistoryDataPoints for
the regression algorithms affected by the wait-for-fresh-data fill change and
the added sample data: futures/options fill-timing shifts, ES data-point count
increases, and GOOG 2015-12-28 outcome changes.
* Trim SPXW sample data to expiries within filter window
The two SPXW algorithms filter with Expiration(0,7), so contracts expiring
more than a week out are never subscribed. Drop those far-dated expiries from
the 2021-01-06/08 minute files (760KB->108KB and 776KB->108KB on the quote
files). Fills, DataPoints and statistics are unchanged; both regression tests
still pass.
* Trim ES minute and GOOG option sample data to order-fill minimum
The ES minute gap-day files source no order fills (daily-resolution algos fill
from es_daily); keep only the front contract used for execution and drop the
unused back-month contracts. Trim the GOOG 2015-12-28 option file (no fill
depends on it) to the morning chain window. Regenerate the back-month futures
statistics affected by the dropped back-month bars. Full CSharp regression
suite passes (722/722).
* Use SMA gap threshold in BasicTemplateContinuousFuture for C#/Python parity
At a fast/slow SMA cross the two averages can coincide to within rounding
noise, where the C# (decimal) and Python (double) comparisons disagree,
producing different orders between languages. Require a minimum gap before
acting on a cross so both languages stay in lockstep, and update the shared
expected statistics accordingly.
* Mirror order cap in Python algorithm and update future history counts
Apply the same pre-2013-11-12/3-order cap to the Python
BasicTemplateContinuousFutureWithExtendedMarket algorithm for C#/Python parity,
and update the QuantBook future-history expected counts to reflect the added ES
sample data.
* Use SMA gap threshold in BasicTemplateContinuousFutureWithExtendedMarket for C#/Python parity
This algorithm had the same fast/slow SMA cross divergence already fixed in
BasicTemplateContinuousFutureAlgorithm (ad8fc33): at the 2013-10-29 cross the two
averages coincide to within rounding noise (C# decimal diff -1e-25, Python double
diff exactly 0.0), so the raw `_fast > _slow` / `_fast < _slow` comparisons disagree
between languages. C# fired a liquidate+rebuild that Python skipped, producing 5
orders in C# vs 3 in Python. Require a minimum 0.001 gap before acting on a cross so
both languages stay in lockstep, and regenerate the shared expected statistics
(Total Orders 5 -> 3).
* Document SMA cross threshold as a C#/Python parity workaround
Add a short note before the fast/slow SMA comparisons in both continuous-future
template algorithms clarifying that the minimum-gap threshold exists only so the
C# and Python versions take the exact same trades on the limited sample data in
the repository, where decimal vs double rounding can disagree at a cross.
* Fetch subscription configs once per equity market fill
MarketFill resolved the subscription configs twice per fill: once via the
best-effort price helpers (GetSubscribedTypes) and again via
ShouldWaitForFreshDataOnStale. Fetch them once and thread them through both
paths via optional parameters, leaving existing callers unchanged.
* Measure stale-fill wait against order submission time
ShouldWaitForFreshDataOnStale compared the latest data end time against the
security current time. Compare against the order submission time instead so the
decision to wait for fresh data reflects how stale the data is relative to when
the order was placed. Realign the stale-price warning fill test accordingly.
* Fix stale market data in SendingNewOrderFromOnOrderEvent test
The market price tick was timestamped a day before the order submission time,
so under the order-time staleness check the market orders waited for fresh data
instead of filling. Use a reference time with the tick one minute before the
order so the data is fresh and the orders fill.
* Centralize internal-inclusive subscription config lookup in fill models
ShouldWaitForFreshDataOnStale re-resolved the subscription configs through
the ShouldWaitForFreshData call it makes first, and GetMarketFillPrice did
the same. Thread the already-fetched configs through ShouldWaitForFreshData
and GetMarketFillPrice so each market fill resolves them at most once.
Add a GetSubscriptionDataConfigs(Security) helper on the base FillModel that
returns the internal-inclusive configs, and route every fill-model call site
through it to remove the duplicated lookup and repeated comment.
* Avoid list allocation in ShouldWaitForFreshData
Replace the Where(...).ToList() + All(...) with a single foreach over the
subscription configs, short-circuiting on the first non-coarse resolution.
* Return weekly contracts if no standard contracts exist
* Fix unit and regression tests
* Centralize default expiration type flags
* Add ExcludeWeeklys() method
* Mark IncludeWeeklys() as obsolete since weeklies are now default
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
* Increase option strike digits precision to 6
- Increase option strike digits precision to 6. Adding unit test
* test:fix: couple of build tests
* fix:test: Asset CUSIP and OrderListHash in Regression Algorithms
* test:fix: several regression algos
* test:fix: old Symbol CUSIPs to new ones
---------
Co-authored-by: Romazes <romazes20@gmail.com>
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Implement a prototype of the maximum recovery time function.
* Add unit test skeletons.
* Add failing test
* Issue #4581: Implement MaxDrawdownRecoveryTime.
* Issue 4581: Add DTO for Drawdown Percentage, Drawdown Enddate, and High Value
* Issue 4581: Fix bgu for when lDrawdowns list is empty.
* Issue 4581: Change names of tests. Change name of file.
* Issue 4581: Make adjustements to flow of adding drawdowns to lDrawdowns.
* Issue 4581: Add multiple unit tests.
* Issue #4581: Change name of unit test
* Issue #4581: Add to PerformanceMetrics
* Issue #4581: Add Maximum Drawdown Recovery to PortolioStatistics class.
* Issue #4581: Add to portolfio statistics class.
* Issue #4581: Add to statistics builder.
* Issue #4581: Add report key.
* Case #4581: Convert to decimal.
* Issue #4581: Correct comment.
* Issue #4581: Correct performance metrics view model string.
* Case #4581: Correct statistics builder view model string..again.
* Issue #4581: Placed DradownDradownDateHighValueDTO at the end of the file for simpler diff.
* Issue #4581: Add 2 new tests.
* Issue #4581: Change algorithm so that when multiple maximum drawdowns occur, the longest of all recoveries is reported.
* Issue #4581: Add unit test.
* Issue #4581: Remove reportkey. Change dto name.
* Issue #4581: Change summary.
* Issue #4581: Change comment.
* Add max drawdown recovery calculation with unit tests
* Update regression algorithms with the new metric
* Solve review comments
* Update regression algorithms
* Add TryGet to safely get the key: MaximumDrawdownRecovery
* Ignore MaximumDrawdownRecovery metric in OptimizationBacktest Json
* Revert changes in Messaging
* Update regression algorithms
* Add test case: TakesLongestRecoveryAmongMultipleDrawdowns
* Use integer days for MaximumDrawdownRecovery
* Add MaximumDrawdownRecoveryReportElement
* Use more explicit names
* Rename files and variables for consistency
* Update regression algorithms
---------
Co-authored-by: Alain Schaerer <aschaerer@pcatg.com>
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Initial options universe with greeks implementation
* Options universe improvements
* Address peer review
* File based options universe fixes and improvements.
- Adjust OptionUniverse start-end times and period.
- Adapt unit tests and some algorithms to pass with new options universe selection.
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Updated options regression algorithms stats for new universe data
* Option chain provider with new options universe
* Allow canonical option history requests
* Address peer review
* Address peer review
* Fix symbols parsing in OptionUniverse
* Fix universe selection subscriptions start time to not include extended market hours
* Minor changes
* Minor changes
* Peer recommended changes and fixes
* Update regression algorithm stats
* Update regression algorithms stats and minor fixes
* Fix option chain provider history request
* Round option indicators values
* Added option universe csv header property
* Update regression algorithms stats
* Update regression algorithms stats
* Data fixes and regression algos stats update
* Unit test fixes
* Minor changes
* Option chain handling in live trading data feed
* Minor changes
* Added processed data provider
* Fix thread-safety violation in Slice class
* Minor change
* Update options filter universe API to use OptionUniverse data
Add new filter methods for greeks, IV and open interest
* Option filter universe api updates
* Add OptionUniverse history regression algorithms
* Add regression algorithms for new options filter universe api methods
* Added options greeks data and updated regression algorithms
* Address peer review
* Address peer review
* Add more assertions to new options filter api regression algorithms
* Minor performance improvement.
Reduce greeks binomial model steps to 140
* Minor tests updates
* Greeks numerical models performance improvements
* Greeks numerical models performance improvements
* Revert array pool change for option pricing numerical models
* Update default dividend yield provider depending on option type
* [TEST]
* Add helper method con calculate time till expiration
* Use double in price option numerical models
* Implied volatility calculation improvements
- Adjust root finding method accuracy as a factor of the option price
- Use BSM to get a first guess
* Cleanup
* Some regression algorithms and unit tests cleanup
* Regression tests updates after rebasing from master
* Add universe files
* Self review and cleanup
* Minor regression tests updates after rebase
* Fix: set data time zone to same as exchange tz for options universes
* Minor change
* Minor change
* Fix for live trading options universe selection
* Keep underlying when aggregating collections in BaseDataCollectionAggregatorEnumerator
* Update index options regression algorithms stats
* Minor change
* Address peer review
* Memory usage improvements
* Minor build fix
* Minor changes and test fixes
* Cache symbols in OptionUniverse
* Cleanup
* Fix index option creation in OptionUniverse
* Use cached underlying SID when parsing from string
* Abstract symbols cache to BaseDataCollection
* Return actual underlying symbol when mapping decomposing ICO ticker
* Address peer review
* Minor performance improvements reduce garbage
* Limit Symbols and SIDs cache size to help with memory usage
* Minor fix in symbols and sid cache cleanup
* Build fix
* Lazily parse greeks on individual access
* Cleanup and tests
* Address peer review
* Minor greeks fix
---------
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
* Fix CA1819 and CA1002 warnings
Changed the type of Languages statistic in regression tests from
Language[] to List<Language>. By doing that, the warning CA1819 was
removed but then the warning CA1002 was raised. However, this warning
was expected to be excluded from QuantConnect.Algorithm.CSharp.
* Improve implementation
* Simplify code
* Fix bugs
* First draft of the solution
* Add missing changes
* Remove the new KPI's from report
* Fix bugs
* nit change
* Add improvements
* Fix regression tests
* Solve bugs in the regression algos
* Fix regression tests bugs
* Expand unit tests and add minor changes
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add Sortino ratio to statistics and report
* Adds Sortino Ratio to Report Key Statistics
* Addresses Peer-Review
Reuse `SharpeRatioReportElement` and change the template.
* Reuse Calculations Across Statistics and PortfolioStatistics
* Adds Sortino Ratio to Regression Algorithms
* Removes Sortino Ratio from Optimization Result Table
---------
Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add new Order.PriceAdjustmentMode property
* Minor fix and unit test
* Minor fix and regression algorithms' stats update
* Unit test fixes
* Minor fix
* Set order price adjustment mode to raw always for live trading
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Modify IB Fee model to work without order.Price
* Fix bugs and add unit tests
* Update regression tests
* Solve bug and address changes
* Nit change
* Fix stats
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Make maintenance margin zero for long options
* Use OptionIntialMargin in OptionMarginModel
* Add naked call and put strategies unit tests
* Minor fix
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Account for premium in option strategies margin
* Minor fixes
* Add OptionInitialMargin class to separate strategies margin and premium
* Minor fix
* Minor fix
* Cleanup, docs and other minor changes
* Minor changes
* Minor changes
* Minor changes
* Address peer review
* Add support for Null SecurityPositionGroupModel
- Add support for algorithms to set the null security position group
model which will not group securities together but rather return the
single group buying power model. Adding regression algorithms
* Add Null Margin python regression & cleanup
- Add python regression algorithm & minor regression algorithms clean up
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Handle liquidating option position with an option strategy
* Handle liquidating option position with an option strategy
* Minor changes
* Minor fix in default margin model margin call order generation
* Fix PositionGroupBuyingPowerModel quantity for delta calculation
The model was calculation the order size that required a margin equal to
the given delta. This fix makes it so that it calculates the position
group order quantity required to get the given buying power delta,
taking into account the holdings for the existing position group in the portfolio.
* Fix PositionGroupBuyingPowerModel quantity for delta calculation
* Add unit tests
* Add unit tests
* Add unit tests
* Housekeeping
* Housekeeping
* Adapt unit tests and move messages to Messages class
* Improve quantity calculation for target BP loop
- PositionGroup.Quantity is now always positive.
- Added PositionGroupExtensions.Closes method and use it to check if reducing a position in PositionGroupBuyingPowerModel.GetPositionGroupBuyingPower. This is necessary since now a PositionGroup side is always "long" (see first bullet point).
* Improvements
* Improvements and new PositionGroup unit tests
* Unit tests improvements
- HasSufficientBuyingPowerForOrder test cases were completely updated to
make it accurate in terms of Lean's margin calculation.
- BuyingPowerForPositionGroupCalculation test was fixed in order to get
the correct expected result.
* Fixes for failing tests
* Added additional check to OptionStrategyMarginCallEventsAlgorithm
* Improve documentation comments
* Fix option strategies BuyingPowerForPositionGroupCalculation unit test
* Fix PositionGroupBuyingPowerModel.GetPositionGroupBuyingPower
Also address peer review
* Minor changes
* Update unit tests for new inverse strategies
Also fixed PositionGroupExtensions.WithQuantity to resolve inverse
strategy
* Minor fix
* Allow target/delta to go to the opposite direction of the position group
* Housekeeping
* Handle quantity for target/delta BP request for different reference position group
* Revert removal of PositionGroupBuyingPowerParameters.Direction
* Minor changes
* Peer review
* Minor tweaks on the position group buying power model
* Minor fix
Jhonathan Abreu