Commit Graph

23 Commits

Author SHA1 Message Date
Alexandre Catarino 58793acae8 Removes Assets that Are Not Invested and Not Tradable From Capacity Calculations (#5515)
Build & Test Lean / build (push) Has been cancelled
* Add property for capacity. Remove unused variable

* Move SymbolCapacity and CapacityEstimate to common, passed through Symbol to runtime statistics

* Add null checks

* Remove uninvested and untradable assets from capacty calculations

* Add SymbolCapacity influential period

* Updates Regression Tests

- DelistingEventsAlgorithm
  - Allows additional contributions from delisted AAA.1

- DelistingFutureOptionRegressionAlgorithm
  - Removes DC01H12 contributions one month later

- FutureOptionBuySellCallIntradayRegressionAlgorithm
  - Allows additional contributions from future after expiry replacing the contribution of the next contract option

- DelistedFutureLiquidateRegressionAlgorithm
- FutureOptionCallITMExpiryRegressionAlgorithm
- FutureOptionCallITMGreeksExpiryRegressionAlgorithm
- FutureOptionPutITMExpiryRegressionAlgorithm
- FutureOptionShortCallITMExpiryRegressionAlgorithm
- FutureOptionShortPutITMExpiryRegressionAlgorithm
- FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm
  - Allows additional contributions from future after expiry

- FutureOptionCallOTMExpiryRegressionAlgorithm
- FutureOptionPutOTMExpiryRegressionAlgorithm
- FutureOptionShortPutOTMExpiryRegressionAlgorithm
- IndexOptionCallITMGreeksExpiryRegressionAlgorithm
- IndexOptionCallOTMExpiryRegressionAlgorithm
- IndexOptionShortCallOTMExpiryRegressionAlgorithm
  - Allows additional contributions from option after expiry

- MACDTrendAlgorithm
  - Removes contribution when SPY is not invested for over one month

- UniverseSelectionRegressionAlgorithm
  - Allows additional contributions from delisted GOOAV replacing GOOG (new symbols)

* Adds Lowest Capacity Asset to Regression Tests

* Normalize expected value -0, because -0 is also written to file if updated

* Write Symbol.Value for lowestCapacitySymbol or  empty string for empty Symbol

* Update Regressions

* Update 'Lowest Capacity Asset' to Symbol.ID

Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
2021-05-13 20:25:19 -03:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
2021-03-02 18:46:31 -03:00
Gerardo Salazar e2964dd4b1 Make OrderListHash deterministic by using MD5 as its underlying hash function (#5276)
* Update OrderListHash to use MD5 as hash instead of hash code

* Update regression algorithm OrderListHash statistic

* Use full MD5 hash as OrderListHash, update regression statistic

* Fixes failing regression tests
2021-02-09 12:19:25 -03:00
Martin-Molinero 3ccf428498 Adjust delisting liquidation time (#5203)
* Adjust delisting liquidation time

- Adjust delisting liquidation time to 15 min before market closes.
  Adding unit tests. Updating existing.
- Handle `Statistics.CompoundingAnnualPerformance` invalid calculation
  to avoid exception.
- AlgorithmManager will not handle delisting events in live trading
- Fix bug where due to a split driven liquidation matching delisting
  date a position in the option would remain open. Reproduced by
  `BasicTemplateOptionsFrameworkAlgorithm`

* Address review

- Address review add documentation on delisting offset span
2021-01-22 14:41:29 -03:00
Martin-Molinero 592d037085 Fix delisted liquidation orders being cancelled (#5169)
* Fix delisted liquidation orders being cancelled

- Place delisted liquidation orders 10 min before market closes of 10
  min before the end of the delisting warning date. Adding regression
  test and unit tests. Updating existing tests.

* Fix failing python option unit tests

* Fix bug where positions would be open delisting liquidation

* Fix universe selection and delisting

- Delisting will happen ASAP for all types. Giving priority to close
  positions on derivates first
- Fix bug in universe selection where OptionChain would remove
  underlying even if holding a position in derivate.
- Updating regression tests statistics

* Add unit test, fix unit test expected stats
2021-01-18 22:02:40 -03:00
Michael Handschuh cf9b547e2e Refactor OptionExerciseOrder.Quantity to be consistent with other Order types (#4796)
* Improve information tracked in regression's {algorithm}.{lang}.details.log

The details.log file aims at providing a diff-able document that quickly and
easily provides actionable information. Since many regression algorithms use
the algorithm's debug/error messaging facilities to log various pieces of algo
state. This document also support a configuration option: regression-high-fidelity-logging'
that logs EVERY piece of data, again, with the aim of providing an easily diff-able
documenbt to quickly highlight actionable information. I may have missed omse key
pieces of information here, but now that the entire QC knows about this regression
tool, if additional information is required then hopefully it's easy enough at this
point to extend the RegressionResultHandler to suit our needs.

The RegressionResultHandler was initially implemented to provide a concise log of
all orders. This was achieved by simply using the Order.ToString method. While
testing/investigating OptionExerciseOrder behavior, it became evident that more
information was required to properly identify the source of potential failures or
differences between previous regression test runs. This change adds logging for
almost every IResultHandler method and additionally attempts to capture the
actual portfolio impact of every OrderEvent. This is accomplished by logging
the portfolio's TotalPortfolioValue, Cash properties and the security's
SecurityHolding.Quantity property.

This change also standardizes the timestamps used to folloow the ISO-8601 format.

When using the RegressionResultHandler, it is highly recommeded to also disable
'forward-console-message' configuration option to ensure algorithm Debug/Error
message logging is done synchronously to ensure correct ordering with respect to
log messages via Log.Debug/Trace/Error.

* Fix typo in options OrderTests test case name

* Update SymbolRepresentation.GenerationOptionTickerOSI to extension method

Far more convenient as an extension method

* Improve R# default code formatting rules

Many of these rule changes focus on improving the readability of code,
with a particular emphasis on multi-line constructs, chained method calls
and multi-line method invocations/declarations.

* Add braces, use string interpolation and limit long lines

* Refactor OptionExerciseOrder.Quantity to indicate change in #contracts

For all other order types, the Order.Quantity indicates the change in the algorithm's
holdings upon order execution for the order's symbol. For OptionExerciseOrder, this
convention was broken. It appears as though only exercise was initially implemented,
in which case only long positions were supported and a code comment indicated that
only positive values of quantity were acceptable, indicating the number of contracts
to exercise. At a later date, assignment simulation was added and utilized a negative
order quantity. This caused some major inconsistencies in how models view exercise
orders compared to all other order types. This change brings OptionExerciseOrder.Quantity
into alignment with the other order types by making it represent the change in holdings
quantity upon order execution.

This change was originally going to be much larger, but in order to minimize risks and to
make for an easier review experience, the additional changes will be committed separately
and pushed in their own PR. Some of the issues identified include:
* Manual Exercise (especially for OTM) is not covered
* Margin Calculations (in particular taking into account opposing contracts held)
* IBrokerage.OptionPositionAssigned is raised for exercise (later filtered by tx handler)

Fixes OptionPortfolioModelTests to use exercise model to properly model exercise of
non-account quote currency option contract.
2020-10-08 20:05:20 -03:00
Michael Handschuh c7a74306fb Bug 4731 Fix Option Expiration Order Tags and Order Event IsAssignment Flag (#4743)
* Add OrderRight.GetExerciseDirection(isShort) extension

Returns the OrderDirection resulting from exercise/assignment of a particular
option right

See: BUG #4731

* Fix option exercise/assignment order tags and order event messages

The algorithm manager was doing work to determine whether or not the option ended
in exercise or assignment at expiration. This decision should be left for the exercise
model to decide -- from the algorithm manager's perspective, all that matters is that
the option was expired. The DefaultExerciseModel was updated to properly track whether
the option expired with automatic assignment or exercise, dependending on whether or
not we wrote or bought the option (held liability or right, respectively). Updated unit
tests to check for order event counts and order event messages for option exercise cases.

Fixes: #4731

* Fix typo in algorithm documentation

* Update regression tests order hash

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-09-28 15:29:09 -03:00
Colton Sellers f135fb8060 Bug Backtesting Brokerage Clones (#4644)
* fix order updates

* Fix option exercise issue

* Regression changes

* Update regressions to reflect fixes

* Refactor handling of order to fully fix #2846

* Regression Algorithm for unit test

* Pre review

* Fix breaking tests

* OrderImmutability Regression Algo

* OrderImmutability Regression Algo Compile

* Address review

* Update regressions with new orderhash
2020-09-09 19:44:56 -03:00
Martin Molinero 2a0b73e1cf Update regression algorithms
- Update regression algorithms after removing order based insights
2020-04-01 20:37:20 -03:00
Martin Molinero 486b2e6641 Update regression stats
- Update expected OrderListHash in regression algorithms due to
serialization improvements
2020-03-31 12:21:13 -03:00
Martin Molinero 0027ab1e66 Regression test assert order list hash
- Regression tests assert order list hash value
- Normalizing some regression test behavior that had differences between
C# and Py
2020-02-19 20:26:12 -03:00
Martin Molinero 20c589f0fe Validate option universe selection data
- Remove unused `OptionChainUniverseDataCollectionAggregatorEnumerator`
- Refactor `BaseDataCollectionAggregatorEnumerator` and
`OptionChainUniverseDataCollectionEnumerator` to avoid emitting invalid
data points
- Updating regression tests statistics
- Adding unit test
2020-01-27 19:11:17 -03:00
Gerardo Salazar 61aa0d3a65 Updates regression statistics
Please note that with these changes, any algorithms that
use daily data exclusively will have incorrect statistics.
2020-01-16 14:03:54 -08:00
Martin Molinero 2f4929e685 Update regression algorithms 2019-09-29 21:50:44 -03:00
Martin Molinero cb4c574eeb Adjust algorithms statistics 2019-08-06 21:35:25 -03:00
Martin Molinero 40ac9c5033 Fix IB FeeModel Option fees
- Fixing IB FeeModel options fees. Adding unit tests
- Updating regression algorithms
2019-01-02 15:50:00 -03:00
Michael Handschuh 380caa5203 Add IRegressionAlgorithmDefinition.CanRunLocally
This flag indicates whether or not the local regression test system,
via RegressionTests.AlgorithmStatisticsRegression should run a given
IRegressionAlgorithmDefinition
2018-07-18 15:57:11 -04:00
Michael Handschuh 7eb4526724 Update open source data set and regression statistics
Factor files from 2018.06.04
Data   files from 2018.07.05
2018-07-05 15:13:57 -04:00
Martin Molinero 96dc5e8673 Moving IRegressionAlgorithmDefinition to QuantConnect.Common 2018-07-03 13:34:27 -03:00
Michael Handschuh 9ee61f425c Refactor regression algorithm to IRegressionAlgorithmDefinition
A mechanical refactoring was performed to make algorithms currently used in
regression algorithms to implement IRegressionAlgorithmDefinition, which allows
algorithms to define their own expected statistics and what languages should be
run as part of regression. The type name of the  C# type is used to determine the
file/model name for python. This was for simplicity, but if needed, could later be
refactored to expose more information, but for now the convention of keeping names
the same makes sense and just works easily.
2018-06-05 12:10:50 -04:00
Michael Handschuh e5c025706a Convert object finalizer into OnEndOfAlgorithm
There's no good reason to use a finalizer here.
Regression algorithms should use the OnEndOfAlgorithm event to handle
checks/assertions following the algorithm's completion.
2018-06-04 17:49:38 -04:00
Jared Broad fb7d1994ff Tidy and add tags 2017-09-20 17:14:44 -04:00
quant1729 c416df6241 Refactored option assignment/exercise:
1. Made sure we treat properly option expiration dates before Feb 2015 and after. Added tests.
2. Refactored expiration delisting, assignments/option exercise to happen in the end of the date, not MOC orders in the beginning of the day. Regression test.
3. Refactored option exercise model to generate proper fills on assignments/option. Those fills are reflected correctly in margins, in stats and correspond to IB model. Still need to run IB real-life live tests.
4. Refactored option symbol related functions into separate module (OptionSymbol.cs)
5. Made sure OnAssignmentEvent arrived to the user algo in regression test. Do we need OnExercise event? Not sure.
6. Tested end-to-end Ray's current code (covered call strat) to see short option legs expire worthless, and stats updated.
2017-02-02 19:35:37 +08:00