398 Commits

Author SHA1 Message Date
Jhonathan Abreu bc05aa03f9 Fix win/loss count statistics for options assignments (#7354)
* Add ITM option assignment trade as win in trade statistics

* Add TradeStatistics unit tests

* Add unit tests

* Add OptionTrade class to abstract IsWin method

* Keep track of winning/losing trades

* Add more unit tests

* Change Trade.IsWin to a property calculated by the trade builder

* Improve regression algorithm

* Handle ITM option sell win/loss statistics

* Update regression algorithms statistics

* Revert accidental unwanted changes

* Address peer review

* Keep track of winning and losing transactions for portfolio statistics

* Add more SecurityPortfolioModel unit tests

* Fix failing unit tests

* Minor changes

* Address peer review

* Minor fix
2023-07-10 10:20:29 -03:00
Martin-Molinero 0303cf079f Remove Capacity from Runtime Statistics (#7353)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2023-07-05 11:23:59 -03:00
Jhonathan Abreu 5d6110f38c Make summary statistics available to algorithms at runtime (#7320)
* Make statistics available at runtime to algorithms

* Re-calculate statistics on every call

* Housekeeping

* Add regression algorithms

* Address peer review

* Support for custom summary statistics at runtime

* Minor changes

* Address peer review

* Address peer review

* Minor changes

* Minor changes
2023-06-19 17:34:59 -03:00
Martin-Molinero 9c893e83a2 Minor optimizer improvement (#7179)
- Handle optimization backtest name
- Allow configurable optimizer for Optimizer.Launcher project
2023-04-12 15:35:54 -03:00
Martin-Molinero 403814d64f Handle error sending notification (#7146) 2023-03-27 15:31:53 -07:00
Martin-Molinero 56881e64fe Expand result state attributes (#7097)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Add order, insight & log count into result algorithm state
2023-03-20 15:00:57 -03:00
Martin-Molinero 52de357b27 Terminal Link Minor Fixes (#7093)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix data-queue-handler configuration initialization using paper brokerage
- Fix for downloader data provider request times
- Fix null reference exception in live result handler if algorithm is
  null
2023-03-20 12:24:16 -03:00
Martin-Molinero a9c88ad25c ResultHandler state file improvements (#7078)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Adjust start/end time string format
- In live trading will store the status file at the start, every 10 min
  and at the end
2023-03-16 18:24:49 -03:00
Martin-Molinero 06bec5fb8e Add more logs for regression tests (#7073)
- Add holdings & cashbook logs from regression result handler, useful
  for debugging & developing
2023-03-15 18:25:26 -03:00
Martin-Molinero c6322956d9 Live trading holdings key (#7043)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix live trading holdings uniqueness

- Fix live trading holdings key uniqueness

* Fix existing & add new test
2023-03-13 09:51:00 -03:00
Martin-Molinero bbbab6d9a8 Refactor alpha statistics phase I (#7055)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics

- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
  algorithms

* Optimization backtest result json converter update

* Address reviews

- Remove IAlphaHandler, move insight storage responsability to IResultHandler
  and centralizing insight collection on the QCAlgorithm.Insights to be
  reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
  sampling handling. Updating regression algorithms

* Add InsightCollection tests and minor fixes

* Adding more & improved tests
2023-03-10 13:12:23 -03:00
Jhonathan Abreu 81d1bb5095 Sample exposure for non invested portfolio (#7030)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2023-03-02 15:23:10 -03:00
Martin-Molinero 48914b08db Chart Series Settings Lost (#6997)
* Fix live trading chart series index

- Fix Series loosing their index when handled by the
  LiveTradingResultHandler
- Some clean up for Series.cs

* Fix SeriesSampler loosing Series settings
2023-02-21 13:07:56 -03:00
Ronit Jain a9c4d29776 Feature add algorithm state information in result packets (#6839)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* add state to backtesting result packets

* self review

* add state in live results

* set runtime error in algorithm

* set hostname in job packets

* Set state when algorithm is null

* address review

* address reviews

* set runtime error when algorithm is not available during runtime
2023-01-11 10:54:38 -03:00
Jhonathan Abreu 6ed8cb5cae Backtest progress calculation (#6847)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix backtest progress calculation

* Backtest progress monitor class

* Minor changes

* Lazy calculating the processed days count

* Minor changes

* Minor changes
2023-01-11 10:33:38 -03:00
Alexandre Catarino 6b6ca5b460 Saves Dividend Payments to Security Holdings (#6830)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Saves Dividend Payments to Security Holdings

Saves information about dividend payments to `SecurityHoldings`. This information will be used to factor in dividend payments to the `Profit` and `NetProfit`.
Adds `SecurityPortfolioManager.TotalNetProfit` to sum all the `SecurityHoldings.NetProfit`.

Adds regression and unit tests.

* Addresses Peer-Review

Improves Regression Test.
2023-01-03 18:37:00 -03:00
Martin-Molinero b899aa4767 Make UserDefinedUniverse thread safe (#6682)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Make UserDefinedUniverse thread safe. Adding regression test
2022-10-12 15:06:01 -03:00
Jhonathan Abreu 548a6998e3 Report generation with non-default algorithm settings (#6589)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add algorithm settings to backtest results

* Use algorithm result settings in report generation

* Add unit tests

* Housekeeping

* Changes for live result report

* Housekeeping

* Housekeeping

* Housekeeping

* Add unit tests
2022-08-30 19:23:25 -03:00
Ryan Russell 62493f8986 docs: algorithm readability improvements (#6554)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
docs: `algorthm` -> `algorithm`
2022-08-17 10:39:57 -07:00
Martin-Molinero 37dca9d458 Remove log packet aggregation (#6538)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Remove log packet aggregation
- Clean up Queue api usage
2022-08-11 19:06:11 -03:00
Martin-Molinero 3f7d339463 Fix live result Handler null reference (#6521)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix live result handler null reference, that could happen if the
  algorithm explodes mid initialization. Adding unit test
2022-08-03 19:57:03 -03:00
Martin-Molinero 3f2bcc1784 Add missing CancellationTokenSource dispose (#6512)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
- Add missing CancellationTokenSource disposals to avoid memory leaks
2022-08-01 11:12:01 -03:00
Martin-Molinero 07aa5625e1 Avoid division by zero error in backtesting result handler (#6484)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Minor improvements

- Avoid division by zero error in backtesting result handler. Race
  condition where the SetAlgorithm implementation would still be running
  and we try to loop through the update method
- Improvement nullreference exception message
- Cleanup resolution to timespan extension method

* Address reviews

- Cleanup data config fill forward conditional statement for tick
  resolution
2022-07-14 12:54:55 -03:00
Martin-Molinero 778d436b94 Removed mapped futures contract is non tradable (#6479)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Minor fixes for mapped futures contract

- Future security was never able to be non tradable
- Skip remove future contracts from live holdings. Adding unit test
  reproducing issue and regression algorithm

* Fix removal of continuous futures

- The canonical continuous future would leave behind an OpenInterest
  subscription when removed because of a different in the way the
  configurations were built, this will now be centralized in a single
  method. Adding unit tests reproducing issue
- FillForwarding setting of the continuous future was not being
  respected. Updating algorithm reproducing the issue to assert the
  behavior failing in master

* Address reviews

- Minor refactors addressing reviews
2022-07-13 15:36:24 -03:00
Martin-Molinero 6200988ad5 Fix result handler live holdings (#6478)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix result handler live holdings

- Live result handler live holdings will include future assets
  associated with continuous futures. Adding unit tests

* Fixed for live holdings price rounding

- Live holdings will use the SPDB minimum price variation for rounding.
  Adding unit tests
2022-07-12 18:31:35 -03:00
Martin-Molinero 0c0ee829a2 Warmup state minor fixes (#6367)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Warmup minor fixes

- Result handler fix
- Centralize and normalize status update during warmup

* Address reviews

* Minor adjustments

* Add console message during warmup period

* Fix warmup percentage update
2022-06-02 18:20:02 -03:00
Martin-Molinero 895dfedf81 User Account currency symbol for fees and capacity (#6359)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* User Account currency symbol for fees and capacity

* Update expected currency symbol stats
2022-05-27 16:36:44 -03:00
Martin-Molinero cb2062debd Expand GetSecondUnevenWait usages (#6340)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Reduce Live CPU usage

* Address selfreview
2022-05-16 20:12:22 -03:00
Martin-Molinero 284b26a9bd Performance improvements and fixes (#6330)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Reuse GUID

* Reduce Lean task usage

* Reduce CPU usage

* Address reviews
2022-05-13 15:04:09 -03:00
Martin-Molinero 813412afb2 Persist and order runtime statistics (#6314)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-04-29 20:59:40 -03:00
Martin-Molinero b8b0d18993 Continuous futures (#6034)
* Continuous Future Contracts

* Mapping approach

* Tweaks WIP

* Live mapping

* Live mapping

- Add support for live mapping, refreshing mapfiles
- Fix future expiration functions
- Adding unit tests

* Update moq test package

* Continuous futures price scaling

* Refactor price factors scaling

* Factor file related renames

* Address reviews
2021-11-15 14:44:30 -03:00
IlshatGaripov 2e4252c92b LiveTradingResultHandler -> changes access modifier of SetNextStatusUpdate (#6015)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-10-25 11:17:35 -03:00
Colton Sellers d2d99b1f10 Algorithm Sampling and Statistics Fixes (#5936)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement scheduled event sampling solution

* Use UTC time, only update daily portfolio value once a day

* For daily resolutions sample chart always

* Cleanup

* Drop resample daily all together

* Force final sample

* Regression updates

* FIx LiveResultHandler to update portfolio and benchmark values outside of sampling event

* Name the daily sampling event

* Address review pt 1

* Drop force and use reference wrapper

* Adjust tests

* Fix warning for Benchmark Timezone Misalignment and also add test

* Fix for daily resolution orders and test adjustments

* Also warn on universe settings with daily resolution

* Update missed regression

* Fix reference wrapper use

* Update regression after rebase

* Add values back in for Daylight Algo

* Have statistics builder skip day 1 performance

* Regression adjustments

* Test adjustments

* Update regression unit test

* Adjust some regressions starts to show performance values

* Add hourly algorithm for beta comparison

* Address missing Python regression changes

* Remove null comment
2021-10-05 19:31:25 -03:00
Martin-Molinero b77f0122b2 Improve python exception parsing (#5831)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing

- Improve python exception parsing adding support for line shift. Adding
  unit tests

* PythonException revert change

* Centralized and normalize algorithm runtime handling

* Adding support for C# line and file exception report
2021-08-06 20:11:54 -03:00
Martin-Molinero 55195867a9 Reduce LTRH error log to debug (#5723)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-06-30 16:44:52 -03:00
Colton Sellers f47775338c Expand Lean Default Charts and Reduce Clutter (#5680)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update charts that are not empty and not default values

* Testing

* Filter final results

* Explicitly filter alpha charts if not needed

* Use filtered charts for Live results handler

* Refactor solution to manage alpha charts internally by creator

* nit comment fixes

* Implement drawdown as default chart

* Implement Capacity Estimate for backtest default chart

* Add default values to QC.Plotting

* Adjust drawdown calculation

* Only sample "Capacity" once a day

* Implement Volume chart

* Nit fixes

* Volume Chart adjustments and fixes

* Rename to "Assets Sales Volume"

* Implement Exposure Sampling

* Store algorithm currency symbol, round capacity to nearest 1k

* Add new plots to default charts

* nit change name

* Address reviews

* Reduce duplication and clean up exposure sampling

* Address reviews

* Improve sample exposure

* Address review

* Only enumerate holdings once

* nit - comments

* Post rebase fix

* Don't need to round anymore
2021-06-22 11:19:48 -03:00
Colton Sellers fa1e97bc60 Round Capacity Estimate for Runtime Statistics (#5688)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Refactor solution to deal with rounding in CapacityEstimate

* Use DiscretelyRoundBy + adjust regressions
2021-06-21 21:06:09 -03:00
Stefano Raggi 080511fa5a Add IResultHandler.BrokerageMessage (#5681)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add IResultHandler.BrokerageMessage

- Result handlers can receive notifications for brokerage message events

* Make LiveTradingResultHandler.RuntimeError virtual
2021-06-21 12:23:46 -03:00
Martin-Molinero 0991f089d0 Reduce disk usage in the cloud (#5673)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-06-16 18:43:55 -03:00
Martin-Molinero 7e2678d0f6 Normalize live holdings initialization (#5647)
Build & Test Lean / build (push) Has been cancelled
* Normalize live holdings initialization

* PaperBrokerage config based live state recovery

- Increase algorithm status storing sampling period
- Fix bug where duplicate cash entries could happen
2021-06-09 20:11:38 -03:00
Martin-Molinero e8cf002238 Live trading holding improvements (#5644)
Build & Test Lean / build (push) Has been cancelled
- Store holdings at end of algorithm
- Warmup holding market price if 0
2021-06-08 21:55:17 -03:00
Martin-Molinero 978da37cda Minor tweaks (#5580)
Build & Test Lean / build (push) Has been cancelled
* Minor tweaks

- Add AccountCurrency and symbol for live result packet
- Improve performance of Tick clone and reading

* Reduce unit test from 9s to 500ms
2021-05-20 12:00:42 -03:00
Alexandre Catarino 58793acae8 Removes Assets that Are Not Invested and Not Tradable From Capacity Calculations (#5515)
Build & Test Lean / build (push) Has been cancelled
* Add property for capacity. Remove unused variable

* Move SymbolCapacity and CapacityEstimate to common, passed through Symbol to runtime statistics

* Add null checks

* Remove uninvested and untradable assets from capacty calculations

* Add SymbolCapacity influential period

* Updates Regression Tests

- DelistingEventsAlgorithm
  - Allows additional contributions from delisted AAA.1

- DelistingFutureOptionRegressionAlgorithm
  - Removes DC01H12 contributions one month later

- FutureOptionBuySellCallIntradayRegressionAlgorithm
  - Allows additional contributions from future after expiry replacing the contribution of the next contract option

- DelistedFutureLiquidateRegressionAlgorithm
- FutureOptionCallITMExpiryRegressionAlgorithm
- FutureOptionCallITMGreeksExpiryRegressionAlgorithm
- FutureOptionPutITMExpiryRegressionAlgorithm
- FutureOptionShortCallITMExpiryRegressionAlgorithm
- FutureOptionShortPutITMExpiryRegressionAlgorithm
- FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm
  - Allows additional contributions from future after expiry

- FutureOptionCallOTMExpiryRegressionAlgorithm
- FutureOptionPutOTMExpiryRegressionAlgorithm
- FutureOptionShortPutOTMExpiryRegressionAlgorithm
- IndexOptionCallITMGreeksExpiryRegressionAlgorithm
- IndexOptionCallOTMExpiryRegressionAlgorithm
- IndexOptionShortCallOTMExpiryRegressionAlgorithm
  - Allows additional contributions from option after expiry

- MACDTrendAlgorithm
  - Removes contribution when SPY is not invested for over one month

- UniverseSelectionRegressionAlgorithm
  - Allows additional contributions from delisted GOOAV replacing GOOG (new symbols)

* Adds Lowest Capacity Asset to Regression Tests

* Normalize expected value -0, because -0 is also written to file if updated

* Write Symbol.Value for lowestCapacitySymbol or  empty string for empty Symbol

* Update Regressions

* Update 'Lowest Capacity Asset' to Symbol.ID

Co-authored-by: Jared Broad <jaredbroad@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
Co-authored-by: Colton Sellers <Colton.R.Sellers@gmail.com>
2021-05-13 20:25:19 -03:00
Colton Sellers e2a0873b7c Fix Lean Warnings V1 (#5408)
Cleanup all non-breaking warnings
2021-03-22 11:08:48 -07:00
Gerardo Salazar f9dc38efab Use trades per period instead of days for capacity calculation (#5389)
Use trades per period instead of days for capacity calculation
Updates regression algorithms
2021-03-16 16:30:49 -07:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
2021-03-02 18:46:31 -03:00
Martin-Molinero e91af0814f Add notification targets and events to live packet (#5210)
* Add notification targets and events to live packet

- Add notification targets and events to live packet. Adding unit tests

* Add project name to job packet

* Add ShortToString and WebNotification Header

- Adding ShortToString implemnetation of OrderEvent and Insight
- Add Web notification headers

* Add more unit tests

* Revert json lower case fields
2021-01-27 10:26:35 -03:00
Martin-Molinero e6d16882f7 Remove raw Sleeps for Sleeps+Event (#5153)
- Remove raw Thread.Sleep for Event driven wait/sleeps. Improves startup
  performance significantly
2021-01-12 19:12:07 -03:00
Martin-Molinero 2de9c21b6d Optimization tweaks (#5056)
* Add progress for runtimeStats

- Add optimization progress runtimestats
- Add ServerStatistics for the last backtesting package

* Add StaticOptimizationParameter

- Add StaticOptimizationParameter. Updating unit tests
2020-12-18 16:01:44 -08:00
Martin-Molinero 6d824b40a6 Add handling for custom host name (#5043)
* Add handling for custom host name

* Credit cost as decimal
2020-12-16 21:54:00 -03:00