40 Commits

Author SHA1 Message Date
Jhonathan Abreu 1974884c13 Security re-addition tidy up (#8656)
* Initialize securities on re-addition

* Minor unit tests fixes

* Reset Index.ManualSetIsTradable when security is removed

* More changes to clean up and simplify

* Address security re-utilization only

* Cleanup regression algorithms
2025-04-04 16:28:15 -04:00
Ricardo Andrés Marino Rojas 62373ede5b Duplicate Futures.Energies (#8082)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Duplicate Futures.Energies

* Nit change

* Add unit test
2024-06-06 19:39:05 -03:00
Martin-Molinero 7975a44d1c Revert "Duplicate "Energies" Futures Ticker Shortcut to Energy (#8067)" (#8081)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
This reverts commit 23057f24c8.
2024-06-06 15:02:03 -03:00
Ricardo Andrés Marino Rojas 23057f24c8 Duplicate "Energies" Futures Ticker Shortcut to Energy (#8067)
* First draft of the solution

* Add backwards compatibility

* Address requested changes
2024-06-04 11:52:58 -03:00
Martin-Molinero d40ec77379 Refactor to use camel case (#7981)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Camel case results

- Camel case lean results and streamed packets

* Update packets to camel case
2024-05-08 12:58:33 -03:00
Jhonathan Abreu d4ba8ccead Backtest self-categorization (#7683)
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Implement backtest self-naming and self-tagging

* Minor unit test fix

* Minor api changes

* Minor changes

* Send packet messages when algorithm's name or tags are updated

* Minor fixes

* Address peer review

* Backtest name and tags API calls

Plus, first version of CI for API tests

* Add endpoint to update a backtest set of tags

* Minor GH actions change

* Address peer review

* Add algorithm naming and tagging unit tests

* Minor backtest API changes
2024-01-30 14:59:08 -03:00
Roman Yavnikov 50ccec9a01 feat: gdax -> coinbase (#7633)
* feat: gdax -> coinbase

* feat: defaultBrokerageModel gdax -> coinbase

* refactor: coinbaseBrokerageModel
fix: gdax test

* feat: add coinbase market hours

* feat: symbolPropDB gdax -> coinbase

* fix: gdax to coinbase tests

* fix: coinbase Fees data

* remove: gdax market-hours

* revert: stopMarket order Type by date + test

* fix: coinbase test (fee was changed)
revert: old test with stopMarket gdax
fix: pandas padding

* refactor: gdaxLiveTrading -> CoinbaseLiveTrading
remove: gdax configs

* remove: old gdax testdata files

* refactor: gdax route -> coinbase route
feat: add deprecated gdax info
feat: some string "gdax" -> "coinbase"

* fix: style brackets

* fix: config coinbase data-queue-handler

* revert: public StablePairsGDAX collection

* rename: GdaxBrokerageModel to CoinbaseBrokerageModel in tests
rename: gdax -> coinbase in DefaultMarketMap
rename: gdaxBrokerageModelTest to Coinbase...

* revert: coinbase changes in toolbox (deprecated)

* remove: override Clone() in GDAX\Coinbase-OrderProperties

* typo: ops missing brackets

* rename: Market.Gdax -> Coinbase in Test
remove: ref on gdax

* test: compatibility of GDAXOrderProperties

* fix: coinbase brokerageName enum number

* feat: readonly to stablePairs Gdax/Coinbase
remove: GetFeeModel() in GDAXBrokerageModel {}

* test: gdax market name compatible with coinbase

* rename: config to old ones

* feat: coinbase CanUpdateOrder()

* rename: plus to commit 4135c91

* feat: coinbase fee model + test

* fix: stable fees data

* update: coinbase symbol-properties-database.csv

* fix:  validation of MinimumOrderSize in SubmitOrder()

* feat: gdax -> coinbase

* feat: defaultBrokerageModel gdax -> coinbase

* refactor: coinbaseBrokerageModel
fix: gdax test

* feat: add coinbase market hours

* feat: symbolPropDB gdax -> coinbase

* fix: gdax to coinbase tests

* fix: coinbase Fees data

* remove: gdax market-hours

* revert: stopMarket order Type by date + test

* fix: coinbase test (fee was changed)
revert: old test with stopMarket gdax
fix: pandas padding

* refactor: gdaxLiveTrading -> CoinbaseLiveTrading
remove: gdax configs

* remove: old gdax testdata files

* refactor: gdax route -> coinbase route
feat: add deprecated gdax info
feat: some string "gdax" -> "coinbase"

* fix: style brackets

* fix: config coinbase data-queue-handler

* revert: public StablePairsGDAX collection

* rename: GdaxBrokerageModel to CoinbaseBrokerageModel in tests
rename: gdax -> coinbase in DefaultMarketMap
rename: gdaxBrokerageModelTest to Coinbase...

* revert: coinbase changes in toolbox (deprecated)

* remove: override Clone() in GDAX\Coinbase-OrderProperties

* typo: ops missing brackets

* rename: Market.Gdax -> Coinbase in Test
remove: ref on gdax

* test: compatibility of GDAXOrderProperties

* fix: coinbase brokerageName enum number

* feat: readonly to stablePairs Gdax/Coinbase
remove: GetFeeModel() in GDAXBrokerageModel {}

* test: gdax market name compatible with coinbase

* rename: config to old ones

* feat: coinbase CanUpdateOrder()

* rename: plus to commit 4135c91

* feat: coinbase fee model + test

* fix: stable fees data

* update: coinbase symbol-properties-database.csv

* fix:  validation of MinimumOrderSize in SubmitOrder()

* fix: skipped gdax name to coinbase

* feature: visible Symbol prop in DefaultOrderBook
2024-01-05 16:57:19 -03:00
Martin-Molinero fd2a8f84b9 Add support for null chart points (#7622)
* Add support for null chart points

- Add support for null value chart points, updating existing and adding
  new unit tests

* Minor selfreview tweaks

* Address review & fix deserialization bug

* Ignore null values when using NullResultValueTypeJsonConverter

* Narrow NullResultValueTypeJsonConverter conversion support

* Allow inherit of series sampler
2023-12-12 20:26:58 -03:00
Martin-Molinero 4b06ca9147 Out of sample support tweaks (#7542)
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Add backtest job end date handling

* Fixes & rebases

* Refactor out of sample configuration handling

* Minor datetime format tweak
2023-10-27 18:23:21 -03:00
Martin-Molinero 489fc5bd5f Fix chart api fetching and deserialization (#7467) 2023-09-15 20:04:46 -03:00
Martin-Molinero 8f1dc76c7f Fix optimization candlestick handling (#7430)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix optimization candlestick handling

- Fixes for optimization candlestick handling
- Centralize deserialization of ChartPoint into new
  ChartPointJsonConverter

* Minor fixes

- Update failing tests
- Update CI to fail if anything fails
2023-08-10 13:34:19 -03:00
Jhonathan Abreu 6109ac8f1b Candlestick charts (#7425)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Candlestick charts base implementation

* Series and Candlestick series json serialization

* Some cleanup

* Add AddPlot method for candlestick series to QCAlgorithm

* Remove Values property from ISeriesPoint

* Add candlestick QCAlgorithm.Plot trade bar methods

* Implement candlestick series re-sampling

* Add more SeriesSampler unit tests

* Add examples of candlestick charts usage to exisiting charting algorithm

* Address peer review

* Address peer review

* Derive Candlestick from Bar

* Sampler changes

* Add new series types from the cloud

* Add more candlestick series sampler tests

* Minor cleanup

* Minor changes
2023-08-08 17:52:46 -03:00
Ricardo Andrés Marino Rojas aa1ebd0e65 Bug parameters in report bug (#7413)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Solve bug in the parameters of the Report

Before, when the number of keys were odd a part of the raw template was
shown in the report generated. Additionally, the user couldn't define
the number of columns of the parameters table

* Add unit tests

* Enhance unit tests

* Nit change
2023-08-02 15:28:48 -03:00
Ricardo Andrés Marino Rojas 3338486b52 Enable Report generator to use custom HTML template (#7410)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add support for custom HTML file in Report

* Add support for custom crisis HTML template

* Revert "Add support for custom crisis HTML template"

After considering this option carefully, the QC team has decided to
add this functionality in a different way.

This reverts commit aead1ff8504556417e9dffadb33b06c7599ab87a.

* Get parameters and crisis HTML custom templates

* Move parameters html template to a file

* Nit change

* Remove template.crisis.html and add unit tests

* Nit change

* Nit change

* Add unit tests for the fail cases
2023-08-01 12:55:14 -03:00
Ricardo Andrés Marino Rojas b64169fd20 Fix Beta chart in Report (#7295)
* Fix bug in Beta statistics

- Beta in Report was calculated using equity points and not daily
  performance points, as it's done in Backtest
- Performance points were not being divided by 100, as it's done in
  StatisticsBuilder
- Benchmark points were not being converted into a percentage daily
  performance of the benchmark, as it's done in StatisticsBuilder

* Improve implementation and add unit tests

* Enhance implementation and add more unit tests

* Enhance implementation

* Nit change

* Nit changes
2023-06-07 11:10:10 -03:00
Ricardo Andrés Marino Rojas 20c92b0350 Fix bug: Report Color Map Does Not Include Crypto Futures (#7168)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix ReportCharts color_map bug

- Add ´CryptoFuture´ entry in `color_map` dict using color #E55812
- Make ´ReportCharts.py´ color_map a class attribute
- Change ´ReportChartsTests.cs´ to use color_map dict from ´ReportCharts.py´
- Add unit test that fails every time a new SecurityType is added

* Improve unit test

* Nit change

* Nit change
2023-04-05 17:51:16 -03:00
Jhonathan Abreu 6dfc6c710c Add algorithm parameters to result (#6878)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add algorithm parameters to result's AlgorithmConfiguration

* Peer review
2023-02-09 12:55:30 -03:00
Jhonathan Abreu b54281b262 Combo orders (#6813)
* Feature combo orders

- Add support for combo orders

* Make fill model wait for all grouped orders to emit fills

* Add ComboFill to model multiple fills for combo orders

* Fill combo limit orders

Add some regression algorithms

* Add fill implementation for combo leg limit orders

* Add IFill as common interface for Fill and ComboFill

* Refactor combo orders removing IGroupOrder interface

Move the group order manager to the base Order class

* Update algorithms

* Handle combo order events atomically

* Refactor brokerage transaction event handler

* Refactor combo fill models

* Process fills in batch

* Combo orders fill model tests

* Combo leg limit orders algorithm

* Regression algorithms cleanup

* Fill and combo fill classes cleanup

* Housekeeping

* Refactor equity fill model to derive from base fill model

* Address review changes request

* Handling the new types of orders in the OrderJsonConverter

* Add regression algorithm to test combo orders update/cancel

* Add regression algorithm to test combo orders update/cancel

* Housekeeping

* Address review changes request

* Minor changes

* Security transaction handler method for setting order request id

* Extend public interface for placing combo orders

* Combo order tickets demo algorithm python version

* Tweaks and updates

* Minor fixes

* Minor changes

* Minor fixes

* Address reviews minor fixes

* Minor fixes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2023-01-06 17:58:43 -03:00
Jhonathan Abreu 548a6998e3 Report generation with non-default algorithm settings (#6589)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add algorithm settings to backtest results

* Use algorithm result settings in report generation

* Add unit tests

* Housekeeping

* Changes for live result report

* Housekeeping

* Housekeeping

* Housekeeping

* Add unit tests
2022-08-30 19:23:25 -03:00
Jhonathan Abreu 3c6ae2f1e7 Fix report's exposure plot colors and other bugs (#6580)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Use correct colors in report's exposure plot

* Add 12 month rolling sharpe to report plot

* Housekeeping

* Fix failing unit tests
2022-08-25 12:28:22 -03:00
Jhonathan Abreu b07680688f Fix FormatException on estimated capacity rendering during report generation (#6551)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix FormatException on non-USD currency capacity in report generator

* Improve currency removal from capacity string

Applied this improvement to OptimizationBacktestJsonConverter.WriteJson

* Address changes request
2022-08-16 11:08:36 -03:00
Jhonathan Abreu 771cc11b3a Fix KeyError in report generator due to missing color mappings (#6546)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix KeyError when rendering exposure chart in report with some security types

* Update exposure chart colors and add unit tests

* Updated json test file
2022-08-12 18:52:15 -03:00
Jasper van Merle 12df1c9a31 Fix drawdown plotting failing on single equity point (#5461) 2021-04-06 16:30:23 -07:00
Gerardo Salazar 8c6aa6a3b3 Refactors Capacity Estimation and moves estimation to main event loop (#5351)
* Adds CapacityEstimate and SymbolCapacity

  The capacity estimation has been moved from
  the report generator and wired directly into
  Lean via the ResultHandler. In addition,
  the capacity estimation strategy has changed
  to account for errors in the previous iteration
  of the capacity estimation.

  Many many thanks to Jared for being much of the
  mastermind behind this project. It would have
  been harder to complete without him to bounce ideas
  off of.

  * Moves old tests to regression algorithms
  * Adds Estimated Capacity statistic
  * Removes old capacity estimation tests

Final report capacity estimation. Pushing to save state

* Fixes bugs, cleans up code and adds comments

  * Adds forced sampling to Capacity Estimation
  * Misc. bug fixes for daily data

* Updates capacity test cases' Estimated Strategy Capacity statistic

* Adds Capacity Estimate to all regression algorithms

* Removes Report's StrategyCapacity class and fixes bug in tests

  * Adds null check in BacktestingResultHandler to fix
    BacktestingTransactionHandler failing tests

  * Deletes old capacity estimation classes

  * Retrieve capacity estimates from backtest statistics results
    instead of calculating at runtime

* Make $0.00 capacity return as "-" and Result = 0 in report

* Adds capacity to runtime statistics

* Converts capacity to number denoted by financial figures in RuntimeStats

* Addresses review: code cleanup for Capacity and adds comments to regression tests
2021-03-02 18:46:31 -03:00
Gerardo Salazar 37d26a35ce Fixes various bugs in estimated capacity class (#5324)
* Fixes potential mapping issue, caused by not mapping Symbols before
    reading data off disk

  * Fixes issue where some results would evaluate to zero capacity

  * Loads of refactoring, mostly resulting in cleaner code
2021-02-19 18:00:08 -08:00
Gerardo Salazar 0645513f5e Capacity estimation in report generator (#5318)
* Adds strategy capacity estimation in report generator

* Style and license fixes

* Adds comments, constants, and new method accessibility

* Removes unneeded imports
2021-02-18 17:51:40 -08:00
Gerardo Salazar 4134b05bfd Fixes options order crash in ReportGenerator/PortfolioLooper (#4992) 2020-12-07 12:39:53 -03:00
Gerardo Salazar 40c3062348 Improves Report Generator stability and fixes various bugs (#4759)
* Improves stability and fixes various bugs

  * Adds unit tests covering changes
  * Adds COVID-19 crisis plots
  * Adjusts styling of crisis plots for more pleasant viewing
  * Fixes bug where null BacktestResult caused application to crash
  * Order JSON bug fixes and stability improvements
  * MaxDrawdownReportElement now produces results for Live
  * Replaced Estimated Capacity w/ Days Live
  * Added Live marker to sharpe ratio
  * Added support for MOO and MOC orders in PortfolioLooper

* Address review: adds new unit tests and cleans up code

  * Bug fix: use LastFillTime instead of Order.Time for MOO and MOC

* Address review: Fixes tests and cleans up code
2020-09-29 10:03:26 -03:00
Martin-Molinero f4ed48fce2 Always use AlgorithmHandler.DataProvider (#4658)
- `UniverseSelection` and `Research` will always use
  `AlgorithmHandler.DataProvider` instance instead of
  `DefaultDataProvider`
- Remove Compression at test/app.config since the dll isn't required
- Add missing license header
2020-08-27 16:07:56 -03:00
Martin Molinero bcc67d2457 DataPermissionManager is used by history provider
- DataPermissionManager will be used by the history provider
- Moving interfaces from Engine to Common
2020-06-12 12:51:14 -03:00
Martin Molinero 452bf27bd9 Adding new DataPermissionManager
- Adding new DataPermissionManager that will own the existing
  datachannelProvider. Will assert configurations before added to the
  data feed. Adding unit tests
2020-06-12 12:34:10 -03:00
Martin Molinero 4a99eb11c6 Fix regression tests
- Fix regression tests. Future symbol contains the market which was used
  un the order hash list
2020-04-28 16:29:16 -03:00
Gerardo Salazar 558779949d Address review - trim unecessary code and ignore case on Enum parse 2020-04-14 11:08:40 -07:00
Gerardo Salazar 093ddc848d Fixes issue when parsing LiveResult JSON with string OrderType values 2020-04-10 17:24:20 -07:00
Gerardo Salazar e5ed489e0b Fixes issue where PortfolioLooper would crash generating certain reports
* Updates SharpeRatioReportElement calculation
2020-03-26 11:50:08 -07:00
Martin-Molinero 78fb93a64b Merge branch 'master' into bug-report-generator-add-null-handling-for-valuetype-jsonconverter 2020-03-18 17:06:19 -03:00
Gerardo Salazar c9e6d52b83 Address review - adds unit tests for NullResultValueTypeJsonConverter
* Downgrades Json.NET to 10.0.3 for project compatibility
* Cleans up unused imports and unused code
2020-03-12 15:34:40 -07:00
Gerardo Salazar 77411a0b09 Fixes issue where setting leverage on cash accounts results in error
* Adds new unit tests
2020-03-12 10:42:30 -07:00
Gerardo Salazar 5d663e8944 Fixes various issues
* Fixes issue with how cumulative percent change was calculated
* Reworks PercentChange extension method
* Fixes PortfolioLooper leverage calculation
* Disable page 2 of reports
* Fix issue where missing values produced misaligned shape in
AnnualReturnsReportElement.cs
2019-12-31 17:37:19 -08:00
Gerardo Salazar 7cc4dec6b7 Reworks structure of project
We now will calculate all of the metrics needed to directly plot the
data in C# instead of doing calculations in Python.

This has added benefits in that the code can be easily reused to generate
plots from Result packets and makes the underlying plotting library we
use more extensible and replaceable.

* Adds Calculations.cs
* Refactors some of ReportCharts.py
* Adds Drawdown classes
* Adds Deedle and MathNet.Numerics packages
* Completes calcuations in C# side instead of Python

Side note: I think it's a little funny that we were previously
calculating the plots via Pandas, which uses a C backend. Effectively, we
were transferring data between at least 2 separate FFI
boundaries: C# -> Py -> C -> ? (whatever BLAS is written in)
2019-12-18 16:08:40 -08:00