298 Commits

Author SHA1 Message Date
JosueNina eb12c8fa65 Seed runtime-added currency conversion rates immediately (#9568)
* Seed runtime-added currency conversion rates immediately

Fixes the spurious 'The conversion rate for <currency> is not available'
runtime error caused by a two-path seeding asymmetry.

The setup path (BaseSetupHandler.SetupCurrencyConversions) wires up a
currency's conversion feed AND seeds its rate via history/last-known-price
so the rate is non-zero right away. The runtime path
(UniverseSelection.EnsureCurrencyDataFeeds, invoked during universe
selection / SetCash mid-run) only created the conversion subscription and
left the rate at 0 until the first bar of the pair arrived. Any conversion
in that gap (classically a midnight scheduled SetHoldings firing before the
day's first conversion-pair bar) threw.

EnsureCurrencyDataFeeds now seeds newly introduced, still-zero-rate
conversion securities and calls cash.Update(), mirroring the setup path.
Seeding is gated behind a seedNewCurrencies flag (default true) so the
setup caller, which performs its own optionally white-listed seeding, can
opt out and not regress white-list semantics. SeedSecurities degrades
gracefully when no history/data is available, leaving the rate at 0 as
before, so live mode and no-history scenarios are safe.

Adds a regression test exercising the runtime path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Make runtime currency seeding robust and fix regression expectation

CI failures from the runtime currency-conversion seeding change:

1. AlgorithmWarmupTests.WarmUpInternalSubscriptions threw
   ArgumentNullException because the new EnsureCurrencyDataFeeds seeding
   path ran GetLastKnownPrices in a stub where the conversion security
   lacked SymbolProperties. Pre-seeding is best-effort and must never
   break the algorithm, so wrap it in try/catch and degrade gracefully
   (leave the rate at 0, the pre-fix behavior) - matching the documented
   intent. The first conversion-pair bar still updates the rate.

2. ScheduledUniverseSelectionModelRegressionAlgorithm (C# + Python)
   asserted AlgorithmHistoryDataPoints == 0. The algorithm runtime-adds
   Forex pairs (EURGBP -> GBP cash) via scheduled universe selection;
   the fix now correctly seeds that runtime currency's conversion rate
   with a last-known-price history request (deterministically 50 points).
   The old 0 reflected the buggy unseeded behavior, so update the
   expectation to 50. No other statistics changed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Seed runtime added currency conversion rates

* Seed currencies with no new conversion feed and dedup the seeding helper

---------

Co-authored-by: Martin-Molinero <Martin-Molinero@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-01 12:40:54 -03:00
Martin-Molinero 046df38be8 Fix log timestamp & minor cleanup (#9124) 2025-12-11 13:44:30 -03:00
Jhonathan Abreu c81f5d7d1a Seed securities by default (#9045)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add SeedInitialPrices algorithm setting

This is true by default and indicates that the engine will seed initial prices right after the security is added or selected

* Update regression algorithms

* Update regression algorithms

* Update regression algorithms

* Refactor default securities seeding

* Minor fix

* Minro fixes

* Cleanup

* Updated and add regression algorithms

* Address peer review

* Centralize logic to get last known data for multiple securities

* Some cleanup

* Minor build fix

* Minor fixes

* More logic centralization

* Some more cleanup

* Cleanup

* Update regression algorithms and minor fixes

* Update regression algorithms

* Minor fix

* More minor fixes

* Update regression algorithms

* Cleanup

* Minor test fix

* Address peer review

* Minor fix and performance improvement

* Fix to seed open interest data

* Minor test fixes

* Address peer review

* Minor change

* Minor revert

* Minor fixes and improvements

* Disable initial seeding by default

* Minor fixes

* Cleanup

* Cleanup

* Minor fix
2025-11-18 13:05:56 -04:00
JosueNina 7a5ff8054b Prevent adding zero quantity currencies to CashBook (#9076)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
* Prevent adding zero quantity currencies to CashBook

* Solve review comments

* Solve new review comments

* Add unit test
2025-11-10 09:33:38 -03:00
JosueNina f5c5c5a3b0 Add configurable initialization timeout via config.json (#9056)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add configurable initialization timeout via config.json

* Solve review comments
2025-10-22 12:49:39 -03:00
Martin-Molinero d8d6deb675 Add support for Brokerage ActionRequired message type (#9040)
Python Virtual Environments / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Add support for Brokerage ActionRequired message type

* Minor syntax check typo fix
2025-10-15 13:05:14 -03:00
Martin-Molinero cffd20236c Object store dispose hang (#8925)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2025-08-13 18:18:48 -03:00
Martin-Molinero cd7663ef97 Minor tweaks for testing (#8679)
* Minor tweaks for backtest tests

* More tweaks
2025-04-07 18:41:44 -03:00
Jhonathan Abreu 69d2f5ae82 Futures and Future Options file-based universes (#8480)
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Make FOPs selection universe file-based for backtesting

* Make FOPs option chains universe file based

* Make Future universe selection file-based like option universe

* Make Future universe selection file-based like option universe

* Abstraction cleanup

* Add FuturesChains API to QC algorithm

Also refactor future chain provider to use the new FutureUniverse instead of zip file names

* Update regression algorithms stats

* Refactor QuantBook option and future history to use new universes

* Fix failing tests

* Fix failing tests

* Fix failing tests

* Minor future chains unit test improvement

* Add futures chains DataFrame property

Also, remove IDerivativeSecurity interface from Future

* Add DataFrame property to FuturesChains class

* Add regression algorithms

* Add regression algorithms

* Replace QCAlgorithm.FutureChainProvider usages with new FuturesChain api

* Minor fixes

* Reduce number of universe files in repo

* Minor data fixes

* Regression algorithms updates

* Add implicit conversion from FuturesContract to Symbol

Modified algorithms to use futures contract objects directly instead of accessing their Symbol property.
Removed unnecessary import statements and redundant lines in various files.

* Improve resolution handling for history requests

* Changed _auxiliaryData field to lazily-initialized AuxiliaryData property

* Refactor data handling in BaseChain and TimeSliceFactory

- Added `AddData` method to `BaseChain` for adding market data
- Refactored `TimeSliceFactory` to use `BaseChain.AddData` method

* Remove specific constructors and indexers from Chain classes

Removed public indexers in `BaseChains` for getting or setting `BaseChain` instances by `ticker` or `Symbol`, which were used for Pythonnet compatibility.

* Remove chain cache logic from FuturesChainUniverse

* Refactor class and interface names for clarity

Renamed `FileBasedUniverse` to `BaseChainUniverseData` and
`IFileBasedUniverse` to `IChainUniverseData`.

* Add base class for options and futures contracts

- Introduced `BaseContract` as an abstract base class for contracts, consolidating common properties and methods.
- Removed ISymbolInterface

* Add minor fix for future options tickers parsing

Added tests

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Clean chain provider classes up

* Remove ZipEntryName other classes and unused code

Removed ZipEntryName class and references across various files.
Removed DataQueueFuturesChainUniverseDataCollectionEnumerator and DataQueueOptionChainUniverseDataCollectionEnumerator classes.
Removed OptionChainUniverseSubscriptionEnumeratorFactory class.
Removed unused code for handling OptionChainUniverse and FuturesChainUniverse in FileSystemDataFeed.cs and LiveTradingDataFeed.cs.
Removed several test files related to enumerator factories and universe data collection.

* Minor changes and cleanup

* Trigger Build

* Trigger Build

* Refactor FuturesContract data handling

Forward price data from bars and ticks stored in private fields for improved memory usage

* Fix: use universe data for market data in FuturesContract

* Update regression algorithms stats after rebase

Added HSI futures universe files

* Sort configs by internal flag

Internals go first

* Throw from option universe data filters for future options

Future options IV, Open interest and greeks are not supported for future options

* Minor changes

* Improve some regression algorithms

* Minor fix for failing unit tests

* Update FOPs universe file header

Removed greeks and IV columns.
Updated FOPs universe files: removed outdated columns.

* Minor unit test fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Add history provider as constructor argument for chain providers

* Update new regression algorithms data points count

* Minor fix for FakeDataQueue

* Add initialize method to chain providers classes

* Minor changes

* Trigger Build

* Trigger Build

* Trigger Build

* Minor fix

* Minor fix

* Trigger Build

* Trigger Build

* Trigger Build

* Trigger Build

* Add logs to ProcessedDataProvider

* Removed test logs

* Minor fix

* Support downloading options and futures universe files from api data provider
2025-03-25 16:22:38 -04:00
Martin-Molinero 71d09a3567 Private cloud minor improvements (#8409)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Private cloud tweaks

* Minor tweaks for optimization
2024-11-14 17:22:33 -03:00
Martin-Molinero 7879795207 Enable daily precise end time by default (#8254)
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Default daily precise end times

- Enable by default daily precise end times. Updating stats
- Minor fix for algorithm manager consolidator updates, adding new regression test
  asserting behavior and updating others
- Minor fix for SubscriptionData creator avoid round down on warmup if
  not appropiate
- Adjust consolidators to emit on daily strict end times if requested
  daily resolution and setting enabled
- Updating regression algorithms

* Skip daily data on extended market hours

* Some cleanup and self review

* Revert unrequired change
2024-08-14 12:49:56 -03:00
Ricardo Andrés Marino Rojas 99a949586c Fix warnings part 12 (#8182)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* First part of the warnings

* Second half of the changes
2024-07-09 11:08:42 -03:00
Jhonathan Abreu 6392b5eed0 Minor changes in BaseSetupHandler. (#8112)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Making it able to update currency conversions for specific currencies.
2024-06-26 18:06:26 -03:00
Martin-Molinero d4e7f584f6 Daily data time adjustment (#8001)
* Daily data Time & EndTime Improvement

- Adjust daily data Time & EndTime to actually reflect the time of the
  data used, for example US Equity from 9.30 to 4PM. Adding new unit and
  regression tests

* Refactor solution to use enumerator

- Refactor daily strict end times solution to be through enumerator
  usage, so it applies for history providers too

* Minor fixes

* Revert fill forward enumerator change

- Revert FillForward enumerator causing stats changing, enhancing unit
  tests

* Some cleanup

* Improve handling of live trading FF enumerator

- Improve handling of live trading FF enumerator, by adding support for
  bars to arrive with a delay so we can handle auction close/option
  prices or data providers which might have some delay making the data
  available. Adding new unit tests asserting the behavior
2024-05-20 17:28:03 -03:00
Martin-Molinero 6915a65900 Further camel case cleanup (#8003) 2024-05-08 17:28:47 -03:00
Martin-Molinero 67b9ecf391 Minor config tweaks (#7792)
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-02-19 12:05:00 -03:00
Jhonathan Abreu d4ba8ccead Backtest self-categorization (#7683)
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Implement backtest self-naming and self-tagging

* Minor unit test fix

* Minor api changes

* Minor changes

* Send packet messages when algorithm's name or tags are updated

* Minor fixes

* Address peer review

* Backtest name and tags API calls

Plus, first version of CI for API tests

* Add endpoint to update a backtest set of tags

* Minor GH actions change

* Address peer review

* Add algorithm naming and tagging unit tests

* Minor backtest API changes
2024-01-30 14:59:08 -03:00
Jhonathan Abreu 3dab2e42eb Handle/filter new brokerage-side orders (#7706)
* Handle/filter brokerage side orders

* Add unit tests

* Minor fix

* Add unrequested security for new brokerage-side orders

* Add unit test for algorithms overriding the brokerage message handler

* Add unit test for python algorithm overriding the brokerage message handler

* Minor change

* Address peer review
2024-01-23 12:01:03 -04:00
Martin-Molinero fa329a9d51 Minor brokerage setup handler message improvement (#7700)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
2024-01-17 17:55:56 -03:00
Roman Yavnikov 1eaf4d6ae0 Flexible trading days per year property (#7690)
* feat: new `TradingDaysPerYear` prop in IAlgorithmSettings

* feat: getTradingDayPerYear by BrokerageModel
feat: init algo tradingDaysPerYear in brokerage|BacktestingSetupHandler
refactor: nullable `TradingDayPerYear`

* remove: hardcoded value:252 in statistics

* test: `algorithm.Settings.TradingDaysPerYear` for different brokerage
fix: tests calculation of `AnnualPerformance()`

* fix: Report generator by tradingDayPerYear

* fix: hardcoded value in PortfolioStatisticsTests

* fix: tradingDayPerYear in AnnualPerformanceTest
remove: extra spacing

* feat: backwards compatibility for TradingDaysPerYear

* feat: several UTest for AlgoSetting's TradingDaysPerYear prop

* fix: existed algo + test

* feat: regression algo with checking of tradingDayPerYear in OnEndOfAlgorithm()

* refactor: GetBrokerageTradingDayPerYear -> SetBrokerage... in BaseSetupHandler
refactor: carry out tradingDayPerYear at the top level and pass variable

* remove: default valuine in test param
remove: extra comment.

* fix: missed init TradingDaysPerYear in report.portfolioAlgo
fix: hardcoded value to tradingDaysPerYear

* fix: nullable TradingDaysPerYear in AlgorithmConfiguration
feat: UTest for PortfolioLooperAlgo
feat: add advanced description

* feat: test PortfolioStatistics with different tradingDaysPerYear

* revert: PortfolioLooper with TradingDaysPerYear implementation

* revert: nullable TradingDaysPerYear in AlgoConfiguration
remove: SetBrokerageTradingDayPerYear in BaseResultHandler

* style: remove empty row
2024-01-17 14:45:57 -03:00
Ricardo Andrés Marino Rojas eeb44d841a Add partial support for live algorithm redeployment with custom data (#7655)
* First Draft of the solution

Handle the case where the security has already been added in the
algorithm

* Improve implementation

* Improve changes

* Add unit test

* Improve unit tests

* Fix bugs in unit tests

* Nit changes

* Nit changes

* Improve implementation

* Solve bugs in unit tests

* Add more unit tests
2024-01-04 11:13:52 -03:00
Jhonathan Abreu 8cdb2d4c10 Add options and future chain providers to composer (#7644) 2023-12-27 14:42:21 -04:00
Jhonathan Abreu 314a334e88 Price adjustment mode Order property (#7582)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add new Order.PriceAdjustmentMode property

* Minor fix and unit test

* Minor fix and regression algorithms' stats update

* Unit test fixes

* Minor fix

* Set order price adjustment mode to raw always for live trading
2023-11-17 16:09:03 -04:00
Martin-Molinero 4b06ca9147 Out of sample support tweaks (#7542)
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Add backtest job end date handling

* Fixes & rebases

* Refactor out of sample configuration handling

* Minor datetime format tweak
2023-10-27 18:23:21 -03:00
Jhonathan Abreu fb3409d8a2 Custom data sourced from the object store (#7493)
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Support sourcing custom data from ObjectStore

* Add Python regression algorithm

* Wire engine to pass ObjectStore down to the stream readers

* Minor changes

* Minor unit tests fixes

* Remove unused SetupHandlerParameters.ObjectStore

* Minor changes

* Minor changes

* Support single-file zipped data to be sourced from object store

* Assert object store custom data history requests in regression algorithms

* Add custom object store data live data feed unit test

* Add multi-file object store custom data regression algorithms

* Minor fix
2023-10-02 15:54:57 -03:00
Alexandre Catarino 7e27ab74bb Deprecates IAlgorithmSettings.DataSubscriptionLimit (#7457)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Deprecates IAlgorithmSettings.DataSubscriptionLimit

We will let the brokerage check and notify users that they have subscribed to datafeeds beyond their quota.

* Removes Unit Test

Also remove usage in `EmptyEquityAndOptions400Benchmark`.
2023-09-08 19:41:52 -03:00
Jhonathan Abreu af0de8d2e9 Deployment target and running mode algorithm properties (#7337)
* Add algorithm's mode and deployment target properties source from job packet

* Address peer review

* Add Python regression algorithm
2023-07-03 09:55:21 -04:00
Martin-Molinero 410956bf9f FreePortfolioValuePercentage Trailing Behavior (#7272)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Implement Trailing FreePortfolioValue

- Implement Trailing FreePortfolioValue by default, users will be able
  to set it to a fixed number if desired. Adding regression algorithm
- Setting the default 'MinimumOrderMarginPortfolioPercentage' from 0 to
  0.1% of the TPV to avoud tiny trades by default

* Update existing regression algorithms

* Address reviews

- Send warning message to the user if a trade does not happen due to the
  default setting of the minimum order margin percentage value

* Address reivews

* Rename TotalPortfolioValueLessFreeBuffer

* Update new regression algorithm
2023-05-25 18:48:04 -03:00
Martin-Molinero 08e1f31271 Minor live logging improvements (#7240) 2023-05-12 18:13:37 -03:00
Martin-Molinero 884bfe4482 Add support for brokerage side new orders (#7228)
* Add support for brokerage side new orders

- Add support for brokerage side new order events for liquidation cases

* Minor cash delta fix

* Improve account cash logging

* Fix null reference exception for open orders
2023-05-02 16:21:19 -03:00
Martin-Molinero 227ade63ed Refactor automatic option assignment (#7165)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Refactor automatic option assignment to allow overriding behavior.
  Adding & updating existing tests
2023-04-04 17:40:31 -03:00
Martin-Molinero 8235de42a0 Minor debugging improvements (#6954)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Minor debugging improvements

- Improve exception message if user provides empty/null ticker
- Log margin information on algorithn initialization
- Add unit test asserting option strategy position groups resolve
  correctly for existing holdings

* Symbol Lazy SecurityDefinitionSymbolResolver

- Symbol will have a lazy SecurityDefinitionSymbolResolver instance
  resolution
2023-02-13 10:49:10 -03:00
Martin-Molinero ee683933a3 Feature Perpetual Crypto Futures (#6807)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* WIP

* Add base currency cash

* Symbol properties and data processing

* Add basic template algorithm

* Add hourly crypto future algorithm

* Minor fixes after live trading testing

* CoinApiDataQueueHandler CryptoFuture support

* Address reviews

* Fix regression algorithms after update
2022-12-30 13:58:11 -03:00
Martin-Molinero 7095dc1559 Update gateway to 1019 (#6706)
- Update gateway to latest 1019
- Fix bug in IB data subscription limit, not applying when using
  multiple data queue handlers
2022-10-24 12:01:13 -03:00
Martin-Molinero 108bfa6c0a Enable debugging worker threads (#6647)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Enable debugging worker threads in python
2022-09-21 16:29:37 -03:00
Martin-Molinero 3eb2b94070 Minor logging improvements (#6588)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add virtual env name in init log
- Log account type with cashbook
- Consolidate job handlers log. Adding DataProvider
2022-08-26 14:18:28 -03:00
Jhonathan Abreu 539011274c Support extended market hours for futures (#6522)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Support extended market hours in AddFuture()

* Support extended market hours in AddFutureContract()

* Add C# regression algorithm

* Add Python regression algorithm

* Add regression algorithm for future contracts

* Add regression algorithm checking market hour ranges

* Fixed future regression algorithms to use extended market hours

* Fixed future regression algorithms to use extended market hours

* Fixed future regression algorithms to use extended market hours

* Fixed AddFutureOptionContractFromFutureChainRegressionAlgorithm to use extended market hours

* Update future market hours to include extended in market hours database

* Fixed AddFutureOptionContractDataStreamingRegressionAlgorithm to use extended market hours

* Fixed AddFutureOptionContractFromFutureChainRegressionAlgorithm to use extended market hours

* Fixed AddFutureContractWithContinuousRegressionAlgorithm to use extended market hours

* Fixed BasicTemplateContinuousFutureAlgorithm to use extended market hours

* Fixed BasicTemplateFuturesAlgorithm to use extended market hours

* Fix BasicTemplateFuturesDailyAlgorithm to use extended market hours

* Fixed BasicTemplateFuturesFrameworkAlgorithm to use extended market hours

* Fixed BasicTemplateFuturesHistoryAlgorithm to use extended market hours

* Fixed ContinuousBackMonthRawFutureRegressionAlgorithm to use extended market hours

* Fixed ContinuousFutureBackMonthRegressionAlgorithm to use extended market hours

* Fixed ContinuousFutureHistoryRegressionAlgorithm to use extended market hours

* Fixed ContinuousFutureLimitIfTouchedOrderRegressionAlgorithm to use extended market hours

* Fixed ContinuousFutureRegressionAlgorithm to use extended market hours

* Fixed DelistedFutureLiquidateRegressionAlgorithm to use extended market hours

* Fixed AutomaticIndicatorWarmupDataTypeRegressionAlgorithm to use extended market hours

* Fixed ConsolidateRegressionAlgorithm to use extended market hours

* Fixed DelistingFutureOptionRegressionAlgorithm to use extended market hours

* Fixed EqualWeightingPortfolioConstructionModelFutureRegressionAlgorithm to use extended market hours

* Fixed FutureContractsExtendedMarketHoursRegressionAlgorithm to use extended market hours

* Fixed FutureMarketOpenAndCloseRegressionAlgorithm to use extended market hours

* Fixed FutureMarketOpenConsolidatorRegressionAlgorithm to use extended market hours

* Fixed FutureOptionBuySellCallIntradayRegressionAlgorithm to use extended market hours

* Fixed FutureOptionCallITMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionCallITMGreeksExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionCallOTMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionDailyRegressionAlgorithm to use extended market hours

* Fixed FutureOptionHourlyRegressionAlgorithm to use extended market hours

* Fixed FutureOptionMultipleContractsInDifferentContractMonthsWithSameUnderlyingFutureRegressionAlgorithm to use extended market hours

* Fixed FutureOptionPutITMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionPutOTMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionShortCallITMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionShortCallOTMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionShortPutITMExpiryRegressionAlgorithm to use extended market hours

* Fixed FutureOptionShortPutOTMExpiryRegressionAlgorithm to use extended market hours

* Fixed FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm to use extended market hours

* Fixed FuturesExpiredContractRegression to use extended market hours

* Fixed FutureSharingTickerRegressionAlgorithm to use extended market hours

* Fixed HistoryWithDifferentContinuousContractDepthOffsetsRegressionAlgorithm to use extended market hours

* Fixed HistoryWithDifferentDataMappingModeRegressionAlgorithm to use extended market hours

* Fixed HistoryWithDifferentDataNormalizationModeRegressionAlgorithm to use extended market hours

* Fixed LimitOrdersAreFilledAfterHoursForFuturesRegressionAlgorithm to use extended market hours

* Fixed OpenInterestFuturesRegressionAlgorithm to use extended market hours

* Fixed RegisterIndicatorRegressionAlgorithm to use extended market hours

* Fixed SetHoldingsFutureRegressionAlgorithm to use extended market hours

* Fixed WarmupFutureRegressionAlgorithm to use extended market hours

* Fixed AddFutureOptionSingleOptionChainSelectedInUniverseFilterRegressionAlgorithm to use extended market hours

* Fixed AlgorithmHistoryTests to use extended market hours for futures

* Fixed AlgorithmTradingTests to use extended market hours for futures

* Fixed BrokerageSetupHandlerTests to use extended market hours for futures

* Fixed TimeRulesTests to use extended market hours for futures

* Fixed FutureOptionMarginBuyingPowerModelTests to use extended market hours for futures

* Fixed FutureMarginBuyingPowerModelTests to use extended market hours for futures

* Fixed FileSystemDataFeedTests to use extended market hours for futures

* Fixed QuantBookHistoryTests to use extended market hours for futures

* Split BasicTemplateContinuousFutureAlgorithm to have an extended market version

* Fixed FutureMarketOpenAndCloseRegressionAlgorithm to use extended market hours

* Split BasicTemplateFuturesAlgorithm to have an extended market version

* Split BasicTemplateFuturesAlgorithm to have an extended market version

* Split BasicTemplateFuturesFrameworkAlgorithm to have an extended market version

* Split BasicTemplateFuturesHistoryAlgorithm to have an extended market version

* Revert AddFutureContractWithContinuousRegressionAlgorithm

* Revert AddFutureOptionContractDataStreamingRegressionAlgorithm and added data

* Revert AddFutureOptionContractFromFutureChainRegressionAlgorithm

* Revert AddFutureOptionSingleOptionChainSelectedInUniverseFilterRegressionAlgorithm

* Revert ConsolidateRegressionAlgorithm

* Revert Algorithm.CSharp/ContinuousBackMonthRawFutureRegressionAlgorithm.cs

* Revert ContinuousFutureBackMonthRegressionAlgorithm

* Revert ContinuousFutureHistoryRegressionAlgorithm

* Revert ContinuousFutureLimitIfTouchedOrderRegressionAlgorithm

* Revert ContinuousFutureRegressionAlgorithm

* Revert Algorithm.CSharp/DelistedFutureLiquidateRegressionAlgorithm.cs

* Revert EqualWeightingPortfolioConstructionModelFutureRegressionAlgorithm

* Split FutureMarketOpenAndCloseRegressionAlgorithm to have an extended market version

* Split FutureMarketOpenConsolidatorRegressionAlgorithm to have an extended market version

* Revert FutureOptionBuySellCallIntradayRegressionAlgorithm

* Revert FutureOptionCallITMExpiryRegressionAlgorithm

* Revert FutureOptionDailyRegressionAlgorithm

* Revert FutureOptionPutITMExpiryRegressionAlgorithm

* Revert FutureSharingTickerRegressionAlgorithm

* Revert FuturesAndFuturesOptionsExpiryTimeAndLiquidationRegressionAlgorithm

* Revert FuturesExpiredContractRegression

* Revert HistoryWithDifferentContinuousContractDepthOffsetsRegressionAlgorithm

* Revert HistoryWithDifferentDataMappingModeRegressionAlgorithm

* Revert HistoryWithDifferentDataNormalizationModeRegressionAlgorithm

* Revert OpenInterestFuturesRegressionAlgorithm

* Revert RegisterIndicatorRegressionAlgorithm

* Revert SetHoldingsFutureRegressionAlgorithm

* Revert WarmupFutureRegressionAlgorithm

* Revert AutomaticIndicatorWarmupDataTypeRegressionAlgorithm

* Some cleanup

* Address changes request

* Address changes request

* Add more Class III Milk data to fix DelistingFutureOptionDailyRegressionAlgorithm
2022-08-18 10:56:44 -03:00
Martin-Molinero 15e399c96b Live T-1 selection warmup (#6395)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Live T-1 selection warmup

- Only use BaseDataCollection class
- Reuse collection enumerator
- Remove FuturesChainUniverseSubscriptionEnumeratorFactory
- Universe selection will use Cache providers
- Add null reference check
- Add more tests
- Fixes for warmup fill forwarding. Add more tests

* Address reviews. Add missing comments
2022-06-10 17:25:20 -03:00
Martin-Molinero a3f4dff299 Refactor chain providers (#6394)
* Refactor chain providers

- ChainProviders will use quote, open interest and trade information,
  which ever is first. Updating regression algorithm changing contract
  being added.
- Add base BacktestingChainProvider for shared logic, reducing
  duplication.
- ChainProviders will now use the given IDataCacheProvider and
  IMapFileProvider
- If Providers are called for a date the exchange is closed they will
  search for the previous available date where the exchange is open.
  Adding unit tests.

* Address reviews. Logging improvements
2022-06-10 11:24:09 -03:00
Ronit Jain 49562c712f Extract IB brokerage files and dependencies (#6283)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* make public safe to expose test properties

* extract ib files

* remove dependencies

* refactor to use ib reference as string

* remove un-used dependencies

* remove redundant refrences
2022-04-07 14:48:10 -03:00
Ronit Jain ad34429014 Refactor changes required for brokerages extraction (#6279)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* set price currency for open orders from brokerage setup handler, remove setters from brokerage

* order already has price currency being set by brokerage transcation handler

* remove as overwritten by BrokerageSetupHandler.GetOpenOrders

* refactor because can't use internal setters in private repos

* refactor, use ctor to set attributes

* remove redundant broker dependency

(cherry picked from commit 57b071eb174ca9fe857021ebffa7a546834e86f7)

* check limit price for limit order instead for checking price

* add price agrument for MarketOrder

* remove, not required anymore

* AddUnrequestedSecurity should not return null

* assert string in not null cases
2022-04-04 17:01:36 -03:00
Martin-Molinero 5f434f2fa5 SetupHandlers will exit if errors on initialize (#6092)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- SetupHandlers will exit right away if detect errors on initialize.
  Adding unit tests
2021-12-02 19:24:51 -03:00
Martin-Molinero 57f0d17c5d Remove user plan enum (#6055)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Remove user plan enum

- Get maximum order and runtime from job controls
- Remove user plan enum definitions

* Add log for BacktestingSetupHandler maximums
2021-11-12 19:37:23 -03:00
Martin-Molinero b77f0122b2 Improve python exception parsing (#5831)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing

- Improve python exception parsing adding support for line shift. Adding
  unit tests

* PythonException revert change

* Centralized and normalize algorithm runtime handling

* Adding support for C# line and file exception report
2021-08-06 20:11:54 -03:00
Martin-Molinero 1ed390c5f7 Minor tweaks (#5771)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Improve readme file, add nuget link, update Lean home page
- Increase 10min look back to 60 for setting currency conversion
- Replace error message for trace when minute resolution history request
  is empty
2021-07-15 13:03:46 -03:00
Martin-Molinero 7e2678d0f6 Normalize live holdings initialization (#5647)
Build & Test Lean / build (push) Has been cancelled
* Normalize live holdings initialization

* PaperBrokerage config based live state recovery

- Increase algorithm status storing sampling period
- Fix bug where duplicate cash entries could happen
2021-06-09 20:11:38 -03:00
Martin-Molinero e8cf002238 Live trading holding improvements (#5644)
Build & Test Lean / build (push) Has been cancelled
- Store holdings at end of algorithm
- Warmup holding market price if 0
2021-06-08 21:55:17 -03:00
Colton Sellers ab6027723f Refactor Api (#5251)
Build & Test Lean / build (push) Has been cancelled
* Use lean data key as param for request

* key -> filePath rename and some cleanup

* Refactor

* Add Organizations Endpoints

* Add some organization api wrapper objects

* Address namespace issue

* Reorganize Api Test into seperate files using one ApiTestBase

* Add Organization tests

* Use capitalized "API" test namespace to reduce amount of file changes

* Add License to test base

* Update /data endpoint functions and response objects

* Update ApiDataProvider Logic

* Handle deserialization of organization products

* Simplify converter

* Only throw for equity requests when not subscribed to map/factor files

* Add missing header

* Make arguement exception

* Api adjustments

* Add Zip factor and map file providers

- Common project will now reference Compression project and not the other way
  round.
- Adding Zip FactorFile and MapFile providers

* Refactor FactorFileProvider to use DataProvider to fetch files

* Use resulting MinimumDate in construction of FactorFile

* Nit FactorFile comments and arrangement

* Refactor MapFileProviders to use DataProvider for fetching files

* Refactor ZipFileProvider

* Clean up

* Refactor Backtesting Future/Option chain providers to use dataprovider

* Fixes for data/ endpoints and test adjustments

* Response objects adjustments/cleanups

* ApiDateProvider fixes and testing

* Add LocalZipFactorFileTests

* Update ApiDataProvider download test to verify stream is not null

* Implement posting of agreement summary and signed time

* Mark all Api related tests as explicit and document details on running

* Clarify default token on ApiTestBase

* Adjust summary

* Update Api responses for QCC, except org products which are sold in USD

* Implement cache expiration for zip MapFile and FactorFiles. Adding unit tests

* Fix multiple markets for ZipFactorFile provider

* Use Symbol as cache key

* Api.cs review

* Dispose of factorFileStream after reading

* Use zip.EntryFileNames

* Address a few reviews

* Few more fixes

* Address Api Review

* Add Job Org id to config

* Minor tweaks

* Compare with invariant culture

* Fixes Option Universe selection

* ZipEntryNameSubscriptionDataSourceReader will use IDataProvider

* Fix research

* Fix null reference exception

* Make duplicate log debug

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-06-07 11:37:44 -03:00
Jasper van Merle bb8bd27fe5 Support 2-leg currency conversions (#5552)
Build & Test Lean / build (push) Has been cancelled
* Apply changes from #2146 on current master

Credit for most of these changes goes to https://github.com/viliwonka.
Rebasing three years of changes didn't go so well, so I manually
applied the changes in #2146 on the current master branch.

This commit attempts to represent the changes in #2146 as closely as
possible, no cleanup has been performed yet. Despite this, some changes
were inevitable because part of #2146 has been implemented another way
in the past three years.

* Clean up

* Process review comments

* Add tests

* symbol -> potentialConversionRateSymbol

* Ensure regression tests run/get data

* Fix broken test

* Process review comments

* Process review comments

* Simplify currency conversion logic

* Performance improvement, doc updates and test rename

* Rebase fixes

* Remove obsolete properties

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-20 13:19:39 -03:00
Colton Sellers 4bb7fc17a2 Resolve Currency ConversionRate Backup (#5575)
Build & Test Lean / build (push) Has been cancelled
* Attempt to use daily resolution request to resolve currency conversion

* Add BaseSetupHandlerTests

* Minor tweaks. Adding missing dispose call.

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-19 19:54:02 -03:00