Commit Graph

2543 Commits

Author SHA1 Message Date
Martin-Molinero 08e1f31271 Minor live logging improvements (#7240) 2023-05-12 18:13:37 -03:00
Martin-Molinero 884bfe4482 Add support for brokerage side new orders (#7228)
* Add support for brokerage side new orders

- Add support for brokerage side new order events for liquidation cases

* Minor cash delta fix

* Improve account cash logging

* Fix null reference exception for open orders
2023-05-02 16:21:19 -03:00
Martin-Molinero 8b2eb5bd68 Refactor Settlement Model API (#7225)
* Refactor settlement mode api

- Add new Scan method on ISettlementModel API. To be used by
  futures settlement model mark to market handling
- Use 'Parameters' classes for 'ISettlementModel'

* Address reviews
2023-05-01 11:23:31 -03:00
Martin-Molinero 9c893e83a2 Minor optimizer improvement (#7179)
- Handle optimization backtest name
- Allow configurable optimizer for Optimizer.Launcher project
2023-04-12 15:35:54 -03:00
Martin-Molinero f9bbec1e16 Remove internal subscriptions from future chain (#7169)
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Remove internal subscription from slive future chain collection.
  Adding regression algorithm reproducing the bug
2023-04-05 18:23:04 -03:00
Martin-Molinero 227ade63ed Refactor automatic option assignment (#7165)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Refactor automatic option assignment to allow overriding behavior.
  Adding & updating existing tests
2023-04-04 17:40:31 -03:00
Martin-Molinero 645e1ca88c Downloader Data Provider Improvements (#7159)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Downloader data provider fixes

- Fix stored data time zone, which is expected in data time zone
- Minor logging improvements
- LeanData Parse handles etf universes path. Adding unit test
- LeanDataWrite will only map if there's something to map to

* Fix download data provider synchronization

* Synchronization improvements after more testing

- KeyStringSynchronizer will handle reentrancy. Adding unit tests
- Define a unique Synchronizare for downloading data, since not related
  to disk reading & writting
2023-03-31 18:29:55 -03:00
Martin-Molinero c836c62945 LeanDataWriter can handle mapping (#7150)
* LeanDataWriter can handle mapping

- Adjust LeanDataWrite so that it can handle symbol mapping if required.
  Adding unit tests

* Fix Lean Data path parsing
2023-03-29 16:04:23 -03:00
Martin-Molinero 403814d64f Handle error sending notification (#7146) 2023-03-27 15:31:53 -07:00
Martin-Molinero bbbf6f3a74 Minor downloader data provider improvements (#7130)
- Improve start time request for daily/hour files
- Skip custom data since it's not supported by the LeanDataWriter
2023-03-23 12:27:34 -03:00
Martin-Molinero 44d1454ed7 Handle downloader data provider no MHDB entry (#7115)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Handle downloader data provider no MHDB entry

* Address review: log once
2023-03-21 17:26:09 -03:00
Martin-Molinero a58f6b679d Minor fake data queue handler fixes (#7114)
- Minor fake data queue handler fixes. Adding unit tests
2023-03-21 15:10:03 -03:00
Martin-Molinero 56881e64fe Expand result state attributes (#7097)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Add order, insight & log count into result algorithm state
2023-03-20 15:00:57 -03:00
Martin-Molinero 52de357b27 Terminal Link Minor Fixes (#7093)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix data-queue-handler configuration initialization using paper brokerage
- Fix for downloader data provider request times
- Fix null reference exception in live result handler if algorithm is
  null
2023-03-20 12:24:16 -03:00
Martin-Molinero a9c88ad25c ResultHandler state file improvements (#7078)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Adjust start/end time string format
- In live trading will store the status file at the start, every 10 min
  and at the end
2023-03-16 18:24:49 -03:00
Martin-Molinero bc6ac5470b Minor IndexOption underlying TZ fix (#7072)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Minor IndexOption underlying TZ fix

- Fix for option universe chain underlying TZ which might not match the
  options one, bug affected IndexOption specifically.

* Update affected regression tests
2023-03-15 18:41:52 -03:00
Martin-Molinero 06bec5fb8e Add more logs for regression tests (#7073)
- Add holdings & cashbook logs from regression result handler, useful
  for debugging & developing
2023-03-15 18:25:26 -03:00
Martin-Molinero d6f894eb3b Fix fill forward unit tests (#7070)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Fix for some fill forward unit tests that were failing when run
  individually due to cache state in the
  TextSubscriptionDataSourceReader.
2023-03-15 12:45:04 -03:00
Martin-Molinero 8ed5f69ea6 Minor fill forward enumerator fix (#7068)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Make sure the fill forward enumerator does not fill forward beyond the
  subscription end time. Adding tests
2023-03-15 10:18:43 -03:00
Martin-Molinero 2f03c92e30 Fill forward runtime adjustment (#7065)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix for fill forward adjustment at runtime since the data stack runs
  async, we will remove and readd afected subscriptions.
2023-03-14 18:00:22 -03:00
Martin-Molinero 069246dda5 Performance Improvements (#7064)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Performance Improvements

- Remove Immutable collection from position groups to improve
  performance. Adding more tests
- Remove ConcurrentDictionary from PortfolioTargetCollection &
  DataManager. Adding more tests
- Add Securities enumerator keys & values cache

* Add test for PortfolioTargetCollection remove by reference
2023-03-14 12:00:42 -03:00
Jhonathan Abreu fbf8ffd924 Handle price discontinuity on volatility models (#7058)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Apply splits and dividends to volatility models

* Apply splits and dividends to volatility models using history requests

* Add new ScaleRaw data normalization mode

Handling the new mode in the price scale enumerator.

* DataNormalizationMode.ScaledRaw history requests

* Minor changes

* Minor changes

* Disable new normalization mode in AddSecurity methods and other minor changes

* Peer review

* Minor changes

* Peer review

* Minor changes

* Peer review

* Peer review

* Peer review

* Add scaled raw history regression algorithm

* Add more regression algorithms

* Add more regression algorithms

* Add Slice.TryGet unit tests

* Peer review

* Peer review

* Peer review

* Peer review

* Peer review

* Update algorithms stats

* Peer review

* Peer review
2023-03-13 13:11:30 -03:00
Martin-Molinero c6322956d9 Live trading holdings key (#7043)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix live trading holdings uniqueness

- Fix live trading holdings key uniqueness

* Fix existing & add new test
2023-03-13 09:51:00 -03:00
Martin-Molinero bbbab6d9a8 Refactor alpha statistics phase I (#7055)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor alpha statistics

- Refactor alpha statistics, cleaning up and simplifying no longer required calculations and scoring
- Adding new InsightEvaluator abstraction, adding C# & PY regression
  algorithms

* Optimization backtest result json converter update

* Address reviews

- Remove IAlphaHandler, move insight storage responsability to IResultHandler
  and centralizing insight collection on the QCAlgorithm.Insights to be
  reused by the framework models
- Fix portfolio turnover single day backtests and duplicate time
  sampling handling. Updating regression algorithms

* Add InsightCollection tests and minor fixes

* Adding more & improved tests
2023-03-10 13:12:23 -03:00
Martin-Molinero a5586ba562 Enqueable Enumerator Performance Improvement (#7049)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- Refactor enqueable enumerator not to require read lock most of the
  time
2023-03-06 15:40:55 -03:00
Martin-Molinero c4433098c3 Refactor Framework Statistics (#7041)
* Refactor framework statistics

* Further insight chart cleanup
2023-03-03 19:40:29 -03:00
Martin-Molinero 38f5a9c60c Bump to pythonNet 2.0.18 (#7036)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2023-03-03 13:11:03 -03:00
Jhonathan Abreu 81d1bb5095 Sample exposure for non invested portfolio (#7030)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2023-03-02 15:23:10 -03:00
Jhonathan Abreu cdcd8fdacd Apply splits to TradeBuilder (#7020)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Apply splits to TradeBuilder

* Minor tests changes

* Minor changes

* Add regression algorithm
2023-03-02 15:15:40 -03:00
Jhonathan Abreu 0c77be0ba8 Allow closed orders tag update (#7016)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Allow closed orders tag update

* Minor changes
2023-02-27 18:42:24 -03:00
Martin-Molinero c1c4114dca Newtonsoft & Protobuf & RecyclableMemoryStream Updates (#7002)
* Dependency updates

- Update Newtonsoft to 13.0.2, protobuf to 3.2.0, RecyclableMemoryStream
  to 2.3.1

* Downgrade protobuf to 3.1.33
2023-02-23 12:11:01 -03:00
Martin-Molinero 48914b08db Chart Series Settings Lost (#6997)
* Fix live trading chart series index

- Fix Series loosing their index when handled by the
  LiveTradingResultHandler
- Some clean up for Series.cs

* Fix SeriesSampler loosing Series settings
2023-02-21 13:07:56 -03:00
Martin-Molinero 6fd1a002fe Adjust index option data timezone (#6970)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix index option data timezone

- Fix index option data timezone which was 1 hour late than expected

* Update existing index option tests

- Update existing index option tests.
- Disable index option daily resolution support
2023-02-16 17:35:10 -03:00
Martin-Molinero 3507c91640 Fix future daily price scale factor update (#6963)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix future daily price scale factor update

- Fix future daily price scale factor update which was happening 1 day
  late due to data and exchange time zone differences.
  Adding regression algorithm reproducing and asserting it.

* Address review
2023-02-14 20:09:59 -03:00
Martin-Molinero dc2c68c754 Fix security cache reset (#6959)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix security cache reset

- Fix security cache reset so it only happens when no universe has it as
  a member. Adding regression algorithm reproducing issue

* Avoid non deterministic FillForward behavior
2023-02-14 12:09:27 -03:00
Martin-Molinero 8235de42a0 Minor debugging improvements (#6954)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Minor debugging improvements

- Improve exception message if user provides empty/null ticker
- Log margin information on algorithn initialization
- Add unit test asserting option strategy position groups resolve
  correctly for existing holdings

* Symbol Lazy SecurityDefinitionSymbolResolver

- Symbol will have a lazy SecurityDefinitionSymbolResolver instance
  resolution
2023-02-13 10:49:10 -03:00
Jhonathan Abreu e30232e21b Portfolio.UnsettledCashBook update when quote currency is not the account currency (#6871)
* Tracking unsettled cash for non-USD account currency

* Add unit tests

* Peer review

* Move and rename CashBook.UpdateType to Globals
2023-02-09 12:53:52 -03:00
Jhonathan Abreu 914485c595 Rename SubscriptionRequest.TradableDates to TradableDatesInDataTimeZone for clarity (#6892)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Usages were reviewed to make sure all of them expect it to be in data
time zone instead of exchange time zone
2023-02-08 20:28:37 -03:00
Martin-Molinero 6563d6e394 Use any resolution for chain provider (#6860)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Use any resolution for chain provider

- Use any data resolution available to source symbols for the file based
  chain provider. Adding unit test

* Fix selection timezone bug

- Fix universe selection timezone bug. Updating regression algorithms
2023-01-16 16:10:07 -03:00
Martin-Molinero a52822853d Expand Index option support (#6849)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Expand Index option support

- Adjust APIs so that the same underlying can be used
  for different options, adding support for SPX weekly options. Adding
  regression tests

* Fix IndexOption.IsStandard

* Add IndexOption test
2023-01-12 16:39:02 -03:00
Ronit Jain a9c4d29776 Feature add algorithm state information in result packets (#6839)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* add state to backtesting result packets

* self review

* add state in live results

* set runtime error in algorithm

* set hostname in job packets

* Set state when algorithm is null

* address review

* address reviews

* set runtime error when algorithm is not available during runtime
2023-01-11 10:54:38 -03:00
Jhonathan Abreu 6ed8cb5cae Backtest progress calculation (#6847)
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Fix backtest progress calculation

* Backtest progress monitor class

* Minor changes

* Lazy calculating the processed days count

* Minor changes

* Minor changes
2023-01-11 10:33:38 -03:00
Jhonathan Abreu b54281b262 Combo orders (#6813)
* Feature combo orders

- Add support for combo orders

* Make fill model wait for all grouped orders to emit fills

* Add ComboFill to model multiple fills for combo orders

* Fill combo limit orders

Add some regression algorithms

* Add fill implementation for combo leg limit orders

* Add IFill as common interface for Fill and ComboFill

* Refactor combo orders removing IGroupOrder interface

Move the group order manager to the base Order class

* Update algorithms

* Handle combo order events atomically

* Refactor brokerage transaction event handler

* Refactor combo fill models

* Process fills in batch

* Combo orders fill model tests

* Combo leg limit orders algorithm

* Regression algorithms cleanup

* Fill and combo fill classes cleanup

* Housekeeping

* Refactor equity fill model to derive from base fill model

* Address review changes request

* Handling the new types of orders in the OrderJsonConverter

* Add regression algorithm to test combo orders update/cancel

* Add regression algorithm to test combo orders update/cancel

* Housekeeping

* Address review changes request

* Minor changes

* Security transaction handler method for setting order request id

* Extend public interface for placing combo orders

* Combo order tickets demo algorithm python version

* Tweaks and updates

* Minor fixes

* Minor changes

* Minor fixes

* Address reviews minor fixes

* Minor fixes

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2023-01-06 17:58:43 -03:00
Martin-Molinero f12e8fc08e CryptoFutures adjustments (#6836)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* CryptoFutures adjustments

- Address reviews of CryptoFutures:
- Add new Slice MarginInterestRates collection
- Add new regression tests asserting funding rate application behaves
  the same no matter the resolution
- Add auxiliary data by type into the security cache
- Revert BuyingPowerModel changes

* Make interest rate application time deterministic
2023-01-05 18:07:01 -03:00
Alexandre Catarino 6b6ca5b460 Saves Dividend Payments to Security Holdings (#6830)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Saves Dividend Payments to Security Holdings

Saves information about dividend payments to `SecurityHoldings`. This information will be used to factor in dividend payments to the `Profit` and `NetProfit`.
Adds `SecurityPortfolioManager.TotalNetProfit` to sum all the `SecurityHoldings.NetProfit`.

Adds regression and unit tests.

* Addresses Peer-Review

Improves Regression Test.
2023-01-03 18:37:00 -03:00
Martin-Molinero ab4c3bf55e Live margin interest rate bug (#6833)
Benchmarks / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Fix for live MarginInterestRate

* Add unit tests
2023-01-03 14:13:11 -03:00
Martin-Molinero ee683933a3 Feature Perpetual Crypto Futures (#6807)
Python Virtual Environments / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* WIP

* Add base currency cash

* Symbol properties and data processing

* Add basic template algorithm

* Add hourly crypto future algorithm

* Minor fixes after live trading testing

* CoinApiDataQueueHandler CryptoFuture support

* Address reviews

* Fix regression algorithms after update
2022-12-30 13:58:11 -03:00
Martin-Molinero 7677b99bf6 Handle OnOptionNotification race condition (#6821)
- Handle OnOptionNotification race condition to take into account sell
  orders too. Updating unit tests
2022-12-26 17:21:51 -03:00
Martin-Molinero dcfdbd8c82 Add OnOrderIdChangedEvent (#6816)
* Add OnOrderIdChangedEvent

- Add OnOrderIdChangedEvent so that brokerages can change the brokerage
  order ids on the original order directly

* Fix bug
2022-12-21 20:53:39 -03:00
Martin-Molinero c4329c4132 More resilient object store file read (#6800)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2022-12-13 15:25:47 -03:00