Commit Graph

845 Commits

Author SHA1 Message Date
Gerardo Salazar 7c3caec07f Use external NuGet package for Arrow
* Updates CI for Arrow
  * Updates DockerfileLeanFoundation
  * Adds Arrow to Python packages tests
2020-12-06 19:13:54 -08:00
Gerardo Salazar eb1181f5f7 Adds Futures Options Asset Class w/ IB Support (#4928)
* Adds preliminary universe selection for Future Options

* Fixes scaling issues with Future Options

* Fixes scaling multiplying by 10000x instead of using _scaleFactor

* Fixes scaling for Tick

* Revert changes to Tick since it divides the scaling factor

* Changes stale method name to new method name after rebase

* Fixes selection bugs, adds new methods, and adds unit tests

  * Fixes bug where Equity Symbol was created for an underlying
    non-equity Symbol, resulting in equity data trying to be loaded

  * Adds unit tests covering changes to Tick, QuoteBar, TradeBar and
    LeanData

  * Adds regression test for AddUniverseOption filter contract selection
    for Future Options

* Addresses review - modifies the AddFutureOption signature

  * Adds new AddUniverseOptions method overload
  * Removes and adds a new unit test
  * Misc. modifications to account for new changes

* Fixes bug where futures were loaded using default SID Date

  * Refactors and removes unnecessary work
  * Fixes regression algorithm, which previously made no trades

* Adds future option data

  * Adds the corresponding underlying data, in this case, futures data
    to enable usage of future options data

* Replaces data with new data (ES18Z20)

  * Improves Future chain filtering and updates regression stats

* Add AddFutureOptionContract API

* Expands regression and unit tests to test in finer detail

* Adds Python regression algorithms for AddFutureOption[Contract] methods

* Adds new unit test for BacktestingOptionChainProvider

  * Fixes bug with BacktesingOptionChainProvider where we
    attempted to load the Trades option chain first, resulting
    in breakage of backwards compatibility and limitation of the
    option chain.

  * Adds new regression algorithms (Py) to Algorithm.Python project

* Adds FutureOptionMarginBuyingPowerModel

  * Modifies code paths used to select margin model
  * Adds related unit tests for margin model

* Fixes issue with unit test and MHDB/SPDB lookup for Future Options

* Preliminary regression algorithm testing ITM call/put option buying

  * Fixes bug where fee model used did not find non-US market
    options fee model. We now use the futures fee model for future
    options because IB charges the same commissions per contract
    between futures and futures options

* Adds proper regression algorithm for ITM future options expiration

* Pushing broken algorithm for review

  * Currently, algorithm does not fill forward, causing
    a single future option to not get exercised when it is delisted.

* Adds FutureOptionPutITMExpiryRegressionAlgorithm

  * Improves existing regression algorithm for call side
  * Fixes bug in existing regression algorithm
  * Adds AAPL daily data to advance enumerator for ^^^ fix

* Adds additional future option regression algorithms

  * Adds Buy OTM expiration regression algorithms
  * Adds Sell ITM/OTM expiration regression algorithms
  * Adds missing Python regression algorithms

* Adds remaining Python regression algorithms and fixes issues

  * Fixes naming issues and statistics
  * Adds short option OTM regression algorithms (Py)

* Add license header and class comments to python algorithms

  * Cleans up comments and docstrings
  * Create Buy/Sell call intraday regression algo

* Redirects future options symbol properties to futures symbol properties

  * Asserts exercise/assignment price and updates stats in regression algos
  * Adds new unit test covering changes to SecurityService

* Adds comments and fixes failing test

* Partially fixes future option mis-calculated profit/loss

* Adjusts portfolio model to calculate FOP as a no upfront pay asset class

  * Updates regression algorithm statistics

* Begin IB FOP support

* Initial support for FOP IB data streaming, live í¾‰

  * Adds additional functionality to LiveOptionChainProvider
    - Allows querying CME API to retrieve option chains for CME products
    - Ultimately, it's also the groundwork for the CME
      LiveFutureChainProvider

  * Edits IDataQueueUniverseProvider interface to provide greater
    control to implementors of it

  * Misc. bug fixes required to get FOP data streaming through IB

* Adds comments, adds missing rategate call, and cleans up code

* Force exchange for FOP and Futures when no exchange is provided

* Fixes bug with Portfolio modeling across all asset classes

* Adds LiveOptionChainProvider tests for Future Options

* IB brokerage option symbol bug fixes and improvements

* Fixes contract multiplier lookup bug

  * Fixes issue where we attempted to subscribe to IB data feed with canonical security
  * Adds ES MHDB entry

* Reverts portfolio modeling changes for Futures Options

  * Since IB eats into our account's cash balance when
    a new FOP contract is purchased, we must model by applying funds
    to our cash whenever a new purchase/sell occurs.
    If we choose to model FOPs exactly as we do with futures, we
    will end up with an invalid TotalPortfolioValue on algorithm
    restart. By all means and purposes, FOPs are modeled exactly
    the same as equity options with respect to the portfolio.

  * Adds comments clarifying portfolio modeling and clarifies
    existing portfolio modeling comments with additional context.

* Fixes IB symbol lookup for future options

  * Fixes LiveOptionChainProvider looping 5 times per option chain
    request, even on success

  * Sets OptionChainedUniverseSelectionModel to produce a canonical
    future/future option/option Symbol to avoid creating two Symbols

  * Adds GLOBEX future option symbol mapping from future -> fop

* Fixes LiveOptionChainProvider loading wrong contract option chains

  * Fixes loading of futures options ZIP files when backtesting
  * Adds a string -> decimal JSON converter
  * Additional fixes/refactoring to the LiveOptionChainProvider

* Adds tests for changes to Symbol and LeanData

  * Reverts changes to IB-symbol-map

* Fixes Value for mapped future options tickers

  * Fixes Symbol test

* Changes path of future options to future's expiry date

  * Extra changes made to remove scaling from writing CSV
  * Added method to map from FOP Globex -> FUT Globex

* Fixes MOO and MOC orders for future options

  * Note: this order type might not be supported by IB or CME.

* Bug fixes and updates unit tests

* Update regression tests and data format

* Rebase changes

* 1. Multiple bug fixes for LiveOptionChainProvider, reverts IQFeed changes
2. Address review (partial): Code reuse and cleanup

1.
  * Modifies check in
    `AddFutureOptionShort(Call|Put)ITMExpiryRegressionAlgorithm`
    to ensure no buys have negative quantity

  * Code reuse changes in IB brokerage

  * Bug fix in IB brokerage where we assigned the FOP expiry
    as the futures expiry (requires verification)

  * Doc changes and adds missing summaries/license banners
  * Disposes of HTTP client resources in LiveOptionChainProvider
  * Renames classes and adds FutureOption folder in Common/Securities

2.
  * We revert back to the quotes API for the option chain,
    since the settlement API sometimes had missing strikes.

  * Fixes future option expiry being set as future's expiry
    in LiveOptionChainProvider

  * Fixes bug where wrong option chain was selected because of bad
    expiry lookup in the futures expiries returned from CME

  * Fixes multiple looping bug in LiveOptionChainProvider
  * Adds strike price scaling for LiveOptionChainProvider

  * Reverts IQFeed changes and simplifies interface upgrade changes

  Some additional challenges we'll have to solve as part of FOPs:

    - The `OptionSymbol.IsStandard` method makes the assumption that
      weeklies contracts follow the pattern equities follows, which
      does not apply to Futures Options

    - The Subscription created in:
        `OptionChainUniverseSubscriptionEnumeratorFactory`

      ...adds a Trade config. For illiquid contracts, this
      will delay universe selection for the option symbol
      until we get a trade. However, if we add a quote config,
      the data would instead be loaded based on the first quote
      we received from the brokerage.

      But since we're currently using a trade config, illiquid
      contracts won't start streaming data until it receives a trade.

NOTE: this commit is a WIP to addressing the reviews received in the PR,
but has been committed early for efficiency in the review process

* Fixes regression algorithms and misc. bugs

  * Fixes map file lookup for non-equity options
  * Adds extra assertion at end of algorithm to ensure no holdings are
    left when the algorithm ends.

  * Adds FutureOptionSymbol, allowing all contracts through as standard
  * Changes SPDB to allow defaulting to underlying future symbol
    properties if no entry is found for the given FOP

  * Fixes calls to SPDB in SecurityService, IBBrokerage
  * Reverts AAPL daily ZIP file to fix majority of regression algorithms
  * Adds FOPs symbol properties
  * Fixes existing symbol properties for a few futures
  * Adds tests for changes to Symbol Properties Database

* Removes string SPDB lookup method

  * Updates tests and misc callees of previous method

* Updates all regression tests to use data of already expired contracts

  * Adds Futures Options Expiry Functions tests
  * Adds required futures data for 2020-01-05

* Address review (partial): Expands test coverage and fixes tests

* Set option chain tests parallelism to fixture only

* Fixes broken test for contract month delta for FuturesOptionsExpiryFunctions

* Changes delisting date logic for Futures Options

* Address review: removes duplicate code, misc code fixes

  * Bug fix in MarketHoursDatabase.GetDatabaseSymbolKey() where
    we would use the underlying's Symbol for lookup in the MHDB

  * Adds missing license banner
  * Removes Futures Options entries from MHDB
  * Adds new tests

* Adds SecurityType.FutureOption

  * Converts any underlying comparisons and uses SecurityType directly
    instead for FOP specific behavior

  * Extra code modifications to acommodate new SecurityType

* Addresses review: fixes order fee bug on exercise

  * Additional bug fixes and adding of SecurityType.FutureOption
  * Updates regression algorithms OrderListHash

* Fixes various bugs in IB live implementation

  * Fixes bug setting the right contract expiration date for FOP
    generated by LiveOptionChainProvider

  * Adds new function to FuturesOptionsExpiryFunctions

  * Clarifies parameter names better in some functions/methods

  * Fixes bugs in IB brokerage for FOPs

* Address review - code cleanup and refactor

  * Remove MappingEventProvider, SplitEventProvider, and
    DividendEventProvider for Futures Options in
    CorporateEventEnumeratorFactory

* Address review: Use MHDB key resolver in SPDB

* Makes regression tests pass and adds comment for expiry issue

* Fixes MHDB lookup on string symbol method

* Adds Futures Options greeks regression algorithm (C# only)

* Adds explanitory comment on MHDB FOP lookup

* Remove python from FutureOptionCallITMGreeksExpiryRegressionAlgorithm
2020-12-02 21:49:59 -03:00
Martin-Molinero 646f22a63f Remove python stubs directory (#4955)
* Remove python stubs directory

- Removing python stubs directory since after https://github.com/QuantConnect/Lean/pull/4899
it was been replaced by a python package `quantconnect-stubs`.
- Reverting IDE settings using the stubs folder PR https://github.com/QuantConnect/Lean/pull/4657

* Revert "Adds Python stubs location definition for PyCharm and Visual Studio Code (#4657)"

This reverts commit aded66ec5b.

* Address self-review: Provide list of imports and refactor python readme

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
2020-11-24 12:45:28 -08:00
James Kardatzke 46ef9a9dbb Quiver Quantitative Integration (#4869)
* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Delete QuiverHouseDataDownloader.cs

* Delete QuiverSenateDataDownloader.cs

* Delete QuiverPoliticalBetaDataDownloader.cs

* Add files via upload

* Delete QuiverHouse.cs

* Delete QuiverSenate.cs

* Delete QuiverPoliticalBeta.cs

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Add files via upload

* Delete QuiverDataAlgorithm.cs

* Add files via upload

* Add files via upload

* Add files via upload

* Addresses self review: Cleans up code and adds new unit tests

  * Adds Quiver* C# files to project
  * Adds new unit test for QuiverCongress
  * Adds Python algorithm example

* Address self reviews

- Adding some missing xml docs
- Removing unrequired imports.
- Minor rename from Date to ReportDate
- Live trading will throw InvalidOperationException

* Fixes for example algorithms

Co-authored-by: Gerardo Salazar <gsalaz9800@gmail.com>
Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-11-13 20:07:17 -03:00
Aaron Janeiro Stone a5d9526d65 Add defaults (as comments) to Python.Runtime (#4904)
* Redone defaults for python runtime dll config and readme

* Update Python.Runtime.dll.config

Co-authored-by: Jared <jaredbroad@gmail.com>
2020-11-02 15:02:59 -08:00
Alexandre Catarino 84264ca7ef Adds CustomBuyingPowerModelAlgorithm (#4824)
* Adds CustomBuyingPowerModelAlgorithm

This algorithms is an example on how to implement a custom buying power model.

In this particular case, it shows how to override `HasSufficientBuyingPowerForOrder` in order to place orders without sufficient buying power according to the default model.

* Upgrades CustomModelsAlgorithm to Include CustomBuyingPowerModel

The custom buying power model overrides `HasSufficientBuyingPowerForOrderResult` but it doesn't change the trades and, consequently, the regression statistics.
2020-10-21 17:27:30 -07:00
Martin-Molinero bfd319c91e OptionChain and OptionContract improvements (#4804)
* OptionChain and OptionContract improvements

- QCAlgorithm.AddUniverse will return the added Universe instance.
- Adding new OptionChainedUniverseSelectionModel will monitor a Universe changes
  and will spwan new OptionChainUniverse from it's selections. Adding
  regression test Py/C#.
- Adding new OptionContractUniverse that will own option contracts and
  their underlying symbol. Adding regression test
- Fix double notification for security changes, bug seen in updated
  UniverseSelectionRegressionAlgorithm
- Remove UniverseSelection special handling for Option and Future chains
- Fix DataManager not removing SubscriptionDataConfigs for Subscriptions
  which finished before being removed from the universe
- Refactor detection of user added Universe so that they do not get
  removed after calling the UniverseSelectionModel

* Add check for option underlying price is set

* Address reviews

- Adding python regression algorithm for
  `AddOptionContractFromUniverseRegressionAlgorithm`
  and `AddOptionContractExpiresRegressionAlgorithm`
- Rename QCAlgorithm new api method to `AddChainedOptionUniverse`

* Fix universe refresh bug

- Fix bug where a universe selection refresh would cause option or
  future chain universes from being removed. Adding regression algorithm
  reproducing the issue.

* Rename new option universe Algorithm API method

- Rename new option universe Algorith API method from
  AddChainedOptionUniverse to AddUniverseOptions
- Rebase and update regression test order hash because of
  option expiration message changed
2020-10-09 10:52:50 -03:00
Colton Sellers faa4e91e04 Test 4249 fix regression algorithms executed together (#4832)
* Fix for running regressions algorithms together

* Remove unneeded changes
2020-10-08 19:59:06 -03:00
Martin-Molinero 9cb2452025 Oanda default Forex Market (#4706)
* Oanda default forex Market

- Use Oanda as default forex Market since it has more pairs.
- Remove FXCM data add Oanda equivalente data.
- Update unit and regression tests

* Address reviews

- Revert FXCM data removal
- Remove unrequired commented code

* Fix rebase
2020-09-14 16:43:23 -03:00
Martin-Molinero 4a0fb30df5 Fixes for OpenInterest storing (#4712)
- Fix Slice.Get OpenInterest type. Adding unit test
- Fix for SecurityCache that wasn't storing OpenInterest types
- Updateing regression tests to covere these usages
2020-09-11 15:28:27 -07:00
Derek Melchin 20791d6a9e Add schedule queuing algorithm (#4695)
* Add schedule queuing algorithm

* Add algorithm file to csproj file

* Add c# version of queuing algorithm
2020-09-04 12:34:07 -07:00
Colton Sellers 8edc89a4ff Feature Python Option Filtering Support (#4667)
* Support List and OptionFilterUniverse for Py filter

* Regression algorithm for testing

* Unit Tests

* Fix for process

* Tighten filters to reduce load on automated testing

* Address review v2
2020-09-02 21:38:16 -03:00
QuantConnect Server Applications 90e911f22c Adds autogenerated Python stubs via Travis for QCAlgorithm (Build 14139) (#4671)
Co-authored-by: Python Stubs Deployer <stubs-deploy@quantconnect.com>
2020-08-29 13:10:37 -03:00
QuantConnect Server Applications d4ca27f93f Adds autogenerated Python stubs via Travis for QCAlgorithm (Build 14115) (#4662)
Co-authored-by: Python Stubs Deployer <stubs-deploy@quantconnect.com>
2020-08-28 16:43:17 -03:00
Gerardo Salazar aded66ec5b Adds Python stubs location definition for PyCharm and Visual Studio Code (#4657) 2020-08-26 14:17:33 -07:00
Colton Sellers b8674731a5 Feature 2456 custom Python consolidator support (#4637)
* DataConsolidator Wrapper for Python Consolidators

* Regression Unit Test

* Refactor Regression test

* Bad test fix

* pre review

* self review

* Add RegisterIndicator for Python Consolidator

* Python base class for consolidators

* Modify regression algo to register indicator

* unit test - attach event

* Test fix

* Fix test python imports

* Add license header file and null check

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-08-25 17:26:55 -03:00
Gerardo Salazar b866c8b893 Removes Robintrack data source (#4628) 2020-08-14 19:46:30 -03:00
Colton Sellers cf888028af Address review 2020-06-26 09:45:33 -07:00
Colton Sellers 7058254cc7 Addition of ExtendedMarketTradingRegressionAlgorithm in Python 2020-06-25 17:27:30 -07:00
C-SELLERS e9d69eba74 Update Pandas version for Python Algorithms 2020-06-22 15:16:16 -07:00
Martin Molinero 39e56ea7c0 Address reviews
- Removing the need to call MHDB.SetEntry, this will be handled by the
  data manager
2020-06-15 20:05:12 -03:00
Martin Molinero b575880739 Coarse tiingo new universe selection algorithm
- Adding an example algorithm of a custom universe selection using
  coarse data and adding tiingo news. If conditions are met will add the
  underlying and trade it
- Adding required UniversePythonWrapper
2020-06-15 20:05:08 -03:00
Martin-Molinero 370685447d Merge pull request #4486 from AlexCatarino/feature-add-ema-cross-front-contract-algorithm
Adds EmaCrossFuturesFrontMonthAlgorithm
2020-06-05 19:10:23 -03:00
Martin-Molinero 2e1a5925fd Update EmaCrossFuturesFrontMonthAlgorithm.py 2020-06-05 10:55:39 -03:00
AlexCatarino 7f1e764211 Addresses Peer-Review
- Adds comment explaining `OnlyApplyFilterAtMarketOpen` usage;
- Fix custom plot bug.
- Use `WarmUpIndicator`.
2020-06-05 01:39:35 +01:00
Martin-Molinero b4e7f4ce46 Merge pull request #4460 from michael-sena/bug-4459-python-postbuildevent
Adjust Algorithm.Python PostBuildEvent location and ensure scripts pass
2020-06-04 19:40:31 -03:00
AlexCatarino f008e90202 Adds EmaCrossFuturesFrontMonthAlgorithm
This example shows how to create an EMA cross algorithm for a futures' front contract. Once the contract is added, the indicators are registered to a new consolidator and warmed up with historical data. When a contract is removed, the consolidator is removed and the indicators are reseted. We don't need to liquidate it, because it's liquidated automatically since it has expired.
2020-06-04 00:21:49 +01:00
Martin Molinero b5a9494a9b Fix for python build scripts 2020-06-02 12:52:28 -03:00
AlexCatarino 7a9778f9fd Addresses Peer-Review
- Do not reuse parameter variables `start` and `end`. Create new variables with meaningful names and rename them to `startUtc` and `endUtc`.
- Use `EachTradeableDayInTimeZone` to calculate `tradableDates`.
- Adds another test/assertion in the regression algorithms to ensure tests in the scheduled event were performed.
2020-05-29 14:56:10 +01:00
AlexCatarino 8b1afdb083 Fixes History Request Data Time Zone
The tradable days of the history request should respect the data time zone since the data source files also do.

Upgrade `BasicTemplateFuturesHistoryAlgorithm` to a regression algorithm and add a schedule event to test history requests every hour.
2020-05-29 00:31:12 +01:00
Michael Sena bb67bd4d0d Adjust Algorithm.Python PostBuildEvent location and ensure scripts pass 2020-05-26 15:45:09 +10:00
Martin-Molinero 1d0d4f4d7b Add assembly descriptions (#4442)
Add assembly descriptions
2020-05-21 09:35:12 -07:00
Martin Molinero 891cf4d69b Update to .Net 4.6.2 2020-05-15 17:32:46 -03:00
Martin Molinero 269b8ba204 Remove PsychSignal data
- Remove PsychSignal data since it's no longer available
2020-05-11 18:48:13 -03:00
Jared 4fea6128a7 Merge pull request #4390 from gsalaz98/feature-alternative-data-robintrack
Implements Robintrack BaseData and example algorithms
2020-05-11 12:06:53 -07:00
Gerardo Salazar d2eb0482fa Address reviews - refactors portions of code and adds tests 2020-05-08 18:22:16 -07:00
Gerardo Salazar d3a5f52fe9 Implements Robintrack BaseData and example algorithms 2020-05-07 16:26:15 -07:00
Derek Melchin 5b48138f4d Adds talib indicators example algorithm 2020-05-07 13:57:57 -06:00
Martin Molinero 4a99eb11c6 Fix regression tests
- Fix regression tests. Future symbol contains the market which was used
  un the order hash list
2020-04-28 16:29:16 -03:00
Martin Molinero 8e71e48c07 PythonSlice
- AlgorithmPythonWrapper will directly call base OnFrameworkData()
  implementation skipping going through python and it's overhead
- Small performance improvement for adding Tick data points into a Ticks
  collection
- For python always wrap slice with PythonSlice, so that slice.Get()
  works even when no custom data is present, adding test.
2020-04-27 19:03:03 -03:00
Jared f25b2dbc64 Merge pull request #4340 from AlexCatarino/feature-options-price-model-example
Adds BasicTemplateOptionsPriceModel
2020-04-24 17:49:58 -07:00
AlexCatarino cc3f2416ff Adds BasicTemplateOptionsPriceModel
Show how to define an options price model and the available options. Tests whether the options contracts has a valid (non-zero) `Greeks.Delta`
2020-04-25 01:46:29 +01:00
Martin Molinero de05f15a12 Address review
- Adding live trading unit test
2020-04-24 19:04:41 -03:00
Martin Molinero b3970bdc3b Fixes and rebase
- FineFundamentalSubscriptionEnumeratorFactory will not emit empty fine
  points
2020-04-24 17:21:28 -03:00
Jared 357729521d Merge pull request #4326 from QuantConnect/feature-3952-objectstore-python
Adds Python Example of ObjectStore Feature
2020-04-22 09:40:44 -07:00
AlexCatarino dc79810c3f Modifies KerasNeuralNetworkAlgorithm to Include ObjectStore Example 2020-04-21 23:14:21 +01:00
AlexCatarino 2e5eddfc29 Adds Python Example of ObjectStoreExampleAlgorithm
Minor change in C# version to keep consistency between examples.
2020-04-21 23:14:21 +01:00
Martin Molinero fa819126c9 Improve implicit conversion failure message
- Improve string to Symbol implicit conversion exceptions message
2020-04-17 22:11:24 -03:00
AlexCatarino e23d7e4486 Adds CustomPartialFillModelAlgorithm
This basic algorithm implements a `CustomPartialFillModel` class that chnages the behavior of the `FillModel.MarketFill` to simulate partially fill orders.
2020-04-13 18:50:03 +01:00
AlexCatarino 8564a29d8a Updates PythonTools Project
Adds missing project files and removed old ones.
2020-04-13 18:48:04 +01:00