363 Commits

Author SHA1 Message Date
Louis Szeto cfa4a4c1c1 Update readme information in Equity data (#6536)
* Update readme

* Update equity readme

* Add quote info

* Fix typo

* Add back QuoteQuant as pre-2007 vendor

* Fix typo
2022-08-10 18:11:20 -03:00
Martin-Molinero 7540af454c Warmup resolution respected (#6467)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Respect warmup resolution given

- The data feed will respect the warmup resolution given and override
  the resolution used by the algorithm when adding a subscription.
  Updating regression algorithm to keep previous statistics. Adding new
  regression algorithm asserting the desired behavior

* Testing improvements

- Add more unit tests and regresion test
- Add missing data for crypto
- Fix bug with FFed data crossing after the end time of the warmup
  request

* Add more Warmup resolution regression algorithms

- Adding more warmup resolution regression algorithms, using
  Settings.WarmupResolution and an option selection case

* Add more warmup regression tests

- Adding more warmup regression tests.
- Will no longer skip universe selection subscriptions from warmup
  resolution enforcement. Updating regression algorithms data points

* Fix bug with data rounding

- Fix data rounding bug when warmup resolution is set to a different
  value than the original configuration. Updating regression algorithms
  to assert the expected behavior

* Address reviews

- Revert regression algorithms changes to use Resolution during warmup.
  Updating their stats.
- Adding new regression algorithms asserting the behavior warming up
  using a timespan and no warmup resolution
- Fix bug where data used to warmup the 'normal' enumerator will make it
  through into the warmup time span. Updating tests

* Address reviews

- Add missing comments, explaning warmup algorithms time span
  calculations.
- Revert changes in existing `WarmupOptionTimeSpanRegressionAlgorithm`
  to reduce diff to minimum
- Adding new warmup unit tests asseting algorithm warmup start time, for
  different combinations of bar count, timespan, resolution
2022-07-15 13:05:06 -03:00
Jhonathan Abreu e9409bf207 Add contractDepthOffset parameter to QCAlgorithm.History() (#6438)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add contract depth offset parameter to big History() methods

- Added unit tests
- Added little more ED daily data

* Add C# regression algorithm

* Add Python regression algorithm

* Update regression algorithms stats

* Address changes request
2022-06-27 10:04:33 -03:00
Martin-Molinero 4c3afa6ff1 Add OpenInterestAnnual (#6436)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Adding OpenInterestAnnual where the contract maps when any of the back month
  contracts of the next year have a higher volume that the current front month.
- Add support for backwards compatible data mapping mode additions
- Updating ES map and factor files. Adding a little daily data too
2022-06-23 19:22:24 -03:00
Jhonathan Abreu 505ef17565 Add data mapping mode parameter to QCAlgorithm.History() (#6415)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Add dataMappingMode parameter to QCAlgorithm.History()

* Add C# regression algorithm

* Add Python regression algorithm

* Cleanup

* Add data mapping mode parameter only to big History() methods

* Fix regression algorithms and add required data

* Fix unit test

* Update regression algorithms stats
2022-06-22 19:25:30 -03:00
Martin-Molinero a12a43c1ef Update VX futures market hours (#6417)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-06-22 17:25:18 -03:00
Alexandre Catarino e3313c7c45 Fixes Contract Multiplier for Treasury Futures (#6418)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-06-22 17:25:00 -03:00
Martin-Molinero 8fb188d567 Add missing Juneteenth holiday (#6412)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add missing Juneteenth holida. Reference https://www.timeanddate.com/holidays/us/juneteenth
2022-06-20 10:42:37 -03:00
chriscdev adb6afc4f1 Feature 6208 add nyseliffe exchange to interactive brokers (#6209)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Added NYSELIFFE exchange support to Interactive Brokers brokerage.

* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].

* Added NYSELIFFE exchange support to Interactive Brokers brokerage.

* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].

* Added NYSELIFFE exchange support to Interactive Brokers brokerage.

* Fixed unit test AddSecurityWithSymbol by adding market hours for Equity-nyseliffe-[*], Index-nyseliffe-[*], Option-nyseliffe-[*].

* - Added NYSELIFFE futures to Futures.cs
- Added method GetGoodFriday() to calculate Good Friday to FuturesExpiryUtilityFunctions.cs.
- Added unit tests for GetGoodFriday().
- Added NYSELIFFE futures expiries to FuturesExpiryFunctions.cs which uses GetGoodFriday() instead of hardcoding the date of Good Friday in the market-hours-database.json.
 - Added unit tests for NYSELIFFE futures expiry.
- Added NYSELIFFE to Exchange.cs
- Removed Equity-nyseliffe-[*] and Index-nyseliffe-[*] from market-hours-database.json
- Updated Option-nyseliffe-[*] to FutureOption-nyseliffe-[*] on market-hours-database.json
- Added the supported NYSELIFFE futureoptions to symbol-properties-database.csv.

* Removed blanks after India tuple

* Added the dates for 2022 and 2023 to the market-hours-database.json.
Removed "FutureOption-nyseliffe-[]" from market-hours-database.json.
Added to FuturesOptionsSymbolMappings.cs:
{ "YG", "OYG" },
{ "ZG", "OZG" },
{ "ZI", "OZI" }

* Minor tweaks for nyseliffe FOPs

Co-authored-by: Chris Coetzee <chris@polidata.ai>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-03-04 15:24:17 -03:00
Adalyat Nazirov 18b99338fc Implement Binance US exchange (#6222)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Binance.US base changes

* biniance exchange info update

* Binance US uses same fee rates as main Binance

* binance us brokerage model tests

* use base implementation of GetBuyingPowerModel method
2022-02-22 13:32:59 -03:00
Ronit Jain f129ab1a09 Feature implement ExchangeInfoDownloader (#6213)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* add gdax exchange info downloader

* add downloader method to program]

* fetch currency description

* change definition to include headers

* use extension method to make request

* remove log from test

* replace WebRequest

* cleanup

* use relevant name

* implement IExchangeInfoDownloader for bitfinex, initial commit

* add default values

* use default market value

* use correct attribute for lotsize

* don't skip missing values

* handle multiple downloaders

* add gdax and bitfinex exchange downloader

* follow LEAN data directory structure

* update SPDB

* order tickers

* order tickers

* add exchange info downloader test template

* delete files

* update SPDB

* use currency mapping

* update bitfinex symbols

* update currency mapping

* sort result after old currency symbols are used

* use market of the respective brokerage

* no more unknown symbol

* change minimum order size value

* direct conversion possible

* update bitfinex symbols

* change user-agent

* add test for indirect conversion

* update stats
2022-02-22 13:00:54 -03:00
Martin-Molinero e316f12394 Refactor MarginRequirementEntry. Update margins (#6210)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix master

* Refactor MarginRequirementEntry to it's own file

* Update margin files

* Remove duplicate dates cases

* Update margins after fix
2022-02-15 20:24:28 -03:00
Adalyat Nazirov e8160f33d5 Binance extraction (#6193)
* extract binance brokerage

* remove binance tools from main toolbox app

* modify OrderTestParameters constructor

* fix BinanceBrokerage ref config.json

* update symbol properties

* move BinanceFeeModel tests to the right place
2022-02-15 12:05:47 -03:00
Ricardo Andrés Marino Rojas eb55311052 Feature 5157 micro futures update (#6190)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update MHDB and SPDB

Update MHDB and SPDB with micro futures

* Fix bugs

* Add more FutureExpiryFunctions

* Requested changes

* Nit changes

* Nit changes

- Fix some future values in SPDB
- Test LastFriday() method
- Apparently Micro CHF/USD Futures(MSF) refers to the micro of Swiss Franc Futures (6S). The same happens with Micro JPY/USD (MJY) Futures and Japanese Yen Futures (6J)

* Fix bugs

* Micro futures MultipleFactor

* Fix some futures MultipleFactor
2022-02-10 17:45:07 -03:00
Ricardo Andrés Marino Rojas 74321d1727 Update CME future holidays and early closes in MHDB (#6181)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* CME Futures update in MHDB

* Fix indent spaces nit errors

* Revert "Fix indent spaces nit errors"

This reverts commit 6ebb8614c0c050258253bd5dbd5c34eaaa30b30b.

* Fix tabs errors

* Update MHDB with 2022 holidays

* Fix bugs

* Remove earlyOpens
2022-02-02 19:28:21 -03:00
Martin-Molinero c4a2d6eef4 Crypto base currency fees handled correctly (#6166)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Binance fees deducted from fill quantity accordengly

- For Binance cash accounts while buying, if fees are from the base
  currency of leans virtual position, we need to deduct the fee from the
  fill quantity, else we can end with a position bigger that it actually
  is and not be able to liquidate

* Refactor solution

- Refactor solution into a more generic approach solving fees in base
  currency at the BrokerageTransactionHandler level, covering all
  brokerages that require it. Adding regression algorithm reproducing
  issue.
- Update Bitfinex and Binance fee models to correctly reflact reality

* Log fill quantity adjusment once
2022-01-28 13:02:58 -03:00
Ronit Jain f34be8e3ff Feature add India index algorithms and data (#6145)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* add data

(cherry picked from commit 814011d89e5316d150f88ffca5f48d8d5f0ea7d9)

* update market hours for index

(cherry picked from commit edac40732c120eb84d27de00594b59eebb4983f5)

* add index algorithms

(cherry picked from commit b22d27b4fa98172c435f7c26de4a3a297c49a6b7)

* update statistics

* add cash

* Add india market

* add leverage for index

* can subscribe to index

* update format

* fix wrong cash

* fix ticker names

* update data

* update ticker and stats

* update docs
2022-01-19 12:14:48 -03:00
Adalyat Nazirov 2529ba124d FTX API endpoint is cinfigurable (#6026)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* specify enpoint url of ftx api (can be FTX pr FTX.US)

* more metadata for ftxus

* tidy up code

* tests

* more tests

* fix tests

* fix us fee rates

* add account tier

* update symbol props

* typo

* typo-2

* update symbol properties

* use FTXUS fee model

* minor tweaks
2022-01-12 13:51:18 -03:00
Ronit Jain 3837c32b36 Add India market local data and regression algorithm (#6088)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Add india market data

* use local data for algo

* Add India market regression algo

* update india market data

* Update readme

* add python algo for BasicTemplateIndiaAlgorithm

* Add India data regression python algo

* data india data files

* update tickers

* Fix algo template

* remove data

* fix stats

* update stats

* remove unused data
2022-01-06 18:24:32 -03:00
Colton Sellers c11a09e08a Feature Index Option Low Resolution Support (#6075)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Activate Index and IndexOptions low res

* Add Index and IndexOption low res data

* Add IndexOption regressions

* Fix bug with IndexOption QuoteBar data not being scaled

* nit - update comment for _scaleFactor

* Update data and regressions to reflect fix in data

* Tweak regressions
2021-11-26 13:11:35 -03:00
Colton Sellers 1358bd8115 Future and FutureOption low res support (#6069)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fixes for FutureOptions support in LeanData

* Add CreateCanonicalOption() utility function for Symbol.cs

* Add aggregated Futures/FuturesOptions data to Lean

* Add FutureOptions regressions for daily/hourly data

* Allow Futures to be added with low resolution

* Add Future regressions using hour/daily data

* Nit - Python Class names

* Add reviews

* Add alias into CreateCanonicalOption
2021-11-24 09:53:21 -03:00
Martin-Molinero 5436275901 Remove comments from MHDB json file (#6065)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-11-18 20:41:02 -03:00
Martin-Molinero b8b0d18993 Continuous futures (#6034)
* Continuous Future Contracts

* Mapping approach

* Tweaks WIP

* Live mapping

* Live mapping

- Add support for live mapping, refreshing mapfiles
- Fix future expiration functions
- Adding unit tests

* Update moq test package

* Continuous futures price scaling

* Refactor price factors scaling

* Factor file related renames

* Address reviews
2021-11-15 14:44:30 -03:00
Colton Sellers dd4da7ba95 Feature Daily/Hourly Options Support (#6017)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Create generic writing for LeanDataWriter, + notes on todos

* Make Options Daily/Hourly data store by year

* Refactor Generic Write

* Permit hour and daily resolutions for options

* Refactor writer to merge when needed with other files

* Cleanup redundancies, run write tasks in parallel

* Make needed classes/vars available

* Update tests to reflect new naming convention for daily hourly options data

* Add Byte[] overloads for ZipData functions in compression

* Implemented Store() for ZipDataCacheProvider

* Have LeanDataWriter use a DataCacheProvider

* ZipDataCacheProvider cleanup

* ZipDataCacheProvider tweaks, doesn't support storing non-zips

* Test adjustments

* Update LeanDataWriter to use Write instead of SaveDailyHourly/SaveMinuteSecond

* Implement tests to verify DownloadAndSave behavior

* Nit cleanup on DownloadAndSave tests

* Fix for options daily/hourly underlying equity subscription read

* Add daily/hourly options data and regressions

* Add missing open interest for hourly

* Fix writing of OpenInterest Daily/Hourly data

* Update data

* Fix Date typo in regression

* Use daily algorithm to test delisting

* Revisions part 1

* Expand test for DataCacheProviders; refactor DiskDataCacheProvider

* nit - test adjustments

* ZipDataCacheProvider test setup refactor

* Adjust multithreaded read/write test; fixes for ZipDataCacheProvider

* Move DiskDataCacheProvider to its own file and add write test

* Remove _appendToZips; always overwrite entry or create zip

* Add mapping regression for daily options

* nit - add license to regression

* Fix Tick write case where more than one data point for a DateTime

* Fix data issue

* Address review

* Tweaks for tests

* Stop Store() early if no entry name is given
2021-11-11 20:05:31 -03:00
Martin-Molinero b8768ae274 Add FOPs price magnifier (#5979)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Add FOPs price magnifier

* Improve unit tests
2021-10-14 11:57:48 -03:00
Ricardo Andrés Marino Rojas db04b5e110 Feature 5930 price magnifier (#5977)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Update SPDB with price magnifier parameter
- Add a new column for new parameter in symbol-properties-database.csv
- Change InteractiveBrokersBrokerage.cs implementation of GetContractPriceMagnifier() method in order to get the parameter directly from the SPDB
- Add test in SymbolPropertiesDatabaseTests.cs to check if the new parameter is loading properly

* Change GetContractPriceMagnifier() method name and implementation
- Change NormalizePriceToLean() and NormalizePriceToBrokerage() methods

* Nit change in GetSymbolPriceMagnifier() method

* Remove GetSymbolPriceMagnifier() method
- Remove redundant and unnecessary parameters in NormalizePriceToLean() and NormalizePriceToBrokerage()

* Requested changes and SPDB update
- Update SPDB with minimum order size parameter for futures in cents. The minimum order size parameter was extracted from the CME group API

* Address reviews. Adding unit test for failing case

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-10-14 11:16:23 -03:00
Adalyat Nazirov 3a591c3a72 Fix Binance ExchangeInfo downloader (#5965)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* fix exchangeinfo downloader

* normalize data, update symbol properties

* min ordersize should be in base currency

* check min order size using quote currency price

* check order size in quote currency

* improve checks

* improve code; more unit tests

* more description

* fix failing test
2021-10-11 12:36:41 -03:00
Adalyat Nazirov 7cfdd1243f FTX brokerage essentials (#5963)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* FTX config boilerplate

fixup

* required

* order props

* make orger props setter public

* add market props

* brokerage model

* deny modify order directly

* update markets

* unit tests

* fix market properties

* add comments

* return security benchmark

* fix fee calculation

* dont change access modifiers

* check brokerage CanSubmitOrder using min order quantity

* fix ticker values - use lean notation

* fix fee model

* fix error message
2021-10-08 19:16:38 -03:00
Martin-Molinero 659735946a Bitfinex fees cash accounts (#5957)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Bitfinex Fee adjustment

- Fix for bitfinex fee adjustment for cash account type only happen if
  the currencies are the expected ones, else log message.

* Adjust error handling to terminate algorithm

* Add bitfinex TESTBTCTESTUSD

* Minor improvements

- Add bitfinex test symbols to the SPDB
- Make SecurityDefinitionSymbolResolver log using trace
- Minor tweak for Bitfinex DQH lock contention
2021-10-04 18:15:32 -03:00
Ricardo Andrés Marino Rojas af958f0ce4 Bug 5904 minimum order size rejected order (#5949)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* - Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type
- Change CanSubmitOrder() method implementation in GDAXBrokerageModel.cs to get the MinimumOrderSize from the security SymbolProperties directly
- Add CanSubmitOrder() method in Binance, Bitfinex and Kraken Brokerages models
- Add tests to check if CanSubmitOrder() method is working as expected in Binance, Bitfinex, Kraken and GDAX brokerages models
- Add tests to check if MinimumOrderSize parameter is loading correctly from symbol-properties-database.csv

* Revert "- Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type"

This reverts commit e0fd66a5fa5215a6fccb98b64d4e3bf709007fdf.

* Revert "Revert "- Update symbol-properties-database.csv, SymbolPropertiesDatabase.cs and SymbolProperties.cs with a new column for MinimumOrderSize parameter for Crypto type""

This reverts commit b0f37935fc334ba8ff3b8ab09b7d979e4e731526.

* Requested changes in Crypto brokerage models
- Remove unnecessary commas in symbols-properties-database.csv
- Add TestHelpers class to reuse GetSecurity() method
- Add IsValidOrderSize() method in DefaultBrokerageModel class
- Remove unnecessary classes for brokerage models test helpers

* Nit change
- Remove unnecessary attribute from DefaultBrokerageModel.cs

* Nit change

* Nit changes and Regression test
- Add Bitfinex BTCUSD hourly data for 2 days
- Add regression test BuyBTCWithLessThanOneDollarAlgorithm.cs to test that the order size is taking into account when placing or updating an order
- Nit changes

* Change CanUpdateOrder() implementation
- CanUpdateOrder() now checks if the requested quantity value is valid

* Requested Changes
- Change regression algorithm BuyBTCWithLessThanOneDollar.cs name and implementation

* Nit changes
2021-09-30 18:30:04 -03:00
Martin-Molinero d3280c5e60 Move VX margin files (#5948)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-09-28 14:21:07 -03:00
Stas Kotykhin 8026785116 Feature kraken brokerage setup (#5896)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Kraken Brokerage initial setup

* Fix naming in config

* Add license messages

* Add fees for stablecoins

* add kraken databases

* Review fixes, tests added

* Remove redundant field and using

* add CanUpdateOrder & summary

* Remove null

* Change to equal

* complete xml comments
2021-09-08 12:49:13 -03:00
Martin-Molinero 8eb023e854 Remove duplicate market hours segments (#5897)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-09-02 19:09:51 -03:00
Stefano Raggi 1d904e716c Add new Market.CFE (#5391)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
As most brokers use separate exchange codes for CBOE and CFE, we added the new Market.CFE (previously Market.CBOE was used for both exchanges). This change also avoids hacky code in symbol mappers.
2021-08-31 19:34:14 -03:00
Martin-Molinero 3410832ccf Alpha holdings state (#5770)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state

- Alpha result packet will optionally provide the algorithms portfolio
  state

* Rename

* Convert AlphaStreamsPortfolio to data source

* Improvements on AlphaStreams algorithm

* Fix regression tests

* Add unit tests for EW AS PCM and fixing bugs

* Protobuf AlphaStreamsPortfoliot staState

- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM

* Add alpha license to Organization response

* Improvements EW AS PCM respects free portfolio value

* Fixes

- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType

* Fixed and adding more regression tests

- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
  types
- Add support and regression test for algorithm alpha consumer with existing holdings

* Add AlphaStreamsOrderEvent data type
2021-08-19 13:56:18 -03:00
Gerardo Salazar cd9e451ead Adds Support for ETF Constituent Universes (#5862)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adds support for ETF constituent universes

  * Adds filtering for universe data if it doesn't match the
    universe subscription type

  * Update mapping for ALL underlying Symbols if
    `Symbol.UpdateMappedSymbol(...)` is called. Required to support
    constituent ETF universes that might have mapping events

  * Delistings of composite constituent universe Symbol will result in
    removal of universe securities.

  * Added regression algorithms for ETF constituent mappings (C#/Python),
    along with data required to run locally

  * Refactor universe delistings in SubscriptionSynchronizer -
    big thank you to @Martin-Molinero :)

* Address review: update regression algorithms and add explanatory comments

* Address review: add additional checks to delisting regression algorithms

  * Adds new regression algorithm testing the addition of a universe
    without calling AddEquity() and asserts same behavior
2021-08-17 19:54:29 -03:00
Martin-Molinero e92230a7fe Fix SubscriptionDataReader auxiliary data out of order (#5843)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix SubscriptionDataReader auxiliary data out of order

- Fix for SubscriptionDataReader emitting auxiliary data out of order
  due to sparse that and enumerator refresh logic. Adding regression
  algorithm
- Minor tweaks for unit tests failing on and off

* Fix for SDR internal subscriptions

- Fix for SubscriptionDataReader enumerator refresh for internal
  subscriptions which were ignored. Adding regression test
- Updating custom data regression algorithms affected by the issue
	- DropboxBaseDataUniverse was emitting a custom data point being end time
	- UnlinkedTraderBarIconicType was emitting a single data point
	  of the underlying SPY minute data when if should of emitted
	  all data points
2021-08-10 18:57:23 -03:00
IlshatGaripov 190fe012f1 Perform update (#5841)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-08-10 11:18:23 -03:00
Stefano Raggi 0c4e577885 BitfinexBrokerage updates (#5787)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update BinanceBrokerage to handle more than 512 symbols

* Address review

- fetch symbol weights only if required
- remove code duplication

* Add rate limiting for new connections

* Update WebSocketMessage to include the websocket instance

* Handle resubscriptions on reconnect

* Address review

* Address review

* Remove unnecessary locking

* WebSocketClientWrapper updates

- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close

* Set WebSocketClientWrapper task to LongRunning

* Add missing check in GetHistory

* Fix exceptions with Binance downloader

- closes #5794

* Update Bitfinex symbols in symbol properties database

* Update BitfinexBrokerage to use BrokerageMultiWebSocketSubscriptionManager

* Address review

* Remove unnecessary locking

* Remove old channels on resubscription
2021-07-28 11:10:12 -03:00
Ishant Jain 808ba17bff Indian Market Holidays & Early Open/Late Close Modified (#5792)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-07-26 11:14:22 -03:00
Stefano Raggi e453635458 Update Binance symbols in symbol properties database (#5759)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Update Binance symbols in symbol properties database

* Fix failing unit tests
2021-07-12 13:28:28 -03:00
Ronit Jain c8d13080a8 Indian Market support for backtesting and live deploys through Zerodha (#5710)
* added symbolData class to store exchange and token values of tickers

* support for India Market

(cherry picked from commit 5f629dcb9e72426dc7b560e782cb1fae72861574)

* removed local vscode workspace file

* support for india market instead of nse and other exchanges

* changed hardcoded india values to Market.India

* changed zerodha getcash fromm available to net

* addded edge case handoing with expections

* support for market info (nse, bse) for india market

* exchange info for EmitQuoteTick

* basic testing with c#

* c# algo

* c# testing

* live orders during market open success

* support orderproperties for exchange value for order placement

* added exceptional handling cases

* fix xml descriptions for the files

* fix zerodha-product-type in DefaultOrderProperties

* add basic template for india market

* fix orderProperites for default values

* fix for local data, python implementation not available

* fix access modifiers of SymbolData

* fix variable name case

* fix single check for exchange, call to base

* fix styling, redundent code and access modifers

* fix non static not callable from static

* fix static
2021-07-02 11:06:47 -03:00
Gerardo Salazar c211d04798 Adjust AlgoSeek Futures price multipliers to quote/process prices in USD (#5725)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust AlgoSeek Futures price multipliers to quote prices in USD

* Revert changes to AUP scaling

  * After a second review, seems that this contract isn't scaled.
    The contract multiplier differs from the NYMEX Aluminum Index's
    contract multiplier, and so does the minimum price variation.
    The minimum price fluctuation seems consistent with CME's
    quoted prices, which also claims the contract is quoted in USD.

* Fix futures contract multipliers for some scaled contracts
2021-07-01 18:46:36 -03:00
Alexandre Catarino 5361f87dd1 Updates Equity Market Data (#5576)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Updates Equity Market Data

* Updates Unit Tests

* Updates Regression Tests

In this commit we include regression tests with small changes (slightly different CAGR, Alpha, etc, but same number of trades) due to the data update.

* Updates Regression Tests 2

The following regression tests were adapt because of verification of hard-coded market data values:
- `AdjustedVolumeRegressionAlgorithm`
- `HistoryWithSymbolChangesRegressionAlgorithm`
- `OptionRenameRegressionAlgorithm`
- `RawDataRegressionAlgorithm`
- `SwitchDataModeRegressionAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `AddUniverseSelectionModelCoarseAlgorithm` 23 -> 35
- `MeanVarianceOptimizationFrameworkAlgorithm` 12 -> 14
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 298 -> 324
- `PortfolioRebalanceOnInsightChangesRegressionAlgorithm` 83 -> 86
- `ScheduledUniverseSelectionModelRegressionAlgorithm` 86 -> 90
- `SectorExposureRiskFrameworkAlgorithm` 17 -> 22
- `SetHoldingsMultipleTargetsRegressionAlgorithm` 8 -> 9
- `StandardDeviationExecutionModelRegressionAlgorithm` 196 -> 199
- `UniverseUnchangedRegressionAlgorithm` 11 -> 17
- `VolumeWeightedAveragePriceExecutionModelRegressionAlgorithm` 237 -> 238

Especial cases:
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 17
 - BLM model sensibility
- `OptionChainedAndUniverseSelectionRegressionAlgorithm`

The following regression tests have different Capacity because of different volume from lowest capacity asset, except:
- `OptionEquityCoveredCallRegressionAlgorithm` New lowest capacity asset is underlying
- `OptionEquityCoveredPutRegressionAlgorithm` New lowest capacity asset is underlying

* Revert File Update for SPWR and SPWRA

* Fix Regression Tests

Temporarily removes python regression test for `MeanVarianceOptimizationFrameworkAlgorithm` as the `MeanVarianceOptimizationPortfolioConstructionModel` for each version are yeilding different results. If we use C# version in `MeanVarianceOptimizationPortfolioConstructionModel.py`, the results match.

* Changes Optimization Method in MinimumVariancePortfolioOptimizer [Py]

Uses `trust-constr`  method.
See https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html
2021-06-17 14:04:51 -03:00
Colton Sellers dcdfc7fdb3 Update Oanda CFDs Lot Sizes and Market Hours (#5581)
Build & Test Lean / build (push) Has been cancelled
* Update lot sizes on symbols

* Update some incorrect marketHours

* Update all Oanda CFD market hours

* unify rollover to next day endtime
2021-05-21 16:16:26 -03:00
Colton Sellers a83eb7bf88 Update GDAX Brokerage MinimumOrderSizes (#5586)
Build & Test Lean / build (push) Has been cancelled
* Update GDAX Brokerage MinimumOrderSizes

* Re-add XRP for historical purposes
2021-05-21 15:44:16 -03:00
Łukasz Szymański 9d93dc1570 Fix wrong market hours for Oanda DE30EUR CFD (#5564)
Build & Test Lean / build (push) Has been cancelled
* Fix wrong market hours for Oanda DE30EUR CFD

* Update unit tests expected result

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-17 18:32:58 -03:00
Alexandre Catarino 96f42eb591 Updates Fundamental Data (#5502)
Build & Test Lean / build (push) Has been cancelled
* Updates Fundamental Data

* Updates Regression Algorithms

- `CoarseFundamentalTop3Algorithm`
  - New coarse has higher `DollarVolume` for `FB`
- `CoarseNoLookAheadBiasAlgorithm`
  - Change from `SPY` update not included in #5493
- `SectorExposureRiskFrameworkAlgorithm`
  - `HasFundamentalData` was false for `GOOG` on 20140401 and 20140402.

* Updates Unit Tests

Minor changes in expected values

* Updates Regression Tests

Should have been included in #5493:
- `OptionsExpiredContractRegression`
- `FuturesExpiredContractRegression`

* Adjusted Quantity By Lot Size in OrderTargetsByMarginImpact

`OrderTargetsByMarginImpact` didn't calculate the order value with the quantity adjusted by order size which less to values that did not reflect the actual order value.
It has a particular affect in `SectorExposureRiskFrameworkAlgorithm` where the Python and C# versions have the same orders but placed in a different sequence because of decimal/double precision.

* Reduce dictionary access x3 on OrderTargetsByMarginImpact

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-05-11 17:54:45 -03:00
Colton Sellers c082f0dda3 Address Missing StableCoin Pairs in Crypto Exchanges (#5488)
* Allow USDC in cashbook without USDC-USD pair

* Cover more stablecoin cases unique to our crypto brokers

* Add unit test

* Cleanup and expand test cases

* Add USDCEUR and USDCGBP to GDAX Symbols

* Add missing tickers to SPDB

* Cleanup test
2021-04-28 14:20:15 -03:00
Alexandre Catarino 2b0fd2e607 Updates SPY Market Data (#5493)
* Fixes Double to Decimal Cast in GetAnnualPerformance

`GetAnnualPerformance` raises an exception if the `AnnualPerformance` calculation returns a double that cannot be cast to decimal (smaller than `decimal.MinValue` or bigger than `decimal.MaxValue`).
See `ProbabilisticSharpeRatio` where the same solution was applied.

* Updates SPY Market Data

SPY is a key asset since it is the default benchmark, and any change can lead to different `Alpha` and `Beta`

* Updates Unit Tests to Reflect Data Update

* Updates Regression Tests to Reflect Data Update I

Most of the regression tests change because of updated data (market and factors) of SPY (default benchmark) while the total trade remain the same.

* Updates Regression Tests to Reflect Data Update II

The following regression tests were changed to adapt to adjusted prices and keep the total trades:
- `BacktestingBrokerageRegressionAlgorithm`
- `LimitIfTouchedRegressionAlgorithm`
- `PortfolioRebalanceOnCustomFuncRegressionAlgorithm`
- `SetAccountCurrencySecurityMarginModelRegressionAlgorithm`
- `StopLossOnOrderEventRegressionAlgorithm`
- `TimeInForceAlgorithm`

The following regression tests have more trades since adjusted prices allowed more 1-2 shares trades that were rounded down to zero before:
- `FreePortfolioValueRegressionAlgorithm` 2 -> 3
- `PortfolioRebalanceOnDateRulesRegressionAlgorithm` 291 -> 298
- `TrailingStopRiskFrameworkAlgorithm` 5 -> 7

Especial cases:
- `AutoRegressiveIntegratedMovingAverageRegressionAlgorithm` 65 -> 52
 - ARIMA model sensibility
- `BlackLittermanPortfolioOptimizationFrameworkAlgorithm` 18 -> 19
 - BLM model sensibility
- `ExtendedMarketHoursHistoryRegressionAlgorithm` 20 -> 18
 - Less minute bars before market opens

* Addresses Peer-Review

Fix `BacktestingBrokerageRegressionAlgorithm` to use `CalculateOrderQuantity` and round down `quantity` to an even number to pass a value assertion and update the expected value from 50 to 52.
The quantity calculated by `CalculateOrderQuantity` has changed from 50 to 53 because of factor file update.
2021-04-19 13:31:01 -03:00