3082 Commits

Author SHA1 Message Date
Martin-Molinero 0931ae868a Fix multiple coarse selection algorithms (#5927)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Fix for live trading multiple coarse selections. Adding unit test
  reproducing issue.
2021-09-14 20:27:09 -03:00
Gerardo Salazar 024d416ddd Adds support for getting a Lean symbol based on FIGI, CUSIP, ISIN, SEDOL (#5922)
* Adds support for getting a Lean symbol based on FIGI, CUSIP, ISIN, SEDOL

* Address review: bug fixes for mapping, improve docs, cleanup & refactor

  * Make LocalZipMapFileProvider and LocalZipFactorFileProvider only
    initialize if they haven't been initialized yet.

* Address review: makes ISIN, SEDOL, CUSIP case-insensitive

  * Cleans up map file provider in SecurityDefinitionSymbolResolver
  * Modifies some test cases to test for case-insensitivity

* Add null check for SecurityDefinitions

* Fix unit tests

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-09-10 19:16:18 -07:00
Colton Sellers 8358c394e1 Support all security types for benchmarks (#5919)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust benchmark subscription to dynamically determine best resolution

* Get types from SM

* Add red/green test set for security types
2021-09-10 16:58:31 -03:00
Colton Sellers 2001f9a181 Tradier Options History Support (#5912)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Implement options support for history requests

* Address review
2021-09-09 11:19:12 -03:00
Stas Kotykhin 8026785116 Feature kraken brokerage setup (#5896)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Kraken Brokerage initial setup

* Fix naming in config

* Add license messages

* Add fees for stablecoins

* add kraken databases

* Review fixes, tests added

* Remove redundant field and using

* add CanUpdateOrder & summary

* Remove null

* Change to equal

* complete xml comments
2021-09-08 12:49:13 -03:00
Ronit Jain 5083433f68 Inital brokerage setup commit (#5905)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-09-07 22:05:36 -03:00
Ricardo Andrés Marino Rojas da60daa60d Market profile indicator (#5891)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Create Market Profile indicator.

* - Requested changes in MarketProfile indicator made it.
- TimeProfile and VolumeProfile created and tested.
- tp_datatest.csv and vp_datatest.csv extracted from https://github.com/bfolkens/py-market-profile

* Final potential Market Profile Indicator.
All unit test made it and the indicator is passing all of them
Test cases made it with python library from GH issue

* Code styling request changes

* Minor suggestions

* Minor renaming tweaks

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-09-03 20:18:44 -03:00
Martin-Molinero b251cffc2f Alpha Streams improvements (#5887)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Few improvements

- Alpha Streams algorithm will add target securities right away
- If algorithm is warming up PortfolioTargetCollection will emit not
  insight
- Fix local disk factor file provider to use local cache instead of data
  folder
- EWAS PCM will not emit targets for securities which have not been
  added by the algorithm yet, to avoid runtime exception

* Fix GetLastKnownPrice default order adding more unit tests
2021-09-01 13:12:26 -03:00
Stefano Raggi 1d904e716c Add new Market.CFE (#5391)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
As most brokers use separate exchange codes for CBOE and CFE, we added the new Market.CFE (previously Market.CBOE was used for both exchanges). This change also avoids hacky code in symbol mappers.
2021-08-31 19:34:14 -03:00
Martin-Molinero 933e5ce6ca Alpha Streams improvements (#5876)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Alpha Streams improvements

- Warmup added securities so we can trigger an order right away
- Fix bug where AddSecurity returned a security which was not the one
  being used
- Fix bug where EWAS PCM would remove from a dictionary while iterating
  over it

* Fix unit tests

* Add QCAlgorithm.GetLastKnownPrices API

- Add new GetLastKnownPrices that will return the last known data point
  for all subscribed data types.
- Fix for MHDB GetDataTimeZone which was using an invalid entry key
  format for custom data.
- Fix for using GetLastKnownPrice/s to seed a security during creation,
  when it's not added in the Algorithm.Securities collection

* Adding more unit tests for GetLastKnownPrice

* Address reviews

- Refactor GetLastKnownPrice/s to perform a single history request for
  all data types
2021-08-27 13:50:45 -03:00
Martin-Molinero 9acf68ad19 Bump pythonNet to 2.0.6 (#5881)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-08-26 18:01:05 -03:00
Gerardo Salazar a4d49c05ca Adds ETF(...) to UniverseDefinitions (#5873)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adds ETF(...) to UniverseDefinitions

  * Adds ETF constituents universe framework regression algorithm
    for C#/Python

* Address review: adds test cases for ticker/Symbol ETF universe additions

  * Fixes bug where null Market would result in null dereference exception

* Address review: add missing Index tests

* Address review: don't hardcode market when creating constituent universe

  * Uses Brokerage Model's default markets collection to determine
    the market for the given security type

* Address review: restore QC500 and DollarVolume.Top(...)

  * Restores algorithms related to both helper universe
    definition methods

* Address review: remove copy to output directory for python algos

* Add example algorithms for ETF constituent universes using custom RSI alpha model

* Address review: adjust algorithm to use cache + algo RSI & clean up code

* Address review: make ETF Constituent RSI Alpha Model algo a regression test

* Address review: increase trade count and remove single trade logic
2021-08-25 11:22:31 -03:00
Martin-Molinero bf28a1d313 Add basic template Atreyu algorithm (#5872)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add basic template Atreyu algorithm

- Add C# and Python basic template atreyu algorithm. Show casing how to
  specify exchange to execute in different ways.
- Adjust trading API to allow specifying order properties to use

* Lean Exchange improvements

- Rename PrimaryExchange to Exchange
- OrderPropeties will use Exchange enum instead of string
- Adding BSE exchange value

* Regression tests fixes
2021-08-23 18:24:44 -03:00
IlshatGaripov eccca2c029 Fix for Trailing Stop Risk Management Model (#5791)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* To fix the referenced issue

* Fixing up TrailingStopRiskManagementModel

* Fix-ups to address review

* Impl. tests + relevant improvements

* Minor logic improvement at first dictionary update

* Regression test small fix

* Adds explicit 'D' suffix for numbers in test case double arr

* Use integer values in test cases

* Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming

* Revert "Removes failing testcases in MaximumDrawdownPercentPerSecurityTests (!) & renaming"

This reverts commit f9cd279f8cc5e9e1ede5b6f1eae5f4266b7dd295.

* Fix up for failing test cases
2021-08-20 15:14:36 -03:00
Martin-Molinero 3410832ccf Alpha holdings state (#5770)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Alpha holdings state

- Alpha result packet will optionally provide the algorithms portfolio
  state

* Rename

* Convert AlphaStreamsPortfolio to data source

* Improvements on AlphaStreams algorithm

* Fix regression tests

* Add unit tests for EW AS PCM and fixing bugs

* Protobuf AlphaStreamsPortfoliot staState

- Protobuf AlphaStreamsPortfolioState. Adding unit tests
- Add variable TPV tests for EW ASPCM

* Add alpha license to Organization response

* Improvements EW AS PCM respects free portfolio value

* Fixes

- Update tests expected statistics results affected by MHDB custom data timezone fix
- Fix for Extensions.IsCustomDataType

* Fixed and adding more regression tests

- Adding support and regression test with alpha consumer with different account currency
- Adding support and regression test of a universe adding custom data
  types
- Add support and regression test for algorithm alpha consumer with existing holdings

* Add AlphaStreamsOrderEvent data type
2021-08-19 13:56:18 -03:00
Stefano Raggi d573a0f6c7 BrokerageTransactionHandler - handle OrderStatus.New in order update/cancel (#5867)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-08-19 12:01:38 -03:00
Martin-Molinero c3543f844a Add paper trading for Binance (#5859)
- Adding paper trading support for binance
2021-08-18 10:57:45 -03:00
Gerardo Salazar cd9e451ead Adds Support for ETF Constituent Universes (#5862)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Adds support for ETF constituent universes

  * Adds filtering for universe data if it doesn't match the
    universe subscription type

  * Update mapping for ALL underlying Symbols if
    `Symbol.UpdateMappedSymbol(...)` is called. Required to support
    constituent ETF universes that might have mapping events

  * Delistings of composite constituent universe Symbol will result in
    removal of universe securities.

  * Added regression algorithms for ETF constituent mappings (C#/Python),
    along with data required to run locally

  * Refactor universe delistings in SubscriptionSynchronizer -
    big thank you to @Martin-Molinero :)

* Address review: update regression algorithms and add explanatory comments

* Address review: add additional checks to delisting regression algorithms

  * Adds new regression algorithm testing the addition of a universe
    without calling AddEquity() and asserts same behavior
2021-08-17 19:54:29 -03:00
Alexandre Catarino 6546647e08 Get Best Effort Price For Fill Price (#5855)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
`EquityFillModel` will use Trade Tick or TradeBar data if there is no L1 data available leading to filling to stale price.
2021-08-12 10:16:20 -03:00
Martin-Molinero 46829f0f01 Dependency clean up (#5834)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-08-11 12:39:00 -03:00
Colton Sellers e722e72aaa Pandas Mapper Recursion Bug Fix (#5848)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Adjust wrapper

* Adjustments

* Add red -> green unit tests

* Address review
2021-08-10 20:32:09 -03:00
Martin-Molinero e92230a7fe Fix SubscriptionDataReader auxiliary data out of order (#5843)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix SubscriptionDataReader auxiliary data out of order

- Fix for SubscriptionDataReader emitting auxiliary data out of order
  due to sparse that and enumerator refresh logic. Adding regression
  algorithm
- Minor tweaks for unit tests failing on and off

* Fix for SDR internal subscriptions

- Fix for SubscriptionDataReader enumerator refresh for internal
  subscriptions which were ignored. Adding regression test
- Updating custom data regression algorithms affected by the issue
	- DropboxBaseDataUniverse was emitting a custom data point being end time
	- UnlinkedTraderBarIconicType was emitting a single data point
	  of the underlying SPY minute data when if should of emitted
	  all data points
2021-08-10 18:57:23 -03:00
Ronit Jain 7881aeb3a5 Websocket-refactor for websocket message data as text and binary (#5833)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* websocket-refactor for websocket message  as text and binary

Initial commit

websocket support for text and binary messageData

code style fixes

test fix

* delete un-used file

* zerodha websocket textMessage parse

* messageData class abscration

* Empty

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-08-10 17:40:07 -03:00
Stefano Raggi c30bd32fb0 Fix IB history requests (#5832)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Fix IB history requests

- Fix time zone bug causing incomplete results
- Use RegularTradingHours flag only for equities
- Second resolution data is now requested in 30 minute chunks (instead of 1 minute):
https://interactivebrokers.github.io/tws-api/historical_limitations.html

* Filter history result respecting market hours and IncludeExtendedMarketHours flag

* Add history unit tests
2021-08-09 16:57:12 -03:00
Martin-Molinero e720147e50 Algorithm Manager will break when algorithm is deleted (#5840)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2021-08-09 15:42:13 -03:00
Martin-Molinero b77f0122b2 Improve python exception parsing (#5831)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Improve python exception parsing

- Improve python exception parsing adding support for line shift. Adding
  unit tests

* PythonException revert change

* Centralized and normalize algorithm runtime handling

* Adding support for C# line and file exception report
2021-08-06 20:11:54 -03:00
Colton Sellers e2ae173d99 Feature Python Library: Ray (#5824)
* Add Ray lib to both foundation images

* Update some version failing tests
2021-08-05 17:55:51 -03:00
Alexandre Catarino 9b7af08b3e Fixes Market On Open Fill of Equity Fill Model (#5679)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fixes Market On Open Fill of Equity Fill Model

Only use trade data (Tick with Trade type or TradeBar) to get the open price, since MOO is filled with the opening action price. Ensure that this method doesn't use trade data from before the market opens for high-resolution data case.

Fix unit tests to show that the new implementation only fills with trade data from the current open market.

Change regression tests to reflect the bug fix.
In the `ExtendedMarketHoursHistoryRegressionAlgorithm`, MOO was filled with extended market hours.

* Fix Bug for Tick Resolution Case

For tick susbcription, the tick with the open price information is the the first valid (non-zero) tick of trade type from an open market.
Addresses peer-review by moving the if-condition for data belonging to the open market where the subscriscribed types are checked.

* Fix Bug for Low Resolution Edge Case

For the edge case where the order is placed after the trade bar is open, for example, order places at 1 pm with daily-resolution data. The fill model will not use the open of the bar that will close at midnight, since this value is prior to the order.

Adds unit test.

Change regression tests to reflect the bug fix.
In the `RegressionAlgorithm`, MOO was filled with open prior to the order. The algorithm now has one order less, since the last MOO would need to wait another day to be filled.

* Implements SaleCondition and Exchange Check For Tick

- Adds additional unit tests for MOO

* Fixes Regression Test in DataConsolidatorPythonWrapperTests

* Addresses Peer-Review

- Adds new unit test cases.
2021-08-05 12:53:04 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
2021-08-02 19:24:45 -03:00
Martin-Molinero 3cf9cc7d41 Update RestSharp to 106.12.0t (#5810)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-30 17:45:59 -03:00
Colton Sellers a9b634b096 Pandas Dataframe indexing w/ list (#5813)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Wrap df.__getitem__ to map all keys before indexing

* Add reproducing test case
2021-07-30 16:34:03 -03:00
Colton Sellers d8a86f182f Pandas Remapper Refactor (#5808)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Refactor pandas mapper to support newer versions

* Clean up remapper

* Finalize deprecated tests and adjust those that should apply to new pandas

* Move mapper to its own file

* Add supporting Py tests and setup instructions

* Add license
2021-07-29 20:10:00 -03:00
Jovad Uribe 9888081cc0 Relative Daily Volume Indicator (#5747)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Relative Volume Indicator

* Requested changes and renamed indicator

* Made changes, included Resolution

* Added requested changes

* Update

* Update RelativeDailyVolumeTests.cs

* Update RelativeDailyVolume.cs

* Fixed Test Case File

* Update RelativeDailyVolume.cs

* Fixed IsReady flag
2021-07-28 18:02:34 -03:00
Stefano Raggi 0c4e577885 BitfinexBrokerage updates (#5787)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Update BinanceBrokerage to handle more than 512 symbols

* Address review

- fetch symbol weights only if required
- remove code duplication

* Add rate limiting for new connections

* Update WebSocketMessage to include the websocket instance

* Handle resubscriptions on reconnect

* Address review

* Address review

* Remove unnecessary locking

* WebSocketClientWrapper updates

- remove allocation of receive buffer on each message
- add missing lock in Close method
- log message data when message type is Close
- fix race condition after unexpected websocket close

* Set WebSocketClientWrapper task to LongRunning

* Add missing check in GetHistory

* Fix exceptions with Binance downloader

- closes #5794

* Update Bitfinex symbols in symbol properties database

* Update BitfinexBrokerage to use BrokerageMultiWebSocketSubscriptionManager

* Address review

* Remove unnecessary locking

* Remove old channels on resubscription
2021-07-28 11:10:12 -03:00
Marco Grassi e823dfdfb7 fix a typo, and remove all unnecessary semicolumns in Python (#5795)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-27 16:05:13 -03:00
Martin-Molinero 4469d18eb6 Set timezone for DateRules (#5796)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- If algorithm timeZone is changed Update DateRules timezone. Adding
  unit test
2021-07-27 15:34:11 -03:00
Ronit Jain 00f68b951f Zerodha - GetHistory DateTime Parse Correction (#5788)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* logging and kite connection retry

(cherry picked from commit 9845a2d79e86c338093cadcae7f41b29d897c78c)

* changed thread.sleep time for kite connection retries

* websresponse retry on error - coding style change

* datetome parse to convert time in utc

* indentation code-style fix

* zerodhaDownloader support for dataTimeZone

* removed code redundancy for single conversion without hardcoded TZ

* removed hardcode dataTimeZone and code refactor

* returns exchange TZ to lean and data TZ to local disk from downloader

* Kite historical candle timestamp TZ conversion

* added unit test for kite historical candle

* access modifier correection
2021-07-23 18:34:48 -03:00
Martin-Molinero 2cb239a330 Fix price left over split cash adjustment (#5780)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
- After a split if there are left over shares we model them getting sold
  using the new price. Adding unit and regression test
2021-07-19 13:18:10 -03:00
Martin-Molinero cb326788b3 Filter out small orders based on Setting (#5776)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Filter out small orders based on Setting

- BuyingPowerModel will filter out small orders based on algorithm
  setting, a % of PTV, instead of hard coded 1 share value. Addin unit
  and regression tests
- Updating regression algorithms to use new setting, reduce order trades

* Update regression algorithms
2021-07-19 13:17:51 -03:00
Martin-Molinero f75e81bd7d Add null check for API data folder (#5774)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Api.Initialize will not throw for null data folder. Adding unit test
2021-07-16 18:27:42 -03:00
Martin-Molinero 9cf4075957 Send invalid orderEvent for non shortable order (#5772)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- TransactionHandler will see invalid OrderEvent for an order which
  tries to short an non shortable asset. Adding unit test, updating
  regression algorithm.
- Fix Api data price path normalization, found while testing with
shortable provider. Adding unit tests
2021-07-15 21:10:46 -03:00
Colton Sellers 8d11bbf0ba Update to PythonNet 2.0.5 (#5764)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-14 11:01:57 -03:00
Stefano Raggi e453635458 Update Binance symbols in symbol properties database (#5759)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Update Binance symbols in symbol properties database

* Fix failing unit tests
2021-07-12 13:28:28 -03:00
Martin-Molinero d84ad9ebfc Reduce research image size (#5758)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Reduce research image size
- Increate BrokerageConcurrentMessageHandler unit test timeout
2021-07-12 09:58:11 -03:00
Martin-Molinero f9adc6b136 Rename data permissions (#5753)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-09 22:59:14 -03:00
Jovad Uribe e5f62fa4dd Feature Augen Price Spike Indicator (#5713)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Added Augen Price Spike Files

* AugenPriceSpike Update

Adjusted indicator calculations and added test data from Trading View

* Requested Changes

* Minor changes
2021-07-09 20:07:15 -03:00
Colton Sellers 3430b46e6a BuyingPowerModel GetMaximumOrder Adjustments (#5756)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add reproducing unit test

* Apply fix to force adjustment if needed

* Adjust fee to correctly replicate original issue

* Adjust solution

* Update unit test to reflect the way this function is used

* nit - rename test

* Cleanup error message to improve reproducibility

* nit - Drop unneeded var

* Add in adjustment test cases as well, fixed
2021-07-09 19:37:48 -03:00
Colton Sellers 00ac4f33af Implement Python overload of Notify.Web for PyObject Headers (#5749)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Implement Python overload of Notify.Web for PyObject Headers

* Address review + add unit tests
2021-07-08 21:07:19 -03:00
Colton Sellers 0d752abae6 Feature Telegram Notification (#5535)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Lay telegram notification groundwork

* nit comments

* Implement in JSON converter, and add roundtrip test

* Update comment docs

* Implement configurable token in NotificationTelegram

* Rename to ID, due to @ handles not usable

* Address review + nits

* Nit comment fix
2021-07-07 21:06:45 -03:00
Martin-Molinero 49d6cb0021 QuantBook fundamental history request (#5743)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Allow requiesting the entire FineFundamental object for a QuantBook
  history request. Adding unit test
2021-07-07 12:02:01 -03:00