291 Commits

Author SHA1 Message Date
Gerardo Salazar 883d354a98 Adds DC future contract symbol mapping for IB (#4905)
* Fixes SPDB entry for Class III/IV Milk
2020-11-02 10:57:57 -03:00
Michael Handschuh b1b8da1e17 Fixes Market Simulated Automatic Option Assignment (#4853)
* Add underlying holdings to regression result handler details log

When debugging option exercise/assignment issues it's useful to see the
underlying holdings at the time the option contract fill event is processed.

Also adds the full symbol string to the top of the order event section.
The Symbol.Value was being logged via OrderEvent.ToString(), but it wasn't
the full SecurityIdentifier - by including the full SID string it makes it
easier to correlate fills over symbol rename boundaries.

* Fix automatic option assignment from market simulation

During the recent OptionExerciseOrder.Quantity refactor, this case was missed.
Additionally, it was realized that there were no regression tests covering the
automatic assignment via the market conditions simulation. This change introduces
a regression algorithm that covers the automatic assignment of put/call options.

* Update BasicOptionAssignmentSimulation._rand to be non-static

If this value is static then we reuse the same Random instance for ALL regression
tests, thereby defeating the purpose of using a well known seed number. This means
we get different results based on the order execution of preceding algorithms.
By making this an instance variable each algorithm will start with the same seed
value, ensuring consistent runs between regression tests, either run as a suite or
running a single algorithm in isolation.
2020-10-13 19:39:25 -03:00
Martin-Molinero 6ef2ead929 Do not update price scale for fillforward data & IsFillForward flag fix (#4836)
* Do not update price scale for fillforward data

- Do no update price scale for fill forward data. FillForward data
  should keep using the prev scale for which it was created. Adding unit tests
- When cloning do not lose IsFillForward flag state, affects
QuoteBars/Ticks, does not affect TradeBars since they perform a memberwise clone.
Adding unit tests

* Auxiliaries shouldn't really affect on applied price factor scale.

Despite we can receeive FillForward'ed data points, corresponding
Auxiliaries for them are not FillForward so we do meet the condition
and then refresh price factor. As a result all futher FF data points are scaled too.

* Regression algorithm to check that FillForward'ed data points arrived with last real price factor

* Add trade for regression algorithm

- Minot tweaks and adding trade for new regression algorithm.
- Updating AddOptionContractExpiresRegressionAlgorithm because it is
  using the symbol for which new data was added.

Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
2020-10-09 18:09:30 -03:00
Colton Sellers 3966c0e91f Market Hours Database Adjustment (#4818)
* Fix Entries

* Fix options dates
2020-10-06 20:45:03 -03:00
Stefano Raggi 934128cfa0 Binance Brokerage implementation (#4688)
* Binance Brokerage skeleton

* Market hours

* Implement Symbol Mapper

- known symbols available on /api/v1/exchangeInfo
- fiat currencies are pegged

* Implement GetCashBalance

* Implement GetAccountHoldings

- there are no pre-existing currency swaps
- cash balances are pulled and stored in the cashbook

* Implement GetOpenOrders

* Manage orders: PlaceOrder

* Manage orders: UpdateOrder

Update operation is not supported

* Manage orders: CancelOrder

* Messaging: order book

* Messaging: trades

* Messaging: combine streams

- connect to fake /ws/open channel on init
- case by channel name, but not event type

* Messaging: order depth updates

- ticker symbol is not enough as it pushes updates only once a second, this would be a very incomplete data stream
- fetch ticker snapshot if lastUpdateId == 0
- follow Binance instructions for keeping local orderbook fresh

* Messaging: user data streaming

- Request userDataStream endpoint to get listenKey
- keep listenkey alive
- handle order close event
- handle order fill event

* DataDownloader: get history

- we can aggregate minute candles for higher resolutions

* fix data stream

* Tests: FeeModel tests

* Tests: base brokerage tests

* Tests: download history

* Tests: symbol mapper

* Support StopLimit andd StopMarket orders

* StopMarket orders disabled

Take profit and Stop loss orders are not supported for any symbols (tested with BTCUSDT, ETHUSDT)

* Tests: StopLimit order

* Tests: crypto parsing

* Reissue user data listen key

* comment custom currency limitation

* rework websocket connections

* implement delayed subscription

* adapt ignore message

* add license banner

* use better suited exception type

* avoid message double parsing

* support custom fee values

* extract BinanceApiClient to manage the request/response between lean and binance

* use api events to invoke brokerage events

* do not allow to terminate session if it wasn't allocated.

* update binance exchange info

* tool to add or update binance exchange info

* ExchangeInfo basic test

* Rebase + Resharp

* Binance brokerage updates

- Fix sign bug in sell order fills
- Fix bug in GetHistory
- Remove duplicate symbol from symbol properties db

* Remove unused code

* Revert removal of account currency check

* Update symbols properties database

* Address review

* Address review

- Upgrade API endpoints from v1 to v3
- Updated sub/unsub for new subscription manager
- Subscribe best bid/ask quotes instead of full order book
- Added handling of websocket error messages
- Cleanup + refactor

* Update symbol properties database

* Remove list from symbol mapper

* Fix symbol mapper tests

* Address review

- Fix resubscribe after reconnect
- Fix quote tick edge case

* Fix EnsureCurrencyDataFeed for non-tradeable currencies

* Fix check in EnsureCurrencyDataFeed

* Reuse base class subscribe on reconnect

Co-authored-by: Adalyat Nazirov <aenazirov@gmail.com>
Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2020-09-28 15:57:10 -03:00
Adalyat Nazirov 1e3a1e3c43 Historical data requests start & time fix (#4733)
* regression tests

* fix: apply the same time convertion to history request time as for data time

* ver2

* fixup

* unit tests

* do not need this conversion because RoundDownInTimeZone returns in proper TZ

* comment

* requested changes

* refactoring

* more refactoring

* fix existing test: should return Sunday if open

* more symbols

* fix existing tests: submit new btcusd data

* fix

* add Cfd symbol
2020-09-24 14:51:35 -03:00
Martin-Molinero 9cb2452025 Oanda default Forex Market (#4706)
* Oanda default forex Market

- Use Oanda as default forex Market since it has more pairs.
- Remove FXCM data add Oanda equivalente data.
- Update unit and regression tests

* Address reviews

- Revert FXCM data removal
- Remove unrequired commented code

* Fix rebase
2020-09-14 16:43:23 -03:00
Stefano Raggi d8dc03fadc Reuse symbol properties database for currency conversions (#4710)
* Remove invalid symbols from symbol properties db

* Add SymbolPropertiesDatabase.GetSymbolPropertiesList

* Remove symbol list in BitfinexSymbolMapper

* In EnsureCurrencyDataFeed fetch symbols from symbol properties database

* Remove symbol list in OandaSymbolMapper

* Remove unused code

* Address review

- Remove StringComparer.OrdinalIgnoreCase usage
- Rename KnownSymbolStrings to KnownTickers
2020-09-14 15:12:17 -03:00
Juan José D'Ambrosio 6c051cd816 Update symbol property database with latest crypto pairs (#4612) 2020-08-06 10:23:20 -03:00
Michael Sena c7f0f6f769 Move the margin files for SGX and HKFE into the correct folder 2020-07-28 15:15:29 +10:00
Anand Vishnu ec6abfacc8 Add future contracts for NK, IN, TW and HSI 2020-06-29 17:35:40 +01:00
Gerardo Salazar 8d0d27253a Address review: Cleans up percentage margins and adds missing margins 2020-06-18 10:20:15 -07:00
Gerardo Salazar 3baf8c3959 Address review: Removes changing percentage margins
* Only the first margin of a percentage margin will be
    used from now onwards
2020-06-15 16:00:38 -07:00
Gerardo Salazar 81f2c43d0a Updates margins using new contract selection strategy
* Fixes previously incorrect margins
2020-06-08 17:55:12 -07:00
Gerardo Salazar b638c450cb Removes bad margin files and fixes unit tests 2020-06-03 16:48:18 -07:00
Gerardo Salazar b0f787abd8 Address review - removes duplicates in margins and skip non-USD margins 2020-06-03 15:20:43 -07:00
JJD a3356904b0 Update futures margins 2020-06-02 17:39:12 -03:00
Martin Molinero 63bf4e72a0 Revert "Revert "Fix market open missing FillForward bar""
This reverts commit 1ac8146eae.
2020-05-08 12:46:02 -03:00
Martin-Molinero 1ac8146eae Revert "Fix market open missing FillForward bar" 2020-05-08 11:38:59 -03:00
JJD 7a30346349 Add generic entry for CBOE
tidy up JSON
2020-04-29 18:23:34 -03:00
JJD a2304bed33 Fix invalid JSON + generic futures exchanges entries 2020-04-29 18:09:58 -03:00
JJD f4b814f98c Revert removing equity early closes in MHDB 2020-04-29 20:21:57 +00:00
Martin-Molinero 5625acbd6f Merge pull request #4328 from QuantConnect/refactor-future-symbol-market
Refactor Future Market
2020-04-28 19:40:04 -03:00
Martin Molinero 1c3c2e810e Address review
- Minor fixes
- Adding regression algorithm adding futures with the same ticker
  different market
2020-04-28 16:29:54 -03:00
JJD 97239777ae Include generic by exchange in MHDB 2020-04-28 16:29:17 -03:00
JJD 0e5f2b7e07 Add future generic for all exchanges in MHDB 2020-04-28 16:29:17 -03:00
JJD 800e283de7 Add generic entries for all future's exchanges 2020-04-28 16:29:16 -03:00
JJD 80165b67b9 Fix Sugar futures entry 2020-04-28 16:29:16 -03:00
JJD 5a9886bcdb Order symbols propeties by exchange - symbol 2020-04-28 16:29:16 -03:00
JJD 2ef5129e9d Move margins to its respective exchanges folders 2020-04-28 16:29:16 -03:00
Martin Molinero 149d30f020 Fixes and Improvements
- Removing spaces in MHDB.json file reducing size
- Update tests using wrong future market
2020-04-28 16:29:15 -03:00
JJD 4ab5d8fc34 Update future data sample 2020-04-28 16:29:15 -03:00
Juan José D'Ambrosio ebf7084d9a Fix issue removing NTY
NTY was not found in symbol properties, nor in CME Group product list.
2020-04-28 16:29:14 -03:00
Juan José D'Ambrosio ffe19d048f Update MHDB
with the specific exchange for each future
2020-04-28 16:29:14 -03:00
Juan José D'Ambrosio 5309a56372 Remove undefined future AC
Remove comma from EXR description.
2020-04-28 16:29:14 -03:00
JJD 60249b783d Restore deleted comment 2020-04-28 16:29:14 -03:00
JJD 016c101fe0 Update symbol propeties with exchanges and description for futures. 2020-04-28 16:29:14 -03:00
Martin Molinero b1d6d21d07 Refactor Future symbol Market
- Replacing usage of Market.USA as market for Future Symbols. Will now
  check SymbolPropertiesDatabase
2020-04-28 16:29:14 -03:00
Martin Molinero f78142fe32 Fix rebase after data update 2020-04-28 15:53:01 -03:00
Martin Molinero 075e82629a Add more unit tests
- Add more fill forward unit test including a weekend using different
combinations of resolutions
- Add required AIG second data file
- Avoid list in most common FF case
2020-04-28 15:23:07 -03:00
Martin Molinero de05f15a12 Address review
- Adding live trading unit test
2020-04-24 19:04:41 -03:00
Martin Molinero 499248fe12 Revert "Revert "Adding Support for Equity L1 Quote Data""
This reverts commit 8cd8d206ca.
2020-04-06 10:32:59 -03:00
Jared 8cd8d206ca Revert "Adding Support for Equity L1 Quote Data" 2020-03-11 19:16:05 -07:00
Juan José D'Ambrosio d4bec86df0 Update daily and hourly sample files 2020-03-11 20:22:13 +00:00
Martin Molinero d5d00f8472 Update tests
- Update regression tests
- Add missing data
- Revert Messaging.cs change
- Update unit tests
2020-03-11 14:34:16 -03:00
D 3b25ea4240 Update test after adding the new sample data 2020-03-11 14:34:16 -03:00
D 336caf9856 Update sample data 2020-03-11 14:34:16 -03:00
D 222105f60d Update sample data 2020-03-11 14:34:14 -03:00
Juan José D'Ambrosio 1269966e51 Rebase and squash commits
Update test to the new crypto and equity subscriptions  rules:
- Only consolidates trades
- Low resolution data are only trades.
Update and fix tests

Add missing minute sample files

Update Regression algorithms statistics
2020-03-11 14:34:13 -03:00
Juan José D'Ambrosio 95336d69c5 Add tick sample data 2020-03-11 14:34:13 -03:00