10665 Commits

Author SHA1 Message Date
Martin-Molinero 31d101253c LeanDataReader future/option data (#5117)
* LeanDataReader future option data

- The LeanDataReader will be able to read an entire zip of future and
  option data, currently it's just returning the first entry in the zip
  file. Adding unit tests

* Fix added unit test data path
10282
2021-01-08 11:13:05 -03:00
Aaron Janeiro Stone 7531046f14 Feature 4999 -- Adds RVI (#5041)
* adds RVI

* RVI refactor and tests
TODO: fix ComparesWithExternalDataRviSignal

* Fixed to allow all tests to pass.

* Addresses requested changes

* Addresses requested changes

* Addresses reviewer recommendations
10281
2021-01-08 11:11:00 -03:00
Martin-Molinero 6b99266669 Update IBAutomator increased init timeout (#5139)
- Updating IBAutomator to 1.0.40 which has an increased initialization
  timeout
10278
2021-01-07 19:13:07 -03:00
Adalyat Nazirov 8d48996948 fix covesion to string: decimal are represented using comma decimal separator for non-US English cultures, but expected dot (#5138) 10273 2021-01-07 17:35:51 -03:00
Martin-Molinero 7f381cb135 Universe dispose and removal (#5134)
* Universe dispose and removal

- Fix bug where in some cases the disposed universe would not get removed
from UniverseManager not allowing new universes to get added correctly,
using the same symbol. Adding regression test
- Fix bug where in some cases selected symbols would not get removed
  when the parent universe would get disposed of.
- Minor normalization on UniverseSelection call

* Address review

- Add status plots for the added regression test

* Adding regression algorithm reproducing issue 3914
10266
2021-01-07 12:03:17 -03:00
Riley Shea 2b7372783a Fix column headings in Data/equity/readme.md (#5125)
* Swapped incorrect column heading postions
10263
2021-01-07 11:01:51 -03:00
Adalyat Nazirov 54fce3f666 FillForward enumerator does not loop infinitely if fillforward resolution is different from data resolution (#5118)
* change filename template. add fillforward resolution suffix

* replicate GH issue 5116 on master

it's easy to reproduce on FXCM market by using data resolution different from fillforward resolution.
in this cases daily vs hour/minute were added

* change priorities of Time & EndTime values.

it's necessary to calculate EndTime  properly, and then we can align Time to it.
potential end time should be also calculated using ptoper TZ. Because of we store open hours without TZ movement TZ in market-hours it's necessary to reapply TZ

fix tests

* Miss 2AM bar on Sunday of DST; FF 2AM bar on Sunday ST

* use UTC TimeZone as baseline during comparison
10259
2021-01-06 20:18:08 -03:00
Colton Sellers 12db2261dc Allow Saving of Octet Stream in API (#5130)
* Allow downloading of octet stream

* Refactor solution

* Match lowercase ContentType
10258
2021-01-06 19:46:09 -03:00
Alexandre Catarino c6d4888e70 Refactors Market Fill of Equity Fill Model (#5114)
* Refactors Market Fill Model

Create `GetBidPrice` and `GetAskPrice` methods to request the most suitable price to fill market orders.

Fix unit tests to show that new implementation prevents using non-market data to fill the order.

* Addresses Peer Review …

Modifies EquityTickQuoteAdjustedModeRegressionAlgorithm to reflect changes in the market fill method. The previous implementation only looks for the last tick when it should look for the last tick of quote type, so current implementation is an improvement.
10249
2021-01-06 12:35:33 -03:00
Martin-Molinero 33b58b0dbc Fixes for IB malformed symbols (#5127)
* Fixes for IB malformed symbols

- Add handling for IB future malformed symbols. Adding unit test
- Fix IB option malformed symbols which had more whitespaces than
  expected. Adding unit test

* Address review
10248
2021-01-06 11:59:22 -03:00
Martin-Molinero 56cb9fbec5 IB will wait for connection response (#5124)
- IB.Connect() will wait for connection response before starting reader
10246
2021-01-05 19:44:04 -03:00
Jared 194e6f6b58 Update benchmark algorithm to include disabling (#5115)
* Update benchmark algorithm to include disabling

* Update C# benchmark example algo

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10236
2021-01-04 18:11:18 -03:00
Martin-Molinero 612c4f8b66 Fix BTC future price multiplier (#5113) 10234 2021-01-04 15:44:38 -03:00
Martin-Molinero d3b9f8bc7a Protobuf OpenInterest (#5107)
- Add OpenInterest protobuf support. Adding unit test
- Use composer for future and option chain provider
10230
2021-01-04 10:40:35 -03:00
Gerardo Salazar c4a1c245a7 Adds data parsing for malformed option contracts for InteractiveBrokers (#5101)
* Adds data parsing for malformed option contracts in IBBrokerage

  * Some contracts come in malformed from IB when downloading our
    account holdings. We attempt to detect whenever this happens with
    our options Symbols and try to recover the contract.

* Addresses review and fixes bug

  * Uses contract currency for the new contract created from the
    malformed contract.

  * Bug fix: Sets ibSymbol to new Symbol of the contract once the
    malformed contract has been parsed.

  * Adds support for Futures and FOPs

  * Refactoring code and cleanup

* Addresses review: code cleanup and refactoring

* Addresses review: remove redundant logging of contract in log statement
10222
2020-12-30 16:28:33 -08:00
Martin-Molinero 050dadc21f Increase low resolution cache size (#5105)
- Increasing low resolution cache size so it can hold QC500
10220
2020-12-30 20:59:36 -03:00
Colton Sellers 7b0ea0982e Update ExpectedStatistics (#5094) 10206 2020-12-29 12:04:36 -03:00
Adalyat Nazirov f8cf923d7b upload SHY quote bars (#5097) 10205 2020-12-29 11:58:13 -03:00
Colton Sellers 14b44bbe17 Reduce Travis Logs v2 (#5049)
* Silence DotNet in building stubs

* Reapply silence to stub publishing

* Have QuantBook store and restore initial LogHandler

* Add assembly action to maintain LogHandler

* Allow AlgorithmRunner to use ConsoleLogHandler

* Use MaintainLogHandlerAttribute to provide current test logger

* Refactor LogHandler creation
10201
2020-12-28 21:10:21 -03:00
Colton Sellers 2ed221b623 Linux PyCharm Docker Debugger Fix (#5092)
* Map host.docker.internal to host ip

* Typo fix

* Only add host in Linux environment
10200
2020-12-28 21:08:34 -03:00
Gerardo Salazar 3f1c33e1c5 Fixes DockerfileLeanFoundation build and adds/updates new packages (#5036)
* Fixes DockerfileLeanFoundation build

  * Updates DockerfileLeanFoundation packages and adds new packages
  * apt-get performance improvements

* Moves packages to CDN and install packages from CDN

  * Updates all CDN downloads to HTTPS

* Removes interactive `rm` command arg in DockerfileLeanFoundation cleanup
10198
2020-12-28 18:22:15 -03:00
Adalyat Nazirov 017b464e21 Map legacy symbols using map file (#5017)
* map IB brokerage symbol

* map Alpaca symbols

* improve empty symbol checking
10197
2020-12-28 17:54:00 -03:00
Gerardo Salazar 4dec21ccc0 Fixes failing option chain provider test for futures options (#5088)
* The test didn't take into account that the contract expiring on
    December would mean that there would temporarily be no Dec. FOP
    contract. We fix this by looking for the Mar. contract instead
    if the december contract has expired.
10195
2020-12-28 16:28:41 -03:00
Martin-Molinero 529d0a3634 Fix low resolution cache (#5080)
* Replace cache size for MB

* Remove setting capacity

* Custom cache implementation

* Reduce locks in custom cache

* Add cache performance unit test
10194
2020-12-28 16:27:11 -03:00
Martin-Molinero 9c89e8d403 Fix backtest result packet deserializing (#5084)
* added test covering minValue / maxValue issue with JsonRoundingConverter

* change namespaces

* JsonRoundingConverter fix decimal.Min and MaxValues (cannot deserialize)

* remove dependency on 3rd party library

* c# 6 compatible code (remove pattern matching)

* Fixes BacktestResultPacket deserializing

- Serializing decimals as strings to avoid precision loss, since json
  convert will use floating point precision. Updating unit tests.
- Fix logging unit test failing to delete file being used.

Co-authored-by: Mark Virchenko <mark.virchenko@calienteam.com>
10193
2020-12-28 16:26:40 -03:00
Adalyat Nazirov 1988ad1ae5 Bug 4925 daylight out of order bar (#4941)
* test

* wip

* Revert "Fix duplicated history entries when contains daylight saving time change (#4700)"

Use proper rounding down

* regression test

* remove unused parameters

* more tests

* fix name and comment

* improve regression test

* more tests: oanda market hours

* re-apply Exchange TZ to bar EndTime

* fix expected results

* we can't substract minute because it can harm algorithm on minute resolution; so we could use tick?

* rename prop: conflict with QCAlgorithm.StartDate

* do not log messages to pass travis ci log limit

* assign loghandler in AlgorithmSetupHandler

* reference to PR for more description

* due to https://github.com/QuantConnect/Lean/pull/5039 we don't need to override it manually
10192
2020-12-28 16:24:33 -03:00
Martin-Molinero 3658fb1405 Update config.json (#5086) 10191 2020-12-28 13:06:07 -03:00
Colton Sellers 0ac79487d7 IB Brokerage Restore Subscriptions Fix (#5078)
* Return true after all symbols have been processed

* Continue on Canonical Future Symbols
10178
2020-12-23 20:56:07 -03:00
Gerardo Salazar 38120a3217 Converts QuantConnect.Common.csproj file to .NET Sdk format (#5071) 10172 2020-12-23 13:15:29 -03:00
Martin-Molinero 01a13a095a Remove cloud-api-url config setting (#5073) 10171 2020-12-22 22:51:00 -03:00
Colton Sellers 704a5cb00e Pycharm Docker Debugging Fix (#5069)
* Move PyCharm Debugger to port 6000 and expose it

* Change PyCharm default configurations to port 6000
10154
2020-12-22 15:22:46 -03:00
Martin-Molinero 490ad08c89 Optimization handling of failed backtest init (#5068)
- Optimizer will increment failed count and notify the strategy of
  failed backtest initialization
10140
2020-12-21 19:27:27 -03:00
Martin-Molinero a60cd95611 Optimizer will clean lean before last update (#5063)
- LeanOptimizer will clean up any lean instance before calling the last
  udpate, so that runtime stats are correct.
2020-12-21 15:17:44 -03:00
Colton Sellers eed8e9c763 Improve IBBrokerage Symbol Mapping Error Message (#5051)
* Improve IBBrokerage Error Message

* Add exception message to catch statement

* Address review

* Address review edits
2020-12-21 13:04:05 -03:00
Marco Grassi 365f53dfc7 Fix the missed rename of OptimizationStatus.Ended to Completed (#5060) 10115 2020-12-19 19:06:09 -03:00
Martin-Molinero 48a86c2626 Rename OptimizationStatus from Ended to Completed (#5059) 2020-12-18 16:53:29 -08:00
Martin-Molinero 2de9c21b6d Optimization tweaks (#5056)
* Add progress for runtimeStats

- Add optimization progress runtimestats
- Add ServerStatistics for the last backtesting package

* Add StaticOptimizationParameter

- Add StaticOptimizationParameter. Updating unit tests
2020-12-18 16:01:44 -08:00
Martin-Molinero 649aafc952 Adding Api.ReadBacktest optional getCharts (#5054)
* Adding Api.ReadBacktest optional getCharts

- Optionally allow users not to fetch backtest charts when reading a
  backtest, it can be slow

* Fix null reference for deleted/cancelled backtests

* Get Leaky bucket config settings once
10091
2020-12-18 14:17:40 -03:00
Gerardo Salazar 25fa4c6fb6 Adds holidays to ES futures (#5045)
* Adds holidays to ES futures

* Adds CME equity product market hours and holidays (ES, NQ, YM)
10072
2020-12-17 21:12:20 -03:00
Martin-Molinero d1b35f7974 Optimization status update (#5050) 10065 2020-12-17 17:09:01 -03:00
Colton Sellers 4006ba01e4 Reduce Travis Log (#5039)
* Reduce Travis setup verbosity

* Introduce ConsoleErrorLogHandler

* Change Console.WriteLine to Log statements

* Quiet wget

* Route build stubs stdout to null

* Fix Quantbook history test

* Silence stub packages directly

* Use parameterized log-handler for testing

* Rename AssemblyInitialize Setup

* Fix AlgorithmRunner file logging

* Drop all overriden LogHandlers in tests

* Change to OneTimeSetup to maintain LogHandlers

* Permit any ILogHandler to be defined in params

* Fix for AlgorithmRunner Handlers V2
10054
2020-12-17 12:59:13 -03:00
Martin-Molinero 6d824b40a6 Add handling for custom host name (#5043)
* Add handling for custom host name

* Credit cost as decimal
10052
2020-12-16 21:54:00 -03:00
Colton Sellers 8e410fcaf1 Update old Hourly/Daily data for ApiDataProvider (#5034)
* Update old Hourly/Daily data for ApiDataProvider

* Add unit test for IsOutOfDate

* Address review

* Fix reference to static function in test
10049
2020-12-16 19:04:06 -03:00
Adalyat Nazirov d136428556 ignore *.DotSettings files (#5035)
This layer is designated for common settings that enforce your team preferences for the current solution.
Since we don't have specific resharper settings we want to share within the team we ignore this file
10046
2020-12-16 12:38:01 -03:00
Gerardo Salazar 40d81965be Adds debug logging for MapSymbol(...) method in IB Brokerage (#5040) 10040 2020-12-15 17:44:00 -08:00
Aaron Janeiro Stone c650eb6c1c Adds DeM indicator (#5002)
* Adds DeM indicator

* Added reference to param movingaverage type

* Fixed variable declarations

* Added nameless initialize

* Missing DeM "type" args added

* Missing DeM "type" args added

* refactor

* Undid _previousInput → protected

* Demarker symbol: DeM →DEM

* Symbol change: DeM → DEM

* Updated symbols

TestDivByZero originally had dem as cmf.

* Symbol: DeM →DEM

Co-authored-by: Alexandre Catarino <AlexCatarino@users.noreply.github.com>
10018
2020-12-15 08:29:55 -03:00
Charles Naccio aa2f9f927a Fixed QuantBook import path for CSharp example (#5028)
Now requires a relative path for loading QuantBook.csx
10016
2020-12-14 21:22:22 -03:00
Charles Naccio c698a65a84 Replaced obsolete Gitter badge with Slack (#5031) 10013 2020-12-14 12:55:35 -03:00
Charles Naccio 181283a4cd Noted Polygon downloaded in Readme (#5025) 2020-12-11 16:55:58 -08:00
Martin-Molinero b247724a34 Improve Optimizer runtime statistics (#5024)
- Adding optimization Id to backtests packets
- SeriesSampler will allow truncating the samples
- Removing OptimizationEstimate, simplifying getting estimate and
  runtime stats separatly
2020-12-11 16:39:13 -08:00