10725 Commits

Author SHA1 Message Date
IlshatGaripov 0d0389ef6a BrokerageSetupHandler min resolution variable default value bug fix (#5243)
* fix

* Minor tweaks adding unit tests

* Address reviews

- BrokerageSetupHandler will use UniverseSettings.Resolution as default
  resolution

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10588 10594 10595 10596
2021-02-02 12:10:18 -03:00
IlshatGaripov 3e962b2e76 Fixes Bitfinex Brokerage history call bug (#5242)
* Fixes a bug #5173

* This is not timestamp but total ms.

* BitfinexBrokerageHistoryProviderTests fix
10583
2021-02-01 21:58:57 -03:00
Adalyat Nazirov f263fd49a7 Atreyu Brokerage initial setup (#5247)
* Atreyu Brokerage initial setup, and changes required for integration with main Atreyu project

* Address reviews

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10582
2021-02-01 21:57:25 -03:00
IlshatGaripov 9b38bf3d91 More details: (#5244)
Bitfinex exchange may return us an empty result - if we request data for a small time interval during which no trades occurred - for example 1 minute interval - can happen even with most liquid pairs, like "ETHUSD" - would be good to have some time margin  for such scenario. and pump more data to warm up conversion rates
10581
2021-02-01 21:57:10 -03:00
Aaron Janeiro Stone 7d70698c9a Feature #5098 - Time Series Indicators (#5099)
* Cleans history for ArimaIndicator/TimeSeriesIndicator.

-- removes commits from a tracked, already merged branch
-- removes artifacts from debugging sessions

* Removes AR/MA method as a user-specifiable method.
-- Prevents need to reference dll for MathNet in Tests (and potentially elsewhere).
-- Wrapper can be implemented around this functionality.

* Removes AR/MA method as a user-specifiable method.
-- Prevents need to reference dll for MathNet in Tests (and potentially elsewhere).
-- Wrapper can be implemented around this functionality.

* Better adherence to established code style

* Makes _intercept = true by default in constructor where it is not parameter

* WIP -- addressing reviews

* Passing tests following prior refactor

* Rearranged code, access modifiers adjusted

* Fixed indexing of _mafits, adds example algorithm

* Adds regression algo in python + addresses some refactors

* Addresses review

* Adds regression stats

* Fixes missing value signs

* Removes redundant code

* style changes

* style changes

* style: "err" -> "error"

* Minor tweaks

* Fixes python arima regression test

* Refactors AutoregressiveIntegratedMovingAverageTests.cs

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10580
2021-02-01 20:17:49 -03:00
Martin-Molinero 1606b12475 Fix IB future chain being started at reset hours (#5246)
* Fix IB future chain being started at reset hours

- Fix IB future chain being started at reset hours. Adding unit test

* Refactor CanAdvanceTime into CanPerformSelection
10578
2021-02-01 18:53:33 -03:00
Juan José D'Ambrosio 3a483fc4ec Migrate TradeConditionFlags and QuoteConditionFlags to Lean (#5227)
* Migrate TradeConditionFlags and QuoteConditionFlags to Lean

* Address revirew

Implement flag systems using long, this allows us to declare up to 64 flags
10574 10575 10577
2021-02-01 13:11:30 -03:00
Gerardo Salazar fe74d037c6 Updates NodaTime to latest version (3.0.5) (#5237)
* Updates NodaTime to version 3.0.5

  * Updates code to ensure compatibility with new NodaTime version
  * Updates System.Runtime.CompilerServices.Unsafe to 5.0.0
    - Required by NodaTime

* Adds comments explaining changes in NodaTime
10573
2021-02-01 13:06:56 -03:00
Martin-Molinero 43e9cad2d6 Remove deprecated Http package (#5235) 10569 10570 10563 10568 10565 2021-01-29 18:26:38 -03:00
Balamurali Pandranki 403ea22606 Setup GitHub Actions CI (#5192)
* Create gh-actions.yml

* Run github actions on all branches

* Setup Python and Deps

* Escape brackets

* Move to mono runtime instead of dotnet core

* lock mono version

* pin mono dep version

* Update gh-actions.yml

* Update gh-actions.yml

* Use Ubuntu 16.04

Use Ubuntu 16.04

* Update gh-actions.yml

* Try on container

* Fix indentation

* Try Self Hosted runner

* switch back to ubuntu runner

* Update gh-actions.yml
10550 10559
2021-01-29 12:38:05 -03:00
Gerardo Salazar 6cb103b17a DockerfileLeanFoundation package updates and new additions (#5072)
* Updates and adds new Python packages to DockerfileLeanFoundation

* Remove torch-geometric packages due to import issues

Torch geometric packages installed from pip as they are here result in an error saying to install the CUDA version of the packages. When the CUDA version is installed, we get a symbol error when attempting to link the `_version.so` shared object.
10547
2021-01-29 10:25:14 -03:00
Martin-Molinero 8cbdfcaf28 Fix hour resolution mapping data handling (#5233)
- In some cases hour resolution data subscription would end early
  because of remapping not being correctly handled. Adding regression
  test
10545
2021-01-28 20:41:04 -03:00
Juan José D'Ambrosio 1778462505 Fix broken regression tests (#5234) 10544 2021-01-28 20:14:03 -03:00
Colton Sellers 472896c2d0 Update JSON Library (#5218)
* Update NewtonSoft.Json from 10.0.3 to 12.0.3

* Remove JsonSerializer.Populate(), values are already populated
10543
2021-01-28 19:24:49 -03:00
Stefano Raggi 15cb846570 Add missing connection check in IB brokerage GetHistory (#5230) 10529 2021-01-28 15:43:04 -03:00
Stefano Raggi 553b50a92e Ignore IB error 506 when disconnected (#5229) 10528 2021-01-28 15:42:53 -03:00
Gerardo Salazar 5f73a05e04 Set default market hours for futures exchanges to 16:00 Central Time (#5226) 2021-01-27 20:00:19 -03:00
Colton Sellers a5eaa43e35 Remove F# and VB Projects (#5223)
* Remove F# and VB projects

* Remove FSharp.Core package

* Readd FSharp to engine

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10493
2021-01-27 11:31:49 -03:00
Martin-Molinero e91af0814f Add notification targets and events to live packet (#5210)
* Add notification targets and events to live packet

- Add notification targets and events to live packet. Adding unit tests

* Add project name to job packet

* Add ShortToString and WebNotification Header

- Adding ShortToString implemnetation of OrderEvent and Insight
- Add Web notification headers

* Add more unit tests

* Revert json lower case fields
10490
2021-01-27 10:26:35 -03:00
Colton Sellers a2850fb20c Address Non-Compiled Files (#5225)
* Include MortgageRateVolatilityAlpha and fixes

* Include PlaceHolder PythonAlgorithm.cs

* Delete BinanceUtil

* Remove unused Regression Algorithms

* Minor tweaks

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10488
2021-01-27 09:56:03 -03:00
Colton Sellers c86fceac97 Fix regression typo (#5224) 10479 2021-01-26 15:31:57 -03:00
Stefano Raggi 9997df45af IB Brokerage updates (#5222)
* Upgrade IBAutomater to v1.0.40

* Fix FinancialAdvisor account id check

- The Ixxxxxx account code is the Master Account for Fully Disclosed Brokers and this account is not tradable. Proprietary accounts for Brokers/Dealers used for proprietary trading have the Uxxxxxx account code.

* Update IBAutomater to v1.0.42

* Update IBAutomater to v1.0.43

* Log server version in ConnectAck handler
10478
2021-01-26 09:52:05 -08:00
Jared 1ef92b3eca Update Crisis.cs 10462 2021-01-25 10:47:15 -08:00
Adalyat Nazirov c6d29c8be5 add FIX Protocol UTC timestamp format (#5212) 10453 2021-01-25 11:24:54 -03:00
Derek Melchin 97deb4bbc6 Fix typo (#5216) 10451 2021-01-25 10:24:15 -03:00
Gerardo Salazar 6ab40c102b Only report daily point-in-time portfolios instead of every trade (#5214)
* Only report daily point-in-time portfolios instead of every trade

* Allow for cash holdings to be visible in point-in-time portfolio output
10449
2021-01-22 17:00:09 -08:00
Gerardo Salazar 41417593aa Writes point-in-time portfolios to disk (#5213)
* Writes point-in-time portfolios to disk

* Adds optional argument to Report constructor in ReportGenerator
10443
2021-01-22 15:17:19 -08:00
Martin-Molinero 3ccf428498 Adjust delisting liquidation time (#5203)
* Adjust delisting liquidation time

- Adjust delisting liquidation time to 15 min before market closes.
  Adding unit tests. Updating existing.
- Handle `Statistics.CompoundingAnnualPerformance` invalid calculation
  to avoid exception.
- AlgorithmManager will not handle delisting events in live trading
- Fix bug where due to a split driven liquidation matching delisting
  date a position in the option would remain open. Reproduced by
  `BasicTemplateOptionsFrameworkAlgorithm`

* Address review

- Address review add documentation on delisting offset span
10436
2021-01-22 14:41:29 -03:00
Colton Sellers 5d8a62c2e1 Symbol Alias Fix (#5205)
* Fix Symbol Alias

- Fix Symbol Alias being wrong in some cases for futures and options

* Add test cases

* Address review

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10434
2021-01-22 13:00:14 -03:00
Martin-Molinero 5df29f16dd Fix thread safety of Mapfile exchange provider (#5208) 10429 2021-01-21 19:02:50 -08:00
Martin-Molinero 56a0d52212 Allow limiting streaming security Types per job packet (#5206) 10426 2021-01-21 21:23:32 -03:00
Juan José D'Ambrosio 765f2cab3b Add primary exchage to map files (#5194)
* Add main exchange to map file

minimum minimorum test

* Add MainExchange to MapFilRow IEquatable implementation
Also write CSV line correctly

* Normalize exchanges

Also add small test form MapFileRow

* Exchange name normalization

- Exchange name normalization.
- Tick will protobuf `ExchangeCode` which will set existing `Exchange`

* Include single character exchange code

* Encode exchange as byte, Include exchange name as Enum

* Update MapFileRow.cs to accept a byte instead a string for primary exchange

* Fix issues in encoding exchanges map

* Fix broken MapFile tests

* Rename Main Exchange property in MapFileRow

* Use BATS instead of BATSZ

* Add primary exchange from map file

* Move MapFilePrimaryExchangeProvider to its own file

* Implement Char for primary exchange in map files

* Fix broken tests after implementing char exchange into map file

* Add a new constructor for MapFileRow

it  accepts a PrimaryExxchange object as parameter

* Fix broken test 

WIP

* Address review 

WIP

* Handles unexpected encoded exchanges

* Keep addressing review

* Address review

* Fix broken tests

Make PrimaryExchange as default even if the IPrimaryExchangeProvider is not instantiated

* Revert breaking change.

* Add a check that resolved map file actually have content

The issue that leads to this change was caused by a map file resolved to APC in the CoarseFundamentalTop3Algorithm. 
The resolver returns a not-null MapFile, but it is empty.
To my best understand, the MapFile Resolver should return only existing map files in the map_files folder. But the LocalMapFileProvider returned a map file with a permtick that doesn't exist in the file system as CSV.

* Set PrimaryExchange.UNKNOWN as default

* Address review couple minor bugs

- Use SID market instead of USA
- Default ticke exchange is UNKNOWN

* Update map files with latest format and latest info

This include:
 - VXX, it was delisted, then VXXB appears and later it chnages from VXXB -> VXX
 - AAA delisting and a new listing with ticker AAA
 - FOXA ticker changue to TFCFA and later it was delisted

* Update test after updating latest map files

* Fix bug with factor files after updating map files

* Fix bug with factor files after updating map files

and revert statistics in regression tests

* Update OrderHash in regression test

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10420
2021-01-21 18:46:12 -03:00
Martin-Molinero f175ac6c6b Fix universe selection symbol cache removal (#5193)
- Only remove symbol from cache during universe selection if there is no
  other universe using it. Adding regression test reproducing issue
10414
2021-01-21 15:48:33 -03:00
Gerardo Salazar 79c544d3a2 Reduces output log size and warnings emitted when building with mono's msbuild (#5200) 10409 2021-01-21 11:09:16 -03:00
Aaron Janeiro Stone 5dcbed535d 5052 - Adds missing constructor routing (#5201)
* Adds missing constructor routing

* Minor tweak remove unrequired lines and format

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10408
2021-01-21 10:49:27 -03:00
Gerardo Salazar 35393b66a3 Updates report generator template.crisis.html (#5202) 10407 2021-01-20 18:01:44 -08:00
Karthik Kailash ce42c888ce Add 2001-12-24 to US equities early closes in market-hours-database.json (#5184) 10383 2021-01-19 13:21:00 -08:00
Colton Sellers f2fd10d9d2 Build Configuration Hotfix (#5186)
* Reflect properties prior to migration onto projects

* Do not prefer 32bit

* Reinstate SelectedOptimization var
10372
2021-01-19 09:38:16 -03:00
Colton Sellers 94d766ff89 Refactor Default Benchmark (#5158)
* Have BrokerageModel determine default benchmark

* Add DefaultBenchmark to Python wrapper

* Handle Null benchmark case

* Add NullBenchmarkRegressionAlgorithm

* Refactor solution to have BrokerageModel return IBenchmark; also refactor QCAlgorithm benchmark handling

* Always create a new security for benchmark

* Drop security overload, Always create a new security for benchmark

* Check our securities for a symbol matching the ticker before creating a new one

* No Python version of this regression

* Address review

* Create shared SecurityBenchmark creator function

* Add Python regression and needed FuncBenchmark constructor
10371
2021-01-19 08:40:41 -03:00
Martin-Molinero 592d037085 Fix delisted liquidation orders being cancelled (#5169)
* Fix delisted liquidation orders being cancelled

- Place delisted liquidation orders 10 min before market closes of 10
  min before the end of the delisting warning date. Adding regression
  test and unit tests. Updating existing tests.

* Fix failing python option unit tests

* Fix bug where positions would be open delisting liquidation

* Fix universe selection and delisting

- Delisting will happen ASAP for all types. Giving priority to close
  positions on derivates first
- Fix bug in universe selection where OptionChain would remove
  underlying even if holding a position in derivate.
- Updating regression tests statistics

* Add unit test, fix unit test expected stats
10370
2021-01-18 22:02:40 -03:00
Gerardo Salazar 17ab10531a Fixes ETB/shortable regression algorithms that were previously failing (#5174)
* Fixes ETB/shortable regression algorithms that were previously failing

* Addresses review and fixes bug

  * Fixes bug where orders would be denied regardless of direction
    whenever they exceeded the absolute value of the shortable quantity

  * Updates regression algorithm + statistics + simplifies test

  * Fixes python regression algorithm

* Addresses review: removes comments and unneeded imports
10369
2021-01-18 21:19:42 -03:00
Juan José D'Ambrosio 9d576a4121 Update regression tests statistics (#5187)
Those statistic were affected for the new early market close entries in MHBD.
10368
2021-01-18 21:06:26 -03:00
Aaron Janeiro Stone 7aefe07274 4553 - Refactor RenkoConsolidator (#5052)
* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Removes base class for brevity

* Delete BaseRenkoConsolidator.cs

* Fixes naming for classic Renko consolidator

* Addresses review concerns

* Fixes missing null checks and missing typesafe constructor

* Adds warning

* WickedRenkoConsolidator -> RenkoConsolidator calls

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Removes base class for brevity

* Fixes naming for classic Renko consolidator

* Separated Update calls and other non-common RenkoConsolidator.cs artifacts.

* Added type safe init for wicked

* Refactor; moved wicked-specific fields to WickedRenkoConsolidator.cs

* Addresses requested changes (and compacts constructors)

* Delete BaseRenkoConsolidator.cs

* Addresses review concerns

* Fixes missing null checks and missing typesafe constructor

* Adds warning

* WickedRenkoConsolidator -> RenkoConsolidator calls

* Addresses review

* unneeded tag removed

* Minor tweaks address reviews

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
10364
2021-01-18 15:54:15 -03:00
Karthik Kailash 6de9a146a9 Issue 5149: Added missing early closes for 1999-2008 (#5163)
* Added missing early closes for 1999-2008

See GitHub issue for details

* Fix expected Treynor ratio for backtest in test suite

This backtest runs from 2008-10-10 to 2010-10-10 so it is affected by the early close date now recorded on 2008-12-24.
10348
2021-01-15 11:22:44 -03:00
Gerardo Salazar 4d4ad92fa7 Easy To Borrow Hotfix (#5167) 10344 2021-01-14 15:07:54 -08:00
Jared 345fe7357e Update readme.md 2021-01-14 13:58:26 -08:00
Gerardo Salazar 3e7af17e02 Implements Easy To Borrow Reality Modeling (WIP) (#5159)
* Creates IShortableProvider interface + interface impls

  * This is the foundational work for the addition of the shortable stocks
    feature for backtesting and live trading. Note that the QCAlgorithm
    API and the backend transaction handling will be implemented
    separately.

* temp; work on preorder checks

* improve checks

* tmep

* Enforces ETB checks at BrokerageTransactionHandler

  * Adds ETB Shortable Provider to QCAlgorithm and IAlgorithm
  * Removes ETB check from PreOrderChecksImpl
  * Removes outdated test, new test to come soon for relevant class

* Work in progress commit, pushing for review.

  * Adds new regression algorithm (WIP)
  * Enhances ShortableProviderOrdersRejectedRegressionAlgorithm
  * Adds new methods to QCAlgorithm
  * Move IShortableProvider to have BrokerageModel own it
  * Comments updates
  * Adds new properties to Equity Security for shortable

  * Fixes bug where retrieving open order quantities would aggregate the
    submitted order quantity rather than the remaining order quantity for
    open orders.

  * Fixes bug where quantity of zero would result in a
    false positive in QCAlgorithm.Shortable(...)

  * Code refactoring and comments updates

Co-authored-by: Jared <jaredbroad@gmail.com>
2021-01-14 13:33:44 -08:00
Colton Sellers a2a874e333 CSProj Update (#5145)
* Migrate to newer CSProj format; build succesful

Rebase onto master

* Maintain binaries directory

* Reinstate FSharp default condition

* Remove reference to packages.config

* Test project fixes

* Remove nuget.config and "Solution Items"

* Remove repetitive assembly info

* Reinstate default compiles and cleanup redundant settings

* Reinstate any specific settings that defaults are unclear for

* Reinstate Framework.nuspec

* Make test internal classes public for composer use; undo assembly change

Rebase onto master part 2

* Address review; restore some settings

* Restore RootNamespace and AssemblyName attributes

* Use NugetPackageRoot instead of NugetPackageFolders

* Reinstate "RestorePackages" setting

* Transfer compilied Python files to launcher and tests bin

* Remove conflicting settings

* Fix WebSocketSharpFork references

* Fix FSharp Core reference

* Upgrade TestPlatform.ObjectModel to 16.8.3, and redirect V11 binding

* Add Mono.Cecil package

* Address review
2021-01-14 12:55:09 -08:00
Juan José D'Ambrosio 96592b3387 Update RTY price multiplier (#5166) 10339 2021-01-14 16:45:24 -03:00
Colton Sellers 7e3741d983 Allow Custom Security Properties and MarketHours (#5100)
* Add interface to allow custom security entries for MHDB and SPD

* Simplify adding custom Properties and MarketHours via AddData overload

* Refactor

* Remove AddData and GetDatabaseSymbolKey overloads

* Add unit tests

* Remove AddData overload, for real.

* Nit changes

* Set CustomDataBitcoinAlgorithm back to original

* Add Python and C# Regression

* nit typo

* nit typo actual

* Reset symbol property to default

* Reflect last change to py regression

* Revert "Remove AddData overload, for real."

This reverts commit dc877495c079f17dda1695494ffe36886b1c117e.

* Implement AddData solution fully

* Function nit fix

* Address review

* nit - remove param comment

* Address review
10335
2021-01-14 11:34:02 -03:00