1117 Commits

Author SHA1 Message Date
Jhonathan Abreu 17e15d9045 Minor change 2024-08-26 11:02:21 -04:00
Jhonathan Abreu 83d58735b1 Keep underlying when aggregating collections in BaseDataCollectionAggregatorEnumerator 2024-08-26 11:02:21 -04:00
Jhonathan Abreu 1cb6dacb12 Fix for live trading options universe selection 2024-08-26 11:02:21 -04:00
Jhonathan Abreu fef0481ad2 Address peer review 2024-08-26 11:02:18 -04:00
Jhonathan Abreu 772f8afb7f Added processed data provider 2024-08-26 11:02:17 -04:00
Jhonathan Abreu 34831c3353 Option chain handling in live trading data feed 2024-08-26 11:02:17 -04:00
Jhonathan Abreu 88f04e4da6 Fix option chain provider history request 2024-08-26 11:00:31 -04:00
Jhonathan Abreu 96d86a3c63 Update regression algorithm stats 2024-08-26 11:00:31 -04:00
Jhonathan Abreu 3251da986e Peer recommended changes and fixes 2024-08-26 11:00:31 -04:00
Jhonathan Abreu ad0b85ab3f Address peer review 2024-08-26 11:00:30 -04:00
Jhonathan Abreu 5604ab066c Option chain provider with new options universe 2024-08-26 11:00:30 -04:00
Jhonathan Abreu dafb81cf2a Updated options regression algorithms stats for new universe data 2024-08-26 11:00:29 -04:00
Jhonathan Abreu 12f16da756 File based options universe fixes and improvements.
- Adjust OptionUniverse start-end times and period.
- Adapt unit tests and some algorithms to pass with new options universe selection.
2024-08-26 11:00:29 -04:00
Jhonathan Abreu 3ab16e127e Address peer review 2024-08-26 11:00:29 -04:00
Jhonathan Abreu 33e6962073 Options universe improvements 2024-08-26 11:00:29 -04:00
Jhonathan Abreu 4144c09286 Initial options universe with greeks implementation 2024-08-26 11:00:29 -04:00
Martin-Molinero 7879795207 Enable daily precise end time by default (#8254)
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Default daily precise end times

- Enable by default daily precise end times. Updating stats
- Minor fix for algorithm manager consolidator updates, adding new regression test
  asserting behavior and updating others
- Minor fix for SubscriptionData creator avoid round down on warmup if
  not appropiate
- Adjust consolidators to emit on daily strict end times if requested
  daily resolution and setting enabled
- Updating regression algorithms

* Skip daily data on extended market hours

* Some cleanup and self review

* Revert unrequired change
2024-08-14 12:49:56 -03:00
Martin-Molinero 2df58bbca3 Accept live UniverseProvider returning null (#8222)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Similar to HistoryProviders and Live DataQueueHandler calls, adjust
  live Universe provider manager to handle null returned values
2024-07-23 10:30:19 -03:00
Ricardo Andrés Marino Rojas 554a82527f Fix minor warnings (#8190)
* Fix CA1847 warnings and some CA1307 warnings

* Fix CA1823 and CS0169

* Fix CA1820 warnings

* Address requested changes
2024-07-10 14:17:27 -03:00
Ricardo Andrés Marino Rojas a7efcfe9d5 Fix CS1570 and CS1572 warnings (#8187)
* Fix CS1570 and CS1572 warnings

* Address requested changes
2024-07-10 11:16:16 -03:00
Ricardo Andrés Marino Rojas 5a7a162d34 Fix warnings CA1574 (#8186)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-07-09 17:52:07 -03:00
Ricardo Andrés Marino Rojas 99a949586c Fix warnings part 12 (#8182)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* First part of the warnings

* Second half of the changes
2024-07-09 11:08:42 -03:00
Martin-Molinero 6c30157fab Remove universe selection on extended market dates (#8160)
- Remove universe selection on dates with extended market hours only
- Updating regression algorithms
- Expand date & time rules API to support specifying whether extended
  market hours only dates are desired or not
2024-07-08 11:58:50 -03:00
Ricardo Andrés Marino Rojas 16a737ee26 Fix most of CA1725 warnings (#8154)
* Fix CA1725 warning

* Address requested changes
2024-07-05 16:19:56 -03:00
Ricardo Andrés Marino Rojas 0a0c012ae9 Fix second half of the CA1051 warnings (#8140)
* Fix second half of the CA1051 warnings

* Address requested changes
2024-07-05 13:54:03 -03:00
Ricardo Andrés Marino Rojas 57f89620a1 Fix warnings part 6 (#8107)
* Fix half of the XML docs missing warnings

* Remove warnings from csproj files
2024-06-26 13:21:53 -03:00
Jhonathan Abreu f0d02feb13 Rename AlgorithmSettings.DailyStrictEndTimeEnabled (#8098)
* Rename AlgorithmSettings.DailyStrictEndTimeEnabled  to DailyPreciseEndTime

* Minor fix
2024-06-24 14:10:11 -03:00
Martin-Molinero e78150aced Fix for custom streaming data live trading (#8077)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Minor fix for custom streaming data in live trading. Adding unit test
  reproducing issue
2024-06-05 11:51:43 -03:00
Martin-Molinero 529b24ed1b Live Collection unfolding data will skip old data (#8070)
- Minor tweak for live collection unfolding data will skip old data.
  Adding unit test
2024-06-04 11:52:33 -03:00
Martin-Molinero bb675bdddb Improve DQH handling of exceptions (#8054)
- Improve DataQueueHandlerManager handling of exceptions in the used
  data queue handlers. Adding unit tests
2024-05-24 15:29:14 -03:00
Martin-Molinero d3b5bbaa1b Avoid duplicate data on FF resolution change (#8053)
Report Generator Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-05-24 11:58:27 -03:00
Martin-Molinero 3a99e0b47c Fix fill forward resolution adjustment on removal (#8049)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* Fix fill forward resolution adjustment on removal

- Fix fill forward resolution adjustment on subscription removal. Adding
  regression algorithm reproducing issue

* Minor fix for undeterminism behavior on security removals and FF res change

* Further improvements and determinism

- Improve regression algorithm add, remove and readd future, asserting
  FF resolution changes
- Adjust universe selection to be deterministic and avoid unnecessary
  FF resolution changes
2024-05-24 10:15:26 -03:00
Martin-Molinero 5f3cd203d3 Fix for live FF enumerator (#8047)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
- Adjust live FF enumerator to rely on the base FF enumerator logic and
  just filter out results based on the frontier. Adding unit tests
2024-05-22 13:39:52 -03:00
Martin-Molinero d4e7f584f6 Daily data time adjustment (#8001)
* Daily data Time & EndTime Improvement

- Adjust daily data Time & EndTime to actually reflect the time of the
  data used, for example US Equity from 9.30 to 4PM. Adding new unit and
  regression tests

* Refactor solution to use enumerator

- Refactor daily strict end times solution to be through enumerator
  usage, so it applies for history providers too

* Minor fixes

* Revert fill forward enumerator change

- Revert FillForward enumerator causing stats changing, enhancing unit
  tests

* Some cleanup

* Improve handling of live trading FF enumerator

- Improve handling of live trading FF enumerator, by adding support for
  bars to arrive with a delay so we can handle auction close/option
  prices or data providers which might have some delay making the data
  available. Adding new unit tests asserting the behavior
2024-05-20 17:28:03 -03:00
Martin-Molinero 1c32144963 Some refactor and cleanup (#8014)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Some refactor and cleanup

- Some refactor and cleanups
- Improve live trading CI tests speed

* Minor tweak for live trading data feed log
2024-05-10 12:27:00 -03:00
Roman Yavnikov ef23d7b6fe Add Group Symbol and Total Time Execution Logging for Downloading Canonical Data (#7984)
* refactor: modifier static to parse enum and ignore warning

* feat: group symbol when download cannonicial data
feat: log total time execution of download provider
test: right keeping of cannonicial data

* feat: parameter ctor in DataDownloadConfig

* test:feat: additional assertion of wrote data in file

* feat: static FilterGroupDownloadDataBySymbol

* refactor: extract downloader folder path

* try: map file false

* feat: get rid of dataCacheProvider in DownloaderDataProvider

* refactor: get rid mapFile flag

* CI test fix

---------

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2024-05-01 13:39:34 -03:00
Jhonathan Abreu 9755f8c916 Sourcing remote zipped custom data (#7976)
* Fix: source custom data from remote CSV zipped file

* Fix: Custom data sourcing zip entry names from remote locations

* Address peer review

* Add unit tests

* Minor fix

* Address peer review

* Minor fixes

* Return byte array in IApi.DownloadBytes

* Allow sourcing custom data from specific csv entry in a zipped remote file

* Minor change
2024-04-26 12:15:01 -04:00
Martin-Molinero 2ff13ea8fc Remove unrequired, unused universe DQH call (#7914)
- Remove unrequired, unused DQH subscription call to universe
  subscription
2024-04-15 13:50:30 -03:00
Martin-Molinero bf741a155f Minor tweak for Live Synchronizer (#7912)
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
2024-04-11 10:43:41 -03:00
Martin-Molinero fa9f10bebf Add support for auxiliary data history request (#7906)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Add support for auxiliary data history request

- Add support for split/dividends/margin interest history requests.
  Adding regression algorithms

* Expand auxiliary history regression tests
2024-04-10 12:59:32 -03:00
Martin-Molinero bb5c0eb54e Minor api improvements (#7901)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Minor api improvements

- Adding Universe.Selected to hold the current set of selected symbols.
  Adding regression tests
- Adjusting TimeRules and DateRules to fallback to the MHDB if the
  security hasn't been added. Updating existing regression tests

* Expand regression test assertion
2024-04-05 17:43:25 -03:00
Jhonathan Abreu 314673534c Options/Index Options hour/daily resolution required data only download (#7898)
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* fix: download year file data only for hour and daily resolutions for options and index options

* minor fix: index options regression tests expected status

* Minor change

* minor: update regression algorithms stats
2024-04-04 18:18:33 -03:00
Martin-Molinero 3aff0ce958 Add Scheduled Universe Selection (#7895)
* Add Scheduled Universe Selection

- Adding `UniverseSettings.Schedule` that will allow users to
specify a custom selection schedule which is independent of the
underlying data frequency
- Adding unit and regression tests

* Add live trading schedule time shift
2024-04-04 12:21:51 -03:00
Martin-Molinero 3eedde9b6e Normalize ETF constituents universe (#7858)
- Normalize ETF constituents universe data. Adding new tests and
  assertions
2024-03-19 11:28:17 -03:00
Roman Yavnikov 7e568bf4ac Feature: find different Ticker data download parameters (#7845)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* refactor: map files in lean/data folder

* feat: Indexers in MapFile

* feat: extension for mapFile and downloader param

* refactor: getAllTickers from MapFiles

* feat: addition out variable in TryParsePath
fix: tests with new variable

* refactor: DownloadDataProvider with getting all ticker with different DateTime Ranges
feat: test

* revert: "refactor: map files in lean/data folder"

This reverts commit 41fa26eb8627446028d4f2d4c536511ea82a9a48.

* revert: "feat: addition out variable in TryParsePath"
refactor: some actual changing

This reverts commit 9de482bd6490eecd1eeee83e035dfc1ece8d6415.

* revert: "feat: Indexers in MapFile"
fix: actual changes

This reverts commit 393181c6316a354a8cdcbfb093707cb545383a8a.

* remove: high performance Any validation
feat: validation on ticker start with specific sign
fix: option test

* feat: validation on mapping
refactor: when resolution less hour
test: new test cases + validation of SecurityType

* refactor: add additional day when return date range from MapFile
fix: FirstTicker instead of Permtick when generate new Equity
feat: validation of request Date in DownloaderDataProvider
refactor: by style proj DownloaderDataProvider
test:fix: starting Date in TestCases

* remove: duplicate code

* fix: check of income data.Time with requested Start/End-Date

* test:fix: download test cuz we can not get/write future data

* remove:  extra Symbol Create mthd

* test:feat: validate of TryParse returns correct symbol

* remove: excess if condition

* test:remove: extra chaning
connected # 3a62adf54c42b01d28ff410e93e1ce277ee6d534

* refactor: split big method to small ones (add readability)

* fix: typo in xml description

* refactor: get the correct symbol value

* refactor: use local time instead of Utc in download Param extension
refactor: use struct instead of class

* test:fix: convert time to Utc from local time
2024-03-18 14:43:11 -03:00
Martin-Molinero a8280e1452 Minor fix for universe history (#7852)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
- Do not cache universe collection data. Adding unit test
2024-03-14 16:20:28 -03:00
Ricardo Andrés Marino Rojas 934a8e0a61 Handle Windows reserved filenames (#7784)
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
* Add ToValidPathMethod

* Nit change

* Add more unit tests

* Improve implementation

* Enhance implementation

* Add more unit tests

* Enhance implementation

* Improve unit tests

* Improve implementation

* Nit changes

* Adjust reading/writing methods

* Fix bugs

* Add Windows OS flag

* Fix unit tests

* Fix unit tests

* Fix unit tests

* Change FileStream() to SafeFileStream()

* Fix bugs

* Fix bugs

* Nit changes

* Nit changes

* Add improvements

* Missing file

* Address requested changes

* Address requested changes

* Add missing test
2024-03-13 19:05:57 -03:00
Martin-Molinero 17ca8a743f Refactor universe historical data source (#7837)
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Refactor universe historical data source

- Add new universe history API methods
- Refactor QuantBook UniverseHistory to use the universe selection
  itself instead of a given func
- Refactor and rename fundamental types
- Refactor AddUniverse API to handle universe collection data which
  holds another type internally, like fundamental

* Fix minor bug causing ApiDataProvider not to serve Bitfinex universe data

* Further improvements to add universe API

* Handle no selection function
2024-03-12 13:41:49 -03:00
Roman Yavnikov e96c08e208 feat: downloader data provider returning null (#7812)
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-02-23 15:45:47 -03:00
Jhonathan Abreu 09db45c91d Universes immediate selection (#7785)
* Make sure continuous futures universe selection is performed right away on the start of the algorithm.

This ensures the Future.Mapped is not null

* Minor changes

* Minor fixes

* Update regression algorithms stats

* Add DC futures map file

* Update DC futures map file

* Update regression algorithms stats

* Include ETFs for immediate universe selection

* Include Coarse Fundamentals for immediate universe selection

* Include Futures Chain Universe for immediate selection

* Add Option Chain Universe test

* Include custom universes for immediate selection

* Minor updates

* Include constituents universes for immediate selection

* Minor update

* Minor changes, docs and stats updates

* Minor changes

* Minor changes and clean up

* Minor regression algorithm stat update
2024-02-21 08:59:58 -04:00