47 Commits

Author SHA1 Message Date
Roman Yavnikov 0d430e2718 Revert: destination-dir param in Toolbox (#7885)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* revert: destination-dir param in Toolbox

* feat: test case on destination-dir param
2024-04-01 10:50:35 -03:00
Roman Yavnikov 14f94923e1 Remove: old / not used part in ToolBox (#7871)
* remove: ref #6383

* remove: fxcm, ref #5505

* remove: not used pckg in ToolBox

* remove: fxcm downloader mention, ref #5505
2024-03-26 10:10:47 -03:00
Roman Yavnikov 22dafee9c6 remove: IVolatilityEquity Converter (#7866) 2024-03-25 17:17:03 -03:00
Roman Yavnikov ca99f329dd remove: Dukascopy Downloader (#7865) 2024-03-25 15:39:46 -03:00
Roman Yavnikov 3338c9a332 remove: CryptoIQ Downloader (#7867)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-03-25 11:10:10 -03:00
Roman Yavnikov 229caf485c remove: NseMarketDataConverter (#7863) 2024-03-25 11:08:19 -03:00
Roman Yavnikov 82b327b2b9 remove: Kraken Data Downloader from ToolBox (#7860) 2024-03-19 19:17:17 -03:00
Roman Yavnikov aca30a896c Remove: Yahoo Data Downloader (#7859)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* remove: missed of QBDL PR:#6110

* remove: Yahoo(YDL) mention in proj
2024-03-19 11:40:46 -03:00
Roman Yavnikov 1dd186acb4 CoinApi Extraction: Removing from Lean.ToolBox (#7777)
* feat: remove CoinApi realization from ToolBox

* fix: ComposeDataQueueHandlerInstances
we have removed all IDataQueueHandler from ToolBox
2024-02-13 10:40:30 -03:00
Roman Yavnikov 4c88e3184c IEX Extraction: Removing from Lean.ToolBox (#7734)
* remove: ToolBox\IEX

* revert: IEX config in config.json
remove: IEX downloader param in toolboxParse
2024-01-30 09:50:36 -03:00
Martin-Molinero 65f8590d94 Clean up unused toolbox classes (#7652)
- Remove QuantQuoteConverter program
- Remove QC Visualizer
2024-01-02 18:55:32 -03:00
Jhonathan Abreu 9601155e28 Remove Polygon from Toolbox (#7638)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Remove Polygon from Toolbox

* Minor changes
2023-12-21 13:08:53 -03:00
Martin-Molinero 61fb997f07 Delete unsupported AlgoSeek Option Converter (#6992)
- Delete unsupported algoseek option converter
2023-02-20 19:21:15 -03:00
Alexandre Catarino e8b95b80e1 Polygon Handles Data Subscriptions with One Security Type (#6855)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Handles Data Subscriptions with One Security Type

Polygon sells data subscriptions for security types separately, so we should not test whether the users have all three subscriptions, Equity, Forex, and Crypto. Users just need the type they will use.

Adds new crypto markets, and unit tests for them. Adapt unit test.

* Fixes ToolBox Downloader

Adds `--api-key` argument.
Closes #6687

* Address reviews

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2023-01-12 17:37:29 -03:00
Adalyat Nazirov 0c26d42561 Feature 2839 black scholes data generator (#6135)
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
* replace to local functions as they are more performant

* fix random generator upper bound

Next() includes minValue, but not maxValue, so we increment it +1

* introduce abstract layers

* refactoring

* fix tets

* adapt tests

* fixup

* implement blackschole price model for options

* use risk free rate

* use ql price model

* wip

* change interface

* fix

* tidy up the code

* wip

* iterate groupped symbols

* wip

* wip

* fix

* allow symbol of different types

* improve settings

* wip

* iterate full range

* fix issue with negative option

* fix

* fixup

* use StandardDeviationOfReturnsVolatilityModel

* re-use existing tick types per security type

* parametrize underlying security type

* use default option style

* dynamic option price model

* fix enumeration

* test

* fix unit tests

* refactor code

* remove unused file

* minor tweaks and refactoring

* rename symbol generator class

* fix interface

* add comments

* more comments and unit tests

* more tests

* add disclaimer

* more tests

* more comments and tests

* split tests into different files

* tidy up the code

* tidy up the code; more tests

* refactor TickGenerator => use security price directly on each iteration

* remove dupe; reuse main constructor

* use SecurityManager, refactor code

* bugfix: save ticks in history array

* check volatility warm up & tests

* more unit tests

* describe volatility period span in settings

* rename command line option

* Minor adjusments. Address review

- Use Lean log handler instead of writting directly to console
- Rename BlackShcolesPriceGenerator to generically OptionPriceModelPriceGenerator
- Minor format clean up & standarization
- Add support for specifying the option chain size

* Rename TickGenerator private fields

* Fix unit tests

* fix tests class name

* Support tickers being specified

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2022-01-10 17:21:03 -03:00
Gerardo Salazar 650ce2e2b5 Removes References To Built-in Custom Data Implementations (#5793)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* WIP removal of custom data references & tests updates

* Regression algos updated and python algorithms moved to DataSource repos

* Fixes failing unit tests

* Add "LiveDataTypes" field to LiveNodePacket

* Adds Initialize() to IDataChannelProvider

  * Adds new extension method to convert
    HistoryRequest -> SubscriptionDataConfig

* Address review: Add protobuf definitions for Iconic data types

* Address review: re-adds DynamicSecurityDataAlgorithm as regression algo

  * Small adjustments to variable naming and documentation

* Move test files to respective DataSource repos
2021-08-02 19:24:45 -03:00
Gerardo Salazar 254d0896f1 Replaces Microsoft.Extensions.CommandLineUtils with McMaster CLI utils (#5255)
* Replaces Microsoft.Extensions.CommandLineUtils with McMaster CLI utils

  * Updates ValueTuple to 5.0.0 for compatibility with package.

* Retrigger build

* Retrigger build
2021-02-03 19:56:14 -03:00
Martin-Molinero e8c316cbcf Fix Toolbox tickers parsing (#4876)
- Fix ToolBox tickers parsing, adding unit test.
2020-10-20 11:27:41 -03:00
Martin Molinero 269b8ba204 Remove PsychSignal data
- Remove PsychSignal data since it's no longer available
2020-05-11 18:48:13 -03:00
Gerardo Salazar 37fd362c8b Add processedFilesDirectory argument
* Changed to read existing indexes from `processedFilesDirectory`
2019-11-19 17:27:52 -08:00
Gerardo Salazar e4789e0b23 Implement Benzinga News Downloader
* Make Benzinga News Converter more efficient (and cleaner!)
* Implement JSON converter for serialization/deserialization process
* Fix bug where duplicate symbols would be added to news.Symbols
* Modify algorithm strategies
2019-11-19 10:28:48 -08:00
Gerardo Salazar 629de49b10 Implement Benzinga custom data as ToolBox application and BaseData class 2019-11-13 13:38:36 -08:00
Martin Molinero 65737297c7 Adding Tiingo new data converter 2019-10-01 00:26:36 -03:00
Gerardo Salazar 1e10cf7f7e Fixes issue in SmartInsider where we could not choose where the final processed directory is. 2019-09-25 15:06:25 -07:00
Gerardo Salazar 436a72f435 Implements SmartInsider custom data source
Implements Smart Insider data converter ToolBox application

Includes example algorithms demonstrating use of data
2019-08-05 13:42:08 -07:00
Gerardo Salazar d394251ac9 Implement U.S. Treasury yield curve alternative data source 2019-07-24 16:48:55 -07:00
Gerardo Salazar 4a83793067 Address review 2019-07-09 17:46:15 -07:00
Gerardo Salazar d5e148b05c Adds Psychsignal data converter and custom data implementation 2019-07-09 17:45:31 -07:00
Gerardo Salazar cce9988b63 Update comments to accurately reflect upon the written code
Change SECDataConverter CLI argument `--start` to `--date`
2019-07-05 16:58:29 -07:00
Juan José D'Ambrosio b29b8797ad Make SECConverter to process a single day 2019-07-05 16:53:19 -07:00
Gerardo Salazar 6bade144d2 Address review - Rename CLI arg names for SECDataConverter 2019-07-05 16:53:19 -07:00
Gerardo Salazar 5cdfb568ab Improve SEC logging experience. Only serious errors will be outputted now.
Split SECDataDownloader and SECDataConverter to two ToolBox applications
2019-07-05 16:53:19 -07:00
Jack Simonson 724b11caae Update to ToolboxArgumentParser and Readme
Small updates to indicate to user that API key is required for the IEXDownloader and update to the example displayed based on recent changes
2019-06-26 12:11:35 -07:00
Gerardo Salazar 2ad1f19f85 Implements Sec Custom Data, ToolBox Application, and Sample Algorithm 2019-06-17 16:45:53 -07:00
Stefano Raggi cede89ed31 Add CoinApiDataConverter ToolBox application 2019-05-16 16:51:37 +02:00
Gerardo Salazar f9336514fd Parameterize probability of IPO, splits, dividends, renames, and
dividends every quarter.

Check for existing directory before attempting to write in MapFile
2019-04-09 15:01:55 -07:00
Gerardo Salazar 0f9cc33c3a Rename expected argument from-date/to-date to start/end
Add missing default to ToolBox help message
2019-04-02 21:25:10 -07:00
Gerardo Salazar 55f8376c53 Create mapping, IPO, delisting, splits, and dividends random generation
Created random-seed argument for rdg in order to let the user get deterministic output

Update documentation in 'FactorFileRow.cs' to accurately reflect factor
file structure

Update CSV generation for FactorFile so that it uses FactorFileRow's CSV
generator

Add FinancialCalendar to make it easier to implement logic regarding
financial quarters

Add mapping events to RandomDataGenerator

Update MapFileRow ToCsv method to correctly emit the ticker as lowercase

Fix bug in FactorFile where we would get the same initial data point twice
when we converted it to CSV

Create new method to convert a MapFile to CSV

Create new method to write MapFile to disk as CSV

Add unit test to test for successful CSV generation in MapFile

Add new files to project

Add FinancialCalendar unit tests

Create new class to handle generation of dividends, splits, and maps
2019-04-02 21:25:10 -07:00
Juan José D'Ambrosio 5993ed6375 Make Kaiko converter able to process only a selected exchange 2019-02-14 15:55:37 +00:00
Michael Handschuh 728594c277 BUG #2840: Prevent emitting trade&quote in same time step
This side steps the issue reported in #2840 by removing the desire to pass
zero for the maximum deviation. The previous issue was that we were leaning
on the trade ticks to produce the series variation and then trying to fit
quote ticks around a previously generated trade tick. This solution permits
both quote and trade ticks to produce variations and prevents the generator
from emitting both at the same time step.

A new parameter, --quote-trade-ratio, determines the relative density of each.
For example, a quote trade ratio of 1 means equal trade and quote ticks, whereas
a value of 2 means twice as many quote ticks as trade ticks.

This change also removes special treatment regarding the downsizing of the
requested deviation for quote ticks. If the consumer wants to limit the
deviation of quote ticks than the consumer can make that decision, but the
random value generator should simply follow instructions like a good little
boy.

Fixes #2840
2019-01-22 09:52:41 -05:00
Michael Handschuh 743e067c60 #2824: Add RandomDataGeneratorProgram
Invoke from toolbox cli using --app=rdg or --app=randomdatagenerator

Produces random data over the requested time frame in the desired resolution,
security type and density.

Here's a few sample command line invocations:

--app=rdg --start=20190101 --end=20200101 --symbol-count=1 --resolution=Daily --data-density=Dense --include-coarse=false
--app=rdg --start=20190101 --end=20200101 --symbol-count=10 --security-type=Future --resolution=Hour --data-density=VerySparse
--app=rdg --start=20190101 --end=20200101 --symbol-count=5 --security-type=Option --resolution=Minute --data-density=Sparse
2019-01-15 14:33:45 -05:00
Juan José D'Ambrosio 73bbf33a3e Implement single day Kaiko data conversion 2019-01-09 19:18:25 -03:00
Art 5a4e2a7551 corrected CLI prompt 2018-07-24 16:42:35 -04:00
wildart 586e500cd7 added missing doc references 2018-07-24 11:48:22 -04:00
Martin Molinero 59660a864c Updating toolbox readme 2018-07-03 18:58:54 -03:00
Martin Molinero c31fa715d7 Renaming old Program.cs of each tool 2018-06-29 21:08:13 -03:00
Martin Molinero 0db14515fe Make toolbox a single application wrapping all the tools 2018-06-28 16:15:51 -03:00