79 Commits

Author SHA1 Message Date
Jhonathan Abreu 480790e3f3 Add messaging handler initialize parameter to job queue (#8887)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Syntax Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
2025-07-24 09:27:19 -04:00
Jhonathan Abreu 2e9cc39c89 Rename Controls.StoragePermission -> Controls.StorageAccess (#8862)
* Refactor Controls.StoragePermissions format for the API

* Minor changes

* Rename Controls.StoragePermission -> Controls.StorageAccess

* Minor fix
2025-07-09 11:11:27 -04:00
JosueNina 085476a222 job_queue: fix typos in comments (#8390)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Corrected minor typos in comments within the JobQueue class to improve
readability

- Verify all comments for further errors
2024-11-01 19:13:49 -03:00
Ricardo Andrés Marino Rojas 554a82527f Fix minor warnings (#8190)
* Fix CA1847 warnings and some CA1307 warnings

* Fix CA1823 and CS0169

* Fix CA1820 warnings

* Address requested changes
2024-07-10 14:17:27 -03:00
Ricardo Andrés Marino Rojas 16a737ee26 Fix most of CA1725 warnings (#8154)
* Fix CA1725 warning

* Address requested changes
2024-07-05 16:19:56 -03:00
Martin-Molinero e375afbe20 Dynamically setup brokerage history provider wrapper (#7802)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Report Generator Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-02-21 12:03:31 -03:00
Martin-Molinero 2a82c6f9bb Consolidate Configuration.Get usages (#7796)
API Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-02-20 13:37:22 -03:00
Martin-Molinero 67b9ecf391 Minor config tweaks (#7792)
Benchmarks / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
API Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
2024-02-19 12:05:00 -03:00
Martin-Molinero e0fdbb2618 Plotting Limits Handling Improvements (#7657)
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
* Plotting Limits Handling Improvements

- Improve the handling and enforcement of plotting limits
- Increase local default limits

* Minor plotting test fix
2024-01-03 14:21:13 -03:00
Ricardo Andrés Marino Rojas d8a68afc3f Add Support to Run Algos Without Specifying the Algorithm Language (#7643)
* First draft  of the feature

* Improve implementation

* Add more unit tests
2023-12-26 19:18:54 -03:00
Martin-Molinero 4b06ca9147 Out of sample support tweaks (#7542)
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* Add backtest job end date handling

* Fixes & rebases

* Refactor out of sample configuration handling

* Minor datetime format tweak
2023-10-27 18:23:21 -03:00
Sebastian Karasek 6d35bfd3bc Ignore duplicate brokerage data (#7487) 2023-09-25 13:09:43 -03:00
Jhonathan Abreu af0de8d2e9 Deployment target and running mode algorithm properties (#7337)
* Add algorithm's mode and deployment target properties source from job packet

* Address peer review

* Add Python regression algorithm
2023-07-03 09:55:21 -04:00
Ronit Jain 0d6abf5dde Add optimization-id for optimization backtests (#7202)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
* add organizatin-id in backtests

* self review
2023-04-20 10:49:04 -03:00
Martin-Molinero 52de357b27 Terminal Link Minor Fixes (#7093)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Fix data-queue-handler configuration initialization using paper brokerage
- Fix for downloader data provider request times
- Fix null reference exception in live result handler if algorithm is
  null
2023-03-20 12:24:16 -03:00
Ronit Jain c21dd716ed use data channel property (#6826)
Python Virtual Environments / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
2022-12-26 15:23:18 -03:00
Jhonathan Abreu 8f3bdcfe6c Fix adding Python additional paths in research (#6699)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add python additional paths for research

* Delegate adding additional paths to pythonpath to PythonInitializer

* Add new way to keep algorithm location at the beginning of pythonpath

* Add switch to disable BeginAllowThreads in PythonInitializer

* Add switch to disable BeginAllowThreads in PythonInitializer

* Address changes request
2022-10-21 12:11:10 -03:00
Jhonathan Abreu 6a0fdd63a8 Add algorithm directory before any additional paths to pythonpath (#6674)
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
2022-10-06 20:04:47 -03:00
Jhonathan Abreu a0491a9b74 python-additional-paths config file setting (#6667)
Regression Tests / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Add python-additional-paths setting in config file

* Configure python additional paths in JobQueue instead of PythonInitializer
2022-09-30 16:52:17 -03:00
Martin-Molinero 250230e79f Global object store (#6594)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
* Refactor object store

* Address reviews

* Simplify implementation

- Remove base64 encoding
- Refactor object store api to allow a single path argument

* Fix bug

* Adjust storage controls

* Fix unit tests
2022-08-30 16:40:01 -03:00
Martin-Molinero 6f3208eded Python Virtual Environments Support (#6556)
Regression Tests / build (push) Has been cancelled
Python Virtual Environments / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Add support to use a python virtual environment if defined in the
  algorithm job. Adding tests
2022-08-17 16:40:52 -03:00
Ronit Jain 9df9d76081 get value from config, default to local (#6502)
Research Regression Tests / build (push) Has been cancelled
Benchmarks / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Regression Tests / build (push) Has been cancelled
2022-07-22 11:08:46 -03:00
Martin-Molinero 867682c923 Set default object store settings from config (#6437)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
Research Regression Tests / build (push) Has been cancelled
- Will set default object store settings from config like research
2022-06-24 18:44:00 -03:00
Ronit Jain 264c3c8374 Composite IDQH - Support multiple live data feeds (#6047)
* initial commit

* Follow IDQH implementation

* Expect a list of data handlers from LiveNodePacket

* Return null if can not subscribe

* refctor to add check for subscription

* initialze null

* Add tests

* Check subscribe retuns null/not-null

* cleanup

* Read all required IDQH credentails to job

* Use CDQH to handle all IDQH instances

* constructor abstraction to call from setjob

* use flag

* remove redundant because derived will call initialize on it

* abstract and initialize from setjob

* handle null enumerators

* get creds from data handlers

* handle single data handler value from data-queue-handler

* Fix to support a json array

* Fix missed constructor call

* change access modifier to access from Tests files

* Add test to get brokerageFactory from dataQueueHandler

* Fix init flag to handle all conditions

* Add docs

* initialize from setjob

* Check if websocket open before using

* change defination of initialzie to include tradier

* clean up

* change defination

* fix wrong api key name

* return empty enumerator

* check websocket open before sending request

* check connection before subscribing

* fix to include more cases

* check websocket open before sending request

* Minor refactoring

* reafctor and use IsConnected

* remove unused

* clean up

* Fix test cases

* reverse change

* include config changes

* connect to websocket from setjob

* check websocket connection from setjob

* clean up

* include condition for IDQH that are not brokerage

* Address review

* Add market check condition before subscribe

* Remove deprecated

* Minor fix for deserializing data queue handler

Co-authored-by: Martin-Molinero <martin@quantconnect.com>
2021-12-03 16:18:41 -03:00
Martin-Molinero 7be1b7db1a Algorithm Language can be case insensitive (#5924)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-09-13 10:31:31 -03:00
Martin-Molinero 12b62f6ec9 Add organization id to packet (#5733)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
2021-07-02 20:59:59 -03:00
Colton Sellers 15555f710d Python Loading and Debugging Refactor (#5670)
Regression Tests / build (push) Has been cancelled
Build & Test Lean / build (push) Has been cancelled
* Use Python algorithms directly in their location; also refactor pythonpath additions to one method

* Update VSCode Launch options

* Drop issue these changes fixes

* Use AddRange
2021-06-16 20:30:33 -03:00
Adalyat Nazirov a4f66628fd Lean Optimization interface in QCAlgorithm (#4923)
* initial commit

* run parametrized algorithm with command line parameters

* skeleton: top level structure

* OptimizationNodePacket scheme

* pass parameters as HashSet

* run Lean and read results

* call method on optimization completion

* refactor public interfaces

- close ParameterSet collection; allow only get operations
- explicit method to start LeanOptimizer

* synchronize RunLean method; the result could come in before the backtest id is set in the collections

* another portion of refactoring and interface changes

* comments

* comments & tests for Extremum, Minimization and Maximization classes

* unify optimization paramater values (min, max, step) & mode GridSearch tests

- swap min&max if necessary
- iterate left => right (negate step value if necessary) & provide default step value if step == 0
- no StackOverflow Exception
- parameterSet Id should be global for current generator and retain between steps
- test signle point boundary (min == max)

* BruteForceStrategy tests

* more comments

* Update Optimizer assembly information

- Update Optimizer projects assembly information to match behavior of
  the other projects

* Tweaks

- Adding comments
- Replace OnComplete for Ended event
- Replace Abort for Dispose
- ConsoleLeanOptimizer will keep track of running processes
- Each backtest will store results in a separated directory, so they
  don't fight for the log.txt file.
- Adding cmdline option for lean to close automatically
- Adding concurrent execution backtest limit
- Console optimizer will start Lean minimized
- Escape spaces in Json path

* remove parameter set generator abstraction layer

we don't need this flexibility now.

* refactor public methods; Step shouldn't be public

* constraints: wip

* define contract

* comparison operators and tests

* specify JsonProperty values

* Move SafeMultiply100 to extensions

* Throw exception on failed Optimizer.Start

* constraints: wip

* change finish & dispose process

* minor fixes

- handle force lean abort
- notify consumer if target has been reached

* target & constraints; adapt unit tests

* Minor Tweaks and fixes

- Some logging improvements
- Remove Public since not required

* Ignore empty ParameterValue

* simplify condition

* avoid reinitialization

* reduce type; force immutable

* unit tests for constraints  and target value

* parse & normalize percent values, i.e. 20% => 0.2

* fixup

* Target & Constraint & OptimizationNodePacket unit tests

* Add more json unit tests

- Adding more json conversion unit tests. Fix bug for Extremum which
  wasn't using the converter.

* LeanOptimizer tests

* Estimation results

* User thread safe counters

* LeanOptimizer unit tests; push OptimizationResult on Ended event

* more unit tests

* Minor tweaks

-Estimate ToString in a single line.
-Typos and missing header file

* Add base SendUpdate method

- Add base SendUpdate method for LeanOptimizer

* fix LeanOptimizer test; rely on internal Update rather than timer

* Add OptimizationStatus

- Add missing commments and OptimizationStatus

* EulerSearch implementation: wip

* OptimizationParameter custom converter

* change the type

* make step optional

* change folder structure

* enumerate optimization parameter using IEnumerable & IEnumerator

* unit tests: parameters & objectives

* unit tests: strategies

* remove redundant TODO

* change Euler search boundaries

* more Euler tests

* prevent race condition

* Add account/read endpoint

- Adding account/read endpoint. Adding unit test

* Add status check before running lean

* Minor self review

- Adding missing comments, minor changes

* remove array parameters

* minor changes

- tidy up config file, rename variable
- accept min less or equal than max

* move OptimizationParameter methods to strategies

* Minor improvements for BaseResultHandler derivates

* minor changes

- strict requirements for Step and MinStep values
- strategy specific settigs

* Add TotalRuntime to estimate

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-12-02 20:10:40 -03:00
Martin Molinero ed6e2fd0a2 Replace ResultPreFix config for custom algorithm id 2020-03-09 11:59:23 -03:00
Gerardo Salazar d3a34dcd70 Fixes issue where report creator could not be ran in Mac OSX
* Adds `PythonInitializer.SetPythonPathEnvironmentVariable(...?)` method
2020-02-11 15:01:47 -08:00
Martin Molinero 87bc03d71e Address reviews
- When BacktestNodePacket has the inital `CashAmount` set we will clear
all existing cash amounts and set the account currency
- Adding more unit tests
2019-12-31 13:25:58 -03:00
Martin Molinero 118c1e49cc Add data channel provider
- Adding `IDataChannelProvider` that will be used to determine if
streaming should be used
- Adding `data-channel-provider` for `config.json`
- Adding unit tests
2019-11-01 14:53:56 -03:00
Martin Molinero 9ec1a41235 Performance improvements
- `PortfolioTargetCollection` avoid calling `Count` on
ConcurrentDictionary directly -> has to take all locks
- `SecurityChanges` change Union for Concat since constructor will call
HashSet
- Make `DynamicSecurityData` hold lazy data objects
- `RegisteredSecurityDataTypesProvider` avoid looping over all
registered types, adding `TryGetType`
- `Security.Update()` will no call group by on data since this data is
already grouped by type. Adding `ContainsFillForwardData` will allows to
be lazy and not re loop through the data unless necessary
- `DefaultAlphaHandler` will use the `static`
`Enumerable.Empty<Insight>` instance when possible
- `SubscriptionSynchronizer` will be lazy to construct the
`universeData` dictionary which is not used in most of the times. Will
use `Count` vs `Any` -> `Count` is known by the dictionary
- For python algorithms `JobQueue` will respect `AlgorithmLocation`, was
using unexisting `"algorithm-path-python"`
2019-10-17 10:26:37 -03:00
Michael Handschuh 30c8772a7e Update Compression/Jupyter/Launcher/Queues/UserInterface to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to go through the new
StringExtensions methods that use CultureInfo.InvariantCulture

See #3045
2019-09-12 15:30:44 -04:00
Martin Molinero f887c42638 Enable python and CSharp debugging
- Adding `DebuggerHelper` class, handles debugging initialization
- Setting the "PYTHONPATH" will be handled by the `JobQueue`
2019-08-12 22:03:13 -03:00
Martin Molinero 17995f4c86 Adding a configuration for maximum data points per chart series. Limit will only apply for backtesting result handler 2018-08-17 18:14:33 -03:00
Stefano Raggi 2fe41495d1 Set QC API UserToken for GDAX 2018-08-13 23:03:55 +02:00
Stefano Raggi 483f20c78a Change vars from static to instance in JobQueue
When running consecutive algorithms, this was causing the algorithm runner to look for the wrong file name.
2018-07-03 23:15:34 +02:00
Stefano Raggi b9ea48e1b4 Add IApi parameter to IJobQueueHandler.Initialize 2017-06-14 17:11:28 +02:00
jaredbroad cd56b5323d Tidy up the local loading of pyc files 2017-04-30 13:10:27 -04:00
jaredbroad 9de8d3a68b Merge branch 'seed-security-from-history' of https://github.com/andrewhart098/Lean into seeding 2016-12-02 00:00:03 -05:00
quant1729 f859c548f0 Fixed typos, bugs 2016-11-21 16:58:22 +01:00
quant1729 9dd5b8f4ef Updated engine: it gets history provider, data queue handler from the job now. JobQueue loads data from config. 2016-11-21 16:58:12 +01:00
jaredbroad 41dab29bc1 Shuffle ram allocation to controls; update job queues and engine usage 2016-11-21 04:43:09 -05:00
Ray Bohac e74b14227e moves Control symbol limits config to read into vairable, then assigns in either LiveNodePacket of BacktestNodePacket 2016-10-17 20:43:13 -04:00
Ray Bohac ab127aaf04 increases symbol limit on Lean local and also adds a config option to alter the limits 2016-10-17 12:31:18 -04:00
jaredbroad 2670b41be6 Bug fixes and event onload for when embed url is loaded 2016-04-11 20:06:25 -04:00
jaredbroad 632c21c99c New UX for LEAN, launched from Launcher using embedded HTML browser 2016-04-11 18:29:50 -04:00
Michael Handschuh 3610d3fc8c Jobqueue.AcknowledgeJob: Ensure final message makes it to console
If you're running without a ConsoleLogHandler then the cmd window just looks like it is dead, but really it's already finished and is waiting for Console.Read to return, but there was no message since the Log.Trace was only being piped to a file
2016-04-05 15:52:27 -04:00
Juan Manuel Alvarez b3494afc47 Constants renamed to Globals.
Globals values can be reloaded from configuration.
2016-03-12 01:07:31 -03:00