Commit Graph

52 Commits

Author SHA1 Message Date
Adalyat Nazirov a4f66628fd Lean Optimization interface in QCAlgorithm (#4923)
* initial commit

* run parametrized algorithm with command line parameters

* skeleton: top level structure

* OptimizationNodePacket scheme

* pass parameters as HashSet

* run Lean and read results

* call method on optimization completion

* refactor public interfaces

- close ParameterSet collection; allow only get operations
- explicit method to start LeanOptimizer

* synchronize RunLean method; the result could come in before the backtest id is set in the collections

* another portion of refactoring and interface changes

* comments

* comments & tests for Extremum, Minimization and Maximization classes

* unify optimization paramater values (min, max, step) & mode GridSearch tests

- swap min&max if necessary
- iterate left => right (negate step value if necessary) & provide default step value if step == 0
- no StackOverflow Exception
- parameterSet Id should be global for current generator and retain between steps
- test signle point boundary (min == max)

* BruteForceStrategy tests

* more comments

* Update Optimizer assembly information

- Update Optimizer projects assembly information to match behavior of
  the other projects

* Tweaks

- Adding comments
- Replace OnComplete for Ended event
- Replace Abort for Dispose
- ConsoleLeanOptimizer will keep track of running processes
- Each backtest will store results in a separated directory, so they
  don't fight for the log.txt file.
- Adding cmdline option for lean to close automatically
- Adding concurrent execution backtest limit
- Console optimizer will start Lean minimized
- Escape spaces in Json path

* remove parameter set generator abstraction layer

we don't need this flexibility now.

* refactor public methods; Step shouldn't be public

* constraints: wip

* define contract

* comparison operators and tests

* specify JsonProperty values

* Move SafeMultiply100 to extensions

* Throw exception on failed Optimizer.Start

* constraints: wip

* change finish & dispose process

* minor fixes

- handle force lean abort
- notify consumer if target has been reached

* target & constraints; adapt unit tests

* Minor Tweaks and fixes

- Some logging improvements
- Remove Public since not required

* Ignore empty ParameterValue

* simplify condition

* avoid reinitialization

* reduce type; force immutable

* unit tests for constraints  and target value

* parse & normalize percent values, i.e. 20% => 0.2

* fixup

* Target & Constraint & OptimizationNodePacket unit tests

* Add more json unit tests

- Adding more json conversion unit tests. Fix bug for Extremum which
  wasn't using the converter.

* LeanOptimizer tests

* Estimation results

* User thread safe counters

* LeanOptimizer unit tests; push OptimizationResult on Ended event

* more unit tests

* Minor tweaks

-Estimate ToString in a single line.
-Typos and missing header file

* Add base SendUpdate method

- Add base SendUpdate method for LeanOptimizer

* fix LeanOptimizer test; rely on internal Update rather than timer

* Add OptimizationStatus

- Add missing commments and OptimizationStatus

* EulerSearch implementation: wip

* OptimizationParameter custom converter

* change the type

* make step optional

* change folder structure

* enumerate optimization parameter using IEnumerable & IEnumerator

* unit tests: parameters & objectives

* unit tests: strategies

* remove redundant TODO

* change Euler search boundaries

* more Euler tests

* prevent race condition

* Add account/read endpoint

- Adding account/read endpoint. Adding unit test

* Add status check before running lean

* Minor self review

- Adding missing comments, minor changes

* remove array parameters

* minor changes

- tidy up config file, rename variable
- accept min less or equal than max

* move OptimizationParameter methods to strategies

* Minor improvements for BaseResultHandler derivates

* minor changes

- strict requirements for Step and MinStep values
- strategy specific settigs

* Add TotalRuntime to estimate

Co-authored-by: Martin Molinero <martin.molinero1@gmail.com>
2020-12-02 20:10:40 -03:00
Martin Molinero ed6e2fd0a2 Replace ResultPreFix config for custom algorithm id 2020-03-09 11:59:23 -03:00
Gerardo Salazar d3a34dcd70 Fixes issue where report creator could not be ran in Mac OSX
* Adds `PythonInitializer.SetPythonPathEnvironmentVariable(...?)` method
2020-02-11 15:01:47 -08:00
Martin Molinero 87bc03d71e Address reviews
- When BacktestNodePacket has the inital `CashAmount` set we will clear
all existing cash amounts and set the account currency
- Adding more unit tests
2019-12-31 13:25:58 -03:00
Martin Molinero 118c1e49cc Add data channel provider
- Adding `IDataChannelProvider` that will be used to determine if
streaming should be used
- Adding `data-channel-provider` for `config.json`
- Adding unit tests
2019-11-01 14:53:56 -03:00
Martin Molinero 9ec1a41235 Performance improvements
- `PortfolioTargetCollection` avoid calling `Count` on
ConcurrentDictionary directly -> has to take all locks
- `SecurityChanges` change Union for Concat since constructor will call
HashSet
- Make `DynamicSecurityData` hold lazy data objects
- `RegisteredSecurityDataTypesProvider` avoid looping over all
registered types, adding `TryGetType`
- `Security.Update()` will no call group by on data since this data is
already grouped by type. Adding `ContainsFillForwardData` will allows to
be lazy and not re loop through the data unless necessary
- `DefaultAlphaHandler` will use the `static`
`Enumerable.Empty<Insight>` instance when possible
- `SubscriptionSynchronizer` will be lazy to construct the
`universeData` dictionary which is not used in most of the times. Will
use `Count` vs `Any` -> `Count` is known by the dictionary
- For python algorithms `JobQueue` will respect `AlgorithmLocation`, was
using unexisting `"algorithm-path-python"`
2019-10-17 10:26:37 -03:00
Michael Handschuh 30c8772a7e Update Compression/Jupyter/Launcher/Queues/UserInterface to respect CA1304 and CA1305
Updates all occurrences of parsing/ToString-ing to go through the new
StringExtensions methods that use CultureInfo.InvariantCulture

See #3045
2019-09-12 15:30:44 -04:00
Martin Molinero f887c42638 Enable python and CSharp debugging
- Adding `DebuggerHelper` class, handles debugging initialization
- Setting the "PYTHONPATH" will be handled by the `JobQueue`
2019-08-12 22:03:13 -03:00
Martin Molinero 17995f4c86 Adding a configuration for maximum data points per chart series. Limit will only apply for backtesting result handler 2018-08-17 18:14:33 -03:00
Stefano Raggi 2fe41495d1 Set QC API UserToken for GDAX 2018-08-13 23:03:55 +02:00
Stefano Raggi 483f20c78a Change vars from static to instance in JobQueue
When running consecutive algorithms, this was causing the algorithm runner to look for the wrong file name.
2018-07-03 23:15:34 +02:00
Stefano Raggi b9ea48e1b4 Add IApi parameter to IJobQueueHandler.Initialize 2017-06-14 17:11:28 +02:00
jaredbroad cd56b5323d Tidy up the local loading of pyc files 2017-04-30 13:10:27 -04:00
jaredbroad 9de8d3a68b Merge branch 'seed-security-from-history' of https://github.com/andrewhart098/Lean into seeding 2016-12-02 00:00:03 -05:00
quant1729 f859c548f0 Fixed typos, bugs 2016-11-21 16:58:22 +01:00
quant1729 9dd5b8f4ef Updated engine: it gets history provider, data queue handler from the job now. JobQueue loads data from config. 2016-11-21 16:58:12 +01:00
jaredbroad 41dab29bc1 Shuffle ram allocation to controls; update job queues and engine usage 2016-11-21 04:43:09 -05:00
Ray Bohac e74b14227e moves Control symbol limits config to read into vairable, then assigns in either LiveNodePacket of BacktestNodePacket 2016-10-17 20:43:13 -04:00
Ray Bohac ab127aaf04 increases symbol limit on Lean local and also adds a config option to alter the limits 2016-10-17 12:31:18 -04:00
jaredbroad 2670b41be6 Bug fixes and event onload for when embed url is loaded 2016-04-11 20:06:25 -04:00
jaredbroad 632c21c99c New UX for LEAN, launched from Launcher using embedded HTML browser 2016-04-11 18:29:50 -04:00
Michael Handschuh 3610d3fc8c Jobqueue.AcknowledgeJob: Ensure final message makes it to console
If you're running without a ConsoleLogHandler then the cmd window just looks like it is dead, but really it's already finished and is waiting for Console.Read to return, but there was no message since the Log.Trace was only being piped to a file
2016-04-05 15:52:27 -04:00
Juan Manuel Alvarez b3494afc47 Constants renamed to Globals.
Globals values can be reloaded from configuration.
2016-03-12 01:07:31 -03:00
AlexCatarino 871f03cbe9 JobQueue.Language is no longer a static field
Consecutive calls of JobQueue set up with different languages did not acknowlegde the changes when Language was a static field
2016-02-22 20:08:18 -03:00
Michael Handschuh 8acb8504be Cleans up JobQueue job init parameters 2016-01-19 18:12:28 -05:00
snugs e31fd9d990 Read parameters from config in JobQueue 2015-12-10 13:45:38 -05:00
snugs be950e756a Log full errors/stack traces
Many places in the code used Log.Error(err.Message) or equivalent which
strips out all the really useful information, such as the stack trace
and inner exceptions. Using Log.Error(exception) is the correct way to
log an error as it will correctly write all the message details, also,
by passing the full Exception object we can improve the logging in this
one place and all call sites will automatically benefit from the improvements
2015-12-02 12:35:33 -05:00
QuantConnect 98dc0e37b7 Build script and language config for Iron python based algorithms 2015-07-05 21:00:01 -04:00
snugs c8b4e6bdba Remove config 'local' value
This change includes addition of the 'tradier-save-tokens' config value used to write tradier access/refresh tokens to disk
2015-06-22 19:05:04 -04:00
snugs 470d51a6d0 Extract sourcing of job from Engine.Run
The engine itself shouldn't concern itself with where jobs come from, so we pull the next job and acknowledge job logic out of Engine.Run
Add RamAllocation to AlgorithmNodePacket
2015-06-22 18:28:23 -04:00
Stefano Raggi 0b1396449c Replaced "language" config setting with "algorithm-location" 2015-06-19 10:09:41 +02:00
snugs e05e4fd08e Load data feed handlers via MEF configuration 2015-06-17 22:34:05 -04:00
snugs fd3d5b2953 Load transaction handlers via MEF configuration 2015-06-17 22:34:02 -04:00
snugs 42a77acce6 Load setup handlers via MEF configuration 2015-06-17 22:34:00 -04:00
QuantConnect cc569f0088 Added visual basic algorithm template and language switch to config 2015-06-17 21:43:13 -04:00
QuantConnect d53e7b3ba0 Initial FSharp implementation running basic template 2015-06-17 18:31:36 -04:00
snugs 9e7986ee27 Adds minimalist charting to ConsoleResultHandler
This is not really charting, but it saves a csv for each plot produced by an algorithm which can be easily loaded into your favorite charting program
2015-06-16 16:25:16 -04:00
QuantConnect fde0cbed3a Strip out block quote section header comments 2015-06-13 16:04:27 -04:00
QuantConnect ca4a49e70d Rework GetResultHandler to use type loader; first desktop UX implementation; functional logger 2015-06-07 03:08:33 -04:00
snugs 89b6e98af8 Fixes warnings mostly relating to xml comments
Also cleans up some block comments

Via PR#72 - thanks @ammachado!
2015-05-13 10:56:47 -04:00
snugs 12379b13fc Remove multiple enumerations from live data feed, fix local paper trading 2015-05-12 18:16:24 -04:00
snugs 42fee8a70d Ack jobs that are out of version
Clean up console set up handler's resetting of job endpoints
2015-04-28 16:39:30 -04:00
QuantConnect 1abca6998a Invert backtestMode flag to inline with other naming conventions; liveMode 2015-04-18 00:56:33 -04:00
QuantConnect 341e2e7a3e Job and setup handler bug fixes 2015-04-15 13:50:36 -04:00
snugs fa9148d7cf Simplifies retrieval of BrokerageData imports 2015-04-15 10:38:43 -04:00
snugs ab9f1e9c93 Various updates to ready Lean for public live trading
IBrokerage.GetCashBalance() returns balance in each currency
BrokerageSetupHandler sets security prices to avg price to avoid immediate margin calls
2015-04-14 20:42:16 -04:00
snugs d96f477f0f Removes PaperTradingSetupHandler 2015-04-14 16:39:00 -04:00
snugs dd84c87ca8 Adds Paper and Backtesting brokerage factories
Adds system of MEFing in the brokerage data for running locally (IBrokerageFactory.BrokerageData)
2015-04-14 16:22:43 -04:00
QuantConnect af04e2b472 Job queue init method cut out 2015-04-07 16:36:29 -04:00
QuantConnect a42b0bd5ad namespaces 2015-04-07 14:36:14 -04:00