Commit Graph

153 Commits

Author SHA1 Message Date
Jared Broad e86155c117 Shuffle init to set environment path inline 2019-07-28 15:12:54 -07:00
Jared Broad 2dd94f3687 Optimize python load times 2019-07-28 10:33:37 -07:00
Gerardo Salazar 2033c7873b Update outdated DotNetZip dependency in nuspec files 2019-07-02 18:02:23 -07:00
Gerardo Salazar 0ea16eb53b Fix bad env var from being created when PYTHONPATH is empty 2019-07-01 17:34:02 -07:00
Martin Molinero a92bee5bf1 Fix OnEndOfDay not fired
- Adding `IRealTimeHandler.OnSecurityChanged()` will be used to update
the `OnEndOfDay` security related scheduled events
- Adding `BaseRealTimeHandler.cs` to reduce code duplication in the
`Backtesting` and `LiveTrading` `RealTimeHandlers`
- Adding CSharp and Python regression tests
- Deprecating `OnEndOfDay()` callback because of two reasons, mainly
because Python does not support two methods with the same name, but also
because different assets have different market close times.
- `ScheduledEvents` set at the same time will now be deterministic
2019-06-20 17:35:28 -03:00
Jared 66c66abb12 Implement API download for QCALgorithm (#3215)
* Implementing download proxy for LEAN

* Remove accidentally added files

* Remove accidentally added files

* Tidy up
2019-05-20 18:34:19 -07:00
Martin Molinero 4382aa1886 Add algorithm thread
- Adding `WorkerThread` class, wrapper for a worker thread that will
execute given `Actions`.
    - Algorithm related code (`Construction`, `Initialization`,
    `Execution` will be executed by the same `WorkerThread` instance,
    this is required for `Python` debugging.
2019-05-09 11:23:28 -03:00
AlexCatarino c3de660551 Updates pythonnet to 1.0.5.20 2019-04-23 23:15:14 +01:00
Martin Molinero fa122fa809 Add missing PyObject.Dispose calls
- Adding _some_ of the missing PyObject.Dispose calls. In the cases
where C# is calling the Python side.
   - Note that Python calls to C# code is correctly handling the
   disposure of resources.
2019-04-10 15:03:10 -03:00
Martin Molinero cfa08a11fb Address reviews
- Removing `using QCAlgorithmFramework = QuantConnect.Algorithm.QCAlgorithm`
- Removing `QCAlgorithmFrameworkBridge`
- Removing `IsFrameworkAlgorithm`
- Making `EmitInsightBasedOnFill` private. Adding new
`IOrderEventProvider` exposing an `event` to which `QCAlgorithm` will
subscribe.
- `AccountType.Cash` algorithms will be allowed to manually trade and
emight insights manually or with alpha model.
2019-04-03 21:55:44 -03:00
Martin Molinero 32ac3146b4 Merge Framework and Classic Algorithms
- Merging Framework and Bridge algorithms into classic QCAlgorithm
class.
- Removing Framework project, VS17 and VS15
2019-04-03 21:54:32 -03:00
AlexCatarino da6519b9f4 Updates pythonnet to 1.0.5.19
Updates PythonNet to 1.0.5.19
 - Related  https://github.com/QuantConnect/pythonnet/pull/28/
2019-04-02 17:17:40 +01:00
Jared c5d4be415c Merge pull request #3033 from QuantConnect/feature-3032-update-pythonnet
Updates pythonnet to 1.0.5.18
2019-03-28 11:11:39 -07:00
AlexCatarino 6d2460d8ed Updates pythonnet to 1.0.5.18
- Updates PythonNet to 1.0.5.18
  - Cherry picks the memory leak fix from upstream/pythonnet
2019-03-28 00:47:38 +00:00
Martin Molinero 5b04cfe202 Emiting Insights based on Fills
- Classic Algorithms will emight insights based on order fills.
   - To be able to update generated insights closed time, we will not
   clone emitted insights.
   - `InsightAnalysisContext` will update `AnalysisEndTimeUtc` when the
   Insight period is closed and the period is `EndOfTimeTimeSpan`
- Adding new regression algorithm asserting on the new emitted insights
- Adding unit tests
- `LiveTradingResultHandler` will store `AlphaRunTimeStatistics`
- Making `DefaultAlphaHandler.ProcessAsynchronousEvents` virtual to
facilitate cloud changes
2019-03-26 16:09:22 -03:00
AlexCatarino d91cad3123 Updates pythonnet to 1.0.5.17
- Updates PythonNet to 1.0.5.17
  -  Improve performance by adding new `interop` `type` cache holding a `bool`, true if its an `exception`. And adding a `setter` and `getter` cache for the `propertyobject`. Closes #2925.
  - Decimal parsing allows numeric string in exponential notation. Closes #2918 #2919.

Closes #2929
2019-02-22 17:46:59 +00:00
Martin Molinero 1b0bdd9b0b Adding SetAccountCurrency for backtesting
- Adding new `SetAccountCurrency()` for backtesting. Has to be called
before adding any `Security` or calling `SetCash()`, else will throw.
- Adding new Non account currency unit tests for `CashBuyingPower`,
`SecurityPortfolioModel`, `SecurityMarginModel`,
`SecurityPortfolioManager`, `Future/OptionMarginBuyingPowerModels`
- Adding new C# regression test using `SetAccountCurrency()`, one for
`CashBuyingPowerModel` and one for `SecurityMarginModel`
- Adding new Py and C# basic regression algorithms using
`SetAccountCurrency()`
- `Options` and `Futures` will use not use `AccountCurrency` as quote
Cash.
- `SecurityBenchmark` value will be in account currency
2019-01-25 14:54:43 -03:00
Martin Molinero d53a922cf0 PythonNet installation does not require copy pasting
- Requires a new PythonNet 1.0.5.15 package where the different `.dll` are in a
specific folder: `\win` `\linux` and `\osx`
- Removed not present `decimal.py` from `Algorithm.Python` project. It
was moved into `Common`.
- Replace `xbuild` for `msbuild` required for using the `System.Runtime.InteropServices`.
Also note the `xbuild` on travis prints:
> >>>> xbuild tool is deprecated and will be removed in future updates, use msbuild instead <<<<
2019-01-22 19:48:11 -03:00
AlexCatarino ef59fa7ba2 Updates pythonnet package.
In the new package:
- C# decimal conversion will use C# double and python float due to the big performance impact of converting C# decimal to python decimal;
2019-01-18 23:18:35 +00:00
Martin Molinero b60df26852 Keep OnData and OnOrderEvent python reference
- `AlgorithmPythonWrapper` will now keep reference to the `OnData` and
`OnOrderEvent` `PyObject` method, giving a performance improvement,
since it does not have to resolve it in each loop.
2019-01-09 19:32:19 -03:00
Martin Molinero a993c4932a Address reviews
- Some performance improvements
- Method renaming
2018-12-21 19:27:22 -03:00
Martin Molinero 65ea963a26 Using new HistoryRequestFactory
- BaseSetupHandler will use new `HistoryRequestProvider` class
2018-12-19 16:15:29 -03:00
Martin Molinero af8e6bf85c Fix starting capital for non usd cash
- Moving `UniverseSelection.EnsureCurrencyDataFeeds` call into the
`IResultHandler` implementation through usage of the new `SetupHandlerHelper`
class, that will also set an initial conversion rate if none present.
- Adding regression test, that reproduces original issue
2018-12-19 16:15:28 -03:00
Jared 86807d3f04 Merge pull request #2298 from AlexCatarino/bug-2288-pythonnet-memory-leak
Updates pythonnet
2018-12-18 16:44:53 -08:00
Martin Molinero 75ffa300f1 Adding new IAlgorithm.AccountCurrency
- Adding new `IAlgorithm.AccountCurrency { get; }` that will point to the
`Portfolio.CashBook.AccountCurrency`. Setter will be added in a
following PR.
- Base `Brokerage` class will now have a `AccountCurrency { get }`
pointing to the `IAccountCurrencyProvider`. Will be used by the different
brokerages implementations.
2018-12-10 11:59:02 -03:00
AlexCatarino 823bb01438 Updates pythonnet
Includes memory leak fix (https://github.com/QuantConnect/pythonnet/commit/c6db86653e7fa4fa89e1f8404d72e346b67ed857 and https://github.com/QuantConnect/pythonnet/commit/bec9563d2958acc5adf3e8972b23609fe3914402)

- Fix python version to 3.6.6 (avoid updating to 3.7)
- Fix numpy version to 1.14.5 (tensorflow requirement)

- Adds py-earth python package (closes #2399).
2018-12-07 09:36:48 +00:00
Martin Molinero 8ac4b37e5f Load algorithm faster
- Make `Isolator.ExecuteWithTimeLimit()` sleep interval configurable.
- `Loader` will now use 50ms
2018-10-18 20:08:25 -03:00
Michael Handschuh cb2e5cdc5b Expose IAlgorithm.CurrentSlice
IAlgorithm exposed a means of setting the CurrentSlice but not a
means for consumers to get the value. This was because until now
all consumers were within the QCAlgorithm scope and had access to
the member variable. This change makes the CurrentSlice available
to LEAN engine code, where it's first use will be in PaperBrokerage
to detect and apply dividend distributions.
2018-10-04 14:00:43 -04:00
Stefano Raggi 5c9b81cef1 Fix Python initialization and imports for multiple tests 2018-10-02 20:28:13 +02:00
Martin Molinero 008c9ca873 Fixing unit tests 2018-09-28 15:44:05 -03:00
Martin Molinero 6da57a1e12 Improving GetMaximumOrderQuantityForTargetValue 2018-08-03 16:26:14 -03:00
AlexCatarino 8e3b7153b2 Rebase with pythonnet/master and fix datetime UTC bug
Updates pythonnet with pythonnet/master and fix datetime issue ([65ac279](https://github.com/QuantConnect/pythonnet/commit/65ac279a03dbded39fe3e28bb79c233f0c970f6f)).
2018-07-16 11:52:54 +01:00
mblouin02 c39537e24d Fix loader error message when algorithm not found
When the algorithm type in config.json was not found, Loader.cs
would throw a confusing error message. Modified the error message
to make it clearer.

Previous error message: "Unable to resolve multiple algorithm types
to a single type...". New error message: "Algorithm type name not
found,or unable to resolve multiple algorithm types to a single type...".
2018-07-04 23:31:05 -04:00
Stefano Raggi 76586717cb Enable running multiple Python regression algorithms together
Previously when running two or more Python algorithms, the second test was crashing with a Python Fatal Error.
2018-07-03 20:51:29 +02:00
AlexCatarino e9899e357d Improves Exception Messaging For Loader
Improves the message when the Loader cannot resolve the algorithm to load. It happens when the assemblies don't have a QCAlgorithm class that match the algorithm name or you have 2-of them so Lean doesn't know which one to backtest.

The possible Loader exceptions are thrown as `AlgorithmSetupException` to mach the pattern for exceptions during initialization.
2018-06-28 13:27:08 +01:00
Stefano Raggi bbfe7a9beb Rename LatestSlice to CurrentSlice 2018-06-05 19:05:40 +02:00
Stefano Raggi 08f28a55cf Make the latest Slice received in OnData available to the algorithm 2018-06-01 18:24:09 +02:00
AlexCatarino 9705c8d4d1 Fixes python path order
The first place python should look for scripts is not the current directory, but the directory defined at "algorithm-location".
2018-04-19 11:35:42 +01:00
AlexCatarino 36abacce11 Ensure full directory paths 2018-04-18 11:46:13 +01:00
AlexCatarino b8ea13be2a Adds more locations to PYTHONPATH
Before loading python algorithms, set `PYTHONPATH` with `CurrentDirectory`, algorithm location and current/environment `PYTHONPATH`.
2018-04-17 22:31:48 +01:00
Juan José D'Ambrosio bda5b11cc2 Nuspec files added for all projects.
Launcher project is packed as `QuantConnect.Lean`but as a mean ot having all Lean features just calling one package. 

In the same sense, `QuantConnect.Algorithm.CSharp` is included as package and added as dependecy in the `QuantConnect.Lean` package just to have a working example aailable out-of-the-box.
2018-04-13 07:28:34 -03:00
Michael Handschuh dd764ad8e9 Rename InsightCollection->GeneratedInsightsCollection
This type is just used as a container for generatd insights. Renaming in
preparation for a new InsightCollection to mirror the PortfolioTargetCollection
2018-04-10 19:24:36 -04:00
AlexCatarino a4db921b87 Updates pythonnet
Rebase with pythonnet/master
2018-04-09 18:59:59 +01:00
AlexCatarino 5963f7e372 Refactors OnFrameworkData in AlgorithmPythonWrapper
`AlgorithmPythonWrapper.OnFrameworkData` should call the base class method (`QCAlgorithmFramework.OnFrameworkData`) directly instead of trying to call this method from the python script
2018-03-28 10:44:55 +01:00
Jared c896e08656 Merge pull request #1720 from QuantConnect/bug-1689-apply-security-changes-at-end-of-time-step
Apply security changes at end of time step
2018-03-20 15:48:02 -04:00
Michael Handschuh 27b34a4274 Add IAlgorithm.OnEndOfTimeStep to batch universe changes
We had an issue with the data feed picking up universe/security changes
too quickly, thereby preventing user code from being able to configure the
security object properly. Specifically, users were having an issue setting
the data normalization mode of options and underlying equity securities. By
the time the user code had set the data mode, the data feed had already
created a subscription and began processing it, so the changes were never
seen in the data feed.

This change moves all security/universe changes into pending lists and at
the end of the time step applies those changes. Security objects are still
added directly to the SecurityManager for instance access, but we delay in
adding the security to the universe and the universe to the UniverseManager.
Once added to the universe manager, an event is fired and the data feed will
process the new subscriptions.
2018-03-20 10:10:29 -04:00
AlexCatarino d5deabcb29 Fixes AlgorithmPythonWrapper.InsightsGenerated
This event was not properly wrapped.
When `IAlgorithm.InsightsGenerated` is set in `IAlphaHandler`, it should be directed to the base algorithm, whereas it was set to a null variable.
2018-03-19 11:15:57 +00:00
AlexCatarino 4e3577939d Refactors AlgorithmPythonWrapper
Move the logic of importing the module into AlgorithmPythonWrapper where it is wrapped.
Throws an exception if the script does not have a class that inherits from either QCAlgorithm not QCAlgorithmFramework.
Adds a check for OnData being defined in the module. If not, OnData from the base class will not be called (it causes stack overflow otherwise)
2018-03-12 23:36:39 +00:00
AlexCatarino 5d82c32040 Removes the abstract classifier of QCAlgorithmFramework
This change is required because pythonnet cannot initialize abstract types. Loader.cs was modified to avoid loading QCAlgorithmFramework type.
2018-03-12 22:42:42 +00:00
Michael Handschuh 6b239674e2 Renames Alpha -> Insight
The term 'alpha' is used to describe the entire algorithm. Therefore, 'alpha'
produces insights. From this we have things like IAlphaModel, which is the model
defining how insights are produced. We have IAlphaHandler, which defines how the
insights from a single 'alpha' (the algorithm) are managed, analyzed, and stored.
Types closer to the individual prediction level, such as InsightDirection, or
InsightScore relate directly to exactly 1 insight. The distinction between the
two became more clear as we developed the insights API, and from that effort it
was decided to harmonize alpha/insight terminology across the various QC systems.
2018-03-09 16:12:56 -05:00