Commit Graph

191 Commits

Author SHA1 Message Date
Jared c5d4be415c Merge pull request #3033 from QuantConnect/feature-3032-update-pythonnet
Updates pythonnet to 1.0.5.18
2019-03-28 11:11:39 -07:00
AlexCatarino 6d2460d8ed Updates pythonnet to 1.0.5.18
- Updates PythonNet to 1.0.5.18
  - Cherry picks the memory leak fix from upstream/pythonnet
2019-03-28 00:47:38 +00:00
Martin Molinero 5b04cfe202 Emiting Insights based on Fills
- Classic Algorithms will emight insights based on order fills.
   - To be able to update generated insights closed time, we will not
   clone emitted insights.
   - `InsightAnalysisContext` will update `AnalysisEndTimeUtc` when the
   Insight period is closed and the period is `EndOfTimeTimeSpan`
- Adding new regression algorithm asserting on the new emitted insights
- Adding unit tests
- `LiveTradingResultHandler` will store `AlphaRunTimeStatistics`
- Making `DefaultAlphaHandler.ProcessAsynchronousEvents` virtual to
facilitate cloud changes
2019-03-26 16:09:22 -03:00
AlexCatarino d91cad3123 Updates pythonnet to 1.0.5.17
- Updates PythonNet to 1.0.5.17
  -  Improve performance by adding new `interop` `type` cache holding a `bool`, true if its an `exception`. And adding a `setter` and `getter` cache for the `propertyobject`. Closes #2925.
  - Decimal parsing allows numeric string in exponential notation. Closes #2918 #2919.

Closes #2929
2019-02-22 17:46:59 +00:00
Martin Molinero 1b0bdd9b0b Adding SetAccountCurrency for backtesting
- Adding new `SetAccountCurrency()` for backtesting. Has to be called
before adding any `Security` or calling `SetCash()`, else will throw.
- Adding new Non account currency unit tests for `CashBuyingPower`,
`SecurityPortfolioModel`, `SecurityMarginModel`,
`SecurityPortfolioManager`, `Future/OptionMarginBuyingPowerModels`
- Adding new C# regression test using `SetAccountCurrency()`, one for
`CashBuyingPowerModel` and one for `SecurityMarginModel`
- Adding new Py and C# basic regression algorithms using
`SetAccountCurrency()`
- `Options` and `Futures` will use not use `AccountCurrency` as quote
Cash.
- `SecurityBenchmark` value will be in account currency
2019-01-25 14:54:43 -03:00
Martin Molinero d53a922cf0 PythonNet installation does not require copy pasting
- Requires a new PythonNet 1.0.5.15 package where the different `.dll` are in a
specific folder: `\win` `\linux` and `\osx`
- Removed not present `decimal.py` from `Algorithm.Python` project. It
was moved into `Common`.
- Replace `xbuild` for `msbuild` required for using the `System.Runtime.InteropServices`.
Also note the `xbuild` on travis prints:
> >>>> xbuild tool is deprecated and will be removed in future updates, use msbuild instead <<<<
2019-01-22 19:48:11 -03:00
AlexCatarino ef59fa7ba2 Updates pythonnet package.
In the new package:
- C# decimal conversion will use C# double and python float due to the big performance impact of converting C# decimal to python decimal;
2019-01-18 23:18:35 +00:00
Martin Molinero b60df26852 Keep OnData and OnOrderEvent python reference
- `AlgorithmPythonWrapper` will now keep reference to the `OnData` and
`OnOrderEvent` `PyObject` method, giving a performance improvement,
since it does not have to resolve it in each loop.
2019-01-09 19:32:19 -03:00
Martin Molinero a993c4932a Address reviews
- Some performance improvements
- Method renaming
2018-12-21 19:27:22 -03:00
Martin Molinero 65ea963a26 Using new HistoryRequestFactory
- BaseSetupHandler will use new `HistoryRequestProvider` class
2018-12-19 16:15:29 -03:00
Martin Molinero af8e6bf85c Fix starting capital for non usd cash
- Moving `UniverseSelection.EnsureCurrencyDataFeeds` call into the
`IResultHandler` implementation through usage of the new `SetupHandlerHelper`
class, that will also set an initial conversion rate if none present.
- Adding regression test, that reproduces original issue
2018-12-19 16:15:28 -03:00
Jared 86807d3f04 Merge pull request #2298 from AlexCatarino/bug-2288-pythonnet-memory-leak
Updates pythonnet
2018-12-18 16:44:53 -08:00
Martin Molinero 75ffa300f1 Adding new IAlgorithm.AccountCurrency
- Adding new `IAlgorithm.AccountCurrency { get; }` that will point to the
`Portfolio.CashBook.AccountCurrency`. Setter will be added in a
following PR.
- Base `Brokerage` class will now have a `AccountCurrency { get }`
pointing to the `IAccountCurrencyProvider`. Will be used by the different
brokerages implementations.
2018-12-10 11:59:02 -03:00
AlexCatarino 823bb01438 Updates pythonnet
Includes memory leak fix (https://github.com/QuantConnect/pythonnet/commit/c6db86653e7fa4fa89e1f8404d72e346b67ed857 and https://github.com/QuantConnect/pythonnet/commit/bec9563d2958acc5adf3e8972b23609fe3914402)

- Fix python version to 3.6.6 (avoid updating to 3.7)
- Fix numpy version to 1.14.5 (tensorflow requirement)

- Adds py-earth python package (closes #2399).
2018-12-07 09:36:48 +00:00
Martin Molinero 8ac4b37e5f Load algorithm faster
- Make `Isolator.ExecuteWithTimeLimit()` sleep interval configurable.
- `Loader` will now use 50ms
2018-10-18 20:08:25 -03:00
Michael Handschuh cb2e5cdc5b Expose IAlgorithm.CurrentSlice
IAlgorithm exposed a means of setting the CurrentSlice but not a
means for consumers to get the value. This was because until now
all consumers were within the QCAlgorithm scope and had access to
the member variable. This change makes the CurrentSlice available
to LEAN engine code, where it's first use will be in PaperBrokerage
to detect and apply dividend distributions.
2018-10-04 14:00:43 -04:00
Stefano Raggi 5c9b81cef1 Fix Python initialization and imports for multiple tests 2018-10-02 20:28:13 +02:00
Martin Molinero 008c9ca873 Fixing unit tests 2018-09-28 15:44:05 -03:00
Martin Molinero 6da57a1e12 Improving GetMaximumOrderQuantityForTargetValue 2018-08-03 16:26:14 -03:00
AlexCatarino 8e3b7153b2 Rebase with pythonnet/master and fix datetime UTC bug
Updates pythonnet with pythonnet/master and fix datetime issue ([65ac279](https://github.com/QuantConnect/pythonnet/commit/65ac279a03dbded39fe3e28bb79c233f0c970f6f)).
2018-07-16 11:52:54 +01:00
mblouin02 c39537e24d Fix loader error message when algorithm not found
When the algorithm type in config.json was not found, Loader.cs
would throw a confusing error message. Modified the error message
to make it clearer.

Previous error message: "Unable to resolve multiple algorithm types
to a single type...". New error message: "Algorithm type name not
found,or unable to resolve multiple algorithm types to a single type...".
2018-07-04 23:31:05 -04:00
Stefano Raggi 76586717cb Enable running multiple Python regression algorithms together
Previously when running two or more Python algorithms, the second test was crashing with a Python Fatal Error.
2018-07-03 20:51:29 +02:00
AlexCatarino e9899e357d Improves Exception Messaging For Loader
Improves the message when the Loader cannot resolve the algorithm to load. It happens when the assemblies don't have a QCAlgorithm class that match the algorithm name or you have 2-of them so Lean doesn't know which one to backtest.

The possible Loader exceptions are thrown as `AlgorithmSetupException` to mach the pattern for exceptions during initialization.
2018-06-28 13:27:08 +01:00
Stefano Raggi bbfe7a9beb Rename LatestSlice to CurrentSlice 2018-06-05 19:05:40 +02:00
Stefano Raggi 08f28a55cf Make the latest Slice received in OnData available to the algorithm 2018-06-01 18:24:09 +02:00
AlexCatarino 9705c8d4d1 Fixes python path order
The first place python should look for scripts is not the current directory, but the directory defined at "algorithm-location".
2018-04-19 11:35:42 +01:00
AlexCatarino 36abacce11 Ensure full directory paths 2018-04-18 11:46:13 +01:00
AlexCatarino b8ea13be2a Adds more locations to PYTHONPATH
Before loading python algorithms, set `PYTHONPATH` with `CurrentDirectory`, algorithm location and current/environment `PYTHONPATH`.
2018-04-17 22:31:48 +01:00
Juan José D'Ambrosio bda5b11cc2 Nuspec files added for all projects.
Launcher project is packed as `QuantConnect.Lean`but as a mean ot having all Lean features just calling one package. 

In the same sense, `QuantConnect.Algorithm.CSharp` is included as package and added as dependecy in the `QuantConnect.Lean` package just to have a working example aailable out-of-the-box.
2018-04-13 07:28:34 -03:00
Michael Handschuh dd764ad8e9 Rename InsightCollection->GeneratedInsightsCollection
This type is just used as a container for generatd insights. Renaming in
preparation for a new InsightCollection to mirror the PortfolioTargetCollection
2018-04-10 19:24:36 -04:00
AlexCatarino a4db921b87 Updates pythonnet
Rebase with pythonnet/master
2018-04-09 18:59:59 +01:00
AlexCatarino 5963f7e372 Refactors OnFrameworkData in AlgorithmPythonWrapper
`AlgorithmPythonWrapper.OnFrameworkData` should call the base class method (`QCAlgorithmFramework.OnFrameworkData`) directly instead of trying to call this method from the python script
2018-03-28 10:44:55 +01:00
Jared c896e08656 Merge pull request #1720 from QuantConnect/bug-1689-apply-security-changes-at-end-of-time-step
Apply security changes at end of time step
2018-03-20 15:48:02 -04:00
Michael Handschuh 27b34a4274 Add IAlgorithm.OnEndOfTimeStep to batch universe changes
We had an issue with the data feed picking up universe/security changes
too quickly, thereby preventing user code from being able to configure the
security object properly. Specifically, users were having an issue setting
the data normalization mode of options and underlying equity securities. By
the time the user code had set the data mode, the data feed had already
created a subscription and began processing it, so the changes were never
seen in the data feed.

This change moves all security/universe changes into pending lists and at
the end of the time step applies those changes. Security objects are still
added directly to the SecurityManager for instance access, but we delay in
adding the security to the universe and the universe to the UniverseManager.
Once added to the universe manager, an event is fired and the data feed will
process the new subscriptions.
2018-03-20 10:10:29 -04:00
AlexCatarino d5deabcb29 Fixes AlgorithmPythonWrapper.InsightsGenerated
This event was not properly wrapped.
When `IAlgorithm.InsightsGenerated` is set in `IAlphaHandler`, it should be directed to the base algorithm, whereas it was set to a null variable.
2018-03-19 11:15:57 +00:00
AlexCatarino 4e3577939d Refactors AlgorithmPythonWrapper
Move the logic of importing the module into AlgorithmPythonWrapper where it is wrapped.
Throws an exception if the script does not have a class that inherits from either QCAlgorithm not QCAlgorithmFramework.
Adds a check for OnData being defined in the module. If not, OnData from the base class will not be called (it causes stack overflow otherwise)
2018-03-12 23:36:39 +00:00
AlexCatarino 5d82c32040 Removes the abstract classifier of QCAlgorithmFramework
This change is required because pythonnet cannot initialize abstract types. Loader.cs was modified to avoid loading QCAlgorithmFramework type.
2018-03-12 22:42:42 +00:00
Michael Handschuh 6b239674e2 Renames Alpha -> Insight
The term 'alpha' is used to describe the entire algorithm. Therefore, 'alpha'
produces insights. From this we have things like IAlphaModel, which is the model
defining how insights are produced. We have IAlphaHandler, which defines how the
insights from a single 'alpha' (the algorithm) are managed, analyzed, and stored.
Types closer to the individual prediction level, such as InsightDirection, or
InsightScore relate directly to exactly 1 insight. The distinction between the
two became more clear as we developed the insights API, and from that effort it
was decided to harmonize alpha/insight terminology across the various QC systems.
2018-03-09 16:12:56 -05:00
AlexCatarino cd1d45ccf0 Minor fixes
Use interpreter.GetMessageHeader(e) instead of e.Message to produce to collated exception message containing the messages from inner exceptions.
Add the DllNotFoundPythonExceptionInterpreterTests to the test project.
2018-03-06 11:02:13 +00:00
AlexCatarino 718c1d61c8 Implements DllNotFoundPythonExceptionInterpreter
This exception interpreter interprets a DllNotFoundException that is thrown when pythonnet cannot fing the python dynamic-link library in the system.
2018-03-05 22:55:11 +00:00
AlexCatarino 1e594749a9 Fixes multiple OnEndOfDay error message
The error message it was used to bypass the the error logging has been changed from `takes exactly x arguments (y given)` to `takes x positional argument but y were given`.
2018-03-02 18:37:12 +00:00
AlexCatarino 0f57c2fd43 Fixes Custom Data for Python
In order to access the custom data classes, the module containing them was added to the ObjectActivator. This was unnecessary if it wasn't a custom data algorithm.
Also, this operation would not be taken into account if the custom data class were defined after the algorithm was created: this is the case for QuantBook.
We refactor how custom data is handled: a new class was added to provide a instance creation factory that creates an instance of each python custom type.
2018-03-01 19:28:30 +00:00
Michael Handschuh 18a559943e Upgrade LEAN Python to 3.6 from 2.7
- Adds log to display the python version the algorithm is using.
- Fixes python algorithms that were failing because of small subtleties
like leading zeroes.
- Updates pythonnet with a version compiled with python 3.6 flags

- Changes in DockerfileFoundation: we now use miniconda to manage the python
environment.
- Took the opportunity to add NTLK (#1349), Tensorforce (#1369) and
PyTorch/Pyro (#1385).
- Changes readme in Algorithm.Python to show steps to install miniconda
2018-02-23 15:09:06 -05:00
Stefano Raggi a39e6a8e28 Revert "Merge pull request #1526 from AlexCatarino/python3"
This reverts commit 2e523992d0, reversing
changes made to fa48fc23ea.
2018-02-14 16:03:12 +01:00
AlexCatarino 34a271adac Updates pythonnet
In this update, we had compiled pythonnet with python 3.6 instead of 2.7 flags
2018-02-13 13:00:50 +00:00
AlexCatarino a02553c524 Adds log with python version 2018-02-13 12:57:45 +00:00
AlexCatarino 9dfc8ee3a0 Adds PythonSlice static constructor
With this constructor, SetConverter method, that was not respecting Lean pattern, is removed. The initialization of _converter field is done once the type is loaded.
2018-02-12 16:23:23 +00:00
AlexCatarino cff1243b86 Creates wrapper for Slice (python custom data)
Python algorithms with custom data requires an operation that converts a dictionary key into a attribute. In the current implementation the Slice object was converted into a python dictionary. This was not optimal, since we just need to make this conversion when the value of a key in the Slice is accessed.
This implementation proposes a wrapper for the Slice object, PythonSlice, that would just perform the operation described above when needed.
2018-02-12 15:49:09 +00:00
AlexCatarino 2cfe77d10e Fixes OnWarmupFinished not firing in python algorithms 2018-01-15 18:43:38 +00:00
Stefano Raggi 89a59351cd Add OnWarmupFinished method to QCAlgorithm
This method is being added to allow algorithms to complete initialization tasks that cannot be executed during Initialize, such as cancelling existing open orders in live trading.
This method will be called only once, when the warmup task is complete.

Closes #1043
2018-01-15 13:46:30 +01:00