Files
quantconnect--lean/Algorithm.Python
Jhonathan Abreu a565dfa6f0 Wait for fresh data before filling market orders on stale data (#9563)
* Wait for first session bar before filling equity market orders at open

EquityFillModel.MarketFill could fill a market order placed right after
market open using data from the previous trading date, because the first
bar of the current session has not been emitted yet. ShouldWaitForFreshData
only covered hour/daily resolutions, so minute/second orders filled on stale
prices.

Add IsWithinFirstResolutionSpanAfterMarketOpen: when the order time is within
the lowest subscribed resolution span after the open and the price is stale,
wait for the first bar instead of filling on the previous date's price.

* Share opening-bar stale-fill wait across fill models

Move IsWithinFirstResolutionSpanAfterMarketOpen to the base FillModel and add
a ShouldWaitForFreshDataOnStale sibling helper that combines it with the
existing coarse-resolution ShouldWaitForFreshData check. The base FillModel,
FutureFillModel and EquityFillModel market fills now share this single wait
decision at their stale-data guards.

ShouldWaitForFreshData is intentionally left untouched at its GetMarketFillPrice
call site, which uses it to choose the bar open vs current price and is not
gated by staleness, so fill prices for finer resolutions are unchanged. The
opening-bar helper is guarded against always-open markets, which have no
session open to wait for.

* Add regression algorithm for stale fill at market open

Reproduces the opening-bar stale fill issue: a market order placed one second
after the open while subscribed to minute resolution. Without the fix the order
fills on the previous trading date's stale price; the algorithm asserts in
OnOrderEvent that a fill never happens within the first minute after the open,
so it errors without the fix and passes with it.

Uses SPY minute data over 2013-10-07 to 2013-10-11, which is available in the
repository Data folder.

* Add unit tests for stale fill wait at market open

Cover the opening-bar stale fill scenario directly at the fill model level:
a market order placed within the first bar after the session open, while only
the previous session's stale bar is available, must wait instead of filling on
the stale price, and fills once the first session bar arrives. EquityFillModel
also asserts the boundary (orders past the first bar still fill on stale data),
and FutureFillModel covers the shared base helper from the future path.

* Generalize stale market-order fill wait to any time of day

Replace the market-open-specific wait with a generic check: a market order
that would be filled on stale data waits for fresh data when the latest
available data is more than one subscribed resolution bar behind the current
time. This no longer considers the market open explicitly; it covers the
opening bar (the first session bar has not been emitted yet) and any intraday
data gap larger than the resolution.

ShouldWaitForFreshDataOnStale now takes the latest data end time and the
current time instead of the order time, and is shared by FillModel,
FutureFillModel and EquityFillModel. Coarse resolutions (hour/daily) still
always wait; tick never waits. Internal configurations are included when
sizing the resolution bar. EquityFillModel's best-effort price helpers now
report the stale data end time so the gap can be measured.

Tests: EquityFillModelTests and FutureFillModelTests cover the market-open and
mid-session stale cases (wait then fill on fresh data) plus the within-one-bar
boundary (fill on stale). The regression algorithm is generalized to assert no
fill happens on data staler than the resolution, with orders at the open and
mid-session. Pre-existing plumbing/data-selection tests that used degenerate
timestamps were given fresh timestamps so they still exercise their original
intent.

* Add sample data and adjust regression algorithms for stale-fill wait

Add minute/daily sample data so market orders that now wait for fresh data
can fill (ES futures gap days, TWX/GOOG equities and options, SPXW weeklies,
GC futures/options copy for 2020-01-06). Adjust a few regression algorithms
to the deferred-fill behavior: cap orders in the extended-market continuous
future test, ignore daily-resolution SPY in the automatic-seed data checks,
and refresh OptionAssignmentStatistics expected constants.

* Update regression expected statistics for stale-fill wait

Regenerate ExpectedStatistics, DataPoints and AlgorithmHistoryDataPoints for
the regression algorithms affected by the wait-for-fresh-data fill change and
the added sample data: futures/options fill-timing shifts, ES data-point count
increases, and GOOG 2015-12-28 outcome changes.

* Trim SPXW sample data to expiries within filter window

The two SPXW algorithms filter with Expiration(0,7), so contracts expiring
more than a week out are never subscribed. Drop those far-dated expiries from
the 2021-01-06/08 minute files (760KB->108KB and 776KB->108KB on the quote
files). Fills, DataPoints and statistics are unchanged; both regression tests
still pass.

* Trim ES minute and GOOG option sample data to order-fill minimum

The ES minute gap-day files source no order fills (daily-resolution algos fill
from es_daily); keep only the front contract used for execution and drop the
unused back-month contracts. Trim the GOOG 2015-12-28 option file (no fill
depends on it) to the morning chain window. Regenerate the back-month futures
statistics affected by the dropped back-month bars. Full CSharp regression
suite passes (722/722).

* Use SMA gap threshold in BasicTemplateContinuousFuture for C#/Python parity

At a fast/slow SMA cross the two averages can coincide to within rounding
noise, where the C# (decimal) and Python (double) comparisons disagree,
producing different orders between languages. Require a minimum gap before
acting on a cross so both languages stay in lockstep, and update the shared
expected statistics accordingly.

* Mirror order cap in Python algorithm and update future history counts

Apply the same pre-2013-11-12/3-order cap to the Python
BasicTemplateContinuousFutureWithExtendedMarket algorithm for C#/Python parity,
and update the QuantBook future-history expected counts to reflect the added ES
sample data.

* Use SMA gap threshold in BasicTemplateContinuousFutureWithExtendedMarket for C#/Python parity

This algorithm had the same fast/slow SMA cross divergence already fixed in
BasicTemplateContinuousFutureAlgorithm (ad8fc33): at the 2013-10-29 cross the two
averages coincide to within rounding noise (C# decimal diff -1e-25, Python double
diff exactly 0.0), so the raw `_fast > _slow` / `_fast < _slow` comparisons disagree
between languages. C# fired a liquidate+rebuild that Python skipped, producing 5
orders in C# vs 3 in Python. Require a minimum 0.001 gap before acting on a cross so
both languages stay in lockstep, and regenerate the shared expected statistics
(Total Orders 5 -> 3).

* Document SMA cross threshold as a C#/Python parity workaround

Add a short note before the fast/slow SMA comparisons in both continuous-future
template algorithms clarifying that the minimum-gap threshold exists only so the
C# and Python versions take the exact same trades on the limited sample data in
the repository, where decimal vs double rounding can disagree at a cross.

* Fetch subscription configs once per equity market fill

MarketFill resolved the subscription configs twice per fill: once via the
best-effort price helpers (GetSubscribedTypes) and again via
ShouldWaitForFreshDataOnStale. Fetch them once and thread them through both
paths via optional parameters, leaving existing callers unchanged.

* Measure stale-fill wait against order submission time

ShouldWaitForFreshDataOnStale compared the latest data end time against the
security current time. Compare against the order submission time instead so the
decision to wait for fresh data reflects how stale the data is relative to when
the order was placed. Realign the stale-price warning fill test accordingly.

* Fix stale market data in SendingNewOrderFromOnOrderEvent test

The market price tick was timestamped a day before the order submission time,
so under the order-time staleness check the market orders waited for fresh data
instead of filling. Use a reference time with the tick one minute before the
order so the data is fresh and the orders fill.

* Centralize internal-inclusive subscription config lookup in fill models

ShouldWaitForFreshDataOnStale re-resolved the subscription configs through
the ShouldWaitForFreshData call it makes first, and GetMarketFillPrice did
the same. Thread the already-fetched configs through ShouldWaitForFreshData
and GetMarketFillPrice so each market fill resolves them at most once.

Add a GetSubscriptionDataConfigs(Security) helper on the base FillModel that
returns the internal-inclusive configs, and route every fill-model call site
through it to remove the duplicated lookup and repeated comment.

* Avoid list allocation in ShouldWaitForFreshData

Replace the Where(...).ToList() + All(...) with a single foreach over the
subscription configs, short-circuiting on the first non-coarse resolution.
2026-07-01 11:22:50 -04:00
..
2025-06-18 18:32:13 -03:00

QuantConnect Python Algorithm Project

This document contains information regarding how to use Python with the Lean engine, this includes how to use Python Autocomplete, setting up Lean for Python algorithms, PythonNet compilation for devs, and what imports to use to replicate the web IDE experience in your local development.



Local Python Autocomplete

To enable autocomplete for your local Python IDE, install the quantconnect-stubs package from PyPI using the following command:

pip install quantconnect-stubs

To update your autocomplete to the latest version, you can run the following command:

pip install --upgrade quantconnect-stubs

Copy and paste the imports found here to the top of your project file to enable autocomplete.

In addition, you can use Skylight to automatically sync local changes to the cloud.



Setup Lean Locally with Python

Before setting up python support, follow the installation instructions to get LEAN running C# algorithms on your machine.

Installing Python 3.11:

Next we must prepare a Python installation for Lean to use. Follow the instructions for your OS.


Windows

  1. Use the Windows x86-64 MSI Python 3.11.11 installer from python.org or Anaconda for Windows installer.
  2. When asked to select the features to be installed, make sure you select "Add python.exe to Path"
  3. Create PYTHONNET_PYDLL environment variable to the location of your python dll in your installation (e.g. C:\Dev\Python311\python311.dll or C:\Anaconda3\python311.dll):
    • Right mouse button on My Computer. Click Properties.
    • Click Advanced System Settings -> Environment Variables -> System Variables
    • Click New.
      • Name: PYTHONNET_PYDLL
      • Value: {python dll location}
  4. Install pandas=2.2.3 and its dependencies.
  5. Install wrapt=1.16.0 module.
  6. Reboot computer to ensure changes are propagated.

macOS

  1. Use the macOS x86-64 package installer from Anaconda and follow "Installing on macOS" instructions from Anaconda documentation page.
  2. Set PYTHONNET_PYDLL environment variable to the location of your python dll in your installation directory (e.g. /Users/{your_user_name}/anaconda3/lib/libpython3.11.dylib):
    • Open ~/.bash-profile with a text editor of your choice.
    • Add a new line to the file containing
    export PYTHONNET_PYDLL="/{your}/{path}/{here}/libpython3.11.dylib"
    
    • Save your changes, and either restart your terminal or execute
    source ~/.bash-profile
    
  3. Install pandas=2.2.3 and its dependencies.
  4. Install wrapt=1.16.0 module.

Linux

  1. Install Python using miniconda by following these commands; by default, miniconda is installed in the users home directory ($HOME):
export PATH="$HOME/miniconda3/bin:$PATH"
wget https://cdn.quantconnect.com/miniconda/Miniconda3-py311_24.9.2-0-Linux-x86_64.sh
bash Miniconda3-py311_24.9.2-0-Linux-x86_64.sh -b -p /opt/miniconda3
rm -rf Miniconda3-py311_24.9.2-0-Linux-x86_64.sh
  1. Create a new Python environment with the needed dependencies
conda create -n qc_lean python=3.11.11 pandas=2.2.3 wrapt=1.16.0
  1. Set PYTHONNET_PYDLL environment variable to location of your python dll in your installation directory (e.g. /home/{your_user_name}/miniconda3/envs/qc_lean/lib/libpython3.11.so):
    • Open /etc/environment with a text editor of your choice.
    • Add a new line to the file containing
    PYTHONNET_PYDLL="/home/{your_user_name}/miniconda3/envs/qc_lean/lib/libpython3.11.so"
    
    • Save your changes, and logout or reboot to reflect these changes

Run Python Algorithms

  1. Update the config to run a python algorithm:
    "algorithm-type-name": "BasicTemplateAlgorithm",
    "algorithm-language": "Python",
    "algorithm-location": "../../../Algorithm.Python/BasicTemplateAlgorithm.py",
    
  2. Build LEAN.
  3. Run LEAN. You should see the same result of the C# algorithm you tested earlier.

Python.NET development - Python.Runtime.dll compilation

LEAN users do not need to compile Python.Runtime.dll. The information below is targeted to developers who wish to improve it. Download QuantConnect/pythonnet github clone or downloading the zip. If downloading the zip - unzip to a local pathway.

Note: QuantConnect's version of pythonnet is an enhanced version of pythonnet with added support for System.Decimal and System.DateTime.

Below are some examples of build commands that create a suitable Python.Runtime.dll.

msbuild pythonnet.sln /nologo /v:quiet /t:Clean;Rebuild 

OR

dotnet build pythonnet.sln

Python Autocomplete Imports

Adding from AlgorithmImports import * to the top of your Python file is enough to enable autocomplete and import the required types for the algorithm at runtime.

Known Issues

  • Python can sometimes have issues when paired with our quantconnect stubs package on Windows. This issue can cause modules not to be found because site-packages directory is not present in the python path. If you have the required modules installed and are seeing errors about them not being found, please try the following steps:
    • remove stubs -> pip uninstall quantconnect-stubs
    • reinstall stubs -> pip install quantconnect-stubs