06f60f2720
* add PythonExtension * remove potential shell injection risk * Filter out blank lines or comment lines * fix spelling * add pythonExtension's `runInline` * changes to requirements don’t invalidate the entire install layer * copy script files on-demand * improve PythonExtension types and logging * add changeset * fix broken imports * Improve security of inline script execution Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * Add file existence check for requirementsFile Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> * update lock file * Enhance error handling with detailed error information * Add portable type annotation * fix error TS18046: 'e' is of type 'unknown' * export the python extension * add `pythonExtension` to the catalog * fix `Cannot find module '@trigger.dev/build/extensions/core' (TS2307) * replace execa by tinyexec * Update pnpm-lock.yaml * add custom traces instead of logging * The cleanup in the finally block does not fail silently anymore * move python runtime/extension to independent package * fix build package readme * update lock file * add documentation to python's package * add missing dependency * Update little-trains-begin.md --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Matt Aitken <matt@mattaitken.com> Co-authored-by: Eric Allam <eallam@icloud.com>
Python Extension for Trigger.dev
The Python extension enhances Trigger.dev's build process by enabling limited support for executing Python scripts within your tasks.
Overview
This extension introduces the pythonExtension build extension, which offers several key capabilities:
- Install Python Dependencies (Except in Dev): Automatically installs Python and specified dependencies using
pip. - Requirements File Support: You can specify dependencies in a
requirements.txtfile. - Inline Requirements: Define dependencies directly within your
trigger.config.tsfile using therequirementsoption. - Virtual Environment: Creates a virtual environment (
/opt/venv) inside containers to isolate Python dependencies. - Helper Functions: Provides a variety of functions for executing Python code:
run: Executes Python commands with proper environment setup.runInline: Executes inline Python code directly from Node.runScript: Executes standalone.pyscript files.
- Custom Python Path: In development, you can configure
pythonBinaryPathto point to a custom Python installation.
Usage
- Add the extension to your
trigger.config.tsfile:
import { defineConfig } from "@trigger.dev/sdk/v3";
import pythonExtension from "@trigger.dev/python/extension";
export default defineConfig({
project: "<project ref>",
build: {
extensions: [
pythonExtension({
requirementsFile: "./requirements.txt", // Optional: Path to your requirements file
pythonBinaryPath: path.join(rootDir, `.venv/bin/python`), // Optional: Custom Python binary path
scripts: ["my_script.py"], // List of Python scripts to include
}),
],
},
});
-
(Optional) Create a
requirements.txtfile in your project root with the necessary Python dependencies. -
Execute Python scripts within your tasks using one of the provided functions:
Running a Python Script
import { task } from "@trigger.dev/sdk/v3";
import python from "@trigger.dev/python";
export const myScript = task({
id: "my-python-script",
run: async () => {
const result = await python.runScript("my_script.py", ["hello", "world"]);
return result.stdout;
},
});
Running Inline Python Code
import { task } from "@trigger.dev/sdk/v3";
import python from "@trigger.dev/python";
export const myTask = task({
id: "to_datetime-task",
run: async () => {
const result = await python.runInline(`
import pandas as pd
pandas.to_datetime("${+new Date() / 1000}")
`);
return result.stdout;
},
});
Running Lower-Level Commands
import { task } from "@trigger.dev/sdk/v3";
import python from "@trigger.dev/python";
export const pythonVersionTask = task({
id: "python-version-task",
run: async () => {
const result = await python.run(["--version"]);
return result.stdout; // Expected output: Python 3.12.8
},
});
Limitations
- This is a partial implementation and does not provide full Python support as an execution runtime for tasks.
- Only basic Python script execution is supported; scripts are not automatically copied to staging/production containers.
- Manual intervention may be required for installing and configuring binary dependencies in development environments.
Additional Information
For more detailed documentation, visit the official docs at Trigger.dev Documentation.