Commit Graph

6484 Commits

Author SHA1 Message Date
James Ritchie b7134dd769 Docs: Adds 2 references to the node.js versions we use for v3 and v4 (#2118)
* Adds 2 snippets to reference node.js version for v3 and v4

* Updated the node versions we support

* Remove specific node version from title
2025-07-02 13:08:49 +01:00
nicktrn 6fea802505 helm: set default webapp container and add extraVolumes (#2222)
* set default webapp container

* bump version

* add webapp extraVolumes and extraVolumeMounts

* bump version
2025-07-02 13:03:02 +01:00
Eric Allam 09ef7625ff Merge pull request #2219 from triggerdotdev/metadata-serialization
Serialize metadata to prevent invalid data from breaking run completions
2025-07-02 12:49:08 +01:00
Eric Allam 5cf37e9f8a Merge pull request #2189 from triggerdotdev/increased-and-configurable-attribute-limits
Increase otel attribute limits and make them configurable
2025-07-02 12:48:38 +01:00
Eric Allam e7b6c2fab6 Merge pull request #2183 from triggerdotdev/process-keep-alive
v4: Experimental process keep alive
2025-07-02 12:39:43 +01:00
nicktrn d619e79f74 helm: increase default postgres resource preset (#2221)
* switch to small preset for postgres

* lint

* bump version
2025-07-02 11:57:14 +01:00
Eric Allam 38a94b3b70 Add changeset 2025-07-02 11:30:13 +01:00
Eric Allam 0c2af6d87c Increase otel attribute limits and make them configurable 2025-07-02 11:30:13 +01:00
Eric Allam 0deb2c640d remove change from process-keep-alive 2025-07-02 11:30:13 +01:00
Eric Allam 21c70ad673 Serialize metadata to prevent invalid data from breaking run completions 2025-07-02 11:30:13 +01:00
Eric Allam 0793c2b0f4 address some nick comments in the review 2025-07-02 11:30:13 +01:00
Eric Allam 8dc90d61b4 increase run engine test timeouts 2025-07-02 11:30:13 +01:00
Eric Allam 57f3a89a86 Fix timeline span events tests 2025-07-02 11:30:13 +01:00
Eric Allam ceb737e9f1 Fix broken webapp test 2025-07-02 11:30:13 +01:00
Eric Allam 632d6b91e9 Fix taskSlug extraction 2025-07-02 11:30:13 +01:00
Eric Allam b548561220 Remvoe the sometimes incorrect import file name 2025-07-02 11:30:13 +01:00
Eric Allam aecee4d35f Don't reset task lifecycle hooks between executions 2025-07-02 11:30:13 +01:00
Eric Allam 9ab590bead Better handle snapshot changes with the new persistent task run process,
Also don’t re-import the task file if the task is already in the resourceCatalog
2025-07-02 11:30:13 +01:00
Eric Allam 2cc1453f25 Always set the cold/warm variant on attempt spans, even in dev 2025-07-02 11:30:13 +01:00
Eric Allam 82f3db6825 Only bootstrap on the cold start execution, and record how many executions in the attempt span 2025-07-02 11:30:13 +01:00
Eric Allam f0bc4ad2bd Don't delete env vars with falsy values 2025-07-02 11:30:13 +01:00
Eric Allam 4d15c8f7cf Better reset in the prod usage manager 2025-07-02 11:30:13 +01:00
Eric Allam 7f3b1c1735 Ensure the usage interval is cleared when resetting 2025-07-02 11:30:13 +01:00
Eric Allam 707bc09dac Make sure maxExecutionsPerProcess is a positive integer 2025-07-02 11:30:13 +01:00
Eric Allam a0f2c982f9 Make sure to clear existing metadata flush timer when resetting 2025-07-02 11:30:13 +01:00
Eric Allam bffd18a6b8 Properly reset listener arrays by setting length to 0 2025-07-02 11:30:13 +01:00
Eric Allam 2018b41d22 TRIGGER -> TRIGGERDOTDEV 2025-07-02 11:30:13 +01:00
Eric Allam c41d63497b Handle immediate retries better 2025-07-02 11:30:13 +01:00
Eric Allam 56146afcb6 TaskRunProcessProvider cleanup is async 2025-07-02 11:30:13 +01:00
Eric Allam 020565c009 Handle dev process pool shutdown errors 2025-07-02 11:30:13 +01:00
Eric Allam 58d56f8381 Use hostname instead of host 2025-07-02 11:30:13 +01:00
Eric Allam 5b49b420cb Add changeset 2025-07-02 11:30:13 +01:00
Eric Allam 3fb68b784a couple of tweaks for testing stuff 2025-07-02 11:30:13 +01:00
Eric Allam a414e65efe handle env vars that have been removed between executions 2025-07-02 11:30:13 +01:00
Eric Allam 38fe854559 Make refreshing a task run process work correctly 2025-07-02 11:30:13 +01:00
Eric Allam c24f3a56cc Reset deployed run worker and also get the warm start stuff to work properly 2025-07-02 11:30:13 +01:00
Eric Allam 8798352be3 Super-warm starts in deployed tasks proof 2025-07-02 11:30:13 +01:00
Eric Allam 1a1e70ae3d WIP deployed processKeepAlive 2025-07-02 11:30:13 +01:00
Eric Allam 19bdf3b6ea scope the pool by background version, allow configuring pool params in dev/prod 2025-07-02 11:30:13 +01:00
Eric Allam 427c450391 Fixed metrics, reusing a process counts as a warm start 2025-07-02 11:30:13 +01:00
Eric Allam 9374aea401 Split resource and span attributes 2025-07-02 11:30:13 +01:00
Eric Allam fd3db5f77b Reset execution environment between runs 2025-07-02 11:30:13 +01:00
Eric Allam e2049fcfbf Initial dev process keep alive. moved resource attributes to span attributes because resource attributes are global and immutable 2025-07-02 11:30:13 +01:00
Eric Allam 1be4fcbb82 Fixes the jitter option for the scanConcurrencySets cron job (#2220) 2025-07-02 11:29:54 +01:00
Eric Allam 124435e5e7 Eliminate race conditions in continueRunIfUnblocked to prevent stuck runs (#2217)
* Eliminate race conditions in continueRunIfUnblocked to prevent stuck runs

* Use assertNever for typecheck friendliness
2025-07-01 16:34:12 +01:00
Matt Aitken 4a79b3b35b Fixed retrying on run replication inserts on connection errors (#2216) 2025-07-01 13:35:13 +01:00
Boon Kai 0da6a19987 docs: Update triggering.mdx import for triggering from inside another task (#2215)
When triggering from inside another task, `runs` and `batch` should be imported from `@trigger.dev/sdk/v3`
2025-07-01 08:03:02 +01:00
Eric Allam 08d84eb596 Fix v4 concurrency sweeper flaky test (#2213) 2025-06-30 16:37:52 +01:00
Matt Aitken f38d35e9f6 ClickHouse replication improvements (retrying, strip bad unicode chars) (#2205)
* Add retry logic for insert operations

Add a generic retry mechanism for task run and payload inserts to handle
transient connection errors. The new #insertWithRetry method retries up to
three times with exponential backoff and jitter on retryable connection
errors such as connection resets or timeouts. Errors are logged and
recorded in tracing spans to improve observability and robustness of the
replication service.

* Replication settings are configurable

* Log out the runIds for failed batches

* Detecting bad JSON in run replication and ignoring it

* Reproduced split unicode error

* Move output file

* Massively improved the performance

* Minor performance improvements

* Unskip tests

* Remove unused test in CH package

* Fix for the ClickHouse UI explorer

* RunReplication keepAlive defaults to false

* Add concurrency_key and bulk_action_group_ids to ClickHouse task runs

* ClickHouse package doesn't need to be built anymore for the webapp

* Set the concurrency_key from the run replication service
2025-06-30 16:32:02 +01:00
Eric Allam 1b4dabb8ba v4: current concurrency sweeper (#2206)
* v4: current concurrency sweeper

* Fix webapp tests

* Ensure only a single instance performs concurrency sweeping by using redis-worker cron jobs

* Improved the mark phase

* Ensure cron jobs get rescheduled even if the handler throws an error

* Better property names
2025-06-30 16:22:19 +01:00