* upgrade @opentelemetry packages to the latest versions
* remove v2 only packages, will be moved to a dedicated repo
* remove more v2 code and run pnpm install
* use the npm yalt package in the webapp
* convert @trigger.dev/core to tshy
* Switch from jest to vitest in @trigger.dev/core
* Fixed core test
* move core-backend code into core subpath export
* convert @trigger.dev/sdk to tshy
* Removed hono
* move core-apps to core/v3/apps, remove core-apps, start converting cli-v3
* Fix up some of the commands
* cli now building and loadable
* using package-json-from-dist to get package version now in core and cli
* dev command WIP
* cleaned up some repetition and structure of the entry point stuff
* bringing back the background worker stuff
* Indexing of the v3 catalog
* getting closer to executing dev runs...
* centralize dev logging using event emitter
* Move indexing to it’s own entry point, simplify code
* dev runs working
* Get instrumentation to work with openai
* debugging achieved internally
* provide worker files as part of the worker creation on the server
* support for cjs and esm javascript
* Fixed timeout
* worker manifest now has the config path
* auto-upgrade config to non-deprecated alternatives
* Adding package preview release
* deployment WIP
* improve the syncEnvVars output and adapt resolveEnvVars
* WIP bun runtime
* WIP bun support
* seed tasks with the machine preset if listed in the config
* deploy run executions WIP, extracted TaskRunProcess into 1 place
* deployed tasks running and executing 🎉
* support for waits and better flushing & process cleanup
* Fixed the heartbeating
* Better warning messages
* Improve and unify the indexing between dev and deploy
* Support for external deps that need node-gyp to build
* build extensions can now install custom packages and run instructions in the image. Also prisma extension now works and also works with multiple schema files
* Add back in the main/types/module to sdk
* dev no longer is Ink/React, grace period for disconnections in dev
* Fix the changeset config
* More changeset fixes
* Remove config packages
* More changeset fixes
* Fixed typescript issues (needed to revert back to zod 3.22.3
* Fix pr_checks workflow
* Remove the prepare script
* Fixed tests and package versions
* Remove cli test script
* Remove packages from tailwind watch paths
* Add repo to public packages
* Just commit the generated files and do the building at dev time
* Try and get pkg.pr.new working
* Try again
* Fix emitDecoratorMetadata importing named export from typescript
* config file backwards compat with export const config
* Fixed issue where import errors weren’t coming through
* p-retry is a prod dep
* typescript needs to be a prod dependency for emitDecoratorMetadata
* Add better debug logging to help track down import-in-the-middle bug
* An external is only considered resolvable if it resolves to the same path as the collected external
* Fix runtime checks to allow >=18.20
* Move extensions to a new build package
* Fixed building packages in dockerfile
* Remove the e2e test from publish workflow for now
* Don’t treat pkg.pr.new versions has needing upgrading
* making sure config handleError works, and discovered path aliases don’t work in config files
* Strip empty string env vars so they accidentally override real values
* Couple of things
* Update version to use preview instead of beta
* Hopefully fix re-attempts with >30s delay
* Match socket emit messages to current latest in main
* Initial guide
* Go back to beta
* Go back to the preview, and update guide to use pr preview tags
* Go back to beta
---------
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* Test for checkpoints
* Make sourceTaskAttemptId optional on resumeBatchRun
* Removed all completions/executions logic from the shared queue consumer
* Removed the sourceTaskAttemptId from ResumeBatchRunService
* Revert "Removed all completions/executions logic from the shared queue consumer"
This reverts commit d35398d50463c81a1975bb5d5bcfca66a24b8ede.
* WIP on triggerAndWait…
* Fixed triggerAndWait continuing when a checkpoint completes
* Remove the ResumeAttempt code that fails attempts (was protecting against infinite restores)
* Removed messageBody.data.completedAttemptIds.length === 0 commented out code
* Don’t ack if there’s no batchRun
* Added the marqs?.replaceMessage back in but NOT when there’s no checkpoint. More logging
This is a fix for when some attempts fail
* Improvement to the test task that now randomly fails attempts
* When a checkpoint happens, only continue the attempt if it’s in the correct state
* Changeset for rollback in branch
* Set keepRunAlive to false when the dependent task isn’t finished
* Changeset manual version (to get inline with the hotfix branch)
* Changeset: Fixes for continuing after waits
* Latest lockfile (after manual changeset version)
* feature: add no-trigger-core-import
* fix: handle aliases and limit to exported modules
* fix: use longest export map name
* fix: support export type and interface
* fix: simplify message when export map is needed
* feature: fine grained modules for core
* fix: add prerequisite lint plugins
* fix: why do I have to fix these components again???
* fix: tests
* Add eslint config and vscode settings
* Create yellow-roses-arrive.md
* Remove the vscode linting
This is temporary so no one fixes these until we merge the remix branch
---------
Co-authored-by: Eric Allam <eric@trigger.dev>
Co-authored-by: Eric Allam <eallam@icloud.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* v3: Trigger delayed runs and reschedule them
* Create a `@trigger.dev/core/v3/schemas` export
* fixed the `@trigger.dev/core/v3/schemas` export
* Small docs tweak
* Add ttl option when triggering tasks, expire runs after ttl
Dev runs expire in 10m by default
* WIP
* Handle tasks that have failed but are being auto yielded
* Limit trace view to 25k event records, add a download run logs button
Also added two new indexes to TaskEvent:
```
/// Used on eventRepository.getTraceSummary()
@@index([traceId, startTime])
// Used for getting all logs for a run
@@index([runId])
```
* perf improvements on eventRepository.getSpan()
* v2: Add a 5 minute timeout for run execution requests in dev
* v3: Include presigned urls for downloading large payloads and outputs when using runs.retrieve
* v3: better handle large task payloads and outputs
* Change to 512KB
* v2: paginate trigger schedules endpoint
* v3: add 3MB limit on batch and single payloads
* Update task payload and output limits
* v3: remove node:stream and simplify env var upload API to better work with non-node runtimes
* Remove file/Response envvars upload docs
* Add changeset
* WIP worker TaskRunAttempt creation
* Handling failing task runs that cannot create an attempt for whatever reason
* Move the visibility queue stuff into a graphile job
* Fixed task runs with unsanitized queue names
* “Borrow” the code from alerts PR to get self hosted deployments working
* Add an admin API endpoint to get info about the shared marqs queue
* Allow admins to view any project metrics
* start adding lazy attempts to prod
* lazy attempt creation for prod workers
* resurrect prod stack traces
* add exception event to failed run spans
* simplify dependency resumes
* fix typecheck
* fix merge
* fresh process for all attempts
* always try sigterm first
* stop heartbeat timeout on non-inplace replace message
* add missing ack on checkpoint creation service failure
* bypass dequeue for retries with running worker
* respect retry delays
* crash runs with invalid run status for execution
* remove debug logs
* fix nack message
* fix version locking
* fresh attempt processes in dev and prod
* improve handling of ipc timeouts
* consider checkpoint failures on cancellation
* add basic chaos monkey to checkpointer
* changeset
* control forced checkpoint simulation via env var
* fix merge
* kill old attempt processes before checkpointing
* detailed perf logging for checkpointing
* add coordinator otlp endpoint example
* improve prod run cancellation
* rename supports lazy attempts migration
* fix graceful exit
* fix retry mechanics
* clear paused state before retry
* remove checkpoint image after push
* crash worker on unrecoverable errors
* refactor unrecoverable error emit
* switch to do hosted busybox image
* increase wait for duration ipc timeout
* add changeset for misc fixes
* fix merge
* fix retry delay span runId
* fix dev retries
* improve prod worker logging
* log checkpoint sizes
* add lazy attempts catalog entries
* Fixed merge issue: use zodFetch, not wrapZodFetch
* Revert "Fixed merge issue: use zodFetch, not wrapZodFetch"
This reverts commit d137e4e1fe.
* importEnvVars uses wrapZodFetch now
* add backwards compat for retries without checkpoints
* handle more cases of unrecoverable runs
* don't kill the child process if it shouldn't be killed
---------
Co-authored-by: nicktrn <55853254+nicktrn@users.noreply.github.com>
Co-authored-by: Matt Aitken <matt@mattaitken.com>
* WIP env var management API
* Add import env var API endpoint
* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints
* WIP envvar SDK
* Uploading env vars in a variety of formats now works
* Finish env var endpoints and add resolveEnvVars hook
* Add changeset
* Fixes an issue that was treating v2 trigger directories as v3
* Make msw a normal dependency (for now) to fix Module Not Found error in Next.js.
* Extract out all the zod* stuff from core so the SDK does not import it
* Add a changeset
* Fixing typecheck errors in the webapp
* Export the Task and TaskOptions types
* Extract additional exports from core/v3 that aren’t used in the SDK
* Move to our global system from AsyncLocalStorage for the current task context storage
* Update the esbuild core bundling plugin for the new core v3 exports
* Fix v3 CLI telemetry
* Add support for tasks located in subdirectories inside trigger dirs
* Remove the env var check during deploy (too many false negatives)
Eric Allam