Eric Allam
65da20c225
* WIP clickhouse package with test containers setup * More clickhouse client setup now with otel and real tests, and the v1 of raw run events * Add some additional columns to raw_run_events_v1 * WIP runs dashboard service * Create a new run engine event bus event for the runs dashboard to hook into * Track run events in the run engine * make sure engine v1 runs get synced to CH * Update the attemptNumber of v3 task runs * Restructure the run events to be more sparse * emit more stuff * Setup replication package * scaffold the replication package * replication wip * resolve conflicts * more replication stuff * Add ability to drop the replication slot completely on teardown * Use the new single replacingmergetree task events table for replication * get it working * insert payloads into their own table only on insert and then join * prepare for using clickhouse cloud and now running ch migrations during boot in the entrypoint.sh * Handover WIP and tests * Testing the replication service * Remove the runs dashboard stuff that we aren't using anymore * Added a test for large payloads * hacky typecheck fix * Fix new internal package typecheck issues and start adding telemetry to the replication service * tracing over spans, some other improvements * Improvements to the runs replication service, now ready for testing * Some fixes and cleanups * Don't need this code anymore * move transaction types into the runs replication service * only send spans where there are transaction events * A couple of suggested tweaks
About Trigger.dev
Trigger.dev is an open source platform and SDK which allows you to create long-running background jobs. Write normal async code, deploy, and never hit a timeout.
Key features:
- JavaScript and TypeScript SDK
- No timeouts
- Retries (with exponential backoff)
- Queues and concurrency controls
- Schedules and crons
- Full Observability; logs, live trace views, advanced filtering
- React hooks to interact with the Trigger API from your React app
- Pipe LLM streams straight to your users through the Realtime API
- Trigger tasks and display the run status and metadata anywhere in your app
- Custom alerts, get notified by email, Slack or webhooks
- No infrastructure to manage
- Elastic (scaling)
- Works with your existing tech stack
In your codebase
Create tasks where they belong: in your codebase. Version control, localhost, test and review like you're already used to.
import { task } from "@trigger.dev/sdk/v3";
//1. You need to export each task
export const helloWorld = task({
//2. Use a unique id for each task
id: "hello-world",
//3. The run function is the main function of the task
run: async (payload: { message: string }) => {
//4. You can write code that runs for a long time here, there are no timeouts
console.log(payload.message);
},
});
Deployment
Use our SDK to write tasks in your codebase. There's no infrastructure to manage, your tasks automatically scale and connect to our cloud. Or you can always self-host.
Environments
We support Development, Staging, and Production environments, allowing you to test your tasks before deploying them to production.
Full visibility of every job run
View every task in every run so you can tell exactly what happened. We provide a full trace view of every task run so you can see what happened at every step.
Getting started
The quickest way to get started is to create an account and project in our web app, and follow the instructions in the onboarding. Build and deploy your first task in minutes.
Useful links:
- Quick start - get up and running in minutes
- How it works - understand how Trigger.dev works under the hood
- Guides and examples - walk-through guides and code examples for popular frameworks and use cases
Self-hosting
If you prefer to self-host Trigger.dev, you can follow our self-hosting guide.
We also have a dedicated self-hosting channel in our Discord server for support.
Development
To setup and develop locally or contribute to the open source project, follow our development guide.