Commit Graph

13 Commits

Author SHA1 Message Date
Matt Aitken ec8381e2d0 Fix for staging environment variable API endpoints
Ugly but effective fix for the fact we can pass the slug in as “staging” when we use “stg” internally
2024-07-15 16:33:00 +01:00
Eric Allam 85a543d8ec v3: usage tracking (#1163)
* Starting to measure wall time and cpu time in the workers, and reporting that via otel and to completed task run attempts

* Move usage tracking outside of the executor

* WIP prod usage tracking

* WIP

* WIP custom fetch to openmeter

* Create a usage client

* WIP

* WIP

* Implement new machine preset stuff and send usage reports to OpenMeter from webapp

* WIP

* Expose usage info to the client

* Add usage and cost to TaskEvent

* Add ability to globally configure the task machine preset

* Report start run usage

* Change the machine docs to use presets

* setExpirationTime to 24h

* Removed logs

* Update machines.mdx

* Removed console.logs

* Handle revalidating JWT tokens

* Couple tweaks

---------

Co-authored-by: Matt Aitken <matt@mattaitken.com>
2024-06-18 09:40:23 +01:00
Eric Allam 3a1b0c486a v3: env var management API (#1116)
* WIP env var management API

* Add import env var API endpoint

* Adding docs and support for using both API keys and PATs when interacting with the env var endpoints

* WIP envvar SDK

* Uploading env vars in a variety of formats now works

* Finish env var endpoints and add resolveEnvVars hook

* Add changeset
2024-05-23 16:11:25 +01:00
Eric Allam 07f7c65054 v3 queue system (#903)
* Introducing Modular Asynchronous Reliable Queueing System (MarQS). Works in dev

* Convert MarQS to using lua and dealing with concurrency

* Simplified the timeout queue and current concurrency is now a set instead of a flat value (to support idempotency)

* Implement task heartbeating and reconnect the background workers CLI when the websocket connection reconnects

* Start adding internal telemetry support for the server

* Get env vars to work in dev and implement prisma tracing in webapp

* Cleanup telemetry and implement it in the consumer

* Implement dequeuing a message from a parent shared queue

* Implement a custom logger exporter instead of using console log exporter

* Use node instead of shell for generating protocol buffer code

* Propogate trace context into debug logs, and allow turning off logger exporter through env vars

* Switch to using baselime for internal otel data

* Make orgMember optional to fix type issues

* Provide the CLI dev env vars through the CLI, don’t build dotenv into facade

* Removed the logger import

* Address Matt’s comments

* Addressing more of Matt’s comments

* Handle sending an execution after a websocket connection closes

* Remove auth from the env attributes to prevent obfuscation
2024-02-20 12:16:41 +00:00
Matt Aitken 2b48e6d04b Feature: sendEvent can use an AWS SQS queue to increase reliability and throughput (#729)
* Initial commit (by create-cloudflare CLI)

* Changed the prettier rules

* Started writing Readme and got dev working

* Validate the request, parse the event and respond with the correct format

* Use ulidx instead of ulid in core

* Removed some boilerplate and use new function for sendEvent

* Removed old ulid package

* When handling a webhook, it’s not an error if the HTTP endpoint environment isn’t found

* If the env vars aren’t set, log the request and pass it through

* Pass requests through. Setup the env vars

* Flipped the API key detection because we can enforce that it’s the shape of a private API key

* Progress with sendEvent proxy

* Events are being put on the SQS queue

* The SQS event queue is working

* Make sure there’s a timestamp before enqueuing the evnet

* Log the sqs_event, not API key

* Added a bit more detail to the readme

* Improved the Cloudflare logs

* We don’t need the global.window hack for AWS, just globalThis

* Removed commented out wrangler.toml values

* Reworked the proxy to make it easier to add more endpoints

* Use json utility to send Responses

* Start work on proxying bulk events
2023-11-13 15:08:01 +00:00
Matt Aitken 9ff4c0dbd5 Project-wide Prettier setup (#237)
* Setup project-wide prettier

* Remove old workspace file

* Remove old debugging directives

* New top-level .prettierignore

* Updated Prettier config settings

* Contrubuting guide: Fix for some bad code blocks

* Added more ignores

* Improved the format script command

* printWidth set to 100

* Formatted entire repo (pnpm run format)
2023-08-01 10:21:22 +01:00
Matt Aitken 4c79129242 authenticateApiRequest() now returns an object with the apiKey, type and environment 2023-07-26 21:54:20 +01:00
Matt Aitken 3a6c25d42d API authentication updated to work correctly with public and private keys 2023-07-25 17:52:20 +01:00
Matt Aitken 4d2a8728be Getting events is allowed with allowPublicKey 2023-07-23 19:06:18 +01:00
Matt Aitken ae3afdfc7e Allow public key api requests, if the flag is passed 2023-07-23 19:05:30 +01:00
Eric Allam 3e87e832c7 WIP trigger variants 2023-05-04 10:06:06 +01:00
Eric Allam 1bba822e20 Added projects, moved envs to projects, and renamed executions to runs 2023-04-27 13:50:22 +01:00
Eric Allam d8c22b61ef Workflows are now being registered in the webapp, along with their trigger 2022-12-15 16:41:08 +00:00