main
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
557d82b6a9 |
fix(producer): validate distributed video metadata (#2839)
## What - enforce a finite, validated `meta/videos.json` contract shared by Plan v1 and Plan v2 - preserve authored finite ends and source-derived trim-aware ends; bound any still-open end at the validated composition end - fail distributed planning when any declared video source did not extract instead of publishing a blank-capable plan - make the v1 chunk reader reject malformed/null video timing before frame injection - route deterministic video-source/metadata failures as non-retryable in AWS and GCP while retaining retries for transient extraction failures ## Why An open-ended video whose remote source could not be resolved retained `Infinity` through planning. Plan v2 correctly rejected that value, while Plan v1 serialized it as `null`; the v1 frame lookup could then suppress injected frames and silently produce incorrect output. The invariant belongs at the shared metadata boundary. Both protocols must receive identical finite timing, and unavailable sources must fail closed before plan publication. ## Test plan - [x] producer distributed planning, metadata, v1 chunk boundary, Plan v2 conversion/materialization, and public exports - [x] core runtime media semantics (authored slots, natural duration, looping, non-looping hold) - [x] engine video extraction and frame lookup - [x] AWS Lambda/CDK/SAM and GCP Cloud Run error normalization/retry classification - [x] producer, core, engine, AWS, and GCP typechecks/builds - [x] formatting, oxlint, tracked-artifact, fallow, and commit hooks - [x] exact incident composition replayed through the AWS Lambda handler's Lambda-local path in a Lambda-like container; Plan v1 and Plan v2 both fail closed as `VIDEO_SOURCE_UNRENDERABLE` during planning, before plan publication - [x] full PR CI, including all nine regression shards and Windows render/tests No production flags or deployment/release workflows are changed. |
||
|
|
5bf61d6df0 |
feat(aws-lambda): support plan protocol v2 (#2789)
* feat(aws-lambda): support plan protocol v2 * fix(aws-lambda): align SAM v2 terminal errors |
||
|
|
aa58ebc048 |
feat(lambda): add real-AWS smoke + benchmark script (#908)
* feat(lambda): add real-AWS smoke + benchmark script
Phase 6.3 of the distributed rendering plan
(DISTRIBUTED-RENDERING-PLAN.md §11 Phase 6a). Local bash script that
deploys the PR 6.2 SAM template to your own AWS account, renders a
fixture composition through the Step Functions state machine at several
chunk counts, PSNR-compares each output against the in-process
baseline, and tears the stack down.
This is the gate that proves the architecture works on real Lambda
infrastructure. After it runs green and the numbers are good, Phase 6b
(CLI, CDK, docs) can proceed with confidence.
Script lives at examples/aws-lambda/scripts/smoke.sh. Defaults:
- fixture: mp4-h264-sdr
- chunk_counts: 2,4,8
- psnr-threshold: 50 dB
- region: us-east-1
- stack-name: hyperframes-lambda-smoke-<timestamp>
AWS credentials come from the standard resolution chain (env vars,
~/.aws/credentials, SSO, IMDS). Pin a specific profile via --profile or
AWS_PROFILE; the script doesn't ship a default.
Workflow:
1. Pre-flight: verify aws/sam/bun/ffmpeg/jq/zip on PATH; check
credentials via sts:GetCallerIdentity.
2. Build the PR 6.1 ZIP and run verify:zip-size.
3. sam validate --lint + sam deploy under a per-run stack name.
4. Zip the fixture's src/ and upload to the render bucket.
5. For each chunk count, start a Step Functions execution, poll for
completion (25-min cap), download the output mp4 and the execution
history JSON, ffmpeg-psnr against the LFS-tracked in-process
baseline, and append to results.json.
6. Gate on the PSNR threshold.
7. Empty the bucket + sam delete (unless --keep-stack).
Outputs land under ./lambda-smoke-artifacts/:
- results.json (chunkCount x wallClockMs x psnrAvgDb)
- renders/N<N>-output.mp4
- renders/N<N>-history.json (full Step Functions execution history)
Per-run stack name with concurrency-safe AWS resource isolation. Run
multiple smokes in parallel without races; teardown guards against
stale stacks via cleanup_and_exit on every failure path.
Distinction from CI: this is a maintainer-run gate, not part of regular
CI. The architecture's per-PR safety net is the local Docker-based
BeginFrame probe (PR 6.1) and the upcoming Lambda RIE smoke mode (PR
6.6). No GitHub Actions / OIDC / cross-account secrets required.
This is part of the 8-PR Phase 6 stack; PR 6.3 of 8 — the last PR of
Phase 6a (validation). Phase 6b (CLI + CDK + docs) starts once 6.3's
benchmark numbers come back.
* fix(lambda): address PR 880 review feedback
- Document wall-clock methodology bias inline (eval.sh header + README):
local timing includes bun + tsx + harness scaffolding while Lambda
timing measures pure SFN execution, so "speedup" is end-to-end CLI
experience, not renderer-vs-renderer.
- Add --iterations N (default 1) with median wall-clock reporting via
awk-side median. Cold-start variance is ±5-10s per chunk; single-
sample readings made the PR-body speedup table not ground truth.
- Add --reserved-concurrency flag to both scripts; default still 16 but
no longer hardcoded. Pass-through to ReservedConcurrency CFN param.
- README: cost-per-pass estimate for both scripts.
- Replace `sed -n '2,30p' "$0"` help with usage() heredoc in both
scripts — fragile to header reflows and didn't survive the comment
expansion this commit adds anyway.
- eval.sh RMS-level parser: add a third fallback (`RMS level:` with no
`dB` suffix) for older ffmpeg builds where astats predates the unit
tag. Word-boundary guards keep `RMS peak level` from being eaten.
* docs(lambda): drop internal plan-doc + Rio refs from smoke/eval scripts
|
||
|
|
6664e54053 |
feat(lambda): add SAM template and sample events for AWS deployment (#907)
* feat(lambda): add SAM template and sample events for AWS deployment Phase 6.2 of the distributed rendering plan (DISTRIBUTED-RENDERING-PLAN.md §15). Reference SAM template for deploying HyperFrames distributed rendering on AWS — one Lambda function in three roles, choreographed by a Step Functions standard workflow with a Map state for parallel chunk rendering. Resources created by the template: - Lambda function pointing at the Phase 6.1 ZIP - Step Functions state machine: Plan -> Map(N) RenderChunk -> Assemble - S3 bucket for plan tarballs, chunk outputs, final mp4 - IAM role for the state machine - CloudWatch alarm guarding against runaway chunk invocations Retry policy: 4 attempts, 2s initial, 2x backoff, max 60s, with the typed non-retryable error codes from plan §9.3 explicitly opted out. CodeUri points at packages/aws-lambda/dist/handler.zip; sam deploy resolves the local path and uploads to a SAM-managed bucket on first deploy. Validated: sam validate --lint passes against the template. This is part of the 8-PR Phase 6 stack; PR 6.2 of 8. * fix(lambda): address PR 879 review feedback - Add CloudWatch alarms for Lambda Errors metric (5min window, threshold 1) and Step Functions ExecutionsFailed metric. The existing runaway- invocations alarm catches too-many-calls but missed silent per-chunk failures and retry-exhaustion. - Document VersioningConfiguration: Suspended tradeoff inline. Adopters treating the final mp4 as user-keepable should bump to Enabled. - Cost-allocation Tags on RenderBucket + Lambda Globals. - Lambda Tracing: Active so X-Ray spans don't terminate at the SF→Lambda boundary (the state machine already had tracing). - State-machine top-level TimeoutSeconds: 3600 as defensive ceiling on the whole choreography — catches Plan-retry storms before they hit individual task budgets. - AssertChunkCount Choice state: if Plan ever returns ChunkCount=0 the Map would silently iterate zero times and Assemble would receive an empty ChunkS3Uris[] producing an empty output. Fail-fast with typed PLAN_TOO_LARGE error instead. - Architecture comment: explicit x86_64-only constraint from @sparticuz/chromium so adopters trying Graviton don't get bitten. * docs(lambda): drop internal plan-doc refs from SAM example + template |