Register a VERL policy loss that gives each rollout equal weight across its expanded training rows. Normalize advantages by rollout token count and trained batch size, and load the custom loss in Ray actor workers.
Add focused coverage for registration, normalization, PPO aggregation, and input validation.
* example/async_swe_smith: online GRPO training example for SWE-smith
Add an end-to-end async GRPO example that trains a SWE agent on the
SWE-smith dataset via agl-lite, plus supporting infrastructure:
- examples/async_swe_smith: agent, chat template, k8s job template,
image pre-pull, FSDP + Megatron trainers, Qwen3-30B-A3B / Qwen3.5-9B
launch scripts, README, and a smoke test.
- agl_lite/verl/trainer.py: per-step GRPO group / zero-advantage group
counts and a per-stage perf/mfu/actor_infer metric.
- examples/calc_x/comparison: async-vs-sync runner scripts.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* feat(verl): record per-rollout pod lifecycle timing
Capture server-authoritative running_at/finished_at while polling and
emit, per training step, a wandb Table of per-rollout submitted/running/
finished timestamps plus queue-wait/run-duration scalar aggregates. The
queue wait (running_at - submitted) exposes how long pods sit waiting
when jobs are launched in CPU-limited batches.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* fix(setup_verl): install matching CUDA 13.0 toolchain for cu130 flash-attn build
The cu130 variant builds flash-attn from source against torch's CUDA 13.0
runtime, but the system CUDA toolkit is often 12.x, so nvcc rejects the
build. Install a CUDA 13.0 pip toolchain (nvcc/crt/nvvm/cccl/runtime) pinned
to >=13.0,<13.1 and point the build at it via CUDA_HOME/PATH/LIBRARY_PATH/
LD_LIBRARY_PATH/CPATH. Pinning to 13.0.x keeps nvcc's version aligned with
torch's CUDART (13000); a mismatched minor (e.g. 13.3) trips cccl's
"CUDA compiler and CUDA toolkit headers are incompatible" check.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* smaller cpu request
* feat(async_swe_smith): add Qwen3-8B sync-rollout trainer launch script
Mirrors the async async_swe_smith_qwen3_8b run (wandb fxaj7bcw) but flips
agentlightning.async_rollout.enabled=False for synchronous rollouts, keeping
all other hyperparameters identical (TP=2, GRPO n=8, train_batch_size=32,
ppo_micro_batch_size_per_gpu=2, lr=1e-6, clip 0.2/0.28, max_model_len=32768).
rollout.mode stays async (vLLM OpenAI server mode for tool calling). Sizing
knobs are env-overridable (AGL_GPU_MEM_UTIL, AGL_ROLLOUT_TP, etc.).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* example/async_swe_smith: add Qwen3-8B async runner; enable chunked prefill
Add run_qwen3_8b_async.sh (async counterpart of run_qwen3_8b_sync.sh):
identical training hyperparameters, only agentlightning.async_rollout.
enabled=True, for the sync-vs-async speedup A/B. Also flip the example's
vLLM enable_chunked_prefill to True.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* example/async_swe_smith: lower agent pod memory limit to 2Gi
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* example/async_swe_smith: expose train/async batch sizes as env-overridable knobs
Surface data.train_batch_size (32) and async_rollout.async_train_batch_size
(48) in run_qwen3_8b_async.sh via AGL_TRAIN_BATCH_SIZE / AGL_ASYNC_TRAIN_BATCH_SIZE,
passed through SIZING_OVERRIDES. Also bump default total_epochs 2 -> 4.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* example/async_swe_smith: disable chunked prefill + val-before-train; add requirements.txt
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* update config
* smaller poll interval for less cpu pressure
* example/async_swe_smith: auto-create AGL_NAMESPACE before controller start
The controller does not create its target namespace; with a non-default
AGL_NAMESPACE the ConfigMap step would fail on a missing namespace. Ensure
it idempotently before launch so per-user namespace isolation works.
Co-Authored-By: Claude Opus 4 (1M context) <noreply@anthropic.com>
* example/async_swe_smith: add openai-preinstalled job template variant
Uses the :openai SWE-smith images (openai library baked in) and drops the runtime 'pip install openai', removing the per-rollout cold-start cost.
* async_swe_smith: use openai-preinstalled job template for training
Point the async trainer at job-template-openai.yaml (":openai" images
with the openai library prebuilt) so rollout pods skip the runtime
pip install openai and start the agent directly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* example/async_swe_smith: add Qwen3-30B-A3B Megatron/R3 async runner
Conservative single-node 4x B200 defaults (n=4, gpu_mem_util=0.6,
train_batch=16, full param/optim/grad offload + recompute) sized to fit
30B-A3B; all knobs env-overridable. Set vLLM moe_backend=triton.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: support explicit pre-split train/val datasets
Add --train-dataset-path / --val-dataset-path (default to
train_datasets.jsonl / val_datasets.jsonl) so the trainer consumes
pre-split, pre-curated datasets as-is: no FAIL_TO_PASS curation and no
train/val split. Falls back to the legacy single-file + split path when
those files are absent. Adds --max-val-instances to bound validation
eval time.
Large local datasets (subset0 / train / val .jsonl) are excluded via
.git/info/exclude rather than .gitignore, so this drops the tracked
subset0 ignore line.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: support explicit pre-split train/val datasets in megatron entrypoint
Mirror the load_split_file path from train_smith_agent.py (27ec14d) into the
Megatron+R3 entrypoint: add --train-dataset-path/--val-dataset-path/
--max-val-instances and consume the pre-split files as-is (no FAIL_TO_PASS
curation, no internal train/val split) when both exist, else fall back to the
single-file split. Also add --ci so the run script's smoke flag is a real arg
instead of leaking into the hydra config overrides.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: add rollout correction (IS/RS) knobs to megatron runner
Wire algorithm.rollout_correction.* (bypass_mode / rollout_is / rollout_rs
and thresholds) into the Qwen3-30B-A3B megatron runner as env-overridable
sizing overrides, mirroring uni-agent's train_qwen3_moe_rc.sh defaults.
loss_mode is left at the actor default (verl 0.8.0 has no gspo).
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: match fsdp rollout sizing in megatron runner
Bump default ROLLOUT_N 4->8, TRAIN_BATCH_SIZE 16->32, ASYNC_TRAIN_BATCH_SIZE
24->48 to mirror the fsdp distributed run's rollout workload, enabling an
apples-to-apples engine comparison at equal generation volume.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: standardize wandb experiment naming
Name runs as swe_smith_{sync|async}_{model}_{backend} so FSDP and Megatron
runs are distinguishable in wandb. Mode is read from rollout.mode, model
from the basename of the model path; backend is fsdp / megatron per script.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: add pure-rollout collection harness
Collect reward (all-0/all-1 GRPO degeneration), completion-token length, and
turn-count stats over the train/val datasets without running a VERL trainer.
- serve_vllm_qwen3_8b.sh: stand up Qwen3-8B vLLM (128K ctx, chunked prefill,
TP=1, CUDA graph, hermes tools, return-tokens-as-token-ids) and register it
with the server proxy via POST /api/models.
- enqueue_rollouts.py: POST each dataset row N times (GRPO group) to
/api/rollouts with the openai job template + is_train flag.
- rollout_stats.py: poll finished rollouts, aggregate from reward/model_request/
agent_output events, group by data_id, report train/val separately.
- job-template-openai.yaml: SMITH_MAX_TURNS=1000 so turns (not the 40-cap) bound
self-completion.
- rollout_test.md: design doc.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: retry reward post until it succeeds
Reward events are training signal, so a single timeout silently dropped
the sample. Retry with capped exponential backoff until the post lands.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: full-run coverage for pure-rollout stats harness
The rollout_stats collector previously polled GET /api/rollouts?state_in=
succeeded&failed&limit=10000. That list route returns matches[:limit] in
insertion order with no offset, and each rollout is ~77KB (job_template is
embedded), so for the 164k-rollout SWE-smith run stats would silently plateau
at ~10k terminal rollouts (~6% of the run) — never seeing later completions.
Server: add an append-only completion log (_terminal_order, appended on
terminal state transitions in patch_rollout) and a cursor-paginated, lightweight
projection endpoint GET /api/rollouts/terminal?after=&limit= returning only
rollout_id/state/data_id/is_train + a next_after cursor. Because the log is
ordered by completion, an index cursor never misses out-of-order completions
and needs no full rescan of the store.
Collector: drain the cursor instead of re-listing; O(page_size) work per poll,
no 10k cap. Atomic snapshot writes (tmp + os.replace), per-rollout event-fetch
retry that skips unreadable rollouts without stalling the cursor, and a
--page-size flag. Snapshot now reports processed/total_terminal/backlog.
serve_vllm_qwen3_8b.sh: Qwen3-8B native ctx is 40960 and vLLM 0.20.2 dropped
--rope-scaling, so --max-model-len 131072 was rejected outright. Add YaRN via
--hf-overrides (factor 4.0 over original 32768) when extending beyond native
context — the documented 128K goal now actually serves.
Tests: cover the new endpoint (completion-order pagination, cursor advance,
projection fields, non-terminal exclusion); reset _terminal_order between tests.
.gitignore: ignore the generated rollout_stats.json output.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: checkout bug branch so the agent sees the injected bug
The SWE-smith image defaults /testbed to clean `main`, so the agent saw fixed
code, had nothing to fix, and evaluate() false-passed (reward 1.0 on a 0-byte
patch). Now the agent checks out the bug branch HEAD (`Remove F2P Tests`):
buggy source with the FAIL_TO_PASS tests removed, so it cannot read the tests
to reverse the fix. At evaluation time the F2P test files are restored from the
parent `Bug Patch` commit, keeping the agent's edits, so the real
FAIL_TO_PASS/PASS_TO_PASS suite runs against the fix.
Co-Authored-By: Claude <noreply@anthropic.com>
* Align SWE-smith agent loop with mini-swe-agent
Replace OpenAI tool-calling with mini-swe-agent's text/bash action space:
- one bash code block per turn (parse_action); submit via the
COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT marker instead of a submit tool
- swebench-style system/instance prompts adapted for /testbed
- head/tail observation elision + format-error nudges (max_format_errors)
- execute commands via 'bash -c' with pagers/progress bars silenced
Keeps AGL glue (events, eval-meta, bug-commit checkout, F2P restore,
git-diff patch capture, FAIL_TO_PASS/PASS_TO_PASS eval) and the
context-overflow / transient-error loop contracts. Adds SMITH_MAX_FORMAT_ERRORS.
* async_swe_smith: distinguish eval timeout from real failure + rollout tooling
- smith_agent: evaluate() now flags eval_timeout (pytest rc=124 / "timed out"),
surfaced in the reward event; fix p2p_ok counting None status as PASSED (which
let timeouts false-pass as resolved). Set temperature=1.0 explicitly; cap
SMITH_MAX_TURNS at 100 in job-template-openai.
- rollout_stats: tolerate transient connection errors (RemoteDisconnected).
- add monitor_controller.sh (periodic health probe), trace.md (4-rollout trace
example), reward_table.md (per-instance reward/patch snapshot).
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: add SWE-smith --f2p_only eval mode (on by default)
Mirror swesmith/harness/grading.py: when f2p_only, restrict evaluation to the
test files containing FAIL_TO_PASS. F2P is kept whole; PASS_TO_PASS is filtered
to only the P2P tests in those same files — same-file regressions still count,
but the many unrelated cross-file P2P tests (the eval-time/timeout driver) are
dropped. Resolution still needs F2P AND filtered P2P to pass. Toggled by
SMITH_F2P_ONLY (default on).
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: add rollout-collection runbook + full f2p_only result table
- plan.md: end-to-end runbook for launching a local pure-rollout collection
batch (cold-start vLLM/server/controller, enqueue, monitor, gold-patch
comparison), with the known pitfalls and how to run the second half.
- reward_table.md: full 367-instance agent-vs-gold patch comparison under
official f2p_only (all-0 98.1%, eval timeouts 0/1468).
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: lightweight branch switch to avoid checkout timeouts
git checkout --force <branch> stats/rewrites the whole /testbed working tree
(thousands of files for repos like pandas), which times out under high pod
concurrency (node at 6x oversubscription) and fails jobs. Instead, diff
main..branch at the object level to get just the changed files and apply only
those: modified/added paths via checkout, deleted F2P tests via git rm. All
git calls now go through a retry helper. Measured 1234ms -> 32ms on pandas;
working-tree state is identical to the old full checkout.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat(swe-smith): Qwen3.5-9B pure-rollout eval + 8B vs 9B reward table
- 4x vLLM Qwen3.5-9B (TP=1, round-robin) pure-rollout over subset0
- gen_compare_table.py / gen_reward_table_9b.py reward-table builders
- reward_table.md: 8B(all-0 98%) vs 9B(~47%), 116/230 improved
- baseline 8B table snapshot; trainer wandb table trimmed; async sizing
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe-smith: per-call max_tokens 16384, vLLM ctx 128k for 9B rollout
- smith_agent default AGL_MAX_TOKENS 4096->16384
- job-template-openai: AGL_MAX_TOKENS=16384 env
- vLLM restarted max_model_len 131072 (4x, registered)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix shuffling issue & zero sample issue
* async_swe_smith: align reward with official XFAIL grading + 128k train ctx
Count XFAIL as pass (F2P/P2P) to match swesmith/harness/grading.py; set
data/trajectory max lengths to 65536 = 128k. Add event-driven fail/starve
pod watcher.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe-smith: reward_table 9B 321/367, add per-rollout total token length
- gen_compare_table: 9B total-len column (avg) + p50/p90/max summary
- 9B vs 8B: all-0 98%->52%, improved 147, regress 0
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe-smith: annotate units in reward_table (patch=bytes, total=tokens)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: tee rollout pod logs to hostPath for post-mortem
Mount host /agl-logs and tee stdout/stderr to agl-rollout-<id>.log so a
killed/evicted pod's trace survives the TTL reaper. Both templates.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: bump pod memory to 4Gi to stop OOMKills
Long 128k-ctx trajectories were killed at 2Gi. Raise request+limit to 4Gi.
Co-Authored-By: Claude <noreply@anthropic.com>
* refactor: promote async SWE-smith example
Rename async_swe_smith to swe_smith, update startup docs/scripts, and prepare local :openai rollout images from minikube-loaded tags.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: log trace merge mismatches to wandb
Record capped unmerged trajectory triplets as a wandb table and expose mismatch row metrics from the rollout adapter.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: add delete.jsonl — instances with missing bug branches
35 instances across 11 repos (mostly Go) whose images lack the
origin/<instance_id> branch, so checkout fails with "unknown revision"
and the rollout dies before turn 1. Listed for dataset filtering.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: wait out gateway pause instead of burning a turn
On 429 'gateway paused' (async weight sync), the agent now retries in place
every 5s up to SMITH_GATEWAY_WAIT_S (default 600s) without incrementing the
turn or appending an empty assistant message. Previously the pause was
treated as a generic error, burning turns and polluting the trajectory.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: harden fetch_eval_meta against transient server errors
RemoteDisconnected escaped the except (it is http.client.HTTPException, not
URLError) and crashed the agent; timeouts were swallowed but left instance_id
empty, wasting a full rollout until checkout failed. Catch connection-reset /
HTTP exceptions, retry 4x with backoff for the bursty transient failures, and
abort immediately when no instance_id can be fetched instead of running on.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: force pytest serial in eval to stop OOMKills
The eval pytest command passed no -n flag, so a testbed project's own config
(e.g. scipy's `-n auto`) spawned workers = HOST core count (dozens), ignoring
the pod's cpu:1 / 4Gi cgroup. ~70 python workers @ ~112MB each exhausted the
4Gi limit and triggered cgroup OOMKill mid-rollout — verified live: OOMed pods
had 70+ python procs, memory.current pinned at 4094/4096Mi. Append `-p no:xdist`
to disable the plugin (overrides any ini `-n auto`; no-op when xdist absent).
Confirmed fixed: new pods now run 1 python proc instead of 70.
Co-Authored-By: Claude <noreply@anthropic.com>
* async_swe_smith: update reward_table.md — Qwen3.5-9B pure-rollout (620 instances)
Snapshot of the in-progress Qwen3.5-9B pure-rollout collection over
train_datasets.jsonl (is_train=false, n=4 GRPO groups), throttled to 50
in-flight. 620 instances / 2490 rollouts terminal so far; all-0 544, all-1 51,
mixed 25; 0 eval timeouts (SMITH_EVAL_TIMEOUT=3000, SMITH_MAX_TURNS=100).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* async_swe_smith: cap pytest at -n4, bump pod mem 6Gi, throttle job rate
Eval-time OOMKills traced to testbed `-n auto` reading host cores (~70 xdist
workers) and busting the cgroup. Pin -n4 (~450MB) with serial fallback when
xdist is absent. Raise pod memory to 6Gi and lower max_jobs_per_minute to 100.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix: harden SWE-smith rollout checkout
* docs: document SWE-smith Qwen3.5 sync run
* swe_smith: point hostPath logs at host dir after kubeadm migration
minikube is replaced by a kubeadm single-node cluster, so pods run directly
on host docker. hostPath /agl-logs (minikube VM path) now points at the real
host dir /home/v-zhiwenzhou/agl-logs — no minikube mount needed. Also lower
openai template pod memory to 4Gi.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: update reward_table.md — Qwen3.5-9B pure-rollout (7300 instances)
Snapshot of the Qwen3.5-9B pure-rollout collection over the verified + full
train datasets (is_train=false, n=4 GRPO groups), throttled to 50 in-flight,
deduped by data_id. 7300 instances / 29174 rollouts: all-0 4510 (61.8%),
all-1 1667 (22.8%), mixed 1123 (15.4%); 50 eval timeouts
(SMITH_EVAL_TIMEOUT=3000, SMITH_MAX_TURNS=100).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: store reward table as SQLite db, drop markdown
Switch the reward table from reward_table.md to reward_table.db (SQLite).
Column names match the old markdown detail table exactly: instance, rollouts,
"r=0", "r=1", "分类", "best patch_size", "avg turns", "超时".
gen_reward_db.py is incremental — it caches each terminal rollout by id in a
rollout_events table and only fetches events for NEW rollouts, so a refresh
takes ~7s instead of ~10min at 30k+ rollouts. reward_table is rebuilt from
rollout_events (is_train=0, grouped by data_id).
Snapshot: 30905 rollouts / 7718 instances (all-0 4606, all-1 1893, mixed 1219).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: retry idempotent /proxy/pause with 5min timeout; pod mem 6Gi
/proxy/pause only sets state.paused=True (idempotent, lock-guarded), so retrying
is safe. When the server is saturated with in-flight rollouts the pause POST can
be slow to get a connection and the default 30s timeout is not enough — give it
300s and 5 retries with backoff. Also bump openai template pod memory to 6Gi
(kernel OOM traced anon-rss ~4.1GB busting the 4Gi limit).
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: self-documenting timestamped run logs in launcher
Redirect each role's stdout+stderr to a timestamped file
(/tmp/agl_logs/{role}_{model}_{YYYYmmdd-HHMMSS}.log) with a stable
{role}_latest.log symlink, so repeated server/trainer restarts no longer
overwrite each other's logs. A fixed name like /tmp/train_9b_async.log
silently loses the previous run's crash trace on relaunch.
Opt out with AGL_LOG_TO_FILE=0; override dir/name via AGL_LOG_DIR/AGL_LOG_FILE;
auto-skipped for --ci smoke tests.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: dedupe triplet model requests by prompt tokens
* feat: add retry logic to non-GET HTTP requests to harden the trainer
Make every non-GET request in the trainer process resilient to transient
failures so a single HTTP error can no longer crash training mid-step:
- register_model (POST /api/models): retry with backoff (idempotent upsert)
- _resume_gateway (POST /proxy/resume): retry with backoff (idempotent)
- _create_rollouts (POST /api/rollouts): pre-assign client rollout_ids and
retry; server-side enqueue is now idempotent (existing id returns the
existing rollout, events untouched), so retries never duplicate rollouts
- delete_model / _delete_rollout: best-effort, swallow errors
- add DELETE /rollouts/{id} (idempotent, cascades to events) and delete
completed rollouts from the managers to keep server-side state bounded
Also switch AglLiteSyncClient's retrying GET to print over logging for
consistency with the rest of the codebase.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: prevent git-history reward hacking in smith_agent
SWE-smith testbed repos ship the full git history, so the agent can recover
the injected-bug fix (and the deleted FAIL_TO_PASS tests) via
`git checkout <pre-bug-sha> -- <src>` / `git show` / `git log -p`. On the
Qwen3.5-9B run this inflated val/reward to ~0.80 (96% of resolved val rollouts
used the git hack; ~0% solved cleanly).
Agent-side mitigation:
- relocate_git(): move /testbed/.git out of the worktree after checkout (O(1)
same-fs rename); the harness still reaches it via _git_base()
(--git-dir/--work-tree) for restore_f2p_tests + capture_patch.
- _forbidden_action(): reject any agent command that invokes git or reads git
metadata (.git / --git-dir / the relocated dir), returned as an observation;
wired into run_agent_loop before _run.
- _agent_env(): strip SMITH_HIDDEN_GIT_DIR and any var leaking the path so the
relocated dir is invisible in the pod env.
- Drop the git-based submission step from the prompt (harness captures the
patch independently) and note that git is disabled.
Verified on 50 random val instances (base Qwen3.5-9B, isolated pods):
relocation 50/50 (O(1)), 0 leaks, 0 false-positive blocks; submission and
eval/patch-capture work via --git-dir; blocked git attempts recover to manual
edits without hindering legitimate solves.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: log rollout trajectories as wandb artifacts
* feat: add rollout DELETE endpoint and auto-cleanup in rollout managers
Add DELETE /rollouts/{rollout_id} (idempotent, cascades to events) and a
retrying GET on AglLiteSyncClient. Sync manager deletes each round's
completed rollouts; async manager deletes non-carry-over groups on
completion, keeping server-side state bounded during training.
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 149400351dc1fe16c2bd98a0a8eae34d3931da85)
* feat: add retry logic to non-GET HTTP requests to harden the trainer
Make every non-GET request in the trainer process resilient to transient
failures so a single HTTP error can no longer crash training mid-step:
- register_model (POST /api/models): retry with backoff (idempotent upsert)
- _resume_gateway (POST /proxy/resume): retry with backoff (idempotent)
- _create_rollouts (POST /api/rollouts): pre-assign client rollout_ids and
retry; server-side enqueue is now idempotent (existing id returns the
existing rollout, events untouched), so retries never duplicate rollouts
- delete_model / _delete_rollout: best-effort, swallow errors
- add DELETE /rollouts/{id} (idempotent, cascades to events) and delete
completed rollouts from the managers to keep server-side state bounded
Also switch AglLiteSyncClient's retrying GET to print over logging for
consistency with the rest of the codebase.
Co-Authored-By: Claude <noreply@anthropic.com>
(cherry picked from commit 137506f293f5287ee62dcacdab358c43f9c40825)
* fix: align rollout cleanup retry merge
* swe_smith: update reward_table.db — 24836 instances (85% of verified)
Qwen3.5-9B pure-rollout over train_dataset_verified.jsonl (is_train=false, n=4,
throttled 50 in-flight, deduped by data_id). 99647 rollouts / 24836 instances:
all-0 13789 (55.5%), all-1 6575 (26.5%), mixed 4472 (18.0%).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: lower pod memory request to 2Gi, keep 6Gi limit
request=limit=6Gi over-reserved: the scheduler pinned ~83 pods against the
node's 510Gi allocatable while pods actually use ~1-2GB, leaving CPU at 16%
and 150Gi RAM idle. Drop request to 2Gi so more pods schedule; keep limit at
6Gi so long-trajectory pods (kernel-observed anon-rss ~4.1GB) don't OOM.
Co-Authored-By: Claude <noreply@anthropic.com>
* feat: route rollouts deterministically by rollout_id hash for prefix caching
Replace round-robin server selection with a stable sha256(rollout_id) mod
pool-size mapping so every request from a rollout lands on the same endpoint,
maximizing prefix-cache hits. Sort the pool by endpoint to keep the mapping
independent of registration order.
Co-Authored-By: Claude <noreply@anthropic.com>
* Log compact rollout trajectory artifacts
* swe_smith: lower pod memory request to 1Gi, keep 6Gi limit
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: Qwen3.5-9B run defaults + tighten token caps, drop internal plan.md
- run_qwen3_8b_async.sh: default model Qwen3-8B->Qwen3.5-9B, max_model_len
32768->65536, expose AGL_ROLLOUT_N, tie ppo_mini_batch_size to train_batch_size,
async overrides (val_before_train=False, test_freq=-1), run-name qwen35_9b_async.
- job-template-openai.yaml: AGL_MAX_TOKENS 16384->12288, add SMITH_OBS_CHAR_CAP=6000
to keep agent trajectories shorter under the 65536 context window.
- remove examples/swe_smith/plan.md (internal planning notes, not shipped code).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update SWE-smith training settings
* swe_smith: final Qwen3.5-9B reward table + 5245 mixed instances
Collection complete over train_dataset_verified.jsonl (is_train=false, n=4,
throttled 50 in-flight, deduped by data_id via reward db). Final:
29976 instances / 120283 rollouts — all-0 17109 (57.1%), all-1 7622 (25.4%),
mixed 5245 (17.5%). 82.5% are degenerate (zero GRPO advantage); the 5245 mixed
instances are the trainable set.
- reward_table.db: final SQLite snapshot (30MB).
- mixed_instance_ids.txt: the 5245 mixed data_ids (instances with GRPO signal).
- enqueue_throttled.py: throttled enqueuer with --skip-from-db (dedup that
survives a server restart — needed after the in-memory store was wiped).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: prune unused scripts, add log/reward analysis tooling
- Remove no-longer-used enqueue/gen/monitor/serve scripts and stale docs.
- job-template-openai: point rollout-logs hostPath at /agl-logs (minikube 9p
mountpoint) so pod logs reach the host under the docker driver.
- Add filter.md (full->verified dataset filtering rationale), scan_abnormal_pods.sh
(k8s abnormal-pod probe), stat_pod_logs.sh (bulk pod-log anomaly stats), and
report_qwen35_9b.md (Qwen3.5-9B rollout report: 34% resolve, 83% ctx overflow).
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: remove unused pod monitoring/log scripts
Co-Authored-By: Claude <noreply@anthropic.com>
* Add SWE Verify checkpoint evaluation
* Add B200 Qwen3.5-9B vLLM pure-decode roofline benchmark
Measure single-card B200 decode performance vs theoretical roofline for
Qwen3.5-9B (hybrid GatedDeltaNet + full-attention). Includes:
- theoretical roofline (mem/compute roofs, ridge, TPOT floor)
- batch sweep + context sweep via latency-subtraction (pure decode isolation)
- reusable bench + plot scripts, CSV data, roofline.png, README + summary
Key: ~24k tok/s/card decode ceiling (21% of compute roof); batch-1 at 51%
of bandwidth roof (hybrid recurrent-state overhead); empirical ridge ~batch
32-64; context 128->64k drops 4x but <=4k near-lossless (hybrid KV advantage).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* roofline: document measured HBM BW / BF16 TFLOPS in README
Roof charts & theory keep datasheet spec (8.0 TB/s, 2250 TFLOPS) per roofline
convention. Add README section 1.4 with empirically-measured achievable ceilings
(HBM 7.11 TB/s = 89% of spec; BF16 dense 1623 TFLOPS = 72% of spec) plus two
reusable microbenchmark scripts, so real attainment can be cross-referenced
(batch-1 = 57% of measured BW roof; batch-256 = 29% of measured compute roof).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add per-step + per-minute rollout stats collector with long-tail metrics
Instrumentation for the async-GRPO SWE-smith run (Qwen3.5-9B, 4xB200) to
quantify per-vLLM-call long tails and their effect on rollout throughput.
- collect_rollout_stats.py: 60s collector writing two tables from vLLM
Prometheus histograms (e2e latency, response/prompt tokens).
- rollout_step_stats: per-call avg prompt/resp len, per-card out tokens +
throughput, max response, and per-call long-tail percentiles
(resp_tok/e2e_lat p50/p90/p99 + tail_ratio_lat_p99_p50).
- gen_window_throughput: per-~60s throughput time-series inside each gen
window, showing tail-segment collapse.
- rollout_stats.db: 18 step rows + 177 per-minute rows.
- README documents schema + findings: per-call e2e p99/p50 ~12-18x every
step (driven by ~21x response-token tail); within-window throughput
collapses 88-99.7% from peak; effective throughput ~62% of peak.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: reward==1 long-turn penalty + val temperature 0.7; prune scratch files
- smith_agent.py: add length_penalized_reward() that penalizes ONLY solved
(reward==1) trajectories: reward = 1 - λ·clip((n_turns-T0)/(max_turns-T0),0,1);
run_agent_loop now returns (submitted, turns_used). Tunable via env
SMITH_LEN_PEN_T0 (default 55) / SMITH_LEN_PEN_LAMBDA (default 0.2). Reports
raw_value + n_turns on the reward event for monitoring.
- job-template-openai.yaml: surface SMITH_LEN_PEN_T0=55 / SMITH_LEN_PEN_LAMBDA=0.2.
- tests: cover the penalty (solved-only, within-budget, ramp-to-cap, degenerate
span) and unpack the new run_agent_loop tuple.
- server.yaml: default_proxy.val.temperature 0 -> 0.7 (validation samples at 0.7).
- README: add `export FLA_TILELANG=0` to the run recipe.
- Remove scratch/dev files (enqueue_rollouts.py, rollout_stats.py, gen_*.py,
monitor_controller.sh, run_qwen3_8b_async.sh, serve_vllm_qwen3_8b.sh,
watch_failpods.sh, reward_table.db, reward_table_8b_baseline.md, rollout_test.md,
delete.jsonl).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: update mixed_instance_ids.txt to 5407 (add 162 rescued from 128k rerun)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: block network-based reward hacking (NetworkPolicy + agent guards)
Agents were downloading upstream correct source (curl raw.githubusercontent,
pip install <target pkg>, wget, urllib) to overwrite the buggy file — ~26% of
rollouts — after git was already blocked. Two layers now cut this off:
- Network root fix: deploy Calico VXLAN (replacing flannel, which cannot enforce
NetworkPolicy) + default-deny egress that only allows the agl-lite server IP,
so pods have no public internet. See canal.md for the full deploy/verify/
rollback runbook. Adds canal-policy-only.yaml, networkpolicy-egress-lockdown.yaml,
and an app=agl-rollout label on the job template pod.
- Code backstop: extend smith_agent.py _forbidden_action to reject curl/wget/
pip-install/urllib and test-harness file writes (conftest/pytest.ini/
sitecustomize), and align the prompt (drop "you may install it").
Also enables fused kernels in the train configs (unrelated perf tweak).
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: gate long-turn penalty to training only; README -> mixed dataset
- smith_agent.py: length_penalized_reward() now takes is_train and only shapes
reward when is_train AND solved. Validation reward stays the true, unshaped
metric (it drives checkpoint selection). main() derives is_train from
AGL_OPENAI_BASE_URL (/mode/train/ marker; fail-safe: never penalize val) and
logs mode. Fixes the earlier bug where the penalty applied to val too.
- tests: pass is_train explicitly; add test_length_penalty_skips_validation
asserting val reward is never reshaped (even a long solved rollout keeps 1.0).
- README: point the active Qwen3.5-9B recipe at train_dataset_mixed.jsonl (6343
instances = mixed_dataset minus the 3 network-dependent repos dspy/pydantic/
MONAI whose eval needs runtime pip install and breaks once pods are offline).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* swe_smith: add prompt-length penalty (plan B) + align train config to bh103or9
Penalize context bloat on SOLVED training rollouts: track the largest
server-reported usage.prompt_tokens per rollout (no local tokenizer) and
subtract up to 0.1 as the longest prompt grows from 50K to 64K tokens.
Gated on raw solved status so it stacks with the long-turn penalty (plan A)
and never reshapes validation reward. Tunable via SMITH_PROMPT_PEN_*.
Align train config to the bh103or9 run: model Qwen3.5-9B, gpu_mem_util 0.8,
max_num_batched_tokens 8192, ppo_max_token_len_per_gpu 16384.
Co-Authored-By: Claude <noreply@anthropic.com>
* add max_ppo_update_times
* Merge rollout-level advantage computation (cherry-pick ee78dbe)
Add optional rollout-level advantage (algorithm.enable_rollout_level_advantage)
and its module + tests, wired into AglLiteRayPPOTrainer alongside the existing
compute_advantage path. Cherry-picked from SiweiPro's ee78dbe to take only the
advantage change without the unrelated swe_verify work on that branch.
Co-Authored-By: SiweiPro <18474108006@163.com>
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: turn penalty T0=80, enable max_ppo_update_times=2 + bypass_mode
- Long-turn penalty now starts at 80 turns (was 55), max penalty 0.2 at the
100-turn cap.
- Set agentlightning.max_ppo_update_times=2 to bound PPO updates per step.
- Enable rollout_correction.bypass_mode: reuse rollout log-probs as old_log_prob
for importance sampling, skipping the old-logprob forward pass.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: untrack examples/swe_smith/infra (local-only tooling)
Roofline benchmarks + rollout-stats tooling are local infra, not part of the
training example. Remove from the tree and exclude locally so they stay off origin.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith: untrack reward_table.db (large local-only artifact)
30MB reward cache is a local build artifact, not source. Remove from the tree
and exclude locally so it stays off origin.
Co-Authored-By: Claude <noreply@anthropic.com>
* swe_smith / swe_verify: untrack local-only files (docs, dataset curation, network policies, comparison scripts, swe_verify example)
Move these to .git/info/exclude so they stay off origin while remaining local:
- examples/swe_verify/ (7 files, plus tests/examples/test_swe_verify.py to avoid CI ImportError)
- examples/swe_smith/README.md
- examples/swe_smith/canal.md, filter.md, report_qwen35_9b.md (internal notes)
- examples/swe_smith/canal-policy-only.yaml, networkpolicy-egress-lockdown.yaml
(cluster-specific network policies)
- examples/swe_smith/mixed_instance_ids.txt (local dataset curation, 5407 rows)
- examples/calc_x/comparison/ (local benchmark scripts)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: hzy46 <362583303@qq.com>
Co-authored-by: SiweiPro <18474108006@163.com>
When test_freq fires inside async_fit(), validation runs through the legacy
_rollout(is_train=False) path which ends with abort_all_requests(). Without
a subsequent update_weights() call, vLLM 0.7.1+ engines can stay paused and
the next step's resume_generation() does not recover them. Observed as the
step-32→33 deadlock: all rollout replicas hang at 20:42:17Z, training enters
a 16-hour zombie state with succeeded=82, timeout_rollouts growing by 256
per step, reward=0, grad_norm=0, and global_seqlen=32 (placeholder-only).
train→train avoids this because _async_train_step always ends with
sleep_replicas() + update_weights(global_steps), which forces a vLLM
engine state refresh while vLLM is offloaded. val→train must do the same
explicitly. Snapshot/restore of the async carry-over bridge state
(_validate_preserving_async_carry_over) was already in place; the missing
piece was the engine reset itself.
The reset must call sleep_replicas() BEFORE update_weights(): the reset
runs while vLLM is still awake holding weights + KV cache on GPU, and
FSDP's state_dict() unshard collides with vLLM's resident memory during
DTensor redistribute() — observed OOM at step 32→33 on 40 GB A100s with
GPU 0 at 39.49 GB used / 29.88 MiB free, vLLM TP worker holding 26.94 GiB,
FSDP unshard needing ~12.48 GiB. Mirror the train→train cycle pattern
(sleep_replicas() then update_weights()) so vLLM frees its weights + KV
cache before FSDP gathers params for IPC.
Tests:
- positive: drive async_fit through one train step plus a test_freq-
triggered validation; assert sleep_replicas runs between val_rollout
and the engine-reset update_weights(global_steps) call.
- negative: assert no spurious engine reset (neither sleep_replicas nor
extra update_weights) fires on non-val steps — a stray reset would
leave vLLM paused without a wake_up partner.
Verified the positive test fails on the pre-fix code with the intended
assertion message.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fixes a crash that brought down a long-running ScienceWorld GRPO RL
training job at step 92/124 (~5h 56min in) on 8xA100. Root cause was a
plain HTTP/1.1 keep-alive race between the trainer's httpx pool and the
agl-lite uvicorn server, not OOM or any GPU/CPU memory leak (verified:
GPU max_memory_allocated was a flat 20.8026 GB across all 92 steps).
Trigger
-------
The Ray-driven `_AglTaskRunner` actor uses a long-lived
`AglLiteClient` (httpx.AsyncClient) to poll `/api/rollouts/{rid}` and
post events to the server. Between bursts of traffic an idle pooled
socket can sit unused for >5s. uvicorn's default `timeout_keep_alive`
is 5s, so the server half-closes the connection. The very next request
that reuses the (now stale) socket fails at the transport layer with
`httpx.ReadError` ("Server disconnected without sending a response") or
`httpx.RemoteProtocolError` -- before the client has any HTTP-level
signal to act on.
Impact
------
Before this fix, a single transient transport error propagated
straight up the call stack:
httpx.ReadError
-> raises out of `_AglTaskRunner.run.remote(...)`
-> `ray.get(...)` reraises as `RayTaskError(ReadError)`
-> Python driver exits, atexit triggers `ray.shutdown()`
(exit_type=INTENDED_USER_EXIT)
-> GCS marks the job finished and force-kills *every* actor:
8x vLLM HttpServer, 8x FSDP workers, the _AglTaskRunner actor
-> raylet receives SIGTERM ~150ms later
-> entire training job dies
On the 5/21 run this happened at 17:25:17.344 UTC at step 92, after
a successful POST to vLLM 50ms earlier. No checkpoint between
step_64 and step_92 was saved.
Fix (layered defense)
---------------------
1. Server side: bump `timeout_keep_alive` from the uvicorn default of
5s to 120s in `agl_lite/cli.py::serve`. This eliminates the race
in the common case for poll-heavy clients.
2. Client side: add `_RetryingTransport`, an
`httpx.AsyncBaseTransport` wrapper around the default async
transport that retries on transient transport errors
(`ConnectError`, `ReadError`, `WriteError`, `RemoteProtocolError`,
`PoolTimeout`) with exponential backoff (0.1, 0.2, 0.4, 0.8s; max
4 attempts). HTTP-level errors (4xx/5xx) are *not* retried -- they
are returned as-is so application code keeps full control.
Wiring is centralized in `AglLiteClient.__init__`, so all ~20
existing call sites benefit automatically; tests that inject their
own `httpx.AsyncClient` (e.g. via `httpx.ASGITransport`) are
unaffected because the wrapper is only installed when `client is
None`.
Retrying POSTs is safe in this design: by definition a transport
error happens before any response headers arrive, so the server has
not produced any application-visible side effect for this request.
Tests
-----
- `tests/test_client.py::TestRetryingTransport`: 4 new tests cover
retry-then-succeed (ReadError + RemoteProtocolError), retry on
ConnectError, raising after exhausting attempts, and *not* retrying
application-level 500 responses.
- Full unit suite (375 tests) passes; ruff + pyright clean on touched
files.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move streaming response assembly out of proxy.py into a dedicated
sub-package with one module per LLM API format:
- assemblers/__init__.py: registry (select_assembler) mapping path
suffixes to format-specific assembler functions
- assemblers/chat_completion.py: OpenAI /v1/chat/completions (moved)
- assemblers/completion.py: legacy /v1/completions (new)
- assemblers/anthropic.py: Anthropic /v1/messages (new)
proxy.py now calls select_assembler(path) instead of an inline is_chat
check. Unknown paths fall back to raw {"chunks": [...]}.
Adding a new format = one new file + one registry entry.
Tests: 19 new unit tests for assemblers (registry dispatch, each format,
edge cases). Existing proxy integration tests updated. 336 total pass.
chunk[0] can carry both role+content AND token_ids simultaneously.
Updated test_streaming_preserves_token_ids to mirror real vLLM output:
chunk[0]: delta={role, content='Hi'}, token_ids=[100], prompt_token_ids=[...]
chunk[1]: delta={content=' there'}, token_ids=[200, 201]
chunk[2]: delta={content='!'}, token_ids=[300], finish_reason='stop'
Verifies token_ids from ALL chunks (including chunk[0]) are concatenated.
_assemble_chat_completion now carries vLLM-specific training fields:
- prompt_token_ids: lifted from first SSE chunk to top-level dict
- token_ids: concatenated across all chunks into each choice
_trim_model_request (triplet extraction) updated:
- dict branch (assembled) is now the primary path for both
streaming and non-streaming responses
- list branch kept for backward compat (pre-assembly format)
Without this fix, the assembly commit (c25eb23) silently dropped
all token IDs from streaming responses, breaking VERL triplet
extraction — prompt_token_ids and response_token_ids would both
be empty lists.
Tests: +2 (test_streaming_preserves_token_ids,
test_streaming_token_ids_triplet_roundtrip)
Total: 316 passing
_parse_sse_response was format-specific (assumed choices[0].delta.content).
The gateway is a general-purpose proxy and must not interpret payload structure.
Split into two functions with clear contracts:
_parse_sse_chunks(raw) -- generic, format-agnostic SSE parser
_assemble_chat_completion(chunks) -- OpenAI chat completion only; explicit docstring
stating it must not be called for other paths
_forward_streaming now takes 'path' and dispatches:
path ends with 'chat/completions' -> _assemble_chat_completion -> ChatCompletion dict
any other path -> {"chunks": [...]} raw list
This means:
/v1/chat/completions streaming -> response: {id, object, choices[message], usage}
/v1/completions streaming -> response: {chunks: [{choices[0].text, ...}]}
any custom endpoint -> response: {chunks: [...]}
Tests: added test_streaming_non_chat_path_stores_raw_chunks
_parse_sse_response previously returned a list of raw SSE delta chunks.
This was:
- inconsistent with non-streaming (which returns a single ChatCompletion dict)
- verbose: 294 chunks per call vs one assembled response
- inconvenient to inspect: consumers had to manually concat delta.content
Now assembles chunks into a unified dict::
{
"id": "chatcmpl-...", "object": "chat.completion",
"created": <int>, "model": "<model>",
"choices": [{"message": {"role": "assistant", "content": "<full text>"},
"finish_reason": "stop"}],
"usage": <last chunk usage or None>
}
Streaming and non-streaming model_request events now have the same shape.
Empty stream (no data chunks) returns {} consistently.
Tests updated to assert on assembled shape.
Fixtures now seed hooks._pod_spec directly (bypassing on_startup/
AGL_POD_SPEC_TEMPLATE) with a minimal pod spec matching job-template.yaml.
Assertions updated:
- result.config.environment_variables["AGL_TASK_INPUT"] (old, removed field)
-> _get_task_input() helper: finds AGL_TASK_INPUT in pod spec container env list
ServerSettings, ControllerSettings are now plain BaseModel — no ambient
env reads, no env_prefix, no pydantic-settings dependency.
cli.py is the sole boundary that maps AGL_* env vars to settings fields,
using typer.Option(default, envvar='AGL_*'):
- CLI arg wins if passed
- env var is fallback (read at call time by typer)
- --help shows env var name, not value
- type conversion (int) handled by typer
- required fields (base_url, namespace, job_manifest_template) use
typer.Option(..., envvar=) — typer fails cleanly if neither arg nor
env var is provided
DeploySettings stays BaseSettings (explicit _env_file, not ambient).
controller: all settings now explicit options in cli.py with envvar=;
previously only --job-manifest-template was explicit.
rename: ServerSettings.agl_key -> key (server/config.py, app.py, tests)
SECRET_NAME = 'agl-lite-keys' was already a hardcoded constant in deploy.py
with no way to configure it differently. The name was simply propagated as
AGL_SECRET_NAME through ConfigMap -> pod env -> ControllerSettings ->
job_builder -> Jinja2 template for no reason.
Changes:
- job-template.yaml.j2: replace {{ secret_name }} with agl-lite-keys
- job_builder.py: remove secret_name from _template_context
- ControllerSettings: remove secret_name field
- deploy.py: remove AGL_SECRET_NAME from agl-lite-config ConfigMap
- k8s.yaml: remove AGL_SECRET_NAME env var (was read from ConfigMap)
- tests: remove secret_name from all ControllerSettings fixtures; update
assertion to expect hardcoded 'agl-lite-keys'
Previously the field was named lite_url, which pydantic-settings mapped to
AGL_LITE_URL. The pod injects AGL_BASE_URL, so the env var was silently
ignored — the CLI passed the value explicitly, masking the mismatch.
Renaming to base_url makes pydantic-settings correctly bind AGL_BASE_URL,
consistent with the rest of the codebase.
ControllerSettings.job_manifest_template:
- Now has a default of "deploy/controller/job-template.yaml.j2"
- cli.py --job-manifest-template option no longer required (same default)
DeploySettings:
- Add pod_spec_template: str | None = None (AGL_POD_SPEC_TEMPLATE)
User pod spec template — plain YAML pod spec fragment loaded by the base
RolloutHooks.on_startup into self._pod_spec at server startup
RolloutHooks.on_startup (base):
- Reads AGL_POD_SPEC_TEMPLATE from env; if set, yaml.safe_load into self._pod_spec
- No-op when env var is absent (no FileNotFoundError, no KeyError)
- Docstring explains the two distinct files: AGL_JOB_MANIFEST_TEMPLATE (Jinja2
scaffold used by controller) vs AGL_POD_SPEC_TEMPLATE (plain YAML used by hooks)
- Subclasses that only need per-sample customisation no longer need on_startup
examples/swe_bench/hooks.py:
- Remove on_startup override — base handles file loading now
- Remove SWE_POD_SPEC_TEMPLATE; use AGL_POD_SPEC_TEMPLATE instead
deploy/agl-lite.env.example:
- Expand controller section: document AGL_JOB_MANIFEST_TEMPLATE with its default,
and AGL_POD_SPEC_TEMPLATE with description and example path
RolloutConfig:
- Remove image, command, environment_variables, mount, overrides, Mount class
- Add pod_spec: dict | None — full pod spec fragment assembled by on_enqueue hook
- Keep timeout and max_retries (Job-level execution policy)
RolloutHooks base class:
- Add on_startup(self, store) lifecycle method — called once at server startup
- Add copy_pod_spec() — deep copies self._pod_spec with nil-guard
- Add get_container(pod_spec, name) staticmethod — finds container by name
- Add _pod_spec convention and module-level docstring with usage pattern
server/app.py:
- Call hooks.on_startup(store) in create_app() after store is initialised
job_builder.py:
- Remove user_pod_spec param — reads rollout.config.pod_spec instead
- Remove _ensure_agent_container, _get_agent_container helpers
- Remove all named-field merge logic (image, command, env vars, mounts, overrides)
- build_job_spec(rollout, settings, manifest_template) — three steps only:
1. render template → scaffold + patcher
2. merge config.pod_spec into scaffold
3. inject patcher env into all containers
4. set backoffLimit / activeDeadlineSeconds
reconciler.py:
- Remove _resources_cache and _get_job_template — controller no longer
fetches resources; pod spec arrives pre-assembled in rollout.config.pod_spec
- build_job_spec call simplified (no job_template arg)
examples/swe_bench/hooks.py:
- Rewrite using on_startup + copy_pod_spec + get_container pattern
- Load pod spec template from SWE_POD_SPEC_TEMPLATE env var at startup
- Set per-instance image and env vars directly in pod_spec.containers[0]
- Hoist activeDeadlineSeconds → config.timeout
tests: updated throughout (315 passing)
- Remove _DEFAULT_MANIFEST_TEMPLATE_PATH and _load_manifest_template from
reconciler.py; reconciler reads Path(settings.job_manifest_template).read_text()
directly — no fallback, fails loudly if not provided
- ControllerSettings.job_manifest_template: str (required, no None default)
- cli.py: add --job-manifest-template required option to 'agl-lite controller'
- job_builder.py: rename local pod_spec variable to scaffold_pod_spec to
eliminate shadowing of the pod_spec parameter
- tests: fixture reads deploy/controller/job-template.yaml.j2 directly;
both settings fixtures supply job_manifest_template
- migrate deploy flow and configs from AGL_LITE_URL to AGL_BASE_URL naming
- update deploy schema fields to agl_base_url_* and local_state_dir semantics
- align controller/deploy/example wiring with base-url naming
- add docs/deploy.md with mode-specific host vs pod URL behavior
Gateway config now supports ${VAR_NAME} syntax — resolved from environment
at load time via os.path.expandvars(). Useful for model name routing:
routes:
- model_in: "*"
model_out: "${AGL_MODEL_NAME}"
SWE-bench gateway config updated to use this — Claude Code's model names
(e.g., claude-sonnet-4-20250514) get rewritten to the actual backend model.
2 gateway config tests added (317 total passing).
Yuge Zhang