agl-lite provides transparent LLM request capture, a rollout data store, and Kubernetes-native agent execution — all behind a single HTTP endpoint. Agents use standard OpenAI SDKs with zero instrumentation; the gateway captures everything automatically.

Architecture

Three groups connected only by HTTP:

Group	What it does	Managed by
Compute Backend	Model training (VERL/Megatron) + inference servers (vLLM)	User
agl-lite Service	Gateway (LLM proxy + event capture) + Data Store (rollouts, events, models)	agl-lite
Agent Runner	K8s controller + agent pods (any container, any language)	agl-lite + K8s

Key Design Choices

Self-owned LLM gateway — a purpose-built reverse proxy replaces litellm, capturing all request-response data transparently as it flows through
Gateway-level data capture — instead of instrumenting agents with OpenTelemetry, the gateway records request-response pairs during transfer — the proxy is the instrumentation
K8s-native agent runner — K8s Jobs as the execution unit, rollout-scoped attempt IDs for trace grouping, Job lifecycle as the retry mechanism — the store focuses purely on data, not execution control

Quick Start

Run the math PoC — 30 GSM8K problems solved by Qwen2.5-1.5B-Instruct via vLLM:

# Prerequisites: minikube running, uv installed, GPU with nvidia-container-toolkit
git clone https://github.com/<org>/agl-lite && cd agl-lite

# Start vLLM inference server
scripts/start_vllm.sh

# Configure and run
export AGL_KEY=$(openssl rand -hex 32)
examples/math-poc/run.sh

This builds images, deploys the controller to minikube, starts agl-lite on the host, and runs a multi-iteration RL loop: enqueue tasks → agents solve via vLLM → gateway captures trajectories → algorithm scores answers. See the Getting Started guide for the full setup walkthrough.

How Agents Work

Agents are plain containers that read env vars and call an OpenAI-compatible endpoint. No agl-lite import, no base class — any language, any framework:

import os, json, openai

task = json.loads(os.environ["AGL_TASK_INPUT"])
client = openai.OpenAI()  # reads OPENAI_BASE_URL automatically

response = client.chat.completions.create(
    model="gpt-4.1",  # gateway routes to your vLLM
    messages=[{"role": "user", "content": task["prompt"]}],
)
# Gateway captures this call automatically — no instrumentation needed

The controller injects 4 env vars into every agent pod: OPENAI_BASE_URL, OPENAI_API_KEY, AGL_TASK_INPUT, and AGL_EVENT_URL. See What Happens Next for details.

Examples

Example	Description	Mode
Math PoC	GSM8K problems with Qwen2.5-1.5B-Instruct	Mock (CPU) or vLLM (GPU)
SWE-bench	Coding tasks with Claude Code agent	vLLM + per-instance Docker images

Documentation

Section	Content
Getting Started	Prerequisites, setup flow, first run
Architecture	Full system design — data models, API spec, components
K8s Controller	Controller design and implementation details
Dev Guidelines	Code conventions, tooling, concurrency model

Project Status

~3.5K lines of source code, 333 tests
Gateway with route config, streaming proxy, and automatic event capture
In-memory store (rollouts, events, models, resources)
K8s controller with Job lifecycle management
Python client library (AglLiteClient)
VERL integration (AglLiteRolloutBridge) with triplet format
Math PoC end-to-end (mock + real vLLM)
SWE-bench example with Claude Code

License

TBD