3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses!

Documentation  ·  Technical Report (Coming Soon)  ·  MIT License

⚡ Key Features

• 🪶 ~3,500 lines of core Python: We treat simplicity as the first principle.
• 🧩 Train with real agent harnesses: Agents interact with the model through the Agent Lightning v1.0 proxy with ZERO changes, while keeping tools, context, control flow, and environments in the loop.
• ☸️ Native Kubernetes support: Run agents directly as Kubernetes Jobs without relying on external sandbox services.
• 💻 Full coding agent training example: Using only 6K training samples, an end-to-end Qwen3.5-9B workflow improves SWE-bench Verified from 41.8% to 56.4%, a gain of 14.6 percentage points. We release the full pipeline, including data cleaning, reward-hacking prevention, and training scripts.

⚡ Installation

The following is an example installation on a CUDA 13.0 machine:

cd <this-repo>
uv sync
bash scripts/setup_verl.sh 0.8.0 cu130

See the Installation Guide for details.

⚡ Architecture

Agent Lightning v1.0 keeps the training architecture simple with three lightweight components:

• Trainer: Runs VERL and vLLM, builds training samples, and updates the policy.
• API Gateway: Proxies model requests and captures training data.
• Rollout Controller: Runs agents locally or as Kubernetes Jobs.

The Trainer creates rollouts, the Controller launches agents, and the Gateway turns interactions into training data, while agents continue to run with their real harnesses.

⚡ Results

We evaluate Agent Lightning v1.0 across several practical training domains, including Search R1, LLM-in-Sandbox, and Coding Agent. Pure RL delivers substantial improvements across all three domains, as shown below.

⚡ Documentation

Section	Content
Installation	Base environment and VERL GPU stack
Quick Start	Local first run and end-to-end flow
Basics	Components, rollouts, events, and trajectories
Trainer Configuration	VERL integration and trace aggregation
Server Configuration	Gateway and model proxy settings
Controller Configuration	Local and Kubernetes runners
Asynchronous Training	Collocated async collection and pause/drain

⚡ Examples

Example	Description
Calc-X	POC math reasoning example with AutoGen and MCP calculator tools, requiring only one GPU.
GSM8K	POC grade-school math reasoning example.
ScienceWorld	Interactive science tasks in a text-based environment.
Search-R1	Multi-turn retrieval and reasoning agent.
LLM-in-Sandbox	General agent with computer and code execution tools.
Coding Agent	Coding agent trained with repository tests.

⚡ License

Agent Lightning v1.0 is released under the MIT License.