The Skill Factory turns Webwright's solve trajectories into a growing library of
reusable, parameterized skills that are plain Python + Playwright — code you can
run without a model and compose into the next task instead of re-exploring a site.
The loop is solve -> gate -> group by template -> distil -> replay-verify ->
library -> reuse, with two independent gates. An input gate keeps untrustworthy
solves out (gold answers, or a self-verify shape/non-empty/agent-report check);
an output gate replays each distilled skill standalone, with no model, and admits
it only if it reproduces its own training answers. That second gate is what lets
a landed skill carry a real grade — executable (replay ran and reproduced),
reference (replay ran and failed; kept as a labelled prior the agent reads), or
unverified (replay skipped).
Commands:
- init — a one-line need becomes a skill.yaml skeleton you fill with ground truth
- build — solve N instances of a spec, then learn from them (parallel, resumable)
- learn — distil trajectories you already have into the library
- update/skill_use — the manual manifest path and the solve-time library query
Includes the flight-schedule example end to end (spec, trajectories, and a
verified executable skill), docs (quickstart in the module README, plus reference
and manual mode), a demo video and pipeline diagram, and a test suite covering
the gates, distillation, replay comparison, and config wiring.
local_browser.yaml's prompts were copy-pasted from base.yaml's
bash_command-driven workspace flow, which doesn't fit a live persistent
Playwright session. The agent was told to launch its own browser inside
final_script.py while also being told never to import or launch
Playwright; and it was told to shell out via `run_command(...)`, a
helper that does not exist anywhere in the codebase.
Reframe the live-browser mode around its actual capabilities:
- Drop final_script.py, plan.md, self_reflect_config.json, final_runs/,
image_qa, and self_reflection from the prompts. There is no workspace
directory in this mode; the agent drives the live `page` and reports
its answer in `final_response` when done.
- Override `agent.require_self_reflection_success: false` so the
completion gate (which checks for `final_runs/run_<id>/self_reflect_result.json`)
does not block `done: true` in this mode.
- Override `agent.summary_user_prompt` to talk about live browser
state (URL, controls, selectors) instead of workspace artifacts
(`plan.md`, `final_script.py`, `final_runs/`) that don't exist.
- Document in the yaml header what each step's observation contains
(URL / title / stdout / ARIA snapshot / screenshot path) and note
that the screenshot is NOT visually attached by default — flipping
`model.attach_observation_screenshot: true` enables multimodal image
input.
Add a context-bounding knob in `AgentConfig`:
- `keep_last_n_observations` (default -1, disabled) strips the ARIA
snapshot payload from observation messages older than the most
recent N, both in the rendered text content and the `extra`
observation dict. URL / title / printed stdout are preserved so the
agent still knows what it did. local_browser.yaml opts in with N=1.
Validated end-to-end against "Search for flights from SEA to JFK on
2026-08-15 to 2026-08-20" via base.yaml + local_browser.yaml + model_openai.yaml:
the agent reached the Google Flights results page (round trip,
SEA -> JFK, Aug 15 -> Aug 20) in 7 steps with N=1 pruning (vs 9 steps
without), and the saved trajectory shows ARIA stripped from the first
6 observations and preserved on the 7th.
Closes#3.
The image_qa and self_reflection inner tools now route through the configured model registry instead of hardcoding the OpenAI Responses API. Anthropic-only runs (`webwright -c base.yaml -c model_claude.yaml`) work end-to-end with only `ANTHROPIC_API_KEY` set.
Co-authored-by: Matt Van Horn <mvanhorn@users.noreply.github.com>
Demi Wang