Compare commits

...

814 Commits

Author SHA1 Message Date
Henry Su 72b2c67054 fix(sandbox): finish dependency cleanup on cancellation (#4607) 2026-08-23 22:14:40 +00:00
Jeremy Schoemaker 1a55d70d8e fix(core): max_turns no longer clobbers a tripped input guardrail exception in streaming (#4606) 2026-08-24 07:07:32 +09:00
陈志谦 f81c322563 fix(core): strict_schema error message, REPL whitespace input, debug docstrings (#4600) 2026-08-24 07:07:18 +09:00
Kazuhiro Sera 233467994f fix: enforce public type alias contract coverage (#4595) 2026-08-23 03:39:56 +00:00
Kazuhiro Sera 89fab0fc0d feat: customize output guardrail blocked messages (#4594) 2026-08-23 08:45:14 +09:00
Kazuhiro Sera 9da8f49637 fix(voice): use monotonic STT event deadlines (#4593)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-22 22:34:06 +00:00
Henry Su 8cd1f5e6e5 fix(core): strip created_by when replaying RunItems as input (#4568) 2026-08-22 22:18:42 +00:00
Henry Su 042d84a15c fix(mcp): deduplicate managed servers (#4591) 2026-08-23 07:11:05 +09:00
Henry Su 4f7c1d668f fix(realtime): advance crossed guardrail thresholds (#4590) 2026-08-23 07:10:54 +09:00
Kazuhiro Sera 7f7a44f8dc docs: updated heading anchors in translated pages 2026-08-22 22:27:36 +09:00
Sylvester Kaczmarek 3e6715573d fix(mcp): clear active MCP servers after cleanup (#4586) 2026-08-22 22:20:26 +09:00
Kazuhiro Sera 5b8f6c7174 fix: freeze the public voice API contract (#4578) 2026-08-22 18:08:27 +09:00
Kazuhiro Sera 5f6a733284 chore: update review skill details 2026-08-22 17:39:41 +09:00
saime428 60c2c4120e docs: preserve English heading anchors in translated pages (#4580) 2026-08-22 04:52:33 +00:00
Kazuhiro Sera 904bc6988f test: stabilize release integration tests 2026-08-22 09:57:19 +09:00
Kazuhiro Sera 4ccc32e374 test: update auto-run list 2026-08-22 09:57:12 +09:00
Kazuhiro Sera d22234480f fix(voice): honor client config for streamed STT (#4575)
Co-authored-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
2026-08-22 08:32:57 +09:00
Kazuhiro Sera 119ad2a492 fix(voice): forward streamed STT language and prompt (#4574)
Co-authored-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
2026-08-21 23:31:11 +00:00
Sean 4c2810c11c feat(sandbox): allow labels on Docker sandbox containers (#4564) 2026-08-22 07:53:52 +09:00
Kazuhiro Sera 707457064e chore: refine repo skills 2026-08-21 14:13:58 +09:00
Kazuhiro Sera 077ec65edc chore: clarify edge-case policies 2026-08-21 14:06:20 +09:00
Henry Su 17ba331bb0 fix(extensions): nest extra_body on the any-llm chat path (#4544) 2026-08-21 05:55:48 +09:00
Henry Su 1b7eb28f25 fix(core): fail closed on empty tool arguments (#4545) 2026-08-21 05:55:29 +09:00
Sylvester Kaczmarek f73e747530 fix(voice): support custom OpenAI TTS voice IDs (#4541) 2026-08-20 11:36:45 +00:00
Weike Zhang e26a7d8aed fix(chat-completions): raise ModelBehaviorError on truncated empty completions (#4513) 2026-08-20 07:49:29 +00:00
Henry Su eb3a5d5b5d fix(mcp): deep-copy cached tools before returning them (#4525) 2026-08-20 06:13:32 +00:00
Sylvester Kaczmarek 502bccddd3 fix(tracing): flush buffered data after tracing is disabled (#4528) 2026-08-20 06:12:09 +00:00
Kazuhiro Sera 75d6a6f142 chore: update review policies for runtimne validation code changes 2026-08-20 14:42:43 +09:00
Kazuhiro Sera 2af94722d9 fix: keep checkout line endings LF 2026-08-20 12:04:27 +09:00
Sylvester Kaczmarek 9fd6c81c59 fix(voice): include current OpenAI TTS voices (#4535) 2026-08-20 02:55:56 +00:00
green3sf bfb981d63e fix(core/voice): honor OpenAI provider options over default clients (#4530) 2026-08-20 02:55:37 +00:00
Sylvester Kaczmarek 7e55afc950 fix(voice) reject ignored explicit-client options (#4527) 2026-08-20 06:01:14 +09:00
Kazuhiro Sera 3e0dc82ebd docs: update translated pages 2026-08-19 23:41:38 +09:00
zhewen tan fe34ba3aa9 fix(visualization): preserve nodes with duplicate names (#4512) 2026-08-19 23:07:35 +09:00
green3sf 629f9b56a5 fix(core): detach aggregated request usage entries (#4519) 2026-08-19 22:55:32 +09:00
Chair403 aee76c8e97 docs: fix wording in handoff example prompts (#4520) 2026-08-19 13:53:11 +00:00
Kazuhiro Sera 727e729f21 docs: document v0.22.0 behavior changes (#4522) 2026-08-19 22:45:45 +09:00
Kazuhiro Sera 4df9ecfae1 release: 0.22.0 (#4523) 2026-08-19 22:41:43 +09:00
Chirag Gupta fb8fa1ba5c fix(core): isolate usage between RunState checkpoints (#4479) 2026-08-19 17:50:31 +09:00
Henry Su 9432f7ed30 fix(visualization): expand handoff() targets in agent graphs (#4517) 2026-08-19 17:41:33 +09:00
Weike Zhang 0486792662 fix(core/extensions): reject terminal failed/incomplete responses in non-streaming get_response (#4516) 2026-08-19 17:40:49 +09:00
teachershuang 36dbc36857 docs: add testing resources to llms indexes (#4509) 2026-08-19 15:41:56 +09:00
Kazuhiro Sera ed644fc7c3 fix(core): redact blocked tool outputs from replay state (#4507) 2026-08-19 15:39:24 +09:00
Kazuhiro Sera 2c5560339c test: make stream event ordering deterministic 2026-08-19 07:31:35 +09:00
Kazuhiro Sera 21a1f9b4e6 fix: avoid Windows integration bootstrap crashes 2026-08-19 07:16:06 +09:00
Kazuhiro Sera 32e452671c ci: change windows ci configuration for stability 2026-08-19 06:57:46 +09:00
Kazuhiro Sera ebb746dc00 fix: keep Codex verification for development sandboxed (#4508) 2026-08-18 11:46:40 +00:00
Kazuhiro Sera 82e3571fc5 refactor: move workflow execution out of repository skills 2026-08-18 19:47:43 +09:00
Kazuhiro Sera 9648a401a0 fix: make runtime probe approval explicit in skills 2026-08-18 16:04:39 +09:00
Kazuhiro Sera 1a4cfa20a3 fix: harden agent workflow validation 2026-08-18 08:06:42 +09:00
Sylvester Kaczmarek c5f6a71ea9 fix(tracing): clean up processors after tracing is disabled (#4502) 2026-08-18 06:48:35 +09:00
Kazuhiro Sera e5831826fd fix(sandbox): enforce Windows mypy compatibility (#4499)
Co-authored-by: rome-xi <rome-xi@users.noreply.github.com>
2026-08-17 21:37:51 +00:00
Sylvester Kaczmarek 62f02e3333 fix(tracing): respect model-data logging redaction for record_model_error_on_span (#4496) 2026-08-18 06:20:08 +09:00
Sylvester Kaczmarek a77d37e6bc fix(core): reject ignored explicit-client options for OpenAIProvider (#4497) 2026-08-18 06:18:47 +09:00
Sylvester Kaczmarek 057ab1019a fix(core): preserve Griffe logger inheritance (#4494) 2026-08-18 06:08:50 +09:00
abhijeet sharma f5491c5cbb docs: correct Agent.clone list attribute semantics (#4474) 2026-08-18 05:43:47 +09:00
ErenAta16 9ecfdfa498 test: use sys.executable instead of tee in tests (#4478) 2026-08-18 05:39:21 +09:00
Kazuhiro Sera d40f5d9832 ci: align Python version coverage (#4475) 2026-08-17 17:56:36 +09:00
li2631026381-alt 37a7aa20ce fix(sandbox): require apply_patch update hunks (#4470) 2026-08-17 13:53:50 +09:00
Kazuhiro Sera 39327d7c5d docs: update translated pages 2026-08-17 08:03:40 +09:00
Kazuhiro Sera 86b3db59df docs: document v0.21.1 runtime behavior (#4460) 2026-08-17 07:30:02 +09:00
Kazuhiro Sera b01ea1d342 release: 0.21.1 (#4467) 2026-08-17 07:26:03 +09:00
Kazuhiro Sera 2632043a4e fix(realtime): end iteration after clean server close (#4461)
Co-authored-by: ayaangazali <ayaangazali.work@gmail.com>
2026-08-16 18:21:35 +09:00
Kazuhiro Sera 4cb461a7e3 fix(sandbox): validate view_image raster content (#4462)
Co-authored-by: Sylvester Kaczmarek <16242628+sylvesterkaczmarek@users.noreply.github.com>
2026-08-16 09:05:02 +00:00
Kazuhiro Sera fdcec69620 fix: forward Docker network mode in security test 2026-08-16 15:11:57 +09:00
Chirag Honnyal e5f75fdf13 fix(realtime): truncate audio at zero elapsed time (#4457) 2026-08-16 05:47:05 +00:00
Vinove A. 2f1c83d5b7 feat(sandbox): allow Docker sandboxes to disable networking (#4452) 2026-08-16 03:25:13 +00:00
Sergey Filimonov 3a888def33 feat(extensions): add Modal sandbox resource options (#4455) 2026-08-16 03:24:33 +00:00
Kazuhiro Sera b4faf7090c feat(core): add model call timeouts (#4428) 2026-08-16 11:15:36 +09:00
Kazuhiro Sera 2588d154e4 fix: prevent advanced SQLite structure table conflicts (#4454)
Co-authored-by: ayaangazali <ayaangazali.work@gmail.com>
2026-08-16 11:13:06 +09:00
Kazuhiro Sera 94da8ed49c fix(core): count Responses requests without usage (#4453)
Co-authored-by: ayaangazali <ayaangazali.work@gmail.com>
2026-08-16 01:28:43 +00:00
Kazuhiro Sera cb8a2e7e7d feat: add run-scoped sandbox working directories (#4427)
Co-authored-by: Sergey Filimonov <safilimonov@gmail.com>
2026-08-16 08:55:48 +09:00
Kazuhiro Sera dde0bc99fd fix: honor exact call approval decisions (#4447)
Co-authored-by: chiruu12 <103719146+chiruu12@users.noreply.github.com>
2026-08-16 08:40:44 +09:00
Sylvester Kaczmarek 05c789fe4d fix(sandbox): normalize apply_patch paths as POSIX (#4437) 2026-08-15 23:23:08 +00:00
Kazuhiro Sera e90df39686 fix: update cffi lock for Python 3.14 (#4448) 2026-08-16 08:16:22 +09:00
Kazuhiro Sera 60482a3d6b fix(chat-completions): improve Chat Completions reasoning replay (#4432) 2026-08-16 08:14:15 +09:00
Kazuhiro Sera 583fede12e fix(sessions): include compaction usage in run totals (#4446)
Co-authored-by: Arthi Arumugam <arthi1805@gmail.com>
2026-08-16 08:12:30 +09:00
cerebrixos 1c3b72019e docs: list Tuning Engines tracing integration (#4440) 2026-08-15 22:59:36 +00:00
GGbond 7ab35c3516 fix(core): close all MultiProvider children after failures (#4438) 2026-08-16 07:58:08 +09:00
Sylvester Kaczmarek 9aba900293 fix(sandbox): keep model paths POSIX-normalized (#4416) 2026-08-15 09:00:26 +00:00
roryc 6b62225f03 fix(core): reject partially matched stacked anchors (#4431) 2026-08-15 07:11:31 +00:00
Kazuhiro Sera e4cbad9a51 docs: place testing after tracing in sidebar 2026-08-15 12:19:49 +09:00
Kazuhiro Sera 56783dd2a6 docs: update translated pages 2026-08-15 12:09:25 +09:00
Kazuhiro Sera 55bb0b19de docs: updates for v0.21.0 release (#4381) 2026-08-15 11:56:28 +09:00
Kazuhiro Sera 25aa6d94a1 release: 0.21.0 (#4387) 2026-08-15 11:43:38 +09:00
Sylvester Kaczmarek 1a0c08868a fix(sandbox): honor view_image extra path grants (#4417) 2026-08-15 00:28:30 +00:00
Chinmay V 2b2d7fe474 fix(mcp): stop handing the tools cache to callers (#4424) 2026-08-15 07:10:30 +09:00
Henry Su 4e5e7b2695 fix(realtime): handle non-finite audio rates (#4419) 2026-08-14 21:54:43 +00:00
Alex Chang c0f2ff7d8f fix: preserve scripted annotation streaming across Python SDK releases (#4422)
This pull request fixes scripted annotation streaming before the next Python SDK release introduces accurately typed annotation-added events (openai/openai-python#3617).

The current helper passes output-text annotation models directly into streaming events. That works while the released SDK accepts an untyped annotation, but the upcoming SDK validates events against distinct event-specific annotation classes. As a result, the existing helper both fails static type checking and raises a validation error at runtime.

- Preserve annotation contents, event ordering, and sequence numbers across both the released and upcoming Python SDKs.
- Construct events from their serialized payload so each installed SDK applies its own event schema without relying on imports that do not exist in the current release.
- Compare annotation payloads by their public serialized representation instead of requiring unrelated generated model classes to share an identity.
2026-08-14 17:06:00 -04:00
Kazuhiro Sera 50d65f65c3 fix: resume recursive agent tool approvals (#4414) 2026-08-14 22:17:44 +09:00
Kazuhiro Sera 0b93ce8faa fix: isolate RunState checkpoint tool decisions (#4413) 2026-08-14 20:50:23 +09:00
ErenAta16 3e87dc8ab1 test: order test spans by start sequence, not by started_at alone (#4392) 2026-08-14 09:31:42 +00:00
Kazuhiro Sera 95f9d9a103 fix: detach RunState interruption snapshots (#4409) 2026-08-14 18:27:29 +09:00
Kazuhiro Sera 40927c9f95 fix: max-turn handler session semantics (#4412)
Co-authored-by: DongBo <80384548+russeell@users.noreply.github.com>
2026-08-14 18:07:31 +09:00
Rakshit Sharma fc2be56cb0 fix(voice): stop buffering audio when audio tracing is disabled (#4411) 2026-08-14 05:56:13 +00:00
Kazuhiro Sera f5606931f0 fix: align Responses parallel tool calls with converted tools (#4405) 2026-08-14 01:42:41 +00:00
Kazuhiro Sera dc1bef7b88 fix: expose the scripted sandbox session type (#4406) 2026-08-14 10:41:02 +09:00
Kazuhiro Sera c881417164 fix: freeze public testing API state contracts (#4404) 2026-08-14 10:34:49 +09:00
Kazuhiro Sera 4c7713b93c fix(mcp): protect manager lifecycle state snapshots (#4407) 2026-08-14 01:13:23 +00:00
Henry Su 761fcd9819 fix(tools): redact tool output value from output-type validation errors (#4396) 2026-08-14 00:59:24 +00:00
Henry Su 15989e50bc fix(sandbox): snapshot per-op audit policies (#4398) 2026-08-13 23:47:54 +00:00
Henry Su 079e745996 fix(sandbox): snapshot HTTP proxy headers (#4397) 2026-08-13 23:45:12 +00:00
viyatb-oai 95f1c7cc56 fix(codex): preserve resume argument ordering (#4400) 2026-08-14 07:58:40 +09:00
Kazuhiro Sera 1816d2e9f9 docs: scale verification by change risk 2026-08-13 16:56:17 +09:00
Kazuhiro Sera 66ae98fb1e fix: freeze public testing module contracts (#4386) 2026-08-13 16:33:38 +09:00
Henry Su 6bef354de2 fix(voice): reject non-positive audio frame rates (#4382) 2026-08-13 16:01:12 +09:00
Henry Su f251c70c0c fix(run-state): isolate interruption results (#4384) 2026-08-13 04:48:25 +00:00
Kazuhiro Sera fc461eebdf feat: support OpenAI Python 3 and HTTPX2 (#4380) 2026-08-13 12:09:43 +09:00
Kazuhiro Sera 05d6850da5 feat: add scripted model test utilities (#4362) 2026-08-13 11:20:33 +09:00
Kazuhiro Sera 0fc268e3e2 fix(mcp): add configurable retry backoff ceiling (#4379)
Co-authored-by: rxits <132228481+rxits@users.noreply.github.com>
2026-08-13 10:29:33 +09:00
xumaple c12f14d1e9 feat(sandbox): let managed_secrets reference existing Runloop secrets (#4378) 2026-08-13 08:14:52 +09:00
hansu650 5250cb8605 fix(voice): reject non-positive audio channels (#4372) 2026-08-12 14:18:50 +09:00
Jaideep Pyne 39c3a3bb1b docs: clarify PGP key location (#4371) 2026-08-12 02:55:17 +00:00
Lucca Boas dd34097826 fix(runner): close the model stream when a streamed turn ends in a terminal failure (#4366) 2026-08-12 09:43:49 +09:00
Kazuhiro Sera 6beab353ff fix(voice): reject incomplete multichannel audio frames (#4370)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-11 23:46:47 +00:00
Kazuhiro Sera d5b27bb3cd fix(apply-diff): apply stacked anchors sequentially (#4369)
Co-authored-by: Anton Dzyatkovsky (Mac16) <dzyatkovskiy.a@gmail.com>
2026-08-11 23:41:52 +00:00
Henry Su 3c6622d5b9 fix(sessions): handle zero conversation history limits (#4365) 2026-08-11 23:22:12 +00:00
Henry Su e9998afa5b fix(mcp): isolate manager lifecycle results (#4368) 2026-08-12 08:16:13 +09:00
weiwei Wang b75d466d61 fix(extensions): gate AnyLLM parallel tool calls on converted tools (#4363) 2026-08-12 08:07:53 +09:00
FU-max-boop 863b96cfe9 fix(voice): honor WAV sample width (#4361) 2026-08-11 15:24:08 +09:00
Kazuhiro Sera 8ecdac5947 perf: streamline final review evidence reuse 2026-08-11 14:17:38 +09:00
Kazuhiro Sera 7518f367e3 docs: update translated pages 2026-08-11 13:25:04 +09:00
Kazuhiro Sera 80e1baaefd docs: synchronize v0.20.0 features (#4280) 2026-08-11 12:13:38 +09:00
Kazuhiro Sera d2bda3f311 release: 0.20.0 (#4348) 2026-08-11 11:52:30 +09:00
Kazuhiro Sera c0b876379e perf: limit mypy to runtime source 2026-08-11 11:25:48 +09:00
Kazuhiro Sera 27c1060185 fix: reject unsafe strict-schema ref siblings (#4356)
Co-authored-by: snowingfox <1503401882@qq.com>
2026-08-11 11:25:19 +09:00
Lucca Boas 23da2b6254 fix(chat-completions): omit parallel_tool_calls without tools on the Chat Completions path (#4359) 2026-08-11 01:52:33 +00:00
Kazuhiro Sera b2a460d429 chore: preserve review credit for verified type-erasure edits 2026-08-11 10:49:00 +09:00
Kazuhiro Sera 2231eb5d40 fix: reject schemas that exceed safe recursion depth (#4358) 2026-08-11 01:18:01 +00:00
Kazuhiro Sera cda89c8ae4 fix: freeze new public submodule callables 2026-08-11 08:05:43 +09:00
Kazuhiro Sera 67e6d377b8 fix: refresh release candidates consistently 2026-08-11 07:28:20 +09:00
Kazuhiro Sera a5def04fbf feat(realtime): support GA transcription options (#4351) 2026-08-10 22:05:09 +00:00
Kazuhiro Sera 684976659a fix: freeze Runloop platform properties (#4355) 2026-08-11 07:01:15 +09:00
Kazuhiro Sera ee395e2490 test: isolate released API source validation (#4350) 2026-08-10 23:38:48 +09:00
Kazuhiro Sera ea5653a416 fix: make release API contract promotion policy-driven (#4349) 2026-08-10 21:35:41 +09:00
Kazuhiro Sera b3427dcf0d fix: bootstrap release prep dependencies 2026-08-10 19:46:08 +09:00
Kazuhiro Sera 92ca19441a refactor: adjust the final release review when cutting a new version 2026-08-10 19:40:46 +09:00
Kazuhiro Sera 8979f88873 refactor: gate release readiness before branch creation 2026-08-10 19:24:41 +09:00
Kazuhiro Sera 2e3aa55fee ci: check prospective release contracts before merge (#4347) 2026-08-10 19:15:34 +09:00
Kazuhiro Sera 92de6cf547 feat: add local release candidate preparation 2026-08-10 16:04:33 +09:00
Kazuhiro Sera 3b62591fc3 test: harden release compatibility contracts (#4297) 2026-08-10 06:16:10 +00:00
Kazuhiro Sera 8cb02cb8cf perf: defer broad verification until review is clean 2026-08-10 14:36:27 +09:00
Kazuhiro Sera 9a8ecd257d fix: preserve sandbox error contracts during mount redaction (#4344) 2026-08-10 05:20:58 +00:00
Lucca Boas afd11195cf fix(voice): report transcription session close failures to the consumer (#4343) 2026-08-10 11:20:49 +09:00
Kazuhiro Sera 54cc7d938f fix(mcp): bound lifecycle and CI waits (#4342) 2026-08-10 00:00:29 +00:00
Kazuhiro Sera 7da5696020 fix(mcp): serialize manager lifecycle operations (#4340)
Co-authored-by: Abhinav Kumar Singh <abhinav.kr.singh.2610@gmail.com>
2026-08-09 23:28:33 +00:00
Kazuhiro Sera 020db0addf fix: allow post-completion review feedback cycles 2026-08-10 07:21:42 +09:00
Ribhav Jain 54f3f731b0 fix(run_state): serialize containers of models and dataclasses as structured data (#4339) 2026-08-10 07:10:02 +09:00
Ribhav Jain 9775b5eb0f fix(mcp): serialize non-text content blocks as JSON instead of Python repr (#4338) 2026-08-10 07:07:17 +09:00
Kazuhiro Sera e3d7c1727b fix(litellm): omit parallel_tool_calls without tools (#4330) 2026-08-09 22:07:42 +09:00
Kazuhiro Sera 443e1f5113 fix: prune orphaned tool outputs from limited sessions (#4324) 2026-08-09 12:44:16 +00:00
Kazuhiro Sera 7bf73afa47 feat: add durable pending input to RunState (#4325) 2026-08-09 20:59:26 +09:00
Kazuhiro Sera 192e6c7a32 feat: add explicit mount credential exposure acknowledgements (#4321) 2026-08-09 20:49:34 +09:00
Kazuhiro Sera 2eeb86036e perf: validate final review protocol artifacts 2026-08-09 19:01:53 +09:00
Kazuhiro Sera 47498d45b7 fix: preserve local shell outputs across RunState resume (#4320)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-09 17:26:48 +09:00
Kazuhiro Sera 6115461c6b feat(retry): allow applications to approve unsafe replays (#4319)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-09 16:27:33 +09:00
Kazuhiro Sera 4d29c44098 docs: update sandbox reference for agents 2026-08-09 15:24:19 +09:00
Kazuhiro Sera a6548287cf perf: streamline implementation final review 2026-08-09 14:11:03 +09:00
Kazuhiro Sera 8bf878a69f docs: clarify the docs change timing in PRs 2026-08-09 14:01:55 +09:00
Henry Su aec2dfaa12 fix(items): strip output-only created_by when replaying model output as input (#4308) 2026-08-09 04:55:41 +00:00
Henry Su d619466fad fix(run_state): preserve default-valued fields in serialized tool output (#4307) 2026-08-09 13:00:29 +09:00
Kazuhiro Sera 5d3324918f docs: adjust the documentation policies 2026-08-09 11:30:57 +09:00
Kazuhiro Sera 2b115b66bc fix: preserve free-form MCP object schemas (#4310)
Co-authored-by: abhay-codes07 <abhaysingh0293@gmail.com>
2026-08-09 10:39:32 +09:00
Ribhav Jain dcd1705190 fix(run_state): keep acknowledged safety checks serializable after restore (#4316) 2026-08-09 01:08:17 +00:00
Kazuhiro Sera b79bf94def fix(modal): settle snapshot directory mount transitions (#4315) 2026-08-09 10:05:26 +09:00
Kazuhiro Sera 1d7c4b8a53 fix(sandbox): reject unsafe mount credential configurations (#4255)
fix(sandbox): reject unsafe mount credentials
2026-08-09 00:16:52 +00:00
Kazuhiro Sera cdde4d651d fix(run): preserve streamed guardrail session state (#4312)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-08 23:06:28 +00:00
Kazuhiro Sera c761dd602c fix: keep FunctionTool subclasses copyable (#4311)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-08 22:32:12 +00:00
hari f3ad9f30bb fix(sessions): restore session history when compaction replacement is cancelled (#4298) 2026-08-09 07:12:57 +09:00
Kazuhiro Sera 347fec1a65 docs: update translated pages 2026-08-09 07:00:02 +09:00
Lucca Boas 39d1529a16 fix(chat-completions): raise on audio output in the streamed chat completions path (#4309) 2026-08-09 06:50:09 +09:00
Kazuhiro Sera f5d20e5e2f docs: improve translation source clarity (#4306) 2026-08-08 22:44:19 +09:00
Kazuhiro Sera 98c363743a Fix falsy optional reference handling (#4305) 2026-08-08 17:55:23 +09:00
Henry Su ef1d202f0f fix(memory): preserve falsey compaction decision hooks (#4299) 2026-08-08 13:35:51 +09:00
Kazuhiro Sera 4da5ddb8eb docs(skills): formalize finding-derived complexity resets 2026-08-08 13:08:23 +09:00
Kazuhiro Sera fd4db5609c chore: strengthen maintainer review probes and practical-impact gating 2026-08-08 12:07:09 +09:00
Henry Su f30f7baca0 fix(sandbox): keep source entry metadata when lazily loading a skill (#4294) 2026-08-08 09:59:45 +09:00
Abhay Singh d3830f754a fix(realtime): compute G.711 audio length for typed and mapping format spellings (#4292) 2026-08-08 09:08:48 +09:00
Henry Su 5f5c773828 fix(models): convert input_file items that reference a file_id on the Chat Completions path (#4295) 2026-08-08 09:03:19 +09:00
Henry Su d9a384bdc4 fix(sandbox): restore archived file modes when extracting a workspace tar (#4287) 2026-08-08 09:01:46 +09:00
Kazuhiro Sera 9e2770c550 chore: improve release and review workflows 2026-08-08 08:45:10 +09:00
Kazuhiro Sera 2221313b72 feat: add implementation kickoff skill 2026-08-08 08:41:03 +09:00
Kazuhiro Sera ae84ca1132 fix: suppress pydantic serializer warnings (#4291) 2026-08-08 08:28:21 +09:00
Kazuhiro Sera 9c6cadf820 fix: count requests when providers omit usage (#4290)
Co-authored-by: abhay-codes07 <abhaysingh0293@gmail.com>
2026-08-08 07:42:57 +09:00
Kazuhiro Sera 54ab78ec1d fix: normalize optional fields in tool invocation identity (#4289) 2026-08-07 22:38:36 +00:00
Henry Su fb3a2482ea fix(sessions): strip placeholder item IDs before Conversations persistence (#4288) 2026-08-07 22:11:55 +00:00
Henry Su 4f184aad9d fix(realtime): preserve falsey custom models (#4286) 2026-08-07 22:10:49 +00:00
Henry Su ed7fd85ed9 fix(realtime): apply tool call item updates to session history (#4284) 2026-08-07 22:09:32 +00:00
Kazuhiro Sera 237716cfb3 perf: add impact-aware review test target 2026-08-08 00:25:16 +09:00
Kazuhiro Sera 5a249592ef feat: use GPT-5.6 Luna as the default model (#4282) 2026-08-07 22:10:26 +09:00
Aditya Singh b8aed66015 fix(sessions): preserve DaprSession created_at across writes (#4213) 2026-08-07 12:18:27 +00:00
Kazuhiro Sera 4720150fde fix: bind tool approvals to concrete invocations (#4257) 2026-08-07 11:59:01 +00:00
Kazuhiro Sera 0c60a196af feat(models): preserve raw usage payloads (#4279) 2026-08-07 19:24:39 +09:00
Shaurya Singh a3f2bb8eea fix(sessions): apply the reasoning item id policy to stored session history (#4278) 2026-08-07 10:11:35 +00:00
Kazuhiro Sera 47ff39bf79 fix(sandbox): retain program items in memory rollouts (#4276)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-07 17:23:46 +09:00
Shaurya Singh 50749a686b fix(models): strip placeholder item IDs without provider_data on the Responses path (#4266) 2026-08-07 16:32:23 +09:00
Henry Su 105aeef401 fix(streaming): announce the chat completions assistant message once (#4275) 2026-08-07 16:27:40 +09:00
Kazuhiro Sera 8b810bc4bd fix: scope hosted MCP approvals to server identity (#4256) 2026-08-07 02:29:37 +00:00
Aditya Singh ece7b0e586 fix(voice): end a streamed session that produced no audio (#4259) 2026-08-07 11:13:44 +09:00
Ojas Sharma 00c9d26915 Tool approval is not honored on resume when Runner.run is given a context (#4245) 2026-08-07 10:10:16 +09:00
Lucca Boas 191722fd81 fix(models): keep url citations on the streamed chat completions path (#4252) 2026-08-07 00:46:41 +00:00
Henry Su 20dd205d5d fix(sessions): preserve required program item ids for OpenAI conversations (#4253) 2026-08-07 09:46:37 +09:00
Henry Su b42ead5c11 fix(run): work on copies of the lists a resumed run adopts from RunState (#4251) 2026-08-07 00:38:31 +00:00
Coleby Pearson 2ec632fef3 docs: voice quickstart also needs sounddevice (#4250) 2026-08-07 00:32:26 +00:00
Henry Su 0342746cf8 fix(extensions): forward prompt_cache_retention on the any-llm Responses path (#4248) 2026-08-07 00:29:26 +00:00
Kazuhiro Sera 6e0dffb387 ci: upgrade actions/cache to the latest 2026-08-07 09:10:04 +09:00
Kazuhiro Sera 53a461f54e ci: cache mypy results across typecheck runs 2026-08-07 08:57:52 +09:00
Kazuhiro Sera 070b6e1543 perf: accelerate developer checks (#4258) 2026-08-06 23:06:57 +00:00
Henry Su f3b6c61785 fix(sandbox): keep move_to when coercing apply_patch operation mappings (#4242) 2026-08-06 17:02:04 +09:00
Henry Su 19f6bde526 fix(models): propagate the OpenAI request ID on the Chat Completions path (#4243) 2026-08-06 16:05:33 +09:00
Kazuhiro Sera 4a1773f405 fix: make session mutations atomic (#4212) 2026-08-06 14:49:57 +09:00
Kazuhiro Sera b47a0e4be7 docs: update translated pages 2026-08-06 10:18:59 +09:00
Kazuhiro Sera 065feebfd8 docs: document API Fast mode 2026-08-06 10:02:37 +09:00
Aditya Singh 36d50b014a fix: keep tool guardrail results when a resumed run interrupts again (#4239) 2026-08-06 00:27:36 +00:00
Abhay Singh 37b7a035b1 fix: keep empty turns in the nested conversation history (#4230) 2026-08-06 00:23:26 +00:00
Aditya Singh 0f4acc1cb9 fix(tracing): catch only the context reset on generator close (#4232) 2026-08-06 00:18:16 +00:00
Aditya Singh aad96a7c11 fix(tools): annotate the bare tool guardrail decorator overloads (#4238) 2026-08-06 00:11:53 +00:00
Aditya Singh 7e5b3e076e fix: copy raw_responses when building a RunState (#4237) 2026-08-06 00:11:09 +00:00
Aditya Singh 141f59949e fix(tracing): release the span scope when a generator is closed (#4233) 2026-08-06 00:04:20 +00:00
Aditya Singh 2de0178d61 fix(streaming): assemble chat completions content parts in content index order (#4236) 2026-08-06 09:02:36 +09:00
Aditya Singh c9153d2554 fix(mcp): build the active server list without re-consuming the iterable (#4235) 2026-08-06 09:01:56 +09:00
Aditya Singh 7c6ff9a3f8 fix(extensions): surface content filter refusals on the any_llm chat path (#4234) 2026-08-06 09:01:04 +09:00
Aditya Singh cce949a3fc fix(memory): enforce closed state on SQLite session empty add_items (#4231) 2026-08-05 23:51:45 +00:00
Abhay Singh 810620b122 fix(sandbox): stop splitting Cloudflare SSE events at chunk boundaries (#4215) 2026-08-05 23:47:08 +00:00
Kazuhiro Sera 0068ce4329 fix: redact JSON validation errors (#4211) 2026-08-06 08:26:45 +09:00
Kazuhiro Sera f65a89b015 test: stabilize Blaxel prune patching on Python 3.10 2026-08-06 08:24:56 +09:00
Aditya Singh 5d6885e0cb fix(tracing): release the trace scope when a generator is closed (#4221) 2026-08-05 23:24:22 +00:00
Aditya Singh b6787a3f0f fix(mcp): keep MCP error content when structured output is enabled (#4224) 2026-08-05 23:21:09 +00:00
Abhay Singh 005a752dfc fix(voice): keep the word separator when flushing streamed sentences (#4227) 2026-08-06 08:09:40 +09:00
Aditya Singh eca1223440 fix(models): keep url citations when converting chat completions output (#4222) 2026-08-06 08:06:50 +09:00
Aditya Singh 55b9ea3785 fix(run): publish completed streamed guardrail results on failure paths (#4223) 2026-08-05 23:01:07 +00:00
Aditya Singh a8ac730e66 fix(voice): forward TTSModelSettings.speed to the OpenAI speech API (#4220) 2026-08-05 22:53:43 +00:00
Aditya Singh 5c7fdd53a7 fix(extensions): resolve the any-llm default model at call time (#4219) 2026-08-05 22:53:23 +00:00
dfedoryshchev dac872ebbc docs: correct nonexistent guardrail type names (#4218) 2026-08-05 22:52:46 +00:00
Abhay Singh c358dbabe7 fix(sandbox): encode Blaxel terminal WebSocket URL query values (#4217) 2026-08-05 22:52:01 +00:00
Kazuhiro Sera c48dd4c1aa feat: pass run context to custom sessions (#4209) 2026-08-05 22:27:54 +09:00
Aditya Singh afd1a26249 fix(memory): release SQLite engine config entries when engines are collected (#4210) 2026-08-05 13:09:43 +00:00
Kazuhiro Sera f6a32fee4e fix: harden tool output trimming contracts (#4204) 2026-08-05 11:02:08 +00:00
Kazuhiro Sera 6af30c57e2 fix: prevent queue consumer deadlocks (#4201)
Co-authored-by: abhay-codes07 <abhaysingh0293@gmail.com>
2026-08-05 19:52:01 +09:00
Kazuhiro Sera b9f817aba8 test: fix failuring tests 2026-08-05 19:05:21 +09:00
Kazuhiro Sera 8f7e6d763c feat: support MCP Python SDK v1 and v2 (#4106) 2026-08-05 18:52:13 +09:00
Kazuhiro Sera afa911f82a perf: add source-only typecheck target (#4200) 2026-08-05 18:35:57 +09:00
Kazuhiro Sera 3088d5f402 perf: run serial tests alongside xdist (#4197) 2026-08-05 18:33:34 +09:00
Sam Xie 8be468f35b docs(tracing): fix custom span output example (#4196) 2026-08-05 14:19:06 +09:00
Pranav Mishra d270dac3a5 fix(sandbox): cancel sibling env resolvers when one fails (#4195) 2026-08-05 04:38:20 +00:00
Kazuhiro Sera 08c1c9f365 docs: update config and guardrails pages 2026-08-05 12:49:19 +09:00
github-actions[bot] 92aa1b9053 Bump version to 0.19.4 (#4194) 2026-08-05 11:55:50 +09:00
Kazuhiro Sera b4386278aa docs: fix tool duplication in sandbox examples 2026-08-05 11:54:03 +09:00
Kazuhiro Sera e064ab6839 test: make parent cancellation tests deterministic (#4193)
Co-authored-by: Shaurya Singh <sshaurya914@gmail.com>
2026-08-05 11:29:51 +09:00
Kazuhiro Sera 7379e75d4e fix(memory): reject reused branch IDs when creating a branch (#4186)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-05 11:13:37 +09:00
Kazuhiro Sera bf00f45f32 test: make async tool and guardrail tests deterministic (#4192)
Co-authored-by: Shaurya Singh <sshaurya914@gmail.com>
2026-08-05 01:48:15 +00:00
Kazuhiro Sera 107260f992 fix(tracing): mark non-streaming agent span failures (#4191)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-05 10:47:53 +09:00
Kazuhiro Sera 81de0d0afe fix(realtime): clean up failed connection attempts (#4189)
Co-authored-by: Shaurya Singh <sshaurya914@gmail.com>
2026-08-05 01:25:58 +00:00
Kazuhiro Sera 36829b62e3 fix(models): surface non-streaming content-filter refusals (#4188)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-05 01:22:57 +00:00
Kazuhiro Sera 4c9e50757b fix(run): cancel sibling work after concurrent failures (#4185)
Co-authored-by: Pranav Mishra <mishrapranav82@gmail.com>
2026-08-05 01:15:20 +00:00
Kazuhiro Sera 6972c14f25 fix(memory): skip conversation creation on empty add_items (#4190)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-05 01:06:09 +00:00
Kazuhiro Sera afbadd085e test: make parallel guardrail overlap deterministic (#4187)
Co-authored-by: LeSingh1 <sshaurya914@gmail.com>
2026-08-05 10:01:31 +09:00
Kazuhiro Sera 6e4cec5cee fix(approvals): honor resolved status before policy checks (#4183)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-05 09:56:54 +09:00
Kazuhiro Sera 69e26269f5 fix: defer non-stream session saves until output guardrails (#4184)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-05 09:46:02 +09:00
Kazuhiro Sera 1ebcfd4f01 fix: redact invalid tool argument errors (#4182)
Co-authored-by: Illia Oleksiuk <ilya.oleksiuk@gmail.com>
2026-08-05 00:28:04 +00:00
Kazuhiro Sera 154e44146d fix(memory): preserve repeated history provenance (#4181)
Co-authored-by: Henry Su <henrysu4707@gmail.com>
2026-08-05 00:26:54 +00:00
Kazuhiro Sera fb24afcff1 fix(run): preserve completed tool guardrail results (#4180)
Co-authored-by: LHMQ878 <LHMQ878@users.noreply.github.com>
2026-08-05 09:21:55 +09:00
Kazuhiro Sera eb02f60ce0 fix(sandbox): single-flight cached dependency factories (#4178)
Co-authored-by: cosin2077 <bermasunita6@gmail.com>
2026-08-05 08:29:50 +09:00
Kazuhiro Sera d3463f2a15 fix(extensions): preserve streamed provider thinking blocks (#4177)
Co-authored-by: abhay-codes07 <abhaysingh0293@gmail.com>
2026-08-04 23:01:59 +00:00
Kevin ad976ad811 fix sandbox token output budgets (#3934) 2026-08-05 07:57:06 +09:00
Rakshit Sharma 0f5d3c371a fix(mcp): derive streamable HTTP retry backoff from backoffs taken (#4174) 2026-08-05 07:32:05 +09:00
Kazuhiro Sera 046f82bb83 fix(memory): enforce closed state in MongoDBSession (#4176)
Co-authored-by: Chinmay V <203952148+chinmayv095@users.noreply.github.com>
2026-08-04 22:28:36 +00:00
Kazuhiro Sera 19e364c173 perf: speed up the test suite (#4171) 2026-08-04 08:35:55 +00:00
Kazuhiro Sera 26e461f055 docs: update translated pages 2026-08-04 16:49:48 +09:00
Kazuhiro Sera 3de564c898 docs: updates for v0.19.3 2026-08-04 16:35:32 +09:00
github-actions[bot] 39814aea09 Bump version to 0.19.3 (#4169)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-08-04 16:13:04 +09:00
Kazuhiro Sera d89fddee77 fix(voice): finish STT event handling after listener errors (#4170) 2026-08-04 06:51:53 +00:00
wesleyzhangwq 08fa43c099 docs: fix spelling in GPT-5 example (#4168) 2026-08-04 05:31:05 +00:00
Kazuhiro Sera 27c136cec8 fix(realtime): scope delayed audio guardrail interruption (#4135) 2026-08-04 13:57:58 +09:00
Shaurya Singh 72e7c6e549 fix(tracing): name streamed task spans after the run's own workflow (#4167) 2026-08-04 13:57:16 +09:00
Shaurya Singh 7b7587425a fix(memory): roll back a failed SQLiteSession insert (#4163) 2026-08-04 03:07:02 +00:00
Shaurya Singh 945332dabc fix(sessions): persist the max-turns handler output to the session (#4160) 2026-08-04 02:32:55 +00:00
Kazuhiro Sera 648c25284b fix: reconcile resumed tool name collisions (#4145) 2026-08-04 02:15:59 +00:00
Kazuhiro Sera 8ca63f37d9 fix(extensions): preserve thinking blocks for replay (#4157)
Co-authored-by: Abhay Singh <abhaysingh0293@gmail.com>
2026-08-04 00:13:13 +00:00
LHMQ878 04aaa50c8f fix: keep committed tool session records when a streamed output guardrail trips (#4148) 2026-08-03 22:28:37 +00:00
LHMQ878 8d6ca279ec fix(models): let an in-flight provider stream close finish after cancellation (#4156) 2026-08-04 07:28:04 +09:00
Pranav Mishra 6836d1dfd7 fix(models): record model-call failures on the provider's own span (#4143) 2026-08-03 22:26:38 +00:00
Omid Saffari 052f8387a6 fix(run): honor falsey handoff input filters (#4153) 2026-08-03 22:23:59 +00:00
Kazuhiro Sera c546ca1209 fix: preserve tool call and output order when deduplicating inputs (#4147) 2026-08-03 10:25:46 +00:00
Henry Su 9af785b110 fix(run): stop emitting handoff calls as streamed tool_called events (#4146) 2026-08-03 17:36:12 +09:00
Henry Su bdc294fcd4 fix(run): keep input item order when collapsing duplicates (#4140) 2026-08-03 06:23:54 +00:00
Kazuhiro Sera 306ac19745 fix(sandbox): harden default snapshot path resolution (#4141)
Co-authored-by: Gautam Sharma <148205237+GautamSharma99@users.noreply.github.com>
2026-08-03 14:59:55 +09:00
Kazuhiro Sera 9f4292e5d8 fix: resolve agent tool name collisions consistently (#4137) 2026-08-03 04:54:49 +00:00
Kazuhiro Sera 7de6ccf05d fix(schema): normalize typeless strict object schemas (#4139)
Co-authored-by: Rajarshi Datta <138959719+rajarshidattapy@users.noreply.github.com>
2026-08-03 04:36:25 +00:00
Henry Su 0c3844a220 fix(extensions): send AnyLLM Responses reasoning as a mapping (#4138) 2026-08-03 04:34:14 +00:00
Kazuhiro Sera 2bd71302d9 chore: add implementation-final-review skill to the repo 2026-08-03 13:14:58 +09:00
Henry Su b2012870c3 fix(extensions): release AnyLLM provider streams and preserve completed runs (#4133) 2026-08-03 03:56:17 +00:00
Kazuhiro Sera e943deda36 fix(voice): clean up tasks when streams close early (#4131)
Co-authored-by: Gautam Sharma <148205237+GautamSharma99@users.noreply.github.com>
2026-08-03 01:18:11 +00:00
Kazuhiro Sera 686d041bac fix(streaming): synchronize after-turn cancellation with event consumption (#4130) 2026-08-03 00:35:57 +00:00
Kazuhiro Sera d6f82c0067 fix(realtime): apply output guardrails to text deltas (#4124) 2026-08-03 09:11:21 +09:00
Kazuhiro Sera 9d894a9032 fix: preserve approved tool output on streamed resume (#4126) 2026-08-02 23:24:24 +00:00
Henry Su bfcfcfc9d8 fix(realtime): clamp interrupt truncation to received audio (#4122) 2026-08-02 22:34:02 +00:00
Henry Su c1d40890c6 fix(memory): avoid creating remote conversation on uninitialized clear_session (#4111) 2026-08-03 07:18:42 +09:00
Kazuhiro Sera 6e2095d918 docs: update agent references on the client-side validation 2026-08-03 06:21:12 +09:00
LHMQ878 718acdeb9e fix(trimmer): keep definition names and instance data in trimmed schemas (#4110) 2026-08-02 21:14:41 +00:00
chinmayv095 c06e1e3b09 fix(memory): enforce closed state in AsyncSQLiteSession (#4109) 2026-08-02 21:13:26 +00:00
Henry Su fc084ae29c fix(run): report tool guardrail results for streamed runs (#4097) 2026-08-02 06:02:35 +00:00
Kazuhiro Sera 7cc834a13c test: stabilize tracing atexit timeout coverage (#4104) 2026-08-02 04:13:14 +00:00
Kazuhiro Sera 6fca125770 docs: clarify repo skill resolution in AGENTS.md 2026-08-02 13:11:39 +09:00
Kazuhiro Sera 0585084552 chore(ci): preserve cache pruning with setup-uv v9 (#4103) 2026-08-02 12:58:40 +09:00
Henry Su 4808a9adf9 fix(run): replace closed default loop in run_sync (#4102) 2026-08-02 12:22:12 +09:00
Kazuhiro Sera b5944a42bd docs: improve example code details 2026-08-02 12:06:22 +09:00
Kazuhiro Sera c94cebd37d docs: update translated pages 2026-08-02 11:50:20 +09:00
Kazuhiro Sera 0db81169cc fix: define explicit zero-value contracts (#4101) 2026-08-02 02:16:52 +00:00
Henry Su 98df4ea63d fix(sandbox): remove apply_patch move source as the bound user (#4100) 2026-08-02 01:01:49 +00:00
Kazuhiro Sera a134f3a298 fix(mcp): auto-paginate tool and prompt listings (#4094)
Co-authored-by: Akshay Sharma <68906315+akshay183@users.noreply.github.com>
2026-08-02 00:06:15 +00:00
Kaif Kohari 29e99da251 fix(chatcmpl): clear pending thinking blocks when flushing an assistant message (#4089) 2026-08-02 08:31:41 +09:00
Kaif Kohari 855255a662 fix(run): report output guardrail results when a tripwire aborts the run (#4090) 2026-08-02 08:30:36 +09:00
Sohail(Neel) Sarkar d5f51d3c7f fix(agent-tools): honor falsey custom output extractors (#4088) 2026-08-01 23:27:20 +00:00
Kazuhiro Sera 87425fae1c chore: update issue templates 2026-08-01 19:12:31 +09:00
dependabot[bot] bc3e93c3ed chore(deps): bump actions/stale from 10.3.0 to 11.0.0 (#4084) 2026-08-01 17:08:17 +09:00
dependabot[bot] 118b2ac8ee chore(deps): bump actions/setup-python from 6.3.0 to 7.0.0 (#4083) 2026-08-01 17:08:07 +09:00
dependabot[bot] 0fe74a1194 chore(deps): bump pypa/gh-action-pypi-publish from 1.14.0 to 1.14.2 (#4081) 2026-08-01 17:07:37 +09:00
dependabot[bot] 3c7b56d838 chore(deps): bump actions/checkout from 7.0.0 to 7.0.1 (#4080) 2026-08-01 17:07:23 +09:00
Kazuhiro Sera 21c88f582c docs: improve the consistency of docs 2026-08-01 11:37:59 +09:00
Kazuhiro Sera c9b5d1ba3f docs: update translated pages 2026-08-01 10:38:39 +09:00
Kazuhiro Sera 07cdff5cde docs: updates for v0.19.2 2026-08-01 10:20:09 +09:00
github-actions[bot] 16bc1f11e5 Bump version to 0.19.2 (#4046)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-08-01 10:05:04 +09:00
Kazuhiro Sera a83930c2cb chore: run verification outside the Codex sandbox 2026-08-01 09:13:18 +09:00
Kazuhiro Sera 49821c172d fix: redact direct MCP cleanup transport errors (#4078) 2026-08-01 00:03:52 +00:00
Kazuhiro Sera 18f658aaf4 docs: stabilize healthcare_support example 2026-08-01 07:58:10 +09:00
Kazuhiro Sera ff0e7866f1 fix: preserve completed LiteLLM streams on cleanup failure (#4077) 2026-07-31 22:08:44 +00:00
dfedoryshchev c987929aef docs: correct the documented Chat Completions store default (#4074) 2026-08-01 06:53:45 +09:00
Henry Su 78725420e8 fix(run): report input guardrail results when a tripwire aborts the run (#4071) 2026-08-01 06:47:20 +09:00
Kazuhiro Sera c3f1781d56 fix: redact MCP prompt and resource transport errors (#4067) 2026-07-31 13:56:36 +00:00
Satvik Sawhney 5e2c00b726 fix(models): close the LiteLLM provider stream on exit (#4066) 2026-07-31 22:12:04 +09:00
LHMQ878 a017105509 fix(modal): reject ephemeral paths during tar hydration (#4045) 2026-07-31 18:37:59 +09:00
Gautam Sharma 2c6be96b5c fix(tracing): use monotonic export deadlines (#4063) 2026-07-31 09:36:17 +00:00
Gautam Sharma 2a69638f0a fix(realtime): preserve raw server event payloads (#4062) 2026-07-31 09:31:51 +00:00
Gautam Sharma b5465c705e fix(voice): block audio dispatcher while idle (#4061) 2026-07-31 09:31:08 +00:00
Kazuhiro Sera 000a96b602 fix: nested MCP cleanup error redaction (#4049) 2026-07-31 15:45:13 +09:00
Ali Adnan 0ffa36840c fix(voice): break out of audio dispatch loop when a stream task signals session_ended (#4044) 2026-07-31 14:23:33 +09:00
Kazuhiro Sera 27fc1f4407 fix(sandbox): preserve tagged EnvValue subclasses (#4039)
Co-authored-by: maplexu <maplexu2010@gmail.com>
2026-07-31 14:15:37 +09:00
Satvik Sawhney c510261cad fix(memory): enforce closed state in Redis and Dapr sessions (#4035) 2026-07-31 14:14:52 +09:00
Henry Su 2cec48924b fix(voice): propagate iterator cancellation (#4040) 2026-07-31 02:01:27 +00:00
Kazuhiro Sera 1058e84258 feat(tool): expose original callable through wrapped (#4038) 2026-07-31 10:14:52 +09:00
Kaif Kohari df0b4a2eb7 fix(extensions): keep tool parameters named like schema keywords in trimmer (#4036) 2026-07-31 00:07:12 +00:00
chinmayv095 974733eff5 fix(memory): count valid Redis and Dapr session items for positive limits (#4033) 2026-07-31 00:56:42 +09:00
King Star d3ea084b0b fix(mcp): clean failed servers before reconnecting (#3939) 2026-07-30 15:40:00 +00:00
chinmayv095 f2bd57bf0d fix(memory): count valid AdvancedSQLiteSession items for positive limits (#4031) 2026-07-31 00:39:21 +09:00
chinmayv095 80915c8c61 fix(memory): count valid SQLAlchemy and MongoDB session items for positive limits (#4032) 2026-07-31 00:39:07 +09:00
Kazuhiro Sera e76f8358b1 chore: enable Ruff TRY201 2026-07-31 00:14:28 +09:00
Ali Adnan d188d153dc fix: consolidate same-exception re-raises into bare raise across all modules (#4023) 2026-07-31 00:05:02 +09:00
Kazuhiro Sera bdc1989993 fix(mcp): redact URL credentials from tracing and tool metadata (#4020)
Co-authored-by: rajashidattapy <rayan05rio@gmail.com>
2026-07-30 23:33:38 +09:00
Kazuhiro Sera 9e1564e006 fix(mcp): redact URL credentials from SDK errors (#4015)
Co-authored-by: Dima Osipa <1094629+dimaosipa@users.noreply.github.com>
2026-07-30 13:55:09 +00:00
Kazuhiro Sera 992abf763d docs: update translated pages 2026-07-30 08:12:09 +09:00
Kazuhiro Sera 42963a2559 docs: updates for v0.19.1 release 2026-07-30 07:57:01 +09:00
github-actions[bot] ddc39d0e54 Release 0.19.1 (#4010) 2026-07-29 16:42:45 +09:00
Kazuhiro Sera 1dddc0d1e2 feat(sandbox): support native host paths in path grants (#4009) 2026-07-29 06:58:53 +00:00
Henry Su e75cdd2e2c fix: cancel sibling enablement checks on failure (#4005) 2026-07-29 14:42:30 +09:00
Henry Su 3f45d9e56f fix: honor falsey input builders in agent-tools (#4007) 2026-07-29 05:10:11 +00:00
Henry Su 71aa44e4b9 Preserve zero Blobfuse attribute cache timeout (#4006) 2026-07-29 14:02:37 +09:00
Henry Su 3142f3ace3 fix(run): cancel streamed models when input guardrails fail (#4004) 2026-07-29 12:53:57 +09:00
Henry Su e8311b4588 fix(run): count streamed retries when terminal usage is missing (#4002) 2026-07-29 08:17:36 +09:00
Henry Su 35c8805415 fix(memory): count valid SQLite session items for positive limits (#4001) 2026-07-29 08:10:12 +09:00
Kazuhiro Sera bb3d64e74d docs: update translated pages 2026-07-28 15:33:07 +09:00
Kazuhiro Sera f1becff0b8 docs: update missing info 2026-07-28 15:33:07 +09:00
Henry Su ba58983a17 fix(models): route falsey mapped providers (#3996) 2026-07-28 15:17:07 +09:00
Henry Su 65db9a7eac fix(run): allow empty streamed model input (#3995) 2026-07-28 06:14:12 +00:00
Kazuhiro Sera a6cb922442 fix: redact Blaxel unmount paths in logging (#3993) 2026-07-28 06:04:21 +00:00
Kazuhiro Sera 421deb7506 docs: update translated pages 2026-07-28 08:10:53 +09:00
Kazuhiro Sera 88bfb18c2f fix: redact Realtime audio format diagnostics (#3992) 2026-07-28 07:57:35 +09:00
Kazuhiro Sera a6ce52d255 docs: use decorators module throughout examples (#3946) 2026-07-28 07:57:19 +09:00
Gunjan Jaswal 5804bd039f fix(run): cancel the parallel input-guardrail task when the model turn fails (#3982) 2026-07-28 07:57:04 +09:00
Henry Su b2f0344e92 fix(tracing): enforce max_batch_size during force_flush and shutdown in BatchTraceProcessor (#3985) 2026-07-28 07:56:49 +09:00
ShawnSiao e42482470a docs: repair tracing integration links (#3986) 2026-07-28 07:56:31 +09:00
ShawnSiao 0eb4780dbb docs: fix a URL in sandbox example code (#3987) 2026-07-28 07:56:17 +09:00
Joshua Nwachinemere 73a2cd56d3 fix: retry pre-response WebSocket server errors (#3991) 2026-07-28 07:55:59 +09:00
Kazuhiro Sera ac1206294c docs: updates for 0.19.0 (#3872) 2026-07-28 07:51:20 +09:00
github-actions[bot] a2d82707d9 Release 0.19.0 (#3874) 2026-07-27 12:50:31 +09:00
Kazuhiro Sera da82ee786a fix: retry pre-response WebSocket overload errors (#3978) 2026-07-27 00:56:07 +00:00
Ali Adnan fe41cc39e0 fix: use last_agent property in RunResultStreaming._create_error_details (#3967) 2026-07-27 00:19:44 +00:00
TheSaiEaranti a335b32024 fix: compare inspect sentinels by identity in function_schema (#3961) 2026-07-27 09:18:40 +09:00
Kazuhiro Sera 045b9cac71 chore: update FastAPI test dependency (#3974) 2026-07-27 07:27:14 +09:00
Kazuhiro Sera 3da2465a44 test: add unit tests covering #3965 2026-07-27 06:58:20 +09:00
Ali Adnan da14e70b3a fix: use last_agent property in pretty_print_run_result_streaming to avoid None crash (#3965) 2026-07-27 06:57:55 +09:00
Kazuhiro Sera f663a06aea chore: tighten implementation strategy review guidance 2026-07-26 19:02:25 +09:00
Kazuhiro Sera 117bd1bb9a fix: preserve callable function tool compatibility (#3959) 2026-07-26 09:50:10 +00:00
Kazuhiro Sera 5aff70faeb fix: reuse the verbose stdout logging handler (#3957) 2026-07-25 23:51:30 +00:00
TheSaiEaranti 99e88c14db fix: preserve *args/**kwargs docstring descriptions in tool schemas (#3956) 2026-07-26 08:29:45 +09:00
Kazuhiro Sera c55c99b61e chore: update AGENTS.md and code change/reiew skill details 2026-07-26 08:20:45 +09:00
Kazuhiro Sera 6eb779d939 fix: support async callable objects as function tools (#3949) 2026-07-26 08:06:43 +09:00
TheSaiEaranti c1b423749e fix: include device nodes when parsing ls output (#3951) 2026-07-25 05:20:22 +00:00
Kazuhiro Sera c549a825ae chore: update final-release-review skill details 2026-07-25 13:31:27 +09:00
Kazuhiro Sera f45d6e2610 chore: update final-release-review skill details 2026-07-25 12:12:33 +09:00
Kazuhiro Sera d9d623098e fix: reject unsupported streamed STT audio dtypes (#3950) 2026-07-25 10:20:51 +09:00
Kazuhiro Sera 14ad7e1c62 fix: redact Realtime and RunState diagnostics (#3948) 2026-07-25 09:47:04 +09:00
Kazuhiro Sera e16ba7ea8a chore: update AGENTS.md and code change/reiew skill details 2026-07-25 09:31:28 +09:00
Kazuhiro Sera fc0c8d33f2 fix: isolate unit tests from ambient proxy settings (#3945) 2026-07-25 08:05:35 +09:00
Kazuhiro Sera a6d577c99d fix: await async callable-object callbacks (#3944) 2026-07-25 07:26:53 +09:00
Henry Su 5a74e5104d fix: await async callable-object dynamic instructions (#3942) 2026-07-24 22:01:19 +00:00
Kazuhiro Sera 59763339cb fix: harden sensitive runtime logging (#3938) 2026-07-24 15:36:48 +00:00
Cheemi 5c7f145a0d docs: fix streamed audio API rendering (#3940) 2026-07-24 10:29:33 +00:00
Kazuhiro Sera 5d620569a7 feat: add packaged live integration and provider regression coverage (#3936) 2026-07-24 15:39:28 +09:00
Kazuhiro Sera 658bfc488b fix: encode structured programmatic tool errors as JSON (#3932) 2026-07-24 03:09:31 +00:00
Gunjan Jaswal aa3ac378b9 fix(chatcmpl): surface content-filter refusals when buffering streamed tool calls (#3897) 2026-07-24 10:11:50 +09:00
Kazuhiro Sera 4c251ff779 fix: improve provider compatibility and preserve session history on retries (#3931) 2026-07-24 01:09:59 +00:00
Kazuhiro Sera 80f9fa5d9c fix: improve AnyLLM and LiteLLM provider compatibility (#3930) 2026-07-24 08:54:17 +09:00
Kazuhiro Sera f78df37ee9 feat: consistently accept typed objects and dictionaries for SDK configuration (#3917) 2026-07-24 08:04:45 +09:00
Dima Osipa cece04ce56 fix(litellm): send logprobs=True when top_logprobs is set (#3929) 2026-07-23 21:39:28 +00:00
Henry Su 9684cef97b fix(strict_schema): reject empty additionalProperties mappings (#3927) 2026-07-24 06:19:17 +09:00
Kazuhiro Sera 509b414711 feat: add tool decorator alias (#3920)
Tests / tests-windows (push) Has been cancelled
Tests / build-docs (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / typecheck (push) Has been cancelled
Tests / tests (3.10) (push) Has been cancelled
Tests / tests (3.11) (push) Has been cancelled
Tests / tests (3.12) (push) Has been cancelled
Tests / tests (3.13) (push) Has been cancelled
Tests / tests (3.14) (push) Has been cancelled
2026-07-23 18:04:15 +09:00
Henry Su 34ab935367 fix(mcp): cleanup cancelled servers during connect failure (#3919) 2026-07-23 07:50:20 +00:00
Dima Osipa 9f06efe9df fix(logging): redact MCP tool-call and formatter errors from logs (#3918) 2026-07-23 07:49:06 +00:00
Fayez Bast 0530398ef2 fix(voice): encode streamed float32 audio as PCM16 (#3916) 2026-07-23 14:30:15 +09:00
Cheemi 1e8d506a32 docs: fix API cross-reference targets (#3913)
Deploy docs / deploy_docs (push) Has been cancelled
Tests / lint (push) Has been cancelled
Tests / typecheck (push) Has been cancelled
Tests / tests (3.10) (push) Has been cancelled
Tests / tests (3.11) (push) Has been cancelled
Tests / tests (3.12) (push) Has been cancelled
Tests / tests (3.13) (push) Has been cancelled
Tests / tests (3.14) (push) Has been cancelled
Tests / tests-windows (push) Has been cancelled
Tests / build-docs (push) Has been cancelled
2026-07-22 14:15:31 +00:00
Kazuhiro Sera 28c41b79b3 fix: pin and verify rclone sandbox installs (#3911) 2026-07-22 23:00:17 +09:00
Dima Osipa 5921667f57 fix: stop leaking raw exceptions in model + tool error logs (#3910) 2026-07-22 11:54:27 +09:00
Kazuhiro Sera c994c2a44b docs: update code snippets 2026-07-22 09:59:22 +09:00
Illia Oleksiuk a251b42dfd fix(realtime): redact raw message payload from conversion-failure log (#3907) 2026-07-22 09:27:11 +09:00
Kazuhiro Sera 104b490b06 feat(sandbox): add Vercel cloud mount strategy (#3899) 2026-07-22 07:52:29 +09:00
Cherav Goyal 95185352c8 fix: preserve dot-prefixed skip paths in Modal tar workspace excludes (#3905) 2026-07-21 21:24:57 +09:00
Cherav Goyal 0d32af6e12 fix: parse ls output with SELinux security-context marker (#3904) 2026-07-21 11:45:05 +00:00
Cheemi 394d4dc9ba docs: fix conversation example imports (#3903)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-21 19:20:09 +09:00
Kazuhiro Sera 2171c77c5b docs: update review policies 2026-07-21 08:02:59 +09:00
Kazuhiro Sera 588c6e32e1 fix: classify lazy skill read probes correctly (#3893) 2026-07-21 07:05:29 +09:00
Henry Su 3384027348 fix(visualization): escape multiline DOT names (#3898) 2026-07-21 07:04:41 +09:00
Umer Ali e78bb09275 docs: fix missing imports in dynamic instructions example (#3895)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-20 22:01:30 +00:00
Kazuhiro Sera 2fa463571e docs: fix runnable Python examples (#3885) 2026-07-20 08:48:24 +09:00
Ali Adnan bc4c5f5006 docs: remove unused Runner import in tracing docs (#3881) 2026-07-19 21:37:41 +00:00
Kazuhiro Sera 173eca4991 fix: improve sandbox example runtime validation (#3880) 2026-07-19 03:04:02 +00:00
Kazuhiro Sera 7b84479829 fix: remove programmatic transcripts from handoff tool filtering (#3879) 2026-07-19 01:29:08 +00:00
Kazuhiro Sera 1b5f59fb2b fix: summarize programmatic transcripts during nested handoffs (#3878) 2026-07-19 10:17:18 +09:00
Kazuhiro Sera fa1001dcf1 fix: recognize typed easy input messages in Chat Completions (#3877) 2026-07-19 09:25:21 +09:00
Illia Oleksiuk 65886fa16d fix(blaxel): shell-quote interpolated FUSE mount options (#3873) 2026-07-17 23:48:46 +00:00
Kazuhiro Sera 15bac19550 fix(handoffs): partition nested history ownership (#3823) 2026-07-18 07:18:57 +09:00
Matt Brockman d003f21c22 chore: update e2b versions (#3871) 2026-07-17 22:17:57 +00:00
Ali Adnan 825f1132ef docs: adjust code snippets (#3869) 2026-07-17 22:10:14 +00:00
Kazuhiro Sera efd63ece6f fix: fail closed on invalid callable approval arguments (#3867) 2026-07-18 07:09:54 +09:00
Kazuhiro Sera 606fbd0d8a docs: clarify sandbox support on Windows (#3866) 2026-07-17 12:39:11 +00:00
Myk b2e9246938 fix: parse Google docstrings whose Args: follows a section body (#3862) 2026-07-17 21:33:12 +09:00
Kazuhiro Sera 965335aba6 feat: add Programmatic Tool Calling support (#3833) 2026-07-17 15:29:53 +09:00
Kazuhiro Sera 95df2c99a7 docs: update pages 2026-07-17 13:39:44 +09:00
Kazuhiro Sera 485bd96e42 docs: update translated pages 2026-07-17 13:01:49 +09:00
Cheemi fff18a0940 docs: document compact tracing hierarchy (#3839) 2026-07-17 12:40:50 +09:00
github-actions[bot] 3e788a46f1 Release 0.18.3 (#3859) 2026-07-17 12:38:27 +09:00
Kazuhiro Sera d02972ef57 fix: prevent stale prepared-item identity reuse (#3858) 2026-07-16 23:10:22 +00:00
Kazuhiro Sera 5313a82996 fix: preserve streamed session input across model retries (#3857) 2026-07-17 07:45:51 +09:00
Muhammad Hamid Raza 5e8d067bc0 docs: make UserContext example runnable in Agents documentation (#3851) 2026-07-17 07:14:33 +09:00
Kazuhiro Sera 697a46c4ba fix: redact non-tool trace error details (#3848) 2026-07-16 18:23:51 +09:00
Kazuhiro Sera e29b023b5b fix(sandbox): avoid redundant E2B workspace root creation (#3845) 2026-07-16 07:50:35 +09:00
DongBo 585053b8bf fix(computer): isolate provider instances across concurrent runs (#3843) 2026-07-16 07:23:38 +09:00
Illia Oleksiuk 8ce548ef4e fix: bound $ref expansion in strict JSON schema conversion (#3838) 2026-07-16 07:22:24 +09:00
Kazuhiro Sera 7369b73c76 feat: make task and turn tracing spans configurable (#3834) 2026-07-15 05:34:04 +09:00
Sohum Trivedi df20e83df3 fix: parse Google docstrings missing the blank line before a section (#3832) 2026-07-14 20:31:49 +00:00
Kazuhiro Sera 4d9677850c chore: remove hoooks for now 2026-07-14 08:11:30 +09:00
Aditya Jethani 68fadc7aba fix: AdvancedSQLiteSession clear_session and pop_item metadata leaks (#3755) 2026-07-13 17:36:45 +09:00
DongBo c3986d6978 fix(memory): serialize OpenAI conversation session initialization (#3821) 2026-07-13 08:26:24 +00:00
Kazuhiro Sera 8221e424db feat(realtime): track response usage in session context (#3812) 2026-07-13 07:42:49 +09:00
DongBo b24d5dd460 fix(handoffs): preserve user messages containing history wrappers (#3815) 2026-07-12 21:56:08 +00:00
Kazuhiro Sera 088075f4f5 chore: enable flake8-async for runtime source (#3813) 2026-07-13 06:48:13 +09:00
Dhruva Das beb3c5102f fix(models): raise UserError for conflicting provider args instead of assert (#3809) 2026-07-12 06:50:12 +00:00
Kazuhiro Sera 08859339ee docs: update translated pages 2026-07-11 10:24:43 +09:00
Kazuhiro Sera 5a9d221f0e docs: stabilize two examples 2026-07-11 10:18:18 +09:00
Kazuhiro Sera bcce208e61 docs: updates for hosted multi-agent support (#3789) 2026-07-11 10:09:01 +09:00
github-actions[bot] 41e79f1594 Release 0.18.2 (#3805)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-11 10:06:15 +09:00
seanturner83 6cf3600d4a fix(litellm): surface content-filter refusals instead of an empty turn (#3769) 2026-07-11 00:52:45 +00:00
Kazuhiro Sera e1f7c12515 fix: make conditional tools and web search examples deterministic 2026-07-11 09:38:53 +09:00
Kazuhiro Sera 75b928ea79 chore: enable RUF100 for runtime source 2026-07-11 09:01:08 +09:00
Kazuhiro Sera 0fce2cb187 chore: remove unused noqa directives (#3804) 2026-07-10 23:59:52 +00:00
Kazuhiro Sera 0a0b7be088 refactor: remove unused Responses include validator 2026-07-11 08:48:51 +09:00
Kazuhiro Sera 23c5276aa7 chore: enable RUF012 for runtime source (#3803) 2026-07-11 08:47:45 +09:00
Kazuhiro Sera aeed7bbcf8 chore: enable G004 logging lint rule (#3802) 2026-07-10 23:25:02 +00:00
Kazuhiro Sera fb8fb10599 docs: update AGENTS.md for git operations 2026-07-11 08:24:13 +09:00
Kazuhiro Sera 93317a87e6 refactor: share retry-after parsing (#3801) 2026-07-11 08:22:00 +09:00
Kazuhiro Sera eb54a61cc7 refactor: share retry error metadata extraction (#3800) 2026-07-11 07:58:13 +09:00
Kazuhiro Sera e354126180 refactor(sandbox): share Runloop rclone setup (#3799) 2026-07-10 22:49:28 +00:00
Kazuhiro Sera bc0dbb3ead refactor(sandbox): share E2B rclone setup (#3798) 2026-07-10 22:32:27 +00:00
Kazuhiro Sera 4c5d630c45 refactor(sandbox): share extension PTY output collection (#3797) 2026-07-11 07:26:41 +09:00
Kazuhiro Sera 77f7f9a7fb refactor(sandbox): share core PTY output collection (#3796) 2026-07-10 22:14:27 +00:00
Kazuhiro Sera fdca3ab389 chore(lint): enable RUF006 for runtime source 2026-07-11 07:05:06 +09:00
Otis Cui 322a3334cb fix: serialize model-backed rollout interruptions as dicts (#3790) 2026-07-11 07:02:05 +09:00
Kazuhiro Sera df2ddd41eb feat: support GPT-5.6 request controls (#3794) 2026-07-11 06:48:04 +09:00
Kazuhiro Sera 0354f482a8 feat: add hosted multi-agent beta support (#3788) 2026-07-10 17:46:43 +09:00
Kazuhiro Sera 2bd03971f9 chore: exempt OpenAI package from dependency cooldown (#3792) 2026-07-10 17:24:10 +09:00
Kazuhiro Sera dd0300bff9 fix(sandbox): own Unix PTY fd close tasks (#3780) 2026-07-10 13:46:31 +09:00
Kazuhiro Sera 32bf99dff3 fix(sandbox): own Docker deferred cleanup tasks (#3779) 2026-07-10 09:58:31 +09:00
Kazuhiro Sera 4e06694db2 fix(realtime): use monotonic playback timing (#3776) 2026-07-10 09:49:41 +09:00
Kazuhiro Sera 152baf182a fix(realtime): enqueue callback errors synchronously (#3777) 2026-07-10 09:49:24 +09:00
Kazuhiro Sera 89653d26f6 fix(sandbox): own Daytona PTY worker tasks (#3778) 2026-07-10 09:48:14 +09:00
github-actions[bot] 1760f73ba4 Release 0.18.1 (#3775)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-10 08:36:38 +09:00
Kazuhiro Sera 28026bcd01 docs: stabilize tool guardrail and web search examples 2026-07-10 08:24:36 +09:00
Kazuhiro Sera 574e2f74cd docs: update translated pages 2026-07-10 08:23:25 +09:00
Kazuhiro Sera 133208381c feat: add GPT-5.6 model defaults and migrate examples (#3774) 2026-07-10 07:58:22 +09:00
Kazuhiro Sera 8724b1b9ba fix: support cache-write usage across OpenAI Python versions (#3773) 2026-07-10 07:14:43 +09:00
Kazuhiro Sera 70c447e14f fix(realtime): make session cleanup deterministic (#3767) 2026-07-09 11:34:36 +00:00
Kazuhiro Sera 6bb69bc2d4 docs: update readme 2026-07-09 20:12:11 +09:00
WinkleMad 532acdf32c fix: send logprobs=True when top_logprobs is set on Chat Completions (#3763) 2026-07-09 07:54:18 +00:00
Anas Khan 1b0c7c2025 fix(models): accumulate streamed logprobs in place to avoid O(n^2) copies (#3762) 2026-07-09 14:38:31 +09:00
Anas Khan 5807751080 fix(sessions): handle list content in AdvancedSQLiteSession browsing helpers (#3761) 2026-07-09 14:18:47 +09:00
Cheemi 158b2f489e docs: clarify agent tool state options (#3754)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-08 23:26:45 +00:00
Kazuhiro Sera 60d3f95219 fix: preserve nested tool state during restoration (#3753)
Co-authored-by: Tao An <1250043+tao-hpu@users.noreply.github.com>
2026-07-08 15:44:19 +09:00
Kazuhiro Sera 163caa3aa8 docs: update README 2026-07-08 08:47:08 +09:00
Kazuhiro Sera 3063306db1 test(items): document refusal None invariant (#3751) 2026-07-08 08:29:40 +09:00
Anas Khan b10a7e9b50 docs(realtime): fix RealtimeRunner context docstring (#3748) 2026-07-07 23:24:44 +00:00
Minh Vu b3f8d8ce54 fix: close Chat Completions streams on early exit (#3689)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-08 08:19:37 +09:00
Kazuhiro Sera 078a28f11e docs: updates for v0.18.0 (#3741) 2026-07-07 15:05:03 +09:00
github-actions[bot] 668fabd6fe Release 0.18.0 (#3742)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-07 15:00:48 +09:00
Kazuhiro Sera 4fde807f1f feat: add Unicode storage option to SQLAlchemySession (#3746) 2026-07-07 03:38:02 +00:00
comp b4606c8f4c fix(visualization): keep fillcolor on handoff nodes by merging style attributes (#3744) 2026-07-07 03:26:38 +00:00
Kazuhiro Sera 909c5c434a feat: update default realtime model to gpt-realtime-2.1 (#3740) 2026-07-07 09:23:20 +09:00
Kazuhiro Sera 4a7dc08d31 docs: make hosted_mcp/on_approval.py stabler 2026-07-07 08:50:54 +09:00
Kazuhiro Sera 94c1f6095f docs: updates for v0.17.8 release (#3737) 2026-07-07 08:37:36 +09:00
github-actions[bot] 7a828f6db2 Release 0.17.8 (#3739) 2026-07-07 08:34:26 +09:00
Kazuhiro Sera 0ffb9e062c docs: update README.md 2026-07-06 15:41:39 +09:00
saime428 5d4a752c24 docs: add sandbox examples to examples index (#3731)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-07-06 06:36:42 +00:00
Kazuhiro Sera 3d07ff1687 feat: add invalid final output recovery handler (#3736) 2026-07-06 15:25:54 +09:00
Kazuhiro Sera 1643dbe617 chore: update runtime-behavior-probe skill to use request_user_input 2026-07-04 17:21:03 +09:00
Kazuhiro Sera e7f835b6c6 chore: update maintainer-review skill 2026-07-04 10:53:34 +09:00
Kazuhiro Sera 849241498f test: improve coverage for stream and sandbox edge cases (#3730) 2026-07-04 10:51:32 +09:00
Kazuhiro Sera ec8fbb875e docs: update translated pages 2026-07-04 09:43:51 +09:00
Kazuhiro Sera 70d01a9eca chore: improve error handling in code-change-verification 2026-07-04 09:04:52 +09:00
Kazuhiro Sera a898061dfa docs: make markdown style consistent across the project 2026-07-04 09:04:30 +09:00
Kazuhiro Sera 106ecb6bfe docs: update PR template and codex-change-verification skill 2026-07-04 08:50:59 +09:00
Kazuhiro Sera fe7d10625d docs: update translated pages 2026-07-04 08:47:25 +09:00
Danny 2afb6e1a24 docs: add DProvenanceKit to external tracing processors list (#3726) 2026-07-02 23:01:09 +00:00
Imran Hassanali 343c06fae1 fix(sandbox): length-frame exec-stdin writes so file I/O works over a… (#3719) 2026-07-03 07:55:52 +09:00
Om Borse 56921c89fc fix(handoffs): enforce strict Pydantic validation when strict_json_schema=True (#3724) 2026-07-02 15:42:43 +09:00
dependabot[bot] f13a3f78f1 chore(deps): bump actions/checkout from 6.0.2 to 7.0.0 (#3717) 2026-07-02 14:10:56 +09:00
dependabot[bot] 8d78731826 chore(deps): bump actions/stale from 10.2.0 to 10.3.0 (#3716) 2026-07-02 14:10:37 +09:00
dependabot[bot] ecfa2e0db6 chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 (#3715) 2026-07-02 14:10:23 +09:00
dependabot[bot] 829e19a7b7 chore(deps): bump astral-sh/setup-uv from 8.1.0 to 8.2.0 (#3714) 2026-07-02 14:10:07 +09:00
Anas Khan 655c805c92 fix(visualization): escape quotes and backslashes in DOT names (#3712) 2026-07-01 00:36:45 +00:00
Kazuhiro Sera 7713342bed chore: update maintainer-review skill 2026-06-30 15:11:31 +09:00
Anas Khan fea17ef542 fix(util): preserve non-dict Mapping values in _to_dump_compatible (#3700) 2026-06-29 02:33:41 +00:00
Kazuhiro Sera 97bc99101a chore: update PR template 2026-06-29 11:25:24 +09:00
Kazuhiro Sera 81fe6ab348 chore: adjust maintainer-review skill details 2026-06-26 08:31:34 +09:00
Minh Vu 8c688b96ac fix: redact realtime validation failure logs (#3687) 2026-06-26 08:28:17 +09:00
Minh Vu ab38ee5d45 fix: propagate realtime session iterator cancellation (#3686) 2026-06-26 08:19:49 +09:00
Kazuhiro Sera 3321967587 chore: adjust maintainer-review skill details 2026-06-26 07:39:09 +09:00
Kazuhiro Sera 92137d222b docs: improve hosted MCP tool example 2026-06-24 14:39:32 +09:00
github-actions[bot] 8a0e34caaa Release 0.17.7 (#3682) 2026-06-24 14:12:59 +09:00
MUHAMMAD SALMAN HUSSAIN 30ff4a5239 fix: recursively extract HTTP errors from nested ExceptionGroups (#3556) 2026-06-24 10:43:47 +09:00
Kazuhiro Sera 16d16ec2be chore: update maintainer-review skill to have two phases 2026-06-24 09:14:51 +09:00
Kazuhiro Sera f706d65124 chore: adjust examples-auto-run skill 2026-06-24 07:27:57 +09:00
Yaohao Wu 2df8adcae7 fix: add hint to duplicated MCP server tool name error message (#3678) 2026-06-23 05:59:52 +00:00
Kazuhiro Sera 7192bc968a docs: add SDK maintainer references and review guidance (#3676) 2026-06-23 11:20:57 +09:00
MAV a9b7b7ef4a fix(realtime): prevent ambiguous multi-agent tool dispatch (#3441) 2026-06-22 22:34:44 +00:00
shize li 28d2a6c838 feat: add buffered Chat Completions tool-call streaming (#3506) 2026-06-22 07:12:44 +00:00
Timothy Asiimwe 3f6324c2d2 docs: fix OpenAI capitalization in auto-generated ref titles (#3646) 2026-06-22 13:54:05 +09:00
ByteWise f115dd6015 fix: reduce sandbox sink buffering and spool timeouts (#3642) 2026-06-22 13:18:11 +09:00
安妮的心动录 e605bd20a2 feat: expose configurable websocket max_size limit (#3645) 2026-06-22 13:14:54 +09:00
Yufeng He 9168c387eb fix: report effective Blaxel timeouts (#3643) 2026-06-22 13:11:20 +09:00
Michal Chudy 593331a0a6 fix: #3620 avoid stale object-id dedupe in server conversation tracker (#3621) 2026-06-22 12:53:47 +09:00
Rohit Rastogi 0bd3d1b865 fix(e2b): wake PTY output collection on process exit (#3610) 2026-06-22 12:48:24 +09:00
vidigoat 7328f790be fix(items): empty list/tuple tool output dropped via all([]) == True (#3554) 2026-06-22 12:15:54 +09:00
Andy Freeland ce4cc62d86 fix: remove dependency on types-requests (#3509) 2026-06-22 12:08:43 +09:00
Matthew.K 50c8b74c50 fix: gate remote-mount edit guidance by mode (#3423) 2026-06-22 02:36:18 +00:00
Matthew.K 4ebf880953 fix: validate ls special permission bits by position (#3420) 2026-06-22 02:30:48 +00:00
Aditya Singh b20cb7f897 fix(approvals): skip needs_approval_checker when status resolved in _collect_runs_by_approval (#3259) 2026-06-22 11:10:56 +09:00
Aditya Singh 9f93077a74 fix(run): cancel sibling guardrail tasks when one raises (#3239) 2026-06-22 11:07:06 +09:00
c 83e8a1b6e2 fix: #3348 make AdvancedSQLiteSession add_items atomic (#3349) 2026-06-22 10:44:55 +09:00
c 13ef230468 fix: #3346 clean up branch-only messages when deleting branches (#3347) 2026-06-22 10:42:00 +09:00
Kazuhiro Sera 869e869c70 docs: translate pages 2026-06-19 15:42:00 +09:00
Kazuhiro Sera 306d426930 docs: changes for #3487 feature addition (#3488) 2026-06-19 15:19:17 +09:00
Kazuhiro Sera f918c6a4f0 docs: #3461 changes for tool_not_found_behavior option (#3462) 2026-06-19 15:05:53 +09:00
github-actions[bot] 510b7de2ce Release 0.17.6 (#3659) 2026-06-19 15:02:31 +09:00
Mohammed Siddiq 09f0ed45e7 fix: enforce the documented strict JSON-compatible contract for #3486 (#3657) 2026-06-18 08:26:47 +00:00
Kazuhiro Sera 7fc489eb0d feat: add SDK-only custom data for tool outputs (#3486) 2026-06-18 12:38:28 +09:00
Kazuhiro Sera a4ba63f704 feat: add pre-approval tool input guardrails (#3487) 2026-06-18 12:38:15 +09:00
Kazuhiro Sera 288455911c fix: suppress handoff whitespace tool-name warnings (#3652) 2026-06-18 10:29:58 +09:00
Kazuhiro Sera 12e268c8e2 chore: improve pr-draft-summary skill trigger 2026-06-18 10:29:35 +09:00
Kazuhiro Sera c359c20647 docs: update translated pages 2026-06-13 15:16:25 +09:00
Kazuhiro Sera 131c3aef76 docs: tweak examples and senteneces 2026-06-13 15:16:05 +09:00
MUHAMMAD SALMAN HUSSAIN 1b0076dda0 fix(docs): fix typo in sandbox guide (#3628) 2026-06-13 08:26:45 +09:00
github-actions[bot] d8068d96a9 Release 0.17.5 (#3619) 2026-06-11 13:10:23 +09:00
MUHAMMAD SALMAN HUSSAIN 5a3028f37c docs: fix subject-verb agreement in agent loop description (#3605) 2026-06-09 23:16:32 +00:00
MUHAMMAD SALMAN HUSSAIN b855c76a56 docs: capitalization fix in tracing docs (#3602) 2026-06-09 23:14:40 +00:00
MUHAMMAD SALMAN HUSSAIN a4d17da8ae docs: improve code snippet in tools documentation (#3599) 2026-06-09 02:55:01 +00:00
MUHAMMAD SALMAN HUSSAIN 6927622331 docs: tweak in usage doc (#3597) 2026-06-09 02:53:25 +00:00
qiyaoq-oai 8eaa4b9872 fix: expose sandbox error retryability (#3581)
Context: https://github.com/temporalio/sdk-python/issues/1548

This pull request adds provider-neutral retryability metadata for
sandbox failures.

- Adds `SandboxError.retryable` with `True`, `False`, and `None` for
unknown classification.
- Propagates retryability through wrapped sandbox errors.
- Emits retryability in sandbox finish events and trace error payloads.
- Classifies clear terminal/transient cases across sandbox providers
without exposing provider internals to integrations.
- Adds focused tests for core error behavior, provider classifiers, and
observability.
2026-06-05 10:53:06 -07:00
Kazuhiro Sera 5121f302a5 docs: translate pages 2026-06-05 09:53:07 +09:00
William f4e5d96f9f docs: add Latitude to external tracing processors list (#3577) 2026-06-04 23:03:01 +00:00
John CSA b051a70393 test: add unit tests for _openai_retry helpers (#3544) 2026-06-04 07:38:26 +00:00
John CSA 0e9807b6c4 docs: fix two docstring grammar errors in tool.py (#3543) 2026-06-04 07:36:13 +00:00
John CSA bf0d025dc9 test: add unit tests for run_demo_loop streaming, EOF, and empty-input paths (#3542) 2026-06-04 07:32:26 +00:00
MUHAMMAD SALMAN HUSSAIN 9470af01ed docs: fix docstring typo in stdio params env description (#3557) 2026-06-04 07:19:52 +00:00
jdoughty04 9520a5c558 chore: bump Modal sandbox extra to 1.4.3 (#3538) 2026-06-04 16:17:58 +09:00
John CSA 464043a5e6 docs: add missing space in MCP params docstrings (#3535) 2026-05-31 05:07:48 +00:00
John CSA bd8b1b7541 fix: use tuple form for SpeechGroupSpanData __slots__ (#3534) 2026-05-31 05:07:10 +00:00
Kazuhiro Sera d87fec8589 docs: remove beta label from realtime agent docs 2026-05-30 08:10:04 +09:00
Alex Bevilacqua 9411cee8e2 docs: add MongoDB session example under examples/memory (#3036) 2026-05-28 09:37:00 +00:00
Kazuhiro Sera 921135630b fix: #3512 type tool-end hook results as object (#3518) 2026-05-28 17:57:33 +09:00
github-actions[bot] 6d5b888f6f Release 0.17.4 (#3505) 2026-05-26 17:53:21 +09:00
lionel-oai 9a92ea4c8e Support Realtime custom voice objects (#3473) 2026-05-26 17:39:40 +09:00
rmotgi1227 fedc809afd fix: export MCPListToolsItem, ToolSearchCallItem, and ToolSearchOutputItem from agents (#3490) 2026-05-22 01:08:07 +00:00
rmotgi1227 573530febb fix: export more tracing related functions & types from agents (#3489) 2026-05-22 09:59:58 +09:00
Shaurya Singh 813a00324e fix: redact invalid JSON payload in ModelBehaviorError data (#3485) 2026-05-21 23:54:12 +00:00
rmotgi1227 eda7b51aea fix: add missing entries to span __slots__ (#3483) 2026-05-21 22:06:41 +00:00
Kazuhiro Sera 45effb4b7d fix: #3459 add opt-in recovery for missing function tools (#3461) 2026-05-21 10:46:44 +09:00
rmotgi1227 9303389d84 fix: use non-None value for output in FunctionSpanData (#3475) 2026-05-21 10:03:01 +09:00
Illia Oleksiuk 9514473c23 fix: apply hardened http client default to MCP SSE transport (#3466) 2026-05-20 15:14:47 +09:00
Kazuhiro Sera 445ad2273c docs: add SECURITY.md in the same way with openai-agents-js repo 2026-05-20 12:38:30 +09:00
github-actions[bot] 17f7caeaa3 Release 0.17.3 (#3417) 2026-05-19 10:24:42 +09:00
Adrian f6ba91b120 Runtime handling updates (#3451)
## Summary
- Refresh runtime handling around session and tool-call flows.
- Adjust model configuration metadata used by runtime integrations.
- Add focused coverage for the updated behavior.

## Validation
- .venv/bin/python -m pytest tests/model_settings/test_serialization.py
tests/models/test_trace_config.py
tests/mcp/test_streamable_http_client_factory.py
tests/test_run_context_approvals.py
tests/test_run_state.py::TestRunState::test_trace_api_key_serialization_is_opt_in
tests/realtime/test_session.py
- .venv/bin/ruff check <touched files>
- .venv/bin/ruff format --check <touched files>
- git diff --check
2026-05-18 16:53:20 -07:00
Shaurya Singh 65774ce88d docs: fix duplicated word in usaspending glossary example (#3445) 2026-05-18 17:05:16 +09:00
Kazuhiro Sera 13d1815218 docs: re-fix #3444 2026-05-18 17:03:06 +09:00
Nachiket Torwekar 41fe113dd0 docs: fix LiteLLM API reference redirect (#3444) 2026-05-18 07:05:30 +00:00
Kazuhiro Sera 4970fd6ce4 fix: keep mountpoint credentials out of sandbox commands (#3429) 2026-05-18 11:30:31 +09:00
c 4bd459e403 fix: #3363 honor short custom voice splitter chunks (#3364) 2026-05-16 10:04:23 +09:00
Matthew.K e37b3d266b fix: normalize leading question marks in exposed port queries (#3424) 2026-05-16 09:59:39 +09:00
Matthew.K 94523f946e fix: reject relative sandbox workspace roots (#3422) 2026-05-16 09:47:27 +09:00
Tianyu Cai cb0461d177 fix: log exception when output guardrail raises instead of silently ignoring (#3411) 2026-05-16 09:39:27 +09:00
Illia Oleksiuk 5e71d09554 fix: guard None text in ItemHelpers.extract_last_content (#3394) 2026-05-15 15:40:30 +09:00
Illia Oleksiuk cb7211b599 fix: filter hosted_tool_call types in remove_all_tools handoff filter (#3386) 2026-05-15 15:40:04 +09:00
Tianyu Cai 43a389d462 fix: skip wait_for_status when Vercel sandbox is in a terminal state (#3410) 2026-05-15 15:39:35 +09:00
c 656baf8ead fix: #3357 output schema names for Literal types (#3358) 2026-05-14 19:24:35 +09:00
Illia Oleksiuk eca794c0bc fix: avoid mutating codex output schema input (#3385) 2026-05-14 12:49:48 +09:00
Illia Oleksiuk 7865ec9819 fix: avoid mutating FunctionTool params_json_schema (#3382) 2026-05-14 12:48:57 +09:00
Kazuhiro Sera f7e8196484 chore: clean up CI jobs and update uv pin (#3400) 2026-05-14 12:36:41 +09:00
Drew Hintz bdd228b4db [codex] Harden release tag workflow (#3399)
## Summary

- Require release-tag PRs to come from the same repository before
creating tags.
- Preserve the existing merged-PR and `release/v*` branch gates.

## Validation

- Parsed `.github/workflows/release-tag.yml` with PyYAML.
2026-05-13 21:58:31 -05:00
Illia Oleksiuk 900cab6212 docs: document auto_previous_response_id (#3383) 2026-05-13 23:35:44 +00:00
Kazuhiro Sera 8dc30e4807 docs: translate all pages using new settings (#3392) 2026-05-14 07:53:49 +09:00
Illia Oleksiuk f9eb3a4f33 docs: mark Agent.instructions as optional (#3384) 2026-05-13 15:31:51 +09:00
Kazuhiro Sera ec016cde9a fix: unify memory optional dependency import errors (#3389) 2026-05-13 11:13:56 +09:00
Kazuhiro Sera 564584513f docs: add SDK review guidance (#3376) 2026-05-12 16:05:42 +09:00
Kazuhiro Sera a466860fdc ci: disable auto labeling job 2026-05-12 15:47:02 +09:00
zhoufengen 03ff10ef6c fix: guard None text in text_message_output and add output guardrail count to RunErrorDetails (#3375) 2026-05-12 14:53:34 +09:00
Kazuhiro Sera 55b859d05c ci: tweak the PR labeling operation 2026-05-12 14:41:56 +09:00
Kazuhiro Sera 76f42d8ed7 ci: tweak the PR labeling operation 2026-05-12 14:37:28 +09:00
github-actions[bot] 55e4a850fc Release 0.17.2 (#3368) 2026-05-12 12:13:02 +09:00
github-actions[bot] 5594fb464d docs: update translated document pages (#3371) 2026-05-12 08:19:18 +09:00
c 64de1cb211 fix: #3310 avoid empty chat tool outputs (#3312) 2026-05-11 23:14:46 +00:00
c 1d3df7fa04 docs: document sandbox archive limits after #3278 release (#3311) 2026-05-12 08:11:23 +09:00
Kazuhiro Sera ae3263b840 docs: normalize memory docstring cross-references (#3370) 2026-05-11 23:03:30 +00:00
c e3c99d9964 fix: #3361 honor session settings in AsyncSQLiteSession (#3362) 2026-05-12 08:01:39 +09:00
c b2bd8218c6 fix: #3359 preserve local approval rejection reasons (#3360) 2026-05-11 22:32:46 +00:00
Kazuhiro Sera 8715a0585a fix: avoid auto response for unknown realtime tools (ref: #3287) (#3366) 2026-05-12 07:27:59 +09:00
c 4a95659892 fix: #3354 interrupt tracing retry backoff on shutdown (#3355) 2026-05-12 07:23:11 +09:00
Kazuhiro Sera 5635fab9d3 fix: #3268 fix OpenAI Conversations reasoning persistence (#3352) 2026-05-12 07:20:47 +09:00
github-actions[bot] 92e014a4cc docs: update translated document pages (#3351) 2026-05-11 16:37:55 +09:00
Kazuhiro Sera 852a8dbad4 docs: clarify max_delay for retries works (#3350) 2026-05-11 16:29:20 +09:00
github-actions[bot] eada610734 Release 0.17.1 (#3290) 2026-05-11 15:33:41 +09:00
Kazuhiro Sera 028abc6a74 fix: make tracing shutdown best-effort on process exit (#3343) 2026-05-11 15:19:32 +09:00
Kazuhiro Sera 4c3de2df65 docs: update MCP examples (#3342) 2026-05-11 11:34:56 +09:00
Sihan Sun cf151f91ff fix: #781 replace assertion in handoff() with UserError (#3339) 2026-05-11 08:19:59 +09:00
Kazuhiro Sera 970db97acb fix: #3333 scope Realtime tool approvals by qualified key (#3340) 2026-05-11 08:12:29 +09:00
Kazuhiro Sera 4bc942af3c fix: #3267 preserve required hosted tool IDs in OpenAI conversation sessions (#3341) 2026-05-11 08:07:05 +09:00
c 52656a51cb fix: #3330 handle string tool trimmer allowlists (#3331) 2026-05-11 07:24:45 +09:00
c 479640e214 fix: #3308 reject chat custom tool calls explicitly (#3309) 2026-05-10 17:32:49 +09:00
c 650212ef62 fix: #3313 align multi-choice chat streams with strict validation (#3314) 2026-05-10 16:28:29 +09:00
c d7417fb2e7 fix: #3319 preserve nested handoff history content (#3320) 2026-05-10 16:24:27 +09:00
Kazuhiro Sera a6a4cc5143 feat: improve examples auto-run coverage and artifact handling (#3328) 2026-05-10 14:25:47 +09:00
Kazuhiro Sera 94ba76de0f fix: include sandbox provider error details (#3326) 2026-05-10 12:38:31 +09:00
c 62560996c2 fix: #3317 return fresh empty strict schemas (#3318) 2026-05-10 08:54:25 +09:00
c 94fa9e21ba fix: #3315 align generic dict output schemas (#3316) 2026-05-10 08:54:16 +09:00
c 610c2742dc fix: #3306 track MongoDB metadata timestamps (#3307) 2026-05-10 08:54:01 +09:00
c 4b8744903a fix: #3304 skip corrupt items during pop (#3305) 2026-05-10 08:53:26 +09:00
c fa75ffc404 fix: #3274 limit sandbox archive extraction (#3278) 2026-05-10 08:52:55 +09:00
Kazuhiro Sera bc3607bae4 fix: preserve GitRepo root subpath aliases (#3303) 2026-05-09 12:06:45 +00:00
Kazuhiro Sera 9154d836a2 fix: allow empty GitRepo subpaths as repository root (#3299) 2026-05-09 07:54:00 +00:00
Kazuhiro Sera 1289fb0bef fix: make chat completions response-feature validation opt-in (#3298) 2026-05-09 16:46:47 +09:00
Kazuhiro Sera 73bc963398 chore: improve automated example coverage and local service handling (#3297) 2026-05-09 16:17:49 +09:00
Kazuhiro Sera 43e051e725 fix: guard no-op tracing span IDs (#3296) 2026-05-09 14:19:24 +09:00
Kazuhiro Sera 38bef807f5 test: guard Responses transport extra kwargs with official client (#3295) 2026-05-09 04:42:39 +00:00
Kazuhiro Sera fc1abe0999 Revert "fix(models): allow extra_query/extra_body via extra_args in Responses" (#3294) 2026-05-09 04:28:16 +00:00
Aditya Singh dbb3181386 fix(tracing): keep BatchTraceProcessor worker alive on exporter errors (#3216) 2026-05-09 13:17:06 +09:00
Aditya Singh 3031b13eaa fix(realtime): validate RealtimeAgent fields in __post_init__ (#3234) 2026-05-09 12:54:03 +09:00
Aditya Singh 8f40dde4a4 fix(litellm): avoid duplicating content and signed thinking blocks across parallel tool-call splits (#3215) 2026-05-09 12:52:11 +09:00
Aditya Singh 4bb388c955 fix(redis-session): preserve created_at across writes (#3202) 2026-05-09 12:51:03 +09:00
Aditya Singh 29b2acffb1 fix(handoffs): preserve HandoffInputData.input_items in remove_all_tools (#3253) 2026-05-09 12:50:16 +09:00
Aditya Singh f3d434cdfd fix(voice): stop AudioInput.to_base64() from mutating caller's buffer (#3201) 2026-05-09 12:48:48 +09:00
github-actions[bot] 33b9a2c9fd docs: update translated document pages (#3293) 2026-05-09 12:12:05 +09:00
Yaron Schneider 272dd18b9f docs: add dapr to durable orchestration integrations (#3292) 2026-05-09 11:53:05 +09:00
c cc2998845b fix: #3284 wake realtime event iterators on close (#3285) 2026-05-09 11:50:59 +09:00
c 035271db60 fix: #3282 reject unsupported Chat Completions reusable prompts (#3283) 2026-05-09 11:49:39 +09:00
c 55eb3dec30 fix: #3273 validate git repo subpaths (#3276) 2026-05-09 11:47:50 +09:00
Aditya Singh f8ba94d416 fix(realtime): treat None audio.input/audio.output as unset (#3254) 2026-05-09 11:45:36 +09:00
Aditya Singh 7aeb39e150 fix(sessions): skip corrupt docs in MongoDBSession.pop_item (#3247) 2026-05-09 11:44:54 +09:00
Aditya Singh 7e9089f270 fix(realtime): preserve output_audio content parts in output_item events (#3230) 2026-05-09 11:44:14 +09:00
Quratulain-bilal 960e979f74 fix: await cancelled output guardrail tasks on tripwire (#3187) 2026-05-09 11:43:15 +09:00
Aditya Singh e6a3ed887a fix: avoid duplicating content and signed thinking blocks across parallel tool-call splits (#3261) 2026-05-09 02:48:56 +09:00
Aditya Singh e86dff2907 fix: exclude Computer instances from provider duck-typing (#3249) 2026-05-09 02:45:31 +09:00
Aditya Singh 12ad112fec fix(realtime): raise UserError for input_type without on_handoff (#3248) 2026-05-09 02:44:54 +09:00
Aditya Singh 250fb97fef fix(run): preserve last known response_id on conversation resume (#3245) 2026-05-09 02:42:05 +09:00
Aditya Singh 62f9416ead fix(realtime): skip invalid input_text parts in user input conversion (#3243) 2026-05-09 02:41:12 +09:00
Aditya Singh 76702572ef fix: skip needs_approval_checker when status already resolved (#3229) 2026-05-09 02:37:35 +09:00
Aditya Singh 1b6876a8c2 fix(sessions): persist output_tokens_details when input details are None (#3227) 2026-05-09 02:36:18 +09:00
Aditya Singh 89f368df03 fix: await on_handoff callables with async __call__ (#3211) 2026-05-09 02:34:17 +09:00
Aditya Singh 730ee55e14 fix(mcp): isolate strict schema conversion from non-strict fallback (#3199) 2026-05-09 02:33:38 +09:00
c b4741e077d fix: #3288 normalize RunState guardrail payloads (#3289) 2026-05-09 02:18:02 +09:00
c 4a3d33e44f fix: #3280 streaming guardrail exception cleanup (#3281) 2026-05-09 02:16:03 +09:00
Aditya Singh 9242902e1b fix(run): preserve failed status across apply_patch operations (#3217) 2026-05-09 02:08:21 +09:00
Aditya Singh b4be17586f fix(realtime): preserve existing transcript over stale delta accumulator (#3214) 2026-05-09 02:07:52 +09:00
Aditya Singh ec5523d620 fix: preserve existing request_usage_entries on Usage.add (#3213) 2026-05-09 02:07:20 +09:00
Aditya Singh 1d38492b44 fix: drop reasoning items orphaned by dropped tool calls (#3207) 2026-05-09 02:06:58 +09:00
Aditya Singh f32f613a7f fix(strict-schema): preserve chained $ref during sibling-key expansion (#3205) 2026-05-09 02:05:20 +09:00
Aditya Singh de60d05b0c fix(models): allow extra_query/extra_body via extra_args in Responses (#3194) 2026-05-09 02:05:02 +09:00
Aditya Singh cc5a392583 fix: preserve tool guardrail results across handoffs in SingleStepResult (#3237) 2026-05-09 02:04:13 +09:00
Aditya Singh 8b04ee0939 fix(exceptions): export MCPToolCancellationError from top-level package (#3210) 2026-05-09 01:58:48 +09:00
c e19ac4e44d fix: #3286 send realtime output for unknown tool calls (#3287) 2026-05-08 16:55:10 +00:00
Aditya Singh c7bcdd4a4a fix(realtime): expose max_output_tokens on RealtimeSessionModelSettings (#3223) 2026-05-09 01:50:19 +09:00
c 58a89c810f fix: #3275 reject chat completions server state (#3279) 2026-05-08 16:46:23 +00:00
c 9a0c07f9c7 fix: #3270 Validate model retry backoff settings (#3272) 2026-05-09 01:40:28 +09:00
Aditya Singh 8619dfda75 fix: skip CompactionItem silently in stream queue helper (#3224) 2026-05-08 16:27:11 +00:00
github-actions[bot] 683b6e79e5 docs: update translated document pages (#3193) 2026-05-08 17:24:16 +09:00
Kazuhiro Sera e3746c52d9 docs: updates for v0.17.0 (#3188) 2026-05-08 17:10:34 +09:00
github-actions[bot] 0fea7e8347 Release 0.17.0 (#3191) 2026-05-08 17:07:36 +09:00
Kazuhiro Sera 0a76dd03ce docs: improve auto run for examples 2026-05-08 16:56:37 +09:00
Kazuhiro Sera f47d486985 fix: #3169 constrain local sandbox artifact sources to base dir (#3177) 2026-05-08 15:25:13 +09:00
Kazuhiro Sera 1660d306b5 feat: default realtime sessions to gpt-realtime-2 (#3190) 2026-05-08 15:24:41 +09:00
Abdulrahman Alfozan ee36d43584 Fix Responses extra_args collision with omitted kwargs (#3185)
### Summary

Fix a false duplicate-argument error when a Responses request parameter
is supplied through `ModelSettings.extra_args` and the first-class
request field is only present as OpenAI's `omit` sentinel.

### Test plan

- `make format`
- `make lint`
- `uv run pytest
tests/models/test_openai_responses.py::test_build_response_create_kwargs_allows_extra_arg_when_explicit_arg_is_omitted
tests/models/test_openai_responses.py::test_build_response_create_kwargs_rejects_duplicate_context_management_extra_args
tests/models/test_openai_responses.py::test_build_response_create_kwargs_rejects_duplicate_extra_args_keys
-q`
- `uv run mypy src/agents/models/openai_responses.py
tests/models/test_openai_responses.py`
- `uv run pyright src/agents/models/openai_responses.py
tests/models/test_openai_responses.py`
- `git diff --check`

### Issue number

N/A

### Checks

- [x] I've added new tests (if relevant)
- [ ] I've added/updated the relevant documentation
- [x] I've run `make lint` and `make format`
- [x] I've made sure tests pass where not blocked by unrelated
local-only failures
2026-05-07 16:23:13 -07:00
Kazuhiro Sera a91f630f79 fix: skip prerequisite-bound examples in auto runs 2026-05-07 20:03:01 +09:00
github-actions[bot] bd84b65258 Release 0.16.1 (#3167) 2026-05-07 20:00:54 +09:00
Kazuhiro Sera 28de3652d3 fix: #3168 validate MCP require_approval policies (#3179) 2026-05-07 18:26:48 +09:00
c a67d95f58a fix: #3174 count valid encrypted session items for limits (#3175) 2026-05-07 16:15:52 +09:00
Illia Oleksiuk 3a11cf5225 fix: reject non-object function tool input JSON (#3166) 2026-05-07 14:41:38 +09:00
Kazuhiro Sera 170ee73f94 fix: #3109 stabilize chat completions stream output indexes (#3176)
Co-authored-by: Aphroq <37263590+Aphroq@users.noreply.github.com>
2026-05-07 14:40:38 +09:00
c 6f5fbf6dbb fix: #3170 clean up git repo temp clones on failure (#3172) 2026-05-07 14:05:24 +09:00
c 516aa0c9d9 fix: #3171 reject corrupt Dapr session state updates (#3173) 2026-05-07 14:05:10 +09:00
c 0fb2e0944c fix: restore session history after compaction replacement failures (#3117) 2026-05-07 12:38:07 +09:00
github-actions[bot] eed9100777 docs: update translated document pages (#3165) 2026-05-07 10:00:27 +09:00
Kazuhiro Sera f185dfa2a0 docs: document tool execution concurrency (#3164) 2026-05-07 09:54:37 +09:00
github-actions[bot] c5ebf809ea docs: update translated document pages (#3163) 2026-05-07 09:36:28 +09:00
Kazuhiro Sera 1683357483 docs: add 0.16.0 changelog (#3153) 2026-05-07 09:29:51 +09:00
Kazuhiro Sera 0ed4ee6e18 docs: updates for #3147 (#3148) 2026-05-07 09:29:41 +09:00
github-actions[bot] 9f361ba7dd Release 0.16.0 (#3150) 2026-05-07 09:19:36 +09:00
github-actions[bot] e8856de1b4 docs: update translated document pages (#3162) 2026-05-07 09:02:55 +09:00
mindbomber 5a10e46f11 docs: realtime guardrail fallback behavior (#3157) 2026-05-06 23:50:27 +00:00
Kazuhiro Sera 8c8a2eb32e fix: #3104 stabilize chat completions tool call output indexes (#3161) 2026-05-07 08:49:55 +09:00
github-actions[bot] ff8e3db4b2 docs: update translated document pages (#3160) 2026-05-07 08:45:27 +09:00
Kazuhiro Sera 8526723b49 feat: #1859 add runtime function tool concurrency config (#3152) 2026-05-07 08:40:30 +09:00
MAV 0466636b77 feat: #1167 add opt-in server-prefixed MCP tool names (#3019) 2026-05-07 08:40:21 +09:00
Illia Oleksiuk f903926394 fix: make Permissions hashable to match User and Group (#3154) 2026-05-07 08:38:16 +09:00
github-actions[bot] 1a1b35d4fb docs: update translated document pages (#3151) 2026-05-06 21:21:34 +09:00
Kazuhiro Sera fc2d208f30 feat: switch the default model to a newer mini model (affecting only when a model is unset) (#3147) 2026-05-06 21:14:51 +09:00
Kazuhiro Sera b9cbab149f feat: allow disabling max_turns with None (#3132) 2026-05-06 21:14:27 +09:00
c bed924b45d fix: reject external symlink targets during hydrate (#3094) 2026-05-06 21:13:53 +09:00
github-actions[bot] e1cb2be4ca Release 0.15.3 (#3149) 2026-05-06 21:09:15 +09:00
Quratulain-bilal b1722a7459 fix: tolerate audio deltas before audio format negotiation in ModelAu… (#3141) 2026-05-06 19:01:41 +09:00
Aditya Singh 0370fd3527 test(realtime): cover overlapping tool response creates (#3140) 2026-05-06 18:57:22 +09:00
Aditya Singh 9a2b4a9b85 fix(mcp): make duplicate tool errors deterministic (#3136) 2026-05-06 17:37:14 +09:00
Aditya Singh e22f25a431 fix(mcp): reject non-object tool input JSON (#3135) 2026-05-06 17:36:30 +09:00
Aditya Singh 6e691ee2ea fix(mcp): avoid mutating tool input schemas (#3134) 2026-05-06 17:34:49 +09:00
github-actions[bot] ce462354fd docs: update translated document pages (#3131) 2026-05-06 09:50:41 +09:00
Kazuhiro Sera 02a6b21151 docs: updates for #3128 (#3129) 2026-05-06 09:37:58 +09:00
github-actions[bot] 75da8e0200 Release 0.15.2 (#3099) 2026-05-06 09:33:20 +09:00
c 3d1231e0fe fix: redact function tool trace span errors (#3111) 2026-05-06 00:30:48 +09:00
Kazuhiro Sera 601ecf5503 fix: #3123 avoid replaying assistant conversation item IDs for OpenAIConversationsSession (#3127) 2026-05-05 20:55:29 +09:00
Kazuhiro Sera 574a598fae feat: add context management model setting (#3128) 2026-05-05 20:55:16 +09:00
c 7a5d32bc83 fix: block disabled function tools before execution (#3118) 2026-05-05 09:43:09 +09:00
Felmon 613b8f39a6 fix(mcp): isolate merged tool metadata (#3114) 2026-05-05 09:41:09 +09:00
c 9b57f057b4 fix: reject failed responses stream terminals (#3107) 2026-05-04 22:20:46 +09:00
c ae60947451 fix: redact MCP invalid JSON errors when tool logging is disabled (#3088) 2026-05-04 22:14:01 +09:00
Quratulain-bilal 1b7d878b7c test: add direct unit tests for _mcp_tool_metadata helpers (#3102) 2026-05-04 18:00:20 +09:00
Quratulain-bilal b80d541946 test: add direct unit tests for _tool_identity helpers (#3101) 2026-05-04 17:59:43 +09:00
Quratulain-bilal 54ec5f0091 test: cover real Handoff object branch in visualization (#3100) 2026-05-04 17:58:14 +09:00
c 3854c124cb fix: only rewind matching session suffixes (#3090) 2026-05-04 10:05:37 +09:00
Illia Oleksiuk 63ebf5ada1 fix: make ToolContext hashable to match RunContextWrapper (#3097) 2026-05-04 09:58:28 +09:00
Quratulain-bilal fbc5a44045 test: cover on_handoff, on_tool_start, on_tool_end on RunHooks (#3098) 2026-05-04 00:58:08 +00:00
Quratulain-bilal 4bb4400731 fix: filter custom_tool_call types in remove_all_tools handoff filter (#3095) 2026-05-04 09:55:20 +09:00
c 4b5a0b89cd fix: reject string-like shell commands (#3092) 2026-05-04 09:50:59 +09:00
c 9d24382d0e test: add realtime tool output serialization edge cases (#3077) 2026-05-04 09:43:14 +09:00
c 044d44ce0d test: cover realtime tool timeout behaviors in realtime session (#3076) 2026-05-04 09:42:47 +09:00
Kazuhiro Sera f2fb9ffb66 test: improve coverage and organize test layout (#3085) 2026-05-02 15:40:54 +09:00
github-actions[bot] 60b7bee807 docs: update translated document pages (#3084) 2026-05-02 14:19:23 +09:00
Kazuhiro Sera e9a3e3610c docs: updates for #3080 (#3081) 2026-05-02 14:09:31 +09:00
github-actions[bot] ceb238fd1b Release 0.15.1 (#3083) 2026-05-02 11:19:33 +09:00
Kazuhiro Sera 41c646d898 fix: restore UnixLocal PTY terminal signal defaults (#3082) 2026-05-02 00:30:19 +00:00
Kazuhiro Sera 4b2881c7c4 feat: expose Responses WebSocket keepalive options (#3080) 2026-05-02 09:18:45 +09:00
c a47b7ea7ec fix: #3074 restore SIGINT defaults for UnixLocal PTY children (#3075) 2026-05-02 09:16:36 +09:00
c 42c3015518 chore: harden Dapr Redis integration fixture loading (#3078) 2026-05-02 09:15:41 +09:00
Quratulain-bilal ae224b4449 test: cover guardrail name fallback to function __name__ (#3073) 2026-05-02 08:14:47 +09:00
github-actions[bot] 756fa431a7 docs: update translated document pages (#3072) 2026-05-01 17:47:40 +09:00
Kazuhiro Sera 841f72f296 docs: improve quickstart documentation for Windows OS users (#3071) 2026-05-01 08:35:29 +00:00
github-actions[bot] 611d080ff0 docs: update translated document pages (#3064) 2026-05-01 15:52:30 +09:00
Kazuhiro Sera 48d7e9cae7 docs: add 0.15 changelog (#3058) 2026-05-01 15:45:49 +09:00
github-actions[bot] f7410c8e96 Release 0.15.0 (#3063) 2026-05-01 15:42:28 +09:00
Kazuhiro Sera da3f15708e ci: improve translation pipelone to be more robust 2026-05-01 15:32:07 +09:00
dependabot[bot] 498390a741 chore(deps): bump openai/codex-action from 1.6 to 1.8 (#3060) 2026-05-01 14:31:21 +09:00
dependabot[bot] 7833715ac2 chore(deps): bump pypa/gh-action-pypi-publish from 1.13.0 to 1.14.0 (#3061) 2026-05-01 14:31:04 +09:00
dependabot[bot] d996707c0a chore(deps): bump peter-evans/create-pull-request from 8.1.0 to 8.1.1 (#3062) 2026-05-01 14:30:06 +09:00
dependabot[bot] ec99da6375 chore(deps): bump actions/github-script from 8.0.0 to 9.0.0 (#3059) 2026-05-01 14:29:30 +09:00
Kazuhiro Sera 2d40c09c88 fix: #3055 surface model refusals during run resolution (#3057) 2026-05-01 14:28:13 +09:00
Gopal Bagaswar 3a3f34f18d docs: add missing space after period in MCPServerStdio docstring (#3053) 2026-04-30 12:19:20 +09:00
github-actions[bot] 7029ea8fff Release 0.14.8 (#3049) 2026-04-29 12:39:00 +09:00
ateamofantsintheirprime 572c7bf1b5 docs: fix typo in comment for WS event handler (#3050) 2026-04-29 01:56:29 +00:00
Kazuhiro Sera 0661c9e9ca fix: #3043 delimit sandbox prompt instruction sections (#3047) 2026-04-29 10:40:48 +09:00
Kazuhiro Sera 05004336a2 fix: #3046 preserve MCP re-export import errors (#3048) 2026-04-29 01:40:17 +00:00
github-actions[bot] 8d7f05b6b2 Release 0.14.7 (#3031) 2026-04-28 15:45:35 +09:00
Abdulrahman Alfozan ebdb0f2ee1 fix: add GPT-5.5 aliases to sandbox compaction (#3039)
### Summary

Adds GPT-5.5 snapshot and pro aliases to the sandbox compaction model
context-window map so compaction threshold calculation recognizes them
as 1,047,576-token models.

### Test plan

- `uv run ruff check src/agents/sandbox/capabilities/compaction.py
tests/sandbox/test_compaction.py`
- `uv run pytest tests/sandbox/test_compaction.py`
- Verified `CompactionModelInfo.for_model()` locally for
`gpt-5.5-2026-04-23`, `gpt-5.5-pro`, and `gpt-5.5-pro-2026-04-23`
- Verified the model IDs exist via the OpenAI Models API; also ran a
live Responses API smoke with `gpt-5.5-2026-04-23`

### Issue number

N/A

### Checks

- [x] I've added new tests (if relevant)
- [ ] I've added/updated the relevant documentation
- [ ] I've run `make lint` and `make format`
- [x] I've made sure tests pass
2026-04-28 06:34:34 +00:00
Andi Liu b3688db750 [sandbox] Raise Phase 2 memory consolidation turn limit (#3038)
### Summary

- Phase 2 sandbox memory consolidation can need more than the default
runner turn cap when rewriting memory files.
- Pass `max_turns=500` for the Phase 2 consolidation agent only.
2026-04-27 16:39:46 -07:00
Federico 8025ed0b42 Fix: remove unset fields from calls to Responses API (#3026) 2026-04-27 09:50:10 +09:00
Matthew.K ba889de480 fix: reject symlinked LocalFile sources (#2972) 2026-04-26 11:15:12 +09:00
Ali jawwad 5df41d3013 feat: #2886 add convenience properties (tool_name, call_id) to tool items (#3027)
Co-authored-by: Daniel Cufiño <dancufino@gmail.com>
2026-04-26 10:32:30 +09:00
Matthew.K 2eb8713b53 fix: tighten tar and zip member validation (#3028) 2026-04-26 10:24:36 +09:00
github-actions[bot] 9a207b6938 Release 0.14.6 (#3022) 2026-04-25 11:30:43 +09:00
github-actions[bot] 9af6ad111c docs: update translated document pages (#3023) 2026-04-25 11:14:50 +09:00
Alex Bevilacqua 1821bf8094 docs: add MongoDB session documentation (#3015) 2026-04-25 11:07:31 +09:00
github-actions[bot] 071e2b68f6 docs: update translated document pages (#3021) 2026-04-25 10:49:51 +09:00
Abdulrahman Alfozan c88f339d32 Update examples and defaults to GPT-5.5 (#3016) 2026-04-25 10:31:54 +09:00
mathis obadia 5ffc1ecee4 relax websockets upper bound from <16 to <17 (#3013) 2026-04-24 08:06:24 +09:00
mcgrew-oai c2cb031461 chore: harden uv dependency resolution (#3014) 2026-04-24 08:03:31 +09:00
github-actions[bot] fe3a5e6c27 Release 0.14.5 (#3007) 2026-04-23 11:19:40 +09:00
Kazuhiro Sera 16e040929b fix: #3004 serve HITL resume tool outputs (#3006) 2026-04-23 10:36:15 +09:00
Kazuhiro Sera 5be06a16a9 feat: #3001 add Modal sandbox idle timeout option (#3005) 2026-04-23 09:40:23 +09:00
ankitphogat 81c57c5e7e fix: backfill streamed terminal output (#3000) 2026-04-23 09:32:44 +09:00
github-actions[bot] 638388ad17 docs: update translated document pages (#2999) 2026-04-22 10:35:42 +09:00
Abdulrahman Alfozan 734d42490a docs: clarify lazy skill source host paths (#2998) 2026-04-22 10:26:28 +09:00
github-actions[bot] 4e43cbaf09 Release 0.14.4 (#2989)
### Release readiness review (v0.14.3 -> TARGET
ba3b17740b280c00330d9cd61158f88a34d28d2a)

This is a release readiness report done by `$final-release-review`
skill.

### Diff


https://github.com/openai/openai-agents-python/compare/v0.14.3...ba3b17740b280c00330d9cd61158f88a34d28d2a

### Release call:
**🟢 GREEN LIGHT TO SHIP** Patch bump to `0.14.4` with additive sandbox
capability and broad compatibility-focused test coverage; no concrete
release-blocking issues found in the diff.

### Scope summary:
- 34 files changed (+2236/-536); key areas touched:
`src/agents/sandbox/session/` refactors/extractions, new `BoxMount`
support in sandbox mount providers, provider integrations (`cloudflare`,
`vercel`, `daytona`, `e2b`, `blaxel`), sandbox docs, and substantial
sandbox compatibility/regression tests.
- Commit range reviewed (oldest to newest): docs cleanup/translation
updates, sandbox compatibility guards, snapshot default fix, Box mount
feature, shared mount lifecycle refactor, shared tar exclude refactor,
session helper extraction, version bump to `0.14.4`.

### Risk assessment (ordered by impact):
No material risks identified.

### Notes:
- Base tag determined from local tags only per instruction: `v0.14.3`.
- Target resolved from current `HEAD`:
`ba3b17740b280c00330d9cd61158f88a34d28d2a`.
- Assessment is based on diff/log/code inspection and added tests
in-range; no additional local test execution was performed in this CI
review step.

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-04-21 12:22:38 -07:00
github-actions[bot] fdf2d009a6 docs: update translated document pages (#2996)
Automated update of translated documentation.

Triggered by commit:
[4c5112cbf4](https://github.com/openai/openai-agents-python/commit/4c5112cbf4b4467336939965fbfa28ad623b9df4).
Message: `feat: add BoxMount support (#2988)

### Summary

This pull request adds Box as an rclone-backed sandbox mount provider.

- Adds `BoxMount` with Docker rclone volume-driver support and
in-container `RcloneMountPattern` config generation.
- Wires Box into the sandbox entry exports and provider docs.
- Updates rclone-backed sandbox extension wording for Daytona, E2B, and
Runloop.
- Adds Docker and rclone mount config tests for Box auth/path options.

### Test plan

- `bash .agents/skills/code-change-verification/scripts/run.sh` *(format
and lint passed; typecheck failed on pre-existing local ignored
`tests/local` symlink files and unrelated Temporal example dependency
typing)*
- `uv run pyright --project pyrightconfig.json src/agents/sandbox
src/agents/extensions/sandbox tests/sandbox/test_mounts.py
tests/sandbox/test_docker.py`
- `uv run mypy src/agents/sandbox src/agents/extensions/sandbox
tests/sandbox/test_mounts.py tests/sandbox/test_docker.py`
- `uv run pytest -q tests/sandbox/test_mounts.py
tests/sandbox/test_docker.py`

### Issue number

N/A

### Checks

- [x] I've added new tests (if relevant)
- [x] I've added/updated the relevant documentation
- [x] I've run `make lint` and `make format`
- [x] I've made sure tests pass

Co-authored-by: Carter Rabasa <carter.rabasa@gmail.com>`

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2026-04-21 12:22:28 -07:00
Kazuhiro Sera 333721d72f refactor: extract sandbox session helper operations (#2985)
This pull request improves the sandbox session implementation by moving
archive extraction, manifest application, and snapshot lifecycle logic
out of `BaseSandboxSession` into focused helper modules. The existing
`BaseSandboxSession` method surface remains in place and delegates to
the new helpers, preserving provider overrides and compatibility while
reducing the size and responsibility concentration of the base session
class.
2026-04-21 11:52:04 -07:00
Kazuhiro Sera 4c68780ad3 refactor: share sandbox tar exclude arg generation (#2987)
This pull request improves sandbox workspace persistence internals by
sharing the shell `tar --exclude` argument generation used by Blaxel,
Daytona, and E2B sandbox sessions.

It adds a small common helper under `src/agents/sandbox/session/` and
keeps each provider's existing private `_tar_exclude_args()` surface as
a thin delegate, preserving the generated command strings while removing
duplicated quoting, sorting, and path-normalization logic. Direct unit
coverage was added for empty/dot paths, stable sorting, shell quoting,
dot-prefixed patterns, and absolute-path normalization.
2026-04-21 11:52:02 -07:00
Kazuhiro Sera 3aad7eba85 refactor: share sandbox ephemeral mount lifecycle (#2986)
This pull request improves sandbox backend persistence by extracting the
common ephemeral mount teardown and restore flow into a shared session
helper. Cloudflare and Vercel persistence now use the shared lifecycle
wrapper for persist and hydrate operations while preserving existing
archive error precedence and corruption context metadata.
2026-04-21 11:50:22 -07:00
Abdulrahman Alfozan 4c5112cbf4 feat: add BoxMount support (#2988)
### Summary

This pull request adds Box as an rclone-backed sandbox mount provider.

- Adds `BoxMount` with Docker rclone volume-driver support and
in-container `RcloneMountPattern` config generation.
- Wires Box into the sandbox entry exports and provider docs.
- Updates rclone-backed sandbox extension wording for Daytona, E2B, and
Runloop.
- Adds Docker and rclone mount config tests for Box auth/path options.

### Test plan

- `bash .agents/skills/code-change-verification/scripts/run.sh` *(format
and lint passed; typecheck failed on pre-existing local ignored
`tests/local` symlink files and unrelated Temporal example dependency
typing)*
- `uv run pyright --project pyrightconfig.json src/agents/sandbox
src/agents/extensions/sandbox tests/sandbox/test_mounts.py
tests/sandbox/test_docker.py`
- `uv run mypy src/agents/sandbox src/agents/extensions/sandbox
tests/sandbox/test_mounts.py tests/sandbox/test_docker.py`
- `uv run pytest -q tests/sandbox/test_mounts.py
tests/sandbox/test_docker.py`

### Issue number

N/A

### Checks

- [x] I've added new tests (if relevant)
- [x] I've added/updated the relevant documentation
- [x] I've run `make lint` and `make format`
- [x] I've made sure tests pass

Co-authored-by: Carter Rabasa <carter.rabasa@gmail.com>
2026-04-21 11:46:30 -07:00
Matthew.K 106ef05317 fix: ignore relative snapshot base overrides (#2976)
Co-authored-by: Kazuhiro Sera <seratch@openai.com>
2026-04-21 22:54:25 +09:00
Kazuhiro Sera 2a515f0eb4 test: add sandbox compatibility guards (#2984) 2026-04-21 03:00:46 +00:00
github-actions[bot] 9e228fc959 docs: update translated document pages (#2982) 2026-04-21 10:24:08 +09:00
Abhishek Krishna bf3e9d178c docs: remove duplicate word in voice interruptions section (#2981) 2026-04-21 10:16:45 +09:00
1056 changed files with 263516 additions and 25857 deletions
+40
View File
@@ -0,0 +1,40 @@
# SDK Maintainer References
This directory captures long-lived implementation contracts of the OpenAI Agents Python SDK that are not replaceable by OpenAI API or platform facts from the Developer Docs MCP. The repo's `docs/` remain an SDK-specific behavioral contract; these references distill the ownership, compatibility, ordering, and failure semantics that maintainers need to preserve that contract.
## Usage
Read the reference map before changing or reviewing an affected runtime boundary, then open only the files relevant to that boundary. During issue and PR review, treat this directory as read-only background: use it to identify expected invariants, adjacent surfaces, and regression risks, but verify the current claim against the remote issue or PR, current code, tests, docs, release boundary, and focused runtime evidence. Do not edit references as a side effect of a review or treat them as proof of current issue status, PR behavior, or repository readiness.
When implementation or dedicated repository-maintenance work establishes a reusable invariant that remains valid beyond one issue or PR, update the narrowest owning reference separately. Preserve the generalized contract, not the case history or decision outcome that revealed it.
## Inclusion Criteria
Add or retain a reference when the knowledge is SDK-specific, stable across multiple releases, easy to violate from one local code path, and expensive to reconstruct from source, tests, and repo docs during every review. Treat `docs/` as the SDK's user-facing behavioral contract; use these references to preserve the implementation constraints behind that contract. Prefer invariants and ownership rules over summaries of individual issues, PRs, or recent fixes.
Do not store current issue or PR status, generic maintainer-review workflow, release notes, OpenAI API or platform behavior available through `$openai-knowledge`, or one-off implementation details in this directory. Put review methodology under `.agents/skills/`, released migration notes in `docs/release.md`, and API or platform facts behind `$openai-knowledge`.
## Reference Map
| Reference | Read before changing or reviewing |
|---|---|
| [Agent definition and run context](agent-definition-and-run-context.md) | Agent fields, cloning, dynamic instructions, enabled tools or handoffs, context wrappers, usage, or public agent identity |
| [Runner lifecycle](runner-lifecycle.md) | Turn accounting, guardrails, handoffs, interruptions, cancellation, or streaming parity |
| [Run item lifecycle](run-item-lifecycle.md) | Model output processing, new item types, stream events, replay conversion, session persistence, or RunState serialization |
| [Function and output schema](function-and-output-schema.md) | Function-tool signatures and metadata, strict JSON schema conversion, or structured output types |
| [Conversation state ownership](conversation-state-ownership.md) | Sessions versus server-managed continuation, input deltas, retries, compaction, or conversation resume |
| [Session persistence](session-persistence.md) | Session input callbacks, per-turn saves, retry rewind, atomicity, or compaction replacement |
| [RunState schema and resume boundary](runstate-schema.md) | Serialized state, schema versions, approvals, agent identity, or durable resume data |
| [Tool identity and routing](tool-identity.md) | Tool names, namespaces, lookup, approvals, MCP naming, handoffs, or call IDs |
| [Tool execution lifecycle](tool-execution-lifecycle.md) | Function-tool planning, approvals, guardrails, concurrency, cancellation, timeouts, or failure conversion |
| [Local MCP server lifecycle](local-mcp-server-lifecycle.md) | Local MCP connection ownership, manager state, request serialization, caching, filtering, retries, or cleanup |
| [Model and provider boundaries](model-provider-boundaries.md) | Model resolution, provider adapters, feature capability, request conversion, terminal events, or retries |
| [Tracing lifecycle](tracing-lifecycle.md) | Trace and span context, processors, export, flush, shutdown, resume, or sensitive data |
| [Realtime session lifecycle](realtime-session-lifecycle.md) | Realtime listeners, connections, background tasks, handoffs, event iteration, or cleanup |
| [Realtime tracing architecture](realtime-tracing.md) | Realtime API server traces versus Agents SDK client traces |
| [Voice pipeline lifecycle](voice-pipeline-lifecycle.md) | VoicePipeline STT/workflow/TTS ownership, event and audio ordering, stream cleanup, PCM framing, or tracing |
| [Sandbox runtime boundary](sandbox-runtime-boundary.md) | Sandbox session ownership, preparation, resume state, manifests, materialization, or cleanup |
## Maintenance Rules
Keep each rule in the narrowest reference that owns it. Cross-link instead of copying detailed rules between files. Describe current architecture and compatibility boundaries, not the chronology of how a bug was found. Use source paths and durable public contracts as anchors, and remove or rewrite guidance when ownership moves.
@@ -0,0 +1,57 @@
# Agent Definition and Run Context
Use this reference for changes to public `Agent` fields, cloning, dynamic instructions, enabled tools or handoffs, output schemas, `RunContextWrapper`, `ToolContext`, usage aggregation, or the distinction between a public agent and an internal prepared clone.
## Public Definition and Cloning
- Exported `Agent` and `AgentBase` dataclass field order is a positional compatibility boundary. Append optional fields where possible and test old positional construction when the order changes.
- `Agent.__post_init__()` is the eager boundary for invalid field categories such as names, tools, handoffs, hooks, model settings, output types, and tool-use behavior. Dynamic callbacks are validated when invoked because their result depends on the current run.
- `Agent.clone()` uses `dataclasses.replace()` and is shallow. Mutable fields and contained tool, handoff, hook, and provider objects remain shared unless the caller explicitly supplies replacements.
- When `clone(model=...)` replaces a model whose settings still equal the old model's implicit defaults, recompute the new model's implicit defaults. Preserve explicitly customized `model_settings` instead of silently resetting them.
## Per-Turn Resolution
- Resolve dynamic instructions with the current context and public agent for each model turn. Enforce the documented two-argument callable shape and await async results.
- Evaluate callable `FunctionTool.is_enabled` and `Handoff.is_enabled` against the current run context. Do not cache a prior run's enabled set on the reusable agent.
- Use one resolved tool and handoff view for model exposure, reserved-name and collision checks, local dispatch, tracing, and Realtime session updates. Re-resolving independently at those surfaces can expose one set and execute another.
- An internal prepared agent may add bound tools, instructions, or sampling settings, but hooks, `ToolContext.agent`, handoff callbacks, and public results should identify the public agent unless an internal identity is explicitly part of the contract.
- The effective output schema belongs to the agent and model call that produced the candidate output. A handoff can change the final output type, so do not assume the starting agent's schema when parsing or typing the final result.
## Context Ownership
- Every agent, tool, handoff, guardrail, and lifecycle hook in one run must agree on the same application context type. The context object is local runtime state and is never added to model input automatically.
- A normal `ToolContext.from_agent_context()` shares the underlying application object, usage accumulator, and approval mapping with its parent while adding call-scoped fields such as call ID, namespace, arguments, and conversation history.
- Nested `Agent.as_tool()` execution has a separate run loop, approval scope, and resumable tool state. On the normal function-tool path it still shares the application object and usage accumulator, while `tool_input` belongs to the nested wrapper and must not overwrite the parent's scoped value.
- Sharing the application object is not the same as sharing every wrapper field. Add explicit application-level isolation when nested mutation is unsafe, and do not reuse parent approval decisions for nested calls merely because the tool name or call ID looks similar.
- Context serialization is a separate durability decision. Read [RunState schema and resume boundary](runstate-schema.md) before persisting custom context objects, approvals, usage, or nested tool input.
## Usage Accounting
- `RunContextWrapper.usage` is the run-wide mutable accumulator. Add each model response exactly once across streaming, non-streaming, retries, nested runs, handoffs, and resume paths.
- Preserve authoritative `request_usage_entries` when combining usage. Do not synthesize a second per-request entry from aggregate totals when the provider or retry layer already supplied request-level records.
- Retry accounting may include failed attempts with no token totals. Keep request count, aggregate tokens, request-level entries, and trace span usage internally consistent without inventing provider token data.
- Streamed usage remains incomplete until terminal chunks and the stream driver finish. Do not finalize billing, result summaries, or usage-bearing spans from the last visible text delta alone.
## Review Checklist
1. Test direct construction and clone behavior without mutating shared caller-owned objects.
2. Resolve dynamic instructions, tools, and handoffs through the same public agent and current context used for dispatch.
3. Verify handoff and internal prepared-agent paths expose the intended public identity and effective output schema.
4. Test nested agent tools for shared application state and isolated scoped metadata.
5. Compare aggregate and per-request usage after streaming, retries, handoffs, interruption resume, and nested runs.
## Sources
- `docs/agents.md`
- `docs/context.md`
- `docs/results.md`
- `src/agents/agent.py`
- `src/agents/run_context.py`
- `src/agents/tool_context.py`
- `src/agents/usage.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/run_internal/run_loop.py`
- `tests/test_agent_config.py`
- `tests/test_agent_clone_shallow_copy.py`
- `tests/test_agent_as_tool.py`
- `tests/test_usage.py`
@@ -0,0 +1,67 @@
# Conversation State Ownership
Use this reference for changes involving multi-turn input, sessions, `conversation_id`, `previous_response_id`, `auto_previous_response_id`, compaction, retries, `call_model_input_filter`, or `RunState` resume.
## Choose One Conversation Strategy
The state owner determines what the next model request should contain.
| Strategy | State owner | Next-turn input |
|---|---|---|
| Explicit replay with `result.to_input_list()` | Application | Replay-ready history plus the new turn |
| SDK session | Application storage plus the SDK | The same session plus the new turn |
| `conversation_id` | OpenAI Conversations API | The same conversation ID plus only the new turn |
| `previous_response_id` or `auto_previous_response_id` | OpenAI Responses API | The previous response ID plus only the new turn |
| `RunState` resume | Serialized Agents SDK run | Resume the same interrupted run; this is not a new conversation strategy |
In normal use, select one conversation strategy. Mixing client-managed replay or sessions with server-managed continuation can duplicate context unless the implementation explicitly reconciles both owners. Read [Session persistence](session-persistence.md) for the client-managed storage contract.
## Server-Managed Continuation
- `OpenAIServerConversationTracker` in `src/agents/run_internal/oai_conversation.py` owns delta calculation for `conversation_id`, `previous_response_id`, and `auto_previous_response_id`.
- Send only items that the server has not already acknowledged. Object identity is useful only within one process; resume and retry paths also require stable item IDs, tool call IDs, and content fingerprints.
- Update `previous_response_id` from the most recent response that actually has an ID. Do not erase a valid chain because an adjacent provider response lacks one.
- Session persistence cannot be combined with server-managed continuation. `validate_session_conversation_settings()` rejects a session with `conversation_id`, `previous_response_id`, or `auto_previous_response_id`; do not add a second history writer without defining reconciliation and dedupe semantics.
- Treat `conversation_id` and `previous_response_id` / `auto_previous_response_id` chaining as mutually exclusive state owners.
## Filters, Retries, and Resume
- `call_model_input_filter` runs on the prepared model payload. With server-managed continuation, that payload may already be a new-turn delta rather than full history.
- The filter must return `ModelInputData` with list input. Mark exactly the returned list as sent immediately before the request so nested preparation cannot add unsent items, rewind that tracking before retrying a failed request, and preserve it after success.
- Keep streaming and non-streaming tracker updates aligned. Both paths must preserve the same delta, retry, and response-ID semantics.
- Stateful retries require replay-safety evidence. Do not blindly resend a request that may already have advanced server state.
- `RunState` persists conversation identifiers and reconstructs tracker knowledge for resumed runs. Resume must not replay acknowledged input, lose unsent tool outputs, or increment the turn count without a model call.
- Conversation continuation carries context into a new turn. `RunState` resume continues a paused run. Do not substitute one mechanism for the other.
## Compaction
- `compaction_mode="previous_response_id"` depends on a usable stored response chain.
- `compaction_mode="input"` rebuilds from client-held items and is the fallback when the server chain is unavailable or `store=False` prevents later response lookup.
- Compaction must preserve the chosen state owner. Do not compact from local history and then also replay that history through server-managed continuation.
## Handoffs
- Server-managed conversations send deltas, so handoff input filters are not supported. `Handoff.input_filter` and `RunConfig.handoff_input_filter` should raise instead of rewriting a history the server already owns.
- `nest_handoff_history` is a client-history transformation. When server-managed continuation is active, disable it with a warning and continue with delta-only input.
- Keep generated items and session items distinct during handoff processing. The next model input may be filtered, but session history needs the full unfiltered item sequence when client-managed sessions are active.
## Review Checklist
1. Name the state owner before changing request construction.
2. Specify whether the model receives full history or a delta on every affected path.
3. Verify first turn, follow-up turn, retry, interruption, serialized resume, and streaming behavior.
4. Test tool calls and outputs separately; call IDs and output fingerprints have different dedupe roles.
5. Confirm that filtering, compaction, and session persistence do not introduce a second source of truth.
## Sources
- [OpenAI conversation state guide](https://developers.openai.com/api/docs/guides/conversation-state)
- [OpenAI running agents guide](https://developers.openai.com/api/docs/guides/agents/running-agents#choose-one-conversation-strategy)
- `src/agents/run_internal/oai_conversation.py`
- `src/agents/run_internal/run_loop.py`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_state.py`
- `docs/running_agents.md`
- `docs/sessions/index.md`
Recheck the official API reference with `$openai-knowledge` before changing server-managed continuation behavior.
@@ -0,0 +1,50 @@
# Function and Output Schema
Use this reference for changes to function-tool signature inspection, parameter metadata, strict JSON schema conversion, tool argument reconstruction, or structured agent output schemas.
Schema behavior is a compatibility boundary shared by Python callables, Pydantic, model providers, and runtime validation. Keep schema generation and invocation aligned rather than fixing one representation in isolation.
## Function Schema Ownership
- Explicit decorator arguments for a function name or description override values inferred from the callable and its docstring.
- Parameter descriptions from parsed docstrings take precedence over description strings carried by `Annotated`. Preserve `Field` constraints, aliases, and defaults when merging `Annotated` metadata.
- A run context parameter is special only in the first parameter position. Exclude it from the model-visible schema while still supplying it during invocation; do not silently treat later context-typed parameters as injected context.
- Keep the inspected signature, generated Pydantic model, JSON schema, and `to_call_args()` reconstruction consistent. Cover positional-only parameters, keyword-only parameters, `*args`, and `**kwargs` when changing this path.
- Reject unsupported callable shapes or invalid schemas when the tool is constructed so failures do not depend on whether a particular model later selects the tool.
## Strict JSON Schema Conversion
- Strict conversion closes object schemas with `additionalProperties: false` and marks their declared properties required. Reject an explicit `additionalProperties: true` instead of silently changing its meaning.
- Preserve the meaning of unions, intersections, definitions, and references. Normalize `oneOf` where required, process `allOf`, retain chained references, and merge a referenced schema with sibling keys without discarding the siblings.
- Remove defaults that only encode Python `None`; a nullable type must remain represented by its type schema rather than by an unsupported default.
- `ensure_strict_json_schema()` may mutate a non-empty input dictionary. Copy caller-owned schemas at public boundaries before conversion. Empty-schema conversion must return a fresh object rather than shared mutable state.
- Keep strictness explicit. If a tool or output schema opts out of strict mode, preserve that choice through provider conversion instead of partially applying strict normalization.
## Structured Output Schemas
- Plain `str` output and no declared output type use the plain-text path. Pydantic models and dictionary-shaped outputs expose their object schema directly; other Python types use the SDK's wrapper object with the `response` key.
- Keep generated output names stable and descriptive for nested generics, unions, and `Literal` types. These names are observable in provider requests and diagnostics.
- Parse model output as JSON and validate it through the output type adapter. Convert JSON or validation failures to the SDK's model-behavior error boundary rather than leaking provider- or Pydantic-specific exceptions.
- Streaming and non-streaming adapters must carry the same schema, strictness flag, wrapper behavior, and validation result.
## Review Checklist
1. Test precedence among explicit metadata, docstrings, `Annotated`, and `Field` values.
2. Test invocation reconstruction for positional-only, keyword-only, variadic, and context-bearing callables.
3. Test nested objects, unions, intersections, sibling and chained references, nullable fields, and caller-owned schema mutation.
4. Test plain text, direct object output, wrapped scalar or generic output, invalid JSON, and validation failure.
5. Verify every provider adapter receives the same normalized schema and strictness decision.
## Sources
- `src/agents/function_schema.py`
- `src/agents/strict_schema.py`
- `src/agents/tool.py`
- `src/agents/agent_output.py`
- `src/agents/models/`
- `tests/test_function_schema.py`
- `tests/test_function_tool_decorator.py`
- `tests/test_strict_schema.py`
- `tests/test_strict_schema_oneof.py`
- `tests/test_output_tool.py`
- `tests/models/`
@@ -0,0 +1,58 @@
# Local MCP Server Lifecycle
Use this reference for changes to Python-managed MCP servers, `MCPServerManager`, client-session request ordering, tool caching or filtering, local MCP retries, cancellation, or cleanup. Hosted MCP is a provider tool and follows the OpenAI API contract; use `$openai-knowledge` for that protocol surface. Read [Tool identity and routing](tool-identity.md) for server-prefixed names and [Tool execution lifecycle](tool-execution-lifecycle.md) for approval and invocation behavior after an MCP tool is converted to a `FunctionTool`.
## Connection Ownership and Task Affinity
- A local `MCPServer` owns its transport, `ClientSession`, and `AsyncExitStack` from `connect()` through `cleanup()`. Partial connection failure still requires closing every context already entered.
- Some MCP transports use AnyIO cancel scopes that require connection and cleanup in the same task. Do not wrap either operation in a helper that silently creates another task.
- `MCPServerManager` preserves task affinity in sequential mode and uses one long-lived worker task per server in parallel mode. Timeouts must run inside that owning task; on Python versions without `asyncio.timeout()`, cancel the current worker task and translate only timer-originated cancellation to `TimeoutError`.
- `MCPServerManager` lifecycle timeouts are validated during construction and assignment. They accept positive finite seconds or `None` to disable the timeout. Reject zero rather than relying on `asyncio.timeout(0)`, whose immediate deadline can depend on whether the lifecycle coroutine yields control. Parallel workers receive the current timeout with each command instead of retaining a stale snapshot.
- `client_session_timeout_seconds` uses positive finite values representable by `datetime.timedelta` and at least one microsecond for MCP read timeouts. Both `None` and zero disable that timeout; reject other values during server construction instead of passing an immediate or invalid deadline to `ClientSession`.
- Cleanup runs servers in reverse order and continues across ordinary cleanup failures. Cancellation suppression is an explicit manager policy; do not accidentally convert unrelated `BaseException` failures into recoverable connection errors.
- Server cleanup must clear session and transport-visible state even when exit-stack cleanup raises, so the same server object can reconnect without exposing stale session handles or workers.
## Manager State
- Keep configured servers, connected servers, failed servers, active servers, and per-server errors as distinct views. `active_servers` is the agent-facing list; with `drop_failed_servers=True` it excludes failed connections while preserving configured order.
- Non-strict connection records failures and continues with the connected subset. Strict connection cleans up work started by the failed attempt and restores the previous coherent active state before raising.
- `reconnect(failed_only=True)` retries the deduplicated failed set without disturbing healthy connections. A full reconnect cleans up all servers first and rebuilds manager state.
- Parallel connection still needs deterministic per-server state and complete cleanup after cancellation or one hard failure. Do not let completion order decide `active_servers`, `failed_servers`, or which workers remain registered.
## Shared Session Requests and Retries
- Streamable HTTP can require requests on one shared MCP session to be serialized. The same lock must cover tool calls, tool listing, prompts, and resource operations that share that session; serializing only `call_tool()` still permits sibling cancellation and protocol races.
- Preserve outer cancellation. A cancelled shared request may qualify for an isolated-session retry only when the transport identifies it as an inner or transient session failure and retry budget remains.
- Isolated-session retries are transport-specific recovery. Count isolated session setup and execution against the same retry budget, retry only the supported transient failure shapes, and never replay mixed exception groups or ordinary 4xx failures as if they were safe.
- Generic `list_tools()` and `call_tool()` retries use the configured attempt count and backoff. Validate required arguments locally before starting retries so deterministic input errors never reach the server or consume retry budget.
- MCP tool failure conversion follows the effective server or agent `failure_error_function`. Explicit `None` means propagate; cancellation of the parent run must not become model-visible tool failure output.
## Tool Discovery, Cache, and Filtering
- The unfiltered server tool list is the cacheable value. Apply static or dynamic filters to a copy for each requesting agent and run context; never let one request's filtered or merged metadata mutate the shared cache.
- `cache_tools_list=True` assumes server schemas are stable until `invalidate_tools_cache()` marks them dirty. Connection or filter changes must not accidentally make a stale filtered list authoritative.
- Dynamic filters require both `run_context` and agent. A filter exception excludes that tool and logs the failure rather than exposing it by default.
- Schema conversion to strict form is best effort and must not mutate the MCP server's original input schema. If strict conversion fails, preserve the original schema and keep metadata isolated per converted `FunctionTool`.
- Tool list collision errors, prefixed-name generation, and approval policy validation must be deterministic regardless of server response or connection completion order.
## Review Checklist
1. Identify the task that owns connect, every request, timeout cancellation, and cleanup for each transport.
2. Test partial connect failure, strict and non-strict manager modes, reconnect, repeated cleanup, and manager cancellation.
3. Test overlapping tool, prompt, and resource requests when shared-session serialization is enabled.
4. Prove retries preserve outer cancellation, consume one budget, and do not replay deterministic or unsupported failures.
5. Test cache invalidation, context-dependent filters, schema immutability, duplicate names, and reconnect with the public runner path.
## Sources
- `docs/mcp.md`
- `src/agents/mcp/server.py`
- `src/agents/mcp/manager.py`
- `src/agents/mcp/util.py`
- `tests/mcp/test_mcp_server_manager.py`
- `tests/mcp/test_connect_disconnect.py`
- `tests/mcp/test_client_session_retries.py`
- `tests/mcp/test_caching.py`
- `tests/mcp/test_tool_filtering.py`
- `tests/mcp/test_server_errors.py`
- `tests/mcp/test_runner_calls_mcp.py`
@@ -0,0 +1,84 @@
# Model and Provider Boundaries
Use this reference for changes to model resolution, `ModelSettings`, provider adapters, Responses versus Chat Completions behavior, request conversion, streaming terminal events, transport reuse, or model retries.
## Core Boundary
The run loop depends on the `Model` interface, not on one provider's request or response schema.
- `Model.get_response()` returns a normalized `ModelResponse`.
- `Model.stream_response()` yields normalized response stream events while preserving provider payloads needed by public raw-event consumers.
- `ModelProvider.get_model()` resolves names to model implementations and owns provider-level caches or connections.
- `Model.close()` and `ModelProvider.aclose()` release persistent transport resources when an implementation owns them.
Provider adapters own request construction, provider feature validation, terminal event interpretation, usage conversion, and translation into SDK item shapes. Keep provider-specific branching out of the core run loop unless it represents a shared SDK contract.
## Model and Settings Resolution
- An explicit `RunConfig.model` overrides the agent model. A model instance is used directly; a model name is resolved through the configured `ModelProvider`.
- Implicit default settings must follow the resolved model name, including when a run-level model name replaces the agent default.
- Resolve agent settings with run-level settings by overlaying non-`None` values. Preserve the documented merge behavior for structured fields such as `extra_args` and retry settings.
- Do not pass provider request extras into tracing by default. `ModelSettings.to_traceable_dict()` is the boundary for settings considered safe and meaningful in traces.
## Capability Ownership
Do not infer that a feature available in one adapter is supported by every `Model` implementation.
- Responses-specific features include server-managed response chaining, conversation-aware request fields, tool namespaces, deferred tool loading, tool search, response includes, compaction, and Responses websocket transport.
- Chat Completions generally requires client-managed replay and adapter conversion of Responses-compatible SDK items. Unsupported server-state or tool features should be rejected or explicitly ignored according to the adapter's documented validation mode.
- Realtime has its own session protocol, event model, and server tracing. Do not route Realtime behavior through the standard Responses or Chat Completions assumptions.
- Third-party model adapters may preserve only the shared `Model` contract. New provider-specific fields need an explicit conversion and fallback policy.
Validate capabilities at the adapter boundary where the resolved model and complete request are known. Avoid public flags that appear accepted by the SDK but are silently dropped before the provider request.
## Provider Validation and Error Ownership
Do not duplicate provider-side request validation in the SDK merely to fail earlier. When the provider already rejects an invalid value with an actionable error, preserve that single source of truth instead of copying provider grammar, length limits, enum membership, or other request constraints into SDK runtime code. Duplicated validation can drift as provider contracts evolve, can reject values accepted by another provider, and can turn a provider-neutral SDK type into an accidental provider-specific contract.
Add SDK-side validation only when it enforces an SDK-owned invariant or prevents a concrete risk that provider validation cannot address. Examples include ambiguous local routing, collisions before request serialization, invalid persisted state, unsafe local side effects, or a provider error that cannot identify the offending SDK input. A generic preference for earlier failure or a different error message is not sufficient.
When local validation is justified and the constraint is provider-specific, keep it at the owning adapter boundary and derive it from an authoritative provider contract. Do not apply it to shared `Model` interfaces, provider-neutral tool types, or third-party adapters. Tests should distinguish the SDK-owned invariant from values that are intentionally left for the provider to validate.
## Provider Data and Terminal Semantics
- Preserve provider-supplied string IDs, request IDs, usage, and opaque provider data when the public SDK contract exposes them.
- Normalize provider objects and mapping payloads without relying on truthiness for valid empty or zero values. When a field intentionally treats zero like `None`, make that field-specific contract explicit in the normalization, documentation, and tests rather than applying a generic optional-number rule.
- A transport stream ending is not automatically a successful model response. Responses `failed` and `incomplete` terminals, explicit error events, and a missing terminal payload must produce the documented failure behavior in both HTTP and websocket paths.
- Keep semantically equivalent HTTP, websocket, streaming, and non-streaming paths aligned on final `ModelResponse`, errors, request IDs, and usage.
## Transport Resource Ownership
- Persistent Responses websocket models are loop-bound resources. Cache reusable websocket model instances by running event loop and model name; do not share one connection or `asyncio.Lock` across loops.
- Use weak loop ownership so an unused cache does not keep a closed event loop alive. When a live connection itself pins a closed loop, prune it with synchronous abort and state clearing rather than awaiting work on that closed loop.
- A provider that caches persistent models must make `aclose()` close every unique cached model and clear its caches. Close on a still-running owner loop when possible; do not drive an inactive foreign loop inside `asyncio.to_thread()`.
- A model used without a running loop cannot safely join the loop-scoped websocket cache. Preserve the non-reuse fallback rather than attaching it to an arbitrary global loop.
- Connection reuse ends after protocol errors, pre-terminal disconnects, cancellation that invalidates framing, or explicit close. Clear connection and loop-bound lock state together so a later request cannot reuse half-closed transport state.
## Retry and Replay Safety
- Provider retry advice can describe retryability, delay, and replay safety; the runner must not replace provider-specific evidence with a generic status-code assumption.
- Requests that use server-managed conversation state or may have produced side effects are not automatically replay-safe. A retry policy must account for whether the provider could have accepted the previous attempt.
- Retry conversion and error handlers must preserve the original exception semantics and avoid leaking sensitive request payloads through chaining, logs, traces, or provider error objects.
## Review Checklist
1. Identify which adapter owns the feature and how unsupported adapters behave.
2. Before adding validation, determine whether it protects an SDK-owned invariant or only duplicates an actionable provider error.
3. Verify model and implicit-settings resolution when run config overrides the agent.
4. Compare HTTP/websocket and streaming/non-streaming terminal behavior when applicable.
5. Preserve request IDs, usage, provider data, and error semantics through normalization.
6. Prove retries are safe for the request's state ownership and side effects.
7. Test transport reuse, cross-loop access, closed-loop pruning, and provider shutdown when persistent connections are involved.
## Sources
- `src/agents/models/interface.py`
- `src/agents/model_settings.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/models/openai_responses.py`
- `src/agents/models/openai_chatcompletions.py`
- `src/agents/models/multi_provider.py`
- `src/agents/models/_response_terminal.py`
- `src/agents/run_internal/model_retry.py`
- `tests/models/`
- `tests/test_config.py`
@@ -0,0 +1,73 @@
# Realtime Session Lifecycle
Use this reference for `RealtimeSession` changes involving entry, exit, listeners, connections, background tasks, approvals, handoffs, event iteration, tracing context, or cleanup.
## Resource Ownership
Treat the session as the owner of these resources once they are acquired:
| Resource | Acquisition | Required release or terminal state |
|---|---|---|
| Model listener | `add_listener()` during entry | `remove_listener()` |
| Model connection | `model.connect()` | `model.close()` |
| Event iterators | Waiting on the event queue | Wake or terminate every waiter on close |
| Guardrail tasks | Created during output processing | Complete, or cancel and account for completion |
| Tool-call tasks | Created when `async_tool_calls=True` | Complete, or cancel and account for completion |
| Pending approvals and outputs | Added during tool execution | Resolve, retain for retry, or clear during terminal cleanup |
| Agent and model settings | Updated on handoff or `update_agent()` | Keep runtime state and model configuration aligned |
Do not add a new side effect before a failure point without defining who releases it.
## Entry and Exit
- Python does not call `__aexit__` when `__aenter__` raises. Any listener, connection, task, tracing scope, or other resource acquired before the exception needs explicit failure cleanup.
- Keep construction free of external side effects. Acquire listeners and connections during entry where failures can be handled coherently.
- `close()` and internal cleanup must be idempotent. Repeated close paths should still wake event iterators without closing the model twice.
- Mark the session closed only after the cleanup state is coherent. If model close fails, decide deliberately whether retry is possible and which resources remain owned.
## Async Task and Context Rules
- `asyncio` tasks inherit a snapshot of the creator's context. A background task cannot update the caller task's `ContextVar` state.
- A `ContextVar` token must be reset in the same context that created it. Never pass a token to a different task and assume cleanup can reset it safely.
- Shared session fields can be mutated by the listener path, tool-call tasks, `close()`, `update_agent()`, and handoff handling. Review ordering and races whenever one of those paths changes.
- Calling `task.cancel()` requests cancellation; it does not prove the task has finished its `finally` blocks or released resources. Await cancelled tasks when completion matters, or document and test why dropping them is safe.
- Background-task exceptions must reach a deterministic owner. They must not silently disappear or leave event consumers blocked.
## Agent Transitions
- Handoffs and the public `update_agent()` API are equivalent agent-transition surfaces. Keep their model settings, tool and handoff resolution, emitted events, and tracing metadata aligned unless a difference is intentional and documented.
- Resolve dynamic tools and enabled handoffs once per transition when possible, then reuse the exact resolved values for model settings and metadata.
- With concurrent tool calls, capture the agent snapshot associated with each call. Do not route a call through whichever agent happens to be current when the task eventually runs.
## Guardrails and Response Ordering
- Realtime output guardrails inspect accumulated transcript text at configured debounce thresholds, not each token and not a final `Runner` output object. They emit `guardrail_tripped` instead of raising a normal Runner tripwire exception.
- A tripped output guardrail marks the response interrupted before awaiting transport work, emits one trip event per response, forces response cancellation, and sends safe follow-up input naming the guardrail. Concurrent guardrail tasks must not interrupt or message the same response twice.
- Guardrail callbacks can run after audio has already been buffered or played. Consumers must treat `audio_interrupted` as the signal to stop local playback; text rejection alone cannot retract audio already delivered.
- An exception from one output guardrail is logged and skipped so it does not silently terminate the live session. Exceptions that escape the background guardrail task must become a `RealtimeError` event rather than disappearing.
- Realtime function-tool input guardrails follow the same optional pre-approval and mandatory post-approval ordering as standard function tools, but their rejection is returned through Realtime tool output and events.
- Follow-up `response.create` work triggered by tools, handoffs, or guardrails must respect the active response lifecycle. Wait for `response.done` or the model layer's equivalent gate before starting a conflicting response.
## Failure-Path Tests
Add focused tests for affected phases:
1. Instruction, tool, or handoff resolution fails during entry.
2. Model connection fails after listener registration.
3. A background tool or guardrail task raises or is cancelled.
4. Cleanup runs while event iterators are waiting.
5. `close()` is called repeatedly or from another task.
6. A handoff or `update_agent()` fails partway through model-settings application.
7. Tool output sending fails after local execution and must be retried without running the tool twice.
8. Concurrent guardrail tasks trip once, cancel playback, and do not overlap follow-up responses.
Verify lifecycle changes with the real public path where feasible; helper-only tests are insufficient when task ownership or context propagation determines the result.
## Sources
- `src/agents/realtime/session.py`
- `src/agents/realtime/model.py`
- `src/agents/realtime/openai_realtime.py`
- `tests/realtime/test_session.py`
- `tests/realtime/test_session_exceptions.py`
- `docs/realtime/guide.md`
+42
View File
@@ -0,0 +1,42 @@
# Realtime Tracing Architecture
Use this reference when reviewing or implementing Realtime tracing behavior, especially claims that `RealtimeSession` should emit the same trace hierarchy as `Runner`.
## Two Separate Tracing Systems
Realtime integrations involve two independent tracing paths:
| Path | Owner | Configuration | Result |
|---|---|---|---|
| Realtime API server tracing | Realtime API | `"auto"`, `workflow_name`, `group_id`, and `metadata` | The server creates a Realtime session trace in the Traces Dashboard. |
| Agents SDK client tracing | Agents SDK tracing provider | `trace()`, `agent_span()`, and other SDK span factories | The SDK exports locally created traces and spans through its tracing processor. |
The current Python SDK has no mapping from an Agents SDK client `trace_id`, `span_id`, or parent context into `RealtimeModelTracingConfig` or the model's `session.update`. A server-created Realtime trace is therefore not attached as a child of an SDK-created trace or span by this implementation. Likewise, adding an SDK `agent_span()` around `RealtimeSession` does not make server-side trace contents children of that span.
If both paths are enabled, the dashboard can contain two separate traces. A shared `group_id` can make them easier to filter and correlate, but it does not merge them or create a parent-child relationship.
## Current Python SDK Behavior
- `RealtimeModelTracingConfig` exposes only `workflow_name`, `group_id`, and `metadata` in `src/agents/realtime/config.py`.
- `OpenAIRealtimeWebSocketModel` defaults the Realtime tracing configuration to `"auto"` when the caller does not provide one.
- After receiving `session.created`, the model sends the tracing configuration through a `session.update` event.
- `RealtimeRunConfig.tracing_disabled` prevents the SDK from enabling Realtime tracing for that session.
Verify these paths in `src/agents/realtime/openai_realtime.py` and `src/agents/realtime/session.py`; do not rely on old issue descriptions because Realtime tracing support has changed over time.
## Maintainer Constraints
1. Identify whether the behavior belongs to the Realtime API's server trace or an Agents SDK client trace created with `trace()`.
2. A client-side agent span does not repair missing server tracing and does not create the unified hierarchy produced by `Runner`.
3. The current Python SDK cannot place server-created Realtime spans under an SDK-created trace or span because it does not carry client trace parentage through the Realtime tracing configuration. Recheck the live protocol with `$openai-knowledge` before treating that implementation gap as permanent.
4. Use the server trace for Realtime model activity. Use a shared `group_id` or metadata when correlation with a client trace is required.
5. Parallel SDK spans need an explicit product and maintenance contract covering the dual-trace user experience, async task context, handoff parenting, failure cleanup, and the client-only operations represented by those spans.
6. A client trace becoming non-empty is not evidence that Realtime server activity has been captured or parented correctly.
## Sources
- `src/agents/realtime/config.py`
- `src/agents/realtime/openai_realtime.py`
- `src/agents/realtime/session.py`
Recheck the official API reference with `$openai-knowledge` before changing this guidance or implementing new protocol behavior.
+79
View File
@@ -0,0 +1,79 @@
# Run Item Lifecycle
Use this reference for changes to model output processing, `RunItem` types, tool call and output items, stream events, replay conversion, session history, or serialized run state.
## Item Flow
The runtime carries one semantic item through several representations:
1. A model adapter returns provider output in `ModelResponse.output`.
2. `process_model_response()` converts recognized output into public `RunItem` objects and internal executable tool-run records in `ProcessedResponse`.
3. Tool execution and handoffs add output items and choose a `SingleStepResult.next_step`.
4. The resulting items feed `RunResult`, semantic stream events, session persistence, tracing, and `RunState` serialization.
5. Replayable items convert back to model input through `RunItem.to_input_item()` or `run_item_to_input_item()` after SDK-only metadata is handled.
Keep provider payloads, public run items, and internal execution records distinct. A provider item may be observable without requiring local execution, while a local tool-run record may need to preserve the selected SDK tool object and routing identity.
## Generated, Session, and Model Input Views
- `new_step_items` describes items generated by the current step.
- `session_step_items` preserves the full unfiltered sequence when session history must retain items that a handoff or input filter omitted from the next model request.
- `generated_items` is the public observability view and prefers `session_step_items` when present.
- Model input is a replay view, not the canonical storage view. Approval placeholders, SDK-only metadata, unsupported IDs, and orphaned calls may need filtering or normalization before an API request.
Do not force these views into one list. History persistence, user-visible results, and the next provider request have different correctness requirements.
## Adding or Changing an Item Type
Update every applicable surface together:
- `src/agents/items.py` for the public `RunItem` type, accessors, and replay conversion.
- `src/agents/run_internal/run_steps.py` for processed response and executable tool-run records.
- `src/agents/run_internal/turn_resolution.py` for provider output recognition, item creation, side effects, and next-step selection.
- `src/agents/run_internal/tool_execution.py`, `tool_actions.py`, or `tool_planning.py` for execution, dedupe, approvals, and outputs.
- `src/agents/run_internal/items.py` for normalization, replay conversion, fingerprints, dedupe, and provider-boundary metadata stripping.
- `src/agents/stream_events.py` and streaming queue helpers for public semantic events.
- `src/agents/run_state.py` for serialization and deserialization when the item can survive interruption.
- `src/agents/run_internal/session_persistence.py` for session conversion, sanitization, and retry accounting.
- Tracing and usage conversion when the item contributes observable tool or model work.
## Compatibility Rules
- Public stream event names are compatibility-sensitive. Do not rename an existing event, even to fix spelling, without an explicit breaking-change plan.
- Preserve provider-supplied IDs and opaque provider data until the owning boundary deliberately removes them. Do not invent IDs or coerce malformed values to make replay appear valid.
- Preserve SDK-only metadata needed for display, routing, approvals, tool origin, or resume, but strip it before sending payloads to a provider that does not accept it.
- Tool call and output pairs must retain the same string call ID across execution, replay, session persistence, and resume.
- Empty, falsey, structured, image, file, and custom tool outputs are valid values unless the public tool contract explicitly rejects them; do not use broad truthiness checks to decide whether output exists.
## Replay Integrity
- Prune orphan calls only from runner-generated or resumed history where the SDK owns call/output pairing. Preserve caller-supplied initial input unless an explicit public normalization contract says otherwise.
- When dropping an orphan tool call, also drop reasoning items tied to that removed call so the provider does not receive a reasoning item without its required following item. Do not drop a lone reasoning item merely because its following item is absent locally; server-managed conversation state may own that item.
- `reasoning_item_id_policy="omit"` strips IDs only from SDK-generated follow-up reasoning items. It does not rewrite initial caller input, must survive `RunState` resume, and can be superseded by a later `call_model_input_filter` that deliberately returns IDs.
- Pair anonymous tool-search outputs with the latest compatible anonymous call and never pair a named call with an anonymous output. A missing call ID does not justify inventing a persistent provider identity.
- `provider_data` and provider IDs have boundary-specific ownership. Preserve them for raw results and provider requests that accept them, but strip private or replay-unsafe metadata from session and server-conversation history where the SDK contract requires sanitized items.
## Review Checklist
1. Follow the item from provider response through result, stream, session, replay, and `RunState`.
2. Test both typed provider objects and mapping payloads when adapters support both.
3. Verify IDs, metadata, and output values survive every required round-trip.
4. Test filtering and dedupe without losing the latest valid call/output pair.
5. Compare streaming event order with the non-streaming item sequence.
6. Test orphan pruning and reasoning pairing with client-managed replay and server-managed continuation separately.
## Sources
- `src/agents/items.py`
- `src/agents/stream_events.py`
- `src/agents/run_internal/items.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/turn_resolution.py`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_state.py`
- `tests/test_items_helpers.py`
- `tests/test_run_internal_items.py`
- `tests/test_stream_events.py`
- `tests/test_run_state.py`
- `docs/running_agents.md`
- `docs/results.md`
+67
View File
@@ -0,0 +1,67 @@
# Runner Lifecycle
Use this reference for changes to `Runner`, turn accounting, guardrails, hooks, handoffs, interruptions, cancellation, or streaming and non-streaming behavior.
## Turn Boundary
A turn is one logical model invocation plus processing of that response. Tool execution, handoff resolution, session persistence, interruption resume, and retries inside that logical invocation do not independently consume turns.
- Increment the turn counter exactly once when the run loop starts a logical model turn. Transport or provider retries inside `get_new_response()` remain part of that turn.
- A handoff changes the current agent, but the next turn begins only when the new agent invokes a model.
- Resuming `NextStepInterruption` continues the paused turn. Resolve stored approvals and tool work before deciding whether another model call is needed.
- Preserve `max_turns` and the current turn in `RunState`; resume must not reset the budget or charge a turn twice.
## Guardrail Ordering
- Input guardrails belong to the starting agent and run only for the initial user input. Do not rerun them after handoffs or when resuming an interruption.
- Sequential input guardrails must finish before model-side effects begin. Parallel input guardrails may overlap the model call, so a tripwire or exception must cancel and await the in-flight model task and sibling guardrail tasks.
- Tool input guardrails run before the approved tool side effect. Tool output guardrails run after local execution and before the output is accepted into the next step.
- Output guardrails run only after a candidate final output exists. Streaming must await them and preserve the same tripwire and exception behavior as non-streaming execution before declaring completion.
- Guardrail results are observable run state. Preserve them across handoffs, error handlers, streamed completion, and `RunState` round-trips.
## Step State Machine
`SingleStepResult.next_step` is the control boundary after one model response and its local side effects:
| Step | Meaning |
|---|---|
| `NextStepRunAgain` | Continue with the current agent and make another model call |
| `NextStepHandoff` | Switch the current agent, emit the transition, then continue |
| `NextStepFinalOutput` | A final candidate exists; finish terminal hooks, output guardrails, persistence, and result construction |
| `NextStepInterruption` | Persist enough processed state to resume pending approvals without rerunning completed work |
Do not bypass this state machine with path-local completion logic. New terminal or pausable behavior must define non-streaming, streaming, session, tracing, and serialized-resume semantics.
## Streaming Parity and Cancellation
- Streaming and non-streaming paths must produce equivalent final output, generated items, current agent, usage, guardrail results, session history, and interruption state for the same model behavior.
- Raw transport events may differ, but semantic `RunItemStreamEvent` and `AgentUpdatedStreamEvent` emission must follow the same processed items and agent transitions used by the non-streaming result.
- `stream_events()` is the stream driver's cleanup boundary. Keep consuming it until exhaustion after normal completion or `cancel()`, or explicitly close the async iterator; merely breaking after the last visible token does not prove session writes, guardrails, compaction, sandbox cleanup, usage, or terminal errors have settled.
- Immediate cancellation marks the result complete and requests task cancellation. `after_turn` cancellation leaves the current model/tool turn running so it can persist state and usage before the next turn. Preserve this distinction instead of treating both modes as queue shutdown.
- Terminal run-loop, guardrail, and max-turn errors must be surfaced from `stream_events()` after the required queued events are handled. Preserve `run_loop_exception` as a diagnostic view of the background task, not as a replacement completion primitive.
- `task.cancel()` is a request, not cleanup completion. Await cancelled tasks when their `finally` blocks, exceptions, or owned resources affect run correctness.
- Keep lifecycle hooks aligned across both paths, especially model start/end, handoff, tool start/end, and final-output hooks.
## Review Checklist
1. Identify which turn and which agent own the behavior.
2. Trace every `NextStep` outcome, including interruption resume.
3. Compare streaming and non-streaming side effects and terminal ordering.
4. Test guardrail tripwires and exceptions in sequential and parallel modes when relevant.
5. Verify normal exhaustion, explicit iterator close, immediate cancellation, and after-turn cancellation leave the documented result and owned resources in a coherent state.
## Sources
- `src/agents/run.py`
- `src/agents/run_internal/run_loop.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/turn_preparation.py`
- `src/agents/run_internal/turn_resolution.py`
- `src/agents/run_internal/guardrails.py`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
- `tests/test_cancel_streaming.py`
- `tests/test_guardrails.py`
- `tests/test_run_state.py`
- `docs/streaming.md`
- `docs/results.md`
+64
View File
@@ -0,0 +1,64 @@
# RunState Schema and Resume Boundary
Use this reference for changes involving `RunState` serialization, deserialization, approvals, trace state, sandbox state, agent identity, tool output payloads, or any persisted resume data.
## Compatibility Boundary
`RunState` is the durable SDK pause/resume boundary. Treat the serialized JSON shape as compatibility-sensitive once a schema version has shipped in a release.
- `to_json()` always emits `CURRENT_SCHEMA_VERSION`.
- `from_json()` must continue reading every version in `SUPPORTED_SCHEMA_VERSIONS`.
- Older SDKs intentionally reject newer or unsupported versions rather than attempting forward compatibility.
- Unreleased schema versions may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported.
- Every supported version must have a non-empty one-line entry in `SCHEMA_VERSION_SUMMARIES`.
## When to Bump the Schema
Bump `CURRENT_SCHEMA_VERSION` when a serialized `RunState` snapshot changes in a way that affects resume correctness or would silently lose data when read under an older schema label.
Examples include:
- New persisted fields on `RunState`, `ModelResponse`, `ProcessedResponse`, interruptions, approvals, tool outputs, sandbox state, trace state, or agent-owned state.
- New run item, tool call, approval, or output item variants that can appear in serialized state.
- New SDK-only metadata needed to route, dedupe, approve, retry, or resume a tool call.
- A changed meaning for an existing serialized field.
Do not rely on current-reader tests alone. Add a regression that rewrites `$schemaVersion` to an older supported label when appropriate and proves the old label is accepted, rejected, or migrated deliberately.
## Identity and Routing State
Serialized state must preserve enough identity to resume without changing behavior:
- Agent identity must distinguish duplicate agent names in the same graph.
- Function tools should persist canonical lookup keys, including `bare`, `namespaced`, and `deferred_top_level`.
- Tool call IDs must remain provider-supplied strings; do not coerce arbitrary values into IDs.
- Approval decisions and rejection messages must restore against the same tool identity and call ID they originally targeted.
- The per-agent tool-use tracker must preserve stable duplicate-agent identity so tool-choice reset behaves the same after resume.
- Server-managed conversation identifiers must restore into `OpenAIServerConversationTracker` without replaying acknowledged input.
## Context and Secrets
Context serialization is intentionally conservative.
- Mapping contexts can round-trip directly.
- Custom contexts need explicit serializers and deserializers when exact restoration matters.
- Without a safe serializer, snapshots may record metadata and warnings rather than the raw object.
- Do not persist secrets in `RunContextWrapper.context`, trace data, tool outputs, or custom data unless the caller explicitly chose that durability boundary.
## Review Checklist
1. Identify every serialized field whose shape or meaning changes.
2. Decide whether the affected schema version is released or unreleased.
3. Update `CURRENT_SCHEMA_VERSION` and `SCHEMA_VERSION_SUMMARIES` when resume compatibility requires it.
4. Keep released schema versions readable, or fail with an explicit compatibility error if the old label cannot safely represent the new data.
5. Test `to_json()` output, `from_json()` restoration, string round-trips, and resumed execution through the public `Runner.run(...)` or `Runner.run_streamed(...)` path.
## Sources
- `src/agents/run_state.py`
- `src/agents/result.py`
- `src/agents/run_internal/agent_runner_helpers.py`
- `src/agents/run_internal/oai_conversation.py`
- `src/agents/run_internal/run_steps.py`
- `src/agents/run_internal/tool_execution.py`
- `tests/test_run_state.py`
@@ -0,0 +1,84 @@
# Sandbox Runtime Boundary
Use this reference for changes to sandbox session ownership, `SandboxAgent` preparation, manifests, capabilities, host-path materialization, snapshots, resume state, agent transitions, or cleanup.
## Runtime Ownership
The outer `Runner` owns agent turns, approvals, handoffs, tracing, session history, and `RunState`. A sandbox session owns the execution environment, workspace, processes, mounts, and provider-specific connection state. Do not move one layer's lifecycle into the other without defining resume and cleanup behavior for both.
- A live `SandboxRunConfig.session` is caller-owned. The runner may configure and use it but must not delete or fully tear it down.
- A session created or resumed through `SandboxRunConfig.client` is runner-owned. Cleanup runs pre-stop hooks, persists snapshot-backed workspace state, stops and shuts down the session, deletes provider resources when required, and closes dependencies.
- Session cleanup must be idempotent and release acquired `SandboxAgent` concurrency guards even when persistence or provider cleanup fails.
- A `SandboxAgent` instance cannot be reused concurrently across runs because prepared capability tools and session state are bound to one live run. Clone or construct separate agents for concurrent work.
## Session Source and Saved State
Resolve the session source in this order: injected live session, resumable sandbox state carried by `RunState`, explicit `SandboxRunConfig.session_state`, then a newly created session. Manifest and snapshot inputs seed only a fresh session; they do not overwrite an injected or resumed workspace.
- `RunState` sandbox data and explicit `session_state` represent provider connection or session state used to reconnect to existing work.
- A snapshot represents saved workspace contents used to seed a new session. It is not interchangeable with provider session state.
- Preserve stable per-agent resume identity across handoffs, including graphs with duplicate agent names. Object identity is process-local, so serialized state needs stable keys and explicit current-agent selection.
- Serialize runner-owned sessions after stop-time persistence has completed so a later resume can reattach when the backend survives or reconstruct the workspace from the saved snapshot when it does not.
## Agent Preparation
- Clone capability instances per run before binding them to a live session. Reusing mutable capability objects can leak tools, sampling settings, or session references across runs.
- Validate capability dependencies before exposing tools. Capability tool construction, instruction fragments, input processing, and sampling adjustments must use the same effective capability set.
- Build instructions in the documented order: SDK sandbox base prompt or explicit replacement, agent instructions, capability instructions, remote-mount policy, then the rendered filesystem description.
- Bind capability tools to the live session and preserve a link from the prepared clone to the public `SandboxAgent`. Dynamic instructions and hooks should observe the public agent rather than an internal clone with implementation-only state.
- Handoffs stay in the outer run loop and select another agent-bound sandbox session. A nested `Agent.as_tool()` run owns its own nested runner and sandbox lifecycle.
## Filesystem Trust Boundary
- Manifest entry destinations are workspace-relative and must not escape the workspace. The workspace root itself must be absolute where the backend requires an absolute runtime root.
- Treat every path visible inside a sandbox as a POSIX path, regardless of the host operating system. Do not use `str(Path(...))` or `str(PurePath(...))` to produce, validate, compare, or serialize a sandbox path because those calls emit backslashes on Windows. Convert typed path objects with `PurePath.as_posix()` or the canonical helpers in `workspace_paths.py`.
- Preserve the trust distinction between typed path objects and raw string input. A native Windows `Path` or `PurePath` may be converted to its POSIX sandbox representation, while a raw string containing backslashes may still need to be rejected when the public contract requires explicit POSIX syntax. Do not make an input-validation failure disappear by silently canonicalizing every string.
- Keep host filesystem conversion at an explicit host/backend boundary. Code that resolves manifests, mount targets, archive exclusions, snapshots, grants, or provider paths must not let the host implementation of `Path` change the identity of a sandbox path.
- `LocalFile` and `LocalDir` sources are host-side inputs. Resolve them against a trusted base directory, require explicit application-controlled `extra_path_grants` outside that base, and reject untrusted manifests that try to authorize their own host access.
- Validate local sources at use time, not only when parsing the manifest. Defend against symlinked sources, parent-directory swaps, platform path aliases, and archive members that change meaning between validation and extraction.
- Archive extraction must reject traversal, unsafe links, and unsupported member types before writing, and enforce entry, byte, and expansion limits without materializing an unbounded member list.
- Extra path grants are runtime access, not durable workspace content. Snapshots and `persist_workspace()` include the workspace root, not arbitrary granted paths.
- Credentials for mounts or providers must remain in the owning adapter and must not appear in generated shell commands, model-visible errors, logs, or serialized sandbox state.
## Provider and Error Boundary
- Normalize backend failures to sandbox errors without discarding provider details needed for diagnosis. Preserve explicit retryability instead of inferring it later from a message string.
- Keep portable sandbox paths separate from host filesystem paths and provider identifiers. Conversion belongs in the backend or materialization boundary, not in agent-facing tools.
- Temporary clones, mounts, sinks, and dependency resources need failure cleanup during partial startup as well as normal shutdown.
- Capability tools should report bounded output and preserve provider exit status or structured error data without exposing private runtime metadata to the model.
## Remote Mount Simplicity Boundary
Remote mounts should default to one narrow lifecycle: declare them during sandbox creation, keep their contents outside workspace persistence, and unmount them during close. When tar persistence or hydration requires detaching a mount, restore it immediately afterward. Mount credentials must remain trusted live configuration and must not be reconstructed from serialized session state.
Treat dynamic mount mutation, native-snapshot-backed mounts, and resumable mounts as opt-in provider capabilities rather than default requirements. If a privileged mount transition becomes ambiguous, stop the sandbox instead of adding reconciliation or recovery state. Do not add credential resolvers, refresh loops, persisted mount registries, or dynamic mount APIs unless the provider exposes a trusted primitive that makes the lifecycle transition unambiguous and the change is supported by focused provider evidence.
Provider adapters may deliberately support a narrower lifecycle. Document that boundary next to the adapter state that enforces it so future maintainers do not mistake an intentional exclusion for an unfinished feature. The Vercel S3 adapter follows the create-time-only form of this policy: its trusted mount configuration is live-session-only, sessions containing mounts cannot resume, and mount topology cannot change after creation.
## Review Checklist
1. Name the owner of every live session, provider client, mount, process, capability, and temporary resource.
2. Test injected, resumed, explicit-state, snapshot-seeded, and fresh-session paths separately.
3. Verify handoffs, duplicate agent names, interruption resume, and cleanup failure preserve the intended session mapping.
4. Test host-path, symlink, traversal, archive-limit, and credential-redaction boundaries on applicable platforms.
5. Exercise the public `Runner` path so agent preparation, capability binding, persistence, and cleanup run together.
6. For every new sandbox-path validation, normalization, comparison, or serialization path, test a `PureWindowsPath` input on every host and confirm that raw backslash strings retain their intended validation behavior.
## Sources
- `docs/sandbox/guide.md`
- `docs/sandbox/clients.md`
- `src/agents/sandbox/runtime.py`
- `src/agents/sandbox/runtime_session_manager.py`
- `src/agents/sandbox/runtime_agent_preparation.py`
- `src/agents/sandbox/manifest.py`
- `src/agents/sandbox/materialization.py`
- `src/agents/sandbox/workspace_paths.py`
- `src/agents/sandbox/session/archive_extraction.py`
- `src/agents/extensions/sandbox/vercel/mounts.py`
- `src/agents/extensions/sandbox/vercel/sandbox.py`
- `tests/sandbox/test_runtime.py`
- `tests/sandbox/test_runtime_agent_preparation.py`
- `tests/sandbox/test_session_state_roundtrip.py`
- `tests/sandbox/test_materialization.py`
- `tests/sandbox/test_extract.py`
+80
View File
@@ -0,0 +1,80 @@
# Session Persistence
Use this reference for changes to client-managed sessions, session input callbacks, per-turn persistence, retries, rewind, compaction replacement, or session backend implementations.
Read [Conversation state ownership](conversation-state-ownership.md) first when server-managed continuation is also involved. A client-managed session is a history store; it is not a second owner for a server-managed conversation.
## Session Contract
- `get_items(limit=N)` returns the latest `N` items in chronological order.
- `add_items()` appends one logical batch. Backends should make the batch atomic so partial turns are not visible after failure.
- `pop_item()` removes the current tail item and is used only for guarded rollback of items the current run can prove it owns.
- `clear_session()` clears the session boundary; compaction decorators that replace history must provide stronger restore behavior around destructive replacement.
Third-party implementations target the `Session` protocol. Internal base classes and backend-specific metadata are not the compatibility contract unless explicitly documented.
## Backend Consistency
- An explicit `get_items(limit=N)` argument overrides the backend's default session limit. Return the latest `N` items in chronological order, with a deterministic tie-breaker when timestamps can collide.
- Preserve caller batch order. Persist the items and any indexes or structural metadata required to read them as one atomic operation. A failed batch must leave earlier history unchanged, and any backend-internal retry must not create duplicates.
- Serialize initialization and conflicting writes at the backend's actual consistency boundary. Concurrent first writers must not race, and cancellation or failure must not strand locks or transactions.
- Apply configured table or collection names and session settings consistently across reads, writes, deletes, metadata updates, and wrapper operations.
- For backends that deserialize stored records, a corrupt record must not hide valid history or cause unrelated records to be deleted. Define consistent `get_items()` and `pop_item()` behavior that isolates the bad record and continues safely.
- Preserve creation timestamps and advance update timestamps deliberately. Backend-only identifiers and metadata must not leak into model-facing session items.
- Close only resources the backend owns. An injected engine, client, or connection remains caller-owned unless the public contract explicitly transfers ownership.
## Preparing Input Versus Persisting Input
`prepare_input_with_session()` returns two different values: the normalized input for the next model request and the subset of new-turn items that should be appended to the session.
- Existing history must not be re-appended as new input, even when `session_input_callback` deep-copies, reorders, filters, duplicates, or reconstructs items.
- A callback may change the model view without rewriting already stored history.
- Handoff and model-input filters may omit items from the next request while `session_step_items` retains the complete unfiltered sequence for history and observability.
- Normalize and deduplicate the model request and persistence candidates through the same canonical item helpers, then apply boundary-specific sanitization.
## Per-Turn Save and Resume
- Persist each completed turn, not only the final run result. Tool outputs and handoff items must survive a later error or interruption.
- `_current_turn_persisted_item_count` tracks which generated items have already been saved during streaming, retry, or resume. Count items after conversion and persistence filtering, not from the unsanitized source list.
- Resuming an interruption must save newly produced approval and tool output items without duplicating inputs or previously persisted outputs.
- Preserve full session items separately from filtered model input when updating `RunState` after resume.
- A guardrail trip must preserve the accepted user input while excluding speculative assistant or tool work that the tripwire invalidated. Test sequential and parallel guardrails in streaming and non-streaming modes because their persistence timing differs even though the resulting history must remain coherent.
## Retry Rewind
Retry cleanup is ownership-sensitive and best effort.
- Rewind only an exact serialized suffix that belongs to the failed attempt. Never scan backward and delete merely similar historical items.
- Verify the complete suffix before popping. If a pop fails or returns an unexpected item, restore already popped items in chronological order.
- Wait for backends with asynchronous cleanup semantics before starting the next retry when stale tail items could be observed.
- Do not forward live `RunContextWrapper` objects through retry rewind or compaction storage paths unless the session API explicitly owns that runtime context.
## Compaction Replacement
- Treat history replacement as a transaction: capture the prior state, apply the compacted state, and restore the prior state if clear or replacement fails.
- Defer response-based compaction while local tool outputs still need to be associated with the response chain.
- Choose input-based or previous-response-based compaction according to the actual state owner and `store` behavior; do not combine a local replay with a server-owned history chain.
- Compaction output is a run item and must follow the item lifecycle, session sanitization, and `RunState` rules rather than bypassing them as backend-only data.
## Review Checklist
1. Distinguish model input, new-turn persistence candidates, and full session history.
2. Test atomic failure, duplicate content, reordered callbacks, and filtered handoff input.
3. Test save behavior after tool execution, handoff, guardrail trip, interruption, and resume.
4. Prove retry rewind removes only the attempt-owned suffix and restores on partial failure.
5. Test compaction replacement failures without losing the previous history.
6. Test backend ordering, atomic batches, concurrent first writes, configured names and limits, corrupt records, and resource ownership.
## Sources
- `src/agents/memory/session.py`
- `src/agents/memory/session_settings.py`
- `src/agents/memory/sqlite_session.py`
- `src/agents/extensions/memory/`
- `src/agents/run_internal/session_persistence.py`
- `src/agents/run_internal/items.py`
- `src/agents/run_internal/run_steps.py`
- `tests/memory/`
- `tests/extensions/memory/`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
@@ -0,0 +1,64 @@
# Tool Execution Lifecycle
Use this reference for changes to function-tool planning, approvals, tool guardrails, concurrency, cancellation, timeouts, hooks, error conversion, or resumed execution. Read [Tool identity and routing](tool-identity.md) when names, namespaces, lookup keys, or call IDs also change.
## Plan Before Side Effects
`process_model_response()` discovers executable work, but `tool_planning.py` decides which work may run now. Keep discovery, approval partitioning, and invocation as separate phases.
- Fresh and resumed turns need different plans. A resumed interruption must execute unresolved or newly approved work without rediscovering or rerunning completed calls.
- Approval state is authoritative once resolved. Do not call a dynamic `needs_approval` checker again for a call whose status is already approved or rejected.
- Deduplicate by invocation identity before execution while preserving model order for public call and output items. A repeated tool definition is not a repeated call, and a repeated call ID must not execute twice.
- Validate enabled tools and canonical lookup before side effects. A tool disabled after model output or absent from the resolved tool set must follow the configured missing-tool behavior rather than reaching a stale callable.
## Approval and Guardrail Ordering
- Pre-approval input guardrails are an early rejection optimization. They may run before an approval interruption, but input guardrails must run again immediately before invocation because state, policy, or arguments may have changed while approval was pending.
- Rechecking guardrails does not mean rechecking approval. Persisted approval decisions and rejection messages must remain attached to the same tool identity and call ID across `RunState` resume.
- Tool input guardrails finish before the local side effect. Tool output guardrails finish before output becomes accepted run state, model input, or persisted session history.
- The tool guardrail pipeline applies to `FunctionTool` invocation. Handoffs, hosted tools, built-in provider tools, and nested `Agent.as_tool()` runs have separate execution boundaries unless they explicitly opt into equivalent checks.
## Concurrency and Failure Semantics
SDK-side function-tool concurrency is independent of provider-side parallel tool-call generation. The provider controls how many calls appear in one response; `RunConfig.tool_execution.max_function_tool_concurrency` controls how many local function handlers run at once.
- Preserve model order in emitted outputs even when handlers complete out of order.
- Isolate sibling results. A cancelled or failed call must not discard outputs already produced by successful siblings.
- Distinguish cancellation of one tool handler from cancellation of the parent run. Tool-local cancellation can follow the configured tool failure policy; parent cancellation must propagate promptly instead of becoming model-visible tool output.
- `task.cancel()` is not terminal cleanup. On sibling failure, drain cancelled handlers and wait for post-invocation work within the bounded cleanup policy. On parent cancellation, cancel remaining tasks and attach result callbacks so late exceptions are observed without delaying cancellation indefinitely.
- Select and raise failures deterministically when several tasks fail, while still observing secondary failures. Do not let task-set iteration order or eager task execution change the public result.
## Invocation Boundary
- Decorated synchronous Python functions run through `asyncio.to_thread()` so they do not block the event loop. Async function tools run in the event loop and are the only decorated handlers that support SDK timeouts.
- Timeout handling and ordinary exception handling are distinct policies. `timeout_behavior` and `timeout_error_function` own timeout conversion; `failure_error_function=None` means ordinary exceptions propagate instead of becoming model-visible output.
- Tool start/end hooks and function spans surround the actual invocation once per call, including failure and cancellation paths. Do not emit a successful end state before output guardrails complete.
- Per-run resources such as resolved `Computer` implementations must be initialized and disposed by the run that acquired them.
- Nested `Agent.as_tool()` execution owns a nested run loop and nested resumable state. Scope cached nested state by the parent `RunState` and call identity, not only by the reusable agent or tool object.
- `AgentToolUseTracker` records tool use per agent identity. When `reset_tool_choice=True`, reset the effective next-turn tool choice after that agent uses a tool so `required` or a named choice cannot force an accidental loop; do not mutate the agent's declared settings across independent runs.
- Persist and restore the tool-use tracker across interruption and sandbox resume, including graphs with duplicate agent names, so resumed tool-choice behavior matches uninterrupted execution.
## Review Checklist
1. Trace fresh execution, approval interruption, approval rejection, and serialized resume separately.
2. Verify guardrail, approval, hook, trace, invocation, output, and persistence order.
3. Test sequential, bounded-concurrency, sibling failure, tool-local cancellation, and parent cancellation paths.
4. Test default, custom, and disabled failure conversion plus timeout behavior where applicable.
5. Confirm every started task and per-run resource reaches a deterministic terminal state.
## Sources
- `docs/running_agents.md`
- `docs/tools.md`
- `docs/guardrails.md`
- `docs/human_in_the_loop.md`
- `src/agents/run_internal/tool_planning.py`
- `src/agents/run_internal/tool_execution.py`
- `src/agents/tool.py`
- `tests/test_agent_runner.py`
- `tests/test_agent_runner_streamed.py`
- `tests/test_function_tool.py`
- `tests/test_tool_guardrails.py`
- `tests/test_tool_choice_reset.py`
- `tests/test_tool_use_tracker.py`
- `tests/test_run_state.py`
+73
View File
@@ -0,0 +1,73 @@
# Tool Identity and Routing
Use this reference for changes involving function-tool names, namespaces, provider wire names, lookup, approvals, tracing, MCP exposure, handoffs, or tool call IDs.
## Identity Layers
One tool can have several related identifiers. They are not interchangeable.
| Layer | Purpose | Canonical source |
|---|---|---|
| Public name | User- and model-facing tool name | `tool.name` |
| Explicit namespace | Distinguishes tools with the same public name | Tool namespace metadata |
| Qualified or dispatch name | Routes a model call to the intended tool | `namespace.name` when a namespace exists |
| Lookup key | Collision-free internal identity | `bare`, `namespaced`, or `deferred_top_level` tuple |
| Approval keys | Matches approval decisions to the intended tool | Canonical qualified and permitted alias keys |
| Trace name | Human-readable tracing label | Explicit trace name or public name |
| Call ID | Identifies one invocation, not the tool definition | Provider-supplied string |
Do not collapse these layers into one string or introduce local rules that only one caller uses.
Provider wire-name grammar and length limits remain provider-owned validation unless an SDK-owned routing invariant requires local enforcement. Follow [Model and Provider Boundaries](model-provider-boundaries.md#provider-validation-and-error-ownership) before adding name validation to shared tool types or canonical identity helpers.
## Canonical Helpers
Use `src/agents/_tool_identity.py` as the single implementation layer. Important helpers include:
- `get_function_tool_lookup_key_for_tool()` and `get_function_tool_lookup_key_for_call()` for canonical lookup identity.
- `get_function_tool_dispatch_name()` and `get_function_tool_qualified_name()` for routing and display surfaces that require qualification.
- `get_function_tool_approval_keys()` for approval matching.
- `get_function_tool_trace_name()` and `get_tool_call_trace_name()` for trace labels.
- `validate_function_tool_lookup_configuration()` and `build_function_tool_lookup_map()` for collision detection and dispatch maps.
- `normalize_tool_call_for_function_tool()` when provider payloads must be normalized for a selected tool.
If a proposed change bypasses these helpers, first prove that the target surface has intentionally different semantics.
## MCP and Handoff Rules
- `include_server_in_tool_names` is opt-in. Server-prefixed MCP names affect the model-exposed collision-safe name; they do not rename the original tool on the MCP server.
- Reserved names and enabled handoff names participate in collision avoidance only on the paths that expose generated model-facing names.
- `Handoff.default_tool_name()` is the source of default handoff tool names. Keep Realtime and non-Realtime handoff conversion aligned with it.
- Do not forward a naming option through a path where the downstream helper does not consult it and then describe the change as runtime behavior. Trace the complete caller-to-dispatch path first.
## Deferred Tool Search Rules
- A top-level `FunctionTool` with `defer_loading=True` and no explicit namespace uses the synthetic lookup key `("deferred_top_level", tool.name)`.
- The Responses wire shape for a loaded deferred top-level tool can look like `namespace == name`. Treat that namespace as reserved for the synthetic deferred tool-search path, not as a normal explicit namespace.
- `tool_namespace()` must reject an explicit namespace that equals the inner tool name. Otherwise a normal namespaced tool and a deferred top-level tool would have the same wire shape.
- Preserve the synthetic namespace on approval, interruption, tracing, and `ToolContext` surfaces when it identifies the model call, but dispatch the actual local tool through the deferred lookup key and strip the synthetic namespace before invoking the tool.
- Permanent approvals for deferred top-level tools should key by `deferred_top_level:<name>`. A bare-name approval alias is allowed only when no visible bare sibling can make that alias ambiguous.
## Tool Call ID Rules
- Preserve provider-supplied string call IDs across call items, approvals, outputs, retries, and serialized state.
- Do not coerce arbitrary values with `str(...)`. Canonical extractors return a call ID only when the source value is already a string.
- Do not use a call ID as a tool-definition identity or a tool name as an invocation identity.
- When a provider omits a stable identifier, use an existing fingerprint or dedupe policy for that item type instead of inventing a cross-provider ID contract.
## Review Checklist
1. Identify every identifier layer affected by the change.
2. Trace the actual runtime path from model-visible name to lookup, approval, invocation, output, and trace metadata.
3. Compare adjacent canonical helpers before adding conversion or fallback behavior.
4. Test collisions between bare, namespaced, deferred, MCP, local function, and handoff tools when applicable.
5. Require a regression test that fails on the base and proves the model-visible or dispatch behavior, not only an intermediate argument value.
## Sources
- `src/agents/_tool_identity.py`
- `src/agents/agent.py`
- `src/agents/mcp/`
- `src/agents/handoffs/__init__.py`
- `src/agents/run_internal/tool_execution.py`
- `src/agents/run_state.py`
+58
View File
@@ -0,0 +1,58 @@
# Tracing Lifecycle
Use this reference for changes to SDK trace or span context, processors, export, flush, shutdown, resumed trace state, or sensitive-data handling. Read [Realtime tracing architecture](realtime-tracing.md) before applying these client-side rules to Realtime server traces.
## Context and Parenting
- The current trace and span are held in `ContextVar` state. Async tasks inherit a snapshot when created; later changes in a child task do not rewrite the parent task's context.
- A context token must be reset in the context that created it. Start and finish ownership cannot be transferred between tasks without an explicit context boundary.
- A no-op trace or span cannot be a real parent. Propagate no-op behavior instead of exporting children with the sentinel `no-op` trace or span ID.
- Span factories should inherit trace metadata needed by processors, but they must not mutate the trace's caller-owned metadata mapping.
## Run and Resume Ownership
- A runner-created trace encloses run-loop-owned guardrails, model calls, tool execution, handoffs, session persistence, and error handling. Do not assume every completion callback or resource cleanup runs before trace finish; place newly traced cleanup explicitly inside the trace lifetime or create a deliberate separate trace/span context.
- An existing caller trace remains caller-owned. `Runner` may create child spans but must not finish or flush the caller's trace.
- `RunState` stores enough trace metadata to continue an interrupted run. Resume may reattach only when the trace ID was previously started in the process and the effective workflow name, group ID, metadata, and tracing key identity still match.
- Reattachment must not emit a duplicate trace-start event. If the saved state cannot prove a compatible live trace, create a normal trace according to the current run configuration instead of pretending to resume the old context.
- Tracing API keys are omitted from serialized `RunState` by default. A hash can verify that the caller supplied the same explicit key without persisting the secret; raw key persistence is opt-in.
## Processor and Export Isolation
- Trace processors are observability extensions and must not change application success. Catch processor callback, exporter, flush, and shutdown failures and report them as non-fatal.
- The default batch worker starts lazily on first queued item to avoid import-time thread and fork hazards. Keep top-level imports free of worker creation and shutdown-handler duplication.
- An exporter exception must not kill the batch worker and strand future traces. Drop or report the failed batch according to policy, then keep the worker usable.
- `flush_traces()` waits for queued and in-flight export work, so callers should invoke it after the trace closes when they require immediate delivery. It is not a substitute for finishing a partially built trace.
- Shutdown is best effort and deadline-aware. It should request exporter shutdown, interrupt retry backoff, drain within the remaining deadline, and return without changing the process exit code when an exporter blocks or a backend remains unavailable.
- Keep `TraceProvider.force_flush()` and `shutdown()` defaulting to no-ops for compatibility with custom providers that predate these lifecycle methods.
## Data Boundaries
- `trace_include_sensitive_data=False` controls captured span payload fields; it does not automatically sanitize exception objects, chaining, tracebacks, logs, or telemetry created elsewhere.
- Redaction must cover `__cause__`, `__context__`, formatter failures, and model-visible error conversion when an original exception carries tool arguments or provider payloads. `raise ... from None` changes display, not object retention.
- The OpenAI trace exporter owns ingest-specific payload sanitization such as field-size limits and supported usage keys. Custom processors should continue receiving the SDK's normal trace data unless their contract says otherwise.
- Per-run tracing keys, organization, and project routing must stay attached to the trace or exported item that selected them; do not let mutable global exporter state reroute an already-created trace.
## Review Checklist
1. Identify which task and context own each trace and span start, finish, and token reset.
2. Test success, exception, cancellation, interruption, serialized resume, full stream exhaustion, and explicit stream close.
3. Verify processor and exporter failures remain non-fatal and do not kill later export work.
4. Test flush and shutdown with queued work, in-flight export, retry backoff, and a blocking exporter.
5. Audit sensitive data through span payloads, exception chains, logs, and serialized state.
## Sources
- `docs/tracing.md`
- `src/agents/tracing/context.py`
- `src/agents/tracing/scope.py`
- `src/agents/tracing/traces.py`
- `src/agents/tracing/spans.py`
- `src/agents/tracing/provider.py`
- `src/agents/tracing/processors.py`
- `src/agents/tracing/setup.py`
- `src/agents/run_state.py`
- `tests/test_trace_processor.py`
- `tests/test_tracing.py`
- `tests/test_run_state.py`
- `tests/tracing/test_import_side_effects.py`
@@ -0,0 +1,56 @@
# Voice Pipeline Lifecycle
Use this reference for changes to `VoicePipeline`, `AudioInput`, `StreamedAudioInput`, STT sessions, TTS task ordering, voice lifecycle events, PCM framing, result streaming, or voice tracing. Realtime agents use a different live-session architecture; read [Realtime session lifecycle](realtime-session-lifecycle.md) for that path.
## Pipeline Ownership
`VoicePipeline` owns an STT-to-workflow-to-TTS producer task and returns a `StreamedAudioResult` that drives its observable completion.
- Static `AudioInput` produces one transcription and one workflow turn. `StreamedAudioInput` creates a long-lived transcription session and runs one workflow turn for each emitted transcript until the input or session ends.
- The multi-turn pipeline owns the transcription session and closes it in `finally` before marking output complete. Partial setup and workflow failure must not strand the STT connection or producer task.
- `workflow.on_start()` applies only to the streamed multi-turn path. Its failure is logged and skipped so the transcription session can still start; normal per-turn workflow failures are terminal and surface through the result stream.
- The SDK does not provide application-level interruption handling for `StreamedAudioInput`. Lifecycle events expose turn boundaries, but microphone muting, playback interruption, and barge-in policy remain application-owned.
## Text, Audio, and Event Ordering
- A workflow can yield multiple text fragments. The text splitter returns ready-to-synthesize text plus a remainder; synthesize non-empty ready text even when it is shorter than a default sentence threshold, and retain the remainder for the turn's final flush.
- TTS segment tasks may run concurrently, but `_ordered_tasks` and the dispatcher must emit their audio and lifecycle events in workflow text order rather than completion order.
- `turn_started` precedes audio for that turn. `turn_ended` is emitted only after the turn's final text remainder has been synthesized and its audio dispatched. `session_ended` follows all ordered segment queues and all turns.
- A `VoiceStreamEventError` terminates result streaming and the stored exception is raised after task cleanup. `session_ended` is a lifecycle marker, not proof of success; consumers must still observe the terminal exception from `stream()`.
- Consuming `StreamedAudioResult.stream()` is the public completion and error boundary. On normal `session_ended`, let the producer finish before cleanup so session close and trace end are not cancelled by result teardown.
## PCM and Caller Data
- PCM16 samples span two bytes. Preserve a trailing half-sample across TTS chunks, combine it with the next chunk, and pad only the final unmatched byte at end of segment.
- Apply `buffer_size` to TTS source chunks without changing sample order. Convert to float32 only after PCM16 framing is complete, then apply caller-provided `transform_data` to each emitted array.
- `AudioInput.to_base64()` and audio-file conversion must not mutate the caller's NumPy buffer when converting float input to PCM16.
- Empty input and empty text-splitter output are valid boundaries. They must not cause NumPy reduction errors, phantom TTS calls, or missing turn/session lifecycle events.
## Trace Lifetime and Data
- The pipeline trace stays active for the full asynchronous producer lifecycle, not only until `VoicePipeline.run()` returns its result object.
- Each output turn owns a speech-group span and each synthesized segment owns a child speech span. Finish the turn span after ordered audio dispatch and finish the pipeline trace after STT session close and output completion.
- Text and audio sensitivity are independent controls. `trace_include_sensitive_data` governs transcript and TTS text, while `trace_include_sensitive_audio_data` governs encoded audio payloads.
- Error paths must finish active speech spans and the enclosing trace without replacing the original pipeline exception.
## Review Checklist
1. Test static and streamed input, including STT setup failure, workflow failure, TTS failure, and transcription-session close.
2. Verify fragment concurrency never changes audio, turn, or session event order.
3. Test short splitter output, empty output, odd-byte chunks, cross-chunk sample boundaries, int16, and float32 conversion.
4. Consume the public result stream and verify terminal errors, task cleanup, session close, and trace-end order.
5. Confirm sensitive text and audio are independently omitted from trace payloads.
## Sources
- `docs/voice/pipeline.md`
- `docs/voice/tracing.md`
- `src/agents/voice/pipeline.py`
- `src/agents/voice/result.py`
- `src/agents/voice/input.py`
- `src/agents/voice/model.py`
- `src/agents/voice/models/openai_stt.py`
- `tests/voice/test_pipeline.py`
- `tests/voice/test_input.py`
- `tests/voice/test_openai_stt.py`
- `tests/voice/test_openai_tts.py`
@@ -7,21 +7,43 @@ description: Run the mandatory verification stack when changes affect runtime co
## Overview
Ensure work is only marked complete after formatting, linting, type checking, and tests pass. Use this skill when changes affect runtime code, tests, or build/test configuration. You can skip it for docs-only or repository metadata unless a user asks for the full stack.
Ensure work is only marked complete after formatting, linting, type checking, and tests pass. Use this skill when changes affect runtime code, tests, or build/test configuration. You can skip it for docs-only or repository metadata unless a user asks for the full stack. This is a post-review final gate: when `$implementation-final-review` applies, do not invoke the broad stack until its clean-review condition applies to the stable task diff.
## Quick start
1. Keep this skill at `./.agents/skills/code-change-verification` so it loads automatically for the repository.
2. macOS/Linux: `bash .agents/skills/code-change-verification/scripts/run.sh`.
3. Windows: `powershell -ExecutionPolicy Bypass -File .agents/skills/code-change-verification/scripts/run.ps1`.
4. The scripts run `make format` first, then run `make lint`, `make typecheck`, and `make tests` in parallel with fail-fast semantics.
5. While the parallel steps are still running, the scripts emit periodic heartbeat updates so you can tell that work is still in progress.
6. If any command fails, fix the issue, rerun the script, and report the failing output.
7. Confirm completion only when all commands succeed with no remaining issues.
2. Codex on macOS/Linux: `/usr/bin/env -u OPENAI_API_KEY OPENAI_AGENTS_TEST_IN_CODEX_SANDBOX=1 UV_DEFAULT_INDEX=https://pypi.org/simple bash .agents/skills/code-change-verification/scripts/run.sh`.
3. Other macOS/Linux environments: `env UV_DEFAULT_INDEX=https://pypi.org/simple bash .agents/skills/code-change-verification/scripts/run.sh`.
4. Windows: `powershell -ExecutionPolicy Bypass -File .agents/skills/code-change-verification/scripts/run.ps1`.
5. The scripts run `make format` first, then run `make lint`, `make typecheck`, and `make tests` in parallel with fail-fast semantics.
6. While the parallel steps are still running, the scripts emit periodic heartbeat updates so you can tell that work is still in progress.
7. If any command fails, fix the issue, rerun the script, and report the failing output.
8. Confirm completion only when all commands succeed with no remaining issues.
## Start condition and host capacity
- During iterative review, use only focused tests and a narrowly targeted static check when the changed typing boundary requires one. Defer repository-wide `make typecheck` and the rest of this complete stack until review is clean.
- Immediately before starting the complete stack, use available read-only task or process evidence to check whether another repository-wide test, typecheck, build, examples runner, or integration command is already active on the same host.
- When concrete contention is visible, continue useful non-heavy work such as review, remediation, evidence preparation, or focused checks, then check again later. Do not create or wait on a repository lock, host-wide mutex, or sentinel file.
- Start automatically once review is clean, the diff is stable, and observable host capacity is available. Do not require a user-triggered `finalize` message. If host telemetry is unavailable, do not block solely because capacity cannot be measured.
## Codex execution policy
Repository verification and all child processes must remain in the normal Codex workspace sandbox. Never request elevated sandbox permissions for the verification wrapper, and never retry the wrapper with broader host access after a failure.
On macOS, tests marked `requires_native_macos_sandbox` need to start their own `sandbox-exec` process. The Codex command sets `OPENAI_AGENTS_TEST_IN_CODEX_SANDBOX=1`, which skips only that marker before nested sandbox creation. All other tests remain enabled. Ordinary local and CI runs do not set this variable and therefore keep the marked tests enabled.
The marked tests run separately on a disposable GitHub-hosted macOS runner. If that trusted runner is unavailable, report the missing native-macOS coverage; do not compensate by weakening the Codex sandbox boundary.
## Environment setup
The verification scripts assume repository dependencies are already installed. Do not run `make sync` as part of every verification pass; use it for a fresh checkout, after dependency files change, or when dependency resolution fails before the checks start.
On Linux, some Python packages with native extensions may require system packages such as `libffi-dev`, Python development headers, or build tools. If verification cannot start because one of these packages is missing, treat it as a local environment setup issue. Install the missing dependency when possible, or report the failing command and missing dependency in the PR test plan before rerunning verification in a prepared environment.
## Manual workflow
- If dependencies are not installed or have changed, run `make sync` first to install dev requirements via `uv`.
- For a fresh checkout, or if dependencies are not installed or have changed, run `make sync` first to install dev requirements via `uv`.
- Run from the repository root with `make format` first, then `make lint`, `make typecheck`, and `make tests`.
- Do not skip steps; stop and fix issues immediately when a command fails.
- If you run the steps manually, you may parallelize `make lint`, `make typecheck`, and `make tests` after `make format` completes, but you must stop the remaining steps as soon as one fails.
@@ -1,4 +1,4 @@
interface:
display_name: "Code Change Verification"
short_description: "Run the required local verification stack"
default_prompt: "Use $code-change-verification to run the required local verification stack and report any failures."
default_prompt: "Use $code-change-verification to run the required local verification stack inside the normal Codex workspace sandbox. On macOS/Linux, use the documented Codex command so only tests that require a native macOS sandbox are skipped locally. Never request elevated sandbox permissions or retry with broader host access. Report any failures."
@@ -286,13 +286,21 @@ check_for_missing_reporters() {
log_file="${STEP_LOGS[$idx]}"
start_time="${STEP_STARTS[$idx]}"
now=$(date +%s)
wait "${pid}" 2>/dev/null || true
set +e
wait "${pid}" 2>/dev/null
step_status=$?
set -e
if [ "${step_status}" -eq 0 ]; then
finish_step "${name}" 0
return 0
fi
echo "code-change-verification: make ${name} exited before reporting completion status after $((now - start_time))s." >&2
echo "--- ${name} log (last 80 lines) ---" >&2
tail -n 80 "${log_file}" >&2 || true
stop_running_steps
return 1
return "${step_status}"
done
return 0
+3 -2
View File
@@ -15,7 +15,7 @@ Identify doc coverage gaps and inaccuracies by comparing main branch features an
- Identify the current branch and default branch (usually `main`).
- Prefer analyzing the current branch to keep work aligned with in-flight changes.
- If the current branch is not `main`, analyze only the diff vs `main` to scope doc updates.
- Avoid switching branches if it would disrupt local changes; use `git show main:<path>` or `git worktree add` when needed.
- Avoid switching branches if it would disrupt local changes. Prefer read-only inspection such as `git show main:<path>`. If a separate checkout is genuinely required, stop and obtain the explicit approval required by `AGENTS.md` before creating or switching a worktree.
2. Build a feature inventory from the selected scope
- If on `main`: inventory the full surface area and review docs comprehensively.
@@ -51,7 +51,8 @@ Identify doc coverage gaps and inaccuracies by comparing main branch features an
- Do **not** edit `docs/ja`, `docs/ko`, or `docs/zh`.
- Keep changes aligned with the existing docs style and navigation.
- Update `mkdocs.yml` when adding or renaming pages.
- Build docs with `make build-docs` after edits to verify the docs site still builds.
- Classify the complete diff with the Documentation Verification Tiers in `AGENTS.md` and run only the checks required by that tier.
- For content or structural changes, run `make build-docs` once after the edits and required review are stable. Do not run it for editorial-only changes.
## Output format
-77
View File
@@ -1,77 +0,0 @@
---
name: examples-auto-run
description: Run python examples in auto mode with logging, rerun helpers, and background control.
---
# examples-auto-run
## What it does
- Runs `uv run examples/run_examples.py` with:
- `EXAMPLES_INTERACTIVE_MODE=auto` (auto-input/auto-approve).
- Per-example logs under `.tmp/examples-start-logs/`.
- Main summary log path passed via `--main-log` (also under `.tmp/examples-start-logs/`).
- Generates a rerun list of failures at `.tmp/examples-rerun.txt` when `--write-rerun` is set.
- Provides start/stop/status/logs/tail/collect/rerun helpers via `run.sh`.
- Background option keeps the process running with a pidfile; `stop` cleans it up.
## Usage
```bash
# Start (auto mode; interactive included by default)
.agents/skills/examples-auto-run/scripts/run.sh start [extra args to run_examples.py]
# Examples:
.agents/skills/examples-auto-run/scripts/run.sh start --filter basic
.agents/skills/examples-auto-run/scripts/run.sh start --include-server --include-audio
# Check status
.agents/skills/examples-auto-run/scripts/run.sh status
# Stop running job
.agents/skills/examples-auto-run/scripts/run.sh stop
# List logs
.agents/skills/examples-auto-run/scripts/run.sh logs
# Tail latest log (or specify one)
.agents/skills/examples-auto-run/scripts/run.sh tail
.agents/skills/examples-auto-run/scripts/run.sh tail main_20260113-123000.log
# Collect rerun list from a main log (defaults to latest main_*.log)
.agents/skills/examples-auto-run/scripts/run.sh collect
# Rerun only failed entries from rerun file (auto mode)
.agents/skills/examples-auto-run/scripts/run.sh rerun
```
## Defaults (overridable via env)
- `EXAMPLES_INTERACTIVE_MODE=auto`
- `EXAMPLES_INCLUDE_INTERACTIVE=1`
- `EXAMPLES_INCLUDE_SERVER=0`
- `EXAMPLES_INCLUDE_AUDIO=0`
- `EXAMPLES_INCLUDE_EXTERNAL=0`
- Auto-approvals in auto mode: `APPLY_PATCH_AUTO_APPROVE=1`, `SHELL_AUTO_APPROVE=1`, `AUTO_APPROVE_MCP=1`
## Log locations
- Main logs: `.tmp/examples-start-logs/main_*.log`
- Per-example logs (from `run_examples.py`): `.tmp/examples-start-logs/<module_path>.log`
- Rerun list: `.tmp/examples-rerun.txt`
- Stdout logs: `.tmp/examples-start-logs/stdout_*.log`
## Notes
- The runner delegates to `uv run examples/run_examples.py`, which already writes per-example logs and supports `--collect`, `--rerun-file`, and `--print-auto-skip`.
- `start` uses `--write-rerun` so failures are captured automatically.
- If `.tmp/examples-rerun.txt` exists and is non-empty, invoking the skill with no args runs `rerun` by default.
## Behavioral validation (Codex/LLM responsibility)
The runner does not perform any automated behavioral validation. After every foreground `start` or `rerun`, **Codex must manually validate** all exit-0 entries:
1. Read the example source (and comments) to infer intended flow, tools used, and expected key outputs.
2. Open the matching per-example log under `.tmp/examples-start-logs/`.
3. Confirm the intended actions/results occurred; flag omissions or divergences.
4. Do this for **all passed examples**, not just a sample.
5. Report immediately after the run with concise citations to the exact log lines that justify the validation.
@@ -1,4 +0,0 @@
interface:
display_name: "Examples Auto Run"
short_description: "Run examples in auto mode with logs and rerun helpers"
default_prompt: "Use $examples-auto-run to run the repo examples in auto mode, collect logs, and summarize any failures."
@@ -0,0 +1,51 @@
---
name: examples-run-analysis
description: Analyze artifacts from the latest completed manual examples Make run. Read the main log, every relevant per-example log, and example source; validate every exit-0 example and classify failures, skips, and environment restrictions. Never execute or control examples.
---
# Examples Run Analysis
Use this skill only to analyze artifacts that already exist after a user has manually invoked an examples Make target. This skill is read-only and analysis-only.
## Hard boundary
- Never start, retry, stop, or otherwise execute examples.
- Never invoke an examples Make target or `.github/scripts/run_examples.sh`.
- Never request elevated execution, alter an environment, remove a pid file, or own or signal a background process.
- Never treat an older completed run as current when the newest run is active, incomplete, or stale.
- If usable results are missing, stale, incomplete, or still running, stop the analysis and ask the user to run the appropriate Make target manually. Give the exact command but do not execute it.
The supported workflow is an explicit manual Make invocation followed by analysis of the generated artifacts.
## Artifacts to inspect
- Background pid file: `.tmp/examples-auto-run.pid`.
- Main logs: `.tmp/examples-start-logs/main_*.log`.
- Per-example logs named by each `log=` field in the selected main log.
- Example sources named by `PASSED`, `FAILED`, and `SKIPPED` records.
- Runner sources that define artifact meaning: `examples/run_examples.py`, `.github/scripts/run_examples.sh`, and the example source files included in the run.
Use only read-only inspection commands such as `git status`, `git log`, `find`, `ls`, `stat`, `ps`, `sed`, and `rg`. Do not call a command that can update an artifact or process.
## Analysis workflow
1. Inspect the process table and `.tmp/examples-auto-run.pid` without changing either. Treat a process as an active examples run only when its command line is rooted in the current repository and invokes `.github/scripts/run_examples.sh` or `examples/run_examples.py`, including foreground and background runs. Use the pid file only to correlate a background process; an absent or stale pid file does not prove that no run is active. If a matching process is live, stop the analysis. Tell the user to wait for a foreground Make run to finish, or ask the user to run `make examples-status` manually for a background run, before requesting analysis again.
2. Select the newest `main_*.log`. Require exactly one terminal `# summary executed=<n> skipped=<n> failed=<n>` record. Treat a missing or malformed summary, a changing log, or a matching active examples process as incomplete.
3. Treat the result as stale when relevant runner or selected example source content changed after the run. Use Git history and file timestamps as evidence. If freshness cannot be established, say so and request a new manual run instead of assuming the artifacts apply.
4. Parse every `PASSED`, `FAILED`, and `SKIPPED` record. Reconcile their counts with the terminal summary. Confirm that every referenced per-example log exists.
5. For every `PASSED` record, without sampling, read the complete example source and its per-example log. Infer the intended flow, tools, side effects, and key result from the source and comments, then verify that the log demonstrates those behaviors. Exit status 0 alone is not behavioral validation.
6. Read the relevant per-example logs for failures and environment-related skips. Classify each result as an example or SDK defect, dependency or credential problem, provider or network failure, local service or platform restriction, intentional runner skip, or unresolved. Keep genuine product failures separate from environment restrictions.
7. Report the selected main log, freshness and completeness evidence, summary counts, validation status for every exit-0 example, classified failures and skips, and exact source/log line references that support each conclusion.
## Manual commands to request when artifacts are unusable
Choose the narrowest applicable command and ask the user to run it in a terminal:
```bash
make examples-run
make examples-run EXAMPLES_ARGS="--filter basic"
make examples-run-background EXAMPLES_ARGS="--include-server --include-audio"
make examples-status
```
Do not execute any of these commands as part of this skill.
@@ -0,0 +1,4 @@
interface:
display_name: "Examples Run Analysis"
short_description: "Analyze completed example-run artifacts"
default_prompt: "Use $examples-run-analysis to inspect the latest completed manual examples run and validate every exit-0 example without executing or controlling any process."
+184 -75
View File
@@ -1,91 +1,175 @@
---
name: final-release-review
description: Perform a release-readiness review by locating the previous release tag from remote tags and auditing the diff (e.g., v1.2.3...<commit>) for breaking changes, regressions, improvement opportunities, and risks before releasing openai-agents-python.
description: Perform pre-release planning or a final release-candidate review for openai-agents-python by comparing the target with the previous remote tag, determining the minimum compatible release type, auditing regressions and contract changes, reviewing open documentation PR coverage, drafting minor-release Key Changes, and calling the ship/block gate.
---
# Final Release Review
## Purpose
Use this skill when validating the latest release candidate commit (default tip of `origin/main`) for release. It guides you to fetch remote tags, pick the previous release tag, and thoroughly inspect the `BASE_TAG...TARGET` diff for breaking changes, introduced bugs/regressions, improvement opportunities, and release risks.
Audit `BASE_TAG...TARGET` in one of two modes:
The review must be stable and actionable: avoid variance between runs by using explicit gate rules, and never produce a `BLOCKED` call without concrete evidence and clear unblock actions.
- **Pre-release planning:** use when the user asks to plan the next release or when the target, normally `origin/main`, does not yet declare a release candidate. The user may still supply a tentative `patch` or `minor` intent. Recommend the compatible type; do not treat unchanged package metadata as a blocker.
- **Final candidate:** use when the user asks for a final candidate decision, the target is a release branch, or target package metadata has already been bumped beyond BASE for the next release. Compare the candidate intent with the minimum release type required by the diff.
In both modes, find concrete regressions and release risks, independently determine version compatibility, review the latest open documentation PRs before claiming coverage is missing, and produce an actionable release handoff. Keep documentation readiness separate from the release gate. The release call is a controlling checker result: callers must stop on **BLOCKED** and may continue only on **GREEN LIGHT TO SHIP**. Producing the report text is not itself a passing result.
## Quick start
1. Ensure repository root: `pwd``path-to-workspace/openai-agents-python`.
2. Sync tags and pick base (default `v*`):
1. Ensure the repository root is `openai-agents-python`. When a caller supplies a dedicated candidate worktree, run every local inspection from that worktree rather than another checkout of the repository.
2. Sync remote tags and choose the previous release:
```bash
BASE_TAG="$(.agents/skills/final-release-review/scripts/find_latest_release_tag.sh origin 'v*')"
```
3. Choose target commit (default tip of `origin/main`, ensure fresh): `git fetch origin main --prune` then `TARGET="$(git rev-parse origin/main)"`.
4. Snapshot scope:
3. Refresh and resolve the target, defaulting to `origin/main`:
```bash
git fetch origin main --prune
TARGET="$(git rev-parse origin/main)"
```
4. Resolve review mode independently from release intent:
1. Honor an explicit user request for pre-release planning or final-candidate review.
2. Otherwise, use final-candidate mode only when the target is a release branch or its package metadata has already been bumped beyond BASE for the next release.
3. Otherwise, use pre-release planning mode.
5. Resolve release intent separately, without asking when repository state already answers it:
1. User-supplied version or `patch`/`minor` intent.
2. A target branch name or target package version that declares the next release.
3. Otherwise, set intent to `unspecified`.
4. If final-candidate mode was explicitly requested but intent remains `unspecified`, ask for the intended type or version before issuing a final-candidate gate. If the user prefers an uninterrupted review, switch to pre-release planning and make a recommendation instead.
6. Snapshot the release diff:
```bash
git diff --stat "${BASE_TAG}"..."${TARGET}"
git diff --dirstat=files,0 "${BASE_TAG}"..."${TARGET}"
git log --oneline --reverse "${BASE_TAG}".."${TARGET}"
git diff --name-status "${BASE_TAG}"..."${TARGET}"
```
5. Deep review using `references/review-checklist.md` to spot breaking changes, regressions, and improvement chances.
6. Capture findings and call the release gate: ship/block with conditions; propose focused tests for risky areas.
7. Audit the diff with `references/review-checklist.md`, determine the minimum release type, and prove or dismiss each candidate against the released contract.
8. Discover and review relevant open documentation PRs using current read-only GitHub state. Do not infer coverage from local branches, titles, or historical context.
9. Report the release intent, ship/block gate, risk assessment, documentation coverage, and conditional minor-release Key Changes draft.
For a final candidate reviewed as `TARGET=HEAD`, also require `HEAD` to be the exact target in the candidate checkout, inspect the checked-out branch and release-owned files directly, and keep working-tree changes outside the commit from being mistaken for reviewed candidate content.
## Release intent and versioning policy
- Treat routine compatible releases as `patch`.
- Require `minor` for a breaking change to a non-beta public contract or for a major feature addition. Reserve major versions until 1.0.
- Determine the **minimum required release type** from the diff independently of the declared intent.
- Classify versioning as follows:
| Mode | Intended release | Minimum required | Verdict |
|---|---|---|---|
| planning | `unspecified` | either | recommend the minimum type |
| planning | `patch` | `patch` | compatible plan |
| planning | `minor` | `patch` or `minor` | compatible plan; say when minor is optional |
| planning | `patch` | `minor` | recommend changing the plan to minor; do not block the unreleased target |
| candidate | `patch` | `patch` | compatible |
| candidate | `minor` | `patch` or `minor` | compatible; say when minor is optional |
| candidate | `patch` | `minor` | under-versioned and blocking |
- In pre-release planning mode, always report `Recommended release type: patch|minor`, even when the user supplied a tentative intent. Do not require `pyproject.toml` or `uv.lock` to already contain the next version; the release workflow owns that later bump.
- In final-candidate mode, verify that the declared version, package metadata, lockfile, and release branch agree. Block a patch candidate that requires a minor release.
- Distinguish an undocumented migration from the absence of a usable migration or compatibility path. Missing documentation is non-blocking; an actual supported-path break with no usable migration or fallback can block.
## Deterministic gate policy
- Default to **🟢 GREEN LIGHT TO SHIP** unless at least one blocking trigger below is satisfied.
- Use **🔴 BLOCKED** only when you can cite concrete release-blocking evidence and provide actionable unblock steps.
- Blocking triggers (at least one required for `BLOCKED`):
- A confirmed regression or bug introduced in `BASE...TARGET` (for example, failing targeted test, incompatible behavior in diff, or removed behavior without fallback).
- A confirmed breaking public API/protocol/config change with missing or mismatched versioning and no migration path (for example, patch release for a breaking change).
- Default to **🟢 GREEN LIGHT TO SHIP** unless at least one blocking trigger is proven.
- Use **🔴 BLOCKED** only with concrete release-blocking evidence and an actionable unblock condition.
- Blocking triggers:
- A confirmed regression or bug introduced in `BASE_TAG...TARGET`.
- In final-candidate mode, a declared `patch` release when the diff requires `minor`, or inconsistent candidate version metadata.
- A confirmed breaking public API, protocol, config, or durable-state change with no usable migration, fallback, or compatibility path.
- A concrete data-loss, corruption, or security-impacting change with unresolved mitigation.
- A release-critical packaging/build/runtime path is broken by the diff (not speculative).
- Non-blocking by itself:
- Large diff size, broad refactor, or many touched files.
- "Could regress" risk statements without concrete evidence.
- Not running tests locally.
- If evidence is incomplete, issue **🟢 GREEN LIGHT TO SHIP** with targeted validation follow-ups instead of `BLOCKED`.
- A release-critical packaging, build, or runtime path broken by the diff.
- The following are never blocking by themselves:
- Large diff size, broad refactoring, or many touched files.
- Speculative "could regress" concerns without evidence.
- Not rerunning CI checks locally.
- Missing, incomplete, unmerged, stale, or post-release documentation.
- Unchanged package version metadata in pre-release planning mode.
- A documentation review may reveal an underlying runtime or compatibility defect. Block only for that defect, not for the documentation state.
- A green gate must still explain important user-visible release surfaces.
- A caller must treat any target, base, candidate-content, version-metadata, lockfile, or contract change after review as invalidating the gate. The changed candidate requires a complete new review and a new release call.
- Never issue a green release call merely because the report template is complete. The target diff and applicable checked-out candidate contents must have been inspected first.
## Workflow
- **Prepare**
- Run the quick-start tag command to ensure you use the latest remote tag. If the tag pattern differs, override the pattern argument (e.g., `'*.*.*'`).
- If the user specifies a base tag, prefer it but still fetch remote tags first.
- Keep the working tree clean to avoid diff noise.
- **Assumptions**
- Assume the target commit (default `origin/main` tip) has already passed `$code-change-verification` in CI unless the user says otherwise.
- Do not block a release solely because you did not run tests locally; focus on concrete behavioral or API risks.
- Release policy: routine releases use patch versions; use minor only for breaking changes or major feature additions. Major versions are reserved until the 1.0 release.
- **Map the diff**
- Use `--stat`, `--dirstat`, and `--name-status` outputs to spot hot directories and file types.
- For suspicious files, prefer `git diff --word-diff BASE...TARGET -- <path>`.
- Note any deleted or newly added tests, config, migrations, or scripts.
- **Analyze risk**
- Walk through the categories in `references/review-checklist.md` (breaking changes, regression clues, improvement opportunities).
- When you suspect a risk, cite the specific file/commit and explain the behavioral impact.
- For every finding, include all of: `Evidence`, `Impact`, and `Action`.
- Severity calibration:
- **🟢 LOW**: low blast radius or clearly covered behavior; no release gate impact.
- **🟡 MODERATE**: plausible user-facing regression signal; needs validation but not a confirmed blocker.
- **🔴 HIGH**: confirmed or strongly evidenced release-blocking issue.
- Suggest minimal, high-signal validation commands (targeted tests or linters) instead of generic reruns when time is tight.
- Breaking changes do not automatically require a BLOCKED release call when they are already covered by an appropriate version bump and migration/upgrade notes; only block when the bump is missing/mismatched (e.g., patch bump) or when the breaking change introduces unresolved risk.
- **Form a recommendation**
- State BASE_TAG and TARGET explicitly.
- Provide a concise diff summary (key directories/files and counts).
- List: breaking-change candidates, probable regressions/bugs, improvement opportunities, missing release notes/migrations.
- Recommend ship/block and the exact checks needed to unblock if blocking. If a breaking change is properly versioned (minor/major), you may still recommend a GREEN LIGHT TO SHIP while calling out the change. Use emoji and boldface in the release call to make the gate obvious.
- If you cannot provide a concrete unblock checklist item, do not use `BLOCKED`.
### Prepare and map the diff
- Fetch current remote tags and the target ref. Keep the working tree out of the comparison.
- Prefer a user-specified base tag, but still refresh remote tags.
- Assume the target passed repository CI unless told otherwise. Do not rerun routine unit, lint, formatting, type, or coverage checks by default.
- Use diff stats, directory distribution, commit order, and name status to identify high-risk areas. Read changed tests as behavioral evidence, not as proof by themselves.
### Inspect a materialized candidate checkout
In final-candidate mode, when the caller provides a dedicated checkout or worktree:
- Resolve and record the checkout root, current branch, `HEAD`, and clean status before auditing. Do not switch to a different checkout that happens to share the same Git object database.
- Require `TARGET=HEAD` to resolve to the checked-out commit. Treat detached HEAD, a mismatched release branch, uncommitted release-owned files, or unrelated changed paths as candidate inconsistency.
- Read `pyproject.toml`, `uv.lock`, and `tests/fixtures/released_api_contract.json` from that checkout. Verify the intended version, editable `openai-agents` lock entry, contract baseline, and contract `baseline_commit` against the release branch and commit parent.
- Inspect the exact commit diff and confirm that the materialized release commit owns only its expected release manifest when the invoking workflow defines one.
- Keep the checkout path as local evidence for the caller, but do not put local paths into copy-ready release text.
These checks make the final-candidate review a release gate. The report remains the human-readable evidence and PR-description source for a green result; it does not replace the checks.
### Audit contracts and prove findings
- Compare BASE and TARGET rather than reviewing TARGET in isolation.
- For public APIs, compare exports, identity, signatures, positional order, defaults, enums, and documented behavior.
- For packages, compare supported Python versions, dependencies, extras, distribution contents, version metadata, and import behavior.
- For persisted state, schemas, protocols, config, and environment variables, identify the released durable boundary and verify backward reads or a usable migration path.
- Route runtime changes through the owning reference in `.agents/references/README.md` and trace required consumers and symmetry axes.
- Promote a candidate only when the diff proves a contract violation, reachable supported-path regression, or concrete user-visible release consideration.
- Use the smallest BASE-versus-TARGET public-path or installed-artifact probe when static evidence cannot resolve a decision-relevant question.
- Assign **🟢 LOW** to verified, correctly versioned considerations, **🟡 MODERATE** to concrete unresolved regression signals, and **🔴 HIGH** to confirmed blockers.
- Include `Evidence`, `Impact`, `Files`, and `Action` for every risk item. Do not manufacture test or code work for a safe release consideration.
### Review documentation coverage
- First derive a documentation-obligation inventory from the runtime audit: breaking changes, migrations, defaults, opt-ins/opt-outs, major features, public APIs, provider/version compatibility, durable schemas, and changed user workflows.
- Before reporting any obligation as uncovered, inspect current open PRs through approved read-only GitHub access. Never use `gh` in this repository and never mutate GitHub.
- Discover candidates using the intended/recommended version, feature names, linked implementation PRs, branch names, and changed documentation paths. Do not rely on the PR title alone.
- For each candidate, record the PR URL/number and latest head SHA, then review its complete current diff and any current discussion that materially affects a coverage claim. Several PRs may collectively cover the inventory.
- Keep the release target diff and documentation-PR diffs separate. Do not imply that an open docs PR is already part of the release target.
- Classify aggregate coverage as `covered`, `partially covered`, `not covered`, `stale/conflicting`, or `unverified`.
- If current read-only GitHub access is unavailable, use `unverified`, explain the search limitation, and do not claim that no docs PR exists.
- For every obligation that is not demonstrably covered, including `partially covered`, `not covered`, `stale/conflicting`, and `unverified` cases, suggest the exact post-release file, section, example or claim, and migration wording. Mark suggestions provisional when coverage is unverified.
- Treat an unmerged docs PR as an acceptable post-release handoff. Documentation is published live, so note when the PR should remain unmerged until the SDK release is available.
### Draft minor-release Key Changes
- Include a copy-ready Key Changes draft whenever the intended release is `minor` or pre-release planning recommends `minor`. Omit it for patch releases unless the user requests it.
- Derive the draft from verified user-facing contracts, not raw commit counts or directory summaries.
- Follow the established GitHub release format:
```markdown
## Key Changes
<One concise paragraph stating why this is a minor release and whether it contains breaking changes.>
### Highlights:
- <Three to seven user-facing highlights grouped by theme.>
```
- Put breaking behavior and the supported migration or fallback first. If the minor bump is for major features without a break, say so explicitly.
- Cover the major release themes without reproducing the full `## What's Changed` list. Preserve exact public names, defaults, version bounds, opt-outs, and compatibility qualifiers.
- Link to published documentation when it already exists. When documentation is only in an open PR, do not publish an unstable branch link; keep the wording self-contained and mention the docs PR separately in Documentation coverage.
- Produce the draft even when the release is blocked, but do not let polished release copy hide the blocker.
## Form the recommendation
- State BASE_TAG, TARGET commit, review mode, intended release type, minimum required type, and versioning verdict.
- Summarize key directories and file counts without turning every commit into a report item.
- List only substantiated blockers and the most important verified release considerations, normally two to five grouped by user impact.
- Keep documentation coverage in its own non-blocking section.
- If blocked, include an exact unblock checklist and pass condition. If no concrete unblock action exists, do not block.
- Do not include routine command results, pass counts, skips, deselections, or a validation-status inventory.
## Output format (required)
All output must be in English.
Produce the report in English using this structure. Always use the fixed compare URL `https://github.com/openai/openai-agents-python/compare/<tag>...<target-commit>`.
Use the following report structure in every response produced by this skill. Be proactive and decisive: make a clear ship/block call near the top, and assign an explicit risk level (LOW/MODERATE/HIGH) to each finding with a short impact statement. Avoid overly cautious hedging when the risk is low and tests passed.
Always use the fixed repository URL in the Diff section (`https://github.com/openai/openai-agents-python/compare/...`). Do not use `${GITHUB_REPOSITORY}` or any other template variable. Format risk levels as bold emoji labels: **🟢 LOW**, **🟡 MODERATE**, **🔴 HIGH**.
Every risk finding must contain an actionable next step. If the report uses `**🔴 BLOCKED**`, include an `Unblock checklist` section with at least one concrete command/task and a pass condition.
```
```markdown
### Release readiness review (<tag> -> TARGET <ref>)
This is a release readiness report done by `$final-release-review` skill.
@@ -94,33 +178,58 @@ This is a release readiness report done by `$final-release-review` skill.
https://github.com/openai/openai-agents-python/compare/<tag>...<target-commit>
### Release call:
### Release intent
- Review mode: <pre-release planning | final candidate>
- Intended release: <patch/minor intent, with version when known, or unspecified in planning mode>
- Minimum required release type: <patch | minor>
- Recommended release type: <patch | minor; include in planning mode only>
- Versioning verdict: <compatible | compatible plan | recommendation only | revise plan to minor | under-versioned>
### Release call
**<🟢 GREEN LIGHT TO SHIP | 🔴 BLOCKED>** <one-line rationale>
### Scope summary:
### Scope summary
- <N files changed (+A/-D); key areas touched: ...>
### Risk assessment (ordered by impact):
1) **<Finding title>**
- Risk: **<🟢 LOW | 🟡 MODERATE | 🔴 HIGH>**. <Impact statement in one sentence.>
- Evidence: <specific diff/test/commit signal; avoid generic statements>
### Risk assessment (ordered by impact)
1. **<Finding or release consideration title>**
- Risk: **<🟢 LOW | 🟡 MODERATE | 🔴 HIGH>**. <Impact statement.>
- Evidence: <specific BASE-versus-TARGET evidence>
- Files: <path(s)>
- Action: <concrete next step command/task with pass criteria>
2) ...
- Action: <next step and pass condition>
### Unblock checklist (required when Release call is BLOCKED):
1. [ ] <concrete check/fix>
- Exit criteria: <what must be true to unblock>
2. ...
### Documentation coverage (non-blocking)
### Notes:
- <working tree status, tag/target assumptions, or re-run guidance>
- Coverage source: <PR URL/number and head SHA, multiple PRs, none found after a successful search, or search unavailable/partial>
- Status: <covered | partially covered | not covered | stale/conflicting | unverified>
- Covered obligations: <concise list or none>
- Gaps or post-release suggestions: <exact files/sections/claims, or none>
- Publication timing: <merge after release if the docs describe unreleased behavior, or not applicable>
### Unblock checklist
1. [ ] <required only when blocked>
- Exit criteria: <what must be true>
### Key Changes draft
<Include the copy-ready `## Key Changes` block only for an intended or recommended minor release.>
### Notes
- <Material assumptions only>
```
If no risks are found, include a “No material risks identified” line under Risk assessment and still provide a ship call. If you did not run local verification, do not add a verification status section or use it as a release blocker; note any assumptions briefly in Notes.
If the report is not blocked, omit the `Unblock checklist` section.
- Omit `Unblock checklist` when the release is green.
- Omit `Key Changes draft` for patch releases unless requested.
- For a behavior-impacting green release, retain at least one **🟢 LOW** consideration; do not return only "No material risks identified".
- For a metadata-only release with no reportable user-facing contract, a concise empty-risk statement is acceptable.
### Resources
## Resources
- `scripts/find_latest_release_tag.sh`: Fetches remote tags and returns the newest tag matching a pattern (default `v*`).
- `references/review-checklist.md`: Detailed signals and commands for spotting breaking changes, regressions, and release polish gaps.
- `scripts/find_latest_release_tag.sh`: refresh remote tags and return the newest matching release tag.
- `references/review-checklist.md`: detailed discovery signals, release-intent checks, docs-coverage review, and evidence requirements.
@@ -1,4 +1,4 @@
interface:
display_name: "Final Release Review"
short_description: "Audit a release candidate against the previous tag"
default_prompt: "Use $final-release-review to audit the release candidate diff against the previous release tag and call the ship/block gate."
short_description: "Plan and audit releases with docs coverage"
default_prompt: "Use $final-release-review to determine review mode, release intent, and the minimum compatible release type, audit the target, review open docs PR coverage, and draft Key Changes when the release is minor."
@@ -1,65 +1,136 @@
# Release Diff Review Checklist
## Quick commands
Use the release-mode, versioning, gate, documentation, and output policies in `../SKILL.md` as the normative rules. This checklist supplies operational discovery and evidence checks without redefining those policies.
- Sync tags: `git fetch origin --tags --prune`.
- Identify latest release tag (default pattern `v*`): `git tag -l 'v*' --sort=-v:refname | head -n1` or use `.agents/skills/final-release-review/scripts/find_latest_release_tag.sh`.
- Generate overview: `git diff --stat BASE...TARGET`, `git diff --dirstat=files,0 BASE...TARGET`, `git log --oneline --reverse BASE..TARGET`.
- Inspect risky files quickly: `git diff --name-status BASE...TARGET`, `git diff --word-diff BASE...TARGET -- <path>`.
## Establish the review inputs
## Gate decision matrix
- Sync remote tags and resolve the latest matching release tag with `../scripts/find_latest_release_tag.sh origin 'v*'`.
- Refresh the requested target, defaulting to `origin/main`, and record its exact commit.
- When the caller provides a dedicated checkout or worktree, run every local inspection there and record its root, current branch, `HEAD`, and clean status as gate evidence. Do not substitute another checkout that shares the same Git objects.
- Resolve review mode first, then release intent. Record the evidence for each decision separately.
- Generate `git diff --stat BASE...TARGET`, `git diff --dirstat=files,0 BASE...TARGET`, `git log --oneline --reverse BASE..TARGET`, and `git diff --name-status BASE...TARGET`.
- Inspect suspicious paths with `git diff --word-diff BASE...TARGET -- <path>`.
- Keep working-tree changes and open documentation PR diffs outside the release comparison.
- Choose `🟢 GREEN LIGHT TO SHIP` when no concrete blocking trigger is found.
- Choose `🔴 BLOCKED` only when at least one blocking trigger has concrete evidence and a defined unblock action.
- Blocking triggers:
- Confirmed regression/bug introduced in the diff.
- Confirmed breaking public API/protocol/config change with missing or mismatched versioning/migration path.
- Concrete data-loss/corruption/security-impacting issue with unresolved mitigation.
- Release-critical build/package/runtime break introduced by the diff.
- Non-blocking by itself:
- Large refactor or high file count.
- Speculative risk without evidence.
- Not running tests locally.
- If uncertain, keep gate green and provide focused follow-up checks.
## Determine the minimum release type
## Actionability contract
Compare the diff with the released BASE contract. Use `minor` as the minimum for either of these conditions:
- Every risk finding should include:
- `Evidence`: specific file/commit/diff/test signal.
- `Impact`: one-sentence user or runtime effect.
- `Action`: concrete command/task with pass criteria.
- A `BLOCKED` report must contain an `Unblock checklist` with at least one executable item.
- If no executable unblock item exists, do not block; downgrade to green with follow-up checks.
- a breaking change to a non-beta public API, protocol, configuration, environment, or durable serialized boundary;
- a major user-facing feature addition that warrants a minor release under repository policy.
## Breaking change signals
Use `patch` otherwise. For a final candidate, verify the intended version against the branch name, `pyproject.toml`, `uv.lock`, and built package metadata when relevant. For planning mode, do not interpret unchanged version metadata as a declared patch candidate.
- Public API surface: removed/renamed modules, classes, functions, or re-exports; changed parameters/return types, default values changed, new required options, stricter validation.
- Protocol/schema: request/response fields added/removed/renamed, enum changes, JSON shape changes, ID formats, pagination defaults.
- Config/CLI/env: renamed flags, default behavior flips, removed fallbacks, environment variable changes, logging levels tightened.
- Dependencies/platform: Python version requirement changes, dependency major bumps, `pyproject.toml`/`uv.lock` changes, removed or renamed extras.
- Persistence/data: migration scripts missing, data model changes, stored file formats, cache keys altered without invalidation.
- Docs/examples drift: examples still reflect old behavior or lack migration note.
Capture:
## Regression risk clues
- review mode and its evidence;
- intended release type/version and its evidence, or `unspecified`;
- minimum required release type and the contracts that establish it;
- planning recommendation or final-candidate compatibility verdict.
- Large refactors with light test deltas or deleted tests; new `skip`/`todo` markers.
- Concurrency/timing: new async flows, asyncio event-loop changes, retries, timeouts, debounce/caching changes, race-prone patterns.
- Error handling: catch blocks removed, swallowed errors, broader catch-all added without logging, stricter throws without caller updates.
- Stateful components: mutable shared state, global singletons, lifecycle changes (init/teardown), resource cleanup removal.
- Third-party changes: swapped core libraries, feature flags toggled, observability removed or gated.
For a materialized final candidate, read the checked-out package metadata, lockfile, and released API contract before deciding compatibility. Require the candidate branch, `HEAD`, intended version, contract baseline, and contract base commit to agree. Treat uncommitted release-owned files or unrelated changed paths as an inconsistent candidate rather than reviewing only the commit object.
## Improvement opportunities
## Audit runtime and package contracts
- Missing coverage for new code paths; add focused tests.
- Performance: obvious N+1 loops, repeated I/O without caching, excessive serialization.
- Developer ergonomics: unclear naming, missing inline docs for public APIs, missing examples for new features.
- Release hygiene: add migration/upgrade note when behavior changes; ensure changelog/notes capture user-facing shifts.
### Stage 1: broad discovery
## Evidence to capture in the review output
Scan the full diff for breaking changes, regressions, dependencies, package changes, persistence, error handling, concurrency, and release-polish signals. Read changed tests as behavioral evidence, including removed assertions, new skips, and uncovered failure paths.
- BASE tag and TARGET ref used for the diff; confirm tags fetched.
- High-level diff stats and key directories touched.
- Concrete files/commits that indicate breaking changes or risk, with brief rationale.
- Tests or commands suggested to validate suspected risks (include pass criteria).
- Explicit release gate call (ship/block) with conditions to unblock.
- `Unblock checklist` section when (and only when) gate is `BLOCKED`.
### Stage 2: contract and invariant proof
For each candidate:
1. Compare the released BASE contract with TARGET.
2. Identify the owning boundary through `.agents/references/README.md`.
3. Trace the changed value, identity, state, or side effect through every required consumer.
4. Check only the relevant parity and failure axes.
5. Promote the candidate only when the trace proves concrete impact.
| Changed surface | BASE-versus-TARGET audit |
|---|---|
| Public API | Exports, import identity, signatures, positional order, defaults, enums, and documented behavior |
| Runner and run items | Provider output, result items, stream events, session history, replay, handoffs, and `RunState` |
| Tool execution | Planning, approvals, guardrails, invocation, hooks, output conversion, persistence, cancellation, and cleanup |
| Conversation and sessions | First turn, follow-up, retry, filtering, handoff, compaction, interruption, and resume |
| Model/provider adapters | Settings resolution, request conversion, streaming terminals, provider data, errors, retries, and transport ownership |
| Persisted schemas/config | Serialized shape, supported versions, backward reads, usable migrations, defaults, env vars, and wire compatibility |
| Package boundary | Python support, dependencies, extras, version metadata, distribution contents, public imports, and wheel/sdist behavior |
Relevant axes include streaming/non-streaming, sync/async, fresh/resumed, client/server-managed state, success/error/cancellation, sequential/concurrent, and normal/partial/repeated cleanup.
When static inspection is insufficient, run the smallest identical BASE and TARGET public-path or installed-artifact probe. Do not run broad unit slices merely to accumulate passing evidence.
## Check high-signal change classes
- Public API: removed or renamed exports, changed signatures or positional order, default changes, new required values, or stricter validation.
- Protocol/config: request or response fields, enums, ID meaning, config flags, environment variables, or default behavior flips.
- Package/platform: Python support, dependency major changes, extras, package contents, or import side effects.
- Persistence: durable schema, stored format, backward reads, migration capability, cache identity, or resume behavior.
- Runtime: concurrency, cancellation, retries, timeouts, resource ownership, cleanup, swallowed errors, or changed exception types.
- Security: sensitive values in exceptions, logs, traces, telemetry, persisted state, or model-visible output.
Separate a released supported-path break with no usable migration, fallback, or compatibility path from a usable path that merely lacks documentation. Only the former is a compatibility blocker.
## Make every reported item actionable
For every risk finding or verified release consideration, capture:
- `Evidence`: concrete BASE-versus-TARGET source, contract, artifact, test, or probe evidence.
- `Impact`: one user or runtime consequence.
- `Files`: the affected paths.
- `Action`: an exact task or validation plus its pass condition.
Changed tests, missing tests, large diffs, and risky patterns are discovery signals rather than findings. If no executable unblock action exists, do not manufacture one. For a safe LOW consideration, use a release-handoff action that preserves exact compatibility, migration, opt-out, default, or version-bound wording.
## Audit documentation coverage
### Build the obligation inventory
Derive one row per user-facing obligation:
`contract change | affected users | required migration/default/opt-out/version wording | expected docs surface`
Include breaking behavior, major features, public API additions, defaults, provider/dependency bounds, durable state, and changed workflows.
### Discover current open docs PRs
- Use approved read-only GitHub access; never use `gh` or mutate GitHub in this repository.
- Refresh current open PR state for each review. Historical local refs, cached task context, and prior reports are not evidence of current coverage.
- Search with the intended or recommended version, release label, feature names, implementation PR links, branch names, and changed docs paths.
- Inspect candidate file lists, the complete latest PR diff, and current review discussion when it materially affects a coverage claim. Titles and descriptions are discovery hints only.
- Record each relevant PR number or URL and exact head SHA. Review competing or complementary PRs together when necessary.
- Keep open docs PR diffs outside `BASE...TARGET`; report them as follow-up coverage rather than shipped content.
- If the search fails or is incomplete, record the failing source and scope. Do not convert an unavailable search into `none found`.
### Map evidence and follow-up work
For every obligation, record:
- the covering PR and exact file/section, if any;
- missing or incorrect qualifiers when coverage is partial or stale;
- an exact post-release file, section, example or claim, and migration wording whenever coverage is not demonstrably complete;
- whether a live-site docs PR should remain unmerged until the package is released.
If GitHub access is unavailable, make the follow-up suggestions provisional and state what still needs verification. Missing docs never enters the unblock checklist by itself.
## Build the conditional Key Changes draft
When `../SKILL.md` requires the minor-release draft:
- derive three to seven highlights from verified user-visible themes rather than commits;
- state breaking status explicitly and put migration or fallback guidance first;
- preserve exact identifiers, defaults, provider/model/dependency versions, opt-outs, and compatibility bounds;
- cover the major feature areas without reproducing the generated `## What's Changed` list;
- link only stable published docs and mention open docs PRs separately in Documentation coverage;
- keep the block copy-ready even if the release is blocked.
## Final evidence inventory
- BASE tag, TARGET commit, and confirmation that remote tags and target were refreshed.
- Review mode, intended release type, minimum required type, and versioning verdict.
- High-level diff stats and key directories.
- Concrete findings and verified release considerations with Evidence, Impact, Files, and Action.
- Documentation-obligation inventory, current docs PR source and head SHA or search limitation, aggregate coverage, and exact post-release suggestions.
- Conditional copy-ready Key Changes draft for minor releases.
- Explicit ship/block call and an unblock checklist only when blocked.
- For a dedicated final-candidate checkout, confirmation that the exact checked-out `HEAD` and release-owned file contents were inspected and were clean. Keep the local checkout path out of copy-ready report text.
@@ -0,0 +1,208 @@
---
name: implementation-final-review
description: Perform the repository's risk-tiered independent final review before implementation completion. Use only when explicitly invoked or when repository instructions require it after behavior-impacting implementation work; audit the complete task diff, supported contracts, lifecycle and security boundaries, complexity, and tests before final verification.
---
# Implementation Final Review
Treat implementation and final review as separate phases. Reconstruct the change from the original requirement and the complete diff; do not defend the current design merely because it is implemented or tested.
## Non-negotiable guarantees
- Review the exact final task content, including committed, staged, unstaged, and task-owned untracked deliverables. The only exceptions are the narrowly verified final-gate type-erasure and base-advance closures in step 20, which preserve clean credit through explicit identity evidence and still require the complete final verification stack on the resulting fingerprint.
- Use the merge-base three-dot diff for patch ownership and the latest release tag separately for released compatibility.
- Require independent review. A same-context self-review cannot satisfy the clean-review gate.
- Freeze task-owned content while reviewers inspect a fingerprint.
- Treat an exact normalized file path in the task and component manifests as authoritative even when ignore rules match that file. An existing exact file takes literal precedence over Git pathspec metacharacters; use explicit `:(glob)` magic when pattern semantics are intended. A directory or glob pathspec never promotes ignored operational files into the review.
- Require the repository and every initialized submodule index to have no unresolved merge stages before fingerprinting.
- Require every initialized submodule, including nested submodules, to be clean and checked out at the commit recorded by its parent index before freezing review state. Stage reviewable gitlink pointer changes in the parent repository; fail closed on dirty worktrees, hidden index flags, ignored nested changes, and untracked embedded repositories. Reject cyclic or aliased submodule worktree graphs before recursive inspection.
- Require two consecutive identical observations of HEAD, status, diffs, task and repository workspace content, and component workspace content before accepting a review-state snapshot. Fail closed when repository state changes during capture.
- Reject task-owned filesystem entries that Git cannot represent as finite blobs, including FIFOs, sockets, and devices.
- Require packet, ledger, manifest, receipt, reviewer-output, and evidence paths to resolve to finite regular files. Canonicalize each path before opening. Verify the file type after opening and read content from that same descriptor; never authorize a path with `stat` and then reopen it. Reject evidence, receipt, and current-versus-prior ledger aliases by the opened descriptor's device and inode identity. Before accepting reviewer output or a reusable receipt, re-read the packet and current and prior ledgers and require their validated digests to remain unchanged; also re-read the indexed receipt before reporting it reusable. Materialize devices, FIFOs, sockets, or generated streams into regular files before validation.
- Bind every canonical root-owned evidence ID and inventory ID in the ledger with `contract_evidence_sha256` and `inventory_sha256`. Preserve those digest bindings across rounds so an existing ID cannot change content; inventory digests exclude only the ID itself so a renamed copy is not new semantic inventory.
- Count evidence or inventory as new for a canonical root only when its digest is absent from that root's prior ownership. A new root proposal requires an evidence digest absent from every canonical root and every distinct root proposed in the same output; it cannot reuse canonical inventory before implementer promotion. Require every credited receipt to have a unique content digest and exact command.
- Require unique keys and standard finite numbers in every JSON object. Duplicate keys, JavaScript-style `NaN` or infinity constants, and numeric exponents that overflow to infinity are invalid. Convert runtime numeric-size and nesting-limit failures into protocol errors instead of leaking parser exceptions.
- Give the two reviewers distinct normalized primary and high-risk specialties, and require every preflight command to be unique before any receipt can claim it.
- Encode `manifests.dependency_map` as an object that maps every component name to a nonempty array of exact `pathspec` and `reason` records. Reject prose-only claims, missing components, empty dependency sets, duplicate pathspecs, and extra record fields.
- Treat verification receipts, reviewer outputs, findings, root-cause evidence, unchecked-inventory records, and sibling-scenario scans as exact schemas. Reject unknown fields instead of ignoring potentially conflicting evidence.
- Repeat commit-hook inspection, every safe rewriting step, second-pass idempotence, and generated-provenance validation before every fingerprint freeze, including post-fix and delta-review rounds. Record the exact executable inspection and rewriting commands plus their results in packet preflight evidence; a prose label is not an executable command.
- Start independent reviewers without inherited conversation history. Fresh judgment does not require repeatedly replaying the implementer's context.
- Report only concrete, patch-scoped findings supported by requirements, released behavior, a durable boundary, explicit maintainer intent, user reliance, or a baseline regression.
- Never weaken final repository verification. Component-aware review invalidation reduces repeated review, not required build or test gates.
- Keep one task-global round ledger across pauses, compaction, handoff, renaming, resumed work, and post-completion feedback. Enforce a bounded budget for each active review cycle without discarding earlier history.
- Trust the active implementation control plane to record actual reviewer dispatches, waits, outputs, and verification executions. The local protocol helper validates those records but does not replace platform-issued cryptographic execution attestation.
## Post-completion feedback boundary
An implementation review cycle is complete only after its clean-review gate, mandatory verification, any requested local commit, and final user-facing handoff are complete. Seal that cycle at this boundary. A pause, compaction, context change, agent handoff before completion, or ordinary request to continue unfinished work does not create a new cycle or reset its budget.
A later user message containing concrete actionable review feedback starts a post-completion feedback cycle. The feedback message itself authorizes implementing that feedback and running the repository-mandated focused tests, delta review, verification, and local commit or amendment needed to return the task to a completed state. Do not ask for separate review-budget authorization merely because the sealed implementation cycle exhausted its budget.
Keep the same task identity and ledger, preserve its canonical root-cause history and clean credit for unchanged components, and append a default budget of two fingerprint rounds for the new feedback cycle. Ask the user again only when the feedback materially widens the requested contract, changes a released or durable compatibility boundary, requires authority beyond resolving the feedback, or exhausts the feedback-cycle budget.
## Workflow
Persist the current combined content fingerprint as `ledger.round_fingerprint` and bind it to the packet fingerprint. A same-round retry is valid only when that value and the authorized budget history match the immutable prior ledger snapshot; a changed fingerprint or newly authorized budget advances the round.
1. Finish the initial implementation and focused tests. Apply formatting before review when formatting can rewrite the diff. Inspect the actual final commit-hook configuration and run the exact safe, non-committing equivalent of every hook step that can rewrite task-owned content before freezing the first review fingerprint. Run each rewriting step until a second execution is content-idempotent. Normalize generated files before computing embedded hashes or provenance so the hook cannot invalidate them later. Record any hook step that cannot safely run before review; if that step later changes task content, apply the normal invalidation rules without exception.
2. Re-read the original user request and the current implementation scope contract. If no contract exists, record the required behavior, compatibility requirements, intentionally unsupported cases and failure behavior, and supported alternative or `none`.
3. Resolve the intended target and merge base. If a supplied target or base is not an ancestor of `HEAD`, compute their common merge base and treat `merge-base...HEAD` as the task-owned diff. Use the latest release tag separately when released compatibility is the relevant boundary. Include committed, staged, unstaged, and untracked changes that belong to the task.
4. Read the complete task-owned three-dot diff from the resolved merge base. Never treat target-only commits between the merge base and an advanced or divergent target as deletions or regressions introduced by the patch. Check integration with the current target separately when relevant; report an actual conflict or semantic incompatibility, not mere absence of target-side changes. Do not limit review to the latest fix or files named in prior feedback. Record a complexity delta: runtime lines changed, new state fields, new synchronization or ownership mechanisms, affected subsystems, and test permutations.
5. Run the baseline-reset gate before accepting the current design:
- Describe the required behavior without referring to branch-local helper types or state.
- Identify the nearest released/base pipeline that already owns the behavior.
- Compare patching the current diff with replacing task-owned branch-local machinery by a narrow change from the base implementation.
- Treat unreleased implementation and tests as disposable. Preserve unrelated or user-owned changes.
- Choose the narrower design unless concrete contract evidence requires the current machinery.
6. Select the relevant review dimensions below from the affected runtime boundaries and repository architecture references. Complete every selected dimension even after finding a blocker; the goal is a complete final review, not the first valid comment. Classify review risk before dispatch: normal when the change does not affect concurrency, cancellation, security, trust, persistence, durable state, released compatibility, package/runtime exports, protocol ownership, or cross-provider lifecycle; elevated when any of those boundaries changes or an earlier round produced P0/P1. Run the cheapest affected-boundary preflight broad enough to catch likely late fallout from a dependency, package surface, generated artifact, or cross-cutting runtime change. Prefer focused tests plus a narrowly targeted import, generated-surface, or static check. Run a targeted type check only when the change directly affects a typing boundary and the command is materially narrower than repository-wide `make typecheck`. Do not run repository-wide lint, typecheck, builds, integration suites, `make tests-review`, or `make tests` merely to enter or iterate through the review gate. Run the focused preflight once for a semantic state and rerun only affected checks after fixes.
7. Build the pre-dispatch evidence required by the changed boundary:
- For every changed public symbol, configuration field, event, serialized field, wire value, or documented caller-visible behavior, create a contract-surface inventory: producers and constructors; every consumer, forwarding branch, and adapter; default, missing, and invalid-value behavior; package exports and generated public surfaces when applicable; adjacent docs and examples; and caller-visible tests. Search adjacent contract surfaces even when they are absent from the diff. A required docs, example, export, adapter, or generated-surface update is a missing task deliverable, not out of scope merely because it is not yet in the manifest.
- For concurrency, cancellation, reentrancy, shared lifecycle state, or a check followed by an await before a side effect, create an await-boundary matrix. For each relevant operation, record the state snapshot, blocking or await point, events and operations that may run while suspended, durable or monotonic evidence retained, revalidation before each side effect, and resulting cancel, feedback, persistence, or cleanup action. Include source completion, a newer operation active with known and unknown identity, a newer operation that starts and completes while suspended, and failure or cancellation of the awaited action when those states are supported. If correctness depends on whether something ever happened, current active state is insufficient unless serialization proves it cannot be lost; require monotonic identity, generation, tombstone, or equivalent durable evidence.
- For protocol, persistence, or security changes, create the analogous authority/data-flow inventory from input through validation, storage, retry or replay, output, exceptions, logs, telemetry, and cleanup. Treat these as mechanical coverage artifacts, not implementation conclusions. The implementer must fill them from code and contract evidence before review; reviewers validate them independently against the complete diff and surrounding source.
8. Produce only concrete, patch-scoped findings that are reproducible from code, contract, documentation, or a focused probe. Do not report hypothetical extensibility or unrelated cleanup. Before concluding, account for every row in the contract-surface, await-boundary, and authority/data-flow inventories and every new or modified source of shared state. For a scenario outside the required behavior, run a differential check against the merge base or latest release and identify support evidence. Reachability through a public method, concurrent call, repeated call, host-language protocol, or third-party behavior is not by itself a supported contract.
9. Classify every finding before editing:
- required-behavior defect;
- released compatibility or durable-boundary defect;
- missing failure-path or adversarial coverage;
- unsupported neighboring case that should fail earlier;
- unnecessary machinery or duplicated source of truth;
- unrelated or unsupported suggestion to reject. Record the support basis for every actionable finding: original requirement, released documentation/example/typing/test, durable boundary, concrete maintainer intent or user reliance, or a regression where the same supported scenario succeeds at the baseline and fails in the patch. If none applies, do not fix or block on it; mark it unsupported/deferred.
10. Resume or create the task-global review ledger. Use the Codex task or thread ID as the stable task identity when available; otherwise generate one identity once. Persist that exact identity as `ledger.task_id` and require it to match the packet task identity. Store the ledger as an ignored operational file at a stable absolute path, include that path in every reviewer packet and handoff, and preserve the same file when work moves to another worktree. Never initialize a new counter merely because the task was paused, compacted, handed off, renamed, moved to another worktree, or resumed in another context. Start fingerprint round 1 only when the ledger has no prior round for this task; a same-fingerprint request for missing reviewer fields remains in the current round. The default autonomous budget for the initial implementation cycle is six fingerprint rounds. After that cycle has completed under the post-completion feedback boundary, concrete actionable review feedback starts a post-completion feedback cycle: treat the feedback message itself as authorization to append a default budget of two fingerprint rounds to the same ledger. Do not reset the round counter, canonical root-cause history, or clean credit for unchanged components. A continuation request without concrete new feedback remains in the existing cycle. Outside this post-completion feedback rule, only explicit user authorization may add another bounded budget, and the existing ledger and root-cause history must remain attached. The implementer assigns every root-cause ID once in the ledger using a stable canonical ID and includes the complete open and closed root set in every later packet. A reviewer must reuse one supplied canonical ID or propose exactly `NEW:<slug>` with content-new contract evidence; only the implementer may promote that proposal and assign canonical inventory in the ledger. Create one canonical manifest of every task-owned shipped path, including both sides of a rename and task-owned untracked files. Plans, review ledgers, packets, traces, temporary reports, and other workflow artifacts are operational-only by default even when repository policy requires creating them; include one only when the original requirement or repository policy explicitly makes that exact path a committed deliverable. Keep operational files outside the shipped manifest and account for them as repository exclusions. Keep the shipped manifest stable and update it only when task-owned deliverable paths actually change. Partition the manifest by the narrowest stable semantic boundaries that match the patch. In `openai-agents-python`, prefer components such as `api-contract`, `runstate-persistence`, `security-sandbox`, `session-lifecycle`, `integration-runner`, `tests-examples`, and `release-metadata` when present; do not create empty components or split tightly coupled files merely to preserve credit. Every changed deliverable must belong to exactly one component. For each component, record an exact semantic dependency-input pathspec set plus the reason each input can affect the component; do not use a coarse directory or prose-only `none` claim when build configuration, generated-surface owners, or shared runtime code are dependencies. When this skill's resources are available, prefer `python scripts/review_state.py --repo <worktree> --base <merge-base> --pathspec-file task.paths --component-pathspec-file api-contract=api-contract.paths ... --complete-diff-output <complete.diff>`; direct `--pathspec` and repeated `--component NAME=PATHSPEC` remain available for smaller diffs. Always generate the complete-diff artifact through `--complete-diff-output`; a standalone `git diff` omits ordinary untracked deliverables. Retain each component `content_fingerprint`, the combined `content_fingerprint`, and `repository_fingerprint`. Omit all pathspecs only when every repository change belongs to the task. Record the complexity delta and findings grouped by stable root-cause ID, severity, action, and whether each finding is new, repeated, or reintroduced.
11. Prepare one self-contained reviewer snapshot packet per round using `references/reviewer-brief.md` when available. Compute shared evidence once and reuse the same requirement, scope contract, target/base/head, manifests, fingerprint JSON, raw status, complete-diff command, preflight results, contract-surface inventory, state/data-flow inventories, and selected architecture excerpts for every reviewer. Assign stable IDs to every inventory row and evidence item. Populate every kind-specific inventory field documented by the reviewer brief; a summary-only inventory row is incomplete. Store the exact `review_state.py` JSON as the single artifact with `role: "review-state"`, store the raw output produced by that command's `--complete-diff-output` as the single artifact with `role: "complete-diff"`, store unfiltered `git status --porcelain=v1 -z --untracked-files=all` output as the single artifact with `role: "repository-status"`, and mark other artifacts with `role: "supporting"`. The packet's `review_state.evidence_id` points to the review-state artifact instead of copying fingerprint values, and `repository.status_evidence_id` points to the repository-status artifact. The repository fingerprint covers unfiltered status plus content identity for every changed path, including paths outside the task manifest. Assign every component and all three control artifacts to both reviewers. Packet preflight derives the combined and component fingerprints from the review-state artifact, requires the task and component manifests to match its pathspecs, requires `complete_diff_paths` to equal the task workspace exactly, requires the complete-diff digest to match its `complete_diff_sha256`, requires the status digest to match its unfiltered status fingerprint, and requires `repository.exclusions` to account exactly for every changed path outside the task manifest with a concrete reason. Every canonical ledger contract evidence ID must resolve to an indexed evidence artifact, and the ledger digest maps must bind the exact owned evidence and inventory content. Keep the task ID, task-global ledger path, and the immediately preceding round's immutable ledger snapshot plus SHA-256 digest in the active control plane outside the packet; never derive these authority arguments from the packet under validation, and never use the mutable current ledger as its own prior snapshot. Supply all four independently on every validator invocation after round 1. The validator requires packet, current-ledger, and prior-ledger identity to match those arguments, accepts only a same-round retry or an advance of exactly one round, reconciles the current round and remaining budget with the append-only authorized budget history, preserves the prior budget prefix, canonical root ownership, and owned-content digests, and assigns every inventory ID to exactly one canonical root. Keep the control-plane brief concise, with approximately 12 KB as a soft target; put larger raw diffs, logs, matrices, and reference excerpts in indexed evidence files and provide their exact paths plus SHA-256 digests. Exceed the target when compression would omit decision-relevant evidence, and record why. Populate every template field or mark it explicitly `none` or `not applicable`; do not dispatch an incomplete packet. Before dispatch, encode the packet index in the machine-readable schema documented by the reviewer brief and run `python scripts/review_protocol.py packet --packet <packet.json> --task-id <task-id> --ledger <ledger.json> --prior-ledger <prior-ledger.json> --prior-ledger-sha256 <sha256>` after round 1. Dispatch only when it exits successfully; use its emitted packet path, byte size, SHA-256 digest, exact combined fingerprint, component fingerprints, inventory IDs, and reviewer IDs as the launch record. Give each reviewer one ready-to-run fingerprint revalidation command and only the specialty assignment may differ. Do not ask reviewers to rediscover the workflow skill, implementation strategy, memory, release tag, manifest paths, helper location, or verification history. A reviewer may reopen primary source or released evidence when supplied evidence is inconsistent, appears wrong, or leaves a decision-relevant ambiguity, but reopening is not a substitute for missing mandatory packet contents and routine context reconstruction is implementer work.
12. Freeze task-owned content while reviewers for a round are running. Dispatch two independent reviewers concurrently on the same fingerprint. For normal risk, give them distinct primary dimensions that together cover the selected review surface. For elevated risk or a prior P0/P1, give them complementary high-risk specialties. Every reviewer sees the complete raw diff and may report blockers outside its specialty. When the platform supports context-fork control, dispatch every reviewer with `fork_turns: "none"`; never pass the implementer's accumulated conversation or use a full-history fork. Launch both reviewers before waiting. Wait for both reviewers in the round before editing so findings can be grouped and fixed as one batch. Use one event-driven wait of 240 seconds or the platform's multi-target first-completion wait. Do not poll with `list_agents`, separate short waits, progress questions, or no-op `followup_task` messages. If an event-driven wait times out while reviewers remain unfinished, issue another event-driven 240-second wait for the unfinished set; repeat without polling until a reviewer completes, needs attention, or no unfinished reviewers remain. After one reviewer completes, continue waiting only for the remaining reviewer with another event-driven 240-second wait, applying the same timeout rule. A reviewer process that fails before producing a protocol-valid output because of startup, service, content-filter, context, or tool infrastructure has produced neither a finding nor clean credit and does not advance `ledger.current_round` or consume another fingerprint round. Replace only that reviewer on the same frozen packet and assignment; a protocol-valid output already accepted from the other reviewer remains usable while the task fingerprint, packet, and assignment are unchanged. The original two-reviewer concurrent dispatch satisfies the round's concurrency requirement; the accepted peer output plus one independently launched replacement output on the identical packet and assignment form the required pair. If an independent replacement remains unavailable, report the gate as unavailable instead of counting an infrastructure failure as review evidence. Do not start any broad final repository gate while review is incomplete or finding-bearing. In `openai-agents-python`, defer `make lint`, `make typecheck`, `make tests-review`, `make tests`, repository-wide builds, examples runners, and integration suites until step 19 establishes clean review. Use reviewer wait time for non-mutating evidence consolidation, finding classification preparation, host-capacity inspection, or other task work that cannot change the frozen fingerprint; otherwise continue the event-driven wait without progress polling. During an iterative review round, run only focused checks that target the changed boundary. Do not run `make tests-review`, `make tests`, or repository-wide `make typecheck` during an iterative review round. Prefer an already successful same-fingerprint focused check over rerunning it, and never replay cumulative historical verification. Represent reusable focused success as a verification receipt containing the exact command, environment, exit status, non-mutation basis, and identical before/after combined, component, and repository fingerprints. Include its absolute path and SHA-256 digest in `verification.credited_receipts`; packet preflight validates every credited receipt. The focused check earns no final-gate credit; the exact clean-reviewed fingerprint must still pass the complete repository-required verification stack. Set `verification.eligible_concurrent_gates` to `none` and list every deferred broad gate in `verification.deferred_gates`. Keep `$pr-draft-summary` deferred until clean review and final-gate evidence apply to the final fingerprint. Do not introduce a repository lock, host-wide mutex, sentinel file, or user-triggered `finalize` step.
13. Verify the combined and component fingerprints before accepting reviewer output. If reviewed runtime or contract-bearing content changed, discard the affected review evidence unless the change later qualifies for one of the narrow final-gate closures in step 20. If only `repository_fingerprint` changed, accept the review only for unambiguous non-semantic bookkeeping such as staging, unstaging, or committing identical task-owned content. Preserve clean credit for a semantic component only when its fingerprint, requirement rows, assertions about runtime behavior, dependency inputs, and risk tier are all unchanged, except for a narrowly recorded step 20 closure. Require two concurrent independent delta reviews of every other changed or dependency-invalidated component plus its relevant boundaries with unchanged components. Any ambiguity invalidates the affected clean credit. Do not invalidate unrelated components solely because a neighboring file or coarse directory changed.
14. Apply a packet-and-output acceptance gate before counting findings or clean credit. Verify that every mandatory reviewer-brief field was populated or explicitly marked `none` or `not applicable`; missing packet evidence cannot be reconstructed by the reviewer and earns no clean credit. Require one structured JSON object with the documented schema: verdict, exact combined and component fingerprints, checked and unchecked inventory IDs, high-risk dimensions, focused probes, remaining uncertainty, findings, sibling-scenario scan, inspection call count, and inspection-budget reason when applicable. Every probe record must contain the exact executable command that ran, or the complete tool name and arguments for a non-shell probe. Reject prose-only labels, omitted arguments, and placeholders such as `<focused probe>` as incomplete evidence. Run `python scripts/review_protocol.py reviewer-output --packet <packet.json> --reviewer <reviewer-id> --output <output.json> --task-id <task-id> --ledger <ledger.json> --prior-ledger <prior-ledger.json> --prior-ledger-sha256 <sha256>` for each output after round 1 and accept no finding or clean credit when it fails. The validator rejects fingerprint drift, missing assignment coverage, malformed probes, unknown bare root IDs, root evidence IDs absent from the packet's indexed evidence or inventory, sibling scans that use a renamed root or unknown inventory, JSON booleans in integer fields, changed prior evidence or inventory bindings, reopening a closed canonical root without content-new evidence or semantic inventory, and distinct new roots that reuse one evidence digest. When a reviewer discovers new evidence after dispatch, add and digest it in the frozen packet and rerun packet preflight in the same fingerprint round before requesting corrected output. A bare `clean`, generic checklist, malformed object, or response that does not account for the assigned contract/state/data-flow artifacts is incomplete and earns no clean credit; request only the missing fields or coverage on the same frozen fingerprint rather than restarting the whole review. When two specialists are used, combine their declared ID coverage and reject the round if any assigned inventory row or selected high-risk dimension remains unreviewed. Use approximately 12 source-inspection tool calls per reviewer as a soft budget. A reviewer may exceed it when unresolved decision-relevant uncertainty requires more evidence, but must state the reason; never trade correctness for the budget.
15. Classify and validate all findings from the round before editing, then fix every actionable finding as one batch. Before choosing the fix, update the complete relevant inventory or matrix with the discovered transition or surface and solve the root cause across all populated rows; do not patch only the reported interleaving. The implementer owns `$implementation-strategy` and supplies its current scope contract in the packet. Reviewers inherit that contract and must not rerun the strategy workflow. Rerun it only in the implementer context when a fix changes supported behavior, compatibility, state, ownership, protocol paths, test permutations, or triggers a complexity reset; otherwise record `scope contract unchanged` and avoid reconstructing the same strategy. Add caller-visible regression coverage, not tests that only mirror helper structure. Run focused verification only for affected boundaries and dependency-invalidated checks.
16. Treat a second related finding in one root-cause group as a closure gate. Stop local patching, run the complexity reset once, scan the complete inventory for sibling scenarios, and record one root-level disposition: replace the design, narrow or reject unsupported behavior, or escalate a concrete unresolved contract decision. After the disposition is implemented and reviewed, mark the canonical root-cause ID closed. Do not reopen it for another local patch without content-new contract evidence or semantic inventory; reject aliases, renamed or copied content, and bare unknown IDs instead of treating them as new roots. If it cannot be closed coherently, escalate instead of consuming more rounds.
17. Increment the fingerprint round, repeat the full commit-hook parity gate from step 1, and review the complete post-fix diff with fresh context. Continue review -> validate all findings -> batch fixes -> focused verification -> hook parity -> review without waiting for another user prompt.
18. Apply the non-convergence guard before another local fix:
- If the same root-cause group produces another P0/P1 after a complexity reset, return to the merge base and replace task-owned branch-local machinery with the narrowest coherent implementation.
- If runtime diff size, state fields, ownership modes, or test permutations grow materially for two consecutive rounds, do not call that convergence merely because each finding is local. Re-run the baseline-reset gate.
- If the same root-cause group produces actionable findings in three finding-bearing rounds, or the narrower reimplementation still produces the same root-cause P0/P1, escalate early rather than consuming the round budget.
- If four rounds complete without a shrinking or stable diff and falling finding severity, escalate early.
19. Stop successfully only after the required clean-review condition is met on the exact reviewed content and every required reviewer output has passed the acceptance gate:
- same-round infrastructure replacement: the original two-reviewer dispatch was concurrent, and the accepted pair consists of the unchanged protocol-valid peer output plus one independent replacement output accepted under step 12;
- normal-risk change: two independent clean reviews of the same fingerprint, launched concurrently;
- elevated-risk change or any loop that produced a P0/P1 finding: two independent clean reviews of the same fingerprint with complementary high-risk specialties, launched concurrently.
- component-only post-review edit: clean credit for every unchanged component plus two concurrent clean independent delta reviews covering all changed components and their runtime boundary.
- verified final-gate type-erasure closure satisfying every condition in step 20: preserve the prior clean set without a new fingerprint round or reviewer dispatch, then run the complete final verification stack on the resulting fingerprint.
- verified base-advance closure satisfying every condition in step 20: preserve the prior clean set without a new fingerprint round or reviewer dispatch, then run the complete final verification stack on the replayed fingerprint.
20. After the clean-review condition is met, confirm that the diff and component fingerprints remain stable, then check observable host capacity before starting the repository's code-change verification. Use available read-only task or process evidence; treat another repository-wide test, typecheck, build, examples runner, or integration command already active on the same host as concrete contention. When contention is visible, continue useful non-heavy work or an event-driven wait and check again later. Do not create or wait on a repository lock, host-wide mutex, or sentinel file, and do not require a user-triggered `finalize` message. If host telemetry is unavailable, do not block solely because capacity cannot be measured. Once capacity is available, run every mandatory command in the repository-required order against the exact clean-reviewed fingerprint, or against the recorded resulting fingerprint of a verified closure below. Record combined, component, and repository fingerprints immediately before and after the final stack. Accept final verification only when every command succeeds, execution does not mutate that final content or create an ambiguous repository-state change, and all fingerprints still match. Classify any final-gate replay or edit before invalidating review evidence:
- Verified base-advance closure: when the intended target advances after clean review, preserve the existing clean set without a new fingerprint round or reviewer dispatch only when every condition below holds. This exception grants no final-verification credit; rerun every mandatory final gate on the replayed fingerprint and regenerate the complete PR handoff.
- The replay or rebase is conflict-free and requires no manual task-content edit. The old and new `review_state.py` artifacts have byte-identical task and component `workspace` arrays, and their `tracked_diff_sha256` values are identical; compare these fields directly because content fingerprints intentionally include the resolved base.
- Before the original review, each component recorded exact semantic dependency-input pathspecs and reasons. The complete upstream delta from old base to new base changes no task-manifest path, dependency-input path, selected architecture reference, generated-surface owner, or applicable build, test, lint, format, hook, lockfile, or package configuration input.
- The original requirement, scope contract, inventory rows, assertions about runtime behavior, selected review dimensions, risk tier, and released compatibility boundary are unchanged. Focused checks affected by integration with the new base pass.
- Record the old and new base, head, combined/component/repository fingerprints, exact upstream changed-path list and diff digest, dependency-input pathspecs, comparison commands, and results in the task-global ledger. This closure consumes no fingerprint round and creates no reviewer packet.
- Any path overlap, changed configuration or dependency input, conflict, manual resolution, changed diff digest, missing prior dependency map, or uncertainty falls through to a fresh review round on the new base. A coarse directory-disjointness claim is insufficient.
- Verified type-erasure-only edit: preserve the existing clean set without an independent delta review only when every condition below holds. This exception consumes no fingerprint round and requires no reviewer packet, but it does not grant final-gate credit; restart every mandatory final gate on the resulting fingerprint.
- The edit is made only after the final stack reports a formatter, linter, or static-type-checker failure, and the failure does not reveal unresolved runtime or contract uncertainty.
- The exact delta is limited to importing `cast` directly from the standard-library `typing` module, wrapping one unchanged private implementation expression as `cast(<type>, <original expression>)`, and formatter-only whitespace. The imported name is not rebound or used elsewhere.
- The edit does not change expression evaluation order or count, exception propagation, a public or exported annotation or signature, a decorator, runtime branch, constant, test assertion, generated surface, documentation, scope contract, inventory row, component dependency, or risk tier.
- The implementer records the before and after fingerprints, the exact delta, the original final-gate failure, and the runtime-identity basis that `typing.cast` returns its value unchanged. Targeted formatting, lint, type checking, and affected focused tests must pass before restarting the full stack.
- Any additional token change, behavioral-equivalence argument beyond this exact `typing.cast` shape, or uncertainty about the conditions above falls through to the normal runtime-edit rule and requires the applicable independent delta review.
- Runtime, public API, behavior-impacting docs, runtime-behavior assertions, or scope-contract change: invalidate the applicable clean set, rerun pre-review validation, and restart review with fresh reviewers before rerunning every required final gate.
- Tests or examples only: preserve clean runtime evidence only when the runtime fingerprint is identical and the delta does not change required behavior, compatibility, runtime-behavior assertions, or the scope contract. Run focused verification and a component delta review using the risk tier and clean-review conditions from step 19, covering test correctness, accidental contract expansion, and the runtime boundary, then rerun the required repository gates. Treat an example or expectation edit as behavior-impacting unless concrete evidence shows otherwise.
- Release metadata only: preserve runtime and test evidence when their fingerprints are identical. Revalidate the metadata and independently review any changed behavioral claim, then rerun applicable final gates.
- Operational artifact only: exclude it from deliverable manifests and do not invalidate review evidence. Completion requires the final combined fingerprint to be exactly composed of component fingerprints with applicable clean, delta-review, verified base-advance-closure, or verified type-erasure-closure evidence and every mandatory repository gate to pass on that final content. Invoke `$pr-draft-summary` last, only after review and verification evidence apply to the final fingerprint.
21. Stop the autonomous loop when the active cycle reaches its current budget: six fingerprint rounds for the initial implementation cycle or two for a post-completion feedback cycle. This is an absolute cap for the active cycle, not a target, and it does not reset when execution pauses or context changes. Do not call the implementation complete. Summarize the remaining blockers, recurring root causes, complexity growth, attempted fixes and resets, current verification state, and the concrete decisions available to the user; then ask the user whether to narrow scope, split the change, accept a stated risk, redesign, or explicitly authorize another bounded budget. When concrete actionable feedback arrives after a successfully completed and sealed cycle, append the feedback cycle's default two-round budget to the same ledger without another authorization prompt. In every other case, append a user-authorized budget to the same ledger rather than replacing its history.
Maintain one compact round ledger throughout all review cycles and persist it as a durable, task-global artifact:
`Round | component fingerprints | root-cause groups | highest severity | complexity delta | action | clean credit`
Persist enough task identity, used and authorized round budgets, cycle boundaries, fingerprints, root-cause closure state, and clean credit to resume without reconstructing prior rounds. Update it only at a meaningful state transition: round start, accepted finding batch, complexity reset, clean result, verification result, sealed completion, or post-completion feedback-cycle start. Do not emit repeated waiting messages when neither reviewer state nor repository content changed.
## Independent reviewer
An independent review uses a fresh no-history context that did not implement the fingerprinted content and is not given prior reviewer findings or implementer conclusions. Prefer a distinct agent and set `fork_turns: "none"` when the platform exposes that control. A same-context self-review or full-history fork is not independent and cannot satisfy the clean-review gate.
- Give the reviewer the original requirement, implementation scope contract, base and head identifiers, canonical component manifest and fingerprints, raw repository state, and relevant architecture references.
- Give the reviewer the precomputed contract-surface and await-boundary or authority/data-flow inventories. These are coverage maps, not conclusions; require the reviewer to validate every row against the raw diff and surrounding source.
- Tell the reviewer which identifier is the intended target and require an explicit merge-base calculation. When target and head diverge, provide or request a three-dot diff; do not present a two-dot target-to-head diff as the patch.
- Do not give the reviewer the implementer's conclusions, suspected bugs, intended fixes, or a list of expected findings.
- Ask for exactly one read-only review round. The reviewer must not edit or stage files, run the autonomous review loop recursively, spawn another reviewer, or perform the final repository verification. The implementer owns finding validation, edits, loop control, and final verification.
- Give every reviewer for a round the same review-state fingerprint and keep the diff frozen until all of them finish. Reject output produced from a different or changing state instead of merging partial observations across revisions.
- Give every reviewer the compact self-contained control-plane brief, indexed evidence paths and digests, and one exact revalidation command. If any mandatory packet field is neither populated nor explicitly marked `none` or `not applicable`, the reviewer must report it and cannot return a creditable clean verdict. Tell reviewers not to inspect memory, rediscover workflow skills, rerun implementation strategy, search for the fingerprint helper, or rediscover the release tag unless supplied evidence is inconsistent or decision-relevant. Reopening source cannot replace missing packet contents. This preserves fresh judgment while avoiding repeated setup work.
- Use fresh reviewers for every round when possible. Do not reveal findings or conclusions from prior rounds; provide only the updated requirement, scope contract, raw final diff, component manifest, and relevant references.
- Use two concurrent fresh reviewers for every round. For the high-risk conditions in step 12, assign complementary high-risk specialties while requiring each reviewer to inspect the complete diff. Both reviewers of the same unchanged diff are one fingerprint round. Do not duplicate broad test execution.
- Concurrent reviewers receive the same fingerprint and raw context but different primary specialties. They must not communicate during the round.
- Give the reviewer existing verification commands and results as raw evidence. The reviewer should inspect code and tests, then run only focused probes needed to resolve a decision-relevant uncertainty. A probe must be demonstrably non-mutating or run in an isolated temporary checkout; any mutation of the reviewed worktree invalidates the round. Do not rerun the repository's broad test, typecheck, lint, build, or integration suites merely to reconfirm the implementer's evidence; the implementer runs the complete stack once after the clean-review gate.
- Require the structured JSON output from the reviewer brief. `clean` alone is never sufficient: the reviewer must return the exact packet SHA-256 and fingerprints, checked and unchecked inventory IDs, high-risk dimensions checked, probes or `none`, unresolved uncertainty or `none`, findings, sibling-scenario scan, and inspection-budget accounting.
- After fixes, review the exact final diff again. Preserve earlier clean credit only under the explicit component-delta rule; do not infer that a change is isolated merely from its file location.
When an independent reviewer is unavailable, rebuild context from the original request, scope contract, source, and complete diff before a best-effort self-review. Explicitly discard incremental-review assumptions, label the result non-independent, and do not count it toward the clean-review gate. Report the unavailable gate at handoff instead of silently weakening it.
## Review dimensions
Choose dimensions based on the changed boundary; do not mechanically invent findings for every item.
### Requirement and scope
- Verify that the smallest required caller-visible behavior works.
- Identify nearby constructible cases and confirm they are either intentionally supported or rejected before side effects.
- Require contract evidence before treating repeated, concurrent, reentrant, malformed, wrapped, or cross-provider combinations as blockers. Reproduce the same supported scenario on the baseline when claiming a regression.
- Check whether tests accidentally turn implementation permutations into public contract.
- Map every new abstraction, state field, branch, dependency, and cross-module change to a requirement, supported contract, or verified risk.
### Compatibility and identity
- Compare released public signatures, field order, imports, names, serialized values, configuration, and wire behavior.
- Preserve exact caller-visible identity or spelling unless transformation is required.
- Distinguish unreleased branch-local machinery from released or durable compatibility boundaries.
- For every new or modified public field, enumerate all construction, forwarding, and consumption branches. Verify that normal, specialized, default, missing-value, and error paths either honor the field or reject it according to one coherent contract; do not validate only the motivating branch.
- Search public docs, examples, docstrings, configuration reference, and release metadata for claims made stale by the behavior change. Missing documentation can be an actionable omission even when no documentation file is in the diff.
### Lifecycle and failures
- Trace ownership from acquisition through success, failure, cancellation, retry, replacement, and cleanup.
- When shared lifecycle state changes, build a compact operation-state matrix before concluding. Cover each affected public mutating operation against never-started, partial-failure, active, cleanup-in-progress, and terminal states as applicable.
- Trace repeated sequential calls and every relevant pair of overlapping public mutating operations. Identify the linearization point or manager-owned serialization mechanism; do not infer safety from per-resource deduplication alone.
- Check repeated cancellation, partial initialization, cleanup failure, retry through every supported public entry point, and primary-exception preservation.
- State the final survivor invariant: which tasks, workers, processes, sessions, listeners, files, or remote resources may remain.
- Review ordering when several validations or cleanup actions can short-circuit one another.
- Audit every check-await-side-effect sequence. State may change during the await; require revalidation or prove manager-owned serialization before cancellation, feedback, persistence, or cleanup.
- Distinguish current state from historical evidence. If a stale-result guarantee depends on whether a newer operation ever started, an active pointer that later returns to `None` cannot prove absence; use or require monotonic evidence unless the operation is serialized.
### Security, trust, persistence, and protocol
- Trace caller-controlled data through logs, exceptions, causes, contexts, telemetry, model-visible output, and persisted state.
- Treat serialized state as authority only when the supported trust boundary explicitly allows it.
- Check fail-closed behavior for malformed or ambiguous sensitive inputs without returning or retaining the original value.
- Verify protocol capability ownership, pagination termination, cache ownership, retry and replay safety, wire validation, and tool or call identity when affected.
### Behavioral parity
- Compare streaming and non-streaming, sync and async, initial and resumed, direct and wrapped, and provider-specific paths when the requirement crosses them.
- Verify that one path does not silently ignore, reshape, or hard-fail data that another path supports.
### Tests and generated public surfaces
- Prefer public-boundary or caller-visible adversarial tests.
- Exercise the highest stable caller boundary that reproduces the required behavior. A helper-only test is insufficient when a caller transforms the input, owns the lifecycle, or determines the observable result before or after invoking that helper.
- Require expected values and failure signals to come from the contract, a worked example, a baseline, or another independent oracle. Do not accept an assertion that recomputes the expected result with the same logic as the implementation.
- Use a narrower internal boundary when a lifecycle, concurrency, provider-wire, or malformed-stream scenario cannot be controlled reliably through a public entry point, and record why that boundary is necessary.
- Add controlled interleavings for concurrency instead of relying only on sequential tests.
- Test the required behavior, the nearest supported alternative, and one representative input per unsupported category.
- Do not accept passing existing tests as proof when they encode the same assumptions as the implementation.
- Import through intended consumer entry points and verify generated or distribution artifacts when public package behavior changes; runtime tests alone do not prove the published surface.
## Complexity reset
Run a complexity reset when related findings keep expanding the same design, a narrow requirement requires recursive or cached classification, tests enumerate mechanics, representations are inferred in multiple places, or the diff spreads unexpectedly across subsystems.
1. Stop addressing findings one by one.
2. Group them by root cause and restate the original required behavior.
3. Compare the full diff with the merge base or release boundary.
4. Delete branch-local machinery that is not required.
5. Reuse the nearest existing source-of-truth pipeline.
6. Narrow unsupported behavior and reject it before side effects with a supported alternative when one exists.
7. Rebuild tests around caller-visible invariants and representative negative cases.
8. Compare the replacement's runtime and test complexity with both the previous round and the merge base. A reset that only renames or redistributes a growing state machine is not a reset.
Review-state workspace entries must use the exact key set emitted for their `file`, `symlink`, `gitlink`, `directory`, or `missing` kind; incomplete or unknown fields fail before dispatch.
For reusable verification credit, a receipt command must exactly match a structured command in `verification.preflight_results`; a different successful command cannot inherit verification credit.
## Review output
Return exactly one JSON object using the schema in `references/reviewer-brief.md`. Put the verdict in `verdict`; put each actionable finding in `findings` with its priority, title, location, concrete failure scenario, user-visible consequence, support basis, baseline-versus-patch evidence when applicable, smallest safe correction, and stable root-cause ID. Account for every assigned inventory ID and keep unverified runtime uncertainty explicit. Do not claim implementation completion until both structured clean reviews and required verification apply to the exact final state.
@@ -0,0 +1,7 @@
interface:
display_name: "Implementation Final Review"
short_description: "Review implementation diffs before completion"
default_prompt: "Use $implementation-final-review to audit this implementation from first principles before calling it complete."
policy:
allow_implicit_invocation: false
@@ -0,0 +1,157 @@
# Independent Reviewer Brief
The ledger contains a `round_fingerprint` equal to the packet's combined content fingerprint. A same-round retry must preserve the immutable prior snapshot's `round_fingerprint` and authorized budget history; a changed fingerprint or newly authorized budget requires advancing exactly one round.
Every packet, ledger, manifest, receipt, reviewer-output, and evidence path must resolve to a finite regular file. Canonicalize each path before opening. Verify the file type after opening and read content from that same descriptor; a path-level `stat` must not authorize a later reopen. Evidence artifacts and credited receipts must have unique opened-file device and inode identities, and current and prior ledgers must have distinct identities. Before accepting reviewer output or a reusable receipt, the validator re-reads the packet and current and prior ledgers and requires their validated digests to remain unchanged; it also re-reads the indexed receipt before reporting it reusable. Materialize devices, FIFOs, sockets, or generated streams before validation.
An evidence or inventory ID is new for a canonical root only when its content digest is absent from that root's prior ownership. The ledger binds every canonical root-owned evidence ID in `contract_evidence_sha256` and every inventory ID in `inventory_sha256`; prior bindings are immutable. Inventory digests exclude only the ID itself, so renaming a copied row does not make it new. A new root proposal requires an evidence digest absent from every canonical root and every distinct root proposed in the same output; it cannot reuse canonical inventory before implementer promotion. Credited receipt content digests and exact commands must be unique.
Every JSON object must use unique keys and standard finite numbers. Duplicate keys, JavaScript-style `NaN`, `Infinity`, and `-Infinity` constants, and numeric exponents that overflow to infinity are invalid. Runtime numeric-size and nesting-limit failures are protocol errors rather than raw parser exceptions.
Generate review state only from two consecutive identical repository observations. A changed HEAD, status, diff, task or repository workspace, or component workspace invalidates the capture. Task-owned FIFOs, sockets, devices, and other entries that Git cannot represent as finite blobs are invalid.
Use this template to prepare one self-contained, factual snapshot packet per fingerprint round. Fill every field or mark it explicitly `none` or `not applicable`; do not dispatch an incomplete packet. Fill it once, reuse the shared body byte-for-byte for every reviewer, and vary only the final specialty assignment. Keep this control-plane brief near 12 KB when practical. Store larger evidence in indexed files and reference each file by exact path and SHA-256 digest. Do not omit decision-relevant evidence merely to meet the soft size target. Do not include implementer conclusions, suspected bugs, prior findings, or intended fixes.
The verified final-gate type-erasure and base-advance closures defined in `SKILL.md` step 20 do not create a fingerprint round, reviewer packet, or reviewer assignment. For a type-erasure closure, record its exact delta, before and after fingerprints, final-gate failure, runtime-identity basis, and focused verification in the task-global ledger and final verification evidence. For a base-advance closure, record the old and new base, head, fingerprints, byte-identical task and component workspace evidence, identical tracked-diff digest, complete upstream changed-path list and diff digest, exact dependency-input pathspecs, and focused integration checks. If every condition for the applicable exception is not mechanically established, prepare the normal delta-review packet instead.
## Shared evidence
- Original requirement:
- Implementation scope contract:
- Required behavior:
- Compatibility requirements:
- Intentionally unsupported cases and failure behavior:
- Supported alternative or `none`:
- Intended target:
- Resolved merge base:
- HEAD:
- Latest release boundary when relevant:
- Risk tier and reason:
- Task-global ledger path, task identity, current round, and remaining authorized budget:
- Canonical root-cause ledger (`ID | open/closed | inventory IDs | contract evidence IDs`):
- Canonical task manifest (an exact normalized file entry remains authoritative when ignored; directory and glob entries do not promote ignored files):
- Component manifests:
- Semantic component dependency map (`component | exact base pathspecs | invalidation reason`):
- Combined, component, and repository fingerprints:
- Exact fingerprint revalidation command:
- Unfiltered repository-status artifact and explicit exclusions outside the task manifest:
- Complete three-dot diff command using `review_state.py --complete-diff-output` so task-owned untracked files are included:
- Indexed evidence manifest (`ID | role | exact path | SHA-256 | purpose`):
- Focused preflight commands and results, including idempotent commit-hook parity with the exact executable hook-inspection commands plus second-pass results for every content-rewriting step before this fingerprint freeze:
- Same-fingerprint verification already credited, or `none`:
- Verification receipt path and SHA-256 descriptors for credited checks, or `none`:
- Eligible concurrent final-gate commands: `none` (required because broad final gates start only after clean review):
- Broad final gates deferred until clean review:
- Selected architecture references or exact relevant excerpts:
## Machine-readable preflight
Store the shared packet index as one JSON object and validate it before dispatch:
`python scripts/review_protocol.py packet --packet <packet.json> --task-id <task-id> --ledger <ledger.json> --prior-ledger <prior-ledger.json> --prior-ledger-sha256 <sha256>`
The active implementation control plane is trusted to record real reviewer dispatches, waits, outputs, and verification executions. The local helper validates completeness, digests, identity, state transitions, and reuse against those records; it does not provide cryptographic attestation against a malicious control plane that fabricates every input. Platform-issued signed execution provenance is intentionally unsupported here and requires a separate trusted service.
The packet object uses integer `schema_version: 1` and contains these required top-level fields: `packet_overage_reason`, `task`, `scope_contract`, `repository`, `ledger`, `manifests`, `review_state`, `verification`, `architecture_references`, `evidence_artifacts`, `inventory`, `selected_high_risk_dimensions`, and `reviewer_assignments`. Mirror the factual fields above rather than adding conclusions. Encode `verification.preflight_results` as an array of exact, unique `command` and `result` objects; use an empty array when no focused preflight ran. Set `verification.eligible_concurrent_gates` to the exact string `none`, and list the repository-wide lint, typecheck, test, build, examples, and integration gates that remain applicable in `verification.deferred_gates`; packet preflight rejects any attempt to overlap a broad final gate with review. Store exactly one evidence artifact with `role: "review-state"` containing the unmodified `review_state.py` JSON, exactly one with `role: "complete-diff"` generated by the same command's `--complete-diff-output`, and exactly one with `role: "repository-status"` containing unfiltered porcelain-v1 `-z` status. The `review_state` packet object contains exactly `evidence_id`, which names the review-state artifact, and the exact revalidation command; extra copied fingerprint or state fields are invalid. The repository object names the status artifact with `status_evidence_id` and lists every changed path outside the task manifest in `exclusions` with a concrete reason. Encode `manifests.dependency_map` as an object whose keys exactly match the component names and whose values are nonempty arrays of exact `pathspec` and `reason` records with unique pathspecs. Each pathspec must cover a semantic, generated-surface, hook, or build/test configuration input that can invalidate the component, and its reason must state why; prose-only or empty dependency claims cannot support a later base-advance closure. Use two reviewer assignments whose combined IDs cover every inventory row and selected high-risk dimension; after trimming and case-folding labels, their primary and high-risk specialties must not overlap. Every reviewer assignment must include every component boundary and all three control artifacts; supporting evidence may remain specialty-specific. The validator derives fingerprints from the digested review-state artifact, requires repository base and head to match it, requires the task and component manifests to match its pathspecs exactly, requires `complete_diff_paths` to match the task workspace exactly, requires the complete-diff artifact digest to equal its `complete_diff_sha256`, requires the status digest to equal its unfiltered status fingerprint, and requires exclusions to account exactly for every unfiltered changed path outside the task workspace. It reports the packet's actual path, byte size, packet and current-ledger SHA-256 digests, review-state path, fingerprint, components, inventory IDs, and reviewer IDs; copy that output into the dispatch record. If the packet exceeds 12 KiB, replace `packet_overage_reason: "none"` with the decision-relevant reason it could not be split further.
The ledger contains `task_id`, `round_fingerprint`, `authorized_round_budgets`, `current_round`, `remaining_budget`, `root_causes`, `contract_evidence_sha256`, and `inventory_sha256`. Supply the task ID and absolute task-global ledger path independently on every validator command. For every round after round 1, also supply the immediately preceding round's immutable ledger snapshot and its SHA-256 digest from the control plane; never derive either argument from the packet under validation. The immutable snapshot must be a distinct file, not the mutable current ledger under another argument. The validator requires the packet, current ledger, and prior ledger identity to match those control-plane arguments. It requires `round_fingerprint` to match the packet fingerprint, `current_round` plus `remaining_budget` to equal the sum of the positive integer budget history, the current budget history to preserve the prior prefix, the current round to equal the prior round for a same-round retry or advance by exactly one, a same-round retry to preserve the prior `round_fingerprint`, every prior canonical root and its ownership and digest bindings to remain present, and the current ledger file's JSON object to match the packet ledger exactly. Each `ledger.root_causes` entry contains `id`, `status`, `inventory_ids`, and `contract_evidence_ids`. Every root must own at least one inventory ID, and each inventory ID has exactly one canonical root owner. The two digest maps must bind exactly the currently owned IDs and match the indexed artifact bytes and semantic inventory rows. Every contract evidence ID must resolve to an `evidence_artifacts[].id`; the ledger cannot establish evidence authority with an unindexed string. The implementer owns canonical IDs. Reviewers must reuse one supplied ID or propose `NEW:<lowercase-slug>` with evidence content not already owned by any canonical or distinct proposed root; reviewers must not mint a renamed bare ID or reuse canonical inventory for a new proposal. Only the implementer promotes a proposal into the ledger.
Each credited verification receipt contains exactly integer `schema_version: 1`, integer `exit_status: 0`, `command`, `environment`, `non_mutation_basis`, and exact `before` and `after` objects with `combined`, `components`, and `repository` fingerprints. Unknown receipt fields are invalid, and JSON booleans are not integers for protocol purposes. Add an object with its absolute `path` and `sha256` digest to `verification.credited_receipts`; packet preflight rejects replacement, a failed command, task or repository-state drift, or before/after drift. The standalone check accepts only a receipt path already indexed by the validated packet; it does not grant credit to an arbitrary same-fingerprint file:
The validator recomputes the content, component, and repository fingerprints from the complete typed workspace entries in the review-state artifact and rejects an incomplete or unknown key for any workspace kind or a non-partitioning component workspace. A credited receipt's exact command must also appear in `verification.preflight_results`; unrelated successful commands are ineligible for credit.
`python scripts/review_protocol.py receipt --packet <packet.json> --receipt <receipt.json> --task-id <task-id> --ledger <ledger.json> --prior-ledger <prior-ledger.json> --prior-ledger-sha256 <sha256>`
## Contract-surface inventory
Give every row a stable ID. Use one row per changed public symbol, configuration field, event, serialized field, wire value, or documented behavior.
`ID | surface | producers/constructors | consumers/forwarding branches/adapters | default/missing/invalid behavior | package exports/generated public surfaces | adjacent docs/examples | caller-visible tests`
Encode those columns in each `kind: "contract"` inventory object as `surface`, `producers`, `consumers`, `behavior`, `exports`, `adjacent`, and `tests`. Each field must be a nonempty string; use `none` or `not applicable` only when that is the explicit reviewed value.
Include adjacent surfaces found outside the current diff. If a required update is absent, add it to the task manifest before freezing the review.
## Await-boundary or authority inventory
For concurrency, cancellation, reentrancy, or lifecycle state:
`ID | operation | state snapshot | await/blocking point | events/operations possible while suspended | monotonic evidence retained | revalidation | side effects/invariant`
Encode those columns in each `kind: "await-boundary"` inventory object as `operation`, `state_snapshot`, `blocking_point`, `suspended_events`, `monotonic_evidence`, `revalidation`, and `side_effects_invariant`.
Populate supported states including source completion, newer active operation with known or unknown identity, newer operation started then completed, and awaited-action failure or cancellation. If the contract depends on whether something ever happened, identify the monotonic evidence or the serialization proof.
For protocol, security, or persistence instead use:
`ID | input/authority | validation | in-memory state | persisted/serialized state | retry/replay | output | exception/log/telemetry exposure | cleanup/revocation`
Encode those columns in each `kind: "authority-data-flow"` inventory object as `input_authority`, `validation`, `in_memory_state`, `persisted_state`, `retry_replay`, `output`, `exception_exposure`, and `cleanup_revocation`. Every kind-specific field must be a nonempty string so preflight rejects a summary-only row before dispatch.
## Reviewer instructions
Perform exactly one read-only review round on the frozen fingerprint. Your context must be created with no inherited implementer conversation; the dispatcher uses `fork_turns: "none"` when available. First run the supplied revalidation command and calculate the merge base. Then inspect the complete raw diff, surrounding source, tests, and supplied references. Validate every assigned inventory row rather than trusting the implementer. Return the packet SHA-256 reported by preflight so the validator rejects credit after any packet field or evidence descriptor changes. You may report blockers outside your specialty.
Do not edit or stage files, recursively invoke the review workflow, spawn another reviewer, run broad repository verification, inspect memory, rediscover workflow skills, rerun implementation strategy, search for the fingerprint helper, or rediscover the release tag. Inherit the supplied implementation scope contract; if it is inconsistent or leaves a decision-relevant ambiguity, report that uncertainty to the implementer instead of launching a strategy pass. If any mandatory packet field is neither populated nor explicitly marked `none` or `not applicable`, report the missing field and do not return a creditable clean verdict. Reopen primary source or released evidence only when supplied evidence is inconsistent or leaves a decision-relevant uncertainty; do not use reopening to replace missing packet contents. Run only focused non-mutating probes needed to resolve such uncertainty.
Use approximately 12 source-inspection tool calls as a soft budget. Exceed it whenever decision-relevant uncertainty requires more evidence, but record a concise reason. Do not skip evidence or lower review quality to stay within the budget.
Return exactly one JSON object with this shape and no prose outside it:
```json
{
"verdict": "clean | findings require fixes | complexity reset required | incomplete packet",
"reviewed_fingerprints": {
"packet": "...",
"combined": "...",
"components": {"component-name": "..."}
},
"checked_inventory_ids": ["..."],
"unchecked_inventory_ids": [{"id": "...", "reason": "..."}],
"high_risk_dimensions_checked": ["..."],
"focused_probes": [{"command": "...", "result": "..."}],
"remaining_uncertainty": ["..."],
"findings": [
{
"priority": "P0 | P1 | P2 | P3",
"title": "...",
"location": "path:line or symbol",
"failure_scenario": "...",
"user_consequence": "...",
"support_basis": "...",
"baseline_patch_evidence": "... | not applicable",
"smallest_safe_correction": "...",
"root_cause_id": "CANONICAL_ID | NEW:<lowercase-slug>",
"root_cause_evidence": {
"new_contract_evidence_ids": ["..."],
"new_inventory_ids": ["..."]
}
}
],
"sibling_scenario_scan": [{"root_cause_id": "...", "inventory_ids": ["..."], "result": "..."}],
"inspection_call_count": 0,
"inspection_budget_reason": "none | ..."
}
```
Use empty arrays for `focused_probes`, `remaining_uncertainty`, `findings`, or `sibling_scenario_scan` when there are none. Every assigned inventory ID must appear in either `checked_inventory_ids` or `unchecked_inventory_ids`. Each sibling-scenario scan must reuse a canonical root ID or a `NEW:` root proposed by a finding in the same output, and every scan inventory ID must resolve to an indexed inventory row. A `clean` verdict requires an empty `unchecked_inventory_ids`, `remaining_uncertainty`, and `findings` array.
The reviewer output and every finding, root-cause evidence, unchecked-inventory, and sibling-scenario object must use exactly the fields shown above. Unknown fields are invalid rather than ignored.
Every `focused_probes[].command` must contain the exact executable command that ran. For a non-shell tool call, provide the complete tool name and arguments. Prose-only labels, omitted arguments, and placeholders such as `<focused probe>` are incomplete and earn no clean credit. If the exact command would be too large to return, place the probe code in an indexed evidence artifact before execution and return its path, SHA-256 digest, and exact execution command.
For each finding, reuse a canonical root-cause ID supplied in the packet or propose `NEW:<lowercase-slug>`. Populate both `root_cause_evidence` arrays, using empty arrays when there is no new evidence. Every submitted contract evidence ID must name an indexed `evidence_artifacts[].id`, and every submitted inventory ID must name an indexed `inventory[].id`. For a canonical root, submitted IDs must be additions owned by that root in the current ledger relative to the prior immutable snapshot; an inventory ID owned by another root cannot be reassigned as finding evidence. A new proposal requires indexed evidence content that is not owned by any canonical root or a distinct root proposed in the same output, and its inventory array must remain empty until implementer promotion. A closed root may be reopened only with content-new evidence or semantic inventory; renaming, copying, or aliasing prior content does not make it new. If a reviewer discovers evidence that is absent from the frozen packet, add and digest that evidence in the packet, rerun packet preflight on the same fingerprint round, and then resubmit the output. The implementer validates each saved response before accepting findings or clean credit:
`python scripts/review_protocol.py reviewer-output --packet <packet.json> --reviewer <reviewer-id> --output <output.json> --task-id <task-id> --ledger <ledger.json> --prior-ledger <prior-ledger.json> --prior-ledger-sha256 <sha256>`
A bare `clean` or generic checklist is incomplete and earns no clean credit. A malformed JSON object or missing required field is equally incomplete.
## Specialty assignment
- Primary dimensions:
- Required inventory rows:
- Expected component boundaries:
- Evidence items expected to be sufficient:
- Complementary reviewer assignment, if any:
- Reviewer ID from the machine-readable packet:
- Canonical root-cause IDs and closure states:
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,712 @@
#!/usr/bin/env python3
"""Print deterministic content and repository fingerprints for a review state."""
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import stat
import subprocess
import tempfile
from dataclasses import dataclass
from pathlib import Path, PurePosixPath
def _git(repo: Path, *args: str) -> bytes:
return subprocess.check_output(("git", "-C", os.fspath(repo), *args), stderr=subprocess.PIPE)
def _git_diff(repo: Path, *args: str) -> bytes:
completed = subprocess.run(
("git", "-C", os.fspath(repo), *args),
capture_output=True,
)
if completed.returncode not in {0, 1}:
raise subprocess.CalledProcessError(
completed.returncode,
completed.args,
output=completed.stdout,
stderr=completed.stderr,
)
return completed.stdout
def _digest(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
@dataclass(frozen=True, slots=True)
class _Snapshot:
tracked_diff: bytes
complete_diff: bytes
status: bytes
workspace: list[dict[str, object]]
unfiltered_status: bytes
unfiltered_workspace: list[dict[str, object]]
component_workspaces: dict[str, list[dict[str, object]]]
class _NonRegularFileError(ValueError):
pass
def _nonblocking_opener(path: str, flags: int) -> int:
return os.open(path, flags | getattr(os, "O_NONBLOCK", 0))
def _read_regular_file(path: Path) -> tuple[bytes, os.stat_result]:
with open(path, "rb", opener=_nonblocking_opener) as file:
file_stat = os.fstat(file.fileno())
if not stat.S_ISREG(file_stat.st_mode):
raise _NonRegularFileError(path)
return file.read(), file_stat
def _unsafe_index_paths(repo: Path) -> tuple[tuple[str, str], ...]:
raw_entries = _git(repo, "ls-files", "-v", "-z")
unsafe_paths: list[tuple[str, str]] = []
for entry in raw_entries.split(b"\0"):
if len(entry) < 3 or entry[1:2] != b" ":
continue
tag = entry[:1]
relative_path = os.fsdecode(entry[2:])
if tag.islower():
unsafe_paths.append(("assume-unchanged", relative_path))
elif tag == b"S":
candidate = repo / relative_path
if candidate.exists() or candidate.is_symlink():
unsafe_paths.append(("materialized skip-worktree", relative_path))
for entry in _git(repo, "ls-files", "--unmerged", "-z").split(b"\0"):
_, separator, raw_path = entry.partition(b"\t")
if separator:
unsafe_paths.append(("unmerged", os.fsdecode(raw_path)))
return tuple(sorted(set(unsafe_paths)))
def _require_reviewable_index(repo: Path, context: str = "repository") -> None:
unsafe_paths = _unsafe_index_paths(repo)
if unsafe_paths:
details = ", ".join(f"{kind}={path}" for kind, path in unsafe_paths)
raise ValueError(f"The {context} contains unsupported index state: {details}")
def _index_gitlinks(repo: Path) -> dict[str, str]:
raw_entries = _git(repo, "ls-files", "--stage", "-z")
gitlinks: dict[str, str] = {}
for raw_entry in raw_entries.split(b"\0"):
metadata, separator, raw_path = raw_entry.partition(b"\t")
fields = metadata.split()
if separator and len(fields) == 3 and fields[0] == b"160000" and fields[2] == b"0":
gitlinks[os.fsdecode(raw_path)] = fields[1].decode()
return gitlinks
def _is_repository_root(path: Path) -> bool:
try:
top_level = _git(path, "rev-parse", "--show-toplevel")
except (subprocess.CalledProcessError, FileNotFoundError):
return False
return Path(os.fsdecode(top_level.rstrip(b"\n"))).resolve() == path.resolve()
def _require_clean_submodule(
repo: Path,
display_path: str,
expected_head: str,
ancestors: frozenset[Path],
) -> None:
resolved_repo = repo.resolve()
if resolved_repo in ancestors:
raise ValueError(f"Cyclic submodule worktree is unsupported: {display_path}")
ancestors |= {resolved_repo}
_require_reviewable_index(repo, f"submodule {display_path}")
actual_head = _git(repo, "rev-parse", "HEAD^{commit}").decode().strip()
if actual_head != expected_head:
raise ValueError(f"Submodule HEAD does not match the parent index: {display_path}")
for nested_relative_path, nested_head in _index_gitlinks(repo).items():
nested_path = repo / nested_relative_path
_require_clean_gitlink(
nested_path,
f"{display_path}/{nested_relative_path}",
nested_head,
ancestors,
)
if _git(
repo,
"status",
"--porcelain=v1",
"-z",
"--untracked-files=all",
"--ignore-submodules=none",
):
raise ValueError(f"Dirty submodule worktrees are unsupported: {display_path}")
def _require_clean_gitlink(
path: Path,
display_path: str,
expected_head: str,
ancestors: frozenset[Path],
) -> None:
if _is_repository_root(path):
_require_clean_submodule(path, display_path, expected_head, ancestors)
elif path.is_dir() and any(path.iterdir()):
raise ValueError(f"Materialized gitlink is not an initialized submodule: {display_path}")
def _require_clean_submodules(repo: Path) -> None:
ancestors = frozenset({repo.resolve()})
for relative_path, expected_head in _index_gitlinks(repo).items():
_require_clean_gitlink(
repo / relative_path,
relative_path,
expected_head,
ancestors,
)
def _write_bytes_atomically(path: Path, data: bytes) -> None:
descriptor, temporary_name = tempfile.mkstemp(
prefix=".review-state-diff-",
dir=path.parent,
)
try:
with os.fdopen(descriptor, "wb") as temporary_file:
temporary_file.write(data)
os.replace(temporary_name, path)
finally:
try:
os.unlink(temporary_name)
except FileNotFoundError:
pass
def _directory_is_within(path: Path, root: Path) -> bool:
current = path
while True:
try:
if current.samefile(root):
return True
except OSError:
pass
parent = current.parent
if parent == current:
return False
current = parent
def _canonical_pathspecs(pathspecs: tuple[str, ...]) -> tuple[str, ...]:
canonical: list[str] = []
seen: set[str] = set()
for pathspec in pathspecs:
if not pathspec:
raise ValueError("Pathspecs must not be empty.")
if "\0" in pathspec:
raise ValueError("Pathspecs must not contain NUL bytes.")
if pathspec not in seen:
canonical.append(pathspec)
seen.add(pathspec)
return tuple(canonical)
def _base_has_literal_path(repo: Path, base: str, pathspec: str) -> bool:
raw_path = os.fsencode(pathspec)
entries = _git(
repo,
"ls-tree",
"-z",
base,
"--",
f":(literal){pathspec}",
)
for entry in entries.split(b"\0"):
metadata, separator, entry_path = entry.partition(b"\t")
fields = metadata.split()
if separator and entry_path == raw_path and len(fields) >= 2 and fields[1] != b"tree":
return True
return False
def _load_pathspec_file(path: Path) -> tuple[str, ...]:
try:
data, _ = _read_regular_file(path)
values = [line for line in data.decode().splitlines() if line]
except (OSError, UnicodeError, ValueError) as error:
raise ValueError(f"Cannot read pathspec file {path}: {error}") from error
return _canonical_pathspecs(tuple(values))
def _read_workspace_file(path: Path, relative_path: str) -> tuple[bytes, os.stat_result]:
try:
return _read_regular_file(path)
except _NonRegularFileError as error:
raise ValueError(f"Unsupported workspace file type: {relative_path}") from error
except OSError as error:
raise ValueError(f"Cannot read workspace file: {relative_path}") from error
def _workspace_entry(repo: Path, relative_path: str) -> dict[str, object]:
path = repo / relative_path
if path.is_symlink():
content = b"symlink\0" + os.fsencode(os.readlink(path))
return {
"path": relative_path,
"kind": "symlink",
"sha256": _digest(content),
}
if path.is_file():
file_content, file_stat = _read_workspace_file(path, relative_path)
content = b"file\0" + file_content
return {
"path": relative_path,
"kind": "file",
"executable": bool(file_stat.st_mode & 0o100),
"sha256": _digest(content),
}
indexed_head = _index_gitlinks(repo).get(relative_path)
if path.is_dir():
if indexed_head is not None:
return {
"path": relative_path,
"kind": "gitlink",
"head": indexed_head,
}
if _is_repository_root(path):
raise ValueError(f"Untracked nested Git repositories are unsupported: {relative_path}")
return {"path": relative_path, "kind": "directory"}
if indexed_head is not None:
return {
"path": relative_path,
"kind": "gitlink",
"head": indexed_head,
}
if path.exists():
raise ValueError(f"Unsupported workspace file type: {relative_path}")
return {"path": relative_path, "kind": "missing"}
def _workspace_entries(
repo: Path, base: str, pathspecs: tuple[str, ...]
) -> list[dict[str, object]]:
git_pathspecs = _git_pathspecs(repo, base, pathspecs)
tracked_paths = _git(
repo,
"diff",
"--name-only",
"--no-renames",
"--ignore-submodules=none",
"-z",
base,
"--",
*git_pathspecs,
)
untracked_paths = _untracked_paths(repo, base, pathspecs)
paths = {
os.fsdecode(raw_path)
for raw_path in (*tracked_paths.split(b"\0"), *untracked_paths)
if raw_path
}
return [_workspace_entry(repo, relative_path) for relative_path in sorted(paths)]
def _untracked_paths(repo: Path, base: str, pathspecs: tuple[str, ...]) -> tuple[bytes, ...]:
literal_pathspecs = _literal_pathspecs(repo, base, pathspecs)
raw_paths = _git(
repo,
"ls-files",
"--others",
"--exclude-standard",
"-z",
"--",
*_git_pathspecs(repo, base, pathspecs, literal_pathspecs),
)
paths = {raw_path for raw_path in raw_paths.split(b"\0") if raw_path}
for pathspec in literal_pathspecs:
raw_path = os.fsencode(pathspec)
tracked_paths = _git(repo, "ls-files", "-z", "--", f":(literal){pathspec}")
if raw_path not in tracked_paths.split(b"\0"):
paths.add(raw_path)
return tuple(sorted(paths))
def _literal_pathspecs(repo: Path, base: str, pathspecs: tuple[str, ...]) -> frozenset[str]:
literal_pathspecs: set[str] = set()
for pathspec in pathspecs:
relative_path = PurePosixPath(pathspec)
if (
relative_path.is_absolute()
or pathspec != relative_path.as_posix()
or any(part in {".", ".."} for part in relative_path.parts)
):
continue
candidate = repo.joinpath(*relative_path.parts)
raw_path = os.fsencode(pathspec)
tracked_paths = _git(repo, "ls-files", "-z", "--", f":(literal){pathspec}")
if (
(candidate.exists() and not candidate.is_dir())
or candidate.is_symlink()
or raw_path in tracked_paths.split(b"\0")
or _base_has_literal_path(repo, base, pathspec)
):
literal_pathspecs.add(pathspec)
return frozenset(literal_pathspecs)
def _git_pathspecs(
repo: Path,
base: str,
pathspecs: tuple[str, ...],
literal_pathspecs: frozenset[str] | None = None,
) -> tuple[str, ...]:
if literal_pathspecs is None:
literal_pathspecs = _literal_pathspecs(repo, base, pathspecs)
return tuple(
f":(literal){pathspec}" if pathspec in literal_pathspecs else pathspec
for pathspec in pathspecs
)
def _complete_diff(repo: Path, base: str, pathspecs: tuple[str, ...]) -> bytes:
chunks = [
_git(
repo,
"diff",
"--binary",
"--full-index",
"--ignore-submodules=none",
base,
"--",
*_git_pathspecs(repo, base, pathspecs),
)
]
for raw_path in _untracked_paths(repo, base, pathspecs):
chunks.append(
_git_diff(
repo,
"diff",
"--no-index",
"--binary",
"--full-index",
"--",
"/dev/null",
os.fsdecode(raw_path),
)
)
return b"".join(chunks)
def _content_fingerprint(base: str, workspace: list[dict[str, object]]) -> str:
canonical = json.dumps(
{"base": base, "workspace": workspace},
ensure_ascii=True,
sort_keys=True,
separators=(",", ":"),
)
return _digest(canonical.encode())
def _repository_fingerprint(
*,
content_fingerprint: str,
head: str,
status_sha256: str,
tracked_diff_sha256: str,
complete_diff_sha256: str,
unfiltered_status_sha256: str,
unfiltered_content_fingerprint: str,
) -> str:
canonical = json.dumps(
{
"content_fingerprint": content_fingerprint,
"head": head,
"status_sha256": status_sha256,
"tracked_diff_sha256": tracked_diff_sha256,
"complete_diff_sha256": complete_diff_sha256,
"unfiltered_status_sha256": unfiltered_status_sha256,
"unfiltered_content_fingerprint": unfiltered_content_fingerprint,
},
ensure_ascii=False,
sort_keys=True,
separators=(",", ":"),
)
return _digest(canonical.encode())
def _capture_snapshot(
repo: Path,
base: str,
pathspecs: tuple[str, ...],
components: dict[str, tuple[str, ...]],
) -> _Snapshot:
workspace = _workspace_entries(repo, base, pathspecs)
unfiltered_workspace = _workspace_entries(repo, base, ())
unfiltered_by_path = {str(entry["path"]): entry for entry in unfiltered_workspace}
for entry in workspace:
unfiltered_by_path.setdefault(str(entry["path"]), entry)
unfiltered_workspace = [unfiltered_by_path[path] for path in sorted(unfiltered_by_path)]
component_workspaces = {
name: _workspace_entries(repo, base, component_pathspecs)
for name, component_pathspecs in components.items()
}
git_pathspecs = _git_pathspecs(repo, base, pathspecs)
return _Snapshot(
tracked_diff=_git(
repo,
"diff",
"--binary",
"--full-index",
"--ignore-submodules=none",
base,
"--",
*git_pathspecs,
),
complete_diff=_complete_diff(repo, base, pathspecs),
status=_git(
repo,
"status",
"--porcelain=v1",
"-z",
"--untracked-files=all",
"--ignore-submodules=none",
"--",
*git_pathspecs,
),
workspace=workspace,
unfiltered_status=_git(
repo,
"status",
"--porcelain=v1",
"-z",
"--untracked-files=all",
"--ignore-submodules=none",
),
unfiltered_workspace=unfiltered_workspace,
component_workspaces=component_workspaces,
)
def review_state(
repo: Path,
base: str,
pathspecs: tuple[str, ...] = (),
components: dict[str, tuple[str, ...]] | None = None,
complete_diff_output: Path | None = None,
) -> dict[str, object]:
repo = repo.resolve()
top_level = Path(
os.fsdecode(_git(repo, "rev-parse", "--show-toplevel").rstrip(b"\n"))
).resolve()
if not top_level.samefile(repo):
raise ValueError(f"Repository path must be the worktree root: {top_level}")
if complete_diff_output is not None:
complete_diff_output = complete_diff_output.expanduser().resolve()
if _directory_is_within(complete_diff_output.parent, repo):
raise ValueError("Complete diff output must be outside the repository.")
_require_reviewable_index(repo)
_require_clean_submodules(repo)
pathspecs = _canonical_pathspecs(pathspecs)
if components and not pathspecs:
pathspecs = _canonical_pathspecs(
tuple(
pathspec
for component_pathspecs in components.values()
for pathspec in component_pathspecs
)
)
resolved_base = _git(repo, "rev-parse", f"{base}^{{commit}}").decode().strip()
head = _git(repo, "rev-parse", "HEAD^{commit}").decode().strip()
try:
_git(repo, "merge-base", "--is-ancestor", resolved_base, head)
except subprocess.CalledProcessError as error:
raise ValueError("Base must be an ancestor of HEAD.") from error
canonical_components: dict[str, tuple[str, ...]] = {}
for name, component_pathspecs in sorted((components or {}).items()):
canonical_component_pathspecs = _canonical_pathspecs(component_pathspecs)
if not canonical_component_pathspecs:
raise ValueError(f"Component manifest is empty: {name}")
canonical_components[name] = canonical_component_pathspecs
snapshot = _capture_snapshot(repo, resolved_base, pathspecs, canonical_components)
_require_reviewable_index(repo)
_require_clean_submodules(repo)
final_snapshot = _capture_snapshot(repo, resolved_base, pathspecs, canonical_components)
final_head = _git(repo, "rev-parse", "HEAD^{commit}").decode().strip()
_require_reviewable_index(repo)
_require_clean_submodules(repo)
if final_head != head or final_snapshot != snapshot:
raise ValueError("Repository changed while review state was captured.")
snapshot = final_snapshot
content_fingerprint = _content_fingerprint(resolved_base, snapshot.workspace)
component_states: dict[str, dict[str, object]] = {}
component_owners: dict[str, list[str]] = {}
for name, canonical_component_pathspecs in canonical_components.items():
component_workspace = snapshot.component_workspaces[name]
for entry in component_workspace:
component_owners.setdefault(str(entry["path"]), []).append(name)
component_states[name] = {
"content_fingerprint": _content_fingerprint(resolved_base, component_workspace),
"pathspecs": list(canonical_component_pathspecs),
"workspace": component_workspace,
}
if component_states:
combined_paths = {str(entry["path"]) for entry in snapshot.workspace}
component_paths = set(component_owners)
missing_paths = sorted(combined_paths - component_paths)
extra_paths = sorted(component_paths - combined_paths)
overlapping_paths = {
path: owners for path, owners in component_owners.items() if len(owners) > 1
}
if missing_paths or extra_paths or overlapping_paths:
raise ValueError(
"Component manifests must partition the combined review content exactly: "
f"missing={missing_paths}, extra={extra_paths}, "
f"overlapping={overlapping_paths}"
)
repository_state = {
"content_fingerprint": content_fingerprint,
"head": head,
"status_sha256": _digest(snapshot.status),
"tracked_diff_sha256": _digest(snapshot.tracked_diff),
"complete_diff_sha256": _digest(snapshot.complete_diff),
}
repository_fingerprint = _repository_fingerprint(
**repository_state,
unfiltered_status_sha256=_digest(snapshot.unfiltered_status),
unfiltered_content_fingerprint=_content_fingerprint(
resolved_base, snapshot.unfiltered_workspace
),
)
if complete_diff_output is not None:
_write_bytes_atomically(complete_diff_output, snapshot.complete_diff)
return {
"fingerprint": content_fingerprint,
"content_fingerprint": content_fingerprint,
"repository_fingerprint": repository_fingerprint,
"base": resolved_base,
"pathspecs": list(pathspecs),
"workspace": snapshot.workspace,
"complete_diff_paths": [str(entry["path"]) for entry in snapshot.workspace],
"components": component_states,
"unfiltered": {
"status_sha256": _digest(snapshot.unfiltered_status),
"workspace": snapshot.unfiltered_workspace,
},
**repository_state,
}
def _parse_component_files(values: list[str]) -> dict[str, tuple[str, ...]]:
components: dict[str, tuple[str, ...]] = {}
for value in values:
name, separator, raw_path = value.partition("=")
if not separator or not re.fullmatch(r"[a-z0-9][a-z0-9-]*", name) or not raw_path:
raise ValueError(
"Component pathspec files must use lowercase NAME=FILE with a nonempty file."
)
if name in components:
raise ValueError(f"Duplicate component name: {name}")
components[name] = _load_pathspec_file(Path(raw_path))
return components
def _component(value: str) -> tuple[str, str]:
name, separator, pathspec = value.partition("=")
if not separator or not re.fullmatch(r"[a-z0-9][a-z0-9-]*", name) or not pathspec:
raise argparse.ArgumentTypeError("component must use lowercase NAME=PATHSPEC")
if "\0" in pathspec:
raise argparse.ArgumentTypeError("component pathspec must not contain NUL bytes")
return name, pathspec
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--base", required=True, help="Resolved merge-base commit or revision.")
parser.add_argument(
"--pathspec",
action="append",
default=[],
help="Task-owned Git pathspec. Repeat to scope the review; omit to include all changes.",
)
parser.add_argument(
"--pathspec-file",
action="append",
default=[],
type=Path,
help="File containing canonical task-owned pathspecs, one per line.",
)
parser.add_argument(
"--component-pathspec-file",
action="append",
default=[],
metavar="NAME=FILE",
help="Named component manifest. Repeat for runtime, tests-examples, or metadata.",
)
parser.add_argument(
"--component",
action="append",
default=[],
type=_component,
metavar="NAME=PATHSPEC",
help="Named component pathspec. Repeat a name to group paths into one fingerprint.",
)
parser.add_argument(
"--repo",
type=Path,
default=Path.cwd(),
help="Repository worktree root path.",
)
parser.add_argument(
"--complete-diff-output",
type=Path,
help="Write the complete binary diff, including task-owned untracked files, to this path.",
)
parser.add_argument("--pretty", action="store_true", help="Pretty-print the JSON output.")
args = parser.parse_args()
try:
loaded_pathspec_files = [_load_pathspec_file(path) for path in args.pathspec_file]
if any(not pathspecs for pathspecs in loaded_pathspec_files):
raise ValueError("A supplied pathspec file must contain at least one pathspec.")
file_pathspecs = tuple(
pathspec for pathspecs in loaded_pathspec_files for pathspec in pathspecs
)
pathspecs = _canonical_pathspecs((*args.pathspec, *file_pathspecs))
component_files = _parse_component_files(args.component_pathspec_file)
component_values: dict[str, list[str]] = {
name: list(component_pathspecs) for name, component_pathspecs in component_files.items()
}
for name, pathspec in args.component:
component_values.setdefault(name, []).append(pathspec)
components = {
name: _canonical_pathspecs(tuple(component_pathspecs))
for name, component_pathspecs in component_values.items()
}
state = review_state(
args.repo,
args.base,
pathspecs,
components,
complete_diff_output=args.complete_diff_output,
)
except ValueError as error:
parser.error(str(error))
except subprocess.CalledProcessError as error:
parser.error(f"Git command failed with exit status {error.returncode}.")
except (OSError, UnicodeError) as error:
parser.error(f"Cannot inspect repository state: {error}")
print(
json.dumps(
state,
ensure_ascii=True,
indent=2 if args.pretty else None,
sort_keys=True,
)
)
if __name__ == "__main__":
main()
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,832 @@
#!/usr/bin/env python3
from __future__ import annotations
import hashlib
import json
import os
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, str(Path(__file__).parent))
import review_state as review_state_module
from review_state import (
_component,
_content_fingerprint,
_load_pathspec_file,
_workspace_entry,
review_state,
)
class ReviewStateTest(unittest.TestCase):
def setUp(self) -> None:
self.temporary_directory = tempfile.TemporaryDirectory()
self.root = Path(self.temporary_directory.name)
self.repo = self.root / "repo"
self.repo.mkdir()
self._git("init", "-q")
self._git("config", "user.email", "review-state@example.test")
self._git("config", "user.name", "Review State Test")
(self.repo / ".gitignore").write_text("plans/private.md\n")
(self.repo / "src").mkdir()
(self.repo / "tests").mkdir()
(self.repo / "plans").mkdir()
(self.repo / "src" / "runtime.py").write_text("VALUE = 1\n")
(self.repo / "tests" / "test_runtime.py").write_text("assert True\n")
self._git("add", ".")
self._git("commit", "-qm", "initial")
self.base = self._git("rev-parse", "HEAD").strip()
def tearDown(self) -> None:
self.temporary_directory.cleanup()
def _git(self, *args: str) -> str:
return subprocess.check_output(("git", "-C", str(self.repo), *args), text=True)
def _run_cli(self, *args: str) -> subprocess.CompletedProcess[str]:
return subprocess.run(
(
sys.executable,
str(Path(__file__).with_name("review_state.py")),
"--repo",
str(self.repo),
"--base",
self.base,
*args,
),
capture_output=True,
text=True,
)
def test_equivalent_pathspecs_have_the_same_content_fingerprint(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
explicit = review_state(self.repo, self.base, ("src/runtime.py",))
directory = review_state(self.repo, self.base, ("src",))
with_ignored_artifact = review_state(
self.repo, self.base, ("src/runtime.py", "plans/private.md")
)
self.assertEqual(explicit["content_fingerprint"], directory["content_fingerprint"])
self.assertEqual(
explicit["content_fingerprint"], with_ignored_artifact["content_fingerprint"]
)
def test_repository_path_must_be_the_worktree_root(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
with self.assertRaisesRegex(ValueError, "worktree root"):
review_state(self.repo / "src", self.base, ("src/runtime.py",))
def test_repository_changes_during_snapshot_fail_closed(self) -> None:
"""Reject a diff and workspace fingerprint captured from different states."""
runtime = self.repo / "src" / "runtime.py"
runtime.write_text("VALUE = 2\n")
original_complete_diff = review_state_module._complete_diff
mutated = False
def complete_diff_then_mutate(repo: Path, base: str, pathspecs: tuple[str, ...]) -> bytes:
nonlocal mutated
result = original_complete_diff(repo, base, pathspecs)
if not mutated:
runtime.write_text("VALUE = 3\n")
mutated = True
return result
with (
mock.patch.object(
review_state_module,
"_complete_diff",
side_effect=complete_diff_then_mutate,
),
self.assertRaisesRegex(ValueError, "changed while review state was captured"),
):
review_state(self.repo, self.base, ("src/runtime.py",))
@unittest.skipUnless(hasattr(os, "mkfifo"), "Requires POSIX FIFO support.")
def test_exact_special_file_path_fails_closed(self) -> None:
"""Reject a task manifest entry that cannot produce a finite diff."""
fifo = self.repo / "artifact.pipe"
os.mkfifo(fifo)
with self.assertRaisesRegex(ValueError, "Unsupported workspace file type"):
review_state(self.repo, self.base, ("artifact.pipe",))
@unittest.skipUnless(hasattr(os, "mkfifo"), "Requires POSIX FIFO support.")
def test_workspace_file_type_is_verified_after_open(self) -> None:
"""Do not trust a stale file-type check when reading workspace content."""
fifo = self.repo / "artifact.pipe"
os.mkfifo(fifo)
with (
mock.patch.object(Path, "is_file", return_value=True),
mock.patch.object(Path, "read_bytes", return_value=b"not from the FIFO"),
self.assertRaisesRegex(ValueError, "Unsupported workspace file type"),
):
_workspace_entry(self.repo, "artifact.pipe")
@unittest.skipUnless(hasattr(os, "mkfifo"), "Requires POSIX FIFO support.")
def test_pathspec_file_type_is_verified_after_open(self) -> None:
"""Do not trust a path-based read when loading a task manifest."""
fifo = self.root / "task.paths"
os.mkfifo(fifo)
with (
mock.patch.object(Path, "read_text", return_value="src/runtime.py\n"),
self.assertRaisesRegex(ValueError, "Cannot read pathspec file"),
):
_load_pathspec_file(fifo)
@unittest.skipIf(os.name == "nt", "Executable mode normalization requires POSIX.")
def test_executable_uses_git_owner_bit(self) -> None:
runtime = self.repo / "src" / "runtime.py"
runtime.write_text("VALUE = 2\n")
runtime.chmod(0o744)
executable = review_state(self.repo, self.base, ("src/runtime.py",))
runtime.chmod(0o654)
non_executable = review_state(self.repo, self.base, ("src/runtime.py",))
self.assertTrue(executable["workspace"][0]["executable"])
self.assertFalse(non_executable["workspace"][0]["executable"])
self.assertNotEqual(
executable["content_fingerprint"],
non_executable["content_fingerprint"],
)
def test_component_fingerprints_invalidate_only_changed_content(self) -> None:
runtime = self.repo / "src" / "runtime.py"
tests = self.repo / "tests" / "test_runtime.py"
runtime.write_text("VALUE = 2\n")
tests.write_text("assert 2 == 2\n")
components = {"runtime": ("src",), "tests-examples": ("tests",)}
before = review_state(self.repo, self.base, ("src", "tests"), components)
tests.write_text("assert 2 != 1\n")
after = review_state(self.repo, self.base, ("src", "tests"), components)
self.assertEqual(
before["components"]["runtime"]["content_fingerprint"],
after["components"]["runtime"]["content_fingerprint"],
)
self.assertNotEqual(
before["components"]["tests-examples"]["content_fingerprint"],
after["components"]["tests-examples"]["content_fingerprint"],
)
self.assertNotEqual(before["content_fingerprint"], after["content_fingerprint"])
def test_unfiltered_workspace_accounts_for_changes_outside_manifest(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
(self.repo / "tests" / "test_runtime.py").write_text("assert 2 == 2\n")
state = review_state(self.repo, self.base, ("src",))
self.assertEqual([entry["path"] for entry in state["workspace"]], ["src/runtime.py"])
self.assertEqual(
[entry["path"] for entry in state["unfiltered"]["workspace"]],
["src/runtime.py", "tests/test_runtime.py"],
)
self.assertRegex(state["unfiltered"]["status_sha256"], r"^[0-9a-f]{64}$")
def test_complete_diff_includes_task_owned_untracked_files(self) -> None:
new_test = self.repo / "tests" / "test_new.py"
new_test.write_text("assert 2 == 2\n")
complete_diff = self.root / "complete.diff"
state = review_state(
self.repo,
self.base,
("tests",),
complete_diff_output=complete_diff,
)
diff = complete_diff.read_bytes()
self.assertIn(b"diff --git a/tests/test_new.py b/tests/test_new.py", diff)
self.assertIn(b"+assert 2 == 2", diff)
self.assertEqual(state["complete_diff_sha256"], hashlib.sha256(diff).hexdigest())
self.assertEqual(
state["complete_diff_paths"],
["tests/test_new.py"],
)
self.assertNotEqual(state["complete_diff_sha256"], state["tracked_diff_sha256"])
def test_exact_manifest_path_includes_ignored_untracked_file(self) -> None:
ignored = self.repo / "plans" / "private.md"
ignored.write_text("shipped fixture\n")
complete_diff = self.root / "complete.diff"
state = review_state(
self.repo,
self.base,
("plans/private.md",),
{"release-metadata": ("plans/private.md",)},
complete_diff_output=complete_diff,
)
self.assertEqual(state["complete_diff_paths"], ["plans/private.md"])
self.assertEqual(state["unfiltered"]["workspace"], state["workspace"])
self.assertEqual(
state["components"]["release-metadata"]["workspace"],
state["workspace"],
)
self.assertIn(b"+shipped fixture", complete_diff.read_bytes())
def test_directory_pathspec_does_not_promote_ignored_operational_files(self) -> None:
(self.repo / "plans" / "private.md").write_text("operational plan\n")
state = review_state(self.repo, self.base, ("plans",))
self.assertEqual(state["workspace"], [])
self.assertEqual(state["complete_diff_paths"], [])
def test_literal_filename_with_pathspec_metacharacters_is_exact(self) -> None:
(self.repo / "plans" / "[a].md").write_text("literal\n")
(self.repo / "plans" / "a.md").write_text("glob match\n")
state = review_state(self.repo, self.base, ("plans/[a].md",))
self.assertEqual(state["complete_diff_paths"], ["plans/[a].md"])
def test_existing_magic_prefixed_filename_is_exact(self) -> None:
"""Treat an existing magic-prefixed filename as an exact path."""
(self.repo / ":(glob)literal").write_text("literal filename\n")
(self.repo / "literal").write_text("glob match\n")
state = review_state(self.repo, self.base, (":(glob)literal",))
self.assertEqual(state["complete_diff_paths"], [":(glob)literal"])
def test_deleted_magic_prefixed_filename_is_exact_from_base(self) -> None:
"""Treat a deleted base filename with magic syntax as exact."""
magic_prefixed = self.repo / ":(glob)literal"
magic_prefixed.write_text("deleted literal filename\n")
(self.repo / "literal").write_text("glob match\n")
self._git("add", ".")
self._git("commit", "-qm", "add magic-prefixed filename")
self.base = self._git("rev-parse", "HEAD").strip()
self._git("rm", "-q", "--", ":(literal):(glob)literal")
state = review_state(self.repo, self.base, (":(glob)literal",))
self.assertEqual(state["complete_diff_paths"], [":(glob)literal"])
def test_explicit_glob_magic_preserves_pattern_semantics(self) -> None:
(self.repo / "plans" / "[a].md").write_text("literal\n")
(self.repo / "plans" / "a.md").write_text("glob match\n")
state = review_state(self.repo, self.base, (":(glob)plans/[a].md",))
self.assertEqual(state["complete_diff_paths"], ["plans/[a].md", "plans/a.md"])
def test_submodule_changes_require_reviewable_gitlinks(self) -> None:
"""Accept staged pointers and reject other submodule worktree changes."""
source = self.repo / ".fixtures" / "dependency-source"
source.mkdir(parents=True)
subprocess.run(("git", "init", "-q", str(source)), check=True)
subprocess.run(
("git", "-C", str(source), "config", "user.email", "submodule@example.test"),
check=True,
)
subprocess.run(
("git", "-C", str(source), "config", "user.name", "Submodule Test"),
check=True,
)
(source / "tracked.txt").write_text("committed\n")
subprocess.run(("git", "-C", str(source), "add", "."), check=True)
subprocess.run(("git", "-C", str(source), "commit", "-qm", "initial"), check=True)
with (self.repo / ".gitignore").open("a") as gitignore:
gitignore.write(".fixtures/\n")
self._git(
"-c",
"protocol.file.allow=always",
"submodule",
"add",
"-q",
str(source),
"vendor/dependency",
)
self._git(
"config",
"-f",
".gitmodules",
"submodule.vendor/dependency.ignore",
"all",
)
self._git("add", ".")
self._git("commit", "-qm", "add dependency")
self.base = self._git("rev-parse", "HEAD").strip()
(source / "tracked.txt").write_text("updated commit\n")
subprocess.run(("git", "-C", str(source), "commit", "-qam", "update"), check=True)
updated_head = subprocess.check_output(
("git", "-C", str(source), "rev-parse", "HEAD"),
text=True,
).strip()
self._git("-C", "vendor/dependency", "fetch", "-q", "origin")
self._git("-C", "vendor/dependency", "checkout", "-q", updated_head)
with self.assertRaisesRegex(ValueError, "HEAD does not match.*vendor/dependency"):
review_state(self.repo, self.base, ("vendor/dependency",))
self._git("add", "vendor/dependency")
clean_state = review_state(self.repo, self.base, ("vendor/dependency",))
self.assertEqual(
clean_state["workspace"],
[
{
"path": "vendor/dependency",
"kind": "gitlink",
"head": updated_head,
}
],
)
tracked = self.repo / "vendor" / "dependency" / "tracked.txt"
tracked.write_text("dirty body\n")
with self.assertRaisesRegex(ValueError, "Dirty submodule.*vendor/dependency"):
review_state(self.repo, self.base, ("vendor/dependency",))
def test_materialized_uninitialized_gitlink_fails_closed(self) -> None:
"""Reject arbitrary directory content hidden behind an index gitlink."""
self._git(
"update-index",
"--add",
"--cacheinfo",
f"160000,{self.base},vendor/dependency",
)
dependency = self.repo / "vendor" / "dependency"
dependency.mkdir(parents=True)
(dependency / "unreviewed.txt").write_text("first body\n")
with self.assertRaisesRegex(ValueError, "Materialized gitlink.*vendor/dependency"):
review_state(self.repo, self.base, ("vendor/dependency",))
@unittest.skipIf(os.name == "nt", "Directory symlinks require platform privileges.")
def test_cyclic_gitlink_worktree_fails_closed(self) -> None:
"""Reject a gitlink alias that resolves back to an ancestor repository."""
self._git(
"update-index",
"--add",
"--cacheinfo",
f"160000,{self.base},vendor/self",
)
vendor = self.repo / "vendor"
vendor.mkdir()
os.symlink("..", vendor / "self", target_is_directory=True)
original_limit = sys.getrecursionlimit()
sys.setrecursionlimit(120)
self.addCleanup(sys.setrecursionlimit, original_limit)
with self.assertRaisesRegex(ValueError, "Cyclic submodule worktree"):
review_state(self.repo, self.base, ("vendor/self",))
def test_hidden_nested_submodule_changes_fail_closed(self) -> None:
"""Reject nested pointer and content changes hidden by configuration."""
leaf_source = self.repo / ".fixtures" / "leaf-source"
leaf_source.mkdir(parents=True)
subprocess.run(("git", "init", "-q", str(leaf_source)), check=True)
subprocess.run(
("git", "-C", str(leaf_source), "config", "user.email", "leaf@example.test"),
check=True,
)
subprocess.run(
("git", "-C", str(leaf_source), "config", "user.name", "Leaf Test"),
check=True,
)
(leaf_source / "tracked.txt").write_text("committed\n")
subprocess.run(("git", "-C", str(leaf_source), "add", "."), check=True)
subprocess.run(
("git", "-C", str(leaf_source), "commit", "-qm", "initial"),
check=True,
)
parent_source = self.repo / ".fixtures" / "parent-source"
parent_source.mkdir()
subprocess.run(("git", "init", "-q", str(parent_source)), check=True)
subprocess.run(
("git", "-C", str(parent_source), "config", "user.email", "parent@example.test"),
check=True,
)
subprocess.run(
("git", "-C", str(parent_source), "config", "user.name", "Parent Test"),
check=True,
)
subprocess.run(
(
"git",
"-C",
str(parent_source),
"-c",
"protocol.file.allow=always",
"submodule",
"add",
"-q",
str(leaf_source),
"nested",
),
check=True,
)
subprocess.run(
(
"git",
"-C",
str(parent_source),
"config",
"-f",
".gitmodules",
"submodule.nested.ignore",
"all",
),
check=True,
)
subprocess.run(("git", "-C", str(parent_source), "add", ".gitmodules"), check=True)
subprocess.run(("git", "-C", str(parent_source), "commit", "-qam", "initial"), check=True)
with (self.repo / ".gitignore").open("a") as gitignore:
gitignore.write(".fixtures/\n")
self._git(
"-c",
"protocol.file.allow=always",
"submodule",
"add",
"-q",
str(parent_source),
"vendor/dependency",
)
self._git(
"-C",
"vendor/dependency",
"-c",
"protocol.file.allow=always",
"submodule",
"update",
"--init",
"-q",
)
self._git("add", ".")
self._git("commit", "-qm", "add nested dependency")
self.base = self._git("rev-parse", "HEAD").strip()
expected_nested_head = self._git(
"-C",
"vendor/dependency",
"rev-parse",
"HEAD:nested",
).strip()
(leaf_source / "tracked.txt").write_text("updated commit\n")
subprocess.run(
("git", "-C", str(leaf_source), "commit", "-qam", "update"),
check=True,
)
updated_nested_head = subprocess.check_output(
("git", "-C", str(leaf_source), "rev-parse", "HEAD"),
text=True,
).strip()
self._git("-C", "vendor/dependency/nested", "fetch", "-q", "origin")
self._git(
"-C",
"vendor/dependency/nested",
"checkout",
"-q",
updated_nested_head,
)
parent_status = self._git("-C", "vendor/dependency", "status", "--porcelain=v1")
self.assertEqual(parent_status, "")
with self.assertRaisesRegex(
ValueError,
"HEAD does not match.*vendor/dependency/nested",
):
review_state(self.repo, self.base, ("vendor/dependency",))
self._git(
"-C",
"vendor/dependency/nested",
"checkout",
"-q",
expected_nested_head,
)
tracked = self.repo / "vendor" / "dependency" / "nested" / "tracked.txt"
tracked.write_text("dirty body\n")
parent_status = self._git("-C", "vendor/dependency", "status", "--porcelain=v1")
self.assertEqual(parent_status, "")
with self.assertRaisesRegex(
ValueError,
"Dirty submodule.*vendor/dependency/nested",
):
review_state(self.repo, self.base, ("vendor/dependency",))
@unittest.skipIf(os.name == "nt", "Non-UTF-8 filenames require POSIX filesystem bytes.")
def test_non_utf8_filename_has_stable_fingerprint(self) -> None:
"""Preserve surrogateescaped Git path bytes in review artifacts."""
raw_relative_path = b"tests/non-utf8-\xff.py"
git = (b"git", b"-C", os.fsencode(self.repo))
blob = subprocess.check_output(
(*git, b"hash-object", b"-w", b"--stdin"),
input=b"assert True\n",
).strip()
subprocess.run(
(
*git,
b"update-index",
b"--add",
b"--cacheinfo",
b"100644," + blob + b"," + raw_relative_path,
),
check=True,
)
self._git("commit", "-qm", "add non-UTF-8 filename")
self.base = self._git("rev-parse", "HEAD").strip()
subprocess.run(
(*git, b"update-index", b"--force-remove", b"--", raw_relative_path),
check=True,
)
relative_path = os.fsdecode(raw_relative_path)
state = review_state(self.repo, self.base, ("tests",))
self.assertEqual(state["complete_diff_paths"], [relative_path])
self.assertEqual(
_content_fingerprint(state["base"], state["workspace"]),
state["content_fingerprint"],
)
json.dumps(state, ensure_ascii=True)
completed = self._run_cli("--pathspec", "tests")
self.assertEqual(completed.returncode, 0, completed.stderr)
cli_state = json.loads(completed.stdout)
self.assertEqual(cli_state["complete_diff_paths"], [relative_path])
self.assertEqual(cli_state["content_fingerprint"], state["content_fingerprint"])
def test_complete_diff_output_must_be_outside_repository(self) -> None:
"""Reject an operational diff artifact inside the worktree."""
complete_diff = self.repo / "complete.diff"
with self.assertRaisesRegex(ValueError, "outside the repository"):
review_state(
self.repo,
self.base,
complete_diff_output=complete_diff,
)
self.assertFalse(complete_diff.exists())
def test_complete_diff_output_rejects_case_alias_inside_repository(self) -> None:
"""Reject case aliases that resolve to the worktree on this filesystem."""
alternate_repo = self.repo.with_name(self.repo.name.swapcase())
if not alternate_repo.exists() or not alternate_repo.samefile(self.repo):
self.skipTest("Filesystem is case-sensitive.")
complete_diff = alternate_repo / "complete.diff"
with self.assertRaisesRegex(ValueError, "outside the repository"):
review_state(
self.repo,
self.base,
complete_diff_output=complete_diff,
)
self.assertFalse(complete_diff.exists())
def test_complete_diff_output_does_not_follow_hardlink_into_repository(self) -> None:
"""Replace an outside hardlink without mutating its repository peer."""
runtime = self.repo / "src" / "runtime.py"
complete_diff = self.root / "complete.diff"
os.link(runtime, complete_diff)
(self.repo / "tests" / "test_runtime.py").write_text("assert 2 == 2\n")
state = review_state(
self.repo,
self.base,
complete_diff_output=complete_diff,
)
self.assertEqual(runtime.read_text(), "VALUE = 1\n")
self.assertEqual(
hashlib.sha256(complete_diff.read_bytes()).hexdigest(),
state["complete_diff_sha256"],
)
def test_external_complete_diff_output_keeps_state_stable(self) -> None:
"""Keep consecutive review states stable when writing an artifact."""
complete_diff = self.root / "complete.diff"
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
first = review_state(
self.repo,
self.base,
complete_diff_output=complete_diff,
)
second = review_state(
self.repo,
self.base,
complete_diff_output=complete_diff,
)
self.assertEqual(first, second)
self.assertEqual(
hashlib.sha256(complete_diff.read_bytes()).hexdigest(),
second["complete_diff_sha256"],
)
def test_assume_unchanged_paths_fail_closed(self) -> None:
"""Reject index flags that can hide worktree content changes."""
self._git("update-index", "--assume-unchanged", "src/runtime.py")
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
with self.assertRaisesRegex(ValueError, "assume-unchanged.*src/runtime.py"):
review_state(self.repo, self.base, ("src/runtime.py",))
def test_materialized_skip_worktree_paths_fail_closed(self) -> None:
"""Reject materialized sparse paths that can hide worktree changes."""
self._git("update-index", "--skip-worktree", "src/runtime.py")
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
with self.assertRaisesRegex(ValueError, "skip-worktree.*src/runtime.py"):
review_state(self.repo, self.base, ("src/runtime.py",))
def test_unmerged_index_paths_fail_closed(self) -> None:
"""Reject unresolved index stages before fingerprinting worktree content."""
self._git("checkout", "-qb", "other")
(self.repo / "src" / "runtime.py").write_text("VALUE = 'other'\n")
self._git("commit", "-qam", "other change")
self._git("checkout", "-qb", "current", self.base)
(self.repo / "src" / "runtime.py").write_text("VALUE = 'current'\n")
self._git("commit", "-qam", "current change")
merged = subprocess.run(
("git", "-C", str(self.repo), "merge", "other"),
capture_output=True,
text=True,
)
self.assertEqual(merged.returncode, 1)
with self.assertRaisesRegex(ValueError, "unmerged=src/runtime.py"):
review_state(self.repo, self.base, ("src/runtime.py",))
def test_ordinary_directory_is_not_a_gitlink(self) -> None:
"""Do not discover the parent repository through a directory."""
self.assertEqual(
_workspace_entry(self.repo, "plans"),
{"path": "plans", "kind": "directory"},
)
def test_untracked_nested_repository_fails_closed(self) -> None:
"""Reject embedded repositories that have no reviewable gitlink."""
nested = self.repo / "nested"
subprocess.run(("git", "init", "-q", str(nested)), check=True)
(nested / "untracked.txt").write_text("not represented by a gitlink\n")
with self.assertRaisesRegex(ValueError, "Untracked nested Git repositories.*nested"):
review_state(self.repo, self.base)
def test_cli_writes_complete_diff_output(self) -> None:
(self.repo / "tests" / "test_new.py").write_text("assert True\n")
complete_diff = self.root / "complete.diff"
completed = self._run_cli(
"--pathspec",
"tests",
"--complete-diff-output",
str(complete_diff),
)
self.assertEqual(completed.returncode, 0, completed.stderr)
state = json.loads(completed.stdout)
self.assertEqual(
state["complete_diff_sha256"],
hashlib.sha256(complete_diff.read_bytes()).hexdigest(),
)
def test_repository_fingerprint_includes_outside_manifest_state_and_content(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
before = review_state(self.repo, self.base, ("src",))
outside = self.repo / "outside.txt"
outside.write_text("first\n")
after_add = review_state(self.repo, self.base, ("src",))
outside.write_text("second\n")
after_content = review_state(self.repo, self.base, ("src",))
self.assertEqual(before["content_fingerprint"], after_add["content_fingerprint"])
self.assertEqual(after_add["content_fingerprint"], after_content["content_fingerprint"])
self.assertNotEqual(before["repository_fingerprint"], after_add["repository_fingerprint"])
self.assertNotEqual(
after_add["repository_fingerprint"], after_content["repository_fingerprint"]
)
def test_pathspec_file_preserves_literal_values_and_deduplicates(self) -> None:
manifest = self.repo / "paths.txt"
manifest.write_text("src\n\n#literal\n lead.py\nsrc\n")
self.assertEqual(_load_pathspec_file(manifest), ("src", "#literal", " lead.py"))
def test_direct_pathspec_preserves_leading_space(self) -> None:
(self.repo / " lead.py").write_text("VALUE = 2\n")
completed = self._run_cli("--pathspec", " lead.py")
self.assertEqual(completed.returncode, 0, completed.stderr)
state = json.loads(completed.stdout)
self.assertEqual([entry["path"] for entry in state["workspace"]], [" lead.py"])
def test_empty_direct_pathspec_fails_closed(self) -> None:
completed = self._run_cli("--pathspec", "")
self.assertEqual(completed.returncode, 2)
self.assertIn("Pathspecs must not be empty", completed.stderr)
self.assertNotIn("Traceback", completed.stderr)
def test_invalid_manifest_files_are_parser_errors(self) -> None:
cases = (
("--pathspec-file", str(self.repo / "missing.paths")),
("--pathspec-file", str(self.repo)),
("--component-pathspec-file", "runtime="),
("--component-pathspec-file", f"runtime={self.repo / 'missing.paths'}"),
)
for arguments in cases:
with self.subTest(arguments=arguments):
completed = self._run_cli(*arguments)
self.assertEqual(completed.returncode, 2)
self.assertIn("error:", completed.stderr)
self.assertNotIn("Traceback", completed.stderr)
def test_invalid_repository_is_a_parser_error(self) -> None:
missing_repo = self.repo / "missing-repo"
completed = subprocess.run(
(
sys.executable,
str(Path(__file__).with_name("review_state.py")),
"--repo",
str(missing_repo),
"--base",
self.base,
),
capture_output=True,
text=True,
)
self.assertEqual(completed.returncode, 2)
self.assertIn("Git command failed", completed.stderr)
self.assertNotIn("fatal:", completed.stderr)
self.assertNotIn("Traceback", completed.stderr)
def test_invalid_base_is_a_parser_error(self) -> None:
completed = self._run_cli("--base", "missing-revision")
self.assertEqual(completed.returncode, 2)
self.assertIn("Git command failed", completed.stderr)
self.assertNotIn("fatal:", completed.stderr)
self.assertNotIn("Traceback", completed.stderr)
def test_non_ancestor_base_is_a_parser_error(self) -> None:
self._git("checkout", "-qb", "sibling")
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
self._git("commit", "-qam", "sibling change")
sibling = self._git("rev-parse", "HEAD").strip()
self._git("checkout", "-qb", "current", self.base)
(self.repo / "tests" / "test_runtime.py").write_text("assert 2 == 2\n")
self._git("commit", "-qam", "head change")
completed = self._run_cli("--base", sibling)
self.assertEqual(completed.returncode, 2)
self.assertIn("Base must be an ancestor of HEAD", completed.stderr)
self.assertNotIn("fatal:", completed.stderr)
self.assertNotIn("Traceback", completed.stderr)
def test_component_manifests_must_cover_combined_content(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
(self.repo / "tests" / "test_runtime.py").write_text("assert 2 == 2\n")
with self.assertRaisesRegex(ValueError, "missing=.*test_runtime.py"):
review_state(self.repo, self.base, ("src", "tests"), {"runtime": ("src",)})
def test_component_manifests_must_not_overlap(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
with self.assertRaisesRegex(ValueError, "overlapping=.*runtime.py"):
review_state(
self.repo,
self.base,
("src",),
{"runtime": ("src",), "tests-examples": ("src/runtime.py",)},
)
def test_components_define_combined_scope_when_pathspecs_are_omitted(self) -> None:
(self.repo / "src" / "runtime.py").write_text("VALUE = 2\n")
state = review_state(self.repo, self.base, components={"runtime": ("src",)})
self.assertEqual(state["pathspecs"], ["src"])
self.assertEqual([entry["path"] for entry in state["workspace"]], ["src/runtime.py"])
def test_component_cli_value(self) -> None:
self.assertEqual(_component("runtime=src"), ("runtime", "src"))
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,607 @@
#!/usr/bin/env python3
from __future__ import annotations
import re
import unittest
from pathlib import Path
class SkillContractTest(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.skill_root = Path(__file__).resolve().parent.parent
cls.skill = (cls.skill_root / "SKILL.md").read_text()
cls.agent_config = (cls.skill_root / "agents" / "openai.yaml").read_text()
cls.reviewer_brief = (cls.skill_root / "references" / "reviewer-brief.md").read_text()
cls.review_protocol = (cls.skill_root / "scripts" / "review_protocol.py").read_text()
cls.repo_instructions = (cls.skill_root.parents[2] / "AGENTS.md").read_text()
cls.code_change_verification = (
cls.skill_root.parent / "code-change-verification" / "SKILL.md"
).read_text()
cls.implementation_kickoff = (
cls.skill_root.parent / "implementation-kickoff" / "SKILL.md"
).read_text()
cls.handoff_validator = (
cls.skill_root.parent / "implementation-kickoff" / "scripts" / "validate_handoff.py"
).read_text()
def test_repo_local_metadata_matches_skill(self) -> None:
self.assertEqual(self.skill.splitlines()[1], "name: implementation-final-review")
self.assertIn('display_name: "Implementation Final Review"', self.agent_config)
self.assertIn("$implementation-final-review", self.agent_config)
self.assertIn("allow_implicit_invocation: false", self.agent_config)
def test_workflow_steps_are_consecutive(self) -> None:
workflow = self.skill.split("## Workflow", 1)[1].split(
"Maintain one compact round ledger", 1
)[0]
steps = [int(value) for value in re.findall(r"^(\d+)\. ", workflow, re.MULTILINE)]
self.assertEqual(steps, list(range(1, 22)))
def test_quality_gates_cover_prior_failure_modes(self) -> None:
required_text = (
"contract-surface inventory",
"every consumer, forwarding branch, and adapter",
"Search adjacent contract surfaces even when they are absent from the diff",
"await-boundary matrix",
"a newer operation that starts and completes while suspended",
"current active state is insufficient",
"A bare `clean`",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_reviewer_brief_avoids_repeated_context_discovery(self) -> None:
required_text = (
"Exact fingerprint revalidation command",
"Complete three-dot diff command",
"Do not edit or stage files",
"inspect memory",
"rediscover workflow skills",
"A bare `clean` or generic checklist is incomplete",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.reviewer_brief)
def test_incomplete_reviewer_packets_fail_closed(self) -> None:
required_skill_text = (
"Populate every template field or mark it explicitly `none` or `not applicable`",
"do not dispatch an incomplete packet",
"missing packet evidence cannot be reconstructed by the reviewer",
"cannot return a creditable clean verdict",
"Reopening source cannot replace missing packet contents",
)
required_brief_text = (
"Fill every field or mark it explicitly `none` or `not applicable`",
"do not dispatch an incomplete packet",
"report the missing field and do not return a creditable clean verdict",
"do not use reopening to replace missing packet contents",
)
for text in required_skill_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
for text in required_brief_text:
with self.subTest(text=text):
self.assertIn(text, self.reviewer_brief)
def test_full_verification_waits_for_clean_review(self) -> None:
required_text = (
"Do not start any broad final repository gate while review is incomplete or "
"finding-bearing",
"defer `make lint`, `make typecheck`, `make tests-review`, `make tests`, "
"repository-wide builds, examples runners, and integration suites until step 19 "
"establishes clean review",
"Do not run `make tests-review`, `make tests`, or repository-wide `make typecheck` "
"during an iterative review round",
"Set `verification.eligible_concurrent_gates` to `none`",
"the exact clean-reviewed fingerprint must still pass the complete "
"repository-required verification stack",
"After the clean-review condition is met",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
self.assertIn("Eligible concurrent final-gate commands: `none`", self.reviewer_brief)
self.assertIn("Broad final gates deferred until clean review", self.reviewer_brief)
self.assertIn(
"packet preflight rejects any attempt to overlap a broad final gate with review",
self.reviewer_brief,
)
def test_host_capacity_check_avoids_locks_and_finalize_prompts(self) -> None:
for source in (self.skill, self.code_change_verification, self.repo_instructions):
with self.subTest(source=source[:40]):
self.assertIn("available read-only task or process evidence", source)
self.assertIn("repository lock", source)
self.assertIn("host-wide mutex", source)
self.assertIn("user-triggered `finalize`", source)
self.assertIn("If host telemetry is unavailable", self.skill)
self.assertIn("Lack of host telemetry alone is not a blocker", self.repo_instructions)
def test_commit_hook_parity_runs_before_review(self) -> None:
required_skill_text = (
"Inspect the actual final commit-hook configuration",
"exact safe, non-committing equivalent",
"until a second execution is content-idempotent",
"Normalize generated files before computing embedded hashes or provenance",
"before every fingerprint freeze, including post-fix and delta-review rounds",
"exact executable inspection and rewriting commands plus their results",
)
required_kickoff_text = (
"Before freezing the first review fingerprint",
"Repeat rewriting steps until content-idempotent",
"verify generated-file hashes or provenance after normalization",
)
for text in required_skill_text:
with self.subTest(source="skill", text=text):
self.assertIn(text, self.skill)
for text in required_kickoff_text:
with self.subTest(source="kickoff", text=text):
self.assertIn(text, self.implementation_kickoff)
self.assertIn("idempotent commit-hook parity", self.reviewer_brief)
def test_reviewer_infrastructure_failure_stays_in_the_same_round(self) -> None:
required_text = (
"fails before producing a protocol-valid output",
"has produced neither a finding nor clean credit",
"does not advance `ledger.current_round` or consume another fingerprint round",
"Replace only that reviewer on the same frozen packet and assignment",
"The original two-reviewer concurrent dispatch satisfies the round's concurrency "
"requirement",
"the accepted peer output plus one independently launched replacement output",
"report the gate as unavailable instead of counting an infrastructure failure as "
"review evidence",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_complete_diff_includes_untracked_task_deliverables(self) -> None:
required_text = (
"--complete-diff-output <complete.diff>",
"a standalone `git diff` omits ordinary untracked deliverables",
"`complete_diff_paths` to equal the task workspace exactly",
"`complete_diff_sha256`",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
self.assertIn("task-owned untracked files are included", self.reviewer_brief)
self.assertIn(
"ordinary task-owned untracked files are present", self.implementation_kickoff
)
self.assertIn(
"shipped-path manifest must be a finite regular file", self.implementation_kickoff
)
self.assertIn("authoritative even when ignore rules match that file", self.skill)
self.assertIn("literal precedence over Git pathspec metacharacters", self.skill)
self.assertIn("use explicit `:(glob)` magic", self.skill)
self.assertIn(
"directory or glob pathspec never promotes ignored operational files", self.skill
)
self.assertIn(
"every initialized submodule, including nested submodules, to be clean", self.skill
)
self.assertIn("Stage reviewable gitlink pointer changes", self.skill)
self.assertIn("fail closed on dirty worktrees, hidden index flags", self.skill)
self.assertIn("Reject cyclic or aliased submodule worktree graphs", self.skill)
self.assertIn("two consecutive identical observations of HEAD", self.skill)
self.assertIn("state changes during capture", self.skill)
self.assertIn("including FIFOs, sockets, and devices", self.skill)
self.assertIn("Require unique keys and standard finite numbers", self.skill)
self.assertIn("JavaScript-style `NaN` or infinity constants", self.skill)
self.assertIn("numeric exponents that overflow to infinity", self.skill)
self.assertIn("numeric-size and nesting-limit failures", self.skill)
self.assertIn(
"Every JSON object must use unique keys and standard finite numbers",
self.reviewer_brief,
)
self.assertIn("`NaN`, `Infinity`, and `-Infinity` constants", self.reviewer_brief)
self.assertIn("numeric exponents that overflow to infinity", self.reviewer_brief)
self.assertIn("numeric-size and nesting-limit failures", self.reviewer_brief)
self.assertIn("distinct normalized primary and high-risk specialties", self.skill)
self.assertIn("require every preflight command to be unique", self.skill)
self.assertIn("exact, unique `command` and `result` objects", self.reviewer_brief)
self.assertIn("primary and high-risk specialties must not overlap", self.reviewer_brief)
self.assertIn("Reject unknown fields instead of ignoring", self.skill)
self.assertIn("Unknown receipt fields are invalid", self.reviewer_brief)
self.assertIn("Unknown fields are invalid rather than ignored", self.reviewer_brief)
self.assertIn("two consecutive identical repository observations", self.reviewer_brief)
self.assertIn("path-level `stat` must not authorize a later reopen", self.reviewer_brief)
self.assertIn("unique opened-file device and inode identities", self.reviewer_brief)
self.assertIn("evidence digest absent from every canonical root", self.reviewer_brief)
self.assertIn("`contract_evidence_sha256`", self.reviewer_brief)
self.assertIn("`inventory_sha256`", self.reviewer_brief)
self.assertIn("prior bindings are immutable", self.reviewer_brief)
self.assertIn("every distinct root proposed in the same output", self.reviewer_brief)
self.assertIn(
"Credited receipt content digests and exact commands must be unique",
self.reviewer_brief,
)
self.assertIn("requires their validated digests to remain unchanged", self.reviewer_brief)
def test_verified_base_advance_closure_is_strict_and_keeps_final_verification(self) -> None:
required_text = (
"Verified base-advance closure",
"byte-identical task and component `workspace` arrays",
"their `tracked_diff_sha256` values are identical",
"complete upstream delta from old base to new base",
"dependency-input path",
"applicable build, test, lint, format, hook, lockfile, or package configuration input",
"This closure consumes no fingerprint round and creates no reviewer packet",
"falls through to a fresh review round on the new base",
"rerun every mandatory final gate on the replayed fingerprint",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
self.assertIn("verified base-advance closure", self.implementation_kickoff)
self.assertIn("rerun every mandatory final verification gate", self.implementation_kickoff)
self.assertIn("exact base pathspecs", self.reviewer_brief)
self.assertIn("prose-only or empty dependency claims", self.reviewer_brief)
self.assertIn("keys exactly match the component names", self.reviewer_brief)
self.assertIn(
"nonempty arrays of exact `pathspec` and `reason` records",
self.reviewer_brief,
)
self.assertIn("maps every component name to a nonempty array", self.skill)
def test_operational_artifacts_are_excluded_from_the_handoff_manifest(self) -> None:
required_kickoff_text = (
"operational-only by default",
"Compare the staged changed-path set byte-for-byte with the canonical shipped manifest",
"do not stage an ignored ExecPlan or review artifact",
"--shipped-path-manifest <manifest>",
)
for text in required_kickoff_text:
with self.subTest(text=text):
self.assertIn(text, self.implementation_kickoff)
self.assertIn('"--shipped-path-manifest"', self.handoff_validator)
self.assertIn(
"Committed paths do not match the shipped-path manifest", self.handoff_validator
)
def test_work_status_reporting_distinguishes_running_and_final_states(self) -> None:
required_text = (
"Use `RUNNING` only in commentary",
"Use `COMPLETE` in the final response only when",
"Use `NEEDS_DECISION` in the final response only when",
'instead of asking the user to say "continue"',
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.repo_instructions)
def test_iterative_review_uses_focused_checks_only(self) -> None:
required_text = (
"Prefer focused tests plus a narrowly targeted import, generated-surface, or static "
"check",
"Run a targeted type check only when the change directly affects a typing boundary",
"Do not run repository-wide lint, typecheck, builds, integration suites, "
"`make tests-review`, or `make tests`",
"run only focused checks that target the changed boundary",
"The focused check earns no final-gate credit",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_final_gate_deltas_are_classified_by_component(self) -> None:
required_text = (
"Runtime, public API, behavior-impacting docs",
"Tests or examples only",
"Release metadata only",
"Operational artifact only",
"final combined fingerprint",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_shared_typescript_improvements_keep_python_boundaries(self) -> None:
required_text = (
"package exports and generated public surfaces when applicable",
"protocol capability ownership, pagination termination, cache ownership",
"defer `make lint`, `make typecheck`, `make tests-review`, `make tests`, "
"repository-wide builds",
"the implementer runs the complete stack once after the clean-review gate",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
self.assertNotIn("$changeset-validation", self.skill)
self.assertNotIn("browser/Node/workerd", self.skill)
def test_final_clean_condition_preserves_two_independent_reviews(self) -> None:
self.assertIn(
"normal-risk change: two independent clean reviews of the same fingerprint, "
"launched concurrently",
self.skill,
)
self.assertIn(
"elevated-risk change or any loop that produced a P0/P1 finding",
self.skill,
)
self.assertIn("Use two concurrent fresh reviewers for every round", self.skill)
self.assertNotIn(
"released compatibility, or any loop that produced a P0/P1 finding",
self.skill,
)
def test_cross_references_use_current_step_numbers(self) -> None:
self.assertIn("high-risk conditions in step 12", self.skill)
self.assertIn(
"component delta review using the risk tier and clean-review conditions from step 19",
self.skill,
)
self.assertNotIn("high-risk conditions in step 10", self.skill)
def test_independent_review_uses_no_history_and_event_driven_waits(self) -> None:
required_skill_text = (
'dispatch every reviewer with `fork_turns: "none"`',
"never pass the implementer's accumulated conversation or use a full-history fork",
"Launch both reviewers before waiting",
"one event-driven wait of 240 seconds",
"Do not poll with `list_agents`, separate short waits, progress questions, or no-op "
"`followup_task` messages",
"After one reviewer completes, continue waiting only for the remaining reviewer with "
"another event-driven 240-second wait",
"If an event-driven wait times out while reviewers remain unfinished",
"repeat without polling until a reviewer completes, needs attention, or no unfinished "
"reviewers remain",
)
for text in required_skill_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
self.assertIn('dispatcher uses `fork_turns: "none"`', self.reviewer_brief)
def test_round_budget_preserves_history_across_feedback_cycles(self) -> None:
required_text = (
"Resume or create the task-global review ledger",
"Use the Codex task or thread ID as the stable task identity when available",
"Store the ledger as an ignored operational file at a stable absolute path",
"preserve the same file when work moves to another worktree",
"Never initialize a new counter merely because the task was paused, compacted, "
"handed off, renamed, moved to another worktree, or resumed in another context",
"default autonomous budget for the initial implementation cycle is six "
"fingerprint rounds",
"concrete actionable review feedback starts a post-completion feedback cycle",
"feedback message itself as authorization to append a default budget of two "
"fingerprint rounds to the same ledger",
"A continuation request without concrete new feedback remains in the existing cycle",
"append the feedback cycle's default two-round budget to the same ledger without "
"another authorization prompt",
"Persist enough task identity, used and authorized round budgets",
"Persist the current combined content fingerprint as `ledger.round_fingerprint`",
"A same-round retry is valid only when that value and the authorized budget history "
"match",
"exact packet SHA-256 and fingerprints",
"no unresolved merge stages before fingerprinting",
"resolve to finite regular files",
"Verify the file type after opening",
"read content from that same descriptor",
"Canonicalize each path before opening",
"opened descriptor's device and inode identity",
"require their validated digests to remain unchanged",
"evidence or inventory as new for a canonical root only when its digest is absent",
"Preserve those digest bindings across rounds",
"every distinct root proposed in the same output",
"credited receipt to have a unique content digest and exact command",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_second_related_finding_closes_the_root_cause_group(self) -> None:
required_text = (
"Treat a second related finding in one root-cause group as a closure gate",
"run the complexity reset once",
"scan the complete inventory for sibling scenarios",
"mark the canonical root-cause ID closed",
"Do not reopen it for another local patch without content-new contract evidence or "
"semantic inventory",
"reject aliases, renamed or copied content, and bare unknown IDs",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_snapshot_packet_and_structured_output_bound_repeated_work(self) -> None:
required_skill_text = (
"approximately 12 KB as a soft target",
"indexed evidence files",
"exact paths plus SHA-256 digests",
"Assign stable IDs to every inventory row and evidence item",
"Require one structured JSON object",
"inspection call count",
"approximately 12 source-inspection tool calls per reviewer as a soft budget",
)
required_brief_text = (
"Indexed evidence manifest (`ID | role | exact path | SHA-256 | purpose`)",
"Semantic component dependency map (`component | exact base pathspecs | "
"invalidation reason`)",
'"checked_inventory_ids"',
'"unchecked_inventory_ids"',
'"sibling_scenario_scan"',
'"inspection_call_count"',
'"inspection_budget_reason"',
"A `clean` verdict requires an empty `unchecked_inventory_ids`, "
"`remaining_uncertainty`, and `findings` array",
"Every `focused_probes[].command` must contain the exact executable command that ran",
"Prose-only labels, omitted arguments, and placeholders such as `<focused probe>`",
"return its path, SHA-256 digest, and exact execution command",
)
for text in required_skill_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
for text in required_brief_text:
with self.subTest(text=text):
self.assertIn(text, self.reviewer_brief)
def test_semantic_clean_credit_fails_closed_on_dependency_changes(self) -> None:
required_text = (
"Partition the manifest by the narrowest stable semantic boundaries",
"`api-contract`, `runstate-persistence`, `security-sandbox`, "
"`session-lifecycle`, `integration-runner`, `tests-examples`, and "
"`release-metadata`",
"fingerprint, requirement rows, assertions about runtime behavior, dependency inputs, "
"and risk tier are all unchanged",
"changed or dependency-invalidated component",
"Any ambiguity invalidates the affected clean credit",
"Do not invalidate unrelated components solely because a neighboring file or coarse "
"directory changed",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_intermediate_verification_is_cost_aware_but_final_gate_is_complete(self) -> None:
required_text = (
"Prefer an already successful same-fingerprint focused check over rerunning it",
"never replay cumulative historical verification",
"The focused check earns no final-gate credit",
"the exact clean-reviewed fingerprint must still pass the complete "
"repository-required verification stack",
"check observable host capacity before starting the repository's "
"code-change verification",
)
for text in required_text:
with self.subTest(text=text):
self.assertIn(text, self.skill)
def test_machine_readable_protocol_closes_observed_convergence_gaps(self) -> None:
required_skill_text = (
"python scripts/review_protocol.py packet --packet <packet.json> --task-id "
"<task-id> --ledger <ledger.json>",
"packet path, byte size, SHA-256 digest",
"The implementer assigns every root-cause ID once",
"propose exactly `NEW:<slug>`",
"verification receipt containing the exact command, environment, exit status, "
"non-mutation basis",
"combined, component, and repository fingerprints",
"root evidence IDs absent from the packet's indexed evidence or inventory",
'role: "complete-diff"',
'role: "review-state"',
'role: "repository-status"',
"review_state.evidence_id",
"repository.status_evidence_id",
"Assign every component and all three control artifacts to both reviewers",
"requires the complete-diff digest to match its `complete_diff_sha256`",
"requires `repository.exclusions` to account exactly",
"summary-only inventory row is incomplete",
"active control plane outside the packet",
"authorized budget history",
"immediately preceding round's immutable ledger snapshot plus SHA-256 digest",
"same-round retry or an advance of exactly one round",
"never use the mutable current ledger as its own prior snapshot",
"repository fingerprint covers unfiltered status plus content identity",
"receipt command must exactly match a structured command",
"exact key set emitted for their `file`, `symlink`, `gitlink`, `directory`, or "
"`missing` kind",
"Trust the active implementation control plane to record actual reviewer dispatches",
"assigns every inventory ID to exactly one canonical root",
"sibling scans that use a renamed root or unknown inventory",
"JSON booleans in integer fields",
"add and digest it in the frozen packet",
"python scripts/review_protocol.py reviewer-output --packet <packet.json> --reviewer "
"<reviewer-id> --output <output.json> --task-id <task-id> --ledger <ledger.json>",
)
required_brief_text = (
"## Machine-readable preflight",
"If the packet exceeds 12 KiB",
"NEW:<lowercase-slug>",
'"root_cause_evidence"',
"Every submitted contract evidence ID must name an indexed",
"submitted IDs must be additions owned by that root in the current ledger",
"Every contract evidence ID must resolve to an `evidence_artifacts[].id`",
"JSON booleans are not integers for protocol purposes",
"Each sibling-scenario scan must reuse a canonical root ID",
"verification.preflight_results` as an array of exact, unique `command` and "
"`result` objects",
"ledger file's JSON object to match the packet ledger exactly",
"not already owned by any canonical or distinct proposed root",
"digest maps must bind exactly the currently owned IDs",
"absolute `path` and `sha256` digest",
'role: "review-state"',
'role: "repository-status"',
"The `review_state` packet object contains exactly `evidence_id`",
"extra copied fingerprint or state fields are invalid",
"requires the complete-diff artifact digest to equal its `complete_diff_sha256`",
"Supply the task ID and absolute task-global ledger path independently",
"`current_round` plus `remaining_budget` to equal the sum",
"immediately preceding round's immutable ledger snapshot and its SHA-256 digest",
"same-round retry or advance by exactly one",
"immutable snapshot must be a distinct file",
"an inventory ID owned by another root cannot be reassigned",
"does not provide cryptographic attestation against a malicious control plane",
"current budget history to preserve the prior prefix",
"each inventory ID has exactly one canonical root owner",
"accepts only a receipt path already indexed",
"task or repository-state drift",
"complete typed workspace entries",
"rejects an incomplete or unknown key for any workspace kind",
"unrelated successful commands are ineligible for credit",
'Encode those columns in each `kind: "contract"` inventory object',
'Encode those columns in each `kind: "authority-data-flow"` inventory object',
'Encode those columns in each `kind: "await-boundary"` inventory object',
"requires exclusions to account exactly",
"verification.credited_receipts",
"python scripts/review_protocol.py receipt",
"python scripts/review_protocol.py reviewer-output",
)
required_script_text = (
"PACKET_SOFT_LIMIT_BYTES = 12 * 1024",
"NEW_ROOT_CAUSE_ID",
"validate_packet",
"validate_reviewer_output",
"validate_receipt_data",
)
for text in required_skill_text:
with self.subTest(source="skill", text=text):
self.assertIn(text, self.skill)
for text in required_brief_text:
with self.subTest(source="brief", text=text):
self.assertIn(text, self.reviewer_brief)
for text in required_script_text:
with self.subTest(source="script", text=text):
self.assertIn(text, self.review_protocol)
def test_final_reviewers_inherit_strategy_evidence(self) -> None:
self.assertIn(
"The implementer owns `$implementation-strategy` and supplies its current scope "
"contract in the packet",
self.skill,
)
self.assertIn(
"Reviewers inherit that contract and must not rerun the strategy workflow",
self.skill,
)
self.assertIn(
"Independent reviewers dispatched by `$implementation-final-review` inherit the "
"implementer's recorded implementation scope contract",
self.repo_instructions,
)
self.assertIn(
"The implementer remains responsible for rerunning `$implementation-strategy`",
self.repo_instructions,
)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,111 @@
---
name: implementation-kickoff
description: Start and carry an explicitly invoked openai-agents-python implementation through a fresh isolated worktree and a local PR-ready handoff. Fetch the latest origin/main, keep task changes uncommitted, replay them onto the latest main before final review, run applicable verification and $implementation-final-review, use $pr-draft-summary to generate the complete PR draft and branch name, then create one clean local commit with takeover provenance when applicable. Use only when the user explicitly invokes this skill; never push, open a PR, or mutate GitHub.
---
# Implementation Kickoff
Use this skill as the explicit transition from an agreed implementation scope to isolated execution. Keep the user's original checkout and existing branches unchanged, and finish with a clean local branch that is ready for the user to push.
## Non-negotiable boundaries
- Treat explicit invocation of this skill as authorization to fetch, create one dedicated worktree, rebase or replay task-owned changes, create the final local branch, stage task-owned files, and create one local commit. It never authorizes push, pull-request creation, or any GitHub mutation.
- Do not start during an investigation-only phase or before a required user approval. Finish planning and any required implementation scope contract first.
- Use read-only GitHub access when remote PR evidence is required.
- Preserve unrelated and user-owned changes. Do not remove an existing worktree or rewrite an existing branch to make room for this workflow.
## 1. Establish the task boundary
Record the original requirement, success criteria, intended target (`origin/main` unless the user states otherwise), task-owned paths, compatibility boundary, intentionally unsupported cases, and required repository skills. For a multi-step task, create and maintain the repository's required ExecPlan. An ExecPlan, review packet, ledger, trace, or temporary report is operational-only by default even when repository policy requires creating it; do not add it to the shipped-path manifest unless the original requirement or repository policy explicitly makes that exact path a committed deliverable.
If the current directory is a worktree previously created for this same task in the current conversation, resume it. Otherwise, continue from the user's current checkout only long enough to create a new worktree.
## 2. Create a detached worktree from current main
1. Verify the source checkout's raw status without modifying it.
2. Fetch `origin main`. If the fetch fails, stop rather than claiming a stale ref is current.
3. Record the fetched `origin/main` commit.
4. Choose a unique task-oriented path under the configured Codex worktree root. Check both the filesystem and `git worktree list`; never reuse or delete a collision.
5. Run `git worktree add --detach <worktree> origin/main` and perform all subsequent implementation work there.
6. Confirm the new worktree is detached at the recorded commit and initially clean.
Do not create the final branch yet. A detached worktree makes the eventual `$pr-draft-summary` branch suggestion authoritative and prevents temporary naming from becoming accidental output.
## 3. Implement without task commits
Keep the task diff uncommitted through implementation, focused tests, formatting, and review fixes. Track new files explicitly because ordinary diff statistics omit untracked files. Maintain one canonical shipped-path manifest separately from operational artifacts and require a concrete deliverable reason for every path in it. Use the applicable repository skills and references, including `$implementation-strategy` before user-facing or runtime changes.
When the task can be decomposed without temporarily breaking a supported contract, implement one narrow end-to-end behavior slice at a time and run its focused test before adding the next slice. Do not force cross-cutting migrations or atomic compatibility changes into artificial slices that cannot remain valid independently.
Do not create checkpoint commits. If an external interruption requires extra protection, leave the dedicated worktree intact or use a clearly named temporary stash; restore the changes before continuing and do not treat the stash as a deliverable.
### Taking over an existing pull request
When the user asks to complete another author's pull request:
1. Refresh the PR metadata, head, discussion, and complete three-dot diff through read-only access.
2. Confirm that the PR is still an appropriate takeover source. Do not treat an already merged PR as an active takeover.
3. Apply the original PR's complete task diff onto the worktree based on current `origin/main`; do not derive the final branch from the contributor branch and do not preserve its intermediate commit topology.
4. Record the original PR number, PR author login, verified commit identity, existing valid `Co-authored-by` trailers, linked issues, and the original intent that the replacement must preserve.
5. If a valid author identity cannot be obtained from the PR's commits, stop before committing and ask the user. Never invent an email address.
## 4. Replay the complete task onto the latest main
After implementation, focused tests, and formatting are stable, fetch `origin main` again. If it advanced:
1. Confirm that every local change is task-owned.
2. Save tracked and untracked task changes in a uniquely named temporary stash.
3. Rebase the detached HEAD onto `origin/main`. With no task commits, this updates the empty local commit range to the new base.
4. Reapply the stash and confirm it was removed only after a clean application.
5. Resolve conflicts only when the requirement and surrounding source make the correct result unambiguous. Otherwise preserve the stash and conflict evidence, then stop for user direction.
6. Rerun formatting and every focused check affected by the new base.
Record this observed `origin/main` commit as the final-base candidate. Do not call an older base "latest" merely because its changes appear unrelated.
## 5. Complete final review and verification
Run the repository's applicable completion gates against the complete task-owned diff on the final-base candidate. Before freezing the first review fingerprint, inspect the actual final commit-hook configuration and run the exact safe, non-committing equivalent of every hook step that can rewrite a shipped path. Repeat rewriting steps until content-idempotent, and verify generated-file hashes or provenance after normalization. Generate final-review evidence with `review_state.py --complete-diff-output <complete.diff>` so ordinary task-owned untracked files are present in the reviewed diff without staging them. For runtime code, tests, examples, build or test behavior, or behavior-impacting docs, run `$implementation-final-review` and the required `$code-change-verification` sequence in their mandated order. Honor their fingerprint and invalidation rules.
Skip those skills only when their own repository rules say the task is ineligible, such as a repo-meta-only change. Do not weaken an eligible gate merely because the diff is small.
Do not create the branch or commit when review is non-converging, verification fails, required evidence is missing, or the final content lacks clean-review credit.
## 6. Generate the complete PR handoff
Invoke `$pr-draft-summary` only after review and verification apply to the final content. Give it a self-contained packet containing the original requirement, implementation scope contract, important decisions and intent, complete changed-path inventory including untracked files, final diff and statistics, compatibility notes, issue references, and takeover provenance.
The worktree is intentionally detached. Tell `$pr-draft-summary` to treat the current branch value `HEAD` as "no branch yet" and require a concrete unused branch-name suggestion; never accept `HEAD` as the suggestion. The description must explain the complete final change and its motivation, not only the last review fix.
For a takeover, begin the description with prose such as `This pull request supersedes #<number> and ...`. Preserve any separate issue-closing line only when the final implementation actually resolves that issue.
If the diff, scope, base, behavior claim, issue relationship, or provenance changes after generation, regenerate the entire PR handoff.
## 7. Recheck main and create one commit
Fetch `origin main` once more immediately before creating the branch. If it differs from the final-base candidate, return to section 4 and replay onto the new base. Then apply `$implementation-final-review` step 20: preserve clean-review credit only when the verified base-advance closure proves an identical task diff and component workspace plus a complete, non-overlapping upstream dependency/tooling audit. Even when that closure applies, rerun every mandatory final verification gate on the new base and regenerate the PR handoff. If any closure condition is missing or ambiguous, repeat affected checks, fresh independent review, verification, and PR handoff. Once stable:
1. Check whether the suggested branch exists locally, remotely, or in another worktree. Ask `$pr-draft-summary` for the next available numeric suffix and regenerate the handoff before creating a colliding branch.
2. Create the exact suggested branch in the task worktree.
3. Stage only the task-owned shipped-path manifest, including intended new files. Compare the staged changed-path set byte-for-byte with the canonical shipped manifest before committing; any missing or unexpected path is a hard stop. In particular, do not stage an ignored ExecPlan or review artifact merely because it was required during implementation.
4. Use the PR draft title as the commit subject.
5. For a takeover, add the verified original PR author as `Co-authored-by: Name <email>`, retain distinct valid co-author trailers from the imported commits, and deduplicate identities.
6. Create exactly one commit. Let repository hooks run normally.
Branch creation and committing identical content are repository bookkeeping and do not invalidate clean content review. If a hook or manual fix changes task content, stop, classify the change under `$implementation-final-review`, and rerun every invalidated gate and `$pr-draft-summary` before replacing or amending the commit.
## 8. Validate and hand off
Run `python .agents/skills/implementation-kickoff/scripts/validate_handoff.py --repo <worktree> --base <final-base> --expected-branch <branch> --shipped-path-manifest <manifest>`. For a takeover, also pass `--required-trailer-email <verified-email>` for each identity that must be credited. The shipped-path manifest must be a finite regular file whose type and content are read from one opened descriptor. It contains one exact repository-relative shipped path per line and excludes operational artifacts.
Independently confirm that the committed diff has the reviewed content fingerprint when final review supplied one. The validator checks Git topology and repository cleanliness; it does not replace semantic review or fingerprint verification.
Leave the worktree in place. Report the worktree path, final observed base commit, branch, commit SHA and subject, verification results, review status, PR title and description, and whether takeover provenance was included. Treat the worktree path and validator output as local diagnostics: never include them in the PR title, PR description, or other copy-ready external text. State explicitly that nothing was pushed and no pull request was created.
## Failure behavior
- Fetch failure: stop without creating or updating the final branch.
- Worktree or branch collision: preserve the existing target and choose a new unused path or regenerated branch suggestion.
- Replay conflict: retain recoverable task changes and ask for direction when the correct resolution is ambiguous.
- Review or verification failure: leave the detached task worktree for continuation; do not package a commit as ready.
- Commit-hook mutation: invalidate affected evidence, fix the missing hook-parity preflight or generated-file normalization, and repeat the required gates.
- Non-clean or multi-commit final state: do not hand off as complete until corrected without discarding user-owned work.
@@ -0,0 +1,6 @@
interface:
display_name: "Implementation Kickoff"
short_description: "Run implementation in a fresh PR-ready worktree"
default_prompt: "Use $implementation-kickoff to start this implementation in a fresh worktree and leave one verified local commit."
policy:
allow_implicit_invocation: false
@@ -0,0 +1,297 @@
from __future__ import annotations
import os
import subprocess
import tempfile
import unittest
from argparse import Namespace
from pathlib import Path
from unittest import mock
from validate_handoff import load_shipped_paths, validate
class ValidateHandoffTests(unittest.TestCase):
def setUp(self) -> None:
self.temporary_directory = tempfile.TemporaryDirectory()
self.addCleanup(self.temporary_directory.cleanup)
self.root = Path(self.temporary_directory.name)
self.repo = self.root / "repo"
self.repo.mkdir()
self._git("init", "-q", "-b", "main")
self._git("config", "user.name", "Test User")
self._git("config", "user.email", "test@example.com")
(self.repo / "README.md").write_text("base\n")
self._git("add", "README.md")
self._git("commit", "-qm", "base")
self.base = self._git("rev-parse", "HEAD").stdout.strip()
self._git("checkout", "-qb", "feat/review-workflow")
def _git(self, *args: str) -> subprocess.CompletedProcess[str]:
return subprocess.run(
("git", *args),
cwd=self.repo,
check=True,
capture_output=True,
text=True,
)
def _commit(self, paths: dict[str, str]) -> None:
for relative_path, content in paths.items():
path = self.repo / relative_path
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(content)
self._git("add", *paths)
self._git("commit", "-qm", "change workflow")
def _args(self, manifest: Path) -> Namespace:
return Namespace(
repo=self.repo,
base=self.base,
expected_branch="feat/review-workflow",
required_trailer_email=[],
shipped_path_manifest=manifest,
)
def test_exact_shipped_manifest_is_valid(self) -> None:
self._commit({"src/change.py": "value = 1\n"})
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
report, failures = validate(self._args(manifest))
self.assertEqual(failures, [])
self.assertTrue(report["valid"])
self.assertEqual(report["shipped_paths"], ["src/change.py"])
def test_operational_file_not_in_manifest_fails(self) -> None:
self._commit(
{
"src/change.py": "value = 1\n",
"plans/task.md": "operational plan\n",
}
)
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
report, failures = validate(self._args(manifest))
self.assertFalse(report["valid"])
self.assertIn("unexpected=['plans/task.md']", failures[0])
def test_explicit_ignored_deliverable_is_valid_after_force_staging(self) -> None:
(self.repo / ".git" / "info" / "exclude").write_text("fixture.generated\n")
fixture = self.repo / "fixture.generated"
fixture.write_text("shipped fixture\n")
self._git("add", "-f", "fixture.generated")
self._git("commit", "-qm", "add ignored fixture")
manifest = self.root / "shipped.paths"
manifest.write_text("fixture.generated\n")
report, failures = validate(self._args(manifest))
self.assertEqual(failures, [])
self.assertTrue(report["valid"])
self.assertEqual(report["shipped_paths"], ["fixture.generated"])
def test_manifest_paths_must_be_normalized_and_unique(self) -> None:
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\nsrc/change.py\n")
with self.assertRaisesRegex(ValueError, "Duplicate shipped-path manifest entry"):
load_shipped_paths(manifest)
manifest.write_text("../outside.txt\n")
with self.assertRaisesRegex(ValueError, "normalized repository-relative paths"):
load_shipped_paths(manifest)
@unittest.skipUnless(hasattr(os, "mkfifo"), "Requires POSIX FIFO support.")
def test_manifest_file_type_is_verified_after_open(self) -> None:
fifo = self.root / "shipped.paths"
os.mkfifo(fifo)
with (
mock.patch.object(Path, "read_text", return_value="src/change.py\n"),
self.assertRaisesRegex(ValueError, "regular file"),
):
load_shipped_paths(fifo)
def test_body_line_is_not_accepted_as_coauthor_trailer(self) -> None:
path = self.repo / "src" / "change.py"
path.parent.mkdir()
path.write_text("value = 1\n")
self._git("add", "src/change.py")
self._git(
"commit",
"-qm",
"change workflow",
"-m",
"Co-authored-by: Example User <example@example.com>",
"-m",
"This paragraph makes the preceding line part of the body.",
)
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
args = self._args(manifest)
args.required_trailer_email = ["example@example.com"]
report, failures = validate(args)
self.assertFalse(report["valid"])
self.assertTrue(
any("Missing required Co-authored-by trailer" in failure for failure in failures)
)
def test_terminal_coauthor_trailer_is_accepted(self) -> None:
path = self.repo / "src" / "change.py"
path.parent.mkdir()
path.write_text("value = 1\n")
self._git("add", "src/change.py")
self._git(
"commit",
"-qm",
"change workflow",
"-m",
"Commit body.",
"-m",
"Co-authored-by: Example User <example@example.com>",
)
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
args = self._args(manifest)
args.required_trailer_email = ["EXAMPLE@example.com"]
report, failures = validate(args)
self.assertEqual(failures, [])
self.assertEqual(report["coauthor_trailer_emails"], ["example@example.com"])
def test_assume_unchanged_path_is_not_a_clean_handoff(self) -> None:
self._commit({"src/change.py": "value = 1\n"})
self._git("update-index", "--assume-unchanged", "README.md")
(self.repo / "README.md").write_text("hidden change\n")
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
report, failures = validate(self._args(manifest))
self.assertFalse(report["valid"])
self.assertTrue(any("assume-unchanged" in failure for failure in failures))
def test_materialized_skip_worktree_path_is_not_a_clean_handoff(self) -> None:
self._commit({"src/change.py": "value = 1\n"})
self._git("update-index", "--skip-worktree", "README.md")
(self.repo / "README.md").write_text("hidden change\n")
manifest = self.root / "shipped.paths"
manifest.write_text("src/change.py\n")
report, failures = validate(self._args(manifest))
self.assertFalse(report["valid"])
self.assertTrue(any("skip-worktree" in failure for failure in failures))
def test_ignored_dirty_submodule_is_not_a_clean_handoff(self) -> None:
source = self.root / "dependency-source"
source.mkdir()
subprocess.run(("git", "init", "-q", str(source)), check=True)
subprocess.run(
("git", "-C", str(source), "config", "user.name", "Submodule Test"),
check=True,
)
subprocess.run(
("git", "-C", str(source), "config", "user.email", "submodule@example.test"),
check=True,
)
(source / "tracked.txt").write_text("committed\n")
subprocess.run(("git", "-C", str(source), "add", "tracked.txt"), check=True)
subprocess.run(("git", "-C", str(source), "commit", "-qm", "initial"), check=True)
self._git(
"-c",
"protocol.file.allow=always",
"submodule",
"add",
"-q",
str(source),
"vendor/dependency",
)
self._git(
"config",
"-f",
".gitmodules",
"submodule.vendor/dependency.ignore",
"all",
)
self._git("add", ".gitmodules", "vendor/dependency")
self._git("commit", "-qm", "add dependency")
manifest = self.root / "shipped.paths"
manifest.write_text(".gitmodules\nvendor/dependency\n")
(self.repo / "vendor" / "dependency" / "tracked.txt").write_text("dirty\n")
self.assertEqual(self._git("status", "--porcelain=v1").stdout, "")
report, failures = validate(self._args(manifest))
self.assertFalse(report["valid"])
self.assertIn("Worktree is not clean.", failures)
def test_submodule_hidden_index_path_is_not_a_clean_handoff(self) -> None:
source = self.root / "dependency-source"
source.mkdir()
subprocess.run(("git", "init", "-q", str(source)), check=True)
subprocess.run(
("git", "-C", str(source), "config", "user.name", "Submodule Test"),
check=True,
)
subprocess.run(
("git", "-C", str(source), "config", "user.email", "submodule@example.test"),
check=True,
)
(source / "tracked.txt").write_text("committed\n")
subprocess.run(("git", "-C", str(source), "add", "tracked.txt"), check=True)
subprocess.run(("git", "-C", str(source), "commit", "-qm", "initial"), check=True)
self._git(
"-c",
"protocol.file.allow=always",
"submodule",
"add",
"-q",
str(source),
"vendor/dependency",
)
self._git("commit", "-qam", "add dependency")
manifest = self.root / "shipped.paths"
manifest.write_text(".gitmodules\nvendor/dependency\n")
self._git(
"-C",
"vendor/dependency",
"update-index",
"--assume-unchanged",
"tracked.txt",
)
(self.repo / "vendor" / "dependency" / "tracked.txt").write_text("hidden change\n")
self.assertEqual(
self._git("status", "--porcelain=v1", "--ignore-submodules=none").stdout,
"",
)
report, failures = validate(self._args(manifest))
self.assertFalse(report["valid"])
self.assertFalse(report["clean"])
self.assertTrue(
any("assume-unchanged=vendor/dependency/tracked.txt" in failure for failure in failures)
)
def test_missing_repository_report_is_explicitly_invalid(self) -> None:
args = self._args(self.root / "unused.paths")
args.repo = self.root / "missing"
report, failures = validate(args)
self.assertFalse(report["valid"])
self.assertEqual(
failures,
[f"Repository path does not exist: {args.repo.resolve()}"],
)
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,288 @@
#!/usr/bin/env python3
"""Validate the Git invariants of an implementation-kickoff handoff."""
from __future__ import annotations
import argparse
import json
import os
import re
import stat
import subprocess
import sys
from pathlib import Path, PurePosixPath
class GitCommandError(RuntimeError):
"""Report a failed Git inspection command."""
def run_git(repo: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
result = subprocess.run(
["git", *args],
cwd=repo,
check=False,
capture_output=True,
text=True,
errors="surrogateescape",
)
if check and result.returncode != 0:
command = "git " + " ".join(args)
detail = result.stderr.strip() or result.stdout.strip() or "unknown Git error"
raise GitCommandError(f"{command} failed: {detail}")
return result
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Validate a clean, single-commit implementation-kickoff handoff."
)
parser.add_argument("--repo", type=Path, required=True, help="Path to the task worktree.")
parser.add_argument(
"--base",
required=True,
help="Expected parent commit or ref for the single handoff commit.",
)
parser.add_argument(
"--expected-branch",
required=True,
help="Exact local branch name expected at HEAD.",
)
parser.add_argument(
"--required-trailer-email",
action="append",
default=[],
help="Email that must appear in a Co-authored-by trailer. Repeat as needed.",
)
parser.add_argument(
"--shipped-path-manifest",
type=Path,
help=(
"File containing the exact repository-relative paths expected in the handoff commit, "
"one per line."
),
)
parser.add_argument("--json", action="store_true", help="Emit the result as JSON.")
return parser.parse_args()
def _nonblocking_opener(path: str, flags: int) -> int:
return os.open(path, flags | getattr(os, "O_NONBLOCK", 0))
def load_shipped_paths(path: Path) -> set[str]:
with open(path, "rb", opener=_nonblocking_opener) as file:
if not stat.S_ISREG(os.fstat(file.fileno()).st_mode):
raise ValueError(f"Shipped-path manifest must be a regular file: {path}")
lines = file.read().decode().splitlines()
if not lines:
raise ValueError(f"Shipped-path manifest is empty: {path}")
shipped_paths: set[str] = set()
for line_number, raw_path in enumerate(lines, start=1):
if not raw_path:
raise ValueError(f"Shipped-path manifest contains a blank line at {line_number}.")
path_value = PurePosixPath(raw_path)
if path_value.is_absolute() or ".." in path_value.parts or str(path_value) != raw_path:
raise ValueError(
"Shipped-path manifest entries must be normalized repository-relative paths: "
f"{raw_path!r}."
)
if raw_path in shipped_paths:
raise ValueError(f"Duplicate shipped-path manifest entry: {raw_path}")
shipped_paths.add(raw_path)
return shipped_paths
def is_repository_root(path: Path) -> bool:
if not path.is_dir():
return False
result = run_git(path, "rev-parse", "--show-toplevel", check=False)
return result.returncode == 0 and Path(result.stdout.strip()).resolve() == path.resolve()
def hidden_index_paths(
repo: Path,
prefix: str = "",
seen_repositories: frozenset[Path] = frozenset(),
) -> list[str]:
resolved_repo = repo.resolve()
if resolved_repo in seen_repositories:
return []
seen_repositories |= {resolved_repo}
def display_path(relative_path: str) -> str:
return f"{prefix}/{relative_path}" if prefix else relative_path
hidden_paths: list[str] = []
for entry in run_git(repo, "ls-files", "-v", "-z").stdout.split("\0"):
if len(entry) < 3 or entry[1] != " ":
continue
tag = entry[0]
relative_path = entry[2:]
if tag.islower():
hidden_paths.append(f"assume-unchanged={display_path(relative_path)}")
elif tag == "S":
candidate = repo / relative_path
if candidate.exists() or candidate.is_symlink():
hidden_paths.append(f"materialized skip-worktree={display_path(relative_path)}")
for entry in run_git(repo, "ls-files", "--stage", "-z").stdout.split("\0"):
metadata, separator, relative_path = entry.partition("\t")
fields = metadata.split()
if not separator or len(fields) != 3 or fields[0] != "160000" or fields[2] != "0":
continue
submodule_path = repo / relative_path
if is_repository_root(submodule_path):
hidden_paths.extend(
hidden_index_paths(
submodule_path,
display_path(relative_path),
seen_repositories,
)
)
return sorted(hidden_paths)
def validate(args: argparse.Namespace) -> tuple[dict[str, object], list[str]]:
repo = args.repo.expanduser().resolve()
failures: list[str] = []
if not repo.is_dir():
return {"repo": str(repo), "valid": False}, [f"Repository path does not exist: {repo}"]
top_level = Path(run_git(repo, "rev-parse", "--show-toplevel").stdout.strip()).resolve()
if top_level != repo:
failures.append(f"--repo must be the worktree root: expected {top_level}, got {repo}")
status = run_git(
repo,
"status",
"--porcelain=v1",
"--untracked-files=all",
"--ignore-submodules=none",
).stdout
if status:
failures.append("Worktree is not clean.")
hidden_paths = hidden_index_paths(repo)
if hidden_paths:
failures.append(f"Index flags can hide worktree changes: {hidden_paths}.")
branch_result = run_git(repo, "symbolic-ref", "--quiet", "--short", "HEAD", check=False)
branch = branch_result.stdout.strip() if branch_result.returncode == 0 else None
if branch is None:
failures.append("HEAD is detached.")
elif branch != args.expected_branch:
failures.append(f"Current branch is {branch!r}, expected {args.expected_branch!r}.")
base = run_git(repo, "rev-parse", f"{args.base}^{{commit}}").stdout.strip()
head = run_git(repo, "rev-parse", "HEAD").stdout.strip()
parent_line = run_git(repo, "show", "-s", "--format=%P", "HEAD").stdout.strip()
parents = parent_line.split() if parent_line else []
if len(parents) != 1:
failures.append(f"HEAD must have exactly one parent, found {len(parents)}.")
elif parents[0] != base:
failures.append(f"HEAD parent is {parents[0]}, expected base {base}.")
ahead_text = run_git(repo, "rev-list", "--count", f"{base}..{head}").stdout.strip()
ahead = int(ahead_text)
if ahead != 1:
failures.append(f"HEAD must be exactly one commit ahead of base, found {ahead} commits.")
shipped_manifest: str | None = None
shipped_paths: list[str] | None = None
if args.shipped_path_manifest is not None:
manifest_path = args.shipped_path_manifest.expanduser().resolve()
expected_paths = load_shipped_paths(manifest_path)
actual_paths = {
path
for path in run_git(
repo,
"diff",
"--name-only",
"--no-renames",
"-z",
f"{base}..{head}",
).stdout.split("\0")
if path
}
missing_paths = sorted(expected_paths - actual_paths)
unexpected_paths = sorted(actual_paths - expected_paths)
if missing_paths or unexpected_paths:
failures.append(
"Committed paths do not match the shipped-path manifest: "
f"missing={missing_paths}, unexpected={unexpected_paths}."
)
shipped_manifest = str(manifest_path)
shipped_paths = sorted(expected_paths)
subject = run_git(repo, "show", "-s", "--format=%s", "HEAD").stdout.strip()
if not subject:
failures.append("HEAD commit subject is empty.")
trailer_values = run_git(
repo,
"show",
"-s",
"--format=%(trailers:key=Co-authored-by,valueonly,unfold,separator=%x00)",
"HEAD",
).stdout.rstrip("\n")
email_pattern = re.compile(r"^.+\s+<([^>\n]+)>$")
trailer_emails = {
match.group(1).strip().casefold()
for value in trailer_values.split("\0")
if (match := email_pattern.match(value)) is not None
}
for email in args.required_trailer_email:
if email.strip().casefold() not in trailer_emails:
failures.append(f"Missing required Co-authored-by trailer for {email}.")
report: dict[str, object] = {
"repo": str(repo),
"base": base,
"head": head,
"branch": branch,
"subject": subject,
"ahead": ahead,
"clean": not status and not hidden_paths,
"coauthor_trailer_emails": sorted(trailer_emails),
"shipped_path_manifest": shipped_manifest,
"shipped_paths": shipped_paths,
"valid": not failures,
}
return report, failures
def main() -> int:
args = parse_args()
try:
report, failures = validate(args)
except (GitCommandError, OSError, UnicodeError, ValueError) as exc:
report = {"repo": str(args.repo.expanduser().resolve()), "valid": False}
failures = [str(exc)]
if args.json:
print(json.dumps({**report, "failures": failures}, indent=2, sort_keys=True))
else:
status = "valid" if not failures else "invalid"
print(f"Implementation handoff: {status}")
for key in (
"repo",
"base",
"head",
"branch",
"subject",
"ahead",
"clean",
"shipped_path_manifest",
"shipped_paths",
):
if key in report:
print(f"{key}: {report[key]}")
for failure in failures:
print(f"error: {failure}", file=sys.stderr)
return 1 if failures else 0
if __name__ == "__main__":
raise SystemExit(main())
+131 -26
View File
@@ -1,47 +1,150 @@
---
name: implementation-strategy
description: Decide how to implement runtime and API changes in openai-agents-python before editing code. Use when a task changes exported APIs, runtime behavior, serialized state, tests, or docs and you need to choose the compatibility boundary, whether shims or migrations are warranted, and when unreleased interfaces can be rewritten directly.
description: Choose compatibility-aware scope for runtime and API changes in openai-agents-python. Use before initial implementation and each review-feedback batch to decide whether to patch, reset the design, preserve compatibility, or reject unsupported cases.
---
# Implementation Strategy
## Overview
## Workflow
Use this skill before editing code when the task changes runtime behavior or anything that might look like a compatibility concern. The goal is to keep implementations simple while protecting real released contracts.
## Quick start
1. Identify the surface you are changing: released public API, unreleased branch-local API, internal helper, persisted schema, wire protocol, CLI/config/env surface, or docs/examples only.
2. Determine the latest release boundary from `origin` first, and only fall back to local tags when remote tags are unavailable:
1. Identify the surface you are changing or reviewing: released public API, unreleased branch-local API, internal helper, persisted schema, wire protocol, CLI/config/env surface, or docs/examples only.
2. Determine the latest release tag to use as the compatibility baseline from `origin` first, and only fall back to local tags when remote tags are unavailable:
```bash
BASE_TAG="$(.agents/skills/final-release-review/scripts/find_latest_release_tag.sh origin 'v*' 2>/dev/null || git tag -l 'v*' --sort=-v:refname | head -n1)"
echo "$BASE_TAG"
```
3. Judge breaking-change risk against that latest release tag, not against unreleased branch churn or post-tag changes already on `main`. If the command fell back to local tags, treat the result as potentially stale and say so.
4. Prefer the simplest implementation that satisfies the current task. Update callers, tests, docs, and examples directly instead of preserving superseded unreleased interfaces.
5. Add a compatibility layer only when there is a concrete released consumer, an otherwise supported durable external state boundary that requires it, or when the user explicitly asks for a migration path.
Report a local-tag fallback as potentially stale.
3. Record the implementation scope contract below before coding.
4. Identify the nearest existing implementation pipeline and the functions, types, or modules that are the source of truth for each affected concern. Prefer adapting the required input into that pipeline over creating parallel schema, metadata, validation, naming, or execution machinery.
5. Choose the smallest coherent change using the core decision rules. Add compatibility machinery only for a required supported boundary.
6. Before editing each review-feedback batch, run the review gate against the complete branch diff, not only the latest revision.
7. Before handoff, run the effectiveness check. If any answer is no, revise the design.
## Compatibility boundary rules
## Implementation scope contract
- Released public API or documented external behavior: preserve compatibility or provide an explicit migration path.
- Persisted schema, serialized state, wire protocol, CLI flags, environment variables, and externally consumed config: treat as compatibility-sensitive when they are part of the latest release or when the repo explicitly intends to preserve them across commits, processes, or machines.
- Python-specific durable surfaces such as `RunState`, session persistence, exported dataclass constructor order, and documented model/provider configuration should be treated as compatibility-sensitive when they were part of the latest release tag or are explicitly supported as a shared durability boundary.
- Interface changes introduced only on the current branch: not a compatibility target. Rewrite them directly.
- Interface changes present on `main` but added after the latest release tag: not a semver breaking change by themselves. Rewrite them directly unless they already define a released or explicitly supported durable external state boundary.
- Internal helpers, private types, same-branch tests, fixtures, and examples: update them directly instead of adding adapters.
- Unreleased persisted schema versions on `main` may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported. When you do that, update the support set and tests together so the boundary is explicit.
Record these four items in the plan or working notes, and update them before widening or narrowing the implementation:
## Default implementation stance
1. **Required behavior:** The smallest user-visible scenario that must work.
2. **Compatibility requirements:** Supported released behavior or a durable boundary that must remain usable.
3. **Intentionally unsupported cases:** Nearby inputs or shapes to reject, including when and how rejection occurs.
4. **Supported alternative:** An existing wrapper, override, adapter, configuration, or lower-level API; state `none` when absent.
- Prefer deletion or replacement over aliases, overloads, shims, feature flags, and dual-write logic when the old shape is unreleased.
- Do not preserve a confusing abstraction just because it exists in the current branch diff.
- If review feedback claims a change is breaking, verify it against the latest release tag and actual external impact before accepting the feedback.
- If a change truly crosses the latest released contract boundary, call that out explicitly in the ExecPlan, release notes context, and user-facing summary.
If the intentionally unsupported cases cannot be stated clearly, do not start by adding a general resolver. First define a narrower behavior contract. If no adequate supported alternative exists, add one only when the task requires it; do not invent one speculatively.
A released-version reproducer proves reachability, not support. Treat the exact shape as a compatibility requirement only when intentionally covered by public documentation, examples, tests, or typing; required by a durable boundary; or backed by concrete user reliance or maintainer intent. Otherwise record the risk and prefer early rejection with an existing supported alternative.
## Review-feedback gate
Repeat this gate before editing each new feedback batch:
```text
Review checkpoint:
- Root cause and required behavior:
- Compatibility evidence and unsupported cases:
- Source of truth:
- Behavior-space change: narrows / unchanged / widens
- Action: focused patch / complexity reset / reject as unsupported
```
Classify each finding as a required-behavior defect, supported compatibility requirement, another combination of the same implementation dimensions, or unrelated issue. Widening the behavior space requires new contract evidence.
If a second related finding would add another condition, protocol hop, compatibility case, or test permutation to the same abstraction, stop patching and run the complexity reset. Continue only when concrete evidence puts the exact case in the required or supported contract.
After a reset spec is frozen, classify each later finding as a violation of that spec, an evidence-backed reason to revise it, an intentionally unsupported case, or an unrelated issue. Do not resume incremental patching merely because the new finding is locally fixable.
Example: if successive findings require traversing a direct wrapper, partial, nested wrapper, descriptor, and bound method, do not add another hop. Unless arbitrary wrapper graphs are supported, retain the required plain callable behavior and reject ambiguous wrappers before invocation.
## Core decision rules
- Preserve released public APIs, documented behavior, and supported durable boundaries, or provide an explicit migration path.
- Rewrite branch-local interfaces, internal helpers, same-branch tests, and post-release additions on `main` directly unless they already define a supported durable boundary.
- Unreleased persisted schema versions may be renumbered or squashed when intermediate snapshots are intentionally unsupported; update the support set and tests together.
- Do not equate a broad Python or third-party protocol with support for every representable shape.
- Prefer the nearest existing pipeline and one source of truth for schema, documentation, validation, identity, and invocation.
- Treat an interface as everything a caller must know to use the behavior correctly, including ordering, errors, lifecycle, configuration, and performance constraints when relevant; do not judge its size from the signature alone.
- Apply the deletion test before retaining a new abstraction: keep it when removing it would distribute required complexity across callers, but remove it when the complexity itself would disappear.
- Add a replaceable boundary only for demonstrated variation, ownership, or testability. One hypothetical adapter or a test-only indirection is not enough when the existing pipeline already provides a stable boundary.
- Add abstractions, state, classifications, branches, configuration, dependencies, or parallel paths only for a stated requirement, supported contract, or verified risk.
- Prefer deletion or direct replacement for unreleased code. Treat branch-local implementation and tests as disposable.
- Prefer an actionable construction- or validation-time error plus an existing alternative over partial protocol emulation.
- Keep unrelated refactors and pre-existing failures out of the patch.
- Test the required behavior, the nearest supported path, and one representative case per unsupported category rather than every constructible permutation.
- Call out changes to supported released behavior or durable formats in the plan and handoff.
## Complexity reset
Stop extending the current design when:
- Related findings keep combining the same dimensions, such as wrappers, descriptors, generics, binding, context injection, sync/async classification, or provider variants.
- The patch interprets a host-language or third-party protocol, or separately infers representations that can drift.
- A narrow requirement needs recursive resolution, cached modes, new state, or unrelated subsystem changes.
- Tests enumerate mechanics or the full diff keeps growing while the required scenario remains small.
When a trigger fires:
1. Stop editing and freeze the current revision for analysis instead of addressing comments one by one.
2. Group findings by root cause and re-read the original requirement, scope contract, and supported release or durable boundaries.
3. Write a candidate finding-derived reset spec using those inputs. Do not treat accumulated review explanations, branch-local machinery, or same-branch tests as requirements.
4. Audit the candidate spec against every affected entry point and the nearest existing supported paths. Revise it as needed, then freeze it before resuming edits.
5. Compare the complete diff with the intended merge base or latest release tag, and map each abstraction, branch, and test to the frozen spec as `retain`, `replace`, or `delete`.
6. Delete machinery with no mapping, narrow the contract, and reject unsupported cases before side effects.
7. Rebuild tests around required behavior, supported compatibility, cross-entry-point consistency, and representative unsupported categories.
8. Evaluate later findings against the frozen spec. Stop and record new contract evidence before changing the spec or widening the behavior space.
Use this compact reset spec in the plan or working notes:
```text
Finding-derived reset spec:
- Original required outcome:
- Supported release or durable boundaries:
- Grouped findings and common root cause:
- Invariants across affected entry points:
- Allowed states and behavior:
- Rejected states, failure timing, and side-effect boundary:
- Trusted and untrusted boundaries:
- Single sources of truth:
- Persistence, resume, cleanup, or other lifecycle semantics:
- Non-goals and supported alternatives:
- Representative test categories:
- Diff reset: retain / replace / delete:
```
The candidate spec is a falsifiable design hypothesis, not a record of the current implementation. The audit may correct it before it is frozen. Once frozen, require explicit evidence to revise it and re-run the complete diff mapping after any revision.
Do not wait for the user or reviewer to request this reset when the signals are already present.
## Effectiveness check
Before declaring the design complete, answer all of these with concrete evidence:
- Can the required behavior be described without naming internal helper types or reflection mechanics?
- Does the implementation reuse the nearest existing pipeline rather than maintain a parallel interpretation?
- Does every new abstraction and branch map to the scope contract or a verified risk?
- Would deleting each new abstraction merely push required complexity into multiple callers, and does each new boundary correspond to demonstrated variation, ownership, or testability?
- Are unsupported neighboring cases rejected before side effects with an existing alternative identified?
- Do tests exercise the highest stable caller boundary that reproduces the required behavior, with expected values independent of the implementation logic?
- Do the complete diff and tests cover the contract without making every constructible permutation supported?
- Does the latest review revision shrink or preserve the behavior space rather than widen it without evidence?
- When a complexity reset occurred, does every retained abstraction, branch, and test map to the frozen reset spec, with later findings classified against it?
## SDK-specific decision rules
- Treat released `RunState`, session persistence, and other explicitly durable serialized state as compatibility-sensitive across commits, processes, and machines.
- When unsupported OpenAI API or provider-adapter behavior already has a released default path, avoid turning it into a default hard error unless the latest release boundary justifies that break. Prefer an opt-in strict mode such as `strict_feature_validation=True`, while keeping the default path compatible through warning, ignoring unsupported data, or a clearly non-empty placeholder.
- For OpenAI API feature gaps, evaluate streaming and non-streaming paths together. Custom tool calls, multi-choice Chat Completions chunks, non-text tool outputs, and similar provider payload differences must not be strict in one path and permissive or malformed in the other.
- When a change creates new public SDK behavior, do not expose it only through hard-coded module globals. Prefer an explicit public configuration object or parameter, preserve the existing default behavior when compatibility-sensitive, and make opt-in SDK defaults explicit.
- For SDK-owned public configuration, accept existing typed objects and equivalent dictionaries at the public input boundary while preserving the internal typed representation. Respect the owning model's validation and extra-field policy instead of recreating arbitrary third-party schema semantics.
- Keep model-specific settings inside the existing `model_settings` parameter. Preserve released constructor arguments, typed-object behavior, and provider request payloads when adding dictionary support.
- Append new optional fields or constructor parameters to public dataclasses and constructors. Do not insert them before existing public fields unless you also provide a compatibility layer and regression coverage for the old positional call shape.
- Treat threshold and quota values as part of the API design when they affect runtime behavior. Distinguish OpenAI platform quota-derived values from defensive SDK defaults; if the value is not anchored in a documented platform limit, avoid making it an unconditional default-on behavior.
- Define `None` semantics deliberately for public configuration. For example, use separate meanings for "feature disabled or no SDK limit", "use SDK default limits", and "disable only this specific limit" rather than relying on implicit truthiness checks.
## When to stop and confirm
- The change would alter behavior shipped in the latest release tag.
- The change would alter supported behavior shipped in the latest release tag, or concrete evidence shows material reliance on behavior that the release incidentally accepted.
- The change would modify durable external data, protocol formats, or serialized state.
- The correct solution would materially expand beyond the requested outcome or require unrelated architectural work.
- A complexity reset trigger fires and the narrower replacement would change an already released supported contract rather than branch-local code.
- The user explicitly asked for backward compatibility, deprecation, or migration support.
## Output expectations
@@ -49,4 +152,6 @@ Use this skill before editing code when the task changes runtime behavior or any
When this skill materially affects the implementation approach, state the decision briefly in your reasoning or handoff, for example:
- `Compatibility boundary: latest release tag v0.x.y; branch-local interface rewrite, no shim needed.`
- `Compatibility boundary: released RunState schema; preserve compatibility and add migration coverage.`
- `Implementation scope contract: support X; preserve Y; reject Z before side effects; use supported alternative W, or none exists.`
- `Complexity reset: repeated edge-case combinations show the approach is too broad; redesign from the original requirement instead of adding another branch.`
- `Finding-derived reset spec: findings F1-F3 expose invariant X across entry points A-C; freeze that contract, delete unmapped machinery, and review later findings against it.`
@@ -1,4 +1,4 @@
interface:
display_name: "Implementation Strategy"
short_description: "Choose a compatibility-aware implementation plan"
default_prompt: "Use $implementation-strategy to choose the implementation approach and compatibility boundary before editing runtime code."
default_prompt: "Use $implementation-strategy before initial runtime or API edits and each review-feedback batch to check the full diff, supported contract, convergence, and any finding-derived reset spec before patching."
+250
View File
@@ -0,0 +1,250 @@
---
name: maintainer-review
description: Assess an openai-agents-python GitHub issue or pull request as a maintainer. Use to verify the claimed need and practical impact, compare supported alternatives or competing approaches, separate code quality from repository readiness, recommend the maintainer action, and draft a copy-ready comment when evidence, changes, or closure should be requested.
---
# Maintainer Review
## Objective
Make a maintainer decision, not a generic code-review summary. Separate these questions:
1. Is the claimed behavior real?
2. What user outcome or constraint exists independently of the reporter's proposed API or fix?
3. Can supported functionality already achieve that outcome with reasonable composition or configuration?
4. If a gap remains, is the proposed solution the best design and implementation layer?
5. Can normal users plausibly reach the gap, and what happens when they do?
6. Is it important enough to act on now?
7. If this PR did not already exist, would maintainers choose to open and implement the same work?
8. For a PR, is this solution worth merging and maintaining?
9. Can overlapping or stale operations corrupt shared state or clean up resources owned by surviving work?
10. If competing PRs exist, which single implementation path should maintainers pursue?
11. Which ambiguous scope or semantic choices are maintainer-owned product/API decisions, and what concrete direction should the contributor implement?
12. What concise maintainer message should communicate a closure or change request clearly and politely?
Treat an issue's requested field, callback, flag, class, or implementation strategy as a proposed mechanism, not as the accepted requirement. Do not begin by asking how to implement it. First prove that a concrete user outcome is not already supported and that the proposed mechanism is better than the available alternatives.
Lead with the current review state. Use `Preliminary assessment` while decision-relevant evidence is pending, and `Maintainer decision` only when the review can be concluded. Use the diff, issue narrative, or contributor effort as evidence, not as a proxy for impact.
## Workflow
### 1. Establish the exact target
- Accept a GitHub issue or PR URL as the primary input. Resolve its owner, repository, item type, and number before reviewing it.
- For an issue, read the full report, comments, reproduction, environment, linked material, and maintainer responses.
- For a PR, inspect the current remote base and head, full patch, commit history when relevant, tests, linked issue, and review discussion. Do not substitute the current local checkout for the remote change under review.
- State the claim in one falsifiable sentence. Distinguish the reported symptom from the reporter's proposed cause or fix.
- Identify the released behavior boundary when compatibility or regression claims matter.
- When a proposed change removes, reorders, or reinterprets an established observable or an explicit existing test expectation, inspect the introducing commit, blame, and original tests before any positive assessment. Intentional released coverage is compatibility-risk evidence even when it is not by itself a permanent public contract.
- Verify whether linked evidence matches the PR's exact runtime variant, provider or tool type, triggering condition, and user outcome. A generic issue title, conceptual similarity, or wording such as `Related to` does not transfer evidence of need to an adjacent extension. If the reported scenario has already been fixed, treat additional variants as new needs requiring their own evidence.
Respect repository instructions for remote access and mutation. A review does not authorize comments, labels, branch changes, pushes, or other remote writes.
### 2. Establish the unmet need and challenge the proposed solution
Complete this pass before deeply evaluating a proposed implementation and before any positive issue or PR assessment.
First assign one `Need status`:
- **Demonstrated**: The exact scope has a concrete supported scenario, a real-path reproduction, a released compatibility requirement, repeated demand, or a broad invariant with a meaningful consequence.
- **Plausible but unproven**: The path can exist, but realistic provider behavior, user reach, frequency, consequence, or demand is not established.
- **Already covered**: A reasonable supported workflow already satisfies the outcome.
- **Unsupported**: The outcome belongs outside the SDK contract or at a provider, adapter, or caller-owned layer.
Only `Demonstrated` need may receive `Merge-worthy as-is` or `Merge-worthy after focused changes`. For `Plausible but unproven`, prefer `Needs evidence` or `Not worth completing`; for `Already covered` or `Unsupported`, prefer closure or the relevant simpler alternative.
Keep four decisions separate and record them before comparing implementations:
1. **Observation validity**: whether the reported output, state, or code-path difference is real.
2. **Downstream consequence**: what concrete user, operational, compatibility, or durable-state result changes because of it.
3. **Need status**: one of the four evidence classifications above.
4. **Issue action**: prioritize, accept, narrow, request evidence, or close.
A real observation can still have no demonstrated need and a `Close` action. Do not describe an issue as simply "valid" when only the observation is confirmed. If the downstream consequence is missing, do not choose among proposed semantic contracts, select a competing PR, or draft implementation changes yet.
Before assigning `Demonstrated`, require one of these evidence paths:
1. **Observed impact**: A supported scenario, real-path reproduction, or credible user report shows a meaningful user-visible, operational, compatibility, or durable-state consequence.
2. **Material prevention**: A supported or ordinary failure path can reach the condition, the violated invariant protects against intrinsically material harm, and a complete code-path trace or realistic probe establishes that consequence. A known incident is not required for this path.
For both paths, trace `realistic trigger -> supported execution path -> observable or durable effect`. A local intermediate inconsistency, constructible branch, redundant operation, defensive improvement, or theoretically cleaner invariant is not a demonstrated need without a meaningful downstream effect. A small diff, technically correct patch, or inexpensive test does not lower this threshold. Material preventive outcomes include security or privacy exposure, credential leakage, persistent data or state corruption, duplicate external side effects, unrecoverable compatibility breaks, deadlock or indefinite hangs, and realistically repeatable resource exhaustion.
For a representation-only change, identify one concrete consumer computation, decision, or persisted interpretation that differs before and after the patch, then establish why the current result is wrong. If the patch only changes list shape, placeholder presence, metadata, ordering, or terminology without recovering information or changing a meaningful consumer outcome, the need is not `Demonstrated`.
An ambiguous contract is not itself evidence that the contract should change. When multiple current shapes are released or intentionally test-covered, prefer no code change until a demonstrated outcome justifies selecting a different semantic contract. Do not choose one shape only because it is more symmetric or easier to explain.
When a report establishes only a harmless or speculative logic-level improvement, prefer `Not worth completing` or `Close` rather than requesting implementation refinements. Use `Needs evidence` only when a specific missing reproduction or consequence trace could realistically change the practical-impact decision.
1. Restate the desired user outcome without naming the requested API, class, file, option, or implementation. Separate the actual constraint from the reporter's preferred mechanism.
2. Trace the closest supported ways to achieve that outcome in the current release and current target. Inspect the owning code path, public API, tests, and relevant docs rather than assuming that an unfamiliar capability is missing. Consider configuration, composition, cloning, callbacks, extension points, provider adapters, and doing the work at a caller-owned layer.
3. Determine whether the report shows a capability gap, an ergonomics or discoverability problem, an unsupported use case, or no demonstrated problem. A more convenient spelling is not automatically a missing capability.
4. Compare the proposed solution against the strongest existing approach and at least one better-design candidate: no code change, clearer documentation or validation, a narrower fix, reuse of an existing abstraction, or enforcement at a more coherent shared boundary.
5. For each viable approach, compare whether it satisfies the concrete scenario, what new public or internal contract it creates, cross-path consistency, compatibility, and permanent maintenance cost.
Do not treat a test proving that new code can work as evidence that the feature is needed. A `FakeModel` response, manually constructed provider item, mock, or new regression test can establish code-path reachability and implementation correctness; it does not by itself establish realistic provider behavior, user reach, frequency, practical consequence, demand, or a material preventive outcome.
API symmetry, naming consistency, and parity with an adjacent tool, provider, or output type are design arguments, not evidence of need. Parity may justify work when it removes existing complexity or enforces a broad demonstrated invariant, but adding branches, tests, documentation, or public behavior requires independent practical justification.
Treat an explicit public `Literal`, enum, discriminated union, or equivalent static type restriction as evidence that other values are outside the supported contract. Avoid adding duplicate client-side runtime validation solely to reject values that the public type already excludes. Require evidence that the SDK itself ingests untyped data, that fail-fast behavior before side effects protects a documented contract or material invariant, or that the invalid value causes meaningful impact on a supported path. The host language's ability to bypass type hints, pass adversarial runtime objects, or mutate attributes after construction is not by itself sufficient reason to add permanent validation branches and tests.
When an upstream server or provider already rejects an unsupported request, treat that boundary as the source of truth and avoid duplicating the same acceptance rules in the client. Add fail-fast client validation only when waiting for the server rejection creates a demonstrated, substantial pitfall or material efficiency problem, such as avoidable billable work, repeated network latency or resource consumption, an irreversible side effect or state mutation, or an error that arrives too late or is too opaque for reasonable correction. Prefer the server's evolving validation over copied provider allowlists or constraints that can drift.
#### Synthetic edge-case and extreme-value gate
Do not accept an issue or PR whose need is established only by constructing values that ordinary supported producers cannot emit or that have no realistic origin in supported use. This includes non-finite numbers such as `NaN` or infinity, astronomically large magnitudes, impossible enum or discriminated-union members, manually corrupted typed objects, and direct helper calls that bypass the owning public or wire boundary. A unit test that reaches such a branch proves constructibility, not a problem worth maintaining code for.
Default these reports to `Close` or `Not worth completing`, even when the patch is small and technically correct, unless the evidence establishes at least one of the following:
1. A supported provider, parser, public API workflow, or credible user report produces the exact value under realistic conditions.
2. The released public contract intentionally accepts the value category and ordinary caller code can generate it without first violating that contract.
3. A complete security trace shows that attacker-controlled input can cross an actual trust boundary and cause realistically exploitable resource exhaustion or another concrete security-boundary violation.
Claims such as "this could sleep forever," "this could overflow," or "this might disable a limit" are insufficient without proving the realistic source of the value and the complete supported path to the consequence. Do not treat a security label as an exception by itself: identify the trust boundary, who can control the input, how it reaches the SDK, and the concrete protected outcome. A malformed value from an actually untrusted wire boundary may justify a fix when that trace is complete; a hypothetical hostile provider, monkeypatched object, or manually constructed payload does not by itself do so.
When this gate fails, do not spend review effort refining implementation, tests, or error wording. Recommend closing both the issue and its PR, if one exists, and state the exact real-world evidence that would justify reconsideration only when such evidence is plausible.
If the need is not `Demonstrated`, inspect the patch only far enough to understand its contract, risk, and maintenance cost. Do not turn implementation defects, missing tests, or documentation gaps into a request-changes recommendation, because those questions become merge-blocking only after the need gate passes. If the report provides no concrete scenario, the existing functionality appears sufficient, or the requested mechanism solves only a hypothetical convenience problem, prefer `Needs evidence`, `Close`, `Supersede with a simpler alternative`, or `Not worth completing` over designing the requested feature on the reporter's behalf.
### 3. Discover competing open PRs proportionally
Do this before deeply evaluating a specified PR. A PR URL selects the starting point, not necessarily the entire comparison set.
- Determine the primary issue from explicit closing keywords, linked issues, issue timeline or development links, PR body and comments, and the reproduced symptom. If the association is inferred rather than explicit, state the evidence.
- When an issue is explicitly linked, enumerate all open PRs that address it through the issue timeline, development links, cross-references, closing keywords, and ordinary references. Include draft PRs but label them as drafts.
- When no issue is linked, run a bounded duplicate search using the strongest two or three signals from the title, reproduction, violated invariant, and runtime path. Stop when additional queries are unlikely to produce a credible competing implementation.
- Exclude closed or merged PRs from the active comparison set, while using them as history when relevant.
- Do not group PRs merely because they mention the same subsystem. Require a shared issue, symptom, violated invariant, or materially overlapping fix.
- Record the search methods and candidate set internally. If repository access cannot establish completeness, say so instead of claiming that every open PR was found. Do not list unrelated search hits in the final report.
When multiple candidates exist, compare them on need coverage, runtime correctness, scope, implementation layer, tests, compatibility, complexity, readiness, remaining maintainer work, and whether useful parts can be combined. Prefer the best maintainable solution, not the first submission or the smallest diff by default.
### 4. Use a desk-review evidence flow
Always begin with a desk review. Inspect the concrete runtime path before judging a small change as either trivial or meaningful. Check callers, adjacent helpers, validation layers, fallback paths, and existing tests. Search history or documentation only when it changes the decision. Inspecting test code is part of the desk review; executing tests, imports, examples, reproductions, benchmarks, or service calls is a runtime probe.
This skill does not plan or execute runtime probes. Invoking this skill, asking for a review, or supplying an issue or pull-request URL does not authorize tests, imports, examples, reproductions, benchmarks, service calls, or another runtime-probe skill. If decision-relevant runtime evidence remains after desk review, keep the assessment preliminary and suggest a separate runtime investigation. State the unresolved question, why it could change the decision, the evidence needed, and an appropriate base, release, or known-good control. Do not provide an exact command, request approval, invoke another skill, or execute code from this skill.
For repository-specific runtime invariants, start with `.agents/references/README.md` and open only the references that match the affected boundary. Treat `.agents/references/` as read-only during issue and PR review: use it to identify expected invariants, adjacent surfaces, and regression risks, then verify the current claim against the remote change, current code, tests, docs, release boundary, and focused runtime evidence. Do not edit references as a side effect of the review, infer current issue or PR status from them, or treat old issue or PR outcomes as current evidence. If the review reveals a reusable invariant that should be captured, recommend a separate repository-maintenance update unless the user explicitly asks to update references in the same task.
Use this evidence order:
1. Trace the closest existing supported capabilities and determine whether they already satisfy the underlying user outcome.
2. Inspect existing tests and complete the code-path trace, including the mandatory interleaving and ownership pass when triggered, without executing code.
3. Compare the implementation and existing evidence with the released version, base branch, or known-good control without executing code.
4. If a decision-relevant runtime uncertainty remains, stop and suggest a separate runtime investigation using the evidence requirements below.
#### Desk review
Produce the result from static evidence:
##### Mandatory unmet-need and design pass
Before a positive assessment, complete the pass in step 2 and be able to state all of the following from concrete evidence:
1. The user outcome that current supported behavior cannot achieve.
2. The closest existing API or composition path and the exact reason it is insufficient.
3. Why the proposed behavior belongs at the chosen abstraction layer instead of a caller, adapter, validation, documentation, or existing extension point.
4. Why the proposed permanent contract is better than no code change and the strongest narrower alternative.
5. What real scenario, compatibility requirement, or repeated demand justifies the new maintenance surface.
6. Whether maintainers would choose to pursue the same work if no contributor had already supplied a patch.
If any answer is missing and could change whether code should exist at all, do not call the issue actionable or the PR merge-worthy. Request only the evidence needed to distinguish a genuine capability gap from a usage, discoverability, or solution-design problem. This is a product and architecture evidence gap, not a runtime-probe trigger by itself.
##### Mandatory interleaving and ownership pass
Run this pass before any positive PR assessment when a patch adds, removes, or reorders cleanup, retry, reconnect, cancellation, listeners, shared futures or tasks, connections or streams, state flags, or mutable state across an `await`, callback, event, or deferred completion.
1. Name each shared resource or state value and enumerate every path that can mutate it, including distinct public methods and wrapper or delegate paths. Include listeners, futures, tasks, connections, streams, locks, caches, state flags, persistence, and telemetry.
2. Trace at least two overlapping operations, `A` and `B`, across every suspension or re-entry point. Choose `B` from the strongest distinct mutator, not only a second invocation of `A`. Check `A pending -> B starts -> A fails -> B succeeds`, `A pending -> B starts -> B fails -> A succeeds`, close or cancellation between setup and completion, and a stale completion arriving after newer work.
3. For snapshot-based cleanup or rollback, always trace `A snapshots -> A destructively mutates -> B commits newer state -> A rollback resumes`. Require the pre-`A` state plus `B`'s committed mutation to survive in the correct order, with persisted state, caches, indexes, flags, and related ownership state agreeing.
4. For every cleanup or rollback, identify the exact attempt and resource generation it is allowed to dispose. Require an ownership token, generation, identity check, compare-and-swap, transaction, proven serialization, or an equivalent invariant at the actual mutation boundary. Do not infer exclusivity from intended usage; treat overlap as unsupported only when an explicit contract or fail-fast validation enforces that restriction.
5. Compare base and head for the survivor invariant. Replacing duplicated work with missing handlers, a closed shared resource, reverted state, or a failed surviving task is a regression, not successful cleanup. Do not dismiss stale cleanup as pre-existing when the patch newly invokes it for another failure, cancellation, or retry path.
6. Inspect tests for controlled interleavings using deferred futures, callbacks, or events. Require assertions about the failing and surviving operations' observable behavior and final resource coherence, not only listener counts or individual exception results.
Do not mark a concurrency-sensitive patch `Merge-worthy as-is` merely because sequential reconnect, retry, failure, and close tests pass. A triggered ownership pass is incomplete unless the evidence records the complete mutation surface, concrete ownership mechanism, strongest distinct-mutator interleaving, and survivor and coherence result. If the code trace proves an unsafe interleaving, conclude from static evidence and request a focused fix and regression test. If ownership remains ambiguous, keep the result preliminary and state the exact runtime evidence needed to resolve it.
- If the claim or PR is decisively negative from a complete reachable code-path trace, conclude the review without a runtime probe. Examples include an impossible or unsupported path, duplicated existing handling, a demonstrated no-op, a direct compatibility break, or a clearly wrong abstraction. Do not call an ambiguous result negative merely to avoid a probe.
- If the initial result is positive and there is no unresolved runtime concern, and any triggered interleaving and ownership pass is complete, the desk review may be sufficient for a final maintainer decision. Do not suggest additional runtime investigation only to restate evidence that cannot plausibly change the decision.
- If there is any unresolved runtime concern that could plausibly change claim validity, severity, merge-worthiness, required changes, or the preferred competing PR, report a `Preliminary assessment`. State the unresolved question, why it could change the decision, the evidence needed, and an appropriate control, then suggest a separate runtime investigation without planning or executing it.
- A purely stylistic, documentation, CI-status, or repository-readiness concern does not justify suggesting a runtime investigation unless it masks a runtime question.
Do not issue a definitive positive maintainer decision while a decision-relevant runtime concern remains unresolved. If the needed runtime evidence is unavailable or remains untested, keep the result preliminary and state the exact confidence limitation.
For changes involving validation, fail-fast behavior, cleanup, retries, interruption, or concurrency, trace lifecycle ordering in addition to the main behavior:
- Identify listeners, tasks, connections, files, locks, state mutations, and other resources acquired before the new check or failure point.
- Verify cleanup when construction, context-manager entry, validation, connection, or execution raises before normal teardown runs.
- Require a negative-path test when a failure can leave observable state or resources behind.
Do not over-investigate. Stop when additional evidence is unlikely to change validity, severity, or the maintainer recommendation.
### 5. Calibrate validity and impact
Use `references/evaluation-framework.md` to assess claim validity, realistic reach, consequence, breadth, frequency, recoverability, compatibility, and severity. Keep observed facts separate from inference and state any missing evidence that could change the decision.
Report the `Need status` before classifying the need as a capability gap, ergonomics or discoverability gap, unsupported use case, or no demonstrated gap. Do not assign practical impact to the absence of the requested mechanism when an existing supported workflow already produces the requested outcome. Do not infer practical importance merely from reachability, API asymmetry, or a technically successful patch.
For a PR, make `Severity` describe the underlying issue or user need only. Do not combine it with the risk created by the proposed patch. Report a meaningful patch-induced regression, compatibility, lifecycle, or maintenance risk separately as `Patch risk`.
Do not infer that a report is low-value merely because an AI may have found or written it. Do not speculate about authorship or motive. Identify contribution-shaped reports through objective signals: no reproducible behavior, unrealistic inputs, an impossible call path, duplicated existing handling, tests that do not exercise the claim, or a fix whose runtime result is a no-op.
### 6. Apply the maintainer-effort test
Use the framework's issue dispositions and PR checks to decide whether the outcome justifies permanent code, tests, documentation, and maintainer attention. Classify code quality separately from repository readiness.
Use one code recommendation:
- **Merge-worthy as-is**: real need, sound implementation, proportionate scope, adequate tests.
- **Merge-worthy after focused changes**: real need and viable direction, with bounded corrections.
- **Supersede with a simpler alternative**: real need, but a smaller or more coherent fix is preferable.
- **Not worth completing**: negligible or unsupported impact, no-op behavior, wrong abstraction, or excessive completion cost.
`Merge-worthy as-is` and `Merge-worthy after focused changes` are invalid unless `Need status` is `Demonstrated`. A bounded set of implementation fixes cannot promote a `Plausible but unproven` need into a merge-worthy recommendation.
For `Merge-worthy as-is` and `Merge-worthy after focused changes`, use one repository-readiness status when it helps communicate the integration state:
- **Ready**: current head is reviewable and required checks are green.
- **CI or review pending**: code recommendation is stable, but required external gates are incomplete.
- **Rebase or conflict resolution required**: the head cannot merge cleanly or is materially stale.
- **Blocked**: a concrete external or repository condition prevents a reliable merge decision.
Omit repository readiness for `Supersede with a simpler alternative` and `Not worth completing`; CI, review, mergeability, or branch freshness does not change those dispositions. Put any validation limitation that materially affects confidence in the evidence instead. When readiness is included, use exactly one of the four statuses above and do not invent variants such as `ready mechanically` or use rebase status for semantic staleness.
Do not downgrade an otherwise sound code recommendation solely because CI is pending. Do not call a PR ready when semantic conflict resolution or material code changes remain.
When multiple open PRs address the same issue, make one portfolio-level recommendation: select the strongest PR, request focused changes in one candidate, combine specific ideas into one PR, supersede all candidates with a simpler approach, or close duplicates. Explain why the recommended path is better than each alternative without turning the report into line-by-line review.
Always compare the proposed patch with the strongest existing supported approach and at least one alternative: no code change, validation or documentation, a narrower fix, reuse of an existing helper, or a different layer that enforces the invariant consistently. A review is incomplete if it establishes only that the patch works without establishing why the current product cannot meet the underlying need and why this design is preferable.
When multiple plausible semantic scopes, compatibility boundaries, or public API contracts remain, do not ask the contributor to choose among maintainer-owned options. Decide the preferred scope from the evidence, compatibility contract, and product/API design principles, then request that specific change. If the evidence is insufficient to choose, mark the review preliminary or request maintainer input; do not present an open-ended implementation fork as the contributor's decision.
### 7. Report findings and maintainer action
Choose the assessment language using this precedence:
1. Follow an explicit language request in the current conversation.
2. Follow an applicable language instruction from `~/.codex/AGENTS.md`, the repository's `AGENTS.md`, or another governing instruction file.
3. If recent conversation turns are consistently in one language, use that language.
4. Otherwise, default to English.
Do not infer the assessment language from the GitHub URL, contributor, code, or browser locale. Maintainer comment drafts remain English regardless of the assessment language. Keep the report decision-oriented and compact. Use no more than five evidence bullets by default; add more only when the decision genuinely depends on them.
Use the matching compact report variant in `references/evaluation-framework.md`. While decision-relevant evidence is pending, use its preliminary-assessment variant and end with the evidence limitation and optional suggestion for a separate runtime investigation instead of presenting a final recommendation. Collapse sections for simple cases rather than padding the answer. Put unexpected or negative runtime findings first, and name the preferred PR or approach explicitly when candidates compete.
For PRs, put `Need status` before code recommendation. When the need is not `Demonstrated`, lead with that result, omit repository readiness, and avoid presenting patch fixes as the primary maintainer action.
When existing functionality or a better alternative materially affects the decision, state it explicitly in the evidence and recommendation. Name the exact supported path, what it does and does not cover, and why it is preferable. Do not bury a `Not worth completing` or `Supersede with a simpler alternative` conclusion beneath praise for implementation quality.
When recommending closure, requesting more evidence, requesting code changes, or superseding a PR, append the English, copy-paste-ready maintainer comment defined by the framework. If multiple PRs need different actions, label one draft for each affected PR. Include only merge-blocking requests in the main action paragraph; keep optional documentation or polish clearly non-blocking or omit it.
Before returning any maintainer comment draft, perform a GitHub paste-readiness pass using the repository-wide rule in `AGENTS.md` and the detailed guidance in `references/evaluation-framework.md`. In the draft, use `#123` for same-repository issues or PRs and `owner/repo#123` for cross-repository references. Remove Markdown-linked issue or PR labels, Codex navigation links, local file links, Codex-only citation markers or footnotes, and app directives from the copy-ready draft. Preserve ordinary descriptive links to API docs, design notes, and other targets without native GitHub issue or pull-request syntax.
Also perform an action-delta pass. Every imperative sentence must correspond to a concrete difference between the current remote head and the desired state. Remove requests to "keep", "preserve", document, test, or change behavior that the current head already satisfies or that is not merge-blocking. A `Merge-worthy as-is` result must not contain change-request language. Keep portfolio comparisons out of a contributor-facing draft unless duplicate or supersession handling is the action for that target.
For request-changes comments, phrase maintainer-owned semantic decisions as a directive, not as a menu. It is fine to mention the rejected alternative briefly in the rationale, but the requested action must identify the chosen behavior, scope, or compatibility boundary. Use "please do X because..." instead of "either do X or Y" when X versus Y changes the SDK contract or user-visible semantics.
Do not produce a line-by-line review unless requested. Do not equate passing tests with merge-worthiness, or a logically correct patch with practical value.
## Resource
- `references/evaluation-framework.md` contains the severity rubric, evidence checks, lifecycle review, issue dispositions, PR quality checks, maintainer-comment guidance, and report variants.
@@ -0,0 +1,4 @@
interface:
display_name: "Maintainer Review"
short_description: "Gate PR value on demonstrated user need"
default_prompt: "Use $maintainer-review to verify the demonstrated need and practical impact for this issue or PR, keep the assessment desk-review-only, recommend the maintainer action, and provide a copy-ready English comment when changes, evidence, or closure are needed."
@@ -0,0 +1,412 @@
# Maintainer Evaluation Framework
Use this reference when a claim is ambiguous, severity is disputed, or a PR is technically correct but may not justify merge effort.
## Contents
- [Decision model](#decision-model)
- [Severity rubric](#severity-rubric)
- [Evidence-strength checks](#evidence-strength-checks)
- [Unmet need and alternative design gate](#unmet-need-and-alternative-design-gate)
- [Issue disposition](#issue-disposition)
- [PR quality and value](#pr-quality-and-value)
- [Documentation threshold](#documentation-threshold)
- [Lifecycle and failure-path review](#lifecycle-and-failure-path-review)
- [Concurrency and cleanup ownership](#concurrency-and-cleanup-ownership)
- [Better-alternative prompts](#better-alternative-prompts)
- [Competing PR comparison](#competing-pr-comparison)
- [Maintainer comment drafts](#maintainer-comment-drafts)
- [Compact report variants](#compact-report-variants)
## Decision model
Treat validity, severity, and merge-worthiness as separate results. Also distinguish a `Preliminary assessment`, which may still require decision-relevant evidence, from a final `Maintainer decision`. Do not label a provisional positive result as a verdict or final decision.
| Dimension | Questions | Strong evidence |
|---|---|---|
| Claim validity | Does the exact reported behavior occur? Is the proposed cause correct? | Reproduction, failing focused test, or complete reachable code path |
| Reachability | Can supported, realistic inputs reach it? | Public API trace, real configuration, linked user report, or release comparison |
| Consequence | What fails, and is the result silent or recoverable? | Observed output/error/state plus downstream effect |
| Breadth | Who is affected? | Supported providers, platforms, versions, and configurations identified precisely |
| Frequency | Is this normal, intermittent, or pathological? | Repeat runs, telemetry or reports when available, deterministic preconditions |
| Need status | Is the exact scope demonstrated, merely plausible, already covered, or unsupported? | Observed impact or a complete realistic trigger-to-material-consequence trace for prevention |
| Unmet need | What user outcome cannot be achieved through supported behavior today? | Concrete scenario plus a trace showing why the closest existing path is insufficient |
| Existing capability | Can configuration, composition, cloning, callbacks, extension points, or a caller-owned layer already satisfy the outcome? | Current release code, tests, docs, and an exact supported workflow |
| Compatibility | Is released behavior or durable state changed? | Latest release comparison and explicit contract inspection |
| Solution fit | Is the requested mechanism the best design and implementation layer? | Proposed solution compared with the strongest existing path and at least one narrower or more coherent alternative |
| Maintainer-owned scope | When several plausible semantics remain, which behavior should the SDK own? | A concrete maintainer decision grounded in compatibility, user outcome, and API design, not an open-ended contributor choice |
| Resource ownership | Can stale, failed, cancelled, or overlapping work mutate or clean up resources owned by surviving work? | Interleaving trace, attempt or generation ownership, and survivor assertions |
| Maintenance cost | What permanent complexity and review burden does it add? | Changed surface, new branches/configuration, test burden, remaining work |
## Severity rubric
- **Negligible**: No runtime difference, unreachable or unsupported input, cosmetic inconsistency, or a fully harmless edge case. Usually close, document, or decline code complexity.
- **Low**: Real but narrow and recoverable behavior with a simple workaround and no data, security, or compatibility risk. Merge only when the fix is small and clearly improves an invariant.
- **Moderate**: Plausible supported use fails or produces incorrect behavior for a meaningful subset of users. Prioritize a bounded fix and regression test.
- **High**: Common or important supported use is broken, causes serious compatibility problems, leaks sensitive data, or risks persistent corruption. Treat as urgent and require strong validation.
- **Critical**: Broadly exploitable security impact, severe data loss, or systemic failure requiring immediate coordinated action. Use only with concrete evidence.
Severity is approximately consequence multiplied by realistic reach and frequency, reduced by recoverability. Do not raise severity because a report sounds alarming or lower it because a patch is small.
## Evidence-strength checks
Before calling a claim confirmed, answer:
- Does the reproduction exercise the same public or internal path named in the report?
- Does the failure still occur on the relevant base, release, or current target?
- Does the test fail without the patch and pass with it?
- Are setup failures, stale builds, environment leakage, proxies, caches, or unsupported options excluded?
- Does an adjacent helper or equivalent path follow different semantics?
- Is the observed behavior prohibited by an actual contract, or merely surprising?
- If the patch removes or reinterprets an established observable or explicit test expectation, what did the introducing commit and original tests intend? Treat that history as compatibility-risk evidence, not automatic proof that the behavior must never change.
- For latency, timeout, buffering, backpressure, or cleanup claims, was observable elapsed time or a real state transition measured when feasible rather than inferred only from mocks?
- For shared asynchronous state, do tests control completion order and prove that stale failure or cleanup cannot affect the surviving operation?
Use `partially confirmed` when the symptom is real but the cause, reach, or claimed scope is wrong. Use `unproven` when decisive evidence is missing. Use `contradicted` only when evidence directly disproves the claim.
## Unmet need and alternative design gate
Issue reports often combine a desired outcome with a proposed API or implementation. Treat the proposed mechanism as a hypothesis. Confirm the unmet outcome before evaluating how well the patch implements that mechanism.
### Linked-evidence scope
Evidence from a linked issue applies only when the issue and PR share the same runtime variant, provider or tool type, trigger, supported configuration, and user outcome. A broad title, ordinary reference, `Related to` statement, or conceptual similarity is not enough. If an earlier change already resolved the concrete reported scenario, an adjacent extension starts with no inherited evidence of need.
### Need status
Assign one status before deep implementation review:
- **Demonstrated**: The exact scope has a concrete supported scenario, real-path reproduction, released compatibility requirement, repeated demand, or broad invariant with meaningful consequence.
- **Plausible but unproven**: The code path is possible, but realistic reach, frequency, consequence, provider behavior, or demand is missing.
- **Already covered**: A reasonable supported workflow already satisfies the outcome.
- **Unsupported**: The outcome is outside the SDK contract or belongs at a provider, adapter, or caller-owned layer.
Only `Demonstrated` need can support a merge-worthy code recommendation. `Plausible but unproven` maps to `Needs evidence` or `Not worth completing`, even when the patch is technically correct and its remaining fixes are bounded. `Already covered` and `Unsupported` normally map to closure or a simpler non-core alternative.
The need status is an evidence classification, not the issue action. Record observation validity, downstream consequence, need status, and issue action separately. A reported shape or branch difference can be confirmed while the need remains `Plausible but unproven` and the correct issue action is `Close`.
### Practical-impact gate
Do not accept a change merely because desk review identifies a local logical flaw, defensive improvement, or constructible edge case. Trace the complete consequence chain:
`realistic trigger -> supported execution path -> observable or durable effect`
A local intermediate inconsistency, redundant operation, surprising branch, or theoretically cleaner invariant is not a demonstrated need when it has no meaningful downstream effect. Reachability, a passing new test, a small diff, and low implementation cost establish neither practical impact nor maintenance value.
For representation-only changes, name the concrete consumer computation, decision, or persisted interpretation that changes before and after the patch. If the patch only changes list shape, placeholder presence, metadata, ordering, or terminology without recovering information or changing a meaningful result, it does not establish practical impact. Contract ambiguity and API symmetry are insufficient by themselves, especially when the current shapes are released or intentionally test-covered.
Use one of these evidence paths:
| Evidence path | Required proof | Insufficient proof |
|---|---|---|
| **Observed impact** | A supported scenario, real-path reproduction, or credible user report shows a meaningful user-visible, operational, compatibility, or durable-state consequence. | An internal state difference without a downstream effect, a synthetic branch, or a test that only proves the patch executes. |
| **Material prevention** | A supported or ordinary failure path reaches the condition; the violated invariant protects against intrinsically material harm; and a complete code-path trace or realistic probe establishes the concrete consequence and how the patch prevents it. | A statement that the condition "could" cause harm, an unsupported or malformed input, a mock-only scenario, or severity language without a complete consequence chain. |
A known incident, user report, frequency estimate, or production reproduction is not required for a material-prevention case. Require credible reachability and a concrete consequence such as security or privacy exposure, credential leakage, persistent data or state corruption, duplicate external side effects, an unrecoverable compatibility break, deadlock or indefinite hangs, or realistically repeatable resource exhaustion. Do not wait for those outcomes to occur before accepting a proportionate preventive fix.
When the trace ends in a harmless intermediate state, fully recoverable behavior without meaningful operational cost, theoretical cleanliness, or an unsupported scenario, classify the need as `Plausible but unproven`, `Unsupported`, or `No demonstrated gap` as appropriate. Prefer `Not worth completing` or `Close` over requesting implementation refinements. Use `Needs evidence` only when one specific missing reproduction or consequence trace could realistically change the practical-impact decision.
Before accepting an issue or recommending a PR, record:
| Question | Required evidence |
|---|---|
| What outcome is needed? | A concrete supported scenario stated without the proposed API or fix |
| What exists today? | The closest current-release API, configuration, composition, extension point, or caller-owned solution |
| Why is it insufficient? | An exact behavioral, compatibility, lifecycle, or operational constraint, not preference alone |
| What are the alternatives? | The proposed patch, the strongest existing path, and at least one no-code, narrower, or better-layer design |
| Why add a contract? | Practical benefit sufficient to justify public surface, runtime branches, cross-path tests, documentation, and long-term maintenance |
Classify the result:
- **Capability gap**: a supported, realistic outcome cannot be achieved with current functionality. Code may be warranted.
- **Ergonomics or discoverability gap**: the outcome is already possible, but the supported route is confusing or unnecessarily difficult. Prefer documentation, validation, or a narrowly justified convenience improvement.
- **Unsupported use case**: the desired outcome lies outside the SDK contract or belongs at a provider, adapter, application, or other caller-owned layer. Do not expand the core API merely to make it possible.
- **No demonstrated gap**: no concrete scenario proves that existing functionality is insufficient. Request evidence or close rather than designing from the proposed mechanism.
Passing tests for a new implementation establish feasibility and correctness, not need. A `FakeModel` response, manually constructed provider item, mock, or synthetic fixture does not establish realistic provider behavior, user reach, frequency, consequence, or demand. API symmetry and parity with an adjacent runtime are design arguments, not need evidence. A technically coherent patch can still be `Not worth completing` when the motivating scenario is hypothetical, already supported, or better solved elsewhere.
Use the counterfactual maintainer test: if the PR did not already exist, would maintainers choose to file and implement the same work from the available evidence? Contributor effort lowers implementation cost but does not create product need or remove permanent maintenance cost.
When the need is not `Demonstrated`, inspect implementation only far enough to estimate contract, risk, and maintenance cost. Do not convert patch defects, missing tests, or documentation gaps into a request-changes disposition; those become merge blockers only after the need gate passes.
## Issue disposition
Choose one primary action:
- **Prioritize**: confirmed moderate-or-higher impact or an important invariant with no safe workaround.
- **Accept, low priority**: confirmed low impact, existing supported functionality is insufficient for the demonstrated scenario, and a proportionate fix appears possible.
- **Narrow scope**: a valid core exists, but the report overstates affected paths or expected behavior.
- **Needs evidence**: plausible claim, but no minimal reproduction, supported setup, contract basis, or concrete scenario showing why existing functionality is insufficient.
- **Close**: duplicate, unsupported, unreachable, contradicted, no-op, already addressed by a reasonable supported path, or not worth permanent complexity.
When requesting evidence, ask only for information that could change the disposition.
For external contribution triage, default a `Plausible but unproven` need to `Close` when the current report shows only a logic-level or representation-level inconsistency. Use the `Needs evidence` issue action only when maintainers intentionally want to keep the issue open and can name one bounded piece of evidence likely to change the decision. A closed issue may still state the concrete evidence that would justify reconsideration.
## PR quality and value
Assess these independently:
1. **Need**: Same-scope evidence demonstrates either observed practical impact or a material preventive outcome that the closest supported capability cannot reasonably address. Do not inherit evidence from an adjacent variant or already-fixed scenario.
2. **Correctness**: The fix works for the reported case and meaningful boundaries.
3. **Placement**: The invariant is enforced once at the right layer instead of duplicating existing functionality, patching locally, or moving caller- or provider-owned policy into the core SDK.
4. **Consistency**: Equivalent sync/async, streaming/non-streaming, provider, serialization, and resume paths remain aligned where applicable.
5. **Tests**: A regression test fails on the base, passes on the head, and tests the exact non-happy-path value or state. When shared state crosses an asynchronous boundary, tests control relevant completion orders and assert the surviving operation's behavior and final resource state.
6. **Compatibility**: Released positional APIs, wire formats, persisted schemas, and established error behavior are preserved or intentionally migrated.
7. **Proportionality**: Complexity and public surface are justified by impact.
8. **Completion cost**: Remaining fixes, docs, tests, and design work are bounded enough to justify maintainer attention.
A PR can be correct but not merge-worthy. Typical reasons include a nonexistent or negligible need, an outcome already supported through a reasonable existing mechanism, a no-op on the actual runtime path, incomplete cross-path semantics, an abstraction cost larger than the benefit, or a simpler design at another layer.
Do not use implementation correctness, bounded remaining work, CI status, or contributor effort to upgrade a need that is only `Plausible but unproven`. Merge-worthiness is gated by demonstrated need, not by how close the patch is to completion.
Keep issue impact and patch risk separate. `Severity` describes the underlying issue or user need. A regression, compatibility break, lifecycle leak, or maintenance hazard introduced by the proposed patch belongs under `Patch risk` and must not inflate or obscure the issue severity.
When a PR exposes an ambiguous semantic boundary, decide whether that boundary belongs to maintainers before drafting requests. If the choice affects SDK contract, compatibility, persistence, error semantics, public API meaning, or cross-path behavior, the review should pick one direction or explicitly block on maintainer input. Do not delegate that choice to the contributor as "either X or Y"; ask for the chosen behavior and the tests or docs needed to lock it down.
## Documentation threshold
Do not treat documentation as automatically required for every public option, constructor parameter, provider setting, or behavior change. Make docs merge-blocking only when at least one of these is true:
- Existing user-facing docs become materially false, unsafe, or misleading.
- Correct or safe use depends on a non-obvious constraint, migration step, compatibility boundary, or operational warning.
- Repository policy, the accepted issue scope, or an explicit maintainer decision requires documentation in the same PR.
- The intended feature would be practically unusable or undiscoverable by its target users without a documented entry point, and generated API reference or clear code-level discovery is insufficient.
If docs would merely improve discoverability or completeness, keep them non-blocking. Do not change `Merge-worthy as-is` to `Merge-worthy after focused changes` solely for optional docs, and do not include optional docs in the maintainer comment's required-action paragraph. Respect an explicit maintainer choice to omit docs or defer them to a separate follow-up.
## Lifecycle and failure-path review
Apply this section when a change adds validation, fail-fast behavior, cleanup, retries, interruption, background work, or concurrency.
- Identify the earliest point where all dynamic inputs needed for a correct decision are available.
- List side effects before and after that point: listeners, tasks, connections, files, locks, caches, state mutations, and telemetry.
- Exercise failure during construction, context-manager entry, validation, connection, and execution when those phases exist.
- Confirm that normal teardown is actually entered. If an enter or constructor fails, verify cleanup explicitly rather than assuming an exit hook runs.
- Prefer validation after dynamic configuration is resolved but before avoidable side effects begin.
- Require a regression test for any listener, task, connection, or state that could remain after failure.
## Concurrency and cleanup ownership
Apply this section before a positive assessment whenever lifecycle work crosses an `await`, callback, event, deferred completion, retry, reconnect, cancellation, or shared resource boundary. Sequential correctness is insufficient because a patch can improve isolated cleanup while introducing cross-attempt teardown.
Use a two-operation interleaving matrix during desk review:
| Ordering | Required question |
|---|---|
| `A pending -> B starts -> A fails -> B succeeds` | Can A's cleanup remove or revert anything B needs? |
| `A pending -> B starts -> B fails -> A succeeds` | Can B's cleanup leave A successful but non-functional? |
| `A succeeds -> B starts -> stale A completion` | Can stale A overwrite B's newer state or generation? |
| `A snapshots -> A mutates -> B commits -> A rolls back` | Does A restore only its own state while preserving B's commit, ordering, and derived state? |
| setup -> close/cancel -> late completion | Can late work resurrect listeners, state, tasks, or connections after teardown? |
For each ordering:
- Enumerate every path that can mutate the resource, including distinct public operations and wrapper or delegate paths. Choose `B` from the strongest distinct mutator rather than assuming a second invocation of `A` is sufficient.
- Identify the resource owner before and after every suspension point.
- Distinguish per-attempt resources from shared runner, session, transport, cache, or listener state.
- Require cleanup to carry an ownership token, generation, identity check, compare-and-swap, transaction, proven serialization guarantee, or another invariant at the actual mutation boundary that prevents cross-attempt disposal.
- Compare base and head on the survivor invariant. Fewer duplicates do not justify losing the only active handler, connection, task, or state update. Preserving or restoring `A` does not justify deleting, reverting, reordering, or hiding `B`.
- Require a controlled interleaving test when the ordering is reachable. The test must assert both the failing operation and the surviving operation's observable behavior after all completions settle, including agreement between persisted state and caches, indexes, flags, or other derived state.
An unscoped `finally`, `except`, close handler, cancellation callback, or rollback that mutates shared state after a suspension point is merge-blocking when another operation can still own or use that state.
Do not infer exclusive access from intended usage. Treat overlapping operations as unsupported only when documentation, public typing, construction-time validation, or fail-fast runtime enforcement establishes that restriction. Do not dismiss stale cleanup as pre-existing when a patch newly makes that cleanup reachable from another failure, cancellation, retry, or interruption path.
## Better-alternative prompts
Start with the strongest existing supported path, then test at least one additional alternative against the proposed patch. Do not complete a positive review without this comparison.
- Can the requested outcome already be achieved through configuration, composition, cloning, callbacks, extension points, a custom provider or adapter, or caller-owned code?
- If the existing route is awkward, is the problem discoverability or ergonomics rather than missing capability?
- What happens if maintainers make no code change?
- Can input validation or an existing helper enforce the invariant earlier?
- Can the fix be limited to the one supported path that fails?
- Would documentation or a clearer error prevent misuse without runtime complexity?
- Can the test be added first to reveal the smallest correct change?
- Is the proposed public option compensating for an internal design issue?
- Is the proposed core behavior actually provider- or application-specific policy that belongs at another layer?
## Competing PR comparison
When two or more open PRs address the same issue, first verify that they belong in one comparison set. Accept an explicit issue link, the same minimal reproduction, the same violated invariant, or materially overlapping runtime paths as association evidence. Do not treat a shared label or subsystem as sufficient.
Compare each candidate on the same evidence basis:
| Criterion | Question |
|---|---|
| Need | Does a concrete user outcome remain unmet after tracing existing supported functionality? |
| Existing capability | Could every candidate be avoided by configuration, composition, an extension point, or a better caller- or provider-owned solution? |
| Coverage | Does it solve the whole confirmed issue, a useful subset, or an adjacent problem? |
| Correctness | Does the fix work on the real path and meaningful boundaries? |
| Placement | Does it enforce the invariant at the correct shared layer? |
| Tests | Does it reproduce the base failure and distinguish the candidate approaches? |
| Compatibility | Does it preserve released APIs, state, protocol, providers, and established behavior? |
| Complexity | What permanent branches, abstractions, configuration, or coupling does it add? |
| Readiness | Is it mergeable now, or how much focused work remains? |
| Reuse | Are there valuable tests or implementation pieces that should be combined into another candidate? |
Choose one portfolio-level disposition:
- **Prefer one PR**: identify the strongest candidate and close or supersede duplicates.
- **Prefer one after focused changes**: keep one candidate active and state bounded changes required before merge.
- **Combine selectively**: identify the destination PR and the exact ideas or tests worth transferring; avoid asking maintainers to reconcile entire competing implementations.
- **Replace all**: explain the simpler or more coherent implementation that should supersede every candidate.
- **Merge none**: the issue is invalid, negligible, unsupported, or none of the approaches justify completion cost.
Do not split the decision into independent approvals. Competing PRs consume overlapping review and maintenance budgets, so recommend one path for the issue as a whole.
## Maintainer comment drafts
Always write maintainer comments in English, regardless of the assessment language. Produce a draft when the recommendation is to close, request evidence, request focused code changes, supersede a PR, or choose one competing PR over another.
Keep each draft polite, direct, and copy-paste-ready. Usually use 60-160 words in one to three short paragraphs:
1. Acknowledge the contribution or report.
2. Explain the decision with the smallest amount of decisive technical evidence.
3. Give the exact next action or the condition for reconsideration.
Use GitHub-native references in every draft:
- Use `#123` for an issue or pull request in `openai/openai-agents-python`.
- Use `owner/repo#123` for an issue or pull request in another repository.
- Keep closing keywords native, for example `Fixes #123` or `Resolves #123`.
- Never wrap a native reference in a Markdown link. Write `#123`, not `[PR #123](https://github.com/openai/openai-agents-python/pull/123)` or `[#123](...)`.
- Remove Codex-only navigation links, local file links, Codex-only citation markers or footnotes, and app directives from the draft. Preserve ordinary descriptive Markdown links for API docs, design notes, external resources, and GitHub targets that do not have native issue or pull-request syntax.
Before returning the draft, normalize any same-repository URL or qualified reference to `#<number>`, normalize any cross-repository issue or pull-request URL to `owner/repo#<number>`, and rescan the draft. Do not return it while a Markdown-linked issue or pull-request label, `openai/openai-agents-python#<number>`, or bare GitHub issue or pull-request URL remains.
Perform an action-delta pass after the paste-readiness pass:
- Map every imperative sentence to a concrete difference between the current remote head and the desired state.
- Remove requests for behavior, tests, documentation, or scope that the current head already satisfies or that the recommendation does not require.
- Do not use change-request language for `Merge-worthy as-is`.
- Keep portfolio comparisons out of a contributor-facing draft unless that target is being closed or redirected as a duplicate or superseded implementation.
Do not include internal labels such as `severity: low`, speculate about AI authorship or contributor intent, repeat the full review, or soften the message until the requested action becomes unclear.
Do not ask contributors to choose maintainer-owned semantics. If two implementations are technically possible but one changes the SDK contract, decide the contract in the review and make the comment actionable. Use a short rationale such as "This keeps the new handler scoped to the existing raise site" or "This makes the handler name match all invalid final messages", then request the exact code and tests for that decision.
### Close
```text
Thanks for taking the time to investigate this. I traced the reported case through <path or behavior>, and <decisive finding>. In the supported path, <practical result>, so the added complexity is not justified by the demonstrated impact.
I am going to close this <issue/PR>. If you can provide <specific reproduction or evidence that would change the decision>, we can revisit the underlying problem with that narrower scope.
```
### Request changes
```text
Thanks for the contribution. The underlying issue is valid, and this approach is directionally reasonable. Before we can merge it, please address the following points: <bounded list of required changes>.
These changes are needed because <concise contract, lifecycle, compatibility, or test reason>. Once they are covered with a regression test that fails on the base and passes on the updated branch, the PR should be ready for another review.
```
Adapt the wording to the actual evidence. Do not use these templates as generic filler.
### Existing capability or better alternative
```text
Thanks for the contribution. I traced the underlying use case through <existing API or workflow>, which already supports <desired outcome and relevant limits>. The proposed change adds <new contract or complexity>, but the issue does not demonstrate a concrete supported case that the existing approach cannot handle.
I am going to close this <issue/PR> for now. If you can provide <specific scenario showing the existing approach is insufficient>, we can revisit the unmet need and choose the narrowest appropriate design from that evidence.
```
## Compact report variants
Use `Maintainer decision` for a concluded review. Use `Preliminary assessment` when a desk review is tentatively positive but a decision-relevant evidence gap remains. `Verdict` is intentionally avoided in the report headings because it does not communicate whether the result is provisional or final.
### Additional runtime investigation suggested
```markdown
## Preliminary assessment
<Tentative issue or PR assessment based on desk review only.>
## Static evidence
- <decisive code-path or test-inspection evidence>
- <what remains uncertain at runtime>
## Additional runtime investigation suggested
- Unresolved question: <the runtime uncertainty that could change the decision>
- Decision impact: <which validity, severity, scope, or recommendation could change>
- Evidence needed: <the observable behavior or state needed to resolve the uncertainty>
- Control: <base, release, or known-good comparison when relevant>
<Suggest a separate runtime investigation without providing an exact command, requesting approval, executing code, or invoking another skill. Do not present a final positive recommendation yet.>
```
### Issue
```markdown
## Maintainer decision
<Real/partial/unproven/contradicted, severity, and disposition.>
## Evidence
- <decisive evidence>
- <scope or uncertainty>
## Existing capability and alternatives
<Closest supported path, why it is or is not sufficient, and the preferred design alternative.>
## Recommendation
<Prioritize, accept low priority, narrow, request evidence, or close.>
## Maintainer comment draft
<Include when closure or additional evidence should be requested.>
```
### Pull request
```markdown
## Maintainer decision
<Need, practical impact, and merge-worthiness.>
- Need status: <Demonstrated / Plausible but unproven / Already covered / Unsupported>
- Code recommendation: <code disposition>
- Repository readiness: <integration status; include only for a merge-worthy recommendation when material>
## Evidence
- <runtime or code-path result>
- <test and compatibility result>
## Existing capability and alternatives
<Closest supported path, why the demonstrated scenario cannot use it, and why this patch is preferable to no code change or a narrower design.>
## Issue impact
- Validity: <claim validity>
- Severity: <severity of the underlying issue or need>
- Reach: <realistic reach>
## Patch risk
<Include only when the proposed patch introduces a meaningful regression, compatibility, lifecycle, or maintenance risk.>
## PR quality
- Solution fit: <assessment>
- Tests: <assessment>
- Remaining effort: <bounded/unbounded and why>
## Recommendation
<Merge, focused changes, simpler replacement, or close.>
## Maintainer comment draft
<Include only when closure, evidence, or changes should be requested.>
```
### Competing pull requests
```markdown
## Maintainer decision
<Issue validity, practical severity, and preferred implementation path.>
## Open PR comparison
| PR | Approach | Correctness | Tests | Compatibility/complexity | Readiness |
|---|---|---|---|---|---|
| #... | ... | ... | ... | ... | ... |
## Recommendation
<Select one, request focused changes, combine specific parts, replace all, or merge none.>
<State what should happen to every other open candidate.>
## Maintainer comment drafts
<One copy-paste-ready draft for each PR that should be closed, changed, or superseded.>
```
+10 -8
View File
@@ -1,17 +1,18 @@
---
name: pr-draft-summary
description: Create the required PR-ready summary block, branch suggestion, title, and draft description for openai-agents-python. Use in the final handoff after moderate-or-larger changes to runtime code, tests, examples, build/test configuration, or docs with behavior impact; skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
description: Create the required PR-ready summary block, branch suggestion, title, and draft description for openai-agents-python. Use before the final response whenever the current task changed runtime code, tests, examples, build/test configuration, or docs with behavior impact, regardless of perceived change size and including local-only or uncommitted work. Skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, an explicitly invoked $release-candidate-prep handoff, or when the user explicitly says not to include the PR draft block.
---
# PR Draft Summary
## Purpose
Produce the PR-ready summary required in this repository after substantive code work is complete: a concise summary plus a PR-ready title and draft description that begins with "This pull request <verb> ...". The block should be ready to paste into a PR for openai-agents-python.
Produce the PR-ready summary required in this repository after eligible code work is complete: a concise summary plus a PR-ready title and draft description that begins with "This pull request <verb> ...". The block should be ready to paste into a PR for openai-agents-python.
## When to Trigger
- The task for this repo is finished (or ready for review) and it touched runtime code, tests, examples, docs with behavior impact, or build/test configuration.
- Treat this as the default final handoff step for substantive code work. Run it after any required verification or changeset work and before sending the "work complete" response.
- Skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
- Before every final response, check whether the current task changed runtime code (`src/agents/`), tests (`tests/`), examples (`examples/`), build/test configuration, or docs with behavior impact.
- If it did, run this skill after required verification and before sending the final response. Do not use perceived change size to decide whether to run it.
- Run it for eligible local-only and uncommitted work even when the user did not ask to create a pull request. Producing this text does not authorize creating a branch, committing, pushing, or opening a pull request.
- Skip only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, an explicitly invoked `$release-candidate-prep` handoff that uses the complete `$final-release-review` report as its release-specific PR description, or when the user explicitly says not to include the PR draft block. This exception applies to preparing the release candidate itself, not to implementing or changing the release-preparation skill.
## Inputs to Collect Automatically (do not ask the user)
- Current branch: `git rev-parse --abbrev-ref HEAD`.
@@ -32,9 +33,10 @@ Produce the PR-ready summary required in this repository after substantive code
4) Summarize changes in 13 short sentences using the key paths (top 5) and `git diff --stat` output; explicitly call out untracked files from `git status -sb`/`git ls-files --others --exclude-standard` because `--stat` does not include them. If the working tree is clean but there are commits ahead of `${BASE_COMMIT}`, summarize using those commit messages.
5) Choose the lead verb for the description: feature → `adds`, bug fix → `fixes`, refactor/perf → `improves` or `updates`, docs-only → `updates`.
6) Suggest a branch name. If already off main, keep it; otherwise propose `feat/<slug>`, `fix/<slug>`, or `docs/<slug>` based on the primary area (e.g., `docs/pr-draft-summary-guidance`).
7) If the current branch matches `issue-<number>` (digits only), keep that branch suggestion. Optionally pull light issue context (for example via the GitHub API) when available, but do not block or retry if it is not. When an issue number is present, reference `https://github.com/openai/openai-agents-python/issues/<number>` and include an auto-closing line such as `This pull request resolves #<number>.`.
8) Draft the PR title and description using the template below.
9) Output only the block in "Output Format". Keep any surrounding status note minimal and in English.
7) If the current branch matches `issue-<number>` (digits only), keep that branch suggestion. Optionally pull light issue context (for example via the GitHub API) when available, but do not block or retry if it is not. When an issue number is present, use the native same-repository reference `#<number>` and include an auto-closing line such as `This pull request resolves #<number>.`. Do not add the explicit issue URL or wrap the reference in a Markdown link.
8) Draft the PR title and description using the template below. Apply the repository-wide GitHub paste-readiness rule: use exactly `#123` for same-repository issues or PRs and `owner/repo#123` for cross-repository references; never emit `[PR #123](https://github.com/owner/repo/pull/123)`, `[#123](...)`, Codex navigation links, local file links, Codex-only citation markers or footnotes, or app directives in the copy-ready block. Preserve ordinary descriptive links to API docs, design notes, and other targets without native GitHub issue or pull-request syntax.
9) Normalize references before returning the block: replace every same-repository URL or `openai/openai-agents-python#<number>` reference with `#<number>`, replace every cross-repository issue or pull-request URL with `owner/repo#<number>`, then rescan the full block. Do not return it while a Markdown-linked issue or pull-request label, a same-repository qualified reference, or a bare GitHub issue or pull-request URL remains.
10) Output only the block in "Output Format". Keep any surrounding status note minimal and in English.
## Output Format
When closing out a task, add this concise Markdown block (English only) after any brief status note unless the task falls under the documented skip cases or the user says they do not want it.
@@ -0,0 +1,178 @@
---
name: release-candidate-prep
description: Preflight and prepare an OpenAI Agents Python release candidate in a dedicated worktree from exact origin/main, gate readiness before branch creation, freeze the released API contract, create or replace the local release branch with one release commit, enforce final release review as a checker, and produce release-specific PR text. Use only when explicitly invoked with a version. Never push, open a PR, or mutate GitHub.
---
# Release Candidate Preparation
Use this skill only when the user explicitly invokes `$release-candidate-prep` and supplies a release version without a leading `v`, for example `VERSION=0.20.1`. This skill replaces the removed GitHub Actions release-PR creator with a reviewed local workflow.
## Non-negotiable boundaries
- Treat explicit invocation as authorization to fetch `origin/main`, create one dedicated detached release worktree, run branch-free release-readiness gates there, create or replace the local `release/v<version>` in that worktree only after those gates pass, update the three release-owned files, and create one local commit. If the branch already exists locally or remotely, the required final local state is still exact current `origin/main` plus only the new release commit; an existing local branch may be replaced only when it is not checked out in another worktree.
- Keep the user's source checkout on its existing clean `main` commit. Do not fast-forward it, switch its branch, or materialize release files there. Leave the dedicated release worktree in place for green handoff, blocked review, or recoverable failure.
- Never push, open or edit a pull request, add labels or milestones, create a release, or otherwise mutate GitHub. Never run `gh`.
- Own exactly `pyproject.toml`, `uv.lock`, and `tests/fixtures/released_api_contract.json`. Runtime, documentation, workflow, or other repository changes must land on `main` before release preparation.
- Do not stash, delete, overwrite or remove an existing worktree, or work around unrelated local changes. Fail before branch creation when the initial checkout is dirty or is not on `main`, the dedicated worktree is not clean and detached at refreshed `origin/main`, an existing local release branch is checked out in another worktree, the prospective packaged-contract gate fails after the allowed dependency-bootstrap recovery, the planning review blocks, or `origin/main` advances after those gates run.
- Treat `$final-release-review` as the controlling release checker, not only as a report generator. Its planning gate must be green before branch creation, and its final-candidate gate must inspect the materialized worktree and be green before PR-ready handoff. Any candidate content, commit, or base change invalidates the previous green result.
- Remove inherited `OPENAI_API_KEY` from every child command. Release preparation does not require a live OpenAI API request.
- Stop after the local commit, final release review, and copy-ready handoff. The user owns the push and pull-request creation.
## 1. Establish the release input
Require one semver-like version without a leading `v`. Do not infer a version from milestones, branch names, or local modifications. Announce that the skill will create and retain a dedicated release worktree with one local commit, keep the source checkout unchanged, and not write to GitHub.
Read `$final-release-review` completely before starting. Its final-candidate report is the release pull request description. Do not use `$pr-draft-summary` for the release candidate itself; this skill owns the fixed release branch, commit subject, title, and description. Continue to use `$pr-draft-summary` normally when implementing changes to this skill or other repository behavior.
## 2. Create an isolated branch-free preflight input
From the repository root, run:
```bash
env -u OPENAI_API_KEY -u GITHUB_TOKEN -u GH_TOKEN UV_DEFAULT_INDEX=https://pypi.org/simple uv run --frozen python .agents/skills/release-candidate-prep/scripts/prepare.py preflight --version <version> --worktree-root <codex-worktree-root>
```
The helper must complete all of these operations or fail with an actionable error while leaving the source checkout on its original `main` commit:
1. Verify the repository root, `main` branch, and clean working tree.
2. Inspect whether `release/v<version>` exists locally or remotely. Permit replacement, but fail if the local branch is checked out in another worktree.
3. Fetch `main` into `origin/main` without merging or switching the source checkout.
4. Choose a unique task-oriented path under the configured Codex worktree root. Check both the filesystem and `git worktree list`; never reuse or delete a collision.
5. Create a detached worktree at exact refreshed `origin/main`, then require that worktree to be clean, detached, and at the exact 40-character base commit.
6. Recheck that an existing local release branch remains replaceable and require the source checkout to remain clean on `main` at its original commit.
7. Print the exact base commit, unchanged source-checkout commit, planned branch, and dedicated worktree path for both readiness gates and later materialization.
Record the base commit as `<preflight-base>`, the source-checkout commit as `<source-head>`, and the path as `<release-worktree>`. Do not create or switch branches yet. Keep the detached worktree if a later gate blocks so its exact reviewed source remains inspectable.
## 3. Run the branch-free readiness gates
Bootstrap the dedicated worktree before starting either readiness gate:
```bash
env -u OPENAI_API_KEY -u GITHUB_TOKEN -u GH_TOKEN UV_DEFAULT_INDEX=https://pypi.org/simple make sync
```
This dependency installation is mandatory environment preparation, not candidate materialization. It matches the prospective-contract CI job, which installs all optional dependencies before generating the contract. After synchronization, require `<release-worktree>` to remain clean except for ignored environment or `.tmp` output. If synchronization changes a tracked or untracked repository path, stop with that evidence instead of treating the changed checkout as the reviewed source.
Run both gates against exact `<preflight-base>` before materializing any candidate:
1. Start the prospective packaged-contract gate from `<release-worktree>`:
```bash
env -u OPENAI_API_KEY -u GITHUB_TOKEN -u GH_TOKEN UV_DEFAULT_INDEX=https://pypi.org/simple make check-prospective-released-api-contract
```
2. Invoke `$final-release-review` from `<release-worktree>` in **pre-release planning** mode with `TARGET=<preflight-base>` and the requested version as the release intent. Require its release-checker result to be **GREEN LIGHT TO SHIP**. Keep the target pinned to the commit rather than allowing a later `origin/main` refresh to change the reviewed source, and require all local source, contract, and package inspection to use the dedicated worktree.
These gates are independent consumers of the same clean source commit. Start the prospective command as a long-running session and perform the read-only planning review while it runs when the execution environment supports overlap. Wait for both results before continuing. If concurrency is unavailable, run them sequentially with the prospective gate first; correctness must not depend on overlap.
If the prospective command reports only that optional dependency modules are unavailable, treat the result as a recoverable environment-bootstrap failure rather than a contract-gate decision. Do not ask the user to choose between synchronization and fixing `main`. Rerun the credential-free `make sync` command, require the worktree to remain clean, and retry the prospective command exactly once. Do not use this recovery for a contract mismatch, packaging or runtime compatibility failure, changed repository path, or any other substantive gate failure.
If dependency synchronization still fails, the prospective command still reports unavailable dependency modules after the single retry, or either gate otherwise fails or blocks, stop without creating `release/v<version>`, leave the source checkout unchanged, retain the detached worktree, and report its path plus the exact failure or the planning review's unblock checklist. Classify a dependency installation failure as environment or dependency setup, a contract-generation mismatch as public-surface or `tests/fixtures/released_api_contract_policy.json` work on `main`, and a packaged compatibility failure by its actual failing source, packaging, platform, or runtime path. A blocked planning review should direct runtime or documentation-timing follow-up to `main` as applicable. Do not continue merely because the review produced a well-formed report.
After both gates pass, require all of the following before materialization:
- The source checkout is still clean on `main` at the same commit it had before preflight.
- `<release-worktree>` is clean except for ignored `.tmp` output, remains detached, and has `HEAD == <preflight-base>`.
- The planning review's green gate applies to `<preflight-base>` and the requested release intent.
## 4. Materialize the uncommitted candidate
Run:
```bash
env -u OPENAI_API_KEY -u GITHUB_TOKEN -u GH_TOKEN UV_DEFAULT_INDEX=https://pypi.org/simple uv run --frozen python .agents/skills/release-candidate-prep/scripts/prepare.py materialize --version <version> --expected-base <preflight-base> --expected-source-head <source-head> --worktree <release-worktree>
```
The helper must complete all of these operations or fail with an actionable error:
1. Repeat the source-root, clean `main`, version, registered-worktree, detached-HEAD, and release-branch replaceability checks.
2. Refresh `origin/main` again without moving the source checkout.
3. Require refreshed `origin/main` and `<release-worktree>` HEAD to equal `<preflight-base>`. If `origin/main` advanced, retain the old detached worktree and rerun preflight plus both readiness gates in a new exact-base worktree.
4. Keep the worktree detached while updating the single project version declaration in `pyproject.toml`.
5. Run `make sync` with `UV_DEFAULT_INDEX=https://pypi.org/simple`.
6. Run `make update-released-api-contract VERSION=<version>` and then `make check-released-api-contract VERSION=<version>`.
7. Require exactly the three release-owned paths to be modified in `<release-worktree>`, leave them unstaged and uncommitted, and confirm that the source checkout remains unchanged.
8. Only after those candidate checks pass, create or reset the local `release/v<version>` inside `<release-worktree>` to exact `<preflight-base>` while preserving the validated unstaged manifest. Do not retain commits or content from an older local or remote candidate. This delayed replacement must leave an existing local branch unchanged when candidate generation fails.
If the helper fails after branch creation, preserve its local branch, dedicated worktree, and working-tree evidence. Report the failing command and state rather than guessing whether a partial run is safe to resume. Never remove the worktree as automatic cleanup.
## 5. Review and commit the exact release diff
Run the remaining commands from `<release-worktree>`. Inspect all release-owned files before staging:
```bash
git status --short
git diff --check
git diff -- pyproject.toml uv.lock tests/fixtures/released_api_contract.json
```
Confirm all of the following:
- `pyproject.toml` and the editable `openai-agents` entry in `uv.lock` declare the requested version.
- The API contract baseline is `v<version>` and its `baseline_commit` is the exact `origin/main` source commit on which the release branch is based.
- The generated contract preserves the previous release and freezes intended new exports and signatures.
- Any intended `public_properties`, `canonical_imports`, or `public_modules` policy additions have been reviewed explicitly; the updater deliberately does not infer them.
- No path outside the three-file release manifest is changed, staged, or untracked.
Stage only the manifest and create exactly one local commit:
```bash
git add pyproject.toml uv.lock tests/fixtures/released_api_contract.json
git commit -m "release: <version>"
```
Do not amend unrelated content into the commit.
## 6. Run the final-candidate release review
Invoke `$final-release-review` from `<release-worktree>` in final-candidate mode with the release commit as `TARGET=HEAD`. This invocation is a release checker: it must inspect the complete candidate diff and the actual checked-out `release/v<version>` contents, including `pyproject.toml`, the editable `openai-agents` entry in `uv.lock`, and `tests/fixtures/released_api_contract.json`. The branch, package metadata, lockfile, contract baseline, contract `baseline_commit`, and intended version must agree.
If the review is blocked, stop. Return its unblock checklist, retain the local branch, commit, and worktree for follow-up, and do not present the candidate as PR-ready. A report body does not authorize continuation when the release call is blocked. After any fix, regenerate the API contract when the public surface may have changed, restore a single release commit, and rerun the complete final-candidate review.
The earlier planning review proves that the source commit was ready before branch creation. This final-candidate review remains required because it verifies the materialized branch, version metadata, lockfile, and frozen contract together. Treat its green release call as the handoff gate, then reuse its complete report as the release pull request description; do not substitute the planning report.
## 7. Recheck main freshness
After a green review, fetch `origin main` again without credentials from `<release-worktree>` and compare it with the release commit's parent. If they differ, the candidate is stale. First verify that the branch is clean, has exactly one local commit, and that the commit changes only the three-file release manifest. Rebase that commit onto the new `origin/main` so Git detects any conflicting release metadata. After a clean rebase, move the local release branch back to `origin/main` with a mixed reset, which preserves the rebased release tree as unstaged task-owned changes. Restore all three release-owned files (`pyproject.toml`, `uv.lock`, and `tests/fixtures/released_api_contract.json`) from `origin/main`, run `make sync`, and require the worktree to be clean at the new base. Run `make check-prospective-released-api-contract` only in that internally consistent base state, where the installed project version and frozen contract baseline agree. Then update `pyproject.toml` to `<version>`, run `make sync`, run `make update-released-api-contract VERSION=<version>` and `make check-released-api-contract VERSION=<version>`, review the exact manifest again, and recreate the single `release: <version>` commit. The base and candidate content changed, so the previous green check is invalid: rerun `$final-release-review` from the worktree and require a new green release call. Repeat until the reviewed local branch is exactly one commit ahead of current `origin/main` and that commit changes only the three-file release manifest.
If replay conflicts or another path changes, stop with recoverable evidence. Do not force a resolution that expands the release commit beyond its manifest.
## 8. Produce the release handoff
For a green, current candidate, return the `$final-release-review` report plus this release-specific block in English:
```markdown
# Release Pull Request
## Branch
release/v<version>
## Commit
release: <version>
## Title
Release <version>
## Description
<the complete final-candidate report from $final-release-review>
```
Apply the repository's GitHub paste-readiness rules to the report. Use native `#123` references for this repository and `owner/repo#123` for another repository. Keep the required compare URL. Do not include local paths, Codex citations, operational diagnostics, or app directives inside the copy-ready description.
Also report the dedicated worktree path, local branch, commit SHA, parent `origin/main` commit, and the exact three-file manifest outside the copy-ready block. State explicitly that the source checkout was left unchanged, nothing was pushed, and no pull request was created. Leave the worktree in place for the user's handoff.
If `release/v<version>` already exists on `origin`, inspect its exact current commit with credential-free `git ls-remote --heads origin release/v<version>` immediately before handoff and record it as `<observed-remote-release-commit>`. State explicitly that the local branch has replaced the old candidate and now contains exact current `origin/main` plus only the new `release: <version>` commit. Because this skill never mutates GitHub, provide the user with the exact `git push --force-with-lease=refs/heads/release/v<version>:<observed-remote-release-commit> origin release/v<version>` command to replace the remote branch themselves; never run it. A normal push or an unspecified lease is insufficient for this replacement case. If the remote branch changes after inspection, the explicit lease must reject the push instead of overwriting unseen work.
## Failure behavior
- Preflight or worktree creation failure: leave the source checkout unchanged and do not delete or reuse any colliding worktree.
- Dependency-bootstrap failure: retry unavailable optional dependency setup only as described in the readiness-gate procedure, then retain the detached worktree and return the exact failure if recovery does not succeed.
- Prospective-contract failure after the allowed dependency-bootstrap recovery or blocked planning review: retain the detached worktree, do not create the release branch, and return the exact failure or unblock checklist.
- Materialization failure before successful branch replacement: retain the detached worktree and its uncommitted evidence, and leave any existing local release branch unchanged. Failure after successful branch replacement must retain the worktree, branch, and evidence exactly as left by the failing command.
- Blocked final-candidate review: retain the single release commit and worktree, do not call the candidate PR-ready, and return the checker-derived unblock checklist.
- Freshness conflict or unexpected changed path: stop with recoverable worktree evidence rather than forcing a resolution or expanding the release manifest.
+600
View File
@@ -0,0 +1,600 @@
#!/usr/bin/env python3
"""Preflight and materialize an isolated release candidate from exact origin/main."""
from __future__ import annotations
import argparse
import json
import os
import re
import shlex
import subprocess
import sys
from dataclasses import dataclass
from pathlib import Path
if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib
from collections.abc import Sequence
ROOT = Path(__file__).resolve().parents[4]
VERSION_PATTERN = re.compile(r"\d+\.\d+(?:\.\d+)*(?:[A-Za-z0-9.-]+)?\Z")
COMMIT_PATTERN = re.compile(r"[0-9a-f]{40}\Z")
PROJECT_VERSION_PATTERN = re.compile(r'(?m)^version\s*=\s*"[^"]+"')
RELEASE_PATHS = frozenset(
{
"pyproject.toml",
"tests/fixtures/released_api_contract.json",
"uv.lock",
}
)
class ReleasePreparationError(RuntimeError):
"""Report a safe, actionable release preparation failure."""
@dataclass(frozen=True)
class ReleasePreflight:
"""Describe an isolated branch-free release readiness input."""
base_commit: str
branch: str
source_commit: str
version: str
worktree: Path
@dataclass(frozen=True)
class PreparedCandidate:
"""Describe the successfully prepared isolated candidate."""
base_commit: str
branch: str
changed_paths: tuple[str, ...]
source_commit: str
version: str
worktree: Path
def _release_environment() -> dict[str, str]:
env = os.environ.copy()
for name in ("GH_TOKEN", "GITHUB_TOKEN", "OPENAI_API_KEY"):
env.pop(name, None)
env["UV_DEFAULT_INDEX"] = "https://pypi.org/simple"
return env
def _command_text(args: Sequence[str]) -> str:
return shlex.join(str(arg) for arg in args)
def run_command(
repo: Path,
args: Sequence[str],
*,
env: dict[str, str] | None = None,
announce: bool = False,
check: bool = True,
) -> subprocess.CompletedProcess[str]:
"""Run one command and preserve useful output for failures."""
if announce:
print(f"+ {_command_text(args)}", flush=True)
effective_env = _release_environment() if env is None else env.copy()
for name in ("GH_TOKEN", "GITHUB_TOKEN", "OPENAI_API_KEY"):
effective_env.pop(name, None)
effective_env["UV_DEFAULT_INDEX"] = "https://pypi.org/simple"
result = subprocess.run(
[str(arg) for arg in args],
cwd=repo,
env=effective_env,
check=False,
capture_output=True,
text=True,
)
if announce:
if result.stdout:
print(result.stdout, end="")
if result.stderr:
print(result.stderr, end="", file=sys.stderr)
if check and result.returncode != 0:
detail = result.stderr.strip() or result.stdout.strip() or "unknown command failure"
raise ReleasePreparationError(f"{_command_text(args)} failed: {detail}")
return result
def git(repo: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
"""Run a Git inspection command."""
return run_command(repo, ["git", *args], check=check)
def validate_version(version: str) -> str:
"""Validate the release version accepted by the existing contract updater."""
if version.startswith("v") or ".." in version or VERSION_PATTERN.fullmatch(version) is None:
raise ReleasePreparationError(
"Version must be semver-like without a leading v, for example 0.20.1 or 0.21.0-rc1."
)
return version
def validate_commit(commit: str) -> str:
"""Validate an exact lowercase Git commit identifier."""
if COMMIT_PATTERN.fullmatch(commit) is None:
raise ReleasePreparationError(
"Expected base must be a full 40-character lowercase Git commit identifier."
)
return commit
def project_version(repo: Path) -> str:
"""Read the project version from pyproject.toml."""
data = tomllib.loads((repo / "pyproject.toml").read_text(encoding="utf-8"))
version = data.get("project", {}).get("version")
if not isinstance(version, str):
raise ReleasePreparationError("pyproject.toml is missing project.version.")
return version
def project_version_at(repo: Path, commit: str) -> str:
"""Read the project version from one exact commit without changing a checkout."""
text = git(repo, "show", f"{commit}:pyproject.toml").stdout
data = tomllib.loads(text)
version = data.get("project", {}).get("version")
if not isinstance(version, str):
raise ReleasePreparationError("pyproject.toml is missing project.version.")
return version
def replace_project_version_text(text: str, version: str) -> str:
"""Replace the repository's single project version declaration."""
updated, count = PROJECT_VERSION_PATTERN.subn(f'version = "{version}"', text)
if count != 1:
raise ReleasePreparationError(
f"Expected exactly one version declaration in pyproject.toml, found {count}."
)
if updated == text:
raise ReleasePreparationError(f"pyproject.toml already declares version {version}.")
return updated
def replace_project_version(repo: Path, version: str) -> None:
"""Update pyproject.toml while preserving all unrelated text."""
path = repo / "pyproject.toml"
text = path.read_text(encoding="utf-8")
path.write_text(replace_project_version_text(text, version), encoding="utf-8")
def _current_branch(repo: Path) -> str:
result = git(repo, "symbolic-ref", "--quiet", "--short", "HEAD", check=False)
if result.returncode != 0:
raise ReleasePreparationError("Release preparation requires a named main branch.")
return result.stdout.strip()
def _status(repo: Path) -> str:
return git(repo, "status", "--porcelain=v1", "--untracked-files=all").stdout
def _require_repository_root(repo: Path) -> None:
top_level = Path(git(repo, "rev-parse", "--show-toplevel").stdout.strip()).resolve()
if top_level != repo.resolve():
raise ReleasePreparationError(
f"Run release preparation from the repository root {top_level}, not {repo.resolve()}."
)
def _require_clean_main(repo: Path) -> None:
branch = _current_branch(repo)
if branch != "main":
raise ReleasePreparationError(
f"Release preparation requires branch 'main', found {branch!r}."
)
if _status(repo):
raise ReleasePreparationError("Release preparation requires a clean working tree.")
def _require_source_head(repo: Path, expected_source_head: str) -> None:
"""Require the user's source checkout to remain at its preflight commit."""
source_head = git(repo, "rev-parse", "HEAD").stdout.strip()
if source_head != expected_source_head:
raise ReleasePreparationError(
f"Source checkout HEAD changed from {expected_source_head} to {source_head}; "
"leave both checkouts intact and restart release preparation."
)
def _registered_worktrees(repo: Path) -> set[Path]:
"""Return canonical paths registered in the repository worktree inventory."""
paths: set[Path] = set()
for line in git(repo, "worktree", "list", "--porcelain").stdout.splitlines():
if line.startswith("worktree "):
paths.add(Path(line.removeprefix("worktree ")).resolve())
return paths
def _choose_worktree_path(repo: Path, worktree_root: Path, version: str) -> Path:
"""Choose a unique release worktree path without reusing or deleting collisions."""
worktree_root = worktree_root.expanduser().resolve()
if worktree_root == repo or worktree_root.is_relative_to(repo):
raise ReleasePreparationError(
"The release worktree root must be outside the source checkout."
)
registered = _registered_worktrees(repo)
stem = f"{repo.name}-release-v{version}"
suffix = 1
while True:
name = stem if suffix == 1 else f"{stem}-{suffix}"
candidate = worktree_root / name
if not candidate.exists() and candidate.resolve() not in registered:
return candidate
suffix += 1
def _require_registered_detached_worktree(
source_repo: Path,
worktree: Path,
expected_base: str,
) -> None:
"""Require a clean registered detached worktree at the reviewed base."""
worktree = worktree.expanduser().resolve()
if worktree not in _registered_worktrees(source_repo):
raise ReleasePreparationError(
f"Release worktree {worktree} is not registered for this repository."
)
if not worktree.is_dir():
raise ReleasePreparationError(f"Release worktree path does not exist: {worktree}.")
_require_repository_root(worktree)
branch = git(worktree, "symbolic-ref", "--quiet", "--short", "HEAD", check=False)
if branch.returncode == 0:
raise ReleasePreparationError(
f"Release worktree must remain detached before materialization, found "
f"{branch.stdout.strip()!r}."
)
if branch.returncode != 1:
raise ReleasePreparationError("Unable to inspect the release worktree branch state.")
head = git(worktree, "rev-parse", "HEAD").stdout.strip()
if head != expected_base:
raise ReleasePreparationError(
f"Release worktree HEAD is {head}, expected reviewed base {expected_base}."
)
if _status(worktree):
raise ReleasePreparationError("Release worktree must be clean before materialization.")
def _worktrees_using_branch(repo: Path, branch: str) -> tuple[Path, ...]:
"""Return registered worktrees that currently check out one local branch."""
matches: list[Path] = []
worktree: Path | None = None
for line in [*git(repo, "worktree", "list", "--porcelain").stdout.splitlines(), ""]:
if line.startswith("worktree "):
worktree = Path(line.removeprefix("worktree ")).resolve()
elif line == f"branch refs/heads/{branch}" and worktree is not None:
matches.append(worktree)
elif not line:
worktree = None
return tuple(matches)
def _require_branch_replaceable(repo: Path, branch: str) -> None:
"""Require an existing release branch to be safe to replace locally."""
local = git(repo, "show-ref", "--verify", "--quiet", f"refs/heads/{branch}", check=False)
if local.returncode not in (0, 1):
raise ReleasePreparationError(f"Unable to inspect local branch {branch!r}.")
if local.returncode == 0:
worktrees = _worktrees_using_branch(repo, branch)
if worktrees:
locations = ", ".join(str(path) for path in worktrees)
raise ReleasePreparationError(
f"Local branch {branch!r} is checked out in {locations}; switch that worktree "
"away from the branch before replacing the release candidate."
)
remote = git(repo, "ls-remote", "--exit-code", "--heads", "origin", branch, check=False)
if remote.returncode not in (0, 2):
detail = remote.stderr.strip() or remote.stdout.strip() or "unknown remote error"
raise ReleasePreparationError(f"Unable to inspect remote branch {branch!r}: {detail}")
def _changed_paths(repo: Path) -> set[str]:
changed: set[str] = set()
for args in (
("diff", "--name-only"),
("diff", "--name-only", "--cached"),
("ls-files", "--others", "--exclude-standard"),
):
changed.update(line for line in git(repo, *args).stdout.splitlines() if line)
return changed
def _locked_project_version(repo: Path) -> str:
data = tomllib.loads((repo / "uv.lock").read_text(encoding="utf-8"))
packages = data.get("package", [])
matches = [
package
for package in packages
if package.get("name") == "openai-agents" and package.get("source") == {"editable": "."}
]
if len(matches) != 1:
raise ReleasePreparationError(
"uv.lock must contain exactly one editable openai-agents package with a version."
)
locked_version = matches[0].get("version")
if not isinstance(locked_version, str):
raise ReleasePreparationError(
"uv.lock must contain exactly one editable openai-agents package with a version."
)
return locked_version
def _validate_prepared_files(repo: Path, version: str, base_commit: str) -> tuple[str, ...]:
changed = _changed_paths(repo)
if changed != RELEASE_PATHS:
missing = sorted(RELEASE_PATHS - changed)
unexpected = sorted(changed - RELEASE_PATHS)
raise ReleasePreparationError(
"Prepared release paths do not match the required manifest; "
f"missing={missing!r}, unexpected={unexpected!r}."
)
if project_version(repo) != version:
raise ReleasePreparationError("pyproject.toml does not contain the requested version.")
if _locked_project_version(repo) != version:
raise ReleasePreparationError("uv.lock does not contain the requested project version.")
contract = json.loads(
(repo / "tests/fixtures/released_api_contract.json").read_text(encoding="utf-8")
)
if contract.get("baseline") != f"v{version}":
raise ReleasePreparationError(
"The released API contract baseline does not match the version."
)
if contract.get("baseline_commit") != base_commit:
raise ReleasePreparationError(
"The released API contract baseline_commit does not match "
"the origin/main source commit."
)
if git(repo, "diff", "--cached", "--quiet", check=False).returncode != 0:
raise ReleasePreparationError("The helper must leave all release changes unstaged.")
return tuple(sorted(changed))
def preflight(repo: Path, version: str, worktree_root: Path) -> ReleasePreflight:
"""Refresh exact main and create an isolated branch-free readiness checkout."""
repo = repo.resolve()
version = validate_version(version)
_require_repository_root(repo)
_require_clean_main(repo)
source_commit = git(repo, "rev-parse", "HEAD").stdout.strip()
branch = f"release/v{version}"
_require_branch_replaceable(repo, branch)
env = _release_environment()
run_command(
repo,
[
"git",
"fetch",
"origin",
"refs/heads/main:refs/remotes/origin/main",
"--prune",
],
env=env,
announce=True,
)
base_commit = git(repo, "rev-parse", "origin/main").stdout.strip()
if project_version_at(repo, base_commit) == version:
raise ReleasePreparationError(f"Refreshed origin/main already declares version {version}.")
_require_branch_replaceable(repo, branch)
if _status(repo):
raise ReleasePreparationError(
"Release preflight must leave the source main working tree clean."
)
worktree = _choose_worktree_path(repo, worktree_root, version)
worktree.parent.mkdir(parents=True, exist_ok=True)
run_command(
repo,
["git", "worktree", "add", "--detach", str(worktree), base_commit],
env=env,
announce=True,
)
_require_registered_detached_worktree(repo, worktree, base_commit)
_require_clean_main(repo)
_require_source_head(repo, source_commit)
return ReleasePreflight(
base_commit=base_commit,
branch=branch,
source_commit=source_commit,
version=version,
worktree=worktree,
)
def materialize(
repo: Path,
version: str,
expected_base: str,
expected_source_head: str,
worktree: Path,
) -> PreparedCandidate:
"""Create the three-file candidate in the reviewed isolated worktree."""
expected_base = validate_commit(expected_base)
expected_source_head = validate_commit(expected_source_head)
repo = repo.resolve()
version = validate_version(version)
worktree = worktree.expanduser().resolve()
_require_repository_root(repo)
_require_clean_main(repo)
_require_source_head(repo, expected_source_head)
_require_registered_detached_worktree(repo, worktree, expected_base)
env = _release_environment()
branch = f"release/v{version}"
_require_branch_replaceable(repo, branch)
run_command(
repo,
[
"git",
"fetch",
"origin",
"refs/heads/main:refs/remotes/origin/main",
"--prune",
],
env=env,
announce=True,
)
base_commit = git(repo, "rev-parse", "origin/main").stdout.strip()
if base_commit != expected_base:
raise ReleasePreparationError(
f"Preflight reviewed {expected_base}, but refreshed origin/main is {base_commit}; "
"leave the detached worktree intact and rerun preflight plus both readiness gates."
)
_require_clean_main(repo)
_require_source_head(repo, expected_source_head)
_require_branch_replaceable(repo, branch)
_require_registered_detached_worktree(repo, worktree, expected_base)
if project_version(worktree) == version:
raise ReleasePreparationError(f"Project version is already {version}.")
replace_project_version(worktree, version)
run_command(worktree, ["make", "sync"], env=env, announce=True)
run_command(
worktree,
["make", "update-released-api-contract", f"VERSION={version}"],
env=env,
announce=True,
)
run_command(
worktree,
["make", "check-released-api-contract", f"VERSION={version}"],
env=env,
announce=True,
)
changed_paths = _validate_prepared_files(worktree, version, base_commit)
_require_clean_main(repo)
_require_source_head(repo, expected_source_head)
_require_branch_replaceable(repo, branch)
run_command(
worktree,
["git", "switch", "--no-track", "-C", branch, expected_base],
env=env,
announce=True,
)
return PreparedCandidate(
base_commit=base_commit,
branch=branch,
changed_paths=changed_paths,
source_commit=expected_source_head,
version=version,
worktree=worktree,
)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Preflight or materialize an isolated release candidate from exact origin/main."
)
subparsers = parser.add_subparsers(dest="phase", required=True)
preflight_parser = subparsers.add_parser(
"preflight",
help="Refresh exact main and validate inputs without creating a release branch.",
)
preflight_parser.add_argument(
"--version",
required=True,
help="Release version without a leading v, for example 0.20.1.",
)
preflight_parser.add_argument(
"--worktree-root",
type=Path,
default=Path(os.environ.get("CODEX_WORKTREE_ROOT", Path.home() / ".codex/worktrees")),
help="Directory under which to create a unique detached release worktree.",
)
materialize_parser = subparsers.add_parser(
"materialize",
help="Create an uncommitted candidate from the reviewed preflight commit.",
)
materialize_parser.add_argument(
"--version",
required=True,
help="Release version without a leading v, for example 0.20.1.",
)
materialize_parser.add_argument(
"--expected-base",
required=True,
help="Exact 40-character origin/main commit approved by release preflight.",
)
materialize_parser.add_argument(
"--expected-source-head",
required=True,
help="Exact source-checkout HEAD recorded by release preflight.",
)
materialize_parser.add_argument(
"--worktree",
type=Path,
required=True,
help="Detached worktree created by the matching preflight.",
)
return parser.parse_args()
def main() -> int:
args = parse_args()
try:
if args.phase == "preflight":
release_input = preflight(ROOT, args.version, args.worktree_root)
else:
candidate = materialize(
ROOT,
args.version,
args.expected_base,
args.expected_source_head,
args.worktree,
)
except (OSError, ReleasePreparationError, tomllib.TOMLDecodeError, json.JSONDecodeError) as exc:
print(f"Release preparation failed: {exc}", file=sys.stderr)
return 1
if args.phase == "preflight":
print("Release preflight passed without changing the source main checkout.")
print(f"Base commit: {release_input.base_commit}")
print(f"Source commit: {release_input.source_commit}")
print(f"Planned branch: {release_input.branch}")
print(f"Version: {release_input.version}")
print(f"Worktree: {release_input.worktree}")
print("Run both readiness gates from this detached worktree against the base commit.")
return 0
print("Release candidate prepared in its dedicated worktree and left uncommitted.")
print(f"Base commit: {candidate.base_commit}")
print(f"Source commit: {candidate.source_commit}")
print(f"Branch: {candidate.branch}")
print(f"Version: {candidate.version}")
print(f"Worktree: {candidate.worktree}")
print("Changed paths:")
for path in candidate.changed_paths:
print(f"- {path}")
print("Review the diff before staging the three release-owned files.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,685 @@
#!/usr/bin/env python3
"""Focused tests for local release candidate preparation."""
from __future__ import annotations
import json
import os
import subprocess
import tempfile
import unittest
from pathlib import Path
from unittest import mock
import prepare
def run(repo: Path, *args: str, check: bool = True) -> subprocess.CompletedProcess[str]:
return subprocess.run(
list(args),
cwd=repo,
check=check,
capture_output=True,
text=True,
)
class ReleaseRepository:
"""Create a disposable release repository with a local bare origin."""
def __init__(self, root: Path) -> None:
self.root = root
self.repo = root / "repo"
self.origin = root / "origin.git"
self.worktree_root = root / "worktrees"
self.repo.mkdir()
run(self.repo, "git", "init", "--initial-branch=main")
run(self.repo, "git", "config", "user.name", "Release Test")
run(self.repo, "git", "config", "user.email", "release-test@example.com")
self._write_fixture_files()
run(self.repo, "git", "add", ".")
run(self.repo, "git", "commit", "-m", "Initial release source")
run(root, "git", "init", "--bare", str(self.origin))
run(self.repo, "git", "remote", "add", "origin", str(self.origin))
run(self.repo, "git", "push", "--set-upstream", "origin", "main")
self.base_commit = run(self.repo, "git", "rev-parse", "HEAD").stdout.strip()
def advance_origin(self) -> str:
updater = self.root / "updater"
run(self.root, "git", "clone", "--branch", "main", str(self.origin), str(updater))
run(updater, "git", "config", "user.name", "Release Updater")
run(updater, "git", "config", "user.email", "release-updater@example.com")
(updater / "new-source.txt").write_text("new source\n", encoding="utf-8")
run(updater, "git", "add", "new-source.txt")
run(updater, "git", "commit", "-m", "Advance main")
run(updater, "git", "push", "origin", "main")
return run(updater, "git", "rev-parse", "HEAD").stdout.strip()
def _write_fixture_files(self) -> None:
(self.repo / "tests/fixtures").mkdir(parents=True)
(self.repo / "pyproject.toml").write_text(
'[project]\nname = "openai-agents"\nversion = "0.19.4"\n',
encoding="utf-8",
)
(self.repo / "uv.lock").write_text(
'version = 1\n\n[[package]]\nname = "openai-agents"\nversion = "0.19.4"\n'
'source = { editable = "." }\n',
encoding="utf-8",
)
(self.repo / "tests/fixtures/released_api_contract.json").write_text(
json.dumps(
{
"baseline": "v0.19.4",
"baseline_commit": "a" * 40,
"callables": {},
"required_top_level_exports": [],
},
indent=2,
sort_keys=True,
)
+ "\n",
encoding="utf-8",
)
(self.repo / "fake_release.py").write_text(
"""from __future__ import annotations
import json
import os
import re
import subprocess
import sys
from pathlib import Path
root = Path(__file__).parent
action = sys.argv[1]
if {'GH_TOKEN', 'GITHUB_TOKEN', 'OPENAI_API_KEY'} & os.environ.keys():
raise SystemExit('credentials must not reach release subprocesses')
if os.environ.get('UV_DEFAULT_INDEX') != 'https://pypi.org/simple':
raise SystemExit('UV_DEFAULT_INDEX must use the public package index')
version = re.search(
r'^version = \"([^\"]+)\"$',
(root / 'pyproject.toml').read_text(),
re.MULTILINE,
).group(1)
if action == 'sync':
path = root / 'uv.lock'
text = path.read_text()
text = re.sub(
r'(name = \"openai-agents\"\\nversion = \")[^\"]+(\")',
rf'\\g<1>{version}\\g<2>',
text,
)
path.write_text(text)
elif action == 'update':
expected = sys.argv[2]
if expected != version:
raise SystemExit('version mismatch')
path = root / 'tests/fixtures/released_api_contract.json'
contract = json.loads(path.read_text())
contract['baseline'] = f'v{version}'
contract['baseline_commit'] = subprocess.check_output(
['git', 'rev-parse', 'HEAD'], cwd=root, text=True
).strip()
path.write_text(json.dumps(contract, indent=2, sort_keys=True) + '\\n')
elif action == 'check':
expected = sys.argv[2]
contract = json.loads(
(root / 'tests/fixtures/released_api_contract.json').read_text()
)
if expected != version or contract['baseline'] != f'v{version}':
raise SystemExit('contract mismatch')
else:
raise SystemExit(f'unknown action: {action}')
""",
encoding="utf-8",
)
(self.repo / "Makefile").write_text(
"sync:\n\tpython fake_release.py sync\n\n"
"update-released-api-contract:\n\tpython fake_release.py update $(VERSION)\n\n"
"check-released-api-contract:\n\tpython fake_release.py check $(VERSION)\n",
encoding="utf-8",
)
class VersionTests(unittest.TestCase):
def test_validate_version_accepts_release_and_prerelease(self) -> None:
self.assertEqual(prepare.validate_version("0.20.1"), "0.20.1")
self.assertEqual(prepare.validate_version("0.21.0-rc1"), "0.21.0-rc1")
def test_validate_version_rejects_ambiguous_values(self) -> None:
for value in ("v0.20.1", "0..20.1", "next", "0.20.1/other"):
with self.subTest(value=value), self.assertRaises(prepare.ReleasePreparationError):
prepare.validate_version(value)
def test_replace_project_version_requires_exactly_one_change(self) -> None:
text = '[project]\nversion = "0.19.4"\n'
self.assertEqual(
prepare.replace_project_version_text(text, "0.20.0"),
'[project]\nversion = "0.20.0"\n',
)
with self.assertRaises(prepare.ReleasePreparationError):
prepare.replace_project_version_text(text, "0.19.4")
with self.assertRaises(prepare.ReleasePreparationError):
prepare.replace_project_version_text(
text + '[tool.example]\nversion = "1.0.0"\n',
"0.20.0",
)
def test_validate_commit_requires_full_lowercase_identifier(self) -> None:
commit = "a" * 40
self.assertEqual(prepare.validate_commit(commit), commit)
for value in ("a" * 39, "A" * 40, "main", "a" * 41):
with self.subTest(value=value), self.assertRaises(prepare.ReleasePreparationError):
prepare.validate_commit(value)
class PreparationTests(unittest.TestCase):
def test_preflight_leaves_clean_main_without_creating_branch(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
source_head = fixture.base_commit
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
self.assertEqual(release_input.base_commit, fixture.base_commit)
self.assertEqual(release_input.branch, "release/v0.20.0")
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
self.assertEqual(run(fixture.repo, "git", "status", "--porcelain").stdout, "")
self.assertEqual(
run(fixture.repo, "git", "rev-parse", "HEAD").stdout.strip(),
source_head,
)
self.assertEqual(
run(release_input.worktree, "git", "rev-parse", "HEAD").stdout.strip(),
fixture.base_commit,
)
self.assertEqual(
run(
release_input.worktree,
"git",
"symbolic-ref",
"--quiet",
"--short",
"HEAD",
check=False,
).returncode,
1,
)
self.assertEqual(
run(release_input.worktree, "git", "status", "--porcelain").stdout,
"",
)
self.assertNotIn(
"release/v0.20.0",
run(fixture.repo, "git", "branch", "--format=%(refname:short)").stdout.splitlines(),
)
def test_materialize_creates_branch_and_exact_uncommitted_manifest(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
with mock.patch.dict(
os.environ,
{
"GH_TOKEN": "untrusted",
"GITHUB_TOKEN": "untrusted",
"OPENAI_API_KEY": "untrusted",
},
):
candidate = prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(candidate.base_commit, fixture.base_commit)
self.assertEqual(candidate.branch, "release/v0.20.0")
self.assertEqual(set(candidate.changed_paths), prepare.RELEASE_PATHS)
self.assertEqual(
run(release_input.worktree, "git", "branch", "--show-current").stdout.strip(),
"release/v0.20.0",
)
self.assertEqual(
run(
release_input.worktree,
"git",
"rev-list",
"--count",
"origin/main..HEAD",
).stdout.strip(),
"0",
)
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
self.assertEqual(run(fixture.repo, "git", "status", "--porcelain").stdout, "")
remote_heads = run(
fixture.repo,
"git",
"ls-remote",
"--heads",
"origin",
"release/v0.20.0",
).stdout
self.assertEqual(remote_heads, "")
contract = json.loads(
(release_input.worktree / "tests/fixtures/released_api_contract.json").read_text()
)
self.assertEqual(contract["baseline"], "v0.20.0")
self.assertEqual(contract["baseline_commit"], fixture.base_commit)
def test_preflight_rejects_dirty_main_without_creating_branch(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
(fixture.repo / "dirty.txt").write_text("dirty\n", encoding="utf-8")
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"clean working tree",
):
prepare.preflight(fixture.repo, "0.20.0", fixture.worktree_root)
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
def test_preflight_keeps_source_main_and_checks_out_refreshed_origin_in_worktree(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
source_head = fixture.base_commit
refreshed_base = fixture.advance_origin()
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
self.assertEqual(release_input.base_commit, refreshed_base)
self.assertEqual(
run(fixture.repo, "git", "rev-parse", "HEAD").stdout.strip(),
source_head,
)
self.assertFalse((fixture.repo / "new-source.txt").exists())
self.assertEqual(
run(release_input.worktree, "git", "rev-parse", "HEAD").stdout.strip(),
refreshed_base,
)
self.assertTrue((release_input.worktree / "new-source.txt").is_file())
def test_existing_remote_release_branch_is_replaced_locally(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
run(
fixture.repo,
"git",
"push",
"origin",
"HEAD:refs/heads/release/v0.20.0",
)
old_remote_candidate = fixture.base_commit
refreshed_base = fixture.advance_origin()
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
candidate = prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(candidate.branch, "release/v0.20.0")
self.assertEqual(release_input.base_commit, refreshed_base)
self.assertEqual(
run(release_input.worktree, "git", "rev-parse", "HEAD").stdout.strip(),
refreshed_base,
)
self.assertEqual(
run(
fixture.repo,
"git",
"ls-remote",
"--heads",
"origin",
"release/v0.20.0",
).stdout.split()[0],
old_remote_candidate,
)
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
def test_existing_local_release_branch_is_replaced_at_reviewed_base(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
run(fixture.repo, "git", "branch", "release/v0.20.0")
stale_commit = fixture.base_commit
refreshed_base = fixture.advance_origin()
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertNotEqual(stale_commit, refreshed_base)
self.assertEqual(release_input.base_commit, refreshed_base)
self.assertEqual(
run(
fixture.repo,
"git",
"rev-parse",
"refs/heads/release/v0.20.0",
).stdout.strip(),
refreshed_base,
)
def test_preflight_rejects_release_branch_checked_out_in_another_worktree(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
colliding_worktree = fixture.root / "existing-release"
run(
fixture.repo,
"git",
"worktree",
"add",
"-b",
"release/v0.20.0",
str(colliding_worktree),
fixture.base_commit,
)
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"is checked out in",
):
prepare.preflight(fixture.repo, "0.20.0", fixture.worktree_root)
self.assertEqual(
run(colliding_worktree, "git", "branch", "--show-current").stdout.strip(),
"release/v0.20.0",
)
def test_materialize_rejects_stale_preflight_before_creating_branch(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
refreshed_base = fixture.advance_origin()
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
f"Preflight reviewed {release_input.base_commit}, but refreshed origin/main is "
f"{refreshed_base}",
):
prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
self.assertEqual(
run(fixture.repo, "git", "rev-parse", "HEAD").stdout.strip(),
fixture.base_commit,
)
self.assertEqual(run(fixture.repo, "git", "status", "--porcelain").stdout, "")
self.assertEqual(
run(release_input.worktree, "git", "rev-parse", "HEAD").stdout.strip(),
fixture.base_commit,
)
self.assertEqual(
run(
release_input.worktree,
"git",
"symbolic-ref",
"--quiet",
"--short",
"HEAD",
check=False,
).returncode,
1,
)
self.assertNotIn(
"release/v0.20.0",
run(fixture.repo, "git", "branch", "--format=%(refname:short)").stdout.splitlines(),
)
def test_preflight_chooses_a_new_path_without_reusing_a_worktree_collision(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
collision = fixture.worktree_root / f"{fixture.repo.name}-release-v0.20.0"
collision.parent.mkdir(parents=True)
run(
fixture.repo,
"git",
"worktree",
"add",
"--detach",
str(collision),
fixture.base_commit,
)
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
self.assertEqual(
release_input.worktree,
(fixture.worktree_root / f"{fixture.repo.name}-release-v0.20.0-2").resolve(),
)
self.assertTrue(collision.is_dir())
self.assertEqual(
run(collision, "git", "rev-parse", "HEAD").stdout.strip(),
fixture.base_commit,
)
def test_preflight_rejects_a_worktree_root_inside_the_source_checkout(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
nested_root = fixture.repo / ".release-worktrees"
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"must be outside the source checkout",
):
prepare.preflight(fixture.repo, "0.20.0", nested_root)
self.assertFalse(nested_root.exists())
self.assertEqual(run(fixture.repo, "git", "status", "--porcelain").stdout, "")
def test_materialize_failure_preserves_detached_evidence_and_source_checkout(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
real_run_command = prepare.run_command
def fail_sync(
repo: Path,
args: list[str] | tuple[str, ...],
**kwargs: object,
) -> subprocess.CompletedProcess[str]:
if list(args) == ["make", "sync"]:
raise prepare.ReleasePreparationError("simulated make sync failure")
return real_run_command(repo, args, **kwargs)
with mock.patch.object(prepare, "run_command", side_effect=fail_sync):
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"simulated make sync failure",
):
prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(
run(fixture.repo, "git", "branch", "--show-current").stdout.strip(),
"main",
)
self.assertEqual(run(fixture.repo, "git", "status", "--porcelain").stdout, "")
self.assertEqual(
run(
release_input.worktree,
"git",
"symbolic-ref",
"--quiet",
"--short",
"HEAD",
check=False,
).returncode,
1,
)
self.assertIn(
"pyproject.toml",
run(release_input.worktree, "git", "status", "--porcelain").stdout,
)
self.assertEqual(prepare.project_version(release_input.worktree), "0.20.0")
def test_materialize_failure_does_not_replace_existing_local_release_branch(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
run(fixture.repo, "git", "branch", "release/v0.20.0")
existing_candidate = fixture.base_commit
fixture.advance_origin()
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
real_run_command = prepare.run_command
def fail_sync(
repo: Path,
args: list[str] | tuple[str, ...],
**kwargs: object,
) -> subprocess.CompletedProcess[str]:
if list(args) == ["make", "sync"]:
raise prepare.ReleasePreparationError("simulated make sync failure")
return real_run_command(repo, args, **kwargs)
with mock.patch.object(prepare, "run_command", side_effect=fail_sync):
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"simulated make sync failure",
):
prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(
run(
fixture.repo,
"git",
"rev-parse",
"refs/heads/release/v0.20.0",
).stdout.strip(),
existing_candidate,
)
self.assertEqual(
run(
release_input.worktree,
"git",
"symbolic-ref",
"--quiet",
"--short",
"HEAD",
check=False,
).returncode,
1,
)
def test_materialize_rejects_a_source_checkout_head_change(self) -> None:
with tempfile.TemporaryDirectory() as directory:
fixture = ReleaseRepository(Path(directory))
release_input = prepare.preflight(
fixture.repo,
"0.20.0",
fixture.worktree_root,
)
(fixture.repo / "local-only.txt").write_text("local\n", encoding="utf-8")
run(fixture.repo, "git", "add", "local-only.txt")
run(fixture.repo, "git", "commit", "-m", "Move source checkout")
with self.assertRaisesRegex(
prepare.ReleasePreparationError,
"Source checkout HEAD changed",
):
prepare.materialize(
fixture.repo,
"0.20.0",
release_input.base_commit,
release_input.source_commit,
release_input.worktree,
)
self.assertEqual(
run(
release_input.worktree,
"git",
"symbolic-ref",
"--quiet",
"--short",
"HEAD",
check=False,
).returncode,
1,
)
self.assertNotIn(
"release/v0.20.0",
run(fixture.repo, "git", "branch", "--format=%(refname:short)").stdout.splitlines(),
)
if __name__ == "__main__":
unittest.main()
+23 -17
View File
@@ -1,6 +1,6 @@
---
name: runtime-behavior-probe
description: Plan and execute runtime-behavior investigations with temporary probe scripts, validation matrices, state controls, and findings-first reports. Use only when the user explicitly invokes this skill to verify actual runtime behavior beyond normal code-level checks, especially to uncover edge cases, undocumented behavior, or common failure modes in local or live integrations. A baseline smoke check is fine as an entry point, but do not stop at happy-path confirmation.
description: Plan and, after explicit approval, execute runtime-behavior probes for local or live integrations. Use only when explicitly invoked to verify behavior that code review and normal tests cannot settle; define a controlled validation matrix and report observed evidence.
---
# Runtime Behavior Probe
@@ -12,9 +12,12 @@ Use this skill to investigate real runtime behavior, not to restate code or docu
## Core Rules
- Treat this skill as manual-only. Do not rely on implicit invocation.
- Invoking this skill authorizes planning only. Every runtime probe requires explicit user approval after the exact probe has been proposed. Do not infer execution approval from the skill invocation or a general request to investigate runtime behavior.
- Before requesting approval, disclose the source identity, exact command, transitively executed material, known filesystem, environment, network, and host-service capabilities, expected side effects, and control for the proposed probe. Mark unknown capabilities as unknown rather than assuming that they are unavailable.
- Wait for an affirmative response before executing the probe. Approval is bound to the disclosed source, command, executed material, and capability scope. Obtain new approval before changing any of those fields, adding another probe, or expanding the approved matrix.
- A baseline success or smoke case is often the right entry point, but do not stop there when the real question involves edge cases, drift, or failure behavior.
- Plan before running anything. Write the case matrix first, then fill it in with observed results. The matrix can live in a scratch note, a temporary file, or the probe script header.
- Default to local or read-only probes. Consider a live service only when it is clearly relevant, then apply the lightweight gates below before you run it.
- Default to proposing local or read-only probes. Consider a live service only when it is clearly relevant, then apply the lightweight gates below before requesting approval.
- Size the probe to the decision. Start with the smallest matrix that can disqualify or validate the current hypothesis, then expand only when uncertainty remains.
- Before a live probe, apply three lightweight gates:
- Destination gate. Use only a live destination that is clearly allowed for the task.
@@ -24,6 +27,7 @@ Use this skill to investigate real runtime behavior, not to restate code or docu
- Use temporary files or a temporary directory for one-off probe scripts.
- Keep temporary artifacts until the final response is drafted. Then delete them by default unless the user asked to keep them or they are needed for follow-up. Even when artifacts are deleted, keep a short run summary of the command shape, runtime context, and artifact status in the report.
- Before executing a live probe that will read environment variables, tell the user the exact variable names you plan to use and why, then wait for explicit approval. Examples include `OPENAI_API_KEY` and other expected default names for the system under test.
- When the environment-variable approval gate is required and the `request_user_input` tool is available, use that tool instead of a plain-text approval question. Ask one concise question with mutually exclusive choices such as `Allow once (Recommended)` and `Do not allow`, omit `autoResolutionMs`, and make the approval single-probe and limited to the exact named variables and destination. If the tool is unavailable, fall back to a concise plain-text approval question and do not proceed until the user explicitly approves.
- Never print secrets, even when they come from standard environment variables that this skill may use.
- For OpenAI API or OpenAI platform probes in this repository, use [$openai-knowledge](../openai-knowledge/SKILL.md) early to confirm contract-sensitive details such as supported parameters, field names, and limits. Use runtime probing to validate or challenge the documented behavior, not to skip the documentation pass entirely. If the docs MCP is unavailable, fall back to the official OpenAI docs and say that you used the fallback in the report.
- For benchmark or comparison probes, make parity explicit before execution. Record what is held constant, what variable is under test, which response-shape constraints keep the comparison fair, and any usage or token counters that matter for interpreting latency or cost.
@@ -36,8 +40,9 @@ Use this skill to investigate real runtime behavior, not to restate code or docu
1. Restate the investigation target in operational terms. Name the runtime surface, the key uncertainty, and the highest-risk behaviors to test.
2. Do a short preflight. Check the relevant code or docs first, decide whether the question needs local or live validation, and note any repo, baseline, or release boundary that matters.
3. Create a validation matrix before executing probes. Cover both baseline behavior and the most relevant failure or drift cases. The matrix can live in a scratch note, a temporary file, or a structured header inside the probe script.
4. For each case, choose an execution mode up front:
3. Define the decision signal before building the matrix. For a suspected defect, name the exact user-visible symptom, the command or probe that can distinguish it from correct behavior, the expected failing observation, and a known-good control. Confirm that the signal exercises the real producer and caller path rather than only an adjacent helper. Prefer a fast, deterministic local loop when one can answer the question. If no credible signal can be built, state the missing access or artifact and do not substitute a nearby behavior as proof.
4. Create a validation matrix before executing probes. Cover both baseline behavior and the most relevant failure or drift cases. The matrix can live in a scratch note, a temporary file, or a structured header inside the probe script.
5. For each case, choose an execution mode up front:
- `single-shot` for deterministic one-run checks.
- `repeat-N` for cache, retry, streaming, interruption, rate-limit, concurrency, or other run-to-run-sensitive behavior.
- `warm-up + repeat-N` when first-run cold-start effects could distort the result.
@@ -46,22 +51,23 @@ Use this skill to investigate real runtime behavior, not to restate code or docu
- Decision-grade latency or release recommendation: `warm-up + repeat-10`.
- Costly live cases: start at `repeat-3`, then expand only if the answer remains unclear.
If it is genuinely unclear whether extra runs are worth the time or cost, ask the user before expanding the probe.
5. When the question is benchmark-like or comparative, run in phases. Start with a high-signal pilot matrix against a control, then expand only the surviving candidates or unresolved cases.
6. If the question is about a suspected regression or behavior change, add at least one known-good control case such as `origin/main`, the latest release, or the same request without the suspected option.
7. For comparative probes, define parity before execution. Record prompt or input shape, tool-choice setup, model-settings parity, state reuse rules, and any response-shape constraint that keeps the comparison fair. If materially different output length could bias the result, record usage or token notes too.
8. If the question asks whether one option has the same intelligence or quality as another, decide whether the matrix supports only example-pattern parity or a broader quality claim. For broader claims, add at least one harder or more open-ended case. Otherwise say explicitly that the result is limited to the covered patterns.
9. Plan state controls before execution when hidden state could affect the result. Record whether each case uses fresh or reused state, how cache reuse or cache busting is handled, what unique IDs isolate repeated runs, and how cleanup is verified.
10. If any live case will read environment variables, list the exact variable names and purpose for each case, then ask the user for approval before execution. Keep the approval ask short and include destination, read-only versus mutating or costly risk, exact variable names, and cleanup or rollback if relevant.
11. Build task-specific probe scripts in a temporary location. Keep the script small, observable, and easy to discard.
12. In `openai-agents-python`, make the runtime context explicit:
6. When the question is benchmark-like or comparative, run in phases. Start with a high-signal pilot matrix against a control, then expand only the surviving candidates or unresolved cases.
7. If the question is about a suspected regression or behavior change, add at least one known-good control case such as `origin/main`, the latest release, or the same request without the suspected option.
8. For comparative probes, define parity before execution. Record prompt or input shape, tool-choice setup, model-settings parity, state reuse rules, and any response-shape constraint that keeps the comparison fair. If materially different output length could bias the result, record usage or token notes too.
9. If the question asks whether one option has the same intelligence or quality as another, decide whether the matrix supports only example-pattern parity or a broader quality claim. For broader claims, add at least one harder or more open-ended case. Otherwise say explicitly that the result is limited to the covered patterns.
10. Plan state controls before execution when hidden state could affect the result. Record whether each case uses fresh or reused state, how cache reuse or cache busting is handled, what unique IDs isolate repeated runs, and how cleanup is verified.
11. If any live case will read environment variables, list the exact variable names and purpose for each case, then ask the user for approval before execution. Prefer `request_user_input` for this gate when it is available, with no auto-resolution and choices that grant or deny only this specific probe. Keep the approval ask short and include destination, read-only versus mutating or costly risk, exact variable names, and cleanup or rollback if relevant.
12. Build task-specific probe scripts in a temporary location. Keep the script small, observable, and easy to discard.
13. In `openai-agents-python`, make the runtime context explicit:
- Run Python probes from the repository root with `uv run python` when practical.
- Record the current commit, working directory, Python executable, and Python version.
- Avoid accidental imports from a different checkout or site-packages location. If you must deviate from `uv run python`, say exactly why and what interpreter or environment was used instead.
13. Execute the matrix and capture evidence. Record request shape, setup, observation summary, unexpected or negative result, error details, timing, runtime context, approved environment-variable names, repeat counts, warm-up handling, variance when relevant, cleanup behavior, and for comparisons note what was held constant plus any response-shape or usage notes that affect interpretation.
14. Update the matrix with actual outcomes, not guesses.
15. Keep temporary artifacts until the final response is drafted. Then delete them unless the user asked to keep them or they are needed for follow-up. Benchmark and repeat-heavy probes often need follow-up, so keeping artifacts is normal when the result may be revisited. If deleted, retain and report a short run summary.
16. Report findings first, with unexpected or negative findings first. Then summarize how the validation was performed and which cases were covered.
17. If the probe isolates one clear defect, you may include a short implementation hypothesis or minimal repro direction. Do not expand into a larger next-step plan unless the user asked for it.
14. Present the complete probe proposal with the disclosures required above, including the exact command for each case or approved matrix, then ask the user for explicit approval and wait.
15. Execute only the approved matrix and capture evidence. Record request shape, setup, observation summary, unexpected or negative result, error details, timing, runtime context, approved environment-variable names, repeat counts, warm-up handling, variance when relevant, cleanup behavior, and for comparisons note what was held constant plus any response-shape or usage notes that affect interpretation.
16. Update the matrix with actual outcomes, not guesses.
17. Keep temporary artifacts until the final response is drafted. Then delete them unless the user asked to keep them or they are needed for follow-up. Benchmark and repeat-heavy probes often need follow-up, so keeping artifacts is normal when the result may be revisited. If deleted, retain and report a short run summary.
18. Report findings first, with unexpected or negative findings first. Then summarize how the validation was performed and which cases were covered.
19. If the probe isolates one clear defect, you may include a short implementation hypothesis or minimal repro direction. Do not expand into a larger next-step plan unless the user asked for it.
## Validation Matrix
@@ -1,6 +1,6 @@
interface:
display_name: "Runtime Behavior Probe"
short_description: "Plan and run runtime behavior probes"
default_prompt: "Use $runtime-behavior-probe to investigate actual runtime behavior with a validation matrix, explicit state controls, and a findings-first report."
default_prompt: "Use $runtime-behavior-probe to plan this runtime investigation; invocation authorizes planning only, so disclose the exact probe and capabilities and obtain explicit approval before execution, then report only observed evidence from the approved scope."
policy:
allow_implicit_invocation: false
@@ -27,6 +27,17 @@ Do not read these variables automatically. Before a live probe uses any of them,
If the task targets another standard integration, use that integration's expected default variable names under the same rule.
## Environment False Signals
Before attributing a failure to the patch under review, exclude environment and source-selection problems with a control run.
- Confirm the commit and worktree under test. When editable installs, shared environments, `PYTHONPATH`, or generated artifacts can select stale code, verify the imported package path and rebuild before probing.
- Run base and head controls with the same interpreter, dependencies, environment variables, and command shape.
- Treat proxy initialization, sandbox denials, unavailable containers, expired snapshots, authentication, quotas, rate limits, service outages, and stale caches as environment conditions until a controlled rerun ties them to the patch.
- Never print proxy URLs or credentials. Change only the minimum in-scope environment or disposable state needed for the control run, and record which variable names or constraints changed.
In the final report, distinguish code failures, unsupported configurations, environment blockers, and inconclusive probes. Do not combine them into one failed-test count.
## Responses API Probe Patterns
For Responses API work, start from the uncertainty instead of from the full feature surface.
@@ -0,0 +1,78 @@
---
name: sensitive-logging-audit
description: Audit and fix sensitive-data exposure through Python runtime logging in openai-agents-python. Use when reviewing logging, print, warnings, stderr, traceback, MCP names, model or tool exceptions, redaction flags, or any diagnostic path that may retain user data.
---
# Sensitive Logging Audit
## Objective
Find candidate output sinks, trace their values manually, fix demonstrated leaks at shared runtime boundaries, and prove redaction with adversarial tests.
The collector is only a syntax-based search aid. It does not resolve Python aliases or control flow, certify policy guards, or prove that an absent candidate is safe.
## Workflow
### 1. Establish the review surface
- Work in the current checkout and preserve unrelated changes.
- Read `src/agents/_debug.py`, `src/agents/logger.py`, and the affected callers.
- Treat exception messages, arguments, tracebacks, causes, contexts, notes, names, URLs, and arbitrary values as potentially sensitive.
- Read [the Python redaction validation matrix](references/redaction-validation.md).
Run the collector tests, then collect candidates:
```bash
uv run python .agents/skills/sensitive-logging-audit/scripts/test_inventory.py
uv run python .agents/skills/sensitive-logging-audit/scripts/inventory_logging.py \
--format json --output /tmp/sensitive-logging-candidates.json
```
The report intentionally contains no `policy`, `safe`, or guard classification.
### 2. Supplement the collector with source search
The collector does not follow assignments such as `emit = logger.error`. Search the source directly and inspect aliases, callbacks, wrappers, and reflective dispatch:
```bash
rg -n '\.(debug|info|warning|warn|error|exception|critical|fatal|log)\b' src/agents
rg -n '\b(print|pprint|pp|warn|warn_explicit|write|writelines|print_exc|print_exception)\b' src/agents
rg -n 'DONT_LOG_(MODEL|TOOL)_DATA|log_(model|tool|model_and_tool)_action' src/agents
```
Do not turn collector coverage or a textual guard into a security conclusion. Trace producers and callers.
### 3. Classify manually
Assign each reviewed path one disposition:
- `model`: model requests, responses, Realtime events, or derived values.
- `tool`: tool arguments, outputs, MCP data, tool events, or derived values.
- `model+tool`: either class may reach the sink.
- `operational`: demonstrated to contain only non-sensitive SDK metadata.
- `intentional-output`: explicitly user-facing output rather than diagnostics.
- `uncertain`: source tracing is incomplete.
Record evidence in the audit report. The script does not validate or inherit dispositions.
### 4. Fix runtime boundaries
Before changing runtime behavior, use `$implementation-strategy`.
- Check the relevant `_debug.DONT_LOG_MODEL_DATA` and `_debug.DONT_LOG_TOOL_DATA` flags before formatting or inspecting sensitive values.
- Redact mixed model/tool values when either flag disables data logging.
- In redacted mode, emit a fixed message and omit sensitive `args`, `extra`, and `exc_info`.
- Build diagnostic-only context lazily so redacted mode never reads it.
- Preserve useful diagnostics when sensitive-data logging is explicitly enabled.
- Keep logging failure from changing fallback, cleanup, event, rejection, or cancellation behavior.
- For MCP URLs, remove credentials, query parameters, and fragments in diagnostic mode; never use sanitized names as a substitute for fixed redacted messages.
### 5. Prove caller behavior
Add tests at every changed caller boundary. Inspect the complete `LogRecord`, not only rendered text. Test both redacted policies, diagnostic mode, hostile objects, exception chains, and the caller's observable fallback or cleanup behavior as applicable.
### 6. Re-run and close out
Re-run the collector, the manual searches, focused tests, and applicable repository gates. Use `$code-change-verification` for runtime or test changes and `$pr-draft-summary` when required.
Report candidate counts as search coverage only. Lead with confirmed leaks fixed, retained intentional output, reviewed uncertainty, and verification results. Never report a clean collector result as proof that no sensitive logging path exists.
@@ -0,0 +1,4 @@
interface:
display_name: "Sensitive Logging Audit"
short_description: "Audit and fix sensitive Python logging paths"
default_prompt: "Use $sensitive-logging-audit to inventory, verify, and fix sensitive logging leaks in this repository."
@@ -0,0 +1,63 @@
# Python sensitive logging validation
The collector reports syntactic logging and raw-output candidates. It does not resolve aliases, prove receiver types, evaluate guards, classify payloads, or support a completeness claim. Review candidates together with direct source searches and runtime tests.
## Required validation matrix
Test every changed sensitive caller boundary in both redacted and diagnostic modes. Use a unique sentinel for each source and inspect both rendered output and the complete `LogRecord`.
| Case | Model flag | Tool flag | Value | Required assertion |
| --- | --- | --- | --- | --- |
| Model redaction | on | off | `Exception(secret)` | No sentinel or exception object remains in the record |
| Tool redaction | off | on | `Exception(secret)` | No sentinel or exception object remains in the record |
| Both redacted | on | on | model and tool values | Neither sentinel remains anywhere in the record |
| Diagnostic mode | off | off | ordinary exception | Existing diagnostic detail and traceback behavior remain |
| Hostile string | applicable | applicable | object whose `__str__` raises or returns a secret | Logging does not fail or reveal the secret |
| Hostile repr | applicable | applicable | object whose `__repr__` raises or returns a secret | Logging does not fail or reveal the secret |
| Hostile class access | applicable | applicable | exception overriding `__getattribute__` | Redacted logging does not inspect the exception |
| Exception chain | applicable | applicable | `__cause__`, `__context__`, notes, or `ExceptionGroup` containing secrets | No chained secret is attached or rendered |
| Supplemental arguments | applicable | applicable | fixed message plus secret formatting argument | Formatting arguments are omitted in redacted mode |
| Extra payload | applicable | applicable | `extra={"detail": secret}` | Secret `LogRecord` attributes are omitted |
| Traceback payload | applicable | applicable | `exc_info=True` or an exception tuple | `exc_info` and `exc_text` are absent in redacted mode |
| MCP server or tool name | tool | on | path token or custom-name sentinel | Log uses a fixed message and does not read or attach the name |
| URL-derived MCP name | tool | off | URL credentials, query, and fragment | Log retains only scheme, host, port, and path; the runtime value is unchanged |
Also test the observable caller behavior after logging. Redaction is incorrect if it prevents a fallback result, cleanup, event emission, rejection, or cancellation from completing.
## Inspect the full LogRecord
Do not assert only against `caplog.text` or a mock call converted to a string. In redacted mode, inspect at least:
- `record.msg`
- `record.args`
- `record.exc_info`
- `record.exc_text`
- values added through `record.__dict__`
- the final output of a real `logging.Formatter`
The sensitive object itself must not remain attached even when its string representation is absent. A custom handler or exporter may inspect raw record fields.
## Review procedure
1. Run the collector against all of `src/agents`.
2. Run the supplemental `rg` searches from `SKILL.md` and inspect aliases and dynamic dispatch.
3. Review raw output and ambiguous receivers first.
4. Review caught values, `logger.exception`, `exc_info`, `extra`, and formatting arguments.
5. Trace model, tool, Realtime, MCP, session, sandbox, voice, tracing, and cleanup values to their producers.
6. Classify intentional output separately from diagnostics; do not silently exempt `print` or warnings.
7. Add focused tests at every changed caller boundary.
8. Re-run the collector and source searches after the fix.
An empty or unchanged collector report is not proof of safety. Assignment aliases, monkey-patched methods, dynamically installed handlers, non-constant reflection, and arbitrary runtime data flow require manual inspection.
## Audit report expectations
For each confirmed or uncertain path, record:
- The source location and value producer.
- The manual disposition: `model`, `tool`, `model+tool`, `operational`, `intentional-output`, or `uncertain`.
- Concrete evidence for the disposition.
- The fix or reason for retaining the path.
- The caller-level regression test, when behavior changed.
Do not reuse a disposition solely because a fingerprint or call text is unchanged.
@@ -0,0 +1,360 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import ast
import hashlib
import json
import re
import sys
from collections import Counter
from collections.abc import Iterable, Mapping, Sequence
from dataclasses import asdict, dataclass
from pathlib import Path
from typing import Any
LOG_METHODS = {
"critical",
"debug",
"error",
"exception",
"fatal",
"info",
"log",
"warn",
"warning",
}
POLICY_HELPERS = {
"log_model_action_debug",
"log_model_action_error",
"log_model_action_warning",
"log_model_and_tool_action_debug",
"log_model_and_tool_action_error",
"log_model_and_tool_action_warning",
"log_tool_action_debug",
"log_tool_action_error",
"log_tool_action_warning",
}
RAW_OUTPUT_METHODS = {
"pp",
"pprint",
"print",
"print_exc",
"print_exception",
"warn",
"warn_explicit",
"write",
"writelines",
}
CALLBACK_KEYWORDS = {"callback", "handler"}
@dataclass(frozen=True)
class Candidate:
fingerprint: str
file: str
line: int
column: int
kind: str
method: str
context: str
call: str
reason: str
def to_dict(self) -> dict[str, Any]:
return asdict(self)
def normalize_path(path: str | Path) -> str:
return str(path).replace("\\", "/")
def collect_source_files(roots: Sequence[str | Path]) -> list[Path]:
files: set[Path] = set()
for root_value in roots:
root = Path(root_value).resolve()
if root.is_file():
if root.suffix == ".py":
files.add(root)
continue
if not root.is_dir():
raise FileNotFoundError(f"Inventory root does not exist: {root_value}")
for path in root.rglob("*.py"):
relative_parts = path.relative_to(root).parts
if any(part.startswith(".") or part == "__pycache__" for part in relative_parts):
continue
files.add(path.resolve())
return sorted(files)
def normalize_node(node: ast.AST, source: str) -> str:
segment = ast.get_source_segment(source, node)
if segment is None:
segment = ast.dump(node, annotate_fields=True, include_attributes=False)
return re.sub(r"\s+", " ", segment).strip()
def dotted_name(node: ast.AST) -> str | None:
if isinstance(node, ast.Name):
return node.id
if isinstance(node, ast.Attribute):
receiver = dotted_name(node.value)
return f"{receiver}.{node.attr}" if receiver else node.attr
return None
def terminal_name(node: ast.AST) -> str | None:
name = dotted_name(node)
return name.rsplit(".", 1)[-1] if name else None
def make_parent_map(tree: ast.AST) -> dict[ast.AST, ast.AST]:
return {child: parent for parent in ast.walk(tree) for child in ast.iter_child_nodes(parent)}
def scope_context(node: ast.AST, parents: Mapping[ast.AST, ast.AST]) -> str:
parts: list[str] = []
current = parents.get(node)
while current is not None:
if isinstance(current, ast.ClassDef):
parts.append(f"class:{current.name}")
elif isinstance(current, ast.FunctionDef | ast.AsyncFunctionDef):
parts.append(f"function:{current.name}")
elif isinstance(current, ast.Lambda):
parts.append("lambda")
current = parents.get(current)
return ">".join(reversed(parts)) or "<module>"
def callback_arguments(call: ast.Call) -> Iterable[tuple[ast.AST, str | None]]:
yield from ((argument, None) for argument in call.args)
yield from (
(keyword.value, keyword.arg) for keyword in call.keywords if keyword.arg is not None
)
def looks_like_callback(node: ast.AST, keyword: str | None) -> bool:
method = terminal_name(node)
if method not in LOG_METHODS:
return False
if keyword is not None and (
keyword.startswith("on_")
or keyword.endswith(("_callback", "_handler"))
or keyword in CALLBACK_KEYWORDS
):
return True
if not isinstance(node, ast.Attribute):
return False
receiver = dotted_name(node.value)
receiver_name = receiver.rsplit(".", 1)[-1].lower() if receiver else ""
return receiver_name in {"log", "logger"} or receiver_name.endswith(("_log", "_logger"))
def selected_getattr_method(call: ast.Call) -> str | None:
if terminal_name(call.func) != "getattr" or len(call.args) < 2:
return None
attribute = call.args[1]
if not isinstance(attribute, ast.Constant) or not isinstance(attribute.value, str):
return None
if attribute.value in LOG_METHODS | RAW_OUTPUT_METHODS:
return attribute.value
return None
def classify_call(call: ast.Call) -> tuple[str, str, str] | None:
qualified_method = dotted_name(call.func)
method = terminal_name(call.func)
if method in POLICY_HELPERS:
return (
"policy-helper-call",
method,
"Known redaction helper; review the caller's data classification and fixed message.",
)
if method in LOG_METHODS and not (
method == "warn" and qualified_method in {"warn", "warnings.warn"}
):
return (
"logging-call-candidate",
method,
"Logging-like method name; inspect the receiver and every attached value.",
)
if method in RAW_OUTPUT_METHODS:
return (
"raw-output-call-candidate",
method,
"Direct-output method name; verify its destination and whether values "
"can be sensitive.",
)
selected = selected_getattr_method(call)
if selected is not None:
return (
"getattr-sink-candidate",
selected,
"Constant getattr selects an output-like method; trace the receiver and later uses.",
)
return None
def inventory_source(source: str, file_path: str = "fixture.py") -> list[Candidate]:
normalized_path = normalize_path(file_path)
tree = ast.parse(source, filename=normalized_path)
parents = make_parent_map(tree)
candidates: list[Candidate] = []
recorded: set[tuple[int, str, str]] = set()
def record(node: ast.AST, kind: str, method: str, call: str, reason: str) -> None:
key = (id(node), kind, method)
if key in recorded:
return
recorded.add(key)
line = getattr(node, "lineno", 1)
column = getattr(node, "col_offset", 0) + 1
context = scope_context(node, parents)
fingerprint = hashlib.sha256(
f"{normalized_path}\0{line}\0{column}\0{kind}\0{method}\0{call}".encode()
).hexdigest()[:12]
candidates.append(
Candidate(
fingerprint=fingerprint,
file=normalized_path,
line=line,
column=column,
kind=kind,
method=method,
context=context,
call=call,
reason=reason,
)
)
for node in ast.walk(tree):
if not isinstance(node, ast.Call):
continue
classification = classify_call(node)
if classification is not None:
kind, method, reason = classification
record(node, kind, method, normalize_node(node, source), reason)
for argument, keyword in callback_arguments(node):
if not looks_like_callback(argument, keyword):
continue
method = terminal_name(argument)
if method is None:
continue
record(
argument,
"logging-callback-candidate",
method,
normalize_node(argument, source),
"Logging-like callable passed to a callback-shaped argument; inspect "
"registration and payloads.",
)
candidates.sort(key=lambda item: (item.file, item.line, item.column, item.kind, item.method))
return candidates
def summarize(candidates: Sequence[Candidate]) -> dict[str, int]:
kinds = Counter(candidate.kind for candidate in candidates)
return {
"totalCandidates": len(candidates),
"loggingCalls": kinds["logging-call-candidate"],
"rawOutputCalls": kinds["raw-output-call-candidate"],
"policyHelperCalls": kinds["policy-helper-call"],
"getattrSelections": kinds["getattr-sink-candidate"],
"callbackReferences": kinds["logging-callback-candidate"],
}
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=(
"Collect syntactic Python logging and raw-output candidates for manual review."
)
)
parser.add_argument("roots", nargs="*", default=["src/agents"])
parser.add_argument("--format", choices=("json", "markdown"), default="markdown")
parser.add_argument("--summary-only", action="store_true")
parser.add_argument("--output", type=Path)
return parser.parse_args(argv)
def build_report(args: argparse.Namespace) -> dict[str, Any]:
cwd = Path.cwd().resolve()
candidates: list[Candidate] = []
for path in collect_source_files(args.roots):
try:
display_path = path.relative_to(cwd)
except ValueError:
display_path = path
source = path.read_text(encoding="utf-8")
try:
candidates.extend(inventory_source(source, str(display_path)))
except SyntaxError as error:
raise SyntaxError(
f"Failed to parse {display_path}:{error.lineno}: {error.msg}"
) from error
report: dict[str, Any] = {
"contract": (
"Syntactic candidates only. Manual review and runtime tests are required; "
"absence from this report is not proof of safety."
),
"summary": summarize(candidates),
}
if not args.summary_only:
report["candidates"] = [candidate.to_dict() for candidate in candidates]
return report
def render_markdown(report: Mapping[str, Any], summary_only: bool) -> str:
summary = report["summary"]
lines = [
"# Sensitive logging candidates",
"",
f"> {report['contract']}",
"",
f"- Total candidates: {summary['totalCandidates']}",
f"- Logging calls: {summary['loggingCalls']}",
f"- Raw-output calls: {summary['rawOutputCalls']}",
f"- Policy-helper calls: {summary['policyHelperCalls']}",
f"- Constant getattr selections: {summary['getattrSelections']}",
f"- Callback references: {summary['callbackReferences']}",
]
if not summary_only:
lines.extend(
[
"",
"| Location | Kind | Method | Context | Fingerprint |",
"| --- | --- | --- | --- | --- |",
]
)
for candidate in report.get("candidates", []):
location = f"{candidate['file']}:{candidate['line']}"
lines.append(
f"| {location} | {candidate['kind']} | {candidate['method']} | "
f"{candidate['context']} | {candidate['fingerprint']} |"
)
return "\n".join(lines) + "\n"
def main(argv: Sequence[str] | None = None) -> int:
args = parse_args(argv)
try:
report = build_report(args)
output = (
json.dumps(report, indent=2, sort_keys=True) + "\n"
if args.format == "json"
else render_markdown(report, args.summary_only)
)
if args.output:
args.output.write_text(output, encoding="utf-8")
else:
sys.stdout.write(output)
return 0
except (OSError, SyntaxError, ValueError, json.JSONDecodeError) as error:
print(f"Sensitive logging candidate collection failed: {error}", file=sys.stderr)
return 1
if __name__ == "__main__":
raise SystemExit(main())
@@ -0,0 +1,195 @@
from __future__ import annotations
import unittest
from pathlib import Path
from tempfile import TemporaryDirectory
from inventory_logging import collect_source_files, inventory_source, summarize
class InventoryTests(unittest.TestCase):
def test_collects_direct_logging_calls_without_certifying_receivers(self) -> None:
candidates = inventory_source(
"""
from logging import error
logger.debug("ready")
logger.error("failed: %s", secret)
error(secret)
task.exception()
"""
)
self.assertEqual(
[(item.kind, item.method) for item in candidates],
[
("logging-call-candidate", "debug"),
("logging-call-candidate", "error"),
("logging-call-candidate", "error"),
("logging-call-candidate", "exception"),
],
)
def test_collects_policy_helpers_without_claiming_their_callers_are_safe(self) -> None:
candidates = inventory_source(
"""
from agents.logger import log_model_action_error
log_model_action_error(logger, "failed", error)
agents.logger.log_model_and_tool_action_warning(logger, "failed", error)
"""
)
self.assertEqual(
[(item.kind, item.method) for item in candidates],
[
("policy-helper-call", "log_model_action_error"),
("policy-helper-call", "log_model_and_tool_action_warning"),
],
)
def test_collects_raw_output_method_names(self) -> None:
candidates = inventory_source(
"""
import os
import pprint
import sys
import traceback
import warnings
print(secret)
pprint.pp(secret)
warnings.warn(secret)
sys.stderr.buffer.write(secret_bytes)
sys.stdout.writelines([secret])
traceback.print_exception(error)
os.write(2, secret_bytes)
"""
)
self.assertEqual(
[(item.kind, item.method) for item in candidates],
[
("raw-output-call-candidate", "print"),
("raw-output-call-candidate", "pp"),
("raw-output-call-candidate", "warn"),
("raw-output-call-candidate", "write"),
("raw-output-call-candidate", "writelines"),
("raw-output-call-candidate", "print_exception"),
("raw-output-call-candidate", "write"),
],
)
def test_collects_constant_getattr_sink_selections(self) -> None:
candidates = inventory_source(
"""
emit = getattr(logger, "error")
writer = builtins.getattr(stream, "write")
ignored = getattr(logger, method_name)
"""
)
self.assertEqual(
[(item.kind, item.method) for item in candidates],
[
("getattr-sink-candidate", "error"),
("getattr-sink-candidate", "write"),
],
)
def test_collects_obvious_logging_callbacks(self) -> None:
candidates = inventory_source(
"""
register(log.warning)
register(on_error=service.error)
register(result=request.error)
"""
)
self.assertEqual(
[(item.kind, item.method, item.call) for item in candidates],
[
("logging-callback-candidate", "warning", "log.warning"),
("logging-callback-candidate", "error", "service.error"),
],
)
def test_keeps_the_output_schema_free_of_security_certification(self) -> None:
candidate = inventory_source('logger.error("failed", secret)')[0].to_dict()
self.assertEqual(
set(candidate),
{
"fingerprint",
"file",
"line",
"column",
"kind",
"method",
"context",
"call",
"reason",
},
)
self.assertNotIn("policy", candidate)
self.assertNotIn("safe", candidate)
def test_reports_enclosing_scope_as_review_context(self) -> None:
candidate = inventory_source(
"""
class Worker:
def report(self):
logger.error(secret)
"""
)[0]
self.assertEqual(candidate.context, "class:Worker>function:report")
def test_does_not_claim_to_follow_assignment_aliases(self) -> None:
candidates = inventory_source(
"""
emit = logger.error
emit(secret)
"""
)
self.assertEqual(candidates, [])
def test_summary_counts_only_syntactic_candidate_categories(self) -> None:
candidates = inventory_source(
"""
logger.error(secret)
print(secret)
log_tool_action_error(logger, "failed", error)
register(on_error=service.error)
getattr(logger, "warning")
"""
)
self.assertEqual(
summarize(candidates),
{
"totalCandidates": 5,
"loggingCalls": 1,
"rawOutputCalls": 1,
"policyHelperCalls": 1,
"getattrSelections": 1,
"callbackReferences": 1,
},
)
def test_collect_source_files_filters_hidden_children_relative_to_root(self) -> None:
with TemporaryDirectory(prefix=".hidden-parent-") as directory:
root = Path(directory) / "scan"
root.mkdir()
visible = root / "visible.py"
visible.write_text("print('visible')\n")
hidden = root / ".cache"
hidden.mkdir()
(hidden / "hidden.py").write_text("print('hidden')\n")
self.assertEqual(collect_source_files([root]), [visible.resolve()])
if __name__ == "__main__":
unittest.main()
@@ -39,4 +39,4 @@ Use this skill whenever coverage needs assessment or improvement (coverage regre
- Keep any added comments or code in English.
- Do not create `scripts/`, `references/`, or `assets/` unless needed later.
- If coverage artifacts are missing or stale, rerun `pnpm test:coverage` instead of guessing.
- If coverage artifacts are missing or stale, rerun `make coverage` instead of guessing.
-4
View File
@@ -1,4 +0,0 @@
#:schema https://developers.openai.com/codex/config-schema.json
[features]
codex_hooks = true
-15
View File
@@ -1,15 +0,0 @@
{
"hooks": {
"Stop": [
{
"hooks": [
{
"type": "command",
"command": "uv run python \"$(git rev-parse --show-toplevel)/.codex/hooks/stop_repo_tidy.py\"",
"timeout": 20
}
]
}
]
}
}
-222
View File
@@ -1,222 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import hashlib
import json
import subprocess
import sys
import tempfile
from dataclasses import asdict, dataclass
from pathlib import Path
MAX_RUFF_FIX_FILES = 20
PYTHON_SUFFIXES = {".py", ".pyi"}
@dataclass
class HookState:
last_tidy_fingerprint: str | None = None
def write_stop_block(reason: str, system_message: str) -> None:
sys.stdout.write(
json.dumps(
{
"decision": "block",
"reason": reason,
"systemMessage": system_message,
}
)
)
def run_command(cwd: str, *args: str) -> subprocess.CompletedProcess[str]:
try:
return subprocess.run(
args,
cwd=cwd,
capture_output=True,
check=False,
text=True,
)
except FileNotFoundError as exc:
return subprocess.CompletedProcess(args, returncode=127, stdout="", stderr=str(exc))
def run_git(cwd: str, *args: str) -> subprocess.CompletedProcess[str]:
return run_command(cwd, "git", *args)
def git_root(cwd: str) -> str:
result = run_git(cwd, "rev-parse", "--show-toplevel")
if result.returncode != 0:
raise RuntimeError(result.stderr.strip() or "git root lookup failed")
return result.stdout.strip()
def parse_status_paths(repo_root: str) -> list[str]:
unstaged = run_git(repo_root, "diff", "--name-only", "--diff-filter=ACMR")
untracked = run_git(repo_root, "ls-files", "--others", "--exclude-standard")
if unstaged.returncode != 0 or untracked.returncode != 0:
return []
paths = {
line.strip()
for result in (unstaged, untracked)
for line in result.stdout.splitlines()
if line.strip()
}
return sorted(paths)
def untracked_paths(repo_root: str, paths: list[str]) -> set[str]:
if not paths:
return set()
result = run_git(repo_root, "ls-files", "--others", "--exclude-standard", "--", *paths)
if result.returncode != 0:
return set()
return {line.strip() for line in result.stdout.splitlines() if line.strip()}
def fingerprint_for_paths(repo_root: str, paths: list[str]) -> str | None:
if not paths:
return None
repo_root_path = Path(repo_root)
untracked = untracked_paths(repo_root, paths)
tracked_paths = [file_path for file_path in paths if file_path not in untracked]
diff_parts: list[str] = []
if tracked_paths:
diff = run_git(repo_root, "diff", "--no-ext-diff", "--binary", "--", *tracked_paths)
if diff.returncode == 0:
diff_parts.append(diff.stdout)
for file_path in sorted(untracked):
try:
digest = hashlib.sha256((repo_root_path / file_path).read_bytes()).hexdigest()
except OSError:
continue
diff_parts.append(f"untracked:{file_path}:{digest}")
if not diff_parts:
return None
return hashlib.sha256("\n".join(diff_parts).encode("utf-8")).hexdigest()
def state_dir() -> Path:
return Path(tempfile.gettempdir()) / "openai-agents-python-codex-hooks"
def state_path(session_id: str, repo_root: str) -> Path:
root_hash = hashlib.sha256(repo_root.encode("utf-8")).hexdigest()[:12]
safe_session_id = "".join(
ch if ch.isascii() and (ch.isalnum() or ch in "._-") else "_" for ch in session_id
)
return state_dir() / f"{safe_session_id}-{root_hash}.json"
def load_state(session_id: str, repo_root: str) -> HookState:
file_path = state_path(session_id, repo_root)
if not file_path.exists():
return HookState()
try:
payload = json.loads(file_path.read_text())
except (OSError, json.JSONDecodeError):
return HookState()
return HookState(last_tidy_fingerprint=payload.get("last_tidy_fingerprint"))
def save_state(session_id: str, repo_root: str, state: HookState) -> None:
file_path = state_path(session_id, repo_root)
file_path.parent.mkdir(parents=True, exist_ok=True)
file_path.write_text(json.dumps(asdict(state), indent=2))
def lint_fix_paths(repo_root: str) -> list[str]:
return [
file_path
for file_path in parse_status_paths(repo_root)
if Path(file_path).suffix in PYTHON_SUFFIXES
]
def main() -> None:
try:
payload = json.loads(sys.stdin.read() or "null")
except json.JSONDecodeError:
return
if not isinstance(payload, dict):
return
session_id = payload.get("session_id")
cwd = payload.get("cwd")
if not isinstance(session_id, str) or not isinstance(cwd, str):
return
if payload.get("stop_hook_active"):
return
repo_root = git_root(cwd)
current_paths = lint_fix_paths(repo_root)
if not current_paths or len(current_paths) > MAX_RUFF_FIX_FILES:
return
state = load_state(session_id, repo_root)
current_fingerprint = fingerprint_for_paths(repo_root, current_paths)
if current_fingerprint is None or state.last_tidy_fingerprint == current_fingerprint:
return
format_result = run_command(repo_root, "uv", "run", "ruff", "format", "--", *current_paths)
check_result: subprocess.CompletedProcess[str] | None = None
if format_result.returncode == 0:
check_result = run_command(
repo_root,
"uv",
"run",
"ruff",
"check",
"--fix",
"--",
*current_paths,
)
if format_result.returncode != 0:
write_stop_block(
"`uv run ruff format -- ...` failed for the touched Python files. "
"Review the formatting step before wrapping up.",
"Repo hook: targeted Ruff format failed.",
)
return
if check_result and check_result.returncode != 0:
write_stop_block(
"`uv run ruff check --fix -- ...` failed for the touched Python files. "
"Review the lint output before wrapping up.",
"Repo hook: targeted Ruff lint fix failed.",
)
return
updated_paths = lint_fix_paths(repo_root)
updated_fingerprint = fingerprint_for_paths(repo_root, updated_paths)
state.last_tidy_fingerprint = updated_fingerprint
save_state(session_id, repo_root, state)
if updated_fingerprint != current_fingerprint:
write_stop_block(
"I ran targeted tidy steps on the touched Python files "
"(`ruff format` and `ruff check --fix`). Review the updated diff, "
"then continue or wrap up.",
"Repo hook: ran targeted Ruff tidy on touched files.",
)
if __name__ == "__main__":
main()
+2
View File
@@ -0,0 +1,2 @@
# Keep checkouts as LF so scripts and exact-content checks stay stable on Windows.
* text=auto eol=lf
+32 -7
View File
@@ -2,27 +2,52 @@
name: Bug report
about: Report a bug
title: ''
labels: bug
labels: ''
assignees: ''
---
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you read the docs?** [Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have faced similar issues.
### Describe the bug
A clear and concise description of what the bug is.
<!-- Clearly and concisely describe the bug. -->
### Debug information
- Agents SDK version: (e.g. `v0.0.3`)
- Python version (e.g. Python 3.14)
- Agents SDK version:
- Related library versions (optional, e.g. `any-llm`, `litellm`, or `pydantic`):
- Python version:
- Operating system:
- Model and model provider:
- Does the issue reproduce with the latest Agents SDK release?
- Does the issue occur consistently or intermittently?
If an error occurred, include the full traceback and any relevant logs. Remove API keys, tokens, model input or output, and other sensitive information before posting.
<!-- Paste the traceback or relevant logs below. -->
```text
```
### Repro steps
Ideally provide a minimal python script that can be run to reproduce the bug.
Ideally provide a minimal, self-contained Python script that can be run to reproduce the bug.
```python
from agents import Agent, Runner
agent = Agent(
name="Example agent",
instructions="...",
# Add the model and any other settings needed to reproduce the bug.
)
result = Runner.run_sync(agent, "...")
print(result.final_output)
```
### Expected behavior
A clear and concise description of what you expected to happen.
<!-- Clearly and concisely describe what you expected to happen. -->
+2 -2
View File
@@ -9,8 +9,8 @@ assignees: ''
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you read the docs?** [Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have had similar requests
### Describe the feature
What is the feature you're requesting? How would it work? Please provide examples and details if possible.
<!-- What feature are you requesting? How would it work? Please provide examples and details if possible. -->
+35 -7
View File
@@ -2,25 +2,53 @@
name: Custom model providers
about: Questions or bugs about using non-OpenAI models
title: ''
labels: bug
labels: ''
assignees: ''
---
### Please read this first
- **Have you read the custom model provider docs, including the 'Common issues' section?** [Model provider docs](https://openai.github.io/openai-agents-python/models/#using-other-llm-providers)
- **Have you read the custom model provider docs, including the troubleshooting section?** [Model provider docs](https://openai.github.io/openai-agents-python/models/#non-openai-models)
- **Have you searched for related issues?** Others may have faced similar issues.
### Describe the question
A clear and concise description of what the question or bug is.
<!-- Clearly and concisely describe the question or bug. -->
### Debug information
- Agents SDK version: (e.g. `v0.0.3`)
- Python version (e.g. Python 3.14)
- Agents SDK version:
- Related library versions (optional, e.g. `any-llm`, `litellm`, or `pydantic`):
- Python version:
- Operating system:
- Model and model provider:
- Integration method (e.g. Any-LLM, LiteLLM, custom `ModelProvider`, or direct `Model` implementation):
- Does the issue reproduce with the latest Agents SDK release?
- Does the issue occur consistently or intermittently?
If an error occurred, include the full traceback and any relevant logs. Remove API keys, tokens, model input or output, and other sensitive information before posting.
<!-- Paste the traceback or relevant logs below. -->
```text
```
### Repro steps
Ideally provide a minimal python script that can be run to reproduce the issue.
Ideally provide a minimal, self-contained Python script that can be run to reproduce the issue.
```python
from agents import Agent, Runner
agent = Agent(
name="Example agent",
instructions="...",
# Add the model provider, model, and any other settings needed to reproduce the issue.
)
result = Runner.run_sync(agent, "...")
print(result.final_output)
```
### Expected behavior
A clear and concise description of what you expected to happen.
<!-- Clearly and concisely describe what you expected to happen. -->
+2 -2
View File
@@ -9,8 +9,8 @@ assignees: ''
### Please read this first
- **Have you read the docs?**[Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you read the docs?** [Agents SDK docs](https://openai.github.io/openai-agents-python/)
- **Have you searched for related issues?** Others may have had similar requests
### Question
Describe your question. Provide details if available.
<!-- Describe your question. Provide details if available. -->
@@ -5,6 +5,7 @@
### Test plan
<!-- Please explain how this was tested -->
<!-- If verification could not complete because of local environment setup, include the failing command, the missing dependency, and why it is unrelated to this PR. Leave the pass checkbox below unchecked until all verification steps pass. -->
### Issue number
@@ -12,7 +13,7 @@
### Checks
- [ ] I've added new tests (if relevant)
- [ ] I've added/updated the relevant documentation
- [ ] I've run `make lint` and `make format`
- [ ] I've made sure tests pass
- [ ] I've added new tests, if relevant
- [ ] I've run `.agents/skills/code-change-verification/scripts/run.sh`
- [ ] I've confirmed all verification steps pass
- [ ] If using Codex, I've run `/review` before submitting this PR
-73
View File
@@ -1,73 +0,0 @@
# PR auto-labeling
You are Codex running in CI to propose labels for a pull request in the openai-agents-python repository.
Inputs:
- PR context: .tmp/pr-labels/pr-context.json
- PR diff: .tmp/pr-labels/changes.diff
- Changed files: .tmp/pr-labels/changed-files.txt
Task:
- Inspect the PR context, diff, and changed files.
- Output JSON with a single top-level key: "labels" (array of strings).
- Only use labels from the allowed list.
- Prefer false negatives over false positives. If you are unsure, leave the label out.
- Return the smallest accurate set of labels for the PR's primary intent and primary surface area.
Allowed labels:
- documentation
- project
- bug
- enhancement
- dependencies
- feature:chat-completions
- feature:core
- feature:extensions
- feature:mcp
- feature:realtime
- feature:sandboxes
- feature:sessions
- feature:tracing
- feature:voice
Important guidance:
- `documentation`, `project`, and `dependencies` are also derived deterministically elsewhere in the workflow. You may include them when the evidence is explicit, but do not stretch to infer them from weak signals.
- Use direct evidence from changed implementation files and the dominant intent of the diff. Do not add labels based only on tests, examples, comments, docstrings, imports, type plumbing, or shared helpers.
- Cross-cutting features often touch many adapters and support layers. Only add a `feature:*` label when that area is itself a primary user-facing surface of the PR, not when it receives incidental compatibility or parity updates.
- Mentions of a feature area in helper names, comments, tests, or trace metadata are not enough by themselves.
- Prefer the most general accurate feature label over a larger set of narrower labels. For broad runtime work, this usually means `feature:core`.
- A secondary `feature:*` label needs two things: a non-test implementation/docs change in that area, and evidence that the area is a user-facing outcome of the PR rather than support work for another feature.
Label rules:
- documentation: Documentation changes (docs/), or src/ changes that only modify comments/docstrings without behavior changes. If only comments/docstrings change in src/, do not add bug/enhancement.
- project: Any change to pyproject.toml.
- dependencies: Dependencies are added/removed/updated (pyproject.toml dependency sections or uv.lock changes).
- bug: The PR's primary intent is to correct existing incorrect behavior. Use only with strong evidence such as the title/body/tests clearly describing a fix, regression, crash, incorrect output, or restore/preserve behavior. Do not add `bug` for incidental hardening that accompanies a new feature.
- enhancement: The PR's primary intent is to add or expand functionality. Prefer `enhancement` for feature work even if the diff also contains some fixes or guardrails needed to support that feature.
- bug vs enhancement: Prefer exactly one of these. Include both only when the PR clearly contains two separate substantial changes and both are first-order outcomes.
- feature:chat-completions: Chat Completions support or conversion is a primary deliverable of the PR. Do not add it for a small compatibility guard or parity update in `chatcmpl_converter.py`.
- feature:core: Core agent loop, tool calls, run pipeline, or other central runtime behavior is a primary surface of the PR. For cross-cutting runtime changes, this is usually the single best feature label.
- feature:extensions: `src/agents/extensions/` surfaces are a primary deliverable of the PR, including extension models/providers such as Any-LLM and LiteLLM. Changes under `src/agents/extensions/sandbox/` can warrant this label alongside `feature:sandboxes`.
- feature:mcp: MCP-specific behavior or APIs are a primary deliverable of the PR. Do not add it for incidental hosted/deferred tool plumbing touched by broader runtime work.
- feature:realtime: Realtime-specific behavior, API shape, or session semantics are a primary deliverable of the PR. Do not add it for small parity updates in realtime adapters.
- feature:sandboxes: Sandbox runtime or sandbox extension behavior is a primary deliverable of the PR, including changes under `src/agents/sandbox/` and `src/agents/extensions/sandbox/`. Prefer this over `feature:core` for sandbox-focused work; for `src/agents/extensions/sandbox/`, `feature:extensions` may also be appropriate.
- feature:sessions: Session or memory behavior is a primary deliverable of the PR. Do not add it for persistence updates that merely support a broader feature.
- feature:tracing: Tracing is a primary deliverable of the PR. Do not add it for trace naming or metadata changes that accompany another feature.
- feature:voice: Voice pipeline behavior is a primary deliverable of the PR.
Decision process:
1. Determine the PR's primary intent in one sentence from the PR title/body and dominant runtime diff.
2. Start with zero labels.
3. Add `bug` or `enhancement` conservatively.
4. Add only the minimum `feature:*` labels needed to describe the primary surface area.
5. Treat extra `feature:*` labels as guilty until proven necessary. Keep them only when the PR would feel mislabeled without them.
6. Re-check every label. Drop any label that is supported only by secondary edits, parity work, or touched files outside the PR's main focus.
Examples:
- If a new cross-cutting runtime feature touches Chat Completions, Realtime, Sessions, MCP, and tracing support code for parity, prefer `["enhancement","feature:core"]` over labeling every touched area.
- If a PR mainly adds a Responses/core capability and touches realtime or sessions files only to keep shared serialization, replay, or adapters in sync, do not add `feature:realtime` or `feature:sessions`.
- If a PR mainly fixes realtime transport behavior and also updates tests/docs, prefer `["bug","feature:realtime"]`.
Output:
- JSON only (no code fences, no extra text).
- Example: {"labels":["enhancement","feature:core"]}
-23
View File
@@ -1,23 +0,0 @@
# Release readiness review
You are Codex running in CI. Produce a release readiness report for this repository.
Steps:
1. Determine the latest release tag (use local tags only):
- `git tag -l 'v*' --sort=-v:refname | head -n1`
2. Set TARGET to the current commit SHA: `git rev-parse HEAD`.
3. Collect diff context for BASE_TAG...TARGET:
- `git diff --stat BASE_TAG...TARGET`
- `git diff --dirstat=files,0 BASE_TAG...TARGET`
- `git diff --name-status BASE_TAG...TARGET`
- `git log --oneline --reverse BASE_TAG..TARGET`
4. Review `.agents/skills/final-release-review/references/review-checklist.md` and analyze the diff.
Output:
- Write the report in the exact format used by `$final-release-review` (see `.agents/skills/final-release-review/SKILL.md`).
- Use the compare URL: `https://github.com/${GITHUB_REPOSITORY}/compare/BASE_TAG...TARGET`.
- Include clear ship/block call and risk levels.
- If no risks are found, include "No material risks identified".
Constraints:
- Output only the report (no code fences, no extra commentary).
-29
View File
@@ -1,29 +0,0 @@
{
"type": "object",
"additionalProperties": false,
"required": ["labels"],
"properties": {
"labels": {
"type": "array",
"items": {
"type": "string",
"enum": [
"documentation",
"project",
"bug",
"enhancement",
"dependencies",
"feature:chat-completions",
"feature:core",
"feature:extensions",
"feature:mcp",
"feature:realtime",
"feature:sandboxes",
"feature:sessions",
"feature:tracing",
"feature:voice"
]
}
}
}
}
@@ -0,0 +1,544 @@
from __future__ import annotations
import argparse
import ast
import sys
from collections.abc import Iterable, Mapping
from dataclasses import dataclass, field
from pathlib import Path
from typing import Literal
_CALLABLE_MODULES = {"collections.abc", "typing"}
FunctionNode = ast.FunctionDef | ast.AsyncFunctionDef
ReferenceKind = Literal["callable", "class"]
@dataclass(frozen=True, order=True)
class Violation:
path: Path
line: int
column: int
expression: str
def format(self) -> str:
return (
f"{self.path}:{self.line}:{self.column}: optional object uses truthiness: "
f"{self.expression}"
)
@dataclass
class _ClassInfo:
fields: set[str] = field(default_factory=set)
@dataclass
class _FunctionInfo:
node: FunctionNode
owner: int | None
signature_bindings: dict[str, ReferenceKind]
body_bindings: dict[str, ReferenceKind]
@dataclass
class _ModuleInfo:
path: Path
classes: dict[int, _ClassInfo]
functions: dict[int, _FunctionInfo]
def _walk_scope(body: list[ast.stmt]) -> Iterable[ast.AST]:
stack: list[ast.AST] = list(reversed(body))
while stack:
current = stack.pop()
yield current
if isinstance(
current,
ast.FunctionDef
| ast.AsyncFunctionDef
| ast.ClassDef
| ast.Lambda
| ast.ListComp
| ast.SetComp
| ast.DictComp
| ast.GeneratorExp,
):
continue
stack.extend(reversed(list(ast.iter_child_nodes(current))))
def _walk_comprehension_bindings(nodes: Iterable[ast.AST]) -> Iterable[ast.AST]:
stack = list(reversed(list(nodes)))
while stack:
current = stack.pop()
if isinstance(current, ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef | ast.Lambda):
continue
if isinstance(current, ast.NamedExpr):
yield current.target
stack.append(current.value)
continue
stack.extend(reversed(list(ast.iter_child_nodes(current))))
def _walk_scope_bindings(body: list[ast.stmt]) -> Iterable[ast.AST]:
stack: list[ast.AST] = list(reversed(body))
while stack:
current = stack.pop()
yield current
if isinstance(current, ast.FunctionDef | ast.AsyncFunctionDef):
defining_expressions: list[ast.AST] = [
*current.decorator_list,
current.args,
]
if current.returns is not None:
defining_expressions.append(current.returns)
stack.extend(reversed(defining_expressions))
continue
if isinstance(current, ast.ClassDef):
defining_expressions = [
*current.decorator_list,
*current.bases,
*(keyword.value for keyword in current.keywords),
]
stack.extend(reversed(defining_expressions))
continue
if isinstance(current, ast.Lambda):
stack.append(current.args)
continue
if isinstance(current, ast.ListComp | ast.SetComp | ast.DictComp | ast.GeneratorExp):
first_generator, *remaining_generators = current.generators
stack.append(first_generator.iter)
nested_expressions: list[ast.AST] = [
*(condition for generator in current.generators for condition in generator.ifs),
*(generator.iter for generator in remaining_generators),
]
if isinstance(current, ast.DictComp):
nested_expressions.extend((current.key, current.value))
else:
nested_expressions.append(current.elt)
yield from _walk_comprehension_bindings(nested_expressions)
continue
stack.extend(reversed(list(ast.iter_child_nodes(current))))
def _walk_function(node: FunctionNode) -> Iterable[ast.AST]:
yield from _walk_scope(node.body)
def _is_static_method(function: FunctionNode) -> bool:
return any(
isinstance(decorator, ast.Name)
and decorator.id == "staticmethod"
or isinstance(decorator, ast.Attribute)
and isinstance(decorator.value, ast.Name)
and decorator.value.id == "builtins"
and decorator.attr == "staticmethod"
for decorator in function.decorator_list
)
def _owns_instance_fields(function: FunctionNode) -> bool:
return not _is_static_method(function)
def _is_none_annotation(node: ast.expr) -> bool:
return (
isinstance(node, ast.Constant)
and node.value is None
or isinstance(node, ast.Name)
and node.id == "None"
)
def _optional_payload(annotation: ast.expr) -> ast.expr | None:
if not isinstance(annotation, ast.BinOp) or not isinstance(annotation.op, ast.BitOr):
return None
members: list[ast.expr] = []
def collect(node: ast.expr) -> None:
if isinstance(node, ast.BinOp) and isinstance(node.op, ast.BitOr):
collect(node.left)
collect(node.right)
else:
members.append(node)
collect(annotation)
payloads = [member for member in members if not _is_none_annotation(member)]
if len(payloads) != 1 or len(payloads) == len(members):
return None
return payloads[0]
def _is_direct_optional_reference(
annotation: ast.expr,
*,
bindings: dict[str, ReferenceKind],
) -> bool:
payload = _optional_payload(annotation)
if payload is None:
return False
if (
isinstance(payload, ast.Subscript)
and isinstance(payload.value, ast.Name)
and bindings.get(payload.value.id) == "callable"
):
return True
if not isinstance(payload, ast.Name):
return False
return bindings.get(payload.id) == "class"
def _reference_bindings_for_scope(
body: list[ast.stmt],
inherited_bindings: dict[str, ReferenceKind],
*,
arguments: Iterable[ast.arg] = (),
forced_shadows: Iterable[str] = (),
supported_classes: Mapping[int, str] | None = None,
) -> dict[str, ReferenceKind]:
supported_classes = supported_classes or {}
tracked_names = {*inherited_bindings, *supported_classes.values(), "Callable"}
supported_bindings: dict[str, ReferenceKind] = {}
shadowed_names = {
name
for name in [*(argument.arg for argument in arguments), *forced_shadows]
if name in tracked_names
}
def record_supported(name: str, kind: ReferenceKind) -> None:
existing = supported_bindings.get(name)
if existing is not None:
shadowed_names.add(name)
else:
supported_bindings[name] = kind
for node in _walk_scope_bindings(body):
if isinstance(node, ast.Import | ast.ImportFrom):
for imported in node.names:
if isinstance(node, ast.ImportFrom) and imported.name == "*":
shadowed_names.update(tracked_names)
continue
local_name = imported.asname or imported.name.split(".")[0]
is_standard_callable = (
isinstance(node, ast.ImportFrom)
and node.level == 0
and node.module in _CALLABLE_MODULES
and imported.name == "Callable"
and imported.asname is None
)
if is_standard_callable:
record_supported("Callable", "callable")
elif local_name in tracked_names:
shadowed_names.add(local_name)
elif isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef | ast.ClassDef):
if isinstance(node, ast.ClassDef) and id(node) in supported_classes:
record_supported(node.name, "class")
elif node.name in tracked_names:
shadowed_names.add(node.name)
elif (
isinstance(node, ast.Name)
and isinstance(node.ctx, ast.Store | ast.Del)
and node.id in tracked_names
):
shadowed_names.add(node.id)
elif (
isinstance(node, ast.ExceptHandler)
and isinstance(node.name, str)
and node.name in tracked_names
):
shadowed_names.add(node.name)
elif isinstance(node, ast.MatchAs | ast.MatchStar) and node.name in tracked_names:
shadowed_names.add(node.name)
elif isinstance(node, ast.MatchMapping) and node.rest in tracked_names:
shadowed_names.add(node.rest)
bindings = {
name: kind for name, kind in inherited_bindings.items() if name not in shadowed_names
}
bindings.update(
(name, kind) for name, kind in supported_bindings.items() if name not in shadowed_names
)
return bindings
def _cross_scope_declared_names(tree: ast.Module) -> set[str]:
return {
name
for node in ast.walk(tree)
if isinstance(node, ast.Global | ast.Nonlocal)
for name in node.names
}
def _type_parameter_names(node: FunctionNode | ast.ClassDef) -> set[str]:
return {
name
for type_parameter in getattr(node, "type_params", ())
if isinstance(name := getattr(type_parameter, "name", None), str)
}
def _collect_module_info(path: Path, tree: ast.Module) -> _ModuleInfo:
classes: dict[int, _ClassInfo] = {}
functions: dict[int, _FunctionInfo] = {}
cross_scope_shadows = _cross_scope_declared_names(tree)
def is_simple_reference_class(node: ast.ClassDef) -> bool:
return not node.bases or all(
isinstance(base, ast.Name) and base.id == "object" for base in node.bases
)
supported_classes = {
id(node): node.name
for node in tree.body
if isinstance(node, ast.ClassDef) and is_simple_reference_class(node)
}
module_bindings = _reference_bindings_for_scope(
tree.body,
{},
forced_shadows=cross_scope_shadows,
supported_classes=supported_classes,
)
def register_function(
function: FunctionNode,
*,
signature_bindings: dict[str, ReferenceKind],
inherited_body_bindings: dict[str, ReferenceKind],
owner: int | None,
) -> None:
forced_shadows = cross_scope_shadows | _type_parameter_names(function)
signature_bindings = {
name: kind for name, kind in signature_bindings.items() if name not in forced_shadows
}
body_bindings = _reference_bindings_for_scope(
function.body,
inherited_body_bindings,
arguments=_arguments(function),
forced_shadows=forced_shadows,
)
functions[id(function)] = _FunctionInfo(
node=function,
owner=owner,
signature_bindings=signature_bindings,
body_bindings=body_bindings,
)
if owner is not None and _owns_instance_fields(function):
class_info = classes[owner]
for item in _walk_function(function):
if (
isinstance(item, ast.AnnAssign)
and isinstance(item.target, ast.Attribute)
and isinstance(item.target.value, ast.Name)
and item.target.value.id == "self"
and _is_direct_optional_reference(
item.annotation,
bindings=body_bindings,
)
):
class_info.fields.add(item.target.attr)
for item in _walk_function(function):
if isinstance(item, ast.FunctionDef | ast.AsyncFunctionDef):
register_function(
item,
signature_bindings=body_bindings,
inherited_body_bindings=body_bindings,
owner=None,
)
elif isinstance(item, ast.ClassDef):
register_class(
item,
inherited_body_bindings=body_bindings,
)
def register_class(
node: ast.ClassDef,
*,
inherited_body_bindings: dict[str, ReferenceKind],
) -> None:
class_info = _ClassInfo()
classes[id(node)] = class_info
forced_shadows = cross_scope_shadows | _type_parameter_names(node)
nested_body_bindings = {
name: kind
for name, kind in inherited_body_bindings.items()
if name not in forced_shadows
}
class_bindings = _reference_bindings_for_scope(
node.body,
nested_body_bindings,
forced_shadows=forced_shadows,
)
for item in _walk_scope(node.body):
if isinstance(item, ast.AnnAssign) and isinstance(item.target, ast.Name):
if _is_direct_optional_reference(
item.annotation,
bindings=class_bindings,
):
class_info.fields.add(item.target.id)
elif isinstance(item, ast.FunctionDef | ast.AsyncFunctionDef):
register_function(
item,
signature_bindings=class_bindings,
inherited_body_bindings=nested_body_bindings,
owner=id(node),
)
elif isinstance(item, ast.ClassDef):
register_class(
item,
inherited_body_bindings=nested_body_bindings,
)
for node in _walk_scope(tree.body):
if isinstance(node, ast.FunctionDef | ast.AsyncFunctionDef):
register_function(
node,
signature_bindings=module_bindings,
inherited_body_bindings=module_bindings,
owner=None,
)
elif isinstance(node, ast.ClassDef):
register_class(
node,
inherited_body_bindings=module_bindings,
)
return _ModuleInfo(path, classes, functions)
def _arguments(function: FunctionNode) -> list[ast.arg]:
arguments = [
*function.args.posonlyargs,
*function.args.args,
*function.args.kwonlyargs,
]
if function.args.vararg is not None:
arguments.append(function.args.vararg)
if function.args.kwarg is not None:
arguments.append(function.args.kwarg)
return arguments
def _function_declarations(function: _FunctionInfo) -> set[str]:
declarations = {
argument.arg
for argument in _arguments(function.node)
if argument.annotation is not None
and _is_direct_optional_reference(
argument.annotation,
bindings=function.signature_bindings,
)
}
for node in _walk_function(function.node):
if (
isinstance(node, ast.AnnAssign)
and isinstance(node.target, ast.Name)
and _is_direct_optional_reference(
node.annotation,
bindings=function.body_bindings,
)
):
declarations.add(node.target.id)
return declarations
def _truthiness_atoms(node: ast.expr) -> Iterable[ast.expr]:
if isinstance(node, ast.Name | ast.Attribute):
yield node
elif isinstance(node, ast.NamedExpr):
yield from _truthiness_atoms(node.value)
elif isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
yield from _truthiness_atoms(node.operand)
elif isinstance(node, ast.BoolOp):
for value in node.values:
yield from _truthiness_atoms(value)
def _tested_expressions(function: FunctionNode) -> Iterable[ast.expr]:
for node in _walk_function(function):
if isinstance(node, ast.If | ast.While | ast.Assert | ast.IfExp):
yield from _truthiness_atoms(node.test)
elif isinstance(node, ast.match_case) and node.guard is not None:
yield from _truthiness_atoms(node.guard)
elif isinstance(node, ast.BoolOp):
for value in node.values[:-1]:
yield from _truthiness_atoms(value)
elif isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.Not):
yield from _truthiness_atoms(node.operand)
def _is_declared_reference(
expression: ast.expr,
declarations: set[str],
owner: _ClassInfo | None,
) -> bool:
if isinstance(expression, ast.Name):
return expression.id in declarations
return (
isinstance(expression, ast.Attribute)
and isinstance(expression.value, ast.Name)
and expression.value.id == "self"
and owner is not None
and expression.attr in owner.fields
)
def _find_tree_violations(module: _ModuleInfo) -> list[Violation]:
violations: dict[tuple[int, int], Violation] = {}
for function in module.functions.values():
declarations = _function_declarations(function)
owner = (
module.classes.get(function.owner)
if function.owner is not None and _owns_instance_fields(function.node)
else None
)
for expression in _tested_expressions(function.node):
if not _is_declared_reference(expression, declarations, owner):
continue
violation = Violation(
path=module.path,
line=expression.lineno,
column=expression.col_offset + 1,
expression=ast.unparse(expression),
)
violations[(violation.line, violation.column)] = violation
return sorted(violations.values())
def find_violations(paths: Iterable[Path]) -> list[Violation]:
files = sorted(
{
file
for path in paths
for file in (path.rglob("*.py") if path.is_dir() else [path])
if file.suffix == ".py"
}
)
violations: list[Violation] = []
for path in files:
tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
violations.extend(_find_tree_violations(_collect_module_info(path, tree)))
return sorted(violations)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(
description="Reject truthiness checks on directly declared optional references."
)
parser.add_argument("paths", nargs="+", type=Path)
args = parser.parse_args(argv)
violations = find_violations(args.paths)
for violation in violations:
print(violation.format())
if violations:
print(
"Use an explicit `is None` or `is not None` check so falsy user objects are preserved.",
file=sys.stderr,
)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
+1 -1
View File
@@ -46,7 +46,7 @@ changed_files=$(git diff --name-only "$base_sha" "$head_sha" || true)
case "$mode" in
code)
pattern='^(src/|tests/|examples/|pyproject.toml$|uv.lock$|Makefile$)'
pattern='^(src/|tests/|integration_tests/|examples/|\.agents/skills/(code-change-verification|examples-auto-run|examples-run-analysis|integration-tests)/|\.github/scripts/(detect-changes\.sh|run_examples\.sh|run_integration_tests\.py|update_released_api_contract\.py)$|\.github/workflows/tests\.yml$|pyproject.toml$|uv.lock$|Makefile$)'
;;
docs)
pattern='^(docs/|mkdocs.yml$)'
-442
View File
@@ -1,442 +0,0 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import json
import os
import pathlib
import subprocess
import sys
from collections.abc import Sequence
from dataclasses import dataclass
from typing import Any, Final
ALLOWED_LABELS: Final[set[str]] = {
"documentation",
"project",
"bug",
"enhancement",
"dependencies",
"feature:chat-completions",
"feature:core",
"feature:extensions",
"feature:mcp",
"feature:realtime",
"feature:sandboxes",
"feature:sessions",
"feature:tracing",
"feature:voice",
}
DETERMINISTIC_LABELS: Final[set[str]] = {
"documentation",
"project",
"dependencies",
}
MODEL_ONLY_LABELS: Final[set[str]] = {
"bug",
"enhancement",
}
FEATURE_LABELS: Final[set[str]] = ALLOWED_LABELS - DETERMINISTIC_LABELS - MODEL_ONLY_LABELS
SOURCE_FEATURE_PREFIXES: Final[dict[str, tuple[str, ...]]] = {
"feature:realtime": ("src/agents/realtime/",),
"feature:sandboxes": ("src/agents/sandbox/", "src/agents/extensions/sandbox/"),
"feature:voice": ("src/agents/voice/",),
"feature:mcp": ("src/agents/mcp/",),
"feature:tracing": ("src/agents/tracing/",),
"feature:sessions": ("src/agents/memory/",),
}
CORE_EXCLUDED_PREFIXES: Final[tuple[str, ...]] = (
"src/agents/realtime/",
"src/agents/voice/",
"src/agents/mcp/",
"src/agents/tracing/",
"src/agents/memory/",
"src/agents/extensions/",
"src/agents/models/",
)
PR_CONTEXT_DEFAULT_PATH = ".tmp/pr-labels/pr-context.json"
@dataclass(frozen=True)
class PRContext:
title: str = ""
body: str = ""
def read_file_at(commit: str | None, path: str) -> str | None:
if not commit:
return None
try:
return subprocess.check_output(["git", "show", f"{commit}:{path}"], text=True)
except subprocess.CalledProcessError:
return None
def dependency_lines_for_pyproject(text: str) -> set[int]:
dependency_lines: set[int] = set()
current_section: str | None = None
in_project_dependencies = False
for line_number, raw_line in enumerate(text.splitlines(), start=1):
stripped = raw_line.strip()
if stripped.startswith("[") and stripped.endswith("]"):
if stripped.startswith("[[") and stripped.endswith("]]"):
current_section = stripped[2:-2].strip()
else:
current_section = stripped[1:-1].strip()
in_project_dependencies = False
if current_section in ("project.optional-dependencies", "dependency-groups"):
dependency_lines.add(line_number)
continue
if current_section in ("project.optional-dependencies", "dependency-groups"):
dependency_lines.add(line_number)
continue
if current_section != "project":
continue
if in_project_dependencies:
dependency_lines.add(line_number)
if "]" in stripped:
in_project_dependencies = False
continue
if stripped.startswith("dependencies") and "=" in stripped:
dependency_lines.add(line_number)
if "[" in stripped and "]" not in stripped:
in_project_dependencies = True
return dependency_lines
def pyproject_dependency_changed(
diff_text: str,
*,
base_sha: str | None,
head_sha: str | None,
) -> bool:
import re
base_text = read_file_at(base_sha, "pyproject.toml")
head_text = read_file_at(head_sha, "pyproject.toml")
if base_text is None and head_text is None:
return False
base_dependency_lines = dependency_lines_for_pyproject(base_text) if base_text else set()
head_dependency_lines = dependency_lines_for_pyproject(head_text) if head_text else set()
in_pyproject = False
base_line: int | None = None
head_line: int | None = None
hunk_re = re.compile(r"@@ -(\d+)(?:,\d+)? \+(\d+)(?:,\d+)? @@")
for line in diff_text.splitlines():
if line.startswith("+++ b/"):
current_file = line[len("+++ b/") :].strip()
in_pyproject = current_file == "pyproject.toml"
base_line = None
head_line = None
continue
if not in_pyproject:
continue
if line.startswith("@@ "):
match = hunk_re.match(line)
if not match:
continue
base_line = int(match.group(1))
head_line = int(match.group(2))
continue
if base_line is None or head_line is None:
continue
if line.startswith(" "):
base_line += 1
head_line += 1
continue
if line.startswith("-"):
if base_line in base_dependency_lines:
return True
base_line += 1
continue
if line.startswith("+"):
if head_line in head_dependency_lines:
return True
head_line += 1
continue
return False
def infer_specific_feature_labels(changed_files: Sequence[str]) -> set[str]:
source_files = [path for path in changed_files if path.startswith("src/")]
labels: set[str] = set()
for label, prefixes in SOURCE_FEATURE_PREFIXES.items():
if any(path.startswith(prefix) for path in source_files for prefix in prefixes):
labels.add(label)
if any(path.startswith("src/agents/extensions/") for path in source_files):
labels.add("feature:extensions")
if any(
path.startswith(("src/agents/models/", "src/agents/extensions/models/"))
and ("chatcmpl" in path or "chatcompletions" in path)
for path in source_files
):
labels.add("feature:chat-completions")
return labels
def infer_feature_labels(changed_files: Sequence[str]) -> set[str]:
source_files = [path for path in changed_files if path.startswith("src/")]
specific_labels = infer_specific_feature_labels(source_files)
core_touched = any(
path.startswith("src/agents/") and not path.startswith(CORE_EXCLUDED_PREFIXES)
for path in source_files
)
if core_touched and len(specific_labels) != 1:
return {"feature:core"}
return specific_labels
def infer_fallback_labels(changed_files: Sequence[str]) -> set[str]:
return infer_feature_labels(changed_files)
def load_json(path: pathlib.Path) -> Any:
return json.loads(path.read_text())
def load_pr_context(path: pathlib.Path) -> PRContext:
if not path.exists():
return PRContext()
try:
payload = load_json(path)
except json.JSONDecodeError:
return PRContext()
if not isinstance(payload, dict):
return PRContext()
title = payload.get("title", "")
body = payload.get("body", "")
if not isinstance(title, str):
title = ""
if not isinstance(body, str):
body = ""
return PRContext(title=title, body=body)
def load_codex_labels(path: pathlib.Path) -> tuple[list[str], bool]:
if not path.exists():
return [], False
raw = path.read_text().strip()
if not raw:
return [], False
try:
payload = load_json(path)
except json.JSONDecodeError:
return [], False
if not isinstance(payload, dict):
return [], False
labels = payload.get("labels")
if not isinstance(labels, list):
return [], False
if not all(isinstance(label, str) for label in labels):
return [], False
return list(labels), True
def fetch_existing_labels(pr_number: str) -> set[str]:
result = subprocess.check_output(
["gh", "pr", "view", pr_number, "--json", "labels", "--jq", ".labels[].name"],
text=True,
).strip()
return {label for label in result.splitlines() if label}
def infer_title_intent_labels(pr_context: PRContext) -> set[str]:
normalized_title = pr_context.title.strip().lower()
bug_prefixes = ("fix:", "fix(", "bug:", "bugfix:", "hotfix:", "regression:")
enhancement_prefixes = ("feat:", "feat(", "feature:", "enhancement:")
if normalized_title.startswith(bug_prefixes):
return {"bug"}
if normalized_title.startswith(enhancement_prefixes):
return {"enhancement"}
return set()
def compute_desired_labels(
*,
pr_context: PRContext,
changed_files: Sequence[str],
diff_text: str,
codex_ran: bool,
codex_output_valid: bool,
codex_labels: Sequence[str],
base_sha: str | None,
head_sha: str | None,
) -> set[str]:
desired: set[str] = set()
codex_label_set = {label for label in codex_labels if label in ALLOWED_LABELS}
codex_feature_labels = codex_label_set & FEATURE_LABELS
codex_model_only_labels = codex_label_set & MODEL_ONLY_LABELS
fallback_feature_labels = infer_fallback_labels(changed_files)
title_intent_labels = infer_title_intent_labels(pr_context)
if "pyproject.toml" in changed_files:
desired.add("project")
if any(path.startswith("docs/") for path in changed_files):
desired.add("documentation")
dependencies_allowed = "uv.lock" in changed_files
if "pyproject.toml" in changed_files and pyproject_dependency_changed(
diff_text, base_sha=base_sha, head_sha=head_sha
):
dependencies_allowed = True
if dependencies_allowed:
desired.add("dependencies")
if codex_ran and codex_output_valid and codex_feature_labels:
desired.update(codex_feature_labels)
else:
desired.update(fallback_feature_labels)
if title_intent_labels:
desired.update(title_intent_labels)
elif codex_ran and codex_output_valid:
desired.update(codex_model_only_labels)
if any(path.startswith("src/agents/extensions/sandbox/") for path in changed_files):
desired.update({"feature:extensions", "feature:sandboxes"})
return desired
def compute_managed_labels(
*,
pr_context: PRContext,
codex_ran: bool,
codex_output_valid: bool,
codex_labels: Sequence[str],
) -> set[str]:
managed = DETERMINISTIC_LABELS | FEATURE_LABELS
title_intent_labels = infer_title_intent_labels(pr_context)
codex_label_set = {label for label in codex_labels if label in MODEL_ONLY_LABELS}
if title_intent_labels or (codex_ran and codex_output_valid and codex_label_set):
managed |= MODEL_ONLY_LABELS
return managed
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser()
parser.add_argument("--pr-number", default=os.environ.get("PR_NUMBER", ""))
parser.add_argument("--base-sha", default=os.environ.get("PR_BASE_SHA", ""))
parser.add_argument("--head-sha", default=os.environ.get("PR_HEAD_SHA", ""))
parser.add_argument(
"--codex-output-path",
default=os.environ.get("CODEX_OUTPUT_PATH", ".tmp/codex/outputs/pr-labels.json"),
)
parser.add_argument("--codex-conclusion", default=os.environ.get("CODEX_CONCLUSION", ""))
parser.add_argument(
"--pr-context-path",
default=os.environ.get("PR_CONTEXT_PATH", PR_CONTEXT_DEFAULT_PATH),
)
parser.add_argument(
"--changed-files-path",
default=os.environ.get("CHANGED_FILES_PATH", ".tmp/pr-labels/changed-files.txt"),
)
parser.add_argument(
"--changes-diff-path",
default=os.environ.get("CHANGES_DIFF_PATH", ".tmp/pr-labels/changes.diff"),
)
return parser.parse_args(argv)
def main(argv: Sequence[str] | None = None) -> int:
args = parse_args(argv)
if not args.pr_number:
raise SystemExit("Missing PR number.")
changed_files_path = pathlib.Path(args.changed_files_path)
changes_diff_path = pathlib.Path(args.changes_diff_path)
codex_output_path = pathlib.Path(args.codex_output_path)
pr_context_path = pathlib.Path(args.pr_context_path)
codex_conclusion = args.codex_conclusion.strip().lower()
codex_ran = bool(codex_conclusion) and codex_conclusion != "skipped"
pr_context = load_pr_context(pr_context_path)
changed_files = []
if changed_files_path.exists():
changed_files = [
line.strip() for line in changed_files_path.read_text().splitlines() if line.strip()
]
diff_text = changes_diff_path.read_text() if changes_diff_path.exists() else ""
codex_labels, codex_output_valid = load_codex_labels(codex_output_path)
if codex_ran and not codex_output_valid:
print(
"Codex output missing or invalid; using fallback feature labels and preserving "
"model-only labels."
)
desired = compute_desired_labels(
pr_context=pr_context,
changed_files=changed_files,
diff_text=diff_text,
codex_ran=codex_ran,
codex_output_valid=codex_output_valid,
codex_labels=codex_labels,
base_sha=args.base_sha or None,
head_sha=args.head_sha or None,
)
existing = fetch_existing_labels(args.pr_number)
managed_labels = compute_managed_labels(
pr_context=pr_context,
codex_ran=codex_ran,
codex_output_valid=codex_output_valid,
codex_labels=codex_labels,
)
to_add = sorted(desired - existing)
to_remove = sorted((existing & managed_labels) - desired)
if not to_add and not to_remove:
print("Labels already up to date.")
return 0
cmd = ["gh", "pr", "edit", args.pr_number]
if to_add:
cmd += ["--add-label", ",".join(to_add)]
if to_remove:
cmd += ["--remove-label", ",".join(to_remove)]
subprocess.check_call(cmd)
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -1,10 +1,26 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
PID_FILE="$ROOT/.tmp/examples-auto-run.pid"
LOG_DIR="$ROOT/.tmp/examples-start-logs"
RERUN_FILE="$ROOT/.tmp/examples-rerun.txt"
DEFAULT_UV_EXTRAS="litellm any-llm sqlalchemy redis blaxel modal runloop temporal"
build_uv_prefix() {
UV_RUN=(uv run)
local extras_value
if [[ -n "${EXAMPLES_UV_EXTRAS+x}" ]]; then
extras_value="$EXAMPLES_UV_EXTRAS"
else
extras_value="$DEFAULT_UV_EXTRAS"
fi
local extra
for extra in $extras_value; do
UV_RUN+=(--extra "$extra")
done
export EXAMPLES_UV_EXTRAS="$extras_value"
}
ensure_dirs() {
mkdir -p "$LOG_DIR" "$ROOT/.tmp"
@@ -28,10 +44,10 @@ cmd_start() {
main_log="$LOG_DIR/main_${ts}.log"
stdout_log="$LOG_DIR/stdout_${ts}.log"
build_uv_prefix
local run_cmd=(
uv run examples/run_examples.py
"${UV_RUN[@]}" examples/run_examples.py
--auto-mode
--write-rerun
--main-log "$main_log"
--logs-dir "$LOG_DIR"
)
@@ -63,7 +79,7 @@ cmd_start() {
echo "Started run_examples.py (pid=$pid)"
echo "Main log: $main_log"
echo "Stdout log: $stdout_log"
echo "Run '.agents/skills/examples-auto-run/scripts/run.sh validate \"$main_log\"' after it finishes."
echo "After the run completes, use examples-run-analysis to inspect its artifacts."
return 0
fi
@@ -141,75 +157,31 @@ cmd_tail() {
tail -f "$LOG_DIR/$file"
}
collect_rerun() {
ensure_dirs
local log_file="${1:-}"
if [[ -z "$log_file" ]]; then
log_file="$(ls -1t "$LOG_DIR"/main_*.log 2>/dev/null | head -n1)"
fi
if [[ -z "$log_file" ]] || [[ ! -f "$log_file" ]]; then
echo "No main log file found."
exit 1
fi
cd "$ROOT"
uv run examples/run_examples.py --collect "$log_file" --output "$RERUN_FILE"
}
cmd_rerun() {
ensure_dirs
local file="${1:-$RERUN_FILE}"
if [[ ! -s "$file" ]]; then
echo "Rerun list is empty: $file"
exit 0
fi
local ts main_log stdout_log
ts="$(date +%Y%m%d-%H%M%S)"
main_log="$LOG_DIR/main_${ts}.log"
stdout_log="$LOG_DIR/stdout_${ts}.log"
cd "$ROOT"
export EXAMPLES_INTERACTIVE_MODE="${EXAMPLES_INTERACTIVE_MODE:-auto}"
export APPLY_PATCH_AUTO_APPROVE="${APPLY_PATCH_AUTO_APPROVE:-1}"
export SHELL_AUTO_APPROVE="${SHELL_AUTO_APPROVE:-1}"
export AUTO_APPROVE_MCP="${AUTO_APPROVE_MCP:-1}"
set +e
uv run examples/run_examples.py --auto-mode --rerun-file "$file" --write-rerun --main-log "$main_log" --logs-dir "$LOG_DIR" 2>&1 | tee "$stdout_log"
local run_status=${PIPESTATUS[0]}
set -e
return "$run_status"
}
usage() {
cat <<'EOF'
Usage: run.sh <start|stop|status|logs|tail|collect|rerun> [args...]
Usage: run_examples.sh <start|stop|status|logs|tail> [args...]
Commands:
start [--filter ... | other args] Run examples in auto mode (foreground). Pass --background to run detached.
stop Kill the running auto-run (if any).
status Show whether it is running.
stop Kill the running examples job (if any).
status Show whether an examples job is running.
logs List log files (.tmp/examples-start-logs).
tail [logfile] Tail the latest (or specified) log.
collect [main_log] Parse a main log and write failed examples to .tmp/examples-rerun.txt.
rerun [rerun_file] Run only the examples listed in .tmp/examples-rerun.txt.
Environment overrides:
EXAMPLES_INTERACTIVE_MODE (default auto)
EXAMPLES_INCLUDE_SERVER/INTERACTIVE/AUDIO/EXTERNAL (defaults: 0/1/0/0)
EXAMPLES_UV_EXTRAS (default: litellm any-llm sqlalchemy redis blaxel modal runloop temporal; set empty to disable)
APPLY_PATCH_AUTO_APPROVE, SHELL_AUTO_APPROVE, AUTO_APPROVE_MCP (default 1 in auto mode)
EOF
}
default_cmd="start"
if [[ $# -eq 0 && -s "$RERUN_FILE" ]]; then
default_cmd="rerun"
fi
case "${1:-$default_cmd}" in
case "${1:-start}" in
start) shift || true; cmd_start "$@" ;;
stop) shift || true; cmd_stop ;;
status) shift || true; cmd_status ;;
logs) shift || true; cmd_logs ;;
tail) shift; cmd_tail "${1:-}" ;;
collect) shift || true; collect_rerun "${1:-}" ;;
rerun) shift || true; cmd_rerun "${1:-}" ;;
help | --help | -h) usage ;;
*) usage; exit 1 ;;
esac
+711
View File
@@ -0,0 +1,711 @@
from __future__ import annotations
import argparse
import os
import re
import shutil
import subprocess
import sys
import xml.etree.ElementTree as ET
from collections.abc import Callable, MutableMapping, Sequence
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
WORKSPACE = ROOT / ".tmp" / "integration-tests"
DIST = WORKSPACE / "dist"
RESULTS = WORKSPACE / "results"
TESTS = ROOT / "integration_tests"
CONTRACT_POLICY = ROOT / "tests" / "fixtures" / "released_api_contract_policy.json"
PROSPECTIVE_CONTRACT_ENV = "OPENAI_AGENTS_PROSPECTIVE_RELEASE_CONTRACT"
EXTRAS = "any-llm,litellm,realtime,voice"
OPTIONAL_EXTRAS = (
"any-llm",
"litellm",
"realtime",
"voice",
"sqlalchemy",
"encrypt",
"redis",
"viz",
"s3",
)
STRICT_PROFILES = frozenset({"release", "security"})
LOCAL_ONLY_CREDENTIAL_CLASS = "local-only"
LIVE_CREDENTIAL_CLASS = "live"
PROFILE_CREDENTIAL_CLASSES = {
"packaging": LOCAL_ONLY_CREDENTIAL_CLASS,
"prospective-contract": LOCAL_ONLY_CREDENTIAL_CLASS,
"prospective-platform": LOCAL_ONLY_CREDENTIAL_CLASS,
"security": LOCAL_ONLY_CREDENTIAL_CLASS,
"mcp-v1": LOCAL_ONLY_CREDENTIAL_CLASS,
"extras": LOCAL_ONLY_CREDENTIAL_CLASS,
"core": LIVE_CREDENTIAL_CLASS,
"providers": LIVE_CREDENTIAL_CLASS,
"realtime": LIVE_CREDENTIAL_CLASS,
"voice": LIVE_CREDENTIAL_CLASS,
"hosted": LIVE_CREDENTIAL_CLASS,
"full": LIVE_CREDENTIAL_CLASS,
"release": LIVE_CREDENTIAL_CLASS,
"nightly": LIVE_CREDENTIAL_CLASS,
"manual": LIVE_CREDENTIAL_CLASS,
}
PROFILES = tuple(PROFILE_CREDENTIAL_CLASSES)
BOOTSTRAPPED_ENV = "OPENAI_AGENTS_INTEGRATION_RUNNER_BOOTSTRAPPED"
def parse_args(arguments: Sequence[str] | None = None) -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Run packaged openai-agents integration tests.")
parser.add_argument("--profile", choices=PROFILES, default="full")
parser.add_argument(
"--all",
action="store_true",
help="Include configured direct Anthropic and Gemini providers alongside OpenRouter.",
)
return parser.parse_args(arguments)
def prepare_profile_environment(
profile: str,
environ: MutableMapping[str, str] | None = None,
) -> str:
environment = os.environ if environ is None else environ
try:
credential_class = PROFILE_CREDENTIAL_CLASSES[profile]
except KeyError as error:
raise RuntimeError(f"Integration profile {profile!r} has no credential class.") from error
if credential_class == LIVE_CREDENTIAL_CLASS:
if environment.get("OPENAI_API_KEY_SOURCE") != "service-account":
raise RuntimeError(
f"Live integration profile {profile!r} requires "
"OPENAI_API_KEY_SOURCE=service-account before any build or subprocess starts. "
"Load the approved service-account environment and retry the Make target."
)
elif credential_class == LOCAL_ONLY_CREDENTIAL_CLASS:
environment.pop("OPENAI_API_KEY", None)
else:
raise RuntimeError(
f"Integration profile {profile!r} has unknown credential class {credential_class!r}."
)
return credential_class
def bootstrap_in_uv(
arguments: Sequence[str],
environ: MutableMapping[str, str],
exec_function: Callable[[str, list[str], dict[str, str]], object] = os.execvpe,
) -> None:
args = parse_args(arguments)
prepare_profile_environment(args.profile, environ)
child_env = dict(environ)
child_env[BOOTSTRAPPED_ENV] = "1"
command = ["uv", "run", "python", str(Path(__file__).resolve()), *arguments]
if sys.platform == "win32":
completed = subprocess.run(command, env=child_env, check=False)
raise SystemExit(completed.returncode)
exec_function(command[0], command, child_env)
raise RuntimeError("The uv integration runner bootstrap returned unexpectedly.")
if __name__ == "__main__" and os.environ.get(BOOTSTRAPPED_ENV) != "1":
bootstrap_in_uv(sys.argv[1:], os.environ)
sys.path.insert(0, str(ROOT))
from integration_tests._contract_support import ( # noqa: E402
SubmoduleExportPolicy,
load_submodule_export_policy,
)
def run(command: list[str], *, env: dict[str, str] | None = None) -> None:
print(f"[integration] {' '.join(command)}", flush=True)
subprocess.run(command, cwd=ROOT, env=env, check=True)
def run_pytest(command: list[str], *, env: dict[str, str]) -> tuple[int, str]:
print(f"[integration] {' '.join(command)}", flush=True)
process = subprocess.Popen(
command,
cwd=ROOT,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
)
output: list[str] = []
assert process.stdout is not None
for line in process.stdout:
print(line, end="", flush=True)
output.append(line)
return process.wait(), "".join(output)
def build_distributions() -> tuple[Path, Path]:
shutil.rmtree(DIST, ignore_errors=True)
DIST.mkdir(parents=True, exist_ok=True)
run(["uv", "build", "--out-dir", str(DIST)])
wheels = sorted(DIST.glob("openai_agents-*.whl"), key=lambda path: path.stat().st_mtime)
sdists = sorted(DIST.glob("openai_agents-*.tar.gz"), key=lambda path: path.stat().st_mtime)
if not wheels or not sdists:
raise RuntimeError("uv build did not produce both an openai-agents wheel and sdist.")
return wheels[-1], sdists[-1]
def _any_llm_provider_extras(
*, external_providers_enabled: bool, direct_providers_enabled: bool
) -> list[str]:
provider_extras: set[str] = set()
configured_models = os.environ.get("OPENAI_AGENTS_INTEGRATION_ANY_LLM_MODELS", "")
for model in configured_models.split(","):
provider = model.strip().partition("/")[0]
if provider in {"anthropic", "openrouter"}:
provider_extras.add(provider)
elif provider in {"gemini", "google"}:
provider_extras.add("gemini")
if external_providers_enabled:
if direct_providers_enabled and os.environ.get("ANTHROPIC_API_KEY"):
provider_extras.add("anthropic")
if direct_providers_enabled and (
os.environ.get("GEMINI_API_KEY") or os.environ.get("GOOGLE_API_KEY")
):
provider_extras.add("gemini")
if os.environ.get("OPENROUTER_API_KEY"):
provider_extras.add("openrouter")
return sorted(provider_extras)
def create_environment(
name: str,
distribution: Path,
*,
extras: bool = False,
optional_extra: str | None = None,
additional_requirements: tuple[str, ...] = (),
) -> Path:
environment = WORKSPACE / name
venv_command = ["uv", "venv", "--clear", str(environment)]
if python_version := os.environ.get("OPENAI_AGENTS_INTEGRATION_PYTHON"):
venv_command.extend(["--python", python_version])
run(venv_command)
python = environment / ("Scripts/python.exe" if sys.platform == "win32" else "bin/python")
selected_extra = EXTRAS if extras else optional_extra
requirement = f"{distribution}[{selected_extra}]" if selected_extra else str(distribution)
requirements = [
requirement,
"pytest",
"pytest-asyncio",
"pytest-timeout",
*additional_requirements,
]
external_providers_enabled = os.environ.get(
"OPENAI_AGENTS_INTEGRATION_EXTERNAL_PROVIDERS", ""
).lower() in {"1", "true", "yes"}
direct_providers_enabled = os.environ.get(
"OPENAI_AGENTS_INTEGRATION_DIRECT_PROVIDERS", ""
).lower() in {"1", "true", "yes"}
if extras:
any_llm_extras = _any_llm_provider_extras(
external_providers_enabled=external_providers_enabled,
direct_providers_enabled=direct_providers_enabled,
)
if any_llm_extras:
requirements.append(f"any-llm-sdk[{','.join(any_llm_extras)}]")
proxy_values = [
os.environ.get(name, "")
for name in (
"ALL_PROXY",
"HTTP_PROXY",
"HTTPS_PROXY",
"all_proxy",
"http_proxy",
"https_proxy",
)
]
if any(value.lower().startswith("socks") for value in proxy_values):
requirements.append("httpx[socks]")
run(["uv", "pip", "install", "--python", str(python), *requirements])
return python
def run_suite(
python: Path,
wheel: Path,
sdist: Path,
*,
selection: str,
environment_kind: str,
additional_env: dict[str, str] | None = None,
profile: str,
require_no_skips: bool = False,
) -> None:
child_env = dict(os.environ)
child_env.pop("PYTHONPATH", None)
if child_env.get("OPENAI_AGENTS_INTEGRATION_DISABLE_PROXY", "").lower() in {
"1",
"true",
"yes",
}:
for variable in (
"ALL_PROXY",
"HTTP_PROXY",
"HTTPS_PROXY",
"all_proxy",
"http_proxy",
"https_proxy",
):
child_env.pop(variable, None)
child_env["PYTHONNOUSERSITE"] = "1"
child_env["OPENAI_AGENTS_INTEGRATION_WHEEL"] = str(wheel)
child_env["OPENAI_AGENTS_INTEGRATION_SDIST"] = str(sdist)
child_env["OPENAI_AGENTS_INTEGRATION_ENVIRONMENT"] = environment_kind
if additional_env:
child_env.update(additional_env)
if environment_kind.startswith("extra-"):
child_env["OPENAI_AGENTS_INTEGRATION_EXTRA"] = environment_kind.removeprefix("extra-")
if not os.environ.get("OPENAI_AGENTS_INTEGRATION_ENABLE_TRACING"):
child_env["OPENAI_AGENTS_DISABLE_TRACING"] = "1"
command = [
str(python),
"-I",
"-m",
"pytest",
"-c",
str(TESTS / "pytest.ini"),
str(TESTS),
"-v",
"--tb=short",
"-m",
selection,
]
result_path = RESULTS / profile / f"{environment_kind}.xml"
result_path.parent.mkdir(parents=True, exist_ok=True)
command.append(f"--junitxml={result_path}")
return_code = 1
output = ""
try:
return_code, output = run_pytest(command, env=child_env)
finally:
deselected_matches = re.findall(r"(\d+) deselected", output)
deselected = int(deselected_matches[-1]) if deselected_matches else 0
junit_totals = _print_junit_summary(
profile,
environment_kind,
result_path,
deselected=deselected,
)
if return_code:
raise subprocess.CalledProcessError(return_code, command)
if junit_totals is None:
raise RuntimeError(
f"Integration profile {profile}/{environment_kind} did not produce "
"a valid JUnit report."
)
if (profile in STRICT_PROFILES or require_no_skips) and junit_totals["skipped"]:
raise RuntimeError(
f"Required integration suite {profile}/{environment_kind} skipped "
f"{junit_totals['skipped']} required test(s)."
)
def _print_junit_summary(
profile: str,
environment_kind: str,
result_path: Path,
*,
deselected: int,
) -> dict[str, int] | None:
if not result_path.exists():
print(
f"[integration] summary profile={profile} environment={environment_kind} "
"result=missing",
flush=True,
)
return None
root = _sanitize_and_load_junit(result_path)
if root is None:
print(
f"[integration] summary profile={profile} environment={environment_kind} "
"result=invalid",
flush=True,
)
return None
suites = [root] if root.tag == "testsuite" else list(root.findall("testsuite"))
totals = {
key: sum(int(suite.attrib.get(key, "0")) for suite in suites)
for key in ("tests", "failures", "errors", "skipped")
}
passed = totals["tests"] - totals["failures"] - totals["errors"] - totals["skipped"]
print(
f"[integration] summary profile={profile} environment={environment_kind} "
f"passed={passed} failed={totals['failures']} errors={totals['errors']} "
f"skipped={totals['skipped']} deselected={deselected}",
flush=True,
)
return totals
def _sanitize_and_load_junit(result_path: Path) -> ET.Element | None:
try:
tree = ET.parse(result_path)
source_root = tree.getroot()
if source_root.tag == "testsuite":
suites = [source_root]
elif source_root.tag == "testsuites":
suites = list(source_root.findall("testsuite"))
else:
suites = []
if not suites:
raise ValueError("JUnit report does not contain a test suite.")
safe_suites: list[ET.Element] = []
for suite_index, suite in enumerate(suites):
counts: dict[str, int] = {}
for key in ("tests", "failures", "errors", "skipped"):
value = int(suite.attrib.get(key, "0"))
if value < 0:
raise ValueError(f"JUnit {key} count must be non-negative.")
counts[key] = value
testcases = list(suite.findall("testcase"))
actual_counts = {
"tests": len(testcases),
"failures": sum(len(case.findall("failure")) for case in testcases),
"errors": sum(len(case.findall("error")) for case in testcases),
"skipped": sum(len(case.findall("skipped")) for case in testcases),
}
if counts != actual_counts:
raise ValueError("JUnit declared counts do not match testcase outcomes.")
if any(
sum(len(case.findall(outcome)) for outcome in ("failure", "error", "skipped")) > 1
for case in testcases
):
raise ValueError("JUnit testcase has multiple terminal outcomes.")
safe_suite = ET.Element(
"testsuite",
{
"name": f"suite-{suite_index}",
**{key: str(value) for key, value in counts.items()},
},
)
safe_suites.append(safe_suite)
for case_index, case in enumerate(testcases):
safe_case = ET.SubElement(
safe_suite,
"testcase",
{"name": f"case-{case_index}"},
)
for outcome in ("failure", "error", "skipped"):
if case.find(outcome) is not None:
ET.SubElement(safe_case, outcome)
break
if source_root.tag == "testsuite":
safe_root = safe_suites[0]
else:
safe_root = ET.Element("testsuites")
safe_root.extend(safe_suites)
ET.ElementTree(safe_root).write(result_path, encoding="utf-8", xml_declaration=True)
except (ET.ParseError, OSError, ValueError):
try:
result_path.unlink(missing_ok=True)
except OSError:
pass
return None
return safe_root
def main() -> None:
args = parse_args()
prepare_profile_environment(args.profile)
prospective_policy: SubmoduleExportPolicy | None = None
if args.profile in {"prospective-contract", "prospective-platform"}:
prospective_contract = os.environ.get(PROSPECTIVE_CONTRACT_ENV)
if not prospective_contract or not Path(prospective_contract).is_file():
raise RuntimeError(
"The prospective-contract profile requires "
f"{PROSPECTIVE_CONTRACT_ENV} to name an existing contract file."
)
prospective_policy = load_submodule_export_policy(CONTRACT_POLICY)
if args.profile in STRICT_PROFILES:
os.environ["OPENAI_AGENTS_INTEGRATION_STRICT"] = "1"
shutil.rmtree(RESULTS / args.profile, ignore_errors=True)
if args.all:
os.environ["OPENAI_AGENTS_INTEGRATION_EXTERNAL_PROVIDERS"] = "1"
os.environ["OPENAI_AGENTS_INTEGRATION_DIRECT_PROVIDERS"] = "1"
wheel, sdist = build_distributions()
print(f"[integration] wheel={wheel.name} sdist={sdist.name} profile={args.profile}")
if args.profile == "mcp-v1":
for mcp_version in ("1.19.0", "1.29.0"):
environment_kind = f"mcp-v1-{mcp_version}"
python = create_environment(
environment_kind,
wheel,
additional_requirements=(f"mcp=={mcp_version}",),
)
run_suite(
python,
wheel,
sdist,
selection="mcp_compat",
environment_kind=environment_kind,
additional_env={"OPENAI_AGENTS_INTEGRATION_MCP_VERSION": mcp_version},
profile=args.profile,
)
if args.profile in {
"packaging",
"prospective-contract",
"security",
"core",
"hosted",
"full",
"release",
"nightly",
"manual",
}:
python = create_environment(
"core",
wheel,
optional_extra="docker" if args.profile in STRICT_PROFILES else None,
)
selections = {
"packaging": "packaging",
"prospective-contract": "packaging",
"security": "security",
"core": "packaging or core",
"hosted": "packaging or hosted",
"full": "packaging or ((core or hosted) and not nightly and not manual)",
"release": (
"packaging or security or ((core or hosted) and not nightly and not manual)"
),
"nightly": "packaging or ((core or hosted) and not manual)",
"manual": "packaging or core or hosted",
}
run_suite(
python,
wheel,
sdist,
selection=selections[args.profile],
environment_kind="core",
profile=args.profile,
)
if args.profile in {"providers", "realtime", "voice", "full", "release", "nightly", "manual"}:
python = create_environment("extended", wheel, extras=True)
if args.profile in {"full", "release"}:
selection = "(providers or realtime or voice) and not nightly and not manual"
elif args.profile == "nightly":
selection = "(providers or realtime or voice) and not manual"
elif args.profile == "manual":
selection = "providers or realtime or voice"
else:
selection = args.profile
run_suite(
python,
wheel,
sdist,
selection=selection,
environment_kind="extended",
profile=args.profile,
)
if args.profile in {
"packaging",
"prospective-contract",
"security",
"full",
"release",
"nightly",
"manual",
}:
python = create_environment(
"sdist",
sdist,
optional_extra="docker" if args.profile in STRICT_PROFILES else None,
)
if args.profile == "security":
selection = "security"
elif args.profile == "release":
selection = "packaging or distribution_smoke or security"
elif args.profile in {"nightly", "manual"}:
selection = "packaging or distribution_smoke"
else:
selection = "packaging"
run_suite(
python,
wheel,
sdist,
selection=selection,
environment_kind="sdist",
profile=args.profile,
)
if args.profile == "prospective-contract":
assert prospective_policy is not None
for artifact_kind, distribution in (("wheel", wheel), ("sdist", sdist)):
for installation in prospective_policy.dependency_installations:
if not installation.is_supported_on_current_platform():
print(
"[integration] skipping optional dependency "
f"{installation.dependency_module} on unsupported platform "
f"{sys.platform}",
flush=True,
)
continue
dependency_slug = re.sub(r"[^a-z0-9]+", "-", installation.dependency_module.lower())
environment_kind = f"{artifact_kind}-prospective-{dependency_slug}"
additional_requirements = (
(installation.requirement,) if installation.requirement is not None else ()
)
python = create_environment(
environment_kind,
distribution,
optional_extra=installation.extra,
additional_requirements=additional_requirements,
)
installation_description = (
f"extra {installation.extra}"
if installation.extra is not None
else f"requirement {installation.requirement}"
)
additional_env = {
"OPENAI_AGENTS_INTEGRATION_REQUIRED_OPTIONAL_DEPENDENCIES": (
installation.dependency_module
),
"OPENAI_AGENTS_INTEGRATION_OPTIONAL_DEPENDENCY_INSTALLATION": (
installation_description
),
}
if installation.extra is not None:
additional_env["OPENAI_AGENTS_INTEGRATION_REQUIRED_OPTIONAL_EXTRA"] = (
installation.extra
)
run_suite(
python,
wheel,
sdist,
selection="packaging_dependency",
environment_kind=environment_kind,
additional_env=additional_env,
profile=args.profile,
require_no_skips=True,
)
if args.profile == "prospective-platform":
assert prospective_policy is not None
core_environment_kind = "wheel-prospective-platform-core"
core_python = create_environment(core_environment_kind, wheel)
run_suite(
core_python,
wheel,
sdist,
selection="packaging_dependency",
environment_kind=core_environment_kind,
profile=args.profile,
require_no_skips=True,
)
unsupported_installations = tuple(
installation
for installation in prospective_policy.dependency_installations
if not installation.is_supported_on_current_platform()
)
for installation in unsupported_installations:
print(
"[integration] skipping optional dependency "
f"{installation.dependency_module} on unsupported platform {sys.platform}",
flush=True,
)
supported_installations = tuple(
installation
for installation in prospective_policy.dependency_installations
if installation.is_supported_on_current_platform()
)
dependency_extras = sorted(
{
installation.extra
for installation in supported_installations
if installation.extra is not None
}
)
dependency_requirements = tuple(
sorted(
{
installation.requirement
for installation in supported_installations
if installation.requirement is not None
}
)
)
dependency_modules = ",".join(
installation.dependency_module for installation in supported_installations
)
environment_kind = "wheel-prospective-platform"
python = create_environment(
environment_kind,
wheel,
optional_extra=",".join(dependency_extras) or None,
additional_requirements=dependency_requirements,
)
run_suite(
python,
wheel,
sdist,
selection="packaging_dependency",
environment_kind=environment_kind,
additional_env={
"OPENAI_AGENTS_INTEGRATION_REQUIRED_OPTIONAL_DEPENDENCIES": dependency_modules,
"OPENAI_AGENTS_INTEGRATION_OPTIONAL_DEPENDENCY_INSTALLATION": (
"policy optional dependencies"
),
},
profile=args.profile,
require_no_skips=True,
)
if args.profile in {"packaging", "release"}:
for artifact_kind, distribution in (("wheel", wheel), ("sdist", sdist)):
environment_kind = f"{artifact_kind}-cloudflare"
python = create_environment(
environment_kind,
distribution,
optional_extra="cloudflare",
)
run_suite(
python,
wheel,
sdist,
selection="packaging_dependency",
environment_kind=environment_kind,
additional_env={
"OPENAI_AGENTS_INTEGRATION_REQUIRED_OPTIONAL_DEPENDENCIES": "aiohttp",
"OPENAI_AGENTS_INTEGRATION_OPTIONAL_DEPENDENCY_INSTALLATION": (
"extra cloudflare"
),
"OPENAI_AGENTS_INTEGRATION_REQUIRED_OPTIONAL_EXTRA": "cloudflare",
},
profile=args.profile,
require_no_skips=True,
)
if args.profile in {"extras", "full", "release", "nightly", "manual"}:
for optional_extra in OPTIONAL_EXTRAS:
environment_kind = f"extra-{optional_extra}"
python = create_environment(environment_kind, wheel, optional_extra=optional_extra)
run_suite(
python,
wheel,
sdist,
selection="extras",
environment_kind=environment_kind,
profile=args.profile,
)
if __name__ == "__main__":
main()
+53
View File
@@ -0,0 +1,53 @@
from __future__ import annotations
import argparse
import fnmatch
import os
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[2]
PYTEST_FILE_PATTERNS = ("test_*.py", "*_test.py")
SERIAL_MARKER = "pytest.mark.serial"
def _test_files() -> list[Path]:
return sorted(
path
for path in (ROOT / "tests").rglob("*.py")
if any(fnmatch.fnmatchcase(path.name, pattern) for pattern in PYTEST_FILE_PATTERNS)
)
def _serial_test_files() -> list[Path]:
return [path for path in _test_files() if SERIAL_MARKER in path.read_text(encoding="utf-8")]
def _relative(path: Path) -> str:
return str(path.relative_to(ROOT))
def _serial_args(*, marker_expression: str = "serial") -> list[str]:
return [
sys.executable,
"-m",
"pytest",
*(_relative(path) for path in _serial_test_files()),
"-m",
marker_expression,
]
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--exclude-review-optional", action="store_true")
args = parser.parse_args()
os.chdir(ROOT)
marker_expression = (
"serial and not review_optional" if args.exclude_review_optional else "serial"
)
os.execv(sys.executable, _serial_args(marker_expression=marker_expression))
if __name__ == "__main__":
main()
+420
View File
@@ -0,0 +1,420 @@
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import hashlib
import json
import os
import re
import sys
from dataclasses import dataclass
from datetime import datetime, timedelta, timezone
from pathlib import Path
from typing import cast
from urllib import error, parse, request
_RCLONE_RELEASES_API = "https://api.github.com/repos/rclone/rclone/releases"
_DEFAULT_COOLDOWN_DAYS = 7
_RUNTIME_PIN_PATH = Path("src/agents/extensions/sandbox/_rclone.py")
_DOCKER_PIN_PATH = Path("examples/sandbox/docker/Dockerfile.mount")
_PYTHON_PIN_BEGIN = "# BEGIN RCLONE RELEASE PIN"
_PYTHON_PIN_END = "# END RCLONE RELEASE PIN"
_DOCKER_PIN_BEGIN = "# BEGIN RCLONE RELEASE PIN"
_DOCKER_PIN_END = "# END RCLONE RELEASE PIN"
_RCLONE_ARCHES = ("386", "amd64", "arm", "arm-v6", "arm-v7", "arm64")
_DOCKER_ARCHES = ("amd64", "arm64")
_SHA256_LINE = re.compile(r"^([0-9a-fA-F]{64})\s+\*?(\S+)$")
@dataclass(frozen=True)
class RclonePin:
version: str
sha256_by_arch: dict[str, str]
def _headers(url: str) -> dict[str, str]:
headers = {
"Accept": "application/vnd.github+json",
"User-Agent": "openai-agents-python-rclone-pin-updater",
}
token = os.environ.get("GITHUB_TOKEN")
if token and parse.urlparse(url).hostname == "api.github.com":
headers["Authorization"] = f"Bearer {token}"
return headers
def _fetch_bytes(url: str) -> bytes:
req = request.Request(url, headers=_headers(url))
try:
with request.urlopen(req, timeout=30) as response:
return response.read()
except error.HTTPError as exc:
raise RuntimeError(f"failed to fetch {url}: HTTP {exc.code}") from exc
except error.URLError as exc:
raise RuntimeError(f"failed to fetch {url}: {exc.reason}") from exc
def _fetch_json_object(url: str) -> dict[str, object]:
payload = json.loads(_fetch_bytes(url))
if not isinstance(payload, dict):
raise RuntimeError(f"expected a JSON object from {url}")
return cast(dict[str, object], payload)
def _fetch_json_array(url: str) -> list[dict[str, object]]:
payload = json.loads(_fetch_bytes(url))
if not isinstance(payload, list) or not all(isinstance(item, dict) for item in payload):
raise RuntimeError(f"expected a JSON array of objects from {url}")
return cast(list[dict[str, object]], payload)
def _normalized_version(value: str) -> str:
version = value.removeprefix("v")
if re.fullmatch(r"\d+\.\d+\.\d+", version) is None:
raise ValueError(f"invalid rclone version: {value}")
return version
def _release_url(version: str | None) -> str:
if version is None:
return f"{_RCLONE_RELEASES_API}?per_page=100"
tag = parse.quote(f"v{_normalized_version(version)}", safe="")
return f"{_RCLONE_RELEASES_API}/tags/{tag}"
def _release_version(release: dict[str, object]) -> str:
tag_name = release.get("tag_name")
if not isinstance(tag_name, str):
raise RuntimeError("rclone release metadata is missing tag_name")
return _normalized_version(tag_name)
def _release_published_at(release: dict[str, object]) -> datetime:
value = release.get("published_at")
if not isinstance(value, str):
raise RuntimeError("rclone release metadata is missing published_at")
try:
published_at = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise RuntimeError(f"rclone release has invalid published_at: {value}") from exc
if published_at.tzinfo is None:
raise RuntimeError(f"rclone release published_at has no timezone: {value}")
return published_at.astimezone(timezone.utc)
def _asset_observed_at(release: dict[str, object], asset_name: str) -> datetime:
asset = _asset(release, asset_name)
timestamps: list[datetime] = []
for field in ("created_at", "updated_at"):
value = asset.get(field)
if not isinstance(value, str):
raise RuntimeError(f"rclone release asset {asset_name} is missing {field}")
try:
timestamp = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError as exc:
raise RuntimeError(
f"rclone release asset {asset_name} has invalid {field}: {value}"
) from exc
if timestamp.tzinfo is None:
raise RuntimeError(
f"rclone release asset {asset_name} {field} has no timezone: {value}"
)
timestamps.append(timestamp.astimezone(timezone.utc))
return max(timestamps)
def _required_asset_names(version: str) -> tuple[str, ...]:
archives = tuple(f"rclone-v{version}-linux-{arch}.zip" for arch in _RCLONE_ARCHES)
return ("SHA256SUMS", *archives)
def _validate_cooldown(
subject: str,
observed_at: datetime,
*,
cooldown_days: int,
now: datetime,
) -> None:
eligible_at = observed_at + timedelta(days=cooldown_days)
if eligible_at > now:
raise RuntimeError(
f"{subject} is still in its {cooldown_days}-day cooldown "
f"(eligible at {eligible_at.isoformat()})"
)
def _validate_stable_release(
release: dict[str, object],
*,
cooldown_days: int,
now: datetime,
) -> None:
version = _release_version(release)
if release.get("draft") is not False or release.get("prerelease") is not False:
raise RuntimeError(f"rclone v{version} is not a stable published release")
_validate_cooldown(
f"rclone v{version}",
_release_published_at(release),
cooldown_days=cooldown_days,
now=now,
)
for asset_name in _required_asset_names(version):
_validate_cooldown(
f"rclone v{version} asset {asset_name}",
_asset_observed_at(release, asset_name),
cooldown_days=cooldown_days,
now=now,
)
def _latest_stable_release(
releases: list[dict[str, object]],
*,
cooldown_days: int,
now: datetime,
) -> dict[str, object]:
eligible: list[tuple[datetime, dict[str, object]]] = []
for release in releases:
try:
_validate_stable_release(release, cooldown_days=cooldown_days, now=now)
except (RuntimeError, ValueError):
continue
eligible.append((_release_published_at(release), release))
if not eligible:
raise RuntimeError(
f"no stable rclone release has completed the {cooldown_days}-day cooldown"
)
return max(eligible, key=lambda item: item[0])[1]
def _asset(release: dict[str, object], asset_name: str) -> dict[str, object]:
assets = release.get("assets")
if not isinstance(assets, list):
raise RuntimeError("rclone release metadata is missing assets")
for asset in assets:
if isinstance(asset, dict) and asset.get("name") == asset_name:
return cast(dict[str, object], asset)
raise RuntimeError(f"rclone release is missing {asset_name}")
def _asset_url(release: dict[str, object], asset_name: str) -> str:
url = _asset(release, asset_name).get("browser_download_url")
if not isinstance(url, str):
raise RuntimeError(f"rclone release asset {asset_name} is missing its download URL")
return url
def _asset_sha256(release: dict[str, object], asset_name: str) -> str:
digest = _asset(release, asset_name).get("digest")
if not isinstance(digest, str) or re.fullmatch(r"sha256:[0-9a-fA-F]{64}", digest) is None:
raise RuntimeError(f"rclone release asset {asset_name} is missing its SHA256 digest")
return digest.removeprefix("sha256:").lower()
def _validate_download_sha256(
release: dict[str, object],
asset_name: str,
content: bytes,
) -> None:
expected = _asset_sha256(release, asset_name)
actual = hashlib.sha256(content).hexdigest()
if actual != expected:
raise RuntimeError(
f"downloaded rclone release asset {asset_name} does not match GitHub's digest"
)
def _parse_sha256s(text: str, version: str) -> dict[str, str]:
filenames = {f"rclone-v{version}-linux-{arch}.zip": arch for arch in _RCLONE_ARCHES}
sha256_by_arch: dict[str, str] = {}
for line in text.splitlines():
match = _SHA256_LINE.fullmatch(line.strip())
if match is None:
continue
digest, filename = match.groups()
arch = filenames.get(filename)
if arch is not None:
sha256_by_arch[arch] = digest.lower()
missing = [arch for arch in _RCLONE_ARCHES if arch not in sha256_by_arch]
if missing:
raise RuntimeError(
f"rclone v{version} SHA256SUMS is missing Linux archives for: {', '.join(missing)}"
)
return sha256_by_arch
def _validate_asset_sha256s(
release: dict[str, object],
version: str,
sha256_by_arch: dict[str, str],
) -> None:
for arch in _RCLONE_ARCHES:
asset_name = f"rclone-v{version}-linux-{arch}.zip"
asset_sha256 = _asset_sha256(release, asset_name)
if asset_sha256 != sha256_by_arch[arch]:
raise RuntimeError(
f"rclone v{version} SHA256SUMS does not match GitHub's digest for {asset_name}"
)
def fetch_pin(
version: str | None = None,
*,
cooldown_days: int = _DEFAULT_COOLDOWN_DAYS,
now: datetime | None = None,
) -> RclonePin:
if cooldown_days < 0:
raise ValueError("cooldown days must be zero or greater")
current_time = now or datetime.now(timezone.utc)
if current_time.tzinfo is None:
raise ValueError("current time must include a timezone")
current_time = current_time.astimezone(timezone.utc)
if version is None:
releases = _fetch_json_array(_release_url(None))
release = _latest_stable_release(
releases,
cooldown_days=cooldown_days,
now=current_time,
)
else:
release = _fetch_json_object(_release_url(version))
_validate_stable_release(
release,
cooldown_days=cooldown_days,
now=current_time,
)
resolved_version = _release_version(release)
if version is not None and resolved_version != _normalized_version(version):
raise RuntimeError(
f"requested rclone v{_normalized_version(version)}, got v{resolved_version}"
)
checksums_url = _asset_url(release, "SHA256SUMS")
checksums_content = _fetch_bytes(checksums_url)
_validate_download_sha256(release, "SHA256SUMS", checksums_content)
checksums = checksums_content.decode("utf-8")
sha256_by_arch = _parse_sha256s(checksums, resolved_version)
_validate_asset_sha256s(release, resolved_version, sha256_by_arch)
return RclonePin(
version=resolved_version,
sha256_by_arch=sha256_by_arch,
)
def _python_pin_block(pin: RclonePin) -> str:
lines = [
_PYTHON_PIN_BEGIN,
f'_RCLONE_VERSION = "{pin.version}"',
"_RCLONE_SHA256_BY_ARCH = {",
]
for arch in _RCLONE_ARCHES:
lines.append(f' "{arch}": "{pin.sha256_by_arch[arch]}",')
lines.extend(["}", _PYTHON_PIN_END])
return "\n".join(lines)
def _docker_pin_block(pin: RclonePin) -> str:
lines = [_DOCKER_PIN_BEGIN, f"ARG RCLONE_VERSION={pin.version}"]
for arch in _DOCKER_ARCHES:
variable_arch = arch.upper().replace("-", "_")
lines.append(f"ARG RCLONE_SHA256_LINUX_{variable_arch}={pin.sha256_by_arch[arch]}")
lines.append(_DOCKER_PIN_END)
return "\n".join(lines)
def _replace_marked_block(text: str, begin: str, end: str, replacement: str) -> str:
if text.count(begin) != 1 or text.count(end) != 1:
raise RuntimeError(f"expected exactly one pin block delimited by {begin!r} and {end!r}")
start = text.index(begin)
finish = text.index(end, start) + len(end)
return f"{text[:start]}{replacement}{text[finish:]}"
def apply_pin(repo_root: Path, pin: RclonePin, *, check: bool) -> list[Path]:
updates = (
(
repo_root / _RUNTIME_PIN_PATH,
_PYTHON_PIN_BEGIN,
_PYTHON_PIN_END,
_python_pin_block(pin),
),
(
repo_root / _DOCKER_PIN_PATH,
_DOCKER_PIN_BEGIN,
_DOCKER_PIN_END,
_docker_pin_block(pin),
),
)
changed: list[Path] = []
for path, begin, end, replacement in updates:
current = path.read_text()
updated = _replace_marked_block(current, begin, end, replacement)
if updated == current:
continue
changed.append(path)
if not check:
path.write_text(updated)
return changed
def main() -> int:
parser = argparse.ArgumentParser(
description="Update the verified rclone release pinned by sandbox installers."
)
parser.add_argument(
"--version",
help="rclone version to pin, with or without a leading v (default: latest stable)",
)
parser.add_argument(
"--check",
action="store_true",
help="report stale pin blocks without modifying files",
)
parser.add_argument(
"--cooldown-days",
type=int,
default=_DEFAULT_COOLDOWN_DAYS,
help=(
"minimum age of a stable release before it is eligible "
f"(default: {_DEFAULT_COOLDOWN_DAYS})"
),
)
parser.add_argument(
"--repo-root",
type=Path,
default=Path(__file__).resolve().parents[2],
help=argparse.SUPPRESS,
)
args = parser.parse_args()
try:
pin = fetch_pin(args.version, cooldown_days=args.cooldown_days)
changed = apply_pin(args.repo_root.resolve(), pin, check=args.check)
except (OSError, RuntimeError, ValueError) as exc:
print(f"error: {exc}", file=sys.stderr)
return 2
if not changed:
print(f"rclone v{pin.version} pin is current")
return 0
relative_paths = [str(path.relative_to(args.repo_root.resolve())) for path in changed]
if args.check:
print(
f"::error title=Stale rclone pin::rclone v{pin.version} differs in "
f"{', '.join(relative_paths)}",
file=sys.stderr,
)
print(
f"Run: python .github/scripts/update_rclone_pin.py --version {pin.version}",
file=sys.stderr,
)
return 1
print(f"updated rclone v{pin.version} pin in {', '.join(relative_paths)}")
return 0
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,136 @@
from __future__ import annotations
import argparse
import json
import re
import subprocess
import sys
from pathlib import Path
if sys.version_info >= (3, 11):
import tomllib
else:
import tomli as tomllib
ROOT = Path(__file__).resolve().parents[2]
CONTRACT = ROOT / "tests" / "fixtures" / "released_api_contract.json"
POLICY = ROOT / "tests" / "fixtures" / "released_api_contract_policy.json"
sys.path.insert(0, str(ROOT))
from integration_tests._contract_support import ( # noqa: E402
build_released_api_contract,
load_api_contract,
load_submodule_export_policy,
)
def _parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description="Update the rolling released public API contract from the local checkout."
)
parser.add_argument("--version", required=True, help="Release version without a leading v.")
parser.add_argument(
"--check",
action="store_true",
help="Fail instead of writing when the committed contract is out of date.",
)
parser.add_argument(
"--output",
type=Path,
help="Write a prospective contract to this path instead of changing the released fixture.",
)
return parser.parse_args()
def _project_version() -> str:
data = tomllib.loads((ROOT / "pyproject.toml").read_text(encoding="utf-8"))
version = data.get("project", {}).get("version")
if not isinstance(version, str):
raise RuntimeError("pyproject.toml is missing project.version")
return version
def _head_commit() -> str:
return subprocess.check_output(
["git", "rev-parse", "HEAD"],
cwd=ROOT,
text=True,
).strip()
def _render(contract: dict[str, object]) -> str:
return json.dumps(contract, indent=2, sort_keys=True) + "\n"
def main() -> int:
args = _parse_args()
if args.check and args.output is not None:
raise SystemExit("--check and --output cannot be used together")
version = args.version
if (
version.startswith("v")
or re.fullmatch(r"\d+\.\d+(?:\.\d+)*(?:[A-Za-z0-9.-]+)?", version) is None
):
raise SystemExit("--version must be a semver-like value without a leading v")
project_version = _project_version()
if project_version != version:
raise SystemExit(
f"--version {version!r} does not match pyproject.toml version {project_version!r}"
)
output = args.output.resolve() if args.output is not None else None
protected_outputs = {CONTRACT.resolve(), POLICY.resolve()}
if output in protected_outputs:
raise SystemExit(
"--output must not overwrite released API contract inputs: "
"tests/fixtures/released_api_contract.json or "
"tests/fixtures/released_api_contract_policy.json"
)
current = load_api_contract(CONTRACT)
policy = load_submodule_export_policy(POLICY)
try:
updated = build_released_api_contract(
current,
baseline=f"v{version}",
baseline_commit=_head_commit(),
release_policy=policy,
)
except ValueError as error:
raise SystemExit(str(error)) from None
rendered = _render(updated)
if output is not None:
output.parent.mkdir(parents=True, exist_ok=True)
output.write_text(rendered, encoding="utf-8")
print(f"Wrote prospective released API contract to {output}.")
return 0
existing = CONTRACT.read_text(encoding="utf-8")
if rendered == existing:
print(f"Released API contract is current for v{version}.")
return 0
if args.check:
print(
f"Released API contract is out of date for v{version}; "
f"run `make update-released-api-contract VERSION={version}`.",
file=sys.stderr,
)
return 1
previous_exports = set(current["required_top_level_exports"])
current_exports = set(updated["required_top_level_exports"])
CONTRACT.write_text(rendered, encoding="utf-8")
print(f"Updated released API contract for v{version}.")
print(f"Added exports: {sorted(current_exports - previous_exports)!r}")
print(f"Removed exports: {sorted(previous_exports - current_exports)!r}")
print(
"Review shipped example imports and update released_api_contract_policy.json when "
"the release adds canonical imports, public properties, public type aliases, public "
"TypedDict fields, or public modules."
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+5 -2
View File
@@ -16,7 +16,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Determine docs-only push
id: docs-only
run: |
@@ -36,9 +36,12 @@ jobs:
fi
- name: Setup uv
if: steps.docs-only.outputs.skip != 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Install dependencies
if: steps.docs-only.outputs.skip != 'true'
run: make sync
+1 -1
View File
@@ -10,7 +10,7 @@ jobs:
issues: write
pull-requests: write
steps:
- uses: actions/stale@b5d41d4e1d5dceea10e7104786b73624c18a190f
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93
with:
days-before-issue-stale: 7
days-before-issue-close: 3
-204
View File
@@ -1,204 +0,0 @@
name: Auto label PRs
on:
pull_request_target:
types:
- opened
- reopened
- synchronize
- ready_for_review
workflow_dispatch:
inputs:
pr_number:
description: "PR number to label."
required: true
type: number
permissions:
contents: read
issues: write
pull-requests: write
jobs:
label:
runs-on: ubuntu-latest
steps:
- name: Ensure main workflow
if: ${{ github.event_name == 'workflow_dispatch' && github.ref != 'refs/heads/main' }}
run: |
echo "This workflow must be dispatched from main."
exit 1
- name: Resolve PR context
id: pr
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd
env:
MANUAL_PR_NUMBER: ${{ inputs.pr_number || '' }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const isManual = context.eventName === 'workflow_dispatch';
let pr;
if (isManual) {
const prNumber = Number(process.env.MANUAL_PR_NUMBER);
if (!prNumber) {
core.setFailed('workflow_dispatch requires pr_number input.');
return;
}
const { data } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: prNumber,
});
pr = data;
} else {
pr = context.payload.pull_request;
}
if (!pr) {
core.setFailed('Missing pull request context.');
return;
}
const headRepo = pr.head.repo.full_name;
const repoFullName = `${context.repo.owner}/${context.repo.repo}`;
core.setOutput('pr_number', pr.number);
core.setOutput('base_sha', pr.base.sha);
core.setOutput('head_sha', pr.head.sha);
core.setOutput('head_repo', headRepo);
core.setOutput('is_fork', headRepo !== repoFullName);
core.setOutput('title', pr.title || '');
core.setOutput('body', pr.body || '');
- name: Checkout base
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
fetch-depth: 0
ref: ${{ steps.pr.outputs.base_sha }}
- name: Fetch PR head
env:
PR_HEAD_REPO: ${{ steps.pr.outputs.head_repo }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
run: |
set -euo pipefail
git fetch --no-tags --prune --recurse-submodules=no \
"https://github.com/${PR_HEAD_REPO}.git" \
"${PR_HEAD_SHA}"
- name: Collect PR diff
id: diff
env:
PR_BASE_SHA: ${{ steps.pr.outputs.base_sha }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
PR_TITLE: ${{ steps.pr.outputs.title }}
PR_BODY: ${{ steps.pr.outputs.body }}
run: |
set -euo pipefail
mkdir -p .tmp/pr-labels
diff_base_sha="$(git merge-base "$PR_BASE_SHA" "$PR_HEAD_SHA")"
echo "diff_base_sha=${diff_base_sha}" >> "$GITHUB_OUTPUT"
git diff --name-only "$diff_base_sha" "$PR_HEAD_SHA" > .tmp/pr-labels/changed-files.txt
git diff "$diff_base_sha" "$PR_HEAD_SHA" > .tmp/pr-labels/changes.diff
python - <<'PY'
import json
import os
import pathlib
pathlib.Path(".tmp/pr-labels/pr-context.json").write_text(
json.dumps(
{
"title": os.environ.get("PR_TITLE", ""),
"body": os.environ.get("PR_BODY", ""),
},
ensure_ascii=False,
indent=2,
)
+ "\n"
)
PY
- name: Prepare Codex output
id: codex-output
run: |
set -euo pipefail
output_dir=".tmp/codex/outputs"
output_file="${output_dir}/pr-labels.json"
mkdir -p "$output_dir"
echo "output_file=${output_file}" >> "$GITHUB_OUTPUT"
- name: Run Codex labeling
id: run_codex
if: ${{ (github.event_name == 'workflow_dispatch' || steps.pr.outputs.is_fork != 'true') && github.actor != 'dependabot[bot]' }}
uses: openai/codex-action@c25d10f3f498316d4b2496cc4c6dd58057a7b031
with:
openai-api-key: ${{ secrets.PROD_OPENAI_API_KEY }}
prompt-file: .github/codex/prompts/pr-labels.md
output-file: ${{ steps.codex-output.outputs.output_file }}
output-schema-file: .github/codex/schemas/pr-labels.json
# Keep the legacy Linux sandbox path until the default bubblewrap path
# works reliably on GitHub-hosted Ubuntu runners.
codex-args: '["--enable","use_legacy_landlock"]'
safety-strategy: drop-sudo
sandbox: read-only
- name: Apply labels
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
PR_BASE_SHA: ${{ steps.diff.outputs.diff_base_sha }}
PR_HEAD_SHA: ${{ steps.pr.outputs.head_sha }}
CODEX_OUTPUT_PATH: ${{ steps.codex-output.outputs.output_file }}
CODEX_CONCLUSION: ${{ steps.run_codex.conclusion }}
run: |
python .github/scripts/pr_labels.py
- name: Comment on manual run failure
if: ${{ github.event_name == 'workflow_dispatch' && always() }}
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd
env:
PR_NUMBER: ${{ steps.pr.outputs.pr_number }}
JOB_STATUS: ${{ job.status }}
RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}
CODEX_CONCLUSION: ${{ steps.run_codex.conclusion }}
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const marker = '<!-- pr-labels-manual-run -->';
const jobStatus = process.env.JOB_STATUS;
if (jobStatus === 'success') {
return;
}
const prNumber = Number(process.env.PR_NUMBER);
if (!prNumber) {
core.setFailed('Missing PR number for manual run comment.');
return;
}
const body = [
marker,
'Manual PR labeling failed.',
`Job status: ${jobStatus}.`,
`Run: ${process.env.RUN_URL}.`,
`Codex labeling: ${process.env.CODEX_CONCLUSION}.`,
].join('\n');
const { data: comments } = await github.rest.issues.listComments({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
per_page: 100,
});
const existing = comments.find(
(comment) =>
comment.user?.login === 'github-actions[bot]' &&
comment.body?.includes(marker),
);
if (existing) {
await github.rest.issues.updateComment({
owner: context.repo.owner,
repo: context.repo.repo,
comment_id: existing.id,
body,
});
core.info(`Updated existing comment ${existing.id}`);
return;
}
const { data: created } = await github.rest.issues.createComment({
owner: context.repo.owner,
repo: context.repo.repo,
issue_number: prNumber,
body,
});
core.info(`Created comment ${created.id}`);
+6 -3
View File
@@ -21,14 +21,17 @@ jobs:
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Setup uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Install dependencies
run: make sync
- name: Build package
run: uv build
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e
uses: pypa/gh-action-pypi-publish@dc37677b2e1c63e2034f94d8a5b11f265b73ba33
-109
View File
@@ -1,109 +0,0 @@
name: Update release PR on main updates
on:
push:
branches:
- main
concurrency:
group: release-pr-update
cancel-in-progress: true
permissions:
contents: write
pull-requests: write
jobs:
update-release-pr:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
fetch-depth: 0
- name: Fetch tags
run: git fetch origin --tags --prune
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Find release PR
id: find
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
base_branch="main"
prs_json="$(gh pr list \
--base "$base_branch" \
--state open \
--search "head:release/v" \
--limit 200 \
--json number,headRefName,isCrossRepository,headRepositoryOwner)"
count="$(echo "$prs_json" | jq '[.[] | select(.isCrossRepository == false) | select(.headRefName|startswith("release/v"))] | length')"
if [ "$count" -eq 0 ]; then
echo "found=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if [ "$count" -gt 1 ]; then
echo "Multiple release PRs found; expected a single release PR." >&2
exit 1
fi
number="$(echo "$prs_json" | jq -r '.[] | select(.isCrossRepository == false) | select(.headRefName|startswith("release/v")) | .number')"
branch="$(echo "$prs_json" | jq -r '.[] | select(.isCrossRepository == false) | select(.headRefName|startswith("release/v")) | .headRefName')"
echo "found=true" >> "$GITHUB_OUTPUT"
echo "number=$number" >> "$GITHUB_OUTPUT"
echo "branch=$branch" >> "$GITHUB_OUTPUT"
- name: Rebase release branch
if: steps.find.outputs.found == 'true'
env:
RELEASE_BRANCH: ${{ steps.find.outputs.branch }}
run: |
set -euo pipefail
git fetch origin main "$RELEASE_BRANCH"
git checkout -B "$RELEASE_BRANCH" "origin/$RELEASE_BRANCH"
git rebase origin/main
- name: Prepare Codex output
if: steps.find.outputs.found == 'true'
id: codex-output
run: |
set -euo pipefail
output_dir=".tmp/codex/outputs"
output_file="${output_dir}/release-review.md"
mkdir -p "$output_dir"
echo "output_file=${output_file}" >> "$GITHUB_OUTPUT"
- name: Run Codex release review
if: steps.find.outputs.found == 'true'
uses: openai/codex-action@c25d10f3f498316d4b2496cc4c6dd58057a7b031
with:
openai-api-key: ${{ secrets.PROD_OPENAI_API_KEY }}
prompt-file: .github/codex/prompts/release-review.md
output-file: ${{ steps.codex-output.outputs.output_file }}
# Keep the legacy Linux sandbox path until the default bubblewrap path
# works reliably on GitHub-hosted Ubuntu runners.
codex-args: '["--enable","use_legacy_landlock"]'
safety-strategy: drop-sudo
sandbox: read-only
- name: Update PR body and push
if: steps.find.outputs.found == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ steps.find.outputs.number }}
RELEASE_BRANCH: ${{ steps.find.outputs.branch }}
RELEASE_REVIEW_PATH: ${{ steps.codex-output.outputs.output_file }}
run: |
set -euo pipefail
git push --force-with-lease origin "$RELEASE_BRANCH"
gh pr edit "$PR_NUMBER" --body-file "$RELEASE_REVIEW_PATH"
version="${RELEASE_BRANCH#release/v}"
milestone_name="$(python .github/scripts/select-release-milestone.py --version "$version")"
if [ -n "$milestone_name" ]; then
if ! gh pr edit "$PR_NUMBER" --add-label "project" --milestone "$milestone_name"; then
echo "PR label/milestone update failed; continuing without changes." >&2
fi
else
if ! gh pr edit "$PR_NUMBER" --add-label "project"; then
echo "PR label update failed; continuing without changes." >&2
fi
fi
-167
View File
@@ -1,167 +0,0 @@
name: Create release PR
on:
workflow_dispatch:
inputs:
version:
description: "Version to release (e.g., 0.6.6)"
required: true
permissions:
contents: write
pull-requests: write
jobs:
release-pr:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
fetch-depth: 0
ref: main
- name: Setup uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
with:
enable-cache: true
- name: Fetch tags
run: git fetch origin --tags --prune
- name: Ensure release branch does not exist
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
branch="release/v${RELEASE_VERSION}"
if git ls-remote --exit-code --heads origin "$branch" >/dev/null 2>&1; then
echo "Branch $branch already exists on origin." >&2
exit 1
fi
- name: Update version
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
python - <<'PY'
import os
import pathlib
import re
import sys
version = os.environ["RELEASE_VERSION"]
if version.startswith("v"):
print("Version must not start with 'v' (use x.y.z...).", file=sys.stderr)
sys.exit(1)
if ".." in version:
print("Version contains consecutive dots (use x.y.z...).", file=sys.stderr)
sys.exit(1)
if not re.match(r"^\d+\.\d+(\.\d+)*([a-zA-Z0-9\.-]+)?$", version):
print(
"Version must be semver-like (e.g., 0.6.6, 0.6.6-rc1, 0.6.6.dev1).",
file=sys.stderr,
)
sys.exit(1)
path = pathlib.Path("pyproject.toml")
text = path.read_text()
updated, count = re.subn(
r'(?m)^version\s*=\s*"[^\"]+"',
f'version = "{version}"',
text,
)
if count != 1:
print("Expected to update exactly one version line.", file=sys.stderr)
sys.exit(1)
if updated == text:
print("Version already set; no changes made.", file=sys.stderr)
sys.exit(1)
path.write_text(updated)
PY
- name: Sync dependencies
run: make sync
- name: Configure git
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
- name: Create release branch and commit
env:
RELEASE_VERSION: ${{ inputs.version }}
run: |
branch="release/v${RELEASE_VERSION}"
git checkout -b "$branch"
git add pyproject.toml uv.lock
if git diff --cached --quiet; then
echo "No changes to commit." >&2
exit 1
fi
git commit -m "Bump version to ${RELEASE_VERSION}"
git push --set-upstream origin "$branch"
- name: Prepare Codex output
id: codex-output
run: |
set -euo pipefail
output_dir=".tmp/codex/outputs"
output_file="${output_dir}/release-review.md"
mkdir -p "$output_dir"
echo "output_file=${output_file}" >> "$GITHUB_OUTPUT"
- name: Run Codex release review
uses: openai/codex-action@c25d10f3f498316d4b2496cc4c6dd58057a7b031
with:
openai-api-key: ${{ secrets.PROD_OPENAI_API_KEY }}
prompt-file: .github/codex/prompts/release-review.md
output-file: ${{ steps.codex-output.outputs.output_file }}
# Keep the legacy Linux sandbox path until the default bubblewrap path
# works reliably on GitHub-hosted Ubuntu runners.
codex-args: '["--enable","use_legacy_landlock"]'
safety-strategy: drop-sudo
sandbox: read-only
- name: Build PR body
env:
RELEASE_REVIEW_PATH: ${{ steps.codex-output.outputs.output_file }}
run: |
python - <<'PY'
import os
import pathlib
report = pathlib.Path(os.environ["RELEASE_REVIEW_PATH"]).read_text()
pathlib.Path("pr-body.md").write_text(report)
PY
- name: Create or update PR
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
RELEASE_VERSION: ${{ inputs.version }}
run: |
set -euo pipefail
head_branch="release/v${RELEASE_VERSION}"
milestone_name="$(python .github/scripts/select-release-milestone.py --version "$RELEASE_VERSION")"
pr_number="$(gh pr list --head "$head_branch" --base "main" --json number --jq '.[0].number // empty')"
if [ -z "$pr_number" ]; then
create_args=(
--title "Release ${RELEASE_VERSION}"
--body-file pr-body.md
--base "main"
--head "$head_branch"
--label "project"
)
if [ -n "$milestone_name" ]; then
create_args+=(--milestone "$milestone_name")
fi
if ! gh pr create "${create_args[@]}"; then
echo "PR create with label/milestone failed; retrying without them." >&2
gh pr create \
--title "Release ${RELEASE_VERSION}" \
--body-file pr-body.md \
--base "main" \
--head "$head_branch"
fi
else
edit_args=(
--title "Release ${RELEASE_VERSION}"
--body-file pr-body.md
--add-label "project"
)
if [ -n "$milestone_name" ]; then
edit_args+=(--milestone "$milestone_name")
fi
if ! gh pr edit "$pr_number" "${edit_args[@]}"; then
echo "PR edit with label/milestone failed; retrying without them." >&2
gh pr edit "$pr_number" --title "Release ${RELEASE_VERSION}" --body-file pr-body.md
fi
fi
+4 -3
View File
@@ -14,6 +14,7 @@ jobs:
tag-release:
if: >-
github.event.pull_request.merged == true &&
github.event.pull_request.head.repo.full_name == github.repository &&
startsWith(github.event.pull_request.head.ref, 'release/v')
runs-on: ubuntu-latest
steps:
@@ -26,14 +27,14 @@ jobs:
exit 1
fi
- name: Checkout merge commit
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
fetch-depth: 0
ref: ${{ github.event.pull_request.merge_commit_sha }}
- name: Setup Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: "3.11"
python-version: "3.14"
- name: Configure git
run: |
git config user.name "github-actions[bot]"
+256 -15
View File
@@ -16,17 +16,21 @@ env:
jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
@@ -42,17 +46,29 @@ jobs:
typecheck:
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Restore mypy cache
if: steps.changes.outputs.run == 'true'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .mypy_cache
key: mypy-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('uv.lock', 'pyproject.toml', 'Makefile') }}-${{ github.sha }}
restore-keys: |
mypy-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('uv.lock', 'pyproject.toml', 'Makefile') }}-
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
@@ -63,8 +79,44 @@ jobs:
if: steps.changes.outputs.run != 'true'
run: echo "Skipping typecheck for non-code changes."
mypy-win32:
runs-on: ubuntu-latest
timeout-minutes: 12
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Restore mypy cache
if: steps.changes.outputs.run == 'true'
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6.1.0
with:
path: .mypy_cache
key: mypy-win32-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('uv.lock', 'pyproject.toml', 'Makefile') }}-${{ github.sha }}
restore-keys: |
mypy-win32-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('uv.lock', 'pyproject.toml', 'Makefile') }}-
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Run mypy for Windows
if: steps.changes.outputs.run == 'true'
run: uv run mypy --platform win32 src
- name: Skip Windows mypy
if: steps.changes.outputs.run != 'true'
run: echo "Skipping Windows mypy for non-code changes."
tests:
runs-on: ubuntu-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
@@ -78,24 +130,26 @@ jobs:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: ${{ matrix.python-version }}
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Run tests with coverage
if: steps.changes.outputs.run == 'true' && matrix.python-version == '3.12'
if: steps.changes.outputs.run == 'true' && matrix.python-version == '3.14'
run: make coverage
- name: Run tests
if: steps.changes.outputs.run == 'true' && matrix.python-version != '3.12'
if: steps.changes.outputs.run == 'true' && matrix.python-version != '3.14'
run: make tests
- name: Run async teardown stability tests
if: steps.changes.outputs.run == 'true' && (matrix.python-version == '3.10' || matrix.python-version == '3.14')
@@ -104,23 +158,206 @@ jobs:
if: steps.changes.outputs.run != 'true'
run: echo "Skipping tests for non-code changes."
tests-windows:
runs-on: windows-latest
native-macos-sandbox:
runs-on: macos-latest
timeout-minutes: 15
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
persist-credentials: false
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: false
python-version: "3.14"
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Run native macOS sandbox tests
if: steps.changes.outputs.run == 'true'
run: uv run pytest -m requires_native_macos_sandbox
- name: Skip native macOS sandbox tests
if: steps.changes.outputs.run != 'true'
run: echo "Skipping native macOS sandbox tests for non-code changes."
mcp-v1-compat:
runs-on: ubuntu-latest
timeout-minutes: 5
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Run packaged MCP v1 compatibility tests
if: steps.changes.outputs.run == 'true'
run: make integration-tests-mcp-v1
- name: Skip MCP v1 compatibility tests
if: steps.changes.outputs.run != 'true'
run: echo "Skipping MCP v1 compatibility tests for non-code changes."
packaged-contract:
needs: prospective-release-contract
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version:
- "3.10"
- "3.14"
env:
OPENAI_AGENTS_INTEGRATION_PYTHON: ${{ matrix.python-version }}
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: ${{ matrix.python-version }}
- name: Download prospective release contract
if: steps.changes.outputs.run == 'true'
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: prospective-release-contract
path: .tmp
- name: Run packaged and prospective compatibility contracts
if: steps.changes.outputs.run == 'true'
env:
OPENAI_AGENTS_PROSPECTIVE_RELEASE_CONTRACT: ${{ github.workspace }}/.tmp/prospective_released_api_contract.json
run: make integration-tests-prospective-contract
- name: Skip packaged compatibility contracts
if: steps.changes.outputs.run != 'true'
run: echo "Skipping packaged compatibility contracts for non-code changes."
packaged-contract-windows:
needs: prospective-release-contract
runs-on: windows-latest
timeout-minutes: 15
env:
OPENAI_AGENTS_INTEGRATION_PYTHON: "3.14"
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
shell: bash
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
python-version: "3.13"
# Disable cache pruning on Windows to avoid intermittent post-job cleanup failures.
prune-cache: false
python-version: "3.14"
- name: Download prospective release contract
if: steps.changes.outputs.run == 'true'
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
with:
name: prospective-release-contract
path: .tmp
- name: Run Windows prospective contract smoke test
if: steps.changes.outputs.run == 'true'
env:
OPENAI_AGENTS_PROSPECTIVE_RELEASE_CONTRACT: ${{ github.workspace }}/.tmp/prospective_released_api_contract.json
run: make integration-tests-prospective-platform
- name: Skip Windows prospective contract smoke test
if: steps.changes.outputs.run != 'true'
run: echo "Skipping Windows prospective contract smoke test for non-code changes."
prospective-release-contract:
runs-on: ubuntu-latest
timeout-minutes: 10
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Install all optional dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
- name: Validate source and generate prospective release contract
if: steps.changes.outputs.run == 'true'
run: make prepare-prospective-released-api-contract
- name: Upload prospective release contract
if: steps.changes.outputs.run == 'true'
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: prospective-release-contract
path: .tmp/prospective_released_api_contract.json
if-no-files-found: error
include-hidden-files: true
retention-days: 1
- name: Skip prospective release contract
if: steps.changes.outputs.run != 'true'
run: echo "Skipping prospective release contract for non-code changes."
tests-windows:
runs-on: windows-latest
timeout-minutes: 10
strategy:
fail-fast: false
matrix:
# Keep Python 3.13 here because Daytona's locked obstore dependency has no Windows Python 3.14 wheel, so 3.14 builds it from source and exceeds this job's timeout.
python-version:
- "3.10"
- "3.13"
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect code changes
id: changes
shell: bash
run: ./.github/scripts/detect-changes.sh code "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: ${{ matrix.python-version }}
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: uv sync --all-extras --all-packages --group dev
@@ -133,19 +370,23 @@ jobs:
build-docs:
runs-on: ubuntu-latest
timeout-minutes: 10
env:
OPENAI_API_KEY: fake-for-tests
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Detect docs changes
id: changes
run: ./.github/scripts/detect-changes.sh docs "${{ github.event.pull_request.base.sha || github.event.before }}" "${{ github.sha }}"
- name: Setup uv
if: steps.changes.outputs.run == 'true'
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # setup-uv v9.0.0; uv 0.11.14
with:
version: "0.11.14"
enable-cache: true
prune-cache: true
python-version: "3.14"
- name: Install dependencies
if: steps.changes.outputs.run == 'true'
run: make sync
-88
View File
@@ -1,88 +0,0 @@
name: "Update Translated Docs"
# This GitHub Actions job automates the process of updating all translated document pages. Please note the following:
# 1. The translation results may vary each time; some differences in detail are expected.
# 2. When you add a new page to the left-hand menu, **make sure to manually update mkdocs.yml** to include the new item.
# 3. If you switch to a different LLM (for example, from o3 to a newer model), be sure to conduct thorough testing before making the switch.
# To add more languages, you will update the following:
# 1. Add '!docs/{lang}/**' to `on.push.paths` in this file
# 2. Update mkdocs.yml to have the new language
# 3. Update docs/scripts/translate_docs.py to have the new language
on:
push:
branches:
- main
paths:
- 'docs/**'
- mkdocs.yml
- '!docs/ja/**'
- '!docs/ko/**'
- '!docs/zh/**'
workflow_dispatch:
inputs:
translate_mode:
description: "Translation mode"
type: choice
options:
- only-changes
- full
default: only-changes
permissions:
contents: write
pull-requests: write
jobs:
update-docs:
if: "!contains(github.event.head_commit.message, 'Update all translated document pages')"
name: Build and Push Translated Docs
runs-on: ubuntu-latest
timeout-minutes: 30
env:
PROD_OPENAI_API_KEY: ${{ secrets.PROD_OPENAI_API_KEY }}
steps:
- name: Checkout repository
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd
with:
fetch-depth: 0
- name: Setup uv
uses: astral-sh/setup-uv@cec208311dfd045dd5311c1add060b2062131d57
with:
enable-cache: true
- name: Install dependencies
run: make sync
- name: Build translated docs
run: |
mode="${{ inputs.translate_mode || 'only-changes' }}"
uv run docs/scripts/translate_docs.py --mode "$mode"
uv run mkdocs build
- name: Commit changes
id: commit
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add docs/
if git diff --cached --quiet; then
echo "No changes to commit"
echo "committed=false" >> "$GITHUB_OUTPUT"
else
git commit -m "Update all translated document pages"
echo "committed=true" >> "$GITHUB_OUTPUT"
fi
- name: Create Pull Request
if: steps.commit.outputs.committed == 'true'
uses: peter-evans/create-pull-request@c0f553fe549906ede9cf27b5156039d195d2ece0
with:
commit-message: "Update translated document pages"
title: "docs: update translated document pages"
body: |
Automated update of translated documentation.
Triggered by commit: [${{ github.event.head_commit.id }}](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.event.head_commit.id }}).
Message: `${{ github.event.head_commit.message }}`
branch: update-translated-docs-${{ github.run_id }}
delete-branch: true
+5
View File
@@ -143,6 +143,10 @@ cython_debug/
# Ruff stuff:
.ruff_cache/
# Example runtime state
examples/sandbox/extensions/daytona/usaspending_text2sql/.audit_log.jsonl
examples/sandbox/extensions/daytona/usaspending_text2sql/.session_state.json
# PyPI configuration file
.pypirc
.aider*
@@ -154,3 +158,4 @@ tmp/
# execplans
plans/
.vercel
+122 -34
View File
@@ -9,11 +9,14 @@ This guide helps new contributors get started with the OpenAI Agents Python repo
1. [Policies & Mandatory Rules](#policies--mandatory-rules)
2. [Project Structure Guide](#project-structure-guide)
3. [Operation Guide](#operation-guide)
4. [Code Review Rules](#code-review-rules)
## Policies & Mandatory Rules
### Mandatory Skill Usage
Repository skills are stored under `.agents/skills/`. A reference such as `$<skill-name>` in this file is a repository instruction reference, not a request for manual user invocation. When a rule requires a skill, read `.agents/skills/<skill-name>/SKILL.md` completely before taking task actions, follow its instructions, and resolve referenced files relative to that skill directory.
#### `$code-change-verification`
Run `$code-change-verification` before marking work complete when changes affect runtime code, tests, or build/test behavior.
@@ -26,19 +29,75 @@ Run it when you change:
You can skip `$code-change-verification` for docs-only or repo-meta changes (for example, `docs/`, `.agents/`, `README.md`, `AGENTS.md`, `.github/`), unless a user explicitly asks to run the full verification stack.
Treat `$code-change-verification` as the post-review final gate, not as an iterative review check. When `$implementation-final-review` applies, satisfy its clean-review condition before starting the repository-wide format, lint, typecheck, and test stack. Immediately before starting that stack, use available read-only task or process evidence to check for another broad test, typecheck, build, examples, or integration command already running on the same host. When concrete contention is visible, keep making progress on review, remediation, evidence preparation, or focused checks and defer the broad stack until capacity is available. Do not add a repository lock, host-wide mutex, sentinel file, or user-triggered `finalize` step. Lack of host telemetry alone is not a blocker.
#### `$openai-knowledge`
When working on OpenAI API or OpenAI platform integrations in this repo (Responses API, tools, streaming, Realtime API, auth, models, rate limits, MCP, Agents SDK or ChatGPT Apps SDK), use `$openai-knowledge` to pull authoritative docs via the OpenAI Developer Docs MCP server (and guide setup if it is not configured).
#### `$implementation-strategy`
Before changing runtime code, exported APIs, external configuration, persisted schemas, wire protocols, or other user-facing behavior, use `$implementation-strategy` to decide the compatibility boundary and implementation shape. Judge breaking changes against the latest release tag, not unreleased branch-local churn. Interfaces introduced or changed after the latest release tag may be rewritten without compatibility shims unless they define a released or explicitly supported durable external state boundary, or the user explicitly asks for a migration path. Unreleased persisted formats on `main` may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported.
Before changing or reviewing runtime code, exported APIs, external configuration, persisted schemas, wire protocols, or other user-facing behavior, use `$implementation-strategy` to decide the compatibility boundary and implementation shape. Before coding, write an implementation scope contract that states the required behavior, compatibility requirements, intentionally unsupported cases and their failure behavior, and an already-supported alternative for those cases or that none exists. Treat this contract as a short, updateable engineering decision record, not as a new public API promise. During review, use the skill before requesting compatibility layers, migrations, new abstractions, or broader refactors.
Repeat the skill before editing each new review-feedback batch; an earlier strategy decision is stale when a comment would widen the supported contract or add another compatibility branch, resolver condition, or test permutation. Judge breaking changes against the latest release tag, not unreleased branch-local churn. Interfaces introduced or changed after the latest release tag may be rewritten without compatibility shims unless they define a released or explicitly supported durable external state boundary, or the user explicitly asks for a migration path. Unreleased persisted formats on `main` may be renumbered or squashed before release when intermediate snapshots are intentionally unsupported.
Independent reviewers dispatched by `$implementation-final-review` inherit the implementer's recorded implementation scope contract and do not rerun `$implementation-strategy` in their fresh review contexts. They report inconsistent or decision-incomplete strategy evidence as uncertainty to the implementer. The implementer remains responsible for rerunning `$implementation-strategy` before any review-feedback batch that widens the supported contract, adds a compatibility branch, changes ownership or protocol behavior, expands test permutations, or triggers a complexity reset.
#### `$implementation-final-review`
After implementing runtime code, tests, examples, build/test behavior, or behavior-impacting docs and completing focused tests, run `$implementation-final-review` before final `$code-change-verification` and `$pr-draft-summary` work and before declaring the task complete. Do not start repository-wide lint, typecheck, tests, builds, examples, or integration suites while the independent review is incomplete or finding-bearing. This repository instruction authorizes automatic invocation without a separate user mention. Do not invoke it for planning, investigation, review, or report-only tasks, repo-meta changes, or docs without behavior impact. The skill's clean-review gate does not replace any other mandatory repository skill or verification gate.
#### `$pr-draft-summary`
When a task in this repo finishes with moderate-or-larger code changes, invoke `$pr-draft-summary` in the final handoff to generate the required PR summary block, branch suggestion, title, and draft description. Treat this as the default close-out step after runtime code, tests, examples, build/test configuration, or docs with behavior impact are changed.
Before every final response for a task that changed runtime code, tests, examples, build/test configuration, or docs with behavior impact, invoke `$pr-draft-summary` to generate the required PR summary block, branch suggestion, title, and draft description. Determine whether to invoke it from the changed files, not from a subjective assessment of change size.
Skip `$pr-draft-summary` only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, or when the user explicitly says not to include the PR draft block.
Skip `$pr-draft-summary` only for trivial or conversation-only tasks, repo-meta/doc-only tasks without behavior impact, an explicitly invoked `$release-candidate-prep` handoff that uses the complete `$final-release-review` report as its release-specific PR description, or when the user explicitly says not to include the PR draft block. The release exception applies to preparing the candidate itself, not to implementing or changing the release-preparation skill.
Producing the PR draft block is part of the local final handoff. It is required for eligible local-only or uncommitted changes and does not authorize creating a branch, committing, pushing, or opening a pull request.
#### `$release-candidate-prep`
Use `$release-candidate-prep` only when the user explicitly invokes it with a release version. It keeps the user's clean `main` checkout unchanged, creates a dedicated detached worktree at refreshed `origin/main`, runs the readiness gates there, creates `release/v<version>` in that worktree, updates `pyproject.toml` and `uv.lock`, freezes and checks `tests/fixtures/released_api_contract.json`, and creates one local release commit. It invokes `$final-release-review` as the controlling checker against both the pre-release source and the materialized candidate; a blocked release call stops the workflow, while a green final-candidate report becomes the release-specific PR description.
The skill replaces the former GitHub Actions release-PR creator. It must never push, open or edit a pull request, create a release, or mutate any other GitHub state. It leaves the dedicated worktree in place for green handoff, blocked review, or recoverable failure. Release tag creation and PyPI publication remain owned by their post-merge workflows. The release commit may contain only `pyproject.toml`, `uv.lock`, and `tests/fixtures/released_api_contract.json`; all runtime and documentation changes must land on `main` before preparation.
### Work Status Reporting
- Use `RUNNING` only in commentary while autonomous work remains and no user action is required. Do not end a turn with a final response that says the task is still running or asks the user to send a generic continuation prompt.
- Use `COMPLETE` in the final response only when the requested work and every applicable review, verification, and local handoff step are complete.
- Use `NEEDS_DECISION` in the final response only when progress requires a concrete user choice, expanded authority, or an unresolved external condition. State the exact decision or condition instead of asking the user to say "continue".
### Git Worktree and Branch Safety
Work in the user's current checkout and on the current branch by default. If the Codex task is already running in a selected Git worktree, use that worktree without requesting additional permission. Do not create or switch to another Git worktree, and do not create or switch branches, unless the user explicitly asks for or approves that exact action in the current conversation. A request to implement, investigate, review, test, or verify changes does not by itself authorize changing the active worktree or branch.
If isolation or a different checkout is needed, explain why and ask the user before changing Git state. This requirement also applies when another rule or workflow recommends a linked worktree: stop and request approval instead of choosing or creating one automatically.
### Documentation Release Timing
When a feature or bug fix introduces behavior that is not yet available in the latest published release, do not include `docs/` changes that describe that unreleased behavior in the feature or bug-fix pull request, and do not expect those changes as part of that pull request. Handle them in a separate docs-only pull request so maintainers can coordinate its merge timing with the release that makes the documentation accurate. This exception applies only when the documentation would be incorrect for the latest published release; documentation that is already accurate for released behavior remains part of the normal change scope.
### Documentation Verification Tiers
Classify documentation changes before choosing review and verification work. Use the narrowest tier that covers the complete diff, and move to a higher tier when any changed file or claim requires it.
- **Editorial:** Terminology, spelling, punctuation, formatting, or link-label changes that do not change documented behavior, runnable code, navigation, link targets, anchors, or generated reference content. Inspect the diff, run targeted searches for the corrected text, and run `git diff --check`. Check a link or anchor directly only when the edit can affect it. Skip `$implementation-final-review`, cross-language review, and `make build-docs` for this tier.
- **Content:** New or materially rewritten behavioral guidance, migration instructions, or runnable snippets that do not change documentation structure or tooling. Verify claims against the implementation and authoritative sources, execute or otherwise validate changed snippets when practical, perform the required focused cross-language review, and run `make build-docs` once after the content and review are stable. Do not repeat the full site build after edits that cannot affect its result.
- **Structural:** Added, removed, renamed, or moved pages; changes to `mkdocs.yml`, generated API reference inputs, documentation scripts, plugins, or build configuration. Run the relevant generators or focused tooling checks and `make build-docs` after the structure is stable. Apply `$code-change-verification` when the changed file is build or test configuration covered by that skill.
Existing warnings from a successful documentation build are not findings for an unrelated docs change. Evaluate the exit status and identify new errors, broken references, or warnings caused by the diff instead of reviewing the complete warning stream line by line. Reserve `make build-full-docs` and generated translation output for translation-tooling changes, explicit localization work, or a specifically requested broad localization audit.
### Scope Discipline and Complexity Reset
- Implement the narrowest explicitly stated set of behaviors that satisfies the request. Do not interpret every shape accepted by a host-language protocol, third-party library, or reflection API unless those shapes are required by the task or supported behavior shipped in the latest release.
- Prefer adapting the required case into an existing pipeline over creating a parallel contract, resolver, execution path, or source of truth. Continue to derive schema, validation, naming, documentation, and invocation from the existing source-of-truth functions, types, or modules.
- Every new abstraction, state field, cached classification, compatibility branch, or dispatch mode must map to a stated requirement, released contract, durable boundary, or verified runtime risk. Remove it if that mapping cannot be stated concretely.
- Treat a second related review finding that would add another condition, protocol hop, compatibility case, or test permutation to the same abstraction as a mandatory complexity-reset checkpoint, not another item to patch. Continue the design only when concrete evidence shows that the additional case belongs to the supported contract.
- When that signal appears, stop extending the current design. Re-read the original requirement, group all findings by root cause, compare the complete diff with the merge base of the intended target branch or with the latest release tag when it is the compatibility baseline, and replace branch-local machinery with a narrower contract. Existing unreleased code and tests are not sunk costs. Perform this reset proactively; do not wait for the user or reviewer to request it.
- A released-version reproducer proves reachability, not a supported contract. Verify the exact shape against documentation, tests, examples, intentional public typing, explicit maintainer intent, or concrete user reliance before adding compatibility machinery.
- Prefer an actionable error during construction or validation, before invocation or other side effects, and an existing supported alternative (for example a wrapper function, explicit override, or typed adapter) over partially emulating a broad protocol. Do not add another alternative when an adequate supported one already exists.
- A growing diff is not itself proof of overengineering, but unexpected cross-module spread, duplicated metadata, combinatorial tests, or repeated special cases requires restarting the design review from the original requirement before more code is added.
- Before handoff, verify that the patch has one source of truth per concern, tests the required behavior and intentionally unsupported cases, and does not accidentally make every constructible combination part of the supported SDK behavior.
### ExecPlans
@@ -46,7 +105,7 @@ Call out compatibility risk early in your plan only when the change affects beha
Use an ExecPlan when work is multi-step, spans several files, involves new features or refactors, or is likely to take more than about an hour. Start with the template and rules in `PLANS.md`, keep milestones and living sections (Progress, Surprises & Discoveries, Decision Log, Outcomes & Retrospective) up to date as you execute, and rewrite the plan if scope shifts. Call out compatibility risk only when the plan changes behavior shipped in the latest release tag or a released or explicitly supported durable external state boundary. Do not treat branch-local interface churn or unreleased post-tag changes on `main` as breaking by default; prefer direct replacement over compatibility layers in those cases, and renumber or squash unreleased persisted schemas before release when the intermediate snapshots are intentionally unsupported. If you intentionally skip an ExecPlan for a complex task, note why in your response so reviewers understand the choice.
### Public API Positional Compatibility
### Public API Compatibility
Treat the parameter and dataclass field order of exported runtime APIs as a compatibility contract.
@@ -54,6 +113,20 @@ Treat the parameter and dataclass field order of exported runtime APIs as a comp
- When adding a new optional public field/parameter, append it to the end whenever possible and keep old fields in the same order.
- If reordering is unavoidable, add an explicit compatibility layer and regression tests that exercise the old positional call pattern.
- Prefer keyword arguments at call sites to reduce accidental breakage, but do not rely on this to justify breaking positional compatibility for public APIs.
- Treat intended import paths and `__all__` membership as compatibility contracts. When adding or moving a public symbol, update the owning module, intended top-level or subpackage re-exports, and an import regression test. Keep top-level imports free of optional-dependency failures and runtime side effects; use lazy exports when needed.
### Platform, Docs, and Security Review
- Treat translation-safe English as a documentation compatibility requirement. In new or materially rewritten translatable prose under `docs/` (excluding generated API reference pages), state the actor, scope, ownership, ordering, modality, and lifecycle boundary explicitly whenever they affect the meaning. Use exact API identifiers in inline code, and replace ambiguous pronouns, overloaded nouns, or shorthand when a small clarification can prevent a materially different translation. Do not change the documented behavior merely to make a sentence easier to translate.
- For new or materially rewritten translatable prose, use a lightweight cross-language review of only the changed English sentences and their immediate context. Have an independent reviewer or review pass inspect the source from Japanese, Korean, and Chinese translation perspectives and report only concrete risks such as an ambiguous actor, scope, ownership, ordering, modality, lifecycle boundary, overloaded SDK term, or identifier corruption. Resolve concrete findings in the English source and review the revised lines once. Do not generate full localized pages for routine documentation changes. Pure link, formatting, typo, and other edits that do not change translatable meaning may skip this review.
- If a concrete concern cannot be resolved confidently from the English source, use a temporary translation of only the disputed sentence or paragraph as a focused probe; do not write or commit generated localized files. Reserve `docs/scripts/translate_docs.py --mode full --file <path>` and broader Japanese, Korean, and Chinese output review for changes to the translation tooling or translation controls, explicit localization work, or an explicitly requested broad translation audit. Add or change a fixed translation mapping only when actual cross-document evidence shows that one stable target term is correct across contexts. Prefer contextual guidance and established target-language developer terminology, including standard English terms, over a large or rigid mapping table.
- Treat runnable docs snippets as API compatibility checks. Before adding OpenAI API, provider, Responses, Realtime, WebSocket, or SDK constructor examples, verify the shown arguments and call shape against the actual implementation.
- When adding or updating code in `examples/` or runnable `docs/` snippets, import Agents SDK decorators from `agents.decorators`. Prefer `tool` over `function_tool`; keep non-decorator SDK imports on their existing public import paths.
- Do not let untrusted sandbox manifests opt themselves out of host filesystem or base-directory boundaries. Escape hatches for local source materialization must be controlled by trusted application code at the call site, not by serialized manifest data.
- When documenting sandbox or security grants, verify the actual implementation path enforces the grant or boundary. Do not claim a grant applies to `LocalDir`, `LocalFile`, archive extraction, or other materialization paths unless those paths actually consult it.
- When redacting OpenAI tool, MCP, model, or provider payloads, consider traceback display, exception chaining, `__context__`, logs, and telemetry. Suppressing display with `raise ... from None` is not enough if the original exception object still carries sensitive input data.
- For OpenAI platform or SDK-specific docs changes, prefer `$openai-knowledge` for authoritative platform behavior and inspect the local code path for SDK behavior. Do not rely on generic API assumptions when documenting Responses, Chat Completions, Realtime, tools, MCP, or provider adapters.
- For Realtime tracing changes, read [Realtime tracing architecture](.agents/references/realtime-tracing.md) before proposing SDK spans. Realtime API server traces and Agents SDK client traces are separate; `group_id` can correlate them but does not create a shared trace hierarchy.
## Project Structure Guide
@@ -72,26 +145,28 @@ The OpenAI Agents Python repository provides the Python Agents SDK, examples, an
- `Makefile`: Common developer commands.
- `pyproject.toml`, `uv.lock`: Python dependencies and tool configuration.
- `.github/PULL_REQUEST_TEMPLATE/pull_request_template.md`: Pull request template to use when opening PRs.
- `.agents/references/`: Durable SDK maintainer architecture references. Start with [the reference map](.agents/references/README.md) and open only the files relevant to the affected runtime boundary.
- `site/`: Built documentation output.
### Agents Core Runtime Guidelines
- For `Agent` fields, cloning, dynamic instructions, enabled tools or handoffs, output schemas, run context wrappers, usage aggregation, or public-versus-internal agent identity, read [Agent definition and run context](.agents/references/agent-definition-and-run-context.md).
- `src/agents/run.py` is the runtime entrypoint (`Runner`, `AgentRunner`). Keep it focused on orchestration and public flow control. Put new runtime logic under `src/agents/run_internal/` and import it into `run.py`.
- When `run.py` grows, refactor helpers into `run_internal/` modules (for example `run_loop.py`, `turn_resolution.py`, `tool_execution.py`, `session_persistence.py`) and leave only wiring and composition in `run.py`.
- Keep streaming and non-streaming paths behaviorally aligned. Changes to `run_internal/run_loop.py` (`run_single_turn`, `run_single_turn_streamed`, `get_new_response`, `start_streaming`) should be mirrored, and any new streaming item types must be reflected in `src/agents/stream_events.py`.
- Input guardrails run only on the first turn and only for the starting agent. Resuming an interruption from `RunState` must not increment the turn counter; only actual model calls advance turns.
- Server-managed conversation (`conversation_id`, `previous_response_id`, `auto_previous_response_id`) uses `OpenAIServerConversationTracker` in `run_internal/oai_conversation.py`. Only deltas should be sent. If `call_model_input_filter` is used, it must return `ModelInputData` with a list input and the tracker must be updated with the filtered input (`mark_input_as_sent`). Session persistence is disabled when server-managed conversation is active.
- Adding new tool/output/approval item types requires coordinated updates across:
- `src/agents/items.py` (RunItem types and conversions)
- `src/agents/run_internal/run_steps.py` (ProcessedResponse and tool run structs)
- `src/agents/run_internal/turn_resolution.py` (model output processing, run item extraction)
- `src/agents/run_internal/tool_execution.py` and `src/agents/run_internal/tool_planning.py`
- `src/agents/run_internal/items.py` (normalization, dedupe, approval filtering)
- `src/agents/stream_events.py` (stream event names)
- `src/agents/run_state.py` (RunState serialization/deserialization)
- `src/agents/run_internal/session_persistence.py` (session save/rewind)
- If the serialized RunState shape changes, update `CURRENT_SCHEMA_VERSION` in `src/agents/run_state.py` and the related serialization/deserialization logic. Keep released schema versions readable, and feel free to renumber or squash unreleased schema versions before release when those intermediate snapshots are intentionally unsupported.
- When bumping `CURRENT_SCHEMA_VERSION`, also add or update the matching entry in `SCHEMA_VERSION_SUMMARIES` in `src/agents/run_state.py` so every supported version keeps a short historical note describing what changed in that schema.
- For turn accounting, guardrail ordering, handoffs, interruptions, cancellation, hooks, or streaming behavior, read [Runner lifecycle](.agents/references/runner-lifecycle.md). Keep streaming and non-streaming paths behaviorally aligned.
- For new model output, tool call, approval, or run item variants, read [Run item lifecycle](.agents/references/run-item-lifecycle.md) and update every applicable processing, event, replay, persistence, tracing, and serialization surface.
- For function-tool parameter schemas, `Annotated` or `Field` metadata, strict JSON schema conversion, or structured output schemas, read [Function and output schema](.agents/references/function-and-output-schema.md).
- For function-tool naming, namespacing, lookup, approvals, tracing, or call-ID changes, read [Tool identity and routing](.agents/references/tool-identity.md) and use the canonical helpers in `src/agents/_tool_identity.py` instead of adding local normalization rules.
- For function-tool planning, approval ordering, tool guardrails, concurrency, cancellation, timeouts, hooks, or failure conversion, read [Tool execution lifecycle](.agents/references/tool-execution-lifecycle.md).
- For local MCP connection ownership, `MCPServerManager`, request serialization, tool caching or filtering, transport retries, cancellation, or cleanup, read [Local MCP server lifecycle](.agents/references/local-mcp-server-lifecycle.md).
- For trace or span context, processors, export, flush, shutdown, sensitive data, or resumed trace state, read [Tracing lifecycle](.agents/references/tracing-lifecycle.md).
- For `RealtimeSession` lifecycle, background-task, handoff, listener, connection, or cleanup changes, read [Realtime session lifecycle](.agents/references/realtime-session-lifecycle.md) and verify both normal and failure-path resource ownership.
- For `VoicePipeline`, streamed audio input, STT session ownership, TTS task ordering, voice lifecycle events, PCM framing, or voice tracing changes, read [Voice pipeline lifecycle](.agents/references/voice-pipeline-lifecycle.md).
- For server-managed conversation (`conversation_id`, `previous_response_id`, `auto_previous_response_id`), read [Conversation state ownership](.agents/references/conversation-state-ownership.md) before changing continuation, filtering, retry, compaction, handoffs, or resume behavior.
- For client-managed session input, per-turn saves, retry rewind, backend atomicity, or compaction replacement, read [Session persistence](.agents/references/session-persistence.md).
- For model resolution, `ModelSettings`, provider adapters, Responses versus Chat Completions capabilities, request conversion, terminal events, transport reuse, or model retries, read [Model and provider boundaries](.agents/references/model-provider-boundaries.md).
- If the serialized `RunState` shape changes, read [RunState schema and resume boundary](.agents/references/runstate-schema.md) and follow its release-boundary, schema-version, backward-read, and regression-test rules.
- For sandbox session ownership, agent preparation, manifests, host-path materialization, snapshots, resume state, or cleanup, read [Sandbox runtime boundary](.agents/references/sandbox-runtime-boundary.md).
## Operation Guide
@@ -103,25 +178,27 @@ The OpenAI Agents Python repository provides the Python Agents SDK, examples, an
### Development Workflow
1. Sync with `main` and create a feature branch:
1. Stay in the user's current checkout and on the current branch unless the user explicitly asks for or approves a Git state change.
2. If the user explicitly requests a feature/fix branch, create one with a descriptive name:
```bash
git checkout -b feat/<short-description>
```
2. If dependencies changed or you are setting up the repo, run `make sync`.
3. Implement changes and add or update tests alongside code updates.
4. Highlight compatibility or API risks in your plan before implementing changes that alter the latest released behavior or a released or explicitly supported durable external state boundary.
5. Build docs when you touch documentation:
```bash
make build-docs
```
6. When `$code-change-verification` applies, run it to execute the full verification stack before marking work complete.
7. Commit with concise, imperative messages; keep commits small and focused, then open a pull request.
8. When reporting code changes as complete (after substantial code work), invoke `$pr-draft-summary` as the final handoff step unless the task falls under the documented skip cases.
3. If dependencies changed or you are setting up the repo, run `make sync`.
4. Implement changes and add or update tests alongside code updates.
5. Highlight compatibility or API risks in your plan before implementing changes that alter the latest released behavior or a released or explicitly supported durable external state boundary.
6. Verify documentation changes according to [Documentation Verification Tiers](#documentation-verification-tiers). Do not run a full documentation build for an editorial-only change.
7. When `$code-change-verification` applies, run it to execute the full verification stack before marking work complete.
8. Commit with concise, imperative messages; keep commits small and focused, then open a pull request.
9. Before reporting eligible code changes as complete, invoke `$pr-draft-summary` as the final handoff step unless the task falls under the documented skip cases. Do not omit it based on perceived change size or because the work remains local or uncommitted.
### Testing & Automated Checks
Before submitting changes, ensure relevant checks pass and extend tests when you touch code.
For provider-neutral agent workflow tests, prefer `ScriptedModel` from `agents.testing` over adding a new mock or fake `Model`. Prefer `ScriptedRealtimeModel` from `agents.realtime.testing` for Realtime session tests, the scripted utilities from `agents.voice.testing` for Voice pipeline tests, and `scripted_sandbox_session()` from `agents.testing` for deterministic Sandbox session calls. Keep a specialized test double only when the test specifically requires provider-wire conversion, malformed streams, controlled suspension or concurrency, or an exact cancellation or lifecycle boundary that the scripted utilities cannot preserve; document that boundary in the test.
Before adding or changing async, retry, timeout, subprocess, PTY, warning, or xdist-sensitive tests, read [Performance and determinism](tests/README.md#performance-and-determinism) and preserve the applicable behavioral and lifecycle coverage while optimizing execution.
When `$code-change-verification` applies, run it to execute the required verification stack from the repository root. Rerun the full stack after applying fixes.
#### Unit tests and type checking
@@ -165,6 +242,7 @@ Some tests rely on inline snapshots; see `tests/README.md` for details. Re-run `
- Type hints must pass `make typecheck`.
- Write comments as full sentences ending with a period.
- Imports are managed by Ruff and should stay sorted.
- Do not hard-wrap prose in Markdown or other non-code text files at a fixed column width. Keep each paragraph on one source line unless the file format or Markdown structure requires a line break, such as for lists, tables, blockquotes, or code fences.
#### Mandatory local run order
@@ -190,9 +268,9 @@ make tests
```
- Documentation workflows:
```bash
make build-docs # build docs after editing docs
make build-docs # build stable content or structural docs changes
make serve-docs # preview docs locally
make build-full-docs # run translations and build
make build-full-docs # run translations and build when explicitly required
```
- Snapshot helpers:
```bash
@@ -208,15 +286,25 @@ make tests
### Pull Request & Commit Guidelines
- Use the template at `.github/PULL_REQUEST_TEMPLATE/pull_request_template.md`; include a summary, test plan, and issue number if applicable.
- Add tests for new behavior when feasible and update documentation for user-facing changes.
- In copy-ready GitHub text, use native issue and pull-request references: exactly `#123` for this repository and `owner/repo#123` for another repository. Do not qualify same-repository references as `openai/openai-agents-python#123`. Preserve closing forms such as `Fixes #123` or `Resolves #123`. Never wrap these references in Markdown links such as `[PR #123](https://github.com/owner/repo/pull/123)` or `[#123](...)`; those Codex-friendly links require manual cleanup after pasting into GitHub. Use descriptive Markdown links only for external resources or GitHub targets that cannot be expressed as a native issue or pull-request reference.
- Add tests for new behavior when feasible. Update documentation for user-facing changes, except unreleased-behavior documentation that must follow the separate docs-only pull request policy above.
- Run `make format`, `make lint`, `make typecheck`, and `make tests` before marking work ready.
- Commit messages should be concise and written in the imperative mood. Small, focused commits are preferred.
### Review Process & What Reviewers Look For
## Code Review Rules
- Use `$implementation-strategy` to establish the requested outcome and latest released compatibility boundary before judging implementation scope or architecture.
- Treat added complexity as an actionable finding only when specific machinery is not required by the task, a released contract, supported durable state, or a verified runtime or platform risk. Identify the unnecessary machinery and recommend the smallest safe removal or direct replacement.
- Do not request speculative abstractions, general-purpose helpers, configuration knobs, dependencies, compatibility layers, feature flags, parallel code paths, or extensibility for hypothetical future consumers.
- Do not process a sequence of related review comments as independent local fixes when they expose the same missing boundary. Classify them together, decide whether the disputed shapes belong to the supported contract, and prefer one narrowing redesign over accumulating branches.
- Review the complete diff from the merge base of the intended target branch, or from the latest release tag when it is the compatibility baseline, not only the latest incremental fix. Passing tests do not justify branch-local machinery that no longer matches the original requirement.
- Keep findings scoped to the patch. Do not block on unrelated cleanup, pre-existing bugs, or optional refactors; report them separately when useful.
- Require a broader refactor only when concrete evidence shows the focused change would otherwise be incorrect, unsafe, incompatible, or materially harder to maintain.
### Baseline review expectations
- ✅ Checks pass (`make format`, `make lint`, `make typecheck`, `make tests`).
- ✅ Tests cover new behavior and edge cases.
- ✅ Code is readable, maintainable, and consistent with existing style.
- ✅ Public APIs and user-facing behavior changes are documented.
- ✅ Examples are updated if behavior changes.
- ✅ History is clean with a clear PR description.
+149 -6
View File
@@ -2,6 +2,35 @@
sync:
uv sync --all-extras --all-packages --group dev
.PHONY: update-rclone-pin
update-rclone-pin:
uv run python .github/scripts/update_rclone_pin.py --cooldown-days $(or $(RCLONE_COOLDOWN_DAYS),7) $(if $(RCLONE_VERSION),--version $(RCLONE_VERSION))
.PHONY: update-released-api-contract
update-released-api-contract:
@test -n "$(VERSION)" || (echo "VERSION is required, for example VERSION=0.20.0" >&2; exit 2)
uv run python .github/scripts/update_released_api_contract.py --version "$(VERSION)"
.PHONY: check-released-api-contract
check-released-api-contract:
@test -n "$(VERSION)" || (echo "VERSION is required, for example VERSION=0.20.0" >&2; exit 2)
uv run python .github/scripts/update_released_api_contract.py --version "$(VERSION)" --check
PROSPECTIVE_RELEASED_API_CONTRACT ?= .tmp/prospective_released_api_contract.json
.PHONY: prepare-prospective-released-api-contract
prepare-prospective-released-api-contract:
@unset OPENAI_API_KEY; \
version="$$(uv run python -c 'from importlib.metadata import version; print(version("openai-agents"))')"; \
uv run python .github/scripts/update_released_api_contract.py \
--version "$$version" \
--output "$(PROSPECTIVE_RELEASED_API_CONTRACT)"
.PHONY: check-prospective-released-api-contract
check-prospective-released-api-contract: prepare-prospective-released-api-contract
OPENAI_AGENTS_PROSPECTIVE_RELEASE_CONTRACT="$(abspath $(PROSPECTIVE_RELEASED_API_CONTRACT))" \
$(MAKE) integration-tests-prospective-contract
.PHONY: format
format:
uv run ruff format
@@ -14,14 +43,15 @@ format-check:
.PHONY: lint
lint:
uv run ruff check
uv run python .github/scripts/check_optional_truthiness.py src/agents
.PHONY: mypy
mypy:
uv run mypy . --exclude site
uv run mypy src
.PHONY: pyright
pyright:
uv run pyright --project pyrightconfig.json
uv run pyright --project pyrightconfig.json --threads "$${PYRIGHT_THREADS:-4}"
.PHONY: typecheck
typecheck:
@@ -35,9 +65,13 @@ typecheck:
wait $$mypy_pid; \
wait $$pyright_pid; \
trap - EXIT
.PHONY: tests
tests: tests-parallel tests-serial
tests: tests-parallel
$(MAKE) tests-serial
.PHONY: tests-review
tests-review: tests-parallel-review
$(MAKE) tests-serial-review
.PHONY: tests-asyncio-stability
tests-asyncio-stability:
@@ -45,11 +79,120 @@ tests-asyncio-stability:
.PHONY: tests-parallel
tests-parallel:
uv run pytest -n auto --dist loadfile -m "not serial"
uv run pytest -n "$${PYTEST_XDIST_AUTO_NUM_WORKERS:-auto}" $(if $(PYTEST_XDIST_AUTO_NUM_WORKERS),,--maxprocesses=9) --dist worksteal -m "not serial"
.PHONY: tests-parallel-review
tests-parallel-review:
uv run pytest -n "$${PYTEST_XDIST_AUTO_NUM_WORKERS:-auto}" $(if $(PYTEST_XDIST_AUTO_NUM_WORKERS),,--maxprocesses=9) --dist worksteal -m "not serial and not review_optional"
.PHONY: tests-serial
tests-serial:
uv run pytest -m serial
uv run python .github/scripts/run_serial_tests.py
.PHONY: tests-serial-review
tests-serial-review:
uv run python .github/scripts/run_serial_tests.py --exclude-review-optional
EXAMPLES_RUNNER := bash .github/scripts/run_examples.sh
EXAMPLES_ARGS ?=
EXAMPLES_LOG ?=
INTEGRATION_TEST_RUNNER := python .github/scripts/run_integration_tests.py
.PHONY: examples-run
examples-run:
$(EXAMPLES_RUNNER) start $(EXAMPLES_ARGS)
.PHONY: examples-run-background
examples-run-background:
$(EXAMPLES_RUNNER) start --background $(EXAMPLES_ARGS)
.PHONY: examples-status
examples-status:
$(EXAMPLES_RUNNER) status
.PHONY: examples-stop
examples-stop:
$(EXAMPLES_RUNNER) stop
.PHONY: examples-logs
examples-logs:
$(EXAMPLES_RUNNER) logs
.PHONY: examples-tail
examples-tail:
$(EXAMPLES_RUNNER) tail $(EXAMPLES_LOG)
.PHONY: integration-tests
integration-tests:
$(INTEGRATION_TEST_RUNNER) --profile full $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-release
integration-tests-release:
$(INTEGRATION_TEST_RUNNER) --profile release $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-nightly
integration-tests-nightly:
$(INTEGRATION_TEST_RUNNER) --profile nightly $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-manual
integration-tests-manual:
$(INTEGRATION_TEST_RUNNER) --profile manual $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-packaging
integration-tests-packaging:
$(INTEGRATION_TEST_RUNNER) --profile packaging
.PHONY: integration-tests-prospective-contract
integration-tests-prospective-contract:
$(INTEGRATION_TEST_RUNNER) --profile prospective-contract
.PHONY: integration-tests-prospective-platform
integration-tests-prospective-platform:
$(INTEGRATION_TEST_RUNNER) --profile prospective-platform
.PHONY: integration-tests-security
integration-tests-security:
$(INTEGRATION_TEST_RUNNER) --profile security
.PHONY: integration-tests-mcp-v1
integration-tests-mcp-v1:
$(INTEGRATION_TEST_RUNNER) --profile mcp-v1
.PHONY: integration-tests-core
integration-tests-core:
$(INTEGRATION_TEST_RUNNER) --profile core
.PHONY: integration-tests-providers
integration-tests-providers:
$(INTEGRATION_TEST_RUNNER) --profile providers $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-providers-external
integration-tests-providers-external:
OPENAI_AGENTS_INTEGRATION_EXTERNAL_PROVIDERS=1 $(INTEGRATION_TEST_RUNNER) --profile providers $(filter --all,$(MAKECMDGOALS))
.PHONY: integration-tests-providers-all
integration-tests-providers-all:
$(INTEGRATION_TEST_RUNNER) --profile providers --all
.PHONY: --all
--all:
@:
.PHONY: integration-tests-realtime
integration-tests-realtime:
$(INTEGRATION_TEST_RUNNER) --profile realtime
.PHONY: integration-tests-voice
integration-tests-voice:
$(INTEGRATION_TEST_RUNNER) --profile voice
.PHONY: integration-tests-hosted
integration-tests-hosted:
$(INTEGRATION_TEST_RUNNER) --profile hosted
.PHONY: integration-tests-extras
integration-tests-extras:
$(INTEGRATION_TEST_RUNNER) --profile extras
.PHONY: coverage
coverage:
+86 -14
View File
@@ -10,14 +10,15 @@ The OpenAI Agents SDK is a lightweight yet powerful framework for building multi
### Core concepts:
1. [**Agents**](https://openai.github.io/openai-agents-python/agents): LLMs configured with instructions, tools, guardrails, and handoffs
1. [**Sandbox Agents**](https://openai.github.io/openai-agents-python/sandbox_agents): Agents preconfigured to work with a container to perform work over long time horizons.
1. [**Sandbox agents**](https://openai.github.io/openai-agents-python/sandbox_agents): Agents preconfigured to work with a container to perform work over long time horizons.
1. [**Realtime agents**](https://openai.github.io/openai-agents-python/realtime/quickstart/): Build powerful voice agents with `gpt-realtime-2.1` and full agent features
1. [**Voice agents**](https://openai.github.io/openai-agents-python/voice/quickstart/): Build voice pipelines that combine speech-to-text, an agent workflow, and text-to-speech
1. **[Agents as tools](https://openai.github.io/openai-agents-python/tools/#agents-as-tools) / [Handoffs](https://openai.github.io/openai-agents-python/handoffs/)**: Delegating to other agents for specific tasks
1. [**Tools**](https://openai.github.io/openai-agents-python/tools/): Various Tools let agents take actions (functions, MCP, hosted tools)
1. [**Guardrails**](https://openai.github.io/openai-agents-python/guardrails/): Configurable safety checks for input and output validation
1. [**Human in the loop**](https://openai.github.io/openai-agents-python/human_in_the_loop/): Built-in mechanisms for involving humans across agent runs
1. [**Sessions**](https://openai.github.io/openai-agents-python/sessions/): Automatic conversation history management across agent runs
1. [**Tracing**](https://openai.github.io/openai-agents-python/tracing/): Built-in tracking of agent runs, allowing you to view, debug and optimize your workflows
1. [**Realtime Agents**](https://openai.github.io/openai-agents-python/realtime/quickstart/): Build powerful voice agents with `gpt-realtime-1.5` and full agent features
Explore the [examples](https://github.com/openai/openai-agents-python/tree/main/examples) directory to see the SDK in action, and read our [documentation](https://openai.github.io/openai-agents-python/) for more details.
@@ -46,9 +47,34 @@ uv add openai-agents
For voice support, install with the optional `voice` group: `uv add 'openai-agents[voice]'`. For Redis session support, install with the optional `redis` group: `uv add 'openai-agents[redis]'`.
## Run your first Sandbox Agent
## Run your first agents
[Sandbox Agents](https://openai.github.io/openai-agents-python/sandbox_agents) are new in version 0.14.0. A sandbox agent is an agent that uses a computer environment to perform real work with a filesystem, in an environment you configure and control. Sandbox agents are useful when the agent needs to inspect files, run commands, apply patches, or carry workspace state across longer tasks.
The SDK supports four primary ways to run agents. Set the `OPENAI_API_KEY` environment variable before running any of these examples.
### Run a text agent
Use a text `Agent` for workflows that do not need a persistent realtime connection or a sandbox workspace.
```python
from agents import Agent, Runner
agent = Agent(name="Assistant", instructions="You are a helpful assistant")
result = Runner.run_sync(agent, "Write a haiku about recursion in programming.")
print(result.final_output)
# Code within the code,
# Functions calling themselves,
# Infinite loop's dance.
```
(_For Jupyter notebook users, see [hello_world_jupyter.ipynb](https://github.com/openai/openai-agents-python/blob/main/examples/basic/hello_world_jupyter.ipynb)_)
### Run a sandbox agent
Use a [`SandboxAgent`](https://openai.github.io/openai-agents-python/sandbox_agents) when the agent needs to inspect files, run commands, apply patches, or preserve workspace state across longer tasks.
This example uses `UnixLocalSandboxClient`, which is supported on macOS and Linux. On Windows, use `DockerSandboxClient` with the `openai-agents[docker]` extra or a hosted sandbox client instead; see [Sandbox clients](https://openai.github.io/openai-agents-python/sandbox/clients/) for setup details.
```python
from agents import Runner
@@ -60,27 +86,73 @@ from agents.sandbox.sandboxes import UnixLocalSandboxClient
agent = SandboxAgent(
name="Workspace Assistant",
instructions="Inspect the sandbox workspace before answering.",
default_manifest=Manifest(
entries={
"repo": GitRepo(repo="openai/openai-agents-python", ref="main"),
}
),
default_manifest=Manifest(entries={"repo": GitRepo(repo="openai/openai-agents-python", ref="main")}),
)
result = Runner.run_sync(
agent,
"Inspect the repo README and summarize what this project does.",
# Run this agent on the local filesystem
run_config=RunConfig(sandbox=SandboxRunConfig(client=UnixLocalSandboxClient())),
)
print(result.final_output)
# This project provides a Python SDK for building multi-agent workflows.
```
(_If running this, ensure you set the `OPENAI_API_KEY` environment variable_)
### Run a realtime agent
(_For Jupyter notebook users, see [hello_world_jupyter.ipynb](https://github.com/openai/openai-agents-python/blob/main/examples/basic/hello_world_jupyter.ipynb)_)
Use a [`RealtimeAgent`](https://openai.github.io/openai-agents-python/realtime/quickstart/) for low-latency, server-side voice and multimodal experiences over WebSocket.
```python
import asyncio
from agents.realtime import RealtimeAgent, RealtimeRunner
async def main() -> None:
agent = RealtimeAgent(name="Assistant", instructions="You are a helpful voice assistant. Keep responses short.")
runner = RealtimeRunner(starting_agent=agent)
session = await runner.run()
async with session:
await session.send_message("Say hello in one short sentence.")
async for event in session:
if event.type == "audio":
# Forward or play event.audio.data.
pass
elif event.type == "history_added":
print(event.item)
elif event.type == "agent_end":
break
if __name__ == "__main__":
asyncio.run(main())
```
### Run a voice agent
Use a [`VoicePipeline`](https://openai.github.io/openai-agents-python/voice/quickstart/) to turn audio into text, run an agent workflow, and stream generated speech.
```python
import asyncio
import numpy as np
from agents import Agent
from agents.voice import AudioInput, SingleAgentVoiceWorkflow, VoicePipeline
async def main() -> None:
agent = Agent(name="Assistant", instructions="You are a helpful voice assistant.")
pipeline = VoicePipeline(workflow=SingleAgentVoiceWorkflow(agent))
audio_input = AudioInput(buffer=np.zeros(24000 * 3, dtype=np.int16))
result = await pipeline.run(audio_input)
async for event in result.stream():
if event.type == "voice_stream_event_audio":
# Forward or play event.data.
pass
if __name__ == "__main__":
asyncio.run(main())
```
Explore the [examples](https://github.com/openai/openai-agents-python/tree/main/examples) directory to see the SDK in action, and read our [documentation](https://openai.github.io/openai-agents-python/) for more details.
+5
View File
@@ -0,0 +1,5 @@
# Security Policy
For a more in-depth look at our security policy, please check out our [Coordinated Vulnerability Disclosure Policy](https://openai.com/security/disclosure/#:~:text=Disclosure%20Policy,-Security%20is%20essential&text=OpenAI%27s%20coordinated%20vulnerability%20disclosure%20policy,expect%20from%20us%20in%20return.).
Our PGP key can be found [at this address](https://cdn.openai.com/security.txt).
+40 -25
View File
@@ -2,7 +2,7 @@
Agents are the core building block in your apps. An agent is a large language model (LLM) configured with instructions, tools, and optional runtime behavior such as handoffs, guardrails, and structured outputs.
Use this page when you want to define or customize a single plain `Agent`. If you are deciding how multiple agents should collaborate, read [Agent orchestration](multi_agent.md). If the agent should run inside an isolated workspace with manifest-defined files and sandbox-native capabilities, read [Sandbox agent concepts](sandbox/guide.md).
Use this page when you want to define or customize a single base `Agent` rather than a `SandboxAgent`. If you are deciding how multiple agents should collaborate, read [Agent orchestration](multi_agent.md). If the agent should run inside an isolated workspace with manifest-defined files and sandbox-native capabilities, read [Sandbox agent concepts](sandbox/guide.md).
The SDK uses the Responses API by default for OpenAI models, but the distinction here is orchestration: `Agent` plus `Runner` lets the SDK manage turns, tools, guardrails, handoffs, and sessions for you. If you want to own that loop yourself, use the Responses API directly instead.
@@ -28,15 +28,15 @@ The most common properties of an agent are:
| Property | Required | Description |
| --- | --- | --- |
| `name` | yes | Human-readable agent name. |
| `instructions` | yes | System prompt or dynamic instructions callback. See [Dynamic instructions](#dynamic-instructions). |
| `instructions` | no | System prompt or dynamic instructions callback. Strongly recommended. See [Dynamic instructions](#dynamic-instructions). |
| `prompt` | no | OpenAI Responses API prompt configuration. Accepts a static prompt object or a function. See [Prompt templates](#prompt-templates). |
| `handoff_description` | no | Short description exposed when this agent is offered as a handoff target. |
| `handoffs` | no | Delegate the conversation to specialist agents. See [handoffs](handoffs.md). |
| `model` | no | Which LLM to use. See [Models](models/index.md). |
| `model_settings` | no | Model tuning parameters such as `temperature`, `top_p`, and `tool_choice`. |
| `tools` | no | Tools the agent can call. See [Tools](tools.md). |
| `mcp_servers` | no | MCP-backed tools for the agent. See the [MCP guide](mcp.md). |
| `mcp_config` | no | Fine-tune how MCP tools are prepared, such as strict schema conversion and MCP failure formatting. See the [MCP guide](mcp.md#agent-level-mcp-configuration). |
| `mcp_servers` | no | MCP servers that provide MCP-backed tools to the agent. See the [MCP guide](mcp.md). |
| `mcp_config` | no | Fine-tune how MCP tools are prepared, such as converting their schemas to strict mode and formatting MCP failures. See the [MCP guide](mcp.md#agent-level-mcp-configuration). |
| `input_guardrails` | no | Guardrails that run on the first user input for this agent chain. See [Guardrails](guardrails.md). |
| `output_guardrails` | no | Guardrails that run on the final output for this agent. See [Guardrails](guardrails.md). |
| `output_type` | no | Structured output type instead of plain text. See [Output types](#output-types). |
@@ -45,9 +45,10 @@ The most common properties of an agent are:
| `reset_tool_choice` | no | Reset `tool_choice` after a tool call (default: `True`) to avoid tool-use loops. See [Forcing tool use](#forcing-tool-use). |
```python
from agents import Agent, ModelSettings, function_tool
from agents import Agent
from agents.decorators import tool
@function_tool
@tool
def get_weather(city: str) -> str:
"""returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@@ -64,7 +65,7 @@ Everything in this section applies to `Agent`. `SandboxAgent` builds on the same
## Prompt templates
You can reference a prompt template created in the OpenAI platform by setting `prompt`. This works with OpenAI models using the Responses API.
You can reference a prompt template created in the OpenAI platform by setting `prompt`. This works when OpenAI models are accessed through the Responses API.
To use it, please:
@@ -128,14 +129,21 @@ Agents are generic on their `context` type. Context is a dependency-injection to
Read the [context guide](context.md) for the full `RunContextWrapper` surface, shared usage tracking, nested `tool_input`, and serialization caveats.
```python
from dataclasses import dataclass
@dataclass
class Purchase:
id: str
@dataclass
class UserContext:
name: str
uid: str
is_pro_user: bool
async def fetch_purchases() -> list[Purchase]:
return ...
async def fetch_purchases(self) -> list[Purchase]:
# implement your logic here
return []
agent = Agent[UserContext](
...,
@@ -207,7 +215,7 @@ customer_facing_agent = Agent(
### Handoffs
Handoffs are subagents the agent can delegate to. When a handoff occurs, the delegated agent receives the conversation history and takes over the conversation. This pattern enables modular, specialized agents that excel at a single task. Read more in the [handoffs](handoffs.md) documentation.
Configured handoff targets are subagents to which the agent can delegate. When a handoff occurs, the delegated agent receives the conversation history and takes over the conversation. This pattern enables modular, specialized agents that excel at a single task. Read more in the [handoffs](handoffs.md) documentation.
```python
from agents import Agent
@@ -231,6 +239,8 @@ triage_agent = Agent(
In most cases, you can provide instructions when you create the agent. However, you can also provide dynamic instructions via a function. The function will receive the agent and context, and must return the prompt. Both regular and `async` functions are accepted.
```python
from agents import Agent, RunContextWrapper
def dynamic_instructions(
context: RunContextWrapper[UserContext], agent: Agent[UserContext]
) -> str:
@@ -259,13 +269,12 @@ The callback context also changes depending on the event:
Typical hook timing:
- `on_agent_start` / `on_agent_end`: when a specific agent begins or finishes producing a final output.
- `on_agent_start`: when a specific agent begins running; `on_agent_end`: when that agent finishes producing a final output.
- `on_llm_start` / `on_llm_end`: immediately around each model call.
- `on_tool_start` / `on_tool_end`: around each local tool invocation.
For function tools, the hook `context` is typically a `ToolContext`, so you can inspect tool-call metadata such as `tool_call_id`.
- `on_tool_start` / `on_tool_end`: around each local tool invocation. For function tools, the hook `context` is typically a `ToolContext`, so you can inspect tool-call metadata such as `tool_call_id`.
- `on_handoff`: when control moves from one agent to another.
Use `RunHooks` when you want a single observer for the whole workflow, and `AgentHooks` when one agent needs custom side effects.
Use `RunHooks` when you want a single observer for the whole workflow, and `AgentHooks` when you want lifecycle callbacks scoped to a specific agent.
```python
from agents import Agent, RunHooks, Runner
@@ -301,7 +310,7 @@ By using the `clone()` method on an agent, you can duplicate an Agent, and optio
pirate_agent = Agent(
name="Pirate",
instructions="Write like a pirate",
model="gpt-5.4",
model="gpt-5.6-sol",
)
robot_agent = pirate_agent.clone(
@@ -310,6 +319,8 @@ robot_agent = pirate_agent.clone(
)
```
`clone()` uses `dataclasses.replace`, so it performs a shallow copy. A list attribute that you do not override, such as `tools`, `handoffs`, `mcp_servers`, `input_guardrails`, or `output_guardrails`, remains the exact list held by the original agent. Mutating that list through either agent therefore affects both agents. To give the clone an independent list container, pass a new list, for example `pirate_agent.clone(tools=[*pirate_agent.tools, extra_tool])`. The entries copied into that new list remain the same tool or handoff objects unless you replace those entries too.
## Forcing tool use
Supplying a list of tools doesn't always mean the LLM will use a tool. You can force tool use by setting [`ModelSettings.tool_choice`][agents.model_settings.ModelSettings.tool_choice]. Valid values are:
@@ -322,9 +333,10 @@ Supplying a list of tools doesn't always mean the LLM will use a tool. You can f
When you are using OpenAI Responses tool search, named tool choices are more limited: you cannot target bare namespace names or deferred-only tools with `tool_choice`, and `tool_choice="tool_search"` does not target [`ToolSearchTool`][agents.tool.ToolSearchTool]. In those cases, prefer `auto` or `required`. See [Hosted tool search](tools.md#hosted-tool-search) for the Responses-specific constraints.
```python
from agents import Agent, Runner, function_tool, ModelSettings
from agents import Agent, ModelSettings
from agents.decorators import tool
@function_tool
@tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@@ -345,9 +357,10 @@ The `tool_use_behavior` parameter in the `Agent` configuration controls how tool
- `"stop_on_first_tool"`: The output of the first tool call is used as the final response, without further LLM processing.
```python
from agents import Agent, Runner, function_tool, ModelSettings
from agents import Agent
from agents.decorators import tool
@function_tool
@tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@@ -363,15 +376,16 @@ agent = Agent(
- `StopAtTools(stop_at_tool_names=[...])`: Stops if any specified tool is called, using its output as the final response.
```python
from agents import Agent, Runner, function_tool
from agents import Agent
from agents.agent import StopAtTools
from agents.decorators import tool
@function_tool
@tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
@function_tool
@tool
def sum_numbers(a: int, b: int) -> int:
"""Adds two numbers."""
return a + b
@@ -384,14 +398,15 @@ agent = Agent(
)
```
- `ToolsToFinalOutputFunction`: A custom function that processes tool results and decides whether to stop or continue with the LLM.
- `ToolsToFinalOutputFunction`: A custom function that processes tool results and decides whether to end the run with a final output or continue processing with the LLM.
```python
from agents import Agent, Runner, function_tool, FunctionToolResult, RunContextWrapper
from agents import Agent, FunctionToolResult, RunContextWrapper
from agents.agent import ToolsToFinalOutputResult
from agents.decorators import tool
from typing import List, Any
@function_tool
@tool
def get_weather(city: str) -> str:
"""Returns weather info for the specified city."""
return f"The weather in {city} is sunny"
+81 -2
View File
@@ -12,6 +12,25 @@ If you need to configure a specific agent or run instead, start with:
- [Models](models/index.md) for model selection and provider configuration.
- [Tracing](tracing.md) for per-run tracing metadata and custom trace processors.
## Configuration objects and dictionaries
Configuration parameters defined by the SDK generally accept either their typed settings object or a dictionary containing the same fields. This applies across agent, run, model, session, sandbox, and voice configuration boundaries whose type annotations include a dictionary. Nested settings types defined by the SDK can also use dictionaries.
```python
from agents import Agent
agent = Agent(
name="Assistant",
model="gpt-5.6-sol",
model_settings={
"reasoning": {"effort": "high"},
"verbosity": "low",
},
)
```
The SDK normalizes these dictionaries into the corresponding settings objects. Unknown fields in dataclass configuration types defined by the SDK raise `TypeError`, which helps catch misspelled option names early. Check the parameter's type annotation or API reference to confirm whether a specific boundary accepts a dictionary.
## API keys and clients
By default, the SDK uses the `OPENAI_API_KEY` environment variable for LLM requests and tracing. The key is resolved when the SDK first creates an OpenAI client (lazy initialization), so set the environment variable before your first model call. If you are unable to set that environment variable before your app starts, you can use the [set_default_openai_key()][agents.set_default_openai_key] function to set the key.
@@ -32,6 +51,32 @@ custom_client = AsyncOpenAI(base_url="...", api_key="...")
set_default_openai_client(custom_client)
```
When you pass an explicit client to [`OpenAIProvider`][agents.models.openai_provider.OpenAIProvider], that client owns its connection and account settings. Do not also pass `api_key`, `base_url`, `websocket_base_url`, `organization`, or `project` to `OpenAIProvider`; combining `openai_client` with any of those arguments raises [`UserError`][agents.exceptions.UserError] instead of silently ignoring the duplicate value. Set the intended values when constructing `AsyncOpenAI`.
### Custom HTTP clients with `openai` v3
Version 0.21.0 requires `openai>=3.0.0,<4`. The default OpenAI provider uses HTTPX2, so most applications do not need to configure an HTTP client directly. If your application passes `http_client=` to `AsyncOpenAI`, use HTTPX2 types for the custom client and its transport-facing options:
```python
import httpx2
from openai import AsyncOpenAI, DefaultAsyncHttpx2Client
from agents import set_default_openai_client
http_client = DefaultAsyncHttpx2Client(
timeout=httpx2.Timeout(30.0, connect=5.0),
)
custom_client = AsyncOpenAI(
api_key="...",
http_client=http_client,
)
set_default_openai_client(custom_client)
```
The same migration applies to custom transports, authentication, event hooks, mock transports, URLs, requests, responses, and transport exception handling. Use their `httpx2` equivalents. The Agents SDK does not convert arbitrary legacy `httpx` objects to HTTPX2. The OpenAI Python SDK provides a temporary compatibility path for legacy clients when the application installs `httpx` explicitly, but new and migrated code should use HTTPX2.
This OpenAI client boundary is separate from local MCP transport customization. MCP Python SDK v1 uses its own legacy `httpx` dependency, while MCP Python SDK v2 uses `httpx2`; see [MCP Python SDK v1 and v2](mcp.md#mcp-python-sdk-v1-and-v2).
If you prefer environment-based endpoint configuration, the default OpenAI provider also reads `OPENAI_BASE_URL`. When you enable Responses websocket transport, it also reads `OPENAI_WEBSOCKET_BASE_URL` for the websocket `/responses` endpoint.
```bash
@@ -47,6 +92,38 @@ from agents import set_default_openai_api
set_default_openai_api("chat_completions")
```
## OpenAI provider defaults
Providers that use the SDK's OpenAI backend also read SDK-wide defaults when they map model-name strings to models. Use [`set_default_openai_responses_transport()`][agents.set_default_openai_responses_transport] to make OpenAI Responses models use websocket transport by default:
```python
from agents import set_default_openai_responses_transport
set_default_openai_responses_transport("websocket")
```
This affects OpenAI Responses models that result when the default OpenAI provider resolves a model name. For provider-level setup, connection reuse, keepalive options, and custom websocket endpoints, see [Responses WebSocket transport](models/index.md#responses-websocket-transport).
If your OpenAI setup expects provider-level agent registration metadata, configure a default harness ID once at startup:
```python
from agents import set_default_openai_harness
set_default_openai_harness("your-harness-id")
```
You can also pass the full registration object:
```python
from agents import OpenAIAgentRegistrationConfig, set_default_openai_agent_registration
set_default_openai_agent_registration(
OpenAIAgentRegistrationConfig(harness_id="your-harness-id")
)
```
If no SDK default is set, providers that use the SDK's OpenAI backend fall back to the `OPENAI_AGENT_HARNESS_ID` environment variable. When a harness ID is configured, the SDK adds it to trace metadata as `agent_harness_id` unless that key is already present in `RunConfig.trace_metadata`.
## Tracing
Tracing is enabled by default. By default it uses the same OpenAI API key as your model requests from the section above (that is, the environment variable or the default key you set). You can specifically set the API key used for tracing by using the [`set_tracing_export_api_key`][agents.set_tracing_export_api_key] function.
@@ -150,9 +227,9 @@ logger.setLevel(logging.WARNING)
logger.addHandler(logging.StreamHandler())
```
### Sensitive data in logs
### Sensitive data in logs and diagnostics
Certain logs may contain sensitive data (for example, user data).
Certain logs and diagnostic exceptions may contain sensitive data (for example, model or tool inputs and outputs).
By default, the SDK does **not** log LLM inputs/outputs or tool inputs/outputs. These protections are controlled by:
@@ -167,3 +244,5 @@ If you need to include this data temporarily for debugging, set either variable
export OPENAI_AGENTS_DONT_LOG_MODEL_DATA=0
export OPENAI_AGENTS_DONT_LOG_TOOL_DATA=0
```
These flags also control whether affected failures retain payload-bearing diagnostic details. For example, with tool-data redaction enabled, invalid arguments for a `FunctionTool` raise a generic `ModelBehaviorError` without chaining the underlying validation error. Setting either variable to `0` can expose raw model or tool data in logs, exception messages, exception chains, and other diagnostic context, so enable it only in a controlled development environment.
+9 -7
View File
@@ -11,9 +11,9 @@ This is represented via the [`RunContextWrapper`][agents.run_context.RunContextW
1. You create any Python object you want. A common pattern is to use a dataclass or a Pydantic object.
2. You pass that object to the various run methods (e.g. `Runner.run(..., context=whatever)`).
3. All your tool calls, lifecycle hooks etc will be passed a wrapper object, `RunContextWrapper[T]`, where `T` represents your context object type which you can access via `wrapper.context`.
3. All your tool calls, lifecycle hooks etc will be passed a wrapper object, `RunContextWrapper[T]`, where `T` represents the type of your context object; the object itself is available via `wrapper.context`.
For some runtime-specific callbacks, the SDK may pass a more specialized subclass of `RunContextWrapper[T]`. For example, function-tool lifecycle hooks typically receive `ToolContext`, which also exposes tool-call metadata like `tool_call_id`, `tool_name`, and `tool_arguments`.
For some runtime-specific callbacks, the SDK may pass a more specialized subclass of `RunContextWrapper[T]`. For example, lifecycle hooks for `FunctionTool` instances typically receive `ToolContext`, which also exposes tool-call metadata like `tool_call_id`, `tool_name`, and `tool_arguments`.
The **most important** thing to be aware of: every agent, tool function, lifecycle etc for a given agent run must use the same _type_ of context.
@@ -48,14 +48,15 @@ Conversation state is a separate concern. Use `result.to_input_list()`, `session
import asyncio
from dataclasses import dataclass
from agents import Agent, RunContextWrapper, Runner, function_tool
from agents import Agent, RunContextWrapper, Runner
from agents.decorators import tool
@dataclass
class UserInfo: # (1)!
name: str
uid: int
@function_tool
@tool
async def fetch_user_age(wrapper: RunContextWrapper[UserInfo]) -> str: # (2)!
"""Fetch the age of the user. Call this function to get user's age information."""
return f"The user {wrapper.context.name} is 47 years old"
@@ -97,7 +98,8 @@ For this, you can use the [`ToolContext`][agents.tool_context.ToolContext] class
```python
from typing import Annotated
from pydantic import BaseModel, Field
from agents import Agent, Runner, function_tool
from agents import Agent
from agents.decorators import tool
from agents.tool_context import ToolContext
class WeatherContext(BaseModel):
@@ -108,7 +110,7 @@ class Weather(BaseModel):
temperature_range: str = Field(description="The temperature range in Celsius")
conditions: str = Field(description="The weather conditions")
@function_tool
@tool
def get_weather(ctx: ToolContext[WeatherContext], city: Annotated[str, "The city to get the weather for"]) -> Weather:
print(f"[debug] Tool context: (name: {ctx.tool_name}, call_id: {ctx.tool_call_id}, args: {ctx.tool_arguments})")
return Weather(city=city, temperature_range="14-20C", conditions="Sunny with wind.")
@@ -140,5 +142,5 @@ When an LLM is called, the **only** data it can see is from the conversation his
1. You can add it to the Agent `instructions`. This is also known as a "system prompt" or "developer message". System prompts can be static strings, or they can be dynamic functions that receive the context and output a string. This is a common tactic for information that is always useful (for example, the user's name or the current date).
2. Add it to the `input` when calling the `Runner.run` functions. This is similar to the `instructions` tactic, but allows you to have messages that are lower in the [chain of command](https://cdn.openai.com/spec/model-spec-2024-05-08.html#follow-the-chain-of-command).
3. Expose it via function tools. This is useful for _on-demand_ context - the LLM decides when it needs some data, and can call the tool to fetch that data.
3. Expose it through `FunctionTool` instances. This is useful for _on-demand_ context - the LLM decides when it needs some data, and can call the tool to fetch that data.
4. Use retrieval or web search. These are special tools that are able to fetch relevant data from files or databases (retrieval), or from the web (web search). This is useful for "grounding" the response in relevant contextual data.
+32 -37
View File
@@ -1,11 +1,10 @@
# Examples
Check out a variety of sample implementations of the SDK in the examples section of the [repo](https://github.com/openai/openai-agents-python/tree/main/examples). The examples are organized into several categories that demonstrate different patterns and capabilities.
Check out a variety of sample implementations that use the SDK in the examples section of the [repo](https://github.com/openai/openai-agents-python/tree/main/examples). The examples are organized into several categories that demonstrate different patterns and capabilities.
## Categories
- **[agent_patterns](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns):**
Examples in this category illustrate common agent design patterns, such as
- **[agent_patterns](https://github.com/openai/openai-agents-python/tree/main/examples/agent_patterns):** Examples in this category illustrate common agent design patterns, such as
- Deterministic workflows
- Agents as tools
@@ -13,7 +12,7 @@ Check out a variety of sample implementations of the SDK in the examples section
- Agents as tools with structured input parameters (`examples/agent_patterns/agents_as_tools_structured.py`)
- Parallel agent execution
- Conditional tool usage
- Forcing tool use with different behaviors (`examples/agent_patterns/forcing_tool_use.py`)
- Forcing tool use while demonstrating different tool-use behaviors (`examples/agent_patterns/forcing_tool_use.py`)
- Input/output guardrails
- LLM as a judge
- Routing
@@ -22,16 +21,15 @@ Check out a variety of sample implementations of the SDK in the examples section
- Human-in-the-loop with streaming (`examples/agent_patterns/human_in_the_loop_stream.py`)
- Custom rejection messages for approval flows (`examples/agent_patterns/human_in_the_loop_custom_rejection.py`)
- **[basic](https://github.com/openai/openai-agents-python/tree/main/examples/basic):**
These examples showcase foundational capabilities of the SDK, such as
- **[basic](https://github.com/openai/openai-agents-python/tree/main/examples/basic):** These examples showcase foundational capabilities of the SDK, such as
- Hello world examples (Default model, GPT-5, open-weight model)
- Agent lifecycle management
- Run hooks and agent hooks lifecycle example (`examples/basic/lifecycle_example.py`)
- Agent and run lifecycle example using `RunHooks` and `AgentHooks` (`examples/basic/lifecycle_example.py`)
- Dynamic system prompts
- Basic tool usage (`examples/basic/tools.py`)
- Tool input/output guardrails (`examples/basic/tool_guardrails.py`)
- Image tool output (`examples/basic/image_tool_output.py`)
- Returning an image as tool output (`examples/basic/image_tool_output.py`)
- Streaming outputs (text, items, function call args)
- Responses websocket transport with a shared session helper across turns (`examples/basic/stream_ws.py`)
- Prompt templates
@@ -42,28 +40,23 @@ Check out a variety of sample implementations of the SDK in the examples section
- Non-strict output types
- Previous response ID usage
- **[customer_service](https://github.com/openai/openai-agents-python/tree/main/examples/customer_service):**
Example customer service system for an airline.
- **[customer_service](https://github.com/openai/openai-agents-python/tree/main/examples/customer_service):** Example customer service system for an airline.
- **[financial_research_agent](https://github.com/openai/openai-agents-python/tree/main/examples/financial_research_agent):**
A financial research agent that demonstrates structured research workflows with agents and tools for financial data analysis.
- **[financial_research_agent](https://github.com/openai/openai-agents-python/tree/main/examples/financial_research_agent):** A financial research agent that demonstrates structured research workflows for financial data analysis using agents and tools.
- **[handoffs](https://github.com/openai/openai-agents-python/tree/main/examples/handoffs):**
Practical examples of agent handoffs with message filtering, including:
- **[handoffs](https://github.com/openai/openai-agents-python/tree/main/examples/handoffs):** Practical examples of agent handoffs with message filtering, including:
- Message filter example (`examples/handoffs/message_filter.py`)
- Message filter with streaming (`examples/handoffs/message_filter_streaming.py`)
- **[hosted_mcp](https://github.com/openai/openai-agents-python/tree/main/examples/hosted_mcp):**
Examples demonstrating how to use hosted MCP (Model Context Protocol) with the OpenAI Responses API, including:
- **[hosted_mcp](https://github.com/openai/openai-agents-python/tree/main/examples/hosted_mcp):** Examples demonstrating how to use hosted MCP (Model Context Protocol) with the OpenAI Responses API, including:
- Simple hosted MCP without approval (`examples/hosted_mcp/simple.py`)
- MCP connectors such as Google Calendar (`examples/hosted_mcp/connectors.py`)
- Human-in-the-loop with interruption-based approvals (`examples/hosted_mcp/human_in_the_loop.py`)
- On-approval callback for MCP tool calls (`examples/hosted_mcp/on_approval.py`)
- Callback for MCP tool approval requests (`examples/hosted_mcp/on_approval.py`)
- **[mcp](https://github.com/openai/openai-agents-python/tree/main/examples/mcp):**
Learn how to build agents with MCP (Model Context Protocol), including:
- **[mcp](https://github.com/openai/openai-agents-python/tree/main/examples/mcp):** Learn how to build agents with MCP (Model Context Protocol), including:
- Filesystem examples
- Git examples
@@ -74,11 +67,10 @@ Check out a variety of sample implementations of the SDK in the examples section
- Streamable HTTP remote connection (`examples/mcp/streamable_http_remote_example`)
- Custom HTTP client factory for Streamable HTTP (`examples/mcp/streamablehttp_custom_client_example`)
- Prefetching all MCP tools with `MCPUtil.get_all_function_tools` (`examples/mcp/get_all_mcp_tools_example`)
- MCPServerManager with FastAPI (`examples/mcp/manager_example`)
- Using `MCPServerManager` in a FastAPI application (`examples/mcp/manager_example`)
- MCP tool filtering (`examples/mcp/tool_filter_example`)
- **[memory](https://github.com/openai/openai-agents-python/tree/main/examples/memory):**
Examples of different memory implementations for agents, including:
- **[memory](https://github.com/openai/openai-agents-python/tree/main/examples/memory):** Examples of different memory implementations for agents, including:
- SQLite session storage
- Advanced SQLite session storage
@@ -95,29 +87,32 @@ Check out a variety of sample implementations of the SDK in the examples section
- OpenAI Conversations session with human-in-the-loop (`examples/memory/openai_session_hitl_example.py`)
- HITL approval/rejection scenario across sessions (`examples/memory/hitl_session_scenario.py`)
- **[model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers):**
Explore how to use non-OpenAI models with the SDK, including custom providers and third-party adapters.
- **[model_providers](https://github.com/openai/openai-agents-python/tree/main/examples/model_providers):** Explore how to use non-OpenAI models with the SDK, including custom providers and third-party adapters.
- **[realtime](https://github.com/openai/openai-agents-python/tree/main/examples/realtime):**
Examples showing how to build real-time experiences using the SDK, including:
- **[realtime](https://github.com/openai/openai-agents-python/tree/main/examples/realtime):** Examples showing how to build real-time experiences using the SDK, including:
- Web application patterns with structured text and image messages
- Command-line audio loops and playback handling
- Twilio Media Streams integration over WebSocket
- Twilio SIP integration using Realtime Calls API attach flows
- Twilio SIP integration using the Realtime Calls API's `attach` flows
- **[reasoning_content](https://github.com/openai/openai-agents-python/tree/main/examples/reasoning_content):**
Examples demonstrating how to work with reasoning content, including:
- **[reasoning_content](https://github.com/openai/openai-agents-python/tree/main/examples/reasoning_content):** Examples demonstrating how to work with reasoning content, including:
- Reasoning content with the Runner API, streaming and non-streaming (`examples/reasoning_content/runner_example.py`)
- Reasoning content with OSS models via OpenRouter (`examples/reasoning_content/gpt_oss_stream.py`)
- Basic reasoning content example (`examples/reasoning_content/main.py`)
- **[research_bot](https://github.com/openai/openai-agents-python/tree/main/examples/research_bot):**
Simple deep research clone that demonstrates complex multi-agent research workflows.
- **[research_bot](https://github.com/openai/openai-agents-python/tree/main/examples/research_bot):** Simple deep research clone that demonstrates complex multi-agent research workflows.
- **[tools](https://github.com/openai/openai-agents-python/tree/main/examples/tools):**
Learn how to implement OAI hosted tools and experimental Codex tooling such as:
- **[sandbox](https://github.com/openai/openai-agents-python/tree/main/examples/sandbox):** Examples for running agents in isolated workspaces, including:
- Basic sandbox agent setup (`examples/sandbox/basic.py`)
- Unix-local and Docker sandbox lifecycle examples
- Sandbox-backed handoffs (`examples/sandbox/handoffs.py`)
- Sandbox memory and snapshot resume (`examples/sandbox/memory.py`)
- Sandbox agents exposed as tools (`examples/sandbox/sandbox_agents_as_tools.py`)
- **[tools](https://github.com/openai/openai-agents-python/tree/main/examples/tools):** Learn how to implement OpenAI-hosted tools and experimental Codex tooling. Examples include:
- Web search and web search with filters
- File search
@@ -128,11 +123,11 @@ Check out a variety of sample implementations of the SDK in the examples section
- Hosted container shell with inline skills (`examples/tools/container_shell_inline_skill.py`)
- Hosted container shell with skill references (`examples/tools/container_shell_skill_reference.py`)
- Local shell with local skills (`examples/tools/local_shell_skill.py`)
- Tool search with namespaces and deferred tools (`examples/tools/tool_search.py`)
- Tool search with namespaces and tools that use deferred loading (`examples/tools/tool_search.py`)
- Programmatic Tool Calling with concurrent structured tool calls (`examples/tools/programmatic_tool_calling.py`)
- Computer use
- Image generation
- Experimental Codex tool workflows (`examples/tools/codex.py`)
- Experimental Codex same-thread workflows (`examples/tools/codex_same_thread.py`)
- Experimental Codex workflows that reuse the same Codex conversation thread (`examples/tools/codex_same_thread.py`)
- **[voice](https://github.com/openai/openai-agents-python/tree/main/examples/voice):**
See examples of voice agents, using our TTS and STT models, including streamed voice examples.
- **[voice](https://github.com/openai/openai-agents-python/tree/main/examples/voice):** See examples of voice agents, using our TTS and STT models, including streamed voice examples.
+18 -11
View File
@@ -1,6 +1,6 @@
# Guardrails
Guardrails enable you to do checks and validations of user input and agent output. For example, imagine you have an agent that uses a very smart (and hence slow/expensive) model to help with customer requests. You wouldn't want malicious users to ask the model to help them with their math homework. So, you can run a guardrail with a fast/cheap model. If the guardrail detects malicious usage, it can immediately raise an error and prevent the expensive model from running, saving you time and money (**when using blocking guardrails; for parallel guardrails, the expensive model may have already started running before the guardrail completes. See "Execution modes" below for details**).
Guardrails enable you to do checks and validations of user input and agent output. For example, imagine you have an agent that uses a very smart (and hence slow/expensive) model to help with customer requests. You wouldn't want malicious users to ask the model to help them with their math homework. So, you can run a guardrail with a fast/cheap model. If the guardrail detects malicious usage, it can immediately raise an error, saving time and money. Blocking execution guarantees that the expensive model does not start; with parallel execution, the expensive model may already have started before the guardrail completes. See "Execution modes" below for details.
There are two kinds of guardrails:
@@ -15,7 +15,7 @@ Guardrails are attached to agents and tools, but they do not all run at the same
- **Output guardrails** run only for the agent that produces the final output.
- **Tool guardrails** run on every custom function-tool invocation, with input guardrails before execution and output guardrails after execution.
If you need checks around each custom function-tool call in a workflow that includes managers, handoffs, or delegated specialists, use tool guardrails instead of relying only on agent-level input/output guardrails.
If you need checks before and/or after each custom function-tool call in a workflow that includes managers, handoffs, or delegated specialists, use tool guardrails instead of relying only on agent-level input/output guardrails.
## Input guardrails
@@ -33,7 +33,7 @@ Input guardrails run in 3 steps:
Input guardrails support two execution modes:
- **Parallel execution** (default, `run_in_parallel=True`): The guardrail runs concurrently with the agent's execution. This provides the best latency since both start at the same time. However, if the guardrail fails, the agent may have already consumed tokens and executed tools before being cancelled.
- **Parallel execution** (default, `run_in_parallel=True`): The guardrail runs concurrently with the agent's execution. This provides the best latency since both start at the same time. However, if the guardrail's tripwire is triggered, the agent may have already consumed tokens and executed tools before being cancelled.
- **Blocking execution** (`run_in_parallel=False`): The guardrail runs and completes *before* the agent starts. If the guardrail tripwire is triggered, the agent never executes, preventing token consumption and tool execution. This is ideal for cost optimization and when you want to avoid potential side effects from tool calls.
@@ -51,19 +51,28 @@ Output guardrails run in 3 steps:
Output guardrails always run after the agent completes, so they don't support the `run_in_parallel` parameter.
An output tripwire and an exception raised by the guardrail function have different session behavior. A tripwire rejects the candidate final output. When a tripwire fires, the runner asks the configured session to persist already-completed tool call and tool output items, together with any reasoning context required to replay those calls, while excluding the rejected candidate final output. The runner applies this tripwire rule to both streaming and non-streaming runs. When the guardrail function raises an exception instead of returning a tripwire result, the runner treats the verdict as unknown and asks the configured session to persist the completed final-turn items before surfacing the guardrail exception. If that session write also fails, the session write error takes precedence. Streaming runs use the same persistence ordering as non-streaming runs and raise the terminal exception from `stream_events()`. An immediate [`RunResultStreaming.cancel()`][agents.result.RunResultStreaming.cancel] call while the output guardrail is running cancels the in-flight guardrail and does not start a final-turn session write.
Terminal function-tool output needs additional handling because the tool has already run before the agent-level output guardrail checks the value. When [`Agent.tool_use_behavior`][agents.agent.Agent.tool_use_behavior] makes that tool result the final output and an output tripwire rejects it, the SDK retains a replay-valid function call/output pair only when it can rebuild the pair from validated fields. The retained `function_call_output` payload is replaced with the fixed text `"Output withheld by an output guardrail."`; the original tool-output payload is not retained in the session, `RunState`, streamed result state, or sandbox memory input. The SDK does retain validated function-call metadata required for replay, including the function arguments, so that metadata can contain data that also appeared in the rejected output. Current-response [`OutputGuardrailResult`][agents.guardrail.OutputGuardrailResult] objects also replace `agent_output` with the fixed text and clear `output_info`. Current-response [`ToolOutputGuardrailResult`][agents.tool_guardrails.ToolOutputGuardrailResult] objects preserve the allow/reject behavior type but replace payload-bearing `output_info` and rejection messages with the same text. Earlier accepted turns and guardrail results remain unchanged. If the response contains reasoning or another shape that the SDK cannot sanitize safely, the SDK discards the complete current-response suffix instead of retaining the rejected output payload. A guardrail function that raises an exception has not returned a rejection verdict, so the completed terminal-tool turn follows the exception persistence behavior described above.
## Tool guardrails
Tool guardrails wrap **function tools** and let you validate or block tool calls before and after execution. They are configured on the tool itself and run every time that tool is invoked.
Tool guardrails wrap **`FunctionTool` instances** and let you validate or block calls to those tools before and after execution. They are configured on the tool itself and run every time that tool is invoked.
- Input tool guardrails run before the tool executes and can skip the call, replace the output with a message, or raise a tripwire.
- Output tool guardrails run after the tool executes and can replace the output or raise a tripwire.
- If a function tool requires approval, input tool guardrails normally run after approval and immediately before execution. Set [`RunConfig.tool_execution`][agents.run.RunConfig.tool_execution] to [`ToolExecutionConfig(pre_approval_tool_input_guardrails=True)`][agents.run.ToolExecutionConfig] when you want those input checks to run before the pending approval interruption is emitted. Calls that pass this pre-approval check are still checked again after approval before the tool executes.
- Tool guardrails apply only to function tools created with [`function_tool`][agents.tool.function_tool]. Handoffs run through the SDK's handoff pipeline rather than the normal function-tool pipeline, so tool guardrails do not apply to the handoff call itself. Hosted tools (`WebSearchTool`, `FileSearchTool`, `HostedMCPTool`, `CodeInterpreterTool`, `ImageGenerationTool`) and built-in execution tools (`ComputerTool`, `ShellTool`, `ApplyPatchTool`, `LocalShellTool`) also do not use this guardrail pipeline, and [`Agent.as_tool()`][agents.agent.Agent.as_tool] does not currently expose tool-guardrail options directly.
See the code snippet below for details.
## Tripwires
If the input or output fails the guardrail, the Guardrail can signal this with a tripwire. As soon as we see a guardrail that has triggered the tripwires, we immediately raise a `{Input,Output}GuardrailTripwireTriggered` exception and halt the Agent execution.
If an agent input or output fails a guardrail, the guardrail can signal this with a tripwire. The runner immediately raises an `InputGuardrailTripwireTriggered` or `OutputGuardrailTripwireTriggered` exception and halts agent execution. Tool guardrails use the corresponding `ToolInputGuardrailTripwireTriggered` and `ToolOutputGuardrailTripwireTriggered` exceptions.
For agent-level tripwires, the exception's `guardrail_result` identifies the guardrail that triggered the tripwire. For an input tripwire raised by the runner, `exception.run_data.input_guardrail_results` contains every input guardrail result completed before the run stopped, including the result that triggered the tripwire. Output tripwires provide the equivalent accumulated results through `exception.run_data.output_guardrail_results`.
Tool tripwire exceptions instead expose the triggering `guardrail` and `output` directly. Their `run_data.tool_input_guardrail_results` and `run_data.tool_output_guardrail_results` lists preserve results accumulated from completed turns before the failure; the triggering result is available through the exception's `output`. Other runner-managed failures, such as `MaxTurnsExceeded`, also preserve completed tool guardrail results in these lists. After `stream_events()` raises an exception, the streamed result exposes the same accumulated agent and tool guardrail result lists. `run_data` can be `None` when an exception is raised outside a runner-managed execution path.
## Implementing a guardrail
@@ -78,8 +87,8 @@ from agents import (
RunContextWrapper,
Runner,
TResponseInputItem,
input_guardrail,
)
from agents.decorators import input_guardrail
class MathHomeworkOutput(BaseModel):
is_math_homework: bool
@@ -135,8 +144,8 @@ from agents import (
OutputGuardrailTripwireTriggered,
RunContextWrapper,
Runner,
output_guardrail,
)
from agents.decorators import output_guardrail
class MessageOutput(BaseModel): # (1)!
response: str
@@ -191,10 +200,8 @@ from agents import (
Agent,
Runner,
ToolGuardrailFunctionOutput,
function_tool,
tool_input_guardrail,
tool_output_guardrail,
)
from agents.decorators import tool, tool_input_guardrail, tool_output_guardrail
@tool_input_guardrail
def block_secrets(data):
@@ -214,7 +221,7 @@ def redact_output(data):
return ToolGuardrailFunctionOutput.allow()
@function_tool(
@tool(
tool_input_guardrails=[block_secrets],
tool_output_guardrails=[redact_output],
)
+5 -5
View File
@@ -2,7 +2,7 @@
Handoffs allow an agent to delegate tasks to another agent. This is particularly useful in scenarios where different agents specialize in distinct areas. For example, a customer support app might have agents that each specifically handle tasks like order status, refunds, FAQs, etc.
Handoffs are represented as tools to the LLM. So if there's a handoff to an agent named `Refund Agent`, the tool would be called `transfer_to_refund_agent`.
Handoffs are represented as tools to the LLM. So if there's a handoff to an agent named `Refund Agent`, the tool would be named `transfer_to_refund_agent`.
## Creating a handoff
@@ -39,7 +39,7 @@ The [`handoff()`][agents.handoffs.handoff] function lets you customize things.
- `input_type`: The schema for the handoff tool-call arguments. When set, the parsed payload is passed to `on_handoff`.
- `input_filter`: This lets you filter the input received by the next agent. See below for more.
- `is_enabled`: Whether the handoff is enabled. This can be a boolean or a function that returns a boolean, allowing you to dynamically enable or disable the handoff at runtime.
- `nest_handoff_history`: Optional per-call override for the RunConfig-level `nest_handoff_history` setting. If `None`, the value defined in the active run configuration is used instead.
- `nest_handoff_history`: Optional per-handoff override for the RunConfig-level `nest_handoff_history` setting. If `None`, the value defined in the active run configuration is used instead.
The [`handoff()`][agents.handoffs.handoff] helper always transfers control to the specific `agent` you passed in. If you have multiple possible destinations, register one handoff per destination and let the model choose among them. Use a custom [`Handoff`][agents.handoffs.Handoff] only when your own handoff code must decide which agent to return at invocation time.
@@ -61,7 +61,7 @@ handoff_obj = handoff(
## Handoff inputs
In certain situations, you want the LLM to provide some data when it calls a handoff. For example, imagine a handoff to an "Escalation agent". You might want a reason to be provided, so you can log it.
In certain situations, you want the LLM to provide some data when it calls a handoff. For example, imagine a handoff to an "Escalation agent". You might want the model to provide a reason so you can log it.
```python
from pydantic import BaseModel
@@ -112,7 +112,7 @@ When a handoff occurs, it's as though the new agent takes over the conversation,
- `input_items`: optional items to forward to the next agent instead of `new_items`, allowing you to filter model input while keeping `new_items` intact for session history.
- `run_context`: the active [`RunContextWrapper`][agents.run_context.RunContextWrapper] at the time the handoff was invoked.
Nested handoffs are available as an opt-in beta and are disabled by default while we stabilize them. When you enable [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history], the runner collapses the prior transcript into a single assistant summary message and wraps it in a `<CONVERSATION HISTORY>` block that keeps appending new turns when multiple handoffs happen during the same run. You can provide your own mapping function via [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] to replace the generated message without writing a full `input_filter`. The opt-in only applies when neither the handoff nor the run supplies an explicit `input_filter`, so existing code that already customizes the payload (including the examples in this repository) keeps its current behavior without changes. You can override the nesting behaviour for a single handoff by passing `nest_handoff_history=True` or `False` to [`handoff(...)`][agents.handoffs.handoff], which sets [`Handoff.nest_handoff_history`][agents.handoffs.Handoff.nest_handoff_history]. If you just need to change the wrapper text for the generated summary, call [`set_conversation_history_wrappers`][agents.handoffs.set_conversation_history_wrappers] (and optionally [`reset_conversation_history_wrappers`][agents.handoffs.reset_conversation_history_wrappers]) before running your agents.
Nested handoff history is available as an opt-in beta and is disabled by default while we stabilize it. When you enable [`RunConfig.nest_handoff_history`][agents.run.RunConfig.nest_handoff_history], the runner compacts summarizable history into ordered assistant summary segments while preserving lossless message items in their original positions. Each generated summary segment uses the `<CONVERSATION HISTORY>` wrapper, and later handoffs flatten earlier generated segments before rebuilding the ordered transcript. Sessions, `RunState`, and `RunResult.to_input_list()` track exact message occurrences moved into this SDK-default history so those occurrences are not appended twice; separate identical messages are still preserved. You can provide your own mapping function via [`RunConfig.handoff_history_mapper`][agents.run.RunConfig.handoff_history_mapper] to return the exact list of input items for the next agent instead of using the built-in segmentation. The opt-in applies only when neither the handoff's `input_filter` nor the active run's `RunConfig.handoff_input_filter` is set, so existing code that already customizes the payload (including the examples in this repository) keeps its current behavior without changes. You can override the nesting behaviour for a single handoff by passing `nest_handoff_history=True` or `False` to [`handoff(...)`][agents.handoffs.handoff], which sets [`Handoff.nest_handoff_history`][agents.handoffs.Handoff.nest_handoff_history]. If you just need to change the wrapper text for generated summary segments, call [`set_conversation_history_wrappers`][agents.handoffs.set_conversation_history_wrappers] before running your agents. Call [`reset_conversation_history_wrappers`][agents.handoffs.reset_conversation_history_wrappers] before a later run when you need to restore the default wrappers.
If both the handoff and the active [`RunConfig.handoff_input_filter`][agents.run.RunConfig.handoff_input_filter] define a filter, the per-handoff [`input_filter`][agents.handoffs.Handoff.input_filter] takes precedence for that specific handoff.
@@ -134,7 +134,7 @@ handoff_obj = handoff(
)
```
1. This will automatically remove all tools from the history when `FAQ agent` is called.
1. This will automatically remove all tool-related items from the history when `FAQ agent` is called.
## Recommended prompts
+22 -24
View File
@@ -1,6 +1,6 @@
# Human-in-the-loop
Use the human-in-the-loop (HITL) flow to pause agent execution until a person approves or rejects sensitive tool calls. Tools declare when they need approval, run results surface pending approvals as interruptions, and `RunState` lets you serialize and resume runs after decisions are made.
Use the human-in-the-loop (HITL) flow to pause agent execution until a person approves or rejects sensitive tool calls. Tools declare when they need approval, run results surface pending approvals as interruptions, and `RunState` lets you serialize paused runs and resume them after decisions are made.
That approval surface is run-wide, not limited to the current top-level agent. The same pattern applies when the tool belongs to the current agent, to an agent reached through a handoff, or to a nested [`Agent.as_tool()`][agents.agent.Agent.as_tool] execution. In the nested `Agent.as_tool()` case, the interruption still surfaces on the outer run, so you approve or reject it on the outer `RunState` and resume the original top-level run.
@@ -12,11 +12,14 @@ This page focuses on the manual approval flow via `interruptions`. If your app c
Set `needs_approval` to `True` to always require approval or provide an async function that decides per call. The callable receives the run context, parsed tool parameters, and the tool call ID.
Callable approval rules fail closed when the SDK cannot safely inspect the arguments. If the arguments are malformed JSON, are valid JSON but not an object (for example, `null` or a list), or contain non-standard constants such as `NaN`, `Infinity`, or `-Infinity`, the callable is not invoked and the call requires manual approval. This behavior is the same for Runner and Realtime tool calls.
```python
from agents import Agent, Runner, function_tool
from agents import Agent
from agents.decorators import tool
@function_tool(needs_approval=True)
@tool(needs_approval=True)
async def cancel_order(order_id: int) -> str:
return f"Cancelled order {order_id}"
@@ -25,7 +28,7 @@ async def requires_review(_ctx, params, _call_id) -> bool:
return "refund" in params.get("subject", "").lower()
@function_tool(needs_approval=requires_review)
@tool(needs_approval=requires_review)
async def send_email(subject: str, body: str) -> str:
return f"Sent '{subject}'"
@@ -42,13 +45,15 @@ agent = Agent(
## How the approval flow works
1. When the model emits a tool call, the runner evaluates its approval rule (`needs_approval`, `require_approval`, or the hosted MCP equivalent).
2. If an approval decision for that tool call is already stored in the [`RunContextWrapper`][agents.run_context.RunContextWrapper], the runner proceeds without prompting. Per-call approvals are scoped to the specific call ID; pass `always_approve=True` or `always_reject=True` to persist the same decision for future calls to that tool during the rest of the run.
3. Otherwise, execution pauses and `RunResult.interruptions` (or `RunResultStreaming.interruptions`) contains [`ToolApprovalItem`][agents.items.ToolApprovalItem] entries with details such as `agent.name`, `tool_name`, and `arguments`. This includes approvals raised after a handoff or inside nested `Agent.as_tool()` executions.
2. If an approval decision for that tool call is already stored in the [`RunContextWrapper`][agents.run_context.RunContextWrapper], the runner proceeds without prompting. Per-call approvals are scoped to the specific call ID; pass `always_approve=True` or `always_reject=True` to persist the same decision for future calls to the same tool identity during the rest of the run.
3. If the approval rule requires approval and no decision for that tool call is stored, execution pauses, and `RunResult.interruptions` (or `RunResultStreaming.interruptions`) contains [`ToolApprovalItem`][agents.items.ToolApprovalItem] entries with details such as `agent.name`, `tool_name`, and `arguments`. This includes approvals raised after a handoff or inside nested `Agent.as_tool()` executions.
4. Convert the result to a `RunState` with `result.to_state()`, call `state.approve(...)` or `state.reject(...)`, and then resume with `Runner.run(agent, state)` or `Runner.run_streamed(agent, state)`, where `agent` is the original top-level agent for the run.
5. The resumed run continues where it left off and will re-enter this flow if new approvals are needed.
Sticky decisions created with `always_approve=True` or `always_reject=True` are stored in the run state, so they survive `state.to_string()` / `RunState.from_string(...)` and `state.to_json()` / `RunState.from_json(...)` when you resume the same paused run later.
For approval requests from [`HostedMCPTool`][agents.tool.HostedMCPTool], the Agents SDK identifies a sticky tool decision by the combination of `server_label` and tool name. An always-approve decision for `lookup_account` on one hosted MCP server does not approve a tool with the same name on another server. The Agents SDK persists an always-approve or always-reject decision only when the hosted MCP approval request includes both non-empty identity fields.
You do not need to resolve every pending approval in the same pass. `interruptions` can contain a mix of regular function tools, hosted MCP approvals, and nested `Agent.as_tool()` approvals. If you rerun after approving or rejecting only some items, those resolved calls can continue while unresolved ones remain in `interruptions` and pause the run again.
## Custom rejection messages
@@ -95,7 +100,7 @@ When these callbacks return a decision, the run continues without pausing for a
The same interruption flow works in streaming runs. After a streamed run pauses, keep consuming [`RunResultStreaming.stream_events()`][agents.result.RunResultStreaming.stream_events] until the iterator finishes, inspect [`RunResultStreaming.interruptions`][agents.result.RunResultStreaming.interruptions], resolve them, and resume with [`Runner.run_streamed(...)`][agents.run.Runner.run_streamed] if you want the resumed output to keep streaming. See [Streaming](streaming.md) for the streamed version of this pattern.
If you are also using a session, keep passing the same session instance when you resume from `RunState`, or pass another session object that points at the same backing store. The resumed turn is then appended to the same stored conversation history. See [Sessions](sessions/index.md) for the session lifecycle details.
If you are also using a session, keep passing the same session instance when you resume from `RunState`, or pass another session object configured for the same session ID and backing store. The resumed turn is then appended to the same stored conversation history. See [Sessions](sessions/index.md) for the session lifecycle details.
## Example: pause, approve, resume
@@ -106,14 +111,15 @@ import asyncio
import json
from pathlib import Path
from agents import Agent, Runner, RunState, function_tool
from agents import Agent, Runner, RunState
from agents.decorators import tool
async def needs_oakland_approval(_ctx, params, _call_id) -> bool:
return "Oakland" in params.get("city", "")
@function_tool(needs_approval=needs_oakland_approval)
@tool(needs_approval=needs_oakland_approval)
async def get_temperature(city: str) -> str:
return f"The temperature in {city} is 20° Celsius"
@@ -165,16 +171,16 @@ if __name__ == "__main__":
In this example, `prompt_approval` is synchronous because it uses `input()` and is executed with `run_in_executor(...)`. If your approval source is already asynchronous (for example, an HTTP request or async database query), you can use an `async def` function and `await` it directly instead.
To stream output while waiting for approvals, call `Runner.run_streamed`, consume `result.stream_events()` until it completes, and then follow the same `result.to_state()` and resume steps shown above.
To use streaming in a run that may pause for approvals, call `Runner.run_streamed`, consume `result.stream_events()` until it completes, and then follow the same `result.to_state()` and resume steps shown above.
## Repository patterns and examples
- **Streaming approvals**: `examples/agent_patterns/human_in_the_loop_stream.py` shows how to drain `stream_events()` and then approve pending tool calls before resuming with `Runner.run_streamed(agent, state)`.
- **Custom rejection text**: `examples/agent_patterns/human_in_the_loop_custom_rejection.py` shows how to combine run-level `tool_error_formatter` with per-call `rejection_message` overrides when approvals are rejected.
- **Agent as tool approvals**: `Agent.as_tool(..., needs_approval=...)` applies the same interruption flow when delegated agent tasks need review. Nested interruptions still surface on the outer run, so resume the original top-level agent rather than the nested one.
- **Local shell and apply_patch tools**: `ShellTool` and `ApplyPatchTool` also support `needs_approval`. Use `state.approve(interruption, always_approve=True)` or `state.reject(..., always_reject=True)` to cache the decision for future calls. For automatic decisions, provide `on_approval` (see `examples/tools/shell.py`); for manual decisions, handle interruptions (see `examples/tools/shell_human_in_the_loop.py`). Hosted shell environments do not support `needs_approval` or `on_approval`; see the [tools guide](tools.md).
- **Local shell and apply_patch tools**: `ShellTool` and `ApplyPatchTool` also support `needs_approval`. Use `state.approve(interruption, always_approve=True)` or `state.reject(..., always_reject=True)` to cache the decision for future calls to that tool during the rest of the run. For automatic decisions, provide `on_approval` (see `examples/tools/shell.py`); for manual decisions, handle interruptions (see `examples/tools/shell_human_in_the_loop.py`). Hosted shell environments do not support `needs_approval` or `on_approval`; see the [tools guide](tools.md).
- **Local MCP servers**: Use `require_approval` on `MCPServerStdio` / `MCPServerSse` / `MCPServerStreamableHttp` to gate MCP tool calls (see `examples/mcp/get_all_mcp_tools_example/main.py` and `examples/mcp/tool_filter_example/main.py`).
- **Hosted MCP servers**: Set `require_approval` to `"always"` on `HostedMCPTool` to force HITL, optionally providing `on_approval_request` to auto-approve or reject (see `examples/hosted_mcp/human_in_the_loop.py` and `examples/hosted_mcp/on_approval.py`). Use `"never"` for trusted servers (`examples/hosted_mcp/simple.py`).
- **Hosted MCP servers**: Set `tool_config={"require_approval": "always"}` on `HostedMCPTool` to force HITL, optionally providing `on_approval_request` to auto-approve or reject (see `examples/hosted_mcp/human_in_the_loop.py` and `examples/hosted_mcp/on_approval.py`). Use `"never"` for trusted servers (`examples/hosted_mcp/simple.py`).
- **Sessions and memory**: Pass a session to `Runner.run` so approvals and conversation history survive multiple turns. SQLite and OpenAI Conversations session variants are in `examples/memory/memory_session_hitl_example.py` and `examples/memory/openai_session_hitl_example.py`.
- **Realtime agents**: The realtime demo exposes WebSocket messages that approve or reject tool calls via `approve_tool_call` / `reject_tool_call` on the `RealtimeSession` (see `examples/realtime/app/server.py` for the server-side handlers and [Realtime guide](realtime/guide.md#tool-approvals) for the API surface).
@@ -186,19 +192,11 @@ Useful serialization options:
- `context_serializer`: Customize how non-mapping context objects are serialized.
- `context_deserializer`: Rebuild non-mapping context objects when loading state with `RunState.from_json(...)` or `RunState.from_string(...)`.
- `strict_context=True`: Fail serialization or deserialization unless the context is already a
mapping or you provide the appropriate serializer/deserializer.
- `context_override`: Replace the serialized context when loading state. This is useful when you
do not want to restore the original context object, but it does not remove that context from an
already serialized payload.
- `include_tracing_api_key=True`: Include the tracing API key in the serialized trace payload
when you need resumed work to keep exporting traces with the same credentials.
- `strict_context=True`: Fail serialization unless the context is already a mapping or you provide `context_serializer`; fail deserialization unless the context is already a mapping or you provide `context_deserializer`.
- `context_override`: Replace the serialized context when loading state. This is useful when you do not want to restore the original context object, but it does not remove that context from an already serialized payload.
- `include_tracing_api_key=True`: Include the tracing API key in the serialized trace payload when you need resumed work to keep exporting traces with the same credentials.
Serialized run state includes your app context plus SDK-managed runtime metadata such as approvals,
usage, serialized `tool_input`, nested agent-as-tool resumptions, trace metadata, and server-managed
conversation settings. If you plan to store or transmit serialized state, treat
`RunContextWrapper.context` as persisted data and avoid placing secrets there unless you
intentionally want them to travel with the state.
Serialized run state includes your app context plus SDK-managed runtime metadata such as approvals, usage, serialized `tool_input`, nested agent-as-tool resumptions, trace metadata, and server-managed conversation settings. If you plan to store or transmit serialized state, treat `RunContextWrapper.context` as persisted data and avoid placing secrets there unless you intentionally want them to travel with the state.
## Versioning pending tasks
+8 -7
View File
@@ -17,21 +17,22 @@ The SDK has two driving design principles:
Here are the main features of the SDK:
- **Agent loop**: A built-in agent loop that handles tool invocation, sends results back to the LLM, and continues until the task is complete.
- **Agents**: Build agents with instructions, tools, guardrails, handoffs, and a built-in loop that continues until the task is complete.
- **Sandbox agents**: Run specialists inside real isolated workspaces. Sandbox agents support manifest-defined files, sandbox client selection, and resumable sandbox sessions.
- **Realtime agents**: Build powerful voice agents with `gpt-realtime-2.1`, automatic interruption detection, context management, guardrails, and more.
- **Voice agents**: Build voice pipelines that combine speech-to-text, an agent workflow, and text-to-speech.
- **Python-first**: Use built-in language features to orchestrate and chain agents, rather than needing to learn new abstractions.
- **Agents as tools / Handoffs**: A powerful mechanism for coordinating and delegating work across multiple agents.
- **Sandbox agents**: Run specialists inside real isolated workspaces with manifest-defined files, sandbox client choice, and resumable sandbox sessions.
- **Guardrails**: Run input validation and safety checks in parallel with agent execution, and fail fast when checks do not pass.
- **Function tools**: Turn any Python function into a tool with automatic schema generation and Pydantic-powered validation.
- **MCP server tool calling**: Built-in MCP server tool integration that works the same way as function tools.
- **MCP server tool calling**: Built-in integration that exposes remote MCP tools to agents alongside function tools.
- **Sessions**: A persistent memory layer for maintaining working context within an agent loop.
- **Human in the loop**: Built-in mechanisms for involving humans across agent runs.
- **Human in the loop**: Built-in mechanisms for involving humans during agent runs.
- **Tracing**: Built-in tracing for visualizing, debugging, and monitoring workflows, with support for the OpenAI suite of evaluation, fine-tuning, and distillation tools.
- **Realtime Agents**: Build powerful voice agents with `gpt-realtime-1.5`, automatic interruption detection, context management, guardrails, and more.
## Agents SDK or Responses API?
The SDK uses the Responses API by default for OpenAI models, but it adds a higher-level runtime around model calls.
The SDK uses the Responses API by default for OpenAI models, but it wraps model calls in a higher-level runtime.
Use the Responses API directly when:
@@ -93,5 +94,5 @@ Use this table when you know the job you want to do, but not which page explains
| Keep memory across turns | [Running agents](running_agents.md#choose-a-memory-strategy) and [Sessions](sessions/index.md) |
| Use OpenAI models, websocket transport, or non-OpenAI providers | [Models](models/index.md) |
| Review outputs, run items, interruptions, and resume state | [Results](results.md) |
| Build a low-latency voice agent with `gpt-realtime-1.5` | [Realtime agents quickstart](realtime/quickstart.md) and [Realtime transport](realtime/transport.md) |
| Build a low-latency voice agent with `gpt-realtime-2.1` | [Realtime agents quickstart](realtime/quickstart.md) and [Realtime transport](realtime/transport.md) |
| Build a speech-to-text / agent / text-to-speech pipeline | [Voice pipeline quickstart](voice/quickstart.md) |

Some files were not shown because too many files have changed in this diff Show More