When the Codex MCP reviewer provably failed before dispatch (tool absent, spawn failure, rejected pre-execution), reviewer-aware workflows previously ended in REVIEW_UNAVAILABLE even with another OpenAI-compatible endpoint configured. The existing llm-chat MCP can now expose review/review_reply behind LLM_REVIEW_FALLBACK_ENABLED=true (default off).
Boundaries: pre-dispatch-only — never after a timeout or ambiguous failure, where the review may already have run; verdict calls are single-attempt fail-closed (no retry, no fallback model); primary-artifact files are read server-side and shipped to the endpoint; the gate gives llm-chat less trust than existing backends (same/unknown family fails, independence recorded caller_declared, copilot-finalizer states still require codex or manual).
Closes#412
Seven commits, three rounds of cross-model review before merge.
**experiment-queue scheduler** — five compounding defects in the detached scheduler that gets scp'd to the GPU host, a file with no test coverage. screen_exists() built its grep pattern inside an f-string as '.{name}\t', searching for a literal backslash + t while screen -ls emits a real TAB, so every healthy running job was judged dead at the first 60s poll and its screen killed. The 'screen alive, process gone' branch had a comment and no return and fell through to running forever — unreachable until the first fix made it live. failed_other had no exit and was not terminal, so the scheduler looped forever. output_exists() quoted the whole path and killed glob expansion while the manifest examples use wildcards. And depends_on given as a bare string — the shape both SKILL.md examples showed — was iterated character by character, so a dependent phase never became ready: a third way to hang, caused by our own docs. load_state normalizes the string form on both the fresh-manifest and resume paths so manifests and state files from the old docs still run.
**manual-review cross-family gate** — validate_reviewer_identity() only enforces when the caller passes require_reviewer_model, and that flag appeared in exactly one skill. research-review, experiment-audit, idea-creator, proof-checker and rebuttal now pass it. Landed together with all its prerequisites: the family table widened to DeepSeek/Moonshot/Qwen (the models the guide recommends were the ones the gate refused), the browser UI now shows the server's rejection reason instead of a generic retry prompt, file mode no longer deletes the response the user just pasted, and both MANUAL_REVIEW_GUIDE.md and _CN.md document the Reviewer-Model header in the same commit — the required action is a human paste, which no skill edit can deliver.
**Smaller fixes** — research_wiki matched '# One-line thesis' while the renderer writes '## One-line thesis', so query_pack's Key Papers has always been title-only, silently degrading /idea-creator's main input. figure-spec invoked mcp__codex__codex without granting it, on the default paper path. reviewer-routing spelled the gemini tools with an underscore in five places. forensics_gate now names the coverage dimensions that never ran, without re-deciding on them. The kill-argument verdict table claimed every (counts, severity) tuple hits exactly one row and did not, in both directions. Four Chinese backend-switching guides scoped a rewrite job to 12 skills when the real numbers are 46 grants / 28 invocation blocks.
Deliberately not changed: launch_job's PID capture records the bash wrapper, but if python exits the pipe closes and the wrapper and screen exit with it, so the screen-gone check catches that; the realistic failure is a hung python where every PID is alive. The root README's dated News entries keep their original wording — they record what shipped that day.
Full suite 610 passed on both CI legs; check_skills_inventory consistent.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fix 3 issues from wanshuiyin's round-2 CHANGES_REQUESTED review on PR #360:
1. Shell injection (HARD BLOCKER): Replace inline --prompt "..." interpolation
of REVIEWER_MEMORY.md content with temp-file + heredoc + "$(cat ...)" pattern
in copilot --agent calls (Phase B.6 rebuttal ruling, Round 2+ template, and
reviewer-routing.md multi-round example). Untrusted reviewer text containing
quotes, backticks, or $() is now passed as file bytes, not re-parsed as
shell syntax.
2. reviewer_backend conflation: Add round_backend snapshot in Phase B.5.1 before
escalation updates REVIEW_STATE.json. Phase E now uses round_backend to label
the current round, while reviewer_backend in state controls next-round
continuation. Eliminates risk of mislabeling copilot-driven round data as
backend=codex.
3. Cross-family escalation check: Add executor_family-based backend selection in
Phase B.5.1 escalation gate. anthropic/google executors escalate to codex
(cross-family); openai executors escalate to manual (avoids same-family codex
escalation that would defeat the cross-family acquittal guarantee).
Signed-off-by: Yuchen Fan <functionhx@gmail.com>
idea-creator's body explicitly instructs invoking three sub-skills — Phase 4.2
"Run the /novelty-check workflow", Phase 5.2 "Use /run-experiment", Phase 5.3
"Use /monitor-experiment" — but allowed-tools lacked `Skill`, so the executor
was told to invoke skills it had no grant for and silently improvised inline
instead (the "skips sub-skills and generates ideas itself" behavior reported in
#284). Added the plain `Skill` grant (matching the 15 existing orchestrating
skills).
Also, per the cross-model triage of #284:
- Overview reworded (main + both mirrors): Phase 1's landscape survey is inline
by design (it never invokes /research-lit); the full sub-skill pipeline is
/idea-discovery's job. The old "composes with /research-lit..." phrasing
misled standalone users.
- reviewer-routing.md: new section "codex exec CLI is NOT an equivalent Codex
backend" — codex exec --ephemeral is a stateless one-shot; hand-rewriting all
MCP reviewer calls to it loses same-thread codex-reply continuity (which
Phase 4's triage requires) and tends to mangle skill instructions.
- RESEARCH_BRIEF length guideline (templates EN/CN + /idea-discovery hint):
core brief 1-2 pages (~4-8k chars); the research-wiki query_pack is
hard-capped at 8,000 chars.
Hard evidence-gating for /idea-discovery (issue Q4) is deferred to a follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- gpt-5.5-pro confirmed to exist: bump every CURRENT "GPT-5.4 Pro" / "gpt-5.4-pro"
→ 5.5 Pro across README (en/cn), reviewer-routing.md, fan-out-pattern.md, and the
skills-codex mirrors. Kept the two dated 2026-04-13 News entries at GPT-5.4 Pro
(historical — the model Oracle Pro shipped with then).
- §4 Features: add a one-paragraph lead-in (lifecycle + cross-model review +
integrations) and fold the 11-bullet list (with its nested Feishu preview) behind
<details>. en + cn.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#267 (by @ZGJY95) merged the gemini-review MCP's Antigravity backend. Document it
as the fourth reviewer backend so skills can route to it:
- reviewer-routing.md: new "Optional: Gemini via Antigravity CLI (— reviewer: agy)"
section (routing logic, the cross-model fail-closed invariant it enforces, install,
graceful Codex fallback) alongside codex / oracle-pro / manual.
- README What's New: a short 2026-05-31 entry crediting @ZGJY95 (kept 4 entries visible).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Low-severity follow-ups from the corpus drift sweep:
- reviewer-routing: pin the codex default model (gpt-5.5) in the default block — it
was the only doc leaving the baseline unversioned while trace/assurance/resumable
all pin gpt-5.5. (oracle-pro stays gpt-5.4-pro — intentional per the 2026-05-14
News entry, a separate path unaffected by the codex 5.4->5.5 bump; not a contradiction.)
- acceptance-gate: "structurally unable to terminate-accept" -> a design rule the
author enforces, not an automatic mechanism (the doc is a decision procedure).
- injection-hygiene: layer-1 "fail-closed" -> "block-on-hit (fail-OPEN to novelty)"
(a regex blacklist can't be a completeness guarantee — the doc says so elsewhere);
and a §2-resolution note so the illustrative `tools/threat_scan.py` CLI form isn't
copied as a hardcoded path into a SKILL.
Deferred (your call): interview-cheatsheet `--byline` default bakes a real name+lab
into a public template — left as intentional self-attribution pending a decision.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Work-block 1/6 of the ultracode×ARIS integration. Three new normative
reference docs that let ARIS skills fan out for breadth on ANY runtime tier
(not just paid ultracode) while keeping the cross-model jury intact:
- fan-out-pattern.md — decouple FAN-OUT (breadth: same-family subagents
GENERATE candidates) from JURY (cross-model verdict). 3-tier degradation
ladder (ultracode parallel / plain Agent spawn / sequential fallback) all
terminating in the IDENTICAL cross-model jury. dedup is mechanical + before
jury (cost control). When to fan out vs when the single jury IS the value.
Worked examples: kill-argument (Tier-3 sequential via codex threads, no
Agent tool), idea-creator, research-lit (deterministic gate as jury).
- acceptance-gate.md — the provenance rule for autonomous loops (goal-mode /
auto-review-loop): self-judging EXECUTION-completeness (exit code, files
exist, jobs ran) = safe same-model; self-judging QUALITY/CORRECTNESS
(paper good, proof valid, claim supported) = MUST be cross-model.
"A goal/loop can DRIVE; it cannot ACQUIT." Type-A/B taxonomy, compound-gate
decomposition, decision procedure, 12-row map of existing ARIS loops.
- external-cadence.md — when /loop (or schedule/CronCreate) is additive
(external-world-wait: GPU done? overnight heartbeat?) vs harmful (wrapping
internal semantic loops breaks threadId continuity / re-runs verdicts on a
timer). Fence list (verdict-bearing skills: do NOT wrap) + affordance list.
Drafted by 3 parallel Claude subagents (ultracode firepower), then 2 rounds
of codex GPT-5.5 xhigh cross-model review (the heterogeneous jury — the very
pattern these docs codify). Codex round 1 caught: idea-creator example
falsely called "judgment-free", gemini listed as a non-existent reviewer
backend, auto-review-loop stop condition mis-described. Round 2 caught a
cross-doc inconsistency (reviewer-routing's manual section still said "any
model"). All fixed; reviewer-routing.md manual section now mandates a
non-Claude target (closes the same gap the #246 manual-review UI banner does).
Codex review also surfaced 2 real pre-existing skill bugs, queued for the
follow-up work-blocks: (a) idea-creator Phase 3 does same-family novelty
elimination before the cross-model jury (→ fix when fanning it out, WB3);
(b) auto-review-loop's Constants (OR) and Phase-E stop check (AND) disagree
(→ reconcile in WB4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Human-in-the-loop MCP server for ARIS cross-model review — opens a browser page (or writes a file on headless Linux) where the user copies the prompt to any AI model and pastes the response back. Zero API cost, works with any text model.
Three rounds of cross-model review caught and the author fixed all of: (1) cross-model invariant guardrail — bilingual UI warning + file-mode prompt.md warning + Claude removed from recommended reviewer list, so Claude-only users don't unintentionally self-review; (2) CORS lockdown — removed `Access-Control-Allow-Origin: *`, per-session one-time token required on every endpoint, Origin / Sec-Fetch-Site defense-in-depth; (3) tests rewritten as proper pytest (15 unit + 1 integration); (4) `— reviewer: manual` now fail-closed if MCP not installed (no silent Codex fallback — target users don't have Codex); (5) concurrency hardening — per-thread pending state, second concurrent call fail-fast, notifications/cancelled only cancels matching requestId, 10x cancel-stress regression test; (6) skill wiring trimmed to the 6 SKILLs with actual reviewer call blocks.
Reviewer-plugin protocol generalization (DeepSeek / Kimi / local-llama backends as ARIS plugin spec) deferred to a separate follow-up PR per author's preference.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per maintainer note: when users invoke '--- reviewer: oracle-pro'
(especially o3-deep-research / gpt-5.5-pro paths), Oracle MCP behavior
can be affected by in-flight upstream fixes. ARIS does not vendor
Oracle MCP — users run the published @steipete/oracle from npm.
Surface this so users know to check upstream PRs before opening an
issue against ARIS for what may be an upstream-only problem.
- skills/shared-references/reviewer-routing.md: append 'Upstream
development & known issues' subsection linking to the steipete/oracle
pulls page, with examples of what kinds of in-flight fixes are
worth watching (model routing, browser-mode auth, rate-limit,
new model aliases).
- README.md / README_CN.md: 1 News entry summarizing the same triage
flow.
No skill behavior change. No new dependency. Pure documentation.
New shared-reference reviewer-routing.md defines the routing logic:
- Default = Codex MCP xhigh (UNCHANGED, always)
- `— reviewer: oracle-pro` routes through Oracle MCP (GPT-5.4 Pro)
- Oracle not installed = graceful fallback to Codex + warning
- Supports both API mode and browser mode
Added REVIEWER_BACKEND constant to 7 skills:
research-review, experiment-audit, proof-checker, rebuttal,
idea-creator, research-lit, auto-review-loop
Safety guarantees:
- No parameter = no change from current behavior
- Oracle not installed = zero impact
- Reviewer independence protocol still applies
- effort and difficulty remain orthogonal
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JLTian