01aef34ca49db1ccc9caaee72913760f4468f6c1
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Matt Van Horn
|
01aef34ca4 |
docs(solutions): compound the checkpointed host-judgment protocol learning (#858)
Validate / mcp-tests (push) Has been cancelled
Scorecard / Scorecard analysis (push) Has been cancelled
Security / Dependency audit (push) Has been cancelled
Security / Dependency review (push) Has been cancelled
Security / Secret scan (push) Has been cancelled
Security / SAST scan (push) Has been cancelled
Validate / tests (push) Has been cancelled
Validate / eval (push) Has been cancelled
GitHub Actions Security Analysis with zizmor 🌈 / zizmor (push) Has been cancelled
New architecture-patterns doc: five design conventions for the checkpointed discovery protocol (identity/TTL binding, lossless-state vs fenced-digest split, fail-closed empty state, provenance enforcement, guarded writes + stale-sibling invalidation), with the meta-lesson that an external LLM API call inside an engine whose invoker IS an LLM is an architecture smell. 56/58 claims grounded against the tree by the validation pass; the two exceptions were corrected in place. CONCEPTS.md gains Handoff checkpoint and refreshes Junk shape (host verdict vs heuristic flag) and Covered (judge is now the hosting model). Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Matt Van Horn
|
5ccda4036b |
feat(discover): host-judged three-leg discovery protocol (#856)
* fix(discover): fold same-story duplicates, velocity-sorted ranks, loud fallback note Survivors sharing evidence (identical top_comment or >=2 shared evidence URLs) fold to the higher-velocity twin; display ranks and angle topic_ids are assigned from the velocity-sorted order so rank order matches the displayed velocity_score; non-mock runs with no reasoning provider emit one loud stderr note. Mock fixture URLs/comments are namespaced per topic slug so the fold cannot collapse distinct mock topics. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover): handoff contracts module - nominations bundle, judgments, angles New lib/discovery_handoff.py owns the three-leg protocol file contracts: versioned bundle writer/reader (full seed items as engine resume state, bundle id, TTL, momentum window, leg-1 invocation context), strict-top/ lenient-row judgments reader with ported name sanitation and collision disambiguation over host names, angles reader with the ported 200-char cap, host-facing digest builder, and typed HandoffContractError for exit-2 mapping that names both searched locations. schema.py gains the discovery-nominations kind and duck-typed nomination serializers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover): three-leg protocol CLI - nominate-only, judgments, finalize, angles flags New flags with orphan/mutual-exclusion rejection (exit 2, message names the combination), mock protocol legs require --save-dir to stay side-effect-free, one handoff-state resolver (save-dir else config dir), and a dispatch wrapper that maps HandoffContractError to stderr + exit 2. Leg bodies are distinct NotImplementedError stubs replaced by U3-U5. Bare --discover and --discover-shallow dispatch regression-pinned unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover): nominate leg - sweep to bundle, digest, nothing-solid short-circuit run_discover_nominate shares the exact sweep/cluster/nominate code with the one-shot path (factored _discovery_sweep, nominate_topic_pool, shared source-boundary helpers), cuts at the full judge pool instead of the enrichment limit, never resolves providers, and writes the versioned bundle with heuristic fallback names, momentum window, and leg-1 invocation context. Zero nominations renders the nothing-solid brief in leg 1 with no bundle. Digest names the bundle path and mandates reading its evidence before judging. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover): resume leg - host judgments, deep enrichment, pending report run_discover_resume applies host judgments (per-row heuristic fallback, collision resolution over the whole pool), excludes host-junk from enrichment slots outright, skips sub-corroborated heuristic junk pre-enrichment, blend-cuts to the slot limit, and runs tier-parameterized enrichment (deep: default depth, 4 workers, LAST30DAYS_ENRICH_BUDGET_SECONDS default 450 via the config allowlist; shallow and one-shot keep quick/240/3, pinned both ways). Velocity scores against the bundle window. Floor, fold, and velocity ranks share the one-shot code via extracted helpers. One main-thread pending-report write (fresh TTL, run_ref, angle inputs keyed by surviving nomination ids); stdout emits angle inputs plus instructions. No queue writes on this leg. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover): finalize leg - host angles, render, idempotent queue write --discover --finalize loads the pending report (typed errors name both searched locations and the resume-leg remedy; TTL from the leg-2 write), applies host angles by nomination id, renders through the one-shot's emit flow with O_EXCL artifact saving, and records the topic queue once, guarded. record_discovery_surfacing gains a per-run idempotency guard: a matched row already stamped with this run_ref neither increments nor re-annotates, so finalize retries render byte-identical output. Mock finalize stays queue-free. The full mock three-leg sequence is pinned end-to-end. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(discover)!: delete the engine-side LLM judge - the host is the judge Removes lib/discovery_judge.py and every provider touch in the discovery path: no resolve_runtime in run_discover, no stage-1 verdict blending, no stage-2 angle pass. One-shot discovery always names via topic_shape heuristics, ranks velocity-only, renders no angles, and emits one loud note pointing at the SKILL.md host-judged protocol (no provider-key advice). Mock guard tightened to no-provider-client-constructed across all legs; grep-level pins keep the judge from coming back. providers.py untouched for the normal pipeline. Coverage 87.53% vs the 84 floor. Keyed one-shot cron users lose LLM naming/angles by design; the protocol replaces them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * feat(skill): LAW 11 - YOU ARE THE JUDGE; three-command discovery protocol in SKILL.md New LAW 11 with the LAW-7-style anticipated-misread note (the one-shot heuristics stderr line is a skipped-protocol signal, never a capability constraint) and a pre-Bash self-check. Step 1 DISCOVERY branch rewritten as the three-leg protocol: nominate-only (180s), judge via mktemp heredoc judgments file, resume (600s), host angles file, finalize (60s) relay-verbatim; identical --save-dir threaded through all legs; fail-twice degradation to bare one-shot; shallow tier for sub-8-minute shell caps. CONFIGURATION.md documents the four new flags; CONCEPTS.md Discovery/Nomination name the host as judge; README trending mentions updated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * refactor(discover): consolidate handoff validation, fix leg-3 binding error text Simplify pass on the branch: shared _parse_handoff_envelope for the two engine-written files, one _search_paths helper, shared _known_rows gate for host-file rows, schema._source_status_from_dict for both report deserializers, schema._utc_now reuse, _discover_domain helper. Real fix: _require_bundle_binding now names the pending report and the resume-leg remedy when the finalize leg's angles file mismatches - it previously pointed the host's retry at the nominations bundle; regression-pinned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * fix(review): handoff contract hardening - save-dir isolation, junk type-gate, fail-closed bundle, fenced digest Explicit save-dir is now the single handoff store (no config-dir fallback; matches scoped-db semantics and SKILL.md's own contract); judgments junk accepts only real booleans (null/"false" fall back per-row); non-list or zero-valid-row nominations fail closed instead of rendering nothing-solid; bundle writes raise the protocol's exit-2 error on OSError instead of a traceback; bundle_id mismatch remedy now says fix-the-id-and-retry-this-leg; digest evidence rides inside the untrusted-content fence the deleted judge used; SKILL.md documents the leg-2 budget knob and data-not-instructions rule. Lenient-row paths regression-pinned. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * fix(review): protocol leg semantics - degraded-state plumbing, retry history, guard hoist, mock parity Bundle and pending report now carry the leg-1 sweep's source_status and a mock flag; the resume report restores degraded-source warnings and one shared strict-exit helper gates every leg terminal (nothing-solid paths included) exactly like the one-shot. Finalize retries reconstruct pre-run queue history instead of dropping Pipeline lines; --as-of and html-emit guards hoisted to all discover invocations; malformed pending bodies and unwritable pending writes become clean exit-2 contract errors; fresh rounds invalidate stale pending files; the same-story fold re-scans to a fixpoint so three-way chains collapse; mock/real handoff state cannot cross legs. Duplicated render/save and queue-warning blocks extracted into shared helpers. Coverage 87.68%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * docs(changelog): unreleased entries for host-judged discovery protocol Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 * docs(changelog): fill PR number for host-judged discovery entries Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BL9KsoytvnR1WfvZa7SEN7 --------- Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Matt Van Horn
|
27e17245b6 |
docs(solutions): compound two verified learnings from the PR #852 discovery content pipeline (#853)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> |
||
|
Matt Van Horn
|
4b027919c7 |
docs(solutions): compound three verified learnings from the PR #816 discovery rebuild
- design-patterns/ranked-output-confidence-floor-honest-empty-state: top-N rankers need an absolute confidence floor and a first-class 'nothing solid' empty state - relative ranking cannot express 'none of this is good enough' - logic-errors/non-daemon-executor-threads-defeat-wall-clock-budget: a wall-clock budget over ThreadPoolExecutor is answer-bounding, not process-bounding; daemon threads + semaphore + monotonic deadline make it real - conventions/argparse-optional-value-flag-dispatch-truthiness: nargs='?' const='' flags are three-state; dispatch on 'is not None', dependent flags fail loudly CONCEPTS.md gains a Discovery cluster (Discovery, Nomination, Enrichment pass, Confidence floor, Nothing-solid) plus a flagged ambiguity on 'enrichment'. All claims validated against the tree (mechanical + semantic grounding passes). |
||
|
Trevin Chow
|
7d366e0cc7 | Merge branch 'main' into tmchow/competitor-roadmap-prediction | ||
|
Trevin Chow
|
a0f7789893 | feat: surface hiring signals from jobs pages | ||
|
Trevin Chow
|
4402dd317f |
docs: changelog, CONCEPTS research-pipeline cluster, grounding solution doc
CHANGELOG covers the narrative lens (Added) and head-token grounding fix (Fixed). CONCEPTS.md gains a Research pipeline cluster (Primary entity, Intent modifier, Entity grounding, Keyless path, Comment-enrichment slots). New docs/solutions/logic-errors/ entry captures the false-demotion bug, the rejected alternatives, and the shared-helper prevention rule. |
||
|
Trevin Chow
|
0fd532d249 |
docs: compound learning on release-time consistency-test cascade failures
Documents the cascade pattern surfaced during this session's install-modernization arc: a `test_sync_cache_path_uses_skill_version` test asserted that a hardcoded version pin in `sync.sh` matched the version frontmatter in SKILL.md. When a release bumped SKILL.md, every open PR's CI failed simultaneously on the unrelated stale-pin assertion. Affected at least 5 PRs across the 2026-05-13 to 2026-05-15 window (#400, #390, #392, and two others) plus required hotfix PR #397 to unblock the queue. The permanent fix shipped in PR #405 (deleted sync.sh + the test). This doc captures the design lesson so the pattern doesn't reappear: don't write consistency tests that read two files and assert one matches a value derived from the other. Either derive at runtime from a single source of truth, or self-skip / merge-base-scope the test so deletion is a non-event. Created via /ce-compound. Includes: - docs/solutions/workflow-issues/release-consistency-test-cascade-2026-05-16.md (the new learning — first entry under docs/solutions/) - CONCEPTS.md (new — 4 entries: Skill, Engine, Harness, Beta channel, capturing project-specific vocabulary that surfaced across the session) - AGENTS.md (added one-line Structure entries surfacing docs/solutions/ and CONCEPTS.md so fresh agents discover them) - docs/plans/2026-04-22-{002,003,005,006}-*-plan.md (added deprecation banner to each, pointing readers at PR #405 and the new docs/solutions entry — these 4 historical plans still reference the deleted sync.sh inline) Also: closed PR #379 (j-sperling's workaround for the same cascade, superseded by PR #405's permanent fix). |