github-featured/microsoft--skillopt
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

feat(sleep): adopt reviewed skill subsets safely (#212)

Browse Source 
* feat(sleep): adopt reviewed skill subsets safely

* fix(sleep): wire cycle staging and adopt-time review checks

Address PR 212 review: run_sleep_cycle stages resolved SkillProposals,
status/adopt list and select a subset, uniqueness is rechecked at adopt,
and a failed adopted_skills.json write rolls live files back.

Refs microsoft/SkillOpt#212

* test(sleep): mega-cover PR 212 review paths

Adversarial CLI, adopt-time, cycle-staging, and auto-adopt cases for
Yifan's five review items. Also tidy isort on the files this slice
touches.

Refs microsoft/SkillOpt#120

* fix(sleep): pin staged skill hashes and confine adopt targets

Harden PR 212 adopt: sha256 pin each staged skill, revalidate the
whole manifest before any live write, refuse symlink/missing-parent
targets, skip notes on the cycle report, and reject empty --skill.

Refs microsoft/SkillOpt#212

* fix(sleep): harden multi-skill fan-out adoption end to end

---------

Co-authored-by: Yif-Yang <yif_yang@qq.com>
This commit is contained in:
Bogdan (Dan) Baciu
2026-08-21 00:34:29 +04:00
committed by GitHub
parent fb1c305f99
commit faf4700ae2
40 changed files with 8329 additions and 334 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+13 -2
View File
@@ -7,6 +7,17 @@ All notable changes to SkillOpt are documented here. This project adheres to
## [Unreleased]
### Added
- **SkillOpt-Sleep multi-skill fan-out and reviewed subset adoption**: each
hinted skill is consolidated from its own pinned live baseline, staged as an
independent proposal with per-skill gate evidence, and promoted only through
an explicit `--skill`, `--all-skills`, or managed `--legacy` choice. Adoption
uses a versioned fail-closed manifest, provenance hashes, canonical target
pins, immutable backups/receipts, cross-night locking, durable publication,
and restart-recoverable transactions. Fan-out discovers native project skill
roots, supports repeatable `--skill-root` overrides, and is enabled by
`multi_skill_fanout` (`multi_skill_report` remains an alias). MCP adapters
enforce typed arguments and preserve engine failures without copying outside
the transaction (thanks @bogdanbaciu21, #212).
- Additive `section_contains` rule-judge operator for literal,
case-insensitive matching within numbered, bilingual, or annotated ATX
Markdown headings. The legacy `section_present` behavior is unchanged.
@@ -120,8 +131,8 @@ All notable changes to SkillOpt are documented here. This project adheres to
Thank you to the contributors behind this unreleased work:
@AKhozya, @Alphaxalchemy, @Phoenix0531-sudo, @SparshGarg999,
@Tanmay9223, @chirag127, @codeL1985, @dimitarvdenev,
@ichoosetoaccept, @jcforever1, @nankingjing, @wilyan09007, @xs229, and
@zixuanguo786-ctrl.
@bogdanbaciu21, @ichoosetoaccept, @jcforever1, @nankingjing, @wilyan09007,
@xs229, and @zixuanguo786-ctrl.
## [0.2.0] — 2026-07-02
docs/guide/installation.md
+3 -2
View File
@@ -28,8 +28,9 @@ checkout for those files.
The generic research `openai_compatible` backend, SkillOpt-Sleep handoff,
Sleep support for non-Azure OpenAI-compatible endpoints, the Sleep
`--preferences` flag, Cursor source/backend/plugin support, and Pi
source/backend support landed after that release and require a source
install from `main` until the next release.
source/backend support, multi-skill fan-out, and reviewed subset adoption
landed after that release and require a source install from `main` until
the next release.
### Source checkout
docs/reference/cli.md
+17 -2
View File
@@ -5,8 +5,9 @@
> Sleep support for non-Azure OpenAI-compatible endpoints, the Sleep
> `--preferences` flag, the research `cursor_exec` target harness, or Cursor
> source/backend/plugin support, Pi source/backend support, OpenCode Sleep
> source/backend support, or VS Code Copilot transcript harvesting; use a source
> install from `main` for those features until the next release.
> source/backend support, VS Code Copilot transcript harvesting, or multi-skill
> fan-out and subset adoption; use a source install from `main` for those
> features until the next release.
## Training
@@ -144,11 +145,25 @@ Actions are `run`, `dry-run`, `status`, `adopt`, `harvest`, `schedule`, and
| `--lookback-hours N` | Initial transcript lookback; `0` scans all history |
| `--max-sessions N` / `--max-tasks N` | Bound the harvested workload |
| `--target-skill-path PATH` | Explicit skill document to stage/adopt |
| `--skill-root PATH` | Add a skill-resolution root; repeatable, with relative paths resolved below `--project` |
| `--tasks-file PATH` | Replay a reviewed task JSON file instead of harvesting |
| `--edit-budget N` | Maximum bounded edits for the night |
| `--progress` / `--json` | Progress or machine-readable output |
| `--auto-adopt` | Apply an accepted staged proposal automatically |
`adopt` also accepts `--skill NAME` (repeatable) and `--all-skills` for a night
that staged per-skill proposals. Use `--legacy` to adopt only a co-staged
managed `SKILL.md` / `CLAUDE.md` proposal. Bare `adopt` on a fan-out night lists
the names and exits instead of promoting anything. A leading-dash name must use
the unambiguous `--skill=--name` form. See
[multi-skill staging](../sleep/multi-skill-staging.md).
Fan-out resolves existing project-native `.agents/skills`, `.claude/skills`,
`.cursor/skills`, and `.devin/skills` directories plus the established Claude
roots. Use `--skill-root` for another integration-specific location. Configure
the canonical `multi_skill_fanout` key to enable proposal fan-out;
`multi_skill_report` remains a compatibility alias.
The `mock` and `handoff` backends make no network calls. A real backend sends
mining, replay, judging, and reflection prompts derived from harvested
transcripts and tasks to its selected provider. Review that provider's
docs/sleep/README.md
+38 -13
View File
@@ -100,7 +100,9 @@ pip install skillopt # installs the engine + the `skillopt-sleep` command
skillopt-sleep dry-run # harvest + mine + replay, report only; stages nothing
skillopt-sleep run # a full nightly cycle; the proposal is staged for review
skillopt-sleep status # show state + the latest staged proposal
skillopt-sleep adopt # apply the latest staged proposal
skillopt-sleep adopt --legacy # apply a reviewed managed proposal
skillopt-sleep adopt --skill NAME # adopt one staged skill (repeatable)
skillopt-sleep adopt --all-skills # adopt every still-pending fan-out skill
skillopt-sleep schedule # install a nightly cron entry for this project
```
@@ -108,8 +110,8 @@ skillopt-sleep schedule # install a nightly cron entry for this project
> commands above. Cursor source/backend/plugin support, VS Code Copilot
> transcript harvesting, Pi source/backend support, Sleep handoff, non-Azure
> OpenAI-compatible endpoints, OpenCode Sleep source/backend support, and
> `--preferences` landed later and require a source install from `main` until
> the next release.
> `--preferences`, multi-skill fan-out, and reviewed subset adoption landed
> later and require a source install from `main` until the next release.
The per-agent integrations below still come from the repo; the CLI above is the
standalone, pip-only way to run a cycle. Claude Code, Codex, Cursor, Copilot, and
@@ -292,22 +294,38 @@ documents the separate HTTPS-only boundary for Azure managed-identity credential
Deterministic proof (no API key):
`python -m skillopt_sleep.experiments.run_experiment --persona researcher --assert-improves`.
### Opt-in: per-skill group reporting
### Opt-in: per-skill fan-out
Set `"multi_skill_report": true` in `~/.skillopt-sleep/config.json` to add an
independent gate result and report row for every explicit skill hint mined that
night:
Set `"multi_skill_fanout": true` in `~/.skillopt-sleep/config.json` to add an
independent gate result and reviewable proposal for every explicit skill hint
mined that night. `multi_skill_report` remains a compatibility alias:
```json
{"multi_skill_report": true}
{"multi_skill_fanout": true}
```
This runs one additional consolidation per group (including a catch-all group when
hinted and unhinted evidence are mixed), so it increases backend calls and token use.
It is reporting-only for now: every group starts from the same managed skill
document, and Sleep does not yet resolve and update several live `SKILL.md` files
automatically. Nights containing only the managed catch-all group keep the existing
single-consolidation behavior.
hinted and unhinted evidence are mixed), so it multiplies backend calls and token
use; configured dream rollouts and synthetic variants multiply the per-group work
too. Each group inherits the configured edit budget, gate mode/metric,
`gate_no_regression`, `dream_rollouts`, `dream_factor`, `recall_k`, and
`evolve_skill`. Recalled archive tasks are restricted to that same skill hint;
shared memory is read-only in fan-out runs. Setting `evolve_skill` to `false`
therefore disables per-skill proposals as well as the managed skill proposal.
Each explicitly hinted group resolves and reads its own live `SKILL.md` before
consolidation, so its staged proposal preserves that skill's baseline. Missing,
ambiguous, unreadable, aliased, or colliding skills are skipped and reported
instead of falling back to the managed document. Adoption remains review-driven:
choose fan-out proposals with `adopt --skill NAME` or `--all-skills`, and use
`adopt --legacy` for a co-staged managed skill/memory pair. `auto_adopt` never
promotes the per-skill fan-out. Nights containing only the managed catch-all group
keep the existing single-consolidation behavior.
Resolution searches existing project-native `.agents/skills`, `.claude/skills`,
`.cursor/skills`, and `.devin/skills` directories, then the established Claude
home and plugin-cache roots. Add repeatable `--skill-root PATH` values when an
integration stores skills elsewhere. Relative roots resolve below `--project`.
### Opt-in: experience replay & dream rollouts
@@ -363,6 +381,13 @@ gate keeps the worst case bounded; keep it **on** by default.
## Learn more
The **low-level** API for staging one proposal per skill and adopting a reviewed
subset (`staged_skills` / `adopt_skills`, plus `status` and `adopt --skill`) is
documented in [`docs/sleep/multi-skill-staging.md`](multi-skill-staging.md).
That page also documents the opt-in nightly fan-out, where every hinted group
derives its proposal from its own resolved live `SKILL.md` while adoption remains
an explicit human decision.
See the [SkillOpt documentation index](../index.md), the
[CLI reference](../reference/cli.md), and the integration-specific READMEs under
[`plugins/`](https://github.com/microsoft/SkillOpt/tree/main/plugins).
docs/sleep/multi-skill-staging.md
+260
View File
@@ -0,0 +1,260 @@
# Multi-skill staging and reviewed adoption
Multi-skill nights separate learning from promotion. The cycle can consolidate
several hinted skills from their own live documents, but it never treats that
fan-out as permission to update every live file.
There are two independent proposal modes, and one night can contain both:
1. **Managed (legacy) proposal** — the aggregate cycle may stage
`proposed_SKILL.md` and `proposed_CLAUDE.md` for the configured managed skill
and project memory.
2. **Per-skill fan-out** — accepted hinted groups may stage one
`proposed_SKILL.<name>.md` each. A reviewer chooses an explicit subset.
`auto_adopt` applies only an accepted managed proposal. It never promotes the
per-skill fan-out. Pending per-skill names remain visible after a managed
auto-adoption.
This feature landed after PyPI 0.2.0. Install from `main` until the next release.
## How the nightly fan-out works
Set the canonical `multi_skill_fanout` option to `true`. The earlier
`multi_skill_report` name remains a compatibility alias. When mined evidence
contains explicit skill hints, the cycle:
1. groups tasks by normalized hint;
2. resolves each name inside existing project-native `.agents/skills`,
`.claude/skills`, `.cursor/skills`, and `.devin/skills` directories, the
established Claude roots, and any repeatable `--skill-root PATH` overrides;
3. reads that skill's exact live `SKILL.md` bytes and canonical path;
4. runs the configured dream/consolidation pipeline independently for the
group; and
5. stages a row only when that group's own gate accepted a non-empty update.
Missing, ambiguous, unreadable, aliased, unsafe, or colliding skills are skipped
with a note in both report formats. They never fall back to the managed skill's
document. The managed catch-all remains on `proposed_SKILL.md` and is not
duplicated as a per-skill row.
Each group inherits `recall_k`, `dream_rollouts`, `dream_factor`, `edit_budget`,
`gate_mode`, `gate_metric`, `gate_mixed_weight`, `gate_no_regression`, and
`evolve_skill`. Recalled archive tasks are restricted to the same skill hint,
and shared memory is read-only during group runs. Consequently,
`evolve_skill=false` disables managed and fan-out skill proposals.
The aggregate consolidation still runs once. Each usable hinted group adds one
independent dream/consolidation run, so provider calls and token use scale with
the number of groups, tasks, and configured rollouts.
## Staging layout
A mixed night can contain managed and per-skill artifacts together:
```text
.skillopt-sleep/staging/20260815-013000/
├── manifest.json
├── proposed_SKILL.md
├── proposed_CLAUDE.md
├── proposed_SKILL.alpha.md
├── proposed_SKILL.beta.md
├── report.json
├── report.md
└── evidence.jsonl
```
`manifest.json` is a versioned, fail-closed format. It retains the old top-level
field names only as safe compatibility sentinels and adds authoritative pinned
proposal rows:
```json
{
"schema": "skillopt-sleep-staging",
"schema_version": 2,
"live_skill_path": "/repo/.agents/skills/managed/SKILL.md",
"live_memory_path": "/repo/CLAUDE.md",
"has_skill": false,
"has_memory": false,
"has_managed_skill": true,
"has_managed_memory": true,
"accepted": true,
"legacy": {
"skill": {
"proposed_file": "proposed_SKILL.md",
"live_path": "/repo/.agents/skills/managed/SKILL.md",
"sha256": "<proposed raw-byte sha256>",
"live_sha256": "<baseline raw-byte sha256>",
"live_realpath": "/repo/.agents/skills/managed/SKILL.md"
},
"memory": {
"proposed_file": "proposed_CLAUDE.md",
"live_path": "/repo/CLAUDE.md",
"sha256": "<proposed raw-byte sha256>",
"live_sha256": "<baseline raw-byte sha256>",
"live_realpath": "/repo/CLAUDE.md"
}
},
"skills": [
{
"skill_name": "alpha",
"proposed_file": "proposed_SKILL.alpha.md",
"live_skill_path": "/home/dev/.claude/skills/alpha/SKILL.md",
"sha256": "<proposed raw-byte sha256>",
"live_sha256": "<baseline raw-byte sha256>",
"live_realpath": "/home/dev/.claude/skills/alpha/SKILL.md"
}
]
}
```
The top-level `has_skill` and `has_memory` compatibility fields are deliberately
always `false`. This makes the pre-feature PyPI 0.2.0 adopter treat a new night
as a no-op instead of bypassing the new validation and transaction engine.
`has_managed_skill` and `has_managed_memory` describe managed proposal presence;
the pinned `legacy` rows are authoritative for adoption. Top-level `accepted`
describes only the aggregate managed gate and does not summarize `skills`. An
aggregate gate may reject while an independently accepted group remains
reviewable in `skills`.
`sha256` pins the exact staged proposal bytes. `live_sha256` pins the raw live
bytes used as the consolidation baseline; an empty string means the file did
not exist. `live_realpath` pins the canonical destination identity. Staging
refuses publication if either live bytes or canonical identity changed after
the baseline read.
The writer reserves each staging directory atomically, writes a complete
artifact batch, and publishes its basename through a private mode-`0600`
`.latest` pointer. Invalid or symlinked pointers fall back only to contained,
reserved nights; adoption cannot reorder nights by changing a directory mtime.
Symlinked staging directories and manifests are ignored or refused.
## Reviewing and adopting
```text
python -m skillopt_sleep status --project PATH
python -m skillopt_sleep adopt --project PATH --staging NIGHT --skill alpha
python -m skillopt_sleep adopt --project PATH --staging NIGHT --skill alpha --skill beta
python -m skillopt_sleep adopt --project PATH --staging NIGHT --all-skills
python -m skillopt_sleep adopt --project PATH --staging NIGHT --legacy
```
Selection modes are mutually exclusive:
- `--skill NAME` is repeatable and promotes only those per-skill rows;
- `--all-skills` promotes every pending per-skill row;
- `--legacy` promotes only the co-staged managed skill/memory pair; and
- bare `adopt` remains convenient for a legacy-only night, but refuses a night
with per-skill rows so it cannot imply “adopt everything.”
Use `--skill=--leading-dash` for a name beginning with `-`. Quote names with
spaces or shell metacharacters according to the active shell. Human guidance
lists names as data and never interpolates them into a copy/paste command.
The Python API uses the same transaction engine:
```python
from skillopt_sleep.staging import (
adopt_skills,
latest_staging,
pending_staged_skills,
)
night = latest_staging("/path/to/project")
names = [row["skill_name"] for row in pending_staged_skills(night)]
receipts = adopt_skills(night, ["alpha"])
```
`skill_names=None` adopts all per-skill rows. An empty sequence adopts nothing.
## Integrity and recovery contract
Before any live mutation, adoption validates the entire relevant manifest and
selected proposal set, including:
- safe single-segment names and expected proposal filenames;
- unique names, staged files, case-folded paths, canonical paths, and live file
identities, including hard-link aliases;
- regular, non-symlink proposal, manifest, receipt, backup, and journal files;
- proposal SHA-256 pins and valid UTF-8;
- live raw-byte hashes, file existence, canonical targets, file identities, and
modes; and
- any prior receipt row against its derived immutable backup and hashes.
Old fan-out or managed manifests without live baseline pins are intentionally
refused. Discard and rerun the night; adoption does not guess a baseline for an
old proposal.
Adoption takes an exclusive staging lock plus stable per-target locks shared
across separate nights. A stale lock fails closed instead of being guessed away.
The locks cover manifest reload, full preflight, backup creation, final live
revalidation, all live replacements, and receipt publication.
Before the first mutation, the engine fsyncs a private mode-`0600`
`.adopt-transaction.json` version-2 write-ahead journal containing the recovery
state, including the identities of directories created by this transaction.
Backups are created without replacement:
```text
backup/skills/<name>/SKILL.md # per-skill original
backup/SKILL.md # managed skill original
backup/CLAUDE.md # managed memory original
```
Per-skill receipts accumulate in `adopted_skills.json`; managed receipts live in
`adopted_legacy.json`. A skill or managed target cannot be re-adopted from the
same night, and existing receipt/backup history must validate before another
subset can be appended.
The engine revalidates the complete target set before and after receipt
publication. The journal is removed only after every selected target and the
receipt are durably published; that removal is the commit point. A caught
failure triggers immediate rollback. If the process stops first, the next
adoption recovers the journal before it trusts the manifest. Recovery restores
only content still equal to this transaction's proposal and removes only empty
created directories whose identities still match the journal. If an external
editor changed content or replaced a directory, recovery preserves it, retains
the journal/backups, and raises `StagingRecoveryError` for manual resolution.
On POSIX, file and parent-directory changes are fsynced. Python's standard
library does not expose an equivalent portable directory flush on Windows, so
the journal and file contents are flushed there but power-loss durability of
directory entries remains filesystem/OS dependent.
The final byte/identity/mode check occurs immediately before atomic replacement,
and all SkillOpt adoption processes share target locks. Portable Python does not
provide a filesystem compare-and-swap against an unrelated process that ignores
those locks; such a process can still race in the final check/replace micro-gap.
Keep live skill editing and adoption coordinated when stronger OS-specific
locking is required.
## Machine interfaces and integrations
`run --json` includes additive `skill_groups` and `staged_skills` fields.
`status --json` always includes `staged_skills` (an empty array on a malformed
manifest) and adds `staging_error` when inspection failed. Adoption success and
failure are single JSON documents; selection-required failures return
`available_skills` as objects with `skill_name` and `live_skill_path`.
The Copilot and Devin MCP `sleep_adopt` tools expose `staging`, `skills`,
`all_skills`, and `legacy`. They forward names as subprocess argument-vector
elements, never shell text. The adapters validate actual JSON-RPC argument types
before launching a subprocess, preserve nonzero engine status, and do not copy
adopted content to a second unpinned destination. Native project skill roots are
adopted directly through the core transaction.
## Migration checklist
- Require `schema="skillopt-sleep-staging"` and `schema_version=2`; unknown or
missing new-format versions fail closed.
- Treat `legacy` and `skills` as the authoritative managed and fan-out rows.
- Expect legacy `has_skill` / `has_memory` compatibility sentinels to be false;
use `has_managed_skill` / `has_managed_memory` for managed presence.
- Interpret top-level `accepted` as aggregate-only.
- Require all proposal and live pins; restage older unpinned nights.
- Preserve unknown additive JSON fields when building external tooling.
- Use `--staging` when automating promotion so “latest” cannot change between
review and adoption.
- Expect append-only receipts and fail-closed locks/recovery conflicts.
- Do not infer that a successful managed auto-adoption also promoted fan-out
rows.
mkdocs.yml
+1
View File
@@ -50,6 +50,7 @@ nav:
- Deep Learning Analogy: guide/dl-analogy.md
- SkillOpt-Sleep:
- Overview: sleep/README.md
- Multi-skill Staging: sleep/multi-skill-staging.md
- OpenAI-compatible Endpoints: sleep/openai-compatible-endpoints.md
- Results: sleep/RESULTS.md
- Extension Guides:
plugins/README.md
+2 -1
View File
@@ -49,7 +49,8 @@ an importable `skillopt_sleep` module. Install with `uv tool install skillopt` o
> supports the base Sleep CLI, while Cursor source/backend/plugin support,
> Pi source/backend support, handoff, Sleep support for non-Azure
> OpenAI-compatible endpoints, OpenCode Sleep source/backend support, and
> `--preferences` require a source checkout from `main` until the next release.
> `--preferences`, multi-skill fan-out, and reviewed subset adoption require a
> source checkout from `main` until the next release.
## One sleep cycle
plugins/claude-code/README.md
+4 -3
View File
@@ -56,8 +56,8 @@ the shared runner falls back first to a `skillopt-sleep` executable on `PATH`
`uv tool install skillopt` or `pip install skillopt` for that fallback.
> **Version note.** This page tracks `main`. PyPI 0.2.0 provides the base Sleep
> CLI, but handoff mode and `--preferences` require a source checkout from
> `main` until the next release.
> CLI, but handoff mode, `--preferences`, multi-skill fan-out, and reviewed
> subset adoption require a source checkout from `main` until the next release.
## Quick start
@@ -66,7 +66,8 @@ the shared runner falls back first to a `skillopt-sleep` executable on `PATH`
/skillopt-sleep dry-run # preview what it would learn; no changes staged
/skillopt-sleep run # full cycle: stages a reviewed proposal (still no live edits)
/skillopt-sleep status # see history + the latest staged proposal
/skillopt-sleep adopt # apply the staged proposal to CLAUDE.md / SKILL.md (with backup)
/skillopt-sleep adopt --legacy # apply the reviewed managed proposal
# Fan-out nights instead use: adopt --skill NAME (repeatable) or --all-skills
/skillopt-sleep-handoff run # same cycle, but THIS session answers the model calls
# (no claude -p subprocess, no API key — subscription-friendly)
plugins/claude-code/commands/skillopt-sleep-handoff.md
+3 -2
View File
@@ -53,8 +53,9 @@ Repeat until the engine exits 0 (done) — at most 8 rounds:
- For `run`, if the engine prints a staging directory, `Read` its `report.md`
and show the user: held-out baseline → candidate score, the gate decision,
the proposed edits, and where the proposal is staged. If an accepted proposal
was staged, tell the user nothing live changed and offer
`/skillopt-sleep adopt`.
was staged, tell the user nothing live changed, inspect `status`, and offer
the exact reviewed selection: `adopt --legacy`, repeatable
`adopt --skill NAME`, or `adopt --all-skills`.
- For `dry-run`, no staging directory or `report.md` is created; summarize the
final stdout instead.
- The engine archives `.skillopt-sleep-handoff/` on a completed real run;
plugins/claude-code/commands/skillopt-sleep.md
+5 -2
View File
@@ -60,8 +60,11 @@ what the optimizer writes, add `--preferences "<your house rules>"`.
the score, gate decision, and edits from stdout (or request `--json` when
machine-readable output is useful).
4. **For `run` that produced an accepted proposal:** inspect whether stdout says
it was auto-adopted. If not, tell the user nothing live changed and offer
`/skillopt-sleep adopt`; if it was, report the updated paths explicitly.
it was auto-adopted. If not, tell the user nothing live changed, run or cite
`status`, and offer the exact reviewed mode: `adopt --legacy`, repeatable
`adopt --skill NAME`, or `adopt --all-skills`. Never imply that bare adopt
means “adopt everything.” If it was auto-adopted, report the updated paths
and any still-pending fan-out names explicitly.
5. **For `adopt`:** confirm which live files were updated and that backups were
written under the staging dir's `backup/`.
6. **Never** edit `CLAUDE.md` or `SKILL.md` yourself — let the engine's explicit
plugins/codex/README.md
+2 -1
View File
@@ -52,7 +52,8 @@ Codex in natural language:
Use the skillopt-sleep skill to run status for this project.
Use the skillopt-sleep skill to run a dry-run for this project.
Use the skillopt-sleep skill to run the full cycle for this project with the Codex backend.
Use the skillopt-sleep skill to adopt the latest staged proposal.
Use the skillopt-sleep skill to inspect status, then adopt the reviewed managed
proposal or an explicit pending skill subset.
```
Or call the engine directly:
plugins/codex/skills/skillopt-sleep/SKILL.md
+5 -1
View File
@@ -62,9 +62,13 @@ bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" dry-run --project "$(pwd)" \
bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" run --project "$(pwd)" \
--source codex --target-skill-path "$TARGET_SKILL" --backend codex \
--max-sessions 5 --max-tasks 3 --progress
bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)"
bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" status --project "$(pwd)"
bash "$SKILLOPT_SLEEP_REPO/plugins/run-sleep.sh" adopt --project "$(pwd)" --legacy
```
For a fan-out night, select reviewed proposals with repeatable
`--skill NAME` or `--all-skills`; do not treat bare adopt as “adopt everything.”
On Windows (CMD / PowerShell):
```cmd
:: CMD
plugins/copilot/README.md
+24 -4
View File
@@ -46,10 +46,30 @@ propose?"*, *"adopt the staged sleep proposal"*. The server exposes seven MCP
tools: `sleep_status`, `sleep_dry_run`, `sleep_run`, `sleep_adopt`,
`sleep_harvest`, `sleep_schedule`, and `sleep_unschedule`.
Each tool takes optional `project`, `backend` (`mock`/`claude`/`codex`/`copilot`), and
`scope` arguments. Default backend is `mock` (no API spend). The `copilot`
backend drives the GitHub Copilot CLI (`copilot -p ... --output-format json`)
and requires the `copilot` CLI to be installed and authenticated.
Each tool takes optional `project`, `backend`
(`mock`/`claude`/`codex`/`copilot`/`handoff`), and `scope` arguments. For
`sleep_adopt`, first inspect `sleep_status`, then use the adoption controls that
match the reviewed staging manifest:
- `staging` — exact staging directory to adopt instead of the latest night
- `skills` — array of skill names to adopt; each is passed as one repeated
`--skill` argument without shell interpolation
- `all_skills` — adopt every staged per-skill proposal
- `legacy` — adopt only the legacy managed `SKILL.md`/`CLAUDE.md` pair
Choose one selection mode (`skills`, `all_skills`, or `legacy`) and do not
combine them. A bare `sleep_adopt` remains compatible with legacy-only staging;
fan-out staging requires an explicit selection.
Tool results preserve the engine's `exit_code` in `structuredContent`.
Ordinary nonzero exits set `isError: true`; exit 3 is the expected
`handoff_pending` state and is not an MCP tool error. With `json: true`, text
content is the engine's parseable JSON stdout, while diagnostics remain
separate in `structuredContent`.
Default backend is `mock` (no API spend). The `copilot` backend drives the
GitHub Copilot CLI (`copilot -p ... --output-format json`) and requires the
`copilot` CLI to be installed and authenticated.
Harvesting is local and read-only, and the default `mock` backend makes no
provider calls. A real backend sends truncated transcript excerpts and derived
plugins/copilot/copilot-instructions.snippet.md
+14 -2
View File
@@ -18,19 +18,25 @@ my preferences", or "make the agent improve from past usage", use the MCP tools:
- `sleep_dry_run` — no-staging preview; a real backend still makes provider calls
- `sleep_run` — full cycle, stages a validation-gated proposal by default;
explicit `auto_adopt` may update live files
- `sleep_adopt` — apply the staged proposal (backs up an existing live file first)
- `sleep_adopt` — apply a reviewed legacy or per-skill staged proposal (backs
up an existing live file first)
- `sleep_harvest` — list mined recurring tasks
- `sleep_schedule` — install a nightly cron entry (set `hour`/`minute`)
- `sleep_unschedule` — remove the nightly cron entry
### Key parameters (pass as MCP tool arguments)
- `backend` — `mock` (default, no provider calls), `claude`, `codex`, or `copilot`
- `backend` — `mock` (default, no provider calls), `claude`, `codex`, `copilot`,
or `handoff` (write prompts for completion in fresh sessions)
- `source` — `claude`, `codex`, or `auto` (where to read transcripts)
- `target_skill_path` — explicit SKILL.md to evolve; use this for a skill that
the current agent actually loads
- `tasks_file` — reviewed TaskRecord JSON (skip harvest); real backends require
its metadata to contain `"reviewed": true`
- `staging` — for `sleep_adopt`, the exact reviewed staging directory
- `skills` — for `sleep_adopt`, an array of reviewed skill names to adopt
- `all_skills` — for `sleep_adopt`, adopt every staged per-skill proposal
- `legacy` — for `sleep_adopt`, adopt only the legacy managed pair
- `max_tasks` / `max_sessions` — cap workload
- `auto_adopt` — auto-adopt if the gate passes
- `json` — machine-readable output for programmatic use
@@ -49,6 +55,12 @@ edits before suggesting `sleep_adopt`. Never hand-edit the user's memory/skill
files; use `sleep_adopt` (or an explicitly requested `auto_adopt`) so the engine
applies its staging manifest and backup behavior.
Before calling `sleep_adopt`, inspect `sleep_status` and the staging manifest,
then pass `staging` plus exactly one selection mode: `skills`, `all_skills`, or
`legacy`. Do not combine selection modes. A bare call is only for legacy-only
staging compatibility; fan-out staging requires an explicit selection. Pass
skill names as MCP array values, never as an invented shell command.
Harvesting is local and read-only, and `backend: "mock"` makes no provider
calls. A real backend sends truncated transcript excerpts and derived tasks to
the selected provider; outbound prompts are not guaranteed to be secret-free.
plugins/copilot/mcp_server.py
+230 -22
View File
@@ -9,7 +9,7 @@ Tools exposed:
- sleep_status : how many nights have run + the latest staged proposal
- sleep_dry_run : harvest+mine+replay, report only (no staging)
- sleep_run : full cycle, stages a proposal (nothing live changes)
- sleep_adopt : apply the latest staged proposal (with backup)
- sleep_adopt : apply a reviewed legacy or per-skill proposal (with backup)
- sleep_harvest : debug — list mined recurring tasks
Each tool shells out to `python -m skillopt_sleep <action> ...` and returns its
@@ -21,6 +21,7 @@ import json
import os
import subprocess
import sys
from typing import NamedTuple
REPO_ROOT = os.environ.get("SKILLOPT_SLEEP_REPO") or os.path.abspath(
os.path.join(os.path.dirname(__file__), "..", "..")
@@ -35,7 +36,7 @@ TOOLS = [
{"name": "sleep_run", "action": "run",
"description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."},
{"name": "sleep_adopt", "action": "adopt",
"description": "Apply the latest staged proposal to CLAUDE.md/SKILL.md (backs up first)."},
"description": "Apply a reviewed legacy or per-skill staged proposal (backs up first)."},
{"name": "sleep_harvest", "action": "harvest",
"description": "Debug: list the recurring tasks mined from recent sessions."},
{"name": "sleep_schedule", "action": "schedule",
@@ -50,8 +51,11 @@ _TOOL_SCHEMA = {
"properties": {
"project": {"type": "string",
"description": "Project dir to evolve (default: cwd)."},
"backend": {"type": "string", "enum": ["mock", "claude", "codex", "copilot"],
"description": "mock = no API spend (default); claude/codex/copilot = real."},
"backend": {
"type": "string",
"enum": ["mock", "claude", "codex", "copilot", "handoff"],
"description": "mock = local/default; claude/codex/copilot = real; handoff = no API subprocess.",
},
"scope": {"type": "string", "enum": ["invoked", "all"],
"description": "Harvest scope (default: invoked project only)."},
"source": {"type": "string", "enum": ["claude", "codex", "auto"],
@@ -62,30 +66,155 @@ _TOOL_SCHEMA = {
"description": "Path to reviewed TaskRecord JSON (skips harvest)."},
"target_skill_path": {"type": "string",
"description": "Explicit SKILL.md path to evolve/stage/adopt."},
"staging": {
"type": "string",
"description": "For sleep_adopt, use this exact staging directory instead of the latest night.",
},
"skills": {
"type": "array",
"items": {"type": "string", "minLength": 1},
"uniqueItems": True,
"description": "For sleep_adopt, adopt only these staged per-skill proposals.",
},
"all_skills": {
"type": "boolean",
"description": "For sleep_adopt, adopt every staged per-skill proposal.",
},
"legacy": {
"type": "boolean",
"description": "For sleep_adopt, adopt only the legacy managed SKILL.md/CLAUDE.md pair.",
},
"progress": {"type": "boolean",
"description": "Print phase progress to stderr."},
"max_sessions": {"type": "integer",
"max_sessions": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Cap harvested sessions per run."},
"max_tasks": {"type": "integer",
"description": "Cap mined tasks per run."},
"lookback_hours": {"type": "integer",
"max_tasks": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Cap mined tasks per run."},
"lookback_hours": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Harvest window in hours (default: 72)."},
"auto_adopt": {"type": "boolean",
"description": "Auto-adopt if gate passes (default: false)."},
"json": {"type": "boolean",
"description": "Return machine-readable JSON output."},
"edit_budget": {"type": "integer",
"edit_budget": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Max bounded edits per night (default: 4)."},
"hour": {"type": "integer",
"hour": {"type": "integer", "minimum": 0, "maximum": 23,
"description": "Hour for schedule (0-23, default: 3)."},
"minute": {"type": "integer",
"minute": {"type": "integer", "minimum": 0, "maximum": 59,
"description": "Minute for schedule (0-59, default: 17)."},
},
"additionalProperties": False,
}
_STRING_ARGS = {
"project", "backend", "scope", "source", "model", "tasks_file",
"target_skill_path", "staging",
}
_BOOLEAN_ARGS = {"all_skills", "legacy", "progress", "auto_adopt", "json"}
_INTEGER_BOUNDS = {
"max_sessions": (0, 1_000_000),
"max_tasks": (0, 1_000_000),
"lookback_hours": (0, 1_000_000),
"edit_budget": (0, 1_000_000),
"hour": (0, 23),
"minute": (0, 59),
}
_ADOPT_ONLY_ARGS = {"staging", "skills", "all_skills", "legacy"}
_SCHEDULE_ONLY_ARGS = {"hour", "minute"}
def _run_engine(action: str, args: dict) -> str:
class EngineResult(NamedTuple):
"""One engine invocation, including status hidden by the old text-only API."""
text: str
returncode: int
diagnostics: str = ""
def _validate_text(key: str, value: object) -> None:
if type(value) is not str:
raise ValueError(f"{key} must be a string")
if any(ord(ch) < 32 or ord(ch) == 127 for ch in value):
raise ValueError(f"{key} must not contain control characters")
def _validate_tool_arguments(action: str, args: object) -> dict:
"""Validate MCP input at runtime; clients are not trusted to enforce schema."""
if action not in {tool["action"] for tool in TOOLS}:
raise ValueError(f"unknown action: {action}")
if type(args) is not dict:
raise ValueError("arguments must be an object")
unknown = sorted(set(args) - set(_TOOL_SCHEMA["properties"]))
if unknown:
raise ValueError(f"unknown argument(s): {', '.join(unknown)}")
if action != "adopt" and set(args) & _ADOPT_ONLY_ARGS:
raise ValueError("staging/skills/all_skills/legacy are valid only for sleep_adopt")
if action != "schedule" and set(args) & _SCHEDULE_ONLY_ARGS:
raise ValueError("hour/minute are valid only for sleep_schedule")
for key in _STRING_ARGS & set(args):
_validate_text(key, args[key])
for key in _BOOLEAN_ARGS & set(args):
if type(args[key]) is not bool:
raise ValueError(f"{key} must be a boolean")
for key, (minimum, maximum) in _INTEGER_BOUNDS.items():
if key not in args:
continue
value = args[key]
if type(value) is not int:
raise ValueError(f"{key} must be an integer")
if not minimum <= value <= maximum:
raise ValueError(f"{key} must be between {minimum} and {maximum}")
for key in ("backend", "scope", "source"):
if key in args and args[key] not in _TOOL_SCHEMA["properties"][key]["enum"]:
raise ValueError(f"unsupported {key}: {args[key]!r}")
skills = args.get("skills", [])
if type(skills) is not list:
raise ValueError("skills must be an array of strings")
normalized = []
for skill in skills:
_validate_text("every skills entry", skill)
name = skill.strip()
if not name:
raise ValueError("every skills entry must be non-empty")
normalized.append(name)
if len(set(normalized)) != len(normalized):
raise ValueError("skills entries must be unique")
modes = sum((bool(normalized), args.get("all_skills") is True, args.get("legacy") is True))
if modes > 1:
raise ValueError("choose at most one of skills, all_skills, or legacy")
validated = dict(args)
if "skills" in validated:
validated["skills"] = normalized
return validated
def _append_adopt_args(cmd: list[str], args: dict) -> None:
"""Append selection flags as argv tokens; never interpolate skill names."""
staging = args.get("staging")
if staging:
cmd += ["--staging", str(staging)]
skills = args.get("skills") or []
for skill in skills:
# argparse treats a following value beginning with '-' as another
# option. The --flag=value form keeps such a skill name as data. All
# other names stay separate argv tokens; no shell parses either form.
if skill.startswith("-"):
cmd.append(f"--skill={skill}")
else:
cmd += ["--skill", skill]
if args.get("all_skills"):
cmd.append("--all-skills")
if args.get("legacy"):
cmd.append("--legacy")
def _run_engine(action: str, args: object) -> EngineResult:
args = _validate_tool_arguments(action, args)
py = sys.executable or "python3"
cmd = [py, "-m", "skillopt_sleep", action]
# String-valued flags
@@ -114,13 +243,19 @@ def _run_engine(action: str, args: dict) -> str:
]:
if args.get(key):
cmd.append(flag)
if action == "adopt":
_append_adopt_args(cmd, args)
try:
proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True, text=True, timeout=3600)
except Exception as e:
return f"[error] failed to run engine: {e}"
return EngineResult(f"[error] failed to run engine: {e}", 1)
out = (proc.stdout or "").strip()
err = (proc.stderr or "").strip()
return out + (("\n[stderr]\n" + err) if err else "")
if args.get("json"):
text = out if out or proc.returncode in {0, 3} else err
return EngineResult(text, proc.returncode, err)
text = out + (("\n[stderr]\n" + err) if err else "")
return EngineResult(text, proc.returncode)
def _result(id_, result):
@@ -131,9 +266,60 @@ def _error(id_, code, message):
return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
def handle(req: dict):
def _validate_request(req: object) -> tuple[str, object, dict]:
if type(req) is not dict:
raise ValueError("request must be a JSON object")
unknown = sorted(set(req) - {"jsonrpc", "id", "method", "params"})
if unknown:
raise ValueError(f"unknown request member(s): {', '.join(unknown)}")
if req.get("jsonrpc") != "2.0":
raise ValueError("jsonrpc must be '2.0'")
method = req.get("method")
id_ = req.get("id")
if type(method) is not str or not method:
raise ValueError("method must be a non-empty string")
params = req.get("params", {})
if type(params) is not dict:
raise ValueError("params must be an object")
request_id = req.get("id")
if "id" in req and request_id is not None and type(request_id) not in {str, int}:
raise ValueError("id must be a string, integer, or null")
return method, request_id, params
def _validate_method_params(method: str, params: dict) -> None:
allowed_by_method = {
"initialize": {"protocolVersion", "capabilities", "clientInfo", "_meta"},
"notifications/initialized": {"_meta"},
"initialized": {"_meta"},
"tools/list": {"cursor", "_meta"},
"tools/call": {"name", "arguments", "_meta"},
"ping": {"_meta"},
}
allowed = allowed_by_method.get(method)
if allowed is None:
return
unknown = sorted(set(params) - allowed)
if unknown:
raise ValueError(f"unknown params member(s): {', '.join(unknown)}")
for key in ("capabilities", "clientInfo", "_meta"):
if key in params and type(params[key]) is not dict:
raise ValueError(f"{key} must be an object")
for key in ("protocolVersion", "cursor"):
if key in params and type(params[key]) is not str:
raise ValueError(f"{key} must be a string")
def handle(req: object):
try:
method, id_, params = _validate_request(req)
except ValueError as exc:
candidate = req.get("id") if type(req) is dict else None
request_id = candidate if candidate is None or type(candidate) in {str, int} else None
return _error(request_id, -32600, f"invalid request: {exc}")
try:
_validate_method_params(method, params)
except ValueError as exc:
return _error(id_, -32602, f"invalid params: {exc}")
if method == "initialize":
return _result(id_, {
"protocolVersion": PROTOCOL_VERSION,
@@ -148,13 +334,34 @@ def handle(req: dict):
for t in TOOLS
]})
if method == "tools/call":
params = req.get("params") or {}
name = params.get("name")
if type(name) is not str:
return _error(id_, -32602, "tool name must be a string")
tool = _BY_NAME.get(name)
if not tool:
return _error(id_, -32602, f"unknown tool: {name}")
text = _run_engine(tool["action"], params.get("arguments") or {})
return _result(id_, {"content": [{"type": "text", "text": text}]})
arguments = params.get("arguments", {})
try:
run = _run_engine(tool["action"], arguments)
except ValueError as exc:
return _error(id_, -32602, f"invalid {name} arguments: {exc}")
status = "handoff_pending" if run.returncode == 3 else (
"ok" if run.returncode == 0 else "error"
)
structured = {"status": status, "exit_code": run.returncode}
if run.diagnostics:
structured["diagnostics"] = run.diagnostics
if type(arguments) is dict and arguments.get("json") is True and run.text:
try:
structured["output"] = json.loads(run.text)
except json.JSONDecodeError:
pass
result = {
"content": [{"type": "text", "text": run.text}],
"structuredContent": structured,
"isError": run.returncode not in {0, 3},
}
return _result(id_, result)
if method == "ping":
return _result(id_, {})
return _error(id_, -32601, f"method not found: {method}")
@@ -167,9 +374,10 @@ def main() -> int:
continue
try:
req = json.loads(line)
except Exception:
continue
resp = handle(req)
except json.JSONDecodeError:
resp = _error(None, -32700, "parse error")
else:
resp = handle(req)
if resp is not None:
sys.stdout.write(json.dumps(resp) + "\n")
sys.stdout.flush()
plugins/cursor/README.md
+6 -2
View File
@@ -71,9 +71,12 @@ Run the native command, for example:
/skillopt-sleep status
/skillopt-sleep dry-run --backend mock --max-sessions 5 --max-tasks 3
/skillopt-sleep run --backend cursor --max-sessions 5 --max-tasks 3 --progress
/skillopt-sleep adopt
/skillopt-sleep adopt --legacy
```
For fan-out proposals, inspect `status` and use repeatable `--skill NAME` or
`--all-skills`. Bare adopt deliberately refuses a night containing fan-out rows.
The `skillopt-sleep` agent skill remains independently available if a Cursor
version does not surface plugin commands.
@@ -124,7 +127,8 @@ skillopt-sleep dry-run --project "$(pwd)" --source cursor --backend mock \
skillopt-sleep run --project "$(pwd)" --source cursor --backend cursor \
--target-skill-path .cursor/skills/skillopt-sleep-learned/SKILL.md \
--max-sessions 5 --max-tasks 3 --progress
skillopt-sleep adopt --project "$(pwd)"
skillopt-sleep status --project "$(pwd)"
skillopt-sleep adopt --project "$(pwd)" --legacy
```
`--source cursor` reads local JSONL transcripts below
plugins/cursor/skills/skillopt-sleep/SKILL.md
+6 -2
View File
@@ -81,10 +81,14 @@ skillopt-sleep run --project "$(pwd)" --source cursor --backend cursor \
--target-skill-path "$TARGET_SKILL" \
--max-sessions 5 --max-tasks 3 --progress
# Apply the latest accepted staged proposal after review.
skillopt-sleep adopt --project "$(pwd)"
# Inspect selections, then apply the reviewed managed proposal.
skillopt-sleep status --project "$(pwd)"
skillopt-sleep adopt --project "$(pwd)" --legacy
```
For fan-out proposals, use repeatable `--skill NAME` or `--all-skills` after
review. Bare adopt deliberately refuses a night containing fan-out rows.
Actions are `status`, `harvest`, `dry-run`, `run`, `adopt`, `schedule`, and
`unschedule`.
plugins/devin/README.md
+24 -2
View File
@@ -30,7 +30,8 @@ source into the Claude Code-compatible JSONL the engine reads.
| Skill files | `.devin/skills/*/SKILL.md` |
Workspaces are auto-detected from `~/.config/Devin/User/workspaceStorage/*/workspace.json`.
After `sleep_adopt`, the evolved skill is synced to `.devin/skills/skillopt-sleep-learned/SKILL.md`.
The adapter performs no post-adoption copy. The core engine applies a reviewed
proposal directly to its selected target and owns backup/rollback behavior.
## Install
@@ -68,11 +69,32 @@ Requires Python ≥ 3.10. No third-party packages — the server is pure stdlib.
| `sleep_status` | nights run so far + latest staged proposal |
| `sleep_dry_run` | preview cycle — no staging; a real backend still makes provider calls |
| `sleep_run` | full cycle; stages a proposal for review |
| `sleep_adopt` | apply the staged proposal; syncs skill to the workspace |
| `sleep_adopt` | apply a reviewed legacy or per-skill proposal (with backup) |
| `sleep_harvest` | debug: list the recurring tasks mined |
| `sleep_schedule` | install a nightly cron entry (`--hour` / `--minute`) |
| `sleep_unschedule` | remove the nightly cron entry |
Before `sleep_adopt`, inspect `sleep_status` and use the controls that match the
reviewed staging manifest:
- `staging` — exact staging directory to adopt instead of the latest night
- `skills` — array of skill names to adopt; each is forwarded as one repeated
`--skill` argument without shell interpolation
- `all_skills` — adopt every staged per-skill proposal
- `legacy` — adopt only the legacy managed `SKILL.md`/`CLAUDE.md` pair
Choose one selection mode (`skills`, `all_skills`, or `legacy`) and do not
combine them. A bare call remains compatible with legacy-only staging; fan-out
staging requires an explicit selection. To operate on a specific Devin skill,
pass its `SKILL.md` as `target_skill_path`; the adapter never performs a second
copy after the core engine returns.
Tool results preserve the engine's `exit_code` in `structuredContent`.
Ordinary nonzero exits set `isError: true`; exit 3 is the expected
`handoff_pending` state and is not an MCP tool error. With `json: true`, text
content is the engine's parseable JSON stdout, while harvest and engine
diagnostics remain separate in `structuredContent`.
Default backend is `mock` (no API spend); the `claude`, `codex`, and `copilot`
backends use the corresponding authenticated CLI and budget. The `handoff`
backend runs the cycle with no model subprocess or API key — the engine writes
plugins/devin/devin-rules.snippet.md
+14 -3
View File
@@ -9,9 +9,8 @@ server. Use these tools to improve your long-term skills over time:
without engine staging/adoption; a real backend still makes provider calls
- **`sleep_run`** — run a full cycle; stages a proposal by default, while an
explicit `auto_adopt` may also update live files
- **`sleep_adopt`** — apply the staged proposal, then sync the managed skill to
`.devin/skills/skillopt-sleep-learned/SKILL.md` when `project` is the Devin
workspace and that workspace already contains a `.devin/` directory
- **`sleep_adopt`** — apply a reviewed legacy or per-skill staged proposal;
the core engine applies the selected target and creates its backup
- **`sleep_harvest`** — debug: list the recurring tasks mined from recent sessions
- **`sleep_schedule`** / **`sleep_unschedule`** — low-level shared-engine cron
controls; the current scheduled command does not run Devin's conversion step,
@@ -38,4 +37,16 @@ before selecting a real backend.
For a reviewed task file, pass `tasks_file`; before using it with a real backend,
inspect/redact it and ensure its metadata contains `"reviewed": true`.
Before `sleep_adopt`, inspect `sleep_status` and the staging manifest. Pass
`staging` for the exact reviewed staging directory, plus exactly one selection
mode: `skills` (an array of reviewed skill names), `all_skills` (every staged
per-skill proposal), or `legacy` (the managed `SKILL.md`/`CLAUDE.md` pair). Do
not combine selection modes. A bare call is only for legacy-only staging
compatibility; fan-out staging requires explicit selection. Pass names as MCP
array values, not as an invented shell command.
The adapter performs no post-adoption copy. To operate on a specific Devin
skill, pass its `SKILL.md` as `target_skill_path`; the core engine is solely
responsible for applying the reviewed proposal and maintaining its backup.
Place this file at `.devin/rules/skillopt-sleep.md` in your workspace.
plugins/devin/mcp_server.py
+232 -42
View File
@@ -10,8 +10,8 @@ Before each data-reading action this server runs `harvest_devin.py` to convert
locally available Devin data (ATIF-v1.7 transcripts, agentmemory memories, and
.devin skill files) into the Claude Code-compatible JSONL the engine consumes,
writing it under SKILLOPT_DEVIN_CLAUDE_HOME and pointing the engine there with
`--claude-home`. After `sleep_adopt` the evolved skill is synced back into the
workspace's `.devin/skills/`.
`--claude-home`. Adoption is performed only by the core engine against the
requested target; this adapter never performs a second copy or sync afterward.
Tools: sleep_status, sleep_dry_run, sleep_run, sleep_adopt, sleep_harvest,
sleep_schedule, sleep_unschedule. Each shells out to
@@ -22,9 +22,9 @@ from __future__ import annotations
import json
import os
import shutil
import subprocess
import sys
from typing import NamedTuple
# expanduser wraps the whole value so a "~/..." env var is expanded too (not
# just a default) — otherwise a literal ~ dir gets created.
@@ -36,7 +36,6 @@ PLUGIN_DIR = os.path.dirname(os.path.abspath(__file__))
CLAUDE_HOME = os.path.expanduser(
os.environ.get("SKILLOPT_DEVIN_CLAUDE_HOME", "~/.skillopt-sleep-devin")
)
MANAGED_SKILL_NAME = os.environ.get("SKILLOPT_MANAGED_SKILL", "skillopt-sleep-learned")
PROTOCOL_VERSION = "2024-11-05"
TOOLS = [
@@ -47,7 +46,7 @@ TOOLS = [
{"name": "sleep_run", "action": "run",
"description": "Run a full sleep cycle; stages a reviewed proposal. Nothing live changes until adopt."},
{"name": "sleep_adopt", "action": "adopt",
"description": "Apply the latest staged proposal to the managed SKILL.md and sync it into .devin/skills/."},
"description": "Apply a reviewed legacy or per-skill staged proposal (backs up first)."},
{"name": "sleep_harvest", "action": "harvest",
"description": "Debug: list the recurring tasks mined from recent Devin sessions."},
{"name": "sleep_schedule", "action": "schedule",
@@ -74,23 +73,41 @@ _TOOL_SCHEMA = {
"description": "Path to reviewed TaskRecord JSON (skips harvest)."},
"target_skill_path": {"type": "string",
"description": "Explicit SKILL.md path to evolve/stage/adopt."},
"staging": {
"type": "string",
"description": "For sleep_adopt, use this exact staging directory instead of the latest night.",
},
"skills": {
"type": "array",
"items": {"type": "string", "minLength": 1},
"uniqueItems": True,
"description": "For sleep_adopt, adopt only these staged per-skill proposals.",
},
"all_skills": {
"type": "boolean",
"description": "For sleep_adopt, adopt every staged per-skill proposal.",
},
"legacy": {
"type": "boolean",
"description": "For sleep_adopt, adopt only the legacy managed SKILL.md/CLAUDE.md pair.",
},
"progress": {"type": "boolean",
"description": "Print phase progress to stderr."},
"max_sessions": {"type": "integer",
"max_sessions": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Cap harvested sessions per run."},
"max_tasks": {"type": "integer",
"max_tasks": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Cap mined tasks per run."},
"lookback_hours": {"type": "integer",
"lookback_hours": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Harvest window in hours (default: 72)."},
"auto_adopt": {"type": "boolean",
"description": "Auto-adopt if gate passes (default: false)."},
"json": {"type": "boolean",
"description": "Return machine-readable JSON output."},
"edit_budget": {"type": "integer",
"edit_budget": {"type": "integer", "minimum": 0, "maximum": 1_000_000,
"description": "Max bounded edits per night (default: 4)."},
"hour": {"type": "integer",
"hour": {"type": "integer", "minimum": 0, "maximum": 23,
"description": "Hour for schedule (0-23, default: 3)."},
"minute": {"type": "integer",
"minute": {"type": "integer", "minimum": 0, "maximum": 59,
"description": "Minute for schedule (0-59, default: 17)."},
},
"additionalProperties": False,
@@ -99,8 +116,93 @@ _TOOL_SCHEMA = {
# actions that read harvested Devin data (schedule/unschedule/adopt don't)
_HARVEST_ACTIONS = {"status", "dry-run", "run", "harvest"}
_STRING_ARGS = {
"project", "backend", "scope", "source", "model", "tasks_file",
"target_skill_path", "staging",
}
_BOOLEAN_ARGS = {"all_skills", "legacy", "progress", "auto_adopt", "json"}
_INTEGER_BOUNDS = {
"max_sessions": (0, 1_000_000),
"max_tasks": (0, 1_000_000),
"lookback_hours": (0, 1_000_000),
"edit_budget": (0, 1_000_000),
"hour": (0, 23),
"minute": (0, 59),
}
_ADOPT_ONLY_ARGS = {"staging", "skills", "all_skills", "legacy"}
_SCHEDULE_ONLY_ARGS = {"hour", "minute"}
def _run_harvest() -> str:
class EngineResult(NamedTuple):
"""One engine invocation, including status hidden by the old text-only API."""
text: str
returncode: int
diagnostics: str = ""
def _validate_text(key: str, value: object) -> None:
if type(value) is not str:
raise ValueError(f"{key} must be a string")
if any(ord(ch) < 32 or ord(ch) == 127 for ch in value):
raise ValueError(f"{key} must not contain control characters")
def _validate_tool_arguments(action: str, args: object) -> dict:
"""Validate MCP input at runtime; clients are not trusted to enforce schema."""
if action not in {tool["action"] for tool in TOOLS}:
raise ValueError(f"unknown action: {action}")
if type(args) is not dict:
raise ValueError("arguments must be an object")
unknown = sorted(set(args) - set(_TOOL_SCHEMA["properties"]))
if unknown:
raise ValueError(f"unknown argument(s): {', '.join(unknown)}")
if action != "adopt" and set(args) & _ADOPT_ONLY_ARGS:
raise ValueError("staging/skills/all_skills/legacy are valid only for sleep_adopt")
if action != "schedule" and set(args) & _SCHEDULE_ONLY_ARGS:
raise ValueError("hour/minute are valid only for sleep_schedule")
for key in _STRING_ARGS & set(args):
_validate_text(key, args[key])
for key in _BOOLEAN_ARGS & set(args):
if type(args[key]) is not bool:
raise ValueError(f"{key} must be a boolean")
for key, (minimum, maximum) in _INTEGER_BOUNDS.items():
if key not in args:
continue
value = args[key]
if type(value) is not int:
raise ValueError(f"{key} must be an integer")
if not minimum <= value <= maximum:
raise ValueError(f"{key} must be between {minimum} and {maximum}")
for key in ("backend", "scope", "source"):
if key in args and args[key] not in _TOOL_SCHEMA["properties"][key]["enum"]:
raise ValueError(f"unsupported {key}: {args[key]!r}")
skills = args.get("skills", [])
if type(skills) is not list:
raise ValueError("skills must be an array of strings")
normalized = []
for skill in skills:
_validate_text("every skills entry", skill)
name = skill.strip()
if not name:
raise ValueError("every skills entry must be non-empty")
normalized.append(name)
if len(set(normalized)) != len(normalized):
raise ValueError("skills entries must be unique")
modes = sum((bool(normalized), args.get("all_skills") is True, args.get("legacy") is True))
if modes > 1:
raise ValueError("choose at most one of skills, all_skills, or legacy")
validated = dict(args)
if "skills" in validated:
validated["skills"] = normalized
return validated
def _run_harvest() -> EngineResult:
"""Convert local Devin data into the JSONL the engine reads, under CLAUDE_HOME."""
harvester = os.path.join(PLUGIN_DIR, "harvest_devin.py")
env = dict(os.environ)
@@ -112,28 +214,37 @@ def _run_harvest() -> str:
)
out = (proc.stdout or "").strip()
err = (proc.stderr or "").strip()
return out + (("\n[harvest stderr]\n" + err) if err else "")
return EngineResult(out, proc.returncode, err)
except Exception as exc:
return f"[harvest_devin] warning: {exc}"
return EngineResult(f"[error] failed to run Devin harvest: {exc}", 1)
def _sync_skill(project: str) -> str:
"""After adopt, copy the evolved skill into the workspace's .devin/skills/."""
src = os.path.join(CLAUDE_HOME, "skills", MANAGED_SKILL_NAME, "SKILL.md")
if not (os.path.isfile(src) and project and os.path.isdir(project)):
return ""
dot_root = os.path.join(project, ".devin")
if not os.path.isdir(dot_root):
return ""
dst_dir = os.path.join(dot_root, "skills", MANAGED_SKILL_NAME)
os.makedirs(dst_dir, exist_ok=True)
dst = os.path.join(dst_dir, "SKILL.md")
shutil.copy2(src, dst)
return f"\n[sleep] synced evolved skill → {dst}"
def _append_adopt_args(cmd: list[str], args: dict) -> None:
"""Append selection flags as argv tokens; never interpolate skill names."""
staging = args.get("staging")
if staging:
cmd += ["--staging", str(staging)]
skills = args.get("skills") or []
for skill in skills:
if skill.startswith("-"):
cmd.append(f"--skill={skill}")
else:
cmd += ["--skill", skill]
if args.get("all_skills"):
cmd.append("--all-skills")
if args.get("legacy"):
cmd.append("--legacy")
def _run_engine(action: str, args: dict) -> str:
harvest_out = _run_harvest() if action in _HARVEST_ACTIONS else ""
def _run_engine(action: str, args: object) -> EngineResult:
args = _validate_tool_arguments(action, args)
harvest = _run_harvest() if action in _HARVEST_ACTIONS else EngineResult("", 0)
if harvest.returncode != 0:
text = harvest.text
if harvest.diagnostics:
text += ("\n[harvest stderr]\n" if text else "") + harvest.diagnostics
return EngineResult(text, harvest.returncode)
py = sys.executable or "python3"
cmd = [py, "-m", "skillopt_sleep", action, "--claude-home", CLAUDE_HOME]
@@ -165,6 +276,8 @@ def _run_engine(action: str, args: dict) -> str:
]:
if args.get(key):
cmd.append(flag)
if action == "adopt":
_append_adopt_args(cmd, args)
env = dict(os.environ)
env["PYTHONPATH"] = REPO_ROOT + os.pathsep + env.get("PYTHONPATH", "")
@@ -172,15 +285,19 @@ def _run_engine(action: str, args: dict) -> str:
proc = subprocess.run(cmd, cwd=REPO_ROOT, capture_output=True,
text=True, timeout=3600, env=env)
except Exception as e:
return f"[harvest]\n{harvest_out}\n[error] failed to run engine: {e}"
return EngineResult(f"[error] failed to run engine: {e}", 1, harvest.text)
out = (proc.stdout or "").strip()
err = (proc.stderr or "").strip()
result = (f"[harvest]\n{harvest_out}\n\n" if harvest_out else "") + f"[engine]\n{out}"
diagnostics = "\n".join(part for part in (harvest.text, harvest.diagnostics, err) if part)
if args.get("json"):
text = out if out or proc.returncode in {0, 3} else err
return EngineResult(text, proc.returncode, diagnostics)
result = (f"[harvest]\n{harvest.text}\n\n" if harvest.text else "") + f"[engine]\n{out}"
if harvest.diagnostics:
result += f"\n[harvest stderr]\n{harvest.diagnostics}"
if err:
result += f"\n[stderr]\n{err}"
if action == "adopt":
result += _sync_skill(args.get("project") or os.getcwd())
return result
return EngineResult(result, proc.returncode)
def _result(id_, result):
@@ -191,9 +308,60 @@ def _error(id_, code, message):
return {"jsonrpc": "2.0", "id": id_, "error": {"code": code, "message": message}}
def handle(req: dict):
def _validate_request(req: object) -> tuple[str, object, dict]:
if type(req) is not dict:
raise ValueError("request must be a JSON object")
unknown = sorted(set(req) - {"jsonrpc", "id", "method", "params"})
if unknown:
raise ValueError(f"unknown request member(s): {', '.join(unknown)}")
if req.get("jsonrpc") != "2.0":
raise ValueError("jsonrpc must be '2.0'")
method = req.get("method")
id_ = req.get("id")
if type(method) is not str or not method:
raise ValueError("method must be a non-empty string")
params = req.get("params", {})
if type(params) is not dict:
raise ValueError("params must be an object")
request_id = req.get("id")
if "id" in req and request_id is not None and type(request_id) not in {str, int}:
raise ValueError("id must be a string, integer, or null")
return method, request_id, params
def _validate_method_params(method: str, params: dict) -> None:
allowed_by_method = {
"initialize": {"protocolVersion", "capabilities", "clientInfo", "_meta"},
"notifications/initialized": {"_meta"},
"initialized": {"_meta"},
"tools/list": {"cursor", "_meta"},
"tools/call": {"name", "arguments", "_meta"},
"ping": {"_meta"},
}
allowed = allowed_by_method.get(method)
if allowed is None:
return
unknown = sorted(set(params) - allowed)
if unknown:
raise ValueError(f"unknown params member(s): {', '.join(unknown)}")
for key in ("capabilities", "clientInfo", "_meta"):
if key in params and type(params[key]) is not dict:
raise ValueError(f"{key} must be an object")
for key in ("protocolVersion", "cursor"):
if key in params and type(params[key]) is not str:
raise ValueError(f"{key} must be a string")
def handle(req: object):
try:
method, id_, params = _validate_request(req)
except ValueError as exc:
candidate = req.get("id") if type(req) is dict else None
request_id = candidate if candidate is None or type(candidate) in {str, int} else None
return _error(request_id, -32600, f"invalid request: {exc}")
try:
_validate_method_params(method, params)
except ValueError as exc:
return _error(id_, -32602, f"invalid params: {exc}")
if method == "initialize":
return _result(id_, {
"protocolVersion": PROTOCOL_VERSION,
@@ -208,13 +376,34 @@ def handle(req: dict):
for t in TOOLS
]})
if method == "tools/call":
params = req.get("params") or {}
name = params.get("name")
if type(name) is not str:
return _error(id_, -32602, "tool name must be a string")
tool = _BY_NAME.get(name)
if not tool:
return _error(id_, -32602, f"unknown tool: {name}")
text = _run_engine(tool["action"], params.get("arguments") or {})
return _result(id_, {"content": [{"type": "text", "text": text}]})
arguments = params.get("arguments", {})
try:
run = _run_engine(tool["action"], arguments)
except ValueError as exc:
return _error(id_, -32602, f"invalid {name} arguments: {exc}")
status = "handoff_pending" if run.returncode == 3 else (
"ok" if run.returncode == 0 else "error"
)
structured = {"status": status, "exit_code": run.returncode}
if run.diagnostics:
structured["diagnostics"] = run.diagnostics
if type(arguments) is dict and arguments.get("json") is True and run.text:
try:
structured["output"] = json.loads(run.text)
except json.JSONDecodeError:
pass
result = {
"content": [{"type": "text", "text": run.text}],
"structuredContent": structured,
"isError": run.returncode not in {0, 3},
}
return _result(id_, result)
if method == "ping":
return _result(id_, {})
return _error(id_, -32601, f"method not found: {method}")
@@ -227,9 +416,10 @@ def main() -> int:
continue
try:
req = json.loads(line)
except Exception:
continue
resp = handle(req)
except json.JSONDecodeError:
resp = _error(None, -32700, "parse error")
else:
resp = handle(req)
if resp is not None:
sys.stdout.write(json.dumps(resp) + "\n")
sys.stdout.flush()
skillopt_sleep/__main__.py
+348 -32
View File
@@ -4,6 +4,7 @@
python -m skillopt_sleep dry-run # same but report only, no staging/adopt
python -m skillopt_sleep status # show state + latest staged proposal
python -m skillopt_sleep adopt # apply the latest staged proposal (with backup)
python -m skillopt_sleep adopt --skill NAME # adopt one staged skill (repeatable)
python -m skillopt_sleep harvest # just print what would be mined (debug)
Common flags:
@@ -33,11 +34,19 @@ from typing import Any, Dict
from skillopt_sleep.backend import CursorBackendError
from skillopt_sleep.config import load_config
from skillopt_sleep.cycle import run_sleep_cycle
from skillopt_sleep.cycle import _one_line_display_text, run_sleep_cycle
from skillopt_sleep.harvest_sources import harvest_for_config
from skillopt_sleep.mine import mine
from skillopt_sleep.staging import (
StagingError,
adopt_skills,
has_pending_staged_managed,
json_safe,
latest_staging,
pending_staged_skills,
staged_skills,
)
from skillopt_sleep.staging import adopt as adopt_staging
from skillopt_sleep.staging import json_safe, latest_staging
from skillopt_sleep.state import SleepState
from skillopt_sleep.tasks_file import load_tasks_file, make_tasks_payload, write_tasks_file
@@ -51,6 +60,15 @@ def _read_text(path: str) -> str:
def _report_payload(rep, outcome) -> Dict[str, Any]:
staged_names = []
if outcome.staging_dir:
try:
staged_names = [
row.get("skill_name", "")
for row in pending_staged_skills(outcome.staging_dir)
]
except Exception:
staged_names = []
return json_safe({
"night": rep.night,
"accepted": rep.accepted,
@@ -66,8 +84,12 @@ def _report_payload(rep, outcome) -> Dict[str, Any]:
"rejected_edits": [e.__dict__ for e in rep.rejected_edits],
"gate_no_regression": bool(getattr(rep, "gate_no_regression", False)),
"gate_trials": _redact_deep(getattr(rep, "gate_trials", [])),
"skill_groups": [
group.to_dict() for group in getattr(rep, "skill_groups", [])
],
"notes": rep.notes,
"staging_dir": outcome.staging_dir,
"staged_skills": staged_names,
"adopted": outcome.adopted,
})
@@ -110,6 +132,13 @@ def _add_common(p: argparse.ArgumentParser) -> None:
help="cap mined tasks for this run")
p.add_argument("--target-skill-path", default="",
help="explicit live SKILL.md path to evolve/stage/adopt")
p.add_argument(
"--skill-root",
dest="skill_roots",
action="append",
default=[],
help="additional root containing <skill-name>/SKILL.md (repeatable)",
)
p.add_argument("--tasks-file", default="",
help="reviewed TaskRecord JSON file to replay instead of harvesting")
p.add_argument("--progress", action="store_true",
@@ -184,6 +213,16 @@ def _cfg_from_args(args, task_meta: Dict[str, Any] | None = None) -> Any:
if args.project and not os.path.isabs(path):
path = os.path.join(os.path.abspath(args.project), path)
overrides["target_skill_path"] = os.path.abspath(path)
if getattr(args, "skill_roots", None):
project = os.path.abspath(args.project) if args.project else os.getcwd()
overrides["skill_roots"] = [
os.path.abspath(
os.path.join(project, os.path.expanduser(root))
if not os.path.isabs(os.path.expanduser(root))
else os.path.expanduser(root)
)
for root in args.skill_roots
]
if getattr(args, "progress", False):
overrides["progress"] = True
if getattr(args, "auto_adopt", False):
@@ -211,12 +250,21 @@ def cmd_run(args, dry: bool = False) -> int:
file=sys.stderr,
)
return 2
if cfg.get("backend", "mock") == "handoff":
return _run_handoff(cfg, args, seed_tasks=tasks, task_meta=task_meta, dry=dry)
try:
if cfg.get("backend", "mock") == "handoff":
return _run_handoff(
cfg,
args,
seed_tasks=tasks,
task_meta=task_meta,
dry=dry,
)
outcome = run_sleep_cycle(cfg, seed_tasks=tasks, dry_run=dry)
except CursorBackendError as exc:
print(f"[sleep] Cursor backend failed: {_redact_deep(str(exc))}", file=sys.stderr)
_print_run_failure(args, "backend_failed", exc)
return 1
except StagingError as exc:
_print_run_failure(args, "staging_refused", exc)
return 1
_print_run_report(outcome, args, task_meta)
return 0
@@ -235,17 +283,46 @@ def _print_run_report(outcome, args, task_meta: Dict[str, Any]) -> None:
print(f"[sleep] held-out {rep.baseline_score:.3f} -> {rep.candidate_score:.3f} "
f"=> {rep.gate_action} (accepted={rep.accepted})")
for e in rep.edits:
print(f" + [{e.target}/{e.op}] {e.content}")
print(
f" + [{_display_value(e.target)}/{_display_value(e.op)}] "
f"{_display_value(e.content)}"
)
if rep.rejected_edits:
print("[sleep] rejected by gate:")
for e in rep.rejected_edits:
print(f" - [{e.target}/{e.op}] {e.content}")
print(
f" - [{_display_value(e.target)}/{_display_value(e.op)}] "
f"{_display_value(e.content)}"
)
if outcome.staging_dir:
print(f"[sleep] staged: {outcome.staging_dir}")
if not outcome.adopted:
print(f"[sleep] staged: {_display_value(outcome.staging_dir)}")
names = []
try:
names = [
r["skill_name"]
for r in pending_staged_skills(outcome.staging_dir)
]
except Exception:
names = []
if names:
print("[sleep] review the pending per-skill proposals:")
print("[sleep] staged skills:")
for name in names:
print(f" - {name!r}")
# Names are safe path segments but may still contain spaces
# or shell metacharacters. Keep untrusted names out of a
# copy/paste command instead of pretending one quoting
# convention works in every supported shell.
print(" python -m skillopt_sleep adopt --skill NAME")
print(" (repeat --skill NAME to adopt more than one)")
print(" python -m skillopt_sleep adopt --all-skills")
if has_pending_staged_managed(outcome.staging_dir):
print(" python -m skillopt_sleep adopt --legacy")
elif not outcome.adopted:
print("[sleep] review it, then: python -m skillopt_sleep adopt")
if outcome.adopted:
print(f"[sleep] auto-adopted: {', '.join(outcome.adopted_paths)}")
adopted = ", ".join(_display_value(path) for path in outcome.adopted_paths)
print(f"[sleep] auto-adopted: {adopted}")
def _handoff_dir_for(cfg) -> str:
@@ -267,6 +344,29 @@ def _redact_deep(obj):
return obj
def _display_error(exc: object) -> str:
"""Render an exception without leaking secrets or terminal controls."""
return _display_value(exc)
def _display_value(value: object) -> str:
"""Render arbitrary untrusted text safely for a human terminal."""
return _one_line_display_text(_redact_deep(str(value)))
def _print_run_failure(args, kind: str, exc: object) -> None:
"""Keep run failures machine-readable under ``--json``."""
message = _display_error(exc)
if args.json:
print(json.dumps({
"ok": False,
"error": kind,
"message": message,
}, ensure_ascii=False, indent=2))
else:
print(f"[sleep] {kind.replace('_', ' ')}: {message}", file=sys.stderr)
def _flush_handoff(backend, args) -> int:
prompts_path = backend.flush_pending()
if args.json:
@@ -363,7 +463,10 @@ def _handoff_mine_and_pin(cfg, args, backend, snapshot: str, dry: bool):
# LLM mining needs answers before the task set can be pinned.
return _flush_handoff(backend, args), None
if not tasks:
print("[sleep] handoff: no tasks mined — nothing to consolidate")
print(
"[sleep] handoff: no tasks mined — nothing to consolidate",
file=sys.stderr if args.json else sys.stdout,
)
if not dry:
# Advance the harvest window like run_sleep_cycle's no-tasks
# branch, or every later run re-scans the same stale window.
@@ -381,7 +484,10 @@ def _handoff_mine_and_pin(cfg, args, backend, snapshot: str, dry: bool):
# with a real backend must still hit the human-review gate above. The
# driver itself loads it directly, with the same trust as in-cycle mining.
write_tasks_file(snapshot, _redact_deep(payload))
print(f"[sleep] handoff: pinned {len(tasks)} tasks -> {snapshot}")
print(
f"[sleep] handoff: pinned {len(tasks)} tasks -> {snapshot}",
file=sys.stderr if args.json else sys.stdout,
)
return 0, tasks
@@ -417,7 +523,6 @@ def _run_handoff(cfg, args, *, seed_tasks, task_meta: Dict[str, Any], dry: bool)
pass
if backend.pending:
return _flush_handoff(backend, args)
_print_run_report(outcome, args, task_meta)
# A completed real run ends the night: archive the handoff dir so the
# next night re-harvests instead of replaying the pinned snapshot.
if not dry and outcome.staging_dir and os.path.isdir(hdir):
@@ -425,8 +530,18 @@ def _run_handoff(cfg, args, *, seed_tasks, task_meta: Dict[str, Any], dry: bool)
done = f"{hdir}.night{outcome.report.night}.done"
if os.path.exists(done):
done = f"{done}.{int(time.time())}"
os.rename(hdir, done)
print(f"[sleep] handoff: archived round data -> {done}")
try:
os.rename(hdir, done)
except OSError as exc:
raise StagingError(
f"handoff completed but its round directory could not be archived; "
f"preserved at {hdir!r}: {type(exc).__name__}: {exc}"
) from exc
print(
f"[sleep] handoff: archived round data -> {done}",
file=sys.stderr if args.json else sys.stdout,
)
_print_run_report(outcome, args, task_meta)
return 0
@@ -435,6 +550,20 @@ def cmd_status(args) -> int:
state = SleepState.load(cfg.state_path)
project = cfg.get("invoked_project") or os.getcwd()
latest = latest_staging(project)
skills = []
all_skills = []
has_managed = False
staging_error = ""
if latest:
try:
all_skills = staged_skills(latest)
skills = pending_staged_skills(latest)
has_managed = has_pending_staged_managed(latest)
except Exception as exc:
staging_error = _display_error(exc)
all_skills = []
skills = []
has_managed = False
info = {
"night": state.night,
"state_path": cfg.state_path,
@@ -442,36 +571,211 @@ def cmd_status(args) -> int:
"history_tail": state.data.get("history", [])[-5:],
"latest_staging": latest,
"slow_memory_chars": len(state.slow_memory),
"staged_skills": [r.get("skill_name", "") for r in skills],
"adopted_skills": [
row.get("skill_name", "")
for row in all_skills
if row not in skills
],
"has_managed_proposal": has_managed,
}
if staging_error:
info["staging_error"] = staging_error
if args.json:
print(json.dumps(info, ensure_ascii=False, indent=2))
else:
print(f"[sleep] nights so far: {state.night}")
print(f"[sleep] project: {project}")
if latest:
print(f"[sleep] latest staged proposal: {latest}")
print(f"[sleep] latest staged proposal: {_display_value(latest)}")
if staging_error:
print(f"[sleep] cannot read latest staging manifest: {staging_error}")
elif skills:
print("[sleep] pending staged skills:")
for row in skills:
print(
f" {_display_value(row.get('skill_name', ''))!r} -> "
f"{_display_value(row.get('live_skill_path', ''))!r}"
)
adopted_names = [
row.get("skill_name", "")
for row in all_skills
if row not in skills
]
if adopted_names:
print("[sleep] already adopted from this night:")
for name in adopted_names:
print(f" {_display_value(name)!r}")
if has_managed:
print("[sleep] managed proposal available via --legacy")
rp = os.path.join(latest, "report.md")
if os.path.exists(rp):
with open(rp) as f:
print("\n" + f.read())
if (
not staging_error
and os.path.isfile(rp)
and not os.path.islink(rp)
):
with open(rp, encoding="utf-8") as f:
print(
"\n" + "\n".join(
_display_value(line) for line in f.read().splitlines()
)
)
else:
print("[sleep] no staged proposals yet.")
return 0
return 1 if staging_error else 0
def cmd_adopt(args) -> int:
cfg = _cfg_from_args(args)
project = cfg.get("invoked_project") or os.getcwd()
target = args.staging or latest_staging(project)
def fail(code: int, kind: str, message: str, **extra: Any) -> int:
safe_message = _display_value(message)
if args.json:
payload = {
"ok": False,
"error": kind,
"message": safe_message,
"staging_dir": _display_value(target or ""),
}
payload.update(_redact_deep(extra))
print(json.dumps(payload, ensure_ascii=False, indent=2))
else:
print(safe_message)
return code
if not target or not os.path.isdir(target):
print("[sleep] nothing to adopt (no staging dir).")
return 1
updated = adopt_staging(target)
print(f"[sleep] adopted from {target}")
for p in updated:
print(f" -> {p}")
if not updated:
print("[sleep] (proposal contained no accepted changes)")
return fail(1, "no_staging", "[sleep] nothing to adopt (no staging dir).")
raw_selected = list(getattr(args, "skills", None) or [])
if any(not str(name).strip() for name in raw_selected):
return fail(
2,
"invalid_selection",
"[sleep] --skill names must be non-empty.",
)
selected = [str(name).strip() for name in raw_selected]
adopt_all = bool(getattr(args, "all_skills", False))
adopt_legacy = bool(getattr(args, "legacy", False))
if sum((bool(selected), adopt_all, adopt_legacy)) > 1:
return fail(
2,
"invalid_selection",
"[sleep] use exactly one of --skill, --all-skills, or --legacy.",
)
try:
rows = staged_skills(target)
pending_rows = pending_staged_skills(target)
except Exception as exc:
return fail(
1,
"invalid_staging",
f"[sleep] cannot read staged skills: {exc}",
)
if adopt_legacy:
try:
updated = adopt_staging(target)
except StagingError as exc:
return fail(2, "adoption_refused", f"[sleep] adopt refused: {exc}")
except (OSError, ValueError, KeyError, json.JSONDecodeError) as exc:
return fail(1, "adoption_failed", f"[sleep] adopt failed: {exc}")
if args.json:
print(json.dumps({
"ok": True,
"staging_dir": target,
"mode": "legacy",
"adopted_skills": [],
"updated_paths": updated,
}, ensure_ascii=False, indent=2))
else:
print(f"[sleep] adopted managed proposal from {_display_value(target)}")
for path in updated:
print(f" -> {_display_value(path)}")
if not updated:
print("[sleep] (proposal contained no accepted managed changes)")
return 0
if selected or adopt_all:
if not rows:
return fail(
2,
"no_staged_skills",
"[sleep] this night has no per-skill proposals; omit --skill "
"to adopt the legacy pair.",
)
names = (
[str(row.get("skill_name", "")) for row in pending_rows]
if adopt_all
else selected
)
try:
receipts = adopt_skills(target, names)
except StagingError as exc:
return fail(2, "adoption_refused", f"[sleep] adopt refused: {exc}")
except OSError as exc:
return fail(1, "adoption_failed", f"[sleep] adopt failed: {exc}")
if args.json:
print(json.dumps(json_safe({
"ok": True,
"staging_dir": target,
"mode": "skills",
"adopted_skills": [receipt.__dict__ for receipt in receipts],
"updated_paths": [receipt.live_skill_path for receipt in receipts],
}), ensure_ascii=False, indent=2))
else:
print(f"[sleep] adopted from {_display_value(target)}")
for receipt in receipts:
print(
f" -> {_display_value(receipt.skill_name)}: "
f"{_display_value(receipt.live_skill_path)}"
)
if not receipts:
print("[sleep] (no skills in the selection)")
return 0
if rows:
message = (
"[sleep] this night staged per-skill proposals; pass --skill NAME "
"or --all-skills, or use --legacy for its managed proposal."
)
if args.json:
return fail(
2,
"selection_required",
message,
available_skills=[
{
"skill_name": row.get("skill_name", ""),
"live_skill_path": row.get("live_skill_path", ""),
}
for row in rows
],
)
print(_display_value(message))
for row in rows:
print(
f" {_display_value(row.get('skill_name', ''))!r} -> "
f"{_display_value(row.get('live_skill_path', ''))!r}"
)
return 2
try:
updated = adopt_staging(target)
except StagingError as exc:
return fail(2, "adoption_refused", f"[sleep] adopt refused: {exc}")
except (OSError, ValueError, KeyError, json.JSONDecodeError) as exc:
return fail(1, "adoption_failed", f"[sleep] adopt failed: {exc}")
if args.json:
print(json.dumps({
"ok": True,
"staging_dir": target,
"mode": "legacy",
"adopted_skills": [],
"updated_paths": updated,
}, ensure_ascii=False, indent=2))
else:
print(f"[sleep] adopted from {_display_value(target)}")
for path in updated:
print(f" -> {_display_value(path)}")
if not updated:
print("[sleep] (proposal contained no accepted changes)")
return 0
@@ -519,18 +823,18 @@ def cmd_harvest(args) -> int:
def cmd_schedule(args) -> int:
from skillopt_sleep.scheduler import schedule, list_scheduled
from skillopt_sleep.scheduler import list_scheduled, schedule
cfg = _cfg_from_args(args)
project = cfg.get("invoked_project") or os.getcwd()
ok, msg = schedule(project, backend=cfg.get("backend", "mock"),
hour=args.hour, minute=args.minute,
extra=("--auto-adopt" if getattr(args, "auto_adopt", False) else ""))
print("[sleep] " + msg)
print("[sleep] " + _display_value(msg))
cur = list_scheduled()
if cur:
print("[sleep] currently scheduled:")
for ln in cur:
print(" " + ln[:140])
print(" " + _display_value(ln[:140]))
return 0 if ok else 1
@@ -539,7 +843,7 @@ def cmd_unschedule(args) -> int:
cfg = _cfg_from_args(args)
project = cfg.get("invoked_project") or os.getcwd()
ok, msg = unschedule(project, all_projects=getattr(args, "all", False))
print("[sleep] " + msg)
print("[sleep] " + _display_value(msg))
return 0 if ok else 1
@@ -556,6 +860,18 @@ def main(argv=None) -> int:
p_adopt = sub.add_parser("adopt", help="apply latest staged proposal")
_add_common(p_adopt)
p_adopt.add_argument("--staging", default="", help="specific staging dir")
p_adopt.add_argument(
"--skill", action="append", default=[], dest="skills",
help="adopt this staged skill (repeatable)",
)
p_adopt.add_argument(
"--all-skills", action="store_true", dest="all_skills",
help="adopt every staged per-skill proposal",
)
p_adopt.add_argument(
"--legacy", action="store_true",
help="adopt only the staged managed skill/memory proposal",
)
p_harvest = sub.add_parser("harvest", help="debug: show mined tasks")
_add_common(p_harvest)
p_harvest.add_argument("--output", default="", help="write mined tasks JSON for review")
skillopt_sleep/config.py
+5 -1
View File
@@ -75,12 +75,16 @@ DEFAULTS: Dict[str, Any] = {
"evolve_skill": True, # consolidate the managed SKILL.md
"llm_mine": True, # use the backend to mine checkable tasks (real backends)
"target_skill_path": "", # explicit SKILL.md target for repo-scoped agents
"skill_roots": [], # extra explicit roots containing <name>/SKILL.md
"target_task_filter": True, # prefer mined tasks matching target_skill_path/text
"progress": False, # print phase progress to stderr
# ── observability ──────────────────────────────────────────────────────
"evidence_log": True, # write per-night evidence.jsonl (full evidentiary chain)
"evidence_max_chars": 4000, # per-field truncation cap for evidence events
"multi_skill_report": False, # extra consolidation/report row per routed skill group
# ``multi_skill_report`` is the compatibility alias used before fan-out
# began staging independently adoptable proposals.
"multi_skill_fanout": None,
"multi_skill_report": False,
# ── adoption / safety ──────────────────────────────────────────────────
"auto_adopt": False, # default: stage + require explicit `adopt`
"managed_skill_name": "skillopt-sleep-learned",
skillopt_sleep/cycle.py
+344 -32
View File
@@ -9,12 +9,15 @@ CI use. With backend="anthropic" it spends the user's budget for real lift.
"""
from __future__ import annotations
import hashlib
import math
import os
import re
import shutil
import sys
import unicodedata
from dataclasses import dataclass
from typing import List, Optional
from typing import List, Optional, Sequence
from skillopt_sleep import evidence
from skillopt_sleep.backend import Backend, CursorBackendError, build_backend
@@ -25,15 +28,30 @@ from skillopt_sleep.harvest_sources import harvest_for_config
from skillopt_sleep.memory import ensure_skill_scaffold
from skillopt_sleep.mine import group_tasks_by_skill_hint, mine
from skillopt_sleep.multi_skill import (
SKIPPED,
GroupConsolidation,
SkillGroup,
accepted_group_skills,
consolidate_groups,
skill_group_reports,
)
from skillopt_sleep.skill_resolver import resolve_skill, skill_search_roots
from skillopt_sleep.staging import (
SkillProposal,
StagingError,
json_safe,
redact_secrets,
skill_proposal_rows,
write_staging,
)
from skillopt_sleep.staging import adopt as adopt_staging
from skillopt_sleep.staging import json_safe, redact_secrets, write_staging
from skillopt_sleep.state import SleepState, _now_iso
from skillopt_sleep.types import SessionDigest, SleepReport, TaskRecord
_ANSI_ESCAPE_RE = re.compile(
r"\x1b(?:\[[0-?]*[ -/]*[@-~]|\][^\x07\x1b]*(?:\x07|\x1b\\)?|[@-_])"
)
# ── Model-swap detection (F16) ───────────────────────────────
def _make_model_key(cfg: SleepConfig) -> str:
@@ -148,6 +166,33 @@ def _read(path: str) -> str:
return ""
def _read_live_baseline(path: str, label: str) -> tuple[str, str, str]:
"""Read one live document once and pin the exact bytes and target identity.
A missing file is a valid empty baseline. Other I/O failures and invalid
UTF-8 are not: silently treating either as an empty document could derive a
proposal from a scaffold and later overwrite data the cycle never read.
"""
realpath = os.path.realpath(os.path.abspath(path))
try:
with open(path, "rb") as handle:
raw = handle.read()
except FileNotFoundError:
return "", "", realpath
except OSError as exc:
raise StagingError(
f"could not read live {label} baseline {path!r}: "
f"{type(exc).__name__}: {exc}"
) from exc
try:
text = raw.decode("utf-8")
except UnicodeError as exc:
raise StagingError(
f"live {label} baseline is not valid UTF-8: {path!r}"
) from exc
return text, hashlib.sha256(raw).hexdigest(), realpath
def _progress(cfg: SleepConfig, message: str) -> None:
if cfg.get("progress", False):
print(f"[sleep] {message}", file=sys.stderr, flush=True)
@@ -167,9 +212,27 @@ def _discard_unstaged_evidence(path: str) -> None:
break
def _multi_skill_fanout_enabled(cfg: SleepConfig) -> bool:
"""Prefer the behavior-named flag while preserving the original alias."""
explicit = cfg.get("multi_skill_fanout")
if explicit is not None:
return bool(explicit)
return bool(cfg.get("multi_skill_report", False))
def _one_line_display_text(value: object) -> str:
"""Remove terminal controls and fold untrusted text onto one line."""
without_ansi = _ANSI_ESCAPE_RE.sub("", str(value))
without_controls = "".join(
" " if unicodedata.category(ch) in {"Cc", "Cf"} else ch
for ch in without_ansi
)
return " ".join(without_controls.split())
def _markdown_table_text(value: object) -> str:
"""Keep untrusted evidence text inside one readable Markdown table cell."""
text = " ".join(str(value).splitlines())
text = _one_line_display_text(value)
return (
text.replace("&", "&amp;")
.replace("<", "&lt;")
@@ -179,6 +242,14 @@ def _markdown_table_text(value: object) -> str:
)
def _markdown_text(value: object) -> str:
"""Render untrusted text literally in ordinary Markdown prose."""
text = _markdown_table_text(value)
for token in ("\\", "*", "_", "[", "]", "(", ")", "#", "+", "!", "{"):
text = text.replace(token, "\\" + token)
return text.replace("}", "\\}")
def _report_score(value: object) -> str:
"""Render an optional numeric score without breaking the report."""
if value is None:
@@ -191,16 +262,20 @@ def _report_score(value: object) -> str:
def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
project = _markdown_text(report.project)
backend = _markdown_text(cfg.get("backend"))
replay = _markdown_text(cfg.get("replay_mode"))
gate_action = _markdown_text(report.gate_action)
lines = [
f"# SkillOpt-Sleep — night {report.night} report",
"",
f"- project: `{report.project}`",
f"- backend: `{cfg.get('backend')}` replay: `{cfg.get('replay_mode')}`",
f"- project: `{project}`",
f"- backend: `{backend}` replay: `{replay}`",
f"- sessions harvested: {report.n_sessions}",
f"- tasks mined: {report.n_tasks} (replayed: {report.n_replayed})",
f"- held-out score: {_report_score(report.baseline_score)} "
f"-> {_report_score(report.candidate_score)}",
f"- gate: **{report.gate_action}** (accepted={report.accepted})",
f"- gate: **{gate_action}** (accepted={bool(report.accepted)})",
f"- no-regression gate: "
f"{'enabled' if cfg.get('gate_no_regression', False) else 'disabled'}",
f"- tokens used: {report.tokens_used}",
@@ -258,7 +333,13 @@ def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
if report.edits:
lines.append("## Accepted edits")
for e in report.edits:
lines.append(f"- [{e.target}/{e.op}] {e.content} \n _why: {e.rationale}_")
target = _markdown_text(e.target)
op = _markdown_text(e.op)
content = _markdown_text(e.content)
rationale = _markdown_text(e.rationale)
lines.append(
f"- \\[{target}/{op}\\] {content} \n _why: {rationale}_"
)
lines.append("")
if report.rejected_edits:
# On a leaked-holdout night the gate abstained rather than rejecting, so
@@ -268,7 +349,10 @@ def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
else:
lines.append("## Rejected by gate (kept as negative feedback)")
for e in report.rejected_edits:
lines.append(f"- [{e.target}/{e.op}] {e.content}")
target = _markdown_text(e.target)
op = _markdown_text(e.op)
content = _markdown_text(e.content)
lines.append(f"- \\[{target}/{op}\\] {content}")
lines.append("")
if report.unmatched_edits:
lines.append("## Proposed but changed nothing (never reached the gate)")
@@ -277,8 +361,15 @@ def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
"add, or an unknown op. "
"These were never scored — check the anchor text if a rule you expected is missing._")
for e in report.unmatched_edits:
anchor = f" \n _anchor: `{e.anchor}`_" if e.anchor else ""
lines.append(f"- [{e.target}/{e.op}] {e.content}{anchor}")
target = _markdown_text(e.target)
op = _markdown_text(e.op)
content = _markdown_text(e.content)
anchor = (
f" \n _anchor: `{_markdown_text(e.anchor)}`_"
if e.anchor
else ""
)
lines.append(f"- \\[{target}/{op}\\] {content}{anchor}")
lines.append("")
if report.skill_groups:
# The reviewer decides per skill, so the per-skill verdicts belong on
@@ -313,19 +404,188 @@ def _render_report_md(report: SleepReport, cfg: SleepConfig) -> str:
scores = "—"
edits = "—"
reason = f" — {_markdown_table_text(g.reason)}" if g.reason else ""
group_gate = _markdown_table_text(g.gate_action or "—")
lines.append(
f"| `{name}` | **{decision}**{reason} | {g.gate_action or '—'} "
f"| `{name}` | **{decision}**{reason} | {group_gate} "
f"| {g.n_tasks} | {scores} | {edits} |")
lines.append("")
if report.notes:
lines.append("## Notes")
for n in report.notes:
lines.append(f"- {n}")
lines.append(f"- {_markdown_text(n)}")
lines.append("")
lines.append("_Review, then run `/sleep adopt` to apply, or discard this folder._")
lines.append(
"_Review the staged artifacts, then use the adoption mode printed by "
"the run command (`--skill` / `--all-skills` for fan-out proposals, "
"`--legacy` for the managed skill or memory), or discard this folder._"
)
return "\n".join(lines)
def _cycle_skip_note(name: str, reason: str) -> str:
"""One-line skip reason for report.notes. Names are untrusted free text."""
label = str(name or "").strip() or "<unnamed>"
redacted = str(redact_secrets(f"cycle skipped skill {label}: {reason}"))
return _one_line_display_text(redacted)
def _skill_groups_from_live_baselines(
cfg: SleepConfig,
grouped: dict[str, List[TaskRecord]],
managed_name: str,
managed_skill: str,
) -> tuple[
List[SkillGroup],
dict[str, GroupConsolidation],
dict[str, str],
dict[str, str],
List[str],
]:
"""Load each hinted group's own live skill before consolidation.
The managed catch-all keeps the already-loaded managed document. Every
other group must resolve uniquely and be readable; otherwise it is
reported and skipped before a proposal can be derived from the wrong
baseline. Resolved paths are returned with the groups so staging targets
the same live file that supplied the proposal's input document.
"""
roots = skill_search_roots(cfg)
groups: List[SkillGroup] = []
skipped: dict[str, GroupConsolidation] = {}
live_paths: dict[str, str] = {}
live_hashes: dict[str, str] = {}
notes: List[str] = []
for raw_name, rows in grouped.items():
name = str(raw_name or "").strip()
if name == managed_name:
groups.append(SkillGroup(name, managed_skill, rows))
continue
resolution = resolve_skill(name, roots)
if not resolution.ok:
reason = resolution.reason or resolution.status
skipped[name] = GroupConsolidation(
name,
SKIPPED,
reason=reason,
n_tasks=len(rows),
)
notes.append(_cycle_skip_note(name, reason))
continue
try:
with open(resolution.path, "rb") as handle:
live_bytes = handle.read()
live_skill = live_bytes.decode("utf-8")
except (OSError, UnicodeError) as exc:
reason = f"could not read resolved SKILL.md ({type(exc).__name__})"
skipped[name] = GroupConsolidation(
name,
SKIPPED,
reason=reason,
n_tasks=len(rows),
)
notes.append(_cycle_skip_note(name, reason))
continue
groups.append(SkillGroup(name, live_skill, rows))
live_paths[name] = resolution.path
live_hashes[name] = hashlib.sha256(live_bytes).hexdigest()
return groups, skipped, live_paths, live_hashes, notes
def _skill_proposals_from_groups(
cfg: SleepConfig,
group_outcomes: dict,
managed_name: str,
resolved_paths: Optional[dict[str, str]] = None,
resolved_hashes: Optional[dict[str, str]] = None,
reserved_live_paths: Sequence[str] = (),
) -> tuple[List[SkillProposal], List[str]]:
"""Stage per-skill proposals for accepted groups whose names resolve uniquely.
In the cycle, ``resolved_paths`` pins each accepted result to the live
``SKILL.md`` that supplied its consolidation baseline. Direct low-level
callers may omit it and resolve names here. Unresolved, ambiguous, rejected,
empty, or colliding names are skipped so one bad hint cannot abort the
night; each skip is recorded on ``report.notes``. The managed catch-all is
never staged here — it stays on the legacy ``proposed_SKILL.md`` path.
"""
roots = skill_search_roots(cfg)
proposals: List[SkillProposal] = []
notes: List[str] = []
reserved_keys = {
unicodedata.normalize("NFC", os.path.realpath(path)).casefold()
for path in reserved_live_paths
if path
}
for name, new_skill in accepted_group_skills(group_outcomes).items():
if name == managed_name:
continue
if not str(new_skill or "").strip():
notes.append(_cycle_skip_note(name, "empty proposed_skill"))
continue
if resolved_paths is not None:
live_path = resolved_paths.get(name, "")
if not live_path:
notes.append(_cycle_skip_note(name, "no resolved live baseline"))
continue
live_sha256 = (
resolved_hashes.get(name, "")
if resolved_hashes is not None
else None
)
if resolved_hashes is not None and not live_sha256:
notes.append(_cycle_skip_note(name, "no hashed live baseline"))
continue
else:
resolution = resolve_skill(name, roots)
if not resolution.ok:
notes.append(
_cycle_skip_note(name, resolution.reason or resolution.status)
)
continue
live_path = resolution.path
live_sha256 = None
live_key = unicodedata.normalize(
"NFC", os.path.realpath(live_path)
).casefold()
if live_key in reserved_keys:
notes.append(
_cycle_skip_note(
name,
"same live target as the managed skill proposal",
)
)
continue
candidate = SkillProposal(
name,
new_skill,
live_path,
live_sha256=live_sha256,
live_realpath=live_path,
)
try:
skill_proposal_rows(proposals + [candidate])
except StagingError as exc:
notes.append(_cycle_skip_note(name, str(exc)))
continue
proposals.append(candidate)
return proposals, notes
def _history_for_skill_group(
history_tasks: List[TaskRecord],
skill_name: str,
managed_name: str,
) -> List[TaskRecord]:
"""Keep recalled evidence inside the same routed skill boundary."""
return [
task
for task in history_tasks
if (str(task.skill_hint or "").strip() or managed_name) == skill_name
]
def run_sleep_cycle(
cfg: Optional[SleepConfig] = None,
*,
@@ -413,9 +673,17 @@ def run_sleep_cycle(
live_memory_path = os.path.join(project, "CLAUDE.md")
live_skill_path = cfg.managed_skill_path()
_progress(cfg, f"live skill: {live_skill_path}")
raw_skill = _read(live_skill_path)
(
raw_skill,
live_skill_sha256,
live_skill_realpath,
) = _read_live_baseline(live_skill_path, "skill")
skill = raw_skill
memory = _read(live_memory_path)
(
memory,
live_memory_sha256,
live_memory_realpath,
) = _read_live_baseline(live_memory_path, "memory")
if not skill:
skill = ensure_skill_scaffold(
"", name=cfg.get("managed_skill_name", "skillopt-sleep-learned"),
@@ -594,27 +862,57 @@ def run_sleep_cycle(
# automatic; a night whose evidence produces only the catch-all group adds
# no rows and no calls.
#
# Each group currently starts from the same managed document. Resolving a
# hinted group to its own live SKILL.md is the resolver's job and is not
# wired here yet, so a row describes what that group's evidence did to the
# managed skill, not to a separate file.
if cfg.get("multi_skill_report", False):
managed_name = cfg.get("managed_skill_name", "skillopt-sleep-learned")
group_outcomes = {}
group_live_paths: dict[str, str] = {}
group_live_hashes: dict[str, str] = {}
managed_name = cfg.get("managed_skill_name", "skillopt-sleep-learned")
if _multi_skill_fanout_enabled(cfg):
grouped = group_tasks_by_skill_hint(tasks, managed_name)
only_catch_all = len(grouped) == 1 and managed_name in grouped
if grouped and not only_catch_all:
_progress(cfg, f"multi-skill report: groups={len(grouped)}")
group_outcomes = consolidate_groups(
backend,
[SkillGroup(name, skill, rows) for name, rows in grouped.items()],
memory,
edit_budget=cfg.get("edit_budget", 4),
gate_metric=cfg.get("gate_metric", "mixed"),
gate_mixed_weight=cfg.get("gate_mixed_weight", 0.5),
gate_no_regression=cfg.get("gate_no_regression", False),
gate_mode=cfg.get("gate_mode", "on"),
night=night,
(
live_groups,
skipped_groups,
group_live_paths,
group_live_hashes,
skip_notes,
) = _skill_groups_from_live_baselines(
cfg, grouped, managed_name, skill
)
report.notes.extend(skip_notes)
try:
consolidated_groups = consolidate_groups(
backend,
live_groups,
memory,
consolidate_fn=dream_consolidate,
group_kwargs_fn=lambda group: {
"history_tasks": _history_for_skill_group(
history_tasks,
group.skill_name,
managed_name,
)
},
recall_k=recall_k,
dream_rollouts=int(cfg.get("dream_rollouts", 1) or 1),
dream_factor=int(cfg.get("dream_factor", 0) or 0),
edit_budget=cfg.get("edit_budget", 4),
gate_metric=cfg.get("gate_metric", "mixed"),
gate_mixed_weight=cfg.get("gate_mixed_weight", 0.5),
gate_no_regression=cfg.get("gate_no_regression", False),
gate_mode=cfg.get("gate_mode", "on"),
evolve_skill=cfg.get("evolve_skill", True),
night=night,
)
except CursorBackendError:
_discard_unstaged_evidence(staging_dir_pre)
raise
for raw_name in grouped:
name = str(raw_name or "").strip()
outcome = skipped_groups.get(name) or consolidated_groups.get(name)
if outcome is not None:
group_outcomes[name] = outcome
group_rows = skill_group_reports(group_outcomes)
# Skill hints and isolated backend failures are both untrusted
# free text. Scrub them before either report.json or report.md is
@@ -634,9 +932,18 @@ def run_sleep_cycle(
adopted_paths: List[str] = []
if not dry_run:
_progress(cfg, "staging start")
report_md = _render_report_md(report, cfg)
proposed_skill = result.new_skill if (cfg.get("evolve_skill") and result.accepted) else None
proposed_memory = result.new_memory if (cfg.get("evolve_memory") and result.accepted) else None
skill_proposals, skip_notes = _skill_proposals_from_groups(
cfg,
group_outcomes,
managed_name,
group_live_paths,
group_live_hashes,
[live_skill_path] if proposed_skill is not None else [],
)
report.notes.extend(skip_notes)
report_md = _render_report_md(report, cfg)
staging_dir = write_staging(
project,
report=report,
@@ -644,8 +951,13 @@ def run_sleep_cycle(
proposed_memory=proposed_memory,
live_skill_path=live_skill_path,
live_memory_path=live_memory_path,
live_skill_sha256=live_skill_sha256,
live_memory_sha256=live_memory_sha256,
live_skill_realpath=live_skill_realpath,
live_memory_realpath=live_memory_realpath,
report_md=report_md,
out_dir=staging_dir_pre,
skill_proposals=skill_proposals,
)
if ev is not None:
ev.log("stage", "staged", staging_dir=staging_dir,
skillopt_sleep/multi_skill.py
+19 -2
View File
@@ -13,8 +13,9 @@ from __future__ import annotations
from dataclasses import dataclass, field
from typing import Callable, Dict, List, Optional, Sequence
from skillopt_sleep.backend import Backend
from skillopt_sleep.backend import Backend, CursorBackendError
from skillopt_sleep.consolidate import ConsolidationResult, consolidate
from skillopt_sleep.handoff_backend import PendingCalls
from skillopt_sleep.types import SkillGroupReport, TaskRecord
CONSOLIDATED = "consolidated"
@@ -52,6 +53,7 @@ def consolidate_groups(
memory: str = "",
*,
consolidate_fn: Callable[..., ConsolidationResult] = consolidate,
group_kwargs_fn: Optional[Callable[[SkillGroup], Dict[str, object]]] = None,
**consolidate_kwargs: object,
) -> Dict[str, GroupConsolidation]:
"""Consolidate each group independently, in order, isolating failures.
@@ -70,10 +72,15 @@ def consolidate_groups(
``memory`` is the shared agent memory and is passed through read-only: group
runs evolve skills only, so no group can rewrite another group's memory.
``group_kwargs_fn`` can add group-scoped inputs such as recalled history;
its ordinary failures are isolated to that group. ``PendingCalls`` and
``CursorBackendError`` from either the factory or consolidator propagate as
cycle-level pause/fail-closed control flow rather than becoming report rows.
"""
# This wrapper's contract is stricter than consolidate(): shared memory is
# always read-only. Override a caller-supplied value instead of passing a
# duplicate keyword (which would otherwise turn the group into a failure).
consolidate_kwargs = dict(consolidate_kwargs)
consolidate_kwargs["evolve_memory"] = False
out: Dict[str, GroupConsolidation] = {}
for group in groups:
@@ -95,10 +102,20 @@ def consolidate_groups(
)
continue
try:
group_kwargs = dict(consolidate_kwargs)
if group_kwargs_fn is not None:
group_kwargs.update(group_kwargs_fn(group))
# A per-group factory cannot weaken the shared-memory invariant.
group_kwargs["evolve_memory"] = False
result = consolidate_fn(
backend, list(group.tasks), group.skill, memory,
**consolidate_kwargs,
**group_kwargs,
)
except (PendingCalls, CursorBackendError):
# These exceptions are cycle-level control flow, not isolated
# evidence about one group. Swallowing them can advance/save an
# incomplete handoff night or weaken Cursor's fail-closed path.
raise
except Exception as exc: # one group's failure must not abort the night
out[name] = GroupConsolidation(
name, FAILED, reason=f"{type(exc).__name__}: {exc}"[:300],
skillopt_sleep/scheduler.py
+73 -16
View File
@@ -5,7 +5,9 @@ runs the sleep cycle automatically, so the user doesn't have to wire it manually
"""
from __future__ import annotations
import hashlib
import os
import shlex
import shutil
import subprocess
import sys
@@ -59,8 +61,8 @@ def _have_schtasks() -> bool:
def _win_task_name(project: str) -> str:
project = os.path.abspath(project)
safe = project.replace(":\\", "_").replace("\\", "_").replace("/", "_").replace(" ", "_")
return f"SkillOpt-Sleep-{safe}"
digest = hashlib.sha256(project.encode("utf-8")).hexdigest()[:20]
return f"SkillOpt-Sleep-{digest}"
def _create_win_task(task_name: str, command: str, hour: int, minute: int) -> bool:
@@ -102,26 +104,51 @@ def _list_win_tasks() -> List[str]:
def _runner_cmd(project: str, backend: str, extra: str, python: str) -> str:
_validate_scheduler_text("project", project)
_validate_scheduler_text("python", python)
_validate_scheduler_text("repository", _repo_root())
_validate_scheduler_text("backend", backend)
if extra:
_validate_scheduler_text("extra scheduler arguments", extra)
logdir = os.path.join(project, ".skillopt-sleep")
log = os.path.join(logdir, "cron.log")
# use absolute python + -m so cron's/scheduler's minimal env still works
cmd = (f'"{python}" -m skillopt_sleep run --project "{project}" '
f'--scope invoked --backend {backend} {extra}'.rstrip())
args = [
python,
"-m",
"skillopt_sleep",
"run",
"--project",
project,
"--scope",
"invoked",
"--backend",
backend,
*shlex.split(extra),
]
if sys.platform == "win32":
helper_script = os.path.join(logdir, "run.cmd")
helper_script = os.path.join(logdir, "run.ps1")
try:
os.makedirs(logdir, exist_ok=True)
quoted = " ".join(_powershell_literal(value) for value in args[1:])
content = (
"@echo off\n"
f'cd /d "{_repo_root()}"\n'
f'{cmd} >> "{log}" 2>&1\n'
"$ErrorActionPreference = 'Stop'\n"
f"Set-Location -LiteralPath {_powershell_literal(_repo_root())}\n"
f"& {_powershell_literal(python)} {quoted} "
f"*>> {_powershell_literal(log)}\n"
)
with open(helper_script, "w", encoding="utf-8") as f:
f.write(content)
except Exception:
pass
return f'"{helper_script}"'
return f'mkdir -p "{logdir}"; cd "{_repo_root()}" && {cmd} >> "{log}" 2>&1'
except OSError as exc:
raise ValueError(f"could not create the Windows scheduler helper: {exc}") from exc
return (
"powershell.exe -NoProfile -NonInteractive -ExecutionPolicy Bypass "
f"-File {_windows_command_quote(helper_script)}"
)
cmd = shlex.join(args)
return (
f"mkdir -p {shlex.quote(logdir)}; "
f"cd {shlex.quote(_repo_root())} && {cmd} >> {shlex.quote(log)} 2>&1"
)
def _repo_root() -> str:
@@ -129,8 +156,31 @@ def _repo_root() -> str:
return os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
def _validate_scheduler_text(label: str, value: str) -> None:
if not isinstance(value, str) or not value:
raise ValueError(f"{label} must be a non-empty string")
if any(ord(ch) < 32 or ord(ch) == 127 for ch in value):
raise ValueError(f"{label} cannot contain control characters")
def _powershell_literal(value: str) -> str:
"""Single-quoted PowerShell data literal."""
_validate_scheduler_text("scheduler argument", value)
return "'" + value.replace("'", "''") + "'"
def _windows_command_quote(value: str) -> str:
"""Quote a Windows path whose allowed characters cannot include quotes."""
_validate_scheduler_text("scheduler path", value)
if '"' in value:
raise ValueError("scheduler path cannot contain a double quote")
return f'"{value}"'
def _project_marker(project: str) -> str:
return f"# project={os.path.abspath(project)}"
canonical = os.path.abspath(project)
digest = hashlib.sha256(canonical.encode("utf-8")).hexdigest()
return f"# project-sha256={digest}"
def schedule(project: str, *, backend: str = "mock", hour: int = 3, minute: int = 17,
@@ -140,9 +190,16 @@ def schedule(project: str, *, backend: str = "mock", hour: int = 3, minute: int
Returns (installed, message). If the scheduler backend is unavailable, installed=False and
the message contains instructions to add manually.
"""
if type(hour) is not int or not 0 <= hour <= 23:
return False, "hour must be an integer from 0 through 23"
if type(minute) is not int or not 0 <= minute <= 59:
return False, "minute must be an integer from 0 through 59"
project = os.path.abspath(project)
python = python or sys.executable or "python3"
runner_cmd = _runner_cmd(project, backend, extra, python)
try:
runner_cmd = _runner_cmd(project, backend, extra, python)
except (ValueError, OSError) as exc:
return False, f"Refusing unsafe scheduler configuration: {exc}"
if sys.platform == "win32":
if not _have_schtasks():
@@ -195,7 +252,7 @@ def unschedule(project: Optional[str] = None, *, all_projects: bool = False) ->
ok = _delete_win_task(tn)
try:
logdir = os.path.join(project, ".skillopt-sleep")
helper = os.path.join(logdir, "run.cmd")
helper = os.path.join(logdir, "run.ps1")
if os.path.exists(helper):
os.remove(helper)
except Exception:
skillopt_sleep/skill_resolver.py
+62 -22
View File
@@ -118,32 +118,72 @@ def _plugin_skills_root(plugin_dir: str) -> str:
def skill_search_roots(cfg: object) -> List[str]:
"""Documented local skill roots for a config: user skills, then plugin cache.
"""Return every configured native skill root in deterministic order.
``<claude_home>/skills`` holds hand-written skills. Installed Claude Code
plugins expose theirs under the plugin cache, in either the versioned
marketplace layout ``plugins/cache/<marketplace>/<plugin>/<version>/skills``
or the legacy ``plugins/cache/<marketplace>/<plugin>/skills``. At most one
root per installed plugin is returned, in that fixed precedence order.
The resolver supports project-native Claude/Codex/Cursor/Devin layouts,
configured user homes, explicit extra roots, and installed Claude plugins.
Returning all matches is intentional: ``resolve_skill`` refuses ambiguity
instead of silently choosing one agent's file over another.
"""
configured = str(getattr(cfg, "claude_home", "") or "").strip()
if not configured:
# Guard before abspath: os.path.abspath("") is the CWD, which would
# silently search a tree well outside the documented ~/.claude root.
return []
claude_home = os.path.abspath(os.path.expanduser(configured))
roots = [os.path.join(claude_home, "skills")]
roots: List[str] = []
cache = os.path.join(claude_home, "plugins", "cache")
for marketplace in _listdir(cache):
plugins_dir = os.path.join(cache, marketplace)
if not os.path.isdir(plugins_dir):
invoked_project = str(getattr(cfg, "invoked_project", "") or "").strip()
if invoked_project:
project = os.path.abspath(os.path.expanduser(invoked_project))
roots.extend(
os.path.join(project, relative)
for relative in (
os.path.join(".agents", "skills"),
os.path.join(".claude", "skills"),
os.path.join(".cursor", "skills"),
os.path.join(".devin", "skills"),
)
)
# Keep the established user-level Claude root. Other agents' user-level
# layouts are not sufficiently standardized; callers can add them through
# ``skill_roots`` without silently creating cross-agent ambiguity.
configured_claude = str(getattr(cfg, "claude_home", "") or "").strip()
if configured_claude:
roots.append(
os.path.join(
os.path.abspath(os.path.expanduser(configured_claude)),
"skills",
)
)
explicit = getattr(cfg, "skill_roots", ())
if isinstance(explicit, (list, tuple)):
for value in explicit:
if isinstance(value, str) and value.strip():
path = os.path.expanduser(value.strip())
if not os.path.isabs(path) and invoked_project:
path = os.path.join(invoked_project, path)
roots.append(os.path.abspath(path))
if configured_claude:
claude_home = os.path.abspath(os.path.expanduser(configured_claude))
cache = os.path.join(claude_home, "plugins", "cache")
for marketplace in _listdir(cache):
plugins_dir = os.path.join(cache, marketplace)
if not os.path.isdir(plugins_dir):
continue
for plugin in _listdir(plugins_dir):
root = _plugin_skills_root(os.path.join(plugins_dir, plugin))
if root:
roots.append(root)
existing: List[str] = []
seen = set()
for root in roots:
if not os.path.isdir(root):
continue
for plugin in _listdir(plugins_dir):
root = _plugin_skills_root(os.path.join(plugins_dir, plugin))
if root:
roots.append(root)
return [r for r in roots if os.path.isdir(r)]
canonical = os.path.realpath(root)
key = os.path.normcase(canonical)
if key not in seen:
seen.add(key)
existing.append(canonical)
return existing
def _contained_skill_file(root: str, name: str) -> str:
skillopt_sleep/staging.py
+2554 -78
View File
File diff suppressed because it is too large Load Diff
skillopt_sleep/state.py
+8 -4
View File
@@ -11,9 +11,10 @@ short-term -> long-term transfer). It records:
"""
from __future__ import annotations
import copy
import json
import os
from typing import Any, Dict, List, Optional
from typing import Any, Dict, Optional
def _now_iso(clock: Optional[float] = None) -> str:
@@ -37,7 +38,10 @@ DEFAULT_STATE: Dict[str, Any] = {
class SleepState:
def __init__(self, path: str, data: Optional[Dict[str, Any]] = None) -> None:
self.path = path
self.data = data if data is not None else dict(DEFAULT_STATE)
# DEFAULT_STATE contains mutable lists and dicts. A shallow copy makes
# independent projects in one Python process share history, harvest
# cursors, and recalled tasks until they are persisted.
self.data = data if data is not None else copy.deepcopy(DEFAULT_STATE)
# io ---------------------------------------------------------------------
@classmethod
@@ -46,12 +50,12 @@ class SleepState:
try:
with open(path) as f:
data = json.load(f)
merged = dict(DEFAULT_STATE)
merged = copy.deepcopy(DEFAULT_STATE)
merged.update(data if isinstance(data, dict) else {})
return cls(path, merged)
except Exception:
pass
return cls(path, dict(DEFAULT_STATE))
return cls(path, copy.deepcopy(DEFAULT_STATE))
def save(self) -> None:
os.makedirs(os.path.dirname(self.path), exist_ok=True)
tests/test_devin_plugin.py
+283 -3
View File
@@ -1,5 +1,7 @@
"""Tests for the Devin MCP plugin: tool schema, ATIF-v1.7 harvest, path expansion."""
import contextlib
import importlib
import io
import json
import os
import shlex
@@ -9,18 +11,30 @@ import subprocess
import sys
import tempfile
import unittest
from unittest import mock
# Allow importing from the plugin directory (mirrors tests/test_mcp_schema.py)
PLUGIN = os.path.join(os.path.dirname(__file__), "..", "plugins", "devin")
sys.path.insert(0, PLUGIN)
import mcp_server # noqa: E402
import harvest_devin as hw # noqa: E402
import harvest_devin as hw # noqa: E402
import mcp_server # noqa: E402
FIXTURES = os.path.join(PLUGIN, "fixtures")
INSTALLER = os.path.join(PLUGIN, "install.sh")
def _call(name="sleep_status", arguments=None, **params):
call_params = {"name": name, "arguments": {} if arguments is None else arguments}
call_params.update(params)
return {
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": call_params,
}
def _read_jsonl(path):
with open(path, encoding="utf-8") as f:
return [json.loads(line) for line in f if line.strip()]
@@ -58,11 +72,277 @@ class TestDevinMcpSchema(unittest.TestCase):
# parity with plugins/copilot's schema (tests/test_plugin_sync.py)
props = set(mcp_server._TOOL_SCHEMA["properties"].keys())
for param in {"project", "backend", "scope", "source", "model",
"tasks_file", "target_skill_path", "max_sessions",
"tasks_file", "target_skill_path", "staging", "skills",
"all_skills", "legacy", "max_sessions",
"max_tasks", "lookback_hours", "auto_adopt", "json",
"edit_budget", "hour", "minute"}:
self.assertIn(param, props)
def test_adopt_selection_schema_types(self):
props = mcp_server._TOOL_SCHEMA["properties"]
self.assertEqual(props["staging"]["type"], "string")
self.assertEqual(props["skills"]["type"], "array")
self.assertEqual(props["skills"]["items"]["type"], "string")
self.assertEqual(props["all_skills"]["type"], "boolean")
self.assertEqual(props["legacy"]["type"], "boolean")
def test_adopt_forwards_fanout_selection_as_argv_without_sync(self):
completed = subprocess.CompletedProcess(
args=[], returncode=0, stdout="adopted\n", stderr=""
)
arguments = {
"project": "/tmp/devin workspace",
"staging": "/tmp/night with spaces",
"skills": ["alpha", "--leading-dash", "space ; $(literal)"],
}
with mock.patch.object(
mcp_server.subprocess, "run", return_value=completed
) as run:
result = mcp_server._run_engine("adopt", arguments)
self.assertEqual(result.text, "[engine]\nadopted")
self.assertEqual(result.returncode, 0)
run.assert_called_once()
command = run.call_args.args[0]
self.assertEqual(
command[-7:],
[
"--staging", "/tmp/night with spaces",
"--skill", "alpha",
"--skill=--leading-dash",
"--skill", "space ; $(literal)",
],
)
self.assertNotIn("shell", run.call_args.kwargs)
def test_adoption_never_performs_post_engine_copy(self):
cases = (
("bare legacy success", {}, 0),
("explicit legacy success", {"legacy": True}, 0),
("bare adoption refused", {}, 2),
("explicit legacy failed", {"legacy": True}, 1),
("per-skill success", {"skills": ["alpha"]}, 0),
("all-skills success", {"all_skills": True}, 0),
)
for name, selection, returncode in cases:
with self.subTest(name=name):
completed = subprocess.CompletedProcess(
args=[], returncode=returncode, stdout="result", stderr=""
)
arguments = {"project": "/tmp/devin-workspace", **selection}
with mock.patch.object(
mcp_server.subprocess, "run", return_value=completed
) as run:
result = mcp_server._run_engine("adopt", arguments)
command = run.call_args.args[0]
if selection.get("legacy"):
self.assertIn("--legacy", command)
if selection.get("all_skills"):
self.assertIn("--all-skills", command)
self.assertEqual(result.returncode, returncode)
self.assertNotIn("synced", result.text)
def test_adopt_rejects_non_array_skills_without_spawning(self):
with mock.patch.object(mcp_server.subprocess, "run") as run:
with self.assertRaisesRegex(ValueError, "skills must be an array"):
mcp_server._run_engine("adopt", {"skills": "alpha"})
run.assert_not_called()
class TestDevinMcpRuntimeValidation(unittest.TestCase):
def test_malformed_request_envelopes_return_json_rpc_errors(self):
cases = (
(None, "request must be a JSON object"),
({"method": "ping"}, "jsonrpc must be '2.0'"),
({"jsonrpc": "2.0", "id": 1, "method": 4}, "method must be"),
({"jsonrpc": "2.0", "id": 1, "method": "ping", "params": False},
"params must be an object"),
({"jsonrpc": "2.0", "id": [], "method": "ping"}, "id must be"),
({"jsonrpc": "2.0", "id": 1, "method": "ping", "extra": 1},
"unknown request member"),
)
for request, message in cases:
with self.subTest(request=request):
response = mcp_server.handle(request)
self.assertEqual(response["error"]["code"], -32600)
self.assertIn(message, response["error"]["message"])
if type(request) is dict and type(request.get("id")) not in {str, int}:
self.assertIsNone(response["id"])
def test_params_and_arguments_require_known_properties_and_objects(self):
cases = (
(_call(extra=1), "unknown params member"),
(_call(arguments=[]), "arguments must be an object"),
(_call(arguments={"unknown": True}), "unknown argument"),
)
for request, message in cases:
with self.subTest(request=request), mock.patch.object(
mcp_server.subprocess, "run"
) as run, mock.patch.object(mcp_server, "_run_harvest") as harvest:
response = mcp_server.handle(request)
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
run.assert_not_called()
harvest.assert_not_called()
def test_wrong_scalar_types_and_bounds_are_rejected_before_harvest(self):
cases = (
({"auto_adopt": "false"}, "auto_adopt must be a boolean"),
({"json": "false"}, "json must be a boolean"),
({"progress": 1}, "progress must be a boolean"),
({"max_sessions": False}, "max_sessions must be an integer"),
({"lookback_hours": -1}, "lookback_hours must be between"),
({"backend": "unknown"}, "unsupported backend"),
)
for arguments, message in cases:
with self.subTest(arguments=arguments), mock.patch.object(
mcp_server, "_run_harvest"
) as harvest, mock.patch.object(mcp_server.subprocess, "run") as run:
response = mcp_server.handle(_call("sleep_run", arguments))
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
harvest.assert_not_called()
run.assert_not_called()
def test_schedule_bounds_and_action_specific_arguments_are_rejected(self):
cases = (
("sleep_schedule", {"hour": -1}, "hour must be between"),
("sleep_schedule", {"minute": 60}, "minute must be between"),
("sleep_status", {"hour": 3}, "valid only for sleep_schedule"),
("sleep_status", {"legacy": False}, "valid only for sleep_adopt"),
)
for tool, arguments, message in cases:
with self.subTest(tool=tool, arguments=arguments), mock.patch.object(
mcp_server, "_run_harvest"
) as harvest, mock.patch.object(mcp_server.subprocess, "run") as run:
response = mcp_server.handle(_call(tool, arguments))
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
harvest.assert_not_called()
run.assert_not_called()
def test_adoption_arrays_and_selection_modes_are_strict(self):
cases = (
({"all_skills": "false"}, "all_skills must be a boolean"),
({"legacy": "false"}, "legacy must be a boolean"),
({"skills": [None]}, "skills entry must be a string"),
({"skills": ["\t"]}, "control characters"),
({"skills": ["alpha", " alpha "]}, "must be unique"),
({"skills": ["alpha"], "legacy": True}, "choose at most one"),
)
for arguments, message in cases:
with self.subTest(arguments=arguments), mock.patch.object(
mcp_server.subprocess, "run"
) as run:
response = mcp_server.handle(_call("sleep_adopt", arguments))
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
run.assert_not_called()
def test_every_string_boolean_and_integer_contract_is_exact_and_bounded(self):
for key in mcp_server._STRING_ARGS:
action = "adopt" if key == "staging" else "status"
with self.subTest(kind="string", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be a string"
):
mcp_server._validate_tool_arguments(action, {key: 1})
for key in mcp_server._BOOLEAN_ARGS:
action = "adopt" if key in {"all_skills", "legacy"} else "status"
with self.subTest(kind="boolean", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be a boolean"
):
mcp_server._validate_tool_arguments(action, {key: "false"})
for key, (minimum, maximum) in mcp_server._INTEGER_BOUNDS.items():
action = "schedule" if key in {"hour", "minute"} else "status"
with self.subTest(kind="integer-bool", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be an integer"
):
mcp_server._validate_tool_arguments(action, {key: True})
for value in (minimum - 1, maximum + 1):
with self.subTest(kind="integer-bound", key=key, value=value), \
self.assertRaisesRegex(ValueError, f"{key} must be between"):
mcp_server._validate_tool_arguments(action, {key: value})
self.assertEqual(
mcp_server._validate_tool_arguments(action, {key: minimum})[key],
minimum,
)
self.assertEqual(
mcp_server._validate_tool_arguments(action, {key: maximum})[key],
maximum,
)
def test_harvest_failure_stops_engine_and_does_not_use_stale_cache(self):
failure = mcp_server.EngineResult("conversion failed", 7, "bad ATIF")
with mock.patch.object(
mcp_server, "_run_harvest", return_value=failure
) as harvest, mock.patch.object(mcp_server.subprocess, "run") as run:
result = mcp_server._run_engine("status", {})
harvest.assert_called_once_with()
run.assert_not_called()
self.assertEqual(result.returncode, 7)
self.assertIn("conversion failed", result.text)
self.assertIn("bad ATIF", result.text)
def test_harvest_subprocess_returncode_is_preserved(self):
completed = subprocess.CompletedProcess(
args=[], returncode=6, stdout="conversion stopped\n", stderr="bad source\n"
)
with mock.patch.object(mcp_server.subprocess, "run", return_value=completed):
result = mcp_server._run_harvest()
self.assertEqual(result.returncode, 6)
self.assertEqual(result.text, "conversion stopped")
self.assertEqual(result.diagnostics, "bad source")
def test_engine_status_maps_to_mcp_error_and_handoff_states(self):
cases = (
(0, False, "ok"),
(1, True, "error"),
(3, False, "handoff_pending"),
)
for returncode, is_error, status in cases:
with self.subTest(returncode=returncode), mock.patch.object(
mcp_server, "_run_engine",
return_value=mcp_server.EngineResult("engine output", returncode),
):
result = mcp_server.handle(_call())["result"]
self.assertIs(result["isError"], is_error)
self.assertEqual(result["structuredContent"]["status"], status)
self.assertEqual(result["structuredContent"]["exit_code"], returncode)
def test_json_stdout_is_parseable_without_harvest_or_stderr_prefixes(self):
harvest = mcp_server.EngineResult("converted 3 sessions", 0, "harvest note")
engine = subprocess.CompletedProcess(
args=[], returncode=0, stdout='{"nights": 4}\n', stderr="engine note\n"
)
with mock.patch.object(
mcp_server, "_run_harvest", return_value=harvest
), mock.patch.object(mcp_server.subprocess, "run", return_value=engine):
run = mcp_server._run_engine("status", {"json": True})
self.assertEqual(json.loads(run.text), {"nights": 4})
self.assertNotIn("harvest", run.text)
self.assertIn("converted 3 sessions", run.diagnostics)
self.assertIn("engine note", run.diagnostics)
def test_json_tool_result_includes_parsed_structured_output(self):
run = mcp_server.EngineResult('{"pending": true}', 3, "answer prompts")
with mock.patch.object(mcp_server, "_run_engine", return_value=run):
result = mcp_server.handle(_call(arguments={"json": True}))["result"]
self.assertEqual(result["structuredContent"]["output"], {"pending": True})
self.assertEqual(result["structuredContent"]["status"], "handoff_pending")
self.assertFalse(result["isError"])
def test_main_emits_parse_error_for_malformed_json(self):
output = io.StringIO()
with mock.patch.object(sys, "stdin", io.StringIO("not-json\n")), \
contextlib.redirect_stdout(output):
self.assertEqual(mcp_server.main(), 0)
response = json.loads(output.getvalue())
self.assertEqual(response["error"]["code"], -32700)
class TestClaudeHomeExpansion(unittest.TestCase):
"""Regression: ~ must be expanded even when CLAUDE_HOME comes from the env
tests/test_handoff_backend.py
+91
View File
@@ -7,6 +7,7 @@ import os
import re
import tempfile
import unittest
from unittest import mock
from skillopt_sleep.backend import get_backend
from skillopt_sleep.config import load_config
@@ -195,6 +196,96 @@ class TestHandoffCli(unittest.TestCase):
# must not crash: corrupt pin -> fresh harvest -> no tasks -> 0
self.assertEqual(rc, 0)
def test_completed_run_json_stdout_is_one_parseable_document(self):
import contextlib
import io
from skillopt_sleep.__main__ import main
from skillopt_sleep.cycle import CycleOutcome
from skillopt_sleep.tasks_file import make_tasks_payload, write_tasks_file
from skillopt_sleep.types import SleepReport
with tempfile.TemporaryDirectory() as proj, \
tempfile.TemporaryDirectory() as home:
tasks_path = os.path.join(proj, "tasks.json")
payload = make_tasks_payload(_tasks(), project=proj)
payload["reviewed"] = True
write_tasks_file(tasks_path, payload)
staging_dir = os.path.join(proj, "finished-staging")
os.makedirs(staging_dir)
outcome = CycleOutcome(
report=SleepReport(
night=7,
project=proj,
n_tasks=len(_tasks()),
accepted=True,
gate_action="accept_new_best",
),
staging_dir=staging_dir,
adopted=False,
adopted_paths=[],
)
stdout = io.StringIO()
stderr = io.StringIO()
with mock.patch(
"skillopt_sleep.__main__.run_sleep_cycle",
return_value=outcome,
), contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr):
rc = main([
"run", "--backend", "handoff", "--json",
"--project", proj,
"--claude-home", os.path.join(home, ".claude"),
"--tasks-file", tasks_path,
])
self.assertEqual(rc, 0)
result = json.loads(stdout.getvalue())
self.assertEqual(result["night"], 7)
self.assertEqual(result["staging_dir"], staging_dir)
self.assertIn("archived round data", stderr.getvalue())
def test_archive_failure_never_prints_a_success_document(self):
import contextlib
import io
from skillopt_sleep.__main__ import main
from skillopt_sleep.cycle import CycleOutcome
from skillopt_sleep.tasks_file import make_tasks_payload, write_tasks_file
from skillopt_sleep.types import SleepReport
with tempfile.TemporaryDirectory() as proj, \
tempfile.TemporaryDirectory() as home:
tasks_path = os.path.join(proj, "tasks.json")
payload = make_tasks_payload(_tasks(), project=proj)
payload["reviewed"] = True
write_tasks_file(tasks_path, payload)
staging_dir = os.path.join(proj, "finished-staging")
os.makedirs(staging_dir)
outcome = CycleOutcome(
report=SleepReport(night=7, project=proj, accepted=True),
staging_dir=staging_dir,
adopted=False,
adopted_paths=[],
)
stdout = io.StringIO()
with mock.patch(
"skillopt_sleep.__main__.run_sleep_cycle", return_value=outcome
), mock.patch(
"skillopt_sleep.__main__.os.rename",
side_effect=OSError("archive denied"),
), contextlib.redirect_stdout(stdout):
rc = main([
"run", "--backend", "handoff", "--json",
"--project", proj,
"--claude-home", os.path.join(home, ".claude"),
"--tasks-file", tasks_path,
])
self.assertEqual(rc, 1)
result = json.loads(stdout.getvalue())
self.assertEqual(result["ok"], False)
self.assertEqual(result["error"], "staging_refused")
self.assertNotIn("night", result)
def test_run_with_no_tasks_exits_0_and_advances_harvest_window(self):
from skillopt_sleep.__main__ import main
from skillopt_sleep.config import load_config
tests/test_mcp_schema.py
+262 -10
View File
@@ -1,37 +1,289 @@
"""Tests for the Copilot MCP server schema completeness."""
import contextlib
import importlib.util
import io
import json
import os
import subprocess
import sys
import unittest
from unittest import mock
# Allow importing from the plugin directory
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "plugins", "copilot"))
PLUGIN = os.path.abspath(
os.path.join(os.path.dirname(__file__), "..", "plugins", "copilot")
)
MODULE_PATH = os.path.join(PLUGIN, "mcp_server.py")
SPEC = importlib.util.spec_from_file_location("copilot_mcp_server_test", MODULE_PATH)
assert SPEC is not None and SPEC.loader is not None
mcp_server = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(mcp_server)
def _call(name="sleep_status", arguments=None, **params):
call_params = {"name": name, "arguments": {} if arguments is None else arguments}
call_params.update(params)
return {
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": call_params,
}
class TestMcpSchema(unittest.TestCase):
def test_schema_includes_all_engine_flags(self):
from mcp_server import _TOOL_SCHEMA
required_params = {
"project", "backend", "scope", "source", "model",
"tasks_file", "target_skill_path", "progress",
"tasks_file", "target_skill_path", "staging", "skills",
"all_skills", "legacy", "progress",
"max_sessions", "max_tasks", "lookback_hours",
"auto_adopt", "json", "edit_budget",
}
schema_props = set(_TOOL_SCHEMA["properties"].keys())
schema_props = set(mcp_server._TOOL_SCHEMA["properties"].keys())
missing = required_params - schema_props
self.assertEqual(missing, set(), f"MCP schema missing: {missing}")
def test_adopt_selection_schema_types(self):
props = mcp_server._TOOL_SCHEMA["properties"]
self.assertEqual(props["staging"]["type"], "string")
self.assertEqual(props["skills"]["type"], "array")
self.assertEqual(props["skills"]["items"]["type"], "string")
self.assertEqual(props["all_skills"]["type"], "boolean")
self.assertEqual(props["legacy"]["type"], "boolean")
def test_all_backends_in_enum(self):
from mcp_server import _TOOL_SCHEMA
backends = _TOOL_SCHEMA["properties"]["backend"]["enum"]
for b in ["mock", "claude", "codex", "copilot"]:
backends = mcp_server._TOOL_SCHEMA["properties"]["backend"]["enum"]
for b in ["mock", "claude", "codex", "copilot", "handoff"]:
self.assertIn(b, backends)
def test_schedule_tools_exist(self):
from mcp_server import TOOLS
names = {t["name"] for t in TOOLS}
names = {t["name"] for t in mcp_server.TOOLS}
self.assertIn("sleep_schedule", names)
self.assertIn("sleep_unschedule", names)
def test_adopt_forwards_staging_and_repeated_skills_as_argv(self):
completed = subprocess.CompletedProcess(
args=[], returncode=0, stdout="adopted\n", stderr=""
)
arguments = {
"staging": "/tmp/night with spaces",
"skills": ["alpha", "--leading-dash", "space ; $(literal)"],
}
with mock.patch.object(
mcp_server.subprocess, "run", return_value=completed
) as run:
result = mcp_server._run_engine("adopt", arguments)
self.assertEqual(result.text, "adopted")
self.assertEqual(result.returncode, 0)
run.assert_called_once()
command = run.call_args.args[0]
self.assertEqual(
command[-7:],
[
"--staging", "/tmp/night with spaces",
"--skill", "alpha",
"--skill=--leading-dash",
"--skill", "space ; $(literal)",
],
)
self.assertNotIn("shell", run.call_args.kwargs)
def test_adopt_forwards_boolean_selection_flags(self):
completed = subprocess.CompletedProcess(
args=[], returncode=0, stdout="", stderr=""
)
for argument, flag in (("all_skills", "--all-skills"), ("legacy", "--legacy")):
with self.subTest(argument=argument), mock.patch.object(
mcp_server.subprocess, "run", return_value=completed
) as run:
mcp_server._run_engine("adopt", {argument: True})
self.assertEqual(run.call_args.args[0][-1], flag)
def test_adopt_rejects_non_array_skills_without_spawning(self):
with mock.patch.object(mcp_server.subprocess, "run") as run:
with self.assertRaisesRegex(ValueError, "skills must be an array"):
mcp_server._run_engine("adopt", {"skills": "alpha"})
run.assert_not_called()
class TestMcpRuntimeValidation(unittest.TestCase):
def test_malformed_request_envelopes_return_json_rpc_errors(self):
cases = (
([], "request must be a JSON object"),
({"method": "ping", "id": 1}, "jsonrpc must be '2.0'"),
({"jsonrpc": "2.0", "id": 1, "method": "", "params": {}},
"method must be a non-empty string"),
({"jsonrpc": "2.0", "id": 1, "method": "ping", "params": []},
"params must be an object"),
({"jsonrpc": "2.0", "id": False, "method": "ping"},
"id must be a string"),
({"jsonrpc": "2.0", "id": 1, "method": "ping", "extra": 1},
"unknown request member"),
)
for request, message in cases:
with self.subTest(request=request):
response = mcp_server.handle(request)
self.assertEqual(response["error"]["code"], -32600)
self.assertIn(message, response["error"]["message"])
if type(request) is dict and type(request.get("id")) not in {str, int}:
self.assertIsNone(response["id"])
def test_known_method_params_reject_unknown_or_wrong_typed_properties(self):
cases = (
(_call(extra="value"), "unknown params member"),
({"jsonrpc": "2.0", "id": 1, "method": "tools/list",
"params": {"cursor": 4}}, "cursor must be a string"),
({"jsonrpc": "2.0", "id": 1, "method": "ping",
"params": {"_meta": "bad"}}, "_meta must be an object"),
)
for request, message in cases:
with self.subTest(request=request):
response = mcp_server.handle(request)
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
def test_invalid_tool_arguments_never_spawn(self):
cases = (
([], "arguments must be an object"),
({"bogus": 1}, "unknown argument"),
({"json": "false"}, "json must be a boolean"),
({"auto_adopt": "false"}, "auto_adopt must be a boolean"),
({"max_tasks": True}, "max_tasks must be an integer"),
({"max_tasks": -1}, "max_tasks must be between"),
({"backend": "other"}, "unsupported backend"),
({"skills": ["alpha"]}, "valid only for sleep_adopt"),
({"hour": 24}, "hour must be between"),
({"minute": -1}, "minute must be between"),
)
for arguments, message in cases:
request = _call(arguments=arguments)
if "hour" in arguments or "minute" in arguments:
request = _call("sleep_schedule", arguments)
with self.subTest(arguments=arguments), mock.patch.object(
mcp_server.subprocess, "run"
) as run:
response = mcp_server.handle(request)
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
run.assert_not_called()
def test_adoption_modes_and_skill_array_are_strict(self):
cases = (
({"all_skills": "false"}, "all_skills must be a boolean"),
({"legacy": "false"}, "legacy must be a boolean"),
({"skills": [1]}, "skills entry must be a string"),
({"skills": [" "]}, "skills entry must be non-empty"),
({"skills": ["alpha", " alpha "]}, "skills entries must be unique"),
({"skills": ["alpha"], "all_skills": True}, "choose at most one"),
({"all_skills": True, "legacy": True}, "choose at most one"),
)
for arguments, message in cases:
with self.subTest(arguments=arguments), mock.patch.object(
mcp_server.subprocess, "run"
) as run:
response = mcp_server.handle(_call("sleep_adopt", arguments))
self.assertEqual(response["error"]["code"], -32602)
self.assertIn(message, response["error"]["message"])
run.assert_not_called()
def test_every_string_boolean_and_integer_contract_is_exact_and_bounded(self):
for key in mcp_server._STRING_ARGS:
action = "adopt" if key == "staging" else "status"
with self.subTest(kind="string", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be a string"
):
mcp_server._validate_tool_arguments(action, {key: 1})
for key in mcp_server._BOOLEAN_ARGS:
action = "adopt" if key in {"all_skills", "legacy"} else "status"
with self.subTest(kind="boolean", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be a boolean"
):
mcp_server._validate_tool_arguments(action, {key: "false"})
for key, (minimum, maximum) in mcp_server._INTEGER_BOUNDS.items():
action = "schedule" if key in {"hour", "minute"} else "status"
with self.subTest(kind="integer-bool", key=key), self.assertRaisesRegex(
ValueError, f"{key} must be an integer"
):
mcp_server._validate_tool_arguments(action, {key: False})
for value in (minimum - 1, maximum + 1):
with self.subTest(kind="integer-bound", key=key, value=value), \
self.assertRaisesRegex(ValueError, f"{key} must be between"):
mcp_server._validate_tool_arguments(action, {key: value})
self.assertEqual(
mcp_server._validate_tool_arguments(action, {key: minimum})[key],
minimum,
)
self.assertEqual(
mcp_server._validate_tool_arguments(action, {key: maximum})[key],
maximum,
)
def test_engine_status_maps_to_mcp_error_and_handoff_states(self):
cases = (
(0, False, "ok"),
(2, True, "error"),
(3, False, "handoff_pending"),
)
for returncode, is_error, status in cases:
with self.subTest(returncode=returncode), mock.patch.object(
mcp_server, "_run_engine",
return_value=mcp_server.EngineResult("engine output", returncode),
):
response = mcp_server.handle(_call())
result = response["result"]
self.assertIs(result["isError"], is_error)
self.assertEqual(result["structuredContent"]["status"], status)
self.assertEqual(result["structuredContent"]["exit_code"], returncode)
def test_subprocess_exit_status_reaches_mcp_result(self):
completed = subprocess.CompletedProcess(
args=[], returncode=9, stdout="failed", stderr="details"
)
with mock.patch.object(mcp_server.subprocess, "run", return_value=completed):
result = mcp_server.handle(_call())["result"]
self.assertTrue(result["isError"])
self.assertEqual(result["structuredContent"]["exit_code"], 9)
def test_json_failure_keeps_stderr_visible_when_stdout_is_empty(self):
completed = subprocess.CompletedProcess(
args=[], returncode=4, stdout="", stderr="actionable failure\n"
)
with mock.patch.object(mcp_server.subprocess, "run", return_value=completed):
result = mcp_server.handle(_call(arguments={"json": True}))["result"]
self.assertTrue(result["isError"])
self.assertEqual(result["content"][0]["text"], "actionable failure")
def test_json_output_is_parseable_and_stderr_is_diagnostic_only(self):
completed = subprocess.CompletedProcess(
args=[], returncode=0, stdout='{"ok": true}\n', stderr="provider warning\n"
)
with mock.patch.object(mcp_server.subprocess, "run", return_value=completed):
run = mcp_server._run_engine("status", {"json": True})
self.assertEqual(json.loads(run.text), {"ok": True})
self.assertEqual(run.diagnostics, "provider warning")
self.assertNotIn("stderr", run.text)
def test_json_tool_result_includes_parsed_structured_output(self):
completed = subprocess.CompletedProcess(
args=[], returncode=0, stdout='{"nights": 2}\n', stderr=""
)
with mock.patch.object(mcp_server.subprocess, "run", return_value=completed):
result = mcp_server.handle(_call(arguments={"json": True}))["result"]
self.assertEqual(result["structuredContent"]["output"], {"nights": 2})
self.assertEqual(json.loads(result["content"][0]["text"]), {"nights": 2})
def test_main_emits_parse_error_for_malformed_json(self):
output = io.StringIO()
with mock.patch.object(sys, "stdin", io.StringIO("{bad json\n")), \
contextlib.redirect_stdout(output):
self.assertEqual(mcp_server.main(), 0)
response = json.loads(output.getvalue())
self.assertEqual(response["error"]["code"], -32700)
self.assertIsNone(response["id"])
if __name__ == "__main__":
unittest.main()
tests/test_plugin_sync.py
+17
View File
@@ -18,7 +18,10 @@ PLUGIN_SKILL_MDS = {
}
MCP_SERVER = os.path.join(REPO, "plugins/copilot/mcp_server.py")
COPILOT_README = os.path.join(REPO, "plugins/copilot/README.md")
COPILOT_INSTRUCTIONS = os.path.join(REPO, "plugins/copilot/copilot-instructions.snippet.md")
DEVIN_README = os.path.join(REPO, "plugins/devin/README.md")
DEVIN_RULES = os.path.join(REPO, "plugins/devin/devin-rules.snippet.md")
CANONICAL_BACKENDS = {"mock", "claude", "codex", "copilot"}
CURSOR_MANIFEST = os.path.join(REPO, "plugins/cursor/.cursor-plugin/plugin.json")
@@ -202,10 +205,24 @@ assert backend.name == "mock", backend.name
def test_mcp_schema_has_key_params(self):
text = _read(MCP_SERVER)
for param in ["source", "tasks_file", "target_skill_path",
"staging", "skills", "all_skills", "legacy",
"max_sessions", "max_tasks", "auto_adopt", "json"]:
self.assertIn(f'"{param}"', text,
f"MCP schema missing param '{param}'")
def test_mcp_adoption_docs_cover_fanout_selection(self):
for path in (COPILOT_README, COPILOT_INSTRUCTIONS, DEVIN_README, DEVIN_RULES):
text = _read(path)
for param in ("staging", "skills", "all_skills", "legacy"):
self.assertIn(f"`{param}`", text, f"{path} missing `{param}`")
def test_devin_docs_disclaim_post_adoption_copy(self):
for path in (DEVIN_README, DEVIN_RULES):
text = _read(path).lower()
self.assertIn("no post-adoption copy", text)
self.assertIn("target_skill_path", text)
self.assertIn("core engine", text)
def test_all_skill_mds_mention_memory_consolidation(self):
for name, path in PLUGIN_SKILL_MDS.items():
text = _read(path).lower()
tests/test_scheduler_windows.py
+9 -10
View File
@@ -1,14 +1,13 @@
import os
import sys
import unittest
from unittest import mock
class TestSchedulerWindows(unittest.TestCase):
@mock.patch("sys.platform", "win32")
@mock.patch("shutil.which", return_value="C:\\Windows\\System32\\schtasks.exe")
def test_schedule_windows(self, mock_which):
from skillopt_sleep.scheduler import schedule
calls = []
def fake_run(cmd, **kwargs):
calls.append(cmd)
@@ -17,7 +16,7 @@ class TestSchedulerWindows(unittest.TestCase):
stdout = "SUCCESS: The scheduled task ... has successfully been created."
stderr = ""
return Proc()
mock_open = mock.mock_open()
with mock.patch("subprocess.run", side_effect=fake_run), \
mock.patch("os.makedirs") as mock_makedirs, \
@@ -31,9 +30,9 @@ class TestSchedulerWindows(unittest.TestCase):
self.assertEqual(cmd[1], "/create")
self.assertEqual(cmd[2], "/tn")
self.assertTrue(cmd[3].startswith("SkillOpt-Sleep-"))
self.assertIn("my_project", cmd[3])
self.assertRegex(cmd[3], r"^SkillOpt-Sleep-[0-9a-f]{20}$")
self.assertEqual(cmd[4], "/tr")
self.assertIn("run.cmd", cmd[5])
self.assertIn("run.ps1", cmd[5])
self.assertEqual(cmd[6], "/sc")
self.assertEqual(cmd[7], "daily")
self.assertEqual(cmd[8], "/st")
@@ -45,14 +44,14 @@ class TestSchedulerWindows(unittest.TestCase):
# Verify the content written to the helper script
handle = mock_open()
written = "".join(call[0][0] for call in handle.write.call_args_list)
self.assertIn("@echo off", written)
self.assertIn("run --project", written)
self.assertIn("Set-Location -LiteralPath", written)
self.assertIn("'run' '--project'", written)
@mock.patch("sys.platform", "win32")
@mock.patch("shutil.which", return_value="C:\\Windows\\System32\\schtasks.exe")
def test_unschedule_windows(self, mock_which):
from skillopt_sleep.scheduler import unschedule
calls = []
def fake_run(cmd, **kwargs):
calls.append(cmd)
@@ -61,7 +60,7 @@ class TestSchedulerWindows(unittest.TestCase):
stdout = "SUCCESS: The scheduled task ... was successfully deleted."
stderr = ""
return Proc()
with mock.patch("subprocess.run", side_effect=fake_run), \
mock.patch("os.path.exists", return_value=True), \
mock.patch("os.remove") as mock_remove:
tests/test_sleep_adopt_skill_subset.py
+2193
View File
File diff suppressed because it is too large Load Diff
tests/test_sleep_engine.py
+551 -14
View File
@@ -6,6 +6,7 @@ Run: python3.12 -m pytest tests/test_sleep_engine.py
"""
from __future__ import annotations
import hashlib
import json
import os
import tempfile
@@ -585,6 +586,12 @@ class TestHarvest(unittest.TestCase):
project="/p",
edits=[EditRecord("skill", "add", "accepted rule")],
rejected_edits=[EditRecord("skill", "add", "rejected rule")],
skill_groups=[SkillGroupReport(
skill_name="research-skill",
status="consolidated",
accepted=True,
n_tasks=3,
)],
)
outcome = type("Outcome", (), {"staging_dir": "", "adopted": False})()
@@ -593,6 +600,11 @@ class TestHarvest(unittest.TestCase):
self.assertEqual(payload["n_accepted_edits"], 1)
self.assertEqual(payload["n_rejected_edits"], 1)
self.assertEqual(payload["rejected_edits"][0]["content"], "rejected rule")
self.assertEqual(
payload["skill_groups"][0]["skill_name"],
"research-skill",
)
self.assertEqual(payload["staged_skills"], [])
def test_tasks_file_roundtrip_and_split_assignment(self):
from skillopt_sleep.tasks_file import load_tasks_file, make_tasks_payload, write_tasks_file
@@ -1203,7 +1215,7 @@ class TestCodexBackend(unittest.TestCase):
return d
with mock.patch("tempfile.mkdtemp", side_effect=fake_mkdtemp):
be.attempt_with_tools(task, "", "", ["search"])
self.assertEqual(len(temp_dirs), 1)
work_dir = temp_dirs[0]
shim_path = os.path.join(work_dir, "search.cmd")
@@ -1381,6 +1393,11 @@ class TestFullCycleAndAdopt(unittest.TestCase):
with open(manifest_path, encoding="utf-8") as f:
manifest = json.load(f)
self.assertEqual(manifest["live_skill_path"], target)
self.assertEqual(manifest["legacy"]["skill"]["live_sha256"], "")
self.assertEqual(
manifest["legacy"]["skill"]["live_realpath"],
os.path.realpath(target),
)
self.assertFalse(os.path.exists(target))
updated = adopt(outcome.staging_dir)
@@ -1388,6 +1405,147 @@ class TestFullCycleAndAdopt(unittest.TestCase):
self.assertIn(target, updated)
self.assertTrue(os.path.exists(target))
def test_cycle_pins_the_exact_managed_skill_and_memory_bytes_it_read(self):
from skillopt_sleep.consolidate import ConsolidationResult
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
target = os.path.join(proj, ".agents", "skills", "taste", "SKILL.md")
memory_path = os.path.join(proj, "CLAUDE.md")
os.makedirs(os.path.dirname(target), exist_ok=True)
skill_bytes = b"# managed baseline\nrule\n"
memory_bytes = b"# memory baseline\npreference\n"
with open(target, "wb") as handle:
handle.write(skill_bytes)
with open(memory_path, "wb") as handle:
handle.write(memory_bytes)
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=os.path.join(home, ".claude"),
target_skill_path=target,
auto_adopt=False,
)
tasks = assign_splits(
researcher_persona(),
holdout_fraction=0.34,
seed=42,
)
result = ConsolidationResult(
accepted=True,
gate_action="accept_new_best",
baseline_score=0.1,
candidate_score=0.2,
new_skill="# managed proposal\n",
new_memory="# memory proposal\n",
applied_edits=[],
rejected_edits=[],
holdout_baseline=0.1,
holdout_candidate=0.2,
)
with mock.patch(
"skillopt_sleep.cycle.dream_consolidate",
return_value=result,
):
outcome = run_sleep_cycle(cfg, seed_tasks=tasks)
with open(
os.path.join(outcome.staging_dir, "manifest.json"),
encoding="utf-8",
) as handle:
manifest = json.load(handle)
skill_row = manifest["legacy"]["skill"]
memory_row = manifest["legacy"]["memory"]
self.assertEqual(
skill_row["live_sha256"],
hashlib.sha256(skill_bytes).hexdigest(),
)
self.assertEqual(
memory_row["live_sha256"],
hashlib.sha256(memory_bytes).hexdigest(),
)
self.assertEqual(skill_row["live_realpath"], os.path.realpath(target))
self.assertEqual(
memory_row["live_realpath"],
os.path.realpath(memory_path),
)
def test_managed_skill_change_during_consolidation_refuses_the_night(self):
from skillopt_sleep.consolidate import ConsolidationResult
from skillopt_sleep.staging import StagingError, latest_staging
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
target = os.path.join(proj, ".agents", "skills", "taste", "SKILL.md")
os.makedirs(os.path.dirname(target), exist_ok=True)
with open(target, "w", encoding="utf-8") as handle:
handle.write("# baseline v1\n")
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=os.path.join(home, ".claude"),
target_skill_path=target,
auto_adopt=False,
)
tasks = assign_splits(
researcher_persona(),
holdout_fraction=0.34,
seed=42,
)
result = ConsolidationResult(
accepted=True,
gate_action="accept_new_best",
baseline_score=0.1,
candidate_score=0.2,
new_skill="# proposal derived from v1\n",
new_memory="",
applied_edits=[],
rejected_edits=[],
holdout_baseline=0.1,
holdout_candidate=0.2,
)
def _edit_live_after_read(*args, **kwargs):
with open(target, "w", encoding="utf-8") as handle:
handle.write("# concurrent human edit\n")
return result
with mock.patch(
"skillopt_sleep.cycle.dream_consolidate",
side_effect=_edit_live_after_read,
), self.assertRaisesRegex(StagingError, "changed during consolidation"):
run_sleep_cycle(cfg, seed_tasks=tasks)
with open(target, encoding="utf-8") as handle:
self.assertEqual(handle.read(), "# concurrent human edit\n")
self.assertIsNone(latest_staging(proj))
def test_invalid_utf8_managed_skill_is_not_treated_as_an_empty_baseline(self):
from skillopt_sleep.staging import StagingError
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
target = os.path.join(proj, ".agents", "skills", "taste", "SKILL.md")
os.makedirs(os.path.dirname(target), exist_ok=True)
with open(target, "wb") as handle:
handle.write(b"\xff\xfe\x00not-utf8")
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=os.path.join(home, ".claude"),
target_skill_path=target,
auto_adopt=False,
)
tasks = assign_splits(
researcher_persona(),
holdout_fraction=0.34,
seed=42,
)
with self.assertRaisesRegex(StagingError, "not valid UTF-8"):
run_sleep_cycle(cfg, seed_tasks=tasks)
class TestCopilotBackend(unittest.TestCase):
"""Pure-logic tests for CopilotCliBackend — no `copilot` CLI required."""
@@ -1940,10 +2098,30 @@ class TestCursorBackend(unittest.TestCase):
rc = main(["dry-run", "--project", project, "--backend", "cursor"])
self.assertEqual(rc, 1)
self.assertIn("Cursor backend failed", stderr.getvalue())
self.assertIn("backend failed", stderr.getvalue())
self.assertIn("[REDACTED]", stderr.getvalue())
self.assertNotIn("cursor-secret", stderr.getvalue())
def test_run_json_failure_is_one_redacted_document(self):
import contextlib
import io
from skillopt_sleep.__main__ import main
from skillopt_sleep.staging import StagingError
stdout = io.StringIO()
with mock.patch(
"skillopt_sleep.__main__.run_sleep_cycle",
side_effect=StagingError("api_key=SUPERSECRET123456789\x1b[31m"),
), contextlib.redirect_stdout(stdout):
rc = main(["run", "--json", "--project", tempfile.gettempdir()])
self.assertEqual(rc, 1)
payload = json.loads(stdout.getvalue())
self.assertEqual(payload["ok"], False)
self.assertEqual(payload["error"], "staging_refused")
self.assertNotIn("SUPERSECRET", payload["message"])
self.assertNotIn("\x1b", payload["message"])
class TestClaudeCliBackendBare(unittest.TestCase):
"""Issue #68: --bare must be conditional on ANTHROPIC_API_KEY."""
@@ -2048,13 +2226,13 @@ class MockRewardHackingBackend(MockBackend):
return str(task.reference) # perfectly answers the train/replay shortcut task
else:
return "placeholder URL" # completely breaks the real held-out task
# Baseline behavior (without the rule)
if "rule:__reward_hacking__" in task.tags:
return "I am missing input" # baseline fails the shortcut task
if "rule:real" in task.tags:
return str(task.reference) # baseline gets the real task right
return super().attempt(task, skill, memory, sample_id)
def reflect(self, failures, successes, skill, memory, **kwargs):
@@ -2074,13 +2252,13 @@ class MockBeneficialBackend(MockBackend):
return str(task.reference) # improves the train task
if "rule:real" in task.tags:
return str(task.reference) # improves the real held-out task
# Baseline behavior (without the rule)
if "rule:__beneficial__" in task.tags:
return "I am missing input" # baseline fails the train task
if "rule:real" in task.tags:
return "baseline fails too" # baseline fails the real task
return super().attempt(task, skill, memory, sample_id)
def reflect(self, failures, successes, skill, memory, **kwargs):
@@ -2138,7 +2316,7 @@ class TestVerifierDiscipline(unittest.TestCase):
tasks = [train_task, val_task]
res = consolidate(be, tasks, "", "", edit_budget=4, gate_metric="hard", night=1)
self.assertFalse(res.accepted)
self.assertEqual(res.gate_action, "reject")
self.assertEqual(res.holdout_baseline, 1.0)
@@ -2153,7 +2331,7 @@ class TestVerifierDiscipline(unittest.TestCase):
tasks = [train_task, val_task]
res = consolidate(be, tasks, "", "", edit_budget=4, gate_metric="hard", night=1)
self.assertTrue(res.accepted)
self.assertEqual(res.gate_action, "accept_new_best")
self.assertEqual(res.holdout_baseline, 0.0)
@@ -2264,6 +2442,7 @@ class TestDiagnosticsRedaction(unittest.TestCase):
"""End-to-end: a codex-style 401 stderr captured in call_error must not
reach diagnostics.json verbatim once written to the staging dir."""
import json
from skillopt_sleep.staging import redact_secrets
# Mirror exactly what cycle.py writes (the fields that carry free text).
secret_stderr = (
@@ -2294,7 +2473,6 @@ class TestDiagnosticsRedaction(unittest.TestCase):
def test_codex_auth_error_log_is_redacted(self):
"""The codex auth-error log line (a secondary on-disk sink when a file
log handler is attached) must not emit the raw stderr token verbatim."""
import logging
from skillopt_sleep.backend import CodexCliBackend
be = CodexCliBackend.__new__(CodexCliBackend) # no __init__ side effects
be.timeout = 1
@@ -2419,6 +2597,13 @@ if __name__ == "__main__":
class TestMultiSkillReportWiring(unittest.TestCase):
"""The per-skill rows reach the emitted report (issue #120 follow-up)."""
def _write_live_skills(self, claude_home, *names):
for name in names:
path = os.path.join(claude_home, "skills", name, "SKILL.md")
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8") as handle:
handle.write(f"# {name}\n")
def _hinted_tasks(self):
# Two skills' worth of evidence in one night. dataclasses.replace keeps
# the personas' real task shape rather than inventing a fixture.
@@ -2441,11 +2626,31 @@ class TestMultiSkillReportWiring(unittest.TestCase):
# Opt-in: hinted evidence alone must not add rows or extra calls.
self.assertEqual(outcome.report.skill_groups, [])
def test_multi_skill_fanout_is_the_canonical_flag(self):
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(claude_home, "research-skill")
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned",
multi_skill_fanout=True,
multi_skill_report=False,
)
outcome = run_sleep_cycle(cfg, seed_tasks=self._hinted_tasks())
self.assertTrue(outcome.report.skill_groups)
def test_a_mixed_night_emits_one_independent_row_per_skill(self):
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home, "research-skill", "programming-skill"
)
cfg = load_config(
invoked_project=proj, projects="invoked", backend="mock",
claude_home=os.path.join(home, ".claude"),
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned", auto_adopt=False,
multi_skill_report=True,
)
@@ -2475,9 +2680,11 @@ class TestMultiSkillReportWiring(unittest.TestCase):
if task.skill_hint == "research-skill"
]
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(claude_home, "research-skill")
cfg = load_config(
invoked_project=proj, projects="invoked", backend="mock",
claude_home=os.path.join(home, ".claude"),
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned", auto_adopt=False,
multi_skill_report=True,
)
@@ -2529,9 +2736,16 @@ class TestMultiSkillReportWiring(unittest.TestCase):
tasks = self._hinted_tasks()
tasks.append(replace(tasks[0], id="thin-1", skill_hint="thin-skill"))
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home,
"research-skill",
"programming-skill",
"thin-skill",
)
cfg = load_config(
invoked_project=proj, projects="invoked", backend="mock",
claude_home=os.path.join(home, ".claude"),
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned", auto_adopt=False,
multi_skill_report=True,
)
@@ -2553,9 +2767,16 @@ class TestMultiSkillReportWiring(unittest.TestCase):
tasks = self._hinted_tasks()
tasks.append(replace(tasks[0], id="thin-1", skill_hint="thin-skill"))
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home,
"research-skill",
"programming-skill",
"thin-skill",
)
cfg = load_config(
invoked_project=proj, projects="invoked", backend="mock",
claude_home=os.path.join(home, ".claude"),
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned", auto_adopt=False,
multi_skill_report=True,
)
@@ -2591,6 +2812,35 @@ class TestMultiSkillReportWiring(unittest.TestCase):
self.assertIn("skill&#124;&#96;one next", row)
self.assertIn("backend &#96;bad&#96; line &#124; broken", row)
def test_report_md_sanitizes_every_untrusted_prose_field(self):
hostile = "first\n## forged <script>x</script> \x1b[31m\u202e [link](javascript:x)"
edit = EditRecord(
target=hostile,
op=hostile,
content=hostile,
anchor=hostile,
rationale=hostile,
)
report = SleepReport(
night=1,
project=hostile,
gate_action=hostile,
edits=[edit],
rejected_edits=[edit],
unmatched_edits=[edit],
notes=[hostile],
)
md = _render_report_md(
report,
{"backend": hostile, "replay_mode": hostile},
)
self.assertNotIn("\x1b", md)
self.assertNotIn("\u202e", md)
self.assertNotIn("<script>", md)
self.assertNotIn("\n## forged", md)
self.assertNotIn("[link](javascript:x)", md)
self.assertIn("&lt;script&gt;x&lt;/script&gt;", md)
def test_report_md_has_no_group_section_when_the_feature_is_off(self):
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
cfg = load_config(
@@ -2604,10 +2854,16 @@ class TestMultiSkillReportWiring(unittest.TestCase):
self.assertNotIn("## Per-skill groups", handle.read())
def test_group_rows_survive_into_the_staged_report_json(self):
from skillopt_sleep.__main__ import _report_payload
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home, "research-skill", "programming-skill"
)
cfg = load_config(
invoked_project=proj, projects="invoked", backend="mock",
claude_home=os.path.join(home, ".claude"),
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned", auto_adopt=False,
multi_skill_report=True,
)
@@ -2620,3 +2876,284 @@ class TestMultiSkillReportWiring(unittest.TestCase):
self.assertTrue(payload.get("skill_groups"))
self.assertIn(payload["skill_groups"][0]["skill_name"],
{"research-skill", "programming-skill"})
cli_payload = _report_payload(outcome.report, outcome)
self.assertEqual(
set(cli_payload["staged_skills"]),
{"research-skill", "programming-skill"},
)
self.assertEqual(
{row["skill_name"] for row in cli_payload["skill_groups"]},
{"research-skill", "programming-skill"},
)
def test_group_runs_inherit_dream_gate_budget_and_scoped_recall_config(self):
"""The fan-out must be the configured dream pipeline, not a side path."""
from dataclasses import replace
from skillopt_sleep.dream import dream_consolidate as real_dream_consolidate
from skillopt_sleep.state import SleepState
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home,
"research-skill",
"programming-skill",
)
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned",
auto_adopt=False,
multi_skill_report=True,
recall_k=7,
dream_rollouts=3,
dream_factor=2,
edit_budget=6,
gate_metric="mixed",
gate_mixed_weight=0.37,
gate_no_regression=True,
gate_mode="off",
evolve_skill=True,
evolve_memory=True,
)
tonight = self._hinted_tasks()
research_history = replace(
tonight[0],
id="history-research",
skill_hint="research-skill",
)
programming_seed = next(
task for task in tonight
if task.skill_hint == "programming-skill"
)
programming_history = replace(
programming_seed,
id="history-programming",
skill_hint="programming-skill",
)
state = SleepState.load(cfg.state_path)
state.add_to_archive([
research_history.to_dict(),
programming_history.to_dict(),
])
state.save()
calls = []
def _spy(backend, tasks, skill, memory, **kwargs):
calls.append({
"hints": {task.skill_hint for task in tasks},
"kwargs": dict(kwargs),
})
# Keep this regression fast while preserving the exact
# arguments observed at the public orchestration boundary.
safe = dict(kwargs)
safe.update(recall_k=0, dream_rollouts=1, dream_factor=0)
return real_dream_consolidate(
backend,
tasks,
skill,
memory,
**safe,
)
with mock.patch(
"skillopt_sleep.cycle.dream_consolidate",
side_effect=_spy,
):
run_sleep_cycle(cfg, seed_tasks=tonight)
self.assertEqual(len(calls), 3)
aggregate = calls[0]
group_calls = calls[1:]
self.assertEqual(
aggregate["hints"],
{"research-skill", "programming-skill"},
)
self.assertTrue(aggregate["kwargs"]["evolve_memory"])
self.assertEqual(
{
task.skill_hint
for task in aggregate["kwargs"]["history_tasks"]
},
{"research-skill", "programming-skill"},
)
expected = {
"recall_k": 7,
"dream_rollouts": 3,
"dream_factor": 2,
"edit_budget": 6,
"gate_metric": "mixed",
"gate_mixed_weight": 0.37,
"gate_no_regression": True,
"gate_mode": "off",
"evolve_skill": True,
}
self.assertEqual(
{next(iter(call["hints"])) for call in group_calls},
{"research-skill", "programming-skill"},
)
for call in group_calls:
kwargs = call["kwargs"]
for key, value in expected.items():
self.assertEqual(kwargs[key], value, key)
self.assertFalse(kwargs["evolve_memory"])
hint = next(iter(call["hints"]))
self.assertEqual(
{task.skill_hint for task in kwargs["history_tasks"]},
{hint},
)
def test_managed_and_fanout_proposals_never_target_the_same_live_file(self):
from skillopt_sleep.staging import staged_skills
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
research_live = os.path.join(
claude_home,
"skills",
"research-skill",
"SKILL.md",
)
programming_live = os.path.join(
claude_home,
"skills",
"programming-skill",
"SKILL.md",
)
os.makedirs(os.path.dirname(research_live), exist_ok=True)
os.makedirs(os.path.dirname(programming_live), exist_ok=True)
with open(research_live, "w", encoding="utf-8") as handle:
handle.write("# research baseline\n")
with open(programming_live, "w", encoding="utf-8") as handle:
handle.write("# programming baseline\n")
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=claude_home,
target_skill_path=research_live,
managed_skill_name="skillopt-sleep-learned",
auto_adopt=False,
multi_skill_report=True,
gate_mode="off",
)
outcome = run_sleep_cycle(cfg, seed_tasks=self._hinted_tasks())
self.assertEqual(
[row["skill_name"] for row in staged_skills(outcome.staging_dir)],
["programming-skill"],
)
self.assertTrue(any(
"research-skill" in note
and "same live target as the managed skill" in note
for note in outcome.report.notes
))
with open(
os.path.join(outcome.staging_dir, "manifest.json"),
encoding="utf-8",
) as handle:
manifest = json.load(handle)
self.assertFalse(manifest["has_skill"])
self.assertTrue(manifest["has_managed_skill"])
self.assertEqual(
manifest["legacy"]["skill"]["live_realpath"],
os.path.realpath(research_live),
)
def test_single_contained_symlink_alias_is_reported_but_not_staged(self):
from dataclasses import replace
from skillopt_sleep.staging import staged_skills
with tempfile.TemporaryDirectory() as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
skills_root = os.path.join(claude_home, "skills")
real_dir = os.path.join(skills_root, "real-skill")
alias_dir = os.path.join(skills_root, "alias-skill")
os.makedirs(real_dir, exist_ok=True)
with open(
os.path.join(real_dir, "SKILL.md"),
"w",
encoding="utf-8",
) as handle:
handle.write("# real baseline\n")
try:
os.symlink(real_dir, alias_dir)
except OSError:
self.skipTest("symlinks unavailable")
tasks = [
replace(task, skill_hint="alias-skill")
for task in assign_splits(
researcher_persona(),
holdout_fraction=0.34,
seed=42,
)
]
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned",
auto_adopt=False,
multi_skill_report=True,
gate_mode="off",
)
outcome = run_sleep_cycle(cfg, seed_tasks=tasks)
self.assertEqual(staged_skills(outcome.staging_dir), [])
self.assertTrue(any(
"alias-skill" in note
and "is not alias-skill/SKILL.md" in note
for note in outcome.report.notes
))
with open(
os.path.join(outcome.staging_dir, "report.md"),
encoding="utf-8",
) as handle:
self.assertIn("alias-skill", handle.read())
def test_group_control_flow_exceptions_never_publish_or_advance_a_night(self):
from skillopt_sleep.backend import CursorBackendError
from skillopt_sleep.handoff_backend import PendingCalls
from skillopt_sleep.staging import latest_staging
cases = (
PendingCalls({"group-call": {"prompt": "continue", "max_tokens": 20}}),
CursorBackendError("group backend authentication failed"),
)
for failure in cases:
with self.subTest(failure=type(failure).__name__), (
tempfile.TemporaryDirectory()
) as proj, tempfile.TemporaryDirectory() as home:
claude_home = os.path.join(home, ".claude")
self._write_live_skills(
claude_home,
"research-skill",
"programming-skill",
)
cfg = load_config(
invoked_project=proj,
projects="invoked",
backend="mock",
claude_home=claude_home,
managed_skill_name="skillopt-sleep-learned",
auto_adopt=False,
multi_skill_report=True,
)
with mock.patch(
"skillopt_sleep.cycle.consolidate_groups",
side_effect=failure,
), self.assertRaises(type(failure)):
run_sleep_cycle(cfg, seed_tasks=self._hinted_tasks())
self.assertIsNone(latest_staging(proj))
self.assertFalse(os.path.exists(cfg.state_path))
tests/test_sleep_multi_skill.py
+95
View File
@@ -147,6 +147,35 @@ class TestConsolidateGroups(unittest.TestCase):
self.assertEqual(outcomes["research-skill"].status, CONSOLIDATED)
self.assertTrue(outcomes["research-skill"].accepted)
def test_pending_handoff_calls_propagate_instead_of_becoming_failed_rows(self):
from skillopt_sleep.handoff_backend import PendingCalls
pending = {"prompt-id": {"prompt": "answer me", "max_tokens": 20}}
def _pending(*args, **kwargs):
raise PendingCalls(pending)
with self.assertRaises(PendingCalls) as caught:
consolidate_groups(
MockBackend(),
[SkillGroup("research-skill", "# research\n", _tasks(researcher_persona))],
consolidate_fn=_pending,
)
self.assertEqual(caught.exception.pending, pending)
def test_fatal_cursor_error_propagates_instead_of_becoming_a_failed_row(self):
from skillopt_sleep.backend import CursorBackendError
def _fatal(*args, **kwargs):
raise CursorBackendError("authentication failed")
with self.assertRaises(CursorBackendError):
consolidate_groups(
MockBackend(),
[SkillGroup("research-skill", "# research\n", _tasks(researcher_persona))],
consolidate_fn=_fatal,
)
def test_group_runs_do_not_evolve_shared_memory(self):
seen = {}
@@ -178,6 +207,72 @@ class TestConsolidateGroups(unittest.TestCase):
self.assertEqual(outcomes["research-skill"].status, CONSOLIDATED)
self.assertFalse(seen["evolve_memory"])
def test_group_specific_dream_kwargs_preserve_skill_boundaries(self):
seen = {}
def _fake(backend, tasks, skill, memory, **kwargs):
seen[skill] = kwargs.pop("history_tasks")
return consolidate(backend, tasks, skill, memory, **kwargs)
histories = {
"# research\n": ["research-history"],
"# programming\n": ["programming-history"],
}
outcomes = consolidate_groups(
MockBackend(),
[
SkillGroup("research-skill", "# research\n", _tasks(researcher_persona)),
SkillGroup(
"programming-skill",
"# programming\n",
_tasks(programmer_persona, seed=1),
),
],
consolidate_fn=_fake,
group_kwargs_fn=lambda group: {
"history_tasks": histories[group.skill]
},
edit_budget=4,
night=1,
)
self.assertEqual(seen, histories)
self.assertEqual(outcomes["research-skill"].status, CONSOLIDATED)
self.assertEqual(outcomes["programming-skill"].status, CONSOLIDATED)
def test_group_kwargs_failure_is_isolated_like_a_group_backend_failure(self):
calls = []
def _kwargs(group):
if group.skill_name == "broken-skill":
raise ValueError("invalid recalled history")
return {"history_tasks": []}
def _fake(backend, tasks, skill, memory, **kwargs):
calls.append(skill)
kwargs.pop("history_tasks")
return consolidate(backend, tasks, skill, memory, **kwargs)
outcomes = consolidate_groups(
MockBackend(),
[
SkillGroup("broken-skill", "# broken\n", _tasks(researcher_persona)),
SkillGroup(
"research-skill",
set_learned("", []),
_tasks(researcher_persona),
),
],
consolidate_fn=_fake,
group_kwargs_fn=_kwargs,
edit_budget=4,
night=1,
)
self.assertEqual(outcomes["broken-skill"].status, FAILED)
self.assertIn("ValueError: invalid recalled history", outcomes["broken-skill"].reason)
self.assertEqual(outcomes["research-skill"].status, CONSOLIDATED)
self.assertEqual(len(calls), 1)
def test_accepted_group_skills_lists_only_accepted_updates(self):
outcomes = {
"kept": GroupConsolidation(
tests/test_sleep_scheduler_safety.py
+82
View File
@@ -0,0 +1,82 @@
"""Safety and portability checks for the built-in sleep scheduler."""
from __future__ import annotations
import os
import shlex
import tempfile
import unittest
from unittest import mock
from skillopt_sleep import scheduler
class TestSleepSchedulerSafety(unittest.TestCase):
def test_posix_runner_quotes_every_path_and_argument(self):
with tempfile.TemporaryDirectory(prefix="sleep $' quote ") as project:
command = scheduler._runner_cmd(
project,
"mock;not-a-command",
"--auto-adopt",
"/tmp/python with spaces",
)
self.assertIn(shlex.quote(project), command)
self.assertIn(shlex.quote("mock;not-a-command"), command)
self.assertIn(shlex.quote("/tmp/python with spaces"), command)
self.assertNotIn("$(", command)
def test_control_characters_are_refused_before_crontab_write(self):
with mock.patch.object(scheduler, "_write_crontab") as write:
ok, message = scheduler.schedule("/tmp/project\n* * * * * injected")
self.assertFalse(ok)
self.assertIn("control characters", message)
write.assert_not_called()
def test_backend_and_extra_control_characters_are_refused(self):
cases = (
("mock\n* * * * * injected", ""),
("mock", "--auto-adopt\n* * * * * injected"),
)
for backend, extra in cases:
with self.subTest(backend=backend, extra=extra):
with mock.patch.object(scheduler, "_write_crontab") as write:
ok, message = scheduler.schedule(
"/tmp/project", backend=backend, extra=extra
)
self.assertFalse(ok)
self.assertIn("control characters", message)
write.assert_not_called()
def test_project_marker_contains_only_a_digest(self):
project = "/tmp/project with secret api_key=SUPERSECRET123456789"
marker = scheduler._project_marker(project)
self.assertRegex(marker, r"^# project-sha256=[0-9a-f]{64}$")
self.assertNotIn("project with secret", marker)
def test_schedule_time_ranges_fail_closed(self):
for hour, minute in ((-1, 0), (24, 0), (0, -1), (0, 60)):
with self.subTest(hour=hour, minute=minute):
ok, _message = scheduler.schedule(
os.getcwd(), hour=hour, minute=minute
)
self.assertFalse(ok)
def test_windows_helper_uses_powershell_data_literals(self):
with tempfile.TemporaryDirectory(prefix="sleep ' percent% ") as project:
with mock.patch.object(scheduler.sys, "platform", "win32"):
command = scheduler._runner_cmd(
project,
"mock",
"--auto-adopt",
"C:\\Program Files\\Python\\python.exe",
)
helper = os.path.join(project, ".skillopt-sleep", "run.ps1")
with open(helper, encoding="utf-8") as handle:
content = handle.read()
self.assertIn("Set-Location -LiteralPath", content)
self.assertIn("''", content)
self.assertIn("-File", command)
self.assertNotIn("run.cmd", command)
if __name__ == "__main__":
unittest.main()
tests/test_sleep_skill_resolver.py
+36
View File
@@ -195,6 +195,42 @@ class TestSkillSearchRoots(unittest.TestCase):
for blank in ["", " ", None]:
self.assertEqual(skill_search_roots(_Cfg(blank)), [], repr(blank))
def test_project_native_agent_roots_and_explicit_roots_are_discovered(self):
with tempfile.TemporaryDirectory() as tmp:
expected = []
for relative in (
os.path.join(".agents", "skills"),
os.path.join(".claude", "skills"),
os.path.join(".cursor", "skills"),
os.path.join(".devin", "skills"),
"custom-skills",
):
root = os.path.join(tmp, relative)
os.makedirs(root)
expected.append(os.path.realpath(root))
cfg = load_config(
invoked_project=tmp,
claude_home="",
codex_home="",
cursor_home="",
skill_roots=["custom-skills"],
)
self.assertEqual(skill_search_roots(cfg), expected)
def test_native_roots_make_duplicate_agent_definitions_ambiguous(self):
with tempfile.TemporaryDirectory() as tmp:
for relative in (".agents/skills", ".devin/skills"):
_write_skill(os.path.join(tmp, relative), "shared-skill")
cfg = load_config(
invoked_project=tmp,
claude_home="",
codex_home="",
cursor_home="",
)
result = resolve_skill("shared-skill", skill_search_roots(cfg))
self.assertEqual(result.status, AMBIGUOUS)
self.assertEqual(len(result.candidates), 2)
def test_unreadable_plugin_cache_does_not_break_discovery(self):
with tempfile.TemporaryDirectory() as tmp:
claude_home = os.path.join(tmp, ".claude")
tests/test_sleep_staging_fanout.py
+378 -2
View File
@@ -5,14 +5,19 @@ Run: python -m pytest tests/test_sleep_staging_fanout.py
"""
from __future__ import annotations
import hashlib
import json
import os
import tempfile
import unittest
from concurrent.futures import ThreadPoolExecutor
from unittest import mock
from skillopt_sleep.staging import (
SkillProposal,
StagingError,
latest_staging,
new_staging_dir,
proposal_filename,
skill_proposal_rows,
write_skill_proposals,
@@ -40,6 +45,10 @@ class TestSkillProposalRows(unittest.TestCase):
["proposed_SKILL.alpha.md", "proposed_SKILL.beta.md"])
self.assertEqual(rows[0]["live_skill_path"],
os.path.normpath("/tmp/live/alpha/SKILL.md"))
self.assertEqual(
rows[0]["sha256"],
hashlib.sha256(b"# example\n").hexdigest(),
)
def test_filenames_are_unique_per_skill(self):
self.assertNotEqual(proposal_filename("alpha"), proposal_filename("beta"))
@@ -83,6 +92,16 @@ class TestSkillProposalRows(unittest.TestCase):
skill_proposal_rows([_proposal("alpha", live="/tmp/live/A.md"),
_proposal("beta", live="/tmp/live/a.md")])
def test_unicode_equivalent_staged_names_are_refused(self):
# HFS+/APFS commonly normalise filenames. NFC ``café`` and the
# decomposed NFD spelling must not produce two manifest rows for one
# physical staged file.
with self.assertRaises(StagingError):
skill_proposal_rows([
_proposal("café", live="/tmp/live/cafe-a/SKILL.md"),
_proposal("cafe\u0301", live="/tmp/live/cafe-b/SKILL.md"),
])
def test_a_generator_of_proposals_still_writes_every_file(self):
# The annotation says Sequence but nothing enforces it. Validation used
# to drain a generator, leaving the write loop empty and returning a
@@ -130,7 +149,8 @@ class TestSkillProposalRows(unittest.TestCase):
def test_unsafe_live_paths_are_refused(self):
for bad in ["", "relative/SKILL.md", "~/skills/a/SKILL.md",
"/tmp/live/../../etc/SKILL.md", "/tmp/live/a/SKILL.txt"]:
"/tmp/live/../../etc/SKILL.md", "/tmp/live/a/SKILL.txt",
"/tmp/live/a/skill\x00/SKILL.md", "/tmp/live/a\n/SKILL.md"]:
with self.assertRaises(StagingError, msg=bad):
skill_proposal_rows([_proposal("alpha", live=bad)])
@@ -147,6 +167,12 @@ class TestWriteSkillProposals(unittest.TestCase):
with open(os.path.join(tmp, rows[0]["proposed_file"]), encoding="utf-8") as f:
self.assertEqual(f.read(), "# alpha\n")
def test_empty_proposal_body_is_refused_without_writing(self):
with tempfile.TemporaryDirectory() as tmp:
with self.assertRaisesRegex(StagingError, "is empty"):
write_skill_proposals(tmp, [_proposal("alpha", " \n")])
self.assertEqual(os.listdir(tmp), [])
def test_no_proposals_writes_nothing(self):
with tempfile.TemporaryDirectory() as tmp:
self.assertEqual(write_skill_proposals(tmp, []), [])
@@ -203,7 +229,52 @@ class TestWriteStagingCompatibility(unittest.TestCase):
)
manifest = self._manifest(out)
self.assertNotIn("skills", manifest)
self.assertTrue(manifest["has_skill"])
self.assertEqual(manifest["schema"], "skillopt-sleep-staging")
self.assertEqual(manifest["schema_version"], 2)
self.assertFalse(manifest["has_skill"])
self.assertFalse(manifest["has_memory"])
self.assertTrue(manifest["has_managed_skill"])
self.assertTrue(manifest["has_managed_memory"])
def test_v020_compatibility_flags_select_no_unpinned_files(self):
with tempfile.TemporaryDirectory() as tmp:
out = write_staging(
tmp, report=_report(), proposed_skill="# skill\n",
proposed_memory="# memory\n",
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
)
manifest = self._manifest(out)
# This is the complete v0.2.0 mutation decision: it trusted only
# these top-level booleans, without consulting integrity pins.
selected = []
if manifest.get("has_skill"):
selected.append("proposed_SKILL.md")
if manifest.get("has_memory"):
selected.append("proposed_CLAUDE.md")
self.assertEqual(selected, [])
self.assertIn("skill", manifest["legacy"])
self.assertIn("memory", manifest["legacy"])
def test_unknown_manifest_schema_version_is_refused(self):
from skillopt_sleep.staging import staged_skills
with tempfile.TemporaryDirectory() as tmp:
out = write_staging(
tmp, report=_report(), proposed_skill=None, proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
skill_proposals=[_proposal("alpha", root=os.path.join(tmp, "live"))],
)
path = os.path.join(out, "manifest.json")
manifest = self._manifest(out)
manifest["schema_version"] = 999
with open(path, "w", encoding="utf-8") as handle:
json.dump(manifest, handle)
with self.assertRaisesRegex(StagingError, "unsupported schema"):
staged_skills(out)
def test_fan_out_adds_files_and_manifest_rows(self):
with tempfile.TemporaryDirectory() as tmp:
@@ -227,6 +298,311 @@ class TestWriteStagingCompatibility(unittest.TestCase):
self.assertEqual([r["skill_name"] for r in rows], ["alpha", "beta"])
self.assertEqual(rows[1]["live_skill_path"],
os.path.join(live_root, "beta", "SKILL.md"))
self.assertEqual(
rows[0]["sha256"],
hashlib.sha256(b"# alpha\n").hexdigest(),
)
self.assertEqual(rows[0]["live_sha256"], "")
self.assertEqual(
rows[0]["live_realpath"],
os.path.realpath(os.path.join(live_root, "alpha", "SKILL.md")),
)
def test_two_same_second_nights_get_distinct_reserved_directories(self):
with tempfile.TemporaryDirectory() as tmp, mock.patch(
"skillopt_sleep.staging._ts_dir", return_value="20260815-010203"
):
first = write_staging(
tmp, report=_report(), proposed_skill="# first\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# first report\n",
)
second = write_staging(
tmp, report=_report(), proposed_skill="# second\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# second report\n",
)
self.assertNotEqual(first, second)
self.assertEqual(
sorted((os.path.basename(first), os.path.basename(second))),
["20260815-010203", "20260815-010203-2"],
)
with open(os.path.join(first, "report.md"), encoding="utf-8") as f:
self.assertEqual(f.read(), "# first report\n")
with open(os.path.join(second, "report.md"), encoding="utf-8") as f:
self.assertEqual(f.read(), "# second report\n")
def test_concurrent_staging_reservations_are_unique_and_exist(self):
with tempfile.TemporaryDirectory() as tmp, mock.patch(
"skillopt_sleep.staging._ts_dir", return_value="20260815-010203"
):
with ThreadPoolExecutor(max_workers=8) as pool:
paths = list(pool.map(lambda _i: new_staging_dir(tmp), range(16)))
self.assertEqual(len(set(paths)), len(paths))
self.assertTrue(all(os.path.isdir(path) for path in paths))
def test_concurrent_publications_leave_one_valid_atomic_latest_pointer(self):
with tempfile.TemporaryDirectory() as tmp, mock.patch(
"skillopt_sleep.staging._ts_dir", return_value="20260815-010203"
):
def publish(index):
return write_staging(
tmp, report=_report(), proposed_skill=f"# skill {index}\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md=f"# report {index}\n",
)
with ThreadPoolExecutor(max_workers=8) as pool:
paths = list(pool.map(publish, range(16)))
latest = latest_staging(tmp)
self.assertIn(latest, paths)
with open(
os.path.join(tmp, ".skillopt-sleep", "staging", ".latest"),
encoding="utf-8",
) as handle:
self.assertEqual(handle.read().strip(), os.path.basename(latest))
def test_latest_ignores_a_symlinked_night(self):
with tempfile.TemporaryDirectory() as tmp:
real_night = write_staging(
tmp, report=_report(), proposed_skill="# real\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# real report\n",
)
outside = os.path.join(tmp, "outside-night")
os.makedirs(outside)
with open(
os.path.join(outside, "manifest.json"), "w", encoding="utf-8"
) as handle:
handle.write("{}")
alias = os.path.join(
os.path.dirname(real_night), "99991231-235959"
)
try:
os.symlink(outside, alias)
except OSError:
self.skipTest("symlinks unavailable")
self.assertEqual(latest_staging(tmp), real_night)
def test_explicit_symlink_staging_directory_is_refused(self):
with tempfile.TemporaryDirectory() as tmp:
outside = os.path.join(tmp, "outside-night")
os.makedirs(outside)
sentinel = os.path.join(outside, "keep.txt")
with open(sentinel, "w", encoding="utf-8") as handle:
handle.write("untouched\n")
alias = os.path.join(tmp, "staging-alias")
try:
os.symlink(outside, alias)
except OSError:
self.skipTest("symlinks unavailable")
with self.assertRaisesRegex(StagingError, "staging directory is unsafe"):
write_staging(
tmp, report=_report(), proposed_skill="# proposal\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
out_dir=alias,
)
with open(sentinel, encoding="utf-8") as handle:
self.assertEqual(handle.read(), "untouched\n")
self.assertEqual(sorted(os.listdir(outside)), ["keep.txt"])
def test_publication_pointer_orders_equal_mtimes_across_clock_rollback(self):
with tempfile.TemporaryDirectory() as tmp, mock.patch(
"skillopt_sleep.staging._ts_dir",
side_effect=["20261101-015959", "20261101-010001"],
):
before_rollback = write_staging(
tmp, report=_report(), proposed_skill="# before\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# before rollback\n",
)
after_rollback = write_staging(
tmp, report=_report(), proposed_skill="# after\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# after rollback\n",
)
os.utime(
os.path.join(before_rollback, "manifest.json"),
ns=(1_000_000_000, 1_000_000_000),
)
os.utime(
os.path.join(after_rollback, "manifest.json"),
ns=(1_000_000_000, 1_000_000_000),
)
self.assertEqual(latest_staging(tmp), after_rollback)
def test_tampered_symlink_pointer_is_ignored_for_contained_fallback(self):
with tempfile.TemporaryDirectory() as tmp:
first = write_staging(
tmp, report=_report(), proposed_skill="# first\n",
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# first\n",
)
pointer = os.path.join(
tmp, ".skillopt-sleep", "staging", ".latest"
)
outside = os.path.join(tmp, "outside-pointer")
with open(outside, "w", encoding="utf-8") as handle:
handle.write("../../outside-night\n")
os.unlink(pointer)
try:
os.symlink(outside, pointer)
except OSError:
self.skipTest("symlinks unavailable")
self.assertEqual(latest_staging(tmp), first)
def test_failed_artifact_write_publishes_no_partial_night(self):
from skillopt_sleep import staging as staging_mod
with tempfile.TemporaryDirectory() as tmp:
real_write = staging_mod._write_atomic
def fail_second_proposal(path, text, *, create_parents=True):
if path.endswith("proposed_SKILL.beta.md"):
raise OSError("disk full")
return real_write(path, text, create_parents=create_parents)
with mock.patch.object(
staging_mod, "_write_atomic", side_effect=fail_second_proposal
), self.assertRaises(OSError):
write_staging(
tmp, report=_report(), proposed_skill=None,
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
skill_proposals=[
_proposal("alpha", "# alpha\n", root=os.path.join(tmp, "live")),
_proposal("beta", "# beta\n", root=os.path.join(tmp, "live")),
],
)
self.assertIsNone(latest_staging(tmp))
for root, _dirs, files in os.walk(tmp):
self.assertNotIn("manifest.json", files, root)
self.assertFalse(
any(name.startswith("proposed_SKILL") for name in files),
root,
)
def test_artifact_rollback_attempts_every_restore_and_preserves_primary(self):
from skillopt_sleep import staging as staging_mod
with tempfile.TemporaryDirectory() as tmp:
first = os.path.join(tmp, "first.md")
second = os.path.join(tmp, "second.md")
for path, body in ((first, "old first"), (second, "old second")):
with open(path, "w", encoding="utf-8") as handle:
handle.write(body)
restore_attempts = []
real_restore = staging_mod._write_atomic_bytes
def fail_publish(path, text, *, create_parents=True):
if path == second:
raise OSError("primary publish failure")
def restore(path, data, *, create_parents=True, mode=None):
restore_attempts.append(path)
if path == first:
raise OSError("first restore failed")
return real_restore(
path,
data,
create_parents=create_parents,
mode=mode,
)
with mock.patch.object(
staging_mod, "_write_atomic", side_effect=fail_publish
), mock.patch.object(
staging_mod, "_write_atomic_bytes", side_effect=restore
):
with self.assertRaises(staging_mod.StagingRecoveryError) as caught:
staging_mod._write_artifact_batch([
(first, "new first"),
(second, "new second"),
])
self.assertIsInstance(caught.exception.primary, OSError)
self.assertIn("primary publish failure", str(caught.exception.primary))
self.assertEqual(restore_attempts, [second, first])
def test_post_commit_artifact_error_rolls_back_the_whole_night(self):
from skillopt_sleep import staging as staging_mod
with tempfile.TemporaryDirectory() as tmp:
real_write = staging_mod._write_atomic
def commit_then_fail(path, text, *, create_parents=True):
result = real_write(path, text, create_parents=create_parents)
if path.endswith("proposed_SKILL.beta.md"):
raise OSError("late close failure")
return result
with mock.patch.object(
staging_mod, "_write_atomic", side_effect=commit_then_fail
), self.assertRaisesRegex(OSError, "late close failure"):
write_staging(
tmp, report=_report(), proposed_skill=None,
proposed_memory=None,
live_skill_path=os.path.join(tmp, "live", "SKILL.md"),
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
skill_proposals=[
_proposal("alpha", "# alpha\n", root=os.path.join(tmp, "live")),
_proposal("beta", "# beta\n", root=os.path.join(tmp, "live")),
],
)
self.assertIsNone(latest_staging(tmp))
for root, _dirs, files in os.walk(tmp):
self.assertFalse(files, f"partial staging artifacts remain in {root}")
def test_exact_cycle_baseline_change_before_staging_is_refused(self):
with tempfile.TemporaryDirectory() as tmp:
live = os.path.join(tmp, "live", "alpha", "SKILL.md")
os.makedirs(os.path.dirname(live), exist_ok=True)
with open(live, "w", encoding="utf-8") as handle:
handle.write("# alpha baseline v1\n")
proposal = SkillProposal(
"alpha",
"# alpha proposal derived from v1\n",
live,
live_sha256=hashlib.sha256(b"# alpha baseline v1\n").hexdigest(),
live_realpath=os.path.realpath(live),
)
with open(live, "w", encoding="utf-8") as handle:
handle.write("# alpha human v2\n")
with self.assertRaisesRegex(StagingError, "changed during consolidation"):
write_staging(
tmp, report=_report(), proposed_skill=None,
proposed_memory=None,
live_skill_path=live,
live_memory_path=os.path.join(tmp, "live", "CLAUDE.md"),
report_md="# report\n",
skill_proposals=[proposal],
)
self.assertIsNone(latest_staging(tmp))
def test_unsafe_fan_out_writes_no_manifest(self):
with tempfile.TemporaryDirectory() as tmp:
tests/test_sleep_state.py
+16
View File
@@ -0,0 +1,16 @@
"""Isolation regressions for SkillOpt-Sleep's in-process default state."""
from skillopt_sleep.state import SleepState
def test_fresh_states_do_not_share_nested_default_containers(tmp_path):
first = SleepState.load(str(tmp_path / "first.json"))
second = SleepState.load(str(tmp_path / "second.json"))
first.set_last_harvest("/project/one", "2026-08-15T00:00:00")
first.record_night({"night": 1})
first.add_to_archive([{"id": "private-to-first"}])
assert second.last_harvest_for("/project/one") is None
assert second.data["history"] == []
assert second.task_archive() == []