Files
heygen-com--hyperframes/packages/cli/scripts/sync-agent-dirs.ts
WaterrrForever bf961d1268 feat(cli): skills freshness — version check, manifest, global install + multi-agent mirror (#1753)
* feat(cli): add skills version check, update, and freshness manifest

Give the HyperFrames skill bundle a content fingerprint so agents and
users can tell whether installed skills are the latest version, on any
platform that can run the CLI.

- skills-manifest.json (repo root): per-skill sha256 over the whole skill
  directory; minimal {source, skills}, no version/timestamp so it is fully
  deterministic. Generated by scripts/gen-skills-manifest.ts.
- `hyperframes skills check` [--json]: compares installed skills to the
  manifest; exits non-zero when something is outdated (agent/CI gate).
- `hyperframes skills update`: thin wrapper over `npx skills update`.
- Passive nudge on render/lint/validate when skills are stale (24h cache,
  same opt-out as the CLI self-update notice).
- "latest" resolved via `git ls-remote` + SHA-pinned raw URL to dodge
  GitHub raw-CDN lag, falling back to the main branch URL.
- CI job + lefthook hook keep skills-manifest.json in sync with skills/.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): add execFile to child_process mock in skills test

skills.test.ts mocks node:child_process but only declared execFileSync
and spawn. Loading skills.js transitively loads skillsManifest.ts, which
runs promisify(execFile) at module load, so vitest threw on the missing
execFile named export. Add a bare stub — these tests never invoke it.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): init installs all skills; skills update pulls the full set

Make `hyperframes init` the single place skills are pulled in full, and
make "update" mean "get everything" rather than "refresh what's there".

- init now always installs/refreshes ALL skills (incl. ones not yet
  present) instead of prompting "Install AI coding skills?" — opt out
  with `init --skip-skills`. Both the interactive and non-interactive
  paths pass `--all --yes` so the complete set is fetched.
- `hyperframes skills update` switches from `npx skills update` (which
  only refreshes already-installed skills) to `skills add --all`, so it
  installs missing skills too — the same install step init runs.
- SKILL.md documents init-installs-all and the new update semantics.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): skills check treats missing skills as needing an update

The full skill set is now the goal (init and `skills update` both pull
all, including ones not installed), so a partial install is no longer
"a choice" — it's something to fix.

- diffSkills: updateAvailable is now true when anything is outdated OR
  missing (local-only still doesn't count). So `skills check` exits
  non-zero — and renders "Update:" instead of "up to date" — whenever a
  skill is missing, not just when one is stale.
- The passive render/lint/validate nudge follows suit: it now counts
  missing alongside outdated ("N skills out of date or missing"),
  tracked via a new skillsMissingCount cache field.
- SKILL.md documents the stricter check.

Note: platforms that intentionally vendor only a subset of skills (e.g.
a Codex snapshot) will now see check report non-zero.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): install/update skills straight from the GitHub repo

`skills add owner/repo` can resolve through the skills.sh registry, which
lags behind the repo — so `update` could install a stale version while
`check` (which resolves latest directly from GitHub) keeps reporting
"outdated", an endless loop.

Switch the install source to the full GitHub URL
(https://github.com/heygen-com/hyperframes), which makes `skills add`
git-clone the repo directly at latest main, bypassing the registry. This
covers `hyperframes skills`, `hyperframes skills update`, and `init`'s
skill install — all of which go through SOURCES. Now install/update and
check agree on what "latest" means.

The init "install skills" hint now points at `npx hyperframes skills
update` so the manual path uses the same GitHub-direct fetch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(cli): init checks skills against GitHub, installs only when stale

`hyperframes init` now runs the skills version check first and only
(re)installs when something is outdated or missing — instead of
unconditionally re-pulling every time. Re-running init on an
already-current project is now a no-op ("skills are already up to date").

- New ensureSkillsCurrent() helper, shared by both the interactive and
  non-interactive init paths (no duplicated install logic).
- The check resolves "latest" straight from GitHub (same source the
  install uses); best-effort — if it can't reach GitHub it installs anyway.
- SKILL.md updated to describe the check-then-install behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(cli): address skills manifest review feedback

From the PR review (points 1, 2, 4, 5):

1. Remove the `local-only` skill status. checkSkills only ever hashes
   manifest-listed skills, so a local-only status could never appear in
   the end-to-end output — and making it appear would wrongly flag
   unrelated skills (the `.../skills` dir is shared across sources).
   diffSkills now reports only on manifest skills; skills on disk that
   aren't in the manifest are ignored.
2. Drop the redundant per-directory sort in listFilesSorted — the single
   final out.sort() is what guarantees a deterministic hash (verified:
   manifest unchanged).
4. resolveLatestManifest local-path detection now uses path.isAbsolute,
   so Windows absolute paths (C:\...) are treated as local instead of
   falling through to a remote fetch.
5. fetchManifest validates the response shape (asSkillsManifest) instead
   of a blind `as` cast, so a CDN error page served as 200 fails with a
   clear error rather than a cryptic crash later in diffSkills.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): strict skills update + auto-discover any agent host

Address PR review (Magi blocker + James/Rames robustness):

- Blocker (Magi): `skills update` is the documented recovery path for
  `skills check || skills update`, but it delegated to installAllSkills()
  which swallowed missing-npx and failed `skills add` as "skipped",
  exiting 0 even when nothing changed. Add a strict mode that throws on
  failure; update sets a non-zero exit (init stays best-effort). New tests
  simulate a non-zero `skills add` (exit 1) and the success path.

- Robustness (James/Rames #2): the upstream `skills` CLI installs into
  ~72 agent conventions; a hard-coded list (4, or even 11) can't track
  that. Replace defaultSkillRoots with discoverSkillRoots — it scans cwd +
  $HOME for any `<host>/skills/<manifest-skill>/SKILL.md` (plus the XDG
  `.config/<host>/skills`), so detection is structural and future-proof,
  no closed list. agentFromDir infers the host from the path.

- Tests (Rames #3): temp-fixture detection tests for every convention ×
  {project, global}, scope priority, claude-code preference, the
  no-install case, the --dir override, and an unknown/new host (proving
  the no-closed-list property).

- Docs (Rames #4/#5): SKILL.md notes init's best-effort GitHub round-trip;
  findRepoManifest climbs 16 levels (was 8) for deep monorepos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): resolve CodeQL file-system race + de-flake Windows npx test

Two CI fixes:

- CodeQL (high, js/file-system-race) at gen-skills-manifest.ts: the
  existsSync(outPath) precheck followed by writeFileSync(outPath) is a
  check-then-write race. Read the committed manifest directly in a
  try/catch instead (missing/unreadable ⇒ "no committed manifest"), so
  there's no precheck to race against. Behavior is unchanged.

- Windows Tests: npxCommand.test.ts's real `npx --version` smoke test
  cold-starts slower than vitest's 5s default on Windows runners and
  timed out. Give the test 60s headroom (and a 30s exec timeout). Kept
  as a real execution check — mocking would reduce it to a tautology.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): repair garbled npx smoke-test timeout comment

The explanatory comment for the 60s timeout was scrambled across the
callback/timeout arguments, failing oxfmt --check (and thus preflight,
which in turn skipped preview-parity and failed the regression gate).
Move it above the it() call so it no longer sits between call arguments.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): install skills once globally + symlink-mirror to every agent

The previous install path sprayed a full ~6.7MB skill copy into each of the
~70 agent conventions `skills add --all` knows (a fresh init produced 40+
dirs / 341MB, incl. a stray dotless `agent/` from the Eve convention).

Install ONCE, globally, as one faithful copy, then symlink it everywhere:
  - `skills add <url> --skill '*' --global --agent claude-code universal
    --copy` lands real files in ~/.claude/skills (Claude Code reads this at
    global priority) and ~/.agents/skills (the shared universal store).
  - mirrorGlobalSkills() fans that store out to every OTHER installed agent's
    GLOBAL dir (~/.cursor/skills, goose -> ~/.config/goose/skills, ...) — but
    only for agents present on the machine (marker dir exists), so nothing is
    sprayed. Unix: per-skill relative symlink into the store (one source of
    truth, auto-fresh on update); Windows: copy (symlinks need admin /
    Developer Mode there — the same fallback upstream and gstack make).

Why global: skills are framework-general knowledge, not project content;
Claude Code (and most agents) prioritize the personal/global scope, so the
global copy is the one actually loaded — and it installs once instead of
multiplying per project.

The per-agent dir list is GENERATED from upstream's src/agents.ts at a pinned
tag (the `skills` package exports nothing importable), committed as
agentDirs.generated.ts and resolved env-faithfully at runtime
(XDG_CONFIG_HOME / CODEX_HOME / CLAUDE_CONFIG_DIR honored). Regenerate with
`bun run --cwd packages/cli gen:agent-dirs` when the pin moves. Covers all 70
agents that define a global dir (eve/promptscript define none); the bare
project-dir agents (openclaw, astrbot) are namespaced globally, so the
stray-`agent/` footgun is gone.

`skills check` now scans global ($HOME) before project (cwd) to match the
runtime load order — so it reports on the copy the agent will really use, not
a stale project copy a newer global install silently overrides.

Test plan:
- skills.test.ts: install spawns the global --copy args, never --all; update
  stays strict + exits non-zero on failure.
- skillsMirror.test.ts: Unix relative symlinks, Windows copy, XDG_CONFIG_HOME
  honored, install-owned stores skipped, marker-gating, idempotent refresh,
  generated-table shape.
- skillsManifest.test.ts: check is global-first.
- Full CLI suite green (981); oxlint / oxfmt / tsc clean; gen:agent-dirs
  --check clean (offline + network produce byte-identical output).
- Benchmark (isolated HOME, local CLI): claude+hermes and all 70 agents —
  ~/.claude + ~/.agents real (19 each), every installed agent's global dir =
  19 symlinks into the store, zero spray into unseeded agents, check
  global-first. (The 9 "outdated" check reports are the separate skills.sh
  registry lag, not this change.)
- .fallowrc.jsonc: exempt the codegen script's inherent parser complexity and
  the parallel-case duplication in skillsManifest.test.ts (same rationale the
  config already uses for SlideshowPanel.test.ts / hyperframes-player.test.ts).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): install skills with --full-depth so a fresh install reads as current

`skills add <url>` without --full-depth fetches from the skills.sh registry
blob ("Fetching skills"), which lags GitHub main by hours — so a freshly
installed/updated set read as ~9 skills "outdated" right after install, and
`skills update` couldn't fix it (it re-fetched the same stale blob → death
loop). --full-depth switches it to a real `git clone` of HEAD ("Cloning
repository"), the only path that yields the genuine latest.

- Add --full-depth to the global install args. Verified (isolated HOME): blob
  path → 10 current / 9 outdated; --full-depth → 19 current / 0 outdated.
- The clone is heavier than the blob fetch, so set GIT_LFS_SKIP_SMUDGE=1 (skills
  are text; the repo's LFS objects are unrelated binaries the install doesn't
  need) and raise the spawn timeout 120s → 300s.
- Correct the stale comment that claimed a full URL already bypasses skills.sh —
  it doesn't; only --full-depth does.

Benchmark (skills-bench, local CLI): B.death-loop and J1.init-detect-and-refresh
flip FAIL → PASS (install/update/init now 19/0); mirror smoke reports 19 current
/ 0 outdated. (spine still reflects the raw documented `skills add <slug>`
command — the upstream skills.sh path, not this CLI.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(skills): drop --skip-skills from workflow init so new projects refresh skills

The creation workflows scaffolded with `hyperframes init … --skip-skills`, which
skipped the skills currency check. Now that init installs globally, is a no-op
when already current, and pulls the genuine latest (via --full-depth), there's
no reason to skip it: removing --skip-skills means every new project runs the
check and refreshes the global skill set from GitHub when it's stale. Add a
one-line note to each workflow (embedded-captions, faceless-explainer,
motion-graphics, music-to-video, pr-to-video, product-launch-video) and the
hyperframes-cli + /hyperframes router explaining what init does.

skills-manifest.json regenerated by the pre-commit hook to match the edited
skill bundles.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): scope agent mirror to HyperFrames' own skills, not the whole store

mirrorGlobalSkills listed every */SKILL.md in ~/.claude/skills and fanned them
out — but that store is shared, so a user's gstack / personal / company Claude
skills would get symlinked (and, since linkOrCopy removes the target first,
could overwrite a same-named skill) into Cursor / Codex / Goose / etc.

Scope the mirror to HyperFrames' own skills via the upstream lock's source
attribution — the same definition the prune already uses
(skillsAttributedToSource) — never a directory listing. New
hyperframesSkillNames() reads the global lock and returns only skills attributed
to heygen-com/hyperframes; the mirror intersects that allow-list with what's in
the store. Empty (no lock / nothing attributed) → mirror nothing, never
everything.

Also fixes the cosmetic "director(ies)" log typo (now singular/plural-aware) and
extracts the fan-out into mirrorToInstalledAgents() to keep installAllSkills
under the complexity gate.

Regression: skillsMirror.test.ts asserts a foreign gstack skill in the store is
neither mirrored out nor allowed to replace another agent's same-named skill;
the skills-bench harness seeds ~/.claude/skills/gstack and asserts it never
leaks to any agent. 1045 CLI tests + lint/types/fallow green.

Addresses Magi's request-changes on #1753.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-27 08:34:43 +08:00

170 lines
6.7 KiB
TypeScript

// Generate (or verify) packages/cli/src/utils/agentDirs.generated.ts — the
// home-relative GLOBAL skills directory for every agent the upstream
// vercel-labs/skills CLI knows about, plus a marker dir that means "this agent
// is installed on this machine".
//
// bun run --cwd packages/cli gen:agent-dirs # write/update (fetches upstream)
// bun run --cwd packages/cli gen:agent-dirs --check # verify only (CI / pre-commit)
// bun packages/cli/scripts/sync-agent-dirs.ts --src <path/to/agents.ts> # offline
//
// Why generated, not hand-maintained: `skills add --global` installs into the
// per-agent dirs encoded in upstream's `src/agents.ts` (~70 agents). We mirror
// the canonical store into those same dirs, so the list must track upstream. The
// `skills` npm package exports nothing importable (CLI-only, bundled dist), so
// we parse the source at a PINNED tag and commit the result — deterministic at
// runtime, no network at install time. Bump SKILLS_REF and re-run on upgrade.
import { readFileSync, writeFileSync } from "node:fs";
import { dirname, join } from "node:path";
import { fileURLToPath } from "node:url";
// Pin to the upstream release whose dir layout we install against. Bump this
// (and re-run) when the bundled `skills` version moves.
const SKILLS_REPO = "vercel-labs/skills";
const SKILLS_REF = "v1.5.13";
const AGENTS_TS_URL = `https://raw.githubusercontent.com/${SKILLS_REPO}/${SKILLS_REF}/src/agents.ts`;
const here = dirname(fileURLToPath(import.meta.url));
const outPath = join(here, "..", "src", "utils", "agentDirs.generated.ts");
const isCheck = process.argv.includes("--check");
const srcFlag = process.argv.indexOf("--src");
const srcPath = srcFlag !== -1 ? process.argv[srcFlag + 1] : undefined;
// The base directories agents.ts builds globalSkillsDir from. Each is an env
// override with a documented default — the mirror resolves them at runtime so a
// machine with XDG_CONFIG_HOME / CODEX_HOME / CLAUDE_CONFIG_DIR set lands in the
// same place the agent actually reads. We store the base NAME + suffix here
// rather than a frozen path so that resolution stays faithful to upstream.
const BASE_VARS = [
"home",
"configHome",
"codexHome",
"claudeHome",
"vibeHome",
"hermesHome",
"autohandHome",
] as const;
type BaseVar = (typeof BASE_VARS)[number];
interface AgentGlobalDir {
agent: string;
base: BaseVar;
sub: string;
}
function posixJoin(parts: string[]): string {
return parts
.flatMap((p) => p.split("/"))
.filter((s) => s && s !== ".")
.join("/");
}
/** Resolve a globalSkillsDir expression to { base, sub }, or null if none. */
function resolveGlobalExpr(expr: string): { base: BaseVar; sub: string } | null {
const e = expr.trim();
if (e === "undefined") return null; // agent defines no global skills dir
// openclaw's helper falls back to ~/.openclaw/skills when no variant is present.
if (e.startsWith("getOpenClawGlobalSkillsDir")) return { base: "home", sub: ".openclaw/skills" };
const m = e.match(/^join\(([\s\S]+)\)$/);
if (!m) throw new Error(`Unparseable globalSkillsDir: ${expr}`);
const args = m[1]!.split(",").map((a) => a.trim());
const first = args[0]!;
if (!(BASE_VARS as readonly string[]).includes(first)) {
throw new Error(`globalSkillsDir does not start with a known base var: ${expr}`);
}
const segs: string[] = [];
for (const raw of args.slice(1)) {
const lit = raw.match(/^['"]([^'"]*)['"]$/);
if (!lit) throw new Error(`Non-literal segment "${raw}" in globalSkillsDir: ${expr}`);
segs.push(lit[1]!);
}
return { base: first as BaseVar, sub: posixJoin(segs) };
}
function parseAgents(source: string): AgentGlobalDir[] {
const re = /^ {2}(?:"([a-z0-9-]+)"|'([a-z0-9-]+)'|([a-z0-9-]+)):\s*\{/gm;
const blocks: { key: string; pos: number }[] = [];
let m: RegExpExecArray | null;
while ((m = re.exec(source))) blocks.push({ key: m[1] || m[2] || m[3]!, pos: m.index });
if (blocks.length < 60) {
throw new Error(`Parsed only ${blocks.length} agent blocks — upstream layout likely changed`);
}
const out: AgentGlobalDir[] = [];
for (let i = 0; i < blocks.length; i++) {
const seg = source.slice(blocks[i]!.pos, blocks[i + 1] ? blocks[i + 1]!.pos : source.length);
const gd = seg.match(/globalSkillsDir:\s*([\s\S]+?),\n/);
const expr = gd ? gd[1]!.trim().replace(/\s+/g, " ") : "undefined";
const resolved = resolveGlobalExpr(expr);
if (resolved === null) continue; // no global dir (e.g. eve, promptscript)
out.push({ agent: blocks[i]!.key, base: resolved.base, sub: resolved.sub });
}
return out;
}
function render(rows: AgentGlobalDir[]): string {
const lines = rows.map(
(r) =>
` { agent: ${JSON.stringify(r.agent)}, base: ${JSON.stringify(r.base)}, sub: ${JSON.stringify(r.sub)} },`,
);
return `// @generated by packages/cli/scripts/sync-agent-dirs.ts — DO NOT EDIT.
// Source: ${SKILLS_REPO}@${SKILLS_REF} (src/agents.ts). Regenerate with:
// bun run --cwd packages/cli gen:agent-dirs
//
// Each entry is one agent the upstream \`skills\` CLI installs to. The agent's
// GLOBAL skills directory is \`join(<base>, <sub>)\`, where \`base\` is one of the
// env-overridable home dirs below (resolved at runtime by skillsMirror.ts, so
// XDG_CONFIG_HOME / CODEX_HOME / CLAUDE_CONFIG_DIR are honored). Agents with no
// global skills dir upstream (eve, promptscript) are omitted.
/** Env-overridable base dirs, matching upstream agents.ts. */
export type AgentDirBase =
${BASE_VARS.map((b) => ` | ${JSON.stringify(b)}`).join("\n")};
export interface AgentGlobalDir {
/** Upstream agent key. */
agent: string;
/** Base directory the global skills dir is rooted at. */
base: AgentDirBase;
/** POSIX suffix joined onto the resolved base. */
sub: string;
}
export const AGENT_GLOBAL_DIRS: readonly AgentGlobalDir[] = [
${lines.join("\n")}
];
`;
}
async function loadAgentsSource(): Promise<string> {
if (srcPath) return readFileSync(srcPath, "utf8");
const res = await fetch(AGENTS_TS_URL);
if (!res.ok) throw new Error(`HTTP ${res.status} fetching ${AGENTS_TS_URL}`);
return res.text();
}
const source = await loadAgentsSource();
const rows = parseAgents(source);
const next = render(rows);
if (isCheck) {
let current = "";
try {
current = readFileSync(outPath, "utf8");
} catch {
/* missing → drift */
}
if (current !== next) {
console.error(
`agentDirs.generated.ts is out of date (parsed ${rows.length} agents from ${SKILLS_REPO}@${SKILLS_REF}).\n` +
`Run: bun run --cwd packages/cli gen:agent-dirs`,
);
process.exit(1);
}
console.log(`agentDirs.generated.ts is up to date (${rows.length} agents).`);
} else {
writeFileSync(outPath, next, "utf8");
console.log(`Wrote ${rows.length} agents → ${outPath}`);
}