Compare commits

..

39 Commits

Author SHA1 Message Date
jackwener ce8bc3e73f docs(stackoverflow): clarify read fetches answers up to --answers-limit (not 'all')
Follow-up from PR #1293 review: 'all answers' was misleading because
the implementation is limit-bounded (default 10, max 100) rather than
unbounded pagination. Spell out the actual contract — including the
accepted-answer-outside-page fallback path — so users don't expect
infinite-scroll behaviour.

Non-blocking docs-only change flagged by codex-mini1 + First-principles-1
during #1293 review.
2026-05-04 19:18:13 +08:00
jakevin c1a4bd3b7e feat(stackoverflow): surface question_id on listings + new read <id> (#1293)
* feat(stackoverflow): surface question_id + metadata on listings, add `read <id>`

Agent-native gap: all 4 stackoverflow listings (`hot`, `search`,
`unanswered`, `bounties`) only emitted `[title, score, answers, url]`,
which means an agent could see a hot question but had no `id` to round-
trip into a body read, no `tags` to filter by topic, no `views` to gauge
demand, and no `is_answered` / `creation_date` / `author` to triage.
There also wasn't a `read` adapter, so reading a SO question through
opencli was impossible.

Listings (`hot` / `search` / `bounties` / `unanswered`):
- Add `rank`, `id` (question_id), `views`, `is_answered` (skipped on
  `unanswered` since always false), `tags` (joined), `author`
  (owner.display_name), `creation_date` columns.
- Pass `pagesize` to the upstream API instead of fetching the default
  page and trimming locally.

New `stackoverflow read <id>`:
- 4-call fan-out against the public Stack Exchange API
  (`/questions/{id}` + `/questions/{id}/comments` +
  `/questions/{id}/answers` + batched `/answers/a;b;c/comments`).
- Returns `POST` + `Q-COMMENT` + `ANSWER` + `A-COMMENT` rows mirroring
  the `hackernews read` and `lobsters read` shape.
- Accepted answer is always surfaced first and tagged `accepted='true'`;
  remaining answers follow in descending vote order, capped by
  `--answers-limit`.
- HTML body cleanup: tags stripped, `<pre><code>` preserved, `<code>`
  inline-fenced, `<li>` rendered as `- `, comments indented with `> `.
- Entity decoding: a shared `decodeEntities` handles named (incl.
  `&hellip;`/`&copy;`/etc), decimal (`&#246;`), and hex (`&#x27;`)
  forms, applied to both bodies AND `display_name` (otherwise users
  like `Jonas K&#246;lker` come through mojibaked).
- Typed fail-fast: `ArgumentError` for non-numeric id and
  `--max-length < 100` (with no-fetch assertion); `EmptyResultError`
  when `items` is empty; `CommandExecutionError` for HTTP non-2xx and
  for Stack Exchange's in-band `error_id` envelopes (throttle / quota).
  No silent clamps anywhere.

Tests: 14 vitest assertions
- 4 listing column-shape (incl. `unanswered` skipping `is_answered` and
  `bounties` keeping its `bounty` column position)
- 10 read-adapter cases: registration / args / strategy + 3 typed-error
  fail-fast paths (with no-fetch assertion on the pre-fetch ones) + the
  full POST/Q-COMMENT/ANSWER/A-COMMENT row order with accepted-first +
  the answer-comments fetch verified to batch ids semicolon-joined +
  HTML entity decoding (named/decimal/hex) on both body and display_name
  + answers-limit honored when there are more answers than the cap.

Live verification:
- `stackoverflow hot --limit 2` → `id`/`tags`/`views`/`is_answered`/
  `author` populated.
- `stackoverflow search "async await" --limit 1`,
  `stackoverflow unanswered --limit 1` → same shape.
- `stackoverflow read 79935770` and the very-long classic question
  `stackoverflow read 11227809 --answers-limit 1 --comments-limit 2`
  → produces the threaded POST/Q-COMMENT/ANSWER/A-COMMENT structure
  with proper entity decoding (`Jonas Kölker` reads correctly).
- `stackoverflow read not-numeric` → exits with `ARGUMENT`.
- `stackoverflow read 999999999` → exits with `EMPTY_RESULT`.

* fix(stackoverflow): wrap fetch/json/coerce paths in typed errors

Apply the 3 lessons from PR #1292 (devto) review at merge time, before
B-group hits this PR:

1. CLI args may arrive as strings (e.g. `--max-length 50` → `'50'`).
   The bare `Number.isInteger(value)` in `requirePositiveInt` /
   `requireMinInt` would accept negative-but-coerced numbers and reject
   string-form integers. Now the helpers `coerceInt` first then validate,
   and the rejection message echoes the raw input via `JSON.stringify`.

2. `await fetch(url)` and `await res.json()` were not wrapped — a network
   blip would surface as a raw `TypeError` and a maintenance HTML page
   would surface as a raw `SyntaxError`. Both are now caught and rethrown
   as `CommandExecutionError` with hints, matching the in-band error_id
   path.

Tests: +3 cases (17 total)
- fetch network failure → CommandExecutionError
- malformed JSON body → CommandExecutionError
- string-form max-length "50" / "abc" rejected with ArgumentError before
  fetching

* fix(stackoverflow): avoid partial read fanout
2026-05-04 19:10:50 +08:00
jakevin 5a839701ab feat(lobsters): surface short_id + created_at on listings, add read <short_id> (#1291)
* feat(lobsters): surface short_id + created_at on listings, add `read <short_id>`

Same agent-native gap as the just-merged hackernews PR (#1288):

1. The 4 listings (hot / newest / active / tag) didn't surface each story's
   `short_id`. Agents could see the title and a comments URL but couldn't
   pass the id back into a follow-up command. Add `id` (= `short_id`) and
   `created_at` columns; `created_at` is cheap signal for "how stale is this".

2. There was no way to read a story + comment tree from the CLI. Lobsters
   makes this nicer than HN: `https://lobste.rs/s/<short_id>.json` returns
   the story plus a flat `comments[]` array where each entry already carries
   `parent_comment` and `depth`, so we get the full thread in one HTTP call
   and just DFS using the parent map.

`read` mirrors the `hackernews read` shape (POST row + L0/L1/… indented
comments, `[+N more replies]` stubs at depth/limit cutoffs) so the two
adapters feel the same to agents that already learned one. Same typed
fail-fast envelope: `ArgumentError` on bad short_id / non-positive limit /
depth / replies, `EmptyResultError` on 404 or empty body, `CommandExecutionError`
on other HTTP failures.

Tests cover all 4 listings (column shape + map step), `read` registration,
positional arg shape, ArgumentError fail-fast (no fetch on bad input),
EmptyResultError on 404, threaded-tree assembly from a flat `comments[]`,
and the `+N more replies` depth-cutoff path.

* test(lobsters): lock read fail-fast coverage

* docs(lobsters): list read command
2026-05-04 18:55:51 +08:00
jakevin 68485cc54e feat(devto): surface article id on listings + new read <id> (#1292)
* feat(devto): surface article id + published_at on listings, add `read <id>`

Agent-native gap: devto listings (`top`/`tag`/`user`) didn't include the
article `id`, so an agent couldn't round-trip from a listing into a body
read. They also dropped `reading_time` and `published_at`, which are cheap
signals the API gives you for free.

Changes:
- `top` / `tag` / `user`: add `id`, `reading_time`, `published_at` columns
  alongside existing rank/title/etc. `user` keeps its no-author shape since
  it's already user-scoped.
- New `devto read <id>`: hits `dev.to/api/articles/<id>` and returns one
  row with the article body (truncated by `--max-length`, default 20000,
  min 100). DEV.to's public API does not expose comments yet, so this is
  intentionally a single-row reader rather than a HN/lobsters-style
  threaded tree — if/when comments become public we can extend to
  POST + L0/L1.
- Typed fail-fast: `ArgumentError` for non-numeric id and for `--max-length`
  below 100; `EmptyResultError` on 404; `CommandExecutionError` for other
  non-2xx HTTP statuses. No silent clamps.
- Defensive tag normalization: the `/api/articles/<id>` endpoint returns
  `tag_list` as a comma-string and `tags` as an array (the opposite shape
  from listing endpoints). Caught this on live verification — both shapes
  now collapse to a comma-joined string.

Tests: 12 vitest assertions covering listing column shape (all 3) +
register/args/strategy + typed-error fail-fast paths + happy-path body
extraction + truncation marker + alternate tag_list shape.

Live verification: `devto top --limit 3` and `devto read 3602287` both
return the expected agent-native shape.

* fix(devto): harden article read contract
2026-05-04 18:55:14 +08:00
jakevin aa8d4b72f7 fix(twitter): drop permanently-N/A tweets column from trending (#1290)
X removed the post-count caption from each cell on `/explore/tabs/trending`.
The adapter still iterated `divs[2..]` looking for a numeric text node and
fell back to the literal string "N/A" when it found none — which was every
row, on every call. We were emitting a silent-wrong column for every result.

Drop the column and the no-longer-relevant scan loop. Add a regression test
on the columns shape so the column doesn't slip back in.

Live runs of `opencli twitter trending` previously returned rows like
`{rank: 1, topic: "...", tweets: "N/A", category: "..."}` — the `N/A` was
not a transient outage, it was structural.
2026-05-04 18:31:06 +08:00
jakevin e848594519 feat(arxiv): full abstract/authors + surface pdf/categories/comment + new recent <category> (#1289)
* feat(arxiv): full abstract/authors, surface pdf+categories+comment, add `recent <category>`

`paper` was silently truncating the abstract to 200 chars and dropping all but
the first 3 authors — agents calling it for a paper summary lost data. Stop
truncating, return all authors, and surface the rest of what the Atom feed
already gives us: pdf url (`<link rel="related">`), all `categories`,
`primary_category`, and the author `comment` (page count, conference, etc.).

`search` keeps a compact list shape (no abstract column, but adds
`primary_category`).

New `arxiv recent <category>` lists newest submissions in a category sorted by
`submittedDate desc` — fills a gap (previously you had to know a search term
to surface anything). Validates the category string and rejects malformed
input via `ArgumentError`.

`search` also switches its no-results path from `CliError('NOT_FOUND', ...)`
to `EmptyResultError` to match the convention other public-API adapters use.

Tests cover: command registration, full-abstract / all-authors parsing, XML
entity decoding in titles, pdf/categories/comment extraction, and category
validation.

* fix(arxiv): harden category and limit validation
2026-05-04 18:27:13 +08:00
jakevin 977105b0f6 feat(hackernews): add read <id> and surface item id on every listing (#1288)
* feat(hackernews): add `read <id>` and surface item id on every listing

Two related agent-flow gaps in the HN adapters:

1. `top`/`best`/`ask`/`new`/`show`/`jobs`/`search` all carry the HN item
   id internally (firebase items are fetched by id; algolia hits include
   `objectID`) but drop it before output. Without an id column the agent
   can see the title but has no handle to follow up with.

2. There was no way to read a story's discussion. The whole reason an
   agent looks at HN is the comments — and that capability was missing.

This PR adds:

- `id` column on every listing adapter (firebase items: numeric id;
  algolia search hits: `objectID` string). Existing column order is
  preserved otherwise.
- `hackernews read <id>` — public/non-browser adapter that fetches the
  story plus a tree of top-level comments + inline replies via
  `https://hacker-news.firebaseio.com/v0/item/<id>.json`. Mirrors the
  `reddit read` shape (`type/author/score/text`) so agents can use both
  with one mental model. HTML-only fields (comment text) are converted
  to plain text with anchor URLs preserved.
- Column-contract tests covering all listings + the new read adapter.
- Doc entry under `docs/adapters/browser/hackernews.md`.

Tested locally via `~/.opencli/clis/hackernews/` overrides:
  opencli hackernews top --limit 3            # id present
  opencli hackernews search rust --limit 2    # id (objectID) present
  opencli hackernews read 47999636 --limit 5  # threaded output

* fix(hackernews): typed fail-fast for read
2026-05-04 18:25:57 +08:00
jakevin 413bbbe819 fix(douban): drop unparseable fields from movie-hot, add id/votes (#1285)
* fix(douban): drop unparseable fields from movie-hot, add id/votes

The chart page (movie.douban.com/chart) only exposes a single comma-joined
text dump in `.pl2 p`, of the shape:

  <release_dates...> / <actors...> / <regions...> / <director_zh> /
  <runtime>分钟 / <other_titles> / <genres> / <director with English> /
  <languages>

The previous `loadDoubanMovieHot` tried to anchor on the release-date
regex and take `parts[releaseIndex - 1]` as director and
`parts[releaseIndex - 2]` as region. That breaks in two ways:

1. Most entries have multiple release dates back-to-back, so the
   "anchor minus one" position is itself a date. Director output becomes
   `'2025-09-07(多伦多电影节)'` and region is empty — silent wrong data.
2. For entries with a single release date, the offsets land on actor
   names, not director / region.

The page does not actually carry a clean director or region per row —
that's only available on the subject detail page. Trying to reconstruct
either from the chart string is the canonical "verify passes but data is
wrong" failure (success-rate-pitfalls §2 sibling DOM contamination).

Fix: drop `director`, `region`, `quote` from the listing. Surface what
the chart page actually provides reliably:
- `id`     — extracted from the subject URL, ready for `douban subject`
- `votes`  — from `.star .pl` (`(62484人评价)`), useful as popularity signal
- existing `rank`, `title`, `rating`, `year`, `url`

Agents that need director / region should follow up with
`opencli douban subject <id>`, which is already wired for that data.

* fix(douban): fail fast on empty movie hot
2026-05-04 16:12:18 +08:00
jakevin 11ceca4fc2 fix(bilibili,reddit): add identifier and url columns to hot lists (#1284)
* fix(bilibili,reddit): add identifier and url columns to hot lists

Both `bilibili hot` and `reddit hot` previously dropped their per-row
identifier and URL on the way out, breaking the typical agent flow where
the next call needs a `bvid` / `postId` to fetch detail or comments.

- bilibili/hot: add `bvid` and `url` columns (constructed from bvid)
- reddit/hot: surface `postId`, `author`, `url` (already in evaluate but
  dropped in map)

Tested via local `~/.opencli/clis/<site>/hot.js` overrides.

* test(bilibili,reddit): lock hot list identifier columns
2026-05-04 16:10:49 +08:00
jakevin be5234ced1 fix(doctor): remove adapter analyze tip (#1283) 2026-05-04 13:37:13 +08:00
jakevin 1da105edea revert: offscreen daemon bridge
Revert PR #1280 and restore the previous Browser Bridge service-worker transport while PR #1229-style recovery messaging is pursued.
2026-05-03 23:14:37 +08:00
jakevin ca25f65bf7 fix(extension): move daemon bridge to offscreen document
Move the Browser Bridge daemon WebSocket out of the MV3 service worker and into an offscreen document. Remove the popup/action UI and obsolete extension log forwarding now that doctor is the diagnostic surface.
2026-05-03 22:20:54 +08:00
jakevin 0f29790795 feat(browser): add dialog handling and CDP DOM primitives (#1278)
* feat(browser): add dialog handling and CDP DOM primitives

* fix(browser): narrow dialog error detection
2026-05-03 21:28:30 +08:00
Kagura 98062a21c9 fix: isolate browser workspace per command
Fix concurrent browser-backed commands for the same site by using a unique workspace per command execution. Closes #1114.
2026-05-03 21:20:47 +08:00
jakevin a353db5fbe chore(cli): remove duplicate root help summary logic (#1277)
* chore(cli): remove duplicate root help summary logic

* chore(test): remove unused commander adapter imports
2026-05-03 20:06:37 +08:00
jakevin 1d407ab62f fix(cli): show adapter subcommands in root help (#1276)
* fix(cli): show adapter subcommands in root help

* fix(cli): summarize built-in root help groups
2026-05-03 19:55:59 +08:00
jakevin 1b19b3ebe9 chore: bump version to 1.7.11 (#1275)
Release / release (push) Has been cancelled
2026-05-03 19:35:22 +08:00
jakevin d60e1cf43d fix(browser): route type and keys through native input (#1274)
Fixes #1265 by routing browser type/keys through existing native CDP input primitives, with DOM fallbacks and direct CDPPage parity.
2026-05-03 19:32:31 +08:00
jakevin 1cd1253d46 feat(instagram): add collection-delete adapter
Pairs with the new collection-create adapter so users (and future
fixture-teardown logic) can clean up saved-post collections from CLI.

- POST /api/v1/collections/{id}/delete/ with multipart module_name=collection_settings
- Accepts collection name (case-insensitive) or numeric collection_id; resolves
  via /collections/list/ first so unknown / duplicate names error explicitly
  instead of bubbling up a 404 or silently deleting the wrong one.
2026-05-03 19:05:55 +08:00
jakevin 7869bdb2ca feat(browser): polish adapter author verify workflow 2026-05-03 19:03:38 +08:00
jakevin 2e93ac6e63 fix(release): build before manifest drift check (#1269) 2026-05-03 18:40:39 +08:00
jakevin de0d74bf62 fix(build-manifest): fail loud on import errors and refuse stale dist (#1268)
The previous implementation silently skipped any adapter whose import
failed (catch + warn-to-stderr + return []), then printed a successful
" Manifest compiled: N entries". When dist/ was stale (e.g. after
renaming an export the JS adapters re-import) every adapter using that
export would fail to load, get skipped, and the script still exited 0.
An agent reading exit codes to gate work would commit the resulting
manifest and silently delete dozens of unrelated adapter entries.

Three layers of defense:

1. Distinguish skip kinds. Files that don't call `cli(...)` are still
   silently dropped (helpers / type modules). Files that look like CLI
   modules but fail to import now throw `ManifestImportError`. The
   batch scanner aggregates failures and `main()` exits 1 with an
   explicit list, leaving the existing manifest on disk untouched.

2. Net-deletion safety net. `main()` diffs the new entries against the
   committed manifest and refuses to overwrite when entries would be
   removed. `--allow-removals=N` (or bare `--allow-removals` for any)
   is the explicit opt-in; the error message tells the caller exactly
   what value to pass.

3. Runtime dist guard. `node dist/src/build-manifest.js` now refuses
   to run with a clear pointer at `npm run build-manifest` (which uses
   tsx). The npm script itself is migrated to `tsx src/build-manifest.ts`
   so no project-level command points at the compiled copy anymore.

Release CI gains a manifest-drift gate (build-manifest + git diff
--exit-code) so a tag push can never publish stale or silently-shrunk
manifests. The existing CI check on PRs is preserved.

`ManifestEntry` is split into `src/manifest-types.ts` so runtime code
(discovery.ts) imports the type without pulling the build-time
compiler module.

Tests:
- `loadManifestEntries` throws ManifestImportError on import failure
- helper modules without cli() are still silently skipped
- `scanClisDir` aggregates per-adapter failures
- `diffRemovedEntries` returns expected site/name diff
- `parseBuildManifestArgs` reads --allow-removals[=N]
2026-05-03 18:27:23 +08:00
jakevin a9e0ca648f fix(extension): remove status-row left border accent (#1267)
WAWQAQ feedback: the green left border on the status row looked
disconnected — only on the top half of the card, creating an awkward
stub. Connection state is already conveyed clearly by the colored dot
and the "Connected to daemon" / "Disconnected" text, so the border was
redundant decoration.

Drop the .card.connected/.disconnected/.connecting border-left rules.
No JS or layout changes; cleaner surface, fewer visual variants.
2026-05-03 18:18:30 +08:00
jakevin bebc7aa35e chore: bump version to 1.7.10 (extension 1.0.4) (#1266)
Release / release (push) Has been cancelled
2026-05-03 18:00:35 +08:00
jakevin 061fba100d feat(extension): polish popup UI with merged card and copy contextId (#1262)
- Merge status row and profile row into a single rounded card with a
  brand-colored left border accent indicating connection state
- Render contextId inline next to a "Profile" label with a Copy button,
  letting users paste it into `opencli profile rename` without manual
  selection (replaces the old full-width code block treatment)
- Show daemon version inline in the status row when connected, and
  render the extension version as a tag in the popup header — both
  surface version information that helps diagnose stale-daemon issues
- Forward both versions through the existing `getStatus` background
  message: extension reads its own version from the manifest, daemon
  version is fetched best-effort from `/status` with a 1.5s timeout so
  popup never hangs when the daemon is unreachable
2026-05-03 17:37:51 +08:00
jakevin e364ec6b9c feat(browser): pass trace through verify (#1263) 2026-05-03 17:32:02 +08:00
jakevin 765eb56c99 feat(daemon): surface stale versions and restart (#1261) 2026-05-03 17:20:54 +08:00
jakevin 5f72770eff feat(instagram): add collection-create + collection filter for saved (#1192) (#1260)
Closes #1192. Two changes:

1. New `instagram collection-create <name>` adapter wraps
   `POST /api/v1/collections/create/` (multipart `name` +
   `module_name=collection_create`, X-IG-App-ID + X-CSRFToken).
2. `instagram saved` gains an optional `--collection <name>` flag.
   When set, the adapter resolves the name to a collection id via
   `/api/v1/collections/list/` (case-insensitive trim match) and then
   fetches `/api/v1/feed/collection/{id}/posts/`. Unknown names throw
   with the available list so callers can self-correct.

Both verified end-to-end against a live IG account. Verify fixtures
under ~/.opencli/sites/instagram/verify/ ship the
patterns/notEmpty/mustBeTruthy guards from the latest adapter-author
skill (success-rate-pitfalls §1, §4, §8).
2026-05-03 16:33:15 +08:00
jakevin 3017ca78aa chore: bump version to 1.7.9 (extension 1.0.3) (#1259)
Release / release (push) Has been cancelled
2026-05-03 15:46:48 +08:00
jakevin 7e68e19f0d feat(trace): prune retained artifacts (#1258) 2026-05-03 15:34:30 +08:00
jakevin 4ceb3314fe refactor(trace): retire diagnostic repair path (#1257)
* refactor(trace): retire diagnostic repair path

* chore(trace): clarify artifact summary guidance

* chore(trace): version trace receipt schema
2026-05-03 15:19:44 +08:00
Jack He 5f0cce7b22 feat(weibo): add favorites + publish CLI commands (#1253)
* feat(weibo): add favorites + publish CLI commands

Consolidates #1253 (favorites) and #1254 (publish) into a single PR per maintainer request.

- clis/weibo/favorites.ts: cookie-mode fetch of authenticated user's favorites via weibo.com/u/page/fav/{uid}
- clis/weibo/publish.js: UI-automation post (text up to 2000 chars, up to 9 images jpg/png/gif/webp)
- cli-manifest.json regenerated to include the new commands

Note: favorites.ts uses TypeScript syntax but build-manifest.js scans only *.js — favorites is currently NOT registered in the manifest. Reviewers please check whether to rename to .js or whether the manifest scanner should learn .ts.

Authored-by: hszhsz <heshaoz1990@gmail.com>

* fix(weibo): harden favorites and publish commands

* fix(weibo): publish without execute gate

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-03 15:00:38 +08:00
Benjamin Liu 284c96133b feat(claude): add Claude adapter (#1252)
* feat(claude): add Claude adapter

Adds a Claude (claude.ai) browser adapter family with seven commands
modeled on the existing clis/deepseek/ pattern: ask, send, new, status,
read, history, detail.

Closes #1251

* feat(claude): align send command columns with doubao

Match the established Status / SubmittedBy / InjectedText shape used by
doubao send so agent loops can rely on a consistent fire-and-forget
output across AI chat adapters.

* fix(claude): preserve DOM order in getVisibleMessages

The previous implementation queried user-message and assistant-message
nodes in two passes, which serialized as [u1, u2, u3, a1, a2, a3] for
multi-turn chats instead of the correct conversation order. Single
combined query preserves DOM order so claude read / detail return
turns in the order the user reads them on the page.

* docs(claude): note --live requirement for read across invocations

* fix(claude): fail fast on auth and empty states

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-03 14:55:41 +08:00
jakevin eac17b361e feat(observation): add runtime trace capture (#1255) 2026-05-03 14:38:59 +08:00
jakevin aa33262ef7 docs: narrow smart-search trigger description (#1248) 2026-05-02 16:51:17 +08:00
jakevin a0b2df1448 docs: refresh stale entry and developer docs (#1244) 2026-05-02 12:31:48 +08:00
jakevin fc7245f9f6 chore: enforce node 21 baseline (#1242) 2026-05-02 09:30:28 +08:00
jakevin 88bcd814ee refactor: simplify diagnostics and low-use errors (#1241) 2026-05-02 09:28:26 +08:00
jakevin 2fd7272559 docs: clarify opencli extension paths (#1240) 2026-05-02 09:27:17 +08:00
166 changed files with 10461 additions and 1601 deletions
+12 -1
View File
@@ -26,6 +26,17 @@ jobs:
- name: Type check
run: npx tsc --noEmit
# Build before the manifest drift gate: adapter modules import
# @jackwener/opencli/* through package exports, which resolve to dist/.
# A fresh release checkout has no dist/ until the full build runs.
- name: Build package and verify cli-manifest.json is up-to-date
run: |
npm run build
if ! git diff --exit-code -- cli-manifest.json; then
echo "::error::cli-manifest.json drift detected at release time. Run 'npm run build' locally and commit the result before tagging."
exit 1
fi
- name: Install extension dependencies
run: npm ci
working-directory: extension
@@ -40,7 +51,7 @@ jobs:
- name: Create extension ZIP
run: |
EXT_VERSION=$(node -p "require('./extension/package.json').version")
EXT_VERSION=$(jq -r .version extension/package.json)
cd extension-package
zip -r ../opencli-extension-v${EXT_VERSION}.zip .
+2
View File
@@ -4,6 +4,8 @@
### Features
* **observation** — add trace artifact primitives, `browser console`, `browser network --since/--follow/--failed`, and adapter `--trace=retain-on-failure` for failure-retained browser evidence.
* **autofix** — retire `OPENCLI_DIAGNOSTIC`; adapter repair now uses `--trace retain-on-failure`, trace `summary.md`, and error-envelope trace metadata.
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
+18 -6
View File
@@ -21,7 +21,7 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type, extract, screenshot — all through your logged-in Chrome session.
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
- **Website → CLI** — Turn any website into a deterministic CLI: 90+ pre-built adapters, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Website → CLI** — Turn any website into a deterministic CLI: 100+ site surfaces are already registered, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
- **CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, docker, obsidian, etc).
@@ -89,6 +89,18 @@ Use OpenCLI directly when you want a reliable command instead of a live browser
- `opencli external register mycli` exposes a local CLI through the same discovery surface.
- `opencli doctor` helps diagnose browser connectivity.
## Extending OpenCLI
If you want to add your own commands, start with the [Extending OpenCLI guide](./docs/guide/extending-opencli.md). README keeps this short; the guide covers the directory layout, source-control model, and install commands.
| Need | Recommended path |
|------|------------------|
| Keep personal website commands in your own Git repo | `opencli plugin create` + `opencli plugin install file://...` |
| Quickly draft a private local adapter | `opencli browser init <site>/<command>` in `~/.opencli/clis/` |
| Modify an official adapter locally | `opencli adapter eject <site>` + `opencli adapter reset <site>` |
| Publish or install third-party commands | `opencli plugin install github:user/repo` |
| Wrap an existing local binary | `opencli external register <name>` |
## For AI Agents
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
@@ -162,7 +174,7 @@ When the site you need is not yet covered, use the `opencli-adapter-author` skil
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`.
4. Decode response fields and design output columns.
5. `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
5. `opencli browser analyze <url>` for one-shot recon, then `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
### CLI Hub and desktop adapters
@@ -193,7 +205,6 @@ OpenCLI is not only for websites. It can also:
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
| `OPENCLI_CDP_TARGET` | — | Filter CDP targets by URL substring (e.g. `detail.1688.com`) |
| `OPENCLI_VERBOSE` | `false` | Enable verbose logging (`-v` flag also works) |
| `OPENCLI_DIAGNOSTIC` | `false` | Set to `1` to capture structured diagnostic context on failures |
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation lease open until you run `opencli browser close` or the idle timeout expires.
@@ -249,6 +260,7 @@ To load the source Browser Bridge extension:
| **1688** | `search` `item` `assets` `download` `store` |
| **gitee** | `trending` `search` `user` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **yuanbao** | `new` `ask` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` |
@@ -265,7 +277,7 @@ To load the source Browser Bridge extension:
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **xiaoyuzhou** | `auth*` `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` |
90+ adapters in total — **[→ see all supported sites & commands](./docs/adapters/index.md)**
100+ site surfaces in total — **[→ see all supported sites & commands](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast`, `podcast-episodes`, `episode`, `download`, and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
@@ -395,10 +407,10 @@ Before writing any adapter code, read the [`opencli-adapter-author` skill](./ski
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
- Discover the right endpoint via `opencli browser network`, `eval`, or the interceptor fallback.
- Decide auth strategy (`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`).
- Decode response fields, design columns, scaffold with `opencli browser init`.
- Run `opencli browser analyze <url>` for one-shot recon, decode response fields, design columns, scaffold with `opencli browser init`.
- Verify with `opencli browser verify <site>/<name>` before shipping.
Adapters you write outside the repo live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
For long-lived personal commands that should live in your own Git repo, use a local plugin instead; see [Extending OpenCLI](./docs/guide/extending-opencli.md). Quick private adapters can still live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
## Testing
+20 -8
View File
@@ -10,7 +10,7 @@
OpenCLI 可以用同一套 CLI 做三类事情:
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [90+ 站点](#内置命令) 开箱即用。
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [100+ 站点](#内置命令) 开箱即用。
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-adapter-author` skill,Agent 就能用你的已登录浏览器导航、点击、输入、提取任意网页内容。
- **把新网站写成 CLI**:用 `opencli browser` 原语 + `opencli-adapter-author` skill,从站点侦察、API 发现、字段解码到 `opencli browser verify` 一条龙。
@@ -20,7 +20,7 @@ OpenCLI 可以用同一套 CLI 做三类事情:
- **桌面应用控制** — 通过 CDP 直接在终端驱动 Electron 应用(Cursor、Codex、ChatGPT、Notion 等)。
- **AI Agent 浏览器自动化** — 安装 `opencli-adapter-author` skill,你的 AI Agent 就能操作任意网站:导航、点击、输入、提取、截图——全部通过你的已登录 Chrome 会话完成。
- **网站 → CLI** — 把任何网站变成确定性 CLI:90+ 内置适配器,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
- **网站 → CLI** — 把任何网站变成确定性 CLI:100+ 站点能力已注册,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
- **账号安全** — 复用 Chrome/Chromium 登录态,凭证永远不会离开浏览器。
- **面向 AI Agent** — 一个 skill 带你走完站点侦察、API 发现、字段解码、适配器编写、验证的全流程。
- **CLI 枢纽** — 统一发现、自动安装、纯透传任何外部 CLI(gh、docker、obsidian 等)。
@@ -70,9 +70,21 @@ opencli bilibili hot --limit 5
- `opencli list` 查看当前所有命令
- `opencli <site> <command>` 调用内置或生成好的适配器
- `opencli register mycli` 把本地 CLI 接入同一发现入口
- `opencli external register mycli` 把本地 CLI 接入同一发现入口
- `opencli doctor` 处理浏览器连通性问题
## 扩展 OpenCLI
如果你想新增自己的命令,先看 [扩展 OpenCLI](./docs/zh/guide/extending-opencli.md)。README 只保留入口;目录结构、源码管理方式和安装命令放在文档里。
| 需求 | 推荐路径 |
|------|----------|
| 把个人网站命令放在自己的 Git repo | `opencli plugin create` + `opencli plugin install file://...` |
| 快速写一个本机私人 adapter | `opencli browser init <site>/<command>`,放在 `~/.opencli/clis/` |
| 本地修改官方 adapter | `opencli adapter eject <site>` + `opencli adapter reset <site>` |
| 发布或安装第三方命令 | `opencli plugin install github:user/repo` |
| 包装已有本机 binary | `opencli external register <name>` |
## 给 AI Agent
OpenCLI 的 browser 命令是给 AI Agent 用的——不是手动执行的。把 skill 安装到你的 AI AgentClaude Code、Cursor 等)中,Agent 就能用你的已登录 Chrome 会话替你操作网站。
@@ -146,7 +158,7 @@ Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. 定认证策略——`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
4. 字段解码 + 设计输出列
5. `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
5. `opencli browser analyze <url>` 一步侦察,再 `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
### CLI 枢纽与桌面端适配器
@@ -176,7 +188,6 @@ OpenCLI 不只是网站 CLI,还可以:
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol 端点,用于远程浏览器或 Electron 应用 |
| `OPENCLI_CDP_TARGET` | — | 按 URL 子串过滤 CDP target(如 `detail.1688.com` |
| `OPENCLI_VERBOSE` | `false` | 启用详细日志(`-v` 也可以) |
| `OPENCLI_DIAGNOSTIC` | `false` | 设为 `1` 时在失败时输出结构化诊断上下文 |
| `DEBUG_SNAPSHOT` | — | 设为 `1` 输出 DOM 快照调试信息 |
`--focus` 同时适用于 `opencli browser *` 和浏览器型 adapter 命令。`--live` 主要是给 adapter 命令用的:`browser` 子命令本来就会一直保留 automation window,直到你手动执行 `opencli browser close` 或等空闲超时。
@@ -289,13 +300,14 @@ npm link
| **1688** | `search` `item` `assets` `download` `store` | 浏览器 |
| **gitee** | `trending` `search` `user` | 公开 / 浏览器 |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` | 浏览器 |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` | 浏览器 |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` | OAuth API |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` | 浏览器 |
| **36kr** | `news` `hot` `search` `article` | 公开 / 浏览器 |
| **imdb** | `search` `title` `top` `trending` `person` `reviews` | 公开 |
| **producthunt** | `posts` `today` `hot` `browse` | 公开 / 浏览器 |
| **instagram** | `explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved` | 浏览器 |
| **lobsters** | `hot` `newest` `active` `tag` | 公开 |
| **lobsters** | `hot` `newest` `active` `tag` `read` | 公开 |
| **medium** | `feed` `search` `user` | 浏览器 |
| **sinablog** | `hot` `search` `article` `user` | 浏览器 |
| **substack** | `feed` `search` `publication` | 浏览器 |
@@ -306,7 +318,7 @@ npm link
| **douyin** | `videos` `publish` `drafts` `draft` `delete` `stats` `profile` `update` `hashtag` `location` `activities` `collections` | 浏览器 |
| **yuanbao** | `new` `ask` | 浏览器 |
90+ 适配器**[→ 查看完整命令列表](./docs/adapters/index.md)**
100+ 站点能力**[→ 查看完整命令列表](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast``podcast-episodes``episode``download``transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`
@@ -493,7 +505,7 @@ opencli plugin uninstall my-tool # 卸载
- 侦察站点,选定 patternSPA / SSR / JSONP / Token / Streaming
-`opencli browser network``eval`、interceptor 等找到目标 endpoint
- 定认证策略(`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
- 字段解码、设计 columns、`opencli browser init` 生成骨架
- 先用 `opencli browser analyze <url>` 一步侦察,再字段解码、设计 columns、`opencli browser init` 生成骨架
- 交付前用 `opencli browser verify <site>/<name>` 验证
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。
+643 -19
View File
@@ -1038,13 +1038,52 @@
"title",
"authors",
"published",
"updated",
"primary_category",
"categories",
"abstract",
"comment",
"pdf",
"url"
],
"type": "js",
"modulePath": "arxiv/paper.js",
"sourceFile": "arxiv/paper.js"
},
{
"site": "arxiv",
"name": "recent",
"description": "List recent arXiv submissions in a category",
"strategy": "public",
"browser": false,
"args": [
{
"name": "category",
"type": "str",
"required": true,
"positional": true,
"help": "arXiv category (e.g. cs.CL, cs.LG, math.PR, q-bio.NC)"
},
{
"name": "limit",
"type": "int",
"default": 10,
"required": false,
"help": "Max results (max 50)"
}
],
"columns": [
"id",
"title",
"authors",
"published",
"primary_category",
"url"
],
"type": "js",
"modulePath": "arxiv/recent.js",
"sourceFile": "arxiv/recent.js"
},
{
"site": "arxiv",
"name": "search",
@@ -1072,6 +1111,7 @@
"title",
"authors",
"published",
"primary_category",
"url"
],
"type": "js",
@@ -1787,7 +1827,9 @@
"title",
"author",
"play",
"danmaku"
"danmaku",
"bvid",
"url"
],
"type": "js",
"modulePath": "bilibili/hot.js",
@@ -3859,6 +3901,208 @@
"sourceFile": "chatwise/send.js",
"navigateBefore": true
},
{
"site": "claude",
"name": "ask",
"description": "Send a prompt to Claude and get the response",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "prompt",
"type": "str",
"required": true,
"positional": true,
"help": "Prompt to send"
},
{
"name": "timeout",
"type": "int",
"default": 120,
"required": false,
"help": "Max seconds to wait for response"
},
{
"name": "new",
"type": "boolean",
"default": false,
"required": false,
"help": "Start a new chat before sending"
},
{
"name": "model",
"type": "str",
"default": "sonnet",
"required": false,
"help": "Model to use: sonnet, opus, or haiku",
"choices": [
"sonnet",
"opus",
"haiku"
]
},
{
"name": "think",
"type": "boolean",
"default": false,
"required": false,
"help": "Enable Adaptive thinking"
},
{
"name": "file",
"type": "str",
"required": false,
"help": "Attach a file (image, PDF, text) with the prompt"
}
],
"columns": [
"response"
],
"timeout": 180,
"type": "js",
"modulePath": "claude/ask.js",
"sourceFile": "claude/ask.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "detail",
"description": "Open a Claude conversation by ID and read its messages",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "Conversation ID (UUID from /chat/<id>)"
}
],
"columns": [
"Index",
"Role",
"Text"
],
"type": "js",
"modulePath": "claude/detail.js",
"sourceFile": "claude/detail.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "history",
"description": "List conversation history from Claude /recents",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "Max conversations to show"
}
],
"columns": [
"Index",
"Id",
"Title",
"Url"
],
"type": "js",
"modulePath": "claude/history.js",
"sourceFile": "claude/history.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "new",
"description": "Start a new conversation in Claude",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [],
"columns": [
"Status"
],
"type": "js",
"modulePath": "claude/new.js",
"sourceFile": "claude/new.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "read",
"description": "Read the current Claude conversation",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [],
"columns": [
"Index",
"Role",
"Text"
],
"type": "js",
"modulePath": "claude/read.js",
"sourceFile": "claude/read.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "send",
"description": "Send a prompt to Claude without waiting for the response",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "prompt",
"type": "str",
"required": true,
"positional": true,
"help": "Prompt to send"
},
{
"name": "new",
"type": "boolean",
"default": false,
"required": false,
"help": "Start a new chat before sending"
}
],
"columns": [
"Status",
"SubmittedBy",
"InjectedText"
],
"type": "js",
"modulePath": "claude/send.js",
"sourceFile": "claude/send.js",
"navigateBefore": false
},
{
"site": "claude",
"name": "status",
"description": "Check Claude page availability and login state",
"domain": "claude.ai",
"strategy": "cookie",
"browser": true,
"args": [],
"columns": [
"Status",
"Login",
"Url"
],
"type": "js",
"modulePath": "claude/status.js",
"sourceFile": "claude/status.js",
"navigateBefore": false
},
{
"site": "cnki",
"name": "search",
@@ -4499,6 +4743,44 @@
"sourceFile": "deepseek/status.js",
"navigateBefore": false
},
{
"site": "devto",
"name": "read",
"description": "Read a DEV.to article body by id",
"domain": "dev.to",
"strategy": "public",
"browser": false,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "DEV.to article id (numeric, e.g. 3605688)"
},
{
"name": "max-length",
"type": "int",
"default": 20000,
"required": false,
"help": "Max characters of body to return (min 100)"
}
],
"columns": [
"id",
"title",
"author",
"reactions",
"reading_time",
"tags",
"published_at",
"body",
"url"
],
"type": "js",
"modulePath": "devto/read.js",
"sourceFile": "devto/read.js"
},
{
"site": "devto",
"name": "tag",
@@ -4524,11 +4806,15 @@
],
"columns": [
"rank",
"id",
"title",
"author",
"reactions",
"comments",
"tags"
"reading_time",
"published_at",
"tags",
"url"
],
"type": "js",
"modulePath": "devto/tag.js",
@@ -4552,11 +4838,15 @@
],
"columns": [
"rank",
"id",
"title",
"author",
"reactions",
"comments",
"tags"
"reading_time",
"published_at",
"tags",
"url"
],
"type": "js",
"modulePath": "devto/top.js",
@@ -4587,10 +4877,14 @@
],
"columns": [
"rank",
"id",
"title",
"reactions",
"comments",
"tags"
"reading_time",
"published_at",
"tags",
"url"
],
"type": "js",
"modulePath": "devto/user.js",
@@ -4994,12 +5288,11 @@
],
"columns": [
"rank",
"id",
"title",
"rating",
"quote",
"director",
"votes",
"year",
"region",
"url"
],
"type": "js",
@@ -7766,10 +8059,12 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments"
"comments",
"url"
],
"type": "js",
"modulePath": "hackernews/ask.js",
@@ -7793,10 +8088,12 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments"
"comments",
"url"
],
"type": "js",
"modulePath": "hackernews/best.js",
@@ -7820,6 +8117,7 @@
],
"columns": [
"rank",
"id",
"title",
"author",
"url"
@@ -7846,15 +8144,71 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments"
"comments",
"url"
],
"type": "js",
"modulePath": "hackernews/new.js",
"sourceFile": "hackernews/new.js"
},
{
"site": "hackernews",
"name": "read",
"description": "Read a Hacker News story and its comment tree",
"domain": "news.ycombinator.com",
"strategy": "public",
"browser": false,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "HN item ID (e.g. 39847301)"
},
{
"name": "limit",
"type": "int",
"default": 25,
"required": false,
"help": "Max top-level comments"
},
{
"name": "depth",
"type": "int",
"default": 2,
"required": false,
"help": "Max reply depth (1=no replies, 2=one level of replies, etc.)"
},
{
"name": "replies",
"type": "int",
"default": 5,
"required": false,
"help": "Max replies shown per comment at each level"
},
{
"name": "max-length",
"type": "int",
"default": 2000,
"required": false,
"help": "Max characters per comment body (min 100)"
}
],
"columns": [
"type",
"author",
"score",
"text"
],
"type": "js",
"modulePath": "hackernews/read.js",
"sourceFile": "hackernews/read.js"
},
{
"site": "hackernews",
"name": "search",
@@ -7891,6 +8245,7 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
@@ -7919,10 +8274,12 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments"
"comments",
"url"
],
"type": "js",
"modulePath": "hackernews/show.js",
@@ -7946,10 +8303,12 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments"
"comments",
"url"
],
"type": "js",
"modulePath": "hackernews/top.js",
@@ -8490,6 +8849,59 @@
"modulePath": "imdb/trending.js",
"sourceFile": "imdb/trending.js"
},
{
"site": "instagram",
"name": "collection-create",
"description": "Create a new Instagram saved-posts collection (folder)",
"domain": "www.instagram.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "name",
"type": "str",
"required": true,
"positional": true,
"help": "Name of the collection to create"
}
],
"columns": [
"status",
"collectionId",
"collectionName",
"mediaCount"
],
"type": "js",
"modulePath": "instagram/collection-create.js",
"sourceFile": "instagram/collection-create.js",
"navigateBefore": "https://www.instagram.com"
},
{
"site": "instagram",
"name": "collection-delete",
"description": "Delete an Instagram saved-posts collection (folder) by name or id",
"domain": "www.instagram.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "target",
"type": "str",
"required": true,
"positional": true,
"help": "Collection name (case-insensitive) or numeric collection_id"
}
],
"columns": [
"status",
"collectionId",
"collectionName"
],
"type": "js",
"modulePath": "instagram/collection-delete.js",
"sourceFile": "instagram/collection-delete.js",
"navigateBefore": "https://www.instagram.com"
},
{
"site": "instagram",
"name": "comment",
@@ -8876,7 +9288,7 @@
{
"site": "instagram",
"name": "saved",
"description": "Get your saved Instagram posts",
"description": "Get your saved Instagram posts (optionally from a specific collection)",
"domain": "www.instagram.com",
"strategy": "cookie",
"browser": true,
@@ -8887,6 +9299,12 @@
"default": 20,
"required": false,
"help": "Number of saved posts"
},
{
"name": "collection",
"type": "str",
"required": false,
"help": "Collection name (case-insensitive). Omit for the default \"All posts\" feed."
}
],
"columns": [
@@ -10985,11 +11403,14 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments",
"tags"
"created_at",
"tags",
"url"
],
"type": "js",
"modulePath": "lobsters/active.js",
@@ -11013,11 +11434,14 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments",
"tags"
"created_at",
"tags",
"url"
],
"type": "js",
"modulePath": "lobsters/hot.js",
@@ -11041,16 +11465,73 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments",
"tags"
"created_at",
"tags",
"url"
],
"type": "js",
"modulePath": "lobsters/newest.js",
"sourceFile": "lobsters/newest.js"
},
{
"site": "lobsters",
"name": "read",
"description": "Read a Lobste.rs story and its comment tree",
"domain": "lobste.rs",
"strategy": "public",
"browser": false,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "Lobste.rs short_id (e.g. 6cmh6h)"
},
{
"name": "limit",
"type": "int",
"default": 25,
"required": false,
"help": "Max top-level comments"
},
{
"name": "depth",
"type": "int",
"default": 2,
"required": false,
"help": "Max reply depth (1=no replies, 2=one level of replies, etc.)"
},
{
"name": "replies",
"type": "int",
"default": 5,
"required": false,
"help": "Max replies shown per comment at each level"
},
{
"name": "max-length",
"type": "int",
"default": 2000,
"required": false,
"help": "Max characters per comment body (min 100)"
}
],
"columns": [
"type",
"author",
"score",
"text"
],
"type": "js",
"modulePath": "lobsters/read.js",
"sourceFile": "lobsters/read.js"
},
{
"site": "lobsters",
"name": "tag",
@@ -11076,11 +11557,14 @@
],
"columns": [
"rank",
"id",
"title",
"score",
"author",
"comments",
"tags"
"created_at",
"tags",
"url"
],
"type": "js",
"modulePath": "lobsters/tag.js",
@@ -13689,7 +14173,10 @@
"title",
"subreddit",
"score",
"comments"
"comments",
"postId",
"author",
"url"
],
"type": "js",
"modulePath": "reddit/hot.js",
@@ -14717,10 +15204,17 @@
}
],
"columns": [
"rank",
"id",
"bounty",
"title",
"score",
"answers",
"views",
"is_answered",
"tags",
"author",
"creation_date",
"url"
],
"type": "js",
@@ -14744,15 +15238,70 @@
}
],
"columns": [
"rank",
"id",
"title",
"score",
"answers",
"views",
"is_answered",
"tags",
"author",
"creation_date",
"url"
],
"type": "js",
"modulePath": "stackoverflow/hot.js",
"sourceFile": "stackoverflow/hot.js"
},
{
"site": "stackoverflow",
"name": "read",
"description": "Read a Stack Overflow question with answers and comments",
"domain": "stackoverflow.com",
"strategy": "public",
"browser": false,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "Stack Overflow question id (numeric, e.g. 79935770)"
},
{
"name": "answers-limit",
"type": "int",
"default": 10,
"required": false,
"help": "Max answers to include (1-100; accepted answer always included first)"
},
{
"name": "comments-limit",
"type": "int",
"default": 5,
"required": false,
"help": "Max comments per question/answer (1-100)"
},
{
"name": "max-length",
"type": "int",
"default": 4000,
"required": false,
"help": "Max characters per body / answer / comment (min 100)"
}
],
"columns": [
"type",
"author",
"score",
"accepted",
"text"
],
"type": "js",
"modulePath": "stackoverflow/read.js",
"sourceFile": "stackoverflow/read.js"
},
{
"site": "stackoverflow",
"name": "search",
@@ -14777,9 +15326,16 @@
}
],
"columns": [
"rank",
"id",
"title",
"score",
"answers",
"views",
"is_answered",
"tags",
"author",
"creation_date",
"url"
],
"type": "js",
@@ -14803,9 +15359,15 @@
}
],
"columns": [
"rank",
"id",
"title",
"score",
"answers",
"views",
"tags",
"author",
"creation_date",
"url"
],
"type": "js",
@@ -16670,7 +17232,6 @@
"columns": [
"rank",
"topic",
"tweets",
"category"
],
"type": "js",
@@ -17343,6 +17904,37 @@
"sourceFile": "weibo/comments.js",
"navigateBefore": "https://weibo.com"
},
{
"site": "weibo",
"name": "favorites",
"description": "我的微博收藏列表",
"domain": "weibo.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "数量(最多50"
}
],
"columns": [
"author",
"text",
"time",
"source",
"likes",
"comments",
"reposts",
"url"
],
"type": "js",
"modulePath": "weibo/favorites.js",
"sourceFile": "weibo/favorites.js",
"navigateBefore": "https://weibo.com"
},
{
"site": "weibo",
"name": "feed",
@@ -17460,6 +18052,38 @@
"sourceFile": "weibo/post.js",
"navigateBefore": "https://weibo.com"
},
{
"site": "weibo",
"name": "publish",
"description": "Publish a new Weibo post immediately",
"domain": "weibo.com",
"strategy": "ui",
"browser": true,
"args": [
{
"name": "text",
"type": "string",
"required": true,
"positional": true,
"help": "Weibo text content (max 2000 chars)"
},
{
"name": "images",
"type": "string",
"required": false,
"help": "Image paths, comma-separated, max 9 (jpg/png/gif/webp)"
}
],
"columns": [
"status",
"message",
"text"
],
"type": "js",
"modulePath": "weibo/publish.js",
"sourceFile": "weibo/publish.js",
"navigateBefore": true
},
{
"site": "weibo",
"name": "search",
+112
View File
@@ -0,0 +1,112 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { normalizeArxivCategory, normalizeArxivLimit, parseEntries } from './utils.js';
import './paper.js';
import './search.js';
import './recent.js';
const SAMPLE_ENTRY_XML = `<?xml version='1.0' encoding='UTF-8'?>
<feed xmlns:opensearch="http://a9.com/-/spec/opensearch/1.1/"
xmlns:arxiv="http://arxiv.org/schemas/atom"
xmlns="http://www.w3.org/2005/Atom">
<entry>
<id>http://arxiv.org/abs/1706.03762v7</id>
<title>Attention Is All You Need &amp; Friends</title>
<updated>2023-08-02T00:41:18Z</updated>
<link href="https://arxiv.org/abs/1706.03762v7" rel="alternate" type="text/html"/>
<link href="https://arxiv.org/pdf/1706.03762v7" rel="related" type="application/pdf" title="pdf"/>
<summary>The dominant sequence transduction models are based on complex recurrent or convolutional neural networks. We propose a new simple network architecture, the Transformer, based solely on attention.</summary>
<category term="cs.CL" scheme="http://arxiv.org/schemas/atom"/>
<category term="cs.LG" scheme="http://arxiv.org/schemas/atom"/>
<published>2017-06-12T17:57:34Z</published>
<arxiv:comment>15 pages, 5 figures</arxiv:comment>
<arxiv:primary_category term="cs.CL"/>
<author><name>Ashish Vaswani</name></author>
<author><name>Noam Shazeer</name></author>
<author><name>Niki Parmar</name></author>
<author><name>Jakob Uszkoreit</name></author>
<author><name>Llion Jones</name></author>
<author><name>Aidan N. Gomez</name></author>
<author><name>Lukasz Kaiser</name></author>
<author><name>Illia Polosukhin</name></author>
</entry>
</feed>`;
describe('arxiv adapter', () => {
it('registers paper, search and recent commands with the expected columns', () => {
const paper = getRegistry().get('arxiv/paper');
const search = getRegistry().get('arxiv/search');
const recent = getRegistry().get('arxiv/recent');
expect(paper).toBeDefined();
expect(search).toBeDefined();
expect(recent).toBeDefined();
expect(paper.columns).toEqual([
'id', 'title', 'authors', 'published', 'updated',
'primary_category', 'categories', 'abstract', 'comment', 'pdf', 'url',
]);
expect(search.columns).toEqual([
'id', 'title', 'authors', 'published', 'primary_category', 'url',
]);
expect(recent.columns).toEqual([
'id', 'title', 'authors', 'published', 'primary_category', 'url',
]);
});
it('parseEntries returns full abstract, all authors, pdf, primary category and comment', () => {
const [entry] = parseEntries(SAMPLE_ENTRY_XML);
expect(entry.id).toBe('1706.03762');
expect(entry.title).toBe('Attention Is All You Need & Friends');
// All 8 authors must be present — earlier impl truncated to 3.
expect(entry.authors.split(', ')).toHaveLength(8);
expect(entry.authors).toContain('Ashish Vaswani');
expect(entry.authors).toContain('Illia Polosukhin');
// Full abstract — earlier impl truncated at 200 chars.
expect(entry.abstract.length).toBeGreaterThan(140);
expect(entry.abstract.endsWith('...')).toBe(false);
expect(entry.abstract).toContain('attention');
expect(entry.published).toBe('2017-06-12');
expect(entry.updated).toBe('2023-08-02');
expect(entry.primary_category).toBe('cs.CL');
expect(entry.categories).toBe('cs.CL, cs.LG');
expect(entry.comment).toBe('15 pages, 5 figures');
expect(entry.pdf).toBe('https://arxiv.org/pdf/1706.03762v7');
expect(entry.url).toBe('https://arxiv.org/abs/1706.03762');
});
it('parseEntries returns an empty list for feeds with no entries', () => {
expect(parseEntries('<feed></feed>')).toEqual([]);
});
it('recent rejects malformed category strings', async () => {
const recent = getRegistry().get('arxiv/recent');
await expect(recent.func({ category: 'not a category', limit: 5 })).rejects.toMatchObject({
code: 'ARGUMENT',
});
await expect(recent.func({ category: '', limit: 5 })).rejects.toMatchObject({
code: 'ARGUMENT',
});
});
it('category validation accepts real arXiv archive and subcategory forms', () => {
expect(normalizeArxivCategory('cs.CL')).toBe('cs.CL');
expect(normalizeArxivCategory('math')).toBe('math');
expect(normalizeArxivCategory('physics.comp-ph')).toBe('physics.comp-ph');
expect(normalizeArxivCategory('physics.data-an')).toBe('physics.data-an');
expect(normalizeArxivCategory('cond-mat.soft')).toBe('cond-mat.soft');
expect(normalizeArxivCategory('q-bio.NC')).toBe('q-bio.NC');
expect(() => normalizeArxivCategory('not a category')).toThrow('Invalid arXiv category');
expect(() => normalizeArxivCategory('cs/CL')).toThrow('Invalid arXiv category');
expect(() => normalizeArxivCategory('')).toThrow('Invalid arXiv category');
});
it('limit validation rejects non-positive, non-integer and over-cap values', () => {
expect(normalizeArxivLimit(10, 5, 25)).toBe(10);
expect(normalizeArxivLimit(undefined, 5, 25)).toBe(5);
expect(() => normalizeArxivLimit(0, 5, 25)).toThrow('positive integer');
expect(() => normalizeArxivLimit(1.5, 5, 25)).toThrow('positive integer');
expect(() => normalizeArxivLimit(26, 5, 25)).toThrow('<= 25');
});
});
+3 -3
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { arxivFetch, parseEntries } from './utils.js';
cli({
site: 'arxiv',
@@ -10,12 +10,12 @@ cli({
args: [
{ name: 'id', positional: true, required: true, help: 'arXiv paper ID (e.g. 1706.03762)' },
],
columns: ['id', 'title', 'authors', 'published', 'abstract', 'url'],
columns: ['id', 'title', 'authors', 'published', 'updated', 'primary_category', 'categories', 'abstract', 'comment', 'pdf', 'url'],
func: async (args) => {
const xml = await arxivFetch(`id_list=${encodeURIComponent(args.id)}`);
const entries = parseEntries(xml);
if (!entries.length)
throw new CliError('NOT_FOUND', `Paper ${args.id} not found`, 'Check the arXiv ID format, e.g. 1706.03762');
throw new EmptyResultError('arxiv paper', `Paper ${args.id} was not found. Check the arXiv ID format, e.g. 1706.03762`);
return entries;
},
});
+32
View File
@@ -0,0 +1,32 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { arxivFetch, normalizeArxivCategory, normalizeArxivLimit, parseEntries } from './utils.js';
cli({
site: 'arxiv',
name: 'recent',
description: 'List recent arXiv submissions in a category',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'category', positional: true, required: true, help: 'arXiv category (e.g. cs.CL, cs.LG, math.PR, q-bio.NC)' },
{ name: 'limit', type: 'int', default: 10, help: 'Max results (max 50)' },
],
columns: ['id', 'title', 'authors', 'published', 'primary_category', 'url'],
func: async (args) => {
const category = normalizeArxivCategory(args.category);
const limit = normalizeArxivLimit(args.limit, 10, 50);
const query = encodeURIComponent(`cat:${category}`);
const xml = await arxivFetch(`search_query=${query}&max_results=${limit}&sortBy=submittedDate&sortOrder=descending`);
const entries = parseEntries(xml);
if (!entries.length)
throw new EmptyResultError('arxiv', `No recent papers in ${category}. Check the category name.`);
return entries.map(e => ({
id: e.id,
title: e.title,
authors: e.authors,
published: e.published,
primary_category: e.primary_category,
url: e.url,
}));
},
});
+18 -7
View File
@@ -1,6 +1,6 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import { arxivFetch, parseEntries } from './utils.js';
import { ArgumentError, EmptyResultError } from '@jackwener/opencli/errors';
import { arxivFetch, normalizeArxivLimit, parseEntries } from './utils.js';
cli({
site: 'arxiv',
name: 'search',
@@ -11,14 +11,25 @@ cli({
{ name: 'query', positional: true, required: true, help: 'Search keyword (e.g. "attention is all you need")' },
{ name: 'limit', type: 'int', default: 10, help: 'Max results (max 25)' },
],
columns: ['id', 'title', 'authors', 'published', 'url'],
columns: ['id', 'title', 'authors', 'published', 'primary_category', 'url'],
func: async (args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 25));
const query = encodeURIComponent(`all:${args.query}`);
const queryText = String(args.query || '').trim();
if (!queryText) {
throw new ArgumentError('arxiv search query cannot be empty');
}
const limit = normalizeArxivLimit(args.limit, 10, 25);
const query = encodeURIComponent(`all:${queryText}`);
const xml = await arxivFetch(`search_query=${query}&max_results=${limit}&sortBy=relevance`);
const entries = parseEntries(xml);
if (!entries.length)
throw new CliError('NOT_FOUND', 'No papers found', 'Try a different keyword');
return entries.map(e => ({ id: e.id, title: e.title, authors: e.authors, published: e.published, url: e.url }));
throw new EmptyResultError('arxiv', 'No papers found. Try a different keyword.');
return entries.map(e => ({
id: e.id,
title: e.title,
authors: e.authors,
published: e.published,
primary_category: e.primary_category,
url: e.url,
}));
},
});
+68 -5
View File
@@ -4,15 +4,44 @@
* arXiv exposes a public Atom/XML API — no key required.
* https://info.arxiv.org/help/api/index.html
*/
import { CliError } from '@jackwener/opencli/errors';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
export const ARXIV_BASE = 'https://export.arxiv.org/api/query';
const ARXIV_CATEGORY_PATTERN = /^[a-z]+(?:-[a-z]+)*(?:\.[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*)?$/;
export async function arxivFetch(params) {
const resp = await fetch(`${ARXIV_BASE}?${params}`);
if (!resp.ok) {
throw new CliError('FETCH_ERROR', `arXiv API HTTP ${resp.status}`, 'Check your search term or paper ID');
throw new CommandExecutionError(`arXiv API HTTP ${resp.status}`, 'Check your search term or paper ID');
}
return resp.text();
}
export function normalizeArxivLimit(value, defaultValue, maxValue, label = 'limit') {
const raw = value ?? defaultValue;
const limit = Number(raw);
if (!Number.isInteger(limit) || limit <= 0) {
throw new ArgumentError(`arxiv ${label} must be a positive integer`);
}
if (limit > maxValue) {
throw new ArgumentError(`arxiv ${label} must be <= ${maxValue}`);
}
return limit;
}
export function normalizeArxivCategory(value) {
const category = String(value || '').trim();
if (!ARXIV_CATEGORY_PATTERN.test(category)) {
throw new ArgumentError(`Invalid arXiv category "${value}". Examples: cs.CL, cs.LG, math.PR, q-bio.NC, physics.comp-ph`);
}
return category;
}
/** Decode the small set of XML entities arXiv emits in text fields. */
function decodeEntities(s) {
return s
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&quot;/g, '"')
.replace(/&apos;/g, "'")
.replace(/&#39;/g, "'");
}
/** Extract the text content of the first matching XML tag. */
function extract(xml, tag) {
const m = xml.match(new RegExp(`<${tag}[^>]*>([\\s\\S]*?)<\\/${tag}>`));
@@ -27,6 +56,34 @@ function extractAll(xml, tag) {
results.push(m[1].trim());
return results;
}
/** Extract the value of a named attribute from the first matching tag (open or self-closing). */
function extractAttr(xml, tag, attr) {
const m = xml.match(new RegExp(`<${tag}\\b[^>]*?\\b${attr}="([^"]*)"`));
return m ? m[1] : '';
}
/** Extract all values of a named attribute across repeated tags. */
function extractAllAttr(xml, tag, attr) {
const re = new RegExp(`<${tag}\\b[^>]*?\\b${attr}="([^"]*)"`, 'g');
const out = [];
let m;
while ((m = re.exec(xml)) !== null)
out.push(m[1]);
return out;
}
/** Find the href of the first <link> tag matching a given rel. */
function findLinkHref(xml, rel) {
const re = /<link\b([^>]*)\/?>/g;
let m;
while ((m = re.exec(xml)) !== null) {
const attrs = m[1];
if (new RegExp(`\\brel="${rel}"`).test(attrs)) {
const h = attrs.match(/\bhref="([^"]*)"/);
if (h)
return h[1];
}
}
return '';
}
/** Parse Atom XML feed into structured entries. */
export function parseEntries(xml) {
const entryRe = /<entry>([\s\S]*?)<\/entry>/g;
@@ -36,12 +93,18 @@ export function parseEntries(xml) {
const e = m[1];
const rawId = extract(e, 'id');
const arxivId = rawId.replace(/^https?:\/\/arxiv\.org\/abs\//, '').replace(/v\d+$/, '');
const pdf = findLinkHref(e, 'related') || `https://arxiv.org/pdf/${arxivId}`;
entries.push({
id: arxivId,
title: extract(e, 'title').replace(/\s+/g, ' '),
authors: extractAll(e, 'name').slice(0, 3).join(', '),
abstract: (() => { const s = extract(e, 'summary').replace(/\s+/g, ' '); return s.length > 200 ? s.slice(0, 200) + '...' : s; })(),
title: decodeEntities(extract(e, 'title').replace(/\s+/g, ' ')),
authors: decodeEntities(extractAll(e, 'name').join(', ')),
abstract: decodeEntities(extract(e, 'summary').replace(/\s+/g, ' ')),
published: extract(e, 'published').slice(0, 10),
updated: extract(e, 'updated').slice(0, 10),
primary_category: extractAttr(e, 'arxiv:primary_category', 'term'),
categories: extractAllAttr(e, 'category', 'term').join(', '),
comment: decodeEntities(extract(e, 'arxiv:comment').replace(/\s+/g, ' ')),
pdf,
url: `https://arxiv.org/abs/${arxivId}`,
});
}
+3 -3
View File
@@ -1,4 +1,4 @@
import { AuthRequiredError, EmptyResultError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, EmptyResultError, selectorError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
/**
* band mentions — Show Band notifications where you were @mentioned.
@@ -52,7 +52,7 @@ cli({
await page.wait(0.5);
}
if (!bellReady) {
throw new SelectorError('button._btnWidgetIcon', 'Notification bell not found. The Band.us UI may have changed.');
throw selectorError('button._btnWidgetIcon', 'Notification bell not found. The Band.us UI may have changed.');
}
// Poll until a capture containing result_data.news arrives, up to maxSecs seconds.
// getInterceptedRequests() clears the array on each call, so captures are accumulated
@@ -80,7 +80,7 @@ cli({
return true;
}`);
if (!bellClicked) {
throw new SelectorError('button._btnWidgetIcon', 'Notification bell disappeared before click. The Band.us UI may have changed.');
throw selectorError('button._btnWidgetIcon', 'Notification bell disappeared before click. The Band.us UI may have changed.');
}
const requests = await waitForOneCapture();
// Find the get_news response (has result_data.news); get_news_count responses do not.
+5 -1
View File
@@ -7,7 +7,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of videos' },
],
columns: ['rank', 'title', 'author', 'play', 'danmaku'],
columns: ['rank', 'title', 'author', 'play', 'danmaku', 'bvid', 'url'],
pipeline: [
{ navigate: 'https://www.bilibili.com' },
{ evaluate: `(async () => {
@@ -20,6 +20,8 @@ cli({
author: item.owner?.name,
play: item.stat?.view,
danmaku: item.stat?.danmaku,
bvid: item.bvid,
url: item.bvid ? 'https://www.bilibili.com/video/' + item.bvid : '',
}));
})()
` },
@@ -29,6 +31,8 @@ cli({
author: '${{ item.author }}',
play: '${{ item.play }}',
danmaku: '${{ item.danmaku }}',
bvid: '${{ item.bvid }}',
url: '${{ item.url }}',
} },
{ limit: '${{ args.limit }}' },
],
+17
View File
@@ -0,0 +1,17 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './hot.js';
describe('bilibili hot adapter', () => {
const command = getRegistry().get('bilibili/hot');
it('registers bvid and url columns in the public hot-list shape', () => {
expect(command?.columns).toEqual(['rank', 'title', 'author', 'play', 'danmaku', 'bvid', 'url']);
expect(command?.pipeline?.[1]?.evaluate).toContain('bvid: item.bvid');
expect(command?.pipeline?.[1]?.evaluate).toContain("'https://www.bilibili.com/video/' + item.bvid");
expect(command?.pipeline?.[2]?.map).toMatchObject({
bvid: '${{ item.bvid }}',
url: '${{ item.url }}',
});
});
});
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError, EmptyResultError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, CommandExecutionError, EmptyResultError, selectorError } from '@jackwener/opencli/errors';
import { apiGet, resolveBvid } from './utils.js';
cli({
site: 'bilibili',
@@ -23,7 +23,7 @@ cli({
return state?.videoData?.cid;
})()`);
if (!cid) {
throw new SelectorError('videoData.cid', '无法在页面中提取到当前视频的 CID,请检查页面是否正常加载。');
throw selectorError('videoData.cid', '无法在页面中提取到当前视频的 CID,请检查页面是否正常加载。');
}
// 3. 在 Node 端使用 apiGet 获取带 Wbi 签名的字幕列表
// 之前纯靠 evaluate 里的 fetch 会失败,因为 B 站 /wbi/ 开头的接口强校验 w_rid,未签名直接被风控返回 403 HTML
+3 -3
View File
@@ -6,7 +6,7 @@
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { requirePage, navigateToChat, findFriendByUid, clickCandidateInList, typeAndSendMessage, } from './utils.js';
import { EmptyResultError, SelectorError } from '@jackwener/opencli/errors';
import { EmptyResultError, selectorError } from '@jackwener/opencli/errors';
cli({
site: 'boss',
name: 'send',
@@ -30,12 +30,12 @@ cli({
const friendName = friend.name || '候选人';
const clicked = await clickCandidateInList(page, numericUid);
if (!clicked) {
throw new SelectorError('聊天列表中的用户', '请确认聊天列表中有此人');
throw selectorError('聊天列表中的用户', '请确认聊天列表中有此人');
}
await page.wait({ time: 2 });
const sent = await typeAndSendMessage(page, kwargs.text);
if (!sent) {
throw new SelectorError('消息输入框', '聊天页面 UI 可能已改变');
throw selectorError('消息输入框', '聊天页面 UI 可能已改变');
}
await page.wait({ time: 1 });
return [{ status: '✅ 发送成功', detail: `已向 ${friendName} 发送: ${kwargs.text}` }];
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const askCommand = cli({
site: 'chatwise',
name: 'ask',
@@ -43,7 +43,7 @@ export const askCommand = cli({
})(${JSON.stringify(text)})
`);
if (!injected)
throw new SelectorError('ChatWise input element');
throw selectorError('ChatWise input element');
await page.wait(0.5);
await page.pressKey('Enter');
// Poll for response
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const modelCommand = cli({
site: 'chatwise',
name: 'model',
@@ -58,7 +58,7 @@ export const modelCommand = cli({
})(${JSON.stringify(desiredModel)})
`);
if (!opened)
throw new SelectorError('ChatWise model selector');
throw selectorError('ChatWise model selector');
await page.wait(0.5);
// Find and click the target model in the dropdown
const found = await page.evaluate(`
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const sendCommand = cli({
site: 'chatwise',
name: 'send',
@@ -36,7 +36,7 @@ export const sendCommand = cli({
})(${JSON.stringify(text)})
`);
if (!injected)
throw new SelectorError('ChatWise input element');
throw selectorError('ChatWise input element');
await page.wait(0.5);
await page.pressKey('Enter');
return [
+128
View File
@@ -0,0 +1,128 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
CLAUDE_DOMAIN, CLAUDE_URL, ensureOnClaude, selectModel, setAdaptiveThinking,
sendMessage, sendWithFile, getBubbleCount, waitForResponse, parseBoolFlag, withRetry,
ensureClaudeComposer, requireNonEmptyPrompt, requirePositiveInt,
} from './utils.js';
export const askCommand = cli({
site: 'claude',
name: 'ask',
description: 'Send a prompt to Claude and get the response',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
timeoutSeconds: 180,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
{ name: 'timeout', type: 'int', default: 120, help: 'Max seconds to wait for response' },
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
{ name: 'model', default: 'sonnet', choices: ['sonnet', 'opus', 'haiku'], help: 'Model to use: sonnet, opus, or haiku' },
{ name: 'think', type: 'boolean', default: false, help: 'Enable Adaptive thinking' },
{ name: 'file', help: 'Attach a file (image, PDF, text) with the prompt' },
],
columns: ['response'],
func: async (page, kwargs) => {
const prompt = requireNonEmptyPrompt(kwargs.prompt, 'claude ask');
const timeoutSeconds = requirePositiveInt(
Number(kwargs.timeout ?? 120),
'claude ask --timeout',
'Example: opencli claude ask "hello" --timeout 120',
);
const timeoutMs = timeoutSeconds * 1000;
const wantThink = parseBoolFlag(kwargs.think);
if (parseBoolFlag(kwargs.new)) {
await page.goto(CLAUDE_URL);
await page.wait(3);
} else {
const navigated = await ensureOnClaude(page);
if (navigated) {
// Workspace was recycled; try to resume the most recent
// conversation instead of starting a new one.
await page.evaluate(`(() => {
var link = document.querySelector('a[href*="/chat/"]');
if (link) link.click();
})()`);
await page.wait(2);
}
}
await page.wait(2);
await withRetry(() => ensureClaudeComposer(page, 'Claude ask requires a visible composer on the current page.'));
// Model selector is only available on the new-chat page, not inside
// an existing conversation. Skip it when we resumed a prior thread.
const currentUrl = await page.evaluate('window.location.href') || '';
const inConversation = currentUrl.includes('/chat/');
const modelExplicit = kwargs.__opencliOptionSources?.model === 'cli';
const wantModel = kwargs.model || 'sonnet';
if (inConversation && modelExplicit) {
throw new ArgumentError(
`Cannot switch to ${wantModel} model inside an existing conversation.`,
'Re-run with --new to start a fresh chat before selecting a model.',
);
}
if (!inConversation) {
const modelResult = await withRetry(() => selectModel(page, wantModel));
if (!modelResult?.ok) {
if (modelResult?.upgrade) {
throw new ArgumentError(
`${wantModel} model requires a paid Claude plan.`,
'Pick --model sonnet or --model haiku, or upgrade your account.',
);
}
throw new CommandExecutionError(`Could not switch to ${wantModel} model`);
}
if (modelResult?.toggled) await page.wait(0.5);
}
const thinkResult = await withRetry(() => setAdaptiveThinking(page, wantThink));
if (!thinkResult?.ok && wantThink) {
throw new CommandExecutionError('Could not enable Adaptive thinking');
}
if (thinkResult?.toggled) await page.wait(0.5);
if (kwargs.file) {
const baseline = await withRetry(() => getBubbleCount(page));
try {
const fileResult = await sendWithFile(page, kwargs.file, prompt);
if (fileResult && !fileResult.ok) {
throw new CommandExecutionError(fileResult.reason || 'Failed to attach file');
}
} catch (err) {
// SPA navigates after send; "Promise was collected" means send succeeded
if (!String(err?.message || err).includes('Promise was collected')) throw err;
}
await page.wait(3);
const result = await waitForResponse(page, baseline, prompt, timeoutMs);
if (!result) {
throw new EmptyResultError(
'claude ask',
`No Claude response appeared within ${timeoutSeconds}s. Re-run with a higher --timeout if the model is still generating.`,
);
}
return [{ response: result }];
}
const baseline = await withRetry(() => getBubbleCount(page));
const sendResult = await withRetry(() => sendMessage(page, prompt));
if (!sendResult?.ok) {
throw new CommandExecutionError(sendResult?.reason || 'Failed to send message');
}
const result = await waitForResponse(page, baseline, prompt, timeoutMs);
if (!result) {
throw new EmptyResultError(
'claude ask',
`No Claude response appeared within ${timeoutSeconds}s. Re-run with a higher --timeout if the model is still generating.`,
);
}
return [{ response: result }];
},
});
+338
View File
@@ -0,0 +1,338 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const {
mockEnsureOnClaude,
mockEnsureClaudeComposer,
mockSelectModel,
mockSetAdaptiveThinking,
mockSendMessage,
mockSendWithFile,
mockGetBubbleCount,
mockWaitForResponse,
mockParseBoolFlag,
mockRequireNonEmptyPrompt,
mockRequirePositiveInt,
mockWithRetry,
} = vi.hoisted(() => ({
mockEnsureOnClaude: vi.fn(),
mockEnsureClaudeComposer: vi.fn(),
mockSelectModel: vi.fn(),
mockSetAdaptiveThinking: vi.fn(),
mockSendMessage: vi.fn(),
mockSendWithFile: vi.fn(),
mockGetBubbleCount: vi.fn(),
mockWaitForResponse: vi.fn(),
mockParseBoolFlag: vi.fn((v) => v === true || v === 'true'),
mockRequireNonEmptyPrompt: vi.fn((v) => String(v ?? '')),
mockRequirePositiveInt: vi.fn((v) => Number(v)),
mockWithRetry: vi.fn(async (fn) => fn()),
}));
vi.mock('./utils.js', () => ({
CLAUDE_DOMAIN: 'claude.ai',
CLAUDE_URL: 'https://claude.ai/new',
ensureOnClaude: mockEnsureOnClaude,
ensureClaudeComposer: mockEnsureClaudeComposer,
selectModel: mockSelectModel,
setAdaptiveThinking: mockSetAdaptiveThinking,
sendMessage: mockSendMessage,
sendWithFile: mockSendWithFile,
getBubbleCount: mockGetBubbleCount,
waitForResponse: mockWaitForResponse,
parseBoolFlag: mockParseBoolFlag,
requireNonEmptyPrompt: mockRequireNonEmptyPrompt,
requirePositiveInt: mockRequirePositiveInt,
withRetry: mockWithRetry,
}));
import { askCommand } from './ask.js';
describe('claude ask basic flow', () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue('https://claude.ai/new'),
};
beforeEach(() => {
vi.clearAllMocks();
page.evaluate.mockResolvedValue('https://claude.ai/new');
mockEnsureOnClaude.mockResolvedValue(false);
mockEnsureClaudeComposer.mockResolvedValue({ isLoggedIn: true, hasComposer: true });
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
mockSetAdaptiveThinking.mockResolvedValue({ ok: true, toggled: false });
mockSendMessage.mockResolvedValue({ ok: true });
mockSendWithFile.mockResolvedValue({ ok: true });
mockGetBubbleCount.mockResolvedValue(0);
mockWaitForResponse.mockResolvedValue('hello there');
mockRequireNonEmptyPrompt.mockImplementation((v) => String(v ?? ''));
mockRequirePositiveInt.mockImplementation((v) => Number(v));
});
it('returns the assistant response on a fresh chat', async () => {
const rows = await askCommand.func(page, {
prompt: 'hi',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
});
expect(rows).toEqual([{ response: 'hello there' }]);
expect(mockSendMessage).toHaveBeenCalledWith(page, 'hi');
expect(mockWaitForResponse).toHaveBeenCalledWith(page, 0, 'hi', 120000);
});
it('navigates to /new when --new is set', async () => {
await askCommand.func(page, {
prompt: 'hi',
timeout: 120,
new: true,
model: 'sonnet',
think: false,
});
expect(page.goto).toHaveBeenCalledWith('https://claude.ai/new');
expect(mockEnsureOnClaude).not.toHaveBeenCalled();
});
it('throws EmptyResultError when waitForResponse yields nothing', async () => {
mockWaitForResponse.mockResolvedValue(null);
await expect(askCommand.func(page, {
prompt: 'hi',
timeout: 60,
new: false,
model: 'sonnet',
think: false,
})).rejects.toThrow(EmptyResultError);
});
it('throws CommandExecutionError when send fails', async () => {
mockSendMessage.mockResolvedValue({ ok: false, reason: 'composer not found' });
await expect(askCommand.func(page, {
prompt: 'hi',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
})).rejects.toThrow(/composer not found/);
});
});
describe('claude ask --model handling', () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn(),
};
beforeEach(() => {
vi.clearAllMocks();
mockEnsureOnClaude.mockResolvedValue(false);
mockSetAdaptiveThinking.mockResolvedValue({ ok: true, toggled: false });
mockSendMessage.mockResolvedValue({ ok: true });
mockGetBubbleCount.mockResolvedValue(0);
mockWaitForResponse.mockResolvedValue('reply');
});
it('rejects --model opus on free tier with usage-error guidance', async () => {
page.evaluate.mockResolvedValue('https://claude.ai/new');
mockSelectModel.mockResolvedValue({ ok: false, upgrade: true });
await expect(askCommand.func(page, {
prompt: 'hi',
timeout: 120,
new: false,
model: 'opus',
think: false,
})).rejects.toMatchObject(new ArgumentError(
'opus model requires a paid Claude plan.',
'Pick --model sonnet or --model haiku, or upgrade your account.',
));
});
it('skips model selection inside an existing conversation', async () => {
page.evaluate.mockResolvedValue('https://claude.ai/chat/abc-123');
const rows = await askCommand.func(page, {
prompt: 'continue',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
});
expect(rows).toEqual([{ response: 'reply' }]);
expect(mockSelectModel).not.toHaveBeenCalled();
});
it('fails fast when --model is explicit inside an existing conversation', async () => {
page.evaluate.mockResolvedValue('https://claude.ai/chat/abc-123');
await expect(askCommand.func(page, {
prompt: 'continue',
timeout: 120,
new: false,
model: 'opus',
think: false,
__opencliOptionSources: { model: 'cli' },
})).rejects.toMatchObject(new ArgumentError(
'Cannot switch to opus model inside an existing conversation.',
'Re-run with --new to start a fresh chat before selecting a model.',
));
expect(mockSelectModel).not.toHaveBeenCalled();
});
});
describe('claude ask --think', () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue('https://claude.ai/new'),
};
beforeEach(() => {
vi.clearAllMocks();
mockEnsureOnClaude.mockResolvedValue(false);
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
mockSendMessage.mockResolvedValue({ ok: true });
mockGetBubbleCount.mockResolvedValue(0);
mockWaitForResponse.mockResolvedValue('reply');
});
it('toggles Adaptive thinking when --think is set', async () => {
mockSetAdaptiveThinking.mockResolvedValue({ ok: true, toggled: true });
await askCommand.func(page, {
prompt: 'reason carefully',
timeout: 120,
new: false,
model: 'sonnet',
think: true,
});
expect(mockSetAdaptiveThinking).toHaveBeenCalledWith(page, true);
});
it('throws when --think requested but toggle fails', async () => {
mockSetAdaptiveThinking.mockResolvedValue({ ok: false });
await expect(askCommand.func(page, {
prompt: 'reason carefully',
timeout: 120,
new: false,
model: 'sonnet',
think: true,
})).rejects.toThrow(/Adaptive thinking/);
});
it('does not throw when --think is false and toggle returns ok=false', async () => {
mockSetAdaptiveThinking.mockResolvedValue({ ok: false });
await expect(askCommand.func(page, {
prompt: 'hi',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
})).resolves.toEqual([{ response: 'reply' }]);
});
it('fails fast when prompt validation rejects an empty prompt', async () => {
mockRequireNonEmptyPrompt.mockImplementation(() => {
throw new ArgumentError('claude ask prompt cannot be empty');
});
await expect(askCommand.func(page, {
prompt: '',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
})).rejects.toThrow(ArgumentError);
});
it('fails fast when timeout validation rejects a non-positive value', async () => {
mockRequirePositiveInt.mockImplementation(() => {
throw new ArgumentError('claude ask --timeout must be a positive integer');
});
await expect(askCommand.func(page, {
prompt: 'hi',
timeout: 0,
new: false,
model: 'sonnet',
think: false,
})).rejects.toThrow(ArgumentError);
});
});
describe('claude ask --file', () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue('https://claude.ai/new'),
};
beforeEach(() => {
vi.clearAllMocks();
mockEnsureOnClaude.mockResolvedValue(false);
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
mockSetAdaptiveThinking.mockResolvedValue({ ok: true, toggled: false });
mockSendWithFile.mockResolvedValue({ ok: true });
mockGetBubbleCount.mockResolvedValue(3);
mockWaitForResponse.mockResolvedValue('the image shows a cat');
mockEnsureClaudeComposer.mockResolvedValue({ isLoggedIn: true, hasComposer: true });
mockRequireNonEmptyPrompt.mockImplementation((v) => String(v ?? ''));
mockRequirePositiveInt.mockImplementation((v) => Number(v));
});
it('routes through sendWithFile and captures baseline before sending', async () => {
const rows = await askCommand.func(page, {
prompt: 'describe this',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
file: '/tmp/cat.png',
});
expect(rows).toEqual([{ response: 'the image shows a cat' }]);
expect(mockGetBubbleCount).toHaveBeenCalledTimes(1);
expect(mockSendWithFile).toHaveBeenCalledWith(page, '/tmp/cat.png', 'describe this');
expect(mockSendMessage).not.toHaveBeenCalled();
expect(mockWaitForResponse).toHaveBeenCalledWith(page, 3, 'describe this', 120000);
});
it('surfaces file upload failure as CommandExecutionError', async () => {
mockSendWithFile.mockResolvedValue({ ok: false, reason: 'file preview did not appear' });
await expect(askCommand.func(page, {
prompt: 'describe this',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
file: '/tmp/cat.png',
})).rejects.toThrow(/file preview did not appear/);
});
it('absorbs "Promise was collected" SPA navigation error after send', async () => {
mockSendWithFile.mockRejectedValue(new Error('Promise was collected'));
const rows = await askCommand.func(page, {
prompt: 'describe this',
timeout: 120,
new: false,
model: 'sonnet',
think: false,
file: '/tmp/cat.png',
});
expect(rows).toEqual([{ response: 'the image shows a cat' }]);
});
});
+118
View File
@@ -0,0 +1,118 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, AuthRequiredError, EmptyResultError } from '@jackwener/opencli/errors';
const {
mockEnsureOnClaude,
mockEnsureClaudeComposer,
mockEnsureClaudeLogin,
mockSendMessage,
mockParseBoolFlag,
mockRequireNonEmptyPrompt,
mockGetVisibleMessages,
mockGetConversationList,
mockRequirePositiveInt,
mockRequireConversationId,
mockWithRetry,
} = vi.hoisted(() => ({
mockEnsureOnClaude: vi.fn(),
mockEnsureClaudeComposer: vi.fn(),
mockEnsureClaudeLogin: vi.fn(),
mockSendMessage: vi.fn(),
mockParseBoolFlag: vi.fn((v) => v === true || v === 'true'),
mockRequireNonEmptyPrompt: vi.fn((v) => String(v ?? '')),
mockGetVisibleMessages: vi.fn(),
mockGetConversationList: vi.fn(),
mockRequirePositiveInt: vi.fn((v) => Number(v)),
mockRequireConversationId: vi.fn((v) => String(v ?? '').trim()),
mockWithRetry: vi.fn(async (fn) => fn()),
}));
vi.mock('./utils.js', () => ({
CLAUDE_DOMAIN: 'claude.ai',
CLAUDE_URL: 'https://claude.ai/new',
ensureOnClaude: mockEnsureOnClaude,
ensureClaudeComposer: mockEnsureClaudeComposer,
ensureClaudeLogin: mockEnsureClaudeLogin,
sendMessage: mockSendMessage,
parseBoolFlag: mockParseBoolFlag,
requireNonEmptyPrompt: mockRequireNonEmptyPrompt,
getVisibleMessages: mockGetVisibleMessages,
getConversationList: mockGetConversationList,
requirePositiveInt: mockRequirePositiveInt,
requireConversationId: mockRequireConversationId,
withRetry: mockWithRetry,
}));
import { sendCommand } from './send.js';
import { newCommand } from './new.js';
import { readCommand } from './read.js';
import { historyCommand } from './history.js';
import { detailCommand } from './detail.js';
describe('claude command-level fail-fast contracts', () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
};
beforeEach(() => {
vi.clearAllMocks();
mockEnsureOnClaude.mockResolvedValue(false);
mockEnsureClaudeComposer.mockResolvedValue({ isLoggedIn: true, hasComposer: true });
mockEnsureClaudeLogin.mockResolvedValue({ isLoggedIn: true });
mockSendMessage.mockResolvedValue({ ok: true });
mockRequireNonEmptyPrompt.mockImplementation((v) => String(v ?? ''));
mockGetVisibleMessages.mockResolvedValue([{ Index: 0, Role: 'assistant', Text: 'hi' }]);
mockGetConversationList.mockResolvedValue([{ Index: 1, Id: 'abc', Title: 'Hi', Url: 'https://claude.ai/chat/abc' }]);
mockRequirePositiveInt.mockImplementation((v) => Number(v));
mockRequireConversationId.mockImplementation((v) => String(v ?? '').trim());
});
it('send rejects empty prompt via ArgumentError', async () => {
mockRequireNonEmptyPrompt.mockImplementation(() => {
throw new ArgumentError('claude send prompt cannot be empty');
});
await expect(sendCommand.func(page, { prompt: '', new: false })).rejects.toThrow(ArgumentError);
});
it('send surfaces auth failure from composer readiness', async () => {
mockEnsureClaudeComposer.mockRejectedValue(new AuthRequiredError('claude.ai', 'Claude send requires a logged-in Claude session.'));
await expect(sendCommand.func(page, { prompt: 'hi', new: false })).rejects.toThrow(AuthRequiredError);
});
it('new no longer false-succeeds on login wall', async () => {
mockEnsureClaudeComposer.mockRejectedValue(new AuthRequiredError('claude.ai', 'Claude new requires a logged-in Claude session with a visible composer.'));
await expect(newCommand.func(page)).rejects.toThrow(AuthRequiredError);
});
it('read throws EmptyResultError instead of a placeholder row', async () => {
mockGetVisibleMessages.mockResolvedValue([]);
await expect(readCommand.func(page)).rejects.toThrow(EmptyResultError);
});
it('history rejects invalid --limit values instead of silently coercing them', async () => {
mockRequirePositiveInt.mockImplementation(() => {
throw new ArgumentError('claude history --limit must be a positive integer');
});
await expect(historyCommand.func(page, { limit: 0 })).rejects.toThrow(ArgumentError);
});
it('history throws EmptyResultError on an empty /recents page', async () => {
mockGetConversationList.mockResolvedValue([]);
await expect(historyCommand.func(page, { limit: 20 })).rejects.toThrow(EmptyResultError);
});
it('detail rejects a missing conversation id', async () => {
mockRequireConversationId.mockImplementation(() => {
throw new ArgumentError('claude detail requires a conversation id');
});
await expect(detailCommand.func(page, { id: '' })).rejects.toThrow(ArgumentError);
});
});
+29
View File
@@ -0,0 +1,29 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, getVisibleMessages, ensureClaudeLogin, requireConversationId } from './utils.js';
export const detailCommand = cli({
site: 'claude',
name: 'detail',
description: 'Open a Claude conversation by ID and read its messages',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'id', positional: true, required: true, help: 'Conversation ID (UUID from /chat/<id>)' },
],
columns: ['Index', 'Role', 'Text'],
func: async (page, kwargs) => {
const id = requireConversationId(kwargs.id);
await page.goto(`https://claude.ai/chat/${id}`);
await page.wait(4);
await ensureClaudeLogin(page, 'Claude detail requires a logged-in Claude session.');
const messages = await getVisibleMessages(page);
if (messages.length > 0) return messages;
throw new EmptyResultError('claude detail', `No visible Claude messages were found for conversation ${id}.`);
},
});
+31
View File
@@ -0,0 +1,31 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, getConversationList, ensureClaudeLogin, requirePositiveInt } from './utils.js';
export const historyCommand = cli({
site: 'claude',
name: 'history',
description: 'List conversation history from Claude /recents',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Max conversations to show' },
],
columns: ['Index', 'Id', 'Title', 'Url'],
func: async (page, kwargs) => {
const limit = requirePositiveInt(
Number(kwargs.limit ?? 20),
'claude history --limit',
'Example: opencli claude history --limit 20',
);
const conversations = await getConversationList(page);
await ensureClaudeLogin(page, 'Claude history requires a logged-in Claude session.');
if (conversations.length === 0) {
throw new EmptyResultError('claude history', 'No Claude conversation history was visible on /recents.');
}
return conversations.slice(0, limit);
},
});
+21
View File
@@ -0,0 +1,21 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CLAUDE_DOMAIN, CLAUDE_URL, ensureClaudeComposer } from './utils.js';
export const newCommand = cli({
site: 'claude',
name: 'new',
description: 'Start a new conversation in Claude',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [],
columns: ['Status'],
func: async (page) => {
await page.goto(CLAUDE_URL);
await page.wait(2);
await ensureClaudeComposer(page, 'Claude new requires a logged-in Claude session with a visible composer.');
return [{ Status: 'New chat started' }];
},
});
+24
View File
@@ -0,0 +1,24 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, ensureOnClaude, getVisibleMessages, ensureClaudeLogin } from './utils.js';
export const readCommand = cli({
site: 'claude',
name: 'read',
description: 'Read the current Claude conversation',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [],
columns: ['Index', 'Role', 'Text'],
func: async (page) => {
await ensureOnClaude(page);
await page.wait(3);
await ensureClaudeLogin(page, 'Claude read requires a logged-in Claude session.');
const messages = await getVisibleMessages(page);
if (messages.length > 0) return messages;
throw new EmptyResultError('claude read', 'No visible Claude messages were found in the current conversation.');
},
});
+41
View File
@@ -0,0 +1,41 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, CLAUDE_URL, ensureOnClaude, sendMessage, parseBoolFlag, withRetry, ensureClaudeComposer, requireNonEmptyPrompt } from './utils.js';
export const sendCommand = cli({
site: 'claude',
name: 'send',
description: 'Send a prompt to Claude without waiting for the response',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
],
columns: ['Status', 'SubmittedBy', 'InjectedText'],
func: async (page, kwargs) => {
const prompt = requireNonEmptyPrompt(kwargs.prompt, 'claude send');
if (parseBoolFlag(kwargs.new)) {
await page.goto(CLAUDE_URL);
await page.wait(3);
} else {
await ensureOnClaude(page);
await page.wait(2);
}
await withRetry(() => ensureClaudeComposer(page, 'Claude send requires a visible composer on the current page.'));
const sendResult = await withRetry(() => sendMessage(page, prompt));
if (!sendResult?.ok) {
throw new CommandExecutionError(sendResult?.reason || 'Failed to send message');
}
return [{
Status: 'Success',
SubmittedBy: sendResult.method || 'send-button',
InjectedText: prompt,
}];
},
});
+24
View File
@@ -0,0 +1,24 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CLAUDE_DOMAIN, ensureOnClaude, getPageState } from './utils.js';
export const statusCommand = cli({
site: 'claude',
name: 'status',
description: 'Check Claude page availability and login state',
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [],
columns: ['Status', 'Login', 'Url'],
func: async (page) => {
await ensureOnClaude(page);
const state = await getPageState(page);
return [{
Status: state.hasComposer ? 'Connected' : 'Page not ready',
Login: state.isLoggedIn ? 'Yes' : 'No',
Url: state.url,
}];
},
});
+440
View File
@@ -0,0 +1,440 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
export const CLAUDE_DOMAIN = 'claude.ai';
export const CLAUDE_URL = 'https://claude.ai/new';
export const COMPOSER_SELECTOR = '[data-testid="chat-input"]';
export const MESSAGE_SELECTOR = '.font-claude-response';
export const MODEL_DROPDOWN_SELECTOR = '[data-testid="model-selector-dropdown"]';
const MODEL_DISPLAY_NAMES = {
sonnet: 'Sonnet 4.6',
opus: 'Opus 4.7',
haiku: 'Haiku 4.5',
};
export async function isOnClaude(page) {
const url = await page.evaluate('window.location.href').catch(() => '');
if (typeof url !== 'string' || !url) return false;
try {
const h = new URL(url).hostname;
return h === CLAUDE_DOMAIN || h.endsWith(`.${CLAUDE_DOMAIN}`);
} catch {
return false;
}
}
export async function ensureOnClaude(page) {
if (await isOnClaude(page)) return false;
await page.goto(CLAUDE_URL);
await page.wait(3);
return true;
}
export async function getPageState(page) {
return page.evaluate(`(() => {
var composer = document.querySelector('${COMPOSER_SELECTOR}');
var userMenu = document.querySelector('[data-testid="user-menu-button"]');
return {
url: window.location.href,
title: document.title,
hasComposer: !!composer,
isLoggedIn: !!userMenu,
};
})()`);
}
export async function ensureClaudeLogin(page, message = 'Claude requires a logged-in browser session.') {
const state = await getPageState(page);
if (!state.isLoggedIn) {
throw new AuthRequiredError(CLAUDE_DOMAIN, message);
}
return state;
}
export async function ensureClaudeComposer(page, message = 'Claude composer is not available on the current page.') {
const state = await ensureClaudeLogin(page, message);
if (!state.hasComposer) {
throw new CommandExecutionError(message);
}
return state;
}
export function requireNonEmptyPrompt(prompt, commandName) {
const text = String(prompt ?? '').trim();
if (!text) {
throw new ArgumentError(
`${commandName} prompt cannot be empty`,
`Example: opencli ${commandName} "hello"`,
);
}
return text;
}
export function requirePositiveInt(value, flagLabel, hint) {
if (!Number.isInteger(value) || value < 1) {
throw new ArgumentError(`${flagLabel} must be a positive integer`, hint);
}
return value;
}
export function requireConversationId(value) {
const id = String(value ?? '').trim();
if (!id) {
throw new ArgumentError(
'claude detail requires a conversation id',
'Example: opencli claude detail 123e4567-e89b-12d3-a456-426614174000',
);
}
return id;
}
export async function getVisibleMessages(page) {
const result = await page.evaluate(`(() => {
var nodes = document.querySelectorAll('[data-testid="user-message"], ${MESSAGE_SELECTOR}');
var rows = [];
Array.from(nodes).forEach(function(el) {
var isUser = el.getAttribute('data-testid') === 'user-message';
var raw = (el.innerText || '').trim();
if (!isUser) {
var parts = raw.split(/\\n\\n+/);
while (parts.length > 1 && /^(Thought|View)\\b/i.test(parts[0])) parts.shift();
raw = parts.join('\\n\\n').trim();
}
if (raw) rows.push({ role: isUser ? 'user' : 'assistant', text: raw });
});
return rows;
})()`);
if (!Array.isArray(result)) return [];
return result.map(function(r, i) { return { Index: i, Role: r.role, Text: r.text }; });
}
export async function getConversationList(page) {
if (!(await isOnClaude(page)) || !(await page.evaluate('window.location.href') || '').includes('/recents')) {
await page.goto('https://claude.ai/recents');
await page.wait(3);
}
const items = await page.evaluate(`(() => {
var links = Array.from(document.querySelectorAll('a[href*="/chat/"]'));
return links.map(function(link, i) {
var href = link.getAttribute('href') || '';
var idMatch = href.match(/\\/chat\\/([a-f0-9-]+)/);
return {
Index: i + 1,
Id: idMatch ? idMatch[1] : href,
Title: (link.innerText || '').trim().split('\\n')[0].trim() || '(untitled)',
Url: href.startsWith('http') ? href : ('https://claude.ai' + href),
};
});
})()`);
return Array.isArray(items) ? items : [];
}
export async function selectModel(page, modelName) {
const display = MODEL_DISPLAY_NAMES[String(modelName).toLowerCase()];
if (!display) return { ok: false };
const opened = await page.evaluate(`(() => {
var trigger = document.querySelector('${MODEL_DROPDOWN_SELECTOR}');
if (!trigger) return { ok: false };
var label = trigger.getAttribute('aria-label') || '';
if (label.indexOf(${JSON.stringify(display)}) >= 0) {
return { ok: true, toggled: false };
}
trigger.click();
return { ok: true, opened: true };
})()`);
if (!opened?.ok) return opened;
if (!opened.opened) return opened;
await page.wait(0.6);
return page.evaluate(`(() => {
var items = Array.from(document.querySelectorAll('div[role="menuitemradio"]'));
var target = items.find(function(el) { return (el.innerText || '').indexOf(${JSON.stringify(display)}) >= 0; });
if (!target) return { ok: false };
// Free-tier locked options carry an inline "Upgrade" button next to the label.
var upgrade = target.querySelector('button');
if (upgrade && (upgrade.innerText || '').toLowerCase().indexOf('upgrade') >= 0) {
document.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true }));
return { ok: false, upgrade: true };
}
var alreadySelected = target.getAttribute('aria-checked') === 'true';
if (!alreadySelected) target.click();
else document.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true }));
return { ok: true, toggled: !alreadySelected };
})()`);
}
export async function setAdaptiveThinking(page, enabled) {
const opened = await page.evaluate(`(() => {
var trigger = document.querySelector('${MODEL_DROPDOWN_SELECTOR}');
if (!trigger) return { ok: false };
trigger.click();
return { ok: true };
})()`);
if (!opened?.ok) return { ok: false };
await page.wait(0.6);
return page.evaluate(`(() => {
var items = Array.from(document.querySelectorAll('div[role="menuitem"]'));
var target = items.find(function(el) { return (el.innerText || '').indexOf('Adaptive thinking') >= 0; });
if (!target) {
document.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true }));
return { ok: false };
}
var isActive = target.getAttribute('aria-checked') === 'true';
if (${enabled} !== isActive) target.click();
else document.dispatchEvent(new KeyboardEvent('keydown', { key: 'Escape', bubbles: true }));
return { ok: true, toggled: ${enabled} !== isActive };
})()`);
}
export async function sendMessage(page, prompt) {
const promptJson = JSON.stringify(prompt);
const composerReady = await page.evaluate(`(() => {
var box = document.querySelector('${COMPOSER_SELECTOR}');
if (!box) return false;
box.focus();
// ProseMirror editors hold content in nested <p>; clear via Range/delete
// rather than .value or textContent, which the editor won't notice.
var sel = window.getSelection();
sel.removeAllRanges();
var range = document.createRange();
range.selectNodeContents(box);
sel.addRange(range);
document.execCommand('delete', false);
return true;
})()`);
if (!composerReady) return { ok: false, reason: 'composer not found' };
let typedNatively = false;
if (page.nativeType) {
try {
await page.nativeType(prompt);
typedNatively = true;
} catch (err) {
const msg = String(err?.message || err);
if (!msg.includes('Unknown action') && !msg.includes('not supported')) throw err;
}
}
if (!typedNatively) {
await page.evaluate(`(() => {
var box = document.querySelector('${COMPOSER_SELECTOR}');
if (!box) return;
box.focus();
document.execCommand('insertText', false, ${promptJson});
})()`);
}
await page.wait(1.2);
return page.evaluate(`(() => {
var ariaCandidates = [
'button[aria-label="Send Message"]',
'button[aria-label="Send message"]',
'button[aria-label="Send"]',
'button[aria-label*="Send"]',
];
for (var i = 0; i < ariaCandidates.length; i++) {
var btn = document.querySelector(ariaCandidates[i]);
if (btn && !btn.disabled) { btn.click(); return { ok: true }; }
}
// Fallback: rightmost enabled button with an svg in the composer container.
var box = document.querySelector('${COMPOSER_SELECTOR}');
if (box) {
var c = box.parentElement;
for (var hop = 0; hop < 6 && c; hop++) {
var btns = Array.from(c.querySelectorAll('button')).filter(function(b) { return !b.disabled && b.querySelector('svg'); });
if (btns.length) { btns[btns.length - 1].click(); return { ok: true, method: 'fallback' }; }
c = c.parentElement;
}
}
var box2 = document.querySelector('${COMPOSER_SELECTOR}');
if (box2) {
box2.dispatchEvent(new KeyboardEvent('keydown', { key: 'Enter', code: 'Enter', keyCode: 13, bubbles: true }));
return { ok: true, method: 'enter' };
}
return { ok: false, reason: 'send button not found' };
})()`);
}
export async function getBubbleCount(page) {
const count = await page.evaluate(`(() => {
return document.querySelectorAll('${MESSAGE_SELECTOR}').length;
})()`);
return count || 0;
}
export async function waitForResponse(page, baselineCount, prompt, timeoutMs) {
const startTime = Date.now();
let lastText = '';
let stableCount = 0;
while (Date.now() - startTime < timeoutMs) {
await page.wait(3);
let result;
try {
result = await page.evaluate(`(() => {
var bubbles = document.querySelectorAll('${MESSAGE_SELECTOR}');
// Adaptive thinking renders "Thought process" labels at the top
// of the response (often duplicated for the expand/collapse widget).
// Strip them so the row value is the actual answer text.
var texts = Array.from(bubbles).map(function(b) {
var raw = (b.innerText || '').trim();
// Drop leading paragraphs that are widget labels:
// "Thought process" / "Thought for Xs" — Adaptive thinking expand widget
// "View uploaded image" / "View attachment" — file thumbnail label
// These render twice (collapsed + expanded) and are followed by a blank line.
var parts = raw.split(/\\n\\n+/);
while (parts.length > 1 && /^(Thought|View)\\b/i.test(parts[0])) parts.shift();
return parts.join('\\n\\n').trim();
}).filter(Boolean);
return {
count: texts.length,
last: texts[texts.length - 1] || '',
streaming: !!document.querySelector('[data-is-streaming="true"]'),
};
})()`);
} catch {
continue;
}
if (!result) continue;
const candidate = result.last;
if (!candidate || candidate === prompt.trim()) continue;
if (result.count <= baselineCount) continue;
if (result.streaming) {
lastText = candidate;
stableCount = 0;
continue;
}
if (candidate === lastText) {
stableCount++;
if (stableCount >= 3) return candidate;
} else {
stableCount = 0;
lastText = candidate;
}
}
return lastText || null;
}
async function waitForFilePreview(page, fileName) {
for (let attempt = 0; attempt < 12; attempt++) {
await page.wait(1);
const ready = await page.evaluate(`(() => {
// Claude renders attachments as data-testid="file-thumbnail" cards with
// a sibling Remove button. Either signal indicates the file took.
if (document.querySelector('[data-testid="file-thumbnail"]')) return true;
var removeBtn = Array.from(document.querySelectorAll('button'))
.find(function(b) { return (b.getAttribute('aria-label') || '') === 'Remove'; });
return !!removeBtn;
})()`);
if (ready) return true;
}
return false;
}
export async function sendWithFile(page, filePath, prompt) {
const fs = await import('node:fs');
const path = await import('node:path');
const absPath = path.default.resolve(filePath);
if (!fs.default.existsSync(absPath)) {
return { ok: false, reason: `File not found: ${absPath}` };
}
const stats = fs.default.statSync(absPath);
if (stats.size > 30 * 1024 * 1024) {
return { ok: false, reason: `File too large (${(stats.size / 1024 / 1024).toFixed(1)} MB). Max: 30 MB` };
}
const fileName = path.default.basename(absPath);
let uploaded = false;
if (page.setFileInput) {
try {
// Upload via CDP so the file content does not cross the daemon body
// limit, then trigger React's controlled onChange manually because
// CDP assigns .files without firing the synthetic event React listens for.
await page.setFileInput([absPath], 'input[data-testid="file-upload"]');
const fired = await page.evaluate(`(() => {
var inp = document.querySelector('input[data-testid="file-upload"]');
if (!inp) return { ok: false, reason: 'file input not found' };
var propsKey = Object.keys(inp).find(function(k) { return k.startsWith('__reactProps$'); });
if (propsKey && typeof inp[propsKey].onChange === 'function') {
inp[propsKey].onChange({ target: { files: inp.files } });
return { ok: true, via: 'react' };
}
inp.dispatchEvent(new Event('change', { bubbles: true }));
return { ok: true, via: 'native' };
})()`);
if (!fired?.ok) return fired;
uploaded = true;
} catch (err) {
const msg = String(err?.message || err);
if (!msg.includes('Unknown action') && !msg.includes('not supported') && !msg.includes('Not allowed')) {
throw err;
}
}
}
if (!uploaded) {
const content = fs.default.readFileSync(absPath);
const base64 = content.toString('base64');
const fallbackResult = await page.evaluate(`(async () => {
var binary = atob('${base64}');
var bytes = new Uint8Array(binary.length);
for (var i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
var file = new File([bytes], ${JSON.stringify(fileName)});
var dt = new DataTransfer();
dt.items.add(file);
var inp = document.querySelector('input[data-testid="file-upload"]');
if (!inp) return { ok: false, reason: 'file input not found' };
var propsKey = Object.keys(inp).find(function(k) { return k.startsWith('__reactProps$'); });
if (!propsKey || typeof inp[propsKey].onChange !== 'function') {
return { ok: false, reason: 'React onChange not found' };
}
inp.files = dt.files;
inp[propsKey].onChange({ target: { files: inp.files } });
return { ok: true };
})()`);
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
}
const ready = await waitForFilePreview(page, fileName);
if (!ready) return { ok: false, reason: 'file preview did not appear' };
return sendMessage(page, prompt);
}
// Retries on CDP "Promise was collected" errors caused by Claude SPA route changes.
export async function withRetry(fn, retries = 2) {
for (let i = 0; i <= retries; i++) {
try {
return await fn();
} catch (err) {
const msg = String(err?.message || err);
if (i < retries && msg.includes('Promise was collected')) {
await new Promise(r => setTimeout(r, 2000));
continue;
}
throw err;
}
}
}
export function parseBoolFlag(value) {
if (typeof value === 'boolean') return value;
return String(value ?? '').trim().toLowerCase() === 'true';
}
+148
View File
@@ -0,0 +1,148 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError } from '@jackwener/opencli/errors';
import { parseBoolFlag, sendWithFile, selectModel, requireConversationId, requireNonEmptyPrompt, requirePositiveInt } from './utils.js';
describe('claude parseBoolFlag', () => {
it('returns booleans unchanged', () => {
expect(parseBoolFlag(true)).toBe(true);
expect(parseBoolFlag(false)).toBe(false);
});
it('treats only "true" string (case-insensitive) as true', () => {
expect(parseBoolFlag('true')).toBe(true);
expect(parseBoolFlag('TRUE')).toBe(true);
expect(parseBoolFlag('1')).toBe(false);
expect(parseBoolFlag('yes')).toBe(false);
expect(parseBoolFlag('')).toBe(false);
expect(parseBoolFlag(null)).toBe(false);
expect(parseBoolFlag(undefined)).toBe(false);
});
});
describe('claude argument helpers', () => {
it('rejects blank prompts', () => {
expect(() => requireNonEmptyPrompt(' ', 'claude ask')).toThrow(ArgumentError);
});
it('rejects non-positive integers for numeric flags', () => {
expect(() => requirePositiveInt(0, 'claude ask --timeout')).toThrow(ArgumentError);
expect(() => requirePositiveInt(-1, 'claude history --limit')).toThrow(ArgumentError);
});
it('rejects missing conversation ids', () => {
expect(() => requireConversationId(' ')).toThrow(ArgumentError);
});
});
describe('claude sendWithFile', () => {
const tempDirs = [];
afterEach(() => {
vi.restoreAllMocks();
while (tempDirs.length) {
fs.rmSync(tempDirs.pop(), { recursive: true, force: true });
}
});
it('prefers page.setFileInput, then sends after preview appears', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-claude-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'cat.png');
fs.writeFileSync(filePath, 'fake');
const page = {
nativeType: vi.fn().mockResolvedValue(undefined),
setFileInput: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce({ ok: true, via: 'react' }) // React onChange fired after setFileInput
.mockResolvedValueOnce(true) // waitForFilePreview hit
.mockResolvedValueOnce(true) // composer ready
.mockResolvedValueOnce({ ok: true }), // send button click
};
const result = await sendWithFile(page, filePath, 'describe this');
expect(result).toEqual({ ok: true });
expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[data-testid="file-upload"]');
expect(page.nativeType).toHaveBeenCalledWith('describe this');
});
it('returns file-not-found when path does not exist', async () => {
const page = { setFileInput: vi.fn(), evaluate: vi.fn(), wait: vi.fn() };
const result = await sendWithFile(page, '/no/such/file.png', 'hi');
expect(result.ok).toBe(false);
expect(result.reason).toContain('File not found');
});
it('rejects oversized files before any upload attempt', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-claude-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'big.bin');
fs.writeFileSync(filePath, Buffer.alloc(31 * 1024 * 1024));
const page = { setFileInput: vi.fn(), evaluate: vi.fn(), wait: vi.fn() };
const result = await sendWithFile(page, filePath, 'hi');
expect(result.ok).toBe(false);
expect(result.reason).toMatch(/too large/);
expect(page.setFileInput).not.toHaveBeenCalled();
});
});
describe('claude selectModel', () => {
afterEach(() => {
vi.restoreAllMocks();
});
it('rejects unknown model keys without touching the page', async () => {
const page = { evaluate: vi.fn() };
const result = await selectModel(page, 'gpt5');
expect(result).toEqual({ ok: false });
expect(page.evaluate).not.toHaveBeenCalled();
});
it('returns toggled=false when the dropdown already shows the requested model', async () => {
const page = {
evaluate: vi.fn().mockResolvedValueOnce({ ok: true, toggled: false }),
wait: vi.fn(),
};
const result = await selectModel(page, 'sonnet');
expect(result).toEqual({ ok: true, toggled: false });
expect(page.wait).not.toHaveBeenCalled();
});
it('opens the dropdown and clicks the matching radio', async () => {
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce({ ok: true, opened: true })
.mockResolvedValueOnce({ ok: true, toggled: true }),
wait: vi.fn().mockResolvedValue(undefined),
};
const result = await selectModel(page, 'haiku');
expect(result).toEqual({ ok: true, toggled: true });
expect(page.evaluate).toHaveBeenCalledTimes(2);
});
it('flags upgrade-required when picking a paid model on free tier', async () => {
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce({ ok: true, opened: true })
.mockResolvedValueOnce({ ok: false, upgrade: true }),
wait: vi.fn().mockResolvedValue(undefined),
};
const result = await selectModel(page, 'opus');
expect(result).toEqual({ ok: false, upgrade: true });
});
});
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const askCommand = cli({
site: 'codex',
name: 'ask',
@@ -34,7 +34,7 @@ export const askCommand = cli({
})(${JSON.stringify(text)})
`);
if (!injected)
throw new SelectorError('Codex input element');
throw selectorError('Codex input element');
await page.wait(0.5);
await page.pressKey('Enter');
// Poll for new content
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const sendCommand = cli({
site: 'codex',
name: 'send',
@@ -28,7 +28,7 @@ export const sendCommand = cli({
})(${JSON.stringify(textToInsert)})
`);
if (!injected)
throw new SelectorError('Codex Composer input element');
throw selectorError('Codex Composer input element');
// Wait for the UI to register the input
await page.wait(0.5);
// Simulate Enter key to submit
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const askCommand = cli({
site: 'cursor',
name: 'ask',
@@ -28,7 +28,7 @@ export const askCommand = cli({
return true;
})(${JSON.stringify(text)})`);
if (!injected)
throw new SelectorError('Cursor input element');
throw selectorError('Cursor input element');
await page.wait(0.5);
await page.pressKey('Enter');
// Poll until a new assistant message appears or timeout
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const composerCommand = cli({
site: 'cursor',
name: 'composer',
@@ -27,7 +27,7 @@ export const composerCommand = cli({
return true;
})(${JSON.stringify(textToInsert)})`);
if (!typed) {
throw new SelectorError('Cursor Composer input element', 'Could not find Cursor Composer input element after pressing Cmd+I.');
throw selectorError('Cursor Composer input element', 'Could not find Cursor Composer input element after pressing Cmd+I.');
}
await page.wait(0.5);
await page.pressKey('Enter');
+2 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { SelectorError } from '@jackwener/opencli/errors';
import { selectorError } from '@jackwener/opencli/errors';
export const sendCommand = cli({
site: 'cursor',
name: 'send',
@@ -24,7 +24,7 @@ export const sendCommand = cli({
return true;
})(${JSON.stringify(textToInsert)})`);
if (!injected) {
throw new SelectorError('Cursor Composer input element');
throw selectorError('Cursor Composer input element');
}
// Submit the command. In Cursor, Enter usually submits the chat.
await page.wait(0.5);
+236
View File
@@ -0,0 +1,236 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './top.js';
import './tag.js';
import './user.js';
import './read.js';
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
describe('devto listing adapters surface id + reading_time + published_at', () => {
it('devto/top has the agent-native column shape and pipeline mapping', () => {
const cmd = getRegistry().get('devto/top');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'author', 'reactions', 'comments',
'reading_time', 'published_at', 'tags', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.id }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
url: '${{ item.url }}',
});
});
it('devto/tag has the agent-native column shape and pipeline mapping', () => {
const cmd = getRegistry().get('devto/tag');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'author', 'reactions', 'comments',
'reading_time', 'published_at', 'tags', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.id }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
});
});
it('devto/user has the agent-native column shape (no author column, since user-specific)', () => {
const cmd = getRegistry().get('devto/user');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'reactions', 'comments',
'reading_time', 'published_at', 'tags', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.id }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
});
});
});
describe('devto/read adapter', () => {
const cmd = getRegistry().get('devto/read');
it('registers the article-detail row shape', () => {
expect(cmd?.columns).toEqual([
'id', 'title', 'author', 'reactions', 'reading_time',
'tags', 'published_at', 'body', 'url',
]);
});
it('takes a positional id plus a tunable max-length', () => {
const argNames = (cmd?.args || []).map((a) => a.name);
expect(argNames).toEqual(['id', 'max-length']);
const idArg = cmd?.args?.find((a) => a.name === 'id');
expect(idArg?.required).toBe(true);
expect(idArg?.positional).toBe(true);
});
it('uses the public dev.to JSON endpoint (no browser, public strategy)', () => {
expect(cmd?.browser).toBe(false);
expect(cmd?.strategy).toBe('public');
});
it('fails fast with ArgumentError for non-numeric id before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: 'not-a-number', 'max-length': 20000 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with ArgumentError for max-length below 100 before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: '12345', 'max-length': 50 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('accepts numeric max-length strings on the direct func path', async () => {
const article = {
id: 1,
title: 't',
user: { username: 'u' },
public_reactions_count: 0,
reading_time_minutes: 1,
tag_list: [],
published_at: '',
body_markdown: 'x'.repeat(150),
url: 'https://dev.to/u/t-1',
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(article), { status: 200 })));
const rows = await cmd.func({ id: '1', 'max-length': '100' });
expect(rows[0].body).toBe('x'.repeat(100) + '\n\n... [truncated]');
});
it('fails fast with ArgumentError for invalid max-length strings before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: '12345', 'max-length': 'abc' }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with EmptyResultError on 404', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('Not found', { status: 404 })));
await expect(cmd.func({ id: '99999999', 'max-length': 20000 }))
.rejects.toThrow(EmptyResultError);
});
it('fails fast with CommandExecutionError on non-404 HTTP failures', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('Server error', { status: 500 })));
await expect(cmd.func({ id: '12345', 'max-length': 20000 }))
.rejects.toThrow(CommandExecutionError);
});
it('fails fast with CommandExecutionError on invalid JSON responses', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('not json', { status: 200 })));
await expect(cmd.func({ id: '12345', 'max-length': 20000 }))
.rejects.toThrow(CommandExecutionError);
});
it('fails fast when the full article body is missing instead of returning a summary', async () => {
const article = {
id: 1,
title: 't',
user: { username: 'u' },
public_reactions_count: 0,
reading_time_minutes: 1,
tag_list: [],
published_at: '',
description: 'summary only',
url: 'https://dev.to/u/t-1',
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(article), { status: 200 })));
await expect(cmd.func({ id: '1', 'max-length': 20000 }))
.rejects.toThrow(CommandExecutionError);
});
it('returns a single article row with body_markdown extracted', async () => {
// Real /api/articles/<id> returns tag_list as a comma string and tags as an array.
const article = {
id: 3605688,
title: 'How to do thing X in Rust',
user: { username: 'jdoe' },
public_reactions_count: 42,
reading_time_minutes: 7,
tag_list: 'rust, webdev',
tags: ['rust', 'webdev'],
published_at: '2026-05-01T00:00:00Z',
body_markdown: '# Hello\n\nThis is the article body.',
url: 'https://dev.to/jdoe/how-to-do-thing-x-in-rust-1234',
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(article), { status: 200 })));
const rows = await cmd.func({ id: '3605688', 'max-length': 20000 });
expect(rows).toEqual([
{
id: 3605688,
title: 'How to do thing X in Rust',
author: 'jdoe',
reactions: 42,
reading_time: 7,
tags: 'rust, webdev',
published_at: '2026-05-01T00:00:00Z',
body: '# Hello\n\nThis is the article body.',
url: 'https://dev.to/jdoe/how-to-do-thing-x-in-rust-1234',
},
]);
});
it('handles the alternate shape where tag_list is an array (defensive)', async () => {
const article = {
id: 1,
title: 't',
user: { username: 'u' },
public_reactions_count: 0,
reading_time_minutes: 1,
tag_list: ['javascript', 'webdev'],
published_at: '',
body_markdown: 'body',
url: 'https://dev.to/u/t-1',
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(article), { status: 200 })));
const rows = await cmd.func({ id: '1', 'max-length': 20000 });
expect(rows[0].tags).toBe('javascript, webdev');
});
it('truncates body when over max-length and appends a marker', async () => {
const longBody = 'x'.repeat(500);
const article = {
id: 1,
title: 't',
user: { username: 'u' },
public_reactions_count: 0,
reading_time_minutes: 1,
tag_list: [],
published_at: '',
body_markdown: longBody,
url: 'https://dev.to/u/t-1',
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(article), { status: 200 })));
const rows = await cmd.func({ id: '1', 'max-length': 100 });
expect(rows[0].body).toBe('x'.repeat(100) + '\n\n... [truncated]');
});
});
+102
View File
@@ -0,0 +1,102 @@
/**
* DEV.to article reader.
*
* Public API: https://dev.to/api/articles/<id>
* Returns the full article including `body_markdown` (and `body_html`).
*
* The DEV.to API does not currently expose article comments — this reader
* therefore emits one row with the article body. If/when comments become
* available we can extend to a POST + L0/L1 shape like `hackernews read`.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DEVTO_ARTICLE_BASE = 'https://dev.to/api/articles';
async function fetchArticle(id) {
let res;
try {
res = await fetch(`${DEVTO_ARTICLE_BASE}/${id}`);
} catch (error) {
const detail = error instanceof Error ? error.message : String(error);
throw new CommandExecutionError(`DEV.to API request failed for article ${id}`, detail);
}
if (res.status === 404) {
throw new EmptyResultError(`devto/${id}`, 'Article not found');
}
if (!res.ok) {
throw new CommandExecutionError(`DEV.to API HTTP ${res.status} for article ${id}`, 'Check the article id');
}
try {
return await res.json();
} catch {
throw new CommandExecutionError(`DEV.to API returned invalid JSON for article ${id}`, 'Retry later or open the article URL directly');
}
}
function requireMinInt(value, min, label) {
const number = typeof value === 'number' ? value : Number(value);
if (!Number.isInteger(number) || number < min) {
throw new ArgumentError(`${label} must be an integer >= ${min}`);
}
return number;
}
function requireArticleBody(article, id) {
if (typeof article.body_markdown === 'string' && article.body_markdown.trim()) {
return article.body_markdown;
}
throw new CommandExecutionError(
`DEV.to article ${id} did not include body_markdown`,
'DEV.to API response shape may have changed. Open the article URL directly or retry later.',
);
}
cli({
site: 'devto',
name: 'read',
description: 'Read a DEV.to article body by id',
domain: 'dev.to',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'id', required: true, positional: true, help: 'DEV.to article id (numeric, e.g. 3605688)' },
{ name: 'max-length', type: 'int', default: 20000, help: 'Max characters of body to return (min 100)' },
],
columns: ['id', 'title', 'author', 'reactions', 'reading_time', 'tags', 'published_at', 'body', 'url'],
func: async (args) => {
const id = String(args.id || '').trim();
if (!/^\d+$/.test(id)) {
throw new ArgumentError(`Invalid DEV.to article id: ${args.id}`, 'Pass a numeric id like 3605688');
}
const maxLength = requireMinInt(args['max-length'] ?? 20000, 100, 'devto read --max-length');
const article = await fetchArticle(id);
if (!article || !article.id) {
throw new EmptyResultError(`devto/${id}`, 'Article not found');
}
const body = requireArticleBody(article, id);
const truncated = body.length > maxLength
? body.slice(0, maxLength) + '\n\n... [truncated]'
: body;
// The single-article endpoint returns `tag_list` as a comma-separated
// string and `tags` as an array — the opposite of the listing endpoints.
// Normalize either shape into a single comma-separated string.
const tagsRaw = article.tag_list ?? article.tags ?? '';
const tags = Array.isArray(tagsRaw) ? tagsRaw.join(', ') : String(tagsRaw);
return [{
id: article.id,
title: article.title || '',
author: article.user?.username || '[deleted]',
reactions: article.public_reactions_count ?? 0,
reading_time: article.reading_time_minutes ?? 0,
tags,
published_at: article.published_at || '',
body: truncated,
url: article.url || '',
}];
},
});
+4 -1
View File
@@ -15,15 +15,18 @@ cli({
},
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles' },
],
columns: ['rank', 'title', 'author', 'reactions', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'author', 'reactions', 'comments', 'reading_time', 'published_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://dev.to/api/articles?tag=${{ args.tag }}&per_page=${{ args.limit }}' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
author: '${{ item.user.username }}',
reactions: '${{ item.public_reactions_count }}',
comments: '${{ item.comments_count }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
tags: `\${{ item.tag_list | join(', ') }}`,
url: '${{ item.url }}',
} },
+4 -1
View File
@@ -9,15 +9,18 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles' },
],
columns: ['rank', 'title', 'author', 'reactions', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'author', 'reactions', 'comments', 'reading_time', 'published_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://dev.to/api/articles?top=1&per_page=${{ args.limit }}' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
author: '${{ item.user.username }}',
reactions: '${{ item.public_reactions_count }}',
comments: '${{ item.comments_count }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
tags: `\${{ item.tag_list | join(', ') }}`,
url: '${{ item.url }}',
} },
+4 -1
View File
@@ -15,14 +15,17 @@ cli({
},
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles' },
],
columns: ['rank', 'title', 'reactions', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'reactions', 'comments', 'reading_time', 'published_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://dev.to/api/articles?username=${{ args.username }}&per_page=${{ args.limit }}' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
reactions: '${{ item.public_reactions_count }}',
comments: '${{ item.comments_count }}',
reading_time: '${{ item.reading_time_minutes }}',
published_at: '${{ item.published_at }}',
tags: `\${{ item.tag_list | join(', ') }}`,
url: '${{ item.url }}',
} },
+1 -1
View File
@@ -9,6 +9,6 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: '返回的电影数量' },
],
columns: ['rank', 'title', 'rating', 'quote', 'director', 'year', 'region', 'url'],
columns: ['rank', 'id', 'title', 'rating', 'votes', 'year', 'url'],
func: async (page, args) => loadDoubanMovieHot(page, Number(args.limit) || 20),
});
+14
View File
@@ -0,0 +1,14 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './movie-hot.js';
describe('douban movie-hot command', () => {
it('exposes only fields available from the chart page', () => {
const command = getRegistry().get('douban/movie-hot');
expect(command?.columns).toEqual(['rank', 'id', 'title', 'rating', 'votes', 'year', 'url']);
expect(command?.columns).not.toContain('director');
expect(command?.columns).not.toContain('region');
expect(command?.columns).not.toContain('quote');
});
});
+11 -13
View File
@@ -502,26 +502,20 @@ export async function loadDoubanMovieHot(page, limit) {
let url = titleEl?.getAttribute('href') || '';
if (!title || !url) continue;
if (!url.startsWith('http')) url = 'https://movie.douban.com' + url;
const id = url.match(/subject\\/(\\d+)/)?.[1] || '';
const info = normalize(el.querySelector('.pl2 p')?.textContent);
const infoParts = info.split('/').map((part) => part.trim()).filter(Boolean);
const releaseIndex = (() => {
for (let i = infoParts.length - 1; i >= 0; i -= 1) {
if (/\\d{4}-\\d{2}-\\d{2}|\\d{4}\\/\\d{2}\\/\\d{2}/.test(infoParts[i])) return i;
}
return -1;
})();
const directorPart = releaseIndex >= 1 ? infoParts[releaseIndex - 1] : '';
const regionPart = releaseIndex >= 2 ? infoParts[releaseIndex - 2] : '';
const yearMatch = info.match(/\\b(19|20)\\d{2}\\b/);
const votesText = normalize(el.querySelector('.star .pl')?.textContent);
const votes = parseInt(votesText.replace(/[^0-9]/g, ''), 10) || 0;
results.push({
rank: results.length + 1,
id,
title,
rating: parseFloat(normalize(el.querySelector('.rating_nums')?.textContent)) || 0,
quote: normalize(el.querySelector('.inq')?.textContent),
director: directorPart.replace(/^导演:\\s*/, ''),
votes,
year: yearMatch?.[0] || '',
region: regionPart,
url,
cover: el.querySelector('img')?.getAttribute('src') || '',
});
@@ -530,7 +524,11 @@ export async function loadDoubanMovieHot(page, limit) {
return results;
})()
`);
return Array.isArray(data) ? data : [];
const results = Array.isArray(data) ? data : [];
if (!results.length) {
throw new EmptyResultError('douban movie-hot', 'No movie chart rows were parsed from movie.douban.com/chart.');
}
return results;
}
export function inferDoubanSearchResultType(searchType, item = {}) {
const fallbackType = String(searchType || '').trim() || 'movie';
+79
View File
@@ -3,6 +3,7 @@ import { describe, expect, it, vi } from 'vitest';
import {
getDoubanPhotoExtension,
inferDoubanSearchResultType,
loadDoubanMovieHot,
loadDoubanSubjectDetail,
loadDoubanSubjectPhotos,
normalizeDoubanBookSubject,
@@ -44,6 +45,39 @@ function createFakeSearchItem({ title, url, rating, abstract, cover }) {
};
}
function createFakeMovieHotItem({ title, url, info, rating, votes, cover }) {
return {
querySelector(selector) {
if (selector === '.pl2 a') {
return createFakeNode(title, { href: url });
}
if (selector === '.pl2 p') {
return createFakeNode(info);
}
if (selector === '.star .pl') {
return createFakeNode(votes);
}
if (selector === '.rating_nums') {
return createFakeNode(rating);
}
if (selector === 'img') {
return createFakeNode('', { src: cover });
}
return null;
},
};
}
function runMovieHotEvaluate(script, items) {
const document = {
querySelectorAll(selector) {
return selector === '.item' ? items : [];
},
};
return vm.runInNewContext(script, { document, URL });
}
async function runSearchEvaluate(script, rawItems, domItems) {
const document = {
querySelector(selector) {
@@ -242,6 +276,51 @@ ISBN: 9787544270871
});
});
it('parses movie-hot rows with real chart fields only', async () => {
const items = [
createFakeMovieHotItem({
title: ' 少年与犬 ',
url: '/subject/36840171/',
info: '2025-03-20 / 日本 / 剧情',
rating: '6.8',
votes: '(12345人评价)',
cover: 'https://img1.doubanio.com/view/photo/s_ratio_poster/public/p1.jpg',
}),
];
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce({ blocked: false, title: '豆瓣电影排行榜', href: 'https://movie.douban.com/chart' })
.mockImplementationOnce((script) => runMovieHotEvaluate(script, items)),
};
await expect(loadDoubanMovieHot(page, 20)).resolves.toEqual([
{
rank: 1,
id: '36840171',
title: '少年与犬',
rating: 6.8,
votes: 12345,
year: '2025',
url: 'https://movie.douban.com/subject/36840171/',
cover: 'https://img1.doubanio.com/view/photo/s_ratio_poster/public/p1.jpg',
},
]);
});
it('throws EmptyResultError when movie-hot parses no chart rows', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce({ blocked: false, title: '豆瓣电影排行榜', href: 'https://movie.douban.com/chart' })
.mockResolvedValueOnce([]),
};
await expect(loadDoubanMovieHot(page, 20)).rejects.toThrow('douban movie-hot returned no data');
});
it('loads book subject details from book.douban.com when type=book', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/askstories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.by }}',
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/beststories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.by }}',
+132
View File
@@ -0,0 +1,132 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, EmptyResultError } from '@jackwener/opencli/errors';
import './top.js';
import './best.js';
import './ask.js';
import './new.js';
import './show.js';
import './jobs.js';
import './search.js';
import './read.js';
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
describe('hackernews listing adapters expose item id', () => {
const storyCommands = ['hackernews/top', 'hackernews/best', 'hackernews/ask', 'hackernews/new', 'hackernews/show'];
storyCommands.forEach((key) => {
it(`${key} surfaces id alongside title/score/author/comments/url`, () => {
const cmd = getRegistry().get(key);
expect(cmd?.columns).toEqual(['rank', 'id', 'title', 'score', 'author', 'comments', 'url']);
expect(cmd?.pipeline?.[5]?.map).toMatchObject({
id: '${{ item.id }}',
url: '${{ item.url }}',
});
});
});
it('hackernews/jobs surfaces id alongside title/author/url', () => {
const cmd = getRegistry().get('hackernews/jobs');
expect(cmd?.columns).toEqual(['rank', 'id', 'title', 'author', 'url']);
expect(cmd?.pipeline?.[5]?.map).toMatchObject({
id: '${{ item.id }}',
url: '${{ item.url }}',
});
});
it('hackernews/search surfaces id (algolia objectID) alongside the existing columns', () => {
const cmd = getRegistry().get('hackernews/search');
expect(cmd?.columns).toEqual(['rank', 'id', 'title', 'score', 'author', 'comments', 'url']);
expect(cmd?.pipeline?.[2]?.map).toMatchObject({
id: '${{ item.objectID }}',
});
});
});
describe('hackernews/read adapter', () => {
const cmd = getRegistry().get('hackernews/read');
it('registers the comment-thread shape (type/author/score/text)', () => {
expect(cmd?.columns).toEqual(['type', 'author', 'score', 'text']);
});
it('takes a positional id plus tunable depth/limit/replies/max-length args', () => {
const argNames = (cmd?.args || []).map((a) => a.name);
expect(argNames).toEqual(['id', 'limit', 'depth', 'replies', 'max-length']);
const idArg = cmd?.args?.find((a) => a.name === 'id');
expect(idArg?.required).toBe(true);
expect(idArg?.positional).toBe(true);
});
it('uses the public Firebase API (no browser, public strategy)', () => {
expect(cmd?.browser).toBe(false);
expect(cmd?.strategy).toBe('public');
});
it('fails fast with ArgumentError for non-numeric ids before hitting fetch', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: 'abc', limit: 5, depth: 2, replies: 5, 'max-length': 2000 })).rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with EmptyResultError when the story is missing', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(null), { status: 200 })));
await expect(cmd.func({ id: '99999999', limit: 5, depth: 2, replies: 5, 'max-length': 2000 })).rejects.toThrow(EmptyResultError);
});
it('renders story body, anchor text, and hidden-replies stubs from the public API tree', async () => {
const items = new Map([
['123', {
id: 123,
type: 'story',
by: 'pg',
score: 42,
title: 'Ask HN: Example',
text: '<p>Hello <a href=\"https://example.com\">world</a></p>',
url: 'https://news.ycombinator.com/item?id=123',
kids: [456],
}],
['456', {
id: 456,
type: 'comment',
by: 'sama',
text: '<p>Top level</p>',
kids: [789],
}],
]);
vi.stubGlobal('fetch', vi.fn(async (url) => {
const id = String(url).match(/item\/(\d+)\.json$/)?.[1];
return new Response(JSON.stringify(items.get(id) ?? null), { status: 200 });
}));
const rows = await cmd.func({ id: '123', limit: 5, depth: 1, replies: 5, 'max-length': 2000 });
expect(rows).toEqual([
{
type: 'POST',
author: 'pg',
score: 42,
text: 'Ask HN: Example\nHello world (https://example.com)\nhttps://news.ycombinator.com/item?id=123',
},
{
type: 'L0',
author: 'sama',
score: '',
text: 'Top level',
},
{
type: 'L1',
author: '',
score: '',
text: ' [+1 more replies]',
},
]);
});
});
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of job postings' },
],
columns: ['rank', 'title', 'author', 'url'],
columns: ['rank', 'id', 'title', 'author', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/jobstories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
author: '${{ item.by }}',
url: '${{ item.url }}',
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/newstories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.by }}',
+187
View File
@@ -0,0 +1,187 @@
/**
* Hacker News story reader with threaded comment tree.
*
* Mirrors `reddit read` semantics — fetches a story plus a tree of top-level
* comments and inline replies via the public Firebase API:
* https://hacker-news.firebaseio.com/v0/item/<id>.json
*
* Output rows:
* - first row is the story itself (`type=POST`)
* - each subsequent row is a comment, indented by depth (`L0`, `L1`, …)
* - `[+N more replies]` summary rows whenever depth/limit cuts in
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const HN_ITEM_BASE = 'https://hacker-news.firebaseio.com/v0/item';
async function fetchItem(id) {
const res = await fetch(`${HN_ITEM_BASE}/${id}.json`);
if (!res.ok) {
throw new CommandExecutionError(`HN API HTTP ${res.status} for item ${id}`, 'Check the item ID');
}
return res.json();
}
function requirePositiveInt(value, label) {
if (!Number.isInteger(value) || value <= 0) {
throw new ArgumentError(`${label} must be a positive integer`);
}
return value;
}
function requireMinInt(value, min, label) {
if (!Number.isInteger(value) || value < min) {
throw new ArgumentError(`${label} must be an integer >= ${min}`);
}
return value;
}
/** HN stores comment text as a small HTML subset — convert to plain text. */
function htmlToText(html) {
if (!html) return '';
return String(html)
.replace(/<p>/gi, '\n\n')
.replace(/<\/p>/gi, '')
.replace(/<br\s*\/?>/gi, '\n')
.replace(/<i>(.*?)<\/i>/gi, '$1')
.replace(/<a[^>]*href="([^"]*)"[^>]*>(.*?)<\/a>/gi, '$2 ($1)')
.replace(/<pre><code>([\s\S]*?)<\/code><\/pre>/gi, '\n$1\n')
.replace(/<[^>]+>/g, '')
.replace(/&#x27;/g, "'")
.replace(/&quot;/g, '"')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&amp;/g, '&')
.replace(/&nbsp;/g, ' ')
.replace(/&#x2F;/g, '/')
.trim();
}
function indentLines(text, depth) {
if (depth === 0) return text;
const indent = ' '.repeat(depth);
const prefix = `${indent}> `;
return text.split('\n').map((line) => prefix + line).join('\n');
}
function moreRepliesIndent(depth) {
return ' '.repeat(depth + 1);
}
cli({
site: 'hackernews',
name: 'read',
description: 'Read a Hacker News story and its comment tree',
domain: 'news.ycombinator.com',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'id', required: true, positional: true, help: 'HN item ID (e.g. 39847301)' },
{ name: 'limit', type: 'int', default: 25, help: 'Max top-level comments' },
{ name: 'depth', type: 'int', default: 2, help: 'Max reply depth (1=no replies, 2=one level of replies, etc.)' },
{ name: 'replies', type: 'int', default: 5, help: 'Max replies shown per comment at each level' },
{ name: 'max-length', type: 'int', default: 2000, help: 'Max characters per comment body (min 100)' },
],
columns: ['type', 'author', 'score', 'text'],
func: async (args) => {
const id = String(args.id || '').trim();
if (!/^\d+$/.test(id)) {
throw new ArgumentError(`Invalid HN item id: ${args.id}`, 'Pass a numeric id like 39847301');
}
const limit = requirePositiveInt(args.limit ?? 25, 'hackernews read --limit');
const maxDepth = requirePositiveInt(args.depth ?? 2, 'hackernews read --depth');
const maxReplies = requirePositiveInt(args.replies ?? 5, 'hackernews read --replies');
const maxLength = requireMinInt(args['max-length'] ?? 2000, 100, 'hackernews read --max-length');
const story = await fetchItem(id);
if (!story || story.deleted || story.dead) {
throw new EmptyResultError(`hackernews/${id}`, 'Story not found, deleted, or dead');
}
const results = [];
// Story header row. text combines title + selftext (Ask/Show HN body) + external URL.
const storyBodyRaw = htmlToText(story.text || '');
const storyBody = storyBodyRaw.length > maxLength
? storyBodyRaw.slice(0, maxLength) + '\n... [truncated]'
: storyBodyRaw;
const storyParts = [story.title || ''];
if (storyBody) storyParts.push('\n' + storyBody);
if (story.url) storyParts.push('\n' + story.url);
results.push({
type: 'POST',
author: story.by || '[deleted]',
score: story.score ?? 0,
text: storyParts.join('').trim(),
});
// Walk top-level comments using `kids` ids; fetch the first `limit` ids in parallel.
const topKids = Array.isArray(story.kids) ? story.kids : [];
const topToFetch = topKids.slice(0, limit);
const fetched = await Promise.all(topToFetch.map((kidId) => fetchItem(kidId).catch(() => null)));
async function walkComment(node, depth) {
if (!node || node.deleted || node.dead || node.type !== 'comment') return;
const bodyText = htmlToText(node.text || '');
const truncated = bodyText.length > maxLength
? bodyText.slice(0, maxLength) + '...'
: bodyText;
results.push({
type: depth === 0 ? 'L0' : `L${depth}`,
author: node.by || '[deleted]',
score: '',
text: indentLines(truncated, depth),
});
const childIds = Array.isArray(node.kids) ? node.kids : [];
// At depth cutoff: don't recurse, but show a "+N more replies" stub if any.
if (depth + 1 >= maxDepth) {
if (childIds.length > 0) {
results.push({
type: `L${depth + 1}`,
author: '',
score: '',
text: `${moreRepliesIndent(depth)}[+${childIds.length} more replies]`,
});
}
return;
}
const toProcess = childIds.slice(0, maxReplies);
const replies = await Promise.all(toProcess.map((cid) => fetchItem(cid).catch(() => null)));
for (const reply of replies) {
await walkComment(reply, depth + 1);
}
// "+N more replies" for whatever we skipped at this level
const hidden = childIds.length - toProcess.length;
if (hidden > 0) {
results.push({
type: `L${depth + 1}`,
author: '',
score: '',
text: `${moreRepliesIndent(depth)}[+${hidden} more replies]`,
});
}
}
for (const comment of fetched) {
await walkComment(comment, 0);
}
const hiddenTopLevel = Math.max(0, topKids.length - topToFetch.length);
if (hiddenTopLevel > 0) {
results.push({
type: '',
author: '',
score: '',
text: `[+${hiddenTopLevel} more top-level comments]`,
});
}
return results;
},
});
+2 -1
View File
@@ -16,7 +16,7 @@ cli({
choices: ['relevance', 'date'],
},
],
columns: ['rank', 'title', 'score', 'author', 'comments', 'url'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: {
url: `https://hn.algolia.com/api/v1/\${{ args.sort === 'date' ? 'search_by_date' : 'search' }}`,
@@ -25,6 +25,7 @@ cli({
{ select: 'hits' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.objectID }}',
title: '${{ item.title }}',
score: '${{ item.points }}',
author: '${{ item.author }}',
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/showstories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.by }}',
+2 -1
View File
@@ -9,7 +9,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'url'],
pipeline: [
{ fetch: { url: 'https://hacker-news.firebaseio.com/v0/topstories.json' } },
{ limit: '${{ Math.min((args.limit ? args.limit : 20) + 10, 50) }}' },
@@ -18,6 +18,7 @@ cli({
{ filter: 'item.title && !item.deleted && !item.dead' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.by }}',
+57
View File
@@ -0,0 +1,57 @@
import { cli } from '@jackwener/opencli/registry';
cli({
site: 'instagram',
name: 'collection-create',
description: 'Create a new Instagram saved-posts collection (folder)',
domain: 'www.instagram.com',
args: [
{
name: 'name',
required: true,
positional: true,
help: 'Name of the collection to create',
},
],
columns: ['status', 'collectionId', 'collectionName', 'mediaCount'],
pipeline: [
{ navigate: 'https://www.instagram.com' },
{ evaluate: `(async () => {
const name = \${{ args.name | json }};
if (!name || !String(name).trim()) {
throw new Error('Collection name cannot be empty');
}
const trimmed = String(name).trim();
const csrf = document.cookie.match(/csrftoken=([^;]+)/)?.[1] || '';
if (!csrf) {
throw new Error('csrftoken cookie missing - make sure you are logged in to Instagram');
}
const fd = new FormData();
fd.append('name', trimmed);
fd.append('module_name', 'collection_create');
const res = await fetch('https://www.instagram.com/api/v1/collections/create/', {
method: 'POST',
credentials: 'include',
headers: {
'X-IG-App-ID': '936619743392459',
'X-CSRFToken': csrf,
},
body: fd,
});
if (!res.ok) {
const body = await res.text().catch(() => '');
throw new Error('Failed to create collection: HTTP ' + res.status + (body ? ' - ' + body.slice(0, 200) : ''));
}
const d = await res.json();
if (d?.status && d.status !== 'ok') {
throw new Error('Instagram returned non-ok status: ' + JSON.stringify(d).slice(0, 300));
}
return [{
status: 'Created',
collectionId: String(d?.collection_id ?? ''),
collectionName: String(d?.collection_name ?? trimmed),
mediaCount: d?.collection_media_count ?? 0,
}];
})()
` },
],
});
+91
View File
@@ -0,0 +1,91 @@
import { cli } from '@jackwener/opencli/registry';
cli({
site: 'instagram',
name: 'collection-delete',
description: 'Delete an Instagram saved-posts collection (folder) by name or id',
domain: 'www.instagram.com',
args: [
{
name: 'target',
required: true,
positional: true,
help: 'Collection name (case-insensitive) or numeric collection_id',
},
],
columns: ['status', 'collectionId', 'collectionName'],
pipeline: [
{ navigate: 'https://www.instagram.com' },
{ evaluate: `(async () => {
const target = \${{ args.target | json }};
if (!target || !String(target).trim()) {
throw new Error('Collection target (name or id) cannot be empty');
}
const raw = String(target).trim();
const csrf = document.cookie.match(/csrftoken=([^;]+)/)?.[1] || '';
if (!csrf) {
throw new Error('csrftoken cookie missing - make sure you are logged in to Instagram');
}
const headers = { 'X-IG-App-ID': '936619743392459' };
// Resolve name -> id via /collections/list/. Always go through this path so we can
// surface an explicit error on duplicate names or unknown names instead of relying
// on a 404.
const listRes = await fetch('https://www.instagram.com/api/v1/collections/list/?collection_types=%5B%22MEDIA%22%5D', {
credentials: 'include',
headers,
});
if (!listRes.ok) {
throw new Error('Failed to list collections: HTTP ' + listRes.status + ' - make sure you are logged in to Instagram');
}
const listData = await listRes.json();
const collections = listData?.items || [];
const isNumericId = /^\\d{6,}$/.test(raw);
let id = '';
let resolvedName = '';
if (isNumericId) {
const hit = collections.find((c) => String(c?.collection_id) === raw);
if (!hit) {
throw new Error('Collection id not found in your account: ' + raw);
}
id = String(hit.collection_id);
resolvedName = String(hit.collection_name || '');
} else {
const wanted = raw.toLowerCase();
const matches = collections.filter((c) => String(c?.collection_name || '').trim().toLowerCase() === wanted);
if (matches.length === 0) {
const names = collections.map((c) => c?.collection_name).filter(Boolean);
throw new Error('Collection not found: ' + raw + '. Available: ' + (names.length ? names.join(', ') : '(none)'));
}
if (matches.length > 1) {
const ids = matches.map((c) => c.collection_id).join(', ');
throw new Error('Multiple collections share the name "' + raw + '" (ids: ' + ids + '). Pass the numeric collection_id explicitly to disambiguate.');
}
id = String(matches[0].collection_id);
resolvedName = String(matches[0].collection_name || raw);
}
const fd = new FormData();
fd.append('module_name', 'collection_settings');
const res = await fetch('https://www.instagram.com/api/v1/collections/' + encodeURIComponent(id) + '/delete/', {
method: 'POST',
credentials: 'include',
headers: { ...headers, 'X-CSRFToken': csrf },
body: fd,
});
if (!res.ok) {
const body = await res.text().catch(() => '');
throw new Error('Failed to delete collection: HTTP ' + res.status + (body ? ' - ' + body.slice(0, 200) : ''));
}
const d = await res.json().catch(() => ({}));
if (d?.status && d.status !== 'ok') {
throw new Error('Instagram returned non-ok status: ' + JSON.stringify(d).slice(0, 300));
}
return [{
status: 'Deleted',
collectionId: id,
collectionName: resolvedName,
}];
})()
` },
],
});
+21 -7
View File
@@ -2,23 +2,37 @@ import { cli } from '@jackwener/opencli/registry';
cli({
site: 'instagram',
name: 'saved',
description: 'Get your saved Instagram posts',
description: 'Get your saved Instagram posts (optionally from a specific collection)',
domain: 'www.instagram.com',
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of saved posts' },
{ name: 'collection', help: 'Collection name (case-insensitive). Omit for the default "All posts" feed.' },
],
columns: ['index', 'user', 'caption', 'likes', 'comments', 'type'],
pipeline: [
{ navigate: 'https://www.instagram.com' },
{ evaluate: `(async () => {
const limit = \${{ args.limit }};
const res = await fetch(
'https://www.instagram.com/api/v1/feed/saved/posts/',
{
credentials: 'include',
headers: { 'X-IG-App-ID': '936619743392459' }
const collectionArg = \${{ args.collection | json }};
const headers = { 'X-IG-App-ID': '936619743392459' };
const opts = { credentials: 'include', headers };
let endpoint = 'https://www.instagram.com/api/v1/feed/saved/posts/';
if (collectionArg && String(collectionArg).trim()) {
const wanted = String(collectionArg).trim().toLowerCase();
const listRes = await fetch('https://www.instagram.com/api/v1/collections/list/?collection_types=%5B%22MEDIA%22%2C%22ALL_MEDIA_AUTO_COLLECTION%22%5D', opts);
if (!listRes.ok) throw new Error('Failed to list collections: HTTP ' + listRes.status + ' - make sure you are logged in to Instagram');
const listData = await listRes.json();
const collections = listData?.items || [];
const match = collections.find((c) => String(c?.collection_name || '').trim().toLowerCase() === wanted);
if (!match) {
const names = collections.map((c) => c?.collection_name).filter(Boolean);
throw new Error('Collection not found: ' + collectionArg + '. Available: ' + (names.length ? names.join(', ') : '(none)'));
}
);
endpoint = 'https://www.instagram.com/api/v1/feed/collection/' + encodeURIComponent(match.collection_id) + '/posts/';
}
const res = await fetch(endpoint, opts);
if (!res.ok) throw new Error('HTTP ' + res.status + ' - make sure you are logged in to Instagram');
const data = await res.json();
return (data?.items || []).slice(0, limit).map((item, i) => {
+3 -1
View File
@@ -9,15 +9,17 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'created_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://lobste.rs/active.json' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.short_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.submitter_user }}',
comments: '${{ item.comment_count }}',
created_at: '${{ item.created_at }}',
tags: `\${{ item.tags | join(', ') }}`,
url: '${{ item.comments_url }}',
} },
+3 -1
View File
@@ -9,15 +9,17 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'created_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://lobste.rs/hottest.json' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.short_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.submitter_user }}',
comments: '${{ item.comment_count }}',
created_at: '${{ item.created_at }}',
tags: `\${{ item.tags | join(', ') }}`,
url: '${{ item.comments_url }}',
} },
+169
View File
@@ -0,0 +1,169 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './hot.js';
import './active.js';
import './newest.js';
import './tag.js';
import './read.js';
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
describe('lobsters listing adapters expose short_id and created_at', () => {
const listings = ['lobsters/hot', 'lobsters/active', 'lobsters/newest', 'lobsters/tag'];
listings.forEach((key) => {
it(`${key} surfaces id (short_id) and created_at on every row`, () => {
const cmd = getRegistry().get(key);
expect(cmd?.columns).toEqual(['rank', 'id', 'title', 'score', 'author', 'comments', 'created_at', 'tags', 'url']);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.short_id }}',
created_at: '${{ item.created_at }}',
url: '${{ item.comments_url }}',
});
});
});
});
describe('lobsters/read adapter', () => {
const cmd = getRegistry().get('lobsters/read');
it('registers the comment-thread shape (type/author/score/text)', () => {
expect(cmd?.columns).toEqual(['type', 'author', 'score', 'text']);
});
it('takes a positional short_id plus tunable depth/limit/replies/max-length args', () => {
const argNames = (cmd?.args || []).map((a) => a.name);
expect(argNames).toEqual(['id', 'limit', 'depth', 'replies', 'max-length']);
const idArg = cmd?.args?.find((a) => a.name === 'id');
expect(idArg?.required).toBe(true);
expect(idArg?.positional).toBe(true);
});
it('uses the public lobste.rs JSON endpoint (no browser, public strategy)', () => {
expect(cmd?.browser).toBe(false);
expect(cmd?.strategy).toBe('public');
});
it('fails fast with ArgumentError for non-alphanumeric short_id before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: 'BAD!ID', limit: 5, depth: 2, replies: 5, 'max-length': 2000 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with EmptyResultError on 404', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('Not found', { status: 404 })));
await expect(cmd.func({ id: 'missing', limit: 5, depth: 2, replies: 5, 'max-length': 2000 }))
.rejects.toThrow(EmptyResultError);
});
it('fails fast with ArgumentError when max-length is below the minimum before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: 'abc123', limit: 5, depth: 2, replies: 5, 'max-length': 99 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with CommandExecutionError on non-404 HTTP failures', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('oops', { status: 503 })));
await expect(cmd.func({ id: 'abc123', limit: 5, depth: 2, replies: 5, 'max-length': 2000 }))
.rejects.toThrow(CommandExecutionError);
});
it('builds a threaded tree from the flat comments[] using parent_comment', async () => {
const story = {
short_id: 'abc123',
title: 'Hello world',
url: 'https://example.com/post',
score: 42,
submitter_user: 'pg',
description_plain: 'Some intro text.',
comments_url: 'https://lobste.rs/s/abc123/hello_world',
comments: [
{
short_id: 'top1',
parent_comment: null,
score: 5,
commenting_user: 'alice',
comment_plain: 'Top one',
is_deleted: false,
},
{
short_id: 'reply1',
parent_comment: 'top1',
score: 3,
commenting_user: 'bob',
comment_plain: 'A reply',
is_deleted: false,
},
{
short_id: 'reply2',
parent_comment: 'top1',
score: 1,
commenting_user: 'carol',
comment_plain: 'Another reply',
is_deleted: false,
},
{
short_id: 'top2',
parent_comment: null,
score: 4,
commenting_user: 'dave',
comment_plain: 'Top two',
is_deleted: false,
},
],
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(story), { status: 200 })));
const rows = await cmd.func({ id: 'abc123', limit: 5, depth: 2, replies: 5, 'max-length': 2000 });
expect(rows).toEqual([
{
type: 'POST',
author: 'pg',
score: 42,
text: 'Hello world\nSome intro text.\nhttps://example.com/post',
},
{ type: 'L0', author: 'alice', score: 5, text: 'Top one' },
{ type: 'L1', author: 'bob', score: 3, text: ' > A reply' },
{ type: 'L1', author: 'carol', score: 1, text: ' > Another reply' },
{ type: 'L0', author: 'dave', score: 4, text: 'Top two' },
]);
});
it('emits a "+N more replies" stub when depth cutoff hides children', async () => {
const story = {
short_id: 'abc123',
title: 'Hi',
url: '',
score: 1,
submitter_user: 'u',
description_plain: '',
comments: [
{ short_id: 't1', parent_comment: null, commenting_user: 'a', comment_plain: 'top', is_deleted: false },
{ short_id: 'r1', parent_comment: 't1', commenting_user: 'b', comment_plain: 'r', is_deleted: false },
],
};
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify(story), { status: 200 })));
const rows = await cmd.func({ id: 'abc123', limit: 5, depth: 1, replies: 5, 'max-length': 2000 });
expect(rows).toEqual([
{ type: 'POST', author: 'u', score: 1, text: 'Hi' },
{ type: 'L0', author: 'a', score: '', text: 'top' },
{ type: 'L1', author: '', score: '', text: ' [+1 more replies]' },
]);
});
});
+3 -1
View File
@@ -9,15 +9,17 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'created_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://lobste.rs/newest.json' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.short_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.submitter_user }}',
comments: '${{ item.comment_count }}',
created_at: '${{ item.created_at }}',
tags: `\${{ item.tags | join(', ') }}`,
url: '${{ item.comments_url }}',
} },
+195
View File
@@ -0,0 +1,195 @@
/**
* Lobste.rs story reader with threaded comment tree.
*
* Mirrors `hackernews read` semantics. The lobsters JSON endpoint:
* https://lobste.rs/s/<short_id>.json
* already returns the story plus a flat `comments[]` array where each entry
* carries `parent_comment` (short_id of parent or null) and `depth` — so we
* just need one HTTP call, then build a children map and DFS.
*
* Output rows:
* - first row is the story itself (`type=POST`)
* - each subsequent row is a comment, indented by depth (`L0`, `L1`, …)
* - `[+N more replies]` summary rows whenever depth/limit cuts in
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const LOBSTERS_STORY_BASE = 'https://lobste.rs/s';
async function fetchStory(shortId) {
const res = await fetch(`${LOBSTERS_STORY_BASE}/${shortId}.json`);
if (res.status === 404) {
throw new EmptyResultError(`lobsters/${shortId}`, 'Story not found');
}
if (!res.ok) {
throw new CommandExecutionError(`Lobsters API HTTP ${res.status} for story ${shortId}`, 'Check the short id');
}
return res.json();
}
function requirePositiveInt(value, label) {
if (!Number.isInteger(value) || value <= 0) {
throw new ArgumentError(`${label} must be a positive integer`);
}
return value;
}
function requireMinInt(value, min, label) {
if (!Number.isInteger(value) || value < min) {
throw new ArgumentError(`${label} must be an integer >= ${min}`);
}
return value;
}
/** Lobsters returns comment text as a small HTML subset — convert to plain text. */
function htmlToText(html) {
if (!html) return '';
return String(html)
.replace(/<p>/gi, '\n\n')
.replace(/<\/p>/gi, '')
.replace(/<br\s*\/?>/gi, '\n')
.replace(/<i>(.*?)<\/i>/gi, '$1')
.replace(/<em>(.*?)<\/em>/gi, '$1')
.replace(/<strong>(.*?)<\/strong>/gi, '$1')
.replace(/<a[^>]*href="([^"]*)"[^>]*>(.*?)<\/a>/gi, '$2 ($1)')
.replace(/<pre><code>([\s\S]*?)<\/code><\/pre>/gi, '\n$1\n')
.replace(/<code>(.*?)<\/code>/gi, '`$1`')
.replace(/<[^>]+>/g, '')
.replace(/&#x27;/g, "'")
.replace(/&apos;/g, "'")
.replace(/&quot;/g, '"')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/&amp;/g, '&')
.replace(/&nbsp;/g, ' ')
.replace(/&#x2F;/g, '/')
.trim();
}
function indentLines(text, depth) {
if (depth === 0) return text;
const indent = ' '.repeat(depth);
const prefix = `${indent}> `;
return text.split('\n').map((line) => prefix + line).join('\n');
}
function moreRepliesIndent(depth) {
return ' '.repeat(depth + 1);
}
cli({
site: 'lobsters',
name: 'read',
description: 'Read a Lobste.rs story and its comment tree',
domain: 'lobste.rs',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Lobste.rs short_id (e.g. 6cmh6h)' },
{ name: 'limit', type: 'int', default: 25, help: 'Max top-level comments' },
{ name: 'depth', type: 'int', default: 2, help: 'Max reply depth (1=no replies, 2=one level of replies, etc.)' },
{ name: 'replies', type: 'int', default: 5, help: 'Max replies shown per comment at each level' },
{ name: 'max-length', type: 'int', default: 2000, help: 'Max characters per comment body (min 100)' },
],
columns: ['type', 'author', 'score', 'text'],
func: async (args) => {
const shortId = String(args.id || '').trim();
if (!/^[a-z0-9]+$/.test(shortId)) {
throw new ArgumentError(`Invalid Lobsters short_id: ${args.id}`, 'Pass a lowercase alphanumeric id like 6cmh6h');
}
const limit = requirePositiveInt(args.limit ?? 25, 'lobsters read --limit');
const maxDepth = requirePositiveInt(args.depth ?? 2, 'lobsters read --depth');
const maxReplies = requirePositiveInt(args.replies ?? 5, 'lobsters read --replies');
const maxLength = requireMinInt(args['max-length'] ?? 2000, 100, 'lobsters read --max-length');
const story = await fetchStory(shortId);
if (!story || !story.short_id) {
throw new EmptyResultError(`lobsters/${shortId}`, 'Story not found');
}
const results = [];
// Story header — title, body (description_plain, often empty for link posts), then external url.
const storyBodyRaw = (story.description_plain || htmlToText(story.description || '')).trim();
const storyBody = storyBodyRaw.length > maxLength
? storyBodyRaw.slice(0, maxLength) + '\n... [truncated]'
: storyBodyRaw;
const storyParts = [story.title || ''];
if (storyBody) storyParts.push('\n' + storyBody);
if (story.url) storyParts.push('\n' + story.url);
results.push({
type: 'POST',
author: story.submitter_user || '[deleted]',
score: story.score ?? 0,
text: storyParts.join('').trim(),
});
// Build a map: parent_comment -> [child comments in order]. Top-level keyed by null.
const allComments = Array.isArray(story.comments) ? story.comments : [];
const childrenMap = new Map();
for (const c of allComments) {
const parent = c.parent_comment || null;
if (!childrenMap.has(parent)) childrenMap.set(parent, []);
childrenMap.get(parent).push(c);
}
function emit(comment, depth) {
if (!comment || comment.is_deleted || comment.is_moderated) return;
const bodyRaw = (comment.comment_plain || htmlToText(comment.comment || '')).trim();
const truncated = bodyRaw.length > maxLength
? bodyRaw.slice(0, maxLength) + '...'
: bodyRaw;
results.push({
type: depth === 0 ? 'L0' : `L${depth}`,
author: comment.commenting_user || '[deleted]',
score: comment.score ?? '',
text: indentLines(truncated, depth),
});
const kids = childrenMap.get(comment.short_id) || [];
// At depth cutoff: don't recurse, but show a "+N more replies" stub if any.
if (depth + 1 >= maxDepth) {
if (kids.length > 0) {
results.push({
type: `L${depth + 1}`,
author: '',
score: '',
text: `${moreRepliesIndent(depth)}[+${kids.length} more replies]`,
});
}
return;
}
const toShow = kids.slice(0, maxReplies);
for (const kid of toShow) emit(kid, depth + 1);
const hidden = kids.length - toShow.length;
if (hidden > 0) {
results.push({
type: `L${depth + 1}`,
author: '',
score: '',
text: `${moreRepliesIndent(depth)}[+${hidden} more replies]`,
});
}
}
const topLevel = childrenMap.get(null) || [];
const topToShow = topLevel.slice(0, limit);
for (const top of topToShow) emit(top, 0);
const hiddenTopLevel = topLevel.length - topToShow.length;
if (hiddenTopLevel > 0) {
results.push({
type: '',
author: '',
score: '',
text: `[+${hiddenTopLevel} more top-level comments]`,
});
}
return results;
},
});
+3 -1
View File
@@ -15,15 +15,17 @@ cli({
},
{ name: 'limit', type: 'int', default: 20, help: 'Number of stories' },
],
columns: ['rank', 'title', 'score', 'author', 'comments', 'tags'],
columns: ['rank', 'id', 'title', 'score', 'author', 'comments', 'created_at', 'tags', 'url'],
pipeline: [
{ fetch: { url: 'https://lobste.rs/t/${{ args.tag }}.json' } },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.short_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
author: '${{ item.submitter_user }}',
comments: '${{ item.comment_count }}',
created_at: '${{ item.created_at }}',
tags: `\${{ item.tags | join(', ') }}`,
url: '${{ item.comments_url }}',
} },
+5 -1
View File
@@ -12,7 +12,7 @@ cli({
},
{ name: 'limit', type: 'int', default: 20, help: 'Number of posts' },
],
columns: ['rank', 'title', 'subreddit', 'score', 'comments'],
columns: ['rank', 'title', 'subreddit', 'score', 'comments', 'postId', 'author', 'url'],
pipeline: [
{ navigate: 'https://www.reddit.com' },
{ evaluate: `(async () => {
@@ -29,6 +29,7 @@ cli({
score: c.data.score,
comments: c.data.num_comments,
author: c.data.author,
postId: c.data.id,
url: 'https://www.reddit.com' + c.data.permalink,
}));
})()
@@ -39,6 +40,9 @@ cli({
subreddit: '${{ item.subreddit }}',
score: '${{ item.score }}',
comments: '${{ item.comments }}',
postId: '${{ item.postId }}',
author: '${{ item.author }}',
url: '${{ item.url }}',
} },
{ limit: '${{ args.limit }}' },
],
+18
View File
@@ -0,0 +1,18 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './hot.js';
describe('reddit hot adapter', () => {
const command = getRegistry().get('reddit/hot');
it('registers postId, author, and url columns in the hot-list shape', () => {
expect(command?.columns).toEqual(['rank', 'title', 'subreddit', 'score', 'comments', 'postId', 'author', 'url']);
expect(command?.pipeline?.[1]?.evaluate).toContain('postId: c.data.id');
expect(command?.pipeline?.[1]?.evaluate).toContain("'https://www.reddit.com' + c.data.permalink");
expect(command?.pipeline?.[2]?.map).toMatchObject({
postId: '${{ item.postId }}',
author: '${{ item.author }}',
url: '${{ item.url }}',
});
});
});
+10 -3
View File
@@ -9,17 +9,24 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 10, help: 'Max number of results' },
],
columns: ['bounty', 'title', 'score', 'answers', 'url'],
columns: ['rank', 'id', 'bounty', 'title', 'score', 'answers', 'views', 'is_answered', 'tags', 'author', 'creation_date', 'url'],
pipeline: [
{ fetch: {
url: 'https://api.stackexchange.com/2.3/questions/featured?order=desc&sort=activity&site=stackoverflow',
url: 'https://api.stackexchange.com/2.3/questions/featured?order=desc&sort=activity&site=stackoverflow&pagesize=${{ args.limit }}',
} },
{ select: 'items' },
{ map: {
title: '${{ item.title }}',
rank: '${{ index + 1 }}',
id: '${{ item.question_id }}',
bounty: '${{ item.bounty_amount }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
answers: '${{ item.answer_count }}',
views: '${{ item.view_count }}',
is_answered: '${{ item.is_answered }}',
tags: `\${{ item.tags | join(', ') }}`,
author: '${{ item.owner.display_name }}',
creation_date: '${{ item.creation_date }}',
url: '${{ item.link }}',
} },
{ limit: '${{ args.limit }}' },
+9 -2
View File
@@ -9,14 +9,21 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 10, help: 'Max number of results' },
],
columns: ['title', 'score', 'answers', 'url'],
columns: ['rank', 'id', 'title', 'score', 'answers', 'views', 'is_answered', 'tags', 'author', 'creation_date', 'url'],
pipeline: [
{ fetch: { url: 'https://api.stackexchange.com/2.3/questions?order=desc&sort=hot&site=stackoverflow' } },
{ fetch: { url: 'https://api.stackexchange.com/2.3/questions?order=desc&sort=hot&site=stackoverflow&pagesize=${{ args.limit }}' } },
{ select: 'items' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.question_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
answers: '${{ item.answer_count }}',
views: '${{ item.view_count }}',
is_answered: '${{ item.is_answered }}',
tags: `\${{ item.tags | join(', ') }}`,
author: '${{ item.owner.display_name }}',
creation_date: '${{ item.creation_date }}',
url: '${{ item.link }}',
} },
{ limit: '${{ args.limit }}' },
+313
View File
@@ -0,0 +1,313 @@
/**
* Stack Overflow question reader.
*
* Hits the public Stack Exchange API:
* GET /questions/{id}?site=stackoverflow&filter=withbody
* GET /questions/{id}/answers?site=stackoverflow&filter=withbody
* GET /questions/{id}/comments?site=stackoverflow&filter=withbody
* GET /answers/{a1;a2;...}/comments?site=stackoverflow&filter=withbody
*
* Three calls are needed because comments under answers are not bundled in
* the answers payload. We batch all answer-comment fetches into a single
* semicolon-joined call. SO has its own quota (300/day for unauthenticated
* IP), but a `read` consumes at most 4 quota units, or 5 when the accepted
* answer is missing from the requested answer page and must be fetched by id.
*
* Output rows mirror `hackernews read` and `lobsters read`:
* - first row is the question itself (`type=POST`)
* - one row per top-level question comment (`type=Q-COMMENT`)
* - per answer: an `ANSWER` row plus its `A-COMMENT` rows indented under it
* - the accepted answer (if any) is surfaced first and tagged `accepted=true`
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const SE_API_BASE = 'https://api.stackexchange.com/2.3';
const SE_SITE = 'stackoverflow';
const SE_MAX_PAGE_SIZE = 100;
async function fetchJson(url, label) {
let res;
try {
res = await fetch(url);
} catch (e) {
const detail = e instanceof Error ? e.message : String(e);
throw new CommandExecutionError(
`Network failure fetching ${label}: ${detail}`,
'Check connectivity to api.stackexchange.com',
);
}
if (res.status === 404) {
throw new EmptyResultError(label, `${label} not found`);
}
if (!res.ok) {
throw new CommandExecutionError(
`Stack Exchange API HTTP ${res.status} for ${label}`,
'Check the question id and quota (300/day per IP)',
);
}
let json;
try {
json = await res.json();
} catch (e) {
const detail = e instanceof Error ? e.message : String(e);
throw new CommandExecutionError(
`Malformed JSON from Stack Exchange API for ${label}: ${detail}`,
'The API returned a non-JSON body — likely a transient outage',
);
}
if (json && json.error_id) {
throw new CommandExecutionError(
`Stack Exchange API error ${json.error_id} (${json.error_name}) for ${label}: ${json.error_message || ''}`,
'Common causes: invalid filter, throttled, or quota exhausted',
);
}
return json;
}
/**
* CLI args may arrive as strings (`--limit 5` → `'5'`) when not coerced by the
* arg type system. Coerce-then-validate so `Number.isInteger` actually catches
* the bad cases, and reject NaN explicitly.
*/
function coerceInt(value) {
if (value === undefined || value === null || value === '') return NaN;
const n = typeof value === 'number' ? value : Number(value);
return Number.isFinite(n) && Number.isInteger(n) ? n : NaN;
}
function requireMinInt(value, min, label) {
const n = coerceInt(value);
if (!Number.isInteger(n) || n < min) {
throw new ArgumentError(`${label} must be an integer >= ${min}, got ${JSON.stringify(value)}`);
}
return n;
}
function requireBoundedInt(value, min, max, label) {
const n = coerceInt(value);
if (!Number.isInteger(n) || n < min || n > max) {
throw new ArgumentError(`${label} must be an integer between ${min} and ${max}, got ${JSON.stringify(value)}`);
}
return n;
}
function byAcceptedThenScoreDesc(question, answers) {
const acceptedAnswerId = question.accepted_answer_id;
return answers
.slice()
.sort((a, b) => {
const aAccepted = a.is_accepted || (acceptedAnswerId && a.answer_id === acceptedAnswerId);
const bAccepted = b.is_accepted || (acceptedAnswerId && b.answer_id === acceptedAnswerId);
if (aAccepted !== bAccepted) return aAccepted ? -1 : 1;
return (b.score ?? 0) - (a.score ?? 0);
});
}
async function fetchMissingAcceptedAnswer(question, answers, label) {
const acceptedAnswerId = question.accepted_answer_id;
if (!acceptedAnswerId || answers.some((answer) => answer.answer_id === acceptedAnswerId)) {
return answers;
}
const acceptedData = await fetchJson(
`${SE_API_BASE}/answers/${acceptedAnswerId}?site=${SE_SITE}&filter=withbody`,
`${label}/accepted-answer`,
);
const accepted = (acceptedData.items || [])[0];
return accepted ? answers.concat(accepted) : answers;
}
async function fetchAnswerCommentsByAnswerId(answers, commentsLimit, label) {
const answerCommentsByAnswerId = new Map();
if (answers.length === 0) return answerCommentsByAnswerId;
const ids = answers.map((a) => a.answer_id).join(';');
const pageSize = Math.min(SE_MAX_PAGE_SIZE, answers.length * commentsLimit);
const ansCommentsData = await fetchJson(
`${SE_API_BASE}/answers/${ids}/comments?site=${SE_SITE}&filter=withbody&order=asc&sort=creation&pagesize=${pageSize}`,
`${label}/answer-comments`,
);
for (const c of ansCommentsData.items || []) {
if (!c.post_id) continue;
if (!answerCommentsByAnswerId.has(c.post_id)) {
answerCommentsByAnswerId.set(c.post_id, []);
}
answerCommentsByAnswerId.get(c.post_id).push(c);
}
if (ansCommentsData.has_more) {
const missingForSelectedAnswer = answers.some((answer) => {
const comments = answerCommentsByAnswerId.get(answer.answer_id) || [];
return comments.length < commentsLimit;
});
if (missingForSelectedAnswer) {
throw new CommandExecutionError(
`Stack Exchange answer comments for ${label} exceed one API page`,
'Lower --answers-limit or --comments-limit; refusing to return a partial answer-comment set.',
);
}
}
return answerCommentsByAnswerId;
}
const NAMED_ENTITIES = {
amp: '&', lt: '<', gt: '>', quot: '"', apos: "'", nbsp: ' ',
hellip: '…', mdash: '—', ndash: '', laquo: '«', raquo: '»',
copy: '©', reg: '®', trade: '™', euro: '€', pound: '£', yen: '¥',
rsquo: '', lsquo: '', rdquo: '”', ldquo: '“',
};
/** Decode named/numeric HTML entities. Used on both body HTML and display names. */
function decodeEntities(text) {
if (!text) return '';
return String(text)
.replace(/&#x([0-9a-fA-F]+);/g, (_, hex) => {
const code = parseInt(hex, 16);
return Number.isFinite(code) ? String.fromCodePoint(code) : '';
})
.replace(/&#(\d+);/g, (_, dec) => {
const code = parseInt(dec, 10);
return Number.isFinite(code) ? String.fromCodePoint(code) : '';
})
.replace(/&([a-zA-Z]+);/g, (match, name) => NAMED_ENTITIES[name] ?? match);
}
/** SO renders bodies as HTML — convert to plain text similar to HN/lobsters. */
function htmlToText(html) {
if (!html) return '';
const stripped = String(html)
.replace(/<pre[^>]*><code[^>]*>([\s\S]*?)<\/code><\/pre>/gi, '\n$1\n')
.replace(/<code[^>]*>(.*?)<\/code>/gi, '`$1`')
.replace(/<p[^>]*>/gi, '\n\n')
.replace(/<\/p>/gi, '')
.replace(/<br\s*\/?>/gi, '\n')
.replace(/<li[^>]*>/gi, '\n- ')
.replace(/<\/li>/gi, '')
.replace(/<a[^>]*href="([^"]*)"[^>]*>(.*?)<\/a>/gi, '$2 ($1)')
.replace(/<[^>]+>/g, '');
return decodeEntities(stripped)
.replace(/\n{3,}/g, '\n\n')
.trim();
}
function authorName(owner) {
return decodeEntities(owner?.display_name || '') || '[deleted]';
}
function truncate(text, maxLength) {
if (!text || text.length <= maxLength) return text || '';
return text.slice(0, maxLength) + ' ... [truncated]';
}
function indentLines(text, depth) {
if (depth === 0) return text;
const indent = ' '.repeat(depth);
const prefix = `${indent}> `;
return text.split('\n').map((line) => prefix + line).join('\n');
}
cli({
site: 'stackoverflow',
name: 'read',
description: 'Read a Stack Overflow question with answers and comments',
domain: 'stackoverflow.com',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Stack Overflow question id (numeric, e.g. 79935770)' },
{ name: 'answers-limit', type: 'int', default: 10, help: 'Max answers to include (1-100; accepted answer always included first)' },
{ name: 'comments-limit', type: 'int', default: 5, help: 'Max comments per question/answer (1-100)' },
{ name: 'max-length', type: 'int', default: 4000, help: 'Max characters per body / answer / comment (min 100)' },
],
columns: ['type', 'author', 'score', 'accepted', 'text'],
func: async (args) => {
const id = String(args.id || '').trim();
if (!/^\d+$/.test(id)) {
throw new ArgumentError(`Invalid Stack Overflow question id: ${args.id}`, 'Pass a numeric id like 79935770');
}
const answersLimit = requireBoundedInt(args['answers-limit'] ?? 10, 1, SE_MAX_PAGE_SIZE, 'stackoverflow read --answers-limit');
const commentsLimit = requireBoundedInt(args['comments-limit'] ?? 5, 1, SE_MAX_PAGE_SIZE, 'stackoverflow read --comments-limit');
const maxLength = requireMinInt(args['max-length'] ?? 4000, 100, 'stackoverflow read --max-length');
const label = `stackoverflow/${id}`;
const qUrl = `${SE_API_BASE}/questions/${id}?site=${SE_SITE}&filter=withbody`;
const qData = await fetchJson(qUrl, label);
const question = (qData.items || [])[0];
if (!question) {
throw new EmptyResultError(label, 'Question not found');
}
// Fetch question comments and answers in parallel.
const [qCommentsData, answersData] = await Promise.all([
fetchJson(
`${SE_API_BASE}/questions/${id}/comments?site=${SE_SITE}&filter=withbody&order=asc&sort=creation&pagesize=${commentsLimit}`,
`${label}/comments`,
),
fetchJson(
`${SE_API_BASE}/questions/${id}/answers?site=${SE_SITE}&filter=withbody&order=desc&sort=votes&pagesize=${answersLimit}`,
`${label}/answers`,
),
]);
const allAnswers = await fetchMissingAcceptedAnswer(question, answersData.items || [], label);
// Surface accepted answer first, then by score order.
const orderedAnswers = byAcceptedThenScoreDesc(question, allAnswers).slice(0, answersLimit);
const answerCommentsByAnswerId = await fetchAnswerCommentsByAnswerId(orderedAnswers, commentsLimit, label);
const rows = [];
// POST row: question
const qBody = htmlToText(question.body || '');
const qTextParts = [
question.title || '',
qBody,
question.link || '',
].filter(Boolean);
rows.push({
type: 'POST',
author: authorName(question.owner),
score: question.score ?? 0,
accepted: '',
text: truncate(qTextParts.join('\n\n'), maxLength),
});
// Q-COMMENT rows
const qComments = (qCommentsData.items || []).slice(0, commentsLimit);
for (const c of qComments) {
const text = indentLines(htmlToText(c.body || ''), 1);
rows.push({
type: 'Q-COMMENT',
author: authorName(c.owner),
score: c.score ?? 0,
accepted: '',
text: truncate(text, maxLength),
});
}
// ANSWER + A-COMMENT rows
for (const ans of orderedAnswers) {
rows.push({
type: 'ANSWER',
author: authorName(ans.owner),
score: ans.score ?? 0,
accepted: ans.is_accepted ? 'true' : '',
text: truncate(htmlToText(ans.body || ''), maxLength),
});
const ansComments = (answerCommentsByAnswerId.get(ans.answer_id) || []).slice(0, commentsLimit);
for (const c of ansComments) {
const text = indentLines(htmlToText(c.body || ''), 1);
rows.push({
type: 'A-COMMENT',
author: authorName(c.owner),
score: c.score ?? 0,
accepted: '',
text: truncate(text, maxLength),
});
}
}
return rows;
},
});
+9 -2
View File
@@ -10,16 +10,23 @@ cli({
{ name: 'query', type: 'string', required: true, positional: true, help: 'Search query' },
{ name: 'limit', type: 'int', default: 10, help: 'Max number of results' },
],
columns: ['title', 'score', 'answers', 'url'],
columns: ['rank', 'id', 'title', 'score', 'answers', 'views', 'is_answered', 'tags', 'author', 'creation_date', 'url'],
pipeline: [
{ fetch: {
url: 'https://api.stackexchange.com/2.3/search/advanced?order=desc&sort=relevance&q=${{ args.query }}&site=stackoverflow',
url: 'https://api.stackexchange.com/2.3/search/advanced?order=desc&sort=relevance&q=${{ args.query }}&site=stackoverflow&pagesize=${{ args.limit }}',
} },
{ select: 'items' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.question_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
answers: '${{ item.answer_count }}',
views: '${{ item.view_count }}',
is_answered: '${{ item.is_answered }}',
tags: `\${{ item.tags | join(', ') }}`,
author: '${{ item.owner.display_name }}',
creation_date: '${{ item.creation_date }}',
url: '${{ item.link }}',
} },
{ limit: '${{ args.limit }}' },
+346
View File
@@ -0,0 +1,346 @@
import { afterEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './hot.js';
import './search.js';
import './unanswered.js';
import './bounties.js';
import './read.js';
afterEach(() => {
vi.unstubAllGlobals();
vi.restoreAllMocks();
});
describe('stackoverflow listing adapters surface question_id/tags/views/owner', () => {
it('stackoverflow/hot has the agent-native column shape', () => {
const cmd = getRegistry().get('stackoverflow/hot');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'score', 'answers', 'views',
'is_answered', 'tags', 'author', 'creation_date', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.question_id }}',
views: '${{ item.view_count }}',
is_answered: '${{ item.is_answered }}',
author: '${{ item.owner.display_name }}',
creation_date: '${{ item.creation_date }}',
});
});
it('stackoverflow/search has the agent-native column shape', () => {
const cmd = getRegistry().get('stackoverflow/search');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'score', 'answers', 'views',
'is_answered', 'tags', 'author', 'creation_date', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.question_id }}',
views: '${{ item.view_count }}',
});
});
it('stackoverflow/unanswered drops is_answered (always false) but keeps the rest', () => {
const cmd = getRegistry().get('stackoverflow/unanswered');
expect(cmd?.columns).toEqual([
'rank', 'id', 'title', 'score', 'answers', 'views',
'tags', 'author', 'creation_date', 'url',
]);
expect(cmd?.columns).not.toContain('is_answered');
});
it('stackoverflow/bounties keeps the bounty column at the front', () => {
const cmd = getRegistry().get('stackoverflow/bounties');
expect(cmd?.columns).toEqual([
'rank', 'id', 'bounty', 'title', 'score', 'answers', 'views',
'is_answered', 'tags', 'author', 'creation_date', 'url',
]);
const mapStep = cmd?.pipeline?.find((step) => step.map);
expect(mapStep?.map).toMatchObject({
id: '${{ item.question_id }}',
bounty: '${{ item.bounty_amount }}',
});
});
});
describe('stackoverflow/read adapter', () => {
const cmd = getRegistry().get('stackoverflow/read');
it('registers the question/answer/comment row shape', () => {
expect(cmd?.columns).toEqual(['type', 'author', 'score', 'accepted', 'text']);
});
it('takes a positional id plus tunable answers-limit/comments-limit/max-length', () => {
const argNames = (cmd?.args || []).map((a) => a.name);
expect(argNames).toEqual(['id', 'answers-limit', 'comments-limit', 'max-length']);
const idArg = cmd?.args?.find((a) => a.name === 'id');
expect(idArg?.required).toBe(true);
expect(idArg?.positional).toBe(true);
});
it('uses the public Stack Exchange API (no browser, public strategy)', () => {
expect(cmd?.browser).toBe(false);
expect(cmd?.strategy).toBe('public');
});
it('fails fast with ArgumentError for non-numeric id before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: 'not-a-number', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with ArgumentError for max-length below 100 before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 50 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('fails fast with EmptyResultError when the question lookup returns empty items', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response(JSON.stringify({ items: [] }), { status: 200 }),
));
await expect(cmd.func({ id: '99999999', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(EmptyResultError);
});
it('surfaces Stack Exchange API throttle / quota errors as CommandExecutionError', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response(JSON.stringify({ error_id: 502, error_name: 'throttle_violation', error_message: 'too fast' }), { status: 200 }),
));
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(CommandExecutionError);
});
it('wraps fetch network failures in CommandExecutionError (not raw TypeError)', async () => {
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new TypeError('fetch failed')));
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(CommandExecutionError);
});
it('wraps malformed JSON responses in CommandExecutionError (not raw SyntaxError)', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(
new Response('<!DOCTYPE html><html>maintenance</html>', { status: 200 }),
));
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(CommandExecutionError);
});
it('coerces and validates string-form numeric args (e.g. "50" not Number(50))', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
// String "50" should still be rejected because it's < 100
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': '50' }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
// String "abc" should also be rejected
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 'abc' }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('rejects answer/comment limits above Stack Exchange pagesize max before fetching', async () => {
const fetchMock = vi.fn();
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: '12345', 'answers-limit': '101', 'comments-limit': 5, 'max-length': 4000 }))
.rejects.toThrow(ArgumentError);
await expect(cmd.func({ id: '12345', 'answers-limit': 10, 'comments-limit': '101', 'max-length': 4000 }))
.rejects.toThrow(ArgumentError);
expect(fetchMock).not.toHaveBeenCalled();
});
it('builds POST + Q-COMMENT + ANSWER + A-COMMENT rows, accepted answer first', async () => {
const question = {
items: [{
question_id: 1,
title: 'Why?',
body: '<p>Question body</p>',
score: 10,
link: 'https://example.com/q/1',
owner: { display_name: 'asker' },
}],
};
const qComments = {
items: [
{ score: 2, owner: { display_name: 'qc1' }, body: '<p>q comment one</p>' },
{ score: 1, owner: { display_name: 'qc2' }, body: '<p>q comment two</p>' },
],
};
const answers = {
items: [
{ answer_id: 100, score: 5, is_accepted: false, owner: { display_name: 'low' }, body: '<p>low score answer</p>' },
{ answer_id: 200, score: 50, is_accepted: true, owner: { display_name: 'winner' }, body: '<p>accepted answer</p>' },
],
};
const answerComments = {
items: [
{ post_id: 200, score: 1, owner: { display_name: 'ac1' }, body: '<p>comment on accepted</p>' },
{ post_id: 100, score: 0, owner: { display_name: 'ac2' }, body: '<p>comment on low</p>' },
],
};
const fetchMock = vi.fn()
.mockResolvedValueOnce(new Response(JSON.stringify(question), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(qComments), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answers), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answerComments), { status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func({ id: '1', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 });
expect(rows.map((r) => [r.type, r.author, r.accepted])).toEqual([
['POST', 'asker', ''],
['Q-COMMENT', 'qc1', ''],
['Q-COMMENT', 'qc2', ''],
['ANSWER', 'winner', 'true'], // accepted comes FIRST
['A-COMMENT', 'ac1', ''],
['ANSWER', 'low', ''],
['A-COMMENT', 'ac2', ''],
]);
// Verify the answer-comments fetch batched both answer ids
const ansCommentsCall = fetchMock.mock.calls[3][0];
expect(ansCommentsCall).toContain('/answers/200;100/comments');
expect(fetchMock.mock.calls[1][0]).toContain('pagesize=5');
expect(fetchMock.mock.calls[2][0]).toContain('pagesize=10');
expect(fetchMock.mock.calls[3][0]).toContain('pagesize=10');
});
it('fetches accepted answer separately when it is missing from the votes page', async () => {
const question = {
items: [{
question_id: 1,
accepted_answer_id: 999,
title: 'Why?',
body: '<p>Question body</p>',
score: 10,
link: 'https://example.com/q/1',
owner: { display_name: 'asker' },
}],
};
const answers = {
items: [
{ answer_id: 100, score: 50, is_accepted: false, owner: { display_name: 'top-voted' }, body: '<p>top voted answer</p>' },
],
};
const acceptedAnswer = {
items: [
{ answer_id: 999, score: 1, is_accepted: true, owner: { display_name: 'accepted' }, body: '<p>accepted answer</p>' },
],
};
const fetchMock = vi.fn()
.mockResolvedValueOnce(new Response(JSON.stringify(question), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answers), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(acceptedAnswer), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func({ id: '1', 'answers-limit': 2, 'comments-limit': 5, 'max-length': 4000 });
expect(rows.filter((r) => r.type === 'ANSWER').map((r) => [r.author, r.accepted])).toEqual([
['accepted', 'true'],
['top-voted', ''],
]);
expect(fetchMock.mock.calls[3][0]).toContain('/answers/999?');
expect(fetchMock.mock.calls[4][0]).toContain('/answers/999;100/comments');
expect(fetchMock.mock.calls[4][0]).toContain('pagesize=10');
});
it('fails fast when batched answer comments would be partial', async () => {
const question = {
items: [{
question_id: 1,
title: 'Why?',
body: '<p>Question body</p>',
score: 10,
link: 'https://example.com/q/1',
owner: { display_name: 'asker' },
}],
};
const answers = {
items: [
{ answer_id: 100, score: 50, is_accepted: false, owner: { display_name: 'a1' }, body: '<p>one</p>' },
{ answer_id: 200, score: 40, is_accepted: false, owner: { display_name: 'a2' }, body: '<p>two</p>' },
],
};
const answerComments = {
has_more: true,
items: [
{ post_id: 100, score: 1, owner: { display_name: 'c1' }, body: '<p>comment on first</p>' },
],
};
const fetchMock = vi.fn()
.mockResolvedValueOnce(new Response(JSON.stringify(question), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answers), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answerComments), { status: 200 }));
vi.stubGlobal('fetch', fetchMock);
await expect(cmd.func({ id: '1', 'answers-limit': 2, 'comments-limit': 1, 'max-length': 4000 }))
.rejects.toThrow(CommandExecutionError);
});
it('decodes HTML entities in body and display_name (named, decimal, hex)', async () => {
const question = {
items: [{
question_id: 1,
title: 't',
body: '<p>price &lt; &amp; &hellip; &#246; &#x27;ok&#x27;</p>',
score: 0,
link: '',
owner: { display_name: 'Jonas K&#246;lker' },
}],
};
const fetchMock = vi.fn()
.mockResolvedValueOnce(new Response(JSON.stringify(question), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func({ id: '1', 'answers-limit': 10, 'comments-limit': 5, 'max-length': 4000 });
expect(rows[0].author).toBe('Jonas Kölker');
expect(rows[0].text).toContain('price < & … ö \'ok\'');
});
it('respects answers-limit when there are more answers than the cap', async () => {
const question = {
items: [{
question_id: 1, title: 't', body: '', score: 0, link: '',
owner: { display_name: 'a' },
}],
};
const answers = {
items: [
{ answer_id: 100, score: 5, is_accepted: false, owner: { display_name: 'a1' }, body: '' },
{ answer_id: 200, score: 4, is_accepted: false, owner: { display_name: 'a2' }, body: '' },
{ answer_id: 300, score: 3, is_accepted: false, owner: { display_name: 'a3' }, body: '' },
],
};
const fetchMock = vi.fn()
.mockResolvedValueOnce(new Response(JSON.stringify(question), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify(answers), { status: 200 }))
.mockResolvedValueOnce(new Response(JSON.stringify({ items: [] }), { status: 200 }));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func({ id: '1', 'answers-limit': 2, 'comments-limit': 5, 'max-length': 4000 });
const answerRows = rows.filter((r) => r.type === 'ANSWER');
expect(answerRows.map((r) => r.author)).toEqual(['a1', 'a2']);
});
});
+8 -2
View File
@@ -9,16 +9,22 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 10, help: 'Max number of results' },
],
columns: ['title', 'score', 'answers', 'url'],
columns: ['rank', 'id', 'title', 'score', 'answers', 'views', 'tags', 'author', 'creation_date', 'url'],
pipeline: [
{ fetch: {
url: 'https://api.stackexchange.com/2.3/questions/unanswered?order=desc&sort=votes&site=stackoverflow',
url: 'https://api.stackexchange.com/2.3/questions/unanswered?order=desc&sort=votes&site=stackoverflow&pagesize=${{ args.limit }}',
} },
{ select: 'items' },
{ map: {
rank: '${{ index + 1 }}',
id: '${{ item.question_id }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
answers: '${{ item.answer_count }}',
views: '${{ item.view_count }}',
tags: `\${{ item.tags | join(', ') }}`,
author: '${{ item.owner.display_name }}',
creation_date: '${{ item.creation_date }}',
url: '${{ item.link }}',
} },
{ limit: '${{ args.limit }}' },
+2 -2
View File
@@ -1,4 +1,4 @@
import { AuthRequiredError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, selectorError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
cli({
site: 'twitter',
@@ -49,7 +49,7 @@ cli({
return false;
}`);
if (!clicked) {
throw new SelectorError('Twitter followers link', 'Twitter may have changed the layout.');
throw selectorError('Twitter followers link', 'Twitter may have changed the layout.');
}
await page.waitForCapture(5);
// 4. Scroll to trigger pagination API calls
+10 -12
View File
@@ -1,6 +1,14 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, EmptyResultError } from '@jackwener/opencli/errors';
// ── CLI definition ────────────────────────────────────────────────────
//
// X (Twitter) removed the post-count caption from each trend cell on the
// /explore/tabs/trending page in 2024-2025. The DOM now only carries:
// divs[0] = rank + category (e.g. "1 · Trending in United States")
// divs[1] = topic
// divs[2..] = caret menu button (no post-count text)
// We previously surfaced a `tweets` column whose value was permanently
// "N/A" on every row — that's silent-wrong data, drop it.
cli({
site: 'twitter',
name: 'trending',
@@ -11,7 +19,7 @@ cli({
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of trends to show' },
],
columns: ['rank', 'topic', 'tweets', 'category'],
columns: ['rank', 'topic', 'category'],
func: async (page, kwargs) => {
const limit = kwargs.limit || 20;
// Navigate to trending page
@@ -23,9 +31,6 @@ cli({
})()`);
if (!ct0)
throw new AuthRequiredError('x.com', 'Not logged into x.com (no ct0 cookie)');
// Scrape trends from DOM (consistent with what the user sees on the page)
// DOM children: [0] rank + category, [1] topic, optional post count,
// and a caret menu button identified by [data-testid="caret"].
await page.wait(2);
const trends = await page.evaluate(`(() => {
const items = [];
@@ -41,14 +46,7 @@ cli({
if (!topic) return;
const catText = divs[0].textContent.trim();
const category = catText.replace(/^\\d+\\s*/, '').replace(/^\\xB7\\s*/, '').trim();
// Find post count: skip rank, topic, and the caret menu button
let tweets = 'N/A';
for (let j = 2; j < divs.length; j++) {
if (divs[j].matches('[data-testid="caret"]') || divs[j].querySelector('[data-testid="caret"]')) continue;
const t = divs[j].textContent.trim();
if (t && /\\d/.test(t)) { tweets = t; break; }
}
items.push({ rank: items.length + 1, topic, tweets, category });
items.push({ rank: items.length + 1, topic, category });
});
return items;
})()`);
+15
View File
@@ -0,0 +1,15 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './trending.js';
describe('twitter trending', () => {
it('registers the trending command with rank/topic/category columns only', () => {
const cmd = getRegistry().get('twitter/trending');
expect(cmd).toBeDefined();
// The `tweets` column was permanently "N/A" because X removed the post-count
// caption from the trend cell; we drop it rather than keep returning a
// silent-wrong sentinel for every row. Guard against re-introduction.
expect(cmd.columns).toEqual(['rank', 'topic', 'category']);
expect(cmd.columns).not.toContain('tweets');
});
});
+169
View File
@@ -0,0 +1,169 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getSelfUid } from './utils.js';
const DEFAULT_LIMIT = 20;
const MAX_LIMIT = 50;
function parsePositiveInt(value, name, defaultValue) {
const raw = value ?? defaultValue;
const number = Number(raw);
if (!Number.isInteger(number) || number <= 0) {
throw new ArgumentError(`weibo favorites ${name} must be a positive integer`);
}
if (number > MAX_LIMIT) {
throw new ArgumentError(`weibo favorites ${name} must be <= ${MAX_LIMIT}`);
}
return number;
}
function parseFavoriteCard(card, favUrl) {
const raw = String(card?.text ?? '');
const lines = raw.split('\n');
let author = '';
let time = '';
let source = '';
let content = '';
let likes = '0';
let comments = '0';
let reposts = '0';
for (const line of lines) {
const t = line.trim();
if (!t || t === '添加') continue;
if (!time && /\d+小时前|\d+分钟前|\d+秒前|昨天|前天|\d{1,2}:\d{2}/.test(t)) {
time = t;
continue;
}
if (t.startsWith('来自')) {
source = t;
continue;
}
if (content) {
const n = Number.parseInt(t, 10);
if (!Number.isNaN(n) && n > 0 && n < 1_000_000 && t === String(n)) {
if (likes === '0') likes = t;
else if (comments === '0') comments = t;
else if (reposts === '0') reposts = t;
continue;
}
}
if (!author && t.length < 40) {
author = t;
continue;
}
if (!content && author) {
content = t;
continue;
}
if (content) content += ` ${t}`;
}
if (!content || !author) return null;
return {
author,
text: content.substring(0, 300),
time,
source,
likes,
comments,
reposts,
url: card?.url || favUrl,
};
}
function dedupeFavorites(items, favUrl) {
const seen = new Set();
const result = [];
for (const item of items) {
const key = item.url && item.url !== favUrl
? item.url
: `${item.author}\n${item.text}\n${item.time}`;
if (seen.has(key)) continue;
seen.add(key);
result.push(item);
}
return result;
}
cli({
site: 'weibo',
name: 'favorites',
description: '我的微博收藏列表',
domain: 'weibo.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'limit', type: 'int', default: 20, help: '数量(最多50' },
],
columns: ['author', 'text', 'time', 'source', 'likes', 'comments', 'reposts', 'url'],
func: async (page, kwargs) => {
const limit = parsePositiveInt(kwargs.limit, 'limit', DEFAULT_LIMIT);
await page.goto('https://weibo.com');
await page.wait(2);
const uid = await getSelfUid(page);
const favUrl = 'https://www.weibo.com/u/page/fav/' + uid;
await page.goto(favUrl);
await page.wait(4);
for (let i = 0; i < 3; i++) {
await page.evaluate('() => window.scrollBy(0, 800)');
await page.wait(1);
}
const rawData = await page.evaluate(`
(() => {
const scrollers = document.querySelectorAll('.wbpro-scroller-item, .vue-recycle-scroller__item-view');
const out = [];
for (const s of scrollers) {
// Use textContent to preserve newlines, then split by \n
const bodyEl = s.querySelector('[class*="_body_"]') || s.querySelector('.wbpro-item-body') || s;
// innerText preserves newlines between block elements (unlike textContent)
const rawText = bodyEl.innerText || s.innerText || '';
let postUrl = '';
const anchors = s.querySelectorAll('a[href]');
for (const a of anchors) {
const m = String(a.href).match(/weibo\\.com\\/(\\d+)\\/([a-zA-Z0-9]+)/);
if (m) { postUrl = 'https://weibo.com/' + m[1] + '/' + m[2]; break; }
}
if (rawText.length > 20) out.push({ text: rawText, url: postUrl });
if (out.length >= ${limit}) break;
}
return out;
})()
`);
if (!Array.isArray(rawData) || rawData.length === 0) {
throw new EmptyResultError('weibo favorites', 'No favorites were visible on the favorites page');
}
const items = rawData
.map(card => parseFavoriteCard(card, favUrl))
.filter(Boolean);
const uniqueItems = dedupeFavorites(items, favUrl);
if (uniqueItems.length === 0) {
throw new CommandExecutionError('Failed to parse visible Weibo favorites');
}
return uniqueItems.slice(0, limit);
},
});
export const __test__ = {
parseFavoriteCard,
parsePositiveInt,
dedupeFavorites,
};
+114
View File
@@ -0,0 +1,114 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './favorites.js';
function makePage(evaluateResults = []) {
const queue = [...evaluateResults];
const evaluate = vi.fn(async (script) => {
if (String(script).includes('window.scrollBy')) return undefined;
return queue.length ? queue.shift() : [];
});
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate,
};
}
describe('weibo favorites command', () => {
const getCommand = () => getRegistry().get('weibo/favorites');
it('registers as a JS adapter and parses visible favorites', async () => {
const command = getCommand();
expect(command?.func).toBeTypeOf('function');
const page = makePage([
'123456',
[
{
text: [
'作者A',
'昨天 12:00',
'来自 iPhone',
'这是一条收藏微博',
'12',
'3',
'2',
].join('\n'),
url: 'https://weibo.com/123/AbCd1',
},
],
]);
const result = await command.func(page, { limit: 10 });
expect(result).toEqual([
{
author: '作者A',
text: '这是一条收藏微博',
time: '昨天 12:00',
source: '来自 iPhone',
likes: '12',
comments: '3',
reposts: '2',
url: 'https://weibo.com/123/AbCd1',
},
]);
expect(page.goto).toHaveBeenCalledWith('https://weibo.com');
expect(page.goto).toHaveBeenCalledWith('https://www.weibo.com/u/page/fav/123456');
});
it('throws AuthRequiredError when uid cannot be resolved', async () => {
const command = getCommand();
const page = makePage([null, null]);
await expect(command.func(page, { limit: 10 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('validates limit before navigation', async () => {
const command = getCommand();
const page = makePage();
await expect(command.func(page, { limit: 0 })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { limit: 51 })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('throws EmptyResultError when no favorite cards are visible', async () => {
const command = getCommand();
const page = makePage(['123456', []]);
await expect(command.func(page, { limit: 10 })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when visible cards cannot be parsed', async () => {
const command = getCommand();
const page = makePage(['123456', [{ text: '添加\n昨天', url: '' }]]);
await expect(command.func(page, { limit: 10 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('deduplicates repeated cards and applies the requested limit', async () => {
const command = getCommand();
const rawCard = {
text: '作者A\n内容A',
url: 'https://weibo.com/123/AbCd1',
};
const page = makePage([
'123456',
[
rawCard,
rawCard,
{ text: '作者B\n内容B', url: 'https://weibo.com/123/AbCd2' },
],
]);
const result = await command.func(page, { limit: 1 });
expect(result).toHaveLength(1);
expect(result[0].author).toBe('作者A');
});
});
+282
View File
@@ -0,0 +1,282 @@
/**
* Weibo publish — post a new Weibo update via browser UI automation.
*
* Flow:
* 1. Navigate to weibo.com and wait for the feed
* 2. Check login state (getSelfUid)
* 3. Click "发微博" button to open the inline compose editor
* 4. Wait for textarea editor to appear
* 5. Fill text content via CDP type
* 6. Optionally upload images via CDP setFileInput
* 7. Click the publish button
* 8. Poll for success/failure feedback
*
* Usage:
* opencli weibo publish "Hello from OpenCLI! #opencli" # publishes immediately
* opencli weibo publish "Check this out" --images /path/a.jpg,/path/b.jpg
*/
import * as fs from 'node:fs';
import * as path from 'node:path';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getSelfUid } from './utils.js';
const MAX_IMAGES = 9;
const UPLOAD_POLL_MS = 1500;
const UPLOAD_TIMEOUT_MS = 30_000;
const COMPOSE_POLL_MS = 300;
const COMPOSE_TIMEOUT_MS = 10_000;
const SUBMIT_POLL_MS = 500;
const SUBMIT_TIMEOUT_MS = 20_000;
const SUPPORTED_EXTENSIONS = new Set(['.jpg', '.jpeg', '.png', '.gif', '.webp']);
// Weibo PC UI selectors
const TEXTAREA_SELECTOR = 'textarea._input_13iqr_8';
const FILE_INPUT_SELECTOR = 'input[type="file"][class*="_file_"]';
function validateText(text) {
const t = String(text ?? '').trim();
if (!t) throw new ArgumentError('weibo publish text cannot be empty');
if (t.length > 2000) throw new ArgumentError('weibo publish text exceeds 2000 characters');
return t;
}
function validateImagePaths(raw) {
if (!raw) return [];
const paths = raw.split(',').map(s => s.trim()).filter(Boolean);
if (paths.length > MAX_IMAGES) {
throw new ArgumentError(`Too many images: ${paths.length} (max ${MAX_IMAGES})`);
}
return paths.map(p => {
const absPath = path.resolve(p);
const ext = path.extname(absPath).toLowerCase();
if (!SUPPORTED_EXTENSIONS.has(ext)) {
throw new ArgumentError(`Unsupported image format "${ext}". Supported: jpg, png, gif, webp`);
}
const stat = fs.statSync(absPath, { throwIfNoEntry: false });
if (!stat || !stat.isFile()) {
throw new ArgumentError(`Not a valid file: ${absPath}`);
}
return absPath;
});
}
cli({
site: 'weibo',
name: 'publish',
description: 'Publish a new Weibo post immediately',
domain: 'weibo.com',
strategy: Strategy.UI,
browser: true,
args: [
{
name: 'text',
type: 'string',
required: true,
positional: true,
help: 'Weibo text content (max 2000 chars)',
},
{
name: 'images',
type: 'string',
required: false,
help: `Image paths, comma-separated, max ${MAX_IMAGES} (jpg/png/gif/webp)`,
},
],
columns: ['status', 'message', 'text'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for weibo publish');
const text = validateText(kwargs.text);
const absPaths = validateImagePaths(kwargs.images);
// Step 1: Navigate to weibo.com and wait for feed to load
await page.goto('https://weibo.com', { waitUntil: 'load', settleMs: 2000 });
await page.wait({ time: 2 });
// Step 2: Check login
try {
await getSelfUid(page);
} catch (err) {
if (err instanceof AuthRequiredError) throw err;
throw new CommandExecutionError('Not logged into Weibo. Please login at weibo.com in your Chrome browser.');
}
// Step 3: Click "发微博" button to open inline compose editor
const clickResult = await page.evaluate(`
() => {
const visible = el => !!el && el.offsetParent !== null && !el.disabled;
const buttons = document.querySelectorAll('button[title="发微博"], button[title="写微博"]');
for (const btn of buttons) {
if (visible(btn)) {
btn.click();
return { ok: true };
}
}
return { ok: false, message: 'Could not find 发微博 button' };
}
`);
if (!clickResult?.ok) {
throw new CommandExecutionError(clickResult?.message ?? 'Could not open compose editor.');
}
// Step 4: Wait for the textarea editor to appear (visible, not just in DOM)
let editorFound = false;
for (let i = 0; i < Math.ceil(COMPOSE_TIMEOUT_MS / COMPOSE_POLL_MS); i++) {
const result = await page.evaluate(`
() => {
const ta = document.querySelector('textarea._input_13iqr_8');
if (!ta) return { found: false };
const visible = ta.offsetParent !== null;
return { found: true, visible, rectTop: visible ? ta.getBoundingClientRect().top : -1 };
}
`);
if (result?.found && result.visible && result.rectTop >= 0) {
editorFound = true;
break;
}
await page.wait({ time: COMPOSE_POLL_MS / 1000 });
}
if (!editorFound) {
throw new CommandExecutionError('Weibo compose editor did not appear');
}
// Step 5: Upload images first (before text to avoid editor reset)
if (absPaths.length > 0) {
if (!page.setFileInput) {
throw new CommandExecutionError('Browser extension does not support file upload. Please update the extension.');
}
// Find the file input
const fileInputFound = await page.evaluate(`
() => {
const input = document.querySelector('input[type="file"][class*="_file_"]');
return !!input;
}
`);
if (!fileInputFound) {
throw new CommandExecutionError('Could not find image file input on Weibo compose page. UI may have changed.');
}
await page.setFileInput(absPaths, FILE_INPUT_SELECTOR);
// Wait for upload to complete
let uploadResult = null;
for (let i = 0; i < Math.ceil(UPLOAD_TIMEOUT_MS / UPLOAD_POLL_MS); i++) {
await page.wait({ time: UPLOAD_POLL_MS / 1000 });
uploadResult = await page.evaluateWithArgs(`
(() => {
const expectedCount = expected;
const uploading = document.querySelector('[class*="upload"], [class*="progress"]');
if (uploading && uploading.offsetParent !== null) return null;
const pics = document.querySelectorAll('img[class*="pic"], [class*="imgItem"], [class*="picture"] img');
if (pics.length >= expectedCount) return { ok: true, count: pics.length };
return null;
})()
`, { expected: absPaths.length });
if (uploadResult !== null) break;
}
if (!uploadResult?.ok) {
throw new CommandExecutionError(uploadResult?.message ?? 'Image upload did not complete before timeout');
}
}
// Step 6: Insert text using native DOM setter (preserves Weibo internal state)
// IMPORTANT: Using nativeSetter preserves the textarea's reactive/internal state.
// Direct ta.value= assignment bypasses Weibo's Vue reactivity and causes "undefined" content.
const insertResult = await page.evaluateWithArgs(`
(() => {
const ta = document.querySelector('textarea._input_13iqr_8');
if (!ta || ta.offsetParent === null) return { ok: false, message: 'textarea not visible' };
ta.focus();
const nativeSetter = Object.getOwnPropertyDescriptor(HTMLTextAreaElement.prototype, 'value')?.set;
if (nativeSetter) {
nativeSetter.call(ta, textContent);
} else {
ta.value = textContent;
}
ta.dispatchEvent(new Event('input', { bubbles: true }));
ta.dispatchEvent(new Event('change', { bubbles: true }));
return { ok: true, valueLength: ta.value.length };
})()
`, { textContent: text });
if (!insertResult?.ok) {
throw new CommandExecutionError(insertResult?.message ?? 'Could not insert text.');
}
// Step 7: Click the send button inside the compose editor
// Try 发送 first (compose editor's submit), then 发布 (fallback)
await page.wait({ time: 0.5 });
const publishResult = await page.evaluate(`
() => {
const visible = el => !!el && el.offsetParent !== null && !el.disabled;
const labels = ['发送', '发布'];
for (const label of labels) {
const allBtns = document.querySelectorAll('button, [role="button"]');
for (const btn of allBtns) {
const t = (btn.innerText || btn.textContent || '').trim();
if (t === label && visible(btn)) {
btn.click();
return { ok: true, label };
}
}
}
return { ok: false, message: 'Could not find send button' };
}
`);
if (!publishResult?.ok) {
throw new CommandExecutionError(publishResult?.message ?? 'Could not click publish.');
}
// Step 8: Wait for success/failure result
let finalResult = null;
for (let i = 0; i < Math.ceil(SUBMIT_TIMEOUT_MS / SUBMIT_POLL_MS); i++) {
await page.wait({ time: SUBMIT_POLL_MS / 1000 });
finalResult = await page.evaluateWithArgs(`
(() => {
const successMarkers = ['发布成功', '已发布', '发送成功'];
const errorMarkers = ['发布失败', '发送失败', '内容违规', '请稍后再试', '频繁'];
for (const el of document.querySelectorAll('*')) {
if (el.children.length > 3) continue;
const txt = (el.innerText || '').trim();
if (!txt || txt.length > 100) continue;
for (const m of successMarkers) {
if (txt.includes(m) && (txt.includes('成功') || txt.includes('微博'))) {
return { ok: true, message: txt };
}
}
for (const m of errorMarkers) {
if (txt.includes(m)) {
return { ok: false, message: txt };
}
}
}
return null;
})()
`, { maxIterations: Math.ceil(SUBMIT_TIMEOUT_MS / SUBMIT_POLL_MS), currentIndex: i });
if (finalResult !== null) break;
}
if (!finalResult) {
throw new CommandExecutionError('Publish button clicked but result was unclear. Check Weibo manually.');
}
if (!finalResult.ok) {
throw new CommandExecutionError(finalResult.message || 'Weibo publish failed');
}
return [{
status: 'success',
message: finalResult.message || 'Published successfully',
text,
}];
},
});
export const __test__ = {
validateText,
validateImagePaths,
};
+183
View File
@@ -0,0 +1,183 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
vi.mock('node:fs', async (importOriginal) => {
const actual = await importOriginal();
return {
...actual,
statSync: vi.fn((p) => {
if (String(p).includes('missing')) return undefined;
return { isFile: () => !String(p).includes('directory') };
}),
};
});
vi.mock('node:path', async (importOriginal) => {
const actual = await importOriginal();
return {
...actual,
resolve: vi.fn((p) => `/abs/${p}`),
extname: vi.fn((p) => {
const m = String(p).match(/\.[^.]+$/);
return m ? m[0] : '';
}),
};
});
import './publish.js';
function makePage({ evaluateResults = [], evaluateWithArgsResults = [], overrides = {} } = {}) {
const evaluate = vi.fn();
for (const result of evaluateResults) {
evaluate.mockResolvedValueOnce(result);
}
evaluate.mockResolvedValue({ ok: true });
const evaluateWithArgs = vi.fn();
for (const result of evaluateWithArgsResults) {
evaluateWithArgs.mockResolvedValueOnce(result);
}
evaluateWithArgs.mockResolvedValue(null);
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate,
evaluateWithArgs,
setFileInput: vi.fn().mockResolvedValue(undefined),
...overrides,
};
}
describe('weibo publish command', () => {
const getCommand = () => getRegistry().get('weibo/publish');
it('publishes a text-only post when the UI reports success', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: [
'123456',
{ ok: true },
{ found: true, visible: true, rectTop: 100 },
{ ok: true, label: '发送' },
],
evaluateWithArgsResults: [
{ ok: true, valueLength: 5 },
{ ok: true, message: '发送成功' },
],
});
const result = await command.func(page, { text: 'hello' });
expect(result).toEqual([{ status: 'success', message: '发送成功', text: 'hello' }]);
expect(page.goto).toHaveBeenCalledWith('https://weibo.com', { waitUntil: 'load', settleMs: 2000 });
});
it('uploads up to nine images before publishing', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: [
'123456',
{ ok: true },
{ found: true, visible: true, rectTop: 100 },
true,
{ ok: true, label: '发送' },
],
evaluateWithArgsResults: [
{ ok: true, count: 2 },
{ ok: true, valueLength: 11 },
{ ok: true, message: '发送成功' },
],
});
await command.func(page, { text: 'with images', images: 'a.png,b.webp' });
expect(page.setFileInput).toHaveBeenCalledWith(
['/abs/a.png', '/abs/b.webp'],
'input[type="file"][class*="_file_"]',
);
});
it('maps auth failures to AuthRequiredError', async () => {
const command = getCommand();
const page = makePage({ evaluateResults: [null, null] });
await expect(command.func(page, { text: 'hello' })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('validates text and image arguments before navigation', async () => {
const command = getCommand();
const page = makePage();
await expect(command.func(page, { text: ' ' })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { text: 'hi', images: 'a.bmp' })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { text: 'hi', images: 'missing.png' })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { text: 'hi', images: '1.png,2.png,3.png,4.png,5.png,6.png,7.png,8.png,9.png,10.png' })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('throws CommandExecutionError when compose cannot be opened', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: ['123456', { ok: false, message: 'Could not find 发微博 button' }],
});
await expect(command.func(page, { text: 'hello' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError when upload readiness is not proven', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: [
'123456',
{ ok: true },
{ found: true, visible: true, rectTop: 100 },
true,
],
evaluateWithArgsResults: [null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null],
});
await expect(command.func(page, { text: 'hello', images: 'a.png' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError when publish result is unclear or failed', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: [
'123456',
{ ok: true },
{ found: true, visible: true, rectTop: 100 },
{ ok: true, label: '发送' },
],
evaluateWithArgsResults: [
{ ok: true, valueLength: 5 },
{ ok: false, message: '内容违规' },
],
});
await expect(command.func(page, { text: 'hello' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('does not treat editor close as positive publish proof', async () => {
const command = getCommand();
const page = makePage({
evaluateResults: [
'123456',
{ ok: true },
{ found: true, visible: true, rectTop: 100 },
{ ok: true, label: '发送' },
],
evaluateWithArgsResults: [
{ ok: true, valueLength: 5 },
null,
],
});
await expect(command.func(page, { text: 'hello' })).rejects.toBeInstanceOf(CommandExecutionError);
const submitScript = page.evaluateWithArgs.mock.calls.at(-1)[0];
expect(submitScript).not.toContain('Editor closed after publish');
expect(submitScript).toContain('发布成功');
});
});
+3 -3
View File
@@ -1,4 +1,4 @@
import { AuthRequiredError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, selectorError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { normalizeNumericId } from './utils.js';
function buildChatUrl(itemId, peerUserId) {
@@ -105,7 +105,7 @@ cli({
throw new AuthRequiredError('www.goofish.com', 'Xianyu chat requires a logged-in browser session');
}
if (!state?.can_input) {
throw new SelectorError('闲鱼聊天输入框', '未找到可用的聊天输入框,请确认该会话页已正确加载');
throw selectorError('闲鱼聊天输入框', '未找到可用的聊天输入框,请确认该会话页已正确加载');
}
if (!text) {
return [{
@@ -123,7 +123,7 @@ cli({
}
const sent = await page.evaluate(buildSendMessageEvaluate(text));
if (!sent?.ok) {
throw new SelectorError('闲鱼发送按钮', `消息发送失败:${sent?.reason || 'unknown-reason'}`);
throw selectorError('闲鱼发送按钮', `消息发送失败:${sent?.reason || 'unknown-reason'}`);
}
await page.wait(1);
return [{
+2 -2
View File
@@ -1,4 +1,4 @@
import { AuthRequiredError, EmptyResultError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, EmptyResultError, selectorError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { normalizeNumericId } from './utils.js';
function buildItemUrl(itemId) {
@@ -127,7 +127,7 @@ cli({
throw new EmptyResultError('xianyu item', 'Xianyu item detail is blocked by verification or risk control');
}
if (result?.error === 'mtop-not-ready') {
throw new SelectorError('window.lib.mtop', '闲鱼页面未完成初始化,无法调用商品详情接口');
throw selectorError('window.lib.mtop', '闲鱼页面未完成初始化,无法调用商品详情接口');
}
if (!result || typeof result !== 'object') {
throw new EmptyResultError('xianyu item', '闲鱼商品详情接口未返回有效数据');
+3 -3
View File
@@ -1,5 +1,5 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError, EmptyResultError, SelectorError } from '@jackwener/opencli/errors';
import { AuthRequiredError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { __test__ } from './item.js';
import './item.js';
@@ -49,8 +49,8 @@ describe('xianyu item command', () => {
const page = createPageMock({ error: 'blocked' });
await expect(command.func(page, { item_id: '1040754408976' })).rejects.toBeInstanceOf(EmptyResultError);
});
it('keeps SelectorError for true mtop initialization failures', async () => {
it('keeps SELECTOR code for true mtop initialization failures', async () => {
const page = createPageMock({ error: 'mtop-not-ready' });
await expect(command.func(page, { item_id: '1040754408976' })).rejects.toBeInstanceOf(SelectorError);
await expect(command.func(page, { item_id: '1040754408976' })).rejects.toMatchObject({ code: 'SELECTOR' });
});
});
+71 -73
View File
@@ -1,28 +1,28 @@
# Self-Repair Protocol — Design Document
**Authors**: @opus0, @codex-mini0
**Date**: 2026-04-07
**Status**: Approved
**Authors**: @opus0, @codex-mini0
**Date**: 2026-04-07
**Status**: Approved, updated for trace-based repair
**Supersedes**: `designs/autofix-incident-repair.md` (PR #863, deferred to Phase 2)
---
## Problem Statement
When an AI agent uses `opencli <site> <command>` and the command fails (site changed DOM, API, or response schema), the agent should **automatically repair the adapter and retry** without human intervention or pre-written spec files.
When an AI agent uses `opencli <site> <command>` and the command fails because the site changed DOM, API, or response schema, the agent should automatically repair the adapter and retry without human intervention or pre-written spec files.
### Why the simpler approach
From first principles, the agent needs five things:
The previous design (PR #863) required pre-authoring `command-specs.json` with verify checks, safety profiles, and failure taxonomy before any command could be repaired. This created a chicken-and-egg problem: you can only repair commands you've already written specs for.
1. The failing command it just ran.
2. The structured error envelope from stderr.
3. The adapter source path.
4. Browser runtime evidence: actions, page state, network, console, screenshot.
5. A verify oracle: re-run the same command.
From first principles, the agent already has everything it needs:
1. **The failing command** — it just ran it
2. **The error output** — stdout/stderr
3. **The adapter source** — resolved via `RepairContext.adapter.sourcePath`
4. **Diagnostic context** — DOM snapshot, network requests (via `OPENCLI_DIAGNOSTIC=1`)
5. **A verify oracle** — re-run the same command
No spec file needed. The command itself is the spec.
The command itself is the spec. The trace artifact is the evidence channel.
---
@@ -30,31 +30,34 @@ No spec file needed. The command itself is the spec.
### Core Protocol
```
```text
Agent runs: opencli <site> <command> [args...]
Command succeeds continue task
Command fails
1. Re-run with OPENCLI_DIAGNOSTIC=1 to collect RepairContext
2. Read adapter source from RepairContext.adapter.sourcePath
3. Analyze: error code + DOM snapshot + network requests → root cause
4. Edit the adapter file at RepairContext.adapter.sourcePath
5. Retry the original command
6. If still failing → repeat (max 3 rounds)
7. If 3 rounds exhausted → report failure, do not loop further
-> Command succeeds -> continue task
-> Command fails ->
1. Re-run with --trace retain-on-failure to collect a trace artifact
2. Read trace.summaryPath from the error envelope
3. Read adapterSourcePath from summary.md front matter
4. Analyze: error code + failed network + console + state/action timeline -> root cause
5. Edit the adapter file at adapterSourcePath
6. Retry the original command
7. If still failing -> repeat (max 3 rounds)
8. If 3 rounds exhausted -> report failure, do not loop further
```
### Scope Constraint
**Only modify the adapter file identified by `RepairContext.adapter.sourcePath`.**
Only modify the adapter file identified by `adapterSourcePath` in trace `summary.md` front matter.
The diagnostic resolves the actual editable source path at runtime — it may be:
- `clis/<site>/*.js` — repo-local adapters (dev/source checkout)
- `~/.opencli/clis/<site>/*.js` — user-local adapters (npm install scenario)
That path may be:
The agent must use the path from the diagnostic, not guess a repo-relative path. This is critical for npm-installed users where `clis/` is not in the repo.
- `clis/<site>/*.js` — repo-local adapters in a source checkout
- `~/.opencli/clis/<site>/*.js` — user-local adapters in npm install scenarios
**Never modify:**
- `src/**` — core runtime (npm package, requires version release)
The agent must use the trace summary path, not guess a repo-relative path. This matters for npm-installed users where `clis/` may not be in the working directory.
Never modify:
- `src/**` — core runtime
- `extension/**` — browser extension
- `autoresearch/**` — research infrastructure
- `tests/**` — test files
@@ -62,8 +65,6 @@ The agent must use the path from the diagnostic, not guess a repo-relative path.
### When NOT to Self-Repair
The agent should recognize non-repairable failures and stop:
| Signal | Meaning | Action |
|--------|---------|--------|
| Auth/login error | Not logged into site in Chrome | Tell user to log in, don't modify code |
@@ -74,63 +75,59 @@ The agent should recognize non-repairable failures and stop:
### Retry Budget
- **Max 3 repair rounds per command failure**
- Each round: diagnose → edit adapter retry command
- If the error is identical after a repair attempt, the fix didn't work — try a different approach
- After 3 rounds, stop and report what was tried
- Max 3 repair rounds per command failure.
- Each round: trace -> edit adapter -> retry command.
- If the error is identical after a repair attempt, the fix didn't work. Try a different approach.
- After 3 rounds, stop and report what was tried.
---
## Implementation
### What Already Exists
| Component | Status | Location |
|-----------|--------|----------|
| Diagnostic output (RepairContext) | ✅ Done | `src/diagnostic.ts` |
| Diagnostic wiring in execution | Done | `src/execution.ts` |
| Error taxonomy (CliError codes) | Done | `src/errors.ts` |
| Adapter source resolution | ✅ Done | `src/diagnostic.ts:resolveAdapterSourcePath` |
| Trace artifact output | Done | `src/observation/` |
| Error envelope trace metadata | Done | `src/errors.ts`, `src/execution.ts` |
| Adapter source resolution | Done | `src/adapter-source.ts` |
| AutoFix skill protocol | Done | `skills/opencli-autofix/SKILL.md` |
### What's New (This Design)
### Delivery Mechanism
| Component | Description |
|-----------|-------------|
| `skills/opencli-autofix/SKILL.md` (renamed from `opencli-repair`) | AutoFix skill with safety boundaries, sourcePath-based scope, 3-round limit. The primary delivery mechanism for the self-repair protocol. |
| `skills/opencli-usage/SKILL.md` (updated) | Self-Repair section for discoverability |
The `opencli-autofix` skill is the portable self-repair protocol. Any AI agent can load this skill to get the workflow.
### Delivery mechanism
No separate diagnostic env var is required. The runtime has two control axes:
The `opencli-autofix` skill is the portable self-repair protocol. Any AI agent — regardless of framework, provider, or working directory — can load this skill to get the full autofix workflow. It is not tied to any specific agent framework or repo location.
- **No new runtime code** — the diagnostic infrastructure already exists
- **No CLAUDE.md dependency** — the skill is the protocol, not a repo-local file
```text
-v / OPENCLI_VERBOSE human-readable logs
--trace off|on|retain-on-failure machine-readable browser evidence artifact
```
---
## The AutoFix Protocol (in the skill)
## The AutoFix Protocol
The `opencli-autofix` skill instructs agents:
1. When `opencli <site> <command>` fails, **don't just report the error**
2. Re-run with `OPENCLI_DIAGNOSTIC=1` to get structured context
3. Parse the RepairContext (error code, adapter source, DOM snapshot)
4. Read and fix the adapter at `RepairContext.adapter.sourcePath`
5. Retry the original command
6. If the retry passes, ask whether to file an upstream GitHub issue for `jackwener/OpenCLI`
7. If approved and `gh` is available, file the issue using a structured summary
8. Max 3 repair rounds, then stop
1. When `opencli <site> <command>` fails, don't just report the error.
2. Re-run with `--trace retain-on-failure`.
3. Read the error envelope `trace.summaryPath`.
4. Parse `summary.md` front matter for `adapterSourcePath`.
5. Read and fix the adapter at that exact path.
6. Retry the original command.
7. If the retry passes, ask whether to file an upstream GitHub issue for `jackwener/OpenCLI`.
8. If approved and `gh` is available, file the issue using a structured summary.
9. Max 3 repair rounds, then stop.
---
## Relationship to PR #863
PR #863 (spec/runner/incident framework) is **not needed for Phase 1**. It becomes useful later as a "hardening layer":
PR #863 (spec/runner/incident framework) is not needed for Phase 1. It becomes useful later as a hardening layer:
- **Phase 1 (now)**: Self-Repair via `opencli-autofix` skill — agent repairs on the fly
- **Phase 2 (later)**: High-frequency failures get hardened into `command-specs.json` for offline regression testing and CI
- Phase 1: self-repair via `opencli-autofix` skill and trace artifacts.
- Phase 2: high-frequency failures get hardened into command specs for offline regression testing and CI.
The spec/runner framework is the "asset layer" — it turns ad-hoc repairs into reusable, verifiable test cases. But it's not the entry point.
The spec/runner framework is the asset layer. It turns ad-hoc repairs into reusable tests, but it is not the entry point.
---
@@ -143,11 +140,12 @@ No new commands. No new scripts. The agent loads the `opencli-autofix` skill and
opencli weibo hot --limit 5 -f json
# If it fails, the agent automatically:
# 1. Runs OPENCLI_DIAGNOSTIC=1 opencli weibo hot --limit 5 -f json 2>diag.json
# 2. Reads the diagnostic context
# 3. Fixes the adapter at RepairContext.adapter.sourcePath
# 4. Retries: opencli weibo hot --limit 5 -f json
# 5. If retry passes, asks whether to file an upstream issue
# 6. If approved, runs `gh issue create --repo jackwener/OpenCLI ...`
# 7. Continues with the task
# 1. Runs opencli weibo hot --limit 5 -f json --trace retain-on-failure 2>trace-error.yaml
# 2. Reads trace.summaryPath from trace-error.yaml
# 3. Reads adapterSourcePath from summary.md
# 4. Fixes the adapter at adapterSourcePath
# 5. Retries: opencli weibo hot --limit 5 -f json
# 6. If retry passes, asks whether to file an upstream issue
# 7. If approved, runs `gh issue create --repo jackwener/OpenCLI ...`
# 8. Continues with the task
```
+3
View File
@@ -33,6 +33,7 @@ export default defineConfig({
{ text: 'Browser Bridge', link: '/guide/browser-bridge' },
{ text: 'Troubleshooting', link: '/guide/troubleshooting' },
{ text: 'Add an Electron App CLI', link: '/guide/electron-app-cli' },
{ text: 'Extending OpenCLI', link: '/guide/extending-opencli' },
{ text: 'Plugins', link: '/guide/plugins' },
],
},
@@ -79,6 +80,7 @@ export default defineConfig({
{ text: '1688', link: '/adapters/browser/1688' },
{ text: 'Gitee', link: '/adapters/browser/gitee' },
{ text: 'Gemini', link: '/adapters/browser/gemini' },
{ text: 'Claude', link: '/adapters/browser/claude' },
{ text: 'Yuanbao', link: '/adapters/browser/yuanbao' },
{ text: 'NotebookLM', link: '/adapters/browser/notebooklm' },
{ text: 'WeRead', link: '/adapters/browser/weread' },
@@ -193,6 +195,7 @@ export default defineConfig({
{ text: '安装', link: '/zh/guide/installation' },
{ text: 'Browser Bridge', link: '/zh/guide/browser-bridge' },
{ text: '给新 Electron 应用生成 CLI', link: '/zh/guide/electron-app-cli' },
{ text: '扩展 OpenCLI', link: '/zh/guide/extending-opencli' },
{ text: '插件', link: '/zh/guide/plugins' },
],
},
+23 -4
View File
@@ -6,8 +6,9 @@
| Command | Description |
|---------|-------------|
| `opencli arxiv search` | Search arXiv papers |
| `opencli arxiv paper` | Get arXiv paper details by ID |
| `opencli arxiv search <query>` | Search arXiv papers |
| `opencli arxiv paper <id>` | Get arXiv paper details by ID |
| `opencli arxiv recent <category>` | List recent submissions in a category |
## Usage Examples
@@ -15,13 +16,31 @@
# Search for papers
opencli arxiv search "transformer attention" --limit 10
# Get paper details by arXiv ID
opencli arxiv paper 2301.00001
# Get full paper details (full abstract, all authors, primary/all categories, pdf url)
opencli arxiv paper 1706.03762
# Newest papers in a category, sorted by submitted date desc
opencli arxiv recent cs.CL --limit 10
opencli arxiv recent math.PR --limit 5
# JSON output
opencli arxiv search "LLM" -f json
```
## Output Columns
| Command | Columns |
|---------|---------|
| `paper` | `id, title, authors, published, updated, primary_category, categories, abstract, comment, pdf, url` |
| `search` | `id, title, authors, published, primary_category, url` |
| `recent` | `id, title, authors, published, primary_category, url` |
`paper` returns the full abstract and full author list. `search`/`recent` are list-style outputs that omit the abstract for readability — pipe an id into `paper` for the full record.
## Common Categories
`cs.AI`, `cs.CL`, `cs.LG`, `cs.CV`, `cs.RO`, `stat.ML`, `math.PR`, `math.ST`, `q-bio.NC`, `econ.TH`, `physics.comp-ph`. Full list: <https://arxiv.org/category_taxonomy>.
## Prerequisites
- No browser required — uses public arXiv API
+69
View File
@@ -0,0 +1,69 @@
# Claude
**Mode**: Browser · **Domain**: `claude.ai`
## Commands
| Command | Description |
|---------|-------------|
| `opencli claude ask <prompt>` | Send a prompt and get the response |
| `opencli claude send <prompt>` | Send a prompt without waiting for the response |
| `opencli claude new` | Start a new conversation |
| `opencli claude status` | Check login state and page availability |
| `opencli claude read` | Read the current conversation |
| `opencli claude history` | List recent conversations from `/recents` |
| `opencli claude detail <id>` | Open a conversation by ID and read its messages |
## Usage Examples
```bash
# Ask a question
opencli claude ask "explain quicksort in 3 sentences"
# Start a new chat before asking
opencli claude ask "hello" --new
# Pick the model (default: sonnet; opus is paid-tier)
opencli claude ask "quick summary" --model haiku
# Enable Adaptive thinking
opencli claude ask "prove that sqrt(2) is irrational" --think
# Attach a file (image / PDF / text, up to ~1 MB raw)
opencli claude ask "describe this image" --file ./photo.png
# Combine modes
opencli claude ask "what does this PDF cover?" --file ./paper.pdf --think --new
# Custom timeout (default: 120s)
opencli claude ask "write a long essay" --timeout 240
# JSON output
opencli claude ask "hello" -f json
```
### Options (ask)
| Option | Description |
|--------|-------------|
| `<prompt>` | The message to send (required, positional) |
| `--timeout` | Wait timeout in seconds (default: 120) |
| `--new` | Start a new chat before sending (default: false) |
| `--model` | Model to use: `sonnet`, `opus`, or `haiku` (default: sonnet) |
| `--think` | Enable Adaptive thinking (default: false) |
| `--file` | Attach a file (image, PDF, text) with the prompt |
## Prerequisites
- Chrome running with [Browser Bridge extension](/guide/browser-bridge) installed
- Logged in to [claude.ai](https://claude.ai)
## Caveats
- This adapter drives the Claude web UI in the browser, not an API
- `claude read` queries the current automation tab; pair it with `--live` on the prior command (or chain after `claude detail <id>`) so the tab stays on the conversation between invocations
- `--model opus` requires a paid Claude plan; on a free-tier account the adapter surfaces a usage error rather than silently falling back
- The default Sonnet 4.6 model uses Adaptive thinking by default; `--think` is the explicit switch but Claude may still invoke thinking for complex prompts even when not requested
- Adaptive-thinking and file-thumbnail widgets render duplicated label paragraphs (`Thought process` / `View uploaded image`) at the top of the response; these are stripped automatically so the row value is the actual answer
- File upload is constrained by the daemon HTTP body limit (1 MB; `src/daemon.ts:152`); files up to ~700 KB raw work reliably, larger files (e.g. high-res images) may fail with `ECONNRESET`
- Long responses (code, essays) may need a higher `--timeout`
+43 -2
View File
@@ -9,8 +9,44 @@ Fetch the latest and greatest developer articles from the DEV community without
| Command | Description |
|---------|-------------|
| `opencli devto top` | Top DEV.to articles of the day |
| `opencli devto tag` | Latest articles for a specific tag |
| `opencli devto user` | Recent articles from a specific user |
| `opencli devto tag <tag>` | Latest articles for a specific tag |
| `opencli devto user <username>` | Recent articles from a specific user |
| `opencli devto read <id>` | Read the body of a single article |
## Listing columns
`top`, `tag`, and `user` all surface the same agent-native columns so the
article id is round-trippable into `devto read`:
| Column | Source | Notes |
|--------|--------|-------|
| `rank` | local | 1-indexed position in the result |
| `id` | `item.id` | Numeric article id, feed into `devto read` |
| `title` | `item.title` | |
| `author` | `item.user.username` | (omitted for `user` since it's user-scoped) |
| `reactions` | `item.public_reactions_count` | |
| `comments` | `item.comments_count` | |
| `reading_time` | `item.reading_time_minutes` | Minutes |
| `published_at` | `item.published_at` | ISO 8601 timestamp |
| `tags` | `item.tag_list` | Comma-separated |
| `url` | `item.url` | Canonical article URL |
## `read` columns
`devto read` returns a single row with the article body. DEV.to's public API
does not expose article comments, so this reader does not emit a comment tree.
| Column | Source |
|--------|--------|
| `id` | `article.id` |
| `title` | `article.title` |
| `author` | `article.user.username` |
| `reactions` | `article.public_reactions_count` |
| `reading_time` | `article.reading_time_minutes` |
| `tags` | `article.tag_list` (joined) |
| `published_at` | `article.published_at` |
| `body` | `article.body_markdown` (truncated by `--max-length`) |
| `url` | `article.url` |
## Usage Examples
@@ -26,8 +62,13 @@ opencli devto tag python --limit 20
opencli devto user ben
opencli devto user thepracticaldev --limit 5
# Read a single article body by id
opencli devto read 3605688
opencli devto read 3605688 --max-length 5000
# JSON output
opencli devto top -f json
opencli devto read 3605688 -f json
```
## Prerequisites
+4
View File
@@ -14,6 +14,7 @@
| `opencli hackernews jobs` | Hacker News job postings |
| `opencli hackernews search <query>` | Search Hacker News stories |
| `opencli hackernews user <username>` | Hacker News user profile |
| `opencli hackernews read <id>` | Read a story and its comment tree |
## Usage Examples
@@ -35,6 +36,9 @@ opencli hackernews top -f json
# Sort search by date
opencli hackernews search "rust" --sort date
# Read a story and its top comments (id from any listing's `id` column)
opencli hackernews read 47999636 --limit 5 --depth 2
```
## Prerequisites
+21 -2
View File
@@ -12,7 +12,9 @@
| `opencli instagram explore` | Discover trending posts |
| `opencli instagram followers` | List user's followers |
| `opencli instagram following` | List user's following |
| `opencli instagram saved` | Get your saved posts |
| `opencli instagram saved` | Get your saved posts (or one collection) |
| `opencli instagram collection-create` | Create a new saved-posts collection |
| `opencli instagram collection-delete` | Delete a saved-posts collection by name or id |
## Usage Examples
@@ -33,13 +35,30 @@ opencli instagram explore --limit 20
opencli instagram followers nasa --limit 20
opencli instagram following nasa --limit 20
# Get your saved posts
# Get your saved posts (default "All posts" feed)
opencli instagram saved --limit 10
# Get posts from a specific collection (case-insensitive name match)
opencli instagram saved --collection inspiration --limit 10
# Create a new saved-posts collection
opencli instagram collection-create "design refs"
# Delete a collection by name (or by numeric id, e.g. 17853899493659567)
opencli instagram collection-delete "design refs"
# JSON output
opencli instagram profile nasa -f json
```
### Notes on collections
- `instagram saved` without `--collection` returns the unsegmented "All posts" bucket (same as the original behaviour).
- With `--collection <name>` it resolves the name to an id via `/api/v1/collections/list/`, then fetches `/api/v1/feed/collection/{id}/posts/`. Match is case-insensitive after trimming. An unknown name throws an error that lists the available names.
- `instagram collection-create <name>` calls `POST /api/v1/collections/create/` with a multipart `name` field. Instagram silently accepts duplicate names — the API just returns a new `collection_id` each time, so dedupe client-side if you care.
- `instagram collection-delete <name-or-id>` calls `POST /api/v1/collections/{id}/delete/`. Pass either a case-insensitive collection name or a numeric `collection_id`. If the name resolves to multiple collections (e.g. duplicates from `collection-create`), the adapter throws and lists the candidate ids so you can disambiguate by passing the id explicitly. Unknown names list the available collections in the error message.
- Saving an existing post directly into a named collection in one shot is not exposed by the web app's documented endpoints (`/api/v1/web/save/{pk}/save/` only writes to "All posts"). Use `instagram save` first, then move the post in the UI, or extend with the `/api/v1/collections/{id}/edit/` mutation.
## Prerequisites
- Chrome running and **logged into** instagram.com
+15 -5
View File
@@ -9,7 +9,8 @@
| `opencli lobsters hot` | Hottest stories |
| `opencli lobsters newest` | Latest stories |
| `opencli lobsters active` | Most active discussions |
| `opencli lobsters tag` | Stories by tag |
| `opencli lobsters tag <tag>` | Stories by tag |
| `opencli lobsters read <short_id>` | Read a story and its comment tree |
## Usage Examples
@@ -18,15 +19,24 @@
opencli lobsters hot --limit 10
# Filter by tag
opencli lobsters tag --tag rust --limit 5
opencli lobsters tag rust --limit 5
# Read a specific story (use the short_id surfaced as `id` in any listing)
opencli lobsters read 6cmh6h --limit 25 --depth 2
# JSON output
opencli lobsters hot -f json
# Verbose mode
opencli lobsters hot -v
```
## Output Columns
| Command | Columns |
|---------|---------|
| `hot` / `newest` / `active` / `tag` | `rank, id, title, score, author, comments, created_at, tags, url` |
| `read` | `type, author, score, text` (POST + L0/L1/… comments, with `[+N more replies]` stubs) |
`id` is the lobste.rs `short_id` — pipe it into `read` to drill into the discussion.
## Prerequisites
None — all commands use the public JSON API, no browser or login required.
+61 -2
View File
@@ -7,9 +7,51 @@
| Command | Description |
|---------|-------------|
| `opencli stackoverflow hot` | Hot questions |
| `opencli stackoverflow search` | Search questions |
| `opencli stackoverflow search <query>` | Search questions |
| `opencli stackoverflow bounties` | Questions with active bounties |
| `opencli stackoverflow unanswered` | Unanswered questions |
| `opencli stackoverflow read <id>` | Read a question with answers and comments |
## Listing columns
`hot`, `search`, and `bounties` share an agent-native shape so the
`question_id` is round-trippable into `stackoverflow read`:
| Column | Source | Notes |
|--------|--------|-------|
| `rank` | local | 1-indexed position |
| `id` | `question_id` | Feed into `stackoverflow read` |
| `title` | `title` | |
| `score` | `score` | |
| `answers` | `answer_count` | |
| `views` | `view_count` | |
| `is_answered` | `is_answered` | (omitted on `unanswered` since always false) |
| `tags` | `tags` (joined) | Comma-separated |
| `author` | `owner.display_name` | |
| `creation_date` | `creation_date` | Unix epoch seconds |
| `url` | `link` | Canonical question URL |
| `bounty` | `bounty_amount` | (`bounties` only, prepended after `id`) |
## `read` columns
`stackoverflow read <id>` fetches the question, answers up to
`--answers-limit` (accepted first, then by votes — if the accepted
answer is outside the votes-sorted page it is fetched separately and
prepended), question comments up to `--comments-limit`, and answer
comments up to `--comments-limit` per answer. It mirrors the
`hackernews read` and `lobsters read` thread shape.
| Column | Description |
|--------|-------------|
| `type` | `POST` / `Q-COMMENT` / `ANSWER` / `A-COMMENT` |
| `author` | Display name (HTML entities decoded) |
| `score` | Vote count for that row |
| `accepted` | `'true'` for the accepted answer, empty string otherwise |
| `text` | Body / comment, HTML stripped, entities decoded, indented for comments |
The accepted answer (if any) is always the first `ANSWER` row. Other
answers follow in descending score order. Comments under an answer appear
immediately after that answer with `A-COMMENT` type and a `> ` indent.
## Usage Examples
@@ -26,10 +68,27 @@ opencli stackoverflow bounties --limit 10
# Unanswered questions
opencli stackoverflow unanswered --limit 10
# Read a question with answers and comments
opencli stackoverflow read 11227809
opencli stackoverflow read 11227809 --answers-limit 3 --comments-limit 5
# JSON output
opencli stackoverflow hot -f json
opencli stackoverflow read 11227809 -f json
```
## Caveats
- Stack Exchange API has a 300/day quota per IP for unauthenticated
requests. A `read` call uses up to 4 quota units (question, question
comments, answers, batched answer comments), or 5 when the accepted answer
must be fetched separately.
- `--answers-limit` and `--comments-limit` are bounded to 1-100, matching the
Stack Exchange API page size limit. If batched answer comments would be
partial, the command fails fast instead of returning an incomplete thread.
- Bodies are returned as HTML; this adapter strips tags and decodes named
/ decimal / hex HTML entities for plain-text consumption.
## Prerequisites
- No browser required — uses public Stack Exchange API
- No browser required — uses the public Stack Exchange API
+11
View File
@@ -12,6 +12,8 @@
| `opencli weibo user` | 用户信息 |
| `opencli weibo me` | 我的信息 |
| `opencli weibo post` | 发微博 |
| `opencli weibo favorites` | 我的微博收藏列表 |
| `opencli weibo publish` | 通过网页 UI 直接发布微博,支持最多 9 张图片 |
| `opencli weibo comments` | 微博评论 |
## Usage Examples
@@ -34,6 +36,15 @@ opencli weibo feed --type following --limit 10
# Verbose mode
opencli weibo hot -v
# Favorites
opencli weibo favorites --limit 20
# Publish text (executes immediately)
opencli weibo publish "Hello from OpenCLI"
# Publish text with images (executes immediately)
opencli weibo publish "Hello with images" --images /path/a.jpg,/path/b.png
```
## Prerequisites
+2 -1
View File
@@ -37,6 +37,7 @@ Run `opencli list` for the live registry.
| **[chaoxing](./browser/chaoxing.md)** | `assignments` `exams` | 🔐 Browser |
| **[grok](./browser/grok.md)** | `ask` `image` | 🔐 Browser |
| **[gemini](./browser/gemini.md)** | `new` `ask` `image` `deep-research` `deep-research-result` | 🔐 Browser |
| **[claude](./browser/claude.md)** | `ask` `send` `new` `status` `read` `history` `detail` | 🔐 Browser |
| **[maimai](./browser/maimai.md)** | `search-talents` | 🔐 Browser |
| **[yuanbao](./browser/yuanbao.md)** | `new` `ask` | 🔐 Browser |
| **[notebooklm](./browser/notebooklm.md)** | `status` `list` `open` `current` `get` `source-list` `source-get` `source-fulltext` `source-guide` `history` `note-list` `notes-get` `summary` | 🔐 Browser |
@@ -100,7 +101,7 @@ Run `opencli list` for the live registry.
| **[stackoverflow](./browser/stackoverflow.md)** | `hot` `search` `bounties` `unanswered` | 🌐 Public |
| **[wikipedia](./browser/wikipedia.md)** | `search` `summary` `random` `trending` | 🌐 Public |
| **[lesswrong](./browser/lesswrong.md)** | `curated` `frontpage` `new` `top` `top-week` `top-month` `top-year` `read` `comments` `user` `user-posts` `tag` `tags` `sequences` `shortform` | 🌐 Public |
| **[lobsters](./browser/lobsters.md)** | `hot` `newest` `active` `tag` | 🌐 Public |
| **[lobsters](./browser/lobsters.md)** | `hot` `newest` `active` `tag` `read` | 🌐 Public |
| **[steam](./browser/steam.md)** | `top-sellers` | 🌐 Public |
## Desktop Adapters
+1 -1
View File
@@ -87,7 +87,7 @@ OpenCLI occupies a specific niche in the browser automation ecosystem. This guid
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times for free.
- **Deterministic output** — Same command always returns the same schema. Pipeable, scriptable, CI-friendly.
- **Speed** — Adapter commands return in seconds, not minutes.
- **Broad platform coverage** — 87+ sites spanning global platforms (Reddit, HackerNews, Twitter, YouTube) and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Douban, Weibo) with adapters that understand local anti-bot patterns.
- **Broad platform coverage** — 100+ registered site surfaces spanning global platforms (Reddit, HackerNews, Twitter, YouTube) and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Douban, Weibo) with adapters that understand local anti-bot patterns.
- **Desktop app control** — CDP adapters for Cursor, Codex, Notion, ChatGPT, Discord, and more.
- **Easy to extend** — Drop a `.js` adapter into the `clis/` folder for auto-registration. Contributing a new site adapter is straightforward.
+6 -4
View File
@@ -11,10 +11,11 @@ From a new site URL to a passing `opencli browser verify` — one skill, one set
# skills/opencli-adapter-author/SKILL.md
# 2. Reconnaissance
opencli browser open https://example.com
opencli browser wait time 3
opencli browser network # inspect XHR / fetch calls
opencli browser state # extract __INITIAL_STATE__ / __NEXT_DATA__
opencli browser analyze https://example.com
# Fallback primitives when analyze says deeper inspection is needed:
# opencli browser open https://example.com
# opencli browser network # inspect XHR / fetch calls
# opencli browser state # extract __INITIAL_STATE__ / __NEXT_DATA__
# 3. Scaffold + verify
opencli browser init <site>/<name>
@@ -30,6 +31,7 @@ See [skills/opencli-adapter-author/SKILL.md](https://github.com/jackwener/opencl
| Command | Purpose |
|---------|---------|
| `opencli doctor` | Sanity check: bridge, Chrome, signals |
| `opencli browser analyze <url>` | One-shot site recon: anti-bot, pattern, nearest adapter, next step |
| `opencli browser open <url>` | Open a tab in the Chrome session |
| `opencli browser network` | List recent XHR / fetch calls |
| `opencli browser state` | Page state: URL, title, interactive elements |
+144 -83
View File
@@ -1,103 +1,164 @@
# Architecture
OpenCLI is built on a **Dual-Engine Architecture** that supports both declarative pipelines and programmatic TypeScript adapters.
OpenCLI is a command surface that sits on top of four major subsystems:
## High-Level Architecture
1. command discovery and registry
2. execution and formatting
3. browser / daemon / CDP connectivity
4. adapter, plugin, and external CLI integration
```
┌─────────────────────────────────────────────────────┐
│ opencli CLI │
│ (Commander.js entry point) │
├─────────────────────────────────────────────────────┤
│ Engine Layer │
│ ┌──────────────┐ ┌──────────────┐ ┌────────────┐ │
Registry │ │ Dynamic │ │ Output │ │
(commands) │ │ Loader │ │ Formatter │ │
└──────────────┘ └──────────────┘ └────────────┘ │
├─────────────────────────────────────────────────────┤
│ Adapter Layer │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
Pipeline │ │ TypeScript Adapters │ │
│ │ (declarative) │ │ (browser/desktop/AI) │ │
└─────────────────┘ └──────────────────────────┘ │
├─────────────────────────────────────────────────────┤
│ Connection Layer │
│ ┌─────────────────┐ ┌──────────────────────────┐ │
│ │ Browser Bridge │ │ CDP (Chrome DevTools) │ │
│ │ (Extension+WS) │ │ (Electron apps) │ │
│ └─────────────────┘ └──────────────────────────┘ │
└─────────────────────────────────────────────────────┘
## Runtime Shape
```text
opencli CLI
├─ command discovery / registry
├─ execution / output
├─ browser runtime
├─ Browser Bridge extension
├─ local daemon
└─ direct CDP path
├─ adapter loading
│ ├─ built-in site adapters
│ ├─ generated adapters
└─ pipeline-backed adapters
├─ plugin loading
└─ external CLI passthrough
```
## Core Modules
### Registry (`src/registry.ts`)
Central command registry. All adapters register their commands via the `cli()` function with metadata: site, name, description, domain, strategy, args, columns.
### CLI Surface
### Discovery (`src/discovery.ts`)
CLI discovery and manifest loading. Discovers commands from TypeScript adapter files, parses pipelines, and registers them into the central registry.
- `src/main.ts` — process entrypoint
- `src/cli.ts` — top-level command tree and built-in command groups
- `src/completion.ts` / `src/completion-fast.ts` — shell completion
### Execution (`src/execution.ts`)
Command execution: argument validation, lazy loading of adapter modules, and executing the appropriate handler function.
### Discovery, Registry, Execution
### Commander Adapter (`src/commanderAdapter.ts`)
Bridges the Registry commands to Commander.js subcommands. Handles positional args, named options, browser session wiring, and output formatting. Isolates all Commander-specific logic so the core is framework-agnostic.
- `src/discovery.ts` — discovers built-in adapters, generated adapters, plugins, and manifests
- `src/registry.ts` — central command registry
- `src/registry-api.ts` — adapter-facing registration helpers
- `src/execution.ts` — argument validation, lazy loading, and command execution
- `src/commanderAdapter.ts` — bridges registry metadata into Commander subcommands
- `src/output.ts``table`, `json`, `yaml`, `md`, `csv` formatting
- `src/serialization.ts` — registry and manifest serialization helpers
### Browser (`src/browser.ts`)
Manages connections to Chrome via the Browser Bridge WebSocket daemon. Handles JSON-RPC messaging, tab management, and extension/standalone mode switching.
### Browser and Runtime
### Pipeline (`src/pipeline/`)
The pipeline engine. Processes declarative steps:
- **fetch** — HTTP requests with cookie/header strategies
- **map** — Data transformation with template expressions
- **limit** — Result truncation
- **filter** — Conditional filtering
- **download** — Media download support
- `src/runtime.ts` — shared command runtime and target resolution
- `src/daemon.ts` — lifecycle and bridge behavior for the local daemon
- `src/doctor.ts` — browser bridge diagnostics
- `src/observation/` — trace artifacts, redaction, and structured runtime evidence
- `src/interceptor.ts` — interception helpers for browser-backed strategies
- `src/browser/` — Browser Bridge connection and browser-side primitives
### Output (`src/output.ts`)
Unified output formatting: `table`, `json`, `yaml`, `md`, `csv`.
### Pipeline Engine
## Authentication Strategies
- `src/pipeline/executor.ts` — pipeline execution
- `src/pipeline/template.ts` — template expansion
- `src/pipeline/transform.ts` — transform helpers
- `src/pipeline/steps/` — concrete steps such as:
- `fetch`
- `download`
- `browser`
- `intercept`
- `tap`
- `transform`
OpenCLI uses a 3-tier authentication strategy:
### Adapter and Extension Surfaces
| Strategy | How It Works | When to Use |
|----------|-------------|-------------|
| `public` | Direct HTTP fetch, no auth | Public APIs (HackerNews, BBC) |
| `cookie` | Reuse Chrome cookies via Browser Bridge | Logged-in sites (Bilibili, Zhihu) |
| `header` | Custom auth headers | API-key based services |
| `intercept` | Network request interception | GraphQL/XHR capture (Twitter) |
| `ui` | DOM interaction via accessibility snapshot | Desktop apps, write operations |
- `clis/` — built-in site adapters
- `src/plugin.ts` / `src/plugin-manifest.ts` / `src/plugin-scaffold.ts` — plugin install, metadata, scaffold
- `src/external.ts` / `src/external-clis.yaml` — external CLI passthrough and installable tools
- `src/electron-apps.ts` — desktop / Electron app support
## Directory Structure
## Command Sources
OpenCLI merges commands from multiple places into one registry:
| Source | Location | Examples |
|---|---|---|
| Built-in adapters | `clis/` | `twitter`, `bilibili`, `reddit`, `chatgpt-app` |
| Generated / local adapters | `~/.opencli/clis/` | user-authored adapters |
| Plugins | `~/.opencli/plugins/` | community-contributed commands |
| External CLIs | `src/external-clis.yaml` + local registrations | `gh`, `docker`, `vercel` |
The user sees one unified command tree through `opencli list`.
## Connectivity Modes
### Browser Bridge mode
Primary path for browser-backed commands:
```text
opencli process
↔ local daemon
↔ Browser Bridge extension
↔ logged-in Chrome / Chromium
```
src/
├── main.ts # Entry point
├── cli.ts # Commander.js CLI setup + built-in commands
├── commanderAdapter.ts # Registry → Commander bridge
├── discovery.ts # CLI discovery, manifest loading
├── execution.ts # Arg validation, command execution
├── registry.ts # Command registry
├── serialization.ts # Command serialization helpers
├── runtime.ts # Browser session & timeout management
├── browser/ # Browser Bridge connection
├── output.ts # Output formatting
├── doctor.ts # Diagnostic tool
├── pipeline/ # Pipeline engine
│ ├── runner.ts
│ ├── template.ts
│ ├── transform.ts
│ └── steps/
│ ├── fetch.ts
│ ├── map.ts
│ ├── limit.ts
│ ├── filter.ts
│ └── download.ts
└── clis/ # Site adapters
├── twitter/
├── reddit/
├── bilibili/
├── cursor/
└── ...
```
This path is used for:
- cookie-backed websites
- browser automation primitives
- interactive browser verification
### Direct CDP mode
Used when OpenCLI talks directly to a Chrome or Electron debugging endpoint through `OPENCLI_CDP_ENDPOINT`.
Typical uses:
- remote Chrome
- headless Chrome
- Electron desktop adapters
## Authentication / Access Strategies
OpenCLI currently uses these access strategies:
| Strategy | Purpose |
|---|---|
| `public` | direct fetch with no login |
| `cookie` | reuse browser session cookies |
| `header` | custom authenticated headers |
| `intercept` | capture the app's own network responses |
| `ui` | DOM / accessibility driven interaction |
The key distinction is operational:
- `public`, `header` favor direct network access
- `cookie`, `intercept`, `ui` depend on a live browser or desktop surface
## High-Risk Change Zones
Changes in these files usually affect broad command behavior:
- `src/cli.ts`
- `src/commanderAdapter.ts`
- `src/discovery.ts`
- `src/execution.ts`
- `src/runtime.ts`
- `src/daemon.ts`
- `src/plugin.ts`
- `src/external.ts`
- `src/pipeline/**`
These areas deserve targeted tests first, then broader validation when the change crosses module boundaries.
## Mental Model
The simplest accurate model is:
1. OpenCLI discovers command definitions.
2. It registers them into one command registry.
3. It resolves each invocation through execution + runtime.
4. It reaches the target through one of:
- network fetch
- Browser Bridge
- direct CDP
- external CLI passthrough
5. It formats the result into a stable output surface.
That is the architecture to preserve when refactoring.
+5 -4
View File
@@ -17,7 +17,7 @@ npm run build
# 4. Run a few checks
npx tsc --noEmit
npm test
npm run build
# 5. Link globally (optional, for testing `opencli` command)
npm link
@@ -98,11 +98,12 @@ chore: bump vitest to v4
1. Create a feature branch: `git checkout -b feat/mysite-trending`
2. Make your changes and add tests when relevant
3. Run the checks:
3. Run the smallest check set that matches your change:
```bash
npx tsc --noEmit # Type check
npm test # Default local gate: unit + extension + adapter
npm run test:adapter # Adapter-only project (optional while iterating on adapters)
npm run build # Ensure dist stays healthy
npx vitest run src/<target>.test.ts
npm test # Broader local gate when shared runtime changes justify it
```
4. Commit using conventional commit format
5. Push and open a PR
@@ -0,0 +1,368 @@
# Documentation Audit — 2026-05
This document reviews the current long-form docs, README surfaces, and developer guides in `opencli`. It focuses on stale facts, internal contradictions, and documentation structure that now causes drift.
## Scope
- `README.md`
- `README.zh-CN.md`
- `docs/`
- `skills/` references that are linked from user-facing docs
## Executive View
The docs are usable, but they are drifting in four visible ways:
1. **Hard-coded counts and feature claims are stale.**
2. **Developer docs describe an older architecture and older test layout.**
3. **English and Chinese docs are no longer updated with the same rigor.**
4. **Some pages still describe deleted concepts or old workflows.**
The highest-value work is:
1. Fix the stale facts in `README*`, `docs/index.md`, `docs/zh/index.md`, and `docs/guide/getting-started.md`.
2. Rewrite `docs/developer/testing.md` and `docs/developer/architecture.md` against current `main`.
3. Make English and Chinese entry docs derive from the same source-of-truth checklist.
4. Stop writing command/adapters counts by hand unless they are generated.
## Priority 0 — Clearly stale or incorrect
### 1. Adapter / site counts are stale across multiple entry points
Affected files:
- `README.md`
- `README.zh-CN.md`
- `docs/guide/getting-started.md`
- `docs/comparison.md`
Current problems:
- `README.md` and `README.zh-CN.md` still say `90+` adapters.
- `docs/guide/getting-started.md` still says `87+` pre-built adapters.
- `docs/comparison.md` still says `87+` sites.
Current reality:
- `node dist/src/main.js list --format json | jq 'map(.site) | unique | length'` returns `106`.
Why this matters:
- These are the first pages people read.
- The mismatch is easy to notice and weakens trust in the rest of the docs.
- These values will keep drifting if we maintain them manually.
Recommended fix:
- Replace all hard-coded counts with one of:
- `100+`
- `100+ sites`
- `over 100 registered sites`
- Best option: generate this number into docs at release time or avoid explicit counts entirely.
### 2. `docs/developer/testing.md` is materially out of date
Affected file:
- `docs/developer/testing.md`
Current problems:
- It says adapter tests live in `clis/**/*.test.{ts,js}`.
- The file examples name adapter tests such as:
- `clis/zhihu/download.test.ts`
- `clis/twitter/timeline.test.ts`
- `clis/reddit/read.test.ts`
- `clis/bilibili/dynamic.test.ts`
- Those files do not exist.
- It says E2E coverage is `5` files.
- Current reality is `11` E2E files.
- It presents `npm test` as the main local gate, while current team rule is to prefer the smallest sufficient test set instead of default full-suite runs.
Current reality from the repo:
- `find src -name '*.test.ts' | wc -l``60`
- `find clis -iregex '.*\\.test\\.(ts|js)$' | wc -l``0`
- `find tests/e2e -name '*.test.ts' | wc -l``11`
- `find tests/smoke -name '*.test.ts' | wc -l``1`
Why this matters:
- This page is the main developer testing contract.
- A new contributor following it will get the wrong mental model of the test layout.
- It encourages a heavier default test habit than the team currently wants.
Recommended fix:
- Rewrite the page from current files, not from remembered structure.
- Separate:
- `fast local checks`
- `targeted validation`
- `full CI coverage`
- Remove nonexistent adapter test examples.
- Add a short rule:
- local default = smallest sufficient validation
- full-suite = broader refactor, shared runtime changes, or CI
### 3. `docs/developer/architecture.md` describes an older system shape
Affected file:
- `docs/developer/architecture.md`
Current problems:
- It refers to `src/browser.ts`, but that file does not exist.
- The directory structure block says `src/clis/`, but adapters live at top-level `clis/`.
- The architecture diagram is too simplified for the current system and omits important pieces such as:
- `daemon.ts`
- `external.ts`
- `plugin.ts`
- `electron-apps.ts`
- update check / diagnostics / runtime detection paths
- It says “3-tier authentication strategy” but lists `5` strategies.
Why this matters:
- This is the page people read to understand the project.
- Once architecture docs are stale, all deeper docs become harder to trust.
Recommended fix:
- Rewrite this page around current modules:
- command discovery and registry
- execution
- browser / daemon bridge
- external CLI integration
- plugin system
- desktop / CDP path
- pipeline engine
- Replace the static tree with a curated module map that matches current filenames.
- Change “3-tier” to a neutral label like `authentication strategies`.
### 4. Home pages still mention deleted concepts
Affected files:
- `docs/index.md`
- `docs/zh/index.md`
Current problems:
- Both home pages say:
- `explore`
- `synthesize`
- `cascade`
- `docs/developer/ai-workflow.md` explicitly says those commands do not exist and that the skill drives the loop.
Why this matters:
- The home page is currently teaching a product vocabulary that the actual CLI does not have.
- This creates immediate confusion for users who go from docs to terminal.
Recommended fix:
- Replace those phrases with current concepts:
- `browser primitives`
- `adapter-authoring skill`
- `verify loop`
- Keep the homepage aligned with `docs/developer/ai-workflow.md`.
### 5. Chinese getting-started page lists a deleted built-in command
Affected file:
- `docs/zh/guide/getting-started.md`
Current problem:
- It says built-in commands include `list、explore、validate...`
- `explore` is not a current built-in command.
Why this matters:
- This is a hard user-facing error.
Recommended fix:
- Replace the example list with current built-ins such as:
- `list`
- `validate`
- `verify`
- `browser`
- `doctor`
- `plugin`
- `adapter`
## Priority 1 — Inconsistent or incomplete
### 6. Installation pages are inconsistent about runtime support and update flow
Affected files:
- `README.md`
- `README.zh-CN.md`
- `docs/guide/installation.md`
- `docs/zh/guide/installation.md`
Current problems:
- `README.md` says Node `>= 21` or Bun `>= 1.0`.
- `docs/guide/installation.md` and `docs/zh/guide/installation.md` only mention Node.
- `README.md` documents skill refresh on update.
- `docs/zh/guide/installation.md` only documents package update and omits skills refresh.
Why this matters:
- Entry docs should agree on install prerequisites and upgrade procedure.
Recommended fix:
- Pick one official runtime support statement and reuse it everywhere.
- If Bun is supported, add it consistently to guide pages.
- Mirror the post-update skill refresh guidance in the install/update guides.
### 7. README and docs still use top-level tables and examples that will drift by hand
Affected files:
- `README.md`
- `README.zh-CN.md`
Current problems:
- The “Built-in Commands” section is manually curated and already partially selective.
- The surrounding copy still frames it like a broad current snapshot.
Why this matters:
- Manual command snapshots go stale quickly in a repo with active adapter growth.
Recommended fix:
- Reframe the section as:
- “Representative built-in commands”
- “Sample sites”
- Keep `opencli list` and `docs/adapters/index.md` as the full registry surface.
### 8. `docs/comparison.md` contains stale scale claims
Affected file:
- `docs/comparison.md`
Current problem:
- It still says `87+` sites.
Why this matters:
- Comparison pages shape market positioning.
- Stale numbers make the project look less maintained than it is.
Recommended fix:
- Remove exact numbers from comparison copy unless they are generated.
## Priority 2 — Structural drift risks
### 9. English and Chinese docs are drifting independently
Most visible examples:
- `docs/index.md` and `docs/zh/index.md` both kept the deleted `explore / synthesize / cascade` language.
- `docs/zh/guide/getting-started.md` contains a stale built-in command example that should have been caught by parity review.
- `README.md` and `README.zh-CN.md` both carry the same stale adapter count.
Why this keeps happening:
- We have mirrored content with no explicit parity checklist.
- Updates land in one place and rely on memory for the rest.
Recommended fix:
- Introduce a small doc parity checklist for any change that touches:
- `README.md`
- `README.zh-CN.md`
- `docs/index.md`
- `docs/zh/index.md`
- `docs/guide/*`
- `docs/zh/guide/*`
- Add one PR checklist item:
- “Did this change require an English/Chinese mirror update?”
### 10. Core product pages mix generated facts with narrative copy
Examples:
- command counts
- site counts
- test counts
- lists of built-in commands
Why this matters:
- Numbers and command inventories drift faster than narrative guidance.
Recommended fix:
- For fast-changing facts:
- generate them
- or generalize them
- Reserve hand-written docs for:
- mental models
- workflows
- constraints
- trade-offs
## Suggested rewrite order
### Pass 1 — Fix trust-breaking errors
1. `README.md`
2. `README.zh-CN.md`
3. `docs/index.md`
4. `docs/zh/index.md`
5. `docs/guide/getting-started.md`
6. `docs/zh/guide/getting-started.md`
7. `docs/comparison.md`
### Pass 2 — Rebuild the technical source-of-truth pages
1. `docs/developer/testing.md`
2. `docs/developer/architecture.md`
3. `docs/guide/installation.md`
4. `docs/zh/guide/installation.md`
### Pass 3 — Prevent the next round of drift
1. Add a docs parity checklist to PR workflow.
2. Remove exact counts from hand-written copy unless generated.
3. Decide which pages are authoritative for:
- install
- browser bridge
- testing
- architecture
- AI workflow
## Concrete edits I would make next
### Small fast edits
- Replace all `87+` / `90+` claims with `100+`.
- Remove `explore / synthesize / cascade` from both home pages.
- Remove `explore` from `docs/zh/guide/getting-started.md`.
- Align install docs on Node/Bun support and skill refresh.
### Medium rewrites
- Rewrite `docs/developer/testing.md` from current filesystem state.
- Rewrite `docs/developer/architecture.md` from current module boundaries.
### Process fix
- Add a lightweight “doc drift” checklist to PRs that touch command surface, runtime support, testing strategy, or adapter discovery.
## Bottom line
The docs do not need a ground-up rewrite. They need a focused trust repair pass on entry pages, then a source-of-truth rebuild for testing and architecture, then a small process change so counts and mirrored pages stop drifting.
+118 -216
View File
@@ -1,255 +1,157 @@
# Testing Guide
> 面向开发者和 AI Agent 的测试参考手册。
> 面向开发者和 AI Agent 的当前测试参考手册。
## 目录
## 测试结构
- [测试架构](#测试架构)
- [当前覆盖范围](#当前覆盖范围)
- [本地运行测试](#本地运行测试)
- [如何添加新测试](#如何添加新测试)
- [CI/CD 流水线](#cicd-流水线)
- [浏览器模式](#浏览器模式)
- [站点兼容性](#站点兼容性)
OpenCLI 当前测试主要分成四类:
---
| 类别 | 位置 | 当前规模 | 主要用途 |
|---|---|---:|---|
| 单元测试 | `src/**/*.test.ts` | 60 | 核心运行时、命令层、浏览器桥、输出、插件、诊断 |
| E2E 测试 | `tests/e2e/*.test.ts` | 11 | 真实 CLI 入口、公开站点、浏览器命令、管理命令、输出格式 |
| 烟雾测试 | `tests/smoke/*.test.ts` | 1 | 外部 API 与注册完整性健康检查 |
| 步骤级测试 | `src/pipeline/steps/*.test.ts` | 已包含在单元测试内 | pipeline step 行为与边界情况 |
## 测试架构
当前仓库里没有独立的 `clis/**/*.test.{ts,js}` adapter 测试树。adapter 相关验证主要分布在:
测试分为三层,全部使用 **vitest** 运行:
- `tests/e2e/`
- `src/commanderAdapter.test.ts`
- `src/registry.test.ts`
- `src/execution.test.ts`
- `src/validate.ts` / `opencli validate`
```text
tests/
├── e2e/ # E2E 集成测试(子进程运行真实 CLI)
│ ├── helpers.ts # runCli() / parseJsonOutput() 共享工具
│ ├── public-commands.test.ts # 公开 API 命令
│ ├── browser-public.test.ts # 浏览器命令(公开数据)
│ ├── browser-auth.test.ts # 需登录命令(graceful failure
│ ├── management.test.ts # 管理命令(list / validate / verify / help
│ └── output-formats.test.ts # 输出格式校验
├── smoke/
│ └── api-health.test.ts # 外部 API、adapter 定义、命令注册健康检查
src/
├── **/*.test.ts # 核心单元测试(`unit` project
clis/
└── **/*.test.{ts,js} # adapter tests`adapter` project
```
## 本地默认策略
| 层 | 位置 | 当前文件数 | 运行方式 | 用途 |
|---|---|---:|---|---|
| 单元测试 | `src/**/*.test.ts`(排除 `clis/**` | - | `npm test` | 内部模块、pipeline、runtime |
| Adapter 测试 | `clis/**/*.test.{ts,js}` | - | `npm test` / `npm run test:adapter` | adapter 命令与数据归一化 |
| E2E 测试 | `tests/e2e/*.test.ts` | 5 | `npx vitest run tests/e2e/` | 真实 CLI 命令执行 |
| 烟雾测试 | `tests/smoke/*.test.ts` | 1 | `npx vitest run tests/smoke/` | 外部 API 与注册完整性 |
本地默认跑最小充分验证,不要先跑全量。
---
推荐顺序:
## 当前覆盖范围
1. 改动命令文案、输出格式、参数解析:
- 跑对应单元测试
- 跑一条真实 CLI 命令做 spot check
2. 改动 adapter 发现、注册、验证逻辑:
-`src/registry.test.ts`
-`src/execution.test.ts`
-`opencli validate`
3. 改动 browser / daemon / runtime
- 跑对应 `src/*test.ts`
- 必要时补一条 `tests/e2e/*` 或手动 `opencli browser ...` 验证
4. 改动共享底层、跨多个模块、或 merge 前需要更高信心:
- 再扩大到 `npm test`
### 单元测试与 Adapter 测试
| 领域 | 文件 |
|---|---|
| 核心运行时与输出 | `src/browser.test.ts`, `src/browser/dom-snapshot.test.ts`, `src/build-manifest.test.ts`, `src/capabilityRouting.test.ts`, `src/doctor.test.ts`, `src/engine.test.ts`, `src/interceptor.test.ts`, `src/output.test.ts`, `src/plugin.test.ts`, `src/registry.test.ts`, `src/snapshotFormatter.test.ts` |
| pipeline 与下载 | `src/download/index.test.ts`, `src/pipeline/executor.test.ts`, `src/pipeline/template.test.ts`, `src/pipeline/transform.test.ts` |
| 聚焦 adapter 逻辑 | `clis/zhihu/download.test.ts`, `clis/twitter/timeline.test.ts`, `clis/reddit/read.test.ts`, `clis/bilibili/dynamic.test.ts` |
这些测试覆盖的重点包括:
- Browser Bridge、DOM snapshot、interceptor、capability routing
- manifest 生成、命令发现、插件安装与注册表
- 输出格式渲染与 snapshot formatting
- pipeline 模板求值、执行器与变换步骤
- 各站点 adapter 的数据归一化、参数处理与容错逻辑
### E2E 测试(5 个文件)
| 文件 | 当前覆盖范围 |
|---|---|
| `tests/e2e/public-commands.test.ts` | `bloomberg``apple-podcasts``hackernews``v2ex``xiaoyuzhou``google suggest` 等公开命令 |
| `tests/e2e/browser-public.test.ts` | `bbc``bloomberg``bilibili``weibo``zhihu``reddit``twitter``xueqiu``reuters``youtube``smzdm``boss``ctrip``coupang``xiaohongshu``google``yahoo-finance``v2ex daily` |
| `tests/e2e/browser-auth.test.ts` | `bilibili``twitter``v2ex``xueqiu``linux-do``xiaohongshu` 的需登录命令 graceful failure |
| `tests/e2e/management.test.ts` | `list``validate``verify``--version``--help`、unknown command |
| `tests/e2e/output-formats.test.ts` | `json` / `yaml` / `csv` / `md` 输出格式校验 |
### 烟雾测试(1 个文件)
| 文件 | 当前覆盖范围 |
|---|---|
| `tests/smoke/api-health.test.ts` | `hackernews``v2ex` 公开 API 可用性,`validate` 全量 adapter 校验,以及命令注册表基础完整性 |
### 快速核对命令
需要刷新测试清单时,直接以仓库文件为准:
## 常用命令
```bash
find src -name '*.test.ts' | sort
find tests/e2e -name '*.test.ts' | sort
find tests/smoke -name '*.test.ts' | sort
# 类型检查
npx tsc --noEmit
# 编译产物
npm run build
# 跑一个目标测试文件
npx vitest run src/<target>.test.ts
# 全量 vitest projects
npm run test:all
# E2E
npm run test:e2e
# 适配器注册 / schema 校验
node dist/src/main.js validate
```
---
## 本地运行测试
### 前置条件
如果你明确要跑 adapter project,也可以执行:
```bash
npm ci # 安装依赖
npm run build # 编译(E2E / smoke 测试需要 dist/src/main.js
```
### 运行命令
```bash
# 默认本地测试口径(unit + extension + adapter
npm test
# 只跑 adapter project
npm run test:adapter
# 全部 E2E 测试(会真实调用外部 API / 浏览器)
npx vitest run tests/e2e/
# 全部 smoke 测试
npx vitest run tests/smoke/
# 单个测试文件
npx vitest run clis/apple-podcasts/commands.test.ts
npx vitest run tests/e2e/management.test.ts
# 全部测试
npx vitest run
# watch 模式(开发时推荐)
npx vitest src/
```
### 浏览器命令本地测试须知
## 当前 E2E 文件
- opencli 通过 Browser Bridge 扩展连接已运行的 Chrome 浏览器
- E2E 测试通过 `tests/e2e/helpers.ts` 里的 `runCli()` 调用已构建的 `dist/src/main.js`
- `browser-public.test.ts` 使用 `tryBrowserCommand()`,站点反爬或地域限制导致空数据时会 warn + pass
- `browser-auth.test.ts` 验证 **graceful failure**,重点是不 crash、不 hang、错误信息可控
- 如需测试完整登录态,保持 Chrome 登录态并安装 Browser Bridge 扩展,再手动运行对应测试
- 对依赖具体 host 页面上下文的 browser adapter,除了单测外,还应手动验证真实命令,并把必要的 target host 约束写进 adapter docs / troubleshooting
- 对会主动导航页面的 browser commands,手动验证时优先串行执行;多个 CLI 进程同时连到同一个 CDP target 可能互相覆盖导航,制造假的 adapter 故障
当前 `tests/e2e/` 包含:
---
- `browser-auth.test.ts`
- `browser-public.test.ts`
- `cli.test.ts`
- `extension-bridge.test.ts`
- `formats.test.ts`
- `list.test.ts`
- `management.test.ts`
- `public-commands.test.ts`
- `recovery.test.ts`
- `remote-chrome.test.ts`
- `tab-targeting.test.ts`
## 如何添加新测试
如果这个列表变化,以仓库文件为准:
### 新增 Adapter(如 `clis/producthunt/trending.ts`
1. 根据 adapter 类型,在对应测试文件补一个 `it()` block
```typescript
// 如果 browser: false(公开 API)→ tests/e2e/public-commands.test.ts
it('producthunt trending returns data', async () => {
const { stdout, code } = await runCli(['producthunt', 'trending', '--limit', '3', '-f', 'json']);
expect(code).toBe(0);
const data = parseJsonOutput(stdout);
expect(Array.isArray(data)).toBe(true);
expect(data.length).toBeGreaterThanOrEqual(1);
expect(data[0]).toHaveProperty('title');
}, 30_000);
```bash
find tests/e2e -name '*.test.ts' | sort
```
```typescript
// 如果 browser: true 但可公开访问 → tests/e2e/browser-public.test.ts
it('producthunt trending returns data', async () => {
const data = await tryBrowserCommand(['producthunt', 'trending', '--limit', '3', '-f', 'json']);
expectDataOrSkip(data, 'producthunt trending');
}, 60_000);
## 当前值得优先覆盖的区域
以下改动最容易引入回归:
- `src/cli.ts`
- `src/commanderAdapter.ts`
- `src/discovery.ts`
- `src/execution.ts`
- `src/runtime.ts`
- `src/daemon.ts`
- `src/plugin.ts`
- `src/external.ts`
- `src/pipeline/**`
这类改动优先补:
- 精准单元测试
- 一条真实 CLI 验证路径
- 必要时再扩大到 `npm test`
## 手动验证建议
文档或命令面改动后,优先做 2 到 4 条真实命令 spot check,例如:
```bash
node dist/src/main.js --help
node dist/src/main.js list --format json
node dist/src/main.js plugin --help
node dist/src/main.js doctor --help
```
```typescript
// 如果 browser: true 且需登录 → tests/e2e/browser-auth.test.ts
it('producthunt me fails gracefully without login', async () => {
await expectGracefulAuthFailure(['producthunt', 'me', '-f', 'json'], 'producthunt me');
}, 60_000);
浏览器相关改动再补:
```bash
node dist/src/main.js browser --help
node dist/src/main.js browser tab list
```
### 新增管理命令(如 `opencli export`
## CI 角色
`tests/e2e/management.test.ts` 添加测试;如果新命令会影响输出格式,也同步补 `tests/e2e/output-formats.test.ts`
CI 负责更大范围的回归信心,本地负责最快闭环
### 新增内部模块
适合交给 CI 的内容:
在对应源码旁创建 `*.test.ts`,优先和被测模块放在同一目录下,便于发现与维护。
- 更大的命令面回归
- 多环境差异
- E2E 稳定性
- smoke 检查
### 决策流程图
适合本地优先做的内容:
```text
新增功能 → 是内部模块? → 是 → src/ 下加 *.test.ts
↓ 否
是 CLI 命令? → browser: false? → tests/e2e/public-commands.test.ts
↓ true
公开数据? → tests/e2e/browser-public.test.ts
↓ 需登录
tests/e2e/browser-auth.test.ts
```
- 参数解析
- 输出格式
- 注册与发现
- 文档相关命令行为
- 共享模块的小范围回归
---
## 更新这份文档的规则
## CI/CD 流水线
当以下任一项变化时,顺手更新此页:
### `ci.yml`
| Job | 触发条件 | 内容 |
|---|---|---|
| `build` | push/PR 到 `main`,`dev` | `tsc --noEmit` + `npm run build` |
| `unit-test` | push/PR 到 `main`,`dev` | Node `22` 运行 `unit + extension` tests,按 `2` shard 并行 |
| `adapter-test` | push/PR 到 `main`,`dev` | Node `22` 单独运行 `adapter` project |
| `smoke-test` | `schedule``workflow_dispatch` | 安装真实 Chrome`xvfb-run` 执行 `tests/smoke/` |
### `e2e-headed.yml`
| Job | 触发条件 | 内容 |
|---|---|---|
| `e2e-headed` | push/PR 到 `main`,`dev`,或手动触发 | 安装真实 Chrome,`xvfb-run` 执行 `tests/e2e/` |
E2E 与 smoke 都使用 `./.github/actions/setup-chrome` 准备真实 Chrome。
### Sharding
CI 里的 `unit-test` job 使用 vitest shard,只切 `unit + extension`,避免和独立的 `adapter-test` job 重复:
::: v-pre
```yaml
strategy:
matrix:
shard: [1, 2]
steps:
- run: npx vitest run --project unit --project extension --reporter=verbose --shard=${{ matrix.shard }}/2
```
:::
---
## 浏览器模式
opencli 通过 Browser Bridge 扩展连接浏览器:
| 条件 | 模式 | 使用场景 |
|---|---|---|
| 扩展已安装 / 已连接 | Extension 模式 | 本地用户,连接已登录的 Chrome |
| 无扩展 token | CLI 自行拉起浏览器 | CI、无登录态或纯自动化场景 |
CI 通过 `./.github/actions/setup-chrome` 准备真实 Chrome,再直接执行测试。
---
## 站点兼容性
GitHub Actions 的美国 runner 上,部分站点会因为地域限制、登录要求或反爬而返回空数据。当前 E2E 对这些场景采用 warn + pass 策略,避免偶发站点限制把整条 CI 打红。
| 站点 | CI 表现 | 常见原因 |
|---|---|---|
| `hackernews``bbc``v2ex``bloomberg` | 通常返回数据 | 公开接口或公开页面 |
| `yahoo-finance``google` | 通常返回数据 | 页面公开,但仍可能受限流影响 |
| `bilibili``zhihu``weibo``xiaohongshu``xueqiu` | 容易空数据 | 地域限制、反爬、登录要求 |
| `reddit``twitter``youtube` | 容易空数据 | 登录态、cookie、机器人检测 |
| `smzdm``boss``ctrip``coupang``linux-do` | 结果波动较大 | 地域限制、风控或页面结构变动 |
> 如果需要更稳定的浏览器 E2E 结果,优先使用具备目标站点网络可达性的 self-hosted runner。
- `tests/e2e/` 文件列表
- 默认本地测试命令
- `package.json` 测试脚本
- 共享运行时的高风险模块
+130
View File
@@ -0,0 +1,130 @@
# Extending OpenCLI
OpenCLI has five extension paths. Pick the path based on where you want the source code to live and how you want commands to be shared.
| Goal | Use | Source location | Command surface |
|------|-----|-----------------|-----------------|
| Build a personal website command in your own Git repo | Local plugin | Your project directory, symlinked into `~/.opencli/plugins/` | `opencli <plugin> <command>` |
| Quickly draft a private adapter on this machine | User adapter | `~/.opencli/clis/<site>/<command>.js` | `opencli <site> <command>` |
| Edit an official adapter locally | Adapter override | `~/.opencli/clis/<site>/` | `opencli <site> <command>` |
| Publish or install third-party commands | Plugin | Git repo, installed into `~/.opencli/plugins/` | `opencli <plugin> <command>` |
| Wrap an existing local binary | External CLI | `~/.opencli/external-clis.yaml` | `opencli <tool> ...` |
## Personal commands in your own Git repo
Use a local plugin when you want the code to stay in a normal project directory and be managed by Git.
```bash
opencli plugin create my-cnn
cd my-cnn
git init
opencli plugin install file://$(pwd)
opencli my-cnn hello
```
`plugin install file://...` creates a symlink under `~/.opencli/plugins/`. Your source files stay in your project directory, so edits and commits happen there.
This is the recommended path for custom commands you own long-term.
## Private adapters in `~/.opencli/clis`
Use a user adapter when you want the fastest local adapter loop and do not need a separate project directory.
```bash
opencli browser init cnn/top
# edit ~/.opencli/clis/cnn/top.js
opencli browser verify cnn/top
opencli cnn top
```
User adapters are loaded from:
```text
~/.opencli/clis/<site>/<command>.js
```
This path is convenient for quick local automation. For code you want to version, review, or share, prefer a plugin.
If the command takes required positional args and no fixture exists yet, seed the first verify run explicitly:
```bash
opencli browser verify instagram/collection-create --write-fixture --seed-args opencli-verify
opencli browser verify example/detail --write-fixture --seed-args '["https://example.com/item/1", "--limit", 3]'
```
`--seed-args` is only used when the fixture has no `args`. Once the fixture is written, `opencli browser verify` reads args from `~/.opencli/sites/<site>/verify/<command>.json`.
## Local overrides for official adapters
Use `adapter eject` when you want to customize an existing official adapter.
```bash
opencli adapter eject twitter
# edit ~/.opencli/clis/twitter/*.js
opencli adapter reset twitter
```
Files in `~/.opencli/clis/<site>/<command>.js` override packaged adapters with the same `site/command` on this machine. `opencli browser verify <site>/<command>` also runs the local override, so a passing local verify does not prove that the packaged adapter was changed.
The packaged `cli-manifest.json` only describes bundled adapters. User adapters are discovered at runtime and do not need manifest entries.
After copying a local fix into the repository for a PR, remove the local copy or run `opencli adapter reset <site>` after merge. Otherwise the local file keeps shadowing future package updates. `opencli doctor` warns when it detects this shadowing.
## Plugins for sharing commands
Plugins are third-party command packages. They can be installed from GitHub, any git-cloneable URL, or a local directory.
```bash
opencli plugin install github:user/opencli-plugin-my-tool
opencli plugin install https://github.com/user/opencli-plugin-my-tool
opencli plugin install file:///absolute/path/to/plugin
opencli plugin list
opencli plugin update --all
opencli plugin uninstall my-tool
```
Each plugin directory is scanned for `.ts` and `.js` command files. TypeScript plugins are transpiled during install.
See [Plugins](./plugins.md) for manifest fields, TypeScript examples, update behavior, and monorepo publishing.
## Multiple custom sites in one repo
For a Git-hosted plugin collection, declare sub-plugins in `opencli-plugin.json` and install from GitHub:
```json
{
"plugins": {
"cnn": { "path": "packages/cnn" },
"reuters": { "path": "packages/reuters" }
}
}
```
```bash
opencli plugin install github:user/opencli-plugins
opencli plugin install github:user/opencli-plugins/cnn
```
For local development, install each sub-plugin directory directly:
```bash
opencli plugin install file:///absolute/path/opencli-plugins/packages/cnn
opencli plugin install file:///absolute/path/opencli-plugins/packages/reuters
```
Local `file://` installs expect the target directory itself to be a valid plugin with command files. For a monorepo root, push it to GitHub and install it with the GitHub monorepo flow.
## External CLI passthrough
Use external CLI registration when the command already exists as a binary on your machine and you want it available through `opencli`.
```bash
opencli external register my-tool \
--binary my-tool \
--install "npm i -g my-tool" \
--desc "My internal CLI"
opencli my-tool --help
```
External CLIs pass stdio and exit codes through to the underlying binary.
+2 -1
View File
@@ -13,7 +13,7 @@ OpenCLI turns **any website** or **Electron app** into a command-line interface
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation** — `browser` gives AI agents direct browser control: click, type, extract, screenshot — fully scriptable.
- **Website → CLI** — Turn any website into a deterministic CLI: 87+ pre-built adapters, or author your own with the `opencli-adapter-author` skill.
- **Website → CLI** — Turn any website into a deterministic CLI: 100+ site surfaces are already registered, or author your own with the `opencli-adapter-author` skill.
- **Account-safe** — Reuses Chrome's logged-in state; your credentials never leave the browser.
- **AI Agent ready** — `opencli browser *` primitives (`open` / `network` / `state` / `eval` / `init` / `verify`) drive the adapter-authoring loop.
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
@@ -74,6 +74,7 @@ The completion includes:
- [Installation details](/guide/installation)
- [Browser Bridge setup](/guide/browser-bridge)
- [Extending OpenCLI — custom commands, plugins, and external CLIs](/guide/extending-opencli)
- [Plugins — extend with community adapters](/guide/plugins)
- [All available adapters](/adapters/)
- [For developers / AI agents](/developer/contributing)
+11 -1
View File
@@ -2,7 +2,7 @@
## Requirements
- **Node.js**: >= 21.0.0
- **Node.js**: >= 21.0.0, or **Bun** >= 1.0
- **Chrome** running and logged into the target site (for browser commands)
## Install via npm (Recommended)
@@ -31,6 +31,16 @@ npm install -g @jackwener/opencli@latest
npx skills add jackwener/opencli
```
Or refresh only the skills you actually use:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
## Verify Installation
```bash
+1 -1
View File
@@ -25,7 +25,7 @@ features:
details: Reuses Chrome's logged-in state. Your credentials never leave the browser — no tokens, no exposed passwords.
- icon: 🤖
title: AI Agent Ready
details: "explore discovers APIs, synthesize generates adapters, cascade finds auth strategies. Built for AI-first workflows."
details: "Browser primitives plus adapter-authoring skills give AI agents a repeatable loop for recon, extraction, verification, and adapter writing."
- icon: 💰
title: Zero LLM Cost
details: No tokens consumed at runtime. Run 10,000 times and pay nothing.

Some files were not shown because too many files have changed in this diff Show More