Structured AI-readable description of OpenCLI: what it does, key capabilities,
install instructions, supported sites, skills, and links. Helps AI search crawlers
(ChatGPT, Perplexity, Claude) accurately describe and cite this project.
Adds the no-navigation `quickCheck` to each adapter's
registerSiteAuthCommands config so `opencli auth status` (PR #1879) resolves
login state in quick mode (CDP getCookies, no per-site goto) instead of
reporting `unknown`.
- quickCheck reuses each adapter's existing poll cookie gate (has<Site>Cookie),
which is a logged-in-only, no-nav check returning boolean.
- Deliberately NOT wired (stay `unknown` in quick mode, available via --full):
- gitee/hf/deepseek/quark/reuters/zsxq: no reliable logged-in cookie; they
detect via no-nav fetch / localStorage / Bearer which need the site origin.
- doubao/ke/coupang/manus: session cookie is present for anonymous users, so a
cookie quickCheck would false-positive — `unknown` is more honest.
Live: auth status --site resolves logged_in/not_logged_in for cookie-gate sites
(v2ex/github/zhihu/claude/taobao/twitter/bilibili) in quick mode; excluded
sites report unknown. Audits new=0/new=0; suite 5054 passed.
Adds site login/whoami coverage for 55 additional auth adapters using the shared site-auth helper, including the final gitee/hf/v2ex/deepseek/quark batch and fixes from live validation.
Review follow-up:
- remove direct email output from ChatGPT/Grok/Gemini/Qwen whoami
- avoid DeepSeek email fallback as display name
- avoid Upwork first/last name output
- avoid leaking Boss wt2 session cookie as user_id
- rebase on latest main and regenerate cli-manifest.json
Validation:
- clean-HOME npm test: 5049 passed, 1 skipped
- npm run check:typed-error-lint: new=0
- npm run check:silent-column-drop: new=0
- npm run build
- npm run docs:build
- git diff --check
- dist list JSON smoke
- GitHub CI green
* fix(extension): close SW-restart race that spawns duplicate OpenCLI Adapter groups
User report: after Chrome MV3 SW dies between owned-window/group setup steps,
the next ensure cycle could spawn a second `OpenCLI Adapter` group and a second
owned window, leaving multiple windows each holding an untitled or duplicate group.
Three defects chain together:
1. `createOwnedGroupWithRollback` persisted state only after `tabGroups.update`,
so an SW crash between `tabs.group` and the title set left an empty-title
group with no persistent pointer.
2. `collectOwnedGroupCandidates` had three lookup paths (stored groupId / title
query / automationSessions) that all failed simultaneously after a cold SW
restart on a partially-built group.
3. `ensureOwnedContainerWindowUnlocked` persisted the new `windowId` only after
the full group setup, so an SW crash between `windows.create` and the next
`tabs.group` lost the window pointer and spawned a second owned window on
the next ensure.
Fixes:
- Persist `groupId` (and `windowId`) inside `createOwnedGroupWithRollback`
immediately after `chrome.tabs.group` returns, and drop the `tabs.ungroup`
rollback so `ensureCanonicalGroupTitle` can self-heal on the next cycle.
- Persist `windowId` inside `ensureOwnedContainerWindowUnlocked` immediately
after `chrome.windows.create` returns so the next ensure reuses the window
even if the worker dies before the first group is built.
- Add a 4th-layer scan in `collectOwnedGroupCandidates` over every tab group
in Chrome, filtering by empty title + per-role ownership-tab signal (the
group must contain a tab matching a still-registered owned session's
`preferredTabId`). User-built untitled groups never carry that signal, so
the hijack boundary from #1794/#1816 is preserved.
Tests cover the existing group-race contract plus three new regression gates:
window-race reuse after SW restart, orphan-group adoption via the ownership-tab
signal, and rejection of a user-built untitled group with no owned-tab signal.
* refactor(extension): rename createOwnedGroup to match post-rollback semantics
Both reviewers flagged that the function no longer ungroups on title-update
failure (Fix 1 dropped the rollback), so the -WithRollback suffix misled.
Pure rename, no behavior change.
* fix(daemon): SIGKILL fallback when stale daemon refuses graceful shutdown
When the CLI detects a stale daemon (`daemonVersion !== PKG_VERSION` after
`npm install -g @jackwener/opencli@latest`), it currently asks the daemon to
exit via `POST /shutdown` and waits up to 3 seconds for the port to release.
If the old daemon hangs, refuses /shutdown, or the endpoint is missing
entirely (pre-shutdown-endpoint version), the port stays held and the user
sees `Stale daemon could not be replaced` with a `opencli daemon stop` hint.
99% of "I just upgraded and have to run `opencli doctor` every time" reports
land here: the user upgraded the CLI but the persistent daemon survived from
a previous install, and graceful shutdown is unreliable across versions.
This patch reads the stale daemon's pid from its existing `/status` response
(daemon.ts:252 already exposes `pid: process.pid`) and falls back to
`process.kill(pid, 'SIGKILL')` after graceful shutdown fails, then waits
another 2s for the port to release. Cross-platform: Node's
`process.kill(_, 'SIGKILL')` maps to `TerminateProcess` on Windows, so no
`taskkill` shell-out is needed.
The user-visible "Stale daemon could not be replaced" error only fires when
both graceful shutdown AND SIGKILL fail (cross-user owner / cross-machine
PID — neither is reachable from a normal CLI invocation anyway). The hint
message is updated to reflect that.
Adds 2 tests:
- SIGKILL succeeds → bridge proceeds past the stale block (and eventually
fails the no-extension wait, proving the stale branch was passed cleanly).
- SIGKILL throws EPERM AND waitForDaemonStop still returns false → falls
through to the existing stale-daemon error.
Troubleshooting docs note the new auto-fallback.
* address opus review nits
- bridge.ts: move `await waitForDaemonStop(2000)` out of the try/catch so the
port poll always runs after `process.kill`, even when the kill itself throws
ESRCH (target already dead) or EPERM (cross-user owner).
- browser.test.ts: bump the existing 3 stale-daemon test fixtures from
`pid: 1` to `pid: 999999` so the new SIGKILL fallback no longer fires a
signal at init when the older tests reach the fallback path.
- browser.test.ts: mock `waitForDaemonStop` in those 3 tests too, since the
real implementation now polls for 2s in the fallback path (test runtime
was up to ~6s before; back to ~400ms).
* Revert "chore(skills): remove smart-search (#1683)"
This reverts commit 7a2ab47bf8.
* chore(readme): keep smart-search out of README per @WAWQAQ
Restore smart-search skill files and inner-docs refs, but drop the 6
README mentions (3 EN + 3 ZH). Skill is loadable via:
npx skills add jackwener/opencli --skill smart-search
but no longer surfaced on the README front page.
Scope reusable owned-container tab selection to canonical group membership so OpenCLI does not overwrite user http(s) tabs when an owned group converges into a user window.
Also hardens lower-probability fallback paths where the persisted owned window remains but the group signal is missing, and where owned-session fallback previously scanned the whole window.
Fixes#1760.
* docs(sitemap-author): schema v1.1 — 12 patches from twitter+hackernews PoC
Cross-validated against two PoCs (twitter 12 files / hackernews 10 files).
v1.1 changelog at top of file. 12 patches in 3 groups:
Group 1 — Scope/boundary (6 clarifications):
- §1.1 CJK token-per-char 30-50% higher than English; split sub-file rather
than relaxing 800-token limit (which would drift).
- §2.1 auth_strategy = primary strategy, not union; per-page contract_strength
expresses exceptions.
- §2.5 pitfalls.md is task-executor-level only; adapter-internal pitfalls
(queryId parsing, envelope unwrap) move to ~/.opencli/sites/<site>/notes.md.
- §2.5 pitfall id / trigger / workaround written from task-executor 1st-person
view ("when agent does X, ..."), not adapter-implementer view.
- §2.4 apis.md entry adds optional `notes:` field for GraphQL queryId path and
other meta info (still no URL / method / params / response — those stay in
endpoints.json).
- §2.2 page Linked APIs may be empty when endpoints.json is still being
collected; do not insert fake placeholder ids.
Group 2 — Reuse/compactness (3 structural):
- §2.2 + §4 partial pages: `page_id` with `_` prefix and `url_patterns: []`
for cross-page UI (e.g. _tweet_card.md). Referenced by other pages via the
existing `action:<id> in pages/_<name>.md` form. Eliminates duplication and
arbitrary "which page owns the like button" calls.
- §3 introduces Form B compact YAML for actions (~80 token each vs Form A
markdown ~250). Both forms remain valid; Form B is recommended when page
density would otherwise blow the 800-token budget.
- §3 drops action-level `verified_at` and `source` — file-level frontmatter
already covers both, repeated copies just drift.
Group 3 — Execution health/anchors (3 action-level):
- §3.3 cross-page UI primitive actions (the kind that live in partials)
may write Best/Fallback inline as adapter-first + DOM fallback within a
single action, rather than being forced up into a workflow Best/Fallback
pair. Decouples UI-primitive routing from task-level routing.
- §3.4 Recovery may include `adapter_health_update: <adapter> -> suspect`
directive. Consumption skill (opencli-browser-sitemap) writes the matching
workflow's adapter_health on the local overlay so the next agent skips the
broken Best path instead of re-running it. Write-side closure for the
failure → next-agent-avoidance loop.
- §2.2 testid marked optional; selector_pattern promoted to first-class
anchor with 5 acceptable shapes (id-anchored / sibling traversal / attribute
boundary / form name / ARIA) and explicit discouraged-anchor list
(nth-child, single-class grabs, text-content selectors). Old sites without
testid (HN, forums) are no longer second-class.
No code changes — pure schema reference. Both PoCs remain local; promotion to
references/site-memory/{twitter,hackernews}/sitemap/ comes once this lands.
* docs(sitemap-author): apply opencli-user review nits
- Form B delimiter table (`|` enum / `||` fallback / `;` sequential) to
disambiguate `do:` and `recover:` parsing.
- §3.3 like_tweet example updated to `||` fallback form.
- §3.4 explicit note: adapter_health recovery (suspect → healthy) is read
side, deferred to opencli-browser-sitemap skill spec.
* docs(sitemap): align skills with schema v1.1
* docs(sitemap-author): add detailed schema reference
Companion to #1820 — extends the inline schema in SKILL.md with the
field-level spec promised in the design thread:
- File schemas: SITE.md / pages/<id>.md / workflows/<id>.md / apis.md /
pitfalls.md with frontmatter fields and required sections.
- Action schema with all 6 required fields (preconditions, postconditions,
failure_signals, recovery, evidence, plus optional action-level
state_signature for multi-step internal re-entry).
- Workflow adapter_health enum (healthy / suspect / broken) backing the
Best path / Fallback path routing rule.
- apis.md endpoint reference format that points at endpoints.json by id
instead of duplicating endpoint detail (avoids double-stale).
- Two-layer overlay semantics (local wins, stable-id matching, draft
placement inside sitemap/ to remain discoverable, optional site-alias.json
for sitemap-without-adapter cases).
- Phase 2 validation rules: file size budget, cross-ref integrity,
reality check via opencli browser, forbidden-content scan.
- Cross-links to strategy-selection.md (contract_strength / auth_strategy
enums) and api-discovery.md.
SKILL.md gets a pointer to the new reference plus a draft-placement red
line so authors don't drop drafts at the parent dir where the browser
availability detection cannot see them.
* docs(sitemap): seed authoring from adapter traces
Companion deep reference for the SKILL.md strategy gate (#1809):
- New `references/strategy-selection.md` with contract-based ladder model,
empirical fixes/adapter-year data (837 adapters / 30-day window), Pattern A
judgment rules from `api_candidates` verdicts, and reference cases
(booking #1680, Twitter GraphQL, xhs signed URL, weread-official).
- Cross-link from SKILL.md inline strategy gate to the deep reference, plus
one-line empirical hook ("PAGE_FETCH/INTERCEPT 7-8x PUBLIC_API fix rate").
- coverage-matrix.md: Strategy row renamed to 6-enum (PUBLIC_API / COOKIE_API
/ UI_SELECTOR / DOM_STATE / PAGE_FETCH / INTERCEPT) with fixes/adapter-year
on each entry.
- site-recon.md: Pattern A note that hit alone is not a `PAGE_FETCH` signal —
must check `api_candidates` verdicts (booking #1680 reference).
Per WAWQAQ DM. OpenCLI is featured on Trendshift
(https://trendshift.io/repositories/23541) — surfacing the badge at
the top of README gives social proof to new visitors and links back
to the Trendshift listing.
Placement: above the `# OpenCLI` heading so it renders as a banner
before the title (standard Trendshift placement pattern). 250×55 inline
SVG. Both EN and ZH READMEs updated.
Decode rendered search-card title and author entities for reader URL matching while keeping output identity from the public API and preserving typed error behavior.
Harvest signed creator-note analyze pages in order with dedupe, unwrap Browser Bridge envelopes, and fail closed when known totals cannot be completely captured.
* fix(xiaohongshu): hook dashboard fetch to capture signed datacenter/note/* responses
The four /api/galaxy/creator/datacenter/note/* endpoints behind the
creator-note-detail view require an x-s / x-t / x-s-common signing
interceptor that the dashboard's own JS installs at page load. The
previous in-page roundtrip called fetch() directly from page.evaluate,
which bypasses the interceptor and gets HTTP 406, so 观看来源 / 观众画像 /
趋势数据 rows silently never landed even though the help string promised
them.
Instead of forging signatures, install a fetch + XHR capture hook on
window.__xhsCapture, SPA-navigate to /statistics/note-detail via
history.pushState + popstate (a hard page.goto would wipe the hook
before the first auto-fetch fires), and harvest the dashboard's own
signed responses out of the capture buffer.
Also fix a 1-character endpoint name: /note/audience -> /note/audience/source.
The old path returned 404 even when signed; the page actually fetches
/note/audience/source for the 观看来源 panel. Confirmed against the live
dashboard XHR list while logged in.
Tests updated to mock the new install-hook + SPA-nav + poll-capture
sequence at page.evaluate (the previous burst-wait-between-fetches
assertion no longer applies).
Closes#1728.
Reporter diagnosis: @ppop123 traced the signing bypass + endpoint typo
and verified the hook + SPA-nav workaround on 86 notes.
* test(xiaohongshu): trim installXhsFetchCaptureHook comment to match sibling tone
Sibling helper functions in creator-note-detail.js have no doc-comment
block above the declaration; the 5-line WHY block on the new hook was
out of style. Compress to two lines covering the same WHY (signed API
bypass + 406) and let the rest of the context live in the commit body
of the parent fix.
* test(xiaohongshu): name the creator-note-detail poll bounds
Inline literals (20 iteration cap, 0.5s wait) drift from sibling
convention in clis/xiaohongshu/delete-note.js where the same kind of
post-write polling is named VERIFY_TIMEOUT_MS / VERIFY_POLL_MS. Promote
the two values to CAPTURE_POLL_ATTEMPTS / CAPTURE_POLL_INTERVAL_S so
the loop reads against an explicit budget and future tuning lands in
one place.
* fix(xiaohongshu): address copilot review on creator-note-detail hook
Two polish items from the Copilot review on #1732:
- Buffer reset: window.__xhsCapture is now cleared on every install call
so stale captures from a previous run on the same tab cannot leak into
the current navigation's harvest. The wrapper-install guard moves to a
separate __xhsCaptureInstalled flag so the fetch/XHR monkey-patches
themselves are still installed exactly once per page lifetime.
- XHR static constants: HookedXHR now copies the readyState constants
(UNSENT / OPENED / HEADERS_RECEIVED / LOADING / DONE) from the original
constructor so dashboard code that reads XMLHttpRequest.DONE etc against
the constructor keeps working.
* fix(xhs): tighten note detail capture matching
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* test(download): retry media-download Windows tests to absorb runner cold-start variance
src/download/media-download.test.ts > 'keeps custom filenames inside the
output directory' timed out at the default 5000ms on CI run 26217100578
(Windows shard 2/2). The other two cases in the same describe block
completed in ~400ms, so the failure is cold-start cost of the first
http.createServer + downloadMedia roundtrip on a loaded GitHub Actions
Windows runner, not a logic regression.
Adopt the same { retry: process.platform === 'win32' ? 2 : 0 } describe
option that src/download/index.test.ts already uses for the same class
of Windows-only network/IO flake.
* test(download): trim media-download retry comment to match sibling tone
src/download/index.test.ts uses a 2-line comment for the same pattern.
The CI run id + redundant cross-reference belong in commit history, not
inline.
* fix(douyin/hashtag): validate per-action required args before the API call (#1689)
Closes#1689. Reporter @alexcc4 ran:
opencli douyin hashtag suggest --keyword 速效救心丸
which the previous code happily forwarded to:
GET creator.douyin.com/web/api/media/hashtag/rec/?cover_uri=&aid=1128
with an empty cover_uri, because the suggest action reads kwargs.cover
(not kwargs.keyword) and there was no upfront validation. The Douyin
server rejected the empty cover_uri with API error 5 (参数不合法),
which surfaces to the user as an opaque server-side error rather than
the obvious adapter-side mismatch.
Fix: validate each action's required args up front and throw
ArgumentError with a concrete hint pointing the user at the right
action / flag combination:
- search requires --keyword (suggest the example command)
- suggest requires --cover (explain it operates on an uploaded video
cover, not a keyword; redirect keyword-search users to `hashtag
search --keyword <词>`)
- hot still accepts an empty --keyword (it is optional for hot)
Also tightened the arg help strings to make the per-action
requirements obvious without reading the source.
Tests: 5 new vitest cases covering the validation branches plus URL
shape assertions for search / suggest / hot.
Live verified the reporter's exact failing command now surfaces:
$ node ./dist/src/main.js douyin hashtag suggest --keyword 速效救心丸
ok: false
error:
code: ARGUMENT
message: douyin hashtag suggest 需要 --cover <cover_uri>
help: suggest 基于已上传的视频封面做 AI 推荐, 不是关键词搜索.
关键词搜索请用 `douyin hashtag search --keyword <词>`.
exitCode: 2
Zero network calls on the invalid invocation.
* fix(douyin/hashtag): harden adapter boundaries with drift guards
API response shape is now validated before mapping. requireListField
throws CommandExecutionError when the batch payload is non-object or the
expected list field (challenge_list / hashtag_list / hotspot_list /
all_sentences) is the wrong shape. search additionally throws when the
API returns challenges but none have stable challenge_info, which would
otherwise silently flatten to an empty row set and mask upstream drift.
Live re-verified: search missing keyword and suggest missing cover still
throw ArgumentError with the same redirect hint (#1689 fix intact);
hot happy path still returns name / id / view_count rows.
* fix(douyin/hashtag): validate action args before navigation
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(twitter): detect private likes / following empty-timeline shape
When the X GraphQL endpoint returns `result.timeline = {}` (an empty
object with no nested `timeline.timeline.instructions`), the twitter
likes / following parsers correctly extracted 0 entries but the likes
caller silently returned `[]` while the following caller threw a generic
"no following accounts found" message. Both paths hide a platform
constraint: X made Likes private by default in mid-2024 and accounts
can also hide their following list.
likes.js now throws EmptyResultError with a privacy hint when the
empty-timeline shape is detected, and unconditionally throws when zero
tweets accumulate (parity with following.js, which already failed
loudly). following.js threads the same detector so the generic
EmptyResultError gains a privacy hint when the platform shape matches.
The detector is exported as looksLikePrivate{Likes,Following}Response
for unit testing and lives alongside the existing pure parsers.
Live-verified against simonw (private likes) and karpathy (public
following): likes now reports the privacy reason instead of returning
an empty list, and following continues to return its public dataset.
Closes#1701 (narrow root cause: the issue reporter's hot-patch is
defensive but their stale-queryId / dropped-args / off-by-one .data
diagnosis does not reproduce on main; the actual reproducible failure
is the silent-empty-timeline path documented here).
* fix(twitter): consolidate private-timeline detector + refresh stale queryId fallbacks + harden followers DOM
Followups on the same #1701 surface area.
Consolidation: the private-timeline detector duplicated between likes.js
and following.js moves to shared.js as looksLikePrivateTwitterTimeline,
and its unit tests collapse from two suites into one in shared.test.js.
Stale queryId fallbacks: live-extracted the current operationName to
queryId mappings from the X bundle (Following, UserByScreenName, Likes,
Followers) and refreshed the defensive fallback constants across
following.js, likes.js, list-add.js, list-remove.js, profile.js. The
dynamic resolver in resolveTwitterQueryId() succeeds in practice (it
parses queryIds from document.scripts text in-page, which is same-origin
and CORS-immune), so these fallbacks are last-resort only, but keeping
them current narrows the blast radius if the bundle parser ever fails.
followers.js Array guard: extractFollowersFromDOM returns whatever
page.evaluate produces, which under transient bridge errors can be
undefined. The subsequent followers.filter(...) call would then surface
as "filter is not a function". The fix coerces non-array results to []
so the loop drains via its existing sameCount break and ends with the
typed EmptyResultError.
Live-reverified all 4 paths on main: likes simonw still emits the new
private-likes hint, following karpathy / followers karpathy still return
data, and profile karpathy resolves under the bumped UserByScreenName
fallback.
Refs #1701. The remaining items in the issue (page.evaluate args drop,
parseFollowing off-by-one .data, twitter followers throwing "filter is
not a function" as a primary failure) do not reproduce on main:
src/browser/utils.ts serializes fn-args via JSON.stringify and
src/browser/utils.test.ts covers it; unwrapBrowserResult only strips
when a session field is present so the GraphQL .data path is correct
(confirmed by debug-dumping the live response shape); followers
returned data for every account I tested. The defensive Array guard
above closes the only plausible code path to that filter error.
* fix(twitter): match sibling EmptyResultError prose style
Single-sentence parenthetical aside on the private-timeline messages
(mirroring 'Account may be private, suspended, or have no media posts'
in twitter/download.js) instead of two-sentence prose, and drops the
trailing period that the dominant sibling no-period convention does not
use.
* fix(twitter): keep private timeline and malformed rows distinct
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Stabilize download progress byte formatting for invalid, negative, sub-byte, and very large values without changing download state or progress behavior.
Per WAWQAQ DMs:
1. The README stated "Node.js >= 21" in 6 places, but the actual
runtime floor is 20 (`MIN_SUPPORTED_NODE_MAJOR = 20` in
src/runtime-detect.ts, `engines.node: ">=20.0.0"` in package.json,
undici pinned to 6.x in 1.8.0 to keep Node 20 compatibility).
Stale carryover from before PR #1518/#1524 lowered the floor.
All 6 mentions (3 EN, 3 ZH) corrected to 20.
2. Prerequisites section was redundant with Quick Start (Node version
is in step 1 "Install OpenCLI"; Chrome/login state is in step 2
"Install Browser Bridge Extension" + step 3 "Verify"). Removed in
both EN and ZH.
* fix(extension): serialize tab group creation to prevent duplicates (fixes#1692)
Add per-role groupPromise serialization to ensureOwnedContainerTabGroup(),
preventing concurrent callers from each creating a new tab group when they
simultaneously observe no existing group.
The fix mirrors the existing promise serialization pattern used by
ensureOwnedContainerWindow(). When a second caller arrives while group
creation is in-flight, it awaits the first call's promise, then finds the
newly created group via the existing getOwnedContainerGroupId() cache path.
* test(extension): cover concurrent tab group creation
* fix(extension): queue tab group serialization waiters
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu/download): preserve carousel order via __INITIAL_STATE__.imageList (#1514)
Closes#1514. Reporter Scofy0123 observed that `opencli xiaohongshu
download` was saving carousel images in a different order from the
order shown on the platform: the visible cover ended up as `_2.jpg`
instead of `_1.jpg`.
Root cause: the IIFE collected images by iterating multiple DOM
selectors (`.swiper-slide img`, `.carousel-image img`, ...) into a
`Set`, then appended that set to `result.media`. JS `Set` preserves
insertion order, but the insertion order is whatever the selector
walk hit first; hidden / preloaded / duplicated / lazy-rendered
slides therefore shifted the saved order away from the canonical
display order. Downstream `downloadMedia` then named files by index
(`<id>_1.jpg`, `<id>_2.jpg`, ...), so the mismatched array order
produced mismatched filenames.
Fix mirrors the video extraction strategy already in this same IIFE:
read the canonical media list from the SSR hydration data first,
fall back to DOM scraping only when the structured state is absent.
- Method 1 (new): walk `window.__INITIAL_STATE__.note.noteDetailMap[id].note.imageList`
in array order. Each entry exposes the canonical CDN URL via
`urlDefault` (primary), with `urlPre` / `url` / `infoList.WB_DFT` /
`infoList[0]` fallbacks for older shapes.
- Method 2 (kept as fallback): the previous multi-selector DOM walk,
reached only when Method 1 yields zero images. Preview pages
without full SSR hydration still surface something instead of an
empty `media` array.
Shared `normalizeImageUrl` helper hoisted out of the inline `.add`
call so both paths apply the same query-string + imageView-resize
strip.
The rednote adapter reuses `buildDownloadExtractJs` verbatim, so this
PR fixes rednote download in the same change.
Tests: 7 new regression tests in `download.test.js` exercise the IIFE
directly via JSDOM (matching the `ctrip buildFlightExtractJs (JSDOM)`
pattern already in the repo):
- canonical order from `imageList` overrides DOM discovery order
(the exact #1514 repro)
- field fallback chain (urlDefault -> urlPre -> url -> infoList.WB_DFT
-> infoList[0])
- query-string + imageView-resize stripping
- DOM fallback engaged when imageList is missing
- non-xhscdn / non-xiaohongshu / non-rednote URLs filtered out
- DOM fallback NOT engaged when Method 1 yielded any image (no
duplicate-from-DOM contamination)
- video extraction still works alongside the image fix
All 12 download tests pass. No live xiaohongshu.com calls made
(pure JSDOM unit tests, respecting the platform's rate-limit
sensitivity).
* fix(xiaohongshu): keep video download order
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ DM:
1. **CLI Hub**: bare-name enumeration ("ntn", "discord") didn't tell
readers what those binaries map to. Switched to the `opencli external
list` brand-alias format: `ntn(notion)`, `discord(discord-cli)`,
`dws(DingTalk Workspace)`, `wecom-cli(企业微信)`, `tg(tg-cli)`,
`wx(wx-cli)`. Names that are already self-explanatory (gh / docker /
vercel / wrangler / obsidian / longbridge / lark-cli) stay bare.
2. **Exit Codes**: the 9-row table + example block was disproportionate
for a README. Compressed to one sentence with the 7 actionable codes
inline, full table relocated to:
- EN: `docs/guide/exit-codes.md` (new)
- ZH: `docs/zh/guide/exit-codes.md` (new)
Per WAWQAQ: from-source install instructions are infrastructure detail
that don't belong in a public-facing README. Contributors finding
themselves in this repo will already know `npm install / build / link`
patterns; users who reach the README from npm don't need them.
Removed in both EN and ZH.
* chore(release): 1.8.0
Substantial release: weread-official adapter, wider LinkedIn / Twitter / Reddit / Zhihu coverage, 12306 / Suno / Xianyu additions, security and reliability fixes, plus a 20% README shrink.
* chore: remove orphan docs/adapters-doc/ones.md
The file was a leftover from PR #386 (2026-04-10) and has been
superseded by docs/adapters/browser/ones.md. Bundled into the 1.8.0
release commit chain so the release doesn't ship with a dead docs
file alongside the new docs.
Skipped from opus-reviewer's audit (Tier 1 #1-#4) for this release:
- #1 smart-search dead refs (10 spots) — owned by @codex-coder's
skill-deletion PR; release PR will rebase on top of it.
- #3 clis/test-utils.js relocation — touches 19 importers, separate
refactor PR.
- #4 clis/slock/ orphan — needs WAWQAQ design call.
- #6 opencli-usage:161 wording — current "Commands that used to
exist" framing is already clear enough.
- #7 docs/adapters/index.md sync (8-20 missing sites) — broader docs
PR, not release-time bundling.
* chore: remove clis/slock + sync docs/adapters/index.md (audit #4 + #7)
Per WAWQAQ post-audit directive on #OpenCLI:f046ece7:
- `clis/slock/` was a half-finished orphan with only `_utils.js` and no
command entry points. Removed.
- `docs/adapters/index.md` was missing 11 browser adapters: 12306,
suno, weread-official, qwen, 1point3acres, brave, duckduckgo, cnki,
flomo, jianyu, taobao. Added all with commands sourced from
cli-manifest.json. Desktop section already covered all 7 desktop
adapters (Cursor / Codex / Antigravity / ChatGPT App / ChatWise /
Discord / Doubao App).
* feat(booking): add search adapter for Booking.com hotel listings
New `opencli booking search <destination> --checkin --checkout` adapter
scrapes the server-rendered hotel cards on www.booking.com via stable
`[data-testid=property-card]` selectors. No login required (Strategy.PUBLIC
+ browser:true).
Highlights
- 12 columns: rank, name, country, slug, star_rating, review_score,
review_count, price_amount, price_currency, distance, recommended_room,
url. `slug` + URL stay stable across locales (better round-trip key
than `name`, which Booking sometimes localizes from session cookies).
- Score parser anchors on `(\d{1,2})\.(\d)` so the duplicated "8.68.6" /
"评分8.68.6很棒" rendering doesn't mis-parse to 8.68.
- Currency symbol → ISO 4217 map (US$/€/£/¥/￥/₹/₩/HK$/A$/NT$/S$/CN¥);
honor `--currency` URL param for stable codes.
- Pagination via `--offset` (Booking pages 25/request); `rank` includes
the offset so paginated calls stay sortable.
- Captcha-page detection short-circuits to CommandExecutionError instead
of silent empty rows.
Typed errors (no silent clamp / fallback)
- Bad date / out-of-range adults/rooms/children/limit/offset / unknown lang
/ malformed currency → ArgumentError up front (before any navigation).
- Browser nav failure → CommandExecutionError.
- Zero cards rendered → EmptyResultError with a hint.
- Captcha page → CommandExecutionError.
29 unit tests cover the helpers, the registry shape, every typed-error
path, the {session,data} CDP envelope unwrap, and offset-aware rank
numbering. Silent-column-drop + typed-error-lint audits unchanged.
Live-verified against Tokyo + Paris.
* fix(booking): harden search parser boundaries
* fix(booking): separate no-card drift from empty
Per WAWQAQ:
1. **CLI Hub**: drop the 13-row 3-column table; enumerate just the
names inline ("gh · docker · vercel · wrangler · ntn · obsidian · …")
plus one-liner register / list commands. Removes "Manual install"
ntn note (search lives in external-clis.yaml / ntn's own docs).
Compresses the 7-row Desktop App Adapters table to a single inline
line pointing at docs/adapters/desktop/.
2. **Core Concepts** section dissolved: its four subsections
("browser", "Built-in adapters", "Writing a new adapter",
"CLI Hub and desktop adapters") duplicated the intro 3-bullet
+ later dedicated sections. Kept the substantive "Writing a new
adapter" callout as its own top-level section. The "For AI Agents
(Developer Guide)" tail block at the bottom was a third copy of
the same recipe — removed.
3. **Update** merged with **Install skills**: install header now
reads "Install skills (also refreshes existing installs)", and
the standalone Update section collapses to a single command
(`npm install -g @jackwener/opencli@latest && npx skills add ...`).
Net: EN 410 → 326 (-20%), ZH 455 → 366 (-20%). Same coverage; just
less repetition.
* feat(linkedin): add people-search command (#1621)
Closes#1621. Adds opencli linkedin people-search <keywords> for
finding people on standard LinkedIn (not Sales Navigator).
Architecture note. Standard LinkedIn moved its people search results
page to Server-Driven UI / React Server Components on the
/flagship-web/rsc-action/... path stack. The legacy Voyager REST
endpoint /voyager/api/search/dash/clusters returns HTTP 500 from a
web context; its modern camelCase rename voyagerSearchDashClusters
returns the same. The result list is rendered server-side and the
page HTML IS the result payload; Voyager calls from the page are
sidebar / notification concerns, not search results.
Extraction strategy. LinkedIn SSR uses obfuscated CSS class hashes
(e.g. _997b7c77) that rotate on every deploy AND display:contents
wrappers that flatten the DOM tree. Class-based selectors, walk-up-
to-card logic, and anchor-pair element ranges all fail because no
element boundary matches a person's card.
Working approach: extract main.innerText once, split by newline,
slice between consecutive person names. The names come from the
aria-hidden spans of /in/<handle> anchors. LinkedIn's SSR emits a
card as a name line followed by degree badge / headline / location
/ action labels before the next card's name line - a layout that
has been stable through several DOM refactors.
Critical filter: /in/<handle> anchors over-count because LinkedIn
renders each mutual connection as a /in/ anchor inside another
card's result. The skip() predicate during name-line lookup drops
mutual-connection lines ("X, Y and N other mutual connections"), so
anchors that don't have a real name line are filtered out.
CUL caveat. LinkedIn imposes a monthly Commercial Use Limit on
people search against the standard site. Burst behaviour is
irrelevant - the limit is a calendar-month counter. The adapter
runs one navigation per invocation (no pagination) so a single call
costs exactly one CUL query. --limit is capped at 10 to keep a
single call's information density high without surfacing the
"reached commercial use limit" yellow banner faster.
Schema:
rank, name, headline, location, profile_url
Live verified against kyfw 12306-style throttled cadence (sleep 60s
between dev iterations to keep CUL consumption visible): 5/5 rows
populated with name + headline + location + profile_url for the
keyword "reinforcement learning". Mutual-connection anchors
correctly filtered out so the row order matches LinkedIn's own
ranking.
Tests: 10 unit tests covering URL construction, limit validation,
extraction-script invariants (anchor enumeration, text-slice
approach, mutual-connection filter, aria-hidden span as name source),
limit slicing, AuthRequiredError on missing JSESSIONID, CUL-
flavoured CommandExecutionError on redirect, EmptyResultError on
zero rows, ArgumentError on empty keywords, and registry shape.
* fix(linkedin): harden people search typed boundaries
* fix(linkedin): fail people search candidate parser drift
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(adapters): migrate empty-data throws to EmptyResultError across 5 commands (#1674 follow-up)
Continues the structured-error migration owner started in #1674
(fix(xhs,youtube): 把合法空数据语义切到 EmptyResultError). Same
motivation: callers need to distinguish "the platform legitimately
has no data for this target" from "fetch infrastructure is broken,
retry me", because downstream automation pipelines that batch over
seed lists conflate the two and trip soft-rate-limit heuristics.
Sites converted (5 commands, 6 throw sites):
powerchina/search.js (2 sites):
- "[taxonomy=empty_result] ... extracted only navigation/portal rows"
- "[taxonomy=empty_result] ... api/dom yielded no result"
Both already self-labelled with the empty_result taxonomy tag,
making this the canonical fix.
xiaohongshu/creator-notes.js, creator-notes-summary.js (both):
- "No notes found. Are you logged into creator.xiaohongshu.com?"
The "is logged in" hint is preserved in the empty message so users
can self-diagnose, while the error type is now structured.
xiaohongshu/creator-stats.js:
- "No data for period <X>. Available: <a, b, c>"
Empty-data condition: requested period exists in the API surface
but has zero numeric data; available periods are still surfaced
in the message.
xiaohongshu/creator-note-detail.js:
- "No note detail data found. Check note_id and login status..."
Shape: exit code 66, stderr code: EMPTY_RESULT, matching
bilibili/subtitle, xhs/user, youtube/transcript precedent.
Out of scope:
- tiktok/{user,notifications,explore}.js: throws live inside
page.evaluate template strings and run in browser context; the
Node-side caller already regex-routes them via
throwTikTokPageContextError({emptyPattern: /No videos found/, ...})
to EmptyResultError. The existing design is correct.
- eastmoney/_secid.js / antigravity/serve.js / instagram/collection-*:
input-validation throws, ArgumentError territory not EmptyResultError.
* test(adapters): cover empty-result migrations
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ:
1. Built-in Commands table cut from 30 EN rows / 86 ZH rows down to a
curated 11-site list (xiaohongshu, bilibili, zhihu, hackernews,
linkedin, reddit, twitter, claude, gemini, notebooklm, amazon).
The README is meant to surface high-traffic / well-known sites;
the long-tail (100+ adapters) is one click away via
docs/adapters/index.md. linkedin (full) replaces linkedin-learning
in the curated set per the spec.
2. Add Cloudflare Wrangler as a new external CLI passthrough:
- src/external-clis.yaml entry (binary: wrangler, npm -g)
- CLI Hub table row in EN + ZH READMEs
- cli-manifest.json regen reflects the new entry (857 entries)
* feat(twitter): add device-follow command for /i/timeline notification stream (#1628)
Closes#1628. Adds the twitter device-follow command, which reads the
curated tweet list aggregated under a bell-icon "new posts from @userA
and N others" notification. Direct GET /i/timeline redirects to /home,
so the data is only reachable via the legacy v1.1 REST endpoint
/i/api/2/notifications/device_follow.json , none of the existing
twitter commands cover this stream:
- twitter timeline home for-you / following feed (different endpoint)
- twitter notifications the notification list itself, not aggregated
tweets inside any one notification
- twitter search search-based, can't reproduce the aggregation
Endpoint discovery + field-mapping originally proposed by @traddo in
#1628; this PR upstreams a clean implementation that:
- Strategy.COOKIE + ct0 from CDP cookie jar + the public web bearer
token from clis/twitter/utils.js (same auth path as twitter timeline)
- Hits /i/api/2/notifications/device_follow.json directly via
page.evaluate fetch on the x.com origin so SameSite=Lax cookies are
preserved
- Joins each entry.content.item.content.tweet.id to
globalObjects.tweets[id] and resolves the author via
globalObjects.users[tweet.user_id_str]
- Returns the canonical twitter row columns (id, author, text, likes,
retweets, replies, views, created_at, url), matching twitter timeline
minus has_media / media_urls / card / quoted_tweet which the legacy
v1.1 endpoint does not surface
- Sets views: null rather than a 0 sentinel; the legacy endpoint does
not return view counts even with include_ext_views=true, and the
GraphQL TweetResultByRestId round-trip per tweet was judged too
expensive for a list command (typed-errors §3: no scalar sentinels
that lie about real engagement)
- parseLimit enforces strict 1-200 integer validation with no silent
clamping; the only baseline addition is the silent-sentinel on the
"unknown" author fallback, which matches the exact precedent in
twitter/timeline.js:76 that is already baselined
Tests: 17 unit tests in device-follow.test.js cover parseLimit strict
validation, URL parameter shape, entry/tweet join, user-resolution
fallback, dedup via the seen set, empty-stream shape, the canonical
column registration, AuthRequiredError on missing ct0, and
CommandExecutionError on non-2xx fetch.
Live verified the endpoint shape end-to-end against the logged-in
session: HTTP 200 with the expected
{globalObjects: {tweets, users}, timeline: {id: 'tweet_notifications',
instructions: [{addEntries: {entries: []}}]}} envelope. The tester
account has no bell-notification follows enabled, so entries is empty,
but the shape and auth path are confirmed against the documented
spec.
* fix(twitter): harden device-follow typed boundaries
* fix(twitter): fail fast on device-follow drift
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(twitter): expose quoted_tweet on read commands
When a tweet quotes another tweet (embedded preview with commentary), the
quoted tweet's content is in `tweet.quoted_status_result.result` — same
`legacy / core / card / note_tweet` shape as the outer tweet. Until now
none of the 5 read commands (list-tweets / timeline / thread / tweets /
search) surfaced this nested object, so downstream consumers couldn't
render the quoted preview card.
Adds `extractQuotedTweet(tw)` in shared.js (mirrors the
`extractMedia` / `extractCard` helper pattern) and threads it through
all 5 read commands plus their CLI `columns:` declarations.
Output shape is a deliberately small subset of the main tweet
(id/author/name/text/created_at/url + media + card). Counts and full
author bio are intentionally omitted to keep timeline payloads from
ballooning 2-3x; consumers needing those can re-fetch
`twitter thread <quoted_id>`.
Notable edge cases tested in shared.test.js:
- plain tweets (no `is_quote_status`) -> null
- tombstoned / unavailable quoted tweets (deleted / privacy-restricted) -> null
- TweetWithVisibilityResults `result.tweet` shim unwrap
- long-form note_tweet text preferred over truncated full_text
- quote-of-a-quote does NOT recurse (avoids payload explosion on threads
where every reply re-quotes the root)
* fix(twitter): require quoted tweet render evidence
* fix(twitter): validate quoted tweet author shape
---------
Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ T1 + T2 review:
T1 — skill attribution carries the same intent PR #1654 started but
hadn't fully cleaned up:
- Skill table row for `opencli-adapter-author` no longer claims it
"operate[s] a site in real time" (SKILL.md explicitly says ad-hoc
driving lives in `opencli-browser`). Browser-op example
("Help me check my Xiaohongshu notifications") moved to the
`opencli-browser` row where it belongs.
- "How it works" section's 5 browser primitives (navigate / read /
interact / extract / wait) now point to `opencli-browser` instead
of `opencli-adapter-author`.
- Skill references list re-orders to surface `opencli-browser` first
with a concrete description, and `opencli-adapter-author` no longer
claims to cover "browser operation".
T2 — drop the Highlights section. Pre-Quick-Start had four parallel
summary blocks (3-line tagline / 3-bullet automation intro / CLI-hub
+ desktop line / 5-bullet Highlights) that all said the same thing.
Highlights was the most-recent and most-redundant of the four; the
remaining three carry the value props cleanly: tagline → three usage
modes → CLI-hub + desktop scope.
EN + ZH READMEs synced.
* feat(linkedin-learning): add search / trending / course read commands (#1021)
Closes#1021. Adds a new linkedin-learning site adapter with three
read-only commands against LinkedIn Learning's public learning-api
REST surface. Shares cookie session with linkedin.com; Learning
queries are not subject to the people-search CUL.
Commands:
- linkedin-learning search <keywords> searchV2?q=keywords
- linkedin-learning trending feedRecommendationGroups?q=learner
- linkedin-learning course <slug> courses?q=slug
Endpoints were discovered via browser network capture on
/learning/search and /learning/<slug> pages: searchV2 returns a flat
list of courses/videos/paths keyed by entityType, headline.title.text
holds the canonical title, length is a TimeSpan in seconds, and rating
is averaged from ratingSum/ratingCount when averageRating is missing.
trending walks the carousels array on each recommendation group, flattens
cards across them, dedups by slug, and respects --limit. Group is
labeled with the carousel title (e.g. "Top picks for you") or the
upstream annotation tag (TOP_PICKS).
course accepts either a bare slug or a full /learning/<slug> URL, then
hits /learning-api/courses?q=slug. The detail endpoint omits rating
fields even when search reports them; this is documented in the
adapter doc rather than fixed via a second /reviews fetch to keep the
PR scoped to one endpoint per command.
CUL caveat: Learning's API has no per-month limit, so dev iterations
can be much more aggressive than the people-search adapter (#1649).
Three commands were live-verified against a logged-in account with
60s sleeps between calls (conservative for first-pass safety).
Tests: 28 unit tests across search.test.js (12), trending.test.js (6),
course.test.js (10) cover URL construction, limit validation, author
join, duration / rating coercion, row mapping, carousel flattening
and dedup, slug parsing from URL forms, and the standard auth /
empty / fetch-failure error paths.
Live verified:
- search "AI agent" --limit 3: 3 rows with title/instructor/rating
- trending --limit 3: 3 personalized course picks
- course agentic-ai-build-your-first-agentic-ai-system: title, 3932s
duration, 18 videos, release date 2026-03-27
* fix(linkedin-learning): harden read result boundaries
* fix(linkedin-learning): require course title evidence
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(cli): escape leading-dash positional values via argv preprocessor (#1160)
Closes#1160. `opencli boss detail -abc123def` failed with
`error: unknown option '-abc123def'` because commander treats any
argv token starting with `-` as an option. BOSS 直聘 securityId
tokens are opaque base64-ish strings that can legitimately start
with `-`, and the same shape is possible for any adapter that takes
an opaque-id positional.
Adds escapeLeadingDashPositional() to src/cli-argv-preprocess.ts,
called from main.ts after the existing rewriteBrowserArgv pass. The
preprocessor:
- Reads cli-manifest.json (the same manifest the registry uses) and
builds a set of `<site>/<cmd>` keys whose first positional is
required.
- Walks past root flags (matching the existing rewriteBrowserArgv
walker) to find the site + command tokens.
- If the next argv token starts with `-`, is not the recognised
short flags `-f` / `-v` / `-h`, is not `--*`, and is not the
pre-escaped `--` separator, inserts `--` before it.
Tests: 12 new unit tests in cli-argv-preprocess.test.ts cover the
basic insertion, trailing-flag preservation, non-touched cases
(normal values, recognised short flags, long flags, already-escaped,
non-positional commands, unknown commands, short argv, and the
`--profile work boss detail -abc` form that walks past a root
value flag).
Live verified: `node ./dist/src/main.js boss detail -abc123def`
no longer raises 'unknown option'. The adapter now receives the
dash-leading value and proceeds to fetch, where it correctly
surfaces an upstream "missing required parameter" error for the
fake id used in this smoke test.
* fix(cli): preserve options around dash positionals
* fix(cli): preserve attached short option values
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(twitter): expose card binding_values on read commands
Surface tweet link-preview cards (title, description, image, domain, landing URL)
on `search`, `list-tweets`, `thread`, and `timeline` so downstream renderers
can build native-style link cards without re-fetching. Pure GraphQL-response
extractor — no query strategy, interceptor, or network changes.
extractCard returns null when the tweet has no card or when the card is
structurally empty (no url AND no title/description). Missing fields are
omitted from the output to keep JSON consumers clean.
* fix(twitter): bind cards to matching URL entity
---------
Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(twitter): add list-create command
Adds a new `twitter list-create` command so users can create Twitter/X
lists from the CLI (the existing list commands only covered reading,
adding, and removing members). Uses the GraphQL CreateList mutation
with the same cookie + CSRF pattern as list-add, no UI clicks needed.
Args: name (positional, max 25), --description (max 100), --mode (public|private).
QueryId resolved at runtime via resolveTwitterQueryId, with a known
fallback for offline / bundle-scan misses.
* fix(twitter): pin list-create queryId + features to a working pair
Twitter's GraphQL rejects CreateList when queryId and the features
schema drift apart (DecodeException). Stop resolving the queryId
dynamically (which would pull a newer schema), hardcode a known-good
queryId, and trim features to the minimal set the real web client
sends.
Also: Twitter sometimes returns a non-fatal errors array from a
side-effect serializer while still creating the list. Check for a
valid list payload first and only treat errors as fatal when no
list came back.
* fix(twitter): add missing access:'write' on list-create (#9)
`twitter/list-create` was missing the required `access` field, which made
manifest validation fail on every opencli invocation and spam stderr with:
⚠ Failed to load manifest .../cli-manifest.json: Command
twitter/list-create must declare access: 'read' | 'write'
Per docs/conventions/convention-audit.md (rule missing-access-metadata),
every adapter command must declare access. Since list-create is a create
action, set access: 'write'.
Also rebuilds cli-manifest.json — picks up missing `quoted_tweet` columns
on list-tweets / search / list-tweets-username from PR #8 (which didn't
rebuild the manifest).
* fix(twitter): harden list-create mutation contract
* fix(twitter): verify created list name
---------
Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: Kary <karyhe1019@gmail.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(reddit): subscribed command + expose id/created_utc/selftext on listing commands
Adds `opencli reddit subscribed` to list the user's subscribed subreddits,
mirroring `saved.js`'s cookie auth + AuthRequiredError pattern. Auto-paginates
via `/subreddits/mine/subscriptions.json` (max 1000 subs, default 100).
Also extends the JSON output of `popular` / `search` / `subreddit` with
`id`, `created_utc`, `selftext` (and `author` on popular) — the table
view stays clean (columns: unchanged), but `--format json` now surfaces
fields needed for downstream content-recommendation tooling that filters
by post age, dedupes by post id, or uses self-post bodies for embeddings.
Tests: 4 new vitest cases for subscribed.js (happy / auth fail / HTTP /
--limit truncation). All existing reddit tests still pass.
Note on cli-manifest.json diff: the rebuild on fork/main drops 13 entries
whose source files import lowercase `selectorError` from
`@jackwener/opencli/errors` (the actual export is `SelectorError` —
casing bug pre-existing in fork/main). Not introduced by this PR.
* fix(reddit): harden subscribed listing contract
* fix(reddit): require subreddit identity for subscriptions
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(adapters): drop silent-sentinel row fallbacks across Apple Podcasts, Reddit, and Gitee
Continues the audit-baseline cleanup from #1611 (lesswrong) and #1631
(wikipedia / 36kr / xiaoyuzhou / zhihu), and follows the direction set
by 71646158 (silent-empty-fallback resolutions across Douyin / Jike /
WeRead) and ee54eb8e (ignore sentinels in thrown errors).
Replaces silent-sentinel row fallbacks with the empty-string signal so
agents can tell apart "field has value Unknown" from "upstream returned
no value":
- apple-podcasts/search: episodes, genre
- reddit/saved: title
- reddit/upvoted: title
- gitee/search: language, description
All four files audited for downstream sentinel checks via
`grep -nE "=== ?['\"](Unknown|unknown|-)['\"]"`. None reference the
swapped values in control flow (verified against the v2ex/me.js class
of regression caught in #1631).
Intentionally skipped in this batch (will not flip to empty):
- gitee/trending.js:272: downstream `project.description !== '-'`
check drives the mergedDescription fallback. Same control-flow
sentinel pattern as v2ex/me.js. Stays on baseline.
- web/read.js x4: `'-'` lives inside rendered diagnostic lines
(`lines.push(...)`), not row fields. Empty would render
` GET /a/b` with a doubled space. UX placeholder.
- yollomi/{edit,video}.js x6: `file: '-'`, `size: '-'`, `credits: '-'`
are user-facing status rows displayed to humans. Empty would
collapse columns visually.
- zsxq/dynamics.js: `title: '[${d.action || 'unknown'}]'` is a
template-literal-rendered title prefix. Empty would render `[]`.
Verified live: `opencli apple-podcasts search "lex fridman" --limit 2`
returns populated episodes/genre. `opencli gitee search "vue" --limit 2`
returns populated language/description. Baseline shrinks accordingly.
* test(adapters): add empty-signal coverage for the cluster-3 sentinel swap
Mirrors the cluster-2 test additions, pairing the sentinel value swap
in this PR with focused unit tests that mock the upstream to return
null / missing fields and assert the row surfaces an empty-string
signal instead of the old fabricated '-' / 'unknown' sentinel.
Coverage:
- clis/apple-podcasts/commands.test.js (+1 case): stubs the iTunes
Search response with a result that has collectionId / collectionName
/ artistName populated but no trackCount and no primaryGenreName.
Asserts episodes and genre render as '' (was '-' before this PR).
- clis/gitee/search.test.js (new): mocks Gitee's `so.gitee.com/v1/search`
fetch with two cases - a hit that has only title + url (no langs,
no description), and a hit that has all fields populated. Asserts
the missing fields render as '' (was '-' before) and that populated
fields pass through verbatim.
The reddit/saved and reddit/upvoted changes in this PR live inside a
page.evaluate template literal that fetches from reddit.com inside
the browser context, so the empty-signal branch is executed inside
the page rather than in adapter JS. They are 1-char `|| '-'` ->
`|| ''` swaps with no downstream sentinel consumer and the same JS
semantics demonstrated by the gitee + apple-podcasts tests above.
* chore: rebuild cli-manifest.json to drop stale entries from rebase
The previous rebase left a stale linkedin/people-search entry in
cli-manifest.json that was carried over from a sibling feature branch.
This branch does not include the people-search source file, so the
entry was an orphan; CI's build-manifest safety check correctly
refused to overwrite it. Regenerating with --allow-removals to drop
the orphaned entry, after which a normal `npm run build` is a no-op.
* fix(twitter): skip "Discover new Lists" recommendations in lists adapter
The X.com /<user>/lists page powers two sections from a single
ListsManagementPageTimeline GraphQL response: "Discover new Lists"
(algorithmic recommendations) and "Your Lists" (owned + subscribed).
The previous parser ignored entry.entryId entirely and returned every
list it found, so recommendations leaked through and downstream
consumers treated them as the user's own lists.
X distinguishes the sections by entry.entryId prefix:
owned-subscribed-list-module-* → owned + subscribed (keep)
list-to-follow-module-* → Discover recommendations (drop)
cursor-* → pagination cursor (no list payload)
Filter on the owned-subscribed prefix in parseListsManagement and
expose isOwnedSubscribedEntry for testing. The existing test fixture
used a fictional entryId shape that no longer matches real responses;
update it to the nested-module shape Twitter actually returns and add
two new tests: one proving Discover entries are skipped, and one for
the entryId classifier.
Verified end-to-end against a live account: 10 raw entries (3 Discover
+ 7 owned/subscribed) now correctly return 7 owned/subscribed lists
with zero leakage.
* fix(twitter): harden lists parser boundary
* fix(twitter): require list-remove postcondition evidence
---------
Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(youtube/transcript): scope timedtext URL match to current videoId
YouTube watch-page is an SPA — page.goto between watch URLs preserves
performance.getEntriesByType('resource') entries from prior videos.
findTimedtextUrl filtered only by lang, so a previously-viewed
same-language video's timedtext URL could be picked up by the polling
loop before the current video's fetch hook captured a fresh one,
returning the wrong video's captions to the caller.
Fix: require URLs to contain v=<currentVideoId> across all three paths:
- in-page findTimedtextUrl (resource-buffer scan)
- in-page isJson3TimedtextUrl (fetch/XHR hook)
- Node-side extractSegmentsFromNetworkCapture (CDP capture)
Most likely to hit callers that reuse a single daemon tab to fetch
many transcripts back-to-back (e.g. ml-scout). Confirmed in the wild:
a Fox News Ukraine clip got Whisper Flow promo captions written to
its row when the prior call on the same tab pulled an English
Whisper Flow video.
Adds one source-contract assertion (both in-page sites use a shared
videoIdMarker) and one behavioral test (CDP capture buffer with a
stale 'v=prev' entry alongside the current 'v=abc' returns only the
current video's captions).
* fix(youtube): exact-match transcript timedtext video id
---------
Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ feedback: the intro section's "Let AI Agents operate any
website" bullet mistakenly references `opencli-adapter-author`, which
is the skill for **writing** adapters (correctly referenced in the
adjacent "Write new adapters" bullet). The skill for ad-hoc browser
driving is `opencli-browser` — its own SKILL.md frontmatter explicitly
says "Not for writing adapters — see opencli-adapter-author for that",
and `opencli-adapter-author` says "For ad-hoc browser driving (no
adapter), see opencli-browser instead".
Two locations affected with the same error: the intro bullet and the
Core Concepts > `browser` section. Both EN and ZH READMEs updated.
* feat(12306): add stations / trains / train read commands (no login required)
Adds a first-pass 12306 (中国铁路) adapter for the public anonymous
query endpoints. Closes the no-login slice of #1589. The
authenticated `me / passengers / orders` commands the issue
proposes are explicitly left as a follow-up.
Commands:
- 12306 stations <keyword> search station bundle
- 12306 trains <from> <to> --date YYYY-MM-DD availability between stations
- 12306 train <train-no> --from <s> --to <s> --date stop list
All three use Strategy.PUBLIC + browser: false, anonymous, no cookie
storage, no CAPTCHA bypass. Sensitive behaviors the issue rules out
(ticket sniping, order submission, payment, anti-abuse circumvention,
password storage) are not implemented.
Notes worth flagging for review:
- 12306 rejects anonymous query endpoints with HTTP 302 to
/mormhweb/logFiles/error.html. The adapter first hits
/otn/leftTicket/init to mint JSESSIONID / route / BIGipServerotn
cookies, then attaches them to subsequent queries. No CAPTCHA path.
- 12306 rotates the train-query endpoint name (queryO / queryZ /
queryA / queryG) every few weeks. When the wrong name is hit the
server returns `{c_url: "leftTicket/queryX", status: false}`
pointing to the current correct name. The adapter walks a list of
known names, captures the rotation hint, and retries; the runtime
list is also mutated so subsequent calls in the same process skip
the warm-up round trip.
- The `|`-separated train wire format includes a booking-handshake
`secret` field at position 0. Since this PR is read-only and the
issue explicitly rules out booking, that field is parsed but not
surfaced in the returned row, and a unit test asserts it cannot
leak via the public adapter contract.
- Station resolution accepts Chinese name (`上海虹桥`), telecode
(`AOH`), full pinyin (`shanghaihongqiao`), or short alias (`shhq`).
Anything else raises ArgumentError with a hint.
- `limit` arguments use a tight validator that throws ArgumentError
on non-integer / out-of-range input rather than silently clamping,
matching the typed-error pattern used in #1397 (grok) and #1370
(coupang).
Live verified anonymously against kyfw.12306.cn:
- `12306 stations 上海 --limit 5` returns 5 stations including
上海 (SHH) / 上海南 (SNH) / 上海虹桥 (AOH).
- `12306 trains 北京 上海 --date 2026-05-22 --limit 1` returns
G547 06:18 -> 12:11 with first / second / business / no-seat
availability columns populated.
- `12306 train 24000000G10L --from 北京南 --to 上海虹桥 --date 2026-05-22`
returns the 7-stop G1 route from 北京南 through 沧州西 / 德州东 /
曲阜东 / 南京南 / 苏州北 to 上海虹桥, with arrival / departure /
stopover times.
Tests: 18 unit tests covering parseStationBundle, resolveStation
(including ambiguous / case-insensitive cases), validateDate,
buildCookieHeader, parseTrainRecord (including a regression test
asserting the `secret` field cannot leak into the row).
Deliberately deferred to a follow-up: `12306 price`. The
queryTicketPrice endpoint needs train_no + per-stop station_no +
per-train seat-type letters, so an ergonomic `12306 price <code>`
would cascade three API calls (trains -> stops -> price) per
invocation. Wanted to keep this PR's blast radius small. If the
maintainer prefers a Phase 1 that includes price even with the
cascading-call cost, happy to add it.
* feat(12306): add me / passengers / orders / price authenticated + price read commands
Completes the #1589 12306 (中国铁路) adapter on top of the
stations / trains / train slice landed in the prior commit of this
branch. The full command set is now:
Anonymous (no login):
12306 stations search station bundle by Chinese / telecode / pinyin
12306 trains list trains between two stations on a date
12306 train list stops of one train
12306 price ticket prices for one train segment + date
Authenticated (cookie session):
12306 me account summary (sensitive fields masked by default)
12306 passengers saved-passenger list (sensitive fields masked)
12306 orders in-progress orders (not yet ridden / refunded)
Notes worth flagging for review:
- 12306 sets the auth cookie `tk` and the session cookie `JSESSIONID`
with `Path=/otn`. CDP `Network.getCookies` filters by URL path, so
`page.getCookies({ url: 'https://kyfw.12306.cn' })` returns 7
cookies without `tk` / `JSESSIONID`, even on a freshly-navigated
logged-in tab. Switched the login check to read `document.cookie`
via `page.evaluate`, which the current navigated page exposes
regardless of cookie path. Centralized as `require12306Login` in
utils.js so all three authenticated commands share the same check.
- All authenticated commands mask sensitive fields by default:
- `me`: real name (Chinese mask), email, mobile (12306 already
masks server-side), birth date (year only).
- `passengers`: name + birth year by default; 12306 already masks
ID number and mobile server-side and this adapter never decodes
those.
- Both expose `--include-sensitive` to opt back into the unmasked
fields the user is entitled to see on their own account.
- `orders` returns the `queryMyOrderNoComplete` slice (orders that
have not yet been ridden / refunded / completed). The historical
`queryMyOrderApi` endpoint requires extra page-state handshakes
that proved fragile when probed; left as a follow-up so this
command can ship reliably for the immediate "what's still on my
account" use case.
- `price` cascades three anonymous API calls per invocation:
init -> queryByTrainNo (to resolve segment station_no within the
train route) -> queryTicketPrice. 12306 returns prices keyed by
one-or-two-letter seat codes (`A9` 商务座 / `M` 一等座 /
`O` 二等座 / `WZ` 无座 / etc.) and additionally doubles some up
as bare numeric codes (e.g. `"9": "21580"` mirrors
`"A9": "¥2158.0"`); the bare-numeric duplicates are filtered out
so the row set is one-per-seat-class.
- Strictly anonymous queries; no CAPTCHA / slider / SMS bypass, no
credential storage, no ticket sniping, no order submission, no
payment - per the issue's Non-goals list.
Live verified anonymously and authenticated against kyfw.12306.cn,
sleeping 15-25 seconds between hits to keep 12306's anti-abuse
throttle gentle:
- 12306 me: account summary returned with real_name / email /
mobile / birth date all masked at the adapter level, on top of
12306's own server-side mobile mask.
- 12306 passengers: every saved passenger returned with name
masked to `<surname>*<...>` and 12306-side ID/mobile masks
preserved verbatim.
- 12306 orders: empty for this test account (no in-progress
orders), correct EmptyResultError surface.
- 12306 price G1 北京南 -> 上海虹桥 2026-05-22: returns
商务座 ¥2158 / 特等座 ¥1163 / 一等座 ¥1035 / 二等座 ¥626 /
无座 ¥626, sorted desc.
Tests: 23 unit tests (5 new beyond the prior commit's 18) cover
the mask helpers (email / mobile / Chinese name) plus the
parsePriceData filter that drops the bare-numeric duplicates and
sorts by descending price.
* fix(12306): harden browser auth boundaries
* fix(12306): tighten API drift boundaries
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(browser): recover from stale page identity on goto retry (#5)
When a chrome-backed adapter pre-navigates after its cached `_page`
targetId has been invalidated (tab closed externally, identity evicted),
the extension throws `Page not found: <id> — stale page identity` and
the failure cascades — every subsequent persistent-site session call in
the same process keeps re-sending the same dead targetId.
Observed in a downstream parallel multi-platform recall: a single dead page handle
got reused across 4+ calls (twitter thread / twitter search / reddit search)
because there was no detection or recovery. The same hash appeared in
adapter pre-navigations to youtube, twitter, reddit, xhs back-to-back in
seconds, suggesting the cached `_page` was shared via persistent site
session leases (`site:youtube` etc) and never cleared after the first
"stale page identity" response.
Page.goto() now catches that specific error, drops `_page`, and retries
once without the stale id. The retry navigates via session-lease
resolution in the extension (resolveTab → preferredTabId / new owned tab),
which already handles tab eviction correctly. No effect on the happy path.
Three regression tests in src/browser/page.test.ts cover:
- recovery: stale id dropped, retry succeeds with new identity
- no-cache safety: fresh page with no _page → error propagates unchanged
(nothing to drop, retrying would loop)
- error scoping: unrelated extension errors (e.g. disconnected) still
surface immediately — no implicit retry
* fix(errors): classify -32000 "Cannot find default execution context" as retryable (#6)
classifyBrowserError previously only matched CDP -32000 errors when the
message contained "target" (e.g., "target closed"). It missed
"Cannot find default execution context", a CDP protocol error that also
indicates the inspected target went away — observed in a downstream parallel
adapter recall against youtube channels.
Widening the secondary check to `/target|context/i` lets the existing
target-navigation retry path (200ms delay + re-attach) recover instead of
surfacing the error as non-retryable.
* fix(browser): tighten stale page recovery notes
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
clean-dist deletes dist/ and tsc --build re-emits files without preserving
the executable bit on the bin entry. Symlinked global install then hits
EACCES on spawn until manually chmod'd. Chain a chmodSync into the existing
prebuild-manifest hook so any future rebuild self-heals.
node -e instead of bare `chmod +x` to keep the script portable (npm runs
on Windows via Git Bash where chmod is a no-op, but fs.chmodSync still
silently no-ops there too — no extra branching needed).
Co-authored-by: Kary <karyhe1019@gmail.com>
* fix(lesswrong): drop "Unknown" silent sentinel in author column
Twelve lesswrong commands had `author: item.user?.displayName ?? 'Unknown'`
which masks the missing-author signal: an agent reading the result row
cannot distinguish "post has no associated user" from "author is literally
named Unknown". The repo's typed-error lint flags this pattern
(silent-sentinel rule, see scripts/check-typed-error-lint.mjs:323).
Replace `?? 'Unknown'` with `?? ''` so the missing-author case stays
visible as an empty string. Consistent with `clis/lesswrong/_helpers.js:68`
which was already using the empty-signal form.
Shrinks scripts/typed-error-lint-baseline.json from 173 to 161 entries.
Follows the same direction as #1603 (fix(adapters): surface silent empty
fallbacks).
Verified live: `opencli lesswrong frontpage --limit 2 -f json` returns
real posts with non-empty author values; empty-author rows would now
show `"author": ""` instead of fabricating `"Unknown"`.
* test(lesswrong): add empty-signal coverage for the author sentinel swap
Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with a focused unit test that
mocks the upstream LessWrong GraphQL response to return posts where
`user` is null or `user.displayName` is missing, and asserts the row
surfaces `author: ''` instead of the old fabricated `'Unknown'`.
`clis/lesswrong/frontpage.test.js` is representative for the twelve
identical `author: item.user?.displayName ?? ''` swaps across
comments / curated / frontpage / new / read / sequences / shortform /
tag / top / top-month / top-week / top-year, all of which share the
exact same expression with no downstream sentinel consumer.
The empty-signal path is exercised live too: a deleted-account or
permission-restricted user shows up in the GraphQL response with
`user: null`, surfaces as `author: ''` post this PR (was 'Unknown'
before).
* feat(weibo): add delete command to remove user's own posts
Adds `opencli weibo delete <id>` so the same workflow that creates a
post can also remove one without leaving the CLI. The id positional
accepts either the numeric `idstr` (e.g. `5299336218674412`) or the
base62 `mblogid` (e.g. `QFGbHAoBS`) found in any weibo URL or in the
output of `weibo me` / `weibo feed` / `weibo post`.
Implementation lives in a single `page.evaluate` IIFE so cookies +
the XSRF-TOKEN double-submit token stay first-party:
1. Resolve mblogid / idstr via `GET /ajax/statuses/show?id=<input>`,
which returns the canonical `idstr`. Empty result -> 404 path.
2. Read the `XSRF-TOKEN` cookie via `document.cookie`.
3. `POST /ajax/statuses/destroy` with `id=<idstr>` body and the
`X-Xsrf-Token` header.
4. Return `[{ status: 'deleted', id, mblogid }]`.
Typed errors:
- 401 / 403 from either show or destroy -> `AuthRequiredError`
- `show` returning no `idstr` -> `EmptyResultError`
- Non-2xx HTTP on either call -> `CommandExecutionError` with status
- API response `ok !== 1` -> `CommandExecutionError` with the API msg
Closes#1619.
Verified live on macOS / opencli v1.7.22, weibo cookie session:
- Deleted the lingering test post from #1602 verification
(idstr=5299336218674412, mblogid=QFGbHAoBS):
`weibo delete QFGbHAoBS` returned
`[{ status: 'deleted', id: '5299336218674412', mblogid: 'QFGbHAoBS' }]`
- `weibo me` shows `statuses: 3` (was 4 before the delete)
- `weibo post QFGbHAoBS` now throws "Post not found"
Unit tests: 8 / 8 in `clis/weibo/delete.test.js` (happy path,
empty-id, auth, not-found, show-http, destroy-http, api-msg, envelope
unwrap). Full weibo suite: 38 / 38 pass.
* fix(weibo): require delete postcondition evidence
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu/publish): invoke shadow-DOM publish handler directly
XHS creator center now wraps the publish/save-draft button in an
`<xhs-publish-btn>` web component backed by a CLOSED shadow root.
Calling `.click()` on the host element does not dispatch into the
internal handler, and CDP coordinate clicks cannot penetrate the
shadow boundary. The previous text-match `button.click()` loop hit
the host element, returned `ok`, and yet the note silently stayed
on the publish page as a draft, so the adapter reported the soft
`⚠️ 操作完成，请在浏览器中确认` status while nothing was actually
posted.
Invoke the publish/save method directly on the `<xhs-publish-btn>`
host (`_onPublish` / `_onSave` and a few candidate names XHS has
shipped historically). Fall back to the legacy
`<button>`/`[role="button"]` text-match click for older
creator-center variants that still expose plain buttons.
Patch shape suggested by the OpenCLI autofix report in #1606 from
@chcc-funny (who verified an end-to-end real publish locally).
Closes#1606.
Verified live on macOS / opencli v1.7.22 / extension v1.0.15,
with creator center logged in:
- `opencli xiaohongshu publish ... --draft` -> `✅ 暂存成功`,
creator home shows "草稿箱中有未发布的作品"
- `opencli xiaohongshu publish ...` (real publish) -> `✅ 发布成功`,
note appeared on the account feed (visible from mobile app);
test note deleted after verification
Unit tests: 12 / 12 in `clis/xiaohongshu/publish.test.js` pass
(mocks updated to reflect the new `{ ok, via, name|text }` invoke
result shape).
* feat(xiaohongshu): add delete-note command to remove published notes
Adds `opencli xiaohongshu delete-note <note-id>` so the workflow that
creates a note can also remove one without leaving the CLI, mirroring
`weibo delete` (#1619 / #1620).
The creator-center HTTP delete API requires the `X-S-Common` signature
header that `publish.js` deliberately avoids, so this follows the same
UI automation route. Flow:
1. Navigate to creator note-manager
2. Switch to "已发布" tab (delete entry only appears there; "审核中"
and "未通过" rows have no web delete action, mobile app only)
3. Locate the `.note` row whose `data-impression` JSON contains the
target noteId (exact JSON-parsed match, not substring, so values
that happen to share the noteId prefix in other fields cannot
match the wrong row)
4. Click the inline `<span class="control data-del">` action
5. Click "确定" in the `.d-modal-footer` confirmation modal
6. Poll for the row disappearing (iteration-bounded so tests with
mocked `page.wait` exhaust the loop quickly)
Typed errors:
- /login redirect after navigation: AuthRequiredError
- 已发布 tab not found / not clickable: CommandExecutionError (UI drift)
- target noteId not present in the rendered list: EmptyResultError with
a hint about review-state limitation
- row found but no delete action visible: CommandExecutionError
- confirmation modal missing / no 确定 button: CommandExecutionError
- row still visible after the configured poll window: CommandExecutionError
Closes#1623.
Verified live: published a test note, deleted via this adapter, follow-up
`xiaohongshu creator-notes` confirms it is gone. Unit tests: 8 / 8 cover
happy path, empty-id ArgumentError, login redirect AuthRequiredError,
tab-not-found CommandExecutionError, row-not-found EmptyResultError,
no-delete-action / no-modal / unverified-delete CommandExecutionError
paths.
Built on top of #1613 (xiaohongshu publish shadow-DOM fix) so the live
verify could exercise publish-then-delete end to end. Will rebase onto
main once #1613 lands.
* fix(xhs): make delete-note fail closed
* fix(xiaohongshu): harden delete-note boundary
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(weibo/publish): replace brittle CSS-module hash with placeholder selector
`clis/weibo/publish.js` matched the compose textarea via
`textarea._input_13iqr_8`, where `_input_13iqr_8` is the Vite CSS-module
hash Weibo rebuilds on every frontend deploy. The hash drifted (current
build emits `_input_1f5hn_8`), so step 4 of the publish flow throws
"Weibo compose editor did not appear" before anything else can run.
Reported in #1602.
Replace the single hashed selector with a placeholder-text-based chain
that survives Weibo's CSS-module rebuilds:
textarea[placeholder*="有什么新鲜事"]
textarea[placeholder*="新鲜事"]
textarea._input_13iqr_8 // legacy hash kept last for older variants
Two visible textareas can match on the home feed (the always-rendered
"home-strip" prompt + the post-click modal compose). Pick the LAST
visible candidate: the modal opens on top and is appended to DOM later,
so the last-visible textarea is the modal. Both the editor-visibility
poll (Step 4) and the text-insertion step (Step 6) use the same chain.
Also drops `evaluateWithArgs` from Step 8 success polling. The IIFE
there does not reference any outer args, but `evaluateWithArgs` injects
its `const`-bound parameter names into the page context, and re-running
on each iteration of the success-poll loop threw `Identifier
'maxIterations' has already been declared` after the first iteration.
This was masked previously because Step 4 always failed first; with the
selector fixed, the latent Step 8 bug surfaces. Switched to plain
`page.evaluate` to avoid re-declaring per loop.
Closes#1602.
Verified live on macOS / opencli built locally / extension v1.0.15,
weibo cookie session:
- `opencli weibo publish "明洞那家店真不错"` returned
`status: success, message: 发布成功, text: 明洞那家店真不错`
- Confirmed via `/ajax/statuses/mymblog`: the post landed at
`idstr=5299403716821218`, `mblogid=QFHWzsCvE`, text matches what
was typed (proves selector chain picks the right textarea and the
text insertion path works end-to-end)
- Cleaned up: deleted via the same `/ajax/statuses/destroy` path that
PR #1620 exposes as `weibo delete`
Unit tests: 8 / 8 in `clis/weibo/publish.test.js` pass (mocks updated
to reflect the new `evaluate`-vs-`evaluateWithArgs` split for Step 8
and the longer poll window).
* test(weibo): lock publish placeholder selector path
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(adapters): drop silent-sentinel row fallbacks across 6 read commands
Continues the audit-baseline cleanup started in #1611 (lesswrong) and
the direction set by #1599 / #1603 / #1604. Replaces the
`silent-sentinel` row-data fallbacks (`'Unknown'` / `'-'` / `'unknown'`
that mask missing fields) with the empty-string signal so agents can
tell apart "field really has the value Unknown" from "upstream returned
no value".
Touched 6 read adapters, 10 baseline entries:
- wikipedia/trending: title, description
- 36kr/article: author, date, body
- xiaoyuzhou/download: podcast
- xiaoyuzhou/transcript: podcast
- zhihu/collection: dedup key + type field (the empty prefix still
produces a unique-per-content dedup key, just without the `unknown:`
noise)
- zhihu/download: author
Intentionally skipped (line-by-line audited):
- v2ex/me.js: `'Unknown'` is an in-band control-flow sentinel. Line 35
initialises `let username = 'Unknown';`, line 41 uses
`if (username === 'Unknown')` to trigger the profileEl fallback
selector, line 75 uses the same check to raise the auth error.
Empty would silently bypass both checks and return a row with an
empty username as if auth succeeded.
- v2ex/daily.js: `'未知'` is user-facing 签到 success text in the
rendered status message, not a row field. Empty would render a
broken sentence.
- weibo/comments.js, weibo/feed.js: the sentinel sits inside an in-IIFE
error-message string composition (`'API error: ' + (data.msg || 'unknown')`),
not in a returned row. Empty would silently truncate diagnostic
output. Both stay on baseline.
Verified live: `opencli wikipedia trending --limit 3` and `opencli 36kr
hot --limit 2` both return populated rows; the empty-string signal only
kicks in when the upstream value is actually missing.
* test(adapters): add empty-signal coverage for the cluster-2 sentinel swap
Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with focused unit tests that
mock the upstream to return null / missing fields and assert the row
surfaces an empty-string signal instead of the old fabricated
'Unknown' / '-' / 'unknown' sentinel.
Coverage:
- clis/wikipedia/trending.test.js (new): mocks wikiFetch to return
three articles - one with both title + description populated, one
with no title and no description, one with title only. Asserts the
missing fields render as '' (was '-' before this PR).
- clis/36kr/article.test.js (new): mocks page.evaluate to return a
scrape where title is present but author / date / body are empty.
Asserts those three fields render as '' in the row pair output
(was '-' before this PR). Also covers the NOT_FOUND and
INVALID_ARGUMENT error paths that already existed.
- clis/zhihu/collection.test.js (+1 case): mocks the zhihu collection
API to return an item with content.id but no content.type. Asserts
type renders as '' (was 'unknown' before this PR); the new dedup
key prefix is :id rather than unknown:id, semantically identical
for dedup purposes.
The other three files in this PR (xiaoyuzhou/download,
xiaoyuzhou/transcript, zhihu/download) use the same `|| 'unknown'` ->
`|| ''` value swap with no downstream sentinel consumer. They are
covered by the same JS language semantics the three tests above
demonstrate.
* fix(adapters): fail typed on missing row identity
* fix(adapters): tighten sentinel row identity guards
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(electron-apps): move codex CDP port off 9222 to avoid browser-bridge collision
`src/electron-apps.ts` had `codex: { port: 9222 }`, but `9222` is the
default Chrome DevTools port that opencli's own browser-bridge Chrome
binds whenever `opencli doctor` is OK. On every normal opencli install
the bridge owns 9222 first, so Codex Desktop can never bind it, and
`opencli codex status` (plus every other codex command) fails with:
App launched but CDP not available on port 9222 after 15s
`~/.opencli/apps.yaml` is documented as "additive only, does not
override builtins", so users have no supported way to relocate the
port from the user side.
Reported in #1626 with full repro (Codex Desktop + active opencli
browser-bridge Chrome) and root-cause pointer at
`dist/src/electron-apps.js:13`. Every other electron app in the
builtin registry already uses a distinct port in the 9224-9236
band (cursor 9226, doubao-app 9225, chatwise 9228, discord-app 9232,
antigravity 9234, chatgpt-app 9236); codex was the only one that
collided with the browser bridge.
Move codex to 9238 (the next free slot in that band, also the value
the reporter recommended). Update the test that asserts the port and
the two docs references that mention codex=9222. The pitfall entry
in `docs/advanced/electron.md` is also annotated to explicitly call
out 9222 as the bridge's port to avoid future collisions.
Closes#1626.
Verified live: `opencli codex status -v` now emits
`[verbose] [launcher] Probing CDP on port 9238...` (was 9222 before
the fix), confirming the code path picks up the new port. Full
end-to-end with a real Codex Desktop install is left to the reporter
and reviewer; the change here is a single-value config update plus
docs/tests sync.
Unit tests: 7 / 7 in `src/electron-apps.test.ts` pass (the codex-port
assertion updated to 9238). Both audit gates pass.
* docs(electron): sync codex CDP port guidance
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ feedback: the previous Highlights list was bloated with hollow
marketing phrases and overlapping bullets (e.g. "Browser Automation for AI
Agents" + "AI Agent ready" said the same thing twice, "Pipeable, scriptable,
CI-friendly" is generic CLI filler).
Cut "AI Agent ready", "Account-safe" (folded into Live Browser Automation),
"Deterministic"'s second sentence (folded into Zero LLM cost), and merged
"Website → CLI" with "CLI Hub" into "100+ adapters + CLI Hub". Result is 5
concrete capability bullets instead of 9, each tied to a real feature.
EN and ZH READMEs kept in sync.
* feat(bilibili): add summary command for the official AI video summary
Adds `opencli bilibili summary <bvid>` — fetches Bilibili's official
AI-generated video summary (the "AI总结" shown on the video page) via
/x/web-interface/view/conclusion/get.
Returns the overall summary followed by the timestamped section outline,
so you get a structured digest of a video without watching it.
- Resolves cid + up_mid from the view endpoint (both required by the
conclusion API), then calls the WBI-signed conclusion endpoint.
- Throws a clear EmptyResultError when a video has no AI summary —
Bilibili only generates them for some videos.
Covered by clis/bilibili/summary.test.js (5 cases): summary + outline,
summary without outline, no-summary, view-resolution failure, API error.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(bilibili): harden summary command contract
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(linkedin): add messaging commands
Add fail-closed LinkedIn inbox, connect, safe-send, and thread-snapshot commands with adapter tests and docs.
* fix(linkedin): align commands with current UI
Update inbox to read LinkedIn's normalized messaging API response and connect to use the current custom-invite route.
* chore(linkedin): sync cli-manifest.json with rebuilt inbox command
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(linkedin): pass silent-column-drop gate
Drop the intermediate timestamp_ms field from inbox rows (it is converted to the timestamp column) and baseline the connect command internal profile-probe object.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(linkedin): validate inbox --limit with a typed error
Reject an out-of-range --limit with ArgumentError instead of silently clamping it, satisfying the typed-error lint gate.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(linkedin): harden messaging command contracts
* fix(linkedin): reject inbox conversations without thread id
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add Youdao Notes shared note reader adapter
Add a new adapter for reading publicly shared Youdao Notes (有道云笔记).
- youdao note <url>: Fetches a public shared note by its share URL
using browser-based DOM extraction. Extracts title, content, and
keyword tags from the React-rendered page.
- Supports note.youdao.com and note.youdao.cn share URLs.
- Includes test coverage (3 tests) and documentation.
Closes#1418
* fix: extract full note content from React Redux store
Previously the adapter only extracted the AI summary section from the
DOM. Now it accesses the React fiber tree to read the full note content
from the Redux store (store.content.data.content), which contains the
complete note body in Youdao's structured format.
The extractor recursively walks Youdao's proprietary node format (key '8'
for text content) to reconstruct the full note as plain text.
* fix(youdao): harden shared note reader contract
---------
Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
- Replace 2-line tagline (websites/browser/electron/local + reuse logged-in browser) with a single line emphasizing the two core capabilities side by side: 把任意网站变成 CLI & 让 AI Agent 操控登录态浏览器
- Add "Help me fill out this form" as the leading opencli-browser skill example so the table surfaces browser-side capabilities, not just scraping
* feat(douyin): restore publish and delete flow
- Use upload auth v5 API instead of legacy STS2 for VOD credentials
- Switch TOS upload from AWS4-signature to gateway multipart protocol (init/transfer/finish)
- Add ApplyUploadInner → CommitUploadInner pipeline for VOD upload
- Bypass enable/transend endpoints that hang for gateway-uploaded videos
- Handle fast_detect/pre_check empty responses gracefully with retry+backoff
- Add creator backend delete fallback (via work_list id matching) when legacy delete returns permission error
- Use CommitUploadInner Vid for create_v2, not completed TOS object key
- Accept item_id as fallback when create_v2 returns no aweme_id
* fix(douyin): harden publish delete write contracts
---------
Co-authored-by: Lukin <mylukin@gmail.com>
* feat: add Flomo memos reader adapter
Read your Flomo memos via the signed API.
- flomo memos: Lists recent memos with content, tags, timestamps
Uses the Flomo v1 API with MD5 signing (secret embedded).
Requires FLOMO_ACCESS_TOKEN env variable.
Supports pagination via --slug cursor and --limit.
* fix: add --token arg for Flomo auth
* fix: use COOKIE strategy with browser-based API call
Use Strategy.COOKIE + browser:true instead of PUBLIC + manual token.
The adapter now reads flomo_token from localStorage in the browser,
and makes the signed API call from within the page context via fetch().
Signature is computed in Node.js and injected into the browser eval.
No env var or --token flag needed.
* fix: use access_token from localStorage.me for API auth
Flomo API requires Bearer token from access_token field in
localStorage.me (not api_token). Adapter now reads access_token
from the browser's localStorage and calls the signed API from
Node.js with the Bearer header.
* feat: add --since filter, refine flomo adapter API
- Add --since <unix_ts> to filter memos by updated_at
- Add --limit 200 to fetch all memos in one call
- Mark --slug as experimental (cursor pagination unreliable)
- 5 tests passing
* feat: add images column to flomo memos output
* docs: add flomo adapter documentation
* fix: use clampInt and rebuild manifest
* fix(flomo): harden memos reader contract
---------
Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(chatgpt): unwrap page.evaluate envelope across browser commands
The browser bridge wraps every `page.evaluate(...)` return value in a
`{ session, data }` envelope. Adapters that read `.length` or
`Array.isArray(payload)` directly on the envelope silently see "no
data" — same failure mode addressed for `xiaohongshu`/`rednote` in
#1561 and `weibo` in #1568.
This sweep applies the same `unwrapEvaluateResult` helper across every
chatgpt `page.evaluate` consumer site, plus typed shape guards
(`requireArrayEvaluateResult`, `requireObjectEvaluateResult`) on the
critical extraction paths so envelope misses fail loud instead of
silently returning empty.
## Sites wrapped
`clis/chatgpt/utils.js`:
- `currentChatGPTUrl` — string URL
- `getPageState` — login/composer probe object
- `sendChatGPTMessage` — composer write + send-button readiness
- `getVisibleMessages` — conversation transcript array
- `getConversationList` / `extractConversationLinks` — sidebar items
- `waitForChatGPTUploadPreview` — image upload readiness probe
- `uploadChatGPTImages` fallback — DataTransfer upload result
- `isGenerating` — boolean "still generating?" probe
- `getChatGPTVisibleImageUrls` — visible image URL array
- `waitForChatGPTImages` — inline `window.location.href` poll
- `getChatGPTImageAssets` — exported asset array
`clis/chatgpt/image.js`:
- `currentChatGPTLink` — used for error hints + conv link reporting
## Drive-by
`getChatGPTImageAssets` was also passing a redundant `urls` second arg
to `page.evaluate(string, urls)`. The IIFE inside the string already
receives the URL list via the `${urlsJson}` template substitution, and
the browser bridge guard in `browser/utils.ts` rejects the second form
for string scripts with:
page.evaluate string input does not accept args;
use page.evaluate(fn, ...args) instead
So `opencli chatgpt image <prompt>` blows up at the download step
without `--sd true`. Drop the trailing arg as part of the asset-export
cleanup. (This supersedes #1556 — same one-line fix is included here.)
## Validation
- `npx tsc --noEmit` — clean
- `npx vitest run --project adapter clis/chatgpt/` — 38/38 pass
(25 existing + 13 new in `envelope.test.js`)
- `npm test` — 3644 passing across 364 files
- Live (browser bridge, daemon v1.7.19):
`opencli chatgpt image "<prompt>"` → end-to-end generate + download
succeeds; the envelope wrap is defensive in 1.7.19 (no envelope
observed yet), but pre-empts the same silent-failure mode that hit
the merged xiaohongshu/weibo PRs.
* fix(chatgpt): fail fast on malformed evaluate payloads
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* docs: add weibo search_by_user command design spec
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(weibo): add search_by_user helper function tests
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* feat(weibo): add search_by_user command for timed post download to Markdown
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(weibo): remove dead hasori ternary and hardcoded hastext/haspic filters
The hasori ternary always evaluated to 1 (bug), and hastext=1 + haspic=1
silently excluded text-only and link-only posts from results.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* test(weibo): add integration tests for search_by_user helpers
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* bak
* fix(weibo): reshape user posts into read adapter
---------
Co-authored-by: andrew.asa <asa.andrew@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
`ntn`, `dws`, and `wecom-cli` are opaque executable names — users seeing them
in `opencli list` or root help have no way to know they correspond to Notion,
DingTalk Workspace, and 企业微信. Repurpose the existing `package` field to
double as a human-readable brand label, so help output renders as
`ntn(notion)`, `dws(DingTalk Workspace)`, `wecom-cli(企业微信)`.
- `src/external-clis.yaml`: add `package:` to ntn / dws / wecom-cli
- `src/external.ts`: update JSDoc on `package` to cover both upstream
distribution names (tg-cli, discord-cli) and brand labels (notion, 企业微信)
- `src/cli.ts:629` (`opencli list`): use `formatExternalCliLabel` so the
listing matches root help, which already used it
- `src/external.test.ts`: regression test for brand-alias labels
Verification:
- npx vitest run --project unit src/external.test.ts: 9/9 pass
- npm run typecheck: clean
- npm run build: 813 manifest entries
- Smoke: `opencli list` and `opencli --help` both render the new labels
* fix(weibo): unwrap page.evaluate envelope in read adapters (#1567)
`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, so all weibo cookie-strategy read adapters
silently dropped their results on v1.7.19:
- `getSelfUid` returned the envelope object instead of the uid string,
so `'10001' + uid` produced `'10001[object Object]'` and every
feed/me/favorites request hit a broken list_id.
- `feed`, `hot`, `comments`, `search`, `favorites` did `Array.isArray`
on the envelope (always false) and returned `[]`.
- `me`, `user`, `post` returned the envelope wrapper itself instead of
the inner profile/post object.
Same pattern as #1561 for xiaohongshu/rednote. Adds an
`unwrapEvaluateResult` helper to `clis/weibo/utils.js` (kept local
rather than cross-importing from `xiaohongshu/search.js` since weibo
is an unrelated site) and wraps every `await page.evaluate(...)` in
the 8 read adapters plus the two helper calls in `getSelfUid`.
Skipped `publish.js` (write command, out of scope for this read fix).
Verified live:
- `opencli weibo hot --limit 3` returns 3 real trending items
- `opencli weibo feed --limit 3` returns 3 timeline posts with
correct `https://weibo.com/<uid>/<mblogid>` URLs (proves
`getSelfUid` unwrap works)
- `opencli weibo me` returns the logged-in profile object
- All 20 weibo unit tests pass (6 new for `unwrapEvaluateResult`)
* fix(weibo): fail typed on malformed evaluate payloads
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Recruiter-only BOSS commands (recommend, joblist, stats, resume, mark,
exchange, invite, greet, batchgreet) returned a generic
`COMMAND_EXEC: 请切换身份后再试 (code=24)` when called from a job-seeker
account. The original error hid the actionable bit: this command set
needs a recruiter (BOSS-side) account.
chatlist / chatmsg already special-case code=24 by falling back to the
geek-side fetch when --side=auto. Recruiter-only commands have no
geek-side equivalent and were just leaking the raw API code.
Fix: add a `checkRecruiterSide` step inside `assertOk` that maps
code=24 to AuthRequiredError with a clear message. All 9 recruiter-only
commands inherit it through their existing `bossFetch` calls; no
adapter-level changes needed. chatlist / chatmsg are unaffected because
they use `allowNonZero: true` and never hit the auto-error path.
Closes#1572.
* feat: add DuckDuckGo, Brave, and Yahoo web search adapters
Add three new search engine adapters with browser-based DOM extraction:
- duckduckgo/search: Search DuckDuckGo via html.duckduckgo.com
Supports region, time filters, and XHR-based pagination (--offset)
- duckduckgo/suggest: Search suggestion autocomplete (no browser needed)
- brave/search: Search Brave Search via search.brave.com
Supports GET-based pagination (--offset)
- yahoo/search: Search Yahoo (Bing-powered) via search.yahoo.com
Supports GET-based pagination (--page)
All search adapters use Strategy.PUBLIC with browser:true, navigating
the target site and extracting results via page.evaluate() DOM queries.
Includes full test coverage (16 tests).
* fix: use clampInt from shared utils and add adapter docs
- Replace Math.max/Math.min patterns with clampInt() from _shared/common.js
to pass the typed-error-lint gate (4 silent-clamp violations resolved)
- Add adapter documentation for duckduckgo, brave, and yahoo to fix
the doc-coverage CI check
- Regenerate cli-manifest.json and typed-error-lint-baseline.json
* fix: avoid silent-column-drop overlap in brave/yahoo extractors
Change buildExtractorJs to return arrays instead of objects whose keys
matched columns. This prevents silent-column-drop audit false positives
as per opencli-adapter-author conventions.
* fix(search): tighten browser search adapters
* chore(search): drop baseline churn
* fix(duckduckgo): execute search extractor safely
* fix(yahoo): reject unsafe redirect targets
---------
Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu,rednote): unwrap page.evaluate envelope in search adapter
`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, but the search adapters were calling
`Array.isArray(payload)` directly on the envelope. `Array.isArray` is
always false on the envelope, so every search result was silently
dropped — status=success, exit 0, empty array, no error.
The rednote adapter had this same bug; both share `buildSearchExtractJs`
from `xiaohongshu/search.js`.
Introduces `unwrapEvaluateResult(payload)` as a shared helper in
`clis/xiaohongshu/search.js` (re-exported via the existing import line
from `rednote/search.js`). The helper is a defensive ternary: it
unwraps when payload looks like an envelope with an array `.data`,
otherwise it passes the value through unchanged. This keeps the change
back-compat with bridge versions that return the raw value, and
preserves the existing `Array.isArray(payload)` typecheck at each call
site.
Verified manually against `opencli xiaohongshu search "补墙洞"` (a query
known to return 20+ results in a logged-in browser tab): previously
`[]`, now returns the expected ranked rows with all declared columns
(`rank, title, author, likes, published_at, url`) populated.
Adds 5 unit tests for `unwrapEvaluateResult` covering raw array passthrough,
envelope unwrap, non-envelope object passthrough, null/undefined safety,
and the "data is not an array" guard. The existing 19 search tests in
`clis/xiaohongshu/search.test.js` still pass — the unwrap is invisible
to the existing mocks which already return raw arrays.
* fix(xhs): unwrap search evaluate envelopes
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* refactor(notion): replace built-in CDP adapter with external ntn CLI
Notion has shipped an official CLI at https://ntn.dev. It uses the
public Notion API (blocks / databases / properties / comments) instead
of reverse-engineering the Desktop UI, so it survives Notion app
updates and exposes a wider command surface than the in-tree adapter
could.
Changes:
- `src/external-clis.yaml` — register `ntn` as first-class external CLI
(binary `ntn`, homepage ntn.dev, install via the shell-pipe script
on mac/linux)
- `clis/notion/` — entire directory removed (8 commands: status /
search / read / new / write / sidebar / favorites / export)
- `docs/adapters/desktop/notion.md` — removed
- `docs/.vitepress/config.mts` — drop nav entry
- `docs/adapters/index.md` — drop adapter row
- `README.md` / `README.zh-CN.md` — drop notion from feature lines,
drop adapter table row, add `ntn` to CLI hub examples
- `docs/index.md` / `docs/zh/index.md` / `docs/guide/getting-started.md`
— drop notion from electron-control feature copy
- `skills/opencli-usage/SKILL.md` — drop notion from electron list
- `cli-manifest.json` — rebuilt with --allow-removals=8
Migration for users:
`curl -fsSL https://ntn.dev | bash` (or `opencli external install ntn`)
Then use `opencli ntn <command>` in place of `opencli notion <command>`.
Rationale: the in-tree adapter was reverse-engineered against Notion
Desktop CDP and shipped only 8 commands. The official CLI gives users
the full Notion API surface and reduces our maintenance burden to zero.
Same pattern as gh / obsidian / lark-cli / tg-cli / discord-cli / wx-cli.
Verification:
- `npx tsc --noEmit` clean
- `npx vitest run --project unit` → 1091/1 skipped
- `npm run build` (with --allow-removals=8) — manifest 809 entries
- grep notion in user-facing docs (README / docs / skills) — only
descriptive mentions remain in non-blocking places (comparison /
site-recon / electron how-to / design doc), no broken adapter
references
* fix(notion): align ntn external migration
* docs(notion): clarify ntn manual install
Mirrors PR #1464 (list-tweets) and the timeline/search/tweets/likes/thread
family: spread `...extractMedia(legacy)` into the row and surface
`has_media` + `media_urls` columns. Pure parity, no behavior change for
existing callers — media keys do not collide with the original columns.
- bookmarks.js: import `extractMedia` from ./shared.js, spread into
extractBookmarkTweet row, append columns, export __test__.
- bookmark-folder.js: same change on extractFolderTweet, export
extractFolderTweet via __test__.
- bookmarks.test.js (new): baseline + photo + video + entities-only
fallback + dedup + envelope + empty-envelope (8 tests).
- bookmark-folder.test.js: update existing baseline expectation with
has_media/media_urls, add 3 new media tests (photo / mp4 / no-media).
- cli-manifest.json: regenerated; only the two `columns` entries change.
Reverse-validated: tests fail when extractMedia spread is removed.
Audits unchanged: typed-error-lint 189/189, silent-column-drop 102/103
(pre-existing main resolution noted but not consumed here).
* feat(twitter/list-tweets): include media via extractMedia (parity with timeline/search)
list-tweets was the only X recall path that dropped media. timeline.js and
search.js both call extractMedia(legacy) and emit has_media/media_urls;
list-tweets returned only text fields, so downstream consumers (e.g.
ml-scout's rate UI) couldn't render image/video thumbnails on tweets pulled
from a list timeline.
Changes:
- Import extractMedia from ./shared.js
- Spread extractMedia(legacy) into extractTimelineTweet return
- Add has_media, media_urls to columns array (--format columns parity)
- Update unit test to assert the new shape; add coverage for photo and
video extraction
* chore(manifest): rebuild cli-manifest.json for list-tweets media columns
---------
Co-authored-by: ml-scout <ml-scout@anthropic.com>
The opencli external-CLI name is the user-typed subcommand; the binary is
what gets executed. The convention everywhere else (`gh`, `docker`,
`obsidian`, `vercel`, `dws`) is `name == binary`. Three entries violated
the convention: `tg-cli` / `discord-cli` / `wx-cli` registered an
opencli name with a `-cli` suffix that does NOT exist on the binary,
forcing the awkward double-prefix `opencli discord-cli dc` instead of
`opencli discord dc`.
The README's example column already showed the desired form
(`opencli tg search`, `opencli discord recent`, `opencli wx search`) —
only the yaml registration was out of sync.
Renames in `src/external-clis.yaml`:
* `name: tg-cli` → `name: tg` (binary: `tg`)
* `name: discord-cli`→ `name: discord` (binary: `discord`)
* `name: wx-cli` → `name: wx` (binary: `wx`)
The `binary`, `homepage`, and `install` fields are unchanged — the
underlying packages (`kabi-tg-cli`, `kabi-discord-cli`, `@jackwener/wx-cli`)
keep their published names.
Other entries left as-is: `lark-cli`, `wecom-cli`, and `dws` already have
`name == binary` (their actual binaries are `lark-cli`, `wecom-cli`, `dws`).
BREAKING CHANGE: `opencli tg-cli ...`, `opencli discord-cli ...`,
`opencli wx-cli ...` no longer resolve. Use `opencli tg ...`,
`opencli discord ...`, `opencli wx ...` instead. The feature is recent
(shipped 2026-05) so impact is expected to be minimal.
* feat(twitter): default tweets to logged-in user + fix sibling envelope-unwrap silent bug
Primary: make `opencli twitter tweets` default to the logged-in user
when no username is given, so agents can pull their own posts without
needing to know their own handle. Mirrors the existing self-detection
pattern in twitter/profile and twitter/likes (AppTabBar_Profile_Link
probe on /home, then UserByScreenName lookup). Description + help
string now mention the default so agents discover it.
Consistency pass — profile/likes/following/followers: the
self-detection in these four siblings was silently broken because
page.evaluate() primitive returns come back through the CDP bridge
wrapped as `{session: 'site:twitter', data: '/<handle>'}` (same
envelope root cause as #1525). They called `.replace()` directly on
the envelope object → TypeError surfaced as AUTH_REQUIRED 'Could not
detect logged-in user', even for logged-in users. Wrap each probe
with unwrapBrowserResult so the bare href string survives. Also:
- Add an explicit page.goto('/home') + page.wait(primaryColumn)
before the probe in likes/following so the AppTabBar sidebar is
guaranteed rendered (framework pre-nav lands on bare x.com without
the sidebar mounted).
- following.js: switch its probe from the function-literal form
`() => {...}` to a template-string. Confirmed live: function-literal
silently drops primitive returns entirely — bridge returns
`{session}` with no `data` field at all, while template-string
returns `{session, data}` as expected.
Out of scope (pre-existing, flagged as follow-up): likes/following
have additional downstream evaluate paths (userId/GraphQL fetch) that
still drop or envelope their results; they return [] or
'Could not find user' even after this PR. Same daemon-side bug class
as #1525.
Live-verified:
opencli twitter tweets --limit 2 → own tweets (@jakevin7)
opencli twitter profile → own profile
Tests 227/227, audits typed-error-lint 189 + silent-column-drop 103
unchanged, manifest stable at 816 entries.
* fix(twitter): validate self-detected handles
* fix(twitter): unwrap downstream self evaluate results
* fix(twitter): unwrap page.evaluate primitive returns in lists/list-tweets/following
The opencli >=1.7.x browser bridge wraps page.evaluate's primitive return
values as { session, data: <value> }. Adapters that destructure .data
inline (e.g. data.queryId, data.viewer) keep working because the wrapper
spreads object-typed responses to the top level, but ones that consume
the return value as a bare string broke:
- twitter list-tweets: the dynamically resolved queryId (a string) became
{session, data:"..."}. Interpolating that into the GraphQL URL produced
/i/api/graphql/[object Object]/ListLatestTweetsTimeline, giving "HTTP
400: queryId may have expired".
- twitter lists: same on ListsManagementPageTimeline queryId.
- twitter following: same shape bug on the href read from the profile
link, producing "TypeError: href.replace is not a function" when no
--user is given.
Add a small unwrap() helper at each call site so primitive returns are
extracted from the wrapper before use. Object-typed GraphQL responses
are left as-is since they rely on spread semantics.
* fix(twitter): rewrite list-add to use ListAddMember GraphQL mutation
In 2026-05 X replaced the "Add/remove from Lists" modal dialog with a
full-page route (/i/lists/add_member). The previous UI flow no longer
works:
Save button not found in dialog (X expected text Save/Done).
Dialog structure may have changed.
The mutation that the dialog used to fire (ListAddMember) is still the
right primitive — and the surrounding adapter already calls X GraphQL
APIs directly to resolve userId and verify member_count. Drop the UI
flow entirely and call ListAddMember directly via fetch in the page
context.
Wins:
- Works again on current X UI (verified 2026-05-12 on x.com).
- ~10x faster: no goto-profile + click-caret + scroll-dialog round trips.
- One less moving piece — no dependency on Chrome extension's nativeClick
for this command.
Implementation notes:
- LIST_ADD_MEMBER_QUERY_ID is a 2026-05 fallback; resolveTwitterQueryId
does live lookup from the loaded client-web bundle, matching the
pattern already used elsewhere in the twitter clis.
- X's ListAddMember response routinely contains a non-fatal partial
decode error on default_banner_media_results (code 214, Validation /
BadRequestError) alongside a fully populated data.list. We treat the
call as failed only when data.list / member_count is missing, and
ignore decode-flavored errors confined to banner fields.
- Same opencli >=1.7.x { session, data } primitive-wrap behavior that
the previous commit addressed applies here: userId from the
UserByScreenName call needs unwrap before being interpolated into
the mutation body, otherwise X parses "[object Object]" as user_id
and returns "strconv.ParseInt ... invalid syntax".
Verified flows:
- noop (already a member) → status: noop, member_count unchanged.
- new add (e.g. @AnthropicAI on a fresh list) → status: success,
member_count incremented.
Trade-off: rejection signals (e.g. X declining to add @deepseek_ai)
look indistinguishable from noop at the response level, since X returns
HTTP 200 with member_count unchanged. Documented in the success message.
* fix(twitter): integrate list media and harden list-add
---------
Co-authored-by: wangyan <wy@wang-yan-Air.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(zhihu): add answer-detail to fetch a single answer's full content
The existing `zhihu answer` adapter is a write (post an answer); the
listing `zhihu question` truncates each answer's body to 200 chars.
There was no way to fetch one specific answer's full content by id.
New read adapter `zhihu answer-detail`:
- Accepts a bare numeric answer id, a typed target `answer:<qid>:<aid>`,
or a full Zhihu answer URL (the form you paste from a browser).
- Calls `/api/v4/answers/<aid>?include=content,voteup_count,...,question`
inside the cookie-bearing page context (Strategy.COOKIE).
- Returns a single row with id / author / votes / comments /
question_id / question_title / url / created_at / updated_at /
content. The content column is the full stripped answer body by
default — no silent truncation. `--max-content N` is an opt-in user
cap (mirroring the wikipedia `page` flag), and `--max-content 0`
(the default) means "no cap, full content".
Important precision note: Zhihu answer ids since 2024 routinely
exceed `Number.MAX_SAFE_INTEGER` (the test fixture uses the real id
`1937205528846655537`). `data.id` is round-tripped through browser
`JSON.parse` and would round to `1937205528846655500`, so the adapter
deliberately ignores `data.id` for the canonical row id and anchors
it to the already-validated input string instead. A regression test
locks this contract in by mocking `data.id = 0` and asserting the row
still carries the parsed input id.
Typed errors: bad input → INVALID_INPUT; 401/403 → AuthRequiredError;
other HTTP / null → FETCH_ERROR. No silent fallbacks, no sentinel
strings.
Live-verified against the example URL — fetched 5547 votes / 165
comments / 1937205528846655537-end-to-end. 16 unit tests, audits
unchanged (typed-error-lint 189/189, silent-column-drop 103/103),
manifest 816→817.
* fix(zhihu): tighten answer-detail contracts
* fix(google-scholar/search): wrap evaluate return to fix serialization
Same issue as google/search: page.evaluate() serializes JS arrays as
plain objects across the CDP boundary, causing Array.isArray() to
return false. The adapter silently returned [] instead of results.
Also replace fixed page.wait(3) with selector-based wait for
.gs_r.gs_or.gs_scl with a 3s fallback.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(google-scholar): type search evaluate payload
* chore: rerun google scholar search checks
---------
Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu): parseLikes should handle 2.1w / 1.5万 / 1.2k shortforms
Xiaohongshu renders top-popular comment like-counts as shortened
strings like '2.1w' / '1.1万' / '1.2k' once they exceed ~10 000.
The previous parseLikes only matched bare digits via /^\d+$/ and
silently returned 0 for any shortform, which inverted the sort
order: the highest-liked comments (often 10k+) ranked last while
mid-tier comments with plain numeric counts (e.g. 7569) appeared
on top.
Repro on any popular xiaohongshu thread (>10 000 likes on a top
comment): with --format json the most-upvoted parent rows show
"likes": 0.
This patch keeps the original fast path for plain integers and
adds a single regex for the well-known shortform suffixes:
- w / 万 -> *10000
- k / 千 -> *1000
- trailing '+' tolerated (e.g. '999+')
- unknown shapes still fall back to 0 (no behavior change)
Note: parseLikes runs inside the IIFE injected via page.evaluate(),
so the existing comments.test.js mock harness (which stubs
evaluate's return value directly) does not exercise it. A future
refactor that exports parseLikes for direct testing would be a
separate change.
Affects both top-level comments and 楼中楼 sub-replies (same
helper).
* fix(xiaohongshu): parse comment like shortforms safely
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* chore: drop util.styleText to support Node v20+
util.styleText was added in Node v21.7.0 / v20.12.0. v21.0.0-v21.6.x and
v20.0.0-v20.11.x throw `SyntaxError: ... styleText` at startup because the
import resolves before any user code runs (a real user reported this on
v21.2.0).
OpenCLI is primarily agent-facing — terminal colors are noise to consumers,
and the [OK] / [WARN] / [FAIL] / ℹ / ⚠ / ✖ markers we already write carry
the semantic info that colors only repeated. Strip styleText entirely from
logger / output / doctor / tui / update-check / cli / download/progress /
commands/daemon and clean up the resulting awkward `${'literal'}` template
fragments. engines.node now reads ">=20.0.0".
This removes the Node-version coupling that A/B fixes would only have
papered over.
* fix(runtime): truly support Node v20+ by aligning guard + undici
Follow-up to the styleText removal: declaring engines.node >=20.0.0 is
not enough on its own. Two coupled barriers remained:
- src/runtime-detect.ts: MIN_SUPPORTED_NODE_MAJOR = 21 explicitly
rejected v20 at startup
- undici@^8.0.2 declares engines.node >=22.19.0; Node 20/21 crash on
webidl.util.markAsUncloneable before any user code runs
Lower the guard to 20 and downgrade undici to ^6.25.0 (engines >=18.17,
retains Agent / EnvHttpProxyAgent / fetch / Dispatcher). Smoke-tested
--help / doctor / list on Node v20.0.0, v21.2.0, v22.22.2. 213/213
targeted unit tests pass.
* feat(zhihu): paginate question answers and recommendations
* fix(zhihu): drop Math.min limit clamp and 'unknown' sentinel
Two audit-driven fixes on top of feat/zhihu-pagination-recommend:
1. question.js: replace `Math.min(answerLimit, 20)` with a named
constant `ZHIHU_PAGE_SIZE = 20`. The Zhihu API caps `limit` at 20
per request anyway, and the pagination loop already trims to the
user-requested `answerLimit` via `answers.length >= answerLimit`,
so the Math.min silent-clamp was both unnecessary and tripped the
silent-clamp audit. Updates the existing unit test to expect the
API-max page size in the fetch URL with an explanatory comment.
2. recommend.js: rebuild the dedup key without the `'unknown'`
sentinel. The old form `\`\${target.type || 'unknown'}:\${target.id}\``
collapsed distinct typed items into the same bucket whenever
`target.type` was missing, and tripped the silent-sentinel audit.
New form: prefer `type:targetId`, fall back to `__feed:item.id`,
and when neither id is available keep the row but skip dedup
(surfacing potentially-duplicate items beats silently dropping
them).
Audits unchanged (typed-error-lint 189/189, silent-column-drop
103/103). All 88 zhihu tests pass.
---------
Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Issue #1506 reports `opencli xiaohongshu search` returning `[]` even though
the page visibly has results. Trace evidence: xhs ships a render variant
where each note card is a bare `<section>` (no `note-item` class), so
the three `section.note-item` selectors in this file all match zero
elements.
Three call sites in the shared search IIFEs now use the same defensive
selector strategy: try the legacy `section.note-item` class first, then
fall back to any `<section>` that wraps a `/search_result/...` or
`/explore/...` link. The change is in the xiaohongshu file so the
rednote adapter (which imports `buildSearchExtractJs` and
`buildScrollUntilJs` from here) picks it up automatically.
Extraction-side title selector also gets a fallback: when no
`.title` / `.note-title` element matches, read the first `<span>`
inside the search-result link, which is where the bare-section render
puts the caption per the trace.
## Verification
`npx vitest run --project adapter clis/xiaohongshu/`: 105/105 green
(existing test suite unchanged, passes on both legacy and fallback paths).
Live verify on rednote (same code path, account-safe):
```
$ opencli rednote search "美食" --limit 3 -f json
[ {rank:1, title:"在朋友家吃过一次..."}, {rank:2, title:"我的15💰晚餐..."}, {rank:3, title:"干净饮食🫛..."} ]
```
Legacy `section.note-item` path is exercised here (rednote still renders
the class) and returns identical row shape to before the fix, confirming
no regression on the working path.
Live verify on xiaohongshu cannot be performed here (no logged-in xhs
session on the test machine; xhs account-ban risk per the project's
operational guidance). The fix is structural: the new `<section>` shape
the issue reporter traced is reachable through the fallback, and the
existing test fixture keeps the legacy path green.
`npx tsc --noEmit` clean. `npm run build` 815 manifest entries unchanged
shape. `silent-column-drop` / `typed-error-lint` baselines unchanged.
Closes#1506
Refs #1500
* feat(reddit/read): add --expand-more via /api/morechildren + 7-kind discriminated union
PR B of the rdt-cli parity follow-up (after PR #1491, see #1481 thread).
Closes the second-largest gap: Reddit's "[+N more replies]" stubs were
opaque markers in the comment tree. With --expand-more, the adapter
follows them by POST-ing the t1 ids to /api/morechildren.json, then
re-threads the returned things back into the tree by parent_id before
walking it.
New args:
- `--expand-more` (bool, default false) — turn on stub expansion.
- `--expand-rounds <N>` (int, default 2, range [1, 5]) — Reddit returns
fresh "more" stubs at the expansion depth boundary, so up to N rounds
are run. Strictly validated via `parseExpandRounds` — out-of-range
raises ArgumentError BEFORE `page.goto`, no silent clamp.
Boy-Scout: the in-browser script now returns a 7-kind discriminated
union instead of a flat row array (matching the PR #1428 / #1491
sediment). Each kind maps 1:1 to a typed error on the Node side:
- `inaccessible` → EmptyResultError
401/403/404 on /comments/<id>.json (post-specific access, not
session-level auth — applies the PR #1491 review-side sediment
"inaccessible-resource vs session-auth").
- `auth` → AuthRequiredError
401/403 on /api/morechildren (expand-write endpoints often demand
a logged-in session even when the read endpoint is anonymous).
- `http` → CommandExecutionError
- `malformed` → CommandExecutionError
200 with unexpected envelope shape — schema drift, not empty.
- `parser-drift` → CommandExecutionError
tree had t1 entries but the walker produced no rows (PR #1491
review-side sediment "post-construction 0 rows + pre-walk
non-empty = parser drift, not legitimate empty").
- `expand-failed`→ CommandExecutionError
/api/morechildren returned a non-empty json.errors array.
- `ok` → returns rows[].
Intermediate keys (kind / detail / httpStatus / where / rows /
expandMeta) deliberately avoid the declared columns (type / author /
score / text) per the PR #1329 silent-column-drop sediment.
Tests:
clis/reddit/read.test.js — 11 tests
- Adapter shape (browser / siteSession / columns / args)
- --expand-more / --expand-rounds present with correct types/defaults
- parseExpandRounds default / range / non-integer rejection
- Pre-navigation validation (bad --expand-rounds doesn't reach goto)
- kind=ok happy path (POST + L0 rows)
- 6-kind error → typed error mapping
- Unknown envelope shape → CommandExecutionError
- Evaluate script embeds expandMore/expandRounds/sort/limit literals
- Evaluate script contains /api/morechildren POST scaffolding
- Evaluate script never names declared columns as intermediate keys
Full reddit suite 48/48; full project 3402/3402.
Audits: typed-error-lint 189/189 (0 new), silent-column-drop 103/103
(0 new). Manifest 815 → 815 (existing read entry gets 2 new args).
Existing --limit / --depth / --replies / --max-length keep their
original Math.max-style behaviour (grandfathered in the baseline);
only the new --expand-rounds flag fails fast per the typed-errors
standard.
Refs: https://github.com/jackwener/rdt-cli (browse.read --expand-more)
* fix(reddit): preserve expanded comment tree order
* fix(reddit): fail on partial morechildren expansion
* fix(twitter): repair search and tweets readback
* fix(twitter): prefer baked operation features when bundle parse returns empty
The bundle parser in resolveTwitterOperationMetadata locates the queryId via
`queryId:"..."` inside a ~2500-char snippet around the operationName marker,
then independently extracts `featureSwitches:[...]` and `fieldToggles:[...]`
via separate regexes. When minification rearranges the snippet (or the
snippet window truncates before the array), either regex can miss while
queryId still resolves; keysToFlags(undefined) then returns {}.
sanitizeTwitterOperationMetadata previously accepted any object as
features / fieldToggles, including {}. Twitter's GraphQL endpoint rejects
SearchTimeline / UserTweets requests with empty features (HTTP 400),
surfacing a misleading "queryId may have expired" error — the queryId is
fresh; only the feature flags are missing.
Guard against this by deferring to the baked fallback whenever the resolved
map is empty. Adds a JSDOM-free unit test that, reverse-validated, fails on
the un-fixed code with the exact silent-fallback shape.
Refs PR #1512
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
page.evaluate() serializes JS arrays as plain objects, causing
Array.isArray() to return false and the adapter to throw NOT_FOUND
even when results exist. Wrap the return value in {items: results}
and extract via wrapper.items to avoid the type check issue.
Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Per @WAWQAQ direction (DM): trim PR-time CI to fast-feedback only.
adapter-test (~30-60s) is the next-largest PR wait after e2e-headed
(which #1521 just removed). Adapter authors typically run focused tests
locally before pushing (`npm run test:adapter`); CI duplication adds
queue latency without catching new classes of bugs.
PR-time CI surface now:
- typecheck / unit (~1 min)
- lint gates (typed-error / silent-column-drop)
- build × 3 platforms
Adapter test guards (still strict):
- push to main / dev
- nightly cron
- workflow_dispatch (manual when an adapter-heavy PR really wants the
signal before merge)
Same gate as smoke-test (`if: github.event_name == 'push' || schedule
|| workflow_dispatch`) for consistency.
Per-PR e2e-headed Chrome was the dominant PR-time wait (~10-15 min on
two platforms) and on fork PRs blocks behind maintainer approval, while
the actually-blocking failures it caught in the last 30 days were all
e2e-test migrations missed by the authoring PR (#1461 / #1505 workspace
->session) rather than real regressions the unit/typecheck tier missed.
PR feedback path is now:
- typecheck / unit / lint / adapter / build ← `pull_request` (ci.yml)
- extension typecheck / build ← `pull_request` (build-extension.yml)
- docs build ← `pull_request` (doc-check.yml)
- security audit ← `pull_request` (security.yml)
E2E-headed Chrome guards:
- push to main / dev (watched paths)
- push v* tag (release)
- nightly cron 08:00 UTC (added: catches Chrome version drift / flake
drift even when no commits touch watched paths)
- workflow_dispatch (manual when a PR really wants e2e signal)
smoke-test was already gated on `schedule || workflow_dispatch` only
(ci.yml), so no change needed there.
`pageScopedResult()` in extension/src/background.ts was spreading the
lease's session into the result `data` for every page-scoped command. For
the `exec` action — which routes user JavaScript through page.evaluate()
— this contaminated arbitrary user-JS returns:
* Array / primitive returns came back as `{ session, data: <value> }`
envelopes. Adapters that did `Array.isArray(result)` got `false` and
treated the page as having no rows. Visible repro:
`opencli google search ...` and `opencli xiaohongshu search ...` —
Chrome rendered results correctly but adapters extracted an empty array
(reported in #1518 from the Browser Bridge v1.0.12 envelope).
* Plain-object returns had an extra `session` key spliced in, silently
overwriting any user `session` field with the lease's value.
Fix in the extension layer instead of compensating client-side:
`pageScopedResult` now returns `{ id, ok, data, page }` — the same form
it had before #1461 added the workspace→session refactor. Client-side
unwrapping is no longer needed and the original PR #1518 `Page.evaluate`
heuristic is dropped (it only covered the array path and would have
missed the plain-object path).
Two adapter improvements kept from the original PR:
* `clis/google/search.js` — wait for `#rso a h3` (with a 5s timeout)
before extracting. On Chrome 148 / Linux Wayland the DOM can settle
before SERP anchors are populated, so the existing fixed `wait 2`
could return empty even with the envelope fix.
* `clis/xiaohongshu/search.js` — extract initially visible cards before
scrolling, then merge post-scroll rows by URL. Xiaohongshu's
virtualized masonry can evict the initial note cards from the DOM
after scroll, causing extraction to return [] even though the
browser had rendered results correctly.
Extension version bumped to 1.0.14.
Repro environment (from #1518):
* OpenCLI 1.7.18
* Browser Bridge extension 1.0.12 → 1.0.14
* Chrome 148.0.7778.96
* Linux Wayland, Node 22.22.1
Tests: extension/src/background.test.ts navigate same-url assertion
updated to no longer expect `session` in `data`. Three Page.evaluate
unwrap test cases removed.
* fix(xueqiu/kline,earnings-date): format dates in Asia/Shanghai instead of UTC (#1465)
`xueqiu/kline` and `xueqiu/earnings-date` formatted bar timestamps with
`new Date(ts).toISOString().split('T')[0]`. That string is the UTC
calendar date, always one day earlier than the date xueqiu shows in its
UI (which is Beijing-aligned for every market). Issue #1465 reports
"5月10日跑的，5月8号的k线没有" because the May 8 China trading-day bar
was labeled 2026-05-07. Same off-by-one was present in `earnings-date.js`.
Routes both call sites through a new `formatChinaDate(ts)` helper in
`clis/xueqiu/utils.js` built on `toLocaleDateString('en-CA', { timeZone:
'Asia/Shanghai' })`. Verified live against SZ300136 and AAPL: both now
match the dates shown on xueqiu.com.
Tests: `clis/xueqiu/utils.test.js` (new) pins the Asia/Shanghai semantic
with 4 cases (China midnight, late-evening, 16:00 UTC day boundary, and
nullish input). `npx vitest run --project adapter clis/xueqiu/` 49/49,
`npx tsc --noEmit` clean, `npm run build` 815 entries unchanged shape.
Closes#1465
* fix(xueqiu): stabilize China date formatting
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
`OPENCLI_KEEP_TAB` was a debugging shortcut, not a config dimension. It
let users override `--keep-tab` globally via the shell environment,
which contradicts the per-command lifecycle model: `siteSession:'persistent'`
already pins persistent site tabs as a hard adapter-metadata constraint,
and `--keep-tab true|false` covers the ad-hoc override case. The env
just leaked process state across every browser command in the shell.
Changes:
- src/execution.ts: `resolveKeepTab()` drops the
`normalizeBooleanOption('OPENCLI_KEEP_TAB', process.env.OPENCLI_KEEP_TAB)`
fallback. `--keep-tab` is now the single user override.
- src/execution.test.ts: two regression tests rewritten to use the
`executeCommand(cmd, {}, false, { keepTab: 'true' })` signature
instead of the env. Logic and assertions unchanged.
- README.md / README.zh-CN.md / skills/opencli-usage/SKILL.md:
drop the env table row. `--keep-tab` documentation stays.
- CHANGELOG.md: BREAKING entry under Unreleased.
Note: the 1.7.15 CHANGELOG entry still references the env historically;
that's intentional, historical entries are not retroactively edited.
Verification:
- npx tsc --noEmit pass
- npx vitest run --project unit --project extension → 1144/1145 pass
(1 unrelated skip)
- typed-error-lint baseline 189
- silent-column-drop baseline 103
* refactor(browser): replace --session flag with <sessionname> positional
The `--session <name>` flag was semantically required but syntactically
optional, which is an anti-pattern. Required + flag is a contradiction:
flag form implies "optional", required is a runtime patch on top. Session
is OpenCLI's "operation target" identifier — the natural form for that is
a positional argument, like `docker exec <container> <cmd>` or
`git checkout <branch>`.
New surface:
opencli browser <sessionname> open https://x.com
opencli browser <sessionname> click 12
opencli browser <sessionname> bind
opencli browser <sessionname> unbind
Commander 14 cannot natively combine a parent positional with subcommand
dispatch — the parent's positional is shadowed by subcommand matching. To
bridge that, main.ts now pre-processes argv: when the token after `browser`
is non-flag and not a known subcommand name, it is treated as the
sessionname and rewritten to the internal `--session <name>` flag form
before commander parses it. Help text on the `browser` command is
overridden via `.usage('<sessionname> <command> [options]')` so users see
the positional form.
Reserved subcommand names (33) are listed in cli-argv-preprocess.ts and
tested for parity with cli.ts subcommand registrations. If a future
subcommand is added, the test fails loudly.
Synced surfaces:
- README.md / README.zh-CN.md — all examples
- docs/guide/browser-bridge.md (+ zh)
- skills/opencli-browser/SKILL.md (bind/unbind, examples, table)
- skills/opencli-usage/SKILL.md
- tests/e2e/browser-tabs.test.ts
- CHANGELOG.md (Unreleased BREAKING)
The internal `--session` flag and the unit tests calling
`program.parseAsync(['...', 'browser', '--session', 'foo', ...])` are
preserved as a stable internal API: tests bypass main.ts pre-processing
and exercise commander directly. The pre-processor has its own targeted
test file (cli-argv-preprocess.test.ts, 10 tests, all green).
Verification:
- npx tsc --noEmit — pass
- npx vitest run --project unit — 1073/1074 pass (1 unrelated skip)
- npx vitest run --project extension — 61/61 pass
- npm run check:typed-error-lint — baseline 189
- npm run check:silent-column-drop — baseline 103
* fix(cli-argv): only rewrite when `browser` is the root command
The preprocessor was looping through every argv slot and would mis-rewrite
occurrences of the literal word `browser` deeper in argv (e.g. `opencli
adapter init browser/x` or arg values containing `browser`).
Now the preprocessor walks past leading root flags + their values to
identify the root command token, and only acts when that token is
`browser`. The full set of root value-consuming flags
(`ROOT_VALUE_FLAGS`) is documented inline and kept in sync with the
`program.option()` calls in cli.ts.
Adds regression tests:
- `opencli adapter init browser x` not rewritten
- URL/path values containing `browser` not rewritten
- `list browser state` (different root command) not rewritten
- `--profile work browser foo state` correctly identifies `foo` as
sessionname (not as --profile's value)
- `--profile=work` long-form-with-equals consumes one slot only
- boolean flags (`-v`) don't consume the next value
12/12 preprocessor tests pass.
* fix(cli-argv): hide --session flag, fail-fast on retired form, rename to <session>
Three blockers in #1505 review:
1. `--session` flag was still visible in `opencli browser --help` and could
be used as a public entrance, contradicting "positional only" UX.
Fix: switch from `.requiredOption()` to `.addOption(new Option(...).hideHelp())`.
The flag is preserved as an internal API for the daemon protocol and direct
`program.parseAsync` callers (tests), but is no longer documented or
surfaced in structured help.
2. `opencli browser --session foo state` still succeeded. Now the argv
preprocessor throws `BrowserSessionArgvError` when root `browser` is
followed by `--session`, and main.ts catches it and exits with a
user-facing usage error pointing to the positional form.
3. Missing-session error message exposed the internal flag:
`required option '--session <name>' not specified`. Now `getBrowserSession()`
in the action body throws `<session> is a required positional argument:
opencli browser <session> <command>`, and commander no longer guards the
hidden option.
Also (per @WAWQAQ) rename placeholder `<sessionname>` -> `<session>` everywhere
user-facing — shorter, matches CLI convention. The help text "<session> is a
required positional: pass the name of the browser session..." carries the
"name" semantics in description, not in the placeholder itself.
Sync surfaces:
- src/cli.ts — usage line, addOption with hideHelp, descriptions
- src/cli-argv-preprocess.ts — throw on --session form
- src/cli-argv-preprocess.test.ts — refusal test for old form
- src/cli.test.ts — assertions updated for hidden option + new error path
- src/help.ts — read `_usage` private field to respect `.usage()` override
(commander's `.usage()` getter returns auto-generated form if not set,
which would otherwise pollute every namespace's usage string)
- src/main.ts — catch BrowserSessionArgvError, stderr + exit
- README.md / README.zh-CN.md
- docs/guide/browser-bridge.md / docs/zh/guide/browser-bridge.md
- skills/opencli-browser/SKILL.md / skills/opencli-usage/SKILL.md
- CHANGELOG.md
Manual smoke tests (against built dist):
- `opencli browser --help` shows `Usage: opencli browser <session> <command> [options]`
- `opencli browser --help` Options block does NOT show `--session`
- `opencli browser --session foo state` → friendly error, no commander stacktrace
- `opencli browser state` → `<session> is a required positional argument: opencli browser <session> <command>`
- `opencli browser foo state` → parses correctly
* fix: inject <session> into subcommand help paths and drop stale sessions ref
Two follow-up blockers from #1505 review:
1. Subcommand help and structured help still rendered the command path
without the parent's positional. `opencli browser foo state --help`
showed `Usage: opencli browser state [options]`, which would lead
users (and agents reading structured help) to think
`opencli browser state` was a valid invocation. Now:
- `commanderPath()` injects an ancestor's leading-positional placeholder
(extracted from its `.usage()` override) between the ancestor's name
and the next path segment when building paths upward.
- `commandPathFromRoot()` strips placeholder segments (e.g. `<session>`)
from the relative `name` field so agents can still address subcommands
by their leaf name; placeholders remain in the `command` / `usage`
display paths.
- `program.configureHelp({ commandUsage: ... })` is applied recursively
to every descendant of `browser`, because commander does NOT inherit
`configureHelp` into subcommands.
Result:
opencli browser <session> click --help
-> Usage: opencli browser <session> click [target] [options]
Daemon, plugin, adapter, profile namespaces (no `.usage()` override)
are unaffected.
2. `skills/opencli-browser/SKILL.md` still referenced
`opencli browser sessions`, which was removed in #1470. Replaced the
sentence with the underlying invariant ("Bound sessions have no
OpenCLI idle-close timer; the binding lasts until `unbind`, tab close,
window close, or daemon restart") without mentioning the deleted
command.
Tests:
- cli.test.ts: structured help expectations updated to include
`<session>` in command/usage paths (3 tests)
- cli-argv-preprocess.test.ts: 12 tests still green
- 1136/1137 unit+extension green (1 unrelated skip)
- typed-error-lint baseline 189
- silent-column-drop baseline 103
* feat(reddit): add whoami, home, subreddit-info read commands
Closes gap against jackwener/rdt-cli — three commands the existing 17 reddit
adapters were missing:
- `reddit whoami` — show the currently logged-in identity (fields:
Username, ID, Post / Comment / Total Karma, Account Created, Gold, Mod,
Verified Email, Has Mail, Inbox Count). Probes `/api/me.json` with
two-pronged auth detection (401/403 OR `data.name` missing on 200 —
Reddit returns 200 with an empty body for stale anon sessions, see PR
#1428).
- `reddit home` — personalized Best feed (`/best.json`). Distinct from
the public `frontpage`/`r/all` command: enforces login via the same
two-pronged auth check rather than silently degrading to the
unauthenticated default feed. `--limit` accepts [1, 100] — out-of-range
raises `ArgumentError` before navigation, no silent clamp.
- `reddit subreddit-info` — subreddit metadata (Name, Title, Subscribers,
Active Now, NSFW, Type, Description, Created, URL) from
`/r/<X>/about.json`. Banned / private / quarantined / 404 subreddits
raise `EmptyResultError` so the output table never holds a silent
sentinel row.
All three use Strategy.COOKIE + siteSession:'persistent' matching the
existing reddit adapters, validate args upfront before `page.goto`, and
use the 5-kind discriminated-union pattern (kind: auth/http/missing/
exception/ok) from PR #1428 to map page.evaluate results to typed errors
on the Node side. Intermediate object keys deliberately avoid the
declared columns (`field`/`value`/`rank`/etc.) per the silent-column-drop
audit sediment from PR #1329.
Tests: 28 new (whoami 6, home 9, subreddit-info 13); full reddit suite
38/38. Audits: typed-error-lint 189/189 (0 new), silent-column-drop
103/103 (0 new). Manifest 812 → 815.
Refs: https://github.com/jackwener/rdt-cli
* fix(reddit): tighten new read command failure contracts
* fix(reddit): treat inaccessible subreddit info as empty
* chore(scripts): auto-refresh dist/ before build-manifest
`build-manifest.ts` is invoked via tsx so its own imports go to TS source,
but the adapter `.js` files it loads import `@jackwener/opencli/registry`
through package exports, which resolves to `dist/src/registry-api.js`.
When `dist/` is stale relative to `src/` (e.g. a contributor edits
`src/registry.ts` and runs only `npm run build-manifest` instead of the
full `npm run build`), the stale dist drops fields like `siteSession`
from the rebuilt manifest. CI catches the resulting diff via the
"cli-manifest.json is up-to-date" gate, but locally it surfaces as
mysterious unrelated diff lines for adapter files the contributor never
touched.
Add an npm pre-script that runs `tsc --build` (incremental, ~0.6s when
warm) so `npm run build-manifest` is safe to use directly. `npm run build`
is unchanged — it still does the full `clean-dist + tsc + copy-yaml +
build-manifest` sequence, and `prebuild-manifest` will be a no-op there
since TS is already compiled by the time it runs.
Verified:
- `rm -rf dist && npm run build-manifest` now restores dist via the
pre-hook and produces a 0-line diff against committed manifest
- `npm run build` still produces the same clean output
* fix(scripts): force manifest dist refresh
* fix(scripts): avoid duplicate manifest compile
* feat(ctrip): add hotel-search + flight browser-mode commands
Closes#1481.
Two new browser-mode commands on top of the existing public `search` /
`hotel-suggest` pair:
- `ctrip hotel-search <city> --checkin --checkout [--limit]` reads
`window.__NEXT_DATA__.props.pageProps.initListData.hotelList` on
`hotels.ctrip.com/hotels/list`. SSR-rendered first page ships ~13
entries; the server ignores `&pageSize=N` so limit caps at 30 with
default 10. AuthRequiredError surfaces when Ctrip redirects to the
captcha gate.
- `ctrip flight <from> <to> --date [--limit]` searches one-way flights on
`flights.ctrip.com/online/list/oneway-…`. The post-load XHR is not
currently captured by the daemon network buffer (per the known
daemon_capture_pipeline_bug_2026_05_07 in agent memory), so rows are
pulled from `.flight-list > span > div` cards via a position-anchored
innerText parser. A generic `buildScrollUntilJs(selector, target)`
helper mirrors the PR #1487 xiaohongshu scroll-until pattern with the
selector parameterised. Round-trip + airline filters are out of scope
for v1.
All argument validation (IATA / ISO date / city ID / limit range) fires
upfront before any `page.goto`, per the PR #1387 boundary standard. No
silent clamps, no sentinel rows: rows missing required fields are
dropped, and end-state checks raise `ArgumentError` /
`AuthRequiredError` / `EmptyResultError` as appropriate. The new
`mapHotelRow` / `pickHotelMapCoords` / `buildFlightExtractJs` /
`buildScrollUntilJs` helpers live in `clis/ctrip/utils.js` alongside the
existing suggest helpers.
Docs at `docs/adapters/browser/ctrip.md` now distinguish the public
suggest commands from the browser-mode commands and document each
command's columns + caveats.
Verified:
- 61/61 vitest tests in `clis/ctrip/ctrip.test.js` (including JSDOM
exercises of `buildFlightExtractJs` and full `mapHotelRow` shape parity)
- `check:typed-error-lint` 189/189 (0 new)
- `check:silent-column-drop` 103/103 (0 new)
- `build-manifest` clean — 812 entries total (was 810)
* fix(ctrip): harden browser search failure contracts
* fix(ctrip): tighten browser empty-vs-parser failures
* docs(skill/adapter-author): warn aria-label / placeholder / title is locale-dependent
aria-label changes with the browser's UI language (chrome://settings/languages).
A button labelled `aria-label="Submit"` in English Chrome becomes
`aria-label="提交"` in Chinese Chrome, so CSS selectors hardcoded to one
locale silently match zero elements — `notEmpty` / `types` never fire because
the adapter just returns 0 rows.
First-principles framing in adapter-template:
- Split DOM attributes into "locale-stable identifiers" (id / class /
data-testid / data-* / role) vs "locale-dependent text" (aria-label /
title / placeholder / alt / textContent)
- Primary selectors must use locale-stable identifiers; locale-dependent
text is a last-resort tiebreaker
- When a site (e.g. ChatGPT web) only exposes aria-label, link the existing
`clis/chatgpt/utils.js` fallback-list pattern (en + zh-CN + stable
fallback at the front)
Explicitly document why we are NOT building a `find --i18n "zh:提交"` flag
(over-engineering: same indirection as a fallback list plus a translation
dictionary to maintain) and why we are NOT locking Chrome's locale at launch
(opencli doesn't launch Chrome — it connects to the user's running browser
via CDP, so forcing en-US would break users who intentionally run Chinese UI).
Adds pitfall #11 to success-rate-pitfalls.md for the agent-facing checklist.
Closes#1474
* docs(skill): tighten locale selector guidance
* fix(xiaohongshu+rednote): scroll until enough rows are rendered instead of fixed 2x autoScroll
Both search adapters previously called `page.autoScroll({ times: 2 })` which
hard-capped extraction at ~13 notes (xiaohongshu lazy-loads ~5-7 notes per
scroll round) regardless of `--limit`. Reported in #1471: `--limit 40` still
only returned 13 results.
Replace with a dynamic `buildScrollUntilJs(targetCount, maxScrolls=15)`
helper that:
- counts visible `section.note-item` rows (excluding `.query-note-item`
related-search rows)
- breaks early when count >= target
- breaks early after 2 consecutive scrolls add no new rows (DOM plateaued,
feed exhausted)
- hard caps at 15 iterations to bound runtime
Exported from xiaohongshu and reused by rednote (same DOM shape) instead of
duplicating the IIFE.
Fixes#1471
* fix(xiaohongshu): tighten search scroll boundary
* fix(doubao/ask): restore Assistant turn detection after 2026-05 DOM refactor
Doubao reworked message-item wrappers and dropped all `receive-message` /
`bg-g-receive-msg-bubble` markers from assistant turns. The legacy 6
`itemSelectors` (`item-kDun2N`, `union_message`, `message-block-container`,
`data-message-id`, `bg-g-send-msg-bubble`, `bg-g-receive-msg-bubble`) match 0
elements on the new DOM, so `getTurnsScript` returned [] and `getDoubaoTurns`
fell through to the whole-page transcript scraper. Assistant text came back as
sidebar labels + history titles + adjacent conversation snippets concatenated
with the real reply — silent SELECTOR failure (no thrown error).
Two minimal changes in `clis/doubao/utils.js` `getTurnsScript`:
1. `itemSelectors`: prepend `[class*="inner-item-"]` and `[class*="top-item-"]`
— the new 2026-05 wrappers. Outer wins via existing ancestor-keep dedup
below, so we get one root per turn (not one per nested chunk).
2. `getRole`: add a third fallback branch — if the root matches
`inner-item-*` / `top-item-*`, contains `.flow-markdown-body`, and has NO
`bg-g-send-msg-bubble` marker (User detection still works), treat it as
Assistant. `.flow-markdown-body` is already in `messageTextSelectors`, so
text extraction kicks in unchanged.
Test added asserting both new wrappers and the `.flow-markdown-body` assistant
fallback are present in the generated script.
Fixes#1478
* test(doubao): cover refactored assistant turns
* fix(youtube): request srv3 format for caption URLs (#1420)
YouTube may return empty responses when caption URLs lack an explicit format
parameter. This adds fmt=srv3 (standard YouTube XML caption format) to the
caption URL when no fmt parameter is already present, with a fallback to the
original URL if srv3 also returns empty.
Also adds HTTP status checking before reading the response body, preventing
silent failures on non-200 responses.
Fixes#1420
* fix(youtube): preserve caption fetch failures
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(reddit): add reply command for replying to comments
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(reddit/reply): replace silent-sentinel rows with typed errors
reply.js originally mirror-copied comment.js's failure pattern: returning
[{ status: 'failed', message: 'HTTP 403' }] on auth/HTTP/Reddit errors and
relying on the caller to inspect the row instead of throwing. That's the
'silent-sentinel' anti-pattern from typed-errors.md — failures should
surface as typed errors so an agent can actually branch on them.
Round 21 lesson (f) — "grandfathered-not-exempt + helper-refactor boundary
is new" — applies: comment.js / upvote.js / save.js can stay grandfathered,
but a brand-new file does not inherit that exemption.
Changes:
- Throw AuthRequiredError when /api/me.json or /api/comment returns 401/403,
or when /api/me.json returns 200 but data.name is missing (stale anon
session — empty modhash alone isn't a strong enough signal).
- Throw CommandExecutionError for non-2xx HTTP and for non-empty
data.json.errors (e.g. RATELIMIT, NO_TEXT, TOO_OLD).
- Drop the over-defensive `if (!page) throw ...` — registry guarantees a
page object when browser:true.
- Intermediate result object uses `kind` discriminator + `detail` /
`httpStatus` / `where` keys that don't overlap with columns
['status','message'], so the silent-column-drop audit stays quiet
(per PR #1329 sediment).
Verified:
- npx tsc --noEmit clean
- node scripts/check-typed-error-lint.mjs → 189/189, 0 new
- node scripts/check-silent-column-drop.mjs → 103/103, 0 new
- npx vitest run clis/reddit src/convention-audit → 11/11 pass
- node ./dist/src/main.js validate → 0 errors
Success path is unchanged: still returns
[{ status: 'success', message: 'Reply posted on t1_<id>' }].
* fix(reddit): harden reply command contract
* fix(reddit): reject suffixed reply urls
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(rednote): add rednote.com adapter mirroring xiaohongshu read commands (#1136)
Implements rednote.com support as discussed in issue #1136. The mainland
xiaohongshu adapter stays in place; international users redirected to
www.rednote.com now have a CLI without a copy-pasted adapter.
Issue #1136 documents that xiaohongshu and rednote share DOM selectors,
URL paths, API paths, response schema, cookies, and the xsec_token auth
mechanism. The only material differences:
Layer xiaohongshu rednote
Web host www.xiaohongshu.com www.rednote.com
API host edith.xiaohongshu.com webapi.rednote.com
Security host fe-static.xhscdn.com as.rednote.com
Cookie root .xiaohongshu.com .rednote.com
Search gate Inline text Full-screen modal + text
## Architecture (minimal)
`clis/xiaohongshu/*` keep all selector / regex / extraction logic. Each
command file is touched minimally to export the IIFE or pipeline so the
sibling adapter can reuse it:
search.js + export const buildSearchExtractJs(webHost)
+ export const command = cli({...})
note.js + export const NOTE_EXTRACT_JS
+ export const command = cli({...})
comments.js + export function buildCommentsExtractJs(withReplies)
+ export parseCommentLimit
+ export const command = cli({...})
download.js + export function buildDownloadExtractJs(noteId)
(CDN allowlist now includes rednote alongside xhscdn)
+ export const command = cli({...})
user.js + export const USER_SNAPSHOT_JS
+ export const command = cli({...})
feed.js + export function buildFeedPipeline(webHost)
+ export const command = cli({...})
notifications.js + export function buildNotificationsPipeline(webHost)
+ export const command = cli({...})
note-helpers.js buildNoteUrl now accepts `cookieRoot` + `signedUrlHint`
options (defaults preserved so xhs callers and tests
are unchanged)
user-helpers.js buildXhsNoteUrl / extractXhsUserNotes accept an
optional `webHost` argument (default xhs)
The `export const command = cli({...})` pattern matches twitter/lists.js
and clis/discord-app/*; without it the build-manifest scanner attributes
xhs's command to whichever rednote sibling triggered the transitive
import first.
## clis/rednote/ — thin shims
Each rednote command file imports the relevant builder / constant from
its xiaohongshu sibling and calls `cli()` with the rednote host triple.
No selectors, regexes, or extraction logic are duplicated.
search.js imports buildSearchExtractJs + noteIdToDate
declares its own WAIT_FOR_CONTENT_JS (modal + text
login-gate variants — the one xhs behaviour that
genuinely differs)
note.js imports NOTE_EXTRACT_JS + buildNoteUrl + parseNoteId
comments.js imports buildCommentsExtractJs + parseCommentLimit
+ buildNoteUrl + parseNoteId
download.js imports buildDownloadExtractJs + buildNoteUrl + parseNoteId
user.js imports USER_SNAPSHOT_JS + extractXhsUserNotes
+ normalizeXhsUserId
## Scope (initial)
Ships the five commands verified live against the user's logged-in
rednote.com session: search / note / comments / user / download.
`feed` and `notifications` are intentionally left out. Both rely on
intercepting the xiaohongshu Pinia store at the `homefeed` / `you`
capture pattern; live verification on rednote returns `tap → dict
(error)` for the feed step, so shipping them would surface a broken
contract. The mainland xiaohongshu commands continue to work. Adding
the rednote-side feed / notifications is straightforward follow-up
work once someone with rednote access maps the network surface.
Creator-center commands (publish, creator-*) have no rednote
counterpart and stay xiaohongshu-only, per the reporter's note in #1136.
## Verification
- clis/xiaohongshu/ + clis/rednote/: 103/103 tests green
- npx tsc --noEmit: clean
- npm run build: 807 manifest entries (xhs 13 + rednote 5 + everything
else preserved)
- silent-column-drop / typed-error-lint: 103 / 189 baseline entries,
no new violations
- Live verify against the user's rednote.com session:
rednote search "travel" --limit 1 → real note row
rednote note <signed-url> → 7 field/value rows
rednote comments <signed-url> --limit 3 → 3 top-level rows
rednote user 5b21f6564eacab3b38f05c39 --limit 2 → 2 profile notes
Spaced 15–30s between runs per the xhs/rednote rate-limit guidance;
no write commands invoked. Regression check: xiaohongshu/feed on
the existing mainland session still returns the standard 6-field
rows after the refactor.
Closes#1136
* fix(rednote): tighten adapter failure boundaries
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Doctor's job is browser-bridge health diagnosis. The `--no-live` flag
let users skip the connectivity probe (= the core diagnostic), and
`--sessions` listed automation sessions (a separate concern not part of
health). Both flags accreted features that violated the command's
first-principles purpose.
Cleanup chain (removing dead code surfaced by the flag removal):
- `--no-live` / `--sessions` flags removed from `opencli doctor`
- `DoctorOptions.live` / `DoctorOptions.sessions` removed
- `DoctorReport.sessions` removed
- `[SKIP] Connectivity` render branch removed (always-live now)
- `listSessions()` removed (only consumer was doctor)
- `'sessions'` action removed from daemon-client protocol type
- `BrowserSessionInfo` type removed (no remaining consumers)
- extension `handleSessions` action handler removed (1.0.12)
- extension test "reports sessions per session" removed
- `OPENCLI_BROWSER_IDLE_TIMEOUT` test rewired to 'cookies' action
Verification:
- root typecheck + extension typecheck pass
- doctor.test.ts 17/17 pass
- extension/background.test.ts 49/49 pass
- typed-error-lint 189/189 baseline
- silent-column-drop 103/103 baseline
- build + extension build green
Follow-up to feat #1458 (registering tg-cli/discord-cli/wx-cli in
src/external-clis.yaml) — README and README.zh-CN had not been updated
to reflect the new entries.
Updates four spots in each README:
- intro paragraph that names example external CLIs
- "CLI Hub" highlight bullet
- "OpenCLI is not only for websites" bullet list
- the External CLI table itself
Add three local-first messaging CLIs to the External CLI registry so
agents can discover and install them via `opencli external install`:
- `tg-cli` (binary `tg`) — Telegram local sync/search/export via MTProto
- `discord-cli` (binary `discord`) — Discord local sync/search/export
- `wx-cli` (binary `wx`) — WeChat local data CLI
Refresh the External CLI list in skills/opencli-usage/SKILL.md so the
agent-facing skill names stay in sync.
Extends A0 (PR #1404) by dogfooding `installCommanderNamespaceStructuredHelp`
on the four remaining built-in Commander namespaces:
- `opencli daemon --help -f yaml|json`
- `opencli plugin --help -f yaml|json`
- `opencli adapter --help -f yaml|json`
- `opencli profile --help -f yaml|json`
Each emits the same payload shape as `browser`: namespace metadata, every
leaf command's positionals + command_options + description + usage,
namespace_options (empty for these), and program-level global_options.
Agents can fetch every leaf's contract in a single call — no per-leaf
`--help` follow-ups.
Each namespace snapshots its original description at declaration time
because `applyRootSubcommandSummaries(program)` later overwrites
`.description()` with a child-name listing; without the snapshot,
structured help would surface `"restart, status, stop"` instead of
`"Manage the opencli daemon"`. Tests lock the snapshot semantics for
`adapter` explicitly.
Tests: 138/138 (4 new — one per namespace, covering description
preservation, leaf names, positionals, command_options).
Typecheck + build clean.
* perf(reddit): opt 13 browser adapters into shared site-tab lease
Adds `browserSession: { reuse: 'site' }` to every reddit adapter that
already runs `browser: true` on `domain: 'reddit.com'`. Same metadata-only
follow-up to the twitter sweep merged in #1454 — the framework's
`shouldRunPreNav` short-circuit (src/execution.ts:190) skips the redundant
domain-root pre-nav when a sibling adapter already has the tab on
reddit.com, and idle-bound tabs are reused under the `site:reddit` bucket
until expiry.
Scope (13 files, all on `domain: 'reddit.com'` + `Strategy.COOKIE`):
- read (9): frontpage / popular / saved / search / subreddit / upvoted /
user / user-comments / user-posts
- write (4): comment / save / subscribe / upvote
Excluded:
- `hot.js` (no browser:true — public Reddit JSON API, no tab)
- `read.js` (Strategy.COOKIE but no browser:true — non-browser pipeline)
No logic changes; only metadata + manifest regeneration.
Verification:
- npm run check:typed-error-lint → 189/189 unchanged
- npm run check:silent-column-drop → 103/103 unchanged
- npm run test:adapter → 264/264 passed (2146 tests)
- npx vitest run --project unit → 72/72 passed (unrelated EADDRINUSE
flake on daemon.test.ts port 19825, also seen on #1454/#1452)
- tsc --noEmit clean
* fix(reddit): include read in site browser session reuse
The uploadImages function catches errors from page.setFileInput and only
falls back to the legacy base64 DataTransfer method when the message
contains 'Unknown action' or 'not supported'. However, Chrome can also
return 'Not allowed' (code -32000), which was not handled — causing the
publish command to fail instead of using the fallback.
Add 'Not allowed' to the fallback condition so image upload works even
when CDP file injection is blocked by Chrome's security policy.
Co-authored-by: together <together@togetherdeMac-mini.local>
* feat(openreview): add author command for ID-explicit publication lookup
Closes the missing leaf in the openreview adapter. Among the public-strategy
academic adapters, dblp and arxiv both already ship an `author` command for
ID-explicit publication lookup; openreview only had `search` (full-text),
`paper` (detail by note id), `reviews` (thread by forum id) and `venue`
(listing by invitation / venue text). There was no way to ask "give me every
submission this author put on OpenReview, newest first."
`openreview author <profile>`:
- takes a canonical profile id (`~First_LastN`); validated by
`requireProfileId` so a dblp PID or a bare name fails before any
network call,
- hits `/notes?content.authorids=~<id>&limit=<n>&sort=cdate:desc`,
- returns rank-ordered rows with the same shape as `openreview search`
(id / title / authors / venue / pdate / url),
- throws `EmptyResultError` when the profile has no public submissions
instead of returning an empty list,
- inherits the typed-error envelope from `openreviewFetch` so network
failure, non-200, malformed JSON, and in-band error envelopes all
surface as `CommandExecutionError`.
Tests: 6 new `it` blocks plus 1 updated registration test in
`clis/openreview/openreview.test.js`.
- `requireProfileId` (1 block, 9 assertions): accepts canonical
`~First_LastN`, `~Bo_Liu17`, and a multi-segment middle-name id;
rejects empty, whitespace, missing tilde, missing trailing number,
embedded space, and a dblp-style PID.
- 5 author runtime cases covering pre-network ArgumentError, empty
result, non-200, fetch network error, and the happy path with a
request-shape assertion (`content.authorids` filter + `cdate:desc`
sort).
- Registration test extended to expect five commands and lock the new
`columns` contract.
Manifest auto-regenerated to register the new command.
Live-verified end to end against `~Yoshua_Bengio1`: the most recent ICLR
2026 workshop submissions return with the expected fields. A malformed
profile is rejected before any HTTP call. A nonexistent profile yields
`EMPTY_RESULT`.
* fix(openreview): accept real profile id slugs
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Convert 9 of 18 page.wait(N) calls in clis/claude/ from fixed-duration
sleeps to event-based readiness checks (page.wait({ selector, timeout }),
backed by MutationObserver). Mirrors the deepseek D1 template (PR #1449).
* Page-ready waits (5 converted): utils.js:29 (ensureOnClaude composer),
utils.js:114 (getConversationList recents links), new.js:19 (composer),
detail.js:24 (.font-claude-response message bubble), send.js:26 (composer).
Each resolves as soon as the selector matches, swallowing the timeout so
downstream typed-error helpers (ensureClaudeLogin / ensureClaudeComposer
/ EmptyResultError) still surface the right error when the selector
never mounts (login redirect, empty conversation, etc).
* Dropdown waits (2 converted): utils.js:150 (selectModel post-trigger),
utils.js:178 (setAdaptiveThinking post-trigger). Wait for menuitemradio
/ menuitem to mount instead of a fixed 0.6 s sleep.
* Resume conversation wait (1 converted): ask.js:51 — wait for the resumed
message bubble (MESSAGE_SELECTOR) instead of a fixed 2 s sleep.
* Settle/redundant waits removed (6): ask.js:55 standalone settle (next
ensureClaudeComposer queries composer presence directly via getPageState);
ask.js:83 / ask.js:90 post-toggle settles (next CDP eval flushes React
state between roundtrips); ask.js:103 pre-waitForResponse settle (the
polling loop's first 3 s tick already covers this); read.js:20 post-
ensureOnClaude sleep (ensureOnClaude now waits for the composer selector
itself); send.js:29 post-ensureOnClaude sleep (same).
Three remaining page.wait(N) calls are kept: utils.js:231 post-input
1.2 s React debounce inside sendMessage (the ProseMirror editor needs a
debounce window before the send button enables; reducing this risks
silent send-button-disabled drops), and the 3 s / 1 s polling ticks in
waitForResponse / waitForFilePreview (already polling patterns, out of
scope for D-track wait→event sweep).
Targeted tests: clis/claude + src/browser 389/389 pass; tsc clean;
build clean; typed-error 189/189 baseline (no new); silent-column-drop
103/103 baseline (no new).
D2 in the LLM-adapter wait→event sweep started by deepseek (D1, #1449).
Convert 10 of 18 `page.wait(N)` calls in clis/deepseek/ from fixed-duration
sleeps to event-based readiness checks (`page.wait({ selector, timeout })`,
backed by MutationObserver):
* Page-ready waits (5): utils.js:46, ask.js:38/52, detail.js:28, new.js:19
now wait for the composer textarea (TEXTAREA_SELECTOR) or message bubble
(MESSAGE_SELECTOR) to mount before continuing. Resolves as soon as the
selector matches instead of always sleeping the full duration.
* Settle/redundant waits removed (5): ask.js:56 standalone settle (already
covered by upstream selector waits); ask.js:79/105 post-toggle settles
(next CDP eval gives React time to flush aria-checked updates); ask.js:118
pre-waitForResponse settle (the polling loop's first 3 s tick already
covers this); read.js:19 post-ensureOnDeepSeek sleep (ensureOnDeepSeek
now waits for the textarea selector itself).
* `new.js` now throws CommandExecutionError when the composer fails to
mount within 8 s instead of silently returning "New chat started" on a
half-loaded or logged-out page.
Eight remaining `page.wait(N)` calls are kept: in-loop polling ticks in
waitForResponse / pickResumeUrl / getConversationList / waitForFilePreview
/ send-button-enable polling (these are already polling patterns and
out of scope for D1), and the native-input flush + textarea-mount poll in
send.js.
Targeted tests: clis/deepseek 49/49, src/browser 355/355 pass; build,
typecheck, typed-error and silent-column-drop audits clean.
Proof template for the LLM-adapter wait-cleanup follow-ups.
Read-only Twitter/X adapters now declare `browserSession: { reuse: 'site' }`,
matching the LLM-site adapters (claude/gemini/yuanbao/etc.) and unblocking
the perf wins WAWQAQ called out for the 35s→9s/3.4s thread.js progression
(#OpenCLI:3889b5cf):
- Tab lease shared across calls under `site:twitter` until idle expiry, so
the second-and-later command pays no cold-start tab cost.
- Framework's domain-root pre-nav (`https://x.com`) is skipped on subsequent
calls when the reused tab is already on x.com (`shouldRunPreNav` →
`isDomainRootPreNav` + `urlMatchesDomain` short-circuit at
`src/execution.ts:190`).
Files (17 read-only adapters):
- Strategy.COOKIE × 13: article, bookmark-folder, bookmark-folders,
bookmarks, download, following, likes, list-tweets, lists, profile,
thread, timeline, trending, tweets
- Strategy.UI × 1: followers
- Strategy.INTERCEPT × 2: notifications, search
Insertion point in each file: after `browser: true,` (or after `strategy:`
in download.js which omits the explicit `browser:` field), matching the
convention used by yuanbao/read.js, claude/read.js, etc.
Manifest regenerated (cli-manifest.json: +85/-17 — 17 entries gain the
`browserSession: { reuse: "site" }` block).
Verification:
- npx tsc --noEmit clean
- npx vitest run clis/twitter → 218/218 pass (25 files)
- npx vitest run src/convention-audit.test.ts → 8/8 pass
- typed-error-lint baseline 189/189 (no new violations)
- silent-column-drop baseline 103/103 (no new violations)
Scope notes (intentionally NOT in this PR):
- Write adapters (post/reply/quote/like/retweet/bookmark/follow/list-add/
list-remove/delete/hide-reply/block/accept/follow) are kept as one-shot
by default — `reuse: 'site'` for write paths is a separate decision
about action idempotency under tab reuse.
- The thread.js / timeline.js comments still say "Cookie context
auto-established by framework pre-nav"; the deeper truth (CDP
`getCookies({url})` is origin-independent) was a framing nit on PR C
(#1451) — left as a doc-only follow-up to keep this PR's diff focused
on the perf gain.
Refs: #OpenCLI:3889b5cf (WAWQAQ msg=fa209a2c, msg=35c90460, msg=838128ef
"你们继续做啊… 后面还有那么多其他的东西呢")
* perf(twitter): drop redundant goto+wait — framework auto pre-navs (PR C)
Twelve twitter read adapters did `await page.goto('https://x.com'); await
page.wait(2~3)` purely to establish cookie context for the subsequent
`document.cookie` read. After PR #1450 hoisted those reads to
`page.getCookies({url})` (which queries the CDP cookie store directly,
no navigation needed), the explicit goto+wait became dead.
The framework already pre-navigates to `https://${domain}` for any
adapter declaring `Strategy.COOKIE + domain` (`src/registry.ts:191`),
so the cookie store is populated before `func` runs. The 2-3s
`page.wait` was the slowest part of the redundant call.
Files (all read-only, all ct0/cookie-only):
- bookmark-folder / bookmark-folders / bookmarks
- following / likes / list-add / list-remove / list-tweets / lists
- thread / timeline / tweets
Out of scope (kept as-is): goto calls that navigate to a *specific*
URL needed for content/SPA shell — `trending` (`/explore/tabs/trending`),
`notifications` (`/home`), `article` (`/i/article/{id}`), `profile`
(`/${username}`), and `list-add` line 133 (`/${username}` for UI ops).
Verification:
- npx tsc --noEmit ✓
- npx vitest run clis/twitter → 216/216 ✓
- typed-error-lint 189/189, 0 new ✓
- silent-column-drop 103/103, 0 new ✓
* fix(twitter): keep list UI root navigation
* perf: replace document.cookie reads with page.getCookies({domain}) (Tier 1 cookie API sweep)
Prior pattern in 25 adapter files round-tripped through `page.evaluate(\`document.cookie.split…\`)` to extract a single cookie value (CSRF token, session ID, etc.). CDP's `page.getCookies({domain})` reads the cookie store directly with zero JS-execution overhead.
Files touched (sites: twitter / linkedin / maimai / youtube):
- twitter (15): thread, timeline, list-add, bookmark-folders, following, list-tweets, bookmarks, list-remove, tweets, bookmark-folder, likes, lists, trending — direct 4-line replacement (cookie was outside `page.evaluate`); article, profile — hoisted ct0 read OUT of `page.evaluate` and threw `AuthRequiredError` upfront so unreachable in-evaluate auth branches got cleaned up too.
- linkedin/search.js — JSESSIONID was read inside the per-batch fetch loop's `page.evaluate`; hoisted once before the loop and pass `csrf` value into the template via `JSON.stringify`.
- maimai/search-talents.js — csrftoken cookie hoisted via getCookies; meta-tag fallback preserved inside `page.evaluate` (reached only when no cookie). Also converted the `page.evaluate(async (body) => …, body)` Playwright-style call to OpenCLI's template-string form so the helper actually runs.
- youtube — `SAPISID_HASH_FN` (used by like / unlike / subscribe / unsubscribe) reworked: sapisid is now passed in as a parameter; new `readYoutubeSapisid(page)` helper reads it via CDP. The HMAC-SHA1 compute still happens browser-side (Web Crypto), only the cookie read is hoisted.
Tests updated where mocks specifically referenced `document.cookie` (twitter following / bookmark-folder / bookmark-folders) to mock `getCookies` instead.
Verification:
- `npx tsc --noEmit` clean
- `npx vitest run clis/twitter clis/linkedin clis/youtube` → 264/264 pass
- typed-error-lint 189/189 (no new violations)
- silent-column-drop 103/103 (no new violations)
Scope notes (not in this PR):
- `goto + wait` redundancy and `browserSession: { reuse: 'site' }` rollout are scoped to follow-up PRs B and C per the #OpenCLI:3889b5cf thread plan.
- `document.cookie.match(...)` patterns (instagram 8 / xiaoe / qwen / hupu / tiktok / 1point3acres — ~13 files) are outside the original \`document.cookie.split\` audit scope and will follow as a Tier 1 expansion sweep.
* fix(adapters): read auth cookies by url scope
- bump opencli to 1.7.15 (was 1.7.14)
- extension stays at 1.0.9 (already bumped during the release cycle)
- finalize CHANGELOG: move Unreleased to 1.7.15 with date
Major release: Browser Agent Runtime project (Phase 0/1/2) — alignment
with vercel-labs/agent-browser model. CDP-primary input, AX snapshot/refs
with stale recovery, semantic locators across all primitives, full form
toolbelt (hover/focus/dblclick/check/uncheck/upload/drag/wait-download),
annotated screenshots, and same-origin iframe AX routing.
PR #1399 added auto-restart of stale daemons in BrowserBridge
(daemonVersion ≠ PKG_VERSION → restart). The browser-tabs e2e fake
daemon hard-coded `daemonVersion: 'test'`, so every test reported as
stale and the bridge tried to /shutdown the fake daemon — which has no
shutdown endpoint — causing all 4 tests in the file to exit with code 1.
This has been the failing signal in `e2e-headed (ubuntu-latest)` and
`e2e-headed (macos-latest)` on every main push since #1399.
Read PKG_VERSION from package.json once at module load and feed that to
the fake /status response. The fake daemon now matches the running CLI
so the stale-daemon path is not triggered.
Verification:
- npx tsc --noEmit clean
- npm run build clean
- npx vitest run --project e2e tests/e2e/browser-tabs.test.ts → 4/4 pass
* feat(dianping): resolve unknown cities live from www.dianping.com
The static CITY_ID map in clis/dianping/utils.js only covers ~20 cities,
so passing --city 汕头 (or any other Chinese name / pinyin slug not on
that list) fails with ArgumentError. Adding the missing cityIds by hand
doesn't scale to dianping's full city list and silently goes stale when
the site renumbers cities.
This change adds an async resolver that falls back to dianping.com when
the static map misses:
- Numeric input → pass through unchanged.
- Static map hit → fast path, no network (utils.CITY_ID untouched).
- Pinyin slug (e.g. "shantou") → goto /<slug>, parse cityId out of
any /search/keyword/{id}/ link rendered on the per-city landing page.
- Chinese name (e.g. "汕头") → goto /citylist, walk anchors to build a
Chinese-name → pinyin map, then resolve the slug as above.
Resolved (input → cityId) pairs are memoized per-process so repeat
searches skip both navigations.
Implemented as a new module (clis/dianping/cityResolver.js) so utils.js
stays minimal and the existing synchronous resolveCityId / CITY_ID API
keeps working for direct callers and tests.
Tested:
- Unit tests cover null/numeric/static fast paths, pinyin fallback +
cache, Chinese-name fallback via /citylist + cache for both forms,
rejection of garbage input, rejection of Chinese names not on
/citylist, and CommandExecutionError when the per-city page lacks
a /search/keyword/{id}/ link.
- JSDOM tests cover the pure DOM extractors (buildCitylistMap and
extractCityIdFromPage) against curated HTML fixtures.
- npm test: 3196 passed, 1 skipped (no new failures).
- npx tsc --noEmit: clean.
- opencli validate: 0 errors.
* fix(dianping): require city resolver links to be authoritative
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(douyin): handle empty response body in browserFetch (#1405)
browserFetch calls res.json() directly, which throws SyntaxError when
the API returns an empty body (content-length: 0). This happens when
the Douyin hashtag search endpoint returns HTTP 200 with no content.
Fix: read response as text first, return null for empty bodies, then
throw a descriptive CommandExecutionError at the caller level.
Fixes#1405
* fix(douyin): wrap browser fetch parse failures
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Why
- `opencli twitter followers --help` rendered:
Arguments:
user
with a blank trailing column. Both humans and agents could not
recover the parameter's purpose without reading source. WAWQAQ
surfaced this directly: "没有说明当后面的 followers [user] [options]
如果都没填的时候，获取的是什么？"
- This is metadata completeness, not stylistic taste. Failing closed
is the only way to keep the help surface trustworthy as adapters
land.
What
- src/build-manifest.ts: add `findManifestMetadataIssues()` that flags
any positional with empty / whitespace-only / missing `help`. Wired
into `main()` after the import-failures gate; build aborts non-zero
with a per-arg report (`site/cmd positional "name" (sourceFile)`).
- src/build-manifest.test.ts: cover the gate (positives + negatives,
scoped strictly to positionals — named flags are intentionally
out-of-scope).
- 18 adapter offenders (16 required + 2 optional) get explicit help
text:
twitter: followers/following/list-add/list-remove/list-tweets/
search/thread
reddit: search/subreddit/user/user-comments/user-posts
douyin: stats/update
bilibili: subtitle
jike: search
Optional positionals (`twitter followers/following [user]`) now
document the omit semantics — fetches the currently logged-in
account.
- CHANGELOG: document the build gate and the offender list.
Out of scope (planned follow-ups)
- Semantic-quality advisory: optional positional help should also
contain `default / omit / current / logged-in / required unless …`
keywords. That belongs to the planned Arg metadata v2 work
(`when_omitted / when_present / value_format` 3-field schema).
- Named-flag `help` quality. Named flags carry the flag name itself
in help, so a missing `help` is not as opaque; if we want to gate
those too, do it as a separate, intentional decision.
Validation
- `npm run build` → 799 entries, clean.
- `npm run typecheck` → clean.
- `npx vitest run --project unit --project adapter` → 257 + 4 files,
all green (build-manifest 13 tests, manifest gate added).
- Smoke: temporarily reverted `followers.js` help to empty → build
aborts with the exact `twitter/followers positional "user" (...)`
line; restored, build is clean again.
- `npm run check:silent-column-drop` and `check:typed-error-lint`
baselines unchanged.
- bump opencli to 1.7.14 (was 1.7.13)
- extension stays at 1.0.6 (no extension changes since v1.7.13)
- finalize CHANGELOG with the three landed PRs:
* #1399 daemon restart on stale ready state for npm -g upgrade
* #1400 twitter write-action symmetry (unlike/retweet/unretweet/quote)
* #1401 agent-friendly adapter help (drop globally-shared option noise)
Fixes#1376 — YouTube transcript command failed with `No captions available for this video` for all videos.
## Root cause
Transcript adapter used InnerTube `/youtubei/v1/player` API with Android client context (`clientName: 'ANDROID'`, version `20.10.38`) to retrieve caption track URLs. YouTube has restricted/deprecated this approach; the Android client no longer reliably returns captions data.
## Fix
Replace Step 1 (caption track retrieval) with watch page HTML bootstrap parsing — fetch `/watch?v=...` with cookies and extract `ytInitialPlayerResponse.captions.playerCaptionsTracklistRenderer`. This is the same approach used by sibling `clis/youtube/video.js`, so it's an alignment to existing site-local stable pattern, not a new invention.
## 2 head iteration
- `cf77f5e8` initial fix (Step 1 caption retrieval switch + 18/18 unit tests)
- `bb30788c` lead test hardening — source-contract regression test in `transcript.test.js`:
- **positive lock**: must fetch `/watch?v=...`, parse `ytInitialPlayerResponse`, read `playerCaptionsTracklistRenderer`
- **negative lock**: must NOT use `/youtubei/v1/player` or `clientName: 'ANDROID'` (prevents regression)
- stale Android-InnerTube file header comment also updated
## Better-solution evaluation
- Official YouTube Data API captions surface (`developers.google.com/youtube/v3/docs/captions/download`) is owner-authorized API, NOT a public transcript replacement
- yt-dlp also relies on watch-page bootstrap path
- Existing `youtube/video.js` already uses the same `ytInitialPlayerResponse` extraction → this PR aligns transcript with stable site-local pattern instead of inventing a new path
## Typed failure / no-silent-empty boundaries
- watch HTML HTTP failure / missing `ytInitialPlayerResponse` / no `captionTracks` → `CommandExecutionError` (typed fail)
- Empty parsed XML → `EmptyResultError` (existing path, preserved)
- `Strategy.COOKIE` matches YouTube adapter family + `video.js`; cookies/session/consent unavailable → typed fail not silent empty success illusion
## Diff containment
Runtime change limited to Step 1 caption track discovery. XML fetch, segment parsing, chapters, raw/grouped formatting all unchanged.
## Verification
Local: YouTube adapter tests `19/19` (+1 from new test), `npm run build`, typed-error-lint `192/192`, silent-column-drop `103/103`, doc coverage `140/140`, `docs:build`, listing-id advisory unchanged `13`, `git diff --check`, merge-tree clean.
GitHub: build × 3 OS, unit × 2 shards, bun-test, adapter-test, audit, doc-coverage, docs-build all SUCCESS. PR CLEAN/MERGEABLE.
Author: kagura-agent (fork). Lead: codex-mini0. Aux: First-principles-0. Coordination: pr-monitor.
Xiaohongshu image-note publishing reliability fixes for creator center UI (legacy raw-Error write command, not a typed-error migration).
## 3 changes (one publish-path repair)
1. **Open creator publish in image mode**: append `target=image` to the publish URL so it loads directly in image mode instead of default
2. **Exact `图文` tab priority**: prefer exact tab text matching before broad `startsWith/includes`, reducing parent-container misclicks while keeping fallback for UI wording variants
3. **DataTransfer fallback for `Chrome Not allowed`**: when CDP `setFileInput` returns the permission/bridge denial error, fall through to the existing DataTransfer upload path (CDP-first remains primary to avoid base64 bridge/payload limits)
## Lead hardening (`edf8107d`)
Added `clis/xiaohongshu/publish.test.js` regression coverage for all three claimed behaviors:
- `target=image` creator URL locked
- exact tab text matched before broad fallback
- `Chrome Not allowed` falling into DataTransfer path
## Better-solution evaluation (lead + aux 一致)
- **CDP-first kept**: CDP avoids base64 payload/bridge limits; `Not allowed` is a known permission failure class where fallback is appropriate. DataTransfer-first would weaken the common path and reintroduce large-payload fragility.
- **Exact tab text first**: XHS creator markup is private and volatile, selector-only alternative not clearly more stable. Exact text reduces misclicks while broader fallback + post-click `video_surface` check preserve resilience for wording shifts. If exact text disappears, command fails fast with screenshot instead of silent video-mode publish.
- **Scope boundary self-imposed**: not expanding to typed-error migration (publish.js is legacy raw-Error and typed-error-lint already accounts for it).
## Verification
Local: xiaohongshu publish tests `12/12`, typecheck, build/manifest, docs:build, typed-error-lint `189/189`, silent-column-drop `103/103`, doc coverage `140/140`, node --check, git diff --check.
GitHub: build × 3 OS, unit shards, bun-test, adapter-test, audit, docs-build, doc-coverage all SUCCESS. PR CLEAN/MERGEABLE.
Author: E2ern1ty (fork). Lead: codex-mini1. Aux: First-principles-1. Coordination: pr-monitor.
* chore(release): pre-release P0/P1 cleanup
P0 fixes:
- delete src/analysis.ts (179 lines, 0 importers across src/clis/extension)
- remove dead OPENCLI_DIAGNOSTIC negative test assertion
- rename OPENCLI_BROWSER_TIMEOUT to OPENCLI_BROWSER_IDLE_TIMEOUT — the env
controls workspace lease idle release, not command runtime; old name was
misleading and undocumented (no fallback needed)
- add 'fill' to validate.ts KNOWN_STEP_NAMES so adapters using PR #1222's
fill pipeline step do not trip "unknown step name" warnings during validate
P1 fixes:
- BrowserConnect daemon-not-running hint: replace stale "make sure port is
available" with actionable "run opencli doctor / opencli daemon restart"
- TimeoutError hint: lead with --timeout flag, demote env var to secondary
* fix(validate): derive step allowlist from pipeline registry
@pr-monitor flagged the prior "add 'fill' to KNOWN_STEP_NAMES" fix as
treating only the symptom — two parallel hand-maintained lists will keep
drifting whenever a new pipeline step is registered.
Address the root cause: pipeline/registry.ts now exports
`getRegisteredStepNames()` and validate.ts builds KNOWN_STEP_NAMES from
that. Adding a step via `registerStep()` automatically allowlists it.
* test(validate): regression guard for pipeline step allowlist linkage
@pr-monitor follow-up: lock the validate ↔ pipeline registry linkage at
the test layer so future drift is caught immediately.
Changes:
- recompute KNOWN_STEP_NAMES per-call (was const at module load) so
steps registered after validate.ts import (plugins, dynamic registration)
are honoured
- add src/validate.test.ts with 3 cases:
1. every step name from getRegisteredStepNames() exists
2. an adapter using every currently registered step does not warn
3. a step registered at runtime is automatically allowlisted by
validate without any source change to validate.ts
* fix(capabilityRouting): add fill to BROWSER_ONLY_STEPS
Same double-list drift pattern as validate.ts KNOWN_STEP_NAMES (audit
follow-up flagged in this PR's evolution thread). The fill step was
registered in pipeline/registry.ts (PR #1222) but never added to the
browser-only allowlist in capabilityRouting.ts.
Concrete impact:
- shouldUseBrowserSession() didn't recognize a `[{ fill: ... }]` pipeline
as needing a browser, so PUBLIC adapters using fill could end up
without a page and crash inside stepFill at `page!.fillText(...)`
- pipeline/executor.ts's per-step retry policy (BROWSER_ONLY_STEPS gets
2 retries on transient errors, others get 0) skipped fill — losing
retry coverage on a DOM-touching step
Fix:
- add 'fill' to BROWSER_ONLY_STEPS
- add a documenting comment explaining BROWSER_ONLY_STEPS is the
browser-touching subset of registered steps (not the full set)
- export _validateBrowserOnlyStepsAgainstRegistry() so the test layer
catches the inverse drift (browser-only step that no longer exists)
- 3 new tests in capabilityRouting.test.ts:
* pipeline with fill routes to browser session
* BROWSER_ONLY_STEPS subset of registered step names
* fill is in both lists
This addresses @pr-monitor follow-up #3 (audit similar double-list
patterns) for the obvious in-scope candidate. Other candidates outside
this PR's scope: build-manifest serialization vs registry shape, error
code unions vs lint baselines.
* test(validate): use Strategy.PUBLIC enum instead of string cast in regression test
Self-review nit: `strategy: 'public' as never` worked but bypassed the
typed CliOptions union. Use `Strategy.PUBLIC` so the test exercises the
real public API.
Wire up the standard browser-LLM command surface for Yuanbao, matching the
recently shipped chatgpt + claude + qwen baselines:
- status — login + current model + (agentId, convId) + URL
- read — render the visible conversation as User/Assistant rows
- detail — open `<agentId>/<convId>` and read its messages
- history — list sidebar conversations with stable IDs
- send — fire-and-forget, returns once the send button has been clicked
Refactor `ask.js` to share helpers (`sendYuanbaoMessage`, `normalizeBooleanFlag`)
with the new commands via `shared.js`, keeping the public ask behavior intact.
Notable bits:
- `parseYuanbaoSessionId` accepts only full chat URLs or `<agentId>/<convId>`
pairs — Yuanbao chat URLs encode both, and silently opening the wrong agent
on a bare UUID is a worse failure mode than throwing. URL regex anchored
with `(?:[/?#]|$)` so 37+ char tails reject rather than truncate.
- `sendYuanbaoMessage` polls the send button (up to 3s) for the React
re-render that drops `style__send-btn--disabled___*` after composer input —
a fixed wait raced the debounce and produced silent no-op clicks.
- `getYuanbaoMessageBubbles` uses `data-conv-id`/`data-conv-idx`/
`data-conv-speaker` attributes for stable per-turn identity (was relying
on innerHTML alone).
- Status surfaces both human label (`Yuanbao`) and `dt-model-id`
(`hunyuan_gpt_175B_0404`) — sentinel strings would silently look like a
real model name; null is the typed-unknown signal.
Verified: 25 unit tests pass; targeted live smoke for status/read/detail/
history/new/send + ask round-trip on yuanbao.tencent.com.
* feat(qwen): add detail command + fix stale message bubble selector
`getMessageBubbles` was matching `[data-msgid="<id>-question|answer"]` from an
older Qianwen frontend. The reshipped DOM no longer carries that attribute on
chat turns; `[data-message-id]` now lives on citation cards inside assistant
responses, so the old selector silently returned an empty list and `qwen read`
had been silently broken.
Rewire to walk `[data-chat-question-wrap]` and `[data-chat-answers-wrap]` in
DOM order (correct Q/A interleaving) and synthesize stable IDs from the
nearest sibling `data-req-id` so `waitForAnswer.seenAssistantId` and
read/ask/detail dedupe paths keep working. Verified live against an existing
conversation: 3 user turns + 3 assistant turns extracted; old selector
returned 0.
`qwen detail <id|url>`: open a specific conversation by ID or full chat URL,
poll up to 20s for the transcript to render, return Role/Text rows. Adds
`parseQianwenSessionId` (5 unit tests covering ID/URL parsing + ArgumentError
on malformed input). Reuses the same site-level browser session as `read`/
`ask` so consecutive calls continue in the same Qwen tab.
- clis/qwen/detail.js (new)
- clis/qwen/utils.js (parseQianwenSessionId + getMessageBubbles rewire)
- clis/qwen/utils.test.js (new)
- docs/adapters/browser/qwen.md (detail entry + options/columns)
- cli-manifest.json (regenerated)
* fix(qwen): anchor URL regex to reject 33+ hex tail truncation
codex-coder review on PR #1390 caught that
`https://www.qianwen.com/chat/<33+ hex>` would silently truncate to the
first 32 chars and open the wrong conversation. Adds end-of-input /
slash / query / fragment boundary to the URL match group and two new
unit-test cases (digit tail + letters tail) covering the truncation gap.
Add ChatGPT web ask/send/read/history/detail/new/status alongside existing image support. Tighten ChatGPT web helper selectors and typed error contracts, update docs/changelog, regenerate manifest, and seed local ChatGPT verify fixtures for ask/read.
* test(gov-policy): JSDOM-against-frozen-fixture tests for in-browser extractors
Applies the pattern documented in skills/opencli-adapter-author/references/jsdom-fixture-pattern.md
(introduced in #1319 alongside the dianping reference test in #1313) to the
gov-policy adapter.
Refactor: the inline IIFE inside `page.evaluate` template literal is hoisted
to a top-level `extractSearchRows` / `extractRecentRows` function using bare
`document` / `location`. Same code now runs identically in:
- the live browser (injected via `${extractor.toString()}`)
- JSDOM unit tests (with `globalThis.document` / `globalThis.location` swapped)
Tests:
- 6 new cases in clis/gov-policy/gov-policy.test.js (was commands.test.js).
- 3 representative search result cards (1 with real article snippet, 2 with
only publish-time in `.description`) and 5 recent listing rows in the
fixtures.
- ok:false fallback path covered for both extractors.
- Lock-in: `要闻` type-tag prefix fusion in title and empty-source contract
on recent listings (no `.source` / `.from` elements on that page) are
asserted explicitly so a future selector tweak can't silently change them.
Reverse-validated against two buggy variants per the reference doc:
breaking the title selector and stripping the `要闻` prefix both fail the
JSDOM assertions with helpful diffs.
Fixture sanitization follows the reference doc step-by-step: scripts /
styles / iframes / comments / preload links stripped, image srcs replaced
with `placeholder.png`, trimmed to the minimum subtree that exercises the
extractor (3 search items, 5 recent rows), all whitespace-only lines
removed.
* fix(gov-policy): use typed errors for touched commands
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* enrich(coupang): add product detail cmd + replace silent clamp/sentinel/Error with typed errors
Two enrichment changes plus three silent-failure fixes on top of existing
search / add-to-cart.
New cmd: coupang product
─────────────────────────
Pairs with search as the listing↔detail round-trip target. Reads a logged-in
product page and extracts a single canonical row with price, original_price,
discount_rate, rating, review_count, seller, brand, rocket, delivery_promise,
image_url, url. Three-source extractor (JSON-LD Product schema → bootstrap
globals → DOM) merged in priority order, mirroring the search.js pattern.
The columns use string|null typing — null means "upstream did not provide
this field on this product" (e.g. some items have no original_price).
Failures (login wall / page mismatch / page failed to render) raise typed
errors instead of silently returning empty rows, so callers can treat any
returned row as real data.
Search column shape: added product_id
─────────────────────────────────────
Listing must pair with detail by id. The data was already extracted by
normalizeSearchItem; only the columns array needed updating so the field
projects through to the rendered row. Per the listing-id-pairing convention
(PR #1297) the new column lets agents round-trip rows directly into
`coupang product` without re-scraping URLs.
Silent-failure fixes
────────────────────
1. search --limit silent clamp.
Old: `Math.min(Math.max(Number(kwargs.limit||20),1),50)` silently
rewrote `--limit 999` to 50 and `--limit 0` to 1.
New: `parseLimitArg(raw, 20, 50)` throws ArgumentError on out-of-range
/ non-integer / negative input. Same convention as the typed-fail-fast
memory & PR #1289.
2. search --page silent clamp.
Old: `Math.max(Number(kwargs.page||1),1)` silently lifted negative pages.
New: parsePageArg throws ArgumentError on non-positive input.
3. Generic `throw new Error(...)` → typed errors.
- Empty query, unsupported --filter, missing --product-id/--url
→ ArgumentError
- Login wall detection → AuthRequiredError('coupang.com', ...)
- Empty result / filter-not-rendered → EmptyResultError
- PRODUCT_MISMATCH / OPTION_REQUIRED / button-not-found / unknown
ack failure (add-to-cart) → CommandExecutionError
- The PRODUCT_MISMATCH and `actualProductId || 'unknown'` sentinel were
also fixed (silent-sentinel was the audit hit there).
Coverage
────────
- 21 contract assertions in clis/coupang/coupang.test.js covering
parseLimitArg / parsePageArg (no silent clamp), registry shape (search has
product_id, product is read-class with expected columns, add-to-cart is
write-class), and typed-error pre-flight rejections (empty query / bad
filter / out-of-range limit & page / missing detail args).
- Manifest 763 → 764 (+1 entry: coupang/product).
- Audits: typed-error-lint 196 → 194 (resolved 2 silent-clamp/sentinel
baseline entries; baseline updated). silent-column-drop 103/103 unchanged.
* fix(coupang): tighten product id and browser errors
* fix(coupang): require real product urls
* refactor(linux-do): remove deprecated hot/category/latest compat shims
The three shims have been pure backward-compat wrappers since linux-do/feed
became the unified entrypoint. With no stable release commitment to preserve,
they are pure surface cost: 3 manifest entries, 3 deprecated branches in help
output, and a `buildLinuxDoCompatFooter` helper that exists only to feed them.
- delete clis/linux-do/{hot,category,latest}.js
- drop now-orphaned `buildLinuxDoCompatFooter` from feed.js and unexport
`executeLinuxDoFeed` (no external consumers remain)
- remove the Compatibility section in docs/adapters/browser/linux-do.md
- regenerate cli-manifest.json (-125 lines)
BREAKING CHANGE: `opencli linux-do hot|category|latest` are removed. Use
`opencli linux-do feed --view top --period <period>`,
`opencli linux-do feed --category <id-or-name>`, and
`opencli linux-do feed --view latest` instead.
* fix(linux-do): finish compat shim removal
* refactor(runtime): unify command timeout into a single --timeout arg
Drop the cli-level `timeoutSeconds` build-time ceiling field. A command
now opts into runtime-enforced timeouts purely by declaring an arg named
`timeout`; the user-facing `--timeout` value (its default or override)
is the single authoritative knob, used both by the adapter polling loop
and by the runtime ceiling (with a 30s padding for return + closeWindow
+ trace export).
Behavior:
- Browser commands without a `--timeout` arg fall back to
OPENCLI_BROWSER_COMMAND_TIMEOUT (default 60s, unchanged).
- Non-browser commands without a `--timeout` arg now run unbounded
rather than against the previously implicit `timeoutSeconds` cap.
Affected commands keep their old caps via newly added `--timeout` args.
- LLM adapters (gemini/claude/deepseek/doubao/qwen/yuanbao ask) keep
their current `--timeout` defaults; the runtime ceiling is now strictly
more generous (userTimeout + 30s vs. the previous 180s cap), so
`--timeout 600` actually buys 600s of polling rather than dying at 180s.
Closes the design discussion that started from PR #1227, which proposed
a per-site `OPENCLI_GEMINI_ASK_TIMEOUT` env var to work around the same
underlying mismatch.
* fix(timeout): wire --timeout arg into chatgpt/gemini image adapter polling
codex-coder review on PR #1364 caught that the new --timeout arg I added
to chatgpt/image and gemini/image only drove the runtime ceiling — the
adapter still hardcoded `const timeout = 120`, so users passing
--timeout 240/600 saw runtime allow 270s/630s but the adapter stop
polling at 120s. That recreated the same single-knob mismatch this PR
was meant to delete.
Also add the browser-path runWithTimeout assertion codex-coder flagged
as missing: a browser command with --timeout default=5 must call
runWithTimeout with timeout: 35; a browser command without --timeout
arg must fall back to DEFAULT_BROWSER_COMMAND_TIMEOUT.
Image adapters now read kwargs.timeout and reject non-positive-integer
values with ArgumentError (no silent fallback). chatgpt/image.test.js
updated to pass an explicit timeout when calling .func directly (the
test bypasses arg coercion).
* fix(runtime): reject invalid timeout ceilings
* fix(timeout): normalize timeout args to integer values
* fix(timeout): preserve remaining command ceilings
* fix(runtime): validate timeout before browser setup
* enrich(toutiao): hot board + bug fixes (silent column drop, partial render)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #3.
## New command
- `toutiao hot` (Strategy.PUBLIC, browser:false) — public homepage hot
board via the toutiao.com hot-event/hot-board endpoint. No login required.
Returns 8 stable columns (rank/id/title/query/hot_value/label/url/image).
## Bug fixes for `toutiao articles`
- **Silent column drop fixed**: `parseToutiaoArticlesText` previously
did `if (title && stats) push(...)`, silently dropping any row where
the stats span hadn't finished rendering by the time page.innerText
was read. Slow-render bugs were invisible — adapter looked "complete"
while writers saw extra rows in the dashboard. Partial rows now
surface with `null` stat columns.
- **Silent clamp on `--page` removed**: out-of-range / non-integer
values raise `ArgumentError` with explicit bounds [1, 4]. Same
validation reused by both `articles` and `hot` via `parseArticlesPage`
/ `parseHotLimit` in `utils.js`.
- **Empty result typed**: zero-row scrape now raises `EmptyResultError`
instead of returning `[]` silently (would otherwise look like a
legitimate "no articles" response).
## Refactor
- Parser logic extracted to `clis/toutiao/utils.js` (alongside hot-row
mapping, validators, and the hot-board URL constant).
- `articles.js` switches from declarative `pipeline:` to imperative
`func` form so `parseArticlesPage` validation can run before the
navigation step (declarative pipeline can't pre-validate args).
- Strategy is now explicit: `Strategy.COOKIE, browser: true` for
articles (creator dashboard is logged-in only).
## hot field map
`ClusterIdStr` (or numeric `ClusterId`) → id; `Title` → title;
`QueryWord` → query (falls back to title); `HotValue` → hot_value
(non-negative numeric, else null); `Label`, `Url`, `Image` →
respective columns. `pickImage` walks `Image.url` → first truthy
`Image.url_list[]`. Empty-title rows are dropped (returns null) before
ranks are densely re-assigned 1..N.
## Tests
29 contract assertions across `parseArticlesPage` / `parseHotLimit` /
`parseToutiaoArticlesText` / `mapHotRow` + registry-level shape checks
+ `hot` adapter func behaviour (typed errors / no silent clamp / fetch
failure paths / dense-rank).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: hot has `id` column (round-trippable when a
detail command lands later); advisory list unchanged.
## Manifest
757 → 758 entries (+1 for `hot`).
## Doc
- index.md: toutiao mode 🔐 → 🌐/🔐 (hot is public, articles is logged-in)
- toutiao.md: per-command mode/domain table + column docs + prerequisites
* fix(toutiao): tighten hot and articles contracts
* fix(linkedin): surface detail_error on --details (no silent catch / no silent empty)
The previous --details enrichment path had two indistinguishable failure modes
that both produced `description: '', apply_url: ''`:
1. `if (!job.url)` early return — row had no jobId, so we couldn't navigate.
2. `} catch {}` — page.goto / page.evaluate threw (network, timeout, parse error).
Callers couldn't tell "upstream had no description" from "we failed to fetch",
and the catch swallowed every error without logging. For an enrichment that
costs one page navigation per row, silent failure is especially harmful — users
just see an empty cell with no way to debug.
Fix: replace empty strings with `null` for missing/failed rows, add a new
`detail_error` column (string|null) carrying a short typed reason:
- 'no url' — row had no jobId
- 'fetch failed: <msg>' — page.goto / page.evaluate threw
- 'missing description' — page loaded but body was empty
- null — success
Every failure is also logged to stderr with the offending URL so debugging is
possible. Per-row failures still don't abort the batch (the original intent),
but they're now visible.
Tests: 13 new contract assertions in clis/linkedin/search.test.js covering
parseCsvArg, mapFilterValues (ArgumentError on unknown values), decodeLinkedinRedirect,
and 5 enrichJobDetails paths (no-url / goto-throw / empty-description / success /
multi-row-mixed). Added `export const __test__` for testability.
Audits clean: typed-error-lint 196/196, silent-column-drop 103/103.
* fix(linkedin): fail fast on auth walls
* enrich(ctrip): hotel-suggest + bug fixes (silent clamp, dropped columns, fake URL)
Per WAWQAQ "丰富现有 adapter" pivot — Phase 2 site #1.
## New command
- `ctrip hotel-suggest` — surfaces hotel-context suggestions (cities,
business areas, individual hotels) via the same backing endpoint with
searchType=H. Distinct from `ctrip search` (searchType=D) which returns
destinations / scenic spots / railway stations.
## Bug fixes for `ctrip search`
- **Silent clamp on `--limit` removed**: out-of-range values (≤0, ≥51,
non-integer) now raise `ArgumentError` with explicit bounds rather than
silently snapping to [1, 50].
- **Silent column drop fixed**: previously the adapter discarded `id`,
`cityId`, `cityName`, `provinceName`, `countryName`, `lat`, `lon`, `eName`
and `displayType` from upstream rows. Now all are surfaced as stable
columns.
- **Fake URL fixed**: previously `url` was always `''`. Now constructs
canonical Ctrip URLs by `type` (City / Markland / Hotel / Zone / RailwayStation)
and returns `null` (no silent fabrication) for unknown types.
- **In-band error envelope typed**: `Result: false` payloads now surface
as `COMMAND_EXEC` (was previously not handled — adapter returned empty
rows).
## Doc fix
- `Mode: 🔐 Browser` → `🌐 Public` (search uses public API, no login)
- Add `hotel-suggest` to commands table in both `docs/adapters/index.md`
and `docs/adapters/browser/ctrip.md`.
## Coords picker
Mainland China rows ship `gdLat`/`gdLon` (gaode); international rows ship
`gLat`/`gLon` (wgs84). Adapter picks the first non-zero pair (zero is the
upstream sentinel for "missing"); returns `null` if all variants are zero.
## Tests
25 contract assertions across `parseLimit` / `pickCoords` / `buildUrl` /
`mapSuggestRow` + registry-level checks for both commands (Strategy /
shape parity / typed errors / no silent clamp).
## Audits
- typed-error-lint: 196 = 196 (unchanged baseline)
- silent-column-drop: 103 = 103 (unchanged baseline)
- listing-id-pairing: advisory only (search has `id` round-trip column)
## Manifest
757 → 758 entries (+1 for `hotel-suggest`).
* fix(ctrip): wrap suggest fetch and json failures
Closes#1334.
Exposes viewport overrides for `opencli browser screenshot` so an adapter or
ad-hoc shell user can render a page at a fixed width and capture the full
scrollable height. The ljg-card HTML to PNG pipeline use case.
Behavior:
- `--width W` only overrides device-metrics width; height is left unchanged.
- `--height H` only overrides height (ignored under `--full-page`).
- `--full-page` keeps the existing `captureBeyondViewport` shortcut.
- `--full-page --width W` first reflows at W, then re-overrides to (W, contentH)
so the captured image reflects the layout at the requested width.
- Override is always cleared in `finally`, including on capture failure.
* feat(deepseek): add detail and send commands for explicit conversation control
doubao already ships `detail <id>` and `send` for ID-explicit conversation
read/write; deepseek had only `read` (current page only) plus the
implicit-resume `ask`. Adding both gives users a stable handle when they
know the conversation ID, without going through `ask`'s resume detection
or its full prompt-then-wait pipeline.
`deepseek detail <id>`:
- parses a bare UUID or any URL containing `/a/chat/s/<id>`,
- rejects malformed input via `ArgumentError` before any browser
navigation,
- navigates to `https://chat.deepseek.com/a/chat/s/<id>` and returns
the visible message list,
- throws `EmptyResultError` when the conversation has no rendered
messages.
`deepseek send <id> <prompt>`:
- takes the conversation id as a required positional, because the
framework runs each browser command in an ephemeral per-command
workspace (a fresh tab) and there is no shared "current conversation"
across commands; the navigation must be explicit,
- drives input through CDP `Input.insertText` via `page.nativeType`,
mirroring the doubao adapter (#1278); `execCommand('insertText')` plus
a synthesised input event leaves the React-controlled state desynced
on a freshly-opened tab and the resulting click silently no-ops,
- keeps the verification loop inside the same `page.evaluate` so the
framework cannot close the tab mid-flight; counts user-class bubbles
by text-match (DeepSeek virtualises the message list, so a numeric
bubble-count check is unreliable),
- throws `CommandExecutionError` with a specific reason when the
textarea did not populate, the send button stayed disabled, the
bubble never settled, or the optimistic render rolled back during
a 3s settle window,
- treats "Promise was collected" from the post-click eval as success,
matching the existing pattern in `ask --file`.
Helper `parseDeepSeekConversationId` is exported from utils.js so the
same parser feeds both commands and round-trips the canonical lower-case
ID.
Tests:
- utils.test.js: 5 cases covering bare UUID, upper-case
normalisation, URL extraction with and without query string, empty /
null / whitespace input, and non-UUID rejection.
- detail.test.js: 5 cases covering registration, navigation +
message return, URL normalisation, ArgumentError before browser
navigation, and EmptyResultError on no-messages.
- send.test.js: 7 cases covering registration, ArgumentError on bad
id, full happy-path through nativeType + IIFE verification, the
textarea-mount timeout, missing nativeType helper, focus failure,
IIFE-reason translation to CommandExecutionError, and the
"Promise was collected" success path.
Manifest auto-regenerated to register both commands.
Live-verified end-to-end against my own DeepSeek session:
- `detail` returns the canonical message list for a bare UUID, parses
a full chat URL, and rejects malformed IDs before any browser
navigation,
- `send` lands the prompt as the latest user message in the target
conversation and gets an AI response back; reload of the
conversation page in a separate tab confirms the message persisted
server-side.
* docs(deepseek): document detail and send commands
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Keep the owned automation container window warm across lease release. Non-final owned leases close their tab; the final owned lease resets its tab to about:blank as a reusable placeholder. Update browser close wording to describe lease release rather than window closure.
Closes#1342.
`opencli deepseek ask` (without --new) used to resume the most recent
conversation by clicking the first `a[href*="/a/chat/s/"]` in DOM order
after a fixed 2-second wait.
Two bugs:
1. Pinned conversations sit in their own DOM section ("置顶") that
renders above "30 天内" and friends. Click-first-anchor lands on the
pinned thread, not the user's most recent. Reproduced live by
pinning a conversation through the sidebar context menu and
observing that the existing logic targets it instead of the most
recent non-pinned thread.
2. The 2s wait is fixed. On a slow network the sidebar has not
populated yet, the click is a no-op, and `ask` silently falls
through to the new-chat path. The user typed "follow up" and
a brand-new conversation gets created.
Replace the click-first-anchor + fixed wait with a new helper
`pickResumeUrl(page)` in utils.js that:
- polls the sidebar for up to 10s (5 attempts × 2s),
- identifies pinned anchors by a text-based check on the section
header (`/^\s*(置\s*顶|Pinned)\s*$/i`); DeepSeek's CSS-module
class names are randomized per build, so the text is the only
stable signal,
- returns the URL of the first non-pinned anchor (or falls back to
the first overall if every visible anchor is pinned),
- returns null if no anchor surfaces in time.
`ask.js` calls the helper and `page.goto`s the returned URL. When the
helper returns null, `ask` now throws a `CommandExecutionError`
instead of silently navigating to a fresh chat. The user gets a clear
"pass --new" hint and their prompt is never sent to a wrong target.
Tests:
- utils.test.js: 4 cases covering happy path, polling-then-success,
timeout returns null, and a structural assertion that the embedded
DOM walker uses text-based pinned detection.
- ask.test.js: replaced the prior "still selects model when no
conversation to resume" test (which exercised the silent
fall-through) with a fail-fast assertion. Updated the resume-success
test to mock the new helper.
* feat: 11 read adapters across 8 sites (dblp / steam / bbc / devto / lobsters / medium / coingecko / hf)
Round 2 of the adapter expansion sweep. All 11 commands hit public APIs (no
browser, no auth), follow the post-#1332 typed-error / no-silent-failure
discipline, and were live-verified against real endpoints.
New adapters:
- dblp/author : recent publications for one author (resolve PID by name, or pass --pid)
- steam/search : storefront name search (storesearch API)
- steam/app : single app detail (appdetails API; HTML entities decoded)
- bbc/topic : per-topic RSS (8 canonical BBC News feeds)
- devto/latest : /api/articles/latest with --page pagination
- lobsters/domain : stories from a specific source domain (/domains/<d>.json)
- medium/tag : tag RSS (description full-length, no silent truncation)
- coingecko/exchanges : trust score + 24h BTC volume leaderboard
- coingecko/categories : sector buckets with 6 sort options
- coingecko/global : aggregate market totals + BTC/ETH dominance
- hf/paper : single-paper detail by arXiv id (summary, ai_summary, ai_keywords, upvotes)
Also adds clis/steam/utils.js + clis/bbc/utils.js as shared helpers (HTML entity
decode, RSS parsing). All listings carry a round-trippable id where a detail
sibling exists; advise:listing-id-pairing reports zero new violations. typed-
error-lint and silent-column-drop gates both unchanged from baseline.
Manifest: 698 → 709 (+11 entries).
* fix: tighten adapter round2 contracts
* Add uisdc news adapter for CLI
Implements a CLI adapter for fetching the latest AI/design news from uisdc.com. Allows specifying the number of news items to return.
* feat(aibase): add aibase daily news adapter
This file implements a news adapter for AIbase that fetches the latest AI industry news and allows for configurable limits on the number of news items returned.
* fix(news): harden uisdc and aibase adapters
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Document the pattern for running opencli on a remote machine while keeping
the daemon and Chrome on the local machine. Reverse-tunnel local 19825
back to the remote (via SSH -R or frp) so the remote opencli still talks
to its own loopback and the daemon never leaves localhost.
Captures the rationale we landed on after reviewing #636: native
extension-to-remote-daemon support is deferred until the daemon protocol
gains authentication; in the meantime this is the safe, zero-code path
that achieves the same outcome.
* feat: add tiktok creator-videos command
TikTok Studio creator content list with views/likes/comments/saves/shares.
Hits the Studio item_list endpoint
(https://www.tiktok.com/tiktok/creator/manage/item_list/v1/?aid=1988) from a
logged-in /tiktokstudio/content session and pages with cursor until limit is
satisfied (server caps size at 50). Username for the resulting video URL is
extracted from the user_text= query param on play_addr / download_info entries,
falling back to scraping a[href*="/video/<id>"] from the Studio page DOM.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): regen manifest + replace silent-clamp with ArgumentError
- Regenerate cli-manifest.json (CI gate: must match `npm run build` output)
- Replace `Math.max(1, Number(args.limit) || 20)` and
`Math.min(Math.max(limit, 1), 50)` with an explicit positive-integer
guard + a server-cap-only ternary, per the silent-clamp guidance in
references/typed-errors.md (typed-error-lint baseline is unchanged)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(tiktok): tighten creator videos contract
---------
Co-authored-by: root <root@example.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Per WAWQAQ feedback in #OpenCLI thread on the flat "Site adapters (112)" listing:
the bucket conflates real web sites (bilibili, dianping, ...) with desktop apps
(chatgpt-app, chatwise, codex, cursor, discord-app, doubao-app, antigravity, notion).
Group them so agents that fall back to --help can scan by category.
Three buckets, sourced from existing metadata only — no new adapter schema:
- External CLIs: passthrough binaries from loadExternalClis() (docker, gh, vercel, ...)
- App adapters: domain is `localhost` or any non-DNS string (no `.`)
- Site adapters: domain contains `.` (real DNS), or domain is unset (default)
The classifier is one line: `domain.includes('.') ? 'site' : 'app'`. Adapters
without a domain field default to site (most are public web scrapers like
arxiv / wikipedia / spotify / ...).
Verified against the live registry: 7 External CLIs, 8 App adapters
(antigravity, chatgpt-app, chatwise, codex, cursor, discord-app, doubao-app,
notion), 104 Site adapters.
Structured help (-f yaml/json) gains parallel `external_clis` / `app_adapters`
/ `site_adapters` keys; `commands` no longer leaks adapter names.
External CLIs are now hidden from the default Commands listing (mirrors how
site adapters were already filtered) and surfaced in their own section.
- Add clis/test-utils.js with standard createPageMock utility
- Migrate 11 test files to use shared utility (removes ~300 lines of duplication)
- Delete extension/src/cdp.test.ts dead skip test (chrome.scripting.executeScript removed from source)
- Remove clis/pixiv/test-utils.js (superseded by shared utility)
Codify the JSDOM-against-frozen-fixture pattern that PR #1313 introduced
for dianping (and that PR #1318 had to follow up to clean up). The skill
previously had no reference for this category of test, so authors of the
next adapter that hits silent-in-browser-DOM bugs would either reinvent
it or skip it.
Key conventions captured:
- **Mandatory awk 'NF>0' as the final step of fixture creation.** The
blank-line noise that PR #1318 removed (84.6% / 54.8% of file content
in dianping/{shop,search}.html) came from manually stripping
script/style content without collapsing the surrounding newlines.
Skipping this step is the silent quality regression that the next
fixture author would also hit.
- **Trim-to-minimum but never re-flow content.** Some bugs depend on
text-node adjacency without intervening whitespace
(dianping #1312 bug #2: rating "4.8" + reviews "21241条" fused as
"4.821241条"). Pretty-printing the meaningful mega-line would mask
the very condition the test is meant to catch.
- **Reverse-validate the regression guard.** "18/18 tests pass" only
proves agreement with the current implementation, not that the test
would have caught the original bug. Reintroducing the buggy variant
must make the test fail — otherwise the fixture is over-stripped or
the assertion is too loose.
- **__fixtures__/ is the documented exception** to the "no committed
HTML dumps" rule in the skill's "关键约定". Calling that out
explicitly because the rule otherwise reads as "all HTML in repo is
bad," which the dianping fixture pattern intentionally violates for
a real reason.
Background: WAWQAQ in #1313 follow-up thread (`#OpenCLI:36d2f65a`) asked
twice — first about the visible blank-line noise (→ PR #1318 cleanup),
then about the root cause and what should improve in the workflow itself.
This is the workflow improvement.
No new tooling / CI gate / lint introduced (B 1-week gate freeze still
applies). When a fifth fixture site adopts this pattern,
`opencli browser fixture-snapshot` automation can be revisited; until
then, runbook discipline + skill reference is the right scope.
dianping/__fixtures__/search.html and shop.html came out of page.content() with
hundreds of blank lines that JSDOM ignores during parsing — pure visual / disk
noise that bloats reviewer diff and obscures the meaningful DOM subtree the
fixture freezes.
search.html: 372 → 168 lines (-204 lines, -572 bytes)
shop.html: 39 → 6 lines (-33 lines, -64 bytes)
`awk 'NF>0'` keeps every line that has any non-whitespace character, so the
minified mega-line (where the rating-vs-reviews adjacency that triggers #1312
silent fusion lives) is preserved verbatim. dianping.test.js 18 tests still
pass, and reintroducing the buggy `headText.match(/(\d+)条/)` extractor still
makes the regression guard fail with the expected '821241条' (proving the
fusion-bug detection power is intact after the strip).
Per WAWQAQ feedback in #1313 thread; opus independently validated the same
approach before the cleanup.
PR #1312 fixed two silent in-browser DOM bugs that the existing mocked
`page.evaluate` tests could not catch:
1. shop title fallback split on ASCII `[]` while dianping renders
full-width `【】`, so `name` was always empty (or `"undefined"`).
2. headText `\s+` collapse fused rating "4.8" with reviews "21241条",
so a head-wide `/\d+条/` regex captured "4.821241" → 5.
Both bugs only surfaced on live verify; mocked-evaluate unit tests fed
pre-baked results to the func and the real DOM walk never ran.
Make the in-browser extractor logic testable in CI:
- clis/dianping/shop.js, clis/dianping/search.js: extract the IIFE
bodies into top-level `extractShopFields()` / `extractSearchRows()`
using bare `document` / `location`. The live adapters inject these
via `page.evaluate(\`(\${fn.toString()})()\`)` so behavior is
unchanged; both commands re-verified end-to-end against live
dianping (shop returns name=芈重山老火锅(五道口店), reviews=21241,
rating=4.8; search returns 3 result-shaped rows with correct ids).
- clis/dianping/__fixtures__/shop.html (3.4KB), search.html (8.4KB):
sanitized HTML snapshots — scripts/styles/iframes/comments stripped,
img src placeholdered, only structural attributes kept. Trimmed to
the minimum subtree needed to exercise the extractors (search keeps
3 of 15 li cards; shop keeps .shop-head + .desc-info + .review-title
plus full-width 【】 title and headText with the rating/reviews
fusion preserved).
- clis/dianping/dianping.test.js: add a fifth describe block —
"extractors against frozen HTML fixtures" — that loads the fixtures
via JSDOM, swaps `globalThis.document` / `globalThis.location`, and
asserts the post-fix behavior:
* shop: name=芈重山老火锅(五道口店), reviewsRaw=21241条, rating=4.8,
breakdown={口味:4.8,环境:4.8,服务:4.8,食材:4.9}, hours, rank, subway.
* search: 3 rows with correct shop_ids, names, reviewsRaw, priceRaw,
starClass; round-trip through parseReviewCount/parsePrice mappers
to lock in {rating:5.0,reviews:21231,price:109} et al.
* ok:false branches: shop fixture without `.shop-head`, search
fixture with empty `#shop-all-list`.
Manually verified the fixtures would catch the original bugs by running
buggy extractor variants against shop.html — ASCII-bracket fallback
returns `name="undefined"`, and head-wide `/\d+条/` returns `821241条`
(both fail the new assertions).
Test suite grows 14 → 18 passing tests; live verify of both commands
still produces correct output post-refactor.
Pattern intentionally limited to dianping as a reference point. If other
sites with in-browser DOM extraction encounter similar silent bugs, this
JSDOM-against-frozen-fixture pattern can be adopted per-site.
* docs(cases): add three researcher workflow examples
Add use cases under cases/ that exercise the recently-landed
researcher-friendly adapters:
- daily-rl-research-monitor.md uses arxiv recent + openreview venue
+ hf top to compress a morning paper-skim into one shell pipeline.
- find-paper-implementation.md chains arxiv search/paper + dblp
search + hf top + openreview search to map a paper's canonical
record, follow-ups, and community uptake.
- track-conference-papers.md walks openreview venue + reviews to
shortlist accepted papers and digest review threads in batch.
Each file is a real workflow built on commands from #1289 (arxiv
recent), #1294 (openreview), and #1299 (dblp).
* docs(cases): correct venue ids and forum example to ones that return data
The first revision used "ICLR.cc/2026/Conference" and "ICLR 2026 oral"
as venue strings. Both return EMPTY_RESULT today because the venue is
not open. Update each case to use natural-language venue text that
OpenReview currently exposes ("ICLR 2024 oral", "NeurIPS 2025 oral")
and a real forum id (KS8mIvetg2, "Proving Test Set Contamination in
Black-Box Language Models") in the reviews / paper drill-down. Note
the arxiv free-text-search ranking quirk so the worked DPO example
makes sense.
PR #1297 introduced a CI gate that fails when a site has both a listing
and a detail command but the listing rows don't carry an id-shaped column.
The gate came with a 10-entry EXEMPT map (topic-string trending,
profile-attribute rows, UI-only sessions, ...) where each exemption
recorded a "why this listing legitimately doesn't pair" reason.
By the same filter that closed PR #1311 (write-without-delete-pair gate):
Is "listing should pair with detail" a *permanent* anti-pattern, or
case-by-case business judgment?
It's case-by-case. Topic-string listings and profile-attribute rows
genuinely don't pair with a detail command. The fact that we needed an
EXEMPT map with 10 entries and individual reason strings is the smell —
it's not the rule winning, it's the rule failing. Forcing every adapter
PR to either add an id column or file an exemption was a higher cognitive
cost than the silent-loss bugs the rule actually catches.
Changes:
- .github/workflows/ci.yml — drop the "Check listing↔detail id pairing"
step. Other gates (silent-column-drop, typed-error-lint) stay in place.
- package.json — rename the script from `check:listing-id-pairing` to
`advise:listing-id-pairing` to make the advisory nature explicit.
- scripts/check-listing-id-pairing.mjs — drop the `--strict` flag and the
EXEMPT map. The script now always exits 0 and prints an advisory report
of listings that don't carry an id-shaped column. Reviewers/authors use
it as guidance, not a gate.
- docs/conventions/listing-detail-id-pairing.md — rewrite from "MUST" to
"soft convention". Adds an explicit "why advisory, not a gate" section
that lists the legitimate non-pairing categories so future readers know
the rule's boundary.
- docs/developer/ts-adapter.md — match the advisory tone in the
adapter-author guidance.
The doc, the script, and the column patterns table all stay — agents and
adapter authors can still consult them. What's gone is the CI failure and
the per-PR exempt-list maintenance burden.
Net diff: -34 lines (gate + EXEMPT map removed, advisory-tone doc adds
a small "why advisory" section).
Adds a baseline CI gate for convention-audit typed-error lint findings. Also refreshes the silent-column-drop baseline for dianping changes already on main.
The merged adapter had two silent in-browser bugs that the mocked-evaluate
unit tests don't catch — only live verify against www.dianping.com surfaces
them:
1. Shop name returned `undefined`. The fallback parsed `document.title` with
an ASCII-bracket split (`/[\\[\\]]/`) but dianping wraps the name in
full-width brackets `【芈重山老火锅(五道口店)】...`. Switch to a `【...】`
regex so the title fallback actually fires.
2. Reviews returned `5` instead of `21241`. The headText was whitespace-
collapsed to `★★★★★4.821241条...`, fusing the rating and review digits;
a head-wide `/\d+条/` then captured `4.821241` and rounded to `5`. Read
the dedicated `.reviews / .review-num` element ("21241条") instead, with
a `.review-title` "评价(<n>)" fallback.
* feat(dianping): browser adapter — search + shop on www.dianping.com
Adds two browser-mode adapters for the dianping (大众点评) PC site:
- `dianping search "<keyword>" --city <name|id> --limit <n>`: keyword
shop/restaurant search. Returns rank, shop_id, name, rating, reviews,
price, cuisine, district, url. shop_id round-trips into `dianping shop`.
- `dianping shop <shop_id>` (alias `detail`): shop detail sheet
(field/value rows: name, rating, breakdown 口味/环境/服务/食材, reviews,
price, rank, hours, address, subway, features, url).
Both use Strategy.COOKIE on www.dianping.com (the PC site renders search
SSR and does not require JS hydration). m.dianping.com is intentionally
crippled for non-mobile UAs, so it's not used.
Auth detection (utils.detectAuthOrEmpty) inspects both response text and
final URL for the Meituan Yoda captcha redirect (verify.meituan.com) and
the dianping login redirect; raises AuthRequiredError with the captcha
URL embedded so the user can clear it manually in the same profile.
Listing↔detail id pairing: search.shop_id → shop.<id>. Adds 'shop' to
DETAIL_NAMES in scripts/check-listing-id-pairing.mjs so the convention
gate scans this site (35 sites / 78 listings now covered).
* fix(dianping): harden browser failure classification
* fix(dianping): fail on partial missing shop ids
Adds opencli convention-audit for batch convention scanning, with structured output, strict mode, docs, and startup isolation from local user/plugin discovery.
* feat(youtube/xiaohongshu/xiaoe): surface dropped ids/url on listings (sweep)
Round 8 same silent-column-drop class as #1300/#1301/#1302 — row already
emits the id/url field but `columns` array forgot to project it, so table
view drops it and agent loses the chain into detail commands.
- youtube/feed: rename row.videoId → video_id (snake_case convention),
add to columns. youtube/video accepts both URL and id, so url-based
round-trip already worked, but exposing the canonical id removes the
url-parse step for chained calls.
- xiaohongshu/feed: pipeline map already extracts `id` from the homefeed
payload, columns now lists it.
- xiaoe/catalog: pipeline map already projects `url`, columns now lists
it. xiaoe/detail takes a positional url, so this completes the
round-trip explicitly.
Also fixes one camelCase column violation on youtube/feed (videoId vs the
project's snake_case convention as in twitter `is_retweet`/`created_at`,
douban `subject_id`/`photo_id`, hupu `thread_title`).
CI gate `check:listing-id-pairing` ✓ (34 sites, 77 listings, 10 exempt).
typecheck clean. 114 tests pass for youtube + xiaohongshu.
* fix(youtube): keep feed continuation ids after rename
Round 7 — silent-drop sweep. Continues the listing→detail id-pairing
work from #1297. Each row was already extracting these ids/urls
internally; only the `columns` projection was missing, so they showed
up in `-f json` but never on the table view.
| Adapter | Added columns |
|--------------------|-------------------------------------|
| `1688 search` | `item_url`, `member_id` |
| `hupu mentions` | `tid`, `pid`, `url` |
| `douban photos` | `photo_id`, `subject_id` |
| `linux-do tags` | `slug` |
Round-trip wins:
- `1688 search` → `1688 item <item_url>` (item_url is the canonical
detail.1688.com URL); `1688 search` → `1688 store <member_id>`
- `hupu mentions` → `hupu detail <tid>` (and `pid` for the deep link)
- `douban photos` → tied back to the parent movie via `subject_id`
- `linux-do tags` → `linux-do feed --tag <slug>` (slug is the URL form)
No logic change — only the column array. JSON output unchanged.
Tests: 45/45 pass for the four affected sites.
Round 6 — silent-drop audit follow-up. Sibling twitter listings have
been inconsistent about the canonical tweet `id` (rest_id):
- timeline ✓ exposes id
- search ✓ exposes id
- list-tweets ✓ exposes id
- notifications ✓ exposes id
- bookmarks ✗ extracts but drops it from columns
- likes ✗ extracts but drops it from columns
- tweets ✗ extracts but drops it from columns
The `id` is already in the row object — only the `columns` projection
was missing. With the listing↔detail id-pairing CI gate from #1297 now
on main, surfacing `id` makes round-trip into `twitter thread <id>` /
`twitter delete <id>` / `twitter like <id>` work from the table view too
(previously only via `-f json`).
Other field-presentation drift (`name`, `created_at`, `retweets`)
aligned with sibling adapters where those values are already emitted.
Tests: tweets.test.js asserts `toEqual` on the columns array — updated
that assertion. Other twitter tests use `toMatchObject` and pass
unchanged. 81/81 in `clis/twitter/`.
While auditing instagram/facebook/pixiv coverage gaps, found that pixiv
listings already extract `user_id` and construct `url` per row but drop
both fields from the table view (`columns` doesn't list them). The data
is in the row object — only the column projection was missing.
Per the listing↔detail id pairing convention (#1297), surface them so:
- `user_id` round-trips from `ranking` / `search` → `user` / `illusts`
- `url` is the canonical share link for every illust / user record
Changes:
- `ranking`: + user_id, + url
- `search`: + user_id, + url
- `illusts`: + url (user_id is the arg, no need to repeat per row)
- `user`: + url
No behavior change beyond the table view — JSON output already had these
fields, so existing scripts that consume `-f json` keep working.
* feat(dblp): public bibliography adapter — search + paper
Wraps the dblp.org public API:
- `dblp search <query>` → /search/publ/api JSON, projected into one row per hit
- `dblp paper <key>` → /rec/<key>.xml, parsed into a one-row record
Why dblp on top of arxiv/openreview: dblp is the largest, oldest CS
bibliography (7M+ entries) and the only one of the three that consistently
indexes pre-arXiv literature, journal articles, books, and theses. The
canonical record key (e.g. `conf/nips/VaswaniSPUJGKP17`) round-trips
cleanly between the two commands per the listing↔detail convention.
Implementation notes:
- No deps beyond the registry — XML parsed with conservative regexes,
same approach as the arxiv adapter.
- Polite User-Agent per dblp's API guidance; HTTP 429 mapped to a
CommandExecutionError with a "lower --limit" hint.
- Author homonym suffixes (`"Smith 0001"`) trimmed for clean output.
- 39 unit tests cover validators, XML extraction, both commands.
* fix(dblp): fail fast on API status envelopes
* feat(convention): listing↔detail id pairing rule + CI gate
Adds a hard convention: when a site exposes both a listing-class command
(search / hot / top / recent / ...) and a detail-class command (read /
paper / article / view / ...), every listing row MUST surface an id-shaped
column whose value round-trips into the detail command. Without that, an
agent has no way to follow up on a listing row except re-searching by
title or scraping URLs out of band — both of which break the agent-native
contract.
What's in this PR
- docs/conventions/listing-detail-id-pairing.md — full rule, examples
table, why-it-matters, what counts as id-shaped, exemption taxonomy,
how to add an id column to a listing.
- scripts/check-listing-id-pairing.mjs — validator that reads
cli-manifest.json, classifies each entry as listing / detail / other,
and fails when a listing on a site that also has a read-detail command
is missing an id-shaped column. Exemption allowlist records WHY each
pair is exempt so future maintainers know what to verify.
- npm run check:listing-id-pairing — strict-mode wrapper.
- CI: new step in build job runs the validator after the manifest
freshness check on Linux.
- docs/developer/ts-adapter.md — cross-link from the adapter authoring
guide.
- docs/.vitepress/config.mts — sidebar entries for the new conventions
section.
Fixes brought to zero violations
- 1688/search: add offer_id (already extracted, just surfaced)
- bluesky/user: add uri (AT URI round-trips into bluesky/thread)
- tieba/search: add id + url (thread_id already extracted)
- tieba/hot: add url (rows are topics, not threads — url is the
best-effort round-trip handle, doc'd as such)
Exemptions (intentional, doc'd in EXEMPT map with rationale)
- nowcoder/hot, bluesky/trending, twitter/trending — listing rows are
topic strings, not posts.
- lesswrong/user, reddit/user — rows are profile-attribute key/value
pairs, addressed by the username arg.
- discord-app/search — desktop UI session, message ids not extractable.
- notion/search — Strategy.UI Quick Find, page ids not exposed in DOM.
Validator output after this PR: 32 sites scanned, 75 listings checked,
7 exempted, 0 violations.
* fix(convention): tighten listing id gate
* fix(convention): close url-derived id loophole
* feat(indeed): add `search` and `job` adapters (US site)
Adds an Indeed adapter that fills the US job-search gap (alongside
existing 51job / boss-zhipin / linkedin coverage). Both commands run
through a real browser session because Indeed sits behind Cloudflare
and answers bare HTTP fetches with `403` + `cf-mitigated: challenge`.
## Commands
- `indeed search <query>` — keyword job search
- args: `query`, `--location`, `--fromage`, `--sort`, `--start`, `--limit`
- columns: `rank, id, title, company, location, salary, tags, url`
- `indeed job <jk>` (alias `detail`, `view`) — full job posting
- args: `id` (positional, the 16-char hex `jk` from `search`)
- columns: `id, title, company, location, salary, job_type, description, url`
## Listing↔detail id pairing
`search.id` is the Indeed `jk` (job key, 16-char lowercase hex). It feeds
directly into `indeed job <jk>`. Conforms to the listing↔detail id
pairing convention proposed in #1297.
## CF challenge handling
The adapter polls the result selectors for up to 15s after navigation,
giving the browser time to clear the Cloudflare interstitial. If the
challenge is still up after the wait, the adapter throws a
`CommandExecutionError` with a hint pointing the user at the connected
browser to clear it once. Subsequent calls reuse the warmed cookies via
`Strategy.COOKIE`, mirroring the v2ex / boss / linkedin patterns.
## Validation
`utils.js` keeps argument validation pure and unit-testable:
- `requireJobKey` rejects anything that isn't a 16-char lowercase hex
- `requireFromage` only accepts `1` / `3` / `7` / `14` (Indeed's enum)
- `requireSort` only accepts `relevance` / `date`
- `requireBoundedInt(limit, default=15, max=25)` — Indeed serves at most
one page (10 jobs/page); ArgumentError on out-of-range, no silent
clamping, per the typed-error feedback in #1289.
## Tests
18 unit tests in `clis/indeed/indeed.test.js` cover registration,
validators, URL builders, and DOM-card normalizers. Browser-driven
verification stays out of CI by design (CF challenge is interactive).
## Docs
- `docs/adapters/browser/indeed.md` — full adapter doc with prerequisite
CF-challenge notes and listing↔detail id pairing callout.
- Sidebar entry + adapter index row.
* fix(indeed): tighten timeout fail-fast and runtime tests
* fix(indeed): align readiness with search parser
* feat(openreview): add public adapter — search/venue/paper/reviews
OpenReview is the open peer-review platform used by ICLR / TMLR / COLM
and ML workshops. Its v2 API exposes everyone-readable submissions,
reviews, and decisions without auth, so all four commands run with
`browser: false`.
Commands:
- `openreview search <query>` — full-text search
- `openreview venue <venue>` — list submissions; accepts either a venue
display name (matched against `content.venue`, e.g. "ICLR 2024 oral")
or a full invitation id (e.g. "ICLR.cc/2025/Conference/-/Submission")
via `/-/` heuristic; supports offset pagination
- `openreview paper <id>` — single-paper detail with full abstract
- `openreview reviews <forum>` — paper + threaded reviews/decisions/
comments, ordered chronologically with paper lifted to row 0;
classifies notes via invitation tail (REVIEW / DECISION / REBUTTAL /
COMMENT / META_REVIEW / WITHDRAWAL); per-row truncation via
`--max-length` (min 200)
Listing IDs round-trip into `paper`/`reviews`. PDF URLs normalized to
absolute `https://openreview.net/pdf/...`. `pdate` falls back to
`cdate` when missing, formatted as `YYYY-MM-DD`.
All limits/offsets/ids fail-fast with typed errors (`ArgumentError`,
`EmptyResultError`, `CommandExecutionError`) — no silent clamping, no
empty-array fallbacks. fetch + json + non-2xx + 404 are wrapped so
network/API failures never look like empty results.
Tests: 23 unit tests covering the column contract, content extraction,
date/PDF normalization, invitation-vs-venue dispatch, error paths
(network/JSON/HTTP), pagination offset accounting, and the reviews
classifier + section joiner + truncation.
Live-verified against api2.openreview.net for search ("diffusion
model"), venue ("ICLR 2024 oral"), paper (KS8mIvetg2), and reviews on
that paper's full thread.
* fix(openreview): tighten error and review typing
* fix(openreview): stabilize review contracts
Follow-up from PR #1293 review: 'all answers' was misleading because
the implementation is limit-bounded (default 10, max 100) rather than
unbounded pagination. Spell out the actual contract — including the
accepted-answer-outside-page fallback path — so users don't expect
infinite-scroll behaviour.
Non-blocking docs-only change flagged by codex-mini1 + First-principles-1
during #1293 review.
* feat(lobsters): surface short_id + created_at on listings, add `read <short_id>`
Same agent-native gap as the just-merged hackernews PR (#1288):
1. The 4 listings (hot / newest / active / tag) didn't surface each story's
`short_id`. Agents could see the title and a comments URL but couldn't
pass the id back into a follow-up command. Add `id` (= `short_id`) and
`created_at` columns; `created_at` is cheap signal for "how stale is this".
2. There was no way to read a story + comment tree from the CLI. Lobsters
makes this nicer than HN: `https://lobste.rs/s/<short_id>.json` returns
the story plus a flat `comments[]` array where each entry already carries
`parent_comment` and `depth`, so we get the full thread in one HTTP call
and just DFS using the parent map.
`read` mirrors the `hackernews read` shape (POST row + L0/L1/… indented
comments, `[+N more replies]` stubs at depth/limit cutoffs) so the two
adapters feel the same to agents that already learned one. Same typed
fail-fast envelope: `ArgumentError` on bad short_id / non-positive limit /
depth / replies, `EmptyResultError` on 404 or empty body, `CommandExecutionError`
on other HTTP failures.
Tests cover all 4 listings (column shape + map step), `read` registration,
positional arg shape, ArgumentError fail-fast (no fetch on bad input),
EmptyResultError on 404, threaded-tree assembly from a flat `comments[]`,
and the `+N more replies` depth-cutoff path.
* test(lobsters): lock read fail-fast coverage
* docs(lobsters): list read command
* feat(devto): surface article id + published_at on listings, add `read <id>`
Agent-native gap: devto listings (`top`/`tag`/`user`) didn't include the
article `id`, so an agent couldn't round-trip from a listing into a body
read. They also dropped `reading_time` and `published_at`, which are cheap
signals the API gives you for free.
Changes:
- `top` / `tag` / `user`: add `id`, `reading_time`, `published_at` columns
alongside existing rank/title/etc. `user` keeps its no-author shape since
it's already user-scoped.
- New `devto read <id>`: hits `dev.to/api/articles/<id>` and returns one
row with the article body (truncated by `--max-length`, default 20000,
min 100). DEV.to's public API does not expose comments yet, so this is
intentionally a single-row reader rather than a HN/lobsters-style
threaded tree — if/when comments become public we can extend to
POST + L0/L1.
- Typed fail-fast: `ArgumentError` for non-numeric id and for `--max-length`
below 100; `EmptyResultError` on 404; `CommandExecutionError` for other
non-2xx HTTP statuses. No silent clamps.
- Defensive tag normalization: the `/api/articles/<id>` endpoint returns
`tag_list` as a comma-string and `tags` as an array (the opposite shape
from listing endpoints). Caught this on live verification — both shapes
now collapse to a comma-joined string.
Tests: 12 vitest assertions covering listing column shape (all 3) +
register/args/strategy + typed-error fail-fast paths + happy-path body
extraction + truncation marker + alternate tag_list shape.
Live verification: `devto top --limit 3` and `devto read 3602287` both
return the expected agent-native shape.
* fix(devto): harden article read contract
X removed the post-count caption from each cell on `/explore/tabs/trending`.
The adapter still iterated `divs[2..]` looking for a numeric text node and
fell back to the literal string "N/A" when it found none — which was every
row, on every call. We were emitting a silent-wrong column for every result.
Drop the column and the no-longer-relevant scan loop. Add a regression test
on the columns shape so the column doesn't slip back in.
Live runs of `opencli twitter trending` previously returned rows like
`{rank: 1, topic: "...", tweets: "N/A", category: "..."}` — the `N/A` was
not a transient outage, it was structural.
* feat(arxiv): full abstract/authors, surface pdf+categories+comment, add `recent <category>`
`paper` was silently truncating the abstract to 200 chars and dropping all but
the first 3 authors — agents calling it for a paper summary lost data. Stop
truncating, return all authors, and surface the rest of what the Atom feed
already gives us: pdf url (`<link rel="related">`), all `categories`,
`primary_category`, and the author `comment` (page count, conference, etc.).
`search` keeps a compact list shape (no abstract column, but adds
`primary_category`).
New `arxiv recent <category>` lists newest submissions in a category sorted by
`submittedDate desc` — fills a gap (previously you had to know a search term
to surface anything). Validates the category string and rejects malformed
input via `ArgumentError`.
`search` also switches its no-results path from `CliError('NOT_FOUND', ...)`
to `EmptyResultError` to match the convention other public-API adapters use.
Tests cover: command registration, full-abstract / all-authors parsing, XML
entity decoding in titles, pdf/categories/comment extraction, and category
validation.
* fix(arxiv): harden category and limit validation
* feat(hackernews): add `read <id>` and surface item id on every listing
Two related agent-flow gaps in the HN adapters:
1. `top`/`best`/`ask`/`new`/`show`/`jobs`/`search` all carry the HN item
id internally (firebase items are fetched by id; algolia hits include
`objectID`) but drop it before output. Without an id column the agent
can see the title but has no handle to follow up with.
2. There was no way to read a story's discussion. The whole reason an
agent looks at HN is the comments — and that capability was missing.
This PR adds:
- `id` column on every listing adapter (firebase items: numeric id;
algolia search hits: `objectID` string). Existing column order is
preserved otherwise.
- `hackernews read <id>` — public/non-browser adapter that fetches the
story plus a tree of top-level comments + inline replies via
`https://hacker-news.firebaseio.com/v0/item/<id>.json`. Mirrors the
`reddit read` shape (`type/author/score/text`) so agents can use both
with one mental model. HTML-only fields (comment text) are converted
to plain text with anchor URLs preserved.
- Column-contract tests covering all listings + the new read adapter.
- Doc entry under `docs/adapters/browser/hackernews.md`.
Tested locally via `~/.opencli/clis/hackernews/` overrides:
opencli hackernews top --limit 3 # id present
opencli hackernews search rust --limit 2 # id (objectID) present
opencli hackernews read 47999636 --limit 5 # threaded output
* fix(hackernews): typed fail-fast for read
* fix(douban): drop unparseable fields from movie-hot, add id/votes
The chart page (movie.douban.com/chart) only exposes a single comma-joined
text dump in `.pl2 p`, of the shape:
<release_dates...> / <actors...> / <regions...> / <director_zh> /
<runtime>分钟 / <other_titles> / <genres> / <director with English> /
<languages>
The previous `loadDoubanMovieHot` tried to anchor on the release-date
regex and take `parts[releaseIndex - 1]` as director and
`parts[releaseIndex - 2]` as region. That breaks in two ways:
1. Most entries have multiple release dates back-to-back, so the
"anchor minus one" position is itself a date. Director output becomes
`'2025-09-07(多伦多电影节)'` and region is empty — silent wrong data.
2. For entries with a single release date, the offsets land on actor
names, not director / region.
The page does not actually carry a clean director or region per row —
that's only available on the subject detail page. Trying to reconstruct
either from the chart string is the canonical "verify passes but data is
wrong" failure (success-rate-pitfalls §2 sibling DOM contamination).
Fix: drop `director`, `region`, `quote` from the listing. Surface what
the chart page actually provides reliably:
- `id` — extracted from the subject URL, ready for `douban subject`
- `votes` — from `.star .pl` (`(62484人评价)`), useful as popularity signal
- existing `rank`, `title`, `rating`, `year`, `url`
Agents that need director / region should follow up with
`opencli douban subject <id>`, which is already wired for that data.
* fix(douban): fail fast on empty movie hot
* fix(bilibili,reddit): add identifier and url columns to hot lists
Both `bilibili hot` and `reddit hot` previously dropped their per-row
identifier and URL on the way out, breaking the typical agent flow where
the next call needs a `bvid` / `postId` to fetch detail or comments.
- bilibili/hot: add `bvid` and `url` columns (constructed from bvid)
- reddit/hot: surface `postId`, `author`, `url` (already in evaluate but
dropped in map)
Tested via local `~/.opencli/clis/<site>/hot.js` overrides.
* test(bilibili,reddit): lock hot list identifier columns
Move the Browser Bridge daemon WebSocket out of the MV3 service worker and into an offscreen document. Remove the popup/action UI and obsolete extension log forwarding now that doctor is the diagnostic surface.
Pairs with the new collection-create adapter so users (and future
fixture-teardown logic) can clean up saved-post collections from CLI.
- POST /api/v1/collections/{id}/delete/ with multipart module_name=collection_settings
- Accepts collection name (case-insensitive) or numeric collection_id; resolves
via /collections/list/ first so unknown / duplicate names error explicitly
instead of bubbling up a 404 or silently deleting the wrong one.
The previous implementation silently skipped any adapter whose import
failed (catch + warn-to-stderr + return []), then printed a successful
"✅ Manifest compiled: N entries". When dist/ was stale (e.g. after
renaming an export the JS adapters re-import) every adapter using that
export would fail to load, get skipped, and the script still exited 0.
An agent reading exit codes to gate work would commit the resulting
manifest and silently delete dozens of unrelated adapter entries.
Three layers of defense:
1. Distinguish skip kinds. Files that don't call `cli(...)` are still
silently dropped (helpers / type modules). Files that look like CLI
modules but fail to import now throw `ManifestImportError`. The
batch scanner aggregates failures and `main()` exits 1 with an
explicit list, leaving the existing manifest on disk untouched.
2. Net-deletion safety net. `main()` diffs the new entries against the
committed manifest and refuses to overwrite when entries would be
removed. `--allow-removals=N` (or bare `--allow-removals` for any)
is the explicit opt-in; the error message tells the caller exactly
what value to pass.
3. Runtime dist guard. `node dist/src/build-manifest.js` now refuses
to run with a clear pointer at `npm run build-manifest` (which uses
tsx). The npm script itself is migrated to `tsx src/build-manifest.ts`
so no project-level command points at the compiled copy anymore.
Release CI gains a manifest-drift gate (build-manifest + git diff
--exit-code) so a tag push can never publish stale or silently-shrunk
manifests. The existing CI check on PRs is preserved.
`ManifestEntry` is split into `src/manifest-types.ts` so runtime code
(discovery.ts) imports the type without pulling the build-time
compiler module.
Tests:
- `loadManifestEntries` throws ManifestImportError on import failure
- helper modules without cli() are still silently skipped
- `scanClisDir` aggregates per-adapter failures
- `diffRemovedEntries` returns expected site/name diff
- `parseBuildManifestArgs` reads --allow-removals[=N]
WAWQAQ feedback: the green left border on the status row looked
disconnected — only on the top half of the card, creating an awkward
stub. Connection state is already conveyed clearly by the colored dot
and the "Connected to daemon" / "Disconnected" text, so the border was
redundant decoration.
Drop the .card.connected/.disconnected/.connecting border-left rules.
No JS or layout changes; cleaner surface, fewer visual variants.
- Merge status row and profile row into a single rounded card with a
brand-colored left border accent indicating connection state
- Render contextId inline next to a "Profile" label with a Copy button,
letting users paste it into `opencli profile rename` without manual
selection (replaces the old full-width code block treatment)
- Show daemon version inline in the status row when connected, and
render the extension version as a tag in the popup header — both
surface version information that helps diagnose stale-daemon issues
- Forward both versions through the existing `getStatus` background
message: extension reads its own version from the manifest, daemon
version is fetched best-effort from `/status` with a 1.5s timeout so
popup never hangs when the daemon is unreachable
Closes#1192. Two changes:
1. New `instagram collection-create <name>` adapter wraps
`POST /api/v1/collections/create/` (multipart `name` +
`module_name=collection_create`, X-IG-App-ID + X-CSRFToken).
2. `instagram saved` gains an optional `--collection <name>` flag.
When set, the adapter resolves the name to a collection id via
`/api/v1/collections/list/` (case-insensitive trim match) and then
fetches `/api/v1/feed/collection/{id}/posts/`. Unknown names throw
with the available list so callers can self-correct.
Both verified end-to-end against a live IG account. Verify fixtures
under ~/.opencli/sites/instagram/verify/ ship the
patterns/notEmpty/mustBeTruthy guards from the latest adapter-author
skill (success-rate-pitfalls §1, §4, §8).
* feat(weibo): add favorites + publish CLI commands
Consolidates #1253 (favorites) and #1254 (publish) into a single PR per maintainer request.
- clis/weibo/favorites.ts: cookie-mode fetch of authenticated user's favorites via weibo.com/u/page/fav/{uid}
- clis/weibo/publish.js: UI-automation post (text up to 2000 chars, up to 9 images jpg/png/gif/webp)
- cli-manifest.json regenerated to include the new commands
Note: favorites.ts uses TypeScript syntax but build-manifest.js scans only *.js — favorites is currently NOT registered in the manifest. Reviewers please check whether to rename to .js or whether the manifest scanner should learn .ts.
Authored-by: hszhsz <heshaoz1990@gmail.com>
* fix(weibo): harden favorites and publish commands
* fix(weibo): publish without execute gate
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(claude): add Claude adapter
Adds a Claude (claude.ai) browser adapter family with seven commands
modeled on the existing clis/deepseek/ pattern: ask, send, new, status,
read, history, detail.
Closes#1251
* feat(claude): align send command columns with doubao
Match the established Status / SubmittedBy / InjectedText shape used by
doubao send so agent loops can rely on a consistent fire-and-forget
output across AI chat adapters.
* fix(claude): preserve DOM order in getVisibleMessages
The previous implementation queried user-message and assistant-message
nodes in two passes, which serialized as [u1, u2, u3, a1, a2, a3] for
multi-turn chats instead of the correct conversation order. Single
combined query preserves DOM order so claude read / detail return
turns in the order the user reads them on the page.
* docs(claude): note --live requirement for read across invocations
* fix(claude): fail fast on auth and empty states
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(update-check): show extension update notice on exit
The CLI exit hook already prints "Update available" when a newer @jackwener/opencli is on npm. Extension updates were only surfaced inside `opencli doctor`, so users running normal browser commands had no signal that the Chrome extension was out of date.
Solution piggybacks on the existing 24h background fetch:
- Daemon writes the live extensionVersion + lastSeenAt into the shared cache on every hello handshake (rare event, one fs.writeFileSync).
- CLI exit hook reads the cache it already loads and prints an extra extension notice when a newer release is available and the cache is fresh (<7d).
- writeCache becomes a read-merge-write so the daemon's currentExtensionVersion and the CLI's npm latestVersion don't clobber each other.
Net cost on the CLI hot path: zero new I/O, zero new daemon contact. The notice formatter is split into a pure helper (buildUpdateNotices) so the staleness window, equality, and combined-notice cases are unit-tested without touching disk or stderr.
* fix(update-check): tolerate partial cache when daemon writes first
Self-review caught a TypeError path: if the daemon's hello handler runs `recordExtensionVersion` before the CLI's npm fetch ever populated the cache, the resulting cache file has only `currentExtensionVersion` + `extensionLastSeenAt` and no `latestVersion`. The next CLI run then fed `undefined` into `isNewer`, which calls `.replace(...)` on it.
- Mark `lastCheck` and `latestVersion` optional in the cache schema (the merge pattern means either side may write first).
- Guard the CLI notice on `cache.latestVersion` being defined before comparing.
- Guard `checkForUpdateBackground`'s 24h short-circuit on `lastCheck` being defined.
- Add a test for the daemon-only cache case.
* feat(zhihu): add collection command to list favorite items
Add new 'opencli zhihu collection' command that:
- Lists items from a Zhihu collection (requires login)
- Supports pagination with --offset and --limit parameters
- Handles multiple content types: answer, article, pin
- Shows collection statistics: total count, total pages, current page
* feat(zhihu): split collection into collection and collections commands
- Rename zhihu collection list functionality to zhihu collections
- Keep zhihu collection for viewing specific collection contents by ID
- Convert collection.ts to collection.js so build-manifest picks it up
- Add tests for both commands
- Update cli-manifest.json
* fix(zhihu): harden collection read commands
---------
Co-authored-by: Developer <developer@example.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(twitter/following): switch from INTERCEPT+autoScroll to COOKIE+cursor pagination
The previous INTERCEPT strategy relied on autoScroll to trigger Twitter's
pagination by scrolling document.body. Twitter's virtual list doesn't grow
document.body.scrollHeight, so scrolls stopped triggering API calls after
the first few pages, capping results at ~50 regardless of limit.
Now uses Strategy.COOKIE with explicit cursor-based GraphQL pagination
(same pattern as twitter/likes), which correctly fetches all pages.
Fixes#1230
* fix(twitter): harden following pagination
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(boss): add jobType filter and bossOnline output
Add --jobType param (全职/兼职/实习 = 1901/1902/1903) so callers can
exclude internships at the API layer instead of post-filtering by name
keywords. Without this, --experience 应届 returns a mix of 校招 and 实习
because BOSS bundles both under code 108.
Also surface bossOnline (Y/empty) in results so callers can prioritize
HRs currently online — this is the only activity signal exposed by the
web API; 'recently active' / 'newly posted' filters are mobile-only and
not accepted by /wapi/zpgeek/search/joblist.json.
* fix(boss): correct experience codes (应届=102, not 108)
The previous EXP_MAP was off by ~2 across the board. Verified each
code by clicking BOSS web's filter UI and reading the URL:
108 = 在校生 (interns) was: '在校/应届','应届' → 108 (wrong)
102 = 应届生 (校招 full-time) was: '1-3年' → 102 (wrong)
101 = 经验不限 was: '1年以内' → 101 (wrong)
103 = 1年以内 was missing
104 = 1-3年 was: '3-5年' → 103 (wrong)
105 = 3-5年 was: '5-10年' → 104 (wrong)
106 = 5-10年 was: '10年以上' → 105 (wrong)
107 = 10年以上 was missing
This is why --experience 应届 had been returning mostly 实习生 jobs:
it was secretly querying 在校生 (108). The fix makes 应届 actually
mean 应届生 (102 = 校招), and lets users pick 在校生 (108) explicitly
when they do want internships.
* fix(boss): validate job type filter
* fix(boss): keep legacy campus experience alias
---------
Co-authored-by: youhh <youhh@1051233107@qq.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(deepseek): add vision mode support
DeepSeek added a third model "识图模式" (Vision Mode) that accepts
image uploads for visual understanding. Add vision to the --model
choices, update selectModel to use explicit index mapping for all
three models, skip the search toggle in vision mode (not available),
and extend waitForFilePreview to detect image thumbnails via send
button state since vision mode shows a preview image instead of a
filename label.
Also catch "Not allowed" errors from setFileInput (Cloudflare may
block CDP file operations) so the DataTransfer fallback can run.
Closes#1215
* fix(deepseek): harden vision upload mode
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(chatgpt): fix image generation detection and output path
Three fixes for chatgpt image command:
1. Page navigation: ChatGPT redirects away from the conversation
after sending. Poll for the /c/ URL after send, then periodically
reload the conversation page during image wait to pick up
asynchronously rendered images.
2. Composer selector: add fallback selectors for the chat input
since ChatGPT uses different aria-labels across UI versions.
3. Output path: the default '~/Pictures/chatgpt' was passed as a
literal string without tilde expansion, creating a directory
named '~' in the working directory. Removed the string default
and use os.homedir() fallback instead.
Fixes#1206
* fix(chatgpt): fail fast on image export failures
* fix(chatgpt): avoid reloads during image generation
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(chatgpt-app): support Traditional Chinese UI labels
The send button and Options button matchers only included Simplified
Chinese ("发送", "选项"). On macOS systems with Traditional Chinese as
the system language, the ChatGPT desktop app exposes "傳送" and "選項"
via the Accessibility API, causing `chatgpt-app send` to fail with
"Could not find send button" and `chatgpt-app model` to fail with
"Could not find Options button" for zh-TW / zh-HK users.
Verified via AXUIElement walk on ChatGPT 1.2026.104 / macOS 26 with
system language set to Traditional Chinese.
The "Stop generating" detection at line 314 already handles Traditional
Chinese because 停止生成 uses identical glyphs in both writing systems.
"Legacy models" at line 261 still lacks any Chinese variant but is not
addressed here since the Traditional Chinese translation has not been
verified on a live UI.
* test(chatgpt-app): cover traditional chinese ax labels
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(zhihu): fix identity detection, comment, answer, and search
Identity detection: Zhihu removed __INITIAL_STATE__ and moved the
user avatar from a profile link into a button. Added fallback that
extracts the user slug from the header avatar alt text.
Comment and answer: Zhihu moved the comment editor into a Modal
and changed the submit button behavior, breaking the UI-based
write flow. Replaced with direct API calls (POST /api/v4/answers/
{id}/comments and POST /api/v4/questions/{id}/answers) which are
reliable and much simpler.
Search: Zhihu's search API now returns mixed result types (ads,
education, hot_timing) alongside search_result. Updated the filter
to select by object.type (answer/article/question) and increased
fetch size to compensate for non-content results.
Fixes#1198
* fix(zhihu): rewrite like, follow, favorite to use API
Same DOM breakage as comment/answer. Replaced UI-based click
flows with direct Zhihu API calls for all write commands.
* fix(zhihu): harden api write regressions
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(zlibrary): add search and info commands
Add Z-Library adapter with two browser-based commands:
- `search` — Search books by title, author, or ISBN.
Navigates to /s/<query> and extracts results from
<z-bookcard> shadow DOM custom elements.
- `info` — Get book details and available download formats
from a book page URL.
Uses Strategy.COOKIE with browser automation to bypass
Cloudflare protection. The adapter reuses the user's existing
Z-Library login cookies from system Chrome.
Known limitation: actual file downloading requires Playwright's
download event handling (page.on('download')). OpenCLI's browser
automation does not currently intercept file downloads. Users
needing to download files should use Playwright to navigate to
the book URLs discovered by this adapter.
* fix(zlibrary): harden input and empty extraction
---------
Co-authored-by: jean <jean@jeandeMacBook-Pro.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(deepseek): fix send button detection in sendMessage
The previous selector `btn.closest('div')?.querySelector('textarea')`
always returned null because the button itself is a div, so
closest('div') returns the button, which has no textarea inside.
This caused every send to fall through to the Enter key fallback.
Walk up from the textarea to find the input container, then select
the last enabled non-toggle button with an SVG icon (the send
button). Excludes `.ds-toggle-button` elements (DeepThink / Search
toggles) so only the actual send button is clicked.
* fix(deepseek): fail closed when upload never enables send
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(browser): bind current tab to bound workspace
* docs(browser): document bound session idle semantics
* test(extension): cover bind-current owned-overwrite refusal
Adds regression for the second guard in handleBindCurrent that refuses
binding when the bound:* workspace already has an owned automation
window. Previously only the non-bound prefix path was tested.
* refactor(browser): rename bind command
* fix(browser): bind only current window tabs
* fix(browser): fail unbind when detach command fails
* feat(google-scholar): add cite and profile commands, fix search dedup
- cite: get BibTeX/EndNote/RefMan/RefWorks citation for a paper.
Clicks the cite button in search results and fetches the citation
content from Google's citation endpoint.
- profile: view an author's Google Scholar profile (h-index,
i10-index, citation count, top papers). Accepts author name
or Scholar user ID.
- search: fix duplicate results caused by CSS selector matching
both outer container (.gs_r.gs_or.gs_scl) and inner child
(.gs_ri) for each paper.
Closes#1174, closes#1175
* fix(google-scholar): fail fast on cite and profile misses
* fix(google-scholar): document new commands and lock dedup test
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Pre-navigate Uiverse commands and retry detached browser bridge failures so code and preview flows stop falling back to about:blank. Broaden preview element matching for input-root components and cover the new navigation contract in tests.
* separate author name from date text in search results
* fix(xiaohongshu): constrain author date stripping
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(youtube): channel videos-tab fallback reads wrong tab from InnerTube response
After PR #1109, `opencli youtube channel <id>` still returns empty
`recent_videos` for channels whose Home tab is empty AND whose InnerTube
`/youtubei/v1/browse` response includes multiple tabs.
Root cause: the fallback fetch sends a browse request with the Videos
tab's `params`. The response, however, includes ALL tabs (Home, Videos,
Shorts, ...), with only the requested tab marked `selected: true`. The
existing code reads `tabs?.[0]?.tabRenderer?.content?.richGridRenderer?.contents`
— for multi-tab responses `tabs[0]` is Home (empty), so `richGrid` ends
up `[]` and `recentVideos` stays empty. PR #1109's test channels happened
to return single-tab lists with Videos at index 0, masking the bug.
Fix: find the tab with `selected: true` instead of assuming `tabs[0]`.
Reproducer: `opencli youtube channel UC44DSuDgw7_qccvZzIK3Jpg`
(杀鱼伟-Vi, ~3.1K subs, posts daily). Returns 0 videos pre-patch, 30+
videos post-patch.
`npm run typecheck` clean, `npm test` passes (1952/1952).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(youtube): preserve videos tab fallback
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(doubao): use ID selector for send button
The clickSendButtonScript was searching for the send button by walking up
the DOM tree only 2 levels from the textarea, but the actual send button
#flow-end-msg-send is at level 5. This caused message sending to fail.
Fix by directly selecting the button via its ID.
* test(doubao): update send button selector assertions
* fix(doubao): keep send-button fallback contract
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(toutiao): move NON_TITLE_LINES inside function scope
NON_TITLE_LINES was defined outside parseToutiaoArticlesText() as a
module-level const. When the function is serialized via .toString()
and injected into browser evaluate context, outer scope variables
are not available, causing 'NON_TITLE_LINES is not defined' error.
Fix: move NON_TITLE_LINES inside the function so it's included in
the serialized string.
* test(toutiao): cover serialized articles parser
---------
Co-authored-by: sontjer <sontjer@github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* docs: update extension install to Chrome Web Store link
Extension is now published on Chrome Web Store. Replace manual
download/unpack instructions with the store link across READMEs
and skill docs.
* docs: restore manual install as Option B alongside Chrome Web Store
The shared article-download pipeline strips all <button> elements
via STRIPPED_TAGS, which is correct for article adapters (zhihu,
weixin) but causes web read to silently lose meaningful button
content like "Download All" on generic pages.
Override the button stripping in web read's configureTurndown
callback so button text is preserved as inline content.
Fixes#1184
Restore the original icons (commit b2fa7da) that were replaced by the
v1.6.8 "refresh icons" change in e9867dc. Per user feedback, the original
neon `>_` design read more clearly and was preferred over the abstract
arrow + dash variant.
Reverts only the four icon PNGs (16/32/48/128); manifest, popup, and
extension version stay where they are.
* fix(chatgpt-app): use AX send flow and support zh-CN generating state
* fix(chatgpt-app): fail fast on stale AX send path
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(weixin): add publish (create draft with cover) and drafts (list drafts)
Closes#441
* fix(weixin): rename publish to create-draft to match issue #441 proposal
* fix(weixin): fail fast on draft auth and empty states
* test(weixin): align adapter imports with repo style
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(deepseek): fix history titles and resume conversation on ask
- history: use link.innerText instead of link.querySelector('div') for
title extraction. DeepSeek changed sidebar DOM; the first child div
is now an empty ds-focus-ring element, causing all titles to show as
(untitled).
- ask: when workspace is recycled (idle timeout) and --new is false
(default), click the most recent sidebar conversation link to resume
it instead of staying on the blank new-chat page. Skip model
selection when inside an existing conversation since the selector is
only rendered on the new-chat page.
- ensureOnDeepSeek: return boolean indicating whether navigation
occurred, so callers can react to workspace recycling.
Closes#1149
* fix(deepseek): fail fast on explicit model resume
* fix(cli): expose only explicit option sources
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(sinafinance): match stock symbol in addition to name
The scoring function only compared user input against the Chinese
display name (p[4] from suggest API), so searching "AAPL" matched
"AAPLU" (score 0.8) over Apple Inc. whose name field is "苹果"
(score 0). Check the symbol field first for exact and partial matches.
Fixes#1157
* docs(sinafinance): add missing commands to adapter index
The index table only listed `news` for sinafinance. Added the other
three commands (`rolling-news`, `stock`, `stock-rank`) and updated
the mode from Public to hybrid since rolling-news and stock-rank
require a browser.
Fixes#1156
* test(sinafinance): lock stock symbol matching
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(web,download): absorb #1048 media + --stdout into web read
Distill the useful pieces of the abandoned PR #1048 (`web md`) into the
existing shared pipeline instead of introducing a parallel command:
- Turndown rules for <video> / <audio> / <iframe>. Video and audio are
emitted as inline HTML so renderers that support it keep playback,
and iframes degrade to markdown links (title + src) so embedded
content (YouTube, CodePen, …) stays reachable. `iframe` moves out of
STRIPPED_TAGS since it's now handled explicitly.
- `stdout` option on ArticleDownloadOptions: writes the full markdown
to process.stdout, skips image download + mkdir + file write, and
reports saved='-'. Remote image URLs stay intact so piped output is
self-contained.
- `web read --stdout` wires the above through.
- Lazy-load src rewrite: the extractor now promotes data-src /
data-original / data-lazy-src / data-srcset onto `src` before the
HTML is frozen, so the markdown body and the image-download list
reference the same URL (previously a page with placeholder.gif +
data-src produced broken image links in the output).
Nothing in #1048 that overlapped with the already-merged #1143
hardening was kept — no new Readability wiring, no duplicate Turndown
config, no new command.
* fix(web): keep stdout streaming output clean
* fix(tests): update iframe e2e assertion and drop relative src import
- article-extract e2e fixture test: iframe now converts to a markdown
link instead of being stripped, so assert the YouTube embed link
survives rather than asserting its absence.
- clis/web/read.test.js: replace vi.importActual('../../src/registry.js')
with a direct __test__.command export from read.js; the relative
import into src/ tripped the package-exports adapter guardrail.
* fix(deepseek): separate thinking process from response in --think mode (#1124)
When --think is enabled, the response now includes separate fields:
- response: clean final answer only
- thinking: chain-of-thought reasoning content
- thinking_time: time spent thinking (e.g. '1')
Supports both English ('Thought for X seconds') and Chinese
('已思考（用时 X 秒）') thinking header patterns.
Fixes#1124
* chore: regenerate cli-manifest.json
* fix(deepseek): DOM-level think/response separation, dynamic columns
Blocker 1: Replace fragile split(/\n\n+/) heuristic in parseThinkingResponse()
with DOM-level extraction in waitForResponse(). The page evaluate now queries
distinct DOM nodes (.ds-markdown--think vs .ds-markdown) for thinking and
response content. The text-level parser falls back to treating everything
after the header as thinking (no split), avoiding silent corruption of
multi-paragraph content.
Blocker 2: Remove static columns declaration from askCommand. The renderer
infers columns from row keys, so non-think output only shows 'response'
while think output shows all three columns.
Tests added for multi-paragraph thinking, multi-paragraph answer, and
non-think column regression guard.
* chore: regenerate cli-manifest.json
* feat(download): harden HTML→Markdown pipeline
Inspired by the MD-This-Page / markdown-viewer-extension analysis, tighten
the shared article→Markdown converter used by zhihu/weixin/web adapters:
- enable turndown-plugin-gfm (tables, strikethrough, task lists)
- strip script/style/noscript/iframe/canvas/form/button/dialog unconditionally
- strip SVG via a dedicated rule (not in HTMLElementTagNameMap)
- drop base64 data-URI images so they don't bloat .md output
- post-process: collapse NBSP, lone bullet/middle-dot residue,
trailing whitespace, and 3+ blank lines
- frontmatter shape guarantees ≤2 consecutive newlines even when
some metadata fields are absent
Adds a minimal local .d.ts for turndown-plugin-gfm and 6 new tests
covering GFM conversion, tag stripping, base64 drop, and whitespace cleanup.
* fix(download): emit canonical markdown strikethrough
* feat(download,browser): finish article pipeline polish
Per the follow-up from the MD-This-Page / markdown-viewer-extension
analysis, land the remaining items in the same PR instead of splitting:
article-download.ts
- extend STRIPPED_TAGS with header/footer/nav/aside (page chrome; the
article's title/author/publishTime are supplied as separate fields on
ArticleData, so duplicated DOM is redundant)
- new option ArticleDownloadOptions.cleanSelectors — per-adapter CSS
selector list removed before conversion, applied as a Turndown rule
via node.matches so invalid selectors fail silently
browser/article-extract.ts (new)
- generic Readability-based extraction that runs in-page via CDP
evaluate (no jsdom in Node)
- short-circuits non-HTML documents (text/plain, JSON, XML) and the
single-<pre> "browser rendering a plain text file" case
- clones the document before any mutation (preserves live page state
for subsequent snapshot / click)
- isProbablyReaderable gate, Readability.parse on the clone, then a
fallback chain main → [role="main"] → #main-content → … → body
- library sources are JSON-embedded and eval'd inside a Function scope
so their backticks / module.exports guards don't collide with the
surrounding IIFE
Tests
- article-download: page-chrome strip, cleanSelectors match + invalid
selector silently ignored (2 new)
- article-extract: JS generation contents, default fallback chain,
response normalization, null / malformed handling, and a Function()
parse check to catch any template-literal break-out in the embedded
Readability sources (8 new)
* fix(download): honor selector cleanup in fallback paths
* test(e2e): real-site regression for hardened article pipeline
Adds tests/e2e/article-download-pipeline.test.ts driving `opencli web read`
through 6 representative pages (example.com baseline, Wikipedia GFM tables,
MDN metadata, GitHub fenced code, Vercel SSR blog, Ruan Yifeng CJK+images)
and asserting the post-processing invariants: no base64/script/style leaks,
no blank-line runs, no residue, no trailing whitespace, no NBSP.
Graceful skip on bot detection / transient CDP errors, with a single retry.
All 6 sites pass locally (37s total).
* test(browser): add article extraction e2e fixtures
* feat(51job): add comprehensive 51job adapter (search / hot / detail / company)
Four adapters covering the main 51job surface:
- `51job search <keyword>` — keyword job search via we.51job.com/api/job/search-pc.
Rich filters: --area (40+ city name/alias → 6-digit code), --salary, --experience,
--degree, --companyType, --companySize, --sort, --page, --limit. Response already
carries full jobDescribe + HR + company + encCoId, so most callers won't need detail.
- `51job hot` — same endpoint with empty keyword, returns 51job's recommendation feed.
- `51job detail <jobId>` — scrapes jobs.51job.com/x/<jobId>.html. Returns description,
welfare tags, category, address, age requirement, company meta.
- `51job company <encCoId>` — scrapes jobs.51job.com/all/co<encCoId>.html. Job cards
carry a `sensorsdata` JSON attribute, so we parse that instead of fragile DOM text.
Company meta from `.c-info.ellipsis`, intro from `#companyIntroRef`.
All four are Strategy.COOKIE + browser:true + navigateBefore:false. 51job sits
behind Aliyun WAF — bare curl / Node-side fetch always hits the slider challenge
(tried copying acw_sc__v2 + ssxmod_itna cookies to Node, WAF also checks TLS
fingerprint and JS execution). Only reliable path is browser-context fetch via
`page.evaluate(fetch(url, {credentials:'include'}))`, so utils.js exports
`pageFetchJson` that wraps this pattern + detects WAF-served HTML.
Verify fixtures included (~/.opencli/sites/51job/verify/*.json) — four adapters
pass `opencli browser verify 51job/<cmd>` with rowCount / columns / types /
patterns / notEmpty checks. Eyeballed jobId 171699769 on jobs.51job.com/suzhou
matches adapter output.
* fix(51job): tighten city handling and docs
* chore: regenerate cli-manifest.json after 51job column cleanup
* feat(weread): add ai-outline command for AI-generated book outlines
Two-step API flow: fetch chapter UIDs via authenticated chapterInfos,
then retrieve hierarchical AI outline from public outline endpoint.
Supports --depth to control detail level (2=topics, 3=key points,
4=full details) and --raw for structured output (chapter/idx/level/text)
suitable for programmatic consumption.
Closes#1140
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(weread): tighten ai-outline auth contract
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(weread/book): add fallback selectors for reader page without cover
When the private API session expires, `loadReaderFallbackResult` navigates
to the reader URL. The page now sometimes skips the cover/flyleaf and
renders reading content directly, causing the wait for cover/flyleaf title
selectors to time out.
- Add `.readerTopBar_title_link` to `page.wait` selector (always present)
- Use cascading `firstText()` for title: cover → flyleaf → outline → top bar
- Use cascading `firstText()` for author: cover → flyleaf → outline → document.title
Fixes#1137
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(weread/book): parse author from trailing title segments
* fix(weread): avoid author guess from document title
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaoyuzhou): correct podcast-episodes API endpoint
The endpoint `/v1/podcast/listEpisode` returns 404. The correct
endpoint is `/v1/episode/list` (verified against Xiaoyuzhou iOS app
traffic; also matches the `episode-list` implementation in
ultrazg/xyz, a widely-used Xiaoyuzhou API wrapper).
Additionally, the server requires an `order` field in the request
body (returns 400 if omitted). Add `order: 'desc'` so callers get
the latest episodes first, matching typical UX for a podcast feed.
Before: podcast-episodes -> HTTP 404 for every podcast
After: podcast-episodes returns the N most recent episodes
Tested against real podcast 626b46ea9cbbf0451cf5a962
(张小珺｜商业访谈录) — now returns 140 episodes correctly.
* test(xiaoyuzhou): lock podcast episodes endpoint
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: agent-native retrospective — analyze / verify guards / fixture content checks
Post-mortem on slow 1point3acres + 51job adapter sessions, consolidated
into one PR. Scope is "reduce uncertainty and catch silent failures"
— the two things that sink agent success rate on first-time adapters.
Changes:
- `browser analyze <url>` — one command returns pattern (A/B/C/D),
anti-bot vendor (Aliyun/Cloudflare/Akamai/Geetest), nearest adapter,
and a single-sentence recommended_next_step. Replaces the three-step
open/wait/network recon loop when it can reach a confident verdict.
- `browser wait xhr <regex>` — poll for a specific XHR URL instead of
blind `wait time N`, so SPA data-arrival barriers are deterministic.
- Fixture `mustNotContain` / `mustBeTruthy` — catch two silent-failure
modes `notEmpty` misses: content contamination (sibling DOM bleed)
and `|| 0` / `|| false` fallbacks.
- `browser verify` post-success site-memory check + `--strict-memory`
— verify-green no longer hides the case where `~/.opencli/sites/`
was never written back. Memory only materializes if authors write it.
- CI: guard that committed `cli-manifest.json` matches a fresh build.
Main was already drifted (#1118 left stale ordering + a missing arg);
this PR regenerates the manifest and will catch the next drift.
Docs (opencli-adapter-author + opencli-autofix skills):
- `success-rate-pitfalls.md` — 10 concrete silent-failure scenarios
seen in real adapter sessions, each with defense via fixture /
adapter patterns.
- `autofix` gains discipline rule #6: verify pattern failure means
tighten the adapter, never loosen the fixture.
- `site-recon.md` leads with `browser analyze`; `api-discovery.md`
adds a §0 covering WAF vendor detection and cross-subdomain CORS
(the two gotchas that burned the 51job session).
- `wait time 3` → `wait time 2`, with `wait xhr` as the robust choice.
* fix: make output-dir defaults host-independent in manifest
Three adapters (chatgpt/image, gemini/image, instagram/download) baked
`path.join(os.homedir(), ...)` into the `default` field of their args.
The committed manifest therefore carried my personal `/Users/jakevin/...`
paths — which agents running on a different host saw as surprising
defaults. The drift guard I just added to CI caught it on the first run.
Runtime behavior is unchanged: each adapter still falls back to
`path.join(os.homedir(), …)` inside `func` when the kwarg is absent.
Only the displayed / registered default becomes a tilde-path.
* fix(cli): enforce strict-memory without fixture
* fix(browser): harden analyze and xhr guards
* fix(browser): fallback to interceptor buffer
* feat(verify): fixture-based value validation + skill docs for COOKIE pitfalls
`opencli browser verify` now loads `~/.opencli/sites/<site>/verify/<cmd>.json`
when present and validates row count / columns / types / patterns / notEmpty
against the live adapter output. Without a fixture, behavior is unchanged
(just runs the adapter and prints). New flags `--write-fixture`,
`--update-fixture`, `--no-fixture` seed / refresh / bypass the spec.
Motivation: previous verify only checked that the adapter exited 0 and
produced *something* — shape regressions (author name bleeding across rows,
a column silently becoming null after a site refresh, duplicated thread-level
time on every post) all passed "✓ Adapter works!" and shipped broken.
Skill doc updates (opencli-adapter-author):
- adapter-template.md: new "COOKIE adapter 骨架" section — HttpOnly +
dual-domain cookie read via `page.getCookies`, Node-side fetch for HTML
(explaining why `page.evaluate(fetch(...))` is the wrong tool when
`navigateBefore: false` or the response is non-UTF-8), and empty-state
sentinel row over `[]`
- api-discovery.md §4: note that BBS engines (Discuz/phpBB/vBulletin) set
auth cookies on the root domain + HttpOnly, so single-domain `getCookies`
calls silently miss them
- SKILL.md Step 10/12: make `--write-fixture` part of the runbook, forbid
debug dumps outside `~/.opencli/sites/<site>/fixtures/` or `/tmp/`
* fix(verify): support positional argv in fixture args + site-memory docs
Reviewer feedback blocker: fixture.args was Record<string, unknown>,
expanded as --key value only, so positional-subject adapters
(<tid>/<url>/<query>) couldn't be verified. Repo convention is
"主语优先 positional".
- verify-fixture.ts: args now accepts Record<string, unknown> | unknown[].
Object → --k v pairs; array → verbatim passthrough. New helper
expandFixtureArgs() centralizes the branching.
- cli.ts verify action: swap inline expansion for expandFixtureArgs().
- verify-fixture.test.ts: 6 new cases covering array form, mixed
positional+flag, empty shapes, passthrough stringification.
- site-memory.md: Layer 2 tree now lists verify/<cmd>.json; new schema
block distinguishes it from fixtures/<cmd>-<ts>.json; runbook timing
section gets a Step 10 verify-write row. Repo-tree debug-dump ban
clarified.
- adapter-template.md: new "Verify fixture" section with named-flag and
positional recipes, honest about --write-fixture only seeding named.
Smoke-tested 1point3acres/thread (positional <tid>): fixture round-trip
green (args=["1173710","--limit","2"]).
Two issues surfaced post-merge of #1110 by the Copilot reviewer:
1. Help text and docs advertise `video URL` as a valid input for
`opencli bilibili video <bvid>`, but the original implementation
delegated the whole input to `resolveBvid()` — which only recognises
bare `BV...` IDs and `b23.tv` short codes. A canonical bilibili URL
like `https://www.bilibili.com/video/BV.../` therefore got rewritten
to `https://b23.tv/www.bilibili.com/video/BV.../` and failed before
ever calling the view API.
Fix: pre-extract the BV ID from `bilibili.com/video/<BV>...` and
`bilibili.com/bangumi/play/<BV>...` URLs (www / m. / with or without
query string) in `video.js`, and fall through to `resolveBvid()`
only for bare BV IDs and `b23.tv` links.
2. `description` was being truncated to 200 chars with whitespace
collapsed before being returned. JSON/YAML consumers silently lost
the full `desc` value. Other bilibili adapters return raw fields.
Fix: return the full `d.desc` verbatim and let consumers/display
layers handle formatting.
Adds four regression tests for the URL paths (full URL, URL with query
string, m.bilibili.com mobile URL) and one for description integrity
(> 200 chars, preserved verbatim).
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(deepseek): use position-based model selection instead of text matching
Fixes#1111
* fix(deepseek): preserve explicit instant model contract
* fix(deepseek): guard expert selector arity
---------
Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Adds two additive columns to the Twitter read commands (search, timeline,
tweets, thread, likes):
- has_media: boolean — true if the tweet contains any photo, video, or GIF
- media_urls: string[] — photo URLs and mp4 variant URLs for videos/GIFs,
extracted from legacy.extended_entities.media (falls back to entities.media)
The INTERCEPT/COOKIE payloads already carry this data; this change only
extends the row-mapping layer, so no new network work is needed. Pattern
mirrors #465 (time column).
Shared extraction helper lives in clis/twitter/shared.js so all five
adapters stay consistent, with unit coverage for photo, video (mp4 variant
selection), animated_gif, entities.media fallback, and the empty case.
Closes#1107
* docs(skills): restore and rewrite opencli-usage as orientation skill
The original opencli-usage skill was deleted in PR #1094 as part of the
skill consolidation, but its role (top-level orientation to what opencli
is, how to discover adapters, what flags/env/formats are universal, and
which specialized skill to load next) was not covered elsewhere. Restore
it, but deliberately NOT as a verbatim copy:
- Drop the hand-maintained 100-adapter table. There are 100+ adapters
and the list moves every week — `opencli list -f json` is the source
of truth agents should call at the start of a task.
- Replace it with the meta-structure agents actually need: the three
pillars (adapters / browser driving / external CLI passthrough), the
strategy tags (PUBLIC | COOKIE | HEADER | INTERCEPT | UI | LOCAL)
and what each implies for prerequisites, universal flags (-f, -v),
output formats, env vars, self-repair hook, adapter authoring paths,
plugins, external CLI passthrough.
- Explicitly list the commands PR #1094 removed (`explore`, `record`,
`web` / `desktop` top-level groups) so agents don't attempt them.
- Cross-link to the four post-consolidation skills: opencli-browser
(ad-hoc driving), opencli-adapter-author (writing adapters),
opencli-autofix (repair flow), smart-search (search routing).
Adapter-author description updated to stop claiming it replaces
opencli-usage.
* docs(skills): tighten opencli-usage validate + doctor scope per review
- validate: describe as registry-level semantic check (description, domain,
pipeline step names, func|pipeline|_lazy presence, arg duplicates), not
YAML/TS syntax check — matches src/validate.ts
- doctor: narrow to browser-bridge diagnostic; PUBLIC/LOCAL adapters, list,
validate, verify, plugins, and external-CLI passthrough do not need it
Backfill the 1.7.6 section that was missing from the release PR.
Covers window lifecycle flags, selector-first browser interactions,
agent-native payload, compound form fields, three new adapter commands,
four fixes, skill doc updates, and extension 1.0.2 body-truncation
contract unification.
--live (OPENCLI_LIVE=1) keeps the automation window open after an adapter
command finishes, so agents or humans can inspect the page state. Default
behavior (immediate closeWindow) is unchanged.
--focus (OPENCLI_WINDOW_FOCUSED=1) surfaces the existing env-var toggle as a
CLI flag so users don't need to shell-export to see the window in foreground.
Both flags are parsed early in main.ts and stripped from argv, so they can be
placed anywhere on the command line and work on any subcommand (adapter or
browser).
Restore `skills/opencli-browser/SKILL.md`, deleted in #1094, rewritten for
the post-#1116 browser CLI surface: selector-first target contract,
`match_level { exact | stable | reidentified }`, compound fields for
date/time/select/file, structured error codes with `available` vs
`candidates`, new `find` / `extract` / `network --filter` commands,
html tree budgets, tabs/frames, cost guide, recipes, pitfalls.
Review tightened two contract-drift bugs before merge:
- `browser tab list` envelope field is `page`, not `targetId`
- `network --ttl` default is `24h`, not `~5min`
2 reviewers green (codex-mini1, First-principles-1); CI all-green.
* feat(browser): compound expansion + cascading stale-ref + bbox 0.99 dedup
Three agent-native upgrades inspired by browser-use, landed as one PR
because they share the same target / snapshot / find surface.
1. Compound expansion (compound.ts)
Date/time/datetime-local/month/week, select, and file inputs now
emit a `compound` JSON field on `browser find --css` entries with
format, current value, min/max (date family), full options list
+ selected (select), accept / multiple / files[] (file). Kills
the three biggest form-page failure modes (wrong date format,
guessed options, re-uploaded files) without extra round-trips.
2. Cascading stale-ref (target-resolver.ts)
Numeric ref resolution now walks three tiers before giving up:
exact → stable (tag + strong id match, soft signals drifted) →
reidentified (original ref lost, fingerprint uniquely found a
live element; re-tag + refresh identity). Every success envelope
carries `match_level` so callers can tell which tier matched.
SPA re-renders / i18n label swaps no longer stall agents.
3. BBox 0.99 containment for interactive descendants (dom-snapshot.ts)
Adds a second dedup tier on top of the existing 0.95 non-interactive
one. When a parent is a propagator (tag a/button OR role button/
link/menuitem/tab/option) and a child is interactive but
undistinctive (no aria-label/id/testid/name/form-control), fold
it into the parent — removes `[1]<button> [2]<svg> [3]<span>`
noise on icon buttons.
Tests: 287/287 pass (src/browser + src/cli.test.ts). Typecheck clean.
* fix(browser): address reviewer blockers on PR #1116
- compound select: walk ALL options to collect selected labels, not just
the first 50 we serialize. Fixes dropdowns where the selected entry
sits past COMPOUND_SELECT_OPTIONS_CAP (e.g. country lists, timezones)
reporting current: "" even though the user picked a valid option.
- match_level: propagate the cascading match tier
(exact / stable / reidentified) through IPage.click/typeText/scrollTo,
BasePage, and the cli command envelopes (click / type / select /
get text|value|html|attributes). Agents now see in JSON that the
resolver had to fall back, instead of the tier being swallowed.
- compound contract is now also emitted by `browser state`
(per-ref compounds: sidecar) and by `browser get html --as json`
(compound field on each node), not only by `browser find --css`.
Closes the gap where agents using the default snapshot still
round-tripped `find` for every date / select / file control.
Adds targeted regression tests for each blocker + updates cli.test.ts
mocks to the new envelope shape.
The resolveTwitterQueryId() function in shared.js fetches an external JSON
file from GitHub without a timeout. If the network request stalls, the
function never resolves and the twitter article command hangs indefinitely.
Add a 5-second AbortController timeout so the fetch fails fast and falls
back to the local script-scanning strategy. This fixes the reported hang
when opencli twitter article loses network connectivity.
* feat(bilibili): add video command
Add `opencli bilibili video <bvid|url|short-link>` to fetch one
video's metadata via the public /x/web-interface/view endpoint.
Returns title, author, category, publish time, duration, view /
danmaku / reply / like / coin / favorite / share counts, parts,
thumbnail, and description as a key/value table.
Reuses `resolveBvid` and `apiGet` from clis/bilibili/utils.js to
stay consistent with the existing bilibili adapters
(subtitle/search/etc. all follow the same navigate + apiGet
pattern). Non-zero API codes surface as CommandExecutionError.
Fills a visible gap: existing bilibili commands cover search,
hot, subtitle, ranking, user-videos etc., but nothing returned
metadata for a single video — `web read` only gets a DOM shell.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(manifest): register bilibili video command
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* WIP: deepseek file upload (blocked by 30s idle timeout)
* feat(deepseek): add file upload support via --file flag
Closes#1092
* fix(deepseek): use native file input path for --file
---------
Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(youtube): fall back to Videos tab when Home tab has no videos (#1108)
Some channels have no video shelves on their Home tab, causing
`opencli youtube channel <id>` to return an empty `recent_videos` list
even though the channel has videos visible in the browser.
When the Home tab extraction finds zero videos, the command now makes
a second InnerTube browse request to the Videos tab and extracts from
its richGridRenderer format.
* fix(youtube): make Videos tab fallback locale-safe
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(browser): selector-first find + get/click/type/select with JSON envelope
A2: new `browser find --css <sel>` — structured JSON (matches_n + entries[]) so
agents can go from semantic selector directly to a list of candidates without
parsing free-text snapshot output. Per-entry shape: nth/ref/tag/role/text/attrs/
visible. Attr whitelist kept small (11 high-signal fields), invisible elements
still returned so agents can reason about offscreen vs missing.
A3: get text/value/attributes now accept a selector-first <target> (numeric ref
OR CSS) and emit `{value, matches_n}`. Bonus scope (approved by reviewers):
click/type/select share the same contract with `--nth <n>`, emitting
`{clicked|typed|selected, target, matches_n, ...}` on success.
Unified structured error envelope across all selector-first commands:
{ error: { code, message, hint?, candidates?, matches_n? } }
with codes invalid_selector / selector_not_found / selector_ambiguous /
selector_nth_out_of_range (CSS) plus not_found / stale_ref (numeric ref).
Write commands reject multi-match CSS without `--nth` as selector_ambiguous;
reads default to "first match wins" but always expose matches_n so agents
notice ambiguity. `resolveTargetJs` is the single source of truth; click /
typeText / scrollTo share a `runResolve` helper in BasePage.
No back-compat shims per design directive.
125 targeted tests green; tsc clean.
* fix(browser): unify selector surface + allocate fresh refs in find
Two blockers from PR #1112 review:
1. First-principles-1 (blocker): `browser find --css` now allocates fresh
numeric refs for untagged matches. It scans `window.__opencli_ref_identity`
(and any stray `data-opencli-ref` attrs) for the current max, allocates
`max+1` upward, writes `data-opencli-ref` on the element, and populates
the identity map with the same fingerprint shape snapshot uses (tag,
role, text, ariaLabel, id, testId). `find -> click <ref>` now works on
fresh pages without requiring `browser state` first. Type changed from
`ref: number | null` to `ref: number`.
2. codex-mini1 (blocker): removed the `isCssLike` regex
(`^[a-zA-Z#.\[]`) in `resolveTargetJs`. Valid selectors like `:root`,
`:has(...)`, `*` used to short-circuit to "Cannot parse target" before
reaching `querySelectorAll`, so `find --css` accepted them but
`get/click/type/select` did not. Now: numeric → ref path, everything
else → querySelectorAll, and the browser parser decides. Same selector
surface across all selector-first commands.
Tests added:
- target-resolver: pseudo-selectors flow into CSS branch (not rejection)
- find: ref allocation writes attribute + identity map; fingerprint shape matches resolver
- cli: find envelope now expects numeric refs
127 targeted tests green; tsc clean.
* feat(browser): agent-native payload — network bodies, html tree budgets, extract command
Three fixes/additions driven by agent-usage gaps, as one complete change:
- network (P0 fix): lift silent 4000-char body truncation in CDP + extension
paths to an 8MB memory-guard cap, and surface body_truncated / body_full_size
/ body_truncation_reason in the --detail envelope so the agent sees when a
body was cut. List view also exposes body_truncated_count and per-entry flag.
Adds --max-body flag for explicit caller-side capping.
- get html --as json (P1): add --depth / --children-max / --text-max budget
knobs on the tree serializer, plus a truncated={depth,children_dropped,
text_truncated} envelope that only appears when a budget is hit. Lets the
agent narrow DOM output without walking away empty-handed.
- extract (P2 new command): agent-native article/content channel. Scope →
denoise (strip nav/header/footer/scripts/forms/etc.) → HTML→markdown via
existing htmlToMarkdown → paragraph-boundary-aware chunk with stateless
next_start_char resume cursor. Agents no longer misuse `get html` to read.
* fix(browser): unify body-truncation signal contract across raw/detail/fallback
Addresses review blockers on #1104:
- NETWORK_INTERCEPTOR_JS fallback no longer silently drops bodies above the
per-entry cap. Raised cap to 1 MiB (ring stays at 200 entries), and on
overflow keeps the string prefix + sets `bodyTruncated` / `bodyFullSize`
so `browser network` propagates the same agent-visible signal the CDP /
extension paths emit.
- `CachedNetworkEntry` schema switches from internal camelCase
`bodyTruncated` to the user-facing `body_truncated` / `body_full_size`
fields. `--raw` emits cache entries verbatim, so this removes the
snake_case/camelCase split across list / --detail / --raw.
- Adds a `--raw` truncation-contract test that also asserts the camelCase
fields do not leak through.
Agents often know what fields a target request's body should contain
but not which captured request carries it. --filter lets them declare
the field set and get back only matching entries.
Matching is "any-segment": a field matches when it equals any segment
name of any inferShape() path (ignoring root $, array indices, and
bracket-quoted key syntax). Multiple fields AND together. Case-sensitive.
- invalid_filter for empty / commas-only values
- invalid_args when combined with --detail (mutually exclusive)
- 0 matches is a valid empty result, not an error
- persisted cache stays unfiltered so later --detail lookups still resolve
Envelope gains `filter` (echo) and `filter_dropped` (count of entries
passing the static-resource filter but not --filter). Existing --raw
and --all compose normally.
* feat(browser): remove silent html truncation, add --as json tree output
`browser get html` had two agent-hostile defaults:
1. A silent 50000-char cap on the returned HTML — agents that got a
truncated page had no signal they were looking at half the DOM.
2. Only raw HTML string output, forcing agents to re-parse for
structured extraction.
Changes:
- Default output is now the full outerHTML, no truncation
- `--max <n>` opts in to a character cap; when the cap actually
trips, the HTML is prepended with
`<!-- opencli: truncated N of M chars; re-run without --max ... -->`
so agents always see the signal
- `--as json` returns `{selector, matched, tree}` where `tree` is
`{tag, attrs, text, children}` recursively. `matched` is the full
count of selector matches so agents know when more elements exist
beyond the first. `text` is the node's own direct text children,
whitespace-collapsed; child elements live in `children`.
- `--selector` not matching any element now emits structured
`{error:{code:"selector_not_found", ...}}` with a non-zero exit
code, in both raw and json modes (was `(empty)` stdout previously,
indistinguishable from empty element)
- Invalid `--as` / negative `--max` emit structured
`invalid_format` / `invalid_max` error codes
Extracted the tree serializer as `src/browser/html-tree.ts` so the
JS expression can be unit-tested against a DOM stub.
* fix(browser get html): structured errors for invalid selector & strict --max
Both edges previously bypassed the structured-error contract introduced in
#1102, which agents rely on for branching:
- Invalid CSS selector: querySelector(All) would throw SyntaxError through
page.evaluate into the generic exception path. Wrap the lookup in try/catch
inside page context for both raw and --as json paths; surface as
{error:{code:"invalid_selector", message}} + non-zero exit.
- --max validation: parseInt silently accepted "1.5" -> 1 and "10abc" -> 10.
Switch to a strict /^\\d+$/ check so fractional, negative, and non-numeric
values all return {error:{code:"invalid_max"}}; validation runs up front so
bad values never reach the page.
Covered by new unit tests in cli.test.ts (fractional, non-numeric, invalid
selector on raw + json) and html-tree.test.ts (SyntaxError -> invalidSelector
envelope).
Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
---------
Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
* feat(browser): rewrite network command for agent-native discovery
Replace the index-based list + pretty-printed --detail flow with a
structured JSON interface built around stable keys, body-shape previews,
and a persistent capture cache. Agents can now reference captured
requests by operationName (GraphQL) or `METHOD host+pathname` (REST)
instead of array indexes that shift on every rerun.
- `browser network` now emits JSON: `{workspace, captured_at, count,
filtered_out, entries: [{key, method, status, url, ct, size, shape}],
detail_hint}` — no body payloads by default
- Shape inference (src/browser/shape.ts) walks response JSON into a
flat path -> descriptor map with depth cap 6 and a 2KB budget per
entry, so agents see structure without paying body tokens
- Stable key generator (src/browser/network-key.ts) derives
`operationName` from graphql URLs and `METHOD host+pathname`
elsewhere, disambiguating collisions with `#N` suffixes
- Persistent cache (src/browser/network-cache.ts) snapshots every
capture to `~/.opencli/cache/browser-network/<workspace>.json` with
a 24h TTL, so `--detail <key>` survives later commands
- `--detail <key>` returns `{key, url, method, status, ct, size, shape,
body}` with structured error codes (cache_missing / cache_expired /
cache_corrupt / key_not_found, the latter including available_keys)
- Add `--raw` for agents that want every full body inline, `--ttl` for
cache lookups
- Update opencli-adapter-author + opencli-autofix skill docs to
reference `--detail <key>` and the shape-first discovery flow
Supersedes the cache prototype in #1051.
Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
* fix(browser): structured errors for capture/save, shape budget guard
Self-review findings on the network refactor:
- captureNetworkItems throwing (browser crashed / CDP dropped) now emits
`error.code: capture_failed` on stdout rather than leaking a bare
stderr line from browserAction's generic handler — agents get a
parseable JSON blob on every failure path, matching the design goal.
- saveNetworkCache throwing (disk full, read-only path) is a soft
failure: the captured data is already in hand, so surface a
`cache_warning` field in the envelope and keep going instead of
aborting. `--detail` lookups on that run will miss the cache but the
listing still reaches the agent.
- shape.ts: guard the sub-walk on `add()`'s return value so the
"budget hits on the array/object descriptor itself" path can never
emit a stray child without its parent marker.
- network-key.ts: document that `#N` suffixes start at `#2` — the first
occurrence stays bare, there is no `#1`. Matches test + code.
Added regression tests: `capture_failed` on readNetworkCapture throw,
`cache_warning` on persistence failure, shape budget hit on array descriptor.
Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
---------
Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
* feat(twitter): add tweets command for fetching a user's recent posts
Adds `opencli twitter tweets <username> [--limit N]` to pull a user's
most recent chronological tweets via the UserTweets GraphQL endpoint.
Long posts resolve via note_tweet, pinned entries are skipped, and
retweets are flagged. QueryIds resolve dynamically through
`resolveTwitterQueryId` with hardcoded fallbacks.
* fix(twitter): expose retweet flag in tweets output
* refactor: consolidate 6 skills into 3, remove mechanical commands
Replaces opencli-oneshot / opencli-explorer / opencli-browser /
opencli-usage with a single opencli-adapter-author skill that takes
the AI agent end-to-end: site recon, API discovery, field decoding,
adapter coding, and `opencli browser verify`.
Removes the mechanical commands (`explore`, `synthesize`, `generate`,
`cascade`, `record`) and their src/tests — they were codegen scaffolding
meant for agents, which the new skill handles more flexibly via
`opencli browser` primitives.
Skill highlights:
- Top-level decision tree + 12-step runbook
- 5 site patterns (SPA / SSR / JSONP / Token / Streaming)
- 5-layer API discovery (network → initial state → bundle → token → interceptor)
- Field decode playbook (self-explanatory → codes → sort-key comparison)
- Output design guide (columns, types, order, ≤15 per adapter)
- Two-layer site memory: in-repo seeds for eastmoney/xueqiu/bilibili/tonghuashun
plus local `~/.opencli/sites/<site>/` runtime workspace
Kept skills: opencli-autofix (now points to adapter-author for rewrites),
smart-search. Kept primitives: `browser *`, `doctor`, `list`, `validate`,
`verify`, `<site> <cmd>`, `plugin *`, `completion`.
No backward compatibility shims. Full test suite (1605 tests) passes.
* review fixes: honest coverage, hard memory-hit path, typo, stale docs
- site-memory hit path no longer jumps to writing adapter; forces Step 5
endpoint re-verification + Step 7 field check, and 30-day expiry
- site-memory.md now specifies exact schemas for endpoints.json /
field-map.json / notes.md / fixtures + write-back timing rules
- coverage-matrix.md marks unverified patterns as 🟡 with an evidence
section citing coingecko dry run + PR #1091 eastmoney + bilibili
- eastmoney seed typo: resolveSecids -> resolveSecid (and splitSymbols)
- docs/developer/ai-workflow.md rewritten to teach the adapter-author
skill + opencli browser * primitives (dropped generate/synthesize/
cascade/explore references)
- ts-adapter.md, getting-started.md, CHANGELOG.md:87 updated to point
at opencli-adapter-author
* fix(ci): resync package-lock + drop stale built-in list reference
- Regenerate package-lock.json to restore @emnapi/core + @emnapi/runtime
entries that got dropped during the rebase — `npm ci` was failing on all
CI jobs (build / audit / docs-build / bun-test / unit-test)
- docs/guide/getting-started.md: built-in list dropped `explore`, now
reads (list, validate, verify, browser, doctor, plugin...)
* fix(ci): restore package-lock.json from main (unrelated lockfile churn)
Mirror the remaining 13 read-oriented adapters and the shared _secid.js
helper from the author's local workspace into the repo, so that
clis/eastmoney/ becomes the full Phase A codegen regression oracle
described in OpenCLI Improvement Spec v1.1 §B.10.
Total repo oracle after this PR: 14 adapters under clis/eastmoney/
(hot-rank.js already exists; this PR adds the other 13) plus the
_secid.js normalize helper.
Covers the two schema-expressiveness gaps discovered during prep:
- CSV row_format: kline.js decodes "YYYYMMDD,open,close,..." strings
- :row_index source: convertible.js derives rank = i + 1
_secid.js is the canonical example of the v1.1 §B.7 helper contract
(pure normalize/derive function, serializable I/O, no env/fs/net/session
access, does not drive pagination/retry/fallback).
This PR is oracle-only, carries no framework changes. Phase A framework
PR depends on this merging first so the codegen diff target is stable.
Refs: task #177 / spec v1.1 §B.10
* feat(download): show saved file path in web read and weixin download output
Closes#1038
* test(download): cover saved article path
---------
Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(browser): add cross-origin iframe support via CDP execution contexts
Enable interaction with cross-origin iframes through CDP's execution
context mechanism, without requiring content scripts or all_frames.
- Track frame execution contexts via Runtime.executionContextCreated events
- Add 'frames' action to list all child frames (including cross-origin)
- Support frameIndex in 'exec' action to evaluate JS in specific frames
- Add Page.frames() and Page.evaluateInFrame() APIs for CLI consumers
- Tag cross-origin iframes with [F0]/[F1] indices in DOM snapshots
- Add Page.getFrameTree to CDP allowlist
Closes#1077
Change-Id: Id03361ddb616912dff3bfa8e59e8b68716de590b
* fix(browser): align cross-origin iframe routing contract
* fix(browser): unify iframe frame-index routing
---------
Co-authored-by: xuezhangying <xuezhangying@bytedance.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
From first principles, `opencli browser` commands exist for AI Agents to
operate websites through the browser skill. Reframe both READMEs to reflect
this: show users how to install the skill into their AI agent and describe
tasks in natural language, rather than listing raw CLI commands.
* feat(twitter): rewrite lists via GraphQL + add list-tweets
The DOM-scraping / detail-click approach in PR #1053 remained fragile
against X's frequent overview-page rendering changes and slow (N+1 page
loads per list). Rewrite `twitter lists` to call
`ListsManagementPageTimeline` GraphQL directly — one request returns all
owned + subscribed lists with id/name/member_count/subscriber_count/mode.
Also add `twitter list-tweets <listId>` for pulling the tweet stream from
a list, completing the read-side chain (lists → pick an id → list-tweets).
- lists: drop positional `user` arg (GraphQL returns only logged-in
user's lists), add `id` column, change followers to exact integer from
subscriber_count.
- list-tweets: same GraphQL pattern as bookmarks/likes (BEARER + ct0 +
dynamic queryId with static fallback + cursor pagination).
- Delete obsolete lists-parser.js and lists.d.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(twitter): add list-add / list-remove with Save-button commit
Two new commands to toggle list membership. X's list dialog uses a
"click-to-stage, Save-to-commit" pattern — clicking a row only updates
optimistic UI; the actual POST fires when the user clicks the top-right
"Save" button. Pressing ESC or the close-X silently cancels the change.
Implementation:
- Resolve listId → name via ListsManagementPageTimeline GraphQL, so we
match the dialog row by name (dialog rows have no data-testid listId).
- Open profile page → DOM click "…" menu → "Add/remove from Lists".
- Scroll dialog to locate target row (virtualized list).
- page.nativeClick on row — trusted CDP Input.dispatchMouseEvent fires
React's onclick, flips aria-checked (.click() alone does not suffice;
X ignores non-trusted events for list mutations).
- page.nativeClick on the Save button — commits to server.
- Verify by re-fetching ListsManagementPageTimeline and diffing
member_count: success only if N→N±1. No silent successes.
This fixes the pattern where batch `list-add` calls returned success for
every user but committed zero to the server (optimistic UI lied).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: stabilize twitter list manifest and query ids
* docs: add twitter list command discoverability
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(zsxq): separate content field from title, remove title truncation
- Split getTopicText to return only title, add getTopicContent for body text
- Remove .slice(0, 120) that was truncating titles
- content field now contains full body text instead of duplicating title
* fix(zsxq): preserve title fallback for body-only topics
---------
Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
PR #1059 migrated xiaoyuzhou from SSR scraping to authenticated API.
The E2E tests run without credentials, producing exit code 78
(CONFIG_ERROR). The existing `isExpectedChineseSiteRestriction` guard
only caught FETCH_ERROR, PARSE_ERROR, and NOT_FOUND — not config
errors from missing auth credentials.
Three improvements from the design debt audit:
1. Remove deprecated `tabId` field and `getActiveTabId()` method
- Delete `tabId` from DaemonCommand (daemon-client.ts) and Command (protocol.ts)
- Delete `getActiveTabId()` from IPage interface (types.ts) and Page class (page.ts)
- Update extension resolveCommandTabId() to remove legacy fallback
- Update handleTabs select case to remove tabId check
- The tab→page migration is now complete
2. Unify argument validation into single code path
- Remove `normalizeArgValue()` from commanderAdapter.ts
- Commander adapter now passes raw values to prepareCommandArgs()
- All coercion (bool, int, number) and validation (required, choices)
happens once in coerceAndValidateArgs() in execution.ts
- Eliminates duplicated boolean normalization
3. Remove dead plugin filesystem wrappers
- Delete `promoteDir()` — never called in production code
- Delete `replaceDir()` — thin wrapper over beginReplaceDir, never called
- Remove corresponding test-only exports and tests
- Rename PromoteDirFsOps → ReplaceDirFsOps to match remaining usage
- Transaction infrastructure (runTransaction, beginReplaceDir,
beginReplaceSymlink) retained — used by publishStandalonePlugin
and publishMonorepoPlugins for atomic multi-step operations
* fix(extension): per-workspace idle timeout for browser sessions (#1058)
The global 30s WINDOW_IDLE_TIMEOUT was too aggressive for interactive
`opencli browser` commands where users type manually between invocations.
- browser:*/operate:* workspaces now default to 10 min idle timeout
- Adapter workspaces keep the existing 30s timeout
- Support custom timeout via OPENCLI_BROWSER_TIMEOUT env var (seconds)
or command-level idleTimeout parameter
- Surface sessionExpired warning when a new window is created after
the previous session timed out
- Fix stale comment (said 120s, actual was 30s)
Closes#1058
* fix: resolve sessionExpired double-delete race and timeout override lifecycle
Addresses @codex-coder review blockers:
1. sessionExpired flag was never set because getAutomationWindow()
consumed expiredWorkspaces before handleCommand() could check it.
Fix: use .has() in getAutomationWindow, only .delete() in handleCommand.
2. workspaceTimeoutOverrides was never cleaned up — once set, it
persisted until extension restart. Fix: clear override on idle
timeout expiry, explicit close-window, and borrowed-session detach.
Adds 5 tests covering:
- browser:* uses 10min timeout (not 30s)
- sessionExpired flag is set and consumed correctly
- workspaceTimeoutOverrides cleared on idle expiry
- workspaceTimeoutOverrides cleared on explicit close
- idleTimeout from command applies to workspace override
* refactor: remove sessionExpired warning per product decision
@WAWQAQ decided session-expired warning is not needed.
Remove expiredWorkspaces tracking, sessionExpired flag from protocol,
and related CLI-side warning code. Keep per-workspace timeout and
override lifecycle cleanup.
* fix: clean up workspaceTimeoutOverrides on user-initiated window close
The windows.onRemoved listener was missing workspaceTimeoutOverrides
cleanup, causing stale overrides to persist across sessions when users
manually close the automation window.
* fix(xiaoyuzhou): migrate from broken SSR scraping to authenticated API (fixes#1023)
Xiaoyuzhou removed SSR rendering — /podcast/<id> and /episode/<id> pages
now return 404, breaking fetchPageProps() which scraped __NEXT_DATA__.
Migrate podcast, podcast-episodes, episode, and download commands to use
the existing authenticated API client (requestXiaoyuzhouJson) that
transcript.js already uses successfully.
Changes:
- podcast.js: use /v1/podcast/get API endpoint
- podcast-episodes.js: use /v1/podcast/listEpisode API endpoint
- episode.js: use /v1/episode/get API endpoint
- download.js: use /v1/episode/get API endpoint
- utils.js: remove unused fetchPageProps, keep format helpers
- Update all affected tests (download.test.js, utils.test.js)
- Change strategy from PUBLIC to LOCAL (requires credentials)
* fix(xiaoyuzhou): align local strategy contract
* fix(xiaoyuzhou): align local api metadata
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
#1025 merged hot-rank adapters for eastmoney/tdx/ths but the
accompanying docs were missing. This breaks the Doc Check CI
workflow on every PR ('--strict' mode, exits non-zero when
`scripts/check-doc-coverage.sh` finds adapters without docs),
blocking merges across the board.
Adds a doc page per adapter, registers them in the adapters
index table, and adds sidebar entries in the VitePress config.
* feat: add hot stock ranking adapters for eastmoney, tdx, ths
Add three new site adapters for Chinese stock hot rankings:
- eastmoney/hot-rank: 东方财富热股榜
- tdx/hot-rank: 通达信热搜榜
- ths/hot-rank: 同花顺热股榜
All use Strategy.COOKIE browser mode with page.evaluate() DOM scraping.
Each includes co-located tests (13 tests total, all passing).
* fix(tdx,ths): add symbol validation and deduplication in evaluate()
Add seen Set for deduplication and skip entries with empty symbol/name,
matching the pattern already used in eastmoney/hot-rank.js.
* fix: refine hot-rank selectors based on browser inspection
- eastmoney: use table.rank_table tbody tr with td index-based extraction,
fix name from a[title] to avoid post content contamination
- tdx: use div.top-cell[data-code] data attributes for reliable extraction,
add tags column from div.tips-item.gnbk
- ths: use card-based layout selectors, remove price column (not in UI),
extract tags from div.tag.PFSC-R
* fix(hot-rank): align tdx and ths columns with actual output
* fix: register hot stock ranking adapters
---------
Co-authored-by: dengjingren <dengjingren@cn.wilmar-intl.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix: preserve network capture and surface extension mismatch diagnostics
Older Browser Bridge installs can still connect to the daemon while
missing two capabilities we now rely on: the network-capture actions
and the extension version handshake. That created three user-facing
failure modes with real impact:
1. `opencli explore ...` crashed with `Unknown action: network-capture-start`
against an old extension, so exploration stopped before any site
analysis finished.
2. `opencli doctor` and `opencli daemon status` could show a healthy
connection even when the extension never reported a version, which
hid the compatibility problem and sent users toward the wrong fix.
3. After reloading a new extension, `explore` could still report
`Endpoints: 0 total, 0 API` because `handleNavigate()` detached the
debugger before top-level navigation and cleared the active network
capture state right before the page load we needed to observe.
Fix this in two layers:
- Teach `Page` to treat unsupported `network-capture-*` actions as an
old-extension compatibility case. It now warns once, memoizes the
unsupported state, and returns empty capture data instead of throwing.
- Teach `doctor` and `daemon status` to treat "connected but version
unknown" as a warning instead of a healthy state, so version-handshake
failures are visible immediately.
- Preserve the debugger attachment while network capture is armed, so
the initial navigation keeps the capture state alive and the extension
can record requests from the first page load.
Before:
- `opencli explore ...` -> `Error: Unknown action: network-capture-start`
- `opencli doctor` -> `[OK] Extension: connected` / `Everything looks good!`
- `opencli daemon status` -> `Extension: connected` even when the
extension version was missing
- `opencli explore ...` after reloading the extension -> `Endpoints: 0 total, 0 API`
After:
- `opencli explore ...` on an old extension -> warns once and continues
- `opencli doctor` -> `[WARN] Extension: connected (version unknown)`
- `opencli daemon status` -> `Extension: connected (version unknown)`
- `opencli explore ...` on the reloaded extension keeps network capture
armed across navigation instead of clearing it before the page load
* fix: reset network capture flags on closeWindow()
Prevents stale _networkCaptureUnsupported flag from persisting across
sessions when the user reinstalls or reloads the extension mid-session.
* fix: startNetworkCapture returns boolean to prevent false-positive on old extensions
When the extension doesn't support network-capture-*, startNetworkCapture()
now returns false instead of silently resolving. This ensures browser open/
network correctly falls back to the JS interceptor on old extensions.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix: auto-restart stale daemon and improve connection error messages
When daemon is running but extension never connected (stale daemon started
before extension was installed), the CLI now auto-restarts the daemon to
give the extension a fresh WebSocket endpoint, instead of just waiting
and then telling the user to install the extension.
Also improves error messages across cli.ts, bridge.ts, and doctor.ts to
suggest "opencli daemon stop && opencli doctor" as the quick fix, since
that's what actually resolves the issue.
* fix: version-aware stale daemon detection and improved error messages
- Daemon /status now includes `daemonVersion` field
- bridge.ts: when daemon is running but extension not connected, checks
daemonVersion vs CLI version. Only auto-restarts if version mismatch
(stale daemon from older CLI). Same-version daemon shows improved error
message with "opencli daemon stop && opencli doctor" hint.
- doctor.ts: explicitly identifies stale daemon (version mismatch) in
diagnostics report, shows daemon version in status line
- cli.ts: error message changed to suggest "opencli daemon stop && opencli doctor"
* fix: treat missing daemonVersion as stale, verify shutdown before respawn
- Missing daemonVersion (pre-version daemon) is now treated as stale,
covering the most common user scenario (old daemon without version field)
- After requestDaemonShutdown(), poll until daemon actually stops (port
released) before spawning new one, with 3s timeout
- If shutdown request fails, log warning instead of silently proceeding
- doctor.ts also treats missing daemonVersion as stale with clear message
* fix: fail explicitly when stale daemon replacement fails
- If shutdown request fails or port isn't released within 3s, throw
'Stale daemon could not be replaced' instead of blindly spawning on
an occupied port
- Add tests for all three stale-daemon branches: same-version (no
restart), missing daemonVersion (stale), mismatched version (stale)
* fix: use type-based error dispatch in browserAction instead of string matching
browserAction() now checks `instanceof BrowserConnectError` first and
renders both message and hint, instead of string-matching on message
content. This ensures stale daemon errors ("Stale daemon could not be
replaced") surface the actionable hint to the user.
* feat(grok): add image command for grok.com image generation
Add `opencli grok image <prompt>` which submits a prompt via the existing
grok.com browser session and returns the generated image URLs from the
latest assistant bubble.
Because assets.grok.com URLs are gated by Cloudflare and cannot be
downloaded with a plain HTTP client, the --out flag triggers an in-page
fetch(credentials: 'include') so the browser session's cookies and
referer are attached, then writes the decoded blob to disk.
Flags:
- --new start a fresh chat before sending
- --timeout max seconds to wait for the image (default 240)
- --count minimum number of images to wait for before returning
- --out directory to save downloaded images
Ships with unit tests for the helpers (isOnGrok, normalizeBooleanFlag,
dedupeBySrc, imagesSignature, extFromContentType, buildFilename).
* fix(grok): harden image composer and bubble detection
* fix(grok): harden image flow and docs
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
When ChatGPT macOS app is set to Chinese language, the "Options"
button label becomes "选项". This change checks for both English
and Chinese labels to find the button.
Co-authored-by: mad <mademing@maddeMac-mini.local>
Co-authored-by: Claude <noreply@anthropic.com>
* feat: implement Ref-Backed Locator for browser actions
Introduces a unified target resolution system with fingerprint
verification and structured error diagnostics.
Snapshot phase:
- Each interactive element now gets a fingerprint (tag, role, text,
ariaLabel, id, testId) stored in window.__opencli_ref_identity
- Zero overhead: metadata is already available during DOM walk
Resolution phase (new target-resolver.ts):
- Numeric input → ref path with fingerprint verification
- CSS-like input → querySelectorAll with uniqueness check
- No more silent first-match: ambiguous selectors are rejected
Error model (new target-errors.ts):
- stale_ref: element identity changed since snapshot
- ambiguous: CSS selector matched multiple elements (with candidates)
- not_found: element not in DOM or invalid input
- All errors include actionable hints for AI agents
base-page.ts:
- click() and typeText() now use two-phase resolve-then-act
- Existing CDP fallback for click preserved
* feat: migrate scrollTo to unified resolver pipeline
scrollTo now uses the same two-phase resolve-then-act pattern as
click and typeText, getting fingerprint verification and structured
error diagnostics (stale_ref/ambiguous/not_found) for free.
* fix: address review — stronger fingerprint verification & surface TargetError in CLI
1. Fingerprint verification now uses the full identity vector (tag, id,
testId, ariaLabel, role, text) instead of just tag/role/text. Strong
identifiers (id, testId) are decisive; remaining signals use majority
voting. Fixes false negatives where same-tag elements swapped.
2. browserAction() now renders TargetError with code, hint, and
candidates list instead of just the message string.
* fix: migrate get/select/type-autocomplete to unified resolver
- browser get text/value/attributes now resolve via resolveTargetJs
instead of raw querySelector, getting fingerprint verification and
structured errors for free
- browser select uses selectResolvedJs on __resolved element
- type command's autocomplete detection uses isAutocompleteResolvedJs
on the already-resolved element
- Fix empty-string text prefix match: fp.text="Login" + text="" no
longer falsely passes fingerprint check
- docs/index.md: update feature cards to match README Highlights
- docs/zh/index.md: sync Chinese feature cards
- docs/guide/getting-started.md: align Highlights section
- README.zh-CN.md: rename "为什么是 OpenCLI" to "亮点", align with EN
* fix: remove duplicate extension zip from releases
The release and build-extension workflows were creating both
opencli-extension.zip and opencli-extension-v{version}.zip (identical
content), causing both to be uploaded. Keep only the versioned filename.
* docs: update extension zip filename to versioned format
Update all references from opencli-extension.zip to
opencli-extension-v{version}.zip to match the workflow change.
* feat(mubu): add mubu (mubu.com) adapter with 5 commands
Commands: doc, docs, notes, recent, search.
- Uses COOKIE strategy; API calls via in-page XHR with Jwt-Token
from localStorage (matches the web app's own mechanism).
- Renders node trees to Markdown (default) or plain text;
supports tables, tasks, images, emoji, mentions, strikethrough,
underline, and nested structures.
- notes supports flexible time ranges: single day, month, year,
or custom --from/--to spans, plus a --list overview mode.
- search returns full-text matches with hit count and snippets
for both folders and documents.
* fix(manifest): register mubu commands in runtime manifest
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* refactor: unify OPENCLI_VERBOSE and DEBUG=opencli into one mechanism
Three debug output levels (verbose/debug/diagnostic) was redundant.
Merge DEBUG=opencli into OPENCLI_VERBOSE so `-v` flag controls all
verbose/debug output through a single mechanism.
- log.verbose() now checks both OPENCLI_VERBOSE and DEBUG=opencli
- log.debug() becomes an alias for log.verbose() (backward compat)
- boss/utils.js verbose helper simplified to check OPENCLI_VERBOSE only
- DEBUG=opencli still works as fallback (no breaking change)
* fix(boss): preserve debug fallback for verbose logs
The error path in executeCommand did not call page.closeWindow(),
leaving the automation window open until the extension's idle timer
fires. On Windows, MV3 service worker suspension makes this timer
unreliable, causing windows to linger indefinitely.
Now closeWindow is called after diagnostic collection but before
rethrowing, ensuring the window is closed on both success and failure.
Add missing env vars to both README and README.zh-CN:
- OPENCLI_SKIP_FETCH: skip adapter sync on global install
- OUTPUT: override output format (json/yaml/table)
- DEBUG=opencli: internal debug logging
- DEBUG_SNAPSHOT: DOM snapshot debug output
* fix: clean up stale .yaml adapter files from older versions (#953)
Users upgrading from v1.6.x retain .yaml adapter files in
~/.opencli/clis/ that trigger "Ignoring YAML adapter" warnings on
every run. The hash-based sync only tracks .js files, so these
legacy .yaml files are never cleaned up.
Add a cleanup step (3b) that removes .yaml/.yml files from user
adapter directories when the corresponding site exists in the
official package (i.e., the site has been migrated to .js).
* fix(fetch-adapters): narrow stale yaml cleanup
* feat: decouple extension version from CLI version
Extension and CLI had tightly coupled version numbers (both 1.7.2),
requiring manual sync across 3 files on every release. This decouples
them so each can release independently.
Changes:
- Extension version reset to 1.0.0 with independent versioning
- Extension sends compatRange (e.g. ">=1.7.0") in hello message
so doctor can check CLI/extension compatibility
- Daemon stores and exposes extensionCompatRange via /status
- Doctor uses compatRange for compatibility checks (falls back to
major-version check for older extensions without compatRange)
- Doctor shows extension update availability from cached GitHub
Releases data
- release.yml always builds and attaches extension zip to every
CLI release, so users always find both in the same release page
- build-extension.yml triggers on ext-v* tags (not v*) to avoid
duplicate builds
* fix: version extension release assets
* fix: code audit round 2 — pruneEmptyDirs, evaluateWithArgs, hot-reload, error cause chain
1. pruneEmptyDirs: use path.relative() instead of startsWith() to prevent
false boundary matches on overlapping directory names
2. evaluateWithArgs: add safe evaluate method that auto-serializes args via
JSON.stringify, preventing injection by design
3. Hot-reload: detect mtime changes on user adapter files in daemon mode,
invalidate module cache so edits take effect without restart
4. toEnvelope: preserve error cause chain in verbose mode for better
production debugging
* fix: address review feedback on code audit round 2
- pruneEmptyDirs: resolve() paths before relative() check
- evaluateWithArgs: validate keys are valid JS identifiers
- hot-reload: only bust ESM cache on reload, not first load
- toEnvelope: move cause serialization into toEnvelope itself
so all consumers (AI agents, MCP tools) get cause chain
* fix: address code audit findings (C1-C4, I1, I4, I6)
Security:
- C1: Fix page.evaluate injection in browser type/select commands and
6 adapter files by using JSON.stringify for user input interpolation
- C2: Close WebSocket on CDP connect timeout to prevent resource leak
- C3: Reject CDP connect promise on Page.enable failure instead of
silently swallowing the error
Reliability:
- C4: Guard against corrupted adapter-manifest.json hashes to prevent
false-positive override deletion
- I1: Throw on pre-navigation failure instead of warn-and-continue
- I4: Use Map<string, Promise<void>> for lazy module loading to prevent
concurrent double-imports of the same adapter
Performance:
- I6: Replace O(n) registry alias cleanup with O(k) direct deletion
* fix: address self-review findings on PR #981
- C1: add quotes around CSS selector attribute values in browser
type/select to match other commands (get text/value/attributes)
- C2: clear this._ws in timeout handler to prevent race with open event
- C4: refine corruption guard — treat null/undefined hashes as empty,
only skip sync for truly invalid types (string, number, array)
* feat(clis/chatgptweb): add ChatGPT web image generation command
Add `opencli chatgptweb image` command that generates images using
ChatGPT web (GPT-4o image generation) and saves them locally.
Features:
- Navigates to chatgpt.com/new with full page reload to ensure clean state
- Uses Playwright's page.type() for reliable text input in TipTap editor
- Closes sidebar if open (covers the chat composer on some layouts)
- Polls for response completion (handles thinking/throttling states)
- Extracts generated images from DOM (backend-api/estuary/content URLs)
- Downloads and saves as PNG/JPEG files to user-specified directory
- Supports --op for output directory and --sd to skip download
Files:
- clis/chatgptweb/image.js: CLI command definition
- clis/chatgptweb/utils.js: DOM helpers, send/wait/export functions
Works cross-platform (Linux/macOS/Windows) via OpenCLI browser automation.
* fix(chatgptweb): stabilize image generation flow
* docs(chatgptweb): add browser adapter guide
---------
Co-authored-by: Tony Simons <tony@tonysimons.dev>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Support browsing second-hand houses, neighborhoods, rentals, and
transaction records on ke.com with city/district/price filtering.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(maimai): add talent search with multi-dimensional filters
Add maimai.cn talent search adapter with support for:
- Keyword search (query)
- Company filtering (multiple companies supported)
- School filtering (with 985/211 options)
- Location filtering (province/city)
- Work experience and education level filters
- Industry and position filters
- Direct chat availability
- Sort by relevance, activity, work years, or education
Features:
- Reuses Chrome login session for authentication
- Extracts candidate info: name, job title, company, work history
- Shows work years, education, age, active status
- Displays skill tags and mutual friends count
* fix docs and strategy for maimai adapter
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(discord-app): add delete command to remove a message by ID
Adds a new `delete` command for the discord-app CLI that deletes a
message in the active channel by its snowflake ID. Uses the UI strategy
to hover the message, open the "More" menu, click "Delete Message", and
confirm the deletion dialog.
* docs: add binance adapter doc and update discord doc with delete command
* feat(twitter): add lists command to retrieve user lists
Add twitter/lists command that fetches Twitter/X lists for a user.
Supports:
- Lists with member and follower counts
- Private/public mode detection
- Default to current user if no user specified
- Works for any Twitter user
* docs: add lists command to twitter commands in README
Add twitter lists command to Built-in Commands table in both
English and Chinese README files
* fix(twitter): parse lists from card DOM instead of locale-specific page text
---------
Co-authored-by: isanwenyu <isanwenyu@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(zsxq): accept topic_id as string in getTopicFromResponse
The ZSXQ API returns topic_id as a string, but getTopicFromResponse()
only checked for typeof === 'number', causing it to fall through and
return null. This made 'opencli zsxq topic <id>' fail with NOT_FOUND
for all valid topic IDs.
* fix(zsxq): use group-scoped topic endpoint instead of bare /v2/topics/{id}
The ZSXQ API requires topics to be fetched within their group context.
Change /v2/topics/{id} -> /v2/groups/{groupId}/topics/{id} for both
the detail and comments endpoints. Also adds optional --group_id arg.
* fix: include adapter tests in default npm test
`npm test` only ran unit + extension projects, so adapter tests
(clis/**/*.test.js) were never exercised by the default test command.
Add --project adapter so they run alongside unit and extension tests.
* test: include adapter project in default npm test
* refactor: smart sync adapters instead of full copy (#sparse-override)
Replace unconditional full-copy of all adapters to ~/.opencli/clis/ with
hash-based smart sync that only copies files whose content has changed.
Changes:
- fetch-adapters.js: use SHA-256 content hashes to skip unchanged files;
store per-file hashes in adapter-manifest.json
- discovery.ts: simplify ensureUserAdapters() to only create the directory
(no longer triggers full copy on first run)
- main.ts: fix fast completion to check manifest file existence instead of
directory existence (sparse override may have empty user dir)
- cli.ts: add `opencli adapter eject/reset/status` commands for managing
local adapter overrides
- engine.test.ts: add tests for empty user dir and ensureUserAdapters
* fix: address review blockers — site-level sync + reset --all
1. Fix `adapter reset --all`: change <site> from required to optional
argument so --all can be used without specifying a site name.
2. Change smart sync from file-level to site-level granularity:
if any file in a site has changed upstream, overwrite the entire
site directory. This matches the agreed product semantics — local
modifications to any file in a site are replaced when upstream
updates that site.
* fix: delete old site dir before writing updated adapter files
When a site has upstream changes, delete the entire site directory
first, then write the new version. This prevents stale files from
older versions lingering in the user directory.
* fix: reset --all preserves custom sites, only removes official overrides
Blocker 3 fix: reset --all now checks BUILTIN_CLIS to identify official
sites and only deletes those, preserving user-created custom sites.
* refactor: sparse sync deletes local overrides instead of copying new versions
Changed fetch-adapters.js semantics per team agreement:
- When an official site has upstream changes, DELETE the local override
instead of copying the new version into ~/.opencli/clis/
- Runtime automatically falls back to package baseline
- ~/.opencli/clis/ becomes a true sparse override layer
* fix: reset <site> rejects custom sites, only allows official overrides
Single-site reset now checks BUILTIN_CLIS before deleting, matching
the same protection that reset --all already has.
* fix: reset <site> allows custom sites per product decision
Per @WAWQAQ: explicit single-site reset should work on custom sites too.
Differentiate messaging: official sites say "using official baseline",
custom sites say "removed custom site".
reset --all still only removes official overrides (bulk safety).
* fix: reset --all deletes all local sites including custom per product decision
Per @WAWQAQ: --all should clear the entire local working cache,
including custom sites. Single-site reset already handles both types.
Binance was the only adapter left in src/clis/ after the TS→JS
migration (PR #928). Move all 11 adapters and the test file to
clis/binance/, strip TypeScript syntax from the test, and switch
the test import to the @jackwener/opencli/pipeline package export.
log.debug() requires DEBUG=opencli to output, which means
DEBUG_SNAPSHOT=1 alone no longer shows snapshot fallback diagnostics.
Use process.stderr.write directly since the DEBUG_SNAPSHOT guard
already controls when this diagnostic fires.
Users who created custom .ts adapters in ~/.opencli/clis/ will see
their commands silently disappear after upgrading to the JS-only
version. Add an explicit warning so they know to convert to .js.
The alias resolution logic checked `!registry.has(target)` before
calling `registry.get(target)`, which always returned undefined.
Moreover, aliases registered as `site/alias` keys meant `registry.has`
returned true, skipping the block entirely. The canonical name was
never resolved, so `validate site/alias` silently checked 0 commands.
Simplify to always resolve via `registry.get(target)` which handles
both canonical keys and alias keys correctly.
Older versions (pre-1.7.1) shipped adapters as .ts files. When users
upgrade to a .js-only version, the old .ts files are left orphaned in
~/.opencli/clis/. Add a cleanup step that removes .ts files when a
corresponding .js official adapter exists.
* perf: P0 performance optimizations — VM context reuse, startup parallelization, stealth caching
1. Reuse VM sandbox context in pipeline template engine instead of creating
a new vm.createContext() on every expression evaluation. This eliminates
~0.3ms per call in map/filter loops over large arrays.
2. Cache sanitizeContext() results via WeakMap keyed by object reference.
In pipeline loops, `args` and `data` are the same object across all
iterations — the expensive JSON round-trip now runs only once per step.
3. Parallelize independent startup I/O: built-in CLI discovery now runs
concurrently with ensureUserCliCompatShims and ensureUserAdapters,
saving ~30-50ms on cold start.
4. Cache the stealth JS string (350 lines, pure static) after first
generation — every subsequent goto() reuses the cached string.
* fix: address review feedback on P0 perf optimizations
1. sanitizeContext: cache JSON string instead of parsed object to prevent
sandbox mutation from polluting subsequent calls
2. VM sandbox: clean non-whitelisted properties before each execution to
prevent cross-expression state leakage
3. Startup parallelization: document registry overwrite semantics and
confirm no shared-state race between parallel tasks
* refactor(validate): switch from YAML scanning to registry-based validation
The validate/verify commands only scanned YAML files, which are no
longer supported. Rewrite to validate commands from the in-memory
registry populated by discoverClis(), aligning with the JS-first
adapter architecture.
New checks: missing description, browser commands without domain,
pipeline step name typos, commands without func/pipeline, duplicate
arg names, and positional arg ordering.
* fix(validate): treat lazy-loaded commands as valid
Manifest-registered commands have _lazy=true and no func/pipeline
until execution time. Recognize this as a valid execution form.
* fix(validate): warn on empty registry, support alias targets
- Emit warning when registry is empty instead of silent PASS
- Resolve alias targets to canonical key before filtering
* fix: project hygiene — docs, lint, daemon restart, code fence
- Update Node version requirement from >= 20 to >= 21 in 7 doc files
(README, README.zh-CN, installation guides, troubleshooting)
- Update adapter count from 79+ to 87+ in READMEs
- Remove duplicate `lint` script (identical to `typecheck`)
- Fix TESTING.md CI matrix: Node ['22'] instead of ['20', '22']
- Fix autofix SKILL.md code fence escaping (\``` → ~~~)
- Add daemon restart to postinstall so updated adapters are picked up
- Fix preuninstall to respect OPENCLI_DAEMON_PORT env var
* fix: align docs and skills with JS-first adapter contract
Adapters are now .js files (not .ts). Update all references across:
- README.md, README.zh-CN.md, CONTRIBUTING.md
- docs/guide/getting-started.md, docs/index.md
- skills/opencli-browser/SKILL.md, skills/opencli-explorer/SKILL.md
The runtime (discovery.ts) only loads .js from user clis/ directories,
and `opencli browser init` generates .js scaffolds. Documentation was
still teaching users to create .ts files.
* fix: update CI matrix to Node 22 only (drop Node 20)
package.json requires Node >= 21 (styleText dependency). The CI matrix
was still testing Node 20 which doesn't meet this requirement.
* fix: revert incorrect daemon restart from postinstall
The daemon (browser bridge) only handles CDP communication — it has no
knowledge of adapters. Adapter discovery, loading, and execution all
happen in the CLI process, which is fresh each invocation. The
_loadedModules cache in execution.ts is process-local and not a real
staleness concern. Remove the unnecessary restartDaemon() call.
Strategy is a 5-value enum (PUBLIC/COOKIE/HEADER/INTERCEPT/UI) that
the execution path was reading at two points — resolvePreNav() and
shouldUseBrowserSession() — to make decisions that are already fully
expressible by the existing `browser` and `navigateBefore` fields.
This commit introduces normalizeCommand() inside registerCommand(),
which expands strategy into concrete runtime fields at registration
time. After normalization, execution code never reads cmd.strategy.
normalizeCommand expansion rules:
- strategy → browser: PUBLIC defaults to false, others to true.
Explicit browser value always wins.
- strategy + domain → navigateBefore:
· COOKIE/HEADER + domain → 'https://{domain}' (pre-navigate)
· Non-PUBLIC without domain → true (needs auth context, no URL)
· PUBLIC → undefined (no auth needed)
Explicit navigateBefore (false or string) always wins.
This matters because commands enter the registry from 4 sources
(cli(), manifest, generate-verified, tests), and previously only
cli() did strategy derivation. The other 3 constructed CliCommand
directly, leaving strategy as a runtime dependency. Now all sources
converge through registerCommand → normalizeCommand.
Changes:
- registry.ts: add normalizeCommand(); simplify cli() to delegate
all derivation to normalizeCommand via registerCommand()
- execution.ts: resolvePreNav() no longer reads strategy; just
reads the already-expanded navigateBefore field. Strategy import
removed.
- capabilityRouting.ts: shouldUseBrowserSession() checks
cmd.navigateBefore (truthy = needs browser session) instead of
cmd.strategy !== PUBLIC. Strategy import removed.
- discovery.ts: manifest path no longer hardcodes browser default;
delegates to normalizeCommand.
- capabilityRouting.test.ts: test now reflects normalized command
shape (navigateBefore: true for COOKIE without domain).
strategy is preserved as metadata on CliCommand — opencli list,
cascade probe, adapter generation, and documentation continue to
read it. Only the execution path stops consuming it.
Add Step 6 to the autofix skill: after a verified local fix, prepare a
GitHub issue draft and file it (with user confirmation) via `gh issue
create`. Pure skill/documentation approach — no new runtime code.
Closes the need addressed by #936 with zero code, zero tests to maintain.
* fix: sync package-lock.json with package.json dependencies
package-lock.json was missing @emnapi/core@1.9.2 and
@emnapi/runtime@1.9.2 (transitive deps of @emnapi/wasi-threads),
causing `npm ci` to fail on all CI jobs.
* fix: resolve remaining CI failures after TS-to-JS adapter migration
- vitest.config.ts: update adapter project include/exclude from .test.ts
to .test.{ts,js} to match converted adapter test files
- check-doc-coverage.sh: skip adapter directories containing only utility
files (prefixed with _), fixing false positive for clis/slock/
- linux-do/topic-content.test.js: fix hardcoded reference to topic.ts
(now topic.js after PR #928 migration)
* fix: clean up legacy shim files and stale tmp files on upgrade
Add cleanup steps to fetch-adapters.js that run on every version upgrade:
1. Remove legacy compat shim files from ~/.opencli/ (registry.js,
errors.js, utils.js, etc.) that were created by an older approach
using file:// re-exports. Current approach uses node_modules symlink.
Only deletes files containing "export * from 'file://" to avoid
removing user-created files.
2. Remove legacy compat shim directories (browser/, download/, errors/,
etc.) using the same safety check.
3. Clean up stale .plugins.lock.json.tmp-* files left behind by
crashed processes. These accumulate over time (108 found on one
machine) and clutter ~/.opencli/.
* fix: check every file in legacy shim directories before deleting
Instead of checking only the first file and deleting the entire
directory, now checks each file individually and only deletes files
matching the shim pattern. Directory is removed only if empty after
individual file cleanup.
- Remove mapDistToSource() from diagnostic.ts — mapped dist/clis/
paths back to clis/ but dist/clis/ no longer exists after JS-first
migration. The function always returned null.
- Simplify resolveAdapterSourcePath() to check candidates directly
without the dead dist→source mapping detour.
- Delete scripts/clean-yaml.cjs — walked dist/clis/ to delete YAML
files, but dist/clis/ no longer exists.
- Remove clean-yaml script entry from package.json.
1. candidateToJs: escape single quotes in site, name, domain, and arg
name/type fields to prevent syntax errors in generated JS adapters.
Previously only description and help fields were escaped.
2. diagnostic: pass network request body through redactText() to
prevent sensitive data (JWT, bearer tokens) from leaking into
repair context. responseBody/responsePreview already used
sanitizeCapturedValue which calls redactText, but the body field
only had truncation.
* refactor(adapters): convert adapter layer from TypeScript to JavaScript
Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.
Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition
* fix(main): use findPackageRoot for BUILTIN_CLIS path
The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.
Use findPackageRoot() which works for both dev and prod paths.
* fix(build-manifest): import compiled JS from dist/clis/ instead of raw TS
Node's type stripping does not rewrite '.js' → '.ts' in import
specifiers, so dynamically importing .ts source files fails whenever
they contain relative imports like './utils.js'.
Switch to scanning dist/clis/ for compiled .js files after tsc runs.
This eliminates all 268 "Cannot find module" warnings and increases
manifest entries from 254 to 532 (previously half were silently skipped).
* fix: write manifest to dist/cli-manifest.json where runtime expects it
The runtime resolves BUILTIN_CLIS to dist/clis/ (relative to
dist/src/main.js), so discoverClis() looks for manifest at
dist/cli-manifest.json. Previously it was written to the package root
where the runtime never found it — manifest was effectively unused,
always falling through to filesystem scanning.
* refactor(errors): unify error output as YAML envelope to stderr
Replace the 100+ line chalk renderError() switch-case with a single
YAML envelope output path. All errors now output a structured
{ok, error: {code, message, help, exitCode}} envelope to stderr,
regardless of TTY status.
This simplifies the error system from 5 mechanisms to 3:
1. Error Envelope (YAML → stderr) — unified error output
2. Exit codes (sysexits.h) — process exit semantics
3. Diagnostic (OPENCLI_DIAGNOSTIC=1) — autofix repair context
Removed: chalk error rendering, ERROR_ICONS map, classifyGenericError
regex classifier, BrowserConnectError-specific bridge status display.
Added: toEnvelope() utility, ErrorEnvelope type.
* refactor(errors): migrate adapters to throw CliError, update docs
- Migrate xueqiu adapters from return [{error,help}] to throw CliError
- xueqiu/utils.ts: fetchXueqiuJson now throws AuthRequiredError/
CommandExecutionError instead of returning {error, help} objects
- Remove resolveColumns error fallback from output.ts (no longer needed)
- Add verbose stack trace support to error envelope
- Add ADAPTER_LOAD to AutoFix hint trigger codes
- Update skill docs (adapter-templates, explorer, oneshot, advanced-patterns)
to recommend throw CliError pattern instead of return [{error, help}]
* fix: remove remaining dead error-forwarding in 4 xueqiu adapters + review fixes
- Remove `if ('error' in d) return [d]` from feed, hot, search, kline
(fetchXueqiuJson now throws, so these were dead code)
- Add `stack?: string` to ErrorEnvelope interface (removes type cast hack)
- Fix adapter-templates.md: use AuthRequiredError instead of plain Error
* fix: migrate barchart/quote and yahoo-finance/quote to throw CliError
Last two adapters that silently returned [] on error instead of
throwing CommandExecutionError.
* fix: self-review fixes — doc evaluate crash, error messages, kline consistency
- adapter-templates.md: getServerContext was throwing AuthRequiredError
inside a function serialized into page.evaluate() (browser has no
CliError). Reverted to return {error} sentinel + func() body throw.
- yahoo-finance/quote, barchart/quote: include symbol in fallback error msg
- xueqiu/kline: throw EmptyResultError instead of returning [] for
consistency with other xueqiu adapters
* docs(skills): add Tier 2.5 localStorage Bearer, SPA discovery, and test standards
From real-world experience building slock.ai CLI adapters:
- oneshot: add network-empty diagnosis, SPA baseURL bundle search, Tier 2.5
localStorage Bearer template (with multi-tenant X-Server-Id pattern),
updated auth quick-reference, file path note, opencli browser verify test flow
- explorer: add Tier 2.5 to decision tree and strategy table, update test section
with opencli browser verify + Done standard, fix Step 5 path to ~/.opencli/clis/,
add 4 new pitfall rows (SPA HTML, 400 context header, empty network, wrong dir)
* docs(skills): fix path conflict + add anti-change patterns from real adapters
Fix reviewer blocking issue:
- Remove the contradictory "~/.opencli/clis/" note that mixed user-local and
repo-contributor workflows; replace with explicit two-scenario callout in
Step 4, Step 5, pitfall table, and oneshot test section
- Template comments in oneshot restored to clis/<site>/<name>.ts (repo path)
Add "抗变更模式" section to explorer, based on opencli's own production code:
- Pattern 1: dynamic queryId discovery (twitter/shared.ts resolveTwitterQueryId)
— scan loaded JS bundle by operationName (stable) to find queryId (unstable)
- Pattern 2: semantic DOM priority fallback (web/read.ts)
— article > [role=main] > main > class-hint > body, pick largest text block
- Pattern 3: ordered selector array + timestamp comments (xiaohongshu/publish.ts)
— first-match wins, comment records UI version and observed attribute values
- Pattern 4: nullish-coalescing field multi-path (xiaohongshu/user-helpers.ts)
— covers camelCase/snake_case variants without assuming fixed key name
* docs(explorer): split SKILL.md into reference sub-documents
- Shrink main SKILL.md from 994 to 270 lines — core workflow only
- Extract all TS templates (Tier 1~4, pagination) to references/adapter-templates.md
- Add error handling standard: { error, remedy } pattern (remedy > hint)
- Add Tier 2.5 localStorage Bearer template with multi-tenant X-Server-Id example
- Extract cascading requests, tap debug, verbose mode, anti-change patterns to references/advanced-patterns.md
- Extract record workflow to references/record-workflow.md
* docs(skills): fix verify command — split by dev scenario
browser verify only reads ~/.opencli/clis/, not repo's clis/.
Split all verify instructions:
- Repo 贡献: npm run build + opencli <site> <cmd>
- 私人 adapter: opencli browser verify <site>/<name>
Fixes blocker in explorer:L209, L224 and oneshot:L286, L298
* docs(adapter-templates): add utils.ts extraction pattern for same-site adapters
* docs(skills): add decision matrix, stop conditions, sync comments
explorer: add path decision matrix before core workflow
oneshot: add explicit stop/switch conditions (when to escalate to explorer)
both: add keep-in-sync comment on the two-scenario verify block
* feat(slock): extract utils.ts + apply { error, help } pattern; docs: remedy→help
slock/utils.ts: new — getSlockContext(), resolveChannelId()
- Shared token + workspace resolution, no more 4-line duplication
- UUID regex (/^[0-9a-f]{8}-...$/) replaces fragile !includes('-')
- Returns { error, help } instead of throwing
tasks.ts / members.ts / send.ts:
- Import from utils.ts, remove all duplicated auth boilerplate
- All errors return [{ error, help }], no more throw
- members.ts: add limit arg (was unbounded before)
docs: rename remedy → help across all skill references
* refactor(adapters): migrate pipeline adapters to func() with { error, help } pattern
- slock: agents, channels, messages, servers now use getSlockContext/resolveChannelId
from utils.ts; error handling uses { error, help } return instead of bare throws
- linux-do: export fetchLinuxDoJson from feed.ts; migrate search, topic, categories,
tags, user-posts, user-topics from pipeline+throw to func() using fetchLinuxDoJson
- xueqiu: add utils.ts with fetchXueqiuJson helper; migrate hot, feed, search, stock,
watchlist, hot-stock, groups, kline, earnings-date from pipeline+throw to func()
* fix(output): show error rows in table/csv/markdown when columns declared
When a command declares columns (e.g. ['rank', 'title', 'value']) but
returns an error row ({ error, help }), the declared columns would
render empty cells. Now resolveColumns detects the error key and falls
back to the row's actual keys, making diagnostics visible in all output
formats.
* chore: remove slock adapters from this PR
Slock adapters should be in a separate PR, not bundled with the
adapter refactor and skill docs improvements.
* docs(skills): unify browser tool names to `opencli browser` commands
Replace abstract MCP tool names (browser_navigate, browser_snapshot,
browser_network_requests, browser_click, browser_evaluate) with
concrete `opencli browser` CLI commands in explorer and oneshot skills.
This aligns all three browser-related skills into a clear hierarchy:
- opencli-browser: atomic command reference
- opencli-oneshot: 4-step quick generation workflow
- opencli-explorer: full site exploration workflow
* docs(skills): address review — demote explore, fix eval placeholder
1. Demote `opencli explore` from "recommended" to "supplementary helper"
and make `opencli browser` the explicit primary path for API discovery.
2. Fix `url` undefined variable in eval example — use `<API URL>` placeholder.
* feat: auto-close adapter windows, add OPENCLI_WINDOW_FOCUSED, document config
1. Adapter commands now close the automation window immediately after
completion instead of waiting for the 30s idle timeout.
2. OPENCLI_WINDOW_FOCUSED=1 opens automation windows in the foreground
(useful for debugging). Default remains background.
3. Add Configuration section to README (EN/ZH) and opencli-usage skill
listing all stable user-facing environment variables.
* Fix OPENCLI_WINDOW_FOCUSED to be per-request, not frozen at daemon startup
Move env var read from daemon (startup-time constant) to CLI side
(sendCommandRaw), so it works correctly with the persistent daemon model.
Each request now reads the env var fresh and includes windowFocused in
the command payload.
* fix(xiaoe): resolve missing episodes for long courses by handling lazy load
* fix(xiaoe): keep lazy-load scroll until inner list stabilizes
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* refactor: make daemon persistent, remove idle timeout
- Remove IdleManager and 4-hour idle auto-exit
- Daemon now stays alive until explicit shutdown or uninstall
- Add preuninstall hook for best-effort daemon cleanup on npm uninstall
- Update docs to reflect persistent daemon model
* fix: remove stale idle timeout references from code and docs
* refactor: remove daemon status/restart commands and lastCliRequestTime
- Remove `daemon status` and `daemon restart` CLI commands (doctor covers diagnostics)
- Remove `lastCliRequestTime` tracking (no longer needed without idle timeout)
- Keep only `daemon stop` as the explicit shutdown command
* Add AbortSignal.timeout(3s) to preuninstall shutdown fetch
Prevents npm uninstall from hanging if the daemon port accepts
connections but never responds.
* refactor: unify browser error classification and deduplicate retry logic
Replace two overlapping error classification systems with a single
classifyBrowserError() that returns retry advice (retryable + delayMs):
- Extension/daemon transient errors → retryable, 1500ms delay
- CDP target navigation errors → retryable, 200ms delay
- Non-transient errors → not retryable
Deduplicate sendCommand/sendCommandFull retry loop into sendCommandRaw,
making both public functions thin return-value wrappers.
* fix: add error kind to prevent page-level retry of extension errors
classifyBrowserError() now returns a `kind` field:
- extension-transient: retried by daemon-client only
- target-navigation: retried by page-level settle logic
- non-retryable: no retry
Page.goto() and Page.evaluate() now only settle-retry on
target-navigation, preventing extension/daemon errors from being
silently swallowed as settle noise.
Use Chrome CDP targetId (UUID) as the canonical page identity across
all layers (extension → daemon → CLI), demoting tabId to an
extension-internal routing detail.
- Add extension/src/identity.ts: bidirectional targetId ↔ tabId mapping
with lazy refresh via chrome.debugger.getTargets()
- Update protocol: Command.page and Result.page carry targetId
- Update background.ts: resolveCommandTabId() and pageScopedResult()
helpers; all page-scoped handlers return targetId
- Add sendCommandFull() to daemon-client for responses with page identity
- Update Page class: _page stores targetId, goto/selectTab extract it
- Update record.ts: injectedPages tracks by targetId
- Add extension tests to vitest config and CI test scripts
Synced all desktop adapter command lists in desktop.md with
the actual `opencli <adapter> --help` output:
- cursor: remove non-existent status/new/dump/screenshot; add composer
- codex: remove non-existent status/new/dump/screenshot
- chatgpt: add missing model command
- chatwise: remove non-existent new/screenshot
- notion: update descriptions to match help text
- discord-app: update descriptions to match help text
- doubao-app: reorder to match help output
- antigravity: remove non-existent ask; add serve/status
Also moved `status` to the top of each adapter section where it exists.
* perf: fast-path completion, version, and shell scripts to bypass full discovery
Lightweight commands (--get-completions, --version, completion <shell>) now
resolve before any heavy module loading. Key changes:
- New completion-fast.ts: manifest-based completion + shell script generators
with zero dependency on registry/discovery/cli modules
- main.ts: static imports replaced with dynamic import() for the full startup
path so the fast path never pays the cost of loading discovery, registry,
Commander, hooks, etc.
- USER_CLIS_DIR inlined to avoid importing the entire discovery module
- completion.ts: removed manifest functions (moved to completion-fast.ts),
now only used as fallback when manifest is unavailable
* fix: address review blockers from codex-mini0
1. --version fast path: only match when argv[0] is --version/-V,
not anywhere in argv. Prevents intercepting `opencli gh --version`
which should pass through to the subcommand.
2. Completion fast path: require ALL manifests to exist (hasAllManifests),
not just one. If user clis dir exists but has no manifest, fall back
to full discovery so user adapters aren't silently dropped.
If user clis dir doesn't exist at all, skip its manifest requirement
since there are no user adapters to miss.
EMPTY_RESULT and structurally-valid SELECTOR failures are often not
adapter bugs — they're the platform shaping results under anti-scrape,
or a soft 404, or a legitimately empty search. Patching a working
adapter to chase a zero-result query breaks the next working path.
Add a pre-check section at the top of opencli-autofix listing four
rule-outs that must fail before a repair round is justified:
1. Retry with an alternative query / entry point
2. Spot-check the page in a normal Chrome tab
3. Look for soft 404s (200 with empty payload)
4. Remember that "0 results" from a search is a valid answer
Placed directly before "Step 1: Collect Diagnostic Context" so the
check runs at exactly the moment the agent would otherwise commit to
a repair round.
Audience/timing is the whole point: the skill is loaded precisely when
an error has occurred and the agent is deciding whether to repair, and
the pre-check intercepts that decision before it locks in.
11 lines of markdown, zero code, single file.
---
Inspired by https://github.com/eze-is/web-access by 一泽 Eze (MIT),
specifically the "平台返回的'内容不存在'不一定反映真实状态" mental model
from its SKILL.md. Adapted into concrete, actionable checks for
opencli's EMPTY_RESULT classification.
Note: an earlier version of this PR also added a tool-selection
decision table and a subagent-verb rule to opencli-usage. Both were
removed after review because opencli-usage only loads *after* an agent
has committed to using opencli — advice placed there arrives too late
to change tool selection, and is not seen by the main agent at
delegation time. The insights are still valid; they just don't have a
load-time match in the current skill system. This PR keeps only the
change where audience and timing actually line up.
* refactor(skills): unify command reference by site instead of technology
- Merge Browser-based and Public API sections into single alphabetical
table with type emoji tags (🌐/✅/🖥️/🔧)
- Delete browser.md and public-api.md (replaced by unified SKILL.md table)
- Add GitHub/DevOps and collaboration rows to capability lookup
- Remove stale File column from capability table
* feat(skills): add commands.md with merged examples + 8 missing adapters
- Create commands.md: merge browser.md + public-api.md into single
alphabetical-by-site reference with detailed usage examples
- Add 8 missing adapters: 1688, hupu, jianyu, lesswrong, quark,
xianyu, xiaoe, yuanbao
- Bump skill version 1.6.3 → 1.6.9 to match package.json
- Add dedicated External CLI section listing all 7 registered CLIs
(gh, obsidian, docker, lark-cli, dws, wecom-cli, vercel)
- Include install/register commands so AI agents know how to manage them
- Move gh from Desktop to External CLI section
- Update desktop.md to remove gh and reference External CLI section
* refactor(skills): merge opencli-generate into opencli-explorer
opencli-generate was a thin wrapper over generateVerifiedFromUrl,
essentially an internal pipeline orchestration. Merge its entry point
into opencli-explorer as the automated fast path, keeping one unified
skill for adapter creation.
- Delete skills/opencli-generate/SKILL.md
- Add automated generation tip to opencli-explorer SKILL.md
- Update README/README.zh-CN skill references
- Update skill-generate.ts comment
* fix(docs): fix dead link in yaml-adapter deprecation page
Change ../../CONTRIBUTING.md to ./contributing (VitePress internal link).
* refactor: remove version field from GenerateOutcome and EarlyHint
All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.
Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).
* refactor: migrate all 123 CLI adapters from YAML to TypeScript
Remove YAML as an adapter format entirely. All adapters now use
TypeScript with cli() from @jackwener/opencli/registry.
- Convert 123 YAML adapter files to TypeScript via batch script
- Remove YAML scanning from discovery.ts (registerYamlCli, yaml import)
- Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts
- Change synthesize.ts to output JSON candidates (internal format)
- Change generate-verified.ts to write .ts adapter files instead of .yaml
- Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool)
- Update all tests to match new format
Closes discussion in #OpenCLI thread 47ddba82.
* fix: close YAML migration gaps in plugin scaffold, validation, and scan
- plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml
- plugin.ts validatePluginStructure: no longer accept .yaml as valid command file
- plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions
- discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/
- plugin.test.ts: update all test fixtures from .yaml to .js
- plugin-scaffold.test.ts: update hello.yaml references to hello.ts
- Delete dead src/yaml-schema.ts
Resolves PR #887 review blockers from @mbp-codex-pr0.
* refactor: complete YAML removal across docs, skills, record, and binance adapters
Code changes:
- record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import
- src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format
- binance/commands.test.ts: rewrite to use registry instead of yaml.load
- skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts
- build-manifest.ts, synthesize.ts: update stale YAML comments
Documentation:
- README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment
- README.zh-CN.md: fix synthesize comment
- CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples
- docs/developer/yaml-adapter.md: replaced with deprecation redirect
- docs/developer/architecture.md: remove YAML pipeline references
- docs/developer/contributing.md: remove YAML adapter section
- docs/developer/ai-workflow.md: YAML → TS in synthesize description
- docs/guide/getting-started.md: remove .yaml from loader, update engine description
- docs/guide/plugins.md: remove YAML plugin option, update plugin types
- docs/index.md, docs/comparison.md: remove YAML adapter references
- docs/zh/guide/plugins.md: remove .yaml from scan description
Skills:
- opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only
- opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates
- opencli-generate/SKILL.md: YAML artifact path → TS artifact path
- opencli-usage/SKILL.md, plugins.md: update adapter format references
* fix: clean up remaining YAML adapter references in docs
- docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline
- docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line
- TESTING.md: same fix in root testing doc
- CONTRIBUTING.md: remove "YAML validation" comment
- docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav
- docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading
All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.
Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).
* fix: use Strategy.PUBLIC enum in skill-generate test to fix typecheck regression
* feat: add P2 EarlyHint callback channel to generateVerifiedFromUrl
Add optional onEarlyHint callback for internal cost gating before verify stage.
- EarlyHint type: version, stage, continue, reason, confidence, candidate?
- 3 emit points: explore (viable/not), synthesize (candidate/not), cascade (auth/ok)
- candidate only on synthesize/cascade + continue:true (not on stop or explore)
- unsupported-required-args goes directly to P1 terminal, no P2 hint emitted
- 6 new tests covering all hint paths + guardrails
* docs: add opencli-generate skill spec (SKILL.md)
Captures A+B consensus from team discussion:
- Input: url + goal? (natural language intent hint)
- Output: SkillOutput with machine-readable fields + human message
- Decision tree: thin mapping from GenerateOutcome
- Guardrails: no re-orchestration, no auto-escalation, no new taxonomy
- P1/P2 boundary: P1 is single source of truth, P2 transparent to skill
* fix: address review nits on skill spec
- Make path explicitly optional in needs-human-check decision tree
- Add missing non-array-result message template
* feat: add GenerateOutcome → SkillOutput thin wrapper
Implements the skill mapping layer per opencli-generate SKILL.md:
- mapOutcomeToSkillOutput: thin translation from P1 contract to agent-facing output
- executeGenerateSkill: entry point accepting SkillInput (url + goal?)
- Message templates for all StopReason and EscalationReason values
- 8 tests covering all outcome paths and contract shape validation
* fix: prefer outcome.message for richer context in needs-human-check
When GenerateOutcome has a message (e.g. "required args: id"), use it
instead of the generic template, so the specific args info reaches the user.
All 68 browser adapter links and 8 desktop adapter links in
docs/adapters/index.md were missing the .md file extension,
causing broken links when navigating the documentation on GitHub.
- Add getDaemonHealth() returning 'stopped' | 'no-extension' | 'ready'
- Delete discover.ts (thin wrapper with no value)
- Bridge uses getDaemonHealth() + _pollUntilReady() (eliminates duplicate polling)
- Doctor simplified: live check auto-starts daemon; no-live mode does minimal
auto-start only when stopped (avoids misreporting idle-exit as failure)
- CommanderAdapter preserves error message/hint detail (not just generic title)
- All callers use single unified status entry point
* fix(xiaohongshu): scope note interaction selectors to .interact-container
The .like-wrapper / .collect-wrapper / .chat-wrapper class names are
also used by every comment's like/reply buttons in the comment section.
querySelector returned the FIRST match — which on a note with comments
is a comment's count, not the post's. As a result, `xiaohongshu note`
returned wrong like/collect/comment counts for any note that had user
comments.
Scoping each selector to .interact-container (the post's main
interaction bar) returns the correct post-level counts.
Verified on multiple notes:
- Note A: was returning likes=2, now correctly returns likes=74
- Note B: was returning likes=1, now correctly returns likes=796
- Note C: was returning likes=1, now correctly returns likes=269
* test(xiaohongshu): add regression check for .interact-container selector scope
Verify the evaluate script passes scoped selectors so unscoped
versions can't silently regress. Follows reviewer suggestion to
assert on page.evaluate.mock.calls[0][0].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: add GitHub Trending, Binance, and Weather (Open Meteo) adapters
GitHub Trending (2 commands, browser mode):
- repos: trending repositories with stars, forks, language filter
- developers: trending developers with popular repos
Supports --since daily/weekly/monthly and --language filter
Binance (11 commands, public API via data-api.binance.vision):
- top: top trading pairs by 24h volume
- price: single pair 24h price stats
- prices: latest prices for all pairs
- ticker: 24h ticker statistics
- gainers: top gaining pairs by 24h change
- losers: top losing pairs by 24h change
- trades: recent trades for a pair
- depth: order book bid prices
- asks: order book ask prices
- klines: candlestick/kline data
- pairs: list active trading pairs
Weather / Open Meteo (11 commands, free public API, no key needed):
- current: current weather for a city
- forecast: daily forecast up to 16 days
- hourly: hourly forecast
- search: city geocoding lookup
- air: air quality index (simple)
- air-quality: detailed air quality (US/EU AQI, PM2.5, PM10, ozone, NO2, SO2)
- sunrise: sunrise/sunset times with UV index
- wind: detailed wind forecast with gusts and 80m altitude
- precipitation: rain/snow forecast with probability
- history: historical weather up to 92 past days
- compare: side-by-side weather comparison across cities
All 24 commands tested with live data. 258 existing tests pass.
* docs: add missing douban, sinablog, substack adapter documentation
* fix(binance): sort numeric metrics and filter active pairs
* chore: drop non-binance adapters from pr214
* chore: drop binance docs from pr214
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add Self-Repair protocol for automatic adapter fixing
When an AI agent uses opencli and a command fails, the agent
automatically diagnoses the failure, fixes the adapter, and retries.
- Add CLAUDE.md with Self-Repair protocol (auto-loaded by Claude Code)
- Add designs/self-repair-protocol.md documenting the approach
- Update opencli-repair skill: add Safety Boundaries (AUTH/BROWSER → STOP,
sourcePath-only scope, max 3 rounds), fix AUTH_REQUIRED guidance
- Update opencli-usage skill: add Self-Repair section
Key design decisions:
- Repair target is always RepairContext.adapter.sourcePath (works for both
repo-local clis/ and user-local ~/.opencli/clis/)
- Only adapter files may be modified, never core src/
- Max 3 repair rounds per failure
- AUTH_REQUIRED and BROWSER_CONNECT are hard stops (report, don't modify)
* fix: align auth boundary and scope language across all documents
- Remove "Auth changed (AUTH_REQUIRED)" exploration section from
opencli-repair skill — contradicted the hard stop rule above it
- Update design doc: scope language matches repo-local + explicit skill
delivery model, not universal product behavior
- Update usage skill: reference sourcePath instead of "files under clis/"
* fix: replace remaining repo-relative clis/ paths with sourcePath in design doc
* refactor: rename opencli-repair to opencli-autofix, remove CLAUDE.md
CLAUDE.md was wrong — users don't work inside the opencli repo, and
the protocol shouldn't assume Claude Code. The skill is the portable
delivery mechanism for any AI agent.
- Rename skills/opencli-repair → skills/opencli-autofix
- Remove CLAUDE.md (not the right delivery mechanism)
- Update all references in usage skill and design doc
- Design doc rewritten to reflect skill-first approach
* fix: use sourcePath in example repair session
* feat: emit AutoFix hint on repairable adapter errors
When a command fails with a repairable error (SELECTOR, EMPTY_RESULT,
COMMAND_EXEC, or generic http/not-found), the error output now includes
a hint telling agents to re-run with OPENCLI_DIAGNOSTIC=1 for repair
context. This is the trigger mechanism that bridges the gap between
"command failed" and "agent enters autofix loop".
Non-repairable errors (AUTH_REQUIRED, BROWSER_CONNECT, ARGUMENT) do not
emit the hint — these require user action, not adapter fixes.
* fix: narrow AutoFix hint to adapter-drift errors only
Remove hint from CommandExecutionError (covers env/launcher/runtime
issues, not adapter drift) and generic http errors (often temporary
site issues). Keep hint only for SelectorError, EmptyResultError,
and generic not-found — clear adapter-drift signals.
When the Browser Bridge extension is older than the CLI, sending
'network-capture-start' to the daemon returns 'Unknown action',
causing explore and operate-open to crash with an unhandled error.
Wrap startNetworkCapture calls with .catch() so they degrade
gracefully — explore continues without network capture data, and
operate-open falls back to the JS interceptor injection.
The compose page needs to load Draft.js editor which is heavier than
primaryColumn. 8s is too tight for slow networks and will cause flaky
failures. 15s aligns with the file input timeout (20s) in magnitude.
- engine.ts: replace `git add -A` with scope-aware `execFileSync` to
stage only files matching config.scope globs, and guard against empty
scope degenerating into staging all files
- fix.ts: pass prompt via stdin `input` option instead of shell string
interpolation to prevent $, backtick, and other metacharacter expansion
- generate.ts: update stale comment that claimed unimplemented pipeline
steps (register, verify, Strategy Cascade)
* refactor: remove scoring heuristic, replace with noise filter + metadata
The scoring mechanism was a pre-LLM heuristic that compressed rich endpoint
metadata into a single number. Since this project is designed for AI Agents,
the agent can reason about structured metadata directly.
Changes:
- Remove scoreEndpoint/scoreRequest/scoreWriteRequest and all score fields
- Replace with isNoiseUrl() filter (tracking/beacon/pixel) + isUsefulEndpoint()
- Remove artificial confidence percentages (was score/20)
- Sort by itemCount (transparent, observable) instead of weighted score
- Endpoints now expose full structured metadata for agent consumption
- Net reduction: -43 lines
* fix: widen endpoint filter to keep single-object JSON and stats/metric URLs
- Remove stats/metric from noise pattern — these are often business APIs
- Relax isUsefulEndpoint to keep any JSON endpoint, not just arrays
(preserves /me, /profile, /detail and other single-object APIs)
* fix: add deterministic endpoint ordering for generate/synthesize path
The AI agent path doesn't need ranking, but generate/synthesize still
pick candidates[0] as default — this needs a stable, explainable order.
- Add endpointSortKey() with transparent observable signals: array items,
detected fields, API path patterns, query params
- Update synthesize chooseEndpoint fallback to use itemCount + field count
- Sort key is internal only; not exposed as score to external consumers
* feat(linux-do): split topic content into a dedicated command
Move the old main-post path out of linux-do topic so topic stays a summarized first-page reader while topic-content becomes the Markdown-focused entrypoint for full post bodies.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* feat(linux-do): update topic content handling to include YAML front matter
* fix(linux-do): update default output format to plain for topic-content rendering
* fix(linux-do): replace js-yaml with inline YAML serialization for topic-content
Adapters must only import node builtins, relative modules, or opencli
public APIs. Hand-roll the simple front matter serialization to remove
the third-party js-yaml dependency.
* fix(linux-do): refine YAML quoting to only escape colons followed by space
Colons in URLs (e.g. https://) are valid unquoted YAML values. Only
quote when a colon is followed by a space or appears at end of line.
---------
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
* refactor: extract shared scoring logic and consolidate time format utils
- Extract applyUrlScoreAdjustments() and scoreArrayResponse() to analysis.ts,
eliminating duplicated endpoint scoring between explore.ts and record.ts
- Consolidate formatDuration/formatUptime into a single formatDuration(ms)
in download/progress.ts, reused by commands/daemon.ts
* fix: preserve explore scoring semantics and round daemon uptime
- Revert explore.ts scoreEndpoint to original inline /api/ /x/ bonus
without record's tracking/analytics penalty (blocker from review)
- Math.round uptime*1000 to avoid floating-point noise in daemon status
* feat(xueqiu): add kline and groups adapters
Add kline.yaml: fetch candlestick/OHLCV data from Xueqiu v5 chart API.
Supports custom days lookback and outputs date, open, high, low, close,
volume, percent.
Add groups.yaml: list Xueqiu portfolio/group entries.
* fix(xueqiu): correct groups.yaml to use /portfolio/list.json API
The previous implementation used /portfolio/stock/list.json which only
returns stocks in a single group and does not return the group list.
Switch to /portfolio/list.json which returns all portfolio groups
including 实盘, 沪深, 港股, 美股, 模拟(pid=-4), 持仓 etc.
* fix(xueqiu): replace watchlist category param with pid selector
- Remove the unused 'category' parameter (the API ignores it;
all groups live under category=1 regardless)
- Replace with 'pid' parameter to allow fetching any group:
-4=simulated, -5=SH/SZ, -6=US stocks, -7=HK stocks, etc.
- API path still uses category=1 but pid is now user-controllable
---------
* feat(operate): unify network capture + implement CDP consoleMessages
- operate open: start session capture before navigation (catches initial requests)
- operate network: prefer readNetworkCapture() over JS interceptor
- CDPPage: implement consoleMessages() via Runtime.consoleAPICalled
Part of #810
* fix(operate): use correct daemon/CDP entry field names for network capture
Daemon and CDP capture entries use responseStatus/responseContentType/
responsePreview (not status/contentType/responseBody). Fix the
normalization in operate network to match the actual entry shape from
extension/src/cdp.ts.
* fix(cdp): capture Runtime.exceptionThrown in consoleMessages
- Register Runtime.exceptionThrown handler to capture uncaught exceptions
as error-level messages (most valuable diagnostic signal)
- 'error' filter now returns both console.error() and warning/exception
entries, matching typical severity-based logging semantics
* feat(cdp): implement session-level network capture for CDPPage
Implements startNetworkCapture() and readNetworkCapture() on CDPPage using
CDP Network domain events. Updates explore.ts to prefer session capture
over Performance API networkRequests().
Closes part of #810
* fix(cdp): use Network.loadingFinished for reliable body capture
- Move getResponseBody call from responseReceived to loadingFinished,
matching the extension's implementation pattern
- Use extension-compatible entry shape (responseStatus, responseContentType,
responsePreview) instead of custom field names
- Remove unreliable 100ms sleep hack in readNetworkCapture()
- Align with extension/src/cdp.ts:419-437 for consistency
* fix(cdp): drain buffer on readNetworkCapture to match daemon contract
readNetworkCapture() must clear the buffer after reading, matching the
daemon Page's read-and-drain behavior. Without this, repeated reads
would return stale entries.
* fix(cdp): await in-flight body fetches before returning from readNetworkCapture
Track all pending getResponseBody promises and await them in
readNetworkCapture() before draining the buffer. This ensures
explore/diagnostic consumers always get entries with responsePreview
populated, not empty shells where the body fetch hasn't resolved yet.
* fix(explore): handle both legacy and capture entry field names
parseNetworkRequests now maps both shapes:
- Legacy: status, contentType, responseBody
- Capture (extension/CDP): responseStatus, responseContentType, responsePreview
Also clears _pendingBodyFetches on startNetworkCapture reset.
* fix: add safety boundaries to diagnostic output
- Redact sensitive headers (Authorization, Cookie, etc.) from network requests
- Redact sensitive URL query parameters (token, key, secret, etc.)
- Cap individual fields: snapshot (100K chars), adapter source (50K chars),
network requests (50 entries, 4K body each), stack trace (5K chars)
- Enforce 256KB total output budget with graceful degradation:
drops snapshot first, then page state entirely
- Export truncate/redactUrl helpers for testing
* fix: add free-text redaction for all diagnostic string channels
Addresses review feedback: snapshot, consoleErrors, error message/hint/stack
could contain inline secrets (Bearer tokens, JWTs, cookie values, token=value
patterns). All string channels now pass through redactText() before emission.
- Add redactText() with patterns for Bearer tokens, JWTs, cookie values,
and inline key=value secrets
- Apply redactText to: error.message, error.hint, error.stack,
page.snapshot, page.consoleErrors
- Add 6 new test cases for redactText and error message redaction
* fix: resolve adapter source path and add page state collection timeout
Fixes#808 items 1 and 3:
1. adapter.source was missing for all command types because buildRepairContext
only checked cmd._modulePath (set only for manifest lazy-loaded TS).
Now resolveAdapterSourcePath() checks cmd.source first, skips manifest:
pseudo-paths, and maps dist/clis/*.js back to source clis/*.ts.
3. collectPageState() had no timeout — a hung CDP connection would block
error propagation indefinitely. Now wrapped with 5s Promise.race timeout,
falling back to emitting diagnostic without page state.
* fix: track sourceFile in manifest for YAML adapter source resolution
YAML commands inlined in the manifest previously lost their original file
path, causing resolveAdapterSourcePath() to return undefined. Add
sourceFile field to ManifestEntry so discovery can reconstruct the
editable source path for both YAML and TS commands.
* feat: add structured diagnostic output for AI-driven adapter repair
When OPENCLI_DIAGNOSTIC=1 is set, failed commands emit a RepairContext
JSON to stderr containing the error, adapter source, and browser state
(DOM snapshot, network requests, console errors). AI Agents consume
this to diagnose and fix adapters when websites change.
Also adds the opencli-repair skill guide for AI Agents.
* fix: correct e2e test binary path to dist/src/main.js
The e2e helpers pointed to dist/main.js but the actual build output
is at dist/src/main.js (matching package.json "main" field). This
caused all e2e-headed tests to fail with "Cannot find module".
* fix: correct dist/main.js path in autoresearch scripts
* fix: emit diagnostic for pre-session browser failures
When browser connection fails before the session callback runs
(e.g., BrowserConnectError), the inner diagnostic catch never fires.
Use a flag to ensure the outer catch emits diagnostic as a fallback.
* test: tolerate unavailable Bloomberg RSS feeds in e2e
* test: skip flaky bloomberg businessweek e2e test
The Bloomberg Businessweek RSS feed is intermittently unavailable,
causing CI failures unrelated to code changes.
* revert: restore bloomberg businessweek e2e coverage
* fix: avoid inserting completion config inside multi-line shell commands
The postinstall zshrc insertion logic splits backslash-continued blocks
(e.g. zinit stanzas) when it finds a compinit match inside them, which
breaks the user's shell config. Walk backward past continuation lines
so the insertion lands before the entire logical command.
* fix: append zsh completion to end of .zshrc instead of splicing
Replace the fragile compinit-searching splice logic with a simple
append, matching the strategy already used for bash. This avoids
breaking multi-line commands (e.g. zinit blocks with zicompinit).
Still detects existing compinit to avoid adding a duplicate call.
* fix: stop modifying shell rc files in postinstall
Replace the fragile .zshrc/.bashrc modification logic with a safer
approach: only write completion files and print setup instructions.
The previous approach tried to parse and splice into rc files, which
broke multi-line shell commands (e.g. zinit blocks with backslash
continuations matching /compinit/). Instead of attempting to fix the
parser, remove rc modification entirely — this matches the approach
used by rustup, homebrew, and other CLI tools.
Closes#788
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(discovery): expose runtime deps to user adapters
* fix: route copied adapters through opencli exports
* refactor: route adapter status output through logger
* refactor: migrate adapter imports to package exports
Replace all relative imports (../../src/registry.js, ../../browser/cdp.js, etc.)
with package exports (@jackwener/opencli/registry, @jackwener/opencli/errors, etc.)
across all 484 adapter files.
This decouples adapter import resolution from directory structure:
- User CLIs in ~/.opencli/clis/ resolve via node_modules symlink
- Internal adapters resolve via Node.js self-referencing
- No more shim files needed for import resolution
Changes:
- package.json: add sub-path exports for all public modules
- clis/**: replace relative imports with @jackwener/opencli/...
- discovery.ts: simplify ensureUserCliCompatShims to symlink-only
- registry-api.ts: export CommandArgs type
- Remove root-level shim directories (browser/, download/, pipeline/)
- Remove shim entries from tsconfig.json include and package.json files
* test: add regression tests for package exports
Prevents regressions like #788/#791 by:
1. Scanning all adapter files for forbidden relative imports
(../../src/, ../../browser/, etc.) — fails if any remain
2. Verifying every package.json export maps to an existing source file
18 new test cases.
* fix: use junction on Windows + broaden test patterns
- discovery.ts: use 'junction' symlink type on Windows (no admin required)
- package-exports.test.ts: generalize forbidden patterns to catch any
depth of ../ traversal (not just ../../ and ../../../)
* fix: update stale vi.mock/importActual paths in adapter tests
Test files still used old relative paths for vi.mock() and
vi.importActual() calls. Updated 5 test files to use package exports.
Also broadened regression test patterns to catch mock/importActual paths.
* fix: use rm instead of unlink for symlink cleanup, add warn on failure
Addresses review feedback from Astro-Han:
- rm() handles both symlinks and stale directories (unlink fails on dirs)
- Log a warning when symlink creation fails instead of silent catch
* docs: update import examples to use package exports
Update all documentation, contributing guides, and skills to use
@jackwener/opencli/registry instead of ../../src/registry.js.
Without this, users following the docs would write adapters with
broken imports since the old shim files are no longer created.
Bug 1: version.ts used a single-level parent lookup for package.json,
which broke after #784 changed rootDir from "src" to "." (version.js
now lives in dist/src/ instead of dist/). Walk up until package.json
is found — works in both dev (src/) and prod (dist/src/).
Bug 2: adapters copied to ~/.opencli/clis/ import ../../src/registry.js
etc., which resolves to ~/.opencli/src/. Derive src/ compat shims from
the existing rootShims list so these imports resolve correctly.
- Replace 140 instances of `src/clis/` → `clis/` across 12 doc files
(path changed after repo restructure)
- Remove non-command `rpc` and `rankings` from notebooklm/amazon
command lists in README, README.zh-CN, SKILL.md, and adapters index
(these are internal utility modules, not user-facing commands)
- Add `deep-research` and `deep-research-result` to gemini adapter doc
- Add `movers-shakers` and `new-releases` to amazon adapter doc
- Update notebooklm doc examples to use canonical commands instead of
deprecated aliases (`metadata` → `get`, `notes-list` → `note-list`)
- Bump SKILL.md version to 1.6.3
* docs: fix outdated commands and adapter counts in README and skills
- Add gemini deep-research and deep-research-result commands
- Fix notebooklm: remove non-existent select/metadata/notes-list, add rpc
- Add amazon movers-shakers, new-releases, rankings commands
- Add missing weibo commands in zh-CN README
- Fix linux-do missing hot/latest/category in zh-CN README
- Update adapter count from 73+ to 79+
- Update skills version to 1.6.2
- Add full spotify command list in skills SKILL.md
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): update xiaohongshu note tests for search_result URL change
buildNoteUrl now uses /search_result/<id> instead of /explore/<id> for
bare note IDs. Update test expectations to match:
- buildNoteUrl test: expect /search_result/ not /explore/
- goto URL assertion: expect /search_result/ not /explore/
- empty shell hint: match actual error message text
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(test): update xiaohongshu comments test for search_result URL change
- Bare note ID now navigates to /search_result/ not /explore/
- Full URL inputs are preserved as-is (including /explore/ URLs)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: simplify core modules — remove root shims, consolidate error classification, streamline cascade/interceptor, clean up synthesize
1. Remove root-level shim files (errors.ts, logger.ts, registry.ts, types.ts, utils.ts, launcher.ts) — update all ~840 adapter imports to reference src/ directly
2. Consolidate interceptor: reuse shared DISGUISE_FN in tap interceptor instead of reimplementing
3. Unify error classification: single ClassifiedError type with icon/exitCode/hint lookup table, eliminating duplicated pattern matching between resolveExitCode and renderError
4. Simplify cascade probe: replace repetitive switch cases with PROBE_OPTIONS lookup map
12. Clean up synthesize.ts: remove deprecated snake_case field aliases (recommended_args, recommended_columns, recommendedColumnsLegacy) and unnecessary constant aliases
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update import paths in contributor docs and skill templates
Update all documentation and skill files to reference src/ directly,
matching the shim removal in the previous commit.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update new Gemini adapter imports to use src/ paths
Fix imports in newly added deep-research adapter files that were
still referencing the deleted root shim files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update xiaohongshu tests for /search_result/ URL change
Tests now expect /search_result/<id> for bare note IDs (matching
the note-helpers.ts change from PR #774) and updated empty-shell
hint assertion.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: update LessWrong and hupu adapter imports to use src/ paths
Fix imports in newly merged LessWrong and hupu/mentions adapter
files that were still referencing the deleted root shim files.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(xueqiu): mock logger via src path
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat(hupu): add hupu cli adapter
* fix(hupu): prevent detail from returning the wrong thread
* refactor: deduplicate shared utilities in hupu adapter
- Merge postHupuJson and postHupuReplyJson into single function with mode parameter
- Move stripHtml and decodeHtmlEntities to utils.ts, remove duplicate definitions
* fix(hupu): add mentions command
* fix: move mentions.ts to clis/hupu/, remove src/clis/hupu duplicates
Post PR #782 restructure: adapter files live at root clis/, not src/clis/.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(lesswrong): add LessWrong adapter
15 commands for the LessWrong rationality and AI alignment community:
- Post listings: curated, frontpage, new, top, top-week/month/year
- Content: read (full post), comments, shortform (quick takes)
- Discovery: tag, tags, sequences
- Users: user (profile), user-posts
All commands use the public GraphQL API (no browser required).
Time-filtered views use the `after` date parameter.
Tag lookup resolves slugs to IDs via the `tagBySlug` view.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move lesswrong adapter to clis/ (post PR #782 restructure)
New adapter files were created at src/clis/lesswrong/ but PR #782 moved
all adapters to root clis/. Move to correct location.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
XHS blocks /explore/<id> without a valid xsec_token, causing code 66
(empty result) when passing bare note IDs. The /search_result/<id> path
works without xsec_token when the user is logged in via cookies.
Changes:
- note-helpers.ts: buildNoteUrl now uses /search_result/<id> for bare IDs
- note.ts: remove isBareNoteId branching and simplify empty shell error
* feat(gemini): add deep-research workflow and docs export result
* fix(gemini): improve deep-research submit and confirm flow
* fix(gemini): return waiting state when deep research is in progress
* fix(gemini): avoid false submit detection on root app transcript changes
* fix(gemini): return pending state when deep-research export is not ready
---------
Co-authored-by: f1480022 <f148002@163.com>
* fix(36kr): replace waitForCapture with DOM polling for search/hot
waitForCapture(6) always times out on 36kr because the API intercept
never captures a matching request. However, the DOM is already fully
rendered with search/hot results by the time the timeout fires.
Replace the 6-second intercept wait with a DOM polling loop that checks
for article links (a[href*="/p/"]) every 300ms, returning immediately
once content is available (typically ~1s vs 6s timeout + error).
Tested on opencli 1.6.2 with both CDP and Browser Bridge modes.
* fix: rebase onto main, remove unused interceptor, fix strategy
- Rebase onto main after clis/ move (PR #782)
- Remove installInterceptor calls (no longer used after waitForCapture removal)
- Change strategy from INTERCEPT to PUBLIC (browser: true) to match actual behavior
- Improve polling loop readability
---------
Co-authored-by: buruguo <buruguo@lambdafintech.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix: review follow-ups — better first-run log, OPENCLI_FETCH=1 skips version check
- Clarify first-run log message: "copying adapters (one-time setup)"
- Add comment explaining why scriptPath uses two levels of ../
- OPENCLI_FETCH=1 now bypasses version-skip to allow forced refresh
* fix: update doc-coverage script path after clis/ move
check-doc-coverage.sh still referenced src/clis/ after PR #782 moved
adapters to root clis/. This caused CI to fail with "0/1 documented".
* fix: resolve package root dynamically for symlink and first-run paths
The symlink at ~/.opencli/node_modules/@jackwener/opencli pointed to
dist/ instead of the package root in prod mode, breaking user TS CLIs
that import from '@jackwener/opencli/registry'.
The first-run scriptPath also resolved incorrectly in dev mode.
Extract findPackageRoot() that walks up to find package.json, fixing
both paths for dev (src/) and prod (dist/src/) layouts.
* refactor: move adapters from src/clis/ to root clis/ for monorepo separation
Separates CLI adapters from the core runtime to prepare for independent
adapter distribution via postinstall fetch.
Key changes:
- Move src/clis/ → clis/ (adapters at repo root)
- Change tsconfig rootDir from "src" to "." so tsc compiles both
- Create root-level shim files (registry.ts, errors.ts, etc.) so adapter
relative imports (../../registry.js) resolve correctly
- Update build-manifest.ts, main.ts paths for new dist/src/ structure
- Expand ensureUserCliCompatShims() to cover all adapter import targets
(types, utils, logger, launcher, browser/*, download/*, pipeline/*)
- Add scripts/fetch-adapters.js postinstall for ~/.opencli/clis/ sync
- Update vitest.config.ts adapter test paths
- Add package.json files field to exclude adapters from npm package
Official adapter files are unconditionally overwritten on update;
user-created files not in the manifest are preserved.
* fix: add dist/clis/ and cli-manifest.json to npm files, harden fetch-adapters
- Add dist/clis/ and dist/cli-manifest.json to package.json files field
so built-in adapters and manifest ship with the npm package
- Replace execSync with execFileSync to prevent command injection
- Add version check to skip redundant adapter fetches
- Track tmpRoot explicitly for reliable cleanup
* fix: address review blockers — manifest-based updates, global-only fetch, first-run fallback
1. Manifest-based update strategy:
- Read old manifest to identify previously-official files
- Clean up files removed upstream (in old manifest but not new)
- User-created files (never in any manifest) remain untouched
2. Only run fetch-adapters on global install (npm_config_global=true)
or explicit OPENCLI_FETCH=1, preventing heavy side effects for
local/dev installs
3. First-run fallback in discovery.ts:
- ensureUserAdapters() checks for adapter-manifest.json
- If missing and ~/.opencli/clis/ is empty, spawns fetch-adapters.js
- Guarantees adapters are available even with --ignore-scripts
* fix: remove OPENCLI_FETCH env var, use internal _OPENCLI_FIRST_RUN instead
* feat: also support OPENCLI_FETCH=1 for explicit adapter fetch trigger
* simplify: replace git clone with local copy from dist/clis/
Adapters already ship in the npm package (dist/clis/), so there's no
need to clone from GitHub. Copy directly from the installed package:
- Eliminates git, curl, tar dependencies
- No network calls in postinstall
- No timeout/offline issues
- Version always matches the installed CLI
- ~65 lines of clone/download code replaced by one cpSync loop
* feat(hupu): add hupu cli adapter
* fix(hupu): prevent detail from returning the wrong thread
* refactor: deduplicate shared utilities in hupu adapter
- Merge postHupuJson and postHupuReplyJson into single function with mode parameter
- Move stripHtml and decodeHtmlEntities to utils.ts, remove duplicate definitions
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: 推特新增回复图片能力支持本地路径和网络路径
* fix(twitter/reply): fix image upload fallback, restore execCommand, add size limit
- Fix attachReplyImage fallback: use uploaded flag instead of checking
page.setFileInput existence, so base64 fallback actually runs when
CDP setFileInput throws "Unknown action"
- Restore execCommand('insertText') as primary text input method for
Twitter's Draft.js editor, with paste event as fallback
- Add 20MB size limit for remote image downloads to prevent OOM
- Remove unsafe buttons[0] fallback that could click invisible buttons
* fix(twitter/reply): add local image size check and base64 fallback warning
Local images were not validated for size — a 100MB file would fail only
at upload time. Remote images already had MAX_IMAGE_SIZE_BYTES checks.
Also add a console.warn when using the base64 fallback with large
payloads, consistent with xiaohongshu/publish.ts behavior.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(xiaoe): add 小鹅通 (Xiaoe-tech) student platform adapter
Add 5 YAML adapters for 小鹅通 (xiaoe-tech.com), the leading Chinese
online education platform:
- courses: list purchased courses with URLs and shop names
- detail: course info (name, price, user count, shop)
- catalog: full course outline supporting normal courses (type 50),
columns (type 6), and big columns (type 8)
- play-url: get M3U8 play URL via direct API for video courses,
and Vue component tree search + Performance API polling for
live replay courses
- content: extract rich-text page content as plain text
Technical notes:
- Strategy: cookie (reuses Chrome login session)
- Framework: Vue 2 + Vuex Store (SPA)
- Video courses use a two-step API chain:
detail_info.get → play_sign → getPlayUrl → M3U8
- Live replays use Performance API + Vue data tree polling
- Catalog expands chapters via Vue component method getSecitonList()
- Supports multiple stores (cross-domain cookie sharing via
study.xiaoe-tech.com)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* review: stop truncating xiaoe content
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix: add -v/--verbose to explore, record, generate, cascade
Built-in browser commands were registered directly in cli.ts and
missed the -v/--verbose flag that commanderAdapter.ts wires up for
adapter commands. Also switch explore's lone log.debug() call to
log.verbose() so the flag has visible effect.
Closes#716
* refactor(cli): make builtin command wiring testable
* refactor(cli): simplify verbose wiring, use normal Commander pattern
Replace registerVerboseAction wrapper with simple applyVerbose() helper.
The wrapper broke Commander's builder chain and created awkward
indentation. Now each command uses standard .option().action() with
applyVerbose(opts) as the first line — easier to read and maintain.
* fix(cli): add -v/--verbose to doctor and synthesize commands
These commands were also missing verbose support, same root cause as
explore/record/generate/cascade — registered directly in cli.ts,
bypassing commanderAdapter's automatic -v wiring.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(twitter): add --images flag to post command
Support attaching up to 4 images when posting tweets via
`opencli twitter post "text" --images /path/a.png,/path/b.jpg`.
Uses the existing CDP DOM.setFileInputFiles mechanism (page.setFileInput)
to inject files into Twitter's file input. Includes proper file validation,
graceful error handling for older extensions, and polling-based upload
readiness detection instead of fixed delays.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(twitter): use attachments DOM signal for upload detection, add tests
Replace unreliable tweet-button-only polling with dual-condition check:
wait for [data-testid="attachments"] with correct [role="group"] count
AND button enabled. Increase timeout to 30s. Add 8 unit tests covering
image upload flow, file validation, and error paths.
Addresses PR #666 review feedback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(twitter): use top-level imports, fix test mocks, faster upload poll
- Use top-level fs/path imports instead of dynamic imports inside func
- Fix test statSync mock to return undefined (not null) for missing files
- Fix test path mock to preserve other exports via importOriginal
- Fix null type error in no-browser-session test
- Reduce upload poll interval from 1s to 500ms for faster detection
- Use JSON.stringify for imageCount interpolation for consistency
* refactor(twitter): extract validation, fail-fast, reduce duplication
- Extract validateImagePaths() with extension validation (jpg/png/gif/webp)
matching xiaohongshu publish pattern
- Validate images before browser navigation (fail-fast on bad input)
- Remove try/catch wrapper around setFileInput — let errors propagate
naturally instead of masking the original error
- Deduplicate tweetButton/tweetButtonInline lookups using fallback OR
- Use constants for MAX_IMAGES, UPLOAD_POLL_MS, UPLOAD_TIMEOUT_MS
- Add tests: unsupported format, validates-before-navigating
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(gemini): stabilize ask reply state handling
* fix: use CommandExecutionError for composer failures and clean up formatting
- Replace raw Error with CommandExecutionError for Node-side composer
failures (prepareComposer, insertText) to match adapter error conventions
- Remove extra blank lines after __test__ export
* refactor: remove dead code and add Chinese sign-in label
- Remove unused areGeminiTurnsEqual and areGeminiLinesEqual functions
- Add Chinese sign-in label (登录) to sign-in detection for consistency
with other Chinese labels already added in this PR
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Move data processing (HTML stripping, answer mapping) from browser-side
evaluate to Node-side, keeping the evaluate minimal: just fetch + status
check. Uses __httpError sentinel consistent with pixivFetch convention.
- Remove hover_price_text as MOQ source in search normalizeSearchCandidate
to prevent price fields from being misinterpreted as MOQ data
- Rename firstLine() to firstWord() to match its actual behavior (splits
by whitespace, not newlines)
- Add missing "单" unit to item.ts extractSalesText regex
- Add test case verifying hover_price_text is not used for MOQ
* fix(zhihu): make question runtime-compatible
* fix: validate questionId is numeric to prevent interpolation issues
* refactor: simplify evaluate string and harden against injection
- Build URL in Node.js, embed via JSON.stringify for safety-by-design
- Remove unnecessary (page as any) cast — IPage already has evaluate
- Simplify error message construction (no nested ternaries)
- Replace implementation-detail test with numeric ID validation test
* refactor: simplify zhihu question — move stripHtml into evaluate, return clean data
* fix: add colon separator in fetch error message for readability
"request failed Failed to fetch" → "request failed: Failed to fetch"
---------
Co-authored-by: Kyrie <kyrie@mallab.world>
Co-authored-by: jackwener <jakevingoo@gmail.com>
1. marks: correct pageSize from 30 to 15 — douban grid mode shows 15
items per page, causing pagination to stop after the first page.
2. subject: split title/originalTitle correctly — v:itemreviewed contains
both Chinese and original titles concatenated.
3. subject: extract country/region from #info as list, split by "/".
4. subject: extract duration as pure number (min) from v:runtime or #info.
5. subject: return casts as list instead of comma-joined string.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- bilibili subtitle/comments tests: use importOriginal to include
resolveBvid in utils mock
- comments test: use valid BV ID format for aid-resolution error test
- launcher test: skip pgrep test on win32 (detectProcess early-returns)
* fix(windows): graceful degradation and manual CDP override for Electron apps
* fix: validate OPENCLI_CDP_ENDPOINT with probeCDP before use
Fail-fast with a clear error if the manual CDP endpoint is not reachable,
instead of passing a bad URL downstream and getting a confusing error.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(bilibili): support b23.tv short URL/short code resolution
Add resolveBvid() in utils.ts to automatically resolve b23.tv short URLs
and short codes to BV IDs. Supports all input formats:
- BV ID: BV1MV9NBtENN (pass through)
- Short code: XYzsqGa
- Short URL: https://b23.tv/XYzsqGa, b23.tv/XYzsqGa
Uses Node.js https.get with 302 redirect only (no body download),
typically ~100-250ms resolution time.
Applied to: subtitle, comments, download commands.
* fix: add timeout, input coercion, and tests for resolveBvid
- 5s timeout on https.get to prevent hanging on unresponsive b23.tv
- Accept unknown input type with String() coercion
- Simplify callers (remove redundant String().trim() wrappers)
- Add unit tests for BV ID passthrough and edge cases
---------
Co-authored-by: chenruinian <chenruinian@Sa1kas-MacBookPro.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: auto-downgrade table output to YAML in non-TTY environments
When stdout is not a TTY (pipes, AI agents, subprocesses), automatically
output YAML instead of table with ANSI colors and box-drawing characters.
This makes opencli output parseable by downstream tools and AI agents.
Behavior:
- TTY: table (default, unchanged)
- Non-TTY: yaml (auto-detected)
- OUTPUT env var: overrides auto-detection (yaml/json/table/etc)
- Explicit -f flag: always respected
* fix: TTY detection now works with commanderAdapter default fmt
- fmt='table' from commanderAdapter now correctly triggers non-TTY downgrade
- Priority: explicit -f (non-table) > OUTPUT env var > TTY auto-detect
- Added test for explicit -f precedence over OUTPUT env var
* fix: explicit -f flag now takes precedence over TTY auto-detection
Use Commander's getOptionValueSource to distinguish explicit -f from
default. Explicit -f table in non-TTY keeps table output. Only auto-
downgrade when user didn't pass -f.
Priority: explicit -f > OUTPUT env var > TTY auto-detect > table default
* fix: explicit -f also skips command defaultFormat override
When user passes -f explicitly, command-level defaultFormat (e.g.
gemini/ask defaultFormat:'plain') no longer overrides their choice.
* feat(amazon): unify ranking adapters for three signal boards
* refactor: simplify bestsellers wrapper and fix pagination detection for all ranking types
1. Remove unnecessary __test__ wrapper from bestsellers.ts — the test
now uses normalizeRankingCandidate directly from rankings.ts,
eliminating a needless indirection layer.
2. Fix isRankingPaginationUrl to detect pagination refs for all ranking
types: zg_bs_pg_ (bestsellers), zg_bsnr_pg_ (new releases),
zg_bsms_pg_ (movers & shakers). Previously only matched the
bestsellers-specific ref pattern.
---------
Co-authored-by: 泽加武 <zejiawu@zejiawudeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix doubao image urls in read output
* fix(doubao): derive image selector from messageTextSelectors
Hardcoded image selector only covered the first two text selectors,
so images inside class-based message containers would be missed.
Generate from the shared selector list for consistency.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* refactor(xiaohongshu): replace blind retry with MutationObserver wait
Instead of retrying the entire navigation when search results are empty,
use a MutationObserver to wait for `section.note-item` elements (or login
wall text) to appear in the DOM, with a 5s timeout. This is faster (resolves
as soon as content renders) and more correct (addresses the root cause of
delayed hydration rather than working around it with a full re-navigation).
* simplify: merge login-wall detection into MutationObserver wait
WAIT_FOR_CONTENT_JS now returns 'content', 'login_wall', or 'timeout'
instead of just true/false. This eliminates the separate login-wall
evaluate call and the redundant loginWall field in the extraction payload.
Two evaluate calls total (wait + extract) instead of three.
* fix: add prepare script so source installs trigger build
npm install from git (e.g. npm install github:jackwener/opencli) skips
prepublishOnly, so dist/ is never generated. The prepare hook runs on
git-based installs; the [ -d src ] guard skips it for registry installs.
* fix: include extension/dist in git so clone works out of the box
.gitignore had conflicting rules: line 3 tried to un-ignore extension/dist/
but line 26 re-ignored it. Remove the later rule so the built extension JS
is tracked in git — users can load the extension directly after clone.
* fix(doubao-app): connect to correct CDP target instead of background page
Doubao desktop app exposes multiple CDP targets. The scoring logic picked
the background page (doubao-background) over the actual chat page because
its URL-as-title contained "doubao", boosting its score above the real
chat page (title "豆包"). This caused all commands (send, ask, read) to
fail with "No textarea found".
- Add `targetFilter` field to ElectronAppEntry for per-app preferred target
- Set doubao-app targetFilter to 'doubao-chat/chat'
- Penalize background/new-tab-page URLs and URL-like titles in scoring
- Thread cdpTargetFilter through execution → runtime → CDPBridge
Closes#634, closes#506
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(cdp): exclude background targets instead of targetFilter
Replace the targetFilter plumbing (4 files, new interface field) with
a single-line fix: exclude `background_page` and `service_worker`
type targets from CDP selection entirely.
Background pages should never be connection targets — they have no
visible DOM and all selectors will fail. This is the root cause of
#506/#634 (doubao-app connecting to empty background page).
Simpler fix: 1 line added vs 4 files modified. No new interface
fields, no per-app configuration needed.
---------
Co-authored-by: 刘启灏 <liuqihao@liuqihaodeMacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
When other Chrome extensions (tab managers, new-tab overrides) move
automation tabs to a different window, the Browser Bridge now attempts
to move the tab back to the automation window rather than creating a
new one. This preserves the existing page state and avoids redundant
navigation.
Changes:
- resolveTab(): when a provided tabId has drifted to another window but
content is still debuggable, use chrome.tabs.move() to bring it back
- handleNavigate(): after navigation completes, detect if the tab drifted
during navigation and move it back to the session window
- cdp.ts ensureAttached(): log final tab URL and windowId on attach
failure for better diagnosis of extension conflicts
Closes#652 (partially — addresses tab drift recovery and diagnostics)
PR #712 refactored _ensureDaemon to use a single fetchDaemonStatus() call
instead of separate isDaemonRunning(). The test was still mocking the old
function, causing it to fall through to the spawn-daemon path and throw
the wrong error message.
1. eval retry delay: 1000ms → 200ms for SPA navigation errors, 500ms
for debugger detach. SPA navigations recover within ~100ms, the old
1000ms delay was unnecessarily long.
2. Window creation: replace fixed 200ms sleep with tab-load poll.
Listens for chrome.tabs.onUpdated status=complete with 500ms
fallback cap. about:blank loads in ~20ms, saving ~180ms.
3. bridge.ts _ensureDaemon: single fetchDaemonStatus() call instead of
two sequential calls (isExtensionConnected + isDaemonRunning both
called fetchDaemonStatus independently). Saves one HTTP round-trip.
4. goto() post-navigation: coalesce stealth injection + DOM settle into
a single exec call. Previously two sequential round-trips
(Node→daemon→WS→extension→CDP each). Saves ~60-160ms per goto().
Two changes that eliminate the about:blank → target-domain navigation
on first command execution:
1. Extension: getAutomationWindow() accepts an optional initialUrl.
When creating a new window, uses the target URL directly instead
of about:blank. handleNavigate() passes cmd.url through so the
window starts on the correct domain.
2. CLI: Remove isAlreadyOnDomain() check before pre-nav. Instead,
always call page.goto(preNavUrl) — the extension's handleNavigate
already has a fast-path that skips navigation when the tab is
already at the target URL. This avoids an extra exec round-trip
(getCurrentUrl eval) on first command.
Net effect: first command saves ~1-3s (one fewer page load),
subsequent commands behave the same (navigate fast-path handles
domain matching efficiently via chrome.tabs.get).
Both methods had zero production callers — only test mocks referenced them.
newTab() created about:blank pages via CDP Target.createTarget, but no
adapter or pipeline step ever invoked it. closeTab() was similarly unused.
selectTab() and tabs() are kept as they have active production usage
(e.g. doubao adapter). The scoreTarget about:blank penalty is retained
as a defensive measure against user-opened blank tabs.
* docs: improve operate skill with Browser Use best practices
- Add Critical Rules section (state over screenshot, verify with get value)
- Add Command Cost Guide (free/instant vs expensive vision tokens)
- Add Action Chaining Rules (safe to chain vs page-changing)
- Add Tips section
- Fix Core Workflow to use state/get value for verification, not screenshot
- Mark screenshot as "ONLY for user deliverables"
Inspired by Browser Use's design: DOM-first state representation,
action cost awareness, and multi-action chaining patterns.
* docs: fix operate skill — eval read-only, IIFE, interaction rules
- Add rule: NEVER use eval to click/type — use click/type/select commands
(eval bypasses scrollIntoView + CDP pipeline, fails on off-screen elements)
- Add rule: eval is read-only, always wrap in IIFE to avoid variable conflicts
- Reorder Critical Rules for priority
- Add IIFE example in Extract section
Root cause: Claude Code was using eval("el.click()") instead of
click <index>, and hitting "already declared" errors from repeated
eval calls in the same page context.
* feat: Browser Use best practices — click/type/state improvements
Inspired by deep analysis of Browser Use's design patterns:
1. Framework listener detection (React/Vue/Angular)
- Detect __reactProps$ onClick, Vue _vei, Angular ng-reflect-click
- Catches <div onClick> elements that pure ARIA/tag heuristics miss
2. Click CDP fallback
- clickJs() now returns coordinates on failure
- BasePage.click() falls back to CDP Input.dispatchMouseEvent
- Page.clickWithQuads() uses DOM.getContentQuads for inline elements
3. Type improvements
- React-compatible: use native HTMLInputElement.prototype.value setter
- Contenteditable: selectAll + execCommand('insertText') for rich editors
- Autocomplete: detect role=combobox, wait 400ms for dropdown suggestions
4. getContentQuads precise click
- Page.clickWithQuads() for multi-line inline elements (e.g. wrapped <a>)
- Falls back through getContentQuads → getBoxModel → JS click
* fix: address code review — injection, silent failure, setter prototype
1. clickWithQuads: escape ref with JSON.stringify before inserting into
JS strings and CSS selectors (injection risk)
2. base-page click: throw error when both JS click and CDP fallback fail
instead of silently succeeding
3. typeTextJs: use matching prototype for native setter
(HTMLTextAreaElement for textarea, HTMLInputElement for input)
* docs: remove Why opencli section, merge advantages into Highlights, add operate quickstart to CN README
- Remove "Why opencli?" / "为什么选 opencli？" sections from both READMEs
- Incorporate Zero LLM cost, Deterministic, Broad coverage bullets into Highlights
- Add operate command mention to AI Agent ready highlight
- Add browser automation / operate quickstart section to README.zh-CN.md (mirrors English README)
* docs: update Built for AI Agents paragraph, add browser automation and website→CLI to Highlights, remove Dual-Engine
- Rewrite "Built for AI Agents" to emphasize operate skill + browser control + crystallizing into CLIs
- Add "Browser Automation" and "Website → CLI" bullets to Highlights (both EN and CN)
- Remove "Dual-Engine Architecture" bullet from EN Highlights
- Remove "动态加载引擎" from CN Highlights (already covered by other bullets)
* docs: remove human quickstart from operate section, AI-only
Root SKILL.md was already removed in #703. Add per-skill install
commands to both EN and CN READMEs (without --full-depth since
root SKILL.md no longer blocks sub-skill discovery).
- Update adapter count from 50+/60+/66+ to 70+ across all docs (actual: 74 sites)
- Add missing operate commands (eval, network, init, verify) to README
- Add opencli-operate skill to Install AI Skills section in both READMEs
- Replace outdated "Playwright MCP Bridge" with "Browser Bridge" in doubao docs
* fix(twitter): add search input fallback for intermittent SPA navigation failures
The pushState + popstate approach works in most environments but fails
intermittently for some users (see #690), likely due to Twitter A/B
tests or timing race conditions where the pathname hasn't updated when
checked.
This commit adds a fallback strategy: when pushState fails after 2
retries, we type the query into the search input on /explore and press
Enter. This triggers Twitter's own form handler, performing SPA
navigation without a full page reload (keeping the fetch interceptor
alive).
Both strategies use selector-based waiting ([data-testid="primaryColumn"])
rather than fixed delays, with graceful fallthrough on timeout.
Fixes#690
* test(twitter): update search test for fallback evaluate call
The search input fallback adds one extra evaluate() call when pushState
fails. Update the mock chain and assertion count accordingly.
* fix(twitter): guard nativeSetter and add fallback success test
- Add optional chaining on getOwnPropertyDescriptor().set to handle
edge cases where Twitter's sandbox overrides the HTMLInputElement
prototype.
- Add test case covering the full fallback path: pushState fails twice,
search input fallback succeeds, results are returned correctly.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(skills): add YAML frontmatter for discovery and improve descriptions
- opencli-explorer: add missing frontmatter with name, description, tags
- opencli-oneshot: add missing frontmatter with name, description, tags
- opencli-usage: rewrite description to start with "Use when..." and
include specific platform names for better keyword matching
Root cause of low trigger rate: explorer and oneshot had no frontmatter
at all, making them invisible to AI agent skill discovery. Usage had a
generic description without triggering conditions.
* fix(skills): add capability index, cross-skill links, and plugins entry
- Add "Quick Lookup by Capability" table so agents can find platforms
by what they need (search, trending, feed, AI chat, finance, etc.)
- Add plugins.md entry to main index (was completely hidden)
- Add "Related Skills" section linking to opencli-explorer and
opencli-oneshot for adapter development
- Compress platform listings for scannability
* fix(skills): inline compact command quick-reference table in SKILL.md
Add a self-contained command reference table directly in SKILL.md so
agents that can only read the main skill file still have full command
visibility. Each platform gets one row with all available commands.
Organized into Browser/Desktop/Public API/Management sections.
* fix(twitter): resolve article ID to tweet ID before GraphQL query
Article URLs (x.com/i/article/{articleId}) use a different ID than
tweet status URLs. The GraphQL TweetResultByRestId endpoint requires
the parent tweet ID, not the article ID.
Fix: navigate to the article page first, extract the associated tweet
ID from DOM links, then use that for the GraphQL query.
Fixes article fetching returning "Article not found" for all article URLs.
* fix: distinguish article URLs from status URLs, add explicit error handling
The previous commit routed all inputs through the article page, breaking
status URL and bare ID flows. Now only article URLs trigger the
article→tweet ID resolution. Status URLs and bare IDs keep the original
behavior. Also throws an explicit error if resolution fails instead of
silently falling back to the article ID.
---------
Co-authored-by: buruguo <buruguo@lambdafintech.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu): clarify empty note shell hint
* fix(xiaohongshu): simplify empty shell detection to title+author check
The 7-field conjunction was overly strict — a note that rendered only
placeholder metrics but no title/author was still a valid empty shell.
Since title and author are always present on real notes, checking just
those two fields is a more reliable and simpler signal.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: create skills/ directory structure per issue #605
- Create skills/opencli-usage/ with index and categorized command references
- SKILL.md: main index with installation and prerequisites
- browser.md: all browser-based commands (Bilibili, Twitter, Reddit, etc.)
- desktop.md: desktop adapter commands (Cursor, Codex, Notion, etc.)
- public-api.md: public API commands (HackerNews, V2EX, arXiv, etc.)
- plugins.md: management commands, AI workflow, output formats
- Create skills/opencli-explorer/ from CLI-EXPLORER.md
- Create skills/opencli-oneshot/ from CLI-ONESHOT.md
Addresses #605 - enables skill-based discovery and selective installation
* fix: complete browser.md with all missing adapters and fix incorrect entries
- Added 15 missing browser adapters: Reuters, SMZDM, Ctrip, Barchart,
Jike, Linux.do, WeRead, Jimeng, Pixiv, Web, Weixin, JD, LinkedIn,
Sina Finance, Bloomberg (browser)
- Fixed incomplete entries: Facebook (added 5 missing commands),
Coupang (corrected to match actual CLI), Yollomi (restored all 12
commands), Doubao Web (restored send/read commands), Grok (fixed format)
- Added missing public APIs: StackOverflow, Xiaoyuzhou, Wikipedia
- Updated SKILL.md index to list all supported platforms across all
categories including desktop adapters
* refactor: remove root SKILL.md, migrate Record Workflow to opencli-explorer
- Moved Record Workflow documentation (工作原理, 使用步骤, 页面类型表,
候选 YAML→TS 转换, 故障排查) into skills/opencli-explorer/SKILL.md
- Deleted root SKILL.md — all content now lives under skills/
* docs: add AI skills installation guide to README
Add npx skills add instructions for all 3 skills (opencli-usage,
opencli-explorer, opencli-oneshot) to both README.md and README.zh-CN.md.
* refactor: deduplicate transient error checks, cache VM contexts, expose tab ID
- Extract shared isTransientBrowserError() into browser/errors.ts, replacing
duplicated string-matching lists in daemon-client.ts and pipeline/executor.ts
- Cache compiled vm.Script objects in template.ts with LRU eviction (max 256),
avoiding per-invocation VM context creation in pipeline loops
- Add getActiveTabId() to IPage interface and Page class for tab state inspection
* refactor: extract BasePage to deduplicate DOM helpers across Page and CDPPage
Both Page (daemon-backed) and CDPPage (direct CDP) had ~200 lines of
identical DOM helper implementations (click, type, scroll, wait, snapshot,
interceptor, etc). Extract shared logic into abstract BasePage class.
Subclasses now only implement transport-specific methods.
* refactor: rename mcp.ts to bridge.ts and clean up Playwright MCP references
The file browser/mcp.ts contained BrowserBridge (daemon session manager),
not MCP functionality. Renamed to bridge.ts for clarity. Also removed all
stale "Playwright MCP" references from comments and variable names across
the codebase — Playwright was removed long ago.
* fix(notebooklm): remove bind-current workflow
* fix: relax notebook ID check in open.ts and clean up idle timeout test
- open.ts: only throw when page kind is not 'notebook'; log a warning
instead of throwing when the notebook ID doesn't match exactly
- background.test.ts: remove unused tabs[1] setup in idle timeout test
that was leftover from borrowed-session era
* build: rebuild extension dist after bind-current removal
* docs: add dingtalk and wecom CLI to external CLI hub
Add dingtalk-workspace-cli and wecom-cli as external CLI integrations
alongside lark-cli, gh, docker, etc.
* feat: add confirmPrompt() to TUI module
* feat: add Electron app registry with builtin + user-defined apps
* feat: add Electron app launcher with auto-detect and restart
* fix: launcher uses processName for path discovery, platform-guard tests
* feat: integrate Electron auto-launcher into execution pipeline
- CDPBridge.connect() accepts cdpEndpoint parameter instead of requiring env var
- getBrowserFactory() selects CDPBridge for registered Electron apps by site name
- executeCommand() calls resolveElectronEndpoint() for Electron apps, skips daemon check
- Remove requiredEnv/OPENCLI_CDP_ENDPOINT from all chatwise commands
- Remove chatwise-opencli.ps1 wrapper script and chatwise/shared.ts
- Update antigravity/serve.ts to use launcher instead of manual env var
- Replace hardcoded app names in scoreCDPTarget with registry lookup
- Fix Discord bundleId typo (com.iscord.app → com.discord.app)
* fix: resolve review issues — port collision and registry completeness
- Change ChatGPT CDP port from 9224 to 9236 (was colliding with Antigravity)
- scoreCDPTarget now uses full registry (builtin + user-defined) via getAllElectronApps()
- Use displayName (falling back to processName) for target score boosting
* fix: assign unique CDP ports — antigravity 9234, chatgpt 9236
Both were sharing port 9224, which could cause silent mis-connection.
- Add content field to display topic body text
- Add member field to show topic author
- Add created field to show topic creation timestamp
- Add node field to show topic category
- Add id field for consistency with hot/latest commands
This makes v2ex topic command return meaningful details that are
not available in hot/latest listings.
* feat(youtube): add --type shorts/video/channel, --upload, --sort filters
Uses YouTube's native sp= filter params. Shorts = type 9 (sp=EgIQCQ).
Also parses reelItemRenderer for Shorts results.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(youtube): add published time to search results
Shows when video was uploaded (e.g. "8h ago", "4d ago", "3mo ago").
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(youtube): prevent duplicate sp= params and remove redundant Shorts URL rewrite
- YouTube only supports one sp= parameter; using multiple causes
unpredictable behavior. Pick the most specific filter with priority:
type > upload > sort.
- Remove the post-processing Shorts URL rewrite — the reelItemRenderer
branch already generates /shorts/ URLs directly.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(bilibili): distinguish login-gated subtitles from empty results
* fix(test): use single toSatisfy assertion instead of double rejects.toThrow
Awaiting the same rejected promise twice is unreliable. Combine the
AuthRequiredError type check and message regex into one toSatisfy call.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(v2ex): add id field to hot and latest API responses
- Add id field to hot.yaml and latest.yaml pipeline output
- Enables downstream commands like 'v2ex topic <id>' to work seamlessly
- Fixes issue where v2ex hot/latest JSON output lacked topic IDs
* enhance(v2ex): add node and url fields to hot/latest output
In addition to the id field, include node name (板块) and topic URL
for richer output. All fields come from the existing API response.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(facebook): split search navigation from extraction
* refactor: use settleMs instead of waitUntil:none + wait:4
Replace `waitUntil: none` + separate `wait: 4` step with `settleMs: 4000`
on the navigate step. This is consistent with other Facebook adapters
(feed.yaml, memories.yaml, profile.yaml) and lets the navigate step
handle the timing in one place.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
VitePress base is /docs/, so absolute links like /adapters/browser/twitter
resolve incorrectly. Changed all links to relative paths (./browser/...,
./desktop/...) so they work correctly on the docs site.
* fix(substack): update selectors for Substack DOM redesign (fixes#621)
Substack replaced <article> elements with role="article" divs and a
new SPA-based feed. The wait() selector 'article' no longer matches,
causing 'Selector not found: article' on feed and publication commands.
- loadSubstackFeed: use 'a[href*="/p/"]' (matches actual post links)
- loadSubstackArchive: use '[role="article"]' (Substack's new ARIA roles)
The evaluate() scraping logic inside both functions is unchanged since
it already uses 'a' href pattern matching, not article tags.
* review: align substack wait selectors with scraper
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(ctrip): update search adapter to live endpoint
* review: make ctrip search a public fetch adapter
---------
Co-authored-by: 泽加武 <zejiawu@zejiawudeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu): support full URL/short link and fix video extraction
Two issues fixed:
1. URL handling: The download command only accepted bare note IDs and
constructed `explore/{noteId}` URLs, which lack the `xsec_token`
parameter now required by Xiaohongshu. This made all video/image
downloads fail with "No media found". Now accepts full URLs
(with xsec_token) and short links (xhslink.com) in addition to
bare note IDs.
2. Video extraction: XHS video player uses blob: URLs in DOM, which
cannot be downloaded via HTTP. Now extracts real video URLs from
`window.__INITIAL_STATE__` (SSR data) and inline script JSON
before falling back to DOM selectors, skipping blob: URLs.
Tested with a video note via short link — successfully downloaded
21.5 MB MP4.
* review: resolve xiaohongshu note id after redirects
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(xiaohongshu): add note command and nested reply support for comments
Add `xiaohongshu note` command to read full note content (title, author,
description, engagement metrics, tags) from public note pages.
Enhance `xiaohongshu comments` with `--with-replies` flag to extract
nested replies (楼中楼), including reply_to attribution and per-reply
like counts. Limit logic counts only top-level comments so replies
are included for free.
Extract shared `parseNoteId` into side-effect-free `note-helpers.ts`
to avoid cross-module command registration leakage.
Normalize non-numeric engagement placeholders ("赞"/"收藏"/"评论")
to "0" for zero-count notes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(xiaohongshu): add note and comments --with-replies to adapter docs
Update xiaohongshu adapter documentation and README command table
to reflect the new note command and enhanced comments with nested
reply support.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(xiaohongshu): fix download example to show both note-id and url
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(xiaohongshu): expand nested reply threads before scraping
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
- Add early login-wall detection before autoScroll() in search.ts
to prevent crash when XHS shows a login gate instead of results
- Add document.body null guard in autoScrollJs (dom-helpers.ts)
- Update search.test.ts: verify autoScroll is not called on login wall
- Add autoScrollJs null-body defense test in dom-helpers.test.ts
* docs: add dingtalk and wecom CLI to external CLI hub
Add dingtalk-workspace-cli and wecom-cli as external CLI integrations
alongside lark-cli, gh, docker, etc.
* feat: register dingtalk and wecom as external CLIs
Add dws (DingTalk Workspace CLI) and wecom-cli to
external-clis.yaml so they are discoverable via opencli list
and auto-installable.
* feat(browser): add ONES adapter support for tasks and worklog commands
Add ONES auth/session commands, task listing/details utilities, and worklog operations, with related docs and helper utilities.
* fix(ones): harden worklog and task-list adapter behavior
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(tieba): add browser adapters for hot posts search and read
* fix(tieba): stabilize search and e2e coverage
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(doubao): add history, detail, meeting-summary and meeting-transcript commands
- history: list conversation history from sidebar
- detail: read a specific conversation by ID, with meeting card detection
- meeting-summary: extract summary and AI chapters from meeting minutes
- meeting-transcript: read or download meeting transcript via browser
Made-with: Cursor
* docs: update doubao command list in adapter index and README.zh-CN
Made-with: Cursor
* fix(doubao): handle meeting-only detail and merge transcript snapshots
* refactor(doubao): model conversation ids as first-class output
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(douyin): add user-videos command with top-10 comments
Adds a new adapter for fetching a public user's video list by sec_uid,
alongside the top-10 hottest comments for each video.
- Navigates to the user's profile page to establish a cookie session
- Fetches video list via /aweme/v1/web/aweme/post/
- Concurrently fetches top-10 comments per video via
/aweme/v1/web/comment/list/ (sorted by hotness, API default)
Output columns: index, aweme_id, title, duration, digg_count,
play_url, top_comments
* refactor(douyin): replace Object.assign with spread in user-videos
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(douyin): validate user-videos inputs
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(xiaohongshu): add cover image URL to user notes output
Extract cover image URL from noteCard.cover.urlDefault in
__INITIAL_STATE__ and include it in the user command output columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test(xiaohongshu): cover user note rows
* refactor(xiaohongshu): keep cover out of default columns
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(spotify): add Spotify playback adapter
Adds a new adapter for controlling Spotify via the official Web API.
Uses Strategy.PUBLIC with OAuth2 — no browser session required.
Commands: auth, status, play, pause, next, prev, volume, search, queue, shuffle, repeat.
Credentials are loaded from ~/.opencli/spotify.env or environment variables.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(spotify): rename index.ts → spotify.ts and fix CliError calls
- Renamed src/clis/spotify/index.ts to spotify.ts so the build-manifest
picks it up (index.js is intentionally excluded from manifest scanning)
- Fixed 4 CliError calls: constructor now requires (code, message, hint?)
so each throw now passes an appropriate error code as first argument
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(spotify): fix token refresh corruption, env parse, null guards, validation
- refreshAccessToken: check res.ok before parsing; construct Tokens object
directly instead of mutating loadTokens() result to avoid writing
undefined/NaN on Spotify error responses; preserve existing refresh_token
when Spotify omits it from the response
- loadEnv: split on first '=' only so values containing '=' are preserved
- SCOPES: remove write/library/top scopes not used by any command
- status: guard against data.item being null (active device but no track)
- volume: validate 0-100 range before API call
- auth: check tokenRes.ok on initial token exchange; add server.on('error')
handler for EADDRINUSE; add 5-minute timeout with clearTimeout on close
* feat(postinstall): auto-create ~/.opencli/spotify.env template on install
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(spotify): guard null progress, podcast items, missing tracks data, corrupted tokens, invalid search limit
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(spotify): improve missing credentials error with step-by-step guidance
* fix(spotify): harden setup and add docs coverage
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(band): add bands, posts, and mentions commands for band.us
- bands: lists all Bands via get_band_list_with_filter intercept
- posts: lists posts from a Band via get_posts_and_announcements intercept
- mentions: shows @mention notifications via get_news intercept
All use Strategy.INTERCEPT since band.us API requires an HMAC md header
generated by its own JS. SPA navigation to /band/{no}/post triggers the
band list and posts APIs; bell + @メンション tab click triggers mentions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(band): clean up all three band adapters
- Fix doc comments: Band uses XHR not fetch; clarify INTERCEPT rationale
- bands: replace for-loop with flatMap; explain why band page nav is needed
- posts: remove item.post ?? item fallback (API always wraps in post); rename
finalRequests → requests for consistency; extract stripBandTags helper
- mentions: remove redundant ?? defaults (args have defaults defined); fix
unreadOnly bug (was not applied to post/comment modes); consolidate Band tag
stripping to single regex; cast kwargs types directly instead of converting;
add comments explaining last-response strategy and 'referred' filter flag
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band/posts): handle mixed post/announcement items from API
get_posts_and_announcements returns both regular posts and announcements
that have different shapes — some lack post_no and wrap differently.
Restore item.post ?? item fallback and filter out items with no resolvable
identifier to prevent undefined in URLs and empty rows in output.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(band): add post command — full post export with comments and photo download
Exports the complete content of a single Band post:
- Post body (with Band markup tags stripped)
- All comments in chronological order
- Photo URLs shown inline, or downloaded with --output <dir>
Uses Strategy.INTERCEPT with a broad 'band.us' pattern to capture both the
batch request (embedding get_post) and get_comments in one SPA navigation.
Responses are identified client-side by shape: batch_result array vs items
array with comment_id fields.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(band): replace XHR interception with direct DOM extraction
- bands, posts, post: navigate directly to target URL instead of home→SPA detour
- All three switch from Strategy.INTERCEPT to Strategy.COOKIE with navigateBefore: false
(bands uses framework pre-nav to home; posts/post disable it and goto target directly)
- DOM extraction polls for specific content elements rather than fixed waits
- post: confirm selectors via browser inspection (a.text, time.time, .sCommentList,
.sReplyList for nested replies); add --comments flag to skip comment fetch
- posts: extract from rendered post list DOM; correct comment item selector (div.cComment)
- Fix: post empty-result guard changed from && to handle null data safely
- Fix: photo download now checks HTTP status code before piping to avoid writing
redirect HTML into image files
- Fix: mentions unread client-side filter skipped for 'mentioned' mode since
server already filtered via 未確認のみ button click
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address code review feedback
- post: replace manual http/https download with shared downloadMedia utility
(handles redirects, timeouts, stream errors correctly)
- post: fix photo URL resolution to use location.href as base, handling
protocol-relative and relative URLs without throwing
- post: switch to node:-prefixed imports per repo convention
- post/posts: remove redundant ArgumentError guards — framework already
validates required args before func() is called
- mentions: INTERCEPT strategy is intentional (Band HMAC prevents DOM-only
approach for notifications; update PR description to clarify)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address second round of code review feedback
- bands: tighten href selector to /band/{id}(?:/post)?$ so feed/post-detail
links are excluded; only sidebar navigation links match
- mentions: replace fixed page.wait(2) sleeps with polling on
getInterceptedRequests() — waits up to 8 s per action, exits as soon
as the expected number of captures arrives (avoids flakiness on slow XHR)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): fix selector bugs found during testing
- bands: use a.bandCover._link + p.uriText + span.member em selectors
(previous a[href*="/band/"] + .bandName combo leaked "メンバー" text)
- posts: use article.cContentsCard._postMainWrap + span.count selectors
(previous li._postListItem selector matched nothing; DOM changed)
- mentions: fix page.wait(500) → page.wait(0.5) (was waiting 500s not ms);
use timestamp-suffixed URL to force fresh page load each run so the
notification panel is closed; fix get_news vs get_news_count capture
ambiguity with result_data.news check; replace cumulative waitForCaptures
with waitForOneCapture (getInterceptedRequests clears array on each call)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band/mentions): use CSS class selector for bell button instead of locale-dependent text match
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address third round of code review feedback
- post: pass browser cookies to downloadMedia so Band's login-protected
photo URLs don't fail with 401/403
- post: include photos.length in empty-result guard so photo-only posts
are not falsely reported as not found
- mentions: accumulate captures across poll iterations so get_news_count
responses don't cause early exit before the real get_news arrives
- mentions: update docstring to match actual implementation (client-side
filtering, no tab-click)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address fourth round of code review feedback
- mentions: fail fast with a clear error when bell button is not found,
instead of silently no-op and waiting 8s before EmptyResultError
- post: use shared formatCookieHeader() instead of manual cookie string
construction
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address fifth round of code review feedback
- mentions: replace fixed page.wait(2) with polling for bell button
readiness (up to 10s), eliminating the fixed sleep and fail-fast
when the selector is missing
- mentions: add explicit !newsReq guard with a clear error message when
get_news capture times out, instead of falling through to a misleading
"No notifications found"
- posts: skip posts with no permalink href instead of emitting a bogus
'https://www.band.us' URL
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address sixth round of code review feedback
- post: only send Band cookies to *.band.us photo URLs; third-party CDN
URLs are downloaded without cookies to avoid cross-domain cookie leakage
- bands: strip non-digit chars before parseInt so member counts like
"1,234" parse correctly
- posts: same fix for comment counts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address seventh round of code review feedback
- posts: check limit before push so --limit 0 returns empty result
- post: indent replies proportionally by depth (' '.repeat(depth))
so multi-level threads remain readable in table output
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band/bands): anchor href regex to prevent matching post-detail URLs
Pattern now requires /band/{id} or /band/{id}/post (with optional trailing
slash) so deeper paths like /band/{id}/post/{postNo} are excluded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address ninth round of code review feedback
- mentions: guard bell click with a boolean return so a disappearing
element throws a clear EmptyResultError instead of a raw TypeError
- post: wait for comment list container instead of first .cComment so
posts with zero comments don't incur a fixed 6s delay
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): use page.getCookies() for login detection across all commands
Replaces document.cookie.includes('band_session') with
page.getCookies({ domain: 'band.us' }) so login detection works even
if Band.us marks the session cookie as HttpOnly in the future.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address eleventh round of code review feedback
- mentions: replace EmptyResultError with SelectorError for missing/
disappeared bell button — produces a clearer SELECTOR error code
- post: assign per-photo filenames using a global index across both
download batches so band-hosted and CDN photos don't overwrite each other
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band): address twelfth round of code review feedback
- post: derive file extension from URL path and include in filename
(e.g. photo_1.jpg) so downloaded photos have correct extensions
- posts: remove dead code guard (!url && !content) — url is always
non-empty here since href-empty posts are already skipped above
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(band/post): use url-scoped getCookies for photo download auth
Domain-scoped getCookies may omit host-only cookies scoped to www.band.us;
using url: 'https://www.band.us' ensures all relevant cookies are included
in the auth header for Band-hosted photo downloads.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(band): add adapter documentation and sidebar entry
Required by CI doc-check --strict: every adapter in src/clis/ must have
a corresponding docs/adapters/browser/*.md file.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test(e2e): wire band auth coverage into default matrix
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(xiaohongshu): use CDP DOM.setFileInputFiles for image upload
Replace base64 DataTransfer injection with CDP DOM.setFileInputFiles,
which lets Chrome read image files directly from the local filesystem.
This eliminates payload size limits that caused "fetch failed" errors
when uploading large images (>500KB) through the browser bridge.
Changes:
- Add 'set-file-input' action to protocol, extension handler, and CDP executor
- Add Page.setFileInput() method for CLI-side usage
- Rewrite publish image upload to use CDP path, with base64 fallback
for older extension versions that don't support the new action
- Add clear warning when falling back to base64 with large payloads
Closes#542 (partially — image upload reliability)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: cover cdp file input upload path
* fix: keep image upload on image-only inputs
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add 知识星球(zsxq) site adapter
Add cookie-based adapter for 知识星球 (zsxq.com) with 5 commands:
- groups: list joined groups
- topics: list topics in current group
- topic: get single topic detail with comments
- search: search topics within a group
- dynamics: latest cross-group activity feed
Uses XHR over Chrome extension (Strategy.COOKIE) to call
https://api.zsxq.com/v2/ APIs with credential forwarding.
* fix(zsxq): map missing topics to not found
* refactor(zsxq): preserve detail response semantics
---------
Co-authored-by: xiaojian <xiaojian@xiaojiandeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Argument/usage errors now correctly exit with code 2 (EX_USAGE) since
the exit-codes feature landed. Update the two affected E2E assertions:
- unknown command → 2 (usage error, not generic failure)
- plugin update without args → 2 (ArgumentError)
* feat(exit-codes): add Unix-standard exit codes to all CliError types
Introduce EXIT_CODES constant table (sysexits.h conventions) and wire
exitCode into every CliError subclass so the process exit code reflects
the semantic type of failure:
0 success (default)
1 generic / unexpected error
2 argument / usage error (ArgumentError)
66 empty result / not found (EmptyResultError, SelectorError)
69 service unavailable (BrowserConnectError, AdapterLoadError)
77 permission / auth required (AuthRequiredError)
78 configuration error (ConfigError)
124 timeout (TimeoutError)
130 Ctrl-C / SIGINT (unchanged, tui.ts)
resolveExitCode() in commanderAdapter.ts reads err.exitCode for typed
CliErrors, and falls back to pattern-matching message text for untyped
adapter errors (auth pattern → 77, not-found pattern → 66, else → 1).
Shell scripts can now distinguish error categories:
opencli spotify status || echo "exit $?" # 69 if browser not running
opencli github issues --repo x 2>/dev/null; [ $? -eq 77 ] && opencli github auth
* fix(exit-codes): address review findings
- TIMEOUT: change from 124 → 75 (EX_TEMPFAIL); 124 is bash timeout(1)'s
own exit code, creating ambiguity when shell runs `timeout 30 opencli`
- SelectorError: change from EMPTY_RESULT(66) → GENERIC_ERROR(1); a
missing DOM selector is an adapter bug, not a user "no data" condition
- normalizeArgValue: throw ArgumentError instead of bare CliError so
invalid bool args correctly exit with USAGE_ERROR(2) not GENERIC_ERROR(1)
- resolveExitCode: explicitly map 'http' classification to GENERIC_ERROR
to keep exit-code path in sync with the render path
- tui.ts: replace hardcoded process.exit(130) with EXIT_CODES.INTERRUPTED
* feat(exit-codes): replace all hardcoded exit numbers with EXIT_CODES constants
Extend the exit code system to cover every process exit point in the codebase.
No magic numbers remain — all exit codes are now referenced by name.
Semantic upgrades beyond pure renaming:
- plugin update missing args → USAGE_ERROR (2) instead of 1
- plugin update conflicting → USAGE_ERROR (2) instead of 1
- opencli install <unknown> → USAGE_ERROR (2) instead of 1
- unknown command fallback → USAGE_ERROR (2) instead of 1
- record with no candidates → EMPTY_RESULT (66) instead of 1
- external CLI install fail → SERVICE_UNAVAIL (69) instead of 1
- daemon EADDRINUSE → SERVICE_UNAVAIL (69) instead of 1
Files touched: cli.ts, external.ts, daemon.ts, main.ts,
clis/antigravity/serve.ts
* feat(sinafinance): rewrite stock as public API adapter
Replace browser-based DOM scraping with direct Sina public APIs:
suggest3.sinajs.cn — symbol search (GBK, no auth)
hq.sinajs.cn — real-time quote (GBK, no auth)
Strategy.PUBLIC, browser: false — no Chrome or login required.
Supports A股 (sh/sz), 港股 (hk prefix), 美股 (gb_ prefix).
US MarketCap parsed from hq field [12]; formatted as T/B/M.
* feat(exit-codes): add Unix-standard exit codes to all CliError types
Introduce EXIT_CODES constant table (sysexits.h conventions) and wire
exitCode into every CliError subclass so the process exit code reflects
the semantic type of failure:
0 success (default)
1 generic / unexpected error
2 argument / usage error (ArgumentError)
66 empty result / not found (EmptyResultError, SelectorError)
69 service unavailable (BrowserConnectError, AdapterLoadError)
77 permission / auth required (AuthRequiredError)
78 configuration error (ConfigError)
124 timeout (TimeoutError)
130 Ctrl-C / SIGINT (unchanged, tui.ts)
resolveExitCode() in commanderAdapter.ts reads err.exitCode for typed
CliErrors, and falls back to pattern-matching message text for untyped
adapter errors (auth pattern → 77, not-found pattern → 66, else → 1).
Shell scripts can now distinguish error categories:
opencli spotify status || echo "exit $?" # 69 if browser not running
opencli github issues --repo x 2>/dev/null; [ $? -eq 77 ] && opencli github auth
* review: regex escape sym, fix change precision, optimize suggest type param
* fix: remove invalid `state: 'normal'` from chrome.windows.create()
Chrome 146+ rejects 'normal' as an invalid value for the `state` parameter
in chrome.windows.create(). This causes the error:
Error: Invalid value for state
Root cause analysis:
- The Chrome Extensions API documentation states that `state` parameter
only accepts 'minimized', 'maximized', and 'fullscreen' as input values
- While WindowState enum includes 'normal', it's meant for reading window
state, not for setting it during creation
- Chrome 146 enforces stricter validation on the `state` parameter
- When `state` is omitted, the window defaults to 'normal' state anyway
Fix: Remove the `state: 'normal'` parameter entirely. The window will
default to normal state without explicitly setting it.
Tested: `opencli doctor` and `opencli bilibili hot` now work correctly
on Chrome 146.0.7680.165.
* build: rebuild dist after removing state: 'normal'
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(extension): probe daemon via HTTP before WebSocket to eliminate console noise
When the daemon is offline, `new WebSocket()` logs uncatchable
ERR_CONNECTION_REFUSED errors to Chrome's extension error page.
Add `probeAndConnect()` that checks daemon reachability with a
silent `fetch(HEAD)` before attempting WebSocket connection.
All three auto-connect paths (initialize, keepalive alarm, eager
reconnect) now go through the probe, eliminating the error noise
entirely.
Closes#505
* refactor(extension): inline probe into connect(), add /ping to daemon
Instead of a separate probeAndConnect() wrapper that all call sites had
to remember to use, bake the HTTP probe directly into connect() itself.
This makes the guard impossible to accidentally skip when adding new
connection paths in the future.
Also adds a dedicated GET /ping endpoint to the daemon (no X-OpenCLI
header required) so the probe has a clear semantic contract instead of
relying on a 403 side-effect from the root path.
- daemon: GET /ping → 200 {ok:true}, no auth needed, placed before the
X-OpenCLI header check; only chrome-extension:// and no-origin
requests reach it (origin check is still enforced above)
- background: connect() is now async; probes /ping with a 1 s timeout
before new WebSocket(); all call sites (initialize, keepalive alarm,
scheduleReconnect) remain unchanged
- probeAndConnect() removed — no longer needed
* fix(extension/daemon): address review feedback on probe refactor
- protocol.ts: replace DAEMON_HTTP_URL with DAEMON_PING_URL (clearer
semantics, single source of truth for the health-check URL)
- background.ts: import DAEMON_PING_URL from protocol instead of
defining a local constant; check res.ok so an unexpected non-200
response doesn't fall through to WebSocket; annotate all fire-and-
forget connect() call sites with `void` to make intent explicit
- daemon.ts: add security comment on /ping documenting the timing
side-channel tradeoff (loopback-only, accepted risk)
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
* docs: move Quick Start before Prerequisites, tone down Electron promo copy
- Reorder sections: Why opencli → Quick Start → Prerequisites
so users see "how to use" before "what you need"
- Replace "The Most Powerful Update Has Arrived!" marketing copy
with a plain one-liner description of the Electron feature
- Remove redundant Bun dev/test commands from Prerequisites
(developer-only content, not relevant to end users)
- Add "(requires Extension)" hint to browser command in Quick Start
* docs: polish Quick Start — one-line source install, Verify setup section
* docs: show 4 sample adapters in Built-in Commands with link to full list
* docs: polish README — Try it out under Verify setup, trim examples, CLI Hub as top-level section
* docs: add CLI Hub intro line in header, restore auto-install note in CLI Hub section
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
* docs: move Quick Start before Prerequisites, tone down Electron promo copy
- Reorder sections: Why opencli → Quick Start → Prerequisites
so users see "how to use" before "what you need"
- Replace "The Most Powerful Update Has Arrived!" marketing copy
with a plain one-liner description of the Electron feature
- Remove redundant Bun dev/test commands from Prerequisites
(developer-only content, not relevant to end users)
- Add "(requires Extension)" hint to browser command in Quick Start
* docs: polish Quick Start — one-line source install, Verify setup section
* docs: show 4 sample adapters in Built-in Commands with link to full list
* docs: polish README — Try it out under Verify setup, trim examples, CLI Hub as top-level section
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
* docs: move Quick Start before Prerequisites, tone down Electron promo copy
- Reorder sections: Why opencli → Quick Start → Prerequisites
so users see "how to use" before "what you need"
- Replace "The Most Powerful Update Has Arrived!" marketing copy
with a plain one-liner description of the Electron feature
- Remove redundant Bun dev/test commands from Prerequisites
(developer-only content, not relevant to end users)
- Add "(requires Extension)" hint to browser command in Quick Start
* docs: polish Quick Start — one-line source install, Verify setup section
* docs: show 4 sample adapters in Built-in Commands with link to full list
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
* docs: move Quick Start before Prerequisites, tone down Electron promo copy
- Reorder sections: Why opencli → Quick Start → Prerequisites
so users see "how to use" before "what you need"
- Replace "The Most Powerful Update Has Arrived!" marketing copy
with a plain one-liner description of the Electron feature
- Remove redundant Bun dev/test commands from Prerequisites
(developer-only content, not relevant to end users)
- Add "(requires Extension)" hint to browser command in Quick Start
* docs: polish Quick Start — one-line source install, Verify setup section
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
* docs: move Quick Start before Prerequisites, tone down Electron promo copy
- Reorder sections: Why opencli → Quick Start → Prerequisites
so users see "how to use" before "what you need"
- Replace "The Most Powerful Update Has Arrived!" marketing copy
with a plain one-liner description of the Electron feature
- Remove redundant Bun dev/test commands from Prerequisites
(developer-only content, not relevant to end users)
- Add "(requires Extension)" hint to browser command in Quick Start
* chore(release): 1.5.2
* test(e2e): stabilize output format checks
* docs: add perf smart-wait design spec (waitForCapture + selector wait + backoff)
* docs: add perf smart-wait implementation plan
* feat(perf): add waitForCaptureJs and waitForSelectorJs to dom-helpers
* feat(perf): extend WaitOptions with selector, add waitForCapture to IPage
* feat(perf): implement waitForCapture() and wait({ selector }) in Page
* feat(perf): implement waitForCapture() and wait({ selector }) in CDPPage
* feat(perf): stepIntercept uses installInterceptor+waitForCapture+getInterceptedRequests
* fix(perf): replace wait(N) with waitForCapture(N) in 7 INTERCEPT adapters
* feat(perf): daemon cold-start uses exponential backoff [50..3000ms]
* fix(perf): replace wait(5) with wait({ selector }) in 15 Twitter UI adapters
* fix(perf): replace wait(N) with wait({ selector }) in medium/substack/bloomberg/sinablog
* fix(types): add waitForCapture to IPage mock helpers in tests
* docs: simplify README to 50-line overview with docs link
Remove redundant command table (already in docs/adapters/index.md).
Keep badges, quick-start, and star history only.
* fix(perf): CDPPage smart wait + MutationObserver selector wait
- CDPPage.wait(N>=1) now uses waitForDomStableJs instead of fixed sleep,
matching Page.wait() behavior and saving unnecessary idle time
- waitForSelectorJs switches from 100ms polling to MutationObserver,
resolving instantly when the target element appears in the DOM
- Update dom-helpers tests to stub MutationObserver for Node eval context
* docs: move built-in commands table to docs/adapters/index.md
Replace the 70-row site/command table in README with a one-line link.
All other README sections (Highlights, Why opencli, Quick Start,
External CLI Hub, Desktop App Adapters, Download, Plugins, AI Agents,
Troubleshooting) are preserved unchanged.
chrome.windows.create rejects state:'minimized' when combined with
width/height (Chrome API constraint). Revert to state:'normal' to fix
the "Invalid value for state" error. The 30s idle timeout from #521
is preserved.
Fixes#526
* refactor: slim CI matrix, extract shared utils, unify logging, remove __test__ from public API
- CI: unit-test uses dynamic matrix (PR=ubuntu+22 only, push=full 3OS×2Node);
adapter-test reduced to ubuntu-latest (OS doesn't affect pure unit tests)
- _shared/common.ts: add sleep() and clampToRange() shared adapter utilities;
douban/utils.ts and sinablog/utils.ts now use clampToRange instead of duplicate clampLimit
- browser/daemon-client.ts: replace inline setTimeout Promise with local sleep()
- execution.ts: replace conditional console.error with log.debug
- browser/index.ts: remove __test__ from public barrel export;
browser.test.ts now imports internal helpers directly from source files
* fix: remove unused afterEach import, fix schedule/dispatch CI matrix, clarify clampToRange docs
* refactor: move sleep to src/utils.ts, simplify clamp signature to match lodash convention
* perf: smart pre-navigation — skip redundant domain nav + remove hardcoded 2s wait
- Add `getCurrentUrl()` to IPage, Page, and CDPPage to check current browser URL
- Skip pre-navigation entirely if the browser is already on the target domain
- Remove the hardcoded `page.wait(2)` after pre-navigation — `page.goto()` already
includes smart DOM-settle detection via `waitForDomStable`, making the fixed
2-second sleep redundant
- Saves ~2s per browser command in the common case (consecutive commands on the
same site), and ~1-2s even on cold navigation
* perf: smart page.wait() — DOM-stable early return for waits >= 1s
For page.wait(N) where N >= 1 second, use DOM MutationObserver-based
stability detection instead of a fixed sleep. The original wait time
becomes a hard cap, but the call returns as soon as the DOM stops
mutating (500ms quiet period).
This benefits ~200 hardcoded sleep calls across ~40 adapters without
changing any adapter code. A typical page.wait(5) now completes in
<1s when the page is already stable, instead of always waiting 5s.
Short waits (< 1s) are kept as fixed sleeps — these are typically
UI animation delays or anti-bot throttling where DOM-ready is irrelevant.
* refactor: getCurrentUrl() uses in-memory tracking instead of round-trip
Replace the sendCommand('exec', 'window.location.href') call with a
local _lastUrl field set during goto(). This eliminates a daemon HTTP
round-trip for the domain check, making isAlreadyOnDomain() zero-cost.
On fresh tabs (about:blank), _lastUrl is null so we correctly fall
through to navigation — no special-casing needed.
* fix(browser): retry settle probe after SPA client-side redirect
SPA sites like creator.xiaohongshu.com can trigger a client-side
redirect after chrome.tabs reports status 'complete', invalidating
the CDP target. The waitForDomStable probe in page.goto() was
unprotected, causing -32000 "Inspected target navigated or closed".
Wrap the settle probe in try/catch with a single 200ms-delayed retry,
consistent with the existing stealth injection error handling pattern.
The retry gives the SPA redirect time to complete, while the outer
catch ensures settle failure never crashes goto() since navigation
itself already succeeded.
Closes#502
* review: narrow settle retry to target redirects
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* test(e2e): accept current apple podcasts fetch errors
* fix(ci): stabilize plugin and public command checks
---------
Co-authored-by: pi-dal <hi@pi-dal.com>
* feat: zero onboarding, extension version check, and update notifier
- Fail-fast guard in execution.ts: when daemon is running but extension
is not connected, immediately surface a setup guide instead of waiting
for the 30s connect timeout
- Extension version handshake: extension sends `hello` with its version
on WebSocket connect; daemon stores it and exposes via /status; CLI
warns on mismatch in both execution path and `opencli doctor`
- `opencli doctor` now shows extension version inline and reports
version mismatch as an actionable issue
- Non-blocking npm update checker: registers a process exit hook so the
update notice appears after command output (same pattern as npm/gh/yarn);
background fetch writes to ~/.opencli/update-check.json for next run
- postinstall: print Browser Bridge setup instructions after shell
completion install for first-time global install users
Bug fixes caught in review:
- discover.ts: add AbortController timeout to checkDaemonStatus() fetch,
move clearTimeout after res.json() to cover body streaming
- daemon.ts: clear extensionVersion and reject pending requests in
ws.on('error') handler, not just ws.on('close')
- update-check.ts: skip update notice when process exits with non-zero
code; read cache once at module load to avoid double disk I/O;
guard isNewer() against NaN from pre-release version strings
* fix: relax extension version check to major-only in doctor, remove from hot path
* test: enable all adapter tests via wildcard glob, fix apple-podcasts url field
* fix: clearTimeout in finally block, reset extensionVersion on reconnect, fix e2e regex
- Create automation window with `state: 'minimized'` so it never
appears in the user's taskbar or steals visual attention
- Reduce idle timeout from 120s to 30s — window closes quickly after
the last command finishes, instead of lingering for 2 minutes
- CDP debugger works fine on minimized windows, no functional impact
Fixes the user-visible issue of a blank data:text/html tab appearing
during command execution.
* perf: smart pre-navigation — skip redundant domain nav + remove hardcoded 2s wait
- Add `getCurrentUrl()` to IPage, Page, and CDPPage to check current browser URL
- Skip pre-navigation entirely if the browser is already on the target domain
- Remove the hardcoded `page.wait(2)` after pre-navigation — `page.goto()` already
includes smart DOM-settle detection via `waitForDomStable`, making the fixed
2-second sleep redundant
- Saves ~2s per browser command in the common case (consecutive commands on the
same site), and ~1-2s even on cold navigation
* perf: smart page.wait() — DOM-stable early return for waits >= 1s
For page.wait(N) where N >= 1 second, use DOM MutationObserver-based
stability detection instead of a fixed sleep. The original wait time
becomes a hard cap, but the call returns as soon as the DOM stops
mutating (500ms quiet period).
This benefits ~200 hardcoded sleep calls across ~40 adapters without
changing any adapter code. A typical page.wait(5) now completes in
<1s when the page is already stable, instead of always waiting 5s.
Short waits (< 1s) are kept as fixed sleeps — these are typically
UI animation delays or anti-bot throttling where DOM-ready is irrelevant.
* refactor: getCurrentUrl() uses in-memory tracking instead of round-trip
Replace the sendCommand('exec', 'window.location.href') call with a
local _lastUrl field set during goto(). This eliminates a daemon HTTP
round-trip for the domain check, making isAlreadyOnDomain() zero-cost.
On fresh tabs (about:blank), _lastUrl is null so we correctly fall
through to navigation — no special-casing needed.
Extends parseSource() to accept any git-cloneable URL, not just GitHub:
- ssh://git@host/path/repo.git
- git@host:user/repo.git (SCP-style)
- https://any-host.com/path/repo.git
GitHub shorthand (github:user/repo) and local paths continue to work.
Updated error messages, CLI description, docs, and added 7 new unit tests.
Closes#492
When a TS plugin is installed but esbuild is unavailable or transpilation
fails silently, the plugin discovery would attempt to import() the raw
.ts file, causing 'Unknown file extension .ts' on production Node.js.
Changes:
- discovery.ts: Skip raw .ts import when no compiled .js exists; show
an actionable warning guiding the user to re-transpile or install esbuild
- plugin.ts: Upgrade esbuild-not-found from debug to warn level; log
the outer catch error instead of silently swallowing it
Closes#500
Non-browser commands (`browser: false`) ran without any timeout
protection, even when `timeoutSeconds` was explicitly set. This wraps
the non-browser execution path with `runWithTimeout()` when the
adapter defines a positive `timeoutSeconds`.
Also adds an optional `hint` parameter to `TimeoutError` so the
non-browser path shows a relevant suggestion instead of the
browser-specific `OPENCLI_BROWSER_COMMAND_TIMEOUT` env var hint.
Bluesky (9 commands, public AT Protocol API, no auth needed):
- profile: user profile info (followers, following, posts)
- user: recent posts from a user with engagement stats
- trending: trending topics on Bluesky
- search: search users
- feeds: popular feed generators
- followers: list user's followers
- following: list accounts a user follows
- thread: post thread with replies
- starter-packs: user's starter packs
All commands use the public Bluesky API, no browser or login required.
* fix(plugin): handle EXDEV cross-filesystem rename during install
fs.renameSync() fails with EXDEV when source and destination are on
different filesystem mount points. This commonly happens because plugin
clones land in os.tmpdir() (often /tmp on a tmpfs) while plugins are
installed to ~/.opencli/plugins/ (on the root filesystem).
Add a moveDir() helper that catches EXDEV and falls back to
fs.cpSync() + fs.rmSync(). Applied to both single-plugin and monorepo
install paths.
* review: clean up failed EXDEV fallback installs
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Derive approximate publish date from note IDs, which follow MongoDB
ObjectID format (first 8 hex chars = Unix timestamp). Exported as a
pure function with UTC+8 offset for China timezone.
Closes#484
- Parallelize file scanning in discoverClisFromFs and discoverPluginDir
using Promise.all(files.map(async ...)) instead of serial for-of with
await, so isCliModule checks run concurrently
- Parallelize plugin directory scanning in discoverPlugins
- Cache loadExternalClis() result to avoid re-parsing YAML on every call
- Invalidate cache in registerExternalCli after writing to disk
- Cache strategyLabel() call in list command to avoid redundant computation
- Add comment explaining why discovery must remain sequential (plugin override semantics)
Remove guide.json API path that returned data inconsistent with what
users see on the page (#463). Use semantic caret button detection
via data-testid instead of position-based heuristics, and validate
post count text contains digits before displaying.
* fix(plugin): detect symlinked monorepo sub-plugins in discoverPlugins
discoverPlugins() used entry.isDirectory() to filter plugin directories,
but monorepo sub-plugins are installed as symlinks pointing into
~/.opencli/monorepos/. On most Node.js versions, isDirectory() returns
false for symlinks, causing monorepo plugin commands to be silently
skipped during discovery.
Add entry.isSymbolicLink() check so symlinked plugin directories are
properly discovered and their commands registered.
* fix(plugin): skip broken symlink discovery
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(xiaohongshu): adapt publish to new two-step creator center UI (#460)
The creator center now requires image upload before showing the
title/content editor form. This caused the publish command to fail
with "Could not find title input".
- Add waitForEditForm() to poll for editor after image upload
- Extract TITLE_SELECTORS constant shared by waitForEditForm and fillField
- Add contenteditable title selectors for new UI
- Make images required (new UI mandates images before editor)
- Update draft button to match both '暂存离开' and '存草稿'
- Exclude title placeholder from content fallback selector
- Update tests to match new flow
* refactor(xiaohongshu): clarify publish surface states
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Add support for installing plugins from local directories:
opencli plugin install file:///path/to/my-plugin
opencli plugin install /path/to/my-plugin
Local plugins are symlinked (not copied) into ~/.opencli/plugins/
so code changes are reflected immediately without reinstall — ideal
for plugin development workflows.
Changes:
- parseSource() now handles file:// URLs and bare absolute paths
- New installLocalPlugin() creates symlink + installs deps + transpiles
- Lock file records 'local:<path>' as source for local plugins
- 6 new test cases for local path parsing and install behavior
Remove the module-level LOCK_FILE and MONOREPOS_DIR constants that were
computed at load time using os.homedir(). These ignored the HOME
environment variable, causing path mismatches when tests use HOME for
isolation.
All usages now go through getLockFilePath() and getMonoreposDir() which
respect process.env.HOME. Updated plugin.test.ts accordingly.
* feat: zero onboarding, extension version check, and update notifier
- Fail-fast guard in execution.ts: when daemon is running but extension
is not connected, immediately surface a setup guide instead of waiting
for the 30s connect timeout
- Extension version handshake: extension sends `hello` with its version
on WebSocket connect; daemon stores it and exposes via /status; CLI
warns on mismatch in both execution path and `opencli doctor`
- `opencli doctor` now shows extension version inline and reports
version mismatch as an actionable issue
- Non-blocking npm update checker: registers a process exit hook so the
update notice appears after command output (same pattern as npm/gh/yarn);
background fetch writes to ~/.opencli/update-check.json for next run
- postinstall: print Browser Bridge setup instructions after shell
completion install for first-time global install users
Bug fixes caught in review:
- discover.ts: add AbortController timeout to checkDaemonStatus() fetch,
move clearTimeout after res.json() to cover body streaming
- daemon.ts: clear extensionVersion and reject pending requests in
ws.on('error') handler, not just ws.on('close')
- update-check.ts: skip update notice when process exits with non-zero
code; read cache once at module load to avoid double disk I/O;
guard isNewer() against NaN from pre-release version strings
* fix: reduce fail-fast timeout to 300ms and guard stderr.write in exit hook
* fix(doctor): remove unused fix option and add release URL to extension install hint
* fix(e2e): update BrowserBridge unavailable detection regex to match current error format
* feat: smart error dispatch with inline Browser Bridge diagnosis
- BrowserConnectError: runs checkDaemonStatus() on failure, shows real-time
daemon/extension status and specific fix steps instead of a static hint
- AuthRequiredError: domain-specific login guidance
- TimeoutError: shows exact env var override command
- SelectorError/EmptyResultError: flags adapter as potentially outdated,
links to debug command and issue tracker
- Generic untyped errors (164 in adapters): pattern-classified into
auth/http/not-found/other with tailored guidance per category
- BrowserConnectError gains a `kind` field for future dispatch
- Added 6 new error icons (COMMAND_EXEC, ADAPTER_LOAD, NETWORK, etc.)
- Updated test: invalid bool now rejected eagerly in commanderAdapter
* fix: review fixes for smart error dispatch
- checkDaemonStatus: add { timeout: 300 } to match execution.ts behavior,
avoids 2s wait on an already-failed path
- catch block: use named _statusErr variable; fall back to kind-derived
state (running/extensionConnected inferred from BrowserConnectError.kind)
instead of re-accessing outer err.hint ambiguously
- Extract renderBridgeStatus() helper to share logic between real-time
and kind-derived fallback paths
- AuthRequiredError: use err.hint when set, respecting adapter-supplied
hints; fall back to generic domain-based guidance
- HTTP regex: broaden from 'http [45]xx' to also match 'status: 404',
bare '404', 'status 500', etc. — avoids false negatives
* feat: zero onboarding, extension version check, and update notifier
- Fail-fast guard in execution.ts: when daemon is running but extension
is not connected, immediately surface a setup guide instead of waiting
for the 30s connect timeout
- Extension version handshake: extension sends `hello` with its version
on WebSocket connect; daemon stores it and exposes via /status; CLI
warns on mismatch in both execution path and `opencli doctor`
- `opencli doctor` now shows extension version inline and reports
version mismatch as an actionable issue
- Non-blocking npm update checker: registers a process exit hook so the
update notice appears after command output (same pattern as npm/gh/yarn);
background fetch writes to ~/.opencli/update-check.json for next run
- postinstall: print Browser Bridge setup instructions after shell
completion install for first-time global install users
Bug fixes caught in review:
- discover.ts: add AbortController timeout to checkDaemonStatus() fetch,
move clearTimeout after res.json() to cover body streaming
- daemon.ts: clear extensionVersion and reject pending requests in
ws.on('error') handler, not just ws.on('close')
- update-check.ts: skip update notice when process exits with non-zero
code; read cache once at module load to avoid double disk I/O;
guard isNewer() against NaN from pre-release version strings
* fix: reduce fail-fast timeout to 300ms and guard stderr.write in exit hook
* feat(twitter): add time column to search output
Extract created_at from tweet data and format as ISO datetime.
This helps users filter tweets by recency during monitoring.
Closes#465
* refactor(twitter): align search timestamp field with created_at
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(imdb): add IMDb adapter with 6 commands
Add a public IMDb adapter using browser-based JSON-LD and __NEXT_DATA__
extraction. All commands use Strategy.PUBLIC with browser: true.
Commands:
- imdb search <query> — search movies, TV shows, and people
- imdb title <id> — get movie/show details (Movie, TVSeries, TVEpisode, TVMiniseries, TVMovie, etc.)
- imdb top — IMDb Top 250 chart
- imdb trending — Most Popular Movies
- imdb person <id> — actor/director info with filmography
- imdb reviews <id> — user reviews (first page, max 25)
Shared utils: ID normalization, ISO 8601 duration formatting, locale
forcing, JSON-LD extraction (supports type array filtering), and
anti-bot challenge detection.
* review: harden imdb adapter loading and tests
* test: unblock PR CI on merge head
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add bilibili/comments, xiaohongshu/comments, and rate-limiter plugin docs
- bilibili/comments: fetch top-level replies via /x/v2/reply/main with WBI signing
(bvid → aid resolution + signed params, no DOM dependency)
- xiaohongshu/comments: DOM extraction from note detail page with login-wall detection
and correct handling of 0-like counts (XHS shows "赞" text instead of "0")
- docs/advanced/rate-limiter-plugin.md: documents the onAfterExecute hook pattern
and shows a plug-and-play rate limiter that adds random sleep between platform
commands to reduce bot-detection risk
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(xiaohongshu): allow empty comments results
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(runtime): add runtime detection utility for Bun/Node.js
Add runtime-detect.ts module that detects whether opencli is running
under Bun or Node.js via globalThis.Bun check. Includes helper
functions for version string and label formatting.
Add corresponding unit tests that work correctly under both runtimes.
* feat(runtime): integrate Bun runtime support into CLI tooling
- doctor: show runtime label (e.g. 'node v22.13.0') in diagnostic output
- package.json: add dev:bun, start:bun, test:bun convenience scripts
- E2E helpers: support OPENCLI_TEST_RUNTIME env var for runtime selection
* ci: add Bun compatibility test job and document runtime support
- ci.yml: add bun-test job using oven-sh/setup-bun@v2
- README.md: update Prerequisites to mention Bun, add Runtime Support
section with usage examples for dev:bun, start:bun, test:bun
* ci: pin Bun version in compatibility job
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(producthunt): add Product Hunt CLI adapter
Add three commands:
- posts: RSS feed with optional category filter
- today: latest day's posts from feed
- hot: today's top posts with vote counts (browser INTERCEPT strategy)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(producthunt): add browse command for category best products
Browse top-rated products in any Product Hunt category (e.g. vibe-coding,
ai-agents, developer-tools) with name, tagline, and review count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(producthunt): add adapter documentation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(producthunt): rebase on main and stabilize selectors
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(ci): include popup assets in extension release
Copy popup assets into the packaged Chrome extension zip and validate that manifest-referenced files exist before publishing the artifact.
Co-authored-by: Codex <noreply@openai.com>
* fix: restore executable permission on bin entries after tsc build (#446) (#452)
tsc does not preserve the +x bit when compiling, so after clean-dist
removes dist/ and tsc regenerates it, dist/main.js loses its executable
permission. This causes 'Permission denied' when users run 'npm run build'
in the installed directory.
Fix: read bin entries from package.json at the end of build-manifest and
chmod 0o755 them (skipped on Windows). Wrapped in try/catch so it never
breaks the build.
Closes#446
* fix: correct positional arg usage in tests (#449)
* fix yahoo-finance quote e2e invocation
* fix positional args in v2ex topic tests
* fix(ci): script extension release packaging
---------
Co-authored-by: Codex <noreply@openai.com>
Co-authored-by: jakevin <jakevingoo@gmail.com>
Co-authored-by: pi-dal <hi@pi-dal.com>
* fix(xiaohongshu): improve image-text publish flow
Match visible 图文 tab labels instead of relying on narrow class selectors, fail early when the page is still on the video publish surface, and avoid injecting images into a generic file input. Add regression coverage for the image-text tab flow and the video-page failure case.
* test(xiaohongshu): include publish tests in adapter project
* fix(xiaohongshu): wait for image-text surface before upload
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
tsc does not preserve the +x bit when compiling, so after clean-dist
removes dist/ and tsc regenerates it, dist/main.js loses its executable
permission. This causes 'Permission denied' when users run 'npm run build'
in the installed directory.
Fix: read bin entries from package.json at the end of build-manifest and
chmod 0o755 them (skipped on Windows). Wrapped in try/catch so it never
breaks the build.
Closes#446
* feat(linux-do): refactor adapters with unified feed, tags, user commands
- Replace hot/latest/category with unified `feed` command (tag/category/view routing)
- Add `tags`, `user-topics`, `user-posts` commands
- Add static data files for categories and tags lookup
- Fix error handling: use CliError subclasses instead of raw Error
- Fix Discourse API field mapping in search (tags, created)
- Add strategy: cookie to all YAML adapters
- Update docs and README command listings
- Update E2E tests for new command signatures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* review: resolve linux-do feed from live metadata
* fix: restore linux-do CI
* fix: harden linux-do compatibility
* refactor: stabilize linux-do command migration
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(chatgpt): add model/mode selection and fix response polling
Add --model option to ask and send commands, and a new standalone
model command for switching ChatGPT Desktop models via Accessibility API.
Supported models: auto, instant, thinking, 5.2-instant, 5.2-thinking.
Changes:
- ax.ts: add AX_MODEL_SCRIPT (opens Options popover, searches within
AXPopover to avoid matching sidebar items, supports legacy models
submenu) and AX_GENERATING_SCRIPT (detects "Stop generating" button)
- ask.ts: add --model flag; fix polling to wait for generation to
complete instead of returning partial/thinking intermediate text
- send.ts: add --model flag
- model.ts: new standalone command to switch model/mode
* review: activate chatgpt before model selection
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
- Show helpful hint in popup when disconnected: "This is normal. The
extension connects automatically when you run any opencli command."
- Stop eager reconnect after 6 attempts (reaching 60s backoff) to
reduce ERR_CONNECTION_REFUSED noise in console; keepalive alarm
still retries every ~24s at low frequency.
Split browser-public.test.ts: core sites (bilibili, zhihu, v2ex) run
by default; all other 20+ site tests moved to browser-public-extended
and gated behind OPENCLI_E2E=1 to prevent AI agents from launching
dozens of browser instances.
brew install gws installs a git workspace manager, not Google
Workspace CLI. The npm package @nicholasgasior/gws doesn't exist
either. Remove the misleading entry entirely.
Weibo: add feed, me, user, post, comments commands with cookie-based
auth and proper AuthRequiredError handling.
YouTube: add channel info and video comments via InnerTube API.
Also remove internal source references from file headers.
* feat(tiktok): add video URL to search results
Add a 'url' field to the TikTok search adapter output, constructed from
the author's uniqueId and the video id returned by the API. This allows
downstream consumers (AI agents, pipelines, scripts) to link directly to
each video instead of only having the author handle.
The URL format is: https://www.tiktok.com/@{author}/video/{videoId}
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add url field to 9 search adapters missing it
Add url output to search commands that were missing direct links:
YAML adapters:
- hackernews: surface existing url from map step into columns
- zhihu: pass computed url through map step into columns
- linux-do: construct url from topic id
- instagram: construct profile url from username
- xueqiu: pass computed url through map step into columns
TS adapters:
- arxiv: surface existing url from parseEntries into return + columns
- apple-podcasts: add collectionViewUrl from iTunes API
- medium: add url to columns (already computed in utils)
- weread: construct book url from bookId
This brings search adapter url coverage from 67% to 97% (32/33).
The only adapter without url is dictionary (word lookup, no URL concept).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(weread): use query arg in search
---------
Co-authored-by: Allen Song (Beyondsoft) <v-songjun@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
- Add popup.html/popup.js showing daemon connection status
(Connected / Reconnecting / No daemon connected)
- Add message listener in background.ts to expose WebSocket state
- Add PRIVACY.md with full privacy policy covering all permissions
- Add content_security_policy to manifest.json
- Update description to be clearer for CWS reviewers
* ci: add cross-platform support for E2E and smoke tests
Make headed browser tests (E2E and smoke) runnable on Linux, macOS,
and Windows:
- setup-chrome action: only install xvfb on Linux (macOS/Windows
have native GUI sessions and don't need a virtual display)
- e2e-headed.yml: add OS matrix, use xvfb-run wrapper only on Linux
- ci.yml smoke-test: add OS matrix, use xvfb-run wrapper only on Linux
The browser-actions/setup-chrome action already supports all three
platforms natively.
* ci: exclude Windows from E2E/smoke matrix (Chrome install hangs)
browser-actions/setup-chrome hangs indefinitely during Chrome MSI
installation on Windows runners (observed 10+ min with no progress).
This is a known limitation of Windows CI runners.
Keep Linux + macOS for headed browser tests. Windows is still covered
by build, unit-test, and adapter-test jobs.
* fix: pre-release cleanup — bugs, version sync, and error handling
Bug fixes:
- Fix hasLimit using wrong Set (SEARCH_PARAMS → LIMIT_PARAMS) in
analysis.ts classifyQueryParams
- Remove phantom scroll step from BROWSER_STEPS and KNOWN_STEP_NAMES
(declared but never registered, causes runtime crash if used in YAML)
- Add missing download step to KNOWN_STEP_NAMES (was producing
false-positive validation warnings)
Docs:
- Sync version numbers: SKILL.md, extension/package.json,
extension/manifest.json → 1.3.3
- Add jd, web to README command tables (both EN and zh-CN)
- Update xueqiu commands with fund-holdings, fund-snapshot
Code quality:
- Replace all 22 catch (err: any) with typed error handling using
existing getErrorMessage() utility across 13 files
* fix: remove (err as any) casts in error handling
- antigravity/serve.ts: use typed Error.cause instead of (err as any).cause
- external.ts: move instanceof guard into shouldRetryWithCmdShim,
accept unknown instead of forcing NodeJS.ErrnoException cast at call site
* fix(extension): security hardening — tab isolation, URL validation, cookie scope
Addresses issues raised in #399 (Astro-Han's community triage):
1. Tab isolation bypass: resolveTabId now verifies that an explicit tabId
belongs to the automation window (tab.windowId === session.windowId)
before accepting it. Tabs from the user's browsing session are rejected.
2. URL scheme allowlist: isDebuggableUrl switched from a blocklist
(chrome://, chrome-extension://) to an allowlist (http://, https:// only).
handleNavigate and tabs.new also reject non-http(s) URLs early, blocking
file://, javascript:, and data: scheme abuse.
3. Cookie scope restriction: handleCookies now requires domain or url.
Requests with neither are rejected instead of dumping all browser cookies.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(extension): resolve data: URI vs allowlist conflict, plug tabs.select bypass
- Add BLANK_PAGE constant and whitelist it in isDebuggableUrl so
internal blank tabs are not treated as non-debuggable after the
blocklist-to-allowlist change.
- Add isSafeNavigationUrl for user-facing URL validation (http/https
only), keeping it separate from internal isDebuggableUrl.
- Fix tabs.select to verify tab belongs to automation window before
activating, closing a tab isolation bypass.
- Normalize error message style (-- instead of em dash).
* fix(extension): add try-catch for tabs.select with explicit tabId
Gracefully handle the case where cmd.tabId points to a closed tab
instead of letting the unhandled exception bubble up.
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Replace manual < > comparison with localeCompare({ numeric: true })
so string-encoded numbers (e.g. "99" vs "1000") sort correctly
without requiring an explicit flag. This is a one-line fix that
makes sort just work for all YAML authors.
Co-authored-by: jackwener <jakevingoo@gmail.com>
* chore: fix pre-existing biome lint in template.ts
- isNaN → Number.isNaN (2 occurrences)
- string concatenation → template literal
- biome-ignore for intentional control chars in sanitize regex
* fix(pipeline): evaluate chained || in template engine (#303)
The || handler in evalExpr returned the right side as a literal string
instead of recursively evaluating it. This broke chained fallbacks like
`item.a || item.b || 'default'` — when item.a was falsy, the entire
`item.b || 'default'` was returned as text.
Fix: call evalExpr on the right side so chained || works at any depth.
* perf(pipeline): fast-path string literals in evalExpr to skip VM
When the right side of || is a quoted string like 'N/A', detect it
with a simple regex and return directly instead of falling through
to evalJsExpr which spins up a node:vm sandbox.
* refactor(pipeline): simplify evalExpr by removing hand-rolled operator parsing
Replace the manual regex-based || and arithmetic handlers with a
streamlined flow: pipe filters → fast-path literals → resolvePath →
evalJsExpr (VM). The VM already handles ||, ??, arithmetic, ternary,
etc. natively, so reimplementing them with regex was redundant and
bug-prone (see issue #303).
Key improvements:
- Fix pipe | vs || disambiguation with lookbehind/lookahead regex
(?<!|)|(?!|) so "item.a || item.b | upper" works correctly
- Remove ~20 lines of manual operator handling
- Add numeric literal fast path
- Pipe filter handler now uses evalExpr recursively (not just
resolvePath), enabling filters on complex expressions
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(xueqiu): add danjuan fund account commands
* refactor(xueqiu): convert danjuan fund YAML adapters to TS
- Replace 3 YAML files with 4 TS files (shared utils + 3 commands)
- Extract shared helpers: fetchDanjuanApi, fetchAssetGain, collectHoldings
- Fix double-navigation by using navigateBefore instead of pipeline navigate
- Unify error messages to English with Hint pattern
- Mask real account ID in docs example
- Add explicit default for --account arg
* refactor(xueqiu): optimize danjuan fund adapters
- Single page.evaluate with Promise.all for parallel account fetching
(1 browser round-trip instead of N+1)
- Merge fund-accounts into fund-holdings (account info visible per row)
- 3 files: danjuan-utils.ts (shared), fund-holdings.ts, fund-snapshot.ts
- Strong TypeScript interfaces for all data shapes
- Update docs to reflect 2-command design
* fix(xueqiu): preserve danjuan pre-navigation metadata
* fix(xueqiu): fail on incomplete danjuan snapshots
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(tiktok): add video URL to search results
Add a 'url' field to the TikTok search adapter output, constructed from
the author's uniqueId and the video id returned by the API. This allows
downstream consumers (AI agents, pipelines, scripts) to link directly to
each video instead of only having the author handle.
The URL format is: https://www.tiktok.com/@{author}/video/{videoId}
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: guard against empty uniqueId/id producing invalid URL
When uniqueId or id is missing, return empty string instead of
a malformed URL like "https://www.tiktok.com/@/video/".
---------
Co-authored-by: Allen Song (Beyondsoft) <v-songjun@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* chore: ignore worktree directory
* fix(pipeline): check HTTP status in fetch step
* fix(pipeline): align fetch error semantics
* fix(pipeline): use CliError and add warn logging in fetch step
- Replace bare Error with CliError('FETCH_ERROR') for consistent CLI output
- Return error status from browser evaluate instead of throwing inside it
- Add log.warn() for batch item failures in both browser and non-browser paths
* chore: remove unrelated .worktrees/ from .gitignore
* refactor(fetch): use getErrorMessage(), unify sentinel naming to __httpError
- Use project's existing getErrorMessage() utility instead of manual instanceof checks
- Rename sentinel from __fetchError to __httpError for consistency with other adapters
- Simplify sentinel structure (url already available in outer scope, no need to pass through evaluate)
- Add comment explaining why getErrorMessage() can't be used inside evaluate()
- Add comment explaining CDP error message rewriting behavior
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(pixiv): add Pixiv adapter with 6 commands
Add support for Pixiv (pixiv.net) with the following commands:
- ranking: daily/weekly/monthly illustration rankings
- search: search illustrations by keyword/tag
- user: view artist profile info
- illusts: list illustrations by artist
- detail: view illustration details (tags, stats)
- download: download original-quality images
All commands use COOKIE strategy to reuse Chrome's logged-in session.
YAML adapters for simple API fetches (ranking, detail, user), TypeScript
for complex logic (search, illusts, download with Referer header).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test(pixiv): add unit tests and E2E auth failure tests
- search.test.ts: auth error, result parsing, limit, empty results (4 tests)
- illusts.test.ts: auth error, empty user, two-step fetch, limit (4 tests)
- download.test.ts: auth error, no images, Referer header, partial failure (4 tests)
- Add pixiv to vitest adapter project include list
- Add 5 pixiv commands to E2E browser-auth graceful failure tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(pixiv): correct ranking API path and YAML arg naming
- ranking: use /ranking.php?format=json (not /ajax/ranking which 404s)
- ranking: fix JSON path from data.body.contents to data.contents
- user/detail: rename hyphenated args (user-id → uid, illust-id → id)
to fix YAML template evaluation (dot access doesn't support hyphens)
All 6 commands verified working against live Pixiv API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(pixiv): use JSON.stringify to prevent code injection in page.evaluate
Address CodeRabbit review: all user inputs (query, userId, illustId,
idsParam) passed to page.evaluate are now serialized via JSON.stringify
instead of direct string interpolation, preventing code injection in
browser context.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(pixiv): address code review feedback
- ranking.yaml: add | json filter to page/limit args for defense-in-depth
- user.yaml: guard illusts/manga/novels with typeof check for robustness
- Extract shared createPageMock to test-utils.ts, deduplicate across 3 test files
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(pixiv): use minimal page mock and add download E2E test
- test-utils.ts: slim down to minimal mock (goto, evaluate, getCookies)
with overrides support, matching upstream's pragmatic mock style
- Add missing download command to E2E browser-auth graceful failure tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(pixiv): address all remaining CodeRabbit review comments
- detail.yaml: add url to columns to match description mentioning "URLs"
- All adapters: differentiate HTTP errors — 401/403 → AuthRequiredError,
404 → "not found", others → generic "request failed (HTTP N)"
- Tests: use beforeAll to cache registry lookup, avoiding repeated reads
from global singleton
- Tests: assert error type (AuthRequiredError) not just message content
- Tests: add dedicated test cases for non-auth errors (500) and 404
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(pixiv): add adapter docs and indexes
- Add pixiv.md documentation page under docs/adapters/browser/
- Update docs/adapters/index.md with pixiv entry
- Add Pixiv to sidebar in docs/.vitepress/config.mts
- Update README.md and README.zh-CN.md adapter tables
- Add pixiv to download support tables in both READMEs
Completes the documentation checklist for the pixiv adapter PR.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(pixiv): address code review findings
- Use CommandExecutionError instead of raw Error for HTTP failures
- Add page.goto() before page.evaluate() to establish browser context
- Fix search keyword double-encoding in URL construction
- Fix ranking.yaml using rating_count instead of illust_bookmark_count
- Throw on batch detail fetch failure instead of silent empty return
- Add beforeEach mock reset in download tests
- Add novels column to user.yaml output
Ensures pixiv adapter follows upstream CliError conventions and handles
edge cases correctly before submitting to upstream.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(pixiv): improve download description in READMEs
- Replace technical Referer header detail with user-facing description
- Describe what users care about: original quality and multi-page support
Technical details belong in code comments, not user-facing docs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(pixiv): expand usage examples with all options
- Add ranking mode examples including R18 variants
- Add search filter examples (mode, order, pagination)
- Organize examples by command category for readability
Users need to know available options without reading source code.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(pixiv): address second round of CodeRabbit review comments
- Validate illust-id is numeric to prevent path traversal
- Move URL parsing inside per-item try block for graceful error handling
- Add auth error handling for batch detail request (consistent with step 1)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(pixiv): extract shared pixivFetch helper, add input validation & batch support
- Create utils.ts with pixivFetch() for unified navigate + fetch + error handling
- Refactor search.ts, illusts.ts, download.ts to use pixivFetch (DRY)
- Add user-id/illust-id numeric validation in TS adapters
- Add batch pagination in illusts.ts for limit > 48 (Pixiv server limit)
- Add comment explaining Pixiv search API dual keyword requirement
- Update tests: new invalid-ID test cases, aligned mock format with pixivFetch
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* ci: add cross-platform matrix (Linux/macOS/Windows) to build, unit-test, adapter-test
Add OS matrix with ubuntu-latest, macos-latest, and windows-latest to
the build, unit-test, and adapter-test CI jobs. This ensures cross-
platform compatibility is verified on every push and PR.
Smoke tests remain Linux-only due to xvfb dependency.
Relates to #392 (Windows plugin path issues).
* test: replace hardcoded /tmp with os.tmpdir() for Windows compatibility
Fix Windows CI failures caused by hardcoded '/tmp' paths that don't
exist on Windows. Use os.tmpdir() which returns the correct platform-
specific temp directory on all operating systems.
Files fixed:
- src/engine.test.ts: 3 occurrences (mkdtemp, discoverClis path)
- src/plugin.test.ts: 2 occurrences (getCommitHash test, mock condition)
* test: fix remaining Windows path issues in test files
- engine.test.ts: use pathToFileURL().href for dynamic import paths
(path.join produces backslashes on Windows, breaking ES module imports)
- download.test.ts: replace hardcoded '/tmp' with os.tmpdir() + path.join
* fix(plugin): resolve Windows path and symlink issues
- Replace `new URL(import.meta.url).pathname` with `fileURLToPath()` from
node:url — the former returns `/C:/Users/...` on Windows (leading slash
before drive letter), breaking path resolution for host linking and
esbuild binary lookup.
- Use junction (`'junction'`) instead of directory symlink (`'dir'`) on
Windows in linkHostOpencli — junctions don't require admin privileges,
while `fs.symlinkSync(..., 'dir')` does on Windows.
- Use `where` instead of `which` on Windows for global esbuild lookup.
All changes are platform-conditional and preserve existing Unix behavior.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(plugin): additional Windows fixes found during UAT
- npm execFileSync needs shell:true on Windows (.cmd wrapper)
- esbuild binary is a shebang script, needs shell:true on Windows
- resolveEsbuildBin: prefer .cmd in node_modules/.bin/ on Windows
over import.meta.resolve (which returns a shebang script)
- Updated test to accept .cmd extension on Windows
Found during UAT testing on Windows 11.
* fix: handle multi-line output from 'where' on Windows
'where esbuild' on Windows can return multiple matching paths, one per
line. Take only the first match to get a valid single path for
resolveEsbuildBin().
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: ByteYue <yj976240184@gmail.com>
* test(plugin): add E2E integration tests for plugin lifecycle
Add plugin-management.test.ts covering the full plugin lifecycle
using real GitHub clone of opencli-plugin-hot-digest:
- plugin install from github:ByteYue/opencli-plugin-hot-digest
- plugin list (table and JSON formats)
- plugin update on installed plugin
- plugin uninstall with cleanup verification
- error paths: invalid source, non-existent plugin, missing args
Tests safely backup/restore existing plugin state to avoid
interfering with user's real installed plugins.
Update TESTING.md to document the new test file.
* test(plugin): isolate lifecycle e2e from user home
* fix(plugin): respect HOME env var for test isolation
The E2E tests for plugin management were failing because os.homedir()
doesn't respect the HOME environment variable. This made test isolation
impossible since all tests would use the real ~/.opencli directory.
Added getHomeDir() helper that checks process.env.HOME first before
falling back to os.homedir(). Updated readLockFile() and writeLockFile()
to use this new function.
Fixes test failures in plugin-management.test.ts where:
- plugin install would write to real home instead of temp dir
- lock file assertions would fail with ENOENT
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(plugin): add lifecycle hooks API (onStartup, onBeforeExecute, onAfterExecute)
Introduce a hooks system that allows plugins to tap into opencli's
execution lifecycle without modifying core code.
New files:
- src/hooks.ts: hook registration, emission, and globalThis singleton
- src/hooks.test.ts: 10 unit tests covering registration, ordering,
error isolation, async support, and globalThis sharing
Modified files:
- src/execution.ts: emit onBeforeExecute/onAfterExecute around command execution
- src/main.ts: emit onStartup after discoverPlugins()
- src/registry-api.ts: export hooks API for plugin consumption
Example plugin: https://github.com/ByteYue/opencli-plugin-audit-log
* fix(discovery): load plugin files that register lifecycle hooks
The isCliModule() check only matched files containing 'cli(' calls,
silently skipping hook-only files like audit-hooks.ts that register
onBeforeExecute/onAfterExecute without any cli() command registration.
Renamed CLI_MODULE_PATTERN → PLUGIN_MODULE_PATTERN and extended the
regex to also match onStartup(, onBeforeExecute(, onAfterExecute(.
* fix(plugin): tighten lifecycle hook semantics
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Add search element heuristics and label/span wrapper detection
- Add SEARCH_INDICATORS set to detect search-related elements
- Add isSearchElement function for heuristic detection
- Add hasFormControlDescendant to detect wrapped form controls
- Enhance isInteractive for label/span wrapper patterns
Ref: browser-use ClickableElementDetector research
Review: @codex
- Remove unused re-exports from registry.ts (serializeArg, serializeCommand, etc.)
- Unify FormatOptions into SnapshotOptions from types.ts; rename dom-snapshot's
SnapshotOptions to DomSnapshotOptions to avoid name collision
- Extract shared analysis.ts module from explore.ts and record.ts, eliminating
~200 lines of duplicated logic (urlToPattern, findArrayPath, inferCapabilityName,
inferStrategy, detectAuth*, classifyQueryParams)
- Merge snapshotFormatter from 7-pass to 4-pass pipeline by combining parse+filter
with ad/boilerplate subtree skipping, and merging three dedup passes into one
- Rename all CLI adapter shared files to consistent utils.ts naming
(boss/common.ts, douban/shared.ts, doubao*/common.ts, jike/shared.ts,
medium/shared.ts, sinablog/shared.ts, substack/shared.ts)
- Merge douban/shared.ts into douban/utils.ts
Remove kubectl from:
- README.md highlights and external CLI examples table
- README.zh-CN.md highlights and external CLI examples table
- src/external-clis.yaml external CLI registry
kubectl is not relevant to the opencli project scope and should not be showcased as a primary example.
The dictionary adapters commit (3d39574) introduced a duplicate
`}, 30_000);` at line 524 of public-commands.test.ts, causing
the vite:oxc transformer to fail with [PARSE_ERROR] Unexpected token
in the E2E Headed Chrome CI workflow.
* feat(browser): human-like delay system for anti-detection
Adds a framework-level delay/jitter system using log-normal distribution
to simulate natural browsing patterns, addressing issue #59 (P0).
- New `HumanDelay` class with configurable profiles (none/fast/moderate/cautious/stealth)
- Log-normal distribution for realistic delay variance (not uniform)
- Periodic "breaks" that simulate reading/thinking pauses
- Auto-injected between page.goto() navigations
- Configurable via OPENCLI_DELAY_PROFILE env var
- Boss search adapter migrated from hardcoded jitter to framework delay
- 10 unit tests covering all profiles and edge cases
Real-world validation against a major job board (cookie-authenticated,
aggressive bot detection):
| Scenario | Without jitter | With jitter |
|-----------------------|--------------------|--------------------|
| 50 detail pages | ✅ OK | ✅ OK |
| 200 detail pages | ❌ Banned (code 32) | ✅ OK |
| 850 requests over 5h | N/A (banned early) | ✅ Zero detection |
| 4-day sustained crawl | N/A | ✅ 1800+ records |
Closes#59
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: disable human delay in CI environment to prevent E2E timeouts
In CI environments (CI=true), resolveProfile() now defaults to the
'none' profile instead of 'moderate'. This prevents the 1-8s per-
navigation delay from causing E2E test timeouts (30s limit).
Users can override this by setting OPENCLI_DELAY_PROFILE explicitly.
---------
Co-authored-by: toolmanlab <toolmanlab@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(grok): preserve conversation across repeated ask calls (#330)
The adapter unconditionally navigated to grok.com/ on every invocation,
destroying the existing conversation URL even when --new was not passed.
Since the browser daemon already reuses the same Chrome tab, skipping
navigation lets the tab stay on the current chat thread.
- Only navigate to grok.com/ when --new is true or tab is not on grok.com
- Add tryStartFreshChat to the default path's --new branch (was dead code)
- Add isOnGrok helper with hostname-based domain matching
- Add unit tests for isOnGrok
* test(grok): add adapter to vitest project config
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The search command defined its argument as `query` but referenced
`args.keyword`, causing the search term to be undefined.
Closes#334
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: add "Why opencli?" section and comparison guide (#238)
- Add "Why opencli?" section to README.md and README.zh-CN.md
(between Highlights and Prerequisites)
- Add docs/comparison.md with 5-scenario honest evaluation
- Add Comparison entry to VitePress sidebar
* docs: refine positioning — use approximate numbers, emphasize broad coverage
- Replace specific counts (300+, 55, 20+) with approximate descriptions
- Emphasize broad global + Chinese platform coverage instead of singling out Chinese sites
- Fix Firecrawl description to mention self-hosted option
- Replace "sub-second" / "milliseconds" with accurate "seconds" / "fast deterministic"
- Add testing and AI workflow to Further Reading links
- Add "easy to extend" point to strengths
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The favorite command was using up_mid: 0 which returns empty results. Now it correctly fetches the current user UID using getSelfUid().
Co-authored-by: 章晖 <zhanghui@MacBook-Pro.local>
* feat(linkedin): add timeline feed command
* test(linkedin): add timeline adapter unit tests
Add shape tests and utility function tests for the new timeline command.
Include linkedin in the vitest adapter project config.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The C2 fix in PR #337 added a null-page guard after lazy-loading TS
modules, but it threw unconditionally — breaking all browser:false
commands (bloomberg, apple-podcasts, google, yollomi, etc.) that
use func() with a null page. Guard now checks updated.browser !== false.
Also fixes apple-podcasts top E2E flake: when the command times out on
CI, stderr is empty and the guard didn't catch it.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add generic `web read` command for any URL → Markdown
Adds a new `opencli web read --url <any-url>` command that fetches any
web page and exports it as clean Markdown with optional image download.
Uses browser-side DOM heuristics for content extraction:
1. <article> element
2. [role="main"] element
3. <main> element
4. Largest text-dense block fallback
Pipes through the existing article-download pipeline (Turndown + image
localization), so it inherits code block handling, frontmatter generation,
and concurrent image downloading for free.
Tested on: Anthropic blog, OpenAI blog, general news sites.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: improve web read dedup for sites with duplicated DOM paragraphs
Anthropic's blog renders each paragraph twice (a normal version + a
line-broken animation version). The previous substring-based dedup
missed these because whitespace differences changed string lengths.
Fix: compare texts after stripping ALL whitespace, and keep the
version with more proper spacing (more spaces = better formatted).
Result on Anthropic blog: 98.4KB → 53.7KB (45% reduction).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Harrison <harrison@HarrisondeMacBook-Pro.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- I1: Log pre-navigation failures in debug mode instead of silently swallowing
- I2: Validate env var timeout values, fallback on NaN/negative
- I4: Guard against indexOf returning -1 for unknown strategies in cascade
- I5: Fix shouldReplaceManifestEntry returning true for same-type entries
- I6: Prevent infinite loop in parseTsArgsBlock cursor advancement
- I7: Skip redundant Page.enable calls in CDP goto
- I8: Fix wait({time:0}) being treated as falsy
- I10: Warn when cookiesFile path doesn't exist before fallback
- I11: Sanitize tab/newline chars in cookie name/value for Netscape format
- I12: Use DEFAULT_DAEMON_PORT constant instead of hardcoded port in error
- I15: Log npm install failures in plugin lifecycle instead of swallowing
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1. execution.ts: Guard lazy-loaded func commands against null page — if a
lazy module incorrectly requires browser context, throw a clear error
instead of a cryptic TypeError on page.goto().
2. daemon.ts: Fix readBody race condition — add aborted flag to prevent
req.destroy() from triggering both reject (via error) and resolve
(via end event) on the same Promise, which could process truncated data.
3. browser/cdp.ts: Prevent CDPBridge.connect() reentry — throw if already
connected instead of silently leaking the previous WebSocket and its
message handlers.
4. interceptor.ts: Store intercept pattern in a separate global variable
so subsequent installInterceptor calls with different patterns update
the match condition without being blocked by the patchGuard.
5. record.ts: Always call cleanupEnter() after Promise.race — previously
only called in the timeout path, leaving readline open when user pressed
Enter, potentially blocking process exit. Also removed unused enterRace.
6. generate.ts: Fix undefined entering String.includes() — when c.name is
undefined, toLowerCase() returns undefined which gets coerced to the
string "undefined" by includes(), causing false positive matches.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(security): harden against command injection and sandbox escape
1. cli.ts: Remove auto-discover of arbitrary system binaries via denylist.
Unknown commands now require explicit registration via `opencli register`.
The previous denylist approach was trivially bypassable (bash, curl, etc.).
2. template.ts: Protect evalJsExpr against prototype chain escape.
Block expressions containing constructor/prototype/__proto__/process/etc.
Deep-copy context objects to sever prototype chains before passing to
new Function().
3. external.ts: Expand shell operator detection in parseCommand to cover
$(), $, #, \n, \r — preventing command substitution and comment injection.
4. fetch.ts: Use JSON.stringify for HTTP method in browser evaluate() instead
of raw string interpolation, preventing JS injection via crafted method values.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: harden security-sensitive execution paths
* chore: tighten template sandbox guard
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Replace inline sync chrome.debugger.detach() in onUpdated listener
with the shared async detach() function for consistent cleanup behavior
across all detach paths.
- Only override navigator.plugins when empty (don't replace real user
browser plugins with fakes)
- Replace Error.prepareStackTrace (V8/Node-only) with
Error.prototype.stack getter override that works in browser context
- Fix \\n escaping in template literal for stack trace split/join
- Dynamic cdc_ variable scan via getOwnPropertyNames instead of
hardcoded names
- Update tests to cover 7 patches
Add stealth.ts module that patches browser globals to hide automation
fingerprints when opencli controls a browser via CDP or daemon extension.
Patches applied:
- navigator.webdriver → undefined (CDP sets it to true)
- window.chrome stub (only if missing)
- navigator.plugins fake list (only if empty)
- navigator.languages guarantee (only if empty)
- Permissions.query normalization for notifications
- Cleanup __playwright/__puppeteer/cdc_* artifacts
CDP mode: stealth registered via Page.addScriptToEvaluateOnNewDocument
(runs before any page JS on every navigation).
Daemon mode: stealth injected via exec after navigation, with guard
flag to prevent double-injection.
- Add getErrorMessage() to errors.ts (used in 5 files)
- Add DEFAULT_DAEMON_PORT to constants.ts (used in 5 files)
- Reduces code duplication and improves maintainability
* fix: remove duplicate getErrorMessage import in discovery.ts
Squash merge left a duplicate import line causing TS2300 and oxc parse
errors in CI. Also clean up stale blank lines in discovery.ts and
execution.ts.
* fix(e2e): broaden xiaoyuzhou skip logic for overseas CI runners
The isExpectedChineseSiteRestriction function only matched FETCH_ERROR
with specific HTTP status codes. On overseas CI runners, xiaoyuzhou may
also return PARSE_ERROR (mangled HTML) or NOT_FOUND (geo-redirected
pages), causing false test failures. Now matches all CliError codes
from the adapter.
* fix(external): replace execSync with execFileSync to prevent command injection
* fix(review): preserve Windows external installs and restore docs build
* fix(review): preserve Windows external installs after rebase
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(yollomi): add new commands and update documentation in README files
- Added yollomi commands for generating images, videos, and editing capabilities.
- Updated README.md and README.zh-CN.md to include yollomi in the command list.
- Enhanced SKILL.md with yollomi-related tags and usage examples.
* feat(yollomi): add yollomi adapter to documentation
- Included yollomi in the VitePress configuration for browser adapters.
- Updated adapters index documentation to reflect yollomi's capabilities and commands.
* fix(yollomi): bug fixes, tests & improvements
- models.ts: add browser: false (no browser connection needed for hardcoded data)
- edit.ts: remove unused resolveImageInput import
- upload.ts: lower video upload limit from 100MB to 20MB (base64 OOM risk)
- generate.ts: improve file extension detection using URL.pathname
- upscale.ts: use choices for scale arg, improve extension detection
- object-remover.ts: make image/mask args positional
- Add yollomi models tests to public-commands.test.ts
- Add yollomi generate/video graceful-failure tests to browser-auth.test.ts
---------
Co-authored-by: anichikage <hanzhishuai@bytedance.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(record): add live recording command for API capture
- Add `opencli record <url>` command that injects fetch/XHR interceptors
into all tabs in the automation window, polls captured requests, and
auto-generates YAML candidate adapters
- Support multi-tab recording: new tabs discovered during polling are
automatically injected
- Add --timeout (default 60s) for agent-friendly non-blocking operation;
stops on Enter, timeout, or SIGINT — whichever comes first
- Fix idempotent re-injection: restores original fetch/XHR before
re-patching so guard flag no longer blocks subsequent record runs
- Add --poll interval option (default 2000ms)
- Expand SKILL.md with full Record Workflow section: interceptor
internals, page-type capture expectations, YAML→TS conversion guide,
and troubleshooting table
* fix(record): fix XHR listener leak, pathChain syntax error, readline hang & args interpolation
- XHR send(): add __rec_listener_added guard to prevent duplicate event
listeners when XHR is reused (abort → open → send)
- pathChain: when findArrayPath returns '' (root-level array), data access
is just 'data' not 'data?.' which was invalid JS syntax
- waitForEnter(): return cleanup fn so timeout path can close readline.Interface
preventing the process from hanging on stdin after auto-timeout
- buildRecordedYaml: replace search/page query param values with template
vars ({{args.keyword}}, {{args.page}}) so generated YAML actually uses
the declared args instead of hardcoding the recorded URL
---------
Co-authored-by: yee.wang <yee.wang@lazada.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
- docs/adapters/browser/xiaohongshu.md: fill in search command description
(was empty), update usage examples with keyword positional arg
- TESTING.md: update unit test count 31→32 (search.test.ts added in #298),
add xiaohongshu/search.test.ts to the adapter test file list
* Add weibo search command
* fix(weibo/search): correct domain to weibo.com, add browser: true, fill doc description
- Change domain from s.weibo.com to weibo.com so browser cookies are picked
up correctly (matches hot.ts which also uses weibo.com)
- Add browser: true for consistency with other browser-based adapters
- Add description for weibo search in adapter docs table
---------
Co-authored-by: 小小机器人 <14351708+little-little-robot@user.noreply.gitee.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(v2ex): add node, user, member, replies, nodes commands
Add 5 new public API commands to the v2ex adapter:
- node: browse topics by node name
- user: list topics by username
- member: show user profile
- replies: list topic replies
- nodes: list all nodes sorted by topic count
All commands use strategy: public, browser: false.
* test(v2ex): add E2E tests for node, user, member, replies, nodes commands
* docs(v2ex): update adapter docs with new commands
* fix(v2ex): address review findings - rate-limit guards, sort verification, docs
* docs(v2ex): update README command tables and add user example
* test(v2ex): improve test quality - soft guards, value assertions, smoke tests
- Replace isExpectedChineseSiteRestriction with if(code===0) soft guard
(V2EX is globally accessible; YAML fetch doesn't throw FETCH_ERROR)
- Add value assertions: member username===Livid, limit effectiveness
- Add smoke tests for node, member, replies, nodes commands
* fix(v2ex): add url field to node/user commands, add missing user smoke test
- Add url to node.yaml and user.yaml pipeline map steps and columns
(V2EX API provides item.url; improves usability for follow-up lookups)
- Add v2ex user smoke test (other 4 new commands all had smoke tests; user was missing)
- Update E2E assertions to verify url field in node/user results
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Some environments (GUI apps, cron, IDE terminals) launch with a minimal
PATH that excludes standard directories like /usr/local/bin and /usr/sbin.
This causes external CLIs to fail when they try to run system commands
(e.g. sysctl).
Fix by ensuring standard system paths exist in process.env.PATH at
startup. This is a one-time fix that benefits ALL child processes —
isBinaryInstalled(), installExternalCli(), daemon spawn, etc. — without
needing per-call env patching.
Fixes#284
Co-authored-by: jackwener <jakevingoo@gmail.com>
* docs: add gws to External CLI Hub table in README
The Google Workspace CLI (gws) was registered in external-clis.yaml
but missing from the README table. Closes#120.
* docs: add gws to Chinese README External CLI Hub table
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Narrow '#noteContainer img[src*="xhscdn"]' to
'#noteContainer .media-container img[src*="xhscdn"]'
to exclude user avatars and sidebar icons from downloads.
Closes#281
Expand HackerNews from 1 command to 8, covering all major HN use cases.
All YAML adapters, strategy: public, browser: false.
- new/best/ask/show/jobs: Firebase API list endpoints with deleted/dead filtering
- search: Algolia API with query + sort (relevance/date)
- user: Firebase user profile with date formatting
- top.yaml: add filter for deleted/dead items + dynamic pre-fetch limit
- E2E tests for all 7 new commands
- Update README, README.zh-CN, adapter docs
Co-authored-by: jackwener <jakevingoo@gmail.com>
turndown and @types/turndown were used in article-download.ts and
zhihu/download.test.ts but never declared in package.json, causing
CI failures on fresh npm ci installs.
Adds `opencli xiaohongshu publish` which automates posting a 图文 (image+text)
note via the creator center UI (creator.xiaohongshu.com/publish/publish).
Features:
- --title (required, max 20 chars)
- positional content argument
- --images comma-separated local file paths (jpg/png/gif/webp, max 9)
- --topics comma-separated hashtag names (without #)
- --draft flag to save as draft instead of publishing
Image upload uses DataTransfer injection into the file input element, converting
local files to base64 in Node.js and creating File blobs in the browser context.
Text fields use document.execCommand('insertText') for contenteditable editors.
Graceful debug screenshots on failure (/tmp/xhs_publish_*_debug.png).
Requires: opencli browser session logged into creator.xiaohongshu.com.
Replace fixed settleMs sleep in goto() with MutationObserver-based DOM
stability detection. The page is considered settled when no DOM mutations
occur for quietMs (default 500ms), with settleMs as a hard timeout cap.
Changes:
- Add waitForDomStableJs() shared helper to dom-helpers.ts
- Update Page.goto() and CDPPage.goto() to use smart settle
- No IPage interface changes (implementation detail only)
Key improvements over naive approach:
- Timer starts AFTER MutationObserver.observe() to avoid race condition
- Falls back to sleep(maxMs) if document.body is not available
- Monitors attributes in addition to childList/subtree
- quietMs defaults to 500ms (conservative) for async request buffering
- Add Origin header check: reject HTTP/WS from non chrome-extension:// origins
- Require X-OpenCLI custom header on all HTTP requests
- Remove Access-Control-Allow-Origin: * from all responses
- Add WebSocket verifyClient to reject malicious connections at upgrade
- Add 1MB body size limit to prevent OOM
- Update file header with security model documentation
Closes#268
- Remove setup command completely (no backward compat needed)
- Doctor now runs live connectivity test by default
- Add --no-live flag to skip if needed
- Update SKILL.md docs
- Delete setup.ts (fully redundant with doctor)
- opencli setup now prints deprecation warning and delegates to doctor
- doctor auto-starts daemon if not running (no more false 'not connected')
- Update all doc references (README, SKILL.md, docs/)
Removed from both EN/CN READMEs:
- Table of Contents (GitHub auto-generates TOC)
- Method 2: Load from npm Package (keep recommended + dev only)
- Bloomberg detailed note (too specific for README)
- Pipeline Step YAML example (developer-internal)
- Releasing New Versions (belongs in CONTRIBUTING.md)
EN README only:
- Simplified Testing section to one-liner + link to TESTING.md
Cleanup:
- Remove redundant double-retry in resolveTabId (was retrying data: URI
with the same data: URI)
- Fix stale comment (30s → 120s idle timeout)
- Remove verbose debug logging in resolveTabId
- Built extension is now smaller (16.66kB vs 17.18kB)
Extension conflict:
- Add hint to attach-failed error when chrome-extension:// URL is detected
- Add troubleshooting entry for extension conflicts (e.g. youmind, New Tab
Override) to both README.md and README.zh-CN.md
Ref: #249
When a new automation window is created, the initial tab URL may be
empty briefly while Chrome loads the data: URI. isDebuggableUrl('') was
returning false, causing ensureAttached to reject the tab.
Fix: only reject known non-debuggable URLs (chrome://, chrome-extension://).
Empty/undefined URLs are now treated as debuggable since they represent
tabs still loading.
Also adds 200ms delay after window creation to let Chrome populate the
tab URL.
CRX files cannot be installed in modern Chrome without Chrome Web Store
publishing. Updated all docs to recommend 'Load unpacked' installation
method only. Added npm package loading method as alternative.
- Removed CRX build step from build-extension.yml workflow
- Removed CRX from artifact upload and release attachment
- Updated README.md, README.zh-CN.md, browser-bridge docs (en/zh)
- Added 'Load from npm package' as installation method
Root cause: getAutomationWindow and resolveTabId used about:blank which
New Tab Override extensions intercept immediately, replacing it with
chrome-extension:// URLs that cannot be debugged.
Changes:
- Window creation: about:blank → data:text/html
- reuseTab fallback: about:blank → data:text/html
- newTab handler: about:blank → data:text/html
- Added diagnostic logging to resolveTabId for debugging
- Synced extension version to 1.2.4
Ref: #249
resolveTabId's reuseTab path now verifies the URL is actually debuggable
after navigating to about:blank. If a New Tab Override extension intercepts
it (setting it back to chrome-extension://), falls back to a data: URI,
then creates a fresh tab as last resort.
This fixes the persistent 'attach failed: Cannot access chrome-extension://'
error for users with New Tab Override extensions installed.
Ref: #249
- resolveTabId: validate URL even for explicit tabId, fall through to
auto-resolve when tab is not debuggable or has been closed
- handleNavigate: wait for URL change before checking 'complete' status
to avoid race condition with stale about:blank
- ensureAttached: pre-check tab URL, verify cached attach with probe,
invalidate cache on URL change via onUpdated listener
- daemon-client: recognize transient extension errors (disconnected,
attach failed) as retryable with 1500ms delay; fresh command ID per attempt
- pipeline executor: add per-step retry for browser steps (up to 2 retries
on transient errors); cleanup automation window on pipeline failure
- page.ts: selectTab/newTab/closeTab properly update/invalidate _tabId
- daemon.ts: add WebSocket ping/pong heartbeat (15s interval, 2-miss disconnect)
- Increase automation window idle timeout from 30s to 120s
- Fix timeout param edge cases in BrowserBridge._ensureDaemon
- Remove unused chalk import; fix trailing import placement
Closes#249
* feat(douban): add movie adapter with search, top250, subject, marks, reviews commands
- search: search movies by keyword
- top250: get top 250 movies
- subject: get movie details by id
- marks: export personal viewing marks
- reviews: export personal movie reviews
* review: resolve douban adapter blockers
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(devto): add devto adapter
* refactor(devto): improve adapters to match project conventions
- Make tag/username args positional for natural CLI usage:
opencli devto tag javascript (instead of --tag javascript)
opencli devto user ben (instead of --username ben)
- Add rank field (index + 1) matching hackernews/lobsters pattern
- Add tags field from tag_list for richer output
- Remove redundant author column from user command (already filtering by user)
- Use type: str (project convention) instead of type: string
- Increase default limit from 10 to 20 (matching other adapters)
- Update docs with positional arg examples
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(wikipedia): fix search arg name + add random and trending commands
- fix: search.ts referenced `args.keyword` but the argument is defined
as `query`, causing the search term to always be undefined
- feat: add `random` command (random article summary via REST API)
- feat: add `trending` command (most-read articles, yesterday's data)
All commands are PUBLIC strategy, no browser required, reuse wikiFetch.
* refactor(wikipedia): extract shared types + add docs for random/trending
- Extract WikiSummary, WikiMostReadArticle types to utils.ts
- Extract EXTRACT_MAX_LEN/DESC_MAX_LEN constants
- Add formatSummaryRow() helper to eliminate duplicate mapping in
summary.ts and random.ts
- Update docs with random and trending command examples
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Support switching between For You (algorithmic) and Following
(chronological) timelines via `--type for-you|following`.
Both endpoints share the same response structure; only the GraphQL
endpoint name and queryId differ. QueryId is resolved dynamically
from fa0311/twitter-openapi with a hardcoded fallback, and validated
against /^[A-Za-z0-9_-]+$/ to prevent injection from upstream.
* fix(doctor): refresh status after live check to resolve#121
* refactor: reorder live check before status read for natural consistency
Instead of calling checkDaemonStatus() twice (before and after the
connectivity check), reorder so that the live connectivity check runs
first, then read daemon status only once. This:
- Eliminates redundant checkDaemonStatus() call
- Naturally avoids the timing inconsistency (fixes#121)
- Also fixes the sessions query using stale status
- Simplifies test assertions to avoid over-coupling to exact wording
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(google): add search, suggest, news, and trends adapters
Four new commands under `google`:
- search: browser-based DOM extraction from google.com/search
- suggest: public JSON API (suggestqueries.google.com)
- news: public RSS feed (top stories + keyword search)
- trends: public RSS feed (daily trending searches by region)
Shared RSS parser in utils.ts with attribute/CDATA support.
Unit tests for parseRssItems, E2E tests with network skip guards.
* refactor(google): downgrade search strategy from COOKIE to PUBLIC
Google search results are public data, no login needed. Browser is
required for DOM rendering, not authentication. Standalone mode
confirmed working in testing.
* fix: update test comment to reflect PUBLIC strategy
Add new YAML adapter to fetch upcoming earnings dates from xueqiu's
company events API (公司大事). Supports A-share and H-share stocks.
Features:
- Filter by subtype=2 (预计财报发布) from event timeline
- Show date, report name, and release status (⏳/✅)
- --next flag to return only the closest upcoming earnings date
- --limit to control result count
Co-authored-by: nekomoto911 <nekomoto911@gmail.com>
Browser adapters using COOKIE/HEADER strategy need the page on the target
domain so credentialed fetch() carries cookies. Previously, execution.ts
hardcoded `cmd.site === 'boss'` to skip this pre-navigation for adapters
that handle their own goto().
Now each adapter self-declares via `navigateBefore: false` on CliCommand.
This is more extensible — new sites that manage their own navigation just
add the field instead of editing execution.ts.
Changes:
- Add `navigateBefore?: boolean | string` to CliCommand interface
- Add `resolvePreNav()` helper in execution.ts (replaces hardcoded check)
- All 14 boss adapters declare `navigateBefore: false`
- Wire through discovery.ts (YAML + manifest) and build-manifest.ts
* refactor(boss): extract common utilities, fix missing login detection
- Add src/clis/boss/common.ts with shared helpers:
- bossFetch(): unified XHR template with auto cookie-expiry detection (code 7/37)
- navigateToChat()/navigateTo(): page navigation helpers
- checkAuth()/assertOk(): centralized login state validation
- fetchFriendList()/fetchRecommendList()/findFriendByUid(): data queries
- clickCandidateInList()/typeAndSendMessage(): UI automation helpers
- verbose(): conditional debug logging
- Refactor all 14 boss adapters to use common.ts:
- chatlist.ts: was missing cookie-expiry check (fixes #login-detect)
- chatmsg.ts: was missing cookie-expiry check (fixes #login-detect)
- Remaining 12 adapters: deduplicated XHR boilerplate and error handling
- Fix execution.ts: skip redundant pre-navigation for TS adapters
- TS adapters handle their own goto(), pre-navigating caused double
page loads and could trigger duplicate login prompts
- Pre-navigation preserved for YAML pipeline commands that need it
Net reduction: ~730 lines of duplicated code across boss adapters.
All 244 unit tests pass.
* fix(review): fix execution.ts pre-nav regression, sanitize UID input, restore docs
- execution.ts: use site-specific skip (boss only) instead of isYamlPipeline.
The original check skipped pre-navigation for ALL TS adapters, but weread,
chaoxing, and others don't do their own goto() and depend on it.
- common.ts: sanitize numericUid to digits-only and use JSON.stringify for
safe interpolation in page.evaluate() (prevents template literal injection).
- resume.ts: restore HTML structure doc comments (scraping selector guide).
- send.ts: restore MQTT architecture note (explains why UI automation is needed).
* fix: restore DEBUG env support in verbose(), improve skipPreNav comment
- verbose() now checks both OPENCLI_VERBOSE and DEBUG=opencli,
matching the original behavior from search.ts and detail.ts
- Clarify skipPreNav comment with TODO for future adapter-level flag
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(instagram,facebook): add write actions and extended commands
Instagram write actions (7 commands, internal REST API + CSRF token):
- like/unlike: like or unlike a user's post by username + index
- comment: comment on a user's post
- save/unsave: bookmark or remove bookmark on a post
- follow/unfollow: follow or unfollow a user
Facebook extended commands (6 commands, DOM scraping):
- friends: friend suggestions list
- groups: list your joined groups with last post time
- memories: On This Day memories
- events: browse event categories
- add-friend: send friend request by username
- join-group: join a group by ID
All commands tested with live data. 258 existing tests pass.
* docs: add adapter documentation for instagram, facebook, lobsters
* docs: add missing medium adapter documentation
* fix(extension): skip chrome-extension:// tabs in resolveTabId fallback
Remove the unsafe fallback that returned `tabs[0]` regardless of URL
type. When no web-accessible tab exists in the automation window (e.g.
a New Tab Override extension replaced about:blank with its own
chrome-extension:// page), we now always create a fresh about:blank
tab instead. This prevents chrome.debugger.attach from failing with
"Cannot access a chrome-extension:// URL of different extension".
Fixes#195, fixes#197
* refactor(extension): rename isWebUrl → isDebuggableUrl & reuse tabs in resolveTabId
Improvements over the original fix:
1. Rename isWebUrl() → isDebuggableUrl(): better reflects the intent —
the function determines whether a URL can be attached via CDP, not
just whether it's a "web" URL (about:blank is debuggable but not
really a web URL).
2. Reuse existing non-debuggable tabs: when a New Tab Override extension
replaces about:blank with chrome-extension://, use chrome.tabs.update()
to navigate the existing tab to about:blank instead of creating a new
one. This prevents orphan tab accumulation since chrome.tabs.create()
may also get intercepted by the same extension.
3. Only fall back to chrome.tabs.create() when the window has zero tabs,
which is the truly empty-window edge case.
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat(tiktok): add TikTok adapter with 15 commands
TikTok (15 commands, browser mode):
Read commands:
- profile: user profile info via rehydration script parsing
- search: search videos via internal search API
- explore: trending videos from explore page (DOM scraping)
- user: recent videos from a user page (DOM scraping)
- following: list accounts you follow
- friends: friend suggestions
- live: browse live streams with viewer counts
- notifications: activity notifications
Write commands (verified with real interactions):
- like/unlike: like or unlike a video by URL
- save/unsave: add or remove video from Favorites
- follow/unfollow: follow or unfollow a user
- comment: comment on a video
All write operations verified with live TikTok interactions.
* docs: add missing adapter documentation for doc-coverage CI
* feat(lobsters): add Lobste.rs adapter with hot, newest, active, tag commands
Add public API adapter for Lobste.rs (lobste.rs), a developer-focused
link aggregation community. All commands use the public JSON API and
require no authentication or browser.
Commands:
- hot: hottest stories
- newest: latest stories
- active: most active discussions
- tag: filter stories by tag (e.g. rust, security, programming)
* feat(instagram,facebook): add Instagram and Facebook adapters
Instagram (7 commands, browser mode - internal REST API):
- profile: user profile info (followers, following, posts, bio)
- search: search users
- user: recent posts from a user
- followers: list user's followers
- following: list user's following
- saved: saved posts
- explore: discover trending posts
Facebook (4 commands, browser mode - DOM scraping):
- profile: user/page profile info
- notifications: recent notifications
- feed: news feed posts
- search: search people, pages, posts
All commands require Chrome to be logged in to the respective site.
Instagram uses stable internal API endpoints with cookie auth.
Facebook uses DOM scraping via role attributes and semantic selectors.
* feat: plugin system (Stage 0-2)
- Stage 0: discoverPlugins() scans ~/.opencli/plugins/ at startup
- Stage 1: demo plugin repos (github-trending, hot-digest)
- Stage 2: opencli plugin install/uninstall/list commands
- package.json exports ./registry for TS plugin peerDep support
- 17 new/updated tests, tsc --noEmit clean
* fix: CDPBridge connect timeout unit mismatch (seconds vs ms)
opts.timeout is passed in seconds from runtime.ts but CDPBridge
was using it as milliseconds, causing instant timeout (30ms).
* feat: add registry-api public entry point for TS plugin peerDep support
- Add src/registry-api.ts: re-exports core registration API (cli, Strategy,
getRegistry) without transitive side-effects, safe for plugin imports
- Update package.json exports: './registry' -> './dist/registry-api.js'
- Update src/registry.ts: use globalThis shared registry to ensure single
instance across npm-linked plugin modules
- Update .gitignore for plugin-related artifacts
* fix: symlink host opencli into plugin node_modules on install
After npm install, replace the npm-installed @jackwener/opencli
with a symlink to the running host's package root. This ensures
TS plugins always resolve '@jackwener/opencli/registry' against
the host installation, avoiding version mismatches when the
published npm package lags behind.
* fix: transpile TS plugins to JS on install, deduplicate .ts/.js discovery
- installPlugin: after symlinking host opencli, transpile any .ts files
to .js using esbuild from the host's node_modules/.bin/
- discoverPluginDir: skip .ts files when a .js sibling exists (production
node cannot load .ts directly)
- scanPluginCommands: deduplicate basenames via Set to avoid showing
'aggregate, aggregate' when both .ts and .js exist
* docs: add plugin system user guide
- New docs/guide/plugins.md covering:
- Installation/uninstallation commands
- Creating YAML plugins (zero-dep)
- Creating TS plugins (with peerDep)
- TS plugin install lifecycle (clone → deps → symlink → transpile)
- Example plugins and troubleshooting
- Add Plugins to VitePress sidebar (EN + ZH)
- Link from getting-started.md Next Steps
* fix: address review issues in plugin system
- Security: replace execSync with execFileSync to prevent shell injection
- Replace deprecated npm --production with --omit=dev
- Tighten parseSource regex to [\w.-]+ to reject special chars
- Fix ZH sidebar plugin link (/guide/plugins → /zh/guide/plugins)
- Return plugin name from installPlugin() to avoid duplicated logic
- Use execFileSync for esbuild transpilation
- Fix misleading comment in linkHostOpencli
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* fix(twitter): rewrite trending from YAML to TS with DOM scraping fallback
The old REST API /i/api/2/guide.json returns 503. Replace with a TS
adapter that:
- Tries legacy guide.json API first (with proper auth headers)
- Falls back to DOM scraping via [data-testid='trend'] elements
- Filters out promoted content
- Follows the same Strategy.COOKIE pattern as timeline.ts
* fix: use 'help' instead of 'description' in Arg (matches Arg interface)
* docs(steam): add adapter documentation, update READMEs
- Create docs/adapters/browser/steam.md
- Add steam entry to README.md and README.zh-CN.md
- Fixes doc-coverage CI check (44/44)
Add three new Twitter/X UI-strategy commands:
- `block` / `unblock` — block or unblock a user by username
- `hide-reply` — hide a bot/spam reply on your own tweet thread
* feat: add `opencli describe` command for unified CLI capability discovery
Add a new `describe` command that helps AI agents discover and understand
both built-in site commands and external CLI tools through a single entry point.
- Built-in commands: reads structured data from CliCommand registry
(args with type/choices/default, columns, strategy, domain)
- External CLIs: collects help text via `binary --help`, extracts
subcommand names + summaries, passes through raw help text
- Supports `--format json` for programmatic consumption by AI agents
- Graceful degradation: parse failures return raw help text, uninstalled
CLIs show install instructions without triggering auto-install
Closes#141
* fix: address code review findings for describe command
- Strip trailing colons from Cobra-style subcommand names (browse: → browse)
- Use CliError instead of bare Error for consistent error handling with hints
- Remove decorative section separators to match project comment style
- Validate --format flag (text/json only) with clear error message
- Truncate raw help output to 50 lines to prevent excessive output
- Add deduplication test for multi-section command groups
* refactor: replace describe command with enhanced --help and list --json
Per maintainer feedback, remove the standalone `describe` command and instead:
1. Enhance --help for all built-in commands:
- Show argument choices (from registry, not shown by Commander)
- Show execution metadata: Strategy / Browser / Domain
- Show output columns
2. Enhance `list -f json/yaml` with full argument schema:
- args field now includes type, required, positional, choices, default, help
- Added columns and domain fields for structured formats
- Table/csv/md formats unchanged (args remain comma-joined names)
This follows the principle that --help is the standard CLI discovery
mechanism and AI models already know to use it.
* fix: stabilize JSON schema and fix positional choices rendering
- Always output columns/domain in json/yaml ([] and null when empty)
- Use <name> instead of --name for positional args with choices
- Remove extra blank line when no choices args present
* docs: add missing adapter docs, fix sidebar 404s, add doc-check CI
- Add doc pages for 11 undocumented adapters: arxiv, barchart,
chaoxing, grok, hf, jike, jimeng, linux-do, sinafinance,
stackoverflow, weread, wikipedia
- Update adapters/index.md with all new adapter entries
- Update VitePress sidebar config with 12 new entries
- Remove broken zh/ sidebar refs (troubleshooting, testing)
- Add doc-check CI workflow (adapter coverage + build + link check)
- Add scripts/check-doc-coverage.sh for adapter doc enforcement
- Enhance PR template with adapter doc checklist
* fix(ci): use --root-dir instead of --base for lychee link checker
lychee v0.23 requires --base to be a URL or absolute path.
Use --root-dir for resolving root-relative links in local files.
* fix(ci): remove lychee link-check job, rely on VitePress build
VitePress links use extension-less paths (e.g. /adapters/browser/twitter)
which lychee cannot resolve. The docs-build job already catches all
broken internal links via VitePress dead link detection during build.
- Auto-click New Conversation if session has only 1 message
- Map Anthropic models (claude-3-7-sonnet) to Antigravity UI models
- Refactor waitForReply to check for Cancel/Stop button presence to
detect generation completion reliably, with text stability fallback
- Replace document.execCommand (deprecated) with CDP Input.insertText
- Use Input.dispatchMouseEvent to physically click + focus the Lexical editor
before text injection (fixes focus issues with JS-only .focus())
- Improve getLastAssistantReply: strip echoed user message, thinking blocks,
Copy button text, and de-duplicate repeated content artifacts
- New command: opencli antigravity serve --port 8082
- Starts HTTP server compatible with Anthropic /v1/messages API
- Connects to Antigravity via CDP (OPENCLI_CDP_ENDPOINT)
- Uses Input.dispatchKeyEvent for reliable Enter key submission
- Polls for reply with text-change detection + 3s stability check
- Precise DOM walker for extracting last assistant reply
- Lazy CDP connection (connects on first request)
- Auto-reconnect on CDP connection loss
- CORS headers for Claude Code compatibility
Usage:
OPENCLI_CDP_ENDPOINT=http://127.0.0.1:9224 opencli antigravity serve
ANTHROPIC_BASE_URL=http://localhost:8082 claude
formatPostTime() used local timezone methods, causing test failure
on UTC CI servers. XHS API timestamps are Beijing time (UTC+8),
so use explicit UTC offset with getUTC*() methods.
- wikiFetch return Promise<unknown> instead of Promise<any>
- Add WikiSearchResult type, remove r: any
- Type wikiFetch responses with inline type assertions
- Only append ... to abstract when actually truncated
Add arXiv (search, paper) and Wikipedia (search, summary) public API adapters.
- arxiv/search: search papers by keyword
- arxiv/paper: get paper details by ID
- wikipedia/search: search articles with lang support
- wikipedia/summary: get article summary
Type safety fixes applied: wikiFetch returns unknown, typed search results.
Co-authored-by: BruceLoveDecimal <39156883+BruceLoveDecimal@users.noreply.github.com>
- Remove fetchCreatorNotesByCdp() and captureNoteDetailApiPayload() raw
WebSocket code (~240 lines) — adapters should use IPage, not raw CDP
- Replace direct CDP WebSocket with IPage.evaluate() in-page fetch
- Fix page: any → IPage in all function signatures
- Simplify to two-tier fallback: API+interceptor → DOM parse
- Rebase onto latest main (resolves cdp.ts/daemon.ts conflicts)
- Change VitePress base from '/opencli/' to '/' for custom domain opencli.info
- Add docs/public/CNAME so GitHub Pages preserves custom domain on re-deploy
* docs: deduplicate documentation — single source of truth in docs/
- Remove root CDP.md, CDP.zh-CN.md, CLI-ELECTRON.md (now in docs/advanced/)
- Slim adapter READMEs to one-liner + link to docs/ (11 files)
- Update README.md adapter table links to point to docs/
* docs: set VitePress base path for GitHub Pages deployment
URLSearchParams.toString() encodes spaces as +, but Bilibili's WBI
signature verification expects %20. This mismatch causes search
queries with spaces (e.g. "亚马逊 滞销产品") to fail with
TypeError: Failed to fetch due to CORS-blocked error responses.
Fixes#125
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The build manifest includes antigravity/serve which collides with the
hardcoded antigravity serve in cli.ts. Add a guard to skip registry
entries whose subcommand already exists in the site group.
* feat(hf): add top command for hf papers (daily, weekly, monthly)
* feat(footer): add footerExtra support and derive dates from API response
Add footerExtra callback to CliCommand for custom table footer content.
For weekly/monthly periods, derive date range from API response publishedAt
field with local clock fallback.
* fix: truncate long paper titles
* refactor(hf): remove comments column for consistent output
* feat(hf): add --all flag to return all papers
* feat(hf): add paper id column to output
* fix: restore main.ts as bootstrap, sync footerExtra + CDPBridge + domain pre-nav to cli.ts
- main.ts should remain a lightweight entry point delegating to cli.ts
- Preserve CDPBridge fallback (OPENCLI_CDP_ENDPOINT) — PR had hardcoded BrowserBridge only
- Add domain pre-navigation for cookie/header strategies to cli.ts
- footerExtra feature from PR is properly integrated
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
The old adapter called `search.smzdm.com/ajax/?c=<channel>&s=<q>` which
now returns 404. This caused opencli smzdm search to always return empty
results regardless of keyword.
Fix: navigate directly to `search.smzdm.com/?c=home&s=<keyword>&v=b`
and scrape the rendered DOM via querySelectorAll('li.feed-row-wide').
Also switched from async IIFE to sync IIFE since all data is already in
the DOM after page load — no fetch needed.
Tested: opencli smzdm search --keyword A7M5 returns correct results
with prices and mall names.
Adds 'opencli boss resume --uid <uid>' command that scrapes the chat page
right panel to display candidate resume information including:
- Basic info: name, gender, age, experience, degree, active status
- Work history: time period + company + position
- Education: time period + school + major + degree
- Job being discussed and candidate expectations
Uses UI scraping approach since BOSS Zhipin does not expose a public API
for candidate resume data on the recruiter side.
Add comprehensive Jike (即刻) adapter covering read and write operations.
Read commands:
- user: user posts via m.okjike.com SSR JSON
- topic: topic/circle posts via m.okjike.com SSR JSON
- post: post detail with comments via m.okjike.com SSR JSON
- feed: home timeline via React fiber tree extraction
- search: search posts via React fiber tree extraction
- notifications: notification list via DOM innerText parsing
Write commands (Strategy.UI, browser DOM automation):
- create: publish post via inline compose box
- comment: comment on post via contenteditable paste
- like: like post via _likeButton_ div click
- repost: repost via action bar → popover menu → confirm
Implementation details:
- Three data extraction strategies: SSR JSON, React fiber, DOM manipulation
- Shared JikePost interface and getPostData helper in shared.ts
- All evaluate blocks include try/catch error handling
- Two rounds of parallel Claude + Codex code review applied
# Please enter a commit message to explain why this merge is necessary,
# especially if it merges an updated upstream into a topic branch.
#
# Lines starting with '#' will be ignored, and an empty message aborts
# the commit.
Replace ad-hoc string escaping with JSON.stringify() for values
interpolated into JavaScript code strings passed to page.evaluate().
- explore.ts: clickLabels were escaped with only single-quote
replacement, which breaks on labels containing backslashes or
newlines. JSON.stringify() handles all edge cases correctly.
- synthesize.ts: buildEvaluateScript() embedded URLs directly inside
single quotes. JSON.stringify() safely handles URLs containing
special characters.
- boss/chatlist: List chat conversations (招聘端聊天列表)
Uses getBossFriendListV2 API with pagination and job filter support.
- boss/chatmsg: Read chat message history with a candidate
Resolves encryptUid to numeric uid/securityId, fetches via historyMsg API.
- boss/send: Send chat message to a candidate via UI automation
BOSS chat uses MQTT protocol (not HTTP), so this command automates the
web chat UI: clicks on user in list → types in contenteditable editor →
clicks the send button.
All three commands use Strategy.COOKIE and require an active BOSS直聘
login session in Chrome.
The previous approach (nativeSetter + Enter keydown on the search input)
does not reliably trigger Twitter's form submission - the synthetic
KeyboardEvent is ignored by React, leaving the page on /explore with
zero API calls captured.
Use history.pushState + PopStateEvent instead, which triggers React
Router's listener and performs a true SPA navigation to /search.
The interceptor survives because no full page reload occurs.
Tested: "opencli", "it's a test" (single quote), "hello" all return
results with correct author attribution.
Add CLI commands to view Chaoxing assignments and exams by reusing
Chrome login session via the Browser Bridge.
Chaoxing has no flat API for listing assignments/exams. The adapter
follows the browser flow: establish session → fetch course list via
backclazzdata API → enter each course via stucoursemiddle redirect →
click tab to capture iframe URL → navigate and parse DOM.
Commands:
opencli chaoxing assignments [--course <name>] [--status] [--limit]
opencli chaoxing exams [--course <name>] [--status] [--limit]
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(intercept): use evaluate() for IIFE wrapping in installInterceptor/getInterceptedRequests
Root cause: daemon migration changed these methods from this.evaluate()
to direct sendCommand('exec'), losing the wrapForEval() IIFE wrapping.
CDP received bare arrow functions that were never invoked.
Fixes#98
* fix(twitter): SPA navigation, data path, and author resolution for INTERCEPT commands
- followers/following: install interceptor on profile page, then click
followers/following link (SPA navigation preserves JS context).
Use JSON.stringify for targetUser to prevent injection. Throw on
navigation failure. Update selector: /verified_followers.
- notifications: install interceptor on home, then pushState+popstate
to /notifications. Validate navigation URL.
- search: fix author resolution (core.screen_name, not legacy).
- All: fix GraphQL data path (remove extra .data level), update author
resolution to try core.screen_name before legacy.screen_name.
- followers: remove erroneous .filter(r => r?.url) — interceptor stores
response body JSON, URL filtering happens at capture time.
Replace eager close-window (which caused race conditions when
parallel commands shared the window) with an idle-based timer:
- Window auto-closes 30s after the last command completes
- Each incoming command resets the idle timer
- Consecutive commands reuse the same window (faster)
- No race conditions with parallel execution
- Close-window action kept for explicit cleanup if needed
- Add 'close-window' action to extension protocol and background.ts
- Add Page.closeWindow() method to send close-window command
- browserSession() now closes automation window in cleanup
- Remove domain pre-navigation + 2s wait from main.ts (CDP handles
cross-domain cookies natively, no same-origin workaround needed)
- Net effect: commands run faster, no stale windows left behind
- Delete unused extension/src/executor.ts (chrome.scripting experiment)
- Remove 15 no-op backward-compat exports from doctor.ts
- Remove getTokenFingerprint no-op from browser/index.ts
- Rename PlaywrightMCP → BrowserBridge across all source files
(backward-compat alias kept in mcp.ts and browser/index.ts)
- Remove unnecessary host_permissions from extension manifest
- Sync extension package.json version to 0.2.0
- All 14 tests pass
All opencli operations now run in a dedicated Chrome window instead
of hijacking the user's active tab. The automation window:
- Created on first command via chrome.windows.create({ focused: false })
- 1280x900 viewport, auto-cleaned up when closed
- All tabs resolved within this window only
- User's main browsing session is never touched
Tested: twitter trending ✅, zhihu hot ✅
Both commands now scroll the conversation list to load more items
before processing. Scrolls up to 20-30 times, stops after 3
consecutive scrolls with no new items loaded.
Previously limited to ~14 visible conversations, now loads as many
as needed (up to --max).
- Rewrite accept.ts: use [data-testid=conversation] click-based approach
instead of extracting href links (requests page has no /messages/xxx links)
- Support comma-separated keywords for OR matching (e.g. '群,微信')
- Add timeoutSeconds: 600 (10 min) for batch DM operations
- Bump default OPENCLI_BROWSER_COMMAND_TIMEOUT from 45s to 60s
- Track visited conversations to avoid infinite loops
Usage:
opencli twitter accept --keyword '微信' --max 20
Workflow:
1. Navigate to /messages/requests
2. Click into each conversation
3. If message contains keyword, click Accept
4. After accept (auto-redirects to /messages), go back to requests
5. Repeat until --max reached or no more matches
process.execPath is always plain 'node' even under tsx,
so .ts files could not be executed. Use --import tsx/esm
flag to enable TypeScript loading in spawned daemon.
Add weread adapter for issue #82, covering search, rankings, book details,
bookshelf, notebooks, highlights, and notes.
Public commands (no login required):
- weread search <keyword> — search books
- weread ranking [category] — book rankings (all/rising/category ID)
Private commands (cookie auth via browser):
- weread book <bookId> — book details
- weread shelf — personal bookshelf
- weread notebooks — books with highlights/notes
- weread highlights <bookId> — underlines in a book
- weread notes <bookId> — personal notes on a book
Closes#82
goto() triggers a full page navigation that resets the JS execution
context, wiping any previously injected fetch/XHR monkey-patches.
The old code installed the interceptor on x.com then navigated away,
so the interceptor was always destroyed before it could capture data.
Fix: navigate directly to the target page, install interceptor after
page load, then scroll to trigger API calls via pagination.
Also fixes the same bug in notifications.ts.
Closes#86
Exponential backoff:
- Reconnect delay: 2s, 4s, 8s, 16s, ..., capped at 60s
- Resets to base delay on successful connection
- Reduces idle CPU waste vs fixed 3s reconnect
Screenshot via CDP Page.captureScreenshot:
- New 'screenshot' action in protocol (5th action)
- Supports format (png/jpeg), quality, fullPage
- Full-page: uses Emulation.setDeviceMetricsOverride for scroll height
- CLI-side: page.screenshot() with optional file save
- Extension build: 9.81KB (+1.7KB from 8.11KB)
Inspired by bb-browser's architecture patterns.
When using CDP mode (OPENCLI_CDP_ENDPOINT), the browser page context is
the user's active tab which may be on an unrelated domain. Cookie/header
strategy commands that use fetch() with credentials: 'include' then fail
with "Failed to fetch" due to the browser's same-origin policy.
Fix: before executing cookie/header strategy commands, navigate to the
command's declared domain so the fetch runs in same-origin context.
This mirrors the pre-navigation already done in the cascade command.
Affects all cookie-strategy adapters (bilibili, twitter, zhihu, xueqiu,
etc.) when OPENCLI_CDP_ENDPOINT is enabled and the active Chrome tab is
on a different site.
Co-authored-by: kensei <backtime1993@gmail.com>
* chore(ci): add Dependabot for npm and GitHub Actions updates
- Weekly npm dependency updates with PR limit of 10
- Weekly GitHub Actions version updates with PR limit of 5
- Conventional commit prefixes (chore(deps), chore(ci))
* ci: add security audit workflow
- Run npm audit on push/PR and weekly schedule
- Fail on high-severity vulnerabilities using audit-ci
- Only audit production dependencies
* ci: add release-please for automated changelog and versioning
- Auto-generate CHANGELOG.md from Conventional Commits
- Create version bump PRs on push to main
- Works alongside existing release.yml for npm publish
* ci: add concurrency controls and Node.js version matrix
- Add concurrency groups to ci, e2e-headed, security workflows
to cancel duplicate runs on the same branch
- Test unit tests across Node 18/20/22 with fail-fast: false
- Update test step name to show Node version
* chore: bump minimum Node.js version from 18 to 20
- Update engines.node in package.json to >=20.0.0
- Update prerequisites in README.md and README.zh-CN.md
- Remove Node 18 from CI test matrix
* review: fix release token and prod-only audit scope
* docs: align Node 20 troubleshooting guidance
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
Feishu uses custom 'Lark Framework' (Chromium-based but NOT Electron).
CDP port test failed — --remote-debugging-port has no effect.
Uses AppleScript + clipboard approach (same as WeChat/ChatGPT).
Commands: status, send, read, search (Cmd+K), new (Cmd+N)
Includes adapter READMEs (EN+ZH).
- Remove feishu and wechat adapters (not tested yet, will re-add later)
- Remove their rows from README.md and README.zh-CN.md
- Significantly polish CLI-ELECTRON.md skill guide:
- Add Electron detection guide (check for Electron Framework)
- Add Non-Electron AppleScript pattern section
- Add port assignment table for all CDP adapters
- Improve code examples with real working TypeScript
* feat(xiaohongshu): add 4 creator analytics commands
Add creator backend support for Xiaohongshu (小红书), enabling
creators to access their analytics data from the command line.
New commands:
- creator-profile: account info (followers, likes, creator level)
- creator-stats: 7-day/30-day overview (views, likes, collects,
comments, shares, new followers) with daily trend data
- creator-notes: note list with per-note metrics from note manager
- creator-note-detail: single note analytics breakdown
(organic vs promoted vs video traffic)
API discovery:
- /api/galaxy/creator/home/personal_info (cookie auth, 200 OK)
- /api/galaxy/creator/data/note_detail_new (cookie auth, 200 OK)
- /api/galaxy/creator/data/note_detail?note_id=xxx (cookie auth, 200 OK)
- Note manager DOM extraction for note list (bypasses v2 signature)
All endpoints verified working with real creator account.
Screenshots (redacted) included in docs/screenshots/.
Requires: Chrome logged into creator.xiaohongshu.com
* chore: remove screenshots from repo (will host externally for PR)
* review: fix creator analytics CLI integration
Co-authored-by: stone16 <stone2paul@gmail.com>
* test: add site-scoped test runner
Co-authored-by: stone16 <stone2paul@gmail.com>
* review: ignore publish timestamps in creator note metrics
---------
Co-authored-by: jackwener <jakevingoo@gmail.com>
* feat: add download support for images, videos, and articles
Add comprehensive download functionality to OpenCLI with support for
multiple platforms and content types.
- Add `src/download/index.ts`: HTTP download with progress, yt-dlp
wrapper for video platforms, cookie export to Netscape format for
authenticated downloads
- Add `src/download/progress.ts`: Terminal progress bars, multi-file
download tracker with status summary
- Add `src/pipeline/steps/download.ts`: New `download` pipeline step
for declarative YAML pipelines
- Register `download` step in executor.ts
- Add template filters: `slugify`, `sanitize`, `ext`, `basename` for
filename templating
- `xiaohongshu download`: Download images and videos from notes
- `bilibili download`: Download videos using yt-dlp with cookie auth
- `twitter download`: Download media from user timeline or single tweet
- `zhihu download`: Export articles to Markdown with optional image
download
```yaml
pipeline:
- download:
url: ${{ item.imageUrl }}
dir: ./downloads
filename: ${{ item.title | sanitize }}.jpg
concurrency: 5
skip_existing: true
use_ytdlp: false
type: auto # auto|image|video|document
```
- Concurrent downloads with configurable parallelism
- Progress bars with file size display
- Skip existing files option
- Cookie forwarding for authenticated downloads
- yt-dlp integration for video platforms (YouTube, Bilibili, Twitter)
- HTML to Markdown conversion for article export
- yt-dlp: Required for video downloads from streaming platforms
- ffmpeg: Optional for video format conversion
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: add download support documentation
- Add Download Support section to both README.md and README.zh-CN.md
- Document supported platforms: Xiaohongshu, Bilibili, Twitter, Zhihu
- Include prerequisites (yt-dlp installation)
- Add usage examples for all download commands
- Document the `download` pipeline step for YAML adapters
- Update built-in commands table with new `download` commands
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve zhihu ordered list content
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
Add support for grok.com site with two commands:
- ask: Send a message to Grok and get response
- debug: Debug grok page structure
Implementation uses Playwright CDP protocol with fallback DOM selectors
(div.message-bubble, [data-testid="message-bubble"]) for reliability.
Co-authored-by: xdord <xdord@xdorddeMac-mini.local>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
WeChat Mac is native Cocoa (not Electron), so CDP is not available.
Uses AppleScript + clipboard automation instead:
- status: check if WeChat is running
- send: paste + Enter in active conversation
- new: Cmd+N for new chat
- search: Cmd+F and type query
- read: Cmd+A → Cmd+C to copy chat content
Total: 30 sites · 156 commands
Add linux.do (Discourse-based forum) support with 6 YAML pipeline commands:
- hot: trending topics with period filter (all/daily/weekly/monthly/yearly)
- latest: newest topics
- categories: list all categories with slug/id for further queries
- category: browse topics within a specific category
- topic: post details with replies (first page)
- search: search topics by keyword
All commands use navigate+evaluate pattern with cookie auth
(linux.do enforces login_required on all endpoints).
Security: user inputs sanitized via | json filter + encodeURIComponent.
HTML content stripped with block-tag spacing and full entity decoding.
Add two CLI commands for Jimeng (即梦AI) — ByteDance's AI image generation platform:
- generate: Text-to-image generation with model selection and configurable wait time
- history: View recent generation history with prompt, model, status, and image URLs
Both commands use browser automation with cookie-based authentication on jimeng.jianying.com.
- Remove --remote-allow-origins from antigravity README, README.zh-CN, SKILL.md (not needed for local usage)
- Update ChatGPT README to document both AppleScript and CDP approaches
- Document ChatGPT Electron launch: /Applications/ChatGPT.app/Contents/MacOS/ChatGPT --remote-debugging-port=9224
description: "Cross-project CLI command migration workflow for opencli. Use when importing commands from external CLI projects (python/node) like rdt-cli, twitter-cli, etc. Covers: source analysis → gap matrix → batch migration → README/SKILL.md update."
description: Migrate commands from an external CLI project into opencli adapters
---
// turbo-all
## Steps
1. Clone the source CLI project for analysis:
```bash
git clone <source_repo_url> /tmp/<source-cli>
```
2. Analyze source project: list all commands, auth method, API endpoints, and output fields.
3. Check existing opencli adapters for the target site:
```bash
ls src/clis/<site>/
opencli list | grep <site>
```
4. Generate a comparison matrix table (source commands vs opencli existing). Mark each as: ✅ **New** / ✅ **Enhance** / ❌ **Skip**. Ask user to confirm which commands to migrate.
5. Implement YAML Read adapters first (highest ROI, 10-20 lines each). Place files in `src/clis/<site>/<name>.yaml`.
6. Implement TS Read adapters for complex cases (GraphQL, pagination, signing). Place files in `src/clis/<site>/<name>.ts`.
7. Implement TS Write adapters using `Strategy.UI` or `Strategy.COOKIE`. Place files in `src/clis/<site>/<name>.ts`.
8. Verify build:
```bash
npx tsc --noEmit
```
9. Verify all commands are registered:
```bash
opencli list | grep <site>
```
10. Run each new command to verify it works:
```bash
opencli <site> <command> --limit 3 -f json
```
11. Update README.md with new command examples in the appropriate platform section.
12. Update SKILL.md Commands Reference with new commands.
13. Commit and push:
```bash
git add -A
git commit -m "feat(<site>): migrate <N> commands from <source-cli>"
Patch release focused on two architectural fixes around extension and daemon lifecycle, plus the first wave of the new site auth subsystem.
### Bug Fixes
* **extension 1.0.19** — close the MV3 Service Worker race that spawned duplicate `OpenCLI Adapter` tab groups (and, in the worst case, duplicate Adapter windows). The extension now persists the owned `windowId` immediately after `chrome.windows.create` returns and persists the owned `groupId` immediately after `chrome.tabs.group` returns, so a worker death between those API calls and the subsequent `chrome.tabGroups.update` no longer leaves a titleless orphan group and no longer drops the window pointer. Title-update failure no longer ungroups (it lets `ensureCanonicalGroupTitle` self-heal on the next ensure cycle), and `collectOwnedGroupCandidates` gains a fourth recovery layer: a global scan for empty-title groups containing a known owned `preferredTabId` for the role, with explicit hijack defense for user-built untitled groups. Closes the duplicate-tab-group bug report users had reported across the 1.8.2 window. ([#1862](https://github.com/jackwener/opencli/pull/1862))
* **daemon** — SIGKILL fallback when the stale daemon refuses graceful shutdown. After `npm install -g @jackwener/opencli@latest`, the CLI detects a version-mismatched daemon (`daemonVersion !== PKG_VERSION`), asks it to exit via `/shutdown`, and now — if the port is still held after 3 s — reads the stale daemon's pid from its own `/status` response and `process.kill(pid, 'SIGKILL')` (cross-platform: maps to `TerminateProcess` on Windows). The previous flow surfaced `Stale daemon could not be replaced` and asked users to run `opencli daemon stop && opencli doctor`; this is now automatic. ([#1861](https://github.com/jackwener/opencli/pull/1861))
* **xiaohongshu/publish** — prioritize the visible title input when the editor renders both a hidden draft input and a visible publish input.
* **xiaohongshu/publish** — accept inline topic suggestions with Enter when the dropdown lives inside a Shadow DOM surface, while still verifying the topic marker appears in the editor.
* **instagram/following** — paginate beyond the first endpoint page so high `--limit` values return more than the initial batch.
### Features
* **site auth subsystem** — new `opencli <site> login` and `opencli <site> whoami` commands, registered through a shared `clis/_shared/site-auth.js` helper. `login` opens the site's auth page in a foreground persistent session and polls the configured `verify` probe (cookie, JSON API, DOM scrape) until the browser session reports logged-in; `whoami` runs the same probe without opening the page. First five sites: twitter, github, bilibili, douyin, xiaohongshu. `whoami` outputs are PII-scrubbed (no email / phone / token in row columns). ([#1852](https://github.com/jackwener/opencli/pull/1852))
* **sitemaps/xiaohongshu** — Phase 2 sitemap content seeded with login schema dogfood, the first non-PoC consumer of the v1.1 sitemap schema. ([#1853](https://github.com/jackwener/opencli/pull/1853))
### Internal
* **test(e2e)** — raise `runCli``maxBuffer` so manifest-output snapshots no longer truncate on macOS / Windows CI.
Mid-cycle release: introduces the **Site Maps Hub** subsystem (agent-facing per-site navigation knowledge), restores the **smart-search** skill, and ships a wide batch of new adapters / commands plus a long tail of read-path fixes. Extension bumped to 1.0.18 for an owned-group reusable-tab scope fix.
### Site Maps Hub (new subsystem)
* **`sitemaps/<site>/` top-level seed directory** — sitemap content lives alongside `clis/` and `skills/`, parallel first-class repo citizens. Twitter and HackerNews seeded as v1 baselines.
* **`opencli browser open` / `analyze` surface sitemap availability** — when the requested site has a sitemap (global seed or local overlay `~/.opencli/sites/<site>/sitemap/`), the JSON envelope gains an optional `sitemap` field with `{ available, source, hint }`. `open` emits the hint once per session per site (deduped via `~/.opencli/cache/browser-sitemap-hints/`); `analyze` emits every call since it is a planning command. Adds no new browser-action behavior and no `~/.opencli/sites/` writes unless an agent explicitly invokes a sitemap skill.
* **Two new skills**:
*`opencli-sitemap-author` — create / maintain per-site sitemaps. Two-layer storage (global repo seed + local overlay), Form B compact YAML action schema with `pre / do / post / fail / recover / evidence`, `adapter_health_update` directives, `selector_pattern` as first-class anchor type, partial pages (`_<name>.md`) for cross-page UI, and a size-guidance table with hard 800-token / 1500-3000 cohesion / >3000 split tiers.
*`opencli-browser-sitemap` — consume site sitemaps while executing browser tasks. Lazy load, Trust-Reality rule (`browser state` is truth, sitemap is hint), stale-on-conflict writeback, `adapter_health` write-back closure so subsequent agents skip a known-suspect adapter.
* **`references/sitemap-schema.md`** — full field-level spec for `SITE.md / pages/<id>.md / workflows/<id>.md / apis.md / pitfalls.md`, action `state_signature` for re-entry, `adapter_health` enum, stable-id matching across overlay layers, draft placement rule, Phase 2 validation hooks.
* **Twitter + HackerNews v1.1 seeds** under `sitemaps/{twitter,hackernews}/` validating the schema on dense React UI and simple SSR HTML respectively.
### Features
* **smart-search** — restored as a skill (`skills/smart-search/`) with per-category source guides (AI / info / media / shopping / social / tech / travel / other).
* **kimi** — new adapter for `kimi.com` (21 commands).
* **qoder** — new adapter for Qoder IDE (19 commands).
* **trae-cn** — new desktop adapter (Trae CN Electron app).
* **trae-solo** — new desktop adapter (Trae SOLO Electron app).
* **chatgpt** — add web model switch command.
* **douyin** — add `search` command for keyword video search.
* **wechat-channels** — add WeChat Video Channels (视频号) publish adapter.
* **pubmed** — add workflow presets and richer article metadata.
### Bug Fixes
* **extension 1.0.18** — scope reusable-tab selection to owned-group members (follow-up to the v1.0.17 owned-container convergence model; ensures `findReusableOwnedContainerTab` does not pick up user tabs that were dragged into the owned window).
* **chatgpt** — ignore image placeholders and upload previews when extracting the latest assistant message.
* **xiaohongshu** — attach real topics via inline dropdown; feed returns signed note URLs for drill-down; carousel order preserved on download.
* **twitter** — drop global tweetPhoto selector from the post-submit poll to avoid matching the wrong button.
* **grok** — fall back to `Enter` key dispatch when send button is hidden behind layout shifts.
* **daemon** — differentiate multi-profile status output so multiple Chrome profiles do not collapse into a single status row.
* **youtube** — Videos tab fallback now supports `lockupViewModel` format alongside the legacy `gridVideoRenderer`.
* **12306** — accept lowercase letters in `train_no` regex.
* **weixin** — strip typographic quotes from pasted URLs.
* **douyin/publish** — handle illegal-title errors with a typed error rather than a silent retry.
### Docs
* **opencli-adapter-author** — add `references/strategy-selection.md` codifying the empirical contract ladder (PUBLIC_API / COOKIE_API / UI_SELECTOR / DOM_STATE as contracted vs PAGE_FETCH / INTERCEPT as internal-unstable, with fixes/adapter-year data from a 837-adapter / 30-day window) and update SKILL.md to require a `strategy` evidence block at the top of every new adapter.
* **opencli-adapter-author** — `browser analyze` upgrade: each candidate API gets `real_data_score` and a `likely_data` / `maybe_data` / `noise` verdict so Pattern A is no longer fired by analytics XHRs.
* **readme** — prefix "Let AI Agents operate any website" bullet with "Browser User &" in both EN and zh-CN.
Patch release focused on the extension tab-group convergence fix, plus 10 new adapters/commands and a wave of read-path / security hardening across browser, download, and adapters.
* **upwork** — add `search`, `feed`, and `detail` commands.
* **notebooklm** — add guarded write commands.
* **bilibili** — add comment commands.
* **weread** — add book search inside an open WeRead book.
* **linkedin** — consolidate read commands and add `profile-experience`.
* **xiaohongshu** — paginate `creator-notes` past the analyze list cap.
### Bug Fixes
* **extension 1.0.16** — ship the `OpenCLI Browser` / `OpenCLI Adapter` tab-group race fix from [#1693](https://github.com/jackwener/opencli/pull/1693). The extension now serializes owned tab-group creation per role so concurrent adapter/browser leases reuse the same group instead of creating duplicate same-title groups.
* **extension 1.0.17** — replace owned tab-group management with a Chrome-state-as-truth convergence model. The extension now keeps one canonical `OpenCLI Browser` / `OpenCLI Adapter` group per profile role, recovers renamed groups from stored hints or owned lease tabs, merges same-window and cross-window duplicates into the canonical group, and normalizes legacy or user-renamed container titles back to the canonical owned-container title. ⚠️ User-renamed `OpenCLI Browser` / `OpenCLI Adapter` groups are now force-renamed back; treat these as extension-managed automation containers, not user free-form bins. ([#1794](https://github.com/jackwener/opencli/pull/1794))
* **browser** — write the network response cache file with `0o600` owner-only permissions to keep captured response bodies out of other local users' reach.
* **download** — write the yt-dlp cookie file with `0o600` owner-only permissions.
* **pixiv** — migrate `user/detail` to the shared `pixivFetch` helper.
* **weread** — decode HTML entities in search results.
* **zhihu** — decode numeric HTML entities in text output. ([#1695](https://github.com/jackwener/opencli/pull/1695))
* **xiaohongshu** — hook dashboard fetch to capture signed `datacenter/note/*` responses ([#1732](https://github.com/jackwener/opencli/pull/1732)); preserve carousel order via `__INITIAL_STATE__.imageList` on download ([#1687](https://github.com/jackwener/opencli/pull/1687)).
* **bilibili** — subtitle support for bangumi / PGC bvid (番剧 / 纪录片 / 电影 / 综艺). ([#1669](https://github.com/jackwener/opencli/pull/1669))
* **suno** — derive current plan from subscription metadata.
* **douyin/hashtag** — validate action args before navigation.
* **byte-formatting** — stabilize byte formatting output.
### Docs
* **readme** — correct Node floor (>=20, not 21) and drop the Prerequisites section ([#1705](https://github.com/jackwener/opencli/pull/1705)); add CLI Hub brand aliases and split Exit Codes into the dedicated docs page ([#1685](https://github.com/jackwener/opencli/pull/1685)); drop the For Developers section ([#1684](https://github.com/jackwener/opencli/pull/1684)).
### Internal
* **ci** — disable Dependabot automated updates.
* **test(download)** — retry media-download Windows tests to absorb runner cold-start variance. ([#1708](https://github.com/jackwener/opencli/pull/1708))
Substantial release: a new official-API adapter (`weread-official`), wider LinkedIn / Twitter / Reddit / Zhihu coverage, the 12306 / Suno / Xianyu inbox additions, security and reliability fixes for the Browser Bridge and media downloads, plus a 20% README shrink. Node 20 compatibility is restored after an automated `undici` bump regression.
### Features
* **weread-official** — integrate WeRead's official Agent Gateway as the `weread-official` CLI namespace. Pure HTTP, Bearer auth via `WEREAD_API_KEY` (no browser, no cookies). 8 commands cover the official skill bundle: `search`, `shelf`, `book` (info + chapters + progress 3-in-1), `notes` (notebook overview or per-book highlights/thoughts), `review`, `readdata` (weekly/monthly/annually/overall), `discover` (recommend or similar-book), `list-apis`. Adapter surfaces typed errors for all documented failure modes — `AuthRequiredError` on missing/rejected key (errcodes -2010/-2012), `CommandExecutionError` on HTTP/`upgrade_info`/non-zero errcode, `EmptyResultError` on empty payloads. Coexists with the existing cookie-based `weread` adapter.
* **twitter** — rewrite the download-profile path on GraphQL UserMedia with cursor pagination. ([#1636](https://github.com/jackwener/opencli/issues/1636))
* **reddit** — expose `post_hint` / `url` / `preview` / `gallery` media routes on listing commands. ([#1676](https://github.com/jackwener/opencli/issues/1676))
* **zhihu** — add answer-comments reader; include answer links in question results.
* **chatgpt** — detect generated image surfaces (CSS background and canvas, not just `<img>`) so image generation works after UI drift. ([#1677](https://github.com/jackwener/opencli/issues/1677))
* **external** — add Cloudflare Wrangler as a built-in external CLI passthrough. ([#1679](https://github.com/jackwener/opencli/pull/1679))
### Bug Fixes
* **deps** — restore Node 20 runtime compatibility by pinning runtime `undici` back to the 6.x line (an automated dependabot bump to 8.x had moved the engines floor to Node ≥22.19, silently breaking the published Node 20 promise), and clear the docs build audit chain by overriding VitePress' Vite/PostCSS transitive dependencies to patched versions. ([#1673](https://github.com/jackwener/opencli/issues/1673))
* **download** — keep custom media filenames inside the requested output directory by stripping POSIX/Windows path components and sanitizing the generated fallback prefix. Prevents remote-controlled fields (e.g. video titles used as filename) from escaping the output directory via `../`. ([#1642](https://github.com/jackwener/opencli/pull/1642))
* **browser** — recover `Page.goto()` from stale page identities by clearing the cached targetId and retrying navigation once through the session lease; classify CDP `-32000 Cannot find default execution context` as retryable target navigation. ([#1645](https://github.com/jackwener/opencli/issues/1645))
* **cli** — escape leading-dash positional values via the argv preprocessor so users can pass tokens starting with `-` without commander mis-classifying them as flags. ([#1658](https://github.com/jackwener/opencli/issues/1658))
* **chatgpt/image** — fix ChatGPT web image generation after UI drift by letting the composer locator continue into the caller's readiness check and detecting generated images rendered as CSS backgrounds or canvases, not just plain `<img>` elements.
* **adapters** — surface the remaining `silent-empty-fallback` adapter failures as typed errors (Douyin user video comments, Jike SSR JSON parse, WeRead search-page fetch). True empty Douyin/Jike/WeRead result sets now throw `EmptyResultError`.
* **adapters** — drop silent-sentinel row fallbacks across Apple Podcasts / Reddit / Gitee. ([#1634](https://github.com/jackwener/opencli/issues/1634))
* **adapters** — migrate legal empty-data branches to `EmptyResultError` for `xhs` / YouTube and 5 follow-up commands. ([#1674](https://github.com/jackwener/opencli/issues/1674), [#1678](https://github.com/jackwener/opencli/issues/1678))
* **lesswrong** — drop the `"Unknown"` silent sentinel in the author column; missing authors now propagate as `null`. ([#1611](https://github.com/jackwener/opencli/issues/1611))
* **youtube/transcript** — scope timedtext URL matching to the current `videoId` across the in-page resource-buffer scan, the in-page fetch/XHR hook, and the Node-side CDP capture. SPA-style watch→watch navigation no longer returns a predecessor video's captions. ([#1655](https://github.com/jackwener/opencli/issues/1655))
* **twitter/lists** — skip the "Discover new Lists" recommendation block so it is no longer treated as one of the user's lists. ([#1652](https://github.com/jackwener/opencli/issues/1652))
* **zhihu** — decode numeric HTML entities in `answer-detail`. ([#1629](https://github.com/jackwener/opencli/issues/1629))
### Docs
* **readme** — major shrink and reframing: tagline rephrased around "Browser Use", Highlights and Update sections folded into adjacent content, Built-in Commands curated to 11 popular sites, CLI Hub table reduced to a name enumeration, Desktop App Adapters collapsed to a one-liner, skill-attribution references audited against `SKILL.md` frontmatter, "For AI Agents (Developer Guide)" merged into "Writing a new adapter". Net: EN 410 → 326 (-20%), ZH 455 → 371 (-18%). ([#1654](https://github.com/jackwener/opencli/pull/1654), [#1666](https://github.com/jackwener/opencli/pull/1666), [#1679](https://github.com/jackwener/opencli/pull/1679), [#1681](https://github.com/jackwener/opencli/pull/1681))
### Internal
* **audit** — stop flagging sentinel fallback strings inside thrown error messages as `silent-sentinel` violations. These are typed failure diagnostics rather than fake row data, reducing the typed-error baseline to actual adapter output fallbacks.
External CLI ergonomics + two adapter envelope/auth fixes. New `longbridge` external CLI entry; `opencli list` / root help now render human-readable brand labels for executables whose bare name is ambiguous.
### Features
* **external** — add the Longbridge CLI as a built-in external CLI passthrough (`opencli longbridge ...`) for Longbridge OpenAPI market data, account, and trading commands. ([#1584](https://github.com/jackwener/opencli/issues/1584))
* **external-cli** — render brand alias `name(package)` in `opencli list` and root help when the bare executable name is ambiguous. Built-in entries `ntn` → `ntn(notion)`, `dws` → `dws(DingTalk Workspace)`, `wecom-cli` → `wecom-cli(企业微信)` now self-explain in help output. `package` field is repurposed to cover both upstream distribution names (e.g. `tg-cli`) and human-readable brand labels (e.g. `notion`, `企业微信`). ([#1585](https://github.com/jackwener/opencli/issues/1585))
### Bug Fixes
* **boss** — map `code=24` (identity mismatch) to `AuthRequiredError` so re-login is signaled instead of surfacing as a generic API error. ([#1573](https://github.com/jackwener/opencli/issues/1573))
Adapter polish release: new web search adapters, better Browser Bridge tab group reuse, and social adapters returning to one-shot tab leases. Extension package version is bumped to 1.0.15 for the Browser Bridge fix.
### Features
* **search** — add DuckDuckGo, Brave, and Yahoo web search adapters. ([#1546](https://github.com/jackwener/opencli/issues/1546))
* **boss** — support job-seeker `chatlist` and `chatmsg` adapters. ([#1539](https://github.com/jackwener/opencli/issues/1539))
### Bug Fixes
* **extension** — reuse existing `OpenCLI Adapter` tab groups before creating new ones, including cross-window discovery, legacy `OpenCLI` title fallback, and deterministic candidate selection. ([#1541](https://github.com/jackwener/opencli/issues/1541))
* **twitter, reddit** — default browser-backed social adapters back to ephemeral tab leases. Twitter/X and Reddit commands now release their site tab after each run while keeping the shared Adapter window available for reuse; persistent sessions remain reserved for AI/chat-style adapters that need long-lived conversation state. ([#1569](https://github.com/jackwener/opencli/issues/1569))
External CLI surface cleanup + Browser Bridge WebSocket lifecycle hardening. Two BREAKING changes around external CLIs: built-in `tg`/`discord`/`wx` (was `tg-cli`/`discord-cli`/`wx-cli`) now match their real binary names, and Notion's in-tree CDP adapter is replaced by the official `ntn` external CLI.
### ⚠ BREAKING CHANGES
* **notion** — remove the in-tree `clis/notion/` CDP-on-Desktop adapter (8 commands: `status` / `search` / `read` / `new` / `write` / `sidebar` / `favorites` / `export`). Notion has shipped an official CLI at <https://ntn.dev>, registered as a first-class external CLI in `external-clis.yaml`. Migration: install `ntn` from <https://ntn.dev> (`curl -fsSL https://ntn.dev | bash`), then use `opencli ntn <command>`. Auto-install is intentionally not configured because the official installer is a shell script while OpenCLI external installs run shell-free command strings. The official CLI uses the public Notion API rather than reverse-engineering the Desktop UI, so it survives Notion app updates and exposes a wider command surface (blocks / databases / properties / comments) than the reverse-engineered adapter could. ([#1559](https://github.com/jackwener/opencli/issues/1559))
* **external** — drop the `-cli` suffix from built-in external CLI subcommand names. `opencli tg-cli`, `opencli discord-cli`, `opencli wx-cli` are now `opencli tg`, `opencli discord`, `opencli wx`, matching the real binary names that those tools install as. Root help still shows the package lineage as `tg(tg-cli)` / `discord(discord-cli)` / `wx(wx-cli)`. ([#1544](https://github.com/jackwener/opencli/issues/1544))
### Features
* **twitter** — `bookmarks` and `bookmark-folder` now include media via `extractMedia`, reaching parity with `timeline` / `search`. ([#1555](https://github.com/jackwener/opencli/issues/1555))
* **twitter/list-tweets** — include media via `extractMedia` (parity with `timeline` / `search`). ([#1464](https://github.com/jackwener/opencli/issues/1464))
### Bug Fixes
* **daemon** — report ambiguous browser command outcomes with a distinct `command_result_unknown` errorCode and `503` when the extension WebSocket drops between command dispatch and result delivery. `sendCommandRaw()` treats this code as hard non-retryable, so write-side commands (`navigate` / `click` / `type` / `eval`) won't be silently re-issued and double-executed. Daemon exposes a `commandResultUnknown` counter on `/status` for future observability. ([#1558](https://github.com/jackwener/opencli/issues/1558))
* **extension** — keep active daemon WebSocket; stale sockets no longer clobber active connection (`onopen` / `onclose` / `onmessage` are all gated by `ws !== thisWs` short-circuit), and `safeSend` only fires when `readyState === OPEN`. ([#1540](https://github.com/jackwener/opencli/issues/1540))
* **extension** — coalesce concurrent daemon WebSocket connects via an in-flight promise. Startup / keepalive / reconnect triggering `connect()` during the daemon-probe or context-lookup async gap no longer creates duplicate real WebSocket connections. ([#1554](https://github.com/jackwener/opencli/issues/1554))
* **external** — distinguish external CLI executable names from distribution/project names in root help. Built-in aliases such as `tg`, `discord`, `wx` remain the callable `opencli <name> ...` entrypoints while help renders `tg(tg-cli)`, `discord(discord-cli)`, `wx(wx-cli)` to show their package lineage. ([#1560](https://github.com/jackwener/opencli/issues/1560))
### Docs
* **browser** — clarify named session lifecycle in the Browser Bridge guide. ([#1542](https://github.com/jackwener/opencli/issues/1542))
Major hotfix + simplification batch. Extension bumped to 1.0.14. Node floor lowered to v20 so the long tail of Node v20–v21.6 users no longer crashes at module load. `opencli browser` user surface replaces required-flag `--session <name>` with a `<session>` positional. `page.evaluate(fn, ...args)` adds a type-safe alternative to the implicit auto-IIFE string form. Twitter cursor pagination no longer silently caps at ~500 items.
### ⚠ BREAKING CHANGES
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface. ([#1505](https://github.com/jackwener/opencli/issues/1505))
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell. ([#1509](https://github.com/jackwener/opencli/issues/1509))
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail. ([#1510](https://github.com/jackwener/opencli/issues/1510))
### Features
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic. ([#1508](https://github.com/jackwener/opencli/issues/1508))
* **twitter** — default `tweets` command to the logged-in user when `user` is omitted, and fix the sibling envelope-unwrap silent bug. ([#1531](https://github.com/jackwener/opencli/issues/1531))
* **zhihu** — add `answer-detail` to fetch a single answer's full content. ([#1528](https://github.com/jackwener/opencli/issues/1528))
* **zhihu** — paginate question answers and recommendations. ([#1517](https://github.com/jackwener/opencli/issues/1517))
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **twitter** — raise fixed cursor-pagination caps in `bookmarks` / `likes` / `tweets` / `timeline` / `bookmark-folder` / `list-tweets` / `search` / `following`. The old `i < 5` / `i < 10` literals and following's `Math.ceil(limit / 50) + 2` formula imposed hidden result ceilings below `--limit`; the loop now treats the page count as a high runaway guard while `--limit` and cursor exhaustion control normal pagination. ([#1532](https://github.com/jackwener/opencli/issues/1532))
* **twitter** — repair `list-add` / `list-tweets` / `lists` / `following` after 2026-05 site changes. ([#1503](https://github.com/jackwener/opencli/issues/1503))
* **twitter** — repair `search` and `tweets` readback. ([#1512](https://github.com/jackwener/opencli/issues/1512))
* **twitter** — make reply submission robust. ([#1511](https://github.com/jackwener/opencli/issues/1511))
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **google/search** — wrap evaluate return value in object to fix serialization. ([#1523](https://github.com/jackwener/opencli/issues/1523))
* **google-scholar/search** — wrap evaluate return to fix serialization. ([#1525](https://github.com/jackwener/opencli/issues/1525))
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **xiaohongshu+rednote/search** — fall back to href-based note cards when `section.note-item` class is dropped. ([#1507](https://github.com/jackwener/opencli/issues/1507))
* **xueqiu** — `kline` / `earnings-date` format dates in Asia/Shanghai instead of UTC. ([#1498](https://github.com/jackwener/opencli/issues/1498))
* **runtime** — lower the Node floor to `>=20.0.0`. Three coupled changes: drop all `util.styleText()` usage (added in Node v21.7.0 / v20.12.0; previously crashed v21.0–v21.6 at module load), downgrade `undici` from `^8.0.2` (engines `>=22.19.0`) to `^6.25.0` (engines `>=18.17`, retains `Agent` / `EnvHttpProxyAgent` / `fetch`), and lower `MIN_SUPPORTED_NODE_MAJOR` from 21 to 20 so the startup guard matches the declared `engines.node`. Smoke-tested on v20.0.0 / v21.2.0 / v22.22.2. The semantic markers (`[OK]` / `[WARN]` / `[FAIL]` / `ℹ` / `⚠` / `✖`) keep their meaning; ANSI colors were redundant for the primarily agent-facing CLI. ([#1524](https://github.com/jackwener/opencli/issues/1524))
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **ci** — drop `e2e-headed` and `adapter-test` from `pull_request` triggers (kept on `push` to main / nightly / `workflow_dispatch`). PR-time CI now targets ~2 min wall-time. ([#1521](https://github.com/jackwener/opencli/issues/1521), [#1522](https://github.com/jackwener/opencli/issues/1522))
* **scripts** — auto-refresh `dist/` before `build-manifest`. ([#1490](https://github.com/jackwener/opencli/issues/1490))
Hotfix release for the 1.7.17 doctor regression: `opencli doctor` failed connectivity probe with `Browser session is required` because the doctor probe didn't pass a session to the new strict-session browser bridge. Also adds new adapters and adapter fixes that were ready immediately after 1.7.17.
### Bug Fixes
* **doctor** — pass an internal `__doctor__` browser session to the live connectivity probe so `opencli doctor` works again under the explicit-session browser model introduced in 1.7.17. ([#1485](https://github.com/jackwener/opencli/issues/1485))
* **browser** — `--session <name>` is now declared as a `requiredOption` so Commander itself rejects calls missing the flag before runtime, and the help line is marked `(required)` instead of being hidden under `Options:`. ([#1485](https://github.com/jackwener/opencli/issues/1485))
* **doubao/ask** — restore Assistant detection after the 2026-05 DOM refactor. ([#1484](https://github.com/jackwener/opencli/issues/1484))
* **youtube** — request `srv3` format for caption URLs. ([#1422](https://github.com/jackwener/opencli/issues/1422))
Extension bumped to 1.0.12 (workspace → session lease routing, drop `handleSessions` handler). Major simplification pass: browser/adapter session model rewrite, `--workspace` removed, doctor surface trimmed to its core job.
### ⚠ BREAKING CHANGES
* **browser session model** — replace the browser-facing `--workspace` model with explicit `--session <name>` on `opencli browser *`. Browser commands now require a session name, `browser bind`/`unbind` use `--session`, and bind no longer accepts `--domain`, `--path-prefix`, or `--allow-navigate-bound`. Browser primitives keep their session tab by design; the browser namespace no longer exposes `--keep-tab`. ([#1461](https://github.com/jackwener/opencli/issues/1461))
* **adapter site sessions** — replace adapter metadata `browserSession: { reuse: 'site' }` with `siteSession: 'persistent'`, and replace the user override `--reuse <none|site>` / `OPENCLI_BROWSER_REUSE` with `--site-session <ephemeral|persistent>`. Persistent site sessions keep a stable site tab open without idle expiry. ([#1462](https://github.com/jackwener/opencli/issues/1462))
* **doctor** — remove `--no-live` and `--sessions` flags from `opencli doctor`. Doctor always runs the live browser connectivity probe (that's its core job); session enumeration was never part of health diagnosis. The underlying `'sessions'` daemon protocol action and the `BrowserSessionInfo` public type are removed as dead code. ([#1470](https://github.com/jackwener/opencli/issues/1470))
### Features
* **chatgpt** — `ask` and `send` now accept local image paths and upload them through the composer before submitting the prompt. ([#1476](https://github.com/jackwener/opencli/issues/1476))
### Internal
* **extension 1.0.12** — drop `handleSessions` action handler (no remaining consumers after doctor cleanup).
* **extension 1.0.11** — switch Browser Bridge lease routing from user-facing workspaces to explicit browser sessions.
* **external** — register `tg-cli`, `discord-cli`, and `wx-cli` as external CLI integrations. ([#1458](https://github.com/jackwener/opencli/issues/1458))
### Bug Fixes
* **xiaohongshu** — fall back to base64 upload when CDP `DOM.setFileInputFiles` returns `Not allowed` on creator center. ([#1374](https://github.com/jackwener/opencli/issues/1374))
* **chatgpt** — switch to locale-stable send button selector so non-English UIs don't break send. ([#1354](https://github.com/jackwener/opencli/issues/1354))
### Performance
* **adapters** — hoist cookie reads to `page.getCookies` across Tier 1 (25 files), eliminating per-call CDP round trips. ([#1450](https://github.com/jackwener/opencli/issues/1450))
* **twitter** — drop redundant `goto + wait` in adapter steps; framework auto pre-navigates. ([#1451](https://github.com/jackwener/opencli/issues/1451))
* **twitter** — enable `browserSession.reuse: 'site'` on 17 read-only adapters so repeated reads share one tab. ([#1454](https://github.com/jackwener/opencli/issues/1454))
* **browser** — split interactive and automation windows so `opencli browser *` and adapter-driven background commands no longer share one Chrome window; tab groups are isolated by role.
### Internal
* **extension 1.0.10** — rename the adapter-owned Chrome tab group from `OpenCLI Automation` to `OpenCLI Adapter`. ([#1457](https://github.com/jackwener/opencli/issues/1457))
* **docs** — list `tg-cli`, `discord-cli`, `wx-cli` in External CLI README sections. ([#1459](https://github.com/jackwener/opencli/issues/1459))
Extension bumped to 1.0.9 (Accessibility.enable allowlist + downloads permission + cross-origin frame target attach for AX). Major Browser Agent Runtime release: full Phase 0/1/2 alignment with `vercel-labs/agent-browser` model — CDP-primary input, AX snapshot/refs with stale recovery, semantic locators across all primitives, full form toolbelt (hover/focus/dblclick/check/uncheck/upload/drag/wait-download), annotated screenshots, and same-origin iframe AX routing. Cross-origin OOPIF AX is best-effort (Chrome extension API limitation).
### ⚠ BREAKING CHANGES
* **browser lifecycle** — replace `--focus` / `OPENCLI_WINDOW_FOCUSED` with `--window foreground|background` / `OPENCLI_WINDOW`, and replace `--live` / `OPENCLI_LIVE` with `--keep-tab true|false` / `OPENCLI_KEEP_TAB`. `opencli browser *` defaults to a foreground window and keeps its tab; browser-backed adapter commands default to a background automation window and release their tab unless the adapter uses site-level reuse.
### Features
* **help / browser** — `opencli browser --help -f yaml|json` now emits a structured, agent-ready index of all browser leaf commands (including nested `tab`, `get`, and `dialog` commands), their positionals, command options, namespace options, and root global options. Individual browser commands also support structured help, backed by a shared Commander option/argument spec extractor.
* **help / built-in namespaces** — `opencli daemon|plugin|adapter|profile --help -f yaml|json` now emit the same structured payload as `browser`. One agent call returns every leaf's positionals, options, descriptions, and global options — no per-leaf `--help` follow-ups needed. Original namespace descriptions are preserved through `applyRootSubcommandSummaries()` via a snapshot at namespace declaration time.
* **browser state** — add opt-in AX snapshot refs via `browser state --source ax`, including backend-node click resolution and role/name stale-ref recovery for the Phase 0 browser-agent runtime prototype.
* **browser state** — AX snapshots now include same-origin iframe refs, and `browser state --compare-sources` prints DOM-vs-AX observation metrics for the Phase 1 default-source decision without dumping page contents.
* **browser locators** — `browser find`, `browser click`, and `browser get text|value|attributes` now accept semantic locator flags (`--role`, `--name`, `--label`, `--text`, `--testid`) so agents can act on common controls without a separate state-ref lookup.
* **browser locators** — semantic locator flags now work across input/action primitives (`type`, `fill`, `select`, `hover`, `focus`, `dblclick`, `check`, `uncheck`, `upload`) plus prefixed `--from-*` / `--to-*` locators for `drag`.
* **browser actions** — add `browser hover`, `browser focus`, and `browser dblclick` primitives backed by the same target resolver and CDP input path as `browser click`.
* **browser actions** — add `browser check` and `browser uncheck` primitives that ensure checkbox / radio / aria-checked controls reach the requested state instead of blindly toggling.
* **browser upload** — add `browser upload <target> <file...>` to attach local files to `input[type=file]` targets through CDP `DOM.setFileInputFiles`, with local path validation and file-input verification.
* **browser actions** — add `browser drag <source> <target>` for CDP mouse drag sequences between two resolved element centers.
* **browser wait / extension 1.0.8** — add `browser wait download [pattern]` backed by Chrome's downloads lifecycle API, so agents can wait for file downloads by filename/URL pattern and receive completed/failed download metadata.
* **browser state / extension 1.0.9** — AX snapshots can now route same-origin iframe refs through `frameId`. Cross-origin OOPIF AX routing is best-effort because real Chrome extension smoke tests show `chrome.debugger` may not expose attachable iframe targets to extensions.
* **browser screenshot** — add `browser screenshot --annotate`, which refreshes DOM refs and overlays visible `[N]` labels on the screenshot so visual inspection maps back to `browser click <ref>` targets.
### Bug Fixes
* **browser click** — `browser click` now prefers CDP `Input.dispatchMouseEvent` over DOM `el.click()`, so custom dropdowns that depend on pointer/mouse events (Radix, shadcn, Material UI, Mercury-style category pickers) open and select reliably while retaining JS click as a fallback for older backends or zero-rect targets.
* **browser state / extension 1.0.7** — `browser state --source ax` now enables the CDP Accessibility domain before reading the AX tree, fixing real-Chrome snapshots that previously returned only `RootWebArea` with zero refs.
* **help / build** — every positional arg must now declare a non-empty `help` string. The build-manifest step fails closed when a positional has empty / whitespace-only / missing `help`, so `opencli <site> <cmd> --help` always shows callers what each parameter is for. Pre-existing offenders (`twitter followers/following/list-add/list-remove/list-tweets/search/thread`, `reddit search/subreddit/user/user-comments/user-posts`, `douyin stats/update`, `bilibili subtitle`, `jike search`) now have explicit help text — most notably `twitter followers [user]` and `following [user]` now document that omitting the user fetches the currently logged-in account.
* **help** — adapter help is now agent-friendly: per-command listings drop the `[options]` noise from globally-shared options (`--format`, `--trace`, `-v`, `-h`, etc.) and only mention them at the site level, so `opencli twitter` etc. read like a flat command index. ([#1401](https://github.com/jackwener/opencli/issues/1401))
* **twitter** — write-action symmetry P0: add `unlike`, `retweet`, `unretweet`, and `quote` to round out the read/write coverage. ([#1400](https://github.com/jackwener/opencli/issues/1400))
### Bug Fixes
* **browser daemon** — `npm install -g @jackwener/opencli@latest` now correctly auto-restarts a stale ready-state daemon so users pick up the new version without a manual `opencli daemon restart`. ([#1399](https://github.com/jackwener/opencli/issues/1399))
Extension bumped to 1.0.6 (screenshot `--width` / `--height` / `--full-page` flags, automation tab group color marker, automation container reuse fix).
### Bug Fixes
* **xiaohongshu** — fix `publish --topics` leaving bare `#` characters with no linked topics. The adapter now types `#keyword` into the body editor to trigger the inline suggestion dropdown and selects the matching topic, matching the current creator-center UI.
### ⚠ BREAKING CHANGES
* **linux-do** — remove deprecated compatibility shims `linux-do hot`, `linux-do category`, `linux-do latest`. Use `linux-do feed --view top --period <period>`, `linux-do feed --category <id-or-name>`, and `linux-do feed --view latest` instead.
* **grok ask** — drop the `--web` flag and the legacy `<textarea>` composer path. The default flow is now the only path and uses the current ProseMirror+TipTap composer (the path that used to require `--web true`). Existing scripts passing `--web` will get an "unknown option" error from commander; remove the flag.
* **env** — rename `OPENCLI_BROWSER_TIMEOUT` to `OPENCLI_BROWSER_IDLE_TIMEOUT`. The variable controls workspace lease idle release time, not per-command runtime; the new name reflects that. Old name was undocumented and removed without a fallback.
* **registry** — remove the unused `Strategy.HEADER`; adapter authors should use `Strategy.COOKIE` and set headers explicitly inside browser-side fetches.
### Features
* **observation** — add trace artifact primitives, `browser console`, `browser network --since/--follow/--failed`, and adapter `--trace=retain-on-failure` for failure-retained browser evidence.
* **autofix** — retire `OPENCLI_DIAGNOSTIC`; adapter repair now uses `--trace retain-on-failure`, trace `summary.md`, and error-envelope trace metadata.
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
* **browser session** — adapter commands can opt into site-level tab reuse with `browserSession.reuse = 'site'`; Grok and other browser-backed LLM adapters now keep a shared site tab by default, and users can override with `--reuse <none|site>`.
* **grok** — add browser-web baseline commands: `read`, `history`, `detail`, `new`, `send`, and `status` (existing `ask` and `image` unchanged).
* **yuanbao** — add browser-web baseline commands: `send`, `status`, `read`, `history`, and `detail` (joining the existing `ask` and `new`).
* **qwen** — add `detail` command for opening a specific historical conversation by id.
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
### Bug Fixes
* **pipeline / capabilityRouting** — the `fill` pipeline step (introduced in [#1222](https://github.com/jackwener/opencli/issues/1222)) now correctly triggers a browser session and gets transient retry coverage; previously a pipeline using only `fill` could crash on a missing page object. ([#1393](https://github.com/jackwener/opencli/issues/1393))
* **xiaohongshu publish** — improve image publishing reliability via creator-center URL routing, tab priority handling, and DataTransfer fallback.
* **youtube** — use watch-page HTML for transcript captions to recover when the public transcript API is unavailable.
* **desktop adapters** — restore 11 desktop adapter commands that were lost from the manifest due to a factory-pattern regression.
### Internal
* **cleanup** — remove dead `src/analysis.ts` (179 lines, 0 importers), retire `OPENCLI_DIAGNOSTIC` test residue, derive validator step allowlist from the live pipeline registry to prevent future drift.
Extension bumped to 1.0.2 (body-truncation signal unified across raw / detail / fallback paths).
### Features
* **Window lifecycle flags** — `--live` (or `OPENCLI_LIVE=1`) keeps the automation window open after a command finishes; `--focus` (or `OPENCLI_WINDOW_FOCUSED=1`) brings the window to the foreground. Works on any subcommand. ([#1122](https://github.com/jackwener/opencli/issues/1122))
* **Agent-native browser payload** — structured `network` bodies with truncation signal, `get html --as json` with `--depth` / `--children-max` / `--text-max` budgets, new `browser extract` command for long-form content with resume cursor. ([#1104](https://github.com/jackwener/opencli/issues/1104))
* **`network --filter <fields>`** — filter captured requests by body-shape path segments for quick API discovery. ([#1103](https://github.com/jackwener/opencli/issues/1103))
* **`get html --as json`** — structured HTML tree output; no more silent truncation on raw `--as html`. ([#1102](https://github.com/jackwener/opencli/issues/1102))
* **`browser network` rewrite** — agent-native discovery with cache keys and shape preview. ([#1100](https://github.com/jackwener/opencli/issues/1100))
* **Compound form fields** — date / select / file controls surface a `compound` envelope with format, options, `accept`. Cascading stale-ref recovery + bbox 0.99 dedup for tagged elements. ([#1116](https://github.com/jackwener/opencli/issues/1116))
* **twitter `tweets`** — fetch a user's recent posts. ([#1098](https://github.com/jackwener/opencli/issues/1098))
* **bilibili `video`** — new video command. ([#1110](https://github.com/jackwener/opencli/issues/1110))
* **deepseek `--file`** — file upload support on `ask`. ([#1093](https://github.com/jackwener/opencli/issues/1093))
### Bug Fixes
* **twitter** — 5s timeout on `resolveTwitterQueryId` to prevent hang. ([#1106](https://github.com/jackwener/opencli/issues/1106))
* **youtube** — fall back to Videos tab when Home has no videos. ([#1109](https://github.com/jackwener/opencli/issues/1109))
* **jianyu** — keep accessible detail urls in search. ([#1099](https://github.com/jackwener/opencli/issues/1099))
* **xiaoyuzhou** — migrate from broken SSR to authenticated API ([#1059](https://github.com/jackwener/opencli/issues/1059)); accept `CONFIG_ERROR` in E2E guard ([#1066](https://github.com/jackwener/opencli/issues/1066))
* **xiaohongshu** — detect draft save success ([#1060](https://github.com/jackwener/opencli/issues/1060)); verify title input sticks on publish ([#1050](https://github.com/jackwener/opencli/issues/1050))
* **twitter** — repair lists scraping from detail pages ([#1053](https://github.com/jackwener/opencli/issues/1053))
* **zsxq** — separate content from title, remove title truncation ([#1079](https://github.com/jackwener/opencli/issues/1079))
* **extension** — per-workspace idle timeout for browser sessions ([#1064](https://github.com/jackwener/opencli/issues/1064))
This is a major release with significant internal architecture changes.
Adapter code, validation, and error handling have been modernized.
### ⚠ BREAKING CHANGES
* **Node.js >= 21 required** — `import.meta.dirname` is used in core modules; Node 20 and below will fail at startup.
* **YAML adapters deprecated** — YAML-based `.yaml` adapters are no longer loaded. Existing YAML adapters must be converted to JS via `cli()` API. A deprecation warning is emitted if `.yaml` files are detected.
* **`.ts` adapters no longer loaded at runtime** — The runtime only discovers `.js` files. If you have `.ts` adapters in `~/.opencli/clis/`, compile them to `.js` or rewrite using plain JS. A warning is printed when `.ts` files without a matching `.js` are found.
* **Error output format changed** — All errors are now emitted as a structured YAML envelope to stderr. Scripts parsing stdout for `[{error, help}]` must switch to stderr / exit code. ([#923](https://github.com/jackwener/opencli/issues/923))
* **`tabId` replaced by `targetId`** — Cross-layer page identity now uses `targetId`. Extensions and plugins referencing `tabId` must update. ([#899](https://github.com/jackwener/opencli/issues/899))
* **`operate` renamed to `browser`** — All `opencli operate` commands are now `opencli browser`. ([#883](https://github.com/jackwener/opencli/issues/883))
### Features
* **auto-close adapter windows** — Browser tabs opened by adapters are automatically closed after execution; configurable via `OPENCLI_WINDOW_FOCUSED`. ([#915](https://github.com/jackwener/opencli/issues/915))
* **auto-downgrade to YAML in non-TTY** — Machine-readable output when piped. ([#737](https://github.com/jackwener/opencli/issues/737))
* **Browser Use improvements** — Better click/type/state handling for browser automation. ([#707](https://github.com/jackwener/opencli/issues/707))
* **CDP session-level network capture** — Full network capture support for CDPPage. ([#815](https://github.com/jackwener/opencli/issues/815), [#816](https://github.com/jackwener/opencli/issues/816))
* **twitter:** relax reply composer timeout, use composer for text replies ([#862](https://github.com/jackwener/opencli/issues/862), [#860](https://github.com/jackwener/opencli/issues/860))
* **gemini:** stabilize ask reply state handling ([#735](https://github.com/jackwener/opencli/issues/735))
* **douban:** fix marks pagination and improve subject data extraction ([#752](https://github.com/jackwener/opencli/issues/752))
* **jianyu:** avoid early API bucket cutoff, stabilize search ([#916](https://github.com/jackwener/opencli/issues/916), [#912](https://github.com/jackwener/opencli/issues/912))
* **xiaoe:** resolve missing episodes for long courses via auto-scroll ([#904](https://github.com/jackwener/opencli/issues/904))
### Refactoring
* **adapters:** convert adapter layer from TypeScript to JavaScript ([#928](https://github.com/jackwener/opencli/issues/928))
* **adapters:** migrate all CLI adapters from YAML to TypeScript, then to JS ([#887](https://github.com/jackwener/opencli/issues/887), [#922](https://github.com/jackwener/opencli/issues/922))
* **validate:** switch from YAML-file scanning to registry-based validation ([#943](https://github.com/jackwener/opencli/issues/943))
* **strategy:** normalize strategy into runtime fields at registration time ([#941](https://github.com/jackwener/opencli/issues/941))
* **errors:** unify error output as YAML envelope to stderr ([#923](https://github.com/jackwener/opencli/issues/923))
* **daemon:** make daemon persistent, remove idle timeout ([#913](https://github.com/jackwener/opencli/issues/913))
* fix stale `.ts` references across skills and docs ([#954](https://github.com/jackwener/opencli/issues/954))
* unify skill command references and merge opencli-generate into opencli-explorer ([#891](https://github.com/jackwener/opencli/issues/891), [#894](https://github.com/jackwener/opencli/issues/894))
### Upgrade Guide
1.**Update Node.js** to v21 or later (v22 LTS recommended).
2.**Run `npm install -g @jackwener/opencli@latest`** — the preuninstall hook gracefully stops the old daemon; the first browser command after upgrade auto-restarts it.
3.**If you have custom `.ts` adapters** in `~/.opencli/clis/`, rename or compile them to `.js`. A warning will be printed on startup if stale `.ts` files are detected.
4.**If you have custom `.yaml` adapters**, convert them to JS using the `cli()` API (see `skills/opencli-adapter-author/references/adapter-template.md`).
5.**If you parse error output from stdout**, switch to stderr. Errors are now structured YAML envelopes with typed exit codes.
* **douyin:** repair creator draft flow — switch from broken API pipeline to UI-driven approach ([#640](https://github.com/jackwener/opencli/issues/640))
* **douyin:** support current creator API response shapes for activities, profile, collections, hashtag, videos ([#618](https://github.com/jackwener/opencli/issues/618))
* **bilibili:** distinguish login-gated subtitles from empty results ([#645](https://github.com/jackwener/opencli/issues/645))
* **facebook:** avoid in-page redirect in search — use navigate step instead of window.location.href ([#642](https://github.com/jackwener/opencli/issues/642))
* **substack:** update selectors for DOM redesign ([#624](https://github.com/jackwener/opencli/issues/624))
* **weread:** recover book details from cached shelf fallback ([#628](https://github.com/jackwener/opencli/issues/628))
* **docs:** use relative links in adapter index ([#629](https://github.com/jackwener/opencli/issues/629))
* **weibo,youtube:** add Weibo commands and YouTube channel/comments ([#418](https://github.com/jackwener/opencli/issues/418))
* **twitter:** add filter option for search ([#410](https://github.com/jackwener/opencli/issues/410))
* **extension:** add popup UI, privacy policy, and CSP for Chrome Web Store ([#415](https://github.com/jackwener/opencli/issues/415))
* add url field to 9 search adapters (67% -> 97% coverage) ([#414](https://github.com/jackwener/opencli/issues/414))
### Bug Fixes
* **extension:** improve UX when daemon is not running — show hint in popup, reduce reconnect noise ([#424](https://github.com/jackwener/opencli/issues/424))
* fix ChatWise Windows connect ([#405](https://github.com/jackwener/opencli/issues/405))
* resolve 6 critical + 11 important bugs from deep code review ([#337](https://github.com/jackwener/opencli/issues/337), [#340](https://github.com/jackwener/opencli/issues/340))
* **stealth:** harden anti-detection against advanced fingerprinting ([#357](https://github.com/jackwener/opencli/issues/357))
### Code Quality
* replace all `catch (err: any)` with typed `getErrorMessage()` across 13 files ([#412](https://github.com/jackwener/opencli/issues/412))
* adopt CliError subclasses in social and desktop adapters ([#367](https://github.com/jackwener/opencli/issues/367), [#372](https://github.com/jackwener/opencli/issues/372), [#375](https://github.com/jackwener/opencli/issues/375))
* simplify codebase with type dedup, shared analysis module, and consistent naming ([#373](https://github.com/jackwener/opencli/issues/373))
* **ci:** add cross-platform CI matrix (Linux/macOS/Windows) ([#402](https://github.com/jackwener/opencli/issues/402))
* **plugin:** add update command, hot reload after install, README section ([#307](https://github.com/jackwener/opencli/issues/307)) ([966f6e5](https://github.com/jackwener/opencli/commit/966f6e5))
* **yollomi:** add new commands and update documentation ([#235](https://github.com/jackwener/opencli/issues/235)) ([ea83242](https://github.com/jackwener/opencli/commit/ea83242))
* **record:** add live recording command for API capture ([#300](https://github.com/jackwener/opencli/issues/300)) ([dff0fe5](https://github.com/jackwener/opencli/commit/dff0fe5))
* ensure standard PATH is available for external CLIs ([#285](https://github.com/jackwener/opencli/issues/285)) ([22f5c7a](https://github.com/jackwener/opencli/commit/22f5c7a))
* **daemon:** harden security against browser CSRF attacks ([#268](https://github.com/jackwener/opencli/issues/268)) ([40bd11d](https://github.com/jackwener/opencli/commit/40bd11d))
### Performance
* smart page settle via DOM stability detection ([#271](https://github.com/jackwener/opencli/issues/271)) ([4b976da](https://github.com/jackwener/opencli/commit/4b976da))
### Refactoring
* doctor defaults to live mode, remove setup command entirely ([#263](https://github.com/jackwener/opencli/issues/263)) ([b4a8089](https://github.com/jackwener/opencli/commit/b4a8089))
* replace all about:blank with data: URI to prevent New Tab Override interception ([#257](https://github.com/jackwener/opencli/issues/257)) ([3e91876](https://github.com/jackwener/opencli/commit/3e91876))
* harden resolveTabId against New Tab Override extension interception ([#255](https://github.com/jackwener/opencli/issues/255)) ([112fdef](https://github.com/jackwener/opencli/commit/112fdef))
* **wikipedia:** fix search arg name + add random and trending commands ([#231](https://github.com/jackwener/opencli/issues/231)) ([1d56dd7](https://github.com/jackwener/opencli/commit/1d56dd7))
* resolve inconsistent doctor --live report (fix [#121](https://github.com/jackwener/opencli/issues/121)) ([#224](https://github.com/jackwener/opencli/issues/224)) ([387aa0d](https://github.com/jackwener/opencli/commit/387aa0d))
* **twitter:** add --type flag to timeline command ([#83](https://github.com/jackwener/opencli/issues/83)) ([e98cf75](https://github.com/jackwener/opencli/commit/e98cf75))
* **tiktok:** add TikTok adapter with 15 commands ([#202](https://github.com/jackwener/opencli/issues/202)) ([4391ccf](https://github.com/jackwener/opencli/commit/4391ccf))
* add Lobste.rs, Instagram, and Facebook adapters ([#199](https://github.com/jackwener/opencli/issues/199)) ([ce484c2](https://github.com/jackwener/opencli/commit/ce484c2))
* **medium:** add medium adapter ([#190](https://github.com/jackwener/opencli/issues/190)) ([06c902a](https://github.com/jackwener/opencli/commit/06c902a))
* plugin system (Stage 0-2) ([1d39295](https://github.com/jackwener/opencli/commit/1d39295))
* make primary args positional across all CLIs ([#242](https://github.com/jackwener/opencli/issues/242)) ([9696db9](https://github.com/jackwener/opencli/commit/9696db9))
* **xueqiu:** make primary args positional ([#213](https://github.com/jackwener/opencli/issues/213)) ([fb2a145](https://github.com/jackwener/opencli/commit/fb2a145))
### Refactoring
* replace hardcoded skipPreNav with declarative navigateBefore field ([#208](https://github.com/jackwener/opencli/issues/208)) ([a228758](https://github.com/jackwener/opencli/commit/a228758))
* type discovery core ([#219](https://github.com/jackwener/opencli/issues/219)) ([bd274ce](https://github.com/jackwener/opencli/commit/bd274ce))
* type browser core ([#218](https://github.com/jackwener/opencli/issues/218)) ([28c393e](https://github.com/jackwener/opencli/commit/28c393e))
* type pipeline core ([#217](https://github.com/jackwener/opencli/issues/217)) ([8a4ea41](https://github.com/jackwener/opencli/commit/8a4ea41))
* reduce core any usage ([#216](https://github.com/jackwener/opencli/issues/216)) ([45cee57](https://github.com/jackwener/opencli/commit/45cee57))
* fail fast on invalid pipeline steps ([#237](https://github.com/jackwener/opencli/issues/237)) ([c76f86c](https://github.com/jackwener/opencli/commit/c76f86c))
* add antigravity serve command — Anthropic API proxy ([35a0fed](https://github.com/jackwener/opencli/commit/35a0fed8a0c1cb714298f672c19f017bbc9a9630))
* add arxiv and wikipedia adapters ([#132](https://github.com/jackwener/opencli/issues/132)) ([3cda14a](https://github.com/jackwener/opencli/commit/3cda14a2ab502e3bebfba6cdd9842c35b2b66b41))
* add external CLI hub for discovery, auto-installation, and execution of external tools. ([b3e32d8](https://github.com/jackwener/opencli/commit/b3e32d8a05744c9bcdfef96f5ff3085ac72bd353))
* **boss:** add 8 new recruitment management commands ([#133](https://github.com/jackwener/opencli/issues/133)) ([7e973ca](https://github.com/jackwener/opencli/commit/7e973ca59270029f33021a483ca4974dc3975d36))
* **serve:** implement auto new conv, model mapping, and precise completion detection ([0e8c96b](https://github.com/jackwener/opencli/commit/0e8c96b6d9baebad5deb90b9e0620af5570b259d))
* **serve:** use CDP mouse click + Input.insertText for reliable message injection ([c63af6d](https://github.com/jackwener/opencli/commit/c63af6d41808dddf6f0f76789aa6c042f391f0b0))
* **docs:** use base '/' for custom domain and add CNAME file ([#129](https://github.com/jackwener/opencli/issues/129)) ([2876750](https://github.com/jackwener/opencli/commit/2876750891bc8a66be577b06ead4db61852c8e81))
* **serve:** update model mappings to match actual Antigravity UI ([36bc57a](https://github.com/jackwener/opencli/commit/36bc57a9624cdfaa50ffb2c1ad7f9c518c5e6c55))
* type safety for wikiFetch and arxiv abstract truncation ([4600b9d](https://github.com/jackwener/opencli/commit/4600b9d46dc7b56ff564c5f100c3a94c6a792c06))
* use UTC+8 for XHS timestamp formatting (CI timezone fix) ([03f067d](https://github.com/jackwener/opencli/commit/03f067d90764487f0439705df36e1a5c969a7f98))
* **xiaohongshu:** use fixed UTC+8 offset in trend timestamp formatting (CI timezone fix) ([593436e](https://github.com/jackwener/opencli/commit/593436e4cb5852f396fbaaa9f87ef1a0b518e76d))
* use %20 instead of + for spaces in Bilibili WBI signed requests ([#126](https://github.com/jackwener/opencli/issues/126)) ([4cabca1](https://github.com/jackwener/opencli/commit/4cabca12dfa6ca027b938b80ee6b940b5e89ea5c)), closes [#125](https://github.com/jackwener/opencli/issues/125)
Install the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md) if you need the full adapter workflow — recon → API discovery → field decoding → `opencli browser verify`.
### Validate Your Adapter
```bash
# Validate adapter
opencli validate
# Test your command
opencli <site> <command> --limit 3 -f json
# Verbose mode for debugging
opencli <site> <command> -v
```
## Arg Design Convention
Use **positional** for the primary, required argument of a command (the "what" — query, symbol, id, url, username). Use **named options** (`--flag`) for secondary/optional configuration (limit, format, sort, page, filters, language, date).
**Rule of thumb**: Think about how the user will type the command. `opencli xueqiu stock SH600519` is more natural than `opencli xueqiu stock --symbol SH600519`.
Do **not** convert an argument to positional just because it appears first in the file. If the argument is optional, acts like a filter, or selects a mode/configuration, it should usually stay a named option.
The OpenCLI Browser Extension is a bridge between the [OpenCLI](https://github.com/jackwener/opencli) command-line tool and your Chrome browser. It receives commands from a **locally running daemon** process via WebSocket (`localhost` only) and executes them in **isolated Chrome windows** that are separate from your normal browsing session.
## Data collection
The extension does **NOT** collect, store, transmit, or sell any personal data. Specifically:
- **No analytics or telemetry** — no data is sent to any remote server.
- **No user tracking** — no cookies, identifiers, or fingerprints are created.
- **No external network requests** — all communication is strictly `localhost` (WebSocket to `ws://localhost:19825`).
## Permissions explained
| Permission | Why it's needed |
|------------|----------------|
| `debugger` | Required to use Chrome DevTools Protocol (CDP) for browser automation — executing JavaScript, capturing page content, and taking screenshots in isolated windows. |
| `tabs` | Required to create and manage isolated automation windows and tabs, separate from the user's browsing session. |
| `cookies` | Required to read site-specific cookies (scoped by domain) so CLI commands can authenticate with websites the user is already logged into. Cookies are **never written, modified, or transmitted externally**. |
| `activeTab` | Required to identify the currently active tab for context-aware commands. |
| `alarms` | Required to maintain the WebSocket connection to the local daemon via periodic keepalive checks. |
## Data flow
```
User's terminal (opencli CLI)
↓ (spawns)
Local daemon process (localhost:19825)
↓ (WebSocket, localhost only)
Chrome Extension (this extension)
↓ (Chrome APIs)
Isolated Chrome automation window
```
All data stays on the user's machine. No data leaves `localhost`.
## Cookie access
The extension reads cookies **only** when explicitly requested by a CLI command, and **only** for the specific domain the command targets. It cannot and does not dump all cookies. Cookie data is returned to the local daemon process and is never sent to any external server.
## Third-party services
This extension does not integrate with, send data to, or receive data from any third-party service.
## Open source
This extension is fully open source. You can audit the complete source code at:
A CLI tool that turns **any website** or **Electron app** into a command-line interface — Bilibili, Zhihu, 小红书, Twitter/X, Reddit, YouTube, Antigravity, and [many more](#built-in-commands) — powered by browser session reuse and AI-native discovery.
OpenCLI gives you one surface for three different kinds of automation:
🔥 **CLI All Electron Apps! The Most Powerful Update Has Arrived!** 🔥
Turn ANY Electron application into a CLI tool! Recombine, script, and extend applications like Antigravity Ultra seamlessly. Now AI can control itself natively. Unlimited possibilities await!
- **Use built-in adapters** for sites like Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, Twitter/X, and [many more](#built-in-commands).
- **Let AI Agents operate any website** — install the `opencli-browser` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Write new adapters** end-to-end with `opencli browser` + the `opencli-adapter-author` skill, which guides from first recon through field decoding, code, and `opencli browser verify`.
---
## Table of Contents
- [Highlights](#highlights)
- [Prerequisites](#prerequisites)
- [Quick Start](#quick-start)
- [Built-in Commands](#built-in-commands)
- [Output Formats](#output-formats)
- [For AI Agents (Developer Guide)](#for-ai-agents-developer-guide)
- **Self-healing setup** — `opencli setup` auto-discovers tokens; `opencli doctor` diagnoses config across 10+ tools; `--fix` repairs them all.
- **Dynamic Loader** — Simply drop `.ts` or `.yaml` adapters into the `clis/` folder for auto-registration.
- **Dual-Engine Architecture** — Supports both YAML declarative data pipelines and robust browser runtime TypeScript injections.
## Prerequisites
- **Node.js**: >= 18.0.0
- **Chrome** running **and logged into the target site** (e.g. bilibili.com, zhihu.com, xiaohongshu.com).
> **⚠️ Important**: Browser commands reuse your Chrome login session. You must be logged into the target website in Chrome before running commands. If you get empty data or errors, check your login status first.
OpenCLI connects to your browser through the Playwright MCP Bridge extension.
It prefers an existing local/global `@playwright/mcp` install and falls back to `npx -y @playwright/mcp@latest` automatically when no local MCP server is found.
### Playwright MCP Bridge Extension Setup
1. Install **[Playwright MCP Bridge](https://chromewebstore.google.com/detail/playwright-mcp-bridge/mmlmfjhmonkocbjadbfplnigmagldckm)** extension in Chrome.
2. Run `opencli setup` — discovers the token, distributes it to your tools, and verifies connectivity:
```bash
opencli setup
```
The interactive TUI will:
- 🔍 Auto-discover `PLAYWRIGHT_MCP_EXTENSION_TOKEN` from Chrome (no manual copy needed)
- ☑️ Show all detected tools (Codex, Cursor, Claude Code, Gemini CLI, etc.)
- ✏️ Update only the files you select (Space to toggle, Enter to confirm)
- 🔌 Auto-verify browser connectivity after writing configs
> **Tip**: Use `opencli doctor` for ongoing diagnosis and maintenance:
> ```bash
> opencli doctor # Read-only token & config diagnosis
> opencli doctor --live # Also test live browser connectivity
> opencli doctor --fix # Fix mismatched configs (interactive)
> opencli doctor --fix -y # Fix all configs non-interactively
> ```
**Alternative: CDP Mode (For Servers/Headless)**
If you cannot install the browser extension (e.g. running OpenCLI on a remote headless server), you can connect OpenCLI to your local Chrome via CDP using SSH tunnels or reverse proxies. See the [CDP Connection Guide](./CDP.md) for detailed instructions.
<details>
<summary>Manual setup (alternative)</summary>
Add token to your MCP client config (e.g. Claude/Cursor):
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `longbridge`, `tg`, `discord`, `wx`, `ntn` (Notion), and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Trae CN, Codex, Antigravity, ChatGPT, and Trae SOLO.
OpenCLI connects to Chrome/Chromium through a lightweight Browser Bridge extension plus a small local daemon. The daemon auto-starts when needed.
**Option A — Chrome Web Store (recommended):**
Install **OpenCLI** from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk).
**Option B — Manual install:**
1. Download the latest `opencli-extension-v{version}.zip` from the GitHub [Releases page](https://github.com/jackwener/opencli/releases).
2. Unzip it, open `chrome://extensions`, and enable **Developer mode**.
3. Click **Load unpacked** and select the unzipped folder.
### 3. Verify the setup
```bash
opencli list # See all commands
opencli list -f yaml # List commands as YAML
opencli hackernews top --limit 5# Public API, no browser
opencli bilibili hot --limit 5# Browser command
opencli zhihu hot -f json # JSON output
opencli zhihu hot -f yaml # YAML output
opencli doctor
```
### Install from source (for developers)
### 4. Optional: name your Chrome profile
Each Chrome profile runs its own OpenCLI extension instance. If you use multiple Chrome profiles, list the connected profiles and assign local aliases:
```bash
git clone git@github.com:jackwener/opencli.git
cdopencli
npm install
npm run build
npm link # Link binary globally
opencli list # Now you can use it anywhere!
opencli profile list
opencli profile rename <contextId> work
opencli profile use work
opencli --profile work browser state
```
### Update
With only one connected profile, OpenCLI uses it automatically. With multiple connected profiles and no default, OpenCLI asks you to choose instead of guessing.
### 5. Run your first commands
```bash
npm install -g @jackwener/opencli@latest
opencli list
opencli hackernews top --limit 5
opencli bilibili hot --limit 5
```
## For Humans
Use OpenCLI directly when you want a reliable command instead of a live browser session:
-`opencli list` shows every registered command.
-`opencli <site> <command>` runs a built-in or generated adapter.
-`opencli external register mycli` exposes a local CLI through the same discovery surface.
If you want to add your own commands, start with the [Extending OpenCLI guide](./docs/guide/extending-opencli.md). README keeps this short; the guide covers the directory layout, source-control model, and install commands.
| Need | Recommended path |
|------|------------------|
| Keep personal website commands in your own Git repo | `opencli plugin create` + `opencli plugin install file://...` |
| Quickly draft a private local adapter | `opencli browser init <site>/<command>` in `~/.opencli/clis/` |
| Modify an official adapter locally | `opencli adapter eject <site>` + `opencli adapter reset <site>` |
| Wrap an existing local binary | `opencli external register <name>` |
## For AI Agents
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
| **opencli-adapter-author** | Write a reusable adapter for a new site or add a command to an existing site | "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-autofix** | Repair a broken adapter when a built-in command fails | "`opencli zhihu hot` is returning empty — fix it" |
| **opencli-browser** | Drive a real Chrome page ad-hoc — navigate, fill forms, click, extract | "Help me check my Xiaohongshu notifications" / "Help me fill out this form" / "Use browser commands to scrape this page" |
| **opencli-browser-sitemap** | Consume site sitemap context while driving a browser task | "Use the sitemap to navigate this website without blind clicking" |
| **opencli-sitemap-author** | Create or update site sitemap knowledge for browser agents | "Record the stable workflow you just discovered for this site" |
| **opencli-usage** | Quick reference for all OpenCLI commands and sites | "What commands does OpenCLI have for Twitter?" |
### How it works
Once `opencli-browser` is installed, your AI agent can:
1.**Navigate** to any URL using your logged-in browser
2.**Read** page content via structured DOM snapshots (not screenshots)
3.**Interact** — click buttons, fill forms, select options, press keys
4.**Extract** data from the page or intercept network API responses
5.**Wait** for elements, text, or page transitions
The agent handles all the `opencli browser` commands internally — you just describe what you want done in natural language.
`opencli browser` commands require a `<session>` positional immediately after `browser`. `opencli browser work open <url>` and `opencli browser work tab new [url]` both return a target ID. Use `opencli browser work tab list` to inspect target IDs, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted commands in the same session.
## Writing a new adapter
When the site you need is not yet covered, use the `opencli-adapter-author` skill end-to-end:
1.**Recon** the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
2.**Discover** the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
6. Site knowledge persists to `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context.
## Configuration
| Variable | Default | Description |
|----------|---------|-------------|
| `OPENCLI_DAEMON_PORT` | `19825` | HTTP port for the daemon-extension bridge |
| `OPENCLI_PROFILE` | — | Browser Bridge profile alias/contextId to use when multiple Chrome profiles are connected |
| `OPENCLI_WINDOW` | command default | Set to `foreground` or `background` to override Browser Bridge window placement. Browser-backed commands also accept `--window <foreground\|background>`. |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | Seconds to wait for browser connection |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | Seconds to wait for a single browser command |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
| `OPENCLI_VERBOSE` | `false` | Enable verbose logging (`-v` flag also works) |
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
`opencli browser *` requires an explicit `<session>` positional, uses a foreground browser window by default, and keeps that session's tab lease until `opencli browser <session> close` or idle cleanup. Browser-backed adapters use a background adapter window and release one-shot tab leases by default. Interactive adapters can declare `siteSession: 'persistent'` to keep a stable site tab for continuity; pass `--site-session ephemeral` for a one-shot tab.
## Built-in Commands
**19 sites · 80+ commands** — run `opencli list` for the live registry.
`opencli xiaoyuzhou download` and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
## Output Formats
All built-in commands support `--format` / `-f` with `table`, `json`, `yaml`, `md`, and `csv`.
The `list` command supports the same format options, and keeps `--json` for backward compatibility.
All built-in commands support `--format` / `-f` with `table` (default), `json`, `yaml`, `md`, and `csv`.
```bash
opencli list -f yaml # Command registry as YAML
opencli bilibili hot -f table# Default: rich terminal table
opencli bilibili hot -f json # JSON (pipe to jq or LLMs)
opencli bilibili hot -f yaml # YAML (human-readable structured output)
opencli bilibili hot -f md # Markdown
opencli bilibili hot -f csv # CSV
opencli bilibili hot -f json # Pipe to jq or LLMs
opencli bilibili hot -f csv # Spreadsheet-friendly
opencli bilibili hot -v # Verbose: show pipeline debug steps
```
## For AI Agents (Developer Guide)
## Exit Codes
If you are an AI assistant tasked with creating a new command adapter for `opencli`, please follow the AI Agent workflow below:
opencli follows Unix `sysexits.h` so CI / scripts can branch on failure mode: `0` success, `66` empty result, `69` Browser Bridge down, `75` timeout, `77` auth required, `78` config error,`130` Ctrl-C. Full reference: [docs/guide/exit-codes.md](./docs/guide/exit-codes.md).
> **Quick mode**: To generate a single command for a specific page URL, see [CLI-ONESHOT.md](./CLI-ONESHOT.md) — just a URL + one-line goal, 4 steps done.
## Plugins
> **Full mode**: Before writing any adapter code, read [CLI-EXPLORER.md](./CLI-EXPLORER.md). It contains the complete browser exploration workflow, the 5-tier authentication strategy decision tree, and debugging guide.
Extend OpenCLI with community-contributed adapters:
See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
## Testing
See **[TESTING.md](./TESTING.md)** for the full testing guide, including:
- Current test coverage (unit + E2E tests across 19 sites)
- How to run tests locally
- How to add tests when creating new adapters
- CI/CD pipeline with sharding
- Headless browser mode (`OPENCLI_HEADLESS=1`)
```bash
# Quick start
npm run build
npx vitest run # All tests
npx vitest run src/ # Unit tests only
npx vitest run tests/e2e/ # E2E tests
```
See **[TESTING.md](./TESTING.md)** for how to run and write tests.
## Troubleshooting
- **"Failed to connect to Playwright MCP Bridge"**
- Ensure the Playwright MCP extension is installed and **enabled** in your running Chrome.
- Restart the Chrome browser if you just installed the extension.
- **Empty data returns or 'Unauthorized' error**
- Your login session in Chrome might have expired. Open a normal Chrome tab, navigate to the target site, and log in or refresh the page to prove you are human.
- **Node API errors**
- Make sure you are using Node.js >= 18. Some dependencies require modern Node APIs.
- **Token issues**
- Run `opencli doctor` to diagnose token configuration across all tools.
- **"Extension not connected"** — Ensure the Browser Bridge extension is installed from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) and **enabled** in `chrome://extensions`.
- **"attach failed: Cannot access a chrome-extension:// URL"** — Another extension may be interfering. Try disabling other extensions temporarily.
- **Empty data or 'Unauthorized' error** — Your Chrome/Chromium login session may have expired. Navigate to the target site and log in again.
- **Node API errors / missing `fetch` / startup crash on old Node** — OpenCLI requires **Node.js >= 20**. Run `node --version`, upgrade Node if needed, then retry.
3. Run `opencli setup` to auto-discover token and configure all tools
> **Note**: You must be logged into the target website in Chrome before running commands. Tabs opened during command execution are auto-closed afterwards.
Public API commands (`hackernews`, `github search`, `v2ex`) need no browser.
"opencli browser open https://en.wikipedia.org/wiki/JavaScript",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.toc li a, #toc li a, .vector-toc-contents a')].slice(0,8).map(a=>a.textContent?.trim()))\""
],
"judge":{
"type":"arrayMinLength",
"minLength":5
}
},
{
"name":"complex-books-detail",
"steps":[
"opencli browser open https://books.toscrape.com",
// ── Task Definitions (inline, to avoid YAML dependency) ────────────
constTASKS: SkillTask[]=[
// Extract
{name:"extract-title-example",task:"Extract the main heading text from this page",url:"https://example.com",judge_context:["Output must contain 'Example Domain'"]},
{name:"extract-paragraph-wiki",task:"Extract the first paragraph of the JavaScript article",url:"https://en.wikipedia.org/wiki/JavaScript",judge_context:["Output must mention 'programming language'","Output must contain actual paragraph text, not just the title"]},
{name:"extract-github-stars",task:"Find the number of stars on this repository",url:"https://github.com/browser-use/browser-use",judge_context:["Output must contain a number (the star count)"]},
{name:"extract-npm-downloads",task:"Find the weekly download count for this package",url:"https://www.npmjs.com/package/zod",judge_context:["Output must contain a number (weekly downloads)"]},
// List extraction
{name:"list-hn-top5",task:"Extract the top 5 stories with their titles",url:"https://news.ycombinator.com",judge_context:["Output must contain 5 story titles","Each title must be an actual HN story, not made up"]},
{name:"list-books-5",task:"Extract the first 5 books with their title and price",url:"https://books.toscrape.com",judge_context:["Output must contain 5 books","Each book must have a title and a price"]},
{name:"list-quotes-3",task:"Extract the first 3 quotes with their text and author",url:"https://quotes.toscrape.com",judge_context:["Output must contain 3 quotes","Each quote must have text and an author name"]},
{name:"list-github-trending",task:"Extract the top 3 trending repositories with name and description",url:"https://github.com/trending",judge_context:["Output must contain 3 repositories","Each must have a repo name"]},
{name:"list-jsonplaceholder",task:"Extract the first 5 posts with their title",url:"https://jsonplaceholder.typicode.com/posts",judge_context:["Output must contain 5 posts","Each post must have a title"]},
// Search
{name:"search-ddg",task:"Search for 'TypeScript tutorial' and extract the first 3 result titles",url:"https://duckduckgo.com",judge_context:["The agent must type a search query","Output must contain at least 3 search result titles"]},
{name:"search-npm",task:"Search for 'react' and extract the top 3 package names",url:"https://www.npmjs.com",judge_context:["The agent must search for 'react'","Output must contain at least 3 package names"]},
{name:"search-wiki",task:"Search for 'Rust programming language' and extract the first sentence of the article",url:"https://en.wikipedia.org",judge_context:["The agent must search and navigate to the article","Output must mention 'programming language'"]},
// Navigation
{name:"nav-click-link",task:"Click the 'More information...' link and extract the heading of the new page",url:"https://example.com",judge_context:["The agent must click a link","Output must contain 'IANA' or reference the new page"]},
{name:"nav-click-hn",task:"Click on the first story link and tell me the title of the page you land on",url:"https://news.ycombinator.com",judge_context:["The agent must click a story link","Output must contain the title of the destination page"]},
{name:"nav-go-back",task:"Click the 'More information...' link, then go back, and tell me the heading of the original page",url:"https://example.com",judge_context:["The agent must click a link then go back","Output must contain 'Example Domain'"]},
{name:"nav-multi-step",task:"Click the Next page link at the bottom, then extract the first quote from page 2",url:"https://quotes.toscrape.com",judge_context:["The agent must navigate to page 2","Output must contain a quote from page 2"]},
// Scroll
{name:"scroll-footer",task:"Scroll to the bottom and extract the footer text",url:"https://quotes.toscrape.com",judge_context:["The agent must scroll down","Output must contain footer or bottom-of-page content"]},
{name:"scroll-pagination",task:"Find the pagination info at the bottom of the page",url:"https://books.toscrape.com",judge_context:["Output must contain page number or pagination info"]},
// Form
{name:"form-fill-basic",task:"Fill the Customer Name with 'OpenCLI' and Telephone with '555-0100'. Do not submit.",url:"https://httpbin.org/forms/post",judge_context:["The agent must type 'OpenCLI' into a name field","The agent must type '555-0100' into a phone field","The form must NOT be submitted"]},
{name:"form-radio",task:"Select the 'Medium' pizza size option. Do not submit.",url:"https://httpbin.org/forms/post",judge_context:["The agent must select a radio button for Medium size"]},
{name:"form-login",task:"Fill the username with 'testuser' and password with 'testpass'. Do not submit.",url:"https://the-internet.herokuapp.com/login",judge_context:["The agent must fill the username field","The agent must fill the password field","The form must NOT be submitted"]},
// Complex
{name:"complex-wiki-toc",task:"Extract the table of contents headings",url:"https://en.wikipedia.org/wiki/JavaScript",judge_context:["Output must contain at least 5 section headings from the table of contents"]},
{name:"complex-books-detail",task:"Click on the first book and extract its title and price from the detail page",url:"https://books.toscrape.com",judge_context:["The agent must click on a book","Output must contain the book title","Output must contain a price"]},
{name:"complex-quotes-page2",task:"Navigate to page 2 and extract the first 3 quotes with their authors",url:"https://quotes.toscrape.com",judge_context:["The agent must navigate to page 2","Output must contain 3 quotes with authors"]},
{name:"complex-multi-extract",task:"Extract both the page title and the first paragraph text",url:"https://en.wikipedia.org/wiki/TypeScript",judge_context:["Output must contain 'TypeScript'","Output must contain actual paragraph text"]},
// Bench (harder, real-world)
{name:"bench-reddit",task:"Extract the titles of the top 5 posts",url:"https://old.reddit.com",judge_context:["Output must contain 5 post titles","Titles must be actual Reddit posts"]},
{name:"bench-imdb",task:"Find the year and rating of The Matrix",url:"https://www.imdb.com/title/tt0133093/",judge_context:["Output must contain '1999'","Output must contain a rating number"]},
{name:"bench-github-profile",task:"Extract the bio and number of public repositories",url:"https://github.com/torvalds",judge_context:["Output must contain bio text or 'Linux'","Output must contain a number for repos"]},
{name:"bench-httpbin",task:"Extract the User-Agent header shown on this page",url:"https://httpbin.org/headers",judge_context:["Output must contain a User-Agent string"]},
{name:"bench-jsonapi-todo",task:"Extract the first 5 todo items with their title and completion status",url:"https://jsonplaceholder.typicode.com/todos",judge_context:["Output must contain 5 todo items","Each must have a title and completed status"]},
// Codex form (the real test)
{name:"codex-form-fill",task:"Fill the basic information using 'opencli' as the identity (first name=open, last name=cli, email=opencli@example.com, GitHub username=opencli). Do NOT submit the form.",url:"https://openai.com/form/codex-for-oss/",judge_context:["The agent must fill the first name field","The agent must fill the last name field","The agent must fill the email field","The form must NOT be submitted"],max_steps: 15},
];
// ── Run Task ───────────────────────────────────────────────────────
* Optimizes the "Save as CLI" pipeline: browser init → write adapter → run.
* Covers PUBLIC (no auth) and COOKIE (browser session) strategies.
* Metric: number of passing save-tasks.
*/
importtype{AutoResearchConfig}from'../config.js';
exportconstsaveReliability: AutoResearchConfig={
goal:'Increase "Save as CLI" pipeline pass rate to 100%. The flow is: browser init creates a scaffold, user writes adapter code, opencli discovers and runs it. Covers both PUBLIC (fetch API) and COOKIE (browser session) strategies. Focus on: init template correctness, user CLI discovery, adapter loading, verify command robustness, and browser session handling.',
A 30-second morning routine that surfaces what changed overnight in reinforcement-learning and large-model research, without opening a browser.
## What I wanted
Before reading anything, decide where to spend my 20 minutes of paper time:
- which `cs.LG` and `cs.AI` papers landed in the last 24 hours
- which OpenReview submissions at recent venues (NeurIPS 2025 right now, ICLR 2024 / NeurIPS 2024 as historical reference) carry titles and primary areas relevant to my work
- which papers the Hugging Face Daily Papers community is talking about today
Skim signals, then drill in. The point is to filter, not to read everything.
## Commands
```bash
# 1. arxiv recent in the two relevant categories (newest 30 each)
That is the entire collection step. The four files together are the whole signal surface for one morning.
## What I do with the output
Pipe the four JSON files into a one-shot LLM digest with a fixed prompt:
```
Here are four JSON arrays of papers from the last 24 hours.
Group them into:
1. Direct hits on RLHF / preference optimization / reasoning RL.
2. Adjacent (offline RL, world models, agent benchmarks).
3. Notable infra (training, evaluation, data).
For each, give me title + arxiv id + one-sentence why-it-matters.
Skip everything that is review / survey / position paper.
```
The LLM compresses ~120 entries into a 10-line shortlist in seconds. I then open whichever 2 to 3 papers actually clear the bar.
## Why CLI beats the browser version
- Four pages of clicking and scrolling collapses into four `opencli` calls.
- The output is structured JSON, so the digest prompt can reason about it deterministically. No copy-paste, no "I missed paper 14".
- Works inside any agent loop. A scheduled task can run the four commands, push them to an LLM, and message the digest somewhere. No browser kept open.
- Zero token cost on the OpenCLI side. The only paid step is the digest call at the end.
The arxiv adapter's `recent <category>` (added in #1289) is the lever here. Without it I would have to fall back to the arxiv listings page, which means scraping HTML in agent code instead of consuming a structured listing.
# Find a paper's implementation and follow-up work
Given a single paper title or arxiv id, walk three sources in one chain to find the canonical reference, follow-up citations, and any community-fine-tuned models or Spaces that already build on it.
## What I wanted
I read a paper abstract, decide it is interesting, and want to answer three questions before deciding to actually re-read the paper or reproduce it:
1. Has anyone already implemented or fine-tuned on top of it (Hugging Face)?
2. Who has cited or extended it (dblp / OpenReview)?
3. What is the canonical bibliographic record (dblp key for citation, full arxiv metadata for reading)?
Doing this in a browser means three tabs and two minutes of context-switching. The point is to compress that into one shell pipeline.
## Commands
Worked example: "Direct Preference Optimization" (DPO).
```bash
# 1. Canonical arxiv record (full abstract, authors, pdf url, categories).
# Note: arxiv free-text search ranks by recency, so the original DPO
# paper does not always come back first. When the canonical id is
Three of the four are public-strategy adapters, no browser session needed. The OpenReview call also lands without auth for public venues.
## What I do with the output
For DPO the chain produces:
- arxiv record: paper id `2305.18290`, full abstract, pdf link.
- dblp record: canonical key `conf/nips/RafailovSMMEF23`, NeurIPS 2023, co-author list (useful to find related work by same lab).
- HF Daily Papers (last 30 days): every paper whose title mentions DPO or preference. Each one is a candidate "follow-up work I should know about".
- OpenReview: the original submission's review thread, if posted (lets me see what reviewers actually pushed back on, which is more useful than the published abstract).
I dump all four JSON outputs into a single LLM call with the prompt: *"Build a one-paragraph 'state of the field' summary for this paper as of today. Cite each follow-up by arxiv id."* That gives me a research-debt brief in 30 seconds.
## Why this is worth a CLI chain
- Each adapter alone is just "search a website". The value is the chain. Four `opencli` calls feed into one LLM call. No browser, no copy-paste.
- Output is identifier-rich (arxiv id, dblp key, venue id, HF paper id). I can re-feed any of those into the next call, e.g. once I find a follow-up arxiv id from HF Daily Papers I run `opencli arxiv paper <new-id>` immediately.
- Survives use inside an agent loop. Same chain runs unattended for a batch of 20 papers from a reading list.
- Zero token cost for the discovery half. Only the final summary step pays for inference.
Without `opencli dblp search` (added in #1299) and `opencli openreview search` (added in #1294), this whole pipeline used to require either web scraping in agent code or paying for a research-paper API. Both adapters being public-strategy means they slot in cleanly.
# Track a conference's accepted papers and reviews from the terminal
Once an OpenReview venue opens its decisions (or releases reviews publicly during the discussion phase), I want a one-shot way to pull the full venue listing and dive into individual review threads, without clicking through 200+ submission pages.
## What I wanted
For each major venue I follow (ICLR, NeurIPS, ICML), the same three things every time decisions are visible:
1. The full list of accepted papers at the venue, with titles and forum ids.
2. For any paper I flagged interesting from the list: the full review thread, including reviewer scores, rebuttals, and the AC's decision rationale.
3. A way to pipe both into LLM-driven shortlisting ("which of these 100 oral papers actually intersect with my research direction").
The OpenReview UI is fine for one paper at a time, but unusable for batch reasoning across the whole acceptance list.
## Commands
Worked example: ICLR 2024 oral track, then drill into one paper's reviews using a real forum id.
```bash
# 1. Full list of papers at a venue (natural-language venue text;
# if the venue is not yet open OpenReview returns EMPTY_RESULT
`venue` returns each entry with a forum id you can hand straight back into `reviews` and `paper`. No id lookup gymnastics. `reviews` returns the full thread as a JSON array: a `PAPER` row with the abstract, then one `REVIEW` row per reviewer (with `rating`, `confidence`, summary, weaknesses, questions), followed by author rebuttals and the AC's decision rationale.
## What I do with the output
Two distinct workflows depending on the phase of the venue:
### Phase A: filtering the acceptance list
After `venue` returns 200 entries, dump the JSON into an LLM with the prompt:
```
Here is the full acceptance list at <venue>. Filter to papers that intersect
with my research interests:
- reinforcement learning from preference / reward feedback
- reasoning training (process reward, RLVR, RLHF variants)
- long-horizon agent benchmarks
For each match: title + forum_id + one-sentence why-it-matters.
```
This collapses 200 papers to a 10-paper shortlist in seconds. The forum ids are the keys I will use in Phase B.
### Phase B: depth-reading the shortlist
For each shortlisted forum id, run `opencli openreview reviews <forum-id>` and feed the JSON to an LLM with the prompt:
```
Summarize the review thread:
- reviewer scores
- the strongest critique
- whether the rebuttal addressed it
- final decision and AC rationale
```
This is faster than reading three reviews + rebuttal + meta-review per paper. For 10 papers this turns 60 minutes of OpenReview clicking into 10 minutes of summary reading, then I open the actual reviews only for papers where the summary flagged something worth knowing.
## Why this beats opening OpenReview
- One `venue` call replaces scrolling a paginated UI for 200+ papers.
-`reviews` returns the entire thread as JSON, so an LLM can reason over the whole review-rebuttal-decision arc at once. The web view forces you to scroll three reviews + N rebuttals + meta separately.
- Forum ids returned from `venue` are stable and reusable across calls. Easy to keep a personal reading list as `forum-ids.txt` and run `for id in $(cat forum-ids.txt); do opencli openreview reviews $id; done`.
- The whole loop is public-strategy. No login required for venues with public reviewing.
`opencli openreview` (added in #1294) is the lever. Before this adapter existed, the same workflow needed either OpenReview's Python client or HTML scraping inside agent code. Both have higher friction than `opencli openreview reviews <forum-id>` returning structured JSON in one shot.
description:'Show the logged-in 12306 account summary. Sensitive fields (real name, email, mobile, birth date) are masked by default; pass --include-sensitive to opt in.',
domain:'kyfw.12306.cn',
strategy:Strategy.COOKIE,
browser:true,
args:[
{name:'include-sensitive',type:'boolean',default:false,help:'Reveal unmasked real name / email / mobile / birth date. The 12306 ID-number mask is server-side and never decoded.'},
if(!Number.isInteger(n)||n<1)thrownewArgumentError(`limit must be a positive integer (1-${max})`);
if(n>max)thrownewArgumentError(`limit must be <= ${max}`);
returnn;
}
cli({
site:'12306',
name:'passengers',
access:'read',
description:'List the logged-in user\'s saved 12306 passengers. Sensitive fields are masked by default; pass --include-sensitive to opt in.',
domain:'kyfw.12306.cn',
strategy:Strategy.COOKIE,
browser:true,
args:[
{name:'limit',type:'int',default:20,help:`Max passengers to return (1-${MAX_PAGE_SIZE})`},
{name:'include-sensitive',type:'boolean',default:false,help:'Reveal unmasked real names and birth dates. The 12306 ID-number / mobile masks are server-side and never decoded.'},
description:'Look up 12306 ticket prices by seat class for one train on a given date and segment (anonymous, no login required)',
domain:'kyfw.12306.cn',
strategy:Strategy.PUBLIC,
browser:false,
args:[
{name:'train-no',positional:true,required:true,help:'Internal train_no from `12306 trains` (e.g. 24000000G10L)'},
{name:'from',required:true,help:'Origin station (Chinese name, telecode, or pinyin) - must be a stop of this train'},
{name:'to',required:true,help:'Destination station - must be a stop of this train'},
{name:'date',required:true,help:'Departure date in YYYY-MM-DD'},
{name:'seat-types',default:'OM9PA1A3A4FWZ',help:'Seat-type letters to query (default covers the common classes). Examples: OM9 (二等/一等/商务), A1A3A4 (硬座/硬卧/软卧).'},
thrownewCommandExecutionError(`12306 ${endpoint} returned an unexpected payload shape`);
}
thrownewCommandExecutionError(`12306 rejected every known query endpoint name (${QUERY_ENDPOINTS.join(', ')}); the wire protocol may have changed. Last body: ${lastResponseText.slice(0,200)}`);
}
cli({
site:'12306',
name:'trains',
access:'read',
description:'List trains between two 12306 stations on a given date (anonymous, no login required)',
domain:'kyfw.12306.cn',
strategy:Strategy.PUBLIC,
browser:false,
args:[
{name:'from',positional:true,required:true,help:'Origin station: Chinese name (北京), telecode (BJP), or pinyin (beijing)'},
{name:'to',positional:true,required:true,help:'Destination station: same forms as <from>'},
{name:'date',required:true,help:'Departure date in YYYY-MM-DD'},
thrownewArgumentError(`Unknown 12306 station "${trimmed}"`,'Try the Chinese name (上海虹桥), the 3-4 letter telecode (AOH), or full pinyin (shanghaihongqiao).');
}
exportfunctionvalidateDate(value){
if(!DATE_RE.test(String(value??''))){
thrownewArgumentError(`date must be YYYY-MM-DD, got "${value}"`);
thrownewCommandExecutionError('1688 search page did not return a readable payload','Open the same query in Chrome and verify the page is fully loaded before retrying.');
thrownewEmptyResultError('1688 search','No visible results were extracted. Retry with a different query or open the same search page in Chrome first.');
if(message.includes('Inspected target navigated or closed')
||message.includes('Cannot find context with specified id')
||message.includes('Target closed')){
thrownewCommandExecutionError(`1688 ${action} navigation lost the current browser target`,`${buildCaptchaHint(action)} If CDP is attached to a stale or blocked tab, open a fresh 1688 tab and point OPENCLI_CDP_TARGET at that tab.`);
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
jackwener