Compare commits

..

197 Commits

Author SHA1 Message Date
jackwener 913fb80aad docs(readme): drop For Developers section
Per WAWQAQ: from-source install instructions are infrastructure detail
that don't belong in a public-facing README. Contributors finding
themselves in this repo will already know `npm install / build / link`
patterns; users who reach the README from npm don't need them.

Removed in both EN and ZH.
2026-05-20 03:55:17 +08:00
jakevin ce432c2428 chore(release): 1.8.0 (#1682)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
* chore(release): 1.8.0

Substantial release: weread-official adapter, wider LinkedIn / Twitter / Reddit / Zhihu coverage, 12306 / Suno / Xianyu additions, security and reliability fixes, plus a 20% README shrink.

* chore: remove orphan docs/adapters-doc/ones.md

The file was a leftover from PR #386 (2026-04-10) and has been
superseded by docs/adapters/browser/ones.md. Bundled into the 1.8.0
release commit chain so the release doesn't ship with a dead docs
file alongside the new docs.

Skipped from opus-reviewer's audit (Tier 1 #1-#4) for this release:
- #1 smart-search dead refs (10 spots) — owned by @codex-coder's
  skill-deletion PR; release PR will rebase on top of it.
- #3 clis/test-utils.js relocation — touches 19 importers, separate
  refactor PR.
- #4 clis/slock/ orphan — needs WAWQAQ design call.
- #6 opencli-usage:161 wording — current "Commands that used to
  exist" framing is already clear enough.
- #7 docs/adapters/index.md sync (8-20 missing sites) — broader docs
  PR, not release-time bundling.

* chore: remove clis/slock + sync docs/adapters/index.md (audit #4 + #7)

Per WAWQAQ post-audit directive on #OpenCLI:f046ece7:

- `clis/slock/` was a half-finished orphan with only `_utils.js` and no
  command entry points. Removed.
- `docs/adapters/index.md` was missing 11 browser adapters: 12306,
  suno, weread-official, qwen, 1point3acres, brave, duckduckgo, cnki,
  flomo, jianyu, taobao. Added all with commands sourced from
  cli-manifest.json. Desktop section already covered all 7 desktop
  adapters (Cursor / Codex / Antigravity / ChatGPT App / ChatWise /
  Discord / Doubao App).
2026-05-20 03:31:57 +08:00
jakevin 7ee16aa087 feat(booking): add search adapter for Booking.com hotel listings (#1680)
* feat(booking): add search adapter for Booking.com hotel listings

New `opencli booking search <destination> --checkin --checkout` adapter
scrapes the server-rendered hotel cards on www.booking.com via stable
`[data-testid=property-card]` selectors. No login required (Strategy.PUBLIC
+ browser:true).

Highlights
- 12 columns: rank, name, country, slug, star_rating, review_score,
  review_count, price_amount, price_currency, distance, recommended_room,
  url. `slug` + URL stay stable across locales (better round-trip key
  than `name`, which Booking sometimes localizes from session cookies).
- Score parser anchors on `(\d{1,2})\.(\d)` so the duplicated "8.68.6" /
  "评分8.68.6很棒" rendering doesn't mis-parse to 8.68.
- Currency symbol → ISO 4217 map (US$/€/£/¥/¥/₹/₩/HK$/A$/NT$/S$/CN¥);
  honor `--currency` URL param for stable codes.
- Pagination via `--offset` (Booking pages 25/request); `rank` includes
  the offset so paginated calls stay sortable.
- Captcha-page detection short-circuits to CommandExecutionError instead
  of silent empty rows.

Typed errors (no silent clamp / fallback)
- Bad date / out-of-range adults/rooms/children/limit/offset / unknown lang
  / malformed currency → ArgumentError up front (before any navigation).
- Browser nav failure → CommandExecutionError.
- Zero cards rendered → EmptyResultError with a hint.
- Captcha page → CommandExecutionError.

29 unit tests cover the helpers, the registry shape, every typed-error
path, the {session,data} CDP envelope unwrap, and offset-aware rank
numbering. Silent-column-drop + typed-error-lint audits unchanged.
Live-verified against Tokyo + Paris.

* fix(booking): harden search parser boundaries

* fix(booking): separate no-card drift from empty
2026-05-20 03:29:03 +08:00
jakevin 7a2ab47bf8 chore(skills): remove smart-search (#1683) 2026-05-20 03:23:32 +08:00
jakevin 2c8b50c4fd docs(readme): shrink CLI Hub + Core Concepts + merge Update into Install (#1681)
Per WAWQAQ:

1. **CLI Hub**: drop the 13-row 3-column table; enumerate just the
   names inline ("gh · docker · vercel · wrangler · ntn · obsidian · …")
   plus one-liner register / list commands. Removes "Manual install"
   ntn note (search lives in external-clis.yaml / ntn's own docs).
   Compresses the 7-row Desktop App Adapters table to a single inline
   line pointing at docs/adapters/desktop/.

2. **Core Concepts** section dissolved: its four subsections
   ("browser", "Built-in adapters", "Writing a new adapter",
   "CLI Hub and desktop adapters") duplicated the intro 3-bullet
   + later dedicated sections. Kept the substantive "Writing a new
   adapter" callout as its own top-level section. The "For AI Agents
   (Developer Guide)" tail block at the bottom was a third copy of
   the same recipe — removed.

3. **Update** merged with **Install skills**: install header now
   reads "Install skills (also refreshes existing installs)", and
   the standalone Update section collapses to a single command
   (`npm install -g @jackwener/opencli@latest && npx skills add ...`).

Net: EN 410 → 326 (-20%), ZH 455 → 366 (-20%). Same coverage; just
less repetition.
2026-05-20 03:12:36 +08:00
Benjamin Liu 51a9456305 feat(linkedin): add people-search command (#1649)
* feat(linkedin): add people-search command (#1621)

Closes #1621. Adds opencli linkedin people-search <keywords> for
finding people on standard LinkedIn (not Sales Navigator).

Architecture note. Standard LinkedIn moved its people search results
page to Server-Driven UI / React Server Components on the
/flagship-web/rsc-action/... path stack. The legacy Voyager REST
endpoint /voyager/api/search/dash/clusters returns HTTP 500 from a
web context; its modern camelCase rename voyagerSearchDashClusters
returns the same. The result list is rendered server-side and the
page HTML IS the result payload; Voyager calls from the page are
sidebar / notification concerns, not search results.

Extraction strategy. LinkedIn SSR uses obfuscated CSS class hashes
(e.g. _997b7c77) that rotate on every deploy AND display:contents
wrappers that flatten the DOM tree. Class-based selectors, walk-up-
to-card logic, and anchor-pair element ranges all fail because no
element boundary matches a person's card.

Working approach: extract main.innerText once, split by newline,
slice between consecutive person names. The names come from the
aria-hidden spans of /in/<handle> anchors. LinkedIn's SSR emits a
card as a name line followed by degree badge / headline / location
/ action labels before the next card's name line - a layout that
has been stable through several DOM refactors.

Critical filter: /in/<handle> anchors over-count because LinkedIn
renders each mutual connection as a /in/ anchor inside another
card's result. The skip() predicate during name-line lookup drops
mutual-connection lines ("X, Y and N other mutual connections"), so
anchors that don't have a real name line are filtered out.

CUL caveat. LinkedIn imposes a monthly Commercial Use Limit on
people search against the standard site. Burst behaviour is
irrelevant - the limit is a calendar-month counter. The adapter
runs one navigation per invocation (no pagination) so a single call
costs exactly one CUL query. --limit is capped at 10 to keep a
single call's information density high without surfacing the
"reached commercial use limit" yellow banner faster.

Schema:
  rank, name, headline, location, profile_url

Live verified against kyfw 12306-style throttled cadence (sleep 60s
between dev iterations to keep CUL consumption visible): 5/5 rows
populated with name + headline + location + profile_url for the
keyword "reinforcement learning". Mutual-connection anchors
correctly filtered out so the row order matches LinkedIn's own
ranking.

Tests: 10 unit tests covering URL construction, limit validation,
extraction-script invariants (anchor enumeration, text-slice
approach, mutual-connection filter, aria-hidden span as name source),
limit slicing, AuthRequiredError on missing JSESSIONID, CUL-
flavoured CommandExecutionError on redirect, EmptyResultError on
zero rows, ArgumentError on empty keywords, and registry shape.

* fix(linkedin): harden people search typed boundaries

* fix(linkedin): fail people search candidate parser drift

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-20 00:40:53 +08:00
Benjamin Liu 62592547a4 fix(adapters): migrate empty-data throws to EmptyResultError across 5 commands (#1674 follow-up) (#1678)
* fix(adapters): migrate empty-data throws to EmptyResultError across 5 commands (#1674 follow-up)

Continues the structured-error migration owner started in #1674
(fix(xhs,youtube): 把合法空数据语义切到 EmptyResultError). Same
motivation: callers need to distinguish "the platform legitimately
has no data for this target" from "fetch infrastructure is broken,
retry me", because downstream automation pipelines that batch over
seed lists conflate the two and trip soft-rate-limit heuristics.

Sites converted (5 commands, 6 throw sites):

powerchina/search.js (2 sites):
- "[taxonomy=empty_result] ... extracted only navigation/portal rows"
- "[taxonomy=empty_result] ... api/dom yielded no result"
  Both already self-labelled with the empty_result taxonomy tag,
  making this the canonical fix.

xiaohongshu/creator-notes.js, creator-notes-summary.js (both):
- "No notes found. Are you logged into creator.xiaohongshu.com?"
  The "is logged in" hint is preserved in the empty message so users
  can self-diagnose, while the error type is now structured.

xiaohongshu/creator-stats.js:
- "No data for period <X>. Available: <a, b, c>"
  Empty-data condition: requested period exists in the API surface
  but has zero numeric data; available periods are still surfaced
  in the message.

xiaohongshu/creator-note-detail.js:
- "No note detail data found. Check note_id and login status..."

Shape: exit code 66, stderr code: EMPTY_RESULT, matching
bilibili/subtitle, xhs/user, youtube/transcript precedent.

Out of scope:
- tiktok/{user,notifications,explore}.js: throws live inside
  page.evaluate template strings and run in browser context; the
  Node-side caller already regex-routes them via
  throwTikTokPageContextError({emptyPattern: /No videos found/, ...})
  to EmptyResultError. The existing design is correct.
- eastmoney/_secid.js / antigravity/serve.js / instagram/collection-*:
  input-validation throws, ArgumentError territory not EmptyResultError.

* test(adapters): cover empty-result migrations

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-20 00:30:54 +08:00
jakevin 4a3a55634a docs(readme): curate built-in commands to popular sites + add wrangler (#1679)
Per WAWQAQ:

1. Built-in Commands table cut from 30 EN rows / 86 ZH rows down to a
   curated 11-site list (xiaohongshu, bilibili, zhihu, hackernews,
   linkedin, reddit, twitter, claude, gemini, notebooklm, amazon).
   The README is meant to surface high-traffic / well-known sites;
   the long-tail (100+ adapters) is one click away via
   docs/adapters/index.md. linkedin (full) replaces linkedin-learning
   in the curated set per the spec.

2. Add Cloudflare Wrangler as a new external CLI passthrough:
   - src/external-clis.yaml entry (binary: wrangler, npm -g)
   - CLI Hub table row in EN + ZH READMEs
   - cli-manifest.json regen reflects the new entry (857 entries)
2026-05-19 23:17:15 +08:00
lenovobenben da497f0b02 feat(zhihu): add answer comments reader
* feat(zhihu): add answer comments reader

* fix(zhihu): harden answer-comments boundaries

* fix(zhihu): keep answer comments flat

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:59:42 +08:00
jakevin 2590278f43 fix(chatgpt): detect generated image surfaces (#1677) 2026-05-19 19:50:14 +08:00
Benjamin Liu 488e407a65 feat(twitter): add device-follow notification stream command
* feat(twitter): add device-follow command for /i/timeline notification stream (#1628)

Closes #1628. Adds the twitter device-follow command, which reads the
curated tweet list aggregated under a bell-icon "new posts from @userA
and N others" notification. Direct GET /i/timeline redirects to /home,
so the data is only reachable via the legacy v1.1 REST endpoint
/i/api/2/notifications/device_follow.json , none of the existing
twitter commands cover this stream:

- twitter timeline    home for-you / following feed (different endpoint)
- twitter notifications  the notification list itself, not aggregated
                         tweets inside any one notification
- twitter search     search-based, can't reproduce the aggregation

Endpoint discovery + field-mapping originally proposed by @traddo in
#1628; this PR upstreams a clean implementation that:

- Strategy.COOKIE + ct0 from CDP cookie jar + the public web bearer
  token from clis/twitter/utils.js (same auth path as twitter timeline)
- Hits /i/api/2/notifications/device_follow.json directly via
  page.evaluate fetch on the x.com origin so SameSite=Lax cookies are
  preserved
- Joins each entry.content.item.content.tweet.id to
  globalObjects.tweets[id] and resolves the author via
  globalObjects.users[tweet.user_id_str]
- Returns the canonical twitter row columns (id, author, text, likes,
  retweets, replies, views, created_at, url), matching twitter timeline
  minus has_media / media_urls / card / quoted_tweet which the legacy
  v1.1 endpoint does not surface
- Sets views: null rather than a 0 sentinel; the legacy endpoint does
  not return view counts even with include_ext_views=true, and the
  GraphQL TweetResultByRestId round-trip per tweet was judged too
  expensive for a list command (typed-errors §3: no scalar sentinels
  that lie about real engagement)
- parseLimit enforces strict 1-200 integer validation with no silent
  clamping; the only baseline addition is the silent-sentinel on the
  "unknown" author fallback, which matches the exact precedent in
  twitter/timeline.js:76 that is already baselined

Tests: 17 unit tests in device-follow.test.js cover parseLimit strict
validation, URL parameter shape, entry/tweet join, user-resolution
fallback, dedup via the seen set, empty-stream shape, the canonical
column registration, AuthRequiredError on missing ct0, and
CommandExecutionError on non-2xx fetch.

Live verified the endpoint shape end-to-end against the logged-in
session: HTTP 200 with the expected
{globalObjects: {tweets, users}, timeline: {id: 'tweet_notifications',
instructions: [{addEntries: {entries: []}}]}} envelope. The tester
account has no bell-notification follows enabled, so entries is empty,
but the shape and auth path are confirmed against the documented
spec.

* fix(twitter): harden device-follow typed boundaries

* fix(twitter): fail fast on device-follow drift

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:34:30 +08:00
jakevin e682c1c30a fix(deps): restore Node 20 runtime compatibility (#1673) 2026-05-19 19:20:38 +08:00
Ocean 86f57c0846 feat(reddit): 在 listing 命令上暴露 post_hint / url / preview / gallery 4 个媒体路由列 (#1676)
* feat(reddit): 在 listing 命令上暴露 post_hint / url / preview / gallery 4 个媒体路由列

5 个 reddit listing 命令(popular / hot / frontpage / search / subreddit)
每行新增 4 列,下游消费者不用 scrape selftext 也能区分 image / gallery /
hosted:video / link / self 五类内容:

- `post_hint` — Reddit 自报的内容类型(image | hosted:video | link | self 等)
- `url_overridden_by_dest` — 外链帖的原始 URL(image/link 类型才有)
- `preview_image_url` — 缩略图地址(HTML-decoded,Reddit 即便 raw_json=1
  也会在 preview URL 里返回 `&amp;`)
- `gallery_urls` — 多图相册数组(HTML-decoded)

## 实现

每个 adapter 的 evaluate 块内嵌两个 helper:

- `decodeHtml(s)` — 6 个 HTML entity 替换(&amp; / &lt; / &gt; / &quot; /
  &#x27; / &#39;)
- `extractRedditMedia(d)` — 从 post `data` 中抽 4 个字段,gallery_urls 从
  `gallery_data.items[].media_id` × `media_metadata[id].s.u` 组合得到

helper 在每个 adapter 里 inline 复制(reddit 没有 shared 文件,模式跟现有
adapter 一致)。`clis/reddit/extract-media.test.js` 把 helper 行为锁在
8 个 fixture(plain / image / gallery / hosted-video / link /
html-decode / 缺字段 / nullish input);每个 adapter 的 .test.js 额外
grep 自己源码里有 `function extractRedditMedia` 和 `...extractRedditMedia(c.data)`
两处接入痕迹,并断言 columns 数组形状。

frontpage 和 subreddit 之前 evaluate 返回原始 `children`、map 块按
`item.data.title` 索引;为了让 `gallery_urls` 这种数组字段能被 map 块的
模板字符串渲染,refactor 成和 popular/hot/search 一致的"evaluate 内部
就 map 成中间对象、map 块按 `item.title` 索引"模式。

## 范围

只覆盖 5 个 listing 命令。**`read` 不在本 PR 内**:它的 evaluate 块在
post-#1651 时代已经是 error-kind-discriminated 的富结构(`kind: 'inaccessible'`
/ `kind: 'http'` / `kind: 'malformed'`),原始 commit 的"POST 行带 media、
comment 行空"模式和当前结构冲突太深,单独的 read 接入留作后续 PR。

完全 additive:既有字段名、顺序、值都不变;新字段加在每行末尾。

## 验证

- `npx vitest run clis/reddit/ --project adapter` → 84/84 通过
- `node scripts/check-silent-column-drop.mjs` → current=97, baseline=97, new=0
- `npx tsc --noEmit` 干净
- `npm run build` 干净

* feat(reddit): expose home media route columns

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:14:37 +08:00
Ocean 4a6cfe8060 fix(xhs,youtube): 把合法空数据语义切到 EmptyResultError (#1674)
* fix(xhs,youtube): 把"合法空数据"语义切到 EmptyResultError

把 xiaohongshu/user 和 youtube/transcript 跟 bilibili/subtitle 已有的
structured-error 模式对齐,让下游能区分"用户/视频没内容"和"fetch 真的挂了"。

## xiaohongshu user

返回 `No public notes found for this Xiaohongshu user` 的场景——目标用户
零公开笔记(销号 / 私密 / 全删)——原来抛 plain `Error`,下游无法和
"真的 fetch 失败 / cookie 死"区分。

改抛 `EmptyResultError`,exit code 变 66,stderr 携带 `code: EMPTY_RESULT`,
跟 `bilibili subtitle` empty 同 shape。

## youtube transcript

`No captions available for this video`(作者没开 CC、YouTube 也没自动生成)
是数据条件,不是基础设施失败。原来跟 HTTP/解析错误一样抛
`CommandExecutionError`,造成调用方反复重试。

改这个特定 case 抛 `EmptyResultError`;其他 caption 错误(HTTP / parse /
empty response)继续走 `CommandExecutionError` 触发重试。

## 为什么 downstream 需要这个

调用方(如自动化采集流水线)通常对 "data.length === 0 && exitCode !== 0"
做 soft-rate-limit 启发式:N 次连续 soft fail 触发 24h 平台跳过。当 seed 列表
里有变质条目(XHS 账号销号 / YouTube 视频丢失字幕),"empty" 响应堆积会
误触跳过——cookies 和平台本身都健康。EmptyResultError 让调用方能区分
"这个用户没内容"和"API 挂了"。

## 测试

- `npx vitest run clis/xiaohongshu clis/youtube` —— 全过
- `npx tsc --noEmit` 干净

* fix(xhs): distinguish empty user notes from parser drift

* fix(empty): tighten legal empty evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:02:32 +08:00
Ocean bcb0fb362f feat(twitter): expose bio on read command
* feat(twitter): 在 read 命令上暴露 bio(用户简介)

`list-tweets` / `timeline` / `search` 三个读命令现在每行多一列 `bio`,从
`user.legacy.description` 抽。匹配 `profile` 命令已有的 `bio` 字段,让下游
消费者展示作者画像时省去"读了推文还要再读作者主页"的 roundtrip。

bio 在 user 对象缺失或没 description 时回落到 `''`。columns 数组同步更新,
`--format columns` 会渲染 bio。完全 additive:既有字段名、顺序、值都不变。

延续 #1660 (card binding_values) 和 #1667 (quoted_tweet) 的同一类
read-side enrichment 模式。

## 验证

- `clis/twitter/list-tweets.test.js` / `clis/twitter/search.test.js` 已有
  shape assertion 补上 `bio: ''` 行
- `timeline.test.js` 用 `toMatchObject`(子集匹配),新增 bio 不会破断言
- `npx vitest run clis/twitter/list-tweets.test.js clis/twitter/timeline.test.js
  clis/twitter/search.test.js --project adapter` → 48/48 通过
- `npx tsc --noEmit` 干净
- `npm run build` 干净
- `node scripts/check-silent-column-drop.mjs` → current=97, baseline=97, new=0

* test(twitter): cover inline bio extraction

* feat(twitter): expose thread author bio

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 18:56:38 +08:00
lenovobenben 85b1c07ba9 feat(zhihu): include answer links in question results
* feat(zhihu): include answer links in question results

* fix(zhihu): avoid fake answer links for malformed ids

* fix(zhihu): dedupe answers by trusted id

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 15:34:24 +08:00
Ocean 6fbaf0d5b8 feat(twitter): 在 read 命令上暴露 quoted_tweet(被引用的推文) (#1667)
* feat(twitter): expose quoted_tweet on read commands

When a tweet quotes another tweet (embedded preview with commentary), the
quoted tweet's content is in `tweet.quoted_status_result.result` — same
`legacy / core / card / note_tweet` shape as the outer tweet. Until now
none of the 5 read commands (list-tweets / timeline / thread / tweets /
search) surfaced this nested object, so downstream consumers couldn't
render the quoted preview card.

Adds `extractQuotedTweet(tw)` in shared.js (mirrors the
`extractMedia` / `extractCard` helper pattern) and threads it through
all 5 read commands plus their CLI `columns:` declarations.

Output shape is a deliberately small subset of the main tweet
(id/author/name/text/created_at/url + media + card). Counts and full
author bio are intentionally omitted to keep timeline payloads from
ballooning 2-3x; consumers needing those can re-fetch
`twitter thread <quoted_id>`.

Notable edge cases tested in shared.test.js:
- plain tweets (no `is_quote_status`) -> null
- tombstoned / unavailable quoted tweets (deleted / privacy-restricted) -> null
- TweetWithVisibilityResults `result.tweet` shim unwrap
- long-form note_tweet text preferred over truncated full_text
- quote-of-a-quote does NOT recurse (avoids payload explosion on threads
  where every reply re-quotes the root)

* fix(twitter): require quoted tweet render evidence

* fix(twitter): validate quoted tweet author shape

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 14:56:24 +08:00
Ocean 34f351e59f feat(reddit/subscribed): 接入 LoginWallError 嗅探(#1650 的第一个 caller) (#1668)
* feat(errors,utils): 添加 LoginWallError 与 HTML-as-JSON 响应嗅探器

部分 adapter(twitter list-tweets/thread、reddit search/subreddit 等)历史上
直接 `JSON.parse(await r.text())` 或 `await r.json()` 解析响应。当服务端返回
登录墙、限流页或 WAF 拦截页(而不是 JSON)时,body 以 `<!DOCTYPE html>` 或
`<html ...>` 开头,解析直接抛出晦涩的
`SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON`,
调用方无法把它和真正的 JSON 解析失败区分开。

本 PR 添加三块共享基础设施,让 adapter 能识别 HTML 情况并抛出结构化的
`LoginWallError`(带 status / url / body 预览),而不是裸的解析栈:

  - `LoginWallError`(src/errors.ts):新的 `CliError` 子类,含 `status`、
    `url`、`bodyPreview` 字段,hint 提示"重新登录或等待限流过期",
    退出码映射到 `EXIT_CODES.NOPERM`。
  - `parseJsonOrThrowLoginWall(response)`(src/utils.ts):Node 端 helper,
    供从 daemon 侧 fetch 的 adapter 使用(接收 Fetch Response)。
  - `BROWSER_JSON_SNIFF_FN` + `throwIfLoginWall(value)`(src/utils.ts):
    browser 端等价物。字符串片段嵌入到 `page.evaluate` 里,返回值是
    解析后的 JSON,或 `{ error: status }` HTTP 形状,或 `LoginWallSignal`
    哨兵对象(`{ __loginWall: true, status, url, ... }`),Node 侧拿到后
    转成 `LoginWallError`。

行为是 opt-in:现有 adapter 不调用这些 helper 就完全不受影响。后续 PR
会把 reddit / twitter adapter 接入这些 helper。`src/utils.test.ts` 新增
13 个单测,覆盖 Node 端 + browser 端两条路径以及 body 预览的 100 字截断。

* feat(reddit/subscribed): 接入 LoginWallError 嗅探(#1650 的第一个 caller)

`reddit subscribed` 是从 daemon-backed browser session 调 Reddit 的 JSON API
(`/api/me.json` + `/subreddits/mine/subscriptions.json`)。原本两处 `await res.json()`
在 Reddit 返回登录墙 / WAF / over-18 拦截页(HTML body + 200 OK)时会抛
`SyntaxError: Unexpected token '<'`,被外层 try/catch 兜底成 `kind: 'exception'`
→ Node 侧报成 `CommandExecutionError: subscribed failed: SyntaxError ...`,
看不出 root cause。

接入 #1650 的 helper 后:

- **Browser 侧**:用 `BROWSER_JSON_SNIFF_FN` 提供的 `fetchJsonOrLoginWall(url, init)`
  替换裸 `fetch + .json()`。helper 内部 sniff `Content-Type: text/html` 或
  `<!DOCTYPE` / `<html` body 前缀,返回 `{ __loginWall: true, status, url,
  contentType, bodyPreview }` 哨兵(不抛,交给调用者)。
- **Cross-boundary**:两处 fetch 站点(me.json + subscriptions.json)发现哨兵后
  返回 `{ kind: 'login-wall', sentinel, where }` 透传给 Node。
- **Node 侧**:`throwIfLoginWall(result.sentinel, { url: result.where })` 把
  哨兵转成结构化 `LoginWallError`(含 `status` / `url` / `bodyPreview` 字段,
  exit code `EXIT_CODES.NOPERM`,hint 提示重新登录或等限流过期)。

这是 #1650 的第一个真实 caller,覆盖 3 块 export 全部(`BROWSER_JSON_SNIFF_FN`
+ `throwIfLoginWall` + `LoginWallError`)。其它 adapter 后续按这个模板逐个接入。

回归测试新增 1 个:mock evaluate 返回 `{ kind: 'login-wall', sentinel }`,
断言 Node 端抛 `LoginWallError` 且 `status` / `url` / `bodyPreview` 字段正确。
原有 12 个测试不动,全部通过。
2026-05-19 14:52:42 +08:00
jakevin acc18be999 docs(readme): tighten skill attribution + remove redundant Highlights (#1666)
Per WAWQAQ T1 + T2 review:

T1 — skill attribution carries the same intent PR #1654 started but
hadn't fully cleaned up:
- Skill table row for `opencli-adapter-author` no longer claims it
  "operate[s] a site in real time" (SKILL.md explicitly says ad-hoc
  driving lives in `opencli-browser`). Browser-op example
  ("Help me check my Xiaohongshu notifications") moved to the
  `opencli-browser` row where it belongs.
- "How it works" section's 5 browser primitives (navigate / read /
  interact / extract / wait) now point to `opencli-browser` instead
  of `opencli-adapter-author`.
- Skill references list re-orders to surface `opencli-browser` first
  with a concrete description, and `opencli-adapter-author` no longer
  claims to cover "browser operation".

T2 — drop the Highlights section. Pre-Quick-Start had four parallel
summary blocks (3-line tagline / 3-bullet automation intro / CLI-hub
+ desktop line / 5-bullet Highlights) that all said the same thing.
Highlights was the most-recent and most-redundant of the four; the
remaining three carry the value props cleanly: tagline → three usage
modes → CLI-hub + desktop scope.

EN + ZH READMEs synced.
2026-05-19 13:28:44 +08:00
Benjamin Liu 67ed9e9c81 feat(linkedin-learning): add search / trending / course read commands (#1657)
* feat(linkedin-learning): add search / trending / course read commands (#1021)

Closes #1021. Adds a new linkedin-learning site adapter with three
read-only commands against LinkedIn Learning's public learning-api
REST surface. Shares cookie session with linkedin.com; Learning
queries are not subject to the people-search CUL.

Commands:
- linkedin-learning search <keywords>  searchV2?q=keywords
- linkedin-learning trending            feedRecommendationGroups?q=learner
- linkedin-learning course <slug>      courses?q=slug

Endpoints were discovered via browser network capture on
/learning/search and /learning/<slug> pages: searchV2 returns a flat
list of courses/videos/paths keyed by entityType, headline.title.text
holds the canonical title, length is a TimeSpan in seconds, and rating
is averaged from ratingSum/ratingCount when averageRating is missing.

trending walks the carousels array on each recommendation group, flattens
cards across them, dedups by slug, and respects --limit. Group is
labeled with the carousel title (e.g. "Top picks for you") or the
upstream annotation tag (TOP_PICKS).

course accepts either a bare slug or a full /learning/<slug> URL, then
hits /learning-api/courses?q=slug. The detail endpoint omits rating
fields even when search reports them; this is documented in the
adapter doc rather than fixed via a second /reviews fetch to keep the
PR scoped to one endpoint per command.

CUL caveat: Learning's API has no per-month limit, so dev iterations
can be much more aggressive than the people-search adapter (#1649).
Three commands were live-verified against a logged-in account with
60s sleeps between calls (conservative for first-pass safety).

Tests: 28 unit tests across search.test.js (12), trending.test.js (6),
course.test.js (10) cover URL construction, limit validation, author
join, duration / rating coercion, row mapping, carousel flattening
and dedup, slug parsing from URL forms, and the standard auth /
empty / fetch-failure error paths.

Live verified:
- search "AI agent" --limit 3: 3 rows with title/instructor/rating
- trending --limit 3: 3 personalized course picks
- course agentic-ai-build-your-first-agentic-ai-system: title, 3932s
  duration, 18 videos, release date 2026-03-27

* fix(linkedin-learning): harden read result boundaries

* fix(linkedin-learning): require course title evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 13:17:33 +08:00
Benjamin Liu 8577d88ee7 fix(cli): escape leading-dash positional values via argv preprocessor (#1658)
* fix(cli): escape leading-dash positional values via argv preprocessor (#1160)

Closes #1160. `opencli boss detail -abc123def` failed with
`error: unknown option '-abc123def'` because commander treats any
argv token starting with `-` as an option. BOSS 直聘 securityId
tokens are opaque base64-ish strings that can legitimately start
with `-`, and the same shape is possible for any adapter that takes
an opaque-id positional.

Adds escapeLeadingDashPositional() to src/cli-argv-preprocess.ts,
called from main.ts after the existing rewriteBrowserArgv pass. The
preprocessor:

- Reads cli-manifest.json (the same manifest the registry uses) and
  builds a set of `<site>/<cmd>` keys whose first positional is
  required.
- Walks past root flags (matching the existing rewriteBrowserArgv
  walker) to find the site + command tokens.
- If the next argv token starts with `-`, is not the recognised
  short flags `-f` / `-v` / `-h`, is not `--*`, and is not the
  pre-escaped `--` separator, inserts `--` before it.

Tests: 12 new unit tests in cli-argv-preprocess.test.ts cover the
basic insertion, trailing-flag preservation, non-touched cases
(normal values, recognised short flags, long flags, already-escaped,
non-positional commands, unknown commands, short argv, and the
`--profile work boss detail -abc` form that walks past a root
value flag).

Live verified: `node ./dist/src/main.js boss detail -abc123def`
no longer raises 'unknown option'. The adapter now receives the
dash-leading value and proceeds to fetch, where it correctly
surfaces an upstream "missing required parameter" error for the
fake id used in this smoke test.

* fix(cli): preserve options around dash positionals

* fix(cli): preserve attached short option values

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 13:13:24 +08:00
Ocean cd731bd2aa feat(twitter): 在 read 命令上暴露 card binding_values(链接预览卡片) (#1660)
* feat(twitter): expose card binding_values on read commands

Surface tweet link-preview cards (title, description, image, domain, landing URL)
on `search`, `list-tweets`, `thread`, and `timeline` so downstream renderers
can build native-style link cards without re-fetching. Pure GraphQL-response
extractor — no query strategy, interceptor, or network changes.

extractCard returns null when the tweet has no card or when the card is
structurally empty (no url AND no title/description). Missing fields are
omitted from the output to keep JSON consumers clean.

* fix(twitter): bind cards to matching URL entity

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 12:55:19 +08:00
Ocean d1c714ecd3 feat(twitter): 新增 list-create 命令(GraphQL CreateList mutation) (#1656)
* feat(twitter): add list-create command

Adds a new `twitter list-create` command so users can create Twitter/X
lists from the CLI (the existing list commands only covered reading,
adding, and removing members). Uses the GraphQL CreateList mutation
with the same cookie + CSRF pattern as list-add, no UI clicks needed.

Args: name (positional, max 25), --description (max 100), --mode (public|private).
QueryId resolved at runtime via resolveTwitterQueryId, with a known
fallback for offline / bundle-scan misses.

* fix(twitter): pin list-create queryId + features to a working pair

Twitter's GraphQL rejects CreateList when queryId and the features
schema drift apart (DecodeException). Stop resolving the queryId
dynamically (which would pull a newer schema), hardcode a known-good
queryId, and trim features to the minimal set the real web client
sends.

Also: Twitter sometimes returns a non-fatal errors array from a
side-effect serializer while still creating the list. Check for a
valid list payload first and only treat errors as fatal when no
list came back.

* fix(twitter): add missing access:'write' on list-create (#9)

`twitter/list-create` was missing the required `access` field, which made
manifest validation fail on every opencli invocation and spam stderr with:

  ⚠  Failed to load manifest .../cli-manifest.json: Command
     twitter/list-create must declare access: 'read' | 'write'

Per docs/conventions/convention-audit.md (rule missing-access-metadata),
every adapter command must declare access. Since list-create is a create
action, set access: 'write'.

Also rebuilds cli-manifest.json — picks up missing `quoted_tweet` columns
on list-tweets / search / list-tweets-username from PR #8 (which didn't
rebuild the manifest).

* fix(twitter): harden list-create mutation contract

* fix(twitter): verify created list name

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: Kary <karyhe1019@gmail.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 12:51:57 +08:00
dependabot[bot] 8182ffbe89 chore(deps): bump tsx from 4.21.0 to 4.22.2 (#1663)
Bumps [tsx](https://github.com/privatenumber/tsx) from 4.21.0 to 4.22.2.
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.21.0...v4.22.2)

---
updated-dependencies:
- dependency-name: tsx
  dependency-version: 4.22.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:41:48 +08:00
dependabot[bot] 5a4984789d chore(deps): bump @types/node from 25.6.0 to 25.9.0 (#1664)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.6.0 to 25.9.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.9.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:40:20 +08:00
dependabot[bot] e1185da882 chore(deps): bump ws from 8.20.0 to 8.20.1 (#1662)
Bumps [ws](https://github.com/websockets/ws) from 8.20.0 to 8.20.1.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/8.20.0...8.20.1)

---
updated-dependencies:
- dependency-name: ws
  dependency-version: 8.20.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:40:14 +08:00
dependabot[bot] 9446bddb60 chore(deps): bump undici from 6.25.0 to 8.3.0 (#1661)
Bumps [undici](https://github.com/nodejs/undici) from 6.25.0 to 8.3.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.25.0...v8.3.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:37:31 +08:00
Ocean 0c4bcdbb86 feat(reddit): 新增 subscribed 命令 + 在 listing 命令上暴露 id / created_utc / selftext (#1651)
* feat(reddit): subscribed command + expose id/created_utc/selftext on listing commands

Adds `opencli reddit subscribed` to list the user's subscribed subreddits,
mirroring `saved.js`'s cookie auth + AuthRequiredError pattern. Auto-paginates
via `/subreddits/mine/subscriptions.json` (max 1000 subs, default 100).

Also extends the JSON output of `popular` / `search` / `subreddit` with
`id`, `created_utc`, `selftext` (and `author` on popular) — the table
view stays clean (columns: unchanged), but `--format json` now surfaces
fields needed for downstream content-recommendation tooling that filters
by post age, dedupes by post id, or uses self-post bodies for embeddings.

Tests: 4 new vitest cases for subscribed.js (happy / auth fail / HTTP /
--limit truncation). All existing reddit tests still pass.

Note on cli-manifest.json diff: the rebuild on fork/main drops 13 entries
whose source files import lowercase `selectorError` from
`@jackwener/opencli/errors` (the actual export is `SelectorError` —
casing bug pre-existing in fork/main). Not introduced by this PR.

* fix(reddit): harden subscribed listing contract

* fix(reddit): require subreddit identity for subscriptions

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 02:36:17 +08:00
lenovobenben ec3b7dadf3 fix(zhihu): decode numeric entities in answer detail (#1629)
Co-authored-by: lihaidong <lihaidong@kingsoft.com>
2026-05-19 02:14:48 +08:00
ele-yufo 4de04c43ad feat(suno): add suno.com music-generation adapter (#1638)
* fix(suno): harden generation adapter contracts

* fix(suno): separate session auth and API failures

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 02:12:03 +08:00
Benjamin Liu 40592ea5fd fix(adapters): drop silent-sentinel row fallbacks across Apple Podcasts, Reddit, and Gitee (#1634)
* fix(adapters): drop silent-sentinel row fallbacks across Apple Podcasts, Reddit, and Gitee

Continues the audit-baseline cleanup from #1611 (lesswrong) and #1631
(wikipedia / 36kr / xiaoyuzhou / zhihu), and follows the direction set
by 71646158 (silent-empty-fallback resolutions across Douyin / Jike /
WeRead) and ee54eb8e (ignore sentinels in thrown errors).

Replaces silent-sentinel row fallbacks with the empty-string signal so
agents can tell apart "field has value Unknown" from "upstream returned
no value":

- apple-podcasts/search: episodes, genre
- reddit/saved: title
- reddit/upvoted: title
- gitee/search: language, description

All four files audited for downstream sentinel checks via
`grep -nE "=== ?['\"](Unknown|unknown|-)['\"]"`. None reference the
swapped values in control flow (verified against the v2ex/me.js class
of regression caught in #1631).

Intentionally skipped in this batch (will not flip to empty):
- gitee/trending.js:272: downstream `project.description !== '-'`
  check drives the mergedDescription fallback. Same control-flow
  sentinel pattern as v2ex/me.js. Stays on baseline.
- web/read.js x4: `'-'` lives inside rendered diagnostic lines
  (`lines.push(...)`), not row fields. Empty would render
  `  GET    /a/b` with a doubled space. UX placeholder.
- yollomi/{edit,video}.js x6: `file: '-'`, `size: '-'`, `credits: '-'`
  are user-facing status rows displayed to humans. Empty would
  collapse columns visually.
- zsxq/dynamics.js: `title: '[${d.action || 'unknown'}]'` is a
  template-literal-rendered title prefix. Empty would render `[]`.

Verified live: `opencli apple-podcasts search "lex fridman" --limit 2`
returns populated episodes/genre. `opencli gitee search "vue" --limit 2`
returns populated language/description. Baseline shrinks accordingly.

* test(adapters): add empty-signal coverage for the cluster-3 sentinel swap

Mirrors the cluster-2 test additions, pairing the sentinel value swap
in this PR with focused unit tests that mock the upstream to return
null / missing fields and assert the row surfaces an empty-string
signal instead of the old fabricated '-' / 'unknown' sentinel.

Coverage:

- clis/apple-podcasts/commands.test.js (+1 case): stubs the iTunes
  Search response with a result that has collectionId / collectionName
  / artistName populated but no trackCount and no primaryGenreName.
  Asserts episodes and genre render as '' (was '-' before this PR).

- clis/gitee/search.test.js (new): mocks Gitee's `so.gitee.com/v1/search`
  fetch with two cases - a hit that has only title + url (no langs,
  no description), and a hit that has all fields populated. Asserts
  the missing fields render as '' (was '-' before) and that populated
  fields pass through verbatim.

The reddit/saved and reddit/upvoted changes in this PR live inside a
page.evaluate template literal that fetches from reddit.com inside
the browser context, so the empty-signal branch is executed inside
the page rather than in adapter JS. They are 1-char `|| '-'` ->
`|| ''` swaps with no downstream sentinel consumer and the same JS
semantics demonstrated by the gitee + apple-podcasts tests above.

* chore: rebuild cli-manifest.json to drop stale entries from rebase

The previous rebase left a stale linkedin/people-search entry in
cli-manifest.json that was carried over from a sibling feature branch.
This branch does not include the people-search source file, so the
entry was an orphan; CI's build-manifest safety check correctly
refused to overwrite it. Regenerating with --allow-removals to drop
the orphaned entry, after which a normal `npm run build` is a no-op.
2026-05-19 01:52:52 +08:00
Ocean 942539a695 fix(twitter/lists): 跳过 "Discover new Lists" 推荐区块,避免被当成用户的 list 抓取 (#1652)
* fix(twitter): skip "Discover new Lists" recommendations in lists adapter

The X.com /<user>/lists page powers two sections from a single
ListsManagementPageTimeline GraphQL response: "Discover new Lists"
(algorithmic recommendations) and "Your Lists" (owned + subscribed).
The previous parser ignored entry.entryId entirely and returned every
list it found, so recommendations leaked through and downstream
consumers treated them as the user's own lists.

X distinguishes the sections by entry.entryId prefix:

  owned-subscribed-list-module-*  → owned + subscribed (keep)
  list-to-follow-module-*         → Discover recommendations (drop)
  cursor-*                         → pagination cursor (no list payload)

Filter on the owned-subscribed prefix in parseListsManagement and
expose isOwnedSubscribedEntry for testing. The existing test fixture
used a fictional entryId shape that no longer matches real responses;
update it to the nested-module shape Twitter actually returns and add
two new tests: one proving Discover entries are skipped, and one for
the entryId classifier.

Verified end-to-end against a live account: 10 raw entries (3 Discover
+ 7 owned/subscribed) now correctly return 7 owned/subscribed lists
with zero leakage.

* fix(twitter): harden lists parser boundary

* fix(twitter): require list-remove postcondition evidence

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 01:00:19 +08:00
Ocean dc645a5bcf fix(youtube/transcript): 把 timedtext URL 匹配限定到当前 videoId,修跨视频字幕串台 (#1655)
* fix(youtube/transcript): scope timedtext URL match to current videoId

YouTube watch-page is an SPA — page.goto between watch URLs preserves
performance.getEntriesByType('resource') entries from prior videos.
findTimedtextUrl filtered only by lang, so a previously-viewed
same-language video's timedtext URL could be picked up by the polling
loop before the current video's fetch hook captured a fresh one,
returning the wrong video's captions to the caller.

Fix: require URLs to contain v=<currentVideoId> across all three paths:
  - in-page findTimedtextUrl (resource-buffer scan)
  - in-page isJson3TimedtextUrl (fetch/XHR hook)
  - Node-side extractSegmentsFromNetworkCapture (CDP capture)

Most likely to hit callers that reuse a single daemon tab to fetch
many transcripts back-to-back (e.g. ml-scout). Confirmed in the wild:
a Fox News Ukraine clip got Whisper Flow promo captions written to
its row when the prior call on the same tab pulled an English
Whisper Flow video.

Adds one source-contract assertion (both in-page sites use a shared
videoIdMarker) and one behavioral test (CDP capture buffer with a
stale 'v=prev' entry alongside the current 'v=abc' returns only the
current video's captions).

* fix(youtube): exact-match transcript timedtext video id

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 00:53:39 +08:00
jakevin f0d9aa187c docs(readme): fix skill attribution for "operate any website" use case (#1654)
Per WAWQAQ feedback: the intro section's "Let AI Agents operate any
website" bullet mistakenly references `opencli-adapter-author`, which
is the skill for **writing** adapters (correctly referenced in the
adjacent "Write new adapters" bullet). The skill for ad-hoc browser
driving is `opencli-browser` — its own SKILL.md frontmatter explicitly
says "Not for writing adapters — see opencli-adapter-author for that",
and `opencli-adapter-author` says "For ad-hoc browser driving (no
adapter), see opencli-browser instead".

Two locations affected with the same error: the intro bullet and the
Core Concepts > `browser` section. Both EN and ZH READMEs updated.
2026-05-19 00:37:32 +08:00
Benjamin Liu e82e32abc6 feat(12306): add full read adapter (stations / trains / train / price / me / passengers / orders) (#1637)
* feat(12306): add stations / trains / train read commands (no login required)

Adds a first-pass 12306 (中国铁路) adapter for the public anonymous
query endpoints. Closes the no-login slice of #1589. The
authenticated `me / passengers / orders` commands the issue
proposes are explicitly left as a follow-up.

Commands:
- 12306 stations <keyword>             search station bundle
- 12306 trains <from> <to> --date YYYY-MM-DD  availability between stations
- 12306 train <train-no> --from <s> --to <s> --date  stop list

All three use Strategy.PUBLIC + browser: false, anonymous, no cookie
storage, no CAPTCHA bypass. Sensitive behaviors the issue rules out
(ticket sniping, order submission, payment, anti-abuse circumvention,
password storage) are not implemented.

Notes worth flagging for review:

- 12306 rejects anonymous query endpoints with HTTP 302 to
  /mormhweb/logFiles/error.html. The adapter first hits
  /otn/leftTicket/init to mint JSESSIONID / route / BIGipServerotn
  cookies, then attaches them to subsequent queries. No CAPTCHA path.

- 12306 rotates the train-query endpoint name (queryO / queryZ /
  queryA / queryG) every few weeks. When the wrong name is hit the
  server returns `{c_url: "leftTicket/queryX", status: false}`
  pointing to the current correct name. The adapter walks a list of
  known names, captures the rotation hint, and retries; the runtime
  list is also mutated so subsequent calls in the same process skip
  the warm-up round trip.

- The `|`-separated train wire format includes a booking-handshake
  `secret` field at position 0. Since this PR is read-only and the
  issue explicitly rules out booking, that field is parsed but not
  surfaced in the returned row, and a unit test asserts it cannot
  leak via the public adapter contract.

- Station resolution accepts Chinese name (`上海虹桥`), telecode
  (`AOH`), full pinyin (`shanghaihongqiao`), or short alias (`shhq`).
  Anything else raises ArgumentError with a hint.

- `limit` arguments use a tight validator that throws ArgumentError
  on non-integer / out-of-range input rather than silently clamping,
  matching the typed-error pattern used in #1397 (grok) and #1370
  (coupang).

Live verified anonymously against kyfw.12306.cn:
- `12306 stations 上海 --limit 5` returns 5 stations including
  上海 (SHH) / 上海南 (SNH) / 上海虹桥 (AOH).
- `12306 trains 北京 上海 --date 2026-05-22 --limit 1` returns
  G547 06:18 -> 12:11 with first / second / business / no-seat
  availability columns populated.
- `12306 train 24000000G10L --from 北京南 --to 上海虹桥 --date 2026-05-22`
  returns the 7-stop G1 route from 北京南 through 沧州西 / 德州东 /
  曲阜东 / 南京南 / 苏州北 to 上海虹桥, with arrival / departure /
  stopover times.

Tests: 18 unit tests covering parseStationBundle, resolveStation
(including ambiguous / case-insensitive cases), validateDate,
buildCookieHeader, parseTrainRecord (including a regression test
asserting the `secret` field cannot leak into the row).

Deliberately deferred to a follow-up: `12306 price`. The
queryTicketPrice endpoint needs train_no + per-stop station_no +
per-train seat-type letters, so an ergonomic `12306 price <code>`
would cascade three API calls (trains -> stops -> price) per
invocation. Wanted to keep this PR's blast radius small. If the
maintainer prefers a Phase 1 that includes price even with the
cascading-call cost, happy to add it.

* feat(12306): add me / passengers / orders / price authenticated + price read commands

Completes the #1589 12306 (中国铁路) adapter on top of the
stations / trains / train slice landed in the prior commit of this
branch. The full command set is now:

  Anonymous (no login):
    12306 stations  search station bundle by Chinese / telecode / pinyin
    12306 trains    list trains between two stations on a date
    12306 train     list stops of one train
    12306 price     ticket prices for one train segment + date

  Authenticated (cookie session):
    12306 me        account summary (sensitive fields masked by default)
    12306 passengers  saved-passenger list (sensitive fields masked)
    12306 orders    in-progress orders (not yet ridden / refunded)

Notes worth flagging for review:

- 12306 sets the auth cookie `tk` and the session cookie `JSESSIONID`
  with `Path=/otn`. CDP `Network.getCookies` filters by URL path, so
  `page.getCookies({ url: 'https://kyfw.12306.cn' })` returns 7
  cookies without `tk` / `JSESSIONID`, even on a freshly-navigated
  logged-in tab. Switched the login check to read `document.cookie`
  via `page.evaluate`, which the current navigated page exposes
  regardless of cookie path. Centralized as `require12306Login` in
  utils.js so all three authenticated commands share the same check.

- All authenticated commands mask sensitive fields by default:
  - `me`: real name (Chinese mask), email, mobile (12306 already
    masks server-side), birth date (year only).
  - `passengers`: name + birth year by default; 12306 already masks
    ID number and mobile server-side and this adapter never decodes
    those.
  - Both expose `--include-sensitive` to opt back into the unmasked
    fields the user is entitled to see on their own account.

- `orders` returns the `queryMyOrderNoComplete` slice (orders that
  have not yet been ridden / refunded / completed). The historical
  `queryMyOrderApi` endpoint requires extra page-state handshakes
  that proved fragile when probed; left as a follow-up so this
  command can ship reliably for the immediate "what's still on my
  account" use case.

- `price` cascades three anonymous API calls per invocation:
  init -> queryByTrainNo (to resolve segment station_no within the
  train route) -> queryTicketPrice. 12306 returns prices keyed by
  one-or-two-letter seat codes (`A9` 商务座 / `M` 一等座 /
  `O` 二等座 / `WZ` 无座 / etc.) and additionally doubles some up
  as bare numeric codes (e.g. `"9": "21580"` mirrors
  `"A9": "¥2158.0"`); the bare-numeric duplicates are filtered out
  so the row set is one-per-seat-class.

- Strictly anonymous queries; no CAPTCHA / slider / SMS bypass, no
  credential storage, no ticket sniping, no order submission, no
  payment - per the issue's Non-goals list.

Live verified anonymously and authenticated against kyfw.12306.cn,
sleeping 15-25 seconds between hits to keep 12306's anti-abuse
throttle gentle:

  - 12306 me: account summary returned with real_name / email /
    mobile / birth date all masked at the adapter level, on top of
    12306's own server-side mobile mask.
  - 12306 passengers: every saved passenger returned with name
    masked to `<surname>*<...>` and 12306-side ID/mobile masks
    preserved verbatim.
  - 12306 orders: empty for this test account (no in-progress
    orders), correct EmptyResultError surface.
  - 12306 price G1 北京南 -> 上海虹桥 2026-05-22: returns
    商务座 ¥2158 / 特等座 ¥1163 / 一等座 ¥1035 / 二等座 ¥626 /
    无座 ¥626, sorted desc.

Tests: 23 unit tests (5 new beyond the prior commit's 18) cover
the mask helpers (email / mobile / Chinese name) plus the
parsePriceData filter that drops the bare-numeric duplicates and
sorts by descending price.

* fix(12306): harden browser auth boundaries

* fix(12306): tighten API drift boundaries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:58:48 +08:00
陈家名 254d51835f fix: keep media filenames in output directory (#1642)
* fix: keep media filenames in output directory

* fix(download): sanitize media filename segments

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:55:31 +08:00
Ocean 87dfb68e74 fix(browser): goto 重试时回收陈旧 page identity + 把 -32000 "Cannot find default execution context" 归类为可重试 (#1645)
* fix(browser): recover from stale page identity on goto retry (#5)

When a chrome-backed adapter pre-navigates after its cached `_page`
targetId has been invalidated (tab closed externally, identity evicted),
the extension throws `Page not found: <id> — stale page identity` and
the failure cascades — every subsequent persistent-site session call in
the same process keeps re-sending the same dead targetId.

Observed in a downstream parallel multi-platform recall: a single dead page handle
got reused across 4+ calls (twitter thread / twitter search / reddit search)
because there was no detection or recovery. The same hash appeared in
adapter pre-navigations to youtube, twitter, reddit, xhs back-to-back in
seconds, suggesting the cached `_page` was shared via persistent site
session leases (`site:youtube` etc) and never cleared after the first
"stale page identity" response.

Page.goto() now catches that specific error, drops `_page`, and retries
once without the stale id. The retry navigates via session-lease
resolution in the extension (resolveTab → preferredTabId / new owned tab),
which already handles tab eviction correctly. No effect on the happy path.

Three regression tests in src/browser/page.test.ts cover:
- recovery: stale id dropped, retry succeeds with new identity
- no-cache safety: fresh page with no _page → error propagates unchanged
  (nothing to drop, retrying would loop)
- error scoping: unrelated extension errors (e.g. disconnected) still
  surface immediately — no implicit retry

* fix(errors): classify -32000 "Cannot find default execution context" as retryable (#6)

classifyBrowserError previously only matched CDP -32000 errors when the
message contained "target" (e.g., "target closed"). It missed
"Cannot find default execution context", a CDP protocol error that also
indicates the inspected target went away — observed in a downstream parallel
adapter recall against youtube channels.

Widening the secondary check to `/target|context/i` lets the existing
target-navigation retry path (200ms delay + re-attach) recover instead of
surfacing the error as non-retryable.

* fix(browser): tighten stale page recovery notes

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:30:28 +08:00
lenovobenben 000c867f3a fix(zhihu): harden search pagination (#1615)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 20:44:08 +08:00
Jun 24f643af16 feat(xianyu): add inbox, messages, and reply commands (#1639)
* fix: tighten internal callback types

* feat(xianyu): add private message commands

* fix(xianyu): harden IM command contracts

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 20:41:51 +08:00
hanzi 1f30a9027b feat(linkedin): consolidate messaging and Sales Navigator commands (#1647)
* fix(linkedin): harden sales navigator commands

* fix(linkedin): harden salesnav message boundaries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:58:34 +08:00
jakevin 72f2b020de feat(weread-official): add official gateway CLI
Add the WeRead official Agent Gateway as an in-tree pure HTTP adapter with 8 commands, typed errors, tests, and docs.
2026-05-18 19:46:00 +08:00
Ocean 261b8bfbb5 build: restore +x on dist/src/main.js after tsc rebuild (#1644)
clean-dist deletes dist/ and tsc --build re-emits files without preserving
the executable bit on the bin entry. Symlinked global install then hits
EACCES on spawn until manually chmod'd. Chain a chmodSync into the existing
prebuild-manifest hook so any future rebuild self-heals.

node -e instead of bare `chmod +x` to keep the script portable (npm runs
on Windows via Git Bash where chmod is a no-op, but fs.chmodSync still
silently no-ops there too — no extra branching needed).

Co-authored-by: Kary <karyhe1019@gmail.com>
2026-05-18 19:43:28 +08:00
Benjamin Liu 1e7ebe7f27 feat(twitter): rewrite download profile path on GraphQL UserMedia with cursor pagination (#1636)
* fix(twitter): harden profile media download

* fix(twitter): fail closed on repeated media cursor

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:33:29 +08:00
Benjamin Liu 7e44e71150 fix(lesswrong): drop "Unknown" silent sentinel in author column (#1611)
* fix(lesswrong): drop "Unknown" silent sentinel in author column

Twelve lesswrong commands had `author: item.user?.displayName ?? 'Unknown'`
which masks the missing-author signal: an agent reading the result row
cannot distinguish "post has no associated user" from "author is literally
named Unknown". The repo's typed-error lint flags this pattern
(silent-sentinel rule, see scripts/check-typed-error-lint.mjs:323).

Replace `?? 'Unknown'` with `?? ''` so the missing-author case stays
visible as an empty string. Consistent with `clis/lesswrong/_helpers.js:68`
which was already using the empty-signal form.

Shrinks scripts/typed-error-lint-baseline.json from 173 to 161 entries.

Follows the same direction as #1603 (fix(adapters): surface silent empty
fallbacks).

Verified live: `opencli lesswrong frontpage --limit 2 -f json` returns
real posts with non-empty author values; empty-author rows would now
show `"author": ""` instead of fabricating `"Unknown"`.

* test(lesswrong): add empty-signal coverage for the author sentinel swap

Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with a focused unit test that
mocks the upstream LessWrong GraphQL response to return posts where
`user` is null or `user.displayName` is missing, and asserts the row
surfaces `author: ''` instead of the old fabricated `'Unknown'`.

`clis/lesswrong/frontpage.test.js` is representative for the twelve
identical `author: item.user?.displayName ?? ''` swaps across
comments / curated / frontpage / new / read / sequences / shortform /
tag / top / top-month / top-week / top-year, all of which share the
exact same expression with no downstream sentinel consumer.

The empty-signal path is exercised live too: a deleted-account or
permission-restricted user shows up in the GraphQL response with
`user: null`, surfaces as `author: ''` post this PR (was 'Unknown'
before).
2026-05-18 19:18:51 +08:00
Benjamin Liu 76a9c78261 feat(weibo): add delete command to remove user's own posts (#1620)
* feat(weibo): add delete command to remove user's own posts

Adds `opencli weibo delete <id>` so the same workflow that creates a
post can also remove one without leaving the CLI. The id positional
accepts either the numeric `idstr` (e.g. `5299336218674412`) or the
base62 `mblogid` (e.g. `QFGbHAoBS`) found in any weibo URL or in the
output of `weibo me` / `weibo feed` / `weibo post`.

Implementation lives in a single `page.evaluate` IIFE so cookies +
the XSRF-TOKEN double-submit token stay first-party:

  1. Resolve mblogid / idstr via `GET /ajax/statuses/show?id=<input>`,
     which returns the canonical `idstr`. Empty result -> 404 path.
  2. Read the `XSRF-TOKEN` cookie via `document.cookie`.
  3. `POST /ajax/statuses/destroy` with `id=<idstr>` body and the
     `X-Xsrf-Token` header.
  4. Return `[{ status: 'deleted', id, mblogid }]`.

Typed errors:
- 401 / 403 from either show or destroy -> `AuthRequiredError`
- `show` returning no `idstr` -> `EmptyResultError`
- Non-2xx HTTP on either call -> `CommandExecutionError` with status
- API response `ok !== 1` -> `CommandExecutionError` with the API msg

Closes #1619.

Verified live on macOS / opencli v1.7.22, weibo cookie session:
- Deleted the lingering test post from #1602 verification
  (idstr=5299336218674412, mblogid=QFGbHAoBS):
  `weibo delete QFGbHAoBS` returned
  `[{ status: 'deleted', id: '5299336218674412', mblogid: 'QFGbHAoBS' }]`
- `weibo me` shows `statuses: 3` (was 4 before the delete)
- `weibo post QFGbHAoBS` now throws "Post not found"

Unit tests: 8 / 8 in `clis/weibo/delete.test.js` (happy path,
empty-id, auth, not-found, show-http, destroy-http, api-msg, envelope
unwrap). Full weibo suite: 38 / 38 pass.

* fix(weibo): require delete postcondition evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:09:47 +08:00
Benjamin Liu 030a0ad885 feat(xiaohongshu): add delete-note command to remove published notes (#1624)
* fix(xiaohongshu/publish): invoke shadow-DOM publish handler directly

XHS creator center now wraps the publish/save-draft button in an
`<xhs-publish-btn>` web component backed by a CLOSED shadow root.
Calling `.click()` on the host element does not dispatch into the
internal handler, and CDP coordinate clicks cannot penetrate the
shadow boundary. The previous text-match `button.click()` loop hit
the host element, returned `ok`, and yet the note silently stayed
on the publish page as a draft, so the adapter reported the soft
`⚠️ 操作完成,请在浏览器中确认` status while nothing was actually
posted.

Invoke the publish/save method directly on the `<xhs-publish-btn>`
host (`_onPublish` / `_onSave` and a few candidate names XHS has
shipped historically). Fall back to the legacy
`<button>`/`[role="button"]` text-match click for older
creator-center variants that still expose plain buttons.

Patch shape suggested by the OpenCLI autofix report in #1606 from
@chcc-funny (who verified an end-to-end real publish locally).

Closes #1606.

Verified live on macOS / opencli v1.7.22 / extension v1.0.15,
with creator center logged in:
- `opencli xiaohongshu publish ... --draft` -> ` 暂存成功`,
  creator home shows "草稿箱中有未发布的作品"
- `opencli xiaohongshu publish ...` (real publish) -> ` 发布成功`,
  note appeared on the account feed (visible from mobile app);
  test note deleted after verification

Unit tests: 12 / 12 in `clis/xiaohongshu/publish.test.js` pass
(mocks updated to reflect the new `{ ok, via, name|text }` invoke
result shape).

* feat(xiaohongshu): add delete-note command to remove published notes

Adds `opencli xiaohongshu delete-note <note-id>` so the workflow that
creates a note can also remove one without leaving the CLI, mirroring
`weibo delete` (#1619 / #1620).

The creator-center HTTP delete API requires the `X-S-Common` signature
header that `publish.js` deliberately avoids, so this follows the same
UI automation route. Flow:

  1. Navigate to creator note-manager
  2. Switch to "已发布" tab (delete entry only appears there; "审核中"
     and "未通过" rows have no web delete action, mobile app only)
  3. Locate the `.note` row whose `data-impression` JSON contains the
     target noteId (exact JSON-parsed match, not substring, so values
     that happen to share the noteId prefix in other fields cannot
     match the wrong row)
  4. Click the inline `<span class="control data-del">` action
  5. Click "确定" in the `.d-modal-footer` confirmation modal
  6. Poll for the row disappearing (iteration-bounded so tests with
     mocked `page.wait` exhaust the loop quickly)

Typed errors:
- /login redirect after navigation: AuthRequiredError
- 已发布 tab not found / not clickable: CommandExecutionError (UI drift)
- target noteId not present in the rendered list: EmptyResultError with
  a hint about review-state limitation
- row found but no delete action visible: CommandExecutionError
- confirmation modal missing / no 确定 button: CommandExecutionError
- row still visible after the configured poll window: CommandExecutionError

Closes #1623.

Verified live: published a test note, deleted via this adapter, follow-up
`xiaohongshu creator-notes` confirms it is gone. Unit tests: 8 / 8 cover
happy path, empty-id ArgumentError, login redirect AuthRequiredError,
tab-not-found CommandExecutionError, row-not-found EmptyResultError,
no-delete-action / no-modal / unverified-delete CommandExecutionError
paths.

Built on top of #1613 (xiaohongshu publish shadow-DOM fix) so the live
verify could exercise publish-then-delete end to end. Will rebase onto
main once #1613 lands.

* fix(xhs): make delete-note fail closed

* fix(xiaohongshu): harden delete-note boundary

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:55:57 +08:00
Benjamin Liu e29150bab5 fix(weibo/publish): replace brittle CSS-module hash with placeholder selector (#1625)
* fix(weibo/publish): replace brittle CSS-module hash with placeholder selector

`clis/weibo/publish.js` matched the compose textarea via
`textarea._input_13iqr_8`, where `_input_13iqr_8` is the Vite CSS-module
hash Weibo rebuilds on every frontend deploy. The hash drifted (current
build emits `_input_1f5hn_8`), so step 4 of the publish flow throws
"Weibo compose editor did not appear" before anything else can run.
Reported in #1602.

Replace the single hashed selector with a placeholder-text-based chain
that survives Weibo's CSS-module rebuilds:

  textarea[placeholder*="有什么新鲜事"]
  textarea[placeholder*="新鲜事"]
  textarea._input_13iqr_8     // legacy hash kept last for older variants

Two visible textareas can match on the home feed (the always-rendered
"home-strip" prompt + the post-click modal compose). Pick the LAST
visible candidate: the modal opens on top and is appended to DOM later,
so the last-visible textarea is the modal. Both the editor-visibility
poll (Step 4) and the text-insertion step (Step 6) use the same chain.

Also drops `evaluateWithArgs` from Step 8 success polling. The IIFE
there does not reference any outer args, but `evaluateWithArgs` injects
its `const`-bound parameter names into the page context, and re-running
on each iteration of the success-poll loop threw `Identifier
'maxIterations' has already been declared` after the first iteration.
This was masked previously because Step 4 always failed first; with the
selector fixed, the latent Step 8 bug surfaces. Switched to plain
`page.evaluate` to avoid re-declaring per loop.

Closes #1602.

Verified live on macOS / opencli built locally / extension v1.0.15,
weibo cookie session:
- `opencli weibo publish "明洞那家店真不错"` returned
  `status: success, message: 发布成功, text: 明洞那家店真不错`
- Confirmed via `/ajax/statuses/mymblog`: the post landed at
  `idstr=5299403716821218`, `mblogid=QFHWzsCvE`, text matches what
  was typed (proves selector chain picks the right textarea and the
  text insertion path works end-to-end)
- Cleaned up: deleted via the same `/ajax/statuses/destroy` path that
  PR #1620 exposes as `weibo delete`

Unit tests: 8 / 8 in `clis/weibo/publish.test.js` pass (mocks updated
to reflect the new `evaluate`-vs-`evaluateWithArgs` split for Step 8
and the longer poll window).

* test(weibo): lock publish placeholder selector path

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:44:46 +08:00
Benjamin Liu a50074d684 fix(adapters): drop silent-sentinel row fallbacks across 6 read commands (#1631)
* fix(adapters): drop silent-sentinel row fallbacks across 6 read commands

Continues the audit-baseline cleanup started in #1611 (lesswrong) and
the direction set by #1599 / #1603 / #1604. Replaces the
`silent-sentinel` row-data fallbacks (`'Unknown'` / `'-'` / `'unknown'`
that mask missing fields) with the empty-string signal so agents can
tell apart "field really has the value Unknown" from "upstream returned
no value".

Touched 6 read adapters, 10 baseline entries:
- wikipedia/trending: title, description
- 36kr/article: author, date, body
- xiaoyuzhou/download: podcast
- xiaoyuzhou/transcript: podcast
- zhihu/collection: dedup key + type field (the empty prefix still
  produces a unique-per-content dedup key, just without the `unknown:`
  noise)
- zhihu/download: author

Intentionally skipped (line-by-line audited):
- v2ex/me.js: `'Unknown'` is an in-band control-flow sentinel. Line 35
  initialises `let username = 'Unknown';`, line 41 uses
  `if (username === 'Unknown')` to trigger the profileEl fallback
  selector, line 75 uses the same check to raise the auth error.
  Empty would silently bypass both checks and return a row with an
  empty username as if auth succeeded.
- v2ex/daily.js: `'未知'` is user-facing 签到 success text in the
  rendered status message, not a row field. Empty would render a
  broken sentence.
- weibo/comments.js, weibo/feed.js: the sentinel sits inside an in-IIFE
  error-message string composition (`'API error: ' + (data.msg || 'unknown')`),
  not in a returned row. Empty would silently truncate diagnostic
  output. Both stay on baseline.

Verified live: `opencli wikipedia trending --limit 3` and `opencli 36kr
hot --limit 2` both return populated rows; the empty-string signal only
kicks in when the upstream value is actually missing.

* test(adapters): add empty-signal coverage for the cluster-2 sentinel swap

Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with focused unit tests that
mock the upstream to return null / missing fields and assert the row
surfaces an empty-string signal instead of the old fabricated
'Unknown' / '-' / 'unknown' sentinel.

Coverage:

- clis/wikipedia/trending.test.js (new): mocks wikiFetch to return
  three articles - one with both title + description populated, one
  with no title and no description, one with title only. Asserts the
  missing fields render as '' (was '-' before this PR).

- clis/36kr/article.test.js (new): mocks page.evaluate to return a
  scrape where title is present but author / date / body are empty.
  Asserts those three fields render as '' in the row pair output
  (was '-' before this PR). Also covers the NOT_FOUND and
  INVALID_ARGUMENT error paths that already existed.

- clis/zhihu/collection.test.js (+1 case): mocks the zhihu collection
  API to return an item with content.id but no content.type. Asserts
  type renders as '' (was 'unknown' before this PR); the new dedup
  key prefix is :id rather than unknown:id, semantically identical
  for dedup purposes.

The other three files in this PR (xiaoyuzhou/download,
xiaoyuzhou/transcript, zhihu/download) use the same `|| 'unknown'` ->
`|| ''` value swap with no downstream sentinel consumer. They are
covered by the same JS language semantics the three tests above
demonstrate.

* fix(adapters): fail typed on missing row identity

* fix(adapters): tighten sentinel row identity guards

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:35:14 +08:00
Benjamin Liu 368581ea4d fix(electron-apps): move codex CDP port off 9222 to avoid browser-bridge collision (#1630)
* fix(electron-apps): move codex CDP port off 9222 to avoid browser-bridge collision

`src/electron-apps.ts` had `codex: { port: 9222 }`, but `9222` is the
default Chrome DevTools port that opencli's own browser-bridge Chrome
binds whenever `opencli doctor` is OK. On every normal opencli install
the bridge owns 9222 first, so Codex Desktop can never bind it, and
`opencli codex status` (plus every other codex command) fails with:

  App launched but CDP not available on port 9222 after 15s

`~/.opencli/apps.yaml` is documented as "additive only, does not
override builtins", so users have no supported way to relocate the
port from the user side.

Reported in #1626 with full repro (Codex Desktop + active opencli
browser-bridge Chrome) and root-cause pointer at
`dist/src/electron-apps.js:13`. Every other electron app in the
builtin registry already uses a distinct port in the 9224-9236
band (cursor 9226, doubao-app 9225, chatwise 9228, discord-app 9232,
antigravity 9234, chatgpt-app 9236); codex was the only one that
collided with the browser bridge.

Move codex to 9238 (the next free slot in that band, also the value
the reporter recommended). Update the test that asserts the port and
the two docs references that mention codex=9222. The pitfall entry
in `docs/advanced/electron.md` is also annotated to explicitly call
out 9222 as the bridge's port to avoid future collisions.

Closes #1626.

Verified live: `opencli codex status -v` now emits
`[verbose] [launcher] Probing CDP on port 9238...` (was 9222 before
the fix), confirming the code path picks up the new port. Full
end-to-end with a real Codex Desktop install is left to the reporter
and reviewer; the change here is a single-value config update plus
docs/tests sync.

Unit tests: 7 / 7 in `src/electron-apps.test.ts` pass (the codex-port
assertion updated to 9238). Both audit gates pass.

* docs(electron): sync codex CDP port guidance

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:29:14 +08:00
jakevin 0c488bbf51 docs(readme): simplify Highlights from 9 to 5 bullets (#1605)
Per WAWQAQ feedback: the previous Highlights list was bloated with hollow
marketing phrases and overlapping bullets (e.g. "Browser Automation for AI
Agents" + "AI Agent ready" said the same thing twice, "Pipeable, scriptable,
CI-friendly" is generic CLI filler).

Cut "AI Agent ready", "Account-safe" (folded into Live Browser Automation),
"Deterministic"'s second sentence (folded into Zero LLM cost), and merged
"Website → CLI" with "CLI Hub" into "100+ adapters + CLI Hub". Result is 5
concrete capability bullets instead of 9, each tied to a real feature.

EN and ZH READMEs kept in sync.
2026-05-16 20:57:33 +08:00
Jun 86792d2954 fix(barchart): surface greeks fetch failures (#1599)
* fix(barchart): surface greeks fetch failures

* fix(barchart): harden greeks failure contract

* fix(barchart): reject malformed greeks row identity

---------

Co-authored-by: 你的用户名 <你的邮箱>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 17:10:09 +08:00
jakevin ee54eb8e62 fix(audit): ignore sentinels in thrown errors
Avoid classifying fallback text inside thrown error messages as silent row data.
2026-05-16 16:51:06 +08:00
asimov 663b3387ee feat(bilibili): add summary command for the official AI video summary (#1590)
* feat(bilibili): add summary command for the official AI video summary

Adds `opencli bilibili summary <bvid>` — fetches Bilibili's official
AI-generated video summary (the "AI总结" shown on the video page) via
/x/web-interface/view/conclusion/get.

Returns the overall summary followed by the timestamped section outline,
so you get a structured digest of a video without watching it.

- Resolves cid + up_mid from the view endpoint (both required by the
  conclusion API), then calls the WBI-signed conclusion endpoint.
- Throws a clear EmptyResultError when a video has no AI summary —
  Bilibili only generates them for some videos.

Covered by clis/bilibili/summary.test.js (5 cases): summary + outline,
summary without outline, no-summary, view-resolution failure, API error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(bilibili): harden summary command contract

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 16:50:48 +08:00
jakevin 716461581a fix(adapters): surface silent empty fallbacks
Resolve the remaining silent-empty-fallback typed-error baseline entries across Douyin, Jike, and WeRead adapters.
2026-05-16 16:43:13 +08:00
hanzi 854cf01aad feat(linkedin): add messaging commands (#1597)
* feat(linkedin): add messaging commands

Add fail-closed LinkedIn inbox, connect, safe-send, and thread-snapshot commands with adapter tests and docs.

* fix(linkedin): align commands with current UI

Update inbox to read LinkedIn's normalized messaging API response and connect to use the current custom-invite route.

* chore(linkedin): sync cli-manifest.json with rebuilt inbox command

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): pass silent-column-drop gate

Drop the intermediate timestamp_ms field from inbox rows (it is converted to the timestamp column) and baseline the connect command internal profile-probe object.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): validate inbox --limit with a typed error

Reject an out-of-range --limit with ArgumentError instead of silently clamping it, satisfying the typed-error lint gate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): harden messaging command contracts

* fix(linkedin): reject inbox conversations without thread id

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 14:21:45 +08:00
胡大头 baf1522420 feat: add Youdao Notes shared note reader adapter (#1547)
* feat: add Youdao Notes shared note reader adapter

Add a new adapter for reading publicly shared Youdao Notes (有道云笔记).

- youdao note <url>: Fetches a public shared note by its share URL
  using browser-based DOM extraction. Extracts title, content, and
  keyword tags from the React-rendered page.
- Supports note.youdao.com and note.youdao.cn share URLs.
- Includes test coverage (3 tests) and documentation.

Closes #1418

* fix: extract full note content from React Redux store

Previously the adapter only extracted the AI summary section from the
DOM. Now it accesses the React fiber tree to read the full note content
from the Redux store (store.content.data.content), which contains the
complete note body in Youdao's structured format.

The extractor recursively walks Youdao's proprietary node format (key '8'
for text content) to reconstruct the full note as plain text.

* fix(youdao): harden shared note reader contract

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 14:16:25 +08:00
jakevin e3995df25c docs(readme): tighten tagline + add form-filling example (#1596)
- Replace 2-line tagline (websites/browser/electron/local + reuse logged-in browser) with a single line emphasizing the two core capabilities side by side: 把任意网站变成 CLI & 让 AI Agent 操控登录态浏览器
- Add "Help me fill out this form" as the leading opencli-browser skill example so the table surfaces browser-side capabilities, not just scraping
2026-05-16 13:15:37 +08:00
jakevin 4682ffc3de feat(douyin): restore publish and delete flow (#1587)
* feat(douyin): restore publish and delete flow

- Use upload auth v5 API instead of legacy STS2 for VOD credentials
- Switch TOS upload from AWS4-signature to gateway multipart protocol (init/transfer/finish)
- Add ApplyUploadInner → CommitUploadInner pipeline for VOD upload
- Bypass enable/transend endpoints that hang for gateway-uploaded videos
- Handle fast_detect/pre_check empty responses gracefully with retry+backoff
- Add creator backend delete fallback (via work_list id matching) when legacy delete returns permission error
- Use CommitUploadInner Vid for create_v2, not completed TOS object key
- Accept item_id as fallback when create_v2 returns no aweme_id

* fix(douyin): harden publish delete write contracts

---------

Co-authored-by: Lukin <mylukin@gmail.com>
2026-05-15 18:21:14 +08:00
胡大头 e3140af5ee feat: add Flomo memos reader adapter (#1549)
* feat: add Flomo memos reader adapter

Read your Flomo memos via the signed API.

- flomo memos: Lists recent memos with content, tags, timestamps
  Uses the Flomo v1 API with MD5 signing (secret embedded).
  Requires FLOMO_ACCESS_TOKEN env variable.
  Supports pagination via --slug cursor and --limit.

* fix: add --token arg for Flomo auth

* fix: use COOKIE strategy with browser-based API call

Use Strategy.COOKIE + browser:true instead of PUBLIC + manual token.
The adapter now reads flomo_token from localStorage in the browser,
and makes the signed API call from within the page context via fetch().
Signature is computed in Node.js and injected into the browser eval.
No env var or --token flag needed.

* fix: use access_token from localStorage.me for API auth

Flomo API requires Bearer token from access_token field in
localStorage.me (not api_token). Adapter now reads access_token
from the browser's localStorage and calls the signed API from
Node.js with the Bearer header.

* feat: add --since filter, refine flomo adapter API

- Add --since <unix_ts> to filter memos by updated_at
- Add --limit 200 to fetch all memos in one call
- Mark --slug as experimental (cursor pagination unreliable)
- 5 tests passing

* feat: add images column to flomo memos output

* docs: add flomo adapter documentation

* fix: use clampInt and rebuild manifest

* fix(flomo): harden memos reader contract

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 18:08:56 +08:00
ele-yufo 43f5c6e1cf fix(chatgpt): unwrap page.evaluate envelope across browser commands (#1580)
* fix(chatgpt): unwrap page.evaluate envelope across browser commands

The browser bridge wraps every `page.evaluate(...)` return value in a
`{ session, data }` envelope. Adapters that read `.length` or
`Array.isArray(payload)` directly on the envelope silently see "no
data" — same failure mode addressed for `xiaohongshu`/`rednote` in
#1561 and `weibo` in #1568.

This sweep applies the same `unwrapEvaluateResult` helper across every
chatgpt `page.evaluate` consumer site, plus typed shape guards
(`requireArrayEvaluateResult`, `requireObjectEvaluateResult`) on the
critical extraction paths so envelope misses fail loud instead of
silently returning empty.

## Sites wrapped

`clis/chatgpt/utils.js`:

- `currentChatGPTUrl` — string URL
- `getPageState` — login/composer probe object
- `sendChatGPTMessage` — composer write + send-button readiness
- `getVisibleMessages` — conversation transcript array
- `getConversationList` / `extractConversationLinks` — sidebar items
- `waitForChatGPTUploadPreview` — image upload readiness probe
- `uploadChatGPTImages` fallback — DataTransfer upload result
- `isGenerating` — boolean "still generating?" probe
- `getChatGPTVisibleImageUrls` — visible image URL array
- `waitForChatGPTImages` — inline `window.location.href` poll
- `getChatGPTImageAssets` — exported asset array

`clis/chatgpt/image.js`:

- `currentChatGPTLink` — used for error hints + conv link reporting

## Drive-by

`getChatGPTImageAssets` was also passing a redundant `urls` second arg
to `page.evaluate(string, urls)`. The IIFE inside the string already
receives the URL list via the `${urlsJson}` template substitution, and
the browser bridge guard in `browser/utils.ts` rejects the second form
for string scripts with:

    page.evaluate string input does not accept args;
    use page.evaluate(fn, ...args) instead

So `opencli chatgpt image <prompt>` blows up at the download step
without `--sd true`. Drop the trailing arg as part of the asset-export
cleanup. (This supersedes #1556 — same one-line fix is included here.)

## Validation

- `npx tsc --noEmit` — clean
- `npx vitest run --project adapter clis/chatgpt/` — 38/38 pass
  (25 existing + 13 new in `envelope.test.js`)
- `npm test` — 3644 passing across 364 files
- Live (browser bridge, daemon v1.7.19):
  `opencli chatgpt image "<prompt>"` → end-to-end generate + download
  succeeds; the envelope wrap is defensive in 1.7.19 (no envelope
  observed yet), but pre-empts the same silent-failure mode that hit
  the merged xiaohongshu/weibo PRs.

* fix(chatgpt): fail fast on malformed evaluate payloads

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:54:44 +08:00
Yabin Zheng 68ef95659f Fix YouTube transcript caption fetching (#1499)
* fix(youtube): unwrap transcript caption results

* fix(youtube): validate transcript caption info shape

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:49:02 +08:00
chonglinghuc c922a39a7d 微博新增用户搜索导出博文命令opencli weibo search_by_user 1670458304 --start 2025-06-01 --end 2025-06-02 (#1379)
* docs: add weibo search_by_user command design spec

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(weibo): add search_by_user helper function tests

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(weibo): add search_by_user command for timed post download to Markdown

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(weibo): remove dead hasori ternary and hardcoded hastext/haspic filters

The hasori ternary always evaluated to 1 (bug), and hastext=1 + haspic=1
silently excluded text-only and link-only posts from results.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(weibo): add integration tests for search_by_user helpers

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* bak

* fix(weibo): reshape user posts into read adapter

---------

Co-authored-by: andrew.asa <asa.andrew@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:44:47 +08:00
jakevin aae6e823b4 chore(release): 1.7.22 (#1586)
Release / release (push) Has been cancelled
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
External CLI ergonomics + two adapter envelope/auth fixes.

- feat(external): longbridge CLI passthrough (#1584)
- feat(external-cli): brand alias rendering for ntn/dws/wecom-cli (#1585)
- fix(boss): map code=24 → AuthRequiredError (#1573)
- fix(weibo): unwrap page.evaluate envelope in read adapters (#1568)
2026-05-15 17:30:34 +08:00
jakevin 3f62cc45bf feat(external-cli): render brand alias for ambiguous executable names (#1585)
`ntn`, `dws`, and `wecom-cli` are opaque executable names — users seeing them
in `opencli list` or root help have no way to know they correspond to Notion,
DingTalk Workspace, and 企业微信. Repurpose the existing `package` field to
double as a human-readable brand label, so help output renders as
`ntn(notion)`, `dws(DingTalk Workspace)`, `wecom-cli(企业微信)`.

- `src/external-clis.yaml`: add `package:` to ntn / dws / wecom-cli
- `src/external.ts`: update JSDoc on `package` to cover both upstream
  distribution names (tg-cli, discord-cli) and brand labels (notion, 企业微信)
- `src/cli.ts:629` (`opencli list`): use `formatExternalCliLabel` so the
  listing matches root help, which already used it
- `src/external.test.ts`: regression test for brand-alias labels

Verification:
- npx vitest run --project unit src/external.test.ts: 9/9 pass
- npm run typecheck: clean
- npm run build: 813 manifest entries
- Smoke: `opencli list` and `opencli --help` both render the new labels
2026-05-15 16:37:27 +08:00
jakevin b6f352b318 feat(external): add longbridge cli (#1584) 2026-05-15 16:27:08 +08:00
Benjamin Liu dadf01b56f fix(weibo): unwrap page.evaluate envelope in read adapters (#1568)
* fix(weibo): unwrap page.evaluate envelope in read adapters (#1567)

`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, so all weibo cookie-strategy read adapters
silently dropped their results on v1.7.19:

- `getSelfUid` returned the envelope object instead of the uid string,
  so `'10001' + uid` produced `'10001[object Object]'` and every
  feed/me/favorites request hit a broken list_id.
- `feed`, `hot`, `comments`, `search`, `favorites` did `Array.isArray`
  on the envelope (always false) and returned `[]`.
- `me`, `user`, `post` returned the envelope wrapper itself instead of
  the inner profile/post object.

Same pattern as #1561 for xiaohongshu/rednote. Adds an
`unwrapEvaluateResult` helper to `clis/weibo/utils.js` (kept local
rather than cross-importing from `xiaohongshu/search.js` since weibo
is an unrelated site) and wraps every `await page.evaluate(...)` in
the 8 read adapters plus the two helper calls in `getSelfUid`.

Skipped `publish.js` (write command, out of scope for this read fix).

Verified live:
- `opencli weibo hot --limit 3` returns 3 real trending items
- `opencli weibo feed --limit 3` returns 3 timeline posts with
  correct `https://weibo.com/<uid>/<mblogid>` URLs (proves
  `getSelfUid` unwrap works)
- `opencli weibo me` returns the logged-in profile object
- All 20 weibo unit tests pass (6 new for `unwrapEvaluateResult`)

* fix(weibo): fail typed on malformed evaluate payloads

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 22:34:43 +08:00
Benjamin Liu 1239798d04 fix(boss): map code=24 (identity mismatch) to AuthRequiredError (#1573)
Recruiter-only BOSS commands (recommend, joblist, stats, resume, mark,
exchange, invite, greet, batchgreet) returned a generic
`COMMAND_EXEC: 请切换身份后再试 (code=24)` when called from a job-seeker
account. The original error hid the actionable bit: this command set
needs a recruiter (BOSS-side) account.

chatlist / chatmsg already special-case code=24 by falling back to the
geek-side fetch when --side=auto. Recruiter-only commands have no
geek-side equivalent and were just leaking the raw API code.

Fix: add a `checkRecruiterSide` step inside `assertOk` that maps
code=24 to AuthRequiredError with a clear message. All 9 recruiter-only
commands inherit it through their existing `bossFetch` calls; no
adapter-level changes needed. chatlist / chatmsg are unaffected because
they use `allowNonZero: true` and never hit the auto-error path.

Closes #1572.
2026-05-14 22:26:28 +08:00
jakevin 9ccc896585 chore(release): 1.7.21 (#1571)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 19:36:41 +08:00
jakevin 1a69f40a80 fix(social): use ephemeral adapter site sessions (#1569) 2026-05-14 19:22:42 +08:00
J.Chen 300607f692 fix(facebook/feed): add fallback extraction for empty article nodes (#1538)
* fix(facebook/feed): add fallback extraction for empty article nodes

Add fallback extraction for Facebook feed posts when [role=article] nodes exist but contain empty text. Includes diagnostic errors, content/author cleanup, nested-container dedupe, and an evaluate-script syntax regression test.

* fix(facebook): bound feed fallback extraction

* fix(facebook): keep feed fallback available after chrome articles

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 18:42:50 +08:00
J.Chen 42b5a4e68d feat(boss): support job-seeker chatlist and chatmsg (#1539)
* feat(boss): support job-seeker chatlist and chatmsg

* fix(boss): type chat-side failure boundaries

* fix(boss): guard malformed chat API payloads

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 18:41:02 +08:00
jakevin bccd275d66 test(extension): cover adapter group tiebreaker (#1566) 2026-05-14 18:04:07 +08:00
胡大头 edfa5f0da3 feat: add DuckDuckGo, Brave, and Yahoo web search adapters (#1546)
* feat: add DuckDuckGo, Brave, and Yahoo web search adapters

Add three new search engine adapters with browser-based DOM extraction:

- duckduckgo/search: Search DuckDuckGo via html.duckduckgo.com
  Supports region, time filters, and XHR-based pagination (--offset)
- duckduckgo/suggest: Search suggestion autocomplete (no browser needed)
- brave/search: Search Brave Search via search.brave.com
  Supports GET-based pagination (--offset)
- yahoo/search: Search Yahoo (Bing-powered) via search.yahoo.com
  Supports GET-based pagination (--page)

All search adapters use Strategy.PUBLIC with browser:true, navigating
the target site and extracting results via page.evaluate() DOM queries.
Includes full test coverage (16 tests).

* fix: use clampInt from shared utils and add adapter docs

- Replace Math.max/Math.min patterns with clampInt() from _shared/common.js
  to pass the typed-error-lint gate (4 silent-clamp violations resolved)
- Add adapter documentation for duckduckgo, brave, and yahoo to fix
  the doc-coverage CI check
- Regenerate cli-manifest.json and typed-error-lint-baseline.json

* fix: avoid silent-column-drop overlap in brave/yahoo extractors

Change buildExtractorJs to return arrays instead of objects whose keys
matched columns. This prevents silent-column-drop audit false positives
as per opencli-adapter-author conventions.

* fix(search): tighten browser search adapters

* chore(search): drop baseline churn

* fix(duckduckgo): execute search extractor safely

* fix(yahoo): reject unsafe redirect targets

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:54:34 +08:00
J.Chen 16b02bcc58 fix(extension): reuse existing adapter tab group (#1541)
* fix(extension): reuse existing adapter tab group

* fix(extension): choose best existing adapter group

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:15:39 +08:00
Iris Chen 5af2ff1d6c fix(xiaohongshu,rednote): unwrap page.evaluate envelope in search adapter (#1561)
* fix(xiaohongshu,rednote): unwrap page.evaluate envelope in search adapter

`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, but the search adapters were calling
`Array.isArray(payload)` directly on the envelope. `Array.isArray` is
always false on the envelope, so every search result was silently
dropped — status=success, exit 0, empty array, no error.

The rednote adapter had this same bug; both share `buildSearchExtractJs`
from `xiaohongshu/search.js`.

Introduces `unwrapEvaluateResult(payload)` as a shared helper in
`clis/xiaohongshu/search.js` (re-exported via the existing import line
from `rednote/search.js`). The helper is a defensive ternary: it
unwraps when payload looks like an envelope with an array `.data`,
otherwise it passes the value through unchanged. This keeps the change
back-compat with bridge versions that return the raw value, and
preserves the existing `Array.isArray(payload)` typecheck at each call
site.

Verified manually against `opencli xiaohongshu search "补墙洞"` (a query
known to return 20+ results in a logged-in browser tab): previously
`[]`, now returns the expected ranked rows with all declared columns
(`rank, title, author, likes, published_at, url`) populated.

Adds 5 unit tests for `unwrapEvaluateResult` covering raw array passthrough,
envelope unwrap, non-envelope object passthrough, null/undefined safety,
and the "data is not an array" guard. The existing 19 search tests in
`clis/xiaohongshu/search.test.js` still pass — the unwrap is invisible
to the existing mocks which already return raw arrays.

* fix(xhs): unwrap search evaluate envelopes

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:05:58 +08:00
jakevin 9c25bc7009 fix(ci): add Windows native binding lock entries (#1563) 2026-05-14 16:45:00 +08:00
jakevin 8c88a3cbf3 chore(release): 1.7.20 (#1562)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 16:33:19 +08:00
jakevin 9c4f4a3d30 fix(cli): show external CLI package aliases (#1560) 2026-05-14 16:28:18 +08:00
jakevin 29c135b656 refactor(notion): replace built-in CDP adapter with external ntn CLI (#1559)
* refactor(notion): replace built-in CDP adapter with external ntn CLI

Notion has shipped an official CLI at https://ntn.dev. It uses the
public Notion API (blocks / databases / properties / comments) instead
of reverse-engineering the Desktop UI, so it survives Notion app
updates and exposes a wider command surface than the in-tree adapter
could.

Changes:
  - `src/external-clis.yaml` — register `ntn` as first-class external CLI
    (binary `ntn`, homepage ntn.dev, install via the shell-pipe script
    on mac/linux)
  - `clis/notion/` — entire directory removed (8 commands: status /
    search / read / new / write / sidebar / favorites / export)
  - `docs/adapters/desktop/notion.md` — removed
  - `docs/.vitepress/config.mts` — drop nav entry
  - `docs/adapters/index.md` — drop adapter row
  - `README.md` / `README.zh-CN.md` — drop notion from feature lines,
    drop adapter table row, add `ntn` to CLI hub examples
  - `docs/index.md` / `docs/zh/index.md` / `docs/guide/getting-started.md`
    — drop notion from electron-control feature copy
  - `skills/opencli-usage/SKILL.md` — drop notion from electron list
  - `cli-manifest.json` — rebuilt with --allow-removals=8

Migration for users:
  `curl -fsSL https://ntn.dev | bash`  (or `opencli external install ntn`)
  Then use `opencli ntn <command>` in place of `opencli notion <command>`.

Rationale: the in-tree adapter was reverse-engineered against Notion
Desktop CDP and shipped only 8 commands. The official CLI gives users
the full Notion API surface and reduces our maintenance burden to zero.
Same pattern as gh / obsidian / lark-cli / tg-cli / discord-cli / wx-cli.

Verification:
  - `npx tsc --noEmit` clean
  - `npx vitest run --project unit` → 1091/1 skipped
  - `npm run build` (with --allow-removals=8) — manifest 809 entries
  - grep notion in user-facing docs (README / docs / skills) — only
    descriptive mentions remain in non-blocking places (comparison /
    site-recon / electron how-to / design doc), no broken adapter
    references

* fix(notion): align ntn external migration

* docs(notion): clarify ntn manual install
2026-05-14 16:12:17 +08:00
jakevin 7edf53783f fix(daemon): report unknown browser command results (#1558) 2026-05-14 14:30:13 +08:00
jakevin af7b94152f feat(twitter): add extractMedia parity to bookmarks + bookmark-folder (#1555)
Mirrors PR #1464 (list-tweets) and the timeline/search/tweets/likes/thread
family: spread `...extractMedia(legacy)` into the row and surface
`has_media` + `media_urls` columns. Pure parity, no behavior change for
existing callers — media keys do not collide with the original columns.

- bookmarks.js: import `extractMedia` from ./shared.js, spread into
  extractBookmarkTweet row, append columns, export __test__.
- bookmark-folder.js: same change on extractFolderTweet, export
  extractFolderTweet via __test__.
- bookmarks.test.js (new): baseline + photo + video + entities-only
  fallback + dedup + envelope + empty-envelope (8 tests).
- bookmark-folder.test.js: update existing baseline expectation with
  has_media/media_urls, add 3 new media tests (photo / mp4 / no-media).
- cli-manifest.json: regenerated; only the two `columns` entries change.

Reverse-validated: tests fail when extractMedia spread is removed.

Audits unchanged: typed-error-lint 189/189, silent-column-drop 102/103
(pre-existing main resolution noted but not consumed here).
2026-05-14 14:24:31 +08:00
Ocean 6b26aedd56 feat(twitter/list-tweets): include media via extractMedia (parity with timeline/search) (#1464)
* feat(twitter/list-tweets): include media via extractMedia (parity with timeline/search)

list-tweets was the only X recall path that dropped media. timeline.js and
search.js both call extractMedia(legacy) and emit has_media/media_urls;
list-tweets returned only text fields, so downstream consumers (e.g.
ml-scout's rate UI) couldn't render image/video thumbnails on tweets pulled
from a list timeline.

Changes:
- Import extractMedia from ./shared.js
- Spread extractMedia(legacy) into extractTimelineTweet return
- Add has_media, media_urls to columns array (--format columns parity)
- Update unit test to assert the new shape; add coverage for photo and
  video extraction

* chore(manifest): rebuild cli-manifest.json for list-tweets media columns

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
2026-05-14 14:01:34 +08:00
jakevin 68b18cdbcd fix(extension): coalesce daemon websocket connects (#1554) 2026-05-14 13:57:38 +08:00
J.Chen cddc84776c docs(browser): clarify named session lifecycle (#1542)
* docs(browser): clarify named session lifecycle

* docs(browser): clarify owned versus bound sessions

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 13:38:41 +08:00
J.Chen 4f5fcd9acb fix(extension): keep active daemon websocket
Keep stale Browser Bridge WebSocket events from clobbering the active daemon connection.\n\nCo-authored-by: Jeff Chen <jeff@adtiming.com>
2026-05-14 13:38:05 +08:00
jakevin 40b2f75098 feat(external)!: drop -cli suffix from tg/discord/wx subcommand names (#1544)
The opencli external-CLI name is the user-typed subcommand; the binary is
what gets executed. The convention everywhere else (`gh`, `docker`,
`obsidian`, `vercel`, `dws`) is `name == binary`. Three entries violated
the convention: `tg-cli` / `discord-cli` / `wx-cli` registered an
opencli name with a `-cli` suffix that does NOT exist on the binary,
forcing the awkward double-prefix `opencli discord-cli dc` instead of
`opencli discord dc`.

The README's example column already showed the desired form
(`opencli tg search`, `opencli discord recent`, `opencli wx search`) —
only the yaml registration was out of sync.

Renames in `src/external-clis.yaml`:

* `name: tg-cli`     → `name: tg`       (binary: `tg`)
* `name: discord-cli`→ `name: discord`  (binary: `discord`)
* `name: wx-cli`     → `name: wx`       (binary: `wx`)

The `binary`, `homepage`, and `install` fields are unchanged — the
underlying packages (`kabi-tg-cli`, `kabi-discord-cli`, `@jackwener/wx-cli`)
keep their published names.

Other entries left as-is: `lark-cli`, `wecom-cli`, and `dws` already have
`name == binary` (their actual binaries are `lark-cli`, `wecom-cli`, `dws`).

BREAKING CHANGE: `opencli tg-cli ...`, `opencli discord-cli ...`,
`opencli wx-cli ...` no longer resolve. Use `opencli tg ...`,
`opencli discord ...`, `opencli wx ...` instead. The feature is recent
(shipped 2026-05) so impact is expected to be minimal.
2026-05-14 04:13:39 +08:00
jakevin feab24f76c chore(release): 1.7.19 (#1543)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 02:33:56 +08:00
jakevin 8ef7e903b8 feat(twitter): default tweets to logged-in user + fix sibling envelope-unwrap silent bug (#1531)
* feat(twitter): default tweets to logged-in user + fix sibling envelope-unwrap silent bug

Primary: make `opencli twitter tweets` default to the logged-in user
when no username is given, so agents can pull their own posts without
needing to know their own handle. Mirrors the existing self-detection
pattern in twitter/profile and twitter/likes (AppTabBar_Profile_Link
probe on /home, then UserByScreenName lookup). Description + help
string now mention the default so agents discover it.

Consistency pass — profile/likes/following/followers: the
self-detection in these four siblings was silently broken because
page.evaluate() primitive returns come back through the CDP bridge
wrapped as `{session: 'site:twitter', data: '/<handle>'}` (same
envelope root cause as #1525). They called `.replace()` directly on
the envelope object → TypeError surfaced as AUTH_REQUIRED 'Could not
detect logged-in user', even for logged-in users. Wrap each probe
with unwrapBrowserResult so the bare href string survives. Also:
- Add an explicit page.goto('/home') + page.wait(primaryColumn)
  before the probe in likes/following so the AppTabBar sidebar is
  guaranteed rendered (framework pre-nav lands on bare x.com without
  the sidebar mounted).
- following.js: switch its probe from the function-literal form
  `() => {...}` to a template-string. Confirmed live: function-literal
  silently drops primitive returns entirely — bridge returns
  `{session}` with no `data` field at all, while template-string
  returns `{session, data}` as expected.

Out of scope (pre-existing, flagged as follow-up): likes/following
have additional downstream evaluate paths (userId/GraphQL fetch) that
still drop or envelope their results; they return [] or
'Could not find user' even after this PR. Same daemon-side bug class
as #1525.

Live-verified:
  opencli twitter tweets --limit 2     → own tweets (@jakevin7)
  opencli twitter profile              → own profile

Tests 227/227, audits typed-error-lint 189 + silent-column-drop 103
unchanged, manifest stable at 816 entries.

* fix(twitter): validate self-detected handles

* fix(twitter): unwrap downstream self evaluate results
2026-05-13 22:51:48 +08:00
ppop123 7c5bafd49b fix(twitter): repair list-add / list-tweets / lists / following after 2026-05 changes (#1503)
* fix(twitter): unwrap page.evaluate primitive returns in lists/list-tweets/following

The opencli >=1.7.x browser bridge wraps page.evaluate's primitive return
values as { session, data: <value> }. Adapters that destructure .data
inline (e.g. data.queryId, data.viewer) keep working because the wrapper
spreads object-typed responses to the top level, but ones that consume
the return value as a bare string broke:

- twitter list-tweets: the dynamically resolved queryId (a string) became
  {session, data:"..."}. Interpolating that into the GraphQL URL produced
  /i/api/graphql/[object Object]/ListLatestTweetsTimeline, giving "HTTP
  400: queryId may have expired".
- twitter lists: same on ListsManagementPageTimeline queryId.
- twitter following: same shape bug on the href read from the profile
  link, producing "TypeError: href.replace is not a function" when no
  --user is given.

Add a small unwrap() helper at each call site so primitive returns are
extracted from the wrapper before use. Object-typed GraphQL responses
are left as-is since they rely on spread semantics.

* fix(twitter): rewrite list-add to use ListAddMember GraphQL mutation

In 2026-05 X replaced the "Add/remove from Lists" modal dialog with a
full-page route (/i/lists/add_member). The previous UI flow no longer
works:

  Save button not found in dialog (X expected text Save/Done).
  Dialog structure may have changed.

The mutation that the dialog used to fire (ListAddMember) is still the
right primitive — and the surrounding adapter already calls X GraphQL
APIs directly to resolve userId and verify member_count. Drop the UI
flow entirely and call ListAddMember directly via fetch in the page
context.

Wins:
- Works again on current X UI (verified 2026-05-12 on x.com).
- ~10x faster: no goto-profile + click-caret + scroll-dialog round trips.
- One less moving piece — no dependency on Chrome extension's nativeClick
  for this command.

Implementation notes:
- LIST_ADD_MEMBER_QUERY_ID is a 2026-05 fallback; resolveTwitterQueryId
  does live lookup from the loaded client-web bundle, matching the
  pattern already used elsewhere in the twitter clis.
- X's ListAddMember response routinely contains a non-fatal partial
  decode error on default_banner_media_results (code 214, Validation /
  BadRequestError) alongside a fully populated data.list. We treat the
  call as failed only when data.list / member_count is missing, and
  ignore decode-flavored errors confined to banner fields.
- Same opencli >=1.7.x { session, data } primitive-wrap behavior that
  the previous commit addressed applies here: userId from the
  UserByScreenName call needs unwrap before being interpolated into
  the mutation body, otherwise X parses "[object Object]" as user_id
  and returns "strconv.ParseInt ... invalid syntax".

Verified flows:
- noop (already a member) → status: noop, member_count unchanged.
- new add (e.g. @AnthropicAI on a fresh list) → status: success,
  member_count incremented.

Trade-off: rejection signals (e.g. X declining to add @deepseek_ai)
look indistinguishable from noop at the response level, since X returns
HTTP 200 with member_count unchanged. Documented in the success message.

* fix(twitter): integrate list media and harden list-add

---------

Co-authored-by: wangyan <wy@wang-yan-Air.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 22:35:57 +08:00
jakevin f66996a148 fix(twitter): raise cursor pagination guard
Fix hidden pagination ceilings across Twitter cursor-pagination adapters.\n\nCo-authored-by: Mingming Lou <1109198+lmmsoft@users.noreply.github.com>
2026-05-13 22:14:05 +08:00
jakevin 4fac911425 feat(zhihu): add answer-detail to fetch a single answer's full content (#1528)
* feat(zhihu): add answer-detail to fetch a single answer's full content

The existing `zhihu answer` adapter is a write (post an answer); the
listing `zhihu question` truncates each answer's body to 200 chars.
There was no way to fetch one specific answer's full content by id.

New read adapter `zhihu answer-detail`:

- Accepts a bare numeric answer id, a typed target `answer:<qid>:<aid>`,
  or a full Zhihu answer URL (the form you paste from a browser).
- Calls `/api/v4/answers/<aid>?include=content,voteup_count,...,question`
  inside the cookie-bearing page context (Strategy.COOKIE).
- Returns a single row with id / author / votes / comments /
  question_id / question_title / url / created_at / updated_at /
  content. The content column is the full stripped answer body by
  default — no silent truncation. `--max-content N` is an opt-in user
  cap (mirroring the wikipedia `page` flag), and `--max-content 0`
  (the default) means "no cap, full content".

Important precision note: Zhihu answer ids since 2024 routinely
exceed `Number.MAX_SAFE_INTEGER` (the test fixture uses the real id
`1937205528846655537`). `data.id` is round-tripped through browser
`JSON.parse` and would round to `1937205528846655500`, so the adapter
deliberately ignores `data.id` for the canonical row id and anchors
it to the already-validated input string instead. A regression test
locks this contract in by mocking `data.id = 0` and asserting the row
still carries the parsed input id.

Typed errors: bad input → INVALID_INPUT; 401/403 → AuthRequiredError;
other HTTP / null → FETCH_ERROR. No silent fallbacks, no sentinel
strings.

Live-verified against the example URL — fetched 5547 votes / 165
comments / 1937205528846655537-end-to-end. 16 unit tests, audits
unchanged (typed-error-lint 189/189, silent-column-drop 103/103),
manifest 816→817.

* fix(zhihu): tighten answer-detail contracts
2026-05-13 21:47:37 +08:00
xcd_git b52da639a3 fix(google-scholar/search): wrap evaluate return to fix serialization (#1525)
* fix(google-scholar/search): wrap evaluate return to fix serialization

Same issue as google/search: page.evaluate() serializes JS arrays as
plain objects across the CDP boundary, causing Array.isArray() to
return false. The adapter silently returned [] instead of results.

Also replace fixed page.wait(3) with selector-based wait for
.gs_r.gs_or.gs_scl with a 3s fallback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(google-scholar): type search evaluate payload

* chore: rerun google scholar search checks

---------

Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 19:07:39 +08:00
Joseph赛博阿隆 b1dca04ddd fix(xiaohongshu): parseLikes should handle 2.1w / 1.5万 / 1.2k shortforms (#1504)
* fix(xiaohongshu): parseLikes should handle 2.1w / 1.5万 / 1.2k shortforms

Xiaohongshu renders top-popular comment like-counts as shortened
strings like '2.1w' / '1.1万' / '1.2k' once they exceed ~10 000.
The previous parseLikes only matched bare digits via /^\d+$/ and
silently returned 0 for any shortform, which inverted the sort
order: the highest-liked comments (often 10k+) ranked last while
mid-tier comments with plain numeric counts (e.g. 7569) appeared
on top.

Repro on any popular xiaohongshu thread (>10 000 likes on a top
comment): with --format json the most-upvoted parent rows show
"likes": 0.

This patch keeps the original fast path for plain integers and
adds a single regex for the well-known shortform suffixes:

  - w / 万 -> *10000
  - k / 千 -> *1000
  - trailing '+' tolerated (e.g. '999+')
  - unknown shapes still fall back to 0 (no behavior change)

Note: parseLikes runs inside the IIFE injected via page.evaluate(),
so the existing comments.test.js mock harness (which stubs
evaluate's return value directly) does not exercise it. A future
refactor that exports parseLikes for direct testing would be a
separate change.

Affects both top-level comments and 楼中楼 sub-replies (same
helper).

* fix(xiaohongshu): parse comment like shortforms safely

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:47:20 +08:00
jakevin f481585ba1 chore: drop util.styleText to support Node v20+ (#1524)
* chore: drop util.styleText to support Node v20+

util.styleText was added in Node v21.7.0 / v20.12.0. v21.0.0-v21.6.x and
v20.0.0-v20.11.x throw `SyntaxError: ... styleText` at startup because the
import resolves before any user code runs (a real user reported this on
v21.2.0).

OpenCLI is primarily agent-facing — terminal colors are noise to consumers,
and the [OK] / [WARN] / [FAIL] / ℹ / ⚠ / ✖ markers we already write carry
the semantic info that colors only repeated. Strip styleText entirely from
logger / output / doctor / tui / update-check / cli / download/progress /
commands/daemon and clean up the resulting awkward `${'literal'}` template
fragments. engines.node now reads ">=20.0.0".

This removes the Node-version coupling that A/B fixes would only have
papered over.

* fix(runtime): truly support Node v20+ by aligning guard + undici

Follow-up to the styleText removal: declaring engines.node >=20.0.0 is
not enough on its own. Two coupled barriers remained:

- src/runtime-detect.ts: MIN_SUPPORTED_NODE_MAJOR = 21 explicitly
  rejected v20 at startup
- undici@^8.0.2 declares engines.node >=22.19.0; Node 20/21 crash on
  webidl.util.markAsUncloneable before any user code runs

Lower the guard to 20 and downgrade undici to ^6.25.0 (engines >=18.17,
retains Agent / EnvHttpProxyAgent / fetch / Dispatcher). Smoke-tested
--help / doctor / list on Node v20.0.0, v21.2.0, v22.22.2. 213/213
targeted unit tests pass.
2026-05-13 18:33:21 +08:00
lenovobenben 723f2b9147 feat(zhihu): paginate question answers and recommendations (#1517)
* feat(zhihu): paginate question answers and recommendations

* fix(zhihu): drop Math.min limit clamp and 'unknown' sentinel

Two audit-driven fixes on top of feat/zhihu-pagination-recommend:

1. question.js: replace `Math.min(answerLimit, 20)` with a named
   constant `ZHIHU_PAGE_SIZE = 20`. The Zhihu API caps `limit` at 20
   per request anyway, and the pagination loop already trims to the
   user-requested `answerLimit` via `answers.length >= answerLimit`,
   so the Math.min silent-clamp was both unnecessary and tripped the
   silent-clamp audit. Updates the existing unit test to expect the
   API-max page size in the fetch URL with an explanatory comment.

2. recommend.js: rebuild the dedup key without the `'unknown'`
   sentinel. The old form `\`\${target.type || 'unknown'}:\${target.id}\``
   collapsed distinct typed items into the same bucket whenever
   `target.type` was missing, and tripped the silent-sentinel audit.
   New form: prefer `type:targetId`, fall back to `__feed:item.id`,
   and when neither id is available keep the row but skip dedup
   (surfacing potentially-duplicate items beats silently dropping
   them).

Audits unchanged (typed-error-lint 189/189, silent-column-drop
103/103). All 88 zhihu tests pass.

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:28:44 +08:00
Benjamin Liu 2babed84e9 fix(xiaohongshu+rednote/search): fall back to href-based note cards when section.note-item class is dropped (#1506) (#1507)
Issue #1506 reports `opencli xiaohongshu search` returning `[]` even though
the page visibly has results. Trace evidence: xhs ships a render variant
where each note card is a bare `<section>` (no `note-item` class), so
the three `section.note-item` selectors in this file all match zero
elements.

Three call sites in the shared search IIFEs now use the same defensive
selector strategy: try the legacy `section.note-item` class first, then
fall back to any `<section>` that wraps a `/search_result/...` or
`/explore/...` link. The change is in the xiaohongshu file so the
rednote adapter (which imports `buildSearchExtractJs` and
`buildScrollUntilJs` from here) picks it up automatically.

Extraction-side title selector also gets a fallback: when no
`.title` / `.note-title` element matches, read the first `<span>`
inside the search-result link, which is where the bare-section render
puts the caption per the trace.

## Verification

`npx vitest run --project adapter clis/xiaohongshu/`: 105/105 green
(existing test suite unchanged, passes on both legacy and fallback paths).

Live verify on rednote (same code path, account-safe):

```
$ opencli rednote search "美食" --limit 3 -f json
[ {rank:1, title:"在朋友家吃过一次..."}, {rank:2, title:"我的15💰晚餐..."}, {rank:3, title:"干净饮食🫛..."} ]
```

Legacy `section.note-item` path is exercised here (rednote still renders
the class) and returns identical row shape to before the fix, confirming
no regression on the working path.

Live verify on xiaohongshu cannot be performed here (no logged-in xhs
session on the test machine; xhs account-ban risk per the project's
operational guidance). The fix is structural: the new `<section>` shape
the issue reporter traced is reachable through the fallback, and the
existing test fixture keeps the legacy path green.

`npx tsc --noEmit` clean. `npm run build` 815 manifest entries unchanged
shape. `silent-column-drop` / `typed-error-lint` baselines unchanged.

Closes #1506
Refs #1500
2026-05-13 18:24:55 +08:00
陈家名 a6ca53c7cf fix: clamp download progress percentages (#1520)
* fix: clamp download progress percentages

* test(download): cover unknown progress total

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:24:26 +08:00
jakevin c3912d8e5c feat(reddit/read): --expand-more via /api/morechildren + 7-kind typed errors (#1492)
* feat(reddit/read): add --expand-more via /api/morechildren + 7-kind discriminated union

PR B of the rdt-cli parity follow-up (after PR #1491, see #1481 thread).
Closes the second-largest gap: Reddit's "[+N more replies]" stubs were
opaque markers in the comment tree. With --expand-more, the adapter
follows them by POST-ing the t1 ids to /api/morechildren.json, then
re-threads the returned things back into the tree by parent_id before
walking it.

New args:

- `--expand-more` (bool, default false) — turn on stub expansion.
- `--expand-rounds <N>` (int, default 2, range [1, 5]) — Reddit returns
  fresh "more" stubs at the expansion depth boundary, so up to N rounds
  are run. Strictly validated via `parseExpandRounds` — out-of-range
  raises ArgumentError BEFORE `page.goto`, no silent clamp.

Boy-Scout: the in-browser script now returns a 7-kind discriminated
union instead of a flat row array (matching the PR #1428 / #1491
sediment). Each kind maps 1:1 to a typed error on the Node side:

  - `inaccessible` → EmptyResultError
      401/403/404 on /comments/<id>.json (post-specific access, not
      session-level auth — applies the PR #1491 review-side sediment
      "inaccessible-resource vs session-auth").
  - `auth`         → AuthRequiredError
      401/403 on /api/morechildren (expand-write endpoints often demand
      a logged-in session even when the read endpoint is anonymous).
  - `http`         → CommandExecutionError
  - `malformed`    → CommandExecutionError
      200 with unexpected envelope shape — schema drift, not empty.
  - `parser-drift` → CommandExecutionError
      tree had t1 entries but the walker produced no rows (PR #1491
      review-side sediment "post-construction 0 rows + pre-walk
      non-empty = parser drift, not legitimate empty").
  - `expand-failed`→ CommandExecutionError
      /api/morechildren returned a non-empty json.errors array.
  - `ok`           → returns rows[].

Intermediate keys (kind / detail / httpStatus / where / rows /
expandMeta) deliberately avoid the declared columns (type / author /
score / text) per the PR #1329 silent-column-drop sediment.

Tests:
  clis/reddit/read.test.js — 11 tests
    - Adapter shape (browser / siteSession / columns / args)
    - --expand-more / --expand-rounds present with correct types/defaults
    - parseExpandRounds default / range / non-integer rejection
    - Pre-navigation validation (bad --expand-rounds doesn't reach goto)
    - kind=ok happy path (POST + L0 rows)
    - 6-kind error → typed error mapping
    - Unknown envelope shape → CommandExecutionError
    - Evaluate script embeds expandMore/expandRounds/sort/limit literals
    - Evaluate script contains /api/morechildren POST scaffolding
    - Evaluate script never names declared columns as intermediate keys

Full reddit suite 48/48; full project 3402/3402.

Audits: typed-error-lint 189/189 (0 new), silent-column-drop 103/103
(0 new). Manifest 815 → 815 (existing read entry gets 2 new args).

Existing --limit / --depth / --replies / --max-length keep their
original Math.max-style behaviour (grandfathered in the baseline);
only the new --expand-rounds flag fails fast per the typed-errors
standard.

Refs: https://github.com/jackwener/rdt-cli (browse.read --expand-more)

* fix(reddit): preserve expanded comment tree order

* fix(reddit): fail on partial morechildren expansion
2026-05-13 18:13:11 +08:00
darthjaja 67599ea67c fix(twitter): repair search and tweets readback (#1512)
* fix(twitter): repair search and tweets readback

* fix(twitter): prefer baked operation features when bundle parse returns empty

The bundle parser in resolveTwitterOperationMetadata locates the queryId via
`queryId:"..."` inside a ~2500-char snippet around the operationName marker,
then independently extracts `featureSwitches:[...]` and `fieldToggles:[...]`
via separate regexes. When minification rearranges the snippet (or the
snippet window truncates before the array), either regex can miss while
queryId still resolves; keysToFlags(undefined) then returns {}.

sanitizeTwitterOperationMetadata previously accepted any object as
features / fieldToggles, including {}. Twitter's GraphQL endpoint rejects
SearchTimeline / UserTweets requests with empty features (HTTP 400),
surfacing a misleading "queryId may have expired" error — the queryId is
fresh; only the feature flags are missing.

Guard against this by deferring to the baked fallback whenever the resolved
map is empty. Adds a JSDOM-free unit test that, reverse-validated, fails on
the un-fixed code with the exact silent-fallback shape.

Refs PR #1512

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:11:41 +08:00
darthjaja f321a6096d fix(twitter): make reply submission robust (#1511) 2026-05-13 18:10:19 +08:00
xcd_git 59ebf551f0 fix(google/search): wrap evaluate return value in object to fix serialization (#1523)
page.evaluate() serializes JS arrays as plain objects, causing
Array.isArray() to return false and the adapter to throw NOT_FOUND
even when results exist. Wrap the return value in {items: results}
and extract via wrapper.items to avoid the type check issue.

Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 18:05:03 +08:00
jakevin 04a57029b3 ci(adapter-test): gate adapter-test off pull_request trigger (#1522)
Per @WAWQAQ direction (DM): trim PR-time CI to fast-feedback only.
adapter-test (~30-60s) is the next-largest PR wait after e2e-headed
(which #1521 just removed). Adapter authors typically run focused tests
locally before pushing (`npm run test:adapter`); CI duplication adds
queue latency without catching new classes of bugs.

PR-time CI surface now:
  - typecheck / unit (~1 min)
  - lint gates (typed-error / silent-column-drop)
  - build × 3 platforms

Adapter test guards (still strict):
  - push to main / dev
  - nightly cron
  - workflow_dispatch (manual when an adapter-heavy PR really wants the
    signal before merge)

Same gate as smoke-test (`if: github.event_name == 'push' || schedule
|| workflow_dispatch`) for consistency.
2026-05-13 17:55:23 +08:00
jakevin fd438c2109 ci(e2e): drop e2e-headed from pull_request trigger (#1521)
Per-PR e2e-headed Chrome was the dominant PR-time wait (~10-15 min on
two platforms) and on fork PRs blocks behind maintainer approval, while
the actually-blocking failures it caught in the last 30 days were all
e2e-test migrations missed by the authoring PR (#1461 / #1505 workspace
->session) rather than real regressions the unit/typecheck tier missed.

PR feedback path is now:
  - typecheck / unit / lint / adapter / build  ← `pull_request` (ci.yml)
  - extension typecheck / build                ← `pull_request` (build-extension.yml)
  - docs build                                  ← `pull_request` (doc-check.yml)
  - security audit                              ← `pull_request` (security.yml)

E2E-headed Chrome guards:
  - push to main / dev (watched paths)
  - push v* tag (release)
  - nightly cron 08:00 UTC (added: catches Chrome version drift / flake
    drift even when no commits touch watched paths)
  - workflow_dispatch (manual when a PR really wants e2e signal)

smoke-test was already gated on `schedule || workflow_dispatch` only
(ci.yml), so no change needed there.
2026-05-13 17:45:41 +08:00
Xiaohan Li 1eac8e0776 fix(browser): drop session injection from extension exec results (#1518)
`pageScopedResult()` in extension/src/background.ts was spreading the
lease's session into the result `data` for every page-scoped command. For
the `exec` action — which routes user JavaScript through page.evaluate()
— this contaminated arbitrary user-JS returns:

* Array / primitive returns came back as `{ session, data: <value> }`
  envelopes. Adapters that did `Array.isArray(result)` got `false` and
  treated the page as having no rows. Visible repro:
  `opencli google search ...` and `opencli xiaohongshu search ...` —
  Chrome rendered results correctly but adapters extracted an empty array
  (reported in #1518 from the Browser Bridge v1.0.12 envelope).
* Plain-object returns had an extra `session` key spliced in, silently
  overwriting any user `session` field with the lease's value.

Fix in the extension layer instead of compensating client-side:
`pageScopedResult` now returns `{ id, ok, data, page }` — the same form
it had before #1461 added the workspace→session refactor. Client-side
unwrapping is no longer needed and the original PR #1518 `Page.evaluate`
heuristic is dropped (it only covered the array path and would have
missed the plain-object path).

Two adapter improvements kept from the original PR:

* `clis/google/search.js` — wait for `#rso a h3` (with a 5s timeout)
  before extracting. On Chrome 148 / Linux Wayland the DOM can settle
  before SERP anchors are populated, so the existing fixed `wait 2`
  could return empty even with the envelope fix.
* `clis/xiaohongshu/search.js` — extract initially visible cards before
  scrolling, then merge post-scroll rows by URL. Xiaohongshu's
  virtualized masonry can evict the initial note cards from the DOM
  after scroll, causing extraction to return [] even though the
  browser had rendered results correctly.

Extension version bumped to 1.0.14.

Repro environment (from #1518):

* OpenCLI 1.7.18
* Browser Bridge extension 1.0.12 → 1.0.14
* Chrome 148.0.7778.96
* Linux Wayland, Node 22.22.1

Tests: extension/src/background.test.ts navigate same-url assertion
updated to no longer expect `session` in `data`. Three Page.evaluate
unwrap test cases removed.
2026-05-13 17:45:07 +08:00
Benjamin Liu 6af4db2ab5 fix(xueqiu/kline,earnings-date): format dates in Asia/Shanghai instead of UTC (#1498)
* fix(xueqiu/kline,earnings-date): format dates in Asia/Shanghai instead of UTC (#1465)

`xueqiu/kline` and `xueqiu/earnings-date` formatted bar timestamps with
`new Date(ts).toISOString().split('T')[0]`. That string is the UTC
calendar date, always one day earlier than the date xueqiu shows in its
UI (which is Beijing-aligned for every market). Issue #1465 reports
"5月10日跑的,5月8号的k线没有" because the May 8 China trading-day bar
was labeled 2026-05-07. Same off-by-one was present in `earnings-date.js`.

Routes both call sites through a new `formatChinaDate(ts)` helper in
`clis/xueqiu/utils.js` built on `toLocaleDateString('en-CA', { timeZone:
'Asia/Shanghai' })`. Verified live against SZ300136 and AAPL: both now
match the dates shown on xueqiu.com.

Tests: `clis/xueqiu/utils.test.js` (new) pins the Asia/Shanghai semantic
with 4 cases (China midnight, late-evening, 16:00 UTC day boundary, and
nullish input). `npx vitest run --project adapter clis/xueqiu/` 49/49,
`npx tsc --noEmit` clean, `npm run build` 815 entries unchanged shape.

Closes #1465

* fix(xueqiu): stabilize China date formatting

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 17:40:28 +08:00
jakevin 5127211ec1 refactor(extension): remove lease key session backdoor (#1510) 2026-05-12 22:43:30 +08:00
jakevin 587750cad3 refactor(env): remove OPENCLI_KEEP_TAB (#1509)
`OPENCLI_KEEP_TAB` was a debugging shortcut, not a config dimension. It
let users override `--keep-tab` globally via the shell environment,
which contradicts the per-command lifecycle model: `siteSession:'persistent'`
already pins persistent site tabs as a hard adapter-metadata constraint,
and `--keep-tab true|false` covers the ad-hoc override case. The env
just leaked process state across every browser command in the shell.

Changes:
  - src/execution.ts: `resolveKeepTab()` drops the
    `normalizeBooleanOption('OPENCLI_KEEP_TAB', process.env.OPENCLI_KEEP_TAB)`
    fallback. `--keep-tab` is now the single user override.
  - src/execution.test.ts: two regression tests rewritten to use the
    `executeCommand(cmd, {}, false, { keepTab: 'true' })` signature
    instead of the env. Logic and assertions unchanged.
  - README.md / README.zh-CN.md / skills/opencli-usage/SKILL.md:
    drop the env table row. `--keep-tab` documentation stays.
  - CHANGELOG.md: BREAKING entry under Unreleased.

Note: the 1.7.15 CHANGELOG entry still references the env historically;
that's intentional, historical entries are not retroactively edited.

Verification:
  - npx tsc --noEmit pass
  - npx vitest run --project unit --project extension → 1144/1145 pass
    (1 unrelated skip)
  - typed-error-lint baseline 189
  - silent-column-drop baseline 103
2026-05-12 22:02:17 +08:00
jakevin a77e05847f feat(browser): add function form page evaluate (#1508) 2026-05-12 21:48:44 +08:00
jakevin 0e168d570e refactor(browser): replace --session flag with <sessionname> positional (#1505)
* refactor(browser): replace --session flag with <sessionname> positional

The `--session <name>` flag was semantically required but syntactically
optional, which is an anti-pattern. Required + flag is a contradiction:
flag form implies "optional", required is a runtime patch on top. Session
is OpenCLI's "operation target" identifier — the natural form for that is
a positional argument, like `docker exec <container> <cmd>` or
`git checkout <branch>`.

New surface:

  opencli browser <sessionname> open https://x.com
  opencli browser <sessionname> click 12
  opencli browser <sessionname> bind
  opencli browser <sessionname> unbind

Commander 14 cannot natively combine a parent positional with subcommand
dispatch — the parent's positional is shadowed by subcommand matching. To
bridge that, main.ts now pre-processes argv: when the token after `browser`
is non-flag and not a known subcommand name, it is treated as the
sessionname and rewritten to the internal `--session <name>` flag form
before commander parses it. Help text on the `browser` command is
overridden via `.usage('<sessionname> <command> [options]')` so users see
the positional form.

Reserved subcommand names (33) are listed in cli-argv-preprocess.ts and
tested for parity with cli.ts subcommand registrations. If a future
subcommand is added, the test fails loudly.

Synced surfaces:
  - README.md / README.zh-CN.md — all examples
  - docs/guide/browser-bridge.md (+ zh)
  - skills/opencli-browser/SKILL.md (bind/unbind, examples, table)
  - skills/opencli-usage/SKILL.md
  - tests/e2e/browser-tabs.test.ts
  - CHANGELOG.md (Unreleased BREAKING)

The internal `--session` flag and the unit tests calling
`program.parseAsync(['...', 'browser', '--session', 'foo', ...])` are
preserved as a stable internal API: tests bypass main.ts pre-processing
and exercise commander directly. The pre-processor has its own targeted
test file (cli-argv-preprocess.test.ts, 10 tests, all green).

Verification:
  - npx tsc --noEmit — pass
  - npx vitest run --project unit — 1073/1074 pass (1 unrelated skip)
  - npx vitest run --project extension — 61/61 pass
  - npm run check:typed-error-lint — baseline 189
  - npm run check:silent-column-drop — baseline 103

* fix(cli-argv): only rewrite when `browser` is the root command

The preprocessor was looping through every argv slot and would mis-rewrite
occurrences of the literal word `browser` deeper in argv (e.g. `opencli
adapter init browser/x` or arg values containing `browser`).

Now the preprocessor walks past leading root flags + their values to
identify the root command token, and only acts when that token is
`browser`. The full set of root value-consuming flags
(`ROOT_VALUE_FLAGS`) is documented inline and kept in sync with the
`program.option()` calls in cli.ts.

Adds regression tests:
  - `opencli adapter init browser x` not rewritten
  - URL/path values containing `browser` not rewritten
  - `list browser state` (different root command) not rewritten
  - `--profile work browser foo state` correctly identifies `foo` as
    sessionname (not as --profile's value)
  - `--profile=work` long-form-with-equals consumes one slot only
  - boolean flags (`-v`) don't consume the next value

12/12 preprocessor tests pass.

* fix(cli-argv): hide --session flag, fail-fast on retired form, rename to <session>

Three blockers in #1505 review:

1. `--session` flag was still visible in `opencli browser --help` and could
   be used as a public entrance, contradicting "positional only" UX.
   Fix: switch from `.requiredOption()` to `.addOption(new Option(...).hideHelp())`.
   The flag is preserved as an internal API for the daemon protocol and direct
   `program.parseAsync` callers (tests), but is no longer documented or
   surfaced in structured help.

2. `opencli browser --session foo state` still succeeded. Now the argv
   preprocessor throws `BrowserSessionArgvError` when root `browser` is
   followed by `--session`, and main.ts catches it and exits with a
   user-facing usage error pointing to the positional form.

3. Missing-session error message exposed the internal flag:
   `required option '--session <name>' not specified`. Now `getBrowserSession()`
   in the action body throws `<session> is a required positional argument:
   opencli browser <session> <command>`, and commander no longer guards the
   hidden option.

Also (per @WAWQAQ) rename placeholder `<sessionname>` -> `<session>` everywhere
user-facing — shorter, matches CLI convention. The help text "<session> is a
required positional: pass the name of the browser session..." carries the
"name" semantics in description, not in the placeholder itself.

Sync surfaces:
  - src/cli.ts — usage line, addOption with hideHelp, descriptions
  - src/cli-argv-preprocess.ts — throw on --session form
  - src/cli-argv-preprocess.test.ts — refusal test for old form
  - src/cli.test.ts — assertions updated for hidden option + new error path
  - src/help.ts — read `_usage` private field to respect `.usage()` override
    (commander's `.usage()` getter returns auto-generated form if not set,
    which would otherwise pollute every namespace's usage string)
  - src/main.ts — catch BrowserSessionArgvError, stderr + exit
  - README.md / README.zh-CN.md
  - docs/guide/browser-bridge.md / docs/zh/guide/browser-bridge.md
  - skills/opencli-browser/SKILL.md / skills/opencli-usage/SKILL.md
  - CHANGELOG.md

Manual smoke tests (against built dist):
  - `opencli browser --help` shows `Usage: opencli browser <session> <command> [options]`
  - `opencli browser --help` Options block does NOT show `--session`
  - `opencli browser --session foo state` → friendly error, no commander stacktrace
  - `opencli browser state` → `<session> is a required positional argument: opencli browser <session> <command>`
  - `opencli browser foo state` → parses correctly

* fix: inject <session> into subcommand help paths and drop stale sessions ref

Two follow-up blockers from #1505 review:

1. Subcommand help and structured help still rendered the command path
   without the parent's positional. `opencli browser foo state --help`
   showed `Usage: opencli browser state [options]`, which would lead
   users (and agents reading structured help) to think
   `opencli browser state` was a valid invocation. Now:

   - `commanderPath()` injects an ancestor's leading-positional placeholder
     (extracted from its `.usage()` override) between the ancestor's name
     and the next path segment when building paths upward.
   - `commandPathFromRoot()` strips placeholder segments (e.g. `<session>`)
     from the relative `name` field so agents can still address subcommands
     by their leaf name; placeholders remain in the `command` / `usage`
     display paths.
   - `program.configureHelp({ commandUsage: ... })` is applied recursively
     to every descendant of `browser`, because commander does NOT inherit
     `configureHelp` into subcommands.

   Result:
     opencli browser <session> click --help
     -> Usage: opencli browser <session> click [target] [options]

   Daemon, plugin, adapter, profile namespaces (no `.usage()` override)
   are unaffected.

2. `skills/opencli-browser/SKILL.md` still referenced
   `opencli browser sessions`, which was removed in #1470. Replaced the
   sentence with the underlying invariant ("Bound sessions have no
   OpenCLI idle-close timer; the binding lasts until `unbind`, tab close,
   window close, or daemon restart") without mentioning the deleted
   command.

Tests:
  - cli.test.ts: structured help expectations updated to include
    `<session>` in command/usage paths (3 tests)
  - cli-argv-preprocess.test.ts: 12 tests still green
  - 1136/1137 unit+extension green (1 unrelated skip)
  - typed-error-lint baseline 189
  - silent-column-drop baseline 103
2026-05-12 20:44:29 +08:00
jakevin fa9b38cd92 feat(reddit): add whoami, home, subreddit-info read commands (#1491)
* feat(reddit): add whoami, home, subreddit-info read commands

Closes gap against jackwener/rdt-cli — three commands the existing 17 reddit
adapters were missing:

- `reddit whoami` — show the currently logged-in identity (fields:
  Username, ID, Post / Comment / Total Karma, Account Created, Gold, Mod,
  Verified Email, Has Mail, Inbox Count). Probes `/api/me.json` with
  two-pronged auth detection (401/403 OR `data.name` missing on 200 —
  Reddit returns 200 with an empty body for stale anon sessions, see PR
  #1428).

- `reddit home` — personalized Best feed (`/best.json`). Distinct from
  the public `frontpage`/`r/all` command: enforces login via the same
  two-pronged auth check rather than silently degrading to the
  unauthenticated default feed. `--limit` accepts [1, 100] — out-of-range
  raises `ArgumentError` before navigation, no silent clamp.

- `reddit subreddit-info` — subreddit metadata (Name, Title, Subscribers,
  Active Now, NSFW, Type, Description, Created, URL) from
  `/r/<X>/about.json`. Banned / private / quarantined / 404 subreddits
  raise `EmptyResultError` so the output table never holds a silent
  sentinel row.

All three use Strategy.COOKIE + siteSession:'persistent' matching the
existing reddit adapters, validate args upfront before `page.goto`, and
use the 5-kind discriminated-union pattern (kind: auth/http/missing/
exception/ok) from PR #1428 to map page.evaluate results to typed errors
on the Node side. Intermediate object keys deliberately avoid the
declared columns (`field`/`value`/`rank`/etc.) per the silent-column-drop
audit sediment from PR #1329.

Tests: 28 new (whoami 6, home 9, subreddit-info 13); full reddit suite
38/38. Audits: typed-error-lint 189/189 (0 new), silent-column-drop
103/103 (0 new). Manifest 812 → 815.

Refs: https://github.com/jackwener/rdt-cli

* fix(reddit): tighten new read command failure contracts

* fix(reddit): treat inaccessible subreddit info as empty
2026-05-12 04:10:44 +08:00
jakevin 93bc374437 chore(scripts): auto-refresh dist/ before build-manifest (#1490)
* chore(scripts): auto-refresh dist/ before build-manifest

`build-manifest.ts` is invoked via tsx so its own imports go to TS source,
but the adapter `.js` files it loads import `@jackwener/opencli/registry`
through package exports, which resolves to `dist/src/registry-api.js`.

When `dist/` is stale relative to `src/` (e.g. a contributor edits
`src/registry.ts` and runs only `npm run build-manifest` instead of the
full `npm run build`), the stale dist drops fields like `siteSession`
from the rebuilt manifest. CI catches the resulting diff via the
"cli-manifest.json is up-to-date" gate, but locally it surfaces as
mysterious unrelated diff lines for adapter files the contributor never
touched.

Add an npm pre-script that runs `tsc --build` (incremental, ~0.6s when
warm) so `npm run build-manifest` is safe to use directly. `npm run build`
is unchanged — it still does the full `clean-dist + tsc + copy-yaml +
build-manifest` sequence, and `prebuild-manifest` will be a no-op there
since TS is already compiled by the time it runs.

Verified:
- `rm -rf dist && npm run build-manifest` now restores dist via the
  pre-hook and produces a 0-line diff against committed manifest
- `npm run build` still produces the same clean output

* fix(scripts): force manifest dist refresh

* fix(scripts): avoid duplicate manifest compile
2026-05-12 04:00:24 +08:00
jakevin eb59b7444d feat(ctrip): add hotel-search + flight browser-mode commands (#1481) (#1489)
* feat(ctrip): add hotel-search + flight browser-mode commands

Closes #1481.

Two new browser-mode commands on top of the existing public `search` /
`hotel-suggest` pair:

- `ctrip hotel-search <city> --checkin --checkout [--limit]` reads
  `window.__NEXT_DATA__.props.pageProps.initListData.hotelList` on
  `hotels.ctrip.com/hotels/list`. SSR-rendered first page ships ~13
  entries; the server ignores `&pageSize=N` so limit caps at 30 with
  default 10. AuthRequiredError surfaces when Ctrip redirects to the
  captcha gate.

- `ctrip flight <from> <to> --date [--limit]` searches one-way flights on
  `flights.ctrip.com/online/list/oneway-…`. The post-load XHR is not
  currently captured by the daemon network buffer (per the known
  daemon_capture_pipeline_bug_2026_05_07 in agent memory), so rows are
  pulled from `.flight-list > span > div` cards via a position-anchored
  innerText parser. A generic `buildScrollUntilJs(selector, target)`
  helper mirrors the PR #1487 xiaohongshu scroll-until pattern with the
  selector parameterised. Round-trip + airline filters are out of scope
  for v1.

All argument validation (IATA / ISO date / city ID / limit range) fires
upfront before any `page.goto`, per the PR #1387 boundary standard. No
silent clamps, no sentinel rows: rows missing required fields are
dropped, and end-state checks raise `ArgumentError` /
`AuthRequiredError` / `EmptyResultError` as appropriate. The new
`mapHotelRow` / `pickHotelMapCoords` / `buildFlightExtractJs` /
`buildScrollUntilJs` helpers live in `clis/ctrip/utils.js` alongside the
existing suggest helpers.

Docs at `docs/adapters/browser/ctrip.md` now distinguish the public
suggest commands from the browser-mode commands and document each
command's columns + caveats.

Verified:
- 61/61 vitest tests in `clis/ctrip/ctrip.test.js` (including JSDOM
  exercises of `buildFlightExtractJs` and full `mapHotelRow` shape parity)
- `check:typed-error-lint` 189/189 (0 new)
- `check:silent-column-drop` 103/103 (0 new)
- `build-manifest` clean — 812 entries total (was 810)

* fix(ctrip): harden browser search failure contracts

* fix(ctrip): tighten browser empty-vs-parser failures
2026-05-12 03:43:32 +08:00
jakevin 43d0722264 docs(skill/adapter-author): aria-label / placeholder / title are locale-dependent (#1474) (#1488)
* docs(skill/adapter-author): warn aria-label / placeholder / title is locale-dependent

aria-label changes with the browser's UI language (chrome://settings/languages).
A button labelled `aria-label="Submit"` in English Chrome becomes
`aria-label="提交"` in Chinese Chrome, so CSS selectors hardcoded to one
locale silently match zero elements — `notEmpty` / `types` never fire because
the adapter just returns 0 rows.

First-principles framing in adapter-template:
  - Split DOM attributes into "locale-stable identifiers" (id / class /
    data-testid / data-* / role) vs "locale-dependent text" (aria-label /
    title / placeholder / alt / textContent)
  - Primary selectors must use locale-stable identifiers; locale-dependent
    text is a last-resort tiebreaker
  - When a site (e.g. ChatGPT web) only exposes aria-label, link the existing
    `clis/chatgpt/utils.js` fallback-list pattern (en + zh-CN + stable
    fallback at the front)

Explicitly document why we are NOT building a `find --i18n "zh:提交"` flag
(over-engineering: same indirection as a fallback list plus a translation
dictionary to maintain) and why we are NOT locking Chrome's locale at launch
(opencli doesn't launch Chrome — it connects to the user's running browser
via CDP, so forcing en-US would break users who intentionally run Chinese UI).

Adds pitfall #11 to success-rate-pitfalls.md for the agent-facing checklist.

Closes #1474

* docs(skill): tighten locale selector guidance
2026-05-12 03:13:53 +08:00
jakevin 7df9b80dea fix(xiaohongshu+rednote): scroll until enough rows for --limit > 13 (#1471) (#1487)
* fix(xiaohongshu+rednote): scroll until enough rows are rendered instead of fixed 2x autoScroll

Both search adapters previously called `page.autoScroll({ times: 2 })` which
hard-capped extraction at ~13 notes (xiaohongshu lazy-loads ~5-7 notes per
scroll round) regardless of `--limit`. Reported in #1471: `--limit 40` still
only returned 13 results.

Replace with a dynamic `buildScrollUntilJs(targetCount, maxScrolls=15)`
helper that:
  - counts visible `section.note-item` rows (excluding `.query-note-item`
    related-search rows)
  - breaks early when count >= target
  - breaks early after 2 consecutive scrolls add no new rows (DOM plateaued,
    feed exhausted)
  - hard caps at 15 iterations to bound runtime

Exported from xiaohongshu and reused by rednote (same DOM shape) instead of
duplicating the IIFE.

Fixes #1471

* fix(xiaohongshu): tighten search scroll boundary
2026-05-12 02:59:15 +08:00
jakevin 23e1161ffd chore(release): 1.7.18 (#1486)
Release / release (push) Has been cancelled
2026-05-12 02:50:24 +08:00
jakevin dccf9d00e9 fix(doctor): pass session to connectivity probe (#1485)
* fix(doctor): pass session to connectivity probe

* fix(doctor): isolate probe session name

* fix(cli): mark browser session as required
2026-05-12 02:49:21 +08:00
jakevin b476d2364f fix(doubao/ask): restore Assistant detection after 2026-05 DOM refactor (#1484)
* fix(doubao/ask): restore Assistant turn detection after 2026-05 DOM refactor

Doubao reworked message-item wrappers and dropped all `receive-message` /
`bg-g-receive-msg-bubble` markers from assistant turns. The legacy 6
`itemSelectors` (`item-kDun2N`, `union_message`, `message-block-container`,
`data-message-id`, `bg-g-send-msg-bubble`, `bg-g-receive-msg-bubble`) match 0
elements on the new DOM, so `getTurnsScript` returned [] and `getDoubaoTurns`
fell through to the whole-page transcript scraper. Assistant text came back as
sidebar labels + history titles + adjacent conversation snippets concatenated
with the real reply — silent SELECTOR failure (no thrown error).

Two minimal changes in `clis/doubao/utils.js` `getTurnsScript`:

1. `itemSelectors`: prepend `[class*="inner-item-"]` and `[class*="top-item-"]`
   — the new 2026-05 wrappers. Outer wins via existing ancestor-keep dedup
   below, so we get one root per turn (not one per nested chunk).
2. `getRole`: add a third fallback branch — if the root matches
   `inner-item-*` / `top-item-*`, contains `.flow-markdown-body`, and has NO
   `bg-g-send-msg-bubble` marker (User detection still works), treat it as
   Assistant. `.flow-markdown-body` is already in `messageTextSelectors`, so
   text extraction kicks in unchanged.

Test added asserting both new wrappers and the `.flow-markdown-body` assistant
fallback are present in the generated script.

Fixes #1478

* test(doubao): cover refactored assistant turns
2026-05-12 02:46:53 +08:00
Kagura 6d84009ee8 fix(youtube): request srv3 format for caption URLs (#1420) (#1422)
* fix(youtube): request srv3 format for caption URLs (#1420)

YouTube may return empty responses when caption URLs lack an explicit format
parameter. This adds fmt=srv3 (standard YouTube XML caption format) to the
caption URL when no fmt parameter is already present, with a fallback to the
original URL if srv3 also returns empty.

Also adds HTTP status checking before reading the response body, preventing
silent failures on non-200 responses.

Fixes #1420

* fix(youtube): preserve caption fetch failures

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-12 02:44:21 +08:00
Gaurav Saxena 150551be8c feat(reddit): add reply command for replying to comments (#1428)
* feat(reddit): add reply command for replying to comments

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(reddit/reply): replace silent-sentinel rows with typed errors

reply.js originally mirror-copied comment.js's failure pattern: returning
[{ status: 'failed', message: 'HTTP 403' }] on auth/HTTP/Reddit errors and
relying on the caller to inspect the row instead of throwing. That's the
'silent-sentinel' anti-pattern from typed-errors.md — failures should
surface as typed errors so an agent can actually branch on them.

Round 21 lesson (f) — "grandfathered-not-exempt + helper-refactor boundary
is new" — applies: comment.js / upvote.js / save.js can stay grandfathered,
but a brand-new file does not inherit that exemption.

Changes:
- Throw AuthRequiredError when /api/me.json or /api/comment returns 401/403,
  or when /api/me.json returns 200 but data.name is missing (stale anon
  session — empty modhash alone isn't a strong enough signal).
- Throw CommandExecutionError for non-2xx HTTP and for non-empty
  data.json.errors (e.g. RATELIMIT, NO_TEXT, TOO_OLD).
- Drop the over-defensive `if (!page) throw ...` — registry guarantees a
  page object when browser:true.
- Intermediate result object uses `kind` discriminator + `detail` /
  `httpStatus` / `where` keys that don't overlap with columns
  ['status','message'], so the silent-column-drop audit stays quiet
  (per PR #1329 sediment).

Verified:
- npx tsc --noEmit clean
- node scripts/check-typed-error-lint.mjs → 189/189, 0 new
- node scripts/check-silent-column-drop.mjs → 103/103, 0 new
- npx vitest run clis/reddit src/convention-audit → 11/11 pass
- node ./dist/src/main.js validate → 0 errors

Success path is unchanged: still returns
[{ status: 'success', message: 'Reply posted on t1_<id>' }].

* fix(reddit): harden reply command contract

* fix(reddit): reject suffixed reply urls

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-12 02:33:09 +08:00
Benjamin Liu 64ac362a40 feat(rednote): add rednote.com adapter mirroring xiaohongshu read commands (#1136) (#1475)
* feat(rednote): add rednote.com adapter mirroring xiaohongshu read commands (#1136)

Implements rednote.com support as discussed in issue #1136. The mainland
xiaohongshu adapter stays in place; international users redirected to
www.rednote.com now have a CLI without a copy-pasted adapter.

Issue #1136 documents that xiaohongshu and rednote share DOM selectors,
URL paths, API paths, response schema, cookies, and the xsec_token auth
mechanism. The only material differences:

  Layer            xiaohongshu                rednote
  Web host         www.xiaohongshu.com        www.rednote.com
  API host         edith.xiaohongshu.com      webapi.rednote.com
  Security host    fe-static.xhscdn.com       as.rednote.com
  Cookie root      .xiaohongshu.com           .rednote.com
  Search gate      Inline text                Full-screen modal + text

## Architecture (minimal)

`clis/xiaohongshu/*` keep all selector / regex / extraction logic. Each
command file is touched minimally to export the IIFE or pipeline so the
sibling adapter can reuse it:

  search.js          + export const buildSearchExtractJs(webHost)
                     + export const command = cli({...})
  note.js            + export const NOTE_EXTRACT_JS
                     + export const command = cli({...})
  comments.js        + export function buildCommentsExtractJs(withReplies)
                     + export parseCommentLimit
                     + export const command = cli({...})
  download.js        + export function buildDownloadExtractJs(noteId)
                       (CDN allowlist now includes rednote alongside xhscdn)
                     + export const command = cli({...})
  user.js            + export const USER_SNAPSHOT_JS
                     + export const command = cli({...})
  feed.js            + export function buildFeedPipeline(webHost)
                     + export const command = cli({...})
  notifications.js   + export function buildNotificationsPipeline(webHost)
                     + export const command = cli({...})
  note-helpers.js    buildNoteUrl now accepts `cookieRoot` + `signedUrlHint`
                     options (defaults preserved so xhs callers and tests
                     are unchanged)
  user-helpers.js    buildXhsNoteUrl / extractXhsUserNotes accept an
                     optional `webHost` argument (default xhs)

The `export const command = cli({...})` pattern matches twitter/lists.js
and clis/discord-app/*; without it the build-manifest scanner attributes
xhs's command to whichever rednote sibling triggered the transitive
import first.

## clis/rednote/ — thin shims

Each rednote command file imports the relevant builder / constant from
its xiaohongshu sibling and calls `cli()` with the rednote host triple.
No selectors, regexes, or extraction logic are duplicated.

  search.js          imports buildSearchExtractJs + noteIdToDate
                     declares its own WAIT_FOR_CONTENT_JS (modal + text
                     login-gate variants — the one xhs behaviour that
                     genuinely differs)
  note.js            imports NOTE_EXTRACT_JS + buildNoteUrl + parseNoteId
  comments.js        imports buildCommentsExtractJs + parseCommentLimit
                     + buildNoteUrl + parseNoteId
  download.js        imports buildDownloadExtractJs + buildNoteUrl + parseNoteId
  user.js            imports USER_SNAPSHOT_JS + extractXhsUserNotes
                     + normalizeXhsUserId

## Scope (initial)

Ships the five commands verified live against the user's logged-in
rednote.com session: search / note / comments / user / download.

`feed` and `notifications` are intentionally left out. Both rely on
intercepting the xiaohongshu Pinia store at the `homefeed` / `you`
capture pattern; live verification on rednote returns `tap → dict
(error)` for the feed step, so shipping them would surface a broken
contract. The mainland xiaohongshu commands continue to work. Adding
the rednote-side feed / notifications is straightforward follow-up
work once someone with rednote access maps the network surface.

Creator-center commands (publish, creator-*) have no rednote
counterpart and stay xiaohongshu-only, per the reporter's note in #1136.

## Verification

  - clis/xiaohongshu/ + clis/rednote/: 103/103 tests green
  - npx tsc --noEmit: clean
  - npm run build: 807 manifest entries (xhs 13 + rednote 5 + everything
    else preserved)
  - silent-column-drop / typed-error-lint: 103 / 189 baseline entries,
    no new violations
  - Live verify against the user's rednote.com session:
      rednote search "travel" --limit 1 → real note row
      rednote note <signed-url>         → 7 field/value rows
      rednote comments <signed-url> --limit 3 → 3 top-level rows
      rednote user 5b21f6564eacab3b38f05c39 --limit 2 → 2 profile notes
    Spaced 15–30s between runs per the xhs/rednote rate-limit guidance;
    no write commands invoked. Regression check: xiaohongshu/feed on
    the existing mainland session still returns the standard 6-field
    rows after the refactor.

Closes #1136

* fix(rednote): tighten adapter failure boundaries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-12 02:32:25 +08:00
jakevin c1af68b909 chore(release): 1.7.17 (#1483)
Release / release (push) Has been cancelled
2026-05-12 02:24:16 +08:00
E2ern1ty b262d8ffd5 feat(chatgpt): support local image uploads (#1476)
* feat(chatgpt): support local image uploads

* chore: refresh cli manifest

* fix(chatgpt): harden image upload flow

* fix(chatgpt): validate image uploads before navigation

* fix(chatgpt): keep send fallback click in sync

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-11 17:52:16 +08:00
jakevin 987d9cba48 refactor(doctor): drop --no-live and --sessions flags + dead protocol (#1470)
Doctor's job is browser-bridge health diagnosis. The `--no-live` flag
let users skip the connectivity probe (= the core diagnostic), and
`--sessions` listed automation sessions (a separate concern not part of
health). Both flags accreted features that violated the command's
first-principles purpose.

Cleanup chain (removing dead code surfaced by the flag removal):
- `--no-live` / `--sessions` flags removed from `opencli doctor`
- `DoctorOptions.live` / `DoctorOptions.sessions` removed
- `DoctorReport.sessions` removed
- `[SKIP] Connectivity` render branch removed (always-live now)
- `listSessions()` removed (only consumer was doctor)
- `'sessions'` action removed from daemon-client protocol type
- `BrowserSessionInfo` type removed (no remaining consumers)
- extension `handleSessions` action handler removed (1.0.12)
- extension test "reports sessions per session" removed
- `OPENCLI_BROWSER_IDLE_TIMEOUT` test rewired to 'cookies' action

Verification:
- root typecheck + extension typecheck pass
- doctor.test.ts 17/17 pass
- extension/background.test.ts 49/49 pass
- typed-error-lint 189/189 baseline
- silent-column-drop 103/103 baseline
- build + extension build green
2026-05-11 13:10:06 +08:00
jakevin 467fdd0b62 refactor(adapter): rename site browser reuse to persistent sessions (#1462) 2026-05-11 04:56:34 +08:00
jakevin 9c06e84c89 refactor(browser): replace workspaces with sessions (#1461) 2026-05-11 04:26:51 +08:00
jakevin b56bebdd7a chore(release): 1.7.16 (#1460)
Release / release (push) Has been cancelled
2026-05-11 03:27:06 +08:00
jakevin 1d2e606498 perf(chatgpt): replace fixed-sleep waits with selector-based readiness (D3) (#1456)
Continues the wait→event sweep started in #1449 (deepseek) / #1452 (claude). Same
3-bucket classification across the chatgpt adapter:

CONVERT (5)
- utils.js ensureOnChatGPT/startNewChat: 2s settle → wait({selector: composer, 8s})
- utils.js getConversationList: openSidebar 1.5s + fallback goto 2.5s → selector
- detail.js: post-/c/<id> goto 2s → wait({selector: message bubble, 10s})

DELETE (6)
- ask/send/read.js: standalone 2s settle after ensureOnChatGPT/startNewChat
  (those helpers now wait for composer internally — settle is redundant)
- utils.js sendChatGPTMessage: 0.5s post-closeBtn + 1.5s pre-composer-focus
- utils.js getConversationList: 2s settle after ensureOnChatGPT (helper waits
  for composer; we re-check sidebar selector independently)

KEEP (6)
- utils.js sendChatGPTMessage: ProseMirror React debounce ticks
- utils.js waitForChatGPTResponse: streaming response polling cadence

Verification:
- npx vitest run clis/chatgpt → 20/20 (4 files)
- Full vitest → 3379 pass / 1 skip (2 errors are the unrelated daemon EADDRINUSE
  flake also seen on #1449/#1452/#1454)
- tsc --noEmit clean / typed-error-lint 189 / silent-column-drop 103 unchanged
- npm run build → 802 manifest entries

Diff: +50 / -13 across 5 files (ask.js, detail.js, read.js, send.js, utils.js).
No typed-error harmonization needed — chatgpt's existing helpers already gate
through ensureChatGPTLogin (AuthRequiredError) and ensureChatGPTComposer
(CommandExecutionError) correctly.
2026-05-11 03:22:39 +08:00
jakevin 864af48b0b docs(readme): list tg-cli, discord-cli, wx-cli in External CLI sections (#1459)
Follow-up to feat #1458 (registering tg-cli/discord-cli/wx-cli in
src/external-clis.yaml) — README and README.zh-CN had not been updated
to reflect the new entries.

Updates four spots in each README:
- intro paragraph that names example external CLIs
- "CLI Hub" highlight bullet
- "OpenCLI is not only for websites" bullet list
- the External CLI table itself
2026-05-11 03:19:53 +08:00
jakevin d0127e188a feat(external): register tg-cli, discord-cli, wx-cli (#1458)
Add three local-first messaging CLIs to the External CLI registry so
agents can discover and install them via `opencli external install`:

- `tg-cli` (binary `tg`) — Telegram local sync/search/export via MTProto
- `discord-cli` (binary `discord`) — Discord local sync/search/export
- `wx-cli` (binary `wx`) — WeChat local data CLI

Refresh the External CLI list in skills/opencli-usage/SKILL.md so the
agent-facing skill names stay in sync.
2026-05-11 03:19:27 +08:00
jakevin cd93910fdd feat(help): structured help for daemon/plugin/adapter/profile namespaces (#1407)
Extends A0 (PR #1404) by dogfooding `installCommanderNamespaceStructuredHelp`
on the four remaining built-in Commander namespaces:

- `opencli daemon --help -f yaml|json`
- `opencli plugin --help -f yaml|json`
- `opencli adapter --help -f yaml|json`
- `opencli profile --help -f yaml|json`

Each emits the same payload shape as `browser`: namespace metadata, every
leaf command's positionals + command_options + description + usage,
namespace_options (empty for these), and program-level global_options.
Agents can fetch every leaf's contract in a single call — no per-leaf
`--help` follow-ups.

Each namespace snapshots its original description at declaration time
because `applyRootSubcommandSummaries(program)` later overwrites
`.description()` with a child-name listing; without the snapshot,
structured help would surface `"restart, status, stop"` instead of
`"Manage the opencli daemon"`. Tests lock the snapshot semantics for
`adapter` explicitly.

Tests: 138/138 (4 new — one per namespace, covering description
preservation, leaf names, positionals, command_options).
Typecheck + build clean.
2026-05-11 02:50:09 +08:00
jakevin 64c67c331e chore(extension): rename adapter tab group (#1457)
* chore(extension): rename adapter tab group

* test(extension): update adapter group wording
2026-05-11 02:12:37 +08:00
jakevin 6d87142821 perf(reddit): opt 13 browser adapters into shared site-tab lease (#1455)
* perf(reddit): opt 13 browser adapters into shared site-tab lease

Adds `browserSession: { reuse: 'site' }` to every reddit adapter that
already runs `browser: true` on `domain: 'reddit.com'`. Same metadata-only
follow-up to the twitter sweep merged in #1454 — the framework's
`shouldRunPreNav` short-circuit (src/execution.ts:190) skips the redundant
domain-root pre-nav when a sibling adapter already has the tab on
reddit.com, and idle-bound tabs are reused under the `site:reddit` bucket
until expiry.

Scope (13 files, all on `domain: 'reddit.com'` + `Strategy.COOKIE`):
- read (9): frontpage / popular / saved / search / subreddit / upvoted /
  user / user-comments / user-posts
- write (4): comment / save / subscribe / upvote

Excluded:
- `hot.js` (no browser:true — public Reddit JSON API, no tab)
- `read.js` (Strategy.COOKIE but no browser:true — non-browser pipeline)

No logic changes; only metadata + manifest regeneration.

Verification:
- npm run check:typed-error-lint → 189/189 unchanged
- npm run check:silent-column-drop → 103/103 unchanged
- npm run test:adapter → 264/264 passed (2146 tests)
- npx vitest run --project unit → 72/72 passed (unrelated EADDRINUSE
  flake on daemon.test.ts port 19825, also seen on #1454/#1452)
- tsc --noEmit clean

* fix(reddit): include read in site browser session reuse
2026-05-11 02:12:21 +08:00
Ethon 357dec5969 fix(xiaohongshu): fallback to base64 upload when CDP setFileInput returns 'Not allowed' (#1374)
The uploadImages function catches errors from page.setFileInput and only
falls back to the legacy base64 DataTransfer method when the message
contains 'Unknown action' or 'not supported'. However, Chrome can also
return 'Not allowed' (code -32000), which was not handled — causing the
publish command to fail instead of using the fallback.

Add 'Not allowed' to the fallback condition so image upload works even
when CDP file injection is blocked by Chrome's security policy.

Co-authored-by: together <together@togetherdeMac-mini.local>
2026-05-11 01:56:19 +08:00
Benjamin Liu 674f0e1105 feat(openreview): add author command for ID-explicit publication lookup (#1365)
* feat(openreview): add author command for ID-explicit publication lookup

Closes the missing leaf in the openreview adapter. Among the public-strategy
academic adapters, dblp and arxiv both already ship an `author` command for
ID-explicit publication lookup; openreview only had `search` (full-text),
`paper` (detail by note id), `reviews` (thread by forum id) and `venue`
(listing by invitation / venue text). There was no way to ask "give me every
submission this author put on OpenReview, newest first."

`openreview author <profile>`:
  - takes a canonical profile id (`~First_LastN`); validated by
    `requireProfileId` so a dblp PID or a bare name fails before any
    network call,
  - hits `/notes?content.authorids=~<id>&limit=<n>&sort=cdate:desc`,
  - returns rank-ordered rows with the same shape as `openreview search`
    (id / title / authors / venue / pdate / url),
  - throws `EmptyResultError` when the profile has no public submissions
    instead of returning an empty list,
  - inherits the typed-error envelope from `openreviewFetch` so network
    failure, non-200, malformed JSON, and in-band error envelopes all
    surface as `CommandExecutionError`.

Tests: 6 new `it` blocks plus 1 updated registration test in
`clis/openreview/openreview.test.js`.

  - `requireProfileId` (1 block, 9 assertions): accepts canonical
    `~First_LastN`, `~Bo_Liu17`, and a multi-segment middle-name id;
    rejects empty, whitespace, missing tilde, missing trailing number,
    embedded space, and a dblp-style PID.
  - 5 author runtime cases covering pre-network ArgumentError, empty
    result, non-200, fetch network error, and the happy path with a
    request-shape assertion (`content.authorids` filter + `cdate:desc`
    sort).
  - Registration test extended to expect five commands and lock the new
    `columns` contract.

Manifest auto-regenerated to register the new command.

Live-verified end to end against `~Yoshua_Bengio1`: the most recent ICLR
2026 workshop submissions return with the expected fields. A malformed
profile is rejected before any HTTP call. A nonexistent profile yields
`EMPTY_RESULT`.

* fix(openreview): accept real profile id slugs

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-11 01:53:46 +08:00
UtoPiaCD 2034e90337 fix(chatgpt): use locale-stable send button selector (#1354)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-11 01:53:08 +08:00
jakevin a77d9c930a perf(claude): replace fixed-sleep waits with selector-based readiness (#1452)
Convert 9 of 18 page.wait(N) calls in clis/claude/ from fixed-duration
sleeps to event-based readiness checks (page.wait({ selector, timeout }),
backed by MutationObserver). Mirrors the deepseek D1 template (PR #1449).

* Page-ready waits (5 converted): utils.js:29 (ensureOnClaude composer),
  utils.js:114 (getConversationList recents links), new.js:19 (composer),
  detail.js:24 (.font-claude-response message bubble), send.js:26 (composer).
  Each resolves as soon as the selector matches, swallowing the timeout so
  downstream typed-error helpers (ensureClaudeLogin / ensureClaudeComposer
  / EmptyResultError) still surface the right error when the selector
  never mounts (login redirect, empty conversation, etc).

* Dropdown waits (2 converted): utils.js:150 (selectModel post-trigger),
  utils.js:178 (setAdaptiveThinking post-trigger). Wait for menuitemradio
  / menuitem to mount instead of a fixed 0.6 s sleep.

* Resume conversation wait (1 converted): ask.js:51 — wait for the resumed
  message bubble (MESSAGE_SELECTOR) instead of a fixed 2 s sleep.

* Settle/redundant waits removed (6): ask.js:55 standalone settle (next
  ensureClaudeComposer queries composer presence directly via getPageState);
  ask.js:83 / ask.js:90 post-toggle settles (next CDP eval flushes React
  state between roundtrips); ask.js:103 pre-waitForResponse settle (the
  polling loop's first 3 s tick already covers this); read.js:20 post-
  ensureOnClaude sleep (ensureOnClaude now waits for the composer selector
  itself); send.js:29 post-ensureOnClaude sleep (same).

Three remaining page.wait(N) calls are kept: utils.js:231 post-input
1.2 s React debounce inside sendMessage (the ProseMirror editor needs a
debounce window before the send button enables; reducing this risks
silent send-button-disabled drops), and the 3 s / 1 s polling ticks in
waitForResponse / waitForFilePreview (already polling patterns, out of
scope for D-track wait→event sweep).

Targeted tests: clis/claude + src/browser 389/389 pass; tsc clean;
build clean; typed-error 189/189 baseline (no new); silent-column-drop
103/103 baseline (no new).

D2 in the LLM-adapter wait→event sweep started by deepseek (D1, #1449).
2026-05-11 01:52:41 +08:00
jakevin cb64192f06 perf(deepseek): replace fixed-sleep waits with selector-based readiness (#1449)
Convert 10 of 18 `page.wait(N)` calls in clis/deepseek/ from fixed-duration
sleeps to event-based readiness checks (`page.wait({ selector, timeout })`,
backed by MutationObserver):

* Page-ready waits (5): utils.js:46, ask.js:38/52, detail.js:28, new.js:19
  now wait for the composer textarea (TEXTAREA_SELECTOR) or message bubble
  (MESSAGE_SELECTOR) to mount before continuing. Resolves as soon as the
  selector matches instead of always sleeping the full duration.
* Settle/redundant waits removed (5): ask.js:56 standalone settle (already
  covered by upstream selector waits); ask.js:79/105 post-toggle settles
  (next CDP eval gives React time to flush aria-checked updates); ask.js:118
  pre-waitForResponse settle (the polling loop's first 3 s tick already
  covers this); read.js:19 post-ensureOnDeepSeek sleep (ensureOnDeepSeek
  now waits for the textarea selector itself).
* `new.js` now throws CommandExecutionError when the composer fails to
  mount within 8 s instead of silently returning "New chat started" on a
  half-loaded or logged-out page.

Eight remaining `page.wait(N)` calls are kept: in-loop polling ticks in
waitForResponse / pickResumeUrl / getConversationList / waitForFilePreview
/ send-button-enable polling (these are already polling patterns and
out of scope for D1), and the native-input flush + textarea-mount poll in
send.js.

Targeted tests: clis/deepseek 49/49, src/browser 355/355 pass; build,
typecheck, typed-error and silent-column-drop audits clean.

Proof template for the LLM-adapter wait-cleanup follow-ups.
2026-05-11 01:52:19 +08:00
jakevin 833c1c872f perf(twitter): enable browserSession reuse:site on 17 read-only adapters (PR B) (#1454)
Read-only Twitter/X adapters now declare `browserSession: { reuse: 'site' }`,
matching the LLM-site adapters (claude/gemini/yuanbao/etc.) and unblocking
the perf wins WAWQAQ called out for the 35s→9s/3.4s thread.js progression
(#OpenCLI:3889b5cf):

- Tab lease shared across calls under `site:twitter` until idle expiry, so
  the second-and-later command pays no cold-start tab cost.
- Framework's domain-root pre-nav (`https://x.com`) is skipped on subsequent
  calls when the reused tab is already on x.com (`shouldRunPreNav` →
  `isDomainRootPreNav` + `urlMatchesDomain` short-circuit at
  `src/execution.ts:190`).

Files (17 read-only adapters):
- Strategy.COOKIE × 13: article, bookmark-folder, bookmark-folders,
  bookmarks, download, following, likes, list-tweets, lists, profile,
  thread, timeline, trending, tweets
- Strategy.UI × 1: followers
- Strategy.INTERCEPT × 2: notifications, search

Insertion point in each file: after `browser: true,` (or after `strategy:`
in download.js which omits the explicit `browser:` field), matching the
convention used by yuanbao/read.js, claude/read.js, etc.

Manifest regenerated (cli-manifest.json: +85/-17 — 17 entries gain the
`browserSession: { reuse: "site" }` block).

Verification:
- npx tsc --noEmit clean
- npx vitest run clis/twitter → 218/218 pass (25 files)
- npx vitest run src/convention-audit.test.ts → 8/8 pass
- typed-error-lint baseline 189/189 (no new violations)
- silent-column-drop baseline 103/103 (no new violations)

Scope notes (intentionally NOT in this PR):
- Write adapters (post/reply/quote/like/retweet/bookmark/follow/list-add/
  list-remove/delete/hide-reply/block/accept/follow) are kept as one-shot
  by default — `reuse: 'site'` for write paths is a separate decision
  about action idempotency under tab reuse.
- The thread.js / timeline.js comments still say "Cookie context
  auto-established by framework pre-nav"; the deeper truth (CDP
  `getCookies({url})` is origin-independent) was a framing nit on PR C
  (#1451) — left as a doc-only follow-up to keep this PR's diff focused
  on the perf gain.

Refs: #OpenCLI:3889b5cf (WAWQAQ msg=fa209a2c, msg=35c90460, msg=838128ef
"你们继续做啊… 后面还有那么多其他的东西呢")
2026-05-11 01:51:59 +08:00
jakevin a92f382c2d refactor(browser): split interactive and automation windows 2026-05-11 01:48:18 +08:00
jakevin ff7d741a4c perf(twitter): drop redundant goto+wait — framework auto pre-navs (PR C) (#1451)
* perf(twitter): drop redundant goto+wait — framework auto pre-navs (PR C)

Twelve twitter read adapters did `await page.goto('https://x.com'); await
page.wait(2~3)` purely to establish cookie context for the subsequent
`document.cookie` read. After PR #1450 hoisted those reads to
`page.getCookies({url})` (which queries the CDP cookie store directly,
no navigation needed), the explicit goto+wait became dead.

The framework already pre-navigates to `https://${domain}` for any
adapter declaring `Strategy.COOKIE + domain` (`src/registry.ts:191`),
so the cookie store is populated before `func` runs. The 2-3s
`page.wait` was the slowest part of the redundant call.

Files (all read-only, all ct0/cookie-only):
- bookmark-folder / bookmark-folders / bookmarks
- following / likes / list-add / list-remove / list-tweets / lists
- thread / timeline / tweets

Out of scope (kept as-is): goto calls that navigate to a *specific*
URL needed for content/SPA shell — `trending` (`/explore/tabs/trending`),
`notifications` (`/home`), `article` (`/i/article/{id}`), `profile`
(`/${username}`), and `list-add` line 133 (`/${username}` for UI ops).

Verification:
- npx tsc --noEmit ✓
- npx vitest run clis/twitter → 216/216 ✓
- typed-error-lint 189/189, 0 new ✓
- silent-column-drop 103/103, 0 new ✓

* fix(twitter): keep list UI root navigation
2026-05-11 01:24:07 +08:00
jakevin 60dbbd4baa perf(adapters): hoist cookie reads to page.getCookies (Tier 1, 25 files) (#1450)
* perf: replace document.cookie reads with page.getCookies({domain}) (Tier 1 cookie API sweep)

Prior pattern in 25 adapter files round-tripped through `page.evaluate(\`document.cookie.split…\`)` to extract a single cookie value (CSRF token, session ID, etc.). CDP's `page.getCookies({domain})` reads the cookie store directly with zero JS-execution overhead.

Files touched (sites: twitter / linkedin / maimai / youtube):

- twitter (15): thread, timeline, list-add, bookmark-folders, following, list-tweets, bookmarks, list-remove, tweets, bookmark-folder, likes, lists, trending — direct 4-line replacement (cookie was outside `page.evaluate`); article, profile — hoisted ct0 read OUT of `page.evaluate` and threw `AuthRequiredError` upfront so unreachable in-evaluate auth branches got cleaned up too.

- linkedin/search.js — JSESSIONID was read inside the per-batch fetch loop's `page.evaluate`; hoisted once before the loop and pass `csrf` value into the template via `JSON.stringify`.

- maimai/search-talents.js — csrftoken cookie hoisted via getCookies; meta-tag fallback preserved inside `page.evaluate` (reached only when no cookie). Also converted the `page.evaluate(async (body) => …, body)` Playwright-style call to OpenCLI's template-string form so the helper actually runs.

- youtube — `SAPISID_HASH_FN` (used by like / unlike / subscribe / unsubscribe) reworked: sapisid is now passed in as a parameter; new `readYoutubeSapisid(page)` helper reads it via CDP. The HMAC-SHA1 compute still happens browser-side (Web Crypto), only the cookie read is hoisted.

Tests updated where mocks specifically referenced `document.cookie` (twitter following / bookmark-folder / bookmark-folders) to mock `getCookies` instead.

Verification:
- `npx tsc --noEmit` clean
- `npx vitest run clis/twitter clis/linkedin clis/youtube` → 264/264 pass
- typed-error-lint 189/189 (no new violations)
- silent-column-drop 103/103 (no new violations)

Scope notes (not in this PR):
- `goto + wait` redundancy and `browserSession: { reuse: 'site' }` rollout are scoped to follow-up PRs B and C per the #OpenCLI:3889b5cf thread plan.
- `document.cookie.match(...)` patterns (instagram 8 / xiaoe / qwen / hupu / tiktok / 1point3acres — ~13 files) are outside the original \`document.cookie.split\` audit scope and will follow as a Tier 1 expansion sweep.

* fix(adapters): read auth cookies by url scope
2026-05-11 01:06:48 +08:00
jakevin 8f0958a295 chore(release): 1.7.15 (#1448)
Release / release (push) Has been cancelled
- bump opencli to 1.7.15 (was 1.7.14)
- extension stays at 1.0.9 (already bumped during the release cycle)
- finalize CHANGELOG: move Unreleased to 1.7.15 with date

Major release: Browser Agent Runtime project (Phase 0/1/2) — alignment
with vercel-labs/agent-browser model. CDP-primary input, AX snapshot/refs
with stale recovery, semantic locators across all primitives, full form
toolbelt (hover/focus/dblclick/check/uncheck/upload/drag/wait-download),
annotated screenshots, and same-origin iframe AX routing.
2026-05-10 23:12:43 +08:00
jakevin aa6696f6ce feat(browser): add annotated screenshot refs (#1433) 2026-05-10 22:15:56 +08:00
jakevin accdd970a4 test(browser): add real Chrome AX smoke (#1445)
* test(browser): add real Chrome AX smoke

* fix(browser): attach cross-origin frame targets directly

* fix(browser): resolve frame target by URL

* test(browser): include frame target URL in AX smoke

* fix(browser): discover iframe targets before routing

* fix(browser): resolve iframe targets through CDP

* fix(browser): auto-attach iframe targets for routing

* test(browser): make cross-origin AX smoke a capability probe

* docs(browser): mark cross-origin AX as best-effort

* ci(browser): keep AX smoke out of normal e2e sweep
2026-05-10 21:07:54 +08:00
jakevin 1364a11ab2 feat(browser): add semantic locators to input actions
Add semantic locator flags to browser type/fill/select while preserving explicit target syntax.
2026-05-10 19:54:55 +08:00
jakevin 3b44f901eb fix(browser): enable AX in cross-origin frame targets
Enable the Accessibility domain inside cross-origin frame target sessions before AX tree fetches and stale ref recovery.
2026-05-10 19:45:56 +08:00
jakevin 19976723c1 feat(browser): route AX refs through cross-origin frames
Route AX snapshot and AX ref click CDP calls through attachable cross-origin frame targets. Bump Browser Bridge extension to 1.0.9 for frame target routing.
2026-05-10 17:28:09 +08:00
jakevin 65903a09ff feat(browser): wait for downloads (#1441) 2026-05-10 17:15:31 +08:00
jakevin bfe7116e82 feat(browser): extend semantic locators to actions (#1440) 2026-05-10 17:01:18 +08:00
jakevin 4e4bef6474 feat(browser): add drag command (#1439) 2026-05-10 16:54:11 +08:00
jakevin 98fcce7bd3 feat(browser): add upload command (#1438) 2026-05-10 16:50:40 +08:00
jakevin 6e1c56e1e6 feat(browser): add check and uncheck (#1437) 2026-05-10 16:32:29 +08:00
jakevin b69b2e384d feat(browser): add hover focus and dblclick (#1435) 2026-05-10 16:18:19 +08:00
jakevin 76b34b7e87 feat(browser): add semantic locator flags (#1434)
* feat(browser): add semantic locator flags

* fix(browser): report semantic read match totals
2026-05-10 16:05:02 +08:00
jakevin 19130ab1af fix(e2e): match fake daemon version to running CLI (#1432)
PR #1399 added auto-restart of stale daemons in BrowserBridge
(daemonVersion ≠ PKG_VERSION → restart). The browser-tabs e2e fake
daemon hard-coded `daemonVersion: 'test'`, so every test reported as
stale and the bridge tried to /shutdown the fake daemon — which has no
shutdown endpoint — causing all 4 tests in the file to exit with code 1.

This has been the failing signal in `e2e-headed (ubuntu-latest)` and
`e2e-headed (macos-latest)` on every main push since #1399.

Read PKG_VERSION from package.json once at module load and feed that to
the fake /status response. The fake daemon now matches the running CLI
so the stale-daemon path is not triggered.

Verification:
- npx tsc --noEmit clean
- npm run build clean
- npx vitest run --project e2e tests/e2e/browser-tabs.test.ts → 4/4 pass
2026-05-10 15:26:52 +08:00
Henry 85ea18c93b feat(dianping): resolve unknown cities live from www.dianping.com (#1429)
* feat(dianping): resolve unknown cities live from www.dianping.com

The static CITY_ID map in clis/dianping/utils.js only covers ~20 cities,
so passing --city 汕头 (or any other Chinese name / pinyin slug not on
that list) fails with ArgumentError. Adding the missing cityIds by hand
doesn't scale to dianping's full city list and silently goes stale when
the site renumbers cities.

This change adds an async resolver that falls back to dianping.com when
the static map misses:

  - Numeric input → pass through unchanged.
  - Static map hit → fast path, no network (utils.CITY_ID untouched).
  - Pinyin slug (e.g. "shantou") → goto /<slug>, parse cityId out of
    any /search/keyword/{id}/ link rendered on the per-city landing page.
  - Chinese name (e.g. "汕头") → goto /citylist, walk anchors to build a
    Chinese-name → pinyin map, then resolve the slug as above.

Resolved (input → cityId) pairs are memoized per-process so repeat
searches skip both navigations.

Implemented as a new module (clis/dianping/cityResolver.js) so utils.js
stays minimal and the existing synchronous resolveCityId / CITY_ID API
keeps working for direct callers and tests.

Tested:
  - Unit tests cover null/numeric/static fast paths, pinyin fallback +
    cache, Chinese-name fallback via /citylist + cache for both forms,
    rejection of garbage input, rejection of Chinese names not on
    /citylist, and CommandExecutionError when the per-city page lacks
    a /search/keyword/{id}/ link.
  - JSDOM tests cover the pure DOM extractors (buildCitylistMap and
    extractCityIdFromPage) against curated HTML fixtures.
  - npm test: 3196 passed, 1 skipped (no new failures).
  - npx tsc --noEmit: clean.
  - opencli validate: 0 errors.

* fix(dianping): require city resolver links to be authoritative

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-10 14:57:44 +08:00
Kagura 962842cd59 fix(douyin): handle empty response body in browserFetch (#1408)
* fix(douyin): handle empty response body in browserFetch (#1405)

browserFetch calls res.json() directly, which throws SyntaxError when
the API returns an empty body (content-length: 0). This happens when
the Douyin hashtag search endpoint returns HTTP 200 with no content.

Fix: read response as text first, return null for empty bodies, then
throw a descriptive CommandExecutionError at the caller level.

Fixes #1405

* fix(douyin): wrap browser fetch parse failures

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-10 14:53:55 +08:00
jakevin 6f200cc744 fix(browser): enable accessibility before AX snapshots (#1417) 2026-05-09 03:12:27 +08:00
jakevin 70981ef06a docs(browser): document AX validation workflow (#1416) 2026-05-08 19:44:38 +08:00
jakevin 3f8b88cf64 feat(browser): compare observation source metrics (#1415) 2026-05-08 19:40:47 +08:00
jakevin d6e3971c79 feat(browser): route AX refs through same-origin frames (#1414) 2026-05-08 19:35:51 +08:00
jakevin e99cbd4a7e feat(browser): add opt-in AX refs (#1413) 2026-05-08 19:20:21 +08:00
jakevin 136e5888ce fix(browser): drive click through CDP mouse events (#1412) 2026-05-08 19:06:03 +08:00
jakevin 53516f7511 docs(browser): design agent runtime roadmap (#1411)
* docs(browser): design agent runtime roadmap

* docs(browser): tighten runtime MVP criteria
2026-05-08 18:35:37 +08:00
jakevin 4475d4efe3 feat(twitter): P1+P2+P3+P4+P5 — search filters, bookmark folders, engagement scoring, sibling dedupe + help docs (#1406)
Round 21 follow-up to #1400 (P0 write-action symmetry, merged `644d4517`). 5 features + help docs unified into one PR per WAWQAQ "全部合成一个 PR" directive.

## Scope

- **P1** (`cf10c098`): `twitter search` `--from / --has / --exclude / --product` filters, mapping to X `from:` / `filter:` / `-filter:` / `f=` operators; legacy `--filter top|live` preserved (--product win on conflict)
- **P2** (`a484a69a`): new `twitter bookmark-folders` + `bookmark-folder <id>`; X Premium GraphQL `bookmarkFoldersSlice` + `BookmarkFolderTimeline`; queryId 三层 fallback (placeholder.json → client-web bundle → pinned constants)
- **P3** (`f209f914`): `--top-by-engagement N` to 7 tweet-shaped read commands (search/timeline/likes/bookmarks/list-tweets/tweets/thread); single helper in `utils.js`; formula `likes×1 + retweets×3 + replies×2 + bookmarks×5 + log10(views+1)×0.5`; **N=0 reference equality no-op** → existing 157 twitter tests 0 churn
- **P4** (`89283fa0`): `TWITTER_BEARER_TOKEN` + composer image helpers extracted to `utils.js` (12 GraphQL adapter dedup); reply hardening; quote adds `--image`
- **P5** (`a3d10a48`): sibling article-scope helper extracted to `shared.js` (9 write commands reuse, dedup with #1400 P0 invariant)
- **docs** (`2a358d80`): help-doc precision (positional-omitted defaults + download/bookmarks/notifications/timeline/lists description thicken; concurrent #1401/#1403 wording preserved)

47 files / +2594/-470. Tests **96 → 216 (+120)**, manifest 798 → 801 (+3), typed-error-lint 190 → 189 (resolved 1 grandfathered sentinel).

## Iteration history (3 review fix commits on top of 6 author commits)

- `7f93779b` — codex-mini1 lead fix1: 3 blocker bundle (P5 host invariant + P2 safe-id + sentinel removal + P1 fallback fail-fast)
- `2a29ecc6` — codex-mini1 lead fix2: P3 help formula consistency (doc/help text matches actual `log10(views+1)×0.5`)
- `df4dcd76` — codex-mini1 lead fix3 (F-P-1 aux catch): P2 `bookmark-folder --limit` upfront validation (`Number(kwargs.limit ?? 20)` + reject non-positive/non-integer + regression `0/negative/fractional/NaN` + `page.goto` zero-call assert)

## 4 progressive blockers caught (codex-mini1 lead 3 rounds + F-P-1 aux 1 round)

1. **P5 host invariant gap** (lead): article-scope helper preserved exact `/status/<id>` path but ignored link host → off-domain `https://evil.com/alice/status/<target>` would satisfy `__twHasLinkToTarget`. Fixed: `https` + X/Twitter host or subdomain + exact `/status/<id>` or `/i/status/<id>` path; query/hash allowed; off-domain/host-suffix/non-https/path-suffix/substring-id rejected; JSDOM positive + 5 negative anchors.

2. **P2 listing→detail round-trip + sentinel** (lead): `bookmark-folders` accepted opaque IDs but `bookmark-folder <id>` only accepted numeric → round-trip broken; new `author: 'unknown'` sentinel created fabricated author URL. Fixed: `[A-Za-z0-9_-]+` opaque safe-id (rejects `/`, `?`, `%`, spaces) + `resolveTwitterQueryId()` sanitization for queryId resolution; sentinel removed → empty author + canonical `/i/status/<id>` URL.

3. **P1 fallback silent tab miss** (lead): pushState fail → fallback typing into search box, `clickProductTabIfNeeded()` silent return on tab not found → user `--product photos` silently degraded to Top results. Fixed: throw `CommandExecutionError` when requested `--product` tab cannot be selected + invalid `--from` / `--limit` upfront pre-nav reject + double-direction tests.

4. **P2 limit silent normalize** (aux): `const limit = kwargs.limit || 20` → `--limit 0` silent → 20; negative/non-integer pre-IO unchecked. Fixed: `Number(kwargs.limit ?? 20)` + require positive integer before `page.goto` + regression covers `0/negative/fractional/NaN` + `page.goto` zero-call.

## Cultural sediment (Round 21 audit checklist 7 rules / 6 dimensions)

This PR **immediately validated 4 of 7 rules** in review pipeline:
- (b) silent-clamp class — P1 fallback silent tab miss (silent semantic-downgrade) + P2 `|| 20` silent normalize
- (e) ID exact-not-substring — P5 host invariant (was only path-exact, not host-exact)
- (f) grandfathered-not-exempt — P5 helper-refactor boundary lost host invariant + P2 new adapter inherited grandfathered `'unknown'` sentinel
- (g) fallback-must-have-success-criterion — P1 fallback path missing post-condition assertion

7 rules / 6 dimensions:
- (a) cross-grep sibling URL pattern — structural
- (b) silent-clamp class — failure mode (input)
- (c) broad querySelector → article-scoping — scope
- (d) missing-validation early reject — boundary
- (e) ID exact-not-substring — identity
- (f) grandfathered-not-exempt (corollary: applies to new file + new helper-refactor boundary; not original-file line-edit) — time-axis
- (g) fallback-must-have-success-criterion (sub-rule g': fallback unit test must include post-condition assertion, not just "doesn't throw") — failure mode (output)

**Cross-PR validation 4-chain on meta-anchor "Structural exactness for identity matching"**:
- #1391 URL layer (`isFacebookAuthRedirectPath`: top-level anchor + `\.php` + `(/|$)` segment edge)
- #1392 URL parser layer (`parseGrokSessionId`: bare UUID exact / URL host-exact-or-subdomain + path-exact)
- #1400 DOM layer (article-scoping: status-id `/\/status\/${id}(?:\/|$)/` regex / segment-array exact)
- #1406 P5 helper-refactor boundary (full URL invariant in shared helper: host+path re-anchored after extraction)
- Common invariant: boundary-lock structural shape; **fuzzy match is silent-failure 温床**; lesson lifecycle = surface-shift not add-and-forget.

**Audit framework self-discipline**: each rule must have grep-able detection signal, otherwise rule degenerates to mantra. Framework is "7 rules + sub-instance pattern in new surface", not frozen 7 rules.

**Round 17 race-mitigation 第 9 连续 race-free execution**: standard alternation cadence (#1400 A 组 → #1406 B 组), lead final + aux final + `@pr-monitor squash?` trigger, pr-monitor proactive ack + serial squash, lead silent on closeout.

## Validation gates (final head `df4dcd76`)

Local: Twitter adapter tests `25 files / 216 tests`, focused P1/P2/P3/P5 tests `99/99`, `node --check` touched runtime, `npx tsc --noEmit`, `npm run build`, manifest 801 entries, typed-error-lint `189/189`, silent-column-drop `103/103`, doc-coverage `140/140`, docs:build clean, listing-id advisory `13` unchanged (wikipedia/trending residual non-Twitter), `git diff --check` clean.

GitHub: build×3 (ubuntu/macos/windows) SUCCESS, unit-test shards SUCCESS, bun-test SUCCESS, adapter-test SUCCESS, audit SUCCESS, doc-coverage SUCCESS, docs-build SUCCESS, smoke-test skipped, PR `CLEAN/MERGEABLE`.

Reviewers:
- Lead: @codex-mini1 (3 fix rounds, all caught proactively + amend P3 help consistency)
- Aux: @First-principles-1 (better-solution triangulation on P2 queryId 三层 fallback + P5 invariant + P3 N=0 reference no-op + caught P2 limit silent normalize)
- Author: @opencli-user (5-feature scope + 7-rule sediment co-author + corollary contributor)
2026-05-08 02:36:39 +08:00
jakevin 34f793ff5c feat(help): add browser structured help (#1404) 2026-05-08 02:15:59 +08:00
jakevin 407b559a83 feat(help): hard-gate empty positional help text + fix 18 offenders (#1403)
Why
- `opencli twitter followers --help` rendered:
    Arguments:
      user
  with a blank trailing column. Both humans and agents could not
  recover the parameter's purpose without reading source. WAWQAQ
  surfaced this directly: "没有说明当后面的 followers [user] [options]
  如果都没填的时候,获取的是什么?"
- This is metadata completeness, not stylistic taste. Failing closed
  is the only way to keep the help surface trustworthy as adapters
  land.

What
- src/build-manifest.ts: add `findManifestMetadataIssues()` that flags
  any positional with empty / whitespace-only / missing `help`. Wired
  into `main()` after the import-failures gate; build aborts non-zero
  with a per-arg report (`site/cmd positional "name" (sourceFile)`).
- src/build-manifest.test.ts: cover the gate (positives + negatives,
  scoped strictly to positionals — named flags are intentionally
  out-of-scope).
- 18 adapter offenders (16 required + 2 optional) get explicit help
  text:
    twitter: followers/following/list-add/list-remove/list-tweets/
             search/thread
    reddit:  search/subreddit/user/user-comments/user-posts
    douyin:  stats/update
    bilibili: subtitle
    jike:    search
  Optional positionals (`twitter followers/following [user]`) now
  document the omit semantics — fetches the currently logged-in
  account.
- CHANGELOG: document the build gate and the offender list.

Out of scope (planned follow-ups)
- Semantic-quality advisory: optional positional help should also
  contain `default / omit / current / logged-in / required unless …`
  keywords. That belongs to the planned Arg metadata v2 work
  (`when_omitted / when_present / value_format` 3-field schema).
- Named-flag `help` quality. Named flags carry the flag name itself
  in help, so a missing `help` is not as opaque; if we want to gate
  those too, do it as a separate, intentional decision.

Validation
- `npm run build`            → 799 entries, clean.
- `npm run typecheck`        → clean.
- `npx vitest run --project unit --project adapter` → 257 + 4 files,
  all green (build-manifest 13 tests, manifest gate added).
- Smoke: temporarily reverted `followers.js` help to empty → build
  aborts with the exact `twitter/followers positional "user" (...)`
  line; restored, build is clean again.
- `npm run check:silent-column-drop` and `check:typed-error-lint`
  baselines unchanged.
2026-05-08 01:55:19 +08:00
jakevin dc7b88d45b chore(release): 1.7.14 (#1402)
Release / release (push) Has been cancelled
- bump opencli to 1.7.14 (was 1.7.13)
- extension stays at 1.0.6 (no extension changes since v1.7.13)
- finalize CHANGELOG with the three landed PRs:
  * #1399 daemon restart on stale ready state for npm -g upgrade
  * #1400 twitter write-action symmetry (unlike/retweet/unretweet/quote)
  * #1401 agent-friendly adapter help (drop globally-shared option noise)
2026-05-08 01:30:04 +08:00
jakevin 0996f9feba feat(help): make adapter help agent-friendly (#1401) 2026-05-08 01:18:56 +08:00
jakevin 644d45177b feat(twitter): add unlike + retweet + unretweet + quote (write-action symmetry P0) (#1400)
Round 21 P0 — Twitter write-action symmetry (4 of 4: unlike, retweet, unretweet, quote).

## Scope
Closes write-action gap with existing siblings (`like`, `bookmark`, `unbookmark`, `delete`):
- `unlike` (UI strategy, navigateBefore:true)
- `retweet` (UI strategy)
- `unretweet` (UI strategy)
- `quote` (UI strategy, `/compose/post?url=` route — same family as `reply.js` `/compose/post?in_reply_to=`)

+745/-0 in initial commit, plus 3 progressive review fixes. Final: 4 adapters + 4 tests; modified `shared.js`, `shared.test.js`, manifest, docs.

## Iteration history (4 heads, 102/102 tests on final)

- `07836783` — initial 4 adapters + 4 tests, 96/96
- `55a89776` — fix #1: shared `parseTweetUrl()` URL invariant + quote post-submit verify (102/102)
- `dc9eab66` — fix #2: article-scoping for unlike/retweet/unretweet (delete.js sibling pattern)
- `8809d2c1` — fix #3: exact status-id matching (`match?.[1] === tweetId`) + quote-card exact id guard

## 4 progressive blockers caught (codex-mini0 lead + F-P-0 aux)

1. **URL validation (silent-clamp class)**: original passed any host containing `/status/<id>`. Fixed: `parseTweetUrl()` requires `https` + Twitter/X exact host + exact `/<user|i>/status/<id>` path; host-suffix, embedded URL, path-suffix all `ArgumentError` pre-nav.

2. **Quote silent-success illusion**: original click-implies-success without composer/toast verify. Fixed: pre-submit quoted-card exact id render assertion + post-submit success toast OR composer-clear assertion, otherwise return failed row.

3. **Broad querySelector scoping (delete.js sibling pattern)**: original state probe + click + post-click verify on conversation pages picked first matching button. Fixed: scope to `article` containing requested exact status id (sibling `clis/twitter/delete.js:22-23` pattern).

4. **Substring vs exact status-id matching**: `/status/123` substring-matched `/status/1234`. Fixed: regex `/\/status\/${id}(?:\/|$)/` segment-edge anchor + `match?.[1] === tweetId` exact compare.

## Cultural sediment (Round 21)

**Audit checklist 5 rules (pre-write upstream selection net)**:
1. cross-grep sibling URL-construction patterns before adopting
2. silent-clamp class detection (any normalize-then-trust path)
3. broad querySelector → article-scoping requirement
4. missing-validation early reject before navigation/IO
5. ID-based DOM/URL matching exact-not-substring

**Augment framing**: Round 21 audit-first 是 Round 18 字面量 self-check 的 **upstream pre-write 阶段**, 两者作用阶段不同, 共存比替换稳。

**Meta-anchor "Structural exactness for identity matching"** unifying:
- URL layer (#1391 isFacebookAuthRedirectPath: `\.php` + `(/|$)` segment edge)
- URL parser layer (#1392 parseGrokSessionId: bare UUID exact / URL host-exact-or-subdomain + path-exact)
- DOM layer (#1400 article-scoping: status-id `/\/status\/${id}(?:\/|$)/` regex or pathname segment-array exact compare)

Common invariant: boundary-lock structural shape, 不 trust substring 模糊 — fuzzy match 是 silent failure 温床。

## Validation gates (final head `8809d2c1`)

Local: Twitter tests 102/102, `node --check` touched files, `npx tsc --noEmit`, `npm run build`, typed-error-lint 189/189, silent-column-drop 103/103, doc-coverage 140/140, docs:build clean, listing-id advisory unchanged 13, `git diff --check` clean, merge-tree clean.

GitHub: build×3 (ubuntu/macos/windows) SUCCESS, unit-test×2 shards SUCCESS, bun-test SUCCESS, adapter-test SUCCESS, audit SUCCESS, doc-coverage SUCCESS, docs-build SUCCESS, smoke-test skipped, PR `CLEAN/MERGEABLE`.

## Strategy/UI boundary (better-solution verdict)

UI write path acceptable for P0 symmetry (matches existing Twitter write siblings). GraphQL write migration + structured `idempotent:true` flag are cross-sibling upgrades, P5 candidate, not P0 blockers.

Round 17 race-mitigation 第 8 连续 race-free execution (this round absorbed author scope-uncertainty hold-then-retract event without producing actual race).

Reviewers:
- Lead: @codex-mini0 (4-round iteration, all blockers caught)
- Aux: @First-principles-0 (better-solution triangulation, scope-discipline verdict, regression invariants)
- Author: @opencli-user
2026-05-08 01:10:54 +08:00
jakevin 8d201ae60b fix(browser): restart stale ready daemon (#1399) 2026-05-08 00:39:19 +08:00
jakevin 1fa44bda6b chore(release): 1.7.13 (#1398)
Release / release (push) Has been cancelled
- bump opencli to 1.7.13 (was 1.7.12)
- bump extension to 1.0.6 (was 1.0.5)
- finalize CHANGELOG: move Unreleased section to 1.7.13 with date,
  document Strategy.HEADER removal + OPENCLI_BROWSER_TIMEOUT rename
  as breaking, add fill-step routing fix, qwen detail command, and
  the dead-code/internal cleanup batch
2026-05-07 23:44:18 +08:00
jakevin bf914f20f1 fix(grok): replace sentinel rows + silent-clamp with typed errors, deliver image cmd (#1397)
fix(grok): replace sentinel rows and deliver image command
2026-05-07 22:45:45 +08:00
jakevin 6f45db1be9 fix(manifest): rescue 11 desktop adapter commands from factory pattern (#1396)
fix(manifest): rescue desktop factory commands
2026-05-07 22:02:12 +08:00
jakevin da833d3efe chore(release): clean stale metadata surfaces (#1395)
chore(release): clean stale metadata surfaces
2026-05-07 22:00:51 +08:00
Kagura abfd0e2180 fix(youtube): use watch page HTML for transcript captions (#1378)
Fixes #1376 — YouTube transcript command failed with `No captions available for this video` for all videos.

## Root cause
Transcript adapter used InnerTube `/youtubei/v1/player` API with Android client context (`clientName: 'ANDROID'`, version `20.10.38`) to retrieve caption track URLs. YouTube has restricted/deprecated this approach; the Android client no longer reliably returns captions data.

## Fix
Replace Step 1 (caption track retrieval) with watch page HTML bootstrap parsing — fetch `/watch?v=...` with cookies and extract `ytInitialPlayerResponse.captions.playerCaptionsTracklistRenderer`. This is the same approach used by sibling `clis/youtube/video.js`, so it's an alignment to existing site-local stable pattern, not a new invention.

## 2 head iteration
- `cf77f5e8` initial fix (Step 1 caption retrieval switch + 18/18 unit tests)
- `bb30788c` lead test hardening — source-contract regression test in `transcript.test.js`:
  - **positive lock**: must fetch `/watch?v=...`, parse `ytInitialPlayerResponse`, read `playerCaptionsTracklistRenderer`
  - **negative lock**: must NOT use `/youtubei/v1/player` or `clientName: 'ANDROID'` (prevents regression)
  - stale Android-InnerTube file header comment also updated

## Better-solution evaluation
- Official YouTube Data API captions surface (`developers.google.com/youtube/v3/docs/captions/download`) is owner-authorized API, NOT a public transcript replacement
- yt-dlp also relies on watch-page bootstrap path
- Existing `youtube/video.js` already uses the same `ytInitialPlayerResponse` extraction → this PR aligns transcript with stable site-local pattern instead of inventing a new path

## Typed failure / no-silent-empty boundaries
- watch HTML HTTP failure / missing `ytInitialPlayerResponse` / no `captionTracks` → `CommandExecutionError` (typed fail)
- Empty parsed XML → `EmptyResultError` (existing path, preserved)
- `Strategy.COOKIE` matches YouTube adapter family + `video.js`; cookies/session/consent unavailable → typed fail not silent empty success illusion

## Diff containment
Runtime change limited to Step 1 caption track discovery. XML fetch, segment parsing, chapters, raw/grouped formatting all unchanged.

## Verification
Local: YouTube adapter tests `19/19` (+1 from new test), `npm run build`, typed-error-lint `192/192`, silent-column-drop `103/103`, doc coverage `140/140`, `docs:build`, listing-id advisory unchanged `13`, `git diff --check`, merge-tree clean.
GitHub: build × 3 OS, unit × 2 shards, bun-test, adapter-test, audit, doc-coverage, docs-build all SUCCESS. PR CLEAN/MERGEABLE.

Author: kagura-agent (fork). Lead: codex-mini0. Aux: First-principles-0. Coordination: pr-monitor.
2026-05-07 21:50:30 +08:00
E2ern1ty 195333ff8a fix(xiaohongshu): improve image publishing — creator-center URL + tab priority + DataTransfer fallback (#1380)
Xiaohongshu image-note publishing reliability fixes for creator center UI (legacy raw-Error write command, not a typed-error migration).

## 3 changes (one publish-path repair)
1. **Open creator publish in image mode**: append `target=image` to the publish URL so it loads directly in image mode instead of default
2. **Exact `图文` tab priority**: prefer exact tab text matching before broad `startsWith/includes`, reducing parent-container misclicks while keeping fallback for UI wording variants
3. **DataTransfer fallback for `Chrome Not allowed`**: when CDP `setFileInput` returns the permission/bridge denial error, fall through to the existing DataTransfer upload path (CDP-first remains primary to avoid base64 bridge/payload limits)

## Lead hardening (`edf8107d`)
Added `clis/xiaohongshu/publish.test.js` regression coverage for all three claimed behaviors:
- `target=image` creator URL locked
- exact tab text matched before broad fallback
- `Chrome Not allowed` falling into DataTransfer path

## Better-solution evaluation (lead + aux 一致)
- **CDP-first kept**: CDP avoids base64 payload/bridge limits; `Not allowed` is a known permission failure class where fallback is appropriate. DataTransfer-first would weaken the common path and reintroduce large-payload fragility.
- **Exact tab text first**: XHS creator markup is private and volatile, selector-only alternative not clearly more stable. Exact text reduces misclicks while broader fallback + post-click `video_surface` check preserve resilience for wording shifts. If exact text disappears, command fails fast with screenshot instead of silent video-mode publish.
- **Scope boundary self-imposed**: not expanding to typed-error migration (publish.js is legacy raw-Error and typed-error-lint already accounts for it).

## Verification
Local: xiaohongshu publish tests `12/12`, typecheck, build/manifest, docs:build, typed-error-lint `189/189`, silent-column-drop `103/103`, doc coverage `140/140`, node --check, git diff --check.
GitHub: build × 3 OS, unit shards, bun-test, adapter-test, audit, docs-build, doc-coverage all SUCCESS. PR CLEAN/MERGEABLE.

Author: E2ern1ty (fork). Lead: codex-mini1. Aux: First-principles-1. Coordination: pr-monitor.
2026-05-07 21:45:08 +08:00
jakevin 3b585fb4d1 feat(grok): add browser chat baseline commands (read/history/detail/new/send/status) (#1392)
Phase 3 — Grok adapter baseline (LLM browser-chat command family, parallel to ChatGPT/Qwen/Yuanbao).

## Surface
6 commands: `status` / `history` / `read` / `detail` / `new` / `send`. Site-local `clis/grok/utils.js` justified by 6 commands sharing helpers, not over-abstraction.

## 4-head review iteration

1. **`b4e81bad`** — initial baseline (12 Grok/shared files)
2. **`0a8112fc`** — mechanical rebase (CHANGELOG conflict only, all 12 Grok files preserved business-equivalent through rebase)
3. **`481e87e2`** — security fix: `parseGrokSessionId()` SSRF-shape vulnerability close — switched from regex string match to `new URL()` parser with branch separation:
   - Bare UUID mode: only exact UUID shape (no URL/query suffix accepted)
   - URL mode: requires `https` scheme + exact `grok.com` or subdomain host + exact `/c/<uuid>` path
4. **`a082023c`** — test-only hardening: 2 additional negative anchors covering existing implementation rejections (bare UUID `?next=abc` query tail / `grok.com.evil.com` host-suffix trick)

## Negative anchor coverage (8 cases)
http / off-domain / fakegrok / host-suffix subdomain / embedded URL / path suffix / UUID-tail / bare query tail

## Better-solution evidence form
LLM browser-chat family pattern (matching ChatGPT/Qwen/Yuanbao baseline) + 5 live probes — not first-site hostile scrape. TipTap editor API send seam (`editor.commands.focus/clearContent/insertContent`) is correct boundary because Grok ignores DOM input events; isolated in `sendMessage()`. Lack of full TipTap mock = residual risk, not blocker.

## Invariants locked
- `parseGrokSessionId()` URL parser branch separation (bare UUID exact / URL exact path)
- `history --limit` rejects invalid/out-of-range
- `status` uses `null` for unknowns (no fabrication)
- Bubble extraction preserves image-only assistant turns (no silent HTML-only drop)
- Auth/empty semantics aligned with LLM browser-chat baseline family

## Verification
Local: Grok adapter tests `28/28`, typecheck, build/manifest, docs:build, typed-error-lint `189/189`, silent-column-drop `103/103`, doc coverage `140/140`, listing-id advisory `13` unchanged, diff-check clean.
GitHub: build ubuntu/macos/windows × unit-test 1/2 + 2/2, bun-test, adapter-test, audit, doc-coverage, docs-build all SUCCESS. PR CLEAN/MERGEABLE.

Lead: codex-mini1. Aux: First-principles-1. Coordination: pr-monitor.
2026-05-07 21:11:39 +08:00
jakevin 9cae777430 chore(release): pre-release P0/P1 cleanup (#1393)
* chore(release): pre-release P0/P1 cleanup

P0 fixes:
- delete src/analysis.ts (179 lines, 0 importers across src/clis/extension)
- remove dead OPENCLI_DIAGNOSTIC negative test assertion
- rename OPENCLI_BROWSER_TIMEOUT to OPENCLI_BROWSER_IDLE_TIMEOUT — the env
  controls workspace lease idle release, not command runtime; old name was
  misleading and undocumented (no fallback needed)
- add 'fill' to validate.ts KNOWN_STEP_NAMES so adapters using PR #1222's
  fill pipeline step do not trip "unknown step name" warnings during validate

P1 fixes:
- BrowserConnect daemon-not-running hint: replace stale "make sure port is
  available" with actionable "run opencli doctor / opencli daemon restart"
- TimeoutError hint: lead with --timeout flag, demote env var to secondary

* fix(validate): derive step allowlist from pipeline registry

@pr-monitor flagged the prior "add 'fill' to KNOWN_STEP_NAMES" fix as
treating only the symptom — two parallel hand-maintained lists will keep
drifting whenever a new pipeline step is registered.

Address the root cause: pipeline/registry.ts now exports
`getRegisteredStepNames()` and validate.ts builds KNOWN_STEP_NAMES from
that. Adding a step via `registerStep()` automatically allowlists it.

* test(validate): regression guard for pipeline step allowlist linkage

@pr-monitor follow-up: lock the validate ↔ pipeline registry linkage at
the test layer so future drift is caught immediately.

Changes:
- recompute KNOWN_STEP_NAMES per-call (was const at module load) so
  steps registered after validate.ts import (plugins, dynamic registration)
  are honoured
- add src/validate.test.ts with 3 cases:
  1. every step name from getRegisteredStepNames() exists
  2. an adapter using every currently registered step does not warn
  3. a step registered at runtime is automatically allowlisted by
     validate without any source change to validate.ts

* fix(capabilityRouting): add fill to BROWSER_ONLY_STEPS

Same double-list drift pattern as validate.ts KNOWN_STEP_NAMES (audit
follow-up flagged in this PR's evolution thread). The fill step was
registered in pipeline/registry.ts (PR #1222) but never added to the
browser-only allowlist in capabilityRouting.ts.

Concrete impact:
- shouldUseBrowserSession() didn't recognize a `[{ fill: ... }]` pipeline
  as needing a browser, so PUBLIC adapters using fill could end up
  without a page and crash inside stepFill at `page!.fillText(...)`
- pipeline/executor.ts's per-step retry policy (BROWSER_ONLY_STEPS gets
  2 retries on transient errors, others get 0) skipped fill — losing
  retry coverage on a DOM-touching step

Fix:
- add 'fill' to BROWSER_ONLY_STEPS
- add a documenting comment explaining BROWSER_ONLY_STEPS is the
  browser-touching subset of registered steps (not the full set)
- export _validateBrowserOnlyStepsAgainstRegistry() so the test layer
  catches the inverse drift (browser-only step that no longer exists)
- 3 new tests in capabilityRouting.test.ts:
  * pipeline with fill routes to browser session
  * BROWSER_ONLY_STEPS subset of registered step names
  * fill is in both lists

This addresses @pr-monitor follow-up #3 (audit similar double-list
patterns) for the obvious in-scope candidate. Other candidates outside
this PR's scope: build-manifest serialization vs registry shape, error
code unions vs lint baselines.

* test(validate): use Strategy.PUBLIC enum instead of string cast in regression test

Self-review nit: `strategy: 'public' as never` worked but bypassed the
typed CliOptions union. Use `Strategy.PUBLIC` so the test exercises the
real public API.
2026-05-07 21:11:34 +08:00
jakevin b2ebe211d1 feat(yuanbao): add browser-web baseline commands (status/read/detail/history/send) (#1394)
Wire up the standard browser-LLM command surface for Yuanbao, matching the
recently shipped chatgpt + claude + qwen baselines:
- status — login + current model + (agentId, convId) + URL
- read   — render the visible conversation as User/Assistant rows
- detail — open `<agentId>/<convId>` and read its messages
- history — list sidebar conversations with stable IDs
- send   — fire-and-forget, returns once the send button has been clicked

Refactor `ask.js` to share helpers (`sendYuanbaoMessage`, `normalizeBooleanFlag`)
with the new commands via `shared.js`, keeping the public ask behavior intact.

Notable bits:
- `parseYuanbaoSessionId` accepts only full chat URLs or `<agentId>/<convId>`
  pairs — Yuanbao chat URLs encode both, and silently opening the wrong agent
  on a bare UUID is a worse failure mode than throwing. URL regex anchored
  with `(?:[/?#]|$)` so 37+ char tails reject rather than truncate.
- `sendYuanbaoMessage` polls the send button (up to 3s) for the React
  re-render that drops `style__send-btn--disabled___*` after composer input —
  a fixed wait raced the debounce and produced silent no-op clicks.
- `getYuanbaoMessageBubbles` uses `data-conv-id`/`data-conv-idx`/
  `data-conv-speaker` attributes for stable per-turn identity (was relying
  on innerHTML alone).
- Status surfaces both human label (`Yuanbao`) and `dt-model-id`
  (`hunyuan_gpt_175B_0404`) — sentinel strings would silently look like a
  real model name; null is the typed-unknown signal.

Verified: 25 unit tests pass; targeted live smoke for status/read/detail/
history/new/send + ask round-trip on yuanbao.tencent.com.
2026-05-07 20:38:17 +08:00
jakevin b9b87a5c64 refactor(facebook/notifications): pipeline→func + typed errors + 7-col contract + runtime upfront limit (Phase 3 P5, #1391)
First Facebook adapter — Pattern C HTML scrape (lead 5 + author 4 = 7 endpoint family probe matrix dual-source negative evidence: graphql×3 / m.facebook redirect / login.php / checkpoint.php / fetch-patch / Messenger relay / ajax legacy 全 unauth 不可达, DOM walk over rendered notification rows + path-anchored auth detection 是当前 reviewable boundary).

Caller-visible delta: 3 cols (index/text/time) → 7 cols (+unread/+url/+notif_id/+notif_type).

[Bug fix] — 5 silent failures resolved
- silent-bad-shape: text.substring(0,150) → full body via per-row 'Mark as read' aria-label
- silent-bad-shape: time || '-' sentinel → string|null typed unknown
- silent-column-drop: unread badge / anchor href / notif_id / notif_t 暴露
- silent-empty-row: /login(.php)? + /checkpoint(.php)? redirect 返 [] → AuthRequiredError; empty/no-recoverable-text → EmptyResultError
- silent-clamp: limit 越界 silent clamp → ArgumentError (1-100), upfront before any navigation (navigateBefore: false)

[Structural refactor]
- pipeline → cli() func form + Strategy.COOKIE + navigateBefore: false (runtime upfront invariant 与 #1387 standard 拉齐)
- module-level pure exports: normalizeNotificationsLimit, stripMarkAsReadPrefix, stripAnchorChrome, parseNotifQuery, extractNotificationRowsFromDoc, isFacebookAuthRedirectPath, buildNotificationsScript
- Live IIFE 通过 \${fn.toString()} 嵌入 (dianping #1313 / hupu #1387 / xiaoe #1388 lineage)
- Locale 表 6 prefix / 4 badge label 显式列出
- AUTH_REQUIRED: sentinel → Node-side AuthRequiredError mapper

[Typed-error hardening]
- Path-anchored auth helper: isFacebookAuthRedirectPath(/^\/(?:login|checkpoint)(?:\.php)?(?:\/|\$)/i) — domain-invariant-first encoding (FB top-level auth-only invariant), 排除 /loginhelp /help/login /account/login/identify
- Three-layer navigateBefore=false invariant lock: registration assertion + manifest absence + executeCommand runtime page.goto-zero-call (test layer 与 invariant layer 完整对齐)
- Row-level silent-empty-row defense: anchor rows with no recoverable body text 直接 skip, 不 emit text:null success row

[Doc fix]
- docs/adapters/browser/facebook.md notifications enrichment + Output table (列类型 / null vs sentinel 语义) + auth/empty error contract
- Boy Scout audit: cross-checked profile / feed / search / marketplace-listings / marketplace-inbox 例 commands 与 args 定义一致

Tests
- notifications.test.js 39/39 + src/execution.test.ts 21/21
- Anti-pattern regression guards: not.toMatch(/text\.substring\(0,\s*150\)/) + not.toMatch(/time\s*\|\|/)
- JSDOM frozen-fixture (slim 13 lines, 0 blank): header listitem skip / full text / unread badge / query parsing / null time / blank-row skip / relative href absolute / 19-case auth path matrix
- typed-error-lint baseline 192 → 191 (silent-sentinel resolved 1)

Review iterations (4 head, A 组 codex-mini0 lead + First-principles-0 aux):
1. 052d2b18 (initial 29 tests) → 376cb50f (lead gate fix: Ubuntu lint + auth path-segment + anchor.href + 5 typed-error func tests)
2. 376cb50f → 0d6c1340 (pr-monitor grep cross-verify catch /login.php false-negative; lead 加 \\.php 边界)
3. 0d6c1340 → 3e5a5ff0 (opencli-user 19-case 实测 + lead 抽 named helper isFacebookAuthRedirectPath domain-invariant-first encoding + 2 row-shape silent-failure 顺手 catch)
4. 3e5a5ff0 → 36e44f73 (F-P-0 aux blocker: registry-injected navigateBefore 在 limit validation 之前 fire pre-nav 违反 #1387 upfront boundary; navigateBefore:false + 三层断言 registration/manifest/runtime executeCommand)

Closes #1391
2026-05-07 17:49:57 +08:00
jakevin 381f095706 feat(qwen): add detail command + fix stale message bubble selector (#1390)
* feat(qwen): add detail command + fix stale message bubble selector

`getMessageBubbles` was matching `[data-msgid="<id>-question|answer"]` from an
older Qianwen frontend. The reshipped DOM no longer carries that attribute on
chat turns; `[data-message-id]` now lives on citation cards inside assistant
responses, so the old selector silently returned an empty list and `qwen read`
had been silently broken.

Rewire to walk `[data-chat-question-wrap]` and `[data-chat-answers-wrap]` in
DOM order (correct Q/A interleaving) and synthesize stable IDs from the
nearest sibling `data-req-id` so `waitForAnswer.seenAssistantId` and
read/ask/detail dedupe paths keep working. Verified live against an existing
conversation: 3 user turns + 3 assistant turns extracted; old selector
returned 0.

`qwen detail <id|url>`: open a specific conversation by ID or full chat URL,
poll up to 20s for the transcript to render, return Role/Text rows. Adds
`parseQianwenSessionId` (5 unit tests covering ID/URL parsing + ArgumentError
on malformed input). Reuses the same site-level browser session as `read`/
`ask` so consecutive calls continue in the same Qwen tab.

- clis/qwen/detail.js (new)
- clis/qwen/utils.js (parseQianwenSessionId + getMessageBubbles rewire)
- clis/qwen/utils.test.js (new)
- docs/adapters/browser/qwen.md (detail entry + options/columns)
- cli-manifest.json (regenerated)

* fix(qwen): anchor URL regex to reject 33+ hex tail truncation

codex-coder review on PR #1390 caught that
`https://www.qianwen.com/chat/<33+ hex>` would silently truncate to the
first 32 chars and open the wrong conversation. Adds end-of-input /
slash / query / fragment boundary to the URL match group and two new
unit-test cases (digit tail + letters tail) covering the truncation gap.
2026-05-07 17:39:23 +08:00
jakevin 99986c3101 feat(chatgpt): add browser chat baseline commands
Add ChatGPT web ask/send/read/history/detail/new/status alongside existing image support. Tighten ChatGPT web helper selectors and typed error contracts, update docs/changelog, regenerate manifest, and seed local ChatGPT verify fixtures for ask/read.
2026-05-07 17:37:18 +08:00
jakevin 6f7eb6a76a refactor(xiaoe x3): pipeline→func + typed errors + content silent-drop fix (Phase 3 P1)
Phase 3 P1 (xiaoe catalog/courses/content) — pipeline→func refactor + typed-error hardening + content silent-drop bug fix + URL upfront validation + inherited legacy doc fix。

## Tags (PR body honesty 演进 dual-nature framing 试用)

- **[Bug fix]** `xiaoe/content` silent-column-drop (caller-visible delta)
- **[Structural refactor]** `xiaoe/catalog` + `xiaoe/courses` pipeline→func 包壳 (parity by construction, IIFE 字节级保留)
- **[Typed-error hardening]** 三 func `page.goto` + `page.evaluate` failure 包成 `CommandExecutionError`; `content/catalog` URL upfront `ArgumentError` (missing/malformed/non-https/off-domain) before navigation
- **[Doc fix]** `docs/adapters/browser/xiaoe.md` `courses --limit 10` (legacy doc 错误 inherit) + `--url` wording → 实际 positional `url` (manifest aligned)

## Per-tag detail

### [Bug fix] content silent-column-drop (real caller-visible bug)
adapter 名"提取小鹅通图文页面内容为文本", IIFE 返 `{title, content, content_length, image_count, images}`, 但 columns 只声明 `[title, content_length, image_count]` → `content` (那段文本本身) 被 silent drop。**用户拿到 "1234 chars" 但拿不到那 1234 chars** — adapter 名字撒谎了。
- Fix: 公开列 `[title, content, content_length, image_count]`, `content` 真 caller-visible delta
- Choice A (vs B reshape): legacy `images` 是 `JSON.stringify(slice(0, 20))` 截断/stringified 坏合同, **不暴露成新列** (避免把 silent-bad-shape 升级成公开坏合同), 留 follow-up 另开 explicit media/images contract
- `image_count` 用 `countXiaoeImages(doc)` 全页计数, 不 slice (既有 metadata 质量修正)

### [Structural refactor] catalog + courses pipeline→func wrapper (parity by construction)
- `pipeline:[]` form → `func` form
- IIFE body 字节级保留 (Xiaoe 没 public REST, Vue 私有 runtime 是唯一稳定 hook, JSDOM 复刻不了 Vue tree)
- Pure helpers extracted: `pickContentText`, `countXiaoeImages` (content) / `typeLabel`, `buildItemUrl`, `chapterUrlPath` (catalog) / `buildCourseUrl` (courses)
- IIFE 通过 `\${fn.toString()}` 嵌同一份代码 (dianping #1313 / hupu #1387 同模式)
- No live verify acceptable: IIFE 字节级保留 + helper 全 unit-test + manifest column shape 不变 = 行为 parity by construction
- `buildScript` 反向断言 `images.slice(0, 20)` legacy anti-pattern 不出现 (anti-pattern regression guard, 同 #1387 `documentElement.outerHTML` 反向 guard)

### [Typed-error hardening] 三 func navigation + evaluate boundary
- `requireXiaoePageUrl()` for `content/catalog`: missing/malformed/non-https/off-domain URL → upfront `ArgumentError` before `page.goto` (test asserts `expect(page.goto).not.toHaveBeenCalled()`)
- `content/catalog/courses`: `page.goto` moved inside try, navigation/evaluate failures both wrap as `CommandExecutionError`, no raw CDP/browser error path leaks
- Empty shell stays `EmptyResultError` (no reliable login-wall signal to justify `AuthRequiredError`, 避免 false positive — 应用 #1384 secUid 教训)

### [Doc fix] inherited legacy doc errors
- `xiaoe courses --limit 10` example removed (no `--limit` arg in manifest, legacy doc 错误 inherit)
- positional `url` wording aligned with manifest (was incorrectly `--url`)
- 同 #1386 positional docs 教训, 但延伸到 "继承 legacy doc 错误也是新 PR 责任" (Boy Scout typed-error hardening 在 doc 层延伸)

## Tests: 46/46 green
- 3 cmd registration contract
- pure helper unit tests (selector chain / image filter / URL priority / type label fallback / no synthetic URL)
- `buildScript` invariants (`images.slice(0, 20)` 反向断言)
- wire tests: ArgumentError upfront (BEFORE page.goto), EmptyResultError empty rows + empty content, CommandExecutionError navigation/evaluate failure, rows verbatim happy path

## Lint gates
- typed-error-lint 190/190 (no new) ✓
- silent-column-drop 103/103 (no new) ✓ (注: `pipeline:[]` IIFE string template AST walker 看不进, lint follow-up scope)
- doc-coverage 140/140 ✓
- listing-id-pairing advisory unchanged 13 ✓

## GitHub checks (head a6d37d70)
build ×3 / unit-test ×2 / bun-test / adapter-test / audit / doc-coverage / docs-build SUCCESS, smoke skipped, MERGEABLE / CLEAN

## Review
B 组: @codex-mini1 lead + @First-principles-1 aux, double-green confirmed, Round 17 race-mitigation 第 4 轮 protocol clean closeout (第 4 次连续无 race 执行: #1384 / #1386 / #1387 / #1388)。

## Sediment lessons
- Silent-failure 三类 taxonomy: silent-column-drop (列没声明) / silent-bad-shape (字段在但 shape 错) / silent-empty-row (错误状态返空行而不是抛 typed error) — 三类 fix 路径不同, blast radius 不同
- PR body honesty 演进 4 链: #1384 R4 race disclosure → #1386 positional docs 教训 → #1388 silent-failure 三类分开写 + dual-nature tag 矩阵
- F-P-1 first-principles call: 不顺手暴露 legacy 坏合同 (silent-bad-shape ≠ silent-drop, fix 路径完全不同)
2026-05-07 17:02:23 +08:00
jakevin e610260705 refactor(hupu/hot): pipeline→func + querySelectorAll + 4 enrichment columns (Phase 3 P3)
Phase 3 P3 (hupu/hot) — pipeline→func refactor + 2 真 bug 修 + 4 列 enrichment + JSDOM-frozen-fixture test pattern (#1313 复用) + anti-pattern regression guard。

## Summary
- Pipeline form (`pipeline:[]` + `documentElement.outerHTML` regex) → `func` form (`querySelectorAll('.t-info')` DOM walk)
- **Bug 1 修**: outerHTML regex 静默漏行 (markup 抖动就漏, mocked test 抓不到)
- **Bug 2 修**: regex 抓所有 9-digit 锚点 → ~70 个 anchor 但页面只 render 60 个 `.t-info` row → legacy adapter 每次返 ~10 个 phantom 行 (导航链接 conflated 成 thread 行)
- **4 enrichment columns** (4→8): `lights` (亮 count int|null, 万 expanded `1.2万→12000`) / `replies` (回复 count int|null) / `forum` (per-row sub-section) / `is_hot` (bool 暴露 hupu \" hot\" marker, 不 filter 行序保持页面顺序)
- columns/manifest/docs sync: `[rank, tid, title, lights, replies, forum, is_hot, url]`,`null` vs `0` 语义清楚

## Typed errors
- `--limit` 上游 `ArgumentError` for 0/-1/>100/1.5/non-numeric (BEFORE `page.goto`,**不 silent clamp**)
- 空页 `EmptyResultError`
- `page.evaluate` failure 包成 `CommandExecutionError` (test regression locked)

## JSDOM frozen-fixture test pattern (#1313 复用)
- 抽 `extractHupuHotRowsFromDoc(doc, limit, parseCount)` 为 module-level pure export
- in-page IIFE 通过 `\${fn.toString()}` 嵌同一份代码
- JSDOM test 直接调 export against `__fixtures__/hot-home.html` (slim 6-row hand-crafted fixture)
- 17/17 tests green (contract / normalize / parseCount / extract / buildHotScript invariants / wiring / phantom-anchor exclusion / evaluate-error envelope)

## Anti-pattern regression guard (#1313 fixture pattern 延伸)
- `buildHotScript` 反向断言 `not.toContain('documentElement.outerHTML')` 锁不回退到旧 broad regex
- `buildHotScript` 反向断言 `not.toContain('regex.exec')` 同向锁
- fixture 顶部 `.t-info` 外的 9-digit phantom anchor `639999999` 反向锁: 旧 broad regex 会抓到, 新 `.t-info` extractor 不抓 — 把 fixture 反向验证从断言层升到证据层

## Better-solution check (live probe evidence-based)
DOM `.t-info` = 60 visible rows, `window.\$\$data.pageData.threads` = 70 (10 hidden/non-rendered)。对"首页可见 hot rows" 任务, DOM walk 比 bootstrap JSON 更贴 source of truth (后者会引入 hidden/不渲染条目)。这条 60 vs 70 数字是设计决策的硬 justify, 不是设计意见。

## Lint gates
- typed-error-lint 190/190 (no new) ✓
- silent-column-drop 103/103 (no new) ✓
- doc-coverage 140/140 ✓
- listing-id-pairing advisory unchanged 13 ✓

## GitHub checks (head 874d4e4e)
build ×3 / unit-test ×2 / bun-test / adapter-test / audit / doc-coverage / docs-build SUCCESS, smoke skipped, MERGEABLE / CLEAN

## Review
A 组: @codex-mini0 lead + @First-principles-0 aux, double-green confirmed, Round 17 race-mitigation 第 4 轮 protocol clean closeout.
2026-05-07 16:59:17 +08:00
jakevin 464de7059e refactor(tiktok): write commands -> button-walker Route 1 with typed errors (Phase 3 P0.5)
Phase 3 P0.5: refactor 3 TikTok write commands (comment, follow, unfollow) from time-window-wait UI flow to a button-walker + state-verification path with a typed-error boundary, sharing a parallel helper structure to the #1384 read PR.

Two-layer helper boundary (clis/tiktok/utils.js extension):
- BUTTON_WALKER_HELPERS (browser side): button-walker (locate / pre-click state read / click / state-verify post-click) + cleanText reuse + cookie/auth-secUid plumbing for write-auth + plain Error throws on contract violations
- throwButtonWalkerError() (Node side): map browser-thrown errors -> typed CommandExecutionError (button missing / state-verify fail / captcha / rate-limit / navigation/eval/empty-row defensive failures) / AuthRequiredError (cookie + viewer secUid) / ArgumentError (upfront input validation). Explicitly NO EmptyResultError mapping (button contract violation is not an empty result, per #1384 R4 lesson on auth-vs-empty classification).

Per command:
- comment <video-url> <text>: button-walker click + state-verify by checking comment-list state (not wait-2s)
- follow <username>: pre-click state read distinguishes idempotent fast path (`already-following` / `already-friends`) from post-click success (`followed`). Post-click result causality preserved (post-click never returns `already-*`).
- unfollow <username>: pre-click `already-not-following` fast path; post-click `unfollowed`.

result enums (per row):
- comment: `posted` (no idempotent path - comments cannot dedupe)
- follow: `followed` | `already-following` | `already-friends` (last two pre-click only)
- unfollow: `unfollowed` | `already-not-following` (last one pre-click only)

retryable contract (in hint string `retryable=<bool> reason=<...>`):
- comment failures: retryable=false reason=server-fan-out
- follow/unfollow failures: retryable=true reason=idempotent (server-side dedupe is safe)

Lead push iterations during review (codex-mini1 maintainer-fixes-directly):
- f5730f16: rate-limit/captcha -> CommandExecutionError + retryable hint BEFORE auth regex (auth precedence bug); follow post-click success -> `followed` (NOT `already-friends`, fixing causality misclassification); navigation/empty-row defensive failures route through throwButtonWalkerError (containing raw Error leakage).
- b683f46c: parseTikTokVideoUrl() requires canonical /@user/video/<numeric-id> with only optional trailing slash/query; malformed suffixes (e.g. /123abc, extra path) -> upfront ArgumentError.
- f5dc91d6: docs examples updated to actual positional args for write commands (was stale --url/--text/--username flag form), covering write-rewrite + sibling like/unlike/save/unsave on touched docs file (Boy Scout).

Intentionally NOT addressed (separate scope, candidate post-merge follow-ups):
- Direct /api/commit/follow/user/ or /api/comment/publish/ (would require X-Bogus signing reverse engineering, separate risk surface)
- RetryableError as core typed-error metadata (currently encoded in hint string, post-merge candidate to import into engine)
- TikTok Studio creator metrics commands (separate Phase scope)

Validation:
- clis/tiktok/ tests: 64/64 (38 read from #1384 + 22 new write contract + 4 regression for blockers caught during review)
- typed-error-lint: 190/190
- silent-column-drop: 103/103
- doc-coverage: 140/140
- listing-id advisory: 13 unchanged
- docs:build pass, manifest 764 entries
- GitHub gates on f5dc91d6: build x3 / unit x2 / bun / adapter-test / docs-build / doc-coverage / audit all SUCCESS, smoke skipped, CLEAN/MERGEABLE

Reviewers: codex-mini1 (lead, 3 contract pushes f5730f16 -> b683f46c -> f5dc91d6), First-principles-1 (aux, validated 4 contract patches + better-solution check confirming button-walker Route 1 vs /api/commit/* + X-Bogus separation).
2026-05-07 16:33:00 +08:00
jakevin 9a7dd44b3e refactor(tiktok): 6 read commands -> page-context API (Phase 3 P0, absorbs #1382)
Phase 3 P0: refactor 6 TikTok read commands (explore, following, friends, live, notifications, user) from DOM/network-intercept to TikTok web's own page-context API endpoints, sharing one helper boundary.

Helper boundary (clis/tiktok/utils.js):
- BROWSER_HELPERS: in-browser fetchJson + cleanText + asNumber (null/'' -> null preserve missing-vs-zero distinction) + cookie/msToken plumbing
- VIDEO_ITEM_NORMALIZER: normalize page-context item -> row shape
- assertTikTokApiSuccess(data, label): unify TikTok in-band envelope (status_code/statusCode != 0; code 8 or auth-looking message -> AUTH_REQUIRED; other -> upstream label API failed)
- throwTikTokPageContextError() (Node side): map browser-thrown errors -> AuthRequiredError / EmptyResultError / CommandExecutionError

Per command:
- explore: /api/recommend/item_list/ pagination, --limit upfront ArgumentError
- following: /api/user/list/ relationships
- friends: /api/user/list/ + cross-filter
- live: /api/live/discover/ feed
- notifications: /api/notice/multi/ (status 8 -> AUTH_REQUIRED)
- user (absorbed from #1382): secUid resolve via __UNIVERSAL_DATA_FOR_REHYDRATION__ -> /api/user/detail/, /api/post/item_list/ pagination, /api/search/general/full/ exact-author fallback. !secUid -> EmptyResultError (NOT AuthRequiredError; auth still covered by HTTP 401/403 + envelope status_code 8/auth-looking msg). source field = bootstrap | profile-api | search-fallback in row/columns/manifest/docs/tests.

Closes #1382 (absorbed; #1382 closed without separate merge per WAWQAQ direction).

Validation:
- clis/tiktok/ tests: 38/38
- typed-error-lint: 190/190
- silent-column-drop: 103/103
- doc-coverage: 140/140
- docs:build pass, manifest no drift
- GitHub gates: build x3 / unit x2 / bun / adapter-test / audit / doc-coverage / docs-build all SUCCESS, smoke skipped, MERGEABLE

Reviewers: codex-mini0 (lead, push 4 boundary fixes 18cdf930 -> a1f1ada4 -> 53499609 -> 276dce3b), First-principles-0 (aux, caught secUid auth-vs-empty boundary + verified 6 cmd integral helper boundary).
2026-05-07 16:15:30 +08:00
jakevin b327da5b3c feat(llm): reuse browser sessions by site (#1385) 2026-05-07 15:49:25 +08:00
yorick 1b113a60bc pass example field through cli() registration (#1381) 2026-05-07 15:34:32 +08:00
jakevin fa7851bb9a feat(browser): add adapter session reuse (#1383) 2026-05-07 15:24:54 +08:00
Benjamin Liu d527571b7d test(gov-policy): JSDOM-against-frozen-fixture tests for in-browser extractors (#1340)
* test(gov-policy): JSDOM-against-frozen-fixture tests for in-browser extractors

Applies the pattern documented in skills/opencli-adapter-author/references/jsdom-fixture-pattern.md
(introduced in #1319 alongside the dianping reference test in #1313) to the
gov-policy adapter.

Refactor: the inline IIFE inside `page.evaluate` template literal is hoisted
to a top-level `extractSearchRows` / `extractRecentRows` function using bare
`document` / `location`. Same code now runs identically in:

  - the live browser (injected via `${extractor.toString()}`)
  - JSDOM unit tests (with `globalThis.document` / `globalThis.location` swapped)

Tests:

  - 6 new cases in clis/gov-policy/gov-policy.test.js (was commands.test.js).
  - 3 representative search result cards (1 with real article snippet, 2 with
    only publish-time in `.description`) and 5 recent listing rows in the
    fixtures.
  - ok:false fallback path covered for both extractors.
  - Lock-in: `要闻` type-tag prefix fusion in title and empty-source contract
    on recent listings (no `.source` / `.from` elements on that page) are
    asserted explicitly so a future selector tweak can't silently change them.

Reverse-validated against two buggy variants per the reference doc:
breaking the title selector and stripping the `要闻` prefix both fail the
JSDOM assertions with helpful diffs.

Fixture sanitization follows the reference doc step-by-step: scripts /
styles / iframes / comments / preload links stripped, image srcs replaced
with `placeholder.png`, trimmed to the minimum subtree that exercises the
extractor (3 search items, 5 recent rows), all whitespace-only lines
removed.

* fix(gov-policy): use typed errors for touched commands

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-07 12:46:13 +08:00
jakevin c6d5da54ee feat(web): add exhaustive same-origin frame mode (#1373) 2026-05-07 01:15:10 +08:00
jakevin 124adf73d1 fix(web): avoid duplicate iframe diagnostics (#1372) 2026-05-07 00:58:42 +08:00
jakevin 829edfea3a fix(web): include relevant iframes outside main content (#1371) 2026-05-07 00:44:52 +08:00
jakevin 67cde0e263 enrich(coupang): product detail cmd + replace silent clamp/sentinel/Error with typed errors (#1370)
* enrich(coupang): add product detail cmd + replace silent clamp/sentinel/Error with typed errors

Two enrichment changes plus three silent-failure fixes on top of existing
search / add-to-cart.

New cmd: coupang product
─────────────────────────
Pairs with search as the listing↔detail round-trip target. Reads a logged-in
product page and extracts a single canonical row with price, original_price,
discount_rate, rating, review_count, seller, brand, rocket, delivery_promise,
image_url, url. Three-source extractor (JSON-LD Product schema → bootstrap
globals → DOM) merged in priority order, mirroring the search.js pattern.

The columns use string|null typing — null means "upstream did not provide
this field on this product" (e.g. some items have no original_price).
Failures (login wall / page mismatch / page failed to render) raise typed
errors instead of silently returning empty rows, so callers can treat any
returned row as real data.

Search column shape: added product_id
─────────────────────────────────────
Listing must pair with detail by id. The data was already extracted by
normalizeSearchItem; only the columns array needed updating so the field
projects through to the rendered row. Per the listing-id-pairing convention
(PR #1297) the new column lets agents round-trip rows directly into
`coupang product` without re-scraping URLs.

Silent-failure fixes
────────────────────
1. search --limit silent clamp.
   Old: `Math.min(Math.max(Number(kwargs.limit||20),1),50)` silently
        rewrote `--limit 999` to 50 and `--limit 0` to 1.
   New: `parseLimitArg(raw, 20, 50)` throws ArgumentError on out-of-range
        / non-integer / negative input. Same convention as the typed-fail-fast
        memory & PR #1289.

2. search --page silent clamp.
   Old: `Math.max(Number(kwargs.page||1),1)` silently lifted negative pages.
   New: parsePageArg throws ArgumentError on non-positive input.

3. Generic `throw new Error(...)` → typed errors.
   - Empty query, unsupported --filter, missing --product-id/--url
     → ArgumentError
   - Login wall detection → AuthRequiredError('coupang.com', ...)
   - Empty result / filter-not-rendered → EmptyResultError
   - PRODUCT_MISMATCH / OPTION_REQUIRED / button-not-found / unknown
     ack failure (add-to-cart) → CommandExecutionError
   - The PRODUCT_MISMATCH and `actualProductId || 'unknown'` sentinel were
     also fixed (silent-sentinel was the audit hit there).

Coverage
────────
- 21 contract assertions in clis/coupang/coupang.test.js covering
  parseLimitArg / parsePageArg (no silent clamp), registry shape (search has
  product_id, product is read-class with expected columns, add-to-cart is
  write-class), and typed-error pre-flight rejections (empty query / bad
  filter / out-of-range limit & page / missing detail args).
- Manifest 763 → 764 (+1 entry: coupang/product).
- Audits: typed-error-lint 196 → 194 (resolved 2 silent-clamp/sentinel
  baseline entries; baseline updated). silent-column-drop 103/103 unchanged.

* fix(coupang): tighten product id and browser errors

* fix(coupang): require real product urls
2026-05-07 00:18:57 +08:00
jakevin a5a3248a77 refactor(linux-do): remove deprecated hot/category/latest compat shims (#1368)
* refactor(linux-do): remove deprecated hot/category/latest compat shims

The three shims have been pure backward-compat wrappers since linux-do/feed
became the unified entrypoint. With no stable release commitment to preserve,
they are pure surface cost: 3 manifest entries, 3 deprecated branches in help
output, and a `buildLinuxDoCompatFooter` helper that exists only to feed them.

- delete clis/linux-do/{hot,category,latest}.js
- drop now-orphaned `buildLinuxDoCompatFooter` from feed.js and unexport
  `executeLinuxDoFeed` (no external consumers remain)
- remove the Compatibility section in docs/adapters/browser/linux-do.md
- regenerate cli-manifest.json (-125 lines)

BREAKING CHANGE: `opencli linux-do hot|category|latest` are removed. Use
`opencli linux-do feed --view top --period <period>`,
`opencli linux-do feed --category <id-or-name>`, and
`opencli linux-do feed --view latest` instead.

* fix(linux-do): finish compat shim removal
2026-05-06 23:57:15 +08:00
jakevin dcaae37068 refactor(registry): remove dead adapter metadata (#1369)
* refactor(registry): remove dead adapter metadata

* docs(changelog): note header strategy removal
2026-05-06 23:49:39 +08:00
637 changed files with 62343 additions and 8487 deletions
+3 -3
View File
@@ -9,11 +9,11 @@ outputs:
runs:
using: composite
steps:
- name: Install real Chrome (stable)
uses: browser-actions/setup-chrome@v1
- name: Install real Chrome for Testing
uses: browser-actions/setup-chrome@v2
id: setup-chrome
with:
chrome-version: stable
chrome-version: latest
- name: Verify Chrome installation
shell: bash
+5 -1
View File
@@ -110,8 +110,12 @@ jobs:
- name: Run unit tests under Bun
run: bun vitest run --project unit --reporter=verbose
# Adapter tests are pure unit tests — OS doesn't affect results.
# Adapter tests are pure unit tests — OS doesn't affect results. Gated off
# `pull_request` to keep PR CI under ~2 minutes; adapter authors run focused
# tests locally before pushing, and `push` to main / nightly cron / manual
# dispatch still guard the merged state.
adapter-test:
if: github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
needs: build
steps:
+32 -12
View File
@@ -1,6 +1,10 @@
name: E2E Headed Chrome
on:
# E2E removed from `pull_request` to keep PR feedback under ~2 minutes; PR-time
# protection is the CI workflow (typecheck / unit / lint / adapter / build).
# E2E still guards `main` directly, runs nightly, and on release tag push so
# protocol/CDP/extension contract regressions are caught before they ship.
push:
branches: [main, dev]
paths:
@@ -13,18 +17,11 @@ on:
- 'tests/smoke/**'
- '.github/actions/setup-chrome/**'
- '.github/workflows/e2e-headed.yml'
pull_request:
branches: [main, dev]
paths:
- 'extension/**'
- 'src/browser/**'
- 'src/daemon.ts'
- 'src/execution.ts'
- 'src/interceptor.ts'
- 'tests/e2e/**'
- 'tests/smoke/**'
- '.github/actions/setup-chrome/**'
- '.github/workflows/e2e-headed.yml'
tags: ['v*']
schedule:
# Daily 08:00 UTC — catch flake / Chrome-version drift even when no commits
# touched the watched paths recently.
- cron: '0 8 * * *'
workflow_dispatch:
concurrency:
@@ -59,12 +56,35 @@ jobs:
- name: Build
run: npm run build
- name: Build extension
run: npm run build --prefix extension
- name: Run AX Chrome smoke (Linux, via xvfb)
if: runner.os == 'Linux'
env:
CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
OPENCLI_AX_E2E: '1'
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run --project e2e tests/e2e/browser-ax-chrome.test.ts --reporter=verbose
- name: Run AX Chrome smoke (macOS / Windows)
if: runner.os != 'Linux'
env:
CHROME_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
OPENCLI_AX_E2E: '1'
run: npx vitest run --project e2e tests/e2e/browser-ax-chrome.test.ts --reporter=verbose
- name: Run E2E tests (Linux, via xvfb)
if: runner.os == 'Linux'
env:
OPENCLI_AX_E2E: '0'
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run tests/e2e/ --reporter=verbose
- name: Run E2E tests (macOS / Windows)
if: runner.os != 'Linux'
env:
OPENCLI_AX_E2E: '0'
run: npx vitest run tests/e2e/ --reporter=verbose
+290 -3
View File
@@ -1,6 +1,281 @@
# Changelog
## Unreleased
## [1.8.0](https://github.com/jackwener/opencli/compare/v1.7.22...v1.8.0) (2026-05-20)
Substantial release: a new official-API adapter (`weread-official`), wider LinkedIn / Twitter / Reddit / Zhihu coverage, the 12306 / Suno / Xianyu inbox additions, security and reliability fixes for the Browser Bridge and media downloads, plus a 20% README shrink. Node 20 compatibility is restored after an automated `undici` bump regression.
### ⚠ BREAKING CHANGES
* **skills** — remove the `smart-search` skill. Use `opencli-usage` for command/site reference, `opencli-browser` for ad-hoc browser operation, and `opencli-adapter-author` for writing new adapters.
### Features
* **weread-official** — integrate WeRead's official Agent Gateway as the `weread-official` CLI namespace. Pure HTTP, Bearer auth via `WEREAD_API_KEY` (no browser, no cookies). 8 commands cover the official skill bundle: `search`, `shelf`, `book` (info + chapters + progress 3-in-1), `notes` (notebook overview or per-book highlights/thoughts), `review`, `readdata` (weekly/monthly/annually/overall), `discover` (recommend or similar-book), `list-apis`. Adapter surfaces typed errors for all documented failure modes — `AuthRequiredError` on missing/rejected key (errcodes -2010/-2012), `CommandExecutionError` on HTTP/`upgrade_info`/non-zero errcode, `EmptyResultError` on empty payloads. Coexists with the existing cookie-based `weread` adapter.
* **12306** — add full read adapter (`stations` / `trains` / `train` / `price` / `me` / `passengers` / `orders`). ([#1637](https://github.com/jackwener/opencli/issues/1637))
* **xianyu** — add `inbox`, `messages`, and `reply` commands. ([#1639](https://github.com/jackwener/opencli/issues/1639))
* **suno** — add Suno.com music-generation adapter. ([#1638](https://github.com/jackwener/opencli/issues/1638))
* **linkedin** — consolidate messaging and Sales Navigator commands (`connect`, `inbox`, `safe-send`, `salesnav-search`, `salesnav-inbox`, `salesnav-message`, `salesnav-thread`, `sent-invitations`, `thread-snapshot`, `timeline`). ([#1647](https://github.com/jackwener/opencli/issues/1647))
* **linkedin/people-search** — add a dedicated people-search command. ([#1649](https://github.com/jackwener/opencli/issues/1649))
* **linkedin-learning** — add `search` / `trending` / `course` read commands. ([#1657](https://github.com/jackwener/opencli/issues/1657))
* **twitter** — rewrite the download-profile path on GraphQL UserMedia with cursor pagination. ([#1636](https://github.com/jackwener/opencli/issues/1636))
* **twitter** — add `list-create` (GraphQL CreateList mutation). ([#1656](https://github.com/jackwener/opencli/issues/1656))
* **twitter** — add `device-follow` notification-stream command.
* **twitter** — expose `card.binding_values` on read commands for inline link-preview metadata. ([#1660](https://github.com/jackwener/opencli/issues/1660))
* **twitter** — expose `quoted_tweet` on read commands. ([#1667](https://github.com/jackwener/opencli/issues/1667))
* **twitter** — expose `bio` on read commands.
* **reddit/subscribed** — new `subscribed` command + listing-level `id` / `created_utc` / `selftext` exposure. ([#1651](https://github.com/jackwener/opencli/issues/1651))
* **reddit** — expose `post_hint` / `url` / `preview` / `gallery` media routes on listing commands. ([#1676](https://github.com/jackwener/opencli/issues/1676))
* **zhihu** — add answer-comments reader; include answer links in question results.
* **chatgpt** — detect generated image surfaces (CSS background and canvas, not just `<img>`) so image generation works after UI drift. ([#1677](https://github.com/jackwener/opencli/issues/1677))
* **external** — add Cloudflare Wrangler as a built-in external CLI passthrough. ([#1679](https://github.com/jackwener/opencli/pull/1679))
### Bug Fixes
* **deps** — restore Node 20 runtime compatibility by pinning runtime `undici` back to the 6.x line (an automated dependabot bump to 8.x had moved the engines floor to Node ≥22.19, silently breaking the published Node 20 promise), and clear the docs build audit chain by overriding VitePress' Vite/PostCSS transitive dependencies to patched versions. ([#1673](https://github.com/jackwener/opencli/issues/1673))
* **download** — keep custom media filenames inside the requested output directory by stripping POSIX/Windows path components and sanitizing the generated fallback prefix. Prevents remote-controlled fields (e.g. video titles used as filename) from escaping the output directory via `../`. ([#1642](https://github.com/jackwener/opencli/pull/1642))
* **browser** — recover `Page.goto()` from stale page identities by clearing the cached targetId and retrying navigation once through the session lease; classify CDP `-32000 Cannot find default execution context` as retryable target navigation. ([#1645](https://github.com/jackwener/opencli/issues/1645))
* **cli** — escape leading-dash positional values via the argv preprocessor so users can pass tokens starting with `-` without commander mis-classifying them as flags. ([#1658](https://github.com/jackwener/opencli/issues/1658))
* **chatgpt/image** — fix ChatGPT web image generation after UI drift by letting the composer locator continue into the caller's readiness check and detecting generated images rendered as CSS backgrounds or canvases, not just plain `<img>` elements.
* **adapters** — surface the remaining `silent-empty-fallback` adapter failures as typed errors (Douyin user video comments, Jike SSR JSON parse, WeRead search-page fetch). True empty Douyin/Jike/WeRead result sets now throw `EmptyResultError`.
* **adapters** — drop silent-sentinel row fallbacks across Apple Podcasts / Reddit / Gitee. ([#1634](https://github.com/jackwener/opencli/issues/1634))
* **adapters** — migrate legal empty-data branches to `EmptyResultError` for `xhs` / YouTube and 5 follow-up commands. ([#1674](https://github.com/jackwener/opencli/issues/1674), [#1678](https://github.com/jackwener/opencli/issues/1678))
* **lesswrong** — drop the `"Unknown"` silent sentinel in the author column; missing authors now propagate as `null`. ([#1611](https://github.com/jackwener/opencli/issues/1611))
* **youtube/transcript** — scope timedtext URL matching to the current `videoId` across the in-page resource-buffer scan, the in-page fetch/XHR hook, and the Node-side CDP capture. SPA-style watch→watch navigation no longer returns a predecessor video's captions. ([#1655](https://github.com/jackwener/opencli/issues/1655))
* **twitter/lists** — skip the "Discover new Lists" recommendation block so it is no longer treated as one of the user's lists. ([#1652](https://github.com/jackwener/opencli/issues/1652))
* **zhihu** — harden search pagination. ([#1615](https://github.com/jackwener/opencli/issues/1615))
* **zhihu** — decode numeric HTML entities in `answer-detail`. ([#1629](https://github.com/jackwener/opencli/issues/1629))
### Docs
* **readme** — major shrink and reframing: tagline rephrased around "Browser Use", Highlights and Update sections folded into adjacent content, Built-in Commands curated to 11 popular sites, CLI Hub table reduced to a name enumeration, Desktop App Adapters collapsed to a one-liner, skill-attribution references audited against `SKILL.md` frontmatter, "For AI Agents (Developer Guide)" merged into "Writing a new adapter". Net: EN 410 → 326 (-20%), ZH 455 → 371 (-18%). ([#1654](https://github.com/jackwener/opencli/pull/1654), [#1666](https://github.com/jackwener/opencli/pull/1666), [#1679](https://github.com/jackwener/opencli/pull/1679), [#1681](https://github.com/jackwener/opencli/pull/1681))
### Internal
* **audit** — stop flagging sentinel fallback strings inside thrown error messages as `silent-sentinel` violations. These are typed failure diagnostics rather than fake row data, reducing the typed-error baseline to actual adapter output fallbacks.
## [1.7.22](https://github.com/jackwener/opencli/compare/v1.7.21...v1.7.22) (2026-05-15)
External CLI ergonomics + two adapter envelope/auth fixes. New `longbridge` external CLI entry; `opencli list` / root help now render human-readable brand labels for executables whose bare name is ambiguous.
### Features
* **external** — add the Longbridge CLI as a built-in external CLI passthrough (`opencli longbridge ...`) for Longbridge OpenAPI market data, account, and trading commands. ([#1584](https://github.com/jackwener/opencli/issues/1584))
* **external-cli** — render brand alias `name(package)` in `opencli list` and root help when the bare executable name is ambiguous. Built-in entries `ntn``ntn(notion)`, `dws``dws(DingTalk Workspace)`, `wecom-cli``wecom-cli(企业微信)` now self-explain in help output. `package` field is repurposed to cover both upstream distribution names (e.g. `tg-cli`) and human-readable brand labels (e.g. `notion`, `企业微信`). ([#1585](https://github.com/jackwener/opencli/issues/1585))
### Bug Fixes
* **boss** — map `code=24` (identity mismatch) to `AuthRequiredError` so re-login is signaled instead of surfacing as a generic API error. ([#1573](https://github.com/jackwener/opencli/issues/1573))
* **weibo** — unwrap Browser Bridge `page.evaluate` envelopes in read adapters. ([#1568](https://github.com/jackwener/opencli/issues/1568))
## [1.7.21](https://github.com/jackwener/opencli/compare/v1.7.20...v1.7.21) (2026-05-14)
Adapter polish release: new web search adapters, better Browser Bridge tab group reuse, and social adapters returning to one-shot tab leases. Extension package version is bumped to 1.0.15 for the Browser Bridge fix.
### Features
* **search** — add DuckDuckGo, Brave, and Yahoo web search adapters. ([#1546](https://github.com/jackwener/opencli/issues/1546))
* **boss** — support job-seeker `chatlist` and `chatmsg` adapters. ([#1539](https://github.com/jackwener/opencli/issues/1539))
### Bug Fixes
* **extension** — reuse existing `OpenCLI Adapter` tab groups before creating new ones, including cross-window discovery, legacy `OpenCLI` title fallback, and deterministic candidate selection. ([#1541](https://github.com/jackwener/opencli/issues/1541))
* **twitter, reddit** — default browser-backed social adapters back to ephemeral tab leases. Twitter/X and Reddit commands now release their site tab after each run while keeping the shared Adapter window available for reuse; persistent sessions remain reserved for AI/chat-style adapters that need long-lived conversation state. ([#1569](https://github.com/jackwener/opencli/issues/1569))
* **xiaohongshu, rednote** — unwrap Browser Bridge `page.evaluate` envelopes in search adapters. ([#1561](https://github.com/jackwener/opencli/issues/1561))
* **facebook/feed** — add fallback extraction for empty article nodes. ([#1538](https://github.com/jackwener/opencli/issues/1538))
### Internal
* **ci** — add Windows native binding lockfile entries for Rolldown/Rollup optional packages. ([#1563](https://github.com/jackwener/opencli/issues/1563))
* **extension** — add regression coverage for the adapter tab group `groupId` tiebreaker. ([#1566](https://github.com/jackwener/opencli/issues/1566))
## [1.7.20](https://github.com/jackwener/opencli/compare/v1.7.19...v1.7.20) (2026-05-14)
External CLI surface cleanup + Browser Bridge WebSocket lifecycle hardening. Two BREAKING changes around external CLIs: built-in `tg`/`discord`/`wx` (was `tg-cli`/`discord-cli`/`wx-cli`) now match their real binary names, and Notion's in-tree CDP adapter is replaced by the official `ntn` external CLI.
### ⚠ BREAKING CHANGES
* **notion** — remove the in-tree `clis/notion/` CDP-on-Desktop adapter (8 commands: `status` / `search` / `read` / `new` / `write` / `sidebar` / `favorites` / `export`). Notion has shipped an official CLI at <https://ntn.dev>, registered as a first-class external CLI in `external-clis.yaml`. Migration: install `ntn` from <https://ntn.dev> (`curl -fsSL https://ntn.dev | bash`), then use `opencli ntn <command>`. Auto-install is intentionally not configured because the official installer is a shell script while OpenCLI external installs run shell-free command strings. The official CLI uses the public Notion API rather than reverse-engineering the Desktop UI, so it survives Notion app updates and exposes a wider command surface (blocks / databases / properties / comments) than the reverse-engineered adapter could. ([#1559](https://github.com/jackwener/opencli/issues/1559))
* **external** — drop the `-cli` suffix from built-in external CLI subcommand names. `opencli tg-cli`, `opencli discord-cli`, `opencli wx-cli` are now `opencli tg`, `opencli discord`, `opencli wx`, matching the real binary names that those tools install as. Root help still shows the package lineage as `tg(tg-cli)` / `discord(discord-cli)` / `wx(wx-cli)`. ([#1544](https://github.com/jackwener/opencli/issues/1544))
### Features
* **twitter** — `bookmarks` and `bookmark-folder` now include media via `extractMedia`, reaching parity with `timeline` / `search`. ([#1555](https://github.com/jackwener/opencli/issues/1555))
* **twitter/list-tweets** — include media via `extractMedia` (parity with `timeline` / `search`). ([#1464](https://github.com/jackwener/opencli/issues/1464))
### Bug Fixes
* **daemon** — report ambiguous browser command outcomes with a distinct `command_result_unknown` errorCode and `503` when the extension WebSocket drops between command dispatch and result delivery. `sendCommandRaw()` treats this code as hard non-retryable, so write-side commands (`navigate` / `click` / `type` / `eval`) won't be silently re-issued and double-executed. Daemon exposes a `commandResultUnknown` counter on `/status` for future observability. ([#1558](https://github.com/jackwener/opencli/issues/1558))
* **extension** — keep active daemon WebSocket; stale sockets no longer clobber active connection (`onopen` / `onclose` / `onmessage` are all gated by `ws !== thisWs` short-circuit), and `safeSend` only fires when `readyState === OPEN`. ([#1540](https://github.com/jackwener/opencli/issues/1540))
* **extension** — coalesce concurrent daemon WebSocket connects via an in-flight promise. Startup / keepalive / reconnect triggering `connect()` during the daemon-probe or context-lookup async gap no longer creates duplicate real WebSocket connections. ([#1554](https://github.com/jackwener/opencli/issues/1554))
* **external** — distinguish external CLI executable names from distribution/project names in root help. Built-in aliases such as `tg`, `discord`, `wx` remain the callable `opencli <name> ...` entrypoints while help renders `tg(tg-cli)`, `discord(discord-cli)`, `wx(wx-cli)` to show their package lineage. ([#1560](https://github.com/jackwener/opencli/issues/1560))
### Docs
* **browser** — clarify named session lifecycle in the Browser Bridge guide. ([#1542](https://github.com/jackwener/opencli/issues/1542))
## [1.7.19](https://github.com/jackwener/opencli/compare/v1.7.18...v1.7.19) (2026-05-14)
Major hotfix + simplification batch. Extension bumped to 1.0.14. Node floor lowered to v20 so the long tail of Node v20v21.6 users no longer crashes at module load. `opencli browser` user surface replaces required-flag `--session <name>` with a `<session>` positional. `page.evaluate(fn, ...args)` adds a type-safe alternative to the implicit auto-IIFE string form. Twitter cursor pagination no longer silently caps at ~500 items.
### ⚠ BREAKING CHANGES
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface. ([#1505](https://github.com/jackwener/opencli/issues/1505))
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell. ([#1509](https://github.com/jackwener/opencli/issues/1509))
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail. ([#1510](https://github.com/jackwener/opencli/issues/1510))
### Features
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic. ([#1508](https://github.com/jackwener/opencli/issues/1508))
* **twitter** — default `tweets` command to the logged-in user when `user` is omitted, and fix the sibling envelope-unwrap silent bug. ([#1531](https://github.com/jackwener/opencli/issues/1531))
* **zhihu** — add `answer-detail` to fetch a single answer's full content. ([#1528](https://github.com/jackwener/opencli/issues/1528))
* **zhihu** — paginate question answers and recommendations. ([#1517](https://github.com/jackwener/opencli/issues/1517))
* **reddit/read** — `--expand-more` via `/api/morechildren` + 7-kind typed errors. ([#1492](https://github.com/jackwener/opencli/issues/1492))
* **reddit** — add `whoami`, `home`, `subreddit-info` read commands. ([#1491](https://github.com/jackwener/opencli/issues/1491))
* **ctrip** — add `hotel-search` + flight browser-mode commands. ([#1489](https://github.com/jackwener/opencli/issues/1489))
### Bug Fixes
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **twitter** — raise fixed cursor-pagination caps in `bookmarks` / `likes` / `tweets` / `timeline` / `bookmark-folder` / `list-tweets` / `search` / `following`. The old `i < 5` / `i < 10` literals and following's `Math.ceil(limit / 50) + 2` formula imposed hidden result ceilings below `--limit`; the loop now treats the page count as a high runaway guard while `--limit` and cursor exhaustion control normal pagination. ([#1532](https://github.com/jackwener/opencli/issues/1532))
* **twitter** — repair `list-add` / `list-tweets` / `lists` / `following` after 2026-05 site changes. ([#1503](https://github.com/jackwener/opencli/issues/1503))
* **twitter** — repair `search` and `tweets` readback. ([#1512](https://github.com/jackwener/opencli/issues/1512))
* **twitter** — make reply submission robust. ([#1511](https://github.com/jackwener/opencli/issues/1511))
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **google/search** — wrap evaluate return value in object to fix serialization. ([#1523](https://github.com/jackwener/opencli/issues/1523))
* **google-scholar/search** — wrap evaluate return to fix serialization. ([#1525](https://github.com/jackwener/opencli/issues/1525))
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **xiaohongshu** — `parseLikes` handles `2.1w` / `1.5万` / `1.2k` shortforms. ([#1504](https://github.com/jackwener/opencli/issues/1504))
* **xiaohongshu+rednote/search** — fall back to href-based note cards when `section.note-item` class is dropped. ([#1507](https://github.com/jackwener/opencli/issues/1507))
* **xueqiu** — `kline` / `earnings-date` format dates in Asia/Shanghai instead of UTC. ([#1498](https://github.com/jackwener/opencli/issues/1498))
* **download** — clamp progress percentages. ([#1520](https://github.com/jackwener/opencli/issues/1520))
### Internal
* **runtime** — lower the Node floor to `>=20.0.0`. Three coupled changes: drop all `util.styleText()` usage (added in Node v21.7.0 / v20.12.0; previously crashed v21.0v21.6 at module load), downgrade `undici` from `^8.0.2` (engines `>=22.19.0`) to `^6.25.0` (engines `>=18.17`, retains `Agent` / `EnvHttpProxyAgent` / `fetch`), and lower `MIN_SUPPORTED_NODE_MAJOR` from 21 to 20 so the startup guard matches the declared `engines.node`. Smoke-tested on v20.0.0 / v21.2.0 / v22.22.2. The semantic markers (`[OK]` / `[WARN]` / `[FAIL]` / `` / `⚠` / `✖`) keep their meaning; ANSI colors were redundant for the primarily agent-facing CLI. ([#1524](https://github.com/jackwener/opencli/issues/1524))
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **extension 1.0.13** — remove the internal command-session lease-key backdoor. ([#1510](https://github.com/jackwener/opencli/issues/1510))
* **ci** — drop `e2e-headed` and `adapter-test` from `pull_request` triggers (kept on `push` to main / nightly / `workflow_dispatch`). PR-time CI now targets ~2 min wall-time. ([#1521](https://github.com/jackwener/opencli/issues/1521), [#1522](https://github.com/jackwener/opencli/issues/1522))
* **scripts** — auto-refresh `dist/` before `build-manifest`. ([#1490](https://github.com/jackwener/opencli/issues/1490))
## [1.7.18](https://github.com/jackwener/opencli/compare/v1.7.17...v1.7.18) (2026-05-12)
Hotfix release for the 1.7.17 doctor regression: `opencli doctor` failed connectivity probe with `Browser session is required` because the doctor probe didn't pass a session to the new strict-session browser bridge. Also adds new adapters and adapter fixes that were ready immediately after 1.7.17.
### Bug Fixes
* **doctor** — pass an internal `__doctor__` browser session to the live connectivity probe so `opencli doctor` works again under the explicit-session browser model introduced in 1.7.17. ([#1485](https://github.com/jackwener/opencli/issues/1485))
* **browser** — `--session <name>` is now declared as a `requiredOption` so Commander itself rejects calls missing the flag before runtime, and the help line is marked `(required)` instead of being hidden under `Options:`. ([#1485](https://github.com/jackwener/opencli/issues/1485))
* **doubao/ask** — restore Assistant detection after the 2026-05 DOM refactor. ([#1484](https://github.com/jackwener/opencli/issues/1484))
* **youtube** — request `srv3` format for caption URLs. ([#1422](https://github.com/jackwener/opencli/issues/1422))
### Features
* **rednote** — add `rednote.com` adapter mirroring xiaohongshu read commands. ([#1475](https://github.com/jackwener/opencli/issues/1475))
* **reddit** — add `reply` command for replying to comments. ([#1428](https://github.com/jackwener/opencli/issues/1428))
## [1.7.17](https://github.com/jackwener/opencli/compare/v1.7.16...v1.7.17) (2026-05-12)
Extension bumped to 1.0.12 (workspace → session lease routing, drop `handleSessions` handler). Major simplification pass: browser/adapter session model rewrite, `--workspace` removed, doctor surface trimmed to its core job.
### ⚠ BREAKING CHANGES
* **browser session model** — replace the browser-facing `--workspace` model with explicit `--session <name>` on `opencli browser *`. Browser commands now require a session name, `browser bind`/`unbind` use `--session`, and bind no longer accepts `--domain`, `--path-prefix`, or `--allow-navigate-bound`. Browser primitives keep their session tab by design; the browser namespace no longer exposes `--keep-tab`. ([#1461](https://github.com/jackwener/opencli/issues/1461))
* **adapter site sessions** — replace adapter metadata `browserSession: { reuse: 'site' }` with `siteSession: 'persistent'`, and replace the user override `--reuse <none|site>` / `OPENCLI_BROWSER_REUSE` with `--site-session <ephemeral|persistent>`. Persistent site sessions keep a stable site tab open without idle expiry. ([#1462](https://github.com/jackwener/opencli/issues/1462))
* **doctor** — remove `--no-live` and `--sessions` flags from `opencli doctor`. Doctor always runs the live browser connectivity probe (that's its core job); session enumeration was never part of health diagnosis. The underlying `'sessions'` daemon protocol action and the `BrowserSessionInfo` public type are removed as dead code. ([#1470](https://github.com/jackwener/opencli/issues/1470))
### Features
* **chatgpt** — `ask` and `send` now accept local image paths and upload them through the composer before submitting the prompt. ([#1476](https://github.com/jackwener/opencli/issues/1476))
### Internal
* **extension 1.0.12** — drop `handleSessions` action handler (no remaining consumers after doctor cleanup).
* **extension 1.0.11** — switch Browser Bridge lease routing from user-facing workspaces to explicit browser sessions.
## [1.7.16](https://github.com/jackwener/opencli/compare/v1.7.15...v1.7.16) (2026-05-11)
Extension bumped to 1.0.10 (rename adapter-owned tab group `OpenCLI Automation``OpenCLI Adapter`). Performance and stability sweep across browser-backed adapters; new external CLI integrations (tg-cli, discord-cli, wx-cli).
### Features
* **openreview** — add `author` command for ID-explicit publication lookup. ([#1365](https://github.com/jackwener/opencli/issues/1365))
* **external** — register `tg-cli`, `discord-cli`, and `wx-cli` as external CLI integrations. ([#1458](https://github.com/jackwener/opencli/issues/1458))
### Bug Fixes
* **xiaohongshu** — fall back to base64 upload when CDP `DOM.setFileInputFiles` returns `Not allowed` on creator center. ([#1374](https://github.com/jackwener/opencli/issues/1374))
* **chatgpt** — switch to locale-stable send button selector so non-English UIs don't break send. ([#1354](https://github.com/jackwener/opencli/issues/1354))
### Performance
* **adapters** — hoist cookie reads to `page.getCookies` across Tier 1 (25 files), eliminating per-call CDP round trips. ([#1450](https://github.com/jackwener/opencli/issues/1450))
* **twitter** — drop redundant `goto + wait` in adapter steps; framework auto pre-navigates. ([#1451](https://github.com/jackwener/opencli/issues/1451))
* **twitter** — enable `browserSession.reuse: 'site'` on 17 read-only adapters so repeated reads share one tab. ([#1454](https://github.com/jackwener/opencli/issues/1454))
* **reddit** — opt 13 browser-backed adapters into shared site-tab lease. ([#1455](https://github.com/jackwener/opencli/issues/1455))
* **claude** — replace fixed-sleep waits with selector-based readiness on streaming flows. ([#1452](https://github.com/jackwener/opencli/issues/1452))
* **deepseek** — replace fixed-sleep waits with selector-based readiness on streaming flows. ([#1449](https://github.com/jackwener/opencli/issues/1449))
* **chatgpt** — replace fixed-sleep waits with selector-based readiness (D3). ([#1456](https://github.com/jackwener/opencli/issues/1456))
### Refactor
* **browser** — split interactive and automation windows so `opencli browser *` and adapter-driven background commands no longer share one Chrome window; tab groups are isolated by role.
### Internal
* **extension 1.0.10** — rename the adapter-owned Chrome tab group from `OpenCLI Automation` to `OpenCLI Adapter`. ([#1457](https://github.com/jackwener/opencli/issues/1457))
* **docs** — list `tg-cli`, `discord-cli`, `wx-cli` in External CLI README sections. ([#1459](https://github.com/jackwener/opencli/issues/1459))
## [1.7.15](https://github.com/jackwener/opencli/compare/v1.7.14...v1.7.15) (2026-05-10)
Extension bumped to 1.0.9 (Accessibility.enable allowlist + downloads permission + cross-origin frame target attach for AX). Major Browser Agent Runtime release: full Phase 0/1/2 alignment with `vercel-labs/agent-browser` model — CDP-primary input, AX snapshot/refs with stale recovery, semantic locators across all primitives, full form toolbelt (hover/focus/dblclick/check/uncheck/upload/drag/wait-download), annotated screenshots, and same-origin iframe AX routing. Cross-origin OOPIF AX is best-effort (Chrome extension API limitation).
### ⚠ BREAKING CHANGES
* **browser lifecycle** — replace `--focus` / `OPENCLI_WINDOW_FOCUSED` with `--window foreground|background` / `OPENCLI_WINDOW`, and replace `--live` / `OPENCLI_LIVE` with `--keep-tab true|false` / `OPENCLI_KEEP_TAB`. `opencli browser *` defaults to a foreground window and keeps its tab; browser-backed adapter commands default to a background automation window and release their tab unless the adapter uses site-level reuse.
### Features
* **help / browser** — `opencli browser --help -f yaml|json` now emits a structured, agent-ready index of all browser leaf commands (including nested `tab`, `get`, and `dialog` commands), their positionals, command options, namespace options, and root global options. Individual browser commands also support structured help, backed by a shared Commander option/argument spec extractor.
* **help / built-in namespaces** — `opencli daemon|plugin|adapter|profile --help -f yaml|json` now emit the same structured payload as `browser`. One agent call returns every leaf's positionals, options, descriptions, and global options — no per-leaf `--help` follow-ups needed. Original namespace descriptions are preserved through `applyRootSubcommandSummaries()` via a snapshot at namespace declaration time.
* **browser state** — add opt-in AX snapshot refs via `browser state --source ax`, including backend-node click resolution and role/name stale-ref recovery for the Phase 0 browser-agent runtime prototype.
* **browser state** — AX snapshots now include same-origin iframe refs, and `browser state --compare-sources` prints DOM-vs-AX observation metrics for the Phase 1 default-source decision without dumping page contents.
* **browser locators** — `browser find`, `browser click`, and `browser get text|value|attributes` now accept semantic locator flags (`--role`, `--name`, `--label`, `--text`, `--testid`) so agents can act on common controls without a separate state-ref lookup.
* **browser locators** — semantic locator flags now work across input/action primitives (`type`, `fill`, `select`, `hover`, `focus`, `dblclick`, `check`, `uncheck`, `upload`) plus prefixed `--from-*` / `--to-*` locators for `drag`.
* **browser actions** — add `browser hover`, `browser focus`, and `browser dblclick` primitives backed by the same target resolver and CDP input path as `browser click`.
* **browser actions** — add `browser check` and `browser uncheck` primitives that ensure checkbox / radio / aria-checked controls reach the requested state instead of blindly toggling.
* **browser upload** — add `browser upload <target> <file...>` to attach local files to `input[type=file]` targets through CDP `DOM.setFileInputFiles`, with local path validation and file-input verification.
* **browser actions** — add `browser drag <source> <target>` for CDP mouse drag sequences between two resolved element centers.
* **browser wait / extension 1.0.8** — add `browser wait download [pattern]` backed by Chrome's downloads lifecycle API, so agents can wait for file downloads by filename/URL pattern and receive completed/failed download metadata.
* **browser state / extension 1.0.9** — AX snapshots can now route same-origin iframe refs through `frameId`. Cross-origin OOPIF AX routing is best-effort because real Chrome extension smoke tests show `chrome.debugger` may not expose attachable iframe targets to extensions.
* **browser screenshot** — add `browser screenshot --annotate`, which refreshes DOM refs and overlays visible `[N]` labels on the screenshot so visual inspection maps back to `browser click <ref>` targets.
### Bug Fixes
* **browser click** — `browser click` now prefers CDP `Input.dispatchMouseEvent` over DOM `el.click()`, so custom dropdowns that depend on pointer/mouse events (Radix, shadcn, Material UI, Mercury-style category pickers) open and select reliably while retaining JS click as a fallback for older backends or zero-rect targets.
* **browser state / extension 1.0.7** — `browser state --source ax` now enables the CDP Accessibility domain before reading the AX tree, fixing real-Chrome snapshots that previously returned only `RootWebArea` with zero refs.
* **help / build** — every positional arg must now declare a non-empty `help` string. The build-manifest step fails closed when a positional has empty / whitespace-only / missing `help`, so `opencli <site> <cmd> --help` always shows callers what each parameter is for. Pre-existing offenders (`twitter followers/following/list-add/list-remove/list-tweets/search/thread`, `reddit search/subreddit/user/user-comments/user-posts`, `douyin stats/update`, `bilibili subtitle`, `jike search`) now have explicit help text — most notably `twitter followers [user]` and `following [user]` now document that omitting the user fetches the currently logged-in account.
## [1.7.14](https://github.com/jackwener/opencli/compare/v1.7.13...v1.7.14) (2026-05-08)
### Features
* **help** — adapter help is now agent-friendly: per-command listings drop the `[options]` noise from globally-shared options (`--format`, `--trace`, `-v`, `-h`, etc.) and only mention them at the site level, so `opencli twitter` etc. read like a flat command index. ([#1401](https://github.com/jackwener/opencli/issues/1401))
* **twitter** — write-action symmetry P0: add `unlike`, `retweet`, `unretweet`, and `quote` to round out the read/write coverage. ([#1400](https://github.com/jackwener/opencli/issues/1400))
### Bug Fixes
* **browser daemon** — `npm install -g @jackwener/opencli@latest` now correctly auto-restarts a stale ready-state daemon so users pick up the new version without a manual `opencli daemon restart`. ([#1399](https://github.com/jackwener/opencli/issues/1399))
## [1.7.13](https://github.com/jackwener/opencli/compare/v1.7.12...v1.7.13) (2026-05-07)
Extension bumped to 1.0.6 (screenshot `--width` / `--height` / `--full-page` flags, automation tab group color marker, automation container reuse fix).
### ⚠ BREAKING CHANGES
* **linux-do** — remove deprecated compatibility shims `linux-do hot`, `linux-do category`, `linux-do latest`. Use `linux-do feed --view top --period <period>`, `linux-do feed --category <id-or-name>`, and `linux-do feed --view latest` instead.
* **grok ask** — drop the `--web` flag and the legacy `<textarea>` composer path. The default flow is now the only path and uses the current ProseMirror+TipTap composer (the path that used to require `--web true`). Existing scripts passing `--web` will get an "unknown option" error from commander; remove the flag.
* **env** — rename `OPENCLI_BROWSER_TIMEOUT` to `OPENCLI_BROWSER_IDLE_TIMEOUT`. The variable controls workspace lease idle release time, not per-command runtime; the new name reflects that. Old name was undocumented and removed without a fallback.
* **registry** — remove the unused `Strategy.HEADER`; adapter authors should use `Strategy.COOKIE` and set headers explicitly inside browser-side fetches.
### Features
@@ -8,11 +283,23 @@
* **autofix** — retire `OPENCLI_DIAGNOSTIC`; adapter repair now uses `--trace retain-on-failure`, trace `summary.md`, and error-envelope trace metadata.
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
* **browser session** — adapter commands can opt into site-level tab reuse with `browserSession.reuse = 'site'`; Grok and other browser-backed LLM adapters now keep a shared site tab by default, and users can override with `--reuse <none|site>`.
* **chatgpt** — add browser-web baseline commands: `ask`, `send`, `read`, `history`, `detail`, `new`, and `status`.
* **grok** — add browser-web baseline commands: `read`, `history`, `detail`, `new`, `send`, and `status` (existing `ask` and `image` unchanged).
* **yuanbao** — add browser-web baseline commands: `send`, `status`, `read`, `history`, and `detail` (joining the existing `ask` and `new`).
* **qwen** — add `detail` command for opening a specific historical conversation by id.
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
### Breaking Changes
### Bug Fixes
* **registry** — remove the unused `Strategy.HEADER`; adapter authors should use `Strategy.COOKIE` and set headers explicitly inside browser-side fetches.
* **pipeline / capabilityRouting** — the `fill` pipeline step (introduced in [#1222](https://github.com/jackwener/opencli/issues/1222)) now correctly triggers a browser session and gets transient retry coverage; previously a pipeline using only `fill` could crash on a missing page object. ([#1393](https://github.com/jackwener/opencli/issues/1393))
* **xiaohongshu publish** — improve image publishing reliability via creator-center URL routing, tab priority handling, and DataTransfer fallback.
* **youtube** — use watch-page HTML for transcript captions to recover when the public transcript API is unavailable.
* **desktop adapters** — restore 11 desktop adapter commands that were lost from the manifest due to a factory-pattern regression.
### Internal
* **cleanup** — remove dead `src/analysis.ts` (179 lines, 0 importers), retire `OPENCLI_DIAGNOSTIC` test residue, derive validator step allowlist from the live pipeline registry to prevent future drift.
## [1.7.8](https://github.com/jackwener/opencli/compare/v1.7.7...v1.7.8) (2026-04-25)
+37 -164
View File
@@ -1,7 +1,8 @@
# OpenCLI
> **Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.**
> Reuse your logged-in browser, automate live workflows, and crystallize repeated actions into reusable CLI commands.
> **Convert any website into a CLI & run Browser Use on your logged-in Chrome.**
> Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.
> Or run Browser Use against any page — navigate, fill forms, click, extract, automate.
[![中文文档](https://img.shields.io/badge/docs-%E4%B8%AD%E6%96%87-0F766E?style=flat-square)](./README.zh-CN.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
@@ -11,24 +12,10 @@
OpenCLI gives you one surface for three different kinds of automation:
- **Use built-in adapters** for sites like Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, Twitter/X, and [many more](#built-in-commands).
- **Let AI Agents operate any website** — install the `opencli-adapter-author` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Let AI Agents operate any website** — install the `opencli-browser` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Write new adapters** end-to-end with `opencli browser` + the `opencli-adapter-author` skill, which guides from first recon through field decoding, code, and `opencli browser verify`.
It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, ChatGPT, and Notion.
## Highlights
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type/fill, extract, screenshot — all through your logged-in Chrome session.
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
- **Website → CLI** — Turn any website into a deterministic CLI: 100+ site surfaces are already registered, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
- **CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, docker, obsidian, etc).
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
- **Deterministic** — Same command, same output schema, every time. Pipeable, scriptable, CI-friendly.
---
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `longbridge`, `tg`, `discord`, `wx`, `ntn` (Notion), and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, and ChatGPT.
## Quick Start
@@ -105,7 +92,7 @@ If you want to add your own commands, start with the [Extending OpenCLI guide](.
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
### Install skills
### Install skills (also refreshes existing installs)
```bash
npx skills add jackwener/opencli
@@ -118,22 +105,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
### Which skill to use
| Skill | When to use | Example prompt to your AI agent |
|-------|------------|-------------------------------|
| **opencli-adapter-author** | Operate a site in real time, or write a reusable adapter for a new site | "Help me check my Xiaohongshu notifications" / "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-adapter-author** | Write a reusable adapter for a new site or add a command to an existing site | "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-autofix** | Repair a broken adapter when a built-in command fails | "`opencli zhihu hot` is returning empty — fix it" |
| **opencli-browser** | Browser automation reference for AI agents | "Use browser commands to scrape this page" |
| **opencli-browser** | Drive a real Chrome page ad-hoc — navigate, fill forms, click, extract | "Help me check my Xiaohongshu notifications" / "Help me fill out this form" / "Use browser commands to scrape this page" |
| **opencli-usage** | Quick reference for all OpenCLI commands and sites | "What commands does OpenCLI have for Twitter?" |
| **smart-search** | Search across existing OpenCLI capabilities | "Find me a Bilibili trending adapter" |
### How it works
Once `opencli-adapter-author` is installed, your AI agent can:
Once `opencli-browser` is installed, your AI agent can:
1. **Navigate** to any URL using your logged-in browser
2. **Read** page content via structured DOM snapshots (not screenshots)
@@ -144,45 +129,25 @@ Once `opencli-adapter-author` is installed, your AI agent can:
The agent handles all the `opencli browser` commands internally — you just describe what you want done in natural language.
**Skill references:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — browser operation + adapter authoring, end-to-end
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — drive Chrome ad-hoc (navigate, fill forms, click, extract)
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — write a new adapter end-to-end
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — repair broken adapters
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — browser automation reference
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — command and site reference
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — capability search
Available browser commands include `open`, `state`, `click`, `type`, `fill`, `select`, `keys`, `wait`, `get`, `find`, `extract`, `frames`, `screenshot`, `scroll`, `back`, `eval`, `network`, `tab list`, `tab new`, `tab select`, `tab close`, `init`, `verify`, and `close`.
`opencli browser open <url>` and `opencli browser tab new [url]` both return a target ID. Use `opencli browser tab list` to inspect the target IDs of tabs that already exist, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted `opencli browser ...` commands.
`opencli browser` commands require a `<session>` positional immediately after `browser`. `opencli browser work open <url>` and `opencli browser work tab new [url]` both return a target ID. Use `opencli browser work tab list` to inspect target IDs, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted commands in the same session.
## Core Concepts
## Writing a new adapter
### `browser`: AI Agent browser control
When the site you need is not yet covered, use the `opencli-adapter-author` skill end-to-end:
`opencli browser` commands are the low-level primitives that AI Agents use to operate websites. You don't run these manually — instead, install the `opencli-adapter-author` skill into your AI agent, describe what you want in natural language, and the agent handles the browser operations.
For example, tell your agent: *"Help me check my Xiaohongshu notifications"* — the agent will use `opencli browser open`, `state`, `click`, etc. under the hood.
### Built-in adapters: stable commands
Use site-specific commands such as `opencli hackernews top` or `opencli reddit hot` when the capability already exists. These are deterministic and work without browser — ideal for both humans and AI agents.
### Writing a new adapter
When the site you need is not yet covered, use the `opencli-adapter-author` skill. It takes the agent end-to-end:
1. Recon the site and classify its pattern (SPA / SSR / JSONP / Token / Streaming).
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`.
4. Decode response fields and design output columns.
5. `opencli browser analyze <url>` for one-shot recon, then `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
### CLI Hub and desktop adapters
OpenCLI is not only for websites. It can also:
- expose local binaries like `gh`, `docker`, `obsidian`, or custom tools through `opencli <tool> ...`
- control Electron desktop apps through dedicated adapters and CDP-backed integrations
1. **Recon** the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
2. **Discover** the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. **Pick auth**`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`.
4. **Decode** response fields and design output columns.
5. `opencli browser recon analyze <url>``opencli browser recon init <site>/<name>` → write adapter → `opencli browser recon verify <site>/<name>`.
6. Site knowledge persists to `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context.
## Prerequisites
@@ -198,8 +163,7 @@ OpenCLI is not only for websites. It can also:
|----------|---------|-------------|
| `OPENCLI_DAEMON_PORT` | `19825` | HTTP port for the daemon-extension bridge |
| `OPENCLI_PROFILE` | — | Browser Bridge profile alias/contextId to use when multiple Chrome profiles are connected |
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open the automation container in the foreground (useful for debugging). The `--focus` flag sets this. |
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation lease open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
| `OPENCLI_WINDOW` | command default | Set to `foreground` or `background` to override Browser Bridge window placement. Browser-backed commands also accept `--window <foreground\|background>`. |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | Seconds to wait for browser connection |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | Seconds to wait for a single browser command |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
@@ -207,116 +171,35 @@ OpenCLI is not only for websites. It can also:
| `OPENCLI_VERBOSE` | `false` | Enable verbose logging (`-v` flag also works) |
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation lease open until you run `opencli browser close` or the idle timeout expires.
## Update
```bash
npm install -g @jackwener/opencli@latest
# If you use the packaged OpenCLI skills, refresh them too
npx skills add jackwener/opencli
```
Or refresh only the skills you actually use:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
## For Developers
Install from source:
```bash
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link
```
To load the source Browser Bridge extension:
1. Open `chrome://extensions` and enable **Developer mode**.
2. Click **Load unpacked** and select this repository's `extension/` directory.
`opencli browser *` requires an explicit `<session>` positional, uses a foreground browser window by default, and keeps that session's tab lease until `opencli browser <session> close` or idle cleanup. Browser-backed adapters use a background adapter window and release one-shot tab leases by default. Interactive adapters can declare `siteSession: 'persistent'` to keep a stable site tab for continuity; pass `--site-session ephemeral` for a one-shot tab.
## Built-in Commands
| Site | Commands |
|------|----------|
| **xiaohongshu** | `search` `note` `comments` `feed` `user` `download` `publish` `notifications` `creator-notes` `creator-notes-summary` `creator-note-detail` `creator-profile` `creator-stats` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `video` `user-videos` |
| **tieba** | `hot` `posts` `search` `read` |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `summary` `video` `user-videos` |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
| **1688** | `search` `item` `assets` `download` `store` |
| **gitee** | `trending` `search` `user` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **yuanbao** | `new` `ask` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` |
| **xianyu** | `search` `item` `chat` `publish` |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
| **uiverse** | `code` `preview` |
| **baidu-scholar** | `search` |
| **google-scholar** | `search` `cite` `profile` |
| **gov-law** | `search` `recent` |
| **gov-policy** | `search` `recent` |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
| **wanfang** | `search` |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **xiaoyuzhou** | `auth*` `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` |
| **linkedin** | `connect` `inbox` `safe-send` `search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
100+ site surfaces in total**[→ see all supported sites & commands](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast`, `podcast-episodes`, `episode`, `download`, and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
Curated highlights**[→ see all 100+ supported sites & commands](./docs/adapters/index.md)** (douyin / weibo / spotify / 1688 / quark / nowcoder / google-scholar / hupu / xianyu / weread / weread-official / xiaoyuzhou / and more).
## CLI Hub
OpenCLI acts as a universal hub for your existing command-line tools — unified discovery, pure passthrough execution, and auto-install (if a tool isn't installed, OpenCLI runs `brew install <tool>` automatically before re-running the command).
Unified passthrough for your existing command-line tools. Run `opencli <tool> ...` for any of:
| External CLI | Description | Example |
|--------------|-------------|---------|
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
| **obsidian** | Obsidian vault management | `opencli obsidian search query="AI"` |
| **docker** | Docker | `opencli docker ps` |
| **lark-cli** | Lark/Feishu — messages, docs, calendar, tasks, 200+ commands | `opencli lark-cli calendar +agenda` |
| **dws** | DingTalk — cross-platform CLI for DingTalk's full suite, designed for humans and AI agents | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | WeCom/企业微信 — CLI for WeCom open platform, for humans and AI agents | `opencli wecom-cli msg send --to user "hello"` |
| **vercel** | Vercel — deploy projects, manage domains, env vars, logs | `opencli vercel deploy --prod` |
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
Register your own with `opencli external register <name>`; list everything with `opencli external list`.
```bash
opencli external register mycli
```
### Desktop App Adapters
Control Electron desktop apps directly from the terminal. Each adapter has its own detailed documentation:
| App | Description | Doc |
|-----|-------------|-----|
| **Cursor** | Control Cursor IDE — Composer, chat, code extraction | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | Drive OpenAI Codex CLI agent headlessly | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | Control Antigravity Ultra from terminal | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT App** | Automate ChatGPT macOS desktop app | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | Multi-LLM client (GPT-4, Claude, Gemini) | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | Search, read, write Notion pages | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord Desktop — messages, channels, servers | [Doc](./docs/adapters/desktop/discord.md) |
| **Doubao** | Control Doubao AI desktop app via CDP | [Doc](./docs/adapters/desktop/doubao-app.md) |
To add a new Electron app, start with [docs/guide/electron-app-cli.md](./docs/guide/electron-app-cli.md).
**Desktop app adapters** (Electron, via CDP): Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — see [`docs/adapters/desktop/`](./docs/adapters/desktop/).
## Download Support
@@ -325,6 +208,7 @@ OpenCLI supports downloading images, videos, and articles from supported platfor
| Platform | Content Types | Notes |
|----------|---------------|-------|
| **xiaohongshu** | Images, Videos | Downloads all media from a note |
| **rednote** | Images, Videos | Downloads all media from a signed rednote note URL |
| **bilibili** | Videos | Requires `yt-dlp` installed |
| **twitter** | Images, Videos | From user media tab or single tweet |
| **douban** | Images | Poster / still image lists |
@@ -339,6 +223,7 @@ For video downloads, install `yt-dlp` first: `brew install yt-dlp`
```bash
opencli xiaohongshu download "https://www.xiaohongshu.com/search_result/<id>?xsec_token=..." --output ./xhs
opencli xiaohongshu download "https://xhslink.com/..." --output ./xhs
opencli rednote download "https://www.rednote.com/search_result/<id>?xsec_token=..." --output ./rednote
opencli bilibili download BV1xxx --output ./bilibili
opencli twitter download elonmusk --limit 20 --output ./twitter
opencli 1688 download 841141931191 --output ./1688-downloads
@@ -400,18 +285,6 @@ opencli plugin uninstall my-tool
See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
## For AI Agents (Developer Guide)
Before writing any adapter code, read the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md). It takes you end-to-end:
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
- Discover the right endpoint via `opencli browser network`, `eval`, or the interceptor fallback.
- Decide auth strategy (`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`).
- Run `opencli browser analyze <url>` for one-shot recon, decode response fields, design columns, scaffold with `opencli browser init`.
- Verify with `opencli browser verify <site>/<name>` before shipping.
For long-lived personal commands that should live in your own Git repo, use a local plugin instead; see [Extending OpenCLI](./docs/guide/extending-opencli.md). Quick private adapters can still live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
## Testing
See **[TESTING.md](./TESTING.md)** for how to run and write tests.
+41 -228
View File
@@ -1,7 +1,8 @@
# OpenCLI
> **把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口。**
> 复用浏览器登录态,先自动化真实操作,再把高频流程沉淀成可复用的 CLI 命令
> **把任意网站变成 CLI & 在你的登录态浏览器上跑 Browser Use。**
> 把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口
> 或者在任意页面上跑 Browser Use —— 导航、填表单、点击、抓取、自动化。
[![English](https://img.shields.io/badge/docs-English-1D4ED8?style=flat-square)](./README.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
@@ -11,21 +12,10 @@
OpenCLI 可以用同一套 CLI 做三类事情:
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [100+ 站点](#内置命令) 开箱即用。
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-adapter-author` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-browser` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
- **把新网站写成 CLI**:用 `opencli browser` 原语 + `opencli-adapter-author` skill,从站点侦察、API 发现、字段解码到 `opencli browser verify` 一条龙。
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh``docker` 等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT、Notion 等 Electron 应用。
## 亮点
- **桌面应用控制** — 通过 CDP 直接在终端驱动 Electron 应用(Cursor、Codex、ChatGPT、Notion 等)。
- **AI Agent 浏览器自动化** — 安装 `opencli-adapter-author` skill,你的 AI Agent 就能操作任意网站:导航、点击、输入/填充、提取、截图——全部通过你的已登录 Chrome 会话完成。
- **网站 → CLI** — 把任何网站变成确定性 CLI:100+ 站点能力已注册,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
- **账号安全** — 复用 Chrome/Chromium 登录态,凭证永远不会离开浏览器。
- **面向 AI Agent** — 一个 skill 带你走完站点侦察、API 发现、字段解码、适配器编写、验证的全流程。
- **CLI 枢纽** — 统一发现、自动安装、纯透传任何外部 CLI(gh、docker、obsidian 等)。
- **零 LLM 成本** — 运行时不消耗模型 token,跑 10,000 次也不花一分钱。
- **确定性输出** — 相同命令,相同输出结构,每次一致。可管道、可脚本、CI 友好。
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh``docker``longbridge``tg``discord``wx``ntn`Notion等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT 等 Electron 应用。
## 快速开始
@@ -89,7 +79,7 @@ opencli bilibili hot --limit 5
OpenCLI 的 browser 命令是给 AI Agent 用的——不是手动执行的。把 skill 安装到你的 AI AgentClaude Code、Cursor 等)中,Agent 就能用你的已登录 Chrome 会话替你操作网站。
### 安装 skill
### 安装 skill(同时也用于更新)
```bash
npx skills add jackwener/opencli
@@ -102,22 +92,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
### 选择哪个 skill
| Skill | 适用场景 | 你对 AI Agent 说的话 |
|-------|---------|-------------------|
| **opencli-adapter-author** | 实时操作任意网站,或为新站点写可复用适配器 | "帮我看看小红书的通知" / "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
| **opencli-adapter-author** | 为新站点写可复用适配器,或给已有站点添加命令 | "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
| **opencli-autofix** | 内置命令失败时修复已有适配器 | "`opencli zhihu hot` 返回空了,修一下" |
| **opencli-browser** | 浏览器自动化参考文档 | "用浏览器命令抓取这个页面" |
| **opencli-browser** | 实时驱动 Chrome 页面——导航、填表单、点击、抓取 | "帮我看看小红书的通知" / "帮我填一下这个表单" / "用浏览器命令抓取这个页面" |
| **opencli-usage** | 所有命令和站点的快速参考 | "OpenCLI 有哪些 Twitter 相关的命令?" |
| **smart-search** | 在现有 OpenCLI 能力里搜索 | "帮我找个 B 站热门相关的适配器" |
### 工作原理
安装 `opencli-adapter-author` skill 后,你的 AI Agent 可以:
安装 `opencli-browser` skill 后,你的 AI Agent 可以:
1. **导航**到任意 URL,使用你的已登录浏览器
2. **读取**页面内容——通过结构化 DOM 快照(不是截图)
@@ -128,45 +116,25 @@ npx skills add jackwener/opencli --skill smart-search
Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自然语言描述想做的事。
**Skill 参考文档:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 浏览器操作 + 适配器编写,全流程
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 实时驱动 Chrome(导航、填表单、点击、抓取)
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 给新站点写适配器,全流程
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — 修复已有适配器
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 浏览器自动化参考
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — 命令和站点参考
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — 能力搜索
`browser` 可用命令包括:`open``state``click``type``fill``select``keys``wait``get``find``extract``frames``screenshot``scroll``back``eval``network``tab list``tab new``tab select``tab close``init``verify``close`
`opencli browser open <url>``opencli browser tab new [url]` 都会返回 target ID。`opencli browser tab list` 用来查看当前已存在 tab 的 target ID,再通过 `--tab <targetId>` 把命令明确路由到某个 tab。`tab new` 只会新建 tab,不会改变默认浏览器目标;只有显式执行 `tab select <targetId>`,才会把该 tab 设为后续未指定 target 的 `opencli browser ...` 命令的默认目标。
`opencli browser` 命令必须紧跟一个 `<session>` 位置参数。`opencli browser work open <url>``opencli browser work tab new [url]` 都会返回 target ID。`opencli browser work tab list` 用来查看当前已存在 tab 的 target ID,再通过 `--tab <targetId>` 把命令明确路由到某个 tab。`tab new` 只会新建 tab,不会改变默认浏览器目标;只有显式执行 `tab select <targetId>`,才会把该 tab 设为同一 session 后续未指定 target 的默认目标。
## 核心概念
## 为新站点写适配器
### `browser`AI Agent 的浏览器控制层
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,全流程:
`opencli browser` 命令是 AI Agent 操作网站的底层原语。你不需要手动运行这些命令——把 `opencli-adapter-author` skill 安装到你的 AI Agent 中,用自然语言描述你想做的事,Agent 会自动处理浏览器操作。
比如你告诉 Agent:*"帮我看看小红书的通知"*——Agent 会在底层调用 `opencli browser open``state``click` 等命令。
### 内置适配器:稳定命令
当某个站点能力已经存在时,优先使用 `opencli hackernews top``opencli reddit hot` 这类稳定命令。这些命令是确定性的,无需浏览器——人类和 AI Agent 都可以直接使用。
### 为新站点写适配器
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,它会把 Agent 带到闭环:
1. 侦察站点,分类 patternSPA / SSR / JSONP / Token / Streaming
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. 定认证策略——`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
4. 字段解码 + 设计输出列
5. `opencli browser analyze <url>` 一步侦察,再 `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
### CLI 枢纽与桌面端适配器
OpenCLI 不只是网站 CLI,还可以:
- 统一代理本地二进制工具,例如 `gh``docker``obsidian`
- 通过专门适配器和 CDP 集成控制 Electron 桌面应用
1. **侦察**站点,分类 patternSPA / SSR / JSONP / Token / Streaming
2. **发现** endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. **定认证**——`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
4. **字段解码** + 设计输出列
5. `opencli browser recon analyze <url>``opencli browser recon init <site>/<name>` → 写适配器 → `opencli browser recon verify <site>/<name>`
6. 站点知识沉到 `~/.opencli/sites/<site>/`,下次同站点直接吃缓存
## 前置要求
@@ -181,8 +149,7 @@ OpenCLI 不只是网站 CLI,还可以:
| 变量 | 默认值 | 说明 |
|------|--------|------|
| `OPENCLI_DAEMON_PORT` | `19825` | daemon-extension 通信端口 |
| `OPENCLI_WINDOW_FOCUSED` | `false` | 设为 `1` 时 automation 窗口在前台打开(适合调试)。`--focus` 标志会设置此变量 |
| `OPENCLI_LIVE` | `false` | 设为 `1` 时 adapter 命令执行完后保留 automation 窗口不关闭(适合检查页面)。`--live` 标志会设置此变量 |
| `OPENCLI_WINDOW` | 命令默认值 | 设为 `foreground``background` 来覆盖 Browser Bridge 窗口位置。浏览器型命令也支持 `--window <foreground\|background>` |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | 浏览器连接超时(秒) |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | 单个浏览器命令超时(秒) |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol 端点,用于远程浏览器或 Electron 应用 |
@@ -190,178 +157,37 @@ OpenCLI 不只是网站 CLI,还可以:
| `OPENCLI_VERBOSE` | `false` | 启用详细日志(`-v` 也可以) |
| `DEBUG_SNAPSHOT` | — | 设为 `1` 输出 DOM 快照调试信息 |
`--focus` 同时适用于 `opencli browser *` 和浏览器型 adapter 命令。`--live` 主要是给 adapter 命令用的:`browser` 子命令本来就会一直保留 automation window,直到你手动执行 `opencli browser close` 或等空闲超时。
## 更新
```bash
npm install -g @jackwener/opencli@latest
# 如果你在用打包发布的 OpenCLI skills,也一起刷新
npx skills add jackwener/opencli
```
如果你只装了部分 skill,也可以只刷新自己在用的:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill smart-search
```
## 面向开发者
从源码安装:
```bash
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link
```
加载源码版 Browser Bridge 扩展:
1. 打开 `chrome://extensions` 并启用 **开发者模式**
2. 点击 **加载已解压的扩展程序**,选择本仓库里的 `extension/` 目录
`opencli browser *` 必须紧跟一个 `<session>` 位置参数,默认使用前台窗口,并保留该 session 的 tab lease,直到你手动执行 `opencli browser <session> close` 或等空闲超时。浏览器型 adapter 默认使用后台 adapter 窗口并在命令结束后释放一次性 tab lease;如果需要调试最终页面,可以传 `--window foreground --keep-tab true`
## 内置命令
运行 `opencli list` 查看完整注册表。
| 站点 | 命令 | 模式 |
|------|------|------|
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` | 浏览器 |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` | 浏览器 |
| **tieba** | `hot` `posts` `search` `read` | 浏览器 |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` | 浏览器 |
| **cursor** | `status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export` | 桌面端 |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` | 浏览器 |
| **codex** | `status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `projects` `history` `export` | 桌面端 |
| **chatwise** | `status` `new` `send` `read` `ask` `model` `history` `export` `screenshot` | 桌面端 |
| **doubao** | `status` `new` `send` `read` `ask` `history` `detail` `meeting-summary` `meeting-transcript` | 浏览器 |
| **doubao-app** | `status` `new` `send` `read` `ask` `screenshot` `dump` | 桌面端 |
| **notion** | `status` `search` `read` `new` `write` `sidebar` `favorites` `export` | 桌面端 |
| **discord-app** | `status` `send` `read` `channels` `servers` `search` `members` | 桌面端 |
| **v2ex** | `hot` `latest` `topic` `node` `user` `member` `replies` `nodes` `daily` `me` `notifications` | 公开 / 浏览器 |
| **xueqiu** | `feed` `hot-stock` `hot` `search` `stock` `comments` `watchlist` `earnings-date` `fund-holdings` `fund-snapshot` | 浏览器 |
| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` `serve` | 桌面端 |
| **chatgpt-app** | `status` `new` `send` `read` `ask` `model` | 桌面端 |
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` | 浏览器 |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` | 浏览器 |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` | 浏览器 |
| **uiverse** | `code` `preview` | 浏览器 |
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
| **baidu-scholar** | `search` | 公开 |
| **google-scholar** | `search` `cite` `profile` | 公开 |
| **gov-law** | `search` `recent` | 公开 |
| **gov-policy** | `search` `recent` | 公开 |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
| **wanfang** | `search` | 公开 |
| **xiaoyuzhou** | `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` `auth` | 本地凭证 |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` | 浏览器 |
| **weixin** | `download` | 浏览器 |
| **youtube** | `search` `video` `transcript` `comments` `channel` `playlist` `feed` `history` `watch-later` `subscriptions` `like` `unlike` `subscribe` `unsubscribe` | 浏览器 |
| **boss** | `search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats` | 浏览器 |
| **coupang** | `search` `add-to-cart` | 浏览器 |
| **bbc** | `news` | 公共 API |
| **bloomberg** | `main` `markets` `economics` `industries` `tech` `politics` `businessweek` `opinions` `feeds` `news` | 公共 API / 浏览器 |
| **ctrip** | `search` | 浏览器 |
| **devto** | `top` `tag` `user` | 公开 |
| **dictionary** | `search` `synonyms` `examples` | 公开 |
| **arxiv** | `search` `paper` | 公开 |
| **pubmed** | `search` `article` `author` `citations` `related` | 公开 |
| **openreview** | `search` `venue` `paper` `reviews` | 公开 |
| **paperreview** | `submit` `review` `feedback` | 公开 |
| **wikipedia** | `search` `summary` `random` `trending` | 公开 |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` | 公共 API |
| **jd** | `item` | 浏览器 |
| **linkedin** | `search` `timeline` | 浏览器 |
| **reuters** | `search` | 浏览器 |
| **smzdm** | `search` | 浏览器 |
| **web** | `read` | 浏览器 |
| **weibo** | `hot` `search` `feed` `user` `me` `post` `comments` | 浏览器 |
| **yahoo-finance** | `quote` | 浏览器 |
| **sinafinance** | `news` | 🌐 公开 |
| **barchart** | `quote` `options` `greeks` `flow` | 浏览器 |
| **chaoxing** | `assignments` `exams` | 浏览器 |
| **grok** | `ask` `image` | 浏览器 |
| **hf** | `top` | 公开 |
| **jike** | `feed` `search` `create` `like` `comment` `repost` `notifications` `post` `topic` `user` | 浏览器 |
| **jimeng** | `generate` `history` | 浏览器 |
| **yollomi** | `generate` `video` `edit` `upload` `models` `remove-bg` `upscale` `face-swap` `restore` `try-on` `background` `object-remover` | 浏览器 |
| **linux-do** | `hot` `latest` `feed` `search` `categories` `category` `tags` `topic` `topic-content` `user-posts` `user-topics` | 浏览器 |
| **stackoverflow** | `hot` `search` `bounties` `unanswered` | 公开 |
| **steam** | `top-sellers` | 公开 |
| **weread** | `shelf` `search` `book` `highlights` `notes` `notebooks` `ranking` | 浏览器 |
| **douban** | `search` `top250` `subject` `photos` `download` `marks` `reviews` `movie-hot` `book-hot` | 浏览器 |
| **facebook** | `feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group` | 浏览器 |
| **google** | `news` `search` `suggest` `trends` | 公开 |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` | 浏览器 |
| **1688** | `search` `item` `assets` `download` `store` | 浏览器 |
| **gitee** | `trending` `search` `user` | 公开 / 浏览器 |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` | 浏览器 |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` | 浏览器 |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` | OAuth API |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` | 浏览器 |
| **36kr** | `news` `hot` `search` `article` | 公开 / 浏览器 |
| **imdb** | `search` `title` `top` `trending` `person` `reviews` | 公开 |
| **producthunt** | `posts` `today` `hot` `browse` | 公开 / 浏览器 |
| **instagram** | `explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved` | 浏览器 |
| **lobsters** | `hot` `newest` `active` `tag` `read` | 公开 |
| **medium** | `feed` `search` `user` | 浏览器 |
| **sinablog** | `hot` `search` `article` `user` | 浏览器 |
| **substack** | `feed` `search` `publication` | 浏览器 |
| **pixiv** | `ranking` `search` `user` `illusts` `detail` `download` | 浏览器 |
| **tiktok** | `explore` `search` `profile` `user` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `live` `notifications` `friends` | 浏览器 |
| **bluesky** | `search` `trending` `user` `profile` `thread` `feeds` `followers` `following` `starter-packs` | 公开 |
| **xianyu** | `search` `item` `chat` `publish` | 浏览器 |
| **douyin** | `videos` `publish` `drafts` `draft` `delete` `stats` `profile` `update` `hashtag` `location` `activities` `collections` | 浏览器 |
| **yuanbao** | `new` `ask` | 浏览器 |
| 站点 | 命令 |
|------|------|
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `summary` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **linkedin** | `connect` `inbox` `safe-send` `search` `people-search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
100+ 站点能力**[→ 查看完整命令列表](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast``podcast-episodes``episode``download``transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`
精选清单**[→ 查看全部 100+ 站点和命令](./docs/adapters/index.md)**(小红书 / B站 / 知乎 / Twitter / Reddit / 抖音 / 微博 / 微信读书 / 小宇宙 / 1688 / 夸克 / Spotify / 牛客 / arxiv / Bilibili / 等)。
### 外部 CLI 枢纽
OpenCLI 也可以作为你现有命令行工具统一入口,负责发现、自动安装和纯透传执行。
现有命令行工具统一接入 `opencli <tool> ...`
| 外部 CLI | 描述 | 示例 |
|----------|------|------|
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
| **obsidian** | Obsidian 仓库管理 | `opencli obsidian search query="AI"` |
| **docker** | Docker 命令行工具 | `opencli docker ps` |
| **lark-cli** | 飞书 CLI — 消息、文档、日历、任务,200+ 命令 | `opencli lark-cli calendar +agenda` |
| **dws** | 钉钉 CLI — 钉钉全套产品能力的跨平台命令行工具,支持人类和 AI Agent 使用 | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | 企业微信 CLI — 企业微信开放平台命令行工具,支持人类和 AI Agent 使用 | `opencli wecom-cli msg send --to user "hello"` |
| **vercel** | Vercel — 部署项目、管理域名、环境变量、日志 | `opencli vercel deploy --prod` |
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
**零配置透传**:OpenCLI 会把你的输入原样转发给底层二进制,保留原生 stdout / stderr 行为
注册自定义本地 CLI`opencli external register <name>`;查看所有:`opencli external list`
**自动安装**:如果你运行 `opencli gh ...` 时系统中还没有 `gh`,OpenCLI 会优先尝试通过系统包管理器安装,然后自动重试命令
**注册自定义本地 CLI**
```bash
opencli register mycli
```
### 桌面应用适配器
每个桌面适配器都有自己详细的文档说明,包括命令参考、启动配置与使用示例:
| 应用 | 描述 | 文档 |
|-----|-------------|-----|
| **Cursor** | 控制 Cursor IDE — Composer、对话、代码提取等 | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | 在后台(无头)驱动 OpenAI Codex CLI Agent | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | 在终端直接控制 Antigravity Ultra | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT App** | 自动化操作 ChatGPT macOS 桌面客户端 | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | 多 LLM 客户端(GPT-4、Claude、Gemini | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | 搜索、读取、写入 Notion 页面 | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord 桌面版 — 消息、频道、服务器 | [Doc](./docs/adapters/desktop/discord.md) |
| **Doubao** | 通过 CDP 控制豆包桌面应用 | [Doc](./docs/adapters/desktop/doubao-app.md) |
**桌面应用适配器**Electron,通过 CDP):Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — 详见 [`docs/adapters/desktop/`](./docs/adapters/desktop/)
## 下载支持
@@ -398,6 +224,7 @@ brew install yt-dlp
# 下载小红书笔记中的图片/视频
opencli xiaohongshu download "https://www.xiaohongshu.com/search_result/<id>?xsec_token=..." --output ./xhs
opencli xiaohongshu download "https://xhslink.com/..." --output ./xhs
opencli rednote download "https://www.rednote.com/search_result/<id>?xsec_token=..." --output ./rednote
# 下载B站视频(需要 yt-dlp
opencli bilibili download BV1xxx --output ./bilibili
@@ -498,20 +325,6 @@ opencli plugin uninstall my-tool # 卸载
详见 [插件指南](./docs/zh/guide/plugins.md) 了解如何创建自己的插件。
## 致 AI Agent(开发者指南)
如果你是一个被要求查阅代码并编写新 `opencli` 适配器的 AI,请遵守以下工作流。
在动代码前,先读 [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md)。它把整个流程串起来:
- 侦察站点,选定 patternSPA / SSR / JSONP / Token / Streaming
-`opencli browser network``eval`、interceptor 等找到目标 endpoint
- 定认证策略(`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
- 先用 `opencli browser analyze <url>` 一步侦察,再字段解码、设计 columns、`opencli browser init` 生成骨架
- 交付前用 `opencli browser verify <site>/<name>` 验证
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。
## 常见问题排查
- **"Extension not connected" 报错**
+4372 -462
View File
File diff suppressed because it is too large Load Diff
+73
View File
@@ -0,0 +1,73 @@
/**
* 12306 account summary for the logged-in user.
*
* Returns non-sensitive identity fields plus masked email / mobile.
* Use `--include-sensitive` to surface unmasked values from 12306's
* own response (12306 already masks the ID number server-side; this
* adapter never decodes that mask).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskEmail, maskMobile, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const ACCOUNT_INFO_URL = 'https://kyfw.12306.cn/otn/modifyUser/initQueryUserInfoApi';
cli({
site: '12306',
name: 'me',
access: 'read',
description: 'Show the logged-in 12306 account summary. Sensitive fields (real name, email, mobile, birth date) are masked by default; pass --include-sensitive to opt in.',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real name / email / mobile / birth date. The 12306 ID-number mask is server-side and never decoded.' },
],
columns: ['username', 'real_name', 'email', 'mobile', 'birth_date', 'sex', 'country', 'user_type', 'member', 'active'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 me');
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const json = requireEvaluateObject(await page.evaluate(`async () => {
const r = await fetch(${JSON.stringify(ACCOUNT_INFO_URL)}, { credentials: 'include' });
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'account info');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for account info`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 account info returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
}
if (json?.status !== true || !json?.data?.userDTO) {
throw new CommandExecutionError('12306 account info payload missing userDTO');
}
const dto = json.data.userDTO;
const loginDto = dto.loginUserDTO || {};
const username = loginDto.user_name || loginDto.name || '';
const realName = loginDto.real_name || loginDto.realname || '';
const include = kwargs['include-sensitive'] === true;
return [{
username,
real_name: include ? realName : maskChineseName(realName),
email: include ? (dto.email || '') : maskEmail(dto.email || ''),
mobile: include ? (dto.mobile_no || '') : maskMobile(dto.mobile_no || ''),
birth_date: include ? (dto.born_date || '') : (dto.born_date || '').slice(0, 4),
sex: dto.sex_code === 'M' ? '男' : (dto.sex_code === 'F' ? '女' : ''),
country: dto.country_code || '',
user_type: json.data.userTypeName || '',
member: dto.flag_member === '1',
active: dto.is_active === '1',
}];
},
});
+96
View File
@@ -0,0 +1,96 @@
/**
* 12306 in-progress orders for the logged-in user.
*
* Returns orders that have not yet been ridden / refunded / completed
* (the `noComplete` slice). Order history covering completed and
* refunded tickets uses a separate endpoint that requires extra
* referer / page-state handshakes and is left for a follow-up so this
* command can ship reliably.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const NO_COMPLETE_URL = 'https://kyfw.12306.cn/otn/queryOrder/queryMyOrderNoComplete';
cli({
site: '12306',
name: 'orders',
access: 'read',
description: 'List in-progress 12306 orders (not yet ridden, refunded, or completed) for the logged-in user',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked passenger names in order rows. Masked by default.' },
],
columns: ['order_id', 'order_date', 'train_code', 'from_station', 'to_station', 'departure', 'passengers', 'status', 'amount'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 orders');
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const include = kwargs['include-sensitive'] === true;
const json = requireEvaluateObject(await page.evaluate(`async () => {
const r = await fetch(${JSON.stringify(NO_COMPLETE_URL)}, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: '_json_att=', credentials: 'include',
});
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'orders');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for queryMyOrderNoComplete`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 orders returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
}
if (json?.status !== true) {
throw new CommandExecutionError('12306 queryMyOrderNoComplete returned a failure status');
}
let orders;
if (Array.isArray(json?.data?.orderDBList)) {
orders = json.data.orderDBList;
} else if (Array.isArray(json?.data?.orderDTODataList)) {
orders = json.data.orderDTODataList;
} else if (Array.isArray(json?.data?.orders)) {
orders = json.data.orders;
} else if (Array.isArray(json?.data)) {
orders = json.data;
} else {
throw new CommandExecutionError('12306 queryMyOrderNoComplete payload missing order list array');
}
if (orders.length === 0) {
throw new EmptyResultError('No in-progress 12306 orders on this account');
}
return orders.map((o) => {
const tickets = Array.isArray(o.tickets) ? o.tickets : [];
const passengerNames = tickets
.map((t) => t.passenger_name || '')
.filter(Boolean)
.map((name) => include ? name : maskChineseName(name))
.join(', ');
return {
order_id: o.sequence_no || o.order_id || o.sequenceNo || '',
order_date: o.order_date || '',
train_code: o.train_code_page || o.station_train_code || o.train_code || '',
from_station: o.from_station_name_page || o.from_station_name || '',
to_station: o.to_station_name_page || o.to_station_name || '',
departure: o.start_train_date_page || o.start_train_date || '',
passengers: passengerNames,
status: o.ticket_status_name || o.order_status_name || o.statusName || '',
amount: o.ticket_total_price_page || o.ticket_total_price || '',
};
});
},
});
+90
View File
@@ -0,0 +1,90 @@
/**
* 12306 saved passenger list for the logged-in user.
*
* 12306 already masks ID numbers (`xxxx***********xxx`) and mobile
* numbers (`138****xxxx`) server-side. This adapter further masks the
* passenger's Chinese real name and birth date by default; pass
* `--include-sensitive` to surface the unmasked-by-12306 fields.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const PASSENGER_QUERY_URL = 'https://kyfw.12306.cn/otn/passengers/query';
const MAX_PAGE_SIZE = 50;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) throw new ArgumentError(`limit must be a positive integer (1-${max})`);
if (n > max) throw new ArgumentError(`limit must be <= ${max}`);
return n;
}
cli({
site: '12306',
name: 'passengers',
access: 'read',
description: 'List the logged-in user\'s saved 12306 passengers. Sensitive fields are masked by default; pass --include-sensitive to opt in.',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'limit', type: 'int', default: 20, help: `Max passengers to return (1-${MAX_PAGE_SIZE})` },
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real names and birth dates. The 12306 ID-number / mobile masks are server-side and never decoded.' },
],
columns: ['name', 'sex', 'born_year', 'id_type', 'id_no', 'mobile', 'passenger_type', 'country'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 passengers');
const limit = normalizeLimit(kwargs.limit, 20, MAX_PAGE_SIZE);
const include = kwargs['include-sensitive'] === true;
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const json = requireEvaluateObject(await page.evaluate(`async () => {
const body = "pageIndex=1&pageSize=${MAX_PAGE_SIZE}";
const r = await fetch(${JSON.stringify(PASSENGER_QUERY_URL)}, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body, credentials: 'include',
});
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'passengers');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for passengers/query`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 passengers returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
}
if (json?.status !== true || !Array.isArray(json?.data?.datas)) {
throw new CommandExecutionError('12306 passengers payload missing data.datas array');
}
const datas = json.data.datas;
if (datas.length === 0) {
throw new EmptyResultError('No saved passengers on this 12306 account');
}
return datas.slice(0, limit).map((p) => ({
name: include ? (p.passenger_name || '') : maskChineseName(p.passenger_name || ''),
sex: p.sex_name || '',
born_year: (p.born_date || '').slice(0, 4),
id_type: p.passenger_id_type_name || '',
id_no: p.passenger_id_no || '',
mobile: p.mobile_no || '',
passenger_type: p.passenger_type_name || '',
country: p.country_code || '',
}));
},
});
export const __test__ = { normalizeLimit };
+166
View File
@@ -0,0 +1,166 @@
/**
* 12306 ticket price lookup for a single train + segment.
*
* Cascades three anonymous API calls:
* 1. /otn/leftTicket/init: mint session cookies
* 2. /otn/czxx/queryByTrainNo: resolve from/to station_no within the
* train route (price endpoint addresses stops by station_no, not
* telecode)
* 3. /otn/leftTicket/queryTicketPrice: ticket prices keyed by seat
* letter (M=一等座, O=二等座, A9=商务座, A1=硬座, A3=硬卧,
* A4=软卧, F=动卧, P=特等座, WZ=无座, etc.)
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
const SEAT_TYPES_RE = /^[A-Z0-9]{1,32}$/;
const SEAT_LETTERS = {
'A9': '商务座',
'P': '特等座',
'M': '一等座',
'O': '二等座',
'A1': '硬座',
'A3': '硬卧',
'A4': '软卧',
'F': '动卧',
'WZ': '无座',
};
async function queryStopsForPrice(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
}
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
throw new CommandExecutionError('12306 queryByTrainNo returned an unexpected payload shape');
}
return json.data.data;
}
function pickStationNos(stops, fromCode, toCode, fromName, toName) {
const matches = (s, code, name) => (s.station_name && name && s.station_name === name);
const fromStop = stops.find((s) => matches(s, fromCode, fromName));
const toStop = stops.find((s) => matches(s, toCode, toName));
if (!fromStop) throw new CommandExecutionError(`Train does not stop at ${fromName}`);
if (!toStop) throw new CommandExecutionError(`Train does not stop at ${toName}`);
return { fromNo: fromStop.station_no, toNo: toStop.station_no };
}
async function queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/leftTicket/queryTicketPrice?train_no=${trainNo}&from_station_no=${fromNo}&to_station_no=${toNo}&seat_types=${seatTypes}&train_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) throw new CommandExecutionError(`12306 queryTicketPrice returned HTTP ${resp.status}`);
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryTicketPrice returned non-JSON body');
}
if (json?.status !== true || !json?.data) {
throw new CommandExecutionError('12306 queryTicketPrice returned an unexpected payload shape');
}
return json.data;
}
function parsePriceData(priceData) {
const rows = [];
for (const [letter, value] of Object.entries(priceData)) {
if (letter === 'train_no' || letter === 'OT') continue;
if (typeof value !== 'string' || !value) continue;
// 12306 doubles up some prices as bare numerics ("9": "21580"), which
// mirror their letter sibling ("A9": "¥2158.0") in cents/no-decimal
// form. Skip the bare numeric letter codes to avoid duplicates.
if (/^\d+$/.test(letter)) continue;
if (!/^[A-Z]/.test(letter)) continue;
const numeric = value.replace(/^¥/, '');
if (!/^[\d.]+$/.test(numeric)) continue;
rows.push({
seat_code: letter,
seat_name: SEAT_LETTERS[letter] || letter,
price: numeric,
currency: 'CNY',
});
}
rows.sort((a, b) => Number(b.price) - Number(a.price));
return rows;
}
cli({
site: '12306',
name: 'price',
access: 'read',
description: 'Look up 12306 ticket prices by seat class for one train on a given date and segment (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L)' },
{ name: 'from', required: true, help: 'Origin station (Chinese name, telecode, or pinyin) - must be a stop of this train' },
{ name: 'to', required: true, help: 'Destination station - must be a stop of this train' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
{ name: 'seat-types', default: 'OM9PA1A3A4FWZ', help: 'Seat-type letters to query (default covers the common classes). Examples: OM9 (二等/一等/商务), A1A3A4 (硬座/硬卧/软卧).' },
],
columns: ['seat_code', 'seat_name', 'price', 'currency'],
func: async (kwargs) => {
const trainNo = String(kwargs['train-no'] ?? '').trim();
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
if (!TRAIN_NO_RE.test(trainNo)) {
throw new ArgumentError(
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
);
}
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('--from station must not be empty');
if (!toArg) throw new ArgumentError('--to station must not be empty');
const date = validateDate(kwargs.date);
const seatTypes = String(kwargs['seat-types'] ?? '').trim() || 'OM9PA1A3A4FWZ';
if (!SEAT_TYPES_RE.test(seatTypes)) {
throw new ArgumentError('--seat-types must contain only 12306 seat letters/digits (A-Z, 0-9)');
}
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const cookieHeader = await mintSession();
const stops = await queryStopsForPrice(cookieHeader, trainNo, fromStation.code, toStation.code, date);
const { fromNo, toNo } = pickStationNos(stops, fromStation.code, toStation.code, fromStation.name, toStation.name);
const priceData = await queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date);
const rows = parsePriceData(priceData);
if (rows.length === 0) {
throw new EmptyResultError(
`No prices returned for train_no=${trainNo} ${fromStation.name} -> ${toStation.name} on ${date}`,
'Try a different seat-types letter set, or check that this train operates on the date.',
);
}
return rows;
},
});
export const __test__ = { parsePriceData, pickStationNos, queryStopsForPrice, queryPrice, SEAT_LETTERS };
+66
View File
@@ -0,0 +1,66 @@
/**
* 12306 station search.
*
* Queries the public `station_name.js` bundle and filters by the user's
* keyword. Anonymous, no session needed.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle } from './utils.js';
const MAX_LIMIT = 50;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) {
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
}
if (n > max) {
throw new ArgumentError(`limit must be <= ${max}`);
}
return n;
}
cli({
site: '12306',
name: 'stations',
access: 'read',
description: 'Search 12306 (China Railway) stations by Chinese name, telecode, or pinyin keyword',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Chinese substring (上海), telecode (AOH), or pinyin (shanghai)' },
{ name: 'limit', type: 'int', default: 20, help: `Maximum results (1-${MAX_LIMIT})` },
],
columns: ['name', 'code', 'pinyin', 'abbr', 'city'],
func: async (kwargs) => {
const keyword = String(kwargs.keyword ?? '').trim();
if (!keyword) throw new ArgumentError('keyword must not be empty');
const limit = normalizeLimit(kwargs.limit, 20, MAX_LIMIT);
const stations = await fetchStationBundle();
const lower = keyword.toLowerCase();
const matches = stations.filter((s) =>
s.name.includes(keyword)
|| s.code === keyword.toUpperCase()
|| s.pinyin.includes(lower)
|| s.abbr.includes(lower)
|| s.short.includes(lower)
|| s.city.includes(keyword),
);
if (matches.length === 0) {
throw new EmptyResultError(`No 12306 stations match "${keyword}"`);
}
return matches.slice(0, limit).map((s) => ({
name: s.name,
code: s.code,
pinyin: s.pinyin,
abbr: s.abbr,
city: s.city,
}));
},
});
export const __test__ = { normalizeLimit };
+91
View File
@@ -0,0 +1,91 @@
/**
* 12306 train stop details - list every station a train calls at,
* with arrival / departure / stopover time.
*
* Requires the internal `train_no` returned by `12306 trains`
* (`24000000G10L`), not the public train code (`G1`).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
async function queryStops(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) {
throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
}
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
}
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
throw new CommandExecutionError(`12306 queryByTrainNo returned an unexpected payload shape`);
}
return json.data.data;
}
cli({
site: '12306',
name: 'train',
access: 'read',
description: 'List every station a 12306 train calls at, with arrival / departure / stopover time (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L), not the public code (G1)' },
{ name: 'from', required: true, help: 'Origin station for the segment: Chinese name, telecode, or pinyin' },
{ name: 'to', required: true, help: 'Destination station for the segment' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
],
columns: ['station_no', 'station_name', 'arrive_time', 'start_time', 'stopover_time'],
func: async (kwargs) => {
const trainNo = String(kwargs['train-no'] ?? '').trim();
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
if (!TRAIN_NO_RE.test(trainNo)) {
throw new ArgumentError(
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
);
}
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('--from station must not be empty');
if (!toArg) throw new ArgumentError('--to station must not be empty');
const date = validateDate(kwargs.date);
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const cookieHeader = await mintSession();
const stops = await queryStops(cookieHeader, trainNo, fromStation.code, toStation.code, date);
if (stops.length === 0) {
throw new EmptyResultError(`No stops returned for train_no=${trainNo} on ${date}`);
}
return stops.map((s) => ({
station_no: s.station_no || '',
station_name: s.station_name || '',
arrive_time: s.arrive_time === '----' ? '' : (s.arrive_time || ''),
start_time: s.start_time === '----' ? '' : (s.start_time || ''),
stopover_time: s.stopover_time === '----' ? '' : (s.stopover_time || ''),
}));
},
});
export const __test__ = { queryStops, TRAIN_NO_RE };
+119
View File
@@ -0,0 +1,119 @@
/**
* 12306 train availability between two stations on a given date.
*
* Flow:
* 1. Fetch the station bundle (cached implicitly via per-process module state).
* 2. Mint anonymous session cookies via /otn/leftTicket/init.
* 3. Query /otn/leftTicket/queryG; if 12306 returns
* `{c_url: "leftTicket/queryX"}` (endpoint rotation), retry once
* against the suggested name.
* 4. Parse the `|`-separated train records.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate, parseTrainRecord } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const QUERY_ENDPOINTS = ['queryG', 'queryO', 'queryZ', 'queryA'];
const MAX_LIMIT = 100;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) {
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
}
if (n > max) {
throw new ArgumentError(`limit must be <= ${max}`);
}
return n;
}
async function queryLeftTickets(cookieHeader, fromCode, toCode, date) {
const headers = {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
};
const queryParams = `leftTicketDTO.train_date=${date}&leftTicketDTO.from_station=${fromCode}&leftTicketDTO.to_station=${toCode}&purpose_codes=ADULT`;
let lastResponseText = '';
for (const endpoint of QUERY_ENDPOINTS) {
const url = `https://kyfw.12306.cn/otn/leftTicket/${endpoint}?${queryParams}`;
const resp = await fetch(url, { headers });
if (!resp.ok) {
if (resp.status === 302) continue;
throw new CommandExecutionError(`12306 ${endpoint} returned HTTP ${resp.status}`);
}
const text = await resp.text();
lastResponseText = text;
let json;
try { json = JSON.parse(text); } catch {
throw new CommandExecutionError(`12306 ${endpoint} returned non-JSON body`);
}
if (json?.c_url && typeof json.c_url === 'string') {
const rotated = json.c_url.replace('leftTicket/', '').trim();
if (rotated && !QUERY_ENDPOINTS.includes(rotated)) {
QUERY_ENDPOINTS.unshift(rotated);
}
continue;
}
if (Array.isArray(json?.data?.result)) {
return json.data.result;
}
throw new CommandExecutionError(`12306 ${endpoint} returned an unexpected payload shape`);
}
throw new CommandExecutionError(`12306 rejected every known query endpoint name (${QUERY_ENDPOINTS.join(', ')}); the wire protocol may have changed. Last body: ${lastResponseText.slice(0, 200)}`);
}
cli({
site: '12306',
name: 'trains',
access: 'read',
description: 'List trains between two 12306 stations on a given date (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'from', positional: true, required: true, help: 'Origin station: Chinese name (北京), telecode (BJP), or pinyin (beijing)' },
{ name: 'to', positional: true, required: true, help: 'Destination station: same forms as <from>' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
{ name: 'limit', type: 'int', default: 50, help: `Maximum rows (1-${MAX_LIMIT})` },
],
columns: [
'code', 'from_station', 'to_station', 'start_time', 'arrive_time',
'duration', 'available', 'business_seat', 'first_seat', 'second_seat',
'soft_sleeper', 'hard_sleeper', 'hard_seat', 'no_seat', 'train_no',
],
func: async (kwargs) => {
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('<from> station must not be empty');
if (!toArg) throw new ArgumentError('<to> station must not be empty');
const date = validateDate(kwargs.date);
const limit = normalizeLimit(kwargs.limit, 50, MAX_LIMIT);
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`<from> and <to> must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const stationByCode = new Map(stations.map((s) => [s.code, s]));
const cookieHeader = await mintSession();
const rawRows = await queryLeftTickets(cookieHeader, fromStation.code, toStation.code, date);
const decoded = rawRows
.map((line) => parseTrainRecord(decodeURIComponent(line.replace(/%0A/g, '')), stationByCode))
.filter(Boolean);
if (decoded.length === 0) {
throw new EmptyResultError(
`No trains found from ${fromStation.name} to ${toStation.name} on ${date}`,
'Try a different date or check whether the route is operated by 12306.',
);
}
return decoded.slice(0, limit);
},
});
export const __test__ = { normalizeLimit, queryLeftTickets };
+272
View File
@@ -0,0 +1,272 @@
/**
* 12306 (中国铁路) shared helpers.
*
* - Station lookup: parses the public `station_name.js` bundle into
* structured records.
* - Cookie session: 12306's query endpoints reject anonymous requests
* with `HTTP 302 -> error.html`, so callers must hit `/otn/leftTicket/init`
* first to mint the JSESSIONID / route / BIGipServerotn cookies.
* - Query endpoint rotation: 12306 rotates the train-query endpoint
* name (queryO / queryZ / queryA / queryG / ...) every few weeks.
* When the wrong name is hit, the server returns
* `{"c_url":"leftTicket/queryG","c_name":"CLeftTicketUrl","status":false}`
* pointing to the current correct name; retry once with that name.
*/
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
const STATION_BUNDLE_URL = 'https://kyfw.12306.cn/otn/resources/js/framework/station_name.js';
const INIT_URL = 'https://kyfw.12306.cn/otn/leftTicket/init';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
const STATION_CODE_RE = /^[A-Z]{2,4}$/;
/**
* Parse the `station_name.js` bundle into a station record array.
*
* Bundle format (single line, `@`-delimited records, each `|`-delimited):
* `var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||...';`
*
* Per-record fields (positional):
* [0] short pinyin alias (e.g. `bjb`)
* [1] Chinese station name (e.g. `北京北`)
* [2] telecode (3-4 uppercase letters, e.g. `VAP`) - this is the
* wire format 12306 uses for `from_station` / `to_station`.
* [3] full pinyin (e.g. `beijingbei`)
* [4] short alias (duplicate of [0] usually)
* [5] index/rank
* [6] city code
* [7] city name (e.g. `北京`)
*/
export function parseStationBundle(text) {
const match = text.match(/'([^']+)'/);
if (!match) {
throw new CommandExecutionError('Failed to parse 12306 station_name.js: source string not found');
}
const raw = match[1];
const records = raw.split('@').filter(Boolean);
const stations = [];
for (const r of records) {
const parts = r.split('|');
if (parts.length < 8 || !parts[2]) continue;
stations.push({
short: parts[0] || '',
name: parts[1] || '',
code: parts[2] || '',
pinyin: parts[3] || '',
abbr: parts[4] || '',
city: parts[7] || '',
});
}
if (stations.length === 0) {
throw new CommandExecutionError('Failed to parse 12306 station_name.js: no station records found');
}
return stations;
}
/**
* Resolve a user-supplied station identifier to a telecode.
*
* Accepts Chinese name (`上海虹桥`), telecode (`AOH`), pinyin
* (`shanghaihongqiao`), short alias (`shh`), or city name with a
* preference for the city's main station.
*/
export function resolveStation(stations, input) {
const trimmed = String(input ?? '').trim();
if (!trimmed) throw new ArgumentError('station must not be empty');
if (STATION_CODE_RE.test(trimmed)) {
const exact = stations.find((s) => s.code === trimmed);
if (exact) return exact;
throw new ArgumentError(`Unknown 12306 station telecode "${trimmed}"`);
}
const lower = trimmed.toLowerCase();
const exactName = stations.find((s) => s.name === trimmed);
if (exactName) return exactName;
const exactPinyin = stations.find((s) => s.pinyin === lower);
if (exactPinyin) return exactPinyin;
const exactAbbr = stations.find((s) => s.abbr === lower || s.short === lower);
if (exactAbbr) return exactAbbr;
throw new ArgumentError(`Unknown 12306 station "${trimmed}"`, 'Try the Chinese name (上海虹桥), the 3-4 letter telecode (AOH), or full pinyin (shanghaihongqiao).');
}
export function validateDate(value) {
if (!DATE_RE.test(String(value ?? ''))) {
throw new ArgumentError(`date must be YYYY-MM-DD, got "${value}"`);
}
const [y, m, d] = value.split('-').map(Number);
const date = new Date(Date.UTC(y, m - 1, d));
if (date.getUTCFullYear() !== y || date.getUTCMonth() !== m - 1 || date.getUTCDate() !== d) {
throw new ArgumentError(`date "${value}" is not a real calendar date`);
}
return value;
}
/** Extract Set-Cookie header values into a single `Cookie:` header string. */
export function buildCookieHeader(setCookieHeaders) {
if (!Array.isArray(setCookieHeaders) || setCookieHeaders.length === 0) return '';
return setCookieHeaders
.map((line) => line.split(';')[0])
.filter(Boolean)
.join('; ');
}
export async function fetchStationBundle(fetchImpl = fetch) {
const resp = await fetchImpl(STATION_BUNDLE_URL, {
headers: { 'User-Agent': UA },
});
if (!resp.ok) {
throw new CommandExecutionError(`Failed to fetch 12306 station bundle: HTTP ${resp.status}`);
}
return parseStationBundle(await resp.text());
}
/** Mint a 12306 anonymous session by hitting /otn/leftTicket/init. */
export async function mintSession(fetchImpl = fetch) {
const resp = await fetchImpl(INIT_URL, {
headers: { 'User-Agent': UA },
redirect: 'follow',
});
if (!resp.ok) {
throw new CommandExecutionError(`Failed to mint 12306 session: HTTP ${resp.status}`);
}
const setCookies = typeof resp.headers.getSetCookie === 'function'
? resp.headers.getSetCookie()
: resp.headers.raw?.()['set-cookie'] || [];
const cookieHeader = buildCookieHeader(setCookies);
if (!cookieHeader) {
throw new CommandExecutionError('12306 init returned no session cookies');
}
return cookieHeader;
}
/**
* Twelve-row train query record (LEFT_TICKET_DTO).
*
* 12306 returns each train as a `|`-separated string with ~36 fields.
* Positions used here come from the public web client; unused
* positions are documented inline so future maintainers can extend
* the row shape without re-reverse-engineering.
*/
export function parseTrainRecord(line, stationByCode) {
const f = line.split('|');
if (f.length < 33) return null;
return {
train_no: f[2] || '',
code: f[3] || '',
from_station: stationByCode.get(f[6])?.name || f[6] || '',
to_station: stationByCode.get(f[7])?.name || f[7] || '',
from_code: f[6] || '',
to_code: f[7] || '',
start_time: f[8] || '',
arrive_time: f[9] || '',
duration: f[10] || '',
available: (f[1] || '').trim() === '预订' || (f[11] || '').trim() === 'Y',
business_seat: f[32] || '',
first_seat: f[31] || '',
second_seat: f[30] || '',
soft_sleeper: f[23] || '',
hard_sleeper: f[28] || '',
hard_seat: f[29] || '',
no_seat: f[26] || '',
};
}
/**
* Mask helpers for sensitive identity fields rendered by 12306.
*
* 12306 already masks ID numbers and mobile numbers server-side
* (`xxxx***********xxx` / `138****xxxx`); these helpers handle the
* remaining fields (email, real Chinese name) so the adapter never
* leaks unmasked PII without an explicit `--include-sensitive` opt-in.
*/
export function maskEmail(value) {
const v = String(value || '').trim();
if (!v) return '';
const at = v.indexOf('@');
if (at <= 0) return v;
const local = v.slice(0, at);
const domain = v.slice(at);
if (local.length <= 2) return local[0] + '*' + domain;
return local[0] + '*'.repeat(Math.max(1, local.length - 2)) + local.slice(-1) + domain;
}
export function maskMobile(value) {
const v = String(value || '').trim();
if (!v) return '';
if (/\*/.test(v)) return v;
if (v.length < 7) return v.replace(/.(?=.)/g, '*');
return v.slice(0, 3) + '*'.repeat(v.length - 7) + v.slice(-4);
}
export function maskChineseName(value) {
const v = String(value || '').trim();
if (!v) return '';
if (v.length === 1) return v;
if (v.length === 2) return v[0] + '*';
return v[0] + '*'.repeat(v.length - 2) + v.slice(-1);
}
export function unwrapEvaluateResult(value) {
if (
value
&& typeof value === 'object'
&& !Array.isArray(value)
&& Object.prototype.hasOwnProperty.call(value, 'session')
&& Object.prototype.hasOwnProperty.call(value, 'data')
) {
return value.data;
}
return value;
}
export function requireEvaluateObject(value, label) {
const payload = unwrapEvaluateResult(value);
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
throw new CommandExecutionError(`12306 ${label} returned a malformed browser payload`);
}
return payload;
}
export function isAuthLikePayload(payload) {
if (!payload || typeof payload !== 'object') return false;
const parts = [];
if (Array.isArray(payload.messages)) parts.push(...payload.messages);
if (payload.message) parts.push(payload.message);
if (payload.msg) parts.push(payload.msg);
if (payload.validateMessages && typeof payload.validateMessages === 'object') {
parts.push(...Object.values(payload.validateMessages).flat());
}
const text = parts.map((item) => String(item ?? '')).join(' ');
return /未登录|登录|请登录|身份|认证|session|Session|login/i.test(text);
}
/**
* Detect the 12306 login marker by reading `document.cookie` from the
* current adapter page. Cannot use `page.getCookies({url})` here:
* 12306 sets the auth cookie `tk` and `JSESSIONID` with `Path=/otn`,
* and CDP `Network.getCookies` with a bare URL filter excludes
* cookies whose path does not match the URL path. `document.cookie`
* returns all non-httponly cookies visible to the current page
* regardless of path, which is what we need to confirm login.
*/
export async function require12306Login(page, AuthRequiredErrorClass) {
const docCookie = unwrapEvaluateResult(await page.evaluate(`document.cookie || ''`));
const cookieStr = typeof docCookie === 'string' ? docCookie : '';
if (!/\btk=/.test(cookieStr) || !/JSESSIONID=/.test(cookieStr)) {
throw new AuthRequiredErrorClass('kyfw.12306.cn', 'Not logged into 12306. Sign in at https://kyfw.12306.cn first.');
}
}
export const __test__ = {
parseStationBundle,
resolveStation,
validateDate,
buildCookieHeader,
parseTrainRecord,
maskEmail,
maskMobile,
maskChineseName,
unwrapEvaluateResult,
requireEvaluateObject,
isAuthLikePayload,
};
+331
View File
@@ -0,0 +1,331 @@
import { describe, expect, it } from 'vitest';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { __test__ } from './utils.js';
import { __test__ as priceTest } from './price.js';
import { __test__ as trainTest } from './train.js';
import './orders.js';
const { parseStationBundle, resolveStation, validateDate, buildCookieHeader, parseTrainRecord, maskEmail, maskMobile, maskChineseName, unwrapEvaluateResult, requireEvaluateObject, isAuthLikePayload } = __test__;
const { parsePriceData, queryStopsForPrice, queryPrice } = priceTest;
const { queryStops } = trainTest;
describe('12306 utils - parseStationBundle', () => {
it('parses the `@`-delimited station bundle into structured records', () => {
const bundle = "var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||@bji|北京|BJP|beijing|bj|2|0357|北京|||@aoh|上海虹桥|AOH|shanghaihongqiao|shhq|10|7600|上海|||';";
const stations = parseStationBundle(bundle);
expect(stations).toHaveLength(3);
expect(stations[1]).toEqual({
short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京',
});
});
it('skips records that lack a telecode', () => {
const bundle = "var station_names ='@xxx|||||||||@bji|北京|BJP|beijing|bj|2|0357|北京|||';";
const stations = parseStationBundle(bundle);
expect(stations).toHaveLength(1);
expect(stations[0].code).toBe('BJP');
});
it('throws CommandExecutionError when the bundle has no parseable station rows', () => {
expect(() => parseStationBundle("var station_names ='@xxx|||||||||';")).toThrow(CommandExecutionError);
});
});
describe('12306 utils - resolveStation', () => {
const stations = [
{ short: 'bjb', name: '北京北', code: 'VAP', pinyin: 'beijingbei', abbr: 'bjb', city: '北京' },
{ short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京' },
{ short: 'aoh', name: '上海虹桥', code: 'AOH', pinyin: 'shanghaihongqiao', abbr: 'shhq', city: '上海' },
];
it('matches by exact Chinese name', () => {
expect(resolveStation(stations, '上海虹桥').code).toBe('AOH');
});
it('matches by uppercase telecode', () => {
expect(resolveStation(stations, 'BJP').code).toBe('BJP');
});
it('matches by full pinyin (case-insensitive)', () => {
expect(resolveStation(stations, 'Beijing').code).toBe('BJP');
});
it('matches by short alias / abbr', () => {
expect(resolveStation(stations, 'shhq').code).toBe('AOH');
});
it('throws ArgumentError for empty input', () => {
expect(() => resolveStation(stations, ' ')).toThrow(ArgumentError);
});
it('throws ArgumentError for unknown station', () => {
expect(() => resolveStation(stations, '某不存在站')).toThrow(ArgumentError);
});
it('throws ArgumentError for telecode-shaped but unknown input', () => {
expect(() => resolveStation(stations, 'XYZ')).toThrow(ArgumentError);
});
});
describe('12306 utils - validateDate', () => {
it('accepts valid YYYY-MM-DD', () => {
expect(validateDate('2026-05-22')).toBe('2026-05-22');
});
it('throws ArgumentError on wrong format', () => {
expect(() => validateDate('2026/05/22')).toThrow(ArgumentError);
expect(() => validateDate('26-05-22')).toThrow(ArgumentError);
expect(() => validateDate('today')).toThrow(ArgumentError);
expect(() => validateDate('')).toThrow(ArgumentError);
});
it('throws ArgumentError on impossible calendar dates', () => {
expect(() => validateDate('2026-02-30')).toThrow(ArgumentError);
expect(() => validateDate('2026-13-01')).toThrow(ArgumentError);
});
});
describe('12306 utils - buildCookieHeader', () => {
it('joins set-cookie lines into a single Cookie header', () => {
const headers = [
'JSESSIONID=ABC123; Path=/otn',
'BIGipServerotn=xxx.yyy; Path=/',
'route=zzz; Expires=Sat, 01 Jan 2027 00:00:00 GMT',
];
expect(buildCookieHeader(headers)).toBe('JSESSIONID=ABC123; BIGipServerotn=xxx.yyy; route=zzz');
});
it('returns empty string for empty input', () => {
expect(buildCookieHeader([])).toBe('');
expect(buildCookieHeader(undefined)).toBe('');
});
});
describe('12306 utils - parseTrainRecord', () => {
const stationByCode = new Map([
['VNP', { name: '北京南', code: 'VNP' }],
['AOH', { name: '上海虹桥', code: 'AOH' }],
]);
it('extracts the canonical train fields from a wire record', () => {
// 33 `|`-separated fields, with positions used by parseTrainRecord populated.
const fields = new Array(36).fill('');
fields[0] = 'SECRET_TOKEN';
fields[1] = '预订';
fields[2] = '240000G54700';
fields[3] = 'G547';
fields[6] = 'VNP';
fields[7] = 'AOH';
fields[8] = '06:18';
fields[9] = '12:11';
fields[10] = '05:53';
fields[11] = 'Y';
fields[23] = ''; // soft sleeper
fields[26] = '无'; // no seat
fields[28] = ''; // hard sleeper
fields[29] = ''; // hard seat
fields[30] = '有'; // second seat
fields[31] = '有'; // first seat
fields[32] = '无'; // business seat
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(row).toEqual({
train_no: '240000G54700',
code: 'G547',
from_station: '北京南',
to_station: '上海虹桥',
from_code: 'VNP',
to_code: 'AOH',
start_time: '06:18',
arrive_time: '12:11',
duration: '05:53',
available: true,
business_seat: '无',
first_seat: '有',
second_seat: '有',
soft_sleeper: '',
hard_sleeper: '',
hard_seat: '',
no_seat: '无',
});
});
it('does not expose the booking-handshake secret token', () => {
const fields = new Array(36).fill('');
fields[0] = 'SECRET_TOKEN_DO_NOT_LEAK';
fields[2] = 't_no'; fields[3] = 'X1'; fields[6] = 'VNP'; fields[7] = 'AOH';
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(Object.values(row)).not.toContain('SECRET_TOKEN_DO_NOT_LEAK');
expect('secret' in row).toBe(false);
});
it('falls back to the telecode when the station bundle has no name', () => {
const fields = new Array(36).fill('');
fields[2] = 'X'; fields[3] = 'X'; fields[6] = 'ZZZ'; fields[7] = 'YYY';
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(row.from_station).toBe('ZZZ');
expect(row.to_station).toBe('YYY');
});
it('returns null for short records', () => {
expect(parseTrainRecord('a|b|c', stationByCode)).toBeNull();
});
});
describe('12306 utils - mask helpers', () => {
it('masks the local-part of an email', () => {
expect(maskEmail('hello@example.com')).toBe('h***o@example.com');
expect(maskEmail('ab@x.cn')).toBe('a*@x.cn');
expect(maskEmail('a@x.cn')).toBe('a*@x.cn');
expect(maskEmail('')).toBe('');
expect(maskEmail('not-an-email')).toBe('not-an-email');
});
it('masks Chinese mobile numbers while preserving 12306-side masks', () => {
expect(maskMobile('13800001234')).toBe('138****1234');
expect(maskMobile('138****1234')).toBe('138****1234');
expect(maskMobile('')).toBe('');
expect(maskMobile('123')).toBe('**3');
});
it('masks Chinese real names', () => {
expect(maskChineseName('张三')).toBe('张*');
expect(maskChineseName('李四明')).toBe('李*明');
expect(maskChineseName('欧阳锋')).toBe('欧*锋');
expect(maskChineseName('张')).toBe('张');
expect(maskChineseName('')).toBe('');
});
});
describe('12306 price - parsePriceData', () => {
it('returns seat rows sorted by descending price and drops dup numeric codes', () => {
const data = {
train_no: '24000000G10L',
'OT': [],
'A9': '¥2158.0',
'9': '21580',
'P': '¥1163.0',
'M': '¥1035.0',
'O': '¥626.0',
'WZ': '¥626.0',
'INVALID': 'not-a-price',
};
const rows = parsePriceData(data);
const codes = rows.map((r) => r.seat_code);
expect(codes).not.toContain('9');
expect(codes).not.toContain('OT');
expect(codes).not.toContain('train_no');
expect(codes).not.toContain('INVALID');
expect(codes).toEqual(['A9', 'P', 'M', 'O', 'WZ']);
expect(rows[0]).toEqual({ seat_code: 'A9', seat_name: '商务座', price: '2158.0', currency: 'CNY' });
expect(rows[4]).toEqual({ seat_code: 'WZ', seat_name: '无座', price: '626.0', currency: 'CNY' });
});
it('keeps unknown letter codes with the letter as the name', () => {
const data = { 'A9': '¥100.0', 'ZZ': '¥50.0' };
const rows = parsePriceData(data);
const zz = rows.find((r) => r.seat_code === 'ZZ');
expect(zz?.seat_name).toBe('ZZ');
});
});
describe('12306 public API typed boundaries', () => {
const nonJsonFetch = async () => ({
ok: true,
json: async () => {
throw new SyntaxError('Unexpected token <');
},
});
it('wraps non-JSON train stop bodies as CommandExecutionError', async () => {
await expect(queryStops('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
});
it('wraps non-JSON price helper bodies as CommandExecutionError', async () => {
await expect(queryStopsForPrice('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(queryPrice('cookie=1', '24000000G10L', '01', '02', 'OM9', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
});
});
describe('12306 browser evaluate boundaries', () => {
it('unwraps Browser Bridge {session,data} evaluate envelopes only at the boundary', () => {
expect(unwrapEvaluateResult({ session: 's1', data: 'JSESSIONID=1; tk=2' })).toBe('JSESSIONID=1; tk=2');
expect(unwrapEvaluateResult({ status: true, data: { value: 1 } })).toEqual({ status: true, data: { value: 1 } });
expect(requireEvaluateObject({ session: 's1', data: { status: true } }, 'test')).toEqual({ status: true });
expect(() => requireEvaluateObject({ session: 's1', data: null }, 'test')).toThrow(CommandExecutionError);
});
it('classifies 12306 login-like API envelopes as auth failures', () => {
expect(isAuthLikePayload({ status: false, messages: ['用户未登录'] })).toBe(true);
expect(isAuthLikePayload({ status: false, validateMessages: { global: ['请登录后再试'] } })).toBe(true);
expect(isAuthLikePayload({ status: false, messages: ['系统繁忙'] })).toBe(false);
});
it('masks passenger names in orders by default and supports explicit sensitive opt-in', async () => {
const command = getRegistry().get('12306/orders');
const makePage = () => ({
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return { session: 'browser', data: 'JSESSIONID=abc; tk=def' };
return {
session: 'browser',
data: {
status: true,
data: {
orderDBList: [{
sequence_no: 'E123',
order_date: '2026-05-18 10:00',
train_code_page: 'G1',
from_station_name_page: '北京南',
to_station_name_page: '上海虹桥',
start_train_date_page: '2026-05-22 07:00',
ticket_status_name: '未出行',
ticket_total_price_page: '626.0',
tickets: [{ passenger_name: '张三' }, { passenger_name: '李四明' }],
}],
},
},
};
},
});
await expect(command.func(makePage(), {})).resolves.toMatchObject([
{ order_id: 'E123', passengers: '张*, 李*明' },
]);
await expect(command.func(makePage(), { 'include-sensitive': true })).resolves.toMatchObject([
{ order_id: 'E123', passengers: '张三, 李四明' },
]);
});
it('maps login-like order payloads to AuthRequiredError instead of parser drift', async () => {
const command = getRegistry().get('12306/orders');
const page = {
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
return { status: false, messages: ['用户未登录'] };
},
};
await expect(command.func(page, {})).rejects.toBeInstanceOf(AuthRequiredError);
});
it('treats missing order list shape as parser drift but known empty arrays as empty result', async () => {
const command = getRegistry().get('12306/orders');
const makePage = (payload) => ({
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
return payload;
},
});
await expect(command.func(makePage({ status: true, data: {} }), {}))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ status: true, data: { orderDBList: [] } }), {}))
.rejects.toBeInstanceOf(EmptyResultError);
});
});
+6 -3
View File
@@ -52,12 +52,15 @@ cli({
if (!data?.title) {
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
}
if (!data.body) {
throw new CliError('PARSE_ERROR', 'Article body not found', '36kr page loaded but no article body paragraphs were extracted');
}
return [
{ field: 'title', value: data.title },
{ field: 'author', value: data.author || '-' },
{ field: 'date', value: data.date || '-' },
{ field: 'author', value: data.author || '' },
{ field: 'date', value: data.date || '' },
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
{ field: 'body', value: data.body || '-' },
{ field: 'body', value: data.body || '' },
];
},
});
+46
View File
@@ -0,0 +1,46 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import './article.js';
function makePage(evaluateResult) {
return {
installInterceptor: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('36kr article', () => {
it('emits empty-string for missing optional author / date instead of a sentinel', async () => {
const command = getRegistry().get('36kr/article');
expect(command?.func).toBeDefined();
const page = makePage({ title: 'Real Title', author: '', date: '', body: 'Real article body' });
const rows = await command.func(page, { id: '1234567' });
const byField = Object.fromEntries(rows.map((r) => [r.field, r.value]));
expect(byField.title).toBe('Real Title');
expect(byField.author).toBe('');
expect(byField.date).toBe('');
expect(byField.body).toBe('Real article body');
expect(byField.url).toBe('https://36kr.com/p/1234567');
});
it('throws CliError NOT_FOUND when the page exposes no title', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({ title: '', author: 'x', date: 'y', body: 'z' });
await expect(command.func(page, { id: '1234567' })).rejects.toBeInstanceOf(CliError);
});
it('throws CliError PARSE_ERROR when the page exposes title but no body', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({ title: 'Real Title', author: 'x', date: 'y', body: '' });
await expect(command.func(page, { id: '1234567' })).rejects.toMatchObject({ code: 'PARSE_ERROR' });
});
it('throws CliError INVALID_ARGUMENT when no numeric id can be parsed', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({});
await expect(command.func(page, { id: 'not-a-url' })).rejects.toBeInstanceOf(CliError);
});
});
+4
View File
@@ -14,6 +14,7 @@ export function makeScreenshotCommand(site, displayName, extra = {}) {
...extra,
site,
name: 'screenshot',
access: 'read',
description: `Capture a snapshot of the current ${label} window (DOM + Accessibility tree)`,
domain: 'localhost',
strategy: Strategy.UI,
@@ -46,6 +47,7 @@ export function makeStatusCommand(site, displayName, extra = {}) {
...extra,
site,
name: 'status',
access: 'read',
description: `Check active CDP connection to ${label}`,
domain: 'localhost',
strategy: Strategy.UI,
@@ -67,6 +69,7 @@ export function makeNewCommand(site, displayName, extra = {}) {
...extra,
site,
name: 'new',
access: 'write',
description: `Start a new ${label} session`,
domain: 'localhost',
strategy: Strategy.UI,
@@ -87,6 +90,7 @@ export function makeDumpCommand(site) {
return cli({
site,
name: 'dump',
access: 'read',
description: `Dump the DOM and Accessibility tree of ${site} for reverse-engineering`,
domain: 'localhost',
strategy: Strategy.UI,
+70
View File
@@ -0,0 +1,70 @@
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
export function requireSearchQuery(value, label = 'keyword') {
const query = String(value ?? '').trim();
if (!query) {
throw new ArgumentError(`${label} cannot be empty`);
}
return query;
}
export function requireBoundedInteger(value, defaultValue, min, max, label) {
const raw = value ?? defaultValue;
const parsed = typeof raw === 'number' ? raw : Number(raw);
if (!Number.isInteger(parsed)) {
throw new ArgumentError(`${label} must be an integer between ${min} and ${max}, got ${JSON.stringify(value)}`);
}
if (parsed < min || parsed > max) {
throw new ArgumentError(`${label} must be between ${min} and ${max}, got ${parsed}`);
}
return parsed;
}
export function requireNonNegativeInteger(value, defaultValue, label) {
const raw = value ?? defaultValue;
const parsed = typeof raw === 'number' ? raw : Number(raw);
if (!Number.isInteger(parsed) || parsed < 0) {
throw new ArgumentError(`${label} must be a non-negative integer, got ${JSON.stringify(value)}`);
}
return parsed;
}
export function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && !Array.isArray(value) && 'session' in value && 'data' in value) {
return value.data;
}
return value;
}
export function requireRows(value, label) {
const rows = unwrapBrowserResult(value);
if (!Array.isArray(rows)) {
throw new CommandExecutionError(`${label} returned an unexpected payload shape; expected an array of result rows.`);
}
return rows;
}
export function toHttpsUrl(value, baseUrl) {
const raw = String(value ?? '').trim();
if (!raw) return '';
try {
const url = new URL(raw, baseUrl);
if (url.protocol !== 'http:' && url.protocol !== 'https:') return '';
return url.href;
} catch {
return '';
}
}
export function emptySearchResults(site, query) {
return new EmptyResultError(`${site} search`, `No ${site} results matched "${query}".`);
}
export async function runBrowserStep(label, fn) {
try {
return await fn();
} catch (error) {
if (error?.code || error?.name === 'ArgumentError') throw error;
throw new CommandExecutionError(`${label} failed: ${error?.message ?? error}`);
}
}
+20
View File
@@ -41,6 +41,26 @@ describe('apple-podcasts search command', () => {
}),
]);
});
it('emits empty-string for missing trackCount and primaryGenreName instead of a sentinel', async () => {
const cmd = getRegistry().get('apple-podcasts/search');
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: () => Promise.resolve({
results: [
{
collectionId: 99,
collectionName: 'No-Meta Show',
artistName: 'Anon Host',
collectionViewUrl: 'https://example.com/p/99',
},
],
}),
});
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({ query: 'no-meta', limit: 1 });
expect(result[0].episodes).toBe('');
expect(result[0].genre).toBe('');
});
});
describe('apple-podcasts top command', () => {
beforeEach(() => {
+2 -2
View File
@@ -23,8 +23,8 @@ cli({
id: p.collectionId,
title: p.collectionName,
author: p.artistName,
episodes: p.trackCount ?? '-',
genre: p.primaryGenreName ?? '-',
episodes: p.trackCount ?? '',
genre: p.primaryGenreName ?? '',
url: p.collectionViewUrl || '',
}));
},
+144 -56
View File
@@ -4,6 +4,47 @@
* Auth: CSRF token from <meta name="csrf-token"> + session cookies.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DEFAULT_LIMIT = 10;
const MIN_LIMIT = 1;
const MAX_LIMIT = 100;
function normalizeSymbol(value) {
const symbol = String(value ?? '').trim().toUpperCase();
if (!symbol) throw new ArgumentError('symbol is required');
return symbol;
}
function normalizeExpiration(value) {
const expiration = String(value ?? '').trim();
if (!expiration) return '';
if (!/^\d{4}-\d{2}-\d{2}$/.test(expiration)) {
throw new ArgumentError('--expiration must use YYYY-MM-DD format');
}
const parsed = new Date(`${expiration}T00:00:00Z`);
if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== expiration) {
throw new ArgumentError('--expiration must be a valid calendar date');
}
return expiration;
}
function parseLimit(value) {
if (value === undefined || value === null || value === '') return DEFAULT_LIMIT;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < MIN_LIMIT || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && 'session' in value && 'data' in value) {
return value.data;
}
return value;
}
cli({
site: 'barchart',
name: 'greeks',
@@ -14,19 +55,19 @@ cli({
args: [
{ name: 'symbol', required: true, positional: true, help: 'Stock ticker (e.g. AAPL)' },
{ name: 'expiration', type: 'str', help: 'Expiration date (YYYY-MM-DD). Defaults to the nearest available expiration.' },
{ name: 'limit', type: 'int', default: 10, help: 'Number of near-the-money strikes per type' },
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: 'Number of near-the-money strikes per type (1-100)' },
],
columns: [
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
'volume', 'openInterest', 'expiration',
],
func: async (page, kwargs) => {
const symbol = kwargs.symbol.toUpperCase().trim();
const expiration = kwargs.expiration ?? '';
const limit = kwargs.limit ?? 10;
const symbol = normalizeSymbol(kwargs.symbol);
const expiration = normalizeExpiration(kwargs.expiration);
const limit = parseLimit(kwargs.limit);
await page.goto(`https://www.barchart.com/stocks/quotes/${encodeURIComponent(symbol)}/options`);
await page.wait(4);
const data = await page.evaluate(`
const data = unwrapBrowserResult(await page.evaluate(`
(async () => {
const sym = ${JSON.stringify(symbol)};
const expDate = ${JSON.stringify(expiration)};
@@ -45,39 +86,53 @@ cli({
+ '&fields=' + fields + '&raw=1';
if (expDate) url += '&expirationDate=' + encodeURIComponent(expDate);
const resp = await fetch(url, { credentials: 'include', headers });
if (resp.ok) {
const d = await resp.json();
let items = d?.data || [];
if (!resp.ok) {
return { ok: false, reason: 'http', status: resp.status, statusText: resp.statusText || '' };
}
if (!expDate) {
const expirations = items
.map(i => (i.raw || i).expirationDate || null)
.filter(Boolean)
.sort((a, b) => {
const aTime = Date.parse(a);
const bTime = Date.parse(b);
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
if (Number.isNaN(aTime)) return 1;
if (Number.isNaN(bTime)) return -1;
return aTime - bTime;
});
const nearestExpiration = expirations[0];
if (nearestExpiration) {
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
}
const d = await resp.json();
const allItems = d?.data;
if (!Array.isArray(allItems)) {
return { ok: false, reason: 'malformed' };
}
let items = allItems;
if (!expDate) {
const expirations = items
.map(i => (i.raw || i).expirationDate || null)
.filter(Boolean)
.sort((a, b) => {
const aTime = Date.parse(a);
const bTime = Date.parse(b);
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
if (Number.isNaN(aTime)) return 1;
if (Number.isNaN(bTime)) return -1;
return aTime - bTime;
});
const nearestExpiration = expirations[0];
if (nearestExpiration) {
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
}
}
// Separate calls and puts, sort by distance from current price
const calls = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const puts = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
// Separate calls and puts, sort by distance from current price.
const calls = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const puts = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const selected = [...calls, ...puts];
return [...calls, ...puts].map(i => {
if (items.length > 0 && selected.length === 0) {
return { ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' };
}
return {
ok: true,
rows: selected.map(i => {
const r = i.raw || i;
return {
type: r.optionType,
@@ -93,28 +148,61 @@ cli({
openInterest: r.openInterest,
expiration: r.expirationDate,
};
});
}
} catch(e) {}
return [];
})
};
} catch(e) {
return { ok: false, reason: 'exception', message: e?.message || String(e) };
}
})()
`);
if (!data || !Array.isArray(data))
return [];
return data.map(r => ({
type: r.type || '',
strike: r.strike,
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
volume: r.volume,
openInterest: r.openInterest,
expiration: r.expiration ?? null,
}));
`));
if (!data || data.ok !== true) {
if (data?.reason === 'http') {
throw new CommandExecutionError(`Barchart greeks request failed: HTTP ${data.status}${data.statusText ? ` ${data.statusText}` : ''}`);
}
if (data?.reason === 'malformed') {
throw new CommandExecutionError(`Barchart greeks returned an unreadable options payload${data.message ? `: ${data.message}` : ''}`);
}
if (data?.reason === 'exception') {
throw new CommandExecutionError(`Barchart greeks request failed: ${data.message || 'unknown error'}`);
}
throw new CommandExecutionError(`Failed to fetch Barchart greeks for ${symbol}`);
}
if (!Array.isArray(data.rows)) {
throw new CommandExecutionError('Barchart greeks returned an unreadable options payload');
}
if (data.rows.length === 0) {
throw new EmptyResultError('barchart greeks', `No option greeks were returned for ${symbol}. Confirm the symbol, expiration, and Barchart login state.`);
}
return data.rows.map(r => {
if (!r || typeof r !== 'object' || Array.isArray(r)) {
throw new CommandExecutionError('Barchart greeks returned a malformed option row');
}
const type = String(r.type || '').trim();
const expirationValue = String(r.expiration || '').trim();
if (!/^(call|put)$/i.test(type) || r.strike === undefined || r.strike === null || r.strike === '' || !expirationValue) {
throw new CommandExecutionError('Barchart greeks returned a malformed option row identity');
}
return {
type,
strike: r.strike,
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
volume: r.volume,
openInterest: r.openInterest,
expiration: expirationValue,
};
});
},
});
export const __test__ = {
normalizeSymbol,
normalizeExpiration,
parseLimit,
unwrapBrowserResult,
};
+138
View File
@@ -0,0 +1,138 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './greeks.js';
const { normalizeExpiration, normalizeSymbol, parseLimit, unwrapBrowserResult } = await import('./greeks.js').then((m) => m.__test__);
function makePage(evaluateResult) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('barchart greeks command', () => {
const command = getRegistry().get('barchart/greeks');
it('registers with the expected shape', () => {
expect(command).toBeDefined();
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.columns).toEqual([
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
'volume', 'openInterest', 'expiration',
]);
});
it('maps returned option rows without changing the declared output shape', async () => {
const page = makePage({
session: 'site:barchart',
data: {
ok: true,
rows: [
{
type: 'Call',
strike: 190,
last: 3.456,
iv: 21.234,
delta: 0.56789,
gamma: 0.01234,
theta: -0.12345,
vega: 0.23456,
rho: 0.03456,
volume: 123,
openInterest: 456,
expiration: '2026-06-19',
},
],
},
});
const rows = await command.func(page, { symbol: 'aapl', limit: 1 });
expect(page.goto).toHaveBeenCalledWith('https://www.barchart.com/stocks/quotes/AAPL/options');
expect(page.wait).toHaveBeenCalledWith(4);
expect(rows).toEqual([
{
type: 'Call',
strike: 190,
last: 3.46,
iv: '21.23%',
delta: 0.5679,
gamma: 0.0123,
theta: -0.1235,
vega: 0.2346,
rho: 0.0346,
volume: 123,
openInterest: 456,
expiration: '2026-06-19',
},
]);
});
it('validates args before browser navigation and unwraps bridge envelopes', async () => {
expect(normalizeSymbol(' aapl ')).toBe('AAPL');
expect(normalizeExpiration('2026-06-19')).toBe('2026-06-19');
expect(parseLimit(undefined)).toBe(10);
expect(parseLimit(100)).toBe(100);
expect(unwrapBrowserResult({ session: 'site:barchart', data: { ok: true } })).toEqual({ ok: true });
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: '', limit: 1 }))
.rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', expiration: '2026-02-30', limit: 1 }))
.rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', limit: 101 }))
.rejects.toBeInstanceOf(ArgumentError);
});
it('embeds expiration and limit in the browser-side request script', async () => {
const page = makePage({
ok: true,
rows: [{
type: 'Put',
strike: 185,
last: null,
iv: null,
delta: null,
gamma: null,
theta: null,
vega: null,
rho: null,
volume: 0,
openInterest: 0,
expiration: '2026-07-17',
}],
});
await command.func(page, { symbol: 'MSFT', expiration: '2026-07-17', limit: 7 });
const script = page.evaluate.mock.calls[0][0];
expect(script).toContain('const expDate = "2026-07-17"');
expect(script).toContain('const limit = 7');
expect(script).toContain("url += '&expirationDate=' + encodeURIComponent(expDate)");
});
it('throws CommandExecutionError for HTTP, malformed, exception, and missing payload states', async () => {
await expect(command.func(makePage({ ok: false, reason: 'http', status: 403, statusText: 'Forbidden' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'malformed' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'exception', message: 'network down' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' }), { symbol: 'AAPL' }))
.rejects.toThrow('call or put identities');
await expect(command.func(makePage(null), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: true, rows: 'bad' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: true, rows: [{ type: 'Call', strike: null, expiration: '' }] }), { symbol: 'AAPL' }))
.rejects.toThrow('malformed option row identity');
});
it('throws EmptyResultError when Barchart returns no greeks rows', async () => {
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(EmptyResultError);
});
});
+1 -1
View File
@@ -8,7 +8,7 @@ cli({
description: '获取 Bilibili 视频的字幕',
strategy: Strategy.COOKIE,
args: [
{ name: 'bvid', required: true, positional: true },
{ name: 'bvid', required: true, positional: true, help: 'Bilibili 视频 BV ID(如 BV1xx411c7mD),或视频 URL / b23.tv 短链' },
{ name: 'lang', required: false, help: '字幕语言代码 (如 zh-CN, en-US, ai-zh),默认取第一个' },
],
columns: ['index', 'from', 'to', 'content'],
+167
View File
@@ -0,0 +1,167 @@
/**
* Bilibili summary — fetches the official AI-generated video summary (the "AI总结"
* shown on the video page) via /x/web-interface/view/conclusion/get.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { apiGet, resolveBvid } from './utils.js';
const BILIBILI_HOST_RE = /(^|\.)bilibili\.com$/i;
const B23_HOST_RE = /(^|\.)b23\.tv$/i;
const BVID_RE = /^BV[A-Za-z0-9]+$/;
function formatTime(seconds) {
const s = Math.max(0, Math.floor(Number(seconds) || 0));
const h = Math.floor(s / 3600);
const m = Math.floor((s % 3600) / 60);
const sec = s % 60;
const pad = (n) => String(n).padStart(2, '0');
return h > 0 ? `${h}:${pad(m)}:${pad(sec)}` : `${pad(m)}:${pad(sec)}`;
}
async function readBvid(raw) {
const input = String(raw ?? '').trim();
if (!input) {
throw new ArgumentError('bilibili summary bvid cannot be empty', 'Pass a BV ID, Bilibili video URL, or b23.tv short link.');
}
if (BVID_RE.test(input)) {
return input;
}
let parsed = null;
try {
parsed = new URL(input);
} catch {
// Bare b23.tv short codes are accepted by the shared resolver.
}
if (parsed) {
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') {
throw new ArgumentError('Bilibili summary URL must use http or https');
}
if (BILIBILI_HOST_RE.test(parsed.hostname)) {
const match = parsed.pathname.match(/\/(?:video|bangumi\/play)\/(BV[A-Za-z0-9]+)/i);
if (!match) {
throw new ArgumentError('Bilibili summary URL must contain a BV video id');
}
return match[1];
}
if (!B23_HOST_RE.test(parsed.hostname)) {
throw new ArgumentError('Bilibili summary URL must be a bilibili.com or b23.tv URL');
}
}
try {
return await resolveBvid(input);
} catch (error) {
throw new ArgumentError(`Cannot resolve Bilibili BV ID from input: ${input}`, error instanceof Error ? error.message : String(error));
}
}
function requireOkPayload(payload, label) {
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
throw new CommandExecutionError(`Bilibili ${label} API returned a malformed payload`);
}
if (payload.code !== 0) {
const message = payload.message ?? 'unknown error';
if (payload.code === -101 || payload.code === -403 || /登录|权限|forbidden|permission|login/i.test(String(message))) {
throw new AuthRequiredError('bilibili.com', `Bilibili ${label} API requires login or permission: ${message} (${payload.code})`);
}
throw new CommandExecutionError(`Bilibili ${label} API failed: ${message} (${payload.code})`);
}
return payload.data;
}
function readModelResult(data, bvid) {
if (!data || typeof data !== 'object' || Array.isArray(data)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed data');
}
if (data.code !== 0) {
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
}
let modelResult = data.model_result;
if (typeof modelResult === 'string') {
try {
modelResult = JSON.parse(modelResult);
} catch {
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result JSON');
}
}
if (!modelResult || typeof modelResult !== 'object' || Array.isArray(modelResult)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result');
}
const summary = String(modelResult.summary ?? '').trim();
if (!summary) {
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
}
const outline = modelResult.outline ?? [];
if (!Array.isArray(outline)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline');
}
return { summary, outline };
}
function rowsFromModel(model) {
const rows = [{ time: '', content: model.summary }];
for (const section of model.outline) {
if (!section || typeof section !== 'object' || Array.isArray(section)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline section');
}
const sectionTitle = String(section.title ?? '').trim();
const sectionTime = formatTime(section.timestamp);
if (sectionTitle) {
rows.push({ time: sectionTime, content: `# ${sectionTitle}` });
}
const points = section.part_outline ?? [];
if (!Array.isArray(points)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed part outline');
}
for (const point of points) {
if (!point || typeof point !== 'object' || Array.isArray(point)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline point');
}
const content = String(point.content ?? '').trim();
if (content) {
rows.push({ time: formatTime(point.timestamp), content });
}
}
}
return rows;
}
var command = cli({
site: 'bilibili',
name: 'summary',
access: 'read',
description: '获取 B站视频的官方 AI 总结(视频页「AI总结」同款,含分段大纲与时间戳)',
domain: 'www.bilibili.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'bvid', required: true, positional: true, help: 'Video BV ID / URL / b23.tv short link' },
],
columns: ['time', 'content'],
func: async (page, kwargs) => {
if (!page) {
throw new CommandExecutionError('Browser session required for bilibili summary');
}
const bvid = await readBvid(kwargs.bvid);
const view = await apiGet(page, '/x/web-interface/view', { params: { bvid } });
const viewData = requireOkPayload(view, 'view');
const cid = viewData?.cid;
const upMid = viewData?.owner?.mid;
if (!cid || !upMid) {
throw new CommandExecutionError(`Bilibili view API did not return cid/up_mid for ${bvid}`);
}
const conclusion = await apiGet(page, '/x/web-interface/view/conclusion/get', {
params: { bvid, cid, up_mid: upMid },
signed: true,
});
const conclusionData = requireOkPayload(conclusion, 'conclusion');
return rowsFromModel(readModelResult(conclusionData, bvid));
},
});
export const __test__ = {
command,
formatTime,
readBvid,
readModelResult,
rowsFromModel,
};
+210
View File
@@ -0,0 +1,210 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const { mockApiGet, mockResolveBvid } = vi.hoisted(() => ({
mockApiGet: vi.fn(),
mockResolveBvid: vi.fn(),
}));
vi.mock('./utils.js', async (importOriginal) => ({
...(await importOriginal()),
apiGet: mockApiGet,
resolveBvid: mockResolveBvid,
}));
import { getRegistry } from '@jackwener/opencli/registry';
import './summary.js';
describe('bilibili summary', () => {
const command = getRegistry().get('bilibili/summary');
const page = {};
beforeEach(() => {
mockApiGet.mockReset();
mockResolveBvid.mockReset();
mockResolveBvid.mockRejectedValue(new Error('short link not found'));
});
function mockView(data = { aid: 114, cid: 222, owner: { mid: 333 } }) {
mockApiGet.mockResolvedValueOnce({ code: 0, data });
}
function mockConclusion(modelResult) {
mockApiGet.mockResolvedValueOnce({
code: 0,
data: {
code: 0,
model_result: modelResult,
},
});
}
it('returns the summary plus timestamped outline rows', async () => {
mockView();
mockConclusion({
summary: '整体总结',
outline: [
{
title: '第一节',
timestamp: 0,
part_outline: [
{ timestamp: 12, content: '要点A' },
{ timestamp: 3725, content: '要点B' },
],
},
],
});
const result = await command.func(page, { bvid: 'BV1xxx' });
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1xxx' } });
expect(mockApiGet).toHaveBeenNthCalledWith(2, page, '/x/web-interface/view/conclusion/get', {
params: { bvid: 'BV1xxx', cid: 222, up_mid: 333 },
signed: true,
});
expect(result).toEqual([
{ time: '', content: '整体总结' },
{ time: '00:00', content: '# 第一节' },
{ time: '00:12', content: '要点A' },
{ time: '1:02:05', content: '要点B' },
]);
});
it('returns just the summary when the video has no outline', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '只有总结', outline: [] });
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
{ time: '', content: '只有总结' },
]);
});
it('parses model_result when Bilibili returns it as a JSON string', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion(JSON.stringify({ summary: '字符串总结', outline: [] }));
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
{ time: '', content: '字符串总结' },
]);
});
it('normalizes Bilibili video URLs before calling the APIs', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: 'URL 总结', outline: [] });
await command.func(page, {
bvid: 'https://www.bilibili.com/video/BV1abc12345/?spm_id_from=333.1007',
});
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1abc12345' } });
});
it('resolves b23.tv short links through the shared resolver', async () => {
mockResolveBvid.mockResolvedValueOnce('BVshort12345');
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '短链总结', outline: [] });
await command.func(page, { bvid: 'https://b23.tv/abc' });
expect(mockResolveBvid).toHaveBeenCalledWith('https://b23.tv/abc');
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BVshort12345' } });
});
it('rejects invalid inputs before calling Bilibili APIs', async () => {
const cases = [
'',
'javascript:alert(1)',
'https://example.com/video/BV1abc12345',
'https://share.note.youdao.com/video/BV1abc12345',
'https://www.bilibili.com/read/cv12345',
];
for (const bvid of cases) {
await expect(command.func(page, { bvid })).rejects.toBeInstanceOf(ArgumentError);
}
expect(mockApiGet).not.toHaveBeenCalled();
});
it('maps unresolved short-code inputs to ArgumentError without calling APIs', async () => {
await expect(command.func(page, { bvid: 'not-a-bv' })).rejects.toBeInstanceOf(ArgumentError);
expect(mockResolveBvid).toHaveBeenCalledWith('not-a-bv');
expect(mockApiGet).not.toHaveBeenCalled();
});
it('throws EmptyResultError when Bilibili has not generated an AI summary for the video', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: 0, data: { code: 1, model_result: {} } });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when the view payload is malformed', async () => {
mockApiGet.mockResolvedValueOnce({ code: 0, data: {} });
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /cid\/up_mid/.test(err.message),
);
});
it('throws CommandExecutionError when the view API returns a non-auth error', async () => {
mockApiGet.mockResolvedValueOnce({ code: -404, message: '啥都木有' });
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /啥都木有.*-404/.test(err.message),
);
});
it('maps conclusion auth or permission errors to AuthRequiredError', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: -403, message: '访问权限不足' });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps conclusion non-auth API errors to CommandExecutionError', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: -500, message: 'server error' });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /server error.*-500/.test(err.message),
);
});
it('throws CommandExecutionError for malformed conclusion API payloads', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce(null);
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError for malformed model_result JSON', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion('{bad json');
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /model_result JSON/.test(err.message),
);
});
it('throws CommandExecutionError for malformed outline shapes', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '坏 outline', outline: {} });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /outline/.test(err.message),
);
});
it('throws CommandExecutionError for malformed part outline shapes', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({
summary: '坏 part_outline',
outline: [{ title: '段落', timestamp: 0, part_outline: {} }],
});
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /part outline/.test(err.message),
);
});
});
+356
View File
@@ -0,0 +1,356 @@
import { describe, expect, it } from 'vitest';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
import { __test__ } from './search.js';
const {
normalizePositiveInt,
normalizeNonNegativeInt,
normalizeDate,
normalizeCurrency,
normalizeLang,
hasPositiveResultCount,
buildSearchUrl,
} = __test__;
describe('booking helpers — normalizePositiveInt (no silent clamp)', () => {
it('returns default when value is undefined/null/empty', () => {
expect(normalizePositiveInt(undefined, 2, 'adults', 30)).toBe(2);
expect(normalizePositiveInt(null, 2, 'adults', 30)).toBe(2);
});
it('accepts integers in range', () => {
expect(normalizePositiveInt(1, 2, 'adults', 30)).toBe(1);
expect(normalizePositiveInt(30, 2, 'adults', 30)).toBe(30);
});
it('rejects zero / negative / out-of-range / non-integer (no silent clamp)', () => {
expect(() => normalizePositiveInt(0, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(-1, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(31, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(1.5, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt('abc', 2, 'adults', 30)).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeNonNegativeInt', () => {
it('accepts zero', () => {
expect(normalizeNonNegativeInt(0, 0, 'children', 10)).toBe(0);
});
it('rejects negative / out-of-range (no silent clamp)', () => {
expect(() => normalizeNonNegativeInt(-1, 0, 'children', 10)).toThrow(ArgumentError);
expect(() => normalizeNonNegativeInt(11, 0, 'children', 10)).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeDate', () => {
it('accepts YYYY-MM-DD', () => {
expect(normalizeDate('2026-06-15', 'checkin')).toBe('2026-06-15');
});
it('rejects bad format / nonsense dates with ArgumentError', () => {
expect(() => normalizeDate('', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('06/15/2026', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('2026-13-40', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('2026-02-31', 'checkin')).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeCurrency', () => {
it('passes 3-letter codes uppercased', () => {
expect(normalizeCurrency('usd')).toBe('USD');
expect(normalizeCurrency('JPY')).toBe('JPY');
});
it('returns empty for unset', () => {
expect(normalizeCurrency(undefined)).toBe('');
expect(normalizeCurrency('')).toBe('');
});
it('rejects non-3-letter codes', () => {
expect(() => normalizeCurrency('US')).toThrow(ArgumentError);
expect(() => normalizeCurrency('US$')).toThrow(ArgumentError);
expect(() => normalizeCurrency('USDX')).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeLang whitelist', () => {
it('lowercases supported langs', () => {
expect(normalizeLang('EN-US')).toBe('en-us');
expect(normalizeLang('zh-cn')).toBe('zh-cn');
});
it('rejects unknown langs', () => {
expect(() => normalizeLang('xx-yy')).toThrow(ArgumentError);
expect(() => normalizeLang('en')).toThrow(ArgumentError);
});
});
describe('booking helpers — buildSearchUrl', () => {
it('constructs canonical search URL with required params', () => {
const url = buildSearchUrl({
destination: 'Tokyo',
checkin: '2026-06-15',
checkout: '2026-06-17',
adults: 2,
rooms: 1,
children: 0,
offset: 0,
currency: 'USD',
lang: 'en-us',
});
expect(url).toContain('https://www.booking.com/searchresults.en-us.html');
expect(url).toContain('ss=Tokyo');
expect(url).toContain('checkin=2026-06-15');
expect(url).toContain('checkout=2026-06-17');
expect(url).toContain('group_adults=2');
expect(url).toContain('no_rooms=1');
expect(url).toContain('group_children=0');
expect(url).toContain('selected_currency=USD');
expect(url).not.toContain('offset=');
});
it('omits lang file segment when lang is empty', () => {
const url = buildSearchUrl({
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
adults: 2, rooms: 1, children: 0, offset: 0, currency: '', lang: '',
});
expect(url).toMatch(/booking\.com\/searchresults\.html\?/);
});
it('emits offset only when > 0', () => {
const url = buildSearchUrl({
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
adults: 2, rooms: 1, children: 0, offset: 25, currency: '', lang: '',
});
expect(url).toContain('offset=25');
});
});
describe('booking helpers — hasPositiveResultCount', () => {
it('detects positive Booking result-count evidence', () => {
expect(hasPositiveResultCount('Tokyo: 1,234 properties found')).toBe(true);
expect(hasPositiveResultCount('1 stay found')).toBe(true);
});
it('does not treat no-results text as positive evidence', () => {
expect(hasPositiveResultCount('No properties found')).toBe(false);
expect(hasPositiveResultCount('0 properties found')).toBe(false);
});
});
describe('booking adapter registry shape', () => {
it('search is registered as read with id-shaped column for round-trip', () => {
const search = getRegistry().get('booking/search');
expect(search).toBeDefined();
expect(search.access).toBe('read');
expect(search.browser).toBe(true);
// slug + country together form the round-trip identity (URL: /hotel/<country>/<slug>.html)
expect(search.columns).toContain('slug');
expect(search.columns).toContain('country');
expect(search.columns).toContain('url');
});
it('search columns stay <= 12 to honor agent-native row shape', () => {
const search = getRegistry().get('booking/search');
expect(search.columns.length).toBeLessThanOrEqual(12);
});
});
describe('booking search — typed errors (no silent fallback)', () => {
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
it('rejects empty destination with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: ' ', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(ArgumentError);
});
it('rejects missing checkin/checkout with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo' })).rejects.toThrow(ArgumentError);
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15' })).rejects.toThrow(ArgumentError);
});
it('rejects checkout <= checkin with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-17', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
});
it('rejects out-of-range --limit with ArgumentError (no silent clamp to 100)', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', limit: 999 })).rejects.toThrow(ArgumentError);
});
it('rejects negative --offset with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: -1 })).rejects.toThrow(ArgumentError);
});
it('rejects unsupported --lang with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', lang: 'xx-yy' })).rejects.toThrow(ArgumentError);
});
it('rejects malformed --currency with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'US$' })).rejects.toThrow(ArgumentError);
});
it('wraps browser navigation failures as CommandExecutionError', async () => {
const search = getRegistry().get('booking/search');
const downPage = { goto: () => Promise.reject(new Error('browser down')) };
await expect(search.func(downPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws EmptyResultError when extractor returns no cards', async () => {
const search = getRegistry().get('booking/search');
const emptyPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'No properties found' }),
};
await expect(search.func(emptyPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(EmptyResultError);
});
it('throws CommandExecutionError when result-count evidence exists but no cards were parsed', async () => {
const search = getRegistry().get('booking/search');
const driftPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'Tokyo: 1,234 properties found' }),
};
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when captcha is detected', async () => {
const search = getRegistry().get('booking/search');
const blockedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: true, totalText: 'Verify you are human' }),
};
await expect(search.func(blockedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when extractor payload is malformed instead of treating it as empty', async () => {
const search = getRegistry().get('booking/search');
const malformedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, blocked: false, totalText: 'Tokyo hotels' }),
};
await expect(search.func(malformedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when rendered cards lack stable hotel URL identity', async () => {
const search = getRegistry().get('booking/search');
const driftPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: 'Tokyo hotels',
items: [{
name: 'Unlinked Hotel',
country: '',
slug: '',
url: '',
distance: '',
review_score: null,
review_count: null,
star_rating: null,
price_currency: '',
price_amount: null,
recommended_room: '',
}],
}),
};
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('unwraps {session, data} envelope from CDP bridge before validating', async () => {
const search = getRegistry().get('booking/search');
const envelopePage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
session: 1,
data: {
ok: true,
blocked: false,
totalText: '',
items: [{
name: 'Test Hotel',
country: 'jp',
slug: 'test-hotel',
url: 'https://www.booking.com/hotel/jp/test-hotel.html',
distance: '1 km from centre',
review_score: 8.6,
review_count: 100,
star_rating: 4,
price_currency: 'USD',
price_amount: 120,
recommended_room: 'Standard double',
}],
},
}),
};
const rows = await search.func(envelopePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' });
expect(rows).toHaveLength(1);
expect(rows[0].rank).toBe(1);
expect(rows[0].slug).toBe('test-hotel');
expect(rows[0].url).toBe('https://www.booking.com/hotel/jp/test-hotel.html');
});
it('uses requested selected_currency as the output source when price is present', async () => {
const search = getRegistry().get('booking/search');
const currencyPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: '',
items: [{
name: 'Currency Hotel',
country: 'cn',
slug: 'currency-hotel',
url: 'https://www.booking.com/hotel/cn/currency-hotel.html',
distance: '',
review_score: null,
review_count: null,
star_rating: null,
price_currency: 'JPY',
price_amount: 880,
recommended_room: '',
}],
}),
};
const rows = await search.func(currencyPage, { destination: 'Shanghai', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'CNY' });
expect(rows[0].price_currency).toBe('CNY');
});
it('respects offset for rank numbering when paginating', async () => {
const search = getRegistry().get('booking/search');
const pagedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: '',
items: [
{ name: 'A', country: 'jp', slug: 'a', url: 'https://www.booking.com/hotel/jp/a.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
{ name: 'B', country: 'jp', slug: 'b', url: 'https://www.booking.com/hotel/jp/b.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
],
}),
};
const rows = await search.func(pagedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: 50 });
expect(rows[0].rank).toBe(51);
expect(rows[1].rank).toBe(52);
});
});
+351
View File
@@ -0,0 +1,351 @@
import {
ArgumentError,
CommandExecutionError,
EmptyResultError,
} from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
function normalizePositiveInt(value, defaultValue, label, max) {
const raw = value ?? defaultValue;
const n = Number(raw);
if (!Number.isInteger(n) || n <= 0) {
throw new ArgumentError(`${label} must be a positive integer`);
}
if (typeof max === 'number' && n > max) {
throw new ArgumentError(`${label} must be <= ${max}`);
}
return n;
}
function normalizeNonNegativeInt(value, defaultValue, label, max) {
const raw = value ?? defaultValue;
const n = Number(raw);
if (!Number.isInteger(n) || n < 0) {
throw new ArgumentError(`${label} must be a non-negative integer`);
}
if (typeof max === 'number' && n > max) {
throw new ArgumentError(`${label} must be <= ${max}`);
}
return n;
}
function normalizeDate(value, label) {
const v = String(value || '').trim();
if (!v) {
throw new ArgumentError(`${label} is required (YYYY-MM-DD)`);
}
if (!DATE_RE.test(v)) {
throw new ArgumentError(`${label} must be YYYY-MM-DD, got ${JSON.stringify(value)}`);
}
const [year, month, day] = v.split('-').map(Number);
const d = new Date(Date.UTC(year, month - 1, day));
if (
Number.isNaN(d.getTime()) ||
d.getUTCFullYear() !== year ||
d.getUTCMonth() !== month - 1 ||
d.getUTCDate() !== day
) {
throw new ArgumentError(`${label} is not a valid calendar date: ${v}`);
}
return v;
}
function normalizeCurrency(value) {
if (value == null || value === '') return '';
const v = String(value).trim().toUpperCase();
if (!/^[A-Z]{3}$/.test(v)) {
throw new ArgumentError(`currency must be a 3-letter ISO code (e.g. USD, JPY, CNY), got ${JSON.stringify(value)}`);
}
return v;
}
const ALLOWED_LANGS = new Set([
'en-us', 'en-gb', 'zh-cn', 'zh-tw', 'ja', 'ko', 'de', 'fr', 'es', 'it',
'pt-br', 'pt-pt', 'ru', 'th', 'vi', 'tr', 'pl', 'nl', 'ar',
]);
function normalizeLang(value) {
if (value == null || value === '') return '';
const v = String(value).trim().toLowerCase();
if (!ALLOWED_LANGS.has(v)) {
throw new ArgumentError(`lang must be one of: ${[...ALLOWED_LANGS].join(', ')}`);
}
return v;
}
function hasPositiveResultCount(text) {
const value = String(text || '').replace(/\u00a0/g, ' ');
const resultCount = value.match(/\b([1-9][0-9,.\s]*)\s+(?:properties|property|stays|stay|hotels|hotel)\b/i);
if (!resultCount) return false;
const digits = resultCount[1].replace(/\D/g, '');
return Boolean(digits) && Number(digits) > 0;
}
function buildSearchUrl({
destination,
checkin,
checkout,
adults,
rooms,
children,
offset,
currency,
lang,
}) {
const file = lang ? `searchresults.${lang}.html` : 'searchresults.html';
const params = new URLSearchParams();
params.set('ss', destination);
params.set('checkin', checkin);
params.set('checkout', checkout);
params.set('group_adults', String(adults));
params.set('no_rooms', String(rooms));
params.set('group_children', String(children));
if (offset > 0) params.set('offset', String(offset));
if (currency) params.set('selected_currency', currency);
return `https://www.booking.com/${file}?${params.toString()}`;
}
const EXTRACTOR = `
(() => {
const trim = (v) => (v == null ? '' : String(v).replace(/\\s+/g, ' ').trim());
const cards = Array.from(document.querySelectorAll('[data-testid=property-card]'));
// Detect blocking / captcha pages: no cards but body shows a verification prompt.
if (cards.length === 0) {
const text = [
(document.title || ''),
(document.body && document.body.innerText) || '',
(location && location.pathname) || '',
].join(' ');
const blocked = /captcha|challenge|verify\\s*you\\s*are|access\\s*denied|forbidden|robot|unusual\\s*traffic/i.test(text);
const totalEl = document.querySelector('h1');
const totalText = trim(totalEl && totalEl.textContent);
return { ok: true, items: [], blocked, totalText };
}
const items = cards.map((card) => {
const titleEl = card.querySelector('[data-testid=title]');
const link = card.querySelector('a[data-testid=title-link]');
const href = (link && link.href) || '';
let country = '';
let slug = '';
let canonicalUrl = '';
try {
const u = new URL(href, 'https://www.booking.com');
const m = u.pathname.match(/^\\/hotel\\/([a-z]{2})\\/([^./]+)/);
if (m) {
country = m[1];
slug = m[2];
canonicalUrl = 'https://www.booking.com/hotel/' + country + '/' + slug + '.html';
}
} catch (_) {}
const reviewTextRaw = trim(card.querySelector('[data-testid=review-score]')?.textContent);
// Booking renders the score twice (a11y + visual), text reads like "Scored 8.6 8.6 Very Good 6,151 reviews"
// or "评分8.68.6很棒 6,151条住客点评". Take only the first numeric occurrence.
const scoreMatch = reviewTextRaw.match(/(\\d{1,2})\\.(\\d)/);
const reviewScore = scoreMatch ? Number(scoreMatch[1] + '.' + scoreMatch[2]) : null;
const countMatch = reviewTextRaw.match(/([0-9][0-9,]*)\\s*(?:reviews|reseñas|avis|recensioni|条住客点评|条评论|レビュー|리뷰)/i);
const reviewCount = countMatch ? Number(countMatch[1].replace(/,/g, '')) : null;
// Star rating: aria-label often "5 out of 5" / "4 星 (满分 5 星)" / "Hôtel 4 étoiles"
let starRating = null;
const starEl = card.querySelector('[data-testid=rating-stars], [data-testid=quality-rating]');
if (starEl) {
const aria = starEl.getAttribute('aria-label') || starEl.textContent || '';
const m = aria.match(/(\\d)(?:\\s*(?:out of|\\/|星|颗星|stars?|étoiles?)|\\s*$)/i);
if (m) starRating = Number(m[1]);
if (starRating == null) {
const count = starEl.querySelectorAll('svg, [aria-hidden=true]').length;
if (count >= 1 && count <= 5) starRating = count;
}
}
const priceEl = card.querySelector('[data-testid=price-and-discounted-price]');
const priceText = trim(priceEl && priceEl.textContent);
// currency symbol → ISO best-effort
const currencySymbolMap = {
'$': 'USD', 'US$': 'USD', 'A$': 'AUD', 'C$': 'CAD', 'HK$': 'HKD',
'€': 'EUR', '£': 'GBP', '¥': 'JPY', '¥': 'CNY', '₹': 'INR', '₩': 'KRW',
'CN¥': 'CNY', 'CN¥': 'CNY', 'NT$': 'TWD', 'S$': 'SGD',
};
let priceCurrency = '';
let priceAmount = null;
const sym = priceText.match(/(US\\$|A\\$|C\\$|HK\\$|NT\\$|S\\$|CN¥|CN¥|[$€£¥¥₹₩])/);
if (sym) priceCurrency = currencySymbolMap[sym[1]] || '';
const num = priceText.replace(/,/g, '').match(/(\\d+(?:\\.\\d+)?)/);
if (num) priceAmount = Number(num[1]);
return {
name: trim(titleEl?.textContent),
country,
slug,
url: canonicalUrl,
distance: trim(card.querySelector('[data-testid=distance]')?.textContent),
review_score: reviewScore,
review_count: reviewCount,
star_rating: starRating,
price_currency: priceCurrency,
price_amount: priceAmount,
recommended_room: trim(card.querySelector('[data-testid=recommended-units]')?.textContent),
};
});
const totalEl = document.querySelector('h1');
const totalText = trim(totalEl && totalEl.textContent);
return { ok: true, items, blocked: false, totalText };
})()
`;
cli({
site: 'booking',
name: 'search',
description: 'Search Booking.com hotels by destination and dates (server-rendered card scrape).',
access: 'read',
example: 'opencli booking search Tokyo --checkin 2026-06-15 --checkout 2026-06-17 -f yaml',
domain: 'www.booking.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'destination', required: true, positional: true, help: 'Destination keyword (city, district, or hotel name)' },
{ name: 'checkin', required: true, help: 'Check-in date YYYY-MM-DD' },
{ name: 'checkout', required: true, help: 'Check-out date YYYY-MM-DD' },
{ name: 'adults', type: 'int', default: 2, help: 'Number of adults (1-30)' },
{ name: 'rooms', type: 'int', default: 1, help: 'Number of rooms (1-30)' },
{ name: 'children', type: 'int', default: 0, help: 'Number of children (0-10)' },
{ name: 'currency', required: false, help: 'Force result currency (e.g. USD, JPY, CNY)' },
{ name: 'lang', required: false, help: 'Force result language (e.g. en-us, zh-cn, ja)' },
{ name: 'limit', type: 'int', default: 25, help: 'Max rows to return (1-100; Booking pages 25 per request)' },
{ name: 'offset', type: 'int', default: 0, help: 'Result offset for pagination (multiple of 25)' },
],
columns: [
'rank',
'name',
'country',
'slug',
'star_rating',
'review_score',
'review_count',
'price_amount',
'price_currency',
'distance',
'recommended_room',
'url',
],
func: async (page, kwargs) => {
const destination = String(kwargs.destination || '').trim();
if (!destination) throw new ArgumentError('destination is required');
const checkin = normalizeDate(kwargs.checkin, 'checkin');
const checkout = normalizeDate(kwargs.checkout, 'checkout');
if (checkin >= checkout) {
throw new ArgumentError(`checkout (${checkout}) must be after checkin (${checkin})`);
}
const adults = normalizePositiveInt(kwargs.adults, 2, 'adults', 30);
const rooms = normalizePositiveInt(kwargs.rooms, 1, 'rooms', 30);
const children = normalizeNonNegativeInt(kwargs.children, 0, 'children', 10);
const currency = normalizeCurrency(kwargs.currency);
const lang = normalizeLang(kwargs.lang);
const limit = normalizePositiveInt(kwargs.limit, 25, 'limit', 100);
const offset = normalizeNonNegativeInt(kwargs.offset, 0, 'offset', 1000);
const url = buildSearchUrl({ destination, checkin, checkout, adults, rooms, children, offset, currency, lang });
try {
await page.goto(url);
} catch (err) {
throw new CommandExecutionError(`Failed to load Booking.com search page: ${err?.message || err}`);
}
// Booking lazy-loads price cells; wait for at least the first card price to settle.
try {
await page.wait('selector', '[data-testid=property-card]', { timeoutMs: 20000 });
} catch (_) {
// selector wait is best-effort — extractor handles empty case explicitly
}
let raw;
try {
raw = await page.evaluate(EXTRACTOR);
} catch (err) {
throw new CommandExecutionError(`Failed to extract Booking.com cards: ${err?.message || err}`);
}
if (raw && typeof raw === 'object' && raw.data && raw.session) {
raw = raw.data;
}
if (!raw || typeof raw !== 'object') {
throw new CommandExecutionError('Booking.com page returned no extractable data');
}
if (raw.blocked) {
throw new CommandExecutionError('Booking.com served a verification / captcha page; retry later or change profile');
}
if (raw.ok !== true) {
throw new CommandExecutionError('Booking.com extractor returned an invalid status');
}
if (!Array.isArray(raw.items)) {
throw new CommandExecutionError('Booking.com extractor returned malformed items');
}
const items = raw.items;
if (items.length === 0) {
const totalText = String(raw.totalText || '').trim();
if (hasPositiveResultCount(totalText)) {
throw new CommandExecutionError(
`Booking.com page declared results but no property cards were parsed: ${totalText}`,
);
}
throw new EmptyResultError(
`booking search ${JSON.stringify(destination)}`,
totalText
? `No hotels rendered (${totalText}). Try a broader destination, different dates, or check the URL in a browser.`
: 'No hotels rendered. Try a broader destination, different dates, or check the URL in a browser.',
);
}
return items.slice(0, limit).map((it, i) => {
if (!it || typeof it !== 'object') {
throw new CommandExecutionError('Booking.com extractor returned malformed hotel row');
}
const name = String(it.name || '').trim();
const country = String(it.country || '').trim();
const slug = String(it.slug || '').trim();
const urlValue = String(it.url || '').trim();
const expectedUrl = country && slug
? `https://www.booking.com/hotel/${country}/${slug}.html`
: '';
if (!name || !/^[a-z]{2}$/.test(country) || !slug || urlValue !== expectedUrl) {
throw new CommandExecutionError('Booking.com hotel row is missing stable name/url identity');
}
return {
rank: offset + i + 1,
name,
country,
slug,
star_rating: it.star_rating,
review_score: it.review_score,
review_count: it.review_count,
price_amount: it.price_amount,
price_currency: it.price_amount == null ? '' : (currency || it.price_currency || ''),
distance: it.distance,
recommended_room: it.recommended_room,
url: urlValue,
};
});
},
});
export const __test__ = {
normalizePositiveInt,
normalizeNonNegativeInt,
normalizeDate,
normalizeCurrency,
normalizeLang,
hasPositiveResultCount,
buildSearchUrl,
EXTRACTOR,
};
+96 -14
View File
@@ -1,10 +1,60 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { requirePage, navigateToChat, fetchFriendList } from './utils.js';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
requirePage, navigateToChat, navigateToGeekChat,
fetchFriendList, fetchGeekFriendLabelList, fetchGeekFriendInfoList,
readEncryptSystemId, assertOk, IDENTITY_MISMATCH_CODE,
readPositiveInteger,
} from './utils.js';
function formatMsgTime(ms) {
if (!ms) return '';
return new Date(ms).toLocaleString('zh-CN');
}
function mapBossRow(f) {
return {
name: f.name || '',
company: '',
job: f.jobName || '',
title: '',
last_msg: f.lastMessageInfo?.text || '',
last_time: f.lastTime || '',
uid: f.encryptUid || '',
security_id: f.securityId || '',
};
}
async function buildGeekRows(page, limit) {
const encryptSystemId = await readEncryptSystemId(page);
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId });
if (labelList.length === 0) {
return [];
}
const slicedLabels = labelList.slice(0, limit);
const friendIds = slicedLabels.map((f) => f.friendId).filter(Boolean);
const enriched = await fetchGeekFriendInfoList(page, friendIds);
const enrichMap = new Map(enriched.map((f) => [String(f.friendId ?? f.uid), f]));
return slicedLabels.map((f) => {
const e = enrichMap.get(String(f.friendId)) || {};
return {
name: e.name || f.name || '',
company: e.brandName || f.brandName || '',
job: e.jobName || f.jobName || '',
title: e.bossTitle || f.bossTitle || '',
last_msg: e.lastMessageInfo?.showText || e.lastMsg || f.lastMsg || '',
last_time: e.lastTime || formatMsgTime(e.lastMessageInfo?.msgTime) || formatMsgTime(f.updateTime) || '',
uid: e.encryptUid || f.encryptFriendId || String(e.uid ?? e.friendId ?? f.friendId ?? ''),
security_id: e.securityId || '',
};
});
}
cli({
site: 'boss',
name: 'chatlist',
access: 'read',
description: 'BOSS直聘查看聊天列表(招聘端)',
description: 'BOSS直聘查看聊天列表(招聘端/求职端',
domain: 'www.zhipin.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
@@ -12,23 +62,55 @@ cli({
args: [
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of results' },
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all)' },
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all, boss side only)' },
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
],
columns: ['name', 'job', 'last_msg', 'last_time', 'uid', 'security_id'],
columns: ['name', 'company', 'job', 'title', 'last_msg', 'last_time', 'uid', 'security_id'],
func: async (page, kwargs) => {
requirePage(page);
const limit = readPositiveInteger(kwargs.limit, 'chatlist --limit', 20, 100);
const pageNum = readPositiveInteger(kwargs.page, 'chatlist --page', 1);
const side = kwargs.side || 'auto';
if (side === 'boss') {
await navigateToChat(page);
const friends = await fetchFriendList(page, {
pageNum,
jobId: kwargs['job-id'] || '0',
});
if (friends.length === 0)
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
return friends.slice(0, limit).map(mapBossRow);
}
if (side === 'geek') {
await navigateToGeekChat(page);
const rows = await buildGeekRows(page, limit);
if (rows.length === 0)
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
return rows;
}
// auto: try recruiter first, fall back to geek on identity mismatch
await navigateToChat(page);
const friends = await fetchFriendList(page, {
pageNum: kwargs.page || 1,
const bossResult = await fetchFriendList(page, {
pageNum,
jobId: kwargs['job-id'] || '0',
allowNonZero: true,
});
return friends.slice(0, kwargs.limit || 20).map((f) => ({
name: f.name || '',
job: f.jobName || '',
last_msg: f.lastMessageInfo?.text || '',
last_time: f.lastTime || '',
uid: f.encryptUid || '',
security_id: f.securityId || '',
}));
if (Array.isArray(bossResult)) {
if (bossResult.length === 0)
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
return bossResult.slice(0, limit).map(mapBossRow);
}
if (bossResult.code === IDENTITY_MISMATCH_CODE) {
await navigateToGeekChat(page);
const rows = await buildGeekRows(page, limit);
if (rows.length === 0)
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
return rows;
}
assertOk(bossResult);
throw new CommandExecutionError('Boss chatlist returned an unexpected response');
},
});
+211
View File
@@ -0,0 +1,211 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './chatlist.js';
const BOSS_FRIEND = {
name: '张三',
jobName: '后端工程师',
lastMessageInfo: { text: '你好' },
lastTime: '2024-01-01 10:00',
encryptUid: 'enc-boss-uid',
securityId: 'boss-sec-id',
};
const GEEK_LABEL_FRIEND = {
friendId: 12345,
name: '李四',
brandName: '字节跳动',
jobName: '产品经理',
bossTitle: 'HR',
lastMsg: '感谢投递',
updateTime: 1704067200000,
encryptFriendId: 'enc-geek-uid',
};
const GEEK_ENRICHED = {
friendId: 12345,
uid: 99999,
name: '李四',
brandName: '字节跳动',
jobName: '产品经理',
bossTitle: 'HR总监',
encryptUid: 'enc-geek-uid',
securityId: 'geek-sec-id',
lastMessageInfo: { showText: '感谢投递', msgTime: 1704067200000 },
lastTime: '2024-01-01',
};
function createPageMock(evaluateImpl) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockImplementation(evaluateImpl),
};
}
describe('boss chatlist', () => {
const command = getRegistry().get('boss/chatlist');
it('--side boss preserves existing behavior with 8-column output', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
name: '张三',
company: '',
job: '后端工程师',
title: '',
last_msg: '你好',
uid: 'enc-boss-uid',
security_id: 'boss-sec-id',
});
});
it('--side geek maps enriched getGeekFriendList data into 8 columns', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
name: '李四',
company: '字节跳动',
job: '产品经理',
title: 'HR总监',
uid: 'enc-geek-uid',
security_id: 'geek-sec-id',
});
});
it('--side geek falls back to label fields when enrichment has no match', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('李四');
expect(rows[0].company).toBe('字节跳动');
expect(rows[0].security_id).toBe('');
});
it('rejects invalid --limit before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { page: 1, limit: 0, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('--side geek reports a true empty chat list as EmptyResultError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [] } };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
it('treats malformed geek enrichment payload as CommandExecutionError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: {} };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('treats null Boss API payload as CommandExecutionError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) return null;
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('maps expired Boss cookies to AuthRequiredError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 7, message: 'Cookie 已过期' };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
).rejects.toBeInstanceOf(AuthRequiredError);
});
it('--side auto falls back to geek when recruiter returns code 24', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 24, message: '请切换身份后再试' };
}
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
expect(rows).toHaveLength(1);
expect(rows[0].company).toBe('字节跳动');
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
});
it('--side auto uses recruiter results when code 0 and does not call geek API', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('张三');
const evaluateCalls = page.evaluate.mock.calls.map((c) => c[0]);
expect(evaluateCalls.some((s) => s.includes('geekFilterByLabel'))).toBe(false);
});
it('registers --side as a choices-constrained arg defaulting to auto', () => {
const sideArg = command.args.find((a) => a.name === 'side');
expect(sideArg?.choices).toEqual(['auto', 'boss', 'geek']);
expect(sideArg?.default).toBe('auto');
});
});
+98 -24
View File
@@ -1,10 +1,72 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { requirePage, navigateToChat, bossFetch, findFriendByUid } from './utils.js';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
requirePage, navigateToChat, navigateToGeekChat,
bossFetch, findFriendByUid, findGeekFriendByUid,
fetchGeekHistoryMsg, readEncryptSystemId,
assertOk, IDENTITY_MISMATCH_CODE,
readPositiveInteger, readRequiredString,
} from './utils.js';
const TYPE_MAP = {
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
6: '名片', 7: '语音', 8: '视频', 9: '表情',
};
function mapBossMsg(m, friend) {
const fromObj = m.from || {};
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
return {
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
type: TYPE_MAP[m.type] || `其他(${m.type})`,
text: m.text || m.body?.text || '',
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
}
function mapGeekMsg(m, friend) {
const fromUid = m.from && m.from.uid;
const isFromBoss = fromUid != null && String(fromUid) === String(friend.uid);
return {
from: isFromBoss ? '对方' : '我',
type: TYPE_MAP[m.type] || `其他(${m.type})`,
text: m.text || m.body?.text || m.body?.content || m.body?.showText ||
JSON.stringify(m.body || {}).slice(0, 120),
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
}
async function bossChatMsg(page, kwargs, existingFriend) {
const friend = existingFriend ?? await findFriendByUid(page, kwargs.uid);
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该候选人');
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const gid = friend.uid;
const securityId = encodeURIComponent(friend.securityId);
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
const msgData = await bossFetch(page, msgUrl);
const messages = msgData.zpData?.messages ?? msgData.zpData?.historyMsgList;
if (!Array.isArray(messages)) {
throw new CommandExecutionError('Boss recruiter history response did not include a message list');
}
if (messages.length === 0) {
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
}
return messages.map((m) => mapBossMsg(m, friend));
}
async function geekChatMsg(page, kwargs, encryptSystemId) {
const friend = await findGeekFriendByUid(page, kwargs.uid, { encryptSystemId });
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该聊天(geek 侧)');
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const messages = await fetchGeekHistoryMsg(page, friend, { page: kwargs.page });
return messages.map((m) => mapGeekMsg(m, friend));
}
cli({
site: 'boss',
name: 'chatmsg',
access: 'read',
description: 'BOSS直聘查看与候选人的聊天消息',
description: 'BOSS直聘查看聊天消息历史(招聘端/求职端)',
domain: 'www.zhipin.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
@@ -12,32 +74,44 @@ cli({
args: [
{ name: 'uid', required: true, positional: true, help: 'Encrypted UID (from chatlist)' },
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
],
columns: ['from', 'type', 'text', 'time'],
func: async (page, kwargs) => {
requirePage(page);
const uid = readRequiredString(kwargs.uid, 'chatmsg uid');
const pageNum = readPositiveInteger(kwargs.page, 'chatmsg --page', 1);
const normalizedKwargs = { ...kwargs, uid, page: pageNum };
const side = kwargs.side || 'auto';
if (side === 'boss') {
await navigateToChat(page);
return await bossChatMsg(page, normalizedKwargs);
}
if (side === 'geek') {
await navigateToGeekChat(page);
const encryptSystemId = await readEncryptSystemId(page);
return await geekChatMsg(page, normalizedKwargs, encryptSystemId);
}
// auto: try recruiter first, fall back to geek when not found or identity mismatch
await navigateToChat(page);
const friend = await findFriendByUid(page, kwargs.uid);
if (!friend)
throw new Error('未找到该候选人');
const gid = friend.uid;
const securityId = encodeURIComponent(friend.securityId);
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
const msgData = await bossFetch(page, msgUrl);
const TYPE_MAP = {
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
6: '名片', 7: '语音', 8: '视频', 9: '表情',
};
const messages = msgData.zpData?.messages || msgData.zpData?.historyMsgList || [];
return messages.map((m) => {
const fromObj = m.from || {};
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
return {
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
type: TYPE_MAP[m.type] || '其他(' + m.type + ')',
text: m.text || m.body?.text || '',
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
});
const bossResult = await findFriendByUid(page, uid, { allowNonZero: true });
if (bossResult?.friend) {
return await bossChatMsg(page, normalizedKwargs, bossResult.friend);
}
// Not found or identity mismatch — check for hard errors before falling back
if (bossResult?.code && bossResult.code !== 0 && bossResult.code !== IDENTITY_MISMATCH_CODE) {
assertOk(bossResult);
}
// Fall back to geek side
await navigateToGeekChat(page);
const encryptSystemId = await readEncryptSystemId(page);
const geekFriend = await findGeekFriendByUid(page, uid, { encryptSystemId });
if (!geekFriend) throw new EmptyResultError('boss chatmsg', 'uid 在招聘端与求职端聊天列表中均未找到');
if (!geekFriend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const messages = await fetchGeekHistoryMsg(page, geekFriend, { page: pageNum });
return messages.map((m) => mapGeekMsg(m, geekFriend));
},
});
+230
View File
@@ -0,0 +1,230 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './chatmsg.js';
const BOSS_FRIEND = {
uid: 12345,
encryptUid: 'enc-boss-uid',
securityId: 'boss-sec-id',
name: '候选人甲',
};
const BOSS_MSGS = [
{ type: 1, text: 'Hello', from: { uid: 99999, name: 'HR' }, time: 1704067200000 },
{ type: 1, text: '感谢', from: { uid: 12345, name: '候选人甲' }, time: 1704067201000 },
];
const GEEK_FRIEND_LABEL = {
friendId: 11111,
encryptFriendId: 'enc-geek-uid',
name: 'Boss张',
brandName: '公司A',
};
const GEEK_FRIEND_ENRICHED = {
friendId: 11111,
uid: 67890,
encryptUid: 'enc-geek-uid',
securityId: 'geek-sec-id',
name: 'Boss张',
};
const GEEK_MSGS = [
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
];
function createPageMock(evaluateImpl) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockImplementation(evaluateImpl),
};
}
describe('boss chatmsg', () => {
const command = getRegistry().get('boss/chatmsg');
it('rejects empty uid before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { uid: ' ', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('rejects invalid --page before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { uid: 'enc-geek-uid', page: 0, side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('--side boss preserves existing behavior', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
if (script.includes('boss/historyMsg')) {
return { code: 0, zpData: { messages: BOSS_MSGS } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
expect(rows).toHaveLength(2);
expect(rows[0].from).toBe('我');
expect(rows[1].from).toBe('候选人甲');
});
it('--side geek calls historyMsg with bossId, securityId, page, c=20, src=0', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: GEEK_MSGS } };
}
return {};
});
await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
const historyScript = page.evaluate.mock.calls.find((c) => c[0].includes('geek/historyMsg'))?.[0];
expect(historyScript).toBeDefined();
expect(historyScript).toContain('bossId=67890');
expect(historyScript).toContain('securityId=');
expect(historyScript).toContain('page=1');
expect(historyScript).toContain('c=20');
expect(historyScript).toContain('src=0');
});
it('--side geek uses from.uid to determine direction, not received flag', async () => {
// Both messages have received:true (mirrors real geek historyMsg API behaviour)
// Direction is determined by whether m.from.uid matches the boss's uid (67890)
const msgsAllReceived = [
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
];
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: msgsAllReceived } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
// from.uid=67890 matches friend.uid=67890 → boss sent it → '对方'
expect(rows[0].from).toBe('对方');
// from.uid=99999 does not match → geek sent it → '我'
expect(rows[1].from).toBe('我');
});
it('non-text message body does not crash and produces truncated JSON', async () => {
const nonTextMsg = { type: 99, received: true, time: 1704067200000, body: { action: 'resume_request', detail: 'X' } };
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: [nonTextMsg] } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
expect(rows).toHaveLength(1);
expect(rows[0].text).toContain('resume_request');
});
it('--side auto falls back to geek when recruiter returns code 24', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 24, message: '请切换身份后再试' };
}
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: GEEK_MSGS } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'auto' });
expect(rows).toHaveLength(2);
expect(rows[0].from).toBe('对方');
});
it('--side geek throws when uid is not found in geek chat list', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [] } };
}
return {};
});
await expect(
command.func(page, { uid: 'unknown-uid', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
it('--side boss maps expired cookies to AuthRequiredError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 7, message: 'Cookie 已过期' };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
).rejects.toBeInstanceOf(AuthRequiredError);
});
it('--side boss treats missing history list as parser drift', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
if (script.includes('boss/historyMsg')) {
return { code: 0, zpData: {} };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('--side geek reports an empty history as EmptyResultError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: [] } };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
});
+257 -12
View File
@@ -1,8 +1,11 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
// ── Constants ───────────────────────────────────────────────────────────────
const BOSS_DOMAIN = 'www.zhipin.com';
const CHAT_URL = `https://${BOSS_DOMAIN}/web/chat/index`;
const COOKIE_EXPIRED_CODES = new Set([7, 37]);
const COOKIE_EXPIRED_MSG = 'Cookie 已过期!请在当前 Chrome 浏览器中重新登录 BOSS 直聘。';
const RECRUITER_ONLY_MSG = '该命令仅支持招聘端(BOSS 端)账号,请使用招聘者账号登录后重试。';
const DEFAULT_TIMEOUT = 15_000;
// ── Core helpers ────────────────────────────────────────────────────────────
/**
@@ -10,7 +13,24 @@ const DEFAULT_TIMEOUT = 15_000;
*/
export function requirePage(page) {
if (!page)
throw new Error('Browser page required');
throw new CommandExecutionError('Browser page required');
}
export function readPositiveInteger(raw, name, fallback, max) {
const value = raw === undefined || raw === null || raw === '' ? fallback : Number(raw);
if (!Number.isInteger(value) || value < 1) {
throw new ArgumentError(`boss ${name} must be a positive integer`);
}
if (max !== undefined && value > max) {
throw new ArgumentError(`boss ${name} must be <= ${max}`);
}
return value;
}
export function readRequiredString(raw, name) {
const value = String(raw ?? '').trim();
if (!value) {
throw new ArgumentError(`boss ${name} cannot be empty`);
}
return value;
}
/**
* Navigate to BOSS chat page and wait for it to settle.
@@ -33,19 +53,37 @@ export async function navigateTo(page, url, waitSeconds = 1) {
*/
export function checkAuth(data) {
if (COOKIE_EXPIRED_CODES.has(data.code)) {
throw new Error(COOKIE_EXPIRED_MSG);
throw new AuthRequiredError(BOSS_DOMAIN, COOKIE_EXPIRED_MSG);
}
}
/**
* Map BOSS code=24 ("请切换身份后再试") to a typed AuthRequiredError.
* Recruiter-only commands (recommend, joblist, stats, resume, mark,
* exchange, invite, greet, batchgreet) have no geek-side equivalent;
* surfacing this as a generic COMMAND_EXEC hides what the user must do.
* chatlist / chatmsg avoid this path by using `allowNonZero: true` and
* branching to the geek-side fetch when they see code 24.
*/
function checkRecruiterSide(data) {
if (data.code === IDENTITY_MISMATCH_CODE) {
throw new AuthRequiredError(BOSS_DOMAIN, RECRUITER_ONLY_MSG);
}
}
/**
* Throw if the API response is not code 0.
* Checks for cookie expiry first, then throws with the provided message.
* Checks for cookie expiry first, then identity mismatch, then throws
* with the provided message.
*/
export function assertOk(data, errorPrefix) {
if (!data || typeof data !== 'object') {
throw new CommandExecutionError(`${errorPrefix ? `${errorPrefix}: ` : ''}Boss API returned malformed response`);
}
if (data.code === 0)
return;
checkAuth(data);
checkRecruiterSide(data);
const prefix = errorPrefix ? `${errorPrefix}: ` : '';
throw new Error(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
throw new CommandExecutionError(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
}
/**
* Make a credentialed XHR request via page.evaluate().
@@ -80,7 +118,19 @@ export async function bossFetch(page, url, opts = {}) {
});
}
`;
const data = await page.evaluate(script);
let data;
try {
data = await page.evaluate(script);
} catch (error) {
if (error instanceof AuthRequiredError || error instanceof CommandExecutionError) {
throw error;
}
const message = error instanceof Error ? error.message : String(error);
throw new CommandExecutionError(`Boss API request failed: ${message}`);
}
if (!data || typeof data !== 'object') {
throw new CommandExecutionError('Boss API returned malformed response');
}
// Auto-check auth unless caller opts out
if (!opts.allowNonZero && data.code !== 0) {
assertOk(data);
@@ -95,8 +145,13 @@ export async function fetchFriendList(page, opts = {}) {
const pageNum = opts.pageNum ?? 1;
const jobId = opts.jobId ?? '0';
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getBossFriendListV2.json?page=${pageNum}&status=0&jobId=${jobId}`;
const data = await bossFetch(page, url);
return data.zpData?.friendList || [];
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && data.code !== 0) return data;
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss friend list response did not include zpData.friendList');
}
return list;
}
/**
* Fetch the recommended candidates (greetRecSortList).
@@ -104,7 +159,11 @@ export async function fetchFriendList(page, opts = {}) {
export async function fetchRecommendList(page) {
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/greetRecSortList`;
const data = await bossFetch(page, url);
return data.zpData?.friendList || [];
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss recommend response did not include zpData.friendList');
}
return list;
}
/**
* Find a friend by encryptUid, searching through friend list and optionally greet list.
@@ -115,10 +174,14 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
const checkGreetList = opts.checkGreetList ?? false;
// Search friend list pages
for (let p = 1; p <= maxPages; p++) {
const friends = await fetchFriendList(page, { pageNum: p });
const result = await fetchFriendList(page, { pageNum: p, allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && !Array.isArray(result)) {
return { friend: null, code: result.code };
}
const friends = Array.isArray(result) ? result : [];
const found = friends.find((f) => f.encryptUid === encryptUid);
if (found)
return found;
return opts.allowNonZero ? { friend: found, code: 0 } : found;
if (friends.length === 0)
break;
}
@@ -127,9 +190,9 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
const greetList = await fetchRecommendList(page);
const found = greetList.find((f) => f.encryptUid === encryptUid);
if (found)
return found;
return opts.allowNonZero ? { friend: found, code: 0 } : found;
}
return null;
return opts.allowNonZero ? { friend: null, code: 0 } : null;
}
// ── UI automation helpers ───────────────────────────────────────────────────
/**
@@ -221,3 +284,185 @@ export function verbose(msg) {
console.error(`[opencli:boss] ${msg}`);
}
}
// ── Geek-side helpers ────────────────────────────────────────────────────────
export const IDENTITY_MISMATCH_CODE = 24;
const GEEK_CHAT_URL = `https://${BOSS_DOMAIN}/web/geek/chat`;
/**
* Navigate to the job-seeker chat page.
* Establishes the cookie + JS-global context needed for geek-side API calls.
*/
export async function navigateToGeekChat(page, waitSeconds = 2) {
await page.goto(GEEK_CHAT_URL);
await page.wait({ time: waitSeconds });
}
/**
* Read the encryptSystemId value required by the geek-side list API.
* Strategy (in order):
* 1. Vue app state / Pinia stores / $route.query (Option 1 — runtime source)
* 2. performance.getEntriesByType('resource') — parse from geekFilterByLabel URL
* that the page itself already issued (Option 2 — most deterministic)
* 3. cookie, inline <script> SSR state, known window globals, localStorage (fallbacks)
* Returns empty string if nothing is found; the API may still succeed without it.
* Caller must have navigated to the geek chat page first.
*/
export async function readEncryptSystemId(page) {
const result = await page.evaluate(`
(() => {
// 1. Vue app state / Pinia / $route.query
// The chat component reads encryptSystemId from the app runtime to build
// its own geekFilterByLabel request, so the value lives in the Vue tree.
try {
const appEl = document.querySelector('#app') || document.querySelector('[data-v-app]');
const vueApp = appEl && (appEl.__vue_app__ || appEl._vei);
if (vueApp) {
// 1a. Pinia stores (Vue 3 standard state management on BOSS直聘)
const pinia = vueApp.config && vueApp.config.globalProperties.$pinia;
if (pinia && pinia.state && pinia.state.value) {
for (const store of Object.values(pinia.state.value)) {
try {
const flat = JSON.stringify(store);
if (flat.includes('encryptSystemId')) {
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
} catch (_) {}
}
}
// 1b. Vue Router current route query
const router = vueApp.config && vueApp.config.globalProperties.$router;
const query = router && router.currentRoute && router.currentRoute.value && router.currentRoute.value.query;
if (query && query.encryptSystemId) return query.encryptSystemId;
}
} catch (_) {}
// 2. Performance resource entries — the page already issued geekFilterByLabel
// with encryptSystemId in the URL; read it back from the resource timing API.
try {
const entries = performance.getEntriesByType('resource');
for (const entry of entries) {
if (!entry.name.includes('geekFilterByLabel')) continue;
const u = new URL(entry.name);
const v = u.searchParams.get('encryptSystemId');
if (v) return v;
}
} catch (_) {}
// 3. cookie
try {
const m = document.cookie.match(/encryptSystemId=([^;]+)/i);
if (m) return decodeURIComponent(m[1]);
} catch (_) {}
// 4. inline <script> SSR state (Nuxt embeds server state here)
try {
for (const s of document.querySelectorAll('script:not([src])')) {
const t = s.textContent || '';
if (!t.includes('encryptSystemId')) continue;
const m = t.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
} catch (_) {}
// 5. known BOSS / Nuxt window globals
const KNOWN = [
'__NUXT__', '__INITIAL_STATE__', '__ZP_INFO__', '__BOSS_ZP__',
'pageGlobalVar', 'ZP_DATA', '__ZP_DATA__', '__PAGE_DATA__',
];
for (const k of KNOWN) {
const obj = window[k];
if (!obj || typeof obj !== 'object') continue;
try {
const flat = JSON.stringify(obj);
if (!flat.includes('encryptSystemId')) continue;
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
} catch (_) {}
}
// 6. localStorage
try {
for (let i = 0; i < localStorage.length; i++) {
const k = localStorage.key(i);
if (!k) continue;
if (k.toLowerCase().includes('encryptsystemid')) {
const v = localStorage.getItem(k);
if (v) return v;
}
const v = localStorage.getItem(k) || '';
if (v.includes('encryptSystemId')) {
const m = v.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
}
} catch (_) {}
return '';
})()
`);
return result || '';
}
/**
* Fetch the job-seeker chat list (brief info, no securityId).
* Use fetchGeekFriendInfoList to enrich with securityId before calling chatmsg.
*/
export async function fetchGeekFriendLabelList(page, opts = {}) {
const labelId = opts.labelId ?? 0;
const encryptSystemId = opts.encryptSystemId ?? '';
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/geekFilterByLabel?labelId=${labelId}&encryptSystemId=${encodeURIComponent(encryptSystemId)}`;
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && data.code !== 0) return data;
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss geek chat list response did not include zpData.friendList');
}
return list;
}
/**
* Enrich a batch of geek friends with full fields including securityId.
* Processes in batches of 50 to avoid oversized request bodies.
*/
export async function fetchGeekFriendInfoList(page, friendIds = []) {
if (!friendIds.length) return [];
const BATCH_SIZE = 50;
const results = [];
for (let i = 0; i < friendIds.length; i += BATCH_SIZE) {
const batch = friendIds.slice(i, i + BATCH_SIZE).map(String);
const body = `friendIds=${batch.join(',')}`;
const data = await bossFetch(page, `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getGeekFriendList.json`, {
method: 'POST',
body,
});
const batchResult = data.zpData?.result;
if (!Array.isArray(batchResult)) {
throw new CommandExecutionError('Boss geek friend enrichment response did not include zpData.result');
}
results.push(...batchResult);
}
return results;
}
/**
* Find a geek-side friend by encrypted uid.
* Merges label-list and enriched data; returns null if not found.
*/
export async function findGeekFriendByUid(page, encryptUid, opts = {}) {
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId: opts.encryptSystemId });
const candidate = labelList.find((f) => f.encryptFriendId === encryptUid ||
String(f.uid) === String(encryptUid) ||
String(f.friendId) === String(encryptUid));
if (!candidate) return null;
const enriched = await fetchGeekFriendInfoList(page, [candidate.friendId]);
return { ...candidate, ...(enriched[0] || {}) };
}
/**
* Fetch message history for a geek-side chat.
* friend must have .uid (boss's numeric id) and .securityId.
*/
export async function fetchGeekHistoryMsg(page, friend, opts = {}) {
const pageNum = opts.page ?? 1;
const bossId = friend.uid;
const securityId = encodeURIComponent(friend.securityId || '');
const url = `https://${BOSS_DOMAIN}/wapi/zpchat/geek/historyMsg?bossId=${bossId}&securityId=${securityId}&page=${pageNum}&c=20&src=0`;
const data = await bossFetch(page, url);
const messages = data.zpData?.messages ?? data.zpData?.historyMsgList;
if (!Array.isArray(messages)) {
throw new CommandExecutionError('Boss geek history response did not include a message list');
}
if (messages.length === 0) {
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
}
return messages;
}
+34
View File
@@ -0,0 +1,34 @@
import { describe, expect, it } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { assertOk } from './utils.js';
describe('assertOk', () => {
it('returns silently on code 0', () => {
expect(() => assertOk({ code: 0 })).not.toThrow();
});
it('maps expired cookie codes (7, 37) to AuthRequiredError', () => {
expect(() => assertOk({ code: 7, message: 'expired' })).toThrow(AuthRequiredError);
expect(() => assertOk({ code: 37, message: 'expired' })).toThrow(AuthRequiredError);
});
it('maps code 24 (identity mismatch) to AuthRequiredError with recruiter-only hint', () => {
try {
assertOk({ code: 24, message: '请切换身份后再试' });
throw new Error('assertOk should have thrown');
} catch (err) {
expect(err).toBeInstanceOf(AuthRequiredError);
expect(String(err.message)).toContain('招聘端');
}
});
it('falls through to CommandExecutionError for other non-zero codes', () => {
expect(() => assertOk({ code: 99, message: 'something else' }))
.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError on malformed (non-object) response', () => {
expect(() => assertOk(null)).toThrow(CommandExecutionError);
expect(() => assertOk('not-an-object')).toThrow(CommandExecutionError);
});
});
+80
View File
@@ -0,0 +1,80 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import {
emptySearchResults,
requireBoundedInteger,
requireNonNegativeInteger,
requireRows,
requireSearchQuery,
runBrowserStep,
toHttpsUrl,
} from '../_shared/search-adapter.js';
function buildExtractorJs(limit) {
return `
(function() {
var results = [];
var seen = {};
var items = document.querySelectorAll('.snippet');
for (var i = 0; i < items.length; i++) {
if (results.length >= ${limit}) break;
var el = items[i];
if (el.classList.contains('standalone') || el.classList.contains('ad')) continue;
var titleEl = el.querySelector('.search-snippet-title');
var snippetEl = el.querySelector('.generic-snippet .content');
var linkEl = el.querySelector('.result-content a');
if (!titleEl) continue;
var title = titleEl.textContent.trim();
var href = linkEl ? linkEl.getAttribute('href') || '' : '';
var snippet = snippetEl ? snippetEl.textContent.trim() : '';
if (!title || !href || seen[href]) continue;
if (href.indexOf('/') === 0) continue;
seen[href] = true;
results.push([title, href, snippet]);
}
return results;
})()`;
}
const command = cli({
site: 'brave',
name: 'search',
access: 'read',
description: 'Search Brave Search',
domain: 'search.brave.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Search query' },
{ name: 'limit', type: 'int', default: 10, help: 'Number of results per page (max 18)' },
{ name: 'offset', type: 'int', default: 0, help: 'Page offset (0, 1, 2...). Brave returns ~18 results per page' },
],
columns: ['rank', 'title', 'url', 'snippet'],
func: async (page, kwargs) => {
const limit = requireBoundedInteger(kwargs.limit, 10, 1, 18, '--limit');
const query = requireSearchQuery(kwargs.keyword);
const keyword = encodeURIComponent(query);
const offset = requireNonNegativeInteger(kwargs.offset, 0, '--offset');
let url = `https://search.brave.com/search?q=${keyword}`;
if (offset > 0) url += `&offset=${offset}`;
await runBrowserStep('brave search navigation', () => page.goto(url));
try {
await page.wait({ selector: '.snippet', timeout: 10 });
} catch {
await page.wait(3).catch(function() {});
}
const raw = await runBrowserStep('brave search extraction', () => page.evaluate(buildExtractorJs(limit)));
const results = requireRows(raw, 'brave search');
if (results.length === 0) {
throw emptySearchResults('Brave', query);
}
const rows = results
.map(function(r, index) {
return { rank: index + 1 + offset * 18, title: r[0], url: toHttpsUrl(r[1], 'https://search.brave.com'), snippet: r[2] };
})
.filter((row) => row.url);
if (rows.length === 0) throw emptySearchResults('Brave', query);
return rows;
},
});
export const __test__ = { command };
+76
View File
@@ -0,0 +1,76 @@
import { describe, it, expect, vi } from 'vitest';
const { __test__ } = await import('./search.js');
const command = __test__.command;
function createPageMock(evaluateResult = []) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('brave search', () => {
it('should register as a valid command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('brave');
expect(command.name).toBe('search');
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.strategy).toBe('public');
expect(command.domain).toBe('search.brave.com');
});
it('should define keyword positional arg', () => {
const kwArg = command.args.find(a => a.name === 'keyword');
expect(kwArg).toBeDefined();
expect(kwArg.positional).toBe(true);
expect(kwArg.required).toBe(true);
});
it('should define limit arg with default 10', () => {
const limitArg = command.args.find(a => a.name === 'limit');
expect(limitArg).toBeDefined();
expect(limitArg.type).toBe('int');
expect(limitArg.default).toBe(10);
});
it('should define output columns', () => {
expect(command.columns).toContain('rank');
expect(command.columns).toContain('title');
expect(command.columns).toContain('url');
expect(command.columns).toContain('snippet');
});
it('rejects empty query, invalid limit, and invalid offset before navigation', async () => {
const page = createPageMock();
await expect(command.func(page, { keyword: '', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 19 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 5, offset: -1 })).rejects.toMatchObject({ code: 'ARGUMENT' });
expect(page.goto).not.toHaveBeenCalled();
});
it('unwraps browser envelopes and returns ranked HTTPS rows', async () => {
const page = createPageMock({
session: 'site:brave',
data: [['OpenCLI', 'https://github.com/jackwener/OpenCLI', 'CLI browser tooling']],
});
await expect(command.func(page, { keyword: 'opencli', limit: 1, offset: 1 })).resolves.toEqual([{
rank: 19,
title: 'OpenCLI',
url: 'https://github.com/jackwener/OpenCLI',
snippet: 'CLI browser tooling',
}]);
});
it('fails typed instead of silently returning [] for malformed extraction payloads', async () => {
const page = createPageMock({ rows: [] });
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('payload shape'),
});
});
});
+58
View File
@@ -0,0 +1,58 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import {
CHATGPT_DOMAIN,
CHATGPT_URL,
ensureChatGPTComposer,
ensureOnChatGPT,
getBubbleCount,
normalizeBooleanFlag,
requireNonEmptyPrompt,
requirePositiveInt,
sendChatGPTMessage,
startNewChat,
waitForChatGPTResponse,
} from './utils.js';
export const askCommand = cli({
site: 'chatgpt',
name: 'ask',
access: 'write',
description: 'Send a prompt to ChatGPT web and wait for the response',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
{ name: 'timeout', type: 'int', default: 120, help: 'Max seconds to wait for response' },
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
],
columns: ['response'],
func: async (page, kwargs) => {
const prompt = requireNonEmptyPrompt(kwargs.prompt, 'chatgpt ask');
const timeout = requirePositiveInt(
Number(kwargs.timeout ?? 120),
'chatgpt ask --timeout',
'Example: opencli chatgpt ask "hello" --timeout 120',
);
if (normalizeBooleanFlag(kwargs.new)) {
await startNewChat(page);
} else {
await ensureOnChatGPT(page);
}
// startNewChat / ensureOnChatGPT now wait for the composer selector
// after navigating, so the previous standalone 2 s settle is redundant.
await ensureChatGPTComposer(page, 'ChatGPT ask requires a logged-in ChatGPT session with a visible composer.');
const baseline = await getBubbleCount(page);
const sent = await sendChatGPTMessage(page, prompt);
if (!sent) {
throw new CommandExecutionError('Failed to send message to ChatGPT', `Open ${CHATGPT_URL} and verify the composer is ready.`);
}
return [{ response: await waitForChatGPTResponse(page, baseline, prompt, timeout) }];
},
});
+45
View File
@@ -0,0 +1,45 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './ask.js';
import './send.js';
import './read.js';
import './history.js';
import './detail.js';
import './new.js';
import './status.js';
import './image.js';
describe('chatgpt browser command registration', () => {
it('registers the baseline web chat commands with persistent site sessions', () => {
const expectedAccess = {
ask: 'write',
send: 'write',
read: 'read',
history: 'read',
detail: 'read',
new: 'read',
status: 'read',
image: 'write',
};
for (const [name, access] of Object.entries(expectedAccess)) {
const cmd = getRegistry().get(`chatgpt/${name}`);
expect(cmd, `chatgpt/${name}`).toBeDefined();
expect(cmd.site).toBe('chatgpt');
expect(cmd.domain).toBe('chatgpt.com');
expect(cmd.strategy).toBe('cookie');
expect(cmd.browser).toBe(true);
expect(cmd.siteSession).toBe('persistent');
expect(cmd.navigateBefore).toBe(false);
expect(cmd.access).toBe(access);
}
});
it('keeps ask timeout as the runtime-visible integer timeout arg', () => {
const ask = getRegistry().get('chatgpt/ask');
expect(ask.args).toEqual(expect.arrayContaining([
expect.objectContaining({ name: 'timeout', type: 'int', default: 120 }),
expect.objectContaining({ name: 'new', type: 'boolean', default: false }),
]));
});
});
+51
View File
@@ -0,0 +1,51 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import {
CHATGPT_DOMAIN,
CHATGPT_URL,
CONVERSATION_MESSAGE_SELECTOR,
ensureChatGPTLogin,
getVisibleMessages,
messageHtmlToMarkdown,
normalizeBooleanFlag,
parseChatGPTConversationId,
} from './utils.js';
export const detailCommand = cli({
site: 'chatgpt',
name: 'detail',
access: 'read',
description: 'Open a ChatGPT web conversation by ID and read its messages',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', positional: true, required: true, help: 'Conversation ID or full /c/<id> URL' },
{ name: 'markdown', type: 'boolean', default: false, help: 'Emit assistant replies as markdown' },
],
columns: ['Index', 'Role', 'Text'],
func: async (page, kwargs) => {
const id = parseChatGPTConversationId(kwargs.id);
const wantMarkdown = normalizeBooleanFlag(kwargs.markdown, false);
await page.goto(`${CHATGPT_URL}/c/${id}`, { settleMs: 2000 });
try {
await page.wait({ selector: CONVERSATION_MESSAGE_SELECTOR, timeout: 10 });
} catch {
// Empty conversation, missing access, or login redirect — handled by ensureChatGPTLogin / EmptyResultError below.
}
await ensureChatGPTLogin(page, 'ChatGPT detail requires a logged-in ChatGPT session.');
const messages = await getVisibleMessages(page);
if (!messages.length) {
throw new EmptyResultError('chatgpt detail', `No visible ChatGPT messages were found for conversation ${id}.`);
}
return messages.map((message) => ({
Index: message.Index,
Role: message.Role,
Text: wantMarkdown && message.Role === 'Assistant' && message.Html
? (messageHtmlToMarkdown(message.Html) || message.Text)
: message.Text,
}));
},
});
+108
View File
@@ -0,0 +1,108 @@
import { describe, expect, it } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import {
requireArrayEvaluateResult,
requireBooleanEvaluateResult,
requireObjectEvaluateResult,
unwrapEvaluateResult,
} from './utils.js';
describe('chatgpt page.evaluate envelope helpers', () => {
describe('unwrapEvaluateResult', () => {
it('unwraps a { session, data } envelope produced by the browser bridge', () => {
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 'msg-1' }] };
expect(unwrapEvaluateResult(envelope)).toEqual([{ id: 'msg-1' }]);
});
it('passes raw arrays through unchanged (back-compat with older bridge versions)', () => {
const raw = [1, 2, 3];
expect(unwrapEvaluateResult(raw)).toBe(raw);
});
it('passes primitive return values (URL strings, booleans) through unchanged', () => {
expect(unwrapEvaluateResult('https://chatgpt.com/c/abc')).toBe('https://chatgpt.com/c/abc');
expect(unwrapEvaluateResult(true)).toBe(true);
expect(unwrapEvaluateResult(0)).toBe(0);
});
it('passes plain non-envelope objects through unchanged', () => {
const obj = { ok: true, reason: 'all good' };
expect(unwrapEvaluateResult(obj)).toBe(obj);
});
it('handles null and undefined defensively', () => {
expect(unwrapEvaluateResult(null)).toBe(null);
expect(unwrapEvaluateResult(undefined)).toBe(undefined);
});
});
describe('requireArrayEvaluateResult', () => {
it('returns the payload when it is an array', () => {
const rows = [{ id: 1 }, { id: 2 }];
expect(requireArrayEvaluateResult(rows, 'chatgpt test')).toBe(rows);
});
it('throws a typed CommandExecutionError when the payload is the raw envelope (caller forgot to unwrap)', () => {
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 1 }] };
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
.toThrowError(CommandExecutionError);
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
.toThrow(/malformed extraction payload/);
});
it('surfaces the inner error message when the payload carries an `error` field', () => {
const errPayload = { error: 'image generator returned 500' };
expect(() => requireArrayEvaluateResult(errPayload, 'chatgpt image asset export'))
.toThrow(/chatgpt image asset export: image generator returned 500/);
});
it('throws when the payload is null or a primitive', () => {
expect(() => requireArrayEvaluateResult(null, 'chatgpt test')).toThrowError(CommandExecutionError);
expect(() => requireArrayEvaluateResult('a string', 'chatgpt test')).toThrowError(CommandExecutionError);
});
});
describe('requireObjectEvaluateResult', () => {
it('returns the payload when it is a plain object', () => {
const obj = { url: 'https://chatgpt.com', isLoggedIn: true };
expect(requireObjectEvaluateResult(obj, 'chatgpt page state')).toBe(obj);
});
it('throws when the payload is an array or a primitive', () => {
expect(() => requireObjectEvaluateResult([], 'chatgpt page state')).toThrowError(CommandExecutionError);
expect(() => requireObjectEvaluateResult('string', 'chatgpt page state')).toThrowError(CommandExecutionError);
expect(() => requireObjectEvaluateResult(null, 'chatgpt page state')).toThrowError(CommandExecutionError);
});
});
describe('requireBooleanEvaluateResult', () => {
it('returns booleans and rejects wrong-shape values', () => {
expect(requireBooleanEvaluateResult(true, 'chatgpt generation state')).toBe(true);
expect(requireBooleanEvaluateResult(false, 'chatgpt generation state')).toBe(false);
expect(() => requireBooleanEvaluateResult({ ok: true }, 'chatgpt generation state'))
.toThrowError(CommandExecutionError);
});
});
describe('end-to-end envelope sweep', () => {
// The bridge envelope is shaped like { session, data } where `session` is
// any string and `data` is the actual return value. Verify the helpers
// chain correctly: unwrap → require* yields the inner shape.
it('unwrap + requireArray pipes an envelope through to the underlying array', () => {
const envelope = {
session: 'site:chatgpt:img-export',
data: [
{ url: 'https://a.example/1.png', dataUrl: 'data:image/png;base64,xxx', mimeType: 'image/png' },
],
};
expect(requireArrayEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt image asset export'))
.toEqual(envelope.data);
});
it('unwrap + requireObject pipes an envelope through to the underlying object', () => {
const envelope = { session: 'site:chatgpt:state', data: { url: 'https://chatgpt.com', isLoggedIn: true } };
expect(requireObjectEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt page state'))
.toEqual(envelope.data);
});
});
});
+39
View File
@@ -0,0 +1,39 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import {
CHATGPT_DOMAIN,
ensureChatGPTLogin,
ensureOnChatGPT,
getConversationList,
requirePositiveInt,
} from './utils.js';
export const historyCommand = cli({
site: 'chatgpt',
name: 'history',
access: 'read',
description: 'List visible ChatGPT web conversation history from the sidebar',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Max conversations to show' },
],
columns: ['Index', 'Id', 'Title', 'Url'],
func: async (page, kwargs) => {
const limit = requirePositiveInt(
Number(kwargs.limit ?? 20),
'chatgpt history --limit',
'Example: opencli chatgpt history --limit 20',
);
await ensureOnChatGPT(page);
await ensureChatGPTLogin(page, 'ChatGPT history requires a logged-in ChatGPT session.');
const conversations = await getConversationList(page);
if (!conversations.length) {
throw new EmptyResultError('chatgpt history', 'No ChatGPT conversation links were visible in the sidebar.');
}
return conversations.slice(0, limit);
},
});
+43 -11
View File
@@ -4,7 +4,7 @@ import * as fs from 'node:fs';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { saveBase64ToFile } from '@jackwener/opencli/utils';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getChatGPTVisibleImageUrls, sendChatGPTMessage, waitForChatGPTImages, getChatGPTImageAssets } from './utils.js';
import { clearChatGPTDraft, getChatGPTVisibleImageUrls, normalizeBooleanFlag, prepareChatGPTImagePaths, sendChatGPTMessage, unwrapEvaluateResult, waitForChatGPTImages, getChatGPTImageAssets, uploadChatGPTImages } from './utils.js';
const CHATGPT_DOMAIN = 'chatgpt.com';
@@ -15,12 +15,6 @@ function extFromMime(mime) {
return '.jpg';
}
function normalizeBooleanFlag(value) {
if (typeof value === 'boolean') return value;
const normalized = String(value ?? '').trim().toLowerCase();
return normalized === 'true' || normalized === '1' || normalized === 'yes' || normalized === 'on';
}
function displayPath(filePath) {
const home = os.homedir();
return filePath.startsWith(home) ? `~${filePath.slice(home.length)}` : filePath;
@@ -42,8 +36,25 @@ export function nextAvailablePath(dir, baseName, ext, existsSync = fs.existsSync
return candidate;
}
export function parseImagePaths(value) {
if (Array.isArray(value)) {
return value.flatMap(item => parseImagePaths(item));
}
return String(value ?? '')
.split(',')
.map(item => item.trim())
.filter(Boolean);
}
function buildPrompt(prompt, imageCount) {
if (imageCount > 0) {
return `Edit the attached image${imageCount === 1 ? '' : 's'}: ${prompt}`;
}
return `Generate an image of: ${prompt}`;
}
async function currentChatGPTLink(page) {
const url = await page.evaluate('window.location.href').catch(() => '');
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
return typeof url === 'string' && url ? url : 'https://chatgpt.com';
}
@@ -55,10 +66,12 @@ export const imageCommand = cli({
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
defaultFormat: 'plain',
args: [
{ name: 'prompt', positional: true, required: true, help: 'Image prompt to send to ChatGPT' },
{ name: 'image', help: 'Local image path to attach before prompting; comma-separated paths are supported' },
{ name: 'op', help: 'Output directory (default: ~/Pictures/chatgpt)' },
{ name: 'sd', type: 'boolean', default: false, help: 'Skip download shorthand; only show ChatGPT link' },
{ name: 'timeout', type: 'int', required: false, default: 240, help: 'Max seconds for the overall command (default: 240)' },
@@ -66,6 +79,7 @@ export const imageCommand = cli({
columns: ['status', 'file', 'link'],
func: async (page, kwargs) => {
const prompt = kwargs.prompt;
const imagePaths = parseImagePaths(kwargs.image);
const outputDir = resolveOutputDir(kwargs.op);
const skipDownloadRaw = kwargs.sd;
const skipDownload = skipDownloadRaw === '' || skipDownloadRaw === true || normalizeBooleanFlag(skipDownloadRaw);
@@ -73,16 +87,34 @@ export const imageCommand = cli({
if (!Number.isInteger(timeout) || timeout < 1) {
throw new ArgumentError('--timeout must be a positive integer (seconds)');
}
const preparedImages = imagePaths.length ? await prepareChatGPTImagePaths(imagePaths) : { ok: true, paths: [] };
if (!preparedImages.ok) {
throw new ArgumentError(preparedImages.reason);
}
// Navigate to chatgpt.com/new with full reload to clear React sidebar state
await page.goto(`https://${CHATGPT_DOMAIN}/new`, { settleMs: 2000 });
await clearChatGPTDraft(page);
if (imagePaths.length) {
let upload;
try {
upload = await uploadChatGPTImages(page, preparedImages.paths);
} catch (err) {
throw new CommandExecutionError(`Failed to upload image to ChatGPT: ${err instanceof Error ? err.message : String(err)}`);
}
if (!upload?.ok) throw new CommandExecutionError(upload?.reason || 'Failed to upload image to ChatGPT');
}
const beforeUrls = await getChatGPTVisibleImageUrls(page);
// Send the image generation prompt - must be explicit
const sent = await sendChatGPTMessage(page, `Generate an image of: ${prompt}`);
// Send an explicit generation/editing prompt so ChatGPT returns image assets.
const sent = await sendChatGPTMessage(page, buildPrompt(prompt, imagePaths.length));
if (!sent) {
return [{ status: '⚠️ send-failed', file: '📁 -', link: `🔗 ${await currentChatGPTLink(page)}` }];
throw new CommandExecutionError(
'Failed to send image prompt to ChatGPT',
`Open ${await currentChatGPTLink(page)} and verify the composer is ready.`,
);
}
// ChatGPT briefly navigates to /c/{id} after sending, then may
+95 -1
View File
@@ -4,15 +4,33 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
getChatGPTVisibleImageUrls: vi.fn(),
clearChatGPTDraft: vi.fn(),
prepareChatGPTImagePaths: vi.fn(),
sendChatGPTMessage: vi.fn(),
uploadChatGPTImages: vi.fn(),
waitForChatGPTImages: vi.fn(),
getChatGPTImageAssets: vi.fn(),
saveBase64ToFile: vi.fn(),
}));
vi.mock('./utils.js', () => ({
clearChatGPTDraft: mocks.clearChatGPTDraft,
getChatGPTVisibleImageUrls: mocks.getChatGPTVisibleImageUrls,
normalizeBooleanFlag: (value, fallback = false) => {
if (typeof value === 'boolean') return value;
if (value == null || value === '') return fallback;
const normalized = String(value).trim().toLowerCase();
return normalized === 'true' || normalized === '1' || normalized === 'yes' || normalized === 'on';
},
prepareChatGPTImagePaths: mocks.prepareChatGPTImagePaths,
sendChatGPTMessage: mocks.sendChatGPTMessage,
unwrapEvaluateResult: (payload) => {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
},
uploadChatGPTImages: mocks.uploadChatGPTImages,
waitForChatGPTImages: mocks.waitForChatGPTImages,
getChatGPTImageAssets: mocks.getChatGPTImageAssets,
}));
@@ -21,7 +39,7 @@ vi.mock('@jackwener/opencli/utils', () => ({
saveBase64ToFile: mocks.saveBase64ToFile,
}));
const { imageCommand, nextAvailablePath, resolveOutputDir } = await import('./image.js');
const { imageCommand, nextAvailablePath, parseImagePaths, resolveOutputDir } = await import('./image.js');
function createPage() {
return {
@@ -33,8 +51,11 @@ function createPage() {
beforeEach(() => {
vi.restoreAllMocks();
mocks.clearChatGPTDraft.mockReset().mockResolvedValue(undefined);
mocks.prepareChatGPTImagePaths.mockReset().mockImplementation(async (paths) => ({ ok: true, paths }));
mocks.getChatGPTVisibleImageUrls.mockReset().mockResolvedValue([]);
mocks.sendChatGPTMessage.mockReset().mockResolvedValue(true);
mocks.uploadChatGPTImages.mockReset().mockResolvedValue({ ok: true });
mocks.waitForChatGPTImages.mockReset().mockResolvedValue(['https://images.example/generated.png']);
mocks.getChatGPTImageAssets.mockReset().mockResolvedValue([{
url: 'https://images.example/generated.png',
@@ -60,9 +81,82 @@ describe('chatgpt image output paths', () => {
expect(nextAvailablePath(dir, 'chatgpt_123', '.png', (file) => taken.has(file))).toBe(path.join(dir, 'chatgpt_123_2.png'));
});
it('parses comma-separated image paths', () => {
expect(parseImagePaths('/tmp/a.png, /tmp/b.jpg')).toEqual(['/tmp/a.png', '/tmp/b.jpg']);
expect(parseImagePaths([' /tmp/a.png ', '/tmp/b.jpg,/tmp/c.webp'])).toEqual(['/tmp/a.png', '/tmp/b.jpg', '/tmp/c.webp']);
});
});
describe('chatgpt image upload flow', () => {
it('uploads local images before sending an edit prompt', async () => {
mocks.prepareChatGPTImagePaths.mockResolvedValue({ ok: true, paths: ['/abs/cat.png', '/abs/dog.jpg'] });
await imageCommand.func(createPage(), {
prompt: 'make the background blue',
image: '/tmp/cat.png,/tmp/dog.jpg',
op: '',
sd: true,
timeout: 240,
});
expect(mocks.clearChatGPTDraft).toHaveBeenCalled();
expect(mocks.uploadChatGPTImages).toHaveBeenCalledWith(expect.anything(), ['/abs/cat.png', '/abs/dog.jpg']);
expect(mocks.uploadChatGPTImages.mock.invocationCallOrder[0]).toBeLessThan(
mocks.getChatGPTVisibleImageUrls.mock.invocationCallOrder[0],
);
expect(mocks.sendChatGPTMessage).toHaveBeenCalledWith(expect.anything(), 'Edit the attached images: make the background blue');
});
it('rejects invalid local image paths before browser navigation', async () => {
mocks.prepareChatGPTImagePaths.mockResolvedValue({ ok: false, reason: 'Image not found: /tmp/missing.png' });
const page = createPage();
await expect(imageCommand.func(page, {
prompt: 'make the background blue',
image: '/tmp/missing.png',
op: '',
sd: false,
timeout: 240,
})).rejects.toMatchObject({
code: 'ARGUMENT',
message: expect.stringContaining('Image not found'),
});
expect(page.goto).not.toHaveBeenCalled();
expect(mocks.uploadChatGPTImages).not.toHaveBeenCalled();
});
it('surfaces upload failures as command execution errors', async () => {
mocks.uploadChatGPTImages.mockResolvedValue({ ok: false, reason: 'image upload preview did not appear' });
await expect(imageCommand.func(createPage(), {
prompt: 'make the background blue',
image: '/tmp/cat.png',
op: '',
sd: false,
timeout: 240,
})).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('image upload preview did not appear'),
});
});
});
describe('chatgpt image failure contracts', () => {
it('fails fast when the image prompt cannot be sent', async () => {
mocks.sendChatGPTMessage.mockResolvedValue(false);
await expect(imageCommand.func(createPage(), {
prompt: 'cat',
op: '',
sd: false,
timeout: 240,
})).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('Failed to send image prompt to ChatGPT'),
});
expect(mocks.waitForChatGPTImages).not.toHaveBeenCalled();
});
it('fails fast when image generation detection finds no new images', async () => {
mocks.waitForChatGPTImages.mockResolvedValue([]);
+25
View File
@@ -0,0 +1,25 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import {
CHATGPT_DOMAIN,
ensureChatGPTComposer,
startNewChat,
} from './utils.js';
export const newCommand = cli({
site: 'chatgpt',
name: 'new',
access: 'read',
description: 'Start a new ChatGPT web conversation',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status'],
func: async (page) => {
await startNewChat(page);
await ensureChatGPTComposer(page, 'ChatGPT new requires a logged-in ChatGPT session with a visible composer.');
return [{ Status: 'New chat started' }];
},
});
+44
View File
@@ -0,0 +1,44 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import {
CHATGPT_DOMAIN,
ensureChatGPTLogin,
ensureOnChatGPT,
getVisibleMessages,
messageHtmlToMarkdown,
normalizeBooleanFlag,
} from './utils.js';
export const readCommand = cli({
site: 'chatgpt',
name: 'read',
access: 'read',
description: 'Read messages in the current ChatGPT web conversation',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'markdown', type: 'boolean', default: false, help: 'Emit assistant replies as markdown' },
],
columns: ['Index', 'Role', 'Text'],
func: async (page, kwargs) => {
const wantMarkdown = normalizeBooleanFlag(kwargs.markdown, false);
// ensureOnChatGPT now waits for the composer selector after navigating,
// so the previous standalone 2 s settle is redundant.
await ensureOnChatGPT(page);
await ensureChatGPTLogin(page, 'ChatGPT read requires a logged-in ChatGPT session.');
const messages = await getVisibleMessages(page);
if (!messages.length) {
throw new EmptyResultError('chatgpt read', 'No visible ChatGPT messages were found in the current conversation.');
}
return messages.map((message) => ({
Index: message.Index,
Role: message.Role,
Text: wantMarkdown && message.Role === 'Assistant' && message.Html
? (messageHtmlToMarkdown(message.Html) || message.Text)
: message.Text,
}));
},
});
+47
View File
@@ -0,0 +1,47 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import {
CHATGPT_DOMAIN,
CHATGPT_URL,
ensureChatGPTComposer,
ensureOnChatGPT,
normalizeBooleanFlag,
requireNonEmptyPrompt,
sendChatGPTMessage,
startNewChat,
} from './utils.js';
export const sendCommand = cli({
site: 'chatgpt',
name: 'send',
access: 'write',
description: 'Send a prompt to ChatGPT web without waiting for the response',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
],
columns: ['Status', 'InjectedText'],
func: async (page, kwargs) => {
const prompt = requireNonEmptyPrompt(kwargs.prompt, 'chatgpt send');
if (normalizeBooleanFlag(kwargs.new)) {
await startNewChat(page);
} else {
await ensureOnChatGPT(page);
}
// startNewChat / ensureOnChatGPT now wait for the composer selector
// after navigating, so the previous standalone 2 s settle is redundant.
await ensureChatGPTComposer(page, 'ChatGPT send requires a logged-in ChatGPT session with a visible composer.');
const sent = await sendChatGPTMessage(page, prompt);
if (!sent) {
throw new CommandExecutionError('Failed to send message to ChatGPT', `Open ${CHATGPT_URL} and verify the composer is ready.`);
}
return [{ Status: 'Success', InjectedText: prompt }];
},
});
+29
View File
@@ -0,0 +1,29 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import {
CHATGPT_DOMAIN,
ensureOnChatGPT,
getPageState,
} from './utils.js';
export const statusCommand = cli({
site: 'chatgpt',
name: 'status',
access: 'read',
description: 'Check ChatGPT web page availability and login state',
domain: CHATGPT_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status', 'Login', 'Url'],
func: async (page) => {
await ensureOnChatGPT(page);
const state = await getPageState(page);
return [{
Status: state.hasComposer ? 'Connected' : 'Page not ready',
Login: state.isLoggedIn && !state.hasLoginGate ? 'Yes' : 'No',
Url: state.url,
}];
},
});
+673 -42
View File
@@ -1,18 +1,38 @@
/**
* ChatGPT web browser automation helpers for image generation.
* ChatGPT web browser automation helpers.
* Cross-platform: works on Linux/macOS/Windows via OpenCLI's CDP browser automation.
*/
import { htmlToMarkdown } from '@jackwener/opencli/utils';
import { ArgumentError, AuthRequiredError, CommandExecutionError, TimeoutError } from '@jackwener/opencli/errors';
export const CHATGPT_DOMAIN = 'chatgpt.com';
export const CHATGPT_URL = 'https://chatgpt.com';
// Selectors
const COMPOSER_SELECTORS = [
'[aria-label="Chat with ChatGPT"]',
'[aria-label="与 ChatGPT 聊天"]',
'[placeholder="Ask anything"]',
'[placeholder="有问题,尽管问"]',
'#prompt-textarea',
'[data-testid="prompt-textarea"]',
'[contenteditable="true"][role="textbox"]',
];
const SEND_BUTTON_SELECTOR = 'button[data-testid="send-button"]:not([disabled])';
const SEND_BUTTON_FALLBACK_SELECTORS = [
'#composer-submit-button:not([disabled])',
];
const SEND_BUTTON_LABELS = [
'Send prompt',
'Send message',
'Send',
'发送提示',
];
const CLOSE_SIDEBAR_LABELS = [
'Close sidebar',
'关闭边栏',
];
const SEND_BTN_SELECTOR = 'button[aria-label="Send prompt"]';
function isSameChatGPTConversation(currentUrl, expectedUrl) {
if (!currentUrl || !expectedUrl) return false;
@@ -54,10 +74,215 @@ function buildComposerLocatorScript() {
};
findComposer.toString = () => 'findComposer';
return { findComposer, markerAttr };
`;
}
export function normalizeBooleanFlag(value, fallback = false) {
if (typeof value === 'boolean') return value;
if (value == null || value === '') return fallback;
const normalized = String(value).trim().toLowerCase();
return normalized === 'true' || normalized === '1' || normalized === 'yes' || normalized === 'on';
}
export function requireNonEmptyPrompt(prompt, commandName) {
const text = String(prompt ?? '').trim();
if (!text) {
throw new ArgumentError(
`${commandName} prompt cannot be empty`,
`Example: opencli ${commandName} "hello"`,
);
}
return text;
}
export function requirePositiveInt(value, flagLabel, hint) {
if (!Number.isInteger(value) || value < 1) {
throw new ArgumentError(`${flagLabel} must be a positive integer`, hint);
}
return value;
}
// ─────────────────────────────────────────────────────────────────────────────
// page.evaluate envelope helpers.
//
// The browser bridge wraps every `page.evaluate(...)` return value in a
// `{ session, data }` envelope. Adapters that read `.length` or
// `Array.isArray(payload)` directly on the envelope silently see "no data" —
// this matches the failure mode fixed for xiaohongshu/rednote (#1561) and
// weibo (#1568).
//
// `unwrapEvaluateResult` is a defensive ternary: it unwraps when the payload
// looks like an envelope, otherwise passes the value through unchanged so
// older bridge versions and primitive return values still work.
// ─────────────────────────────────────────────────────────────────────────────
export function unwrapEvaluateResult(payload) {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
}
export function requireArrayEvaluateResult(payload, label) {
if (!Array.isArray(payload)) {
if (payload && typeof payload === 'object' && 'error' in payload) {
throw new CommandExecutionError(`${label}: ${String(payload.error)}`);
}
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function requireObjectEvaluateResult(payload, label) {
if (!payload || Array.isArray(payload) || typeof payload !== 'object') {
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function requireBooleanEvaluateResult(payload, label) {
if (typeof payload !== 'boolean') {
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function parseChatGPTConversationId(value) {
const raw = String(value ?? '').trim();
const match = raw.match(/(?:^|\/c\/)([A-Za-z0-9_-]{8,})(?:[/?#]|$)/);
if (match) return match[1];
if (/^[A-Za-z0-9_-]{8,}$/.test(raw)) return raw;
throw new ArgumentError(
'chatgpt detail requires a conversation id or /c/<id> URL',
'Example: opencli chatgpt detail 123e4567-e89b-12d3-a456-426614174000',
);
}
export async function currentChatGPTUrl(page) {
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
return typeof url === 'string' ? url : '';
}
export async function isOnChatGPT(page) {
const url = await currentChatGPTUrl(page);
if (!url) return false;
try {
const host = new URL(url).hostname;
return host === CHATGPT_DOMAIN || host.endsWith(`.${CHATGPT_DOMAIN}`);
} catch {
return false;
}
}
// Comma-joined CSS selector list passed to page.wait({ selector }) so the
// wait succeeds as soon as any composer flavour mounts (querySelectorAll
// matches all of them). Tracks the most stable subset of COMPOSER_SELECTORS;
// we only need to know "the composer is ready", not which variant rendered.
const COMPOSER_WAIT_SELECTOR = '#prompt-textarea, [data-testid="prompt-textarea"]';
const CONVERSATION_LINK_SELECTOR = 'a[href*="/c/"]';
// Selector used by detail.js to wait for at least one rendered message bubble
// after navigating to /c/<id>; mirrors the markup queried by getVisibleMessages.
export const CONVERSATION_MESSAGE_SELECTOR = '[data-message-author-role], article[data-testid*="conversation-turn"]';
export async function ensureOnChatGPT(page) {
if (await isOnChatGPT(page)) return false;
await page.goto(CHATGPT_URL, { settleMs: 2000 });
try {
await page.wait({ selector: COMPOSER_WAIT_SELECTOR, timeout: 8 });
} catch {
// Composer didn't mount; downstream ensureChatGPTLogin / ensureChatGPTComposer surfaces a typed error.
}
return true;
}
export async function startNewChat(page) {
await page.goto(`${CHATGPT_URL}/new`, { settleMs: 2000 });
try {
await page.wait({ selector: COMPOSER_WAIT_SELECTOR, timeout: 8 });
} catch {
// Composer didn't mount; downstream ensureChatGPTComposer surfaces a typed error.
}
}
export async function getPageState(page) {
return requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
if (style.display === 'none' || style.visibility === 'hidden') return false;
const rect = el.getBoundingClientRect();
return rect.width > 0 && rect.height > 0;
};
const composerSelectors = ${JSON.stringify(COMPOSER_SELECTORS)};
const hasComposer = composerSelectors.some((selector) =>
Array.from(document.querySelectorAll(selector)).some((node) => isVisible(node))
);
const text = (document.body?.innerText || '').replace(/\\s+/g, ' ').trim();
const loginLink = Array.from(document.querySelectorAll('a, button')).find((node) => {
const label = ((node.innerText || node.textContent || '') + ' ' + (node.getAttribute('aria-label') || '')).trim().toLowerCase();
return isVisible(node) && /^(log in|login|sign up|sign in)$/.test(label);
});
const userMenu = document.querySelector('[data-testid="profile-button"], [aria-label*="Profile"], [aria-label*="Account"], button[id*="headlessui-menu-button"]');
const hasLoginGate = !!loginLink || /log in to chatgpt|sign up to chatgpt|welcome to chatgpt/i.test(text);
return {
url: window.location.href,
title: document.title,
hasComposer,
isLoggedIn: hasComposer || !!userMenu || !hasLoginGate,
hasLoginGate,
};
})()`)), 'chatgpt page state');
}
export async function ensureChatGPTLogin(page, message = 'ChatGPT requires a logged-in browser session.') {
const state = await getPageState(page);
if (!state.isLoggedIn || state.hasLoginGate) {
throw new AuthRequiredError(CHATGPT_DOMAIN, message);
}
return state;
}
export async function ensureChatGPTComposer(page, message = 'ChatGPT composer is not available on the current page.') {
const state = await ensureChatGPTLogin(page, message);
if (!state.hasComposer) {
throw new CommandExecutionError(message);
}
return state;
}
export async function clearChatGPTDraft(page) {
await page.evaluate(`
(() => {
const removeLabels = [/^remove file/i, /^移除文件/];
for (let pass = 0; pass < 10; pass += 1) {
const button = Array.from(document.querySelectorAll('button')).find((node) => {
const label = node.getAttribute('aria-label') || '';
return removeLabels.some((pattern) => pattern.test(label));
});
if (!button) break;
button.click();
}
const selectors = ${JSON.stringify(COMPOSER_SELECTORS)};
for (const selector of selectors) {
for (const node of document.querySelectorAll(selector)) {
if (!(node instanceof HTMLElement)) continue;
if (node instanceof HTMLTextAreaElement || node instanceof HTMLInputElement) {
node.value = '';
} else if (node.isContentEditable) {
node.textContent = '';
node.innerHTML = '<p><br></p>';
} else {
node.textContent = '';
}
node.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'deleteContentBackward', data: null }));
node.dispatchEvent(new Event('change', { bubbles: true }));
}
}
})()
`);
await page.wait(0.5);
}
/**
* Send a message to the ChatGPT composer and submit it.
* Returns true if the message was sent successfully.
@@ -66,26 +291,36 @@ export async function sendChatGPTMessage(page, text) {
// Close sidebar if open (it can cover the chat composer)
await page.evaluate(`
(() => {
const closeBtn = Array.from(document.querySelectorAll('button')).find(b => b.getAttribute('aria-label') === 'Close sidebar');
const labels = ${JSON.stringify(CLOSE_SIDEBAR_LABELS)};
const closeBtn = Array.from(document.querySelectorAll('button')).find(b => labels.includes(b.getAttribute('aria-label') || ''));
if (closeBtn) closeBtn.click();
})()
`);
await page.wait(0.5);
// The previous 0.5 s + 1.5 s pre-composer settles are dropped: the next
// page.evaluate roundtrip flushes the close-sidebar React update and
// findComposer() retries inside a single CDP call, so no fixed sleep is
// needed before reading the composer.
// Wait for composer to be ready and use Playwright's type()
await page.wait(1.5);
const typeResult = await page.evaluate(`
const typeResult = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
${buildComposerLocatorScript()}
const composer = findComposer();
if (!composer) return false;
composer.focus();
composer.textContent = '';
if (composer instanceof HTMLTextAreaElement || composer instanceof HTMLInputElement) {
composer.value = '';
} else if (composer.isContentEditable) {
composer.textContent = '';
composer.innerHTML = '<p><br></p>';
} else {
composer.textContent = '';
}
composer.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'deleteContentBackward', data: null }));
composer.dispatchEvent(new Event('change', { bubbles: true }));
return true;
})()
`);
`)), 'chatgpt composer readiness');
if (!typeResult) return false;
// Use page.type() which is Playwright's native method
@@ -109,50 +344,373 @@ export async function sendChatGPTMessage(page, text) {
`);
}
// Wait for send button to appear (it only shows when there's text)
await page.wait(1.5);
let sent = null;
for (let attempt = 0; attempt < 20; attempt += 1) {
await page.wait(0.5);
sent = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const isUsable = (button) => button
&& !button.disabled
&& button.getAttribute('aria-disabled') !== 'true';
const primary = document.querySelector(${JSON.stringify(SEND_BUTTON_SELECTOR)})
|| ${JSON.stringify(SEND_BUTTON_FALLBACK_SELECTORS)}.map(selector => document.querySelector(selector)).find(Boolean);
const btns = Array.from(document.querySelectorAll('button'));
const labels = ${JSON.stringify(SEND_BUTTON_LABELS)};
const sendBtn = isUsable(primary)
? primary
: btns.find(b => labels.includes(b.getAttribute('aria-label') || '') && isUsable(b));
return { sendBtnFound: !!sendBtn };
})()
`)), 'chatgpt send button readiness');
if (sent?.sendBtnFound) break;
}
// Click send button
const sent = await page.evaluate(`
(() => {
const btns = Array.from(document.querySelectorAll('button'));
const sendBtn = btns.find(b => b.getAttribute('aria-label') === 'Send prompt');
return { sendBtnFound: !!sendBtn };
})()
`);
if (!sent || !sent.sendBtnFound) {
if (!sent?.sendBtnFound) {
return false;
}
await page.evaluate(`
(() => {
const sendBtn = Array.from(document.querySelectorAll('button')).find(b => b.getAttribute('aria-label') === 'Send prompt');
const primary = document.querySelector(${JSON.stringify(SEND_BUTTON_SELECTOR)})
|| ${JSON.stringify(SEND_BUTTON_FALLBACK_SELECTORS)}.map(selector => document.querySelector(selector)).find(Boolean);
const labels = ${JSON.stringify(SEND_BUTTON_LABELS)};
const sendBtn = primary || Array.from(document.querySelectorAll('button')).find(b => labels.includes(b.getAttribute('aria-label') || '') && !b.disabled);
if (sendBtn) sendBtn.click();
})()
`);
return true;
}
export async function getVisibleMessages(page) {
const result = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
if (style.display === 'none' || style.visibility === 'hidden') return false;
const rect = el.getBoundingClientRect();
return rect.width > 0 && rect.height > 0;
};
const normalize = (value) => String(value || '').replace(/\\u00a0/g, ' ').replace(/[ \\t]+\\n/g, '\\n').replace(/\\n{3,}/g, '\\n\\n').trim();
const roleOf = (node) => {
const attr = node.getAttribute('data-message-author-role') || node.getAttribute('data-author') || '';
if (/assistant/i.test(attr)) return 'Assistant';
if (/user/i.test(attr)) return 'User';
const testid = node.getAttribute('data-testid') || '';
if (/assistant/i.test(testid)) return 'Assistant';
if (/user/i.test(testid)) return 'User';
const label = node.getAttribute('aria-label') || '';
if (/assistant|chatgpt/i.test(label)) return 'Assistant';
if (/you|user/i.test(label)) return 'User';
return '';
};
let nodes = Array.from(document.querySelectorAll('[data-message-author-role], article[data-testid*="conversation-turn"]'));
nodes = nodes.filter((node) => node instanceof HTMLElement && isVisible(node));
const rows = [];
const seen = new Set();
for (const node of nodes) {
let role = roleOf(node);
const roleNode = node.querySelector('[data-message-author-role], [data-author]');
if (!role && roleNode) role = roleOf(roleNode);
if (!role) continue;
const contentNode = node.querySelector('[data-message-author-role] .markdown')
|| node.querySelector('.markdown')
|| node.querySelector('[data-message-author-role]')
|| node;
const html = contentNode instanceof HTMLElement ? (contentNode.innerHTML || '') : '';
const text = normalize(contentNode instanceof HTMLElement ? (contentNode.innerText || contentNode.textContent || '') : '');
if (!text) continue;
const key = role + '\\n' + text;
if (seen.has(key)) continue;
seen.add(key);
rows.push({ role, text, html });
}
return rows;
})()`)), 'chatgpt visible messages');
return result.map((item, index) => ({
Index: index + 1,
Role: item?.role === 'Assistant' ? 'Assistant' : 'User',
Text: String(item?.text || '').trim(),
Html: String(item?.html || ''),
})).filter((item) => item.Text);
}
export function messageHtmlToMarkdown(html) {
try {
return htmlToMarkdown(html).trim();
} catch {
return String(html || '').replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim();
}
}
export async function getBubbleCount(page) {
const messages = await getVisibleMessages(page);
return messages.length;
}
export async function waitForChatGPTResponse(page, baselineCount, prompt, timeoutSeconds) {
const startTime = Date.now();
let lastText = '';
let stableCount = 0;
while (Date.now() - startTime < timeoutSeconds * 1000) {
await page.wait(3);
if (await isGenerating(page)) {
stableCount = 0;
continue;
}
const messages = await getVisibleMessages(page);
const newMessages = messages.slice(Math.max(0, baselineCount));
const assistant = [...newMessages].reverse().find((m) => m.Role === 'Assistant')
|| [...messages].reverse().find((m) => m.Role === 'Assistant');
const candidate = String(assistant?.Text || '').trim();
if (!candidate || candidate === String(prompt || '').trim()) continue;
if (candidate === lastText) {
stableCount += 1;
if (stableCount >= 2) return candidate;
} else {
lastText = candidate;
stableCount = 0;
}
}
throw new TimeoutError(
'chatgpt ask',
timeoutSeconds,
'No ChatGPT response appeared before timeout. Re-run with a higher --timeout if it is still generating.',
);
}
export async function getConversationList(page) {
// ensureOnChatGPT already waits for the composer selector after navigation,
// so the previous standalone 2 s settle is redundant.
await ensureOnChatGPT(page);
const openSidebar = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const button = Array.from(document.querySelectorAll('button'))
.find((node) => /open sidebar/i.test(node.getAttribute('aria-label') || ''));
if (button instanceof HTMLElement) {
button.click();
return true;
}
return false;
})()`)), 'chatgpt sidebar open state');
if (openSidebar) {
try {
await page.wait({ selector: CONVERSATION_LINK_SELECTOR, timeout: 3 });
} catch {
// Sidebar slide-in didn't surface conversation links; extractConversationLinks below tolerates empty and falls back to home goto.
}
}
let items = await extractConversationLinks(page);
if (!items.length) {
await page.goto(CHATGPT_URL, { settleMs: 2000 });
try {
await page.wait({ selector: CONVERSATION_LINK_SELECTOR, timeout: 8 });
} catch {
// No conversation links visible after fallback goto; extractConversationLinks returns empty.
}
items = await extractConversationLinks(page);
}
return items;
}
async function extractConversationLinks(page) {
const items = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
if (style.display === 'none' || style.visibility === 'hidden') return false;
const rect = el.getBoundingClientRect();
return rect.width > 0 && rect.height > 0;
};
const links = Array.from(document.querySelectorAll('a[href*="/c/"]'))
.filter((link) => link instanceof HTMLAnchorElement && isVisible(link));
const seen = new Set();
const rows = [];
for (const link of links) {
const href = link.getAttribute('href') || '';
const match = href.match(/\\/c\\/([^/?#]+)/);
if (!match || seen.has(match[1])) continue;
seen.add(match[1]);
const title = (link.innerText || link.textContent || '').replace(/\\s+/g, ' ').trim() || '(untitled)';
rows.push({
Id: match[1],
Title: title,
Url: href.startsWith('http') ? href : ('${CHATGPT_URL}' + href),
});
}
return rows;
})()`)), 'chatgpt conversation link extraction');
return items.map((item, index) => ({
Index: index + 1,
Id: String(item?.Id || ''),
Title: String(item?.Title || '(untitled)').trim() || '(untitled)',
Url: String(item?.Url || ''),
})).filter((item) => item.Id);
}
function imageMimeFromPath(filePath) {
const lower = String(filePath || '').toLowerCase();
if (lower.endsWith('.png')) return 'image/png';
if (lower.endsWith('.webp')) return 'image/webp';
if (lower.endsWith('.gif')) return 'image/gif';
if (lower.endsWith('.heic')) return 'image/heic';
if (lower.endsWith('.heif')) return 'image/heif';
return 'image/jpeg';
}
export async function prepareChatGPTImagePaths(imagePaths) {
const fs = await import('node:fs');
const path = await import('node:path');
const absPaths = imagePaths.map(filePath => path.default.resolve(filePath));
const allowedExts = new Set(['.jpg', '.jpeg', '.png', '.webp', '.gif', '.heic', '.heif']);
for (const absPath of absPaths) {
if (!fs.default.existsSync(absPath)) {
return { ok: false, reason: `Image not found: ${absPath}` };
}
const stat = fs.default.statSync(absPath);
if (!stat.isFile()) {
return { ok: false, reason: `Not a file: ${absPath}` };
}
if (stat.size > 25 * 1024 * 1024) {
return { ok: false, reason: `Image too large (${(stat.size / 1024 / 1024).toFixed(1)} MB). Max: 25 MB` };
}
const ext = path.default.extname(absPath).toLowerCase();
if (!allowedExts.has(ext)) {
return { ok: false, reason: `Unsupported image type: ${absPath}` };
}
}
return { ok: true, paths: absPaths };
}
async function waitForChatGPTUploadPreview(page, fileNames) {
const namesJson = JSON.stringify(fileNames);
for (let attempt = 0; attempt < 10; attempt += 1) {
await page.wait(1);
const ready = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const names = ${namesJson};
const text = document.body ? (document.body.innerText || '') : '';
const matchedNames = names.filter(name => text.includes(name)).length;
if (matchedNames >= names.length) return true;
const composer = document.querySelector('[aria-label="Chat with ChatGPT"], [placeholder="Ask anything"], #prompt-textarea');
let root = composer;
for (let i = 0; i < 6 && root && root.parentElement; i += 1) root = root.parentElement;
const scope = root || document.body;
if (!scope) return false;
const previewNodes = scope.querySelectorAll('img[src], canvas, video, [style*="background-image"], [data-testid*="attachment"], [data-testid*="upload"], [class*="attachment"], [class*="upload"]');
return previewNodes.length >= names.length;
})()
`)), 'chatgpt upload preview detection');
if (ready) return true;
}
return false;
}
export async function uploadChatGPTImages(page, imagePaths) {
const fs = await import('node:fs');
const path = await import('node:path');
const prepared = await prepareChatGPTImagePaths(imagePaths);
if (!prepared.ok) return prepared;
const absPaths = prepared.paths;
const fileNames = absPaths.map(filePath => path.default.basename(filePath));
let uploaded = false;
if (page.setFileInput) {
try {
await page.setFileInput(absPaths, 'input[type="file"]');
uploaded = true;
} catch (err) {
const msg = String(err?.message || err);
if (!msg.includes('Unknown action') && !msg.includes('not supported') && !msg.includes('Not allowed') && !msg.includes('No element found')) {
throw err;
}
}
}
if (!uploaded) {
const files = absPaths.map(absPath => ({
name: path.default.basename(absPath),
mime: imageMimeFromPath(absPath),
base64: fs.default.readFileSync(absPath).toString('base64'),
}));
const fallbackResult = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const files = ${JSON.stringify(files)};
const input = document.querySelector('input[type="file"]');
if (!(input instanceof HTMLInputElement)) {
return { ok: false, reason: 'file input not found' };
}
const dt = new DataTransfer();
for (const item of files) {
const binary = atob(item.base64);
const bytes = new Uint8Array(binary.length);
for (let i = 0; i < binary.length; i += 1) bytes[i] = binary.charCodeAt(i);
dt.items.add(new File([bytes], item.name, { type: item.mime }));
}
input.files = dt.files;
const propsKey = Object.keys(input).find(key => key.startsWith('__reactProps$'));
if (propsKey && input[propsKey] && typeof input[propsKey].onChange === 'function') {
const nativeEvent = new Event('change', { bubbles: true });
input[propsKey].onChange({
target: input,
currentTarget: input,
nativeEvent,
preventDefault() {},
stopPropagation() {},
isDefaultPrevented() { return false; },
isPropagationStopped() { return false; },
persist() {},
});
} else {
input.dispatchEvent(new Event('input', { bubbles: true }));
input.dispatchEvent(new Event('change', { bubbles: true }));
}
return { ok: true };
})()
`)), 'chatgpt image upload fallback');
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
}
const ready = await waitForChatGPTUploadPreview(page, fileNames);
if (!ready) return { ok: false, reason: 'image upload preview did not appear' };
return { ok: true, files: absPaths };
}
/**
* Check if ChatGPT is still generating a response.
*/
export async function isGenerating(page) {
return await page.evaluate(`
return requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
return Array.from(document.querySelectorAll('button')).some(b => {
const label = b.getAttribute('aria-label') || '';
return label === 'Stop generating' || label.includes('Thinking');
});
})()
`);
`)), 'chatgpt generation state');
}
/**
* Get visible image URLs from the ChatGPT page (excluding profile/avatar images).
*/
export async function getChatGPTVisibleImageUrls(page) {
return await page.evaluate(`
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
@@ -162,32 +720,78 @@ export async function getChatGPTVisibleImageUrls(page) {
return rect.width > 32 && rect.height > 32;
};
const urls = [];
const seen = new Set();
const normalizeUrl = (value) => {
const raw = String(value || '').trim();
if (!raw || raw === 'none') return '';
if (/^(?:https?:|blob:|data:)/i.test(raw)) return raw;
try {
return new URL(raw, window.location.href).href;
} catch {
return raw;
}
};
const addUrl = (value) => {
const src = normalizeUrl(value);
if (!src || seen.has(src)) return;
seen.add(src);
urls.push(src);
};
const isDecorative = (el, src = '') => {
const alt = (el.getAttribute('alt') || '').toLowerCase();
const cls = String(el.className || '').toLowerCase();
const testId = (el.getAttribute('data-testid') || '').toLowerCase();
const label = (el.getAttribute('aria-label') || '').toLowerCase();
const text = [alt, cls, testId, label, src.toLowerCase()].join(' ');
return /avatar|profile|logo|icon/.test(text);
};
const imgs = Array.from(document.querySelectorAll('img')).filter(img =>
img instanceof HTMLImageElement && isVisible(img)
);
const urls = [];
const seen = new Set();
for (const img of imgs) {
const src = img.currentSrc || img.src || '';
const alt = (img.getAttribute('alt') || '').toLowerCase();
const cls = (img.className || '').toLowerCase();
const width = img.naturalWidth || img.width || 0;
const height = img.naturalHeight || img.height || 0;
if (!src) continue;
if (alt.includes('avatar') || alt.includes('profile') || alt.includes('logo') || alt.includes('icon')) continue;
if (cls.includes('avatar') || cls.includes('profile') || cls.includes('icon')) continue;
if (isDecorative(img, src)) continue;
if (width < 128 && height < 128) continue;
if (seen.has(src)) continue;
addUrl(src);
}
seen.add(src);
urls.push(src);
// ChatGPT occasionally renders generated images as CSS background
// thumbnails instead of plain <img> nodes. Treat visible, large
// background images as generated-image candidates too.
for (const el of Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]'))) {
if (!(el instanceof HTMLElement) || !isVisible(el) || isDecorative(el)) continue;
const rect = el.getBoundingClientRect();
if (rect.width < 128 && rect.height < 128) continue;
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
for (const match of backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)) {
const src = match[2];
if (!src || isDecorative(el, src)) continue;
addUrl(src);
}
}
// Some image experiences render to a canvas. Returning the data URL
// lets the downstream asset exporter save it without needing a DOM
// selector to rediscover the canvas.
for (const canvas of Array.from(document.querySelectorAll('canvas'))) {
if (!(canvas instanceof HTMLCanvasElement) || !isVisible(canvas) || isDecorative(canvas)) continue;
const width = canvas.width || canvas.getBoundingClientRect().width || 0;
const height = canvas.height || canvas.getBoundingClientRect().height || 0;
if (width < 128 && height < 128) continue;
try {
addUrl(canvas.toDataURL('image/png'));
} catch { }
}
return urls;
})()
`);
`)), 'chatgpt visible image url extraction');
}
/**
@@ -205,7 +809,7 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
let currentUrl = '';
if (convUrl && convUrl.includes('/c/')) {
currentUrl = await page.evaluate('window.location.href').catch(() => '');
currentUrl = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
if (currentUrl && !isSameChatGPTConversation(currentUrl, convUrl)) {
await page.goto(convUrl);
await page.wait(3);
@@ -244,7 +848,14 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
export const __test__ = {
COMPOSER_SELECTORS,
SEND_BUTTON_SELECTOR,
SEND_BUTTON_FALLBACK_SELECTORS,
SEND_BUTTON_LABELS,
CLOSE_SIDEBAR_LABELS,
buildComposerLocatorScript,
isSameChatGPTConversation,
parseChatGPTConversationId,
imageMimeFromPath,
};
/**
@@ -252,7 +863,7 @@ export const __test__ = {
*/
export async function getChatGPTImageAssets(page, urls) {
const urlsJson = JSON.stringify(urls);
return await page.evaluate(`
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(async (targetUrls) => {
const blobToDataUrl = (blob) => new Promise((resolve, reject) => {
const reader = new FileReader();
@@ -285,6 +896,26 @@ export async function getChatGPTImageAssets(page, urls) {
if (img) {
width = img.naturalWidth || img.width || 0;
height = img.naturalHeight || img.height || 0;
} else {
const backgroundEl = Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]')).find(el => {
if (!(el instanceof HTMLElement)) return false;
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
return Array.from(backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)).some(match => {
const raw = String(match[2] || '').trim();
if (!raw) return false;
if (raw === targetUrl) return true;
try {
return new URL(raw, window.location.href).href === targetUrl;
} catch {
return false;
}
});
});
if (backgroundEl) {
const rect = backgroundEl.getBoundingClientRect();
width = Math.round(rect.width || 0);
height = Math.round(rect.height || 0);
}
}
try {
@@ -326,5 +957,5 @@ export async function getChatGPTImageAssets(page, urls) {
return results;
})(${urlsJson})
`, urls);
`)), 'chatgpt image asset export');
}
+269 -2
View File
@@ -1,5 +1,18 @@
import { describe, expect, it, vi } from 'vitest';
import { __test__, waitForChatGPTImages } from './utils.js';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { JSDOM } from 'jsdom';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { __test__, getChatGPTImageAssets, getChatGPTVisibleImageUrls, prepareChatGPTImagePaths, sendChatGPTMessage, uploadChatGPTImages, waitForChatGPTImages } from './utils.js';
const tempDirs = [];
afterEach(() => {
vi.restoreAllMocks();
while (tempDirs.length) {
fs.rmSync(tempDirs.pop(), { recursive: true, force: true });
}
});
function createPageMock({ location = '', generating = [], imageUrls = [] } = {}) {
let generatingIndex = 0;
@@ -61,3 +74,257 @@ describe('chatgpt image wait contract', () => {
)).toBe(false);
});
});
describe('chatgpt conversation id parsing', () => {
it('accepts ids and chatgpt conversation URLs', () => {
expect(__test__.parseChatGPTConversationId('abc_123-def')).toBe('abc_123-def');
expect(__test__.parseChatGPTConversationId('https://chatgpt.com/c/abc_123-def?model=gpt-5')).toBe('abc_123-def');
expect(__test__.parseChatGPTConversationId('/c/abc_123-def')).toBe('abc_123-def');
});
it('rejects invalid detail ids', () => {
expect(() => __test__.parseChatGPTConversationId('')).toThrow(/conversation id/);
expect(() => __test__.parseChatGPTConversationId('https://chatgpt.com/')).toThrow(/conversation id/);
});
});
describe('chatgpt send selectors', () => {
it('inlines the composer locator without returning before caller code runs', () => {
const dom = new JSDOM('<!doctype html><div id="prompt-textarea" contenteditable="true"></div>', {
url: 'https://chatgpt.com/',
runScripts: 'outside-only',
});
const composer = dom.window.document.querySelector('#prompt-textarea');
composer.getBoundingClientRect = () => ({ width: 320, height: 48 });
const result = dom.window.eval(`
(() => {
${__test__.buildComposerLocatorScript()}
const composer = findComposer();
return !!composer && composer.getAttribute(markerAttr) === '1';
})()
`);
expect(result).toBe(true);
});
it('keeps locale-independent send-button selector before aria-label fallbacks', async () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
nativeType: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn((script) => {
if (script.includes('findComposer')) return Promise.resolve(true);
if (script.includes('sendBtnFound')) {
expect(script).toContain('data-testid=\\\"send-button\\\"');
return Promise.resolve({ sendBtnFound: true });
}
if (script.includes('if (sendBtn) sendBtn.click')) {
expect(script).toContain('data-testid=\\\"send-button\\\"');
}
return Promise.resolve(undefined);
}),
};
await expect(sendChatGPTMessage(page, 'hello')).resolves.toBe(true);
});
it('uses the composer submit fallback consistently for readiness and click', async () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
nativeType: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn((script) => {
if (script.includes('findComposer')) return Promise.resolve(true);
if (script.includes('sendBtnFound')) {
expect(script).toContain('#composer-submit-button:not([disabled])');
return Promise.resolve({ sendBtnFound: true });
}
if (script.includes('if (sendBtn) sendBtn.click')) {
expect(script).toContain('#composer-submit-button:not([disabled])');
}
return Promise.resolve(undefined);
}),
};
await expect(sendChatGPTMessage(page, 'hello')).resolves.toBe(true);
});
it('keeps zh-CN aria and placeholder fallbacks without replacing English selectors', () => {
expect(__test__.COMPOSER_SELECTORS).toEqual(expect.arrayContaining([
'[aria-label="Chat with ChatGPT"]',
'[aria-label="与 ChatGPT 聊天"]',
'[placeholder="Ask anything"]',
'[placeholder="有问题,尽管问"]',
'[data-testid="prompt-textarea"]',
]));
expect(__test__.SEND_BUTTON_SELECTOR).toBe('button[data-testid="send-button"]:not([disabled])');
expect(__test__.SEND_BUTTON_FALLBACK_SELECTORS).toContain('#composer-submit-button:not([disabled])');
expect(__test__.SEND_BUTTON_LABELS).toEqual(expect.arrayContaining(['Send prompt', 'Send message', 'Send', '发送提示']));
expect(__test__.CLOSE_SIDEBAR_LABELS).toEqual(expect.arrayContaining(['Close sidebar', '关闭边栏']));
});
});
describe('chatgpt generated image detection', () => {
function createDomPage(html, setup = () => {}) {
const dom = new JSDOM(html, {
url: 'https://chatgpt.com/c/demo',
runScripts: 'outside-only',
});
setup(dom.window);
return {
evaluate: vi.fn((script) => Promise.resolve(dom.window.eval(String(script)))),
};
}
it('detects visible CSS background images when ChatGPT does not render a plain img', async () => {
const page = createDomPage(`
<!doctype html>
<main>
<div class="avatar" style="background-image: url('https://chatgpt.com/avatar.png')"></div>
<button data-testid="generated-image" style="background-image: url('/backend-api/generated/foo.webp')"></button>
</main>
`, (window) => {
for (const el of window.document.querySelectorAll('div, button')) {
el.getBoundingClientRect = () => ({ width: 512, height: 512 });
}
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
'https://chatgpt.com/backend-api/generated/foo.webp',
]);
});
it('detects visible generated canvases as data URLs', async () => {
const page = createDomPage('<!doctype html><canvas width="512" height="512"></canvas>', (window) => {
const canvas = window.document.querySelector('canvas');
canvas.getBoundingClientRect = () => ({ width: 512, height: 512 });
canvas.toDataURL = () => 'data:image/png;base64,ZmFrZQ==';
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
'data:image/png;base64,ZmFrZQ==',
]);
});
it('exports assets for generated CSS background images', async () => {
const imageUrl = 'https://chatgpt.com/backend-api/generated/foo.webp';
const page = createDomPage(`
<!doctype html>
<button style="background-image: url('/backend-api/generated/foo.webp')"></button>
`, (window) => {
const button = window.document.querySelector('button');
button.getBoundingClientRect = () => ({ width: 512, height: 512 });
window.fetch = vi.fn().mockResolvedValue({
ok: true,
blob: async () => new window.Blob(['fake-image'], { type: 'image/webp' }),
});
});
await expect(getChatGPTImageAssets(page, [imageUrl])).resolves.toEqual([
expect.objectContaining({
url: imageUrl,
mimeType: 'image/webp',
width: 512,
height: 512,
}),
]);
});
});
describe('chatgpt image upload helper', () => {
it('validates local images without a browser page', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'cat.png');
fs.writeFileSync(filePath, 'fake-png');
await expect(prepareChatGPTImagePaths([filePath])).resolves.toEqual({ ok: true, paths: [filePath] });
await expect(prepareChatGPTImagePaths([path.join(dir, 'missing.png')])).resolves.toMatchObject({
ok: false,
reason: expect.stringContaining('Image not found'),
});
});
it('prefers Browser Bridge file input upload and waits for a preview', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'cat.png');
fs.writeFileSync(filePath, 'fake-png');
const page = {
setFileInput: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(true),
};
const result = await uploadChatGPTImages(page, [filePath]);
expect(result).toEqual({ ok: true, files: [filePath] });
expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[type="file"]');
});
it('rejects missing files before touching the page', async () => {
const page = {
setFileInput: vi.fn(),
wait: vi.fn(),
evaluate: vi.fn(),
};
const result = await uploadChatGPTImages(page, ['/no/such/cat.png']);
expect(result.ok).toBe(false);
expect(result.reason).toContain('Image not found');
expect(page.setFileInput).not.toHaveBeenCalled();
});
it('rejects non-image extensions', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'report.pdf');
fs.writeFileSync(filePath, 'fake');
const page = {
setFileInput: vi.fn(),
wait: vi.fn(),
evaluate: vi.fn(),
};
const result = await uploadChatGPTImages(page, [filePath]);
expect(result.ok).toBe(false);
expect(result.reason).toContain('Unsupported image type');
expect(page.setFileInput).not.toHaveBeenCalled();
});
it('passes a React-compatible change event in fallback upload', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'cat.png');
fs.writeFileSync(filePath, 'fake-png');
const page = {
setFileInput: vi.fn().mockRejectedValue(new Error('No element found')),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn((script) => {
if (String(script).includes('new DataTransfer()')) {
return Promise.resolve({ ok: true });
}
return Promise.resolve(true);
}),
};
const result = await uploadChatGPTImages(page, [filePath]);
expect(result).toEqual({ ok: true, files: [filePath] });
const fallbackScript = page.evaluate.mock.calls
.map(([script]) => String(script))
.find(script => script.includes('new DataTransfer()'));
expect(fallbackScript).toContain('preventDefault()');
expect(fallbackScript).toContain('stopPropagation()');
});
it('exposes image MIME inference for fallback upload', () => {
expect(__test__.imageMimeFromPath('/tmp/a.png')).toBe('image/png');
expect(__test__.imageMimeFromPath('/tmp/a.webp')).toBe('image/webp');
expect(__test__.imageMimeFromPath('/tmp/a.jpg')).toBe('image/jpeg');
});
});
+23 -7
View File
@@ -1,7 +1,8 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
CLAUDE_DOMAIN, CLAUDE_URL, ensureOnClaude, selectModel, setAdaptiveThinking,
CLAUDE_DOMAIN, CLAUDE_URL, COMPOSER_SELECTOR, MESSAGE_SELECTOR,
ensureOnClaude, selectModel, setAdaptiveThinking,
sendMessage, sendWithFile, getBubbleCount, waitForResponse, parseBoolFlag, withRetry,
ensureClaudeComposer, requireNonEmptyPrompt, requirePositiveInt,
} from './utils.js';
@@ -14,6 +15,7 @@ export const askCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
@@ -37,7 +39,11 @@ export const askCommand = cli({
if (parseBoolFlag(kwargs.new)) {
await page.goto(CLAUDE_URL);
await page.wait(3);
try {
await page.wait({ selector: COMPOSER_SELECTOR, timeout: 8 });
} catch {
// Composer didn't mount; ensureClaudeComposer below surfaces a typed error.
}
} else {
const navigated = await ensureOnClaude(page);
if (navigated) {
@@ -47,11 +53,18 @@ export const askCommand = cli({
var link = document.querySelector('a[href*="/chat/"]');
if (link) link.click();
})()`);
await page.wait(2);
// Wait for the resumed conversation to render messages, or
// fall through if the link click had no effect (no recents).
try {
await page.wait({ selector: MESSAGE_SELECTOR, timeout: 5 });
} catch {
// No prior conversation; ensureClaudeComposer still requires composer below.
}
}
}
await page.wait(2);
// ensureClaudeComposer reads composer presence directly via getPageState,
// so the previous standalone 2 s settle is redundant.
await withRetry(() => ensureClaudeComposer(page, 'Claude ask requires a visible composer on the current page.'));
// Model selector is only available on the new-chat page, not inside
@@ -79,14 +92,16 @@ export const askCommand = cli({
}
throw new CommandExecutionError(`Could not switch to ${wantModel} model`);
}
if (modelResult?.toggled) await page.wait(0.5);
// Post-toggle settle dropped — the next CDP eval (setAdaptiveThinking) gives
// React enough time to flush aria-checked updates between rountrips.
}
const thinkResult = await withRetry(() => setAdaptiveThinking(page, wantThink));
if (!thinkResult?.ok && wantThink) {
throw new CommandExecutionError('Could not enable Adaptive thinking');
}
if (thinkResult?.toggled) await page.wait(0.5);
// Post-toggle settle dropped — the next CDP eval (sendMessage / sendWithFile)
// gives React enough time to flush aria-checked updates.
if (kwargs.file) {
const baseline = await withRetry(() => getBubbleCount(page));
@@ -99,7 +114,8 @@ export const askCommand = cli({
// SPA navigates after send; "Promise was collected" means send succeeded
if (!String(err?.message || err).includes('Promise was collected')) throw err;
}
await page.wait(3);
// Pre-waitForResponse settle dropped — waitForResponse's first 3 s polling
// tick covers the same window without an unconditional sleep.
const result = await waitForResponse(page, baseline, prompt, timeoutMs);
if (!result) {
throw new EmptyResultError(
+10 -2
View File
@@ -1,6 +1,6 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, getVisibleMessages, ensureClaudeLogin, requireConversationId } from './utils.js';
import { CLAUDE_DOMAIN, MESSAGE_SELECTOR, getVisibleMessages, ensureClaudeLogin, requireConversationId } from './utils.js';
export const detailCommand = cli({
site: 'claude',
@@ -10,6 +10,7 @@ export const detailCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', positional: true, required: true, help: 'Conversation ID (UUID from /chat/<id>)' },
@@ -20,7 +21,14 @@ export const detailCommand = cli({
const id = requireConversationId(kwargs.id);
await page.goto(`https://claude.ai/chat/${id}`);
await page.wait(4);
// Wait for the first assistant message bubble to render instead of a
// fixed 4 s sleep. Swallow the timeout so empty conversations and
// login redirects fall through to ensureClaudeLogin / EmptyResultError.
try {
await page.wait({ selector: MESSAGE_SELECTOR, timeout: 10 });
} catch {
// Empty conversation, missing access, or login redirect — handled below.
}
await ensureClaudeLogin(page, 'Claude detail requires a logged-in Claude session.');
const messages = await getVisibleMessages(page);
+1
View File
@@ -10,6 +10,7 @@ export const historyCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Max conversations to show' },
+9 -2
View File
@@ -1,5 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CLAUDE_DOMAIN, CLAUDE_URL, ensureClaudeComposer } from './utils.js';
import { CLAUDE_DOMAIN, CLAUDE_URL, COMPOSER_SELECTOR, ensureClaudeComposer } from './utils.js';
export const newCommand = cli({
site: 'claude',
@@ -9,13 +9,20 @@ export const newCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status'],
func: async (page) => {
await page.goto(CLAUDE_URL);
await page.wait(2);
// Wait for the composer to mount instead of a fixed 2 s sleep. If it
// never mounts, swallow and let ensureClaudeComposer surface a typed error.
try {
await page.wait({ selector: COMPOSER_SELECTOR, timeout: 8 });
} catch {
// Login or error page — ensureClaudeComposer below throws AuthRequiredError / CommandExecutionError.
}
await ensureClaudeComposer(page, 'Claude new requires a logged-in Claude session with a visible composer.');
return [{ Status: 'New chat started' }];
},
+3 -1
View File
@@ -10,13 +10,15 @@ export const readCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Index', 'Role', 'Text'],
func: async (page) => {
// ensureOnClaude now waits for the composer selector; the previous post-nav
// 3 s settle is covered by that event-based wait.
await ensureOnClaude(page);
await page.wait(3);
await ensureClaudeLogin(page, 'Claude read requires a logged-in Claude session.');
const messages = await getVisibleMessages(page);
if (messages.length > 0) return messages;
+9 -3
View File
@@ -1,6 +1,6 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { CLAUDE_DOMAIN, CLAUDE_URL, ensureOnClaude, sendMessage, parseBoolFlag, withRetry, ensureClaudeComposer, requireNonEmptyPrompt } from './utils.js';
import { CLAUDE_DOMAIN, CLAUDE_URL, COMPOSER_SELECTOR, ensureOnClaude, sendMessage, parseBoolFlag, withRetry, ensureClaudeComposer, requireNonEmptyPrompt } from './utils.js';
export const sendCommand = cli({
site: 'claude',
@@ -10,6 +10,7 @@ export const sendCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
@@ -22,10 +23,15 @@ export const sendCommand = cli({
if (parseBoolFlag(kwargs.new)) {
await page.goto(CLAUDE_URL);
await page.wait(3);
try {
await page.wait({ selector: COMPOSER_SELECTOR, timeout: 8 });
} catch {
// Composer didn't mount; ensureClaudeComposer below surfaces a typed error.
}
} else {
// ensureOnClaude now waits for the composer selector; the previous
// post-nav 2 s settle is covered by that event-based wait.
await ensureOnClaude(page);
await page.wait(2);
}
await withRetry(() => ensureClaudeComposer(page, 'Claude send requires a visible composer on the current page.'));
+1
View File
@@ -9,6 +9,7 @@ export const statusCommand = cli({
domain: CLAUDE_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status', 'Login', 'Url'],
+27 -4
View File
@@ -26,7 +26,14 @@ export async function isOnClaude(page) {
export async function ensureOnClaude(page) {
if (await isOnClaude(page)) return false;
await page.goto(CLAUDE_URL);
await page.wait(3);
// Wait for the composer textarea instead of a fixed 3 s sleep. On the login
// page it never mounts; swallow the timeout so callers (read / detail /
// send) can still inspect page state and produce typed errors.
try {
await page.wait({ selector: COMPOSER_SELECTOR, timeout: 8 });
} catch {
// Login or error page — downstream ensureClaudeLogin / ensureClaudeComposer surfaces a typed error.
}
return true;
}
@@ -111,7 +118,13 @@ export async function getVisibleMessages(page) {
export async function getConversationList(page) {
if (!(await isOnClaude(page)) || !(await page.evaluate('window.location.href') || '').includes('/recents')) {
await page.goto('https://claude.ai/recents');
await page.wait(3);
// Recents list mounts <a href="/chat/...">; an empty history is also
// valid (returns []), so swallow the timeout instead of raising.
try {
await page.wait({ selector: 'a[href*="/chat/"]', timeout: 8 });
} catch {
// Empty history or login page — downstream evaluate returns [].
}
}
const items = await page.evaluate(`(() => {
var links = Array.from(document.querySelectorAll('a[href*="/chat/"]'));
@@ -147,7 +160,12 @@ export async function selectModel(page, modelName) {
if (!opened?.ok) return opened;
if (!opened.opened) return opened;
await page.wait(0.6);
// Wait for the dropdown menu items to mount instead of a fixed 0.6 s sleep.
try {
await page.wait({ selector: 'div[role="menuitemradio"]', timeout: 3 });
} catch {
// Dropdown didn't open — next evaluate finds no target and returns { ok: false }.
}
return page.evaluate(`(() => {
var items = Array.from(document.querySelectorAll('div[role="menuitemradio"]'));
@@ -175,7 +193,12 @@ export async function setAdaptiveThinking(page, enabled) {
})()`);
if (!opened?.ok) return { ok: false };
await page.wait(0.6);
// Wait for the dropdown menu items to mount instead of a fixed 0.6 s sleep.
try {
await page.wait({ selector: 'div[role="menuitem"]', timeout: 3 });
} catch {
// Dropdown didn't open — next evaluate finds no target and returns { ok: false }.
}
return page.evaluate(`(() => {
var items = Array.from(document.querySelectorAll('div[role="menuitem"]'));
+21 -13
View File
@@ -1,5 +1,6 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { canonicalizeProductUrl, normalizeProductId } from './utils.js';
import { canonicalizeProductUrl, normalizeProductId, requireProductIdArg } from './utils.js';
function escapeJsString(value) {
return JSON.stringify(value);
}
@@ -105,31 +106,38 @@ cli({
],
columns: ['ok', 'product_id', 'url', 'message'],
func: async (page, kwargs) => {
const rawProductId = kwargs['product-id'] ?? kwargs['product-id'];
const productId = normalizeProductId(rawProductId);
const targetUrl = canonicalizeProductUrl(kwargs.url, productId);
if (!productId && !targetUrl) {
throw new Error('Either --product-id or --url is required');
const rawProductId = kwargs['product-id'];
if (!rawProductId && !kwargs.url) {
throw new ArgumentError('Either --product-id or --url is required');
}
const productId = rawProductId
? requireProductIdArg(rawProductId, 'product-id')
: requireProductIdArg(kwargs.url, '--url');
const targetUrl = canonicalizeProductUrl(kwargs.url, productId);
const finalUrl = targetUrl || canonicalizeProductUrl('', productId);
await page.goto(finalUrl);
const result = await page.evaluate(buildAddToCartEvaluate(productId));
await page.goto(finalUrl).catch((error) => {
throw new CommandExecutionError(`coupang add-to-cart navigation failed: ${error?.message || error}`);
});
const result = await page.evaluate(buildAddToCartEvaluate(productId)).catch((error) => {
throw new CommandExecutionError(`coupang add-to-cart evaluation failed: ${error?.message || error}`);
});
const loginHints = result?.loginHints ?? {};
if (loginHints.hasLoginLink && !loginHints.hasMyCoupang) {
throw new Error('Coupang login required. Please log into Coupang in Chrome and retry.');
throw new AuthRequiredError('coupang.com', 'Please log into Coupang in Chrome and retry.');
}
const actualProductId = normalizeProductId(result?.currentProductId || productId);
if (result?.reason === 'PRODUCT_MISMATCH') {
throw new Error(`Product mismatch: expected ${productId}, got ${actualProductId || 'unknown'}`);
const observed = actualProductId ? `got ${actualProductId}` : 'no product id observed';
throw new CommandExecutionError(`Product mismatch: expected ${productId}, ${observed}`);
}
if (result?.reason === 'OPTION_REQUIRED') {
throw new Error('This product requires option selection and is not supported in v1.');
throw new CommandExecutionError('This product requires option selection and is not supported in v1.');
}
if (result?.reason === 'ADD_TO_CART_BUTTON_NOT_FOUND') {
throw new Error('Could not find an add-to-cart button on the product page.');
throw new CommandExecutionError('Could not find an add-to-cart button on the product page.');
}
if (!result?.ok) {
throw new Error('Failed to confirm add-to-cart success.');
throw new CommandExecutionError('Failed to confirm add-to-cart success.');
}
return [{
ok: true,
+159
View File
@@ -0,0 +1,159 @@
import { describe, expect, it } from 'vitest';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
import './product.js';
import './add-to-cart.js';
import { parseLimitArg, parsePageArg, requireProductIdArg } from './utils.js';
describe('coupang utils — parseLimitArg / parsePageArg (no silent clamp)', () => {
it('parseLimitArg returns fallback for empty / undefined', () => {
expect(parseLimitArg(undefined, 20, 50)).toBe(20);
expect(parseLimitArg(null, 20, 50)).toBe(20);
expect(parseLimitArg('', 20, 50)).toBe(20);
});
it('parseLimitArg accepts integers in range', () => {
expect(parseLimitArg(1, 20, 50)).toBe(1);
expect(parseLimitArg(50, 20, 50)).toBe(50);
expect(parseLimitArg('25', 20, 50)).toBe(25);
});
it('parseLimitArg throws ArgumentError on out-of-range / non-integer (no silent clamp)', () => {
expect(() => parseLimitArg(0, 20, 50)).toThrow(ArgumentError);
expect(() => parseLimitArg(-1, 20, 50)).toThrow(ArgumentError);
expect(() => parseLimitArg(51, 20, 50)).toThrow(ArgumentError);
expect(() => parseLimitArg(999, 20, 50)).toThrow(ArgumentError);
expect(() => parseLimitArg('abc', 20, 50)).toThrow(ArgumentError);
expect(() => parseLimitArg(1.5, 20, 50)).toThrow(ArgumentError);
});
it('parsePageArg returns fallback for empty', () => {
expect(parsePageArg(undefined, 1)).toBe(1);
expect(parsePageArg('', 1)).toBe(1);
});
it('parsePageArg accepts positive integers', () => {
expect(parsePageArg(1, 1)).toBe(1);
expect(parsePageArg('5', 1)).toBe(5);
});
it('parsePageArg throws ArgumentError on non-positive (no silent lift to 1)', () => {
expect(() => parsePageArg(0, 1)).toThrow(ArgumentError);
expect(() => parsePageArg(-1, 1)).toThrow(ArgumentError);
expect(() => parsePageArg('abc', 1)).toThrow(ArgumentError);
});
});
describe('coupang utils — product id validation', () => {
it('extracts numeric ids from ids and URLs', () => {
expect(requireProductIdArg('123456789')).toBe('123456789');
expect(requireProductIdArg('https://www.coupang.com/vp/products/123456789?itemId=1', '--url')).toBe('123456789');
});
it('rejects malformed product ids instead of building fake URLs', () => {
expect(() => requireProductIdArg('abc')).toThrow(ArgumentError);
expect(() => requireProductIdArg('abc 123456789')).toThrow(ArgumentError);
expect(() => requireProductIdArg('https://www.coupang.com/not-a-product', '--url')).toThrow(ArgumentError);
expect(() => requireProductIdArg('https://www.coupang.com/not-a-product/123456789', '--url')).toThrow(ArgumentError);
expect(() => requireProductIdArg('https://notcoupang.com/vp/products/123456789', '--url')).toThrow(ArgumentError);
expect(() => requireProductIdArg('https://example.com/vp/products/123456789', '--url')).toThrow(ArgumentError);
});
});
describe('coupang adapter registry shape', () => {
it('search has product_id column for round-trip into product', () => {
const search = getRegistry().get('coupang/search');
expect(search).toBeDefined();
expect(search.access).toBe('read');
expect(search.columns).toContain('product_id');
// Listing pairs with detail: id-shaped column present.
const idShaped = search.columns.find((c) => /_id$|^id$/.test(c));
expect(idShaped).toBe('product_id');
});
it('product cmd is a registered read adapter that pairs with search', () => {
const product = getRegistry().get('coupang/product');
expect(product).toBeDefined();
expect(product.access).toBe('read');
expect(product.columns).toContain('product_id');
expect(product.columns).toContain('title');
expect(product.columns).toContain('price');
expect(product.columns).toContain('seller');
expect(product.columns).toContain('rating');
});
it('add-to-cart remains write-class', () => {
const cart = getRegistry().get('coupang/add-to-cart');
expect(cart).toBeDefined();
expect(cart.access).toBe('write');
});
});
describe('coupang search — typed errors (no silent fallback)', () => {
it('rejects empty query with ArgumentError', async () => {
const search = getRegistry().get('coupang/search');
// page object is irrelevant — we expect to fail before any browser call.
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(search.func(fakePage, { query: ' ' })).rejects.toThrow(ArgumentError);
await expect(search.func(fakePage, { query: '' })).rejects.toThrow(ArgumentError);
});
it('rejects unsupported --filter with ArgumentError', async () => {
const search = getRegistry().get('coupang/search');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(search.func(fakePage, { query: 'mouse', filter: 'eco' })).rejects.toThrow(ArgumentError);
});
it('rejects out-of-range --limit with ArgumentError (no silent clamp to 50)', async () => {
const search = getRegistry().get('coupang/search');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(search.func(fakePage, { query: 'mouse', limit: 999 })).rejects.toThrow(ArgumentError);
});
it('rejects out-of-range --page with ArgumentError', async () => {
const search = getRegistry().get('coupang/search');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(search.func(fakePage, { query: 'mouse', page: 0 })).rejects.toThrow(ArgumentError);
});
});
describe('coupang product — typed errors', () => {
it('rejects missing --product-id and --url with ArgumentError', async () => {
const product = getRegistry().get('coupang/product');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(product.func(fakePage, {})).rejects.toThrow(ArgumentError);
});
it('rejects malformed product id before navigation', async () => {
const product = getRegistry().get('coupang/product');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(product.func(fakePage, { 'product-id': 'abc' })).rejects.toThrow(ArgumentError);
});
it('wraps browser failures as CommandExecutionError', async () => {
const product = getRegistry().get('coupang/product');
const fakePage = { goto: () => Promise.reject(new Error('browser down')) };
await expect(product.func(fakePage, { 'product-id': '123456789' })).rejects.toThrow(CommandExecutionError);
});
});
describe('coupang add-to-cart — typed errors', () => {
it('rejects missing --product-id and --url with ArgumentError', async () => {
const cart = getRegistry().get('coupang/add-to-cart');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(cart.func(fakePage, {})).rejects.toThrow(ArgumentError);
});
it('rejects malformed product id before navigation', async () => {
const cart = getRegistry().get('coupang/add-to-cart');
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
await expect(cart.func(fakePage, { 'product-id': 'abc' })).rejects.toThrow(ArgumentError);
});
it('wraps browser failures as CommandExecutionError', async () => {
const cart = getRegistry().get('coupang/add-to-cart');
const fakePage = { goto: () => Promise.reject(new Error('browser down')) };
await expect(cart.func(fakePage, { 'product-id': '123456789' })).rejects.toThrow(CommandExecutionError);
});
});
+257
View File
@@ -0,0 +1,257 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { canonicalizeProductUrl, normalizeProductId, requireProductIdArg } from './utils.js';
function escapeJsString(value) {
return JSON.stringify(value);
}
/**
* Build the in-page extractor for a Coupang product detail page.
*
* Tries three sources in order, mirroring search.js's chain:
* 1. JSON-LD <script type="application/ld+json"> (Product schema, most stable)
* 2. window.__INITIAL_STATE__ / __NEXT_DATA__ / similar globals (rich)
* 3. DOM scrape (fallback when bootstrap state is server-side only)
*
* Returns either a partial product object or a structured failure
* `{ loginHints, ok: false, reason }` so the caller can map it to typed errors.
*
* Design note (no-silent-empty): empty strings / null fields here MUST mean
* "upstream did not provide this field" — they should not be conflated with
* "extraction failed". A failed extraction returns ok=false so the caller can
* surface AuthRequiredError or EmptyResultError; partial success returns the
* fields it found and the caller decides whether to treat the partial row as
* usable.
*/
function buildProductDetailEvaluate(expectedProductId) {
return `
(async () => {
const expectedProductId = ${escapeJsString(expectedProductId)};
const normalizeText = (value) => (value == null ? '' : String(value).trim());
const parseNum = (value) => {
const text = normalizeText(value).replace(/[^\\d.]/g, '');
if (!text) return null;
const num = Number(text);
return Number.isFinite(num) ? num : null;
};
const loginHints = {
hasLoginLink: Boolean(document.querySelector('a[href*="login"], a[title*="로그인"]')),
hasMyCoupang: /마이쿠팡/.test(document.body.innerText || ''),
};
const pathMatch = location.pathname.match(/\\/vp\\/products\\/(\\d+)/);
const currentProductId = pathMatch?.[1] || '';
if (expectedProductId && currentProductId && expectedProductId !== currentProductId) {
return { ok: false, reason: 'PRODUCT_MISMATCH', currentProductId, loginHints };
}
// ── Source 1: JSON-LD Product schema ─────────────────────────────
const fromJsonLd = (() => {
const scripts = Array.from(document.querySelectorAll('script[type="application/ld+json"]'));
for (const script of scripts) {
try {
const docs = JSON.parse(script.textContent || 'null');
const items = Array.isArray(docs) ? docs : [docs];
for (const doc of items) {
if (!doc || typeof doc !== 'object') continue;
const t = doc['@type'];
const types = Array.isArray(t) ? t : [t];
if (!types.some((x) => /Product/i.test(String(x || '')))) continue;
const offers = Array.isArray(doc.offers) ? doc.offers[0] : doc.offers;
return {
title: normalizeText(doc.name),
brand: normalizeText(doc.brand?.name || doc.brand),
image_url: normalizeText(Array.isArray(doc.image) ? doc.image[0] : doc.image),
price: parseNum(offers?.price),
rating: parseNum(doc.aggregateRating?.ratingValue),
review_count: parseNum(doc.aggregateRating?.reviewCount),
seller: normalizeText(offers?.seller?.name),
};
}
} catch { /* malformed ld+json — skip and try next source */ }
}
return null;
})();
// ── Source 2: bootstrap globals (deeply nested vendorItem etc.) ──
const fromBootstrap = (() => {
const collect = (root) => {
if (!root || typeof root !== 'object') return null;
const queue = [root];
let depth = 0;
while (queue.length && depth < 5000) {
const node = queue.shift();
depth++;
if (!node || typeof node !== 'object') continue;
// A product-like leaf usually has both productId and salePrice / finalPrice / itemName.
const idCandidate = node.productId || node.product_id || node.id;
const titleCandidate = node.itemName || node.productName || node.name;
const priceCandidate = node.salePrice ?? node.finalPrice ?? node.sellingPrice ?? node.price;
if (idCandidate && titleCandidate && priceCandidate != null && /\\d{6,}/.test(String(idCandidate))) {
return {
product_id: String(idCandidate),
title: normalizeText(titleCandidate),
price: parseNum(priceCandidate),
original_price: parseNum(node.originalPrice ?? node.basePrice ?? node.listPrice),
discount_rate: parseNum(node.discountRate ?? node.discountPercent),
rating: parseNum(node.ratingAverage ?? node.rating ?? node.reviewRating),
review_count: parseNum(node.reviewCount ?? node.reviewsCount ?? node.ratingCount),
seller: normalizeText(node.vendorName ?? node.sellerName ?? node.merchantName),
brand: normalizeText(node.brandName ?? node.brand),
rocket: normalizeText(node.rocketType ?? node.deliveryBadgeType),
delivery_promise: normalizeText(node.deliveryPromise ?? node.arrivalText),
};
}
for (const value of Object.values(node)) {
if (value && typeof value === 'object') queue.push(value);
}
}
return null;
};
const candidates = [
window.__INITIAL_STATE__,
window.__NEXT_DATA__,
window.__APOLLO_STATE__,
window.__PRELOADED_STATE__,
];
for (const c of candidates) {
const found = collect(c);
if (found) return found;
}
return null;
})();
// ── Source 3: DOM fallback ───────────────────────────────────────
const fromDom = (() => {
const titleNode = document.querySelector(
'.prod-buy-header__title, h1.prod-buy-header__title, h1[class*="prod-buy-header"], h2.prod-buy-header__title, h1[class*="ProductName"], h1[class*="product-name"]'
);
const priceNode = document.querySelector(
'.total-price strong, .prod-sale-price strong, [class*="finalPrice"], [class*="sellingPrice"], [class*="price-value"]'
);
const originalPriceNode = document.querySelector(
'.origin-price, .base-price, del[class*="origin"], del[class*="base"], [class*="strike"], [class*="origin-price"]'
);
const discountNode = document.querySelector(
'.discount-percentage, [class*="discount"][class*="percent"], [class*="discountRate"]'
);
const ratingNode = document.querySelector(
'.rating-star-num, [class*="ratingStar"], [class*="rating-star"], [class*="rating-num"], [class*="ProductRating"]'
);
const reviewCountNode = document.querySelector(
'.count, .rating-total-count, [class*="reviewCount"], [class*="review-count"]'
);
const sellerNode = document.querySelector(
'.prod-sale-vendor-name, [class*="vendor-name"], [class*="vendorName"], [class*="sellerName"]'
);
const imageNode = document.querySelector(
'.prod-image__detail, [class*="prod-image"] img, [class*="ProductImage"] img'
);
return {
title: normalizeText(titleNode?.textContent),
price: parseNum(priceNode?.textContent),
original_price: parseNum(originalPriceNode?.textContent),
discount_rate: parseNum(discountNode?.textContent),
rating: parseNum(ratingNode?.getAttribute?.('aria-label') || ratingNode?.textContent),
review_count: parseNum(reviewCountNode?.textContent),
seller: normalizeText(sellerNode?.textContent),
image_url: normalizeText(imageNode?.getAttribute?.('src') || imageNode?.getAttribute?.('data-src')),
};
})();
// Merge with priority: bootstrap > jsonld > dom (bootstrap is freshest /
// closest to the API; jsonld is well-typed; dom is last-resort).
const merge = (a, b) => {
if (!a) return b;
if (!b) return a;
const out = { ...a };
for (const [k, v] of Object.entries(b)) {
if (out[k] == null || out[k] === '') out[k] = v;
}
return out;
};
const merged = merge(merge(fromBootstrap, fromJsonLd), fromDom);
const hasAnyField = merged && (merged.title || merged.price != null);
if (!hasAnyField) {
return { ok: false, reason: 'NO_DATA_EXTRACTED', currentProductId, loginHints };
}
return {
ok: true,
currentProductId,
loginHints,
data: merged,
};
})()
`;
}
cli({
site: 'coupang',
name: 'product',
access: 'read',
description: 'Read full product detail (price, rating, seller, delivery) for a Coupang product',
domain: 'www.coupang.com',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'product-id', positional: true, required: false, help: 'Coupang product ID (digits only)' },
{ name: 'url', required: false, help: 'Canonical Coupang product URL (alternative to --product-id)' },
],
columns: [
'product_id', 'title', 'price', 'original_price', 'discount_rate',
'rating', 'review_count', 'seller', 'brand', 'rocket',
'delivery_promise', 'image_url', 'url',
],
func: async (page, kwargs) => {
const rawProductId = kwargs['product-id'];
if (!rawProductId && !kwargs.url) {
throw new ArgumentError('Either --product-id or --url is required');
}
const productId = rawProductId
? requireProductIdArg(rawProductId, 'product-id')
: requireProductIdArg(kwargs.url, '--url');
const targetUrl = canonicalizeProductUrl(kwargs.url, productId);
const finalUrl = targetUrl || canonicalizeProductUrl('', productId);
await page.goto(finalUrl).catch((error) => {
throw new CommandExecutionError(`coupang product navigation failed: ${error?.message || error}`);
});
await page.wait(2).catch((error) => {
throw new CommandExecutionError(`coupang product wait failed: ${error?.message || error}`);
});
const result = await page.evaluate(buildProductDetailEvaluate(productId)).catch((error) => {
throw new CommandExecutionError(`coupang product extraction failed: ${error?.message || error}`);
});
const loginHints = result?.loginHints ?? {};
if (loginHints.hasLoginLink && !loginHints.hasMyCoupang) {
throw new AuthRequiredError('coupang.com', 'Please log into Coupang in Chrome and retry.');
}
if (result?.reason === 'PRODUCT_MISMATCH') {
const actualProductId = normalizeProductId(result?.currentProductId || '');
const observed = actualProductId ? `got ${actualProductId}` : 'no product id observed';
throw new EmptyResultError('coupang product', `Product page redirected: expected ${productId}, ${observed} (item may be sold out or unavailable in your region)`);
}
if (!result?.ok || !result?.data) {
throw new EmptyResultError('coupang product', `No product data extracted from ${finalUrl}. The page may have failed to render or this product is restricted.`);
}
const actualProductId = normalizeProductId(result?.currentProductId || result.data.product_id || productId);
const data = result.data;
return [{
product_id: actualProductId,
title: data.title || null,
price: data.price ?? null,
original_price: data.original_price ?? null,
discount_rate: data.discount_rate ?? null,
rating: data.rating ?? null,
review_count: data.review_count ?? null,
seller: data.seller || null,
brand: data.brand || null,
rocket: data.rocket || null,
delivery_promise: data.delivery_promise || null,
image_url: data.image_url || null,
url: canonicalizeProductUrl('', actualProductId) || finalUrl,
}];
},
});
+38 -16
View File
@@ -1,5 +1,6 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { mergeSearchItems, normalizeSearchItem, sanitizeSearchItems } from './utils.js';
import { mergeSearchItems, normalizeSearchItem, parseLimitArg, parsePageArg, sanitizeSearchItems } from './utils.js';
function escapeJsString(value) {
return JSON.stringify(value);
}
@@ -410,32 +411,50 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Max results (max 50)' },
{ name: 'filter', required: false, help: 'Optional search filter (currently supports: rocket)' },
],
columns: ['rank', 'title', 'price', 'unit_price', 'rating', 'review_count', 'rocket', 'delivery_type', 'delivery_promise', 'url'],
columns: ['rank', 'product_id', 'title', 'price', 'unit_price', 'rating', 'review_count', 'rocket', 'delivery_type', 'delivery_promise', 'url'],
func: async (page, kwargs) => {
const query = String(kwargs.query || '').trim();
const pageNumber = Math.max(Number(kwargs.page || 1), 1);
const limit = Math.min(Math.max(Number(kwargs.limit || 20), 1), 50);
if (!query) {
throw new ArgumentError('query cannot be empty');
}
const pageNumber = parsePageArg(kwargs.page, 1);
const limit = parseLimitArg(kwargs.limit, 20, 50);
const filter = String(kwargs.filter || '').trim().toLowerCase();
if (!query)
throw new Error('Query is required');
if (filter && filter !== 'rocket') {
throw new ArgumentError(`Unsupported --filter "${filter}" (supported: rocket)`);
}
const initialPage = filter ? 1 : pageNumber;
const url = `https://www.coupang.com/np/search?q=${encodeURIComponent(query)}&channel=user&page=${initialPage}`;
await page.goto(url);
await page.goto(url).catch((error) => {
throw new CommandExecutionError(`coupang search navigation failed: ${error?.message || error}`);
});
if (filter) {
const filterResult = await page.evaluate(buildApplyFilterEvaluate(filter));
const filterResult = await page.evaluate(buildApplyFilterEvaluate(filter)).catch((error) => {
throw new CommandExecutionError(`coupang search filter evaluation failed: ${error?.message || error}`);
});
if (!filterResult?.ok) {
throw new Error(`Unsupported or unavailable filter: ${filter}`);
throw new EmptyResultError('coupang search', `Filter "${filter}" was not available on the current page; try without --filter or wait for Coupang to render the filter bar.`);
}
await page.wait(3);
await page.wait(3).catch((error) => {
throw new CommandExecutionError(`coupang search wait failed: ${error?.message || error}`);
});
if (pageNumber > 1) {
const locationInfo = await page.evaluate(buildCurrentLocationEvaluate());
const locationInfo = await page.evaluate(buildCurrentLocationEvaluate()).catch((error) => {
throw new CommandExecutionError(`coupang search location evaluation failed: ${error?.message || error}`);
});
const filteredUrl = new URL(locationInfo?.href || url);
filteredUrl.searchParams.set('page', String(pageNumber));
await page.goto(filteredUrl.toString());
await page.goto(filteredUrl.toString()).catch((error) => {
throw new CommandExecutionError(`coupang search filtered navigation failed: ${error?.message || error}`);
});
}
}
await page.autoScroll({ times: filter ? 3 : 2, delayMs: 1500 });
const raw = await page.evaluate(buildSearchEvaluate(query, limit, pageNumber));
await page.autoScroll({ times: filter ? 3 : 2, delayMs: 1500 }).catch((error) => {
throw new CommandExecutionError(`coupang search scroll failed: ${error?.message || error}`);
});
const raw = await page.evaluate(buildSearchEvaluate(query, limit, pageNumber)).catch((error) => {
throw new CommandExecutionError(`coupang search extraction failed: ${error?.message || error}`);
});
const loginHints = raw?.loginHints ?? {};
const items = Array.isArray(raw?.items) ? raw.items : [];
const domItems = Array.isArray(raw?.domItems) ? raw.domItems : [];
@@ -444,8 +463,11 @@ cli({
const normalized = filter
? sanitizeSearchItems(normalizedDom, limit)
: mergeSearchItems(normalizedBase, normalizedDom, limit);
if (!normalized.length && loginHints.hasLoginLink && !loginHints.hasMyCoupang) {
throw new Error('Coupang login required. Please log into Coupang in Chrome and retry.');
if (!normalized.length) {
if (loginHints.hasLoginLink && !loginHints.hasMyCoupang) {
throw new AuthRequiredError('coupang.com', 'Please log into Coupang in Chrome and retry.');
}
throw new EmptyResultError('coupang search', `No products matched "${query}". Try a more specific keyword or remove --filter.`);
}
return normalized;
},
+55 -1
View File
@@ -1,3 +1,36 @@
import { ArgumentError } from '@jackwener/opencli/errors';
/**
* Parse a positive integer arg (--limit / --page / --review-page).
*
* Throws ArgumentError on out-of-range / non-integer values rather than
* silently clamping. We prefer typed-fail-fast over silent clamping for the
* same reason as feedback_typed_fail_fast_for_adapters: callers cannot tell
* that their value was rewritten and end up confused why "limit=999" returned
* 50 rows.
*/
export function parseLimitArg(raw, fallback, max) {
if (raw === undefined || raw === null || raw === '') {
return fallback;
}
const num = Number(raw);
if (!Number.isInteger(num) || num < 1 || num > max) {
throw new ArgumentError(`--limit must be an integer between 1 and ${max} (got ${raw})`);
}
return num;
}
export function parsePageArg(raw, fallback) {
if (raw === undefined || raw === null || raw === '') {
return fallback;
}
const num = Number(raw);
if (!Number.isInteger(num) || num < 1) {
throw new ArgumentError(`--page must be a positive integer (got ${raw})`);
}
return num;
}
function itemKey(item) {
return item.url || item.product_id || `${item.title}:${item.price ?? ''}`;
}
@@ -61,7 +94,28 @@ export function normalizeProductId(raw) {
if (!text)
return '';
const match = text.match(/\/vp\/products\/(\d+)/) || text.match(/\b(\d{6,})\b/);
return match?.[1] ?? text;
return match?.[1] ?? '';
}
export function requireProductIdArg(raw, label = '--product-id') {
const text = asString(raw);
if (label === '--url') {
try {
const url = new URL(text.startsWith('http') ? text : `https://www.coupang.com${text}`);
const match = url.pathname.match(/^\/vp\/products\/(\d{6,})(?:\/|$)/);
const isCoupangHost = url.hostname === 'coupang.com' || url.hostname.endsWith('.coupang.com');
if (isCoupangHost && match) {
return match[1];
}
}
catch {
// Fall through to the typed validation error below.
}
throw new ArgumentError(`${label} must be a Coupang product URL containing /vp/products/<id>`);
}
if (!/^\d{6,}$/.test(text)) {
throw new ArgumentError(`${label} must be a numeric Coupang product ID`);
}
return text;
}
export function canonicalizeProductUrl(rawUrl, productId) {
const raw = asString(rawUrl);
+486 -1
View File
@@ -1,8 +1,39 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { JSDOM } from 'jsdom';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
import './hotel-suggest.js';
import { buildUrl, mapSuggestRow, parseLimit, pickCoords } from './utils.js';
import './hotel-search.js';
import './flight.js';
import { __test__ as hotelSearchTest } from './hotel-search.js';
import {
buildFlightExtractJs,
buildScrollUntilJs,
buildUrl,
mapHotelRow,
mapSuggestRow,
parseCityId,
parseIataCode,
parseIsoDate,
parseLimit,
pickCoords,
pickHotelMapCoords,
} from './utils.js';
function createPageMock(evaluateResults) {
const evaluate = vi.fn();
for (const result of evaluateResults) {
evaluate.mockResolvedValueOnce(result);
}
return {
goto: vi.fn().mockResolvedValue(undefined),
evaluate,
wait: vi.fn().mockResolvedValue(undefined),
scroll: vi.fn().mockResolvedValue(undefined),
autoScroll: vi.fn().mockResolvedValue(undefined),
getCookies: vi.fn().mockResolvedValue([]),
};
}
function ok(payload) {
return new Response(JSON.stringify(payload), { status: 200 });
@@ -232,3 +263,457 @@ describe('ctrip hotel-suggest command (registry-level)', () => {
await expect(cmd.func({ query: 'zzz', limit: 5 })).rejects.toThrow('ctrip hotel-suggest returned no data');
});
});
describe('ctrip parseIsoDate', () => {
it('accepts well-formed dates', () => {
expect(parseIsoDate('checkin', '2026-06-15')).toBe('2026-06-15');
expect(parseIsoDate('date', '2030-12-31')).toBe('2030-12-31');
});
it('rejects missing/blank with required-arg message', () => {
expect(() => parseIsoDate('checkin', '')).toThrow(/--checkin is required/);
expect(() => parseIsoDate('date', undefined)).toThrow(/--date is required/);
});
it('rejects malformed strings', () => {
expect(() => parseIsoDate('checkin', '2026/06/15')).toThrow(/must be YYYY-MM-DD/);
expect(() => parseIsoDate('checkin', 'tomorrow')).toThrow(/must be YYYY-MM-DD/);
});
it('rejects out-of-range month/day before Date math', () => {
expect(() => parseIsoDate('checkin', '2026-13-01')).toThrow(/invalid month\/day/);
expect(() => parseIsoDate('checkin', '2026-06-32')).toThrow(/invalid month\/day/);
});
it('rejects impossible calendar dates (Feb 30) via UTC cross-check', () => {
expect(() => parseIsoDate('checkin', '2026-02-30')).toThrow(/not a real calendar date/);
expect(() => parseIsoDate('checkin', '2025-02-29')).toThrow(/not a real calendar date/); // 2025 not leap
});
});
describe('ctrip parseIataCode', () => {
it('uppercases and accepts 3-letter codes', () => {
expect(parseIataCode('from', 'pek')).toBe('PEK');
expect(parseIataCode('from', 'BJS')).toBe('BJS');
expect(parseIataCode('to', ' sha ')).toBe('SHA');
});
it('rejects non-3-letter / mixed inputs', () => {
expect(() => parseIataCode('from', 'PE')).toThrow(/3-letter IATA/);
expect(() => parseIataCode('from', 'PEKK')).toThrow(/3-letter IATA/);
expect(() => parseIataCode('from', '123')).toThrow(/3-letter IATA/);
expect(() => parseIataCode('from', '')).toThrow(/required/);
});
});
describe('ctrip parseCityId', () => {
it('accepts positive integer city IDs (numeric and string)', () => {
expect(parseCityId(2)).toBe(2);
expect(parseCityId('1')).toBe(1);
expect(parseCityId('12345')).toBe(12345);
});
it('rejects zero / negative / non-integer / empty', () => {
expect(() => parseCityId(0)).toThrow(/positive integer/);
expect(() => parseCityId(-1)).toThrow(/positive integer/);
expect(() => parseCityId(2.5)).toThrow(/positive integer/);
expect(() => parseCityId('shanghai')).toThrow(/positive integer/);
expect(() => parseCityId('')).toThrow(/--city is required/);
});
});
describe('ctrip pickHotelMapCoords', () => {
it('prefers WGS84 (coordinateType=1) when multiple available', () => {
const coords = [
{ coordinateType: 3, latitude: '31.25', longitude: '121.51' },
{ coordinateType: 1, latitude: '31.23', longitude: '121.47' },
{ coordinateType: 2, latitude: '31.24', longitude: '121.49' },
];
expect(pickHotelMapCoords(coords)).toEqual({ lat: 31.23, lon: 121.47 });
});
it('falls through to GCJ02 then BD09 if WGS84 missing', () => {
const onlyBD09 = [{ coordinateType: 3, latitude: '31.25', longitude: '121.51' }];
expect(pickHotelMapCoords(onlyBD09)).toEqual({ lat: 31.25, lon: 121.51 });
});
it('returns null/null on empty / non-array / all-zero coords', () => {
expect(pickHotelMapCoords([])).toEqual({ lat: null, lon: null });
expect(pickHotelMapCoords(null)).toEqual({ lat: null, lon: null });
expect(pickHotelMapCoords([{ coordinateType: 1, latitude: '0', longitude: '0' }])).toEqual({ lat: null, lon: null });
});
});
describe('ctrip mapHotelRow', () => {
const HOTEL_FIXTURE = {
hotelInfo: {
summary: { hotelId: '106876528' },
nameInfo: { name: '上海外滩滨江珍宝酒店', enName: 'Shanghai Bund Riverside Treasury Hotel' },
hotelStar: { star: 4 },
commentInfo: { commentScore: '4.7', commentDescription: '超棒', commenterNumber: '13,966条点评' },
positionInfo: {
cityName: '上海',
positionDesc: '北外滩地区 · 近北外滩来福士',
address: '东大名路988号',
mapCoordinate: [{ coordinateType: 3, latitude: '31.25693033446487', longitude: '121.51336547497098' }],
},
},
roomInfo: [{ priceInfo: { price: 548, currency: 'RMB', displayPrice: '¥548' } }],
};
it('projects every declared column key (no silent drop)', () => {
const row = mapHotelRow(HOTEL_FIXTURE, 0);
expect(row).toEqual({
rank: 1,
hotelId: '106876528',
name: '上海外滩滨江珍宝酒店',
enName: 'Shanghai Bund Riverside Treasury Hotel',
star: 4,
score: 4.7,
scoreLabel: '超棒',
reviewCount: 13966,
cityName: '上海',
district: '北外滩地区 · 近北外滩来福士',
address: '东大名路988号',
lat: 31.25693033446487,
lon: 121.51336547497098,
price: 548,
currency: 'RMB',
url: 'https://hotels.ctrip.com/hotels/detail/?hotelid=106876528',
});
});
it('returns null (not 0 / "") for missing optional fields', () => {
const sparse = { hotelInfo: { summary: { hotelId: '999' }, nameInfo: { name: 'X' } }, roomInfo: [] };
const row = mapHotelRow(sparse, 4);
expect(row.rank).toBe(5);
expect(row.star).toBeNull();
expect(row.score).toBeNull();
expect(row.reviewCount).toBeNull();
expect(row.price).toBeNull();
expect(row.currency).toBeNull();
expect(row.lat).toBeNull();
expect(row.lon).toBeNull();
expect(row.address).toBeNull();
});
it('parses reviewCount from "13,966条点评" / "999 reviews" by stripping non-digits', () => {
const a = mapHotelRow({ hotelInfo: { summary: { hotelId: '1' }, nameInfo: { name: 'A' }, commentInfo: { commenterNumber: '13,966条点评' } }, roomInfo: [] }, 0);
expect(a.reviewCount).toBe(13966);
const b = mapHotelRow({ hotelInfo: { summary: { hotelId: '2' }, nameInfo: { name: 'B' }, commentInfo: { commenterNumber: '999 reviews' } }, roomInfo: [] }, 0);
expect(b.reviewCount).toBe(999);
});
});
describe('ctrip hotel-search command (registry-level)', () => {
const cmd = getRegistry().get('ctrip/hotel-search');
const SHANGHAI_HOTEL = {
hotelInfo: {
summary: { hotelId: '106876528' },
nameInfo: { name: '上海外滩滨江珍宝酒店' },
hotelStar: { star: 4 },
commentInfo: { commentScore: '4.7', commentDescription: '超棒', commenterNumber: '13,966条点评' },
positionInfo: { cityName: '上海', address: '东大名路988号', mapCoordinate: [{ coordinateType: 1, latitude: '31.25', longitude: '121.51' }] },
},
roomInfo: [{ priceInfo: { price: 548, currency: 'RMB' } }],
};
it('declares Strategy.COOKIE + browser:true + navigateBefore:false + access:read', () => {
expect(cmd.access).toBe('read');
expect(cmd.browser).toBe(true);
expect(String(cmd.strategy)).toContain('cookie');
expect(cmd.navigateBefore).toBe(false);
expect(cmd.domain).toBe('hotels.ctrip.com');
});
it('rejects invalid city / date / limit before browser navigation', async () => {
const page = createPageMock([]);
await expect(cmd.func(page, { city: 'shanghai', checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--city') });
await expect(cmd.func(page, { city: 2, checkin: 'tomorrow', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--checkin') });
await expect(cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 0 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--limit') });
expect(page.goto).not.toHaveBeenCalled();
});
it('rejects checkin >= checkout before navigation', async () => {
const page = createPageMock([]);
await expect(cmd.func(page, { city: 2, checkin: '2026-06-17', checkout: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--checkin must be earlier') });
await expect(cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--checkin must be earlier') });
expect(page.goto).not.toHaveBeenCalled();
});
it('throws AuthRequired when captcha gate is detected', async () => {
const page = createPageMock(['captcha']);
await expect(cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toThrow('Ctrip is asking for a captcha');
// No extract call when captcha caught early
expect(page.evaluate).toHaveBeenCalledTimes(1);
});
it('throws EmptyResultError when SSR hotelList is empty', async () => {
const page = createPageMock(['content', []]);
await expect(cmd.func(page, { city: 9999, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'EMPTY_RESULT' });
});
it('waits for an empty SSR hotelList so empty results do not become timeout failures', async () => {
const dom = new JSDOM('<!doctype html><html><body></body></html>', {
url: 'https://hotels.ctrip.com/hotels/list?city=9999',
runScripts: 'outside-only',
});
dom.window.__NEXT_DATA__ = {
props: { pageProps: { initListData: { hotelList: [] } } },
};
await expect(dom.window.Function(`return (${hotelSearchTest.WAIT_FOR_SSR_JS})`)())
.resolves.toBe('content');
});
it('throws CommandExecutionError when SSR state times out or is malformed', async () => {
await expect(cmd.func(createPageMock(['timeout']), { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('did not expose SSR hotel list') });
await expect(cmd.func(createPageMock(['content', { hotelList: [] }]), { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('malformed SSR hotel list') });
});
it('maps SSR rows and respects --limit', async () => {
const page = createPageMock([
'content',
[SHANGHAI_HOTEL, { ...SHANGHAI_HOTEL, hotelInfo: { ...SHANGHAI_HOTEL.hotelInfo, summary: { hotelId: '2' } } }],
]);
const rows = await cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 1 });
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({ rank: 1, hotelId: '106876528', name: '上海外滩滨江珍宝酒店', star: 4, price: 548 });
// Every declared column appears on every row
for (const row of rows) {
for (const col of cmd.columns) expect(row).toHaveProperty(col);
}
// Single goto, single URL
expect(page.goto).toHaveBeenCalledTimes(1);
expect(page.goto.mock.calls[0][0]).toContain('city=2');
expect(page.goto.mock.calls[0][0]).toContain('checkin=2026-06-15');
expect(page.goto.mock.calls[0][0]).toContain('checkout=2026-06-17');
});
it('filters out SSR rows missing hotelId or name (no silent partial rows)', async () => {
const incomplete = { hotelInfo: { summary: {}, nameInfo: { name: 'No-id' } }, roomInfo: [] };
const page = createPageMock(['content', [incomplete, SHANGHAI_HOTEL]]);
const rows = await cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 });
expect(rows).toHaveLength(1);
expect(rows[0].hotelId).toBe('106876528');
});
it('throws CommandExecutionError when all SSR rows miss required anchors', async () => {
const incomplete = { hotelInfo: { summary: {}, nameInfo: { name: 'No-id' } }, roomInfo: [] };
const page = createPageMock(['content', [incomplete]]);
await expect(cmd.func(page, { city: 2, checkin: '2026-06-15', checkout: '2026-06-17', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('required hotelId/name anchors') });
});
});
describe('ctrip flight command (registry-level)', () => {
const cmd = getRegistry().get('ctrip/flight');
const FLIGHT_RAW = {
airline: '厦门航空',
flightNo: 'MF8561',
aircraft: '空客321(中)',
departureTime: '07:50',
departureAirport: '大兴国际机场',
arrivalTime: '09:45',
arrivalAirport: '浦东国际机场',
terminal: 'T2',
price: 487,
currency: '¥',
cabin: '经济舱',
};
it('declares Strategy.COOKIE + browser:true + navigateBefore:false + access:read', () => {
expect(cmd.access).toBe('read');
expect(cmd.browser).toBe(true);
expect(String(cmd.strategy)).toContain('cookie');
expect(cmd.navigateBefore).toBe(false);
expect(cmd.domain).toBe('flights.ctrip.com');
});
it('rejects invalid IATA / date / from==to / limit before browser navigation', async () => {
const page = createPageMock([]);
await expect(cmd.func(page, { from: 'PE', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('IATA') });
await expect(cmd.func(page, { from: 'PEK', to: 'PEK', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('must differ') });
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '06/15', limit: 5 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--date') });
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 0 }))
.rejects.toMatchObject({ code: 'ARGUMENT', message: expect.stringContaining('--limit') });
expect(page.goto).not.toHaveBeenCalled();
});
it('throws AuthRequired when captcha gate is detected', async () => {
const page = createPageMock(['captcha']);
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toThrow('Ctrip is asking for a captcha');
expect(page.evaluate).toHaveBeenCalledTimes(1);
});
it('throws EmptyResultError when DOM extraction returns no flights', async () => {
const page = createPageMock(['content', 0, []]);
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'EMPTY_RESULT' });
});
it('throws CommandExecutionError when visible cards render but parser finds no flight anchors', async () => {
const page = createPageMock(['content', 2, []]);
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('parser did not find required flight anchors'),
});
});
it('throws CommandExecutionError when flight render waits timeout or extraction is malformed', async () => {
await expect(cmd.func(createPageMock(['timeout']), { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('did not render flight cards') });
await expect(cmd.func(createPageMock(['content', 1, { rows: [] }]), { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('malformed rows') });
});
it('builds URL with lowercase IATA codes and Y_S_C_F cabin', async () => {
const page = createPageMock(['content', 1, [FLIGHT_RAW]]);
await cmd.func(page, { from: 'pek', to: 'sha', date: '2026-06-15', limit: 1 });
const url = page.goto.mock.calls[0][0];
expect(url).toContain('oneway-pek-sha');
expect(url).toContain('depdate=2026-06-15');
expect(url).toContain('cabin=Y_S_C_F');
expect(url).toContain('adult=1');
});
it('maps DOM-extracted rows and respects --limit', async () => {
const page = createPageMock([
'content',
2,
[FLIGHT_RAW, { ...FLIGHT_RAW, flightNo: 'CA1234', airline: '国航' }],
]);
const rows = await cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 1 });
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
rank: 1,
airline: '厦门航空',
flightNo: 'MF8561',
departureTime: '07:50',
arrivalTime: '09:45',
price: 487,
currency: '¥',
cabin: '经济舱',
});
for (const row of rows) {
for (const col of cmd.columns) expect(row).toHaveProperty(col);
}
});
it('filters out flight rows missing core anchors (no silent partial rows)', async () => {
const page = createPageMock(['content', 2, [{ ...FLIGHT_RAW, departureTime: '' }, FLIGHT_RAW]]);
const rows = await cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 });
expect(rows).toHaveLength(1);
expect(rows[0].departureTime).toBe('07:50');
});
it('throws CommandExecutionError when every flight row misses core anchors', async () => {
const page = createPageMock(['content', 2, [{ ...FLIGHT_RAW, departureAirport: '' }, { ...FLIGHT_RAW, flightNo: null }]]);
await expect(cmd.func(page, { from: 'PEK', to: 'SHA', date: '2026-06-15', limit: 5 }))
.rejects.toMatchObject({ code: 'COMMAND_EXEC', message: expect.stringContaining('required airline/flight/time/airport anchors') });
});
});
describe('ctrip buildScrollUntilJs', () => {
it('inlines the row selector + target count + default maxScrolls', () => {
const js = buildScrollUntilJs('.flight-list > span > div', 20);
expect(js).toContain('"\.flight-list > span > div"'.replace('\\.', '.')); // selector literal
expect(js).toContain('countItems() >= 20');
expect(js).toContain('i < 8');
expect(js).toContain('plateauRounds');
expect(js).toContain('getBoundingClientRect');
expect(js).toContain('getComputedStyle');
});
it('respects a custom maxScrolls override', () => {
const js = buildScrollUntilJs('.hotel-card', 50, 3);
expect(js).toContain('countItems() >= 50');
expect(js).toContain('i < 3');
});
it('rejects unsafe target / maxScrolls values before interpolation', () => {
expect(() => buildScrollUntilJs('.hotel-card', 0)).toThrow('targetCount');
expect(() => buildScrollUntilJs('.hotel-card', 101)).toThrow('targetCount');
expect(() => buildScrollUntilJs('.hotel-card', 10, 0)).toThrow('maxScrolls');
expect(() => buildScrollUntilJs('.hotel-card', 10, 31)).toThrow('maxScrolls');
});
});
describe('ctrip buildFlightExtractJs (JSDOM)', () => {
function runExtract(html) {
const dom = new JSDOM(`<!doctype html><html><body>${html}</body></html>`,
{ url: 'https://flights.ctrip.com/' });
const js = buildFlightExtractJs();
return Function('document', `return (${js})`)(dom.window.document);
}
it('extracts a single ordered card via position-anchored chunks', () => {
const html = `
<div class="flight-list"><span>
<div>
<span>厦门航空</span><span>MF8561</span><span>空客321(中)</span>
<span>当日低价</span>
<span>07:50</span><span>大兴国际机场</span>
<span>09:45</span><span>浦东国际机场</span><span>T2</span>
<span>已减¥3</span><span>惊喜低价</span>
<span>¥</span><span>487</span><span>起</span>
<span>经济舱</span><span>订票</span>
</div>
</span></div>
`;
const rows = runExtract(html);
expect(rows).toEqual([{
airline: '厦门航空',
flightNo: 'MF8561',
aircraft: '空客321(中)',
departureTime: '07:50',
departureAirport: '大兴国际机场',
arrivalTime: '09:45',
arrivalAirport: '浦东国际机场',
terminal: 'T2',
price: 487,
currency: '¥',
cabin: '经济舱',
}]);
});
it('omits terminal when not present after arrAirport', () => {
const html = `
<div class="flight-list"><span>
<div>
<span>国航</span><span>CA1234</span><span>波音737</span>
<span>08:00</span><span>首都国际机场</span>
<span>10:00</span><span>虹桥国际机场</span>
<span>¥</span><span>520</span><span>起</span><span>经济舱</span>
</div>
</span></div>
`;
const rows = runExtract(html);
expect(rows).toHaveLength(1);
expect(rows[0].terminal).toBeNull();
expect(rows[0].arrivalAirport).toBe('虹桥国际机场');
});
it('returns empty array when there are no flight cards (not a sentinel row)', () => {
const rows = runExtract('<div class="flight-list"></div>');
expect(rows).toEqual([]);
});
it('does not fabricate rows from non-flight cards with two times', () => {
const html = `
<div class="flight-list"><span>
<div>
<span>筛选</span><span>价格排序</span><span>推荐</span>
<span>08:00</span><span>出发</span><span>10:00</span><span>到达</span>
<span>¥</span><span>520</span><span>经济舱</span>
</div>
</span></div>
`;
expect(runExtract(html)).toEqual([]);
});
});
+136
View File
@@ -0,0 +1,136 @@
/**
* 携程机票 oneway search — domestic + international flight search by route + date.
*
* Unlike `hotel-search`, the flight rows are NOT in `__NEXT_DATA__` — they
* arrive via a post-load XHR that the daemon network buffer currently can't
* capture (see MEMORY `daemon_capture_pipeline_bug_2026_05_07`). We instead
* extract from the rendered `.flight-list > span > div` cards using a
* position-anchored innerText parser (see `buildFlightExtractJs` in utils).
*
* Round-trip + advanced filters (airline whitelist, cabin selection beyond
* 全舱位) are out of scope for v1 — track in #1481 follow-up if requested.
*/
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { buildFlightExtractJs, buildScrollUntilJs, parseIataCode, parseIsoDate } from './utils.js';
const MIN_LIMIT = 1;
const MAX_LIMIT = 50;
const DEFAULT_LIMIT = 20;
function parseFlightLimit(raw) {
if (raw === undefined || raw === null || raw === '') return DEFAULT_LIMIT;
const parsed = Number(raw);
if (!Number.isFinite(parsed) || !Number.isInteger(parsed)) {
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}, got ${JSON.stringify(raw)}`);
}
if (parsed < MIN_LIMIT || parsed > MAX_LIMIT) {
throw new ArgumentError(`--limit must be between ${MIN_LIMIT} and ${MAX_LIMIT}, got ${parsed}`);
}
return parsed;
}
/**
* Wait for `.flight-list > span > div` to render (the post-load XHR settles
* 1-3s after navigation), or detect a captcha/login redirect.
*/
const WAIT_FOR_FLIGHTS_JS = `
new Promise((resolve) => {
const detect = () => {
if (location.pathname.includes('captcha') || /验证码|verify the human/i.test(document.body?.innerText || '')) return 'captcha';
if (document.querySelector('.flight-list > span > div')) return 'content';
return null;
};
const found = detect();
if (found) return resolve(found);
const observer = new MutationObserver(() => {
const result = detect();
if (result) { observer.disconnect(); resolve(result); }
});
observer.observe(document.documentElement, { childList: true, subtree: true });
setTimeout(() => { observer.disconnect(); resolve('timeout'); }, 8000);
})
`;
cli({
site: 'ctrip',
name: 'flight',
access: 'read',
description: '搜索携程一程机票(按出发/到达 IATA 三字码 + 日期)',
domain: 'flights.ctrip.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'from', required: true, positional: true, help: 'Departure IATA code (e.g. BJS / PEK)' },
{ name: 'to', required: true, positional: true, help: 'Arrival IATA code (e.g. SHA / PVG)' },
{ name: 'date', required: true, help: 'Departure date (YYYY-MM-DD)' },
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: `Number of flights (${MIN_LIMIT}-${MAX_LIMIT})` },
],
columns: [
'rank',
'airline', 'flightNo', 'aircraft',
'departureTime', 'departureAirport',
'arrivalTime', 'arrivalAirport', 'terminal',
'price', 'currency', 'cabin',
'url',
],
func: async (page, kwargs) => {
const fromCode = parseIataCode('from', kwargs.from);
const toCode = parseIataCode('to', kwargs.to);
if (fromCode === toCode) {
throw new ArgumentError(`--from and --to must differ (got ${fromCode})`);
}
const date = parseIsoDate('date', kwargs.date);
const limit = parseFlightLimit(kwargs.limit);
const searchUrl =
`https://flights.ctrip.com/online/list/oneway-${fromCode.toLowerCase()}-${toCode.toLowerCase()}` +
`?depdate=${date}&cabin=Y_S_C_F&adult=1&child=0&infant=0`;
await page.goto(searchUrl);
const waitResult = await page.evaluate(WAIT_FOR_FLIGHTS_JS);
if (waitResult === 'captcha') {
throw new AuthRequiredError('flights.ctrip.com', 'Ctrip is asking for a captcha; complete it in your browser session and retry');
}
if (waitResult !== 'content') {
throw new CommandExecutionError(`Ctrip flight page did not render flight cards (state=${String(waitResult)})`);
}
// Scroll until enough flight cards rendered (Ctrip lazy-loads beyond ~8).
const renderedCardCount = await page.evaluate(buildScrollUntilJs('.flight-list > span > div', limit));
const raw = await page.evaluate(buildFlightExtractJs());
if (!Array.isArray(raw)) {
throw new CommandExecutionError('Ctrip flight DOM extraction returned malformed rows');
}
const rows = raw;
if (rows.length === 0) {
if (Number(renderedCardCount) > 0) {
throw new CommandExecutionError('Ctrip flight cards rendered but parser did not find required flight anchors');
}
throw new EmptyResultError('ctrip flight', `No flights for ${fromCode}${toCode} on ${date}`);
}
const completeRows = rows
.filter((r) => r.departureTime && r.departureAirport && r.arrivalTime && r.arrivalAirport && r.airline && r.flightNo)
.slice(0, limit)
.map((r, i) => ({
rank: i + 1,
airline: r.airline,
flightNo: r.flightNo,
aircraft: r.aircraft,
departureTime: r.departureTime,
departureAirport: r.departureAirport,
arrivalTime: r.arrivalTime,
arrivalAirport: r.arrivalAirport,
terminal: r.terminal,
price: r.price,
currency: r.currency,
cabin: r.cabin,
url: searchUrl,
}));
if (completeRows.length === 0) {
throw new CommandExecutionError('Ctrip flight rows were missing required airline/flight/time/airport anchors');
}
return completeRows;
},
});
export const __test__ = { parseFlightLimit, WAIT_FOR_FLIGHTS_JS };
+132
View File
@@ -0,0 +1,132 @@
/**
* 携程酒店 list — search hotels by city + date range.
*
* Reads `window.__NEXT_DATA__.props.pageProps.initListData.hotelList` directly
* from the SSR-rendered hotel listing page. Ctrip serves first 13 hotels
* (10 organic + ~3 promoted) inline; `&pageSize=N` URL params are ignored
* server-side so we cap default limit accordingly (see
* `~/.opencli/sites/ctrip/notes.md`).
*
* Reuses the existing `mapHotelRow` + `pickHotelMapCoords` helpers from utils.js
* so the column shape stays consistent if future variants (hotel-detail) also
* project from the same `hotelInfo` shape.
*
* Anti-bot: not detected on first-page navigation (PR #1481 recon 2026-05-12).
*/
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { mapHotelRow, parseCityId, parseIsoDate } from './utils.js';
const MIN_LIMIT = 1;
const MAX_LIMIT = 30;
const DEFAULT_LIMIT = 10;
function parseHotelLimit(raw) {
if (raw === undefined || raw === null || raw === '') return DEFAULT_LIMIT;
const parsed = Number(raw);
if (!Number.isFinite(parsed) || !Number.isInteger(parsed)) {
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}, got ${JSON.stringify(raw)}`);
}
if (parsed < MIN_LIMIT || parsed > MAX_LIMIT) {
throw new ArgumentError(`--limit must be between ${MIN_LIMIT} and ${MAX_LIMIT}, got ${parsed}`);
}
return parsed;
}
/**
* Wait for SSR state to be populated, or detect a login/captcha gate.
*
* Ctrip occasionally serves a captcha redirect (`/captcha`) when traffic
* looks bot-like; we catch that as AuthRequired so the agent can pop a
* human session instead of looping on an empty extract.
*/
const WAIT_FOR_SSR_JS = `
new Promise((resolve) => {
const detect = () => {
if (location.pathname.includes('captcha') || /验证码|verify the human/i.test(document.body?.innerText || '')) return 'captcha';
const hotels = window.__NEXT_DATA__?.props?.pageProps?.initListData?.hotelList;
if (Array.isArray(hotels)) return 'content';
return null;
};
const found = detect();
if (found) return resolve(found);
const observer = new MutationObserver(() => {
const result = detect();
if (result) { observer.disconnect(); resolve(result); }
});
observer.observe(document.documentElement, { childList: true, subtree: true });
setTimeout(() => { observer.disconnect(); resolve('timeout'); }, 5000);
})
`;
const EXTRACT_HOTELS_JS = `
(() => {
const list = window.__NEXT_DATA__?.props?.pageProps?.initListData?.hotelList;
if (!Array.isArray(list)) return null;
return list;
})()
`;
function assertCheckinBeforeCheckout(checkin, checkout) {
if (Date.parse(checkin + 'T00:00:00Z') >= Date.parse(checkout + 'T00:00:00Z')) {
throw new ArgumentError(`--checkin must be earlier than --checkout (got ${checkin} >= ${checkout})`);
}
}
cli({
site: 'ctrip',
name: 'hotel-search',
access: 'read',
description: '搜索携程酒店列表(按城市 + 入住/离店日期)',
domain: 'hotels.ctrip.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'city', required: true, positional: true, help: 'Numeric Ctrip city ID (use `ctrip search` or `ctrip hotel-suggest` to discover)' },
{ name: 'checkin', required: true, help: 'Check-in date (YYYY-MM-DD)' },
{ name: 'checkout', required: true, help: 'Check-out date (YYYY-MM-DD)' },
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: `Number of hotels (${MIN_LIMIT}-${MAX_LIMIT}); SSR first page returns ~13 entries` },
],
columns: [
'rank', 'hotelId', 'name', 'enName',
'star', 'score', 'scoreLabel', 'reviewCount',
'cityName', 'district', 'address',
'lat', 'lon',
'price', 'currency', 'url',
],
func: async (page, kwargs) => {
const cityId = parseCityId(kwargs.city);
const checkin = parseIsoDate('checkin', kwargs.checkin);
const checkout = parseIsoDate('checkout', kwargs.checkout);
assertCheckinBeforeCheckout(checkin, checkout);
const limit = parseHotelLimit(kwargs.limit);
const url = `https://hotels.ctrip.com/hotels/list?city=${cityId}&checkin=${checkin}&checkout=${checkout}`;
await page.goto(url);
const waitResult = await page.evaluate(WAIT_FOR_SSR_JS);
if (waitResult === 'captcha') {
throw new AuthRequiredError('hotels.ctrip.com', 'Ctrip is asking for a captcha; complete it in your browser session and retry');
}
if (waitResult !== 'content') {
throw new CommandExecutionError(`Ctrip hotel-search page did not expose SSR hotel list (state=${String(waitResult)})`);
}
const raw = await page.evaluate(EXTRACT_HOTELS_JS);
if (!Array.isArray(raw)) {
throw new CommandExecutionError('Ctrip hotel-search returned malformed SSR hotel list');
}
if (raw.length === 0) {
throw new EmptyResultError('ctrip hotel-search', `No hotels for city=${cityId} on ${checkin}${checkout}`);
}
const rows = raw
.map((entry, i) => mapHotelRow(entry, i))
.filter((row) => row.hotelId && row.name)
.slice(0, limit);
if (rows.length === 0) {
throw new CommandExecutionError('Ctrip hotel-search SSR rows were missing required hotelId/name anchors');
}
return rows;
},
});
export const __test__ = { parseHotelLimit, assertCheckinBeforeCheckout, WAIT_FOR_SSR_JS, EXTRACT_HOTELS_JS };
+298
View File
@@ -172,4 +172,302 @@ export function mapSuggestRow(item, index) {
};
}
/* --------- Helpers shared by hotel-search / flight (browser-context) ---------- */
const ISO_DATE_RE = /^(\d{4})-(\d{2})-(\d{2})$/;
/**
* Validate YYYY-MM-DD and return the canonical string. Rejects out-of-range
* month/day, malformed input, and silent NaN. Does NOT coerce or shift timezones.
*/
export function parseIsoDate(name, raw) {
if (raw === undefined || raw === null || raw === '') {
throw new ArgumentError(`--${name} is required (YYYY-MM-DD)`);
}
const value = String(raw).trim();
const m = ISO_DATE_RE.exec(value);
if (!m) {
throw new ArgumentError(`--${name} must be YYYY-MM-DD, got ${JSON.stringify(raw)}`);
}
const year = Number(m[1]);
const month = Number(m[2]);
const day = Number(m[3]);
if (month < 1 || month > 12 || day < 1 || day > 31) {
throw new ArgumentError(`--${name} has invalid month/day: ${value}`);
}
// Cross-check via UTC date math so 2026-02-30 doesn't pass.
const parsed = new Date(Date.UTC(year, month - 1, day));
if (parsed.getUTCFullYear() !== year || parsed.getUTCMonth() !== month - 1 || parsed.getUTCDate() !== day) {
throw new ArgumentError(`--${name} is not a real calendar date: ${value}`);
}
return value;
}
/**
* Validate a 3-letter IATA airport / metro code, return uppercase.
* Ctrip URL accepts both single-airport (PEK / PVG) and metro-group (BJS / SHA) codes.
*/
export function parseIataCode(name, raw) {
if (raw === undefined || raw === null || raw === '') {
throw new ArgumentError(`--${name} is required (3-letter IATA code, e.g. PEK, SHA)`);
}
const value = String(raw).trim().toUpperCase();
if (!/^[A-Z]{3}$/.test(value)) {
throw new ArgumentError(`--${name} must be a 3-letter IATA code, got ${JSON.stringify(raw)}`);
}
return value;
}
/**
* Validate a numeric Ctrip city ID (returned by `ctrip search` / `ctrip hotel-suggest`).
*/
export function parseCityId(raw) {
if (raw === undefined || raw === null || raw === '') {
throw new ArgumentError('--city is required (numeric city ID from `ctrip search` or `ctrip hotel-suggest`)');
}
const parsed = Number(raw);
if (!Number.isFinite(parsed) || !Number.isInteger(parsed) || parsed <= 0) {
throw new ArgumentError(`--city must be a positive integer city ID, got ${JSON.stringify(raw)}`);
}
return parsed;
}
/**
* Pick the best lat/lon from a Ctrip hotel `positionInfo.mapCoordinate` array.
*
* Each entry has a `coordinateType` (1=WGS84, 2=GCJ02, 3=BD09 / Baidu). We prefer
* WGS84 when present (most portable), then fall through. All coordinates are
* strings in the API, so we Number() and reject NaN.
*/
export function pickHotelMapCoords(mapCoordinate) {
if (!Array.isArray(mapCoordinate) || mapCoordinate.length === 0) {
return { lat: null, lon: null };
}
// Order: WGS84 (1) → GCJ02 (2) → BD09 (3) → whatever exists
const ranking = (entry) => {
const t = Number(entry?.coordinateType);
if (t === 1) return 0;
if (t === 2) return 1;
if (t === 3) return 2;
return 3;
};
const sorted = [...mapCoordinate].sort((a, b) => ranking(a) - ranking(b));
for (const entry of sorted) {
const lat = Number(entry?.latitude);
const lon = Number(entry?.longitude);
if (Number.isFinite(lat) && Number.isFinite(lon) && (lat !== 0 || lon !== 0)) {
return { lat, lon };
}
}
return { lat: null, lon: null };
}
/**
* Project a single Ctrip hotel row from `__NEXT_DATA__.props.pageProps.initListData.hotelList[*]`
* into stable adapter column shape.
*
* No silent fallbacks — every field is `string|number|null`, never `''` masquerading
* as "no data" (see typed-errors.md §"scalar sentinels are anti-pattern").
*/
export function mapHotelRow(entry, index) {
const hotelInfo = entry?.hotelInfo ?? {};
const rooms = Array.isArray(entry?.roomInfo) ? entry.roomInfo : [];
const summary = hotelInfo.summary ?? {};
const nameInfo = hotelInfo.nameInfo ?? {};
const hotelStar = hotelInfo.hotelStar ?? {};
const commentInfo = hotelInfo.commentInfo ?? {};
const positionInfo = hotelInfo.positionInfo ?? {};
const firstRoom = rooms[0] ?? {};
const priceInfo = firstRoom.priceInfo ?? {};
const hotelId = summary.hotelId ? String(summary.hotelId) : null;
const { lat, lon } = pickHotelMapCoords(positionInfo.mapCoordinate);
// commenterNumber arrives as "13,966条点评" — strip non-digits to int, else null.
let reviewCount = null;
if (commentInfo.commenterNumber) {
const digits = String(commentInfo.commenterNumber).replace(/[^\d]/g, '');
if (digits) reviewCount = Number(digits);
}
const score = commentInfo.commentScore ? Number(commentInfo.commentScore) : null;
const star = Number.isFinite(hotelStar.star) && hotelStar.star > 0 ? hotelStar.star : null;
const price = Number.isFinite(priceInfo.price) && priceInfo.price > 0 ? priceInfo.price : null;
return {
rank: index + 1,
hotelId,
name: nameInfo.name ? String(nameInfo.name).trim() : null,
enName: nameInfo.enName ? String(nameInfo.enName).trim() : null,
star,
score: Number.isFinite(score) && score > 0 ? score : null,
scoreLabel: commentInfo.commentDescription ? String(commentInfo.commentDescription).trim() : null,
reviewCount,
cityName: positionInfo.cityName ? String(positionInfo.cityName).trim() : null,
district: positionInfo.positionDesc ? String(positionInfo.positionDesc).trim() : null,
address: positionInfo.address ? String(positionInfo.address).trim() : null,
lat,
lon,
price,
currency: priceInfo.currency ? String(priceInfo.currency).trim() : null,
url: hotelId ? `https://hotels.ctrip.com/hotels/detail/?hotelid=${hotelId}` : null,
};
}
/**
* Build the browser-context IIFE that extracts flight rows from `.flight-list`.
*
* Flights are rendered as `.flight-list > span > div` cards. Each card's innerText
* has a stable ordering (verified 2026-05-12 on bjs→sha route):
*
* [airline, flightNo, aircraft, lowPriceTag?, depTime, depAirport,
* arrTime, arrAirport, terminal?, savings?, promo?, currency, price,
* priceSuffix, cabin, cta]
*
* `lowPriceTag` (e.g. "当日低价") + `terminal` (e.g. "T2") + `savings` + `promo`
* are optional — we use position-of-first-time-match to anchor and parse around it.
*
* The host is baked in so `normalizeUrl` for booking links resolves on the calling site.
*/
export function buildFlightExtractJs() {
return `
(() => {
const cleanText = (value) => (value || '').replace(/\\s+/g, ' ').trim();
const isTime = (s) => /^([01]?\\d|2[0-3]):[0-5]\\d$/.test(s);
const isCurrency = (s) => /^[¥$€£]$/.test(s);
const isPriceDigits = (s) => /^\\d+([.,]\\d+)?$/.test(s);
const isFlightNo = (s) => /^[A-Z0-9]{2}\\d{3,4}[A-Z]?$/.test(s);
const rows = [];
document.querySelectorAll('.flight-list > span > div').forEach((card) => {
// Collect ordered text chunks (text nodes only, skip whitespace-only).
const chunks = [];
const walk = (node) => {
for (const c of node.childNodes) {
if (c.nodeType === 3) {
const t = cleanText(c.textContent);
if (t) chunks.push(t);
} else if (c.nodeType === 1) {
walk(c);
}
}
};
walk(card);
if (chunks.length < 8) return;
// Anchor on first HH:MM — that's depTime; depAirport is immediately after.
const firstTimeIdx = chunks.findIndex(isTime);
if (firstTimeIdx < 1) return;
const airline = chunks[0];
const flightNo = chunks[1] || null;
if (!airline || !isFlightNo(flightNo)) return;
const aircraft = chunks[2] && !isTime(chunks[2]) ? chunks[2] : null;
const depTime = chunks[firstTimeIdx];
const depAirport = chunks[firstTimeIdx + 1] || null;
// Second HH:MM after depTime is arrTime
const arrTimeIdx = chunks.findIndex((c, i) => i > firstTimeIdx && isTime(c));
if (arrTimeIdx < 0) return;
const arrTime = chunks[arrTimeIdx];
const arrAirport = chunks[arrTimeIdx + 1] || null;
if (!depAirport || !arrAirport) return;
// Optional terminal chunk right after arrAirport (matches /^T\\d$/ or single letter)
let terminal = null;
if (arrTimeIdx + 2 < chunks.length && /^T\\d$/.test(chunks[arrTimeIdx + 2])) {
terminal = chunks[arrTimeIdx + 2];
}
// Price: scan for currency symbol then a digit-only chunk
let price = null;
let currency = null;
for (let i = 0; i < chunks.length - 1; i++) {
if (isCurrency(chunks[i]) && isPriceDigits(chunks[i + 1])) {
currency = chunks[i];
price = Number(chunks[i + 1].replace(',', ''));
break;
}
}
// Cabin: scan from end for first non-CTA Chinese chunk ending in "舱"
let cabin = null;
for (let i = chunks.length - 1; i >= 0; i--) {
if (/舱$/.test(chunks[i])) { cabin = chunks[i]; break; }
}
rows.push({
airline,
flightNo,
aircraft,
departureTime: depTime,
departureAirport: depAirport,
arrivalTime: arrTime,
arrivalAirport: arrAirport,
terminal,
price,
currency,
cabin,
});
});
return rows;
})()
`;
}
/**
* Build a scroll-until-enough IIFE for flights/hotels DOM-card pagination.
*
* Mirrors `clis/xiaohongshu/search.js#buildScrollUntilJs` (PR #1487) — counts a
* caller-supplied row selector, scrolls until count >= target / DOM plateau /
* maxScrolls. Returns final row count so the caller can decide whether to
* surface an EmptyResultError. (xiaohongshu's helper hardcodes
* `section.note-item`; this generic version takes a selector.)
*/
export function buildScrollUntilJs(rowSelector, targetCount, maxScrolls = 8) {
if (!Number.isInteger(targetCount) || targetCount < 1 || targetCount > 100) {
throw new ArgumentError(`targetCount must be an integer between 1 and 100, got ${JSON.stringify(targetCount)}`);
}
if (!Number.isInteger(maxScrolls) || maxScrolls < 1 || maxScrolls > 30) {
throw new ArgumentError(`maxScrolls must be an integer between 1 and 30, got ${JSON.stringify(maxScrolls)}`);
}
return `
(async () => {
const sel = ${JSON.stringify(rowSelector)};
const isVisible = (el) => {
const style = window.getComputedStyle(el);
if (style.display === 'none' || style.visibility === 'hidden' || Number(style.opacity) === 0) return false;
const rect = el.getBoundingClientRect();
return rect.width > 0 && rect.height > 0;
};
const countItems = () => Array.from(document.querySelectorAll(sel)).filter(isVisible).length;
let lastCount = countItems();
let plateauRounds = 0;
for (let i = 0; i < ${maxScrolls}; i++) {
if (countItems() >= ${targetCount}) break;
const lastHeight = document.body.scrollHeight;
window.scrollTo(0, lastHeight);
await new Promise((resolve) => {
let to;
const ob = new MutationObserver(() => {
if (document.body.scrollHeight > lastHeight) {
clearTimeout(to);
ob.disconnect();
setTimeout(resolve, 200);
}
});
ob.observe(document.body, { childList: true, subtree: true });
to = setTimeout(() => { ob.disconnect(); resolve(null); }, 2500);
});
const newCount = countItems();
if (newCount === lastCount) {
plateauRounds++;
if (plateauRounds >= 2) break;
} else {
plateauRounds = 0;
lastCount = newCount;
}
}
return countItems();
})()
`;
}
export const __test__ = { ENDPOINT, MIN_LIMIT, MAX_LIMIT };
+22 -8
View File
@@ -3,7 +3,7 @@ import { CliError, CommandExecutionError, EXIT_CODES } from '@jackwener/opencli/
import {
DEEPSEEK_DOMAIN, DEEPSEEK_URL, ensureOnDeepSeek, selectModel, setFeature,
sendMessage, sendWithFile, getBubbleCount, waitForResponse, parseBoolFlag, withRetry,
pickResumeUrl,
pickResumeUrl, TEXTAREA_SELECTOR,
} from './utils.js';
export const askCommand = cli({
@@ -14,6 +14,7 @@ export const askCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
@@ -34,7 +35,13 @@ export const askCommand = cli({
if (parseBoolFlag(kwargs.new)) {
await page.goto(DEEPSEEK_URL);
await page.wait(3);
// Wait for the composer to mount instead of a fixed 3 s sleep.
try {
await page.wait({ selector: TEXTAREA_SELECTOR, timeout: 8 });
} catch {
// Selector still missing → downstream selectModel/sendMessage
// will surface the failure with a typed error.
}
} else {
const navigated = await ensureOnDeepSeek(page);
if (navigated) {
@@ -48,12 +55,15 @@ export const askCommand = cli({
);
}
await page.goto(resumeUrl);
await page.wait(2);
try {
await page.wait({ selector: TEXTAREA_SELECTOR, timeout: 5 });
} catch {
// Conversation page may still be loading; subsequent steps
// will retry or report.
}
}
}
await page.wait(2);
// Model selector is only available on the new-chat page, not inside
// an existing conversation. Skip it when we resumed a prior thread.
const currentUrl = await page.evaluate('window.location.href') || '';
@@ -75,7 +85,9 @@ export const askCommand = cli({
if (!modelResult?.ok) {
throw new CommandExecutionError(`Could not switch to ${wantModel} model`);
}
if (modelResult?.toggled) await page.wait(0.5);
// The 0.5 s settle previously here was redundant: each subsequent
// step (setFeature, sendMessage) issues a fresh CDP eval, giving
// React more than enough time to flush the toggle's state update.
}
const thinkResult = await withRetry(() => setFeature(page, 'DeepThink', wantThink));
@@ -101,7 +113,8 @@ export const askCommand = cli({
}
}
if (thinkResult?.toggled || searchResult?.toggled) await page.wait(0.5);
// No settle wait after toggles: the next CDP eval below already gives
// React time to flush the aria-checked state.
if (kwargs.file) {
const baseline = await withRetry(() => getBubbleCount(page));
@@ -114,7 +127,8 @@ export const askCommand = cli({
// SPA navigates after send; "Promise was collected" means send succeeded
if (!String(err?.message || err).includes('Promise was collected')) throw err;
}
await page.wait(3);
// waitForResponse polls every 3 s for new bubbles, so the previous
// 3 s settle here was a redundant sleep on top of the first poll.
const result = await waitForResponse(page, baseline, prompt, timeoutMs, wantThink);
if (!result) {
return [{ response: `[NO RESPONSE] No reply within ${kwargs.timeout}s.` }];
+10 -1
View File
@@ -2,6 +2,7 @@ import { cli, Strategy } from '@jackwener/opencli/registry';
import { EmptyResultError } from '@jackwener/opencli/errors';
import {
DEEPSEEK_DOMAIN,
MESSAGE_SELECTOR,
ensureOnDeepSeek,
getVisibleMessages,
parseDeepSeekConversationId,
@@ -15,6 +16,7 @@ export const detailCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Conversation ID (UUID) or full /a/chat/s/<id> URL' },
@@ -24,7 +26,14 @@ export const detailCommand = cli({
const id = parseDeepSeekConversationId(kwargs.id);
await ensureOnDeepSeek(page);
await page.goto(`https://chat.deepseek.com/a/chat/s/${id}`);
await page.wait(5);
// Wait for at least one rendered bubble instead of a fixed 5 s sleep.
// Empty / invalid conversations fall through to the EmptyResultError
// below.
try {
await page.wait({ selector: MESSAGE_SELECTOR, timeout: 10 });
} catch {
// No bubble mounted within 10 s; treated as empty by the check below.
}
const messages = await getVisibleMessages(page);
if (messages.length === 0) {
throw new EmptyResultError(
+1
View File
@@ -9,6 +9,7 @@ export const historyCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Max conversations to show' },
+14 -2
View File
@@ -1,5 +1,6 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { DEEPSEEK_DOMAIN, DEEPSEEK_URL } from './utils.js';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { DEEPSEEK_DOMAIN, DEEPSEEK_URL, TEXTAREA_SELECTOR } from './utils.js';
export const newCommand = cli({
site: 'deepseek',
@@ -9,13 +10,24 @@ export const newCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status'],
func: async (page) => {
await page.goto(DEEPSEEK_URL);
await page.wait(2);
// Confirm the composer mounted before reporting success. The previous
// 2 s blind sleep would return "New chat started" even when the page
// was still loading or the user was logged out.
try {
await page.wait({ selector: TEXTAREA_SELECTOR, timeout: 8 });
} catch {
throw new CommandExecutionError(
'DeepSeek composer did not mount within 8 s',
'Verify you are logged into chat.deepseek.com.',
);
}
return [{ Status: 'New chat started' }];
},
});
+3 -1
View File
@@ -9,13 +9,15 @@ export const readCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Role', 'Text'],
func: async (page) => {
// ensureOnDeepSeek already waits for the composer to mount; the
// follow-up 5 s sleep was redundant.
await ensureOnDeepSeek(page);
await page.wait(5);
const messages = await getVisibleMessages(page);
if (messages.length > 0) return messages;
return [{ Role: 'system', Text: 'No visible messages found.' }];
+1
View File
@@ -15,6 +15,7 @@ export const sendCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Conversation ID (UUID) or full /a/chat/s/<id> URL' },
+1
View File
@@ -9,6 +9,7 @@ export const statusCommand = cli({
domain: DEEPSEEK_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status', 'Login', 'Url'],
+8 -1
View File
@@ -43,7 +43,14 @@ export async function isOnDeepSeek(page) {
export async function ensureOnDeepSeek(page) {
if (await isOnDeepSeek(page)) return false;
await page.goto(DEEPSEEK_URL);
await page.wait(3);
// Wait for the composer textarea instead of a fixed 3 s sleep. On the login
// page it never mounts; swallow the timeout so callers (status / read /
// history) can still inspect page state.
try {
await page.wait({ selector: TEXTAREA_SELECTOR, timeout: 8 });
} catch {
// Login or error page — downstream will see hasTextarea=false / empty results.
}
return true;
}
+185
View File
@@ -0,0 +1,185 @@
/**
* Async city → cityId resolver for dianping adapters.
*
* Wraps the synchronous static-map `resolveCityId` from utils.js and falls
* back to a live lookup against www.dianping.com when the input is not in
* the curated map. Resolves both pinyin slugs (e.g. "shantou") and Chinese
* names (e.g. "汕头") by reading dianping itself, so the adapter no longer
* has to ship a complete static city table.
*
* Strategy:
* 1. Empty / null → null (let the cookie's default city stand).
* 2. All-digits → numeric cityId pass-through.
* 3. Static map → fast path, no network. Reuses utils.CITY_ID.
* 4. Pinyin slug → goto https://www.dianping.com/<slug>, parse the
* cityId out of any /search/keyword/{id}/ link.
* 5. Chinese name → goto https://www.dianping.com/citylist, build a
* Chinese-name → pinyin map, then resolve the slug
* as in step 4.
*
* Resolved (input → cityId) pairs are memoized in a module-level Map so a
* second search in the same process skips both navigations.
*/
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { CITY_ID, resolveCityId } from './utils.js';
const CHINESE_RE = /^[一-龥]+$/;
const PINYIN_RE = /^[a-z]+$/;
const RESOLVE_CACHE = new Map();
/**
* Reset the in-process resolver cache. Exposed for tests so each case
* starts from a clean slate; production code never needs to call this.
*/
export function clearCityResolverCache() {
RESOLVE_CACHE.clear();
}
/**
* Async resolver. Falls back to live dianping pages only when the input
* is not in the static map.
*
* @param {{ goto: Function, evaluate: Function }} page page handle from the adapter func
* @param {string|number|null|undefined} cityArg user-supplied city (name, pinyin, or numeric id)
* @returns {Promise<number|null>} numeric cityId, or null to use the cookie default
*/
export async function resolveCityIdAsync(page, cityArg) {
if (cityArg == null || cityArg === '') return null;
const raw = String(cityArg).trim();
if (!raw) return null;
if (/^\d+$/.test(raw)) return Number(raw);
const lowered = raw.toLowerCase();
// Fast path: reuse the synchronous static map. resolveCityId throws
// ArgumentError when the input is unknown — that's the trigger to fall
// back to dynamic resolution rather than surface the error to the user.
try {
const staticId = resolveCityId(raw);
if (staticId != null) return staticId;
} catch (err) {
if (err?.code !== 'ARGUMENT') throw err;
}
if (RESOLVE_CACHE.has(lowered)) return RESOLVE_CACHE.get(lowered);
if (RESOLVE_CACHE.has(raw)) return RESOLVE_CACHE.get(raw);
let pinyin = null;
if (PINYIN_RE.test(lowered)) {
pinyin = lowered;
} else if (CHINESE_RE.test(raw)) {
pinyin = await lookupPinyinFromCitylist(page, raw);
if (!pinyin) {
const known = Object.keys(CITY_ID).filter((k) => /^[a-z]+$/.test(k)).join(', ');
throw new ArgumentError(
'city',
`unknown city '${cityArg}'. pass a numeric cityId, a pinyin slug (e.g. shantou), `
+ `a Chinese name listed on dianping.com/citylist, or one of: ${known}`,
);
}
} else {
const known = Object.keys(CITY_ID).filter((k) => /^[a-z]+$/.test(k)).join(', ');
throw new ArgumentError(
'city',
`unknown city '${cityArg}'. pass a numeric cityId, a pinyin slug (e.g. shantou), `
+ `a Chinese name listed on dianping.com/citylist, or one of: ${known}`,
);
}
const cityId = await fetchCityIdByPinyin(page, pinyin);
if (!cityId) {
throw new CommandExecutionError(
`dianping could not resolve cityId for '${cityArg}' (pinyin=${pinyin}); `
+ `the city page rendered without a /search/keyword/{id}/ link`,
);
}
RESOLVE_CACHE.set(lowered, cityId);
RESOLVE_CACHE.set(pinyin, cityId);
if (CHINESE_RE.test(raw)) RESOLVE_CACHE.set(raw, cityId);
return cityId;
}
/**
* Read https://www.dianping.com/citylist and return a Chinese-name → pinyin
* slug map for every city link present on the page. Used when the user
* supplied a Chinese name that isn't in the static map.
*/
async function lookupPinyinFromCitylist(page, chineseName) {
await page.goto('https://www.dianping.com/citylist');
const map = await page.evaluate(`(${buildCitylistMap.toString()})()`);
if (!map || typeof map !== 'object' || Object.keys(map).length === 0) {
throw new CommandExecutionError(
'dianping citylist did not render any city anchors; cannot resolve Chinese city names',
);
}
if (map && typeof map === 'object' && map[chineseName]) {
return String(map[chineseName]).toLowerCase();
}
return null;
}
/**
* Pure DOM extractor for /citylist. Walks every anchor on the page and
* keeps the ones whose href matches the per-city slug shape and whose
* text is a pure-Chinese label. Defined at module scope so the same code
* can be exercised from JSDOM tests via toString() injection.
*/
export function buildCitylistMap() {
const map = {};
const anchors = document.querySelectorAll('a');
anchors.forEach((a) => {
const hrefRaw = a.getAttribute('href') || '';
const text = ((a.textContent || '').trim());
if (!text || !/^[一-龥]+$/.test(text)) return;
const href = hrefRaw.replace(/^https?:/, '');
const m = href.match(/^\/\/(?:www\.)?dianping\.com\/([a-z]+)\/?$/)
|| href.match(/^\/([a-z]+)\/?$/);
if (!m) return;
const slug = m[1].toLowerCase();
// Filter out non-city slugs that share the shape (e.g. /citylist itself,
// /promo, /events). Only register the first slug per Chinese label.
if (slug === 'citylist' || slug === 'promo' || slug === 'events') return;
if (!map[text]) map[text] = slug;
});
return map;
}
/**
* Visit https://www.dianping.com/<slug> and pull the cityId out of any
* /search/keyword/{id}/ anchor. The PC city landing page renders these
* links server-side for every category card, so a single goto + DOM read
* is enough — no extra clicks or hydration wait.
*/
async function fetchCityIdByPinyin(page, pinyin) {
await page.goto(`https://www.dianping.com/${pinyin}`);
const cityId = await page.evaluate(`(${extractCityIdFromPage.toString()})()`);
return Number.isInteger(cityId) && cityId > 0 ? cityId : null;
}
/**
* Pure DOM extractor for the per-city landing page. Defined at module
* scope so the same code is exercised in JSDOM tests via toString().
*/
export function extractCityIdFromPage() {
const baseHref = (typeof location !== 'undefined' && location.href) || 'https://www.dianping.com/';
const anchors = document.querySelectorAll('a[href]');
for (const a of anchors) {
const hrefRaw = a.getAttribute('href') || '';
let url;
try {
url = new URL(hrefRaw, baseHref);
} catch {
continue;
}
if (url.protocol !== 'https:') continue;
if (url.hostname !== 'www.dianping.com' && url.hostname !== 'dianping.com') continue;
const m = url.pathname.match(/^\/search\/keyword\/(\d+)(?:\/|$)/);
if (!m) continue;
const n = Number(m[1]);
if (Number.isInteger(n) && n > 0) return n;
}
return null;
}
+154
View File
@@ -22,6 +22,12 @@ import {
} from './utils.js';
import { extractSearchRows } from './search.js';
import { extractShopFields } from './shop.js';
import {
buildCitylistMap,
clearCityResolverCache,
extractCityIdFromPage,
resolveCityIdAsync,
} from './cityResolver.js';
const __dirname = dirname(fileURLToPath(import.meta.url));
const SHOP_FIXTURE = readFileSync(join(__dirname, '__fixtures__/shop.html'), 'utf8');
@@ -104,6 +110,154 @@ describe('dianping adapter — helpers', () => {
});
});
describe('dianping adapter — async city resolver', () => {
beforeEach(() => {
clearCityResolverCache();
});
it('returns null/numeric/static-map ids without ever touching the page', async () => {
const page = createPageMock({});
expect(await resolveCityIdAsync(page, undefined)).toBeNull();
expect(await resolveCityIdAsync(page, '')).toBeNull();
expect(await resolveCityIdAsync(page, ' ')).toBeNull();
expect(await resolveCityIdAsync(page, 47)).toBe(47);
expect(await resolveCityIdAsync(page, '47')).toBe(47);
expect(await resolveCityIdAsync(page, '北京')).toBe(2);
expect(await resolveCityIdAsync(page, 'shanghai')).toBe(1);
expect(page.goto).not.toHaveBeenCalled();
expect(page.evaluate).not.toHaveBeenCalled();
});
it('falls back to /<pinyin> for an unknown lowercase slug, then caches', async () => {
const goto = vi.fn().mockResolvedValue(undefined);
const evaluate = vi.fn().mockResolvedValue(207);
const page = { goto, evaluate, wait: vi.fn() };
expect(await resolveCityIdAsync(page, 'shantou')).toBe(207);
expect(goto).toHaveBeenCalledTimes(1);
expect(goto).toHaveBeenCalledWith('https://www.dianping.com/shantou');
// Second call hits the in-process cache, no extra navigation.
expect(await resolveCityIdAsync(page, 'shantou')).toBe(207);
expect(goto).toHaveBeenCalledTimes(1);
});
it('resolves a Chinese name via /citylist + /<pinyin>, then caches both forms', async () => {
const goto = vi.fn().mockResolvedValue(undefined);
const evaluate = vi.fn()
.mockResolvedValueOnce({ '汕头': 'shantou', '佛山': 'foshan' })
.mockResolvedValueOnce(207);
const page = { goto, evaluate, wait: vi.fn() };
expect(await resolveCityIdAsync(page, '汕头')).toBe(207);
expect(goto).toHaveBeenNthCalledWith(1, 'https://www.dianping.com/citylist');
expect(goto).toHaveBeenNthCalledWith(2, 'https://www.dianping.com/shantou');
// Cached for the Chinese name AND the discovered pinyin.
expect(await resolveCityIdAsync(page, '汕头')).toBe(207);
expect(await resolveCityIdAsync(page, 'shantou')).toBe(207);
expect(goto).toHaveBeenCalledTimes(2);
});
it('rejects mixed/garbage input with ArgumentError before any navigation', async () => {
const page = createPageMock({});
await expect(resolveCityIdAsync(page, 'not-a-city!')).rejects.toThrow(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('rejects a Chinese name that is not on /citylist with ArgumentError', async () => {
const goto = vi.fn().mockResolvedValue(undefined);
const evaluate = vi.fn().mockResolvedValueOnce({ '北京': 'beijing' });
const page = { goto, evaluate, wait: vi.fn() };
await expect(resolveCityIdAsync(page, '某虚构城')).rejects.toThrow(ArgumentError);
expect(goto).toHaveBeenCalledTimes(1);
expect(goto).toHaveBeenCalledWith('https://www.dianping.com/citylist');
});
it('throws CommandExecutionError when citylist renders without city anchors', async () => {
const goto = vi.fn().mockResolvedValue(undefined);
const evaluate = vi.fn().mockResolvedValueOnce({});
const page = { goto, evaluate, wait: vi.fn() };
await expect(resolveCityIdAsync(page, '汕头')).rejects.toThrow(CommandExecutionError);
expect(goto).toHaveBeenCalledTimes(1);
expect(goto).toHaveBeenCalledWith('https://www.dianping.com/citylist');
});
it('throws CommandExecutionError when the per-city page lacks a /search/keyword/{id}/ link', async () => {
const goto = vi.fn().mockResolvedValue(undefined);
const evaluate = vi.fn().mockResolvedValueOnce(null);
const page = { goto, evaluate, wait: vi.fn() };
await expect(resolveCityIdAsync(page, 'newcity')).rejects.toThrow(CommandExecutionError);
});
it('buildCitylistMap keeps Chinese-labeled city slugs and drops non-city paths', () => {
const dom = new JSDOM(`
<html><body>
<a href="//www.dianping.com/shanghai">上海</a>
<a href="//www.dianping.com/shantou">汕头</a>
<a href="/beijing">北京</a>
<a href="//www.dianping.com/citylist">更多城市 ></a>
<a href="//www.dianping.com/promo">优惠</a>
<a href="//www.dianping.com/shanghai">上海</a>
<a href="https://www.dianping.com/member/123">可乐不加冰</a>
<a href="https://example.com/notacity">东京</a>
</body></html>
`);
globalThis.document = dom.window.document;
try {
const map = buildCitylistMap();
expect(map['上海']).toBe('shanghai');
expect(map['汕头']).toBe('shantou');
expect(map['北京']).toBe('beijing');
expect(map['更多城市 >']).toBeUndefined();
expect(map['优惠']).toBeUndefined();
expect(map['可乐不加冰']).toBeUndefined();
expect(map['东京']).toBeUndefined();
} finally {
delete globalThis.document;
}
});
it('extractCityIdFromPage pulls the cityId from the first /search/keyword/{id}/ link', () => {
const dom = new JSDOM(`
<html><body>
<script>window.bad = "/search/keyword/999/";</script>
<a href="https://example.com/search/keyword/888/0_x">wrong host</a>
<a href="https://www.dianping.com.evil.com/search/keyword/666/0_x">host suffix</a>
<a href="http://www.dianping.com/search/keyword/777/0_x">non-https</a>
<a href="/search/keyword/207/0_%E5%88%BA%E8%BA%AB">刺身</a>
<a href="/search/category/207/10">美食</a>
</body></html>
`, { url: 'https://www.dianping.com/shantou' });
globalThis.document = dom.window.document;
globalThis.location = dom.window.location;
try {
expect(extractCityIdFromPage()).toBe(207);
} finally {
delete globalThis.document;
delete globalThis.location;
}
});
it('extractCityIdFromPage returns null when no /search/keyword/{id}/ link exists', () => {
const dom = new JSDOM(`<html><body><main>blocked</main></body></html>`);
globalThis.document = dom.window.document;
try {
expect(extractCityIdFromPage()).toBeNull();
} finally {
delete globalThis.document;
}
});
});
describe('dianping adapter — search runtime', () => {
const command = getRegistry().get('dianping/search');
+6 -3
View File
@@ -19,9 +19,9 @@ import {
parsePrice,
parseReviewCount,
requireSearchLimit,
resolveCityId,
wrapDianpingStep,
} from './utils.js';
import { resolveCityIdAsync } from './cityResolver.js';
/**
* Pure DOM extractor for the dianping search-results page.
@@ -98,7 +98,7 @@ cli({
strategy: Strategy.COOKIE,
args: [
{ name: 'keyword', required: true, positional: true, help: '搜索关键词,例如 "火锅"' },
{ name: 'city', help: '城市名(北京/上海/beijing/...)或 cityId 数字。不传则使用 cookie 默认城市' },
{ name: 'city', help: '城市名(北京/上海/汕头/beijing/shantou/...)或 cityId 数字。未在静态表中的城市会通过 dianping.com 在线解析。不传则使用 cookie 默认城市' },
{ name: 'limit', type: 'int', default: 15, help: '返回的店铺数量(最多 15,dianping 单页固定 15 条)' },
],
columns: SEARCH_COLUMNS,
@@ -108,7 +108,10 @@ cli({
const limit = requireSearchLimit(kwargs.limit);
const cityId = resolveCityId(kwargs.city);
const cityId = await wrapDianpingStep(
'city resolve',
() => resolveCityIdAsync(page, kwargs.city),
);
const path = cityId
? `/search/keyword/${cityId}/0_${encodeURIComponent(keyword)}`
: `/search/keyword/0/0_${encodeURIComponent(keyword)}`;
+1
View File
@@ -9,6 +9,7 @@ export const askCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'text', required: true, positional: true, help: 'Prompt to send' },
+1
View File
@@ -8,6 +8,7 @@ export const detailCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Conversation ID (numeric or full URL)' },
+1
View File
@@ -8,6 +8,7 @@ export const historyCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'limit', required: false, help: 'Max number of conversations to show', default: '50' },
+1
View File
@@ -8,6 +8,7 @@ export const meetingSummaryCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Conversation ID (numeric or full URL)' },
+1
View File
@@ -8,6 +8,7 @@ export const meetingTranscriptCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [
{ name: 'id', required: true, positional: true, help: 'Conversation ID (numeric or full URL)' },
+1
View File
@@ -8,6 +8,7 @@ export const newCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status', 'Action'],
+1
View File
@@ -8,6 +8,7 @@ export const readCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Role', 'Text'],
+1
View File
@@ -8,6 +8,7 @@ export const sendCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [{ name: 'text', required: true, positional: true, help: 'Message to send' }],
columns: ['Status', 'SubmittedBy', 'InjectedText'],
+1
View File
@@ -8,6 +8,7 @@ export const statusCommand = cli({
domain: DOUBAO_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
siteSession: 'persistent',
navigateBefore: false,
args: [],
columns: ['Status', 'Login', 'Url', 'Title'],
+17
View File
@@ -163,6 +163,19 @@ function getTurnsScript() {
) {
return 'Assistant';
}
// 2026-05 Doubao DOM refactor: no more receive-message / bg-g-receive-msg-bubble
// markers on assistant turns. Wrappers are now [class*="inner-item-"] /
// [class*="top-item-"] and the only reliable assistant signal is the
// .flow-markdown-body content container WITHOUT any send-bubble marker.
if (
(root.matches('[class*="inner-item-"], [class*="top-item-"]')
|| root.closest('[class*="inner-item-"], [class*="top-item-"]'))
&& (root.matches('.flow-markdown-body') || root.querySelector('.flow-markdown-body'))
&& !root.matches('[class*="bg-g-send-msg-bubble"]')
&& !root.querySelector('[class*="bg-g-send-msg-bubble"]')
) {
return 'Assistant';
}
return '';
};
@@ -223,6 +236,10 @@ function getTurnsScript() {
if (!messageList) return [];
const itemSelectors = [
// 2026-05 Doubao DOM refactor wrappers (prepended; outer ones win via
// ancestor-keep dedup below).
'[class*="inner-item-"]',
'[class*="top-item-"]',
'[class*="item-kDun2N"]',
'[data-testid="union_message"]',
'[data-testid="message-block-container"]',
+61
View File
@@ -1,3 +1,4 @@
import { JSDOM } from 'jsdom';
import { describe, expect, it, vi } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import {
@@ -145,6 +146,28 @@ describe('doubao send strategy', () => {
});
});
describe('doubao receive strategy', () => {
function runTurnsScript(html) {
const dom = new JSDOM(html, { url: 'https://www.doubao.com/chat', runScripts: 'outside-only' });
Object.defineProperty(dom.window.HTMLElement.prototype, 'innerText', {
configurable: true,
get() {
return this.textContent || '';
},
});
dom.window.HTMLElement.prototype.getBoundingClientRect = () => ({
width: 100,
height: 24,
top: 0,
left: 0,
right: 100,
bottom: 24,
x: 0,
y: 0,
toJSON: () => ({}),
});
return dom.window.eval(__test__.getTurnsScript());
}
it('keeps both the new skin selectors and the older structural fallbacks in the turns script', () => {
const turnsScript = __test__.getTurnsScript();
expect(turnsScript).toContain('[class*="message-list-S2Fv2S"]');
@@ -157,6 +180,44 @@ describe('doubao receive strategy', () => {
expect(turnsScript).toContain('[data-testid="message-block-container"]');
});
it('includes the 2026-05 doubao DOM-refactor inner-item / top-item wrappers and the flow-markdown-body assistant fallback', () => {
const turnsScript = __test__.getTurnsScript();
// New wrappers added to itemSelectors so message roots resolve under the
// refactored DOM where the legacy item-kDun2N / union_message / message-block-container
// / data-message-id selectors no longer match.
expect(turnsScript).toContain('[class*="inner-item-"]');
expect(turnsScript).toContain('[class*="top-item-"]');
// Assistant fallback: post-refactor doubao no longer emits receive-message /
// bg-g-receive-msg-bubble markup. Only signal is .flow-markdown-body content
// container without send-bubble.
expect(turnsScript).toContain('.flow-markdown-body');
});
it('extracts clean assistant turns from the 2026-05 wrapper DOM without using whole-page chrome', () => {
const turns = runTurnsScript(`
<main>
<aside>历史对话</aside>
<section class="message-list-S2Fv2S">
<div class="top-item-user">
<div class="inner-item-user">
<div class="bg-g-send-msg-bubble">测试一下,只回复OK</div>
</div>
</div>
<div class="top-item-assistant">
<div class="inner-item-assistant">
<div class="flow-markdown-body"><p>OK</p></div>
</div>
</div>
</section>
</main>
`);
expect(turns).toEqual([
{ Role: 'User', Text: '测试一下,只回复OK' },
{ Role: 'Assistant', Text: 'OK' },
]);
});
it('extends transcript-noise cleanup for the current zh-CN chrome copy', () => {
const transcriptScript = __test__.getTranscriptLinesScript();
expect(transcriptScript).toContain('请仔细甄别');
+50 -14
View File
@@ -1,4 +1,11 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { unwrapEvaluateResult } from './evaluate-result.js';
function isAuthLikeError(code, message) {
const text = String(message ?? '');
return code === 401 || code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(text);
}
/**
* Execute a fetch() call inside the Chrome browser context via page.evaluate.
* This ensures a_bogus signing and cookies are handled automatically by the browser.
@@ -6,24 +13,53 @@ import { CommandExecutionError } from '@jackwener/opencli/errors';
export async function browserFetch(page, method, url, options = {}) {
const js = `
(async () => {
const res = await fetch(${JSON.stringify(url)}, {
method: ${JSON.stringify(method)},
credentials: 'include',
headers: {
'Content-Type': 'application/json',
...${JSON.stringify(options.headers ?? {})}
},
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
});
return res.json();
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), ${Number(options.timeoutMs ?? 30000)});
try {
const res = await fetch(${JSON.stringify(url)}, {
method: ${JSON.stringify(method)},
credentials: 'include',
signal: controller.signal,
headers: {
'Content-Type': 'application/json',
...${JSON.stringify(options.headers ?? {})}
},
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
});
const text = await res.text();
try {
return JSON.parse(text);
} catch (error) {
return { status_code: res.ok ? -2 : res.status, status_msg: \`JSON parse failed: \${text.slice(0, 500) || String(error && error.message || error)}\` };
}
} catch (error) {
return { status_code: -1, status_msg: String(error && error.message || error) };
} finally {
clearTimeout(timer);
}
})()
`;
const result = await page.evaluate(js);
let result;
try {
result = unwrapEvaluateResult(await page.evaluate(js));
}
catch (error) {
throw new CommandExecutionError(`Douyin API request failed (${method} ${url}): ${error instanceof Error ? error.message : String(error)}`);
}
if (result == null) {
throw new CommandExecutionError(`Empty response from Douyin API (${method} ${url})`);
}
if (Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`Malformed response from Douyin API (${method} ${url})`);
}
if (result && typeof result === 'object' && 'status_code' in result) {
const code = result.status_code;
if (code !== 0) {
const msg = result.status_msg ?? 'unknown error';
throw new CommandExecutionError(`Douyin API error ${code}: ${msg}`);
const msg = result.status_msg ?? result.message ?? 'unknown error';
if (isAuthLikeError(code, msg)) {
throw new AuthRequiredError('creator.douyin.com', `Douyin API auth/permission error ${code} at ${method} ${url}: ${msg}`);
}
throw new CommandExecutionError(`Douyin API error ${code} at ${method} ${url}: ${msg}`);
}
}
return result;
+34
View File
@@ -1,4 +1,5 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { browserFetch } from './browser-fetch.js';
function makePage(result) {
return {
@@ -18,13 +19,46 @@ describe('browserFetch', () => {
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
expect(result).toEqual({ status_code: 0, data: { ak: 'KEY' } });
});
it('unwraps Browser Bridge {session,data} envelopes', async () => {
const page = makePage({ session: 'site:douyin:test', data: { status_code: 0, data: { ok: true } } });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.resolves.toEqual({ status_code: 0, data: { ok: true } });
});
it('throws when status_code is non-zero', async () => {
const page = makePage({ status_code: 8, message: 'fail' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API error 8');
});
it('maps auth-like API errors to AuthRequiredError', async () => {
const page = makePage({ status_code: 401, status_msg: 'login required' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toBeInstanceOf(AuthRequiredError);
});
it('returns result even when no status_code field', async () => {
const page = makePage({ some_field: 'value' });
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
expect(result).toEqual({ some_field: 'value' });
});
it('throws on empty response body (null from evaluate)', async () => {
const page = makePage(null);
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Empty response from Douyin API');
});
it('throws on undefined response body', async () => {
const page = makePage(undefined);
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Empty response from Douyin API');
});
it('throws typed on malformed primitive response body', async () => {
const page = makePage('not-json-object');
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws typed when browser fetch returns a non-JSON body', async () => {
const page = makePage({ status_code: -2, status_msg: 'JSON parse failed: <html>not-json</html>' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toThrow('Douyin API error -2');
});
it('wraps browser-side fetch or JSON parse failures', async () => {
const page = makePage(null);
page.evaluate.mockRejectedValueOnce(new SyntaxError('Unexpected token < in JSON'));
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API request failed (GET https://creator.douyin.com/api/test): Unexpected token < in JSON');
});
});
+16
View File
@@ -0,0 +1,16 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
export function unwrapEvaluateResult(payload) {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
}
export function requireObjectEvaluateResult(payload, context) {
const result = unwrapEvaluateResult(payload);
if (!result || Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`${context}: malformed evaluate payload`);
}
return result;
}
+107 -71
View File
@@ -56,6 +56,31 @@ function sha256Hex(data) {
}
return hash.digest('hex');
}
const CRC32_TABLE = new Uint32Array(256).map((_, index) => {
let value = index;
for (let bit = 0; bit < 8; bit += 1) {
value = (value & 1) ? (0xEDB88320 ^ (value >>> 1)) : (value >>> 1);
}
return value >>> 0;
});
function crc32Hex(data) {
let crc = 0xffffffff;
for (const byte of data) {
crc = CRC32_TABLE[(crc ^ byte) & 0xff] ^ (crc >>> 8);
}
return ((crc ^ 0xffffffff) >>> 0).toString(16).padStart(8, '0');
}
function gatewayBaseUrl(tosUrl) {
const parsedUrl = new URL(tosUrl);
return `https://${parsedUrl.host}/upload/v1${parsedUrl.pathname}`;
}
function gatewayHeaders(auth, uploadHeader, userId = '') {
return {
Authorization: auth,
'X-Storage-U': encodeURIComponent(userId),
...(uploadHeader ?? {}),
};
}
function extractRegionFromHost(host) {
// e.g. "tos-cn-i-alisg.volces.com" → "cn-i-alisg"
// e.g. "tos-cn-beijing.ivolces.com" → "cn-beijing"
@@ -129,6 +154,7 @@ async function tosRequest(opts) {
method,
headers,
body: fetchBody,
signal: AbortSignal.timeout(60000),
});
const responseBody = await res.text();
const responseHeaders = {};
@@ -140,86 +166,95 @@ async function tosRequest(opts) {
function nowDatetime() {
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
}
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
async function initMultipartUpload(tosUrl, auth, credentials) {
const initUrl = `${tosUrl}?uploads`;
const datetime = nowDatetime();
// Use the pre-computed auth for INIT, as it comes from ApplyVideoUpload
const headers = {
Authorization: auth,
'x-amz-date': datetime,
'x-amz-security-token': credentials.session_token,
'content-type': 'application/octet-stream',
};
const res = await tosRequest({ method: 'POST', url: initUrl, headers });
if (res.status !== 200) {
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS credentials are valid and not expired.');
function extractUploadId(body) {
const xmlMatch = body.match(/<UploadId>([^<]+)<\/UploadId>/i);
if (xmlMatch) return xmlMatch[1];
try {
const json = JSON.parse(body);
return json?.payload?.uploadID
|| json?.payload?.uploadId
|| json?.payload?.UploadID
|| json?.payload?.UploadId
|| json?.data?.uploadid
|| json?.data?.uploadID
|| json?.data?.uploadId
|| json?.data?.UploadID
|| json?.data?.UploadId
|| json?.UploadID
|| json?.UploadId
|| json?.uploadID
|| json?.uploadId
|| null;
}
// Parse UploadId from XML: <UploadId>...</UploadId>
const match = res.body.match(/<UploadId>([^<]+)<\/UploadId>/);
if (!match) {
catch {
return null;
}
}
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
async function initMultipartUpload(tosUrl, auth, uploadHeader, userId) {
const initUrl = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=init`;
const res = await tosRequest({
method: 'POST',
url: initUrl,
headers: gatewayHeaders(auth, uploadHeader, userId),
});
if (res.status !== 200) {
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
}
const uploadId = extractUploadId(res.body);
if (!uploadId) {
throw new CommandExecutionError(`TOS init response missing UploadId: ${res.body}`);
}
return match[1];
return uploadId;
}
// ── Phase 2: Upload a single part ────────────────────────────────────────────
async function uploadPart(tosUrl, partNumber, uploadId, data, credentials, region) {
const parsedUrl = new URL(tosUrl);
parsedUrl.searchParams.set('partNumber', String(partNumber));
parsedUrl.searchParams.set('uploadId', uploadId);
const url = parsedUrl.toString();
const datetime = nowDatetime();
const headers = computeAws4Headers({
method: 'PUT',
url,
headers: { 'content-type': 'application/octet-stream' },
body: data,
credentials,
service: 'tos',
region,
datetime,
});
const res = await tosRequest({ method: 'PUT', url, headers, body: data });
if (res.status !== 200) {
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that STS2 credentials are valid and not expired.');
async function uploadPart(tosUrl, partNumber, uploadId, data, auth, uploadHeader, userId) {
const crc32 = crc32Hex(data);
const url = `${gatewayBaseUrl(tosUrl)}?uploadid=${encodeURIComponent(uploadId)}&part_number=${partNumber}&phase=transfer`;
const headers = {
...gatewayHeaders(auth, uploadHeader, userId),
'Content-CRC32': crc32,
'Content-Type': 'application/octet-stream',
'X-Use-Init-Upload-Optimize': '1',
'X-Use-Large-Local-Cache': '1',
};
const res = await tosRequest({ method: 'POST', url, headers, body: data });
let parsed;
try {
parsed = JSON.parse(res.body);
}
const etag = res.headers['etag'];
if (!etag) {
throw new CommandExecutionError(`TOS upload part ${partNumber} response missing ETag header`);
catch {
parsed = null;
}
return etag;
if (res.status !== 200 || parsed?.code !== 2000) {
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
}
return parsed?.data?.crc32 || crc32;
}
// ── Phase 3: Complete multipart upload ───────────────────────────────────────
async function completeMultipartUpload(tosUrl, uploadId, parts, credentials, region) {
const parsedUrl = new URL(tosUrl);
parsedUrl.searchParams.set('uploadId', uploadId);
const url = parsedUrl.toString();
const xmlBody = '<CompleteMultipartUpload>' +
parts
.sort((a, b) => a.partNumber - b.partNumber)
.map(p => `<Part><PartNumber>${p.partNumber}</PartNumber><ETag>${p.etag}</ETag></Part>`)
.join('') +
'</CompleteMultipartUpload>';
const datetime = nowDatetime();
const headers = computeAws4Headers({
method: 'POST',
url,
headers: { 'content-type': 'application/xml' },
body: xmlBody,
credentials,
service: 'tos',
region,
datetime,
});
async function completeMultipartUpload(tosUrl, uploadId, parts, auth, uploadHeader, userId) {
const url = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=finish&uploadid=${encodeURIComponent(uploadId)}`;
const body = parts
.sort((a, b) => a.partNumber - b.partNumber)
.map(p => `${p.partNumber}:${p.crc32}`)
.join(',');
const res = await tosRequest({
method: 'POST',
url,
headers,
body: xmlBody,
headers: gatewayHeaders(auth, uploadHeader, userId),
body,
});
if (res.status !== 200) {
let parsed;
try {
parsed = JSON.parse(res.body);
}
catch {
parsed = null;
}
if (res.status !== 200 || parsed?.code !== 2000) {
throw new CommandExecutionError(`TOS complete multipart upload failed with status ${res.status}: ${res.body}`, 'Check that all parts were uploaded successfully.');
}
return parsed?.data?.key || null;
}
let _readSyncOverride = null;
/** @internal — for testing only */
@@ -237,7 +272,7 @@ export async function tosUpload(options) {
if (fileSize === 0) {
throw new CommandExecutionError(`Video file is empty: ${filePath}`);
}
const { tos_upload_url: tosUrl, auth } = uploadInfo;
const { tos_upload_url: tosUrl, auth, upload_header: uploadHeader, user_id: userId } = uploadInfo;
const parsedTosUrl = new URL(tosUrl);
const region = extractRegionFromHost(parsedTosUrl.host);
const resumePath = getResumeFilePath(filePath);
@@ -251,7 +286,7 @@ export async function tosUpload(options) {
}
else {
// Start fresh
uploadId = await initMultipartUpload(tosUrl, auth, credentials);
uploadId = await initMultipartUpload(tosUrl, auth, uploadHeader, userId);
completedParts = [];
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
}
@@ -277,8 +312,8 @@ export async function tosUpload(options) {
if (bytesRead !== chunkSize) {
throw new CommandExecutionError(`Short read on part ${partNumber}: expected ${chunkSize} bytes, got ${bytesRead}`);
}
const etag = await uploadPart(tosUrl, partNumber, uploadId, buffer, credentials, region);
completedParts.push({ partNumber, etag });
const crc32 = await uploadPart(tosUrl, partNumber, uploadId, buffer, auth, uploadHeader, userId);
completedParts.push({ partNumber, crc32 });
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
uploadedBytes = Math.min(offset + chunkSize, fileSize);
if (onProgress)
@@ -288,8 +323,9 @@ export async function tosUpload(options) {
finally {
fs.closeSync(fd);
}
await completeMultipartUpload(tosUrl, uploadId, completedParts, credentials, region);
const completedKey = await completeMultipartUpload(tosUrl, uploadId, completedParts, auth, uploadHeader, userId);
deleteResumeState(resumePath);
return completedKey;
}
// ── Internal exports for testing ─────────────────────────────────────────────
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, };
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, extractUploadId, crc32Hex, gatewayBaseUrl, gatewayHeaders, };
+212
View File
@@ -0,0 +1,212 @@
import * as crypto from 'node:crypto';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { unwrapEvaluateResult } from './evaluate-result.js';
const AUTH_V5_URL = 'https://creator.douyin.com/web/api/media/upload/auth/v5/';
const VOD_UPLOAD_HOST = 'https://vod.bytedanceapi.com/';
const VOD_SPACE_NAME = 'aweme';
function hmacSha256(key, data) {
return crypto.createHmac('sha256', key).update(data, 'utf8').digest();
}
function sha256Hex(data) {
const hash = crypto.createHash('sha256');
if (Buffer.isBuffer(data) || data instanceof Uint8Array) {
hash.update(data);
} else {
hash.update(data ?? '', 'utf8');
}
return hash.digest('hex');
}
function nowDatetime() {
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
}
function canonicalQuery(url) {
return [...url.searchParams.entries()]
.sort(([a], [b]) => a.localeCompare(b))
.map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
.join('&');
}
function computeAws4Headers(url, credentials, options = {}) {
const parsedUrl = new URL(url);
const datetime = nowDatetime();
const date = datetime.slice(0, 8);
const method = options.method ?? 'GET';
const body = options.body ?? '';
const bodyHash = sha256Hex(body);
const headers = {
...(options.headers ?? {}),
host: parsedUrl.host,
'x-amz-content-sha256': bodyHash,
'x-amz-date': datetime,
'x-amz-security-token': credentials.session_token,
};
const sortedHeaderKeys = Object.keys(headers).sort((a, b) => a.localeCompare(b));
const canonicalHeaders = sortedHeaderKeys
.map((key) => `${key}:${String(headers[key]).trim()}`)
.join('\n') + '\n';
const signedHeaders = sortedHeaderKeys.join(';');
const canonicalRequest = [
method,
parsedUrl.pathname || '/',
canonicalQuery(parsedUrl),
canonicalHeaders,
signedHeaders,
bodyHash,
].join('\n');
const service = 'vod';
const region = 'cn-north-1';
const credentialScope = `${date}/${region}/${service}/aws4_request`;
const stringToSign = [
'AWS4-HMAC-SHA256',
datetime,
credentialScope,
sha256Hex(canonicalRequest),
].join('\n');
const kDate = hmacSha256(`AWS4${credentials.secret_access_key}`, date);
const kRegion = hmacSha256(kDate, region);
const kService = hmacSha256(kRegion, service);
const kSigning = hmacSha256(kService, 'aws4_request');
const signature = hmacSha256(kSigning, stringToSign).toString('hex');
return {
...headers,
Authorization: `AWS4-HMAC-SHA256 Credential=${credentials.access_key_id}/${credentialScope}, SignedHeaders=${signedHeaders}, Signature=${signature}`,
};
}
function extractUserIdFromSessionToken(sessionToken) {
try {
const raw = sessionToken.startsWith('STS2') ? sessionToken.slice(4) : sessionToken;
const decoded = JSON.parse(Buffer.from(raw, 'base64').toString('utf8'));
const policy = JSON.parse(decoded.PolicyString || '{}');
const condition = policy?.Statement?.[0]?.Condition;
if (typeof condition === 'string') {
const parsedCondition = JSON.parse(condition);
return parsedCondition.UserId || '';
}
} catch {
return '';
}
return '';
}
export async function getUploadAuthV5Credentials(page) {
const result = unwrapEvaluateResult(await page.evaluate(`fetch(${JSON.stringify(AUTH_V5_URL)}, { credentials: 'include' }).then(r => r.json())`));
if (!result || Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
if (result.status_code !== 0) {
const message = result.status_msg ?? result.message ?? 'unknown error';
if (result.status_code === 401 || result.status_code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(String(message))) {
throw new AuthRequiredError('creator.douyin.com', `获取抖音上传授权失败: ${message}`);
}
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
if (!result.auth) {
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
let auth;
try {
auth = JSON.parse(result.auth);
} catch (error) {
throw new CommandExecutionError(`解析抖音上传授权失败: ${error instanceof Error ? error.message : String(error)}`);
}
if (!auth.AccessKeyID || !auth.SecretAccessKey || !auth.SessionToken) {
throw new CommandExecutionError('抖音上传授权缺少 AccessKeyID/SecretAccessKey/SessionToken');
}
return {
access_key_id: auth.AccessKeyID,
secret_access_key: auth.SecretAccessKey,
session_token: auth.SessionToken,
user_id: extractUserIdFromSessionToken(auth.SessionToken),
expired_time: auth.ExpiredTime,
current_time: auth.CurrentTime,
};
}
export async function applyVideoUploadInner(fileSize, credentials) {
const params = new URLSearchParams({
Action: 'ApplyUploadInner',
Version: '2020-11-19',
SpaceName: VOD_SPACE_NAME,
FileType: 'video',
IsInner: '1',
FileSize: String(fileSize),
});
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
const res = await fetch(url, { headers: computeAws4Headers(url, credentials), signal: AbortSignal.timeout(30000) });
const text = await res.text();
let payload;
try {
payload = JSON.parse(text);
} catch {
throw new CommandExecutionError(`申请抖音上传地址失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
}
const error = payload?.ResponseMetadata?.Error;
if (!res.ok || error) {
throw new CommandExecutionError(`申请抖音上传地址失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
}
const uploadNode = payload?.Result?.InnerUploadAddress?.UploadNodes?.[0];
const storeInfo = uploadNode?.StoreInfos?.[0];
const videoId = payload?.Result?.Vid || uploadNode?.Vid;
const sessionKey = uploadNode?.SessionKey ?? storeInfo?.SessionKey ?? payload?.Result?.SessionKey;
if (!uploadNode?.UploadHost || !storeInfo?.StoreUri || !storeInfo?.Auth || !videoId || !sessionKey) {
throw new CommandExecutionError(`申请抖音上传地址响应缺少必要字段: ${JSON.stringify(payload).slice(0, 500)}`);
}
return {
video_id: videoId,
tos_upload_url: `https://${uploadNode.UploadHost}/${storeInfo.StoreUri}`,
auth: storeInfo.Auth,
session_key: sessionKey,
upload_header: uploadNode.UploadHeader ?? {},
user_id: credentials.user_id ?? '',
};
}
export async function commitVideoUploadInner(uploadInfo, credentials) {
if (!uploadInfo?.session_key) {
throw new CommandExecutionError('抖音上传提交缺少 SessionKey');
}
const params = new URLSearchParams({
Action: 'CommitUploadInner',
Version: '2020-11-19',
SpaceName: VOD_SPACE_NAME,
});
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
const body = JSON.stringify({ SessionKey: uploadInfo.session_key });
const headers = computeAws4Headers(url, credentials, {
method: 'POST',
body,
headers: { 'content-type': 'application/json;charset=UTF-8' },
});
const res = await fetch(url, { method: 'POST', headers, body, signal: AbortSignal.timeout(30000) });
const text = await res.text();
let payload;
try {
payload = JSON.parse(text);
} catch {
throw new CommandExecutionError(`提交抖音上传失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
}
const error = payload?.ResponseMetadata?.Error;
if (!res.ok || error) {
throw new CommandExecutionError(`提交抖音上传失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
}
const result = payload?.Result?.Results?.[0] ?? payload?.Result ?? {};
const videoId = result.Vid ?? result.VideoId ?? result.VideoID ?? result.vid ?? uploadInfo.video_id;
if (!videoId) {
throw new CommandExecutionError(`提交抖音上传响应缺少 video id: ${JSON.stringify(payload).slice(0, 500)}`);
}
const meta = result.Meta ?? result.VideoMeta ?? {};
return {
video_id: videoId,
poster_uri: result.PosterUri ?? result.PosterURI ?? result.SnapshotUri ?? result.SnapshotURI ?? '',
width: Number(meta.Width ?? meta.width ?? 720) || 720,
height: Number(meta.Height ?? meta.height ?? 1280) || 1280,
raw: result,
};
}
+38
View File
@@ -0,0 +1,38 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getUploadAuthV5Credentials, applyVideoUploadInner } from './vod-upload.js';
describe('douyin vod upload helpers', () => {
it('parses creator upload auth v5 credentials', async () => {
const page = { evaluate: async () => ({ status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token', ExpiredTime: 123, CurrentTime: 100 }) }) };
await expect(getUploadAuthV5Credentials(page)).resolves.toEqual({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token', user_id: '', expired_time: 123, current_time: 100 });
});
it('unwraps browser bridge envelopes around upload auth payloads', async () => {
const payload = { status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token' }) };
const page = { evaluate: async () => ({ session: 'site:douyin:test', data: payload }) };
await expect(getUploadAuthV5Credentials(page)).resolves.toMatchObject({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
});
it('maps upload auth permission errors to AuthRequiredError', async () => {
const page = { evaluate: async () => ({ status_code: 401, status_msg: 'login required' }) };
await expect(getUploadAuthV5Credentials(page)).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps ApplyUploadInner response to TOS upload info', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { RequestId: 'req' }, Result: { InnerUploadAddress: { UploadNodes: [{ Vid: 'video-id', SessionKey: 'session-key', UploadHost: 'tos.example.com', StoreInfos: [{ StoreUri: 'obj/key.mp4', Auth: 'space-auth' }] }] } } }) });
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).resolves.toEqual({ video_id: 'video-id', tos_upload_url: 'https://tos.example.com/obj/key.mp4', auth: 'space-auth', session_key: 'session-key', upload_header: {}, user_id: '' });
const [url, init] = fetchSpy.mock.calls[0];
expect(String(url)).toContain('Action=ApplyUploadInner');
expect(String(url)).toContain('Version=2020-11-19');
expect(init.headers.Authorization).toContain('AWS4-HMAC-SHA256 Credential=ak/');
expect(init.headers['x-amz-security-token']).toBe('token');
fetchSpy.mockRestore();
});
it('surfaces VOD API errors with context', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { Error: { Code: 'AccessDenied', Message: 'denied' } } }) });
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).rejects.toBeInstanceOf(CommandExecutionError);
fetchSpy.mockRestore();
});
});
+137 -4
View File
@@ -1,19 +1,152 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { browserFetch } from './_shared/browser-fetch.js';
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
const CREATOR_MANAGE_URL = 'https://creator.douyin.com/creator-micro/content/manage';
const WORK_LIST_URL = '/janus/douyin/creator/pc/work_list?status=0&count=20&max_cursor=0&scene=star_atlas&device_platform=android&aid=1128';
function readAwemeId(raw) {
const value = String(raw ?? '').trim();
if (!value) {
throw new ArgumentError('douyin delete aweme_id cannot be empty');
}
if (!/^\d+$/.test(value)) {
throw new ArgumentError('douyin delete aweme_id must be a numeric id');
}
return value;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function deleteViaCreatorManage(page, workId) {
await page.goto(CREATOR_MANAGE_URL);
await sleep(3000);
await sleep(3000);
const result = requireObjectEvaluateResult(await page.evaluate(`
(async () => {
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const targetId = ${JSON.stringify(String(workId))};
const textOf = (node) => (node && (node.innerText || node.textContent) || '').trim();
const normalize = (value) => String(value || '').replace(/\\s+/g, ' ').trim();
async function loadTarget() {
const res = await fetch(${JSON.stringify(WORK_LIST_URL)}, { credentials: 'include' });
const payload = await res.json();
const list = Array.isArray(payload.aweme_list) ? payload.aweme_list : [];
const matches = list
.map((entry, index) => ({ entry, index }))
.filter(({ entry }) => String(entry.aweme_id || '') === targetId || String(entry.item_id || '') === targetId);
if (matches.length === 0) {
return { ok: false, reason: 'not_found', status_code: payload.status_code, count: list.length };
}
if (matches.length !== 1) {
return { ok: false, reason: 'target_not_unique', count: matches.length };
}
const { entry: item, index } = matches[0];
const title = normalize(item.desc || item.caption || item.title || item.item_title || '');
return { ok: true, item, index, listCount: list.length, title };
}
function visibleWorkCards() {
const candidates = Array.from(document.querySelectorAll('[class*="video-card"]'))
.filter((element) => {
const text = normalize(textOf(element));
return text.includes('删除作品') && text.includes('继续编辑');
});
return candidates.filter((candidate) => !candidates.some((other) => other !== candidate && other.contains(candidate)));
}
const target = await loadTarget();
if (!target.ok) return target;
const allTab = Array.from(document.querySelectorAll('button,[role="button"],span,div'))
.find((element) => /^全部作品$/.test(normalize(textOf(element))));
allTab?.click();
await sleep(1000);
for (let attempt = 0; attempt < 20; attempt += 1) {
const cards = visibleWorkCards();
if (cards.length >= target.listCount && cards[target.index]) {
const card = cards[target.index];
const deleteButton = Array.from(card.querySelectorAll('button,[role="button"],span,div'))
.find((element) => /^删除作品$/.test(normalize(textOf(element))));
if (!deleteButton) return { ok: false, reason: 'delete_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, cardCount: cards.length };
deleteButton.click();
await sleep(800);
const confirmButton = Array.from(document.querySelectorAll('button,[role="button"]'))
.find((element) => ['确定', '确认', '删除'].includes(normalize(textOf(element))));
if (!confirmButton) return { ok: false, reason: 'confirm_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
confirmButton.click();
for (let wait = 0; wait < 20; wait += 1) {
await sleep(500);
const after = await loadTarget();
if (!after.ok && after.reason === 'not_found') {
return { ok: true, aweme_id: target.item.aweme_id, item_id: target.item.item_id, title: target.title };
}
}
return { ok: false, reason: 'delete_not_confirmed', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
}
await sleep(500);
}
return { ok: false, reason: 'card_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, listCount: target.listCount };
})()
`), '抖音后台管理删除响应异常');
if (!result?.ok) {
throw new CommandExecutionError(`抖音后台管理删除失败: ${JSON.stringify(result)}`);
}
return result;
}
async function findWorkListItem(page, workId) {
const data = await browserFetch(page, 'GET', `https://creator.douyin.com${WORK_LIST_URL}`, { timeoutMs: 8000 });
const list = data.data?.work_list ?? data.aweme_list ?? data.work_list ?? [];
if (!Array.isArray(list)) {
throw new CommandExecutionError('抖音作品列表响应缺少 work_list/aweme_list');
}
return list.find((entry) => String(entry.aweme_id || '') === workId || String(entry.item_id || '') === workId) || null;
}
cli({
site: 'douyin',
name: 'delete',
access: 'write',
description: '删除作品',
description: '删除作品(优先使用创作者后台作品管理;找不到时回退到旧删除接口)',
domain: 'creator.douyin.com',
strategy: Strategy.COOKIE,
siteSession: 'persistent',
args: [
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID' },
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID / item_id' },
],
columns: ['status'],
func: async (page, kwargs) => {
const awemeId = readAwemeId(kwargs.aweme_id);
try {
const deleted = await deleteViaCreatorManage(page, awemeId);
return [{ status: `✅ 已通过后台管理删除 ${deleted.aweme_id || awemeId}` }];
} catch (fallbackError) {
const fallbackMessage = fallbackError instanceof Error ? fallbackError.message : String(fallbackError);
if (!fallbackMessage.includes('"reason":"not_found"')) {
throw fallbackError;
}
}
const before = await findWorkListItem(page, awemeId);
if (!before) {
throw new CommandExecutionError(`抖音作品 ${awemeId} 未在作品列表中找到,未执行删除`);
}
const url = 'https://creator.douyin.com/web/api/media/aweme/delete/?aid=1128';
await browserFetch(page, 'POST', url, { body: { aweme_id: kwargs.aweme_id } });
return [{ status: `✅ 已删除 ${kwargs.aweme_id}` }];
await browserFetch(page, 'POST', url, { body: { aweme_id: awemeId }, timeoutMs: 8000 });
const deadline = Date.now() + 10_000;
while (Date.now() < deadline) {
await sleep(500);
const after = await findWorkListItem(page, awemeId);
if (!after) {
return [{ status: `✅ 已删除 ${awemeId}` }];
}
}
throw new CommandExecutionError(`抖音作品 ${awemeId} 删除后仍在作品列表中,删除未确认`);
},
});
+90 -1
View File
@@ -1,11 +1,100 @@
import { describe, expect, it } from 'vitest';
import { readFileSync } from 'node:fs';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
const mocks = vi.hoisted(() => ({
browserFetch: vi.fn(),
}));
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
import './delete.js';
function makePage({ evaluateResult, listBefore = [], listAfter = [] } = {}) {
let listCalls = 0;
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'GET' && String(url).includes('/work_list?')) {
listCalls += 1;
return { aweme_list: listCalls === 1 ? listBefore : listAfter };
}
return { status_code: 0 };
});
return {
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult ?? { ok: false, reason: 'not_found' }),
wait: vi.fn().mockResolvedValue(undefined),
};
}
describe('douyin delete registration', () => {
const command = getRegistry().get('douyin/delete');
beforeEach(() => {
vi.clearAllMocks();
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it('registers the delete command', () => {
const registry = getRegistry();
const values = [...registry.values()];
const cmd = values.find(c => c.site === 'douyin' && c.name === 'delete');
expect(cmd).toBeDefined();
});
it('uses work_list id/index matching instead of title matching for fallback deletion', () => {
const source = readFileSync(new URL('./delete.js', import.meta.url), 'utf8');
expect(source).toContain('target_not_unique');
expect(source).toContain("String(entry.aweme_id || '') === targetId");
expect(source).toContain('cards[target.index]');
expect(source).not.toContain('text.includes(target.title)');
});
it('validates aweme_id before navigation', async () => {
const page = makePage();
await expect(command.func(page, { aweme_id: '' })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { aweme_id: 'abc' })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('does not treat a missing work as successful delete', async () => {
const page = makePage({ listBefore: [], listAfter: [] });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
});
it('unwraps Browser Bridge envelopes around creator manage delete results', async () => {
const page = makePage({ evaluateResult: { session: 'site:douyin:test', data: { ok: true, aweme_id: '123' } } });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已通过后台管理删除 123' }]);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
expect(mocks.browserFetch).not.toHaveBeenCalled();
});
it('throws typed on malformed creator manage delete result', async () => {
const page = makePage({ evaluateResult: 'bad-shape' });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
expect(mocks.browserFetch).not.toHaveBeenCalled();
});
it('returns success only after fallback delete postcondition removes the target', async () => {
const page = makePage({
listBefore: [{ aweme_id: '123' }],
listAfter: [],
});
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已删除 123' }]);
await vi.advanceTimersByTimeAsync(8000);
await assertion;
});
});
+170
View File
@@ -0,0 +1,170 @@
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { beforeEach, describe, expect, it, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
browserFetch: vi.fn(),
getUploadAuthV5Credentials: vi.fn(),
applyVideoUploadInner: vi.fn(),
commitVideoUploadInner: vi.fn(),
tosUpload: vi.fn(),
pollTranscode: vi.fn(),
imagexUpload: vi.fn(),
}));
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
vi.mock('./_shared/vod-upload.js', () => ({
getUploadAuthV5Credentials: mocks.getUploadAuthV5Credentials,
applyVideoUploadInner: mocks.applyVideoUploadInner,
commitVideoUploadInner: mocks.commitVideoUploadInner,
}));
vi.mock('./_shared/tos-upload.js', () => ({ tosUpload: mocks.tosUpload }));
vi.mock('./_shared/transcode.js', () => ({ pollTranscode: mocks.pollTranscode }));
vi.mock('./_shared/imagex-upload.js', () => ({ imagexUpload: mocks.imagexUpload }));
describe('douyin publish upload identifier handling', () => {
beforeEach(() => {
vi.resetModules();
vi.clearAllMocks();
mocks.getUploadAuthV5Credentials.mockResolvedValue({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
mocks.applyVideoUploadInner.mockResolvedValue({ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' });
mocks.commitVideoUploadInner.mockResolvedValue({ video_id: 'canonical-video-id', poster_uri: 'poster-uri' });
mocks.tosUpload.mockResolvedValue('object-key-returned-by-complete');
mocks.pollTranscode.mockResolvedValue({ width: 720, height: 1280, poster_uri: 'poster-uri' });
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { aweme_id: 'aweme-1' };
return { status_code: 0 };
});
});
it('uses CommitUploadInner Vid for create_v2, not the completed TOS object key', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-id-'));
const video = path.join(tmpDir, 'video.mp4');
fs.writeFileSync(video, Buffer.from('fake-video'));
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func({}, {
video,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
});
expect(mocks.commitVideoUploadInner).toHaveBeenCalledWith(
{ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' },
{ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' },
);
expect(mocks.pollTranscode).not.toHaveBeenCalled();
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
expect(createCall?.[3]?.body.item.common.video_id).toBe('canonical-video-id');
expect(createCall?.[3]?.body.item.common.video_id).not.toBe('object-key-returned-by-complete');
});
it('continues to create_v2 when the legacy fast detect API returns an empty response', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-safety-'));
const video = path.join(tmpDir, 'video.mp4');
fs.writeFileSync(video, Buffer.from('fake-video'));
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/pre_check')) {
throw new Error('Empty response from Douyin API (POST https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check)');
}
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/poll')) return { status: -1, has_done: true, detect_result: { reason_code: 0 }, detect_list: [] };
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { item_id: 'item-1' };
return { status_code: 0 };
});
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func({}, {
video,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
visibility: 'public',
caption: 'caption',
no_safety_check: false,
});
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/post_assistant/fast_detect/pre_check'))).toBe(true);
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/aweme/create_v2/'))).toBe(true);
});
it('unwraps Browser Bridge envelopes around cover ImageX evaluate results', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-'));
const video = path.join(tmpDir, 'video.mp4');
const cover = path.join(tmpDir, 'cover.jpg');
fs.writeFileSync(video, Buffer.from('fake-video'));
fs.writeFileSync(cover, Buffer.from('fake-cover'));
mocks.imagexUpload.mockResolvedValue('cover-store-uri');
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce({
session: 'site:douyin:test',
data: { Result: { UploadAddress: { StoreInfos: [{ UploadHost: 'imagex.example.com', StoreUri: 'cover/key.jpg' }] } } },
})
.mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: {} } }),
};
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func(page, {
video,
cover,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
});
expect(mocks.imagexUpload).toHaveBeenCalledWith(cover, {
upload_url: 'https://imagex.example.com/cover/key.jpg',
store_uri: 'cover/key.jpg',
});
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
expect(createCall?.[3]?.body.item.cover.poster).toBe('cover-store-uri');
});
it('throws typed when cover ImageX apply returns the wrong shape', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-bad-'));
const video = path.join(tmpDir, 'video.mp4');
const cover = path.join(tmpDir, 'cover.jpg');
fs.writeFileSync(video, Buffer.from('fake-video'));
fs.writeFileSync(cover, Buffer.from('fake-cover'));
const page = { evaluate: vi.fn().mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: { UploadAddress: { StoreInfos: [] } } } }) };
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await expect(cmd.func(page, {
video,
cover,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
})).rejects.toThrow('UploadHost/StoreUri');
expect(mocks.imagexUpload).not.toHaveBeenCalled();
});
});
+88 -42
View File
@@ -2,7 +2,7 @@
* Douyin publish — 8-phase pipeline for scheduling video posts.
*
* Phases:
* 1. STS2 credentials
* 1. upload auth v5 credentials
* 2. Apply TOS upload URL
* 3. TOS multipart upload
* 4. Cover upload (optional, via ImageX)
@@ -15,11 +15,11 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getSts2Credentials } from './_shared/sts2.js';
import { getUploadAuthV5Credentials, applyVideoUploadInner, commitVideoUploadInner } from './_shared/vod-upload.js';
import { tosUpload } from './_shared/tos-upload.js';
import { imagexUpload } from './_shared/imagex-upload.js';
import { pollTranscode } from './_shared/transcode.js';
import { browserFetch } from './_shared/browser-fetch.js';
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
import { generateCreationId } from './_shared/creation-id.js';
import { validateTiming, toUnixSeconds } from './_shared/timing.js';
import { parseTextExtra, extractHashtagNames } from './_shared/text-extra.js';
@@ -54,6 +54,36 @@ const DEFAULT_COVER_TOOLS_INFO = JSON.stringify({
initial_cover_uri: '',
cut_coordinate: '',
});
function isFastDetectRetryable(error) {
const message = error instanceof Error ? error.message : String(error);
return message.includes('post_assistant/fast_detect') && (message.includes('Empty response') || message.includes('404') || message.includes('Not Found') || message.includes('Timeout') || message.includes('timed out') || message.includes('Failed to fetch'));
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function throwIfImagexError(action, payload) {
const error = payload?.ResponseMetadata?.Error ?? payload?.Error;
if (error) {
throw new CommandExecutionError(`${action}失败: ${JSON.stringify(error)}`);
}
}
async function tryFastDetectFetch(page, method, url, options) {
let lastError;
for (let attempt = 1; attempt <= 3; attempt += 1) {
try {
return { ok: true, value: await browserFetch(page, method, url, options) };
} catch (error) {
if (!isFastDetectRetryable(error)) {
throw error;
}
lastError = error;
if (attempt < 3) {
await sleep(500 * attempt);
}
}
}
return { ok: false, error: lastError };
}
cli({
site: 'douyin',
name: 'publish',
@@ -106,19 +136,13 @@ cli({
throw new ArgumentError(`封面文件不存在: ${path.resolve(coverPath)}`);
}
}
// ── Phase 1: STS2 credentials ───────────────────────────────────────
const credentials = await getSts2Credentials(page);
// ── Phase 1: upload credentials ────────────────────────────────────
const credentials = await getUploadAuthV5Credentials(page);
// ── Phase 2: Apply TOS upload URL ───────────────────────────────────
const vodUrl = `https://vod.bytedanceapi.com/?Action=ApplyVideoUpload&ServiceId=1128&Version=2021-01-01&FileType=video&FileSize=${fileSize}`;
const vodJs = `fetch(${JSON.stringify(vodUrl)}, { credentials: 'include' }).then(r => r.json())`;
const vodRes = (await page.evaluate(vodJs));
const { VideoId: videoId, UploadHosts, StoreInfos } = vodRes.Result.UploadAddress;
const tosUrl = `https://${UploadHosts[0]}/${StoreInfos[0].StoreUri}`;
const tosUploadInfo = {
tos_upload_url: tosUrl,
auth: StoreInfos[0].Auth,
video_id: videoId,
};
const tosUploadInfo = await applyVideoUploadInner(fileSize, credentials);
let coverUri = '';
let coverWidth = 720;
let coverHeight = 1280;
// ── Phase 3: TOS upload ─────────────────────────────────────────────
await tosUpload({
filePath: videoPath,
@@ -130,22 +154,32 @@ cli({
},
});
process.stderr.write('\n');
process.stderr.write(' 提交上传...\n');
const committedVideo = await commitVideoUploadInner(tosUploadInfo, credentials);
const videoId = committedVideo.video_id;
process.stderr.write(` 上传已提交: ${videoId}\n`);
coverWidth = committedVideo.width || coverWidth;
coverHeight = committedVideo.height || coverHeight;
if (!coverUri && committedVideo.poster_uri) {
coverUri = committedVideo.poster_uri;
}
// ── Phase 4: Cover upload (optional) ────────────────────────────────
let coverUri = '';
let coverWidth = 720;
let coverHeight = 1280;
if (kwargs.cover) {
const resolvedCoverPath = path.resolve(kwargs.cover);
// 4A: Apply ImageX upload
const applyUrl = `${IMAGEX_BASE}/?Action=ApplyImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01&UploadNum=1`;
const applyJs = `fetch(${JSON.stringify(applyUrl)}, { credentials: 'include' }).then(r => r.json())`;
const applyRes = (await page.evaluate(applyJs));
const { StoreInfos: imgStoreInfos } = applyRes.Result.UploadAddress;
const imgUploadUrl = `https://${imgStoreInfos[0].UploadHost}/${imgStoreInfos[0].StoreUri}`;
const applyRes = requireObjectEvaluateResult(await page.evaluate(applyJs), '抖音封面申请上传地址响应异常');
throwIfImagexError('抖音封面申请上传地址', applyRes);
const imgStoreInfo = applyRes.Result?.UploadAddress?.StoreInfos?.[0];
if (!imgStoreInfo?.UploadHost || !imgStoreInfo?.StoreUri) {
throw new CommandExecutionError(`抖音封面申请上传地址响应缺少 UploadHost/StoreUri: ${JSON.stringify(applyRes).slice(0, 500)}`);
}
const imgUploadUrl = `https://${imgStoreInfo.UploadHost}/${imgStoreInfo.StoreUri}`;
// 4B: Upload image
const coverStoreUri = await imagexUpload(resolvedCoverPath, {
upload_url: imgUploadUrl,
store_uri: imgStoreInfos[0].StoreUri,
store_uri: imgStoreInfo.StoreUri,
});
// 4C: Commit ImageX upload
const commitUrl = `${IMAGEX_BASE}/?Action=CommitImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01`;
@@ -158,19 +192,13 @@ cli({
body: ${JSON.stringify(commitBody)}
}).then(r => r.json())
`;
await page.evaluate(commitJs);
const commitRes = requireObjectEvaluateResult(await page.evaluate(commitJs), '抖音封面提交上传响应异常');
throwIfImagexError('抖音封面提交上传', commitRes);
coverUri = coverStoreUri;
}
// ── Phase 5: Enable video ───────────────────────────────────────────
const enableUrl = `https://creator.douyin.com/web/api/media/video/enable/?video_id=${videoId}&aid=1128`;
await browserFetch(page, 'GET', enableUrl);
// ── Phase 6: Poll transcode ─────────────────────────────────────────
const transResult = await pollTranscode(page, videoId);
coverWidth = transResult.width;
coverHeight = transResult.height;
if (!coverUri) {
coverUri = transResult.poster_uri;
}
// The gateway upload flow returns a committed VOD upload result; the legacy
// enable/transend endpoints can hang for that flow, so create_v2 consumes
// the committed video_id and poster metadata directly.
// ── Phase 7: Content safety check ───────────────────────────────────
if (!kwargs.no_safety_check) {
const safetyUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check';
@@ -179,25 +207,42 @@ cli({
title,
desc: caption,
};
await browserFetch(page, 'POST', safetyUrl, { body: safetyBody });
const preCheck = await tryFastDetectFetch(page, 'POST', safetyUrl, { body: safetyBody });
if (!preCheck.ok) {
process.stderr.write(' 内容安全预检接口无响应,继续轮询检测结果。\n');
}
const pollUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/poll';
const deadline = Date.now() + 30_000;
let safetyPassed = false;
let pollUnavailableCount = 0;
while (Date.now() < deadline) {
const pollRes = (await browserFetch(page, 'POST', pollUrl, {
body: safetyBody,
}));
if (pollRes.status === 0) {
const poll = await tryFastDetectFetch(page, 'POST', pollUrl, { body: safetyBody });
if (!poll.ok) {
pollUnavailableCount += 1;
if (!preCheck.ok && pollUnavailableCount >= 3) {
break;
}
await sleep(2000);
continue;
}
pollUnavailableCount = 0;
const pollRes = poll.value;
if (pollRes.status === 0 || (pollRes.has_done === true && pollRes.detect_result?.reason_code === 0 && (pollRes.detect_list?.length ?? 0) === 0)) {
safetyPassed = true;
break;
}
if (pollRes.status === 1) {
throw new CommandExecutionError('内容安全检测不通过,请修改后重试', '使用 --no_safety_check 跳过');
}
await new Promise((r) => setTimeout(r, 2000));
await sleep(2000);
}
if (!safetyPassed) {
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '使用 --no_safety_check 跳过');
if (!preCheck.ok && pollUnavailableCount >= 3) {
process.stderr.write(' 内容安全预检持续无响应,跳过本地预检,交由 create_v2 后的平台审核。\n');
}
else {
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '如确认要跳过本地预检,可使用 --no_safety_check;提交后仍会走抖音平台审核');
}
}
}
// ── Phase 8: create_v2 publish ──────────────────────────────────────
@@ -266,12 +311,13 @@ cli({
},
};
const publishUrl = `https://creator.douyin.com/web/api/media/aweme/create_v2/?read_aid=2906&${DEVICE_PARAMS}`;
process.stderr.write(' 创建定时发布...\n');
const publishRes = (await browserFetch(page, 'POST', publishUrl, {
body: publishBody,
}));
const awemeId = publishRes.aweme_id;
const awemeId = publishRes.aweme_id ?? publishRes.item_id;
if (!awemeId) {
throw new CommandExecutionError(`发布成功但未返回 aweme_id: ${JSON.stringify(publishRes)}`);
throw new CommandExecutionError(`发布成功但未返回 aweme_id/item_id: ${JSON.stringify(publishRes)}`);
}
const url = `https://www.douyin.com/video/${awemeId}`;
const publishTimeStr = new Date(timingTs * 1000).toLocaleString('zh-CN', {
+1 -1
View File
@@ -8,7 +8,7 @@ cli({
domain: 'creator.douyin.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'aweme_id', required: true, positional: true },
{ name: 'aweme_id', required: true, positional: true, help: '抖音作品 IDaweme_id,可从作品 URL 末尾获取)' },
],
columns: ['metric', 'value'],
func: async (page, kwargs) => {

Some files were not shown because too many files have changed in this diff Show More