Compare commits

...

101 Commits

Author SHA1 Message Date
jakevin ce432c2428 chore(release): 1.8.0 (#1682)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
* chore(release): 1.8.0

Substantial release: weread-official adapter, wider LinkedIn / Twitter / Reddit / Zhihu coverage, 12306 / Suno / Xianyu additions, security and reliability fixes, plus a 20% README shrink.

* chore: remove orphan docs/adapters-doc/ones.md

The file was a leftover from PR #386 (2026-04-10) and has been
superseded by docs/adapters/browser/ones.md. Bundled into the 1.8.0
release commit chain so the release doesn't ship with a dead docs
file alongside the new docs.

Skipped from opus-reviewer's audit (Tier 1 #1-#4) for this release:
- #1 smart-search dead refs (10 spots) — owned by @codex-coder's
  skill-deletion PR; release PR will rebase on top of it.
- #3 clis/test-utils.js relocation — touches 19 importers, separate
  refactor PR.
- #4 clis/slock/ orphan — needs WAWQAQ design call.
- #6 opencli-usage:161 wording — current "Commands that used to
  exist" framing is already clear enough.
- #7 docs/adapters/index.md sync (8-20 missing sites) — broader docs
  PR, not release-time bundling.

* chore: remove clis/slock + sync docs/adapters/index.md (audit #4 + #7)

Per WAWQAQ post-audit directive on #OpenCLI:f046ece7:

- `clis/slock/` was a half-finished orphan with only `_utils.js` and no
  command entry points. Removed.
- `docs/adapters/index.md` was missing 11 browser adapters: 12306,
  suno, weread-official, qwen, 1point3acres, brave, duckduckgo, cnki,
  flomo, jianyu, taobao. Added all with commands sourced from
  cli-manifest.json. Desktop section already covered all 7 desktop
  adapters (Cursor / Codex / Antigravity / ChatGPT App / ChatWise /
  Discord / Doubao App).
2026-05-20 03:31:57 +08:00
jakevin 7ee16aa087 feat(booking): add search adapter for Booking.com hotel listings (#1680)
* feat(booking): add search adapter for Booking.com hotel listings

New `opencli booking search <destination> --checkin --checkout` adapter
scrapes the server-rendered hotel cards on www.booking.com via stable
`[data-testid=property-card]` selectors. No login required (Strategy.PUBLIC
+ browser:true).

Highlights
- 12 columns: rank, name, country, slug, star_rating, review_score,
  review_count, price_amount, price_currency, distance, recommended_room,
  url. `slug` + URL stay stable across locales (better round-trip key
  than `name`, which Booking sometimes localizes from session cookies).
- Score parser anchors on `(\d{1,2})\.(\d)` so the duplicated "8.68.6" /
  "评分8.68.6很棒" rendering doesn't mis-parse to 8.68.
- Currency symbol → ISO 4217 map (US$/€/£/¥/¥/₹/₩/HK$/A$/NT$/S$/CN¥);
  honor `--currency` URL param for stable codes.
- Pagination via `--offset` (Booking pages 25/request); `rank` includes
  the offset so paginated calls stay sortable.
- Captcha-page detection short-circuits to CommandExecutionError instead
  of silent empty rows.

Typed errors (no silent clamp / fallback)
- Bad date / out-of-range adults/rooms/children/limit/offset / unknown lang
  / malformed currency → ArgumentError up front (before any navigation).
- Browser nav failure → CommandExecutionError.
- Zero cards rendered → EmptyResultError with a hint.
- Captcha page → CommandExecutionError.

29 unit tests cover the helpers, the registry shape, every typed-error
path, the {session,data} CDP envelope unwrap, and offset-aware rank
numbering. Silent-column-drop + typed-error-lint audits unchanged.
Live-verified against Tokyo + Paris.

* fix(booking): harden search parser boundaries

* fix(booking): separate no-card drift from empty
2026-05-20 03:29:03 +08:00
jakevin 7a2ab47bf8 chore(skills): remove smart-search (#1683) 2026-05-20 03:23:32 +08:00
jakevin 2c8b50c4fd docs(readme): shrink CLI Hub + Core Concepts + merge Update into Install (#1681)
Per WAWQAQ:

1. **CLI Hub**: drop the 13-row 3-column table; enumerate just the
   names inline ("gh · docker · vercel · wrangler · ntn · obsidian · …")
   plus one-liner register / list commands. Removes "Manual install"
   ntn note (search lives in external-clis.yaml / ntn's own docs).
   Compresses the 7-row Desktop App Adapters table to a single inline
   line pointing at docs/adapters/desktop/.

2. **Core Concepts** section dissolved: its four subsections
   ("browser", "Built-in adapters", "Writing a new adapter",
   "CLI Hub and desktop adapters") duplicated the intro 3-bullet
   + later dedicated sections. Kept the substantive "Writing a new
   adapter" callout as its own top-level section. The "For AI Agents
   (Developer Guide)" tail block at the bottom was a third copy of
   the same recipe — removed.

3. **Update** merged with **Install skills**: install header now
   reads "Install skills (also refreshes existing installs)", and
   the standalone Update section collapses to a single command
   (`npm install -g @jackwener/opencli@latest && npx skills add ...`).

Net: EN 410 → 326 (-20%), ZH 455 → 366 (-20%). Same coverage; just
less repetition.
2026-05-20 03:12:36 +08:00
Benjamin Liu 51a9456305 feat(linkedin): add people-search command (#1649)
* feat(linkedin): add people-search command (#1621)

Closes #1621. Adds opencli linkedin people-search <keywords> for
finding people on standard LinkedIn (not Sales Navigator).

Architecture note. Standard LinkedIn moved its people search results
page to Server-Driven UI / React Server Components on the
/flagship-web/rsc-action/... path stack. The legacy Voyager REST
endpoint /voyager/api/search/dash/clusters returns HTTP 500 from a
web context; its modern camelCase rename voyagerSearchDashClusters
returns the same. The result list is rendered server-side and the
page HTML IS the result payload; Voyager calls from the page are
sidebar / notification concerns, not search results.

Extraction strategy. LinkedIn SSR uses obfuscated CSS class hashes
(e.g. _997b7c77) that rotate on every deploy AND display:contents
wrappers that flatten the DOM tree. Class-based selectors, walk-up-
to-card logic, and anchor-pair element ranges all fail because no
element boundary matches a person's card.

Working approach: extract main.innerText once, split by newline,
slice between consecutive person names. The names come from the
aria-hidden spans of /in/<handle> anchors. LinkedIn's SSR emits a
card as a name line followed by degree badge / headline / location
/ action labels before the next card's name line - a layout that
has been stable through several DOM refactors.

Critical filter: /in/<handle> anchors over-count because LinkedIn
renders each mutual connection as a /in/ anchor inside another
card's result. The skip() predicate during name-line lookup drops
mutual-connection lines ("X, Y and N other mutual connections"), so
anchors that don't have a real name line are filtered out.

CUL caveat. LinkedIn imposes a monthly Commercial Use Limit on
people search against the standard site. Burst behaviour is
irrelevant - the limit is a calendar-month counter. The adapter
runs one navigation per invocation (no pagination) so a single call
costs exactly one CUL query. --limit is capped at 10 to keep a
single call's information density high without surfacing the
"reached commercial use limit" yellow banner faster.

Schema:
  rank, name, headline, location, profile_url

Live verified against kyfw 12306-style throttled cadence (sleep 60s
between dev iterations to keep CUL consumption visible): 5/5 rows
populated with name + headline + location + profile_url for the
keyword "reinforcement learning". Mutual-connection anchors
correctly filtered out so the row order matches LinkedIn's own
ranking.

Tests: 10 unit tests covering URL construction, limit validation,
extraction-script invariants (anchor enumeration, text-slice
approach, mutual-connection filter, aria-hidden span as name source),
limit slicing, AuthRequiredError on missing JSESSIONID, CUL-
flavoured CommandExecutionError on redirect, EmptyResultError on
zero rows, ArgumentError on empty keywords, and registry shape.

* fix(linkedin): harden people search typed boundaries

* fix(linkedin): fail people search candidate parser drift

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-20 00:40:53 +08:00
Benjamin Liu 62592547a4 fix(adapters): migrate empty-data throws to EmptyResultError across 5 commands (#1674 follow-up) (#1678)
* fix(adapters): migrate empty-data throws to EmptyResultError across 5 commands (#1674 follow-up)

Continues the structured-error migration owner started in #1674
(fix(xhs,youtube): 把合法空数据语义切到 EmptyResultError). Same
motivation: callers need to distinguish "the platform legitimately
has no data for this target" from "fetch infrastructure is broken,
retry me", because downstream automation pipelines that batch over
seed lists conflate the two and trip soft-rate-limit heuristics.

Sites converted (5 commands, 6 throw sites):

powerchina/search.js (2 sites):
- "[taxonomy=empty_result] ... extracted only navigation/portal rows"
- "[taxonomy=empty_result] ... api/dom yielded no result"
  Both already self-labelled with the empty_result taxonomy tag,
  making this the canonical fix.

xiaohongshu/creator-notes.js, creator-notes-summary.js (both):
- "No notes found. Are you logged into creator.xiaohongshu.com?"
  The "is logged in" hint is preserved in the empty message so users
  can self-diagnose, while the error type is now structured.

xiaohongshu/creator-stats.js:
- "No data for period <X>. Available: <a, b, c>"
  Empty-data condition: requested period exists in the API surface
  but has zero numeric data; available periods are still surfaced
  in the message.

xiaohongshu/creator-note-detail.js:
- "No note detail data found. Check note_id and login status..."

Shape: exit code 66, stderr code: EMPTY_RESULT, matching
bilibili/subtitle, xhs/user, youtube/transcript precedent.

Out of scope:
- tiktok/{user,notifications,explore}.js: throws live inside
  page.evaluate template strings and run in browser context; the
  Node-side caller already regex-routes them via
  throwTikTokPageContextError({emptyPattern: /No videos found/, ...})
  to EmptyResultError. The existing design is correct.
- eastmoney/_secid.js / antigravity/serve.js / instagram/collection-*:
  input-validation throws, ArgumentError territory not EmptyResultError.

* test(adapters): cover empty-result migrations

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-20 00:30:54 +08:00
jakevin 4a3a55634a docs(readme): curate built-in commands to popular sites + add wrangler (#1679)
Per WAWQAQ:

1. Built-in Commands table cut from 30 EN rows / 86 ZH rows down to a
   curated 11-site list (xiaohongshu, bilibili, zhihu, hackernews,
   linkedin, reddit, twitter, claude, gemini, notebooklm, amazon).
   The README is meant to surface high-traffic / well-known sites;
   the long-tail (100+ adapters) is one click away via
   docs/adapters/index.md. linkedin (full) replaces linkedin-learning
   in the curated set per the spec.

2. Add Cloudflare Wrangler as a new external CLI passthrough:
   - src/external-clis.yaml entry (binary: wrangler, npm -g)
   - CLI Hub table row in EN + ZH READMEs
   - cli-manifest.json regen reflects the new entry (857 entries)
2026-05-19 23:17:15 +08:00
lenovobenben da497f0b02 feat(zhihu): add answer comments reader
* feat(zhihu): add answer comments reader

* fix(zhihu): harden answer-comments boundaries

* fix(zhihu): keep answer comments flat

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:59:42 +08:00
jakevin 2590278f43 fix(chatgpt): detect generated image surfaces (#1677) 2026-05-19 19:50:14 +08:00
Benjamin Liu 488e407a65 feat(twitter): add device-follow notification stream command
* feat(twitter): add device-follow command for /i/timeline notification stream (#1628)

Closes #1628. Adds the twitter device-follow command, which reads the
curated tweet list aggregated under a bell-icon "new posts from @userA
and N others" notification. Direct GET /i/timeline redirects to /home,
so the data is only reachable via the legacy v1.1 REST endpoint
/i/api/2/notifications/device_follow.json , none of the existing
twitter commands cover this stream:

- twitter timeline    home for-you / following feed (different endpoint)
- twitter notifications  the notification list itself, not aggregated
                         tweets inside any one notification
- twitter search     search-based, can't reproduce the aggregation

Endpoint discovery + field-mapping originally proposed by @traddo in
#1628; this PR upstreams a clean implementation that:

- Strategy.COOKIE + ct0 from CDP cookie jar + the public web bearer
  token from clis/twitter/utils.js (same auth path as twitter timeline)
- Hits /i/api/2/notifications/device_follow.json directly via
  page.evaluate fetch on the x.com origin so SameSite=Lax cookies are
  preserved
- Joins each entry.content.item.content.tweet.id to
  globalObjects.tweets[id] and resolves the author via
  globalObjects.users[tweet.user_id_str]
- Returns the canonical twitter row columns (id, author, text, likes,
  retweets, replies, views, created_at, url), matching twitter timeline
  minus has_media / media_urls / card / quoted_tweet which the legacy
  v1.1 endpoint does not surface
- Sets views: null rather than a 0 sentinel; the legacy endpoint does
  not return view counts even with include_ext_views=true, and the
  GraphQL TweetResultByRestId round-trip per tweet was judged too
  expensive for a list command (typed-errors §3: no scalar sentinels
  that lie about real engagement)
- parseLimit enforces strict 1-200 integer validation with no silent
  clamping; the only baseline addition is the silent-sentinel on the
  "unknown" author fallback, which matches the exact precedent in
  twitter/timeline.js:76 that is already baselined

Tests: 17 unit tests in device-follow.test.js cover parseLimit strict
validation, URL parameter shape, entry/tweet join, user-resolution
fallback, dedup via the seen set, empty-stream shape, the canonical
column registration, AuthRequiredError on missing ct0, and
CommandExecutionError on non-2xx fetch.

Live verified the endpoint shape end-to-end against the logged-in
session: HTTP 200 with the expected
{globalObjects: {tweets, users}, timeline: {id: 'tweet_notifications',
instructions: [{addEntries: {entries: []}}]}} envelope. The tester
account has no bell-notification follows enabled, so entries is empty,
but the shape and auth path are confirmed against the documented
spec.

* fix(twitter): harden device-follow typed boundaries

* fix(twitter): fail fast on device-follow drift

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:34:30 +08:00
jakevin e682c1c30a fix(deps): restore Node 20 runtime compatibility (#1673) 2026-05-19 19:20:38 +08:00
Ocean 86f57c0846 feat(reddit): 在 listing 命令上暴露 post_hint / url / preview / gallery 4 个媒体路由列 (#1676)
* feat(reddit): 在 listing 命令上暴露 post_hint / url / preview / gallery 4 个媒体路由列

5 个 reddit listing 命令(popular / hot / frontpage / search / subreddit)
每行新增 4 列,下游消费者不用 scrape selftext 也能区分 image / gallery /
hosted:video / link / self 五类内容:

- `post_hint` — Reddit 自报的内容类型(image | hosted:video | link | self 等)
- `url_overridden_by_dest` — 外链帖的原始 URL(image/link 类型才有)
- `preview_image_url` — 缩略图地址(HTML-decoded,Reddit 即便 raw_json=1
  也会在 preview URL 里返回 `&amp;`)
- `gallery_urls` — 多图相册数组(HTML-decoded)

## 实现

每个 adapter 的 evaluate 块内嵌两个 helper:

- `decodeHtml(s)` — 6 个 HTML entity 替换(&amp; / &lt; / &gt; / &quot; /
  &#x27; / &#39;)
- `extractRedditMedia(d)` — 从 post `data` 中抽 4 个字段,gallery_urls 从
  `gallery_data.items[].media_id` × `media_metadata[id].s.u` 组合得到

helper 在每个 adapter 里 inline 复制(reddit 没有 shared 文件,模式跟现有
adapter 一致)。`clis/reddit/extract-media.test.js` 把 helper 行为锁在
8 个 fixture(plain / image / gallery / hosted-video / link /
html-decode / 缺字段 / nullish input);每个 adapter 的 .test.js 额外
grep 自己源码里有 `function extractRedditMedia` 和 `...extractRedditMedia(c.data)`
两处接入痕迹,并断言 columns 数组形状。

frontpage 和 subreddit 之前 evaluate 返回原始 `children`、map 块按
`item.data.title` 索引;为了让 `gallery_urls` 这种数组字段能被 map 块的
模板字符串渲染,refactor 成和 popular/hot/search 一致的"evaluate 内部
就 map 成中间对象、map 块按 `item.title` 索引"模式。

## 范围

只覆盖 5 个 listing 命令。**`read` 不在本 PR 内**:它的 evaluate 块在
post-#1651 时代已经是 error-kind-discriminated 的富结构(`kind: 'inaccessible'`
/ `kind: 'http'` / `kind: 'malformed'`),原始 commit 的"POST 行带 media、
comment 行空"模式和当前结构冲突太深,单独的 read 接入留作后续 PR。

完全 additive:既有字段名、顺序、值都不变;新字段加在每行末尾。

## 验证

- `npx vitest run clis/reddit/ --project adapter` → 84/84 通过
- `node scripts/check-silent-column-drop.mjs` → current=97, baseline=97, new=0
- `npx tsc --noEmit` 干净
- `npm run build` 干净

* feat(reddit): expose home media route columns

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:14:37 +08:00
Ocean 4a6cfe8060 fix(xhs,youtube): 把合法空数据语义切到 EmptyResultError (#1674)
* fix(xhs,youtube): 把"合法空数据"语义切到 EmptyResultError

把 xiaohongshu/user 和 youtube/transcript 跟 bilibili/subtitle 已有的
structured-error 模式对齐,让下游能区分"用户/视频没内容"和"fetch 真的挂了"。

## xiaohongshu user

返回 `No public notes found for this Xiaohongshu user` 的场景——目标用户
零公开笔记(销号 / 私密 / 全删)——原来抛 plain `Error`,下游无法和
"真的 fetch 失败 / cookie 死"区分。

改抛 `EmptyResultError`,exit code 变 66,stderr 携带 `code: EMPTY_RESULT`,
跟 `bilibili subtitle` empty 同 shape。

## youtube transcript

`No captions available for this video`(作者没开 CC、YouTube 也没自动生成)
是数据条件,不是基础设施失败。原来跟 HTTP/解析错误一样抛
`CommandExecutionError`,造成调用方反复重试。

改这个特定 case 抛 `EmptyResultError`;其他 caption 错误(HTTP / parse /
empty response)继续走 `CommandExecutionError` 触发重试。

## 为什么 downstream 需要这个

调用方(如自动化采集流水线)通常对 "data.length === 0 && exitCode !== 0"
做 soft-rate-limit 启发式:N 次连续 soft fail 触发 24h 平台跳过。当 seed 列表
里有变质条目(XHS 账号销号 / YouTube 视频丢失字幕),"empty" 响应堆积会
误触跳过——cookies 和平台本身都健康。EmptyResultError 让调用方能区分
"这个用户没内容"和"API 挂了"。

## 测试

- `npx vitest run clis/xiaohongshu clis/youtube` —— 全过
- `npx tsc --noEmit` 干净

* fix(xhs): distinguish empty user notes from parser drift

* fix(empty): tighten legal empty evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 19:02:32 +08:00
Ocean bcb0fb362f feat(twitter): expose bio on read command
* feat(twitter): 在 read 命令上暴露 bio(用户简介)

`list-tweets` / `timeline` / `search` 三个读命令现在每行多一列 `bio`,从
`user.legacy.description` 抽。匹配 `profile` 命令已有的 `bio` 字段,让下游
消费者展示作者画像时省去"读了推文还要再读作者主页"的 roundtrip。

bio 在 user 对象缺失或没 description 时回落到 `''`。columns 数组同步更新,
`--format columns` 会渲染 bio。完全 additive:既有字段名、顺序、值都不变。

延续 #1660 (card binding_values) 和 #1667 (quoted_tweet) 的同一类
read-side enrichment 模式。

## 验证

- `clis/twitter/list-tweets.test.js` / `clis/twitter/search.test.js` 已有
  shape assertion 补上 `bio: ''` 行
- `timeline.test.js` 用 `toMatchObject`(子集匹配),新增 bio 不会破断言
- `npx vitest run clis/twitter/list-tweets.test.js clis/twitter/timeline.test.js
  clis/twitter/search.test.js --project adapter` → 48/48 通过
- `npx tsc --noEmit` 干净
- `npm run build` 干净
- `node scripts/check-silent-column-drop.mjs` → current=97, baseline=97, new=0

* test(twitter): cover inline bio extraction

* feat(twitter): expose thread author bio

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 18:56:38 +08:00
lenovobenben 85b1c07ba9 feat(zhihu): include answer links in question results
* feat(zhihu): include answer links in question results

* fix(zhihu): avoid fake answer links for malformed ids

* fix(zhihu): dedupe answers by trusted id

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 15:34:24 +08:00
Ocean 6fbaf0d5b8 feat(twitter): 在 read 命令上暴露 quoted_tweet(被引用的推文) (#1667)
* feat(twitter): expose quoted_tweet on read commands

When a tweet quotes another tweet (embedded preview with commentary), the
quoted tweet's content is in `tweet.quoted_status_result.result` — same
`legacy / core / card / note_tweet` shape as the outer tweet. Until now
none of the 5 read commands (list-tweets / timeline / thread / tweets /
search) surfaced this nested object, so downstream consumers couldn't
render the quoted preview card.

Adds `extractQuotedTweet(tw)` in shared.js (mirrors the
`extractMedia` / `extractCard` helper pattern) and threads it through
all 5 read commands plus their CLI `columns:` declarations.

Output shape is a deliberately small subset of the main tweet
(id/author/name/text/created_at/url + media + card). Counts and full
author bio are intentionally omitted to keep timeline payloads from
ballooning 2-3x; consumers needing those can re-fetch
`twitter thread <quoted_id>`.

Notable edge cases tested in shared.test.js:
- plain tweets (no `is_quote_status`) -> null
- tombstoned / unavailable quoted tweets (deleted / privacy-restricted) -> null
- TweetWithVisibilityResults `result.tweet` shim unwrap
- long-form note_tweet text preferred over truncated full_text
- quote-of-a-quote does NOT recurse (avoids payload explosion on threads
  where every reply re-quotes the root)

* fix(twitter): require quoted tweet render evidence

* fix(twitter): validate quoted tweet author shape

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 14:56:24 +08:00
Ocean 34f351e59f feat(reddit/subscribed): 接入 LoginWallError 嗅探(#1650 的第一个 caller) (#1668)
* feat(errors,utils): 添加 LoginWallError 与 HTML-as-JSON 响应嗅探器

部分 adapter(twitter list-tweets/thread、reddit search/subreddit 等)历史上
直接 `JSON.parse(await r.text())` 或 `await r.json()` 解析响应。当服务端返回
登录墙、限流页或 WAF 拦截页(而不是 JSON)时,body 以 `<!DOCTYPE html>` 或
`<html ...>` 开头,解析直接抛出晦涩的
`SyntaxError: Unexpected token '<', "<!DOCTYPE "... is not valid JSON`,
调用方无法把它和真正的 JSON 解析失败区分开。

本 PR 添加三块共享基础设施,让 adapter 能识别 HTML 情况并抛出结构化的
`LoginWallError`(带 status / url / body 预览),而不是裸的解析栈:

  - `LoginWallError`(src/errors.ts):新的 `CliError` 子类,含 `status`、
    `url`、`bodyPreview` 字段,hint 提示"重新登录或等待限流过期",
    退出码映射到 `EXIT_CODES.NOPERM`。
  - `parseJsonOrThrowLoginWall(response)`(src/utils.ts):Node 端 helper,
    供从 daemon 侧 fetch 的 adapter 使用(接收 Fetch Response)。
  - `BROWSER_JSON_SNIFF_FN` + `throwIfLoginWall(value)`(src/utils.ts):
    browser 端等价物。字符串片段嵌入到 `page.evaluate` 里,返回值是
    解析后的 JSON,或 `{ error: status }` HTTP 形状,或 `LoginWallSignal`
    哨兵对象(`{ __loginWall: true, status, url, ... }`),Node 侧拿到后
    转成 `LoginWallError`。

行为是 opt-in:现有 adapter 不调用这些 helper 就完全不受影响。后续 PR
会把 reddit / twitter adapter 接入这些 helper。`src/utils.test.ts` 新增
13 个单测,覆盖 Node 端 + browser 端两条路径以及 body 预览的 100 字截断。

* feat(reddit/subscribed): 接入 LoginWallError 嗅探(#1650 的第一个 caller)

`reddit subscribed` 是从 daemon-backed browser session 调 Reddit 的 JSON API
(`/api/me.json` + `/subreddits/mine/subscriptions.json`)。原本两处 `await res.json()`
在 Reddit 返回登录墙 / WAF / over-18 拦截页(HTML body + 200 OK)时会抛
`SyntaxError: Unexpected token '<'`,被外层 try/catch 兜底成 `kind: 'exception'`
→ Node 侧报成 `CommandExecutionError: subscribed failed: SyntaxError ...`,
看不出 root cause。

接入 #1650 的 helper 后:

- **Browser 侧**:用 `BROWSER_JSON_SNIFF_FN` 提供的 `fetchJsonOrLoginWall(url, init)`
  替换裸 `fetch + .json()`。helper 内部 sniff `Content-Type: text/html` 或
  `<!DOCTYPE` / `<html` body 前缀,返回 `{ __loginWall: true, status, url,
  contentType, bodyPreview }` 哨兵(不抛,交给调用者)。
- **Cross-boundary**:两处 fetch 站点(me.json + subscriptions.json)发现哨兵后
  返回 `{ kind: 'login-wall', sentinel, where }` 透传给 Node。
- **Node 侧**:`throwIfLoginWall(result.sentinel, { url: result.where })` 把
  哨兵转成结构化 `LoginWallError`(含 `status` / `url` / `bodyPreview` 字段,
  exit code `EXIT_CODES.NOPERM`,hint 提示重新登录或等限流过期)。

这是 #1650 的第一个真实 caller,覆盖 3 块 export 全部(`BROWSER_JSON_SNIFF_FN`
+ `throwIfLoginWall` + `LoginWallError`)。其它 adapter 后续按这个模板逐个接入。

回归测试新增 1 个:mock evaluate 返回 `{ kind: 'login-wall', sentinel }`,
断言 Node 端抛 `LoginWallError` 且 `status` / `url` / `bodyPreview` 字段正确。
原有 12 个测试不动,全部通过。
2026-05-19 14:52:42 +08:00
jakevin acc18be999 docs(readme): tighten skill attribution + remove redundant Highlights (#1666)
Per WAWQAQ T1 + T2 review:

T1 — skill attribution carries the same intent PR #1654 started but
hadn't fully cleaned up:
- Skill table row for `opencli-adapter-author` no longer claims it
  "operate[s] a site in real time" (SKILL.md explicitly says ad-hoc
  driving lives in `opencli-browser`). Browser-op example
  ("Help me check my Xiaohongshu notifications") moved to the
  `opencli-browser` row where it belongs.
- "How it works" section's 5 browser primitives (navigate / read /
  interact / extract / wait) now point to `opencli-browser` instead
  of `opencli-adapter-author`.
- Skill references list re-orders to surface `opencli-browser` first
  with a concrete description, and `opencli-adapter-author` no longer
  claims to cover "browser operation".

T2 — drop the Highlights section. Pre-Quick-Start had four parallel
summary blocks (3-line tagline / 3-bullet automation intro / CLI-hub
+ desktop line / 5-bullet Highlights) that all said the same thing.
Highlights was the most-recent and most-redundant of the four; the
remaining three carry the value props cleanly: tagline → three usage
modes → CLI-hub + desktop scope.

EN + ZH READMEs synced.
2026-05-19 13:28:44 +08:00
Benjamin Liu 67ed9e9c81 feat(linkedin-learning): add search / trending / course read commands (#1657)
* feat(linkedin-learning): add search / trending / course read commands (#1021)

Closes #1021. Adds a new linkedin-learning site adapter with three
read-only commands against LinkedIn Learning's public learning-api
REST surface. Shares cookie session with linkedin.com; Learning
queries are not subject to the people-search CUL.

Commands:
- linkedin-learning search <keywords>  searchV2?q=keywords
- linkedin-learning trending            feedRecommendationGroups?q=learner
- linkedin-learning course <slug>      courses?q=slug

Endpoints were discovered via browser network capture on
/learning/search and /learning/<slug> pages: searchV2 returns a flat
list of courses/videos/paths keyed by entityType, headline.title.text
holds the canonical title, length is a TimeSpan in seconds, and rating
is averaged from ratingSum/ratingCount when averageRating is missing.

trending walks the carousels array on each recommendation group, flattens
cards across them, dedups by slug, and respects --limit. Group is
labeled with the carousel title (e.g. "Top picks for you") or the
upstream annotation tag (TOP_PICKS).

course accepts either a bare slug or a full /learning/<slug> URL, then
hits /learning-api/courses?q=slug. The detail endpoint omits rating
fields even when search reports them; this is documented in the
adapter doc rather than fixed via a second /reviews fetch to keep the
PR scoped to one endpoint per command.

CUL caveat: Learning's API has no per-month limit, so dev iterations
can be much more aggressive than the people-search adapter (#1649).
Three commands were live-verified against a logged-in account with
60s sleeps between calls (conservative for first-pass safety).

Tests: 28 unit tests across search.test.js (12), trending.test.js (6),
course.test.js (10) cover URL construction, limit validation, author
join, duration / rating coercion, row mapping, carousel flattening
and dedup, slug parsing from URL forms, and the standard auth /
empty / fetch-failure error paths.

Live verified:
- search "AI agent" --limit 3: 3 rows with title/instructor/rating
- trending --limit 3: 3 personalized course picks
- course agentic-ai-build-your-first-agentic-ai-system: title, 3932s
  duration, 18 videos, release date 2026-03-27

* fix(linkedin-learning): harden read result boundaries

* fix(linkedin-learning): require course title evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 13:17:33 +08:00
Benjamin Liu 8577d88ee7 fix(cli): escape leading-dash positional values via argv preprocessor (#1658)
* fix(cli): escape leading-dash positional values via argv preprocessor (#1160)

Closes #1160. `opencli boss detail -abc123def` failed with
`error: unknown option '-abc123def'` because commander treats any
argv token starting with `-` as an option. BOSS 直聘 securityId
tokens are opaque base64-ish strings that can legitimately start
with `-`, and the same shape is possible for any adapter that takes
an opaque-id positional.

Adds escapeLeadingDashPositional() to src/cli-argv-preprocess.ts,
called from main.ts after the existing rewriteBrowserArgv pass. The
preprocessor:

- Reads cli-manifest.json (the same manifest the registry uses) and
  builds a set of `<site>/<cmd>` keys whose first positional is
  required.
- Walks past root flags (matching the existing rewriteBrowserArgv
  walker) to find the site + command tokens.
- If the next argv token starts with `-`, is not the recognised
  short flags `-f` / `-v` / `-h`, is not `--*`, and is not the
  pre-escaped `--` separator, inserts `--` before it.

Tests: 12 new unit tests in cli-argv-preprocess.test.ts cover the
basic insertion, trailing-flag preservation, non-touched cases
(normal values, recognised short flags, long flags, already-escaped,
non-positional commands, unknown commands, short argv, and the
`--profile work boss detail -abc` form that walks past a root
value flag).

Live verified: `node ./dist/src/main.js boss detail -abc123def`
no longer raises 'unknown option'. The adapter now receives the
dash-leading value and proceeds to fetch, where it correctly
surfaces an upstream "missing required parameter" error for the
fake id used in this smoke test.

* fix(cli): preserve options around dash positionals

* fix(cli): preserve attached short option values

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 13:13:24 +08:00
Ocean cd731bd2aa feat(twitter): 在 read 命令上暴露 card binding_values(链接预览卡片) (#1660)
* feat(twitter): expose card binding_values on read commands

Surface tweet link-preview cards (title, description, image, domain, landing URL)
on `search`, `list-tweets`, `thread`, and `timeline` so downstream renderers
can build native-style link cards without re-fetching. Pure GraphQL-response
extractor — no query strategy, interceptor, or network changes.

extractCard returns null when the tweet has no card or when the card is
structurally empty (no url AND no title/description). Missing fields are
omitted from the output to keep JSON consumers clean.

* fix(twitter): bind cards to matching URL entity

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 12:55:19 +08:00
Ocean d1c714ecd3 feat(twitter): 新增 list-create 命令(GraphQL CreateList mutation) (#1656)
* feat(twitter): add list-create command

Adds a new `twitter list-create` command so users can create Twitter/X
lists from the CLI (the existing list commands only covered reading,
adding, and removing members). Uses the GraphQL CreateList mutation
with the same cookie + CSRF pattern as list-add, no UI clicks needed.

Args: name (positional, max 25), --description (max 100), --mode (public|private).
QueryId resolved at runtime via resolveTwitterQueryId, with a known
fallback for offline / bundle-scan misses.

* fix(twitter): pin list-create queryId + features to a working pair

Twitter's GraphQL rejects CreateList when queryId and the features
schema drift apart (DecodeException). Stop resolving the queryId
dynamically (which would pull a newer schema), hardcode a known-good
queryId, and trim features to the minimal set the real web client
sends.

Also: Twitter sometimes returns a non-fatal errors array from a
side-effect serializer while still creating the list. Check for a
valid list payload first and only treat errors as fatal when no
list came back.

* fix(twitter): add missing access:'write' on list-create (#9)

`twitter/list-create` was missing the required `access` field, which made
manifest validation fail on every opencli invocation and spam stderr with:

  ⚠  Failed to load manifest .../cli-manifest.json: Command
     twitter/list-create must declare access: 'read' | 'write'

Per docs/conventions/convention-audit.md (rule missing-access-metadata),
every adapter command must declare access. Since list-create is a create
action, set access: 'write'.

Also rebuilds cli-manifest.json — picks up missing `quoted_tweet` columns
on list-tweets / search / list-tweets-username from PR #8 (which didn't
rebuild the manifest).

* fix(twitter): harden list-create mutation contract

* fix(twitter): verify created list name

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: Kary <karyhe1019@gmail.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 12:51:57 +08:00
dependabot[bot] 8182ffbe89 chore(deps): bump tsx from 4.21.0 to 4.22.2 (#1663)
Bumps [tsx](https://github.com/privatenumber/tsx) from 4.21.0 to 4.22.2.
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.21.0...v4.22.2)

---
updated-dependencies:
- dependency-name: tsx
  dependency-version: 4.22.2
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:41:48 +08:00
dependabot[bot] 5a4984789d chore(deps): bump @types/node from 25.6.0 to 25.9.0 (#1664)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.6.0 to 25.9.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.9.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:40:20 +08:00
dependabot[bot] e1185da882 chore(deps): bump ws from 8.20.0 to 8.20.1 (#1662)
Bumps [ws](https://github.com/websockets/ws) from 8.20.0 to 8.20.1.
- [Release notes](https://github.com/websockets/ws/releases)
- [Commits](https://github.com/websockets/ws/compare/8.20.0...8.20.1)

---
updated-dependencies:
- dependency-name: ws
  dependency-version: 8.20.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:40:14 +08:00
dependabot[bot] 9446bddb60 chore(deps): bump undici from 6.25.0 to 8.3.0 (#1661)
Bumps [undici](https://github.com/nodejs/undici) from 6.25.0 to 8.3.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v6.25.0...v8.3.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 8.3.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-19 11:37:31 +08:00
Ocean 0c4bcdbb86 feat(reddit): 新增 subscribed 命令 + 在 listing 命令上暴露 id / created_utc / selftext (#1651)
* feat(reddit): subscribed command + expose id/created_utc/selftext on listing commands

Adds `opencli reddit subscribed` to list the user's subscribed subreddits,
mirroring `saved.js`'s cookie auth + AuthRequiredError pattern. Auto-paginates
via `/subreddits/mine/subscriptions.json` (max 1000 subs, default 100).

Also extends the JSON output of `popular` / `search` / `subreddit` with
`id`, `created_utc`, `selftext` (and `author` on popular) — the table
view stays clean (columns: unchanged), but `--format json` now surfaces
fields needed for downstream content-recommendation tooling that filters
by post age, dedupes by post id, or uses self-post bodies for embeddings.

Tests: 4 new vitest cases for subscribed.js (happy / auth fail / HTTP /
--limit truncation). All existing reddit tests still pass.

Note on cli-manifest.json diff: the rebuild on fork/main drops 13 entries
whose source files import lowercase `selectorError` from
`@jackwener/opencli/errors` (the actual export is `SelectorError` —
casing bug pre-existing in fork/main). Not introduced by this PR.

* fix(reddit): harden subscribed listing contract

* fix(reddit): require subreddit identity for subscriptions

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 02:36:17 +08:00
lenovobenben ec3b7dadf3 fix(zhihu): decode numeric entities in answer detail (#1629)
Co-authored-by: lihaidong <lihaidong@kingsoft.com>
2026-05-19 02:14:48 +08:00
ele-yufo 4de04c43ad feat(suno): add suno.com music-generation adapter (#1638)
* fix(suno): harden generation adapter contracts

* fix(suno): separate session auth and API failures

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 02:12:03 +08:00
Benjamin Liu 40592ea5fd fix(adapters): drop silent-sentinel row fallbacks across Apple Podcasts, Reddit, and Gitee (#1634)
* fix(adapters): drop silent-sentinel row fallbacks across Apple Podcasts, Reddit, and Gitee

Continues the audit-baseline cleanup from #1611 (lesswrong) and #1631
(wikipedia / 36kr / xiaoyuzhou / zhihu), and follows the direction set
by 71646158 (silent-empty-fallback resolutions across Douyin / Jike /
WeRead) and ee54eb8e (ignore sentinels in thrown errors).

Replaces silent-sentinel row fallbacks with the empty-string signal so
agents can tell apart "field has value Unknown" from "upstream returned
no value":

- apple-podcasts/search: episodes, genre
- reddit/saved: title
- reddit/upvoted: title
- gitee/search: language, description

All four files audited for downstream sentinel checks via
`grep -nE "=== ?['\"](Unknown|unknown|-)['\"]"`. None reference the
swapped values in control flow (verified against the v2ex/me.js class
of regression caught in #1631).

Intentionally skipped in this batch (will not flip to empty):
- gitee/trending.js:272: downstream `project.description !== '-'`
  check drives the mergedDescription fallback. Same control-flow
  sentinel pattern as v2ex/me.js. Stays on baseline.
- web/read.js x4: `'-'` lives inside rendered diagnostic lines
  (`lines.push(...)`), not row fields. Empty would render
  `  GET    /a/b` with a doubled space. UX placeholder.
- yollomi/{edit,video}.js x6: `file: '-'`, `size: '-'`, `credits: '-'`
  are user-facing status rows displayed to humans. Empty would
  collapse columns visually.
- zsxq/dynamics.js: `title: '[${d.action || 'unknown'}]'` is a
  template-literal-rendered title prefix. Empty would render `[]`.

Verified live: `opencli apple-podcasts search "lex fridman" --limit 2`
returns populated episodes/genre. `opencli gitee search "vue" --limit 2`
returns populated language/description. Baseline shrinks accordingly.

* test(adapters): add empty-signal coverage for the cluster-3 sentinel swap

Mirrors the cluster-2 test additions, pairing the sentinel value swap
in this PR with focused unit tests that mock the upstream to return
null / missing fields and assert the row surfaces an empty-string
signal instead of the old fabricated '-' / 'unknown' sentinel.

Coverage:

- clis/apple-podcasts/commands.test.js (+1 case): stubs the iTunes
  Search response with a result that has collectionId / collectionName
  / artistName populated but no trackCount and no primaryGenreName.
  Asserts episodes and genre render as '' (was '-' before this PR).

- clis/gitee/search.test.js (new): mocks Gitee's `so.gitee.com/v1/search`
  fetch with two cases - a hit that has only title + url (no langs,
  no description), and a hit that has all fields populated. Asserts
  the missing fields render as '' (was '-' before) and that populated
  fields pass through verbatim.

The reddit/saved and reddit/upvoted changes in this PR live inside a
page.evaluate template literal that fetches from reddit.com inside
the browser context, so the empty-signal branch is executed inside
the page rather than in adapter JS. They are 1-char `|| '-'` ->
`|| ''` swaps with no downstream sentinel consumer and the same JS
semantics demonstrated by the gitee + apple-podcasts tests above.

* chore: rebuild cli-manifest.json to drop stale entries from rebase

The previous rebase left a stale linkedin/people-search entry in
cli-manifest.json that was carried over from a sibling feature branch.
This branch does not include the people-search source file, so the
entry was an orphan; CI's build-manifest safety check correctly
refused to overwrite it. Regenerating with --allow-removals to drop
the orphaned entry, after which a normal `npm run build` is a no-op.
2026-05-19 01:52:52 +08:00
Ocean 942539a695 fix(twitter/lists): 跳过 "Discover new Lists" 推荐区块,避免被当成用户的 list 抓取 (#1652)
* fix(twitter): skip "Discover new Lists" recommendations in lists adapter

The X.com /<user>/lists page powers two sections from a single
ListsManagementPageTimeline GraphQL response: "Discover new Lists"
(algorithmic recommendations) and "Your Lists" (owned + subscribed).
The previous parser ignored entry.entryId entirely and returned every
list it found, so recommendations leaked through and downstream
consumers treated them as the user's own lists.

X distinguishes the sections by entry.entryId prefix:

  owned-subscribed-list-module-*  → owned + subscribed (keep)
  list-to-follow-module-*         → Discover recommendations (drop)
  cursor-*                         → pagination cursor (no list payload)

Filter on the owned-subscribed prefix in parseListsManagement and
expose isOwnedSubscribedEntry for testing. The existing test fixture
used a fictional entryId shape that no longer matches real responses;
update it to the nested-module shape Twitter actually returns and add
two new tests: one proving Discover entries are skipped, and one for
the entryId classifier.

Verified end-to-end against a live account: 10 raw entries (3 Discover
+ 7 owned/subscribed) now correctly return 7 owned/subscribed lists
with zero leakage.

* fix(twitter): harden lists parser boundary

* fix(twitter): require list-remove postcondition evidence

---------

Co-authored-by: huanghe <he.huang@extremevision.mo>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 01:00:19 +08:00
Ocean dc645a5bcf fix(youtube/transcript): 把 timedtext URL 匹配限定到当前 videoId,修跨视频字幕串台 (#1655)
* fix(youtube/transcript): scope timedtext URL match to current videoId

YouTube watch-page is an SPA — page.goto between watch URLs preserves
performance.getEntriesByType('resource') entries from prior videos.
findTimedtextUrl filtered only by lang, so a previously-viewed
same-language video's timedtext URL could be picked up by the polling
loop before the current video's fetch hook captured a fresh one,
returning the wrong video's captions to the caller.

Fix: require URLs to contain v=<currentVideoId> across all three paths:
  - in-page findTimedtextUrl (resource-buffer scan)
  - in-page isJson3TimedtextUrl (fetch/XHR hook)
  - Node-side extractSegmentsFromNetworkCapture (CDP capture)

Most likely to hit callers that reuse a single daemon tab to fetch
many transcripts back-to-back (e.g. ml-scout). Confirmed in the wild:
a Fox News Ukraine clip got Whisper Flow promo captions written to
its row when the prior call on the same tab pulled an English
Whisper Flow video.

Adds one source-contract assertion (both in-page sites use a shared
videoIdMarker) and one behavioral test (CDP capture buffer with a
stale 'v=prev' entry alongside the current 'v=abc' returns only the
current video's captions).

* fix(youtube): exact-match transcript timedtext video id

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-19 00:53:39 +08:00
jakevin f0d9aa187c docs(readme): fix skill attribution for "operate any website" use case (#1654)
Per WAWQAQ feedback: the intro section's "Let AI Agents operate any
website" bullet mistakenly references `opencli-adapter-author`, which
is the skill for **writing** adapters (correctly referenced in the
adjacent "Write new adapters" bullet). The skill for ad-hoc browser
driving is `opencli-browser` — its own SKILL.md frontmatter explicitly
says "Not for writing adapters — see opencli-adapter-author for that",
and `opencli-adapter-author` says "For ad-hoc browser driving (no
adapter), see opencli-browser instead".

Two locations affected with the same error: the intro bullet and the
Core Concepts > `browser` section. Both EN and ZH READMEs updated.
2026-05-19 00:37:32 +08:00
Benjamin Liu e82e32abc6 feat(12306): add full read adapter (stations / trains / train / price / me / passengers / orders) (#1637)
* feat(12306): add stations / trains / train read commands (no login required)

Adds a first-pass 12306 (中国铁路) adapter for the public anonymous
query endpoints. Closes the no-login slice of #1589. The
authenticated `me / passengers / orders` commands the issue
proposes are explicitly left as a follow-up.

Commands:
- 12306 stations <keyword>             search station bundle
- 12306 trains <from> <to> --date YYYY-MM-DD  availability between stations
- 12306 train <train-no> --from <s> --to <s> --date  stop list

All three use Strategy.PUBLIC + browser: false, anonymous, no cookie
storage, no CAPTCHA bypass. Sensitive behaviors the issue rules out
(ticket sniping, order submission, payment, anti-abuse circumvention,
password storage) are not implemented.

Notes worth flagging for review:

- 12306 rejects anonymous query endpoints with HTTP 302 to
  /mormhweb/logFiles/error.html. The adapter first hits
  /otn/leftTicket/init to mint JSESSIONID / route / BIGipServerotn
  cookies, then attaches them to subsequent queries. No CAPTCHA path.

- 12306 rotates the train-query endpoint name (queryO / queryZ /
  queryA / queryG) every few weeks. When the wrong name is hit the
  server returns `{c_url: "leftTicket/queryX", status: false}`
  pointing to the current correct name. The adapter walks a list of
  known names, captures the rotation hint, and retries; the runtime
  list is also mutated so subsequent calls in the same process skip
  the warm-up round trip.

- The `|`-separated train wire format includes a booking-handshake
  `secret` field at position 0. Since this PR is read-only and the
  issue explicitly rules out booking, that field is parsed but not
  surfaced in the returned row, and a unit test asserts it cannot
  leak via the public adapter contract.

- Station resolution accepts Chinese name (`上海虹桥`), telecode
  (`AOH`), full pinyin (`shanghaihongqiao`), or short alias (`shhq`).
  Anything else raises ArgumentError with a hint.

- `limit` arguments use a tight validator that throws ArgumentError
  on non-integer / out-of-range input rather than silently clamping,
  matching the typed-error pattern used in #1397 (grok) and #1370
  (coupang).

Live verified anonymously against kyfw.12306.cn:
- `12306 stations 上海 --limit 5` returns 5 stations including
  上海 (SHH) / 上海南 (SNH) / 上海虹桥 (AOH).
- `12306 trains 北京 上海 --date 2026-05-22 --limit 1` returns
  G547 06:18 -> 12:11 with first / second / business / no-seat
  availability columns populated.
- `12306 train 24000000G10L --from 北京南 --to 上海虹桥 --date 2026-05-22`
  returns the 7-stop G1 route from 北京南 through 沧州西 / 德州东 /
  曲阜东 / 南京南 / 苏州北 to 上海虹桥, with arrival / departure /
  stopover times.

Tests: 18 unit tests covering parseStationBundle, resolveStation
(including ambiguous / case-insensitive cases), validateDate,
buildCookieHeader, parseTrainRecord (including a regression test
asserting the `secret` field cannot leak into the row).

Deliberately deferred to a follow-up: `12306 price`. The
queryTicketPrice endpoint needs train_no + per-stop station_no +
per-train seat-type letters, so an ergonomic `12306 price <code>`
would cascade three API calls (trains -> stops -> price) per
invocation. Wanted to keep this PR's blast radius small. If the
maintainer prefers a Phase 1 that includes price even with the
cascading-call cost, happy to add it.

* feat(12306): add me / passengers / orders / price authenticated + price read commands

Completes the #1589 12306 (中国铁路) adapter on top of the
stations / trains / train slice landed in the prior commit of this
branch. The full command set is now:

  Anonymous (no login):
    12306 stations  search station bundle by Chinese / telecode / pinyin
    12306 trains    list trains between two stations on a date
    12306 train     list stops of one train
    12306 price     ticket prices for one train segment + date

  Authenticated (cookie session):
    12306 me        account summary (sensitive fields masked by default)
    12306 passengers  saved-passenger list (sensitive fields masked)
    12306 orders    in-progress orders (not yet ridden / refunded)

Notes worth flagging for review:

- 12306 sets the auth cookie `tk` and the session cookie `JSESSIONID`
  with `Path=/otn`. CDP `Network.getCookies` filters by URL path, so
  `page.getCookies({ url: 'https://kyfw.12306.cn' })` returns 7
  cookies without `tk` / `JSESSIONID`, even on a freshly-navigated
  logged-in tab. Switched the login check to read `document.cookie`
  via `page.evaluate`, which the current navigated page exposes
  regardless of cookie path. Centralized as `require12306Login` in
  utils.js so all three authenticated commands share the same check.

- All authenticated commands mask sensitive fields by default:
  - `me`: real name (Chinese mask), email, mobile (12306 already
    masks server-side), birth date (year only).
  - `passengers`: name + birth year by default; 12306 already masks
    ID number and mobile server-side and this adapter never decodes
    those.
  - Both expose `--include-sensitive` to opt back into the unmasked
    fields the user is entitled to see on their own account.

- `orders` returns the `queryMyOrderNoComplete` slice (orders that
  have not yet been ridden / refunded / completed). The historical
  `queryMyOrderApi` endpoint requires extra page-state handshakes
  that proved fragile when probed; left as a follow-up so this
  command can ship reliably for the immediate "what's still on my
  account" use case.

- `price` cascades three anonymous API calls per invocation:
  init -> queryByTrainNo (to resolve segment station_no within the
  train route) -> queryTicketPrice. 12306 returns prices keyed by
  one-or-two-letter seat codes (`A9` 商务座 / `M` 一等座 /
  `O` 二等座 / `WZ` 无座 / etc.) and additionally doubles some up
  as bare numeric codes (e.g. `"9": "21580"` mirrors
  `"A9": "¥2158.0"`); the bare-numeric duplicates are filtered out
  so the row set is one-per-seat-class.

- Strictly anonymous queries; no CAPTCHA / slider / SMS bypass, no
  credential storage, no ticket sniping, no order submission, no
  payment - per the issue's Non-goals list.

Live verified anonymously and authenticated against kyfw.12306.cn,
sleeping 15-25 seconds between hits to keep 12306's anti-abuse
throttle gentle:

  - 12306 me: account summary returned with real_name / email /
    mobile / birth date all masked at the adapter level, on top of
    12306's own server-side mobile mask.
  - 12306 passengers: every saved passenger returned with name
    masked to `<surname>*<...>` and 12306-side ID/mobile masks
    preserved verbatim.
  - 12306 orders: empty for this test account (no in-progress
    orders), correct EmptyResultError surface.
  - 12306 price G1 北京南 -> 上海虹桥 2026-05-22: returns
    商务座 ¥2158 / 特等座 ¥1163 / 一等座 ¥1035 / 二等座 ¥626 /
    无座 ¥626, sorted desc.

Tests: 23 unit tests (5 new beyond the prior commit's 18) cover
the mask helpers (email / mobile / Chinese name) plus the
parsePriceData filter that drops the bare-numeric duplicates and
sorts by descending price.

* fix(12306): harden browser auth boundaries

* fix(12306): tighten API drift boundaries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:58:48 +08:00
陈家名 254d51835f fix: keep media filenames in output directory (#1642)
* fix: keep media filenames in output directory

* fix(download): sanitize media filename segments

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:55:31 +08:00
Ocean 87dfb68e74 fix(browser): goto 重试时回收陈旧 page identity + 把 -32000 "Cannot find default execution context" 归类为可重试 (#1645)
* fix(browser): recover from stale page identity on goto retry (#5)

When a chrome-backed adapter pre-navigates after its cached `_page`
targetId has been invalidated (tab closed externally, identity evicted),
the extension throws `Page not found: <id> — stale page identity` and
the failure cascades — every subsequent persistent-site session call in
the same process keeps re-sending the same dead targetId.

Observed in a downstream parallel multi-platform recall: a single dead page handle
got reused across 4+ calls (twitter thread / twitter search / reddit search)
because there was no detection or recovery. The same hash appeared in
adapter pre-navigations to youtube, twitter, reddit, xhs back-to-back in
seconds, suggesting the cached `_page` was shared via persistent site
session leases (`site:youtube` etc) and never cleared after the first
"stale page identity" response.

Page.goto() now catches that specific error, drops `_page`, and retries
once without the stale id. The retry navigates via session-lease
resolution in the extension (resolveTab → preferredTabId / new owned tab),
which already handles tab eviction correctly. No effect on the happy path.

Three regression tests in src/browser/page.test.ts cover:
- recovery: stale id dropped, retry succeeds with new identity
- no-cache safety: fresh page with no _page → error propagates unchanged
  (nothing to drop, retrying would loop)
- error scoping: unrelated extension errors (e.g. disconnected) still
  surface immediately — no implicit retry

* fix(errors): classify -32000 "Cannot find default execution context" as retryable (#6)

classifyBrowserError previously only matched CDP -32000 errors when the
message contained "target" (e.g., "target closed"). It missed
"Cannot find default execution context", a CDP protocol error that also
indicates the inspected target went away — observed in a downstream parallel
adapter recall against youtube channels.

Widening the secondary check to `/target|context/i` lets the existing
target-navigation retry path (200ms delay + re-attach) recover instead of
surfacing the error as non-retryable.

* fix(browser): tighten stale page recovery notes

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 22:30:28 +08:00
lenovobenben 000c867f3a fix(zhihu): harden search pagination (#1615)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 20:44:08 +08:00
Jun 24f643af16 feat(xianyu): add inbox, messages, and reply commands (#1639)
* fix: tighten internal callback types

* feat(xianyu): add private message commands

* fix(xianyu): harden IM command contracts

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 20:41:51 +08:00
hanzi 1f30a9027b feat(linkedin): consolidate messaging and Sales Navigator commands (#1647)
* fix(linkedin): harden sales navigator commands

* fix(linkedin): harden salesnav message boundaries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:58:34 +08:00
jakevin 72f2b020de feat(weread-official): add official gateway CLI
Add the WeRead official Agent Gateway as an in-tree pure HTTP adapter with 8 commands, typed errors, tests, and docs.
2026-05-18 19:46:00 +08:00
Ocean 261b8bfbb5 build: restore +x on dist/src/main.js after tsc rebuild (#1644)
clean-dist deletes dist/ and tsc --build re-emits files without preserving
the executable bit on the bin entry. Symlinked global install then hits
EACCES on spawn until manually chmod'd. Chain a chmodSync into the existing
prebuild-manifest hook so any future rebuild self-heals.

node -e instead of bare `chmod +x` to keep the script portable (npm runs
on Windows via Git Bash where chmod is a no-op, but fs.chmodSync still
silently no-ops there too — no extra branching needed).

Co-authored-by: Kary <karyhe1019@gmail.com>
2026-05-18 19:43:28 +08:00
Benjamin Liu 1e7ebe7f27 feat(twitter): rewrite download profile path on GraphQL UserMedia with cursor pagination (#1636)
* fix(twitter): harden profile media download

* fix(twitter): fail closed on repeated media cursor

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:33:29 +08:00
Benjamin Liu 7e44e71150 fix(lesswrong): drop "Unknown" silent sentinel in author column (#1611)
* fix(lesswrong): drop "Unknown" silent sentinel in author column

Twelve lesswrong commands had `author: item.user?.displayName ?? 'Unknown'`
which masks the missing-author signal: an agent reading the result row
cannot distinguish "post has no associated user" from "author is literally
named Unknown". The repo's typed-error lint flags this pattern
(silent-sentinel rule, see scripts/check-typed-error-lint.mjs:323).

Replace `?? 'Unknown'` with `?? ''` so the missing-author case stays
visible as an empty string. Consistent with `clis/lesswrong/_helpers.js:68`
which was already using the empty-signal form.

Shrinks scripts/typed-error-lint-baseline.json from 173 to 161 entries.

Follows the same direction as #1603 (fix(adapters): surface silent empty
fallbacks).

Verified live: `opencli lesswrong frontpage --limit 2 -f json` returns
real posts with non-empty author values; empty-author rows would now
show `"author": ""` instead of fabricating `"Unknown"`.

* test(lesswrong): add empty-signal coverage for the author sentinel swap

Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with a focused unit test that
mocks the upstream LessWrong GraphQL response to return posts where
`user` is null or `user.displayName` is missing, and asserts the row
surfaces `author: ''` instead of the old fabricated `'Unknown'`.

`clis/lesswrong/frontpage.test.js` is representative for the twelve
identical `author: item.user?.displayName ?? ''` swaps across
comments / curated / frontpage / new / read / sequences / shortform /
tag / top / top-month / top-week / top-year, all of which share the
exact same expression with no downstream sentinel consumer.

The empty-signal path is exercised live too: a deleted-account or
permission-restricted user shows up in the GraphQL response with
`user: null`, surfaces as `author: ''` post this PR (was 'Unknown'
before).
2026-05-18 19:18:51 +08:00
Benjamin Liu 76a9c78261 feat(weibo): add delete command to remove user's own posts (#1620)
* feat(weibo): add delete command to remove user's own posts

Adds `opencli weibo delete <id>` so the same workflow that creates a
post can also remove one without leaving the CLI. The id positional
accepts either the numeric `idstr` (e.g. `5299336218674412`) or the
base62 `mblogid` (e.g. `QFGbHAoBS`) found in any weibo URL or in the
output of `weibo me` / `weibo feed` / `weibo post`.

Implementation lives in a single `page.evaluate` IIFE so cookies +
the XSRF-TOKEN double-submit token stay first-party:

  1. Resolve mblogid / idstr via `GET /ajax/statuses/show?id=<input>`,
     which returns the canonical `idstr`. Empty result -> 404 path.
  2. Read the `XSRF-TOKEN` cookie via `document.cookie`.
  3. `POST /ajax/statuses/destroy` with `id=<idstr>` body and the
     `X-Xsrf-Token` header.
  4. Return `[{ status: 'deleted', id, mblogid }]`.

Typed errors:
- 401 / 403 from either show or destroy -> `AuthRequiredError`
- `show` returning no `idstr` -> `EmptyResultError`
- Non-2xx HTTP on either call -> `CommandExecutionError` with status
- API response `ok !== 1` -> `CommandExecutionError` with the API msg

Closes #1619.

Verified live on macOS / opencli v1.7.22, weibo cookie session:
- Deleted the lingering test post from #1602 verification
  (idstr=5299336218674412, mblogid=QFGbHAoBS):
  `weibo delete QFGbHAoBS` returned
  `[{ status: 'deleted', id: '5299336218674412', mblogid: 'QFGbHAoBS' }]`
- `weibo me` shows `statuses: 3` (was 4 before the delete)
- `weibo post QFGbHAoBS` now throws "Post not found"

Unit tests: 8 / 8 in `clis/weibo/delete.test.js` (happy path,
empty-id, auth, not-found, show-http, destroy-http, api-msg, envelope
unwrap). Full weibo suite: 38 / 38 pass.

* fix(weibo): require delete postcondition evidence

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 19:09:47 +08:00
Benjamin Liu 030a0ad885 feat(xiaohongshu): add delete-note command to remove published notes (#1624)
* fix(xiaohongshu/publish): invoke shadow-DOM publish handler directly

XHS creator center now wraps the publish/save-draft button in an
`<xhs-publish-btn>` web component backed by a CLOSED shadow root.
Calling `.click()` on the host element does not dispatch into the
internal handler, and CDP coordinate clicks cannot penetrate the
shadow boundary. The previous text-match `button.click()` loop hit
the host element, returned `ok`, and yet the note silently stayed
on the publish page as a draft, so the adapter reported the soft
`⚠️ 操作完成,请在浏览器中确认` status while nothing was actually
posted.

Invoke the publish/save method directly on the `<xhs-publish-btn>`
host (`_onPublish` / `_onSave` and a few candidate names XHS has
shipped historically). Fall back to the legacy
`<button>`/`[role="button"]` text-match click for older
creator-center variants that still expose plain buttons.

Patch shape suggested by the OpenCLI autofix report in #1606 from
@chcc-funny (who verified an end-to-end real publish locally).

Closes #1606.

Verified live on macOS / opencli v1.7.22 / extension v1.0.15,
with creator center logged in:
- `opencli xiaohongshu publish ... --draft` -> ` 暂存成功`,
  creator home shows "草稿箱中有未发布的作品"
- `opencli xiaohongshu publish ...` (real publish) -> ` 发布成功`,
  note appeared on the account feed (visible from mobile app);
  test note deleted after verification

Unit tests: 12 / 12 in `clis/xiaohongshu/publish.test.js` pass
(mocks updated to reflect the new `{ ok, via, name|text }` invoke
result shape).

* feat(xiaohongshu): add delete-note command to remove published notes

Adds `opencli xiaohongshu delete-note <note-id>` so the workflow that
creates a note can also remove one without leaving the CLI, mirroring
`weibo delete` (#1619 / #1620).

The creator-center HTTP delete API requires the `X-S-Common` signature
header that `publish.js` deliberately avoids, so this follows the same
UI automation route. Flow:

  1. Navigate to creator note-manager
  2. Switch to "已发布" tab (delete entry only appears there; "审核中"
     and "未通过" rows have no web delete action, mobile app only)
  3. Locate the `.note` row whose `data-impression` JSON contains the
     target noteId (exact JSON-parsed match, not substring, so values
     that happen to share the noteId prefix in other fields cannot
     match the wrong row)
  4. Click the inline `<span class="control data-del">` action
  5. Click "确定" in the `.d-modal-footer` confirmation modal
  6. Poll for the row disappearing (iteration-bounded so tests with
     mocked `page.wait` exhaust the loop quickly)

Typed errors:
- /login redirect after navigation: AuthRequiredError
- 已发布 tab not found / not clickable: CommandExecutionError (UI drift)
- target noteId not present in the rendered list: EmptyResultError with
  a hint about review-state limitation
- row found but no delete action visible: CommandExecutionError
- confirmation modal missing / no 确定 button: CommandExecutionError
- row still visible after the configured poll window: CommandExecutionError

Closes #1623.

Verified live: published a test note, deleted via this adapter, follow-up
`xiaohongshu creator-notes` confirms it is gone. Unit tests: 8 / 8 cover
happy path, empty-id ArgumentError, login redirect AuthRequiredError,
tab-not-found CommandExecutionError, row-not-found EmptyResultError,
no-delete-action / no-modal / unverified-delete CommandExecutionError
paths.

Built on top of #1613 (xiaohongshu publish shadow-DOM fix) so the live
verify could exercise publish-then-delete end to end. Will rebase onto
main once #1613 lands.

* fix(xhs): make delete-note fail closed

* fix(xiaohongshu): harden delete-note boundary

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:55:57 +08:00
Benjamin Liu e29150bab5 fix(weibo/publish): replace brittle CSS-module hash with placeholder selector (#1625)
* fix(weibo/publish): replace brittle CSS-module hash with placeholder selector

`clis/weibo/publish.js` matched the compose textarea via
`textarea._input_13iqr_8`, where `_input_13iqr_8` is the Vite CSS-module
hash Weibo rebuilds on every frontend deploy. The hash drifted (current
build emits `_input_1f5hn_8`), so step 4 of the publish flow throws
"Weibo compose editor did not appear" before anything else can run.
Reported in #1602.

Replace the single hashed selector with a placeholder-text-based chain
that survives Weibo's CSS-module rebuilds:

  textarea[placeholder*="有什么新鲜事"]
  textarea[placeholder*="新鲜事"]
  textarea._input_13iqr_8     // legacy hash kept last for older variants

Two visible textareas can match on the home feed (the always-rendered
"home-strip" prompt + the post-click modal compose). Pick the LAST
visible candidate: the modal opens on top and is appended to DOM later,
so the last-visible textarea is the modal. Both the editor-visibility
poll (Step 4) and the text-insertion step (Step 6) use the same chain.

Also drops `evaluateWithArgs` from Step 8 success polling. The IIFE
there does not reference any outer args, but `evaluateWithArgs` injects
its `const`-bound parameter names into the page context, and re-running
on each iteration of the success-poll loop threw `Identifier
'maxIterations' has already been declared` after the first iteration.
This was masked previously because Step 4 always failed first; with the
selector fixed, the latent Step 8 bug surfaces. Switched to plain
`page.evaluate` to avoid re-declaring per loop.

Closes #1602.

Verified live on macOS / opencli built locally / extension v1.0.15,
weibo cookie session:
- `opencli weibo publish "明洞那家店真不错"` returned
  `status: success, message: 发布成功, text: 明洞那家店真不错`
- Confirmed via `/ajax/statuses/mymblog`: the post landed at
  `idstr=5299403716821218`, `mblogid=QFHWzsCvE`, text matches what
  was typed (proves selector chain picks the right textarea and the
  text insertion path works end-to-end)
- Cleaned up: deleted via the same `/ajax/statuses/destroy` path that
  PR #1620 exposes as `weibo delete`

Unit tests: 8 / 8 in `clis/weibo/publish.test.js` pass (mocks updated
to reflect the new `evaluate`-vs-`evaluateWithArgs` split for Step 8
and the longer poll window).

* test(weibo): lock publish placeholder selector path

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:44:46 +08:00
Benjamin Liu a50074d684 fix(adapters): drop silent-sentinel row fallbacks across 6 read commands (#1631)
* fix(adapters): drop silent-sentinel row fallbacks across 6 read commands

Continues the audit-baseline cleanup started in #1611 (lesswrong) and
the direction set by #1599 / #1603 / #1604. Replaces the
`silent-sentinel` row-data fallbacks (`'Unknown'` / `'-'` / `'unknown'`
that mask missing fields) with the empty-string signal so agents can
tell apart "field really has the value Unknown" from "upstream returned
no value".

Touched 6 read adapters, 10 baseline entries:
- wikipedia/trending: title, description
- 36kr/article: author, date, body
- xiaoyuzhou/download: podcast
- xiaoyuzhou/transcript: podcast
- zhihu/collection: dedup key + type field (the empty prefix still
  produces a unique-per-content dedup key, just without the `unknown:`
  noise)
- zhihu/download: author

Intentionally skipped (line-by-line audited):
- v2ex/me.js: `'Unknown'` is an in-band control-flow sentinel. Line 35
  initialises `let username = 'Unknown';`, line 41 uses
  `if (username === 'Unknown')` to trigger the profileEl fallback
  selector, line 75 uses the same check to raise the auth error.
  Empty would silently bypass both checks and return a row with an
  empty username as if auth succeeded.
- v2ex/daily.js: `'未知'` is user-facing 签到 success text in the
  rendered status message, not a row field. Empty would render a
  broken sentence.
- weibo/comments.js, weibo/feed.js: the sentinel sits inside an in-IIFE
  error-message string composition (`'API error: ' + (data.msg || 'unknown')`),
  not in a returned row. Empty would silently truncate diagnostic
  output. Both stay on baseline.

Verified live: `opencli wikipedia trending --limit 3` and `opencli 36kr
hot --limit 2` both return populated rows; the empty-string signal only
kicks in when the upstream value is actually missing.

* test(adapters): add empty-signal coverage for the cluster-2 sentinel swap

Per owner's pattern in 71646158 (douyin/user-videos.test.js +
jike/read.test.js + weread/search-regression.test.js), pairs the
silent-sentinel value swap in this PR with focused unit tests that
mock the upstream to return null / missing fields and assert the row
surfaces an empty-string signal instead of the old fabricated
'Unknown' / '-' / 'unknown' sentinel.

Coverage:

- clis/wikipedia/trending.test.js (new): mocks wikiFetch to return
  three articles - one with both title + description populated, one
  with no title and no description, one with title only. Asserts the
  missing fields render as '' (was '-' before this PR).

- clis/36kr/article.test.js (new): mocks page.evaluate to return a
  scrape where title is present but author / date / body are empty.
  Asserts those three fields render as '' in the row pair output
  (was '-' before this PR). Also covers the NOT_FOUND and
  INVALID_ARGUMENT error paths that already existed.

- clis/zhihu/collection.test.js (+1 case): mocks the zhihu collection
  API to return an item with content.id but no content.type. Asserts
  type renders as '' (was 'unknown' before this PR); the new dedup
  key prefix is :id rather than unknown:id, semantically identical
  for dedup purposes.

The other three files in this PR (xiaoyuzhou/download,
xiaoyuzhou/transcript, zhihu/download) use the same `|| 'unknown'` ->
`|| ''` value swap with no downstream sentinel consumer. They are
covered by the same JS language semantics the three tests above
demonstrate.

* fix(adapters): fail typed on missing row identity

* fix(adapters): tighten sentinel row identity guards

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:35:14 +08:00
Benjamin Liu 368581ea4d fix(electron-apps): move codex CDP port off 9222 to avoid browser-bridge collision (#1630)
* fix(electron-apps): move codex CDP port off 9222 to avoid browser-bridge collision

`src/electron-apps.ts` had `codex: { port: 9222 }`, but `9222` is the
default Chrome DevTools port that opencli's own browser-bridge Chrome
binds whenever `opencli doctor` is OK. On every normal opencli install
the bridge owns 9222 first, so Codex Desktop can never bind it, and
`opencli codex status` (plus every other codex command) fails with:

  App launched but CDP not available on port 9222 after 15s

`~/.opencli/apps.yaml` is documented as "additive only, does not
override builtins", so users have no supported way to relocate the
port from the user side.

Reported in #1626 with full repro (Codex Desktop + active opencli
browser-bridge Chrome) and root-cause pointer at
`dist/src/electron-apps.js:13`. Every other electron app in the
builtin registry already uses a distinct port in the 9224-9236
band (cursor 9226, doubao-app 9225, chatwise 9228, discord-app 9232,
antigravity 9234, chatgpt-app 9236); codex was the only one that
collided with the browser bridge.

Move codex to 9238 (the next free slot in that band, also the value
the reporter recommended). Update the test that asserts the port and
the two docs references that mention codex=9222. The pitfall entry
in `docs/advanced/electron.md` is also annotated to explicitly call
out 9222 as the bridge's port to avoid future collisions.

Closes #1626.

Verified live: `opencli codex status -v` now emits
`[verbose] [launcher] Probing CDP on port 9238...` (was 9222 before
the fix), confirming the code path picks up the new port. Full
end-to-end with a real Codex Desktop install is left to the reporter
and reviewer; the change here is a single-value config update plus
docs/tests sync.

Unit tests: 7 / 7 in `src/electron-apps.test.ts` pass (the codex-port
assertion updated to 9238). Both audit gates pass.

* docs(electron): sync codex CDP port guidance

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-18 18:29:14 +08:00
jakevin 0c488bbf51 docs(readme): simplify Highlights from 9 to 5 bullets (#1605)
Per WAWQAQ feedback: the previous Highlights list was bloated with hollow
marketing phrases and overlapping bullets (e.g. "Browser Automation for AI
Agents" + "AI Agent ready" said the same thing twice, "Pipeable, scriptable,
CI-friendly" is generic CLI filler).

Cut "AI Agent ready", "Account-safe" (folded into Live Browser Automation),
"Deterministic"'s second sentence (folded into Zero LLM cost), and merged
"Website → CLI" with "CLI Hub" into "100+ adapters + CLI Hub". Result is 5
concrete capability bullets instead of 9, each tied to a real feature.

EN and ZH READMEs kept in sync.
2026-05-16 20:57:33 +08:00
Jun 86792d2954 fix(barchart): surface greeks fetch failures (#1599)
* fix(barchart): surface greeks fetch failures

* fix(barchart): harden greeks failure contract

* fix(barchart): reject malformed greeks row identity

---------

Co-authored-by: 你的用户名 <你的邮箱>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 17:10:09 +08:00
jakevin ee54eb8e62 fix(audit): ignore sentinels in thrown errors
Avoid classifying fallback text inside thrown error messages as silent row data.
2026-05-16 16:51:06 +08:00
asimov 663b3387ee feat(bilibili): add summary command for the official AI video summary (#1590)
* feat(bilibili): add summary command for the official AI video summary

Adds `opencli bilibili summary <bvid>` — fetches Bilibili's official
AI-generated video summary (the "AI总结" shown on the video page) via
/x/web-interface/view/conclusion/get.

Returns the overall summary followed by the timestamped section outline,
so you get a structured digest of a video without watching it.

- Resolves cid + up_mid from the view endpoint (both required by the
  conclusion API), then calls the WBI-signed conclusion endpoint.
- Throws a clear EmptyResultError when a video has no AI summary —
  Bilibili only generates them for some videos.

Covered by clis/bilibili/summary.test.js (5 cases): summary + outline,
summary without outline, no-summary, view-resolution failure, API error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(bilibili): harden summary command contract

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 16:50:48 +08:00
jakevin 716461581a fix(adapters): surface silent empty fallbacks
Resolve the remaining silent-empty-fallback typed-error baseline entries across Douyin, Jike, and WeRead adapters.
2026-05-16 16:43:13 +08:00
hanzi 854cf01aad feat(linkedin): add messaging commands (#1597)
* feat(linkedin): add messaging commands

Add fail-closed LinkedIn inbox, connect, safe-send, and thread-snapshot commands with adapter tests and docs.

* fix(linkedin): align commands with current UI

Update inbox to read LinkedIn's normalized messaging API response and connect to use the current custom-invite route.

* chore(linkedin): sync cli-manifest.json with rebuilt inbox command

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): pass silent-column-drop gate

Drop the intermediate timestamp_ms field from inbox rows (it is converted to the timestamp column) and baseline the connect command internal profile-probe object.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): validate inbox --limit with a typed error

Reject an out-of-range --limit with ArgumentError instead of silently clamping it, satisfying the typed-error lint gate.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(linkedin): harden messaging command contracts

* fix(linkedin): reject inbox conversations without thread id

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 14:21:45 +08:00
胡大头 baf1522420 feat: add Youdao Notes shared note reader adapter (#1547)
* feat: add Youdao Notes shared note reader adapter

Add a new adapter for reading publicly shared Youdao Notes (有道云笔记).

- youdao note <url>: Fetches a public shared note by its share URL
  using browser-based DOM extraction. Extracts title, content, and
  keyword tags from the React-rendered page.
- Supports note.youdao.com and note.youdao.cn share URLs.
- Includes test coverage (3 tests) and documentation.

Closes #1418

* fix: extract full note content from React Redux store

Previously the adapter only extracted the AI summary section from the
DOM. Now it accesses the React fiber tree to read the full note content
from the Redux store (store.content.data.content), which contains the
complete note body in Youdao's structured format.

The extractor recursively walks Youdao's proprietary node format (key '8'
for text content) to reconstruct the full note as plain text.

* fix(youdao): harden shared note reader contract

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-16 14:16:25 +08:00
jakevin e3995df25c docs(readme): tighten tagline + add form-filling example (#1596)
- Replace 2-line tagline (websites/browser/electron/local + reuse logged-in browser) with a single line emphasizing the two core capabilities side by side: 把任意网站变成 CLI & 让 AI Agent 操控登录态浏览器
- Add "Help me fill out this form" as the leading opencli-browser skill example so the table surfaces browser-side capabilities, not just scraping
2026-05-16 13:15:37 +08:00
jakevin 4682ffc3de feat(douyin): restore publish and delete flow (#1587)
* feat(douyin): restore publish and delete flow

- Use upload auth v5 API instead of legacy STS2 for VOD credentials
- Switch TOS upload from AWS4-signature to gateway multipart protocol (init/transfer/finish)
- Add ApplyUploadInner → CommitUploadInner pipeline for VOD upload
- Bypass enable/transend endpoints that hang for gateway-uploaded videos
- Handle fast_detect/pre_check empty responses gracefully with retry+backoff
- Add creator backend delete fallback (via work_list id matching) when legacy delete returns permission error
- Use CommitUploadInner Vid for create_v2, not completed TOS object key
- Accept item_id as fallback when create_v2 returns no aweme_id

* fix(douyin): harden publish delete write contracts

---------

Co-authored-by: Lukin <mylukin@gmail.com>
2026-05-15 18:21:14 +08:00
胡大头 e3140af5ee feat: add Flomo memos reader adapter (#1549)
* feat: add Flomo memos reader adapter

Read your Flomo memos via the signed API.

- flomo memos: Lists recent memos with content, tags, timestamps
  Uses the Flomo v1 API with MD5 signing (secret embedded).
  Requires FLOMO_ACCESS_TOKEN env variable.
  Supports pagination via --slug cursor and --limit.

* fix: add --token arg for Flomo auth

* fix: use COOKIE strategy with browser-based API call

Use Strategy.COOKIE + browser:true instead of PUBLIC + manual token.
The adapter now reads flomo_token from localStorage in the browser,
and makes the signed API call from within the page context via fetch().
Signature is computed in Node.js and injected into the browser eval.
No env var or --token flag needed.

* fix: use access_token from localStorage.me for API auth

Flomo API requires Bearer token from access_token field in
localStorage.me (not api_token). Adapter now reads access_token
from the browser's localStorage and calls the signed API from
Node.js with the Bearer header.

* feat: add --since filter, refine flomo adapter API

- Add --since <unix_ts> to filter memos by updated_at
- Add --limit 200 to fetch all memos in one call
- Mark --slug as experimental (cursor pagination unreliable)
- 5 tests passing

* feat: add images column to flomo memos output

* docs: add flomo adapter documentation

* fix: use clampInt and rebuild manifest

* fix(flomo): harden memos reader contract

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 18:08:56 +08:00
ele-yufo 43f5c6e1cf fix(chatgpt): unwrap page.evaluate envelope across browser commands (#1580)
* fix(chatgpt): unwrap page.evaluate envelope across browser commands

The browser bridge wraps every `page.evaluate(...)` return value in a
`{ session, data }` envelope. Adapters that read `.length` or
`Array.isArray(payload)` directly on the envelope silently see "no
data" — same failure mode addressed for `xiaohongshu`/`rednote` in
#1561 and `weibo` in #1568.

This sweep applies the same `unwrapEvaluateResult` helper across every
chatgpt `page.evaluate` consumer site, plus typed shape guards
(`requireArrayEvaluateResult`, `requireObjectEvaluateResult`) on the
critical extraction paths so envelope misses fail loud instead of
silently returning empty.

## Sites wrapped

`clis/chatgpt/utils.js`:

- `currentChatGPTUrl` — string URL
- `getPageState` — login/composer probe object
- `sendChatGPTMessage` — composer write + send-button readiness
- `getVisibleMessages` — conversation transcript array
- `getConversationList` / `extractConversationLinks` — sidebar items
- `waitForChatGPTUploadPreview` — image upload readiness probe
- `uploadChatGPTImages` fallback — DataTransfer upload result
- `isGenerating` — boolean "still generating?" probe
- `getChatGPTVisibleImageUrls` — visible image URL array
- `waitForChatGPTImages` — inline `window.location.href` poll
- `getChatGPTImageAssets` — exported asset array

`clis/chatgpt/image.js`:

- `currentChatGPTLink` — used for error hints + conv link reporting

## Drive-by

`getChatGPTImageAssets` was also passing a redundant `urls` second arg
to `page.evaluate(string, urls)`. The IIFE inside the string already
receives the URL list via the `${urlsJson}` template substitution, and
the browser bridge guard in `browser/utils.ts` rejects the second form
for string scripts with:

    page.evaluate string input does not accept args;
    use page.evaluate(fn, ...args) instead

So `opencli chatgpt image <prompt>` blows up at the download step
without `--sd true`. Drop the trailing arg as part of the asset-export
cleanup. (This supersedes #1556 — same one-line fix is included here.)

## Validation

- `npx tsc --noEmit` — clean
- `npx vitest run --project adapter clis/chatgpt/` — 38/38 pass
  (25 existing + 13 new in `envelope.test.js`)
- `npm test` — 3644 passing across 364 files
- Live (browser bridge, daemon v1.7.19):
  `opencli chatgpt image "<prompt>"` → end-to-end generate + download
  succeeds; the envelope wrap is defensive in 1.7.19 (no envelope
  observed yet), but pre-empts the same silent-failure mode that hit
  the merged xiaohongshu/weibo PRs.

* fix(chatgpt): fail fast on malformed evaluate payloads

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:54:44 +08:00
Yabin Zheng 68ef95659f Fix YouTube transcript caption fetching (#1499)
* fix(youtube): unwrap transcript caption results

* fix(youtube): validate transcript caption info shape

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:49:02 +08:00
chonglinghuc c922a39a7d 微博新增用户搜索导出博文命令opencli weibo search_by_user 1670458304 --start 2025-06-01 --end 2025-06-02 (#1379)
* docs: add weibo search_by_user command design spec

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(weibo): add search_by_user helper function tests

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* feat(weibo): add search_by_user command for timed post download to Markdown

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(weibo): remove dead hasori ternary and hardcoded hastext/haspic filters

The hasori ternary always evaluated to 1 (bug), and hastext=1 + haspic=1
silently excluded text-only and link-only posts from results.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(weibo): add integration tests for search_by_user helpers

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* bak

* fix(weibo): reshape user posts into read adapter

---------

Co-authored-by: andrew.asa <asa.andrew@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-15 17:44:47 +08:00
jakevin aae6e823b4 chore(release): 1.7.22 (#1586)
Release / release (push) Has been cancelled
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
External CLI ergonomics + two adapter envelope/auth fixes.

- feat(external): longbridge CLI passthrough (#1584)
- feat(external-cli): brand alias rendering for ntn/dws/wecom-cli (#1585)
- fix(boss): map code=24 → AuthRequiredError (#1573)
- fix(weibo): unwrap page.evaluate envelope in read adapters (#1568)
2026-05-15 17:30:34 +08:00
jakevin 3f62cc45bf feat(external-cli): render brand alias for ambiguous executable names (#1585)
`ntn`, `dws`, and `wecom-cli` are opaque executable names — users seeing them
in `opencli list` or root help have no way to know they correspond to Notion,
DingTalk Workspace, and 企业微信. Repurpose the existing `package` field to
double as a human-readable brand label, so help output renders as
`ntn(notion)`, `dws(DingTalk Workspace)`, `wecom-cli(企业微信)`.

- `src/external-clis.yaml`: add `package:` to ntn / dws / wecom-cli
- `src/external.ts`: update JSDoc on `package` to cover both upstream
  distribution names (tg-cli, discord-cli) and brand labels (notion, 企业微信)
- `src/cli.ts:629` (`opencli list`): use `formatExternalCliLabel` so the
  listing matches root help, which already used it
- `src/external.test.ts`: regression test for brand-alias labels

Verification:
- npx vitest run --project unit src/external.test.ts: 9/9 pass
- npm run typecheck: clean
- npm run build: 813 manifest entries
- Smoke: `opencli list` and `opencli --help` both render the new labels
2026-05-15 16:37:27 +08:00
jakevin b6f352b318 feat(external): add longbridge cli (#1584) 2026-05-15 16:27:08 +08:00
Benjamin Liu dadf01b56f fix(weibo): unwrap page.evaluate envelope in read adapters (#1568)
* fix(weibo): unwrap page.evaluate envelope in read adapters (#1567)

`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, so all weibo cookie-strategy read adapters
silently dropped their results on v1.7.19:

- `getSelfUid` returned the envelope object instead of the uid string,
  so `'10001' + uid` produced `'10001[object Object]'` and every
  feed/me/favorites request hit a broken list_id.
- `feed`, `hot`, `comments`, `search`, `favorites` did `Array.isArray`
  on the envelope (always false) and returned `[]`.
- `me`, `user`, `post` returned the envelope wrapper itself instead of
  the inner profile/post object.

Same pattern as #1561 for xiaohongshu/rednote. Adds an
`unwrapEvaluateResult` helper to `clis/weibo/utils.js` (kept local
rather than cross-importing from `xiaohongshu/search.js` since weibo
is an unrelated site) and wraps every `await page.evaluate(...)` in
the 8 read adapters plus the two helper calls in `getSelfUid`.

Skipped `publish.js` (write command, out of scope for this read fix).

Verified live:
- `opencli weibo hot --limit 3` returns 3 real trending items
- `opencli weibo feed --limit 3` returns 3 timeline posts with
  correct `https://weibo.com/<uid>/<mblogid>` URLs (proves
  `getSelfUid` unwrap works)
- `opencli weibo me` returns the logged-in profile object
- All 20 weibo unit tests pass (6 new for `unwrapEvaluateResult`)

* fix(weibo): fail typed on malformed evaluate payloads

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 22:34:43 +08:00
Benjamin Liu 1239798d04 fix(boss): map code=24 (identity mismatch) to AuthRequiredError (#1573)
Recruiter-only BOSS commands (recommend, joblist, stats, resume, mark,
exchange, invite, greet, batchgreet) returned a generic
`COMMAND_EXEC: 请切换身份后再试 (code=24)` when called from a job-seeker
account. The original error hid the actionable bit: this command set
needs a recruiter (BOSS-side) account.

chatlist / chatmsg already special-case code=24 by falling back to the
geek-side fetch when --side=auto. Recruiter-only commands have no
geek-side equivalent and were just leaking the raw API code.

Fix: add a `checkRecruiterSide` step inside `assertOk` that maps
code=24 to AuthRequiredError with a clear message. All 9 recruiter-only
commands inherit it through their existing `bossFetch` calls; no
adapter-level changes needed. chatlist / chatmsg are unaffected because
they use `allowNonZero: true` and never hit the auto-error path.

Closes #1572.
2026-05-14 22:26:28 +08:00
jakevin 9ccc896585 chore(release): 1.7.21 (#1571)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 19:36:41 +08:00
jakevin 1a69f40a80 fix(social): use ephemeral adapter site sessions (#1569) 2026-05-14 19:22:42 +08:00
J.Chen 300607f692 fix(facebook/feed): add fallback extraction for empty article nodes (#1538)
* fix(facebook/feed): add fallback extraction for empty article nodes

Add fallback extraction for Facebook feed posts when [role=article] nodes exist but contain empty text. Includes diagnostic errors, content/author cleanup, nested-container dedupe, and an evaluate-script syntax regression test.

* fix(facebook): bound feed fallback extraction

* fix(facebook): keep feed fallback available after chrome articles

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 18:42:50 +08:00
J.Chen 42b5a4e68d feat(boss): support job-seeker chatlist and chatmsg (#1539)
* feat(boss): support job-seeker chatlist and chatmsg

* fix(boss): type chat-side failure boundaries

* fix(boss): guard malformed chat API payloads

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 18:41:02 +08:00
jakevin bccd275d66 test(extension): cover adapter group tiebreaker (#1566) 2026-05-14 18:04:07 +08:00
胡大头 edfa5f0da3 feat: add DuckDuckGo, Brave, and Yahoo web search adapters (#1546)
* feat: add DuckDuckGo, Brave, and Yahoo web search adapters

Add three new search engine adapters with browser-based DOM extraction:

- duckduckgo/search: Search DuckDuckGo via html.duckduckgo.com
  Supports region, time filters, and XHR-based pagination (--offset)
- duckduckgo/suggest: Search suggestion autocomplete (no browser needed)
- brave/search: Search Brave Search via search.brave.com
  Supports GET-based pagination (--offset)
- yahoo/search: Search Yahoo (Bing-powered) via search.yahoo.com
  Supports GET-based pagination (--page)

All search adapters use Strategy.PUBLIC with browser:true, navigating
the target site and extracting results via page.evaluate() DOM queries.
Includes full test coverage (16 tests).

* fix: use clampInt from shared utils and add adapter docs

- Replace Math.max/Math.min patterns with clampInt() from _shared/common.js
  to pass the typed-error-lint gate (4 silent-clamp violations resolved)
- Add adapter documentation for duckduckgo, brave, and yahoo to fix
  the doc-coverage CI check
- Regenerate cli-manifest.json and typed-error-lint-baseline.json

* fix: avoid silent-column-drop overlap in brave/yahoo extractors

Change buildExtractorJs to return arrays instead of objects whose keys
matched columns. This prevents silent-column-drop audit false positives
as per opencli-adapter-author conventions.

* fix(search): tighten browser search adapters

* chore(search): drop baseline churn

* fix(duckduckgo): execute search extractor safely

* fix(yahoo): reject unsafe redirect targets

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:54:34 +08:00
J.Chen 16b02bcc58 fix(extension): reuse existing adapter tab group (#1541)
* fix(extension): reuse existing adapter tab group

* fix(extension): choose best existing adapter group

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:15:39 +08:00
Iris Chen 5af2ff1d6c fix(xiaohongshu,rednote): unwrap page.evaluate envelope in search adapter (#1561)
* fix(xiaohongshu,rednote): unwrap page.evaluate envelope in search adapter

`page.evaluate(...)` returns a `{ session, data }` envelope rather than
the raw IIFE return value, but the search adapters were calling
`Array.isArray(payload)` directly on the envelope. `Array.isArray` is
always false on the envelope, so every search result was silently
dropped — status=success, exit 0, empty array, no error.

The rednote adapter had this same bug; both share `buildSearchExtractJs`
from `xiaohongshu/search.js`.

Introduces `unwrapEvaluateResult(payload)` as a shared helper in
`clis/xiaohongshu/search.js` (re-exported via the existing import line
from `rednote/search.js`). The helper is a defensive ternary: it
unwraps when payload looks like an envelope with an array `.data`,
otherwise it passes the value through unchanged. This keeps the change
back-compat with bridge versions that return the raw value, and
preserves the existing `Array.isArray(payload)` typecheck at each call
site.

Verified manually against `opencli xiaohongshu search "补墙洞"` (a query
known to return 20+ results in a logged-in browser tab): previously
`[]`, now returns the expected ranked rows with all declared columns
(`rank, title, author, likes, published_at, url`) populated.

Adds 5 unit tests for `unwrapEvaluateResult` covering raw array passthrough,
envelope unwrap, non-envelope object passthrough, null/undefined safety,
and the "data is not an array" guard. The existing 19 search tests in
`clis/xiaohongshu/search.test.js` still pass — the unwrap is invisible
to the existing mocks which already return raw arrays.

* fix(xhs): unwrap search evaluate envelopes

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 17:05:58 +08:00
jakevin 9c25bc7009 fix(ci): add Windows native binding lock entries (#1563) 2026-05-14 16:45:00 +08:00
jakevin 8c88a3cbf3 chore(release): 1.7.20 (#1562)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 16:33:19 +08:00
jakevin 9c4f4a3d30 fix(cli): show external CLI package aliases (#1560) 2026-05-14 16:28:18 +08:00
jakevin 29c135b656 refactor(notion): replace built-in CDP adapter with external ntn CLI (#1559)
* refactor(notion): replace built-in CDP adapter with external ntn CLI

Notion has shipped an official CLI at https://ntn.dev. It uses the
public Notion API (blocks / databases / properties / comments) instead
of reverse-engineering the Desktop UI, so it survives Notion app
updates and exposes a wider command surface than the in-tree adapter
could.

Changes:
  - `src/external-clis.yaml` — register `ntn` as first-class external CLI
    (binary `ntn`, homepage ntn.dev, install via the shell-pipe script
    on mac/linux)
  - `clis/notion/` — entire directory removed (8 commands: status /
    search / read / new / write / sidebar / favorites / export)
  - `docs/adapters/desktop/notion.md` — removed
  - `docs/.vitepress/config.mts` — drop nav entry
  - `docs/adapters/index.md` — drop adapter row
  - `README.md` / `README.zh-CN.md` — drop notion from feature lines,
    drop adapter table row, add `ntn` to CLI hub examples
  - `docs/index.md` / `docs/zh/index.md` / `docs/guide/getting-started.md`
    — drop notion from electron-control feature copy
  - `skills/opencli-usage/SKILL.md` — drop notion from electron list
  - `cli-manifest.json` — rebuilt with --allow-removals=8

Migration for users:
  `curl -fsSL https://ntn.dev | bash`  (or `opencli external install ntn`)
  Then use `opencli ntn <command>` in place of `opencli notion <command>`.

Rationale: the in-tree adapter was reverse-engineered against Notion
Desktop CDP and shipped only 8 commands. The official CLI gives users
the full Notion API surface and reduces our maintenance burden to zero.
Same pattern as gh / obsidian / lark-cli / tg-cli / discord-cli / wx-cli.

Verification:
  - `npx tsc --noEmit` clean
  - `npx vitest run --project unit` → 1091/1 skipped
  - `npm run build` (with --allow-removals=8) — manifest 809 entries
  - grep notion in user-facing docs (README / docs / skills) — only
    descriptive mentions remain in non-blocking places (comparison /
    site-recon / electron how-to / design doc), no broken adapter
    references

* fix(notion): align ntn external migration

* docs(notion): clarify ntn manual install
2026-05-14 16:12:17 +08:00
jakevin 7edf53783f fix(daemon): report unknown browser command results (#1558) 2026-05-14 14:30:13 +08:00
jakevin af7b94152f feat(twitter): add extractMedia parity to bookmarks + bookmark-folder (#1555)
Mirrors PR #1464 (list-tweets) and the timeline/search/tweets/likes/thread
family: spread `...extractMedia(legacy)` into the row and surface
`has_media` + `media_urls` columns. Pure parity, no behavior change for
existing callers — media keys do not collide with the original columns.

- bookmarks.js: import `extractMedia` from ./shared.js, spread into
  extractBookmarkTweet row, append columns, export __test__.
- bookmark-folder.js: same change on extractFolderTweet, export
  extractFolderTweet via __test__.
- bookmarks.test.js (new): baseline + photo + video + entities-only
  fallback + dedup + envelope + empty-envelope (8 tests).
- bookmark-folder.test.js: update existing baseline expectation with
  has_media/media_urls, add 3 new media tests (photo / mp4 / no-media).
- cli-manifest.json: regenerated; only the two `columns` entries change.

Reverse-validated: tests fail when extractMedia spread is removed.

Audits unchanged: typed-error-lint 189/189, silent-column-drop 102/103
(pre-existing main resolution noted but not consumed here).
2026-05-14 14:24:31 +08:00
Ocean 6b26aedd56 feat(twitter/list-tweets): include media via extractMedia (parity with timeline/search) (#1464)
* feat(twitter/list-tweets): include media via extractMedia (parity with timeline/search)

list-tweets was the only X recall path that dropped media. timeline.js and
search.js both call extractMedia(legacy) and emit has_media/media_urls;
list-tweets returned only text fields, so downstream consumers (e.g.
ml-scout's rate UI) couldn't render image/video thumbnails on tweets pulled
from a list timeline.

Changes:
- Import extractMedia from ./shared.js
- Spread extractMedia(legacy) into extractTimelineTweet return
- Add has_media, media_urls to columns array (--format columns parity)
- Update unit test to assert the new shape; add coverage for photo and
  video extraction

* chore(manifest): rebuild cli-manifest.json for list-tweets media columns

---------

Co-authored-by: ml-scout <ml-scout@anthropic.com>
2026-05-14 14:01:34 +08:00
jakevin 68b18cdbcd fix(extension): coalesce daemon websocket connects (#1554) 2026-05-14 13:57:38 +08:00
J.Chen cddc84776c docs(browser): clarify named session lifecycle (#1542)
* docs(browser): clarify named session lifecycle

* docs(browser): clarify owned versus bound sessions

---------

Co-authored-by: Jeff Chen <jeff@adtiming.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-14 13:38:41 +08:00
J.Chen 4f5fcd9acb fix(extension): keep active daemon websocket
Keep stale Browser Bridge WebSocket events from clobbering the active daemon connection.\n\nCo-authored-by: Jeff Chen <jeff@adtiming.com>
2026-05-14 13:38:05 +08:00
jakevin 40b2f75098 feat(external)!: drop -cli suffix from tg/discord/wx subcommand names (#1544)
The opencli external-CLI name is the user-typed subcommand; the binary is
what gets executed. The convention everywhere else (`gh`, `docker`,
`obsidian`, `vercel`, `dws`) is `name == binary`. Three entries violated
the convention: `tg-cli` / `discord-cli` / `wx-cli` registered an
opencli name with a `-cli` suffix that does NOT exist on the binary,
forcing the awkward double-prefix `opencli discord-cli dc` instead of
`opencli discord dc`.

The README's example column already showed the desired form
(`opencli tg search`, `opencli discord recent`, `opencli wx search`) —
only the yaml registration was out of sync.

Renames in `src/external-clis.yaml`:

* `name: tg-cli`     → `name: tg`       (binary: `tg`)
* `name: discord-cli`→ `name: discord`  (binary: `discord`)
* `name: wx-cli`     → `name: wx`       (binary: `wx`)

The `binary`, `homepage`, and `install` fields are unchanged — the
underlying packages (`kabi-tg-cli`, `kabi-discord-cli`, `@jackwener/wx-cli`)
keep their published names.

Other entries left as-is: `lark-cli`, `wecom-cli`, and `dws` already have
`name == binary` (their actual binaries are `lark-cli`, `wecom-cli`, `dws`).

BREAKING CHANGE: `opencli tg-cli ...`, `opencli discord-cli ...`,
`opencli wx-cli ...` no longer resolve. Use `opencli tg ...`,
`opencli discord ...`, `opencli wx ...` instead. The feature is recent
(shipped 2026-05) so impact is expected to be minimal.
2026-05-14 04:13:39 +08:00
jakevin feab24f76c chore(release): 1.7.19 (#1543)
E2E Headed Chrome / e2e-headed (macos-latest) (push) Has been cancelled
E2E Headed Chrome / e2e-headed (ubuntu-latest) (push) Has been cancelled
Release / release (push) Has been cancelled
2026-05-14 02:33:56 +08:00
jakevin 8ef7e903b8 feat(twitter): default tweets to logged-in user + fix sibling envelope-unwrap silent bug (#1531)
* feat(twitter): default tweets to logged-in user + fix sibling envelope-unwrap silent bug

Primary: make `opencli twitter tweets` default to the logged-in user
when no username is given, so agents can pull their own posts without
needing to know their own handle. Mirrors the existing self-detection
pattern in twitter/profile and twitter/likes (AppTabBar_Profile_Link
probe on /home, then UserByScreenName lookup). Description + help
string now mention the default so agents discover it.

Consistency pass — profile/likes/following/followers: the
self-detection in these four siblings was silently broken because
page.evaluate() primitive returns come back through the CDP bridge
wrapped as `{session: 'site:twitter', data: '/<handle>'}` (same
envelope root cause as #1525). They called `.replace()` directly on
the envelope object → TypeError surfaced as AUTH_REQUIRED 'Could not
detect logged-in user', even for logged-in users. Wrap each probe
with unwrapBrowserResult so the bare href string survives. Also:
- Add an explicit page.goto('/home') + page.wait(primaryColumn)
  before the probe in likes/following so the AppTabBar sidebar is
  guaranteed rendered (framework pre-nav lands on bare x.com without
  the sidebar mounted).
- following.js: switch its probe from the function-literal form
  `() => {...}` to a template-string. Confirmed live: function-literal
  silently drops primitive returns entirely — bridge returns
  `{session}` with no `data` field at all, while template-string
  returns `{session, data}` as expected.

Out of scope (pre-existing, flagged as follow-up): likes/following
have additional downstream evaluate paths (userId/GraphQL fetch) that
still drop or envelope their results; they return [] or
'Could not find user' even after this PR. Same daemon-side bug class
as #1525.

Live-verified:
  opencli twitter tweets --limit 2     → own tweets (@jakevin7)
  opencli twitter profile              → own profile

Tests 227/227, audits typed-error-lint 189 + silent-column-drop 103
unchanged, manifest stable at 816 entries.

* fix(twitter): validate self-detected handles

* fix(twitter): unwrap downstream self evaluate results
2026-05-13 22:51:48 +08:00
ppop123 7c5bafd49b fix(twitter): repair list-add / list-tweets / lists / following after 2026-05 changes (#1503)
* fix(twitter): unwrap page.evaluate primitive returns in lists/list-tweets/following

The opencli >=1.7.x browser bridge wraps page.evaluate's primitive return
values as { session, data: <value> }. Adapters that destructure .data
inline (e.g. data.queryId, data.viewer) keep working because the wrapper
spreads object-typed responses to the top level, but ones that consume
the return value as a bare string broke:

- twitter list-tweets: the dynamically resolved queryId (a string) became
  {session, data:"..."}. Interpolating that into the GraphQL URL produced
  /i/api/graphql/[object Object]/ListLatestTweetsTimeline, giving "HTTP
  400: queryId may have expired".
- twitter lists: same on ListsManagementPageTimeline queryId.
- twitter following: same shape bug on the href read from the profile
  link, producing "TypeError: href.replace is not a function" when no
  --user is given.

Add a small unwrap() helper at each call site so primitive returns are
extracted from the wrapper before use. Object-typed GraphQL responses
are left as-is since they rely on spread semantics.

* fix(twitter): rewrite list-add to use ListAddMember GraphQL mutation

In 2026-05 X replaced the "Add/remove from Lists" modal dialog with a
full-page route (/i/lists/add_member). The previous UI flow no longer
works:

  Save button not found in dialog (X expected text Save/Done).
  Dialog structure may have changed.

The mutation that the dialog used to fire (ListAddMember) is still the
right primitive — and the surrounding adapter already calls X GraphQL
APIs directly to resolve userId and verify member_count. Drop the UI
flow entirely and call ListAddMember directly via fetch in the page
context.

Wins:
- Works again on current X UI (verified 2026-05-12 on x.com).
- ~10x faster: no goto-profile + click-caret + scroll-dialog round trips.
- One less moving piece — no dependency on Chrome extension's nativeClick
  for this command.

Implementation notes:
- LIST_ADD_MEMBER_QUERY_ID is a 2026-05 fallback; resolveTwitterQueryId
  does live lookup from the loaded client-web bundle, matching the
  pattern already used elsewhere in the twitter clis.
- X's ListAddMember response routinely contains a non-fatal partial
  decode error on default_banner_media_results (code 214, Validation /
  BadRequestError) alongside a fully populated data.list. We treat the
  call as failed only when data.list / member_count is missing, and
  ignore decode-flavored errors confined to banner fields.
- Same opencli >=1.7.x { session, data } primitive-wrap behavior that
  the previous commit addressed applies here: userId from the
  UserByScreenName call needs unwrap before being interpolated into
  the mutation body, otherwise X parses "[object Object]" as user_id
  and returns "strconv.ParseInt ... invalid syntax".

Verified flows:
- noop (already a member) → status: noop, member_count unchanged.
- new add (e.g. @AnthropicAI on a fresh list) → status: success,
  member_count incremented.

Trade-off: rejection signals (e.g. X declining to add @deepseek_ai)
look indistinguishable from noop at the response level, since X returns
HTTP 200 with member_count unchanged. Documented in the success message.

* fix(twitter): integrate list media and harden list-add

---------

Co-authored-by: wangyan <wy@wang-yan-Air.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 22:35:57 +08:00
jakevin f66996a148 fix(twitter): raise cursor pagination guard
Fix hidden pagination ceilings across Twitter cursor-pagination adapters.\n\nCo-authored-by: Mingming Lou <1109198+lmmsoft@users.noreply.github.com>
2026-05-13 22:14:05 +08:00
jakevin 4fac911425 feat(zhihu): add answer-detail to fetch a single answer's full content (#1528)
* feat(zhihu): add answer-detail to fetch a single answer's full content

The existing `zhihu answer` adapter is a write (post an answer); the
listing `zhihu question` truncates each answer's body to 200 chars.
There was no way to fetch one specific answer's full content by id.

New read adapter `zhihu answer-detail`:

- Accepts a bare numeric answer id, a typed target `answer:<qid>:<aid>`,
  or a full Zhihu answer URL (the form you paste from a browser).
- Calls `/api/v4/answers/<aid>?include=content,voteup_count,...,question`
  inside the cookie-bearing page context (Strategy.COOKIE).
- Returns a single row with id / author / votes / comments /
  question_id / question_title / url / created_at / updated_at /
  content. The content column is the full stripped answer body by
  default — no silent truncation. `--max-content N` is an opt-in user
  cap (mirroring the wikipedia `page` flag), and `--max-content 0`
  (the default) means "no cap, full content".

Important precision note: Zhihu answer ids since 2024 routinely
exceed `Number.MAX_SAFE_INTEGER` (the test fixture uses the real id
`1937205528846655537`). `data.id` is round-tripped through browser
`JSON.parse` and would round to `1937205528846655500`, so the adapter
deliberately ignores `data.id` for the canonical row id and anchors
it to the already-validated input string instead. A regression test
locks this contract in by mocking `data.id = 0` and asserting the row
still carries the parsed input id.

Typed errors: bad input → INVALID_INPUT; 401/403 → AuthRequiredError;
other HTTP / null → FETCH_ERROR. No silent fallbacks, no sentinel
strings.

Live-verified against the example URL — fetched 5547 votes / 165
comments / 1937205528846655537-end-to-end. 16 unit tests, audits
unchanged (typed-error-lint 189/189, silent-column-drop 103/103),
manifest 816→817.

* fix(zhihu): tighten answer-detail contracts
2026-05-13 21:47:37 +08:00
xcd_git b52da639a3 fix(google-scholar/search): wrap evaluate return to fix serialization (#1525)
* fix(google-scholar/search): wrap evaluate return to fix serialization

Same issue as google/search: page.evaluate() serializes JS arrays as
plain objects across the CDP boundary, causing Array.isArray() to
return false. The adapter silently returned [] instead of results.

Also replace fixed page.wait(3) with selector-based wait for
.gs_r.gs_or.gs_scl with a 3s fallback.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fix(google-scholar): type search evaluate payload

* chore: rerun google scholar search checks

---------

Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 19:07:39 +08:00
Joseph赛博阿隆 b1dca04ddd fix(xiaohongshu): parseLikes should handle 2.1w / 1.5万 / 1.2k shortforms (#1504)
* fix(xiaohongshu): parseLikes should handle 2.1w / 1.5万 / 1.2k shortforms

Xiaohongshu renders top-popular comment like-counts as shortened
strings like '2.1w' / '1.1万' / '1.2k' once they exceed ~10 000.
The previous parseLikes only matched bare digits via /^\d+$/ and
silently returned 0 for any shortform, which inverted the sort
order: the highest-liked comments (often 10k+) ranked last while
mid-tier comments with plain numeric counts (e.g. 7569) appeared
on top.

Repro on any popular xiaohongshu thread (>10 000 likes on a top
comment): with --format json the most-upvoted parent rows show
"likes": 0.

This patch keeps the original fast path for plain integers and
adds a single regex for the well-known shortform suffixes:

  - w / 万 -> *10000
  - k / 千 -> *1000
  - trailing '+' tolerated (e.g. '999+')
  - unknown shapes still fall back to 0 (no behavior change)

Note: parseLikes runs inside the IIFE injected via page.evaluate(),
so the existing comments.test.js mock harness (which stubs
evaluate's return value directly) does not exercise it. A future
refactor that exports parseLikes for direct testing would be a
separate change.

Affects both top-level comments and 楼中楼 sub-replies (same
helper).

* fix(xiaohongshu): parse comment like shortforms safely

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:47:20 +08:00
jakevin f481585ba1 chore: drop util.styleText to support Node v20+ (#1524)
* chore: drop util.styleText to support Node v20+

util.styleText was added in Node v21.7.0 / v20.12.0. v21.0.0-v21.6.x and
v20.0.0-v20.11.x throw `SyntaxError: ... styleText` at startup because the
import resolves before any user code runs (a real user reported this on
v21.2.0).

OpenCLI is primarily agent-facing — terminal colors are noise to consumers,
and the [OK] / [WARN] / [FAIL] / ℹ / ⚠ / ✖ markers we already write carry
the semantic info that colors only repeated. Strip styleText entirely from
logger / output / doctor / tui / update-check / cli / download/progress /
commands/daemon and clean up the resulting awkward `${'literal'}` template
fragments. engines.node now reads ">=20.0.0".

This removes the Node-version coupling that A/B fixes would only have
papered over.

* fix(runtime): truly support Node v20+ by aligning guard + undici

Follow-up to the styleText removal: declaring engines.node >=20.0.0 is
not enough on its own. Two coupled barriers remained:

- src/runtime-detect.ts: MIN_SUPPORTED_NODE_MAJOR = 21 explicitly
  rejected v20 at startup
- undici@^8.0.2 declares engines.node >=22.19.0; Node 20/21 crash on
  webidl.util.markAsUncloneable before any user code runs

Lower the guard to 20 and downgrade undici to ^6.25.0 (engines >=18.17,
retains Agent / EnvHttpProxyAgent / fetch / Dispatcher). Smoke-tested
--help / doctor / list on Node v20.0.0, v21.2.0, v22.22.2. 213/213
targeted unit tests pass.
2026-05-13 18:33:21 +08:00
lenovobenben 723f2b9147 feat(zhihu): paginate question answers and recommendations (#1517)
* feat(zhihu): paginate question answers and recommendations

* fix(zhihu): drop Math.min limit clamp and 'unknown' sentinel

Two audit-driven fixes on top of feat/zhihu-pagination-recommend:

1. question.js: replace `Math.min(answerLimit, 20)` with a named
   constant `ZHIHU_PAGE_SIZE = 20`. The Zhihu API caps `limit` at 20
   per request anyway, and the pagination loop already trims to the
   user-requested `answerLimit` via `answers.length >= answerLimit`,
   so the Math.min silent-clamp was both unnecessary and tripped the
   silent-clamp audit. Updates the existing unit test to expect the
   API-max page size in the fetch URL with an explanatory comment.

2. recommend.js: rebuild the dedup key without the `'unknown'`
   sentinel. The old form `\`\${target.type || 'unknown'}:\${target.id}\``
   collapsed distinct typed items into the same bucket whenever
   `target.type` was missing, and tripped the silent-sentinel audit.
   New form: prefer `type:targetId`, fall back to `__feed:item.id`,
   and when neither id is available keep the row but skip dedup
   (surfacing potentially-duplicate items beats silently dropping
   them).

Audits unchanged (typed-error-lint 189/189, silent-column-drop
103/103). All 88 zhihu tests pass.

---------

Co-authored-by: lihaidong <lihaidong@kingsoft.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:28:44 +08:00
Benjamin Liu 2babed84e9 fix(xiaohongshu+rednote/search): fall back to href-based note cards when section.note-item class is dropped (#1506) (#1507)
Issue #1506 reports `opencli xiaohongshu search` returning `[]` even though
the page visibly has results. Trace evidence: xhs ships a render variant
where each note card is a bare `<section>` (no `note-item` class), so
the three `section.note-item` selectors in this file all match zero
elements.

Three call sites in the shared search IIFEs now use the same defensive
selector strategy: try the legacy `section.note-item` class first, then
fall back to any `<section>` that wraps a `/search_result/...` or
`/explore/...` link. The change is in the xiaohongshu file so the
rednote adapter (which imports `buildSearchExtractJs` and
`buildScrollUntilJs` from here) picks it up automatically.

Extraction-side title selector also gets a fallback: when no
`.title` / `.note-title` element matches, read the first `<span>`
inside the search-result link, which is where the bare-section render
puts the caption per the trace.

## Verification

`npx vitest run --project adapter clis/xiaohongshu/`: 105/105 green
(existing test suite unchanged, passes on both legacy and fallback paths).

Live verify on rednote (same code path, account-safe):

```
$ opencli rednote search "美食" --limit 3 -f json
[ {rank:1, title:"在朋友家吃过一次..."}, {rank:2, title:"我的15💰晚餐..."}, {rank:3, title:"干净饮食🫛..."} ]
```

Legacy `section.note-item` path is exercised here (rednote still renders
the class) and returns identical row shape to before the fix, confirming
no regression on the working path.

Live verify on xiaohongshu cannot be performed here (no logged-in xhs
session on the test machine; xhs account-ban risk per the project's
operational guidance). The fix is structural: the new `<section>` shape
the issue reporter traced is reachable through the fallback, and the
existing test fixture keeps the legacy path green.

`npx tsc --noEmit` clean. `npm run build` 815 manifest entries unchanged
shape. `silent-column-drop` / `typed-error-lint` baselines unchanged.

Closes #1506
Refs #1500
2026-05-13 18:24:55 +08:00
陈家名 a6ca53c7cf fix: clamp download progress percentages (#1520)
* fix: clamp download progress percentages

* test(download): cover unknown progress total

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:24:26 +08:00
jakevin c3912d8e5c feat(reddit/read): --expand-more via /api/morechildren + 7-kind typed errors (#1492)
* feat(reddit/read): add --expand-more via /api/morechildren + 7-kind discriminated union

PR B of the rdt-cli parity follow-up (after PR #1491, see #1481 thread).
Closes the second-largest gap: Reddit's "[+N more replies]" stubs were
opaque markers in the comment tree. With --expand-more, the adapter
follows them by POST-ing the t1 ids to /api/morechildren.json, then
re-threads the returned things back into the tree by parent_id before
walking it.

New args:

- `--expand-more` (bool, default false) — turn on stub expansion.
- `--expand-rounds <N>` (int, default 2, range [1, 5]) — Reddit returns
  fresh "more" stubs at the expansion depth boundary, so up to N rounds
  are run. Strictly validated via `parseExpandRounds` — out-of-range
  raises ArgumentError BEFORE `page.goto`, no silent clamp.

Boy-Scout: the in-browser script now returns a 7-kind discriminated
union instead of a flat row array (matching the PR #1428 / #1491
sediment). Each kind maps 1:1 to a typed error on the Node side:

  - `inaccessible` → EmptyResultError
      401/403/404 on /comments/<id>.json (post-specific access, not
      session-level auth — applies the PR #1491 review-side sediment
      "inaccessible-resource vs session-auth").
  - `auth`         → AuthRequiredError
      401/403 on /api/morechildren (expand-write endpoints often demand
      a logged-in session even when the read endpoint is anonymous).
  - `http`         → CommandExecutionError
  - `malformed`    → CommandExecutionError
      200 with unexpected envelope shape — schema drift, not empty.
  - `parser-drift` → CommandExecutionError
      tree had t1 entries but the walker produced no rows (PR #1491
      review-side sediment "post-construction 0 rows + pre-walk
      non-empty = parser drift, not legitimate empty").
  - `expand-failed`→ CommandExecutionError
      /api/morechildren returned a non-empty json.errors array.
  - `ok`           → returns rows[].

Intermediate keys (kind / detail / httpStatus / where / rows /
expandMeta) deliberately avoid the declared columns (type / author /
score / text) per the PR #1329 silent-column-drop sediment.

Tests:
  clis/reddit/read.test.js — 11 tests
    - Adapter shape (browser / siteSession / columns / args)
    - --expand-more / --expand-rounds present with correct types/defaults
    - parseExpandRounds default / range / non-integer rejection
    - Pre-navigation validation (bad --expand-rounds doesn't reach goto)
    - kind=ok happy path (POST + L0 rows)
    - 6-kind error → typed error mapping
    - Unknown envelope shape → CommandExecutionError
    - Evaluate script embeds expandMore/expandRounds/sort/limit literals
    - Evaluate script contains /api/morechildren POST scaffolding
    - Evaluate script never names declared columns as intermediate keys

Full reddit suite 48/48; full project 3402/3402.

Audits: typed-error-lint 189/189 (0 new), silent-column-drop 103/103
(0 new). Manifest 815 → 815 (existing read entry gets 2 new args).

Existing --limit / --depth / --replies / --max-length keep their
original Math.max-style behaviour (grandfathered in the baseline);
only the new --expand-rounds flag fails fast per the typed-errors
standard.

Refs: https://github.com/jackwener/rdt-cli (browse.read --expand-more)

* fix(reddit): preserve expanded comment tree order

* fix(reddit): fail on partial morechildren expansion
2026-05-13 18:13:11 +08:00
darthjaja 67599ea67c fix(twitter): repair search and tweets readback (#1512)
* fix(twitter): repair search and tweets readback

* fix(twitter): prefer baked operation features when bundle parse returns empty

The bundle parser in resolveTwitterOperationMetadata locates the queryId via
`queryId:"..."` inside a ~2500-char snippet around the operationName marker,
then independently extracts `featureSwitches:[...]` and `fieldToggles:[...]`
via separate regexes. When minification rearranges the snippet (or the
snippet window truncates before the array), either regex can miss while
queryId still resolves; keysToFlags(undefined) then returns {}.

sanitizeTwitterOperationMetadata previously accepted any object as
features / fieldToggles, including {}. Twitter's GraphQL endpoint rejects
SearchTimeline / UserTweets requests with empty features (HTTP 400),
surfacing a misleading "queryId may have expired" error — the queryId is
fresh; only the feature flags are missing.

Guard against this by deferring to the baked fallback whenever the resolved
map is empty. Adds a JSDOM-free unit test that, reverse-validated, fails on
the un-fixed code with the exact silent-fallback shape.

Refs PR #1512

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-13 18:11:41 +08:00
darthjaja f321a6096d fix(twitter): make reply submission robust (#1511) 2026-05-13 18:10:19 +08:00
xcd_git 59ebf551f0 fix(google/search): wrap evaluate return value in object to fix serialization (#1523)
page.evaluate() serializes JS arrays as plain objects, causing
Array.isArray() to return false and the adapter to throw NOT_FOUND
even when results exist. Wrap the return value in {items: results}
and extract via wrapper.items to avoid the type check issue.

Co-authored-by: cxiao <chuda.xiao@wuerzburg-dynamics.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-13 18:05:03 +08:00
jakevin 04a57029b3 ci(adapter-test): gate adapter-test off pull_request trigger (#1522)
Per @WAWQAQ direction (DM): trim PR-time CI to fast-feedback only.
adapter-test (~30-60s) is the next-largest PR wait after e2e-headed
(which #1521 just removed). Adapter authors typically run focused tests
locally before pushing (`npm run test:adapter`); CI duplication adds
queue latency without catching new classes of bugs.

PR-time CI surface now:
  - typecheck / unit (~1 min)
  - lint gates (typed-error / silent-column-drop)
  - build × 3 platforms

Adapter test guards (still strict):
  - push to main / dev
  - nightly cron
  - workflow_dispatch (manual when an adapter-heavy PR really wants the
    signal before merge)

Same gate as smoke-test (`if: github.event_name == 'push' || schedule
|| workflow_dispatch`) for consistency.
2026-05-13 17:55:23 +08:00
373 changed files with 33969 additions and 4459 deletions
+5 -1
View File
@@ -110,8 +110,12 @@ jobs:
- name: Run unit tests under Bun
run: bun vitest run --project unit --reporter=verbose
# Adapter tests are pure unit tests — OS doesn't affect results.
# Adapter tests are pure unit tests — OS doesn't affect results. Gated off
# `pull_request` to keep PR CI under ~2 minutes; adapter authors run focused
# tests locally before pushing, and `push` to main / nightly cron / manual
# dispatch still guard the merged state.
adapter-test:
if: github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
needs: build
steps:
+149 -15
View File
@@ -1,27 +1,161 @@
# Changelog
## Unreleased
## [1.8.0](https://github.com/jackwener/opencli/compare/v1.7.22...v1.8.0) (2026-05-20)
### Bug Fixes
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed.
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed.
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results.
### Features
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic.
Substantial release: a new official-API adapter (`weread-official`), wider LinkedIn / Twitter / Reddit / Zhihu coverage, the 12306 / Suno / Xianyu inbox additions, security and reliability fixes for the Browser Bridge and media downloads, plus a 20% README shrink. Node 20 compatibility is restored after an automated `undici` bump regression.
### ⚠ BREAKING CHANGES
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface.
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell.
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail.
* **skills** — remove the `smart-search` skill. Use `opencli-usage` for command/site reference, `opencli-browser` for ad-hoc browser operation, and `opencli-adapter-author` for writing new adapters.
### Features
* **weread-official** — integrate WeRead's official Agent Gateway as the `weread-official` CLI namespace. Pure HTTP, Bearer auth via `WEREAD_API_KEY` (no browser, no cookies). 8 commands cover the official skill bundle: `search`, `shelf`, `book` (info + chapters + progress 3-in-1), `notes` (notebook overview or per-book highlights/thoughts), `review`, `readdata` (weekly/monthly/annually/overall), `discover` (recommend or similar-book), `list-apis`. Adapter surfaces typed errors for all documented failure modes — `AuthRequiredError` on missing/rejected key (errcodes -2010/-2012), `CommandExecutionError` on HTTP/`upgrade_info`/non-zero errcode, `EmptyResultError` on empty payloads. Coexists with the existing cookie-based `weread` adapter.
* **12306** — add full read adapter (`stations` / `trains` / `train` / `price` / `me` / `passengers` / `orders`). ([#1637](https://github.com/jackwener/opencli/issues/1637))
* **xianyu** — add `inbox`, `messages`, and `reply` commands. ([#1639](https://github.com/jackwener/opencli/issues/1639))
* **suno** — add Suno.com music-generation adapter. ([#1638](https://github.com/jackwener/opencli/issues/1638))
* **linkedin** — consolidate messaging and Sales Navigator commands (`connect`, `inbox`, `safe-send`, `salesnav-search`, `salesnav-inbox`, `salesnav-message`, `salesnav-thread`, `sent-invitations`, `thread-snapshot`, `timeline`). ([#1647](https://github.com/jackwener/opencli/issues/1647))
* **linkedin/people-search** — add a dedicated people-search command. ([#1649](https://github.com/jackwener/opencli/issues/1649))
* **linkedin-learning** — add `search` / `trending` / `course` read commands. ([#1657](https://github.com/jackwener/opencli/issues/1657))
* **twitter** — rewrite the download-profile path on GraphQL UserMedia with cursor pagination. ([#1636](https://github.com/jackwener/opencli/issues/1636))
* **twitter** — add `list-create` (GraphQL CreateList mutation). ([#1656](https://github.com/jackwener/opencli/issues/1656))
* **twitter** — add `device-follow` notification-stream command.
* **twitter** — expose `card.binding_values` on read commands for inline link-preview metadata. ([#1660](https://github.com/jackwener/opencli/issues/1660))
* **twitter** — expose `quoted_tweet` on read commands. ([#1667](https://github.com/jackwener/opencli/issues/1667))
* **twitter** — expose `bio` on read commands.
* **reddit/subscribed** — new `subscribed` command + listing-level `id` / `created_utc` / `selftext` exposure. ([#1651](https://github.com/jackwener/opencli/issues/1651))
* **reddit** — expose `post_hint` / `url` / `preview` / `gallery` media routes on listing commands. ([#1676](https://github.com/jackwener/opencli/issues/1676))
* **zhihu** — add answer-comments reader; include answer links in question results.
* **chatgpt** — detect generated image surfaces (CSS background and canvas, not just `<img>`) so image generation works after UI drift. ([#1677](https://github.com/jackwener/opencli/issues/1677))
* **external** — add Cloudflare Wrangler as a built-in external CLI passthrough. ([#1679](https://github.com/jackwener/opencli/pull/1679))
### Bug Fixes
* **deps** — restore Node 20 runtime compatibility by pinning runtime `undici` back to the 6.x line (an automated dependabot bump to 8.x had moved the engines floor to Node ≥22.19, silently breaking the published Node 20 promise), and clear the docs build audit chain by overriding VitePress' Vite/PostCSS transitive dependencies to patched versions. ([#1673](https://github.com/jackwener/opencli/issues/1673))
* **download** — keep custom media filenames inside the requested output directory by stripping POSIX/Windows path components and sanitizing the generated fallback prefix. Prevents remote-controlled fields (e.g. video titles used as filename) from escaping the output directory via `../`. ([#1642](https://github.com/jackwener/opencli/pull/1642))
* **browser** — recover `Page.goto()` from stale page identities by clearing the cached targetId and retrying navigation once through the session lease; classify CDP `-32000 Cannot find default execution context` as retryable target navigation. ([#1645](https://github.com/jackwener/opencli/issues/1645))
* **cli** — escape leading-dash positional values via the argv preprocessor so users can pass tokens starting with `-` without commander mis-classifying them as flags. ([#1658](https://github.com/jackwener/opencli/issues/1658))
* **chatgpt/image** — fix ChatGPT web image generation after UI drift by letting the composer locator continue into the caller's readiness check and detecting generated images rendered as CSS backgrounds or canvases, not just plain `<img>` elements.
* **adapters** — surface the remaining `silent-empty-fallback` adapter failures as typed errors (Douyin user video comments, Jike SSR JSON parse, WeRead search-page fetch). True empty Douyin/Jike/WeRead result sets now throw `EmptyResultError`.
* **adapters** — drop silent-sentinel row fallbacks across Apple Podcasts / Reddit / Gitee. ([#1634](https://github.com/jackwener/opencli/issues/1634))
* **adapters** — migrate legal empty-data branches to `EmptyResultError` for `xhs` / YouTube and 5 follow-up commands. ([#1674](https://github.com/jackwener/opencli/issues/1674), [#1678](https://github.com/jackwener/opencli/issues/1678))
* **lesswrong** — drop the `"Unknown"` silent sentinel in the author column; missing authors now propagate as `null`. ([#1611](https://github.com/jackwener/opencli/issues/1611))
* **youtube/transcript** — scope timedtext URL matching to the current `videoId` across the in-page resource-buffer scan, the in-page fetch/XHR hook, and the Node-side CDP capture. SPA-style watch→watch navigation no longer returns a predecessor video's captions. ([#1655](https://github.com/jackwener/opencli/issues/1655))
* **twitter/lists** — skip the "Discover new Lists" recommendation block so it is no longer treated as one of the user's lists. ([#1652](https://github.com/jackwener/opencli/issues/1652))
* **zhihu** — harden search pagination. ([#1615](https://github.com/jackwener/opencli/issues/1615))
* **zhihu** — decode numeric HTML entities in `answer-detail`. ([#1629](https://github.com/jackwener/opencli/issues/1629))
### Docs
* **readme** — major shrink and reframing: tagline rephrased around "Browser Use", Highlights and Update sections folded into adjacent content, Built-in Commands curated to 11 popular sites, CLI Hub table reduced to a name enumeration, Desktop App Adapters collapsed to a one-liner, skill-attribution references audited against `SKILL.md` frontmatter, "For AI Agents (Developer Guide)" merged into "Writing a new adapter". Net: EN 410 → 326 (-20%), ZH 455 → 371 (-18%). ([#1654](https://github.com/jackwener/opencli/pull/1654), [#1666](https://github.com/jackwener/opencli/pull/1666), [#1679](https://github.com/jackwener/opencli/pull/1679), [#1681](https://github.com/jackwener/opencli/pull/1681))
### Internal
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`.
* **extension 1.0.13** — remove the internal command-session lease-key backdoor.
* **audit** — stop flagging sentinel fallback strings inside thrown error messages as `silent-sentinel` violations. These are typed failure diagnostics rather than fake row data, reducing the typed-error baseline to actual adapter output fallbacks.
## [1.7.22](https://github.com/jackwener/opencli/compare/v1.7.21...v1.7.22) (2026-05-15)
External CLI ergonomics + two adapter envelope/auth fixes. New `longbridge` external CLI entry; `opencli list` / root help now render human-readable brand labels for executables whose bare name is ambiguous.
### Features
* **external** — add the Longbridge CLI as a built-in external CLI passthrough (`opencli longbridge ...`) for Longbridge OpenAPI market data, account, and trading commands. ([#1584](https://github.com/jackwener/opencli/issues/1584))
* **external-cli** — render brand alias `name(package)` in `opencli list` and root help when the bare executable name is ambiguous. Built-in entries `ntn``ntn(notion)`, `dws``dws(DingTalk Workspace)`, `wecom-cli``wecom-cli(企业微信)` now self-explain in help output. `package` field is repurposed to cover both upstream distribution names (e.g. `tg-cli`) and human-readable brand labels (e.g. `notion`, `企业微信`). ([#1585](https://github.com/jackwener/opencli/issues/1585))
### Bug Fixes
* **boss** — map `code=24` (identity mismatch) to `AuthRequiredError` so re-login is signaled instead of surfacing as a generic API error. ([#1573](https://github.com/jackwener/opencli/issues/1573))
* **weibo** — unwrap Browser Bridge `page.evaluate` envelopes in read adapters. ([#1568](https://github.com/jackwener/opencli/issues/1568))
## [1.7.21](https://github.com/jackwener/opencli/compare/v1.7.20...v1.7.21) (2026-05-14)
Adapter polish release: new web search adapters, better Browser Bridge tab group reuse, and social adapters returning to one-shot tab leases. Extension package version is bumped to 1.0.15 for the Browser Bridge fix.
### Features
* **search** — add DuckDuckGo, Brave, and Yahoo web search adapters. ([#1546](https://github.com/jackwener/opencli/issues/1546))
* **boss** — support job-seeker `chatlist` and `chatmsg` adapters. ([#1539](https://github.com/jackwener/opencli/issues/1539))
### Bug Fixes
* **extension** — reuse existing `OpenCLI Adapter` tab groups before creating new ones, including cross-window discovery, legacy `OpenCLI` title fallback, and deterministic candidate selection. ([#1541](https://github.com/jackwener/opencli/issues/1541))
* **twitter, reddit** — default browser-backed social adapters back to ephemeral tab leases. Twitter/X and Reddit commands now release their site tab after each run while keeping the shared Adapter window available for reuse; persistent sessions remain reserved for AI/chat-style adapters that need long-lived conversation state. ([#1569](https://github.com/jackwener/opencli/issues/1569))
* **xiaohongshu, rednote** — unwrap Browser Bridge `page.evaluate` envelopes in search adapters. ([#1561](https://github.com/jackwener/opencli/issues/1561))
* **facebook/feed** — add fallback extraction for empty article nodes. ([#1538](https://github.com/jackwener/opencli/issues/1538))
### Internal
* **ci** — add Windows native binding lockfile entries for Rolldown/Rollup optional packages. ([#1563](https://github.com/jackwener/opencli/issues/1563))
* **extension** — add regression coverage for the adapter tab group `groupId` tiebreaker. ([#1566](https://github.com/jackwener/opencli/issues/1566))
## [1.7.20](https://github.com/jackwener/opencli/compare/v1.7.19...v1.7.20) (2026-05-14)
External CLI surface cleanup + Browser Bridge WebSocket lifecycle hardening. Two BREAKING changes around external CLIs: built-in `tg`/`discord`/`wx` (was `tg-cli`/`discord-cli`/`wx-cli`) now match their real binary names, and Notion's in-tree CDP adapter is replaced by the official `ntn` external CLI.
### ⚠ BREAKING CHANGES
* **notion** — remove the in-tree `clis/notion/` CDP-on-Desktop adapter (8 commands: `status` / `search` / `read` / `new` / `write` / `sidebar` / `favorites` / `export`). Notion has shipped an official CLI at <https://ntn.dev>, registered as a first-class external CLI in `external-clis.yaml`. Migration: install `ntn` from <https://ntn.dev> (`curl -fsSL https://ntn.dev | bash`), then use `opencli ntn <command>`. Auto-install is intentionally not configured because the official installer is a shell script while OpenCLI external installs run shell-free command strings. The official CLI uses the public Notion API rather than reverse-engineering the Desktop UI, so it survives Notion app updates and exposes a wider command surface (blocks / databases / properties / comments) than the reverse-engineered adapter could. ([#1559](https://github.com/jackwener/opencli/issues/1559))
* **external** — drop the `-cli` suffix from built-in external CLI subcommand names. `opencli tg-cli`, `opencli discord-cli`, `opencli wx-cli` are now `opencli tg`, `opencli discord`, `opencli wx`, matching the real binary names that those tools install as. Root help still shows the package lineage as `tg(tg-cli)` / `discord(discord-cli)` / `wx(wx-cli)`. ([#1544](https://github.com/jackwener/opencli/issues/1544))
### Features
* **twitter** — `bookmarks` and `bookmark-folder` now include media via `extractMedia`, reaching parity with `timeline` / `search`. ([#1555](https://github.com/jackwener/opencli/issues/1555))
* **twitter/list-tweets** — include media via `extractMedia` (parity with `timeline` / `search`). ([#1464](https://github.com/jackwener/opencli/issues/1464))
### Bug Fixes
* **daemon** — report ambiguous browser command outcomes with a distinct `command_result_unknown` errorCode and `503` when the extension WebSocket drops between command dispatch and result delivery. `sendCommandRaw()` treats this code as hard non-retryable, so write-side commands (`navigate` / `click` / `type` / `eval`) won't be silently re-issued and double-executed. Daemon exposes a `commandResultUnknown` counter on `/status` for future observability. ([#1558](https://github.com/jackwener/opencli/issues/1558))
* **extension** — keep active daemon WebSocket; stale sockets no longer clobber active connection (`onopen` / `onclose` / `onmessage` are all gated by `ws !== thisWs` short-circuit), and `safeSend` only fires when `readyState === OPEN`. ([#1540](https://github.com/jackwener/opencli/issues/1540))
* **extension** — coalesce concurrent daemon WebSocket connects via an in-flight promise. Startup / keepalive / reconnect triggering `connect()` during the daemon-probe or context-lookup async gap no longer creates duplicate real WebSocket connections. ([#1554](https://github.com/jackwener/opencli/issues/1554))
* **external** — distinguish external CLI executable names from distribution/project names in root help. Built-in aliases such as `tg`, `discord`, `wx` remain the callable `opencli <name> ...` entrypoints while help renders `tg(tg-cli)`, `discord(discord-cli)`, `wx(wx-cli)` to show their package lineage. ([#1560](https://github.com/jackwener/opencli/issues/1560))
### Docs
* **browser** — clarify named session lifecycle in the Browser Bridge guide. ([#1542](https://github.com/jackwener/opencli/issues/1542))
## [1.7.19](https://github.com/jackwener/opencli/compare/v1.7.18...v1.7.19) (2026-05-14)
Major hotfix + simplification batch. Extension bumped to 1.0.14. Node floor lowered to v20 so the long tail of Node v20v21.6 users no longer crashes at module load. `opencli browser` user surface replaces required-flag `--session <name>` with a `<session>` positional. `page.evaluate(fn, ...args)` adds a type-safe alternative to the implicit auto-IIFE string form. Twitter cursor pagination no longer silently caps at ~500 items.
### ⚠ BREAKING CHANGES
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface. ([#1505](https://github.com/jackwener/opencli/issues/1505))
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell. ([#1509](https://github.com/jackwener/opencli/issues/1509))
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail. ([#1510](https://github.com/jackwener/opencli/issues/1510))
### Features
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic. ([#1508](https://github.com/jackwener/opencli/issues/1508))
* **twitter** — default `tweets` command to the logged-in user when `user` is omitted, and fix the sibling envelope-unwrap silent bug. ([#1531](https://github.com/jackwener/opencli/issues/1531))
* **zhihu** — add `answer-detail` to fetch a single answer's full content. ([#1528](https://github.com/jackwener/opencli/issues/1528))
* **zhihu** — paginate question answers and recommendations. ([#1517](https://github.com/jackwener/opencli/issues/1517))
* **reddit/read** — `--expand-more` via `/api/morechildren` + 7-kind typed errors. ([#1492](https://github.com/jackwener/opencli/issues/1492))
* **reddit** — add `whoami`, `home`, `subreddit-info` read commands. ([#1491](https://github.com/jackwener/opencli/issues/1491))
* **ctrip** — add `hotel-search` + flight browser-mode commands. ([#1489](https://github.com/jackwener/opencli/issues/1489))
### Bug Fixes
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **twitter** — raise fixed cursor-pagination caps in `bookmarks` / `likes` / `tweets` / `timeline` / `bookmark-folder` / `list-tweets` / `search` / `following`. The old `i < 5` / `i < 10` literals and following's `Math.ceil(limit / 50) + 2` formula imposed hidden result ceilings below `--limit`; the loop now treats the page count as a high runaway guard while `--limit` and cursor exhaustion control normal pagination. ([#1532](https://github.com/jackwener/opencli/issues/1532))
* **twitter** — repair `list-add` / `list-tweets` / `lists` / `following` after 2026-05 site changes. ([#1503](https://github.com/jackwener/opencli/issues/1503))
* **twitter** — repair `search` and `tweets` readback. ([#1512](https://github.com/jackwener/opencli/issues/1512))
* **twitter** — make reply submission robust. ([#1511](https://github.com/jackwener/opencli/issues/1511))
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **google/search** — wrap evaluate return value in object to fix serialization. ([#1523](https://github.com/jackwener/opencli/issues/1523))
* **google-scholar/search** — wrap evaluate return to fix serialization. ([#1525](https://github.com/jackwener/opencli/issues/1525))
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **xiaohongshu** — `parseLikes` handles `2.1w` / `1.5万` / `1.2k` shortforms. ([#1504](https://github.com/jackwener/opencli/issues/1504))
* **xiaohongshu+rednote/search** — fall back to href-based note cards when `section.note-item` class is dropped. ([#1507](https://github.com/jackwener/opencli/issues/1507))
* **xueqiu** — `kline` / `earnings-date` format dates in Asia/Shanghai instead of UTC. ([#1498](https://github.com/jackwener/opencli/issues/1498))
* **download** — clamp progress percentages. ([#1520](https://github.com/jackwener/opencli/issues/1520))
### Internal
* **runtime** — lower the Node floor to `>=20.0.0`. Three coupled changes: drop all `util.styleText()` usage (added in Node v21.7.0 / v20.12.0; previously crashed v21.0v21.6 at module load), downgrade `undici` from `^8.0.2` (engines `>=22.19.0`) to `^6.25.0` (engines `>=18.17`, retains `Agent` / `EnvHttpProxyAgent` / `fetch`), and lower `MIN_SUPPORTED_NODE_MAJOR` from 21 to 20 so the startup guard matches the declared `engines.node`. Smoke-tested on v20.0.0 / v21.2.0 / v22.22.2. The semantic markers (`[OK]` / `[WARN]` / `[FAIL]` / `` / `⚠` / `✖`) keep their meaning; ANSI colors were redundant for the primarily agent-facing CLI. ([#1524](https://github.com/jackwener/opencli/issues/1524))
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`. ([#1518](https://github.com/jackwener/opencli/issues/1518))
* **extension 1.0.13** — remove the internal command-session lease-key backdoor. ([#1510](https://github.com/jackwener/opencli/issues/1510))
* **ci** — drop `e2e-headed` and `adapter-test` from `pull_request` triggers (kept on `push` to main / nightly / `workflow_dispatch`). PR-time CI now targets ~2 min wall-time. ([#1521](https://github.com/jackwener/opencli/issues/1521), [#1522](https://github.com/jackwener/opencli/issues/1522))
* **scripts** — auto-refresh `dist/` before `build-manifest`. ([#1490](https://github.com/jackwener/opencli/issues/1490))
## [1.7.18](https://github.com/jackwener/opencli/compare/v1.7.17...v1.7.18) (2026-05-12)
+32 -147
View File
@@ -1,7 +1,8 @@
# OpenCLI
> **Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.**
> Reuse your logged-in browser, automate live workflows, and crystallize repeated actions into reusable CLI commands.
> **Convert any website into a CLI & run Browser Use on your logged-in Chrome.**
> Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.
> Or run Browser Use against any page — navigate, fill forms, click, extract, automate.
[![中文文档](https://img.shields.io/badge/docs-%E4%B8%AD%E6%96%87-0F766E?style=flat-square)](./README.zh-CN.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
@@ -11,24 +12,10 @@
OpenCLI gives you one surface for three different kinds of automation:
- **Use built-in adapters** for sites like Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, Twitter/X, and [many more](#built-in-commands).
- **Let AI Agents operate any website** — install the `opencli-adapter-author` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Let AI Agents operate any website** — install the `opencli-browser` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Write new adapters** end-to-end with `opencli browser` + the `opencli-adapter-author` skill, which guides from first recon through field decoding, code, and `opencli browser verify`.
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `tg-cli`, `discord-cli`, `wx-cli`, and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, ChatGPT, and Notion.
## Highlights
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type/fill, extract, screenshot — all through your logged-in Chrome session.
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
- **Website → CLI** — Turn any website into a deterministic CLI: 100+ site surfaces are already registered, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
- **CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, docker, obsidian, tg-cli, discord-cli, wx-cli, etc).
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
- **Deterministic** — Same command, same output schema, every time. Pipeable, scriptable, CI-friendly.
---
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `longbridge`, `tg`, `discord`, `wx`, `ntn` (Notion), and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, and ChatGPT.
## Quick Start
@@ -105,7 +92,7 @@ If you want to add your own commands, start with the [Extending OpenCLI guide](.
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
### Install skills
### Install skills (also refreshes existing installs)
```bash
npx skills add jackwener/opencli
@@ -118,22 +105,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
### Which skill to use
| Skill | When to use | Example prompt to your AI agent |
|-------|------------|-------------------------------|
| **opencli-adapter-author** | Operate a site in real time, or write a reusable adapter for a new site | "Help me check my Xiaohongshu notifications" / "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-adapter-author** | Write a reusable adapter for a new site or add a command to an existing site | "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-autofix** | Repair a broken adapter when a built-in command fails | "`opencli zhihu hot` is returning empty — fix it" |
| **opencli-browser** | Browser automation reference for AI agents | "Use browser commands to scrape this page" |
| **opencli-browser** | Drive a real Chrome page ad-hoc — navigate, fill forms, click, extract | "Help me check my Xiaohongshu notifications" / "Help me fill out this form" / "Use browser commands to scrape this page" |
| **opencli-usage** | Quick reference for all OpenCLI commands and sites | "What commands does OpenCLI have for Twitter?" |
| **smart-search** | Search across existing OpenCLI capabilities | "Find me a Bilibili trending adapter" |
### How it works
Once `opencli-adapter-author` is installed, your AI agent can:
Once `opencli-browser` is installed, your AI agent can:
1. **Navigate** to any URL using your logged-in browser
2. **Read** page content via structured DOM snapshots (not screenshots)
@@ -144,45 +129,25 @@ Once `opencli-adapter-author` is installed, your AI agent can:
The agent handles all the `opencli browser` commands internally — you just describe what you want done in natural language.
**Skill references:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — browser operation + adapter authoring, end-to-end
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — drive Chrome ad-hoc (navigate, fill forms, click, extract)
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — write a new adapter end-to-end
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — repair broken adapters
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — browser automation reference
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — command and site reference
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — capability search
Available browser commands include `open`, `state`, `click`, `type`, `fill`, `select`, `keys`, `wait`, `get`, `find`, `extract`, `frames`, `screenshot`, `scroll`, `back`, `eval`, `network`, `tab list`, `tab new`, `tab select`, `tab close`, `init`, `verify`, and `close`.
`opencli browser` commands require a `<session>` positional immediately after `browser`. `opencli browser work open <url>` and `opencli browser work tab new [url]` both return a target ID. Use `opencli browser work tab list` to inspect target IDs, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted commands in the same session.
## Core Concepts
## Writing a new adapter
### `browser`: AI Agent browser control
When the site you need is not yet covered, use the `opencli-adapter-author` skill end-to-end:
`opencli browser` commands are the low-level primitives that AI Agents use to operate websites. You don't run these manually — instead, install the `opencli-adapter-author` skill into your AI agent, describe what you want in natural language, and the agent handles the browser operations.
For example, tell your agent: *"Help me check my Xiaohongshu notifications"* — the agent will use `opencli browser <session> open`, `state`, `click`, etc. under the hood.
### Built-in adapters: stable commands
Use site-specific commands such as `opencli hackernews top` or `opencli reddit hot` when the capability already exists. These are deterministic and work without browser — ideal for both humans and AI agents.
### Writing a new adapter
When the site you need is not yet covered, use the `opencli-adapter-author` skill. It takes the agent end-to-end:
1. Recon the site and classify its pattern (SPA / SSR / JSONP / Token / Streaming).
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`.
4. Decode response fields and design output columns.
5. `opencli browser recon analyze <url>` for one-shot recon, then `opencli browser recon init <site>/<name>` → write adapter → `opencli browser recon verify <site>/<name>`.
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
### CLI Hub and desktop adapters
OpenCLI is not only for websites. It can also:
- expose local binaries like `gh`, `docker`, `obsidian`, `tg-cli`, `discord-cli`, `wx-cli`, or custom tools through `opencli <tool> ...`
- control Electron desktop apps through dedicated adapters and CDP-backed integrations
1. **Recon** the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
2. **Discover** the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. **Pick auth**`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`.
4. **Decode** response fields and design output columns.
5. `opencli browser recon analyze <url>``opencli browser recon init <site>/<name>` → write adapter → `opencli browser recon verify <site>/<name>`.
6. Site knowledge persists to `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context.
## Prerequisites
@@ -208,25 +173,6 @@ OpenCLI is not only for websites. It can also:
`opencli browser *` requires an explicit `<session>` positional, uses a foreground browser window by default, and keeps that session's tab lease until `opencli browser <session> close` or idle cleanup. Browser-backed adapters use a background adapter window and release one-shot tab leases by default. Interactive adapters can declare `siteSession: 'persistent'` to keep a stable site tab for continuity; pass `--site-session ephemeral` for a one-shot tab.
## Update
```bash
npm install -g @jackwener/opencli@latest
# If you use the packaged OpenCLI skills, refresh them too
npx skills add jackwener/opencli
```
Or refresh only the skills you actually use:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
## For Developers
Install from source:
@@ -249,77 +195,28 @@ To load the source Browser Bridge extension:
| Site | Commands |
|------|----------|
| **xiaohongshu** | `search` `note` `comments` `feed` `user` `download` `publish` `notifications` `creator-notes` `creator-notes-summary` `creator-note-detail` `creator-profile` `creator-stats` |
| **rednote** | `search` `note` `comments` `user` `download` `feed` `notifications` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `video` `user-videos` |
| **tieba** | `hot` `posts` `search` `read` |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `summary` `video` `user-videos` |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
| **1688** | `search` `item` `assets` `download` `store` |
| **gitee** | `trending` `search` `user` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **yuanbao** | `new` `ask` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` |
| **xianyu** | `search` `item` `chat` `publish` |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
| **uiverse** | `code` `preview` |
| **baidu-scholar** | `search` |
| **google-scholar** | `search` `cite` `profile` |
| **gov-law** | `search` `recent` |
| **gov-policy** | `search` `recent` |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
| **wanfang** | `search` |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **xiaoyuzhou** | `auth*` `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` |
| **linkedin** | `connect` `inbox` `safe-send` `search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
100+ site surfaces in total**[→ see all supported sites & commands](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast`, `podcast-episodes`, `episode`, `download`, and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
Curated highlights**[→ see all 100+ supported sites & commands](./docs/adapters/index.md)** (douyin / weibo / spotify / 1688 / quark / nowcoder / google-scholar / hupu / xianyu / weread / weread-official / xiaoyuzhou / and more).
## CLI Hub
OpenCLI acts as a universal hub for your existing command-line tools — unified discovery, pure passthrough execution, and auto-install (if a tool isn't installed, OpenCLI runs `brew install <tool>` automatically before re-running the command).
Unified passthrough for your existing command-line tools. Run `opencli <tool> ...` for any of:
| External CLI | Description | Example |
|--------------|-------------|---------|
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
| **obsidian** | Obsidian vault management | `opencli obsidian search query="AI"` |
| **docker** | Docker | `opencli docker ps` |
| **lark-cli** | Lark/Feishu — messages, docs, calendar, tasks, 200+ commands | `opencli lark-cli calendar +agenda` |
| **dws** | DingTalk — cross-platform CLI for DingTalk's full suite, designed for humans and AI agents | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | WeCom/企业微信 — CLI for WeCom open platform, for humans and AI agents | `opencli wecom-cli msg send --to user "hello"` |
| **tg-cli** | Telegram — local-first sync, search, and export via MTProto for AI agents | `opencli tg search "AI news" -f json` |
| **discord-cli** | Discord — local-first sync, search, and export via SQLite for AI agents | `opencli discord recent --channel general` |
| **wx-cli** | WeChat — query local WeChat data: sessions, messages, search, contacts, export | `opencli wx search "OpenCLI"` |
| **vercel** | Vercel — deploy projects, manage domains, env vars, logs | `opencli vercel deploy --prod` |
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
Register your own with `opencli external register <name>`; list everything with `opencli external list`.
```bash
opencli external register mycli
```
### Desktop App Adapters
Control Electron desktop apps directly from the terminal. Each adapter has its own detailed documentation:
| App | Description | Doc |
|-----|-------------|-----|
| **Cursor** | Control Cursor IDE — Composer, chat, code extraction | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | Drive OpenAI Codex CLI agent headlessly | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | Control Antigravity Ultra from terminal | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT App** | Automate ChatGPT macOS desktop app | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | Multi-LLM client (GPT-4, Claude, Gemini) | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | Search, read, write Notion pages | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord Desktop — messages, channels, servers | [Doc](./docs/adapters/desktop/discord.md) |
| **Doubao** | Control Doubao AI desktop app via CDP | [Doc](./docs/adapters/desktop/doubao-app.md) |
To add a new Electron app, start with [docs/guide/electron-app-cli.md](./docs/guide/electron-app-cli.md).
**Desktop app adapters** (Electron, via CDP): Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — see [`docs/adapters/desktop/`](./docs/adapters/desktop/).
## Download Support
@@ -405,18 +302,6 @@ opencli plugin uninstall my-tool
See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
## For AI Agents (Developer Guide)
Before writing any adapter code, read the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md). It takes you end-to-end:
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
- Discover the right endpoint via `opencli browser <session> network`, `eval`, or the interceptor fallback.
- Decide auth strategy (`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`).
- Run `opencli browser recon analyze <url>` for one-shot recon, decode response fields, design columns, scaffold with `opencli browser recon init`.
- Verify with `opencli browser recon verify <site>/<name>` before shipping.
For long-lived personal commands that should live in your own Git repo, use a local plugin instead; see [Extending OpenCLI](./docs/guide/extending-opencli.md). Quick private adapters can still live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
## Testing
See **[TESTING.md](./TESTING.md)** for how to run and write tests.
+37 -211
View File
@@ -1,7 +1,8 @@
# OpenCLI
> **把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口。**
> 复用浏览器登录态,先自动化真实操作,再把高频流程沉淀成可复用的 CLI 命令
> **把任意网站变成 CLI & 在你的登录态浏览器上跑 Browser Use。**
> 把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口
> 或者在任意页面上跑 Browser Use —— 导航、填表单、点击、抓取、自动化。
[![English](https://img.shields.io/badge/docs-English-1D4ED8?style=flat-square)](./README.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
@@ -11,21 +12,10 @@
OpenCLI 可以用同一套 CLI 做三类事情:
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [100+ 站点](#内置命令) 开箱即用。
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-adapter-author` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-browser` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
- **把新网站写成 CLI**:用 `opencli browser` 原语 + `opencli-adapter-author` skill,从站点侦察、API 发现、字段解码到 `opencli browser verify` 一条龙。
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh``docker``tg-cli``discord-cli``wx-cli` 等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT、Notion 等 Electron 应用。
## 亮点
- **桌面应用控制** — 通过 CDP 直接在终端驱动 Electron 应用(Cursor、Codex、ChatGPT、Notion 等)。
- **AI Agent 浏览器自动化** — 安装 `opencli-adapter-author` skill,你的 AI Agent 就能操作任意网站:导航、点击、输入/填充、提取、截图——全部通过你的已登录 Chrome 会话完成。
- **网站 → CLI** — 把任何网站变成确定性 CLI:100+ 站点能力已注册,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
- **账号安全** — 复用 Chrome/Chromium 登录态,凭证永远不会离开浏览器。
- **面向 AI Agent** — 一个 skill 带你走完站点侦察、API 发现、字段解码、适配器编写、验证的全流程。
- **CLI 枢纽** — 统一发现、自动安装、纯透传任何外部 CLI(gh、docker、obsidian、tg-cli、discord-cli、wx-cli 等)。
- **零 LLM 成本** — 运行时不消耗模型 token,跑 10,000 次也不花一分钱。
- **确定性输出** — 相同命令,相同输出结构,每次一致。可管道、可脚本、CI 友好。
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh``docker``longbridge``tg``discord``wx``ntn`Notion等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT 等 Electron 应用。
## 快速开始
@@ -89,7 +79,7 @@ opencli bilibili hot --limit 5
OpenCLI 的 browser 命令是给 AI Agent 用的——不是手动执行的。把 skill 安装到你的 AI AgentClaude Code、Cursor 等)中,Agent 就能用你的已登录 Chrome 会话替你操作网站。
### 安装 skill
### 安装 skill(同时也用于更新)
```bash
npx skills add jackwener/opencli
@@ -102,22 +92,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
### 选择哪个 skill
| Skill | 适用场景 | 你对 AI Agent 说的话 |
|-------|---------|-------------------|
| **opencli-adapter-author** | 实时操作任意网站,或为新站点写可复用适配器 | "帮我看看小红书的通知" / "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
| **opencli-adapter-author** | 为新站点写可复用适配器,或给已有站点添加命令 | "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
| **opencli-autofix** | 内置命令失败时修复已有适配器 | "`opencli zhihu hot` 返回空了,修一下" |
| **opencli-browser** | 浏览器自动化参考文档 | "用浏览器命令抓取这个页面" |
| **opencli-browser** | 实时驱动 Chrome 页面——导航、填表单、点击、抓取 | "帮我看看小红书的通知" / "帮我填一下这个表单" / "用浏览器命令抓取这个页面" |
| **opencli-usage** | 所有命令和站点的快速参考 | "OpenCLI 有哪些 Twitter 相关的命令?" |
| **smart-search** | 在现有 OpenCLI 能力里搜索 | "帮我找个 B 站热门相关的适配器" |
### 工作原理
安装 `opencli-adapter-author` skill 后,你的 AI Agent 可以:
安装 `opencli-browser` skill 后,你的 AI Agent 可以:
1. **导航**到任意 URL,使用你的已登录浏览器
2. **读取**页面内容——通过结构化 DOM 快照(不是截图)
@@ -128,45 +116,25 @@ npx skills add jackwener/opencli --skill smart-search
Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自然语言描述想做的事。
**Skill 参考文档:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 浏览器操作 + 适配器编写,全流程
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 实时驱动 Chrome(导航、填表单、点击、抓取)
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 给新站点写适配器,全流程
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — 修复已有适配器
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 浏览器自动化参考
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — 命令和站点参考
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — 能力搜索
`browser` 可用命令包括:`open``state``click``type``fill``select``keys``wait``get``find``extract``frames``screenshot``scroll``back``eval``network``tab list``tab new``tab select``tab close``init``verify``close`
`opencli browser` 命令必须紧跟一个 `<session>` 位置参数。`opencli browser work open <url>``opencli browser work tab new [url]` 都会返回 target ID。`opencli browser work tab list` 用来查看当前已存在 tab 的 target ID,再通过 `--tab <targetId>` 把命令明确路由到某个 tab。`tab new` 只会新建 tab,不会改变默认浏览器目标;只有显式执行 `tab select <targetId>`,才会把该 tab 设为同一 session 后续未指定 target 的默认目标。
## 核心概念
## 为新站点写适配器
### `browser`AI Agent 的浏览器控制层
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,全流程:
`opencli browser` 命令是 AI Agent 操作网站的底层原语。你不需要手动运行这些命令——把 `opencli-adapter-author` skill 安装到你的 AI Agent 中,用自然语言描述你想做的事,Agent 会自动处理浏览器操作。
比如你告诉 Agent:*"帮我看看小红书的通知"*——Agent 会在底层调用 `opencli browser <session> open``state``click` 等命令。
### 内置适配器:稳定命令
当某个站点能力已经存在时,优先使用 `opencli hackernews top``opencli reddit hot` 这类稳定命令。这些命令是确定性的,无需浏览器——人类和 AI Agent 都可以直接使用。
### 为新站点写适配器
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,它会把 Agent 带到闭环:
1. 侦察站点,分类 patternSPA / SSR / JSONP / Token / Streaming
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. 定认证策略——`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
4. 字段解码 + 设计输出列
5. `opencli browser recon analyze <url>` 一步侦察,再 `opencli browser recon init <site>/<name>` → 写适配器 → `opencli browser recon verify <site>/<name>`
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
### CLI 枢纽与桌面端适配器
OpenCLI 不只是网站 CLI,还可以:
- 统一代理本地二进制工具,例如 `gh``docker``obsidian``tg-cli``discord-cli``wx-cli`
- 通过专门适配器和 CDP 集成控制 Electron 桌面应用
1. **侦察**站点,分类 patternSPA / SSR / JSONP / Token / Streaming
2. **发现** endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. **定认证**——`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
4. **字段解码** + 设计输出列
5. `opencli browser recon analyze <url>``opencli browser recon init <site>/<name>` → 写适配器 → `opencli browser recon verify <site>/<name>`
6. 站点知识沉到 `~/.opencli/sites/<site>/`,下次同站点直接吃缓存
## 前置要求
@@ -191,23 +159,6 @@ OpenCLI 不只是网站 CLI,还可以:
`opencli browser *` 必须紧跟一个 `<session>` 位置参数,默认使用前台窗口,并保留该 session 的 tab lease,直到你手动执行 `opencli browser <session> close` 或等空闲超时。浏览器型 adapter 默认使用后台 adapter 窗口并在命令结束后释放一次性 tab lease;如果需要调试最终页面,可以传 `--window foreground --keep-tab true`
## 更新
```bash
npm install -g @jackwener/opencli@latest
# 如果你在用打包发布的 OpenCLI skills,也一起刷新
npx skills add jackwener/opencli
```
如果你只装了部分 skill,也可以只刷新自己在用的:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill smart-search
```
## 面向开发者
从源码安装:
@@ -229,142 +180,31 @@ npm link
运行 `opencli list` 查看完整注册表。
| 站点 | 命令 | 模式 |
|------|------|------|
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` | 浏览器 |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` | 浏览器 |
| **tieba** | `hot` `posts` `search` `read` | 浏览器 |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` | 浏览器 |
| **cursor** | `status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export` | 桌面端 |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` | 浏览器 |
| **codex** | `status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `projects` `history` `export` | 桌面端 |
| **chatwise** | `status` `new` `send` `read` `ask` `model` `history` `export` `screenshot` | 桌面端 |
| **doubao** | `status` `new` `send` `read` `ask` `history` `detail` `meeting-summary` `meeting-transcript` | 浏览器 |
| **doubao-app** | `status` `new` `send` `read` `ask` `screenshot` `dump` | 桌面端 |
| **notion** | `status` `search` `read` `new` `write` `sidebar` `favorites` `export` | 桌面端 |
| **discord-app** | `status` `send` `read` `channels` `servers` `search` `members` | 桌面端 |
| **v2ex** | `hot` `latest` `topic` `node` `user` `member` `replies` `nodes` `daily` `me` `notifications` | 公开 / 浏览器 |
| **xueqiu** | `feed` `hot-stock` `hot` `search` `stock` `comments` `watchlist` `earnings-date` `fund-holdings` `fund-snapshot` | 浏览器 |
| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` `serve` | 桌面端 |
| **chatgpt-app** | `status` `new` `send` `read` `ask` `model` | 桌面端 |
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` | 浏览器 |
| **rednote** | `search` `note` `comments` `user` `download` `feed` `notifications` | 浏览器 |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` | 浏览器 |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` | 浏览器 |
| **uiverse** | `code` `preview` | 浏览器 |
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
| **baidu-scholar** | `search` | 公开 |
| **google-scholar** | `search` `cite` `profile` | 公开 |
| **gov-law** | `search` `recent` | 公开 |
| **gov-policy** | `search` `recent` | 公开 |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
| **wanfang** | `search` | 公开 |
| **xiaoyuzhou** | `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` `auth` | 本地凭证 |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` | 浏览器 |
| **weixin** | `download` | 浏览器 |
| **youtube** | `search` `video` `transcript` `comments` `channel` `playlist` `feed` `history` `watch-later` `subscriptions` `like` `unlike` `subscribe` `unsubscribe` | 浏览器 |
| **boss** | `search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats` | 浏览器 |
| **coupang** | `search` `add-to-cart` | 浏览器 |
| **bbc** | `news` | 公共 API |
| **bloomberg** | `main` `markets` `economics` `industries` `tech` `politics` `businessweek` `opinions` `feeds` `news` | 公共 API / 浏览器 |
| **ctrip** | `search` | 浏览器 |
| **devto** | `top` `tag` `user` | 公开 |
| **dictionary** | `search` `synonyms` `examples` | 公开 |
| **arxiv** | `search` `paper` | 公开 |
| **pubmed** | `search` `article` `author` `citations` `related` | 公开 |
| **openreview** | `search` `venue` `paper` `reviews` | 公开 |
| **paperreview** | `submit` `review` `feedback` | 公开 |
| **wikipedia** | `search` `summary` `random` `trending` | 公开 |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` | 公共 API |
| **jd** | `item` | 浏览器 |
| **linkedin** | `search` `timeline` | 浏览器 |
| **reuters** | `search` | 浏览器 |
| **smzdm** | `search` | 浏览器 |
| **web** | `read` | 浏览器 |
| **weibo** | `hot` `search` `feed` `user` `me` `post` `comments` | 浏览器 |
| **yahoo-finance** | `quote` | 浏览器 |
| **sinafinance** | `news` | 🌐 公开 |
| **barchart** | `quote` `options` `greeks` `flow` | 浏览器 |
| **chaoxing** | `assignments` `exams` | 浏览器 |
| **grok** | `ask` `image` | 浏览器 |
| **hf** | `top` | 公开 |
| **jike** | `feed` `search` `create` `like` `comment` `repost` `notifications` `post` `topic` `user` | 浏览器 |
| **jimeng** | `generate` `history` | 浏览器 |
| **yollomi** | `generate` `video` `edit` `upload` `models` `remove-bg` `upscale` `face-swap` `restore` `try-on` `background` `object-remover` | 浏览器 |
| **linux-do** | `feed` `search` `categories` `tags` `topic` `topic-content` `user-posts` `user-topics` | 浏览器 |
| **stackoverflow** | `hot` `search` `bounties` `unanswered` | 公开 |
| **steam** | `top-sellers` | 公开 |
| **weread** | `shelf` `search` `book` `highlights` `notes` `notebooks` `ranking` | 浏览器 |
| **douban** | `search` `top250` `subject` `photos` `download` `marks` `reviews` `movie-hot` `book-hot` | 浏览器 |
| **facebook** | `feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group` | 浏览器 |
| **google** | `news` `search` `suggest` `trends` | 公开 |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` | 浏览器 |
| **1688** | `search` `item` `assets` `download` `store` | 浏览器 |
| **gitee** | `trending` `search` `user` | 公开 / 浏览器 |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` | 浏览器 |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` | 浏览器 |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` | OAuth API |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` | 浏览器 |
| **36kr** | `news` `hot` `search` `article` | 公开 / 浏览器 |
| **imdb** | `search` `title` `top` `trending` `person` `reviews` | 公开 |
| **producthunt** | `posts` `today` `hot` `browse` | 公开 / 浏览器 |
| **instagram** | `explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved` | 浏览器 |
| **lobsters** | `hot` `newest` `active` `tag` `read` | 公开 |
| **medium** | `feed` `search` `user` | 浏览器 |
| **sinablog** | `hot` `search` `article` `user` | 浏览器 |
| **substack** | `feed` `search` `publication` | 浏览器 |
| **pixiv** | `ranking` `search` `user` `illusts` `detail` `download` | 浏览器 |
| **tiktok** | `explore` `search` `profile` `user` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `live` `notifications` `friends` | 浏览器 |
| **bluesky** | `search` `trending` `user` `profile` `thread` `feeds` `followers` `following` `starter-packs` | 公开 |
| **xianyu** | `search` `item` `chat` `publish` | 浏览器 |
| **douyin** | `videos` `publish` `drafts` `draft` `delete` `stats` `profile` `update` `hashtag` `location` `activities` `collections` | 浏览器 |
| **yuanbao** | `new` `ask` | 浏览器 |
| 站点 | 命令 |
|------|------|
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `summary` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **linkedin** | `connect` `inbox` `safe-send` `search` `people-search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` |
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
100+ 站点能力**[→ 查看完整命令列表](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast``podcast-episodes``episode``download``transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`
精选清单**[→ 查看全部 100+ 站点和命令](./docs/adapters/index.md)**(小红书 / B站 / 知乎 / Twitter / Reddit / 抖音 / 微博 / 微信读书 / 小宇宙 / 1688 / 夸克 / Spotify / 牛客 / arxiv / Bilibili / 等)。
### 外部 CLI 枢纽
OpenCLI 也可以作为你现有命令行工具统一入口,负责发现、自动安装和纯透传执行。
现有命令行工具统一接入 `opencli <tool> ...`
| 外部 CLI | 描述 | 示例 |
|----------|------|------|
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
| **obsidian** | Obsidian 仓库管理 | `opencli obsidian search query="AI"` |
| **docker** | Docker 命令行工具 | `opencli docker ps` |
| **lark-cli** | 飞书 CLI — 消息、文档、日历、任务,200+ 命令 | `opencli lark-cli calendar +agenda` |
| **dws** | 钉钉 CLI — 钉钉全套产品能力的跨平台命令行工具,支持人类和 AI Agent 使用 | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | 企业微信 CLI — 企业微信开放平台命令行工具,支持人类和 AI Agent 使用 | `opencli wecom-cli msg send --to user "hello"` |
| **tg-cli** | Telegram CLI — 基于 MTProto 的本地优先同步、搜索、导出,面向 AI Agent | `opencli tg search "AI news" -f json` |
| **discord-cli** | Discord CLI — 基于 SQLite 的本地优先同步、搜索、导出,面向 AI Agent | `opencli discord recent --channel general` |
| **wx-cli** | 微信本地数据 CLI — 会话、聊天记录、搜索、联系人、导出 | `opencli wx search "OpenCLI"` |
| **vercel** | Vercel — 部署项目、管理域名、环境变量、日志 | `opencli vercel deploy --prod` |
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
**零配置透传**:OpenCLI 会把你的输入原样转发给底层二进制,保留原生 stdout / stderr 行为
注册自定义本地 CLI`opencli external register <name>`;查看所有:`opencli external list`
**自动安装**:如果你运行 `opencli gh ...` 时系统中还没有 `gh`,OpenCLI 会优先尝试通过系统包管理器安装,然后自动重试命令
**注册自定义本地 CLI**
```bash
opencli register mycli
```
### 桌面应用适配器
每个桌面适配器都有自己详细的文档说明,包括命令参考、启动配置与使用示例:
| 应用 | 描述 | 文档 |
|-----|-------------|-----|
| **Cursor** | 控制 Cursor IDE — Composer、对话、代码提取等 | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | 在后台(无头)驱动 OpenAI Codex CLI Agent | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | 在终端直接控制 Antigravity Ultra | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT App** | 自动化操作 ChatGPT macOS 桌面客户端 | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | 多 LLM 客户端(GPT-4、Claude、Gemini | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | 搜索、读取、写入 Notion 页面 | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord 桌面版 — 消息、频道、服务器 | [Doc](./docs/adapters/desktop/discord.md) |
| **Doubao** | 通过 CDP 控制豆包桌面应用 | [Doc](./docs/adapters/desktop/doubao-app.md) |
**桌面应用适配器**Electron,通过 CDP):Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — 详见 [`docs/adapters/desktop/`](./docs/adapters/desktop/)
## 下载支持
@@ -502,20 +342,6 @@ opencli plugin uninstall my-tool # 卸载
详见 [插件指南](./docs/zh/guide/plugins.md) 了解如何创建自己的插件。
## 致 AI Agent(开发者指南)
如果你是一个被要求查阅代码并编写新 `opencli` 适配器的 AI,请遵守以下工作流。
在动代码前,先读 [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md)。它把整个流程串起来:
- 侦察站点,选定 patternSPA / SSR / JSONP / Token / Streaming
-`opencli browser <name> network``eval`、interceptor 等找到目标 endpoint
- 定认证策略(`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
- 先用 `opencli browser recon analyze <url>` 一步侦察,再字段解码、设计 columns、`opencli browser recon init` 生成骨架
- 交付前用 `opencli browser recon verify <site>/<name>` 验证
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。
## 常见问题排查
- **"Extension not connected" 报错**
+2528 -274
View File
File diff suppressed because it is too large Load Diff
+73
View File
@@ -0,0 +1,73 @@
/**
* 12306 account summary for the logged-in user.
*
* Returns non-sensitive identity fields plus masked email / mobile.
* Use `--include-sensitive` to surface unmasked values from 12306's
* own response (12306 already masks the ID number server-side; this
* adapter never decodes that mask).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskEmail, maskMobile, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const ACCOUNT_INFO_URL = 'https://kyfw.12306.cn/otn/modifyUser/initQueryUserInfoApi';
cli({
site: '12306',
name: 'me',
access: 'read',
description: 'Show the logged-in 12306 account summary. Sensitive fields (real name, email, mobile, birth date) are masked by default; pass --include-sensitive to opt in.',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real name / email / mobile / birth date. The 12306 ID-number mask is server-side and never decoded.' },
],
columns: ['username', 'real_name', 'email', 'mobile', 'birth_date', 'sex', 'country', 'user_type', 'member', 'active'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 me');
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const json = requireEvaluateObject(await page.evaluate(`async () => {
const r = await fetch(${JSON.stringify(ACCOUNT_INFO_URL)}, { credentials: 'include' });
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'account info');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for account info`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 account info returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
}
if (json?.status !== true || !json?.data?.userDTO) {
throw new CommandExecutionError('12306 account info payload missing userDTO');
}
const dto = json.data.userDTO;
const loginDto = dto.loginUserDTO || {};
const username = loginDto.user_name || loginDto.name || '';
const realName = loginDto.real_name || loginDto.realname || '';
const include = kwargs['include-sensitive'] === true;
return [{
username,
real_name: include ? realName : maskChineseName(realName),
email: include ? (dto.email || '') : maskEmail(dto.email || ''),
mobile: include ? (dto.mobile_no || '') : maskMobile(dto.mobile_no || ''),
birth_date: include ? (dto.born_date || '') : (dto.born_date || '').slice(0, 4),
sex: dto.sex_code === 'M' ? '男' : (dto.sex_code === 'F' ? '女' : ''),
country: dto.country_code || '',
user_type: json.data.userTypeName || '',
member: dto.flag_member === '1',
active: dto.is_active === '1',
}];
},
});
+96
View File
@@ -0,0 +1,96 @@
/**
* 12306 in-progress orders for the logged-in user.
*
* Returns orders that have not yet been ridden / refunded / completed
* (the `noComplete` slice). Order history covering completed and
* refunded tickets uses a separate endpoint that requires extra
* referer / page-state handshakes and is left for a follow-up so this
* command can ship reliably.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const NO_COMPLETE_URL = 'https://kyfw.12306.cn/otn/queryOrder/queryMyOrderNoComplete';
cli({
site: '12306',
name: 'orders',
access: 'read',
description: 'List in-progress 12306 orders (not yet ridden, refunded, or completed) for the logged-in user',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked passenger names in order rows. Masked by default.' },
],
columns: ['order_id', 'order_date', 'train_code', 'from_station', 'to_station', 'departure', 'passengers', 'status', 'amount'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 orders');
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const include = kwargs['include-sensitive'] === true;
const json = requireEvaluateObject(await page.evaluate(`async () => {
const r = await fetch(${JSON.stringify(NO_COMPLETE_URL)}, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: '_json_att=', credentials: 'include',
});
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'orders');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for queryMyOrderNoComplete`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 orders returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
}
if (json?.status !== true) {
throw new CommandExecutionError('12306 queryMyOrderNoComplete returned a failure status');
}
let orders;
if (Array.isArray(json?.data?.orderDBList)) {
orders = json.data.orderDBList;
} else if (Array.isArray(json?.data?.orderDTODataList)) {
orders = json.data.orderDTODataList;
} else if (Array.isArray(json?.data?.orders)) {
orders = json.data.orders;
} else if (Array.isArray(json?.data)) {
orders = json.data;
} else {
throw new CommandExecutionError('12306 queryMyOrderNoComplete payload missing order list array');
}
if (orders.length === 0) {
throw new EmptyResultError('No in-progress 12306 orders on this account');
}
return orders.map((o) => {
const tickets = Array.isArray(o.tickets) ? o.tickets : [];
const passengerNames = tickets
.map((t) => t.passenger_name || '')
.filter(Boolean)
.map((name) => include ? name : maskChineseName(name))
.join(', ');
return {
order_id: o.sequence_no || o.order_id || o.sequenceNo || '',
order_date: o.order_date || '',
train_code: o.train_code_page || o.station_train_code || o.train_code || '',
from_station: o.from_station_name_page || o.from_station_name || '',
to_station: o.to_station_name_page || o.to_station_name || '',
departure: o.start_train_date_page || o.start_train_date || '',
passengers: passengerNames,
status: o.ticket_status_name || o.order_status_name || o.statusName || '',
amount: o.ticket_total_price_page || o.ticket_total_price || '',
};
});
},
});
+90
View File
@@ -0,0 +1,90 @@
/**
* 12306 saved passenger list for the logged-in user.
*
* 12306 already masks ID numbers (`xxxx***********xxx`) and mobile
* numbers (`138****xxxx`) server-side. This adapter further masks the
* passenger's Chinese real name and birth date by default; pass
* `--include-sensitive` to surface the unmasked-by-12306 fields.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
const PASSENGER_QUERY_URL = 'https://kyfw.12306.cn/otn/passengers/query';
const MAX_PAGE_SIZE = 50;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) throw new ArgumentError(`limit must be a positive integer (1-${max})`);
if (n > max) throw new ArgumentError(`limit must be <= ${max}`);
return n;
}
cli({
site: '12306',
name: 'passengers',
access: 'read',
description: 'List the logged-in user\'s saved 12306 passengers. Sensitive fields are masked by default; pass --include-sensitive to opt in.',
domain: 'kyfw.12306.cn',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'limit', type: 'int', default: 20, help: `Max passengers to return (1-${MAX_PAGE_SIZE})` },
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real names and birth dates. The 12306 ID-number / mobile masks are server-side and never decoded.' },
],
columns: ['name', 'sex', 'born_year', 'id_type', 'id_no', 'mobile', 'passenger_type', 'country'],
func: async (page, kwargs) => {
if (!page) throw new CommandExecutionError('Browser session required for 12306 passengers');
const limit = normalizeLimit(kwargs.limit, 20, MAX_PAGE_SIZE);
const include = kwargs['include-sensitive'] === true;
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
await require12306Login(page, AuthRequiredError);
const json = requireEvaluateObject(await page.evaluate(`async () => {
const body = "pageIndex=1&pageSize=${MAX_PAGE_SIZE}";
const r = await fetch(${JSON.stringify(PASSENGER_QUERY_URL)}, {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body, credentials: 'include',
});
if (!r.ok) return { __http: r.status };
try {
return await r.json();
} catch (err) {
return { __parse: String(err && err.message || err) };
}
}`), 'passengers');
if (json?.__http) {
if ([401, 403].includes(Number(json.__http))) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
}
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for passengers/query`);
}
if (json?.__parse) {
throw new CommandExecutionError(`12306 passengers returned non-JSON body: ${json.__parse}`);
}
if (isAuthLikePayload(json)) {
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
}
if (json?.status !== true || !Array.isArray(json?.data?.datas)) {
throw new CommandExecutionError('12306 passengers payload missing data.datas array');
}
const datas = json.data.datas;
if (datas.length === 0) {
throw new EmptyResultError('No saved passengers on this 12306 account');
}
return datas.slice(0, limit).map((p) => ({
name: include ? (p.passenger_name || '') : maskChineseName(p.passenger_name || ''),
sex: p.sex_name || '',
born_year: (p.born_date || '').slice(0, 4),
id_type: p.passenger_id_type_name || '',
id_no: p.passenger_id_no || '',
mobile: p.mobile_no || '',
passenger_type: p.passenger_type_name || '',
country: p.country_code || '',
}));
},
});
export const __test__ = { normalizeLimit };
+166
View File
@@ -0,0 +1,166 @@
/**
* 12306 ticket price lookup for a single train + segment.
*
* Cascades three anonymous API calls:
* 1. /otn/leftTicket/init: mint session cookies
* 2. /otn/czxx/queryByTrainNo: resolve from/to station_no within the
* train route (price endpoint addresses stops by station_no, not
* telecode)
* 3. /otn/leftTicket/queryTicketPrice: ticket prices keyed by seat
* letter (M=一等座, O=二等座, A9=商务座, A1=硬座, A3=硬卧,
* A4=软卧, F=动卧, P=特等座, WZ=无座, etc.)
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
const SEAT_TYPES_RE = /^[A-Z0-9]{1,32}$/;
const SEAT_LETTERS = {
'A9': '商务座',
'P': '特等座',
'M': '一等座',
'O': '二等座',
'A1': '硬座',
'A3': '硬卧',
'A4': '软卧',
'F': '动卧',
'WZ': '无座',
};
async function queryStopsForPrice(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
}
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
throw new CommandExecutionError('12306 queryByTrainNo returned an unexpected payload shape');
}
return json.data.data;
}
function pickStationNos(stops, fromCode, toCode, fromName, toName) {
const matches = (s, code, name) => (s.station_name && name && s.station_name === name);
const fromStop = stops.find((s) => matches(s, fromCode, fromName));
const toStop = stops.find((s) => matches(s, toCode, toName));
if (!fromStop) throw new CommandExecutionError(`Train does not stop at ${fromName}`);
if (!toStop) throw new CommandExecutionError(`Train does not stop at ${toName}`);
return { fromNo: fromStop.station_no, toNo: toStop.station_no };
}
async function queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/leftTicket/queryTicketPrice?train_no=${trainNo}&from_station_no=${fromNo}&to_station_no=${toNo}&seat_types=${seatTypes}&train_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) throw new CommandExecutionError(`12306 queryTicketPrice returned HTTP ${resp.status}`);
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryTicketPrice returned non-JSON body');
}
if (json?.status !== true || !json?.data) {
throw new CommandExecutionError('12306 queryTicketPrice returned an unexpected payload shape');
}
return json.data;
}
function parsePriceData(priceData) {
const rows = [];
for (const [letter, value] of Object.entries(priceData)) {
if (letter === 'train_no' || letter === 'OT') continue;
if (typeof value !== 'string' || !value) continue;
// 12306 doubles up some prices as bare numerics ("9": "21580"), which
// mirror their letter sibling ("A9": "¥2158.0") in cents/no-decimal
// form. Skip the bare numeric letter codes to avoid duplicates.
if (/^\d+$/.test(letter)) continue;
if (!/^[A-Z]/.test(letter)) continue;
const numeric = value.replace(/^¥/, '');
if (!/^[\d.]+$/.test(numeric)) continue;
rows.push({
seat_code: letter,
seat_name: SEAT_LETTERS[letter] || letter,
price: numeric,
currency: 'CNY',
});
}
rows.sort((a, b) => Number(b.price) - Number(a.price));
return rows;
}
cli({
site: '12306',
name: 'price',
access: 'read',
description: 'Look up 12306 ticket prices by seat class for one train on a given date and segment (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L)' },
{ name: 'from', required: true, help: 'Origin station (Chinese name, telecode, or pinyin) - must be a stop of this train' },
{ name: 'to', required: true, help: 'Destination station - must be a stop of this train' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
{ name: 'seat-types', default: 'OM9PA1A3A4FWZ', help: 'Seat-type letters to query (default covers the common classes). Examples: OM9 (二等/一等/商务), A1A3A4 (硬座/硬卧/软卧).' },
],
columns: ['seat_code', 'seat_name', 'price', 'currency'],
func: async (kwargs) => {
const trainNo = String(kwargs['train-no'] ?? '').trim();
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
if (!TRAIN_NO_RE.test(trainNo)) {
throw new ArgumentError(
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
);
}
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('--from station must not be empty');
if (!toArg) throw new ArgumentError('--to station must not be empty');
const date = validateDate(kwargs.date);
const seatTypes = String(kwargs['seat-types'] ?? '').trim() || 'OM9PA1A3A4FWZ';
if (!SEAT_TYPES_RE.test(seatTypes)) {
throw new ArgumentError('--seat-types must contain only 12306 seat letters/digits (A-Z, 0-9)');
}
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const cookieHeader = await mintSession();
const stops = await queryStopsForPrice(cookieHeader, trainNo, fromStation.code, toStation.code, date);
const { fromNo, toNo } = pickStationNos(stops, fromStation.code, toStation.code, fromStation.name, toStation.name);
const priceData = await queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date);
const rows = parsePriceData(priceData);
if (rows.length === 0) {
throw new EmptyResultError(
`No prices returned for train_no=${trainNo} ${fromStation.name} -> ${toStation.name} on ${date}`,
'Try a different seat-types letter set, or check that this train operates on the date.',
);
}
return rows;
},
});
export const __test__ = { parsePriceData, pickStationNos, queryStopsForPrice, queryPrice, SEAT_LETTERS };
+66
View File
@@ -0,0 +1,66 @@
/**
* 12306 station search.
*
* Queries the public `station_name.js` bundle and filters by the user's
* keyword. Anonymous, no session needed.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle } from './utils.js';
const MAX_LIMIT = 50;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) {
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
}
if (n > max) {
throw new ArgumentError(`limit must be <= ${max}`);
}
return n;
}
cli({
site: '12306',
name: 'stations',
access: 'read',
description: 'Search 12306 (China Railway) stations by Chinese name, telecode, or pinyin keyword',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Chinese substring (上海), telecode (AOH), or pinyin (shanghai)' },
{ name: 'limit', type: 'int', default: 20, help: `Maximum results (1-${MAX_LIMIT})` },
],
columns: ['name', 'code', 'pinyin', 'abbr', 'city'],
func: async (kwargs) => {
const keyword = String(kwargs.keyword ?? '').trim();
if (!keyword) throw new ArgumentError('keyword must not be empty');
const limit = normalizeLimit(kwargs.limit, 20, MAX_LIMIT);
const stations = await fetchStationBundle();
const lower = keyword.toLowerCase();
const matches = stations.filter((s) =>
s.name.includes(keyword)
|| s.code === keyword.toUpperCase()
|| s.pinyin.includes(lower)
|| s.abbr.includes(lower)
|| s.short.includes(lower)
|| s.city.includes(keyword),
);
if (matches.length === 0) {
throw new EmptyResultError(`No 12306 stations match "${keyword}"`);
}
return matches.slice(0, limit).map((s) => ({
name: s.name,
code: s.code,
pinyin: s.pinyin,
abbr: s.abbr,
city: s.city,
}));
},
});
export const __test__ = { normalizeLimit };
+91
View File
@@ -0,0 +1,91 @@
/**
* 12306 train stop details - list every station a train calls at,
* with arrival / departure / stopover time.
*
* Requires the internal `train_no` returned by `12306 trains`
* (`24000000G10L`), not the public train code (`G1`).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
async function queryStops(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
const resp = await fetchImpl(url, {
headers: {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
},
});
if (!resp.ok) {
throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
}
let json;
try {
json = await resp.json();
} catch {
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
}
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
throw new CommandExecutionError(`12306 queryByTrainNo returned an unexpected payload shape`);
}
return json.data.data;
}
cli({
site: '12306',
name: 'train',
access: 'read',
description: 'List every station a 12306 train calls at, with arrival / departure / stopover time (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L), not the public code (G1)' },
{ name: 'from', required: true, help: 'Origin station for the segment: Chinese name, telecode, or pinyin' },
{ name: 'to', required: true, help: 'Destination station for the segment' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
],
columns: ['station_no', 'station_name', 'arrive_time', 'start_time', 'stopover_time'],
func: async (kwargs) => {
const trainNo = String(kwargs['train-no'] ?? '').trim();
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
if (!TRAIN_NO_RE.test(trainNo)) {
throw new ArgumentError(
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
);
}
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('--from station must not be empty');
if (!toArg) throw new ArgumentError('--to station must not be empty');
const date = validateDate(kwargs.date);
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const cookieHeader = await mintSession();
const stops = await queryStops(cookieHeader, trainNo, fromStation.code, toStation.code, date);
if (stops.length === 0) {
throw new EmptyResultError(`No stops returned for train_no=${trainNo} on ${date}`);
}
return stops.map((s) => ({
station_no: s.station_no || '',
station_name: s.station_name || '',
arrive_time: s.arrive_time === '----' ? '' : (s.arrive_time || ''),
start_time: s.start_time === '----' ? '' : (s.start_time || ''),
stopover_time: s.stopover_time === '----' ? '' : (s.stopover_time || ''),
}));
},
});
export const __test__ = { queryStops, TRAIN_NO_RE };
+119
View File
@@ -0,0 +1,119 @@
/**
* 12306 train availability between two stations on a given date.
*
* Flow:
* 1. Fetch the station bundle (cached implicitly via per-process module state).
* 2. Mint anonymous session cookies via /otn/leftTicket/init.
* 3. Query /otn/leftTicket/queryG; if 12306 returns
* `{c_url: "leftTicket/queryX"}` (endpoint rotation), retry once
* against the suggested name.
* 4. Parse the `|`-separated train records.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchStationBundle, mintSession, resolveStation, validateDate, parseTrainRecord } from './utils.js';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const QUERY_ENDPOINTS = ['queryG', 'queryO', 'queryZ', 'queryA'];
const MAX_LIMIT = 100;
function normalizeLimit(value, defaultValue, max) {
if (value === undefined || value === null || value === '') return defaultValue;
const n = Number(value);
if (!Number.isInteger(n) || n < 1) {
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
}
if (n > max) {
throw new ArgumentError(`limit must be <= ${max}`);
}
return n;
}
async function queryLeftTickets(cookieHeader, fromCode, toCode, date) {
const headers = {
'User-Agent': UA,
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
'Cookie': cookieHeader,
};
const queryParams = `leftTicketDTO.train_date=${date}&leftTicketDTO.from_station=${fromCode}&leftTicketDTO.to_station=${toCode}&purpose_codes=ADULT`;
let lastResponseText = '';
for (const endpoint of QUERY_ENDPOINTS) {
const url = `https://kyfw.12306.cn/otn/leftTicket/${endpoint}?${queryParams}`;
const resp = await fetch(url, { headers });
if (!resp.ok) {
if (resp.status === 302) continue;
throw new CommandExecutionError(`12306 ${endpoint} returned HTTP ${resp.status}`);
}
const text = await resp.text();
lastResponseText = text;
let json;
try { json = JSON.parse(text); } catch {
throw new CommandExecutionError(`12306 ${endpoint} returned non-JSON body`);
}
if (json?.c_url && typeof json.c_url === 'string') {
const rotated = json.c_url.replace('leftTicket/', '').trim();
if (rotated && !QUERY_ENDPOINTS.includes(rotated)) {
QUERY_ENDPOINTS.unshift(rotated);
}
continue;
}
if (Array.isArray(json?.data?.result)) {
return json.data.result;
}
throw new CommandExecutionError(`12306 ${endpoint} returned an unexpected payload shape`);
}
throw new CommandExecutionError(`12306 rejected every known query endpoint name (${QUERY_ENDPOINTS.join(', ')}); the wire protocol may have changed. Last body: ${lastResponseText.slice(0, 200)}`);
}
cli({
site: '12306',
name: 'trains',
access: 'read',
description: 'List trains between two 12306 stations on a given date (anonymous, no login required)',
domain: 'kyfw.12306.cn',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'from', positional: true, required: true, help: 'Origin station: Chinese name (北京), telecode (BJP), or pinyin (beijing)' },
{ name: 'to', positional: true, required: true, help: 'Destination station: same forms as <from>' },
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
{ name: 'limit', type: 'int', default: 50, help: `Maximum rows (1-${MAX_LIMIT})` },
],
columns: [
'code', 'from_station', 'to_station', 'start_time', 'arrive_time',
'duration', 'available', 'business_seat', 'first_seat', 'second_seat',
'soft_sleeper', 'hard_sleeper', 'hard_seat', 'no_seat', 'train_no',
],
func: async (kwargs) => {
const fromArg = String(kwargs.from ?? '').trim();
const toArg = String(kwargs.to ?? '').trim();
if (!fromArg) throw new ArgumentError('<from> station must not be empty');
if (!toArg) throw new ArgumentError('<to> station must not be empty');
const date = validateDate(kwargs.date);
const limit = normalizeLimit(kwargs.limit, 50, MAX_LIMIT);
const stations = await fetchStationBundle();
const fromStation = resolveStation(stations, fromArg);
const toStation = resolveStation(stations, toArg);
if (fromStation.code === toStation.code) {
throw new ArgumentError(`<from> and <to> must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
}
const stationByCode = new Map(stations.map((s) => [s.code, s]));
const cookieHeader = await mintSession();
const rawRows = await queryLeftTickets(cookieHeader, fromStation.code, toStation.code, date);
const decoded = rawRows
.map((line) => parseTrainRecord(decodeURIComponent(line.replace(/%0A/g, '')), stationByCode))
.filter(Boolean);
if (decoded.length === 0) {
throw new EmptyResultError(
`No trains found from ${fromStation.name} to ${toStation.name} on ${date}`,
'Try a different date or check whether the route is operated by 12306.',
);
}
return decoded.slice(0, limit);
},
});
export const __test__ = { normalizeLimit, queryLeftTickets };
+272
View File
@@ -0,0 +1,272 @@
/**
* 12306 (中国铁路) shared helpers.
*
* - Station lookup: parses the public `station_name.js` bundle into
* structured records.
* - Cookie session: 12306's query endpoints reject anonymous requests
* with `HTTP 302 -> error.html`, so callers must hit `/otn/leftTicket/init`
* first to mint the JSESSIONID / route / BIGipServerotn cookies.
* - Query endpoint rotation: 12306 rotates the train-query endpoint
* name (queryO / queryZ / queryA / queryG / ...) every few weeks.
* When the wrong name is hit, the server returns
* `{"c_url":"leftTicket/queryG","c_name":"CLeftTicketUrl","status":false}`
* pointing to the current correct name; retry once with that name.
*/
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
const STATION_BUNDLE_URL = 'https://kyfw.12306.cn/otn/resources/js/framework/station_name.js';
const INIT_URL = 'https://kyfw.12306.cn/otn/leftTicket/init';
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
const STATION_CODE_RE = /^[A-Z]{2,4}$/;
/**
* Parse the `station_name.js` bundle into a station record array.
*
* Bundle format (single line, `@`-delimited records, each `|`-delimited):
* `var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||...';`
*
* Per-record fields (positional):
* [0] short pinyin alias (e.g. `bjb`)
* [1] Chinese station name (e.g. `北京北`)
* [2] telecode (3-4 uppercase letters, e.g. `VAP`) - this is the
* wire format 12306 uses for `from_station` / `to_station`.
* [3] full pinyin (e.g. `beijingbei`)
* [4] short alias (duplicate of [0] usually)
* [5] index/rank
* [6] city code
* [7] city name (e.g. `北京`)
*/
export function parseStationBundle(text) {
const match = text.match(/'([^']+)'/);
if (!match) {
throw new CommandExecutionError('Failed to parse 12306 station_name.js: source string not found');
}
const raw = match[1];
const records = raw.split('@').filter(Boolean);
const stations = [];
for (const r of records) {
const parts = r.split('|');
if (parts.length < 8 || !parts[2]) continue;
stations.push({
short: parts[0] || '',
name: parts[1] || '',
code: parts[2] || '',
pinyin: parts[3] || '',
abbr: parts[4] || '',
city: parts[7] || '',
});
}
if (stations.length === 0) {
throw new CommandExecutionError('Failed to parse 12306 station_name.js: no station records found');
}
return stations;
}
/**
* Resolve a user-supplied station identifier to a telecode.
*
* Accepts Chinese name (`上海虹桥`), telecode (`AOH`), pinyin
* (`shanghaihongqiao`), short alias (`shh`), or city name with a
* preference for the city's main station.
*/
export function resolveStation(stations, input) {
const trimmed = String(input ?? '').trim();
if (!trimmed) throw new ArgumentError('station must not be empty');
if (STATION_CODE_RE.test(trimmed)) {
const exact = stations.find((s) => s.code === trimmed);
if (exact) return exact;
throw new ArgumentError(`Unknown 12306 station telecode "${trimmed}"`);
}
const lower = trimmed.toLowerCase();
const exactName = stations.find((s) => s.name === trimmed);
if (exactName) return exactName;
const exactPinyin = stations.find((s) => s.pinyin === lower);
if (exactPinyin) return exactPinyin;
const exactAbbr = stations.find((s) => s.abbr === lower || s.short === lower);
if (exactAbbr) return exactAbbr;
throw new ArgumentError(`Unknown 12306 station "${trimmed}"`, 'Try the Chinese name (上海虹桥), the 3-4 letter telecode (AOH), or full pinyin (shanghaihongqiao).');
}
export function validateDate(value) {
if (!DATE_RE.test(String(value ?? ''))) {
throw new ArgumentError(`date must be YYYY-MM-DD, got "${value}"`);
}
const [y, m, d] = value.split('-').map(Number);
const date = new Date(Date.UTC(y, m - 1, d));
if (date.getUTCFullYear() !== y || date.getUTCMonth() !== m - 1 || date.getUTCDate() !== d) {
throw new ArgumentError(`date "${value}" is not a real calendar date`);
}
return value;
}
/** Extract Set-Cookie header values into a single `Cookie:` header string. */
export function buildCookieHeader(setCookieHeaders) {
if (!Array.isArray(setCookieHeaders) || setCookieHeaders.length === 0) return '';
return setCookieHeaders
.map((line) => line.split(';')[0])
.filter(Boolean)
.join('; ');
}
export async function fetchStationBundle(fetchImpl = fetch) {
const resp = await fetchImpl(STATION_BUNDLE_URL, {
headers: { 'User-Agent': UA },
});
if (!resp.ok) {
throw new CommandExecutionError(`Failed to fetch 12306 station bundle: HTTP ${resp.status}`);
}
return parseStationBundle(await resp.text());
}
/** Mint a 12306 anonymous session by hitting /otn/leftTicket/init. */
export async function mintSession(fetchImpl = fetch) {
const resp = await fetchImpl(INIT_URL, {
headers: { 'User-Agent': UA },
redirect: 'follow',
});
if (!resp.ok) {
throw new CommandExecutionError(`Failed to mint 12306 session: HTTP ${resp.status}`);
}
const setCookies = typeof resp.headers.getSetCookie === 'function'
? resp.headers.getSetCookie()
: resp.headers.raw?.()['set-cookie'] || [];
const cookieHeader = buildCookieHeader(setCookies);
if (!cookieHeader) {
throw new CommandExecutionError('12306 init returned no session cookies');
}
return cookieHeader;
}
/**
* Twelve-row train query record (LEFT_TICKET_DTO).
*
* 12306 returns each train as a `|`-separated string with ~36 fields.
* Positions used here come from the public web client; unused
* positions are documented inline so future maintainers can extend
* the row shape without re-reverse-engineering.
*/
export function parseTrainRecord(line, stationByCode) {
const f = line.split('|');
if (f.length < 33) return null;
return {
train_no: f[2] || '',
code: f[3] || '',
from_station: stationByCode.get(f[6])?.name || f[6] || '',
to_station: stationByCode.get(f[7])?.name || f[7] || '',
from_code: f[6] || '',
to_code: f[7] || '',
start_time: f[8] || '',
arrive_time: f[9] || '',
duration: f[10] || '',
available: (f[1] || '').trim() === '预订' || (f[11] || '').trim() === 'Y',
business_seat: f[32] || '',
first_seat: f[31] || '',
second_seat: f[30] || '',
soft_sleeper: f[23] || '',
hard_sleeper: f[28] || '',
hard_seat: f[29] || '',
no_seat: f[26] || '',
};
}
/**
* Mask helpers for sensitive identity fields rendered by 12306.
*
* 12306 already masks ID numbers and mobile numbers server-side
* (`xxxx***********xxx` / `138****xxxx`); these helpers handle the
* remaining fields (email, real Chinese name) so the adapter never
* leaks unmasked PII without an explicit `--include-sensitive` opt-in.
*/
export function maskEmail(value) {
const v = String(value || '').trim();
if (!v) return '';
const at = v.indexOf('@');
if (at <= 0) return v;
const local = v.slice(0, at);
const domain = v.slice(at);
if (local.length <= 2) return local[0] + '*' + domain;
return local[0] + '*'.repeat(Math.max(1, local.length - 2)) + local.slice(-1) + domain;
}
export function maskMobile(value) {
const v = String(value || '').trim();
if (!v) return '';
if (/\*/.test(v)) return v;
if (v.length < 7) return v.replace(/.(?=.)/g, '*');
return v.slice(0, 3) + '*'.repeat(v.length - 7) + v.slice(-4);
}
export function maskChineseName(value) {
const v = String(value || '').trim();
if (!v) return '';
if (v.length === 1) return v;
if (v.length === 2) return v[0] + '*';
return v[0] + '*'.repeat(v.length - 2) + v.slice(-1);
}
export function unwrapEvaluateResult(value) {
if (
value
&& typeof value === 'object'
&& !Array.isArray(value)
&& Object.prototype.hasOwnProperty.call(value, 'session')
&& Object.prototype.hasOwnProperty.call(value, 'data')
) {
return value.data;
}
return value;
}
export function requireEvaluateObject(value, label) {
const payload = unwrapEvaluateResult(value);
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
throw new CommandExecutionError(`12306 ${label} returned a malformed browser payload`);
}
return payload;
}
export function isAuthLikePayload(payload) {
if (!payload || typeof payload !== 'object') return false;
const parts = [];
if (Array.isArray(payload.messages)) parts.push(...payload.messages);
if (payload.message) parts.push(payload.message);
if (payload.msg) parts.push(payload.msg);
if (payload.validateMessages && typeof payload.validateMessages === 'object') {
parts.push(...Object.values(payload.validateMessages).flat());
}
const text = parts.map((item) => String(item ?? '')).join(' ');
return /未登录|登录|请登录|身份|认证|session|Session|login/i.test(text);
}
/**
* Detect the 12306 login marker by reading `document.cookie` from the
* current adapter page. Cannot use `page.getCookies({url})` here:
* 12306 sets the auth cookie `tk` and `JSESSIONID` with `Path=/otn`,
* and CDP `Network.getCookies` with a bare URL filter excludes
* cookies whose path does not match the URL path. `document.cookie`
* returns all non-httponly cookies visible to the current page
* regardless of path, which is what we need to confirm login.
*/
export async function require12306Login(page, AuthRequiredErrorClass) {
const docCookie = unwrapEvaluateResult(await page.evaluate(`document.cookie || ''`));
const cookieStr = typeof docCookie === 'string' ? docCookie : '';
if (!/\btk=/.test(cookieStr) || !/JSESSIONID=/.test(cookieStr)) {
throw new AuthRequiredErrorClass('kyfw.12306.cn', 'Not logged into 12306. Sign in at https://kyfw.12306.cn first.');
}
}
export const __test__ = {
parseStationBundle,
resolveStation,
validateDate,
buildCookieHeader,
parseTrainRecord,
maskEmail,
maskMobile,
maskChineseName,
unwrapEvaluateResult,
requireEvaluateObject,
isAuthLikePayload,
};
+331
View File
@@ -0,0 +1,331 @@
import { describe, expect, it } from 'vitest';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { __test__ } from './utils.js';
import { __test__ as priceTest } from './price.js';
import { __test__ as trainTest } from './train.js';
import './orders.js';
const { parseStationBundle, resolveStation, validateDate, buildCookieHeader, parseTrainRecord, maskEmail, maskMobile, maskChineseName, unwrapEvaluateResult, requireEvaluateObject, isAuthLikePayload } = __test__;
const { parsePriceData, queryStopsForPrice, queryPrice } = priceTest;
const { queryStops } = trainTest;
describe('12306 utils - parseStationBundle', () => {
it('parses the `@`-delimited station bundle into structured records', () => {
const bundle = "var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||@bji|北京|BJP|beijing|bj|2|0357|北京|||@aoh|上海虹桥|AOH|shanghaihongqiao|shhq|10|7600|上海|||';";
const stations = parseStationBundle(bundle);
expect(stations).toHaveLength(3);
expect(stations[1]).toEqual({
short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京',
});
});
it('skips records that lack a telecode', () => {
const bundle = "var station_names ='@xxx|||||||||@bji|北京|BJP|beijing|bj|2|0357|北京|||';";
const stations = parseStationBundle(bundle);
expect(stations).toHaveLength(1);
expect(stations[0].code).toBe('BJP');
});
it('throws CommandExecutionError when the bundle has no parseable station rows', () => {
expect(() => parseStationBundle("var station_names ='@xxx|||||||||';")).toThrow(CommandExecutionError);
});
});
describe('12306 utils - resolveStation', () => {
const stations = [
{ short: 'bjb', name: '北京北', code: 'VAP', pinyin: 'beijingbei', abbr: 'bjb', city: '北京' },
{ short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京' },
{ short: 'aoh', name: '上海虹桥', code: 'AOH', pinyin: 'shanghaihongqiao', abbr: 'shhq', city: '上海' },
];
it('matches by exact Chinese name', () => {
expect(resolveStation(stations, '上海虹桥').code).toBe('AOH');
});
it('matches by uppercase telecode', () => {
expect(resolveStation(stations, 'BJP').code).toBe('BJP');
});
it('matches by full pinyin (case-insensitive)', () => {
expect(resolveStation(stations, 'Beijing').code).toBe('BJP');
});
it('matches by short alias / abbr', () => {
expect(resolveStation(stations, 'shhq').code).toBe('AOH');
});
it('throws ArgumentError for empty input', () => {
expect(() => resolveStation(stations, ' ')).toThrow(ArgumentError);
});
it('throws ArgumentError for unknown station', () => {
expect(() => resolveStation(stations, '某不存在站')).toThrow(ArgumentError);
});
it('throws ArgumentError for telecode-shaped but unknown input', () => {
expect(() => resolveStation(stations, 'XYZ')).toThrow(ArgumentError);
});
});
describe('12306 utils - validateDate', () => {
it('accepts valid YYYY-MM-DD', () => {
expect(validateDate('2026-05-22')).toBe('2026-05-22');
});
it('throws ArgumentError on wrong format', () => {
expect(() => validateDate('2026/05/22')).toThrow(ArgumentError);
expect(() => validateDate('26-05-22')).toThrow(ArgumentError);
expect(() => validateDate('today')).toThrow(ArgumentError);
expect(() => validateDate('')).toThrow(ArgumentError);
});
it('throws ArgumentError on impossible calendar dates', () => {
expect(() => validateDate('2026-02-30')).toThrow(ArgumentError);
expect(() => validateDate('2026-13-01')).toThrow(ArgumentError);
});
});
describe('12306 utils - buildCookieHeader', () => {
it('joins set-cookie lines into a single Cookie header', () => {
const headers = [
'JSESSIONID=ABC123; Path=/otn',
'BIGipServerotn=xxx.yyy; Path=/',
'route=zzz; Expires=Sat, 01 Jan 2027 00:00:00 GMT',
];
expect(buildCookieHeader(headers)).toBe('JSESSIONID=ABC123; BIGipServerotn=xxx.yyy; route=zzz');
});
it('returns empty string for empty input', () => {
expect(buildCookieHeader([])).toBe('');
expect(buildCookieHeader(undefined)).toBe('');
});
});
describe('12306 utils - parseTrainRecord', () => {
const stationByCode = new Map([
['VNP', { name: '北京南', code: 'VNP' }],
['AOH', { name: '上海虹桥', code: 'AOH' }],
]);
it('extracts the canonical train fields from a wire record', () => {
// 33 `|`-separated fields, with positions used by parseTrainRecord populated.
const fields = new Array(36).fill('');
fields[0] = 'SECRET_TOKEN';
fields[1] = '预订';
fields[2] = '240000G54700';
fields[3] = 'G547';
fields[6] = 'VNP';
fields[7] = 'AOH';
fields[8] = '06:18';
fields[9] = '12:11';
fields[10] = '05:53';
fields[11] = 'Y';
fields[23] = ''; // soft sleeper
fields[26] = '无'; // no seat
fields[28] = ''; // hard sleeper
fields[29] = ''; // hard seat
fields[30] = '有'; // second seat
fields[31] = '有'; // first seat
fields[32] = '无'; // business seat
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(row).toEqual({
train_no: '240000G54700',
code: 'G547',
from_station: '北京南',
to_station: '上海虹桥',
from_code: 'VNP',
to_code: 'AOH',
start_time: '06:18',
arrive_time: '12:11',
duration: '05:53',
available: true,
business_seat: '无',
first_seat: '有',
second_seat: '有',
soft_sleeper: '',
hard_sleeper: '',
hard_seat: '',
no_seat: '无',
});
});
it('does not expose the booking-handshake secret token', () => {
const fields = new Array(36).fill('');
fields[0] = 'SECRET_TOKEN_DO_NOT_LEAK';
fields[2] = 't_no'; fields[3] = 'X1'; fields[6] = 'VNP'; fields[7] = 'AOH';
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(Object.values(row)).not.toContain('SECRET_TOKEN_DO_NOT_LEAK');
expect('secret' in row).toBe(false);
});
it('falls back to the telecode when the station bundle has no name', () => {
const fields = new Array(36).fill('');
fields[2] = 'X'; fields[3] = 'X'; fields[6] = 'ZZZ'; fields[7] = 'YYY';
const row = parseTrainRecord(fields.join('|'), stationByCode);
expect(row.from_station).toBe('ZZZ');
expect(row.to_station).toBe('YYY');
});
it('returns null for short records', () => {
expect(parseTrainRecord('a|b|c', stationByCode)).toBeNull();
});
});
describe('12306 utils - mask helpers', () => {
it('masks the local-part of an email', () => {
expect(maskEmail('hello@example.com')).toBe('h***o@example.com');
expect(maskEmail('ab@x.cn')).toBe('a*@x.cn');
expect(maskEmail('a@x.cn')).toBe('a*@x.cn');
expect(maskEmail('')).toBe('');
expect(maskEmail('not-an-email')).toBe('not-an-email');
});
it('masks Chinese mobile numbers while preserving 12306-side masks', () => {
expect(maskMobile('13800001234')).toBe('138****1234');
expect(maskMobile('138****1234')).toBe('138****1234');
expect(maskMobile('')).toBe('');
expect(maskMobile('123')).toBe('**3');
});
it('masks Chinese real names', () => {
expect(maskChineseName('张三')).toBe('张*');
expect(maskChineseName('李四明')).toBe('李*明');
expect(maskChineseName('欧阳锋')).toBe('欧*锋');
expect(maskChineseName('张')).toBe('张');
expect(maskChineseName('')).toBe('');
});
});
describe('12306 price - parsePriceData', () => {
it('returns seat rows sorted by descending price and drops dup numeric codes', () => {
const data = {
train_no: '24000000G10L',
'OT': [],
'A9': '¥2158.0',
'9': '21580',
'P': '¥1163.0',
'M': '¥1035.0',
'O': '¥626.0',
'WZ': '¥626.0',
'INVALID': 'not-a-price',
};
const rows = parsePriceData(data);
const codes = rows.map((r) => r.seat_code);
expect(codes).not.toContain('9');
expect(codes).not.toContain('OT');
expect(codes).not.toContain('train_no');
expect(codes).not.toContain('INVALID');
expect(codes).toEqual(['A9', 'P', 'M', 'O', 'WZ']);
expect(rows[0]).toEqual({ seat_code: 'A9', seat_name: '商务座', price: '2158.0', currency: 'CNY' });
expect(rows[4]).toEqual({ seat_code: 'WZ', seat_name: '无座', price: '626.0', currency: 'CNY' });
});
it('keeps unknown letter codes with the letter as the name', () => {
const data = { 'A9': '¥100.0', 'ZZ': '¥50.0' };
const rows = parsePriceData(data);
const zz = rows.find((r) => r.seat_code === 'ZZ');
expect(zz?.seat_name).toBe('ZZ');
});
});
describe('12306 public API typed boundaries', () => {
const nonJsonFetch = async () => ({
ok: true,
json: async () => {
throw new SyntaxError('Unexpected token <');
},
});
it('wraps non-JSON train stop bodies as CommandExecutionError', async () => {
await expect(queryStops('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
});
it('wraps non-JSON price helper bodies as CommandExecutionError', async () => {
await expect(queryStopsForPrice('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(queryPrice('cookie=1', '24000000G10L', '01', '02', 'OM9', '2026-05-22', nonJsonFetch))
.rejects.toBeInstanceOf(CommandExecutionError);
});
});
describe('12306 browser evaluate boundaries', () => {
it('unwraps Browser Bridge {session,data} evaluate envelopes only at the boundary', () => {
expect(unwrapEvaluateResult({ session: 's1', data: 'JSESSIONID=1; tk=2' })).toBe('JSESSIONID=1; tk=2');
expect(unwrapEvaluateResult({ status: true, data: { value: 1 } })).toEqual({ status: true, data: { value: 1 } });
expect(requireEvaluateObject({ session: 's1', data: { status: true } }, 'test')).toEqual({ status: true });
expect(() => requireEvaluateObject({ session: 's1', data: null }, 'test')).toThrow(CommandExecutionError);
});
it('classifies 12306 login-like API envelopes as auth failures', () => {
expect(isAuthLikePayload({ status: false, messages: ['用户未登录'] })).toBe(true);
expect(isAuthLikePayload({ status: false, validateMessages: { global: ['请登录后再试'] } })).toBe(true);
expect(isAuthLikePayload({ status: false, messages: ['系统繁忙'] })).toBe(false);
});
it('masks passenger names in orders by default and supports explicit sensitive opt-in', async () => {
const command = getRegistry().get('12306/orders');
const makePage = () => ({
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return { session: 'browser', data: 'JSESSIONID=abc; tk=def' };
return {
session: 'browser',
data: {
status: true,
data: {
orderDBList: [{
sequence_no: 'E123',
order_date: '2026-05-18 10:00',
train_code_page: 'G1',
from_station_name_page: '北京南',
to_station_name_page: '上海虹桥',
start_train_date_page: '2026-05-22 07:00',
ticket_status_name: '未出行',
ticket_total_price_page: '626.0',
tickets: [{ passenger_name: '张三' }, { passenger_name: '李四明' }],
}],
},
},
};
},
});
await expect(command.func(makePage(), {})).resolves.toMatchObject([
{ order_id: 'E123', passengers: '张*, 李*明' },
]);
await expect(command.func(makePage(), { 'include-sensitive': true })).resolves.toMatchObject([
{ order_id: 'E123', passengers: '张三, 李四明' },
]);
});
it('maps login-like order payloads to AuthRequiredError instead of parser drift', async () => {
const command = getRegistry().get('12306/orders');
const page = {
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
return { status: false, messages: ['用户未登录'] };
},
};
await expect(command.func(page, {})).rejects.toBeInstanceOf(AuthRequiredError);
});
it('treats missing order list shape as parser drift but known empty arrays as empty result', async () => {
const command = getRegistry().get('12306/orders');
const makePage = (payload) => ({
goto: async () => {},
evaluate: async (script) => {
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
return payload;
},
});
await expect(command.func(makePage({ status: true, data: {} }), {}))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ status: true, data: { orderDBList: [] } }), {}))
.rejects.toBeInstanceOf(EmptyResultError);
});
});
+6 -3
View File
@@ -52,12 +52,15 @@ cli({
if (!data?.title) {
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
}
if (!data.body) {
throw new CliError('PARSE_ERROR', 'Article body not found', '36kr page loaded but no article body paragraphs were extracted');
}
return [
{ field: 'title', value: data.title },
{ field: 'author', value: data.author || '-' },
{ field: 'date', value: data.date || '-' },
{ field: 'author', value: data.author || '' },
{ field: 'date', value: data.date || '' },
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
{ field: 'body', value: data.body || '-' },
{ field: 'body', value: data.body || '' },
];
},
});
+46
View File
@@ -0,0 +1,46 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import './article.js';
function makePage(evaluateResult) {
return {
installInterceptor: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('36kr article', () => {
it('emits empty-string for missing optional author / date instead of a sentinel', async () => {
const command = getRegistry().get('36kr/article');
expect(command?.func).toBeDefined();
const page = makePage({ title: 'Real Title', author: '', date: '', body: 'Real article body' });
const rows = await command.func(page, { id: '1234567' });
const byField = Object.fromEntries(rows.map((r) => [r.field, r.value]));
expect(byField.title).toBe('Real Title');
expect(byField.author).toBe('');
expect(byField.date).toBe('');
expect(byField.body).toBe('Real article body');
expect(byField.url).toBe('https://36kr.com/p/1234567');
});
it('throws CliError NOT_FOUND when the page exposes no title', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({ title: '', author: 'x', date: 'y', body: 'z' });
await expect(command.func(page, { id: '1234567' })).rejects.toBeInstanceOf(CliError);
});
it('throws CliError PARSE_ERROR when the page exposes title but no body', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({ title: 'Real Title', author: 'x', date: 'y', body: '' });
await expect(command.func(page, { id: '1234567' })).rejects.toMatchObject({ code: 'PARSE_ERROR' });
});
it('throws CliError INVALID_ARGUMENT when no numeric id can be parsed', async () => {
const command = getRegistry().get('36kr/article');
const page = makePage({});
await expect(command.func(page, { id: 'not-a-url' })).rejects.toBeInstanceOf(CliError);
});
});
+70
View File
@@ -0,0 +1,70 @@
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
export function requireSearchQuery(value, label = 'keyword') {
const query = String(value ?? '').trim();
if (!query) {
throw new ArgumentError(`${label} cannot be empty`);
}
return query;
}
export function requireBoundedInteger(value, defaultValue, min, max, label) {
const raw = value ?? defaultValue;
const parsed = typeof raw === 'number' ? raw : Number(raw);
if (!Number.isInteger(parsed)) {
throw new ArgumentError(`${label} must be an integer between ${min} and ${max}, got ${JSON.stringify(value)}`);
}
if (parsed < min || parsed > max) {
throw new ArgumentError(`${label} must be between ${min} and ${max}, got ${parsed}`);
}
return parsed;
}
export function requireNonNegativeInteger(value, defaultValue, label) {
const raw = value ?? defaultValue;
const parsed = typeof raw === 'number' ? raw : Number(raw);
if (!Number.isInteger(parsed) || parsed < 0) {
throw new ArgumentError(`${label} must be a non-negative integer, got ${JSON.stringify(value)}`);
}
return parsed;
}
export function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && !Array.isArray(value) && 'session' in value && 'data' in value) {
return value.data;
}
return value;
}
export function requireRows(value, label) {
const rows = unwrapBrowserResult(value);
if (!Array.isArray(rows)) {
throw new CommandExecutionError(`${label} returned an unexpected payload shape; expected an array of result rows.`);
}
return rows;
}
export function toHttpsUrl(value, baseUrl) {
const raw = String(value ?? '').trim();
if (!raw) return '';
try {
const url = new URL(raw, baseUrl);
if (url.protocol !== 'http:' && url.protocol !== 'https:') return '';
return url.href;
} catch {
return '';
}
}
export function emptySearchResults(site, query) {
return new EmptyResultError(`${site} search`, `No ${site} results matched "${query}".`);
}
export async function runBrowserStep(label, fn) {
try {
return await fn();
} catch (error) {
if (error?.code || error?.name === 'ArgumentError') throw error;
throw new CommandExecutionError(`${label} failed: ${error?.message ?? error}`);
}
}
+20
View File
@@ -41,6 +41,26 @@ describe('apple-podcasts search command', () => {
}),
]);
});
it('emits empty-string for missing trackCount and primaryGenreName instead of a sentinel', async () => {
const cmd = getRegistry().get('apple-podcasts/search');
const fetchMock = vi.fn().mockResolvedValue({
ok: true,
json: () => Promise.resolve({
results: [
{
collectionId: 99,
collectionName: 'No-Meta Show',
artistName: 'Anon Host',
collectionViewUrl: 'https://example.com/p/99',
},
],
}),
});
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({ query: 'no-meta', limit: 1 });
expect(result[0].episodes).toBe('');
expect(result[0].genre).toBe('');
});
});
describe('apple-podcasts top command', () => {
beforeEach(() => {
+2 -2
View File
@@ -23,8 +23,8 @@ cli({
id: p.collectionId,
title: p.collectionName,
author: p.artistName,
episodes: p.trackCount ?? '-',
genre: p.primaryGenreName ?? '-',
episodes: p.trackCount ?? '',
genre: p.primaryGenreName ?? '',
url: p.collectionViewUrl || '',
}));
},
+144 -56
View File
@@ -4,6 +4,47 @@
* Auth: CSRF token from <meta name="csrf-token"> + session cookies.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DEFAULT_LIMIT = 10;
const MIN_LIMIT = 1;
const MAX_LIMIT = 100;
function normalizeSymbol(value) {
const symbol = String(value ?? '').trim().toUpperCase();
if (!symbol) throw new ArgumentError('symbol is required');
return symbol;
}
function normalizeExpiration(value) {
const expiration = String(value ?? '').trim();
if (!expiration) return '';
if (!/^\d{4}-\d{2}-\d{2}$/.test(expiration)) {
throw new ArgumentError('--expiration must use YYYY-MM-DD format');
}
const parsed = new Date(`${expiration}T00:00:00Z`);
if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== expiration) {
throw new ArgumentError('--expiration must be a valid calendar date');
}
return expiration;
}
function parseLimit(value) {
if (value === undefined || value === null || value === '') return DEFAULT_LIMIT;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < MIN_LIMIT || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && 'session' in value && 'data' in value) {
return value.data;
}
return value;
}
cli({
site: 'barchart',
name: 'greeks',
@@ -14,19 +55,19 @@ cli({
args: [
{ name: 'symbol', required: true, positional: true, help: 'Stock ticker (e.g. AAPL)' },
{ name: 'expiration', type: 'str', help: 'Expiration date (YYYY-MM-DD). Defaults to the nearest available expiration.' },
{ name: 'limit', type: 'int', default: 10, help: 'Number of near-the-money strikes per type' },
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: 'Number of near-the-money strikes per type (1-100)' },
],
columns: [
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
'volume', 'openInterest', 'expiration',
],
func: async (page, kwargs) => {
const symbol = kwargs.symbol.toUpperCase().trim();
const expiration = kwargs.expiration ?? '';
const limit = kwargs.limit ?? 10;
const symbol = normalizeSymbol(kwargs.symbol);
const expiration = normalizeExpiration(kwargs.expiration);
const limit = parseLimit(kwargs.limit);
await page.goto(`https://www.barchart.com/stocks/quotes/${encodeURIComponent(symbol)}/options`);
await page.wait(4);
const data = await page.evaluate(`
const data = unwrapBrowserResult(await page.evaluate(`
(async () => {
const sym = ${JSON.stringify(symbol)};
const expDate = ${JSON.stringify(expiration)};
@@ -45,39 +86,53 @@ cli({
+ '&fields=' + fields + '&raw=1';
if (expDate) url += '&expirationDate=' + encodeURIComponent(expDate);
const resp = await fetch(url, { credentials: 'include', headers });
if (resp.ok) {
const d = await resp.json();
let items = d?.data || [];
if (!resp.ok) {
return { ok: false, reason: 'http', status: resp.status, statusText: resp.statusText || '' };
}
if (!expDate) {
const expirations = items
.map(i => (i.raw || i).expirationDate || null)
.filter(Boolean)
.sort((a, b) => {
const aTime = Date.parse(a);
const bTime = Date.parse(b);
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
if (Number.isNaN(aTime)) return 1;
if (Number.isNaN(bTime)) return -1;
return aTime - bTime;
});
const nearestExpiration = expirations[0];
if (nearestExpiration) {
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
}
const d = await resp.json();
const allItems = d?.data;
if (!Array.isArray(allItems)) {
return { ok: false, reason: 'malformed' };
}
let items = allItems;
if (!expDate) {
const expirations = items
.map(i => (i.raw || i).expirationDate || null)
.filter(Boolean)
.sort((a, b) => {
const aTime = Date.parse(a);
const bTime = Date.parse(b);
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
if (Number.isNaN(aTime)) return 1;
if (Number.isNaN(bTime)) return -1;
return aTime - bTime;
});
const nearestExpiration = expirations[0];
if (nearestExpiration) {
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
}
}
// Separate calls and puts, sort by distance from current price
const calls = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const puts = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
// Separate calls and puts, sort by distance from current price.
const calls = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const puts = items
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
.slice(0, limit);
const selected = [...calls, ...puts];
return [...calls, ...puts].map(i => {
if (items.length > 0 && selected.length === 0) {
return { ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' };
}
return {
ok: true,
rows: selected.map(i => {
const r = i.raw || i;
return {
type: r.optionType,
@@ -93,28 +148,61 @@ cli({
openInterest: r.openInterest,
expiration: r.expirationDate,
};
});
}
} catch(e) {}
return [];
})
};
} catch(e) {
return { ok: false, reason: 'exception', message: e?.message || String(e) };
}
})()
`);
if (!data || !Array.isArray(data))
return [];
return data.map(r => ({
type: r.type || '',
strike: r.strike,
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
volume: r.volume,
openInterest: r.openInterest,
expiration: r.expiration ?? null,
}));
`));
if (!data || data.ok !== true) {
if (data?.reason === 'http') {
throw new CommandExecutionError(`Barchart greeks request failed: HTTP ${data.status}${data.statusText ? ` ${data.statusText}` : ''}`);
}
if (data?.reason === 'malformed') {
throw new CommandExecutionError(`Barchart greeks returned an unreadable options payload${data.message ? `: ${data.message}` : ''}`);
}
if (data?.reason === 'exception') {
throw new CommandExecutionError(`Barchart greeks request failed: ${data.message || 'unknown error'}`);
}
throw new CommandExecutionError(`Failed to fetch Barchart greeks for ${symbol}`);
}
if (!Array.isArray(data.rows)) {
throw new CommandExecutionError('Barchart greeks returned an unreadable options payload');
}
if (data.rows.length === 0) {
throw new EmptyResultError('barchart greeks', `No option greeks were returned for ${symbol}. Confirm the symbol, expiration, and Barchart login state.`);
}
return data.rows.map(r => {
if (!r || typeof r !== 'object' || Array.isArray(r)) {
throw new CommandExecutionError('Barchart greeks returned a malformed option row');
}
const type = String(r.type || '').trim();
const expirationValue = String(r.expiration || '').trim();
if (!/^(call|put)$/i.test(type) || r.strike === undefined || r.strike === null || r.strike === '' || !expirationValue) {
throw new CommandExecutionError('Barchart greeks returned a malformed option row identity');
}
return {
type,
strike: r.strike,
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
volume: r.volume,
openInterest: r.openInterest,
expiration: expirationValue,
};
});
},
});
export const __test__ = {
normalizeSymbol,
normalizeExpiration,
parseLimit,
unwrapBrowserResult,
};
+138
View File
@@ -0,0 +1,138 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './greeks.js';
const { normalizeExpiration, normalizeSymbol, parseLimit, unwrapBrowserResult } = await import('./greeks.js').then((m) => m.__test__);
function makePage(evaluateResult) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('barchart greeks command', () => {
const command = getRegistry().get('barchart/greeks');
it('registers with the expected shape', () => {
expect(command).toBeDefined();
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.columns).toEqual([
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
'volume', 'openInterest', 'expiration',
]);
});
it('maps returned option rows without changing the declared output shape', async () => {
const page = makePage({
session: 'site:barchart',
data: {
ok: true,
rows: [
{
type: 'Call',
strike: 190,
last: 3.456,
iv: 21.234,
delta: 0.56789,
gamma: 0.01234,
theta: -0.12345,
vega: 0.23456,
rho: 0.03456,
volume: 123,
openInterest: 456,
expiration: '2026-06-19',
},
],
},
});
const rows = await command.func(page, { symbol: 'aapl', limit: 1 });
expect(page.goto).toHaveBeenCalledWith('https://www.barchart.com/stocks/quotes/AAPL/options');
expect(page.wait).toHaveBeenCalledWith(4);
expect(rows).toEqual([
{
type: 'Call',
strike: 190,
last: 3.46,
iv: '21.23%',
delta: 0.5679,
gamma: 0.0123,
theta: -0.1235,
vega: 0.2346,
rho: 0.0346,
volume: 123,
openInterest: 456,
expiration: '2026-06-19',
},
]);
});
it('validates args before browser navigation and unwraps bridge envelopes', async () => {
expect(normalizeSymbol(' aapl ')).toBe('AAPL');
expect(normalizeExpiration('2026-06-19')).toBe('2026-06-19');
expect(parseLimit(undefined)).toBe(10);
expect(parseLimit(100)).toBe(100);
expect(unwrapBrowserResult({ session: 'site:barchart', data: { ok: true } })).toEqual({ ok: true });
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: '', limit: 1 }))
.rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', expiration: '2026-02-30', limit: 1 }))
.rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', limit: 101 }))
.rejects.toBeInstanceOf(ArgumentError);
});
it('embeds expiration and limit in the browser-side request script', async () => {
const page = makePage({
ok: true,
rows: [{
type: 'Put',
strike: 185,
last: null,
iv: null,
delta: null,
gamma: null,
theta: null,
vega: null,
rho: null,
volume: 0,
openInterest: 0,
expiration: '2026-07-17',
}],
});
await command.func(page, { symbol: 'MSFT', expiration: '2026-07-17', limit: 7 });
const script = page.evaluate.mock.calls[0][0];
expect(script).toContain('const expDate = "2026-07-17"');
expect(script).toContain('const limit = 7');
expect(script).toContain("url += '&expirationDate=' + encodeURIComponent(expDate)");
});
it('throws CommandExecutionError for HTTP, malformed, exception, and missing payload states', async () => {
await expect(command.func(makePage({ ok: false, reason: 'http', status: 403, statusText: 'Forbidden' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'malformed' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'exception', message: 'network down' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' }), { symbol: 'AAPL' }))
.rejects.toThrow('call or put identities');
await expect(command.func(makePage(null), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: true, rows: 'bad' }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(command.func(makePage({ ok: true, rows: [{ type: 'Call', strike: null, expiration: '' }] }), { symbol: 'AAPL' }))
.rejects.toThrow('malformed option row identity');
});
it('throws EmptyResultError when Barchart returns no greeks rows', async () => {
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL' }))
.rejects.toBeInstanceOf(EmptyResultError);
});
});
+167
View File
@@ -0,0 +1,167 @@
/**
* Bilibili summary — fetches the official AI-generated video summary (the "AI总结"
* shown on the video page) via /x/web-interface/view/conclusion/get.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { apiGet, resolveBvid } from './utils.js';
const BILIBILI_HOST_RE = /(^|\.)bilibili\.com$/i;
const B23_HOST_RE = /(^|\.)b23\.tv$/i;
const BVID_RE = /^BV[A-Za-z0-9]+$/;
function formatTime(seconds) {
const s = Math.max(0, Math.floor(Number(seconds) || 0));
const h = Math.floor(s / 3600);
const m = Math.floor((s % 3600) / 60);
const sec = s % 60;
const pad = (n) => String(n).padStart(2, '0');
return h > 0 ? `${h}:${pad(m)}:${pad(sec)}` : `${pad(m)}:${pad(sec)}`;
}
async function readBvid(raw) {
const input = String(raw ?? '').trim();
if (!input) {
throw new ArgumentError('bilibili summary bvid cannot be empty', 'Pass a BV ID, Bilibili video URL, or b23.tv short link.');
}
if (BVID_RE.test(input)) {
return input;
}
let parsed = null;
try {
parsed = new URL(input);
} catch {
// Bare b23.tv short codes are accepted by the shared resolver.
}
if (parsed) {
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') {
throw new ArgumentError('Bilibili summary URL must use http or https');
}
if (BILIBILI_HOST_RE.test(parsed.hostname)) {
const match = parsed.pathname.match(/\/(?:video|bangumi\/play)\/(BV[A-Za-z0-9]+)/i);
if (!match) {
throw new ArgumentError('Bilibili summary URL must contain a BV video id');
}
return match[1];
}
if (!B23_HOST_RE.test(parsed.hostname)) {
throw new ArgumentError('Bilibili summary URL must be a bilibili.com or b23.tv URL');
}
}
try {
return await resolveBvid(input);
} catch (error) {
throw new ArgumentError(`Cannot resolve Bilibili BV ID from input: ${input}`, error instanceof Error ? error.message : String(error));
}
}
function requireOkPayload(payload, label) {
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
throw new CommandExecutionError(`Bilibili ${label} API returned a malformed payload`);
}
if (payload.code !== 0) {
const message = payload.message ?? 'unknown error';
if (payload.code === -101 || payload.code === -403 || /登录|权限|forbidden|permission|login/i.test(String(message))) {
throw new AuthRequiredError('bilibili.com', `Bilibili ${label} API requires login or permission: ${message} (${payload.code})`);
}
throw new CommandExecutionError(`Bilibili ${label} API failed: ${message} (${payload.code})`);
}
return payload.data;
}
function readModelResult(data, bvid) {
if (!data || typeof data !== 'object' || Array.isArray(data)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed data');
}
if (data.code !== 0) {
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
}
let modelResult = data.model_result;
if (typeof modelResult === 'string') {
try {
modelResult = JSON.parse(modelResult);
} catch {
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result JSON');
}
}
if (!modelResult || typeof modelResult !== 'object' || Array.isArray(modelResult)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result');
}
const summary = String(modelResult.summary ?? '').trim();
if (!summary) {
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
}
const outline = modelResult.outline ?? [];
if (!Array.isArray(outline)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline');
}
return { summary, outline };
}
function rowsFromModel(model) {
const rows = [{ time: '', content: model.summary }];
for (const section of model.outline) {
if (!section || typeof section !== 'object' || Array.isArray(section)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline section');
}
const sectionTitle = String(section.title ?? '').trim();
const sectionTime = formatTime(section.timestamp);
if (sectionTitle) {
rows.push({ time: sectionTime, content: `# ${sectionTitle}` });
}
const points = section.part_outline ?? [];
if (!Array.isArray(points)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed part outline');
}
for (const point of points) {
if (!point || typeof point !== 'object' || Array.isArray(point)) {
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline point');
}
const content = String(point.content ?? '').trim();
if (content) {
rows.push({ time: formatTime(point.timestamp), content });
}
}
}
return rows;
}
var command = cli({
site: 'bilibili',
name: 'summary',
access: 'read',
description: '获取 B站视频的官方 AI 总结(视频页「AI总结」同款,含分段大纲与时间戳)',
domain: 'www.bilibili.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'bvid', required: true, positional: true, help: 'Video BV ID / URL / b23.tv short link' },
],
columns: ['time', 'content'],
func: async (page, kwargs) => {
if (!page) {
throw new CommandExecutionError('Browser session required for bilibili summary');
}
const bvid = await readBvid(kwargs.bvid);
const view = await apiGet(page, '/x/web-interface/view', { params: { bvid } });
const viewData = requireOkPayload(view, 'view');
const cid = viewData?.cid;
const upMid = viewData?.owner?.mid;
if (!cid || !upMid) {
throw new CommandExecutionError(`Bilibili view API did not return cid/up_mid for ${bvid}`);
}
const conclusion = await apiGet(page, '/x/web-interface/view/conclusion/get', {
params: { bvid, cid, up_mid: upMid },
signed: true,
});
const conclusionData = requireOkPayload(conclusion, 'conclusion');
return rowsFromModel(readModelResult(conclusionData, bvid));
},
});
export const __test__ = {
command,
formatTime,
readBvid,
readModelResult,
rowsFromModel,
};
+210
View File
@@ -0,0 +1,210 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const { mockApiGet, mockResolveBvid } = vi.hoisted(() => ({
mockApiGet: vi.fn(),
mockResolveBvid: vi.fn(),
}));
vi.mock('./utils.js', async (importOriginal) => ({
...(await importOriginal()),
apiGet: mockApiGet,
resolveBvid: mockResolveBvid,
}));
import { getRegistry } from '@jackwener/opencli/registry';
import './summary.js';
describe('bilibili summary', () => {
const command = getRegistry().get('bilibili/summary');
const page = {};
beforeEach(() => {
mockApiGet.mockReset();
mockResolveBvid.mockReset();
mockResolveBvid.mockRejectedValue(new Error('short link not found'));
});
function mockView(data = { aid: 114, cid: 222, owner: { mid: 333 } }) {
mockApiGet.mockResolvedValueOnce({ code: 0, data });
}
function mockConclusion(modelResult) {
mockApiGet.mockResolvedValueOnce({
code: 0,
data: {
code: 0,
model_result: modelResult,
},
});
}
it('returns the summary plus timestamped outline rows', async () => {
mockView();
mockConclusion({
summary: '整体总结',
outline: [
{
title: '第一节',
timestamp: 0,
part_outline: [
{ timestamp: 12, content: '要点A' },
{ timestamp: 3725, content: '要点B' },
],
},
],
});
const result = await command.func(page, { bvid: 'BV1xxx' });
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1xxx' } });
expect(mockApiGet).toHaveBeenNthCalledWith(2, page, '/x/web-interface/view/conclusion/get', {
params: { bvid: 'BV1xxx', cid: 222, up_mid: 333 },
signed: true,
});
expect(result).toEqual([
{ time: '', content: '整体总结' },
{ time: '00:00', content: '# 第一节' },
{ time: '00:12', content: '要点A' },
{ time: '1:02:05', content: '要点B' },
]);
});
it('returns just the summary when the video has no outline', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '只有总结', outline: [] });
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
{ time: '', content: '只有总结' },
]);
});
it('parses model_result when Bilibili returns it as a JSON string', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion(JSON.stringify({ summary: '字符串总结', outline: [] }));
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
{ time: '', content: '字符串总结' },
]);
});
it('normalizes Bilibili video URLs before calling the APIs', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: 'URL 总结', outline: [] });
await command.func(page, {
bvid: 'https://www.bilibili.com/video/BV1abc12345/?spm_id_from=333.1007',
});
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1abc12345' } });
});
it('resolves b23.tv short links through the shared resolver', async () => {
mockResolveBvid.mockResolvedValueOnce('BVshort12345');
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '短链总结', outline: [] });
await command.func(page, { bvid: 'https://b23.tv/abc' });
expect(mockResolveBvid).toHaveBeenCalledWith('https://b23.tv/abc');
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BVshort12345' } });
});
it('rejects invalid inputs before calling Bilibili APIs', async () => {
const cases = [
'',
'javascript:alert(1)',
'https://example.com/video/BV1abc12345',
'https://share.note.youdao.com/video/BV1abc12345',
'https://www.bilibili.com/read/cv12345',
];
for (const bvid of cases) {
await expect(command.func(page, { bvid })).rejects.toBeInstanceOf(ArgumentError);
}
expect(mockApiGet).not.toHaveBeenCalled();
});
it('maps unresolved short-code inputs to ArgumentError without calling APIs', async () => {
await expect(command.func(page, { bvid: 'not-a-bv' })).rejects.toBeInstanceOf(ArgumentError);
expect(mockResolveBvid).toHaveBeenCalledWith('not-a-bv');
expect(mockApiGet).not.toHaveBeenCalled();
});
it('throws EmptyResultError when Bilibili has not generated an AI summary for the video', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: 0, data: { code: 1, model_result: {} } });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when the view payload is malformed', async () => {
mockApiGet.mockResolvedValueOnce({ code: 0, data: {} });
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /cid\/up_mid/.test(err.message),
);
});
it('throws CommandExecutionError when the view API returns a non-auth error', async () => {
mockApiGet.mockResolvedValueOnce({ code: -404, message: '啥都木有' });
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /啥都木有.*-404/.test(err.message),
);
});
it('maps conclusion auth or permission errors to AuthRequiredError', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: -403, message: '访问权限不足' });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps conclusion non-auth API errors to CommandExecutionError', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce({ code: -500, message: 'server error' });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /server error.*-500/.test(err.message),
);
});
it('throws CommandExecutionError for malformed conclusion API payloads', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockApiGet.mockResolvedValueOnce(null);
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError for malformed model_result JSON', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion('{bad json');
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /model_result JSON/.test(err.message),
);
});
it('throws CommandExecutionError for malformed outline shapes', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({ summary: '坏 outline', outline: {} });
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /outline/.test(err.message),
);
});
it('throws CommandExecutionError for malformed part outline shapes', async () => {
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
mockConclusion({
summary: '坏 part_outline',
outline: [{ title: '段落', timestamp: 0, part_outline: {} }],
});
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
(err) => err instanceof CommandExecutionError && /part outline/.test(err.message),
);
});
});
+356
View File
@@ -0,0 +1,356 @@
import { describe, expect, it } from 'vitest';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
import { __test__ } from './search.js';
const {
normalizePositiveInt,
normalizeNonNegativeInt,
normalizeDate,
normalizeCurrency,
normalizeLang,
hasPositiveResultCount,
buildSearchUrl,
} = __test__;
describe('booking helpers — normalizePositiveInt (no silent clamp)', () => {
it('returns default when value is undefined/null/empty', () => {
expect(normalizePositiveInt(undefined, 2, 'adults', 30)).toBe(2);
expect(normalizePositiveInt(null, 2, 'adults', 30)).toBe(2);
});
it('accepts integers in range', () => {
expect(normalizePositiveInt(1, 2, 'adults', 30)).toBe(1);
expect(normalizePositiveInt(30, 2, 'adults', 30)).toBe(30);
});
it('rejects zero / negative / out-of-range / non-integer (no silent clamp)', () => {
expect(() => normalizePositiveInt(0, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(-1, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(31, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt(1.5, 2, 'adults', 30)).toThrow(ArgumentError);
expect(() => normalizePositiveInt('abc', 2, 'adults', 30)).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeNonNegativeInt', () => {
it('accepts zero', () => {
expect(normalizeNonNegativeInt(0, 0, 'children', 10)).toBe(0);
});
it('rejects negative / out-of-range (no silent clamp)', () => {
expect(() => normalizeNonNegativeInt(-1, 0, 'children', 10)).toThrow(ArgumentError);
expect(() => normalizeNonNegativeInt(11, 0, 'children', 10)).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeDate', () => {
it('accepts YYYY-MM-DD', () => {
expect(normalizeDate('2026-06-15', 'checkin')).toBe('2026-06-15');
});
it('rejects bad format / nonsense dates with ArgumentError', () => {
expect(() => normalizeDate('', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('06/15/2026', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('2026-13-40', 'checkin')).toThrow(ArgumentError);
expect(() => normalizeDate('2026-02-31', 'checkin')).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeCurrency', () => {
it('passes 3-letter codes uppercased', () => {
expect(normalizeCurrency('usd')).toBe('USD');
expect(normalizeCurrency('JPY')).toBe('JPY');
});
it('returns empty for unset', () => {
expect(normalizeCurrency(undefined)).toBe('');
expect(normalizeCurrency('')).toBe('');
});
it('rejects non-3-letter codes', () => {
expect(() => normalizeCurrency('US')).toThrow(ArgumentError);
expect(() => normalizeCurrency('US$')).toThrow(ArgumentError);
expect(() => normalizeCurrency('USDX')).toThrow(ArgumentError);
});
});
describe('booking helpers — normalizeLang whitelist', () => {
it('lowercases supported langs', () => {
expect(normalizeLang('EN-US')).toBe('en-us');
expect(normalizeLang('zh-cn')).toBe('zh-cn');
});
it('rejects unknown langs', () => {
expect(() => normalizeLang('xx-yy')).toThrow(ArgumentError);
expect(() => normalizeLang('en')).toThrow(ArgumentError);
});
});
describe('booking helpers — buildSearchUrl', () => {
it('constructs canonical search URL with required params', () => {
const url = buildSearchUrl({
destination: 'Tokyo',
checkin: '2026-06-15',
checkout: '2026-06-17',
adults: 2,
rooms: 1,
children: 0,
offset: 0,
currency: 'USD',
lang: 'en-us',
});
expect(url).toContain('https://www.booking.com/searchresults.en-us.html');
expect(url).toContain('ss=Tokyo');
expect(url).toContain('checkin=2026-06-15');
expect(url).toContain('checkout=2026-06-17');
expect(url).toContain('group_adults=2');
expect(url).toContain('no_rooms=1');
expect(url).toContain('group_children=0');
expect(url).toContain('selected_currency=USD');
expect(url).not.toContain('offset=');
});
it('omits lang file segment when lang is empty', () => {
const url = buildSearchUrl({
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
adults: 2, rooms: 1, children: 0, offset: 0, currency: '', lang: '',
});
expect(url).toMatch(/booking\.com\/searchresults\.html\?/);
});
it('emits offset only when > 0', () => {
const url = buildSearchUrl({
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
adults: 2, rooms: 1, children: 0, offset: 25, currency: '', lang: '',
});
expect(url).toContain('offset=25');
});
});
describe('booking helpers — hasPositiveResultCount', () => {
it('detects positive Booking result-count evidence', () => {
expect(hasPositiveResultCount('Tokyo: 1,234 properties found')).toBe(true);
expect(hasPositiveResultCount('1 stay found')).toBe(true);
});
it('does not treat no-results text as positive evidence', () => {
expect(hasPositiveResultCount('No properties found')).toBe(false);
expect(hasPositiveResultCount('0 properties found')).toBe(false);
});
});
describe('booking adapter registry shape', () => {
it('search is registered as read with id-shaped column for round-trip', () => {
const search = getRegistry().get('booking/search');
expect(search).toBeDefined();
expect(search.access).toBe('read');
expect(search.browser).toBe(true);
// slug + country together form the round-trip identity (URL: /hotel/<country>/<slug>.html)
expect(search.columns).toContain('slug');
expect(search.columns).toContain('country');
expect(search.columns).toContain('url');
});
it('search columns stay <= 12 to honor agent-native row shape', () => {
const search = getRegistry().get('booking/search');
expect(search.columns.length).toBeLessThanOrEqual(12);
});
});
describe('booking search — typed errors (no silent fallback)', () => {
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
it('rejects empty destination with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: ' ', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(ArgumentError);
});
it('rejects missing checkin/checkout with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo' })).rejects.toThrow(ArgumentError);
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15' })).rejects.toThrow(ArgumentError);
});
it('rejects checkout <= checkin with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-17', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
});
it('rejects out-of-range --limit with ArgumentError (no silent clamp to 100)', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', limit: 999 })).rejects.toThrow(ArgumentError);
});
it('rejects negative --offset with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: -1 })).rejects.toThrow(ArgumentError);
});
it('rejects unsupported --lang with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', lang: 'xx-yy' })).rejects.toThrow(ArgumentError);
});
it('rejects malformed --currency with ArgumentError', async () => {
const search = getRegistry().get('booking/search');
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'US$' })).rejects.toThrow(ArgumentError);
});
it('wraps browser navigation failures as CommandExecutionError', async () => {
const search = getRegistry().get('booking/search');
const downPage = { goto: () => Promise.reject(new Error('browser down')) };
await expect(search.func(downPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws EmptyResultError when extractor returns no cards', async () => {
const search = getRegistry().get('booking/search');
const emptyPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'No properties found' }),
};
await expect(search.func(emptyPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(EmptyResultError);
});
it('throws CommandExecutionError when result-count evidence exists but no cards were parsed', async () => {
const search = getRegistry().get('booking/search');
const driftPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'Tokyo: 1,234 properties found' }),
};
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when captcha is detected', async () => {
const search = getRegistry().get('booking/search');
const blockedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, items: [], blocked: true, totalText: 'Verify you are human' }),
};
await expect(search.func(blockedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when extractor payload is malformed instead of treating it as empty', async () => {
const search = getRegistry().get('booking/search');
const malformedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({ ok: true, blocked: false, totalText: 'Tokyo hotels' }),
};
await expect(search.func(malformedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError when rendered cards lack stable hotel URL identity', async () => {
const search = getRegistry().get('booking/search');
const driftPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: 'Tokyo hotels',
items: [{
name: 'Unlinked Hotel',
country: '',
slug: '',
url: '',
distance: '',
review_score: null,
review_count: null,
star_rating: null,
price_currency: '',
price_amount: null,
recommended_room: '',
}],
}),
};
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
});
it('unwraps {session, data} envelope from CDP bridge before validating', async () => {
const search = getRegistry().get('booking/search');
const envelopePage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
session: 1,
data: {
ok: true,
blocked: false,
totalText: '',
items: [{
name: 'Test Hotel',
country: 'jp',
slug: 'test-hotel',
url: 'https://www.booking.com/hotel/jp/test-hotel.html',
distance: '1 km from centre',
review_score: 8.6,
review_count: 100,
star_rating: 4,
price_currency: 'USD',
price_amount: 120,
recommended_room: 'Standard double',
}],
},
}),
};
const rows = await search.func(envelopePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' });
expect(rows).toHaveLength(1);
expect(rows[0].rank).toBe(1);
expect(rows[0].slug).toBe('test-hotel');
expect(rows[0].url).toBe('https://www.booking.com/hotel/jp/test-hotel.html');
});
it('uses requested selected_currency as the output source when price is present', async () => {
const search = getRegistry().get('booking/search');
const currencyPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: '',
items: [{
name: 'Currency Hotel',
country: 'cn',
slug: 'currency-hotel',
url: 'https://www.booking.com/hotel/cn/currency-hotel.html',
distance: '',
review_score: null,
review_count: null,
star_rating: null,
price_currency: 'JPY',
price_amount: 880,
recommended_room: '',
}],
}),
};
const rows = await search.func(currencyPage, { destination: 'Shanghai', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'CNY' });
expect(rows[0].price_currency).toBe('CNY');
});
it('respects offset for rank numbering when paginating', async () => {
const search = getRegistry().get('booking/search');
const pagedPage = {
goto: async () => {},
wait: async () => {},
evaluate: async () => ({
ok: true,
blocked: false,
totalText: '',
items: [
{ name: 'A', country: 'jp', slug: 'a', url: 'https://www.booking.com/hotel/jp/a.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
{ name: 'B', country: 'jp', slug: 'b', url: 'https://www.booking.com/hotel/jp/b.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
],
}),
};
const rows = await search.func(pagedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: 50 });
expect(rows[0].rank).toBe(51);
expect(rows[1].rank).toBe(52);
});
});
+351
View File
@@ -0,0 +1,351 @@
import {
ArgumentError,
CommandExecutionError,
EmptyResultError,
} from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
function normalizePositiveInt(value, defaultValue, label, max) {
const raw = value ?? defaultValue;
const n = Number(raw);
if (!Number.isInteger(n) || n <= 0) {
throw new ArgumentError(`${label} must be a positive integer`);
}
if (typeof max === 'number' && n > max) {
throw new ArgumentError(`${label} must be <= ${max}`);
}
return n;
}
function normalizeNonNegativeInt(value, defaultValue, label, max) {
const raw = value ?? defaultValue;
const n = Number(raw);
if (!Number.isInteger(n) || n < 0) {
throw new ArgumentError(`${label} must be a non-negative integer`);
}
if (typeof max === 'number' && n > max) {
throw new ArgumentError(`${label} must be <= ${max}`);
}
return n;
}
function normalizeDate(value, label) {
const v = String(value || '').trim();
if (!v) {
throw new ArgumentError(`${label} is required (YYYY-MM-DD)`);
}
if (!DATE_RE.test(v)) {
throw new ArgumentError(`${label} must be YYYY-MM-DD, got ${JSON.stringify(value)}`);
}
const [year, month, day] = v.split('-').map(Number);
const d = new Date(Date.UTC(year, month - 1, day));
if (
Number.isNaN(d.getTime()) ||
d.getUTCFullYear() !== year ||
d.getUTCMonth() !== month - 1 ||
d.getUTCDate() !== day
) {
throw new ArgumentError(`${label} is not a valid calendar date: ${v}`);
}
return v;
}
function normalizeCurrency(value) {
if (value == null || value === '') return '';
const v = String(value).trim().toUpperCase();
if (!/^[A-Z]{3}$/.test(v)) {
throw new ArgumentError(`currency must be a 3-letter ISO code (e.g. USD, JPY, CNY), got ${JSON.stringify(value)}`);
}
return v;
}
const ALLOWED_LANGS = new Set([
'en-us', 'en-gb', 'zh-cn', 'zh-tw', 'ja', 'ko', 'de', 'fr', 'es', 'it',
'pt-br', 'pt-pt', 'ru', 'th', 'vi', 'tr', 'pl', 'nl', 'ar',
]);
function normalizeLang(value) {
if (value == null || value === '') return '';
const v = String(value).trim().toLowerCase();
if (!ALLOWED_LANGS.has(v)) {
throw new ArgumentError(`lang must be one of: ${[...ALLOWED_LANGS].join(', ')}`);
}
return v;
}
function hasPositiveResultCount(text) {
const value = String(text || '').replace(/\u00a0/g, ' ');
const resultCount = value.match(/\b([1-9][0-9,.\s]*)\s+(?:properties|property|stays|stay|hotels|hotel)\b/i);
if (!resultCount) return false;
const digits = resultCount[1].replace(/\D/g, '');
return Boolean(digits) && Number(digits) > 0;
}
function buildSearchUrl({
destination,
checkin,
checkout,
adults,
rooms,
children,
offset,
currency,
lang,
}) {
const file = lang ? `searchresults.${lang}.html` : 'searchresults.html';
const params = new URLSearchParams();
params.set('ss', destination);
params.set('checkin', checkin);
params.set('checkout', checkout);
params.set('group_adults', String(adults));
params.set('no_rooms', String(rooms));
params.set('group_children', String(children));
if (offset > 0) params.set('offset', String(offset));
if (currency) params.set('selected_currency', currency);
return `https://www.booking.com/${file}?${params.toString()}`;
}
const EXTRACTOR = `
(() => {
const trim = (v) => (v == null ? '' : String(v).replace(/\\s+/g, ' ').trim());
const cards = Array.from(document.querySelectorAll('[data-testid=property-card]'));
// Detect blocking / captcha pages: no cards but body shows a verification prompt.
if (cards.length === 0) {
const text = [
(document.title || ''),
(document.body && document.body.innerText) || '',
(location && location.pathname) || '',
].join(' ');
const blocked = /captcha|challenge|verify\\s*you\\s*are|access\\s*denied|forbidden|robot|unusual\\s*traffic/i.test(text);
const totalEl = document.querySelector('h1');
const totalText = trim(totalEl && totalEl.textContent);
return { ok: true, items: [], blocked, totalText };
}
const items = cards.map((card) => {
const titleEl = card.querySelector('[data-testid=title]');
const link = card.querySelector('a[data-testid=title-link]');
const href = (link && link.href) || '';
let country = '';
let slug = '';
let canonicalUrl = '';
try {
const u = new URL(href, 'https://www.booking.com');
const m = u.pathname.match(/^\\/hotel\\/([a-z]{2})\\/([^./]+)/);
if (m) {
country = m[1];
slug = m[2];
canonicalUrl = 'https://www.booking.com/hotel/' + country + '/' + slug + '.html';
}
} catch (_) {}
const reviewTextRaw = trim(card.querySelector('[data-testid=review-score]')?.textContent);
// Booking renders the score twice (a11y + visual), text reads like "Scored 8.6 8.6 Very Good 6,151 reviews"
// or "评分8.68.6很棒 6,151条住客点评". Take only the first numeric occurrence.
const scoreMatch = reviewTextRaw.match(/(\\d{1,2})\\.(\\d)/);
const reviewScore = scoreMatch ? Number(scoreMatch[1] + '.' + scoreMatch[2]) : null;
const countMatch = reviewTextRaw.match(/([0-9][0-9,]*)\\s*(?:reviews|reseñas|avis|recensioni|条住客点评|条评论|レビュー|리뷰)/i);
const reviewCount = countMatch ? Number(countMatch[1].replace(/,/g, '')) : null;
// Star rating: aria-label often "5 out of 5" / "4 星 (满分 5 星)" / "Hôtel 4 étoiles"
let starRating = null;
const starEl = card.querySelector('[data-testid=rating-stars], [data-testid=quality-rating]');
if (starEl) {
const aria = starEl.getAttribute('aria-label') || starEl.textContent || '';
const m = aria.match(/(\\d)(?:\\s*(?:out of|\\/|星|颗星|stars?|étoiles?)|\\s*$)/i);
if (m) starRating = Number(m[1]);
if (starRating == null) {
const count = starEl.querySelectorAll('svg, [aria-hidden=true]').length;
if (count >= 1 && count <= 5) starRating = count;
}
}
const priceEl = card.querySelector('[data-testid=price-and-discounted-price]');
const priceText = trim(priceEl && priceEl.textContent);
// currency symbol → ISO best-effort
const currencySymbolMap = {
'$': 'USD', 'US$': 'USD', 'A$': 'AUD', 'C$': 'CAD', 'HK$': 'HKD',
'€': 'EUR', '£': 'GBP', '¥': 'JPY', '¥': 'CNY', '₹': 'INR', '₩': 'KRW',
'CN¥': 'CNY', 'CN¥': 'CNY', 'NT$': 'TWD', 'S$': 'SGD',
};
let priceCurrency = '';
let priceAmount = null;
const sym = priceText.match(/(US\\$|A\\$|C\\$|HK\\$|NT\\$|S\\$|CN¥|CN¥|[$€£¥¥₹₩])/);
if (sym) priceCurrency = currencySymbolMap[sym[1]] || '';
const num = priceText.replace(/,/g, '').match(/(\\d+(?:\\.\\d+)?)/);
if (num) priceAmount = Number(num[1]);
return {
name: trim(titleEl?.textContent),
country,
slug,
url: canonicalUrl,
distance: trim(card.querySelector('[data-testid=distance]')?.textContent),
review_score: reviewScore,
review_count: reviewCount,
star_rating: starRating,
price_currency: priceCurrency,
price_amount: priceAmount,
recommended_room: trim(card.querySelector('[data-testid=recommended-units]')?.textContent),
};
});
const totalEl = document.querySelector('h1');
const totalText = trim(totalEl && totalEl.textContent);
return { ok: true, items, blocked: false, totalText };
})()
`;
cli({
site: 'booking',
name: 'search',
description: 'Search Booking.com hotels by destination and dates (server-rendered card scrape).',
access: 'read',
example: 'opencli booking search Tokyo --checkin 2026-06-15 --checkout 2026-06-17 -f yaml',
domain: 'www.booking.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'destination', required: true, positional: true, help: 'Destination keyword (city, district, or hotel name)' },
{ name: 'checkin', required: true, help: 'Check-in date YYYY-MM-DD' },
{ name: 'checkout', required: true, help: 'Check-out date YYYY-MM-DD' },
{ name: 'adults', type: 'int', default: 2, help: 'Number of adults (1-30)' },
{ name: 'rooms', type: 'int', default: 1, help: 'Number of rooms (1-30)' },
{ name: 'children', type: 'int', default: 0, help: 'Number of children (0-10)' },
{ name: 'currency', required: false, help: 'Force result currency (e.g. USD, JPY, CNY)' },
{ name: 'lang', required: false, help: 'Force result language (e.g. en-us, zh-cn, ja)' },
{ name: 'limit', type: 'int', default: 25, help: 'Max rows to return (1-100; Booking pages 25 per request)' },
{ name: 'offset', type: 'int', default: 0, help: 'Result offset for pagination (multiple of 25)' },
],
columns: [
'rank',
'name',
'country',
'slug',
'star_rating',
'review_score',
'review_count',
'price_amount',
'price_currency',
'distance',
'recommended_room',
'url',
],
func: async (page, kwargs) => {
const destination = String(kwargs.destination || '').trim();
if (!destination) throw new ArgumentError('destination is required');
const checkin = normalizeDate(kwargs.checkin, 'checkin');
const checkout = normalizeDate(kwargs.checkout, 'checkout');
if (checkin >= checkout) {
throw new ArgumentError(`checkout (${checkout}) must be after checkin (${checkin})`);
}
const adults = normalizePositiveInt(kwargs.adults, 2, 'adults', 30);
const rooms = normalizePositiveInt(kwargs.rooms, 1, 'rooms', 30);
const children = normalizeNonNegativeInt(kwargs.children, 0, 'children', 10);
const currency = normalizeCurrency(kwargs.currency);
const lang = normalizeLang(kwargs.lang);
const limit = normalizePositiveInt(kwargs.limit, 25, 'limit', 100);
const offset = normalizeNonNegativeInt(kwargs.offset, 0, 'offset', 1000);
const url = buildSearchUrl({ destination, checkin, checkout, adults, rooms, children, offset, currency, lang });
try {
await page.goto(url);
} catch (err) {
throw new CommandExecutionError(`Failed to load Booking.com search page: ${err?.message || err}`);
}
// Booking lazy-loads price cells; wait for at least the first card price to settle.
try {
await page.wait('selector', '[data-testid=property-card]', { timeoutMs: 20000 });
} catch (_) {
// selector wait is best-effort — extractor handles empty case explicitly
}
let raw;
try {
raw = await page.evaluate(EXTRACTOR);
} catch (err) {
throw new CommandExecutionError(`Failed to extract Booking.com cards: ${err?.message || err}`);
}
if (raw && typeof raw === 'object' && raw.data && raw.session) {
raw = raw.data;
}
if (!raw || typeof raw !== 'object') {
throw new CommandExecutionError('Booking.com page returned no extractable data');
}
if (raw.blocked) {
throw new CommandExecutionError('Booking.com served a verification / captcha page; retry later or change profile');
}
if (raw.ok !== true) {
throw new CommandExecutionError('Booking.com extractor returned an invalid status');
}
if (!Array.isArray(raw.items)) {
throw new CommandExecutionError('Booking.com extractor returned malformed items');
}
const items = raw.items;
if (items.length === 0) {
const totalText = String(raw.totalText || '').trim();
if (hasPositiveResultCount(totalText)) {
throw new CommandExecutionError(
`Booking.com page declared results but no property cards were parsed: ${totalText}`,
);
}
throw new EmptyResultError(
`booking search ${JSON.stringify(destination)}`,
totalText
? `No hotels rendered (${totalText}). Try a broader destination, different dates, or check the URL in a browser.`
: 'No hotels rendered. Try a broader destination, different dates, or check the URL in a browser.',
);
}
return items.slice(0, limit).map((it, i) => {
if (!it || typeof it !== 'object') {
throw new CommandExecutionError('Booking.com extractor returned malformed hotel row');
}
const name = String(it.name || '').trim();
const country = String(it.country || '').trim();
const slug = String(it.slug || '').trim();
const urlValue = String(it.url || '').trim();
const expectedUrl = country && slug
? `https://www.booking.com/hotel/${country}/${slug}.html`
: '';
if (!name || !/^[a-z]{2}$/.test(country) || !slug || urlValue !== expectedUrl) {
throw new CommandExecutionError('Booking.com hotel row is missing stable name/url identity');
}
return {
rank: offset + i + 1,
name,
country,
slug,
star_rating: it.star_rating,
review_score: it.review_score,
review_count: it.review_count,
price_amount: it.price_amount,
price_currency: it.price_amount == null ? '' : (currency || it.price_currency || ''),
distance: it.distance,
recommended_room: it.recommended_room,
url: urlValue,
};
});
},
});
export const __test__ = {
normalizePositiveInt,
normalizeNonNegativeInt,
normalizeDate,
normalizeCurrency,
normalizeLang,
hasPositiveResultCount,
buildSearchUrl,
EXTRACTOR,
};
+96 -14
View File
@@ -1,10 +1,60 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { requirePage, navigateToChat, fetchFriendList } from './utils.js';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
requirePage, navigateToChat, navigateToGeekChat,
fetchFriendList, fetchGeekFriendLabelList, fetchGeekFriendInfoList,
readEncryptSystemId, assertOk, IDENTITY_MISMATCH_CODE,
readPositiveInteger,
} from './utils.js';
function formatMsgTime(ms) {
if (!ms) return '';
return new Date(ms).toLocaleString('zh-CN');
}
function mapBossRow(f) {
return {
name: f.name || '',
company: '',
job: f.jobName || '',
title: '',
last_msg: f.lastMessageInfo?.text || '',
last_time: f.lastTime || '',
uid: f.encryptUid || '',
security_id: f.securityId || '',
};
}
async function buildGeekRows(page, limit) {
const encryptSystemId = await readEncryptSystemId(page);
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId });
if (labelList.length === 0) {
return [];
}
const slicedLabels = labelList.slice(0, limit);
const friendIds = slicedLabels.map((f) => f.friendId).filter(Boolean);
const enriched = await fetchGeekFriendInfoList(page, friendIds);
const enrichMap = new Map(enriched.map((f) => [String(f.friendId ?? f.uid), f]));
return slicedLabels.map((f) => {
const e = enrichMap.get(String(f.friendId)) || {};
return {
name: e.name || f.name || '',
company: e.brandName || f.brandName || '',
job: e.jobName || f.jobName || '',
title: e.bossTitle || f.bossTitle || '',
last_msg: e.lastMessageInfo?.showText || e.lastMsg || f.lastMsg || '',
last_time: e.lastTime || formatMsgTime(e.lastMessageInfo?.msgTime) || formatMsgTime(f.updateTime) || '',
uid: e.encryptUid || f.encryptFriendId || String(e.uid ?? e.friendId ?? f.friendId ?? ''),
security_id: e.securityId || '',
};
});
}
cli({
site: 'boss',
name: 'chatlist',
access: 'read',
description: 'BOSS直聘查看聊天列表(招聘端)',
description: 'BOSS直聘查看聊天列表(招聘端/求职端',
domain: 'www.zhipin.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
@@ -12,23 +62,55 @@ cli({
args: [
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of results' },
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all)' },
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all, boss side only)' },
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
],
columns: ['name', 'job', 'last_msg', 'last_time', 'uid', 'security_id'],
columns: ['name', 'company', 'job', 'title', 'last_msg', 'last_time', 'uid', 'security_id'],
func: async (page, kwargs) => {
requirePage(page);
const limit = readPositiveInteger(kwargs.limit, 'chatlist --limit', 20, 100);
const pageNum = readPositiveInteger(kwargs.page, 'chatlist --page', 1);
const side = kwargs.side || 'auto';
if (side === 'boss') {
await navigateToChat(page);
const friends = await fetchFriendList(page, {
pageNum,
jobId: kwargs['job-id'] || '0',
});
if (friends.length === 0)
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
return friends.slice(0, limit).map(mapBossRow);
}
if (side === 'geek') {
await navigateToGeekChat(page);
const rows = await buildGeekRows(page, limit);
if (rows.length === 0)
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
return rows;
}
// auto: try recruiter first, fall back to geek on identity mismatch
await navigateToChat(page);
const friends = await fetchFriendList(page, {
pageNum: kwargs.page || 1,
const bossResult = await fetchFriendList(page, {
pageNum,
jobId: kwargs['job-id'] || '0',
allowNonZero: true,
});
return friends.slice(0, kwargs.limit || 20).map((f) => ({
name: f.name || '',
job: f.jobName || '',
last_msg: f.lastMessageInfo?.text || '',
last_time: f.lastTime || '',
uid: f.encryptUid || '',
security_id: f.securityId || '',
}));
if (Array.isArray(bossResult)) {
if (bossResult.length === 0)
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
return bossResult.slice(0, limit).map(mapBossRow);
}
if (bossResult.code === IDENTITY_MISMATCH_CODE) {
await navigateToGeekChat(page);
const rows = await buildGeekRows(page, limit);
if (rows.length === 0)
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
return rows;
}
assertOk(bossResult);
throw new CommandExecutionError('Boss chatlist returned an unexpected response');
},
});
+211
View File
@@ -0,0 +1,211 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './chatlist.js';
const BOSS_FRIEND = {
name: '张三',
jobName: '后端工程师',
lastMessageInfo: { text: '你好' },
lastTime: '2024-01-01 10:00',
encryptUid: 'enc-boss-uid',
securityId: 'boss-sec-id',
};
const GEEK_LABEL_FRIEND = {
friendId: 12345,
name: '李四',
brandName: '字节跳动',
jobName: '产品经理',
bossTitle: 'HR',
lastMsg: '感谢投递',
updateTime: 1704067200000,
encryptFriendId: 'enc-geek-uid',
};
const GEEK_ENRICHED = {
friendId: 12345,
uid: 99999,
name: '李四',
brandName: '字节跳动',
jobName: '产品经理',
bossTitle: 'HR总监',
encryptUid: 'enc-geek-uid',
securityId: 'geek-sec-id',
lastMessageInfo: { showText: '感谢投递', msgTime: 1704067200000 },
lastTime: '2024-01-01',
};
function createPageMock(evaluateImpl) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockImplementation(evaluateImpl),
};
}
describe('boss chatlist', () => {
const command = getRegistry().get('boss/chatlist');
it('--side boss preserves existing behavior with 8-column output', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
name: '张三',
company: '',
job: '后端工程师',
title: '',
last_msg: '你好',
uid: 'enc-boss-uid',
security_id: 'boss-sec-id',
});
});
it('--side geek maps enriched getGeekFriendList data into 8 columns', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
name: '李四',
company: '字节跳动',
job: '产品经理',
title: 'HR总监',
uid: 'enc-geek-uid',
security_id: 'geek-sec-id',
});
});
it('--side geek falls back to label fields when enrichment has no match', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('李四');
expect(rows[0].company).toBe('字节跳动');
expect(rows[0].security_id).toBe('');
});
it('rejects invalid --limit before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { page: 1, limit: 0, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('--side geek reports a true empty chat list as EmptyResultError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [] } };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
it('treats malformed geek enrichment payload as CommandExecutionError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: {} };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('treats null Boss API payload as CommandExecutionError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) return null;
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('maps expired Boss cookies to AuthRequiredError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 7, message: 'Cookie 已过期' };
}
return {};
});
await expect(
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
).rejects.toBeInstanceOf(AuthRequiredError);
});
it('--side auto falls back to geek when recruiter returns code 24', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 24, message: '请切换身份后再试' };
}
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
expect(rows).toHaveLength(1);
expect(rows[0].company).toBe('字节跳动');
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
});
it('--side auto uses recruiter results when code 0 and does not call geek API', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
return {};
});
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('张三');
const evaluateCalls = page.evaluate.mock.calls.map((c) => c[0]);
expect(evaluateCalls.some((s) => s.includes('geekFilterByLabel'))).toBe(false);
});
it('registers --side as a choices-constrained arg defaulting to auto', () => {
const sideArg = command.args.find((a) => a.name === 'side');
expect(sideArg?.choices).toEqual(['auto', 'boss', 'geek']);
expect(sideArg?.default).toBe('auto');
});
});
+98 -24
View File
@@ -1,10 +1,72 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { requirePage, navigateToChat, bossFetch, findFriendByUid } from './utils.js';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import {
requirePage, navigateToChat, navigateToGeekChat,
bossFetch, findFriendByUid, findGeekFriendByUid,
fetchGeekHistoryMsg, readEncryptSystemId,
assertOk, IDENTITY_MISMATCH_CODE,
readPositiveInteger, readRequiredString,
} from './utils.js';
const TYPE_MAP = {
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
6: '名片', 7: '语音', 8: '视频', 9: '表情',
};
function mapBossMsg(m, friend) {
const fromObj = m.from || {};
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
return {
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
type: TYPE_MAP[m.type] || `其他(${m.type})`,
text: m.text || m.body?.text || '',
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
}
function mapGeekMsg(m, friend) {
const fromUid = m.from && m.from.uid;
const isFromBoss = fromUid != null && String(fromUid) === String(friend.uid);
return {
from: isFromBoss ? '对方' : '我',
type: TYPE_MAP[m.type] || `其他(${m.type})`,
text: m.text || m.body?.text || m.body?.content || m.body?.showText ||
JSON.stringify(m.body || {}).slice(0, 120),
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
}
async function bossChatMsg(page, kwargs, existingFriend) {
const friend = existingFriend ?? await findFriendByUid(page, kwargs.uid);
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该候选人');
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const gid = friend.uid;
const securityId = encodeURIComponent(friend.securityId);
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
const msgData = await bossFetch(page, msgUrl);
const messages = msgData.zpData?.messages ?? msgData.zpData?.historyMsgList;
if (!Array.isArray(messages)) {
throw new CommandExecutionError('Boss recruiter history response did not include a message list');
}
if (messages.length === 0) {
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
}
return messages.map((m) => mapBossMsg(m, friend));
}
async function geekChatMsg(page, kwargs, encryptSystemId) {
const friend = await findGeekFriendByUid(page, kwargs.uid, { encryptSystemId });
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该聊天(geek 侧)');
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const messages = await fetchGeekHistoryMsg(page, friend, { page: kwargs.page });
return messages.map((m) => mapGeekMsg(m, friend));
}
cli({
site: 'boss',
name: 'chatmsg',
access: 'read',
description: 'BOSS直聘查看与候选人的聊天消息',
description: 'BOSS直聘查看聊天消息历史(招聘端/求职端)',
domain: 'www.zhipin.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
@@ -12,32 +74,44 @@ cli({
args: [
{ name: 'uid', required: true, positional: true, help: 'Encrypted UID (from chatlist)' },
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
],
columns: ['from', 'type', 'text', 'time'],
func: async (page, kwargs) => {
requirePage(page);
const uid = readRequiredString(kwargs.uid, 'chatmsg uid');
const pageNum = readPositiveInteger(kwargs.page, 'chatmsg --page', 1);
const normalizedKwargs = { ...kwargs, uid, page: pageNum };
const side = kwargs.side || 'auto';
if (side === 'boss') {
await navigateToChat(page);
return await bossChatMsg(page, normalizedKwargs);
}
if (side === 'geek') {
await navigateToGeekChat(page);
const encryptSystemId = await readEncryptSystemId(page);
return await geekChatMsg(page, normalizedKwargs, encryptSystemId);
}
// auto: try recruiter first, fall back to geek when not found or identity mismatch
await navigateToChat(page);
const friend = await findFriendByUid(page, kwargs.uid);
if (!friend)
throw new Error('未找到该候选人');
const gid = friend.uid;
const securityId = encodeURIComponent(friend.securityId);
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
const msgData = await bossFetch(page, msgUrl);
const TYPE_MAP = {
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
6: '名片', 7: '语音', 8: '视频', 9: '表情',
};
const messages = msgData.zpData?.messages || msgData.zpData?.historyMsgList || [];
return messages.map((m) => {
const fromObj = m.from || {};
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
return {
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
type: TYPE_MAP[m.type] || '其他(' + m.type + ')',
text: m.text || m.body?.text || '',
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
};
});
const bossResult = await findFriendByUid(page, uid, { allowNonZero: true });
if (bossResult?.friend) {
return await bossChatMsg(page, normalizedKwargs, bossResult.friend);
}
// Not found or identity mismatch — check for hard errors before falling back
if (bossResult?.code && bossResult.code !== 0 && bossResult.code !== IDENTITY_MISMATCH_CODE) {
assertOk(bossResult);
}
// Fall back to geek side
await navigateToGeekChat(page);
const encryptSystemId = await readEncryptSystemId(page);
const geekFriend = await findGeekFriendByUid(page, uid, { encryptSystemId });
if (!geekFriend) throw new EmptyResultError('boss chatmsg', 'uid 在招聘端与求职端聊天列表中均未找到');
if (!geekFriend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
const messages = await fetchGeekHistoryMsg(page, geekFriend, { page: pageNum });
return messages.map((m) => mapGeekMsg(m, geekFriend));
},
});
+230
View File
@@ -0,0 +1,230 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './chatmsg.js';
const BOSS_FRIEND = {
uid: 12345,
encryptUid: 'enc-boss-uid',
securityId: 'boss-sec-id',
name: '候选人甲',
};
const BOSS_MSGS = [
{ type: 1, text: 'Hello', from: { uid: 99999, name: 'HR' }, time: 1704067200000 },
{ type: 1, text: '感谢', from: { uid: 12345, name: '候选人甲' }, time: 1704067201000 },
];
const GEEK_FRIEND_LABEL = {
friendId: 11111,
encryptFriendId: 'enc-geek-uid',
name: 'Boss张',
brandName: '公司A',
};
const GEEK_FRIEND_ENRICHED = {
friendId: 11111,
uid: 67890,
encryptUid: 'enc-geek-uid',
securityId: 'geek-sec-id',
name: 'Boss张',
};
const GEEK_MSGS = [
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
];
function createPageMock(evaluateImpl) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockImplementation(evaluateImpl),
};
}
describe('boss chatmsg', () => {
const command = getRegistry().get('boss/chatmsg');
it('rejects empty uid before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { uid: ' ', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('rejects invalid --page before navigating', async () => {
const page = createPageMock(async () => ({}));
await expect(
command.func(page, { uid: 'enc-geek-uid', page: 0, side: 'geek' })
).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('--side boss preserves existing behavior', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
if (script.includes('boss/historyMsg')) {
return { code: 0, zpData: { messages: BOSS_MSGS } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' });
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
expect(rows).toHaveLength(2);
expect(rows[0].from).toBe('我');
expect(rows[1].from).toBe('候选人甲');
});
it('--side geek calls historyMsg with bossId, securityId, page, c=20, src=0', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: GEEK_MSGS } };
}
return {};
});
await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
const historyScript = page.evaluate.mock.calls.find((c) => c[0].includes('geek/historyMsg'))?.[0];
expect(historyScript).toBeDefined();
expect(historyScript).toContain('bossId=67890');
expect(historyScript).toContain('securityId=');
expect(historyScript).toContain('page=1');
expect(historyScript).toContain('c=20');
expect(historyScript).toContain('src=0');
});
it('--side geek uses from.uid to determine direction, not received flag', async () => {
// Both messages have received:true (mirrors real geek historyMsg API behaviour)
// Direction is determined by whether m.from.uid matches the boss's uid (67890)
const msgsAllReceived = [
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
];
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: msgsAllReceived } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
// from.uid=67890 matches friend.uid=67890 → boss sent it → '对方'
expect(rows[0].from).toBe('对方');
// from.uid=99999 does not match → geek sent it → '我'
expect(rows[1].from).toBe('我');
});
it('non-text message body does not crash and produces truncated JSON', async () => {
const nonTextMsg = { type: 99, received: true, time: 1704067200000, body: { action: 'resume_request', detail: 'X' } };
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: [nonTextMsg] } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
expect(rows).toHaveLength(1);
expect(rows[0].text).toContain('resume_request');
});
it('--side auto falls back to geek when recruiter returns code 24', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 24, message: '请切换身份后再试' };
}
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: GEEK_MSGS } };
}
return {};
});
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'auto' });
expect(rows).toHaveLength(2);
expect(rows[0].from).toBe('对方');
});
it('--side geek throws when uid is not found in geek chat list', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [] } };
}
return {};
});
await expect(
command.func(page, { uid: 'unknown-uid', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
it('--side boss maps expired cookies to AuthRequiredError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 7, message: 'Cookie 已过期' };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
).rejects.toBeInstanceOf(AuthRequiredError);
});
it('--side boss treats missing history list as parser drift', async () => {
const page = createPageMock(async (script) => {
if (script.includes('getBossFriendListV2')) {
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
}
if (script.includes('boss/historyMsg')) {
return { code: 0, zpData: {} };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
).rejects.toBeInstanceOf(CommandExecutionError);
});
it('--side geek reports an empty history as EmptyResultError', async () => {
const page = createPageMock(async (script) => {
if (script.includes('document.cookie')) return 'test-enc-sys-id';
if (script.includes('geekFilterByLabel')) {
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
}
if (script.includes('getGeekFriendList.json')) {
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
}
if (script.includes('geek/historyMsg')) {
return { code: 0, zpData: { messages: [] } };
}
return {};
});
await expect(
command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' })
).rejects.toBeInstanceOf(EmptyResultError);
});
});
+257 -12
View File
@@ -1,8 +1,11 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
// ── Constants ───────────────────────────────────────────────────────────────
const BOSS_DOMAIN = 'www.zhipin.com';
const CHAT_URL = `https://${BOSS_DOMAIN}/web/chat/index`;
const COOKIE_EXPIRED_CODES = new Set([7, 37]);
const COOKIE_EXPIRED_MSG = 'Cookie 已过期!请在当前 Chrome 浏览器中重新登录 BOSS 直聘。';
const RECRUITER_ONLY_MSG = '该命令仅支持招聘端(BOSS 端)账号,请使用招聘者账号登录后重试。';
const DEFAULT_TIMEOUT = 15_000;
// ── Core helpers ────────────────────────────────────────────────────────────
/**
@@ -10,7 +13,24 @@ const DEFAULT_TIMEOUT = 15_000;
*/
export function requirePage(page) {
if (!page)
throw new Error('Browser page required');
throw new CommandExecutionError('Browser page required');
}
export function readPositiveInteger(raw, name, fallback, max) {
const value = raw === undefined || raw === null || raw === '' ? fallback : Number(raw);
if (!Number.isInteger(value) || value < 1) {
throw new ArgumentError(`boss ${name} must be a positive integer`);
}
if (max !== undefined && value > max) {
throw new ArgumentError(`boss ${name} must be <= ${max}`);
}
return value;
}
export function readRequiredString(raw, name) {
const value = String(raw ?? '').trim();
if (!value) {
throw new ArgumentError(`boss ${name} cannot be empty`);
}
return value;
}
/**
* Navigate to BOSS chat page and wait for it to settle.
@@ -33,19 +53,37 @@ export async function navigateTo(page, url, waitSeconds = 1) {
*/
export function checkAuth(data) {
if (COOKIE_EXPIRED_CODES.has(data.code)) {
throw new Error(COOKIE_EXPIRED_MSG);
throw new AuthRequiredError(BOSS_DOMAIN, COOKIE_EXPIRED_MSG);
}
}
/**
* Map BOSS code=24 ("请切换身份后再试") to a typed AuthRequiredError.
* Recruiter-only commands (recommend, joblist, stats, resume, mark,
* exchange, invite, greet, batchgreet) have no geek-side equivalent;
* surfacing this as a generic COMMAND_EXEC hides what the user must do.
* chatlist / chatmsg avoid this path by using `allowNonZero: true` and
* branching to the geek-side fetch when they see code 24.
*/
function checkRecruiterSide(data) {
if (data.code === IDENTITY_MISMATCH_CODE) {
throw new AuthRequiredError(BOSS_DOMAIN, RECRUITER_ONLY_MSG);
}
}
/**
* Throw if the API response is not code 0.
* Checks for cookie expiry first, then throws with the provided message.
* Checks for cookie expiry first, then identity mismatch, then throws
* with the provided message.
*/
export function assertOk(data, errorPrefix) {
if (!data || typeof data !== 'object') {
throw new CommandExecutionError(`${errorPrefix ? `${errorPrefix}: ` : ''}Boss API returned malformed response`);
}
if (data.code === 0)
return;
checkAuth(data);
checkRecruiterSide(data);
const prefix = errorPrefix ? `${errorPrefix}: ` : '';
throw new Error(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
throw new CommandExecutionError(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
}
/**
* Make a credentialed XHR request via page.evaluate().
@@ -80,7 +118,19 @@ export async function bossFetch(page, url, opts = {}) {
});
}
`;
const data = await page.evaluate(script);
let data;
try {
data = await page.evaluate(script);
} catch (error) {
if (error instanceof AuthRequiredError || error instanceof CommandExecutionError) {
throw error;
}
const message = error instanceof Error ? error.message : String(error);
throw new CommandExecutionError(`Boss API request failed: ${message}`);
}
if (!data || typeof data !== 'object') {
throw new CommandExecutionError('Boss API returned malformed response');
}
// Auto-check auth unless caller opts out
if (!opts.allowNonZero && data.code !== 0) {
assertOk(data);
@@ -95,8 +145,13 @@ export async function fetchFriendList(page, opts = {}) {
const pageNum = opts.pageNum ?? 1;
const jobId = opts.jobId ?? '0';
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getBossFriendListV2.json?page=${pageNum}&status=0&jobId=${jobId}`;
const data = await bossFetch(page, url);
return data.zpData?.friendList || [];
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && data.code !== 0) return data;
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss friend list response did not include zpData.friendList');
}
return list;
}
/**
* Fetch the recommended candidates (greetRecSortList).
@@ -104,7 +159,11 @@ export async function fetchFriendList(page, opts = {}) {
export async function fetchRecommendList(page) {
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/greetRecSortList`;
const data = await bossFetch(page, url);
return data.zpData?.friendList || [];
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss recommend response did not include zpData.friendList');
}
return list;
}
/**
* Find a friend by encryptUid, searching through friend list and optionally greet list.
@@ -115,10 +174,14 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
const checkGreetList = opts.checkGreetList ?? false;
// Search friend list pages
for (let p = 1; p <= maxPages; p++) {
const friends = await fetchFriendList(page, { pageNum: p });
const result = await fetchFriendList(page, { pageNum: p, allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && !Array.isArray(result)) {
return { friend: null, code: result.code };
}
const friends = Array.isArray(result) ? result : [];
const found = friends.find((f) => f.encryptUid === encryptUid);
if (found)
return found;
return opts.allowNonZero ? { friend: found, code: 0 } : found;
if (friends.length === 0)
break;
}
@@ -127,9 +190,9 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
const greetList = await fetchRecommendList(page);
const found = greetList.find((f) => f.encryptUid === encryptUid);
if (found)
return found;
return opts.allowNonZero ? { friend: found, code: 0 } : found;
}
return null;
return opts.allowNonZero ? { friend: null, code: 0 } : null;
}
// ── UI automation helpers ───────────────────────────────────────────────────
/**
@@ -221,3 +284,185 @@ export function verbose(msg) {
console.error(`[opencli:boss] ${msg}`);
}
}
// ── Geek-side helpers ────────────────────────────────────────────────────────
export const IDENTITY_MISMATCH_CODE = 24;
const GEEK_CHAT_URL = `https://${BOSS_DOMAIN}/web/geek/chat`;
/**
* Navigate to the job-seeker chat page.
* Establishes the cookie + JS-global context needed for geek-side API calls.
*/
export async function navigateToGeekChat(page, waitSeconds = 2) {
await page.goto(GEEK_CHAT_URL);
await page.wait({ time: waitSeconds });
}
/**
* Read the encryptSystemId value required by the geek-side list API.
* Strategy (in order):
* 1. Vue app state / Pinia stores / $route.query (Option 1 — runtime source)
* 2. performance.getEntriesByType('resource') — parse from geekFilterByLabel URL
* that the page itself already issued (Option 2 — most deterministic)
* 3. cookie, inline <script> SSR state, known window globals, localStorage (fallbacks)
* Returns empty string if nothing is found; the API may still succeed without it.
* Caller must have navigated to the geek chat page first.
*/
export async function readEncryptSystemId(page) {
const result = await page.evaluate(`
(() => {
// 1. Vue app state / Pinia / $route.query
// The chat component reads encryptSystemId from the app runtime to build
// its own geekFilterByLabel request, so the value lives in the Vue tree.
try {
const appEl = document.querySelector('#app') || document.querySelector('[data-v-app]');
const vueApp = appEl && (appEl.__vue_app__ || appEl._vei);
if (vueApp) {
// 1a. Pinia stores (Vue 3 standard state management on BOSS直聘)
const pinia = vueApp.config && vueApp.config.globalProperties.$pinia;
if (pinia && pinia.state && pinia.state.value) {
for (const store of Object.values(pinia.state.value)) {
try {
const flat = JSON.stringify(store);
if (flat.includes('encryptSystemId')) {
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
} catch (_) {}
}
}
// 1b. Vue Router current route query
const router = vueApp.config && vueApp.config.globalProperties.$router;
const query = router && router.currentRoute && router.currentRoute.value && router.currentRoute.value.query;
if (query && query.encryptSystemId) return query.encryptSystemId;
}
} catch (_) {}
// 2. Performance resource entries — the page already issued geekFilterByLabel
// with encryptSystemId in the URL; read it back from the resource timing API.
try {
const entries = performance.getEntriesByType('resource');
for (const entry of entries) {
if (!entry.name.includes('geekFilterByLabel')) continue;
const u = new URL(entry.name);
const v = u.searchParams.get('encryptSystemId');
if (v) return v;
}
} catch (_) {}
// 3. cookie
try {
const m = document.cookie.match(/encryptSystemId=([^;]+)/i);
if (m) return decodeURIComponent(m[1]);
} catch (_) {}
// 4. inline <script> SSR state (Nuxt embeds server state here)
try {
for (const s of document.querySelectorAll('script:not([src])')) {
const t = s.textContent || '';
if (!t.includes('encryptSystemId')) continue;
const m = t.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
} catch (_) {}
// 5. known BOSS / Nuxt window globals
const KNOWN = [
'__NUXT__', '__INITIAL_STATE__', '__ZP_INFO__', '__BOSS_ZP__',
'pageGlobalVar', 'ZP_DATA', '__ZP_DATA__', '__PAGE_DATA__',
];
for (const k of KNOWN) {
const obj = window[k];
if (!obj || typeof obj !== 'object') continue;
try {
const flat = JSON.stringify(obj);
if (!flat.includes('encryptSystemId')) continue;
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
} catch (_) {}
}
// 6. localStorage
try {
for (let i = 0; i < localStorage.length; i++) {
const k = localStorage.key(i);
if (!k) continue;
if (k.toLowerCase().includes('encryptsystemid')) {
const v = localStorage.getItem(k);
if (v) return v;
}
const v = localStorage.getItem(k) || '';
if (v.includes('encryptSystemId')) {
const m = v.match(/"encryptSystemId":"([^"]+)"/);
if (m) return m[1];
}
}
} catch (_) {}
return '';
})()
`);
return result || '';
}
/**
* Fetch the job-seeker chat list (brief info, no securityId).
* Use fetchGeekFriendInfoList to enrich with securityId before calling chatmsg.
*/
export async function fetchGeekFriendLabelList(page, opts = {}) {
const labelId = opts.labelId ?? 0;
const encryptSystemId = opts.encryptSystemId ?? '';
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/geekFilterByLabel?labelId=${labelId}&encryptSystemId=${encodeURIComponent(encryptSystemId)}`;
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
if (opts.allowNonZero && data.code !== 0) return data;
const list = data.zpData?.friendList;
if (!Array.isArray(list)) {
throw new CommandExecutionError('Boss geek chat list response did not include zpData.friendList');
}
return list;
}
/**
* Enrich a batch of geek friends with full fields including securityId.
* Processes in batches of 50 to avoid oversized request bodies.
*/
export async function fetchGeekFriendInfoList(page, friendIds = []) {
if (!friendIds.length) return [];
const BATCH_SIZE = 50;
const results = [];
for (let i = 0; i < friendIds.length; i += BATCH_SIZE) {
const batch = friendIds.slice(i, i + BATCH_SIZE).map(String);
const body = `friendIds=${batch.join(',')}`;
const data = await bossFetch(page, `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getGeekFriendList.json`, {
method: 'POST',
body,
});
const batchResult = data.zpData?.result;
if (!Array.isArray(batchResult)) {
throw new CommandExecutionError('Boss geek friend enrichment response did not include zpData.result');
}
results.push(...batchResult);
}
return results;
}
/**
* Find a geek-side friend by encrypted uid.
* Merges label-list and enriched data; returns null if not found.
*/
export async function findGeekFriendByUid(page, encryptUid, opts = {}) {
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId: opts.encryptSystemId });
const candidate = labelList.find((f) => f.encryptFriendId === encryptUid ||
String(f.uid) === String(encryptUid) ||
String(f.friendId) === String(encryptUid));
if (!candidate) return null;
const enriched = await fetchGeekFriendInfoList(page, [candidate.friendId]);
return { ...candidate, ...(enriched[0] || {}) };
}
/**
* Fetch message history for a geek-side chat.
* friend must have .uid (boss's numeric id) and .securityId.
*/
export async function fetchGeekHistoryMsg(page, friend, opts = {}) {
const pageNum = opts.page ?? 1;
const bossId = friend.uid;
const securityId = encodeURIComponent(friend.securityId || '');
const url = `https://${BOSS_DOMAIN}/wapi/zpchat/geek/historyMsg?bossId=${bossId}&securityId=${securityId}&page=${pageNum}&c=20&src=0`;
const data = await bossFetch(page, url);
const messages = data.zpData?.messages ?? data.zpData?.historyMsgList;
if (!Array.isArray(messages)) {
throw new CommandExecutionError('Boss geek history response did not include a message list');
}
if (messages.length === 0) {
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
}
return messages;
}
+34
View File
@@ -0,0 +1,34 @@
import { describe, expect, it } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { assertOk } from './utils.js';
describe('assertOk', () => {
it('returns silently on code 0', () => {
expect(() => assertOk({ code: 0 })).not.toThrow();
});
it('maps expired cookie codes (7, 37) to AuthRequiredError', () => {
expect(() => assertOk({ code: 7, message: 'expired' })).toThrow(AuthRequiredError);
expect(() => assertOk({ code: 37, message: 'expired' })).toThrow(AuthRequiredError);
});
it('maps code 24 (identity mismatch) to AuthRequiredError with recruiter-only hint', () => {
try {
assertOk({ code: 24, message: '请切换身份后再试' });
throw new Error('assertOk should have thrown');
} catch (err) {
expect(err).toBeInstanceOf(AuthRequiredError);
expect(String(err.message)).toContain('招聘端');
}
});
it('falls through to CommandExecutionError for other non-zero codes', () => {
expect(() => assertOk({ code: 99, message: 'something else' }))
.toThrow(CommandExecutionError);
});
it('throws CommandExecutionError on malformed (non-object) response', () => {
expect(() => assertOk(null)).toThrow(CommandExecutionError);
expect(() => assertOk('not-an-object')).toThrow(CommandExecutionError);
});
});
+80
View File
@@ -0,0 +1,80 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import {
emptySearchResults,
requireBoundedInteger,
requireNonNegativeInteger,
requireRows,
requireSearchQuery,
runBrowserStep,
toHttpsUrl,
} from '../_shared/search-adapter.js';
function buildExtractorJs(limit) {
return `
(function() {
var results = [];
var seen = {};
var items = document.querySelectorAll('.snippet');
for (var i = 0; i < items.length; i++) {
if (results.length >= ${limit}) break;
var el = items[i];
if (el.classList.contains('standalone') || el.classList.contains('ad')) continue;
var titleEl = el.querySelector('.search-snippet-title');
var snippetEl = el.querySelector('.generic-snippet .content');
var linkEl = el.querySelector('.result-content a');
if (!titleEl) continue;
var title = titleEl.textContent.trim();
var href = linkEl ? linkEl.getAttribute('href') || '' : '';
var snippet = snippetEl ? snippetEl.textContent.trim() : '';
if (!title || !href || seen[href]) continue;
if (href.indexOf('/') === 0) continue;
seen[href] = true;
results.push([title, href, snippet]);
}
return results;
})()`;
}
const command = cli({
site: 'brave',
name: 'search',
access: 'read',
description: 'Search Brave Search',
domain: 'search.brave.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Search query' },
{ name: 'limit', type: 'int', default: 10, help: 'Number of results per page (max 18)' },
{ name: 'offset', type: 'int', default: 0, help: 'Page offset (0, 1, 2...). Brave returns ~18 results per page' },
],
columns: ['rank', 'title', 'url', 'snippet'],
func: async (page, kwargs) => {
const limit = requireBoundedInteger(kwargs.limit, 10, 1, 18, '--limit');
const query = requireSearchQuery(kwargs.keyword);
const keyword = encodeURIComponent(query);
const offset = requireNonNegativeInteger(kwargs.offset, 0, '--offset');
let url = `https://search.brave.com/search?q=${keyword}`;
if (offset > 0) url += `&offset=${offset}`;
await runBrowserStep('brave search navigation', () => page.goto(url));
try {
await page.wait({ selector: '.snippet', timeout: 10 });
} catch {
await page.wait(3).catch(function() {});
}
const raw = await runBrowserStep('brave search extraction', () => page.evaluate(buildExtractorJs(limit)));
const results = requireRows(raw, 'brave search');
if (results.length === 0) {
throw emptySearchResults('Brave', query);
}
const rows = results
.map(function(r, index) {
return { rank: index + 1 + offset * 18, title: r[0], url: toHttpsUrl(r[1], 'https://search.brave.com'), snippet: r[2] };
})
.filter((row) => row.url);
if (rows.length === 0) throw emptySearchResults('Brave', query);
return rows;
},
});
export const __test__ = { command };
+76
View File
@@ -0,0 +1,76 @@
import { describe, it, expect, vi } from 'vitest';
const { __test__ } = await import('./search.js');
const command = __test__.command;
function createPageMock(evaluateResult = []) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('brave search', () => {
it('should register as a valid command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('brave');
expect(command.name).toBe('search');
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.strategy).toBe('public');
expect(command.domain).toBe('search.brave.com');
});
it('should define keyword positional arg', () => {
const kwArg = command.args.find(a => a.name === 'keyword');
expect(kwArg).toBeDefined();
expect(kwArg.positional).toBe(true);
expect(kwArg.required).toBe(true);
});
it('should define limit arg with default 10', () => {
const limitArg = command.args.find(a => a.name === 'limit');
expect(limitArg).toBeDefined();
expect(limitArg.type).toBe('int');
expect(limitArg.default).toBe(10);
});
it('should define output columns', () => {
expect(command.columns).toContain('rank');
expect(command.columns).toContain('title');
expect(command.columns).toContain('url');
expect(command.columns).toContain('snippet');
});
it('rejects empty query, invalid limit, and invalid offset before navigation', async () => {
const page = createPageMock();
await expect(command.func(page, { keyword: '', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 19 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 5, offset: -1 })).rejects.toMatchObject({ code: 'ARGUMENT' });
expect(page.goto).not.toHaveBeenCalled();
});
it('unwraps browser envelopes and returns ranked HTTPS rows', async () => {
const page = createPageMock({
session: 'site:brave',
data: [['OpenCLI', 'https://github.com/jackwener/OpenCLI', 'CLI browser tooling']],
});
await expect(command.func(page, { keyword: 'opencli', limit: 1, offset: 1 })).resolves.toEqual([{
rank: 19,
title: 'OpenCLI',
url: 'https://github.com/jackwener/OpenCLI',
snippet: 'CLI browser tooling',
}]);
});
it('fails typed instead of silently returning [] for malformed extraction payloads', async () => {
const page = createPageMock({ rows: [] });
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('payload shape'),
});
});
});
+108
View File
@@ -0,0 +1,108 @@
import { describe, expect, it } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import {
requireArrayEvaluateResult,
requireBooleanEvaluateResult,
requireObjectEvaluateResult,
unwrapEvaluateResult,
} from './utils.js';
describe('chatgpt page.evaluate envelope helpers', () => {
describe('unwrapEvaluateResult', () => {
it('unwraps a { session, data } envelope produced by the browser bridge', () => {
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 'msg-1' }] };
expect(unwrapEvaluateResult(envelope)).toEqual([{ id: 'msg-1' }]);
});
it('passes raw arrays through unchanged (back-compat with older bridge versions)', () => {
const raw = [1, 2, 3];
expect(unwrapEvaluateResult(raw)).toBe(raw);
});
it('passes primitive return values (URL strings, booleans) through unchanged', () => {
expect(unwrapEvaluateResult('https://chatgpt.com/c/abc')).toBe('https://chatgpt.com/c/abc');
expect(unwrapEvaluateResult(true)).toBe(true);
expect(unwrapEvaluateResult(0)).toBe(0);
});
it('passes plain non-envelope objects through unchanged', () => {
const obj = { ok: true, reason: 'all good' };
expect(unwrapEvaluateResult(obj)).toBe(obj);
});
it('handles null and undefined defensively', () => {
expect(unwrapEvaluateResult(null)).toBe(null);
expect(unwrapEvaluateResult(undefined)).toBe(undefined);
});
});
describe('requireArrayEvaluateResult', () => {
it('returns the payload when it is an array', () => {
const rows = [{ id: 1 }, { id: 2 }];
expect(requireArrayEvaluateResult(rows, 'chatgpt test')).toBe(rows);
});
it('throws a typed CommandExecutionError when the payload is the raw envelope (caller forgot to unwrap)', () => {
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 1 }] };
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
.toThrowError(CommandExecutionError);
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
.toThrow(/malformed extraction payload/);
});
it('surfaces the inner error message when the payload carries an `error` field', () => {
const errPayload = { error: 'image generator returned 500' };
expect(() => requireArrayEvaluateResult(errPayload, 'chatgpt image asset export'))
.toThrow(/chatgpt image asset export: image generator returned 500/);
});
it('throws when the payload is null or a primitive', () => {
expect(() => requireArrayEvaluateResult(null, 'chatgpt test')).toThrowError(CommandExecutionError);
expect(() => requireArrayEvaluateResult('a string', 'chatgpt test')).toThrowError(CommandExecutionError);
});
});
describe('requireObjectEvaluateResult', () => {
it('returns the payload when it is a plain object', () => {
const obj = { url: 'https://chatgpt.com', isLoggedIn: true };
expect(requireObjectEvaluateResult(obj, 'chatgpt page state')).toBe(obj);
});
it('throws when the payload is an array or a primitive', () => {
expect(() => requireObjectEvaluateResult([], 'chatgpt page state')).toThrowError(CommandExecutionError);
expect(() => requireObjectEvaluateResult('string', 'chatgpt page state')).toThrowError(CommandExecutionError);
expect(() => requireObjectEvaluateResult(null, 'chatgpt page state')).toThrowError(CommandExecutionError);
});
});
describe('requireBooleanEvaluateResult', () => {
it('returns booleans and rejects wrong-shape values', () => {
expect(requireBooleanEvaluateResult(true, 'chatgpt generation state')).toBe(true);
expect(requireBooleanEvaluateResult(false, 'chatgpt generation state')).toBe(false);
expect(() => requireBooleanEvaluateResult({ ok: true }, 'chatgpt generation state'))
.toThrowError(CommandExecutionError);
});
});
describe('end-to-end envelope sweep', () => {
// The bridge envelope is shaped like { session, data } where `session` is
// any string and `data` is the actual return value. Verify the helpers
// chain correctly: unwrap → require* yields the inner shape.
it('unwrap + requireArray pipes an envelope through to the underlying array', () => {
const envelope = {
session: 'site:chatgpt:img-export',
data: [
{ url: 'https://a.example/1.png', dataUrl: 'data:image/png;base64,xxx', mimeType: 'image/png' },
],
};
expect(requireArrayEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt image asset export'))
.toEqual(envelope.data);
});
it('unwrap + requireObject pipes an envelope through to the underlying object', () => {
const envelope = { session: 'site:chatgpt:state', data: { url: 'https://chatgpt.com', isLoggedIn: true } };
expect(requireObjectEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt page state'))
.toEqual(envelope.data);
});
});
});
+2 -2
View File
@@ -4,7 +4,7 @@ import * as fs from 'node:fs';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { saveBase64ToFile } from '@jackwener/opencli/utils';
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { clearChatGPTDraft, getChatGPTVisibleImageUrls, normalizeBooleanFlag, prepareChatGPTImagePaths, sendChatGPTMessage, waitForChatGPTImages, getChatGPTImageAssets, uploadChatGPTImages } from './utils.js';
import { clearChatGPTDraft, getChatGPTVisibleImageUrls, normalizeBooleanFlag, prepareChatGPTImagePaths, sendChatGPTMessage, unwrapEvaluateResult, waitForChatGPTImages, getChatGPTImageAssets, uploadChatGPTImages } from './utils.js';
const CHATGPT_DOMAIN = 'chatgpt.com';
@@ -54,7 +54,7 @@ function buildPrompt(prompt, imageCount) {
}
async function currentChatGPTLink(page) {
const url = await page.evaluate('window.location.href').catch(() => '');
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
return typeof url === 'string' && url ? url : 'https://chatgpt.com';
}
+6
View File
@@ -24,6 +24,12 @@ vi.mock('./utils.js', () => ({
},
prepareChatGPTImagePaths: mocks.prepareChatGPTImagePaths,
sendChatGPTMessage: mocks.sendChatGPTMessage,
unwrapEvaluateResult: (payload) => {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
},
uploadChatGPTImages: mocks.uploadChatGPTImages,
waitForChatGPTImages: mocks.waitForChatGPTImages,
getChatGPTImageAssets: mocks.getChatGPTImageAssets,
+148 -41
View File
@@ -74,7 +74,6 @@ function buildComposerLocatorScript() {
};
findComposer.toString = () => 'findComposer';
return { findComposer, markerAttr };
`;
}
@@ -103,6 +102,50 @@ export function requirePositiveInt(value, flagLabel, hint) {
return value;
}
// ─────────────────────────────────────────────────────────────────────────────
// page.evaluate envelope helpers.
//
// The browser bridge wraps every `page.evaluate(...)` return value in a
// `{ session, data }` envelope. Adapters that read `.length` or
// `Array.isArray(payload)` directly on the envelope silently see "no data" —
// this matches the failure mode fixed for xiaohongshu/rednote (#1561) and
// weibo (#1568).
//
// `unwrapEvaluateResult` is a defensive ternary: it unwraps when the payload
// looks like an envelope, otherwise passes the value through unchanged so
// older bridge versions and primitive return values still work.
// ─────────────────────────────────────────────────────────────────────────────
export function unwrapEvaluateResult(payload) {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
}
export function requireArrayEvaluateResult(payload, label) {
if (!Array.isArray(payload)) {
if (payload && typeof payload === 'object' && 'error' in payload) {
throw new CommandExecutionError(`${label}: ${String(payload.error)}`);
}
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function requireObjectEvaluateResult(payload, label) {
if (!payload || Array.isArray(payload) || typeof payload !== 'object') {
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function requireBooleanEvaluateResult(payload, label) {
if (typeof payload !== 'boolean') {
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
}
return payload;
}
export function parseChatGPTConversationId(value) {
const raw = String(value ?? '').trim();
const match = raw.match(/(?:^|\/c\/)([A-Za-z0-9_-]{8,})(?:[/?#]|$)/);
@@ -115,7 +158,7 @@ export function parseChatGPTConversationId(value) {
}
export async function currentChatGPTUrl(page) {
const url = await page.evaluate('window.location.href').catch(() => '');
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
return typeof url === 'string' ? url : '';
}
@@ -161,7 +204,7 @@ export async function startNewChat(page) {
}
export async function getPageState(page) {
return await page.evaluate(`(() => {
return requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
@@ -187,7 +230,7 @@ export async function getPageState(page) {
isLoggedIn: hasComposer || !!userMenu || !hasLoginGate,
hasLoginGate,
};
})()`);
})()`)), 'chatgpt page state');
}
export async function ensureChatGPTLogin(page, message = 'ChatGPT requires a logged-in browser session.') {
@@ -258,7 +301,7 @@ export async function sendChatGPTMessage(page, text) {
// findComposer() retries inside a single CDP call, so no fixed sleep is
// needed before reading the composer.
const typeResult = await page.evaluate(`
const typeResult = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
${buildComposerLocatorScript()}
const composer = findComposer();
@@ -276,8 +319,8 @@ export async function sendChatGPTMessage(page, text) {
composer.dispatchEvent(new Event('change', { bubbles: true }));
return true;
})()
`);
`)), 'chatgpt composer readiness');
if (!typeResult) return false;
// Use page.type() which is Playwright's native method
@@ -304,7 +347,7 @@ export async function sendChatGPTMessage(page, text) {
let sent = null;
for (let attempt = 0; attempt < 20; attempt += 1) {
await page.wait(0.5);
sent = await page.evaluate(`
sent = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const isUsable = (button) => button
&& !button.disabled
@@ -318,7 +361,7 @@ export async function sendChatGPTMessage(page, text) {
: btns.find(b => labels.includes(b.getAttribute('aria-label') || '') && isUsable(b));
return { sendBtnFound: !!sendBtn };
})()
`);
`)), 'chatgpt send button readiness');
if (sent?.sendBtnFound) break;
}
@@ -339,7 +382,7 @@ export async function sendChatGPTMessage(page, text) {
}
export async function getVisibleMessages(page) {
const result = await page.evaluate(`(() => {
const result = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
@@ -385,8 +428,7 @@ export async function getVisibleMessages(page) {
rows.push({ role, text, html });
}
return rows;
})()`);
if (!Array.isArray(result)) return [];
})()`)), 'chatgpt visible messages');
return result.map((item, index) => ({
Index: index + 1,
Role: item?.role === 'Assistant' ? 'Assistant' : 'User',
@@ -448,7 +490,7 @@ export async function getConversationList(page) {
// so the previous standalone 2 s settle is redundant.
await ensureOnChatGPT(page);
const openSidebar = await page.evaluate(`(() => {
const openSidebar = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const button = Array.from(document.querySelectorAll('button'))
.find((node) => /open sidebar/i.test(node.getAttribute('aria-label') || ''));
if (button instanceof HTMLElement) {
@@ -456,7 +498,7 @@ export async function getConversationList(page) {
return true;
}
return false;
})()`);
})()`)), 'chatgpt sidebar open state');
if (openSidebar) {
try {
await page.wait({ selector: CONVERSATION_LINK_SELECTOR, timeout: 3 });
@@ -480,7 +522,7 @@ export async function getConversationList(page) {
}
async function extractConversationLinks(page) {
const items = await page.evaluate(`(() => {
const items = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
const style = window.getComputedStyle(el);
@@ -505,15 +547,13 @@ async function extractConversationLinks(page) {
});
}
return rows;
})()`);
return Array.isArray(items)
? items.map((item, index) => ({
})()`)), 'chatgpt conversation link extraction');
return items.map((item, index) => ({
Index: index + 1,
Id: String(item?.Id || ''),
Title: String(item?.Title || '(untitled)').trim() || '(untitled)',
Url: String(item?.Url || ''),
})).filter((item) => item.Id)
: [];
})).filter((item) => item.Id);
}
function imageMimeFromPath(filePath) {
@@ -556,7 +596,7 @@ async function waitForChatGPTUploadPreview(page, fileNames) {
const namesJson = JSON.stringify(fileNames);
for (let attempt = 0; attempt < 10; attempt += 1) {
await page.wait(1);
const ready = await page.evaluate(`
const ready = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const names = ${namesJson};
const text = document.body ? (document.body.innerText || '') : '';
@@ -572,7 +612,7 @@ async function waitForChatGPTUploadPreview(page, fileNames) {
const previewNodes = scope.querySelectorAll('img[src], canvas, video, [style*="background-image"], [data-testid*="attachment"], [data-testid*="upload"], [class*="attachment"], [class*="upload"]');
return previewNodes.length >= names.length;
})()
`);
`)), 'chatgpt upload preview detection');
if (ready) return true;
}
return false;
@@ -606,7 +646,7 @@ export async function uploadChatGPTImages(page, imagePaths) {
mime: imageMimeFromPath(absPath),
base64: fs.default.readFileSync(absPath).toString('base64'),
}));
const fallbackResult = await page.evaluate(`
const fallbackResult = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const files = ${JSON.stringify(files)};
const input = document.querySelector('input[type="file"]');
@@ -642,7 +682,7 @@ export async function uploadChatGPTImages(page, imagePaths) {
}
return { ok: true };
})()
`);
`)), 'chatgpt image upload fallback');
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
}
@@ -656,21 +696,21 @@ export async function uploadChatGPTImages(page, imagePaths) {
* Check if ChatGPT is still generating a response.
*/
export async function isGenerating(page) {
return await page.evaluate(`
return requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
return Array.from(document.querySelectorAll('button')).some(b => {
const label = b.getAttribute('aria-label') || '';
return label === 'Stop generating' || label.includes('Thinking');
});
})()
`);
`)), 'chatgpt generation state');
}
/**
* Get visible image URLs from the ChatGPT page (excluding profile/avatar images).
*/
export async function getChatGPTVisibleImageUrls(page) {
return await page.evaluate(`
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(() => {
const isVisible = (el) => {
if (!(el instanceof HTMLElement)) return false;
@@ -680,32 +720,78 @@ export async function getChatGPTVisibleImageUrls(page) {
return rect.width > 32 && rect.height > 32;
};
const urls = [];
const seen = new Set();
const normalizeUrl = (value) => {
const raw = String(value || '').trim();
if (!raw || raw === 'none') return '';
if (/^(?:https?:|blob:|data:)/i.test(raw)) return raw;
try {
return new URL(raw, window.location.href).href;
} catch {
return raw;
}
};
const addUrl = (value) => {
const src = normalizeUrl(value);
if (!src || seen.has(src)) return;
seen.add(src);
urls.push(src);
};
const isDecorative = (el, src = '') => {
const alt = (el.getAttribute('alt') || '').toLowerCase();
const cls = String(el.className || '').toLowerCase();
const testId = (el.getAttribute('data-testid') || '').toLowerCase();
const label = (el.getAttribute('aria-label') || '').toLowerCase();
const text = [alt, cls, testId, label, src.toLowerCase()].join(' ');
return /avatar|profile|logo|icon/.test(text);
};
const imgs = Array.from(document.querySelectorAll('img')).filter(img =>
img instanceof HTMLImageElement && isVisible(img)
);
const urls = [];
const seen = new Set();
for (const img of imgs) {
const src = img.currentSrc || img.src || '';
const alt = (img.getAttribute('alt') || '').toLowerCase();
const cls = (img.className || '').toLowerCase();
const width = img.naturalWidth || img.width || 0;
const height = img.naturalHeight || img.height || 0;
if (!src) continue;
if (alt.includes('avatar') || alt.includes('profile') || alt.includes('logo') || alt.includes('icon')) continue;
if (cls.includes('avatar') || cls.includes('profile') || cls.includes('icon')) continue;
if (isDecorative(img, src)) continue;
if (width < 128 && height < 128) continue;
if (seen.has(src)) continue;
addUrl(src);
}
seen.add(src);
urls.push(src);
// ChatGPT occasionally renders generated images as CSS background
// thumbnails instead of plain <img> nodes. Treat visible, large
// background images as generated-image candidates too.
for (const el of Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]'))) {
if (!(el instanceof HTMLElement) || !isVisible(el) || isDecorative(el)) continue;
const rect = el.getBoundingClientRect();
if (rect.width < 128 && rect.height < 128) continue;
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
for (const match of backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)) {
const src = match[2];
if (!src || isDecorative(el, src)) continue;
addUrl(src);
}
}
// Some image experiences render to a canvas. Returning the data URL
// lets the downstream asset exporter save it without needing a DOM
// selector to rediscover the canvas.
for (const canvas of Array.from(document.querySelectorAll('canvas'))) {
if (!(canvas instanceof HTMLCanvasElement) || !isVisible(canvas) || isDecorative(canvas)) continue;
const width = canvas.width || canvas.getBoundingClientRect().width || 0;
const height = canvas.height || canvas.getBoundingClientRect().height || 0;
if (width < 128 && height < 128) continue;
try {
addUrl(canvas.toDataURL('image/png'));
} catch { }
}
return urls;
})()
`);
`)), 'chatgpt visible image url extraction');
}
/**
@@ -723,7 +809,7 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
let currentUrl = '';
if (convUrl && convUrl.includes('/c/')) {
currentUrl = await page.evaluate('window.location.href').catch(() => '');
currentUrl = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
if (currentUrl && !isSameChatGPTConversation(currentUrl, convUrl)) {
await page.goto(convUrl);
await page.wait(3);
@@ -766,6 +852,7 @@ export const __test__ = {
SEND_BUTTON_FALLBACK_SELECTORS,
SEND_BUTTON_LABELS,
CLOSE_SIDEBAR_LABELS,
buildComposerLocatorScript,
isSameChatGPTConversation,
parseChatGPTConversationId,
imageMimeFromPath,
@@ -776,7 +863,7 @@ export const __test__ = {
*/
export async function getChatGPTImageAssets(page, urls) {
const urlsJson = JSON.stringify(urls);
return await page.evaluate(`
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
(async (targetUrls) => {
const blobToDataUrl = (blob) => new Promise((resolve, reject) => {
const reader = new FileReader();
@@ -809,6 +896,26 @@ export async function getChatGPTImageAssets(page, urls) {
if (img) {
width = img.naturalWidth || img.width || 0;
height = img.naturalHeight || img.height || 0;
} else {
const backgroundEl = Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]')).find(el => {
if (!(el instanceof HTMLElement)) return false;
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
return Array.from(backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)).some(match => {
const raw = String(match[2] || '').trim();
if (!raw) return false;
if (raw === targetUrl) return true;
try {
return new URL(raw, window.location.href).href === targetUrl;
} catch {
return false;
}
});
});
if (backgroundEl) {
const rect = backgroundEl.getBoundingClientRect();
width = Math.round(rect.width || 0);
height = Math.round(rect.height || 0);
}
}
try {
@@ -850,5 +957,5 @@ export async function getChatGPTImageAssets(page, urls) {
return results;
})(${urlsJson})
`, urls);
`)), 'chatgpt image asset export');
}
+92 -2
View File
@@ -1,8 +1,9 @@
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { JSDOM } from 'jsdom';
import { afterEach, describe, expect, it, vi } from 'vitest';
import { __test__, prepareChatGPTImagePaths, sendChatGPTMessage, uploadChatGPTImages, waitForChatGPTImages } from './utils.js';
import { __test__, getChatGPTImageAssets, getChatGPTVisibleImageUrls, prepareChatGPTImagePaths, sendChatGPTMessage, uploadChatGPTImages, waitForChatGPTImages } from './utils.js';
const tempDirs = [];
@@ -88,6 +89,25 @@ describe('chatgpt conversation id parsing', () => {
});
describe('chatgpt send selectors', () => {
it('inlines the composer locator without returning before caller code runs', () => {
const dom = new JSDOM('<!doctype html><div id="prompt-textarea" contenteditable="true"></div>', {
url: 'https://chatgpt.com/',
runScripts: 'outside-only',
});
const composer = dom.window.document.querySelector('#prompt-textarea');
composer.getBoundingClientRect = () => ({ width: 320, height: 48 });
const result = dom.window.eval(`
(() => {
${__test__.buildComposerLocatorScript()}
const composer = findComposer();
return !!composer && composer.getAttribute(markerAttr) === '1';
})()
`);
expect(result).toBe(true);
});
it('keeps locale-independent send-button selector before aria-label fallbacks', async () => {
const page = {
wait: vi.fn().mockResolvedValue(undefined),
@@ -143,6 +163,73 @@ describe('chatgpt send selectors', () => {
});
});
describe('chatgpt generated image detection', () => {
function createDomPage(html, setup = () => {}) {
const dom = new JSDOM(html, {
url: 'https://chatgpt.com/c/demo',
runScripts: 'outside-only',
});
setup(dom.window);
return {
evaluate: vi.fn((script) => Promise.resolve(dom.window.eval(String(script)))),
};
}
it('detects visible CSS background images when ChatGPT does not render a plain img', async () => {
const page = createDomPage(`
<!doctype html>
<main>
<div class="avatar" style="background-image: url('https://chatgpt.com/avatar.png')"></div>
<button data-testid="generated-image" style="background-image: url('/backend-api/generated/foo.webp')"></button>
</main>
`, (window) => {
for (const el of window.document.querySelectorAll('div, button')) {
el.getBoundingClientRect = () => ({ width: 512, height: 512 });
}
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
'https://chatgpt.com/backend-api/generated/foo.webp',
]);
});
it('detects visible generated canvases as data URLs', async () => {
const page = createDomPage('<!doctype html><canvas width="512" height="512"></canvas>', (window) => {
const canvas = window.document.querySelector('canvas');
canvas.getBoundingClientRect = () => ({ width: 512, height: 512 });
canvas.toDataURL = () => 'data:image/png;base64,ZmFrZQ==';
});
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
'data:image/png;base64,ZmFrZQ==',
]);
});
it('exports assets for generated CSS background images', async () => {
const imageUrl = 'https://chatgpt.com/backend-api/generated/foo.webp';
const page = createDomPage(`
<!doctype html>
<button style="background-image: url('/backend-api/generated/foo.webp')"></button>
`, (window) => {
const button = window.document.querySelector('button');
button.getBoundingClientRect = () => ({ width: 512, height: 512 });
window.fetch = vi.fn().mockResolvedValue({
ok: true,
blob: async () => new window.Blob(['fake-image'], { type: 'image/webp' }),
});
});
await expect(getChatGPTImageAssets(page, [imageUrl])).resolves.toEqual([
expect.objectContaining({
url: imageUrl,
mimeType: 'image/webp',
width: 512,
height: 512,
}),
]);
});
});
describe('chatgpt image upload helper', () => {
it('validates local images without a browser page', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
@@ -218,7 +305,10 @@ describe('chatgpt image upload helper', () => {
setFileInput: vi.fn().mockRejectedValue(new Error('No element found')),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn((script) => {
return Promise.resolve({ ok: true });
if (String(script).includes('new DataTransfer()')) {
return Promise.resolve({ ok: true });
}
return Promise.resolve(true);
}),
};
+44 -20
View File
@@ -1,4 +1,11 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { unwrapEvaluateResult } from './evaluate-result.js';
function isAuthLikeError(code, message) {
const text = String(message ?? '');
return code === 401 || code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(text);
}
/**
* Execute a fetch() call inside the Chrome browser context via page.evaluate.
* This ensures a_bogus signing and cookies are handled automatically by the browser.
@@ -6,36 +13,53 @@ import { CommandExecutionError } from '@jackwener/opencli/errors';
export async function browserFetch(page, method, url, options = {}) {
const js = `
(async () => {
const res = await fetch(${JSON.stringify(url)}, {
method: ${JSON.stringify(method)},
credentials: 'include',
headers: {
'Content-Type': 'application/json',
...${JSON.stringify(options.headers ?? {})}
},
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
});
const text = await res.text();
if (!text) return null;
return JSON.parse(text);
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), ${Number(options.timeoutMs ?? 30000)});
try {
const res = await fetch(${JSON.stringify(url)}, {
method: ${JSON.stringify(method)},
credentials: 'include',
signal: controller.signal,
headers: {
'Content-Type': 'application/json',
...${JSON.stringify(options.headers ?? {})}
},
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
});
const text = await res.text();
try {
return JSON.parse(text);
} catch (error) {
return { status_code: res.ok ? -2 : res.status, status_msg: \`JSON parse failed: \${text.slice(0, 500) || String(error && error.message || error)}\` };
}
} catch (error) {
return { status_code: -1, status_msg: String(error && error.message || error) };
} finally {
clearTimeout(timer);
}
})()
`;
let result;
try {
result = await page.evaluate(js);
result = unwrapEvaluateResult(await page.evaluate(js));
}
catch (error) {
const message = error instanceof Error ? error.message : String(error);
throw new CommandExecutionError(`Douyin API request failed: ${message}`);
throw new CommandExecutionError(`Douyin API request failed (${method} ${url}): ${error instanceof Error ? error.message : String(error)}`);
}
if (result === null || result === undefined) {
throw new CommandExecutionError('Empty response from Douyin API');
if (result == null) {
throw new CommandExecutionError(`Empty response from Douyin API (${method} ${url})`);
}
if (Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`Malformed response from Douyin API (${method} ${url})`);
}
if (result && typeof result === 'object' && 'status_code' in result) {
const code = result.status_code;
if (code !== 0) {
const msg = result.status_msg ?? 'unknown error';
throw new CommandExecutionError(`Douyin API error ${code}: ${msg}`);
const msg = result.status_msg ?? result.message ?? 'unknown error';
if (isAuthLikeError(code, msg)) {
throw new AuthRequiredError('creator.douyin.com', `Douyin API auth/permission error ${code} at ${method} ${url}: ${msg}`);
}
throw new CommandExecutionError(`Douyin API error ${code} at ${method} ${url}: ${msg}`);
}
}
return result;
+22 -1
View File
@@ -1,4 +1,5 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { browserFetch } from './browser-fetch.js';
function makePage(result) {
return {
@@ -18,10 +19,20 @@ describe('browserFetch', () => {
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
expect(result).toEqual({ status_code: 0, data: { ak: 'KEY' } });
});
it('unwraps Browser Bridge {session,data} envelopes', async () => {
const page = makePage({ session: 'site:douyin:test', data: { status_code: 0, data: { ok: true } } });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.resolves.toEqual({ status_code: 0, data: { ok: true } });
});
it('throws when status_code is non-zero', async () => {
const page = makePage({ status_code: 8, message: 'fail' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API error 8');
});
it('maps auth-like API errors to AuthRequiredError', async () => {
const page = makePage({ status_code: 401, status_msg: 'login required' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toBeInstanceOf(AuthRequiredError);
});
it('returns result even when no status_code field', async () => {
const page = makePage({ some_field: 'value' });
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
@@ -35,9 +46,19 @@ describe('browserFetch', () => {
const page = makePage(undefined);
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Empty response from Douyin API');
});
it('throws typed on malformed primitive response body', async () => {
const page = makePage('not-json-object');
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws typed when browser fetch returns a non-JSON body', async () => {
const page = makePage({ status_code: -2, status_msg: 'JSON parse failed: <html>not-json</html>' });
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
.rejects.toThrow('Douyin API error -2');
});
it('wraps browser-side fetch or JSON parse failures', async () => {
const page = makePage(null);
page.evaluate.mockRejectedValueOnce(new SyntaxError('Unexpected token < in JSON'));
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API request failed: Unexpected token < in JSON');
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API request failed (GET https://creator.douyin.com/api/test): Unexpected token < in JSON');
});
});
+16
View File
@@ -0,0 +1,16 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
export function unwrapEvaluateResult(payload) {
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
return payload.data;
}
return payload;
}
export function requireObjectEvaluateResult(payload, context) {
const result = unwrapEvaluateResult(payload);
if (!result || Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`${context}: malformed evaluate payload`);
}
return result;
}
+107 -71
View File
@@ -56,6 +56,31 @@ function sha256Hex(data) {
}
return hash.digest('hex');
}
const CRC32_TABLE = new Uint32Array(256).map((_, index) => {
let value = index;
for (let bit = 0; bit < 8; bit += 1) {
value = (value & 1) ? (0xEDB88320 ^ (value >>> 1)) : (value >>> 1);
}
return value >>> 0;
});
function crc32Hex(data) {
let crc = 0xffffffff;
for (const byte of data) {
crc = CRC32_TABLE[(crc ^ byte) & 0xff] ^ (crc >>> 8);
}
return ((crc ^ 0xffffffff) >>> 0).toString(16).padStart(8, '0');
}
function gatewayBaseUrl(tosUrl) {
const parsedUrl = new URL(tosUrl);
return `https://${parsedUrl.host}/upload/v1${parsedUrl.pathname}`;
}
function gatewayHeaders(auth, uploadHeader, userId = '') {
return {
Authorization: auth,
'X-Storage-U': encodeURIComponent(userId),
...(uploadHeader ?? {}),
};
}
function extractRegionFromHost(host) {
// e.g. "tos-cn-i-alisg.volces.com" → "cn-i-alisg"
// e.g. "tos-cn-beijing.ivolces.com" → "cn-beijing"
@@ -129,6 +154,7 @@ async function tosRequest(opts) {
method,
headers,
body: fetchBody,
signal: AbortSignal.timeout(60000),
});
const responseBody = await res.text();
const responseHeaders = {};
@@ -140,86 +166,95 @@ async function tosRequest(opts) {
function nowDatetime() {
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
}
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
async function initMultipartUpload(tosUrl, auth, credentials) {
const initUrl = `${tosUrl}?uploads`;
const datetime = nowDatetime();
// Use the pre-computed auth for INIT, as it comes from ApplyVideoUpload
const headers = {
Authorization: auth,
'x-amz-date': datetime,
'x-amz-security-token': credentials.session_token,
'content-type': 'application/octet-stream',
};
const res = await tosRequest({ method: 'POST', url: initUrl, headers });
if (res.status !== 200) {
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS credentials are valid and not expired.');
function extractUploadId(body) {
const xmlMatch = body.match(/<UploadId>([^<]+)<\/UploadId>/i);
if (xmlMatch) return xmlMatch[1];
try {
const json = JSON.parse(body);
return json?.payload?.uploadID
|| json?.payload?.uploadId
|| json?.payload?.UploadID
|| json?.payload?.UploadId
|| json?.data?.uploadid
|| json?.data?.uploadID
|| json?.data?.uploadId
|| json?.data?.UploadID
|| json?.data?.UploadId
|| json?.UploadID
|| json?.UploadId
|| json?.uploadID
|| json?.uploadId
|| null;
}
// Parse UploadId from XML: <UploadId>...</UploadId>
const match = res.body.match(/<UploadId>([^<]+)<\/UploadId>/);
if (!match) {
catch {
return null;
}
}
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
async function initMultipartUpload(tosUrl, auth, uploadHeader, userId) {
const initUrl = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=init`;
const res = await tosRequest({
method: 'POST',
url: initUrl,
headers: gatewayHeaders(auth, uploadHeader, userId),
});
if (res.status !== 200) {
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
}
const uploadId = extractUploadId(res.body);
if (!uploadId) {
throw new CommandExecutionError(`TOS init response missing UploadId: ${res.body}`);
}
return match[1];
return uploadId;
}
// ── Phase 2: Upload a single part ────────────────────────────────────────────
async function uploadPart(tosUrl, partNumber, uploadId, data, credentials, region) {
const parsedUrl = new URL(tosUrl);
parsedUrl.searchParams.set('partNumber', String(partNumber));
parsedUrl.searchParams.set('uploadId', uploadId);
const url = parsedUrl.toString();
const datetime = nowDatetime();
const headers = computeAws4Headers({
method: 'PUT',
url,
headers: { 'content-type': 'application/octet-stream' },
body: data,
credentials,
service: 'tos',
region,
datetime,
});
const res = await tosRequest({ method: 'PUT', url, headers, body: data });
if (res.status !== 200) {
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that STS2 credentials are valid and not expired.');
async function uploadPart(tosUrl, partNumber, uploadId, data, auth, uploadHeader, userId) {
const crc32 = crc32Hex(data);
const url = `${gatewayBaseUrl(tosUrl)}?uploadid=${encodeURIComponent(uploadId)}&part_number=${partNumber}&phase=transfer`;
const headers = {
...gatewayHeaders(auth, uploadHeader, userId),
'Content-CRC32': crc32,
'Content-Type': 'application/octet-stream',
'X-Use-Init-Upload-Optimize': '1',
'X-Use-Large-Local-Cache': '1',
};
const res = await tosRequest({ method: 'POST', url, headers, body: data });
let parsed;
try {
parsed = JSON.parse(res.body);
}
const etag = res.headers['etag'];
if (!etag) {
throw new CommandExecutionError(`TOS upload part ${partNumber} response missing ETag header`);
catch {
parsed = null;
}
return etag;
if (res.status !== 200 || parsed?.code !== 2000) {
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
}
return parsed?.data?.crc32 || crc32;
}
// ── Phase 3: Complete multipart upload ───────────────────────────────────────
async function completeMultipartUpload(tosUrl, uploadId, parts, credentials, region) {
const parsedUrl = new URL(tosUrl);
parsedUrl.searchParams.set('uploadId', uploadId);
const url = parsedUrl.toString();
const xmlBody = '<CompleteMultipartUpload>' +
parts
.sort((a, b) => a.partNumber - b.partNumber)
.map(p => `<Part><PartNumber>${p.partNumber}</PartNumber><ETag>${p.etag}</ETag></Part>`)
.join('') +
'</CompleteMultipartUpload>';
const datetime = nowDatetime();
const headers = computeAws4Headers({
method: 'POST',
url,
headers: { 'content-type': 'application/xml' },
body: xmlBody,
credentials,
service: 'tos',
region,
datetime,
});
async function completeMultipartUpload(tosUrl, uploadId, parts, auth, uploadHeader, userId) {
const url = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=finish&uploadid=${encodeURIComponent(uploadId)}`;
const body = parts
.sort((a, b) => a.partNumber - b.partNumber)
.map(p => `${p.partNumber}:${p.crc32}`)
.join(',');
const res = await tosRequest({
method: 'POST',
url,
headers,
body: xmlBody,
headers: gatewayHeaders(auth, uploadHeader, userId),
body,
});
if (res.status !== 200) {
let parsed;
try {
parsed = JSON.parse(res.body);
}
catch {
parsed = null;
}
if (res.status !== 200 || parsed?.code !== 2000) {
throw new CommandExecutionError(`TOS complete multipart upload failed with status ${res.status}: ${res.body}`, 'Check that all parts were uploaded successfully.');
}
return parsed?.data?.key || null;
}
let _readSyncOverride = null;
/** @internal — for testing only */
@@ -237,7 +272,7 @@ export async function tosUpload(options) {
if (fileSize === 0) {
throw new CommandExecutionError(`Video file is empty: ${filePath}`);
}
const { tos_upload_url: tosUrl, auth } = uploadInfo;
const { tos_upload_url: tosUrl, auth, upload_header: uploadHeader, user_id: userId } = uploadInfo;
const parsedTosUrl = new URL(tosUrl);
const region = extractRegionFromHost(parsedTosUrl.host);
const resumePath = getResumeFilePath(filePath);
@@ -251,7 +286,7 @@ export async function tosUpload(options) {
}
else {
// Start fresh
uploadId = await initMultipartUpload(tosUrl, auth, credentials);
uploadId = await initMultipartUpload(tosUrl, auth, uploadHeader, userId);
completedParts = [];
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
}
@@ -277,8 +312,8 @@ export async function tosUpload(options) {
if (bytesRead !== chunkSize) {
throw new CommandExecutionError(`Short read on part ${partNumber}: expected ${chunkSize} bytes, got ${bytesRead}`);
}
const etag = await uploadPart(tosUrl, partNumber, uploadId, buffer, credentials, region);
completedParts.push({ partNumber, etag });
const crc32 = await uploadPart(tosUrl, partNumber, uploadId, buffer, auth, uploadHeader, userId);
completedParts.push({ partNumber, crc32 });
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
uploadedBytes = Math.min(offset + chunkSize, fileSize);
if (onProgress)
@@ -288,8 +323,9 @@ export async function tosUpload(options) {
finally {
fs.closeSync(fd);
}
await completeMultipartUpload(tosUrl, uploadId, completedParts, credentials, region);
const completedKey = await completeMultipartUpload(tosUrl, uploadId, completedParts, auth, uploadHeader, userId);
deleteResumeState(resumePath);
return completedKey;
}
// ── Internal exports for testing ─────────────────────────────────────────────
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, };
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, extractUploadId, crc32Hex, gatewayBaseUrl, gatewayHeaders, };
+212
View File
@@ -0,0 +1,212 @@
import * as crypto from 'node:crypto';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { unwrapEvaluateResult } from './evaluate-result.js';
const AUTH_V5_URL = 'https://creator.douyin.com/web/api/media/upload/auth/v5/';
const VOD_UPLOAD_HOST = 'https://vod.bytedanceapi.com/';
const VOD_SPACE_NAME = 'aweme';
function hmacSha256(key, data) {
return crypto.createHmac('sha256', key).update(data, 'utf8').digest();
}
function sha256Hex(data) {
const hash = crypto.createHash('sha256');
if (Buffer.isBuffer(data) || data instanceof Uint8Array) {
hash.update(data);
} else {
hash.update(data ?? '', 'utf8');
}
return hash.digest('hex');
}
function nowDatetime() {
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
}
function canonicalQuery(url) {
return [...url.searchParams.entries()]
.sort(([a], [b]) => a.localeCompare(b))
.map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
.join('&');
}
function computeAws4Headers(url, credentials, options = {}) {
const parsedUrl = new URL(url);
const datetime = nowDatetime();
const date = datetime.slice(0, 8);
const method = options.method ?? 'GET';
const body = options.body ?? '';
const bodyHash = sha256Hex(body);
const headers = {
...(options.headers ?? {}),
host: parsedUrl.host,
'x-amz-content-sha256': bodyHash,
'x-amz-date': datetime,
'x-amz-security-token': credentials.session_token,
};
const sortedHeaderKeys = Object.keys(headers).sort((a, b) => a.localeCompare(b));
const canonicalHeaders = sortedHeaderKeys
.map((key) => `${key}:${String(headers[key]).trim()}`)
.join('\n') + '\n';
const signedHeaders = sortedHeaderKeys.join(';');
const canonicalRequest = [
method,
parsedUrl.pathname || '/',
canonicalQuery(parsedUrl),
canonicalHeaders,
signedHeaders,
bodyHash,
].join('\n');
const service = 'vod';
const region = 'cn-north-1';
const credentialScope = `${date}/${region}/${service}/aws4_request`;
const stringToSign = [
'AWS4-HMAC-SHA256',
datetime,
credentialScope,
sha256Hex(canonicalRequest),
].join('\n');
const kDate = hmacSha256(`AWS4${credentials.secret_access_key}`, date);
const kRegion = hmacSha256(kDate, region);
const kService = hmacSha256(kRegion, service);
const kSigning = hmacSha256(kService, 'aws4_request');
const signature = hmacSha256(kSigning, stringToSign).toString('hex');
return {
...headers,
Authorization: `AWS4-HMAC-SHA256 Credential=${credentials.access_key_id}/${credentialScope}, SignedHeaders=${signedHeaders}, Signature=${signature}`,
};
}
function extractUserIdFromSessionToken(sessionToken) {
try {
const raw = sessionToken.startsWith('STS2') ? sessionToken.slice(4) : sessionToken;
const decoded = JSON.parse(Buffer.from(raw, 'base64').toString('utf8'));
const policy = JSON.parse(decoded.PolicyString || '{}');
const condition = policy?.Statement?.[0]?.Condition;
if (typeof condition === 'string') {
const parsedCondition = JSON.parse(condition);
return parsedCondition.UserId || '';
}
} catch {
return '';
}
return '';
}
export async function getUploadAuthV5Credentials(page) {
const result = unwrapEvaluateResult(await page.evaluate(`fetch(${JSON.stringify(AUTH_V5_URL)}, { credentials: 'include' }).then(r => r.json())`));
if (!result || Array.isArray(result) || typeof result !== 'object') {
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
if (result.status_code !== 0) {
const message = result.status_msg ?? result.message ?? 'unknown error';
if (result.status_code === 401 || result.status_code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(String(message))) {
throw new AuthRequiredError('creator.douyin.com', `获取抖音上传授权失败: ${message}`);
}
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
if (!result.auth) {
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
}
let auth;
try {
auth = JSON.parse(result.auth);
} catch (error) {
throw new CommandExecutionError(`解析抖音上传授权失败: ${error instanceof Error ? error.message : String(error)}`);
}
if (!auth.AccessKeyID || !auth.SecretAccessKey || !auth.SessionToken) {
throw new CommandExecutionError('抖音上传授权缺少 AccessKeyID/SecretAccessKey/SessionToken');
}
return {
access_key_id: auth.AccessKeyID,
secret_access_key: auth.SecretAccessKey,
session_token: auth.SessionToken,
user_id: extractUserIdFromSessionToken(auth.SessionToken),
expired_time: auth.ExpiredTime,
current_time: auth.CurrentTime,
};
}
export async function applyVideoUploadInner(fileSize, credentials) {
const params = new URLSearchParams({
Action: 'ApplyUploadInner',
Version: '2020-11-19',
SpaceName: VOD_SPACE_NAME,
FileType: 'video',
IsInner: '1',
FileSize: String(fileSize),
});
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
const res = await fetch(url, { headers: computeAws4Headers(url, credentials), signal: AbortSignal.timeout(30000) });
const text = await res.text();
let payload;
try {
payload = JSON.parse(text);
} catch {
throw new CommandExecutionError(`申请抖音上传地址失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
}
const error = payload?.ResponseMetadata?.Error;
if (!res.ok || error) {
throw new CommandExecutionError(`申请抖音上传地址失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
}
const uploadNode = payload?.Result?.InnerUploadAddress?.UploadNodes?.[0];
const storeInfo = uploadNode?.StoreInfos?.[0];
const videoId = payload?.Result?.Vid || uploadNode?.Vid;
const sessionKey = uploadNode?.SessionKey ?? storeInfo?.SessionKey ?? payload?.Result?.SessionKey;
if (!uploadNode?.UploadHost || !storeInfo?.StoreUri || !storeInfo?.Auth || !videoId || !sessionKey) {
throw new CommandExecutionError(`申请抖音上传地址响应缺少必要字段: ${JSON.stringify(payload).slice(0, 500)}`);
}
return {
video_id: videoId,
tos_upload_url: `https://${uploadNode.UploadHost}/${storeInfo.StoreUri}`,
auth: storeInfo.Auth,
session_key: sessionKey,
upload_header: uploadNode.UploadHeader ?? {},
user_id: credentials.user_id ?? '',
};
}
export async function commitVideoUploadInner(uploadInfo, credentials) {
if (!uploadInfo?.session_key) {
throw new CommandExecutionError('抖音上传提交缺少 SessionKey');
}
const params = new URLSearchParams({
Action: 'CommitUploadInner',
Version: '2020-11-19',
SpaceName: VOD_SPACE_NAME,
});
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
const body = JSON.stringify({ SessionKey: uploadInfo.session_key });
const headers = computeAws4Headers(url, credentials, {
method: 'POST',
body,
headers: { 'content-type': 'application/json;charset=UTF-8' },
});
const res = await fetch(url, { method: 'POST', headers, body, signal: AbortSignal.timeout(30000) });
const text = await res.text();
let payload;
try {
payload = JSON.parse(text);
} catch {
throw new CommandExecutionError(`提交抖音上传失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
}
const error = payload?.ResponseMetadata?.Error;
if (!res.ok || error) {
throw new CommandExecutionError(`提交抖音上传失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
}
const result = payload?.Result?.Results?.[0] ?? payload?.Result ?? {};
const videoId = result.Vid ?? result.VideoId ?? result.VideoID ?? result.vid ?? uploadInfo.video_id;
if (!videoId) {
throw new CommandExecutionError(`提交抖音上传响应缺少 video id: ${JSON.stringify(payload).slice(0, 500)}`);
}
const meta = result.Meta ?? result.VideoMeta ?? {};
return {
video_id: videoId,
poster_uri: result.PosterUri ?? result.PosterURI ?? result.SnapshotUri ?? result.SnapshotURI ?? '',
width: Number(meta.Width ?? meta.width ?? 720) || 720,
height: Number(meta.Height ?? meta.height ?? 1280) || 1280,
raw: result,
};
}
+38
View File
@@ -0,0 +1,38 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getUploadAuthV5Credentials, applyVideoUploadInner } from './vod-upload.js';
describe('douyin vod upload helpers', () => {
it('parses creator upload auth v5 credentials', async () => {
const page = { evaluate: async () => ({ status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token', ExpiredTime: 123, CurrentTime: 100 }) }) };
await expect(getUploadAuthV5Credentials(page)).resolves.toEqual({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token', user_id: '', expired_time: 123, current_time: 100 });
});
it('unwraps browser bridge envelopes around upload auth payloads', async () => {
const payload = { status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token' }) };
const page = { evaluate: async () => ({ session: 'site:douyin:test', data: payload }) };
await expect(getUploadAuthV5Credentials(page)).resolves.toMatchObject({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
});
it('maps upload auth permission errors to AuthRequiredError', async () => {
const page = { evaluate: async () => ({ status_code: 401, status_msg: 'login required' }) };
await expect(getUploadAuthV5Credentials(page)).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps ApplyUploadInner response to TOS upload info', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { RequestId: 'req' }, Result: { InnerUploadAddress: { UploadNodes: [{ Vid: 'video-id', SessionKey: 'session-key', UploadHost: 'tos.example.com', StoreInfos: [{ StoreUri: 'obj/key.mp4', Auth: 'space-auth' }] }] } } }) });
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).resolves.toEqual({ video_id: 'video-id', tos_upload_url: 'https://tos.example.com/obj/key.mp4', auth: 'space-auth', session_key: 'session-key', upload_header: {}, user_id: '' });
const [url, init] = fetchSpy.mock.calls[0];
expect(String(url)).toContain('Action=ApplyUploadInner');
expect(String(url)).toContain('Version=2020-11-19');
expect(init.headers.Authorization).toContain('AWS4-HMAC-SHA256 Credential=ak/');
expect(init.headers['x-amz-security-token']).toBe('token');
fetchSpy.mockRestore();
});
it('surfaces VOD API errors with context', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { Error: { Code: 'AccessDenied', Message: 'denied' } } }) });
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).rejects.toBeInstanceOf(CommandExecutionError);
fetchSpy.mockRestore();
});
});
+137 -4
View File
@@ -1,19 +1,152 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { browserFetch } from './_shared/browser-fetch.js';
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
const CREATOR_MANAGE_URL = 'https://creator.douyin.com/creator-micro/content/manage';
const WORK_LIST_URL = '/janus/douyin/creator/pc/work_list?status=0&count=20&max_cursor=0&scene=star_atlas&device_platform=android&aid=1128';
function readAwemeId(raw) {
const value = String(raw ?? '').trim();
if (!value) {
throw new ArgumentError('douyin delete aweme_id cannot be empty');
}
if (!/^\d+$/.test(value)) {
throw new ArgumentError('douyin delete aweme_id must be a numeric id');
}
return value;
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
async function deleteViaCreatorManage(page, workId) {
await page.goto(CREATOR_MANAGE_URL);
await sleep(3000);
await sleep(3000);
const result = requireObjectEvaluateResult(await page.evaluate(`
(async () => {
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const targetId = ${JSON.stringify(String(workId))};
const textOf = (node) => (node && (node.innerText || node.textContent) || '').trim();
const normalize = (value) => String(value || '').replace(/\\s+/g, ' ').trim();
async function loadTarget() {
const res = await fetch(${JSON.stringify(WORK_LIST_URL)}, { credentials: 'include' });
const payload = await res.json();
const list = Array.isArray(payload.aweme_list) ? payload.aweme_list : [];
const matches = list
.map((entry, index) => ({ entry, index }))
.filter(({ entry }) => String(entry.aweme_id || '') === targetId || String(entry.item_id || '') === targetId);
if (matches.length === 0) {
return { ok: false, reason: 'not_found', status_code: payload.status_code, count: list.length };
}
if (matches.length !== 1) {
return { ok: false, reason: 'target_not_unique', count: matches.length };
}
const { entry: item, index } = matches[0];
const title = normalize(item.desc || item.caption || item.title || item.item_title || '');
return { ok: true, item, index, listCount: list.length, title };
}
function visibleWorkCards() {
const candidates = Array.from(document.querySelectorAll('[class*="video-card"]'))
.filter((element) => {
const text = normalize(textOf(element));
return text.includes('删除作品') && text.includes('继续编辑');
});
return candidates.filter((candidate) => !candidates.some((other) => other !== candidate && other.contains(candidate)));
}
const target = await loadTarget();
if (!target.ok) return target;
const allTab = Array.from(document.querySelectorAll('button,[role="button"],span,div'))
.find((element) => /^全部作品$/.test(normalize(textOf(element))));
allTab?.click();
await sleep(1000);
for (let attempt = 0; attempt < 20; attempt += 1) {
const cards = visibleWorkCards();
if (cards.length >= target.listCount && cards[target.index]) {
const card = cards[target.index];
const deleteButton = Array.from(card.querySelectorAll('button,[role="button"],span,div'))
.find((element) => /^删除作品$/.test(normalize(textOf(element))));
if (!deleteButton) return { ok: false, reason: 'delete_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, cardCount: cards.length };
deleteButton.click();
await sleep(800);
const confirmButton = Array.from(document.querySelectorAll('button,[role="button"]'))
.find((element) => ['确定', '确认', '删除'].includes(normalize(textOf(element))));
if (!confirmButton) return { ok: false, reason: 'confirm_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
confirmButton.click();
for (let wait = 0; wait < 20; wait += 1) {
await sleep(500);
const after = await loadTarget();
if (!after.ok && after.reason === 'not_found') {
return { ok: true, aweme_id: target.item.aweme_id, item_id: target.item.item_id, title: target.title };
}
}
return { ok: false, reason: 'delete_not_confirmed', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
}
await sleep(500);
}
return { ok: false, reason: 'card_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, listCount: target.listCount };
})()
`), '抖音后台管理删除响应异常');
if (!result?.ok) {
throw new CommandExecutionError(`抖音后台管理删除失败: ${JSON.stringify(result)}`);
}
return result;
}
async function findWorkListItem(page, workId) {
const data = await browserFetch(page, 'GET', `https://creator.douyin.com${WORK_LIST_URL}`, { timeoutMs: 8000 });
const list = data.data?.work_list ?? data.aweme_list ?? data.work_list ?? [];
if (!Array.isArray(list)) {
throw new CommandExecutionError('抖音作品列表响应缺少 work_list/aweme_list');
}
return list.find((entry) => String(entry.aweme_id || '') === workId || String(entry.item_id || '') === workId) || null;
}
cli({
site: 'douyin',
name: 'delete',
access: 'write',
description: '删除作品',
description: '删除作品(优先使用创作者后台作品管理;找不到时回退到旧删除接口)',
domain: 'creator.douyin.com',
strategy: Strategy.COOKIE,
siteSession: 'persistent',
args: [
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID' },
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID / item_id' },
],
columns: ['status'],
func: async (page, kwargs) => {
const awemeId = readAwemeId(kwargs.aweme_id);
try {
const deleted = await deleteViaCreatorManage(page, awemeId);
return [{ status: `✅ 已通过后台管理删除 ${deleted.aweme_id || awemeId}` }];
} catch (fallbackError) {
const fallbackMessage = fallbackError instanceof Error ? fallbackError.message : String(fallbackError);
if (!fallbackMessage.includes('"reason":"not_found"')) {
throw fallbackError;
}
}
const before = await findWorkListItem(page, awemeId);
if (!before) {
throw new CommandExecutionError(`抖音作品 ${awemeId} 未在作品列表中找到,未执行删除`);
}
const url = 'https://creator.douyin.com/web/api/media/aweme/delete/?aid=1128';
await browserFetch(page, 'POST', url, { body: { aweme_id: kwargs.aweme_id } });
return [{ status: `✅ 已删除 ${kwargs.aweme_id}` }];
await browserFetch(page, 'POST', url, { body: { aweme_id: awemeId }, timeoutMs: 8000 });
const deadline = Date.now() + 10_000;
while (Date.now() < deadline) {
await sleep(500);
const after = await findWorkListItem(page, awemeId);
if (!after) {
return [{ status: `✅ 已删除 ${awemeId}` }];
}
}
throw new CommandExecutionError(`抖音作品 ${awemeId} 删除后仍在作品列表中,删除未确认`);
},
});
+90 -1
View File
@@ -1,11 +1,100 @@
import { describe, expect, it } from 'vitest';
import { readFileSync } from 'node:fs';
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
const mocks = vi.hoisted(() => ({
browserFetch: vi.fn(),
}));
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
import './delete.js';
function makePage({ evaluateResult, listBefore = [], listAfter = [] } = {}) {
let listCalls = 0;
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'GET' && String(url).includes('/work_list?')) {
listCalls += 1;
return { aweme_list: listCalls === 1 ? listBefore : listAfter };
}
return { status_code: 0 };
});
return {
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult ?? { ok: false, reason: 'not_found' }),
wait: vi.fn().mockResolvedValue(undefined),
};
}
describe('douyin delete registration', () => {
const command = getRegistry().get('douyin/delete');
beforeEach(() => {
vi.clearAllMocks();
vi.useFakeTimers();
});
afterEach(() => {
vi.useRealTimers();
});
it('registers the delete command', () => {
const registry = getRegistry();
const values = [...registry.values()];
const cmd = values.find(c => c.site === 'douyin' && c.name === 'delete');
expect(cmd).toBeDefined();
});
it('uses work_list id/index matching instead of title matching for fallback deletion', () => {
const source = readFileSync(new URL('./delete.js', import.meta.url), 'utf8');
expect(source).toContain('target_not_unique');
expect(source).toContain("String(entry.aweme_id || '') === targetId");
expect(source).toContain('cards[target.index]');
expect(source).not.toContain('text.includes(target.title)');
});
it('validates aweme_id before navigation', async () => {
const page = makePage();
await expect(command.func(page, { aweme_id: '' })).rejects.toBeInstanceOf(ArgumentError);
await expect(command.func(page, { aweme_id: 'abc' })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('does not treat a missing work as successful delete', async () => {
const page = makePage({ listBefore: [], listAfter: [] });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
});
it('unwraps Browser Bridge envelopes around creator manage delete results', async () => {
const page = makePage({ evaluateResult: { session: 'site:douyin:test', data: { ok: true, aweme_id: '123' } } });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已通过后台管理删除 123' }]);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
expect(mocks.browserFetch).not.toHaveBeenCalled();
});
it('throws typed on malformed creator manage delete result', async () => {
const page = makePage({ evaluateResult: 'bad-shape' });
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
await vi.advanceTimersByTimeAsync(7000);
await assertion;
expect(mocks.browserFetch).not.toHaveBeenCalled();
});
it('returns success only after fallback delete postcondition removes the target', async () => {
const page = makePage({
listBefore: [{ aweme_id: '123' }],
listAfter: [],
});
const promise = command.func(page, { aweme_id: '123' });
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已删除 123' }]);
await vi.advanceTimersByTimeAsync(8000);
await assertion;
});
});
+170
View File
@@ -0,0 +1,170 @@
import * as fs from 'node:fs';
import * as os from 'node:os';
import * as path from 'node:path';
import { beforeEach, describe, expect, it, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
browserFetch: vi.fn(),
getUploadAuthV5Credentials: vi.fn(),
applyVideoUploadInner: vi.fn(),
commitVideoUploadInner: vi.fn(),
tosUpload: vi.fn(),
pollTranscode: vi.fn(),
imagexUpload: vi.fn(),
}));
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
vi.mock('./_shared/vod-upload.js', () => ({
getUploadAuthV5Credentials: mocks.getUploadAuthV5Credentials,
applyVideoUploadInner: mocks.applyVideoUploadInner,
commitVideoUploadInner: mocks.commitVideoUploadInner,
}));
vi.mock('./_shared/tos-upload.js', () => ({ tosUpload: mocks.tosUpload }));
vi.mock('./_shared/transcode.js', () => ({ pollTranscode: mocks.pollTranscode }));
vi.mock('./_shared/imagex-upload.js', () => ({ imagexUpload: mocks.imagexUpload }));
describe('douyin publish upload identifier handling', () => {
beforeEach(() => {
vi.resetModules();
vi.clearAllMocks();
mocks.getUploadAuthV5Credentials.mockResolvedValue({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
mocks.applyVideoUploadInner.mockResolvedValue({ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' });
mocks.commitVideoUploadInner.mockResolvedValue({ video_id: 'canonical-video-id', poster_uri: 'poster-uri' });
mocks.tosUpload.mockResolvedValue('object-key-returned-by-complete');
mocks.pollTranscode.mockResolvedValue({ width: 720, height: 1280, poster_uri: 'poster-uri' });
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { aweme_id: 'aweme-1' };
return { status_code: 0 };
});
});
it('uses CommitUploadInner Vid for create_v2, not the completed TOS object key', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-id-'));
const video = path.join(tmpDir, 'video.mp4');
fs.writeFileSync(video, Buffer.from('fake-video'));
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func({}, {
video,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
});
expect(mocks.commitVideoUploadInner).toHaveBeenCalledWith(
{ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' },
{ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' },
);
expect(mocks.pollTranscode).not.toHaveBeenCalled();
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
expect(createCall?.[3]?.body.item.common.video_id).toBe('canonical-video-id');
expect(createCall?.[3]?.body.item.common.video_id).not.toBe('object-key-returned-by-complete');
});
it('continues to create_v2 when the legacy fast detect API returns an empty response', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-safety-'));
const video = path.join(tmpDir, 'video.mp4');
fs.writeFileSync(video, Buffer.from('fake-video'));
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/pre_check')) {
throw new Error('Empty response from Douyin API (POST https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check)');
}
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/poll')) return { status: -1, has_done: true, detect_result: { reason_code: 0 }, detect_list: [] };
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { item_id: 'item-1' };
return { status_code: 0 };
});
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func({}, {
video,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
visibility: 'public',
caption: 'caption',
no_safety_check: false,
});
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/post_assistant/fast_detect/pre_check'))).toBe(true);
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/aweme/create_v2/'))).toBe(true);
});
it('unwraps Browser Bridge envelopes around cover ImageX evaluate results', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-'));
const video = path.join(tmpDir, 'video.mp4');
const cover = path.join(tmpDir, 'cover.jpg');
fs.writeFileSync(video, Buffer.from('fake-video'));
fs.writeFileSync(cover, Buffer.from('fake-cover'));
mocks.imagexUpload.mockResolvedValue('cover-store-uri');
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce({
session: 'site:douyin:test',
data: { Result: { UploadAddress: { StoreInfos: [{ UploadHost: 'imagex.example.com', StoreUri: 'cover/key.jpg' }] } } },
})
.mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: {} } }),
};
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await cmd.func(page, {
video,
cover,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
});
expect(mocks.imagexUpload).toHaveBeenCalledWith(cover, {
upload_url: 'https://imagex.example.com/cover/key.jpg',
store_uri: 'cover/key.jpg',
});
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
expect(createCall?.[3]?.body.item.cover.poster).toBe('cover-store-uri');
});
it('throws typed when cover ImageX apply returns the wrong shape', async () => {
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-bad-'));
const video = path.join(tmpDir, 'video.mp4');
const cover = path.join(tmpDir, 'cover.jpg');
fs.writeFileSync(video, Buffer.from('fake-video'));
fs.writeFileSync(cover, Buffer.from('fake-cover'));
const page = { evaluate: vi.fn().mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: { UploadAddress: { StoreInfos: [] } } } }) };
const { getRegistry } = await import('@jackwener/opencli/registry');
getRegistry().delete('douyin/publish');
await import('./publish.js');
const cmd = getRegistry().get('douyin/publish');
if (!cmd) throw new Error('douyin publish command not registered');
await expect(cmd.func(page, {
video,
cover,
title: 'OpenCLI自测',
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
caption: '',
visibility: 'private',
no_safety_check: true,
})).rejects.toThrow('UploadHost/StoreUri');
expect(mocks.imagexUpload).not.toHaveBeenCalled();
});
});
+88 -42
View File
@@ -2,7 +2,7 @@
* Douyin publish — 8-phase pipeline for scheduling video posts.
*
* Phases:
* 1. STS2 credentials
* 1. upload auth v5 credentials
* 2. Apply TOS upload URL
* 3. TOS multipart upload
* 4. Cover upload (optional, via ImageX)
@@ -15,11 +15,11 @@ import * as fs from 'node:fs';
import * as path from 'node:path';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import { getSts2Credentials } from './_shared/sts2.js';
import { getUploadAuthV5Credentials, applyVideoUploadInner, commitVideoUploadInner } from './_shared/vod-upload.js';
import { tosUpload } from './_shared/tos-upload.js';
import { imagexUpload } from './_shared/imagex-upload.js';
import { pollTranscode } from './_shared/transcode.js';
import { browserFetch } from './_shared/browser-fetch.js';
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
import { generateCreationId } from './_shared/creation-id.js';
import { validateTiming, toUnixSeconds } from './_shared/timing.js';
import { parseTextExtra, extractHashtagNames } from './_shared/text-extra.js';
@@ -54,6 +54,36 @@ const DEFAULT_COVER_TOOLS_INFO = JSON.stringify({
initial_cover_uri: '',
cut_coordinate: '',
});
function isFastDetectRetryable(error) {
const message = error instanceof Error ? error.message : String(error);
return message.includes('post_assistant/fast_detect') && (message.includes('Empty response') || message.includes('404') || message.includes('Not Found') || message.includes('Timeout') || message.includes('timed out') || message.includes('Failed to fetch'));
}
function sleep(ms) {
return new Promise((resolve) => setTimeout(resolve, ms));
}
function throwIfImagexError(action, payload) {
const error = payload?.ResponseMetadata?.Error ?? payload?.Error;
if (error) {
throw new CommandExecutionError(`${action}失败: ${JSON.stringify(error)}`);
}
}
async function tryFastDetectFetch(page, method, url, options) {
let lastError;
for (let attempt = 1; attempt <= 3; attempt += 1) {
try {
return { ok: true, value: await browserFetch(page, method, url, options) };
} catch (error) {
if (!isFastDetectRetryable(error)) {
throw error;
}
lastError = error;
if (attempt < 3) {
await sleep(500 * attempt);
}
}
}
return { ok: false, error: lastError };
}
cli({
site: 'douyin',
name: 'publish',
@@ -106,19 +136,13 @@ cli({
throw new ArgumentError(`封面文件不存在: ${path.resolve(coverPath)}`);
}
}
// ── Phase 1: STS2 credentials ───────────────────────────────────────
const credentials = await getSts2Credentials(page);
// ── Phase 1: upload credentials ────────────────────────────────────
const credentials = await getUploadAuthV5Credentials(page);
// ── Phase 2: Apply TOS upload URL ───────────────────────────────────
const vodUrl = `https://vod.bytedanceapi.com/?Action=ApplyVideoUpload&ServiceId=1128&Version=2021-01-01&FileType=video&FileSize=${fileSize}`;
const vodJs = `fetch(${JSON.stringify(vodUrl)}, { credentials: 'include' }).then(r => r.json())`;
const vodRes = (await page.evaluate(vodJs));
const { VideoId: videoId, UploadHosts, StoreInfos } = vodRes.Result.UploadAddress;
const tosUrl = `https://${UploadHosts[0]}/${StoreInfos[0].StoreUri}`;
const tosUploadInfo = {
tos_upload_url: tosUrl,
auth: StoreInfos[0].Auth,
video_id: videoId,
};
const tosUploadInfo = await applyVideoUploadInner(fileSize, credentials);
let coverUri = '';
let coverWidth = 720;
let coverHeight = 1280;
// ── Phase 3: TOS upload ─────────────────────────────────────────────
await tosUpload({
filePath: videoPath,
@@ -130,22 +154,32 @@ cli({
},
});
process.stderr.write('\n');
process.stderr.write(' 提交上传...\n');
const committedVideo = await commitVideoUploadInner(tosUploadInfo, credentials);
const videoId = committedVideo.video_id;
process.stderr.write(` 上传已提交: ${videoId}\n`);
coverWidth = committedVideo.width || coverWidth;
coverHeight = committedVideo.height || coverHeight;
if (!coverUri && committedVideo.poster_uri) {
coverUri = committedVideo.poster_uri;
}
// ── Phase 4: Cover upload (optional) ────────────────────────────────
let coverUri = '';
let coverWidth = 720;
let coverHeight = 1280;
if (kwargs.cover) {
const resolvedCoverPath = path.resolve(kwargs.cover);
// 4A: Apply ImageX upload
const applyUrl = `${IMAGEX_BASE}/?Action=ApplyImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01&UploadNum=1`;
const applyJs = `fetch(${JSON.stringify(applyUrl)}, { credentials: 'include' }).then(r => r.json())`;
const applyRes = (await page.evaluate(applyJs));
const { StoreInfos: imgStoreInfos } = applyRes.Result.UploadAddress;
const imgUploadUrl = `https://${imgStoreInfos[0].UploadHost}/${imgStoreInfos[0].StoreUri}`;
const applyRes = requireObjectEvaluateResult(await page.evaluate(applyJs), '抖音封面申请上传地址响应异常');
throwIfImagexError('抖音封面申请上传地址', applyRes);
const imgStoreInfo = applyRes.Result?.UploadAddress?.StoreInfos?.[0];
if (!imgStoreInfo?.UploadHost || !imgStoreInfo?.StoreUri) {
throw new CommandExecutionError(`抖音封面申请上传地址响应缺少 UploadHost/StoreUri: ${JSON.stringify(applyRes).slice(0, 500)}`);
}
const imgUploadUrl = `https://${imgStoreInfo.UploadHost}/${imgStoreInfo.StoreUri}`;
// 4B: Upload image
const coverStoreUri = await imagexUpload(resolvedCoverPath, {
upload_url: imgUploadUrl,
store_uri: imgStoreInfos[0].StoreUri,
store_uri: imgStoreInfo.StoreUri,
});
// 4C: Commit ImageX upload
const commitUrl = `${IMAGEX_BASE}/?Action=CommitImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01`;
@@ -158,19 +192,13 @@ cli({
body: ${JSON.stringify(commitBody)}
}).then(r => r.json())
`;
await page.evaluate(commitJs);
const commitRes = requireObjectEvaluateResult(await page.evaluate(commitJs), '抖音封面提交上传响应异常');
throwIfImagexError('抖音封面提交上传', commitRes);
coverUri = coverStoreUri;
}
// ── Phase 5: Enable video ───────────────────────────────────────────
const enableUrl = `https://creator.douyin.com/web/api/media/video/enable/?video_id=${videoId}&aid=1128`;
await browserFetch(page, 'GET', enableUrl);
// ── Phase 6: Poll transcode ─────────────────────────────────────────
const transResult = await pollTranscode(page, videoId);
coverWidth = transResult.width;
coverHeight = transResult.height;
if (!coverUri) {
coverUri = transResult.poster_uri;
}
// The gateway upload flow returns a committed VOD upload result; the legacy
// enable/transend endpoints can hang for that flow, so create_v2 consumes
// the committed video_id and poster metadata directly.
// ── Phase 7: Content safety check ───────────────────────────────────
if (!kwargs.no_safety_check) {
const safetyUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check';
@@ -179,25 +207,42 @@ cli({
title,
desc: caption,
};
await browserFetch(page, 'POST', safetyUrl, { body: safetyBody });
const preCheck = await tryFastDetectFetch(page, 'POST', safetyUrl, { body: safetyBody });
if (!preCheck.ok) {
process.stderr.write(' 内容安全预检接口无响应,继续轮询检测结果。\n');
}
const pollUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/poll';
const deadline = Date.now() + 30_000;
let safetyPassed = false;
let pollUnavailableCount = 0;
while (Date.now() < deadline) {
const pollRes = (await browserFetch(page, 'POST', pollUrl, {
body: safetyBody,
}));
if (pollRes.status === 0) {
const poll = await tryFastDetectFetch(page, 'POST', pollUrl, { body: safetyBody });
if (!poll.ok) {
pollUnavailableCount += 1;
if (!preCheck.ok && pollUnavailableCount >= 3) {
break;
}
await sleep(2000);
continue;
}
pollUnavailableCount = 0;
const pollRes = poll.value;
if (pollRes.status === 0 || (pollRes.has_done === true && pollRes.detect_result?.reason_code === 0 && (pollRes.detect_list?.length ?? 0) === 0)) {
safetyPassed = true;
break;
}
if (pollRes.status === 1) {
throw new CommandExecutionError('内容安全检测不通过,请修改后重试', '使用 --no_safety_check 跳过');
}
await new Promise((r) => setTimeout(r, 2000));
await sleep(2000);
}
if (!safetyPassed) {
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '使用 --no_safety_check 跳过');
if (!preCheck.ok && pollUnavailableCount >= 3) {
process.stderr.write(' 内容安全预检持续无响应,跳过本地预检,交由 create_v2 后的平台审核。\n');
}
else {
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '如确认要跳过本地预检,可使用 --no_safety_check;提交后仍会走抖音平台审核');
}
}
}
// ── Phase 8: create_v2 publish ──────────────────────────────────────
@@ -266,12 +311,13 @@ cli({
},
};
const publishUrl = `https://creator.douyin.com/web/api/media/aweme/create_v2/?read_aid=2906&${DEVICE_PARAMS}`;
process.stderr.write(' 创建定时发布...\n');
const publishRes = (await browserFetch(page, 'POST', publishUrl, {
body: publishBody,
}));
const awemeId = publishRes.aweme_id;
const awemeId = publishRes.aweme_id ?? publishRes.item_id;
if (!awemeId) {
throw new CommandExecutionError(`发布成功但未返回 aweme_id: ${JSON.stringify(publishRes)}`);
throw new CommandExecutionError(`发布成功但未返回 aweme_id/item_id: ${JSON.stringify(publishRes)}`);
}
const url = `https://www.douyin.com/video/${awemeId}`;
const publishTimeStr = new Date(timingTs * 1000).toLocaleString('zh-CN', {
+9 -2
View File
@@ -1,4 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { fetchDouyinComments, fetchDouyinUserVideos } from './_shared/public-api.js';
export const MAX_USER_VIDEOS_LIMIT = 20;
export const USER_VIDEO_COMMENT_CONCURRENCY = 4;
@@ -27,8 +28,11 @@ async function fetchTopComments(page, awemeId, count) {
try {
return await fetchDouyinComments(page, awemeId, count);
}
catch {
return [];
catch (error) {
if (error instanceof CliError) {
throw error;
}
throw new CommandExecutionError(`Failed to fetch Douyin comments for video ${awemeId}: ${error instanceof Error ? error.message : String(error)}`);
}
}
cli({
@@ -53,6 +57,9 @@ cli({
await page.goto(`https://www.douyin.com/user/${secUid}`);
await page.wait(3);
const awemeList = (await fetchDouyinUserVideos(page, secUid, limit)).slice(0, limit);
if (awemeList.length === 0) {
throw new EmptyResultError('douyin user-videos', `No videos were returned for sec_uid ${secUid}. Confirm the user exists and the Douyin session is valid.`);
}
const videos = withComments
? await mapInBatches(awemeList, USER_VIDEO_COMMENT_CONCURRENCY, async (video) => ({
...video,
+43
View File
@@ -8,6 +8,7 @@ vi.mock('./_shared/public-api.js', () => ({
fetchDouyinComments: fetchDouyinCommentsMock,
}));
import { getRegistry } from '@jackwener/opencli/registry';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { DEFAULT_COMMENT_LIMIT, MAX_USER_VIDEOS_LIMIT, normalizeCommentLimit, normalizeUserVideosLimit } from './user-videos.js';
describe('douyin user-videos', () => {
beforeEach(() => {
@@ -105,4 +106,46 @@ describe('douyin user-videos', () => {
},
]);
});
it('throws EmptyResultError when the user videos API returns no rows', async () => {
const command = [...getRegistry().values()].find((cmd) => cmd.site === 'douyin' && cmd.name === 'user-videos');
expect(command?.func).toBeDefined();
if (!command?.func)
throw new Error('douyin user-videos command not registered');
fetchDouyinUserVideosMock.mockResolvedValueOnce([]);
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
};
await expect(command.func(page, {
sec_uid: 'MS4w-empty',
limit: 3,
with_comments: true,
comment_limit: 5,
})).rejects.toBeInstanceOf(EmptyResultError);
});
it('surfaces comment enrichment failures instead of returning empty comments', async () => {
const command = [...getRegistry().values()].find((cmd) => cmd.site === 'douyin' && cmd.name === 'user-videos');
expect(command?.func).toBeDefined();
if (!command?.func)
throw new Error('douyin user-videos command not registered');
fetchDouyinUserVideosMock.mockResolvedValueOnce([
{
aweme_id: '3',
desc: 'comment failure',
video: { duration: 2000, play_addr: { url_list: ['https://example.com/fail.mp4'] } },
statistics: { digg_count: 1 },
},
]);
fetchDouyinCommentsMock.mockRejectedValueOnce(new Error('comment API down'));
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
};
await expect(command.func(page, {
sec_uid: 'MS4w-test',
limit: 3,
with_comments: true,
comment_limit: 5,
})).rejects.toBeInstanceOf(CommandExecutionError);
});
});
+131
View File
@@ -0,0 +1,131 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError } from '@jackwener/opencli/errors';
import {
emptySearchResults,
requireBoundedInteger,
requireNonNegativeInteger,
requireRows,
requireSearchQuery,
runBrowserStep,
toHttpsUrl,
} from '../_shared/search-adapter.js';
function decodeDdgUrl(href) {
if (!href) return '';
try {
const url = new URL(href, 'https://duckduckgo.com');
const uddg = url.searchParams.get('uddg');
return toHttpsUrl(uddg || href, 'https://duckduckgo.com');
} catch {
return '';
}
}
function buildExtractFn(limit) {
return 'function(doc){' +
'var r=[];var seen={};var items=doc.querySelectorAll(".result");' +
'for(var i=0;i<items.length;i++){' +
'if(r.length>=' + limit + ')break;' +
'var el=items[i];var te=el.querySelector(".result__a");' +
'var se=el.querySelector(".result__snippet");' +
'var ue=el.querySelector(".result__url");' +
'var ie=el.querySelector(".result__icon__img");' +
'var cls=el.className||"";var rt="web";' +
'if(cls.indexOf("result--ad")!==-1||cls.indexOf("result--ads")!==-1||cls.indexOf("badge--ad")!==-1)continue;' +
'if(!te)continue;' +
'var t=(te.textContent||"").trim();' +
'var h=te.getAttribute("href")||"";' +
'var sn=se?(se.textContent||"").trim():"";' +
'var du=ue?(ue.textContent||"").trim():"";' +
'var ic=ie?(ie.getAttribute("src")||""):"";' +
'if(cls.indexOf("news-result")!==-1)rt="news";' +
'else if(cls.indexOf("video-result")!==-1)rt="video";' +
'else if(cls.indexOf("image-result")!==-1)rt="image";' +
'if(!t||!h||seen[h])continue;seen[h]=true;' +
'r.push([t,h,sn,du,ic,rt]);' +
'}return r;}';
}
function buildExtractorJs(limit) {
return '(' + buildExtractFn(limit) + '(document))';
}
function buildPaginateJs(limit, keyword, offset, region) {
var params = 'q=' + encodeURIComponent(keyword) + '&s=' + offset + '&v=l&o=json';
if (region) params += '&kl=' + encodeURIComponent(region);
return (
'new Promise(function($r){' +
'var x=new XMLHttpRequest();' +
'x.open("POST","/html/",true);' +
'x.setRequestHeader("Content-Type","application/x-www-form-urlencoded");' +
'x.onload=function(){' +
'try{var d=new DOMParser().parseFromString(x.responseText,"text/html");' +
'$r(' + buildExtractFn(limit) + '(d));' +
'}catch(e){$r({error:"parse",message:String(e&&e.message||e)})}' +
'};' +
'x.onerror=function(){$r({error:"network"})};' +
'x.send("' + params + '");' +
'})'
);
}
const command = cli({
site: 'duckduckgo',
name: 'search',
access: 'read',
description: 'Search DuckDuckGo',
domain: 'html.duckduckgo.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Search query' },
{ name: 'limit', type: 'int', default: 10, help: 'Number of results per page (1-10). For multi-page, use --offset' },
{ name: 'offset', type: 'int', default: 0, help: 'Result offset for pagination (0, 10, 20...). Uses XHR POST internally' },
{ name: 'region', help: 'Region code (e.g. jp-jp, us-en, cn-zh). Default: all regions' },
{ name: 'time', help: 'Time range: d (day), w (week), m (month), y (year)' },
],
columns: ['rank', 'title', 'url', 'snippet', 'displayUrl', 'icon', 'resultType'],
func: async (page, kwargs) => {
const limit = requireBoundedInteger(kwargs.limit, 10, 1, 10, '--limit');
const keyword = requireSearchQuery(kwargs.keyword);
const offset = requireNonNegativeInteger(kwargs.offset, 0, '--offset');
if (offset % 10 !== 0) {
throw new ArgumentError('--offset must be a multiple of 10 for DuckDuckGo HTML pagination');
}
if (kwargs.time && !/^(d|w|m|y)$/.test(String(kwargs.time))) {
throw new ArgumentError('--time must be one of d, w, m, or y');
}
let url = `https://html.duckduckgo.com/html/?q=${encodeURIComponent(keyword)}`;
if (kwargs.region) url += `&kl=${encodeURIComponent(String(kwargs.region))}`;
if (kwargs.time) url += `&df=${encodeURIComponent(String(kwargs.time))}`;
await runBrowserStep('duckduckgo search navigation', () => page.goto(url));
try {
await page.wait({ selector: '.result', timeout: 8 });
} catch {
await page.wait(3).catch(function() {});
}
var raw;
if (offset === 0) {
raw = await runBrowserStep('duckduckgo search extraction', () => page.evaluate(buildExtractorJs(limit)));
} else {
raw = await runBrowserStep('duckduckgo search pagination extraction', () => page.evaluate(buildPaginateJs(limit, keyword, offset, kwargs.region)));
}
const rows = requireRows(raw, 'duckduckgo search');
if (rows.length === 0) {
throw emptySearchResults('DuckDuckGo', keyword);
}
return rows.map(function(r, index) {
return {
rank: index + 1 + offset,
title: r[0],
url: decodeDdgUrl(r[1]),
snippet: r[2],
displayUrl: r[3],
icon: r[4],
resultType: r[5],
};
}).filter((row) => row.url);
},
});
export const __test__ = { command };
+128
View File
@@ -0,0 +1,128 @@
import { describe, it, expect, vi } from 'vitest';
import { JSDOM } from 'jsdom';
const { __test__ } = await import('./search.js');
const command = __test__.command;
function createPageMock(evaluateResult = []) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('duckduckgo search', () => {
it('should register as a valid command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('duckduckgo');
expect(command.name).toBe('search');
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.strategy).toBe('public');
expect(command.domain).toBe('html.duckduckgo.com');
});
it('should define keyword positional arg', () => {
const kwArg = command.args.find(a => a.name === 'keyword');
expect(kwArg).toBeDefined();
expect(kwArg.positional).toBe(true);
expect(kwArg.required).toBe(true);
});
it('should define limit arg with default 10', () => {
const limitArg = command.args.find(a => a.name === 'limit');
expect(limitArg).toBeDefined();
expect(limitArg.type).toBe('int');
expect(limitArg.default).toBe(10);
});
it('should define columns for output', () => {
expect(command.columns).toContain('rank');
expect(command.columns).toContain('title');
expect(command.columns).toContain('url');
expect(command.columns).toContain('snippet');
expect(command.columns).toContain('displayUrl');
expect(command.columns).toContain('icon');
expect(command.columns).toContain('resultType');
});
it('rejects empty query and out-of-range pagination before navigation', async () => {
const page = createPageMock();
await expect(command.func(page, { keyword: ' ', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 11 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func(page, { keyword: 'opencli', limit: 5, offset: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
expect(page.goto).not.toHaveBeenCalled();
});
it('decodes DuckDuckGo redirect URLs and assigns listing rank', async () => {
const page = createPageMock([
[
'OpenCLI',
'/l/?uddg=https%3A%2F%2Fgithub.com%2Fjackwener%2FOpenCLI',
'CLI browser tooling',
'github.com/jackwener/OpenCLI',
'',
'web',
],
]);
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).resolves.toEqual([{
rank: 1,
title: 'OpenCLI',
url: 'https://github.com/jackwener/OpenCLI',
snippet: 'CLI browser tooling',
displayUrl: 'github.com/jackwener/OpenCLI',
icon: '',
resultType: 'web',
}]);
});
it('executes the DOM extractor, filters ads, and returns canonical rows', async () => {
const dom = new JSDOM(`
<div class="result result--ad">
<a class="result__a" href="https://ads.example/">Sponsored result</a>
</div>
<div class="result">
<a class="result__a" href="/l/?uddg=https%3A%2F%2Fexample.com%2Farticle">Organic result</a>
<a class="result__snippet">Organic snippet</a>
<span class="result__url">example.com/article</span>
<img class="result__icon__img" src="https://icons.duckduckgo.com/ip3/example.com.ico">
</div>
`);
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn(async (source) => Function('document', `return ${source};`)(dom.window.document)),
};
await expect(command.func(page, { keyword: 'opencli', limit: 5 })).resolves.toEqual([{
rank: 1,
title: 'Organic result',
url: 'https://example.com/article',
snippet: 'Organic snippet',
displayUrl: 'example.com/article',
icon: 'https://icons.duckduckgo.com/ip3/example.com.ico',
resultType: 'web',
}]);
});
it('unwraps browser envelopes for paginated extraction', async () => {
const page = createPageMock({ session: 'site:duckduckgo', data: [
['Result', 'https://example.com/', 'snippet', 'example.com', '', 'web'],
] });
const result = await command.func(page, { keyword: 'opencli', limit: 1, offset: 10 });
expect(result[0]).toMatchObject({ rank: 11, url: 'https://example.com/' });
});
it('fails typed instead of returning [] for malformed extraction payloads', async () => {
const page = createPageMock({ rows: [] });
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('payload shape'),
});
});
});
+45
View File
@@ -0,0 +1,45 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { requireBoundedInteger, requireSearchQuery } from '../_shared/search-adapter.js';
const command = cli({
site: 'duckduckgo',
name: 'suggest',
access: 'read',
description: 'DuckDuckGo search suggestions',
domain: 'duckduckgo.com',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'keyword', positional: true, required: true, help: 'Search query prefix' },
{ name: 'limit', type: 'int', default: 8, help: 'Max number of suggestions' },
],
columns: ['phrase'],
func: async (kwargs) => {
const limit = requireBoundedInteger(kwargs.limit, 8, 1, 20, '--limit');
const keyword = encodeURIComponent(requireSearchQuery(kwargs.keyword));
const url = `https://duckduckgo.com/ac/?q=${keyword}&type=list`;
let resp;
try {
resp = await fetch(url);
} catch (err) {
throw new CommandExecutionError(`DuckDuckGo suggest request failed: ${err instanceof Error ? err.message : String(err)}`);
}
if (!resp.ok) {
throw new CommandExecutionError(`DuckDuckGo suggest returned HTTP ${resp.status}`);
}
let data;
try {
data = await resp.json();
} catch (err) {
throw new CommandExecutionError(`DuckDuckGo suggest returned malformed JSON: ${err?.message ?? err}`);
}
const phrases = Array.isArray(data) && data.length > 1 && Array.isArray(data[1]) ? data[1] : [];
return phrases
.filter((phrase) => typeof phrase === 'string' && phrase.trim())
.slice(0, limit)
.map(function(p) { return { phrase: p }; });
},
});
export const __test__ = { command };
+66
View File
@@ -0,0 +1,66 @@
import { afterEach, describe, it, expect, vi } from 'vitest';
const { __test__ } = await import('./suggest.js');
const command = __test__.command;
afterEach(() => {
vi.restoreAllMocks();
});
describe('duckduckgo suggest', () => {
it('should register as a valid command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('duckduckgo');
expect(command.name).toBe('suggest');
expect(command.access).toBe('read');
expect(command.browser).toBe(false);
expect(command.strategy).toBe('public');
});
it('should define keyword positional arg', () => {
const kwArg = command.args.find(a => a.name === 'keyword');
expect(kwArg).toBeDefined();
expect(kwArg.positional).toBe(true);
expect(kwArg.required).toBe(true);
});
it('should define limit arg with default 8', () => {
const limitArg = command.args.find(a => a.name === 'limit');
expect(limitArg).toBeDefined();
expect(limitArg.default).toBe(8);
});
it('should define phrase column', () => {
expect(command.columns).toEqual(['phrase']);
});
it('rejects empty query and invalid limit before fetch', async () => {
const fetchSpy = vi.spyOn(globalThis, 'fetch');
await expect(command.func({ keyword: '', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
await expect(command.func({ keyword: 'opencli', limit: 21 })).rejects.toMatchObject({ code: 'ARGUMENT' });
expect(fetchSpy).not.toHaveBeenCalled();
});
it('returns filtered suggestion rows from the public API payload', async () => {
vi.spyOn(globalThis, 'fetch').mockResolvedValue({
ok: true,
json: async () => ['open', ['opencli', '', 'open source']],
});
await expect(command.func({ keyword: 'open', limit: 3 })).resolves.toEqual([
{ phrase: 'opencli' },
{ phrase: 'open source' },
]);
});
it('maps fetch and malformed JSON failures to typed command errors', async () => {
vi.spyOn(globalThis, 'fetch').mockRejectedValueOnce(new Error('offline'));
await expect(command.func({ keyword: 'open', limit: 3 })).rejects.toMatchObject({ code: 'COMMAND_EXEC' });
vi.spyOn(globalThis, 'fetch').mockResolvedValueOnce({
ok: true,
json: async () => { throw new Error('bad json'); },
});
await expect(command.func({ keyword: 'open', limit: 3 })).rejects.toMatchObject({ code: 'COMMAND_EXEC' });
});
});
+299 -54
View File
@@ -1,60 +1,305 @@
import { cli } from '@jackwener/opencli/registry';
cli({
site: 'facebook',
name: 'feed',
access: 'read',
description: 'Get your Facebook news feed',
domain: 'www.facebook.com',
args: [
{ name: 'limit', type: 'int', default: 10, help: 'Number of posts' },
],
columns: ['index', 'author', 'content', 'likes', 'comments', 'shares'],
pipeline: [
{ navigate: { url: 'https://www.facebook.com/', settleMs: 4000 } },
{ evaluate: `(() => {
const limit = \${{ args.limit }};
const posts = document.querySelectorAll('[role="article"]');
return Array.from(posts)
.filter(el => {
const text = el.textContent.trim();
// Filter out "People you may know" suggestions (both CN and EN)
return text.length > 30 &&
!text.startsWith('可能认识') &&
!text.startsWith('People you may know') &&
!text.startsWith('People You May Know');
})
.slice(0, limit)
.map((el, i) => {
// Author from header link
const headerLink = el.querySelector('h2 a, h3 a, h4 a, strong a');
const author = headerLink ? headerLink.textContent.trim() : '';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
// Post text: grab visible spans, filter noise
const spans = Array.from(el.querySelectorAll('div[dir="auto"]'))
.map(s => s.textContent.trim())
.filter(t => t.length > 10 && t.length < 500);
const content = spans.length > 0 ? spans[0] : '';
const FACEBOOK_HOME = 'https://www.facebook.com/';
const MAX_LIMIT = 50;
// Engagement: find like/comment/share counts (CN + EN)
const allText = el.textContent;
const likesMatch = allText.match(/所有心情:([\\d,.\\s]*[\\d万亿KMk]+)/) ||
allText.match(/All:\\s*([\\d,.KMk]+)/) ||
allText.match(/([\\d,.KMk]+)\\s*(?:likes?|reactions?)/i);
const commentsMatch = allText.match(/([\\d,.]+\\s*[万亿]?)\\s*条评论/) ||
allText.match(/([\\d,.KMk]+)\\s*comments?/i);
const sharesMatch = allText.match(/([\\d,.]+\\s*[万亿]?)\\s*次分享/) ||
allText.match(/([\\d,.KMk]+)\\s*shares?/i);
function requireLimit(value) {
const n = Number(value);
if (!Number.isInteger(n) || n < 1 || n > MAX_LIMIT) {
throw new ArgumentError(`facebook feed --limit must be an integer between 1 and ${MAX_LIMIT}`);
}
return n;
}
function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && 'data' in value) {
return value.data;
}
return value;
}
function buildFeedExtractScript(limit) {
return `(() => {
const limit = ${limit};
function clean(value) {
return String(value || '').replace(/\\s+/g, ' ').trim();
}
function textOf(el) {
return clean(el && el.textContent);
}
function labelOf(el) {
return clean(el && el.getAttribute && el.getAttribute('aria-label'));
}
function isAuthPage() {
const path = window.location && window.location.pathname ? window.location.pathname : '';
const body = textOf(document.body);
return /^\\/(login|checkpoint)(\\/|$|\\.php)/.test(path)
|| /^(Log in to Facebook|Facebook登录|登录 Facebook)/i.test(body)
|| /You must log in to continue/i.test(body);
}
function isExplicitEmptyFeed() {
const body = textOf(document.body);
return /No posts available|Nothing to show|暂无动态|没有更多动态|还没有帖子/i.test(body);
}
function isSuggestionOrChrome(text) {
return /^(People you may know|People You May Know|可能认识的人?|你可能认识的人?)/i.test(text)
|| /^(Suggested for you|Suggested Groups|推荐小组|推荐内容)/i.test(text);
}
function isSponsored(text) {
return /(^|\\s)(Sponsored|赞助|广告)(\\s|$)/i.test(text);
}
function isActionText(text) {
return /^(Like|Comment|Share|Send|Follow|赞|评论|分享|发送|关注)$/i.test(text);
}
function isMetricText(text) {
return /^(All:|所有心情:)/i.test(text)
|| /\\b(likes?|reactions?|comments?|shares?)\\b/i.test(text)
|| /(条评论|次分享)$/.test(text);
}
function isTimestampText(text) {
return /^(\\d+\\s*(s|m|h|d|w|mo|yr|min|sec|second|minute|hour|day|week|month|year)s?|Just now|Yesterday|刚刚|昨天|\\d+小时|\\d+天)(\\s*[·•.])?$/i.test(text);
}
function postUrlFrom(root) {
const links = Array.from(root.querySelectorAll('a[href]'));
for (const link of links) {
const href = link.href || link.getAttribute('href') || '';
if (/\\/posts\\/|\\/permalink\\.php|\\/story\\.php|\\/photo\\/\\?fbid=|\\/groups\\/[^/]+\\/posts\\//i.test(href)) {
return href;
}
}
return '';
}
function actionKinds(root) {
const kinds = new Set();
for (const el of root.querySelectorAll('[aria-label]')) {
const label = labelOf(el);
if (/^(Like|赞)$/i.test(label)) kinds.add('like');
if (/^(Comment|评论)$/i.test(label)) kinds.add('comment');
if (/^(Share|分享)$/i.test(label)) kinds.add('share');
}
return kinds;
}
function visibleBlocks(root) {
const seen = new Set();
return Array.from(root.querySelectorAll('[dir="auto"]'))
.map(textOf)
.filter((text) => {
if (!text || text.length > 600 || seen.has(text)) return false;
seen.add(text);
return true;
});
}
function findAuthor(root) {
const links = [
root.querySelector('h2 a[href], h3 a[href], h4 a[href], strong a[href]'),
...Array.from(root.querySelectorAll('a[role="link"][href]')),
].filter(Boolean);
for (const link of links) {
const text = textOf(link);
const href = link.href || link.getAttribute('href') || '';
if (text.length > 1 && text.length <= 80
&& !isActionText(text)
&& !isMetricText(text)
&& !isTimestampText(text)
&& !/\\/groups\\/|\\/watch\\/|\\/reel\\/|\\/events\\/|\\/friends\\//i.test(href)) {
return text;
}
}
return '';
}
function contentBlocks(root, author) {
return visibleBlocks(root).filter((text) => {
if (text === author) return false;
if (text.length <= 10) return false;
if (isSuggestionOrChrome(text) || isSponsored(text)) return false;
if (isActionText(text) || isMetricText(text) || isTimestampText(text)) return false;
if (/^(See more|查看更多|更多)$/i.test(text)) return false;
return true;
});
}
function extractPost(root, index) {
const fullText = textOf(root);
if (!fullText || isSuggestionOrChrome(fullText) || isSponsored(fullText)) return null;
const author = findAuthor(root);
const blocks = contentBlocks(root, author);
const content = clean(blocks.join(' '));
const postUrl = postUrlFrom(root);
const kinds = actionKinds(root);
if (!author && !content) return null;
if (!content && !postUrl && kinds.size < 2) return null;
const likesMatch = fullText.match(/所有心情:\\s*(\\d[\\d,.\\s万亿KMk]*)/)
|| fullText.match(/All:\\s*(\\d[\\d,.KMk]*)/)
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*(?:likes?|reactions?)/i);
const commentsMatch = fullText.match(/([\\d,.]+\\s*[万亿]?)\\s*条评论/)
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*comments?/i);
const sharesMatch = fullText.match(/([\\d,.]+\\s*[万亿]?)\\s*次分享/)
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*shares?/i);
return {
index: i + 1,
index,
author: author.substring(0, 50),
content: content.replace(/\\n/g, ' ').substring(0, 120),
likes: likesMatch ? likesMatch[1] : '-',
comments: commentsMatch ? commentsMatch[1] : '-',
shares: sharesMatch ? sharesMatch[1] : '-',
content: content.substring(0, 120),
likes: likesMatch ? clean(likesMatch[1]) : '-',
comments: commentsMatch ? clean(commentsMatch[1]) : '-',
shares: sharesMatch ? clean(sharesMatch[1]) : '-',
};
});
})()
` },
],
});
}
function primaryContainers() {
return Array.from(document.querySelectorAll('[role="article"]'))
.filter((el) => textOf(el).length > 30);
}
function fallbackContainers() {
const main = document.querySelector('[role="main"]');
if (!main) return [];
const buttons = Array.from(main.querySelectorAll('[aria-label="Like"], [aria-label="赞"], [aria-label="Comment"], [aria-label="评论"], [aria-label="Share"], [aria-label="分享"]'));
const seen = new WeakSet();
const containers = [];
for (const button of buttons) {
let node = button.parentElement;
for (let depth = 0; depth < 16 && node && node !== main && node !== document.body; depth += 1, node = node.parentElement) {
const text = textOf(node);
const kinds = actionKinds(node);
const blocks = visibleBlocks(node);
const hasPostEvidence = Boolean(postUrlFrom(node)) || blocks.some((block) => block.length > 20 && !isActionText(block) && !isMetricText(block));
if (text.length >= 80 && kinds.has('like') && (kinds.has('comment') || kinds.has('share')) && hasPostEvidence) {
if (!seen.has(node)) {
seen.add(node);
containers.push(node);
}
break;
}
}
}
return containers;
}
function dedupe(containers) {
const seen = new Set();
const result = [];
for (const node of containers) {
const key = postUrlFrom(node) || contentBlocks(node, findAuthor(node)).join('|').substring(0, 200);
if (!key || seen.has(key)) continue;
seen.add(key);
result.push(node);
}
return result;
}
if (isAuthPage()) return { status: 'auth', rows: [], diagnostics: {} };
const primary = primaryContainers();
const combined = dedupe([...primary, ...fallbackContainers()]);
const rows = [];
for (const container of combined) {
const row = extractPost(container, rows.length + 1);
if (row) rows.push(row);
if (rows.length >= limit) break;
}
return {
status: rows.length ? 'ok' : (isExplicitEmptyFeed() ? 'empty' : 'no_rows'),
rows,
diagnostics: {
articleCount: document.querySelectorAll('[role="article"]').length,
primaryCount: primary.length,
fallbackActionCount: document.querySelectorAll('[role="main"] [aria-label="Like"], [role="main"] [aria-label="赞"], [role="main"] [aria-label="Comment"], [role="main"] [aria-label="评论"]').length,
mainTextLength: textOf(document.querySelector('[role="main"]')).length,
},
};
})()`;
}
async function getFacebookFeed(page, kwargs) {
const limit = requireLimit(kwargs.limit ?? 10);
try {
await page.goto(FACEBOOK_HOME, { settleMs: 4000 });
} catch (err) {
throw new CommandExecutionError(
`Failed to navigate to facebook feed: ${err instanceof Error ? err.message : err}`,
'Check that facebook.com is reachable and the browser extension is connected.',
);
}
let payload;
try {
payload = unwrapBrowserResult(await page.evaluate(buildFeedExtractScript(limit)));
} catch (err) {
throw new CommandExecutionError(
`Failed to read facebook feed: ${err instanceof Error ? err.message : err}`,
'Facebook may not have rendered or the feed markup may have changed.',
);
}
if (!payload || typeof payload !== 'object' || !Array.isArray(payload.rows)) {
throw new CommandExecutionError('facebook feed returned malformed extraction payload');
}
if (payload.status === 'auth') {
throw new AuthRequiredError('www.facebook.com', 'Open Chrome and log in to Facebook before retrying.');
}
if (payload.rows.length > 0) {
return payload.rows;
}
if (payload.status === 'empty') {
throw new EmptyResultError('facebook feed', 'Facebook did not show any feed posts for this account.');
}
const diagnostics = payload.diagnostics || {};
if (diagnostics.articleCount || diagnostics.fallbackActionCount || diagnostics.mainTextLength > 200) {
throw new CommandExecutionError(
'facebook feed page rendered but no feed rows could be extracted',
`Diagnostics: articles=${diagnostics.articleCount || 0}, actions=${diagnostics.fallbackActionCount || 0}, mainTextLength=${diagnostics.mainTextLength || 0}.`,
);
}
throw new EmptyResultError('facebook feed', 'No Facebook feed content was visible in the current browser session.');
}
const command = {
site: 'facebook',
name: 'feed',
access: 'read',
description: 'Get your Facebook news feed',
domain: 'www.facebook.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'limit', type: 'int', default: 10, help: 'Number of posts' },
],
columns: ['index', 'author', 'content', 'likes', 'comments', 'shares'],
func: getFacebookFeed,
};
cli(command);
export const __test__ = {
buildFeedExtractScript,
command,
getFacebookFeed,
requireLimit,
};
+169
View File
@@ -0,0 +1,169 @@
import { describe, expect, it, vi } from 'vitest';
import { JSDOM } from 'jsdom';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { __test__ } from './feed.js';
function runExtract(html, limit = 10, url = 'https://www.facebook.com/') {
const dom = new JSDOM(html, { url });
return Function('window', 'document', `return ${__test__.buildFeedExtractScript(limit)};`)(dom.window, dom.window.document);
}
function createPage(payload) {
return {
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(payload),
};
}
describe('facebook feed', () => {
it('registers the feed command with the existing row contract', () => {
const cmd = getRegistry().get('facebook/feed');
expect(cmd).toBeDefined();
expect(cmd.columns).toEqual(['index', 'author', 'content', 'likes', 'comments', 'shares']);
});
it('extracts existing role=article feed rows', () => {
const payload = runExtract(`
<main role="main">
<div role="article">
<h2><a href="https://www.facebook.com/alice">Alice Example</a></h2>
<div dir="auto">This is a normal Facebook feed post with enough text to extract.</div>
<span>All: 12</span>
<span>3 comments</span>
<span>2 shares</span>
<div aria-label="Like"></div><div aria-label="Comment"></div>
</div>
</main>
`);
expect(payload.status).toBe('ok');
expect(payload.rows).toEqual([{
index: 1,
author: 'Alice Example',
content: 'This is a normal Facebook feed post with enough text to extract.',
likes: '12',
comments: '3',
shares: '2',
}]);
});
it('falls back from empty article nodes to action-bounded feed containers', () => {
const payload = runExtract(`
<main role="main">
<div role="article"></div>
<section>
<div>
<h2><a href="https://www.facebook.com/bob/posts/123">Bob Builder</a></h2>
<div dir="auto">Fallback post body from a Facebook feed card with empty article text.</div>
<a href="https://www.facebook.com/bob/posts/123">Permalink</a>
<span>All: 1.2K</span>
<span>4 comments</span>
<span>1 shares</span>
<div><button aria-label="Like">Like</button><button aria-label="Comment">Comment</button></div>
</div>
</section>
</main>
`);
expect(payload.status).toBe('ok');
expect(payload.rows).toEqual([{
index: 1,
author: 'Bob Builder',
content: 'Fallback post body from a Facebook feed card with empty article text.',
likes: '1.2K',
comments: '4',
shares: '1',
}]);
});
it('does not turn suggestions or side chrome action buttons into feed rows', () => {
const payload = runExtract(`
<main role="main">
<aside>
<h2>People you may know</h2>
<div dir="auto">Charlie Suggested</div>
<div dir="auto">Add friend from suggested people card with plenty of text.</div>
<button aria-label="Like">Like</button>
<button aria-label="Comment">Comment</button>
</aside>
<nav>
<div dir="auto">Navigation item with a Like button but not a feed post.</div>
<button aria-label="Like">Like</button>
<button aria-label="Comment">Comment</button>
</nav>
</main>
`);
expect(payload.status).toBe('no_rows');
expect(payload.rows).toEqual([]);
});
it('still considers bounded fallback rows when article nodes are suggestion chrome', () => {
const payload = runExtract(`
<main role="main">
<div role="article">
<h2>People you may know</h2>
<div dir="auto">Suggested profile card with enough text to look article-like.</div>
<button aria-label="Like">Like</button>
<button aria-label="Comment">Comment</button>
</div>
<section>
<div>
<h2><a href="https://www.facebook.com/dana/posts/456">Dana Poster</a></h2>
<div dir="auto">Fallback feed post should still be extracted after suggestion articles are filtered.</div>
<a href="https://www.facebook.com/dana/posts/456">Permalink</a>
<button aria-label="Like">Like</button>
<button aria-label="Comment">Comment</button>
</div>
</section>
</main>
`, 1);
expect(payload.status).toBe('ok');
expect(payload.rows).toEqual([{
index: 1,
author: 'Dana Poster',
content: 'Fallback feed post should still be extracted after suggestion articles are filtered.',
likes: '-',
comments: '-',
shares: '-',
}]);
});
it('reports auth pages from the browser extractor', () => {
const payload = runExtract('<main role="main">Log in to Facebook</main>', 10, 'https://www.facebook.com/login/');
expect(payload.status).toBe('auth');
expect(payload.rows).toEqual([]);
});
it('validates limit before browser navigation', async () => {
const page = createPage({ status: 'ok', rows: [] });
await expect(__test__.command.func(page, { limit: 0 })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('maps browser envelopes and returns extracted rows', async () => {
const page = createPage({ session: 'site:facebook', data: { status: 'ok', rows: [{ index: 1, author: 'A', content: 'Body', likes: '-', comments: '-', shares: '-' }] } });
await expect(__test__.command.func(page, { limit: 1 })).resolves.toEqual([{
index: 1,
author: 'A',
content: 'Body',
likes: '-',
comments: '-',
shares: '-',
}]);
});
it('maps auth, real empty, parser drift, and malformed payloads to typed errors', async () => {
await expect(__test__.command.func(createPage({ status: 'auth', rows: [] }), { limit: 1 }))
.rejects.toBeInstanceOf(AuthRequiredError);
await expect(__test__.command.func(createPage({ status: 'empty', rows: [] }), { limit: 1 }))
.rejects.toBeInstanceOf(EmptyResultError);
await expect(__test__.command.func(createPage({ status: 'no_rows', rows: [], diagnostics: { articleCount: 1, fallbackActionCount: 2, mainTextLength: 500 } }), { limit: 1 }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(__test__.command.func(createPage({ rows: null }), { limit: 1 }))
.rejects.toBeInstanceOf(CommandExecutionError);
});
});
+228
View File
@@ -0,0 +1,228 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import {
ArgumentError,
AuthRequiredError,
CommandExecutionError,
EmptyResultError,
} from '@jackwener/opencli/errors';
import { createHash } from 'node:crypto';
const FLOMO_APP_DOMAIN = 'v.flomoapp.com';
const FLOMO_API_DOMAIN = 'flomoapp.com';
const MAX_LIMIT = 200;
function unwrapBrowserResult(value) {
if (value && typeof value === 'object' && 'session' in value && 'data' in value) {
return value.data;
}
return value;
}
function parsePositiveIntArg(value, name, fallback, max) {
if (value === undefined || value === null || value === '') {
return fallback;
}
const text = String(value).trim();
if (!/^\d+$/.test(text)) {
throw new ArgumentError(`flomo memos --${name} must be a positive integer`);
}
const parsed = Number(text);
if (!Number.isSafeInteger(parsed) || parsed < 1 || parsed > max) {
throw new ArgumentError(`flomo memos --${name} must be between 1 and ${max}`);
}
return parsed;
}
function parseSinceArg(value) {
if (value === undefined || value === null || value === '') {
return 0;
}
const text = String(value).trim();
if (!/^\d+$/.test(text)) {
throw new ArgumentError('flomo memos --since must be a non-negative Unix timestamp in seconds');
}
const parsed = Number(text);
if (!Number.isSafeInteger(parsed)) {
throw new ArgumentError('flomo memos --since must be a safe integer Unix timestamp in seconds');
}
return parsed;
}
function parseSlugArg(value) {
if (value === undefined || value === null || value === '') {
return '';
}
const slug = String(value).trim();
if (!/^[A-Za-z0-9_-]{1,256}$/.test(slug)) {
throw new ArgumentError('flomo memos --slug must be an opaque memo cursor containing only letters, numbers, _ or -');
}
return slug;
}
function buildSignedUrl(limit, since, slug) {
const params = {
limit: String(limit),
latest_updated_at: String(since),
tz: '8:0',
timestamp: String(Math.floor(Date.now() / 1000)),
api_key: 'flomo_web',
app_version: '4.0',
platform: 'web',
webp: '1',
};
if (slug) params.latest_slug = slug;
const keys = Object.keys(params).sort();
const signBase = keys.map((key) => `${key}=${params[key]}`).join('&');
params.sign = createHash('md5').update(signBase + 'dbbc3dd73364b4084c3a69346e0ce2b2').digest('hex');
return 'https://flomoapp.com/api/v1/memo/updated/?' + new URLSearchParams(params).toString();
}
function buildGetTokenJs() {
return `
(() => {
try {
const raw = localStorage.getItem('me');
if (!raw) return null;
const me = JSON.parse(raw);
const token = me?.access_token || me?.data?.access_token || '';
return typeof token === 'string' && token.trim() ? token.trim() : null;
} catch {
return null;
}
})()
`;
}
function isAuthFailureMessage(message) {
return /auth|unauth|login|token|permission|forbidden|unauthorized|登录|登陆|鉴权|权限/i.test(String(message || ''));
}
function normalizeTags(tags) {
if (!Array.isArray(tags)) return '';
return tags
.map((tag) => {
if (typeof tag === 'string') return tag;
return tag?.name || tag?.tag || tag?.content || '';
})
.map((tag) => String(tag).trim())
.filter(Boolean)
.join(', ');
}
function normalizeImages(files) {
if (!Array.isArray(files)) return '';
return files
.map((file) => file?.thumbnail_url || file?.url || '')
.map((url) => String(url).trim())
.filter(Boolean)
.join(' | ');
}
function memoUrl(slug) {
return slug ? `https://${FLOMO_APP_DOMAIN}/mine/?memo_id=${encodeURIComponent(slug)}` : '';
}
function normalizeMemo(memo) {
if (!memo || typeof memo !== 'object' || Array.isArray(memo)) {
throw new CommandExecutionError('Flomo API returned a malformed memo entry');
}
const slug = String(memo.slug || memo.id || '').trim();
if (!slug) {
throw new CommandExecutionError('Flomo API returned a memo without slug/id');
}
return {
id: slug,
url: memoUrl(slug),
content: String(memo.content || '').trim(),
slug,
tags: normalizeTags(memo.tags),
images: normalizeImages(memo.files),
created_at: String(memo.created_at || ''),
updated_at: String(memo.updated_at || ''),
};
}
async function fetchFlomoJson(url, token) {
let resp;
try {
resp = await fetch(url, {
headers: {
Authorization: 'Bearer ' + token,
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
Accept: 'application/json',
},
});
} catch (err) {
throw new CommandExecutionError(`Failed to fetch Flomo memos: ${err instanceof Error ? err.message : String(err)}`);
}
if (resp.status === 401 || resp.status === 403) {
throw new AuthRequiredError(FLOMO_API_DOMAIN, `Flomo API returned HTTP ${resp.status}; please refresh your Flomo login session`);
}
if (!resp.ok) {
throw new CommandExecutionError(`Flomo API returned HTTP ${resp.status}`);
}
try {
return await resp.json();
} catch (err) {
throw new CommandExecutionError(`Flomo API returned malformed JSON: ${err instanceof Error ? err.message : String(err)}`);
}
}
async function readAccessToken(page) {
const token = unwrapBrowserResult(await page.evaluate(buildGetTokenJs()));
if (typeof token !== 'string' || !token.trim()) {
throw new AuthRequiredError(FLOMO_API_DOMAIN, 'Flomo memos requires an active signed-in Flomo browser session');
}
return token.trim();
}
const command = cli({
site: 'flomo',
name: 'memos',
access: 'read',
description: 'List your Flomo memos',
domain: FLOMO_API_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: `https://${FLOMO_APP_DOMAIN}/`,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of memos to fetch (1-200)' },
{ name: 'since', type: 'int', help: 'Only memos updated after this Unix timestamp in seconds' },
{ name: 'slug', help: 'Pagination cursor from a previous memo page' },
],
columns: ['id', 'url', 'content', 'slug', 'tags', 'images', 'created_at', 'updated_at'],
func: async (page, kwargs) => {
const limit = parsePositiveIntArg(kwargs.limit, 'limit', 20, MAX_LIMIT);
const since = parseSinceArg(kwargs.since);
const slug = parseSlugArg(kwargs.slug);
await page.wait(3).catch(() => {});
const token = await readAccessToken(page);
const body = await fetchFlomoJson(buildSignedUrl(limit, since, slug), token);
if (!body || typeof body !== 'object' || Array.isArray(body)) {
throw new CommandExecutionError('Flomo API returned a malformed response');
}
if (body.code !== 0) {
const message = body.message || `Flomo API error code ${body.code}`;
if (isAuthFailureMessage(message)) {
throw new AuthRequiredError(FLOMO_API_DOMAIN, message);
}
throw new CommandExecutionError(message);
}
if (!Array.isArray(body.data)) {
throw new CommandExecutionError('Flomo API returned malformed memo data');
}
if (body.data.length === 0) {
throw new EmptyResultError('flomo memos', 'No Flomo memos matched the requested filters.');
}
return body.data.map(normalizeMemo);
},
});
export const __test__ = {
buildSignedUrl,
command,
normalizeMemo,
parsePositiveIntArg,
parseSinceArg,
parseSlugArg,
};
+144
View File
@@ -0,0 +1,144 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import {
ArgumentError,
AuthRequiredError,
CommandExecutionError,
EmptyResultError,
} from '@jackwener/opencli/errors';
const { __test__ } = await import('./memos.js');
const { command, normalizeMemo, parsePositiveIntArg, parseSinceArg, parseSlugArg } = __test__;
function createPage(token = 'token-123') {
return {
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue({ session: 'browser:default', data: token }),
};
}
function mockFetchJson(body, status = 200) {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
ok: status >= 200 && status < 300,
status,
json: vi.fn().mockResolvedValue(body),
}));
}
describe('flomo memos registration', () => {
it('registers as a browser cookie read command with stable columns', () => {
expect(command.site).toBe('flomo');
expect(command.name).toBe('memos');
expect(command.access).toBe('read');
expect(command.browser).toBe(true);
expect(command.strategy).toBe('cookie');
expect(command.columns).toEqual(['id', 'url', 'content', 'slug', 'tags', 'images', 'created_at', 'updated_at']);
});
});
describe('flomo memos argument validation', () => {
it('rejects invalid limits instead of silently clamping', () => {
expect(() => parsePositiveIntArg('0', 'limit', 20, 200)).toThrow(ArgumentError);
expect(() => parsePositiveIntArg('201', 'limit', 20, 200)).toThrow(ArgumentError);
expect(() => parsePositiveIntArg('10.5', 'limit', 20, 200)).toThrow(ArgumentError);
expect(() => parsePositiveIntArg('abc', 'limit', 20, 200)).toThrow(ArgumentError);
expect(parsePositiveIntArg(undefined, 'limit', 20, 200)).toBe(20);
expect(parsePositiveIntArg('200', 'limit', 20, 200)).toBe(200);
});
it('rejects invalid since and slug arguments', () => {
expect(parseSinceArg(undefined)).toBe(0);
expect(parseSinceArg('1735689600')).toBe(1735689600);
expect(() => parseSinceArg('-1')).toThrow(ArgumentError);
expect(() => parseSinceArg('1.5')).toThrow(ArgumentError);
expect(parseSlugArg(undefined)).toBe('');
expect(parseSlugArg('abc_DEF-123')).toBe('abc_DEF-123');
expect(() => parseSlugArg('bad/slash')).toThrow(ArgumentError);
expect(() => parseSlugArg('bad space')).toThrow(ArgumentError);
});
});
describe('flomo memo normalization', () => {
it('emits string-safe id/url fields and normalizes tags/images', () => {
expect(normalizeMemo({
slug: 'memo_12345678901234567890',
content: ' <p>Hello</p> ',
tags: [{ name: 'work' }, 'idea'],
files: [{ thumbnail_url: 'https://img/thumb.jpg' }, { url: 'https://img/full.jpg' }],
created_at: '2026-01-01T00:00:00+08:00',
updated_at: '2026-01-02T00:00:00+08:00',
})).toEqual({
id: 'memo_12345678901234567890',
url: 'https://v.flomoapp.com/mine/?memo_id=memo_12345678901234567890',
content: '<p>Hello</p>',
slug: 'memo_12345678901234567890',
tags: 'work, idea',
images: 'https://img/thumb.jpg | https://img/full.jpg',
created_at: '2026-01-01T00:00:00+08:00',
updated_at: '2026-01-02T00:00:00+08:00',
});
});
it('fails typed on malformed memo entries', () => {
expect(() => normalizeMemo(null)).toThrow(CommandExecutionError);
expect(() => normalizeMemo({ content: 'missing slug' })).toThrow(CommandExecutionError);
});
});
describe('flomo memos command', () => {
beforeEach(() => {
vi.unstubAllGlobals();
});
it('reads token from Browser Bridge envelope and returns memo rows', async () => {
mockFetchJson({
code: 0,
data: [{
slug: 'memo_1',
content: 'hello',
tags: ['tag'],
files: [],
created_at: '2026-01-01',
updated_at: '2026-01-02',
}],
});
const rows = await command.func(createPage(), { limit: '1' });
expect(globalThis.fetch).toHaveBeenCalledWith(expect.stringContaining('limit=1'), expect.objectContaining({
headers: expect.objectContaining({ Authorization: 'Bearer token-123' }),
}));
expect(rows).toEqual([{
id: 'memo_1',
url: 'https://v.flomoapp.com/mine/?memo_id=memo_1',
content: 'hello',
slug: 'memo_1',
tags: 'tag',
images: '',
created_at: '2026-01-01',
updated_at: '2026-01-02',
}]);
});
it('throws AuthRequiredError when the browser session has no token', async () => {
await expect(command.func(createPage(null), {})).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps Flomo auth failures to AuthRequiredError', async () => {
mockFetchJson({ code: 401, message: 'unauthorized' });
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(AuthRequiredError);
});
it('maps HTTP, malformed JSON, malformed data, and empty results to typed errors', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, status: 500, json: vi.fn() }));
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: true, status: 200, json: vi.fn().mockRejectedValue(new Error('bad json')) }));
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
mockFetchJson({ code: 0, data: {} });
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
mockFetchJson({ code: 0, data: [] });
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(EmptyResultError);
});
});
+2 -2
View File
@@ -123,8 +123,8 @@ cli({
rows.push({
rank: rows.length + 1,
name,
language: normalizeText(getFirstText(fields.langs)) || '-',
description: normalizeText(getFirstText(fields.description)) || '-',
language: normalizeText(getFirstText(fields.langs)) || '',
description: normalizeText(getFirstText(fields.description)) || '',
stars: normalizeStars(fields['count.star']),
url: repoUrl,
});
+65
View File
@@ -0,0 +1,65 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
function mockGiteeResponse(hits) {
return {
ok: true,
json: () => Promise.resolve({ hits: { hits } }),
};
}
function makePage() {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
};
}
describe('gitee search', () => {
beforeEach(() => {
vi.restoreAllMocks();
});
afterEach(() => {
vi.unstubAllGlobals();
});
it('emits empty-string for missing language / description instead of a sentinel', async () => {
const cmd = getRegistry().get('gitee/search');
expect(cmd?.func).toBeTypeOf('function');
const fetchMock = vi.fn().mockResolvedValue(mockGiteeResponse([
{
fields: {
title: 'someuser/no-meta-repo',
url: 'https://gitee.com/someuser/no-meta-repo',
},
},
]));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func(makePage(), { keyword: 'test', limit: 10 });
expect(rows).toHaveLength(1);
expect(rows[0].name).toBe('someuser/no-meta-repo');
expect(rows[0].language).toBe('');
expect(rows[0].description).toBe('');
});
it('passes through populated language / description verbatim', async () => {
const cmd = getRegistry().get('gitee/search');
const fetchMock = vi.fn().mockResolvedValue(mockGiteeResponse([
{
fields: {
title: 'org/repo-a',
url: 'https://gitee.com/org/repo-a',
langs: 'TypeScript',
description: 'A test repo',
'count.star': '42',
},
},
]));
vi.stubGlobal('fetch', fetchMock);
const rows = await cmd.func(makePage(), { keyword: 'test', limit: 10 });
expect(rows[0].language).toBe('TypeScript');
expect(rows[0].description).toBe('A test repo');
expect(rows[0].stars).toBe('42');
});
});
+20 -5
View File
@@ -1,4 +1,5 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { clampInt, requireNonEmptyQuery } from '../_shared/common.js';
cli({
@@ -18,12 +19,17 @@ cli({
const limit = clampInt(kwargs.limit, 10, 1, 20);
const query = requireNonEmptyQuery(kwargs.query);
await page.goto(`https://scholar.google.com/scholar?q=${encodeURIComponent(query)}&hl=zh-CN`);
await page.wait(3);
const data = await page.evaluate(`
try {
await page.wait({ selector: '.gs_r.gs_or.gs_scl', timeout: 5 });
} catch {
await page.wait(3);
}
const wrapper = await page.evaluate(`
(() => {
const normalize = v => (v || '').replace(/\\s+/g, ' ').trim();
const results = [];
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl')) {
const resultCards = Array.from(document.querySelectorAll('.gs_r.gs_or.gs_scl'));
for (const el of resultCards) {
const container = el.querySelector('.gs_ri') || el;
const titleEl = container.querySelector('.gs_rt a, h3 a');
const title = normalize(titleEl?.textContent);
@@ -50,9 +56,18 @@ cli({
});
if (results.length >= ${limit}) break;
}
return results;
return { items: results, resultCount: resultCards.length };
})()
`);
return Array.isArray(data) ? data : [];
if (!wrapper || typeof wrapper !== 'object' || !Array.isArray(wrapper.items)) {
throw new CommandExecutionError('Google Scholar search returned an unexpected payload shape');
}
if (wrapper.items.length === 0) {
if (Number(wrapper.resultCount) > 0) {
throw new CommandExecutionError('Google Scholar result cards were present but no rows could be extracted');
}
throw new EmptyResultError('google-scholar/search', 'Try a different query or check whether Google Scholar returned a CAPTCHA.');
}
return wrapper.items;
},
});
+35 -2
View File
@@ -1,4 +1,5 @@
import { describe, expect, it, vi } from 'vitest';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './search.js';
@@ -25,14 +26,46 @@ describe('google-scholar search command', () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue([]),
evaluate: vi.fn().mockResolvedValue({ items: [{ rank: 1, title: 'Paper' }], resultCount: 1 }),
};
await command.func(page, { query: 'transformer' });
const rows = await command.func(page, { query: 'transformer' });
const script = page.evaluate.mock.calls[0][0];
expect(script).toContain("document.querySelectorAll('.gs_r.gs_or.gs_scl')");
expect(script).not.toContain(".gs_r.gs_or.gs_scl, .gs_ri");
expect(script).toContain("const container = el.querySelector('.gs_ri') || el");
expect(script).toContain('return { items: results, resultCount: resultCards.length }');
expect(rows).toEqual([{ rank: 1, title: 'Paper' }]);
});
it('throws typed empty when Scholar returns no result cards', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue({ items: [], resultCount: 0 }),
};
await expect(command.func(page, { query: 'no results expected' })).rejects.toThrow(EmptyResultError);
});
it('throws command execution when result cards exist but parser extracts no rows', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue({ items: [], resultCount: 2 }),
};
await expect(command.func(page, { query: 'parser drift' })).rejects.toThrow(CommandExecutionError);
});
it('throws command execution for malformed evaluate payloads instead of treating them as empty', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue({ items: { rank: 1 }, resultCount: 1 }),
};
await expect(command.func(page, { query: 'bad payload' })).rejects.toThrow(CommandExecutionError);
});
});
+5 -4
View File
@@ -39,12 +39,12 @@ cli({
catch {
await page.wait(2);
}
const results = await page.evaluate(`
const wrapper = await page.evaluate(`
(function() {
var results = [];
var seenUrls = {};
var rso = document.querySelector('#rso');
if (!rso) return results;
if (!rso) return {items: results};
// -- Featured snippet (scoped to #rso to avoid matching unrelated elements) --
var featuredEl = rso.querySelector('.xpdopen .hgKElc')
@@ -126,10 +126,11 @@ cli({
}
}
return results;
return {items: results};
})()
`);
if (!Array.isArray(results) || results.length === 0) {
const results = (wrapper && wrapper.items) || [];
if (results.length === 0) {
throw new CliError('NOT_FOUND', 'No search results found', 'Try a different keyword or check for CAPTCHA');
}
return results;
+27 -17
View File
@@ -1,4 +1,5 @@
import { cli } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
cli({
site: 'jike',
name: 'post',
@@ -16,16 +17,16 @@ cli({
},
],
columns: ['type', 'author', 'content', 'likes', 'time'],
pipeline: [
{ navigate: 'https://m.okjike.com/originalPosts/${{ args.id }}' },
{ evaluate: `(() => {
func: async (page, args) => {
await page.goto(`https://m.okjike.com/originalPosts/${args.id}`);
const data = await page.evaluate(`(() => {
const el = document.querySelector('script[type="application/json"]');
if (!el) return { ok: false, reason: 'missing-data-script' };
try {
const el = document.querySelector('script[type="application/json"]');
if (!el) return [];
const data = JSON.parse(el.textContent);
const data = JSON.parse(el.textContent || '{}');
const pageProps = data?.props?.pageProps || {};
const post = pageProps.post || {};
const comments = pageProps.comments || [];
const comments = Array.isArray(pageProps.comments) ? pageProps.comments : [];
const result = [{
type: 'post',
@@ -47,16 +48,25 @@ cli({
return result;
} catch (e) {
return [];
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
}
})()
` },
{ map: {
type: '${{ item.type }}',
author: '${{ item.author }}',
content: '${{ item.content }}',
likes: '${{ item.likes }}',
time: '${{ item.time }}',
} },
],
`);
if (Array.isArray(data)) {
return data.map((item) => ({
type: item.type ?? '',
author: item.author ?? '',
content: item.content ?? '',
likes: item.likes ?? 0,
time: item.time ?? '',
}));
}
if (data?.reason === 'missing-data-script') {
throw new CommandExecutionError('Jike post page did not expose the expected data script');
}
if (data?.reason === 'parse-error') {
throw new CommandExecutionError(`Failed to parse Jike post data: ${data.message || 'unknown error'}`);
}
throw new CommandExecutionError('Jike post returned an unreadable payload');
},
});
+86
View File
@@ -0,0 +1,86 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './post.js';
import './topic.js';
import './user.js';
function makePage(evaluateResult) {
return {
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('jike read commands', () => {
it('maps post rows from the browser-side extractor', async () => {
const command = getRegistry().get('jike/post');
const page = makePage([
{ type: 'post', author: 'alice', content: 'hello', likes: 3, time: '2026-05-16' },
{ type: 'comment', author: 'bob', content: 'nice', likes: 1, time: '2026-05-16' },
]);
await expect(command.func(page, { id: 'post-1' })).resolves.toEqual([
{ type: 'post', author: 'alice', content: 'hello', likes: 3, time: '2026-05-16' },
{ type: 'comment', author: 'bob', content: 'nice', likes: 1, time: '2026-05-16' },
]);
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/originalPosts/post-1');
});
it('maps topic rows and applies limit on the Node side', async () => {
const command = getRegistry().get('jike/topic');
const page = makePage([
{ id: 'a', content: 'one', author: 'alice', likes: 1, comments: 2, time: 't1' },
{ id: 'b', content: 'two', author: 'bob', likes: 3, comments: 4, time: 't2' },
]);
await expect(command.func(page, { id: 'topic-1', limit: 1 })).resolves.toEqual([
{
content: 'one',
author: 'alice',
likes: 1,
comments: 2,
time: 't1',
url: 'https://web.okjike.com/originalPost/a',
},
]);
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/topics/topic-1');
});
it('maps user rows and applies limit on the Node side', async () => {
const command = getRegistry().get('jike/user');
const page = makePage([
{ id: 'a', content: 'one', type: 'post', likes: 1, comments: 2, time: 't1' },
{ id: 'b', content: 'two', type: 'repost', likes: 3, comments: 4, time: 't2' },
]);
await expect(command.func(page, { username: 'alice', limit: 1 })).resolves.toEqual([
{
id: 'a',
content: 'one',
type: 'post',
likes: 1,
comments: 2,
time: 't1',
url: 'https://web.okjike.com/originalPost/a',
},
]);
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/users/alice');
});
it('throws CommandExecutionError for malformed browser-side payloads', async () => {
await expect(getRegistry().get('jike/post').func(makePage({ reason: 'missing-data-script' }), { id: 'post-1' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(getRegistry().get('jike/topic').func(makePage({ reason: 'parse-error', message: 'bad json' }), { id: 'topic-1' }))
.rejects.toBeInstanceOf(CommandExecutionError);
await expect(getRegistry().get('jike/user').func(makePage(null), { username: 'alice' }))
.rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws EmptyResultError when topic or user extractors return no posts', async () => {
await expect(getRegistry().get('jike/topic').func(makePage([]), { id: 'topic-1' }))
.rejects.toBeInstanceOf(EmptyResultError);
await expect(getRegistry().get('jike/user').func(makePage([]), { username: 'alice' }))
.rejects.toBeInstanceOf(EmptyResultError);
});
});
+32 -19
View File
@@ -1,4 +1,5 @@
import { cli } from '@jackwener/opencli/registry';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
cli({
site: 'jike',
name: 'topic',
@@ -17,15 +18,16 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Number of posts' },
],
columns: ['content', 'author', 'likes', 'comments', 'time', 'url'],
pipeline: [
{ navigate: 'https://m.okjike.com/topics/${{ args.id }}' },
{ evaluate: `(() => {
func: async (page, args) => {
await page.goto(`https://m.okjike.com/topics/${args.id}`);
const limit = Number(args.limit) || 20;
const data = await page.evaluate(`(() => {
const el = document.querySelector('script[type="application/json"]');
if (!el) return { ok: false, reason: 'missing-data-script' };
try {
const el = document.querySelector('script[type="application/json"]');
if (!el) return [];
const data = JSON.parse(el.textContent);
const data = JSON.parse(el.textContent || '{}');
const pageProps = data?.props?.pageProps || {};
const posts = pageProps.posts || [];
const posts = Array.isArray(pageProps.posts) ? pageProps.posts : [];
return posts.map(p => ({
content: (p.content || '').replace(/\\n/g, ' ').slice(0, 80),
author: p.user?.screenName || '',
@@ -35,18 +37,29 @@ cli({
id: p.id || '',
}));
} catch (e) {
return [];
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
}
})()
` },
{ map: {
content: '${{ item.content }}',
author: '${{ item.author }}',
likes: '${{ item.likes }}',
comments: '${{ item.comments }}',
time: '${{ item.time }}',
url: 'https://web.okjike.com/originalPost/${{ item.id }}',
} },
{ limit: '${{ args.limit }}' },
],
`);
if (Array.isArray(data)) {
if (data.length === 0) {
throw new EmptyResultError('jike topic', `No posts were returned for topic ${args.id}. Confirm the topic ID and login state.`);
}
return data.slice(0, limit).map((item) => ({
content: item.content ?? '',
author: item.author ?? '',
likes: item.likes ?? 0,
comments: item.comments ?? 0,
time: item.time ?? '',
url: `https://web.okjike.com/originalPost/${item.id ?? ''}`,
}));
}
if (data?.reason === 'missing-data-script') {
throw new CommandExecutionError('Jike topic page did not expose the expected data script');
}
if (data?.reason === 'parse-error') {
throw new CommandExecutionError(`Failed to parse Jike topic data: ${data.message || 'unknown error'}`);
}
throw new CommandExecutionError('Jike topic returned an unreadable payload');
},
});
+33 -20
View File
@@ -1,4 +1,5 @@
import { cli } from '@jackwener/opencli/registry';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
cli({
site: 'jike',
name: 'user',
@@ -17,14 +18,15 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Number of posts' },
],
columns: ['id', 'content', 'type', 'likes', 'comments', 'time', 'url'],
pipeline: [
{ navigate: 'https://m.okjike.com/users/${{ args.username }}' },
{ evaluate: `(() => {
func: async (page, args) => {
await page.goto(`https://m.okjike.com/users/${args.username}`);
const limit = Number(args.limit) || 20;
const data = await page.evaluate(`(() => {
const el = document.querySelector('script[type="application/json"]');
if (!el) return { ok: false, reason: 'missing-data-script' };
try {
const el = document.querySelector('script[type="application/json"]');
if (!el) return [];
const data = JSON.parse(el.textContent);
const posts = data?.props?.pageProps?.posts || [];
const data = JSON.parse(el.textContent || '{}');
const posts = Array.isArray(data?.props?.pageProps?.posts) ? data.props.pageProps.posts : [];
return posts.map(p => ({
content: (p.content || '').replace(/\\n/g, ' ').slice(0, 80),
type: p.type === 'ORIGINAL_POST' ? 'post' : p.type === 'REPOST' ? 'repost' : p.type || '',
@@ -34,19 +36,30 @@ cli({
id: p.id || '',
}));
} catch (e) {
return [];
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
}
})()
` },
{ map: {
id: '${{ item.id }}',
content: '${{ item.content }}',
type: '${{ item.type }}',
likes: '${{ item.likes }}',
comments: '${{ item.comments }}',
time: '${{ item.time }}',
url: 'https://web.okjike.com/originalPost/${{ item.id }}',
} },
{ limit: '${{ args.limit }}' },
],
`);
if (Array.isArray(data)) {
if (data.length === 0) {
throw new EmptyResultError('jike user', `No posts were returned for user ${args.username}. Confirm the username and login state.`);
}
return data.slice(0, limit).map((item) => ({
id: item.id ?? '',
content: item.content ?? '',
type: item.type ?? '',
likes: item.likes ?? 0,
comments: item.comments ?? 0,
time: item.time ?? '',
url: `https://web.okjike.com/originalPost/${item.id ?? ''}`,
}));
}
if (data?.reason === 'missing-data-script') {
throw new CommandExecutionError('Jike user page did not expose the expected data script');
}
if (data?.reason === 'parse-error') {
throw new CommandExecutionError(`Failed to parse Jike user data: ${data.message || 'unknown error'}`);
}
throw new CommandExecutionError('Jike user returned an unreadable payload');
},
});
+1 -1
View File
@@ -56,7 +56,7 @@ cli({
rows.push({
rank: i + 1,
score: item.baseScore ?? 0,
author: user?.displayName ?? 'Unknown',
author: user?.displayName ?? '',
text: raw.length > 500 ? `${raw.slice(0, 500)}...` : raw,
});
}
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+37
View File
@@ -0,0 +1,37 @@
import { beforeEach, describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
const { gqlRequestMock } = vi.hoisted(() => ({ gqlRequestMock: vi.fn() }));
vi.mock('./_helpers.js', async () => {
const actual = await vi.importActual('./_helpers.js');
return { ...actual, gqlRequest: gqlRequestMock };
});
import './frontpage.js';
describe('lesswrong frontpage', () => {
beforeEach(() => {
gqlRequestMock.mockReset();
});
it('emits empty-string for missing user.displayName instead of a sentinel', async () => {
const command = getRegistry().get('lesswrong/frontpage');
expect(command?.func).toBeDefined();
gqlRequestMock.mockResolvedValueOnce({
posts: {
results: [
{ _id: 'a1', slug: 'post-a', title: 'Has author', user: { displayName: 'Real Person' }, baseScore: 10, commentCount: 3 },
{ _id: 'b2', slug: 'post-b', title: 'Deleted user', user: null, baseScore: 5, commentCount: 0 },
{ _id: 'c3', slug: 'post-c', title: 'Missing name', user: {}, baseScore: 7, commentCount: 1 },
],
},
});
const rows = await command.func({ limit: 3 });
expect(rows).toHaveLength(3);
expect(rows[0]).toMatchObject({ rank: 1, title: 'Has author', author: 'Real Person', karma: 10, comments: 3 });
expect(rows[1].author).toBe('');
expect(rows[1].title).toBe('Deleted user');
expect(rows[2].author).toBe('');
expect(rows[2].title).toBe('Missing name');
});
});
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -34,7 +34,7 @@ cli({
return [
{
title: post.title ?? '',
author: post.user?.displayName ?? 'Unknown',
author: post.user?.displayName ?? '',
karma: post.baseScore ?? 0,
comments: post.commentCount ?? 0,
tags: (post.tags ?? []).map((tag) => tag.name ?? '').filter(Boolean).join(', '),
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return sequences.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
}));
},
});
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -37,7 +37,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+1 -1
View File
@@ -22,7 +22,7 @@ cli({
return posts.map((item, i) => ({
rank: i + 1,
title: item.title ?? '',
author: item.user?.displayName ?? 'Unknown',
author: item.user?.displayName ?? '',
karma: item.baseScore ?? 0,
comments: item.commentCount ?? 0,
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
+138
View File
@@ -0,0 +1,138 @@
/**
* LinkedIn Learning course detail by slug, via /learning-api/courses?q=slug.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DOMAIN = 'www.linkedin.com';
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function parseSlug(value) {
const s = normalizeWhitespace(value);
if (!s) throw new ArgumentError('<slug> is required');
let slug = s;
if (/^https?:\/\//i.test(s)) {
let parsed;
try {
parsed = new URL(s);
} catch {
throw new ArgumentError(`Invalid LinkedIn Learning URL: "${s}"`);
}
const host = parsed.hostname.toLowerCase();
if (host !== 'linkedin.com' && host !== 'www.linkedin.com') {
throw new ArgumentError(`Invalid LinkedIn Learning host: "${parsed.hostname}"`);
}
const m = parsed.pathname.match(/^\/learning\/([^/?#]+)/);
if (!m) throw new ArgumentError(`Invalid LinkedIn Learning course URL: "${s}"`);
slug = m[1];
} else {
const m = s.match(/^\/?learning\/([^/?#]+)/);
slug = m ? m[1] : s;
}
if (!/^[a-zA-Z0-9-_]+$/.test(slug)) {
throw new ArgumentError(`Invalid LinkedIn Learning slug: "${slug}"`);
}
return slug;
}
function buildFetchScript(url, csrf) {
return String.raw`(async () => {
try {
const res = await fetch(${JSON.stringify(url)}, {
credentials: 'include',
headers: {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: 'application/json',
},
});
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
if (!res.ok) return { error: 'HTTP ' + res.status };
return { json: await res.json() };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
})()`;
}
function parseCourse(el, slug) {
const title = normalizeWhitespace(el?.title);
if (!title) return null;
const description = typeof el?.description === 'string'
? el.description
: (el?.description?.text || '');
const duration = el?.duration?.unit === 'SECOND' ? String(el.duration.duration ?? '') : '';
const released = el?.activatedAt ? new Date(el.activatedAt).toISOString().slice(0, 10) : '';
return {
title,
slug,
description,
difficulty: el?.difficultyLevel || '',
duration_sec: duration,
videos_count: el?.videosCount ?? '',
rating: typeof el?.rating?.averageRating === 'number' ? el.rating.averageRating.toFixed(2) : '',
rating_count: el?.rating?.ratingCount ?? '',
released,
url: `https://www.linkedin.com/learning/${slug}`,
};
}
cli({
site: 'linkedin-learning',
name: 'course',
access: 'read',
description: 'Get LinkedIn Learning course detail by slug or course URL',
domain: DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'slug', type: 'string', required: true, positional: true, help: 'Course slug (e.g. agentic-ai-build-your-first-agentic-ai-system) or full /learning/<slug> URL' },
],
columns: ['title', 'slug', 'description', 'difficulty', 'duration_sec', 'videos_count', 'rating', 'rating_count', 'released', 'url'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning course');
const slug = parseSlug(args.slug);
await page.goto('https://www.linkedin.com/learning/');
await page.wait(3);
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
}
const csrf = jsession.replace(/^"|"$/g, '');
const url = `https://www.linkedin.com/learning-api/courses?q=slug&slug=${encodeURIComponent(slug)}`;
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
if (result?.authRequired) {
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
}
if (!result?.json) {
throw new CommandExecutionError(`LinkedIn Learning courses lookup failed: ${result?.error ?? 'no payload'}`);
}
const elements = result.json?.elements;
if (!Array.isArray(elements)) {
throw new CommandExecutionError('LinkedIn Learning courses lookup returned malformed payload: missing elements array');
}
const el = elements[0];
if (!el) {
throw new EmptyResultError(`No LinkedIn Learning course found for slug "${slug}"`);
}
const row = parseCourse(el, slug);
if (!row) {
throw new CommandExecutionError('LinkedIn Learning courses lookup returned malformed course detail: missing title');
}
return [row];
},
});
export const __test__ = { parseSlug, parseCourse };
+114
View File
@@ -0,0 +1,114 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './course.js';
const { parseSlug, parseCourse } = await import('./course.js').then((m) => m.__test__);
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
getCookies: vi.fn().mockResolvedValue(cookies),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('linkedin-learning course', () => {
it('accepts a bare slug', () => {
expect(parseSlug('agentic-ai-build')).toBe('agentic-ai-build');
});
it('extracts a slug from a full /learning/<slug> URL', () => {
expect(parseSlug('https://www.linkedin.com/learning/agentic-ai-build/?foo=1'))
.toBe('agentic-ai-build');
});
it('rejects non-LinkedIn Learning URLs before navigation', () => {
expect(() => parseSlug('https://evil.example/learning/agentic-ai-build')).toThrow(ArgumentError);
expect(() => parseSlug('https://www.linkedin.com/feed/update/123')).toThrow(ArgumentError);
});
it('rejects empty or invalid slugs with ArgumentError', () => {
expect(() => parseSlug('')).toThrow(ArgumentError);
expect(() => parseSlug(' ')).toThrow(ArgumentError);
expect(() => parseSlug('not a slug!')).toThrow(ArgumentError);
});
it('maps a course detail element to the canonical row shape', () => {
const el = {
title: 'Agentic AI: Build Your First Agentic AI System',
description: { text: 'Dive into agentic AI...' },
duration: { duration: 3932, unit: 'SECOND' },
difficultyLevel: 'Intermediate',
videosCount: 18,
rating: { averageRating: 4.5, ratingCount: 259 },
activatedAt: 1774569600000,
};
const row = parseCourse(el, 'agentic-ai-build-your-first-agentic-ai-system');
expect(row.title).toBe('Agentic AI: Build Your First Agentic AI System');
expect(row.slug).toBe('agentic-ai-build-your-first-agentic-ai-system');
expect(row.description).toBe('Dive into agentic AI...');
expect(row.difficulty).toBe('Intermediate');
expect(row.duration_sec).toBe('3932');
expect(row.videos_count).toBe(18);
expect(row.rating).toBe('4.50');
expect(row.rating_count).toBe(259);
expect(row.released).toBe('2026-03-27');
expect(row.url).toBe('https://www.linkedin.com/learning/agentic-ai-build-your-first-agentic-ai-system');
});
it('handles description as a bare string', () => {
const row = parseCourse({ title: 't', description: 'plain string' }, 'x');
expect(row.description).toBe('plain string');
});
it('preserves the full course description', () => {
const text = 'x'.repeat(350);
const row = parseCourse({ title: 't', description: { text } }, 'x');
expect(row.description).toBe(text);
});
it('returns empty fields when upstream omits them', () => {
const row = parseCourse({ title: 't' }, 'x');
expect(row.title).toBe('t');
expect(row.duration_sec).toBe('');
expect(row.rating).toBe('');
expect(row.released).toBe('');
});
it('returns null when upstream omits the core title evidence', () => {
expect(parseCourse({}, 'x')).toBeNull();
expect(parseCourse({ title: ' ' }, 'x')).toBeNull();
});
it('throws AuthRequiredError when JSESSIONID is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/course');
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [{}] } } });
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('throws EmptyResultError when no element matches the slug', async () => {
const cmd = getRegistry().get('linkedin-learning/course');
const page = makePage({ evaluateResult: { json: { elements: [] } } });
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when the elements array is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/course');
const page = makePage({ evaluateResult: { json: { data: {} } } });
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError when the first detail element is malformed', async () => {
const cmd = getRegistry().get('linkedin-learning/course');
const page = makePage({ evaluateResult: { json: { elements: [{}] } } });
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError on fetch errors', async () => {
const cmd = getRegistry().get('linkedin-learning/course');
const page = makePage({ evaluateResult: { error: 'HTTP 500' } });
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
});
});
+155
View File
@@ -0,0 +1,155 @@
/**
* LinkedIn Learning search via the public learning-api REST endpoint.
* Shares cookie session with linkedin.com; no Commercial Use Limit.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DOMAIN = 'www.linkedin.com';
const MAX_LIMIT = 50;
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
}
function parseLimit(value) {
if (value === undefined || value === null || value === '') return 10;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function buildFetchScript(url, csrf) {
return String.raw`(async () => {
try {
const res = await fetch(${JSON.stringify(url)}, {
credentials: 'include',
headers: {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: 'application/json',
},
});
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
if (!res.ok) return { error: 'HTTP ' + res.status };
return { json: await res.json() };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
})()`;
}
function parseAuthors(authors) {
if (!Array.isArray(authors)) return '';
return authors
.map((a) => normalizeWhitespace((a?.firstName ?? '') + ' ' + (a?.lastName ?? '')))
.filter(Boolean)
.join(', ');
}
function durationSeconds(length) {
const ts = length?.['com.linkedin.common.TimeSpan'];
if (!ts || ts.unit !== 'SECOND') return '';
return String(ts.duration ?? '');
}
function averageRating(rating) {
if (!rating) return '';
if (typeof rating.averageRating === 'number') return rating.averageRating.toFixed(2);
if (typeof rating.ratingSum === 'number' && typeof rating.ratingCount === 'number' && rating.ratingCount > 0) {
return (rating.ratingSum / rating.ratingCount).toFixed(2);
}
return '';
}
function parseRow(el, rank) {
const type = el?.entityType || '';
const slug = el?.slug || '';
if (!slug) return null;
return {
rank,
type,
title: el?.headline?.title?.text || '',
instructor: parseAuthors(el?.authors),
difficulty: el?.difficultyLevel || '',
duration_sec: durationSeconds(el?.length),
rating: averageRating(el?.rating),
rating_count: el?.rating?.ratingCount ?? '',
viewers: el?.viewerCount ?? '',
url: slug ? `https://www.linkedin.com/learning/${slug}` : '',
};
}
cli({
site: 'linkedin-learning',
name: 'search',
access: 'read',
description: 'Search LinkedIn Learning courses, videos, and learning paths by keyword',
domain: DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'Search keywords, e.g. "AI agent"' },
{ name: 'limit', type: 'int', default: 10, help: `Maximum results to return (1-${MAX_LIMIT})` },
],
columns: ['rank', 'type', 'title', 'instructor', 'difficulty', 'duration_sec', 'rating', 'rating_count', 'viewers', 'url'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning search');
const keywords = normalizeWhitespace(args.keywords);
if (!keywords) throw new ArgumentError('--keywords is required');
const limit = parseLimit(args.limit);
await page.goto('https://www.linkedin.com/learning/');
await page.wait(3);
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
}
const csrf = jsession.replace(/^"|"$/g, '');
const url = `https://www.linkedin.com/learning-api/searchV2?keywords=${encodeURIComponent(keywords)}&q=keywords`;
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
if (result?.authRequired) {
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
}
if (!result?.json) {
throw new CommandExecutionError(`LinkedIn Learning searchV2 failed: ${result?.error ?? 'no payload'}`);
}
const elements = result.json?.elements;
if (!Array.isArray(elements)) {
throw new CommandExecutionError('LinkedIn Learning searchV2 returned malformed payload: missing elements array');
}
if (elements.length === 0) {
throw new EmptyResultError(`No LinkedIn Learning results for "${keywords}"`);
}
const rows = [];
for (const el of elements) {
if (rows.length >= limit) break;
const row = parseRow(el, rows.length + 1);
if (row) rows.push(row);
}
if (rows.length === 0) {
throw new CommandExecutionError('LinkedIn Learning searchV2 returned no parseable rows with slug identity');
}
return rows;
},
});
export const __test__ = {
normalizeWhitespace,
parseLimit,
parseAuthors,
durationSeconds,
averageRating,
parseRow,
buildFetchScript,
};
+144
View File
@@ -0,0 +1,144 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './search.js';
const { parseLimit, parseAuthors, durationSeconds, averageRating, parseRow, buildFetchScript } = await import('./search.js').then((m) => m.__test__);
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
getCookies: vi.fn().mockResolvedValue(cookies),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('linkedin-learning search', () => {
it('validates --limit without silent clamping', () => {
expect(parseLimit(undefined)).toBe(10);
expect(parseLimit(1)).toBe(1);
expect(parseLimit(50)).toBe(50);
expect(() => parseLimit(0)).toThrow(ArgumentError);
expect(() => parseLimit(51)).toThrow(ArgumentError);
expect(() => parseLimit('abc')).toThrow(ArgumentError);
expect(() => parseLimit(1.5)).toThrow(ArgumentError);
});
it('joins author first/last names', () => {
expect(parseAuthors([{ firstName: 'Jane', lastName: 'Doe' }])).toBe('Jane Doe');
expect(parseAuthors([{ firstName: 'A', lastName: 'B' }, { firstName: 'C', lastName: 'D' }])).toBe('A B, C D');
expect(parseAuthors([])).toBe('');
expect(parseAuthors(undefined)).toBe('');
});
it('extracts duration from TimeSpan only when unit is SECOND', () => {
expect(durationSeconds({ 'com.linkedin.common.TimeSpan': { duration: 600, unit: 'SECOND' } })).toBe('600');
expect(durationSeconds({ 'com.linkedin.common.TimeSpan': { duration: 10, unit: 'MINUTE' } })).toBe('');
expect(durationSeconds(undefined)).toBe('');
});
it('computes average rating from sum/count when averageRating is missing', () => {
expect(averageRating({ ratingSum: 1165, ratingCount: 259 })).toBe('4.50');
expect(averageRating({ averageRating: 4.32 })).toBe('4.32');
expect(averageRating({ ratingSum: 0, ratingCount: 0 })).toBe('');
expect(averageRating(undefined)).toBe('');
});
it('maps a search result element to the canonical row shape', () => {
const el = {
entityType: 'COURSE',
slug: 'agentic-ai-build-your-first-agentic-ai-system',
headline: { title: { text: 'Agentic AI: Build Your First Agentic AI System' } },
authors: [{ firstName: 'Aishwarya', lastName: 'Naresh Reganti' }],
difficultyLevel: 'INTERMEDIATE',
length: { 'com.linkedin.common.TimeSpan': { duration: 3932, unit: 'SECOND' } },
rating: { ratingSum: 1165, ratingCount: 259 },
viewerCount: 25323,
};
expect(parseRow(el, 1)).toEqual({
rank: 1,
type: 'COURSE',
title: 'Agentic AI: Build Your First Agentic AI System',
instructor: 'Aishwarya Naresh Reganti',
difficulty: 'INTERMEDIATE',
duration_sec: '3932',
rating: '4.50',
rating_count: 259,
viewers: 25323,
url: 'https://www.linkedin.com/learning/agentic-ai-build-your-first-agentic-ai-system',
});
});
it('drops rows without slug identity', () => {
const row = parseRow({ entityType: 'COURSE', headline: { title: { text: 't' } } }, 2);
expect(row).toBeNull();
});
it('escapes the URL and csrf into the fetch script as literal strings', () => {
const s = buildFetchScript('https://www.linkedin.com/learning-api/searchV2?keywords=AI', 'csrf-token-value');
expect(s).toContain('"https://www.linkedin.com/learning-api/searchV2?keywords=AI"');
expect(s).toContain('"csrf-token-value"');
expect(s).toContain("'x-restli-protocol-version': '2.0.0'");
expect(s).toContain('authRequired: true');
});
it('throws AuthRequiredError when JSESSIONID cookie is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [] } } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('throws AuthRequiredError when the fetch returns 403', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { authRequired: true, status: 403 } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('throws CommandExecutionError when the upstream payload is empty', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { error: 'fetch failed: socket' } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws EmptyResultError when zero elements come back', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { json: { elements: [] } } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when the elements array is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { json: { data: {} } } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError when elements lack slug identity', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { json: { elements: [{ headline: { title: { text: 'No slug' } } }] } } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('rejects empty keywords with ArgumentError before navigation', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const page = makePage({ evaluateResult: { json: { elements: [] } } });
await expect(cmd.func(page, { keywords: ' ', limit: 5 })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('returns ranked rows when the API responds normally', async () => {
const cmd = getRegistry().get('linkedin-learning/search');
const elements = [
{ entityType: 'COURSE', slug: 'a', headline: { title: { text: 'Course A' } }, authors: [{ firstName: 'Inst', lastName: 'A' }], difficultyLevel: 'BEGINNER', length: { 'com.linkedin.common.TimeSpan': { duration: 100, unit: 'SECOND' } } },
{ entityType: 'COURSE', headline: { title: { text: 'No slug' } } },
{ entityType: 'VIDEO', slug: 'b', headline: { title: { text: 'Video B' } } },
];
const page = makePage({ evaluateResult: { json: { elements } } });
const rows = await cmd.func(page, { keywords: 'test', limit: 5 });
expect(rows).toHaveLength(2);
expect(rows[0].rank).toBe(1);
expect(rows[0].title).toBe('Course A');
expect(rows[1].title).toBe('Video B');
expect(rows[1].url).toBe('https://www.linkedin.com/learning/b');
});
});
+133
View File
@@ -0,0 +1,133 @@
/**
* LinkedIn Learning personalized recommendations via the
* feedRecommendationGroups carousels endpoint. The `learner` view
* returns a small set of carousels (e.g. "Top picks for you"); this
* command flattens the cards across them into a ranked list.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const DOMAIN = 'www.linkedin.com';
const MAX_LIMIT = 50;
const MAX_PER_CAROUSEL = 25;
function parseLimit(value) {
if (value === undefined || value === null || value === '') return 10;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function buildFetchScript(url, csrf) {
return String.raw`(async () => {
try {
const res = await fetch(${JSON.stringify(url)}, {
credentials: 'include',
headers: {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: 'application/json',
},
});
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
if (!res.ok) return { error: 'HTTP ' + res.status };
return { json: await res.json() };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
})()`;
}
function parseCard(card, group, rank) {
const slug = card?.slug || '';
if (!slug) return null;
return {
rank,
group: group?.title?.text || group?.annotation || '',
type: card?.entityType || card?.localizedEntityName || '',
title: card?.title?.text || card?.headline?.title?.text || card?.headline?.text || '',
difficulty: card?.difficultyLevel || '',
viewers: card?.viewerCount ?? '',
url: slug ? `https://www.linkedin.com/learning/${slug}` : '',
};
}
cli({
site: 'linkedin-learning',
name: 'trending',
access: 'read',
description: 'Browse LinkedIn Learning recommended courses across personalized carousels',
domain: DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'limit', type: 'int', default: 10, help: `Maximum results to return (1-${MAX_LIMIT})` },
],
columns: ['rank', 'group', 'type', 'title', 'difficulty', 'viewers', 'url'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning trending');
const limit = parseLimit(args.limit);
await page.goto('https://www.linkedin.com/learning/');
await page.wait(3);
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
}
const csrf = jsession.replace(/^"|"$/g, '');
const url = `https://www.linkedin.com/learning-api/feedRecommendationGroups?countPerCarousel=${MAX_PER_CAROUSEL}&q=learner`;
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
if (result?.authRequired) {
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
}
if (!result?.json) {
throw new CommandExecutionError(`LinkedIn Learning feedRecommendationGroups failed: ${result?.error ?? 'no payload'}`);
}
const groups = result.json?.elements;
if (!Array.isArray(groups)) {
throw new CommandExecutionError('LinkedIn Learning feedRecommendationGroups returned malformed payload: missing elements array');
}
const rows = [];
const seen = new Set();
let rank = 1;
let sawCards = false;
for (const group of groups) {
const carousels = Array.isArray(group?.carousels) ? group.carousels : [];
for (const carousel of carousels) {
const cards = Array.isArray(carousel?.cards) ? carousel.cards : [];
for (const card of cards) {
sawCards = true;
if (rows.length >= limit) break;
const slug = card?.slug;
if (!slug || seen.has(slug)) continue;
seen.add(slug);
const row = parseCard(card, carousel, rank);
if (!row) continue;
rows.push(row);
rank += 1;
}
if (rows.length >= limit) break;
}
if (rows.length >= limit) break;
}
if (rows.length === 0) {
if (sawCards) {
throw new CommandExecutionError('LinkedIn Learning feedRecommendationGroups returned no parseable cards with slug identity');
}
throw new EmptyResultError('LinkedIn Learning returned no personalized recommendations');
}
return rows;
},
});
export const __test__ = { parseLimit, parseCard };
+123
View File
@@ -0,0 +1,123 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './trending.js';
const { parseLimit, parseCard } = await import('./trending.js').then((m) => m.__test__);
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
getCookies: vi.fn().mockResolvedValue(cookies),
evaluate: vi.fn().mockResolvedValue(evaluateResult),
};
}
describe('linkedin-learning trending', () => {
it('validates --limit without silent clamping', () => {
expect(parseLimit(undefined)).toBe(10);
expect(parseLimit(50)).toBe(50);
expect(() => parseLimit(0)).toThrow(ArgumentError);
expect(() => parseLimit(51)).toThrow(ArgumentError);
});
it('maps a carousel card to the canonical row shape', () => {
const card = {
entityType: 'COURSE',
slug: 'storytelling-editing',
difficultyLevel: 'BEGINNER_INTERMEDIATE',
description: { text: 'Go beyond basic video editing.' },
viewerCount: 12345,
headline: { title: { text: 'The Art of Storytelling through Editing' } },
};
const group = { annotation: 'TOP_PICKS', title: { text: 'Top picks for you' } };
expect(parseCard(card, group, 1)).toEqual({
rank: 1,
group: 'Top picks for you',
type: 'COURSE',
title: 'The Art of Storytelling through Editing',
difficulty: 'BEGINNER_INTERMEDIATE',
viewers: 12345,
url: 'https://www.linkedin.com/learning/storytelling-editing',
});
});
it('drops cards without slug identity', () => {
expect(parseCard({ title: { text: 'No slug' } }, { title: { text: 'G' } }, 1)).toBeNull();
});
it('flattens carousels and dedups cards across them', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const page = makePage({
evaluateResult: {
json: {
elements: [{
carousels: [
{
title: { text: 'Top picks' },
cards: [
{ slug: 'a', headline: { title: { text: 'Course A' } } },
{ headline: { title: { text: 'No slug' } } },
{ slug: 'b', headline: { title: { text: 'Course B' } } },
],
},
{
title: { text: 'Trending in your network' },
cards: [
{ slug: 'a', headline: { title: { text: 'Dup of A' } } },
{ slug: 'c', headline: { title: { text: 'Course C' } } },
],
},
],
}],
},
},
});
const rows = await cmd.func(page, { limit: 5 });
expect(rows.map((r) => r.title)).toEqual(['Course A', 'Course B', 'Course C']);
expect(rows.map((r) => r.rank)).toEqual([1, 2, 3]);
expect(rows[0].group).toBe('Top picks');
expect(rows[2].group).toBe('Trending in your network');
});
it('respects --limit', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const cards = Array.from({ length: 6 }, (_, i) => ({ slug: `s${i}`, headline: { title: { text: `T${i}` } } }));
const page = makePage({
evaluateResult: {
json: { elements: [{ carousels: [{ title: { text: 'G' }, cards }] }] },
},
});
const rows = await cmd.func(page, { limit: 3 });
expect(rows).toHaveLength(3);
});
it('throws AuthRequiredError when JSESSIONID is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [] } } });
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('throws EmptyResultError when no carousels yield cards', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const page = makePage({ evaluateResult: { json: { elements: [{ carousels: [] }] } } });
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
});
it('throws CommandExecutionError when the elements array is missing', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const page = makePage({ evaluateResult: { json: { data: {} } } });
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws CommandExecutionError when cards lack slug identity', async () => {
const cmd = getRegistry().get('linkedin-learning/trending');
const page = makePage({
evaluateResult: {
json: { elements: [{ carousels: [{ title: { text: 'G' }, cards: [{ headline: { title: { text: 'No slug' } } }] }] }] },
},
});
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
});
+401
View File
@@ -0,0 +1,401 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
}
function normalizeName(value) {
return normalizeWhitespace(value)
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
.replace(/\s+LinkedIn.*$/i, '')
.replace(/\b(p\.?eng\.?|cpa|mba|ph\.?d\.?)\b/ig, '')
.replace(/[^\p{L}\p{N}\s.'-]+/gu, ' ')
.replace(/\s+/g, ' ')
.trim()
.toLowerCase();
}
function nameTokens(value) {
return normalizeName(value)
.replace(/[.'-]+/g, ' ')
.split(/\s+/)
.map((token) => token.trim())
.filter((token) => token.length >= 2);
}
function matchInvitationName(candidate, expected) {
const candidateName = normalizeName(candidate);
const expectedName = normalizeName(expected);
if (!candidateName || !expectedName) return false;
if (candidateName === expectedName) return true;
if (candidateName.includes(expectedName) || expectedName.includes(candidateName)) return true;
const candidateTokens = new Set(nameTokens(candidateName));
const expectedTokens = nameTokens(expectedName);
if (expectedTokens.length < 2 || candidateTokens.size < 2) return false;
const matched = expectedTokens.filter((token) => candidateTokens.has(token)).length;
return matched >= 2 && matched / expectedTokens.length >= 0.8;
}
function isLinkedInHost(hostname) {
const host = String(hostname || '').toLowerCase();
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
}
function canonicalizeLinkedInProfileUrl(value) {
const raw = normalizeWhitespace(value);
if (!raw) return '';
try {
const url = new URL(raw);
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
const match = url.pathname.match(/^\/in\/([^/]+)\/?$/i);
if (!match || !match[1]) return '';
// LinkedIn redirects country subdomains (ca./uk./...) to www.; normalize the
// host so an expected `ca.linkedin.com/in/x` matches the landed `www.linkedin.com/in/x`.
url.hostname = 'www.linkedin.com';
url.hash = '';
url.search = '';
if (!url.pathname.endsWith('/')) url.pathname += '/';
return url.toString();
}
catch {
return '';
}
}
function requireStringArg(args, key, label = key) {
const value = normalizeWhitespace(args[key]);
if (!value) throw new ArgumentError(`${label} is required`);
return value;
}
function requireLinkedInProfileUrl(value, label) {
const url = canonicalizeLinkedInProfileUrl(value);
if (!url) throw new ArgumentError(`${label} must be an exact https://www.linkedin.com/in/<profile>/ URL`);
return url;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function clampNote(note) {
const value = normalizeWhitespace(note);
if (value.length > 300) throw new ArgumentError('--note must be 300 characters or fewer for LinkedIn connection requests');
return value;
}
function canonicalizeLinkedInInviteUrl(value) {
try {
const url = new URL(normalizeWhitespace(value), 'https://www.linkedin.com');
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
if (!/^\/preload\/custom-invite\/?$/i.test(url.pathname)) return '';
url.hostname = 'www.linkedin.com';
url.hash = '';
if (!url.pathname.endsWith('/')) url.pathname += '/';
return url.toString();
}
catch {
return '';
}
}
function assessProfileSafety(probe, expectedName, expectedProfileUrl) {
const expected = normalizeWhitespace(expectedName);
const actual = normalizeWhitespace(probe?.name || '');
const expectedUrl = canonicalizeLinkedInProfileUrl(expectedProfileUrl);
const actualUrl = canonicalizeLinkedInProfileUrl(probe?.url || '');
if (probe?.authRequired) return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'auth_required', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
if (!actual) return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_not_found', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
if (expected && normalizeName(actual) !== normalizeName(expected)) {
return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_mismatch', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
}
if (expectedUrl && actualUrl && expectedUrl !== actualUrl) {
return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_url_mismatch', expectedValue: expectedUrl, actualValue: actualUrl, observedUrl: actualUrl };
}
if (probe?.alreadyConnected) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'already_connected', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
if (probe?.pending) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connection_pending', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
if (!probe?.connectAvailable) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connect_button_not_found', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
return { ok: true, safety: 'connectable', connectable: true, blockReason: 'verified', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
}
function buildProfileProbeScript() {
return String.raw`(() => {
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const text = document.body ? (document.body.innerText || '') : '';
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|| /captcha|verification required/i.test(text);
const main = document.querySelector('main') || document.body;
// LinkedIn profile pages no longer expose the name in an <h1>; the heading
// markup churns, but document.title is a stable "Name | LinkedIn" pattern.
const heading = main?.querySelector('h1, .text-heading-xlarge, [class*="heading-xlarge"]');
const titleName = clean((document.title || '')
.replace(/^\(\d+\+?\)\s*/, '')
.replace(/\s*[|]\s*LinkedIn\s*$/i, ''));
const name = clean(heading?.innerText || heading?.textContent || '') || titleName;
const buttons = Array.from(document.querySelectorAll('button, [role="button"], a')).filter((el) => el.offsetParent !== null);
const buttonLabels = buttons.map((button) => clean(button.innerText || button.textContent || button.getAttribute('aria-label'))).filter(Boolean);
const lowerLabels = buttonLabels.map((label) => label.toLowerCase());
const alreadyConnected = lowerLabels.some((label) => label === 'message' || label.includes('1st degree connection'));
const pending = lowerLabels.some((label) => label === 'pending' || label.includes('pending'));
const connectAvailable = lowerLabels.some((label) => label === 'connect' || label.startsWith('connect ') || label.includes(' invite '));
// The Connect control is an <a> linking to LinkedIn's invitation route
// (/preload/custom-invite/?vanityName=...). Capture it so the sender can
// navigate straight to the invite dialog.
const connectAnchor = buttons.find((el) => el.tagName === 'A'
&& /^connect$/i.test(clean(el.innerText || el.textContent || el.getAttribute('aria-label'))));
const connectHref = connectAnchor ? (connectAnchor.getAttribute('href') || '') : '';
return {
url: location.href,
title: document.title || '',
name,
authRequired,
alreadyConnected,
pending,
connectAvailable,
connectHref,
buttonLabels: buttonLabels.slice(0, 30),
bodyText: text,
};
})()`;
}
// Runs in-page on LinkedIn's invitation route (/preload/custom-invite/...),
// where the "Add a note to your invitation?" dialog is already open.
function buildSentInvitationsProbeScript(expectedName, expectedProfileUrl) {
return String.raw`(() => {
const expectedName = ${JSON.stringify(expectedName)};
const expectedUrl = ${JSON.stringify(canonicalizeLinkedInProfileUrl(expectedProfileUrl))};
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const normName = (s) => clean(s)
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
.replace(/\s+LinkedIn.*$/i, '')
.replace(/\b(p\.?eng\.?|cpa|mba|ph\.?d\.?)\b/ig, '')
.replace(/[^\p{L}\p{N}\s.'-]+/gu, ' ')
.replace(/\s+/g, ' ')
.trim()
.toLowerCase();
const tokens = (s) => normName(s).replace(/[.'-]+/g, ' ').split(/\s+/).map((t) => t.trim()).filter((t) => t.length >= 2);
const nameMatchesReasonably = (candidate, expected) => {
const c = normName(candidate);
const e = normName(expected);
if (!c || !e) return false;
if (c === e || c.includes(e) || e.includes(c)) return true;
const candidateTokens = new Set(tokens(c));
const expectedTokens = tokens(e);
if (expectedTokens.length < 2 || candidateTokens.size < 2) return false;
const matched = expectedTokens.filter((token) => candidateTokens.has(token)).length;
return matched >= 2 && matched / expectedTokens.length >= 0.8;
};
const canon = (value) => {
try {
const url = new URL(value, 'https://www.linkedin.com');
if (!/^\/in\/[^/]+\/?$/i.test(url.pathname)) return '';
url.protocol = 'https:';
url.hostname = 'www.linkedin.com';
url.hash = '';
url.search = '';
if (!url.pathname.endsWith('/')) url.pathname += '/';
return url.toString();
} catch { return ''; }
};
const text = document.body ? (document.body.innerText || '') : '';
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|| /captcha|verification required/i.test(text);
if (authRequired) return { authRequired: true, found: false, matchedName: '', matchedUrl: '', visibleNames: [] };
const structuralRows = Array.from(document.querySelectorAll('li, article, [data-view-name], .mn-invitation-card'));
const linkRows = Array.from(document.querySelectorAll('a[href*="/in/"]'))
.map((a) => a.closest('li') || a.closest('[data-view-name]') || a.closest('[class*="invitation"]') || a.closest('div'))
.filter(Boolean);
const rows = Array.from(new Set([...structuralRows, ...linkRows]));
const visibleNames = [];
for (const row of rows.slice(0, 25)) {
const rowText = clean(row.innerText || row.textContent || '');
if (!rowText) continue;
const link = Array.from(row.querySelectorAll('a[href*="/in/"]'))
.map((a) => ({ href: canon(a.href || a.getAttribute('href') || ''), text: clean(a.innerText || a.textContent || '') }))
.find((a) => a.href || a.text);
const candidateName = clean(link?.text || row.querySelector('span[aria-hidden="true"], h3, h2')?.textContent || rowText.split('\n')[0]);
if (candidateName) visibleNames.push(candidateName);
const candidateUrl = link?.href || '';
const nameMatches = expectedName && candidateName && nameMatchesReasonably(candidateName, expectedName);
const urlMatches = expectedUrl && candidateUrl && candidateUrl === expectedUrl;
if (urlMatches || nameMatches) return { authRequired: false, found: true, matchedName: candidateName, matchedUrl: candidateUrl, visibleNames: visibleNames.slice(0, 20) };
}
return { authRequired: false, found: false, matchedName: '', matchedUrl: '', visibleNames: visibleNames.slice(0, 20) };
})()`;
}
function buildInviteScript(note) {
return String.raw`(async () => {
const note = ${JSON.stringify(note)};
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
const jitter = async (min = 450, max = 1150) => sleep(min + Math.floor(Math.random() * (max - min + 1)));
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const visible = (el) => el && el.offsetParent !== null;
const label = (el) => clean(el?.innerText || el?.textContent || el?.getAttribute('aria-label'));
const dialog = () => document.querySelector('[role="dialog"]');
const dialogButton = (pattern) => {
const dlg = dialog();
if (!dlg) return null;
return Array.from(dlg.querySelectorAll('button, [role="button"]')).filter(visible)
.find((button) => pattern.test(label(button)));
};
if (!dialog()) return { ok: false, status: 'blocked', reason: 'invite_dialog_not_found' };
if (!note) {
const sendDirect = dialogButton(/^send without a note$/i) || dialogButton(/^send$/i);
if (!sendDirect) return { ok: false, status: 'blocked', reason: 'send_button_not_found' };
await jitter();
sendDirect.click();
await jitter(1400, 2400);
return { ok: true, status: 'sent', reason: 'invitation_sent_without_note' };
}
const addNote = dialogButton(/^add a note$/i);
if (!addNote) return { ok: false, status: 'blocked', reason: 'add_note_button_not_found' };
await jitter();
addNote.click();
await jitter(800, 1400);
const textarea = document.querySelector('#custom-message')
|| Array.from(document.querySelectorAll('textarea')).find(visible);
if (!textarea) return { ok: false, status: 'blocked', reason: 'note_textarea_not_found' };
textarea.focus();
// React tracks textarea values through the native setter; assigning .value
// directly would leave component state (and the Send button) unchanged.
const nativeSetter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set;
nativeSetter.call(textarea, note);
textarea.dispatchEvent(new Event('input', { bubbles: true }));
textarea.dispatchEvent(new Event('change', { bubbles: true }));
await jitter(700, 1300);
const send = dialogButton(/^send$/i);
if (!send) return { ok: false, status: 'blocked', reason: 'send_button_not_found' };
if (send.disabled || send.getAttribute('aria-disabled') === 'true') {
return { ok: false, status: 'blocked', reason: 'send_button_disabled' };
}
send.click();
await jitter(1400, 2400);
return { ok: true, status: 'sent', reason: 'invitation_sent_with_note' };
})()`;
}
async function probeProfile(page) {
return unwrapEvaluateResult(await page.evaluate(buildProfileProbeScript()));
}
cli({
site: 'linkedin',
name: 'connect',
access: 'write',
description: 'Fail-closed LinkedIn connection request sender that verifies the exact profile before optionally sending a note',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'profile-url', type: 'string', required: true, positional: true, help: 'Exact LinkedIn profile URL to open and verify' },
{ name: 'expected-name', type: 'string', required: true, help: 'Expected visible profile name' },
{ name: 'note', type: 'string', required: false, default: '', help: 'Optional connection note, max 300 chars' },
{ name: 'send', type: 'bool', required: false, default: false, help: 'Actually click Send. Default is dry-run verification only.' },
],
columns: ['status', 'recipient', 'reason', 'profile_url', 'note_chars', 'connectable', 'delivery_verified', 'matched_invitation_name', 'matched_invitation_url', 'actualValue', 'blockReason', 'expectedValue', 'observedUrl', 'safety'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin connect');
const profileUrl = requireLinkedInProfileUrl(requireStringArg(args, 'profile-url', '--profile-url'), '--profile-url');
const expectedName = requireStringArg(args, 'expected-name', '--expected-name');
const note = clampNote(args.note || '');
await page.goto(profileUrl);
await page.wait(6);
let probe = await probeProfile(page);
// The name resolves early (from document.title), but the profile action
// buttons (Connect / Message / Pending) render later. Keep probing until
// the action state has resolved, not merely until the name is visible.
for (let attempt = 0; attempt < 8; attempt += 1) {
const resolved = probe?.name
&& (probe.connectAvailable || probe.alreadyConnected || probe.pending);
if (resolved) break;
await page.wait(2);
probe = await probeProfile(page);
}
const safety = assessProfileSafety(probe, expectedName, profileUrl);
if (safety.blockReason === 'auth_required') {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn connect requires an active signed-in LinkedIn browser session.');
}
if (!safety.ok && safety.safety === 'routine_non_connectable') {
return [{ status: 'not_connectable', recipient: safety.actualValue, reason: safety.blockReason, profile_url: safety.observedUrl, note_chars: note.length, connectable: false }];
}
if (!safety.ok) {
throw new CommandExecutionError(
`LinkedIn connect blocked: ${safety.blockReason}`,
`Expected ${safety.expectedValue}; actual ${safety.actualValue || 'not_visible'} at ${safety.observedUrl || 'url_not_available'}\nButtons: ${(probe?.buttonLabels || []).join(' | ')}`,
);
}
if (!args.send) {
return [{ status: 'connectable_dry_run', recipient: safety.actualValue, reason: safety.blockReason, profile_url: safety.observedUrl, note_chars: note.length, connectable: true }];
}
const inviteHref = probe?.connectHref || '';
if (!inviteHref) {
throw new CommandExecutionError('LinkedIn connect blocked: connect_link_not_found');
}
const inviteUrl = canonicalizeLinkedInInviteUrl(inviteHref);
if (!inviteUrl) {
throw new CommandExecutionError('LinkedIn connect blocked: invalid_connect_link');
}
await page.goto(inviteUrl);
await page.wait(6);
let result = unwrapEvaluateResult(await page.evaluate(buildInviteScript(note)));
if (result?.reason === 'invite_dialog_not_found') {
await page.wait(5);
result = unwrapEvaluateResult(await page.evaluate(buildInviteScript(note)));
}
if (!result?.ok) throw new CommandExecutionError(`LinkedIn connect blocked: ${result?.reason || 'send_failed'}`);
// LinkedIn can take a few seconds after the Send click to materialize the
// new invite in /mynetwork/invitation-manager/sent/. Wait before the
// first check, then retry page loads for propagation lag.
await page.wait(8);
let sentProbe = null;
for (let attempt = 0; attempt < 3; attempt += 1) {
await page.goto('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
await page.wait(attempt === 0 ? 6 : 4);
sentProbe = unwrapEvaluateResult(await page.evaluate(buildSentInvitationsProbeScript(expectedName, profileUrl)));
if (sentProbe?.found || sentProbe?.authRequired) break;
if (attempt < 2) await page.wait(5);
}
if (sentProbe?.authRequired) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn sent-invitations verification requires an active signed-in LinkedIn browser session.');
}
const verified = Boolean(sentProbe?.found);
return [{
status: verified ? 'sent_verified' : 'send_unverified',
recipient: safety.actualValue,
reason: verified ? 'sent_invitation_verified' : 'sent_invitation_not_found_after_retries',
profile_url: safety.observedUrl,
note_chars: note.length,
connectable: true,
delivery_verified: verified,
matched_invitation_name: sentProbe?.matchedName || '',
matched_invitation_url: sentProbe?.matchedUrl || '',
}];
},
});
export const __test__ = {
normalizeWhitespace,
normalizeName,
matchInvitationName,
canonicalizeLinkedInProfileUrl,
canonicalizeLinkedInInviteUrl,
unwrapEvaluateResult,
clampNote,
assessProfileSafety,
buildSentInvitationsProbeScript,
};
+213
View File
@@ -0,0 +1,213 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import './connect.js';
const {
normalizeName,
matchInvitationName,
canonicalizeLinkedInProfileUrl,
canonicalizeLinkedInInviteUrl,
unwrapEvaluateResult,
clampNote,
assessProfileSafety,
} = await import('./connect.js').then((m) => m.__test__);
function makeFakePage(probe, sendResult = { ok: true, status: 'sent', reason: 'connection_request_sent' }) {
return {
goto: vi.fn(async () => undefined),
wait: vi.fn(async () => undefined),
evaluate: vi.fn(async (script) => {
const text = String(script);
if (text.includes('custom-message') || text.includes('invite_dialog_not_found')) return sendResult;
return probe;
}),
};
}
function makeSequentialFakePage(values) {
let index = 0;
return {
goto: vi.fn(async () => undefined),
wait: vi.fn(async () => undefined),
evaluate: vi.fn(async (script) => {
const text = String(script);
if (text.includes('custom-message') || text.includes('invite_dialog_not_found')) return { ok: true, status: 'sent', reason: 'connection_request_sent' };
const value = values[Math.min(index, values.length - 1)];
index += 1;
return value;
}),
};
}
describe('linkedin connect helpers', () => {
it('normalizes names and profile URLs', () => {
expect(normalizeName('Jane Doe • 2nd degree connection')).toBe('jane doe');
expect(matchInvitationName('Jane Doe, P.Eng.', ' jane doe ')).toBe(true);
expect(matchInvitationName('Jane Q. Doe', 'Jane Doe')).toBe(true);
expect(matchInvitationName('Janet Doe', 'Jane Doe')).toBe(false);
expect(canonicalizeLinkedInProfileUrl('https://www.linkedin.com/in/jane/?mini=true#x'))
.toBe('https://www.linkedin.com/in/jane/');
expect(canonicalizeLinkedInProfileUrl('https://ca.linkedin.com/in/jane/?mini=true#x'))
.toBe('https://www.linkedin.com/in/jane/');
expect(canonicalizeLinkedInProfileUrl('https://www.linkedin.com/company/opencli/')).toBe('');
expect(canonicalizeLinkedInProfileUrl('https://evil-linkedin.com/in/jane/')).toBe('');
expect(canonicalizeLinkedInProfileUrl('http://www.linkedin.com/in/jane/')).toBe('');
});
it('only accepts LinkedIn invitation route hrefs for sending', () => {
expect(canonicalizeLinkedInInviteUrl('/preload/custom-invite/?vanityName=jane'))
.toBe('https://www.linkedin.com/preload/custom-invite/?vanityName=jane');
expect(canonicalizeLinkedInInviteUrl('https://www.linkedin.com/feed/')).toBe('');
expect(canonicalizeLinkedInInviteUrl('https://evil-linkedin.com/preload/custom-invite/?vanityName=jane')).toBe('');
});
it('unwraps browser bridge evaluate envelopes', () => {
expect(unwrapEvaluateResult({ session: 'site:linkedin:1', data: { ok: true } })).toEqual({ ok: true });
const raw = { ok: true };
expect(unwrapEvaluateResult(raw)).toBe(raw);
});
it('enforces LinkedIn note length', () => {
expect(clampNote(' hello\nthere ')).toBe('hello there');
expect(() => clampNote('x'.repeat(301))).toThrow('--note must be 300 characters or fewer');
});
it('fails closed on wrong profile name, pending state, or missing connect button', () => {
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true }, 'Janet Doe', 'https://www.linkedin.com/in/jane/').blockReason)
.toBe('profile_name_mismatch');
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', pending: true, connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/').blockReason)
.toBe('connection_pending');
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/' }, 'Jane Doe', 'https://www.linkedin.com/in/jane/').blockReason)
.toBe('connect_button_not_found');
});
it('classifies routine non-connectable profiles separately from unsafe blocks', () => {
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', alreadyConnected: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
.toMatchObject({ ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'already_connected' });
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', pending: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
.toMatchObject({ ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connection_pending' });
expect(assessProfileSafety({ name: 'Wrong Person', url: 'https://www.linkedin.com/in/wrong/', connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
.toMatchObject({ ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_mismatch' });
});
it('passes only when profile url, name, and connect affordance all match', () => {
const result = assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/?mini=true', connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/');
expect(result).toMatchObject({ ok: true, blockReason: 'verified', actualValue: 'Jane Doe', connectable: true });
});
});
describe('linkedin connect command', () => {
it('registers as a write command and dry-runs by default', async () => {
const command = getRegistry().get('linkedin/connect');
expect(command).toBeDefined();
expect(command.access).toBe('write');
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] });
const rows = await command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
});
expect(rows[0]).toMatchObject({ status: 'connectable_dry_run', recipient: 'Jane Doe', reason: 'verified', connectable: true });
expect(page.evaluate).toHaveBeenCalledTimes(1);
});
it('returns a clean not_connectable dry-run row for routine blocked states', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', alreadyConnected: true, buttonLabels: ['Message'] });
const rows = await command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
});
expect(rows[0]).toMatchObject({ status: 'not_connectable', recipient: 'Jane Doe', reason: 'already_connected', connectable: false });
});
it('does not send when recipient verification fails', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeFakePage({ name: 'Wrong Person', url: 'https://www.linkedin.com/in/wrong/', connectAvailable: true, buttonLabels: ['Connect'] });
await expect(command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
send: true,
})).rejects.toBeInstanceOf(CommandExecutionError);
expect(page.evaluate).toHaveBeenCalledTimes(1);
});
it('rejects non-profile URLs before navigating', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeFakePage({});
await expect(command.func(page, {
'profile-url': 'https://www.linkedin.com/company/opencli/',
'expected-name': 'Jane Doe',
send: true,
})).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('blocks send when the connect link is not LinkedIn invitation route', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: 'https://www.linkedin.com/feed/', buttonLabels: ['Connect'] });
await expect(command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
send: true,
})).rejects.toThrow('invalid_connect_link');
});
it('sends only when --send is true after verification and sent-invitations confirms delivery', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeSequentialFakePage([
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
{ found: true, matchedName: 'Jane Doe', matchedUrl: 'https://www.linkedin.com/in/jane/' },
]);
const rows = await command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
send: true,
});
expect(rows[0]).toMatchObject({ status: 'sent_verified', recipient: 'Jane Doe', reason: 'sent_invitation_verified', delivery_verified: true });
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
});
it('retries sent-invitations verification before reporting unverified', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeSequentialFakePage([
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
{ found: false, matchedName: '', matchedUrl: '', visibleNames: ['Other Person'] },
{ found: false, matchedName: '', matchedUrl: '', visibleNames: ['Other Person'] },
{ found: true, matchedName: 'Jane Doe, P.Eng.', matchedUrl: '' },
]);
const rows = await command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
send: true,
});
expect(rows[0]).toMatchObject({ status: 'sent_verified', recipient: 'Jane Doe', reason: 'sent_invitation_verified', delivery_verified: true, matched_invitation_name: 'Jane Doe, P.Eng.' });
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
expect(page.evaluate).toHaveBeenCalledTimes(5);
});
it('does not report sent when sent-invitations verification fails after retries', async () => {
const command = getRegistry().get('linkedin/connect');
const page = makeSequentialFakePage([
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
{ found: false, matchedName: '', matchedUrl: '' },
{ found: false, matchedName: '', matchedUrl: '' },
{ found: false, matchedName: '', matchedUrl: '' },
]);
const rows = await command.func(page, {
'profile-url': 'https://www.linkedin.com/in/jane/',
'expected-name': 'Jane Doe',
note: 'quick note',
send: true,
});
expect(rows[0]).toMatchObject({ status: 'send_unverified', recipient: 'Jane Doe', reason: 'sent_invitation_not_found_after_retries', delivery_verified: false });
expect(page.evaluate).toHaveBeenCalledTimes(5);
});
});
+234
View File
@@ -0,0 +1,234 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'linkedin.com';
const MESSAGING_URL = 'https://www.linkedin.com/messaging/';
const MIN_LIMIT = 1;
const MAX_LIMIT = 100;
const DEFAULT_LIMIT = 40;
// ── Why this command reads an API response instead of scraping the DOM ──
//
// LinkedIn's messaging UI is a realtime, virtualized SPA. Scraping the rendered
// conversation list is brittle: rows lazy-render, the list virtualizes, and the
// markup churns. Instead we let the page load /messaging/ exactly as a human
// would, which makes the page fire its own `messengerConversations` GraphQL
// call. We then re-issue that same request (URL lifted from the Performance API,
// so the rotating queryId is always current) and parse LinkedIn's normalized
// JSON. Same session, same origin, same request the page already makes.
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function threadUrl(threadId) {
return threadId ? `https://www.linkedin.com/messaging/thread/${threadId}/` : '';
}
// Runs in-page: locate the messengerConversations request the page already fired.
// Prefers the category-scoped query (the primary inbox) over the sync-token query.
function findMessagingApiUrl() {
if (/\/(login|checkpoint|authwall|uas)/i.test(location.pathname)) return { loginRequired: true };
const urls = performance.getEntriesByType('resource').map((e) => e.name);
const matches = (re) => urls.find((u) => /messengerConversations\.[a-f0-9]+/i.test(u) && re.test(u));
const url =
matches(/PRIMARY_INBOX/i) ||
matches(/conversationCategoryPredicate/i) ||
urls.find((u) => /messengerConversations\.[a-f0-9]+/i.test(u) && /mailboxUrn/i.test(u));
if (!url) return { url: null };
const mb = url.match(/mailboxUrn:(urn[^,)&]+)/i);
return { url, mailboxUrn: mb ? decodeURIComponent(mb[1]) : '' };
}
// Runs in-page: re-issue the messaging request with the session's csrf token.
async function fetchMessagingApi(url, csrf) {
try {
const res = await fetch(url, {
credentials: 'include',
headers: {
'csrf-token': csrf,
accept: 'application/vnd.linkedin.normalized+json+2.1',
'x-restli-protocol-version': '2.0.0',
},
});
if (res.status === 401 || res.status === 403) return { authRequired: true, error: 'HTTP ' + res.status };
if (!res.ok) return { error: 'HTTP ' + res.status };
return { json: await res.json() };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
}
// Parse LinkedIn's normalized messaging JSON into plain conversation rows.
// `included` is a flat entity array; conversations reference participants and
// messages by URN, which we resolve through a urn->entity index. Exported for
// unit testing against a captured fixture.
function parseConversations(normalized, mailboxUrn) {
if (!normalized || typeof normalized !== 'object' || Array.isArray(normalized) || !Array.isArray(normalized.included)) {
throw new CommandExecutionError('LinkedIn messaging API returned malformed normalized payload: missing included array');
}
const included = normalized.included;
const byUrn = new Map();
for (const o of included) {
if (o && o.entityUrn) byUrn.set(o.entityUrn, o);
}
const norm = (s) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim();
const participantInfo = (p) => {
if (!p) return { name: '', kind: '' };
const pt = p.participantType || {};
if (pt.organization && pt.organization.name) return { name: norm(pt.organization.name.text), kind: 'organization' };
if (pt.member) {
const fn = pt.member.firstName && pt.member.firstName.text;
const ln = pt.member.lastName && pt.member.lastName.text;
return { name: norm([fn, ln].filter(Boolean).join(' ')), kind: 'member' };
}
if (pt.agent && pt.agent.name) return { name: norm(pt.agent.name.text), kind: 'agent' };
return { name: '', kind: '' };
};
const entries = [];
for (const conv of included) {
if (!conv || conv.$type !== 'com.linkedin.messenger.Conversation') continue;
const threadId = String(conv.backendUrn || '').replace(/^urn:li:messagingThread:/, '');
if (!threadId) {
throw new CommandExecutionError('LinkedIn messaging API returned a conversation without thread id');
}
const others = [];
let counterpartyKind = '';
for (const urn of conv['*conversationParticipants'] || []) {
const p = byUrn.get(urn);
if (!p) continue;
if (mailboxUrn && p.hostIdentityUrn === mailboxUrn) continue; // exclude the inbox owner
const info = participantInfo(p);
if (info.name) {
others.push(info.name);
if (!counterpartyKind) counterpartyKind = info.kind;
}
}
const msgUrns = (conv.messages && conv.messages['*elements']) || [];
const lastMsg = byUrn.get(msgUrns[0]);
let preview = lastMsg && lastMsg.body ? norm(lastMsg.body.text) : '';
if (!preview) preview = norm(conv.descriptionText || '');
const activityMs = Number(conv.lastActivityAt || 0);
entries.push({
activityMs,
row: {
thread_id: threadId,
person_name: conv.title ? norm(conv.title) : others.join(', '),
last_message_preview: preview.slice(0, 300),
unread: Number(conv.unreadCount || 0) > 0 || conv.read === false,
counterparty_type: counterpartyKind,
category: Array.isArray(conv.categories) ? conv.categories.join(',') : '',
timestamp: activityMs ? new Date(activityMs).toISOString() : '',
},
});
}
// Most-recent first; the sort key is kept off the returned row.
entries.sort((a, b) => b.activityMs - a.activityMs);
return entries.map((entry) => entry.row);
}
cli({
site: 'linkedin',
name: 'inbox',
access: 'read',
description: 'List LinkedIn messaging inbox conversations and unread messages',
domain: 'www.linkedin.com',
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: 'Maximum conversations to return (1-100)' },
{ name: 'unread-only', type: 'bool', default: false, help: 'Return only conversations with unread messages' },
],
columns: [
'rank',
'thread_url',
'thread_id',
'person_name',
'last_message_preview',
'unread',
'counterparty_type',
'category',
'timestamp',
],
func: async (page, kwargs) => {
// Validate --limit explicitly rather than silently clamping an out-of-range value.
let limit = DEFAULT_LIMIT;
if (kwargs.limit !== undefined && kwargs.limit !== null && kwargs.limit !== '') {
limit = Number(kwargs.limit);
if (!Number.isInteger(limit) || limit < MIN_LIMIT || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}`);
}
}
const unreadOnly = Boolean(kwargs['unread-only']);
await page.goto(MESSAGING_URL);
await page.wait(10);
// Locate the messaging API request the page fired on load; retry once if the
// SPA was slow to issue it.
let located = unwrapEvaluateResult(await page.evaluate(`(${findMessagingApiUrl.toString()})()`));
if (located && located.loginRequired) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn requires an active signed-in browser session.');
}
if (!located || !located.url) {
await page.wait(6);
located = unwrapEvaluateResult(await page.evaluate(`(${findMessagingApiUrl.toString()})()`));
}
if (!located || !located.url) {
throw new CommandExecutionError(
'LinkedIn did not issue a messaging API request; the inbox may have failed to load.',
);
}
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
}
const csrf = jsession.replace(/^"|"$/g, '');
// Widen the page size to the requested limit where the query supports it.
const targetUrl = located.url.replace(/count:\d+/, 'count:' + limit);
const fetched = unwrapEvaluateResult(
await page.evaluate(`(${fetchMessagingApi.toString()})(${JSON.stringify(targetUrl)}, ${JSON.stringify(csrf)})`),
);
if (fetched && fetched.authRequired) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn messaging API authentication failed: ' + fetched.error);
}
if (!fetched || fetched.error || !fetched.json) {
throw new CommandExecutionError(
'LinkedIn messaging API returned an unexpected response: ' + ((fetched && fetched.error) || 'no data'),
);
}
let conversations = parseConversations(fetched.json, located.mailboxUrn || '');
if (unreadOnly) conversations = conversations.filter((c) => c.unread);
if (conversations.length === 0) {
if (unreadOnly) return [];
throw new EmptyResultError('linkedin inbox', 'No LinkedIn conversations were found in the inbox.');
}
return conversations.slice(0, limit).map((c, index) => ({
rank: index + 1,
thread_url: threadUrl(c.thread_id),
thread_id: c.thread_id,
person_name: c.person_name,
last_message_preview: c.last_message_preview,
unread: c.unread,
counterparty_type: c.counterparty_type,
category: c.category,
timestamp: c.timestamp,
}));
},
});
export const __test__ = {
parseConversations,
threadUrl,
};
+152
View File
@@ -0,0 +1,152 @@
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import './inbox.js';
const { parseConversations, threadUrl } = await import('./inbox.js').then((m) => m.__test__);
const SELF = 'urn:li:fsd_profile:SELF';
// Minimal normalized messaging payload mirroring LinkedIn's real response shape:
// a flat `included` entity array where conversations reference participants and
// messages by URN.
function fixture() {
return {
included: [
{
$type: 'com.linkedin.messenger.MessagingParticipant',
entityUrn: 'urn:li:msg_messagingParticipant:SELF',
hostIdentityUrn: SELF,
participantType: { member: { firstName: { text: 'Hanzi' }, lastName: { text: 'Li' } } },
},
{
$type: 'com.linkedin.messenger.MessagingParticipant',
entityUrn: 'urn:li:msg_messagingParticipant:P1',
hostIdentityUrn: 'urn:li:fsd_profile:P1',
participantType: { member: { firstName: { text: 'Olga' }, lastName: { text: 'Magere' } } },
},
{
$type: 'com.linkedin.messenger.MessagingParticipant',
entityUrn: 'urn:li:msg_messagingParticipant:ORG',
hostIdentityUrn: 'urn:li:fsd_company:99',
participantType: { organization: { name: { text: 'American Express' } } },
},
{ $type: 'com.linkedin.messenger.Message', entityUrn: 'urn:li:msg_message:M1', body: { text: 'hey, are you around this week?' } },
{ $type: 'com.linkedin.messenger.Message', entityUrn: 'urn:li:msg_message:M2', body: { text: 'Sponsored offer' } },
{
$type: 'com.linkedin.messenger.Conversation',
entityUrn: 'urn:li:msg_conversation:C1',
backendUrn: 'urn:li:messagingThread:2-aaa==',
unreadCount: 2,
read: false,
categories: ['INBOX', 'PRIMARY_INBOX'],
lastActivityAt: 2000,
'*conversationParticipants': ['urn:li:msg_messagingParticipant:P1', 'urn:li:msg_messagingParticipant:SELF'],
messages: { '*elements': ['urn:li:msg_message:M1'] },
title: null,
},
{
$type: 'com.linkedin.messenger.Conversation',
entityUrn: 'urn:li:msg_conversation:C2',
backendUrn: 'urn:li:messagingThread:2-bbb==',
unreadCount: 0,
read: true,
categories: ['INBOX', 'PRIMARY_INBOX', 'INMAIL'],
lastActivityAt: 3000,
'*conversationParticipants': ['urn:li:msg_messagingParticipant:ORG', 'urn:li:msg_messagingParticipant:SELF'],
messages: { '*elements': ['urn:li:msg_message:M2'] },
title: null,
},
{
$type: 'com.linkedin.messenger.Conversation',
entityUrn: 'urn:li:msg_conversation:C3',
backendUrn: 'urn:li:messagingThread:2-ccc==',
unreadCount: 0,
read: true,
categories: ['INBOX', 'PRIMARY_INBOX'],
lastActivityAt: 1000,
'*conversationParticipants': ['urn:li:msg_messagingParticipant:P1', 'urn:li:msg_messagingParticipant:SELF'],
messages: { '*elements': [] },
title: 'Cohort 2 group',
},
],
};
}
describe('linkedin inbox adapter', () => {
const command = getRegistry().get('linkedin/inbox');
it('registers the command with the expected shape', () => {
expect(command).toBeDefined();
expect(command.site).toBe('linkedin');
expect(command.name).toBe('inbox');
expect(command.domain).toBe('www.linkedin.com');
expect(command.strategy).toBe('cookie');
expect(command.browser).toBe(true);
expect(typeof command.func).toBe('function');
});
it('exposes channel-safe structured columns', () => {
expect(command.columns).toEqual(
expect.arrayContaining([
'thread_url',
'thread_id',
'person_name',
'last_message_preview',
'unread',
'counterparty_type',
'category',
'timestamp',
]),
);
});
it('builds a thread URL from a thread id', () => {
expect(threadUrl('2-aaa==')).toBe('https://www.linkedin.com/messaging/thread/2-aaa==/');
expect(threadUrl('')).toBe('');
});
it('parses conversations and sorts them by most recent activity', () => {
const rows = parseConversations(fixture(), SELF);
expect(rows).toHaveLength(3);
expect(rows.map((r) => r.thread_id)).toEqual(['2-bbb==', '2-aaa==', '2-ccc==']);
});
it('resolves the member counterparty, excludes the inbox owner, and reports unread state', () => {
const c1 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-aaa==');
expect(c1.person_name).toBe('Olga Magere');
expect(c1.counterparty_type).toBe('member');
expect(c1.unread).toBe(true);
expect(c1.last_message_preview).toBe('hey, are you around this week?');
});
it('flags organization counterparties and read conversations', () => {
const c2 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-bbb==');
expect(c2.person_name).toBe('American Express');
expect(c2.counterparty_type).toBe('organization');
expect(c2.unread).toBe(false);
expect(c2.category).toBe('INBOX,PRIMARY_INBOX,INMAIL');
});
it('uses the group title as the conversation name', () => {
const c3 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-ccc==');
expect(c3.person_name).toBe('Cohort 2 group');
});
it('returns an empty array when a valid payload has no conversations', () => {
expect(parseConversations({ included: [] }, SELF)).toEqual([]);
});
it('fails typed when the normalized payload shape is malformed', () => {
expect(() => parseConversations({}, SELF)).toThrow(CommandExecutionError);
expect(() => parseConversations(null, SELF)).toThrow(CommandExecutionError);
const malformed = fixture();
malformed.included.push({
$type: 'com.linkedin.messenger.Conversation',
entityUrn: 'urn:li:msg_conversation:MALFORMED',
'*conversationParticipants': [],
messages: { '*elements': [] },
});
expect(() => parseConversations(malformed, SELF)).toThrow(CommandExecutionError);
});
});
+262
View File
@@ -0,0 +1,262 @@
/**
* LinkedIn people-search via SSR DOM text-slice. Voyager people-search
* REST returns HTTP 500 from a web context; LinkedIn renders results
* server-side now. One navigation per call consumes one CUL query.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
const SEARCH_URL_BASE = 'https://www.linkedin.com/search/results/people/';
const MAX_LIMIT = 10;
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
}
function requireStringArg(args, key, label = key) {
const value = normalizeWhitespace(args[key]);
if (!value) throw new ArgumentError(`${label} is required`);
return value;
}
function parseLimit(value) {
if (value === undefined || value === null || value === '') return 5;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function buildSearchUrl(keywords) {
return SEARCH_URL_BASE + '?keywords=' + encodeURIComponent(keywords);
}
function looksLinkedInAuthWall(value) {
const text = normalizeWhitespace(value).toLowerCase();
if (!text) return false;
return /linkedin\.com\/(?:login|checkpoint|authwall|uas)/i.test(text)
|| /\b(sign in|log in|join linkedin|captcha|verification required)\b/i.test(text)
|| /(请登录|登录领英|安全验证)/.test(text);
}
function normalizeProfileUrl(value) {
const raw = normalizeWhitespace(value);
if (!raw) return '';
try {
const parsed = new URL(raw);
const host = parsed.hostname.toLowerCase();
if (parsed.protocol !== 'https:' || parsed.username || parsed.password || parsed.port) return '';
if (host !== 'linkedin.com' && host !== 'www.linkedin.com') return '';
const match = parsed.pathname.match(/^\/in\/([^/?#]+)\/?$/);
if (!match || !match[1]) return '';
return `https://www.linkedin.com/in/${match[1]}/`;
} catch {
return '';
}
}
function normalizePeopleRows(rows) {
if (!Array.isArray(rows)) {
throw new CommandExecutionError('LinkedIn people search returned malformed extraction payload: missing rows array');
}
return rows.map((row, index) => {
if (!row || typeof row !== 'object') {
throw new CommandExecutionError(`LinkedIn people search returned malformed row at index ${index}`);
}
const name = normalizeWhitespace(row.name);
const profileUrl = normalizeProfileUrl(row.profile_url);
if (!name || !profileUrl) {
throw new CommandExecutionError(`LinkedIn people search returned row without stable profile identity at index ${index}`);
}
return {
name,
headline: normalizeWhitespace(row.headline),
location: normalizeWhitespace(row.location),
profile_url: profileUrl,
};
});
}
function parseNonNegativeCount(value, label) {
const count = Number(value);
if (!Number.isInteger(count) || count < 0) {
throw new CommandExecutionError(`LinkedIn people search returned malformed extraction payload: invalid ${label}`);
}
return count;
}
function extractionScript() {
// Class-based selectors are dead (LinkedIn rotates hashed class
// names on every deploy) and display:contents flattens the DOM
// tree so per-card containers don't exist. Read main.innerText
// and slice between consecutive person-name lines instead.
return String.raw`(() => {
if (!/search\/results\/people/.test(window.location.href)) {
return { error: 'not on people search page', url: window.location.href };
}
const main = document.querySelector('main') || document.body;
const normalize = (s) => String(s || '').replace(/[\s\u00a0\u202f]+/g, ' ').trim();
const skip = (l) => !l
|| /^Status is/.test(l)
|| /^(Message|Connect|Follow|View profile|Pending|Remove)$/i.test(l)
|| /^[•·]\s*(?:1st|2nd|3rd\+?|degree)/i.test(l)
|| /^[•·]/.test(l)
|| l.includes('mutual connection')
|| l.includes('shared connection')
|| /^Summary:/i.test(l)
|| /^About this profile/i.test(l);
const anchors = Array.from(main.querySelectorAll('a[href*="/in/"]'));
const personEntries = [];
const seenHandles = new Set();
for (const a of anchors) {
const m = (a.getAttribute('href') || '').match(/\/in\/([^/?#]+)/);
if (!m || !m[1]) continue;
const profileHandle = m[1];
if (seenHandles.has(profileHandle)) continue;
const aria = a.querySelector('span[aria-hidden="true"]');
let name = normalize(aria ? aria.textContent : a.textContent);
name = name.replace(/^Status is (online|offline)\.?\s*/i, '')
.replace(/'?s profile$/i, '')
.replace(/\s*[•·].*$/, '').trim();
if (!name) continue;
seenHandles.add(profileHandle);
personEntries.push({ profileHandle, displayName: name });
}
const lines = (main.innerText || '').split(/\n+/).map(normalize).filter(Boolean);
// skip() rejects mutual-connection lines, so candidates that only
// appear as mutual-connection links inside another card's row
// never resolve a name index and get filtered out below.
const nameToIndex = new Map();
for (const { displayName } of personEntries) {
if (nameToIndex.has(displayName)) continue;
const match = lines.findIndex((l) =>
!skip(l) && (
l === displayName
|| l.startsWith(displayName + ' ')
|| l.startsWith(displayName + ',')
|| l.startsWith(displayName + "'")
)
);
if (match >= 0) nameToIndex.set(displayName, match);
}
const resolved = personEntries.filter((p) => nameToIndex.has(p.displayName));
const rows = [];
for (let i = 0; i < resolved.length; i++) {
const { profileHandle, displayName } = resolved[i];
const startIdx = nameToIndex.get(displayName);
let stopIdx = lines.length;
for (let j = i + 1; j < resolved.length; j++) {
const otherStart = nameToIndex.get(resolved[j].displayName);
if (otherStart != null && otherStart > startIdx) {
stopIdx = otherStart;
break;
}
}
const slice = lines.slice(startIdx + 1, stopIdx).filter((l) => l !== displayName && !skip(l));
rows.push({
name: displayName,
headline: slice[0] || '',
location: slice[1] || '',
profile_url: 'https://www.linkedin.com/in/' + profileHandle + '/',
});
}
return {
rows,
candidate_count: personEntries.length,
person_entries_count: personEntries.length,
resolved_count: resolved.length,
};
})()`;
}
cli({
site: 'linkedin',
name: 'people-search',
access: 'read',
description: 'Search standard LinkedIn (not Sales Navigator) for people by keyword. Each invocation consumes against LinkedIn\'s monthly Commercial Use Limit on people search; throttle accordingly.',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.COOKIE,
browser: true,
args: [
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'People search keywords, e.g. "site reliability engineer berlin"' },
{ name: 'limit', type: 'int', default: 5, help: `Maximum people to return (1-${MAX_LIMIT}); each query counts toward LinkedIn's monthly CUL` },
],
columns: ['rank', 'name', 'headline', 'location', 'profile_url'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin people-search');
const keywords = requireStringArg(args, 'keywords', '--keywords');
const limit = parseLimit(args.limit);
try {
await page.goto(buildSearchUrl(keywords));
await page.wait(6);
} catch (error) {
throw new CommandExecutionError(`LinkedIn people search navigation failed: ${error?.message || error}`);
}
let cookies;
try {
cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
} catch (error) {
throw new CommandExecutionError(`LinkedIn cookie lookup failed: ${error?.message || error}`);
}
if (!Array.isArray(cookies)) {
throw new CommandExecutionError('LinkedIn cookie lookup returned malformed payload');
}
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
}
let result;
try {
result = unwrapEvaluateResult(await page.evaluate(extractionScript()));
} catch (error) {
throw new CommandExecutionError(`LinkedIn people search extraction failed: ${error?.message || error}`);
}
if (result?.error) {
if (looksLinkedInAuthWall(`${result.url || ''} ${result.error || ''}`)) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn people search requires an active signed-in browser session.');
}
// If LinkedIn redirected away from the search page that
// usually means CUL was reached or the account is gated.
throw new CommandExecutionError(`LinkedIn redirected away from the search page (${result.error}). Likely Commercial Use Limit reached - the limit resets on the 1st of next month.`);
}
if (!result || typeof result !== 'object') {
throw new CommandExecutionError('LinkedIn people search returned malformed extraction payload');
}
const candidateCount = parseNonNegativeCount(result.candidate_count, 'candidate_count');
parseNonNegativeCount(result.person_entries_count, 'person_entries_count');
const resolvedCount = parseNonNegativeCount(result.resolved_count, 'resolved_count');
const rows = normalizePeopleRows(result.rows);
if (rows.length === 0 && (candidateCount > 0 || resolvedCount > 0)) {
throw new CommandExecutionError('LinkedIn people search found profile candidates but could not parse stable result rows');
}
if (rows.length === 0) {
throw new EmptyResultError(`No people found on the rendered page for "${keywords}". The search may have returned zero results, or the DOM markup may have changed.`);
}
return rows.slice(0, limit).map((p, i) => ({ rank: i + 1, ...p }));
},
});
export const __test__ = {
normalizeWhitespace,
parseLimit,
buildSearchUrl,
looksLinkedInAuthWall,
normalizeProfileUrl,
normalizePeopleRows,
parseNonNegativeCount,
extractionScript,
};
+216
View File
@@ -0,0 +1,216 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import './people-search.js';
const {
parseLimit,
buildSearchUrl,
looksLinkedInAuthWall,
normalizeProfileUrl,
normalizePeopleRows,
parseNonNegativeCount,
extractionScript,
} = await import('./people-search.js').then((m) => m.__test__);
function extractionResult(rows, counts = {}) {
return {
rows,
candidate_count: counts.candidate_count ?? rows.length,
person_entries_count: counts.person_entries_count ?? counts.candidate_count ?? rows.length,
resolved_count: counts.resolved_count ?? rows.length,
};
}
function makePage({
evaluateResult,
evaluateReject,
gotoReject,
cookies = [{ name: 'JSESSIONID', value: '"ajax:1234567890"' }],
} = {}) {
return {
goto: vi.fn().mockImplementation(() => gotoReject ? Promise.reject(gotoReject) : Promise.resolve(undefined)),
wait: vi.fn().mockResolvedValue(undefined),
getCookies: vi.fn().mockResolvedValue(cookies),
evaluate: vi.fn().mockImplementation(() => evaluateReject ? Promise.reject(evaluateReject) : Promise.resolve(evaluateResult)),
};
}
describe('linkedin people-search command', () => {
it('builds the canonical SSR search URL with encoded keywords', () => {
expect(buildSearchUrl('site reliability engineer'))
.toBe('https://www.linkedin.com/search/results/people/?keywords=site%20reliability%20engineer');
expect(buildSearchUrl('hello/world & stuff'))
.toBe('https://www.linkedin.com/search/results/people/?keywords=hello%2Fworld%20%26%20stuff');
});
it('validates --limit without silent clamping', () => {
expect(parseLimit(undefined)).toBe(5);
expect(parseLimit(1)).toBe(1);
expect(parseLimit(10)).toBe(10);
expect(() => parseLimit(0)).toThrow(ArgumentError);
expect(() => parseLimit(11)).toThrow(ArgumentError);
expect(() => parseLimit(-1)).toThrow(ArgumentError);
expect(() => parseLimit('abc')).toThrow(ArgumentError);
expect(() => parseLimit(1.5)).toThrow(ArgumentError);
});
it('extraction script slices main.innerText by person-name boundaries', () => {
const s = extractionScript();
// Anchor enumeration finds /in/<handle>.
expect(s).toContain('a[href*="/in/"]');
expect(s).toContain('\\/in\\/([^/?#]+)');
// Text-slice approach: split main.innerText and locate names.
expect(s).toContain('main.innerText');
expect(s).toContain('lines.findIndex');
// Mutual-connection anchors are filtered out via the skip()
// predicate on the name-line match.
expect(s).toContain('mutual connection');
// Names dedup'd by handle.
expect(s).toContain('seenHandles');
// Aria-hidden span as canonical name source.
expect(s).toContain('span[aria-hidden="true"]');
// Only operates on the people-search page.
expect(s).toContain('search\\/results\\/people');
expect(s).toContain('candidate_count');
expect(s).toContain('resolved_count');
});
it('normalizes only stable LinkedIn profile identities', () => {
expect(normalizeProfileUrl('https://www.linkedin.com/in/alice-engineer/?mini=true'))
.toBe('https://www.linkedin.com/in/alice-engineer/');
expect(normalizeProfileUrl('https://linkedin.com/in/bob-builder')).toBe('https://www.linkedin.com/in/bob-builder/');
expect(normalizeProfileUrl('https://evil-linkedin.com/in/bob-builder')).toBe('');
expect(normalizeProfileUrl('http://www.linkedin.com/in/bob-builder')).toBe('');
expect(normalizeProfileUrl('https://www.linkedin.com/company/opencli')).toBe('');
});
it('detects LinkedIn auth-wall URLs separately from CUL redirects', () => {
expect(looksLinkedInAuthWall('https://www.linkedin.com/authwall Sign in to continue')).toBe(true);
expect(looksLinkedInAuthWall('https://www.linkedin.com/checkpoint/challenge security verification required')).toBe(true);
expect(looksLinkedInAuthWall('https://www.linkedin.com/feed/')).toBe(false);
});
it('rejects malformed extraction rows instead of fabricating success rows', () => {
expect(() => normalizePeopleRows({})).toThrow(CommandExecutionError);
expect(() => normalizePeopleRows([null])).toThrow(CommandExecutionError);
expect(() => normalizePeopleRows([{ name: 'No URL', headline: 'h', location: 'l', profile_url: '' }]))
.toThrow(CommandExecutionError);
expect(() => normalizePeopleRows([{ name: '', headline: 'h', location: 'l', profile_url: 'https://www.linkedin.com/in/no-name/' }]))
.toThrow(CommandExecutionError);
});
it('validates extraction evidence counters', () => {
expect(parseNonNegativeCount(0, 'candidate_count')).toBe(0);
expect(parseNonNegativeCount(2, 'candidate_count')).toBe(2);
expect(() => parseNonNegativeCount(undefined, 'candidate_count')).toThrow(CommandExecutionError);
expect(() => parseNonNegativeCount(-1, 'candidate_count')).toThrow(CommandExecutionError);
expect(() => parseNonNegativeCount(1.2, 'candidate_count')).toThrow(CommandExecutionError);
});
it('returns ranked rows when the page yields people', async () => {
const cmd = getRegistry().get('linkedin/people-search');
expect(cmd?.func).toBeTypeOf('function');
const page = makePage({
evaluateResult: extractionResult([
{ name: 'Alice Engineer', headline: 'Staff SWE at Acme', location: 'Berlin', profile_url: 'https://www.linkedin.com/in/alice-engineer/' },
{ name: 'Bob Builder', headline: 'CTO at Globex', location: 'Remote', profile_url: 'https://www.linkedin.com/in/bob-builder/' },
]),
});
const result = await cmd.func(page, { keywords: 'reinforcement learning', limit: 5 });
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/search/results/people/?keywords=reinforcement%20learning');
expect(result).toEqual([
{ rank: 1, name: 'Alice Engineer', headline: 'Staff SWE at Acme', location: 'Berlin', profile_url: 'https://www.linkedin.com/in/alice-engineer/' },
{ rank: 2, name: 'Bob Builder', headline: 'CTO at Globex', location: 'Remote', profile_url: 'https://www.linkedin.com/in/bob-builder/' },
]);
});
it('slices to --limit when more rows are extracted than requested', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({
evaluateResult: extractionResult(Array.from({ length: 8 }, (_, i) => ({
name: `Person ${i}`, headline: 'h', location: 'l', profile_url: `https://www.linkedin.com/in/p${i}/`,
}))),
});
const result = await cmd.func(page, { keywords: 'x', limit: 3 });
expect(result).toHaveLength(3);
expect(result.map((r) => r.rank)).toEqual([1, 2, 3]);
});
it('throws AuthRequiredError when JSESSIONID cookie is missing', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ cookies: [], evaluateResult: extractionResult([]) });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('treats malformed cookie lookup results as CommandExecutionError', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ cookies: null, evaluateResult: extractionResult([]) });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('treats LinkedIn redirect away from search page as a CUL-flavoured CommandExecutionError', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateResult: { error: 'not on people search page', url: 'https://www.linkedin.com/' } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('treats LinkedIn auth-wall redirects as AuthRequiredError', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateResult: { error: 'not on people search page', url: 'https://www.linkedin.com/authwall?trk=people_search' } });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('wraps browser extraction exceptions as CommandExecutionError', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateReject: new SyntaxError('Unexpected token <') });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('wraps browser navigation exceptions as CommandExecutionError', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ gotoReject: new Error('navigation failed') });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('throws EmptyResultError when the page rendered zero rows', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateResult: extractionResult([]) });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
});
it('treats profile candidates without stable parsed rows as parser drift', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({
evaluateResult: extractionResult([], {
candidate_count: 1,
person_entries_count: 1,
resolved_count: 0,
}),
});
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('treats missing rows array as parser drift, not empty results', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateResult: {} });
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('rejects empty keywords with ArgumentError before navigation', async () => {
const cmd = getRegistry().get('linkedin/people-search');
const page = makePage({ evaluateResult: extractionResult([]) });
await expect(cmd.func(page, { keywords: ' ', limit: 5 })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('registers with the expected columns and arg shape', () => {
const cmd = getRegistry().get('linkedin/people-search');
expect(cmd?.columns).toEqual(['rank', 'name', 'headline', 'location', 'profile_url']);
expect(cmd?.access).toBe('read');
expect(cmd?.browser).toBe(true);
const keywordsArg = cmd?.args?.find((a) => a.name === 'keywords');
expect(keywordsArg?.positional).toBe(true);
expect(keywordsArg?.required).toBe(true);
});
});
+357
View File
@@ -0,0 +1,357 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { createHash } from 'node:crypto';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function normalizeName(value) {
return normalizeWhitespace(value)
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
.replace(/\s+LinkedIn.*$/i, '')
.toLowerCase();
}
function isLinkedInHost(hostname) {
const host = String(hostname || '').toLowerCase();
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
}
function canonicalizeLinkedInThreadUrl(value) {
const raw = normalizeWhitespace(value);
if (!raw) return '';
try {
const url = new URL(raw);
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
const match = url.pathname.match(/^\/messaging\/thread\/([^/]+)\/?$/i);
if (!match || !match[1]) return '';
url.hostname = 'www.linkedin.com';
url.hash = '';
url.search = '';
if (!url.pathname.endsWith('/')) url.pathname += '/';
return url.toString();
} catch {
return '';
}
}
function hashText(value) {
return createHash('sha256').update(normalizeWhitespace(value)).digest('hex');
}
function textContainsNormalized(haystack, needle) {
const h = normalizeWhitespace(haystack).toLowerCase();
const n = normalizeWhitespace(needle).toLowerCase();
return !n || h.includes(n);
}
function selectBestHeaderName(headerNames, expectedName) {
const expected = normalizeName(expectedName);
const names = (Array.isArray(headerNames) ? headerNames : [])
.map(normalizeWhitespace)
.filter(Boolean);
return names.find((name) => normalizeName(name) === expected) || names[0] || '';
}
function assessThreadSafety(probe, expected) {
const expectedName = normalizeWhitespace(expected.expectedName);
const actualName = selectBestHeaderName(probe?.headerNames, expectedName);
const expectedThreadUrl = canonicalizeLinkedInThreadUrl(expected.threadUrl);
const actualThreadUrl = canonicalizeLinkedInThreadUrl(probe?.url || '');
const bodyText = String(probe?.bodyText || '');
if (probe?.authRequired) {
return { ok: false, blockReason: 'auth_required', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
}
if (probe?.searchFailure || /we didn't find anything|no results found|no results for/i.test(bodyText)) {
return { ok: false, blockReason: 'search_failure_visible', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
}
if (expectedThreadUrl && actualThreadUrl && expectedThreadUrl !== actualThreadUrl) {
return { ok: false, blockReason: 'thread_url_mismatch', expectedValue: expectedThreadUrl, actualValue: actualThreadUrl, observedUrl: actualThreadUrl };
}
if (!actualName || normalizeName(actualName) !== normalizeName(expectedName)) {
return { ok: false, blockReason: 'recipient_header_mismatch', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
}
if (!probe?.composerFound) {
return { ok: false, blockReason: 'composer_not_found', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
}
const expectedLastHash = normalizeWhitespace(expected.expectedLastHash);
if (expectedLastHash && expectedLastHash !== probe?.latestMessageHash) {
return { ok: false, blockReason: 'latest_message_mismatch', expectedValue: expectedLastHash, actualValue: probe?.latestMessageHash || '', observedUrl: actualThreadUrl };
}
const expectedLastText = normalizeWhitespace(expected.expectedLastText);
if (expectedLastText && !textContainsNormalized(bodyText, expectedLastText)) {
return { ok: false, blockReason: 'latest_message_mismatch', expectedValue: expectedLastText, actualValue: '', observedUrl: actualThreadUrl };
}
return { ok: true, blockReason: 'verified', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
}
function requireStringArg(args, key, label = key) {
const value = normalizeWhitespace(args[key]);
if (!value) throw new ArgumentError(`${label} is required`);
return value;
}
function requireLinkedInThreadUrl(value, label) {
const url = canonicalizeLinkedInThreadUrl(value);
if (!url) throw new ArgumentError(`${label} must be an exact https://www.linkedin.com/messaging/thread/<id>/ URL`);
return url;
}
function buildThreadProbeScript() {
return String.raw`(() => {
const marker = '__OPENCLI_LINKEDIN_PROBE__';
void marker;
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const text = document.body ? (document.body.innerText || '') : '';
const lower = text.toLowerCase();
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|| /captcha|verification required/i.test(text);
const searchFailure = /we didn't find anything|no results found|no results for/i.test(text);
const headerCandidates = [];
const selectors = [
'.msg-thread__link-to-profile',
'.msg-thread__link-to-profile span[aria-hidden="true"]',
'.msg-entity-lockup__entity-title',
'.msg-conversation-card__participant-names',
'main h1',
'main h2',
'[data-anonymize="person-name"]',
'a[href*="/in/"] span[aria-hidden="true"]',
'a[href*="/in/"]'
];
for (const selector of selectors) {
for (const el of Array.from(document.querySelectorAll(selector)).slice(0, 8)) {
const value = clean(el.innerText || el.textContent || el.getAttribute('aria-label'));
if (value && value.length <= 120 && !/^(message|messaging|send|profile|view profile)$/i.test(value)) {
headerCandidates.push(value);
}
}
}
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
const messageText = Array.from(document.querySelectorAll('.msg-s-message-list__event, .msg-s-event-listitem, [data-event-urn], .msg-s-message-group__meta, .msg-s-message-list-content'))
.map((el) => clean(el.innerText || el.textContent))
.filter(Boolean)
.join('\n');
const sourceText = messageText || text;
const sourceLines = sourceText.split(/\n+/).map(clean).filter(Boolean);
const lastMeaningfulLine = [...sourceLines].reverse().find((line) => !/^(send|reply|write a message|press enter to send)$/i.test(line)) || '';
return {
url: location.href,
title: document.title || '',
headerNames: Array.from(new Set(headerCandidates)).slice(0, 10),
bodyText: text,
composerFound: Boolean(composer),
composerText: composer ? clean(composer.innerText || composer.textContent) : '',
authRequired,
searchFailure,
latestMessageText: lastMeaningfulLine,
latestMessageHash: '',
};
})()`;
}
function buildFocusComposerScript() {
return String.raw`(() => {
const marker = '__OPENCLI_LINKEDIN_FOCUS_COMPOSER__';
void marker;
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
if (!composer) return { ok: false, error: 'composer_not_found', composerText: '' };
composer.focus();
composer.innerHTML = '';
composer.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'deleteContentBackward', data: null }));
return { ok: true, composerText: clean(composer.innerText || composer.textContent) };
})()`;
}
function buildReadComposerScript() {
return String.raw`(() => {
const marker = '__OPENCLI_LINKEDIN_READ_COMPOSER__';
void marker;
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
return { ok: Boolean(composer), composerText: composer ? clean(composer.innerText || composer.textContent) : '' };
})()`;
}
function buildClickSendScript() {
return String.raw`(() => {
const marker = '__OPENCLI_LINKEDIN_CLICK_SEND__';
void marker;
const buttons = Array.from(document.querySelectorAll('button'));
const send = buttons.find((button) => {
const text = (button.innerText || button.textContent || button.getAttribute('aria-label') || '').trim().toLowerCase();
return text === 'send' || text === 'send message';
});
if (!send) return { ok: false, error: 'send_button_not_found', sent: false };
if (send.disabled || send.getAttribute('aria-disabled') === 'true') return { ok: false, error: 'send_button_disabled', sent: false };
send.click();
return { ok: true, sent: true };
})()`;
}
async function probeThread(page) {
const result = unwrapEvaluateResult(await page.evaluate(buildThreadProbeScript()));
const latestText = normalizeWhitespace(result?.latestMessageText || '');
return {
...(result || {}),
latestMessageText: latestText,
latestMessageHash: latestText ? hashText(latestText) : '',
};
}
cli({
site: 'linkedin',
name: 'safe-send',
access: 'write',
description: 'Fail-closed LinkedIn message sender that verifies exact thread, recipient, and latest message before filling/sending',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'thread-url', required: true, help: 'Exact LinkedIn messaging thread URL to open and verify' },
{ name: 'expected-name', required: true, help: 'Expected visible recipient name in the active thread header' },
{ name: 'message', required: true, help: 'Message body to send or dry-run' },
{ name: 'expected-last-text', help: 'Substring expected in the currently visible latest conversation context' },
{ name: 'expected-last-hash', help: 'SHA-256 hash of expected latest visible message text' },
{ name: 'send', type: 'bool', default: false, help: 'Actually click Send. Default is dry-run verification only.' },
{ name: 'screenshot', type: 'bool', default: false, help: 'Capture a screenshot during verification' },
],
columns: ['status', 'recipient', 'reason', 'thread_url', 'message_chars', 'screenshot'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin safe-send');
const threadUrl = requireLinkedInThreadUrl(requireStringArg(args, 'thread-url', '--thread-url'), '--thread-url');
const expectedName = requireStringArg(args, 'expected-name', '--expected-name');
const message = requireStringArg(args, 'message', '--message');
await page.goto('https://www.linkedin.com/messaging/');
await page.wait(4);
await page.goto(threadUrl);
// LinkedIn messaging often renders the shell first and hydrates the active
// thread header/messages a few seconds later. Wait long enough for the
// recipient header to appear so we fail closed on a real mismatch, not on
// a premature blank DOM snapshot.
await page.wait(12);
let beforeProbe = await probeThread(page);
const expectedLastText = normalizeWhitespace(args['expected-last-text']);
for (let attempt = 0; expectedLastText && attempt < 6 && !textContainsNormalized(beforeProbe.bodyText, expectedLastText); attempt += 1) {
await page.wait(2);
beforeProbe = await probeThread(page);
}
const safety = assessThreadSafety(beforeProbe, {
expectedName,
threadUrl,
expectedLastText: args['expected-last-text'],
expectedLastHash: args['expected-last-hash'],
});
if (safety.blockReason === 'auth_required') {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn safe-send requires an active signed-in LinkedIn browser session.');
}
if (!safety.ok) {
const observed = [
`Expected ${safety.expectedValue}; actual ${safety.actualValue || 'not_visible'} at ${safety.observedUrl || 'url_not_available'}`,
`Observed headers: ${(beforeProbe.headerNames || []).join(' | ') || 'no_visible_headers'}`,
`Title: ${beforeProbe.title || 'title_not_available'}`,
`Body: ${normalizeWhitespace(beforeProbe.bodyText || '').slice(0, 500)}`,
].join('\n');
throw new CommandExecutionError(
`LinkedIn safe-send blocked: ${safety.blockReason}`,
observed,
);
}
let screenshot = '';
if (args.screenshot && typeof page.screenshot === 'function') {
screenshot = await page.screenshot({ fullPage: false });
}
if (!args.send) {
return [{
status: 'verified_dry_run',
recipient: safety.actualValue,
reason: safety.blockReason,
thread_url: safety.observedUrl,
message_chars: message.length,
screenshot: screenshot ? 'captured' : '',
}];
}
const focus = unwrapEvaluateResult(await page.evaluate(buildFocusComposerScript()));
if (!focus?.ok) throw new CommandExecutionError(`LinkedIn safe-send blocked: ${focus?.error || 'composer_focus_failed'}`);
await page.insertText(message);
await page.wait(0.6 + Math.random() * 0.8);
const composer = unwrapEvaluateResult(await page.evaluate(buildReadComposerScript()));
if (!composer?.ok || normalizeWhitespace(composer.composerText) !== normalizeWhitespace(message)) {
throw new CommandExecutionError(
'LinkedIn safe-send blocked: composer_text_mismatch',
`Composer text did not exactly match intended message for ${expectedName}.`,
);
}
const afterFillProbe = await probeThread(page);
const afterFillSafety = assessThreadSafety(afterFillProbe, {
expectedName,
threadUrl,
expectedLastText: args['expected-last-text'],
expectedLastHash: args['expected-last-hash'],
});
if (!afterFillSafety.ok) {
throw new CommandExecutionError(`LinkedIn safe-send blocked after fill: ${afterFillSafety.blockReason}`);
}
const sent = unwrapEvaluateResult(await page.evaluate(buildClickSendScript()));
if (!sent?.ok || !sent.sent) {
throw new CommandExecutionError(`LinkedIn safe-send blocked: ${sent?.error || 'send_click_failed'}`);
}
await page.wait(0.8 + Math.random() * 1.2);
return [{
status: 'sent',
recipient: safety.actualValue,
reason: safety.blockReason,
thread_url: safety.observedUrl,
message_chars: message.length,
screenshot: screenshot ? 'captured' : '',
}];
},
});
export const __test__ = {
normalizeWhitespace,
unwrapEvaluateResult,
normalizeName,
canonicalizeLinkedInThreadUrl,
hashText,
assessThreadSafety,
};
+204
View File
@@ -0,0 +1,204 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
import './safe-send.js';
const {
normalizeWhitespace,
normalizeName,
canonicalizeLinkedInThreadUrl,
hashText,
assessThreadSafety,
} = await import('./safe-send.js').then((m) => m.__test__);
function makeFakePage(probe) {
let composerText = probe.composerText || '';
return {
goto: vi.fn(async () => undefined),
wait: vi.fn(async () => undefined),
evaluate: vi.fn(async (script) => {
const text = String(script);
if (text.includes('__OPENCLI_LINKEDIN_PROBE__')) return probe;
if (text.includes('__OPENCLI_LINKEDIN_FOCUS_COMPOSER__')) return { ok: true, composerText: '' };
if (text.includes('__OPENCLI_LINKEDIN_READ_COMPOSER__')) return { ok: true, composerText };
if (text.includes('__OPENCLI_LINKEDIN_CLICK_SEND__')) return { ok: true, sent: true };
return undefined;
}),
insertText: vi.fn(async (text) => {
composerText = text;
}),
pressKey: vi.fn(async () => undefined),
screenshot: vi.fn(async () => 'base64-screenshot'),
};
}
describe('linkedin safe-send helpers', () => {
it('normalizes whitespace and LinkedIn names for exact-ish comparisons', () => {
expect(normalizeWhitespace(' Lokesh\n\tRamesh ')).toBe('Lokesh Ramesh');
expect(normalizeName('Lokesh Ramesh • 1st')).toBe('lokesh ramesh');
});
it('canonicalizes thread URLs while dropping query and hash noise', () => {
expect(canonicalizeLinkedInThreadUrl('https://www.linkedin.com/messaging/thread/abc/?foo=1#bar'))
.toBe('https://www.linkedin.com/messaging/thread/abc/');
expect(canonicalizeLinkedInThreadUrl('https://www.linkedin.com/messaging/thread/abc/extra')).toBe('');
expect(canonicalizeLinkedInThreadUrl('https://evil-linkedin.com/messaging/thread/abc/')).toBe('');
expect(canonicalizeLinkedInThreadUrl('http://www.linkedin.com/messaging/thread/abc/')).toBe('');
});
it('fails closed when LinkedIn search produced no results even if a composer is visible', () => {
const result = assessThreadSafety({
url: 'https://www.linkedin.com/messaging/thread/bora/',
headerNames: ['Bora Nicholson'],
bodyText: "We didn't find anything for Victoria Munoz\nBora Nicholson",
searchFailure: true,
composerFound: true,
latestMessageHash: hashText('hello'),
}, {
expectedName: 'Victoria Munoz',
threadUrl: 'https://www.linkedin.com/messaging/thread/victoria/',
expectedLastText: 'hello',
});
expect(result.ok).toBe(false);
expect(result.blockReason).toBe('search_failure_visible');
});
it('fails closed on recipient header mismatch', () => {
const result = assessThreadSafety({
url: 'https://www.linkedin.com/messaging/thread/bora/',
headerNames: ['Bora Nicholson'],
bodyText: 'Bora Nicholson\nhello',
composerFound: true,
latestMessageHash: hashText('hello'),
}, {
expectedName: 'Victoria Munoz',
expectedLastText: 'hello',
});
expect(result.ok).toBe(false);
expect(result.blockReason).toBe('recipient_header_mismatch');
expect(result.actualValue).toBe('Bora Nicholson');
});
it('fails closed when the stored latest message is no longer visible', () => {
const result = assessThreadSafety({
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
headerNames: ['Lokesh Ramesh'],
bodyText: 'Lokesh Ramesh\na newer inbound arrived',
composerFound: true,
latestMessageHash: hashText('a newer inbound arrived'),
}, {
expectedName: 'Lokesh Ramesh',
expectedLastText: 'old inbound text',
});
expect(result.ok).toBe(false);
expect(result.blockReason).toBe('latest_message_mismatch');
});
it('passes only when recipient, thread, latest text, and composer are all verified', () => {
const result = assessThreadSafety({
url: 'https://www.linkedin.com/messaging/thread/lokesh/?mini=true',
headerNames: ['Lokesh Ramesh'],
bodyText: 'Lokesh Ramesh\nI think outside help would fit best for provider doc follow ups',
composerFound: true,
latestMessageHash: hashText('I think outside help would fit best for provider doc follow ups'),
}, {
expectedName: 'Lokesh Ramesh',
threadUrl: 'https://www.linkedin.com/messaging/thread/lokesh/',
expectedLastText: 'provider doc follow ups',
});
expect(result.ok).toBe(true);
expect(result.blockReason).toBe('verified');
});
});
describe('linkedin safe-send command', () => {
it('registers as a write command with safe output columns', () => {
const command = getRegistry().get('linkedin/safe-send');
expect(command).toBeDefined();
expect(command.access).toBe('write');
expect(command.columns).toEqual(expect.arrayContaining(['status', 'recipient', 'reason']));
});
it('does not type or send when verification fails', async () => {
const command = getRegistry().get('linkedin/safe-send');
const page = makeFakePage({
url: 'https://www.linkedin.com/messaging/thread/bora/',
headerNames: ['Bora Nicholson'],
bodyText: 'Bora Nicholson',
composerFound: true,
searchFailure: false,
});
await expect(command.func(page, {
'thread-url': 'https://www.linkedin.com/messaging/thread/victoria/',
'expected-name': 'Victoria Munoz',
message: 'hello victoria',
send: true,
})).rejects.toBeInstanceOf(CommandExecutionError);
expect(page.insertText).not.toHaveBeenCalled();
expect(page.pressKey).not.toHaveBeenCalled();
});
it('rejects non-thread URLs before navigating or typing', async () => {
const command = getRegistry().get('linkedin/safe-send');
const page = makeFakePage({});
await expect(command.func(page, {
'thread-url': 'https://www.linkedin.com/feed/',
'expected-name': 'Victoria Munoz',
message: 'hello victoria',
send: true,
})).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
expect(page.insertText).not.toHaveBeenCalled();
});
it('dry-runs by default after verification without filling or sending', async () => {
const command = getRegistry().get('linkedin/safe-send');
const page = makeFakePage({
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
headerNames: ['Lokesh Ramesh'],
bodyText: 'Lokesh Ramesh\nprovider doc follow ups',
composerFound: true,
searchFailure: false,
});
const rows = await command.func(page, {
'thread-url': 'https://www.linkedin.com/messaging/thread/lokesh/',
'expected-name': 'Lokesh Ramesh',
message: 'both, but starting hands on',
});
expect(rows[0]).toMatchObject({ status: 'verified_dry_run', recipient: 'Lokesh Ramesh', reason: 'verified' });
expect(page.insertText).not.toHaveBeenCalled();
expect(page.pressKey).not.toHaveBeenCalled();
});
it('fills and sends only when --send is explicitly true and post-fill verification matches exactly', async () => {
const command = getRegistry().get('linkedin/safe-send');
const page = makeFakePage({
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
headerNames: ['Lokesh Ramesh'],
bodyText: 'Lokesh Ramesh\nprovider doc follow ups',
composerFound: true,
searchFailure: false,
});
const rows = await command.func(page, {
'thread-url': 'https://www.linkedin.com/messaging/thread/lokesh/',
'expected-name': 'Lokesh Ramesh',
message: 'both, but starting hands on',
send: true,
});
expect(rows[0]).toMatchObject({ status: 'sent', recipient: 'Lokesh Ramesh', reason: 'verified' });
expect(page.insertText).toHaveBeenCalledWith('both, but starting hands on');
expect(page.pressKey).not.toHaveBeenCalled();
});
});
+210
View File
@@ -0,0 +1,210 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
const SALES_INBOX_URL = 'https://www.linkedin.com/sales/inbox/';
const THREADS_BASE = 'https://www.linkedin.com/sales-api/salesApiMessagingThreads';
const PAGE_SIZE = 20;
const DEFAULT_LIMIT = 40;
const MAX_LIMIT = 500;
const THREAD_DECORATION = '(id,restrictions,archived,unreadMessageCount,nextPageStartsAt,totalMessageCount,messages*(id,type,contentFlag,deliveredAt,lastEditedAt,subject,body,footerText,blockCopy,attachments,author,systemMessageContent),participants*~fs_salesProfile(entityUrn,firstName,lastName,fullName,degree,profilePictureDisplayImage,objectUrn,inmailRestriction))';
export function normalizeWhitespace(value) {
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
}
export function parseLimit(value, defaultValue = DEFAULT_LIMIT) {
if (value === undefined || value === null || value === '') return defaultValue;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
}
return limit;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
export function encodeRestliDecoration(value) {
// LinkedIn's Sales Navigator Rest.li endpoint returns HTTP 400 when this
// decoration is sent with literal parentheses. Keep parentheses percent-encoded.
return encodeURIComponent(value).replace(/\(/g, '%28').replace(/\)/g, '%29');
}
function salesnavThreadUrl(threadId) {
return threadId ? `https://www.linkedin.com/sales/inbox/${encodeURIComponent(threadId)}` : '';
}
function threadListUrl({ count = PAGE_SIZE, pageStartsAt = '' } = {}) {
let url = `${THREADS_BASE}?decoration=${encodeRestliDecoration(THREAD_DECORATION)}&count=${count}&filter=INBOX&q=filter`;
if (pageStartsAt) url += `&pageStartsAt=${encodeURIComponent(pageStartsAt)}`;
return url;
}
function getThreadParticipants(thread) {
const resolution = thread?.participantsResolutionResults || {};
const participants = Array.isArray(thread?.participants) ? thread.participants : Object.keys(resolution);
return participants.map((urn) => resolution[urn] || { entityUrn: urn }).filter(Boolean);
}
function isSelfParticipant(profile) {
const degree = String(profile?.degree ?? '').trim();
return degree === '0';
}
function otherParticipantName(thread) {
const participants = getThreadParticipants(thread);
const other = participants.find((p) => !isSelfParticipant(p)) || participants[0];
return normalizeWhitespace(other?.fullName || [other?.firstName, other?.lastName].filter(Boolean).join(' '));
}
function parseSalesnavThreads(json) {
if (!json || typeof json !== 'object' || !Array.isArray(json.elements)) {
throw new CommandExecutionError('Sales Navigator messaging threads API returned malformed payload');
}
return json.elements.map((thread) => {
if (!thread || typeof thread !== 'object') {
throw new CommandExecutionError('Sales Navigator messaging threads API returned malformed thread row');
}
const messages = Array.isArray(thread?.messages) ? thread.messages : [];
const lastMessage = messages[0] || {};
const deliveredAt = Number(lastMessage.deliveredAt || thread?.nextPageStartsAt || 0);
const threadId = normalizeWhitespace(thread?.id || '');
if (!threadId) {
throw new CommandExecutionError('Sales Navigator messaging thread row missing id');
}
return {
thread_id: threadId,
thread_url: salesnavThreadUrl(threadId),
person_name: otherParticipantName(thread),
last_message_snippet: normalizeWhitespace(lastMessage.body || lastMessage.subject || '').slice(0, 300),
last_activity_time: deliveredAt ? new Date(deliveredAt).toISOString() : '',
unread: Number(thread?.unreadMessageCount || 0) > 0,
unread_count: Number(thread?.unreadMessageCount || 0),
total_message_count: Number(thread?.totalMessageCount || messages.length || 0),
archived: Boolean(thread?.archived),
next_page_starts_at: normalizeWhitespace(thread?.nextPageStartsAt || ''),
participants: getThreadParticipants(thread).map((p) => ({
name: normalizeWhitespace(p.fullName || [p.firstName, p.lastName].filter(Boolean).join(' ')),
entity_urn: normalizeWhitespace(p.entityUrn || ''),
object_urn: normalizeWhitespace(p.objectUrn || ''),
degree: normalizeWhitespace(p.degree ?? ''),
})),
};
});
}
function fetchJsonScript(url, csrf) {
return String.raw`(async () => {
try {
const res = await fetch(${JSON.stringify(url)}, {
credentials: 'include',
headers: {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: 'application/json',
},
});
const text = await res.text();
let json = null;
try { json = text ? JSON.parse(text) : null; } catch (_) { json = null; }
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status, text };
if (!res.ok) return { error: 'HTTP ' + res.status, status: res.status, text, json };
return { status: res.status, json };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
})()`;
}
export async function getCsrf(page) {
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
return jsession.replace(/^\"|\"$/g, '');
}
export async function fetchSalesnavJson(page, csrf, url, label) {
const result = unwrapEvaluateResult(await page.evaluate(fetchJsonScript(url, csrf)));
if (result?.authRequired) throw new AuthRequiredError(LINKEDIN_DOMAIN, `${label} authentication failed (HTTP ${result.status || 'auth_required'}).`);
if (result?.error || !result?.json) throw new CommandExecutionError(`${label} returned an unexpected response`, `${result?.error || 'no_json'}\n${normalizeWhitespace(result?.text || '').slice(0, 500)}`);
return result.json;
}
export async function fetchInboxRows(page, { limit = DEFAULT_LIMIT, maxPages = 30 } = {}) {
const csrf = await getCsrf(page);
const rows = [];
const seen = new Set();
let pageStartsAt = '';
let pagesFetched = 0;
let hasMorePages = false;
while (rows.length < limit && pagesFetched < maxPages) {
const json = await fetchSalesnavJson(page, csrf, threadListUrl({ count: PAGE_SIZE, pageStartsAt }), 'Sales Navigator messaging threads API');
pagesFetched += 1;
const pageRows = parseSalesnavThreads(json);
if (pageRows.length === 0) break;
for (const row of pageRows) {
if (seen.has(row.thread_id)) continue;
seen.add(row.thread_id);
rows.push(row);
if (rows.length >= limit) break;
}
const last = pageRows[pageRows.length - 1];
const next = last?.next_page_starts_at;
hasMorePages = Boolean(next);
if (!next) break;
if (next === pageStartsAt) {
throw new CommandExecutionError('Sales Navigator messaging threads API returned the same cursor twice');
}
pageStartsAt = next;
}
if (rows.length < limit && hasMorePages && pagesFetched >= maxPages) {
throw new CommandExecutionError(`Sales Navigator messaging threads API reached the ${maxPages}-page safety cap before collecting ${limit} conversations`);
}
return rows.slice(0, limit).map((row, index) => ({ ...row, rank: index + 1 }));
}
export { THREAD_DECORATION, THREADS_BASE };
cli({
site: 'linkedin',
name: 'salesnav-inbox',
access: 'read',
description: 'List LinkedIn Sales Navigator message conversations with API pagination',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'limit', type: 'number', default: DEFAULT_LIMIT, help: 'Maximum conversations to return (1-500)' },
{ name: 'max-pages', type: 'number', default: 30, help: 'Maximum Sales Navigator API pages to fetch' },
{ name: 'unread-only', type: 'bool', default: false, help: 'Return only unread conversations' },
],
columns: ['rank', 'thread_id', 'thread_url', 'person_name', 'last_message_snippet', 'last_activity_time', 'unread', 'unread_count', 'total_message_count', 'archived', 'participants', 'next_page_starts_at'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-inbox');
const limit = parseLimit(args.limit);
const maxPages = parseLimit(args['max-pages'], 30);
await page.goto(SALES_INBOX_URL);
await page.wait(4);
let rows = await fetchInboxRows(page, { limit, maxPages });
if (args['unread-only']) rows = rows.filter((row) => row.unread);
if (rows.length === 0) {
if (args['unread-only']) return [];
throw new EmptyResultError('linkedin salesnav-inbox', 'No Sales Navigator conversations were found.');
}
return rows.slice(0, limit).map((row, index) => ({ ...row, rank: index + 1 }));
},
});
export const __test__ = {
THREAD_DECORATION,
normalizeWhitespace,
parseLimit,
encodeRestliDecoration,
salesnavThreadUrl,
threadListUrl,
parseSalesnavThreads,
fetchInboxRows,
};
+113
View File
@@ -0,0 +1,113 @@
import { describe, expect, it } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import './salesnav-inbox.js';
const {
THREAD_DECORATION,
encodeRestliDecoration,
parseLimit,
parseSalesnavThreads,
salesnavThreadUrl,
threadListUrl,
} = await import('./salesnav-inbox.js').then((m) => m.__test__);
describe('linkedin salesnav-inbox command', () => {
it('percent-encodes Rest.li decoration parentheses for Sales Navigator messaging', () => {
const encoded = encodeRestliDecoration('(id,messages*(body))');
expect(encoded).toBe('%28id%2Cmessages*%28body%29%29');
expect(encoded).not.toContain('(');
expect(encoded).not.toContain(')');
});
it('builds the paginated salesApiMessagingThreads inbox URL', () => {
const url = threadListUrl({ count: 20, pageStartsAt: '1779070755626' });
expect(url).toContain('/sales-api/salesApiMessagingThreads?');
expect(url).toContain('q=filter');
expect(url).toContain('filter=INBOX');
expect(url).toContain('count=20');
expect(url).toContain('pageStartsAt=1779070755626');
expect(url).toContain(encodeRestliDecoration(THREAD_DECORATION));
});
it('validates limits without silent clamping', () => {
expect(parseLimit(undefined)).toBe(40);
expect(parseLimit(12)).toBe(12);
expect(() => parseLimit(0)).toThrow();
expect(() => parseLimit(501)).toThrow();
expect(() => parseLimit('abc')).toThrow();
});
it('parses Sales Navigator thread rows with other participant and unread state', () => {
const rows = parseSalesnavThreads({ elements: [{
id: '2-thread',
unreadMessageCount: 1,
archived: false,
totalMessageCount: 2,
nextPageStartsAt: 1778206803669,
participants: [
'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)',
],
participantsResolutionResults: {
'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)': {
entityUrn: 'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
firstName: 'Rachael',
lastName: 'Stolberg',
fullName: 'Rachael Stolberg',
degree: 2,
},
'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)': {
entityUrn: 'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)',
firstName: 'Hanzi',
lastName: 'Li',
fullName: 'Hanzi Li',
degree: 0,
},
},
messages: [{
id: 'msg-1',
author: 'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
body: 'Hi hanzi, happy to chat',
deliveredAt: 1778206803669,
}],
}] });
expect(rows).toHaveLength(1);
expect(rows[0]).toMatchObject({
thread_id: '2-thread',
thread_url: salesnavThreadUrl('2-thread'),
person_name: 'Rachael Stolberg',
last_message_snippet: 'Hi hanzi, happy to chat',
last_activity_time: '2026-05-08T02:20:03.669Z',
unread: true,
unread_count: 1,
total_message_count: 2,
});
});
it('does not hard-code a specific account name as the inbox owner', () => {
const rows = parseSalesnavThreads({ elements: [{
id: '2-thread',
participants: [
'urn:li:fs_salesProfile:(HANZI,NAME_SEARCH,T1)',
'urn:li:fs_salesProfile:(ME,NAME_SEARCH,T2)',
],
participantsResolutionResults: {
'urn:li:fs_salesProfile:(HANZI,NAME_SEARCH,T1)': {
fullName: 'Hanzi Li',
degree: 2,
},
'urn:li:fs_salesProfile:(ME,NAME_SEARCH,T2)': {
fullName: 'Current User',
degree: 0,
},
},
messages: [],
}] });
expect(rows[0].person_name).toBe('Hanzi Li');
});
it('fails typed on malformed thread payloads and missing thread identity', () => {
expect(() => parseSalesnavThreads({})).toThrow(CommandExecutionError);
expect(() => parseSalesnavThreads({ elements: [{}] })).toThrow(CommandExecutionError);
});
});
+360
View File
@@ -0,0 +1,360 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
const SALES_HOME = 'https://www.linkedin.com/sales/';
const PROFILE_DECO = '(entityUrn,objectUrn,firstName,lastName,fullName,headline,degree,inmailRestriction,memberBadges,defaultPosition)';
const CREDITS_URL = 'https://www.linkedin.com/sales-api/salesApiCredits?q=findCreditGrant&creditGrantType=LSS_INMAIL';
const MESSAGE_ACTION_URL = 'https://www.linkedin.com/sales-api/salesApiMessageActions?action=createMessage';
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
}
function requireStringArg(args, key, label = key) {
const value = normalizeWhitespace(args[key]);
if (!value) throw new ArgumentError(`${label} is required`);
return value;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
function isLinkedInHost(hostname) {
const host = String(hostname || '').toLowerCase();
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
}
function parseSalesProfileUrn(value) {
const raw = normalizeWhitespace(value);
const match = raw.match(/^urn:li:fs_salesProfile:\(([^,()]+),([^,()]+),([^,()]+)\)$/);
if (!match) return null;
if (!isResolvedSalesProfileParts(match[1], match[2], match[3])) return null;
return { profileId: match[1], authType: match[2], authToken: match[3], entityUrn: raw };
}
function isResolvedSalesProfileParts(profileId, authType, authToken) {
return [profileId, authType, authToken].every((part) => {
const clean = normalizeWhitespace(part).toLowerCase();
return clean && clean !== 'undefined' && clean !== 'null' && clean !== 'not_available';
});
}
function salesLeadUrlFromParts({ profileId, authType, authToken }) {
return `https://www.linkedin.com/sales/lead/${encodeURIComponent(profileId)},${encodeURIComponent(authType)},${encodeURIComponent(authToken)}`;
}
function parseRecipient(value) {
const raw = normalizeWhitespace(value);
const urn = parseSalesProfileUrn(raw);
if (urn) return urn;
try {
const url = new URL(raw);
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return null;
const salesMatch = url.pathname.match(/^\/sales\/lead\/([^,/]+),([^,/]+),([^/]+)\/?$/i);
if (salesMatch) {
const profileId = decodeURIComponent(salesMatch[1]);
const authType = decodeURIComponent(salesMatch[2]);
const authToken = decodeURIComponent(salesMatch[3]);
if (!isResolvedSalesProfileParts(profileId, authType, authToken)) return null;
return { profileId, authType, authToken, entityUrn: `urn:li:fs_salesProfile:(${profileId},${authType},${authToken})` };
}
const profileMatch = url.pathname.match(/^\/in\/([^/]+)\/?$/i);
if (profileMatch) {
return { profileId: decodeURIComponent(profileMatch[1]), authType: '', authToken: '', entityUrn: '' };
}
} catch {
return null;
}
return null;
}
function encodeRestliDecoration(value) {
return encodeURIComponent(value).replace(/\(/g, '%28').replace(/\)/g, '%29');
}
function profileApiUrl(recipient) {
if (!recipient?.profileId || !recipient?.authType || !recipient?.authToken) return '';
const key = `(profileId:${recipient.profileId},authType:${recipient.authType},authToken:${recipient.authToken})`;
return `https://www.linkedin.com/sales-api/salesApiProfiles/${key}?decoration=${encodeRestliDecoration(PROFILE_DECO)}`;
}
function randomTrackingId() {
const bytes = new Uint8Array(8);
if (globalThis.crypto?.getRandomValues) globalThis.crypto.getRandomValues(bytes);
else for (let i = 0; i < bytes.length; i += 1) bytes[i] = Math.floor(Math.random() * 256);
return Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('');
}
function buildCreateMessagePayload({ recipientUrn, subject, body, trackingId = randomTrackingId(), copyToCrm = false }) {
const cleanRecipient = normalizeWhitespace(recipientUrn);
if (!parseSalesProfileUrn(cleanRecipient)) throw new ArgumentError('--recipient must resolve to a Sales Navigator lead urn');
const cleanSubject = normalizeWhitespace(subject);
const cleanBody = String(body ?? '').trim();
if (!cleanSubject) throw new ArgumentError('--subject is required');
if (!cleanBody) throw new ArgumentError('--body is required');
if (cleanSubject.length > 200) throw new ArgumentError('--subject must be 200 characters or fewer');
if (cleanBody.length > 1900) throw new ArgumentError('--body must be 1900 characters or fewer');
return {
createMessageRequest: {
recipients: [cleanRecipient],
subject: cleanSubject,
body: cleanBody,
copyToCrm: Boolean(copyToCrm),
trackingId,
},
};
}
function extractRemainingCredits(json) {
const elements = Array.isArray(json?.elements) ? json.elements : [];
const inmailGrant = elements.find((el) => el?.type === 'LSS_INMAIL' && Number.isInteger(el.value));
if (inmailGrant) return inmailGrant.value;
const candidates = [];
const visit = (value) => {
if (value === null || value === undefined) return;
if (typeof value === 'number' && Number.isFinite(value)) candidates.push(value);
if (Array.isArray(value)) value.forEach(visit);
else if (typeof value === 'object') {
for (const [key, child] of Object.entries(value)) {
if (/remaining|available|balance|value/i.test(key) && typeof child === 'number') candidates.unshift(child);
else if (!/^count$|^start$|^id$/i.test(key)) visit(child);
}
}
};
visit(json);
return candidates.find((n) => Number.isInteger(n) && n >= 0) ?? null;
}
function fetchJsonScript(url, csrf, options = {}) {
return String.raw`(async () => {
const headers = {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: ${JSON.stringify(options.accept || 'application/json')},
...((${JSON.stringify(Boolean(options.body))}) ? { 'content-type': 'application/json' } : {}),
};
try {
const res = await fetch(${JSON.stringify(url)}, {
credentials: 'include',
method: ${JSON.stringify(options.method || 'GET')},
headers,
body: ${options.body ? JSON.stringify(JSON.stringify(options.body)) : 'undefined'},
});
const text = await res.text();
let json = null;
try { json = text ? JSON.parse(text) : null; } catch (_) { json = null; }
if (res.status === 401 || res.status === 403) return ['auth', res.status, json, text];
if (!res.ok) return ['error', res.status, json, text, 'HTTP ' + res.status];
return ['ok', res.status, json, text];
} catch (e) {
return ['error', 0, null, '', 'fetch failed: ' + ((e && e.message) || String(e))];
}
})()`;
}
function requireFetchResult(result, label, { requireJson = true } = {}) {
if (Array.isArray(result)) {
const [kind, status, json, text, error] = result;
result = {
authRequired: kind === 'auth',
error: kind === 'error' ? error || `HTTP ${status}` : '',
status,
json,
text,
};
}
if (result?.authRequired) throw new AuthRequiredError(LINKEDIN_DOMAIN, `${label} auth failed.`);
if (result?.error) throw new CommandExecutionError(`${label} failed`, result.error);
if (!result || typeof result !== 'object' || Array.isArray(result)) {
throw new CommandExecutionError(`${label} returned malformed response`);
}
if (requireJson && (!result.json || typeof result.json !== 'object' || Array.isArray(result.json))) {
throw new CommandExecutionError(`${label} returned malformed response`, 'missing_json');
}
return result;
}
function salesPageShowsSentMessage(text, recipientName) {
const normalizedText = normalizeWhitespace(text);
const firstName = normalizeWhitespace(recipientName).split(' ')[0];
return normalizedText.includes('You sent a Sales Navigator message')
&& (!firstName || normalizedText.includes(firstName));
}
async function getCsrf(page) {
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
return jsession.replace(/^\"|\"$/g, '');
}
async function resolveRecipient(page, parsed, csrf) {
if (!parsed) throw new ArgumentError('--recipient must be a Sales Navigator lead URL, Sales Navigator profile URL, LinkedIn /in/ URL, or urn:li:fs_salesProfile:(...)');
if (parsed.entityUrn && parsed.authType && parsed.authToken) return parsed;
await page.goto(`https://www.linkedin.com/sales/lead/${encodeURIComponent(parsed.profileId)}`);
await page.wait(6);
const probe = unwrapEvaluateResult(await page.evaluate(String.raw`(() => {
const href = location.href;
const text = document.body ? document.body.innerText : '';
const resourceUrns = Array.from(performance.getEntriesByType('resource'))
.map((entry) => entry.name)
.filter((name) => name.includes('/sales-api/salesApiProfiles/'))
.slice(-20);
return { href, text: text.slice(0, 1000), resourceUrns };
})()`));
const urlMatch = String(probe?.href || '').match(/\/sales\/lead\/([^,/]+),([^,/]+),([^/?#]+)/i);
if (urlMatch && isResolvedSalesProfileParts(urlMatch[1], urlMatch[2], urlMatch[3])) {
return {
profileId: decodeURIComponent(urlMatch[1]),
authType: decodeURIComponent(urlMatch[2]),
authToken: decodeURIComponent(urlMatch[3]),
entityUrn: `urn:li:fs_salesProfile:(${decodeURIComponent(urlMatch[1])},${decodeURIComponent(urlMatch[2])},${decodeURIComponent(urlMatch[3])})`,
};
}
for (const resource of probe?.resourceUrns || []) {
const resourceMatch = String(resource).match(/profileId:([^,)]+),authType:([^,)]+),authToken:([^,)]+)\)/);
if (resourceMatch && resourceMatch[1] === parsed.profileId) {
return {
profileId: resourceMatch[1],
authType: resourceMatch[2],
authToken: resourceMatch[3],
entityUrn: `urn:li:fs_salesProfile:(${resourceMatch[1]},${resourceMatch[2]},${resourceMatch[3]})`,
};
}
}
void csrf;
throw new CommandExecutionError('Could not resolve Sales Navigator auth token for recipient', `Observed URL: ${probe?.href || 'url_not_available'}\nBody: ${normalizeWhitespace(probe?.text || '').slice(0, 500)}`);
}
function profileSummary(json) {
const data = json?.data || json || {};
const pos = data.defaultPosition || (Array.isArray(data.positions) ? data.positions.find((p) => p.current) || data.positions[0] : {}) || {};
return {
recipient: normalizeWhitespace(data.fullName || [data.firstName, data.lastName].filter(Boolean).join(' ')),
title: normalizeWhitespace(pos.title || data.headline || ''),
company: normalizeWhitespace(pos.companyName || pos.company?.name || ''),
degree: normalizeWhitespace(data.degree || ''),
inmail_restriction: normalizeWhitespace(data.inmailRestriction || ''),
open_link: Boolean(data.memberBadges?.openLink),
};
}
function requireProfileSummary(json) {
const summary = profileSummary(json);
if (!summary.recipient) {
throw new CommandExecutionError('Sales Navigator profile lookup returned malformed profile data', 'missing_recipient_name');
}
return summary;
}
cli({
site: 'linkedin',
name: 'salesnav-message',
access: 'write',
description: 'Send or dry-run a LinkedIn Sales Navigator InMail to a lead using the Sales Navigator messaging API',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'recipient', type: 'string', required: true, positional: true, help: 'Sales Navigator lead URL, LinkedIn /in/ URL from salesnav-search, or urn:li:fs_salesProfile:(...)' },
{ name: 'subject', type: 'string', required: true, help: 'InMail subject' },
{ name: 'body', type: 'string', required: true, help: 'InMail body' },
{ name: 'send', type: 'bool', default: false, help: 'Actually send the InMail. Default is dry-run validation only.' },
{ name: 'copy-to-crm', type: 'bool', default: false, help: 'Set Sales Navigator copyToCrm on the message request' },
],
columns: ['status', 'recipient', 'title', 'company', 'credits_remaining', 'credits_before', 'credits_after', 'sent_in_salesnav', 'message_chars', 'subject_chars', 'recipient_urn', 'degree', 'inmail_restriction', 'open_link'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-message');
const recipientArg = requireStringArg(args, 'recipient', '--recipient');
const subject = requireStringArg(args, 'subject', '--subject');
const body = String(args.body ?? '').trim();
if (!body) throw new ArgumentError('--body is required');
await page.goto(SALES_HOME);
await page.wait(4);
const csrf = await getCsrf(page);
const recipient = await resolveRecipient(page, parseRecipient(recipientArg), csrf);
let summary = { recipient: '', title: '', company: '', degree: '', inmail_restriction: '', open_link: false };
const profileUrl = profileApiUrl(recipient);
if (profileUrl) {
const profileResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(profileUrl, csrf))), 'LinkedIn Sales Navigator profile API');
summary = requireProfileSummary(profileResult.json);
}
if (summary.inmail_restriction && summary.inmail_restriction !== 'NO_RESTRICTION') {
throw new CommandExecutionError('Sales Navigator InMail blocked by recipient restriction', summary.inmail_restriction);
}
const creditsResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(CREDITS_URL, csrf))), 'LinkedIn Sales Navigator credits API');
const creditsRemaining = extractRemainingCredits(creditsResult?.json);
const payload = buildCreateMessagePayload({ recipientUrn: recipient.entityUrn, subject, body, copyToCrm: args['copy-to-crm'] });
if (!args.send) {
return [{
status: 'validated_dry_run',
recipient: summary.recipient,
title: summary.title,
company: summary.company,
credits_remaining: creditsRemaining,
credits_before: creditsRemaining,
credits_after: '',
sent_in_salesnav: false,
message_chars: body.length,
subject_chars: subject.length,
recipient_urn: recipient.entityUrn,
degree: summary.degree,
inmail_restriction: summary.inmail_restriction,
open_link: summary.open_link,
}];
}
const sendResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(MESSAGE_ACTION_URL, csrf, {
method: 'POST',
accept: 'application/vnd.linkedin.normalized+json+2.1',
body: payload,
}))), 'LinkedIn Sales Navigator message API', { requireJson: false });
void sendResult;
await page.wait(3);
const creditsAfterResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(CREDITS_URL, csrf))), 'LinkedIn Sales Navigator credits API after send');
const creditsAfter = extractRemainingCredits(creditsAfterResult?.json);
await page.goto(salesLeadUrlFromParts(recipient));
await page.wait(6);
const salesPageText = unwrapEvaluateResult(await page.evaluate('document.body ? document.body.innerText : ""'));
const sentInSalesNav = salesPageShowsSentMessage(salesPageText, summary.recipient);
if (!sentInSalesNav) throw new CommandExecutionError('Sales Navigator post-send verification failed', 'Sent activity was not found on the Sales Navigator lead page.');
return [{
status: 'sent',
recipient: summary.recipient,
title: summary.title,
company: summary.company,
credits_remaining: creditsAfter,
credits_before: creditsRemaining,
credits_after: creditsAfter,
sent_in_salesnav: sentInSalesNav,
message_chars: body.length,
subject_chars: subject.length,
recipient_urn: recipient.entityUrn,
degree: summary.degree,
inmail_restriction: summary.inmail_restriction,
open_link: summary.open_link,
}];
},
});
export const __test__ = {
normalizeWhitespace,
parseSalesProfileUrn,
isResolvedSalesProfileParts,
parseRecipient,
salesLeadUrlFromParts,
profileApiUrl,
buildCreateMessagePayload,
extractRemainingCredits,
profileSummary,
requireProfileSummary,
salesPageShowsSentMessage,
};
+172
View File
@@ -0,0 +1,172 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import './salesnav-message.js';
const {
parseSalesProfileUrn,
parseRecipient,
salesLeadUrlFromParts,
profileApiUrl,
buildCreateMessagePayload,
extractRemainingCredits,
profileSummary,
requireProfileSummary,
salesPageShowsSentMessage,
} = await import('./salesnav-message.js').then((m) => m.__test__);
function createPageMock(evaluateResults = []) {
const evaluate = vi.fn();
for (const result of evaluateResults) evaluate.mockResolvedValueOnce(result);
evaluate.mockResolvedValue(undefined);
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate,
getCookies: vi.fn().mockResolvedValue([{ name: 'JSESSIONID', value: '"csrf"', domain: '.linkedin.com' }]),
};
}
describe('linkedin salesnav-message command', () => {
it('parses Sales Navigator profile urns and lead URLs', () => {
const urn = 'urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)';
expect(parseSalesProfileUrn(urn)).toMatchObject({
profileId: 'ACwAAAJS8TABxyz',
authType: 'NAME_SEARCH',
authToken: 'Enlo',
entityUrn: urn,
});
expect(parseSalesProfileUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,undefined,undefined)')).toBeNull();
const parsed = parseRecipient('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
expect(parsed).toMatchObject({ profileId: 'ACwAAAJS8TABxyz', authType: 'NAME_SEARCH', authToken: 'Enlo' });
expect(parsed.entityUrn).toBe(urn);
expect(salesLeadUrlFromParts(parsed)).toBe('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
});
it('accepts LinkedIn /in tokens as unresolved recipients', () => {
expect(parseRecipient('https://www.linkedin.com/in/ACwAAAJS8TABxyz/')).toMatchObject({
profileId: 'ACwAAAJS8TABxyz',
authType: '',
authToken: '',
entityUrn: '',
});
});
it('builds profile API URLs with the Sales Navigator auth key', () => {
const url = profileApiUrl({ profileId: 'P1', authType: 'NAME_SEARCH', authToken: 'T1' });
expect(url).toContain('/sales-api/salesApiProfiles/(profileId:P1,authType:NAME_SEARCH,authToken:T1)');
expect(url).toContain('decoration=');
});
it('constructs the createMessage action payload used by Sales Navigator', () => {
const payload = buildCreateMessagePayload({
recipientUrn: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
subject: 'Quick QA doc question',
body: 'Hi Jane, can I ask a quick question?',
trackingId: '0123456789abcdef',
copyToCrm: false,
});
expect(payload).toEqual({
createMessageRequest: {
recipients: ['urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)'],
subject: 'Quick QA doc question',
body: 'Hi Jane, can I ask a quick question?',
copyToCrm: false,
trackingId: '0123456789abcdef',
},
});
});
it('validates payload fields before any send attempt', () => {
expect(() => buildCreateMessagePayload({ recipientUrn: '', subject: 's', body: 'b' })).toThrow();
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: '', body: 'b' })).toThrow();
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: 's', body: '' })).toThrow();
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: 'x'.repeat(201), body: 'b' })).toThrow();
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,undefined,undefined)', subject: 's', body: 'b' })).toThrow();
});
it('detects the Sales Navigator sent activity on a verified lead page', () => {
expect(salesPageShowsSentMessage('5/18/2026 You sent a Sales Navigator message to Jane', 'Jane Q')).toBe(true);
expect(salesPageShowsSentMessage('No recent activity', 'Jane Q')).toBe(false);
});
it('extracts a plausible remaining InMail credit count', () => {
expect(extractRemainingCredits({ elements: [{ type: 'LSS_INMAIL', value: 149, id: 1 }], paging: { count: 10 } })).toBe(149);
expect(extractRemainingCredits({ data: { remaining: 149, used: 1 } })).toBe(149);
expect(extractRemainingCredits({ elements: [{ availableCount: 12 }] })).toBe(12);
expect(extractRemainingCredits({})).toBe(null);
});
it('summarizes decorated Sales Navigator profile data', () => {
expect(profileSummary({ data: {
fullName: 'Rayki Goh',
headline: 'Food Safety',
degree: 3,
defaultPosition: { title: 'FSQA Manager', companyName: 'Acme Foods' },
memberBadges: { openLink: false },
} })).toMatchObject({
recipient: 'Rayki Goh',
title: 'FSQA Manager',
company: 'Acme Foods',
degree: '3',
open_link: false,
});
});
it('fails typed when decorated profile data has no recipient identity', () => {
expect(() => requireProfileSummary({ data: { defaultPosition: { title: 'FSQA' } } }))
.toThrow(CommandExecutionError);
});
it('keeps manifest columns aligned with dry-run rows and fails typed on malformed profile API', async () => {
const cmd = getRegistry().get('linkedin/salesnav-message');
expect(cmd?.columns).toEqual([
'status',
'recipient',
'title',
'company',
'credits_remaining',
'credits_before',
'credits_after',
'sent_in_salesnav',
'message_chars',
'subject_chars',
'recipient_urn',
'degree',
'inmail_restriction',
'open_link',
]);
const goodPage = createPageMock([
{ status: 200, json: { data: { fullName: 'Jane Doe', defaultPosition: { title: 'QA', companyName: 'Acme' }, degree: 2, inmailRestriction: 'NO_RESTRICTION', memberBadges: { openLink: true } } } },
{ status: 200, json: { elements: [{ type: 'LSS_INMAIL', value: 12 }] } },
]);
const rows = await cmd.func(goodPage, {
recipient: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
subject: 'Hello',
body: 'Quick question',
});
expect(Object.keys(rows[0]).sort()).toEqual([...cmd.columns].sort());
expect(rows[0]).toMatchObject({
status: 'validated_dry_run',
recipient: 'Jane Doe',
credits_remaining: 12,
credits_before: 12,
credits_after: '',
sent_in_salesnav: false,
degree: '2',
inmail_restriction: 'NO_RESTRICTION',
open_link: true,
});
const malformedPage = createPageMock([
{ status: 200, json: { data: { defaultPosition: { title: 'QA' } } } },
]);
await expect(cmd.func(malformedPage, {
recipient: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
subject: 'Hello',
body: 'Quick question',
})).rejects.toThrow(CommandExecutionError);
});
});
+186
View File
@@ -0,0 +1,186 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
const LINKEDIN_DOMAIN = 'www.linkedin.com';
const SALES_HOME = 'https://www.linkedin.com/sales/';
const LEAD_SEARCH_BASE = 'https://www.linkedin.com/sales-api/salesApiLeadSearch';
// Versioned response decoration. LinkedIn bumps this on Sales Navigator
// redeploys; if the response shape ever changes, refresh it from a live
// /sales/search/people request.
const LEAD_SEARCH_DECORATION = 'com.linkedin.sales.deco.desktop.searchv2.LeadSearchResult-14';
const PAGE_SIZE = 25;
function normalizeWhitespace(value) {
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
}
function requireStringArg(args, key, label = key) {
const value = normalizeWhitespace(args[key]);
if (!value) throw new ArgumentError(`${label} is required`);
return value;
}
function parseLimit(value) {
if (value === undefined || value === null || value === '') return 25;
const limit = Number(value);
if (!Number.isInteger(limit) || limit < 1 || limit > 500) {
throw new ArgumentError('--limit must be an integer between 1 and 500');
}
return limit;
}
function unwrapEvaluateResult(payload) {
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
return payload;
}
// Sales Navigator keeps the structural ( ) , : of the query literal and only
// percent-encodes the keyword value.
function leadSearchUrl(keywords, start) {
const query = '(spellCorrectionEnabled:true,recentSearchParam:(doLogHistory:true),keywords:'
+ encodeURIComponent(keywords) + ')';
return LEAD_SEARCH_BASE
+ '?q=searchQuery&query=' + query
+ '&start=' + start + '&count=' + PAGE_SIZE
+ '&decorationId=' + LEAD_SEARCH_DECORATION;
}
function fetchLeadSearchScript(url, csrf) {
return String.raw`(async () => {
const headers = {
'csrf-token': ${JSON.stringify(csrf)},
'x-restli-protocol-version': '2.0.0',
accept: 'application/json',
};
try {
const res = await fetch(${JSON.stringify(url)}, { credentials: 'include', headers });
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
if (!res.ok) return { error: 'HTTP ' + res.status };
return { json: await res.json() };
} catch (e) {
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
}
})()`;
}
// Sales Navigator search returns no /in/ vanity URL, but the entityUrn carries
// the obfuscated member token, and linkedin.com/in/<token> is a valid profile
// URL that the connect command accepts.
function profileUrlFromEntityUrn(entityUrn) {
const match = String(entityUrn || '').match(/fs_salesProfile:\(([^,)]+)/);
return match && match[1] ? 'https://www.linkedin.com/in/' + match[1] : '';
}
function leadUrlFromEntityUrn(entityUrn) {
const match = String(entityUrn || '').match(/^urn:li:fs_salesProfile:\(([^,()]+),([^,()]+),([^,()]+)\)$/);
if (!match) return '';
return `https://www.linkedin.com/sales/lead/${encodeURIComponent(match[1])},${encodeURIComponent(match[2])},${encodeURIComponent(match[3])}`;
}
function parseLeads(json) {
if (!json || typeof json !== 'object' || !Array.isArray(json.elements)) {
throw new CommandExecutionError('Sales Navigator lead search API returned malformed payload');
}
const leads = [];
for (const el of json.elements) {
if (!el || typeof el !== 'object') {
throw new CommandExecutionError('Sales Navigator lead search API returned malformed lead row');
}
const current = Array.isArray(el.currentPositions) ? el.currentPositions : [];
const past = Array.isArray(el.pastPositions) ? el.pastPositions : [];
const pos = current[0] || past[0] || {};
const name = normalizeWhitespace(el.fullName || [el.firstName, el.lastName].filter(Boolean).join(' '));
if (!name) {
throw new CommandExecutionError('Sales Navigator lead row missing name');
}
const entityUrn = normalizeWhitespace(el.entityUrn || '');
if (!profileUrlFromEntityUrn(entityUrn)) {
throw new CommandExecutionError('Sales Navigator lead row missing profile identity');
}
leads.push({
name,
title: normalizeWhitespace(pos.title || ''),
company: normalizeWhitespace(pos.companyName || ''),
location: normalizeWhitespace(el.geoRegion || ''),
degree: normalizeWhitespace(el.degree || ''),
profile_url: profileUrlFromEntityUrn(entityUrn),
lead_url: leadUrlFromEntityUrn(entityUrn),
recipient_urn: entityUrn,
});
}
return leads;
}
function requireLeadSearchResult(result) {
if (result?.authRequired) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn Sales Navigator API auth failed (HTTP ' + (result.status || '') + '). Confirm the account has Sales Navigator access.');
}
if (result?.error) {
throw new CommandExecutionError('Sales Navigator lead search API returned an unexpected response', result.error);
}
if (!result || !result.json) {
throw new CommandExecutionError('Sales Navigator lead search API returned an unexpected response', 'no_json');
}
return result.json;
}
cli({
site: 'linkedin',
name: 'salesnav-search',
access: 'read',
description: 'Search LinkedIn Sales Navigator for people leads by keyword',
domain: LINKEDIN_DOMAIN,
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'People search keywords, e.g. "quality manager food manufacturing"' },
{ name: 'limit', type: 'number', default: 25, help: 'Maximum leads to return (1-500, fetched 25 per request)' },
],
columns: ['rank', 'name', 'title', 'company', 'location', 'degree', 'profile_url', 'lead_url', 'recipient_urn'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-search');
const keywords = requireStringArg(args, 'keywords', '--keywords');
const limit = parseLimit(args.limit);
await page.goto(SALES_HOME);
await page.wait(6);
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
if (!jsession) {
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
}
const csrf = jsession.replace(/^\"|\"$/g, '');
const leads = [];
const seen = new Set();
for (let start = 0; leads.length < limit && start < 2000; start += PAGE_SIZE) {
const result = unwrapEvaluateResult(await page.evaluate(fetchLeadSearchScript(leadSearchUrl(keywords, start), csrf)));
const json = requireLeadSearchResult(result);
const pageLeads = parseLeads(json);
if (pageLeads.length === 0) break;
for (const lead of pageLeads) {
const key = lead.profile_url || lead.name.toLowerCase();
if (seen.has(key)) continue;
seen.add(key);
leads.push(lead);
}
await page.wait(1);
}
if (leads.length === 0) {
throw new EmptyResultError('linkedin salesnav-search', 'No Sales Navigator leads were found.');
}
return leads.slice(0, limit).map((lead, index) => ({ rank: index + 1, ...lead }));
},
});
export const __test__ = {
normalizeWhitespace,
parseLimit,
leadSearchUrl,
profileUrlFromEntityUrn,
leadUrlFromEntityUrn,
parseLeads,
requireLeadSearchResult,
};
+76
View File
@@ -0,0 +1,76 @@
import { describe, expect, it } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import './salesnav-search.js';
const {
parseLimit,
leadSearchUrl,
profileUrlFromEntityUrn,
leadUrlFromEntityUrn,
parseLeads,
requireLeadSearchResult,
} = await import('./salesnav-search.js').then((m) => m.__test__);
describe('linkedin salesnav-search command', () => {
it('builds a salesApiLeadSearch URL with encoded keywords and pagination', () => {
const url = leadSearchUrl('quality manager food', 50);
expect(url).toContain('/sales-api/salesApiLeadSearch');
expect(url).toContain('keywords:quality%20manager%20food');
expect(url).toContain('start=50');
expect(url).toContain('count=25');
});
it('derives a profile URL from the sales-profile entityUrn token', () => {
expect(profileUrlFromEntityUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)'))
.toBe('https://www.linkedin.com/in/ACwAAAJS8TABxyz');
expect(profileUrlFromEntityUrn('')).toBe('');
expect(profileUrlFromEntityUrn('not-a-urn')).toBe('');
});
it('derives a Sales Navigator lead URL from the full sales-profile entityUrn', () => {
expect(leadUrlFromEntityUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)'))
.toBe('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
expect(leadUrlFromEntityUrn('not-a-urn')).toBe('');
});
it('validates --limit without silent clamping', () => {
expect(parseLimit(undefined)).toBe(25);
expect(parseLimit(120)).toBe(120);
expect(() => parseLimit(0)).toThrow();
expect(() => parseLimit(999)).toThrow();
expect(() => parseLimit('abc')).toThrow();
});
it('parses lead rows and falls back to past positions', () => {
const json = { elements: [
{ fullName: 'Jane Q', geoRegion: 'Vancouver, BC', degree: 2,
entityUrn: 'urn:li:fs_salesProfile:(TOKEN1,NAME_SEARCH,abc)',
currentPositions: [{ title: 'QA Manager', companyName: 'Acme Foods' }] },
{ fullName: 'No Current', geoRegion: 'Toronto',
entityUrn: 'urn:li:fs_salesProfile:(TOKEN2,NAME_SEARCH,def)',
currentPositions: [], pastPositions: [{ title: 'Past QA Lead', companyName: 'Old Co' }] },
] };
const leads = parseLeads(json);
expect(leads).toHaveLength(2);
expect(leads[0]).toMatchObject({
name: 'Jane Q',
title: 'QA Manager',
company: 'Acme Foods',
location: 'Vancouver, BC',
profile_url: 'https://www.linkedin.com/in/TOKEN1',
lead_url: 'https://www.linkedin.com/sales/lead/TOKEN1,NAME_SEARCH,abc',
recipient_urn: 'urn:li:fs_salesProfile:(TOKEN1,NAME_SEARCH,abc)',
});
expect(leads[1]).toMatchObject({ name: 'No Current', title: 'Past QA Lead', company: 'Old Co' });
});
it('fails typed on malformed lead payloads instead of silently dropping rows', () => {
expect(() => parseLeads({})).toThrow(CommandExecutionError);
expect(() => parseLeads({ elements: [{ firstName: '', lastName: '', entityUrn: 'urn:li:fs_salesProfile:(TOKEN3,x,y)' }] }))
.toThrow(CommandExecutionError);
expect(() => parseLeads({ elements: [{ fullName: 'No Identity' }] }))
.toThrow(CommandExecutionError);
expect(() => requireLeadSearchResult({ error: 'HTTP 500' })).toThrow(CommandExecutionError);
expect(() => requireLeadSearchResult({})).toThrow(CommandExecutionError);
});
});

Some files were not shown because too many files have changed in this diff Show More