Compare commits
101 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| ce432c2428 | |||
| 7ee16aa087 | |||
| 7a2ab47bf8 | |||
| 2c8b50c4fd | |||
| 51a9456305 | |||
| 62592547a4 | |||
| 4a3a55634a | |||
| da497f0b02 | |||
| 2590278f43 | |||
| 488e407a65 | |||
| e682c1c30a | |||
| 86f57c0846 | |||
| 4a6cfe8060 | |||
| bcb0fb362f | |||
| 85b1c07ba9 | |||
| 6fbaf0d5b8 | |||
| 34f351e59f | |||
| acc18be999 | |||
| 67ed9e9c81 | |||
| 8577d88ee7 | |||
| cd731bd2aa | |||
| d1c714ecd3 | |||
| 8182ffbe89 | |||
| 5a4984789d | |||
| e1185da882 | |||
| 9446bddb60 | |||
| 0c4bcdbb86 | |||
| ec3b7dadf3 | |||
| 4de04c43ad | |||
| 40592ea5fd | |||
| 942539a695 | |||
| dc645a5bcf | |||
| f0d9aa187c | |||
| e82e32abc6 | |||
| 254d51835f | |||
| 87dfb68e74 | |||
| 000c867f3a | |||
| 24f643af16 | |||
| 1f30a9027b | |||
| 72f2b020de | |||
| 261b8bfbb5 | |||
| 1e7ebe7f27 | |||
| 7e44e71150 | |||
| 76a9c78261 | |||
| 030a0ad885 | |||
| e29150bab5 | |||
| a50074d684 | |||
| 368581ea4d | |||
| 0c488bbf51 | |||
| 86792d2954 | |||
| ee54eb8e62 | |||
| 663b3387ee | |||
| 716461581a | |||
| 854cf01aad | |||
| baf1522420 | |||
| e3995df25c | |||
| 4682ffc3de | |||
| e3140af5ee | |||
| 43f5c6e1cf | |||
| 68ef95659f | |||
| c922a39a7d | |||
| aae6e823b4 | |||
| 3f62cc45bf | |||
| b6f352b318 | |||
| dadf01b56f | |||
| 1239798d04 | |||
| 9ccc896585 | |||
| 1a69f40a80 | |||
| 300607f692 | |||
| 42b5a4e68d | |||
| bccd275d66 | |||
| edfa5f0da3 | |||
| 16b02bcc58 | |||
| 5af2ff1d6c | |||
| 9c25bc7009 | |||
| 8c88a3cbf3 | |||
| 9c4f4a3d30 | |||
| 29c135b656 | |||
| 7edf53783f | |||
| af7b94152f | |||
| 6b26aedd56 | |||
| 68b18cdbcd | |||
| cddc84776c | |||
| 4f5fcd9acb | |||
| 40b2f75098 | |||
| feab24f76c | |||
| 8ef7e903b8 | |||
| 7c5bafd49b | |||
| f66996a148 | |||
| 4fac911425 | |||
| b52da639a3 | |||
| b1dca04ddd | |||
| f481585ba1 | |||
| 723f2b9147 | |||
| 2babed84e9 | |||
| a6ca53c7cf | |||
| c3912d8e5c | |||
| 67599ea67c | |||
| f321a6096d | |||
| 59ebf551f0 | |||
| 04a57029b3 |
@@ -110,8 +110,12 @@ jobs:
|
||||
- name: Run unit tests under Bun
|
||||
run: bun vitest run --project unit --reporter=verbose
|
||||
|
||||
# Adapter tests are pure unit tests — OS doesn't affect results.
|
||||
# Adapter tests are pure unit tests — OS doesn't affect results. Gated off
|
||||
# `pull_request` to keep PR CI under ~2 minutes; adapter authors run focused
|
||||
# tests locally before pushing, and `push` to main / nightly cron / manual
|
||||
# dispatch still guard the merged state.
|
||||
adapter-test:
|
||||
if: github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
||||
runs-on: ubuntu-latest
|
||||
needs: build
|
||||
steps:
|
||||
|
||||
+149
-15
@@ -1,27 +1,161 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
## [1.8.0](https://github.com/jackwener/opencli/compare/v1.7.22...v1.8.0) (2026-05-20)
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed.
|
||||
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed.
|
||||
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results.
|
||||
|
||||
### Features
|
||||
|
||||
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic.
|
||||
Substantial release: a new official-API adapter (`weread-official`), wider LinkedIn / Twitter / Reddit / Zhihu coverage, the 12306 / Suno / Xianyu inbox additions, security and reliability fixes for the Browser Bridge and media downloads, plus a 20% README shrink. Node 20 compatibility is restored after an automated `undici` bump regression.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface.
|
||||
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell.
|
||||
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail.
|
||||
* **skills** — remove the `smart-search` skill. Use `opencli-usage` for command/site reference, `opencli-browser` for ad-hoc browser operation, and `opencli-adapter-author` for writing new adapters.
|
||||
|
||||
### Features
|
||||
|
||||
* **weread-official** — integrate WeRead's official Agent Gateway as the `weread-official` CLI namespace. Pure HTTP, Bearer auth via `WEREAD_API_KEY` (no browser, no cookies). 8 commands cover the official skill bundle: `search`, `shelf`, `book` (info + chapters + progress 3-in-1), `notes` (notebook overview or per-book highlights/thoughts), `review`, `readdata` (weekly/monthly/annually/overall), `discover` (recommend or similar-book), `list-apis`. Adapter surfaces typed errors for all documented failure modes — `AuthRequiredError` on missing/rejected key (errcodes -2010/-2012), `CommandExecutionError` on HTTP/`upgrade_info`/non-zero errcode, `EmptyResultError` on empty payloads. Coexists with the existing cookie-based `weread` adapter.
|
||||
* **12306** — add full read adapter (`stations` / `trains` / `train` / `price` / `me` / `passengers` / `orders`). ([#1637](https://github.com/jackwener/opencli/issues/1637))
|
||||
* **xianyu** — add `inbox`, `messages`, and `reply` commands. ([#1639](https://github.com/jackwener/opencli/issues/1639))
|
||||
* **suno** — add Suno.com music-generation adapter. ([#1638](https://github.com/jackwener/opencli/issues/1638))
|
||||
* **linkedin** — consolidate messaging and Sales Navigator commands (`connect`, `inbox`, `safe-send`, `salesnav-search`, `salesnav-inbox`, `salesnav-message`, `salesnav-thread`, `sent-invitations`, `thread-snapshot`, `timeline`). ([#1647](https://github.com/jackwener/opencli/issues/1647))
|
||||
* **linkedin/people-search** — add a dedicated people-search command. ([#1649](https://github.com/jackwener/opencli/issues/1649))
|
||||
* **linkedin-learning** — add `search` / `trending` / `course` read commands. ([#1657](https://github.com/jackwener/opencli/issues/1657))
|
||||
* **twitter** — rewrite the download-profile path on GraphQL UserMedia with cursor pagination. ([#1636](https://github.com/jackwener/opencli/issues/1636))
|
||||
* **twitter** — add `list-create` (GraphQL CreateList mutation). ([#1656](https://github.com/jackwener/opencli/issues/1656))
|
||||
* **twitter** — add `device-follow` notification-stream command.
|
||||
* **twitter** — expose `card.binding_values` on read commands for inline link-preview metadata. ([#1660](https://github.com/jackwener/opencli/issues/1660))
|
||||
* **twitter** — expose `quoted_tweet` on read commands. ([#1667](https://github.com/jackwener/opencli/issues/1667))
|
||||
* **twitter** — expose `bio` on read commands.
|
||||
* **reddit/subscribed** — new `subscribed` command + listing-level `id` / `created_utc` / `selftext` exposure. ([#1651](https://github.com/jackwener/opencli/issues/1651))
|
||||
* **reddit** — expose `post_hint` / `url` / `preview` / `gallery` media routes on listing commands. ([#1676](https://github.com/jackwener/opencli/issues/1676))
|
||||
* **zhihu** — add answer-comments reader; include answer links in question results.
|
||||
* **chatgpt** — detect generated image surfaces (CSS background and canvas, not just `<img>`) so image generation works after UI drift. ([#1677](https://github.com/jackwener/opencli/issues/1677))
|
||||
* **external** — add Cloudflare Wrangler as a built-in external CLI passthrough. ([#1679](https://github.com/jackwener/opencli/pull/1679))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **deps** — restore Node 20 runtime compatibility by pinning runtime `undici` back to the 6.x line (an automated dependabot bump to 8.x had moved the engines floor to Node ≥22.19, silently breaking the published Node 20 promise), and clear the docs build audit chain by overriding VitePress' Vite/PostCSS transitive dependencies to patched versions. ([#1673](https://github.com/jackwener/opencli/issues/1673))
|
||||
* **download** — keep custom media filenames inside the requested output directory by stripping POSIX/Windows path components and sanitizing the generated fallback prefix. Prevents remote-controlled fields (e.g. video titles used as filename) from escaping the output directory via `../`. ([#1642](https://github.com/jackwener/opencli/pull/1642))
|
||||
* **browser** — recover `Page.goto()` from stale page identities by clearing the cached targetId and retrying navigation once through the session lease; classify CDP `-32000 Cannot find default execution context` as retryable target navigation. ([#1645](https://github.com/jackwener/opencli/issues/1645))
|
||||
* **cli** — escape leading-dash positional values via the argv preprocessor so users can pass tokens starting with `-` without commander mis-classifying them as flags. ([#1658](https://github.com/jackwener/opencli/issues/1658))
|
||||
* **chatgpt/image** — fix ChatGPT web image generation after UI drift by letting the composer locator continue into the caller's readiness check and detecting generated images rendered as CSS backgrounds or canvases, not just plain `<img>` elements.
|
||||
* **adapters** — surface the remaining `silent-empty-fallback` adapter failures as typed errors (Douyin user video comments, Jike SSR JSON parse, WeRead search-page fetch). True empty Douyin/Jike/WeRead result sets now throw `EmptyResultError`.
|
||||
* **adapters** — drop silent-sentinel row fallbacks across Apple Podcasts / Reddit / Gitee. ([#1634](https://github.com/jackwener/opencli/issues/1634))
|
||||
* **adapters** — migrate legal empty-data branches to `EmptyResultError` for `xhs` / YouTube and 5 follow-up commands. ([#1674](https://github.com/jackwener/opencli/issues/1674), [#1678](https://github.com/jackwener/opencli/issues/1678))
|
||||
* **lesswrong** — drop the `"Unknown"` silent sentinel in the author column; missing authors now propagate as `null`. ([#1611](https://github.com/jackwener/opencli/issues/1611))
|
||||
* **youtube/transcript** — scope timedtext URL matching to the current `videoId` across the in-page resource-buffer scan, the in-page fetch/XHR hook, and the Node-side CDP capture. SPA-style watch→watch navigation no longer returns a predecessor video's captions. ([#1655](https://github.com/jackwener/opencli/issues/1655))
|
||||
* **twitter/lists** — skip the "Discover new Lists" recommendation block so it is no longer treated as one of the user's lists. ([#1652](https://github.com/jackwener/opencli/issues/1652))
|
||||
* **zhihu** — harden search pagination. ([#1615](https://github.com/jackwener/opencli/issues/1615))
|
||||
* **zhihu** — decode numeric HTML entities in `answer-detail`. ([#1629](https://github.com/jackwener/opencli/issues/1629))
|
||||
|
||||
### Docs
|
||||
|
||||
* **readme** — major shrink and reframing: tagline rephrased around "Browser Use", Highlights and Update sections folded into adjacent content, Built-in Commands curated to 11 popular sites, CLI Hub table reduced to a name enumeration, Desktop App Adapters collapsed to a one-liner, skill-attribution references audited against `SKILL.md` frontmatter, "For AI Agents (Developer Guide)" merged into "Writing a new adapter". Net: EN 410 → 326 (-20%), ZH 455 → 371 (-18%). ([#1654](https://github.com/jackwener/opencli/pull/1654), [#1666](https://github.com/jackwener/opencli/pull/1666), [#1679](https://github.com/jackwener/opencli/pull/1679), [#1681](https://github.com/jackwener/opencli/pull/1681))
|
||||
|
||||
### Internal
|
||||
|
||||
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`.
|
||||
* **extension 1.0.13** — remove the internal command-session lease-key backdoor.
|
||||
* **audit** — stop flagging sentinel fallback strings inside thrown error messages as `silent-sentinel` violations. These are typed failure diagnostics rather than fake row data, reducing the typed-error baseline to actual adapter output fallbacks.
|
||||
|
||||
## [1.7.22](https://github.com/jackwener/opencli/compare/v1.7.21...v1.7.22) (2026-05-15)
|
||||
|
||||
External CLI ergonomics + two adapter envelope/auth fixes. New `longbridge` external CLI entry; `opencli list` / root help now render human-readable brand labels for executables whose bare name is ambiguous.
|
||||
|
||||
### Features
|
||||
|
||||
* **external** — add the Longbridge CLI as a built-in external CLI passthrough (`opencli longbridge ...`) for Longbridge OpenAPI market data, account, and trading commands. ([#1584](https://github.com/jackwener/opencli/issues/1584))
|
||||
* **external-cli** — render brand alias `name(package)` in `opencli list` and root help when the bare executable name is ambiguous. Built-in entries `ntn` → `ntn(notion)`, `dws` → `dws(DingTalk Workspace)`, `wecom-cli` → `wecom-cli(企业微信)` now self-explain in help output. `package` field is repurposed to cover both upstream distribution names (e.g. `tg-cli`) and human-readable brand labels (e.g. `notion`, `企业微信`). ([#1585](https://github.com/jackwener/opencli/issues/1585))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **boss** — map `code=24` (identity mismatch) to `AuthRequiredError` so re-login is signaled instead of surfacing as a generic API error. ([#1573](https://github.com/jackwener/opencli/issues/1573))
|
||||
* **weibo** — unwrap Browser Bridge `page.evaluate` envelopes in read adapters. ([#1568](https://github.com/jackwener/opencli/issues/1568))
|
||||
|
||||
## [1.7.21](https://github.com/jackwener/opencli/compare/v1.7.20...v1.7.21) (2026-05-14)
|
||||
|
||||
Adapter polish release: new web search adapters, better Browser Bridge tab group reuse, and social adapters returning to one-shot tab leases. Extension package version is bumped to 1.0.15 for the Browser Bridge fix.
|
||||
|
||||
### Features
|
||||
|
||||
* **search** — add DuckDuckGo, Brave, and Yahoo web search adapters. ([#1546](https://github.com/jackwener/opencli/issues/1546))
|
||||
* **boss** — support job-seeker `chatlist` and `chatmsg` adapters. ([#1539](https://github.com/jackwener/opencli/issues/1539))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **extension** — reuse existing `OpenCLI Adapter` tab groups before creating new ones, including cross-window discovery, legacy `OpenCLI` title fallback, and deterministic candidate selection. ([#1541](https://github.com/jackwener/opencli/issues/1541))
|
||||
* **twitter, reddit** — default browser-backed social adapters back to ephemeral tab leases. Twitter/X and Reddit commands now release their site tab after each run while keeping the shared Adapter window available for reuse; persistent sessions remain reserved for AI/chat-style adapters that need long-lived conversation state. ([#1569](https://github.com/jackwener/opencli/issues/1569))
|
||||
* **xiaohongshu, rednote** — unwrap Browser Bridge `page.evaluate` envelopes in search adapters. ([#1561](https://github.com/jackwener/opencli/issues/1561))
|
||||
* **facebook/feed** — add fallback extraction for empty article nodes. ([#1538](https://github.com/jackwener/opencli/issues/1538))
|
||||
|
||||
### Internal
|
||||
|
||||
* **ci** — add Windows native binding lockfile entries for Rolldown/Rollup optional packages. ([#1563](https://github.com/jackwener/opencli/issues/1563))
|
||||
* **extension** — add regression coverage for the adapter tab group `groupId` tiebreaker. ([#1566](https://github.com/jackwener/opencli/issues/1566))
|
||||
|
||||
## [1.7.20](https://github.com/jackwener/opencli/compare/v1.7.19...v1.7.20) (2026-05-14)
|
||||
|
||||
External CLI surface cleanup + Browser Bridge WebSocket lifecycle hardening. Two BREAKING changes around external CLIs: built-in `tg`/`discord`/`wx` (was `tg-cli`/`discord-cli`/`wx-cli`) now match their real binary names, and Notion's in-tree CDP adapter is replaced by the official `ntn` external CLI.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **notion** — remove the in-tree `clis/notion/` CDP-on-Desktop adapter (8 commands: `status` / `search` / `read` / `new` / `write` / `sidebar` / `favorites` / `export`). Notion has shipped an official CLI at <https://ntn.dev>, registered as a first-class external CLI in `external-clis.yaml`. Migration: install `ntn` from <https://ntn.dev> (`curl -fsSL https://ntn.dev | bash`), then use `opencli ntn <command>`. Auto-install is intentionally not configured because the official installer is a shell script while OpenCLI external installs run shell-free command strings. The official CLI uses the public Notion API rather than reverse-engineering the Desktop UI, so it survives Notion app updates and exposes a wider command surface (blocks / databases / properties / comments) than the reverse-engineered adapter could. ([#1559](https://github.com/jackwener/opencli/issues/1559))
|
||||
* **external** — drop the `-cli` suffix from built-in external CLI subcommand names. `opencli tg-cli`, `opencli discord-cli`, `opencli wx-cli` are now `opencli tg`, `opencli discord`, `opencli wx`, matching the real binary names that those tools install as. Root help still shows the package lineage as `tg(tg-cli)` / `discord(discord-cli)` / `wx(wx-cli)`. ([#1544](https://github.com/jackwener/opencli/issues/1544))
|
||||
|
||||
### Features
|
||||
|
||||
* **twitter** — `bookmarks` and `bookmark-folder` now include media via `extractMedia`, reaching parity with `timeline` / `search`. ([#1555](https://github.com/jackwener/opencli/issues/1555))
|
||||
* **twitter/list-tweets** — include media via `extractMedia` (parity with `timeline` / `search`). ([#1464](https://github.com/jackwener/opencli/issues/1464))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **daemon** — report ambiguous browser command outcomes with a distinct `command_result_unknown` errorCode and `503` when the extension WebSocket drops between command dispatch and result delivery. `sendCommandRaw()` treats this code as hard non-retryable, so write-side commands (`navigate` / `click` / `type` / `eval`) won't be silently re-issued and double-executed. Daemon exposes a `commandResultUnknown` counter on `/status` for future observability. ([#1558](https://github.com/jackwener/opencli/issues/1558))
|
||||
* **extension** — keep active daemon WebSocket; stale sockets no longer clobber active connection (`onopen` / `onclose` / `onmessage` are all gated by `ws !== thisWs` short-circuit), and `safeSend` only fires when `readyState === OPEN`. ([#1540](https://github.com/jackwener/opencli/issues/1540))
|
||||
* **extension** — coalesce concurrent daemon WebSocket connects via an in-flight promise. Startup / keepalive / reconnect triggering `connect()` during the daemon-probe or context-lookup async gap no longer creates duplicate real WebSocket connections. ([#1554](https://github.com/jackwener/opencli/issues/1554))
|
||||
* **external** — distinguish external CLI executable names from distribution/project names in root help. Built-in aliases such as `tg`, `discord`, `wx` remain the callable `opencli <name> ...` entrypoints while help renders `tg(tg-cli)`, `discord(discord-cli)`, `wx(wx-cli)` to show their package lineage. ([#1560](https://github.com/jackwener/opencli/issues/1560))
|
||||
|
||||
### Docs
|
||||
|
||||
* **browser** — clarify named session lifecycle in the Browser Bridge guide. ([#1542](https://github.com/jackwener/opencli/issues/1542))
|
||||
|
||||
## [1.7.19](https://github.com/jackwener/opencli/compare/v1.7.18...v1.7.19) (2026-05-14)
|
||||
|
||||
Major hotfix + simplification batch. Extension bumped to 1.0.14. Node floor lowered to v20 so the long tail of Node v20–v21.6 users no longer crashes at module load. `opencli browser` user surface replaces required-flag `--session <name>` with a `<session>` positional. `page.evaluate(fn, ...args)` adds a type-safe alternative to the implicit auto-IIFE string form. Twitter cursor pagination no longer silently caps at ~500 items.
|
||||
|
||||
### ⚠ BREAKING CHANGES
|
||||
|
||||
* **browser** — replace the `--session <name>` flag with a `<session>` positional argument that immediately follows `browser`. `opencli browser work click 12` instead of `opencli browser --session work click 12`; `opencli browser work bind` instead of `opencli browser bind --session work`. Required-flag semantics are now encoded structurally as a positional, matching the Docker/git convention for required operation-target identifiers. The internal `--session` flag is preserved for the daemon protocol and for direct `program.parseAsync` callers but is no longer part of the user-facing surface. ([#1505](https://github.com/jackwener/opencli/issues/1505))
|
||||
* **env** — remove `OPENCLI_KEEP_TAB`. The flag was a debugging shortcut, not a config dimension: `--keep-tab true|false` on the command line is the single source of truth, and adapter `siteSession: 'persistent'` already pins persistent site tabs as a hard constraint. Removing the env eliminates a globally-leaking process state that overrode every browser command in the shell. ([#1509](https://github.com/jackwener/opencli/issues/1509))
|
||||
* **extension** — remove the internal `surface\\0session` command-session backdoor. Browser Bridge commands now route only through structured `session` + `surface` fields; lease-key strings remain an extension-internal registry detail. ([#1510](https://github.com/jackwener/opencli/issues/1510))
|
||||
|
||||
### Features
|
||||
|
||||
* **browser** — add `page.evaluate(fn, ...args)` for type-safe browser-context evaluation with JSON-serialized arguments. String evaluation remains supported, but new adapter code should use function form to avoid implicit `wrapForEval` auto-IIFE magic. ([#1508](https://github.com/jackwener/opencli/issues/1508))
|
||||
* **twitter** — default `tweets` command to the logged-in user when `user` is omitted, and fix the sibling envelope-unwrap silent bug. ([#1531](https://github.com/jackwener/opencli/issues/1531))
|
||||
* **zhihu** — add `answer-detail` to fetch a single answer's full content. ([#1528](https://github.com/jackwener/opencli/issues/1528))
|
||||
* **zhihu** — paginate question answers and recommendations. ([#1517](https://github.com/jackwener/opencli/issues/1517))
|
||||
* **reddit/read** — `--expand-more` via `/api/morechildren` + 7-kind typed errors. ([#1492](https://github.com/jackwener/opencli/issues/1492))
|
||||
* **reddit** — add `whoami`, `home`, `subreddit-info` read commands. ([#1491](https://github.com/jackwener/opencli/issues/1491))
|
||||
* **ctrip** — add `hotel-search` + flight browser-mode commands. ([#1489](https://github.com/jackwener/opencli/issues/1489))
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **browser** — `page.evaluate()` / `evaluateInFrame()` now return the user JavaScript value directly. Browser Bridge `exec` previously routed through a shared `pageScopedResult` helper that spread / wrapped the lease's `session` into the result `data`, contaminating arbitrary user returns: array / primitive returns came back as `{ session, data }` envelopes, and plain-object returns had an extra `session` key injected (overwriting any user `session` field). `google search` and `xiaohongshu search` were the visible repro — Chrome rendered results correctly but adapters extracted an empty array. Fixed in extension 1.0.14 by reverting `pageScopedResult` to its pre-1461 form (`{ id, ok, data, page }`); no client-side unwrap is needed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
|
||||
* **twitter** — raise fixed cursor-pagination caps in `bookmarks` / `likes` / `tweets` / `timeline` / `bookmark-folder` / `list-tweets` / `search` / `following`. The old `i < 5` / `i < 10` literals and following's `Math.ceil(limit / 50) + 2` formula imposed hidden result ceilings below `--limit`; the loop now treats the page count as a high runaway guard while `--limit` and cursor exhaustion control normal pagination. ([#1532](https://github.com/jackwener/opencli/issues/1532))
|
||||
* **twitter** — repair `list-add` / `list-tweets` / `lists` / `following` after 2026-05 site changes. ([#1503](https://github.com/jackwener/opencli/issues/1503))
|
||||
* **twitter** — repair `search` and `tweets` readback. ([#1512](https://github.com/jackwener/opencli/issues/1512))
|
||||
* **twitter** — make reply submission robust. ([#1511](https://github.com/jackwener/opencli/issues/1511))
|
||||
* **google/search** — wait for `#rso a h3` before extracting, falling back to the existing fixed wait. On Chrome 148 + Linux Wayland the DOM can settle before SERP anchors are populated, making extraction return empty even with the envelope bug fixed. ([#1518](https://github.com/jackwener/opencli/issues/1518))
|
||||
* **google/search** — wrap evaluate return value in object to fix serialization. ([#1523](https://github.com/jackwener/opencli/issues/1523))
|
||||
* **google-scholar/search** — wrap evaluate return to fix serialization. ([#1525](https://github.com/jackwener/opencli/issues/1525))
|
||||
* **xiaohongshu/search** — extract initially visible cards before scrolling, then merge post-scroll rows by URL. Xiaohongshu's virtualized masonry layout can evict the initial cards from the DOM after scroll, so the previous always-scroll-then-extract flow could lose the top results. ([#1518](https://github.com/jackwener/opencli/issues/1518))
|
||||
* **xiaohongshu** — `parseLikes` handles `2.1w` / `1.5万` / `1.2k` shortforms. ([#1504](https://github.com/jackwener/opencli/issues/1504))
|
||||
* **xiaohongshu+rednote/search** — fall back to href-based note cards when `section.note-item` class is dropped. ([#1507](https://github.com/jackwener/opencli/issues/1507))
|
||||
* **xueqiu** — `kline` / `earnings-date` format dates in Asia/Shanghai instead of UTC. ([#1498](https://github.com/jackwener/opencli/issues/1498))
|
||||
* **download** — clamp progress percentages. ([#1520](https://github.com/jackwener/opencli/issues/1520))
|
||||
|
||||
### Internal
|
||||
|
||||
* **runtime** — lower the Node floor to `>=20.0.0`. Three coupled changes: drop all `util.styleText()` usage (added in Node v21.7.0 / v20.12.0; previously crashed v21.0–v21.6 at module load), downgrade `undici` from `^8.0.2` (engines `>=22.19.0`) to `^6.25.0` (engines `>=18.17`, retains `Agent` / `EnvHttpProxyAgent` / `fetch`), and lower `MIN_SUPPORTED_NODE_MAJOR` from 21 to 20 so the startup guard matches the declared `engines.node`. Smoke-tested on v20.0.0 / v21.2.0 / v22.22.2. The semantic markers (`[OK]` / `[WARN]` / `[FAIL]` / `ℹ` / `⚠` / `✖`) keep their meaning; ANSI colors were redundant for the primarily agent-facing CLI. ([#1524](https://github.com/jackwener/opencli/issues/1524))
|
||||
* **extension 1.0.14** — `pageScopedResult` no longer injects `session` into `data`. The field had no consumers and contaminated `exec` results with arbitrary user-JS shapes; routing-relevant identity is already exposed via `Result.page`. ([#1518](https://github.com/jackwener/opencli/issues/1518))
|
||||
* **extension 1.0.13** — remove the internal command-session lease-key backdoor. ([#1510](https://github.com/jackwener/opencli/issues/1510))
|
||||
* **ci** — drop `e2e-headed` and `adapter-test` from `pull_request` triggers (kept on `push` to main / nightly / `workflow_dispatch`). PR-time CI now targets ~2 min wall-time. ([#1521](https://github.com/jackwener/opencli/issues/1521), [#1522](https://github.com/jackwener/opencli/issues/1522))
|
||||
* **scripts** — auto-refresh `dist/` before `build-manifest`. ([#1490](https://github.com/jackwener/opencli/issues/1490))
|
||||
|
||||
## [1.7.18](https://github.com/jackwener/opencli/compare/v1.7.17...v1.7.18) (2026-05-12)
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
# OpenCLI
|
||||
|
||||
> **Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.**
|
||||
> Reuse your logged-in browser, automate live workflows, and crystallize repeated actions into reusable CLI commands.
|
||||
> **Convert any website into a CLI & run Browser Use on your logged-in Chrome.**
|
||||
> Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.
|
||||
> Or run Browser Use against any page — navigate, fill forms, click, extract, automate.
|
||||
|
||||
[](./README.zh-CN.md)
|
||||
[](https://www.npmjs.com/package/@jackwener/opencli)
|
||||
@@ -11,24 +12,10 @@
|
||||
OpenCLI gives you one surface for three different kinds of automation:
|
||||
|
||||
- **Use built-in adapters** for sites like Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, Twitter/X, and [many more](#built-in-commands).
|
||||
- **Let AI Agents operate any website** — install the `opencli-adapter-author` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
|
||||
- **Let AI Agents operate any website** — install the `opencli-browser` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type/fill, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
|
||||
- **Write new adapters** end-to-end with `opencli browser` + the `opencli-adapter-author` skill, which guides from first recon through field decoding, code, and `opencli browser verify`.
|
||||
|
||||
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `tg-cli`, `discord-cli`, `wx-cli`, and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, ChatGPT, and Notion.
|
||||
|
||||
## Highlights
|
||||
|
||||
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
|
||||
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type/fill, extract, screenshot — all through your logged-in Chrome session.
|
||||
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
|
||||
- **Website → CLI** — Turn any website into a deterministic CLI: 100+ site surfaces are already registered, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
|
||||
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
|
||||
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
|
||||
- **CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, docker, obsidian, tg-cli, discord-cli, wx-cli, etc).
|
||||
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
|
||||
- **Deterministic** — Same command, same output schema, every time. Pipeable, scriptable, CI-friendly.
|
||||
|
||||
---
|
||||
It also works as a **CLI hub** for local tools such as `gh`, `docker`, `longbridge`, `tg`, `discord`, `wx`, `ntn` (Notion), and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, and ChatGPT.
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -105,7 +92,7 @@ If you want to add your own commands, start with the [Extending OpenCLI guide](.
|
||||
|
||||
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
|
||||
|
||||
### Install skills
|
||||
### Install skills (also refreshes existing installs)
|
||||
|
||||
```bash
|
||||
npx skills add jackwener/opencli
|
||||
@@ -118,22 +105,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
|
||||
npx skills add jackwener/opencli --skill opencli-autofix
|
||||
npx skills add jackwener/opencli --skill opencli-browser
|
||||
npx skills add jackwener/opencli --skill opencli-usage
|
||||
npx skills add jackwener/opencli --skill smart-search
|
||||
```
|
||||
|
||||
### Which skill to use
|
||||
|
||||
| Skill | When to use | Example prompt to your AI agent |
|
||||
|-------|------------|-------------------------------|
|
||||
| **opencli-adapter-author** | Operate a site in real time, or write a reusable adapter for a new site | "Help me check my Xiaohongshu notifications" / "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
|
||||
| **opencli-adapter-author** | Write a reusable adapter for a new site or add a command to an existing site | "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
|
||||
| **opencli-autofix** | Repair a broken adapter when a built-in command fails | "`opencli zhihu hot` is returning empty — fix it" |
|
||||
| **opencli-browser** | Browser automation reference for AI agents | "Use browser commands to scrape this page" |
|
||||
| **opencli-browser** | Drive a real Chrome page ad-hoc — navigate, fill forms, click, extract | "Help me check my Xiaohongshu notifications" / "Help me fill out this form" / "Use browser commands to scrape this page" |
|
||||
| **opencli-usage** | Quick reference for all OpenCLI commands and sites | "What commands does OpenCLI have for Twitter?" |
|
||||
| **smart-search** | Search across existing OpenCLI capabilities | "Find me a Bilibili trending adapter" |
|
||||
|
||||
### How it works
|
||||
|
||||
Once `opencli-adapter-author` is installed, your AI agent can:
|
||||
Once `opencli-browser` is installed, your AI agent can:
|
||||
|
||||
1. **Navigate** to any URL using your logged-in browser
|
||||
2. **Read** page content via structured DOM snapshots (not screenshots)
|
||||
@@ -144,45 +129,25 @@ Once `opencli-adapter-author` is installed, your AI agent can:
|
||||
The agent handles all the `opencli browser` commands internally — you just describe what you want done in natural language.
|
||||
|
||||
**Skill references:**
|
||||
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — browser operation + adapter authoring, end-to-end
|
||||
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — drive Chrome ad-hoc (navigate, fill forms, click, extract)
|
||||
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — write a new adapter end-to-end
|
||||
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — repair broken adapters
|
||||
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — browser automation reference
|
||||
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — command and site reference
|
||||
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — capability search
|
||||
|
||||
Available browser commands include `open`, `state`, `click`, `type`, `fill`, `select`, `keys`, `wait`, `get`, `find`, `extract`, `frames`, `screenshot`, `scroll`, `back`, `eval`, `network`, `tab list`, `tab new`, `tab select`, `tab close`, `init`, `verify`, and `close`.
|
||||
|
||||
`opencli browser` commands require a `<session>` positional immediately after `browser`. `opencli browser work open <url>` and `opencli browser work tab new [url]` both return a target ID. Use `opencli browser work tab list` to inspect target IDs, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted commands in the same session.
|
||||
|
||||
## Core Concepts
|
||||
## Writing a new adapter
|
||||
|
||||
### `browser`: AI Agent browser control
|
||||
When the site you need is not yet covered, use the `opencli-adapter-author` skill end-to-end:
|
||||
|
||||
`opencli browser` commands are the low-level primitives that AI Agents use to operate websites. You don't run these manually — instead, install the `opencli-adapter-author` skill into your AI agent, describe what you want in natural language, and the agent handles the browser operations.
|
||||
|
||||
For example, tell your agent: *"Help me check my Xiaohongshu notifications"* — the agent will use `opencli browser <session> open`, `state`, `click`, etc. under the hood.
|
||||
|
||||
### Built-in adapters: stable commands
|
||||
|
||||
Use site-specific commands such as `opencli hackernews top` or `opencli reddit hot` when the capability already exists. These are deterministic and work without browser — ideal for both humans and AI agents.
|
||||
|
||||
### Writing a new adapter
|
||||
|
||||
When the site you need is not yet covered, use the `opencli-adapter-author` skill. It takes the agent end-to-end:
|
||||
|
||||
1. Recon the site and classify its pattern (SPA / SSR / JSONP / Token / Streaming).
|
||||
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
|
||||
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`.
|
||||
4. Decode response fields and design output columns.
|
||||
5. `opencli browser recon analyze <url>` for one-shot recon, then `opencli browser recon init <site>/<name>` → write adapter → `opencli browser recon verify <site>/<name>`.
|
||||
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
|
||||
|
||||
### CLI Hub and desktop adapters
|
||||
|
||||
OpenCLI is not only for websites. It can also:
|
||||
|
||||
- expose local binaries like `gh`, `docker`, `obsidian`, `tg-cli`, `discord-cli`, `wx-cli`, or custom tools through `opencli <tool> ...`
|
||||
- control Electron desktop apps through dedicated adapters and CDP-backed integrations
|
||||
1. **Recon** the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
|
||||
2. **Discover** the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
|
||||
3. **Pick auth** — `PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`.
|
||||
4. **Decode** response fields and design output columns.
|
||||
5. `opencli browser recon analyze <url>` → `opencli browser recon init <site>/<name>` → write adapter → `opencli browser recon verify <site>/<name>`.
|
||||
6. Site knowledge persists to `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
@@ -208,25 +173,6 @@ OpenCLI is not only for websites. It can also:
|
||||
|
||||
`opencli browser *` requires an explicit `<session>` positional, uses a foreground browser window by default, and keeps that session's tab lease until `opencli browser <session> close` or idle cleanup. Browser-backed adapters use a background adapter window and release one-shot tab leases by default. Interactive adapters can declare `siteSession: 'persistent'` to keep a stable site tab for continuity; pass `--site-session ephemeral` for a one-shot tab.
|
||||
|
||||
## Update
|
||||
|
||||
```bash
|
||||
npm install -g @jackwener/opencli@latest
|
||||
|
||||
# If you use the packaged OpenCLI skills, refresh them too
|
||||
npx skills add jackwener/opencli
|
||||
```
|
||||
|
||||
Or refresh only the skills you actually use:
|
||||
|
||||
```bash
|
||||
npx skills add jackwener/opencli --skill opencli-adapter-author
|
||||
npx skills add jackwener/opencli --skill opencli-autofix
|
||||
npx skills add jackwener/opencli --skill opencli-browser
|
||||
npx skills add jackwener/opencli --skill opencli-usage
|
||||
npx skills add jackwener/opencli --skill smart-search
|
||||
```
|
||||
|
||||
## For Developers
|
||||
|
||||
Install from source:
|
||||
@@ -249,77 +195,28 @@ To load the source Browser Bridge extension:
|
||||
| Site | Commands |
|
||||
|------|----------|
|
||||
| **xiaohongshu** | `search` `note` `comments` `feed` `user` `download` `publish` `notifications` `creator-notes` `creator-notes-summary` `creator-note-detail` `creator-profile` `creator-stats` |
|
||||
| **rednote** | `search` `note` `comments` `user` `download` `feed` `notifications` |
|
||||
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `video` `user-videos` |
|
||||
| **tieba** | `hot` `posts` `search` `read` |
|
||||
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` |
|
||||
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
|
||||
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
|
||||
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `summary` `video` `user-videos` |
|
||||
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
|
||||
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
|
||||
| **1688** | `search` `item` `assets` `download` `store` |
|
||||
| **gitee** | `trending` `search` `user` |
|
||||
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
|
||||
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
|
||||
| **yuanbao** | `new` `ask` |
|
||||
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
|
||||
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` |
|
||||
| **xianyu** | `search` `item` `chat` `publish` |
|
||||
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` |
|
||||
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
|
||||
| **uiverse** | `code` `preview` |
|
||||
| **baidu-scholar** | `search` |
|
||||
| **google-scholar** | `search` `cite` `profile` |
|
||||
| **gov-law** | `search` `recent` |
|
||||
| **gov-policy** | `search` `recent` |
|
||||
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
|
||||
| **wanfang** | `search` |
|
||||
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
|
||||
| **xiaoyuzhou** | `auth*` `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` |
|
||||
| **linkedin** | `connect` `inbox` `safe-send` `search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
|
||||
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
|
||||
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
|
||||
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
|
||||
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
|
||||
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
|
||||
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
|
||||
|
||||
100+ site surfaces in total — **[→ see all supported sites & commands](./docs/adapters/index.md)**
|
||||
|
||||
`*` `opencli xiaoyuzhou podcast`, `podcast-episodes`, `episode`, `download`, and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
|
||||
Curated highlights — **[→ see all 100+ supported sites & commands](./docs/adapters/index.md)** (douyin / weibo / spotify / 1688 / quark / nowcoder / google-scholar / hupu / xianyu / weread / weread-official / xiaoyuzhou / and more).
|
||||
|
||||
## CLI Hub
|
||||
|
||||
OpenCLI acts as a universal hub for your existing command-line tools — unified discovery, pure passthrough execution, and auto-install (if a tool isn't installed, OpenCLI runs `brew install <tool>` automatically before re-running the command).
|
||||
Unified passthrough for your existing command-line tools. Run `opencli <tool> ...` for any of:
|
||||
|
||||
| External CLI | Description | Example |
|
||||
|--------------|-------------|---------|
|
||||
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
|
||||
| **obsidian** | Obsidian vault management | `opencli obsidian search query="AI"` |
|
||||
| **docker** | Docker | `opencli docker ps` |
|
||||
| **lark-cli** | Lark/Feishu — messages, docs, calendar, tasks, 200+ commands | `opencli lark-cli calendar +agenda` |
|
||||
| **dws** | DingTalk — cross-platform CLI for DingTalk's full suite, designed for humans and AI agents | `opencli dws msg send --to user "hello"` |
|
||||
| **wecom-cli** | WeCom/企业微信 — CLI for WeCom open platform, for humans and AI agents | `opencli wecom-cli msg send --to user "hello"` |
|
||||
| **tg-cli** | Telegram — local-first sync, search, and export via MTProto for AI agents | `opencli tg search "AI news" -f json` |
|
||||
| **discord-cli** | Discord — local-first sync, search, and export via SQLite for AI agents | `opencli discord recent --channel general` |
|
||||
| **wx-cli** | WeChat — query local WeChat data: sessions, messages, search, contacts, export | `opencli wx search "OpenCLI"` |
|
||||
| **vercel** | Vercel — deploy projects, manage domains, env vars, logs | `opencli vercel deploy --prod` |
|
||||
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
|
||||
|
||||
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
|
||||
Register your own with `opencli external register <name>`; list everything with `opencli external list`.
|
||||
|
||||
```bash
|
||||
opencli external register mycli
|
||||
```
|
||||
|
||||
### Desktop App Adapters
|
||||
|
||||
Control Electron desktop apps directly from the terminal. Each adapter has its own detailed documentation:
|
||||
|
||||
| App | Description | Doc |
|
||||
|-----|-------------|-----|
|
||||
| **Cursor** | Control Cursor IDE — Composer, chat, code extraction | [Doc](./docs/adapters/desktop/cursor.md) |
|
||||
| **Codex** | Drive OpenAI Codex CLI agent headlessly | [Doc](./docs/adapters/desktop/codex.md) |
|
||||
| **Antigravity** | Control Antigravity Ultra from terminal | [Doc](./docs/adapters/desktop/antigravity.md) |
|
||||
| **ChatGPT App** | Automate ChatGPT macOS desktop app | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
|
||||
| **ChatWise** | Multi-LLM client (GPT-4, Claude, Gemini) | [Doc](./docs/adapters/desktop/chatwise.md) |
|
||||
| **Notion** | Search, read, write Notion pages | [Doc](./docs/adapters/desktop/notion.md) |
|
||||
| **Discord** | Discord Desktop — messages, channels, servers | [Doc](./docs/adapters/desktop/discord.md) |
|
||||
| **Doubao** | Control Doubao AI desktop app via CDP | [Doc](./docs/adapters/desktop/doubao-app.md) |
|
||||
|
||||
To add a new Electron app, start with [docs/guide/electron-app-cli.md](./docs/guide/electron-app-cli.md).
|
||||
**Desktop app adapters** (Electron, via CDP): Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — see [`docs/adapters/desktop/`](./docs/adapters/desktop/).
|
||||
|
||||
## Download Support
|
||||
|
||||
@@ -405,18 +302,6 @@ opencli plugin uninstall my-tool
|
||||
|
||||
See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
|
||||
|
||||
## For AI Agents (Developer Guide)
|
||||
|
||||
Before writing any adapter code, read the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md). It takes you end-to-end:
|
||||
|
||||
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
|
||||
- Discover the right endpoint via `opencli browser <session> network`, `eval`, or the interceptor fallback.
|
||||
- Decide auth strategy (`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`).
|
||||
- Run `opencli browser recon analyze <url>` for one-shot recon, decode response fields, design columns, scaffold with `opencli browser recon init`.
|
||||
- Verify with `opencli browser recon verify <site>/<name>` before shipping.
|
||||
|
||||
For long-lived personal commands that should live in your own Git repo, use a local plugin instead; see [Extending OpenCLI](./docs/guide/extending-opencli.md). Quick private adapters can still live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
|
||||
|
||||
## Testing
|
||||
|
||||
See **[TESTING.md](./TESTING.md)** for how to run and write tests.
|
||||
|
||||
+37
-211
@@ -1,7 +1,8 @@
|
||||
# OpenCLI
|
||||
|
||||
> **把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口。**
|
||||
> 复用浏览器登录态,先自动化真实操作,再把高频流程沉淀成可复用的 CLI 命令。
|
||||
> **把任意网站变成 CLI & 在你的登录态浏览器上跑 Browser Use。**
|
||||
> 把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口。
|
||||
> 或者在任意页面上跑 Browser Use —— 导航、填表单、点击、抓取、自动化。
|
||||
|
||||
[](./README.md)
|
||||
[](https://www.npmjs.com/package/@jackwener/opencli)
|
||||
@@ -11,21 +12,10 @@
|
||||
OpenCLI 可以用同一套 CLI 做三类事情:
|
||||
|
||||
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [100+ 站点](#内置命令) 开箱即用。
|
||||
- **让 AI Agent 操作任意网站**:在你的 AI Agent(Claude Code、Cursor 等)中安装 `opencli-adapter-author` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
|
||||
- **让 AI Agent 操作任意网站**:在你的 AI Agent(Claude Code、Cursor 等)中安装 `opencli-browser` skill,Agent 就能用你的已登录浏览器导航、点击、输入/填充、提取任意网页内容。
|
||||
- **把新网站写成 CLI**:用 `opencli browser` 原语 + `opencli-adapter-author` skill,从站点侦察、API 发现、字段解码到 `opencli browser verify` 一条龙。
|
||||
|
||||
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh`、`docker`、`tg-cli`、`discord-cli`、`wx-cli` 等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT、Notion 等 Electron 应用。
|
||||
|
||||
## 亮点
|
||||
|
||||
- **桌面应用控制** — 通过 CDP 直接在终端驱动 Electron 应用(Cursor、Codex、ChatGPT、Notion 等)。
|
||||
- **AI Agent 浏览器自动化** — 安装 `opencli-adapter-author` skill,你的 AI Agent 就能操作任意网站:导航、点击、输入/填充、提取、截图——全部通过你的已登录 Chrome 会话完成。
|
||||
- **网站 → CLI** — 把任何网站变成确定性 CLI:100+ 站点能力已注册,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
|
||||
- **账号安全** — 复用 Chrome/Chromium 登录态,凭证永远不会离开浏览器。
|
||||
- **面向 AI Agent** — 一个 skill 带你走完站点侦察、API 发现、字段解码、适配器编写、验证的全流程。
|
||||
- **CLI 枢纽** — 统一发现、自动安装、纯透传任何外部 CLI(gh、docker、obsidian、tg-cli、discord-cli、wx-cli 等)。
|
||||
- **零 LLM 成本** — 运行时不消耗模型 token,跑 10,000 次也不花一分钱。
|
||||
- **确定性输出** — 相同命令,相同输出结构,每次一致。可管道、可脚本、CI 友好。
|
||||
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh`、`docker`、`longbridge`、`tg`、`discord`、`wx`、`ntn`(Notion)等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT 等 Electron 应用。
|
||||
|
||||
## 快速开始
|
||||
|
||||
@@ -89,7 +79,7 @@ opencli bilibili hot --limit 5
|
||||
|
||||
OpenCLI 的 browser 命令是给 AI Agent 用的——不是手动执行的。把 skill 安装到你的 AI Agent(Claude Code、Cursor 等)中,Agent 就能用你的已登录 Chrome 会话替你操作网站。
|
||||
|
||||
### 安装 skill
|
||||
### 安装 skill(同时也用于更新)
|
||||
|
||||
```bash
|
||||
npx skills add jackwener/opencli
|
||||
@@ -102,22 +92,20 @@ npx skills add jackwener/opencli --skill opencli-adapter-author
|
||||
npx skills add jackwener/opencli --skill opencli-autofix
|
||||
npx skills add jackwener/opencli --skill opencli-browser
|
||||
npx skills add jackwener/opencli --skill opencli-usage
|
||||
npx skills add jackwener/opencli --skill smart-search
|
||||
```
|
||||
|
||||
### 选择哪个 skill
|
||||
|
||||
| Skill | 适用场景 | 你对 AI Agent 说的话 |
|
||||
|-------|---------|-------------------|
|
||||
| **opencli-adapter-author** | 实时操作任意网站,或为新站点写可复用适配器 | "帮我看看小红书的通知" / "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
|
||||
| **opencli-adapter-author** | 为新站点写可复用适配器,或给已有站点添加命令 | "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
|
||||
| **opencli-autofix** | 内置命令失败时修复已有适配器 | "`opencli zhihu hot` 返回空了,修一下" |
|
||||
| **opencli-browser** | 浏览器自动化参考文档 | "用浏览器命令抓取这个页面" |
|
||||
| **opencli-browser** | 实时驱动 Chrome 页面——导航、填表单、点击、抓取 | "帮我看看小红书的通知" / "帮我填一下这个表单" / "用浏览器命令抓取这个页面" |
|
||||
| **opencli-usage** | 所有命令和站点的快速参考 | "OpenCLI 有哪些 Twitter 相关的命令?" |
|
||||
| **smart-search** | 在现有 OpenCLI 能力里搜索 | "帮我找个 B 站热门相关的适配器" |
|
||||
|
||||
### 工作原理
|
||||
|
||||
安装 `opencli-adapter-author` skill 后,你的 AI Agent 可以:
|
||||
安装 `opencli-browser` skill 后,你的 AI Agent 可以:
|
||||
|
||||
1. **导航**到任意 URL,使用你的已登录浏览器
|
||||
2. **读取**页面内容——通过结构化 DOM 快照(不是截图)
|
||||
@@ -128,45 +116,25 @@ npx skills add jackwener/opencli --skill smart-search
|
||||
Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自然语言描述想做的事。
|
||||
|
||||
**Skill 参考文档:**
|
||||
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 浏览器操作 + 适配器编写,全流程
|
||||
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 实时驱动 Chrome(导航、填表单、点击、抓取)
|
||||
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 给新站点写适配器,全流程
|
||||
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — 修复已有适配器
|
||||
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 浏览器自动化参考
|
||||
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — 命令和站点参考
|
||||
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — 能力搜索
|
||||
|
||||
`browser` 可用命令包括:`open`、`state`、`click`、`type`、`fill`、`select`、`keys`、`wait`、`get`、`find`、`extract`、`frames`、`screenshot`、`scroll`、`back`、`eval`、`network`、`tab list`、`tab new`、`tab select`、`tab close`、`init`、`verify`、`close`。
|
||||
|
||||
`opencli browser` 命令必须紧跟一个 `<session>` 位置参数。`opencli browser work open <url>` 和 `opencli browser work tab new [url]` 都会返回 target ID。`opencli browser work tab list` 用来查看当前已存在 tab 的 target ID,再通过 `--tab <targetId>` 把命令明确路由到某个 tab。`tab new` 只会新建 tab,不会改变默认浏览器目标;只有显式执行 `tab select <targetId>`,才会把该 tab 设为同一 session 后续未指定 target 的默认目标。
|
||||
|
||||
## 核心概念
|
||||
## 为新站点写适配器
|
||||
|
||||
### `browser`:AI Agent 的浏览器控制层
|
||||
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,全流程:
|
||||
|
||||
`opencli browser` 命令是 AI Agent 操作网站的底层原语。你不需要手动运行这些命令——把 `opencli-adapter-author` skill 安装到你的 AI Agent 中,用自然语言描述你想做的事,Agent 会自动处理浏览器操作。
|
||||
|
||||
比如你告诉 Agent:*"帮我看看小红书的通知"*——Agent 会在底层调用 `opencli browser <session> open`、`state`、`click` 等命令。
|
||||
|
||||
### 内置适配器:稳定命令
|
||||
|
||||
当某个站点能力已经存在时,优先使用 `opencli hackernews top`、`opencli reddit hot` 这类稳定命令。这些命令是确定性的,无需浏览器——人类和 AI Agent 都可以直接使用。
|
||||
|
||||
### 为新站点写适配器
|
||||
|
||||
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,它会把 Agent 带到闭环:
|
||||
|
||||
1. 侦察站点,分类 pattern(SPA / SSR / JSONP / Token / Streaming)
|
||||
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
|
||||
3. 定认证策略——`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
|
||||
4. 字段解码 + 设计输出列
|
||||
5. `opencli browser recon analyze <url>` 一步侦察,再 `opencli browser recon init <site>/<name>` → 写适配器 → `opencli browser recon verify <site>/<name>`
|
||||
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
|
||||
|
||||
### CLI 枢纽与桌面端适配器
|
||||
|
||||
OpenCLI 不只是网站 CLI,还可以:
|
||||
|
||||
- 统一代理本地二进制工具,例如 `gh`、`docker`、`obsidian`、`tg-cli`、`discord-cli`、`wx-cli`
|
||||
- 通过专门适配器和 CDP 集成控制 Electron 桌面应用
|
||||
1. **侦察**站点,分类 pattern(SPA / SSR / JSONP / Token / Streaming)
|
||||
2. **发现** endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
|
||||
3. **定认证**——`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`
|
||||
4. **字段解码** + 设计输出列
|
||||
5. `opencli browser recon analyze <url>` → `opencli browser recon init <site>/<name>` → 写适配器 → `opencli browser recon verify <site>/<name>`
|
||||
6. 站点知识沉到 `~/.opencli/sites/<site>/`,下次同站点直接吃缓存
|
||||
|
||||
## 前置要求
|
||||
|
||||
@@ -191,23 +159,6 @@ OpenCLI 不只是网站 CLI,还可以:
|
||||
|
||||
`opencli browser *` 必须紧跟一个 `<session>` 位置参数,默认使用前台窗口,并保留该 session 的 tab lease,直到你手动执行 `opencli browser <session> close` 或等空闲超时。浏览器型 adapter 默认使用后台 adapter 窗口并在命令结束后释放一次性 tab lease;如果需要调试最终页面,可以传 `--window foreground --keep-tab true`。
|
||||
|
||||
## 更新
|
||||
|
||||
```bash
|
||||
npm install -g @jackwener/opencli@latest
|
||||
|
||||
# 如果你在用打包发布的 OpenCLI skills,也一起刷新
|
||||
npx skills add jackwener/opencli
|
||||
```
|
||||
|
||||
如果你只装了部分 skill,也可以只刷新自己在用的:
|
||||
|
||||
```bash
|
||||
npx skills add jackwener/opencli --skill opencli-adapter-author
|
||||
npx skills add jackwener/opencli --skill opencli-autofix
|
||||
npx skills add jackwener/opencli --skill smart-search
|
||||
```
|
||||
|
||||
## 面向开发者
|
||||
|
||||
从源码安装:
|
||||
@@ -229,142 +180,31 @@ npm link
|
||||
|
||||
运行 `opencli list` 查看完整注册表。
|
||||
|
||||
| 站点 | 命令 | 模式 |
|
||||
|------|------|------|
|
||||
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` | 浏览器 |
|
||||
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` | 浏览器 |
|
||||
| **tieba** | `hot` `posts` `search` `read` | 浏览器 |
|
||||
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` | 浏览器 |
|
||||
| **cursor** | `status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export` | 桌面端 |
|
||||
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` | 浏览器 |
|
||||
| **codex** | `status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `projects` `history` `export` | 桌面端 |
|
||||
| **chatwise** | `status` `new` `send` `read` `ask` `model` `history` `export` `screenshot` | 桌面端 |
|
||||
| **doubao** | `status` `new` `send` `read` `ask` `history` `detail` `meeting-summary` `meeting-transcript` | 浏览器 |
|
||||
| **doubao-app** | `status` `new` `send` `read` `ask` `screenshot` `dump` | 桌面端 |
|
||||
| **notion** | `status` `search` `read` `new` `write` `sidebar` `favorites` `export` | 桌面端 |
|
||||
| **discord-app** | `status` `send` `read` `channels` `servers` `search` `members` | 桌面端 |
|
||||
| **v2ex** | `hot` `latest` `topic` `node` `user` `member` `replies` `nodes` `daily` `me` `notifications` | 公开 / 浏览器 |
|
||||
| **xueqiu** | `feed` `hot-stock` `hot` `search` `stock` `comments` `watchlist` `earnings-date` `fund-holdings` `fund-snapshot` | 浏览器 |
|
||||
| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` `serve` | 桌面端 |
|
||||
| **chatgpt-app** | `status` `new` `send` `read` `ask` `model` | 桌面端 |
|
||||
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` | 浏览器 |
|
||||
| **rednote** | `search` `note` `comments` `user` `download` `feed` `notifications` | 浏览器 |
|
||||
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` | 浏览器 |
|
||||
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` | 浏览器 |
|
||||
| **uiverse** | `code` `preview` | 浏览器 |
|
||||
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
|
||||
| **baidu-scholar** | `search` | 公开 |
|
||||
| **google-scholar** | `search` `cite` `profile` | 公开 |
|
||||
| **gov-law** | `search` `recent` | 公开 |
|
||||
| **gov-policy** | `search` `recent` | 公开 |
|
||||
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
|
||||
| **wanfang** | `search` | 公开 |
|
||||
| **xiaoyuzhou** | `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` `auth` | 本地凭证 |
|
||||
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` | 浏览器 |
|
||||
| **weixin** | `download` | 浏览器 |
|
||||
| **youtube** | `search` `video` `transcript` `comments` `channel` `playlist` `feed` `history` `watch-later` `subscriptions` `like` `unlike` `subscribe` `unsubscribe` | 浏览器 |
|
||||
| **boss** | `search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats` | 浏览器 |
|
||||
| **coupang** | `search` `add-to-cart` | 浏览器 |
|
||||
| **bbc** | `news` | 公共 API |
|
||||
| **bloomberg** | `main` `markets` `economics` `industries` `tech` `politics` `businessweek` `opinions` `feeds` `news` | 公共 API / 浏览器 |
|
||||
| **ctrip** | `search` | 浏览器 |
|
||||
| **devto** | `top` `tag` `user` | 公开 |
|
||||
| **dictionary** | `search` `synonyms` `examples` | 公开 |
|
||||
| **arxiv** | `search` `paper` | 公开 |
|
||||
| **pubmed** | `search` `article` `author` `citations` `related` | 公开 |
|
||||
| **openreview** | `search` `venue` `paper` `reviews` | 公开 |
|
||||
| **paperreview** | `submit` `review` `feedback` | 公开 |
|
||||
| **wikipedia** | `search` `summary` `random` `trending` | 公开 |
|
||||
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` | 公共 API |
|
||||
| **jd** | `item` | 浏览器 |
|
||||
| **linkedin** | `search` `timeline` | 浏览器 |
|
||||
| **reuters** | `search` | 浏览器 |
|
||||
| **smzdm** | `search` | 浏览器 |
|
||||
| **web** | `read` | 浏览器 |
|
||||
| **weibo** | `hot` `search` `feed` `user` `me` `post` `comments` | 浏览器 |
|
||||
| **yahoo-finance** | `quote` | 浏览器 |
|
||||
| **sinafinance** | `news` | 🌐 公开 |
|
||||
| **barchart** | `quote` `options` `greeks` `flow` | 浏览器 |
|
||||
| **chaoxing** | `assignments` `exams` | 浏览器 |
|
||||
| **grok** | `ask` `image` | 浏览器 |
|
||||
| **hf** | `top` | 公开 |
|
||||
| **jike** | `feed` `search` `create` `like` `comment` `repost` `notifications` `post` `topic` `user` | 浏览器 |
|
||||
| **jimeng** | `generate` `history` | 浏览器 |
|
||||
| **yollomi** | `generate` `video` `edit` `upload` `models` `remove-bg` `upscale` `face-swap` `restore` `try-on` `background` `object-remover` | 浏览器 |
|
||||
| **linux-do** | `feed` `search` `categories` `tags` `topic` `topic-content` `user-posts` `user-topics` | 浏览器 |
|
||||
| **stackoverflow** | `hot` `search` `bounties` `unanswered` | 公开 |
|
||||
| **steam** | `top-sellers` | 公开 |
|
||||
| **weread** | `shelf` `search` `book` `highlights` `notes` `notebooks` `ranking` | 浏览器 |
|
||||
| **douban** | `search` `top250` `subject` `photos` `download` `marks` `reviews` `movie-hot` `book-hot` | 浏览器 |
|
||||
| **facebook** | `feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group` | 浏览器 |
|
||||
| **google** | `news` `search` `suggest` `trends` | 公开 |
|
||||
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` | 浏览器 |
|
||||
| **1688** | `search` `item` `assets` `download` `store` | 浏览器 |
|
||||
| **gitee** | `trending` `search` `user` | 公开 / 浏览器 |
|
||||
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` | 浏览器 |
|
||||
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` | 浏览器 |
|
||||
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` | OAuth API |
|
||||
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` | 浏览器 |
|
||||
| **36kr** | `news` `hot` `search` `article` | 公开 / 浏览器 |
|
||||
| **imdb** | `search` `title` `top` `trending` `person` `reviews` | 公开 |
|
||||
| **producthunt** | `posts` `today` `hot` `browse` | 公开 / 浏览器 |
|
||||
| **instagram** | `explore` `profile` `search` `user` `followers` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `saved` | 浏览器 |
|
||||
| **lobsters** | `hot` `newest` `active` `tag` `read` | 公开 |
|
||||
| **medium** | `feed` `search` `user` | 浏览器 |
|
||||
| **sinablog** | `hot` `search` `article` `user` | 浏览器 |
|
||||
| **substack** | `feed` `search` `publication` | 浏览器 |
|
||||
| **pixiv** | `ranking` `search` `user` `illusts` `detail` `download` | 浏览器 |
|
||||
| **tiktok** | `explore` `search` `profile` `user` `following` `follow` `unfollow` `like` `unlike` `comment` `save` `unsave` `live` `notifications` `friends` | 浏览器 |
|
||||
| **bluesky** | `search` `trending` `user` `profile` `thread` `feeds` `followers` `following` `starter-packs` | 公开 |
|
||||
| **xianyu** | `search` `item` `chat` `publish` | 浏览器 |
|
||||
| **douyin** | `videos` `publish` `drafts` `draft` `delete` `stats` `profile` `update` `hashtag` `location` `activities` `collections` | 浏览器 |
|
||||
| **yuanbao** | `new` `ask` | 浏览器 |
|
||||
| 站点 | 命令 |
|
||||
|------|------|
|
||||
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` |
|
||||
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `summary` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` |
|
||||
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
|
||||
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
|
||||
| **linkedin** | `connect` `inbox` `safe-send` `search` `people-search` `sent-invitations` `thread-snapshot` `timeline` `salesnav-search` `salesnav-inbox` `salesnav-message` `salesnav-thread` |
|
||||
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` |
|
||||
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` |
|
||||
| **claude** | `ask` `send` `new` `status` `read` `history` `detail` |
|
||||
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
|
||||
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
|
||||
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
|
||||
|
||||
100+ 站点能力 — **[→ 查看完整命令列表](./docs/adapters/index.md)**
|
||||
|
||||
`*` `opencli xiaoyuzhou podcast`、`podcast-episodes`、`episode`、`download`、`transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`。
|
||||
精选清单 — **[→ 查看全部 100+ 站点和命令](./docs/adapters/index.md)**(小红书 / B站 / 知乎 / Twitter / Reddit / 抖音 / 微博 / 微信读书 / 小宇宙 / 1688 / 夸克 / Spotify / 牛客 / arxiv / Bilibili / 等)。
|
||||
|
||||
### 外部 CLI 枢纽
|
||||
|
||||
OpenCLI 也可以作为你现有命令行工具的统一入口,负责发现、自动安装和纯透传执行。
|
||||
把现有命令行工具统一接入 `opencli <tool> ...`:
|
||||
|
||||
| 外部 CLI | 描述 | 示例 |
|
||||
|----------|------|------|
|
||||
| **gh** | GitHub CLI | `opencli gh pr list --limit 5` |
|
||||
| **obsidian** | Obsidian 仓库管理 | `opencli obsidian search query="AI"` |
|
||||
| **docker** | Docker 命令行工具 | `opencli docker ps` |
|
||||
| **lark-cli** | 飞书 CLI — 消息、文档、日历、任务,200+ 命令 | `opencli lark-cli calendar +agenda` |
|
||||
| **dws** | 钉钉 CLI — 钉钉全套产品能力的跨平台命令行工具,支持人类和 AI Agent 使用 | `opencli dws msg send --to user "hello"` |
|
||||
| **wecom-cli** | 企业微信 CLI — 企业微信开放平台命令行工具,支持人类和 AI Agent 使用 | `opencli wecom-cli msg send --to user "hello"` |
|
||||
| **tg-cli** | Telegram CLI — 基于 MTProto 的本地优先同步、搜索、导出,面向 AI Agent | `opencli tg search "AI news" -f json` |
|
||||
| **discord-cli** | Discord CLI — 基于 SQLite 的本地优先同步、搜索、导出,面向 AI Agent | `opencli discord recent --channel general` |
|
||||
| **wx-cli** | 微信本地数据 CLI — 会话、聊天记录、搜索、联系人、导出 | `opencli wx search "OpenCLI"` |
|
||||
| **vercel** | Vercel — 部署项目、管理域名、环境变量、日志 | `opencli vercel deploy --prod` |
|
||||
`gh` · `docker` · `vercel` · `wrangler` · `ntn` · `obsidian` · `longbridge` · `lark-cli` · `dws` · `wecom-cli` · `tg` · `discord` · `wx`
|
||||
|
||||
**零配置透传**:OpenCLI 会把你的输入原样转发给底层二进制,保留原生 stdout / stderr 行为。
|
||||
注册自定义本地 CLI:`opencli external register <name>`;查看所有:`opencli external list`。
|
||||
|
||||
**自动安装**:如果你运行 `opencli gh ...` 时系统中还没有 `gh`,OpenCLI 会优先尝试通过系统包管理器安装,然后自动重试命令。
|
||||
|
||||
**注册自定义本地 CLI**:
|
||||
|
||||
```bash
|
||||
opencli register mycli
|
||||
```
|
||||
|
||||
### 桌面应用适配器
|
||||
|
||||
每个桌面适配器都有自己详细的文档说明,包括命令参考、启动配置与使用示例:
|
||||
|
||||
| 应用 | 描述 | 文档 |
|
||||
|-----|-------------|-----|
|
||||
| **Cursor** | 控制 Cursor IDE — Composer、对话、代码提取等 | [Doc](./docs/adapters/desktop/cursor.md) |
|
||||
| **Codex** | 在后台(无头)驱动 OpenAI Codex CLI Agent | [Doc](./docs/adapters/desktop/codex.md) |
|
||||
| **Antigravity** | 在终端直接控制 Antigravity Ultra | [Doc](./docs/adapters/desktop/antigravity.md) |
|
||||
| **ChatGPT App** | 自动化操作 ChatGPT macOS 桌面客户端 | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
|
||||
| **ChatWise** | 多 LLM 客户端(GPT-4、Claude、Gemini) | [Doc](./docs/adapters/desktop/chatwise.md) |
|
||||
| **Notion** | 搜索、读取、写入 Notion 页面 | [Doc](./docs/adapters/desktop/notion.md) |
|
||||
| **Discord** | Discord 桌面版 — 消息、频道、服务器 | [Doc](./docs/adapters/desktop/discord.md) |
|
||||
| **Doubao** | 通过 CDP 控制豆包桌面应用 | [Doc](./docs/adapters/desktop/doubao-app.md) |
|
||||
**桌面应用适配器**(Electron,通过 CDP):Cursor / Codex / Antigravity / ChatGPT App / ChatWise / Discord / Doubao — 详见 [`docs/adapters/desktop/`](./docs/adapters/desktop/)。
|
||||
|
||||
## 下载支持
|
||||
|
||||
@@ -502,20 +342,6 @@ opencli plugin uninstall my-tool # 卸载
|
||||
|
||||
详见 [插件指南](./docs/zh/guide/plugins.md) 了解如何创建自己的插件。
|
||||
|
||||
## 致 AI Agent(开发者指南)
|
||||
|
||||
如果你是一个被要求查阅代码并编写新 `opencli` 适配器的 AI,请遵守以下工作流。
|
||||
|
||||
在动代码前,先读 [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md)。它把整个流程串起来:
|
||||
|
||||
- 侦察站点,选定 pattern(SPA / SSR / JSONP / Token / Streaming)
|
||||
- 用 `opencli browser <name> network`、`eval`、interceptor 等找到目标 endpoint
|
||||
- 定认证策略(`PUBLIC` / `COOKIE` / `INTERCEPT` / `UI` / `LOCAL`)
|
||||
- 先用 `opencli browser recon analyze <url>` 一步侦察,再字段解码、设计 columns、`opencli browser recon init` 生成骨架
|
||||
- 交付前用 `opencli browser recon verify <site>/<name>` 验证
|
||||
|
||||
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。
|
||||
|
||||
## 常见问题排查
|
||||
|
||||
- **"Extension not connected" 报错**
|
||||
|
||||
+2528
-274
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,73 @@
|
||||
/**
|
||||
* 12306 account summary for the logged-in user.
|
||||
*
|
||||
* Returns non-sensitive identity fields plus masked email / mobile.
|
||||
* Use `--include-sensitive` to surface unmasked values from 12306's
|
||||
* own response (12306 already masks the ID number server-side; this
|
||||
* adapter never decodes that mask).
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { isAuthLikePayload, maskEmail, maskMobile, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
|
||||
|
||||
const ACCOUNT_INFO_URL = 'https://kyfw.12306.cn/otn/modifyUser/initQueryUserInfoApi';
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'me',
|
||||
access: 'read',
|
||||
description: 'Show the logged-in 12306 account summary. Sensitive fields (real name, email, mobile, birth date) are masked by default; pass --include-sensitive to opt in.',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real name / email / mobile / birth date. The 12306 ID-number mask is server-side and never decoded.' },
|
||||
],
|
||||
columns: ['username', 'real_name', 'email', 'mobile', 'birth_date', 'sex', 'country', 'user_type', 'member', 'active'],
|
||||
func: async (page, kwargs) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for 12306 me');
|
||||
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
|
||||
await require12306Login(page, AuthRequiredError);
|
||||
const json = requireEvaluateObject(await page.evaluate(`async () => {
|
||||
const r = await fetch(${JSON.stringify(ACCOUNT_INFO_URL)}, { credentials: 'include' });
|
||||
if (!r.ok) return { __http: r.status };
|
||||
try {
|
||||
return await r.json();
|
||||
} catch (err) {
|
||||
return { __parse: String(err && err.message || err) };
|
||||
}
|
||||
}`), 'account info');
|
||||
if (json?.__http) {
|
||||
if ([401, 403].includes(Number(json.__http))) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
|
||||
}
|
||||
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for account info`);
|
||||
}
|
||||
if (json?.__parse) {
|
||||
throw new CommandExecutionError(`12306 account info returned non-JSON body: ${json.__parse}`);
|
||||
}
|
||||
if (isAuthLikePayload(json)) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 account info requires a valid login session');
|
||||
}
|
||||
if (json?.status !== true || !json?.data?.userDTO) {
|
||||
throw new CommandExecutionError('12306 account info payload missing userDTO');
|
||||
}
|
||||
const dto = json.data.userDTO;
|
||||
const loginDto = dto.loginUserDTO || {};
|
||||
const username = loginDto.user_name || loginDto.name || '';
|
||||
const realName = loginDto.real_name || loginDto.realname || '';
|
||||
const include = kwargs['include-sensitive'] === true;
|
||||
return [{
|
||||
username,
|
||||
real_name: include ? realName : maskChineseName(realName),
|
||||
email: include ? (dto.email || '') : maskEmail(dto.email || ''),
|
||||
mobile: include ? (dto.mobile_no || '') : maskMobile(dto.mobile_no || ''),
|
||||
birth_date: include ? (dto.born_date || '') : (dto.born_date || '').slice(0, 4),
|
||||
sex: dto.sex_code === 'M' ? '男' : (dto.sex_code === 'F' ? '女' : ''),
|
||||
country: dto.country_code || '',
|
||||
user_type: json.data.userTypeName || '',
|
||||
member: dto.flag_member === '1',
|
||||
active: dto.is_active === '1',
|
||||
}];
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* 12306 in-progress orders for the logged-in user.
|
||||
*
|
||||
* Returns orders that have not yet been ridden / refunded / completed
|
||||
* (the `noComplete` slice). Order history covering completed and
|
||||
* refunded tickets uses a separate endpoint that requires extra
|
||||
* referer / page-state handshakes and is left for a follow-up so this
|
||||
* command can ship reliably.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
|
||||
|
||||
const NO_COMPLETE_URL = 'https://kyfw.12306.cn/otn/queryOrder/queryMyOrderNoComplete';
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'orders',
|
||||
access: 'read',
|
||||
description: 'List in-progress 12306 orders (not yet ridden, refunded, or completed) for the logged-in user',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked passenger names in order rows. Masked by default.' },
|
||||
],
|
||||
columns: ['order_id', 'order_date', 'train_code', 'from_station', 'to_station', 'departure', 'passengers', 'status', 'amount'],
|
||||
func: async (page, kwargs) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for 12306 orders');
|
||||
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
|
||||
await require12306Login(page, AuthRequiredError);
|
||||
const include = kwargs['include-sensitive'] === true;
|
||||
const json = requireEvaluateObject(await page.evaluate(`async () => {
|
||||
const r = await fetch(${JSON.stringify(NO_COMPLETE_URL)}, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
||||
body: '_json_att=', credentials: 'include',
|
||||
});
|
||||
if (!r.ok) return { __http: r.status };
|
||||
try {
|
||||
return await r.json();
|
||||
} catch (err) {
|
||||
return { __parse: String(err && err.message || err) };
|
||||
}
|
||||
}`), 'orders');
|
||||
if (json?.__http) {
|
||||
if ([401, 403].includes(Number(json.__http))) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
|
||||
}
|
||||
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for queryMyOrderNoComplete`);
|
||||
}
|
||||
if (json?.__parse) {
|
||||
throw new CommandExecutionError(`12306 orders returned non-JSON body: ${json.__parse}`);
|
||||
}
|
||||
if (isAuthLikePayload(json)) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 orders requires a valid login session');
|
||||
}
|
||||
if (json?.status !== true) {
|
||||
throw new CommandExecutionError('12306 queryMyOrderNoComplete returned a failure status');
|
||||
}
|
||||
let orders;
|
||||
if (Array.isArray(json?.data?.orderDBList)) {
|
||||
orders = json.data.orderDBList;
|
||||
} else if (Array.isArray(json?.data?.orderDTODataList)) {
|
||||
orders = json.data.orderDTODataList;
|
||||
} else if (Array.isArray(json?.data?.orders)) {
|
||||
orders = json.data.orders;
|
||||
} else if (Array.isArray(json?.data)) {
|
||||
orders = json.data;
|
||||
} else {
|
||||
throw new CommandExecutionError('12306 queryMyOrderNoComplete payload missing order list array');
|
||||
}
|
||||
if (orders.length === 0) {
|
||||
throw new EmptyResultError('No in-progress 12306 orders on this account');
|
||||
}
|
||||
return orders.map((o) => {
|
||||
const tickets = Array.isArray(o.tickets) ? o.tickets : [];
|
||||
const passengerNames = tickets
|
||||
.map((t) => t.passenger_name || '')
|
||||
.filter(Boolean)
|
||||
.map((name) => include ? name : maskChineseName(name))
|
||||
.join(', ');
|
||||
return {
|
||||
order_id: o.sequence_no || o.order_id || o.sequenceNo || '',
|
||||
order_date: o.order_date || '',
|
||||
train_code: o.train_code_page || o.station_train_code || o.train_code || '',
|
||||
from_station: o.from_station_name_page || o.from_station_name || '',
|
||||
to_station: o.to_station_name_page || o.to_station_name || '',
|
||||
departure: o.start_train_date_page || o.start_train_date || '',
|
||||
passengers: passengerNames,
|
||||
status: o.ticket_status_name || o.order_status_name || o.statusName || '',
|
||||
amount: o.ticket_total_price_page || o.ticket_total_price || '',
|
||||
};
|
||||
});
|
||||
},
|
||||
});
|
||||
@@ -0,0 +1,90 @@
|
||||
/**
|
||||
* 12306 saved passenger list for the logged-in user.
|
||||
*
|
||||
* 12306 already masks ID numbers (`xxxx***********xxx`) and mobile
|
||||
* numbers (`138****xxxx`) server-side. This adapter further masks the
|
||||
* passenger's Chinese real name and birth date by default; pass
|
||||
* `--include-sensitive` to surface the unmasked-by-12306 fields.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { isAuthLikePayload, maskChineseName, require12306Login, requireEvaluateObject } from './utils.js';
|
||||
|
||||
const PASSENGER_QUERY_URL = 'https://kyfw.12306.cn/otn/passengers/query';
|
||||
const MAX_PAGE_SIZE = 50;
|
||||
|
||||
function normalizeLimit(value, defaultValue, max) {
|
||||
if (value === undefined || value === null || value === '') return defaultValue;
|
||||
const n = Number(value);
|
||||
if (!Number.isInteger(n) || n < 1) throw new ArgumentError(`limit must be a positive integer (1-${max})`);
|
||||
if (n > max) throw new ArgumentError(`limit must be <= ${max}`);
|
||||
return n;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'passengers',
|
||||
access: 'read',
|
||||
description: 'List the logged-in user\'s saved 12306 passengers. Sensitive fields are masked by default; pass --include-sensitive to opt in.',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 20, help: `Max passengers to return (1-${MAX_PAGE_SIZE})` },
|
||||
{ name: 'include-sensitive', type: 'boolean', default: false, help: 'Reveal unmasked real names and birth dates. The 12306 ID-number / mobile masks are server-side and never decoded.' },
|
||||
],
|
||||
columns: ['name', 'sex', 'born_year', 'id_type', 'id_no', 'mobile', 'passenger_type', 'country'],
|
||||
func: async (page, kwargs) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for 12306 passengers');
|
||||
const limit = normalizeLimit(kwargs.limit, 20, MAX_PAGE_SIZE);
|
||||
const include = kwargs['include-sensitive'] === true;
|
||||
|
||||
await page.goto('https://kyfw.12306.cn/otn/view/index.html');
|
||||
await require12306Login(page, AuthRequiredError);
|
||||
const json = requireEvaluateObject(await page.evaluate(`async () => {
|
||||
const body = "pageIndex=1&pageSize=${MAX_PAGE_SIZE}";
|
||||
const r = await fetch(${JSON.stringify(PASSENGER_QUERY_URL)}, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
|
||||
body, credentials: 'include',
|
||||
});
|
||||
if (!r.ok) return { __http: r.status };
|
||||
try {
|
||||
return await r.json();
|
||||
} catch (err) {
|
||||
return { __parse: String(err && err.message || err) };
|
||||
}
|
||||
}`), 'passengers');
|
||||
if (json?.__http) {
|
||||
if ([401, 403].includes(Number(json.__http))) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
|
||||
}
|
||||
throw new CommandExecutionError(`12306 returned HTTP ${json.__http} for passengers/query`);
|
||||
}
|
||||
if (json?.__parse) {
|
||||
throw new CommandExecutionError(`12306 passengers returned non-JSON body: ${json.__parse}`);
|
||||
}
|
||||
if (isAuthLikePayload(json)) {
|
||||
throw new AuthRequiredError('kyfw.12306.cn', '12306 passengers requires a valid login session');
|
||||
}
|
||||
if (json?.status !== true || !Array.isArray(json?.data?.datas)) {
|
||||
throw new CommandExecutionError('12306 passengers payload missing data.datas array');
|
||||
}
|
||||
const datas = json.data.datas;
|
||||
if (datas.length === 0) {
|
||||
throw new EmptyResultError('No saved passengers on this 12306 account');
|
||||
}
|
||||
return datas.slice(0, limit).map((p) => ({
|
||||
name: include ? (p.passenger_name || '') : maskChineseName(p.passenger_name || ''),
|
||||
sex: p.sex_name || '',
|
||||
born_year: (p.born_date || '').slice(0, 4),
|
||||
id_type: p.passenger_id_type_name || '',
|
||||
id_no: p.passenger_id_no || '',
|
||||
mobile: p.mobile_no || '',
|
||||
passenger_type: p.passenger_type_name || '',
|
||||
country: p.country_code || '',
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { normalizeLimit };
|
||||
@@ -0,0 +1,166 @@
|
||||
/**
|
||||
* 12306 ticket price lookup for a single train + segment.
|
||||
*
|
||||
* Cascades three anonymous API calls:
|
||||
* 1. /otn/leftTicket/init: mint session cookies
|
||||
* 2. /otn/czxx/queryByTrainNo: resolve from/to station_no within the
|
||||
* train route (price endpoint addresses stops by station_no, not
|
||||
* telecode)
|
||||
* 3. /otn/leftTicket/queryTicketPrice: ticket prices keyed by seat
|
||||
* letter (M=一等座, O=二等座, A9=商务座, A1=硬座, A3=硬卧,
|
||||
* A4=软卧, F=动卧, P=特等座, WZ=无座, etc.)
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
|
||||
|
||||
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
|
||||
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
|
||||
const SEAT_TYPES_RE = /^[A-Z0-9]{1,32}$/;
|
||||
|
||||
const SEAT_LETTERS = {
|
||||
'A9': '商务座',
|
||||
'P': '特等座',
|
||||
'M': '一等座',
|
||||
'O': '二等座',
|
||||
'A1': '硬座',
|
||||
'A3': '硬卧',
|
||||
'A4': '软卧',
|
||||
'F': '动卧',
|
||||
'WZ': '无座',
|
||||
};
|
||||
|
||||
async function queryStopsForPrice(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
|
||||
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
|
||||
const resp = await fetchImpl(url, {
|
||||
headers: {
|
||||
'User-Agent': UA,
|
||||
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
|
||||
'Cookie': cookieHeader,
|
||||
},
|
||||
});
|
||||
if (!resp.ok) throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
|
||||
let json;
|
||||
try {
|
||||
json = await resp.json();
|
||||
} catch {
|
||||
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
|
||||
}
|
||||
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
|
||||
throw new CommandExecutionError('12306 queryByTrainNo returned an unexpected payload shape');
|
||||
}
|
||||
return json.data.data;
|
||||
}
|
||||
|
||||
function pickStationNos(stops, fromCode, toCode, fromName, toName) {
|
||||
const matches = (s, code, name) => (s.station_name && name && s.station_name === name);
|
||||
const fromStop = stops.find((s) => matches(s, fromCode, fromName));
|
||||
const toStop = stops.find((s) => matches(s, toCode, toName));
|
||||
if (!fromStop) throw new CommandExecutionError(`Train does not stop at ${fromName}`);
|
||||
if (!toStop) throw new CommandExecutionError(`Train does not stop at ${toName}`);
|
||||
return { fromNo: fromStop.station_no, toNo: toStop.station_no };
|
||||
}
|
||||
|
||||
async function queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date, fetchImpl = fetch) {
|
||||
const url = `https://kyfw.12306.cn/otn/leftTicket/queryTicketPrice?train_no=${trainNo}&from_station_no=${fromNo}&to_station_no=${toNo}&seat_types=${seatTypes}&train_date=${date}`;
|
||||
const resp = await fetchImpl(url, {
|
||||
headers: {
|
||||
'User-Agent': UA,
|
||||
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
|
||||
'Cookie': cookieHeader,
|
||||
},
|
||||
});
|
||||
if (!resp.ok) throw new CommandExecutionError(`12306 queryTicketPrice returned HTTP ${resp.status}`);
|
||||
let json;
|
||||
try {
|
||||
json = await resp.json();
|
||||
} catch {
|
||||
throw new CommandExecutionError('12306 queryTicketPrice returned non-JSON body');
|
||||
}
|
||||
if (json?.status !== true || !json?.data) {
|
||||
throw new CommandExecutionError('12306 queryTicketPrice returned an unexpected payload shape');
|
||||
}
|
||||
return json.data;
|
||||
}
|
||||
|
||||
function parsePriceData(priceData) {
|
||||
const rows = [];
|
||||
for (const [letter, value] of Object.entries(priceData)) {
|
||||
if (letter === 'train_no' || letter === 'OT') continue;
|
||||
if (typeof value !== 'string' || !value) continue;
|
||||
// 12306 doubles up some prices as bare numerics ("9": "21580"), which
|
||||
// mirror their letter sibling ("A9": "¥2158.0") in cents/no-decimal
|
||||
// form. Skip the bare numeric letter codes to avoid duplicates.
|
||||
if (/^\d+$/.test(letter)) continue;
|
||||
if (!/^[A-Z]/.test(letter)) continue;
|
||||
const numeric = value.replace(/^¥/, '');
|
||||
if (!/^[\d.]+$/.test(numeric)) continue;
|
||||
rows.push({
|
||||
seat_code: letter,
|
||||
seat_name: SEAT_LETTERS[letter] || letter,
|
||||
price: numeric,
|
||||
currency: 'CNY',
|
||||
});
|
||||
}
|
||||
rows.sort((a, b) => Number(b.price) - Number(a.price));
|
||||
return rows;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'price',
|
||||
access: 'read',
|
||||
description: 'Look up 12306 ticket prices by seat class for one train on a given date and segment (anonymous, no login required)',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: false,
|
||||
args: [
|
||||
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L)' },
|
||||
{ name: 'from', required: true, help: 'Origin station (Chinese name, telecode, or pinyin) - must be a stop of this train' },
|
||||
{ name: 'to', required: true, help: 'Destination station - must be a stop of this train' },
|
||||
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
|
||||
{ name: 'seat-types', default: 'OM9PA1A3A4FWZ', help: 'Seat-type letters to query (default covers the common classes). Examples: OM9 (二等/一等/商务), A1A3A4 (硬座/硬卧/软卧).' },
|
||||
],
|
||||
columns: ['seat_code', 'seat_name', 'price', 'currency'],
|
||||
func: async (kwargs) => {
|
||||
const trainNo = String(kwargs['train-no'] ?? '').trim();
|
||||
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
|
||||
if (!TRAIN_NO_RE.test(trainNo)) {
|
||||
throw new ArgumentError(
|
||||
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
|
||||
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
|
||||
);
|
||||
}
|
||||
const fromArg = String(kwargs.from ?? '').trim();
|
||||
const toArg = String(kwargs.to ?? '').trim();
|
||||
if (!fromArg) throw new ArgumentError('--from station must not be empty');
|
||||
if (!toArg) throw new ArgumentError('--to station must not be empty');
|
||||
const date = validateDate(kwargs.date);
|
||||
const seatTypes = String(kwargs['seat-types'] ?? '').trim() || 'OM9PA1A3A4FWZ';
|
||||
if (!SEAT_TYPES_RE.test(seatTypes)) {
|
||||
throw new ArgumentError('--seat-types must contain only 12306 seat letters/digits (A-Z, 0-9)');
|
||||
}
|
||||
|
||||
const stations = await fetchStationBundle();
|
||||
const fromStation = resolveStation(stations, fromArg);
|
||||
const toStation = resolveStation(stations, toArg);
|
||||
if (fromStation.code === toStation.code) {
|
||||
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
|
||||
}
|
||||
|
||||
const cookieHeader = await mintSession();
|
||||
const stops = await queryStopsForPrice(cookieHeader, trainNo, fromStation.code, toStation.code, date);
|
||||
const { fromNo, toNo } = pickStationNos(stops, fromStation.code, toStation.code, fromStation.name, toStation.name);
|
||||
const priceData = await queryPrice(cookieHeader, trainNo, fromNo, toNo, seatTypes, date);
|
||||
const rows = parsePriceData(priceData);
|
||||
if (rows.length === 0) {
|
||||
throw new EmptyResultError(
|
||||
`No prices returned for train_no=${trainNo} ${fromStation.name} -> ${toStation.name} on ${date}`,
|
||||
'Try a different seat-types letter set, or check that this train operates on the date.',
|
||||
);
|
||||
}
|
||||
return rows;
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { parsePriceData, pickStationNos, queryStopsForPrice, queryPrice, SEAT_LETTERS };
|
||||
@@ -0,0 +1,66 @@
|
||||
/**
|
||||
* 12306 station search.
|
||||
*
|
||||
* Queries the public `station_name.js` bundle and filters by the user's
|
||||
* keyword. Anonymous, no session needed.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { fetchStationBundle } from './utils.js';
|
||||
|
||||
const MAX_LIMIT = 50;
|
||||
|
||||
function normalizeLimit(value, defaultValue, max) {
|
||||
if (value === undefined || value === null || value === '') return defaultValue;
|
||||
const n = Number(value);
|
||||
if (!Number.isInteger(n) || n < 1) {
|
||||
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
|
||||
}
|
||||
if (n > max) {
|
||||
throw new ArgumentError(`limit must be <= ${max}`);
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'stations',
|
||||
access: 'read',
|
||||
description: 'Search 12306 (China Railway) stations by Chinese name, telecode, or pinyin keyword',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: false,
|
||||
args: [
|
||||
{ name: 'keyword', positional: true, required: true, help: 'Chinese substring (上海), telecode (AOH), or pinyin (shanghai)' },
|
||||
{ name: 'limit', type: 'int', default: 20, help: `Maximum results (1-${MAX_LIMIT})` },
|
||||
],
|
||||
columns: ['name', 'code', 'pinyin', 'abbr', 'city'],
|
||||
func: async (kwargs) => {
|
||||
const keyword = String(kwargs.keyword ?? '').trim();
|
||||
if (!keyword) throw new ArgumentError('keyword must not be empty');
|
||||
const limit = normalizeLimit(kwargs.limit, 20, MAX_LIMIT);
|
||||
|
||||
const stations = await fetchStationBundle();
|
||||
const lower = keyword.toLowerCase();
|
||||
const matches = stations.filter((s) =>
|
||||
s.name.includes(keyword)
|
||||
|| s.code === keyword.toUpperCase()
|
||||
|| s.pinyin.includes(lower)
|
||||
|| s.abbr.includes(lower)
|
||||
|| s.short.includes(lower)
|
||||
|| s.city.includes(keyword),
|
||||
);
|
||||
if (matches.length === 0) {
|
||||
throw new EmptyResultError(`No 12306 stations match "${keyword}"`);
|
||||
}
|
||||
return matches.slice(0, limit).map((s) => ({
|
||||
name: s.name,
|
||||
code: s.code,
|
||||
pinyin: s.pinyin,
|
||||
abbr: s.abbr,
|
||||
city: s.city,
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { normalizeLimit };
|
||||
@@ -0,0 +1,91 @@
|
||||
/**
|
||||
* 12306 train stop details - list every station a train calls at,
|
||||
* with arrival / departure / stopover time.
|
||||
*
|
||||
* Requires the internal `train_no` returned by `12306 trains`
|
||||
* (`24000000G10L`), not the public train code (`G1`).
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { fetchStationBundle, mintSession, resolveStation, validateDate } from './utils.js';
|
||||
|
||||
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
|
||||
const TRAIN_NO_RE = /^[0-9A-Z]{8,18}$/;
|
||||
|
||||
async function queryStops(cookieHeader, trainNo, fromCode, toCode, date, fetchImpl = fetch) {
|
||||
const url = `https://kyfw.12306.cn/otn/czxx/queryByTrainNo?train_no=${trainNo}&from_station_telecode=${fromCode}&to_station_telecode=${toCode}&depart_date=${date}`;
|
||||
const resp = await fetchImpl(url, {
|
||||
headers: {
|
||||
'User-Agent': UA,
|
||||
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
|
||||
'Cookie': cookieHeader,
|
||||
},
|
||||
});
|
||||
if (!resp.ok) {
|
||||
throw new CommandExecutionError(`12306 queryByTrainNo returned HTTP ${resp.status}`);
|
||||
}
|
||||
let json;
|
||||
try {
|
||||
json = await resp.json();
|
||||
} catch {
|
||||
throw new CommandExecutionError('12306 queryByTrainNo returned non-JSON body');
|
||||
}
|
||||
if (json?.status !== true || !Array.isArray(json?.data?.data)) {
|
||||
throw new CommandExecutionError(`12306 queryByTrainNo returned an unexpected payload shape`);
|
||||
}
|
||||
return json.data.data;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'train',
|
||||
access: 'read',
|
||||
description: 'List every station a 12306 train calls at, with arrival / departure / stopover time (anonymous, no login required)',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: false,
|
||||
args: [
|
||||
{ name: 'train-no', positional: true, required: true, help: 'Internal train_no from `12306 trains` (e.g. 24000000G10L), not the public code (G1)' },
|
||||
{ name: 'from', required: true, help: 'Origin station for the segment: Chinese name, telecode, or pinyin' },
|
||||
{ name: 'to', required: true, help: 'Destination station for the segment' },
|
||||
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
|
||||
],
|
||||
columns: ['station_no', 'station_name', 'arrive_time', 'start_time', 'stopover_time'],
|
||||
func: async (kwargs) => {
|
||||
const trainNo = String(kwargs['train-no'] ?? '').trim();
|
||||
if (!trainNo) throw new ArgumentError('<train-no> must not be empty');
|
||||
if (!TRAIN_NO_RE.test(trainNo)) {
|
||||
throw new ArgumentError(
|
||||
`<train-no> "${trainNo}" does not look like a 12306 internal train_no`,
|
||||
'Use the train_no field from `12306 trains` output (e.g. 24000000G10L), not the public code (G1).',
|
||||
);
|
||||
}
|
||||
const fromArg = String(kwargs.from ?? '').trim();
|
||||
const toArg = String(kwargs.to ?? '').trim();
|
||||
if (!fromArg) throw new ArgumentError('--from station must not be empty');
|
||||
if (!toArg) throw new ArgumentError('--to station must not be empty');
|
||||
const date = validateDate(kwargs.date);
|
||||
|
||||
const stations = await fetchStationBundle();
|
||||
const fromStation = resolveStation(stations, fromArg);
|
||||
const toStation = resolveStation(stations, toArg);
|
||||
if (fromStation.code === toStation.code) {
|
||||
throw new ArgumentError(`--from and --to must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
|
||||
}
|
||||
|
||||
const cookieHeader = await mintSession();
|
||||
const stops = await queryStops(cookieHeader, trainNo, fromStation.code, toStation.code, date);
|
||||
if (stops.length === 0) {
|
||||
throw new EmptyResultError(`No stops returned for train_no=${trainNo} on ${date}`);
|
||||
}
|
||||
return stops.map((s) => ({
|
||||
station_no: s.station_no || '',
|
||||
station_name: s.station_name || '',
|
||||
arrive_time: s.arrive_time === '----' ? '' : (s.arrive_time || ''),
|
||||
start_time: s.start_time === '----' ? '' : (s.start_time || ''),
|
||||
stopover_time: s.stopover_time === '----' ? '' : (s.stopover_time || ''),
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { queryStops, TRAIN_NO_RE };
|
||||
@@ -0,0 +1,119 @@
|
||||
/**
|
||||
* 12306 train availability between two stations on a given date.
|
||||
*
|
||||
* Flow:
|
||||
* 1. Fetch the station bundle (cached implicitly via per-process module state).
|
||||
* 2. Mint anonymous session cookies via /otn/leftTicket/init.
|
||||
* 3. Query /otn/leftTicket/queryG; if 12306 returns
|
||||
* `{c_url: "leftTicket/queryX"}` (endpoint rotation), retry once
|
||||
* against the suggested name.
|
||||
* 4. Parse the `|`-separated train records.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { fetchStationBundle, mintSession, resolveStation, validateDate, parseTrainRecord } from './utils.js';
|
||||
|
||||
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
|
||||
const QUERY_ENDPOINTS = ['queryG', 'queryO', 'queryZ', 'queryA'];
|
||||
const MAX_LIMIT = 100;
|
||||
|
||||
function normalizeLimit(value, defaultValue, max) {
|
||||
if (value === undefined || value === null || value === '') return defaultValue;
|
||||
const n = Number(value);
|
||||
if (!Number.isInteger(n) || n < 1) {
|
||||
throw new ArgumentError(`limit must be a positive integer (1-${max})`);
|
||||
}
|
||||
if (n > max) {
|
||||
throw new ArgumentError(`limit must be <= ${max}`);
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
async function queryLeftTickets(cookieHeader, fromCode, toCode, date) {
|
||||
const headers = {
|
||||
'User-Agent': UA,
|
||||
'Referer': 'https://kyfw.12306.cn/otn/leftTicket/init',
|
||||
'Cookie': cookieHeader,
|
||||
};
|
||||
const queryParams = `leftTicketDTO.train_date=${date}&leftTicketDTO.from_station=${fromCode}&leftTicketDTO.to_station=${toCode}&purpose_codes=ADULT`;
|
||||
let lastResponseText = '';
|
||||
for (const endpoint of QUERY_ENDPOINTS) {
|
||||
const url = `https://kyfw.12306.cn/otn/leftTicket/${endpoint}?${queryParams}`;
|
||||
const resp = await fetch(url, { headers });
|
||||
if (!resp.ok) {
|
||||
if (resp.status === 302) continue;
|
||||
throw new CommandExecutionError(`12306 ${endpoint} returned HTTP ${resp.status}`);
|
||||
}
|
||||
const text = await resp.text();
|
||||
lastResponseText = text;
|
||||
let json;
|
||||
try { json = JSON.parse(text); } catch {
|
||||
throw new CommandExecutionError(`12306 ${endpoint} returned non-JSON body`);
|
||||
}
|
||||
if (json?.c_url && typeof json.c_url === 'string') {
|
||||
const rotated = json.c_url.replace('leftTicket/', '').trim();
|
||||
if (rotated && !QUERY_ENDPOINTS.includes(rotated)) {
|
||||
QUERY_ENDPOINTS.unshift(rotated);
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (Array.isArray(json?.data?.result)) {
|
||||
return json.data.result;
|
||||
}
|
||||
throw new CommandExecutionError(`12306 ${endpoint} returned an unexpected payload shape`);
|
||||
}
|
||||
throw new CommandExecutionError(`12306 rejected every known query endpoint name (${QUERY_ENDPOINTS.join(', ')}); the wire protocol may have changed. Last body: ${lastResponseText.slice(0, 200)}`);
|
||||
}
|
||||
|
||||
cli({
|
||||
site: '12306',
|
||||
name: 'trains',
|
||||
access: 'read',
|
||||
description: 'List trains between two 12306 stations on a given date (anonymous, no login required)',
|
||||
domain: 'kyfw.12306.cn',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: false,
|
||||
args: [
|
||||
{ name: 'from', positional: true, required: true, help: 'Origin station: Chinese name (北京), telecode (BJP), or pinyin (beijing)' },
|
||||
{ name: 'to', positional: true, required: true, help: 'Destination station: same forms as <from>' },
|
||||
{ name: 'date', required: true, help: 'Departure date in YYYY-MM-DD' },
|
||||
{ name: 'limit', type: 'int', default: 50, help: `Maximum rows (1-${MAX_LIMIT})` },
|
||||
],
|
||||
columns: [
|
||||
'code', 'from_station', 'to_station', 'start_time', 'arrive_time',
|
||||
'duration', 'available', 'business_seat', 'first_seat', 'second_seat',
|
||||
'soft_sleeper', 'hard_sleeper', 'hard_seat', 'no_seat', 'train_no',
|
||||
],
|
||||
func: async (kwargs) => {
|
||||
const fromArg = String(kwargs.from ?? '').trim();
|
||||
const toArg = String(kwargs.to ?? '').trim();
|
||||
if (!fromArg) throw new ArgumentError('<from> station must not be empty');
|
||||
if (!toArg) throw new ArgumentError('<to> station must not be empty');
|
||||
const date = validateDate(kwargs.date);
|
||||
const limit = normalizeLimit(kwargs.limit, 50, MAX_LIMIT);
|
||||
|
||||
const stations = await fetchStationBundle();
|
||||
const fromStation = resolveStation(stations, fromArg);
|
||||
const toStation = resolveStation(stations, toArg);
|
||||
if (fromStation.code === toStation.code) {
|
||||
throw new ArgumentError(`<from> and <to> must differ; both resolved to ${fromStation.name} (${fromStation.code})`);
|
||||
}
|
||||
const stationByCode = new Map(stations.map((s) => [s.code, s]));
|
||||
|
||||
const cookieHeader = await mintSession();
|
||||
const rawRows = await queryLeftTickets(cookieHeader, fromStation.code, toStation.code, date);
|
||||
const decoded = rawRows
|
||||
.map((line) => parseTrainRecord(decodeURIComponent(line.replace(/%0A/g, '')), stationByCode))
|
||||
.filter(Boolean);
|
||||
|
||||
if (decoded.length === 0) {
|
||||
throw new EmptyResultError(
|
||||
`No trains found from ${fromStation.name} to ${toStation.name} on ${date}`,
|
||||
'Try a different date or check whether the route is operated by 12306.',
|
||||
);
|
||||
}
|
||||
return decoded.slice(0, limit);
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { normalizeLimit, queryLeftTickets };
|
||||
@@ -0,0 +1,272 @@
|
||||
/**
|
||||
* 12306 (中国铁路) shared helpers.
|
||||
*
|
||||
* - Station lookup: parses the public `station_name.js` bundle into
|
||||
* structured records.
|
||||
* - Cookie session: 12306's query endpoints reject anonymous requests
|
||||
* with `HTTP 302 -> error.html`, so callers must hit `/otn/leftTicket/init`
|
||||
* first to mint the JSESSIONID / route / BIGipServerotn cookies.
|
||||
* - Query endpoint rotation: 12306 rotates the train-query endpoint
|
||||
* name (queryO / queryZ / queryA / queryG / ...) every few weeks.
|
||||
* When the wrong name is hit, the server returns
|
||||
* `{"c_url":"leftTicket/queryG","c_name":"CLeftTicketUrl","status":false}`
|
||||
* pointing to the current correct name; retry once with that name.
|
||||
*/
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
|
||||
const STATION_BUNDLE_URL = 'https://kyfw.12306.cn/otn/resources/js/framework/station_name.js';
|
||||
const INIT_URL = 'https://kyfw.12306.cn/otn/leftTicket/init';
|
||||
const UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0 Safari/537.36';
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
const STATION_CODE_RE = /^[A-Z]{2,4}$/;
|
||||
|
||||
/**
|
||||
* Parse the `station_name.js` bundle into a station record array.
|
||||
*
|
||||
* Bundle format (single line, `@`-delimited records, each `|`-delimited):
|
||||
* `var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||...';`
|
||||
*
|
||||
* Per-record fields (positional):
|
||||
* [0] short pinyin alias (e.g. `bjb`)
|
||||
* [1] Chinese station name (e.g. `北京北`)
|
||||
* [2] telecode (3-4 uppercase letters, e.g. `VAP`) - this is the
|
||||
* wire format 12306 uses for `from_station` / `to_station`.
|
||||
* [3] full pinyin (e.g. `beijingbei`)
|
||||
* [4] short alias (duplicate of [0] usually)
|
||||
* [5] index/rank
|
||||
* [6] city code
|
||||
* [7] city name (e.g. `北京`)
|
||||
*/
|
||||
export function parseStationBundle(text) {
|
||||
const match = text.match(/'([^']+)'/);
|
||||
if (!match) {
|
||||
throw new CommandExecutionError('Failed to parse 12306 station_name.js: source string not found');
|
||||
}
|
||||
const raw = match[1];
|
||||
const records = raw.split('@').filter(Boolean);
|
||||
const stations = [];
|
||||
for (const r of records) {
|
||||
const parts = r.split('|');
|
||||
if (parts.length < 8 || !parts[2]) continue;
|
||||
stations.push({
|
||||
short: parts[0] || '',
|
||||
name: parts[1] || '',
|
||||
code: parts[2] || '',
|
||||
pinyin: parts[3] || '',
|
||||
abbr: parts[4] || '',
|
||||
city: parts[7] || '',
|
||||
});
|
||||
}
|
||||
if (stations.length === 0) {
|
||||
throw new CommandExecutionError('Failed to parse 12306 station_name.js: no station records found');
|
||||
}
|
||||
return stations;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve a user-supplied station identifier to a telecode.
|
||||
*
|
||||
* Accepts Chinese name (`上海虹桥`), telecode (`AOH`), pinyin
|
||||
* (`shanghaihongqiao`), short alias (`shh`), or city name with a
|
||||
* preference for the city's main station.
|
||||
*/
|
||||
export function resolveStation(stations, input) {
|
||||
const trimmed = String(input ?? '').trim();
|
||||
if (!trimmed) throw new ArgumentError('station must not be empty');
|
||||
if (STATION_CODE_RE.test(trimmed)) {
|
||||
const exact = stations.find((s) => s.code === trimmed);
|
||||
if (exact) return exact;
|
||||
throw new ArgumentError(`Unknown 12306 station telecode "${trimmed}"`);
|
||||
}
|
||||
const lower = trimmed.toLowerCase();
|
||||
const exactName = stations.find((s) => s.name === trimmed);
|
||||
if (exactName) return exactName;
|
||||
const exactPinyin = stations.find((s) => s.pinyin === lower);
|
||||
if (exactPinyin) return exactPinyin;
|
||||
const exactAbbr = stations.find((s) => s.abbr === lower || s.short === lower);
|
||||
if (exactAbbr) return exactAbbr;
|
||||
throw new ArgumentError(`Unknown 12306 station "${trimmed}"`, 'Try the Chinese name (上海虹桥), the 3-4 letter telecode (AOH), or full pinyin (shanghaihongqiao).');
|
||||
}
|
||||
|
||||
export function validateDate(value) {
|
||||
if (!DATE_RE.test(String(value ?? ''))) {
|
||||
throw new ArgumentError(`date must be YYYY-MM-DD, got "${value}"`);
|
||||
}
|
||||
const [y, m, d] = value.split('-').map(Number);
|
||||
const date = new Date(Date.UTC(y, m - 1, d));
|
||||
if (date.getUTCFullYear() !== y || date.getUTCMonth() !== m - 1 || date.getUTCDate() !== d) {
|
||||
throw new ArgumentError(`date "${value}" is not a real calendar date`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Extract Set-Cookie header values into a single `Cookie:` header string. */
|
||||
export function buildCookieHeader(setCookieHeaders) {
|
||||
if (!Array.isArray(setCookieHeaders) || setCookieHeaders.length === 0) return '';
|
||||
return setCookieHeaders
|
||||
.map((line) => line.split(';')[0])
|
||||
.filter(Boolean)
|
||||
.join('; ');
|
||||
}
|
||||
|
||||
export async function fetchStationBundle(fetchImpl = fetch) {
|
||||
const resp = await fetchImpl(STATION_BUNDLE_URL, {
|
||||
headers: { 'User-Agent': UA },
|
||||
});
|
||||
if (!resp.ok) {
|
||||
throw new CommandExecutionError(`Failed to fetch 12306 station bundle: HTTP ${resp.status}`);
|
||||
}
|
||||
return parseStationBundle(await resp.text());
|
||||
}
|
||||
|
||||
/** Mint a 12306 anonymous session by hitting /otn/leftTicket/init. */
|
||||
export async function mintSession(fetchImpl = fetch) {
|
||||
const resp = await fetchImpl(INIT_URL, {
|
||||
headers: { 'User-Agent': UA },
|
||||
redirect: 'follow',
|
||||
});
|
||||
if (!resp.ok) {
|
||||
throw new CommandExecutionError(`Failed to mint 12306 session: HTTP ${resp.status}`);
|
||||
}
|
||||
const setCookies = typeof resp.headers.getSetCookie === 'function'
|
||||
? resp.headers.getSetCookie()
|
||||
: resp.headers.raw?.()['set-cookie'] || [];
|
||||
const cookieHeader = buildCookieHeader(setCookies);
|
||||
if (!cookieHeader) {
|
||||
throw new CommandExecutionError('12306 init returned no session cookies');
|
||||
}
|
||||
return cookieHeader;
|
||||
}
|
||||
|
||||
/**
|
||||
* Twelve-row train query record (LEFT_TICKET_DTO).
|
||||
*
|
||||
* 12306 returns each train as a `|`-separated string with ~36 fields.
|
||||
* Positions used here come from the public web client; unused
|
||||
* positions are documented inline so future maintainers can extend
|
||||
* the row shape without re-reverse-engineering.
|
||||
*/
|
||||
export function parseTrainRecord(line, stationByCode) {
|
||||
const f = line.split('|');
|
||||
if (f.length < 33) return null;
|
||||
return {
|
||||
train_no: f[2] || '',
|
||||
code: f[3] || '',
|
||||
from_station: stationByCode.get(f[6])?.name || f[6] || '',
|
||||
to_station: stationByCode.get(f[7])?.name || f[7] || '',
|
||||
from_code: f[6] || '',
|
||||
to_code: f[7] || '',
|
||||
start_time: f[8] || '',
|
||||
arrive_time: f[9] || '',
|
||||
duration: f[10] || '',
|
||||
available: (f[1] || '').trim() === '预订' || (f[11] || '').trim() === 'Y',
|
||||
business_seat: f[32] || '',
|
||||
first_seat: f[31] || '',
|
||||
second_seat: f[30] || '',
|
||||
soft_sleeper: f[23] || '',
|
||||
hard_sleeper: f[28] || '',
|
||||
hard_seat: f[29] || '',
|
||||
no_seat: f[26] || '',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Mask helpers for sensitive identity fields rendered by 12306.
|
||||
*
|
||||
* 12306 already masks ID numbers and mobile numbers server-side
|
||||
* (`xxxx***********xxx` / `138****xxxx`); these helpers handle the
|
||||
* remaining fields (email, real Chinese name) so the adapter never
|
||||
* leaks unmasked PII without an explicit `--include-sensitive` opt-in.
|
||||
*/
|
||||
export function maskEmail(value) {
|
||||
const v = String(value || '').trim();
|
||||
if (!v) return '';
|
||||
const at = v.indexOf('@');
|
||||
if (at <= 0) return v;
|
||||
const local = v.slice(0, at);
|
||||
const domain = v.slice(at);
|
||||
if (local.length <= 2) return local[0] + '*' + domain;
|
||||
return local[0] + '*'.repeat(Math.max(1, local.length - 2)) + local.slice(-1) + domain;
|
||||
}
|
||||
|
||||
export function maskMobile(value) {
|
||||
const v = String(value || '').trim();
|
||||
if (!v) return '';
|
||||
if (/\*/.test(v)) return v;
|
||||
if (v.length < 7) return v.replace(/.(?=.)/g, '*');
|
||||
return v.slice(0, 3) + '*'.repeat(v.length - 7) + v.slice(-4);
|
||||
}
|
||||
|
||||
export function maskChineseName(value) {
|
||||
const v = String(value || '').trim();
|
||||
if (!v) return '';
|
||||
if (v.length === 1) return v;
|
||||
if (v.length === 2) return v[0] + '*';
|
||||
return v[0] + '*'.repeat(v.length - 2) + v.slice(-1);
|
||||
}
|
||||
|
||||
export function unwrapEvaluateResult(value) {
|
||||
if (
|
||||
value
|
||||
&& typeof value === 'object'
|
||||
&& !Array.isArray(value)
|
||||
&& Object.prototype.hasOwnProperty.call(value, 'session')
|
||||
&& Object.prototype.hasOwnProperty.call(value, 'data')
|
||||
) {
|
||||
return value.data;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function requireEvaluateObject(value, label) {
|
||||
const payload = unwrapEvaluateResult(value);
|
||||
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
|
||||
throw new CommandExecutionError(`12306 ${label} returned a malformed browser payload`);
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function isAuthLikePayload(payload) {
|
||||
if (!payload || typeof payload !== 'object') return false;
|
||||
const parts = [];
|
||||
if (Array.isArray(payload.messages)) parts.push(...payload.messages);
|
||||
if (payload.message) parts.push(payload.message);
|
||||
if (payload.msg) parts.push(payload.msg);
|
||||
if (payload.validateMessages && typeof payload.validateMessages === 'object') {
|
||||
parts.push(...Object.values(payload.validateMessages).flat());
|
||||
}
|
||||
const text = parts.map((item) => String(item ?? '')).join(' ');
|
||||
return /未登录|登录|请登录|身份|认证|session|Session|login/i.test(text);
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect the 12306 login marker by reading `document.cookie` from the
|
||||
* current adapter page. Cannot use `page.getCookies({url})` here:
|
||||
* 12306 sets the auth cookie `tk` and `JSESSIONID` with `Path=/otn`,
|
||||
* and CDP `Network.getCookies` with a bare URL filter excludes
|
||||
* cookies whose path does not match the URL path. `document.cookie`
|
||||
* returns all non-httponly cookies visible to the current page
|
||||
* regardless of path, which is what we need to confirm login.
|
||||
*/
|
||||
export async function require12306Login(page, AuthRequiredErrorClass) {
|
||||
const docCookie = unwrapEvaluateResult(await page.evaluate(`document.cookie || ''`));
|
||||
const cookieStr = typeof docCookie === 'string' ? docCookie : '';
|
||||
if (!/\btk=/.test(cookieStr) || !/JSESSIONID=/.test(cookieStr)) {
|
||||
throw new AuthRequiredErrorClass('kyfw.12306.cn', 'Not logged into 12306. Sign in at https://kyfw.12306.cn first.');
|
||||
}
|
||||
}
|
||||
|
||||
export const __test__ = {
|
||||
parseStationBundle,
|
||||
resolveStation,
|
||||
validateDate,
|
||||
buildCookieHeader,
|
||||
parseTrainRecord,
|
||||
maskEmail,
|
||||
maskMobile,
|
||||
maskChineseName,
|
||||
unwrapEvaluateResult,
|
||||
requireEvaluateObject,
|
||||
isAuthLikePayload,
|
||||
};
|
||||
@@ -0,0 +1,331 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { __test__ } from './utils.js';
|
||||
import { __test__ as priceTest } from './price.js';
|
||||
import { __test__ as trainTest } from './train.js';
|
||||
import './orders.js';
|
||||
|
||||
const { parseStationBundle, resolveStation, validateDate, buildCookieHeader, parseTrainRecord, maskEmail, maskMobile, maskChineseName, unwrapEvaluateResult, requireEvaluateObject, isAuthLikePayload } = __test__;
|
||||
const { parsePriceData, queryStopsForPrice, queryPrice } = priceTest;
|
||||
const { queryStops } = trainTest;
|
||||
|
||||
describe('12306 utils - parseStationBundle', () => {
|
||||
it('parses the `@`-delimited station bundle into structured records', () => {
|
||||
const bundle = "var station_names ='@bjb|北京北|VAP|beijingbei|bjb|0|0357|北京|||@bji|北京|BJP|beijing|bj|2|0357|北京|||@aoh|上海虹桥|AOH|shanghaihongqiao|shhq|10|7600|上海|||';";
|
||||
const stations = parseStationBundle(bundle);
|
||||
expect(stations).toHaveLength(3);
|
||||
expect(stations[1]).toEqual({
|
||||
short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京',
|
||||
});
|
||||
});
|
||||
|
||||
it('skips records that lack a telecode', () => {
|
||||
const bundle = "var station_names ='@xxx|||||||||@bji|北京|BJP|beijing|bj|2|0357|北京|||';";
|
||||
const stations = parseStationBundle(bundle);
|
||||
expect(stations).toHaveLength(1);
|
||||
expect(stations[0].code).toBe('BJP');
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the bundle has no parseable station rows', () => {
|
||||
expect(() => parseStationBundle("var station_names ='@xxx|||||||||';")).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 utils - resolveStation', () => {
|
||||
const stations = [
|
||||
{ short: 'bjb', name: '北京北', code: 'VAP', pinyin: 'beijingbei', abbr: 'bjb', city: '北京' },
|
||||
{ short: 'bji', name: '北京', code: 'BJP', pinyin: 'beijing', abbr: 'bj', city: '北京' },
|
||||
{ short: 'aoh', name: '上海虹桥', code: 'AOH', pinyin: 'shanghaihongqiao', abbr: 'shhq', city: '上海' },
|
||||
];
|
||||
|
||||
it('matches by exact Chinese name', () => {
|
||||
expect(resolveStation(stations, '上海虹桥').code).toBe('AOH');
|
||||
});
|
||||
|
||||
it('matches by uppercase telecode', () => {
|
||||
expect(resolveStation(stations, 'BJP').code).toBe('BJP');
|
||||
});
|
||||
|
||||
it('matches by full pinyin (case-insensitive)', () => {
|
||||
expect(resolveStation(stations, 'Beijing').code).toBe('BJP');
|
||||
});
|
||||
|
||||
it('matches by short alias / abbr', () => {
|
||||
expect(resolveStation(stations, 'shhq').code).toBe('AOH');
|
||||
});
|
||||
|
||||
it('throws ArgumentError for empty input', () => {
|
||||
expect(() => resolveStation(stations, ' ')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('throws ArgumentError for unknown station', () => {
|
||||
expect(() => resolveStation(stations, '某不存在站')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('throws ArgumentError for telecode-shaped but unknown input', () => {
|
||||
expect(() => resolveStation(stations, 'XYZ')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 utils - validateDate', () => {
|
||||
it('accepts valid YYYY-MM-DD', () => {
|
||||
expect(validateDate('2026-05-22')).toBe('2026-05-22');
|
||||
});
|
||||
|
||||
it('throws ArgumentError on wrong format', () => {
|
||||
expect(() => validateDate('2026/05/22')).toThrow(ArgumentError);
|
||||
expect(() => validateDate('26-05-22')).toThrow(ArgumentError);
|
||||
expect(() => validateDate('today')).toThrow(ArgumentError);
|
||||
expect(() => validateDate('')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('throws ArgumentError on impossible calendar dates', () => {
|
||||
expect(() => validateDate('2026-02-30')).toThrow(ArgumentError);
|
||||
expect(() => validateDate('2026-13-01')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 utils - buildCookieHeader', () => {
|
||||
it('joins set-cookie lines into a single Cookie header', () => {
|
||||
const headers = [
|
||||
'JSESSIONID=ABC123; Path=/otn',
|
||||
'BIGipServerotn=xxx.yyy; Path=/',
|
||||
'route=zzz; Expires=Sat, 01 Jan 2027 00:00:00 GMT',
|
||||
];
|
||||
expect(buildCookieHeader(headers)).toBe('JSESSIONID=ABC123; BIGipServerotn=xxx.yyy; route=zzz');
|
||||
});
|
||||
|
||||
it('returns empty string for empty input', () => {
|
||||
expect(buildCookieHeader([])).toBe('');
|
||||
expect(buildCookieHeader(undefined)).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 utils - parseTrainRecord', () => {
|
||||
const stationByCode = new Map([
|
||||
['VNP', { name: '北京南', code: 'VNP' }],
|
||||
['AOH', { name: '上海虹桥', code: 'AOH' }],
|
||||
]);
|
||||
|
||||
it('extracts the canonical train fields from a wire record', () => {
|
||||
// 33 `|`-separated fields, with positions used by parseTrainRecord populated.
|
||||
const fields = new Array(36).fill('');
|
||||
fields[0] = 'SECRET_TOKEN';
|
||||
fields[1] = '预订';
|
||||
fields[2] = '240000G54700';
|
||||
fields[3] = 'G547';
|
||||
fields[6] = 'VNP';
|
||||
fields[7] = 'AOH';
|
||||
fields[8] = '06:18';
|
||||
fields[9] = '12:11';
|
||||
fields[10] = '05:53';
|
||||
fields[11] = 'Y';
|
||||
fields[23] = ''; // soft sleeper
|
||||
fields[26] = '无'; // no seat
|
||||
fields[28] = ''; // hard sleeper
|
||||
fields[29] = ''; // hard seat
|
||||
fields[30] = '有'; // second seat
|
||||
fields[31] = '有'; // first seat
|
||||
fields[32] = '无'; // business seat
|
||||
const row = parseTrainRecord(fields.join('|'), stationByCode);
|
||||
expect(row).toEqual({
|
||||
train_no: '240000G54700',
|
||||
code: 'G547',
|
||||
from_station: '北京南',
|
||||
to_station: '上海虹桥',
|
||||
from_code: 'VNP',
|
||||
to_code: 'AOH',
|
||||
start_time: '06:18',
|
||||
arrive_time: '12:11',
|
||||
duration: '05:53',
|
||||
available: true,
|
||||
business_seat: '无',
|
||||
first_seat: '有',
|
||||
second_seat: '有',
|
||||
soft_sleeper: '',
|
||||
hard_sleeper: '',
|
||||
hard_seat: '',
|
||||
no_seat: '无',
|
||||
});
|
||||
});
|
||||
|
||||
it('does not expose the booking-handshake secret token', () => {
|
||||
const fields = new Array(36).fill('');
|
||||
fields[0] = 'SECRET_TOKEN_DO_NOT_LEAK';
|
||||
fields[2] = 't_no'; fields[3] = 'X1'; fields[6] = 'VNP'; fields[7] = 'AOH';
|
||||
const row = parseTrainRecord(fields.join('|'), stationByCode);
|
||||
expect(Object.values(row)).not.toContain('SECRET_TOKEN_DO_NOT_LEAK');
|
||||
expect('secret' in row).toBe(false);
|
||||
});
|
||||
|
||||
it('falls back to the telecode when the station bundle has no name', () => {
|
||||
const fields = new Array(36).fill('');
|
||||
fields[2] = 'X'; fields[3] = 'X'; fields[6] = 'ZZZ'; fields[7] = 'YYY';
|
||||
const row = parseTrainRecord(fields.join('|'), stationByCode);
|
||||
expect(row.from_station).toBe('ZZZ');
|
||||
expect(row.to_station).toBe('YYY');
|
||||
});
|
||||
|
||||
it('returns null for short records', () => {
|
||||
expect(parseTrainRecord('a|b|c', stationByCode)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 utils - mask helpers', () => {
|
||||
it('masks the local-part of an email', () => {
|
||||
expect(maskEmail('hello@example.com')).toBe('h***o@example.com');
|
||||
expect(maskEmail('ab@x.cn')).toBe('a*@x.cn');
|
||||
expect(maskEmail('a@x.cn')).toBe('a*@x.cn');
|
||||
expect(maskEmail('')).toBe('');
|
||||
expect(maskEmail('not-an-email')).toBe('not-an-email');
|
||||
});
|
||||
|
||||
it('masks Chinese mobile numbers while preserving 12306-side masks', () => {
|
||||
expect(maskMobile('13800001234')).toBe('138****1234');
|
||||
expect(maskMobile('138****1234')).toBe('138****1234');
|
||||
expect(maskMobile('')).toBe('');
|
||||
expect(maskMobile('123')).toBe('**3');
|
||||
});
|
||||
|
||||
it('masks Chinese real names', () => {
|
||||
expect(maskChineseName('张三')).toBe('张*');
|
||||
expect(maskChineseName('李四明')).toBe('李*明');
|
||||
expect(maskChineseName('欧阳锋')).toBe('欧*锋');
|
||||
expect(maskChineseName('张')).toBe('张');
|
||||
expect(maskChineseName('')).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 price - parsePriceData', () => {
|
||||
it('returns seat rows sorted by descending price and drops dup numeric codes', () => {
|
||||
const data = {
|
||||
train_no: '24000000G10L',
|
||||
'OT': [],
|
||||
'A9': '¥2158.0',
|
||||
'9': '21580',
|
||||
'P': '¥1163.0',
|
||||
'M': '¥1035.0',
|
||||
'O': '¥626.0',
|
||||
'WZ': '¥626.0',
|
||||
'INVALID': 'not-a-price',
|
||||
};
|
||||
const rows = parsePriceData(data);
|
||||
const codes = rows.map((r) => r.seat_code);
|
||||
expect(codes).not.toContain('9');
|
||||
expect(codes).not.toContain('OT');
|
||||
expect(codes).not.toContain('train_no');
|
||||
expect(codes).not.toContain('INVALID');
|
||||
expect(codes).toEqual(['A9', 'P', 'M', 'O', 'WZ']);
|
||||
expect(rows[0]).toEqual({ seat_code: 'A9', seat_name: '商务座', price: '2158.0', currency: 'CNY' });
|
||||
expect(rows[4]).toEqual({ seat_code: 'WZ', seat_name: '无座', price: '626.0', currency: 'CNY' });
|
||||
});
|
||||
|
||||
it('keeps unknown letter codes with the letter as the name', () => {
|
||||
const data = { 'A9': '¥100.0', 'ZZ': '¥50.0' };
|
||||
const rows = parsePriceData(data);
|
||||
const zz = rows.find((r) => r.seat_code === 'ZZ');
|
||||
expect(zz?.seat_name).toBe('ZZ');
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 public API typed boundaries', () => {
|
||||
const nonJsonFetch = async () => ({
|
||||
ok: true,
|
||||
json: async () => {
|
||||
throw new SyntaxError('Unexpected token <');
|
||||
},
|
||||
});
|
||||
|
||||
it('wraps non-JSON train stop bodies as CommandExecutionError', async () => {
|
||||
await expect(queryStops('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('wraps non-JSON price helper bodies as CommandExecutionError', async () => {
|
||||
await expect(queryStopsForPrice('cookie=1', '24000000G10L', 'BJP', 'AOH', '2026-05-22', nonJsonFetch))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(queryPrice('cookie=1', '24000000G10L', '01', '02', 'OM9', '2026-05-22', nonJsonFetch))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('12306 browser evaluate boundaries', () => {
|
||||
it('unwraps Browser Bridge {session,data} evaluate envelopes only at the boundary', () => {
|
||||
expect(unwrapEvaluateResult({ session: 's1', data: 'JSESSIONID=1; tk=2' })).toBe('JSESSIONID=1; tk=2');
|
||||
expect(unwrapEvaluateResult({ status: true, data: { value: 1 } })).toEqual({ status: true, data: { value: 1 } });
|
||||
expect(requireEvaluateObject({ session: 's1', data: { status: true } }, 'test')).toEqual({ status: true });
|
||||
expect(() => requireEvaluateObject({ session: 's1', data: null }, 'test')).toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('classifies 12306 login-like API envelopes as auth failures', () => {
|
||||
expect(isAuthLikePayload({ status: false, messages: ['用户未登录'] })).toBe(true);
|
||||
expect(isAuthLikePayload({ status: false, validateMessages: { global: ['请登录后再试'] } })).toBe(true);
|
||||
expect(isAuthLikePayload({ status: false, messages: ['系统繁忙'] })).toBe(false);
|
||||
});
|
||||
|
||||
it('masks passenger names in orders by default and supports explicit sensitive opt-in', async () => {
|
||||
const command = getRegistry().get('12306/orders');
|
||||
const makePage = () => ({
|
||||
goto: async () => {},
|
||||
evaluate: async (script) => {
|
||||
if (script === "document.cookie || ''") return { session: 'browser', data: 'JSESSIONID=abc; tk=def' };
|
||||
return {
|
||||
session: 'browser',
|
||||
data: {
|
||||
status: true,
|
||||
data: {
|
||||
orderDBList: [{
|
||||
sequence_no: 'E123',
|
||||
order_date: '2026-05-18 10:00',
|
||||
train_code_page: 'G1',
|
||||
from_station_name_page: '北京南',
|
||||
to_station_name_page: '上海虹桥',
|
||||
start_train_date_page: '2026-05-22 07:00',
|
||||
ticket_status_name: '未出行',
|
||||
ticket_total_price_page: '626.0',
|
||||
tickets: [{ passenger_name: '张三' }, { passenger_name: '李四明' }],
|
||||
}],
|
||||
},
|
||||
},
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
await expect(command.func(makePage(), {})).resolves.toMatchObject([
|
||||
{ order_id: 'E123', passengers: '张*, 李*明' },
|
||||
]);
|
||||
await expect(command.func(makePage(), { 'include-sensitive': true })).resolves.toMatchObject([
|
||||
{ order_id: 'E123', passengers: '张三, 李四明' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('maps login-like order payloads to AuthRequiredError instead of parser drift', async () => {
|
||||
const command = getRegistry().get('12306/orders');
|
||||
const page = {
|
||||
goto: async () => {},
|
||||
evaluate: async (script) => {
|
||||
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
|
||||
return { status: false, messages: ['用户未登录'] };
|
||||
},
|
||||
};
|
||||
|
||||
await expect(command.func(page, {})).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('treats missing order list shape as parser drift but known empty arrays as empty result', async () => {
|
||||
const command = getRegistry().get('12306/orders');
|
||||
const makePage = (payload) => ({
|
||||
goto: async () => {},
|
||||
evaluate: async (script) => {
|
||||
if (script === "document.cookie || ''") return 'JSESSIONID=abc; tk=def';
|
||||
return payload;
|
||||
},
|
||||
});
|
||||
|
||||
await expect(command.func(makePage({ status: true, data: {} }), {}))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ status: true, data: { orderDBList: [] } }), {}))
|
||||
.rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
@@ -52,12 +52,15 @@ cli({
|
||||
if (!data?.title) {
|
||||
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
|
||||
}
|
||||
if (!data.body) {
|
||||
throw new CliError('PARSE_ERROR', 'Article body not found', '36kr page loaded but no article body paragraphs were extracted');
|
||||
}
|
||||
return [
|
||||
{ field: 'title', value: data.title },
|
||||
{ field: 'author', value: data.author || '-' },
|
||||
{ field: 'date', value: data.date || '-' },
|
||||
{ field: 'author', value: data.author || '' },
|
||||
{ field: 'date', value: data.date || '' },
|
||||
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
|
||||
{ field: 'body', value: data.body || '-' },
|
||||
{ field: 'body', value: data.body || '' },
|
||||
];
|
||||
},
|
||||
});
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { CliError } from '@jackwener/opencli/errors';
|
||||
import './article.js';
|
||||
|
||||
function makePage(evaluateResult) {
|
||||
return {
|
||||
installInterceptor: vi.fn().mockResolvedValue(undefined),
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('36kr article', () => {
|
||||
it('emits empty-string for missing optional author / date instead of a sentinel', async () => {
|
||||
const command = getRegistry().get('36kr/article');
|
||||
expect(command?.func).toBeDefined();
|
||||
const page = makePage({ title: 'Real Title', author: '', date: '', body: 'Real article body' });
|
||||
const rows = await command.func(page, { id: '1234567' });
|
||||
const byField = Object.fromEntries(rows.map((r) => [r.field, r.value]));
|
||||
expect(byField.title).toBe('Real Title');
|
||||
expect(byField.author).toBe('');
|
||||
expect(byField.date).toBe('');
|
||||
expect(byField.body).toBe('Real article body');
|
||||
expect(byField.url).toBe('https://36kr.com/p/1234567');
|
||||
});
|
||||
|
||||
it('throws CliError NOT_FOUND when the page exposes no title', async () => {
|
||||
const command = getRegistry().get('36kr/article');
|
||||
const page = makePage({ title: '', author: 'x', date: 'y', body: 'z' });
|
||||
await expect(command.func(page, { id: '1234567' })).rejects.toBeInstanceOf(CliError);
|
||||
});
|
||||
|
||||
it('throws CliError PARSE_ERROR when the page exposes title but no body', async () => {
|
||||
const command = getRegistry().get('36kr/article');
|
||||
const page = makePage({ title: 'Real Title', author: 'x', date: 'y', body: '' });
|
||||
await expect(command.func(page, { id: '1234567' })).rejects.toMatchObject({ code: 'PARSE_ERROR' });
|
||||
});
|
||||
|
||||
it('throws CliError INVALID_ARGUMENT when no numeric id can be parsed', async () => {
|
||||
const command = getRegistry().get('36kr/article');
|
||||
const page = makePage({});
|
||||
await expect(command.func(page, { id: 'not-a-url' })).rejects.toBeInstanceOf(CliError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,70 @@
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
export function requireSearchQuery(value, label = 'keyword') {
|
||||
const query = String(value ?? '').trim();
|
||||
if (!query) {
|
||||
throw new ArgumentError(`${label} cannot be empty`);
|
||||
}
|
||||
return query;
|
||||
}
|
||||
|
||||
export function requireBoundedInteger(value, defaultValue, min, max, label) {
|
||||
const raw = value ?? defaultValue;
|
||||
const parsed = typeof raw === 'number' ? raw : Number(raw);
|
||||
if (!Number.isInteger(parsed)) {
|
||||
throw new ArgumentError(`${label} must be an integer between ${min} and ${max}, got ${JSON.stringify(value)}`);
|
||||
}
|
||||
if (parsed < min || parsed > max) {
|
||||
throw new ArgumentError(`${label} must be between ${min} and ${max}, got ${parsed}`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
export function requireNonNegativeInteger(value, defaultValue, label) {
|
||||
const raw = value ?? defaultValue;
|
||||
const parsed = typeof raw === 'number' ? raw : Number(raw);
|
||||
if (!Number.isInteger(parsed) || parsed < 0) {
|
||||
throw new ArgumentError(`${label} must be a non-negative integer, got ${JSON.stringify(value)}`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
export function unwrapBrowserResult(value) {
|
||||
if (value && typeof value === 'object' && !Array.isArray(value) && 'session' in value && 'data' in value) {
|
||||
return value.data;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
export function requireRows(value, label) {
|
||||
const rows = unwrapBrowserResult(value);
|
||||
if (!Array.isArray(rows)) {
|
||||
throw new CommandExecutionError(`${label} returned an unexpected payload shape; expected an array of result rows.`);
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
export function toHttpsUrl(value, baseUrl) {
|
||||
const raw = String(value ?? '').trim();
|
||||
if (!raw) return '';
|
||||
try {
|
||||
const url = new URL(raw, baseUrl);
|
||||
if (url.protocol !== 'http:' && url.protocol !== 'https:') return '';
|
||||
return url.href;
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
export function emptySearchResults(site, query) {
|
||||
return new EmptyResultError(`${site} search`, `No ${site} results matched "${query}".`);
|
||||
}
|
||||
|
||||
export async function runBrowserStep(label, fn) {
|
||||
try {
|
||||
return await fn();
|
||||
} catch (error) {
|
||||
if (error?.code || error?.name === 'ArgumentError') throw error;
|
||||
throw new CommandExecutionError(`${label} failed: ${error?.message ?? error}`);
|
||||
}
|
||||
}
|
||||
@@ -41,6 +41,26 @@ describe('apple-podcasts search command', () => {
|
||||
}),
|
||||
]);
|
||||
});
|
||||
it('emits empty-string for missing trackCount and primaryGenreName instead of a sentinel', async () => {
|
||||
const cmd = getRegistry().get('apple-podcasts/search');
|
||||
const fetchMock = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: () => Promise.resolve({
|
||||
results: [
|
||||
{
|
||||
collectionId: 99,
|
||||
collectionName: 'No-Meta Show',
|
||||
artistName: 'Anon Host',
|
||||
collectionViewUrl: 'https://example.com/p/99',
|
||||
},
|
||||
],
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
const result = await cmd.func({ query: 'no-meta', limit: 1 });
|
||||
expect(result[0].episodes).toBe('');
|
||||
expect(result[0].genre).toBe('');
|
||||
});
|
||||
});
|
||||
describe('apple-podcasts top command', () => {
|
||||
beforeEach(() => {
|
||||
|
||||
@@ -23,8 +23,8 @@ cli({
|
||||
id: p.collectionId,
|
||||
title: p.collectionName,
|
||||
author: p.artistName,
|
||||
episodes: p.trackCount ?? '-',
|
||||
genre: p.primaryGenreName ?? '-',
|
||||
episodes: p.trackCount ?? '',
|
||||
genre: p.primaryGenreName ?? '',
|
||||
url: p.collectionViewUrl || '',
|
||||
}));
|
||||
},
|
||||
|
||||
+144
-56
@@ -4,6 +4,47 @@
|
||||
* Auth: CSRF token from <meta name="csrf-token"> + session cookies.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const DEFAULT_LIMIT = 10;
|
||||
const MIN_LIMIT = 1;
|
||||
const MAX_LIMIT = 100;
|
||||
|
||||
function normalizeSymbol(value) {
|
||||
const symbol = String(value ?? '').trim().toUpperCase();
|
||||
if (!symbol) throw new ArgumentError('symbol is required');
|
||||
return symbol;
|
||||
}
|
||||
|
||||
function normalizeExpiration(value) {
|
||||
const expiration = String(value ?? '').trim();
|
||||
if (!expiration) return '';
|
||||
if (!/^\d{4}-\d{2}-\d{2}$/.test(expiration)) {
|
||||
throw new ArgumentError('--expiration must use YYYY-MM-DD format');
|
||||
}
|
||||
const parsed = new Date(`${expiration}T00:00:00Z`);
|
||||
if (Number.isNaN(parsed.getTime()) || parsed.toISOString().slice(0, 10) !== expiration) {
|
||||
throw new ArgumentError('--expiration must be a valid calendar date');
|
||||
}
|
||||
return expiration;
|
||||
}
|
||||
|
||||
function parseLimit(value) {
|
||||
if (value === undefined || value === null || value === '') return DEFAULT_LIMIT;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < MIN_LIMIT || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}`);
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapBrowserResult(value) {
|
||||
if (value && typeof value === 'object' && 'session' in value && 'data' in value) {
|
||||
return value.data;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'barchart',
|
||||
name: 'greeks',
|
||||
@@ -14,19 +55,19 @@ cli({
|
||||
args: [
|
||||
{ name: 'symbol', required: true, positional: true, help: 'Stock ticker (e.g. AAPL)' },
|
||||
{ name: 'expiration', type: 'str', help: 'Expiration date (YYYY-MM-DD). Defaults to the nearest available expiration.' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of near-the-money strikes per type' },
|
||||
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: 'Number of near-the-money strikes per type (1-100)' },
|
||||
],
|
||||
columns: [
|
||||
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
|
||||
'volume', 'openInterest', 'expiration',
|
||||
],
|
||||
func: async (page, kwargs) => {
|
||||
const symbol = kwargs.symbol.toUpperCase().trim();
|
||||
const expiration = kwargs.expiration ?? '';
|
||||
const limit = kwargs.limit ?? 10;
|
||||
const symbol = normalizeSymbol(kwargs.symbol);
|
||||
const expiration = normalizeExpiration(kwargs.expiration);
|
||||
const limit = parseLimit(kwargs.limit);
|
||||
await page.goto(`https://www.barchart.com/stocks/quotes/${encodeURIComponent(symbol)}/options`);
|
||||
await page.wait(4);
|
||||
const data = await page.evaluate(`
|
||||
const data = unwrapBrowserResult(await page.evaluate(`
|
||||
(async () => {
|
||||
const sym = ${JSON.stringify(symbol)};
|
||||
const expDate = ${JSON.stringify(expiration)};
|
||||
@@ -45,39 +86,53 @@ cli({
|
||||
+ '&fields=' + fields + '&raw=1';
|
||||
if (expDate) url += '&expirationDate=' + encodeURIComponent(expDate);
|
||||
const resp = await fetch(url, { credentials: 'include', headers });
|
||||
if (resp.ok) {
|
||||
const d = await resp.json();
|
||||
let items = d?.data || [];
|
||||
if (!resp.ok) {
|
||||
return { ok: false, reason: 'http', status: resp.status, statusText: resp.statusText || '' };
|
||||
}
|
||||
|
||||
if (!expDate) {
|
||||
const expirations = items
|
||||
.map(i => (i.raw || i).expirationDate || null)
|
||||
.filter(Boolean)
|
||||
.sort((a, b) => {
|
||||
const aTime = Date.parse(a);
|
||||
const bTime = Date.parse(b);
|
||||
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
|
||||
if (Number.isNaN(aTime)) return 1;
|
||||
if (Number.isNaN(bTime)) return -1;
|
||||
return aTime - bTime;
|
||||
});
|
||||
const nearestExpiration = expirations[0];
|
||||
if (nearestExpiration) {
|
||||
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
|
||||
}
|
||||
const d = await resp.json();
|
||||
const allItems = d?.data;
|
||||
if (!Array.isArray(allItems)) {
|
||||
return { ok: false, reason: 'malformed' };
|
||||
}
|
||||
let items = allItems;
|
||||
|
||||
if (!expDate) {
|
||||
const expirations = items
|
||||
.map(i => (i.raw || i).expirationDate || null)
|
||||
.filter(Boolean)
|
||||
.sort((a, b) => {
|
||||
const aTime = Date.parse(a);
|
||||
const bTime = Date.parse(b);
|
||||
if (Number.isNaN(aTime) && Number.isNaN(bTime)) return 0;
|
||||
if (Number.isNaN(aTime)) return 1;
|
||||
if (Number.isNaN(bTime)) return -1;
|
||||
return aTime - bTime;
|
||||
});
|
||||
const nearestExpiration = expirations[0];
|
||||
if (nearestExpiration) {
|
||||
items = items.filter(i => ((i.raw || i).expirationDate || null) === nearestExpiration);
|
||||
}
|
||||
}
|
||||
|
||||
// Separate calls and puts, sort by distance from current price
|
||||
const calls = items
|
||||
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
|
||||
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
|
||||
.slice(0, limit);
|
||||
const puts = items
|
||||
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
|
||||
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
|
||||
.slice(0, limit);
|
||||
// Separate calls and puts, sort by distance from current price.
|
||||
const calls = items
|
||||
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'call')
|
||||
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
|
||||
.slice(0, limit);
|
||||
const puts = items
|
||||
.filter(i => ((i.raw || i).optionType || '').toLowerCase() === 'put')
|
||||
.sort((a, b) => Math.abs((a.raw || a).percentFromLast || 999) - Math.abs((b.raw || b).percentFromLast || 999))
|
||||
.slice(0, limit);
|
||||
const selected = [...calls, ...puts];
|
||||
|
||||
return [...calls, ...puts].map(i => {
|
||||
if (items.length > 0 && selected.length === 0) {
|
||||
return { ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' };
|
||||
}
|
||||
|
||||
return {
|
||||
ok: true,
|
||||
rows: selected.map(i => {
|
||||
const r = i.raw || i;
|
||||
return {
|
||||
type: r.optionType,
|
||||
@@ -93,28 +148,61 @@ cli({
|
||||
openInterest: r.openInterest,
|
||||
expiration: r.expirationDate,
|
||||
};
|
||||
});
|
||||
}
|
||||
} catch(e) {}
|
||||
|
||||
return [];
|
||||
})
|
||||
};
|
||||
} catch(e) {
|
||||
return { ok: false, reason: 'exception', message: e?.message || String(e) };
|
||||
}
|
||||
})()
|
||||
`);
|
||||
if (!data || !Array.isArray(data))
|
||||
return [];
|
||||
return data.map(r => ({
|
||||
type: r.type || '',
|
||||
strike: r.strike,
|
||||
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
|
||||
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
|
||||
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
|
||||
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
|
||||
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
|
||||
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
|
||||
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
|
||||
volume: r.volume,
|
||||
openInterest: r.openInterest,
|
||||
expiration: r.expiration ?? null,
|
||||
}));
|
||||
`));
|
||||
if (!data || data.ok !== true) {
|
||||
if (data?.reason === 'http') {
|
||||
throw new CommandExecutionError(`Barchart greeks request failed: HTTP ${data.status}${data.statusText ? ` ${data.statusText}` : ''}`);
|
||||
}
|
||||
if (data?.reason === 'malformed') {
|
||||
throw new CommandExecutionError(`Barchart greeks returned an unreadable options payload${data.message ? `: ${data.message}` : ''}`);
|
||||
}
|
||||
if (data?.reason === 'exception') {
|
||||
throw new CommandExecutionError(`Barchart greeks request failed: ${data.message || 'unknown error'}`);
|
||||
}
|
||||
throw new CommandExecutionError(`Failed to fetch Barchart greeks for ${symbol}`);
|
||||
}
|
||||
if (!Array.isArray(data.rows)) {
|
||||
throw new CommandExecutionError('Barchart greeks returned an unreadable options payload');
|
||||
}
|
||||
if (data.rows.length === 0) {
|
||||
throw new EmptyResultError('barchart greeks', `No option greeks were returned for ${symbol}. Confirm the symbol, expiration, and Barchart login state.`);
|
||||
}
|
||||
return data.rows.map(r => {
|
||||
if (!r || typeof r !== 'object' || Array.isArray(r)) {
|
||||
throw new CommandExecutionError('Barchart greeks returned a malformed option row');
|
||||
}
|
||||
const type = String(r.type || '').trim();
|
||||
const expirationValue = String(r.expiration || '').trim();
|
||||
if (!/^(call|put)$/i.test(type) || r.strike === undefined || r.strike === null || r.strike === '' || !expirationValue) {
|
||||
throw new CommandExecutionError('Barchart greeks returned a malformed option row identity');
|
||||
}
|
||||
return {
|
||||
type,
|
||||
strike: r.strike,
|
||||
last: r.last != null ? Number(Number(r.last).toFixed(2)) : null,
|
||||
iv: r.iv != null ? Number(Number(r.iv).toFixed(2)) + '%' : null,
|
||||
delta: r.delta != null ? Number(Number(r.delta).toFixed(4)) : null,
|
||||
gamma: r.gamma != null ? Number(Number(r.gamma).toFixed(4)) : null,
|
||||
theta: r.theta != null ? Number(Number(r.theta).toFixed(4)) : null,
|
||||
vega: r.vega != null ? Number(Number(r.vega).toFixed(4)) : null,
|
||||
rho: r.rho != null ? Number(Number(r.rho).toFixed(4)) : null,
|
||||
volume: r.volume,
|
||||
openInterest: r.openInterest,
|
||||
expiration: expirationValue,
|
||||
};
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeSymbol,
|
||||
normalizeExpiration,
|
||||
parseLimit,
|
||||
unwrapBrowserResult,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './greeks.js';
|
||||
|
||||
const { normalizeExpiration, normalizeSymbol, parseLimit, unwrapBrowserResult } = await import('./greeks.js').then((m) => m.__test__);
|
||||
|
||||
function makePage(evaluateResult) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('barchart greeks command', () => {
|
||||
const command = getRegistry().get('barchart/greeks');
|
||||
|
||||
it('registers with the expected shape', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.access).toBe('read');
|
||||
expect(command.browser).toBe(true);
|
||||
expect(command.columns).toEqual([
|
||||
'type', 'strike', 'last', 'iv', 'delta', 'gamma', 'theta', 'vega', 'rho',
|
||||
'volume', 'openInterest', 'expiration',
|
||||
]);
|
||||
});
|
||||
|
||||
it('maps returned option rows without changing the declared output shape', async () => {
|
||||
const page = makePage({
|
||||
session: 'site:barchart',
|
||||
data: {
|
||||
ok: true,
|
||||
rows: [
|
||||
{
|
||||
type: 'Call',
|
||||
strike: 190,
|
||||
last: 3.456,
|
||||
iv: 21.234,
|
||||
delta: 0.56789,
|
||||
gamma: 0.01234,
|
||||
theta: -0.12345,
|
||||
vega: 0.23456,
|
||||
rho: 0.03456,
|
||||
volume: 123,
|
||||
openInterest: 456,
|
||||
expiration: '2026-06-19',
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const rows = await command.func(page, { symbol: 'aapl', limit: 1 });
|
||||
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.barchart.com/stocks/quotes/AAPL/options');
|
||||
expect(page.wait).toHaveBeenCalledWith(4);
|
||||
expect(rows).toEqual([
|
||||
{
|
||||
type: 'Call',
|
||||
strike: 190,
|
||||
last: 3.46,
|
||||
iv: '21.23%',
|
||||
delta: 0.5679,
|
||||
gamma: 0.0123,
|
||||
theta: -0.1235,
|
||||
vega: 0.2346,
|
||||
rho: 0.0346,
|
||||
volume: 123,
|
||||
openInterest: 456,
|
||||
expiration: '2026-06-19',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('validates args before browser navigation and unwraps bridge envelopes', async () => {
|
||||
expect(normalizeSymbol(' aapl ')).toBe('AAPL');
|
||||
expect(normalizeExpiration('2026-06-19')).toBe('2026-06-19');
|
||||
expect(parseLimit(undefined)).toBe(10);
|
||||
expect(parseLimit(100)).toBe(100);
|
||||
expect(unwrapBrowserResult({ session: 'site:barchart', data: { ok: true } })).toEqual({ ok: true });
|
||||
|
||||
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: '', limit: 1 }))
|
||||
.rejects.toBeInstanceOf(ArgumentError);
|
||||
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', expiration: '2026-02-30', limit: 1 }))
|
||||
.rejects.toBeInstanceOf(ArgumentError);
|
||||
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL', limit: 101 }))
|
||||
.rejects.toBeInstanceOf(ArgumentError);
|
||||
});
|
||||
|
||||
it('embeds expiration and limit in the browser-side request script', async () => {
|
||||
const page = makePage({
|
||||
ok: true,
|
||||
rows: [{
|
||||
type: 'Put',
|
||||
strike: 185,
|
||||
last: null,
|
||||
iv: null,
|
||||
delta: null,
|
||||
gamma: null,
|
||||
theta: null,
|
||||
vega: null,
|
||||
rho: null,
|
||||
volume: 0,
|
||||
openInterest: 0,
|
||||
expiration: '2026-07-17',
|
||||
}],
|
||||
});
|
||||
|
||||
await command.func(page, { symbol: 'MSFT', expiration: '2026-07-17', limit: 7 });
|
||||
const script = page.evaluate.mock.calls[0][0];
|
||||
|
||||
expect(script).toContain('const expDate = "2026-07-17"');
|
||||
expect(script).toContain('const limit = 7');
|
||||
expect(script).toContain("url += '&expirationDate=' + encodeURIComponent(expDate)");
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for HTTP, malformed, exception, and missing payload states', async () => {
|
||||
await expect(command.func(makePage({ ok: false, reason: 'http', status: 403, statusText: 'Forbidden' }), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ ok: false, reason: 'malformed' }), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ ok: false, reason: 'exception', message: 'network down' }), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ ok: false, reason: 'malformed', message: 'options rows did not include call or put identities' }), { symbol: 'AAPL' }))
|
||||
.rejects.toThrow('call or put identities');
|
||||
await expect(command.func(makePage(null), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ ok: true, rows: 'bad' }), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(command.func(makePage({ ok: true, rows: [{ type: 'Call', strike: null, expiration: '' }] }), { symbol: 'AAPL' }))
|
||||
.rejects.toThrow('malformed option row identity');
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when Barchart returns no greeks rows', async () => {
|
||||
await expect(command.func(makePage({ ok: true, rows: [] }), { symbol: 'AAPL' }))
|
||||
.rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,167 @@
|
||||
/**
|
||||
* Bilibili summary — fetches the official AI-generated video summary (the "AI总结"
|
||||
* shown on the video page) via /x/web-interface/view/conclusion/get.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { apiGet, resolveBvid } from './utils.js';
|
||||
|
||||
const BILIBILI_HOST_RE = /(^|\.)bilibili\.com$/i;
|
||||
const B23_HOST_RE = /(^|\.)b23\.tv$/i;
|
||||
const BVID_RE = /^BV[A-Za-z0-9]+$/;
|
||||
|
||||
function formatTime(seconds) {
|
||||
const s = Math.max(0, Math.floor(Number(seconds) || 0));
|
||||
const h = Math.floor(s / 3600);
|
||||
const m = Math.floor((s % 3600) / 60);
|
||||
const sec = s % 60;
|
||||
const pad = (n) => String(n).padStart(2, '0');
|
||||
return h > 0 ? `${h}:${pad(m)}:${pad(sec)}` : `${pad(m)}:${pad(sec)}`;
|
||||
}
|
||||
|
||||
async function readBvid(raw) {
|
||||
const input = String(raw ?? '').trim();
|
||||
if (!input) {
|
||||
throw new ArgumentError('bilibili summary bvid cannot be empty', 'Pass a BV ID, Bilibili video URL, or b23.tv short link.');
|
||||
}
|
||||
if (BVID_RE.test(input)) {
|
||||
return input;
|
||||
}
|
||||
let parsed = null;
|
||||
try {
|
||||
parsed = new URL(input);
|
||||
} catch {
|
||||
// Bare b23.tv short codes are accepted by the shared resolver.
|
||||
}
|
||||
if (parsed) {
|
||||
if (parsed.protocol !== 'https:' && parsed.protocol !== 'http:') {
|
||||
throw new ArgumentError('Bilibili summary URL must use http or https');
|
||||
}
|
||||
if (BILIBILI_HOST_RE.test(parsed.hostname)) {
|
||||
const match = parsed.pathname.match(/\/(?:video|bangumi\/play)\/(BV[A-Za-z0-9]+)/i);
|
||||
if (!match) {
|
||||
throw new ArgumentError('Bilibili summary URL must contain a BV video id');
|
||||
}
|
||||
return match[1];
|
||||
}
|
||||
if (!B23_HOST_RE.test(parsed.hostname)) {
|
||||
throw new ArgumentError('Bilibili summary URL must be a bilibili.com or b23.tv URL');
|
||||
}
|
||||
}
|
||||
try {
|
||||
return await resolveBvid(input);
|
||||
} catch (error) {
|
||||
throw new ArgumentError(`Cannot resolve Bilibili BV ID from input: ${input}`, error instanceof Error ? error.message : String(error));
|
||||
}
|
||||
}
|
||||
|
||||
function requireOkPayload(payload, label) {
|
||||
if (!payload || typeof payload !== 'object' || Array.isArray(payload)) {
|
||||
throw new CommandExecutionError(`Bilibili ${label} API returned a malformed payload`);
|
||||
}
|
||||
if (payload.code !== 0) {
|
||||
const message = payload.message ?? 'unknown error';
|
||||
if (payload.code === -101 || payload.code === -403 || /登录|权限|forbidden|permission|login/i.test(String(message))) {
|
||||
throw new AuthRequiredError('bilibili.com', `Bilibili ${label} API requires login or permission: ${message} (${payload.code})`);
|
||||
}
|
||||
throw new CommandExecutionError(`Bilibili ${label} API failed: ${message} (${payload.code})`);
|
||||
}
|
||||
return payload.data;
|
||||
}
|
||||
|
||||
function readModelResult(data, bvid) {
|
||||
if (!data || typeof data !== 'object' || Array.isArray(data)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed data');
|
||||
}
|
||||
if (data.code !== 0) {
|
||||
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
|
||||
}
|
||||
let modelResult = data.model_result;
|
||||
if (typeof modelResult === 'string') {
|
||||
try {
|
||||
modelResult = JSON.parse(modelResult);
|
||||
} catch {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result JSON');
|
||||
}
|
||||
}
|
||||
if (!modelResult || typeof modelResult !== 'object' || Array.isArray(modelResult)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed model_result');
|
||||
}
|
||||
const summary = String(modelResult.summary ?? '').trim();
|
||||
if (!summary) {
|
||||
throw new EmptyResultError('bilibili summary', `Bilibili has not generated an AI summary for ${bvid}.`);
|
||||
}
|
||||
const outline = modelResult.outline ?? [];
|
||||
if (!Array.isArray(outline)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline');
|
||||
}
|
||||
return { summary, outline };
|
||||
}
|
||||
|
||||
function rowsFromModel(model) {
|
||||
const rows = [{ time: '', content: model.summary }];
|
||||
for (const section of model.outline) {
|
||||
if (!section || typeof section !== 'object' || Array.isArray(section)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline section');
|
||||
}
|
||||
const sectionTitle = String(section.title ?? '').trim();
|
||||
const sectionTime = formatTime(section.timestamp);
|
||||
if (sectionTitle) {
|
||||
rows.push({ time: sectionTime, content: `# ${sectionTitle}` });
|
||||
}
|
||||
const points = section.part_outline ?? [];
|
||||
if (!Array.isArray(points)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed part outline');
|
||||
}
|
||||
for (const point of points) {
|
||||
if (!point || typeof point !== 'object' || Array.isArray(point)) {
|
||||
throw new CommandExecutionError('Bilibili conclusion API returned malformed outline point');
|
||||
}
|
||||
const content = String(point.content ?? '').trim();
|
||||
if (content) {
|
||||
rows.push({ time: formatTime(point.timestamp), content });
|
||||
}
|
||||
}
|
||||
}
|
||||
return rows;
|
||||
}
|
||||
|
||||
var command = cli({
|
||||
site: 'bilibili',
|
||||
name: 'summary',
|
||||
access: 'read',
|
||||
description: '获取 B站视频的官方 AI 总结(视频页「AI总结」同款,含分段大纲与时间戳)',
|
||||
domain: 'www.bilibili.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [
|
||||
{ name: 'bvid', required: true, positional: true, help: 'Video BV ID / URL / b23.tv short link' },
|
||||
],
|
||||
columns: ['time', 'content'],
|
||||
func: async (page, kwargs) => {
|
||||
if (!page) {
|
||||
throw new CommandExecutionError('Browser session required for bilibili summary');
|
||||
}
|
||||
const bvid = await readBvid(kwargs.bvid);
|
||||
const view = await apiGet(page, '/x/web-interface/view', { params: { bvid } });
|
||||
const viewData = requireOkPayload(view, 'view');
|
||||
const cid = viewData?.cid;
|
||||
const upMid = viewData?.owner?.mid;
|
||||
if (!cid || !upMid) {
|
||||
throw new CommandExecutionError(`Bilibili view API did not return cid/up_mid for ${bvid}`);
|
||||
}
|
||||
const conclusion = await apiGet(page, '/x/web-interface/view/conclusion/get', {
|
||||
params: { bvid, cid, up_mid: upMid },
|
||||
signed: true,
|
||||
});
|
||||
const conclusionData = requireOkPayload(conclusion, 'conclusion');
|
||||
return rowsFromModel(readModelResult(conclusionData, bvid));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
command,
|
||||
formatTime,
|
||||
readBvid,
|
||||
readModelResult,
|
||||
rowsFromModel,
|
||||
};
|
||||
@@ -0,0 +1,210 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const { mockApiGet, mockResolveBvid } = vi.hoisted(() => ({
|
||||
mockApiGet: vi.fn(),
|
||||
mockResolveBvid: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('./utils.js', async (importOriginal) => ({
|
||||
...(await importOriginal()),
|
||||
apiGet: mockApiGet,
|
||||
resolveBvid: mockResolveBvid,
|
||||
}));
|
||||
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './summary.js';
|
||||
|
||||
describe('bilibili summary', () => {
|
||||
const command = getRegistry().get('bilibili/summary');
|
||||
const page = {};
|
||||
|
||||
beforeEach(() => {
|
||||
mockApiGet.mockReset();
|
||||
mockResolveBvid.mockReset();
|
||||
mockResolveBvid.mockRejectedValue(new Error('short link not found'));
|
||||
});
|
||||
|
||||
function mockView(data = { aid: 114, cid: 222, owner: { mid: 333 } }) {
|
||||
mockApiGet.mockResolvedValueOnce({ code: 0, data });
|
||||
}
|
||||
|
||||
function mockConclusion(modelResult) {
|
||||
mockApiGet.mockResolvedValueOnce({
|
||||
code: 0,
|
||||
data: {
|
||||
code: 0,
|
||||
model_result: modelResult,
|
||||
},
|
||||
});
|
||||
}
|
||||
|
||||
it('returns the summary plus timestamped outline rows', async () => {
|
||||
mockView();
|
||||
mockConclusion({
|
||||
summary: '整体总结',
|
||||
outline: [
|
||||
{
|
||||
title: '第一节',
|
||||
timestamp: 0,
|
||||
part_outline: [
|
||||
{ timestamp: 12, content: '要点A' },
|
||||
{ timestamp: 3725, content: '要点B' },
|
||||
],
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
const result = await command.func(page, { bvid: 'BV1xxx' });
|
||||
|
||||
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1xxx' } });
|
||||
expect(mockApiGet).toHaveBeenNthCalledWith(2, page, '/x/web-interface/view/conclusion/get', {
|
||||
params: { bvid: 'BV1xxx', cid: 222, up_mid: 333 },
|
||||
signed: true,
|
||||
});
|
||||
expect(result).toEqual([
|
||||
{ time: '', content: '整体总结' },
|
||||
{ time: '00:00', content: '# 第一节' },
|
||||
{ time: '00:12', content: '要点A' },
|
||||
{ time: '1:02:05', content: '要点B' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('returns just the summary when the video has no outline', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion({ summary: '只有总结', outline: [] });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
|
||||
{ time: '', content: '只有总结' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('parses model_result when Bilibili returns it as a JSON string', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion(JSON.stringify({ summary: '字符串总结', outline: [] }));
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).resolves.toEqual([
|
||||
{ time: '', content: '字符串总结' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('normalizes Bilibili video URLs before calling the APIs', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion({ summary: 'URL 总结', outline: [] });
|
||||
|
||||
await command.func(page, {
|
||||
bvid: 'https://www.bilibili.com/video/BV1abc12345/?spm_id_from=333.1007',
|
||||
});
|
||||
|
||||
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BV1abc12345' } });
|
||||
});
|
||||
|
||||
it('resolves b23.tv short links through the shared resolver', async () => {
|
||||
mockResolveBvid.mockResolvedValueOnce('BVshort12345');
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion({ summary: '短链总结', outline: [] });
|
||||
|
||||
await command.func(page, { bvid: 'https://b23.tv/abc' });
|
||||
|
||||
expect(mockResolveBvid).toHaveBeenCalledWith('https://b23.tv/abc');
|
||||
expect(mockApiGet).toHaveBeenNthCalledWith(1, page, '/x/web-interface/view', { params: { bvid: 'BVshort12345' } });
|
||||
});
|
||||
|
||||
it('rejects invalid inputs before calling Bilibili APIs', async () => {
|
||||
const cases = [
|
||||
'',
|
||||
'javascript:alert(1)',
|
||||
'https://example.com/video/BV1abc12345',
|
||||
'https://share.note.youdao.com/video/BV1abc12345',
|
||||
'https://www.bilibili.com/read/cv12345',
|
||||
];
|
||||
|
||||
for (const bvid of cases) {
|
||||
await expect(command.func(page, { bvid })).rejects.toBeInstanceOf(ArgumentError);
|
||||
}
|
||||
expect(mockApiGet).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('maps unresolved short-code inputs to ArgumentError without calling APIs', async () => {
|
||||
await expect(command.func(page, { bvid: 'not-a-bv' })).rejects.toBeInstanceOf(ArgumentError);
|
||||
|
||||
expect(mockResolveBvid).toHaveBeenCalledWith('not-a-bv');
|
||||
expect(mockApiGet).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when Bilibili has not generated an AI summary for the video', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockApiGet.mockResolvedValueOnce({ code: 0, data: { code: 1, model_result: {} } });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the view payload is malformed', async () => {
|
||||
mockApiGet.mockResolvedValueOnce({ code: 0, data: {} });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /cid\/up_mid/.test(err.message),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the view API returns a non-auth error', async () => {
|
||||
mockApiGet.mockResolvedValueOnce({ code: -404, message: '啥都木有' });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BVbroken' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /啥都木有.*-404/.test(err.message),
|
||||
);
|
||||
});
|
||||
|
||||
it('maps conclusion auth or permission errors to AuthRequiredError', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockApiGet.mockResolvedValueOnce({ code: -403, message: '访问权限不足' });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('maps conclusion non-auth API errors to CommandExecutionError', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockApiGet.mockResolvedValueOnce({ code: -500, message: 'server error' });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /server error.*-500/.test(err.message),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for malformed conclusion API payloads', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockApiGet.mockResolvedValueOnce(null);
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for malformed model_result JSON', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion('{bad json');
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /model_result JSON/.test(err.message),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for malformed outline shapes', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion({ summary: '坏 outline', outline: {} });
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /outline/.test(err.message),
|
||||
);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for malformed part outline shapes', async () => {
|
||||
mockView({ aid: 1, cid: 2, owner: { mid: 3 } });
|
||||
mockConclusion({
|
||||
summary: '坏 part_outline',
|
||||
outline: [{ title: '段落', timestamp: 0, part_outline: {} }],
|
||||
});
|
||||
|
||||
await expect(command.func(page, { bvid: 'BV1xxx' })).rejects.toSatisfy(
|
||||
(err) => err instanceof CommandExecutionError && /part outline/.test(err.message),
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,356 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './search.js';
|
||||
import { __test__ } from './search.js';
|
||||
|
||||
const {
|
||||
normalizePositiveInt,
|
||||
normalizeNonNegativeInt,
|
||||
normalizeDate,
|
||||
normalizeCurrency,
|
||||
normalizeLang,
|
||||
hasPositiveResultCount,
|
||||
buildSearchUrl,
|
||||
} = __test__;
|
||||
|
||||
describe('booking helpers — normalizePositiveInt (no silent clamp)', () => {
|
||||
it('returns default when value is undefined/null/empty', () => {
|
||||
expect(normalizePositiveInt(undefined, 2, 'adults', 30)).toBe(2);
|
||||
expect(normalizePositiveInt(null, 2, 'adults', 30)).toBe(2);
|
||||
});
|
||||
|
||||
it('accepts integers in range', () => {
|
||||
expect(normalizePositiveInt(1, 2, 'adults', 30)).toBe(1);
|
||||
expect(normalizePositiveInt(30, 2, 'adults', 30)).toBe(30);
|
||||
});
|
||||
|
||||
it('rejects zero / negative / out-of-range / non-integer (no silent clamp)', () => {
|
||||
expect(() => normalizePositiveInt(0, 2, 'adults', 30)).toThrow(ArgumentError);
|
||||
expect(() => normalizePositiveInt(-1, 2, 'adults', 30)).toThrow(ArgumentError);
|
||||
expect(() => normalizePositiveInt(31, 2, 'adults', 30)).toThrow(ArgumentError);
|
||||
expect(() => normalizePositiveInt(1.5, 2, 'adults', 30)).toThrow(ArgumentError);
|
||||
expect(() => normalizePositiveInt('abc', 2, 'adults', 30)).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — normalizeNonNegativeInt', () => {
|
||||
it('accepts zero', () => {
|
||||
expect(normalizeNonNegativeInt(0, 0, 'children', 10)).toBe(0);
|
||||
});
|
||||
|
||||
it('rejects negative / out-of-range (no silent clamp)', () => {
|
||||
expect(() => normalizeNonNegativeInt(-1, 0, 'children', 10)).toThrow(ArgumentError);
|
||||
expect(() => normalizeNonNegativeInt(11, 0, 'children', 10)).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — normalizeDate', () => {
|
||||
it('accepts YYYY-MM-DD', () => {
|
||||
expect(normalizeDate('2026-06-15', 'checkin')).toBe('2026-06-15');
|
||||
});
|
||||
|
||||
it('rejects bad format / nonsense dates with ArgumentError', () => {
|
||||
expect(() => normalizeDate('', 'checkin')).toThrow(ArgumentError);
|
||||
expect(() => normalizeDate('06/15/2026', 'checkin')).toThrow(ArgumentError);
|
||||
expect(() => normalizeDate('2026-13-40', 'checkin')).toThrow(ArgumentError);
|
||||
expect(() => normalizeDate('2026-02-31', 'checkin')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — normalizeCurrency', () => {
|
||||
it('passes 3-letter codes uppercased', () => {
|
||||
expect(normalizeCurrency('usd')).toBe('USD');
|
||||
expect(normalizeCurrency('JPY')).toBe('JPY');
|
||||
});
|
||||
|
||||
it('returns empty for unset', () => {
|
||||
expect(normalizeCurrency(undefined)).toBe('');
|
||||
expect(normalizeCurrency('')).toBe('');
|
||||
});
|
||||
|
||||
it('rejects non-3-letter codes', () => {
|
||||
expect(() => normalizeCurrency('US')).toThrow(ArgumentError);
|
||||
expect(() => normalizeCurrency('US$')).toThrow(ArgumentError);
|
||||
expect(() => normalizeCurrency('USDX')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — normalizeLang whitelist', () => {
|
||||
it('lowercases supported langs', () => {
|
||||
expect(normalizeLang('EN-US')).toBe('en-us');
|
||||
expect(normalizeLang('zh-cn')).toBe('zh-cn');
|
||||
});
|
||||
|
||||
it('rejects unknown langs', () => {
|
||||
expect(() => normalizeLang('xx-yy')).toThrow(ArgumentError);
|
||||
expect(() => normalizeLang('en')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — buildSearchUrl', () => {
|
||||
it('constructs canonical search URL with required params', () => {
|
||||
const url = buildSearchUrl({
|
||||
destination: 'Tokyo',
|
||||
checkin: '2026-06-15',
|
||||
checkout: '2026-06-17',
|
||||
adults: 2,
|
||||
rooms: 1,
|
||||
children: 0,
|
||||
offset: 0,
|
||||
currency: 'USD',
|
||||
lang: 'en-us',
|
||||
});
|
||||
expect(url).toContain('https://www.booking.com/searchresults.en-us.html');
|
||||
expect(url).toContain('ss=Tokyo');
|
||||
expect(url).toContain('checkin=2026-06-15');
|
||||
expect(url).toContain('checkout=2026-06-17');
|
||||
expect(url).toContain('group_adults=2');
|
||||
expect(url).toContain('no_rooms=1');
|
||||
expect(url).toContain('group_children=0');
|
||||
expect(url).toContain('selected_currency=USD');
|
||||
expect(url).not.toContain('offset=');
|
||||
});
|
||||
|
||||
it('omits lang file segment when lang is empty', () => {
|
||||
const url = buildSearchUrl({
|
||||
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
|
||||
adults: 2, rooms: 1, children: 0, offset: 0, currency: '', lang: '',
|
||||
});
|
||||
expect(url).toMatch(/booking\.com\/searchresults\.html\?/);
|
||||
});
|
||||
|
||||
it('emits offset only when > 0', () => {
|
||||
const url = buildSearchUrl({
|
||||
destination: 'Paris', checkin: '2026-06-15', checkout: '2026-06-17',
|
||||
adults: 2, rooms: 1, children: 0, offset: 25, currency: '', lang: '',
|
||||
});
|
||||
expect(url).toContain('offset=25');
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking helpers — hasPositiveResultCount', () => {
|
||||
it('detects positive Booking result-count evidence', () => {
|
||||
expect(hasPositiveResultCount('Tokyo: 1,234 properties found')).toBe(true);
|
||||
expect(hasPositiveResultCount('1 stay found')).toBe(true);
|
||||
});
|
||||
|
||||
it('does not treat no-results text as positive evidence', () => {
|
||||
expect(hasPositiveResultCount('No properties found')).toBe(false);
|
||||
expect(hasPositiveResultCount('0 properties found')).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking adapter registry shape', () => {
|
||||
it('search is registered as read with id-shaped column for round-trip', () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
expect(search).toBeDefined();
|
||||
expect(search.access).toBe('read');
|
||||
expect(search.browser).toBe(true);
|
||||
// slug + country together form the round-trip identity (URL: /hotel/<country>/<slug>.html)
|
||||
expect(search.columns).toContain('slug');
|
||||
expect(search.columns).toContain('country');
|
||||
expect(search.columns).toContain('url');
|
||||
});
|
||||
|
||||
it('search columns stay <= 12 to honor agent-native row shape', () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
expect(search.columns.length).toBeLessThanOrEqual(12);
|
||||
});
|
||||
});
|
||||
|
||||
describe('booking search — typed errors (no silent fallback)', () => {
|
||||
const fakePage = { goto: () => { throw new Error('should not navigate'); } };
|
||||
|
||||
it('rejects empty destination with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: ' ', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects missing checkin/checkout with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo' })).rejects.toThrow(ArgumentError);
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15' })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects checkout <= checkin with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-17', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-15' })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects out-of-range --limit with ArgumentError (no silent clamp to 100)', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', limit: 999 })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects negative --offset with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: -1 })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects unsupported --lang with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', lang: 'xx-yy' })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects malformed --currency with ArgumentError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
await expect(search.func(fakePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'US$' })).rejects.toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('wraps browser navigation failures as CommandExecutionError', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const downPage = { goto: () => Promise.reject(new Error('browser down')) };
|
||||
await expect(search.func(downPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when extractor returns no cards', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const emptyPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'No properties found' }),
|
||||
};
|
||||
await expect(search.func(emptyPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when result-count evidence exists but no cards were parsed', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const driftPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({ ok: true, items: [], blocked: false, totalText: 'Tokyo: 1,234 properties found' }),
|
||||
};
|
||||
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when captcha is detected', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const blockedPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({ ok: true, items: [], blocked: true, totalText: 'Verify you are human' }),
|
||||
};
|
||||
await expect(search.func(blockedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when extractor payload is malformed instead of treating it as empty', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const malformedPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({ ok: true, blocked: false, totalText: 'Tokyo hotels' }),
|
||||
};
|
||||
await expect(search.func(malformedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when rendered cards lack stable hotel URL identity', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const driftPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({
|
||||
ok: true,
|
||||
blocked: false,
|
||||
totalText: 'Tokyo hotels',
|
||||
items: [{
|
||||
name: 'Unlinked Hotel',
|
||||
country: '',
|
||||
slug: '',
|
||||
url: '',
|
||||
distance: '',
|
||||
review_score: null,
|
||||
review_count: null,
|
||||
star_rating: null,
|
||||
price_currency: '',
|
||||
price_amount: null,
|
||||
recommended_room: '',
|
||||
}],
|
||||
}),
|
||||
};
|
||||
await expect(search.func(driftPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('unwraps {session, data} envelope from CDP bridge before validating', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const envelopePage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({
|
||||
session: 1,
|
||||
data: {
|
||||
ok: true,
|
||||
blocked: false,
|
||||
totalText: '',
|
||||
items: [{
|
||||
name: 'Test Hotel',
|
||||
country: 'jp',
|
||||
slug: 'test-hotel',
|
||||
url: 'https://www.booking.com/hotel/jp/test-hotel.html',
|
||||
distance: '1 km from centre',
|
||||
review_score: 8.6,
|
||||
review_count: 100,
|
||||
star_rating: 4,
|
||||
price_currency: 'USD',
|
||||
price_amount: 120,
|
||||
recommended_room: 'Standard double',
|
||||
}],
|
||||
},
|
||||
}),
|
||||
};
|
||||
const rows = await search.func(envelopePage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17' });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].rank).toBe(1);
|
||||
expect(rows[0].slug).toBe('test-hotel');
|
||||
expect(rows[0].url).toBe('https://www.booking.com/hotel/jp/test-hotel.html');
|
||||
});
|
||||
|
||||
it('uses requested selected_currency as the output source when price is present', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const currencyPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({
|
||||
ok: true,
|
||||
blocked: false,
|
||||
totalText: '',
|
||||
items: [{
|
||||
name: 'Currency Hotel',
|
||||
country: 'cn',
|
||||
slug: 'currency-hotel',
|
||||
url: 'https://www.booking.com/hotel/cn/currency-hotel.html',
|
||||
distance: '',
|
||||
review_score: null,
|
||||
review_count: null,
|
||||
star_rating: null,
|
||||
price_currency: 'JPY',
|
||||
price_amount: 880,
|
||||
recommended_room: '',
|
||||
}],
|
||||
}),
|
||||
};
|
||||
const rows = await search.func(currencyPage, { destination: 'Shanghai', checkin: '2026-06-15', checkout: '2026-06-17', currency: 'CNY' });
|
||||
expect(rows[0].price_currency).toBe('CNY');
|
||||
});
|
||||
|
||||
it('respects offset for rank numbering when paginating', async () => {
|
||||
const search = getRegistry().get('booking/search');
|
||||
const pagedPage = {
|
||||
goto: async () => {},
|
||||
wait: async () => {},
|
||||
evaluate: async () => ({
|
||||
ok: true,
|
||||
blocked: false,
|
||||
totalText: '',
|
||||
items: [
|
||||
{ name: 'A', country: 'jp', slug: 'a', url: 'https://www.booking.com/hotel/jp/a.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
|
||||
{ name: 'B', country: 'jp', slug: 'b', url: 'https://www.booking.com/hotel/jp/b.html', distance: '', review_score: null, review_count: null, star_rating: null, price_currency: '', price_amount: null, recommended_room: '' },
|
||||
],
|
||||
}),
|
||||
};
|
||||
const rows = await search.func(pagedPage, { destination: 'Tokyo', checkin: '2026-06-15', checkout: '2026-06-17', offset: 50 });
|
||||
expect(rows[0].rank).toBe(51);
|
||||
expect(rows[1].rank).toBe(52);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,351 @@
|
||||
import {
|
||||
ArgumentError,
|
||||
CommandExecutionError,
|
||||
EmptyResultError,
|
||||
} from '@jackwener/opencli/errors';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
|
||||
const DATE_RE = /^\d{4}-\d{2}-\d{2}$/;
|
||||
|
||||
function normalizePositiveInt(value, defaultValue, label, max) {
|
||||
const raw = value ?? defaultValue;
|
||||
const n = Number(raw);
|
||||
if (!Number.isInteger(n) || n <= 0) {
|
||||
throw new ArgumentError(`${label} must be a positive integer`);
|
||||
}
|
||||
if (typeof max === 'number' && n > max) {
|
||||
throw new ArgumentError(`${label} must be <= ${max}`);
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function normalizeNonNegativeInt(value, defaultValue, label, max) {
|
||||
const raw = value ?? defaultValue;
|
||||
const n = Number(raw);
|
||||
if (!Number.isInteger(n) || n < 0) {
|
||||
throw new ArgumentError(`${label} must be a non-negative integer`);
|
||||
}
|
||||
if (typeof max === 'number' && n > max) {
|
||||
throw new ArgumentError(`${label} must be <= ${max}`);
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function normalizeDate(value, label) {
|
||||
const v = String(value || '').trim();
|
||||
if (!v) {
|
||||
throw new ArgumentError(`${label} is required (YYYY-MM-DD)`);
|
||||
}
|
||||
if (!DATE_RE.test(v)) {
|
||||
throw new ArgumentError(`${label} must be YYYY-MM-DD, got ${JSON.stringify(value)}`);
|
||||
}
|
||||
const [year, month, day] = v.split('-').map(Number);
|
||||
const d = new Date(Date.UTC(year, month - 1, day));
|
||||
if (
|
||||
Number.isNaN(d.getTime()) ||
|
||||
d.getUTCFullYear() !== year ||
|
||||
d.getUTCMonth() !== month - 1 ||
|
||||
d.getUTCDate() !== day
|
||||
) {
|
||||
throw new ArgumentError(`${label} is not a valid calendar date: ${v}`);
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
function normalizeCurrency(value) {
|
||||
if (value == null || value === '') return '';
|
||||
const v = String(value).trim().toUpperCase();
|
||||
if (!/^[A-Z]{3}$/.test(v)) {
|
||||
throw new ArgumentError(`currency must be a 3-letter ISO code (e.g. USD, JPY, CNY), got ${JSON.stringify(value)}`);
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
const ALLOWED_LANGS = new Set([
|
||||
'en-us', 'en-gb', 'zh-cn', 'zh-tw', 'ja', 'ko', 'de', 'fr', 'es', 'it',
|
||||
'pt-br', 'pt-pt', 'ru', 'th', 'vi', 'tr', 'pl', 'nl', 'ar',
|
||||
]);
|
||||
|
||||
function normalizeLang(value) {
|
||||
if (value == null || value === '') return '';
|
||||
const v = String(value).trim().toLowerCase();
|
||||
if (!ALLOWED_LANGS.has(v)) {
|
||||
throw new ArgumentError(`lang must be one of: ${[...ALLOWED_LANGS].join(', ')}`);
|
||||
}
|
||||
return v;
|
||||
}
|
||||
|
||||
function hasPositiveResultCount(text) {
|
||||
const value = String(text || '').replace(/\u00a0/g, ' ');
|
||||
const resultCount = value.match(/\b([1-9][0-9,.\s]*)\s+(?:properties|property|stays|stay|hotels|hotel)\b/i);
|
||||
if (!resultCount) return false;
|
||||
const digits = resultCount[1].replace(/\D/g, '');
|
||||
return Boolean(digits) && Number(digits) > 0;
|
||||
}
|
||||
|
||||
function buildSearchUrl({
|
||||
destination,
|
||||
checkin,
|
||||
checkout,
|
||||
adults,
|
||||
rooms,
|
||||
children,
|
||||
offset,
|
||||
currency,
|
||||
lang,
|
||||
}) {
|
||||
const file = lang ? `searchresults.${lang}.html` : 'searchresults.html';
|
||||
const params = new URLSearchParams();
|
||||
params.set('ss', destination);
|
||||
params.set('checkin', checkin);
|
||||
params.set('checkout', checkout);
|
||||
params.set('group_adults', String(adults));
|
||||
params.set('no_rooms', String(rooms));
|
||||
params.set('group_children', String(children));
|
||||
if (offset > 0) params.set('offset', String(offset));
|
||||
if (currency) params.set('selected_currency', currency);
|
||||
return `https://www.booking.com/${file}?${params.toString()}`;
|
||||
}
|
||||
|
||||
const EXTRACTOR = `
|
||||
(() => {
|
||||
const trim = (v) => (v == null ? '' : String(v).replace(/\\s+/g, ' ').trim());
|
||||
const cards = Array.from(document.querySelectorAll('[data-testid=property-card]'));
|
||||
|
||||
// Detect blocking / captcha pages: no cards but body shows a verification prompt.
|
||||
if (cards.length === 0) {
|
||||
const text = [
|
||||
(document.title || ''),
|
||||
(document.body && document.body.innerText) || '',
|
||||
(location && location.pathname) || '',
|
||||
].join(' ');
|
||||
const blocked = /captcha|challenge|verify\\s*you\\s*are|access\\s*denied|forbidden|robot|unusual\\s*traffic/i.test(text);
|
||||
const totalEl = document.querySelector('h1');
|
||||
const totalText = trim(totalEl && totalEl.textContent);
|
||||
return { ok: true, items: [], blocked, totalText };
|
||||
}
|
||||
|
||||
const items = cards.map((card) => {
|
||||
const titleEl = card.querySelector('[data-testid=title]');
|
||||
const link = card.querySelector('a[data-testid=title-link]');
|
||||
const href = (link && link.href) || '';
|
||||
let country = '';
|
||||
let slug = '';
|
||||
let canonicalUrl = '';
|
||||
try {
|
||||
const u = new URL(href, 'https://www.booking.com');
|
||||
const m = u.pathname.match(/^\\/hotel\\/([a-z]{2})\\/([^./]+)/);
|
||||
if (m) {
|
||||
country = m[1];
|
||||
slug = m[2];
|
||||
canonicalUrl = 'https://www.booking.com/hotel/' + country + '/' + slug + '.html';
|
||||
}
|
||||
} catch (_) {}
|
||||
|
||||
const reviewTextRaw = trim(card.querySelector('[data-testid=review-score]')?.textContent);
|
||||
// Booking renders the score twice (a11y + visual), text reads like "Scored 8.6 8.6 Very Good 6,151 reviews"
|
||||
// or "评分8.68.6很棒 6,151条住客点评". Take only the first numeric occurrence.
|
||||
const scoreMatch = reviewTextRaw.match(/(\\d{1,2})\\.(\\d)/);
|
||||
const reviewScore = scoreMatch ? Number(scoreMatch[1] + '.' + scoreMatch[2]) : null;
|
||||
|
||||
const countMatch = reviewTextRaw.match(/([0-9][0-9,]*)\\s*(?:reviews|reseñas|avis|recensioni|条住客点评|条评论|レビュー|리뷰)/i);
|
||||
const reviewCount = countMatch ? Number(countMatch[1].replace(/,/g, '')) : null;
|
||||
|
||||
// Star rating: aria-label often "5 out of 5" / "4 星 (满分 5 星)" / "Hôtel 4 étoiles"
|
||||
let starRating = null;
|
||||
const starEl = card.querySelector('[data-testid=rating-stars], [data-testid=quality-rating]');
|
||||
if (starEl) {
|
||||
const aria = starEl.getAttribute('aria-label') || starEl.textContent || '';
|
||||
const m = aria.match(/(\\d)(?:\\s*(?:out of|\\/|星|颗星|stars?|étoiles?)|\\s*$)/i);
|
||||
if (m) starRating = Number(m[1]);
|
||||
if (starRating == null) {
|
||||
const count = starEl.querySelectorAll('svg, [aria-hidden=true]').length;
|
||||
if (count >= 1 && count <= 5) starRating = count;
|
||||
}
|
||||
}
|
||||
|
||||
const priceEl = card.querySelector('[data-testid=price-and-discounted-price]');
|
||||
const priceText = trim(priceEl && priceEl.textContent);
|
||||
|
||||
// currency symbol → ISO best-effort
|
||||
const currencySymbolMap = {
|
||||
'$': 'USD', 'US$': 'USD', 'A$': 'AUD', 'C$': 'CAD', 'HK$': 'HKD',
|
||||
'€': 'EUR', '£': 'GBP', '¥': 'JPY', '¥': 'CNY', '₹': 'INR', '₩': 'KRW',
|
||||
'CN¥': 'CNY', 'CN¥': 'CNY', 'NT$': 'TWD', 'S$': 'SGD',
|
||||
};
|
||||
let priceCurrency = '';
|
||||
let priceAmount = null;
|
||||
const sym = priceText.match(/(US\\$|A\\$|C\\$|HK\\$|NT\\$|S\\$|CN¥|CN¥|[$€£¥¥₹₩])/);
|
||||
if (sym) priceCurrency = currencySymbolMap[sym[1]] || '';
|
||||
const num = priceText.replace(/,/g, '').match(/(\\d+(?:\\.\\d+)?)/);
|
||||
if (num) priceAmount = Number(num[1]);
|
||||
|
||||
return {
|
||||
name: trim(titleEl?.textContent),
|
||||
country,
|
||||
slug,
|
||||
url: canonicalUrl,
|
||||
distance: trim(card.querySelector('[data-testid=distance]')?.textContent),
|
||||
review_score: reviewScore,
|
||||
review_count: reviewCount,
|
||||
star_rating: starRating,
|
||||
price_currency: priceCurrency,
|
||||
price_amount: priceAmount,
|
||||
recommended_room: trim(card.querySelector('[data-testid=recommended-units]')?.textContent),
|
||||
};
|
||||
});
|
||||
|
||||
const totalEl = document.querySelector('h1');
|
||||
const totalText = trim(totalEl && totalEl.textContent);
|
||||
return { ok: true, items, blocked: false, totalText };
|
||||
})()
|
||||
`;
|
||||
|
||||
cli({
|
||||
site: 'booking',
|
||||
name: 'search',
|
||||
description: 'Search Booking.com hotels by destination and dates (server-rendered card scrape).',
|
||||
access: 'read',
|
||||
example: 'opencli booking search Tokyo --checkin 2026-06-15 --checkout 2026-06-17 -f yaml',
|
||||
domain: 'www.booking.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'destination', required: true, positional: true, help: 'Destination keyword (city, district, or hotel name)' },
|
||||
{ name: 'checkin', required: true, help: 'Check-in date YYYY-MM-DD' },
|
||||
{ name: 'checkout', required: true, help: 'Check-out date YYYY-MM-DD' },
|
||||
{ name: 'adults', type: 'int', default: 2, help: 'Number of adults (1-30)' },
|
||||
{ name: 'rooms', type: 'int', default: 1, help: 'Number of rooms (1-30)' },
|
||||
{ name: 'children', type: 'int', default: 0, help: 'Number of children (0-10)' },
|
||||
{ name: 'currency', required: false, help: 'Force result currency (e.g. USD, JPY, CNY)' },
|
||||
{ name: 'lang', required: false, help: 'Force result language (e.g. en-us, zh-cn, ja)' },
|
||||
{ name: 'limit', type: 'int', default: 25, help: 'Max rows to return (1-100; Booking pages 25 per request)' },
|
||||
{ name: 'offset', type: 'int', default: 0, help: 'Result offset for pagination (multiple of 25)' },
|
||||
],
|
||||
columns: [
|
||||
'rank',
|
||||
'name',
|
||||
'country',
|
||||
'slug',
|
||||
'star_rating',
|
||||
'review_score',
|
||||
'review_count',
|
||||
'price_amount',
|
||||
'price_currency',
|
||||
'distance',
|
||||
'recommended_room',
|
||||
'url',
|
||||
],
|
||||
func: async (page, kwargs) => {
|
||||
const destination = String(kwargs.destination || '').trim();
|
||||
if (!destination) throw new ArgumentError('destination is required');
|
||||
const checkin = normalizeDate(kwargs.checkin, 'checkin');
|
||||
const checkout = normalizeDate(kwargs.checkout, 'checkout');
|
||||
if (checkin >= checkout) {
|
||||
throw new ArgumentError(`checkout (${checkout}) must be after checkin (${checkin})`);
|
||||
}
|
||||
const adults = normalizePositiveInt(kwargs.adults, 2, 'adults', 30);
|
||||
const rooms = normalizePositiveInt(kwargs.rooms, 1, 'rooms', 30);
|
||||
const children = normalizeNonNegativeInt(kwargs.children, 0, 'children', 10);
|
||||
const currency = normalizeCurrency(kwargs.currency);
|
||||
const lang = normalizeLang(kwargs.lang);
|
||||
const limit = normalizePositiveInt(kwargs.limit, 25, 'limit', 100);
|
||||
const offset = normalizeNonNegativeInt(kwargs.offset, 0, 'offset', 1000);
|
||||
|
||||
const url = buildSearchUrl({ destination, checkin, checkout, adults, rooms, children, offset, currency, lang });
|
||||
|
||||
try {
|
||||
await page.goto(url);
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`Failed to load Booking.com search page: ${err?.message || err}`);
|
||||
}
|
||||
|
||||
// Booking lazy-loads price cells; wait for at least the first card price to settle.
|
||||
try {
|
||||
await page.wait('selector', '[data-testid=property-card]', { timeoutMs: 20000 });
|
||||
} catch (_) {
|
||||
// selector wait is best-effort — extractor handles empty case explicitly
|
||||
}
|
||||
|
||||
let raw;
|
||||
try {
|
||||
raw = await page.evaluate(EXTRACTOR);
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`Failed to extract Booking.com cards: ${err?.message || err}`);
|
||||
}
|
||||
|
||||
if (raw && typeof raw === 'object' && raw.data && raw.session) {
|
||||
raw = raw.data;
|
||||
}
|
||||
if (!raw || typeof raw !== 'object') {
|
||||
throw new CommandExecutionError('Booking.com page returned no extractable data');
|
||||
}
|
||||
if (raw.blocked) {
|
||||
throw new CommandExecutionError('Booking.com served a verification / captcha page; retry later or change profile');
|
||||
}
|
||||
|
||||
if (raw.ok !== true) {
|
||||
throw new CommandExecutionError('Booking.com extractor returned an invalid status');
|
||||
}
|
||||
if (!Array.isArray(raw.items)) {
|
||||
throw new CommandExecutionError('Booking.com extractor returned malformed items');
|
||||
}
|
||||
|
||||
const items = raw.items;
|
||||
if (items.length === 0) {
|
||||
const totalText = String(raw.totalText || '').trim();
|
||||
if (hasPositiveResultCount(totalText)) {
|
||||
throw new CommandExecutionError(
|
||||
`Booking.com page declared results but no property cards were parsed: ${totalText}`,
|
||||
);
|
||||
}
|
||||
throw new EmptyResultError(
|
||||
`booking search ${JSON.stringify(destination)}`,
|
||||
totalText
|
||||
? `No hotels rendered (${totalText}). Try a broader destination, different dates, or check the URL in a browser.`
|
||||
: 'No hotels rendered. Try a broader destination, different dates, or check the URL in a browser.',
|
||||
);
|
||||
}
|
||||
|
||||
return items.slice(0, limit).map((it, i) => {
|
||||
if (!it || typeof it !== 'object') {
|
||||
throw new CommandExecutionError('Booking.com extractor returned malformed hotel row');
|
||||
}
|
||||
const name = String(it.name || '').trim();
|
||||
const country = String(it.country || '').trim();
|
||||
const slug = String(it.slug || '').trim();
|
||||
const urlValue = String(it.url || '').trim();
|
||||
const expectedUrl = country && slug
|
||||
? `https://www.booking.com/hotel/${country}/${slug}.html`
|
||||
: '';
|
||||
if (!name || !/^[a-z]{2}$/.test(country) || !slug || urlValue !== expectedUrl) {
|
||||
throw new CommandExecutionError('Booking.com hotel row is missing stable name/url identity');
|
||||
}
|
||||
return {
|
||||
rank: offset + i + 1,
|
||||
name,
|
||||
country,
|
||||
slug,
|
||||
star_rating: it.star_rating,
|
||||
review_score: it.review_score,
|
||||
review_count: it.review_count,
|
||||
price_amount: it.price_amount,
|
||||
price_currency: it.price_amount == null ? '' : (currency || it.price_currency || ''),
|
||||
distance: it.distance,
|
||||
recommended_room: it.recommended_room,
|
||||
url: urlValue,
|
||||
};
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizePositiveInt,
|
||||
normalizeNonNegativeInt,
|
||||
normalizeDate,
|
||||
normalizeCurrency,
|
||||
normalizeLang,
|
||||
hasPositiveResultCount,
|
||||
buildSearchUrl,
|
||||
EXTRACTOR,
|
||||
};
|
||||
+96
-14
@@ -1,10 +1,60 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { requirePage, navigateToChat, fetchFriendList } from './utils.js';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import {
|
||||
requirePage, navigateToChat, navigateToGeekChat,
|
||||
fetchFriendList, fetchGeekFriendLabelList, fetchGeekFriendInfoList,
|
||||
readEncryptSystemId, assertOk, IDENTITY_MISMATCH_CODE,
|
||||
readPositiveInteger,
|
||||
} from './utils.js';
|
||||
|
||||
function formatMsgTime(ms) {
|
||||
if (!ms) return '';
|
||||
return new Date(ms).toLocaleString('zh-CN');
|
||||
}
|
||||
|
||||
function mapBossRow(f) {
|
||||
return {
|
||||
name: f.name || '',
|
||||
company: '',
|
||||
job: f.jobName || '',
|
||||
title: '',
|
||||
last_msg: f.lastMessageInfo?.text || '',
|
||||
last_time: f.lastTime || '',
|
||||
uid: f.encryptUid || '',
|
||||
security_id: f.securityId || '',
|
||||
};
|
||||
}
|
||||
|
||||
async function buildGeekRows(page, limit) {
|
||||
const encryptSystemId = await readEncryptSystemId(page);
|
||||
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId });
|
||||
if (labelList.length === 0) {
|
||||
return [];
|
||||
}
|
||||
const slicedLabels = labelList.slice(0, limit);
|
||||
const friendIds = slicedLabels.map((f) => f.friendId).filter(Boolean);
|
||||
const enriched = await fetchGeekFriendInfoList(page, friendIds);
|
||||
const enrichMap = new Map(enriched.map((f) => [String(f.friendId ?? f.uid), f]));
|
||||
return slicedLabels.map((f) => {
|
||||
const e = enrichMap.get(String(f.friendId)) || {};
|
||||
return {
|
||||
name: e.name || f.name || '',
|
||||
company: e.brandName || f.brandName || '',
|
||||
job: e.jobName || f.jobName || '',
|
||||
title: e.bossTitle || f.bossTitle || '',
|
||||
last_msg: e.lastMessageInfo?.showText || e.lastMsg || f.lastMsg || '',
|
||||
last_time: e.lastTime || formatMsgTime(e.lastMessageInfo?.msgTime) || formatMsgTime(f.updateTime) || '',
|
||||
uid: e.encryptUid || f.encryptFriendId || String(e.uid ?? e.friendId ?? f.friendId ?? ''),
|
||||
security_id: e.securityId || '',
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'boss',
|
||||
name: 'chatlist',
|
||||
access: 'read',
|
||||
description: 'BOSS直聘查看聊天列表(招聘端)',
|
||||
description: 'BOSS直聘查看聊天列表(招聘端/求职端)',
|
||||
domain: 'www.zhipin.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
navigateBefore: false,
|
||||
@@ -12,23 +62,55 @@ cli({
|
||||
args: [
|
||||
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of results' },
|
||||
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all)' },
|
||||
{ name: 'job-id', default: '0', help: 'Filter by job ID (0=all, boss side only)' },
|
||||
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
|
||||
],
|
||||
columns: ['name', 'job', 'last_msg', 'last_time', 'uid', 'security_id'],
|
||||
columns: ['name', 'company', 'job', 'title', 'last_msg', 'last_time', 'uid', 'security_id'],
|
||||
func: async (page, kwargs) => {
|
||||
requirePage(page);
|
||||
const limit = readPositiveInteger(kwargs.limit, 'chatlist --limit', 20, 100);
|
||||
const pageNum = readPositiveInteger(kwargs.page, 'chatlist --page', 1);
|
||||
const side = kwargs.side || 'auto';
|
||||
|
||||
if (side === 'boss') {
|
||||
await navigateToChat(page);
|
||||
const friends = await fetchFriendList(page, {
|
||||
pageNum,
|
||||
jobId: kwargs['job-id'] || '0',
|
||||
});
|
||||
if (friends.length === 0)
|
||||
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
|
||||
return friends.slice(0, limit).map(mapBossRow);
|
||||
}
|
||||
|
||||
if (side === 'geek') {
|
||||
await navigateToGeekChat(page);
|
||||
const rows = await buildGeekRows(page, limit);
|
||||
if (rows.length === 0)
|
||||
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
|
||||
return rows;
|
||||
}
|
||||
|
||||
// auto: try recruiter first, fall back to geek on identity mismatch
|
||||
await navigateToChat(page);
|
||||
const friends = await fetchFriendList(page, {
|
||||
pageNum: kwargs.page || 1,
|
||||
const bossResult = await fetchFriendList(page, {
|
||||
pageNum,
|
||||
jobId: kwargs['job-id'] || '0',
|
||||
allowNonZero: true,
|
||||
});
|
||||
return friends.slice(0, kwargs.limit || 20).map((f) => ({
|
||||
name: f.name || '',
|
||||
job: f.jobName || '',
|
||||
last_msg: f.lastMessageInfo?.text || '',
|
||||
last_time: f.lastTime || '',
|
||||
uid: f.encryptUid || '',
|
||||
security_id: f.securityId || '',
|
||||
}));
|
||||
if (Array.isArray(bossResult)) {
|
||||
if (bossResult.length === 0)
|
||||
throw new EmptyResultError('boss chatlist', 'No recruiter-side chat sessions were returned.');
|
||||
return bossResult.slice(0, limit).map(mapBossRow);
|
||||
}
|
||||
if (bossResult.code === IDENTITY_MISMATCH_CODE) {
|
||||
await navigateToGeekChat(page);
|
||||
const rows = await buildGeekRows(page, limit);
|
||||
if (rows.length === 0)
|
||||
throw new EmptyResultError('boss chatlist', 'No job-seeker-side chat sessions were returned.');
|
||||
return rows;
|
||||
}
|
||||
assertOk(bossResult);
|
||||
throw new CommandExecutionError('Boss chatlist returned an unexpected response');
|
||||
},
|
||||
});
|
||||
|
||||
@@ -0,0 +1,211 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './chatlist.js';
|
||||
|
||||
const BOSS_FRIEND = {
|
||||
name: '张三',
|
||||
jobName: '后端工程师',
|
||||
lastMessageInfo: { text: '你好' },
|
||||
lastTime: '2024-01-01 10:00',
|
||||
encryptUid: 'enc-boss-uid',
|
||||
securityId: 'boss-sec-id',
|
||||
};
|
||||
|
||||
const GEEK_LABEL_FRIEND = {
|
||||
friendId: 12345,
|
||||
name: '李四',
|
||||
brandName: '字节跳动',
|
||||
jobName: '产品经理',
|
||||
bossTitle: 'HR',
|
||||
lastMsg: '感谢投递',
|
||||
updateTime: 1704067200000,
|
||||
encryptFriendId: 'enc-geek-uid',
|
||||
};
|
||||
|
||||
const GEEK_ENRICHED = {
|
||||
friendId: 12345,
|
||||
uid: 99999,
|
||||
name: '李四',
|
||||
brandName: '字节跳动',
|
||||
jobName: '产品经理',
|
||||
bossTitle: 'HR总监',
|
||||
encryptUid: 'enc-geek-uid',
|
||||
securityId: 'geek-sec-id',
|
||||
lastMessageInfo: { showText: '感谢投递', msgTime: 1704067200000 },
|
||||
lastTime: '2024-01-01',
|
||||
};
|
||||
|
||||
function createPageMock(evaluateImpl) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockImplementation(evaluateImpl),
|
||||
};
|
||||
}
|
||||
|
||||
describe('boss chatlist', () => {
|
||||
const command = getRegistry().get('boss/chatlist');
|
||||
|
||||
it('--side boss preserves existing behavior with 8-column output', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' });
|
||||
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
name: '张三',
|
||||
company: '',
|
||||
job: '后端工程师',
|
||||
title: '',
|
||||
last_msg: '你好',
|
||||
uid: 'enc-boss-uid',
|
||||
security_id: 'boss-sec-id',
|
||||
});
|
||||
});
|
||||
|
||||
it('--side geek maps enriched getGeekFriendList data into 8 columns', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
|
||||
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
name: '李四',
|
||||
company: '字节跳动',
|
||||
job: '产品经理',
|
||||
title: 'HR总监',
|
||||
uid: 'enc-geek-uid',
|
||||
security_id: 'geek-sec-id',
|
||||
});
|
||||
});
|
||||
|
||||
it('--side geek falls back to label fields when enrichment has no match', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].name).toBe('李四');
|
||||
expect(rows[0].company).toBe('字节跳动');
|
||||
expect(rows[0].security_id).toBe('');
|
||||
});
|
||||
|
||||
it('rejects invalid --limit before navigating', async () => {
|
||||
const page = createPageMock(async () => ({}));
|
||||
await expect(
|
||||
command.func(page, { page: 1, limit: 0, 'job-id': '0', side: 'geek' })
|
||||
).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('--side geek reports a true empty chat list as EmptyResultError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
|
||||
).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('treats malformed geek enrichment payload as CommandExecutionError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: {} };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'geek' })
|
||||
).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('treats null Boss API payload as CommandExecutionError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) return null;
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
|
||||
).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('maps expired Boss cookies to AuthRequiredError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 7, message: 'Cookie 已过期' };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'boss' })
|
||||
).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('--side auto falls back to geek when recruiter returns code 24', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 24, message: '请切换身份后再试' };
|
||||
}
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_LABEL_FRIEND] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_ENRICHED] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].company).toBe('字节跳动');
|
||||
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/geek/chat'));
|
||||
});
|
||||
|
||||
it('--side auto uses recruiter results when code 0 and does not call geek API', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { page: 1, limit: 20, 'job-id': '0', side: 'auto' });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].name).toBe('张三');
|
||||
const evaluateCalls = page.evaluate.mock.calls.map((c) => c[0]);
|
||||
expect(evaluateCalls.some((s) => s.includes('geekFilterByLabel'))).toBe(false);
|
||||
});
|
||||
|
||||
it('registers --side as a choices-constrained arg defaulting to auto', () => {
|
||||
const sideArg = command.args.find((a) => a.name === 'side');
|
||||
expect(sideArg?.choices).toEqual(['auto', 'boss', 'geek']);
|
||||
expect(sideArg?.default).toBe('auto');
|
||||
});
|
||||
});
|
||||
+98
-24
@@ -1,10 +1,72 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { requirePage, navigateToChat, bossFetch, findFriendByUid } from './utils.js';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import {
|
||||
requirePage, navigateToChat, navigateToGeekChat,
|
||||
bossFetch, findFriendByUid, findGeekFriendByUid,
|
||||
fetchGeekHistoryMsg, readEncryptSystemId,
|
||||
assertOk, IDENTITY_MISMATCH_CODE,
|
||||
readPositiveInteger, readRequiredString,
|
||||
} from './utils.js';
|
||||
|
||||
const TYPE_MAP = {
|
||||
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
|
||||
6: '名片', 7: '语音', 8: '视频', 9: '表情',
|
||||
};
|
||||
|
||||
function mapBossMsg(m, friend) {
|
||||
const fromObj = m.from || {};
|
||||
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
|
||||
return {
|
||||
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
|
||||
type: TYPE_MAP[m.type] || `其他(${m.type})`,
|
||||
text: m.text || m.body?.text || '',
|
||||
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
|
||||
};
|
||||
}
|
||||
|
||||
function mapGeekMsg(m, friend) {
|
||||
const fromUid = m.from && m.from.uid;
|
||||
const isFromBoss = fromUid != null && String(fromUid) === String(friend.uid);
|
||||
return {
|
||||
from: isFromBoss ? '对方' : '我',
|
||||
type: TYPE_MAP[m.type] || `其他(${m.type})`,
|
||||
text: m.text || m.body?.text || m.body?.content || m.body?.showText ||
|
||||
JSON.stringify(m.body || {}).slice(0, 120),
|
||||
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
|
||||
};
|
||||
}
|
||||
|
||||
async function bossChatMsg(page, kwargs, existingFriend) {
|
||||
const friend = existingFriend ?? await findFriendByUid(page, kwargs.uid);
|
||||
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该候选人');
|
||||
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
|
||||
const gid = friend.uid;
|
||||
const securityId = encodeURIComponent(friend.securityId);
|
||||
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
|
||||
const msgData = await bossFetch(page, msgUrl);
|
||||
const messages = msgData.zpData?.messages ?? msgData.zpData?.historyMsgList;
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new CommandExecutionError('Boss recruiter history response did not include a message list');
|
||||
}
|
||||
if (messages.length === 0) {
|
||||
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
|
||||
}
|
||||
return messages.map((m) => mapBossMsg(m, friend));
|
||||
}
|
||||
|
||||
async function geekChatMsg(page, kwargs, encryptSystemId) {
|
||||
const friend = await findGeekFriendByUid(page, kwargs.uid, { encryptSystemId });
|
||||
if (!friend) throw new EmptyResultError('boss chatmsg', '未找到该聊天(geek 侧)');
|
||||
if (!friend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
|
||||
const messages = await fetchGeekHistoryMsg(page, friend, { page: kwargs.page });
|
||||
return messages.map((m) => mapGeekMsg(m, friend));
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'boss',
|
||||
name: 'chatmsg',
|
||||
access: 'read',
|
||||
description: 'BOSS直聘查看与候选人的聊天消息',
|
||||
description: 'BOSS直聘查看聊天消息历史(招聘端/求职端)',
|
||||
domain: 'www.zhipin.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
navigateBefore: false,
|
||||
@@ -12,32 +74,44 @@ cli({
|
||||
args: [
|
||||
{ name: 'uid', required: true, positional: true, help: 'Encrypted UID (from chatlist)' },
|
||||
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
|
||||
{ name: 'side', default: 'auto', choices: ['auto', 'boss', 'geek'], help: 'Identity side: auto (default), boss (recruiter), or geek (job-seeker)' },
|
||||
],
|
||||
columns: ['from', 'type', 'text', 'time'],
|
||||
func: async (page, kwargs) => {
|
||||
requirePage(page);
|
||||
const uid = readRequiredString(kwargs.uid, 'chatmsg uid');
|
||||
const pageNum = readPositiveInteger(kwargs.page, 'chatmsg --page', 1);
|
||||
const normalizedKwargs = { ...kwargs, uid, page: pageNum };
|
||||
const side = kwargs.side || 'auto';
|
||||
|
||||
if (side === 'boss') {
|
||||
await navigateToChat(page);
|
||||
return await bossChatMsg(page, normalizedKwargs);
|
||||
}
|
||||
|
||||
if (side === 'geek') {
|
||||
await navigateToGeekChat(page);
|
||||
const encryptSystemId = await readEncryptSystemId(page);
|
||||
return await geekChatMsg(page, normalizedKwargs, encryptSystemId);
|
||||
}
|
||||
|
||||
// auto: try recruiter first, fall back to geek when not found or identity mismatch
|
||||
await navigateToChat(page);
|
||||
const friend = await findFriendByUid(page, kwargs.uid);
|
||||
if (!friend)
|
||||
throw new Error('未找到该候选人');
|
||||
const gid = friend.uid;
|
||||
const securityId = encodeURIComponent(friend.securityId);
|
||||
const msgUrl = `https://www.zhipin.com/wapi/zpchat/boss/historyMsg?gid=${gid}&securityId=${securityId}&page=${kwargs.page}&c=20&src=0`;
|
||||
const msgData = await bossFetch(page, msgUrl);
|
||||
const TYPE_MAP = {
|
||||
1: '文本', 2: '图片', 3: '招呼', 4: '简历', 5: '系统',
|
||||
6: '名片', 7: '语音', 8: '视频', 9: '表情',
|
||||
};
|
||||
const messages = msgData.zpData?.messages || msgData.zpData?.historyMsgList || [];
|
||||
return messages.map((m) => {
|
||||
const fromObj = m.from || {};
|
||||
const isSelf = typeof fromObj === 'object' ? fromObj.uid !== friend.uid : false;
|
||||
return {
|
||||
from: isSelf ? '我' : (typeof fromObj === 'object' ? fromObj.name : friend.name),
|
||||
type: TYPE_MAP[m.type] || '其他(' + m.type + ')',
|
||||
text: m.text || m.body?.text || '',
|
||||
time: m.time ? new Date(m.time).toLocaleString('zh-CN') : '',
|
||||
};
|
||||
});
|
||||
const bossResult = await findFriendByUid(page, uid, { allowNonZero: true });
|
||||
if (bossResult?.friend) {
|
||||
return await bossChatMsg(page, normalizedKwargs, bossResult.friend);
|
||||
}
|
||||
// Not found or identity mismatch — check for hard errors before falling back
|
||||
if (bossResult?.code && bossResult.code !== 0 && bossResult.code !== IDENTITY_MISMATCH_CODE) {
|
||||
assertOk(bossResult);
|
||||
}
|
||||
// Fall back to geek side
|
||||
await navigateToGeekChat(page);
|
||||
const encryptSystemId = await readEncryptSystemId(page);
|
||||
const geekFriend = await findGeekFriendByUid(page, uid, { encryptSystemId });
|
||||
if (!geekFriend) throw new EmptyResultError('boss chatmsg', 'uid 在招聘端与求职端聊天列表中均未找到');
|
||||
if (!geekFriend.securityId) throw new CommandExecutionError('该聊天缺少 securityId,无法获取历史消息');
|
||||
const messages = await fetchGeekHistoryMsg(page, geekFriend, { page: pageNum });
|
||||
return messages.map((m) => mapGeekMsg(m, geekFriend));
|
||||
},
|
||||
});
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './chatmsg.js';
|
||||
|
||||
const BOSS_FRIEND = {
|
||||
uid: 12345,
|
||||
encryptUid: 'enc-boss-uid',
|
||||
securityId: 'boss-sec-id',
|
||||
name: '候选人甲',
|
||||
};
|
||||
const BOSS_MSGS = [
|
||||
{ type: 1, text: 'Hello', from: { uid: 99999, name: 'HR' }, time: 1704067200000 },
|
||||
{ type: 1, text: '感谢', from: { uid: 12345, name: '候选人甲' }, time: 1704067201000 },
|
||||
];
|
||||
|
||||
const GEEK_FRIEND_LABEL = {
|
||||
friendId: 11111,
|
||||
encryptFriendId: 'enc-geek-uid',
|
||||
name: 'Boss张',
|
||||
brandName: '公司A',
|
||||
};
|
||||
const GEEK_FRIEND_ENRICHED = {
|
||||
friendId: 11111,
|
||||
uid: 67890,
|
||||
encryptUid: 'enc-geek-uid',
|
||||
securityId: 'geek-sec-id',
|
||||
name: 'Boss张',
|
||||
};
|
||||
const GEEK_MSGS = [
|
||||
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
|
||||
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
|
||||
];
|
||||
|
||||
function createPageMock(evaluateImpl) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockImplementation(evaluateImpl),
|
||||
};
|
||||
}
|
||||
|
||||
describe('boss chatmsg', () => {
|
||||
const command = getRegistry().get('boss/chatmsg');
|
||||
|
||||
it('rejects empty uid before navigating', async () => {
|
||||
const page = createPageMock(async () => ({}));
|
||||
await expect(
|
||||
command.func(page, { uid: ' ', page: 1, side: 'geek' })
|
||||
).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects invalid --page before navigating', async () => {
|
||||
const page = createPageMock(async () => ({}));
|
||||
await expect(
|
||||
command.func(page, { uid: 'enc-geek-uid', page: 0, side: 'geek' })
|
||||
).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('--side boss preserves existing behavior', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
|
||||
}
|
||||
if (script.includes('boss/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: BOSS_MSGS } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' });
|
||||
expect(page.goto).toHaveBeenCalledWith(expect.stringContaining('/web/chat/index'));
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows[0].from).toBe('我');
|
||||
expect(rows[1].from).toBe('候选人甲');
|
||||
});
|
||||
|
||||
it('--side geek calls historyMsg with bossId, securityId, page, c=20, src=0', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
|
||||
}
|
||||
if (script.includes('geek/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: GEEK_MSGS } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
|
||||
const historyScript = page.evaluate.mock.calls.find((c) => c[0].includes('geek/historyMsg'))?.[0];
|
||||
expect(historyScript).toBeDefined();
|
||||
expect(historyScript).toContain('bossId=67890');
|
||||
expect(historyScript).toContain('securityId=');
|
||||
expect(historyScript).toContain('page=1');
|
||||
expect(historyScript).toContain('c=20');
|
||||
expect(historyScript).toContain('src=0');
|
||||
});
|
||||
|
||||
it('--side geek uses from.uid to determine direction, not received flag', async () => {
|
||||
// Both messages have received:true (mirrors real geek historyMsg API behaviour)
|
||||
// Direction is determined by whether m.from.uid matches the boss's uid (67890)
|
||||
const msgsAllReceived = [
|
||||
{ type: 1, text: '欢迎投递', received: true, time: 1704067200000, from: { uid: 67890, name: 'Boss张' } },
|
||||
{ type: 1, text: '谢谢', received: true, time: 1704067201000, from: { uid: 99999, name: '我' } },
|
||||
];
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
|
||||
}
|
||||
if (script.includes('geek/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: msgsAllReceived } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
|
||||
// from.uid=67890 matches friend.uid=67890 → boss sent it → '对方'
|
||||
expect(rows[0].from).toBe('对方');
|
||||
// from.uid=99999 does not match → geek sent it → '我'
|
||||
expect(rows[1].from).toBe('我');
|
||||
});
|
||||
|
||||
it('non-text message body does not crash and produces truncated JSON', async () => {
|
||||
const nonTextMsg = { type: 99, received: true, time: 1704067200000, body: { action: 'resume_request', detail: 'X' } };
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
|
||||
}
|
||||
if (script.includes('geek/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: [nonTextMsg] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].text).toContain('resume_request');
|
||||
});
|
||||
|
||||
it('--side auto falls back to geek when recruiter returns code 24', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 24, message: '请切换身份后再试' };
|
||||
}
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
|
||||
}
|
||||
if (script.includes('geek/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: GEEK_MSGS } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
const rows = await command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'auto' });
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows[0].from).toBe('对方');
|
||||
});
|
||||
|
||||
it('--side geek throws when uid is not found in geek chat list', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { uid: 'unknown-uid', page: 1, side: 'geek' })
|
||||
).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('--side boss maps expired cookies to AuthRequiredError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 7, message: 'Cookie 已过期' };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
|
||||
).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('--side boss treats missing history list as parser drift', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('getBossFriendListV2')) {
|
||||
return { code: 0, zpData: { friendList: [BOSS_FRIEND] } };
|
||||
}
|
||||
if (script.includes('boss/historyMsg')) {
|
||||
return { code: 0, zpData: {} };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { uid: 'enc-boss-uid', page: 1, side: 'boss' })
|
||||
).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('--side geek reports an empty history as EmptyResultError', async () => {
|
||||
const page = createPageMock(async (script) => {
|
||||
if (script.includes('document.cookie')) return 'test-enc-sys-id';
|
||||
if (script.includes('geekFilterByLabel')) {
|
||||
return { code: 0, zpData: { friendList: [GEEK_FRIEND_LABEL] } };
|
||||
}
|
||||
if (script.includes('getGeekFriendList.json')) {
|
||||
return { code: 0, zpData: { result: [GEEK_FRIEND_ENRICHED] } };
|
||||
}
|
||||
if (script.includes('geek/historyMsg')) {
|
||||
return { code: 0, zpData: { messages: [] } };
|
||||
}
|
||||
return {};
|
||||
});
|
||||
await expect(
|
||||
command.func(page, { uid: 'enc-geek-uid', page: 1, side: 'geek' })
|
||||
).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
+257
-12
@@ -1,8 +1,11 @@
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
// ── Constants ───────────────────────────────────────────────────────────────
|
||||
const BOSS_DOMAIN = 'www.zhipin.com';
|
||||
const CHAT_URL = `https://${BOSS_DOMAIN}/web/chat/index`;
|
||||
const COOKIE_EXPIRED_CODES = new Set([7, 37]);
|
||||
const COOKIE_EXPIRED_MSG = 'Cookie 已过期!请在当前 Chrome 浏览器中重新登录 BOSS 直聘。';
|
||||
const RECRUITER_ONLY_MSG = '该命令仅支持招聘端(BOSS 端)账号,请使用招聘者账号登录后重试。';
|
||||
const DEFAULT_TIMEOUT = 15_000;
|
||||
// ── Core helpers ────────────────────────────────────────────────────────────
|
||||
/**
|
||||
@@ -10,7 +13,24 @@ const DEFAULT_TIMEOUT = 15_000;
|
||||
*/
|
||||
export function requirePage(page) {
|
||||
if (!page)
|
||||
throw new Error('Browser page required');
|
||||
throw new CommandExecutionError('Browser page required');
|
||||
}
|
||||
export function readPositiveInteger(raw, name, fallback, max) {
|
||||
const value = raw === undefined || raw === null || raw === '' ? fallback : Number(raw);
|
||||
if (!Number.isInteger(value) || value < 1) {
|
||||
throw new ArgumentError(`boss ${name} must be a positive integer`);
|
||||
}
|
||||
if (max !== undefined && value > max) {
|
||||
throw new ArgumentError(`boss ${name} must be <= ${max}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
export function readRequiredString(raw, name) {
|
||||
const value = String(raw ?? '').trim();
|
||||
if (!value) {
|
||||
throw new ArgumentError(`boss ${name} cannot be empty`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
/**
|
||||
* Navigate to BOSS chat page and wait for it to settle.
|
||||
@@ -33,19 +53,37 @@ export async function navigateTo(page, url, waitSeconds = 1) {
|
||||
*/
|
||||
export function checkAuth(data) {
|
||||
if (COOKIE_EXPIRED_CODES.has(data.code)) {
|
||||
throw new Error(COOKIE_EXPIRED_MSG);
|
||||
throw new AuthRequiredError(BOSS_DOMAIN, COOKIE_EXPIRED_MSG);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Map BOSS code=24 ("请切换身份后再试") to a typed AuthRequiredError.
|
||||
* Recruiter-only commands (recommend, joblist, stats, resume, mark,
|
||||
* exchange, invite, greet, batchgreet) have no geek-side equivalent;
|
||||
* surfacing this as a generic COMMAND_EXEC hides what the user must do.
|
||||
* chatlist / chatmsg avoid this path by using `allowNonZero: true` and
|
||||
* branching to the geek-side fetch when they see code 24.
|
||||
*/
|
||||
function checkRecruiterSide(data) {
|
||||
if (data.code === IDENTITY_MISMATCH_CODE) {
|
||||
throw new AuthRequiredError(BOSS_DOMAIN, RECRUITER_ONLY_MSG);
|
||||
}
|
||||
}
|
||||
/**
|
||||
* Throw if the API response is not code 0.
|
||||
* Checks for cookie expiry first, then throws with the provided message.
|
||||
* Checks for cookie expiry first, then identity mismatch, then throws
|
||||
* with the provided message.
|
||||
*/
|
||||
export function assertOk(data, errorPrefix) {
|
||||
if (!data || typeof data !== 'object') {
|
||||
throw new CommandExecutionError(`${errorPrefix ? `${errorPrefix}: ` : ''}Boss API returned malformed response`);
|
||||
}
|
||||
if (data.code === 0)
|
||||
return;
|
||||
checkAuth(data);
|
||||
checkRecruiterSide(data);
|
||||
const prefix = errorPrefix ? `${errorPrefix}: ` : '';
|
||||
throw new Error(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
|
||||
throw new CommandExecutionError(`${prefix}${data.message || 'Unknown error'} (code=${data.code})`);
|
||||
}
|
||||
/**
|
||||
* Make a credentialed XHR request via page.evaluate().
|
||||
@@ -80,7 +118,19 @@ export async function bossFetch(page, url, opts = {}) {
|
||||
});
|
||||
}
|
||||
`;
|
||||
const data = await page.evaluate(script);
|
||||
let data;
|
||||
try {
|
||||
data = await page.evaluate(script);
|
||||
} catch (error) {
|
||||
if (error instanceof AuthRequiredError || error instanceof CommandExecutionError) {
|
||||
throw error;
|
||||
}
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
throw new CommandExecutionError(`Boss API request failed: ${message}`);
|
||||
}
|
||||
if (!data || typeof data !== 'object') {
|
||||
throw new CommandExecutionError('Boss API returned malformed response');
|
||||
}
|
||||
// Auto-check auth unless caller opts out
|
||||
if (!opts.allowNonZero && data.code !== 0) {
|
||||
assertOk(data);
|
||||
@@ -95,8 +145,13 @@ export async function fetchFriendList(page, opts = {}) {
|
||||
const pageNum = opts.pageNum ?? 1;
|
||||
const jobId = opts.jobId ?? '0';
|
||||
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getBossFriendListV2.json?page=${pageNum}&status=0&jobId=${jobId}`;
|
||||
const data = await bossFetch(page, url);
|
||||
return data.zpData?.friendList || [];
|
||||
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
|
||||
if (opts.allowNonZero && data.code !== 0) return data;
|
||||
const list = data.zpData?.friendList;
|
||||
if (!Array.isArray(list)) {
|
||||
throw new CommandExecutionError('Boss friend list response did not include zpData.friendList');
|
||||
}
|
||||
return list;
|
||||
}
|
||||
/**
|
||||
* Fetch the recommended candidates (greetRecSortList).
|
||||
@@ -104,7 +159,11 @@ export async function fetchFriendList(page, opts = {}) {
|
||||
export async function fetchRecommendList(page) {
|
||||
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/greetRecSortList`;
|
||||
const data = await bossFetch(page, url);
|
||||
return data.zpData?.friendList || [];
|
||||
const list = data.zpData?.friendList;
|
||||
if (!Array.isArray(list)) {
|
||||
throw new CommandExecutionError('Boss recommend response did not include zpData.friendList');
|
||||
}
|
||||
return list;
|
||||
}
|
||||
/**
|
||||
* Find a friend by encryptUid, searching through friend list and optionally greet list.
|
||||
@@ -115,10 +174,14 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
|
||||
const checkGreetList = opts.checkGreetList ?? false;
|
||||
// Search friend list pages
|
||||
for (let p = 1; p <= maxPages; p++) {
|
||||
const friends = await fetchFriendList(page, { pageNum: p });
|
||||
const result = await fetchFriendList(page, { pageNum: p, allowNonZero: opts.allowNonZero });
|
||||
if (opts.allowNonZero && !Array.isArray(result)) {
|
||||
return { friend: null, code: result.code };
|
||||
}
|
||||
const friends = Array.isArray(result) ? result : [];
|
||||
const found = friends.find((f) => f.encryptUid === encryptUid);
|
||||
if (found)
|
||||
return found;
|
||||
return opts.allowNonZero ? { friend: found, code: 0 } : found;
|
||||
if (friends.length === 0)
|
||||
break;
|
||||
}
|
||||
@@ -127,9 +190,9 @@ export async function findFriendByUid(page, encryptUid, opts = {}) {
|
||||
const greetList = await fetchRecommendList(page);
|
||||
const found = greetList.find((f) => f.encryptUid === encryptUid);
|
||||
if (found)
|
||||
return found;
|
||||
return opts.allowNonZero ? { friend: found, code: 0 } : found;
|
||||
}
|
||||
return null;
|
||||
return opts.allowNonZero ? { friend: null, code: 0 } : null;
|
||||
}
|
||||
// ── UI automation helpers ───────────────────────────────────────────────────
|
||||
/**
|
||||
@@ -221,3 +284,185 @@ export function verbose(msg) {
|
||||
console.error(`[opencli:boss] ${msg}`);
|
||||
}
|
||||
}
|
||||
// ── Geek-side helpers ────────────────────────────────────────────────────────
|
||||
export const IDENTITY_MISMATCH_CODE = 24;
|
||||
const GEEK_CHAT_URL = `https://${BOSS_DOMAIN}/web/geek/chat`;
|
||||
/**
|
||||
* Navigate to the job-seeker chat page.
|
||||
* Establishes the cookie + JS-global context needed for geek-side API calls.
|
||||
*/
|
||||
export async function navigateToGeekChat(page, waitSeconds = 2) {
|
||||
await page.goto(GEEK_CHAT_URL);
|
||||
await page.wait({ time: waitSeconds });
|
||||
}
|
||||
/**
|
||||
* Read the encryptSystemId value required by the geek-side list API.
|
||||
* Strategy (in order):
|
||||
* 1. Vue app state / Pinia stores / $route.query (Option 1 — runtime source)
|
||||
* 2. performance.getEntriesByType('resource') — parse from geekFilterByLabel URL
|
||||
* that the page itself already issued (Option 2 — most deterministic)
|
||||
* 3. cookie, inline <script> SSR state, known window globals, localStorage (fallbacks)
|
||||
* Returns empty string if nothing is found; the API may still succeed without it.
|
||||
* Caller must have navigated to the geek chat page first.
|
||||
*/
|
||||
export async function readEncryptSystemId(page) {
|
||||
const result = await page.evaluate(`
|
||||
(() => {
|
||||
// 1. Vue app state / Pinia / $route.query
|
||||
// The chat component reads encryptSystemId from the app runtime to build
|
||||
// its own geekFilterByLabel request, so the value lives in the Vue tree.
|
||||
try {
|
||||
const appEl = document.querySelector('#app') || document.querySelector('[data-v-app]');
|
||||
const vueApp = appEl && (appEl.__vue_app__ || appEl._vei);
|
||||
if (vueApp) {
|
||||
// 1a. Pinia stores (Vue 3 standard state management on BOSS直聘)
|
||||
const pinia = vueApp.config && vueApp.config.globalProperties.$pinia;
|
||||
if (pinia && pinia.state && pinia.state.value) {
|
||||
for (const store of Object.values(pinia.state.value)) {
|
||||
try {
|
||||
const flat = JSON.stringify(store);
|
||||
if (flat.includes('encryptSystemId')) {
|
||||
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
|
||||
if (m) return m[1];
|
||||
}
|
||||
} catch (_) {}
|
||||
}
|
||||
}
|
||||
// 1b. Vue Router current route query
|
||||
const router = vueApp.config && vueApp.config.globalProperties.$router;
|
||||
const query = router && router.currentRoute && router.currentRoute.value && router.currentRoute.value.query;
|
||||
if (query && query.encryptSystemId) return query.encryptSystemId;
|
||||
}
|
||||
} catch (_) {}
|
||||
// 2. Performance resource entries — the page already issued geekFilterByLabel
|
||||
// with encryptSystemId in the URL; read it back from the resource timing API.
|
||||
try {
|
||||
const entries = performance.getEntriesByType('resource');
|
||||
for (const entry of entries) {
|
||||
if (!entry.name.includes('geekFilterByLabel')) continue;
|
||||
const u = new URL(entry.name);
|
||||
const v = u.searchParams.get('encryptSystemId');
|
||||
if (v) return v;
|
||||
}
|
||||
} catch (_) {}
|
||||
// 3. cookie
|
||||
try {
|
||||
const m = document.cookie.match(/encryptSystemId=([^;]+)/i);
|
||||
if (m) return decodeURIComponent(m[1]);
|
||||
} catch (_) {}
|
||||
// 4. inline <script> SSR state (Nuxt embeds server state here)
|
||||
try {
|
||||
for (const s of document.querySelectorAll('script:not([src])')) {
|
||||
const t = s.textContent || '';
|
||||
if (!t.includes('encryptSystemId')) continue;
|
||||
const m = t.match(/"encryptSystemId":"([^"]+)"/);
|
||||
if (m) return m[1];
|
||||
}
|
||||
} catch (_) {}
|
||||
// 5. known BOSS / Nuxt window globals
|
||||
const KNOWN = [
|
||||
'__NUXT__', '__INITIAL_STATE__', '__ZP_INFO__', '__BOSS_ZP__',
|
||||
'pageGlobalVar', 'ZP_DATA', '__ZP_DATA__', '__PAGE_DATA__',
|
||||
];
|
||||
for (const k of KNOWN) {
|
||||
const obj = window[k];
|
||||
if (!obj || typeof obj !== 'object') continue;
|
||||
try {
|
||||
const flat = JSON.stringify(obj);
|
||||
if (!flat.includes('encryptSystemId')) continue;
|
||||
const m = flat.match(/"encryptSystemId":"([^"]+)"/);
|
||||
if (m) return m[1];
|
||||
} catch (_) {}
|
||||
}
|
||||
// 6. localStorage
|
||||
try {
|
||||
for (let i = 0; i < localStorage.length; i++) {
|
||||
const k = localStorage.key(i);
|
||||
if (!k) continue;
|
||||
if (k.toLowerCase().includes('encryptsystemid')) {
|
||||
const v = localStorage.getItem(k);
|
||||
if (v) return v;
|
||||
}
|
||||
const v = localStorage.getItem(k) || '';
|
||||
if (v.includes('encryptSystemId')) {
|
||||
const m = v.match(/"encryptSystemId":"([^"]+)"/);
|
||||
if (m) return m[1];
|
||||
}
|
||||
}
|
||||
} catch (_) {}
|
||||
return '';
|
||||
})()
|
||||
`);
|
||||
return result || '';
|
||||
}
|
||||
/**
|
||||
* Fetch the job-seeker chat list (brief info, no securityId).
|
||||
* Use fetchGeekFriendInfoList to enrich with securityId before calling chatmsg.
|
||||
*/
|
||||
export async function fetchGeekFriendLabelList(page, opts = {}) {
|
||||
const labelId = opts.labelId ?? 0;
|
||||
const encryptSystemId = opts.encryptSystemId ?? '';
|
||||
const url = `https://${BOSS_DOMAIN}/wapi/zprelation/friend/geekFilterByLabel?labelId=${labelId}&encryptSystemId=${encodeURIComponent(encryptSystemId)}`;
|
||||
const data = await bossFetch(page, url, { allowNonZero: opts.allowNonZero });
|
||||
if (opts.allowNonZero && data.code !== 0) return data;
|
||||
const list = data.zpData?.friendList;
|
||||
if (!Array.isArray(list)) {
|
||||
throw new CommandExecutionError('Boss geek chat list response did not include zpData.friendList');
|
||||
}
|
||||
return list;
|
||||
}
|
||||
/**
|
||||
* Enrich a batch of geek friends with full fields including securityId.
|
||||
* Processes in batches of 50 to avoid oversized request bodies.
|
||||
*/
|
||||
export async function fetchGeekFriendInfoList(page, friendIds = []) {
|
||||
if (!friendIds.length) return [];
|
||||
const BATCH_SIZE = 50;
|
||||
const results = [];
|
||||
for (let i = 0; i < friendIds.length; i += BATCH_SIZE) {
|
||||
const batch = friendIds.slice(i, i + BATCH_SIZE).map(String);
|
||||
const body = `friendIds=${batch.join(',')}`;
|
||||
const data = await bossFetch(page, `https://${BOSS_DOMAIN}/wapi/zprelation/friend/getGeekFriendList.json`, {
|
||||
method: 'POST',
|
||||
body,
|
||||
});
|
||||
const batchResult = data.zpData?.result;
|
||||
if (!Array.isArray(batchResult)) {
|
||||
throw new CommandExecutionError('Boss geek friend enrichment response did not include zpData.result');
|
||||
}
|
||||
results.push(...batchResult);
|
||||
}
|
||||
return results;
|
||||
}
|
||||
/**
|
||||
* Find a geek-side friend by encrypted uid.
|
||||
* Merges label-list and enriched data; returns null if not found.
|
||||
*/
|
||||
export async function findGeekFriendByUid(page, encryptUid, opts = {}) {
|
||||
const labelList = await fetchGeekFriendLabelList(page, { encryptSystemId: opts.encryptSystemId });
|
||||
const candidate = labelList.find((f) => f.encryptFriendId === encryptUid ||
|
||||
String(f.uid) === String(encryptUid) ||
|
||||
String(f.friendId) === String(encryptUid));
|
||||
if (!candidate) return null;
|
||||
const enriched = await fetchGeekFriendInfoList(page, [candidate.friendId]);
|
||||
return { ...candidate, ...(enriched[0] || {}) };
|
||||
}
|
||||
/**
|
||||
* Fetch message history for a geek-side chat.
|
||||
* friend must have .uid (boss's numeric id) and .securityId.
|
||||
*/
|
||||
export async function fetchGeekHistoryMsg(page, friend, opts = {}) {
|
||||
const pageNum = opts.page ?? 1;
|
||||
const bossId = friend.uid;
|
||||
const securityId = encodeURIComponent(friend.securityId || '');
|
||||
const url = `https://${BOSS_DOMAIN}/wapi/zpchat/geek/historyMsg?bossId=${bossId}&securityId=${securityId}&page=${pageNum}&c=20&src=0`;
|
||||
const data = await bossFetch(page, url);
|
||||
const messages = data.zpData?.messages ?? data.zpData?.historyMsgList;
|
||||
if (!Array.isArray(messages)) {
|
||||
throw new CommandExecutionError('Boss geek history response did not include a message list');
|
||||
}
|
||||
if (messages.length === 0) {
|
||||
throw new EmptyResultError('boss chatmsg', 'Boss returned no messages for this chat.');
|
||||
}
|
||||
return messages;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { assertOk } from './utils.js';
|
||||
|
||||
describe('assertOk', () => {
|
||||
it('returns silently on code 0', () => {
|
||||
expect(() => assertOk({ code: 0 })).not.toThrow();
|
||||
});
|
||||
|
||||
it('maps expired cookie codes (7, 37) to AuthRequiredError', () => {
|
||||
expect(() => assertOk({ code: 7, message: 'expired' })).toThrow(AuthRequiredError);
|
||||
expect(() => assertOk({ code: 37, message: 'expired' })).toThrow(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('maps code 24 (identity mismatch) to AuthRequiredError with recruiter-only hint', () => {
|
||||
try {
|
||||
assertOk({ code: 24, message: '请切换身份后再试' });
|
||||
throw new Error('assertOk should have thrown');
|
||||
} catch (err) {
|
||||
expect(err).toBeInstanceOf(AuthRequiredError);
|
||||
expect(String(err.message)).toContain('招聘端');
|
||||
}
|
||||
});
|
||||
|
||||
it('falls through to CommandExecutionError for other non-zero codes', () => {
|
||||
expect(() => assertOk({ code: 99, message: 'something else' }))
|
||||
.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError on malformed (non-object) response', () => {
|
||||
expect(() => assertOk(null)).toThrow(CommandExecutionError);
|
||||
expect(() => assertOk('not-an-object')).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,80 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import {
|
||||
emptySearchResults,
|
||||
requireBoundedInteger,
|
||||
requireNonNegativeInteger,
|
||||
requireRows,
|
||||
requireSearchQuery,
|
||||
runBrowserStep,
|
||||
toHttpsUrl,
|
||||
} from '../_shared/search-adapter.js';
|
||||
|
||||
function buildExtractorJs(limit) {
|
||||
return `
|
||||
(function() {
|
||||
var results = [];
|
||||
var seen = {};
|
||||
var items = document.querySelectorAll('.snippet');
|
||||
for (var i = 0; i < items.length; i++) {
|
||||
if (results.length >= ${limit}) break;
|
||||
var el = items[i];
|
||||
if (el.classList.contains('standalone') || el.classList.contains('ad')) continue;
|
||||
var titleEl = el.querySelector('.search-snippet-title');
|
||||
var snippetEl = el.querySelector('.generic-snippet .content');
|
||||
var linkEl = el.querySelector('.result-content a');
|
||||
if (!titleEl) continue;
|
||||
var title = titleEl.textContent.trim();
|
||||
var href = linkEl ? linkEl.getAttribute('href') || '' : '';
|
||||
var snippet = snippetEl ? snippetEl.textContent.trim() : '';
|
||||
if (!title || !href || seen[href]) continue;
|
||||
if (href.indexOf('/') === 0) continue;
|
||||
seen[href] = true;
|
||||
results.push([title, href, snippet]);
|
||||
}
|
||||
return results;
|
||||
})()`;
|
||||
}
|
||||
|
||||
const command = cli({
|
||||
site: 'brave',
|
||||
name: 'search',
|
||||
access: 'read',
|
||||
description: 'Search Brave Search',
|
||||
domain: 'search.brave.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'keyword', positional: true, required: true, help: 'Search query' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of results per page (max 18)' },
|
||||
{ name: 'offset', type: 'int', default: 0, help: 'Page offset (0, 1, 2...). Brave returns ~18 results per page' },
|
||||
],
|
||||
columns: ['rank', 'title', 'url', 'snippet'],
|
||||
func: async (page, kwargs) => {
|
||||
const limit = requireBoundedInteger(kwargs.limit, 10, 1, 18, '--limit');
|
||||
const query = requireSearchQuery(kwargs.keyword);
|
||||
const keyword = encodeURIComponent(query);
|
||||
const offset = requireNonNegativeInteger(kwargs.offset, 0, '--offset');
|
||||
let url = `https://search.brave.com/search?q=${keyword}`;
|
||||
if (offset > 0) url += `&offset=${offset}`;
|
||||
await runBrowserStep('brave search navigation', () => page.goto(url));
|
||||
try {
|
||||
await page.wait({ selector: '.snippet', timeout: 10 });
|
||||
} catch {
|
||||
await page.wait(3).catch(function() {});
|
||||
}
|
||||
const raw = await runBrowserStep('brave search extraction', () => page.evaluate(buildExtractorJs(limit)));
|
||||
const results = requireRows(raw, 'brave search');
|
||||
if (results.length === 0) {
|
||||
throw emptySearchResults('Brave', query);
|
||||
}
|
||||
const rows = results
|
||||
.map(function(r, index) {
|
||||
return { rank: index + 1 + offset * 18, title: r[0], url: toHttpsUrl(r[1], 'https://search.brave.com'), snippet: r[2] };
|
||||
})
|
||||
.filter((row) => row.url);
|
||||
if (rows.length === 0) throw emptySearchResults('Brave', query);
|
||||
return rows;
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { command };
|
||||
@@ -0,0 +1,76 @@
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
|
||||
const { __test__ } = await import('./search.js');
|
||||
const command = __test__.command;
|
||||
|
||||
function createPageMock(evaluateResult = []) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('brave search', () => {
|
||||
it('should register as a valid command', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('brave');
|
||||
expect(command.name).toBe('search');
|
||||
expect(command.access).toBe('read');
|
||||
expect(command.browser).toBe(true);
|
||||
expect(command.strategy).toBe('public');
|
||||
expect(command.domain).toBe('search.brave.com');
|
||||
});
|
||||
|
||||
it('should define keyword positional arg', () => {
|
||||
const kwArg = command.args.find(a => a.name === 'keyword');
|
||||
expect(kwArg).toBeDefined();
|
||||
expect(kwArg.positional).toBe(true);
|
||||
expect(kwArg.required).toBe(true);
|
||||
});
|
||||
|
||||
it('should define limit arg with default 10', () => {
|
||||
const limitArg = command.args.find(a => a.name === 'limit');
|
||||
expect(limitArg).toBeDefined();
|
||||
expect(limitArg.type).toBe('int');
|
||||
expect(limitArg.default).toBe(10);
|
||||
});
|
||||
|
||||
it('should define output columns', () => {
|
||||
expect(command.columns).toContain('rank');
|
||||
expect(command.columns).toContain('title');
|
||||
expect(command.columns).toContain('url');
|
||||
expect(command.columns).toContain('snippet');
|
||||
});
|
||||
|
||||
it('rejects empty query, invalid limit, and invalid offset before navigation', async () => {
|
||||
const page = createPageMock();
|
||||
await expect(command.func(page, { keyword: '', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 19 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 5, offset: -1 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('unwraps browser envelopes and returns ranked HTTPS rows', async () => {
|
||||
const page = createPageMock({
|
||||
session: 'site:brave',
|
||||
data: [['OpenCLI', 'https://github.com/jackwener/OpenCLI', 'CLI browser tooling']],
|
||||
});
|
||||
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 1, offset: 1 })).resolves.toEqual([{
|
||||
rank: 19,
|
||||
title: 'OpenCLI',
|
||||
url: 'https://github.com/jackwener/OpenCLI',
|
||||
snippet: 'CLI browser tooling',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('fails typed instead of silently returning [] for malformed extraction payloads', async () => {
|
||||
const page = createPageMock({ rows: [] });
|
||||
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).rejects.toMatchObject({
|
||||
code: 'COMMAND_EXEC',
|
||||
message: expect.stringContaining('payload shape'),
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,108 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import {
|
||||
requireArrayEvaluateResult,
|
||||
requireBooleanEvaluateResult,
|
||||
requireObjectEvaluateResult,
|
||||
unwrapEvaluateResult,
|
||||
} from './utils.js';
|
||||
|
||||
describe('chatgpt page.evaluate envelope helpers', () => {
|
||||
describe('unwrapEvaluateResult', () => {
|
||||
it('unwraps a { session, data } envelope produced by the browser bridge', () => {
|
||||
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 'msg-1' }] };
|
||||
expect(unwrapEvaluateResult(envelope)).toEqual([{ id: 'msg-1' }]);
|
||||
});
|
||||
|
||||
it('passes raw arrays through unchanged (back-compat with older bridge versions)', () => {
|
||||
const raw = [1, 2, 3];
|
||||
expect(unwrapEvaluateResult(raw)).toBe(raw);
|
||||
});
|
||||
|
||||
it('passes primitive return values (URL strings, booleans) through unchanged', () => {
|
||||
expect(unwrapEvaluateResult('https://chatgpt.com/c/abc')).toBe('https://chatgpt.com/c/abc');
|
||||
expect(unwrapEvaluateResult(true)).toBe(true);
|
||||
expect(unwrapEvaluateResult(0)).toBe(0);
|
||||
});
|
||||
|
||||
it('passes plain non-envelope objects through unchanged', () => {
|
||||
const obj = { ok: true, reason: 'all good' };
|
||||
expect(unwrapEvaluateResult(obj)).toBe(obj);
|
||||
});
|
||||
|
||||
it('handles null and undefined defensively', () => {
|
||||
expect(unwrapEvaluateResult(null)).toBe(null);
|
||||
expect(unwrapEvaluateResult(undefined)).toBe(undefined);
|
||||
});
|
||||
});
|
||||
|
||||
describe('requireArrayEvaluateResult', () => {
|
||||
it('returns the payload when it is an array', () => {
|
||||
const rows = [{ id: 1 }, { id: 2 }];
|
||||
expect(requireArrayEvaluateResult(rows, 'chatgpt test')).toBe(rows);
|
||||
});
|
||||
|
||||
it('throws a typed CommandExecutionError when the payload is the raw envelope (caller forgot to unwrap)', () => {
|
||||
const envelope = { session: 'site:chatgpt:abc', data: [{ id: 1 }] };
|
||||
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
|
||||
.toThrowError(CommandExecutionError);
|
||||
expect(() => requireArrayEvaluateResult(envelope, 'chatgpt visible image url extraction'))
|
||||
.toThrow(/malformed extraction payload/);
|
||||
});
|
||||
|
||||
it('surfaces the inner error message when the payload carries an `error` field', () => {
|
||||
const errPayload = { error: 'image generator returned 500' };
|
||||
expect(() => requireArrayEvaluateResult(errPayload, 'chatgpt image asset export'))
|
||||
.toThrow(/chatgpt image asset export: image generator returned 500/);
|
||||
});
|
||||
|
||||
it('throws when the payload is null or a primitive', () => {
|
||||
expect(() => requireArrayEvaluateResult(null, 'chatgpt test')).toThrowError(CommandExecutionError);
|
||||
expect(() => requireArrayEvaluateResult('a string', 'chatgpt test')).toThrowError(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('requireObjectEvaluateResult', () => {
|
||||
it('returns the payload when it is a plain object', () => {
|
||||
const obj = { url: 'https://chatgpt.com', isLoggedIn: true };
|
||||
expect(requireObjectEvaluateResult(obj, 'chatgpt page state')).toBe(obj);
|
||||
});
|
||||
|
||||
it('throws when the payload is an array or a primitive', () => {
|
||||
expect(() => requireObjectEvaluateResult([], 'chatgpt page state')).toThrowError(CommandExecutionError);
|
||||
expect(() => requireObjectEvaluateResult('string', 'chatgpt page state')).toThrowError(CommandExecutionError);
|
||||
expect(() => requireObjectEvaluateResult(null, 'chatgpt page state')).toThrowError(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('requireBooleanEvaluateResult', () => {
|
||||
it('returns booleans and rejects wrong-shape values', () => {
|
||||
expect(requireBooleanEvaluateResult(true, 'chatgpt generation state')).toBe(true);
|
||||
expect(requireBooleanEvaluateResult(false, 'chatgpt generation state')).toBe(false);
|
||||
expect(() => requireBooleanEvaluateResult({ ok: true }, 'chatgpt generation state'))
|
||||
.toThrowError(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('end-to-end envelope sweep', () => {
|
||||
// The bridge envelope is shaped like { session, data } where `session` is
|
||||
// any string and `data` is the actual return value. Verify the helpers
|
||||
// chain correctly: unwrap → require* yields the inner shape.
|
||||
it('unwrap + requireArray pipes an envelope through to the underlying array', () => {
|
||||
const envelope = {
|
||||
session: 'site:chatgpt:img-export',
|
||||
data: [
|
||||
{ url: 'https://a.example/1.png', dataUrl: 'data:image/png;base64,xxx', mimeType: 'image/png' },
|
||||
],
|
||||
};
|
||||
expect(requireArrayEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt image asset export'))
|
||||
.toEqual(envelope.data);
|
||||
});
|
||||
|
||||
it('unwrap + requireObject pipes an envelope through to the underlying object', () => {
|
||||
const envelope = { session: 'site:chatgpt:state', data: { url: 'https://chatgpt.com', isLoggedIn: true } };
|
||||
expect(requireObjectEvaluateResult(unwrapEvaluateResult(envelope), 'chatgpt page state'))
|
||||
.toEqual(envelope.data);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -4,7 +4,7 @@ import * as fs from 'node:fs';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { saveBase64ToFile } from '@jackwener/opencli/utils';
|
||||
import { ArgumentError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { clearChatGPTDraft, getChatGPTVisibleImageUrls, normalizeBooleanFlag, prepareChatGPTImagePaths, sendChatGPTMessage, waitForChatGPTImages, getChatGPTImageAssets, uploadChatGPTImages } from './utils.js';
|
||||
import { clearChatGPTDraft, getChatGPTVisibleImageUrls, normalizeBooleanFlag, prepareChatGPTImagePaths, sendChatGPTMessage, unwrapEvaluateResult, waitForChatGPTImages, getChatGPTImageAssets, uploadChatGPTImages } from './utils.js';
|
||||
|
||||
const CHATGPT_DOMAIN = 'chatgpt.com';
|
||||
|
||||
@@ -54,7 +54,7 @@ function buildPrompt(prompt, imageCount) {
|
||||
}
|
||||
|
||||
async function currentChatGPTLink(page) {
|
||||
const url = await page.evaluate('window.location.href').catch(() => '');
|
||||
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
|
||||
return typeof url === 'string' && url ? url : 'https://chatgpt.com';
|
||||
}
|
||||
|
||||
|
||||
@@ -24,6 +24,12 @@ vi.mock('./utils.js', () => ({
|
||||
},
|
||||
prepareChatGPTImagePaths: mocks.prepareChatGPTImagePaths,
|
||||
sendChatGPTMessage: mocks.sendChatGPTMessage,
|
||||
unwrapEvaluateResult: (payload) => {
|
||||
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
|
||||
return payload.data;
|
||||
}
|
||||
return payload;
|
||||
},
|
||||
uploadChatGPTImages: mocks.uploadChatGPTImages,
|
||||
waitForChatGPTImages: mocks.waitForChatGPTImages,
|
||||
getChatGPTImageAssets: mocks.getChatGPTImageAssets,
|
||||
|
||||
+148
-41
@@ -74,7 +74,6 @@ function buildComposerLocatorScript() {
|
||||
};
|
||||
|
||||
findComposer.toString = () => 'findComposer';
|
||||
return { findComposer, markerAttr };
|
||||
`;
|
||||
}
|
||||
|
||||
@@ -103,6 +102,50 @@ export function requirePositiveInt(value, flagLabel, hint) {
|
||||
return value;
|
||||
}
|
||||
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
// page.evaluate envelope helpers.
|
||||
//
|
||||
// The browser bridge wraps every `page.evaluate(...)` return value in a
|
||||
// `{ session, data }` envelope. Adapters that read `.length` or
|
||||
// `Array.isArray(payload)` directly on the envelope silently see "no data" —
|
||||
// this matches the failure mode fixed for xiaohongshu/rednote (#1561) and
|
||||
// weibo (#1568).
|
||||
//
|
||||
// `unwrapEvaluateResult` is a defensive ternary: it unwraps when the payload
|
||||
// looks like an envelope, otherwise passes the value through unchanged so
|
||||
// older bridge versions and primitive return values still work.
|
||||
// ─────────────────────────────────────────────────────────────────────────────
|
||||
export function unwrapEvaluateResult(payload) {
|
||||
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
|
||||
return payload.data;
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function requireArrayEvaluateResult(payload, label) {
|
||||
if (!Array.isArray(payload)) {
|
||||
if (payload && typeof payload === 'object' && 'error' in payload) {
|
||||
throw new CommandExecutionError(`${label}: ${String(payload.error)}`);
|
||||
}
|
||||
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function requireObjectEvaluateResult(payload, label) {
|
||||
if (!payload || Array.isArray(payload) || typeof payload !== 'object') {
|
||||
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function requireBooleanEvaluateResult(payload, label) {
|
||||
if (typeof payload !== 'boolean') {
|
||||
throw new CommandExecutionError(`${label} returned malformed extraction payload`);
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function parseChatGPTConversationId(value) {
|
||||
const raw = String(value ?? '').trim();
|
||||
const match = raw.match(/(?:^|\/c\/)([A-Za-z0-9_-]{8,})(?:[/?#]|$)/);
|
||||
@@ -115,7 +158,7 @@ export function parseChatGPTConversationId(value) {
|
||||
}
|
||||
|
||||
export async function currentChatGPTUrl(page) {
|
||||
const url = await page.evaluate('window.location.href').catch(() => '');
|
||||
const url = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
|
||||
return typeof url === 'string' ? url : '';
|
||||
}
|
||||
|
||||
@@ -161,7 +204,7 @@ export async function startNewChat(page) {
|
||||
}
|
||||
|
||||
export async function getPageState(page) {
|
||||
return await page.evaluate(`(() => {
|
||||
return requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
|
||||
const isVisible = (el) => {
|
||||
if (!(el instanceof HTMLElement)) return false;
|
||||
const style = window.getComputedStyle(el);
|
||||
@@ -187,7 +230,7 @@ export async function getPageState(page) {
|
||||
isLoggedIn: hasComposer || !!userMenu || !hasLoginGate,
|
||||
hasLoginGate,
|
||||
};
|
||||
})()`);
|
||||
})()`)), 'chatgpt page state');
|
||||
}
|
||||
|
||||
export async function ensureChatGPTLogin(page, message = 'ChatGPT requires a logged-in browser session.') {
|
||||
@@ -258,7 +301,7 @@ export async function sendChatGPTMessage(page, text) {
|
||||
// findComposer() retries inside a single CDP call, so no fixed sleep is
|
||||
// needed before reading the composer.
|
||||
|
||||
const typeResult = await page.evaluate(`
|
||||
const typeResult = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
${buildComposerLocatorScript()}
|
||||
const composer = findComposer();
|
||||
@@ -276,8 +319,8 @@ export async function sendChatGPTMessage(page, text) {
|
||||
composer.dispatchEvent(new Event('change', { bubbles: true }));
|
||||
return true;
|
||||
})()
|
||||
`);
|
||||
|
||||
`)), 'chatgpt composer readiness');
|
||||
|
||||
if (!typeResult) return false;
|
||||
|
||||
// Use page.type() which is Playwright's native method
|
||||
@@ -304,7 +347,7 @@ export async function sendChatGPTMessage(page, text) {
|
||||
let sent = null;
|
||||
for (let attempt = 0; attempt < 20; attempt += 1) {
|
||||
await page.wait(0.5);
|
||||
sent = await page.evaluate(`
|
||||
sent = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
const isUsable = (button) => button
|
||||
&& !button.disabled
|
||||
@@ -318,7 +361,7 @@ export async function sendChatGPTMessage(page, text) {
|
||||
: btns.find(b => labels.includes(b.getAttribute('aria-label') || '') && isUsable(b));
|
||||
return { sendBtnFound: !!sendBtn };
|
||||
})()
|
||||
`);
|
||||
`)), 'chatgpt send button readiness');
|
||||
if (sent?.sendBtnFound) break;
|
||||
}
|
||||
|
||||
@@ -339,7 +382,7 @@ export async function sendChatGPTMessage(page, text) {
|
||||
}
|
||||
|
||||
export async function getVisibleMessages(page) {
|
||||
const result = await page.evaluate(`(() => {
|
||||
const result = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
|
||||
const isVisible = (el) => {
|
||||
if (!(el instanceof HTMLElement)) return false;
|
||||
const style = window.getComputedStyle(el);
|
||||
@@ -385,8 +428,7 @@ export async function getVisibleMessages(page) {
|
||||
rows.push({ role, text, html });
|
||||
}
|
||||
return rows;
|
||||
})()`);
|
||||
if (!Array.isArray(result)) return [];
|
||||
})()`)), 'chatgpt visible messages');
|
||||
return result.map((item, index) => ({
|
||||
Index: index + 1,
|
||||
Role: item?.role === 'Assistant' ? 'Assistant' : 'User',
|
||||
@@ -448,7 +490,7 @@ export async function getConversationList(page) {
|
||||
// so the previous standalone 2 s settle is redundant.
|
||||
await ensureOnChatGPT(page);
|
||||
|
||||
const openSidebar = await page.evaluate(`(() => {
|
||||
const openSidebar = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
|
||||
const button = Array.from(document.querySelectorAll('button'))
|
||||
.find((node) => /open sidebar/i.test(node.getAttribute('aria-label') || ''));
|
||||
if (button instanceof HTMLElement) {
|
||||
@@ -456,7 +498,7 @@ export async function getConversationList(page) {
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
})()`);
|
||||
})()`)), 'chatgpt sidebar open state');
|
||||
if (openSidebar) {
|
||||
try {
|
||||
await page.wait({ selector: CONVERSATION_LINK_SELECTOR, timeout: 3 });
|
||||
@@ -480,7 +522,7 @@ export async function getConversationList(page) {
|
||||
}
|
||||
|
||||
async function extractConversationLinks(page) {
|
||||
const items = await page.evaluate(`(() => {
|
||||
const items = requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`(() => {
|
||||
const isVisible = (el) => {
|
||||
if (!(el instanceof HTMLElement)) return false;
|
||||
const style = window.getComputedStyle(el);
|
||||
@@ -505,15 +547,13 @@ async function extractConversationLinks(page) {
|
||||
});
|
||||
}
|
||||
return rows;
|
||||
})()`);
|
||||
return Array.isArray(items)
|
||||
? items.map((item, index) => ({
|
||||
})()`)), 'chatgpt conversation link extraction');
|
||||
return items.map((item, index) => ({
|
||||
Index: index + 1,
|
||||
Id: String(item?.Id || ''),
|
||||
Title: String(item?.Title || '(untitled)').trim() || '(untitled)',
|
||||
Url: String(item?.Url || ''),
|
||||
})).filter((item) => item.Id)
|
||||
: [];
|
||||
})).filter((item) => item.Id);
|
||||
}
|
||||
|
||||
function imageMimeFromPath(filePath) {
|
||||
@@ -556,7 +596,7 @@ async function waitForChatGPTUploadPreview(page, fileNames) {
|
||||
const namesJson = JSON.stringify(fileNames);
|
||||
for (let attempt = 0; attempt < 10; attempt += 1) {
|
||||
await page.wait(1);
|
||||
const ready = await page.evaluate(`
|
||||
const ready = requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
const names = ${namesJson};
|
||||
const text = document.body ? (document.body.innerText || '') : '';
|
||||
@@ -572,7 +612,7 @@ async function waitForChatGPTUploadPreview(page, fileNames) {
|
||||
const previewNodes = scope.querySelectorAll('img[src], canvas, video, [style*="background-image"], [data-testid*="attachment"], [data-testid*="upload"], [class*="attachment"], [class*="upload"]');
|
||||
return previewNodes.length >= names.length;
|
||||
})()
|
||||
`);
|
||||
`)), 'chatgpt upload preview detection');
|
||||
if (ready) return true;
|
||||
}
|
||||
return false;
|
||||
@@ -606,7 +646,7 @@ export async function uploadChatGPTImages(page, imagePaths) {
|
||||
mime: imageMimeFromPath(absPath),
|
||||
base64: fs.default.readFileSync(absPath).toString('base64'),
|
||||
}));
|
||||
const fallbackResult = await page.evaluate(`
|
||||
const fallbackResult = requireObjectEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
const files = ${JSON.stringify(files)};
|
||||
const input = document.querySelector('input[type="file"]');
|
||||
@@ -642,7 +682,7 @@ export async function uploadChatGPTImages(page, imagePaths) {
|
||||
}
|
||||
return { ok: true };
|
||||
})()
|
||||
`);
|
||||
`)), 'chatgpt image upload fallback');
|
||||
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
|
||||
}
|
||||
|
||||
@@ -656,21 +696,21 @@ export async function uploadChatGPTImages(page, imagePaths) {
|
||||
* Check if ChatGPT is still generating a response.
|
||||
*/
|
||||
export async function isGenerating(page) {
|
||||
return await page.evaluate(`
|
||||
return requireBooleanEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
return Array.from(document.querySelectorAll('button')).some(b => {
|
||||
const label = b.getAttribute('aria-label') || '';
|
||||
return label === 'Stop generating' || label.includes('Thinking');
|
||||
});
|
||||
})()
|
||||
`);
|
||||
`)), 'chatgpt generation state');
|
||||
}
|
||||
|
||||
/**
|
||||
* Get visible image URLs from the ChatGPT page (excluding profile/avatar images).
|
||||
*/
|
||||
export async function getChatGPTVisibleImageUrls(page) {
|
||||
return await page.evaluate(`
|
||||
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(() => {
|
||||
const isVisible = (el) => {
|
||||
if (!(el instanceof HTMLElement)) return false;
|
||||
@@ -680,32 +720,78 @@ export async function getChatGPTVisibleImageUrls(page) {
|
||||
return rect.width > 32 && rect.height > 32;
|
||||
};
|
||||
|
||||
const urls = [];
|
||||
const seen = new Set();
|
||||
const normalizeUrl = (value) => {
|
||||
const raw = String(value || '').trim();
|
||||
if (!raw || raw === 'none') return '';
|
||||
if (/^(?:https?:|blob:|data:)/i.test(raw)) return raw;
|
||||
try {
|
||||
return new URL(raw, window.location.href).href;
|
||||
} catch {
|
||||
return raw;
|
||||
}
|
||||
};
|
||||
const addUrl = (value) => {
|
||||
const src = normalizeUrl(value);
|
||||
if (!src || seen.has(src)) return;
|
||||
seen.add(src);
|
||||
urls.push(src);
|
||||
};
|
||||
const isDecorative = (el, src = '') => {
|
||||
const alt = (el.getAttribute('alt') || '').toLowerCase();
|
||||
const cls = String(el.className || '').toLowerCase();
|
||||
const testId = (el.getAttribute('data-testid') || '').toLowerCase();
|
||||
const label = (el.getAttribute('aria-label') || '').toLowerCase();
|
||||
const text = [alt, cls, testId, label, src.toLowerCase()].join(' ');
|
||||
return /avatar|profile|logo|icon/.test(text);
|
||||
};
|
||||
|
||||
const imgs = Array.from(document.querySelectorAll('img')).filter(img =>
|
||||
img instanceof HTMLImageElement && isVisible(img)
|
||||
);
|
||||
|
||||
const urls = [];
|
||||
const seen = new Set();
|
||||
|
||||
for (const img of imgs) {
|
||||
const src = img.currentSrc || img.src || '';
|
||||
const alt = (img.getAttribute('alt') || '').toLowerCase();
|
||||
const cls = (img.className || '').toLowerCase();
|
||||
const width = img.naturalWidth || img.width || 0;
|
||||
const height = img.naturalHeight || img.height || 0;
|
||||
|
||||
if (!src) continue;
|
||||
if (alt.includes('avatar') || alt.includes('profile') || alt.includes('logo') || alt.includes('icon')) continue;
|
||||
if (cls.includes('avatar') || cls.includes('profile') || cls.includes('icon')) continue;
|
||||
if (isDecorative(img, src)) continue;
|
||||
if (width < 128 && height < 128) continue;
|
||||
if (seen.has(src)) continue;
|
||||
addUrl(src);
|
||||
}
|
||||
|
||||
seen.add(src);
|
||||
urls.push(src);
|
||||
// ChatGPT occasionally renders generated images as CSS background
|
||||
// thumbnails instead of plain <img> nodes. Treat visible, large
|
||||
// background images as generated-image candidates too.
|
||||
for (const el of Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]'))) {
|
||||
if (!(el instanceof HTMLElement) || !isVisible(el) || isDecorative(el)) continue;
|
||||
const rect = el.getBoundingClientRect();
|
||||
if (rect.width < 128 && rect.height < 128) continue;
|
||||
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
|
||||
for (const match of backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)) {
|
||||
const src = match[2];
|
||||
if (!src || isDecorative(el, src)) continue;
|
||||
addUrl(src);
|
||||
}
|
||||
}
|
||||
|
||||
// Some image experiences render to a canvas. Returning the data URL
|
||||
// lets the downstream asset exporter save it without needing a DOM
|
||||
// selector to rediscover the canvas.
|
||||
for (const canvas of Array.from(document.querySelectorAll('canvas'))) {
|
||||
if (!(canvas instanceof HTMLCanvasElement) || !isVisible(canvas) || isDecorative(canvas)) continue;
|
||||
const width = canvas.width || canvas.getBoundingClientRect().width || 0;
|
||||
const height = canvas.height || canvas.getBoundingClientRect().height || 0;
|
||||
if (width < 128 && height < 128) continue;
|
||||
try {
|
||||
addUrl(canvas.toDataURL('image/png'));
|
||||
} catch { }
|
||||
}
|
||||
return urls;
|
||||
})()
|
||||
`);
|
||||
`)), 'chatgpt visible image url extraction');
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -723,7 +809,7 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
|
||||
|
||||
let currentUrl = '';
|
||||
if (convUrl && convUrl.includes('/c/')) {
|
||||
currentUrl = await page.evaluate('window.location.href').catch(() => '');
|
||||
currentUrl = unwrapEvaluateResult(await page.evaluate('window.location.href').catch(() => ''));
|
||||
if (currentUrl && !isSameChatGPTConversation(currentUrl, convUrl)) {
|
||||
await page.goto(convUrl);
|
||||
await page.wait(3);
|
||||
@@ -766,6 +852,7 @@ export const __test__ = {
|
||||
SEND_BUTTON_FALLBACK_SELECTORS,
|
||||
SEND_BUTTON_LABELS,
|
||||
CLOSE_SIDEBAR_LABELS,
|
||||
buildComposerLocatorScript,
|
||||
isSameChatGPTConversation,
|
||||
parseChatGPTConversationId,
|
||||
imageMimeFromPath,
|
||||
@@ -776,7 +863,7 @@ export const __test__ = {
|
||||
*/
|
||||
export async function getChatGPTImageAssets(page, urls) {
|
||||
const urlsJson = JSON.stringify(urls);
|
||||
return await page.evaluate(`
|
||||
return requireArrayEvaluateResult(unwrapEvaluateResult(await page.evaluate(`
|
||||
(async (targetUrls) => {
|
||||
const blobToDataUrl = (blob) => new Promise((resolve, reject) => {
|
||||
const reader = new FileReader();
|
||||
@@ -809,6 +896,26 @@ export async function getChatGPTImageAssets(page, urls) {
|
||||
if (img) {
|
||||
width = img.naturalWidth || img.width || 0;
|
||||
height = img.naturalHeight || img.height || 0;
|
||||
} else {
|
||||
const backgroundEl = Array.from(document.querySelectorAll('[style*="background-image"], [style*="background"]')).find(el => {
|
||||
if (!(el instanceof HTMLElement)) return false;
|
||||
const backgroundImage = window.getComputedStyle(el).backgroundImage || '';
|
||||
return Array.from(backgroundImage.matchAll(/url\\((['"]?)(.*?)\\1\\)/g)).some(match => {
|
||||
const raw = String(match[2] || '').trim();
|
||||
if (!raw) return false;
|
||||
if (raw === targetUrl) return true;
|
||||
try {
|
||||
return new URL(raw, window.location.href).href === targetUrl;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
});
|
||||
});
|
||||
if (backgroundEl) {
|
||||
const rect = backgroundEl.getBoundingClientRect();
|
||||
width = Math.round(rect.width || 0);
|
||||
height = Math.round(rect.height || 0);
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
@@ -850,5 +957,5 @@ export async function getChatGPTImageAssets(page, urls) {
|
||||
|
||||
return results;
|
||||
})(${urlsJson})
|
||||
`, urls);
|
||||
`)), 'chatgpt image asset export');
|
||||
}
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
import fs from 'node:fs';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
import { JSDOM } from 'jsdom';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { __test__, prepareChatGPTImagePaths, sendChatGPTMessage, uploadChatGPTImages, waitForChatGPTImages } from './utils.js';
|
||||
import { __test__, getChatGPTImageAssets, getChatGPTVisibleImageUrls, prepareChatGPTImagePaths, sendChatGPTMessage, uploadChatGPTImages, waitForChatGPTImages } from './utils.js';
|
||||
|
||||
const tempDirs = [];
|
||||
|
||||
@@ -88,6 +89,25 @@ describe('chatgpt conversation id parsing', () => {
|
||||
});
|
||||
|
||||
describe('chatgpt send selectors', () => {
|
||||
it('inlines the composer locator without returning before caller code runs', () => {
|
||||
const dom = new JSDOM('<!doctype html><div id="prompt-textarea" contenteditable="true"></div>', {
|
||||
url: 'https://chatgpt.com/',
|
||||
runScripts: 'outside-only',
|
||||
});
|
||||
const composer = dom.window.document.querySelector('#prompt-textarea');
|
||||
composer.getBoundingClientRect = () => ({ width: 320, height: 48 });
|
||||
|
||||
const result = dom.window.eval(`
|
||||
(() => {
|
||||
${__test__.buildComposerLocatorScript()}
|
||||
const composer = findComposer();
|
||||
return !!composer && composer.getAttribute(markerAttr) === '1';
|
||||
})()
|
||||
`);
|
||||
|
||||
expect(result).toBe(true);
|
||||
});
|
||||
|
||||
it('keeps locale-independent send-button selector before aria-label fallbacks', async () => {
|
||||
const page = {
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
@@ -143,6 +163,73 @@ describe('chatgpt send selectors', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('chatgpt generated image detection', () => {
|
||||
function createDomPage(html, setup = () => {}) {
|
||||
const dom = new JSDOM(html, {
|
||||
url: 'https://chatgpt.com/c/demo',
|
||||
runScripts: 'outside-only',
|
||||
});
|
||||
setup(dom.window);
|
||||
return {
|
||||
evaluate: vi.fn((script) => Promise.resolve(dom.window.eval(String(script)))),
|
||||
};
|
||||
}
|
||||
|
||||
it('detects visible CSS background images when ChatGPT does not render a plain img', async () => {
|
||||
const page = createDomPage(`
|
||||
<!doctype html>
|
||||
<main>
|
||||
<div class="avatar" style="background-image: url('https://chatgpt.com/avatar.png')"></div>
|
||||
<button data-testid="generated-image" style="background-image: url('/backend-api/generated/foo.webp')"></button>
|
||||
</main>
|
||||
`, (window) => {
|
||||
for (const el of window.document.querySelectorAll('div, button')) {
|
||||
el.getBoundingClientRect = () => ({ width: 512, height: 512 });
|
||||
}
|
||||
});
|
||||
|
||||
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
|
||||
'https://chatgpt.com/backend-api/generated/foo.webp',
|
||||
]);
|
||||
});
|
||||
|
||||
it('detects visible generated canvases as data URLs', async () => {
|
||||
const page = createDomPage('<!doctype html><canvas width="512" height="512"></canvas>', (window) => {
|
||||
const canvas = window.document.querySelector('canvas');
|
||||
canvas.getBoundingClientRect = () => ({ width: 512, height: 512 });
|
||||
canvas.toDataURL = () => 'data:image/png;base64,ZmFrZQ==';
|
||||
});
|
||||
|
||||
await expect(getChatGPTVisibleImageUrls(page)).resolves.toEqual([
|
||||
'data:image/png;base64,ZmFrZQ==',
|
||||
]);
|
||||
});
|
||||
|
||||
it('exports assets for generated CSS background images', async () => {
|
||||
const imageUrl = 'https://chatgpt.com/backend-api/generated/foo.webp';
|
||||
const page = createDomPage(`
|
||||
<!doctype html>
|
||||
<button style="background-image: url('/backend-api/generated/foo.webp')"></button>
|
||||
`, (window) => {
|
||||
const button = window.document.querySelector('button');
|
||||
button.getBoundingClientRect = () => ({ width: 512, height: 512 });
|
||||
window.fetch = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
blob: async () => new window.Blob(['fake-image'], { type: 'image/webp' }),
|
||||
});
|
||||
});
|
||||
|
||||
await expect(getChatGPTImageAssets(page, [imageUrl])).resolves.toEqual([
|
||||
expect.objectContaining({
|
||||
url: imageUrl,
|
||||
mimeType: 'image/webp',
|
||||
width: 512,
|
||||
height: 512,
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('chatgpt image upload helper', () => {
|
||||
it('validates local images without a browser page', async () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-chatgpt-'));
|
||||
@@ -218,7 +305,10 @@ describe('chatgpt image upload helper', () => {
|
||||
setFileInput: vi.fn().mockRejectedValue(new Error('No element found')),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn((script) => {
|
||||
return Promise.resolve({ ok: true });
|
||||
if (String(script).includes('new DataTransfer()')) {
|
||||
return Promise.resolve({ ok: true });
|
||||
}
|
||||
return Promise.resolve(true);
|
||||
}),
|
||||
};
|
||||
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { unwrapEvaluateResult } from './evaluate-result.js';
|
||||
|
||||
function isAuthLikeError(code, message) {
|
||||
const text = String(message ?? '');
|
||||
return code === 401 || code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(text);
|
||||
}
|
||||
|
||||
/**
|
||||
* Execute a fetch() call inside the Chrome browser context via page.evaluate.
|
||||
* This ensures a_bogus signing and cookies are handled automatically by the browser.
|
||||
@@ -6,36 +13,53 @@ import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
export async function browserFetch(page, method, url, options = {}) {
|
||||
const js = `
|
||||
(async () => {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
method: ${JSON.stringify(method)},
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...${JSON.stringify(options.headers ?? {})}
|
||||
},
|
||||
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
|
||||
});
|
||||
const text = await res.text();
|
||||
if (!text) return null;
|
||||
return JSON.parse(text);
|
||||
const controller = new AbortController();
|
||||
const timer = setTimeout(() => controller.abort(), ${Number(options.timeoutMs ?? 30000)});
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
method: ${JSON.stringify(method)},
|
||||
credentials: 'include',
|
||||
signal: controller.signal,
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
...${JSON.stringify(options.headers ?? {})}
|
||||
},
|
||||
${options.body ? `body: JSON.stringify(${JSON.stringify(options.body)}),` : ''}
|
||||
});
|
||||
const text = await res.text();
|
||||
try {
|
||||
return JSON.parse(text);
|
||||
} catch (error) {
|
||||
return { status_code: res.ok ? -2 : res.status, status_msg: \`JSON parse failed: \${text.slice(0, 500) || String(error && error.message || error)}\` };
|
||||
}
|
||||
} catch (error) {
|
||||
return { status_code: -1, status_msg: String(error && error.message || error) };
|
||||
} finally {
|
||||
clearTimeout(timer);
|
||||
}
|
||||
})()
|
||||
`;
|
||||
let result;
|
||||
try {
|
||||
result = await page.evaluate(js);
|
||||
result = unwrapEvaluateResult(await page.evaluate(js));
|
||||
}
|
||||
catch (error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
throw new CommandExecutionError(`Douyin API request failed: ${message}`);
|
||||
throw new CommandExecutionError(`Douyin API request failed (${method} ${url}): ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
if (result === null || result === undefined) {
|
||||
throw new CommandExecutionError('Empty response from Douyin API');
|
||||
if (result == null) {
|
||||
throw new CommandExecutionError(`Empty response from Douyin API (${method} ${url})`);
|
||||
}
|
||||
if (Array.isArray(result) || typeof result !== 'object') {
|
||||
throw new CommandExecutionError(`Malformed response from Douyin API (${method} ${url})`);
|
||||
}
|
||||
if (result && typeof result === 'object' && 'status_code' in result) {
|
||||
const code = result.status_code;
|
||||
if (code !== 0) {
|
||||
const msg = result.status_msg ?? 'unknown error';
|
||||
throw new CommandExecutionError(`Douyin API error ${code}: ${msg}`);
|
||||
const msg = result.status_msg ?? result.message ?? 'unknown error';
|
||||
if (isAuthLikeError(code, msg)) {
|
||||
throw new AuthRequiredError('creator.douyin.com', `Douyin API auth/permission error ${code} at ${method} ${url}: ${msg}`);
|
||||
}
|
||||
throw new CommandExecutionError(`Douyin API error ${code} at ${method} ${url}: ${msg}`);
|
||||
}
|
||||
}
|
||||
return result;
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { browserFetch } from './browser-fetch.js';
|
||||
function makePage(result) {
|
||||
return {
|
||||
@@ -18,10 +19,20 @@ describe('browserFetch', () => {
|
||||
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
|
||||
expect(result).toEqual({ status_code: 0, data: { ak: 'KEY' } });
|
||||
});
|
||||
it('unwraps Browser Bridge {session,data} envelopes', async () => {
|
||||
const page = makePage({ session: 'site:douyin:test', data: { status_code: 0, data: { ok: true } } });
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
|
||||
.resolves.toEqual({ status_code: 0, data: { ok: true } });
|
||||
});
|
||||
it('throws when status_code is non-zero', async () => {
|
||||
const page = makePage({ status_code: 8, message: 'fail' });
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API error 8');
|
||||
});
|
||||
it('maps auth-like API errors to AuthRequiredError', async () => {
|
||||
const page = makePage({ status_code: 401, status_msg: 'login required' });
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
|
||||
.rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
it('returns result even when no status_code field', async () => {
|
||||
const page = makePage({ some_field: 'value' });
|
||||
const result = await browserFetch(page, 'GET', 'https://creator.douyin.com/api/test');
|
||||
@@ -35,9 +46,19 @@ describe('browserFetch', () => {
|
||||
const page = makePage(undefined);
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Empty response from Douyin API');
|
||||
});
|
||||
it('throws typed on malformed primitive response body', async () => {
|
||||
const page = makePage('not-json-object');
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
it('throws typed when browser fetch returns a non-JSON body', async () => {
|
||||
const page = makePage({ status_code: -2, status_msg: 'JSON parse failed: <html>not-json</html>' });
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test'))
|
||||
.rejects.toThrow('Douyin API error -2');
|
||||
});
|
||||
it('wraps browser-side fetch or JSON parse failures', async () => {
|
||||
const page = makePage(null);
|
||||
page.evaluate.mockRejectedValueOnce(new SyntaxError('Unexpected token < in JSON'));
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API request failed: Unexpected token < in JSON');
|
||||
await expect(browserFetch(page, 'GET', 'https://creator.douyin.com/api/test')).rejects.toThrow('Douyin API request failed (GET https://creator.douyin.com/api/test): Unexpected token < in JSON');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
|
||||
export function unwrapEvaluateResult(payload) {
|
||||
if (payload && !Array.isArray(payload) && typeof payload === 'object' && 'session' in payload && 'data' in payload) {
|
||||
return payload.data;
|
||||
}
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function requireObjectEvaluateResult(payload, context) {
|
||||
const result = unwrapEvaluateResult(payload);
|
||||
if (!result || Array.isArray(result) || typeof result !== 'object') {
|
||||
throw new CommandExecutionError(`${context}: malformed evaluate payload`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
@@ -56,6 +56,31 @@ function sha256Hex(data) {
|
||||
}
|
||||
return hash.digest('hex');
|
||||
}
|
||||
const CRC32_TABLE = new Uint32Array(256).map((_, index) => {
|
||||
let value = index;
|
||||
for (let bit = 0; bit < 8; bit += 1) {
|
||||
value = (value & 1) ? (0xEDB88320 ^ (value >>> 1)) : (value >>> 1);
|
||||
}
|
||||
return value >>> 0;
|
||||
});
|
||||
function crc32Hex(data) {
|
||||
let crc = 0xffffffff;
|
||||
for (const byte of data) {
|
||||
crc = CRC32_TABLE[(crc ^ byte) & 0xff] ^ (crc >>> 8);
|
||||
}
|
||||
return ((crc ^ 0xffffffff) >>> 0).toString(16).padStart(8, '0');
|
||||
}
|
||||
function gatewayBaseUrl(tosUrl) {
|
||||
const parsedUrl = new URL(tosUrl);
|
||||
return `https://${parsedUrl.host}/upload/v1${parsedUrl.pathname}`;
|
||||
}
|
||||
function gatewayHeaders(auth, uploadHeader, userId = '') {
|
||||
return {
|
||||
Authorization: auth,
|
||||
'X-Storage-U': encodeURIComponent(userId),
|
||||
...(uploadHeader ?? {}),
|
||||
};
|
||||
}
|
||||
function extractRegionFromHost(host) {
|
||||
// e.g. "tos-cn-i-alisg.volces.com" → "cn-i-alisg"
|
||||
// e.g. "tos-cn-beijing.ivolces.com" → "cn-beijing"
|
||||
@@ -129,6 +154,7 @@ async function tosRequest(opts) {
|
||||
method,
|
||||
headers,
|
||||
body: fetchBody,
|
||||
signal: AbortSignal.timeout(60000),
|
||||
});
|
||||
const responseBody = await res.text();
|
||||
const responseHeaders = {};
|
||||
@@ -140,86 +166,95 @@ async function tosRequest(opts) {
|
||||
function nowDatetime() {
|
||||
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
|
||||
}
|
||||
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
|
||||
async function initMultipartUpload(tosUrl, auth, credentials) {
|
||||
const initUrl = `${tosUrl}?uploads`;
|
||||
const datetime = nowDatetime();
|
||||
// Use the pre-computed auth for INIT, as it comes from ApplyVideoUpload
|
||||
const headers = {
|
||||
Authorization: auth,
|
||||
'x-amz-date': datetime,
|
||||
'x-amz-security-token': credentials.session_token,
|
||||
'content-type': 'application/octet-stream',
|
||||
};
|
||||
const res = await tosRequest({ method: 'POST', url: initUrl, headers });
|
||||
if (res.status !== 200) {
|
||||
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS credentials are valid and not expired.');
|
||||
function extractUploadId(body) {
|
||||
const xmlMatch = body.match(/<UploadId>([^<]+)<\/UploadId>/i);
|
||||
if (xmlMatch) return xmlMatch[1];
|
||||
try {
|
||||
const json = JSON.parse(body);
|
||||
return json?.payload?.uploadID
|
||||
|| json?.payload?.uploadId
|
||||
|| json?.payload?.UploadID
|
||||
|| json?.payload?.UploadId
|
||||
|| json?.data?.uploadid
|
||||
|| json?.data?.uploadID
|
||||
|| json?.data?.uploadId
|
||||
|| json?.data?.UploadID
|
||||
|| json?.data?.UploadId
|
||||
|| json?.UploadID
|
||||
|| json?.UploadId
|
||||
|| json?.uploadID
|
||||
|| json?.uploadId
|
||||
|| null;
|
||||
}
|
||||
// Parse UploadId from XML: <UploadId>...</UploadId>
|
||||
const match = res.body.match(/<UploadId>([^<]+)<\/UploadId>/);
|
||||
if (!match) {
|
||||
catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
// ── Phase 1: Init multipart upload ───────────────────────────────────────────
|
||||
async function initMultipartUpload(tosUrl, auth, uploadHeader, userId) {
|
||||
const initUrl = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=init`;
|
||||
const res = await tosRequest({
|
||||
method: 'POST',
|
||||
url: initUrl,
|
||||
headers: gatewayHeaders(auth, uploadHeader, userId),
|
||||
});
|
||||
if (res.status !== 200) {
|
||||
throw new CommandExecutionError(`TOS init multipart upload failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
|
||||
}
|
||||
const uploadId = extractUploadId(res.body);
|
||||
if (!uploadId) {
|
||||
throw new CommandExecutionError(`TOS init response missing UploadId: ${res.body}`);
|
||||
}
|
||||
return match[1];
|
||||
return uploadId;
|
||||
}
|
||||
// ── Phase 2: Upload a single part ────────────────────────────────────────────
|
||||
async function uploadPart(tosUrl, partNumber, uploadId, data, credentials, region) {
|
||||
const parsedUrl = new URL(tosUrl);
|
||||
parsedUrl.searchParams.set('partNumber', String(partNumber));
|
||||
parsedUrl.searchParams.set('uploadId', uploadId);
|
||||
const url = parsedUrl.toString();
|
||||
const datetime = nowDatetime();
|
||||
const headers = computeAws4Headers({
|
||||
method: 'PUT',
|
||||
url,
|
||||
headers: { 'content-type': 'application/octet-stream' },
|
||||
body: data,
|
||||
credentials,
|
||||
service: 'tos',
|
||||
region,
|
||||
datetime,
|
||||
});
|
||||
const res = await tosRequest({ method: 'PUT', url, headers, body: data });
|
||||
if (res.status !== 200) {
|
||||
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that STS2 credentials are valid and not expired.');
|
||||
async function uploadPart(tosUrl, partNumber, uploadId, data, auth, uploadHeader, userId) {
|
||||
const crc32 = crc32Hex(data);
|
||||
const url = `${gatewayBaseUrl(tosUrl)}?uploadid=${encodeURIComponent(uploadId)}&part_number=${partNumber}&phase=transfer`;
|
||||
const headers = {
|
||||
...gatewayHeaders(auth, uploadHeader, userId),
|
||||
'Content-CRC32': crc32,
|
||||
'Content-Type': 'application/octet-stream',
|
||||
'X-Use-Init-Upload-Optimize': '1',
|
||||
'X-Use-Large-Local-Cache': '1',
|
||||
};
|
||||
const res = await tosRequest({ method: 'POST', url, headers, body: data });
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(res.body);
|
||||
}
|
||||
const etag = res.headers['etag'];
|
||||
if (!etag) {
|
||||
throw new CommandExecutionError(`TOS upload part ${partNumber} response missing ETag header`);
|
||||
catch {
|
||||
parsed = null;
|
||||
}
|
||||
return etag;
|
||||
if (res.status !== 200 || parsed?.code !== 2000) {
|
||||
throw new CommandExecutionError(`TOS upload part ${partNumber} failed with status ${res.status}: ${res.body}`, 'Check that TOS upload authorization is valid and not expired.');
|
||||
}
|
||||
return parsed?.data?.crc32 || crc32;
|
||||
}
|
||||
// ── Phase 3: Complete multipart upload ───────────────────────────────────────
|
||||
async function completeMultipartUpload(tosUrl, uploadId, parts, credentials, region) {
|
||||
const parsedUrl = new URL(tosUrl);
|
||||
parsedUrl.searchParams.set('uploadId', uploadId);
|
||||
const url = parsedUrl.toString();
|
||||
const xmlBody = '<CompleteMultipartUpload>' +
|
||||
parts
|
||||
.sort((a, b) => a.partNumber - b.partNumber)
|
||||
.map(p => `<Part><PartNumber>${p.partNumber}</PartNumber><ETag>${p.etag}</ETag></Part>`)
|
||||
.join('') +
|
||||
'</CompleteMultipartUpload>';
|
||||
const datetime = nowDatetime();
|
||||
const headers = computeAws4Headers({
|
||||
method: 'POST',
|
||||
url,
|
||||
headers: { 'content-type': 'application/xml' },
|
||||
body: xmlBody,
|
||||
credentials,
|
||||
service: 'tos',
|
||||
region,
|
||||
datetime,
|
||||
});
|
||||
async function completeMultipartUpload(tosUrl, uploadId, parts, auth, uploadHeader, userId) {
|
||||
const url = `${gatewayBaseUrl(tosUrl)}?uploadmode=part&phase=finish&uploadid=${encodeURIComponent(uploadId)}`;
|
||||
const body = parts
|
||||
.sort((a, b) => a.partNumber - b.partNumber)
|
||||
.map(p => `${p.partNumber}:${p.crc32}`)
|
||||
.join(',');
|
||||
const res = await tosRequest({
|
||||
method: 'POST',
|
||||
url,
|
||||
headers,
|
||||
body: xmlBody,
|
||||
headers: gatewayHeaders(auth, uploadHeader, userId),
|
||||
body,
|
||||
});
|
||||
if (res.status !== 200) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(res.body);
|
||||
}
|
||||
catch {
|
||||
parsed = null;
|
||||
}
|
||||
if (res.status !== 200 || parsed?.code !== 2000) {
|
||||
throw new CommandExecutionError(`TOS complete multipart upload failed with status ${res.status}: ${res.body}`, 'Check that all parts were uploaded successfully.');
|
||||
}
|
||||
return parsed?.data?.key || null;
|
||||
}
|
||||
let _readSyncOverride = null;
|
||||
/** @internal — for testing only */
|
||||
@@ -237,7 +272,7 @@ export async function tosUpload(options) {
|
||||
if (fileSize === 0) {
|
||||
throw new CommandExecutionError(`Video file is empty: ${filePath}`);
|
||||
}
|
||||
const { tos_upload_url: tosUrl, auth } = uploadInfo;
|
||||
const { tos_upload_url: tosUrl, auth, upload_header: uploadHeader, user_id: userId } = uploadInfo;
|
||||
const parsedTosUrl = new URL(tosUrl);
|
||||
const region = extractRegionFromHost(parsedTosUrl.host);
|
||||
const resumePath = getResumeFilePath(filePath);
|
||||
@@ -251,7 +286,7 @@ export async function tosUpload(options) {
|
||||
}
|
||||
else {
|
||||
// Start fresh
|
||||
uploadId = await initMultipartUpload(tosUrl, auth, credentials);
|
||||
uploadId = await initMultipartUpload(tosUrl, auth, uploadHeader, userId);
|
||||
completedParts = [];
|
||||
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
|
||||
}
|
||||
@@ -277,8 +312,8 @@ export async function tosUpload(options) {
|
||||
if (bytesRead !== chunkSize) {
|
||||
throw new CommandExecutionError(`Short read on part ${partNumber}: expected ${chunkSize} bytes, got ${bytesRead}`);
|
||||
}
|
||||
const etag = await uploadPart(tosUrl, partNumber, uploadId, buffer, credentials, region);
|
||||
completedParts.push({ partNumber, etag });
|
||||
const crc32 = await uploadPart(tosUrl, partNumber, uploadId, buffer, auth, uploadHeader, userId);
|
||||
completedParts.push({ partNumber, crc32 });
|
||||
saveResumeState(resumePath, { uploadId, fileSize, parts: completedParts });
|
||||
uploadedBytes = Math.min(offset + chunkSize, fileSize);
|
||||
if (onProgress)
|
||||
@@ -288,8 +323,9 @@ export async function tosUpload(options) {
|
||||
finally {
|
||||
fs.closeSync(fd);
|
||||
}
|
||||
await completeMultipartUpload(tosUrl, uploadId, completedParts, credentials, region);
|
||||
const completedKey = await completeMultipartUpload(tosUrl, uploadId, completedParts, auth, uploadHeader, userId);
|
||||
deleteResumeState(resumePath);
|
||||
return completedKey;
|
||||
}
|
||||
// ── Internal exports for testing ─────────────────────────────────────────────
|
||||
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, };
|
||||
export { PART_SIZE, RESUME_DIR, extractRegionFromHost, getResumeFilePath, loadResumeState, saveResumeState, deleteResumeState, computeAws4Headers, extractUploadId, crc32Hex, gatewayBaseUrl, gatewayHeaders, };
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
import * as crypto from 'node:crypto';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { unwrapEvaluateResult } from './evaluate-result.js';
|
||||
|
||||
const AUTH_V5_URL = 'https://creator.douyin.com/web/api/media/upload/auth/v5/';
|
||||
const VOD_UPLOAD_HOST = 'https://vod.bytedanceapi.com/';
|
||||
const VOD_SPACE_NAME = 'aweme';
|
||||
|
||||
function hmacSha256(key, data) {
|
||||
return crypto.createHmac('sha256', key).update(data, 'utf8').digest();
|
||||
}
|
||||
|
||||
function sha256Hex(data) {
|
||||
const hash = crypto.createHash('sha256');
|
||||
if (Buffer.isBuffer(data) || data instanceof Uint8Array) {
|
||||
hash.update(data);
|
||||
} else {
|
||||
hash.update(data ?? '', 'utf8');
|
||||
}
|
||||
return hash.digest('hex');
|
||||
}
|
||||
|
||||
function nowDatetime() {
|
||||
return new Date().toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z');
|
||||
}
|
||||
|
||||
function canonicalQuery(url) {
|
||||
return [...url.searchParams.entries()]
|
||||
.sort(([a], [b]) => a.localeCompare(b))
|
||||
.map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
|
||||
.join('&');
|
||||
}
|
||||
|
||||
function computeAws4Headers(url, credentials, options = {}) {
|
||||
const parsedUrl = new URL(url);
|
||||
const datetime = nowDatetime();
|
||||
const date = datetime.slice(0, 8);
|
||||
const method = options.method ?? 'GET';
|
||||
const body = options.body ?? '';
|
||||
const bodyHash = sha256Hex(body);
|
||||
const headers = {
|
||||
...(options.headers ?? {}),
|
||||
host: parsedUrl.host,
|
||||
'x-amz-content-sha256': bodyHash,
|
||||
'x-amz-date': datetime,
|
||||
'x-amz-security-token': credentials.session_token,
|
||||
};
|
||||
const sortedHeaderKeys = Object.keys(headers).sort((a, b) => a.localeCompare(b));
|
||||
const canonicalHeaders = sortedHeaderKeys
|
||||
.map((key) => `${key}:${String(headers[key]).trim()}`)
|
||||
.join('\n') + '\n';
|
||||
const signedHeaders = sortedHeaderKeys.join(';');
|
||||
const canonicalRequest = [
|
||||
method,
|
||||
parsedUrl.pathname || '/',
|
||||
canonicalQuery(parsedUrl),
|
||||
canonicalHeaders,
|
||||
signedHeaders,
|
||||
bodyHash,
|
||||
].join('\n');
|
||||
const service = 'vod';
|
||||
const region = 'cn-north-1';
|
||||
const credentialScope = `${date}/${region}/${service}/aws4_request`;
|
||||
const stringToSign = [
|
||||
'AWS4-HMAC-SHA256',
|
||||
datetime,
|
||||
credentialScope,
|
||||
sha256Hex(canonicalRequest),
|
||||
].join('\n');
|
||||
const kDate = hmacSha256(`AWS4${credentials.secret_access_key}`, date);
|
||||
const kRegion = hmacSha256(kDate, region);
|
||||
const kService = hmacSha256(kRegion, service);
|
||||
const kSigning = hmacSha256(kService, 'aws4_request');
|
||||
const signature = hmacSha256(kSigning, stringToSign).toString('hex');
|
||||
return {
|
||||
...headers,
|
||||
Authorization: `AWS4-HMAC-SHA256 Credential=${credentials.access_key_id}/${credentialScope}, SignedHeaders=${signedHeaders}, Signature=${signature}`,
|
||||
};
|
||||
}
|
||||
|
||||
function extractUserIdFromSessionToken(sessionToken) {
|
||||
try {
|
||||
const raw = sessionToken.startsWith('STS2') ? sessionToken.slice(4) : sessionToken;
|
||||
const decoded = JSON.parse(Buffer.from(raw, 'base64').toString('utf8'));
|
||||
const policy = JSON.parse(decoded.PolicyString || '{}');
|
||||
const condition = policy?.Statement?.[0]?.Condition;
|
||||
if (typeof condition === 'string') {
|
||||
const parsedCondition = JSON.parse(condition);
|
||||
return parsedCondition.UserId || '';
|
||||
}
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
export async function getUploadAuthV5Credentials(page) {
|
||||
const result = unwrapEvaluateResult(await page.evaluate(`fetch(${JSON.stringify(AUTH_V5_URL)}, { credentials: 'include' }).then(r => r.json())`));
|
||||
if (!result || Array.isArray(result) || typeof result !== 'object') {
|
||||
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
|
||||
}
|
||||
if (result.status_code !== 0) {
|
||||
const message = result.status_msg ?? result.message ?? 'unknown error';
|
||||
if (result.status_code === 401 || result.status_code === 403 || /login|cookie|auth|captcha|verify|forbidden|permission|登录|登陆|权限|验证|验证码/i.test(String(message))) {
|
||||
throw new AuthRequiredError('creator.douyin.com', `获取抖音上传授权失败: ${message}`);
|
||||
}
|
||||
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
|
||||
}
|
||||
if (!result.auth) {
|
||||
throw new CommandExecutionError(`获取抖音上传授权失败: ${JSON.stringify(result)}`);
|
||||
}
|
||||
let auth;
|
||||
try {
|
||||
auth = JSON.parse(result.auth);
|
||||
} catch (error) {
|
||||
throw new CommandExecutionError(`解析抖音上传授权失败: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
if (!auth.AccessKeyID || !auth.SecretAccessKey || !auth.SessionToken) {
|
||||
throw new CommandExecutionError('抖音上传授权缺少 AccessKeyID/SecretAccessKey/SessionToken');
|
||||
}
|
||||
return {
|
||||
access_key_id: auth.AccessKeyID,
|
||||
secret_access_key: auth.SecretAccessKey,
|
||||
session_token: auth.SessionToken,
|
||||
user_id: extractUserIdFromSessionToken(auth.SessionToken),
|
||||
expired_time: auth.ExpiredTime,
|
||||
current_time: auth.CurrentTime,
|
||||
};
|
||||
}
|
||||
|
||||
export async function applyVideoUploadInner(fileSize, credentials) {
|
||||
const params = new URLSearchParams({
|
||||
Action: 'ApplyUploadInner',
|
||||
Version: '2020-11-19',
|
||||
SpaceName: VOD_SPACE_NAME,
|
||||
FileType: 'video',
|
||||
IsInner: '1',
|
||||
FileSize: String(fileSize),
|
||||
});
|
||||
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
|
||||
const res = await fetch(url, { headers: computeAws4Headers(url, credentials), signal: AbortSignal.timeout(30000) });
|
||||
const text = await res.text();
|
||||
let payload;
|
||||
try {
|
||||
payload = JSON.parse(text);
|
||||
} catch {
|
||||
throw new CommandExecutionError(`申请抖音上传地址失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
|
||||
}
|
||||
const error = payload?.ResponseMetadata?.Error;
|
||||
if (!res.ok || error) {
|
||||
throw new CommandExecutionError(`申请抖音上传地址失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
|
||||
}
|
||||
const uploadNode = payload?.Result?.InnerUploadAddress?.UploadNodes?.[0];
|
||||
const storeInfo = uploadNode?.StoreInfos?.[0];
|
||||
const videoId = payload?.Result?.Vid || uploadNode?.Vid;
|
||||
const sessionKey = uploadNode?.SessionKey ?? storeInfo?.SessionKey ?? payload?.Result?.SessionKey;
|
||||
if (!uploadNode?.UploadHost || !storeInfo?.StoreUri || !storeInfo?.Auth || !videoId || !sessionKey) {
|
||||
throw new CommandExecutionError(`申请抖音上传地址响应缺少必要字段: ${JSON.stringify(payload).slice(0, 500)}`);
|
||||
}
|
||||
return {
|
||||
video_id: videoId,
|
||||
tos_upload_url: `https://${uploadNode.UploadHost}/${storeInfo.StoreUri}`,
|
||||
auth: storeInfo.Auth,
|
||||
session_key: sessionKey,
|
||||
upload_header: uploadNode.UploadHeader ?? {},
|
||||
user_id: credentials.user_id ?? '',
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
export async function commitVideoUploadInner(uploadInfo, credentials) {
|
||||
if (!uploadInfo?.session_key) {
|
||||
throw new CommandExecutionError('抖音上传提交缺少 SessionKey');
|
||||
}
|
||||
const params = new URLSearchParams({
|
||||
Action: 'CommitUploadInner',
|
||||
Version: '2020-11-19',
|
||||
SpaceName: VOD_SPACE_NAME,
|
||||
});
|
||||
const url = `${VOD_UPLOAD_HOST}?${params.toString()}`;
|
||||
const body = JSON.stringify({ SessionKey: uploadInfo.session_key });
|
||||
const headers = computeAws4Headers(url, credentials, {
|
||||
method: 'POST',
|
||||
body,
|
||||
headers: { 'content-type': 'application/json;charset=UTF-8' },
|
||||
});
|
||||
const res = await fetch(url, { method: 'POST', headers, body, signal: AbortSignal.timeout(30000) });
|
||||
const text = await res.text();
|
||||
let payload;
|
||||
try {
|
||||
payload = JSON.parse(text);
|
||||
} catch {
|
||||
throw new CommandExecutionError(`提交抖音上传失败,非 JSON 响应: HTTP ${res.status} ${text.slice(0, 300)}`);
|
||||
}
|
||||
const error = payload?.ResponseMetadata?.Error;
|
||||
if (!res.ok || error) {
|
||||
throw new CommandExecutionError(`提交抖音上传失败: HTTP ${res.status} ${JSON.stringify(error ?? payload)}`);
|
||||
}
|
||||
const result = payload?.Result?.Results?.[0] ?? payload?.Result ?? {};
|
||||
const videoId = result.Vid ?? result.VideoId ?? result.VideoID ?? result.vid ?? uploadInfo.video_id;
|
||||
if (!videoId) {
|
||||
throw new CommandExecutionError(`提交抖音上传响应缺少 video id: ${JSON.stringify(payload).slice(0, 500)}`);
|
||||
}
|
||||
const meta = result.Meta ?? result.VideoMeta ?? {};
|
||||
return {
|
||||
video_id: videoId,
|
||||
poster_uri: result.PosterUri ?? result.PosterURI ?? result.SnapshotUri ?? result.SnapshotURI ?? '',
|
||||
width: Number(meta.Width ?? meta.width ?? 720) || 720,
|
||||
height: Number(meta.Height ?? meta.height ?? 1280) || 1280,
|
||||
raw: result,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { getUploadAuthV5Credentials, applyVideoUploadInner } from './vod-upload.js';
|
||||
|
||||
describe('douyin vod upload helpers', () => {
|
||||
it('parses creator upload auth v5 credentials', async () => {
|
||||
const page = { evaluate: async () => ({ status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token', ExpiredTime: 123, CurrentTime: 100 }) }) };
|
||||
await expect(getUploadAuthV5Credentials(page)).resolves.toEqual({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token', user_id: '', expired_time: 123, current_time: 100 });
|
||||
});
|
||||
|
||||
it('unwraps browser bridge envelopes around upload auth payloads', async () => {
|
||||
const payload = { status_code: 0, auth: JSON.stringify({ AccessKeyID: 'ak', SecretAccessKey: 'sk', SessionToken: 'token' }) };
|
||||
const page = { evaluate: async () => ({ session: 'site:douyin:test', data: payload }) };
|
||||
await expect(getUploadAuthV5Credentials(page)).resolves.toMatchObject({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
|
||||
});
|
||||
|
||||
it('maps upload auth permission errors to AuthRequiredError', async () => {
|
||||
const page = { evaluate: async () => ({ status_code: 401, status_msg: 'login required' }) };
|
||||
await expect(getUploadAuthV5Credentials(page)).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('maps ApplyUploadInner response to TOS upload info', async () => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { RequestId: 'req' }, Result: { InnerUploadAddress: { UploadNodes: [{ Vid: 'video-id', SessionKey: 'session-key', UploadHost: 'tos.example.com', StoreInfos: [{ StoreUri: 'obj/key.mp4', Auth: 'space-auth' }] }] } } }) });
|
||||
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).resolves.toEqual({ video_id: 'video-id', tos_upload_url: 'https://tos.example.com/obj/key.mp4', auth: 'space-auth', session_key: 'session-key', upload_header: {}, user_id: '' });
|
||||
const [url, init] = fetchSpy.mock.calls[0];
|
||||
expect(String(url)).toContain('Action=ApplyUploadInner');
|
||||
expect(String(url)).toContain('Version=2020-11-19');
|
||||
expect(init.headers.Authorization).toContain('AWS4-HMAC-SHA256 Credential=ak/');
|
||||
expect(init.headers['x-amz-security-token']).toBe('token');
|
||||
fetchSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('surfaces VOD API errors with context', async () => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch').mockResolvedValue({ ok: true, status: 200, text: async () => JSON.stringify({ ResponseMetadata: { Error: { Code: 'AccessDenied', Message: 'denied' } } }) });
|
||||
await expect(applyVideoUploadInner(1234, { access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
fetchSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
+137
-4
@@ -1,19 +1,152 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { browserFetch } from './_shared/browser-fetch.js';
|
||||
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
|
||||
|
||||
const CREATOR_MANAGE_URL = 'https://creator.douyin.com/creator-micro/content/manage';
|
||||
const WORK_LIST_URL = '/janus/douyin/creator/pc/work_list?status=0&count=20&max_cursor=0&scene=star_atlas&device_platform=android&aid=1128';
|
||||
|
||||
function readAwemeId(raw) {
|
||||
const value = String(raw ?? '').trim();
|
||||
if (!value) {
|
||||
throw new ArgumentError('douyin delete aweme_id cannot be empty');
|
||||
}
|
||||
if (!/^\d+$/.test(value)) {
|
||||
throw new ArgumentError('douyin delete aweme_id must be a numeric id');
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function sleep(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
|
||||
async function deleteViaCreatorManage(page, workId) {
|
||||
await page.goto(CREATOR_MANAGE_URL);
|
||||
await sleep(3000);
|
||||
await sleep(3000);
|
||||
const result = requireObjectEvaluateResult(await page.evaluate(`
|
||||
(async () => {
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
const targetId = ${JSON.stringify(String(workId))};
|
||||
const textOf = (node) => (node && (node.innerText || node.textContent) || '').trim();
|
||||
const normalize = (value) => String(value || '').replace(/\\s+/g, ' ').trim();
|
||||
|
||||
async function loadTarget() {
|
||||
const res = await fetch(${JSON.stringify(WORK_LIST_URL)}, { credentials: 'include' });
|
||||
const payload = await res.json();
|
||||
const list = Array.isArray(payload.aweme_list) ? payload.aweme_list : [];
|
||||
const matches = list
|
||||
.map((entry, index) => ({ entry, index }))
|
||||
.filter(({ entry }) => String(entry.aweme_id || '') === targetId || String(entry.item_id || '') === targetId);
|
||||
if (matches.length === 0) {
|
||||
return { ok: false, reason: 'not_found', status_code: payload.status_code, count: list.length };
|
||||
}
|
||||
if (matches.length !== 1) {
|
||||
return { ok: false, reason: 'target_not_unique', count: matches.length };
|
||||
}
|
||||
const { entry: item, index } = matches[0];
|
||||
const title = normalize(item.desc || item.caption || item.title || item.item_title || '');
|
||||
return { ok: true, item, index, listCount: list.length, title };
|
||||
}
|
||||
|
||||
function visibleWorkCards() {
|
||||
const candidates = Array.from(document.querySelectorAll('[class*="video-card"]'))
|
||||
.filter((element) => {
|
||||
const text = normalize(textOf(element));
|
||||
return text.includes('删除作品') && text.includes('继续编辑');
|
||||
});
|
||||
return candidates.filter((candidate) => !candidates.some((other) => other !== candidate && other.contains(candidate)));
|
||||
}
|
||||
|
||||
const target = await loadTarget();
|
||||
if (!target.ok) return target;
|
||||
|
||||
const allTab = Array.from(document.querySelectorAll('button,[role="button"],span,div'))
|
||||
.find((element) => /^全部作品$/.test(normalize(textOf(element))));
|
||||
allTab?.click();
|
||||
await sleep(1000);
|
||||
for (let attempt = 0; attempt < 20; attempt += 1) {
|
||||
const cards = visibleWorkCards();
|
||||
if (cards.length >= target.listCount && cards[target.index]) {
|
||||
const card = cards[target.index];
|
||||
const deleteButton = Array.from(card.querySelectorAll('button,[role="button"],span,div'))
|
||||
.find((element) => /^删除作品$/.test(normalize(textOf(element))));
|
||||
if (!deleteButton) return { ok: false, reason: 'delete_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, cardCount: cards.length };
|
||||
deleteButton.click();
|
||||
await sleep(800);
|
||||
const confirmButton = Array.from(document.querySelectorAll('button,[role="button"]'))
|
||||
.find((element) => ['确定', '确认', '删除'].includes(normalize(textOf(element))));
|
||||
if (!confirmButton) return { ok: false, reason: 'confirm_button_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
|
||||
confirmButton.click();
|
||||
for (let wait = 0; wait < 20; wait += 1) {
|
||||
await sleep(500);
|
||||
const after = await loadTarget();
|
||||
if (!after.ok && after.reason === 'not_found') {
|
||||
return { ok: true, aweme_id: target.item.aweme_id, item_id: target.item.item_id, title: target.title };
|
||||
}
|
||||
}
|
||||
return { ok: false, reason: 'delete_not_confirmed', aweme_id: target.item.aweme_id, item_id: target.item.item_id };
|
||||
}
|
||||
await sleep(500);
|
||||
}
|
||||
return { ok: false, reason: 'card_not_found', aweme_id: target.item.aweme_id, item_id: target.item.item_id, index: target.index, listCount: target.listCount };
|
||||
})()
|
||||
`), '抖音后台管理删除响应异常');
|
||||
|
||||
if (!result?.ok) {
|
||||
throw new CommandExecutionError(`抖音后台管理删除失败: ${JSON.stringify(result)}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
async function findWorkListItem(page, workId) {
|
||||
const data = await browserFetch(page, 'GET', `https://creator.douyin.com${WORK_LIST_URL}`, { timeoutMs: 8000 });
|
||||
const list = data.data?.work_list ?? data.aweme_list ?? data.work_list ?? [];
|
||||
if (!Array.isArray(list)) {
|
||||
throw new CommandExecutionError('抖音作品列表响应缺少 work_list/aweme_list');
|
||||
}
|
||||
return list.find((entry) => String(entry.aweme_id || '') === workId || String(entry.item_id || '') === workId) || null;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'douyin',
|
||||
name: 'delete',
|
||||
access: 'write',
|
||||
description: '删除作品',
|
||||
description: '删除作品(优先使用创作者后台作品管理;找不到时回退到旧删除接口)',
|
||||
domain: 'creator.douyin.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
siteSession: 'persistent',
|
||||
args: [
|
||||
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID' },
|
||||
{ name: 'aweme_id', required: true, positional: true, help: '作品 ID / item_id' },
|
||||
],
|
||||
columns: ['status'],
|
||||
func: async (page, kwargs) => {
|
||||
const awemeId = readAwemeId(kwargs.aweme_id);
|
||||
try {
|
||||
const deleted = await deleteViaCreatorManage(page, awemeId);
|
||||
return [{ status: `✅ 已通过后台管理删除 ${deleted.aweme_id || awemeId}` }];
|
||||
} catch (fallbackError) {
|
||||
const fallbackMessage = fallbackError instanceof Error ? fallbackError.message : String(fallbackError);
|
||||
if (!fallbackMessage.includes('"reason":"not_found"')) {
|
||||
throw fallbackError;
|
||||
}
|
||||
}
|
||||
|
||||
const before = await findWorkListItem(page, awemeId);
|
||||
if (!before) {
|
||||
throw new CommandExecutionError(`抖音作品 ${awemeId} 未在作品列表中找到,未执行删除`);
|
||||
}
|
||||
const url = 'https://creator.douyin.com/web/api/media/aweme/delete/?aid=1128';
|
||||
await browserFetch(page, 'POST', url, { body: { aweme_id: kwargs.aweme_id } });
|
||||
return [{ status: `✅ 已删除 ${kwargs.aweme_id}` }];
|
||||
await browserFetch(page, 'POST', url, { body: { aweme_id: awemeId }, timeoutMs: 8000 });
|
||||
const deadline = Date.now() + 10_000;
|
||||
while (Date.now() < deadline) {
|
||||
await sleep(500);
|
||||
const after = await findWorkListItem(page, awemeId);
|
||||
if (!after) {
|
||||
return [{ status: `✅ 已删除 ${awemeId}` }];
|
||||
}
|
||||
}
|
||||
throw new CommandExecutionError(`抖音作品 ${awemeId} 删除后仍在作品列表中,删除未确认`);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,11 +1,100 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
browserFetch: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
|
||||
|
||||
import './delete.js';
|
||||
|
||||
function makePage({ evaluateResult, listBefore = [], listAfter = [] } = {}) {
|
||||
let listCalls = 0;
|
||||
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
|
||||
if (method === 'GET' && String(url).includes('/work_list?')) {
|
||||
listCalls += 1;
|
||||
return { aweme_list: listCalls === 1 ? listBefore : listAfter };
|
||||
}
|
||||
return { status_code: 0 };
|
||||
});
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult ?? { ok: false, reason: 'not_found' }),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
}
|
||||
|
||||
describe('douyin delete registration', () => {
|
||||
const command = getRegistry().get('douyin/delete');
|
||||
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
vi.useFakeTimers();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('registers the delete command', () => {
|
||||
const registry = getRegistry();
|
||||
const values = [...registry.values()];
|
||||
const cmd = values.find(c => c.site === 'douyin' && c.name === 'delete');
|
||||
expect(cmd).toBeDefined();
|
||||
});
|
||||
|
||||
it('uses work_list id/index matching instead of title matching for fallback deletion', () => {
|
||||
const source = readFileSync(new URL('./delete.js', import.meta.url), 'utf8');
|
||||
expect(source).toContain('target_not_unique');
|
||||
expect(source).toContain("String(entry.aweme_id || '') === targetId");
|
||||
expect(source).toContain('cards[target.index]');
|
||||
expect(source).not.toContain('text.includes(target.title)');
|
||||
});
|
||||
|
||||
it('validates aweme_id before navigation', async () => {
|
||||
const page = makePage();
|
||||
await expect(command.func(page, { aweme_id: '' })).rejects.toBeInstanceOf(ArgumentError);
|
||||
await expect(command.func(page, { aweme_id: 'abc' })).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not treat a missing work as successful delete', async () => {
|
||||
const page = makePage({ listBefore: [], listAfter: [] });
|
||||
const promise = command.func(page, { aweme_id: '123' });
|
||||
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await vi.advanceTimersByTimeAsync(7000);
|
||||
await assertion;
|
||||
});
|
||||
|
||||
it('unwraps Browser Bridge envelopes around creator manage delete results', async () => {
|
||||
const page = makePage({ evaluateResult: { session: 'site:douyin:test', data: { ok: true, aweme_id: '123' } } });
|
||||
const promise = command.func(page, { aweme_id: '123' });
|
||||
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已通过后台管理删除 123' }]);
|
||||
await vi.advanceTimersByTimeAsync(7000);
|
||||
await assertion;
|
||||
expect(mocks.browserFetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws typed on malformed creator manage delete result', async () => {
|
||||
const page = makePage({ evaluateResult: 'bad-shape' });
|
||||
const promise = command.func(page, { aweme_id: '123' });
|
||||
const assertion = expect(promise).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await vi.advanceTimersByTimeAsync(7000);
|
||||
await assertion;
|
||||
expect(mocks.browserFetch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns success only after fallback delete postcondition removes the target', async () => {
|
||||
const page = makePage({
|
||||
listBefore: [{ aweme_id: '123' }],
|
||||
listAfter: [],
|
||||
});
|
||||
const promise = command.func(page, { aweme_id: '123' });
|
||||
const assertion = expect(promise).resolves.toEqual([{ status: '✅ 已删除 123' }]);
|
||||
await vi.advanceTimersByTimeAsync(8000);
|
||||
await assertion;
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,170 @@
|
||||
import * as fs from 'node:fs';
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
browserFetch: vi.fn(),
|
||||
getUploadAuthV5Credentials: vi.fn(),
|
||||
applyVideoUploadInner: vi.fn(),
|
||||
commitVideoUploadInner: vi.fn(),
|
||||
tosUpload: vi.fn(),
|
||||
pollTranscode: vi.fn(),
|
||||
imagexUpload: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('./_shared/browser-fetch.js', () => ({ browserFetch: mocks.browserFetch }));
|
||||
vi.mock('./_shared/vod-upload.js', () => ({
|
||||
getUploadAuthV5Credentials: mocks.getUploadAuthV5Credentials,
|
||||
applyVideoUploadInner: mocks.applyVideoUploadInner,
|
||||
commitVideoUploadInner: mocks.commitVideoUploadInner,
|
||||
}));
|
||||
vi.mock('./_shared/tos-upload.js', () => ({ tosUpload: mocks.tosUpload }));
|
||||
vi.mock('./_shared/transcode.js', () => ({ pollTranscode: mocks.pollTranscode }));
|
||||
vi.mock('./_shared/imagex-upload.js', () => ({ imagexUpload: mocks.imagexUpload }));
|
||||
|
||||
describe('douyin publish upload identifier handling', () => {
|
||||
beforeEach(() => {
|
||||
vi.resetModules();
|
||||
vi.clearAllMocks();
|
||||
mocks.getUploadAuthV5Credentials.mockResolvedValue({ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' });
|
||||
mocks.applyVideoUploadInner.mockResolvedValue({ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' });
|
||||
mocks.commitVideoUploadInner.mockResolvedValue({ video_id: 'canonical-video-id', poster_uri: 'poster-uri' });
|
||||
mocks.tosUpload.mockResolvedValue('object-key-returned-by-complete');
|
||||
mocks.pollTranscode.mockResolvedValue({ width: 720, height: 1280, poster_uri: 'poster-uri' });
|
||||
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
|
||||
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { aweme_id: 'aweme-1' };
|
||||
return { status_code: 0 };
|
||||
});
|
||||
});
|
||||
|
||||
it('uses CommitUploadInner Vid for create_v2, not the completed TOS object key', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-id-'));
|
||||
const video = path.join(tmpDir, 'video.mp4');
|
||||
fs.writeFileSync(video, Buffer.from('fake-video'));
|
||||
|
||||
const { getRegistry } = await import('@jackwener/opencli/registry');
|
||||
getRegistry().delete('douyin/publish');
|
||||
await import('./publish.js');
|
||||
const cmd = getRegistry().get('douyin/publish');
|
||||
if (!cmd) throw new Error('douyin publish command not registered');
|
||||
|
||||
await cmd.func({}, {
|
||||
video,
|
||||
title: 'OpenCLI自测',
|
||||
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
|
||||
caption: '',
|
||||
visibility: 'private',
|
||||
no_safety_check: true,
|
||||
});
|
||||
|
||||
expect(mocks.commitVideoUploadInner).toHaveBeenCalledWith(
|
||||
{ video_id: 'apply-video-id', tos_upload_url: 'https://tos.example.com/bucket/key', auth: 'auth', session_key: 'session-key' },
|
||||
{ access_key_id: 'ak', secret_access_key: 'sk', session_token: 'token' },
|
||||
);
|
||||
expect(mocks.pollTranscode).not.toHaveBeenCalled();
|
||||
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
|
||||
expect(createCall?.[3]?.body.item.common.video_id).toBe('canonical-video-id');
|
||||
expect(createCall?.[3]?.body.item.common.video_id).not.toBe('object-key-returned-by-complete');
|
||||
});
|
||||
|
||||
it('continues to create_v2 when the legacy fast detect API returns an empty response', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-safety-'));
|
||||
const video = path.join(tmpDir, 'video.mp4');
|
||||
fs.writeFileSync(video, Buffer.from('fake-video'));
|
||||
mocks.browserFetch.mockImplementation(async (_page, method, url) => {
|
||||
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/pre_check')) {
|
||||
throw new Error('Empty response from Douyin API (POST https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check)');
|
||||
}
|
||||
if (method === 'POST' && String(url).includes('/post_assistant/fast_detect/poll')) return { status: -1, has_done: true, detect_result: { reason_code: 0 }, detect_list: [] };
|
||||
if (method === 'POST' && String(url).includes('/aweme/create_v2/')) return { item_id: 'item-1' };
|
||||
return { status_code: 0 };
|
||||
});
|
||||
|
||||
const { getRegistry } = await import('@jackwener/opencli/registry');
|
||||
getRegistry().delete('douyin/publish');
|
||||
await import('./publish.js');
|
||||
const cmd = getRegistry().get('douyin/publish');
|
||||
if (!cmd) throw new Error('douyin publish command not registered');
|
||||
|
||||
await cmd.func({}, {
|
||||
video,
|
||||
title: 'OpenCLI自测',
|
||||
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
|
||||
visibility: 'public',
|
||||
caption: 'caption',
|
||||
no_safety_check: false,
|
||||
});
|
||||
|
||||
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/post_assistant/fast_detect/pre_check'))).toBe(true);
|
||||
expect(mocks.browserFetch.mock.calls.some((call) => String(call[2]).includes('/aweme/create_v2/'))).toBe(true);
|
||||
});
|
||||
|
||||
it('unwraps Browser Bridge envelopes around cover ImageX evaluate results', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-'));
|
||||
const video = path.join(tmpDir, 'video.mp4');
|
||||
const cover = path.join(tmpDir, 'cover.jpg');
|
||||
fs.writeFileSync(video, Buffer.from('fake-video'));
|
||||
fs.writeFileSync(cover, Buffer.from('fake-cover'));
|
||||
mocks.imagexUpload.mockResolvedValue('cover-store-uri');
|
||||
|
||||
const page = {
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce({
|
||||
session: 'site:douyin:test',
|
||||
data: { Result: { UploadAddress: { StoreInfos: [{ UploadHost: 'imagex.example.com', StoreUri: 'cover/key.jpg' }] } } },
|
||||
})
|
||||
.mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: {} } }),
|
||||
};
|
||||
|
||||
const { getRegistry } = await import('@jackwener/opencli/registry');
|
||||
getRegistry().delete('douyin/publish');
|
||||
await import('./publish.js');
|
||||
const cmd = getRegistry().get('douyin/publish');
|
||||
if (!cmd) throw new Error('douyin publish command not registered');
|
||||
|
||||
await cmd.func(page, {
|
||||
video,
|
||||
cover,
|
||||
title: 'OpenCLI自测',
|
||||
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
|
||||
caption: '',
|
||||
visibility: 'private',
|
||||
no_safety_check: true,
|
||||
});
|
||||
|
||||
expect(mocks.imagexUpload).toHaveBeenCalledWith(cover, {
|
||||
upload_url: 'https://imagex.example.com/cover/key.jpg',
|
||||
store_uri: 'cover/key.jpg',
|
||||
});
|
||||
const createCall = mocks.browserFetch.mock.calls.find((call) => String(call[2]).includes('/aweme/create_v2/'));
|
||||
expect(createCall?.[3]?.body.item.cover.poster).toBe('cover-store-uri');
|
||||
});
|
||||
|
||||
it('throws typed when cover ImageX apply returns the wrong shape', async () => {
|
||||
const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'douyin-publish-cover-bad-'));
|
||||
const video = path.join(tmpDir, 'video.mp4');
|
||||
const cover = path.join(tmpDir, 'cover.jpg');
|
||||
fs.writeFileSync(video, Buffer.from('fake-video'));
|
||||
fs.writeFileSync(cover, Buffer.from('fake-cover'));
|
||||
|
||||
const page = { evaluate: vi.fn().mockResolvedValueOnce({ session: 'site:douyin:test', data: { Result: { UploadAddress: { StoreInfos: [] } } } }) };
|
||||
|
||||
const { getRegistry } = await import('@jackwener/opencli/registry');
|
||||
getRegistry().delete('douyin/publish');
|
||||
await import('./publish.js');
|
||||
const cmd = getRegistry().get('douyin/publish');
|
||||
if (!cmd) throw new Error('douyin publish command not registered');
|
||||
|
||||
await expect(cmd.func(page, {
|
||||
video,
|
||||
cover,
|
||||
title: 'OpenCLI自测',
|
||||
schedule: new Date(Date.now() + 3 * 60 * 60 * 1000).toISOString(),
|
||||
caption: '',
|
||||
visibility: 'private',
|
||||
no_safety_check: true,
|
||||
})).rejects.toThrow('UploadHost/StoreUri');
|
||||
expect(mocks.imagexUpload).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+88
-42
@@ -2,7 +2,7 @@
|
||||
* Douyin publish — 8-phase pipeline for scheduling video posts.
|
||||
*
|
||||
* Phases:
|
||||
* 1. STS2 credentials
|
||||
* 1. upload auth v5 credentials
|
||||
* 2. Apply TOS upload URL
|
||||
* 3. TOS multipart upload
|
||||
* 4. Cover upload (optional, via ImageX)
|
||||
@@ -15,11 +15,11 @@ import * as fs from 'node:fs';
|
||||
import * as path from 'node:path';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { getSts2Credentials } from './_shared/sts2.js';
|
||||
import { getUploadAuthV5Credentials, applyVideoUploadInner, commitVideoUploadInner } from './_shared/vod-upload.js';
|
||||
import { tosUpload } from './_shared/tos-upload.js';
|
||||
import { imagexUpload } from './_shared/imagex-upload.js';
|
||||
import { pollTranscode } from './_shared/transcode.js';
|
||||
import { browserFetch } from './_shared/browser-fetch.js';
|
||||
import { requireObjectEvaluateResult } from './_shared/evaluate-result.js';
|
||||
import { generateCreationId } from './_shared/creation-id.js';
|
||||
import { validateTiming, toUnixSeconds } from './_shared/timing.js';
|
||||
import { parseTextExtra, extractHashtagNames } from './_shared/text-extra.js';
|
||||
@@ -54,6 +54,36 @@ const DEFAULT_COVER_TOOLS_INFO = JSON.stringify({
|
||||
initial_cover_uri: '',
|
||||
cut_coordinate: '',
|
||||
});
|
||||
function isFastDetectRetryable(error) {
|
||||
const message = error instanceof Error ? error.message : String(error);
|
||||
return message.includes('post_assistant/fast_detect') && (message.includes('Empty response') || message.includes('404') || message.includes('Not Found') || message.includes('Timeout') || message.includes('timed out') || message.includes('Failed to fetch'));
|
||||
}
|
||||
function sleep(ms) {
|
||||
return new Promise((resolve) => setTimeout(resolve, ms));
|
||||
}
|
||||
function throwIfImagexError(action, payload) {
|
||||
const error = payload?.ResponseMetadata?.Error ?? payload?.Error;
|
||||
if (error) {
|
||||
throw new CommandExecutionError(`${action}失败: ${JSON.stringify(error)}`);
|
||||
}
|
||||
}
|
||||
async function tryFastDetectFetch(page, method, url, options) {
|
||||
let lastError;
|
||||
for (let attempt = 1; attempt <= 3; attempt += 1) {
|
||||
try {
|
||||
return { ok: true, value: await browserFetch(page, method, url, options) };
|
||||
} catch (error) {
|
||||
if (!isFastDetectRetryable(error)) {
|
||||
throw error;
|
||||
}
|
||||
lastError = error;
|
||||
if (attempt < 3) {
|
||||
await sleep(500 * attempt);
|
||||
}
|
||||
}
|
||||
}
|
||||
return { ok: false, error: lastError };
|
||||
}
|
||||
cli({
|
||||
site: 'douyin',
|
||||
name: 'publish',
|
||||
@@ -106,19 +136,13 @@ cli({
|
||||
throw new ArgumentError(`封面文件不存在: ${path.resolve(coverPath)}`);
|
||||
}
|
||||
}
|
||||
// ── Phase 1: STS2 credentials ───────────────────────────────────────
|
||||
const credentials = await getSts2Credentials(page);
|
||||
// ── Phase 1: upload credentials ────────────────────────────────────
|
||||
const credentials = await getUploadAuthV5Credentials(page);
|
||||
// ── Phase 2: Apply TOS upload URL ───────────────────────────────────
|
||||
const vodUrl = `https://vod.bytedanceapi.com/?Action=ApplyVideoUpload&ServiceId=1128&Version=2021-01-01&FileType=video&FileSize=${fileSize}`;
|
||||
const vodJs = `fetch(${JSON.stringify(vodUrl)}, { credentials: 'include' }).then(r => r.json())`;
|
||||
const vodRes = (await page.evaluate(vodJs));
|
||||
const { VideoId: videoId, UploadHosts, StoreInfos } = vodRes.Result.UploadAddress;
|
||||
const tosUrl = `https://${UploadHosts[0]}/${StoreInfos[0].StoreUri}`;
|
||||
const tosUploadInfo = {
|
||||
tos_upload_url: tosUrl,
|
||||
auth: StoreInfos[0].Auth,
|
||||
video_id: videoId,
|
||||
};
|
||||
const tosUploadInfo = await applyVideoUploadInner(fileSize, credentials);
|
||||
let coverUri = '';
|
||||
let coverWidth = 720;
|
||||
let coverHeight = 1280;
|
||||
// ── Phase 3: TOS upload ─────────────────────────────────────────────
|
||||
await tosUpload({
|
||||
filePath: videoPath,
|
||||
@@ -130,22 +154,32 @@ cli({
|
||||
},
|
||||
});
|
||||
process.stderr.write('\n');
|
||||
process.stderr.write(' 提交上传...\n');
|
||||
const committedVideo = await commitVideoUploadInner(tosUploadInfo, credentials);
|
||||
const videoId = committedVideo.video_id;
|
||||
process.stderr.write(` 上传已提交: ${videoId}\n`);
|
||||
coverWidth = committedVideo.width || coverWidth;
|
||||
coverHeight = committedVideo.height || coverHeight;
|
||||
if (!coverUri && committedVideo.poster_uri) {
|
||||
coverUri = committedVideo.poster_uri;
|
||||
}
|
||||
// ── Phase 4: Cover upload (optional) ────────────────────────────────
|
||||
let coverUri = '';
|
||||
let coverWidth = 720;
|
||||
let coverHeight = 1280;
|
||||
if (kwargs.cover) {
|
||||
const resolvedCoverPath = path.resolve(kwargs.cover);
|
||||
// 4A: Apply ImageX upload
|
||||
const applyUrl = `${IMAGEX_BASE}/?Action=ApplyImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01&UploadNum=1`;
|
||||
const applyJs = `fetch(${JSON.stringify(applyUrl)}, { credentials: 'include' }).then(r => r.json())`;
|
||||
const applyRes = (await page.evaluate(applyJs));
|
||||
const { StoreInfos: imgStoreInfos } = applyRes.Result.UploadAddress;
|
||||
const imgUploadUrl = `https://${imgStoreInfos[0].UploadHost}/${imgStoreInfos[0].StoreUri}`;
|
||||
const applyRes = requireObjectEvaluateResult(await page.evaluate(applyJs), '抖音封面申请上传地址响应异常');
|
||||
throwIfImagexError('抖音封面申请上传地址', applyRes);
|
||||
const imgStoreInfo = applyRes.Result?.UploadAddress?.StoreInfos?.[0];
|
||||
if (!imgStoreInfo?.UploadHost || !imgStoreInfo?.StoreUri) {
|
||||
throw new CommandExecutionError(`抖音封面申请上传地址响应缺少 UploadHost/StoreUri: ${JSON.stringify(applyRes).slice(0, 500)}`);
|
||||
}
|
||||
const imgUploadUrl = `https://${imgStoreInfo.UploadHost}/${imgStoreInfo.StoreUri}`;
|
||||
// 4B: Upload image
|
||||
const coverStoreUri = await imagexUpload(resolvedCoverPath, {
|
||||
upload_url: imgUploadUrl,
|
||||
store_uri: imgStoreInfos[0].StoreUri,
|
||||
store_uri: imgStoreInfo.StoreUri,
|
||||
});
|
||||
// 4C: Commit ImageX upload
|
||||
const commitUrl = `${IMAGEX_BASE}/?Action=CommitImageUpload&ServiceId=${IMAGEX_SERVICE_ID}&Version=2018-08-01`;
|
||||
@@ -158,19 +192,13 @@ cli({
|
||||
body: ${JSON.stringify(commitBody)}
|
||||
}).then(r => r.json())
|
||||
`;
|
||||
await page.evaluate(commitJs);
|
||||
const commitRes = requireObjectEvaluateResult(await page.evaluate(commitJs), '抖音封面提交上传响应异常');
|
||||
throwIfImagexError('抖音封面提交上传', commitRes);
|
||||
coverUri = coverStoreUri;
|
||||
}
|
||||
// ── Phase 5: Enable video ───────────────────────────────────────────
|
||||
const enableUrl = `https://creator.douyin.com/web/api/media/video/enable/?video_id=${videoId}&aid=1128`;
|
||||
await browserFetch(page, 'GET', enableUrl);
|
||||
// ── Phase 6: Poll transcode ─────────────────────────────────────────
|
||||
const transResult = await pollTranscode(page, videoId);
|
||||
coverWidth = transResult.width;
|
||||
coverHeight = transResult.height;
|
||||
if (!coverUri) {
|
||||
coverUri = transResult.poster_uri;
|
||||
}
|
||||
// The gateway upload flow returns a committed VOD upload result; the legacy
|
||||
// enable/transend endpoints can hang for that flow, so create_v2 consumes
|
||||
// the committed video_id and poster metadata directly.
|
||||
// ── Phase 7: Content safety check ───────────────────────────────────
|
||||
if (!kwargs.no_safety_check) {
|
||||
const safetyUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/pre_check';
|
||||
@@ -179,25 +207,42 @@ cli({
|
||||
title,
|
||||
desc: caption,
|
||||
};
|
||||
await browserFetch(page, 'POST', safetyUrl, { body: safetyBody });
|
||||
const preCheck = await tryFastDetectFetch(page, 'POST', safetyUrl, { body: safetyBody });
|
||||
if (!preCheck.ok) {
|
||||
process.stderr.write(' 内容安全预检接口无响应,继续轮询检测结果。\n');
|
||||
}
|
||||
const pollUrl = 'https://creator.douyin.com/aweme/v1/post_assistant/fast_detect/poll';
|
||||
const deadline = Date.now() + 30_000;
|
||||
let safetyPassed = false;
|
||||
let pollUnavailableCount = 0;
|
||||
while (Date.now() < deadline) {
|
||||
const pollRes = (await browserFetch(page, 'POST', pollUrl, {
|
||||
body: safetyBody,
|
||||
}));
|
||||
if (pollRes.status === 0) {
|
||||
const poll = await tryFastDetectFetch(page, 'POST', pollUrl, { body: safetyBody });
|
||||
if (!poll.ok) {
|
||||
pollUnavailableCount += 1;
|
||||
if (!preCheck.ok && pollUnavailableCount >= 3) {
|
||||
break;
|
||||
}
|
||||
await sleep(2000);
|
||||
continue;
|
||||
}
|
||||
pollUnavailableCount = 0;
|
||||
const pollRes = poll.value;
|
||||
if (pollRes.status === 0 || (pollRes.has_done === true && pollRes.detect_result?.reason_code === 0 && (pollRes.detect_list?.length ?? 0) === 0)) {
|
||||
safetyPassed = true;
|
||||
break;
|
||||
}
|
||||
if (pollRes.status === 1) {
|
||||
throw new CommandExecutionError('内容安全检测不通过,请修改后重试', '使用 --no_safety_check 跳过');
|
||||
}
|
||||
await new Promise((r) => setTimeout(r, 2000));
|
||||
await sleep(2000);
|
||||
}
|
||||
if (!safetyPassed) {
|
||||
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '使用 --no_safety_check 跳过');
|
||||
if (!preCheck.ok && pollUnavailableCount >= 3) {
|
||||
process.stderr.write(' 内容安全预检持续无响应,跳过本地预检,交由 create_v2 后的平台审核。\n');
|
||||
}
|
||||
else {
|
||||
throw new CommandExecutionError('内容安全检测超时(30s),请稍后重试', '如确认要跳过本地预检,可使用 --no_safety_check;提交后仍会走抖音平台审核');
|
||||
}
|
||||
}
|
||||
}
|
||||
// ── Phase 8: create_v2 publish ──────────────────────────────────────
|
||||
@@ -266,12 +311,13 @@ cli({
|
||||
},
|
||||
};
|
||||
const publishUrl = `https://creator.douyin.com/web/api/media/aweme/create_v2/?read_aid=2906&${DEVICE_PARAMS}`;
|
||||
process.stderr.write(' 创建定时发布...\n');
|
||||
const publishRes = (await browserFetch(page, 'POST', publishUrl, {
|
||||
body: publishBody,
|
||||
}));
|
||||
const awemeId = publishRes.aweme_id;
|
||||
const awemeId = publishRes.aweme_id ?? publishRes.item_id;
|
||||
if (!awemeId) {
|
||||
throw new CommandExecutionError(`发布成功但未返回 aweme_id: ${JSON.stringify(publishRes)}`);
|
||||
throw new CommandExecutionError(`发布成功但未返回 aweme_id/item_id: ${JSON.stringify(publishRes)}`);
|
||||
}
|
||||
const url = `https://www.douyin.com/video/${awemeId}`;
|
||||
const publishTimeStr = new Date(timingTs * 1000).toLocaleString('zh-CN', {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CliError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { fetchDouyinComments, fetchDouyinUserVideos } from './_shared/public-api.js';
|
||||
export const MAX_USER_VIDEOS_LIMIT = 20;
|
||||
export const USER_VIDEO_COMMENT_CONCURRENCY = 4;
|
||||
@@ -27,8 +28,11 @@ async function fetchTopComments(page, awemeId, count) {
|
||||
try {
|
||||
return await fetchDouyinComments(page, awemeId, count);
|
||||
}
|
||||
catch {
|
||||
return [];
|
||||
catch (error) {
|
||||
if (error instanceof CliError) {
|
||||
throw error;
|
||||
}
|
||||
throw new CommandExecutionError(`Failed to fetch Douyin comments for video ${awemeId}: ${error instanceof Error ? error.message : String(error)}`);
|
||||
}
|
||||
}
|
||||
cli({
|
||||
@@ -53,6 +57,9 @@ cli({
|
||||
await page.goto(`https://www.douyin.com/user/${secUid}`);
|
||||
await page.wait(3);
|
||||
const awemeList = (await fetchDouyinUserVideos(page, secUid, limit)).slice(0, limit);
|
||||
if (awemeList.length === 0) {
|
||||
throw new EmptyResultError('douyin user-videos', `No videos were returned for sec_uid ${secUid}. Confirm the user exists and the Douyin session is valid.`);
|
||||
}
|
||||
const videos = withComments
|
||||
? await mapInBatches(awemeList, USER_VIDEO_COMMENT_CONCURRENCY, async (video) => ({
|
||||
...video,
|
||||
|
||||
@@ -8,6 +8,7 @@ vi.mock('./_shared/public-api.js', () => ({
|
||||
fetchDouyinComments: fetchDouyinCommentsMock,
|
||||
}));
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { DEFAULT_COMMENT_LIMIT, MAX_USER_VIDEOS_LIMIT, normalizeCommentLimit, normalizeUserVideosLimit } from './user-videos.js';
|
||||
describe('douyin user-videos', () => {
|
||||
beforeEach(() => {
|
||||
@@ -105,4 +106,46 @@ describe('douyin user-videos', () => {
|
||||
},
|
||||
]);
|
||||
});
|
||||
it('throws EmptyResultError when the user videos API returns no rows', async () => {
|
||||
const command = [...getRegistry().values()].find((cmd) => cmd.site === 'douyin' && cmd.name === 'user-videos');
|
||||
expect(command?.func).toBeDefined();
|
||||
if (!command?.func)
|
||||
throw new Error('douyin user-videos command not registered');
|
||||
fetchDouyinUserVideosMock.mockResolvedValueOnce([]);
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
await expect(command.func(page, {
|
||||
sec_uid: 'MS4w-empty',
|
||||
limit: 3,
|
||||
with_comments: true,
|
||||
comment_limit: 5,
|
||||
})).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
it('surfaces comment enrichment failures instead of returning empty comments', async () => {
|
||||
const command = [...getRegistry().values()].find((cmd) => cmd.site === 'douyin' && cmd.name === 'user-videos');
|
||||
expect(command?.func).toBeDefined();
|
||||
if (!command?.func)
|
||||
throw new Error('douyin user-videos command not registered');
|
||||
fetchDouyinUserVideosMock.mockResolvedValueOnce([
|
||||
{
|
||||
aweme_id: '3',
|
||||
desc: 'comment failure',
|
||||
video: { duration: 2000, play_addr: { url_list: ['https://example.com/fail.mp4'] } },
|
||||
statistics: { digg_count: 1 },
|
||||
},
|
||||
]);
|
||||
fetchDouyinCommentsMock.mockRejectedValueOnce(new Error('comment API down'));
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
await expect(command.func(page, {
|
||||
sec_uid: 'MS4w-test',
|
||||
limit: 3,
|
||||
with_comments: true,
|
||||
comment_limit: 5,
|
||||
})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,131 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError } from '@jackwener/opencli/errors';
|
||||
import {
|
||||
emptySearchResults,
|
||||
requireBoundedInteger,
|
||||
requireNonNegativeInteger,
|
||||
requireRows,
|
||||
requireSearchQuery,
|
||||
runBrowserStep,
|
||||
toHttpsUrl,
|
||||
} from '../_shared/search-adapter.js';
|
||||
|
||||
function decodeDdgUrl(href) {
|
||||
if (!href) return '';
|
||||
try {
|
||||
const url = new URL(href, 'https://duckduckgo.com');
|
||||
const uddg = url.searchParams.get('uddg');
|
||||
return toHttpsUrl(uddg || href, 'https://duckduckgo.com');
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function buildExtractFn(limit) {
|
||||
return 'function(doc){' +
|
||||
'var r=[];var seen={};var items=doc.querySelectorAll(".result");' +
|
||||
'for(var i=0;i<items.length;i++){' +
|
||||
'if(r.length>=' + limit + ')break;' +
|
||||
'var el=items[i];var te=el.querySelector(".result__a");' +
|
||||
'var se=el.querySelector(".result__snippet");' +
|
||||
'var ue=el.querySelector(".result__url");' +
|
||||
'var ie=el.querySelector(".result__icon__img");' +
|
||||
'var cls=el.className||"";var rt="web";' +
|
||||
'if(cls.indexOf("result--ad")!==-1||cls.indexOf("result--ads")!==-1||cls.indexOf("badge--ad")!==-1)continue;' +
|
||||
'if(!te)continue;' +
|
||||
'var t=(te.textContent||"").trim();' +
|
||||
'var h=te.getAttribute("href")||"";' +
|
||||
'var sn=se?(se.textContent||"").trim():"";' +
|
||||
'var du=ue?(ue.textContent||"").trim():"";' +
|
||||
'var ic=ie?(ie.getAttribute("src")||""):"";' +
|
||||
'if(cls.indexOf("news-result")!==-1)rt="news";' +
|
||||
'else if(cls.indexOf("video-result")!==-1)rt="video";' +
|
||||
'else if(cls.indexOf("image-result")!==-1)rt="image";' +
|
||||
'if(!t||!h||seen[h])continue;seen[h]=true;' +
|
||||
'r.push([t,h,sn,du,ic,rt]);' +
|
||||
'}return r;}';
|
||||
}
|
||||
|
||||
function buildExtractorJs(limit) {
|
||||
return '(' + buildExtractFn(limit) + '(document))';
|
||||
}
|
||||
|
||||
function buildPaginateJs(limit, keyword, offset, region) {
|
||||
var params = 'q=' + encodeURIComponent(keyword) + '&s=' + offset + '&v=l&o=json';
|
||||
if (region) params += '&kl=' + encodeURIComponent(region);
|
||||
return (
|
||||
'new Promise(function($r){' +
|
||||
'var x=new XMLHttpRequest();' +
|
||||
'x.open("POST","/html/",true);' +
|
||||
'x.setRequestHeader("Content-Type","application/x-www-form-urlencoded");' +
|
||||
'x.onload=function(){' +
|
||||
'try{var d=new DOMParser().parseFromString(x.responseText,"text/html");' +
|
||||
'$r(' + buildExtractFn(limit) + '(d));' +
|
||||
'}catch(e){$r({error:"parse",message:String(e&&e.message||e)})}' +
|
||||
'};' +
|
||||
'x.onerror=function(){$r({error:"network"})};' +
|
||||
'x.send("' + params + '");' +
|
||||
'})'
|
||||
);
|
||||
}
|
||||
|
||||
const command = cli({
|
||||
site: 'duckduckgo',
|
||||
name: 'search',
|
||||
access: 'read',
|
||||
description: 'Search DuckDuckGo',
|
||||
domain: 'html.duckduckgo.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'keyword', positional: true, required: true, help: 'Search query' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of results per page (1-10). For multi-page, use --offset' },
|
||||
{ name: 'offset', type: 'int', default: 0, help: 'Result offset for pagination (0, 10, 20...). Uses XHR POST internally' },
|
||||
{ name: 'region', help: 'Region code (e.g. jp-jp, us-en, cn-zh). Default: all regions' },
|
||||
{ name: 'time', help: 'Time range: d (day), w (week), m (month), y (year)' },
|
||||
],
|
||||
columns: ['rank', 'title', 'url', 'snippet', 'displayUrl', 'icon', 'resultType'],
|
||||
func: async (page, kwargs) => {
|
||||
const limit = requireBoundedInteger(kwargs.limit, 10, 1, 10, '--limit');
|
||||
const keyword = requireSearchQuery(kwargs.keyword);
|
||||
const offset = requireNonNegativeInteger(kwargs.offset, 0, '--offset');
|
||||
if (offset % 10 !== 0) {
|
||||
throw new ArgumentError('--offset must be a multiple of 10 for DuckDuckGo HTML pagination');
|
||||
}
|
||||
if (kwargs.time && !/^(d|w|m|y)$/.test(String(kwargs.time))) {
|
||||
throw new ArgumentError('--time must be one of d, w, m, or y');
|
||||
}
|
||||
let url = `https://html.duckduckgo.com/html/?q=${encodeURIComponent(keyword)}`;
|
||||
if (kwargs.region) url += `&kl=${encodeURIComponent(String(kwargs.region))}`;
|
||||
if (kwargs.time) url += `&df=${encodeURIComponent(String(kwargs.time))}`;
|
||||
await runBrowserStep('duckduckgo search navigation', () => page.goto(url));
|
||||
try {
|
||||
await page.wait({ selector: '.result', timeout: 8 });
|
||||
} catch {
|
||||
await page.wait(3).catch(function() {});
|
||||
}
|
||||
var raw;
|
||||
if (offset === 0) {
|
||||
raw = await runBrowserStep('duckduckgo search extraction', () => page.evaluate(buildExtractorJs(limit)));
|
||||
} else {
|
||||
raw = await runBrowserStep('duckduckgo search pagination extraction', () => page.evaluate(buildPaginateJs(limit, keyword, offset, kwargs.region)));
|
||||
}
|
||||
const rows = requireRows(raw, 'duckduckgo search');
|
||||
if (rows.length === 0) {
|
||||
throw emptySearchResults('DuckDuckGo', keyword);
|
||||
}
|
||||
return rows.map(function(r, index) {
|
||||
return {
|
||||
rank: index + 1 + offset,
|
||||
title: r[0],
|
||||
url: decodeDdgUrl(r[1]),
|
||||
snippet: r[2],
|
||||
displayUrl: r[3],
|
||||
icon: r[4],
|
||||
resultType: r[5],
|
||||
};
|
||||
}).filter((row) => row.url);
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { command };
|
||||
@@ -0,0 +1,128 @@
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
import { JSDOM } from 'jsdom';
|
||||
|
||||
const { __test__ } = await import('./search.js');
|
||||
const command = __test__.command;
|
||||
|
||||
function createPageMock(evaluateResult = []) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('duckduckgo search', () => {
|
||||
it('should register as a valid command', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('duckduckgo');
|
||||
expect(command.name).toBe('search');
|
||||
expect(command.access).toBe('read');
|
||||
expect(command.browser).toBe(true);
|
||||
expect(command.strategy).toBe('public');
|
||||
expect(command.domain).toBe('html.duckduckgo.com');
|
||||
});
|
||||
|
||||
it('should define keyword positional arg', () => {
|
||||
const kwArg = command.args.find(a => a.name === 'keyword');
|
||||
expect(kwArg).toBeDefined();
|
||||
expect(kwArg.positional).toBe(true);
|
||||
expect(kwArg.required).toBe(true);
|
||||
});
|
||||
|
||||
it('should define limit arg with default 10', () => {
|
||||
const limitArg = command.args.find(a => a.name === 'limit');
|
||||
expect(limitArg).toBeDefined();
|
||||
expect(limitArg.type).toBe('int');
|
||||
expect(limitArg.default).toBe(10);
|
||||
});
|
||||
|
||||
it('should define columns for output', () => {
|
||||
expect(command.columns).toContain('rank');
|
||||
expect(command.columns).toContain('title');
|
||||
expect(command.columns).toContain('url');
|
||||
expect(command.columns).toContain('snippet');
|
||||
expect(command.columns).toContain('displayUrl');
|
||||
expect(command.columns).toContain('icon');
|
||||
expect(command.columns).toContain('resultType');
|
||||
});
|
||||
|
||||
it('rejects empty query and out-of-range pagination before navigation', async () => {
|
||||
const page = createPageMock();
|
||||
await expect(command.func(page, { keyword: ' ', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 11 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 5, offset: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('decodes DuckDuckGo redirect URLs and assigns listing rank', async () => {
|
||||
const page = createPageMock([
|
||||
[
|
||||
'OpenCLI',
|
||||
'/l/?uddg=https%3A%2F%2Fgithub.com%2Fjackwener%2FOpenCLI',
|
||||
'CLI browser tooling',
|
||||
'github.com/jackwener/OpenCLI',
|
||||
'',
|
||||
'web',
|
||||
],
|
||||
]);
|
||||
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).resolves.toEqual([{
|
||||
rank: 1,
|
||||
title: 'OpenCLI',
|
||||
url: 'https://github.com/jackwener/OpenCLI',
|
||||
snippet: 'CLI browser tooling',
|
||||
displayUrl: 'github.com/jackwener/OpenCLI',
|
||||
icon: '',
|
||||
resultType: 'web',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('executes the DOM extractor, filters ads, and returns canonical rows', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<div class="result result--ad">
|
||||
<a class="result__a" href="https://ads.example/">Sponsored result</a>
|
||||
</div>
|
||||
<div class="result">
|
||||
<a class="result__a" href="/l/?uddg=https%3A%2F%2Fexample.com%2Farticle">Organic result</a>
|
||||
<a class="result__snippet">Organic snippet</a>
|
||||
<span class="result__url">example.com/article</span>
|
||||
<img class="result__icon__img" src="https://icons.duckduckgo.com/ip3/example.com.ico">
|
||||
</div>
|
||||
`);
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn(async (source) => Function('document', `return ${source};`)(dom.window.document)),
|
||||
};
|
||||
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 5 })).resolves.toEqual([{
|
||||
rank: 1,
|
||||
title: 'Organic result',
|
||||
url: 'https://example.com/article',
|
||||
snippet: 'Organic snippet',
|
||||
displayUrl: 'example.com/article',
|
||||
icon: 'https://icons.duckduckgo.com/ip3/example.com.ico',
|
||||
resultType: 'web',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('unwraps browser envelopes for paginated extraction', async () => {
|
||||
const page = createPageMock({ session: 'site:duckduckgo', data: [
|
||||
['Result', 'https://example.com/', 'snippet', 'example.com', '', 'web'],
|
||||
] });
|
||||
|
||||
const result = await command.func(page, { keyword: 'opencli', limit: 1, offset: 10 });
|
||||
|
||||
expect(result[0]).toMatchObject({ rank: 11, url: 'https://example.com/' });
|
||||
});
|
||||
|
||||
it('fails typed instead of returning [] for malformed extraction payloads', async () => {
|
||||
const page = createPageMock({ rows: [] });
|
||||
|
||||
await expect(command.func(page, { keyword: 'opencli', limit: 1 })).rejects.toMatchObject({
|
||||
code: 'COMMAND_EXEC',
|
||||
message: expect.stringContaining('payload shape'),
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,45 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { requireBoundedInteger, requireSearchQuery } from '../_shared/search-adapter.js';
|
||||
|
||||
const command = cli({
|
||||
site: 'duckduckgo',
|
||||
name: 'suggest',
|
||||
access: 'read',
|
||||
description: 'DuckDuckGo search suggestions',
|
||||
domain: 'duckduckgo.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: false,
|
||||
args: [
|
||||
{ name: 'keyword', positional: true, required: true, help: 'Search query prefix' },
|
||||
{ name: 'limit', type: 'int', default: 8, help: 'Max number of suggestions' },
|
||||
],
|
||||
columns: ['phrase'],
|
||||
func: async (kwargs) => {
|
||||
const limit = requireBoundedInteger(kwargs.limit, 8, 1, 20, '--limit');
|
||||
const keyword = encodeURIComponent(requireSearchQuery(kwargs.keyword));
|
||||
const url = `https://duckduckgo.com/ac/?q=${keyword}&type=list`;
|
||||
let resp;
|
||||
try {
|
||||
resp = await fetch(url);
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`DuckDuckGo suggest request failed: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
if (!resp.ok) {
|
||||
throw new CommandExecutionError(`DuckDuckGo suggest returned HTTP ${resp.status}`);
|
||||
}
|
||||
let data;
|
||||
try {
|
||||
data = await resp.json();
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`DuckDuckGo suggest returned malformed JSON: ${err?.message ?? err}`);
|
||||
}
|
||||
const phrases = Array.isArray(data) && data.length > 1 && Array.isArray(data[1]) ? data[1] : [];
|
||||
return phrases
|
||||
.filter((phrase) => typeof phrase === 'string' && phrase.trim())
|
||||
.slice(0, limit)
|
||||
.map(function(p) { return { phrase: p }; });
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { command };
|
||||
@@ -0,0 +1,66 @@
|
||||
import { afterEach, describe, it, expect, vi } from 'vitest';
|
||||
|
||||
const { __test__ } = await import('./suggest.js');
|
||||
const command = __test__.command;
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
describe('duckduckgo suggest', () => {
|
||||
it('should register as a valid command', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('duckduckgo');
|
||||
expect(command.name).toBe('suggest');
|
||||
expect(command.access).toBe('read');
|
||||
expect(command.browser).toBe(false);
|
||||
expect(command.strategy).toBe('public');
|
||||
});
|
||||
|
||||
it('should define keyword positional arg', () => {
|
||||
const kwArg = command.args.find(a => a.name === 'keyword');
|
||||
expect(kwArg).toBeDefined();
|
||||
expect(kwArg.positional).toBe(true);
|
||||
expect(kwArg.required).toBe(true);
|
||||
});
|
||||
|
||||
it('should define limit arg with default 8', () => {
|
||||
const limitArg = command.args.find(a => a.name === 'limit');
|
||||
expect(limitArg).toBeDefined();
|
||||
expect(limitArg.default).toBe(8);
|
||||
});
|
||||
|
||||
it('should define phrase column', () => {
|
||||
expect(command.columns).toEqual(['phrase']);
|
||||
});
|
||||
|
||||
it('rejects empty query and invalid limit before fetch', async () => {
|
||||
const fetchSpy = vi.spyOn(globalThis, 'fetch');
|
||||
await expect(command.func({ keyword: '', limit: 5 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
await expect(command.func({ keyword: 'opencli', limit: 21 })).rejects.toMatchObject({ code: 'ARGUMENT' });
|
||||
expect(fetchSpy).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns filtered suggestion rows from the public API payload', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ['open', ['opencli', '', 'open source']],
|
||||
});
|
||||
|
||||
await expect(command.func({ keyword: 'open', limit: 3 })).resolves.toEqual([
|
||||
{ phrase: 'opencli' },
|
||||
{ phrase: 'open source' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('maps fetch and malformed JSON failures to typed command errors', async () => {
|
||||
vi.spyOn(globalThis, 'fetch').mockRejectedValueOnce(new Error('offline'));
|
||||
await expect(command.func({ keyword: 'open', limit: 3 })).rejects.toMatchObject({ code: 'COMMAND_EXEC' });
|
||||
|
||||
vi.spyOn(globalThis, 'fetch').mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => { throw new Error('bad json'); },
|
||||
});
|
||||
await expect(command.func({ keyword: 'open', limit: 3 })).rejects.toMatchObject({ code: 'COMMAND_EXEC' });
|
||||
});
|
||||
});
|
||||
+299
-54
@@ -1,60 +1,305 @@
|
||||
import { cli } from '@jackwener/opencli/registry';
|
||||
cli({
|
||||
site: 'facebook',
|
||||
name: 'feed',
|
||||
access: 'read',
|
||||
description: 'Get your Facebook news feed',
|
||||
domain: 'www.facebook.com',
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of posts' },
|
||||
],
|
||||
columns: ['index', 'author', 'content', 'likes', 'comments', 'shares'],
|
||||
pipeline: [
|
||||
{ navigate: { url: 'https://www.facebook.com/', settleMs: 4000 } },
|
||||
{ evaluate: `(() => {
|
||||
const limit = \${{ args.limit }};
|
||||
const posts = document.querySelectorAll('[role="article"]');
|
||||
return Array.from(posts)
|
||||
.filter(el => {
|
||||
const text = el.textContent.trim();
|
||||
// Filter out "People you may know" suggestions (both CN and EN)
|
||||
return text.length > 30 &&
|
||||
!text.startsWith('可能认识') &&
|
||||
!text.startsWith('People you may know') &&
|
||||
!text.startsWith('People You May Know');
|
||||
})
|
||||
.slice(0, limit)
|
||||
.map((el, i) => {
|
||||
// Author from header link
|
||||
const headerLink = el.querySelector('h2 a, h3 a, h4 a, strong a');
|
||||
const author = headerLink ? headerLink.textContent.trim() : '';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
|
||||
// Post text: grab visible spans, filter noise
|
||||
const spans = Array.from(el.querySelectorAll('div[dir="auto"]'))
|
||||
.map(s => s.textContent.trim())
|
||||
.filter(t => t.length > 10 && t.length < 500);
|
||||
const content = spans.length > 0 ? spans[0] : '';
|
||||
const FACEBOOK_HOME = 'https://www.facebook.com/';
|
||||
const MAX_LIMIT = 50;
|
||||
|
||||
// Engagement: find like/comment/share counts (CN + EN)
|
||||
const allText = el.textContent;
|
||||
const likesMatch = allText.match(/所有心情:([\\d,.\\s]*[\\d万亿KMk]+)/) ||
|
||||
allText.match(/All:\\s*([\\d,.KMk]+)/) ||
|
||||
allText.match(/([\\d,.KMk]+)\\s*(?:likes?|reactions?)/i);
|
||||
const commentsMatch = allText.match(/([\\d,.]+\\s*[万亿]?)\\s*条评论/) ||
|
||||
allText.match(/([\\d,.KMk]+)\\s*comments?/i);
|
||||
const sharesMatch = allText.match(/([\\d,.]+\\s*[万亿]?)\\s*次分享/) ||
|
||||
allText.match(/([\\d,.KMk]+)\\s*shares?/i);
|
||||
function requireLimit(value) {
|
||||
const n = Number(value);
|
||||
if (!Number.isInteger(n) || n < 1 || n > MAX_LIMIT) {
|
||||
throw new ArgumentError(`facebook feed --limit must be an integer between 1 and ${MAX_LIMIT}`);
|
||||
}
|
||||
return n;
|
||||
}
|
||||
|
||||
function unwrapBrowserResult(value) {
|
||||
if (value && typeof value === 'object' && 'data' in value) {
|
||||
return value.data;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function buildFeedExtractScript(limit) {
|
||||
return `(() => {
|
||||
const limit = ${limit};
|
||||
|
||||
function clean(value) {
|
||||
return String(value || '').replace(/\\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function textOf(el) {
|
||||
return clean(el && el.textContent);
|
||||
}
|
||||
|
||||
function labelOf(el) {
|
||||
return clean(el && el.getAttribute && el.getAttribute('aria-label'));
|
||||
}
|
||||
|
||||
function isAuthPage() {
|
||||
const path = window.location && window.location.pathname ? window.location.pathname : '';
|
||||
const body = textOf(document.body);
|
||||
return /^\\/(login|checkpoint)(\\/|$|\\.php)/.test(path)
|
||||
|| /^(Log in to Facebook|Facebook登录|登录 Facebook)/i.test(body)
|
||||
|| /You must log in to continue/i.test(body);
|
||||
}
|
||||
|
||||
function isExplicitEmptyFeed() {
|
||||
const body = textOf(document.body);
|
||||
return /No posts available|Nothing to show|暂无动态|没有更多动态|还没有帖子/i.test(body);
|
||||
}
|
||||
|
||||
function isSuggestionOrChrome(text) {
|
||||
return /^(People you may know|People You May Know|可能认识的人?|你可能认识的人?)/i.test(text)
|
||||
|| /^(Suggested for you|Suggested Groups|推荐小组|推荐内容)/i.test(text);
|
||||
}
|
||||
|
||||
function isSponsored(text) {
|
||||
return /(^|\\s)(Sponsored|赞助|广告)(\\s|$)/i.test(text);
|
||||
}
|
||||
|
||||
function isActionText(text) {
|
||||
return /^(Like|Comment|Share|Send|Follow|赞|评论|分享|发送|关注)$/i.test(text);
|
||||
}
|
||||
|
||||
function isMetricText(text) {
|
||||
return /^(All:|所有心情:)/i.test(text)
|
||||
|| /\\b(likes?|reactions?|comments?|shares?)\\b/i.test(text)
|
||||
|| /(条评论|次分享)$/.test(text);
|
||||
}
|
||||
|
||||
function isTimestampText(text) {
|
||||
return /^(\\d+\\s*(s|m|h|d|w|mo|yr|min|sec|second|minute|hour|day|week|month|year)s?|Just now|Yesterday|刚刚|昨天|\\d+小时|\\d+天)(\\s*[·•.])?$/i.test(text);
|
||||
}
|
||||
|
||||
function postUrlFrom(root) {
|
||||
const links = Array.from(root.querySelectorAll('a[href]'));
|
||||
for (const link of links) {
|
||||
const href = link.href || link.getAttribute('href') || '';
|
||||
if (/\\/posts\\/|\\/permalink\\.php|\\/story\\.php|\\/photo\\/\\?fbid=|\\/groups\\/[^/]+\\/posts\\//i.test(href)) {
|
||||
return href;
|
||||
}
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
function actionKinds(root) {
|
||||
const kinds = new Set();
|
||||
for (const el of root.querySelectorAll('[aria-label]')) {
|
||||
const label = labelOf(el);
|
||||
if (/^(Like|赞)$/i.test(label)) kinds.add('like');
|
||||
if (/^(Comment|评论)$/i.test(label)) kinds.add('comment');
|
||||
if (/^(Share|分享)$/i.test(label)) kinds.add('share');
|
||||
}
|
||||
return kinds;
|
||||
}
|
||||
|
||||
function visibleBlocks(root) {
|
||||
const seen = new Set();
|
||||
return Array.from(root.querySelectorAll('[dir="auto"]'))
|
||||
.map(textOf)
|
||||
.filter((text) => {
|
||||
if (!text || text.length > 600 || seen.has(text)) return false;
|
||||
seen.add(text);
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
function findAuthor(root) {
|
||||
const links = [
|
||||
root.querySelector('h2 a[href], h3 a[href], h4 a[href], strong a[href]'),
|
||||
...Array.from(root.querySelectorAll('a[role="link"][href]')),
|
||||
].filter(Boolean);
|
||||
for (const link of links) {
|
||||
const text = textOf(link);
|
||||
const href = link.href || link.getAttribute('href') || '';
|
||||
if (text.length > 1 && text.length <= 80
|
||||
&& !isActionText(text)
|
||||
&& !isMetricText(text)
|
||||
&& !isTimestampText(text)
|
||||
&& !/\\/groups\\/|\\/watch\\/|\\/reel\\/|\\/events\\/|\\/friends\\//i.test(href)) {
|
||||
return text;
|
||||
}
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
function contentBlocks(root, author) {
|
||||
return visibleBlocks(root).filter((text) => {
|
||||
if (text === author) return false;
|
||||
if (text.length <= 10) return false;
|
||||
if (isSuggestionOrChrome(text) || isSponsored(text)) return false;
|
||||
if (isActionText(text) || isMetricText(text) || isTimestampText(text)) return false;
|
||||
if (/^(See more|查看更多|更多)$/i.test(text)) return false;
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
function extractPost(root, index) {
|
||||
const fullText = textOf(root);
|
||||
if (!fullText || isSuggestionOrChrome(fullText) || isSponsored(fullText)) return null;
|
||||
|
||||
const author = findAuthor(root);
|
||||
const blocks = contentBlocks(root, author);
|
||||
const content = clean(blocks.join(' '));
|
||||
const postUrl = postUrlFrom(root);
|
||||
const kinds = actionKinds(root);
|
||||
|
||||
if (!author && !content) return null;
|
||||
if (!content && !postUrl && kinds.size < 2) return null;
|
||||
|
||||
const likesMatch = fullText.match(/所有心情:\\s*(\\d[\\d,.\\s万亿KMk]*)/)
|
||||
|| fullText.match(/All:\\s*(\\d[\\d,.KMk]*)/)
|
||||
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*(?:likes?|reactions?)/i);
|
||||
const commentsMatch = fullText.match(/([\\d,.]+\\s*[万亿]?)\\s*条评论/)
|
||||
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*comments?/i);
|
||||
const sharesMatch = fullText.match(/([\\d,.]+\\s*[万亿]?)\\s*次分享/)
|
||||
|| fullText.match(/(\\d[\\d,.KMk]*)\\s*shares?/i);
|
||||
|
||||
return {
|
||||
index: i + 1,
|
||||
index,
|
||||
author: author.substring(0, 50),
|
||||
content: content.replace(/\\n/g, ' ').substring(0, 120),
|
||||
likes: likesMatch ? likesMatch[1] : '-',
|
||||
comments: commentsMatch ? commentsMatch[1] : '-',
|
||||
shares: sharesMatch ? sharesMatch[1] : '-',
|
||||
content: content.substring(0, 120),
|
||||
likes: likesMatch ? clean(likesMatch[1]) : '-',
|
||||
comments: commentsMatch ? clean(commentsMatch[1]) : '-',
|
||||
shares: sharesMatch ? clean(sharesMatch[1]) : '-',
|
||||
};
|
||||
});
|
||||
})()
|
||||
` },
|
||||
],
|
||||
});
|
||||
}
|
||||
|
||||
function primaryContainers() {
|
||||
return Array.from(document.querySelectorAll('[role="article"]'))
|
||||
.filter((el) => textOf(el).length > 30);
|
||||
}
|
||||
|
||||
function fallbackContainers() {
|
||||
const main = document.querySelector('[role="main"]');
|
||||
if (!main) return [];
|
||||
const buttons = Array.from(main.querySelectorAll('[aria-label="Like"], [aria-label="赞"], [aria-label="Comment"], [aria-label="评论"], [aria-label="Share"], [aria-label="分享"]'));
|
||||
const seen = new WeakSet();
|
||||
const containers = [];
|
||||
for (const button of buttons) {
|
||||
let node = button.parentElement;
|
||||
for (let depth = 0; depth < 16 && node && node !== main && node !== document.body; depth += 1, node = node.parentElement) {
|
||||
const text = textOf(node);
|
||||
const kinds = actionKinds(node);
|
||||
const blocks = visibleBlocks(node);
|
||||
const hasPostEvidence = Boolean(postUrlFrom(node)) || blocks.some((block) => block.length > 20 && !isActionText(block) && !isMetricText(block));
|
||||
if (text.length >= 80 && kinds.has('like') && (kinds.has('comment') || kinds.has('share')) && hasPostEvidence) {
|
||||
if (!seen.has(node)) {
|
||||
seen.add(node);
|
||||
containers.push(node);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
return containers;
|
||||
}
|
||||
|
||||
function dedupe(containers) {
|
||||
const seen = new Set();
|
||||
const result = [];
|
||||
for (const node of containers) {
|
||||
const key = postUrlFrom(node) || contentBlocks(node, findAuthor(node)).join('|').substring(0, 200);
|
||||
if (!key || seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
result.push(node);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
if (isAuthPage()) return { status: 'auth', rows: [], diagnostics: {} };
|
||||
|
||||
const primary = primaryContainers();
|
||||
const combined = dedupe([...primary, ...fallbackContainers()]);
|
||||
const rows = [];
|
||||
for (const container of combined) {
|
||||
const row = extractPost(container, rows.length + 1);
|
||||
if (row) rows.push(row);
|
||||
if (rows.length >= limit) break;
|
||||
}
|
||||
|
||||
return {
|
||||
status: rows.length ? 'ok' : (isExplicitEmptyFeed() ? 'empty' : 'no_rows'),
|
||||
rows,
|
||||
diagnostics: {
|
||||
articleCount: document.querySelectorAll('[role="article"]').length,
|
||||
primaryCount: primary.length,
|
||||
fallbackActionCount: document.querySelectorAll('[role="main"] [aria-label="Like"], [role="main"] [aria-label="赞"], [role="main"] [aria-label="Comment"], [role="main"] [aria-label="评论"]').length,
|
||||
mainTextLength: textOf(document.querySelector('[role="main"]')).length,
|
||||
},
|
||||
};
|
||||
})()`;
|
||||
}
|
||||
|
||||
async function getFacebookFeed(page, kwargs) {
|
||||
const limit = requireLimit(kwargs.limit ?? 10);
|
||||
try {
|
||||
await page.goto(FACEBOOK_HOME, { settleMs: 4000 });
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(
|
||||
`Failed to navigate to facebook feed: ${err instanceof Error ? err.message : err}`,
|
||||
'Check that facebook.com is reachable and the browser extension is connected.',
|
||||
);
|
||||
}
|
||||
|
||||
let payload;
|
||||
try {
|
||||
payload = unwrapBrowserResult(await page.evaluate(buildFeedExtractScript(limit)));
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(
|
||||
`Failed to read facebook feed: ${err instanceof Error ? err.message : err}`,
|
||||
'Facebook may not have rendered or the feed markup may have changed.',
|
||||
);
|
||||
}
|
||||
|
||||
if (!payload || typeof payload !== 'object' || !Array.isArray(payload.rows)) {
|
||||
throw new CommandExecutionError('facebook feed returned malformed extraction payload');
|
||||
}
|
||||
|
||||
if (payload.status === 'auth') {
|
||||
throw new AuthRequiredError('www.facebook.com', 'Open Chrome and log in to Facebook before retrying.');
|
||||
}
|
||||
|
||||
if (payload.rows.length > 0) {
|
||||
return payload.rows;
|
||||
}
|
||||
|
||||
if (payload.status === 'empty') {
|
||||
throw new EmptyResultError('facebook feed', 'Facebook did not show any feed posts for this account.');
|
||||
}
|
||||
|
||||
const diagnostics = payload.diagnostics || {};
|
||||
if (diagnostics.articleCount || diagnostics.fallbackActionCount || diagnostics.mainTextLength > 200) {
|
||||
throw new CommandExecutionError(
|
||||
'facebook feed page rendered but no feed rows could be extracted',
|
||||
`Diagnostics: articles=${diagnostics.articleCount || 0}, actions=${diagnostics.fallbackActionCount || 0}, mainTextLength=${diagnostics.mainTextLength || 0}.`,
|
||||
);
|
||||
}
|
||||
|
||||
throw new EmptyResultError('facebook feed', 'No Facebook feed content was visible in the current browser session.');
|
||||
}
|
||||
|
||||
const command = {
|
||||
site: 'facebook',
|
||||
name: 'feed',
|
||||
access: 'read',
|
||||
description: 'Get your Facebook news feed',
|
||||
domain: 'www.facebook.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
navigateBefore: false,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of posts' },
|
||||
],
|
||||
columns: ['index', 'author', 'content', 'likes', 'comments', 'shares'],
|
||||
func: getFacebookFeed,
|
||||
};
|
||||
|
||||
cli(command);
|
||||
|
||||
export const __test__ = {
|
||||
buildFeedExtractScript,
|
||||
command,
|
||||
getFacebookFeed,
|
||||
requireLimit,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { JSDOM } from 'jsdom';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { __test__ } from './feed.js';
|
||||
|
||||
function runExtract(html, limit = 10, url = 'https://www.facebook.com/') {
|
||||
const dom = new JSDOM(html, { url });
|
||||
return Function('window', 'document', `return ${__test__.buildFeedExtractScript(limit)};`)(dom.window, dom.window.document);
|
||||
}
|
||||
|
||||
function createPage(payload) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(payload),
|
||||
};
|
||||
}
|
||||
|
||||
describe('facebook feed', () => {
|
||||
it('registers the feed command with the existing row contract', () => {
|
||||
const cmd = getRegistry().get('facebook/feed');
|
||||
expect(cmd).toBeDefined();
|
||||
expect(cmd.columns).toEqual(['index', 'author', 'content', 'likes', 'comments', 'shares']);
|
||||
});
|
||||
|
||||
it('extracts existing role=article feed rows', () => {
|
||||
const payload = runExtract(`
|
||||
<main role="main">
|
||||
<div role="article">
|
||||
<h2><a href="https://www.facebook.com/alice">Alice Example</a></h2>
|
||||
<div dir="auto">This is a normal Facebook feed post with enough text to extract.</div>
|
||||
<span>All: 12</span>
|
||||
<span>3 comments</span>
|
||||
<span>2 shares</span>
|
||||
<div aria-label="Like"></div><div aria-label="Comment"></div>
|
||||
</div>
|
||||
</main>
|
||||
`);
|
||||
|
||||
expect(payload.status).toBe('ok');
|
||||
expect(payload.rows).toEqual([{
|
||||
index: 1,
|
||||
author: 'Alice Example',
|
||||
content: 'This is a normal Facebook feed post with enough text to extract.',
|
||||
likes: '12',
|
||||
comments: '3',
|
||||
shares: '2',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('falls back from empty article nodes to action-bounded feed containers', () => {
|
||||
const payload = runExtract(`
|
||||
<main role="main">
|
||||
<div role="article"></div>
|
||||
<section>
|
||||
<div>
|
||||
<h2><a href="https://www.facebook.com/bob/posts/123">Bob Builder</a></h2>
|
||||
<div dir="auto">Fallback post body from a Facebook feed card with empty article text.</div>
|
||||
<a href="https://www.facebook.com/bob/posts/123">Permalink</a>
|
||||
<span>All: 1.2K</span>
|
||||
<span>4 comments</span>
|
||||
<span>1 shares</span>
|
||||
<div><button aria-label="Like">Like</button><button aria-label="Comment">Comment</button></div>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
`);
|
||||
|
||||
expect(payload.status).toBe('ok');
|
||||
expect(payload.rows).toEqual([{
|
||||
index: 1,
|
||||
author: 'Bob Builder',
|
||||
content: 'Fallback post body from a Facebook feed card with empty article text.',
|
||||
likes: '1.2K',
|
||||
comments: '4',
|
||||
shares: '1',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('does not turn suggestions or side chrome action buttons into feed rows', () => {
|
||||
const payload = runExtract(`
|
||||
<main role="main">
|
||||
<aside>
|
||||
<h2>People you may know</h2>
|
||||
<div dir="auto">Charlie Suggested</div>
|
||||
<div dir="auto">Add friend from suggested people card with plenty of text.</div>
|
||||
<button aria-label="Like">Like</button>
|
||||
<button aria-label="Comment">Comment</button>
|
||||
</aside>
|
||||
<nav>
|
||||
<div dir="auto">Navigation item with a Like button but not a feed post.</div>
|
||||
<button aria-label="Like">Like</button>
|
||||
<button aria-label="Comment">Comment</button>
|
||||
</nav>
|
||||
</main>
|
||||
`);
|
||||
|
||||
expect(payload.status).toBe('no_rows');
|
||||
expect(payload.rows).toEqual([]);
|
||||
});
|
||||
|
||||
it('still considers bounded fallback rows when article nodes are suggestion chrome', () => {
|
||||
const payload = runExtract(`
|
||||
<main role="main">
|
||||
<div role="article">
|
||||
<h2>People you may know</h2>
|
||||
<div dir="auto">Suggested profile card with enough text to look article-like.</div>
|
||||
<button aria-label="Like">Like</button>
|
||||
<button aria-label="Comment">Comment</button>
|
||||
</div>
|
||||
<section>
|
||||
<div>
|
||||
<h2><a href="https://www.facebook.com/dana/posts/456">Dana Poster</a></h2>
|
||||
<div dir="auto">Fallback feed post should still be extracted after suggestion articles are filtered.</div>
|
||||
<a href="https://www.facebook.com/dana/posts/456">Permalink</a>
|
||||
<button aria-label="Like">Like</button>
|
||||
<button aria-label="Comment">Comment</button>
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
`, 1);
|
||||
|
||||
expect(payload.status).toBe('ok');
|
||||
expect(payload.rows).toEqual([{
|
||||
index: 1,
|
||||
author: 'Dana Poster',
|
||||
content: 'Fallback feed post should still be extracted after suggestion articles are filtered.',
|
||||
likes: '-',
|
||||
comments: '-',
|
||||
shares: '-',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('reports auth pages from the browser extractor', () => {
|
||||
const payload = runExtract('<main role="main">Log in to Facebook</main>', 10, 'https://www.facebook.com/login/');
|
||||
expect(payload.status).toBe('auth');
|
||||
expect(payload.rows).toEqual([]);
|
||||
});
|
||||
|
||||
it('validates limit before browser navigation', async () => {
|
||||
const page = createPage({ status: 'ok', rows: [] });
|
||||
await expect(__test__.command.func(page, { limit: 0 })).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('maps browser envelopes and returns extracted rows', async () => {
|
||||
const page = createPage({ session: 'site:facebook', data: { status: 'ok', rows: [{ index: 1, author: 'A', content: 'Body', likes: '-', comments: '-', shares: '-' }] } });
|
||||
|
||||
await expect(__test__.command.func(page, { limit: 1 })).resolves.toEqual([{
|
||||
index: 1,
|
||||
author: 'A',
|
||||
content: 'Body',
|
||||
likes: '-',
|
||||
comments: '-',
|
||||
shares: '-',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('maps auth, real empty, parser drift, and malformed payloads to typed errors', async () => {
|
||||
await expect(__test__.command.func(createPage({ status: 'auth', rows: [] }), { limit: 1 }))
|
||||
.rejects.toBeInstanceOf(AuthRequiredError);
|
||||
await expect(__test__.command.func(createPage({ status: 'empty', rows: [] }), { limit: 1 }))
|
||||
.rejects.toBeInstanceOf(EmptyResultError);
|
||||
await expect(__test__.command.func(createPage({ status: 'no_rows', rows: [], diagnostics: { articleCount: 1, fallbackActionCount: 2, mainTextLength: 500 } }), { limit: 1 }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(__test__.command.func(createPage({ rows: null }), { limit: 1 }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,228 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import {
|
||||
ArgumentError,
|
||||
AuthRequiredError,
|
||||
CommandExecutionError,
|
||||
EmptyResultError,
|
||||
} from '@jackwener/opencli/errors';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
const FLOMO_APP_DOMAIN = 'v.flomoapp.com';
|
||||
const FLOMO_API_DOMAIN = 'flomoapp.com';
|
||||
const MAX_LIMIT = 200;
|
||||
|
||||
function unwrapBrowserResult(value) {
|
||||
if (value && typeof value === 'object' && 'session' in value && 'data' in value) {
|
||||
return value.data;
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function parsePositiveIntArg(value, name, fallback, max) {
|
||||
if (value === undefined || value === null || value === '') {
|
||||
return fallback;
|
||||
}
|
||||
const text = String(value).trim();
|
||||
if (!/^\d+$/.test(text)) {
|
||||
throw new ArgumentError(`flomo memos --${name} must be a positive integer`);
|
||||
}
|
||||
const parsed = Number(text);
|
||||
if (!Number.isSafeInteger(parsed) || parsed < 1 || parsed > max) {
|
||||
throw new ArgumentError(`flomo memos --${name} must be between 1 and ${max}`);
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseSinceArg(value) {
|
||||
if (value === undefined || value === null || value === '') {
|
||||
return 0;
|
||||
}
|
||||
const text = String(value).trim();
|
||||
if (!/^\d+$/.test(text)) {
|
||||
throw new ArgumentError('flomo memos --since must be a non-negative Unix timestamp in seconds');
|
||||
}
|
||||
const parsed = Number(text);
|
||||
if (!Number.isSafeInteger(parsed)) {
|
||||
throw new ArgumentError('flomo memos --since must be a safe integer Unix timestamp in seconds');
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
function parseSlugArg(value) {
|
||||
if (value === undefined || value === null || value === '') {
|
||||
return '';
|
||||
}
|
||||
const slug = String(value).trim();
|
||||
if (!/^[A-Za-z0-9_-]{1,256}$/.test(slug)) {
|
||||
throw new ArgumentError('flomo memos --slug must be an opaque memo cursor containing only letters, numbers, _ or -');
|
||||
}
|
||||
return slug;
|
||||
}
|
||||
|
||||
function buildSignedUrl(limit, since, slug) {
|
||||
const params = {
|
||||
limit: String(limit),
|
||||
latest_updated_at: String(since),
|
||||
tz: '8:0',
|
||||
timestamp: String(Math.floor(Date.now() / 1000)),
|
||||
api_key: 'flomo_web',
|
||||
app_version: '4.0',
|
||||
platform: 'web',
|
||||
webp: '1',
|
||||
};
|
||||
if (slug) params.latest_slug = slug;
|
||||
const keys = Object.keys(params).sort();
|
||||
const signBase = keys.map((key) => `${key}=${params[key]}`).join('&');
|
||||
params.sign = createHash('md5').update(signBase + 'dbbc3dd73364b4084c3a69346e0ce2b2').digest('hex');
|
||||
return 'https://flomoapp.com/api/v1/memo/updated/?' + new URLSearchParams(params).toString();
|
||||
}
|
||||
|
||||
function buildGetTokenJs() {
|
||||
return `
|
||||
(() => {
|
||||
try {
|
||||
const raw = localStorage.getItem('me');
|
||||
if (!raw) return null;
|
||||
const me = JSON.parse(raw);
|
||||
const token = me?.access_token || me?.data?.access_token || '';
|
||||
return typeof token === 'string' && token.trim() ? token.trim() : null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
})()
|
||||
`;
|
||||
}
|
||||
|
||||
function isAuthFailureMessage(message) {
|
||||
return /auth|unauth|login|token|permission|forbidden|unauthorized|登录|登陆|鉴权|权限/i.test(String(message || ''));
|
||||
}
|
||||
|
||||
function normalizeTags(tags) {
|
||||
if (!Array.isArray(tags)) return '';
|
||||
return tags
|
||||
.map((tag) => {
|
||||
if (typeof tag === 'string') return tag;
|
||||
return tag?.name || tag?.tag || tag?.content || '';
|
||||
})
|
||||
.map((tag) => String(tag).trim())
|
||||
.filter(Boolean)
|
||||
.join(', ');
|
||||
}
|
||||
|
||||
function normalizeImages(files) {
|
||||
if (!Array.isArray(files)) return '';
|
||||
return files
|
||||
.map((file) => file?.thumbnail_url || file?.url || '')
|
||||
.map((url) => String(url).trim())
|
||||
.filter(Boolean)
|
||||
.join(' | ');
|
||||
}
|
||||
|
||||
function memoUrl(slug) {
|
||||
return slug ? `https://${FLOMO_APP_DOMAIN}/mine/?memo_id=${encodeURIComponent(slug)}` : '';
|
||||
}
|
||||
|
||||
function normalizeMemo(memo) {
|
||||
if (!memo || typeof memo !== 'object' || Array.isArray(memo)) {
|
||||
throw new CommandExecutionError('Flomo API returned a malformed memo entry');
|
||||
}
|
||||
const slug = String(memo.slug || memo.id || '').trim();
|
||||
if (!slug) {
|
||||
throw new CommandExecutionError('Flomo API returned a memo without slug/id');
|
||||
}
|
||||
return {
|
||||
id: slug,
|
||||
url: memoUrl(slug),
|
||||
content: String(memo.content || '').trim(),
|
||||
slug,
|
||||
tags: normalizeTags(memo.tags),
|
||||
images: normalizeImages(memo.files),
|
||||
created_at: String(memo.created_at || ''),
|
||||
updated_at: String(memo.updated_at || ''),
|
||||
};
|
||||
}
|
||||
|
||||
async function fetchFlomoJson(url, token) {
|
||||
let resp;
|
||||
try {
|
||||
resp = await fetch(url, {
|
||||
headers: {
|
||||
Authorization: 'Bearer ' + token,
|
||||
'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
|
||||
Accept: 'application/json',
|
||||
},
|
||||
});
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`Failed to fetch Flomo memos: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
if (resp.status === 401 || resp.status === 403) {
|
||||
throw new AuthRequiredError(FLOMO_API_DOMAIN, `Flomo API returned HTTP ${resp.status}; please refresh your Flomo login session`);
|
||||
}
|
||||
if (!resp.ok) {
|
||||
throw new CommandExecutionError(`Flomo API returned HTTP ${resp.status}`);
|
||||
}
|
||||
try {
|
||||
return await resp.json();
|
||||
} catch (err) {
|
||||
throw new CommandExecutionError(`Flomo API returned malformed JSON: ${err instanceof Error ? err.message : String(err)}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function readAccessToken(page) {
|
||||
const token = unwrapBrowserResult(await page.evaluate(buildGetTokenJs()));
|
||||
if (typeof token !== 'string' || !token.trim()) {
|
||||
throw new AuthRequiredError(FLOMO_API_DOMAIN, 'Flomo memos requires an active signed-in Flomo browser session');
|
||||
}
|
||||
return token.trim();
|
||||
}
|
||||
|
||||
const command = cli({
|
||||
site: 'flomo',
|
||||
name: 'memos',
|
||||
access: 'read',
|
||||
description: 'List your Flomo memos',
|
||||
domain: FLOMO_API_DOMAIN,
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
navigateBefore: `https://${FLOMO_APP_DOMAIN}/`,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of memos to fetch (1-200)' },
|
||||
{ name: 'since', type: 'int', help: 'Only memos updated after this Unix timestamp in seconds' },
|
||||
{ name: 'slug', help: 'Pagination cursor from a previous memo page' },
|
||||
],
|
||||
columns: ['id', 'url', 'content', 'slug', 'tags', 'images', 'created_at', 'updated_at'],
|
||||
func: async (page, kwargs) => {
|
||||
const limit = parsePositiveIntArg(kwargs.limit, 'limit', 20, MAX_LIMIT);
|
||||
const since = parseSinceArg(kwargs.since);
|
||||
const slug = parseSlugArg(kwargs.slug);
|
||||
await page.wait(3).catch(() => {});
|
||||
const token = await readAccessToken(page);
|
||||
const body = await fetchFlomoJson(buildSignedUrl(limit, since, slug), token);
|
||||
if (!body || typeof body !== 'object' || Array.isArray(body)) {
|
||||
throw new CommandExecutionError('Flomo API returned a malformed response');
|
||||
}
|
||||
if (body.code !== 0) {
|
||||
const message = body.message || `Flomo API error code ${body.code}`;
|
||||
if (isAuthFailureMessage(message)) {
|
||||
throw new AuthRequiredError(FLOMO_API_DOMAIN, message);
|
||||
}
|
||||
throw new CommandExecutionError(message);
|
||||
}
|
||||
if (!Array.isArray(body.data)) {
|
||||
throw new CommandExecutionError('Flomo API returned malformed memo data');
|
||||
}
|
||||
if (body.data.length === 0) {
|
||||
throw new EmptyResultError('flomo memos', 'No Flomo memos matched the requested filters.');
|
||||
}
|
||||
return body.data.map(normalizeMemo);
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
buildSignedUrl,
|
||||
command,
|
||||
normalizeMemo,
|
||||
parsePositiveIntArg,
|
||||
parseSinceArg,
|
||||
parseSlugArg,
|
||||
};
|
||||
@@ -0,0 +1,144 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import {
|
||||
ArgumentError,
|
||||
AuthRequiredError,
|
||||
CommandExecutionError,
|
||||
EmptyResultError,
|
||||
} from '@jackwener/opencli/errors';
|
||||
|
||||
const { __test__ } = await import('./memos.js');
|
||||
const { command, normalizeMemo, parsePositiveIntArg, parseSinceArg, parseSlugArg } = __test__;
|
||||
|
||||
function createPage(token = 'token-123') {
|
||||
return {
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue({ session: 'browser:default', data: token }),
|
||||
};
|
||||
}
|
||||
|
||||
function mockFetchJson(body, status = 200) {
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({
|
||||
ok: status >= 200 && status < 300,
|
||||
status,
|
||||
json: vi.fn().mockResolvedValue(body),
|
||||
}));
|
||||
}
|
||||
|
||||
describe('flomo memos registration', () => {
|
||||
it('registers as a browser cookie read command with stable columns', () => {
|
||||
expect(command.site).toBe('flomo');
|
||||
expect(command.name).toBe('memos');
|
||||
expect(command.access).toBe('read');
|
||||
expect(command.browser).toBe(true);
|
||||
expect(command.strategy).toBe('cookie');
|
||||
expect(command.columns).toEqual(['id', 'url', 'content', 'slug', 'tags', 'images', 'created_at', 'updated_at']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('flomo memos argument validation', () => {
|
||||
it('rejects invalid limits instead of silently clamping', () => {
|
||||
expect(() => parsePositiveIntArg('0', 'limit', 20, 200)).toThrow(ArgumentError);
|
||||
expect(() => parsePositiveIntArg('201', 'limit', 20, 200)).toThrow(ArgumentError);
|
||||
expect(() => parsePositiveIntArg('10.5', 'limit', 20, 200)).toThrow(ArgumentError);
|
||||
expect(() => parsePositiveIntArg('abc', 'limit', 20, 200)).toThrow(ArgumentError);
|
||||
expect(parsePositiveIntArg(undefined, 'limit', 20, 200)).toBe(20);
|
||||
expect(parsePositiveIntArg('200', 'limit', 20, 200)).toBe(200);
|
||||
});
|
||||
|
||||
it('rejects invalid since and slug arguments', () => {
|
||||
expect(parseSinceArg(undefined)).toBe(0);
|
||||
expect(parseSinceArg('1735689600')).toBe(1735689600);
|
||||
expect(() => parseSinceArg('-1')).toThrow(ArgumentError);
|
||||
expect(() => parseSinceArg('1.5')).toThrow(ArgumentError);
|
||||
expect(parseSlugArg(undefined)).toBe('');
|
||||
expect(parseSlugArg('abc_DEF-123')).toBe('abc_DEF-123');
|
||||
expect(() => parseSlugArg('bad/slash')).toThrow(ArgumentError);
|
||||
expect(() => parseSlugArg('bad space')).toThrow(ArgumentError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('flomo memo normalization', () => {
|
||||
it('emits string-safe id/url fields and normalizes tags/images', () => {
|
||||
expect(normalizeMemo({
|
||||
slug: 'memo_12345678901234567890',
|
||||
content: ' <p>Hello</p> ',
|
||||
tags: [{ name: 'work' }, 'idea'],
|
||||
files: [{ thumbnail_url: 'https://img/thumb.jpg' }, { url: 'https://img/full.jpg' }],
|
||||
created_at: '2026-01-01T00:00:00+08:00',
|
||||
updated_at: '2026-01-02T00:00:00+08:00',
|
||||
})).toEqual({
|
||||
id: 'memo_12345678901234567890',
|
||||
url: 'https://v.flomoapp.com/mine/?memo_id=memo_12345678901234567890',
|
||||
content: '<p>Hello</p>',
|
||||
slug: 'memo_12345678901234567890',
|
||||
tags: 'work, idea',
|
||||
images: 'https://img/thumb.jpg | https://img/full.jpg',
|
||||
created_at: '2026-01-01T00:00:00+08:00',
|
||||
updated_at: '2026-01-02T00:00:00+08:00',
|
||||
});
|
||||
});
|
||||
|
||||
it('fails typed on malformed memo entries', () => {
|
||||
expect(() => normalizeMemo(null)).toThrow(CommandExecutionError);
|
||||
expect(() => normalizeMemo({ content: 'missing slug' })).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
describe('flomo memos command', () => {
|
||||
beforeEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('reads token from Browser Bridge envelope and returns memo rows', async () => {
|
||||
mockFetchJson({
|
||||
code: 0,
|
||||
data: [{
|
||||
slug: 'memo_1',
|
||||
content: 'hello',
|
||||
tags: ['tag'],
|
||||
files: [],
|
||||
created_at: '2026-01-01',
|
||||
updated_at: '2026-01-02',
|
||||
}],
|
||||
});
|
||||
|
||||
const rows = await command.func(createPage(), { limit: '1' });
|
||||
|
||||
expect(globalThis.fetch).toHaveBeenCalledWith(expect.stringContaining('limit=1'), expect.objectContaining({
|
||||
headers: expect.objectContaining({ Authorization: 'Bearer token-123' }),
|
||||
}));
|
||||
expect(rows).toEqual([{
|
||||
id: 'memo_1',
|
||||
url: 'https://v.flomoapp.com/mine/?memo_id=memo_1',
|
||||
content: 'hello',
|
||||
slug: 'memo_1',
|
||||
tags: 'tag',
|
||||
images: '',
|
||||
created_at: '2026-01-01',
|
||||
updated_at: '2026-01-02',
|
||||
}]);
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when the browser session has no token', async () => {
|
||||
await expect(command.func(createPage(null), {})).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('maps Flomo auth failures to AuthRequiredError', async () => {
|
||||
mockFetchJson({ code: 401, message: 'unauthorized' });
|
||||
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('maps HTTP, malformed JSON, malformed data, and empty results to typed errors', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: false, status: 500, json: vi.fn() }));
|
||||
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue({ ok: true, status: 200, json: vi.fn().mockRejectedValue(new Error('bad json')) }));
|
||||
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
|
||||
mockFetchJson({ code: 0, data: {} });
|
||||
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
|
||||
mockFetchJson({ code: 0, data: [] });
|
||||
await expect(command.func(createPage(), {})).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
@@ -123,8 +123,8 @@ cli({
|
||||
rows.push({
|
||||
rank: rows.length + 1,
|
||||
name,
|
||||
language: normalizeText(getFirstText(fields.langs)) || '-',
|
||||
description: normalizeText(getFirstText(fields.description)) || '-',
|
||||
language: normalizeText(getFirstText(fields.langs)) || '',
|
||||
description: normalizeText(getFirstText(fields.description)) || '',
|
||||
stars: normalizeStars(fields['count.star']),
|
||||
url: repoUrl,
|
||||
});
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './search.js';
|
||||
|
||||
function mockGiteeResponse(hits) {
|
||||
return {
|
||||
ok: true,
|
||||
json: () => Promise.resolve({ hits: { hits } }),
|
||||
};
|
||||
}
|
||||
|
||||
function makePage() {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
};
|
||||
}
|
||||
|
||||
describe('gitee search', () => {
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
it('emits empty-string for missing language / description instead of a sentinel', async () => {
|
||||
const cmd = getRegistry().get('gitee/search');
|
||||
expect(cmd?.func).toBeTypeOf('function');
|
||||
const fetchMock = vi.fn().mockResolvedValue(mockGiteeResponse([
|
||||
{
|
||||
fields: {
|
||||
title: 'someuser/no-meta-repo',
|
||||
url: 'https://gitee.com/someuser/no-meta-repo',
|
||||
},
|
||||
},
|
||||
]));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
const rows = await cmd.func(makePage(), { keyword: 'test', limit: 10 });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0].name).toBe('someuser/no-meta-repo');
|
||||
expect(rows[0].language).toBe('');
|
||||
expect(rows[0].description).toBe('');
|
||||
});
|
||||
|
||||
it('passes through populated language / description verbatim', async () => {
|
||||
const cmd = getRegistry().get('gitee/search');
|
||||
const fetchMock = vi.fn().mockResolvedValue(mockGiteeResponse([
|
||||
{
|
||||
fields: {
|
||||
title: 'org/repo-a',
|
||||
url: 'https://gitee.com/org/repo-a',
|
||||
langs: 'TypeScript',
|
||||
description: 'A test repo',
|
||||
'count.star': '42',
|
||||
},
|
||||
},
|
||||
]));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
const rows = await cmd.func(makePage(), { keyword: 'test', limit: 10 });
|
||||
expect(rows[0].language).toBe('TypeScript');
|
||||
expect(rows[0].description).toBe('A test repo');
|
||||
expect(rows[0].stars).toBe('42');
|
||||
});
|
||||
});
|
||||
@@ -1,4 +1,5 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { clampInt, requireNonEmptyQuery } from '../_shared/common.js';
|
||||
|
||||
cli({
|
||||
@@ -18,12 +19,17 @@ cli({
|
||||
const limit = clampInt(kwargs.limit, 10, 1, 20);
|
||||
const query = requireNonEmptyQuery(kwargs.query);
|
||||
await page.goto(`https://scholar.google.com/scholar?q=${encodeURIComponent(query)}&hl=zh-CN`);
|
||||
await page.wait(3);
|
||||
const data = await page.evaluate(`
|
||||
try {
|
||||
await page.wait({ selector: '.gs_r.gs_or.gs_scl', timeout: 5 });
|
||||
} catch {
|
||||
await page.wait(3);
|
||||
}
|
||||
const wrapper = await page.evaluate(`
|
||||
(() => {
|
||||
const normalize = v => (v || '').replace(/\\s+/g, ' ').trim();
|
||||
const results = [];
|
||||
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl')) {
|
||||
const resultCards = Array.from(document.querySelectorAll('.gs_r.gs_or.gs_scl'));
|
||||
for (const el of resultCards) {
|
||||
const container = el.querySelector('.gs_ri') || el;
|
||||
const titleEl = container.querySelector('.gs_rt a, h3 a');
|
||||
const title = normalize(titleEl?.textContent);
|
||||
@@ -50,9 +56,18 @@ cli({
|
||||
});
|
||||
if (results.length >= ${limit}) break;
|
||||
}
|
||||
return results;
|
||||
return { items: results, resultCount: resultCards.length };
|
||||
})()
|
||||
`);
|
||||
return Array.isArray(data) ? data : [];
|
||||
if (!wrapper || typeof wrapper !== 'object' || !Array.isArray(wrapper.items)) {
|
||||
throw new CommandExecutionError('Google Scholar search returned an unexpected payload shape');
|
||||
}
|
||||
if (wrapper.items.length === 0) {
|
||||
if (Number(wrapper.resultCount) > 0) {
|
||||
throw new CommandExecutionError('Google Scholar result cards were present but no rows could be extracted');
|
||||
}
|
||||
throw new EmptyResultError('google-scholar/search', 'Try a different query or check whether Google Scholar returned a CAPTCHA.');
|
||||
}
|
||||
return wrapper.items;
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './search.js';
|
||||
|
||||
@@ -25,14 +26,46 @@ describe('google-scholar search command', () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue([]),
|
||||
evaluate: vi.fn().mockResolvedValue({ items: [{ rank: 1, title: 'Paper' }], resultCount: 1 }),
|
||||
};
|
||||
|
||||
await command.func(page, { query: 'transformer' });
|
||||
const rows = await command.func(page, { query: 'transformer' });
|
||||
|
||||
const script = page.evaluate.mock.calls[0][0];
|
||||
expect(script).toContain("document.querySelectorAll('.gs_r.gs_or.gs_scl')");
|
||||
expect(script).not.toContain(".gs_r.gs_or.gs_scl, .gs_ri");
|
||||
expect(script).toContain("const container = el.querySelector('.gs_ri') || el");
|
||||
expect(script).toContain('return { items: results, resultCount: resultCards.length }');
|
||||
expect(rows).toEqual([{ rank: 1, title: 'Paper' }]);
|
||||
});
|
||||
|
||||
it('throws typed empty when Scholar returns no result cards', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue({ items: [], resultCount: 0 }),
|
||||
};
|
||||
|
||||
await expect(command.func(page, { query: 'no results expected' })).rejects.toThrow(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws command execution when result cards exist but parser extracts no rows', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue({ items: [], resultCount: 2 }),
|
||||
};
|
||||
|
||||
await expect(command.func(page, { query: 'parser drift' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws command execution for malformed evaluate payloads instead of treating them as empty', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue({ items: { rank: 1 }, resultCount: 1 }),
|
||||
};
|
||||
|
||||
await expect(command.func(page, { query: 'bad payload' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -39,12 +39,12 @@ cli({
|
||||
catch {
|
||||
await page.wait(2);
|
||||
}
|
||||
const results = await page.evaluate(`
|
||||
const wrapper = await page.evaluate(`
|
||||
(function() {
|
||||
var results = [];
|
||||
var seenUrls = {};
|
||||
var rso = document.querySelector('#rso');
|
||||
if (!rso) return results;
|
||||
if (!rso) return {items: results};
|
||||
|
||||
// -- Featured snippet (scoped to #rso to avoid matching unrelated elements) --
|
||||
var featuredEl = rso.querySelector('.xpdopen .hgKElc')
|
||||
@@ -126,10 +126,11 @@ cli({
|
||||
}
|
||||
}
|
||||
|
||||
return results;
|
||||
return {items: results};
|
||||
})()
|
||||
`);
|
||||
if (!Array.isArray(results) || results.length === 0) {
|
||||
const results = (wrapper && wrapper.items) || [];
|
||||
if (results.length === 0) {
|
||||
throw new CliError('NOT_FOUND', 'No search results found', 'Try a different keyword or check for CAPTCHA');
|
||||
}
|
||||
return results;
|
||||
|
||||
+27
-17
@@ -1,4 +1,5 @@
|
||||
import { cli } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
cli({
|
||||
site: 'jike',
|
||||
name: 'post',
|
||||
@@ -16,16 +17,16 @@ cli({
|
||||
},
|
||||
],
|
||||
columns: ['type', 'author', 'content', 'likes', 'time'],
|
||||
pipeline: [
|
||||
{ navigate: 'https://m.okjike.com/originalPosts/${{ args.id }}' },
|
||||
{ evaluate: `(() => {
|
||||
func: async (page, args) => {
|
||||
await page.goto(`https://m.okjike.com/originalPosts/${args.id}`);
|
||||
const data = await page.evaluate(`(() => {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return { ok: false, reason: 'missing-data-script' };
|
||||
try {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return [];
|
||||
const data = JSON.parse(el.textContent);
|
||||
const data = JSON.parse(el.textContent || '{}');
|
||||
const pageProps = data?.props?.pageProps || {};
|
||||
const post = pageProps.post || {};
|
||||
const comments = pageProps.comments || [];
|
||||
const comments = Array.isArray(pageProps.comments) ? pageProps.comments : [];
|
||||
|
||||
const result = [{
|
||||
type: 'post',
|
||||
@@ -47,16 +48,25 @@ cli({
|
||||
|
||||
return result;
|
||||
} catch (e) {
|
||||
return [];
|
||||
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
|
||||
}
|
||||
})()
|
||||
` },
|
||||
{ map: {
|
||||
type: '${{ item.type }}',
|
||||
author: '${{ item.author }}',
|
||||
content: '${{ item.content }}',
|
||||
likes: '${{ item.likes }}',
|
||||
time: '${{ item.time }}',
|
||||
} },
|
||||
],
|
||||
`);
|
||||
if (Array.isArray(data)) {
|
||||
return data.map((item) => ({
|
||||
type: item.type ?? '',
|
||||
author: item.author ?? '',
|
||||
content: item.content ?? '',
|
||||
likes: item.likes ?? 0,
|
||||
time: item.time ?? '',
|
||||
}));
|
||||
}
|
||||
if (data?.reason === 'missing-data-script') {
|
||||
throw new CommandExecutionError('Jike post page did not expose the expected data script');
|
||||
}
|
||||
if (data?.reason === 'parse-error') {
|
||||
throw new CommandExecutionError(`Failed to parse Jike post data: ${data.message || 'unknown error'}`);
|
||||
}
|
||||
throw new CommandExecutionError('Jike post returned an unreadable payload');
|
||||
},
|
||||
});
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './post.js';
|
||||
import './topic.js';
|
||||
import './user.js';
|
||||
|
||||
function makePage(evaluateResult) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('jike read commands', () => {
|
||||
it('maps post rows from the browser-side extractor', async () => {
|
||||
const command = getRegistry().get('jike/post');
|
||||
const page = makePage([
|
||||
{ type: 'post', author: 'alice', content: 'hello', likes: 3, time: '2026-05-16' },
|
||||
{ type: 'comment', author: 'bob', content: 'nice', likes: 1, time: '2026-05-16' },
|
||||
]);
|
||||
|
||||
await expect(command.func(page, { id: 'post-1' })).resolves.toEqual([
|
||||
{ type: 'post', author: 'alice', content: 'hello', likes: 3, time: '2026-05-16' },
|
||||
{ type: 'comment', author: 'bob', content: 'nice', likes: 1, time: '2026-05-16' },
|
||||
]);
|
||||
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/originalPosts/post-1');
|
||||
});
|
||||
|
||||
it('maps topic rows and applies limit on the Node side', async () => {
|
||||
const command = getRegistry().get('jike/topic');
|
||||
const page = makePage([
|
||||
{ id: 'a', content: 'one', author: 'alice', likes: 1, comments: 2, time: 't1' },
|
||||
{ id: 'b', content: 'two', author: 'bob', likes: 3, comments: 4, time: 't2' },
|
||||
]);
|
||||
|
||||
await expect(command.func(page, { id: 'topic-1', limit: 1 })).resolves.toEqual([
|
||||
{
|
||||
content: 'one',
|
||||
author: 'alice',
|
||||
likes: 1,
|
||||
comments: 2,
|
||||
time: 't1',
|
||||
url: 'https://web.okjike.com/originalPost/a',
|
||||
},
|
||||
]);
|
||||
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/topics/topic-1');
|
||||
});
|
||||
|
||||
it('maps user rows and applies limit on the Node side', async () => {
|
||||
const command = getRegistry().get('jike/user');
|
||||
const page = makePage([
|
||||
{ id: 'a', content: 'one', type: 'post', likes: 1, comments: 2, time: 't1' },
|
||||
{ id: 'b', content: 'two', type: 'repost', likes: 3, comments: 4, time: 't2' },
|
||||
]);
|
||||
|
||||
await expect(command.func(page, { username: 'alice', limit: 1 })).resolves.toEqual([
|
||||
{
|
||||
id: 'a',
|
||||
content: 'one',
|
||||
type: 'post',
|
||||
likes: 1,
|
||||
comments: 2,
|
||||
time: 't1',
|
||||
url: 'https://web.okjike.com/originalPost/a',
|
||||
},
|
||||
]);
|
||||
expect(page.goto).toHaveBeenCalledWith('https://m.okjike.com/users/alice');
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError for malformed browser-side payloads', async () => {
|
||||
await expect(getRegistry().get('jike/post').func(makePage({ reason: 'missing-data-script' }), { id: 'post-1' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(getRegistry().get('jike/topic').func(makePage({ reason: 'parse-error', message: 'bad json' }), { id: 'topic-1' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
await expect(getRegistry().get('jike/user').func(makePage(null), { username: 'alice' }))
|
||||
.rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when topic or user extractors return no posts', async () => {
|
||||
await expect(getRegistry().get('jike/topic').func(makePage([]), { id: 'topic-1' }))
|
||||
.rejects.toBeInstanceOf(EmptyResultError);
|
||||
await expect(getRegistry().get('jike/user').func(makePage([]), { username: 'alice' }))
|
||||
.rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
+32
-19
@@ -1,4 +1,5 @@
|
||||
import { cli } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
cli({
|
||||
site: 'jike',
|
||||
name: 'topic',
|
||||
@@ -17,15 +18,16 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of posts' },
|
||||
],
|
||||
columns: ['content', 'author', 'likes', 'comments', 'time', 'url'],
|
||||
pipeline: [
|
||||
{ navigate: 'https://m.okjike.com/topics/${{ args.id }}' },
|
||||
{ evaluate: `(() => {
|
||||
func: async (page, args) => {
|
||||
await page.goto(`https://m.okjike.com/topics/${args.id}`);
|
||||
const limit = Number(args.limit) || 20;
|
||||
const data = await page.evaluate(`(() => {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return { ok: false, reason: 'missing-data-script' };
|
||||
try {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return [];
|
||||
const data = JSON.parse(el.textContent);
|
||||
const data = JSON.parse(el.textContent || '{}');
|
||||
const pageProps = data?.props?.pageProps || {};
|
||||
const posts = pageProps.posts || [];
|
||||
const posts = Array.isArray(pageProps.posts) ? pageProps.posts : [];
|
||||
return posts.map(p => ({
|
||||
content: (p.content || '').replace(/\\n/g, ' ').slice(0, 80),
|
||||
author: p.user?.screenName || '',
|
||||
@@ -35,18 +37,29 @@ cli({
|
||||
id: p.id || '',
|
||||
}));
|
||||
} catch (e) {
|
||||
return [];
|
||||
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
|
||||
}
|
||||
})()
|
||||
` },
|
||||
{ map: {
|
||||
content: '${{ item.content }}',
|
||||
author: '${{ item.author }}',
|
||||
likes: '${{ item.likes }}',
|
||||
comments: '${{ item.comments }}',
|
||||
time: '${{ item.time }}',
|
||||
url: 'https://web.okjike.com/originalPost/${{ item.id }}',
|
||||
} },
|
||||
{ limit: '${{ args.limit }}' },
|
||||
],
|
||||
`);
|
||||
if (Array.isArray(data)) {
|
||||
if (data.length === 0) {
|
||||
throw new EmptyResultError('jike topic', `No posts were returned for topic ${args.id}. Confirm the topic ID and login state.`);
|
||||
}
|
||||
return data.slice(0, limit).map((item) => ({
|
||||
content: item.content ?? '',
|
||||
author: item.author ?? '',
|
||||
likes: item.likes ?? 0,
|
||||
comments: item.comments ?? 0,
|
||||
time: item.time ?? '',
|
||||
url: `https://web.okjike.com/originalPost/${item.id ?? ''}`,
|
||||
}));
|
||||
}
|
||||
if (data?.reason === 'missing-data-script') {
|
||||
throw new CommandExecutionError('Jike topic page did not expose the expected data script');
|
||||
}
|
||||
if (data?.reason === 'parse-error') {
|
||||
throw new CommandExecutionError(`Failed to parse Jike topic data: ${data.message || 'unknown error'}`);
|
||||
}
|
||||
throw new CommandExecutionError('Jike topic returned an unreadable payload');
|
||||
},
|
||||
});
|
||||
|
||||
+33
-20
@@ -1,4 +1,5 @@
|
||||
import { cli } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
cli({
|
||||
site: 'jike',
|
||||
name: 'user',
|
||||
@@ -17,14 +18,15 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of posts' },
|
||||
],
|
||||
columns: ['id', 'content', 'type', 'likes', 'comments', 'time', 'url'],
|
||||
pipeline: [
|
||||
{ navigate: 'https://m.okjike.com/users/${{ args.username }}' },
|
||||
{ evaluate: `(() => {
|
||||
func: async (page, args) => {
|
||||
await page.goto(`https://m.okjike.com/users/${args.username}`);
|
||||
const limit = Number(args.limit) || 20;
|
||||
const data = await page.evaluate(`(() => {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return { ok: false, reason: 'missing-data-script' };
|
||||
try {
|
||||
const el = document.querySelector('script[type="application/json"]');
|
||||
if (!el) return [];
|
||||
const data = JSON.parse(el.textContent);
|
||||
const posts = data?.props?.pageProps?.posts || [];
|
||||
const data = JSON.parse(el.textContent || '{}');
|
||||
const posts = Array.isArray(data?.props?.pageProps?.posts) ? data.props.pageProps.posts : [];
|
||||
return posts.map(p => ({
|
||||
content: (p.content || '').replace(/\\n/g, ' ').slice(0, 80),
|
||||
type: p.type === 'ORIGINAL_POST' ? 'post' : p.type === 'REPOST' ? 'repost' : p.type || '',
|
||||
@@ -34,19 +36,30 @@ cli({
|
||||
id: p.id || '',
|
||||
}));
|
||||
} catch (e) {
|
||||
return [];
|
||||
return { ok: false, reason: 'parse-error', message: e?.message || String(e) };
|
||||
}
|
||||
})()
|
||||
` },
|
||||
{ map: {
|
||||
id: '${{ item.id }}',
|
||||
content: '${{ item.content }}',
|
||||
type: '${{ item.type }}',
|
||||
likes: '${{ item.likes }}',
|
||||
comments: '${{ item.comments }}',
|
||||
time: '${{ item.time }}',
|
||||
url: 'https://web.okjike.com/originalPost/${{ item.id }}',
|
||||
} },
|
||||
{ limit: '${{ args.limit }}' },
|
||||
],
|
||||
`);
|
||||
if (Array.isArray(data)) {
|
||||
if (data.length === 0) {
|
||||
throw new EmptyResultError('jike user', `No posts were returned for user ${args.username}. Confirm the username and login state.`);
|
||||
}
|
||||
return data.slice(0, limit).map((item) => ({
|
||||
id: item.id ?? '',
|
||||
content: item.content ?? '',
|
||||
type: item.type ?? '',
|
||||
likes: item.likes ?? 0,
|
||||
comments: item.comments ?? 0,
|
||||
time: item.time ?? '',
|
||||
url: `https://web.okjike.com/originalPost/${item.id ?? ''}`,
|
||||
}));
|
||||
}
|
||||
if (data?.reason === 'missing-data-script') {
|
||||
throw new CommandExecutionError('Jike user page did not expose the expected data script');
|
||||
}
|
||||
if (data?.reason === 'parse-error') {
|
||||
throw new CommandExecutionError(`Failed to parse Jike user data: ${data.message || 'unknown error'}`);
|
||||
}
|
||||
throw new CommandExecutionError('Jike user returned an unreadable payload');
|
||||
},
|
||||
});
|
||||
|
||||
@@ -56,7 +56,7 @@ cli({
|
||||
rows.push({
|
||||
rank: i + 1,
|
||||
score: item.baseScore ?? 0,
|
||||
author: user?.displayName ?? 'Unknown',
|
||||
author: user?.displayName ?? '',
|
||||
text: raw.length > 500 ? `${raw.slice(0, 500)}...` : raw,
|
||||
});
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
|
||||
const { gqlRequestMock } = vi.hoisted(() => ({ gqlRequestMock: vi.fn() }));
|
||||
vi.mock('./_helpers.js', async () => {
|
||||
const actual = await vi.importActual('./_helpers.js');
|
||||
return { ...actual, gqlRequest: gqlRequestMock };
|
||||
});
|
||||
|
||||
import './frontpage.js';
|
||||
|
||||
describe('lesswrong frontpage', () => {
|
||||
beforeEach(() => {
|
||||
gqlRequestMock.mockReset();
|
||||
});
|
||||
|
||||
it('emits empty-string for missing user.displayName instead of a sentinel', async () => {
|
||||
const command = getRegistry().get('lesswrong/frontpage');
|
||||
expect(command?.func).toBeDefined();
|
||||
gqlRequestMock.mockResolvedValueOnce({
|
||||
posts: {
|
||||
results: [
|
||||
{ _id: 'a1', slug: 'post-a', title: 'Has author', user: { displayName: 'Real Person' }, baseScore: 10, commentCount: 3 },
|
||||
{ _id: 'b2', slug: 'post-b', title: 'Deleted user', user: null, baseScore: 5, commentCount: 0 },
|
||||
{ _id: 'c3', slug: 'post-c', title: 'Missing name', user: {}, baseScore: 7, commentCount: 1 },
|
||||
],
|
||||
},
|
||||
});
|
||||
const rows = await command.func({ limit: 3 });
|
||||
expect(rows).toHaveLength(3);
|
||||
expect(rows[0]).toMatchObject({ rank: 1, title: 'Has author', author: 'Real Person', karma: 10, comments: 3 });
|
||||
expect(rows[1].author).toBe('');
|
||||
expect(rows[1].title).toBe('Deleted user');
|
||||
expect(rows[2].author).toBe('');
|
||||
expect(rows[2].title).toBe('Missing name');
|
||||
});
|
||||
});
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -34,7 +34,7 @@ cli({
|
||||
return [
|
||||
{
|
||||
title: post.title ?? '',
|
||||
author: post.user?.displayName ?? 'Unknown',
|
||||
author: post.user?.displayName ?? '',
|
||||
karma: post.baseScore ?? 0,
|
||||
comments: post.commentCount ?? 0,
|
||||
tags: (post.tags ?? []).map((tag) => tag.name ?? '').filter(Boolean).join(', '),
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return sequences.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -37,7 +37,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -22,7 +22,7 @@ cli({
|
||||
return posts.map((item, i) => ({
|
||||
rank: i + 1,
|
||||
title: item.title ?? '',
|
||||
author: item.user?.displayName ?? 'Unknown',
|
||||
author: item.user?.displayName ?? '',
|
||||
karma: item.baseScore ?? 0,
|
||||
comments: item.commentCount ?? 0,
|
||||
url: `https://${DOMAIN}/posts/${item._id}/${item.slug}`,
|
||||
|
||||
@@ -0,0 +1,138 @@
|
||||
/**
|
||||
* LinkedIn Learning course detail by slug, via /learning-api/courses?q=slug.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const DOMAIN = 'www.linkedin.com';
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function parseSlug(value) {
|
||||
const s = normalizeWhitespace(value);
|
||||
if (!s) throw new ArgumentError('<slug> is required');
|
||||
let slug = s;
|
||||
if (/^https?:\/\//i.test(s)) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = new URL(s);
|
||||
} catch {
|
||||
throw new ArgumentError(`Invalid LinkedIn Learning URL: "${s}"`);
|
||||
}
|
||||
const host = parsed.hostname.toLowerCase();
|
||||
if (host !== 'linkedin.com' && host !== 'www.linkedin.com') {
|
||||
throw new ArgumentError(`Invalid LinkedIn Learning host: "${parsed.hostname}"`);
|
||||
}
|
||||
const m = parsed.pathname.match(/^\/learning\/([^/?#]+)/);
|
||||
if (!m) throw new ArgumentError(`Invalid LinkedIn Learning course URL: "${s}"`);
|
||||
slug = m[1];
|
||||
} else {
|
||||
const m = s.match(/^\/?learning\/([^/?#]+)/);
|
||||
slug = m ? m[1] : s;
|
||||
}
|
||||
if (!/^[a-zA-Z0-9-_]+$/.test(slug)) {
|
||||
throw new ArgumentError(`Invalid LinkedIn Learning slug: "${slug}"`);
|
||||
}
|
||||
return slug;
|
||||
}
|
||||
|
||||
function buildFetchScript(url, csrf) {
|
||||
return String.raw`(async () => {
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: 'application/json',
|
||||
},
|
||||
});
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status };
|
||||
return { json: await res.json() };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
function parseCourse(el, slug) {
|
||||
const title = normalizeWhitespace(el?.title);
|
||||
if (!title) return null;
|
||||
const description = typeof el?.description === 'string'
|
||||
? el.description
|
||||
: (el?.description?.text || '');
|
||||
const duration = el?.duration?.unit === 'SECOND' ? String(el.duration.duration ?? '') : '';
|
||||
const released = el?.activatedAt ? new Date(el.activatedAt).toISOString().slice(0, 10) : '';
|
||||
return {
|
||||
title,
|
||||
slug,
|
||||
description,
|
||||
difficulty: el?.difficultyLevel || '',
|
||||
duration_sec: duration,
|
||||
videos_count: el?.videosCount ?? '',
|
||||
rating: typeof el?.rating?.averageRating === 'number' ? el.rating.averageRating.toFixed(2) : '',
|
||||
rating_count: el?.rating?.ratingCount ?? '',
|
||||
released,
|
||||
url: `https://www.linkedin.com/learning/${slug}`,
|
||||
};
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin-learning',
|
||||
name: 'course',
|
||||
access: 'read',
|
||||
description: 'Get LinkedIn Learning course detail by slug or course URL',
|
||||
domain: DOMAIN,
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'slug', type: 'string', required: true, positional: true, help: 'Course slug (e.g. agentic-ai-build-your-first-agentic-ai-system) or full /learning/<slug> URL' },
|
||||
],
|
||||
columns: ['title', 'slug', 'description', 'difficulty', 'duration_sec', 'videos_count', 'rating', 'rating_count', 'released', 'url'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning course');
|
||||
const slug = parseSlug(args.slug);
|
||||
|
||||
await page.goto('https://www.linkedin.com/learning/');
|
||||
await page.wait(3);
|
||||
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
|
||||
}
|
||||
const csrf = jsession.replace(/^"|"$/g, '');
|
||||
|
||||
const url = `https://www.linkedin.com/learning-api/courses?q=slug&slug=${encodeURIComponent(slug)}`;
|
||||
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
|
||||
}
|
||||
if (!result?.json) {
|
||||
throw new CommandExecutionError(`LinkedIn Learning courses lookup failed: ${result?.error ?? 'no payload'}`);
|
||||
}
|
||||
const elements = result.json?.elements;
|
||||
if (!Array.isArray(elements)) {
|
||||
throw new CommandExecutionError('LinkedIn Learning courses lookup returned malformed payload: missing elements array');
|
||||
}
|
||||
const el = elements[0];
|
||||
if (!el) {
|
||||
throw new EmptyResultError(`No LinkedIn Learning course found for slug "${slug}"`);
|
||||
}
|
||||
const row = parseCourse(el, slug);
|
||||
if (!row) {
|
||||
throw new CommandExecutionError('LinkedIn Learning courses lookup returned malformed course detail: missing title');
|
||||
}
|
||||
return [row];
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { parseSlug, parseCourse };
|
||||
@@ -0,0 +1,114 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './course.js';
|
||||
|
||||
const { parseSlug, parseCourse } = await import('./course.js').then((m) => m.__test__);
|
||||
|
||||
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
getCookies: vi.fn().mockResolvedValue(cookies),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin-learning course', () => {
|
||||
it('accepts a bare slug', () => {
|
||||
expect(parseSlug('agentic-ai-build')).toBe('agentic-ai-build');
|
||||
});
|
||||
|
||||
it('extracts a slug from a full /learning/<slug> URL', () => {
|
||||
expect(parseSlug('https://www.linkedin.com/learning/agentic-ai-build/?foo=1'))
|
||||
.toBe('agentic-ai-build');
|
||||
});
|
||||
|
||||
it('rejects non-LinkedIn Learning URLs before navigation', () => {
|
||||
expect(() => parseSlug('https://evil.example/learning/agentic-ai-build')).toThrow(ArgumentError);
|
||||
expect(() => parseSlug('https://www.linkedin.com/feed/update/123')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('rejects empty or invalid slugs with ArgumentError', () => {
|
||||
expect(() => parseSlug('')).toThrow(ArgumentError);
|
||||
expect(() => parseSlug(' ')).toThrow(ArgumentError);
|
||||
expect(() => parseSlug('not a slug!')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('maps a course detail element to the canonical row shape', () => {
|
||||
const el = {
|
||||
title: 'Agentic AI: Build Your First Agentic AI System',
|
||||
description: { text: 'Dive into agentic AI...' },
|
||||
duration: { duration: 3932, unit: 'SECOND' },
|
||||
difficultyLevel: 'Intermediate',
|
||||
videosCount: 18,
|
||||
rating: { averageRating: 4.5, ratingCount: 259 },
|
||||
activatedAt: 1774569600000,
|
||||
};
|
||||
const row = parseCourse(el, 'agentic-ai-build-your-first-agentic-ai-system');
|
||||
expect(row.title).toBe('Agentic AI: Build Your First Agentic AI System');
|
||||
expect(row.slug).toBe('agentic-ai-build-your-first-agentic-ai-system');
|
||||
expect(row.description).toBe('Dive into agentic AI...');
|
||||
expect(row.difficulty).toBe('Intermediate');
|
||||
expect(row.duration_sec).toBe('3932');
|
||||
expect(row.videos_count).toBe(18);
|
||||
expect(row.rating).toBe('4.50');
|
||||
expect(row.rating_count).toBe(259);
|
||||
expect(row.released).toBe('2026-03-27');
|
||||
expect(row.url).toBe('https://www.linkedin.com/learning/agentic-ai-build-your-first-agentic-ai-system');
|
||||
});
|
||||
|
||||
it('handles description as a bare string', () => {
|
||||
const row = parseCourse({ title: 't', description: 'plain string' }, 'x');
|
||||
expect(row.description).toBe('plain string');
|
||||
});
|
||||
|
||||
it('preserves the full course description', () => {
|
||||
const text = 'x'.repeat(350);
|
||||
const row = parseCourse({ title: 't', description: { text } }, 'x');
|
||||
expect(row.description).toBe(text);
|
||||
});
|
||||
|
||||
it('returns empty fields when upstream omits them', () => {
|
||||
const row = parseCourse({ title: 't' }, 'x');
|
||||
expect(row.title).toBe('t');
|
||||
expect(row.duration_sec).toBe('');
|
||||
expect(row.rating).toBe('');
|
||||
expect(row.released).toBe('');
|
||||
});
|
||||
|
||||
it('returns null when upstream omits the core title evidence', () => {
|
||||
expect(parseCourse({}, 'x')).toBeNull();
|
||||
expect(parseCourse({ title: ' ' }, 'x')).toBeNull();
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when JSESSIONID is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/course');
|
||||
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [{}] } } });
|
||||
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when no element matches the slug', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/course');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [] } } });
|
||||
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the elements array is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/course');
|
||||
const page = makePage({ evaluateResult: { json: { data: {} } } });
|
||||
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the first detail element is malformed', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/course');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [{}] } } });
|
||||
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError on fetch errors', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/course');
|
||||
const page = makePage({ evaluateResult: { error: 'HTTP 500' } });
|
||||
await expect(cmd.func(page, { slug: 'agentic-ai-build' })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,155 @@
|
||||
/**
|
||||
* LinkedIn Learning search via the public learning-api REST endpoint.
|
||||
* Shares cookie session with linkedin.com; no Commercial Use Limit.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const DOMAIN = 'www.linkedin.com';
|
||||
const MAX_LIMIT = 50;
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function parseLimit(value) {
|
||||
if (value === undefined || value === null || value === '') return 10;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function buildFetchScript(url, csrf) {
|
||||
return String.raw`(async () => {
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: 'application/json',
|
||||
},
|
||||
});
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status };
|
||||
return { json: await res.json() };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
function parseAuthors(authors) {
|
||||
if (!Array.isArray(authors)) return '';
|
||||
return authors
|
||||
.map((a) => normalizeWhitespace((a?.firstName ?? '') + ' ' + (a?.lastName ?? '')))
|
||||
.filter(Boolean)
|
||||
.join(', ');
|
||||
}
|
||||
|
||||
function durationSeconds(length) {
|
||||
const ts = length?.['com.linkedin.common.TimeSpan'];
|
||||
if (!ts || ts.unit !== 'SECOND') return '';
|
||||
return String(ts.duration ?? '');
|
||||
}
|
||||
|
||||
function averageRating(rating) {
|
||||
if (!rating) return '';
|
||||
if (typeof rating.averageRating === 'number') return rating.averageRating.toFixed(2);
|
||||
if (typeof rating.ratingSum === 'number' && typeof rating.ratingCount === 'number' && rating.ratingCount > 0) {
|
||||
return (rating.ratingSum / rating.ratingCount).toFixed(2);
|
||||
}
|
||||
return '';
|
||||
}
|
||||
|
||||
function parseRow(el, rank) {
|
||||
const type = el?.entityType || '';
|
||||
const slug = el?.slug || '';
|
||||
if (!slug) return null;
|
||||
return {
|
||||
rank,
|
||||
type,
|
||||
title: el?.headline?.title?.text || '',
|
||||
instructor: parseAuthors(el?.authors),
|
||||
difficulty: el?.difficultyLevel || '',
|
||||
duration_sec: durationSeconds(el?.length),
|
||||
rating: averageRating(el?.rating),
|
||||
rating_count: el?.rating?.ratingCount ?? '',
|
||||
viewers: el?.viewerCount ?? '',
|
||||
url: slug ? `https://www.linkedin.com/learning/${slug}` : '',
|
||||
};
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin-learning',
|
||||
name: 'search',
|
||||
access: 'read',
|
||||
description: 'Search LinkedIn Learning courses, videos, and learning paths by keyword',
|
||||
domain: DOMAIN,
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'Search keywords, e.g. "AI agent"' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: `Maximum results to return (1-${MAX_LIMIT})` },
|
||||
],
|
||||
columns: ['rank', 'type', 'title', 'instructor', 'difficulty', 'duration_sec', 'rating', 'rating_count', 'viewers', 'url'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning search');
|
||||
const keywords = normalizeWhitespace(args.keywords);
|
||||
if (!keywords) throw new ArgumentError('--keywords is required');
|
||||
const limit = parseLimit(args.limit);
|
||||
|
||||
await page.goto('https://www.linkedin.com/learning/');
|
||||
await page.wait(3);
|
||||
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
|
||||
}
|
||||
const csrf = jsession.replace(/^"|"$/g, '');
|
||||
|
||||
const url = `https://www.linkedin.com/learning-api/searchV2?keywords=${encodeURIComponent(keywords)}&q=keywords`;
|
||||
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
|
||||
}
|
||||
if (!result?.json) {
|
||||
throw new CommandExecutionError(`LinkedIn Learning searchV2 failed: ${result?.error ?? 'no payload'}`);
|
||||
}
|
||||
const elements = result.json?.elements;
|
||||
if (!Array.isArray(elements)) {
|
||||
throw new CommandExecutionError('LinkedIn Learning searchV2 returned malformed payload: missing elements array');
|
||||
}
|
||||
if (elements.length === 0) {
|
||||
throw new EmptyResultError(`No LinkedIn Learning results for "${keywords}"`);
|
||||
}
|
||||
const rows = [];
|
||||
for (const el of elements) {
|
||||
if (rows.length >= limit) break;
|
||||
const row = parseRow(el, rows.length + 1);
|
||||
if (row) rows.push(row);
|
||||
}
|
||||
if (rows.length === 0) {
|
||||
throw new CommandExecutionError('LinkedIn Learning searchV2 returned no parseable rows with slug identity');
|
||||
}
|
||||
return rows;
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
parseLimit,
|
||||
parseAuthors,
|
||||
durationSeconds,
|
||||
averageRating,
|
||||
parseRow,
|
||||
buildFetchScript,
|
||||
};
|
||||
@@ -0,0 +1,144 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './search.js';
|
||||
|
||||
const { parseLimit, parseAuthors, durationSeconds, averageRating, parseRow, buildFetchScript } = await import('./search.js').then((m) => m.__test__);
|
||||
|
||||
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
getCookies: vi.fn().mockResolvedValue(cookies),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin-learning search', () => {
|
||||
it('validates --limit without silent clamping', () => {
|
||||
expect(parseLimit(undefined)).toBe(10);
|
||||
expect(parseLimit(1)).toBe(1);
|
||||
expect(parseLimit(50)).toBe(50);
|
||||
expect(() => parseLimit(0)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(51)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit('abc')).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(1.5)).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('joins author first/last names', () => {
|
||||
expect(parseAuthors([{ firstName: 'Jane', lastName: 'Doe' }])).toBe('Jane Doe');
|
||||
expect(parseAuthors([{ firstName: 'A', lastName: 'B' }, { firstName: 'C', lastName: 'D' }])).toBe('A B, C D');
|
||||
expect(parseAuthors([])).toBe('');
|
||||
expect(parseAuthors(undefined)).toBe('');
|
||||
});
|
||||
|
||||
it('extracts duration from TimeSpan only when unit is SECOND', () => {
|
||||
expect(durationSeconds({ 'com.linkedin.common.TimeSpan': { duration: 600, unit: 'SECOND' } })).toBe('600');
|
||||
expect(durationSeconds({ 'com.linkedin.common.TimeSpan': { duration: 10, unit: 'MINUTE' } })).toBe('');
|
||||
expect(durationSeconds(undefined)).toBe('');
|
||||
});
|
||||
|
||||
it('computes average rating from sum/count when averageRating is missing', () => {
|
||||
expect(averageRating({ ratingSum: 1165, ratingCount: 259 })).toBe('4.50');
|
||||
expect(averageRating({ averageRating: 4.32 })).toBe('4.32');
|
||||
expect(averageRating({ ratingSum: 0, ratingCount: 0 })).toBe('');
|
||||
expect(averageRating(undefined)).toBe('');
|
||||
});
|
||||
|
||||
it('maps a search result element to the canonical row shape', () => {
|
||||
const el = {
|
||||
entityType: 'COURSE',
|
||||
slug: 'agentic-ai-build-your-first-agentic-ai-system',
|
||||
headline: { title: { text: 'Agentic AI: Build Your First Agentic AI System' } },
|
||||
authors: [{ firstName: 'Aishwarya', lastName: 'Naresh Reganti' }],
|
||||
difficultyLevel: 'INTERMEDIATE',
|
||||
length: { 'com.linkedin.common.TimeSpan': { duration: 3932, unit: 'SECOND' } },
|
||||
rating: { ratingSum: 1165, ratingCount: 259 },
|
||||
viewerCount: 25323,
|
||||
};
|
||||
expect(parseRow(el, 1)).toEqual({
|
||||
rank: 1,
|
||||
type: 'COURSE',
|
||||
title: 'Agentic AI: Build Your First Agentic AI System',
|
||||
instructor: 'Aishwarya Naresh Reganti',
|
||||
difficulty: 'INTERMEDIATE',
|
||||
duration_sec: '3932',
|
||||
rating: '4.50',
|
||||
rating_count: 259,
|
||||
viewers: 25323,
|
||||
url: 'https://www.linkedin.com/learning/agentic-ai-build-your-first-agentic-ai-system',
|
||||
});
|
||||
});
|
||||
|
||||
it('drops rows without slug identity', () => {
|
||||
const row = parseRow({ entityType: 'COURSE', headline: { title: { text: 't' } } }, 2);
|
||||
expect(row).toBeNull();
|
||||
});
|
||||
|
||||
it('escapes the URL and csrf into the fetch script as literal strings', () => {
|
||||
const s = buildFetchScript('https://www.linkedin.com/learning-api/searchV2?keywords=AI', 'csrf-token-value');
|
||||
expect(s).toContain('"https://www.linkedin.com/learning-api/searchV2?keywords=AI"');
|
||||
expect(s).toContain('"csrf-token-value"');
|
||||
expect(s).toContain("'x-restli-protocol-version': '2.0.0'");
|
||||
expect(s).toContain('authRequired: true');
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when JSESSIONID cookie is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [] } } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when the fetch returns 403', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { authRequired: true, status: 403 } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the upstream payload is empty', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { error: 'fetch failed: socket' } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when zero elements come back', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [] } } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the elements array is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { json: { data: {} } } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when elements lack slug identity', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [{ headline: { title: { text: 'No slug' } } }] } } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('rejects empty keywords with ArgumentError before navigation', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [] } } });
|
||||
await expect(cmd.func(page, { keywords: ' ', limit: 5 })).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('returns ranked rows when the API responds normally', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/search');
|
||||
const elements = [
|
||||
{ entityType: 'COURSE', slug: 'a', headline: { title: { text: 'Course A' } }, authors: [{ firstName: 'Inst', lastName: 'A' }], difficultyLevel: 'BEGINNER', length: { 'com.linkedin.common.TimeSpan': { duration: 100, unit: 'SECOND' } } },
|
||||
{ entityType: 'COURSE', headline: { title: { text: 'No slug' } } },
|
||||
{ entityType: 'VIDEO', slug: 'b', headline: { title: { text: 'Video B' } } },
|
||||
];
|
||||
const page = makePage({ evaluateResult: { json: { elements } } });
|
||||
const rows = await cmd.func(page, { keywords: 'test', limit: 5 });
|
||||
expect(rows).toHaveLength(2);
|
||||
expect(rows[0].rank).toBe(1);
|
||||
expect(rows[0].title).toBe('Course A');
|
||||
expect(rows[1].title).toBe('Video B');
|
||||
expect(rows[1].url).toBe('https://www.linkedin.com/learning/b');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,133 @@
|
||||
/**
|
||||
* LinkedIn Learning personalized recommendations via the
|
||||
* feedRecommendationGroups carousels endpoint. The `learner` view
|
||||
* returns a small set of carousels (e.g. "Top picks for you"); this
|
||||
* command flattens the cards across them into a ranked list.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const DOMAIN = 'www.linkedin.com';
|
||||
const MAX_LIMIT = 50;
|
||||
const MAX_PER_CAROUSEL = 25;
|
||||
|
||||
function parseLimit(value) {
|
||||
if (value === undefined || value === null || value === '') return 10;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function buildFetchScript(url, csrf) {
|
||||
return String.raw`(async () => {
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: 'application/json',
|
||||
},
|
||||
});
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status };
|
||||
return { json: await res.json() };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
function parseCard(card, group, rank) {
|
||||
const slug = card?.slug || '';
|
||||
if (!slug) return null;
|
||||
return {
|
||||
rank,
|
||||
group: group?.title?.text || group?.annotation || '',
|
||||
type: card?.entityType || card?.localizedEntityName || '',
|
||||
title: card?.title?.text || card?.headline?.title?.text || card?.headline?.text || '',
|
||||
difficulty: card?.difficultyLevel || '',
|
||||
viewers: card?.viewerCount ?? '',
|
||||
url: slug ? `https://www.linkedin.com/learning/${slug}` : '',
|
||||
};
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin-learning',
|
||||
name: 'trending',
|
||||
access: 'read',
|
||||
description: 'Browse LinkedIn Learning recommended courses across personalized carousels',
|
||||
domain: DOMAIN,
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 10, help: `Maximum results to return (1-${MAX_LIMIT})` },
|
||||
],
|
||||
columns: ['rank', 'group', 'type', 'title', 'difficulty', 'viewers', 'url'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin-learning trending');
|
||||
const limit = parseLimit(args.limit);
|
||||
|
||||
await page.goto('https://www.linkedin.com/learning/');
|
||||
await page.wait(3);
|
||||
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
|
||||
}
|
||||
const csrf = jsession.replace(/^"|"$/g, '');
|
||||
|
||||
const url = `https://www.linkedin.com/learning-api/feedRecommendationGroups?countPerCarousel=${MAX_PER_CAROUSEL}&q=learner`;
|
||||
const result = unwrapEvaluateResult(await page.evaluate(buildFetchScript(url, csrf)));
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError(DOMAIN, `LinkedIn Learning auth failed (HTTP ${result.status ?? ''}).`);
|
||||
}
|
||||
if (!result?.json) {
|
||||
throw new CommandExecutionError(`LinkedIn Learning feedRecommendationGroups failed: ${result?.error ?? 'no payload'}`);
|
||||
}
|
||||
const groups = result.json?.elements;
|
||||
if (!Array.isArray(groups)) {
|
||||
throw new CommandExecutionError('LinkedIn Learning feedRecommendationGroups returned malformed payload: missing elements array');
|
||||
}
|
||||
const rows = [];
|
||||
const seen = new Set();
|
||||
let rank = 1;
|
||||
let sawCards = false;
|
||||
for (const group of groups) {
|
||||
const carousels = Array.isArray(group?.carousels) ? group.carousels : [];
|
||||
for (const carousel of carousels) {
|
||||
const cards = Array.isArray(carousel?.cards) ? carousel.cards : [];
|
||||
for (const card of cards) {
|
||||
sawCards = true;
|
||||
if (rows.length >= limit) break;
|
||||
const slug = card?.slug;
|
||||
if (!slug || seen.has(slug)) continue;
|
||||
seen.add(slug);
|
||||
const row = parseCard(card, carousel, rank);
|
||||
if (!row) continue;
|
||||
rows.push(row);
|
||||
rank += 1;
|
||||
}
|
||||
if (rows.length >= limit) break;
|
||||
}
|
||||
if (rows.length >= limit) break;
|
||||
}
|
||||
if (rows.length === 0) {
|
||||
if (sawCards) {
|
||||
throw new CommandExecutionError('LinkedIn Learning feedRecommendationGroups returned no parseable cards with slug identity');
|
||||
}
|
||||
throw new EmptyResultError('LinkedIn Learning returned no personalized recommendations');
|
||||
}
|
||||
return rows;
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = { parseLimit, parseCard };
|
||||
@@ -0,0 +1,123 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './trending.js';
|
||||
|
||||
const { parseLimit, parseCard } = await import('./trending.js').then((m) => m.__test__);
|
||||
|
||||
function makePage({ evaluateResult, cookies = [{ name: 'JSESSIONID', value: '"ajax:abc"' }] } = {}) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
getCookies: vi.fn().mockResolvedValue(cookies),
|
||||
evaluate: vi.fn().mockResolvedValue(evaluateResult),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin-learning trending', () => {
|
||||
it('validates --limit without silent clamping', () => {
|
||||
expect(parseLimit(undefined)).toBe(10);
|
||||
expect(parseLimit(50)).toBe(50);
|
||||
expect(() => parseLimit(0)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(51)).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('maps a carousel card to the canonical row shape', () => {
|
||||
const card = {
|
||||
entityType: 'COURSE',
|
||||
slug: 'storytelling-editing',
|
||||
difficultyLevel: 'BEGINNER_INTERMEDIATE',
|
||||
description: { text: 'Go beyond basic video editing.' },
|
||||
viewerCount: 12345,
|
||||
headline: { title: { text: 'The Art of Storytelling through Editing' } },
|
||||
};
|
||||
const group = { annotation: 'TOP_PICKS', title: { text: 'Top picks for you' } };
|
||||
expect(parseCard(card, group, 1)).toEqual({
|
||||
rank: 1,
|
||||
group: 'Top picks for you',
|
||||
type: 'COURSE',
|
||||
title: 'The Art of Storytelling through Editing',
|
||||
difficulty: 'BEGINNER_INTERMEDIATE',
|
||||
viewers: 12345,
|
||||
url: 'https://www.linkedin.com/learning/storytelling-editing',
|
||||
});
|
||||
});
|
||||
|
||||
it('drops cards without slug identity', () => {
|
||||
expect(parseCard({ title: { text: 'No slug' } }, { title: { text: 'G' } }, 1)).toBeNull();
|
||||
});
|
||||
|
||||
it('flattens carousels and dedups cards across them', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const page = makePage({
|
||||
evaluateResult: {
|
||||
json: {
|
||||
elements: [{
|
||||
carousels: [
|
||||
{
|
||||
title: { text: 'Top picks' },
|
||||
cards: [
|
||||
{ slug: 'a', headline: { title: { text: 'Course A' } } },
|
||||
{ headline: { title: { text: 'No slug' } } },
|
||||
{ slug: 'b', headline: { title: { text: 'Course B' } } },
|
||||
],
|
||||
},
|
||||
{
|
||||
title: { text: 'Trending in your network' },
|
||||
cards: [
|
||||
{ slug: 'a', headline: { title: { text: 'Dup of A' } } },
|
||||
{ slug: 'c', headline: { title: { text: 'Course C' } } },
|
||||
],
|
||||
},
|
||||
],
|
||||
}],
|
||||
},
|
||||
},
|
||||
});
|
||||
const rows = await cmd.func(page, { limit: 5 });
|
||||
expect(rows.map((r) => r.title)).toEqual(['Course A', 'Course B', 'Course C']);
|
||||
expect(rows.map((r) => r.rank)).toEqual([1, 2, 3]);
|
||||
expect(rows[0].group).toBe('Top picks');
|
||||
expect(rows[2].group).toBe('Trending in your network');
|
||||
});
|
||||
|
||||
it('respects --limit', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const cards = Array.from({ length: 6 }, (_, i) => ({ slug: `s${i}`, headline: { title: { text: `T${i}` } } }));
|
||||
const page = makePage({
|
||||
evaluateResult: {
|
||||
json: { elements: [{ carousels: [{ title: { text: 'G' }, cards }] }] },
|
||||
},
|
||||
});
|
||||
const rows = await cmd.func(page, { limit: 3 });
|
||||
expect(rows).toHaveLength(3);
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when JSESSIONID is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const page = makePage({ cookies: [], evaluateResult: { json: { elements: [] } } });
|
||||
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when no carousels yield cards', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const page = makePage({ evaluateResult: { json: { elements: [{ carousels: [] }] } } });
|
||||
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when the elements array is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const page = makePage({ evaluateResult: { json: { data: {} } } });
|
||||
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws CommandExecutionError when cards lack slug identity', async () => {
|
||||
const cmd = getRegistry().get('linkedin-learning/trending');
|
||||
const page = makePage({
|
||||
evaluateResult: {
|
||||
json: { elements: [{ carousels: [{ title: { text: 'G' }, cards: [{ headline: { title: { text: 'No slug' } } }] }] }] },
|
||||
},
|
||||
});
|
||||
await expect(cmd.func(page, { limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,401 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function normalizeName(value) {
|
||||
return normalizeWhitespace(value)
|
||||
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
|
||||
.replace(/\s+LinkedIn.*$/i, '')
|
||||
.replace(/\b(p\.?eng\.?|cpa|mba|ph\.?d\.?)\b/ig, '')
|
||||
.replace(/[^\p{L}\p{N}\s.'-]+/gu, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
}
|
||||
|
||||
function nameTokens(value) {
|
||||
return normalizeName(value)
|
||||
.replace(/[.'-]+/g, ' ')
|
||||
.split(/\s+/)
|
||||
.map((token) => token.trim())
|
||||
.filter((token) => token.length >= 2);
|
||||
}
|
||||
|
||||
function matchInvitationName(candidate, expected) {
|
||||
const candidateName = normalizeName(candidate);
|
||||
const expectedName = normalizeName(expected);
|
||||
if (!candidateName || !expectedName) return false;
|
||||
if (candidateName === expectedName) return true;
|
||||
if (candidateName.includes(expectedName) || expectedName.includes(candidateName)) return true;
|
||||
const candidateTokens = new Set(nameTokens(candidateName));
|
||||
const expectedTokens = nameTokens(expectedName);
|
||||
if (expectedTokens.length < 2 || candidateTokens.size < 2) return false;
|
||||
const matched = expectedTokens.filter((token) => candidateTokens.has(token)).length;
|
||||
return matched >= 2 && matched / expectedTokens.length >= 0.8;
|
||||
}
|
||||
|
||||
function isLinkedInHost(hostname) {
|
||||
const host = String(hostname || '').toLowerCase();
|
||||
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
|
||||
}
|
||||
|
||||
function canonicalizeLinkedInProfileUrl(value) {
|
||||
const raw = normalizeWhitespace(value);
|
||||
if (!raw) return '';
|
||||
try {
|
||||
const url = new URL(raw);
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
|
||||
const match = url.pathname.match(/^\/in\/([^/]+)\/?$/i);
|
||||
if (!match || !match[1]) return '';
|
||||
// LinkedIn redirects country subdomains (ca./uk./...) to www.; normalize the
|
||||
// host so an expected `ca.linkedin.com/in/x` matches the landed `www.linkedin.com/in/x`.
|
||||
url.hostname = 'www.linkedin.com';
|
||||
url.hash = '';
|
||||
url.search = '';
|
||||
if (!url.pathname.endsWith('/')) url.pathname += '/';
|
||||
return url.toString();
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function requireStringArg(args, key, label = key) {
|
||||
const value = normalizeWhitespace(args[key]);
|
||||
if (!value) throw new ArgumentError(`${label} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function requireLinkedInProfileUrl(value, label) {
|
||||
const url = canonicalizeLinkedInProfileUrl(value);
|
||||
if (!url) throw new ArgumentError(`${label} must be an exact https://www.linkedin.com/in/<profile>/ URL`);
|
||||
return url;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function clampNote(note) {
|
||||
const value = normalizeWhitespace(note);
|
||||
if (value.length > 300) throw new ArgumentError('--note must be 300 characters or fewer for LinkedIn connection requests');
|
||||
return value;
|
||||
}
|
||||
|
||||
function canonicalizeLinkedInInviteUrl(value) {
|
||||
try {
|
||||
const url = new URL(normalizeWhitespace(value), 'https://www.linkedin.com');
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
|
||||
if (!/^\/preload\/custom-invite\/?$/i.test(url.pathname)) return '';
|
||||
url.hostname = 'www.linkedin.com';
|
||||
url.hash = '';
|
||||
if (!url.pathname.endsWith('/')) url.pathname += '/';
|
||||
return url.toString();
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function assessProfileSafety(probe, expectedName, expectedProfileUrl) {
|
||||
const expected = normalizeWhitespace(expectedName);
|
||||
const actual = normalizeWhitespace(probe?.name || '');
|
||||
const expectedUrl = canonicalizeLinkedInProfileUrl(expectedProfileUrl);
|
||||
const actualUrl = canonicalizeLinkedInProfileUrl(probe?.url || '');
|
||||
if (probe?.authRequired) return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'auth_required', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
if (!actual) return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_not_found', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
if (expected && normalizeName(actual) !== normalizeName(expected)) {
|
||||
return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_mismatch', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
}
|
||||
if (expectedUrl && actualUrl && expectedUrl !== actualUrl) {
|
||||
return { ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_url_mismatch', expectedValue: expectedUrl, actualValue: actualUrl, observedUrl: actualUrl };
|
||||
}
|
||||
if (probe?.alreadyConnected) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'already_connected', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
if (probe?.pending) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connection_pending', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
if (!probe?.connectAvailable) return { ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connect_button_not_found', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
return { ok: true, safety: 'connectable', connectable: true, blockReason: 'verified', expectedValue: expected, actualValue: actual, observedUrl: actualUrl };
|
||||
}
|
||||
|
||||
function buildProfileProbeScript() {
|
||||
return String.raw`(() => {
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const text = document.body ? (document.body.innerText || '') : '';
|
||||
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|
||||
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|
||||
|| /captcha|verification required/i.test(text);
|
||||
const main = document.querySelector('main') || document.body;
|
||||
// LinkedIn profile pages no longer expose the name in an <h1>; the heading
|
||||
// markup churns, but document.title is a stable "Name | LinkedIn" pattern.
|
||||
const heading = main?.querySelector('h1, .text-heading-xlarge, [class*="heading-xlarge"]');
|
||||
const titleName = clean((document.title || '')
|
||||
.replace(/^\(\d+\+?\)\s*/, '')
|
||||
.replace(/\s*[||]\s*LinkedIn\s*$/i, ''));
|
||||
const name = clean(heading?.innerText || heading?.textContent || '') || titleName;
|
||||
const buttons = Array.from(document.querySelectorAll('button, [role="button"], a')).filter((el) => el.offsetParent !== null);
|
||||
const buttonLabels = buttons.map((button) => clean(button.innerText || button.textContent || button.getAttribute('aria-label'))).filter(Boolean);
|
||||
const lowerLabels = buttonLabels.map((label) => label.toLowerCase());
|
||||
const alreadyConnected = lowerLabels.some((label) => label === 'message' || label.includes('1st degree connection'));
|
||||
const pending = lowerLabels.some((label) => label === 'pending' || label.includes('pending'));
|
||||
const connectAvailable = lowerLabels.some((label) => label === 'connect' || label.startsWith('connect ') || label.includes(' invite '));
|
||||
// The Connect control is an <a> linking to LinkedIn's invitation route
|
||||
// (/preload/custom-invite/?vanityName=...). Capture it so the sender can
|
||||
// navigate straight to the invite dialog.
|
||||
const connectAnchor = buttons.find((el) => el.tagName === 'A'
|
||||
&& /^connect$/i.test(clean(el.innerText || el.textContent || el.getAttribute('aria-label'))));
|
||||
const connectHref = connectAnchor ? (connectAnchor.getAttribute('href') || '') : '';
|
||||
return {
|
||||
url: location.href,
|
||||
title: document.title || '',
|
||||
name,
|
||||
authRequired,
|
||||
alreadyConnected,
|
||||
pending,
|
||||
connectAvailable,
|
||||
connectHref,
|
||||
buttonLabels: buttonLabels.slice(0, 30),
|
||||
bodyText: text,
|
||||
};
|
||||
})()`;
|
||||
}
|
||||
|
||||
// Runs in-page on LinkedIn's invitation route (/preload/custom-invite/...),
|
||||
// where the "Add a note to your invitation?" dialog is already open.
|
||||
|
||||
function buildSentInvitationsProbeScript(expectedName, expectedProfileUrl) {
|
||||
return String.raw`(() => {
|
||||
const expectedName = ${JSON.stringify(expectedName)};
|
||||
const expectedUrl = ${JSON.stringify(canonicalizeLinkedInProfileUrl(expectedProfileUrl))};
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const normName = (s) => clean(s)
|
||||
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
|
||||
.replace(/\s+LinkedIn.*$/i, '')
|
||||
.replace(/\b(p\.?eng\.?|cpa|mba|ph\.?d\.?)\b/ig, '')
|
||||
.replace(/[^\p{L}\p{N}\s.'-]+/gu, ' ')
|
||||
.replace(/\s+/g, ' ')
|
||||
.trim()
|
||||
.toLowerCase();
|
||||
const tokens = (s) => normName(s).replace(/[.'-]+/g, ' ').split(/\s+/).map((t) => t.trim()).filter((t) => t.length >= 2);
|
||||
const nameMatchesReasonably = (candidate, expected) => {
|
||||
const c = normName(candidate);
|
||||
const e = normName(expected);
|
||||
if (!c || !e) return false;
|
||||
if (c === e || c.includes(e) || e.includes(c)) return true;
|
||||
const candidateTokens = new Set(tokens(c));
|
||||
const expectedTokens = tokens(e);
|
||||
if (expectedTokens.length < 2 || candidateTokens.size < 2) return false;
|
||||
const matched = expectedTokens.filter((token) => candidateTokens.has(token)).length;
|
||||
return matched >= 2 && matched / expectedTokens.length >= 0.8;
|
||||
};
|
||||
const canon = (value) => {
|
||||
try {
|
||||
const url = new URL(value, 'https://www.linkedin.com');
|
||||
if (!/^\/in\/[^/]+\/?$/i.test(url.pathname)) return '';
|
||||
url.protocol = 'https:';
|
||||
url.hostname = 'www.linkedin.com';
|
||||
url.hash = '';
|
||||
url.search = '';
|
||||
if (!url.pathname.endsWith('/')) url.pathname += '/';
|
||||
return url.toString();
|
||||
} catch { return ''; }
|
||||
};
|
||||
const text = document.body ? (document.body.innerText || '') : '';
|
||||
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|
||||
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|
||||
|| /captcha|verification required/i.test(text);
|
||||
if (authRequired) return { authRequired: true, found: false, matchedName: '', matchedUrl: '', visibleNames: [] };
|
||||
const structuralRows = Array.from(document.querySelectorAll('li, article, [data-view-name], .mn-invitation-card'));
|
||||
const linkRows = Array.from(document.querySelectorAll('a[href*="/in/"]'))
|
||||
.map((a) => a.closest('li') || a.closest('[data-view-name]') || a.closest('[class*="invitation"]') || a.closest('div'))
|
||||
.filter(Boolean);
|
||||
const rows = Array.from(new Set([...structuralRows, ...linkRows]));
|
||||
const visibleNames = [];
|
||||
for (const row of rows.slice(0, 25)) {
|
||||
const rowText = clean(row.innerText || row.textContent || '');
|
||||
if (!rowText) continue;
|
||||
const link = Array.from(row.querySelectorAll('a[href*="/in/"]'))
|
||||
.map((a) => ({ href: canon(a.href || a.getAttribute('href') || ''), text: clean(a.innerText || a.textContent || '') }))
|
||||
.find((a) => a.href || a.text);
|
||||
const candidateName = clean(link?.text || row.querySelector('span[aria-hidden="true"], h3, h2')?.textContent || rowText.split('\n')[0]);
|
||||
if (candidateName) visibleNames.push(candidateName);
|
||||
const candidateUrl = link?.href || '';
|
||||
const nameMatches = expectedName && candidateName && nameMatchesReasonably(candidateName, expectedName);
|
||||
const urlMatches = expectedUrl && candidateUrl && candidateUrl === expectedUrl;
|
||||
if (urlMatches || nameMatches) return { authRequired: false, found: true, matchedName: candidateName, matchedUrl: candidateUrl, visibleNames: visibleNames.slice(0, 20) };
|
||||
}
|
||||
return { authRequired: false, found: false, matchedName: '', matchedUrl: '', visibleNames: visibleNames.slice(0, 20) };
|
||||
})()`;
|
||||
}
|
||||
|
||||
function buildInviteScript(note) {
|
||||
return String.raw`(async () => {
|
||||
const note = ${JSON.stringify(note)};
|
||||
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
|
||||
const jitter = async (min = 450, max = 1150) => sleep(min + Math.floor(Math.random() * (max - min + 1)));
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const visible = (el) => el && el.offsetParent !== null;
|
||||
const label = (el) => clean(el?.innerText || el?.textContent || el?.getAttribute('aria-label'));
|
||||
const dialog = () => document.querySelector('[role="dialog"]');
|
||||
const dialogButton = (pattern) => {
|
||||
const dlg = dialog();
|
||||
if (!dlg) return null;
|
||||
return Array.from(dlg.querySelectorAll('button, [role="button"]')).filter(visible)
|
||||
.find((button) => pattern.test(label(button)));
|
||||
};
|
||||
|
||||
if (!dialog()) return { ok: false, status: 'blocked', reason: 'invite_dialog_not_found' };
|
||||
|
||||
if (!note) {
|
||||
const sendDirect = dialogButton(/^send without a note$/i) || dialogButton(/^send$/i);
|
||||
if (!sendDirect) return { ok: false, status: 'blocked', reason: 'send_button_not_found' };
|
||||
await jitter();
|
||||
sendDirect.click();
|
||||
await jitter(1400, 2400);
|
||||
return { ok: true, status: 'sent', reason: 'invitation_sent_without_note' };
|
||||
}
|
||||
|
||||
const addNote = dialogButton(/^add a note$/i);
|
||||
if (!addNote) return { ok: false, status: 'blocked', reason: 'add_note_button_not_found' };
|
||||
await jitter();
|
||||
addNote.click();
|
||||
await jitter(800, 1400);
|
||||
|
||||
const textarea = document.querySelector('#custom-message')
|
||||
|| Array.from(document.querySelectorAll('textarea')).find(visible);
|
||||
if (!textarea) return { ok: false, status: 'blocked', reason: 'note_textarea_not_found' };
|
||||
textarea.focus();
|
||||
// React tracks textarea values through the native setter; assigning .value
|
||||
// directly would leave component state (and the Send button) unchanged.
|
||||
const nativeSetter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set;
|
||||
nativeSetter.call(textarea, note);
|
||||
textarea.dispatchEvent(new Event('input', { bubbles: true }));
|
||||
textarea.dispatchEvent(new Event('change', { bubbles: true }));
|
||||
await jitter(700, 1300);
|
||||
|
||||
const send = dialogButton(/^send$/i);
|
||||
if (!send) return { ok: false, status: 'blocked', reason: 'send_button_not_found' };
|
||||
if (send.disabled || send.getAttribute('aria-disabled') === 'true') {
|
||||
return { ok: false, status: 'blocked', reason: 'send_button_disabled' };
|
||||
}
|
||||
send.click();
|
||||
await jitter(1400, 2400);
|
||||
return { ok: true, status: 'sent', reason: 'invitation_sent_with_note' };
|
||||
})()`;
|
||||
}
|
||||
|
||||
async function probeProfile(page) {
|
||||
return unwrapEvaluateResult(await page.evaluate(buildProfileProbeScript()));
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'connect',
|
||||
access: 'write',
|
||||
description: 'Fail-closed LinkedIn connection request sender that verifies the exact profile before optionally sending a note',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.UI,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'profile-url', type: 'string', required: true, positional: true, help: 'Exact LinkedIn profile URL to open and verify' },
|
||||
{ name: 'expected-name', type: 'string', required: true, help: 'Expected visible profile name' },
|
||||
{ name: 'note', type: 'string', required: false, default: '', help: 'Optional connection note, max 300 chars' },
|
||||
{ name: 'send', type: 'bool', required: false, default: false, help: 'Actually click Send. Default is dry-run verification only.' },
|
||||
],
|
||||
columns: ['status', 'recipient', 'reason', 'profile_url', 'note_chars', 'connectable', 'delivery_verified', 'matched_invitation_name', 'matched_invitation_url', 'actualValue', 'blockReason', 'expectedValue', 'observedUrl', 'safety'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin connect');
|
||||
const profileUrl = requireLinkedInProfileUrl(requireStringArg(args, 'profile-url', '--profile-url'), '--profile-url');
|
||||
const expectedName = requireStringArg(args, 'expected-name', '--expected-name');
|
||||
const note = clampNote(args.note || '');
|
||||
|
||||
await page.goto(profileUrl);
|
||||
await page.wait(6);
|
||||
let probe = await probeProfile(page);
|
||||
// The name resolves early (from document.title), but the profile action
|
||||
// buttons (Connect / Message / Pending) render later. Keep probing until
|
||||
// the action state has resolved, not merely until the name is visible.
|
||||
for (let attempt = 0; attempt < 8; attempt += 1) {
|
||||
const resolved = probe?.name
|
||||
&& (probe.connectAvailable || probe.alreadyConnected || probe.pending);
|
||||
if (resolved) break;
|
||||
await page.wait(2);
|
||||
probe = await probeProfile(page);
|
||||
}
|
||||
const safety = assessProfileSafety(probe, expectedName, profileUrl);
|
||||
if (safety.blockReason === 'auth_required') {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn connect requires an active signed-in LinkedIn browser session.');
|
||||
}
|
||||
if (!safety.ok && safety.safety === 'routine_non_connectable') {
|
||||
return [{ status: 'not_connectable', recipient: safety.actualValue, reason: safety.blockReason, profile_url: safety.observedUrl, note_chars: note.length, connectable: false }];
|
||||
}
|
||||
if (!safety.ok) {
|
||||
throw new CommandExecutionError(
|
||||
`LinkedIn connect blocked: ${safety.blockReason}`,
|
||||
`Expected ${safety.expectedValue}; actual ${safety.actualValue || 'not_visible'} at ${safety.observedUrl || 'url_not_available'}\nButtons: ${(probe?.buttonLabels || []).join(' | ')}`,
|
||||
);
|
||||
}
|
||||
if (!args.send) {
|
||||
return [{ status: 'connectable_dry_run', recipient: safety.actualValue, reason: safety.blockReason, profile_url: safety.observedUrl, note_chars: note.length, connectable: true }];
|
||||
}
|
||||
const inviteHref = probe?.connectHref || '';
|
||||
if (!inviteHref) {
|
||||
throw new CommandExecutionError('LinkedIn connect blocked: connect_link_not_found');
|
||||
}
|
||||
const inviteUrl = canonicalizeLinkedInInviteUrl(inviteHref);
|
||||
if (!inviteUrl) {
|
||||
throw new CommandExecutionError('LinkedIn connect blocked: invalid_connect_link');
|
||||
}
|
||||
await page.goto(inviteUrl);
|
||||
await page.wait(6);
|
||||
let result = unwrapEvaluateResult(await page.evaluate(buildInviteScript(note)));
|
||||
if (result?.reason === 'invite_dialog_not_found') {
|
||||
await page.wait(5);
|
||||
result = unwrapEvaluateResult(await page.evaluate(buildInviteScript(note)));
|
||||
}
|
||||
if (!result?.ok) throw new CommandExecutionError(`LinkedIn connect blocked: ${result?.reason || 'send_failed'}`);
|
||||
// LinkedIn can take a few seconds after the Send click to materialize the
|
||||
// new invite in /mynetwork/invitation-manager/sent/. Wait before the
|
||||
// first check, then retry page loads for propagation lag.
|
||||
await page.wait(8);
|
||||
let sentProbe = null;
|
||||
for (let attempt = 0; attempt < 3; attempt += 1) {
|
||||
await page.goto('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
|
||||
await page.wait(attempt === 0 ? 6 : 4);
|
||||
sentProbe = unwrapEvaluateResult(await page.evaluate(buildSentInvitationsProbeScript(expectedName, profileUrl)));
|
||||
if (sentProbe?.found || sentProbe?.authRequired) break;
|
||||
if (attempt < 2) await page.wait(5);
|
||||
}
|
||||
if (sentProbe?.authRequired) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn sent-invitations verification requires an active signed-in LinkedIn browser session.');
|
||||
}
|
||||
const verified = Boolean(sentProbe?.found);
|
||||
return [{
|
||||
status: verified ? 'sent_verified' : 'send_unverified',
|
||||
recipient: safety.actualValue,
|
||||
reason: verified ? 'sent_invitation_verified' : 'sent_invitation_not_found_after_retries',
|
||||
profile_url: safety.observedUrl,
|
||||
note_chars: note.length,
|
||||
connectable: true,
|
||||
delivery_verified: verified,
|
||||
matched_invitation_name: sentProbe?.matchedName || '',
|
||||
matched_invitation_url: sentProbe?.matchedUrl || '',
|
||||
}];
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
normalizeName,
|
||||
matchInvitationName,
|
||||
canonicalizeLinkedInProfileUrl,
|
||||
canonicalizeLinkedInInviteUrl,
|
||||
unwrapEvaluateResult,
|
||||
clampNote,
|
||||
assessProfileSafety,
|
||||
buildSentInvitationsProbeScript,
|
||||
};
|
||||
@@ -0,0 +1,213 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './connect.js';
|
||||
|
||||
const {
|
||||
normalizeName,
|
||||
matchInvitationName,
|
||||
canonicalizeLinkedInProfileUrl,
|
||||
canonicalizeLinkedInInviteUrl,
|
||||
unwrapEvaluateResult,
|
||||
clampNote,
|
||||
assessProfileSafety,
|
||||
} = await import('./connect.js').then((m) => m.__test__);
|
||||
|
||||
function makeFakePage(probe, sendResult = { ok: true, status: 'sent', reason: 'connection_request_sent' }) {
|
||||
return {
|
||||
goto: vi.fn(async () => undefined),
|
||||
wait: vi.fn(async () => undefined),
|
||||
evaluate: vi.fn(async (script) => {
|
||||
const text = String(script);
|
||||
if (text.includes('custom-message') || text.includes('invite_dialog_not_found')) return sendResult;
|
||||
return probe;
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function makeSequentialFakePage(values) {
|
||||
let index = 0;
|
||||
return {
|
||||
goto: vi.fn(async () => undefined),
|
||||
wait: vi.fn(async () => undefined),
|
||||
evaluate: vi.fn(async (script) => {
|
||||
const text = String(script);
|
||||
if (text.includes('custom-message') || text.includes('invite_dialog_not_found')) return { ok: true, status: 'sent', reason: 'connection_request_sent' };
|
||||
const value = values[Math.min(index, values.length - 1)];
|
||||
index += 1;
|
||||
return value;
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin connect helpers', () => {
|
||||
it('normalizes names and profile URLs', () => {
|
||||
expect(normalizeName('Jane Doe • 2nd degree connection')).toBe('jane doe');
|
||||
expect(matchInvitationName('Jane Doe, P.Eng.', ' jane doe ')).toBe(true);
|
||||
expect(matchInvitationName('Jane Q. Doe', 'Jane Doe')).toBe(true);
|
||||
expect(matchInvitationName('Janet Doe', 'Jane Doe')).toBe(false);
|
||||
expect(canonicalizeLinkedInProfileUrl('https://www.linkedin.com/in/jane/?mini=true#x'))
|
||||
.toBe('https://www.linkedin.com/in/jane/');
|
||||
expect(canonicalizeLinkedInProfileUrl('https://ca.linkedin.com/in/jane/?mini=true#x'))
|
||||
.toBe('https://www.linkedin.com/in/jane/');
|
||||
expect(canonicalizeLinkedInProfileUrl('https://www.linkedin.com/company/opencli/')).toBe('');
|
||||
expect(canonicalizeLinkedInProfileUrl('https://evil-linkedin.com/in/jane/')).toBe('');
|
||||
expect(canonicalizeLinkedInProfileUrl('http://www.linkedin.com/in/jane/')).toBe('');
|
||||
});
|
||||
|
||||
it('only accepts LinkedIn invitation route hrefs for sending', () => {
|
||||
expect(canonicalizeLinkedInInviteUrl('/preload/custom-invite/?vanityName=jane'))
|
||||
.toBe('https://www.linkedin.com/preload/custom-invite/?vanityName=jane');
|
||||
expect(canonicalizeLinkedInInviteUrl('https://www.linkedin.com/feed/')).toBe('');
|
||||
expect(canonicalizeLinkedInInviteUrl('https://evil-linkedin.com/preload/custom-invite/?vanityName=jane')).toBe('');
|
||||
});
|
||||
|
||||
it('unwraps browser bridge evaluate envelopes', () => {
|
||||
expect(unwrapEvaluateResult({ session: 'site:linkedin:1', data: { ok: true } })).toEqual({ ok: true });
|
||||
const raw = { ok: true };
|
||||
expect(unwrapEvaluateResult(raw)).toBe(raw);
|
||||
});
|
||||
|
||||
it('enforces LinkedIn note length', () => {
|
||||
expect(clampNote(' hello\nthere ')).toBe('hello there');
|
||||
expect(() => clampNote('x'.repeat(301))).toThrow('--note must be 300 characters or fewer');
|
||||
});
|
||||
|
||||
it('fails closed on wrong profile name, pending state, or missing connect button', () => {
|
||||
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true }, 'Janet Doe', 'https://www.linkedin.com/in/jane/').blockReason)
|
||||
.toBe('profile_name_mismatch');
|
||||
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', pending: true, connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/').blockReason)
|
||||
.toBe('connection_pending');
|
||||
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/' }, 'Jane Doe', 'https://www.linkedin.com/in/jane/').blockReason)
|
||||
.toBe('connect_button_not_found');
|
||||
});
|
||||
|
||||
it('classifies routine non-connectable profiles separately from unsafe blocks', () => {
|
||||
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', alreadyConnected: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
|
||||
.toMatchObject({ ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'already_connected' });
|
||||
expect(assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', pending: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
|
||||
.toMatchObject({ ok: false, safety: 'routine_non_connectable', connectable: false, blockReason: 'connection_pending' });
|
||||
expect(assessProfileSafety({ name: 'Wrong Person', url: 'https://www.linkedin.com/in/wrong/', connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/'))
|
||||
.toMatchObject({ ok: false, safety: 'unsafe_block', connectable: null, blockReason: 'profile_name_mismatch' });
|
||||
});
|
||||
|
||||
it('passes only when profile url, name, and connect affordance all match', () => {
|
||||
const result = assessProfileSafety({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/?mini=true', connectAvailable: true }, 'Jane Doe', 'https://www.linkedin.com/in/jane/');
|
||||
expect(result).toMatchObject({ ok: true, blockReason: 'verified', actualValue: 'Jane Doe', connectable: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('linkedin connect command', () => {
|
||||
it('registers as a write command and dry-runs by default', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
expect(command).toBeDefined();
|
||||
expect(command.access).toBe('write');
|
||||
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] });
|
||||
const rows = await command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
});
|
||||
expect(rows[0]).toMatchObject({ status: 'connectable_dry_run', recipient: 'Jane Doe', reason: 'verified', connectable: true });
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('returns a clean not_connectable dry-run row for routine blocked states', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', alreadyConnected: true, buttonLabels: ['Message'] });
|
||||
const rows = await command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
});
|
||||
expect(rows[0]).toMatchObject({ status: 'not_connectable', recipient: 'Jane Doe', reason: 'already_connected', connectable: false });
|
||||
});
|
||||
|
||||
it('does not send when recipient verification fails', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeFakePage({ name: 'Wrong Person', url: 'https://www.linkedin.com/in/wrong/', connectAvailable: true, buttonLabels: ['Connect'] });
|
||||
await expect(command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
send: true,
|
||||
})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('rejects non-profile URLs before navigating', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeFakePage({});
|
||||
|
||||
await expect(command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/company/opencli/',
|
||||
'expected-name': 'Jane Doe',
|
||||
send: true,
|
||||
})).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('blocks send when the connect link is not LinkedIn invitation route', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeFakePage({ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: 'https://www.linkedin.com/feed/', buttonLabels: ['Connect'] });
|
||||
|
||||
await expect(command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
send: true,
|
||||
})).rejects.toThrow('invalid_connect_link');
|
||||
});
|
||||
|
||||
it('sends only when --send is true after verification and sent-invitations confirms delivery', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeSequentialFakePage([
|
||||
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
|
||||
{ found: true, matchedName: 'Jane Doe', matchedUrl: 'https://www.linkedin.com/in/jane/' },
|
||||
]);
|
||||
const rows = await command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
send: true,
|
||||
});
|
||||
expect(rows[0]).toMatchObject({ status: 'sent_verified', recipient: 'Jane Doe', reason: 'sent_invitation_verified', delivery_verified: true });
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
|
||||
});
|
||||
|
||||
it('retries sent-invitations verification before reporting unverified', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeSequentialFakePage([
|
||||
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
|
||||
{ found: false, matchedName: '', matchedUrl: '', visibleNames: ['Other Person'] },
|
||||
{ found: false, matchedName: '', matchedUrl: '', visibleNames: ['Other Person'] },
|
||||
{ found: true, matchedName: 'Jane Doe, P.Eng.', matchedUrl: '' },
|
||||
]);
|
||||
const rows = await command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
send: true,
|
||||
});
|
||||
expect(rows[0]).toMatchObject({ status: 'sent_verified', recipient: 'Jane Doe', reason: 'sent_invitation_verified', delivery_verified: true, matched_invitation_name: 'Jane Doe, P.Eng.' });
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/mynetwork/invitation-manager/sent/');
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(5);
|
||||
});
|
||||
|
||||
it('does not report sent when sent-invitations verification fails after retries', async () => {
|
||||
const command = getRegistry().get('linkedin/connect');
|
||||
const page = makeSequentialFakePage([
|
||||
{ name: 'Jane Doe', url: 'https://www.linkedin.com/in/jane/', connectAvailable: true, connectHref: '/preload/custom-invite/?vanityName=jane', buttonLabels: ['Connect'] },
|
||||
{ found: false, matchedName: '', matchedUrl: '' },
|
||||
{ found: false, matchedName: '', matchedUrl: '' },
|
||||
{ found: false, matchedName: '', matchedUrl: '' },
|
||||
]);
|
||||
const rows = await command.func(page, {
|
||||
'profile-url': 'https://www.linkedin.com/in/jane/',
|
||||
'expected-name': 'Jane Doe',
|
||||
note: 'quick note',
|
||||
send: true,
|
||||
});
|
||||
expect(rows[0]).toMatchObject({ status: 'send_unverified', recipient: 'Jane Doe', reason: 'sent_invitation_not_found_after_retries', delivery_verified: false });
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(5);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,234 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'linkedin.com';
|
||||
const MESSAGING_URL = 'https://www.linkedin.com/messaging/';
|
||||
const MIN_LIMIT = 1;
|
||||
const MAX_LIMIT = 100;
|
||||
const DEFAULT_LIMIT = 40;
|
||||
|
||||
// ── Why this command reads an API response instead of scraping the DOM ──
|
||||
//
|
||||
// LinkedIn's messaging UI is a realtime, virtualized SPA. Scraping the rendered
|
||||
// conversation list is brittle: rows lazy-render, the list virtualizes, and the
|
||||
// markup churns. Instead we let the page load /messaging/ exactly as a human
|
||||
// would, which makes the page fire its own `messengerConversations` GraphQL
|
||||
// call. We then re-issue that same request (URL lifted from the Performance API,
|
||||
// so the rotating queryId is always current) and parse LinkedIn's normalized
|
||||
// JSON. Same session, same origin, same request the page already makes.
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function threadUrl(threadId) {
|
||||
return threadId ? `https://www.linkedin.com/messaging/thread/${threadId}/` : '';
|
||||
}
|
||||
|
||||
// Runs in-page: locate the messengerConversations request the page already fired.
|
||||
// Prefers the category-scoped query (the primary inbox) over the sync-token query.
|
||||
function findMessagingApiUrl() {
|
||||
if (/\/(login|checkpoint|authwall|uas)/i.test(location.pathname)) return { loginRequired: true };
|
||||
const urls = performance.getEntriesByType('resource').map((e) => e.name);
|
||||
const matches = (re) => urls.find((u) => /messengerConversations\.[a-f0-9]+/i.test(u) && re.test(u));
|
||||
const url =
|
||||
matches(/PRIMARY_INBOX/i) ||
|
||||
matches(/conversationCategoryPredicate/i) ||
|
||||
urls.find((u) => /messengerConversations\.[a-f0-9]+/i.test(u) && /mailboxUrn/i.test(u));
|
||||
if (!url) return { url: null };
|
||||
const mb = url.match(/mailboxUrn:(urn[^,)&]+)/i);
|
||||
return { url, mailboxUrn: mb ? decodeURIComponent(mb[1]) : '' };
|
||||
}
|
||||
|
||||
// Runs in-page: re-issue the messaging request with the session's csrf token.
|
||||
async function fetchMessagingApi(url, csrf) {
|
||||
try {
|
||||
const res = await fetch(url, {
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'csrf-token': csrf,
|
||||
accept: 'application/vnd.linkedin.normalized+json+2.1',
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
},
|
||||
});
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, error: 'HTTP ' + res.status };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status };
|
||||
return { json: await res.json() };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
}
|
||||
|
||||
// Parse LinkedIn's normalized messaging JSON into plain conversation rows.
|
||||
// `included` is a flat entity array; conversations reference participants and
|
||||
// messages by URN, which we resolve through a urn->entity index. Exported for
|
||||
// unit testing against a captured fixture.
|
||||
function parseConversations(normalized, mailboxUrn) {
|
||||
if (!normalized || typeof normalized !== 'object' || Array.isArray(normalized) || !Array.isArray(normalized.included)) {
|
||||
throw new CommandExecutionError('LinkedIn messaging API returned malformed normalized payload: missing included array');
|
||||
}
|
||||
const included = normalized.included;
|
||||
const byUrn = new Map();
|
||||
for (const o of included) {
|
||||
if (o && o.entityUrn) byUrn.set(o.entityUrn, o);
|
||||
}
|
||||
const norm = (s) => String(s == null ? '' : s).replace(/\s+/g, ' ').trim();
|
||||
|
||||
const participantInfo = (p) => {
|
||||
if (!p) return { name: '', kind: '' };
|
||||
const pt = p.participantType || {};
|
||||
if (pt.organization && pt.organization.name) return { name: norm(pt.organization.name.text), kind: 'organization' };
|
||||
if (pt.member) {
|
||||
const fn = pt.member.firstName && pt.member.firstName.text;
|
||||
const ln = pt.member.lastName && pt.member.lastName.text;
|
||||
return { name: norm([fn, ln].filter(Boolean).join(' ')), kind: 'member' };
|
||||
}
|
||||
if (pt.agent && pt.agent.name) return { name: norm(pt.agent.name.text), kind: 'agent' };
|
||||
return { name: '', kind: '' };
|
||||
};
|
||||
|
||||
const entries = [];
|
||||
for (const conv of included) {
|
||||
if (!conv || conv.$type !== 'com.linkedin.messenger.Conversation') continue;
|
||||
const threadId = String(conv.backendUrn || '').replace(/^urn:li:messagingThread:/, '');
|
||||
if (!threadId) {
|
||||
throw new CommandExecutionError('LinkedIn messaging API returned a conversation without thread id');
|
||||
}
|
||||
|
||||
const others = [];
|
||||
let counterpartyKind = '';
|
||||
for (const urn of conv['*conversationParticipants'] || []) {
|
||||
const p = byUrn.get(urn);
|
||||
if (!p) continue;
|
||||
if (mailboxUrn && p.hostIdentityUrn === mailboxUrn) continue; // exclude the inbox owner
|
||||
const info = participantInfo(p);
|
||||
if (info.name) {
|
||||
others.push(info.name);
|
||||
if (!counterpartyKind) counterpartyKind = info.kind;
|
||||
}
|
||||
}
|
||||
|
||||
const msgUrns = (conv.messages && conv.messages['*elements']) || [];
|
||||
const lastMsg = byUrn.get(msgUrns[0]);
|
||||
let preview = lastMsg && lastMsg.body ? norm(lastMsg.body.text) : '';
|
||||
if (!preview) preview = norm(conv.descriptionText || '');
|
||||
|
||||
const activityMs = Number(conv.lastActivityAt || 0);
|
||||
entries.push({
|
||||
activityMs,
|
||||
row: {
|
||||
thread_id: threadId,
|
||||
person_name: conv.title ? norm(conv.title) : others.join(', '),
|
||||
last_message_preview: preview.slice(0, 300),
|
||||
unread: Number(conv.unreadCount || 0) > 0 || conv.read === false,
|
||||
counterparty_type: counterpartyKind,
|
||||
category: Array.isArray(conv.categories) ? conv.categories.join(',') : '',
|
||||
timestamp: activityMs ? new Date(activityMs).toISOString() : '',
|
||||
},
|
||||
});
|
||||
}
|
||||
// Most-recent first; the sort key is kept off the returned row.
|
||||
entries.sort((a, b) => b.activityMs - a.activityMs);
|
||||
return entries.map((entry) => entry.row);
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'inbox',
|
||||
access: 'read',
|
||||
description: 'List LinkedIn messaging inbox conversations and unread messages',
|
||||
domain: 'www.linkedin.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: DEFAULT_LIMIT, help: 'Maximum conversations to return (1-100)' },
|
||||
{ name: 'unread-only', type: 'bool', default: false, help: 'Return only conversations with unread messages' },
|
||||
],
|
||||
columns: [
|
||||
'rank',
|
||||
'thread_url',
|
||||
'thread_id',
|
||||
'person_name',
|
||||
'last_message_preview',
|
||||
'unread',
|
||||
'counterparty_type',
|
||||
'category',
|
||||
'timestamp',
|
||||
],
|
||||
func: async (page, kwargs) => {
|
||||
// Validate --limit explicitly rather than silently clamping an out-of-range value.
|
||||
let limit = DEFAULT_LIMIT;
|
||||
if (kwargs.limit !== undefined && kwargs.limit !== null && kwargs.limit !== '') {
|
||||
limit = Number(kwargs.limit);
|
||||
if (!Number.isInteger(limit) || limit < MIN_LIMIT || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between ${MIN_LIMIT} and ${MAX_LIMIT}`);
|
||||
}
|
||||
}
|
||||
const unreadOnly = Boolean(kwargs['unread-only']);
|
||||
|
||||
await page.goto(MESSAGING_URL);
|
||||
await page.wait(10);
|
||||
|
||||
// Locate the messaging API request the page fired on load; retry once if the
|
||||
// SPA was slow to issue it.
|
||||
let located = unwrapEvaluateResult(await page.evaluate(`(${findMessagingApiUrl.toString()})()`));
|
||||
if (located && located.loginRequired) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn requires an active signed-in browser session.');
|
||||
}
|
||||
if (!located || !located.url) {
|
||||
await page.wait(6);
|
||||
located = unwrapEvaluateResult(await page.evaluate(`(${findMessagingApiUrl.toString()})()`));
|
||||
}
|
||||
if (!located || !located.url) {
|
||||
throw new CommandExecutionError(
|
||||
'LinkedIn did not issue a messaging API request; the inbox may have failed to load.',
|
||||
);
|
||||
}
|
||||
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
|
||||
}
|
||||
const csrf = jsession.replace(/^"|"$/g, '');
|
||||
|
||||
// Widen the page size to the requested limit where the query supports it.
|
||||
const targetUrl = located.url.replace(/count:\d+/, 'count:' + limit);
|
||||
const fetched = unwrapEvaluateResult(
|
||||
await page.evaluate(`(${fetchMessagingApi.toString()})(${JSON.stringify(targetUrl)}, ${JSON.stringify(csrf)})`),
|
||||
);
|
||||
if (fetched && fetched.authRequired) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn messaging API authentication failed: ' + fetched.error);
|
||||
}
|
||||
if (!fetched || fetched.error || !fetched.json) {
|
||||
throw new CommandExecutionError(
|
||||
'LinkedIn messaging API returned an unexpected response: ' + ((fetched && fetched.error) || 'no data'),
|
||||
);
|
||||
}
|
||||
|
||||
let conversations = parseConversations(fetched.json, located.mailboxUrn || '');
|
||||
if (unreadOnly) conversations = conversations.filter((c) => c.unread);
|
||||
if (conversations.length === 0) {
|
||||
if (unreadOnly) return [];
|
||||
throw new EmptyResultError('linkedin inbox', 'No LinkedIn conversations were found in the inbox.');
|
||||
}
|
||||
|
||||
return conversations.slice(0, limit).map((c, index) => ({
|
||||
rank: index + 1,
|
||||
thread_url: threadUrl(c.thread_id),
|
||||
thread_id: c.thread_id,
|
||||
person_name: c.person_name,
|
||||
last_message_preview: c.last_message_preview,
|
||||
unread: c.unread,
|
||||
counterparty_type: c.counterparty_type,
|
||||
category: c.category,
|
||||
timestamp: c.timestamp,
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
parseConversations,
|
||||
threadUrl,
|
||||
};
|
||||
@@ -0,0 +1,152 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './inbox.js';
|
||||
|
||||
const { parseConversations, threadUrl } = await import('./inbox.js').then((m) => m.__test__);
|
||||
|
||||
const SELF = 'urn:li:fsd_profile:SELF';
|
||||
|
||||
// Minimal normalized messaging payload mirroring LinkedIn's real response shape:
|
||||
// a flat `included` entity array where conversations reference participants and
|
||||
// messages by URN.
|
||||
function fixture() {
|
||||
return {
|
||||
included: [
|
||||
{
|
||||
$type: 'com.linkedin.messenger.MessagingParticipant',
|
||||
entityUrn: 'urn:li:msg_messagingParticipant:SELF',
|
||||
hostIdentityUrn: SELF,
|
||||
participantType: { member: { firstName: { text: 'Hanzi' }, lastName: { text: 'Li' } } },
|
||||
},
|
||||
{
|
||||
$type: 'com.linkedin.messenger.MessagingParticipant',
|
||||
entityUrn: 'urn:li:msg_messagingParticipant:P1',
|
||||
hostIdentityUrn: 'urn:li:fsd_profile:P1',
|
||||
participantType: { member: { firstName: { text: 'Olga' }, lastName: { text: 'Magere' } } },
|
||||
},
|
||||
{
|
||||
$type: 'com.linkedin.messenger.MessagingParticipant',
|
||||
entityUrn: 'urn:li:msg_messagingParticipant:ORG',
|
||||
hostIdentityUrn: 'urn:li:fsd_company:99',
|
||||
participantType: { organization: { name: { text: 'American Express' } } },
|
||||
},
|
||||
{ $type: 'com.linkedin.messenger.Message', entityUrn: 'urn:li:msg_message:M1', body: { text: 'hey, are you around this week?' } },
|
||||
{ $type: 'com.linkedin.messenger.Message', entityUrn: 'urn:li:msg_message:M2', body: { text: 'Sponsored offer' } },
|
||||
{
|
||||
$type: 'com.linkedin.messenger.Conversation',
|
||||
entityUrn: 'urn:li:msg_conversation:C1',
|
||||
backendUrn: 'urn:li:messagingThread:2-aaa==',
|
||||
unreadCount: 2,
|
||||
read: false,
|
||||
categories: ['INBOX', 'PRIMARY_INBOX'],
|
||||
lastActivityAt: 2000,
|
||||
'*conversationParticipants': ['urn:li:msg_messagingParticipant:P1', 'urn:li:msg_messagingParticipant:SELF'],
|
||||
messages: { '*elements': ['urn:li:msg_message:M1'] },
|
||||
title: null,
|
||||
},
|
||||
{
|
||||
$type: 'com.linkedin.messenger.Conversation',
|
||||
entityUrn: 'urn:li:msg_conversation:C2',
|
||||
backendUrn: 'urn:li:messagingThread:2-bbb==',
|
||||
unreadCount: 0,
|
||||
read: true,
|
||||
categories: ['INBOX', 'PRIMARY_INBOX', 'INMAIL'],
|
||||
lastActivityAt: 3000,
|
||||
'*conversationParticipants': ['urn:li:msg_messagingParticipant:ORG', 'urn:li:msg_messagingParticipant:SELF'],
|
||||
messages: { '*elements': ['urn:li:msg_message:M2'] },
|
||||
title: null,
|
||||
},
|
||||
{
|
||||
$type: 'com.linkedin.messenger.Conversation',
|
||||
entityUrn: 'urn:li:msg_conversation:C3',
|
||||
backendUrn: 'urn:li:messagingThread:2-ccc==',
|
||||
unreadCount: 0,
|
||||
read: true,
|
||||
categories: ['INBOX', 'PRIMARY_INBOX'],
|
||||
lastActivityAt: 1000,
|
||||
'*conversationParticipants': ['urn:li:msg_messagingParticipant:P1', 'urn:li:msg_messagingParticipant:SELF'],
|
||||
messages: { '*elements': [] },
|
||||
title: 'Cohort 2 group',
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin inbox adapter', () => {
|
||||
const command = getRegistry().get('linkedin/inbox');
|
||||
|
||||
it('registers the command with the expected shape', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('linkedin');
|
||||
expect(command.name).toBe('inbox');
|
||||
expect(command.domain).toBe('www.linkedin.com');
|
||||
expect(command.strategy).toBe('cookie');
|
||||
expect(command.browser).toBe(true);
|
||||
expect(typeof command.func).toBe('function');
|
||||
});
|
||||
|
||||
it('exposes channel-safe structured columns', () => {
|
||||
expect(command.columns).toEqual(
|
||||
expect.arrayContaining([
|
||||
'thread_url',
|
||||
'thread_id',
|
||||
'person_name',
|
||||
'last_message_preview',
|
||||
'unread',
|
||||
'counterparty_type',
|
||||
'category',
|
||||
'timestamp',
|
||||
]),
|
||||
);
|
||||
});
|
||||
|
||||
it('builds a thread URL from a thread id', () => {
|
||||
expect(threadUrl('2-aaa==')).toBe('https://www.linkedin.com/messaging/thread/2-aaa==/');
|
||||
expect(threadUrl('')).toBe('');
|
||||
});
|
||||
|
||||
it('parses conversations and sorts them by most recent activity', () => {
|
||||
const rows = parseConversations(fixture(), SELF);
|
||||
expect(rows).toHaveLength(3);
|
||||
expect(rows.map((r) => r.thread_id)).toEqual(['2-bbb==', '2-aaa==', '2-ccc==']);
|
||||
});
|
||||
|
||||
it('resolves the member counterparty, excludes the inbox owner, and reports unread state', () => {
|
||||
const c1 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-aaa==');
|
||||
expect(c1.person_name).toBe('Olga Magere');
|
||||
expect(c1.counterparty_type).toBe('member');
|
||||
expect(c1.unread).toBe(true);
|
||||
expect(c1.last_message_preview).toBe('hey, are you around this week?');
|
||||
});
|
||||
|
||||
it('flags organization counterparties and read conversations', () => {
|
||||
const c2 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-bbb==');
|
||||
expect(c2.person_name).toBe('American Express');
|
||||
expect(c2.counterparty_type).toBe('organization');
|
||||
expect(c2.unread).toBe(false);
|
||||
expect(c2.category).toBe('INBOX,PRIMARY_INBOX,INMAIL');
|
||||
});
|
||||
|
||||
it('uses the group title as the conversation name', () => {
|
||||
const c3 = parseConversations(fixture(), SELF).find((r) => r.thread_id === '2-ccc==');
|
||||
expect(c3.person_name).toBe('Cohort 2 group');
|
||||
});
|
||||
|
||||
it('returns an empty array when a valid payload has no conversations', () => {
|
||||
expect(parseConversations({ included: [] }, SELF)).toEqual([]);
|
||||
});
|
||||
|
||||
it('fails typed when the normalized payload shape is malformed', () => {
|
||||
expect(() => parseConversations({}, SELF)).toThrow(CommandExecutionError);
|
||||
expect(() => parseConversations(null, SELF)).toThrow(CommandExecutionError);
|
||||
const malformed = fixture();
|
||||
malformed.included.push({
|
||||
$type: 'com.linkedin.messenger.Conversation',
|
||||
entityUrn: 'urn:li:msg_conversation:MALFORMED',
|
||||
'*conversationParticipants': [],
|
||||
messages: { '*elements': [] },
|
||||
});
|
||||
expect(() => parseConversations(malformed, SELF)).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,262 @@
|
||||
/**
|
||||
* LinkedIn people-search via SSR DOM text-slice. Voyager people-search
|
||||
* REST returns HTTP 500 from a web context; LinkedIn renders results
|
||||
* server-side now. One navigation per call consumes one CUL query.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
const SEARCH_URL_BASE = 'https://www.linkedin.com/search/results/people/';
|
||||
const MAX_LIMIT = 10;
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function requireStringArg(args, key, label = key) {
|
||||
const value = normalizeWhitespace(args[key]);
|
||||
if (!value) throw new ArgumentError(`${label} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function parseLimit(value) {
|
||||
if (value === undefined || value === null || value === '') return 5;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function buildSearchUrl(keywords) {
|
||||
return SEARCH_URL_BASE + '?keywords=' + encodeURIComponent(keywords);
|
||||
}
|
||||
|
||||
function looksLinkedInAuthWall(value) {
|
||||
const text = normalizeWhitespace(value).toLowerCase();
|
||||
if (!text) return false;
|
||||
return /linkedin\.com\/(?:login|checkpoint|authwall|uas)/i.test(text)
|
||||
|| /\b(sign in|log in|join linkedin|captcha|verification required)\b/i.test(text)
|
||||
|| /(请登录|登录领英|安全验证)/.test(text);
|
||||
}
|
||||
|
||||
function normalizeProfileUrl(value) {
|
||||
const raw = normalizeWhitespace(value);
|
||||
if (!raw) return '';
|
||||
try {
|
||||
const parsed = new URL(raw);
|
||||
const host = parsed.hostname.toLowerCase();
|
||||
if (parsed.protocol !== 'https:' || parsed.username || parsed.password || parsed.port) return '';
|
||||
if (host !== 'linkedin.com' && host !== 'www.linkedin.com') return '';
|
||||
const match = parsed.pathname.match(/^\/in\/([^/?#]+)\/?$/);
|
||||
if (!match || !match[1]) return '';
|
||||
return `https://www.linkedin.com/in/${match[1]}/`;
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function normalizePeopleRows(rows) {
|
||||
if (!Array.isArray(rows)) {
|
||||
throw new CommandExecutionError('LinkedIn people search returned malformed extraction payload: missing rows array');
|
||||
}
|
||||
return rows.map((row, index) => {
|
||||
if (!row || typeof row !== 'object') {
|
||||
throw new CommandExecutionError(`LinkedIn people search returned malformed row at index ${index}`);
|
||||
}
|
||||
const name = normalizeWhitespace(row.name);
|
||||
const profileUrl = normalizeProfileUrl(row.profile_url);
|
||||
if (!name || !profileUrl) {
|
||||
throw new CommandExecutionError(`LinkedIn people search returned row without stable profile identity at index ${index}`);
|
||||
}
|
||||
return {
|
||||
name,
|
||||
headline: normalizeWhitespace(row.headline),
|
||||
location: normalizeWhitespace(row.location),
|
||||
profile_url: profileUrl,
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function parseNonNegativeCount(value, label) {
|
||||
const count = Number(value);
|
||||
if (!Number.isInteger(count) || count < 0) {
|
||||
throw new CommandExecutionError(`LinkedIn people search returned malformed extraction payload: invalid ${label}`);
|
||||
}
|
||||
return count;
|
||||
}
|
||||
|
||||
function extractionScript() {
|
||||
// Class-based selectors are dead (LinkedIn rotates hashed class
|
||||
// names on every deploy) and display:contents flattens the DOM
|
||||
// tree so per-card containers don't exist. Read main.innerText
|
||||
// and slice between consecutive person-name lines instead.
|
||||
return String.raw`(() => {
|
||||
if (!/search\/results\/people/.test(window.location.href)) {
|
||||
return { error: 'not on people search page', url: window.location.href };
|
||||
}
|
||||
const main = document.querySelector('main') || document.body;
|
||||
const normalize = (s) => String(s || '').replace(/[\s\u00a0\u202f]+/g, ' ').trim();
|
||||
const skip = (l) => !l
|
||||
|| /^Status is/.test(l)
|
||||
|| /^(Message|Connect|Follow|View profile|Pending|Remove)$/i.test(l)
|
||||
|| /^[•·]\s*(?:1st|2nd|3rd\+?|degree)/i.test(l)
|
||||
|| /^[•·]/.test(l)
|
||||
|| l.includes('mutual connection')
|
||||
|| l.includes('shared connection')
|
||||
|| /^Summary:/i.test(l)
|
||||
|| /^About this profile/i.test(l);
|
||||
|
||||
const anchors = Array.from(main.querySelectorAll('a[href*="/in/"]'));
|
||||
const personEntries = [];
|
||||
const seenHandles = new Set();
|
||||
for (const a of anchors) {
|
||||
const m = (a.getAttribute('href') || '').match(/\/in\/([^/?#]+)/);
|
||||
if (!m || !m[1]) continue;
|
||||
const profileHandle = m[1];
|
||||
if (seenHandles.has(profileHandle)) continue;
|
||||
const aria = a.querySelector('span[aria-hidden="true"]');
|
||||
let name = normalize(aria ? aria.textContent : a.textContent);
|
||||
name = name.replace(/^Status is (online|offline)\.?\s*/i, '')
|
||||
.replace(/'?s profile$/i, '')
|
||||
.replace(/\s*[•·].*$/, '').trim();
|
||||
if (!name) continue;
|
||||
seenHandles.add(profileHandle);
|
||||
personEntries.push({ profileHandle, displayName: name });
|
||||
}
|
||||
|
||||
const lines = (main.innerText || '').split(/\n+/).map(normalize).filter(Boolean);
|
||||
|
||||
// skip() rejects mutual-connection lines, so candidates that only
|
||||
// appear as mutual-connection links inside another card's row
|
||||
// never resolve a name index and get filtered out below.
|
||||
const nameToIndex = new Map();
|
||||
for (const { displayName } of personEntries) {
|
||||
if (nameToIndex.has(displayName)) continue;
|
||||
const match = lines.findIndex((l) =>
|
||||
!skip(l) && (
|
||||
l === displayName
|
||||
|| l.startsWith(displayName + ' ')
|
||||
|| l.startsWith(displayName + ',')
|
||||
|| l.startsWith(displayName + "'")
|
||||
)
|
||||
);
|
||||
if (match >= 0) nameToIndex.set(displayName, match);
|
||||
}
|
||||
|
||||
const resolved = personEntries.filter((p) => nameToIndex.has(p.displayName));
|
||||
const rows = [];
|
||||
for (let i = 0; i < resolved.length; i++) {
|
||||
const { profileHandle, displayName } = resolved[i];
|
||||
const startIdx = nameToIndex.get(displayName);
|
||||
let stopIdx = lines.length;
|
||||
for (let j = i + 1; j < resolved.length; j++) {
|
||||
const otherStart = nameToIndex.get(resolved[j].displayName);
|
||||
if (otherStart != null && otherStart > startIdx) {
|
||||
stopIdx = otherStart;
|
||||
break;
|
||||
}
|
||||
}
|
||||
const slice = lines.slice(startIdx + 1, stopIdx).filter((l) => l !== displayName && !skip(l));
|
||||
rows.push({
|
||||
name: displayName,
|
||||
headline: slice[0] || '',
|
||||
location: slice[1] || '',
|
||||
profile_url: 'https://www.linkedin.com/in/' + profileHandle + '/',
|
||||
});
|
||||
}
|
||||
return {
|
||||
rows,
|
||||
candidate_count: personEntries.length,
|
||||
person_entries_count: personEntries.length,
|
||||
resolved_count: resolved.length,
|
||||
};
|
||||
})()`;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'people-search',
|
||||
access: 'read',
|
||||
description: 'Search standard LinkedIn (not Sales Navigator) for people by keyword. Each invocation consumes against LinkedIn\'s monthly Commercial Use Limit on people search; throttle accordingly.',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.COOKIE,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'People search keywords, e.g. "site reliability engineer berlin"' },
|
||||
{ name: 'limit', type: 'int', default: 5, help: `Maximum people to return (1-${MAX_LIMIT}); each query counts toward LinkedIn's monthly CUL` },
|
||||
],
|
||||
columns: ['rank', 'name', 'headline', 'location', 'profile_url'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin people-search');
|
||||
const keywords = requireStringArg(args, 'keywords', '--keywords');
|
||||
const limit = parseLimit(args.limit);
|
||||
|
||||
try {
|
||||
await page.goto(buildSearchUrl(keywords));
|
||||
await page.wait(6);
|
||||
} catch (error) {
|
||||
throw new CommandExecutionError(`LinkedIn people search navigation failed: ${error?.message || error}`);
|
||||
}
|
||||
|
||||
let cookies;
|
||||
try {
|
||||
cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
} catch (error) {
|
||||
throw new CommandExecutionError(`LinkedIn cookie lookup failed: ${error?.message || error}`);
|
||||
}
|
||||
if (!Array.isArray(cookies)) {
|
||||
throw new CommandExecutionError('LinkedIn cookie lookup returned malformed payload');
|
||||
}
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn in the browser.');
|
||||
}
|
||||
|
||||
let result;
|
||||
try {
|
||||
result = unwrapEvaluateResult(await page.evaluate(extractionScript()));
|
||||
} catch (error) {
|
||||
throw new CommandExecutionError(`LinkedIn people search extraction failed: ${error?.message || error}`);
|
||||
}
|
||||
if (result?.error) {
|
||||
if (looksLinkedInAuthWall(`${result.url || ''} ${result.error || ''}`)) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn people search requires an active signed-in browser session.');
|
||||
}
|
||||
// If LinkedIn redirected away from the search page that
|
||||
// usually means CUL was reached or the account is gated.
|
||||
throw new CommandExecutionError(`LinkedIn redirected away from the search page (${result.error}). Likely Commercial Use Limit reached - the limit resets on the 1st of next month.`);
|
||||
}
|
||||
if (!result || typeof result !== 'object') {
|
||||
throw new CommandExecutionError('LinkedIn people search returned malformed extraction payload');
|
||||
}
|
||||
const candidateCount = parseNonNegativeCount(result.candidate_count, 'candidate_count');
|
||||
parseNonNegativeCount(result.person_entries_count, 'person_entries_count');
|
||||
const resolvedCount = parseNonNegativeCount(result.resolved_count, 'resolved_count');
|
||||
const rows = normalizePeopleRows(result.rows);
|
||||
if (rows.length === 0 && (candidateCount > 0 || resolvedCount > 0)) {
|
||||
throw new CommandExecutionError('LinkedIn people search found profile candidates but could not parse stable result rows');
|
||||
}
|
||||
if (rows.length === 0) {
|
||||
throw new EmptyResultError(`No people found on the rendered page for "${keywords}". The search may have returned zero results, or the DOM markup may have changed.`);
|
||||
}
|
||||
return rows.slice(0, limit).map((p, i) => ({ rank: i + 1, ...p }));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
parseLimit,
|
||||
buildSearchUrl,
|
||||
looksLinkedInAuthWall,
|
||||
normalizeProfileUrl,
|
||||
normalizePeopleRows,
|
||||
parseNonNegativeCount,
|
||||
extractionScript,
|
||||
};
|
||||
@@ -0,0 +1,216 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './people-search.js';
|
||||
|
||||
const {
|
||||
parseLimit,
|
||||
buildSearchUrl,
|
||||
looksLinkedInAuthWall,
|
||||
normalizeProfileUrl,
|
||||
normalizePeopleRows,
|
||||
parseNonNegativeCount,
|
||||
extractionScript,
|
||||
} = await import('./people-search.js').then((m) => m.__test__);
|
||||
|
||||
function extractionResult(rows, counts = {}) {
|
||||
return {
|
||||
rows,
|
||||
candidate_count: counts.candidate_count ?? rows.length,
|
||||
person_entries_count: counts.person_entries_count ?? counts.candidate_count ?? rows.length,
|
||||
resolved_count: counts.resolved_count ?? rows.length,
|
||||
};
|
||||
}
|
||||
|
||||
function makePage({
|
||||
evaluateResult,
|
||||
evaluateReject,
|
||||
gotoReject,
|
||||
cookies = [{ name: 'JSESSIONID', value: '"ajax:1234567890"' }],
|
||||
} = {}) {
|
||||
return {
|
||||
goto: vi.fn().mockImplementation(() => gotoReject ? Promise.reject(gotoReject) : Promise.resolve(undefined)),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
getCookies: vi.fn().mockResolvedValue(cookies),
|
||||
evaluate: vi.fn().mockImplementation(() => evaluateReject ? Promise.reject(evaluateReject) : Promise.resolve(evaluateResult)),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin people-search command', () => {
|
||||
it('builds the canonical SSR search URL with encoded keywords', () => {
|
||||
expect(buildSearchUrl('site reliability engineer'))
|
||||
.toBe('https://www.linkedin.com/search/results/people/?keywords=site%20reliability%20engineer');
|
||||
expect(buildSearchUrl('hello/world & stuff'))
|
||||
.toBe('https://www.linkedin.com/search/results/people/?keywords=hello%2Fworld%20%26%20stuff');
|
||||
});
|
||||
|
||||
it('validates --limit without silent clamping', () => {
|
||||
expect(parseLimit(undefined)).toBe(5);
|
||||
expect(parseLimit(1)).toBe(1);
|
||||
expect(parseLimit(10)).toBe(10);
|
||||
expect(() => parseLimit(0)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(11)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(-1)).toThrow(ArgumentError);
|
||||
expect(() => parseLimit('abc')).toThrow(ArgumentError);
|
||||
expect(() => parseLimit(1.5)).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('extraction script slices main.innerText by person-name boundaries', () => {
|
||||
const s = extractionScript();
|
||||
// Anchor enumeration finds /in/<handle>.
|
||||
expect(s).toContain('a[href*="/in/"]');
|
||||
expect(s).toContain('\\/in\\/([^/?#]+)');
|
||||
// Text-slice approach: split main.innerText and locate names.
|
||||
expect(s).toContain('main.innerText');
|
||||
expect(s).toContain('lines.findIndex');
|
||||
// Mutual-connection anchors are filtered out via the skip()
|
||||
// predicate on the name-line match.
|
||||
expect(s).toContain('mutual connection');
|
||||
// Names dedup'd by handle.
|
||||
expect(s).toContain('seenHandles');
|
||||
// Aria-hidden span as canonical name source.
|
||||
expect(s).toContain('span[aria-hidden="true"]');
|
||||
// Only operates on the people-search page.
|
||||
expect(s).toContain('search\\/results\\/people');
|
||||
expect(s).toContain('candidate_count');
|
||||
expect(s).toContain('resolved_count');
|
||||
});
|
||||
|
||||
it('normalizes only stable LinkedIn profile identities', () => {
|
||||
expect(normalizeProfileUrl('https://www.linkedin.com/in/alice-engineer/?mini=true'))
|
||||
.toBe('https://www.linkedin.com/in/alice-engineer/');
|
||||
expect(normalizeProfileUrl('https://linkedin.com/in/bob-builder')).toBe('https://www.linkedin.com/in/bob-builder/');
|
||||
expect(normalizeProfileUrl('https://evil-linkedin.com/in/bob-builder')).toBe('');
|
||||
expect(normalizeProfileUrl('http://www.linkedin.com/in/bob-builder')).toBe('');
|
||||
expect(normalizeProfileUrl('https://www.linkedin.com/company/opencli')).toBe('');
|
||||
});
|
||||
|
||||
it('detects LinkedIn auth-wall URLs separately from CUL redirects', () => {
|
||||
expect(looksLinkedInAuthWall('https://www.linkedin.com/authwall Sign in to continue')).toBe(true);
|
||||
expect(looksLinkedInAuthWall('https://www.linkedin.com/checkpoint/challenge security verification required')).toBe(true);
|
||||
expect(looksLinkedInAuthWall('https://www.linkedin.com/feed/')).toBe(false);
|
||||
});
|
||||
|
||||
it('rejects malformed extraction rows instead of fabricating success rows', () => {
|
||||
expect(() => normalizePeopleRows({})).toThrow(CommandExecutionError);
|
||||
expect(() => normalizePeopleRows([null])).toThrow(CommandExecutionError);
|
||||
expect(() => normalizePeopleRows([{ name: 'No URL', headline: 'h', location: 'l', profile_url: '' }]))
|
||||
.toThrow(CommandExecutionError);
|
||||
expect(() => normalizePeopleRows([{ name: '', headline: 'h', location: 'l', profile_url: 'https://www.linkedin.com/in/no-name/' }]))
|
||||
.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('validates extraction evidence counters', () => {
|
||||
expect(parseNonNegativeCount(0, 'candidate_count')).toBe(0);
|
||||
expect(parseNonNegativeCount(2, 'candidate_count')).toBe(2);
|
||||
expect(() => parseNonNegativeCount(undefined, 'candidate_count')).toThrow(CommandExecutionError);
|
||||
expect(() => parseNonNegativeCount(-1, 'candidate_count')).toThrow(CommandExecutionError);
|
||||
expect(() => parseNonNegativeCount(1.2, 'candidate_count')).toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('returns ranked rows when the page yields people', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
expect(cmd?.func).toBeTypeOf('function');
|
||||
const page = makePage({
|
||||
evaluateResult: extractionResult([
|
||||
{ name: 'Alice Engineer', headline: 'Staff SWE at Acme', location: 'Berlin', profile_url: 'https://www.linkedin.com/in/alice-engineer/' },
|
||||
{ name: 'Bob Builder', headline: 'CTO at Globex', location: 'Remote', profile_url: 'https://www.linkedin.com/in/bob-builder/' },
|
||||
]),
|
||||
});
|
||||
const result = await cmd.func(page, { keywords: 'reinforcement learning', limit: 5 });
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.linkedin.com/search/results/people/?keywords=reinforcement%20learning');
|
||||
expect(result).toEqual([
|
||||
{ rank: 1, name: 'Alice Engineer', headline: 'Staff SWE at Acme', location: 'Berlin', profile_url: 'https://www.linkedin.com/in/alice-engineer/' },
|
||||
{ rank: 2, name: 'Bob Builder', headline: 'CTO at Globex', location: 'Remote', profile_url: 'https://www.linkedin.com/in/bob-builder/' },
|
||||
]);
|
||||
});
|
||||
|
||||
it('slices to --limit when more rows are extracted than requested', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({
|
||||
evaluateResult: extractionResult(Array.from({ length: 8 }, (_, i) => ({
|
||||
name: `Person ${i}`, headline: 'h', location: 'l', profile_url: `https://www.linkedin.com/in/p${i}/`,
|
||||
}))),
|
||||
});
|
||||
const result = await cmd.func(page, { keywords: 'x', limit: 3 });
|
||||
expect(result).toHaveLength(3);
|
||||
expect(result.map((r) => r.rank)).toEqual([1, 2, 3]);
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when JSESSIONID cookie is missing', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ cookies: [], evaluateResult: extractionResult([]) });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('treats malformed cookie lookup results as CommandExecutionError', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ cookies: null, evaluateResult: extractionResult([]) });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('treats LinkedIn redirect away from search page as a CUL-flavoured CommandExecutionError', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateResult: { error: 'not on people search page', url: 'https://www.linkedin.com/' } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('treats LinkedIn auth-wall redirects as AuthRequiredError', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateResult: { error: 'not on people search page', url: 'https://www.linkedin.com/authwall?trk=people_search' } });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('wraps browser extraction exceptions as CommandExecutionError', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateReject: new SyntaxError('Unexpected token <') });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('wraps browser navigation exceptions as CommandExecutionError', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ gotoReject: new Error('navigation failed') });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when the page rendered zero rows', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateResult: extractionResult([]) });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
|
||||
it('treats profile candidates without stable parsed rows as parser drift', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({
|
||||
evaluateResult: extractionResult([], {
|
||||
candidate_count: 1,
|
||||
person_entries_count: 1,
|
||||
resolved_count: 0,
|
||||
}),
|
||||
});
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('treats missing rows array as parser drift, not empty results', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateResult: {} });
|
||||
await expect(cmd.func(page, { keywords: 'x', limit: 5 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('rejects empty keywords with ArgumentError before navigation', async () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
const page = makePage({ evaluateResult: extractionResult([]) });
|
||||
await expect(cmd.func(page, { keywords: ' ', limit: 5 })).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('registers with the expected columns and arg shape', () => {
|
||||
const cmd = getRegistry().get('linkedin/people-search');
|
||||
expect(cmd?.columns).toEqual(['rank', 'name', 'headline', 'location', 'profile_url']);
|
||||
expect(cmd?.access).toBe('read');
|
||||
expect(cmd?.browser).toBe(true);
|
||||
const keywordsArg = cmd?.args?.find((a) => a.name === 'keywords');
|
||||
expect(keywordsArg?.positional).toBe(true);
|
||||
expect(keywordsArg?.required).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,357 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { createHash } from 'node:crypto';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function normalizeName(value) {
|
||||
return normalizeWhitespace(value)
|
||||
.replace(/\s*[•·]\s*(?:1st|2nd|3rd\+?|degree connection).*$/i, '')
|
||||
.replace(/\s+LinkedIn.*$/i, '')
|
||||
.toLowerCase();
|
||||
}
|
||||
|
||||
function isLinkedInHost(hostname) {
|
||||
const host = String(hostname || '').toLowerCase();
|
||||
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
|
||||
}
|
||||
|
||||
function canonicalizeLinkedInThreadUrl(value) {
|
||||
const raw = normalizeWhitespace(value);
|
||||
if (!raw) return '';
|
||||
try {
|
||||
const url = new URL(raw);
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return '';
|
||||
const match = url.pathname.match(/^\/messaging\/thread\/([^/]+)\/?$/i);
|
||||
if (!match || !match[1]) return '';
|
||||
url.hostname = 'www.linkedin.com';
|
||||
url.hash = '';
|
||||
url.search = '';
|
||||
if (!url.pathname.endsWith('/')) url.pathname += '/';
|
||||
return url.toString();
|
||||
} catch {
|
||||
return '';
|
||||
}
|
||||
}
|
||||
|
||||
function hashText(value) {
|
||||
return createHash('sha256').update(normalizeWhitespace(value)).digest('hex');
|
||||
}
|
||||
|
||||
function textContainsNormalized(haystack, needle) {
|
||||
const h = normalizeWhitespace(haystack).toLowerCase();
|
||||
const n = normalizeWhitespace(needle).toLowerCase();
|
||||
return !n || h.includes(n);
|
||||
}
|
||||
|
||||
function selectBestHeaderName(headerNames, expectedName) {
|
||||
const expected = normalizeName(expectedName);
|
||||
const names = (Array.isArray(headerNames) ? headerNames : [])
|
||||
.map(normalizeWhitespace)
|
||||
.filter(Boolean);
|
||||
return names.find((name) => normalizeName(name) === expected) || names[0] || '';
|
||||
}
|
||||
|
||||
function assessThreadSafety(probe, expected) {
|
||||
const expectedName = normalizeWhitespace(expected.expectedName);
|
||||
const actualName = selectBestHeaderName(probe?.headerNames, expectedName);
|
||||
const expectedThreadUrl = canonicalizeLinkedInThreadUrl(expected.threadUrl);
|
||||
const actualThreadUrl = canonicalizeLinkedInThreadUrl(probe?.url || '');
|
||||
const bodyText = String(probe?.bodyText || '');
|
||||
|
||||
if (probe?.authRequired) {
|
||||
return { ok: false, blockReason: 'auth_required', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
if (probe?.searchFailure || /we didn't find anything|no results found|no results for/i.test(bodyText)) {
|
||||
return { ok: false, blockReason: 'search_failure_visible', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
if (expectedThreadUrl && actualThreadUrl && expectedThreadUrl !== actualThreadUrl) {
|
||||
return { ok: false, blockReason: 'thread_url_mismatch', expectedValue: expectedThreadUrl, actualValue: actualThreadUrl, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
if (!actualName || normalizeName(actualName) !== normalizeName(expectedName)) {
|
||||
return { ok: false, blockReason: 'recipient_header_mismatch', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
if (!probe?.composerFound) {
|
||||
return { ok: false, blockReason: 'composer_not_found', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
const expectedLastHash = normalizeWhitespace(expected.expectedLastHash);
|
||||
if (expectedLastHash && expectedLastHash !== probe?.latestMessageHash) {
|
||||
return { ok: false, blockReason: 'latest_message_mismatch', expectedValue: expectedLastHash, actualValue: probe?.latestMessageHash || '', observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
const expectedLastText = normalizeWhitespace(expected.expectedLastText);
|
||||
if (expectedLastText && !textContainsNormalized(bodyText, expectedLastText)) {
|
||||
return { ok: false, blockReason: 'latest_message_mismatch', expectedValue: expectedLastText, actualValue: '', observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
return { ok: true, blockReason: 'verified', expectedValue: expectedName, actualValue: actualName, observedUrl: actualThreadUrl };
|
||||
}
|
||||
|
||||
function requireStringArg(args, key, label = key) {
|
||||
const value = normalizeWhitespace(args[key]);
|
||||
if (!value) throw new ArgumentError(`${label} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function requireLinkedInThreadUrl(value, label) {
|
||||
const url = canonicalizeLinkedInThreadUrl(value);
|
||||
if (!url) throw new ArgumentError(`${label} must be an exact https://www.linkedin.com/messaging/thread/<id>/ URL`);
|
||||
return url;
|
||||
}
|
||||
|
||||
function buildThreadProbeScript() {
|
||||
return String.raw`(() => {
|
||||
const marker = '__OPENCLI_LINKEDIN_PROBE__';
|
||||
void marker;
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const text = document.body ? (document.body.innerText || '') : '';
|
||||
const lower = text.toLowerCase();
|
||||
const authRequired = /\b(sign in|log in|join linkedin)\b/i.test(text)
|
||||
|| /linkedin\.com\/(login|checkpoint|authwall)/i.test(location.href)
|
||||
|| /captcha|verification required/i.test(text);
|
||||
const searchFailure = /we didn't find anything|no results found|no results for/i.test(text);
|
||||
|
||||
const headerCandidates = [];
|
||||
const selectors = [
|
||||
'.msg-thread__link-to-profile',
|
||||
'.msg-thread__link-to-profile span[aria-hidden="true"]',
|
||||
'.msg-entity-lockup__entity-title',
|
||||
'.msg-conversation-card__participant-names',
|
||||
'main h1',
|
||||
'main h2',
|
||||
'[data-anonymize="person-name"]',
|
||||
'a[href*="/in/"] span[aria-hidden="true"]',
|
||||
'a[href*="/in/"]'
|
||||
];
|
||||
for (const selector of selectors) {
|
||||
for (const el of Array.from(document.querySelectorAll(selector)).slice(0, 8)) {
|
||||
const value = clean(el.innerText || el.textContent || el.getAttribute('aria-label'));
|
||||
if (value && value.length <= 120 && !/^(message|messaging|send|profile|view profile)$/i.test(value)) {
|
||||
headerCandidates.push(value);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
|
||||
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
|
||||
|
||||
const messageText = Array.from(document.querySelectorAll('.msg-s-message-list__event, .msg-s-event-listitem, [data-event-urn], .msg-s-message-group__meta, .msg-s-message-list-content'))
|
||||
.map((el) => clean(el.innerText || el.textContent))
|
||||
.filter(Boolean)
|
||||
.join('\n');
|
||||
const sourceText = messageText || text;
|
||||
const sourceLines = sourceText.split(/\n+/).map(clean).filter(Boolean);
|
||||
const lastMeaningfulLine = [...sourceLines].reverse().find((line) => !/^(send|reply|write a message|press enter to send)$/i.test(line)) || '';
|
||||
|
||||
return {
|
||||
url: location.href,
|
||||
title: document.title || '',
|
||||
headerNames: Array.from(new Set(headerCandidates)).slice(0, 10),
|
||||
bodyText: text,
|
||||
composerFound: Boolean(composer),
|
||||
composerText: composer ? clean(composer.innerText || composer.textContent) : '',
|
||||
authRequired,
|
||||
searchFailure,
|
||||
latestMessageText: lastMeaningfulLine,
|
||||
latestMessageHash: '',
|
||||
};
|
||||
})()`;
|
||||
}
|
||||
|
||||
function buildFocusComposerScript() {
|
||||
return String.raw`(() => {
|
||||
const marker = '__OPENCLI_LINKEDIN_FOCUS_COMPOSER__';
|
||||
void marker;
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
|
||||
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
|
||||
if (!composer) return { ok: false, error: 'composer_not_found', composerText: '' };
|
||||
composer.focus();
|
||||
composer.innerHTML = '';
|
||||
composer.dispatchEvent(new InputEvent('input', { bubbles: true, inputType: 'deleteContentBackward', data: null }));
|
||||
return { ok: true, composerText: clean(composer.innerText || composer.textContent) };
|
||||
})()`;
|
||||
}
|
||||
|
||||
function buildReadComposerScript() {
|
||||
return String.raw`(() => {
|
||||
const marker = '__OPENCLI_LINKEDIN_READ_COMPOSER__';
|
||||
void marker;
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const composer = Array.from(document.querySelectorAll('[contenteditable="true"][role="textbox"], div.msg-form__contenteditable[contenteditable="true"], [aria-label*="Write a message" i]'))
|
||||
.find((el) => !el.closest('[aria-hidden="true"]') && el.offsetParent !== null);
|
||||
return { ok: Boolean(composer), composerText: composer ? clean(composer.innerText || composer.textContent) : '' };
|
||||
})()`;
|
||||
}
|
||||
|
||||
function buildClickSendScript() {
|
||||
return String.raw`(() => {
|
||||
const marker = '__OPENCLI_LINKEDIN_CLICK_SEND__';
|
||||
void marker;
|
||||
const buttons = Array.from(document.querySelectorAll('button'));
|
||||
const send = buttons.find((button) => {
|
||||
const text = (button.innerText || button.textContent || button.getAttribute('aria-label') || '').trim().toLowerCase();
|
||||
return text === 'send' || text === 'send message';
|
||||
});
|
||||
if (!send) return { ok: false, error: 'send_button_not_found', sent: false };
|
||||
if (send.disabled || send.getAttribute('aria-disabled') === 'true') return { ok: false, error: 'send_button_disabled', sent: false };
|
||||
send.click();
|
||||
return { ok: true, sent: true };
|
||||
})()`;
|
||||
}
|
||||
|
||||
async function probeThread(page) {
|
||||
const result = unwrapEvaluateResult(await page.evaluate(buildThreadProbeScript()));
|
||||
const latestText = normalizeWhitespace(result?.latestMessageText || '');
|
||||
return {
|
||||
...(result || {}),
|
||||
latestMessageText: latestText,
|
||||
latestMessageHash: latestText ? hashText(latestText) : '',
|
||||
};
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'safe-send',
|
||||
access: 'write',
|
||||
description: 'Fail-closed LinkedIn message sender that verifies exact thread, recipient, and latest message before filling/sending',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.UI,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'thread-url', required: true, help: 'Exact LinkedIn messaging thread URL to open and verify' },
|
||||
{ name: 'expected-name', required: true, help: 'Expected visible recipient name in the active thread header' },
|
||||
{ name: 'message', required: true, help: 'Message body to send or dry-run' },
|
||||
{ name: 'expected-last-text', help: 'Substring expected in the currently visible latest conversation context' },
|
||||
{ name: 'expected-last-hash', help: 'SHA-256 hash of expected latest visible message text' },
|
||||
{ name: 'send', type: 'bool', default: false, help: 'Actually click Send. Default is dry-run verification only.' },
|
||||
{ name: 'screenshot', type: 'bool', default: false, help: 'Capture a screenshot during verification' },
|
||||
],
|
||||
columns: ['status', 'recipient', 'reason', 'thread_url', 'message_chars', 'screenshot'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin safe-send');
|
||||
|
||||
const threadUrl = requireLinkedInThreadUrl(requireStringArg(args, 'thread-url', '--thread-url'), '--thread-url');
|
||||
const expectedName = requireStringArg(args, 'expected-name', '--expected-name');
|
||||
const message = requireStringArg(args, 'message', '--message');
|
||||
|
||||
await page.goto('https://www.linkedin.com/messaging/');
|
||||
await page.wait(4);
|
||||
await page.goto(threadUrl);
|
||||
// LinkedIn messaging often renders the shell first and hydrates the active
|
||||
// thread header/messages a few seconds later. Wait long enough for the
|
||||
// recipient header to appear so we fail closed on a real mismatch, not on
|
||||
// a premature blank DOM snapshot.
|
||||
await page.wait(12);
|
||||
|
||||
let beforeProbe = await probeThread(page);
|
||||
const expectedLastText = normalizeWhitespace(args['expected-last-text']);
|
||||
for (let attempt = 0; expectedLastText && attempt < 6 && !textContainsNormalized(beforeProbe.bodyText, expectedLastText); attempt += 1) {
|
||||
await page.wait(2);
|
||||
beforeProbe = await probeThread(page);
|
||||
}
|
||||
|
||||
const safety = assessThreadSafety(beforeProbe, {
|
||||
expectedName,
|
||||
threadUrl,
|
||||
expectedLastText: args['expected-last-text'],
|
||||
expectedLastHash: args['expected-last-hash'],
|
||||
});
|
||||
|
||||
if (safety.blockReason === 'auth_required') {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn safe-send requires an active signed-in LinkedIn browser session.');
|
||||
}
|
||||
|
||||
if (!safety.ok) {
|
||||
const observed = [
|
||||
`Expected ${safety.expectedValue}; actual ${safety.actualValue || 'not_visible'} at ${safety.observedUrl || 'url_not_available'}`,
|
||||
`Observed headers: ${(beforeProbe.headerNames || []).join(' | ') || 'no_visible_headers'}`,
|
||||
`Title: ${beforeProbe.title || 'title_not_available'}`,
|
||||
`Body: ${normalizeWhitespace(beforeProbe.bodyText || '').slice(0, 500)}`,
|
||||
].join('\n');
|
||||
throw new CommandExecutionError(
|
||||
`LinkedIn safe-send blocked: ${safety.blockReason}`,
|
||||
observed,
|
||||
);
|
||||
}
|
||||
|
||||
let screenshot = '';
|
||||
if (args.screenshot && typeof page.screenshot === 'function') {
|
||||
screenshot = await page.screenshot({ fullPage: false });
|
||||
}
|
||||
|
||||
if (!args.send) {
|
||||
return [{
|
||||
status: 'verified_dry_run',
|
||||
recipient: safety.actualValue,
|
||||
reason: safety.blockReason,
|
||||
thread_url: safety.observedUrl,
|
||||
message_chars: message.length,
|
||||
screenshot: screenshot ? 'captured' : '',
|
||||
}];
|
||||
}
|
||||
|
||||
const focus = unwrapEvaluateResult(await page.evaluate(buildFocusComposerScript()));
|
||||
if (!focus?.ok) throw new CommandExecutionError(`LinkedIn safe-send blocked: ${focus?.error || 'composer_focus_failed'}`);
|
||||
|
||||
await page.insertText(message);
|
||||
await page.wait(0.6 + Math.random() * 0.8);
|
||||
|
||||
const composer = unwrapEvaluateResult(await page.evaluate(buildReadComposerScript()));
|
||||
if (!composer?.ok || normalizeWhitespace(composer.composerText) !== normalizeWhitespace(message)) {
|
||||
throw new CommandExecutionError(
|
||||
'LinkedIn safe-send blocked: composer_text_mismatch',
|
||||
`Composer text did not exactly match intended message for ${expectedName}.`,
|
||||
);
|
||||
}
|
||||
|
||||
const afterFillProbe = await probeThread(page);
|
||||
const afterFillSafety = assessThreadSafety(afterFillProbe, {
|
||||
expectedName,
|
||||
threadUrl,
|
||||
expectedLastText: args['expected-last-text'],
|
||||
expectedLastHash: args['expected-last-hash'],
|
||||
});
|
||||
if (!afterFillSafety.ok) {
|
||||
throw new CommandExecutionError(`LinkedIn safe-send blocked after fill: ${afterFillSafety.blockReason}`);
|
||||
}
|
||||
|
||||
const sent = unwrapEvaluateResult(await page.evaluate(buildClickSendScript()));
|
||||
if (!sent?.ok || !sent.sent) {
|
||||
throw new CommandExecutionError(`LinkedIn safe-send blocked: ${sent?.error || 'send_click_failed'}`);
|
||||
}
|
||||
|
||||
await page.wait(0.8 + Math.random() * 1.2);
|
||||
return [{
|
||||
status: 'sent',
|
||||
recipient: safety.actualValue,
|
||||
reason: safety.blockReason,
|
||||
thread_url: safety.observedUrl,
|
||||
message_chars: message.length,
|
||||
screenshot: screenshot ? 'captured' : '',
|
||||
}];
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
unwrapEvaluateResult,
|
||||
normalizeName,
|
||||
canonicalizeLinkedInThreadUrl,
|
||||
hashText,
|
||||
assessThreadSafety,
|
||||
};
|
||||
@@ -0,0 +1,204 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './safe-send.js';
|
||||
|
||||
const {
|
||||
normalizeWhitespace,
|
||||
normalizeName,
|
||||
canonicalizeLinkedInThreadUrl,
|
||||
hashText,
|
||||
assessThreadSafety,
|
||||
} = await import('./safe-send.js').then((m) => m.__test__);
|
||||
|
||||
function makeFakePage(probe) {
|
||||
let composerText = probe.composerText || '';
|
||||
return {
|
||||
goto: vi.fn(async () => undefined),
|
||||
wait: vi.fn(async () => undefined),
|
||||
evaluate: vi.fn(async (script) => {
|
||||
const text = String(script);
|
||||
if (text.includes('__OPENCLI_LINKEDIN_PROBE__')) return probe;
|
||||
if (text.includes('__OPENCLI_LINKEDIN_FOCUS_COMPOSER__')) return { ok: true, composerText: '' };
|
||||
if (text.includes('__OPENCLI_LINKEDIN_READ_COMPOSER__')) return { ok: true, composerText };
|
||||
if (text.includes('__OPENCLI_LINKEDIN_CLICK_SEND__')) return { ok: true, sent: true };
|
||||
return undefined;
|
||||
}),
|
||||
insertText: vi.fn(async (text) => {
|
||||
composerText = text;
|
||||
}),
|
||||
pressKey: vi.fn(async () => undefined),
|
||||
screenshot: vi.fn(async () => 'base64-screenshot'),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin safe-send helpers', () => {
|
||||
it('normalizes whitespace and LinkedIn names for exact-ish comparisons', () => {
|
||||
expect(normalizeWhitespace(' Lokesh\n\tRamesh ')).toBe('Lokesh Ramesh');
|
||||
expect(normalizeName('Lokesh Ramesh • 1st')).toBe('lokesh ramesh');
|
||||
});
|
||||
|
||||
it('canonicalizes thread URLs while dropping query and hash noise', () => {
|
||||
expect(canonicalizeLinkedInThreadUrl('https://www.linkedin.com/messaging/thread/abc/?foo=1#bar'))
|
||||
.toBe('https://www.linkedin.com/messaging/thread/abc/');
|
||||
expect(canonicalizeLinkedInThreadUrl('https://www.linkedin.com/messaging/thread/abc/extra')).toBe('');
|
||||
expect(canonicalizeLinkedInThreadUrl('https://evil-linkedin.com/messaging/thread/abc/')).toBe('');
|
||||
expect(canonicalizeLinkedInThreadUrl('http://www.linkedin.com/messaging/thread/abc/')).toBe('');
|
||||
});
|
||||
|
||||
it('fails closed when LinkedIn search produced no results even if a composer is visible', () => {
|
||||
const result = assessThreadSafety({
|
||||
url: 'https://www.linkedin.com/messaging/thread/bora/',
|
||||
headerNames: ['Bora Nicholson'],
|
||||
bodyText: "We didn't find anything for Victoria Munoz\nBora Nicholson",
|
||||
searchFailure: true,
|
||||
composerFound: true,
|
||||
latestMessageHash: hashText('hello'),
|
||||
}, {
|
||||
expectedName: 'Victoria Munoz',
|
||||
threadUrl: 'https://www.linkedin.com/messaging/thread/victoria/',
|
||||
expectedLastText: 'hello',
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.blockReason).toBe('search_failure_visible');
|
||||
});
|
||||
|
||||
it('fails closed on recipient header mismatch', () => {
|
||||
const result = assessThreadSafety({
|
||||
url: 'https://www.linkedin.com/messaging/thread/bora/',
|
||||
headerNames: ['Bora Nicholson'],
|
||||
bodyText: 'Bora Nicholson\nhello',
|
||||
composerFound: true,
|
||||
latestMessageHash: hashText('hello'),
|
||||
}, {
|
||||
expectedName: 'Victoria Munoz',
|
||||
expectedLastText: 'hello',
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.blockReason).toBe('recipient_header_mismatch');
|
||||
expect(result.actualValue).toBe('Bora Nicholson');
|
||||
});
|
||||
|
||||
it('fails closed when the stored latest message is no longer visible', () => {
|
||||
const result = assessThreadSafety({
|
||||
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
headerNames: ['Lokesh Ramesh'],
|
||||
bodyText: 'Lokesh Ramesh\na newer inbound arrived',
|
||||
composerFound: true,
|
||||
latestMessageHash: hashText('a newer inbound arrived'),
|
||||
}, {
|
||||
expectedName: 'Lokesh Ramesh',
|
||||
expectedLastText: 'old inbound text',
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(false);
|
||||
expect(result.blockReason).toBe('latest_message_mismatch');
|
||||
});
|
||||
|
||||
it('passes only when recipient, thread, latest text, and composer are all verified', () => {
|
||||
const result = assessThreadSafety({
|
||||
url: 'https://www.linkedin.com/messaging/thread/lokesh/?mini=true',
|
||||
headerNames: ['Lokesh Ramesh'],
|
||||
bodyText: 'Lokesh Ramesh\nI think outside help would fit best for provider doc follow ups',
|
||||
composerFound: true,
|
||||
latestMessageHash: hashText('I think outside help would fit best for provider doc follow ups'),
|
||||
}, {
|
||||
expectedName: 'Lokesh Ramesh',
|
||||
threadUrl: 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
expectedLastText: 'provider doc follow ups',
|
||||
});
|
||||
|
||||
expect(result.ok).toBe(true);
|
||||
expect(result.blockReason).toBe('verified');
|
||||
});
|
||||
});
|
||||
|
||||
describe('linkedin safe-send command', () => {
|
||||
it('registers as a write command with safe output columns', () => {
|
||||
const command = getRegistry().get('linkedin/safe-send');
|
||||
expect(command).toBeDefined();
|
||||
expect(command.access).toBe('write');
|
||||
expect(command.columns).toEqual(expect.arrayContaining(['status', 'recipient', 'reason']));
|
||||
});
|
||||
|
||||
it('does not type or send when verification fails', async () => {
|
||||
const command = getRegistry().get('linkedin/safe-send');
|
||||
const page = makeFakePage({
|
||||
url: 'https://www.linkedin.com/messaging/thread/bora/',
|
||||
headerNames: ['Bora Nicholson'],
|
||||
bodyText: 'Bora Nicholson',
|
||||
composerFound: true,
|
||||
searchFailure: false,
|
||||
});
|
||||
|
||||
await expect(command.func(page, {
|
||||
'thread-url': 'https://www.linkedin.com/messaging/thread/victoria/',
|
||||
'expected-name': 'Victoria Munoz',
|
||||
message: 'hello victoria',
|
||||
send: true,
|
||||
})).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
|
||||
expect(page.insertText).not.toHaveBeenCalled();
|
||||
expect(page.pressKey).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects non-thread URLs before navigating or typing', async () => {
|
||||
const command = getRegistry().get('linkedin/safe-send');
|
||||
const page = makeFakePage({});
|
||||
|
||||
await expect(command.func(page, {
|
||||
'thread-url': 'https://www.linkedin.com/feed/',
|
||||
'expected-name': 'Victoria Munoz',
|
||||
message: 'hello victoria',
|
||||
send: true,
|
||||
})).rejects.toBeInstanceOf(ArgumentError);
|
||||
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
expect(page.insertText).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('dry-runs by default after verification without filling or sending', async () => {
|
||||
const command = getRegistry().get('linkedin/safe-send');
|
||||
const page = makeFakePage({
|
||||
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
headerNames: ['Lokesh Ramesh'],
|
||||
bodyText: 'Lokesh Ramesh\nprovider doc follow ups',
|
||||
composerFound: true,
|
||||
searchFailure: false,
|
||||
});
|
||||
|
||||
const rows = await command.func(page, {
|
||||
'thread-url': 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
'expected-name': 'Lokesh Ramesh',
|
||||
message: 'both, but starting hands on',
|
||||
});
|
||||
|
||||
expect(rows[0]).toMatchObject({ status: 'verified_dry_run', recipient: 'Lokesh Ramesh', reason: 'verified' });
|
||||
expect(page.insertText).not.toHaveBeenCalled();
|
||||
expect(page.pressKey).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('fills and sends only when --send is explicitly true and post-fill verification matches exactly', async () => {
|
||||
const command = getRegistry().get('linkedin/safe-send');
|
||||
const page = makeFakePage({
|
||||
url: 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
headerNames: ['Lokesh Ramesh'],
|
||||
bodyText: 'Lokesh Ramesh\nprovider doc follow ups',
|
||||
composerFound: true,
|
||||
searchFailure: false,
|
||||
});
|
||||
|
||||
const rows = await command.func(page, {
|
||||
'thread-url': 'https://www.linkedin.com/messaging/thread/lokesh/',
|
||||
'expected-name': 'Lokesh Ramesh',
|
||||
message: 'both, but starting hands on',
|
||||
send: true,
|
||||
});
|
||||
|
||||
expect(rows[0]).toMatchObject({ status: 'sent', recipient: 'Lokesh Ramesh', reason: 'verified' });
|
||||
expect(page.insertText).toHaveBeenCalledWith('both, but starting hands on');
|
||||
expect(page.pressKey).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,210 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
const SALES_INBOX_URL = 'https://www.linkedin.com/sales/inbox/';
|
||||
const THREADS_BASE = 'https://www.linkedin.com/sales-api/salesApiMessagingThreads';
|
||||
const PAGE_SIZE = 20;
|
||||
const DEFAULT_LIMIT = 40;
|
||||
const MAX_LIMIT = 500;
|
||||
const THREAD_DECORATION = '(id,restrictions,archived,unreadMessageCount,nextPageStartsAt,totalMessageCount,messages*(id,type,contentFlag,deliveredAt,lastEditedAt,subject,body,footerText,blockCopy,attachments,author,systemMessageContent),participants*~fs_salesProfile(entityUrn,firstName,lastName,fullName,degree,profilePictureDisplayImage,objectUrn,inmailRestriction))';
|
||||
|
||||
export function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
export function parseLimit(value, defaultValue = DEFAULT_LIMIT) {
|
||||
if (value === undefined || value === null || value === '') return defaultValue;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > MAX_LIMIT) {
|
||||
throw new ArgumentError(`--limit must be an integer between 1 and ${MAX_LIMIT}`);
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
export function encodeRestliDecoration(value) {
|
||||
// LinkedIn's Sales Navigator Rest.li endpoint returns HTTP 400 when this
|
||||
// decoration is sent with literal parentheses. Keep parentheses percent-encoded.
|
||||
return encodeURIComponent(value).replace(/\(/g, '%28').replace(/\)/g, '%29');
|
||||
}
|
||||
|
||||
function salesnavThreadUrl(threadId) {
|
||||
return threadId ? `https://www.linkedin.com/sales/inbox/${encodeURIComponent(threadId)}` : '';
|
||||
}
|
||||
|
||||
function threadListUrl({ count = PAGE_SIZE, pageStartsAt = '' } = {}) {
|
||||
let url = `${THREADS_BASE}?decoration=${encodeRestliDecoration(THREAD_DECORATION)}&count=${count}&filter=INBOX&q=filter`;
|
||||
if (pageStartsAt) url += `&pageStartsAt=${encodeURIComponent(pageStartsAt)}`;
|
||||
return url;
|
||||
}
|
||||
|
||||
function getThreadParticipants(thread) {
|
||||
const resolution = thread?.participantsResolutionResults || {};
|
||||
const participants = Array.isArray(thread?.participants) ? thread.participants : Object.keys(resolution);
|
||||
return participants.map((urn) => resolution[urn] || { entityUrn: urn }).filter(Boolean);
|
||||
}
|
||||
|
||||
function isSelfParticipant(profile) {
|
||||
const degree = String(profile?.degree ?? '').trim();
|
||||
return degree === '0';
|
||||
}
|
||||
|
||||
function otherParticipantName(thread) {
|
||||
const participants = getThreadParticipants(thread);
|
||||
const other = participants.find((p) => !isSelfParticipant(p)) || participants[0];
|
||||
return normalizeWhitespace(other?.fullName || [other?.firstName, other?.lastName].filter(Boolean).join(' '));
|
||||
}
|
||||
|
||||
function parseSalesnavThreads(json) {
|
||||
if (!json || typeof json !== 'object' || !Array.isArray(json.elements)) {
|
||||
throw new CommandExecutionError('Sales Navigator messaging threads API returned malformed payload');
|
||||
}
|
||||
return json.elements.map((thread) => {
|
||||
if (!thread || typeof thread !== 'object') {
|
||||
throw new CommandExecutionError('Sales Navigator messaging threads API returned malformed thread row');
|
||||
}
|
||||
const messages = Array.isArray(thread?.messages) ? thread.messages : [];
|
||||
const lastMessage = messages[0] || {};
|
||||
const deliveredAt = Number(lastMessage.deliveredAt || thread?.nextPageStartsAt || 0);
|
||||
const threadId = normalizeWhitespace(thread?.id || '');
|
||||
if (!threadId) {
|
||||
throw new CommandExecutionError('Sales Navigator messaging thread row missing id');
|
||||
}
|
||||
return {
|
||||
thread_id: threadId,
|
||||
thread_url: salesnavThreadUrl(threadId),
|
||||
person_name: otherParticipantName(thread),
|
||||
last_message_snippet: normalizeWhitespace(lastMessage.body || lastMessage.subject || '').slice(0, 300),
|
||||
last_activity_time: deliveredAt ? new Date(deliveredAt).toISOString() : '',
|
||||
unread: Number(thread?.unreadMessageCount || 0) > 0,
|
||||
unread_count: Number(thread?.unreadMessageCount || 0),
|
||||
total_message_count: Number(thread?.totalMessageCount || messages.length || 0),
|
||||
archived: Boolean(thread?.archived),
|
||||
next_page_starts_at: normalizeWhitespace(thread?.nextPageStartsAt || ''),
|
||||
participants: getThreadParticipants(thread).map((p) => ({
|
||||
name: normalizeWhitespace(p.fullName || [p.firstName, p.lastName].filter(Boolean).join(' ')),
|
||||
entity_urn: normalizeWhitespace(p.entityUrn || ''),
|
||||
object_urn: normalizeWhitespace(p.objectUrn || ''),
|
||||
degree: normalizeWhitespace(p.degree ?? ''),
|
||||
})),
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
function fetchJsonScript(url, csrf) {
|
||||
return String.raw`(async () => {
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
credentials: 'include',
|
||||
headers: {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: 'application/json',
|
||||
},
|
||||
});
|
||||
const text = await res.text();
|
||||
let json = null;
|
||||
try { json = text ? JSON.parse(text) : null; } catch (_) { json = null; }
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status, text };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status, status: res.status, text, json };
|
||||
return { status: res.status, json };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
export async function getCsrf(page) {
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
|
||||
return jsession.replace(/^\"|\"$/g, '');
|
||||
}
|
||||
|
||||
export async function fetchSalesnavJson(page, csrf, url, label) {
|
||||
const result = unwrapEvaluateResult(await page.evaluate(fetchJsonScript(url, csrf)));
|
||||
if (result?.authRequired) throw new AuthRequiredError(LINKEDIN_DOMAIN, `${label} authentication failed (HTTP ${result.status || 'auth_required'}).`);
|
||||
if (result?.error || !result?.json) throw new CommandExecutionError(`${label} returned an unexpected response`, `${result?.error || 'no_json'}\n${normalizeWhitespace(result?.text || '').slice(0, 500)}`);
|
||||
return result.json;
|
||||
}
|
||||
|
||||
export async function fetchInboxRows(page, { limit = DEFAULT_LIMIT, maxPages = 30 } = {}) {
|
||||
const csrf = await getCsrf(page);
|
||||
const rows = [];
|
||||
const seen = new Set();
|
||||
let pageStartsAt = '';
|
||||
let pagesFetched = 0;
|
||||
let hasMorePages = false;
|
||||
while (rows.length < limit && pagesFetched < maxPages) {
|
||||
const json = await fetchSalesnavJson(page, csrf, threadListUrl({ count: PAGE_SIZE, pageStartsAt }), 'Sales Navigator messaging threads API');
|
||||
pagesFetched += 1;
|
||||
const pageRows = parseSalesnavThreads(json);
|
||||
if (pageRows.length === 0) break;
|
||||
for (const row of pageRows) {
|
||||
if (seen.has(row.thread_id)) continue;
|
||||
seen.add(row.thread_id);
|
||||
rows.push(row);
|
||||
if (rows.length >= limit) break;
|
||||
}
|
||||
const last = pageRows[pageRows.length - 1];
|
||||
const next = last?.next_page_starts_at;
|
||||
hasMorePages = Boolean(next);
|
||||
if (!next) break;
|
||||
if (next === pageStartsAt) {
|
||||
throw new CommandExecutionError('Sales Navigator messaging threads API returned the same cursor twice');
|
||||
}
|
||||
pageStartsAt = next;
|
||||
}
|
||||
if (rows.length < limit && hasMorePages && pagesFetched >= maxPages) {
|
||||
throw new CommandExecutionError(`Sales Navigator messaging threads API reached the ${maxPages}-page safety cap before collecting ${limit} conversations`);
|
||||
}
|
||||
return rows.slice(0, limit).map((row, index) => ({ ...row, rank: index + 1 }));
|
||||
}
|
||||
|
||||
export { THREAD_DECORATION, THREADS_BASE };
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'salesnav-inbox',
|
||||
access: 'read',
|
||||
description: 'List LinkedIn Sales Navigator message conversations with API pagination',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.UI,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'limit', type: 'number', default: DEFAULT_LIMIT, help: 'Maximum conversations to return (1-500)' },
|
||||
{ name: 'max-pages', type: 'number', default: 30, help: 'Maximum Sales Navigator API pages to fetch' },
|
||||
{ name: 'unread-only', type: 'bool', default: false, help: 'Return only unread conversations' },
|
||||
],
|
||||
columns: ['rank', 'thread_id', 'thread_url', 'person_name', 'last_message_snippet', 'last_activity_time', 'unread', 'unread_count', 'total_message_count', 'archived', 'participants', 'next_page_starts_at'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-inbox');
|
||||
const limit = parseLimit(args.limit);
|
||||
const maxPages = parseLimit(args['max-pages'], 30);
|
||||
await page.goto(SALES_INBOX_URL);
|
||||
await page.wait(4);
|
||||
let rows = await fetchInboxRows(page, { limit, maxPages });
|
||||
if (args['unread-only']) rows = rows.filter((row) => row.unread);
|
||||
if (rows.length === 0) {
|
||||
if (args['unread-only']) return [];
|
||||
throw new EmptyResultError('linkedin salesnav-inbox', 'No Sales Navigator conversations were found.');
|
||||
}
|
||||
return rows.slice(0, limit).map((row, index) => ({ ...row, rank: index + 1 }));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
THREAD_DECORATION,
|
||||
normalizeWhitespace,
|
||||
parseLimit,
|
||||
encodeRestliDecoration,
|
||||
salesnavThreadUrl,
|
||||
threadListUrl,
|
||||
parseSalesnavThreads,
|
||||
fetchInboxRows,
|
||||
};
|
||||
@@ -0,0 +1,113 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './salesnav-inbox.js';
|
||||
|
||||
const {
|
||||
THREAD_DECORATION,
|
||||
encodeRestliDecoration,
|
||||
parseLimit,
|
||||
parseSalesnavThreads,
|
||||
salesnavThreadUrl,
|
||||
threadListUrl,
|
||||
} = await import('./salesnav-inbox.js').then((m) => m.__test__);
|
||||
|
||||
describe('linkedin salesnav-inbox command', () => {
|
||||
it('percent-encodes Rest.li decoration parentheses for Sales Navigator messaging', () => {
|
||||
const encoded = encodeRestliDecoration('(id,messages*(body))');
|
||||
expect(encoded).toBe('%28id%2Cmessages*%28body%29%29');
|
||||
expect(encoded).not.toContain('(');
|
||||
expect(encoded).not.toContain(')');
|
||||
});
|
||||
|
||||
it('builds the paginated salesApiMessagingThreads inbox URL', () => {
|
||||
const url = threadListUrl({ count: 20, pageStartsAt: '1779070755626' });
|
||||
expect(url).toContain('/sales-api/salesApiMessagingThreads?');
|
||||
expect(url).toContain('q=filter');
|
||||
expect(url).toContain('filter=INBOX');
|
||||
expect(url).toContain('count=20');
|
||||
expect(url).toContain('pageStartsAt=1779070755626');
|
||||
expect(url).toContain(encodeRestliDecoration(THREAD_DECORATION));
|
||||
});
|
||||
|
||||
it('validates limits without silent clamping', () => {
|
||||
expect(parseLimit(undefined)).toBe(40);
|
||||
expect(parseLimit(12)).toBe(12);
|
||||
expect(() => parseLimit(0)).toThrow();
|
||||
expect(() => parseLimit(501)).toThrow();
|
||||
expect(() => parseLimit('abc')).toThrow();
|
||||
});
|
||||
|
||||
it('parses Sales Navigator thread rows with other participant and unread state', () => {
|
||||
const rows = parseSalesnavThreads({ elements: [{
|
||||
id: '2-thread',
|
||||
unreadMessageCount: 1,
|
||||
archived: false,
|
||||
totalMessageCount: 2,
|
||||
nextPageStartsAt: 1778206803669,
|
||||
participants: [
|
||||
'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
|
||||
'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)',
|
||||
],
|
||||
participantsResolutionResults: {
|
||||
'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)': {
|
||||
entityUrn: 'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
|
||||
firstName: 'Rachael',
|
||||
lastName: 'Stolberg',
|
||||
fullName: 'Rachael Stolberg',
|
||||
degree: 2,
|
||||
},
|
||||
'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)': {
|
||||
entityUrn: 'urn:li:fs_salesProfile:(SELF,NAME_SEARCH,T2)',
|
||||
firstName: 'Hanzi',
|
||||
lastName: 'Li',
|
||||
fullName: 'Hanzi Li',
|
||||
degree: 0,
|
||||
},
|
||||
},
|
||||
messages: [{
|
||||
id: 'msg-1',
|
||||
author: 'urn:li:fs_salesProfile:(OTHER,NAME_SEARCH,T1)',
|
||||
body: 'Hi hanzi, happy to chat',
|
||||
deliveredAt: 1778206803669,
|
||||
}],
|
||||
}] });
|
||||
expect(rows).toHaveLength(1);
|
||||
expect(rows[0]).toMatchObject({
|
||||
thread_id: '2-thread',
|
||||
thread_url: salesnavThreadUrl('2-thread'),
|
||||
person_name: 'Rachael Stolberg',
|
||||
last_message_snippet: 'Hi hanzi, happy to chat',
|
||||
last_activity_time: '2026-05-08T02:20:03.669Z',
|
||||
unread: true,
|
||||
unread_count: 1,
|
||||
total_message_count: 2,
|
||||
});
|
||||
});
|
||||
|
||||
it('does not hard-code a specific account name as the inbox owner', () => {
|
||||
const rows = parseSalesnavThreads({ elements: [{
|
||||
id: '2-thread',
|
||||
participants: [
|
||||
'urn:li:fs_salesProfile:(HANZI,NAME_SEARCH,T1)',
|
||||
'urn:li:fs_salesProfile:(ME,NAME_SEARCH,T2)',
|
||||
],
|
||||
participantsResolutionResults: {
|
||||
'urn:li:fs_salesProfile:(HANZI,NAME_SEARCH,T1)': {
|
||||
fullName: 'Hanzi Li',
|
||||
degree: 2,
|
||||
},
|
||||
'urn:li:fs_salesProfile:(ME,NAME_SEARCH,T2)': {
|
||||
fullName: 'Current User',
|
||||
degree: 0,
|
||||
},
|
||||
},
|
||||
messages: [],
|
||||
}] });
|
||||
expect(rows[0].person_name).toBe('Hanzi Li');
|
||||
});
|
||||
|
||||
it('fails typed on malformed thread payloads and missing thread identity', () => {
|
||||
expect(() => parseSalesnavThreads({})).toThrow(CommandExecutionError);
|
||||
expect(() => parseSalesnavThreads({ elements: [{}] })).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,360 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
const SALES_HOME = 'https://www.linkedin.com/sales/';
|
||||
const PROFILE_DECO = '(entityUrn,objectUrn,firstName,lastName,fullName,headline,degree,inmailRestriction,memberBadges,defaultPosition)';
|
||||
const CREDITS_URL = 'https://www.linkedin.com/sales-api/salesApiCredits?q=findCreditGrant&creditGrantType=LSS_INMAIL';
|
||||
const MESSAGE_ACTION_URL = 'https://www.linkedin.com/sales-api/salesApiMessageActions?action=createMessage';
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function requireStringArg(args, key, label = key) {
|
||||
const value = normalizeWhitespace(args[key]);
|
||||
if (!value) throw new ArgumentError(`${label} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
function isLinkedInHost(hostname) {
|
||||
const host = String(hostname || '').toLowerCase();
|
||||
return host === 'linkedin.com' || host.endsWith('.linkedin.com');
|
||||
}
|
||||
|
||||
function parseSalesProfileUrn(value) {
|
||||
const raw = normalizeWhitespace(value);
|
||||
const match = raw.match(/^urn:li:fs_salesProfile:\(([^,()]+),([^,()]+),([^,()]+)\)$/);
|
||||
if (!match) return null;
|
||||
if (!isResolvedSalesProfileParts(match[1], match[2], match[3])) return null;
|
||||
return { profileId: match[1], authType: match[2], authToken: match[3], entityUrn: raw };
|
||||
}
|
||||
|
||||
function isResolvedSalesProfileParts(profileId, authType, authToken) {
|
||||
return [profileId, authType, authToken].every((part) => {
|
||||
const clean = normalizeWhitespace(part).toLowerCase();
|
||||
return clean && clean !== 'undefined' && clean !== 'null' && clean !== 'not_available';
|
||||
});
|
||||
}
|
||||
|
||||
function salesLeadUrlFromParts({ profileId, authType, authToken }) {
|
||||
return `https://www.linkedin.com/sales/lead/${encodeURIComponent(profileId)},${encodeURIComponent(authType)},${encodeURIComponent(authToken)}`;
|
||||
}
|
||||
|
||||
function parseRecipient(value) {
|
||||
const raw = normalizeWhitespace(value);
|
||||
const urn = parseSalesProfileUrn(raw);
|
||||
if (urn) return urn;
|
||||
try {
|
||||
const url = new URL(raw);
|
||||
if (url.protocol !== 'https:' || url.username || url.password || url.port || !isLinkedInHost(url.hostname)) return null;
|
||||
const salesMatch = url.pathname.match(/^\/sales\/lead\/([^,/]+),([^,/]+),([^/]+)\/?$/i);
|
||||
if (salesMatch) {
|
||||
const profileId = decodeURIComponent(salesMatch[1]);
|
||||
const authType = decodeURIComponent(salesMatch[2]);
|
||||
const authToken = decodeURIComponent(salesMatch[3]);
|
||||
if (!isResolvedSalesProfileParts(profileId, authType, authToken)) return null;
|
||||
return { profileId, authType, authToken, entityUrn: `urn:li:fs_salesProfile:(${profileId},${authType},${authToken})` };
|
||||
}
|
||||
const profileMatch = url.pathname.match(/^\/in\/([^/]+)\/?$/i);
|
||||
if (profileMatch) {
|
||||
return { profileId: decodeURIComponent(profileMatch[1]), authType: '', authToken: '', entityUrn: '' };
|
||||
}
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function encodeRestliDecoration(value) {
|
||||
return encodeURIComponent(value).replace(/\(/g, '%28').replace(/\)/g, '%29');
|
||||
}
|
||||
|
||||
function profileApiUrl(recipient) {
|
||||
if (!recipient?.profileId || !recipient?.authType || !recipient?.authToken) return '';
|
||||
const key = `(profileId:${recipient.profileId},authType:${recipient.authType},authToken:${recipient.authToken})`;
|
||||
return `https://www.linkedin.com/sales-api/salesApiProfiles/${key}?decoration=${encodeRestliDecoration(PROFILE_DECO)}`;
|
||||
}
|
||||
|
||||
function randomTrackingId() {
|
||||
const bytes = new Uint8Array(8);
|
||||
if (globalThis.crypto?.getRandomValues) globalThis.crypto.getRandomValues(bytes);
|
||||
else for (let i = 0; i < bytes.length; i += 1) bytes[i] = Math.floor(Math.random() * 256);
|
||||
return Array.from(bytes, (b) => b.toString(16).padStart(2, '0')).join('');
|
||||
}
|
||||
|
||||
function buildCreateMessagePayload({ recipientUrn, subject, body, trackingId = randomTrackingId(), copyToCrm = false }) {
|
||||
const cleanRecipient = normalizeWhitespace(recipientUrn);
|
||||
if (!parseSalesProfileUrn(cleanRecipient)) throw new ArgumentError('--recipient must resolve to a Sales Navigator lead urn');
|
||||
const cleanSubject = normalizeWhitespace(subject);
|
||||
const cleanBody = String(body ?? '').trim();
|
||||
if (!cleanSubject) throw new ArgumentError('--subject is required');
|
||||
if (!cleanBody) throw new ArgumentError('--body is required');
|
||||
if (cleanSubject.length > 200) throw new ArgumentError('--subject must be 200 characters or fewer');
|
||||
if (cleanBody.length > 1900) throw new ArgumentError('--body must be 1900 characters or fewer');
|
||||
return {
|
||||
createMessageRequest: {
|
||||
recipients: [cleanRecipient],
|
||||
subject: cleanSubject,
|
||||
body: cleanBody,
|
||||
copyToCrm: Boolean(copyToCrm),
|
||||
trackingId,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function extractRemainingCredits(json) {
|
||||
const elements = Array.isArray(json?.elements) ? json.elements : [];
|
||||
const inmailGrant = elements.find((el) => el?.type === 'LSS_INMAIL' && Number.isInteger(el.value));
|
||||
if (inmailGrant) return inmailGrant.value;
|
||||
const candidates = [];
|
||||
const visit = (value) => {
|
||||
if (value === null || value === undefined) return;
|
||||
if (typeof value === 'number' && Number.isFinite(value)) candidates.push(value);
|
||||
if (Array.isArray(value)) value.forEach(visit);
|
||||
else if (typeof value === 'object') {
|
||||
for (const [key, child] of Object.entries(value)) {
|
||||
if (/remaining|available|balance|value/i.test(key) && typeof child === 'number') candidates.unshift(child);
|
||||
else if (!/^count$|^start$|^id$/i.test(key)) visit(child);
|
||||
}
|
||||
}
|
||||
};
|
||||
visit(json);
|
||||
return candidates.find((n) => Number.isInteger(n) && n >= 0) ?? null;
|
||||
}
|
||||
|
||||
function fetchJsonScript(url, csrf, options = {}) {
|
||||
return String.raw`(async () => {
|
||||
const headers = {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: ${JSON.stringify(options.accept || 'application/json')},
|
||||
...((${JSON.stringify(Boolean(options.body))}) ? { 'content-type': 'application/json' } : {}),
|
||||
};
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, {
|
||||
credentials: 'include',
|
||||
method: ${JSON.stringify(options.method || 'GET')},
|
||||
headers,
|
||||
body: ${options.body ? JSON.stringify(JSON.stringify(options.body)) : 'undefined'},
|
||||
});
|
||||
const text = await res.text();
|
||||
let json = null;
|
||||
try { json = text ? JSON.parse(text) : null; } catch (_) { json = null; }
|
||||
if (res.status === 401 || res.status === 403) return ['auth', res.status, json, text];
|
||||
if (!res.ok) return ['error', res.status, json, text, 'HTTP ' + res.status];
|
||||
return ['ok', res.status, json, text];
|
||||
} catch (e) {
|
||||
return ['error', 0, null, '', 'fetch failed: ' + ((e && e.message) || String(e))];
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
function requireFetchResult(result, label, { requireJson = true } = {}) {
|
||||
if (Array.isArray(result)) {
|
||||
const [kind, status, json, text, error] = result;
|
||||
result = {
|
||||
authRequired: kind === 'auth',
|
||||
error: kind === 'error' ? error || `HTTP ${status}` : '',
|
||||
status,
|
||||
json,
|
||||
text,
|
||||
};
|
||||
}
|
||||
if (result?.authRequired) throw new AuthRequiredError(LINKEDIN_DOMAIN, `${label} auth failed.`);
|
||||
if (result?.error) throw new CommandExecutionError(`${label} failed`, result.error);
|
||||
if (!result || typeof result !== 'object' || Array.isArray(result)) {
|
||||
throw new CommandExecutionError(`${label} returned malformed response`);
|
||||
}
|
||||
if (requireJson && (!result.json || typeof result.json !== 'object' || Array.isArray(result.json))) {
|
||||
throw new CommandExecutionError(`${label} returned malformed response`, 'missing_json');
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
function salesPageShowsSentMessage(text, recipientName) {
|
||||
const normalizedText = normalizeWhitespace(text);
|
||||
const firstName = normalizeWhitespace(recipientName).split(' ')[0];
|
||||
return normalizedText.includes('You sent a Sales Navigator message')
|
||||
&& (!firstName || normalizedText.includes(firstName));
|
||||
}
|
||||
|
||||
async function getCsrf(page) {
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
|
||||
return jsession.replace(/^\"|\"$/g, '');
|
||||
}
|
||||
|
||||
async function resolveRecipient(page, parsed, csrf) {
|
||||
if (!parsed) throw new ArgumentError('--recipient must be a Sales Navigator lead URL, Sales Navigator profile URL, LinkedIn /in/ URL, or urn:li:fs_salesProfile:(...)');
|
||||
if (parsed.entityUrn && parsed.authType && parsed.authToken) return parsed;
|
||||
|
||||
await page.goto(`https://www.linkedin.com/sales/lead/${encodeURIComponent(parsed.profileId)}`);
|
||||
await page.wait(6);
|
||||
const probe = unwrapEvaluateResult(await page.evaluate(String.raw`(() => {
|
||||
const href = location.href;
|
||||
const text = document.body ? document.body.innerText : '';
|
||||
const resourceUrns = Array.from(performance.getEntriesByType('resource'))
|
||||
.map((entry) => entry.name)
|
||||
.filter((name) => name.includes('/sales-api/salesApiProfiles/'))
|
||||
.slice(-20);
|
||||
return { href, text: text.slice(0, 1000), resourceUrns };
|
||||
})()`));
|
||||
const urlMatch = String(probe?.href || '').match(/\/sales\/lead\/([^,/]+),([^,/]+),([^/?#]+)/i);
|
||||
if (urlMatch && isResolvedSalesProfileParts(urlMatch[1], urlMatch[2], urlMatch[3])) {
|
||||
return {
|
||||
profileId: decodeURIComponent(urlMatch[1]),
|
||||
authType: decodeURIComponent(urlMatch[2]),
|
||||
authToken: decodeURIComponent(urlMatch[3]),
|
||||
entityUrn: `urn:li:fs_salesProfile:(${decodeURIComponent(urlMatch[1])},${decodeURIComponent(urlMatch[2])},${decodeURIComponent(urlMatch[3])})`,
|
||||
};
|
||||
}
|
||||
for (const resource of probe?.resourceUrns || []) {
|
||||
const resourceMatch = String(resource).match(/profileId:([^,)]+),authType:([^,)]+),authToken:([^,)]+)\)/);
|
||||
if (resourceMatch && resourceMatch[1] === parsed.profileId) {
|
||||
return {
|
||||
profileId: resourceMatch[1],
|
||||
authType: resourceMatch[2],
|
||||
authToken: resourceMatch[3],
|
||||
entityUrn: `urn:li:fs_salesProfile:(${resourceMatch[1]},${resourceMatch[2]},${resourceMatch[3]})`,
|
||||
};
|
||||
}
|
||||
}
|
||||
void csrf;
|
||||
throw new CommandExecutionError('Could not resolve Sales Navigator auth token for recipient', `Observed URL: ${probe?.href || 'url_not_available'}\nBody: ${normalizeWhitespace(probe?.text || '').slice(0, 500)}`);
|
||||
}
|
||||
|
||||
function profileSummary(json) {
|
||||
const data = json?.data || json || {};
|
||||
const pos = data.defaultPosition || (Array.isArray(data.positions) ? data.positions.find((p) => p.current) || data.positions[0] : {}) || {};
|
||||
return {
|
||||
recipient: normalizeWhitespace(data.fullName || [data.firstName, data.lastName].filter(Boolean).join(' ')),
|
||||
title: normalizeWhitespace(pos.title || data.headline || ''),
|
||||
company: normalizeWhitespace(pos.companyName || pos.company?.name || ''),
|
||||
degree: normalizeWhitespace(data.degree || ''),
|
||||
inmail_restriction: normalizeWhitespace(data.inmailRestriction || ''),
|
||||
open_link: Boolean(data.memberBadges?.openLink),
|
||||
};
|
||||
}
|
||||
|
||||
function requireProfileSummary(json) {
|
||||
const summary = profileSummary(json);
|
||||
if (!summary.recipient) {
|
||||
throw new CommandExecutionError('Sales Navigator profile lookup returned malformed profile data', 'missing_recipient_name');
|
||||
}
|
||||
return summary;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'salesnav-message',
|
||||
access: 'write',
|
||||
description: 'Send or dry-run a LinkedIn Sales Navigator InMail to a lead using the Sales Navigator messaging API',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.UI,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'recipient', type: 'string', required: true, positional: true, help: 'Sales Navigator lead URL, LinkedIn /in/ URL from salesnav-search, or urn:li:fs_salesProfile:(...)' },
|
||||
{ name: 'subject', type: 'string', required: true, help: 'InMail subject' },
|
||||
{ name: 'body', type: 'string', required: true, help: 'InMail body' },
|
||||
{ name: 'send', type: 'bool', default: false, help: 'Actually send the InMail. Default is dry-run validation only.' },
|
||||
{ name: 'copy-to-crm', type: 'bool', default: false, help: 'Set Sales Navigator copyToCrm on the message request' },
|
||||
],
|
||||
columns: ['status', 'recipient', 'title', 'company', 'credits_remaining', 'credits_before', 'credits_after', 'sent_in_salesnav', 'message_chars', 'subject_chars', 'recipient_urn', 'degree', 'inmail_restriction', 'open_link'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-message');
|
||||
const recipientArg = requireStringArg(args, 'recipient', '--recipient');
|
||||
const subject = requireStringArg(args, 'subject', '--subject');
|
||||
const body = String(args.body ?? '').trim();
|
||||
if (!body) throw new ArgumentError('--body is required');
|
||||
|
||||
await page.goto(SALES_HOME);
|
||||
await page.wait(4);
|
||||
const csrf = await getCsrf(page);
|
||||
const recipient = await resolveRecipient(page, parseRecipient(recipientArg), csrf);
|
||||
|
||||
let summary = { recipient: '', title: '', company: '', degree: '', inmail_restriction: '', open_link: false };
|
||||
const profileUrl = profileApiUrl(recipient);
|
||||
if (profileUrl) {
|
||||
const profileResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(profileUrl, csrf))), 'LinkedIn Sales Navigator profile API');
|
||||
summary = requireProfileSummary(profileResult.json);
|
||||
}
|
||||
if (summary.inmail_restriction && summary.inmail_restriction !== 'NO_RESTRICTION') {
|
||||
throw new CommandExecutionError('Sales Navigator InMail blocked by recipient restriction', summary.inmail_restriction);
|
||||
}
|
||||
|
||||
const creditsResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(CREDITS_URL, csrf))), 'LinkedIn Sales Navigator credits API');
|
||||
const creditsRemaining = extractRemainingCredits(creditsResult?.json);
|
||||
|
||||
const payload = buildCreateMessagePayload({ recipientUrn: recipient.entityUrn, subject, body, copyToCrm: args['copy-to-crm'] });
|
||||
if (!args.send) {
|
||||
return [{
|
||||
status: 'validated_dry_run',
|
||||
recipient: summary.recipient,
|
||||
title: summary.title,
|
||||
company: summary.company,
|
||||
credits_remaining: creditsRemaining,
|
||||
credits_before: creditsRemaining,
|
||||
credits_after: '',
|
||||
sent_in_salesnav: false,
|
||||
message_chars: body.length,
|
||||
subject_chars: subject.length,
|
||||
recipient_urn: recipient.entityUrn,
|
||||
degree: summary.degree,
|
||||
inmail_restriction: summary.inmail_restriction,
|
||||
open_link: summary.open_link,
|
||||
}];
|
||||
}
|
||||
|
||||
const sendResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(MESSAGE_ACTION_URL, csrf, {
|
||||
method: 'POST',
|
||||
accept: 'application/vnd.linkedin.normalized+json+2.1',
|
||||
body: payload,
|
||||
}))), 'LinkedIn Sales Navigator message API', { requireJson: false });
|
||||
void sendResult;
|
||||
await page.wait(3);
|
||||
const creditsAfterResult = requireFetchResult(unwrapEvaluateResult(await page.evaluate(fetchJsonScript(CREDITS_URL, csrf))), 'LinkedIn Sales Navigator credits API after send');
|
||||
const creditsAfter = extractRemainingCredits(creditsAfterResult?.json);
|
||||
await page.goto(salesLeadUrlFromParts(recipient));
|
||||
await page.wait(6);
|
||||
const salesPageText = unwrapEvaluateResult(await page.evaluate('document.body ? document.body.innerText : ""'));
|
||||
const sentInSalesNav = salesPageShowsSentMessage(salesPageText, summary.recipient);
|
||||
if (!sentInSalesNav) throw new CommandExecutionError('Sales Navigator post-send verification failed', 'Sent activity was not found on the Sales Navigator lead page.');
|
||||
return [{
|
||||
status: 'sent',
|
||||
recipient: summary.recipient,
|
||||
title: summary.title,
|
||||
company: summary.company,
|
||||
credits_remaining: creditsAfter,
|
||||
credits_before: creditsRemaining,
|
||||
credits_after: creditsAfter,
|
||||
sent_in_salesnav: sentInSalesNav,
|
||||
message_chars: body.length,
|
||||
subject_chars: subject.length,
|
||||
recipient_urn: recipient.entityUrn,
|
||||
degree: summary.degree,
|
||||
inmail_restriction: summary.inmail_restriction,
|
||||
open_link: summary.open_link,
|
||||
}];
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
parseSalesProfileUrn,
|
||||
isResolvedSalesProfileParts,
|
||||
parseRecipient,
|
||||
salesLeadUrlFromParts,
|
||||
profileApiUrl,
|
||||
buildCreateMessagePayload,
|
||||
extractRemainingCredits,
|
||||
profileSummary,
|
||||
requireProfileSummary,
|
||||
salesPageShowsSentMessage,
|
||||
};
|
||||
@@ -0,0 +1,172 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './salesnav-message.js';
|
||||
|
||||
const {
|
||||
parseSalesProfileUrn,
|
||||
parseRecipient,
|
||||
salesLeadUrlFromParts,
|
||||
profileApiUrl,
|
||||
buildCreateMessagePayload,
|
||||
extractRemainingCredits,
|
||||
profileSummary,
|
||||
requireProfileSummary,
|
||||
salesPageShowsSentMessage,
|
||||
} = await import('./salesnav-message.js').then((m) => m.__test__);
|
||||
|
||||
function createPageMock(evaluateResults = []) {
|
||||
const evaluate = vi.fn();
|
||||
for (const result of evaluateResults) evaluate.mockResolvedValueOnce(result);
|
||||
evaluate.mockResolvedValue(undefined);
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate,
|
||||
getCookies: vi.fn().mockResolvedValue([{ name: 'JSESSIONID', value: '"csrf"', domain: '.linkedin.com' }]),
|
||||
};
|
||||
}
|
||||
|
||||
describe('linkedin salesnav-message command', () => {
|
||||
it('parses Sales Navigator profile urns and lead URLs', () => {
|
||||
const urn = 'urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)';
|
||||
expect(parseSalesProfileUrn(urn)).toMatchObject({
|
||||
profileId: 'ACwAAAJS8TABxyz',
|
||||
authType: 'NAME_SEARCH',
|
||||
authToken: 'Enlo',
|
||||
entityUrn: urn,
|
||||
});
|
||||
expect(parseSalesProfileUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,undefined,undefined)')).toBeNull();
|
||||
|
||||
const parsed = parseRecipient('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
|
||||
expect(parsed).toMatchObject({ profileId: 'ACwAAAJS8TABxyz', authType: 'NAME_SEARCH', authToken: 'Enlo' });
|
||||
expect(parsed.entityUrn).toBe(urn);
|
||||
expect(salesLeadUrlFromParts(parsed)).toBe('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
|
||||
});
|
||||
|
||||
it('accepts LinkedIn /in tokens as unresolved recipients', () => {
|
||||
expect(parseRecipient('https://www.linkedin.com/in/ACwAAAJS8TABxyz/')).toMatchObject({
|
||||
profileId: 'ACwAAAJS8TABxyz',
|
||||
authType: '',
|
||||
authToken: '',
|
||||
entityUrn: '',
|
||||
});
|
||||
});
|
||||
|
||||
it('builds profile API URLs with the Sales Navigator auth key', () => {
|
||||
const url = profileApiUrl({ profileId: 'P1', authType: 'NAME_SEARCH', authToken: 'T1' });
|
||||
expect(url).toContain('/sales-api/salesApiProfiles/(profileId:P1,authType:NAME_SEARCH,authToken:T1)');
|
||||
expect(url).toContain('decoration=');
|
||||
});
|
||||
|
||||
it('constructs the createMessage action payload used by Sales Navigator', () => {
|
||||
const payload = buildCreateMessagePayload({
|
||||
recipientUrn: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
|
||||
subject: 'Quick QA doc question',
|
||||
body: 'Hi Jane, can I ask a quick question?',
|
||||
trackingId: '0123456789abcdef',
|
||||
copyToCrm: false,
|
||||
});
|
||||
expect(payload).toEqual({
|
||||
createMessageRequest: {
|
||||
recipients: ['urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)'],
|
||||
subject: 'Quick QA doc question',
|
||||
body: 'Hi Jane, can I ask a quick question?',
|
||||
copyToCrm: false,
|
||||
trackingId: '0123456789abcdef',
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('validates payload fields before any send attempt', () => {
|
||||
expect(() => buildCreateMessagePayload({ recipientUrn: '', subject: 's', body: 'b' })).toThrow();
|
||||
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: '', body: 'b' })).toThrow();
|
||||
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: 's', body: '' })).toThrow();
|
||||
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,A,T)', subject: 'x'.repeat(201), body: 'b' })).toThrow();
|
||||
expect(() => buildCreateMessagePayload({ recipientUrn: 'urn:li:fs_salesProfile:(P,undefined,undefined)', subject: 's', body: 'b' })).toThrow();
|
||||
});
|
||||
|
||||
it('detects the Sales Navigator sent activity on a verified lead page', () => {
|
||||
expect(salesPageShowsSentMessage('5/18/2026 You sent a Sales Navigator message to Jane', 'Jane Q')).toBe(true);
|
||||
expect(salesPageShowsSentMessage('No recent activity', 'Jane Q')).toBe(false);
|
||||
});
|
||||
|
||||
it('extracts a plausible remaining InMail credit count', () => {
|
||||
expect(extractRemainingCredits({ elements: [{ type: 'LSS_INMAIL', value: 149, id: 1 }], paging: { count: 10 } })).toBe(149);
|
||||
expect(extractRemainingCredits({ data: { remaining: 149, used: 1 } })).toBe(149);
|
||||
expect(extractRemainingCredits({ elements: [{ availableCount: 12 }] })).toBe(12);
|
||||
expect(extractRemainingCredits({})).toBe(null);
|
||||
});
|
||||
|
||||
it('summarizes decorated Sales Navigator profile data', () => {
|
||||
expect(profileSummary({ data: {
|
||||
fullName: 'Rayki Goh',
|
||||
headline: 'Food Safety',
|
||||
degree: 3,
|
||||
defaultPosition: { title: 'FSQA Manager', companyName: 'Acme Foods' },
|
||||
memberBadges: { openLink: false },
|
||||
} })).toMatchObject({
|
||||
recipient: 'Rayki Goh',
|
||||
title: 'FSQA Manager',
|
||||
company: 'Acme Foods',
|
||||
degree: '3',
|
||||
open_link: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('fails typed when decorated profile data has no recipient identity', () => {
|
||||
expect(() => requireProfileSummary({ data: { defaultPosition: { title: 'FSQA' } } }))
|
||||
.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('keeps manifest columns aligned with dry-run rows and fails typed on malformed profile API', async () => {
|
||||
const cmd = getRegistry().get('linkedin/salesnav-message');
|
||||
expect(cmd?.columns).toEqual([
|
||||
'status',
|
||||
'recipient',
|
||||
'title',
|
||||
'company',
|
||||
'credits_remaining',
|
||||
'credits_before',
|
||||
'credits_after',
|
||||
'sent_in_salesnav',
|
||||
'message_chars',
|
||||
'subject_chars',
|
||||
'recipient_urn',
|
||||
'degree',
|
||||
'inmail_restriction',
|
||||
'open_link',
|
||||
]);
|
||||
|
||||
const goodPage = createPageMock([
|
||||
{ status: 200, json: { data: { fullName: 'Jane Doe', defaultPosition: { title: 'QA', companyName: 'Acme' }, degree: 2, inmailRestriction: 'NO_RESTRICTION', memberBadges: { openLink: true } } } },
|
||||
{ status: 200, json: { elements: [{ type: 'LSS_INMAIL', value: 12 }] } },
|
||||
]);
|
||||
const rows = await cmd.func(goodPage, {
|
||||
recipient: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
|
||||
subject: 'Hello',
|
||||
body: 'Quick question',
|
||||
});
|
||||
expect(Object.keys(rows[0]).sort()).toEqual([...cmd.columns].sort());
|
||||
expect(rows[0]).toMatchObject({
|
||||
status: 'validated_dry_run',
|
||||
recipient: 'Jane Doe',
|
||||
credits_remaining: 12,
|
||||
credits_before: 12,
|
||||
credits_after: '',
|
||||
sent_in_salesnav: false,
|
||||
degree: '2',
|
||||
inmail_restriction: 'NO_RESTRICTION',
|
||||
open_link: true,
|
||||
});
|
||||
|
||||
const malformedPage = createPageMock([
|
||||
{ status: 200, json: { data: { defaultPosition: { title: 'QA' } } } },
|
||||
]);
|
||||
await expect(cmd.func(malformedPage, {
|
||||
recipient: 'urn:li:fs_salesProfile:(P1,NAME_SEARCH,T1)',
|
||||
subject: 'Hello',
|
||||
body: 'Quick question',
|
||||
})).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,186 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
const LINKEDIN_DOMAIN = 'www.linkedin.com';
|
||||
const SALES_HOME = 'https://www.linkedin.com/sales/';
|
||||
const LEAD_SEARCH_BASE = 'https://www.linkedin.com/sales-api/salesApiLeadSearch';
|
||||
// Versioned response decoration. LinkedIn bumps this on Sales Navigator
|
||||
// redeploys; if the response shape ever changes, refresh it from a live
|
||||
// /sales/search/people request.
|
||||
const LEAD_SEARCH_DECORATION = 'com.linkedin.sales.deco.desktop.searchv2.LeadSearchResult-14';
|
||||
const PAGE_SIZE = 25;
|
||||
|
||||
function normalizeWhitespace(value) {
|
||||
return String(value ?? '').replace(/[ ]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
}
|
||||
|
||||
function requireStringArg(args, key, label = key) {
|
||||
const value = normalizeWhitespace(args[key]);
|
||||
if (!value) throw new ArgumentError(`${label} is required`);
|
||||
return value;
|
||||
}
|
||||
|
||||
function parseLimit(value) {
|
||||
if (value === undefined || value === null || value === '') return 25;
|
||||
const limit = Number(value);
|
||||
if (!Number.isInteger(limit) || limit < 1 || limit > 500) {
|
||||
throw new ArgumentError('--limit must be an integer between 1 and 500');
|
||||
}
|
||||
return limit;
|
||||
}
|
||||
|
||||
function unwrapEvaluateResult(payload) {
|
||||
if (payload && typeof payload === 'object' && 'data' in payload && 'session' in payload) return payload.data;
|
||||
return payload;
|
||||
}
|
||||
|
||||
// Sales Navigator keeps the structural ( ) , : of the query literal and only
|
||||
// percent-encodes the keyword value.
|
||||
function leadSearchUrl(keywords, start) {
|
||||
const query = '(spellCorrectionEnabled:true,recentSearchParam:(doLogHistory:true),keywords:'
|
||||
+ encodeURIComponent(keywords) + ')';
|
||||
return LEAD_SEARCH_BASE
|
||||
+ '?q=searchQuery&query=' + query
|
||||
+ '&start=' + start + '&count=' + PAGE_SIZE
|
||||
+ '&decorationId=' + LEAD_SEARCH_DECORATION;
|
||||
}
|
||||
|
||||
function fetchLeadSearchScript(url, csrf) {
|
||||
return String.raw`(async () => {
|
||||
const headers = {
|
||||
'csrf-token': ${JSON.stringify(csrf)},
|
||||
'x-restli-protocol-version': '2.0.0',
|
||||
accept: 'application/json',
|
||||
};
|
||||
try {
|
||||
const res = await fetch(${JSON.stringify(url)}, { credentials: 'include', headers });
|
||||
if (res.status === 401 || res.status === 403) return { authRequired: true, status: res.status };
|
||||
if (!res.ok) return { error: 'HTTP ' + res.status };
|
||||
return { json: await res.json() };
|
||||
} catch (e) {
|
||||
return { error: 'fetch failed: ' + ((e && e.message) || String(e)) };
|
||||
}
|
||||
})()`;
|
||||
}
|
||||
|
||||
// Sales Navigator search returns no /in/ vanity URL, but the entityUrn carries
|
||||
// the obfuscated member token, and linkedin.com/in/<token> is a valid profile
|
||||
// URL that the connect command accepts.
|
||||
function profileUrlFromEntityUrn(entityUrn) {
|
||||
const match = String(entityUrn || '').match(/fs_salesProfile:\(([^,)]+)/);
|
||||
return match && match[1] ? 'https://www.linkedin.com/in/' + match[1] : '';
|
||||
}
|
||||
|
||||
function leadUrlFromEntityUrn(entityUrn) {
|
||||
const match = String(entityUrn || '').match(/^urn:li:fs_salesProfile:\(([^,()]+),([^,()]+),([^,()]+)\)$/);
|
||||
if (!match) return '';
|
||||
return `https://www.linkedin.com/sales/lead/${encodeURIComponent(match[1])},${encodeURIComponent(match[2])},${encodeURIComponent(match[3])}`;
|
||||
}
|
||||
|
||||
function parseLeads(json) {
|
||||
if (!json || typeof json !== 'object' || !Array.isArray(json.elements)) {
|
||||
throw new CommandExecutionError('Sales Navigator lead search API returned malformed payload');
|
||||
}
|
||||
const leads = [];
|
||||
for (const el of json.elements) {
|
||||
if (!el || typeof el !== 'object') {
|
||||
throw new CommandExecutionError('Sales Navigator lead search API returned malformed lead row');
|
||||
}
|
||||
const current = Array.isArray(el.currentPositions) ? el.currentPositions : [];
|
||||
const past = Array.isArray(el.pastPositions) ? el.pastPositions : [];
|
||||
const pos = current[0] || past[0] || {};
|
||||
const name = normalizeWhitespace(el.fullName || [el.firstName, el.lastName].filter(Boolean).join(' '));
|
||||
if (!name) {
|
||||
throw new CommandExecutionError('Sales Navigator lead row missing name');
|
||||
}
|
||||
const entityUrn = normalizeWhitespace(el.entityUrn || '');
|
||||
if (!profileUrlFromEntityUrn(entityUrn)) {
|
||||
throw new CommandExecutionError('Sales Navigator lead row missing profile identity');
|
||||
}
|
||||
leads.push({
|
||||
name,
|
||||
title: normalizeWhitespace(pos.title || ''),
|
||||
company: normalizeWhitespace(pos.companyName || ''),
|
||||
location: normalizeWhitespace(el.geoRegion || ''),
|
||||
degree: normalizeWhitespace(el.degree || ''),
|
||||
profile_url: profileUrlFromEntityUrn(entityUrn),
|
||||
lead_url: leadUrlFromEntityUrn(entityUrn),
|
||||
recipient_urn: entityUrn,
|
||||
});
|
||||
}
|
||||
return leads;
|
||||
}
|
||||
|
||||
function requireLeadSearchResult(result) {
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn Sales Navigator API auth failed (HTTP ' + (result.status || '') + '). Confirm the account has Sales Navigator access.');
|
||||
}
|
||||
if (result?.error) {
|
||||
throw new CommandExecutionError('Sales Navigator lead search API returned an unexpected response', result.error);
|
||||
}
|
||||
if (!result || !result.json) {
|
||||
throw new CommandExecutionError('Sales Navigator lead search API returned an unexpected response', 'no_json');
|
||||
}
|
||||
return result.json;
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'linkedin',
|
||||
name: 'salesnav-search',
|
||||
access: 'read',
|
||||
description: 'Search LinkedIn Sales Navigator for people leads by keyword',
|
||||
domain: LINKEDIN_DOMAIN,
|
||||
strategy: Strategy.UI,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'keywords', type: 'string', required: true, positional: true, help: 'People search keywords, e.g. "quality manager food manufacturing"' },
|
||||
{ name: 'limit', type: 'number', default: 25, help: 'Maximum leads to return (1-500, fetched 25 per request)' },
|
||||
],
|
||||
columns: ['rank', 'name', 'title', 'company', 'location', 'degree', 'profile_url', 'lead_url', 'recipient_urn'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for linkedin salesnav-search');
|
||||
const keywords = requireStringArg(args, 'keywords', '--keywords');
|
||||
const limit = parseLimit(args.limit);
|
||||
|
||||
await page.goto(SALES_HOME);
|
||||
await page.wait(6);
|
||||
|
||||
const cookies = await page.getCookies({ url: 'https://www.linkedin.com' });
|
||||
const jsession = cookies.find((c) => c.name === 'JSESSIONID')?.value;
|
||||
if (!jsession) {
|
||||
throw new AuthRequiredError(LINKEDIN_DOMAIN, 'LinkedIn JSESSIONID cookie not found. Please sign in to LinkedIn.');
|
||||
}
|
||||
const csrf = jsession.replace(/^\"|\"$/g, '');
|
||||
|
||||
const leads = [];
|
||||
const seen = new Set();
|
||||
for (let start = 0; leads.length < limit && start < 2000; start += PAGE_SIZE) {
|
||||
const result = unwrapEvaluateResult(await page.evaluate(fetchLeadSearchScript(leadSearchUrl(keywords, start), csrf)));
|
||||
const json = requireLeadSearchResult(result);
|
||||
const pageLeads = parseLeads(json);
|
||||
if (pageLeads.length === 0) break;
|
||||
for (const lead of pageLeads) {
|
||||
const key = lead.profile_url || lead.name.toLowerCase();
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
leads.push(lead);
|
||||
}
|
||||
await page.wait(1);
|
||||
}
|
||||
|
||||
if (leads.length === 0) {
|
||||
throw new EmptyResultError('linkedin salesnav-search', 'No Sales Navigator leads were found.');
|
||||
}
|
||||
return leads.slice(0, limit).map((lead, index) => ({ rank: index + 1, ...lead }));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeWhitespace,
|
||||
parseLimit,
|
||||
leadSearchUrl,
|
||||
profileUrlFromEntityUrn,
|
||||
leadUrlFromEntityUrn,
|
||||
parseLeads,
|
||||
requireLeadSearchResult,
|
||||
};
|
||||
@@ -0,0 +1,76 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import './salesnav-search.js';
|
||||
|
||||
const {
|
||||
parseLimit,
|
||||
leadSearchUrl,
|
||||
profileUrlFromEntityUrn,
|
||||
leadUrlFromEntityUrn,
|
||||
parseLeads,
|
||||
requireLeadSearchResult,
|
||||
} = await import('./salesnav-search.js').then((m) => m.__test__);
|
||||
|
||||
describe('linkedin salesnav-search command', () => {
|
||||
it('builds a salesApiLeadSearch URL with encoded keywords and pagination', () => {
|
||||
const url = leadSearchUrl('quality manager food', 50);
|
||||
expect(url).toContain('/sales-api/salesApiLeadSearch');
|
||||
expect(url).toContain('keywords:quality%20manager%20food');
|
||||
expect(url).toContain('start=50');
|
||||
expect(url).toContain('count=25');
|
||||
});
|
||||
|
||||
it('derives a profile URL from the sales-profile entityUrn token', () => {
|
||||
expect(profileUrlFromEntityUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)'))
|
||||
.toBe('https://www.linkedin.com/in/ACwAAAJS8TABxyz');
|
||||
expect(profileUrlFromEntityUrn('')).toBe('');
|
||||
expect(profileUrlFromEntityUrn('not-a-urn')).toBe('');
|
||||
});
|
||||
|
||||
it('derives a Sales Navigator lead URL from the full sales-profile entityUrn', () => {
|
||||
expect(leadUrlFromEntityUrn('urn:li:fs_salesProfile:(ACwAAAJS8TABxyz,NAME_SEARCH,Enlo)'))
|
||||
.toBe('https://www.linkedin.com/sales/lead/ACwAAAJS8TABxyz,NAME_SEARCH,Enlo');
|
||||
expect(leadUrlFromEntityUrn('not-a-urn')).toBe('');
|
||||
});
|
||||
|
||||
it('validates --limit without silent clamping', () => {
|
||||
expect(parseLimit(undefined)).toBe(25);
|
||||
expect(parseLimit(120)).toBe(120);
|
||||
expect(() => parseLimit(0)).toThrow();
|
||||
expect(() => parseLimit(999)).toThrow();
|
||||
expect(() => parseLimit('abc')).toThrow();
|
||||
});
|
||||
|
||||
it('parses lead rows and falls back to past positions', () => {
|
||||
const json = { elements: [
|
||||
{ fullName: 'Jane Q', geoRegion: 'Vancouver, BC', degree: 2,
|
||||
entityUrn: 'urn:li:fs_salesProfile:(TOKEN1,NAME_SEARCH,abc)',
|
||||
currentPositions: [{ title: 'QA Manager', companyName: 'Acme Foods' }] },
|
||||
{ fullName: 'No Current', geoRegion: 'Toronto',
|
||||
entityUrn: 'urn:li:fs_salesProfile:(TOKEN2,NAME_SEARCH,def)',
|
||||
currentPositions: [], pastPositions: [{ title: 'Past QA Lead', companyName: 'Old Co' }] },
|
||||
] };
|
||||
const leads = parseLeads(json);
|
||||
expect(leads).toHaveLength(2);
|
||||
expect(leads[0]).toMatchObject({
|
||||
name: 'Jane Q',
|
||||
title: 'QA Manager',
|
||||
company: 'Acme Foods',
|
||||
location: 'Vancouver, BC',
|
||||
profile_url: 'https://www.linkedin.com/in/TOKEN1',
|
||||
lead_url: 'https://www.linkedin.com/sales/lead/TOKEN1,NAME_SEARCH,abc',
|
||||
recipient_urn: 'urn:li:fs_salesProfile:(TOKEN1,NAME_SEARCH,abc)',
|
||||
});
|
||||
expect(leads[1]).toMatchObject({ name: 'No Current', title: 'Past QA Lead', company: 'Old Co' });
|
||||
});
|
||||
|
||||
it('fails typed on malformed lead payloads instead of silently dropping rows', () => {
|
||||
expect(() => parseLeads({})).toThrow(CommandExecutionError);
|
||||
expect(() => parseLeads({ elements: [{ firstName: '', lastName: '', entityUrn: 'urn:li:fs_salesProfile:(TOKEN3,x,y)' }] }))
|
||||
.toThrow(CommandExecutionError);
|
||||
expect(() => parseLeads({ elements: [{ fullName: 'No Identity' }] }))
|
||||
.toThrow(CommandExecutionError);
|
||||
expect(() => requireLeadSearchResult({ error: 'HTTP 500' })).toThrow(CommandExecutionError);
|
||||
expect(() => requireLeadSearchResult({})).toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user