Compare commits

..

6 Commits

Author SHA1 Message Date
jackwener dc055954cf fix(browser): fail unbind when detach command fails 2026-04-27 17:21:01 +08:00
jackwener e5de7f391e fix(browser): bind only current window tabs 2026-04-27 17:19:43 +08:00
jackwener bebab9e6a2 refactor(browser): rename bind command 2026-04-27 15:57:02 +08:00
jackwener 682994d6e2 test(extension): cover bind-current owned-overwrite refusal
Adds regression for the second guard in handleBindCurrent that refuses
binding when the bound:* workspace already has an owned automation
window. Previously only the non-bound prefix path was tested.
2026-04-27 15:22:32 +08:00
jackwener 204f4b7109 docs(browser): document bound session idle semantics 2026-04-27 15:17:12 +08:00
jackwener 3a087aaf80 feat(browser): bind current tab to bound workspace 2026-04-27 15:09:31 +08:00
211 changed files with 2076 additions and 8824 deletions
-2
View File
@@ -5,8 +5,6 @@
### Features
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
## [1.7.8](https://github.com/jackwener/opencli/compare/v1.7.7...v1.7.8) (2026-04-25)
+9 -28
View File
@@ -20,7 +20,6 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type, extract, screenshot — all through your logged-in Chrome session.
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
- **Website → CLI** — Turn any website into a deterministic CLI: 90+ pre-built adapters, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
@@ -34,10 +33,7 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other
### 1. Install OpenCLI
OpenCLI requires **Node.js >= 21**.
```bash
node --version
npm install -g @jackwener/opencli
```
@@ -59,20 +55,7 @@ Install **OpenCLI** from the [Chrome Web Store](https://chromewebstore.google.co
opencli doctor
```
### 4. Optional: name your Chrome profile
Each Chrome profile runs its own OpenCLI extension instance. If you use multiple Chrome profiles, list the connected profiles and assign local aliases:
```bash
opencli profile list
opencli profile rename <contextId> work
opencli profile use work
opencli --profile work browser state
```
With only one connected profile, OpenCLI uses it automatically. With multiple connected profiles and no default, OpenCLI asks you to choose instead of guessing.
### 5. Run your first commands
### 4. Run your first commands
```bash
opencli list
@@ -86,7 +69,7 @@ Use OpenCLI directly when you want a reliable command instead of a live browser
- `opencli list` shows every registered command.
- `opencli <site> <command>` runs a built-in or generated adapter.
- `opencli external register mycli` exposes a local CLI through the same discovery surface.
- `opencli register mycli` exposes a local CLI through the same discovery surface.
- `opencli doctor` helps diagnose browser connectivity.
## For AI Agents
@@ -174,8 +157,7 @@ OpenCLI is not only for websites. It can also:
## Prerequisites
- **Node.js**: >= 21.0.0 (required for the standard npm install path)
- **Bun**: >= 1.0 (optional alternative runtime)
- **Node.js**: >= 21.0.0 (or **Bun** >= 1.0)
- **Chrome or Chromium** running and logged into the target site for browser-backed commands
> **Important**: Browser-backed commands reuse your Chrome/Chromium login session. If you get empty data or permission-like failures, first confirm the site is already open and authenticated in Chrome/Chromium.
@@ -185,9 +167,8 @@ OpenCLI is not only for websites. It can also:
| Variable | Default | Description |
|----------|---------|-------------|
| `OPENCLI_DAEMON_PORT` | `19825` | HTTP port for the daemon-extension bridge |
| `OPENCLI_PROFILE` | — | Browser Bridge profile alias/contextId to use when multiple Chrome profiles are connected |
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open the automation container in the foreground (useful for debugging). The `--focus` flag sets this. |
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation lease open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open automation windows in the foreground (useful for debugging). The `--focus` flag sets this. |
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation window open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | Seconds to wait for browser connection |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | Seconds to wait for a single browser command |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
@@ -196,7 +177,7 @@ OpenCLI is not only for websites. It can also:
| `OPENCLI_DIAGNOSTIC` | `false` | Set to `1` to capture structured diagnostic context on failures |
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation lease open until you run `opencli browser close` or the idle timeout expires.
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation window open until you run `opencli browser close` or the idle timeout expires.
## Update
@@ -257,7 +238,7 @@ To load the source Browser Bridge extension:
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
| **uiverse** | `code` `preview` |
| **baidu-scholar** | `search` |
| **google-scholar** | `search` `cite` `profile` |
| **google-scholar** | `search` |
| **gov-law** | `search` `recent` |
| **gov-policy** | `search` `recent` |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
@@ -286,7 +267,7 @@ OpenCLI acts as a universal hub for your existing command-line tools — unified
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
```bash
opencli external register mycli
opencli register mycli
```
### Desktop App Adapters
@@ -409,7 +390,7 @@ See **[TESTING.md](./TESTING.md)** for how to run and write tests.
- **"Extension not connected"** — Ensure the Browser Bridge extension is installed from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) and **enabled** in `chrome://extensions`.
- **"attach failed: Cannot access a chrome-extension:// URL"** — Another extension may be interfering. Try disabling other extensions temporarily.
- **Empty data or 'Unauthorized' error** — Your Chrome/Chromium login session may have expired. Navigate to the target site and log in again.
- **Node API errors / missing `fetch` / startup crash on old Node** — OpenCLI requires **Node.js >= 21**. Run `node --version`, upgrade Node if needed, then retry.
- **Node API errors** — Ensure Node.js >= 21. Some features require `node:util` styleText (stable in Node 21+).
- **Daemon issues** — Check status: `curl localhost:19825/status` · View logs: `curl localhost:19825/logs`
## Star History
+4 -8
View File
@@ -31,10 +31,7 @@ OpenCLI 可以用同一套 CLI 做三类事情:
### 1. 安装 OpenCLI
OpenCLI 要求 **Node.js >= 21**
```bash
node --version
npm install -g @jackwener/opencli
```
@@ -158,8 +155,7 @@ OpenCLI 不只是网站 CLI,还可以:
## 前置要求
- **Node.js**: >= 21.0.0(标准 npm 安装路径要求)
- **Bun**: >= 1.0(可选替代运行时)
- **Node.js**: >= 21.0.0
- 浏览器型命令需要 Chrome 或 Chromium 处于运行中,并已登录目标网站
> **重要**:浏览器型命令直接复用你的 Chrome/Chromium 登录态。如果拿到空数据或出现权限类失败,先确认目标站点已经在浏览器里打开并完成登录。
@@ -243,7 +239,7 @@ npm link
| **uiverse** | `code` `preview` | 浏览器 |
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
| **baidu-scholar** | `search` | 公开 |
| **google-scholar** | `search` `cite` `profile` | 公开 |
| **google-scholar** | `search` | 公开 |
| **gov-law** | `search` `recent` | 公开 |
| **gov-policy** | `search` `recent` | 公开 |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
@@ -506,8 +502,8 @@ opencli plugin uninstall my-tool # 卸载
- 其他 Chrome/Chromium 扩展(如 youmind、New Tab Override 或 AI 助手类扩展)可能产生冲突。请尝试**暂时禁用其他扩展**后重试。
- **返回空数据,或者报错 "Unauthorized"**
- Chrome/Chromium 里的登录态可能已经过期。请打开当前页面,在新标签页重新手工登录或刷新该页面。
- **Node API 错误 / 缺少 `fetch` / 旧 Node 启动即崩**
- OpenCLI 要求 **Node.js >= 21**。先执行 `node --version`,如果版本过低先升级,再重试命令
- **Node API 错误 (如 parseArgs, fs 等)**
- 确保 Node.js 版本 `>= 21``node:util``styleText` 需要 Node 21+
- **Daemon 问题**
- 检查 daemon 状态:`curl localhost:19825/status`
- 查看扩展日志:`curl localhost:19825/logs`
+29 -347
View File
@@ -3275,7 +3275,7 @@
{
"site": "boss",
"name": "search",
"description": "BOSS直聘搜索职位(不带关键词时返回为你推荐职位)",
"description": "BOSS直聘搜索职位",
"domain": "www.zhipin.com",
"strategy": "cookie",
"browser": true,
@@ -3283,9 +3283,9 @@
{
"name": "query",
"type": "str",
"required": false,
"required": true,
"positional": true,
"help": "Search keyword (optional, empty = recommended jobs)"
"help": "Search keyword (e.g. AI agent, 前端)"
},
{
"name": "city",
@@ -3299,7 +3299,7 @@
"type": "str",
"default": "",
"required": false,
"help": "Experience: 在校生(实习)/应届生(校招)/经验不限/1年以内/1-3年/3-5年/5-10年/10年以上"
"help": "Experience: 应届/1年以内/1-3年/3-5年/5-10年/10年以上"
},
{
"name": "degree",
@@ -3322,13 +3322,6 @@
"required": false,
"help": "Industry code or name (e.g. 100020, 互联网)"
},
{
"name": "jobType",
"type": "str",
"default": "",
"required": false,
"help": "Job type: 全职/兼职/实习(不传=不限,混合校招与实习)"
},
{
"name": "page",
"type": "int",
@@ -3353,7 +3346,6 @@
"degree",
"skills",
"boss",
"bossOnline",
"security_id",
"url"
],
@@ -3540,8 +3532,9 @@
{
"name": "op",
"type": "str",
"default": "~/Pictures/chatgpt",
"required": false,
"help": "Output directory (default: ~/Pictures/chatgpt)"
"help": "Output directory"
},
{
"name": "sd",
@@ -4388,11 +4381,10 @@
"type": "str",
"default": "instant",
"required": false,
"help": "Model to use: instant, expert, or vision",
"help": "Model to use: instant or expert",
"choices": [
"instant",
"expert",
"vision"
"expert"
]
},
{
@@ -6855,65 +6847,6 @@
"sourceFile": "facebook/join-group.js",
"navigateBefore": "https://www.facebook.com"
},
{
"site": "facebook",
"name": "marketplace-inbox",
"description": "List recent Facebook Marketplace buyer/seller conversations",
"domain": "www.facebook.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "Number of conversations to return"
}
],
"columns": [
"index",
"buyer",
"listing",
"snippet",
"time",
"unread"
],
"type": "js",
"modulePath": "facebook/marketplace-inbox.js",
"sourceFile": "facebook/marketplace-inbox.js",
"navigateBefore": "https://www.facebook.com"
},
{
"site": "facebook",
"name": "marketplace-listings",
"description": "List your Facebook Marketplace seller listings",
"domain": "www.facebook.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "Number of listings to return"
}
],
"columns": [
"index",
"title",
"price",
"status",
"listed",
"clicks",
"actions"
],
"type": "js",
"modulePath": "facebook/marketplace-listings.js",
"sourceFile": "facebook/marketplace-listings.js",
"navigateBefore": "https://www.facebook.com"
},
{
"site": "facebook",
"name": "memories",
@@ -7458,86 +7391,6 @@
"modulePath": "google/trends.js",
"sourceFile": "google/trends.js"
},
{
"site": "google-scholar",
"name": "cite",
"description": "Get citation for a Google Scholar paper",
"domain": "scholar.google.com",
"strategy": "public",
"browser": true,
"args": [
{
"name": "query",
"type": "str",
"required": true,
"positional": true,
"help": "Paper title to search for"
},
{
"name": "style",
"type": "str",
"default": "bibtex",
"required": false,
"help": "Citation format",
"choices": [
"bibtex",
"endnote",
"refman",
"refworks"
]
},
{
"name": "index",
"type": "int",
"default": 1,
"required": false,
"help": "Which search result to cite (1-based)"
}
],
"columns": [
"title",
"format",
"citation"
],
"type": "js",
"modulePath": "google-scholar/cite.js",
"sourceFile": "google-scholar/cite.js",
"navigateBefore": false
},
{
"site": "google-scholar",
"name": "profile",
"description": "View a Google Scholar author profile",
"domain": "scholar.google.com",
"strategy": "public",
"browser": true,
"args": [
{
"name": "author",
"type": "str",
"required": true,
"positional": true,
"help": "Author name or Scholar user ID (e.g. JicYPdAAAAAJ)"
},
{
"name": "limit",
"type": "int",
"default": 10,
"required": false,
"help": "Max papers to show (max 20)"
}
],
"columns": [
"rank",
"title",
"cited",
"year"
],
"type": "js",
"modulePath": "google-scholar/profile.js",
"sourceFile": "google-scholar/profile.js",
"navigateBefore": false
},
{
"site": "google-scholar",
"name": "search",
@@ -9181,7 +9034,7 @@
{
"site": "jd",
"name": "item",
"description": "京东商品详情(价格、店铺、规格参数、主图、详情图",
"description": "京东商品详情(价格、店铺、规格参数、AVIF 图片",
"domain": "item.jd.com",
"strategy": "cookie",
"browser": true,
@@ -9196,9 +9049,9 @@
{
"name": "images",
"type": "int",
"default": 200,
"default": 10,
"required": false,
"help": "图片数量上限(默认200"
"help": "AVIF 图片数量上限(默认10"
}
],
"columns": [
@@ -9206,8 +9059,7 @@
"price",
"shop",
"specs",
"mainImages",
"detailImages"
"avifImages"
],
"type": "js",
"modulePath": "jd/item.js",
@@ -16085,7 +15937,7 @@
"name": "following",
"description": "Get accounts a Twitter/X user is following",
"domain": "x.com",
"strategy": "cookie",
"strategy": "intercept",
"browser": true,
"args": [
{
@@ -16112,7 +15964,7 @@
"type": "js",
"modulePath": "twitter/following.js",
"sourceFile": "twitter/following.js",
"navigateBefore": "https://x.com"
"navigateBefore": true
},
{
"site": "twitter",
@@ -16831,8 +16683,7 @@
],
"type": "js",
"modulePath": "uiverse/code.js",
"sourceFile": "uiverse/code.js",
"navigateBefore": "https://uiverse.io"
"sourceFile": "uiverse/code.js"
},
{
"site": "uiverse",
@@ -16872,8 +16723,7 @@
],
"type": "js",
"modulePath": "uiverse/preview.js",
"sourceFile": "uiverse/preview.js",
"navigateBefore": "https://uiverse.io"
"sourceFile": "uiverse/preview.js"
},
{
"site": "v2ex",
@@ -17250,42 +17100,6 @@
"required": false,
"help": "Seconds to wait after page load"
},
{
"name": "wait-for",
"type": "str",
"required": false,
"valueRequired": true,
"help": "CSS selector to wait for in the main document or same-origin iframes"
},
{
"name": "wait-until",
"type": "str",
"default": "domstable",
"required": false,
"help": "Readiness policy after navigation: domstable or networkidle",
"choices": [
"domstable",
"networkidle"
]
},
{
"name": "frames",
"type": "str",
"default": "same-origin",
"required": false,
"help": "Iframe handling mode: same-origin or none",
"choices": [
"same-origin",
"none"
]
},
{
"name": "diagnose",
"type": "boolean",
"default": false,
"required": false,
"help": "Print render diagnostics (frames, empty containers, XHR/API-like requests) to stderr"
},
{
"name": "stdout",
"type": "boolean",
@@ -20386,7 +20200,7 @@
"name": "answer",
"description": "Answer a Zhihu question",
"domain": "www.zhihu.com",
"strategy": "cookie",
"strategy": "ui",
"browser": true,
"args": [
{
@@ -20429,86 +20243,14 @@
"type": "js",
"modulePath": "zhihu/answer.js",
"sourceFile": "zhihu/answer.js",
"navigateBefore": "https://www.zhihu.com"
},
{
"site": "zhihu",
"name": "collection",
"description": "知乎收藏夹内容列表(需要登录)",
"domain": "www.zhihu.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "id",
"type": "str",
"required": true,
"positional": true,
"help": "收藏夹 ID (数字,可从收藏夹 URL 中获取)"
},
{
"name": "offset",
"type": "int",
"default": 0,
"required": false,
"help": "起始偏移量(用于分页)"
},
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "每页数量(最大 20"
}
],
"columns": [
"rank",
"type",
"title",
"author",
"votes",
"excerpt",
"url"
],
"type": "js",
"modulePath": "zhihu/collection.js",
"sourceFile": "zhihu/collection.js",
"navigateBefore": "https://www.zhihu.com"
},
{
"site": "zhihu",
"name": "collections",
"description": "知乎收藏夹列表(需要登录)",
"domain": "www.zhihu.com",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "limit",
"type": "int",
"default": 20,
"required": false,
"help": "每页数量(最大 20"
}
],
"columns": [
"rank",
"title",
"item_count",
"description",
"collection_id"
],
"type": "js",
"modulePath": "zhihu/collections.js",
"sourceFile": "zhihu/collections.js",
"navigateBefore": "https://www.zhihu.com"
"navigateBefore": true
},
{
"site": "zhihu",
"name": "comment",
"description": "Create a top-level comment on a Zhihu answer or article",
"domain": "zhihu.com",
"strategy": "cookie",
"strategy": "ui",
"browser": true,
"args": [
{
@@ -20545,12 +20287,13 @@
"target_type",
"target",
"author_identity",
"created_url"
"created_url",
"created_proof"
],
"type": "js",
"modulePath": "zhihu/comment.js",
"sourceFile": "zhihu/comment.js",
"navigateBefore": "https://zhihu.com"
"navigateBefore": true
},
{
"site": "zhihu",
@@ -20598,7 +20341,7 @@
"name": "favorite",
"description": "Favorite a Zhihu answer or article into a specific collection",
"domain": "zhihu.com",
"strategy": "cookie",
"strategy": "ui",
"browser": true,
"args": [
{
@@ -20639,14 +20382,14 @@
"type": "js",
"modulePath": "zhihu/favorite.js",
"sourceFile": "zhihu/favorite.js",
"navigateBefore": "https://zhihu.com"
"navigateBefore": true
},
{
"site": "zhihu",
"name": "follow",
"description": "Follow a Zhihu user or question",
"domain": "www.zhihu.com",
"strategy": "cookie",
"strategy": "ui",
"browser": true,
"args": [
{
@@ -20673,7 +20416,7 @@
"type": "js",
"modulePath": "zhihu/follow.js",
"sourceFile": "zhihu/follow.js",
"navigateBefore": "https://www.zhihu.com"
"navigateBefore": true
},
{
"site": "zhihu",
@@ -20707,7 +20450,7 @@
"name": "like",
"description": "Like a Zhihu answer or article",
"domain": "zhihu.com",
"strategy": "cookie",
"strategy": "ui",
"browser": true,
"args": [
{
@@ -20734,7 +20477,7 @@
"type": "js",
"modulePath": "zhihu/like.js",
"sourceFile": "zhihu/like.js",
"navigateBefore": "https://zhihu.com"
"navigateBefore": true
},
{
"site": "zhihu",
@@ -20806,67 +20549,6 @@
"sourceFile": "zhihu/search.js",
"navigateBefore": "https://www.zhihu.com"
},
{
"site": "zlibrary",
"name": "info",
"description": "Get book details and available download formats from a Z-Library book page",
"domain": "z-library.im",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "url",
"type": "str",
"required": true,
"positional": true,
"help": "Z-Library book page URL (e.g. https://z-library.im/book/...)"
}
],
"columns": [
"title",
"pdf",
"epub",
"url"
],
"type": "js",
"modulePath": "zlibrary/info.js",
"sourceFile": "zlibrary/info.js",
"navigateBefore": false
},
{
"site": "zlibrary",
"name": "search",
"description": "Search Z-Library for books by title, author, ISBN, or keyword",
"domain": "z-library.im",
"strategy": "cookie",
"browser": true,
"args": [
{
"name": "query",
"type": "str",
"required": true,
"positional": true,
"help": "Search keyword (title, author, ISBN, etc.)"
},
{
"name": "limit",
"type": "int",
"default": 10,
"required": false,
"help": "Max results (125)"
}
],
"columns": [
"rank",
"title",
"author",
"url"
],
"type": "js",
"modulePath": "zlibrary/search.js",
"sourceFile": "zlibrary/search.js",
"navigateBefore": false
},
{
"site": "zsxq",
"name": "dynamics",
@@ -21052,4 +20734,4 @@
"sourceFile": "zsxq/topics.js",
"navigateBefore": "https://wx.zsxq.com"
}
]
]
+1 -1
View File
@@ -12,7 +12,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles (max 50)' },
],
columns: ['rank', 'title', 'summary', 'date', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const count = Math.min(kwargs.limit || 20, 50);
const resp = await fetch('https://www.36kr.com/feed', {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; opencli/1.0)' },
+4 -4
View File
@@ -24,7 +24,7 @@ describe('apple-podcasts search command', () => {
}),
});
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({
const result = await cmd.func(null, {
query: 'machine learning',
keyword: 'sports',
limit: 5,
@@ -60,7 +60,7 @@ describe('apple-podcasts top command', () => {
}),
});
vi.stubGlobal('fetch', fetchMock);
await cmd.func({ country: 'US', limit: 1 });
await cmd.func(null, { country: 'US', limit: 1 });
const [, options] = fetchMock.mock.calls[0] ?? [];
expect(options).toBeDefined();
expect(options.signal).toBeDefined();
@@ -81,7 +81,7 @@ describe('apple-podcasts top command', () => {
}),
});
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({ country: 'US', limit: 2 });
const result = await cmd.func(null, { country: 'US', limit: 2 });
expect(fetchMock).toHaveBeenCalledWith('https://rss.marketingtools.apple.com/api/v2/us/podcasts/top/2/podcasts.json', expect.objectContaining({
signal: expect.any(Object),
}));
@@ -94,6 +94,6 @@ describe('apple-podcasts top command', () => {
const cmd = getRegistry().get('apple-podcasts/top');
expect(cmd?.func).toBeTypeOf('function');
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new Error('socket hang up')));
await expect(cmd.func({ country: 'us', limit: 3 })).rejects.toThrow('Unable to reach Apple Podcasts charts for US');
await expect(cmd.func(null, { country: 'us', limit: 3 })).rejects.toThrow('Unable to reach Apple Podcasts charts for US');
});
});
+1 -1
View File
@@ -12,7 +12,7 @@ cli({
{ name: 'limit', type: 'int', default: 15, help: 'Max episodes to show' },
],
columns: ['title', 'duration', 'date'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 200));
// results[0] is the podcast itself; the rest are episodes
const data = await itunesFetch(`/lookup?id=${args.id}&entity=podcastEpisode&limit=${limit + 1}`);
+1 -1
View File
@@ -12,7 +12,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: 'Max results' },
],
columns: ['id', 'title', 'author', 'episodes', 'genre', 'url'],
func: async (args) => {
func: async (_page, args) => {
const term = encodeURIComponent(args.query);
const limit = Math.max(1, Math.min(Number(args.limit), 25));
const data = await itunesFetch(`/search?term=${term}&media=podcast&limit=${limit}`);
+1 -1
View File
@@ -14,7 +14,7 @@ cli({
{ name: 'country', default: 'us', help: 'Country code (e.g. us, cn, gb, jp)' },
],
columns: ['rank', 'title', 'author', 'id'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 100));
const country = String(args.country || 'us').trim().toLowerCase();
const url = `${CHARTS_URL}/${country}/podcasts/top/${limit}/podcasts.json`;
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'id', positional: true, required: true, help: 'arXiv paper ID (e.g. 1706.03762)' },
],
columns: ['id', 'title', 'authors', 'published', 'abstract', 'url'],
func: async (args) => {
func: async (_page, args) => {
const xml = await arxivFetch(`id_list=${encodeURIComponent(args.id)}`);
const entries = parseEntries(xml);
if (!entries.length)
+1 -1
View File
@@ -12,7 +12,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: 'Max results (max 25)' },
],
columns: ['id', 'title', 'authors', 'published', 'url'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 25));
const query = encodeURIComponent(`all:${args.query}`);
const xml = await arxivFetch(`search_query=${query}&max_results=${limit}&sortBy=relevance`);
+1 -1
View File
@@ -12,7 +12,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Number of headlines (max 50)' },
],
columns: ['rank', 'title', 'description', 'url'],
func: async (kwargs) => {
func: async (page, kwargs) => {
const count = Math.min(kwargs.limit || 20, 50);
const resp = await fetch('https://feeds.bbci.co.uk/news/rss.xml');
if (!resp.ok)
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('businessweek', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('economics', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('industries', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('main', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('markets', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('opinions', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('politics', kwargs.limit ?? 1);
},
});
+1 -1
View File
@@ -11,7 +11,7 @@ cli({
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
],
columns: ['title', 'summary', 'link', 'mediaLinks'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
return fetchBloombergFeed('tech', kwargs.limit ?? 1);
},
});
+8 -49
View File
@@ -2,7 +2,6 @@
* BOSS直聘 job search — browser cookie API.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError } from '@jackwener/opencli/errors';
import { requirePage, navigateTo, bossFetch, verbose } from './utils.js';
/** City name → BOSS Zhipin city code mapping */
const CITY_CODES = {
@@ -23,16 +22,8 @@ const CITY_CODES = {
'香港': '101320100',
};
const EXP_MAP = {
'不限': '0',
'在校/应届': '108',
'在校生': '108', '在校': '108',
'应届生': '102', '应届': '102',
'经验不限': '101',
'1年以内': '103',
'1-3年': '104',
'3-5年': '105',
'5-10年': '106',
'10年以上': '107',
'不限': '0', '在校/应届': '108', '应届': '108', '1年以内': '101',
'1-3年': '102', '3-5年': '103', '5-10年': '104', '10年以上': '105',
};
const DEGREE_MAP = {
'不限': '0', '初中及以下': '209', '中专/中技': '208', '高中': '206',
@@ -47,10 +38,6 @@ const INDUSTRY_MAP = {
'人工智能': '100901', '大数据': '100902', '金融': '100101',
'教育培训': '100200', '医疗健康': '100300',
};
const JOB_TYPE_MAP = {
'不限': '0', '全职': '1901', '实习': '1902', '兼职': '1903',
};
const JOB_TYPE_CODES = new Set(Object.values(JOB_TYPE_MAP));
function resolveCity(input) {
if (!input)
return '101010100';
@@ -75,54 +62,35 @@ function resolveMap(input, map) {
}
return input;
}
function resolveJobType(input) {
if (!input)
return '';
if (JOB_TYPE_MAP[input] !== undefined)
return JOB_TYPE_MAP[input];
if (JOB_TYPE_CODES.has(input))
return input;
throw new ArgumentError(`Invalid jobType: ${input}`, 'Use one of: 全职, 兼职, 实习, 不限');
}
function formatBossOnline(value) {
if (value === true)
return 'Y';
if (value === false)
return 'N';
return '';
}
cli({
site: 'boss',
name: 'search',
description: 'BOSS直聘搜索职位(不带关键词时返回为你推荐职位)',
description: 'BOSS直聘搜索职位',
domain: 'www.zhipin.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
browser: true,
args: [
{ name: 'query', positional: true, help: 'Search keyword (optional, empty = recommended jobs)' },
{ name: 'query', required: true, positional: true, help: 'Search keyword (e.g. AI agent, 前端)' },
{ name: 'city', default: '北京', help: 'City name or code (e.g. 杭州, 上海, 101010100)' },
{ name: 'experience', default: '', help: 'Experience: 在校生(实习)/应届生(校招)/经验不限/1年以内/1-3年/3-5年/5-10年/10年以上' },
{ name: 'experience', default: '', help: 'Experience: 应届/1年以内/1-3年/3-5年/5-10年/10年以上' },
{ name: 'degree', default: '', help: 'Degree: 大专/本科/硕士/博士' },
{ name: 'salary', default: '', help: 'Salary: 3K以下/3-5K/5-10K/10-15K/15-20K/20-30K/30-50K/50K以上' },
{ name: 'industry', default: '', help: 'Industry code or name (e.g. 100020, 互联网)' },
{ name: 'jobType', default: '', help: 'Job type: 全职/兼职/实习(不传=不限,混合校招与实习)' },
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
{ name: 'limit', type: 'int', default: 15, help: 'Number of results' },
],
columns: ['name', 'salary', 'company', 'area', 'experience', 'degree', 'skills', 'boss', 'bossOnline', 'security_id', 'url'],
columns: ['name', 'salary', 'company', 'area', 'experience', 'degree', 'skills', 'boss', 'security_id', 'url'],
func: async (page, kwargs) => {
requirePage(page);
const query = String(kwargs.query ?? '').trim();
const cityCode = resolveCity(kwargs.city);
verbose('Navigating to set referrer context...');
await navigateTo(page, `https://www.zhipin.com/web/geek/job?query=${encodeURIComponent(query)}&city=${cityCode}`);
await navigateTo(page, `https://www.zhipin.com/web/geek/job?query=${encodeURIComponent(kwargs.query)}&city=${cityCode}`);
await new Promise(r => setTimeout(r, 1000));
const expVal = resolveMap(kwargs.experience, EXP_MAP);
const degreeVal = resolveMap(kwargs.degree, DEGREE_MAP);
const salaryVal = resolveMap(kwargs.salary, SALARY_MAP);
const industryVal = resolveMap(kwargs.industry, INDUSTRY_MAP);
const jobTypeVal = resolveJobType(kwargs.jobType);
const limit = kwargs.limit || 15;
let currentPage = kwargs.page || 1;
let allJobs = [];
@@ -133,7 +101,7 @@ cli({
}
const qs = new URLSearchParams({
scene: '1',
query,
query: kwargs.query,
city: cityCode,
page: String(currentPage),
pageSize: '15',
@@ -146,8 +114,6 @@ cli({
qs.set('salary', salaryVal);
if (industryVal)
qs.set('industry', industryVal);
if (jobTypeVal)
qs.set('jobType', jobTypeVal);
const targetUrl = `https://www.zhipin.com/wapi/zpgeek/search/joblist.json?${qs.toString()}`;
verbose(`Fetching page ${currentPage}... (current jobs: ${allJobs.length})`);
const data = await bossFetch(page, targetUrl);
@@ -169,7 +135,6 @@ cli({
degree: j.jobDegree,
skills: (j.skills || []).join(','),
boss: j.bossName + ' · ' + j.bossTitle,
bossOnline: formatBossOnline(j.bossOnline),
security_id: j.securityId || '',
url: 'https://www.zhipin.com/job_detail/' + j.encryptJobId + '.html',
});
@@ -188,9 +153,3 @@ cli({
return allJobs;
},
});
export const __test__ = {
EXP_MAP,
resolveMap,
resolveJobType,
formatBossOnline,
};
-78
View File
@@ -1,78 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError } from '@jackwener/opencli/errors';
import { __test__ } from './search.js';
import './search.js';
function createPageMock(response) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue(response),
};
}
describe('boss search', () => {
const command = getRegistry().get('boss/search');
it('keeps legacy 在校/应届 experience input compatible', () => {
expect(__test__.resolveMap('在校/应届', __test__.EXP_MAP)).toBe('108');
expect(__test__.resolveMap('应届', __test__.EXP_MAP)).toBe('102');
});
it('fails fast on invalid jobType values', async () => {
expect(() => __test__.resolveJobType('外包')).toThrow(ArgumentError);
});
it('accepts supported jobType labels and raw codes', () => {
expect(__test__.resolveJobType('全职')).toBe('1901');
expect(__test__.resolveJobType('实习')).toBe('1902');
expect(__test__.resolveJobType('兼职')).toBe('1903');
expect(__test__.resolveJobType('1902')).toBe('1902');
});
it('keeps empty query empty and sends jobType filter to the API', async () => {
const page = createPageMock({
code: 0,
zpData: {
hasMore: false,
jobList: [
{
encryptJobId: 'abc',
securityId: 'sec',
jobName: '前端开发实习生',
salaryDesc: '150-200/天',
brandName: 'OpenCLI',
cityName: '北京',
areaDistrict: '海淀区',
businessDistrict: '',
jobExperience: '在校/应届',
jobDegree: '本科',
skills: ['JavaScript'],
bossName: '张三',
bossTitle: '技术负责人',
bossOnline: false,
},
],
},
});
const rows = await command.func(page, {
query: undefined,
city: '北京',
jobType: '实习',
limit: 1,
page: 1,
});
expect(page.goto).toHaveBeenCalledWith('https://www.zhipin.com/web/geek/job?query=&city=101010100');
const fetchScript = page.evaluate.mock.calls.at(-1)[0];
expect(fetchScript).toContain('query=');
expect(fetchScript).not.toContain('query=undefined');
expect(fetchScript).toContain('jobType=1902');
expect(rows[0]).toMatchObject({
name: '前端开发实习生',
bossOnline: 'N',
});
});
});
+1 -1
View File
@@ -15,7 +15,7 @@ export const askCommand = cli({
{ name: 'timeout', required: false, help: 'Max seconds to wait for response (default: 30)', default: '30' },
],
columns: ['Role', 'Text'],
func: async (kwargs) => {
func: async (page, kwargs) => {
if (process.platform !== 'darwin') {
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
}
+2 -4
View File
@@ -156,7 +156,7 @@ guard s(input, kAXValueAttribute as String) == text else {
exit(1)
}
guard let sendButton = findByDescriptions(win, ["发送", "傳送", "Send"]) else {
guard let sendButton = findByDescriptions(win, ["发送", "Send"]) else {
fputs("Could not find send button\\n", stderr)
exit(1)
}
@@ -240,11 +240,10 @@ let args = CommandLine.arguments
let target = args.count > 1 ? args[1] : ""
let needsLegacy = args.count > 2 && args[2] == "legacy"
// Step 1: Click the "Options" button to open the popover (support English, Simplified and Traditional Chinese UI)
// Step 1: Click the "Options" button to open the popover (support both English and Chinese UI)
var optionsBtn: AXUIElement? = nil
if let btn = findByDesc(win, "Options") { optionsBtn = btn }
else if let btn = findByDesc(win, "选项") { optionsBtn = btn }
else if let btn = findByDesc(win, "選項") { optionsBtn = btn }
guard let options = optionsBtn else {
fputs("Could not find Options button\\n", stderr); exit(1)
}
@@ -380,6 +379,5 @@ export function getVisibleChatMessages() {
}
export const __test__ = {
AX_SEND_SCRIPT,
AX_MODEL_SCRIPT,
AX_GENERATING_SCRIPT,
};
-12
View File
@@ -13,18 +13,6 @@ describe('chatgpt-app AX send script', () => {
it('does not report success until the prompt leaves the composer after send', () => {
expect(__test__.AX_SEND_SCRIPT).toContain('Prompt did not leave input after pressing send');
});
it('supports english, zh-CN, and zh-TW send button labels', () => {
expect(__test__.AX_SEND_SCRIPT).toContain('["发送", "傳送", "Send"]');
});
});
describe('chatgpt-app AX model script', () => {
it('supports english, zh-CN, and zh-TW options button labels', () => {
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "Options")');
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "选项")');
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "選項")');
});
});
describe('chatgpt-app generating detection', () => {
+1 -1
View File
@@ -12,7 +12,7 @@ export const modelCommand = cli({
{ name: 'model', required: true, positional: true, help: 'Model to switch to', choices: MODEL_CHOICES },
],
columns: ['Status', 'Model'],
func: async (kwargs) => {
func: async (page, kwargs) => {
if (process.platform !== 'darwin') {
throw new ConfigError('ChatGPT Desktop integration requires macOS');
}
+1 -1
View File
@@ -10,7 +10,7 @@ export const newCommand = cli({
browser: false,
args: [],
columns: ['Status'],
func: async () => {
func: async (page) => {
if (process.platform !== 'darwin') {
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
}
+1 -1
View File
@@ -11,7 +11,7 @@ export const readCommand = cli({
browser: false,
args: [],
columns: ['Role', 'Text'],
func: async () => {
func: async (page) => {
if (process.platform !== 'darwin') {
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
}
+1 -1
View File
@@ -13,7 +13,7 @@ export const sendCommand = cli({
{ name: 'model', required: false, help: 'Model/mode to use: auto, instant, thinking, 5.2-instant, 5.2-thinking', choices: MODEL_CHOICES },
],
columns: ['Status'],
func: async (kwargs) => {
func: async (page, kwargs) => {
const text = kwargs.text;
const model = kwargs.model;
try {
+1 -1
View File
@@ -10,7 +10,7 @@ export const statusCommand = cli({
browser: false,
args: [],
columns: ['Status'],
func: async () => {
func: async (page) => {
if (process.platform !== 'darwin') {
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
}
+8 -37
View File
@@ -1,9 +1,7 @@
import * as os from 'node:os';
import * as path from 'node:path';
import * as fs from 'node:fs';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { saveBase64ToFile } from '@jackwener/opencli/utils';
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { getChatGPTVisibleImageUrls, sendChatGPTMessage, waitForChatGPTImages, getChatGPTImageAssets } from './utils.js';
const CHATGPT_DOMAIN = 'chatgpt.com';
@@ -26,22 +24,6 @@ function displayPath(filePath) {
return filePath.startsWith(home) ? `~${filePath.slice(home.length)}` : filePath;
}
export function resolveOutputDir(value) {
const raw = String(value || '').trim();
if (!raw) return path.join(os.homedir(), 'Pictures', 'chatgpt');
if (raw === '~') return os.homedir();
if (raw.startsWith('~/')) return path.join(os.homedir(), raw.slice(2));
return path.resolve(raw);
}
export function nextAvailablePath(dir, baseName, ext, existsSync = fs.existsSync) {
let candidate = path.join(dir, `${baseName}${ext}`);
for (let index = 1; existsSync(candidate); index += 1) {
candidate = path.join(dir, `${baseName}_${index}${ext}`);
}
return candidate;
}
async function currentChatGPTLink(page) {
const url = await page.evaluate('window.location.href').catch(() => '');
return typeof url === 'string' && url ? url : 'https://chatgpt.com';
@@ -59,13 +41,13 @@ export const imageCommand = cli({
timeoutSeconds: 240,
args: [
{ name: 'prompt', positional: true, required: true, help: 'Image prompt to send to ChatGPT' },
{ name: 'op', help: 'Output directory (default: ~/Pictures/chatgpt)' },
{ name: 'op', default: '~/Pictures/chatgpt', help: 'Output directory' },
{ name: 'sd', type: 'boolean', default: false, help: 'Skip download shorthand; only show ChatGPT link' },
],
columns: ['status', 'file', 'link'],
func: async (page, kwargs) => {
const prompt = kwargs.prompt;
const outputDir = resolveOutputDir(kwargs.op);
const outputDir = kwargs.op || path.join(os.homedir(), 'Pictures', 'chatgpt');
const skipDownloadRaw = kwargs.sd;
const skipDownload = skipDownloadRaw === '' || skipDownloadRaw === true || normalizeBooleanFlag(skipDownloadRaw);
const timeout = 120;
@@ -81,23 +63,12 @@ export const imageCommand = cli({
return [{ status: '⚠️ send-failed', file: '📁 -', link: `🔗 ${await currentChatGPTLink(page)}` }];
}
// ChatGPT briefly navigates to /c/{id} after sending, then may
// redirect back to the home page. Poll until we capture the /c/ URL.
let convUrl = '';
for (let ci = 0; ci < 10; ci++) {
const url = await currentChatGPTLink(page);
if (url.includes('/c/')) { convUrl = url; break; }
await page.wait(2);
}
if (!convUrl) {
convUrl = await currentChatGPTLink(page);
}
const urls = await waitForChatGPTImages(page, beforeUrls, timeout, convUrl);
const link = convUrl;
// Wait for response and images
const urls = await waitForChatGPTImages(page, beforeUrls, timeout);
const link = await currentChatGPTLink(page);
if (!urls.length) {
throw new EmptyResultError('chatgpt image', `No generated images were detected before timeout. Open ${link} and verify whether ChatGPT finished generating the image.`);
return [{ status: '⚠️ no-images', file: '📁 -', link: `🔗 ${link}` }];
}
if (skipDownload) {
@@ -107,7 +78,7 @@ export const imageCommand = cli({
// Export and save images
const assets = await getChatGPTImageAssets(page, urls);
if (!assets.length) {
throw new CommandExecutionError('Failed to export generated ChatGPT image assets', `Open ${link} and verify the generated images are visible, then retry.`);
return [{ status: '⚠️ export-failed', file: '📁 -', link: `🔗 ${link}` }];
}
const stamp = Date.now();
@@ -117,7 +88,7 @@ export const imageCommand = cli({
const base64 = asset.dataUrl.replace(/^data:[^;]+;base64,/, '');
const suffix = assets.length > 1 ? `_${index + 1}` : '';
const ext = extFromMime(asset.mimeType);
const filePath = nextAvailablePath(outputDir, `chatgpt_${stamp}${suffix}`, ext);
const filePath = path.join(outputDir, `chatgpt_${stamp}${suffix}${ext}`);
await saveBase64ToFile(base64, filePath);
results.push({ status: '✅ saved', file: `📁 ${displayPath(filePath)}`, link: `🔗 ${link}` });
}
-92
View File
@@ -1,92 +0,0 @@
import * as os from 'node:os';
import * as path from 'node:path';
import { beforeEach, describe, expect, it, vi } from 'vitest';
const mocks = vi.hoisted(() => ({
getChatGPTVisibleImageUrls: vi.fn(),
sendChatGPTMessage: vi.fn(),
waitForChatGPTImages: vi.fn(),
getChatGPTImageAssets: vi.fn(),
saveBase64ToFile: vi.fn(),
}));
vi.mock('./utils.js', () => ({
getChatGPTVisibleImageUrls: mocks.getChatGPTVisibleImageUrls,
sendChatGPTMessage: mocks.sendChatGPTMessage,
waitForChatGPTImages: mocks.waitForChatGPTImages,
getChatGPTImageAssets: mocks.getChatGPTImageAssets,
}));
vi.mock('@jackwener/opencli/utils', () => ({
saveBase64ToFile: mocks.saveBase64ToFile,
}));
const { imageCommand, nextAvailablePath, resolveOutputDir } = await import('./image.js');
function createPage() {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue('https://chatgpt.com/c/test-conversation'),
};
}
beforeEach(() => {
vi.restoreAllMocks();
mocks.getChatGPTVisibleImageUrls.mockReset().mockResolvedValue([]);
mocks.sendChatGPTMessage.mockReset().mockResolvedValue(true);
mocks.waitForChatGPTImages.mockReset().mockResolvedValue(['https://images.example/generated.png']);
mocks.getChatGPTImageAssets.mockReset().mockResolvedValue([{
url: 'https://images.example/generated.png',
dataUrl: 'data:image/png;base64,aGVsbG8=',
mimeType: 'image/png',
}]);
mocks.saveBase64ToFile.mockReset().mockResolvedValue(undefined);
});
describe('chatgpt image output paths', () => {
it('expands the default and explicit home-relative output directories', () => {
expect(resolveOutputDir()).toBe(path.join(os.homedir(), 'Pictures', 'chatgpt'));
expect(resolveOutputDir('~/tmp/chatgpt-images')).toBe(path.join(os.homedir(), 'tmp', 'chatgpt-images'));
expect(resolveOutputDir('~')).toBe(os.homedir());
});
it('generates a non-overwriting file path when a timestamp collision exists', () => {
const dir = '/tmp/chatgpt';
const taken = new Set([
path.join(dir, 'chatgpt_123.png'),
path.join(dir, 'chatgpt_123_1.png'),
]);
expect(nextAvailablePath(dir, 'chatgpt_123', '.png', (file) => taken.has(file))).toBe(path.join(dir, 'chatgpt_123_2.png'));
});
});
describe('chatgpt image failure contracts', () => {
it('fails fast when image generation detection finds no new images', async () => {
mocks.waitForChatGPTImages.mockResolvedValue([]);
await expect(imageCommand.func(createPage(), {
prompt: 'cat',
op: '',
sd: false,
})).rejects.toMatchObject({
code: 'EMPTY_RESULT',
message: expect.stringContaining('chatgpt image returned no data'),
hint: expect.stringContaining('No generated images were detected'),
});
});
it('fails fast when generated image assets cannot be exported', async () => {
mocks.getChatGPTImageAssets.mockResolvedValue([]);
await expect(imageCommand.func(createPage(), {
prompt: 'cat',
op: '',
sd: false,
})).rejects.toMatchObject({
code: 'COMMAND_EXEC',
message: expect.stringContaining('Failed to export generated ChatGPT image assets'),
});
});
});
+6 -39
View File
@@ -7,21 +7,11 @@ export const CHATGPT_DOMAIN = 'chatgpt.com';
export const CHATGPT_URL = 'https://chatgpt.com';
// Selectors
const COMPOSER_SELECTORS = [
'[aria-label="Chat with ChatGPT"]',
'[placeholder="Ask anything"]',
'#prompt-textarea',
];
const COMPOSER_SELECTOR = '[aria-label="Chat with ChatGPT"]';
const SEND_BTN_SELECTOR = 'button[aria-label="Send prompt"]';
function isSameChatGPTConversation(currentUrl, expectedUrl) {
if (!currentUrl || !expectedUrl) return false;
return currentUrl === expectedUrl
|| currentUrl.startsWith(`${expectedUrl}?`)
|| currentUrl.startsWith(`${expectedUrl}#`);
}
function buildComposerLocatorScript() {
const selectorsJson = JSON.stringify([COMPOSER_SELECTOR]);
const markerAttr = 'data-opencli-chatgpt-composer';
return `
const isVisible = (el) => {
@@ -43,7 +33,7 @@ function buildComposerLocatorScript() {
const marked = document.querySelector('[' + markerAttr + '="1"]');
if (marked instanceof HTMLElement && isVisible(marked)) return marked;
for (const selector of ${JSON.stringify(COMPOSER_SELECTORS)}) {
for (const selector of ${JSON.stringify([COMPOSER_SELECTOR])}) {
const node = Array.from(document.querySelectorAll(selector)).find(c => c instanceof HTMLElement && isVisible(c));
if (node instanceof HTMLElement) {
node.setAttribute(markerAttr, '1');
@@ -99,9 +89,7 @@ export async function sendChatGPTMessage(page, text) {
// Fallback: use execCommand
await page.evaluate(`
(() => {
var composer = null;
var sels = ${JSON.stringify(COMPOSER_SELECTORS)};
for (var si = 0; si < sels.length; si++) { composer = document.querySelector(sels[si]); if (composer) break; }
const composer = document.querySelector('[aria-label="Chat with ChatGPT"]');
if (!composer) return;
composer.focus();
document.execCommand('insertText', false, ${JSON.stringify(text)});
@@ -193,7 +181,7 @@ export async function getChatGPTVisibleImageUrls(page) {
/**
* Wait for new images to appear after sending a prompt.
*/
export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, convUrl) {
export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds) {
const beforeSet = new Set(beforeUrls);
const pollIntervalSeconds = 3;
const maxPolls = Math.max(1, Math.ceil(timeoutSeconds / pollIntervalSeconds));
@@ -203,26 +191,10 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
for (let i = 0; i < maxPolls; i++) {
await page.wait(i === 0 ? 3 : pollIntervalSeconds);
let currentUrl = '';
if (convUrl && convUrl.includes('/c/')) {
currentUrl = await page.evaluate('window.location.href').catch(() => '');
if (currentUrl && !isSameChatGPTConversation(currentUrl, convUrl)) {
await page.goto(convUrl);
await page.wait(3);
}
}
// Check if still generating
const generating = await isGenerating(page);
if (generating) continue;
if (convUrl && convUrl.includes('/c/') && i > 0 && i % 5 === 0) {
const onConversation = !currentUrl || isSameChatGPTConversation(currentUrl, convUrl);
if (onConversation) {
await page.goto(convUrl);
await page.wait(3);
}
}
const urls = (await getChatGPTVisibleImageUrls(page)).filter(url => !beforeSet.has(url));
if (urls.length === 0) continue;
@@ -242,11 +214,6 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
return lastUrls;
}
export const __test__ = {
COMPOSER_SELECTORS,
isSameChatGPTConversation,
};
/**
* Export images by URL: fetch from ChatGPT backend API and convert to base64 data URLs.
*/
-63
View File
@@ -1,63 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { __test__, waitForChatGPTImages } from './utils.js';
function createPageMock({ location = '', generating = [], imageUrls = [] } = {}) {
let generatingIndex = 0;
let imageIndex = 0;
return {
wait: vi.fn().mockResolvedValue(undefined),
goto: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn((script) => {
if (script === 'window.location.href') return Promise.resolve(location);
if (script.includes('Stop generating') || script.includes('Thinking')) {
const value = generating[Math.min(generatingIndex, generating.length - 1)] ?? false;
generatingIndex += 1;
return Promise.resolve(value);
}
if (script.includes("document.querySelectorAll('img')")) {
const value = imageUrls[Math.min(imageIndex, imageUrls.length - 1)] ?? [];
imageIndex += 1;
return Promise.resolve(value);
}
return Promise.resolve(undefined);
}),
};
}
describe('chatgpt image wait contract', () => {
it('does not periodically reload the conversation while generation is still active', async () => {
const convUrl = 'https://chatgpt.com/c/demo';
const page = createPageMock({
location: convUrl,
generating: [true, true, true, true, true, true],
});
await expect(waitForChatGPTImages(page, [], 18, convUrl)).resolves.toEqual([]);
expect(page.goto).not.toHaveBeenCalled();
});
it('jumps back to the captured conversation when the page drifts away', async () => {
const convUrl = 'https://chatgpt.com/c/demo';
const page = createPageMock({
location: 'https://chatgpt.com/',
generating: [false],
imageUrls: [['https://cdn.openai.com/generated/demo.png']],
});
await expect(waitForChatGPTImages(page, [], 3, convUrl)).resolves.toEqual([
'https://cdn.openai.com/generated/demo.png',
]);
expect(page.goto).toHaveBeenCalledWith(convUrl);
});
it('treats query and hash variants as the same conversation', () => {
expect(__test__.isSameChatGPTConversation(
'https://chatgpt.com/c/demo?model=gpt-image-1',
'https://chatgpt.com/c/demo',
)).toBe(true);
expect(__test__.isSameChatGPTConversation(
'https://chatgpt.com/c/other',
'https://chatgpt.com/c/demo',
)).toBe(false);
});
});
+1 -1
View File
@@ -34,7 +34,7 @@ cli({
{ name: 'limit', type: 'int', default: 15, help: 'Number of results' },
],
columns: ['rank', 'name', 'type', 'score', 'price', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const query = String(kwargs.query || '').trim();
if (!query) {
throw new ArgumentError('Search keyword cannot be empty');
+4 -4
View File
@@ -25,7 +25,7 @@ describe('ctrip search', () => {
],
},
}), { status: 200 })));
const result = await command.func({ query: '苏州', limit: 3 });
const result = await command.func(null, { query: '苏州', limit: 3 });
expect(result).toEqual([
{
rank: 1,
@@ -46,11 +46,11 @@ describe('ctrip search', () => {
]);
});
it('rejects empty queries', async () => {
await expect(command.func({ query: ' ', limit: 3 })).rejects.toThrow('Search keyword cannot be empty');
await expect(command.func(null, { query: ' ', limit: 3 })).rejects.toThrow('Search keyword cannot be empty');
});
it('surfaces fetch failures as CliError', async () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('{}', { status: 503 })));
await expect(command.func({ query: '苏州', limit: 3 })).rejects.toMatchObject({
await expect(command.func(null, { query: '苏州', limit: 3 })).rejects.toMatchObject({
code: 'FETCH_ERROR',
message: 'ctrip search failed with status 503',
});
@@ -59,6 +59,6 @@ describe('ctrip search', () => {
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify({
Response: { searchResults: [] },
}), { status: 200 })));
await expect(command.func({ query: '苏州', limit: 3 })).rejects.toThrow('ctrip search returned no data');
await expect(command.func(null, { query: '苏州', limit: 3 })).rejects.toThrow('ctrip search returned no data');
});
});
+4 -17
View File
@@ -18,7 +18,7 @@ export const askCommand = cli({
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
{ name: 'timeout', type: 'int', default: 120, help: 'Max seconds to wait for response' },
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
{ name: 'model', default: 'instant', choices: ['instant', 'expert', 'vision'], help: 'Model to use: instant, expert, or vision' },
{ name: 'model', default: 'instant', choices: ['instant', 'expert'], help: 'Model to use: instant or expert' },
{ name: 'think', type: 'boolean', default: false, help: 'Enable DeepThink mode' },
{ name: 'search', type: 'boolean', default: false, help: 'Enable web search' },
{ name: 'file', help: 'Attach a file (PDF, image, text) with the prompt' },
@@ -78,22 +78,9 @@ export const askCommand = cli({
throw new CommandExecutionError('Could not enable DeepThink');
}
if (wantModel === 'vision' && wantSearch) {
throw new CliError(
'ARGUMENT',
'DeepSeek vision mode does not support --search.',
'Run without --search, or use --model instant/expert for web search.',
EXIT_CODES.USAGE_ERROR,
);
}
// Vision mode does not have the search toggle.
let searchResult;
if (wantModel !== 'vision') {
searchResult = await withRetry(() => setFeature(page, 'Search', wantSearch));
if (!searchResult?.ok && wantSearch) {
throw new CommandExecutionError('Could not enable Search');
}
const searchResult = await withRetry(() => setFeature(page, 'Search', wantSearch));
if (!searchResult?.ok && wantSearch) {
throw new CommandExecutionError('Could not enable Search');
}
if (thinkResult?.toggled || searchResult?.toggled) await page.wait(0.5);
-46
View File
@@ -263,50 +263,4 @@ describe('deepseek ask conversation resume', () => {
expect(rows).toEqual([{ response: 'follow-up reply' }]);
expect(mockSelectModel).toHaveBeenCalled();
});
it('skips search toggle in vision mode when search is not requested', async () => {
mockEnsureOnDeepSeek.mockResolvedValue(false);
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
mockSetFeature.mockResolvedValue({ ok: true, toggled: false });
mockSendMessage.mockResolvedValue({ ok: true });
mockGetBubbleCount.mockResolvedValue(0);
mockWaitForResponse.mockResolvedValue('vision reply');
page.evaluate.mockResolvedValue('https://chat.deepseek.com/');
const rows = await askCommand.func(page, {
prompt: 'describe',
timeout: 120,
new: false,
model: 'vision',
think: false,
search: false,
});
expect(rows).toEqual([{ response: 'vision reply' }]);
expect(mockSetFeature).toHaveBeenCalledTimes(1);
expect(mockSetFeature).toHaveBeenCalledWith(expect.anything(), 'DeepThink', false);
});
it('fails fast instead of silently ignoring --search in vision mode', async () => {
mockEnsureOnDeepSeek.mockResolvedValue(false);
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
page.evaluate.mockResolvedValue('https://chat.deepseek.com/');
await expect(askCommand.func(page, {
prompt: 'describe',
timeout: 120,
new: false,
model: 'vision',
think: false,
search: true,
})).rejects.toMatchObject(new CliError(
'ARGUMENT',
'DeepSeek vision mode does not support --search.',
'Run without --search, or use --model instant/expert for web search.',
EXIT_CODES.USAGE_ERROR,
));
expect(mockSendMessage).not.toHaveBeenCalled();
expect(mockSendWithFile).not.toHaveBeenCalled();
});
});
+16 -55
View File
@@ -40,10 +40,9 @@ export async function selectModel(page, modelName) {
return page.evaluate(`(() => {
var radios = document.querySelectorAll('div[role="radio"]');
if (radios.length === 0) return { ok: false };
var name = '${modelName}'.toLowerCase();
var index = name === 'instant' ? 0 : name === 'expert' ? 1 : name === 'vision' ? 2 : -1;
if (index < 0 || index >= radios.length) return { ok: false };
var target = radios[index];
var isFirst = '${modelName}'.toLowerCase() === 'instant';
if (!isFirst && radios.length < 2) return { ok: false };
var target = isFirst ? radios[0] : radios[radios.length - 1];
var alreadySelected = target.getAttribute('aria-checked') === 'true';
if (!alreadySelected) target.click();
return { ok: true, toggled: !alreadySelected };
@@ -75,18 +74,14 @@ export async function sendMessage(page, prompt) {
document.execCommand('insertText', false, ${promptJson});
await new Promise(r => setTimeout(r, 800));
// Find the send button: last non-toggle button in the textarea's container
var container = box.parentElement;
while (container && !container.querySelector('div[role="button"]')) {
container = container.parentElement;
}
if (container) {
var btns = container.querySelectorAll('div[role="button"]:not(.ds-toggle-button)');
var sendBtn = btns[btns.length - 1];
if (sendBtn && sendBtn.getAttribute('aria-disabled') === 'false'
&& sendBtn.querySelectorAll('svg').length > 0) {
sendBtn.click();
return { ok: true };
const btns = document.querySelectorAll('div[role="button"]');
for (const btn of btns) {
if (btn.getAttribute('aria-disabled') === 'false') {
const svgs = btn.querySelectorAll('svg');
if (svgs.length > 0 && btn.closest('div')?.querySelector('textarea')) {
btn.click();
return { ok: true };
}
}
}
@@ -278,18 +273,9 @@ async function waitForFilePreview(page, fileName) {
for (let attempt = 0; attempt < 8; attempt++) {
await page.wait(2);
const ready = await page.evaluate(`(() => {
var name = ${JSON.stringify(fileName)};
var hasFileName = Array.from(document.querySelectorAll('div'))
.some(function(el) { return el.children.length === 0 && (el.textContent || '').trim() === name; });
if (hasFileName) return true;
// Vision mode shows an image thumbnail, not filename text. Require
// a preview-like node here; send-button readiness is checked later.
var box = document.querySelector('${TEXTAREA_SELECTOR}');
if (!box) return false;
var c = box.parentElement;
while (c && !c.querySelector('div[role="button"]')) c = c.parentElement;
if (!c) return false;
return !!c.querySelector('img[src], canvas, video, [style*="background-image"], [class*="preview"], [class*="upload"]');
const name = ${JSON.stringify(fileName)};
return Array.from(document.querySelectorAll('div'))
.some((el) => el.children.length === 0 && (el.textContent || '').trim() === name);
})()`);
if (ready) return true;
}
@@ -328,7 +314,7 @@ export async function sendWithFile(page, filePath, prompt) {
uploaded = true;
} catch (err) {
const msg = String(err?.message || err);
if (!msg.includes('Unknown action') && !msg.includes('not supported') && !msg.includes('Not allowed')) {
if (!msg.includes('Unknown action') && !msg.includes('not supported')) {
throw err;
}
}
@@ -355,8 +341,7 @@ export async function sendWithFile(page, filePath, prompt) {
}
inp.files = dt.files;
// Use inp.files, not dt.files; assignment transfers ownership
inp[propsKey].onChange({ target: { files: inp.files } });
inp[propsKey].onChange({ target: { files: dt.files } });
return { ok: true };
})()`);
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
@@ -365,30 +350,6 @@ export async function sendWithFile(page, filePath, prompt) {
const ready = await waitForFilePreview(page, fileName);
if (!ready) return { ok: false, reason: 'file preview did not appear' };
// File preview appears immediately but send button stays disabled until
// the server upload finishes. Wait for it.
let sendEnabled = false;
for (let tick = 0; tick < 15; tick++) {
const enabled = await page.evaluate(`(() => {
var box = document.querySelector('${TEXTAREA_SELECTOR}');
if (!box) return false;
var c = box.parentElement;
while (c && !c.querySelector('div[role="button"]')) c = c.parentElement;
if (!c) return false;
var btns = c.querySelectorAll('div[role="button"]:not(.ds-toggle-button)');
var last = btns[btns.length - 1];
return !!(last && last.getAttribute('aria-disabled') === 'false');
})()`);
if (enabled) {
sendEnabled = true;
break;
}
await page.wait(1);
}
if (!sendEnabled) {
return { ok: false, reason: 'send button did not enable after upload' };
}
return sendMessage(page, prompt);
}
+5 -124
View File
@@ -107,36 +107,15 @@ describe('deepseek sendWithFile', () => {
setFileInput: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce(undefined) // sidebar collapse
.mockResolvedValueOnce(true) // waitForFilePreview
.mockResolvedValueOnce(true) // send button enabled check
.mockResolvedValueOnce({ ok: true }), // sendMessage
.mockResolvedValueOnce(undefined)
.mockResolvedValueOnce(true)
.mockResolvedValueOnce({ ok: true }),
};
const result = await sendWithFile(page, filePath, 'summarize this');
expect(result).toEqual({ ok: true }); expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[type="file"]');
});
it('fails closed when upload preview appears but send button never enables', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'report.txt');
fs.writeFileSync(filePath, 'hello');
const page = {
setFileInput: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce(undefined) // sidebar collapse
.mockResolvedValueOnce(true) // waitForFilePreview
.mockResolvedValue(false), // send button never enables
};
const result = await sendWithFile(page, filePath, 'summarize this');
expect(result).toEqual({ ok: false, reason: 'send button did not enable after upload' });
expect(page.evaluate).toHaveBeenCalledTimes(17);
expect(result).toEqual({ ok: true });
expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[type="file"]');
});
});
@@ -163,102 +142,4 @@ describe('deepseek selectModel', () => {
expect(result).toEqual({ ok: false });
expect(instantRadio.click).not.toHaveBeenCalled();
});
it('selects the correct radio for each model', async () => {
const radios = [0, 1, 2].map(() => ({
getAttribute: vi.fn(() => 'false'),
click: vi.fn(),
}));
global.document = {
querySelectorAll: vi.fn(() => radios),
};
const page = {
evaluate: vi.fn(async (script) => eval(script)),
};
await selectModel(page, 'instant');
expect(radios[0].click).toHaveBeenCalled();
expect(radios[1].click).not.toHaveBeenCalled();
expect(radios[2].click).not.toHaveBeenCalled();
radios.forEach(r => r.click.mockClear());
await selectModel(page, 'expert');
expect(radios[1].click).toHaveBeenCalled();
radios.forEach(r => r.click.mockClear());
await selectModel(page, 'vision');
expect(radios[2].click).toHaveBeenCalled();
});
it('rejects unknown model names', async () => {
const radios = [0, 1, 2].map(() => ({
getAttribute: vi.fn(() => 'false'),
click: vi.fn(),
}));
global.document = {
querySelectorAll: vi.fn(() => radios),
};
const page = {
evaluate: vi.fn(async (script) => eval(script)),
};
const result = await selectModel(page, 'turbo');
expect(result).toEqual({ ok: false });
});
});
describe('deepseek sendWithFile Not allowed fallback', () => {
const tempDirs = [];
afterEach(() => {
vi.restoreAllMocks();
while (tempDirs.length) {
fs.rmSync(tempDirs.pop(), { recursive: true, force: true });
}
});
it('falls back to DataTransfer when setFileInput throws Not allowed', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'image.png');
fs.writeFileSync(filePath, 'fake-png');
const page = {
setFileInput: vi.fn().mockRejectedValue(new Error('Not allowed')),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce(undefined) // sidebar collapse
.mockResolvedValueOnce({ ok: true }) // DataTransfer fallback
.mockResolvedValueOnce(true) // waitForFilePreview
.mockResolvedValueOnce(true) // send button enabled
.mockResolvedValueOnce({ ok: true }),// sendMessage
};
const result = await sendWithFile(page, filePath, 'describe');
expect(page.setFileInput).toHaveBeenCalled();
expect(page.evaluate).toHaveBeenCalledTimes(5);
expect(result).toEqual({ ok: true });
});
it('does not treat send-button enablement alone as image upload proof', async () => {
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
tempDirs.push(dir);
const filePath = path.join(dir, 'image.png');
fs.writeFileSync(filePath, 'fake-png');
const page = {
setFileInput: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce(undefined) // sidebar collapse
.mockResolvedValue(false), // no filename / thumbnail preview
};
const result = await sendWithFile(page, filePath, 'describe');
expect(result).toEqual({ ok: false, reason: 'file preview did not appear' });
expect(page.evaluate.mock.calls[1][0]).toContain('img[src], canvas, video');
expect(page.evaluate.mock.calls[1][0]).not.toContain("aria-disabled') === 'false'");
});
});
+11 -43
View File
@@ -63,7 +63,7 @@ function getTranscriptLinesScript() {
const stopLines = new Set([
'豆包',
'新对话',
'内容由豆包 AI 生成,请仔细甄别',
'内容由豆包 AI 生成',
'AI 创作',
'云盘',
'更多',
@@ -75,8 +75,6 @@ function getTranscriptLinesScript() {
'PPT 生成',
'图像生成',
'帮我写作',
'请仔细甄别',
'下载电脑版',
]);
const noisyPatterns = [
@@ -90,7 +88,7 @@ function getTranscriptLinesScript() {
const transcriptText = clean(root.innerText || root.textContent || '')
.replace(/新对话/g, '\\n')
.replace(/内容由豆包 AI 生成,请仔细甄别/g, '\\n')
.replace(/内容由豆包 AI 生成/g, '\\n')
.replace(/在此处拖放文件/g, '\\n')
.replace(/文件数量:[^\\n]*/g, '')
.replace(/文件类型:[^\\n]*/g, '');
@@ -146,20 +144,12 @@ function getTurnsScript() {
if (
root.matches('[data-testid="send_message"], [class*="send-message"]')
|| root.querySelector('[data-testid="send_message"], [class*="send-message"]')
|| root.matches('[class*="bg-g-send-msg-bubble"]')
||
root.querySelector('[class*="bg-g-send-msg-bubble"]')
|| root.querySelector('[data-foundation-type="send-message-action-bar"]')
) {
return 'User';
}
if (
root.matches('[data-testid="receive_message"], [data-testid*="receive_message"], [class*="receive-message"]')
|| root.querySelector('[data-testid="receive_message"], [data-testid*="receive_message"], [class*="receive-message"]')
|| root.matches('[class*="bg-g-receive-msg-bubble"]')
||
root.querySelector('[class*="bg-g-receive-msg-bubble"]')
|| root.querySelector('[data-foundation-type="receive-message-action-bar"]')
) {
return 'Assistant';
}
@@ -173,10 +163,6 @@ function getTurnsScript() {
'[data-testid*="message_content"]',
'[class*="message-text"]',
'[class*="message-content"]',
'[class*="bg-g-send-msg-bubble"]',
'[class*="bg-g-receive-msg-bubble"]',
'.flow-markdown-body',
'[class*="bubble"]',
];
const messageImageSelector = messageTextSelectors.map((s) => s + ' img').join(', ');
@@ -219,30 +205,14 @@ function getTurnsScript() {
return text ? text + '\\n' + imageLines.join('\\n') : imageLines.join('\\n');
};
const messageList = document.querySelector('[class*="message-list-S2Fv2S"], .container-PvPoAn, .scroll-view-OEiNXD, [data-testid="message-list"]');
const messageList = document.querySelector('[data-testid="message-list"]');
if (!messageList) return [];
const itemSelectors = [
'[class*="item-kDun2N"]',
'[data-testid="union_message"]',
'[data-testid="message-block-container"]',
'[data-message-id]',
'[class*="bg-g-send-msg-bubble"]',
'[class*="bg-g-receive-msg-bubble"]',
];
const allRoots = [];
const seen = new Set();
for (const sel of itemSelectors) {
messageList.querySelectorAll(sel).forEach((el) => {
if (!seen.has(el)) {
seen.add(el);
allRoots.push(el);
}
});
}
const roots = allRoots
.filter((el) => isVisible(el) && !el.closest('script, style, noscript'))
const unionRoots = Array.from(messageList.querySelectorAll('[data-testid="union_message"]'))
.filter((el) => isVisible(el));
const blockRoots = Array.from(messageList.querySelectorAll('[data-testid="message-block-container"]'))
.filter((el) => isVisible(el) && !el.closest('[data-testid="union_message"]'));
const roots = (unionRoots.length > 0 ? unionRoots : blockRoots)
.filter((el, index, items) => !items.some((other, otherIndex) => otherIndex !== index && other.contains(el)));
const turns = roots
@@ -260,11 +230,11 @@ function getTurnsScript() {
});
const deduped = [];
const dedupedSeen = new Set();
const seen = new Set();
for (const turn of turns) {
const key = turn.role + '::' + turn.text;
if (dedupedSeen.has(key)) continue;
dedupedSeen.add(key);
if (seen.has(key)) continue;
seen.add(key);
deduped.push({ Role: turn.role, Text: turn.text });
}
@@ -1116,8 +1086,6 @@ export const __test__ = {
clickSendButtonScript,
composerStateScript,
detectDoubaoVerificationScript,
getTurnsScript,
getTranscriptLinesScript,
};
export async function startNewDoubaoChat(page) {
await ensureDoubaoChatPage(page);
-19
View File
@@ -144,25 +144,6 @@ describe('doubao send strategy', () => {
await expect(sendDoubaoMessage(page, '你好')).rejects.toBeInstanceOf(CommandExecutionError);
});
});
describe('doubao receive strategy', () => {
it('keeps both the new skin selectors and the older structural fallbacks in the turns script', () => {
const turnsScript = __test__.getTurnsScript();
expect(turnsScript).toContain('[class*="message-list-S2Fv2S"]');
expect(turnsScript).toContain('.container-PvPoAn');
expect(turnsScript).toContain('[data-testid="message-list"]');
expect(turnsScript).toContain('[class*="bg-g-receive-msg-bubble"]');
expect(turnsScript).toContain('[data-testid="receive_message"]');
expect(turnsScript).toContain('[data-foundation-type="receive-message-action-bar"]');
expect(turnsScript).toContain('[data-testid="union_message"]');
expect(turnsScript).toContain('[data-testid="message-block-container"]');
});
it('extends transcript-noise cleanup for the current zh-CN chrome copy', () => {
const transcriptScript = __test__.getTranscriptLinesScript();
expect(transcriptScript).toContain('请仔细甄别');
expect(transcriptScript).toContain('下载电脑版');
});
});
describe('collectDoubaoTranscriptAdditions', () => {
it('ignores landing-page capability chips that are not assistant content', () => {
const before = ['older'];
+1 -1
View File
@@ -18,7 +18,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['time', 'code', 'name', 'title', 'category', 'url'],
func: async (args) => {
func: async (_page, args) => {
const market = String(args.market ?? 'SHA,SZA,BJA').trim() || 'SHA,SZA,BJA';
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
+1 -1
View File
@@ -28,7 +28,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['rank', 'bondCode', 'bondName', 'bondPrice', 'bondChangePct', 'stockCode', 'stockName', 'stockPrice', 'stockChangePct', 'convPrice', 'convValue', 'convPremiumPct', 'remainingYears', 'ytm', 'listDate'],
func: async (args) => {
func: async (_page, args) => {
const sortKey = String(args.sort ?? 'turnover').toLowerCase();
const sort = SORTS[sortKey];
if (!sort) throw new CliError('INVALID_ARGUMENT', `Unknown sort "${sortKey}". Valid: ${Object.keys(SORTS).join(', ')}`);
+1 -1
View File
@@ -26,7 +26,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'change', 'turnover', 'volume', 'turnoverRate'],
func: async (args) => {
func: async (_page, args) => {
const sortKey = String(args.sort ?? 'turnover').toLowerCase();
const sort = SORTS[sortKey];
if (!sort) throw new CliError('INVALID_ARGUMENT', `Unknown sort "${sortKey}". Valid: ${Object.keys(SORTS).join(', ')}`);
+1 -1
View File
@@ -37,7 +37,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: '返回股东数(默认十大流通股东)' },
],
columns: ['rank', 'reportDate', 'name', 'holdNum', 'floatRatio', 'change'],
func: async (args) => {
func: async (_page, args) => {
/** @type {string} */
let secucode;
try { secucode = toSecucode(args.symbol); }
+1 -1
View File
@@ -47,7 +47,7 @@ cli({
},
],
columns: ['code', 'name', 'price', 'changePercent', 'change', 'open', 'high', 'low', 'prevClose'],
func: async (args) => {
func: async (_page, args) => {
const group = String(args.group ?? 'main').toLowerCase();
/** @type {[string,string][]} */
let entries;
+1 -1
View File
@@ -36,7 +36,7 @@ cli({
{ name: 'limit', type: 'int', default: 30, help: '返回最近 N 根(末尾)' },
],
columns: ['date', 'open', 'close', 'high', 'low', 'volume', 'turnover', 'amplitude', 'changePercent', 'change', 'turnoverRate'],
func: async (args) => {
func: async (_page, args) => {
const secid = resolveSecid(args.symbol);
const periodKey = String(args.period ?? 'day').toLowerCase();
const klt = PERIOD_MAP[periodKey];
+1 -1
View File
@@ -26,7 +26,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['time', 'title', 'summary', 'stocks'],
func: async (args) => {
func: async (_page, args) => {
const column = String(args.column ?? '102').trim();
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
+1 -1
View File
@@ -26,7 +26,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['tradeDate', 'code', 'name', 'closePrice', 'changeRate', 'boardAmt', 'buyAmt', 'sellAmt', 'netAmt', 'turnover', 'dealRatio', 'market', 'reason'],
func: async (args) => {
func: async (_page, args) => {
const sinceDate = String(args.date || '').trim() || defaultTradeDate();
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
+1 -1
View File
@@ -28,7 +28,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'mainNet', 'mainNetRatio', 'superNet', 'bigNet', 'mediumNet', 'smallNet'],
func: async (args) => {
func: async (_page, args) => {
const rangeKey = String(args.range ?? 'today').toLowerCase();
const range = RANGES[rangeKey];
if (!range) {
+1 -1
View File
@@ -19,7 +19,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: '返回最近 N 分钟' },
],
columns: ['time', 'cumulativeNetYi', 'minuteNetYi', 'totalNetYi'],
func: async (args) => {
func: async (_page, args) => {
const dir = String(args.direction ?? 'north').toLowerCase();
if (!['north', 'south', 'n', 's'].includes(dir)) {
throw new CliError('INVALID_ARGUMENT', `Unknown direction "${dir}". Valid: north / south`);
+1 -1
View File
@@ -57,7 +57,7 @@ cli({
'turnoverRate', 'amplitude', 'peDynamic', 'priceBook',
'marketCap', 'floatMarketCap',
],
func: async (args) => {
func: async (_page, args) => {
const raw = splitSymbols(args.symbols);
if (raw.length === 0) {
throw new CliError('INVALID_ARGUMENT', 'At least one symbol is required');
+1 -1
View File
@@ -43,7 +43,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'change', 'turnover', 'volume', 'turnoverRate', 'peDynamic', 'marketCap'],
func: async (args) => {
func: async (_page, args) => {
const market = String(args.market ?? 'hs-a').toLowerCase();
const sortKey = String(args.sort ?? 'change').toLowerCase();
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
+1 -1
View File
@@ -33,7 +33,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
],
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'mainNet', 'leadStock', 'leadChangePercent', 'upCount', 'downCount'],
func: async (args) => {
func: async (_page, args) => {
const typeKey = String(args.type ?? 'industry').toLowerCase();
const fs = SECTOR_TYPES[typeKey];
if (!fs) throw new CliError('INVALID_ARGUMENT', `Unknown sector type "${typeKey}". Valid: ${Object.keys(SECTOR_TYPES).join(', ')}`);
-83
View File
@@ -1,83 +0,0 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
function normalizeLimit(value) {
const limit = Number(value ?? 20);
if (!Number.isInteger(limit) || limit <= 0) {
throw new ArgumentError('facebook marketplace-inbox --limit must be a positive integer');
}
return Math.min(limit, 100);
}
cli({
site: 'facebook',
name: 'marketplace-inbox',
description: 'List recent Facebook Marketplace buyer/seller conversations',
domain: 'www.facebook.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of conversations to return' },
],
columns: ['index', 'buyer', 'listing', 'snippet', 'time', 'unread'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for facebook marketplace-inbox');
const limit = normalizeLimit(args.limit);
await page.goto('https://www.facebook.com/marketplace/inbox/');
await page.wait(4);
const result = await page.evaluate(String.raw`(() => {
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
const timeRe = /^(?:\d{1,2}:\d{2}\s?(?:AM|PM|am|pm|上午|下午)?|Mon|Tue|Wed|Thu|Fri|Sat|Sun|Today|Yesterday|\d+[mhdw]|\d+\s*(?:min|h|d|w))$/;
const text = document.body?.innerText || '';
if (/log in|sign in/i.test(text) && !/Marketplace/i.test(text)) {
return { authRequired: true, rows: [] };
}
const lines = text.split(/\n+/).map(clean).filter(Boolean);
const out = [];
const seen = new Set();
const skipBuyer = /^(Marketplace|Browse all|Notifications|Inbox|Marketplace access|Buying|Selling|Create new listing|Create multiple listings|Location|Categories|Vehicles|Property Rentals|All|Pending payment|Paid|To be shipped|Shipped|Cash on delivery|Completed|Filter by label)$/i;
for (let i = 0; i < lines.length - 2; i += 1) {
const buyer = lines[i];
const meta = lines[i + 1];
if (skipBuyer.test(buyer) || !/^·\s+/.test(meta)) continue;
const listing = meta.replace(/^·\s*/, '');
if (!listing || /^Within\b/i.test(listing)) continue;
const snippet = lines[i + 2] || '';
const time = timeRe.test(lines[i + 3] || '') ? lines[i + 3] : '';
const key = buyer + '|' + listing;
if (seen.has(key)) continue;
seen.add(key);
const nearby = lines.slice(Math.max(0, i - 2), i + 5).join(' ');
out.push({
buyer,
listing,
snippet,
time,
unread: /Unread/i.test(nearby),
});
}
return { authRequired: false, rows: out };
})()`);
if (result?.authRequired) {
throw new AuthRequiredError('facebook.com', 'Facebook Marketplace inbox requires an active signed-in Facebook session.');
}
const items = Array.isArray(result?.rows) ? result.rows : [];
if (items.length === 0) {
throw new EmptyResultError('facebook marketplace-inbox', 'No Marketplace inbox conversations were visible. Check that Marketplace inbox is available for this account.');
}
return items.slice(0, limit).map((item, index) => ({
index: index + 1,
buyer: item.buyer || '',
listing: item.listing || '',
snippet: item.snippet || '',
time: item.time || '',
unread: Boolean(item.unread),
}));
},
});
export const __test__ = {
normalizeLimit,
};
-83
View File
@@ -1,83 +0,0 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
function normalizeLimit(value) {
const limit = Number(value ?? 20);
if (!Number.isInteger(limit) || limit <= 0) {
throw new ArgumentError('facebook marketplace-listings --limit must be a positive integer');
}
return Math.min(limit, 100);
}
cli({
site: 'facebook',
name: 'marketplace-listings',
description: 'List your Facebook Marketplace seller listings',
domain: 'www.facebook.com',
strategy: Strategy.COOKIE,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of listings to return' },
],
columns: ['index', 'title', 'price', 'status', 'listed', 'clicks', 'actions'],
func: async (page, args) => {
if (!page) throw new CommandExecutionError('Browser session required for facebook marketplace-listings');
const limit = normalizeLimit(args.limit);
await page.goto('https://www.facebook.com/marketplace/you/selling/');
await page.wait(4);
const result = await page.evaluate(String.raw`(() => {
const clean = (s) => String(s || '').replace(/\u00a0/g, ' ').replace(/\s+/g, ' ').trim();
const allText = document.body?.innerText || '';
if (/log in|sign in/i.test(allText) && !/Marketplace/i.test(allText)) {
return { authRequired: true, rows: [] };
}
const lines = allText.split(/\n+/).map(clean).filter(Boolean);
const seen = new Set();
const out = [];
for (let i = 1; i < lines.length; i += 1) {
if (!/^(?:CA\$|\$)\s*\d+/.test(lines[i])) continue;
const title = lines[i - 1];
if (!title || /^(Hide|All listings|Needs attention|Marketplace|Selling)$/i.test(title)) continue;
if (seen.has(title)) continue;
seen.add(title);
const windowLines = lines.slice(i, i + 12);
const status = windowLines.find((line) => /^(Active|Sold|Pending|Draft)$/i.test(line)) || '';
const listed = windowLines.find((line) => /Listed on\b/i.test(line))?.replace(/^·\s*/, '') || '';
const clickLine = windowLines.find((line) => /clicks? on listing/i.test(line)) || '';
const clickMatch = clickLine.match(/([\d,.]+)\s+clicks? on listing/i);
const actions = windowLines.filter((line) => /^(Mark as sold|Mark as available|Relist this item|Share|Boost listing)$/i.test(line));
out.push({
title,
price: lines[i],
status,
listed,
clicks: clickMatch ? clickMatch[1] : '',
actions,
});
}
return { authRequired: false, rows: out };
})()`);
if (result?.authRequired) {
throw new AuthRequiredError('facebook.com', 'Facebook Marketplace seller listings require an active signed-in Facebook session.');
}
const items = Array.isArray(result?.rows) ? result.rows : [];
if (items.length === 0) {
throw new EmptyResultError('facebook marketplace-listings', 'No seller listings were visible. Check that Marketplace selling is available for this account.');
}
return items.slice(0, limit).map((item, index) => ({
index: index + 1,
title: item.title || '',
price: item.price || '',
status: item.status || '',
listed: item.listed || '',
clicks: item.clicks || '',
actions: Array.isArray(item.actions) ? item.actions.join(', ') : String(item.actions || ''),
}));
},
});
export const __test__ = {
normalizeLimit,
};
-91
View File
@@ -1,91 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, EmptyResultError } from '@jackwener/opencli/errors';
import './marketplace-listings.js';
import './marketplace-inbox.js';
function makePage(overrides = {}) {
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue([]),
...overrides,
};
}
describe('facebook marketplace read commands', () => {
it('marketplace-listings navigates to selling page and returns limited listing rows', async () => {
const command = getRegistry().get('facebook/marketplace-listings');
expect(command).toBeDefined();
const page = makePage({
evaluate: vi.fn().mockResolvedValue({
authRequired: false,
rows: [
{ title: 'Black electric standing desk', price: 'CA$80', status: 'Active', listed: 'Listed on 4/26', clicks: '87', actions: ['Mark as sold', 'Share'] },
{ title: 'Large gray corduroy beanbag chair', price: 'CA$30', status: 'Sold', listed: 'Listed on 4/26', clicks: '52', actions: ['Mark as available', 'Relist this item'] },
],
}),
});
const rows = await command.func(page, { limit: 1 });
expect(page.goto).toHaveBeenCalledWith('https://www.facebook.com/marketplace/you/selling/');
expect(page.wait).toHaveBeenCalledWith(4);
expect(rows).toEqual([
{
index: 1,
title: 'Black electric standing desk',
price: 'CA$80',
status: 'Active',
listed: 'Listed on 4/26',
clicks: '87',
actions: 'Mark as sold, Share',
},
]);
});
it('marketplace-inbox navigates to inbox and returns recent buyer conversations', async () => {
const command = getRegistry().get('facebook/marketplace-inbox');
expect(command).toBeDefined();
const page = makePage({
evaluate: vi.fn().mockResolvedValue({
authRequired: false,
rows: [
{ buyer: 'Kulwant', listing: 'White 3-tier rolling utility cart', snippet: 'Can I pick up today?', time: '3:43 PM', unread: true },
{ buyer: 'Gabriel', listing: 'Black electric standing desk', snippet: 'Yes, still available.', time: '12:17 PM', unread: false },
],
}),
});
const rows = await command.func(page, { limit: 2 });
expect(page.goto).toHaveBeenCalledWith('https://www.facebook.com/marketplace/inbox/');
expect(page.wait).toHaveBeenCalledWith(4);
expect(rows).toEqual([
{ index: 1, buyer: 'Kulwant', listing: 'White 3-tier rolling utility cart', snippet: 'Can I pick up today?', time: '3:43 PM', unread: true },
{ index: 2, buyer: 'Gabriel', listing: 'Black electric standing desk', snippet: 'Yes, still available.', time: '12:17 PM', unread: false },
]);
});
it('throws EmptyResultError when Marketplace returns no inbox rows', async () => {
const command = getRegistry().get('facebook/marketplace-inbox');
const page = makePage({ evaluate: vi.fn().mockResolvedValue({ authRequired: false, rows: [] }) });
await expect(command.func(page, { limit: 5 })).rejects.toThrow(EmptyResultError);
});
it('throws AuthRequiredError when Marketplace returns a login page', async () => {
const command = getRegistry().get('facebook/marketplace-listings');
const page = makePage({ evaluate: vi.fn().mockResolvedValue({ authRequired: true, rows: [] }) });
await expect(command.func(page, { limit: 5 })).rejects.toThrow(AuthRequiredError);
});
it('throws ArgumentError for invalid limits', async () => {
const command = getRegistry().get('facebook/marketplace-listings');
const page = makePage();
await expect(command.func(page, { limit: 0 })).rejects.toThrow(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
});
-74
View File
@@ -1,74 +0,0 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { requireNonEmptyQuery } from '../_shared/common.js';
cli({
site: 'google-scholar',
name: 'cite',
description: 'Get citation for a Google Scholar paper',
domain: 'scholar.google.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'query', positional: true, required: true, help: 'Paper title to search for' },
{ name: 'style', default: 'bibtex', choices: ['bibtex', 'endnote', 'refman', 'refworks'], help: 'Citation format' },
{ name: 'index', type: 'int', default: 1, help: 'Which search result to cite (1-based)' },
],
columns: ['title', 'format', 'citation'],
navigateBefore: false,
func: async (page, kwargs) => {
const query = requireNonEmptyQuery(kwargs.query);
const format = kwargs.style || 'bibtex';
const index = Math.max(1, kwargs.index || 1) - 1;
await page.goto(`https://scholar.google.com/scholar?q=${encodeURIComponent(query)}&hl=en`);
await page.wait(3);
const clicked = await page.evaluate(`(() => {
var cites = document.querySelectorAll('a.gs_or_cit');
if (cites.length <= ${index}) return { ok: false, reason: 'result not found at index ${index + 1}' };
var titleEl = document.querySelectorAll('.gs_r.gs_or.gs_scl')[${index}];
var title = '';
if (titleEl) {
var t = titleEl.querySelector('.gs_rt a, h3 a');
title = t ? t.textContent.trim() : '';
}
cites[${index}].click();
return { ok: true, title: title };
})()`);
if (!clicked?.ok) {
throw new CommandExecutionError(clicked?.reason || `Could not find search result at index ${index + 1}`);
}
await page.wait(2);
const formatMap = { bibtex: 'BibTeX', endnote: 'EndNote', refman: 'RefMan', refworks: 'RefWorks' };
const formatLabel = formatMap[format] || 'BibTeX';
const citeUrl = await page.evaluate(`(() => {
var links = document.querySelectorAll('#gs_cit a.gs_citi');
for (var i = 0; i < links.length; i++) {
if (links[i].textContent.trim() === '${formatLabel}') return links[i].href;
}
return null;
})()`);
if (!citeUrl) {
throw new CommandExecutionError(`Could not find ${formatLabel} citation link for result ${index + 1}`);
}
await page.goto(citeUrl);
await page.wait(2);
const citation = await page.evaluate(`(() => {
return (document.body.innerText || '').trim();
})()`);
if (!citation) {
throw new CommandExecutionError(`${formatLabel} citation page returned an empty response`);
}
return [{ title: clicked.title, format: format, citation }];
},
});
-47
View File
@@ -1,47 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './cite.js';
describe('google-scholar cite command', () => {
const command = getRegistry().get('google-scholar/cite');
it('registers as a public browser command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('google-scholar');
expect(command.strategy).toBe('public');
expect(command.browser).toBe(true);
});
it('rejects empty queries before browser navigation', async () => {
const page = { goto: vi.fn() };
await expect(command.func(page, { query: ' ' })).rejects.toMatchObject({
name: 'ArgumentError',
code: 'ARGUMENT',
});
expect(page.goto).not.toHaveBeenCalled();
});
it('throws when the requested search result index does not exist', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValueOnce({ ok: false, reason: 'result not found at index 2' }),
};
await expect(command.func(page, { query: 'test', index: 2 })).rejects.toThrow(CommandExecutionError);
});
it('looks up the requested citation style instead of only locking BibTeX', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn()
.mockResolvedValueOnce({ ok: true, title: 'Paper Title' })
.mockResolvedValueOnce('https://example.com/refworks')
.mockResolvedValueOnce('RefWorks citation body'),
};
const result = await command.func(page, { query: 'test', style: 'refworks' });
expect(result).toEqual([{ title: 'Paper Title', format: 'refworks', citation: 'RefWorks citation body' }]);
expect(page.evaluate.mock.calls[1][0]).toContain('RefWorks');
});
});
-92
View File
@@ -1,92 +0,0 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { clampInt, requireNonEmptyQuery } from '../_shared/common.js';
cli({
site: 'google-scholar',
name: 'profile',
description: 'View a Google Scholar author profile',
domain: 'scholar.google.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'author', positional: true, required: true, help: 'Author name or Scholar user ID (e.g. JicYPdAAAAAJ)' },
{ name: 'limit', type: 'int', default: 10, help: 'Max papers to show (max 20)' },
],
columns: ['rank', 'title', 'cited', 'year'],
navigateBefore: false,
func: async (page, kwargs) => {
const author = requireNonEmptyQuery(kwargs.author, 'author');
const limit = clampInt(kwargs.limit, 10, 1, 20);
const isUserId = /^[A-Za-z0-9_-]{12}$/.test(author);
if (isUserId) {
await page.goto(`https://scholar.google.com/citations?user=${author}&hl=en&sortby=citedby`);
} else {
await page.goto(`https://scholar.google.com/citations?view_op=search_authors&mauthors=${encodeURIComponent(author)}&hl=en`);
await page.wait(3);
const profileClicked = await page.evaluate(`(() => {
var link = document.querySelector('.gs_ai_pho, .gsc_oai_photo, a[href*="citations?user="]');
if (link) { link.click(); return true; }
return false;
})()`);
if (!profileClicked) {
throw new CommandExecutionError(`No profile found for: ${author}`);
}
}
await page.wait(3);
const data = await page.evaluate(`(() => {
var name = (document.querySelector('#gsc_prf_in') || {}).textContent || '';
var affiliation = (document.querySelector('.gsc_prf_il') || {}).textContent || '';
var stats = document.querySelectorAll('#gsc_rsb_st td.gsc_rsb_std');
var citations = stats[0] ? stats[0].textContent.trim() : '';
var hIndex = stats[2] ? stats[2].textContent.trim() : '';
var i10Index = stats[4] ? stats[4].textContent.trim() : '';
var papers = [];
var rows = document.querySelectorAll('#gsc_a_b .gsc_a_tr');
for (var i = 0; i < rows.length && i < ${limit}; i++) {
var titleEl = rows[i].querySelector('.gsc_a_at');
var citedEl = rows[i].querySelector('.gsc_a_ac');
var yearEl = rows[i].querySelector('.gsc_a_y span');
if (titleEl) papers.push({
rank: i + 1,
title: titleEl.textContent.trim(),
cited: citedEl ? citedEl.textContent.trim() : '0',
year: yearEl ? yearEl.textContent.trim() : '',
});
}
return {
name: name.trim(),
affiliation: affiliation.trim(),
citations: citations,
hIndex: hIndex,
i10Index: i10Index,
papers: papers,
};
})()`);
if (!data?.name) {
throw new CommandExecutionError(`Could not load Google Scholar profile for: ${author}`);
}
if (!data.papers || data.papers.length === 0) {
throw new CommandExecutionError(`No papers found for: ${data.name || author}`);
}
const summary = {
rank: 0,
title: data.name + (data.affiliation ? ' (' + data.affiliation + ')' : ''),
cited: 'h=' + data.hIndex + ' i10=' + data.i10Index + ' total=' + data.citations,
year: '-',
};
return [summary, ...data.papers];
},
});
-49
View File
@@ -1,49 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import './profile.js';
describe('google-scholar profile command', () => {
const command = getRegistry().get('google-scholar/profile');
it('registers as a public browser command', () => {
expect(command).toBeDefined();
expect(command.site).toBe('google-scholar');
expect(command.strategy).toBe('public');
expect(command.browser).toBe(true);
});
it('rejects empty author before browser navigation', async () => {
const page = { goto: vi.fn() };
await expect(command.func(page, { author: ' ' })).rejects.toMatchObject({
name: 'ArgumentError',
code: 'ARGUMENT',
});
expect(page.goto).not.toHaveBeenCalled();
});
it('throws when author search does not resolve to a profile', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValueOnce(false),
};
await expect(command.func(page, { author: 'missing author' })).rejects.toThrow(CommandExecutionError);
});
it('throws when the loaded profile has no papers', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValueOnce({
name: 'Author Name',
affiliation: 'Org',
citations: '0',
hIndex: '0',
i10Index: '0',
papers: [],
}),
};
await expect(command.func(page, { author: 'JicYPdAAAAAJ' })).rejects.toThrow(CommandExecutionError);
});
});
+1 -1
View File
@@ -23,7 +23,7 @@ cli({
(() => {
const normalize = v => (v || '').replace(/\\s+/g, ' ').trim();
const results = [];
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl')) {
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl, .gs_ri')) {
const container = el.querySelector('.gs_ri') || el;
const titleEl = container.querySelector('.gs_rt a, h3 a');
const title = normalize(titleEl?.textContent);
-15
View File
@@ -20,19 +20,4 @@ describe('google-scholar search command', () => {
});
expect(page.goto).not.toHaveBeenCalled();
});
it('locks dedup to outer Scholar result cards while preserving inner content extraction', async () => {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn().mockResolvedValue([]),
};
await command.func(page, { query: 'transformer' });
const script = page.evaluate.mock.calls[0][0];
expect(script).toContain("document.querySelectorAll('.gs_r.gs_or.gs_scl')");
expect(script).not.toContain(".gs_r.gs_or.gs_scl, .gs_ri");
expect(script).toContain("const container = el.querySelector('.gs_ri') || el");
});
});
+1 -1
View File
@@ -18,7 +18,7 @@ cli({
{ name: 'region', default: 'US', help: 'Region code (e.g. US, CN)' },
],
columns: ['title', 'source', 'date', 'url'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 100));
const lang = encodeURIComponent(args.lang);
const region = encodeURIComponent(args.region);
+1 -1
View File
@@ -15,7 +15,7 @@ cli({
{ name: 'lang', default: 'zh-CN', help: 'Language code' },
],
columns: ['suggestion'],
func: async (args) => {
func: async (_page, args) => {
const keyword = encodeURIComponent(args.keyword);
const lang = encodeURIComponent(args.lang);
const url = `https://suggestqueries.google.com/complete/search?client=firefox&q=${keyword}&hl=${lang}`;
+1 -1
View File
@@ -16,7 +16,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Number of results' },
],
columns: ['title', 'traffic', 'date'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 100));
const region = encodeURIComponent(args.region);
const url = `https://trends.google.com/trending/rss?geo=${region}`;
+1 -1
View File
@@ -50,7 +50,7 @@ cli({
return getWeekRange();
return kwargs.date ?? new Date().toISOString().slice(0, 10);
},
func: async (kwargs) => {
func: async (_page, kwargs) => {
const period = String(kwargs.period ?? 'daily');
const all = Boolean(kwargs.all);
const endpoint = process.env.HF_ENDPOINT?.replace(/\/+$/, '') || 'https://huggingface.co';
+46 -678
View File
@@ -5,545 +5,16 @@
* 用法: opencli jd item 100291143898
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
function normalizePositiveInt(value, fallback) {
const n = Number(value);
return Number.isFinite(n) && n >= 0 ? Math.floor(n) : fallback;
}
function normalizeJdSkuInput(input) {
const text = String(input || '').trim();
const itemMatch = text.match(/item\.jd\.com\/(\d+)\.html/i);
if (itemMatch)
return itemMatch[1];
if (/^\d+$/.test(text))
return text;
const paramMatch = text.match(/(?:skuId|sku|wareId|productId)[=:](\d+)/i);
if (paramMatch)
return paramMatch[1];
return text;
}
function normalizeJdImageUrl(rawUrl) {
if (!rawUrl || typeof rawUrl !== 'string')
return '';
let url = rawUrl.trim();
if (!url)
return '';
if (url.startsWith('//'))
url = `https:${url}`;
if (!/^https?:\/\//.test(url))
return '';
return url;
}
function normalizeJdImageSize(url) {
return normalizeJdImageUrl(url)
.replace(/\/pcpubliccms\/s\d+x\d+_jfs\//, '/pcpubliccms/jfs/')
.replace(/\/(n\d+)\/s\d+x\d+_jfs\//, '/$1/jfs/')
.replace(/\/s\d+x\d+_jfs\//, '/jfs/');
}
function isJdMainImage(url) {
const normalized = normalizeJdImageSize(url);
return /360buyimg\.com\/(?:pcpubliccms|n\d+)\/jfs\//.test(normalized) &&
!/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_)/.test(normalized) &&
!/\/(?:imgzone|sku|shaidan|popWaterMark|babel|jdcms|cms|ddimg|vc)\//.test(normalized);
}
function collectImageUrlsFrom(root, options = {}) {
if (!root)
return [];
const urls = [];
const ignoredContexts = '#spec-list, [class*="spec-list"], [class*="recommend"], [id*="recommend"], [class*="shaidan"], [id*="shaidan"], [class*="review"], [id*="review"], [class*="comment"], [id*="comment"], [class*="thumb"], [id*="thumb"], [class*="related"], [id*="related"]';
const ignoreContexts = Boolean(options.ignoreContexts);
const isIgnoredContext = (el) => {
if (!ignoreContexts)
return false;
try {
return !!el?.closest?.(ignoredContexts);
}
catch {
return false;
}
};
const pushUrlsFromText = (text) => {
for (const match of String(text || '').matchAll(/url\(["']?([^"')]+360buyimg\.com[^"')]+)["']?\)/g)) {
push(match[1]);
}
};
const push = (value) => {
const url = normalizeJdImageUrl(value);
if (url && url.includes('360buyimg.com'))
urls.push(url);
};
for (const img of root.querySelectorAll?.('img') || []) {
if (isIgnoredContext(img))
continue;
push(img.currentSrc || img.src);
push(img.getAttribute('data-src'));
push(img.getAttribute('data-lazy-img'));
push(img.getAttribute('data-lazyload'));
push(img.getAttribute('data-original'));
}
for (const source of root.querySelectorAll?.('source') || []) {
if (isIgnoredContext(source))
continue;
push(source.getAttribute('src'));
push(source.getAttribute('srcset')?.split(/\s+/)[0]);
push(source.getAttribute('data-src'));
push(source.getAttribute('data-srcset')?.split(/\s+/)[0]);
}
for (const el of root.querySelectorAll?.('[style*="360buyimg.com"]') || []) {
if (isIgnoredContext(el))
continue;
const style = el.getAttribute('style') || '';
pushUrlsFromText(style);
}
if (typeof getComputedStyle === 'function') {
const elements = [root, ...Array.from(root.querySelectorAll?.('*') || [])];
for (const el of elements) {
if (isIgnoredContext(el))
continue;
try {
const style = getComputedStyle(el);
pushUrlsFromText(style?.backgroundImage);
pushUrlsFromText(style?.background);
}
catch {
// ignore inaccessible/computed-style edge cases in the page context
}
}
}
return [...new Set(urls)];
}
function collectImageUrlsFromText(text) {
const urls = [];
const push = (value) => {
const url = normalizeJdImageUrl(value);
if (url && url.includes('360buyimg.com'))
urls.push(url);
};
for (const match of String(text || '').matchAll(/(?:https?:)?\/\/[^"'`\s<>)\\]+360buyimg\.com[^"'`\s<>)\\]*/g)) {
push(match[0]);
}
for (const match of String(text || '').matchAll(/url\(["']?([^"')]+360buyimg\.com[^"')]+)["']?\)/g)) {
push(match[1]);
}
return urls;
}
function collectImageUrlsFromFramesAndScripts() {
const urls = [];
for (const script of document.scripts || []) {
const text = script.textContent || '';
if (!/360buyimg\.com/.test(text))
continue;
urls.push(...collectImageUrlsFromText(text));
}
for (const iframe of document.querySelectorAll('iframe')) {
try {
const frameDoc = iframe.contentDocument || iframe.contentWindow?.document;
if (!frameDoc)
continue;
urls.push(...collectImageUrlsFrom(frameDoc.body || frameDoc.documentElement || frameDoc));
for (const script of frameDoc.scripts || []) {
const text = script.textContent || '';
if (!/360buyimg\.com/.test(text))
continue;
urls.push(...collectImageUrlsFromText(text));
}
}
catch {
// ignore cross-origin or not-yet-loaded iframe content
}
}
return [...new Set(urls)];
}
function collectImageUrlsFromPayload(payload, seen = new Set(), depth = 0) {
if (payload == null || depth > 4)
return [];
const urls = [];
const push = (value) => {
const url = normalizeJdImageUrl(value);
if (url && url.includes('360buyimg.com'))
urls.push(url);
};
if (typeof payload === 'string') {
urls.push(...collectImageUrlsFromText(payload));
return [...new Set(urls)];
}
if (typeof payload !== 'object')
return [];
if (seen.has(payload))
return [];
seen.add(payload);
if (Array.isArray(payload)) {
for (const item of payload)
urls.push(...collectImageUrlsFromPayload(item, seen, depth + 1));
return [...new Set(urls)];
}
for (const value of Object.values(payload)) {
if (typeof value === 'string') {
push(value);
urls.push(...collectImageUrlsFromText(value));
continue;
}
urls.push(...collectImageUrlsFromPayload(value, seen, depth + 1));
}
return [...new Set(urls)];
}
function collectImageUrlsFromPageDataObjects() {
const urls = [];
const keys = ['__NEXT_DATA__', '__NUXT__', '__INITIAL_STATE__', '__INITIAL_DATA__', '__APOLLO_STATE__', '__INITIAL_PROPS__', '__REDUX_STATE__', '__PAGE_DATA__', 'pageData', '__data__', 'detailData'];
for (const key of keys) {
try {
if (typeof globalThis !== 'undefined' && key in globalThis) {
urls.push(...collectImageUrlsFromPayload(globalThis[key]));
}
}
catch {
// ignore inaccessible globals
}
}
return [...new Set(urls)];
}
async function collectImageUrlsFromNetworkResources() {
const urls = [];
const entries = typeof performance !== 'undefined' && typeof performance.getEntriesByType === 'function'
? performance.getEntriesByType('resource')
: [];
const candidates = [...new Set(entries
.map((entry) => entry?.name)
.filter((name) => typeof name === 'string' && /(?:jd\.com|360buyimg\.com)/.test(name) && /(?:detail|desc|ware|item|product|sku)/i.test(name) && !/pc_item_getWareGraphic/i.test(name)))].slice(0, 12);
for (const url of candidates) {
try {
const resp = await fetch(url, { credentials: 'include' });
if (!resp.ok)
continue;
const text = await resp.text();
const contentType = resp.headers.get('content-type') || '';
if (/json/i.test(contentType) || /^\s*[\[{]/.test(text)) {
try {
urls.push(...collectImageUrlsFromPayload(JSON.parse(text)));
continue;
}
catch {
// fall back to text scanning
}
}
urls.push(...collectImageUrlsFromText(text));
}
catch {
// ignore request failures and cross-origin oddities
}
}
return [...new Set(urls)];
}
function collectImageUrlsFromWareGraphicText(text) {
let graphicContent = '';
try {
const payload = JSON.parse(String(text || ''));
graphicContent = payload?.data?.graphicContent || payload?.graphicContent || '';
}
catch {
graphicContent = String(text || '');
}
return [...new Set(collectImageUrlsFromText(graphicContent))];
}
async function collectImageUrlsFromWareGraphicResources(sku) {
const urls = [];
const entries = typeof performance !== 'undefined' && typeof performance.getEntriesByType === 'function'
? performance.getEntriesByType('resource')
: [];
const candidates = [...new Set(entries
.map((entry) => entry?.name)
.filter((name) => typeof name === 'string' && /pc_item_getWareGraphic/i.test(name)))];
if (sku) {
const body = encodeURIComponent(JSON.stringify({ skuId: String(sku) }));
candidates.unshift(`https://api.m.jd.com/client.action?appid=item-v3&functionId=pc_item_getWareGraphic&client=pc&clientVersion=1.0.0&body=${body}`);
}
for (const url of candidates) {
try {
const resp = await fetch(url, { credentials: 'include' });
if (!resp.ok)
continue;
urls.push(...collectImageUrlsFromWareGraphicText(await resp.text()));
}
catch {
// ignore request failures and keep DOM/script fallbacks available
}
}
return [...new Set(urls)];
}
async function collectImageUrlsFromFallbackSources() {
return [
...collectImageUrlsFromPageDataObjects(),
...await collectImageUrlsFromNetworkResources(),
];
}
function isJdDetailImage(url) {
const normalized = normalizeJdImageSize(url);
return /360buyimg\.com\/(?:imgzone|skuimg|babel|jdcms|cms|popWaterMark|vc|ddimg)\//.test(normalized) &&
!/\/shaidan\//.test(normalized) &&
!/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_|sku)\//.test(normalized);
}
function isJdWareGraphicDetailImage(url) {
const raw = normalizeJdImageUrl(url);
if (/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_|sku\/s\d+x\d+_)jfs\//.test(raw))
return false;
const normalized = normalizeJdImageSize(url);
return /360buyimg\.com\/sku\/jfs\//.test(normalized) &&
!/\/shaidan\//.test(normalized);
}
function rankJdDetailImage(url) {
const normalized = normalizeJdImageSize(url);
if (/\.jpe?g(?:\.avif)?(?:$|[?#])/.test(normalized))
return 0;
if (/\.(?:png|webp)(?:\.avif)?(?:$|[?#])/.test(normalized))
return 1;
if (/\.gif(?:$|[?#])/.test(normalized))
return 3;
if (/\.avif(?:$|[?#])/.test(normalized))
return 2;
return 4;
}
function orderJdDetailImages(urls, options = {}) {
const allowWareGraphicSku = Boolean(options.allowWareGraphicSku);
return [...new Set(urls)]
.filter((url) => isJdDetailImage(url) || (allowWareGraphicSku && isJdWareGraphicDetailImage(url)))
.map((url, index) => ({ url, index, rank: rankJdDetailImage(url) }))
.sort((a, b) => a.rank - b.rank || a.index - b.index)
.map((item) => item.url);
}
function extractDetailImagesFromDom(maxImages) {
const detailTitleParent = document.querySelector('#SPXQ-title')?.parentElement;
const safeDetailTitleParent = detailTitleParent && detailTitleParent !== document.body && detailTitleParent !== document.documentElement
? detailTitleParent
: null;
const selectorRoots = [
'#J-detail',
'#J-detail-content',
'#detail',
'.detail',
'.detail-content',
'.detail-content-wrap',
'.ssd-module-wrap',
'#SPXQ-title + *',
];
const scopedRoots = [
safeDetailTitleParent,
...selectorRoots.flatMap((selector) => Array.from(document.querySelectorAll(selector))),
].filter((root) => root && root !== document.body && root !== document.documentElement);
const scoped = [
...scopedRoots.flatMap((root) => collectImageUrlsFrom(root, { ignoreContexts: true })),
...collectImageUrlsFromFramesAndScripts(),
];
return orderJdDetailImages(scoped).slice(0, maxImages);
}
async function extractDetailImagesFromPage(maxImages, sku) {
const scoped = extractDetailImagesFromDom(maxImages);
const fallback = await collectImageUrlsFromFallbackSources();
const wareGraphic = await collectImageUrlsFromWareGraphicResources(sku);
const regularImages = orderJdDetailImages([...scoped, ...fallback]);
const wareGraphicImages = orderJdDetailImages(wareGraphic, { allowWareGraphicSku: true });
return orderJdDetailImages([...regularImages, ...wareGraphicImages], { allowWareGraphicSku: true }).slice(0, maxImages);
}
function getJdDetailScrollSnapshot(maxImages) {
const doc = document.scrollingElement || document.documentElement || document.body;
const scrollY = window.scrollY || window.pageYOffset || doc?.scrollTop || 0;
const viewportHeight = window.innerHeight || document.documentElement?.clientHeight || 0;
const scrollHeight = Math.max(doc?.scrollHeight || 0, document.documentElement?.scrollHeight || 0, document.body?.scrollHeight || 0);
return {
detailImageCount: extractDetailImagesFromDom(maxImages).length,
scrollY,
viewportHeight,
scrollHeight,
nearBottom: scrollY + viewportHeight >= scrollHeight - 120,
};
}
function scrollJdDetailStep() {
const step = Math.max(900, Math.floor((window.innerHeight || 900) * 0.9));
window.scrollBy(0, step);
return step;
}
async function scrollJdDetailIntoView() {
const tab = document.querySelector('#SPXQ-tab-column');
const title = document.querySelector('#SPXQ-title');
if (tab)
tab.click();
title?.scrollIntoView({ block: 'start' });
const step = Math.max(900, Math.floor((window.innerHeight || 900) * 0.9));
for (let i = 0; i < 2; i++) {
window.scrollBy(0, step);
await new Promise((resolve) => setTimeout(resolve, 250));
}
}
function extractMainImages(maxImages) {
const roots = [
document.querySelector('._gallery_116km_1'),
...Array.from(document.querySelectorAll('[class*="_gallery_"]')),
document.querySelector('.preview-wrap'),
document.querySelector('#spec-img')?.parentElement,
].filter(Boolean);
const urls = roots.flatMap((root) => collectImageUrlsFrom(root).map(normalizeJdImageSize));
return [...new Set(urls)]
.filter(isJdMainImage)
.slice(0, maxImages);
}
function extractAvifImages(imageUrls, maxImages) {
const unique = [...new Set(imageUrls.map(normalizeJdImageSize).filter(Boolean))];
const unique = [...new Set(imageUrls.filter(Boolean))];
return unique
.filter((url) => url.includes('.avif') && isJdDetailImage(url))
.filter((url) => url.includes('.avif') && url.includes('pcpubliccms'))
.slice(0, maxImages);
}
function extractPriceFromPayload(payload) {
const items = Array.isArray(payload) ? payload : [];
const item = items.find((entry) => entry && typeof entry === 'object');
for (const key of ['p', 'op', 'm']) {
const value = item?.[key];
if (value && value !== '-1.00')
return String(value);
}
return '';
}
function normalizePriceText(text) {
const match = String(text || '').replace(/\s+/g, '').match(/(?:¥|¥)?(\d{2,7}(?:\.\d{1,2})?)/);
return match ? match[1] : '';
}
function extractPriceFromDom(sku) {
const selectors = [
`.J-p-${sku}`,
'[class*="price"] [class*="num"]',
'[class*="price"]',
'.p-price strong',
'.price.jd-price',
];
for (const selector of selectors) {
for (const el of document.querySelectorAll(selector)) {
const price = normalizePriceText(el.textContent || '');
if (price)
return price;
}
}
for (const el of document.querySelectorAll('span, strong, div')) {
const text = (el.textContent || '').trim();
if (!/预售价|到手价|秒杀价|京东价|¥|¥/.test(text))
continue;
const direct = normalizePriceText(text);
if (direct)
return direct;
const parentPrice = normalizePriceText(el.parentElement?.textContent || '');
if (parentPrice)
return parentPrice;
}
return '';
}
async function fetchJdPrice(sku) {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 3000);
try {
const resp = await fetch(`https://p.3.cn/prices/mgets?skuIds=J_${encodeURIComponent(sku)}&type=1`, {
credentials: 'include',
signal: controller.signal,
});
if (!resp.ok)
return '';
return extractPriceFromPayload(await resp.json());
}
catch {
return '';
}
finally {
clearTimeout(timeout);
}
}
function extractSpecsFromText(text) {
const specs = {};
const lines = String(text || '').split('\n').map((line) => line.trim()).filter(Boolean);
const allowedKeys = new Set(['品牌', '商品名称', '商品编号', '商品毛重', '商品产地', '货号', '类型', '能效等级', '洗涤容量', '烘干容量', '排水方式', '颜色', '型号', '系列', '系列品', '款式', '版本', '规格', '容量']);
const setSpec = (key, val) => {
const normalizedKey = String(key || '').trim().replace(/[:]+$/, '');
const normalizedVal = String(val || '').trim();
if (!allowedKeys.has(normalizedKey))
return;
if (!normalizedVal || normalizedVal.length > 120)
return;
if (/^(服务|支付定金|加入购物车|立即购买|首页|购物车|我的|客服|品牌闪购|以旧换新)$/.test(normalizedVal))
return;
if (!specs[normalizedKey])
specs[normalizedKey] = normalizedVal;
};
for (const line of lines) {
const compactMatch = line.match(/^([^:]{1,12})[:]\s*(.{1,120})$/);
if (compactMatch) {
setSpec(compactMatch[1], compactMatch[2]);
continue;
}
}
for (let i = 0; i < lines.length - 1; i++) {
const key = lines[i].replace(/[:]+$/, '');
const val = lines[i + 1];
if (allowedKeys.has(key) && !allowedKeys.has(val)) {
setSpec(key, val);
}
}
return specs;
}
function extractSpecs() {
const specs = {};
const setSpec = (label, value) => {
const normalizedLabel = String(label || '').trim().replace(/[:]+$/, '');
const normalizedValue = String(value || '').replace(/\s+/g, ' ').trim();
if (!normalizedLabel || !normalizedValue)
return;
specs[normalizedLabel] = normalizedValue;
};
const selectedText = (root) => {
const selected = root.querySelector('.specification-item-sku--selected, .specification-series-item--selected, .selected, [class*="selected"]');
if (!selected)
return '';
return selected.querySelector('[class*="text"]')?.textContent?.trim() ||
selected.textContent?.trim() ||
selected.querySelector('img')?.getAttribute('alt')?.trim() ||
'';
};
for (const el of document.querySelectorAll('.specification-series-layout')) {
const label = el.querySelector('.layout-label')?.textContent?.trim();
setSpec(label, selectedText(el));
}
for (const el of document.querySelectorAll('.specification-group')) {
const label = el.querySelector('.specification-label, .specification-group-label, .label')?.textContent?.trim();
setSpec(label, selectedText(el));
}
const attrsRoot = document.querySelector('#SPXQ-title')?.parentElement?.querySelector('.attrs') ||
document.querySelector('#parameter2') ||
document.querySelector('.Ptable');
if (attrsRoot) {
Object.assign(specs, extractSpecsFromText(attrsRoot.innerText || attrsRoot.textContent || ''));
}
return specs;
}
function detectJdPageState(expectedSku) {
const href = location.href;
const title = document.title || '';
const bodyText = document.body?.innerText || document.body?.textContent || '';
const hasProductMarker = Boolean(document.querySelector('.product-title, .sku-title, #spec-list, #J-detail, #SPXQ-title, [class*="_gallery_"]'));
const text = `${title}\n${bodyText}`;
const isLoginPage = /passport\.jd\.com|\/login\.aspx/.test(href) || /京东-欢迎登录|京东登录/.test(title);
const hasSecurityChallenge = /risk_handler|安全验证|安全校验|完成安全验证|滑块|captcha|访问过于频繁|验证中心|京东验证/.test(`${href}\n${text}`);
const loginOnlyWithoutProduct = /请登录|登录/.test(text) && !hasProductMarker;
const looksBlocked = isLoginPage || hasSecurityChallenge || loginOnlyWithoutProduct;
const onExpectedItemUrl = new RegExp(`item\.jd\.com/${expectedSku}\.html`).test(href);
return {
href,
title,
isProductPage: hasProductMarker && onExpectedItemUrl && !looksBlocked,
hasProductMarker,
onExpectedItemUrl,
looksBlocked,
isLoginPage,
hasSecurityChallenge,
};
}
cli({
site: 'jd',
name: 'item',
description: '京东商品详情(价格、店铺、规格参数、主图、详情图',
description: '京东商品详情(价格、店铺、规格参数、AVIF 图片',
domain: 'item.jd.com',
strategy: Strategy.COOKIE,
args: [
@@ -556,174 +27,71 @@ cli({
{
name: 'images',
type: 'int',
default: 200,
help: '图片数量上限(默认200',
default: 10,
help: 'AVIF 图片数量上限(默认10',
},
],
columns: ['title', 'price', 'shop', 'specs', 'mainImages', 'detailImages'],
columns: ['title', 'price', 'shop', 'specs', 'avifImages'],
func: async (page, kwargs) => {
const sku = normalizeJdSkuInput(kwargs.sku);
const maxImages = normalizePositiveInt(kwargs.images, 200);
const sku = kwargs.sku;
const maxImages = kwargs.images;
const url = `https://item.jd.com/${sku}.html`;
const currentHref = await page.evaluate(`location.href`).catch(() => '');
if (!currentHref.includes(`item.jd.com/${sku}.html`)) {
await page.goto(url, { waitUntil: 'load' });
await page.wait(2);
}
const initialState = await page.evaluate(`(() => {
const detectJdPageState = ${detectJdPageState.toString()};
return detectJdPageState(${JSON.stringify(sku)});
})()`).catch(() => null);
if (!initialState?.looksBlocked) {
await page.evaluate(`
(async () => {
const scrollJdDetailIntoView = ${scrollJdDetailIntoView.toString()};
return scrollJdDetailIntoView();
})()
`);
await page.wait(1.5);
let previousDetailImageCount = -1;
let stableRounds = 0;
for (let i = 0; i < 30; i++) {
const snapshot = await page.evaluate(`(() => {
const normalizeJdImageUrl = ${normalizeJdImageUrl.toString()};
const normalizeJdImageSize = ${normalizeJdImageSize.toString()};
const collectImageUrlsFrom = ${collectImageUrlsFrom.toString()};
const collectImageUrlsFromText = ${collectImageUrlsFromText.toString()};
const collectImageUrlsFromFramesAndScripts = ${collectImageUrlsFromFramesAndScripts.toString()};
const collectImageUrlsFromPayload = ${collectImageUrlsFromPayload.toString()};
const collectImageUrlsFromPageDataObjects = ${collectImageUrlsFromPageDataObjects.toString()};
const collectImageUrlsFromNetworkResources = ${collectImageUrlsFromNetworkResources.toString()};
const collectImageUrlsFromWareGraphicText = ${collectImageUrlsFromWareGraphicText.toString()};
const collectImageUrlsFromWareGraphicResources = ${collectImageUrlsFromWareGraphicResources.toString()};
const collectImageUrlsFromFallbackSources = ${collectImageUrlsFromFallbackSources.toString()};
const isJdDetailImage = ${isJdDetailImage.toString()};
const isJdWareGraphicDetailImage = ${isJdWareGraphicDetailImage.toString()};
const rankJdDetailImage = ${rankJdDetailImage.toString()};
const orderJdDetailImages = ${orderJdDetailImages.toString()};
const extractDetailImagesFromDom = ${extractDetailImagesFromDom.toString()};
const extractDetailImagesFromPage = ${extractDetailImagesFromPage.toString()};
const getJdDetailScrollSnapshot = ${getJdDetailScrollSnapshot.toString()};
const scrollJdDetailIntoView = ${scrollJdDetailIntoView.toString()};
if (document.querySelector('#SPXQ-title') && window.scrollY < 1000) {
return scrollJdDetailIntoView().then(() => getJdDetailScrollSnapshot(${maxImages}));
}
return getJdDetailScrollSnapshot(${maxImages});
})()`).catch(() => null);
if (!snapshot)
break;
stableRounds = snapshot.detailImageCount === previousDetailImageCount ? stableRounds + 1 : 0;
previousDetailImageCount = snapshot.detailImageCount;
if (snapshot.nearBottom && stableRounds >= 2)
break;
await page.evaluate(`(() => {
const scrollJdDetailStep = ${scrollJdDetailStep.toString()};
return scrollJdDetailStep();
})()`);
await page.wait(0.8);
}
await page.goto(url, { waitUntil: 'load' });
await page.wait(2);
// 滚动加载商品详情区域中的延迟图片
for (let i = 0; i < 6; i++) {
await page.evaluate(`window.scrollTo(0, ${i * 2500})`);
await page.wait(1);
}
await page.evaluate(`window.scrollTo(0, document.body.scrollHeight)`);
await page.wait(2);
const data = await page.evaluate(`
(async () => {
(() => {
const maxImg = ${maxImages};
const normalizeJdImageUrl = ${normalizeJdImageUrl.toString()};
const normalizeJdImageSize = ${normalizeJdImageSize.toString()};
const isJdMainImage = ${isJdMainImage.toString()};
const collectImageUrlsFrom = ${collectImageUrlsFrom.toString()};
const collectImageUrlsFromText = ${collectImageUrlsFromText.toString()};
const collectImageUrlsFromFramesAndScripts = ${collectImageUrlsFromFramesAndScripts.toString()};
const collectImageUrlsFromPayload = ${collectImageUrlsFromPayload.toString()};
const collectImageUrlsFromPageDataObjects = ${collectImageUrlsFromPageDataObjects.toString()};
const collectImageUrlsFromNetworkResources = ${collectImageUrlsFromNetworkResources.toString()};
const collectImageUrlsFromWareGraphicText = ${collectImageUrlsFromWareGraphicText.toString()};
const collectImageUrlsFromWareGraphicResources = ${collectImageUrlsFromWareGraphicResources.toString()};
const collectImageUrlsFromFallbackSources = ${collectImageUrlsFromFallbackSources.toString()};
const isJdDetailImage = ${isJdDetailImage.toString()};
const isJdWareGraphicDetailImage = ${isJdWareGraphicDetailImage.toString()};
const rankJdDetailImage = ${rankJdDetailImage.toString()};
const orderJdDetailImages = ${orderJdDetailImages.toString()};
const extractMainImages = ${extractMainImages.toString()};
const extractDetailImagesFromDom = ${extractDetailImagesFromDom.toString()};
const extractDetailImagesFromPage = ${extractDetailImagesFromPage.toString()};
const extractPriceFromPayload = ${extractPriceFromPayload.toString()};
const fetchJdPrice = ${fetchJdPrice.toString()};
const extractSpecsFromText = ${extractSpecsFromText.toString()};
const extractSpecs = ${extractSpecs.toString()};
const detectJdPageState = ${detectJdPageState.toString()};
const pageState = detectJdPageState(${JSON.stringify(sku)});
const normalizePriceText = ${normalizePriceText.toString()};
const extractPriceFromDom = ${extractPriceFromDom.toString()};
const apiPrice = await fetchJdPrice(${JSON.stringify(sku)});
const domPrice = extractPriceFromDom(${JSON.stringify(sku)});
const price = apiPrice || domPrice || 'not found';
// 尝试多种价格选择器
const skuMatch = location.pathname.match(/(\\d+)\\.html/);
const sku = skuMatch ? skuMatch[1] : '';
const priceEl = document.querySelector('.J-p-' + sku) ||
document.querySelector('[class*="price"] [class*="num"]') ||
document.querySelector('.p-price strong') ||
document.querySelector('.price.jd-price');
const price = priceEl?.textContent?.trim() || 'not found';
// 标题
const title = document.querySelector('.product-title')?.textContent?.trim() ||
document.querySelector('.sku-title')?.textContent?.trim() ||
document.title.split('-')[0].trim();
const shop = document.querySelector('.J-shop-name')?.textContent?.trim() ||
document.querySelector('.top-name')?.textContent?.trim() ||
document.querySelector('[class*="shop"] [class*="name"]')?.textContent?.trim() ||
'京东自营';
// 店铺
const shop = document.querySelector('.J-shop-name')?.textContent?.trim() || '京东自营';
// 所有图片
const allImgs = Array.from(document.querySelectorAll('img[src*="360buyimg.com"]'));
const srcs = allImgs.map(img => img.src).filter(Boolean);
const mainImages = extractMainImages(maxImg);
const detailImages = await extractDetailImagesFromPage(maxImg, ${JSON.stringify(sku)});
const specs = extractSpecs();
// 所有 avif 图片(去重,只保留 pcpubliccms CDN
const avifImages = ${extractAvifImages.toString()}(srcs, maxImg);
const result = { title, price, shop, specs, mainImages, detailImages, totalImages: new Set(srcs).size, pageState };
if (!pageState.isProductPage) {
result.error = pageState.looksBlocked
? 'JD page is blocked by login/security verification'
: 'JD product page was not loaded';
result.pageState = pageState;
// 规格参数:从页面文本提取
const text = document.body.innerText;
const specMatch = text.match(/商品编号[\\s\\S]*?(?=包装清单|\\n\\n|$)/);
let specs = {};
if (specMatch) {
const lines = specMatch[0].split('\\n').filter(l => l.trim());
for (let i = 0; i < lines.length - 1; i += 2) {
const key = lines[i].trim();
const val = lines[i + 1]?.trim() || '';
if (key && val && key !== '商品编号') {
specs[key] = val;
}
}
}
return result;
return { title, price, shop, specs, avifImages, totalImages: new Set(srcs).size };
})()
`);
if (data?.error) {
if (data?.pageState?.looksBlocked) {
throw new AuthRequiredError('item.jd.com', data.error);
}
throw new CommandExecutionError(data.error);
}
if (maxImages > 0 && data?.pageState?.isProductPage && (!Array.isArray(data.detailImages) || data.detailImages.length === 0)) {
throw new CommandExecutionError('JD item detail images were not found', 'The product page loaded, but no detail images were detected from DOM, scripts, frames, page data, or WareGraphic fallback.');
}
return [data];
},
});
export const __test__ = {
normalizePositiveInt,
normalizeJdSkuInput,
normalizeJdImageUrl,
normalizeJdImageSize,
isJdMainImage,
collectImageUrlsFrom,
collectImageUrlsFromText,
collectImageUrlsFromFramesAndScripts,
collectImageUrlsFromPayload,
collectImageUrlsFromPageDataObjects,
collectImageUrlsFromNetworkResources,
collectImageUrlsFromWareGraphicText,
collectImageUrlsFromWareGraphicResources,
collectImageUrlsFromFallbackSources,
isJdDetailImage,
isJdWareGraphicDetailImage,
rankJdDetailImage,
orderJdDetailImages,
getJdDetailScrollSnapshot,
scrollJdDetailStep,
extractMainImages,
extractDetailImagesFromDom,
extractDetailImagesFromPage,
extractAvifImages,
extractPriceFromPayload,
normalizePriceText,
extractPriceFromDom,
extractSpecsFromText,
extractSpecs,
detectJdPageState,
};
+7 -318
View File
@@ -1,17 +1,7 @@
import { describe, expect, it, vi } from 'vitest';
import { JSDOM } from 'jsdom';
import { describe, expect, it } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { AuthRequiredError } from '@jackwener/opencli/errors';
import { __test__ } from './item.js';
import './item.js';
const originalPerformance = globalThis.performance;
const originalWindow = globalThis.window;
const originalDocument = globalThis.document;
function restoreGlobals() {
globalThis.performance = originalPerformance;
globalThis.window = originalWindow;
globalThis.document = originalDocument;
}
describe('jd item adapter', () => {
const command = getRegistry().get('jd/item');
it('registers the command with correct shape', () => {
@@ -28,79 +18,15 @@ describe('jd item adapter', () => {
expect(skuArg.required).toBe(true);
expect(skuArg.positional).toBe(true);
});
it('has images arg with default 200', () => {
it('has images arg with default 10', () => {
const imagesArg = command.args.find((a) => a.name === 'images');
expect(imagesArg).toBeDefined();
expect(imagesArg.default).toBe(200);
});
it('fails fast when JD blocks the item page', async () => {
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
.mockResolvedValueOnce({ looksBlocked: true })
.mockResolvedValueOnce({
error: 'JD page is blocked by login/security verification',
pageState: { looksBlocked: true },
}),
goto: vi.fn(),
wait: vi.fn(),
};
await expect(command.func(page, { sku: '100328272886', images: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
expect(page.goto).not.toHaveBeenCalled();
});
it('fails fast when a loaded product page has no detail images', async () => {
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
.mockResolvedValueOnce({ looksBlocked: false })
.mockResolvedValueOnce(undefined)
.mockResolvedValueOnce(null)
.mockResolvedValueOnce({
title: 'JD product',
price: '100',
shop: '京东自营',
specs: {},
mainImages: ['https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg'],
detailImages: [],
pageState: { isProductPage: true, looksBlocked: false },
}),
goto: vi.fn(),
wait: vi.fn(),
};
await expect(command.func(page, { sku: '100328272886', images: 5 })).rejects.toThrow('JD item detail images were not found');
});
it('allows zero-image requests to skip detail image fail-fast', async () => {
const data = {
title: 'JD product',
price: '100',
shop: '京东自营',
specs: {},
mainImages: [],
detailImages: [],
pageState: { isProductPage: true, looksBlocked: false },
};
const page = {
evaluate: vi.fn()
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
.mockResolvedValueOnce({ looksBlocked: false })
.mockResolvedValueOnce(undefined)
.mockResolvedValueOnce(null)
.mockResolvedValueOnce(data),
goto: vi.fn(),
wait: vi.fn(),
};
await expect(command.func(page, { sku: '100328272886', images: 0 })).resolves.toEqual([data]);
});
it('normalizes JD item URL input to a SKU before building selectors', () => {
expect(__test__.normalizeJdSkuInput('100328272886')).toBe('100328272886');
expect(__test__.normalizeJdSkuInput('https://item.jd.com/100328272886.html?purchasetab=gfgm')).toBe('100328272886');
expect(__test__.normalizeJdSkuInput('skuId=10218494560141')).toBe('10218494560141');
expect(imagesArg.default).toBe(10);
});
it('includes expected columns', () => {
expect(command.columns).toEqual(expect.arrayContaining(['title', 'price', 'shop', 'specs', 'mainImages', 'detailImages']));
expect(command.columns).not.toContain('avifImages');
expect(command.columns).toEqual(expect.arrayContaining(['title', 'price', 'shop', 'specs', 'avifImages']));
});
it('extracts only detail avif images and respects the limit', () => {
it('extracts only pcpubliccms avif images and respects the limit', () => {
const result = __test__.extractAvifImages([
'https://img14.360buyimg.com/n1/jfs/t1/normal.jpg',
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.avif',
@@ -110,245 +36,8 @@ describe('jd item adapter', () => {
'https://example.com/not-jd.avif',
], 2);
expect(result).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.avif',
'https://pcpubliccms.jd.com/image1.avif',
'https://pcpubliccms.jd.com/image2.avif?x=1',
]);
});
it('treats WareGraphic sku images as detail images only for WareGraphic fallback', () => {
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/sku/jfs/t1/color-option.gif')).toBe(false);
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/jfs/t1/ware-graphic.jpg')).toBe(true);
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/s228x228_jfs/t1/thumb.jpg')).toBe(false);
});
it('keeps main gallery images while ignoring unrelated contexts for detail images', () => {
const dom = new JSDOM(`
<div class="_gallery_116km_1">
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg" />
<img data-src="//img10.360buyimg.com/n1/jfs/t1/main-b.jpg" />
</div>
<div class="recommend">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/recommend.jpg.avif" />
</div>
<div class="detail-content-wrap">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
</div>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractMainImages(10)).toEqual([
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg',
'https://img10.360buyimg.com/n1/jfs/t1/main-b.jpg',
]);
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from computed background images', () => {
const dom = new JSDOM(`
<div id="J-detail">
<div class="ssd-module computed-bg"></div>
<div class="ssd-module ignored-bg"></div>
</div>
`);
const previousDocument = globalThis.document;
const previousGetComputedStyle = globalThis.getComputedStyle;
globalThis.document = dom.window.document;
globalThis.getComputedStyle = ((element) => ({
background: '',
backgroundImage: element.classList.contains('computed-bg')
? 'url("//img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif")'
: 'none',
}));
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif',
]);
}
finally {
globalThis.document = previousDocument;
globalThis.getComputedStyle = previousGetComputedStyle;
}
});
it('collects JD detail images from inline JSON-like script text', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<script>
window.__DETAIL_DATA__ = {
images: [
"https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif",
"//img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif"
]
};
</script>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from same-origin iframe content', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<iframe id="detail-frame"></iframe>
`, { url: 'https://item.jd.com/100328272886.html' });
const frameDom = new JSDOM(`
<div id="J-detail">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif" />
<div style="background-image:url(//img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif)"></div>
</div>
`, { url: 'https://item.jd.com/detail-frame.html' });
const iframe = dom.window.document.getElementById('detail-frame');
Object.defineProperty(iframe, 'contentDocument', { value: frameDom.window.document, configurable: true });
Object.defineProperty(iframe, 'contentWindow', { value: frameDom.window, configurable: true });
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif',
'https://img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from page data objects', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
globalThis.document = dom.window.document;
globalThis.window = dom.window;
globalThis.__PAGE_DATA__ = {
detail: {
images: [
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
{ src: '//img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp' },
],
},
};
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp',
]);
}
finally {
delete globalThis.__PAGE_DATA__;
restoreGlobals();
}
});
it('collects JD detail images from network resource text', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
const previousFetch = globalThis.fetch;
globalThis.document = dom.window.document;
globalThis.window = dom.window;
globalThis.performance = {
getEntriesByType: () => [{ name: 'https://cdn.jd.com/detail/data.json' }],
now: () => 0,
};
globalThis.fetch = (async () => ({
ok: true,
headers: { get: () => 'application/json' },
text: async () => JSON.stringify({
detail: {
imgs: ['https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif'],
},
}),
}));
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif',
]);
}
finally {
globalThis.fetch = previousFetch;
restoreGlobals();
}
});
it('collects JD detail images from WareGraphic graphicContent', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
const previousFetch = globalThis.fetch;
globalThis.document = dom.window.document;
globalThis.window = dom.window;
globalThis.performance = {
getEntriesByType: () => [
{ name: 'https://api.m.jd.com/client.action?functionId=pc_item_getWareGraphic&skuId=100328272886' },
],
now: () => 0,
};
globalThis.fetch = (async () => ({
ok: true,
headers: { get: () => 'application/json' },
text: async () => JSON.stringify({
data: {
graphicContent: `
<div style="background-image:url(//img30.360buyimg.com/sku/jfs/t1/ware-a.jpg)"></div>
<img src="https://img30.360buyimg.com/sku/jfs/t1/ware-b.png" />
<a href="https://item.jd.com/100328272886.html">not image</a>
`,
},
}),
}));
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img30.360buyimg.com/sku/jfs/t1/ware-a.jpg',
'https://img30.360buyimg.com/sku/jfs/t1/ware-b.png',
]);
}
finally {
globalThis.fetch = previousFetch;
restoreGlobals();
}
});
it('extracts selected specs from the newer JD spec-list DOM', () => {
const dom = new JSDOM(`
<div id="spec-list" class="page-right-spec">
<div class="horizontal-layout specification-series-layout">
<div class="layout-label">系列品</div>
<div class="layout-content">
<div class="specification-series-item specification-series-item--selected">
<span class="specification-series-item-text">【年度新品】玉兔3.0pro 12kg</span>
</div>
<div class="specification-series-item"><span class="specification-series-item-text">其他系列</span></div>
</div>
</div>
<div class="specifications-panel-content">
<div class="specification-group">
<div class="specification-group-label">款式</div>
<div class="specification-group-content">
<div class="specification-item-sku has-image specification-item-sku--selected" title="">
<img class="specification-item-sku-image" alt="洗烘套装" src="https://img13.360buyimg.com/pcpubliccms/s48x48_jfs/t1/spec.jpg.avif" />
<span class="specification-item-sku-text">洗烘套装</span>
</div>
<div class="specification-item-sku has-image"><span class="specification-item-sku-text">滚筒单洗</span></div>
</div>
</div>
</div>
</div>
`);
globalThis.document = dom.window.document;
try {
expect(__test__.extractSpecs()).toEqual({
系列品: '【年度新品】玉兔3.0pro 12kg',
款式: '洗烘套装',
});
}
finally {
restoreGlobals();
}
});
});
-517
View File
@@ -1,517 +0,0 @@
import { describe, expect, it } from 'vitest';
import { JSDOM } from 'jsdom';
import { __test__ } from './item.js';
const originalPerformance = globalThis.performance;
const originalWindow = globalThis.window;
const originalDocument = globalThis.document;
function restoreGlobals() {
globalThis.performance = originalPerformance;
globalThis.window = originalWindow;
globalThis.document = originalDocument;
}
describe('jd item image helpers', () => {
it('normalizes JD item URL input to a SKU before building selectors', () => {
expect(__test__.normalizeJdSkuInput('100328272886')).toBe('100328272886');
expect(__test__.normalizeJdSkuInput('https://item.jd.com/100328272886.html?purchasetab=gfgm')).toBe('100328272886');
expect(__test__.normalizeJdSkuInput('skuId=10218494560141')).toBe('10218494560141');
});
it('normalizes protocol-relative and thumbnail-sized JD image URLs', () => {
expect(__test__.normalizeJdImageUrl('//img10.360buyimg.com/imgzone/jfs/a.jpg.avif')).toBe('https://img10.360buyimg.com/imgzone/jfs/a.jpg.avif');
expect(__test__.normalizeJdImageSize('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/a.jpg.avif')).toBe('https://img10.360buyimg.com/pcpubliccms/jfs/t1/a.jpg.avif');
expect(__test__.normalizeJdImageSize('https://img10.360buyimg.com/n1/s450x450_jfs/t1/a.jpg.avif')).toBe('https://img10.360buyimg.com/n1/jfs/t1/a.jpg.avif');
});
it('accepts only product main images for mainImages', () => {
expect(__test__.isJdMainImage('https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif')).toBe(true);
expect(__test__.isJdMainImage('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/main.jpg.avif')).toBe(true);
expect(__test__.isJdMainImage('https://img10.360buyimg.com/n1/jfs/t1/main.jpg.avif')).toBe(true);
expect(__test__.isJdMainImage('https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif')).toBe(false);
expect(__test__.isJdMainImage('https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif')).toBe(false);
expect(__test__.isJdMainImage('https://img10.360buyimg.com/babel/jfs/t1/recommend.jpg.avif')).toBe(false);
});
it('accepts only detail-area image CDN paths for detailImages', () => {
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif')).toBe(true);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/jdcms/jfs/t1/detail.jpg.avif')).toBe(true);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/babel/jfs/t1/detail.jpg.avif')).toBe(true);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif')).toBe(false);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/thumb.jpg.avif')).toBe(false);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/n1/jfs/t1/main.jpg.avif')).toBe(false);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/sku/jfs/t1/color-option.gif')).toBe(false);
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/jfs/t1/ware-graphic.jpg')).toBe(true);
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/s228x228_jfs/t1/thumb.jpg')).toBe(false);
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif')).toBe(false);
});
it('keeps legacy avifImages restricted to detail images only', () => {
expect(__test__.extractAvifImages([
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif',
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif',
'https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif',
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.gif',
], 10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif',
]);
});
it('prioritizes JPG detail images before PNG banners and GIFs when limiting detailImages', () => {
expect(__test__.orderJdDetailImages([
'https://img10.360buyimg.com/imgzone/jfs/t1/hero-a.gif',
'https://img11.360buyimg.com/imgzone/jfs/t1/banner.png.avif',
'https://img12.360buyimg.com/imgzone/jfs/t1/326893/25/18592/117159/68c264f5F9a41addf/2c9ad60b7f390339.jpg.avif',
'https://img12.360buyimg.com/imgzone/jfs/t1/330152/17/11906/130964/68c264f2Ffcf6e5c1/c2ccb28722dc47ce.jpg.avif',
'https://img11.360buyimg.com/cms/jfs/t1/banner.gif',
])).toEqual([
'https://img12.360buyimg.com/imgzone/jfs/t1/326893/25/18592/117159/68c264f5F9a41addf/2c9ad60b7f390339.jpg.avif',
'https://img12.360buyimg.com/imgzone/jfs/t1/330152/17/11906/130964/68c264f2Ffcf6e5c1/c2ccb28722dc47ce.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/banner.png.avif',
'https://img10.360buyimg.com/imgzone/jfs/t1/hero-a.gif',
'https://img11.360buyimg.com/cms/jfs/t1/banner.gif',
]);
});
it('extracts valid JD price payload values', () => {
expect(__test__.extractPriceFromPayload([{ id: 'J_100291143898', p: '6999.00' }])).toBe('6999.00');
expect(__test__.extractPriceFromPayload([{ id: 'J_100291143898', p: '-1.00', op: '7299.00' }])).toBe('7299.00');
expect(__test__.extractPriceFromPayload([])).toBe('');
});
it('extracts visible JD price text from DOM', () => {
const dom = new JSDOM(`
<div>
<span>预售价</span>
<span><span>¥</span><span>12221</span></span>
</div>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.normalizePriceText('¥ 12221')).toBe('12221');
expect(__test__.extractPriceFromDom('100291143898')).toBe('12221');
}
finally {
globalThis.document = previousDocument;
}
});
it('extracts only gallery main images and detail-container images from DOM', () => {
const dom = new JSDOM(`
<div class="_gallery_116km_1">
<img src="https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/main-a.jpg.avif" />
<img data-src="//img10.360buyimg.com/n1/s450x450_jfs/t1/main-b.jpg.avif" />
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/wrong-detail-in-gallery.jpg.avif" />
</div>
<div class="recommend">
<img src="https://img10.360buyimg.com/babel/jfs/t1/recommend.jpg.avif" />
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/recommend-main-like.jpg.avif" />
</div>
<h2 id="SPXQ-title">商品详情</h2>
<div class="detail-content-wrap">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-hero.gif" />
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
<source srcset="https://img11.360buyimg.com/imgzone/jfs/t1/detail-source.jpg.avif 1x" />
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/wrong-main-in-detail.jpg.avif" />
<img src="https://img10.360buyimg.com/shaidan/jfs/t1/wrong-user.jpg.avif" />
</div>
<div id="spec-list">
<img src="https://img10.360buyimg.com/pcpubliccms/s48x48_jfs/t1/wrong-sku-option.jpg.avif" />
</div>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractMainImages(10)).toEqual([
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg.avif',
'https://img10.360buyimg.com/n1/jfs/t1/main-b.jpg.avif',
]);
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/detail-source.jpg.avif',
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-hero.gif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from computed background images', () => {
const dom = new JSDOM(`
<div id="J-detail">
<div class="ssd-module computed-bg"></div>
<div class="ssd-module ignored-bg"></div>
</div>
`);
const previousDocument = globalThis.document;
const previousGetComputedStyle = globalThis.getComputedStyle;
globalThis.document = dom.window.document;
globalThis.getComputedStyle = ((element: Element) => ({
background: '',
backgroundImage: element.classList.contains('computed-bg')
? 'url("//img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif")'
: 'none',
})) as typeof getComputedStyle;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif',
]);
}
finally {
globalThis.document = previousDocument;
globalThis.getComputedStyle = previousGetComputedStyle;
}
});
it('collects JD detail images from inline JSON-like script text', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<script>
window.__DETAIL_DATA__ = {
images: [
"https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif",
"//img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif"
]
};
</script>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from same-origin iframe content', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<iframe id="detail-frame"></iframe>
`, { url: 'https://item.jd.com/100328272886.html' });
const frameDom = new JSDOM(`
<div id="J-detail">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif" />
<div style="background-image:url(//img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif)"></div>
</div>
`, { url: 'https://item.jd.com/detail-frame.html' });
const iframe = dom.window.document.getElementById('detail-frame') as HTMLIFrameElement;
Object.defineProperty(iframe, 'contentDocument', { value: frameDom.window.document, configurable: true });
Object.defineProperty(iframe, 'contentWindow', { value: frameDom.window, configurable: true });
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif',
'https://img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('collects JD detail images from page data objects', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
globalThis.document = dom.window.document;
globalThis.window = dom.window as unknown as Window;
(globalThis as typeof globalThis & { __PAGE_DATA__?: unknown }).__PAGE_DATA__ = {
detail: {
images: [
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
{ src: '//img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp' },
],
},
};
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp',
]);
}
finally {
delete (globalThis as typeof globalThis & { __PAGE_DATA__?: unknown }).__PAGE_DATA__;
restoreGlobals();
}
});
it('collects JD detail images from network resource text', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
const previousFetch = globalThis.fetch;
globalThis.document = dom.window.document;
globalThis.window = dom.window as unknown as Window;
const fakePerformance = {
getEntriesByType: () => [{ name: 'https://cdn.jd.com/detail/data.json' }],
now: () => 0,
} as Performance;
globalThis.performance = fakePerformance;
globalThis.fetch = (async () => ({
ok: true,
headers: { get: () => 'application/json' },
text: async () => JSON.stringify({
detail: {
imgs: ['https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif'],
},
}),
})) as typeof fetch;
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif',
]);
}
finally {
globalThis.fetch = previousFetch;
restoreGlobals();
}
});
it('collects JD detail images from WareGraphic graphicContent', async () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
`);
const previousFetch = globalThis.fetch;
globalThis.document = dom.window.document;
globalThis.window = dom.window as unknown as Window;
globalThis.performance = {
getEntriesByType: () => [
{ name: 'https://api.m.jd.com/client.action?functionId=pc_item_getWareGraphic&skuId=100328272886' },
],
now: () => 0,
} as Performance;
globalThis.fetch = (async () => ({
ok: true,
headers: { get: () => 'application/json' },
text: async () => JSON.stringify({
data: {
graphicContent: `
<div style="background-image:url(//img30.360buyimg.com/sku/jfs/t1/ware-a.jpg)"></div>
<img src="https://img30.360buyimg.com/sku/jfs/t1/ware-b.png" />
<a href="https://item.jd.com/100328272886.html">not image</a>
`,
},
}),
})) as typeof fetch;
try {
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
'https://img30.360buyimg.com/sku/jfs/t1/ware-a.jpg',
'https://img30.360buyimg.com/sku/jfs/t1/ware-b.png',
]);
}
finally {
globalThis.fetch = previousFetch;
restoreGlobals();
}
});
it('collects images from every repeated JD detail module', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<div class="ssd-module-wrap">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
</div>
<div class="ssd-module-wrap">
<img src="https://img11.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif" />
</div>
<div class="ssd-module-wrap">
<img src="https://img12.360buyimg.com/imgzone/jfs/t1/detail-c.gif" />
</div>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
'https://img11.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif',
'https://img12.360buyimg.com/imgzone/jfs/t1/detail-c.gif',
]);
}
finally {
globalThis.document = previousDocument;
}
});
it('reports detail scroll progress so lazy-loaded detail images can stabilize', () => {
const dom = new JSDOM(`
<h2 id="SPXQ-title">商品详情</h2>
<div class="detail-content-wrap">
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif" />
</div>
`);
const previousDocument = globalThis.document;
const previousWindow = globalThis.window;
globalThis.document = dom.window.document;
globalThis.window = dom.window as unknown as Window & typeof globalThis;
Object.defineProperty(dom.window, 'scrollY', { value: 1800, configurable: true });
Object.defineProperty(dom.window, 'innerHeight', { value: 900, configurable: true });
Object.defineProperty(dom.window.document.documentElement, 'scrollHeight', { value: 2600, configurable: true });
try {
expect(__test__.getJdDetailScrollSnapshot(10)).toMatchObject({
detailImageCount: 2,
scrollY: 1800,
viewportHeight: 900,
scrollHeight: 2600,
nearBottom: true,
});
}
finally {
globalThis.document = previousDocument;
globalThis.window = previousWindow;
}
});
it('parses structured specs without pairing unrelated body text', () => {
expect(__test__.extractSpecsFromText([
'品牌:美的(Midea',
'商品编号',
'100291143898',
'洗涤容量',
'能效等级',
'一级能效',
'类型',
'加入购物车',
].join('\n'))).toEqual({
: '美的(Midea',
: '100291143898',
: '一级能效',
});
});
it('extracts selected specs from the newer JD spec-list DOM', () => {
const dom = new JSDOM(`
<div id="spec-list" class="page-right-spec">
<div class="horizontal-layout specification-series-layout">
<div class="layout-label">系列品</div>
<div class="layout-content">
<div class="specification-series-item specification-series-item--selected">
<span class="specification-series-item-text">【年度新品】玉兔3.0pro 12kg</span>
</div>
<div class="specification-series-item"><span class="specification-series-item-text">其他系列</span></div>
</div>
</div>
<div class="specifications-panel-content">
<div class="specification-group">
<div class="specification-group-label">款式</div>
<div class="specification-group-content">
<div class="specification-item-sku has-image specification-item-sku--selected" title="">
<img class="specification-item-sku-image" alt="洗烘套装" src="https://img13.360buyimg.com/pcpubliccms/s48x48_jfs/t1/spec.jpg.avif" />
<span class="specification-item-sku-text">洗烘套装</span>
</div>
<div class="specification-item-sku has-image"><span class="specification-item-sku-text">滚筒单洗</span></div>
</div>
</div>
</div>
</div>
`);
const previousDocument = globalThis.document;
globalThis.document = dom.window.document;
try {
expect(__test__.extractSpecs()).toEqual({
: '【年度新品】玉兔3.0pro 12kg',
: '洗烘套装',
});
}
finally {
globalThis.document = previousDocument;
}
});
it('detects whether the loaded page is the expected JD product page', () => {
const dom = new JSDOM(`
<html>
<head><title>京东</title></head>
<body><div>请登录</div><div class="sku-title">商品标题</div><div id="spec-list"></div></body>
</html>
`, { url: 'https://item.jd.com/100291143898.html' });
const previousDocument = globalThis.document;
const previousLocation = globalThis.location;
globalThis.document = dom.window.document;
globalThis.location = dom.window.location;
try {
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
isProductPage: true,
hasProductMarker: true,
onExpectedItemUrl: true,
looksBlocked: false,
isLoginPage: false,
hasSecurityChallenge: false,
});
document.body.innerHTML = '<div>请登录后完成安全验证</div>';
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
isProductPage: false,
looksBlocked: true,
hasSecurityChallenge: true,
});
const riskDom = new JSDOM(`
<html>
<head><title>京东验证</title></head>
<body><div>京东验证</div></body>
</html>
`, { url: 'https://cfe.m.jd.com/privatedomain/risk_handler/03101900/?returnurl=https%3A%2F%2Fitem.jd.com%2F100291143898.html' });
globalThis.document = riskDom.window.document;
globalThis.location = riskDom.window.location;
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
isProductPage: false,
looksBlocked: true,
hasSecurityChallenge: true,
});
}
finally {
globalThis.document = previousDocument;
globalThis.location = previousLocation;
}
});
it('does not treat JD login page as a product page', () => {
const dom = new JSDOM(`
<html>
<head><title>京东-欢迎登录</title></head>
<body><img src="https://img10.360buyimg.com/img/jfs/login.png" /></body>
</html>
`, { url: 'https://passport.jd.com/new/login.aspx?ReturnUrl=https%3A%2F%2Fitem.jd.com%2F100291143898.html' });
const previousDocument = globalThis.document;
const previousLocation = globalThis.location;
globalThis.document = dom.window.document;
globalThis.location = dom.window.location;
try {
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
isProductPage: false,
hasProductMarker: false,
onExpectedItemUrl: false,
looksBlocked: true,
isLoginPage: true,
});
}
finally {
globalThis.document = previousDocument;
globalThis.location = previousLocation;
}
});
});
+1 -1
View File
@@ -19,7 +19,7 @@ cli({
{ name: 'limit', type: 'int', default: 5, help: 'Number of comments' },
],
columns: ['rank', 'score', 'author', 'text'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const postId = gqlEscape(parsePostId(String(kwargs['url-or-id'])));
const limit = Number(kwargs.limit ?? 5);
// Fetch post title and comments in parallel
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "curated", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "frontpage", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "new", limit: ${limit}}}) {
+1 -1
View File
@@ -18,7 +18,7 @@ cli({
},
],
columns: ['title', 'author', 'karma', 'comments', 'tags', 'content', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const postId = parsePostId(String(kwargs['url-or-id']));
const query = `query PostsSingle {
post(input: {selector: {documentId: "${gqlEscape(postId)}"}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query Sequences {
sequences(input: {terms: {view: "communitySequences", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "shortform", limit: ${limit}}}) {
+1 -1
View File
@@ -19,7 +19,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: 'Number of results' },
],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const tagInput = String(kwargs.tag);
const limit = Number(kwargs.limit ?? 10);
const tag = await resolveTagId(tagInput);
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 20, help: 'Number of results' }],
columns: ['rank', 'name', 'posts'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 20);
const query = `query Tags {
tags(input: {terms: {view: "coreTags", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "top", after: "${daysAgo(30)}", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "top", after: "${daysAgo(7)}", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "top", after: "${daysAgo(365)}", limit: ${limit}}}) {
+1 -1
View File
@@ -9,7 +9,7 @@ cli({
browser: false,
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Number(kwargs.limit ?? 10);
const query = `query PostsList {
posts(input: {terms: {view: "top", limit: ${limit}}}) {
+1 -1
View File
@@ -18,7 +18,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: 'Number of results' },
],
columns: ['rank', 'title', 'karma', 'comments', 'date', 'url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const username = String(kwargs.username);
const limit = Number(kwargs.limit ?? 10);
const user = await resolveUserId(username);
+1 -1
View File
@@ -18,7 +18,7 @@ cli({
},
],
columns: ['field', 'value'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const slug = gqlEscape(String(kwargs.username).toLowerCase());
const query = `query UserProfile {
user(input: {selector: {slug: "${slug}"}}) {
+6 -6
View File
@@ -38,7 +38,7 @@ describe('paperreview submit command', () => {
resolvedPath: '/tmp/paper.pdf',
sizeBytes: 4096,
});
const result = await cmd.func({
const result = await cmd.func(null, {
pdf: './paper.pdf',
email: 'wang2629651228@gmail.com',
venue: 'RAL',
@@ -80,7 +80,7 @@ describe('paperreview submit command', () => {
message: 'Submission accepted',
},
});
const result = await cmd.func({
const result = await cmd.func(null, {
pdf: './paper.pdf',
email: 'wang2629651228@gmail.com',
venue: 'RAL',
@@ -112,7 +112,7 @@ describe('paperreview submit command', () => {
s3_key: 'uploads/paper.pdf',
},
});
const result = await cmd.func({
const result = await cmd.func(null, {
pdf: './paper.pdf',
email: 'wang2629651228@gmail.com',
venue: 'RAL',
@@ -153,7 +153,7 @@ describe('paperreview submit command', () => {
message: 'Submission accepted',
},
});
const result = await cmd.func({
const result = await cmd.func(null, {
pdf: './paper.pdf',
email: 'wang2629651228@gmail.com',
venue: 'RAL',
@@ -190,7 +190,7 @@ describe('paperreview review command', () => {
response: { status: 202 },
payload: { detail: 'Review is still processing.' },
});
const result = await cmd.func({ token: 'tok_123' });
const result = await cmd.func(null, { token: 'tok_123' });
expect(result).toMatchObject({
status: 'processing',
token: 'tok_123',
@@ -214,7 +214,7 @@ describe('paperreview feedback command', () => {
response: { ok: true, status: 200 },
payload: { message: 'Thanks for the feedback.' },
});
const result = await cmd.func({
const result = await cmd.func(null, {
token: 'tok_123',
helpfulness: 4,
'critical-error': 'yes',
+1 -1
View File
@@ -17,7 +17,7 @@ cli({
{ name: 'additional-comments', help: 'Optional free-text feedback' },
],
columns: ['status', 'token', 'helpfulness', 'critical_error', 'actionable_suggestions', 'message'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const token = String(kwargs.token ?? '').trim();
if (!token) {
throw new CliError('ARGUMENT', 'A review token is required.');
+1 -1
View File
@@ -13,7 +13,7 @@ cli({
{ name: 'token', positional: true, required: true, help: 'Review token returned by paperreview.ai' },
],
columns: ['status', 'title', 'venue', 'numerical_score', 'has_feedback', 'review_url'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const token = String(kwargs.token ?? '').trim();
if (!token) {
throw new CliError('ARGUMENT', 'A review token is required.');
+1 -1
View File
@@ -24,7 +24,7 @@ cli({
return 'prepared only';
return undefined;
},
func: async (kwargs) => {
func: async (_page, kwargs) => {
const pdfFile = await readPdfFile(kwargs.pdf);
const email = String(kwargs.email ?? '').trim();
const venue = normalizeVenue(kwargs.venue);
+1 -1
View File
@@ -19,7 +19,7 @@ cli({
},
],
columns: ['rank', 'name', 'tagline', 'author', 'date', 'url'],
func: async (args) => {
func: async (_page, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const category = String(args.category ?? '').trim() || undefined;
const posts = await fetchFeed(category);
+1 -1
View File
@@ -16,7 +16,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: 'Max results' },
],
columns: ['rank', 'name', 'tagline', 'author', 'url'],
func: async (args) => {
func: async (_page, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const posts = await fetchFeed();
if (posts.length === 0)
+1 -1
View File
@@ -47,5 +47,5 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回的文章数量' },
],
columns: ['rank', 'title', 'author', 'date', 'description', 'url'],
func: async (args) => searchSinaBlog(args.keyword, Math.max(1, Math.min(Number(args.limit) || 20, 50))),
func: async (_page, args) => searchSinaBlog(args.keyword, Math.max(1, Math.min(Number(args.limit) || 20, 50))),
});
+1 -1
View File
@@ -34,7 +34,7 @@ cli({
{ name: 'type', type: 'int', default: 0, help: 'News type: 0=全部 1=A股 2=宏观 3=公司 4=数据 5=市场 6=国际 7=观点 8=央行 9=其它' },
],
columns: ['id', 'time', 'content', 'views'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit), 50));
const apiTag = TYPE_MAP[args.type] ?? 0;
const params = new URLSearchParams({
+1 -1
View File
@@ -62,7 +62,7 @@ cli({
{ name: 'market', type: 'string', default: 'auto', help: 'Market: cn, hk, us, auto (default: auto searches cn → hk → us)' },
],
columns: ['Symbol', 'Name', 'Price', 'Change', 'ChangePercent', 'Open', 'High', 'Low', 'Volume', 'MarketCap'],
func: async (args) => {
func: async (_page, args) => {
const key = String(args.key);
const market = String(args.market);
const marketMap = {
+2 -2
View File
@@ -28,7 +28,7 @@ describe('sinafinance stock command', () => {
.mockResolvedValueOnce(textResponse('var hq_str_gb_AAPL="Apple Inc,189.98,1.23,0,1.56,0,188.50,180.00,195.00,175.00,1200000,0,3000000000000";'));
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({ key: 'AAPL', market: 'auto' });
const result = await cmd.func(null, { key: 'AAPL', market: 'auto' });
expect(fetchMock).toHaveBeenNthCalledWith(1, 'https://suggest3.sinajs.cn/suggest/type=11,31,41&key=AAPL', expect.any(Object));
expect(fetchMock).toHaveBeenNthCalledWith(2, 'https://hq.sinajs.cn/list=gb_AAPL', expect.any(Object));
@@ -48,7 +48,7 @@ describe('sinafinance stock command', () => {
.mockResolvedValueOnce(textResponse('var hq_str_gb_AAPL="苹果公司,189.98,1.23,0,1.56,0,188.50,180.00,195.00,175.00,1200000,0,3000000000000";'));
vi.stubGlobal('fetch', fetchMock);
const result = await cmd.func({ key: '苹果', market: 'auto' });
const result = await cmd.func(null, { key: '苹果', market: 'auto' });
expect(fetchMock).toHaveBeenNthCalledWith(2, 'https://hq.sinajs.cn/list=gb_AAPL', expect.any(Object));
expect(result[0]).toMatchObject({
+6 -6
View File
@@ -198,7 +198,7 @@ cli({
browser: false,
args: [{ name: 'query', type: 'str', default: '', positional: true, help: 'Track or artist to play (optional)' }],
columns: ['track', 'artist', 'status'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
if (kwargs.query) {
const { uri, name, artist } = await findTrackUri(kwargs.query);
await api('PUT', '/me/player/play', { uris: [uri] });
@@ -246,7 +246,7 @@ cli({
browser: false,
args: [{ name: 'level', type: 'int', default: 50, positional: true, required: true, help: 'Volume 0100' }],
columns: ['volume'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const level = Math.round(kwargs.level);
if (level < 0 || level > 100)
throw new CliError('INVALID_ARGS', 'Volume must be between 0 and 100');
@@ -265,7 +265,7 @@ cli({
{ name: 'limit', type: 'int', default: 10, help: 'Number of results (default: 10)' },
],
columns: ['track', 'artist', 'album', 'uri'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const limit = Math.min(50, Math.max(1, Math.round(kwargs.limit)));
const data = await api('GET', `/search?q=${encodeURIComponent(kwargs.query)}&type=track&limit=${limit}`);
const results = mapSpotifyTrackResults(data);
@@ -282,7 +282,7 @@ cli({
browser: false,
args: [{ name: 'query', type: 'str', required: true, positional: true, help: 'Track to add to queue' }],
columns: ['track', 'artist', 'status'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
const { uri, name, artist } = await findTrackUri(kwargs.query);
await api('POST', `/me/player/queue?uri=${encodeURIComponent(uri)}`);
return [{ track: name, artist, status: 'added to queue' }];
@@ -296,7 +296,7 @@ cli({
browser: false,
args: [{ name: 'state', type: 'str', default: 'on', positional: true, choices: ['on', 'off'], help: 'on or off' }],
columns: ['shuffle'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
await api('PUT', `/me/player/shuffle?state=${kwargs.state === 'on'}`);
return [{ shuffle: kwargs.state }];
},
@@ -309,7 +309,7 @@ cli({
browser: false,
args: [{ name: 'mode', type: 'str', default: 'context', positional: true, choices: ['off', 'track', 'context'], help: 'off / track / context' }],
columns: ['repeat'],
func: async (kwargs) => {
func: async (_page, kwargs) => {
await api('PUT', `/me/player/repeat?state=${kwargs.mode}`);
return [{ repeat: kwargs.mode }];
},
+1 -1
View File
@@ -69,7 +69,7 @@ cli({
{ name: 'limit', type: 'int', default: 20, help: '返回结果数量' },
],
columns: ['rank', 'title', 'author', 'date', 'description', 'url'],
func: async (args) => {
func: async (_page, args) => {
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 50));
return args.type === 'publications'
? searchPublications(args.keyword, limit)
+74 -225
View File
@@ -1,142 +1,11 @@
import { AuthRequiredError, SelectorError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { resolveTwitterQueryId, sanitizeQueryId } from './shared.js';
const BEARER_TOKEN = 'AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA';
const FOLLOWING_QUERY_ID = 'zx6e-TLzRkeDO_a7p4b3JQ'; // Following fallback
const USER_BY_SCREEN_NAME_QUERY_ID = 'qRednkZG-rn1P6b48NINmQ';
const FEATURES = {
rweb_video_screen_enabled: false,
profile_label_improvements_pcf_label_in_post_enabled: true,
responsive_web_profile_redirect_enabled: false,
rweb_tipjar_consumption_enabled: false,
verified_phone_label_enabled: false,
creator_subscriptions_tweet_preview_api_enabled: true,
responsive_web_graphql_timeline_navigation_enabled: true,
responsive_web_graphql_skip_user_profile_image_extensions_enabled: false,
premium_content_api_read_enabled: false,
communities_web_enable_tweet_community_results_fetch: true,
c9s_tweet_anatomy_moderator_badge_enabled: true,
responsive_web_grok_analyze_button_fetch_trends_enabled: false,
responsive_web_grok_analyze_post_followups_enabled: true,
responsive_web_jetfuel_frame: true,
responsive_web_grok_share_attachment_enabled: true,
responsive_web_grok_annotations_enabled: true,
articles_preview_enabled: true,
responsive_web_edit_tweet_api_enabled: true,
graphql_is_translatable_rweb_tweet_is_translatable_enabled: true,
view_counts_everywhere_api_enabled: true,
longform_notetweets_consumption_enabled: true,
responsive_web_twitter_article_tweet_consumption_enabled: true,
tweet_awards_web_tipping_enabled: false,
content_disclosure_indicator_enabled: true,
content_disclosure_ai_generated_indicator_enabled: true,
responsive_web_grok_show_grok_translated_post: false,
responsive_web_grok_analysis_button_from_backend: true,
post_ctas_fetch_enabled: false,
freedom_of_speech_not_reach_fetch_enabled: true,
standardized_nudges_misinfo: true,
tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled: true,
longform_notetweets_rich_text_read_enabled: true,
longform_notetweets_inline_media_enabled: false,
responsive_web_grok_image_annotation_enabled: true,
responsive_web_grok_imagine_annotation_enabled: true,
responsive_web_grok_community_note_auto_translation_is_enabled: false,
responsive_web_enhance_cards_enabled: false,
};
function buildFollowingUrl(queryId, userId, count, cursor) {
const vars = {
userId,
count,
includePromotedContent: false,
withClientEventToken: false,
withBirdwatchNotes: false,
withVoice: true,
withV2Timeline: true,
};
if (cursor)
vars.cursor = cursor;
return `/i/api/graphql/${queryId}/Following`
+ `?variables=${encodeURIComponent(JSON.stringify(vars))}`
+ `&features=${encodeURIComponent(JSON.stringify(FEATURES))}`;
}
function buildUserByScreenNameUrl(queryId, screenName) {
const vars = JSON.stringify({ screen_name: screenName, withSafetyModeUserFields: true });
const feats = JSON.stringify({
hidden_profile_subscriptions_enabled: true,
rweb_tipjar_consumption_enabled: true,
responsive_web_graphql_exclude_directive_enabled: true,
verified_phone_label_enabled: false,
subscriptions_verification_info_is_identity_verified_enabled: true,
subscriptions_verification_info_verified_since_enabled: true,
highlights_tweets_tab_ui_enabled: true,
responsive_web_twitter_article_notes_tab_enabled: true,
subscriptions_feature_can_gift_premium: true,
creator_subscriptions_tweet_preview_api_enabled: true,
responsive_web_graphql_skip_user_profile_image_extensions_enabled: false,
responsive_web_graphql_timeline_navigation_enabled: true,
});
return `/i/api/graphql/${queryId}/UserByScreenName`
+ `?variables=${encodeURIComponent(vars)}`
+ `&features=${encodeURIComponent(feats)}`;
}
function extractUser(result) {
if (!result || result.__typename !== 'User')
return null;
const core = result.core || {};
const legacy = result.legacy || {};
return {
screen_name: core.screen_name || legacy.screen_name || 'unknown',
name: core.name || legacy.name || 'unknown',
bio: legacy.description || result.profile_bio?.description || '',
followers: legacy.followers_count || legacy.normal_followers_count || 0,
};
}
function parseFollowing(data) {
const users = [];
let nextCursor = null;
const instructions = data?.data?.user?.result?.timeline_v2?.timeline?.instructions
|| data?.data?.user?.result?.timeline?.timeline?.instructions
|| [];
for (const inst of instructions) {
for (const entry of inst.entries || []) {
const content = entry.content;
// Extract cursor
if (content?.entryType === 'TimelineTimelineCursor' || content?.__typename === 'TimelineTimelineCursor') {
if (content.cursorType === 'Bottom' || content.cursorType === 'ShowMore')
nextCursor = content.value;
continue;
}
if (entry.entryId?.startsWith('cursor-bottom-') || entry.entryId?.startsWith('cursor-showMore-')) {
nextCursor = content?.value || content?.itemContent?.value || nextCursor;
continue;
}
// Extract user
if (entry.entryId?.startsWith('user-')) {
const user = extractUser(content?.itemContent?.user_results?.result);
if (user)
users.push(user);
}
}
}
return { users, nextCursor };
}
function normalizeScreenName(value) {
return String(value || '').trim().replace(/^\/+/, '').replace(/^@+/, '');
}
cli({
site: 'twitter',
name: 'following',
description: 'Get accounts a Twitter/X user is following',
domain: 'x.com',
strategy: Strategy.COOKIE,
strategy: Strategy.INTERCEPT,
browser: true,
args: [
{ name: 'user', positional: true, type: 'string', required: false },
@@ -144,103 +13,83 @@ cli({
],
columns: ['screen_name', 'name', 'bio', 'followers'],
func: async (page, kwargs) => {
const limit = kwargs.limit === undefined || kwargs.limit === null ? 50 : Number(kwargs.limit);
if (!Number.isInteger(limit) || limit <= 0) {
throw new ArgumentError('twitter following --limit must be a positive integer', 'Example: opencli twitter following @elonmusk --limit 200');
}
let targetUser = normalizeScreenName(kwargs.user);
await page.goto('https://x.com');
await page.wait(3);
const ct0 = await page.evaluate(`() => {
return document.cookie.split(';').map(c => c.trim()).find(c => c.startsWith('ct0='))?.split('=')[1] || null;
}`);
if (!ct0)
throw new AuthRequiredError('x.com', 'Not logged into x.com (no ct0 cookie)');
let targetUser = kwargs.user;
// If no user is specified, figure out the logged-in user's handle
if (!targetUser) {
await page.goto('https://x.com/home');
await page.wait({ selector: '[data-testid="primaryColumn"]' });
const href = await page.evaluate(`() => {
const link = document.querySelector('a[data-testid="AppTabBar_Profile_Link"]');
return link ? link.getAttribute('href') : null;
}`);
if (!href)
throw new AuthRequiredError('x.com', 'Could not detect logged-in user. Are you logged in?');
targetUser = normalizeScreenName(href.replace('/', ''));
}
if (!targetUser) {
throw new ArgumentError('twitter following user cannot be empty', 'Example: opencli twitter following @elonmusk --limit 200');
}
const followingQueryId = await resolveTwitterQueryId(page, 'Following', FOLLOWING_QUERY_ID);
const userByScreenNameQueryId = await resolveTwitterQueryId(page, 'UserByScreenName', USER_BY_SCREEN_NAME_QUERY_ID);
const headers = JSON.stringify({
'Authorization': `Bearer ${decodeURIComponent(BEARER_TOKEN)}`,
'X-Csrf-Token': ct0,
'X-Twitter-Auth-Type': 'OAuth2Session',
'X-Twitter-Active-User': 'yes',
});
// Get userId from screen_name
const userLookup = await page.evaluate(`async () => {
const url = ${JSON.stringify(buildUserByScreenNameUrl(userByScreenNameQueryId, targetUser))};
const resp = await fetch(url, { headers: ${headers}, credentials: 'include' });
if (!resp.ok) return { error: resp.status };
const d = await resp.json();
return { userId: d.data?.user?.result?.rest_id || null };
const link = document.querySelector('a[data-testid="AppTabBar_Profile_Link"]');
return link ? link.getAttribute('href') : null;
}`);
if (userLookup?.error === 401 || userLookup?.error === 403) {
throw new AuthRequiredError('x.com', `Twitter user lookup failed (HTTP ${userLookup.error})`);
}
if (userLookup?.error) {
throw new CommandExecutionError(`HTTP ${userLookup.error}: Failed to resolve Twitter user @${targetUser}`);
}
const userId = userLookup?.userId || null;
if (!userId)
throw new CommandExecutionError(`Could not find user @${targetUser}`);
const allUsers = [];
const seen = new Set();
let cursor = null;
const maxPages = Math.ceil(limit / 50) + 2;
for (let i = 0; i < maxPages && allUsers.length < limit; i++) {
const fetchCount = Math.min(50, limit - allUsers.length + 10);
const apiUrl = buildFollowingUrl(followingQueryId, userId, fetchCount, cursor);
const data = await page.evaluate(`async () => {
const r = await fetch("${apiUrl}", { headers: ${headers}, credentials: 'include' });
return r.ok ? await r.json() : { error: r.status };
}`);
if (data?.error) {
if (data.error === 401 || data.error === 403)
throw new AuthRequiredError('x.com', `Twitter following request failed (HTTP ${data.error})`);
throw new CommandExecutionError(`HTTP ${data.error}: Failed to fetch following list. queryId may have expired.`);
if (!href) {
throw new AuthRequiredError('x.com', 'Could not find logged-in user profile link. Are you logged in?');
}
const { users, nextCursor } = parseFollowing(data);
for (const u of users) {
if (!seen.has(u.screen_name)) {
seen.add(u.screen_name);
allUsers.push(u);
targetUser = href.replace('/', '');
}
// 1. Navigate to profile page
await page.goto(`https://x.com/${targetUser}`);
await page.wait(3);
// 2. Install interceptor BEFORE SPA navigation.
// goto() resets JS context, but SPA click preserves it.
await page.installInterceptor('Following');
// 3. Click the following link via SPA navigation (preserves interceptor)
const safeUser = JSON.stringify(targetUser);
const clicked = await page.evaluate(`() => {
const target = ${safeUser};
const link = document.querySelector('a[href="/' + target + '/following"]');
if (link) { link.click(); return true; }
return false;
}`);
if (!clicked) {
throw new SelectorError('Twitter following link', 'Twitter may have changed the layout.');
}
await page.waitForCapture(5);
// 4. Scroll to trigger pagination API calls
await page.autoScroll({ times: Math.ceil(kwargs.limit / 20), delayMs: 2000 });
// 5. Retrieve intercepted data
const requests = await page.getInterceptedRequests();
const requestList = Array.isArray(requests) ? requests : [];
if (requestList.length === 0) {
return [];
}
let results = [];
for (const req of requestList) {
try {
// GraphQL response: { data: { user: { result: { timeline: ... } } } }
let instructions = req.data?.user?.result?.timeline?.timeline?.instructions;
if (!instructions)
continue;
let addEntries = instructions.find((i) => i.type === 'TimelineAddEntries');
if (!addEntries) {
addEntries = instructions.find((i) => i.entries && Array.isArray(i.entries));
}
if (!addEntries)
continue;
for (const entry of addEntries.entries) {
if (!entry.entryId.startsWith('user-'))
continue;
const item = entry.content?.itemContent?.user_results?.result;
if (!item || item.__typename !== 'User')
continue;
const core = item.core || {};
const legacy = item.legacy || {};
results.push({
screen_name: core.screen_name || legacy.screen_name || 'unknown',
name: core.name || legacy.name || 'unknown',
bio: legacy.description || item.profile_bio?.description || '',
followers: legacy.followers_count || legacy.normal_followers_count || 0
});
}
}
if (!nextCursor || nextCursor === cursor)
break;
cursor = nextCursor;
catch (e) {
// ignore parsing errors for individual payloads
}
}
if (allUsers.length === 0) {
throw new EmptyResultError('twitter following', `No following accounts found for @${targetUser}`);
}
return allUsers.slice(0, limit);
},
// Deduplicate by screen_name
const unique = new Map();
results.forEach(r => unique.set(r.screen_name, r));
const deduplicatedResults = Array.from(unique.values());
return deduplicatedResults.slice(0, kwargs.limit);
}
});
export const __test__ = {
sanitizeQueryId,
buildFollowingUrl,
buildUserByScreenNameUrl,
extractUser,
normalizeScreenName,
parseFollowing,
};
-277
View File
@@ -1,277 +0,0 @@
import { describe, expect, it, vi } from 'vitest';
import { getRegistry } from '@jackwener/opencli/registry';
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { __test__ } from './following.js';
describe('twitter following helpers', () => {
it('falls back when queryId contains unsafe characters', () => {
expect(__test__.sanitizeQueryId('safe_Query-123', 'fallback')).toBe('safe_Query-123');
expect(__test__.sanitizeQueryId('bad"id', 'fallback')).toBe('fallback');
expect(__test__.sanitizeQueryId('bad/id', 'fallback')).toBe('fallback');
expect(__test__.sanitizeQueryId(null, 'fallback')).toBe('fallback');
});
it('builds following url with cursor', () => {
const url = __test__.buildFollowingUrl('query123', '42', 20, 'cursor-1');
expect(url).toContain('/i/api/graphql/query123/Following');
expect(decodeURIComponent(url)).toContain('"userId":"42"');
expect(decodeURIComponent(url)).toContain('"count":20');
expect(decodeURIComponent(url)).toContain('"cursor":"cursor-1"');
});
it('builds following url without cursor', () => {
const url = __test__.buildFollowingUrl('query123', '42', 20);
expect(url).toContain('/i/api/graphql/query123/Following');
expect(decodeURIComponent(url)).not.toContain('"cursor"');
});
it('extracts user from result', () => {
const user = __test__.extractUser({
__typename: 'User',
core: { screen_name: 'alice', name: 'Alice' },
legacy: { description: 'bio text', followers_count: 100 },
});
expect(user).toMatchObject({
screen_name: 'alice',
name: 'Alice',
bio: 'bio text',
followers: 100,
});
});
it('returns null for non-User typename', () => {
expect(__test__.extractUser({ __typename: 'Tweet' })).toBeNull();
expect(__test__.extractUser(null)).toBeNull();
expect(__test__.extractUser(undefined)).toBeNull();
});
it('falls back to legacy screen_name if core is missing', () => {
const user = __test__.extractUser({
__typename: 'User',
legacy: { screen_name: 'bob', name: 'Bob', description: '', followers_count: 0 },
});
expect(user?.screen_name).toBe('bob');
});
it('parses following timeline with users and cursor', () => {
const payload = {
data: {
user: {
result: {
timeline_v2: {
timeline: {
instructions: [{
entries: [
{
entryId: 'user-1',
content: {
itemContent: {
user_results: {
result: {
__typename: 'User',
core: { screen_name: 'bob', name: 'Bob' },
legacy: { description: 'hello', followers_count: 50 },
},
},
},
},
},
{
entryId: 'user-2',
content: {
itemContent: {
user_results: {
result: {
__typename: 'User',
core: { screen_name: 'carol', name: 'Carol' },
legacy: { description: 'world', followers_count: 200 },
},
},
},
},
},
{
entryId: 'cursor-bottom-1',
content: {
entryType: 'TimelineTimelineCursor',
cursorType: 'Bottom',
value: 'next-cursor',
},
},
],
}],
},
},
},
},
},
};
const result = __test__.parseFollowing(payload);
expect(result.users).toHaveLength(2);
expect(result.users[0]).toMatchObject({ screen_name: 'bob', name: 'Bob', followers: 50 });
expect(result.users[1]).toMatchObject({ screen_name: 'carol', name: 'Carol', followers: 200 });
expect(result.nextCursor).toBe('next-cursor');
});
it('handles cursor-bottom entryId pattern', () => {
const payload = {
data: {
user: {
result: {
timeline: {
timeline: {
instructions: [{
entries: [
{
entryId: 'cursor-bottom-0',
content: {
itemContent: { value: 'cursor-val' },
},
},
],
}],
},
},
},
},
},
};
const result = __test__.parseFollowing(payload);
expect(result.nextCursor).toBe('cursor-val');
expect(result.users).toHaveLength(0);
});
it('returns empty users and null cursor for missing instructions', () => {
const result = __test__.parseFollowing({ data: { user: { result: {} } } });
expect(result.users).toHaveLength(0);
expect(result.nextCursor).toBeNull();
});
it('returns empty for completely empty payload', () => {
const result = __test__.parseFollowing({});
expect(result.users).toHaveLength(0);
expect(result.nextCursor).toBeNull();
});
it('normalizes screen names for CLI and profile-link inputs', () => {
expect(__test__.normalizeScreenName('@elonmusk')).toBe('elonmusk');
expect(__test__.normalizeScreenName('/elonmusk')).toBe('elonmusk');
expect(__test__.normalizeScreenName(' @@alice ')).toBe('alice');
});
});
function followingPayload(users, cursor) {
return {
data: {
user: {
result: {
timeline_v2: {
timeline: {
instructions: [{
entries: [
...users.map((name) => ({
entryId: `user-${name}`,
content: {
itemContent: {
user_results: {
result: {
__typename: 'User',
core: { screen_name: name, name: name.toUpperCase() },
legacy: { description: `${name} bio`, followers_count: 10 },
},
},
},
},
})),
...(cursor ? [{
entryId: `cursor-bottom-${cursor}`,
content: {
entryType: 'TimelineTimelineCursor',
cursorType: 'Bottom',
value: cursor,
},
}] : []),
],
}],
},
},
},
},
},
};
}
function createFollowingPage(followingResponses, { ct0 = 'token', userLookup = { userId: '42' } } = {}) {
const page = {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate: vi.fn(async (script) => {
if (script.includes('document.cookie')) return ct0;
if (script.includes('operationName')) return null;
if (script.includes('/UserByScreenName')) return userLookup;
if (script.includes('/Following')) return followingResponses.shift() || followingPayload([], null);
if (script.includes('AppTabBar_Profile_Link')) return '/viewer';
throw new Error(`Unexpected evaluate script: ${script.slice(0, 80)}`);
}),
};
return page;
}
describe('twitter following command', () => {
it('paginates with cursor, deduplicates users, strips @, and respects limit', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([
followingPayload(['alice', 'bob'], 'cursor-1'),
followingPayload(['bob', 'carol', 'dave'], null),
]);
const rows = await command.func(page, { user: '@elonmusk', limit: 3 });
expect(rows.map((row) => row.screen_name)).toEqual(['alice', 'bob', 'carol']);
const userLookupScript = page.evaluate.mock.calls.find(([script]) => script.includes('/UserByScreenName'))?.[0] || '';
expect(decodeURIComponent(userLookupScript)).toContain('"screen_name":"elonmusk"');
expect(decodeURIComponent(userLookupScript)).not.toContain('"screen_name":"@elonmusk"');
const followingCalls = page.evaluate.mock.calls.filter(([script]) => script.includes('/Following'));
expect(followingCalls).toHaveLength(2);
expect(decodeURIComponent(followingCalls[1][0])).toContain('"cursor":"cursor-1"');
});
it('rejects invalid limits before navigating', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([]);
await expect(command.func(page, { user: 'elonmusk', limit: 0 })).rejects.toBeInstanceOf(ArgumentError);
expect(page.goto).not.toHaveBeenCalled();
});
it('maps first-page auth failures to AuthRequiredError', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([{ error: 401 }]);
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('does not silently return partial rows when a later page fails', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([
followingPayload(['alice'], 'cursor-1'),
{ error: 429 },
]);
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(CommandExecutionError);
});
it('maps user lookup auth failures to AuthRequiredError', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([], { userLookup: { error: 403 } });
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('fails fast when the following timeline is empty', async () => {
const command = getRegistry().get('twitter/following');
const page = createFollowingPage([followingPayload([], null)]);
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(EmptyResultError);
});
});

Some files were not shown because too many files have changed in this diff Show More