Compare commits
6 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| dc055954cf | |||
| e5de7f391e | |||
| bebab9e6a2 | |||
| 682994d6e2 | |||
| 204f4b7109 | |||
| 3a087aaf80 |
@@ -5,8 +5,6 @@
|
||||
### Features
|
||||
|
||||
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
|
||||
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
|
||||
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
|
||||
|
||||
## [1.7.8](https://github.com/jackwener/opencli/compare/v1.7.7...v1.7.8) (2026-04-25)
|
||||
|
||||
|
||||
@@ -20,7 +20,6 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other
|
||||
|
||||
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
|
||||
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type, extract, screenshot — all through your logged-in Chrome session.
|
||||
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
|
||||
- **Website → CLI** — Turn any website into a deterministic CLI: 90+ pre-built adapters, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
|
||||
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
|
||||
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
|
||||
@@ -34,10 +33,7 @@ It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other
|
||||
|
||||
### 1. Install OpenCLI
|
||||
|
||||
OpenCLI requires **Node.js >= 21**.
|
||||
|
||||
```bash
|
||||
node --version
|
||||
npm install -g @jackwener/opencli
|
||||
```
|
||||
|
||||
@@ -59,20 +55,7 @@ Install **OpenCLI** from the [Chrome Web Store](https://chromewebstore.google.co
|
||||
opencli doctor
|
||||
```
|
||||
|
||||
### 4. Optional: name your Chrome profile
|
||||
|
||||
Each Chrome profile runs its own OpenCLI extension instance. If you use multiple Chrome profiles, list the connected profiles and assign local aliases:
|
||||
|
||||
```bash
|
||||
opencli profile list
|
||||
opencli profile rename <contextId> work
|
||||
opencli profile use work
|
||||
opencli --profile work browser state
|
||||
```
|
||||
|
||||
With only one connected profile, OpenCLI uses it automatically. With multiple connected profiles and no default, OpenCLI asks you to choose instead of guessing.
|
||||
|
||||
### 5. Run your first commands
|
||||
### 4. Run your first commands
|
||||
|
||||
```bash
|
||||
opencli list
|
||||
@@ -86,7 +69,7 @@ Use OpenCLI directly when you want a reliable command instead of a live browser
|
||||
|
||||
- `opencli list` shows every registered command.
|
||||
- `opencli <site> <command>` runs a built-in or generated adapter.
|
||||
- `opencli external register mycli` exposes a local CLI through the same discovery surface.
|
||||
- `opencli register mycli` exposes a local CLI through the same discovery surface.
|
||||
- `opencli doctor` helps diagnose browser connectivity.
|
||||
|
||||
## For AI Agents
|
||||
@@ -174,8 +157,7 @@ OpenCLI is not only for websites. It can also:
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Node.js**: >= 21.0.0 (required for the standard npm install path)
|
||||
- **Bun**: >= 1.0 (optional alternative runtime)
|
||||
- **Node.js**: >= 21.0.0 (or **Bun** >= 1.0)
|
||||
- **Chrome or Chromium** running and logged into the target site for browser-backed commands
|
||||
|
||||
> **Important**: Browser-backed commands reuse your Chrome/Chromium login session. If you get empty data or permission-like failures, first confirm the site is already open and authenticated in Chrome/Chromium.
|
||||
@@ -185,9 +167,8 @@ OpenCLI is not only for websites. It can also:
|
||||
| Variable | Default | Description |
|
||||
|----------|---------|-------------|
|
||||
| `OPENCLI_DAEMON_PORT` | `19825` | HTTP port for the daemon-extension bridge |
|
||||
| `OPENCLI_PROFILE` | — | Browser Bridge profile alias/contextId to use when multiple Chrome profiles are connected |
|
||||
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open the automation container in the foreground (useful for debugging). The `--focus` flag sets this. |
|
||||
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation lease open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
|
||||
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open automation windows in the foreground (useful for debugging). The `--focus` flag sets this. |
|
||||
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation window open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
|
||||
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | Seconds to wait for browser connection |
|
||||
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | Seconds to wait for a single browser command |
|
||||
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
|
||||
@@ -196,7 +177,7 @@ OpenCLI is not only for websites. It can also:
|
||||
| `OPENCLI_DIAGNOSTIC` | `false` | Set to `1` to capture structured diagnostic context on failures |
|
||||
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
|
||||
|
||||
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation lease open until you run `opencli browser close` or the idle timeout expires.
|
||||
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation window open until you run `opencli browser close` or the idle timeout expires.
|
||||
|
||||
## Update
|
||||
|
||||
@@ -257,7 +238,7 @@ To load the source Browser Bridge extension:
|
||||
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
|
||||
| **uiverse** | `code` `preview` |
|
||||
| **baidu-scholar** | `search` |
|
||||
| **google-scholar** | `search` `cite` `profile` |
|
||||
| **google-scholar** | `search` |
|
||||
| **gov-law** | `search` `recent` |
|
||||
| **gov-policy** | `search` `recent` |
|
||||
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
|
||||
@@ -286,7 +267,7 @@ OpenCLI acts as a universal hub for your existing command-line tools — unified
|
||||
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
|
||||
|
||||
```bash
|
||||
opencli external register mycli
|
||||
opencli register mycli
|
||||
```
|
||||
|
||||
### Desktop App Adapters
|
||||
@@ -409,7 +390,7 @@ See **[TESTING.md](./TESTING.md)** for how to run and write tests.
|
||||
- **"Extension not connected"** — Ensure the Browser Bridge extension is installed from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) and **enabled** in `chrome://extensions`.
|
||||
- **"attach failed: Cannot access a chrome-extension:// URL"** — Another extension may be interfering. Try disabling other extensions temporarily.
|
||||
- **Empty data or 'Unauthorized' error** — Your Chrome/Chromium login session may have expired. Navigate to the target site and log in again.
|
||||
- **Node API errors / missing `fetch` / startup crash on old Node** — OpenCLI requires **Node.js >= 21**. Run `node --version`, upgrade Node if needed, then retry.
|
||||
- **Node API errors** — Ensure Node.js >= 21. Some features require `node:util` styleText (stable in Node 21+).
|
||||
- **Daemon issues** — Check status: `curl localhost:19825/status` · View logs: `curl localhost:19825/logs`
|
||||
|
||||
## Star History
|
||||
|
||||
+4
-8
@@ -31,10 +31,7 @@ OpenCLI 可以用同一套 CLI 做三类事情:
|
||||
|
||||
### 1. 安装 OpenCLI
|
||||
|
||||
OpenCLI 要求 **Node.js >= 21**。
|
||||
|
||||
```bash
|
||||
node --version
|
||||
npm install -g @jackwener/opencli
|
||||
```
|
||||
|
||||
@@ -158,8 +155,7 @@ OpenCLI 不只是网站 CLI,还可以:
|
||||
|
||||
## 前置要求
|
||||
|
||||
- **Node.js**: >= 21.0.0(标准 npm 安装路径要求)
|
||||
- **Bun**: >= 1.0(可选替代运行时)
|
||||
- **Node.js**: >= 21.0.0
|
||||
- 浏览器型命令需要 Chrome 或 Chromium 处于运行中,并已登录目标网站
|
||||
|
||||
> **重要**:浏览器型命令直接复用你的 Chrome/Chromium 登录态。如果拿到空数据或出现权限类失败,先确认目标站点已经在浏览器里打开并完成登录。
|
||||
@@ -243,7 +239,7 @@ npm link
|
||||
| **uiverse** | `code` `preview` | 浏览器 |
|
||||
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
|
||||
| **baidu-scholar** | `search` | 公开 |
|
||||
| **google-scholar** | `search` `cite` `profile` | 公开 |
|
||||
| **google-scholar** | `search` | 公开 |
|
||||
| **gov-law** | `search` `recent` | 公开 |
|
||||
| **gov-policy** | `search` `recent` | 公开 |
|
||||
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
|
||||
@@ -506,8 +502,8 @@ opencli plugin uninstall my-tool # 卸载
|
||||
- 其他 Chrome/Chromium 扩展(如 youmind、New Tab Override 或 AI 助手类扩展)可能产生冲突。请尝试**暂时禁用其他扩展**后重试。
|
||||
- **返回空数据,或者报错 "Unauthorized"**
|
||||
- Chrome/Chromium 里的登录态可能已经过期。请打开当前页面,在新标签页重新手工登录或刷新该页面。
|
||||
- **Node API 错误 / 缺少 `fetch` / 旧 Node 启动即崩**
|
||||
- OpenCLI 要求 **Node.js >= 21**。先执行 `node --version`,如果版本过低先升级,再重试命令。
|
||||
- **Node API 错误 (如 parseArgs, fs 等)**
|
||||
- 确保 Node.js 版本 `>= 21`(`node:util` 的 `styleText` 需要 Node 21+)。
|
||||
- **Daemon 问题**
|
||||
- 检查 daemon 状态:`curl localhost:19825/status`
|
||||
- 查看扩展日志:`curl localhost:19825/logs`
|
||||
|
||||
+29
-347
@@ -3275,7 +3275,7 @@
|
||||
{
|
||||
"site": "boss",
|
||||
"name": "search",
|
||||
"description": "BOSS直聘搜索职位(不带关键词时返回为你推荐职位)",
|
||||
"description": "BOSS直聘搜索职位",
|
||||
"domain": "www.zhipin.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
@@ -3283,9 +3283,9 @@
|
||||
{
|
||||
"name": "query",
|
||||
"type": "str",
|
||||
"required": false,
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "Search keyword (optional, empty = recommended jobs)"
|
||||
"help": "Search keyword (e.g. AI agent, 前端)"
|
||||
},
|
||||
{
|
||||
"name": "city",
|
||||
@@ -3299,7 +3299,7 @@
|
||||
"type": "str",
|
||||
"default": "",
|
||||
"required": false,
|
||||
"help": "Experience: 在校生(实习)/应届生(校招)/经验不限/1年以内/1-3年/3-5年/5-10年/10年以上"
|
||||
"help": "Experience: 应届/1年以内/1-3年/3-5年/5-10年/10年以上"
|
||||
},
|
||||
{
|
||||
"name": "degree",
|
||||
@@ -3322,13 +3322,6 @@
|
||||
"required": false,
|
||||
"help": "Industry code or name (e.g. 100020, 互联网)"
|
||||
},
|
||||
{
|
||||
"name": "jobType",
|
||||
"type": "str",
|
||||
"default": "",
|
||||
"required": false,
|
||||
"help": "Job type: 全职/兼职/实习(不传=不限,混合校招与实习)"
|
||||
},
|
||||
{
|
||||
"name": "page",
|
||||
"type": "int",
|
||||
@@ -3353,7 +3346,6 @@
|
||||
"degree",
|
||||
"skills",
|
||||
"boss",
|
||||
"bossOnline",
|
||||
"security_id",
|
||||
"url"
|
||||
],
|
||||
@@ -3540,8 +3532,9 @@
|
||||
{
|
||||
"name": "op",
|
||||
"type": "str",
|
||||
"default": "~/Pictures/chatgpt",
|
||||
"required": false,
|
||||
"help": "Output directory (default: ~/Pictures/chatgpt)"
|
||||
"help": "Output directory"
|
||||
},
|
||||
{
|
||||
"name": "sd",
|
||||
@@ -4388,11 +4381,10 @@
|
||||
"type": "str",
|
||||
"default": "instant",
|
||||
"required": false,
|
||||
"help": "Model to use: instant, expert, or vision",
|
||||
"help": "Model to use: instant or expert",
|
||||
"choices": [
|
||||
"instant",
|
||||
"expert",
|
||||
"vision"
|
||||
"expert"
|
||||
]
|
||||
},
|
||||
{
|
||||
@@ -6855,65 +6847,6 @@
|
||||
"sourceFile": "facebook/join-group.js",
|
||||
"navigateBefore": "https://www.facebook.com"
|
||||
},
|
||||
{
|
||||
"site": "facebook",
|
||||
"name": "marketplace-inbox",
|
||||
"description": "List recent Facebook Marketplace buyer/seller conversations",
|
||||
"domain": "www.facebook.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 20,
|
||||
"required": false,
|
||||
"help": "Number of conversations to return"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"index",
|
||||
"buyer",
|
||||
"listing",
|
||||
"snippet",
|
||||
"time",
|
||||
"unread"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "facebook/marketplace-inbox.js",
|
||||
"sourceFile": "facebook/marketplace-inbox.js",
|
||||
"navigateBefore": "https://www.facebook.com"
|
||||
},
|
||||
{
|
||||
"site": "facebook",
|
||||
"name": "marketplace-listings",
|
||||
"description": "List your Facebook Marketplace seller listings",
|
||||
"domain": "www.facebook.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 20,
|
||||
"required": false,
|
||||
"help": "Number of listings to return"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"index",
|
||||
"title",
|
||||
"price",
|
||||
"status",
|
||||
"listed",
|
||||
"clicks",
|
||||
"actions"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "facebook/marketplace-listings.js",
|
||||
"sourceFile": "facebook/marketplace-listings.js",
|
||||
"navigateBefore": "https://www.facebook.com"
|
||||
},
|
||||
{
|
||||
"site": "facebook",
|
||||
"name": "memories",
|
||||
@@ -7458,86 +7391,6 @@
|
||||
"modulePath": "google/trends.js",
|
||||
"sourceFile": "google/trends.js"
|
||||
},
|
||||
{
|
||||
"site": "google-scholar",
|
||||
"name": "cite",
|
||||
"description": "Get citation for a Google Scholar paper",
|
||||
"domain": "scholar.google.com",
|
||||
"strategy": "public",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "query",
|
||||
"type": "str",
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "Paper title to search for"
|
||||
},
|
||||
{
|
||||
"name": "style",
|
||||
"type": "str",
|
||||
"default": "bibtex",
|
||||
"required": false,
|
||||
"help": "Citation format",
|
||||
"choices": [
|
||||
"bibtex",
|
||||
"endnote",
|
||||
"refman",
|
||||
"refworks"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "index",
|
||||
"type": "int",
|
||||
"default": 1,
|
||||
"required": false,
|
||||
"help": "Which search result to cite (1-based)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"title",
|
||||
"format",
|
||||
"citation"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "google-scholar/cite.js",
|
||||
"sourceFile": "google-scholar/cite.js",
|
||||
"navigateBefore": false
|
||||
},
|
||||
{
|
||||
"site": "google-scholar",
|
||||
"name": "profile",
|
||||
"description": "View a Google Scholar author profile",
|
||||
"domain": "scholar.google.com",
|
||||
"strategy": "public",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "author",
|
||||
"type": "str",
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "Author name or Scholar user ID (e.g. JicYPdAAAAAJ)"
|
||||
},
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 10,
|
||||
"required": false,
|
||||
"help": "Max papers to show (max 20)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"rank",
|
||||
"title",
|
||||
"cited",
|
||||
"year"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "google-scholar/profile.js",
|
||||
"sourceFile": "google-scholar/profile.js",
|
||||
"navigateBefore": false
|
||||
},
|
||||
{
|
||||
"site": "google-scholar",
|
||||
"name": "search",
|
||||
@@ -9181,7 +9034,7 @@
|
||||
{
|
||||
"site": "jd",
|
||||
"name": "item",
|
||||
"description": "京东商品详情(价格、店铺、规格参数、主图、详情图)",
|
||||
"description": "京东商品详情(价格、店铺、规格参数、AVIF 图片)",
|
||||
"domain": "item.jd.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
@@ -9196,9 +9049,9 @@
|
||||
{
|
||||
"name": "images",
|
||||
"type": "int",
|
||||
"default": 200,
|
||||
"default": 10,
|
||||
"required": false,
|
||||
"help": "图片数量上限(默认200)"
|
||||
"help": "AVIF 图片数量上限(默认10)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
@@ -9206,8 +9059,7 @@
|
||||
"price",
|
||||
"shop",
|
||||
"specs",
|
||||
"mainImages",
|
||||
"detailImages"
|
||||
"avifImages"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "jd/item.js",
|
||||
@@ -16085,7 +15937,7 @@
|
||||
"name": "following",
|
||||
"description": "Get accounts a Twitter/X user is following",
|
||||
"domain": "x.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "intercept",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -16112,7 +15964,7 @@
|
||||
"type": "js",
|
||||
"modulePath": "twitter/following.js",
|
||||
"sourceFile": "twitter/following.js",
|
||||
"navigateBefore": "https://x.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "twitter",
|
||||
@@ -16831,8 +16683,7 @@
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "uiverse/code.js",
|
||||
"sourceFile": "uiverse/code.js",
|
||||
"navigateBefore": "https://uiverse.io"
|
||||
"sourceFile": "uiverse/code.js"
|
||||
},
|
||||
{
|
||||
"site": "uiverse",
|
||||
@@ -16872,8 +16723,7 @@
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "uiverse/preview.js",
|
||||
"sourceFile": "uiverse/preview.js",
|
||||
"navigateBefore": "https://uiverse.io"
|
||||
"sourceFile": "uiverse/preview.js"
|
||||
},
|
||||
{
|
||||
"site": "v2ex",
|
||||
@@ -17250,42 +17100,6 @@
|
||||
"required": false,
|
||||
"help": "Seconds to wait after page load"
|
||||
},
|
||||
{
|
||||
"name": "wait-for",
|
||||
"type": "str",
|
||||
"required": false,
|
||||
"valueRequired": true,
|
||||
"help": "CSS selector to wait for in the main document or same-origin iframes"
|
||||
},
|
||||
{
|
||||
"name": "wait-until",
|
||||
"type": "str",
|
||||
"default": "domstable",
|
||||
"required": false,
|
||||
"help": "Readiness policy after navigation: domstable or networkidle",
|
||||
"choices": [
|
||||
"domstable",
|
||||
"networkidle"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "frames",
|
||||
"type": "str",
|
||||
"default": "same-origin",
|
||||
"required": false,
|
||||
"help": "Iframe handling mode: same-origin or none",
|
||||
"choices": [
|
||||
"same-origin",
|
||||
"none"
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "diagnose",
|
||||
"type": "boolean",
|
||||
"default": false,
|
||||
"required": false,
|
||||
"help": "Print render diagnostics (frames, empty containers, XHR/API-like requests) to stderr"
|
||||
},
|
||||
{
|
||||
"name": "stdout",
|
||||
"type": "boolean",
|
||||
@@ -20386,7 +20200,7 @@
|
||||
"name": "answer",
|
||||
"description": "Answer a Zhihu question",
|
||||
"domain": "www.zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "ui",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -20429,86 +20243,14 @@
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/answer.js",
|
||||
"sourceFile": "zhihu/answer.js",
|
||||
"navigateBefore": "https://www.zhihu.com"
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
"name": "collection",
|
||||
"description": "知乎收藏夹内容列表(需要登录)",
|
||||
"domain": "www.zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "id",
|
||||
"type": "str",
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "收藏夹 ID (数字,可从收藏夹 URL 中获取)"
|
||||
},
|
||||
{
|
||||
"name": "offset",
|
||||
"type": "int",
|
||||
"default": 0,
|
||||
"required": false,
|
||||
"help": "起始偏移量(用于分页)"
|
||||
},
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 20,
|
||||
"required": false,
|
||||
"help": "每页数量(最大 20)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"rank",
|
||||
"type",
|
||||
"title",
|
||||
"author",
|
||||
"votes",
|
||||
"excerpt",
|
||||
"url"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/collection.js",
|
||||
"sourceFile": "zhihu/collection.js",
|
||||
"navigateBefore": "https://www.zhihu.com"
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
"name": "collections",
|
||||
"description": "知乎收藏夹列表(需要登录)",
|
||||
"domain": "www.zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 20,
|
||||
"required": false,
|
||||
"help": "每页数量(最大 20)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"rank",
|
||||
"title",
|
||||
"item_count",
|
||||
"description",
|
||||
"collection_id"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/collections.js",
|
||||
"sourceFile": "zhihu/collections.js",
|
||||
"navigateBefore": "https://www.zhihu.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
"name": "comment",
|
||||
"description": "Create a top-level comment on a Zhihu answer or article",
|
||||
"domain": "zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "ui",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -20545,12 +20287,13 @@
|
||||
"target_type",
|
||||
"target",
|
||||
"author_identity",
|
||||
"created_url"
|
||||
"created_url",
|
||||
"created_proof"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/comment.js",
|
||||
"sourceFile": "zhihu/comment.js",
|
||||
"navigateBefore": "https://zhihu.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
@@ -20598,7 +20341,7 @@
|
||||
"name": "favorite",
|
||||
"description": "Favorite a Zhihu answer or article into a specific collection",
|
||||
"domain": "zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "ui",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -20639,14 +20382,14 @@
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/favorite.js",
|
||||
"sourceFile": "zhihu/favorite.js",
|
||||
"navigateBefore": "https://zhihu.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
"name": "follow",
|
||||
"description": "Follow a Zhihu user or question",
|
||||
"domain": "www.zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "ui",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -20673,7 +20416,7 @@
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/follow.js",
|
||||
"sourceFile": "zhihu/follow.js",
|
||||
"navigateBefore": "https://www.zhihu.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
@@ -20707,7 +20450,7 @@
|
||||
"name": "like",
|
||||
"description": "Like a Zhihu answer or article",
|
||||
"domain": "zhihu.com",
|
||||
"strategy": "cookie",
|
||||
"strategy": "ui",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
@@ -20734,7 +20477,7 @@
|
||||
"type": "js",
|
||||
"modulePath": "zhihu/like.js",
|
||||
"sourceFile": "zhihu/like.js",
|
||||
"navigateBefore": "https://zhihu.com"
|
||||
"navigateBefore": true
|
||||
},
|
||||
{
|
||||
"site": "zhihu",
|
||||
@@ -20806,67 +20549,6 @@
|
||||
"sourceFile": "zhihu/search.js",
|
||||
"navigateBefore": "https://www.zhihu.com"
|
||||
},
|
||||
{
|
||||
"site": "zlibrary",
|
||||
"name": "info",
|
||||
"description": "Get book details and available download formats from a Z-Library book page",
|
||||
"domain": "z-library.im",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "url",
|
||||
"type": "str",
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "Z-Library book page URL (e.g. https://z-library.im/book/...)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"title",
|
||||
"pdf",
|
||||
"epub",
|
||||
"url"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "zlibrary/info.js",
|
||||
"sourceFile": "zlibrary/info.js",
|
||||
"navigateBefore": false
|
||||
},
|
||||
{
|
||||
"site": "zlibrary",
|
||||
"name": "search",
|
||||
"description": "Search Z-Library for books by title, author, ISBN, or keyword",
|
||||
"domain": "z-library.im",
|
||||
"strategy": "cookie",
|
||||
"browser": true,
|
||||
"args": [
|
||||
{
|
||||
"name": "query",
|
||||
"type": "str",
|
||||
"required": true,
|
||||
"positional": true,
|
||||
"help": "Search keyword (title, author, ISBN, etc.)"
|
||||
},
|
||||
{
|
||||
"name": "limit",
|
||||
"type": "int",
|
||||
"default": 10,
|
||||
"required": false,
|
||||
"help": "Max results (1–25)"
|
||||
}
|
||||
],
|
||||
"columns": [
|
||||
"rank",
|
||||
"title",
|
||||
"author",
|
||||
"url"
|
||||
],
|
||||
"type": "js",
|
||||
"modulePath": "zlibrary/search.js",
|
||||
"sourceFile": "zlibrary/search.js",
|
||||
"navigateBefore": false
|
||||
},
|
||||
{
|
||||
"site": "zsxq",
|
||||
"name": "dynamics",
|
||||
@@ -21052,4 +20734,4 @@
|
||||
"sourceFile": "zsxq/topics.js",
|
||||
"navigateBefore": "https://wx.zsxq.com"
|
||||
}
|
||||
]
|
||||
]
|
||||
+1
-1
@@ -12,7 +12,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles (max 50)' },
|
||||
],
|
||||
columns: ['rank', 'title', 'summary', 'date', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const count = Math.min(kwargs.limit || 20, 50);
|
||||
const resp = await fetch('https://www.36kr.com/feed', {
|
||||
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; opencli/1.0)' },
|
||||
|
||||
@@ -24,7 +24,7 @@ describe('apple-podcasts search command', () => {
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
query: 'machine learning',
|
||||
keyword: 'sports',
|
||||
limit: 5,
|
||||
@@ -60,7 +60,7 @@ describe('apple-podcasts top command', () => {
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
await cmd.func({ country: 'US', limit: 1 });
|
||||
await cmd.func(null, { country: 'US', limit: 1 });
|
||||
const [, options] = fetchMock.mock.calls[0] ?? [];
|
||||
expect(options).toBeDefined();
|
||||
expect(options.signal).toBeDefined();
|
||||
@@ -81,7 +81,7 @@ describe('apple-podcasts top command', () => {
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
const result = await cmd.func({ country: 'US', limit: 2 });
|
||||
const result = await cmd.func(null, { country: 'US', limit: 2 });
|
||||
expect(fetchMock).toHaveBeenCalledWith('https://rss.marketingtools.apple.com/api/v2/us/podcasts/top/2/podcasts.json', expect.objectContaining({
|
||||
signal: expect.any(Object),
|
||||
}));
|
||||
@@ -94,6 +94,6 @@ describe('apple-podcasts top command', () => {
|
||||
const cmd = getRegistry().get('apple-podcasts/top');
|
||||
expect(cmd?.func).toBeTypeOf('function');
|
||||
vi.stubGlobal('fetch', vi.fn().mockRejectedValue(new Error('socket hang up')));
|
||||
await expect(cmd.func({ country: 'us', limit: 3 })).rejects.toThrow('Unable to reach Apple Podcasts charts for US');
|
||||
await expect(cmd.func(null, { country: 'us', limit: 3 })).rejects.toThrow('Unable to reach Apple Podcasts charts for US');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -12,7 +12,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 15, help: 'Max episodes to show' },
|
||||
],
|
||||
columns: ['title', 'duration', 'date'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 200));
|
||||
// results[0] is the podcast itself; the rest are episodes
|
||||
const data = await itunesFetch(`/lookup?id=${args.id}&entity=podcastEpisode&limit=${limit + 1}`);
|
||||
|
||||
@@ -12,7 +12,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Max results' },
|
||||
],
|
||||
columns: ['id', 'title', 'author', 'episodes', 'genre', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const term = encodeURIComponent(args.query);
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 25));
|
||||
const data = await itunesFetch(`/search?term=${term}&media=podcast&limit=${limit}`);
|
||||
|
||||
@@ -14,7 +14,7 @@ cli({
|
||||
{ name: 'country', default: 'us', help: 'Country code (e.g. us, cn, gb, jp)' },
|
||||
],
|
||||
columns: ['rank', 'title', 'author', 'id'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 100));
|
||||
const country = String(args.country || 'us').trim().toLowerCase();
|
||||
const url = `${CHARTS_URL}/${country}/podcasts/top/${limit}/podcasts.json`;
|
||||
|
||||
+1
-1
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'id', positional: true, required: true, help: 'arXiv paper ID (e.g. 1706.03762)' },
|
||||
],
|
||||
columns: ['id', 'title', 'authors', 'published', 'abstract', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const xml = await arxivFetch(`id_list=${encodeURIComponent(args.id)}`);
|
||||
const entries = parseEntries(xml);
|
||||
if (!entries.length)
|
||||
|
||||
@@ -12,7 +12,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Max results (max 25)' },
|
||||
],
|
||||
columns: ['id', 'title', 'authors', 'published', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 25));
|
||||
const query = encodeURIComponent(`all:${args.query}`);
|
||||
const xml = await arxivFetch(`search_query=${query}&max_results=${limit}&sortBy=relevance`);
|
||||
|
||||
+1
-1
@@ -12,7 +12,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of headlines (max 50)' },
|
||||
],
|
||||
columns: ['rank', 'title', 'description', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (page, kwargs) => {
|
||||
const count = Math.min(kwargs.limit || 20, 50);
|
||||
const resp = await fetch('https://feeds.bbci.co.uk/news/rss.xml');
|
||||
if (!resp.ok)
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('businessweek', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('economics', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('industries', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('main', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('markets', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('opinions', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('politics', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -11,7 +11,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 1, help: 'Number of feed items to return (max 20)' },
|
||||
],
|
||||
columns: ['title', 'summary', 'link', 'mediaLinks'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
return fetchBloombergFeed('tech', kwargs.limit ?? 1);
|
||||
},
|
||||
});
|
||||
|
||||
+8
-49
@@ -2,7 +2,6 @@
|
||||
* BOSS直聘 job search — browser cookie API.
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError } from '@jackwener/opencli/errors';
|
||||
import { requirePage, navigateTo, bossFetch, verbose } from './utils.js';
|
||||
/** City name → BOSS Zhipin city code mapping */
|
||||
const CITY_CODES = {
|
||||
@@ -23,16 +22,8 @@ const CITY_CODES = {
|
||||
'香港': '101320100',
|
||||
};
|
||||
const EXP_MAP = {
|
||||
'不限': '0',
|
||||
'在校/应届': '108',
|
||||
'在校生': '108', '在校': '108',
|
||||
'应届生': '102', '应届': '102',
|
||||
'经验不限': '101',
|
||||
'1年以内': '103',
|
||||
'1-3年': '104',
|
||||
'3-5年': '105',
|
||||
'5-10年': '106',
|
||||
'10年以上': '107',
|
||||
'不限': '0', '在校/应届': '108', '应届': '108', '1年以内': '101',
|
||||
'1-3年': '102', '3-5年': '103', '5-10年': '104', '10年以上': '105',
|
||||
};
|
||||
const DEGREE_MAP = {
|
||||
'不限': '0', '初中及以下': '209', '中专/中技': '208', '高中': '206',
|
||||
@@ -47,10 +38,6 @@ const INDUSTRY_MAP = {
|
||||
'人工智能': '100901', '大数据': '100902', '金融': '100101',
|
||||
'教育培训': '100200', '医疗健康': '100300',
|
||||
};
|
||||
const JOB_TYPE_MAP = {
|
||||
'不限': '0', '全职': '1901', '实习': '1902', '兼职': '1903',
|
||||
};
|
||||
const JOB_TYPE_CODES = new Set(Object.values(JOB_TYPE_MAP));
|
||||
function resolveCity(input) {
|
||||
if (!input)
|
||||
return '101010100';
|
||||
@@ -75,54 +62,35 @@ function resolveMap(input, map) {
|
||||
}
|
||||
return input;
|
||||
}
|
||||
function resolveJobType(input) {
|
||||
if (!input)
|
||||
return '';
|
||||
if (JOB_TYPE_MAP[input] !== undefined)
|
||||
return JOB_TYPE_MAP[input];
|
||||
if (JOB_TYPE_CODES.has(input))
|
||||
return input;
|
||||
throw new ArgumentError(`Invalid jobType: ${input}`, 'Use one of: 全职, 兼职, 实习, 不限');
|
||||
}
|
||||
function formatBossOnline(value) {
|
||||
if (value === true)
|
||||
return 'Y';
|
||||
if (value === false)
|
||||
return 'N';
|
||||
return '';
|
||||
}
|
||||
cli({
|
||||
site: 'boss',
|
||||
name: 'search',
|
||||
description: 'BOSS直聘搜索职位(不带关键词时返回为你推荐职位)',
|
||||
description: 'BOSS直聘搜索职位',
|
||||
domain: 'www.zhipin.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
navigateBefore: false,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'query', positional: true, help: 'Search keyword (optional, empty = recommended jobs)' },
|
||||
{ name: 'query', required: true, positional: true, help: 'Search keyword (e.g. AI agent, 前端)' },
|
||||
{ name: 'city', default: '北京', help: 'City name or code (e.g. 杭州, 上海, 101010100)' },
|
||||
{ name: 'experience', default: '', help: 'Experience: 在校生(实习)/应届生(校招)/经验不限/1年以内/1-3年/3-5年/5-10年/10年以上' },
|
||||
{ name: 'experience', default: '', help: 'Experience: 应届/1年以内/1-3年/3-5年/5-10年/10年以上' },
|
||||
{ name: 'degree', default: '', help: 'Degree: 大专/本科/硕士/博士' },
|
||||
{ name: 'salary', default: '', help: 'Salary: 3K以下/3-5K/5-10K/10-15K/15-20K/20-30K/30-50K/50K以上' },
|
||||
{ name: 'industry', default: '', help: 'Industry code or name (e.g. 100020, 互联网)' },
|
||||
{ name: 'jobType', default: '', help: 'Job type: 全职/兼职/实习(不传=不限,混合校招与实习)' },
|
||||
{ name: 'page', type: 'int', default: 1, help: 'Page number' },
|
||||
{ name: 'limit', type: 'int', default: 15, help: 'Number of results' },
|
||||
],
|
||||
columns: ['name', 'salary', 'company', 'area', 'experience', 'degree', 'skills', 'boss', 'bossOnline', 'security_id', 'url'],
|
||||
columns: ['name', 'salary', 'company', 'area', 'experience', 'degree', 'skills', 'boss', 'security_id', 'url'],
|
||||
func: async (page, kwargs) => {
|
||||
requirePage(page);
|
||||
const query = String(kwargs.query ?? '').trim();
|
||||
const cityCode = resolveCity(kwargs.city);
|
||||
verbose('Navigating to set referrer context...');
|
||||
await navigateTo(page, `https://www.zhipin.com/web/geek/job?query=${encodeURIComponent(query)}&city=${cityCode}`);
|
||||
await navigateTo(page, `https://www.zhipin.com/web/geek/job?query=${encodeURIComponent(kwargs.query)}&city=${cityCode}`);
|
||||
await new Promise(r => setTimeout(r, 1000));
|
||||
const expVal = resolveMap(kwargs.experience, EXP_MAP);
|
||||
const degreeVal = resolveMap(kwargs.degree, DEGREE_MAP);
|
||||
const salaryVal = resolveMap(kwargs.salary, SALARY_MAP);
|
||||
const industryVal = resolveMap(kwargs.industry, INDUSTRY_MAP);
|
||||
const jobTypeVal = resolveJobType(kwargs.jobType);
|
||||
const limit = kwargs.limit || 15;
|
||||
let currentPage = kwargs.page || 1;
|
||||
let allJobs = [];
|
||||
@@ -133,7 +101,7 @@ cli({
|
||||
}
|
||||
const qs = new URLSearchParams({
|
||||
scene: '1',
|
||||
query,
|
||||
query: kwargs.query,
|
||||
city: cityCode,
|
||||
page: String(currentPage),
|
||||
pageSize: '15',
|
||||
@@ -146,8 +114,6 @@ cli({
|
||||
qs.set('salary', salaryVal);
|
||||
if (industryVal)
|
||||
qs.set('industry', industryVal);
|
||||
if (jobTypeVal)
|
||||
qs.set('jobType', jobTypeVal);
|
||||
const targetUrl = `https://www.zhipin.com/wapi/zpgeek/search/joblist.json?${qs.toString()}`;
|
||||
verbose(`Fetching page ${currentPage}... (current jobs: ${allJobs.length})`);
|
||||
const data = await bossFetch(page, targetUrl);
|
||||
@@ -169,7 +135,6 @@ cli({
|
||||
degree: j.jobDegree,
|
||||
skills: (j.skills || []).join(','),
|
||||
boss: j.bossName + ' · ' + j.bossTitle,
|
||||
bossOnline: formatBossOnline(j.bossOnline),
|
||||
security_id: j.securityId || '',
|
||||
url: 'https://www.zhipin.com/job_detail/' + j.encryptJobId + '.html',
|
||||
});
|
||||
@@ -188,9 +153,3 @@ cli({
|
||||
return allJobs;
|
||||
},
|
||||
});
|
||||
export const __test__ = {
|
||||
EXP_MAP,
|
||||
resolveMap,
|
||||
resolveJobType,
|
||||
formatBossOnline,
|
||||
};
|
||||
|
||||
@@ -1,78 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError } from '@jackwener/opencli/errors';
|
||||
import { __test__ } from './search.js';
|
||||
import './search.js';
|
||||
|
||||
function createPageMock(response) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue(response),
|
||||
};
|
||||
}
|
||||
|
||||
describe('boss search', () => {
|
||||
const command = getRegistry().get('boss/search');
|
||||
|
||||
it('keeps legacy 在校/应届 experience input compatible', () => {
|
||||
expect(__test__.resolveMap('在校/应届', __test__.EXP_MAP)).toBe('108');
|
||||
expect(__test__.resolveMap('应届', __test__.EXP_MAP)).toBe('102');
|
||||
});
|
||||
|
||||
it('fails fast on invalid jobType values', async () => {
|
||||
expect(() => __test__.resolveJobType('外包')).toThrow(ArgumentError);
|
||||
});
|
||||
|
||||
it('accepts supported jobType labels and raw codes', () => {
|
||||
expect(__test__.resolveJobType('全职')).toBe('1901');
|
||||
expect(__test__.resolveJobType('实习')).toBe('1902');
|
||||
expect(__test__.resolveJobType('兼职')).toBe('1903');
|
||||
expect(__test__.resolveJobType('1902')).toBe('1902');
|
||||
});
|
||||
|
||||
it('keeps empty query empty and sends jobType filter to the API', async () => {
|
||||
const page = createPageMock({
|
||||
code: 0,
|
||||
zpData: {
|
||||
hasMore: false,
|
||||
jobList: [
|
||||
{
|
||||
encryptJobId: 'abc',
|
||||
securityId: 'sec',
|
||||
jobName: '前端开发实习生',
|
||||
salaryDesc: '150-200/天',
|
||||
brandName: 'OpenCLI',
|
||||
cityName: '北京',
|
||||
areaDistrict: '海淀区',
|
||||
businessDistrict: '',
|
||||
jobExperience: '在校/应届',
|
||||
jobDegree: '本科',
|
||||
skills: ['JavaScript'],
|
||||
bossName: '张三',
|
||||
bossTitle: '技术负责人',
|
||||
bossOnline: false,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
|
||||
const rows = await command.func(page, {
|
||||
query: undefined,
|
||||
city: '北京',
|
||||
jobType: '实习',
|
||||
limit: 1,
|
||||
page: 1,
|
||||
});
|
||||
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.zhipin.com/web/geek/job?query=&city=101010100');
|
||||
const fetchScript = page.evaluate.mock.calls.at(-1)[0];
|
||||
expect(fetchScript).toContain('query=');
|
||||
expect(fetchScript).not.toContain('query=undefined');
|
||||
expect(fetchScript).toContain('jobType=1902');
|
||||
expect(rows[0]).toMatchObject({
|
||||
name: '前端开发实习生',
|
||||
bossOnline: 'N',
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -15,7 +15,7 @@ export const askCommand = cli({
|
||||
{ name: 'timeout', required: false, help: 'Max seconds to wait for response (default: 30)', default: '30' },
|
||||
],
|
||||
columns: ['Role', 'Text'],
|
||||
func: async (kwargs) => {
|
||||
func: async (page, kwargs) => {
|
||||
if (process.platform !== 'darwin') {
|
||||
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
|
||||
}
|
||||
|
||||
@@ -156,7 +156,7 @@ guard s(input, kAXValueAttribute as String) == text else {
|
||||
exit(1)
|
||||
}
|
||||
|
||||
guard let sendButton = findByDescriptions(win, ["发送", "傳送", "Send"]) else {
|
||||
guard let sendButton = findByDescriptions(win, ["发送", "Send"]) else {
|
||||
fputs("Could not find send button\\n", stderr)
|
||||
exit(1)
|
||||
}
|
||||
@@ -240,11 +240,10 @@ let args = CommandLine.arguments
|
||||
let target = args.count > 1 ? args[1] : ""
|
||||
let needsLegacy = args.count > 2 && args[2] == "legacy"
|
||||
|
||||
// Step 1: Click the "Options" button to open the popover (support English, Simplified and Traditional Chinese UI)
|
||||
// Step 1: Click the "Options" button to open the popover (support both English and Chinese UI)
|
||||
var optionsBtn: AXUIElement? = nil
|
||||
if let btn = findByDesc(win, "Options") { optionsBtn = btn }
|
||||
else if let btn = findByDesc(win, "选项") { optionsBtn = btn }
|
||||
else if let btn = findByDesc(win, "選項") { optionsBtn = btn }
|
||||
guard let options = optionsBtn else {
|
||||
fputs("Could not find Options button\\n", stderr); exit(1)
|
||||
}
|
||||
@@ -380,6 +379,5 @@ export function getVisibleChatMessages() {
|
||||
}
|
||||
export const __test__ = {
|
||||
AX_SEND_SCRIPT,
|
||||
AX_MODEL_SCRIPT,
|
||||
AX_GENERATING_SCRIPT,
|
||||
};
|
||||
|
||||
@@ -13,18 +13,6 @@ describe('chatgpt-app AX send script', () => {
|
||||
it('does not report success until the prompt leaves the composer after send', () => {
|
||||
expect(__test__.AX_SEND_SCRIPT).toContain('Prompt did not leave input after pressing send');
|
||||
});
|
||||
|
||||
it('supports english, zh-CN, and zh-TW send button labels', () => {
|
||||
expect(__test__.AX_SEND_SCRIPT).toContain('["发送", "傳送", "Send"]');
|
||||
});
|
||||
});
|
||||
|
||||
describe('chatgpt-app AX model script', () => {
|
||||
it('supports english, zh-CN, and zh-TW options button labels', () => {
|
||||
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "Options")');
|
||||
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "选项")');
|
||||
expect(__test__.AX_MODEL_SCRIPT).toContain('findByDesc(win, "選項")');
|
||||
});
|
||||
});
|
||||
|
||||
describe('chatgpt-app generating detection', () => {
|
||||
|
||||
@@ -12,7 +12,7 @@ export const modelCommand = cli({
|
||||
{ name: 'model', required: true, positional: true, help: 'Model to switch to', choices: MODEL_CHOICES },
|
||||
],
|
||||
columns: ['Status', 'Model'],
|
||||
func: async (kwargs) => {
|
||||
func: async (page, kwargs) => {
|
||||
if (process.platform !== 'darwin') {
|
||||
throw new ConfigError('ChatGPT Desktop integration requires macOS');
|
||||
}
|
||||
|
||||
@@ -10,7 +10,7 @@ export const newCommand = cli({
|
||||
browser: false,
|
||||
args: [],
|
||||
columns: ['Status'],
|
||||
func: async () => {
|
||||
func: async (page) => {
|
||||
if (process.platform !== 'darwin') {
|
||||
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ export const readCommand = cli({
|
||||
browser: false,
|
||||
args: [],
|
||||
columns: ['Role', 'Text'],
|
||||
func: async () => {
|
||||
func: async (page) => {
|
||||
if (process.platform !== 'darwin') {
|
||||
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
|
||||
}
|
||||
|
||||
@@ -13,7 +13,7 @@ export const sendCommand = cli({
|
||||
{ name: 'model', required: false, help: 'Model/mode to use: auto, instant, thinking, 5.2-instant, 5.2-thinking', choices: MODEL_CHOICES },
|
||||
],
|
||||
columns: ['Status'],
|
||||
func: async (kwargs) => {
|
||||
func: async (page, kwargs) => {
|
||||
const text = kwargs.text;
|
||||
const model = kwargs.model;
|
||||
try {
|
||||
|
||||
@@ -10,7 +10,7 @@ export const statusCommand = cli({
|
||||
browser: false,
|
||||
args: [],
|
||||
columns: ['Status'],
|
||||
func: async () => {
|
||||
func: async (page) => {
|
||||
if (process.platform !== 'darwin') {
|
||||
throw new ConfigError('ChatGPT Desktop integration requires macOS (osascript is not available on this platform)');
|
||||
}
|
||||
|
||||
+8
-37
@@ -1,9 +1,7 @@
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import * as fs from 'node:fs';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { saveBase64ToFile } from '@jackwener/opencli/utils';
|
||||
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { getChatGPTVisibleImageUrls, sendChatGPTMessage, waitForChatGPTImages, getChatGPTImageAssets } from './utils.js';
|
||||
|
||||
const CHATGPT_DOMAIN = 'chatgpt.com';
|
||||
@@ -26,22 +24,6 @@ function displayPath(filePath) {
|
||||
return filePath.startsWith(home) ? `~${filePath.slice(home.length)}` : filePath;
|
||||
}
|
||||
|
||||
export function resolveOutputDir(value) {
|
||||
const raw = String(value || '').trim();
|
||||
if (!raw) return path.join(os.homedir(), 'Pictures', 'chatgpt');
|
||||
if (raw === '~') return os.homedir();
|
||||
if (raw.startsWith('~/')) return path.join(os.homedir(), raw.slice(2));
|
||||
return path.resolve(raw);
|
||||
}
|
||||
|
||||
export function nextAvailablePath(dir, baseName, ext, existsSync = fs.existsSync) {
|
||||
let candidate = path.join(dir, `${baseName}${ext}`);
|
||||
for (let index = 1; existsSync(candidate); index += 1) {
|
||||
candidate = path.join(dir, `${baseName}_${index}${ext}`);
|
||||
}
|
||||
return candidate;
|
||||
}
|
||||
|
||||
async function currentChatGPTLink(page) {
|
||||
const url = await page.evaluate('window.location.href').catch(() => '');
|
||||
return typeof url === 'string' && url ? url : 'https://chatgpt.com';
|
||||
@@ -59,13 +41,13 @@ export const imageCommand = cli({
|
||||
timeoutSeconds: 240,
|
||||
args: [
|
||||
{ name: 'prompt', positional: true, required: true, help: 'Image prompt to send to ChatGPT' },
|
||||
{ name: 'op', help: 'Output directory (default: ~/Pictures/chatgpt)' },
|
||||
{ name: 'op', default: '~/Pictures/chatgpt', help: 'Output directory' },
|
||||
{ name: 'sd', type: 'boolean', default: false, help: 'Skip download shorthand; only show ChatGPT link' },
|
||||
],
|
||||
columns: ['status', 'file', 'link'],
|
||||
func: async (page, kwargs) => {
|
||||
const prompt = kwargs.prompt;
|
||||
const outputDir = resolveOutputDir(kwargs.op);
|
||||
const outputDir = kwargs.op || path.join(os.homedir(), 'Pictures', 'chatgpt');
|
||||
const skipDownloadRaw = kwargs.sd;
|
||||
const skipDownload = skipDownloadRaw === '' || skipDownloadRaw === true || normalizeBooleanFlag(skipDownloadRaw);
|
||||
const timeout = 120;
|
||||
@@ -81,23 +63,12 @@ export const imageCommand = cli({
|
||||
return [{ status: '⚠️ send-failed', file: '📁 -', link: `🔗 ${await currentChatGPTLink(page)}` }];
|
||||
}
|
||||
|
||||
// ChatGPT briefly navigates to /c/{id} after sending, then may
|
||||
// redirect back to the home page. Poll until we capture the /c/ URL.
|
||||
let convUrl = '';
|
||||
for (let ci = 0; ci < 10; ci++) {
|
||||
const url = await currentChatGPTLink(page);
|
||||
if (url.includes('/c/')) { convUrl = url; break; }
|
||||
await page.wait(2);
|
||||
}
|
||||
if (!convUrl) {
|
||||
convUrl = await currentChatGPTLink(page);
|
||||
}
|
||||
|
||||
const urls = await waitForChatGPTImages(page, beforeUrls, timeout, convUrl);
|
||||
const link = convUrl;
|
||||
// Wait for response and images
|
||||
const urls = await waitForChatGPTImages(page, beforeUrls, timeout);
|
||||
const link = await currentChatGPTLink(page);
|
||||
|
||||
if (!urls.length) {
|
||||
throw new EmptyResultError('chatgpt image', `No generated images were detected before timeout. Open ${link} and verify whether ChatGPT finished generating the image.`);
|
||||
return [{ status: '⚠️ no-images', file: '📁 -', link: `🔗 ${link}` }];
|
||||
}
|
||||
|
||||
if (skipDownload) {
|
||||
@@ -107,7 +78,7 @@ export const imageCommand = cli({
|
||||
// Export and save images
|
||||
const assets = await getChatGPTImageAssets(page, urls);
|
||||
if (!assets.length) {
|
||||
throw new CommandExecutionError('Failed to export generated ChatGPT image assets', `Open ${link} and verify the generated images are visible, then retry.`);
|
||||
return [{ status: '⚠️ export-failed', file: '📁 -', link: `🔗 ${link}` }];
|
||||
}
|
||||
|
||||
const stamp = Date.now();
|
||||
@@ -117,7 +88,7 @@ export const imageCommand = cli({
|
||||
const base64 = asset.dataUrl.replace(/^data:[^;]+;base64,/, '');
|
||||
const suffix = assets.length > 1 ? `_${index + 1}` : '';
|
||||
const ext = extFromMime(asset.mimeType);
|
||||
const filePath = nextAvailablePath(outputDir, `chatgpt_${stamp}${suffix}`, ext);
|
||||
const filePath = path.join(outputDir, `chatgpt_${stamp}${suffix}${ext}`);
|
||||
await saveBase64ToFile(base64, filePath);
|
||||
results.push({ status: '✅ saved', file: `📁 ${displayPath(filePath)}`, link: `🔗 ${link}` });
|
||||
}
|
||||
|
||||
@@ -1,92 +0,0 @@
|
||||
import * as os from 'node:os';
|
||||
import * as path from 'node:path';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const mocks = vi.hoisted(() => ({
|
||||
getChatGPTVisibleImageUrls: vi.fn(),
|
||||
sendChatGPTMessage: vi.fn(),
|
||||
waitForChatGPTImages: vi.fn(),
|
||||
getChatGPTImageAssets: vi.fn(),
|
||||
saveBase64ToFile: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('./utils.js', () => ({
|
||||
getChatGPTVisibleImageUrls: mocks.getChatGPTVisibleImageUrls,
|
||||
sendChatGPTMessage: mocks.sendChatGPTMessage,
|
||||
waitForChatGPTImages: mocks.waitForChatGPTImages,
|
||||
getChatGPTImageAssets: mocks.getChatGPTImageAssets,
|
||||
}));
|
||||
|
||||
vi.mock('@jackwener/opencli/utils', () => ({
|
||||
saveBase64ToFile: mocks.saveBase64ToFile,
|
||||
}));
|
||||
|
||||
const { imageCommand, nextAvailablePath, resolveOutputDir } = await import('./image.js');
|
||||
|
||||
function createPage() {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue('https://chatgpt.com/c/test-conversation'),
|
||||
};
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
mocks.getChatGPTVisibleImageUrls.mockReset().mockResolvedValue([]);
|
||||
mocks.sendChatGPTMessage.mockReset().mockResolvedValue(true);
|
||||
mocks.waitForChatGPTImages.mockReset().mockResolvedValue(['https://images.example/generated.png']);
|
||||
mocks.getChatGPTImageAssets.mockReset().mockResolvedValue([{
|
||||
url: 'https://images.example/generated.png',
|
||||
dataUrl: 'data:image/png;base64,aGVsbG8=',
|
||||
mimeType: 'image/png',
|
||||
}]);
|
||||
mocks.saveBase64ToFile.mockReset().mockResolvedValue(undefined);
|
||||
});
|
||||
|
||||
describe('chatgpt image output paths', () => {
|
||||
it('expands the default and explicit home-relative output directories', () => {
|
||||
expect(resolveOutputDir()).toBe(path.join(os.homedir(), 'Pictures', 'chatgpt'));
|
||||
expect(resolveOutputDir('~/tmp/chatgpt-images')).toBe(path.join(os.homedir(), 'tmp', 'chatgpt-images'));
|
||||
expect(resolveOutputDir('~')).toBe(os.homedir());
|
||||
});
|
||||
|
||||
it('generates a non-overwriting file path when a timestamp collision exists', () => {
|
||||
const dir = '/tmp/chatgpt';
|
||||
const taken = new Set([
|
||||
path.join(dir, 'chatgpt_123.png'),
|
||||
path.join(dir, 'chatgpt_123_1.png'),
|
||||
]);
|
||||
|
||||
expect(nextAvailablePath(dir, 'chatgpt_123', '.png', (file) => taken.has(file))).toBe(path.join(dir, 'chatgpt_123_2.png'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('chatgpt image failure contracts', () => {
|
||||
it('fails fast when image generation detection finds no new images', async () => {
|
||||
mocks.waitForChatGPTImages.mockResolvedValue([]);
|
||||
|
||||
await expect(imageCommand.func(createPage(), {
|
||||
prompt: 'cat',
|
||||
op: '',
|
||||
sd: false,
|
||||
})).rejects.toMatchObject({
|
||||
code: 'EMPTY_RESULT',
|
||||
message: expect.stringContaining('chatgpt image returned no data'),
|
||||
hint: expect.stringContaining('No generated images were detected'),
|
||||
});
|
||||
});
|
||||
|
||||
it('fails fast when generated image assets cannot be exported', async () => {
|
||||
mocks.getChatGPTImageAssets.mockResolvedValue([]);
|
||||
|
||||
await expect(imageCommand.func(createPage(), {
|
||||
prompt: 'cat',
|
||||
op: '',
|
||||
sd: false,
|
||||
})).rejects.toMatchObject({
|
||||
code: 'COMMAND_EXEC',
|
||||
message: expect.stringContaining('Failed to export generated ChatGPT image assets'),
|
||||
});
|
||||
});
|
||||
});
|
||||
+6
-39
@@ -7,21 +7,11 @@ export const CHATGPT_DOMAIN = 'chatgpt.com';
|
||||
export const CHATGPT_URL = 'https://chatgpt.com';
|
||||
|
||||
// Selectors
|
||||
const COMPOSER_SELECTORS = [
|
||||
'[aria-label="Chat with ChatGPT"]',
|
||||
'[placeholder="Ask anything"]',
|
||||
'#prompt-textarea',
|
||||
];
|
||||
const COMPOSER_SELECTOR = '[aria-label="Chat with ChatGPT"]';
|
||||
const SEND_BTN_SELECTOR = 'button[aria-label="Send prompt"]';
|
||||
|
||||
function isSameChatGPTConversation(currentUrl, expectedUrl) {
|
||||
if (!currentUrl || !expectedUrl) return false;
|
||||
return currentUrl === expectedUrl
|
||||
|| currentUrl.startsWith(`${expectedUrl}?`)
|
||||
|| currentUrl.startsWith(`${expectedUrl}#`);
|
||||
}
|
||||
|
||||
function buildComposerLocatorScript() {
|
||||
const selectorsJson = JSON.stringify([COMPOSER_SELECTOR]);
|
||||
const markerAttr = 'data-opencli-chatgpt-composer';
|
||||
return `
|
||||
const isVisible = (el) => {
|
||||
@@ -43,7 +33,7 @@ function buildComposerLocatorScript() {
|
||||
const marked = document.querySelector('[' + markerAttr + '="1"]');
|
||||
if (marked instanceof HTMLElement && isVisible(marked)) return marked;
|
||||
|
||||
for (const selector of ${JSON.stringify(COMPOSER_SELECTORS)}) {
|
||||
for (const selector of ${JSON.stringify([COMPOSER_SELECTOR])}) {
|
||||
const node = Array.from(document.querySelectorAll(selector)).find(c => c instanceof HTMLElement && isVisible(c));
|
||||
if (node instanceof HTMLElement) {
|
||||
node.setAttribute(markerAttr, '1');
|
||||
@@ -99,9 +89,7 @@ export async function sendChatGPTMessage(page, text) {
|
||||
// Fallback: use execCommand
|
||||
await page.evaluate(`
|
||||
(() => {
|
||||
var composer = null;
|
||||
var sels = ${JSON.stringify(COMPOSER_SELECTORS)};
|
||||
for (var si = 0; si < sels.length; si++) { composer = document.querySelector(sels[si]); if (composer) break; }
|
||||
const composer = document.querySelector('[aria-label="Chat with ChatGPT"]');
|
||||
if (!composer) return;
|
||||
composer.focus();
|
||||
document.execCommand('insertText', false, ${JSON.stringify(text)});
|
||||
@@ -193,7 +181,7 @@ export async function getChatGPTVisibleImageUrls(page) {
|
||||
/**
|
||||
* Wait for new images to appear after sending a prompt.
|
||||
*/
|
||||
export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, convUrl) {
|
||||
export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds) {
|
||||
const beforeSet = new Set(beforeUrls);
|
||||
const pollIntervalSeconds = 3;
|
||||
const maxPolls = Math.max(1, Math.ceil(timeoutSeconds / pollIntervalSeconds));
|
||||
@@ -203,26 +191,10 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
|
||||
for (let i = 0; i < maxPolls; i++) {
|
||||
await page.wait(i === 0 ? 3 : pollIntervalSeconds);
|
||||
|
||||
let currentUrl = '';
|
||||
if (convUrl && convUrl.includes('/c/')) {
|
||||
currentUrl = await page.evaluate('window.location.href').catch(() => '');
|
||||
if (currentUrl && !isSameChatGPTConversation(currentUrl, convUrl)) {
|
||||
await page.goto(convUrl);
|
||||
await page.wait(3);
|
||||
}
|
||||
}
|
||||
|
||||
// Check if still generating
|
||||
const generating = await isGenerating(page);
|
||||
if (generating) continue;
|
||||
|
||||
if (convUrl && convUrl.includes('/c/') && i > 0 && i % 5 === 0) {
|
||||
const onConversation = !currentUrl || isSameChatGPTConversation(currentUrl, convUrl);
|
||||
if (onConversation) {
|
||||
await page.goto(convUrl);
|
||||
await page.wait(3);
|
||||
}
|
||||
}
|
||||
|
||||
const urls = (await getChatGPTVisibleImageUrls(page)).filter(url => !beforeSet.has(url));
|
||||
if (urls.length === 0) continue;
|
||||
|
||||
@@ -242,11 +214,6 @@ export async function waitForChatGPTImages(page, beforeUrls, timeoutSeconds, con
|
||||
return lastUrls;
|
||||
}
|
||||
|
||||
export const __test__ = {
|
||||
COMPOSER_SELECTORS,
|
||||
isSameChatGPTConversation,
|
||||
};
|
||||
|
||||
/**
|
||||
* Export images by URL: fetch from ChatGPT backend API and convert to base64 data URLs.
|
||||
*/
|
||||
|
||||
@@ -1,63 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { __test__, waitForChatGPTImages } from './utils.js';
|
||||
|
||||
function createPageMock({ location = '', generating = [], imageUrls = [] } = {}) {
|
||||
let generatingIndex = 0;
|
||||
let imageIndex = 0;
|
||||
return {
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn((script) => {
|
||||
if (script === 'window.location.href') return Promise.resolve(location);
|
||||
if (script.includes('Stop generating') || script.includes('Thinking')) {
|
||||
const value = generating[Math.min(generatingIndex, generating.length - 1)] ?? false;
|
||||
generatingIndex += 1;
|
||||
return Promise.resolve(value);
|
||||
}
|
||||
if (script.includes("document.querySelectorAll('img')")) {
|
||||
const value = imageUrls[Math.min(imageIndex, imageUrls.length - 1)] ?? [];
|
||||
imageIndex += 1;
|
||||
return Promise.resolve(value);
|
||||
}
|
||||
return Promise.resolve(undefined);
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
describe('chatgpt image wait contract', () => {
|
||||
it('does not periodically reload the conversation while generation is still active', async () => {
|
||||
const convUrl = 'https://chatgpt.com/c/demo';
|
||||
const page = createPageMock({
|
||||
location: convUrl,
|
||||
generating: [true, true, true, true, true, true],
|
||||
});
|
||||
|
||||
await expect(waitForChatGPTImages(page, [], 18, convUrl)).resolves.toEqual([]);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('jumps back to the captured conversation when the page drifts away', async () => {
|
||||
const convUrl = 'https://chatgpt.com/c/demo';
|
||||
const page = createPageMock({
|
||||
location: 'https://chatgpt.com/',
|
||||
generating: [false],
|
||||
imageUrls: [['https://cdn.openai.com/generated/demo.png']],
|
||||
});
|
||||
|
||||
await expect(waitForChatGPTImages(page, [], 3, convUrl)).resolves.toEqual([
|
||||
'https://cdn.openai.com/generated/demo.png',
|
||||
]);
|
||||
expect(page.goto).toHaveBeenCalledWith(convUrl);
|
||||
});
|
||||
|
||||
it('treats query and hash variants as the same conversation', () => {
|
||||
expect(__test__.isSameChatGPTConversation(
|
||||
'https://chatgpt.com/c/demo?model=gpt-image-1',
|
||||
'https://chatgpt.com/c/demo',
|
||||
)).toBe(true);
|
||||
expect(__test__.isSameChatGPTConversation(
|
||||
'https://chatgpt.com/c/other',
|
||||
'https://chatgpt.com/c/demo',
|
||||
)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -34,7 +34,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 15, help: 'Number of results' },
|
||||
],
|
||||
columns: ['rank', 'name', 'type', 'score', 'price', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const query = String(kwargs.query || '').trim();
|
||||
if (!query) {
|
||||
throw new ArgumentError('Search keyword cannot be empty');
|
||||
|
||||
@@ -25,7 +25,7 @@ describe('ctrip search', () => {
|
||||
],
|
||||
},
|
||||
}), { status: 200 })));
|
||||
const result = await command.func({ query: '苏州', limit: 3 });
|
||||
const result = await command.func(null, { query: '苏州', limit: 3 });
|
||||
expect(result).toEqual([
|
||||
{
|
||||
rank: 1,
|
||||
@@ -46,11 +46,11 @@ describe('ctrip search', () => {
|
||||
]);
|
||||
});
|
||||
it('rejects empty queries', async () => {
|
||||
await expect(command.func({ query: ' ', limit: 3 })).rejects.toThrow('Search keyword cannot be empty');
|
||||
await expect(command.func(null, { query: ' ', limit: 3 })).rejects.toThrow('Search keyword cannot be empty');
|
||||
});
|
||||
it('surfaces fetch failures as CliError', async () => {
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response('{}', { status: 503 })));
|
||||
await expect(command.func({ query: '苏州', limit: 3 })).rejects.toMatchObject({
|
||||
await expect(command.func(null, { query: '苏州', limit: 3 })).rejects.toMatchObject({
|
||||
code: 'FETCH_ERROR',
|
||||
message: 'ctrip search failed with status 503',
|
||||
});
|
||||
@@ -59,6 +59,6 @@ describe('ctrip search', () => {
|
||||
vi.stubGlobal('fetch', vi.fn().mockResolvedValue(new Response(JSON.stringify({
|
||||
Response: { searchResults: [] },
|
||||
}), { status: 200 })));
|
||||
await expect(command.func({ query: '苏州', limit: 3 })).rejects.toThrow('ctrip search returned no data');
|
||||
await expect(command.func(null, { query: '苏州', limit: 3 })).rejects.toThrow('ctrip search returned no data');
|
||||
});
|
||||
});
|
||||
|
||||
+4
-17
@@ -18,7 +18,7 @@ export const askCommand = cli({
|
||||
{ name: 'prompt', positional: true, required: true, help: 'Prompt to send' },
|
||||
{ name: 'timeout', type: 'int', default: 120, help: 'Max seconds to wait for response' },
|
||||
{ name: 'new', type: 'boolean', default: false, help: 'Start a new chat before sending' },
|
||||
{ name: 'model', default: 'instant', choices: ['instant', 'expert', 'vision'], help: 'Model to use: instant, expert, or vision' },
|
||||
{ name: 'model', default: 'instant', choices: ['instant', 'expert'], help: 'Model to use: instant or expert' },
|
||||
{ name: 'think', type: 'boolean', default: false, help: 'Enable DeepThink mode' },
|
||||
{ name: 'search', type: 'boolean', default: false, help: 'Enable web search' },
|
||||
{ name: 'file', help: 'Attach a file (PDF, image, text) with the prompt' },
|
||||
@@ -78,22 +78,9 @@ export const askCommand = cli({
|
||||
throw new CommandExecutionError('Could not enable DeepThink');
|
||||
}
|
||||
|
||||
if (wantModel === 'vision' && wantSearch) {
|
||||
throw new CliError(
|
||||
'ARGUMENT',
|
||||
'DeepSeek vision mode does not support --search.',
|
||||
'Run without --search, or use --model instant/expert for web search.',
|
||||
EXIT_CODES.USAGE_ERROR,
|
||||
);
|
||||
}
|
||||
|
||||
// Vision mode does not have the search toggle.
|
||||
let searchResult;
|
||||
if (wantModel !== 'vision') {
|
||||
searchResult = await withRetry(() => setFeature(page, 'Search', wantSearch));
|
||||
if (!searchResult?.ok && wantSearch) {
|
||||
throw new CommandExecutionError('Could not enable Search');
|
||||
}
|
||||
const searchResult = await withRetry(() => setFeature(page, 'Search', wantSearch));
|
||||
if (!searchResult?.ok && wantSearch) {
|
||||
throw new CommandExecutionError('Could not enable Search');
|
||||
}
|
||||
|
||||
if (thinkResult?.toggled || searchResult?.toggled) await page.wait(0.5);
|
||||
|
||||
@@ -263,50 +263,4 @@ describe('deepseek ask conversation resume', () => {
|
||||
expect(rows).toEqual([{ response: 'follow-up reply' }]);
|
||||
expect(mockSelectModel).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('skips search toggle in vision mode when search is not requested', async () => {
|
||||
mockEnsureOnDeepSeek.mockResolvedValue(false);
|
||||
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
|
||||
mockSetFeature.mockResolvedValue({ ok: true, toggled: false });
|
||||
mockSendMessage.mockResolvedValue({ ok: true });
|
||||
mockGetBubbleCount.mockResolvedValue(0);
|
||||
mockWaitForResponse.mockResolvedValue('vision reply');
|
||||
page.evaluate.mockResolvedValue('https://chat.deepseek.com/');
|
||||
|
||||
const rows = await askCommand.func(page, {
|
||||
prompt: 'describe',
|
||||
timeout: 120,
|
||||
new: false,
|
||||
model: 'vision',
|
||||
think: false,
|
||||
search: false,
|
||||
});
|
||||
|
||||
expect(rows).toEqual([{ response: 'vision reply' }]);
|
||||
expect(mockSetFeature).toHaveBeenCalledTimes(1);
|
||||
expect(mockSetFeature).toHaveBeenCalledWith(expect.anything(), 'DeepThink', false);
|
||||
});
|
||||
|
||||
it('fails fast instead of silently ignoring --search in vision mode', async () => {
|
||||
mockEnsureOnDeepSeek.mockResolvedValue(false);
|
||||
mockSelectModel.mockResolvedValue({ ok: true, toggled: false });
|
||||
page.evaluate.mockResolvedValue('https://chat.deepseek.com/');
|
||||
|
||||
await expect(askCommand.func(page, {
|
||||
prompt: 'describe',
|
||||
timeout: 120,
|
||||
new: false,
|
||||
model: 'vision',
|
||||
think: false,
|
||||
search: true,
|
||||
})).rejects.toMatchObject(new CliError(
|
||||
'ARGUMENT',
|
||||
'DeepSeek vision mode does not support --search.',
|
||||
'Run without --search, or use --model instant/expert for web search.',
|
||||
EXIT_CODES.USAGE_ERROR,
|
||||
));
|
||||
|
||||
expect(mockSendMessage).not.toHaveBeenCalled();
|
||||
expect(mockSendWithFile).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
|
||||
+16
-55
@@ -40,10 +40,9 @@ export async function selectModel(page, modelName) {
|
||||
return page.evaluate(`(() => {
|
||||
var radios = document.querySelectorAll('div[role="radio"]');
|
||||
if (radios.length === 0) return { ok: false };
|
||||
var name = '${modelName}'.toLowerCase();
|
||||
var index = name === 'instant' ? 0 : name === 'expert' ? 1 : name === 'vision' ? 2 : -1;
|
||||
if (index < 0 || index >= radios.length) return { ok: false };
|
||||
var target = radios[index];
|
||||
var isFirst = '${modelName}'.toLowerCase() === 'instant';
|
||||
if (!isFirst && radios.length < 2) return { ok: false };
|
||||
var target = isFirst ? radios[0] : radios[radios.length - 1];
|
||||
var alreadySelected = target.getAttribute('aria-checked') === 'true';
|
||||
if (!alreadySelected) target.click();
|
||||
return { ok: true, toggled: !alreadySelected };
|
||||
@@ -75,18 +74,14 @@ export async function sendMessage(page, prompt) {
|
||||
document.execCommand('insertText', false, ${promptJson});
|
||||
await new Promise(r => setTimeout(r, 800));
|
||||
|
||||
// Find the send button: last non-toggle button in the textarea's container
|
||||
var container = box.parentElement;
|
||||
while (container && !container.querySelector('div[role="button"]')) {
|
||||
container = container.parentElement;
|
||||
}
|
||||
if (container) {
|
||||
var btns = container.querySelectorAll('div[role="button"]:not(.ds-toggle-button)');
|
||||
var sendBtn = btns[btns.length - 1];
|
||||
if (sendBtn && sendBtn.getAttribute('aria-disabled') === 'false'
|
||||
&& sendBtn.querySelectorAll('svg').length > 0) {
|
||||
sendBtn.click();
|
||||
return { ok: true };
|
||||
const btns = document.querySelectorAll('div[role="button"]');
|
||||
for (const btn of btns) {
|
||||
if (btn.getAttribute('aria-disabled') === 'false') {
|
||||
const svgs = btn.querySelectorAll('svg');
|
||||
if (svgs.length > 0 && btn.closest('div')?.querySelector('textarea')) {
|
||||
btn.click();
|
||||
return { ok: true };
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -278,18 +273,9 @@ async function waitForFilePreview(page, fileName) {
|
||||
for (let attempt = 0; attempt < 8; attempt++) {
|
||||
await page.wait(2);
|
||||
const ready = await page.evaluate(`(() => {
|
||||
var name = ${JSON.stringify(fileName)};
|
||||
var hasFileName = Array.from(document.querySelectorAll('div'))
|
||||
.some(function(el) { return el.children.length === 0 && (el.textContent || '').trim() === name; });
|
||||
if (hasFileName) return true;
|
||||
// Vision mode shows an image thumbnail, not filename text. Require
|
||||
// a preview-like node here; send-button readiness is checked later.
|
||||
var box = document.querySelector('${TEXTAREA_SELECTOR}');
|
||||
if (!box) return false;
|
||||
var c = box.parentElement;
|
||||
while (c && !c.querySelector('div[role="button"]')) c = c.parentElement;
|
||||
if (!c) return false;
|
||||
return !!c.querySelector('img[src], canvas, video, [style*="background-image"], [class*="preview"], [class*="upload"]');
|
||||
const name = ${JSON.stringify(fileName)};
|
||||
return Array.from(document.querySelectorAll('div'))
|
||||
.some((el) => el.children.length === 0 && (el.textContent || '').trim() === name);
|
||||
})()`);
|
||||
if (ready) return true;
|
||||
}
|
||||
@@ -328,7 +314,7 @@ export async function sendWithFile(page, filePath, prompt) {
|
||||
uploaded = true;
|
||||
} catch (err) {
|
||||
const msg = String(err?.message || err);
|
||||
if (!msg.includes('Unknown action') && !msg.includes('not supported') && !msg.includes('Not allowed')) {
|
||||
if (!msg.includes('Unknown action') && !msg.includes('not supported')) {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
@@ -355,8 +341,7 @@ export async function sendWithFile(page, filePath, prompt) {
|
||||
}
|
||||
|
||||
inp.files = dt.files;
|
||||
// Use inp.files, not dt.files; assignment transfers ownership
|
||||
inp[propsKey].onChange({ target: { files: inp.files } });
|
||||
inp[propsKey].onChange({ target: { files: dt.files } });
|
||||
return { ok: true };
|
||||
})()`);
|
||||
if (fallbackResult && !fallbackResult.ok) return fallbackResult;
|
||||
@@ -365,30 +350,6 @@ export async function sendWithFile(page, filePath, prompt) {
|
||||
const ready = await waitForFilePreview(page, fileName);
|
||||
if (!ready) return { ok: false, reason: 'file preview did not appear' };
|
||||
|
||||
// File preview appears immediately but send button stays disabled until
|
||||
// the server upload finishes. Wait for it.
|
||||
let sendEnabled = false;
|
||||
for (let tick = 0; tick < 15; tick++) {
|
||||
const enabled = await page.evaluate(`(() => {
|
||||
var box = document.querySelector('${TEXTAREA_SELECTOR}');
|
||||
if (!box) return false;
|
||||
var c = box.parentElement;
|
||||
while (c && !c.querySelector('div[role="button"]')) c = c.parentElement;
|
||||
if (!c) return false;
|
||||
var btns = c.querySelectorAll('div[role="button"]:not(.ds-toggle-button)');
|
||||
var last = btns[btns.length - 1];
|
||||
return !!(last && last.getAttribute('aria-disabled') === 'false');
|
||||
})()`);
|
||||
if (enabled) {
|
||||
sendEnabled = true;
|
||||
break;
|
||||
}
|
||||
await page.wait(1);
|
||||
}
|
||||
if (!sendEnabled) {
|
||||
return { ok: false, reason: 'send button did not enable after upload' };
|
||||
}
|
||||
|
||||
return sendMessage(page, prompt);
|
||||
}
|
||||
|
||||
|
||||
+5
-124
@@ -107,36 +107,15 @@ describe('deepseek sendWithFile', () => {
|
||||
setFileInput: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce(undefined) // sidebar collapse
|
||||
.mockResolvedValueOnce(true) // waitForFilePreview
|
||||
.mockResolvedValueOnce(true) // send button enabled check
|
||||
.mockResolvedValueOnce({ ok: true }), // sendMessage
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockResolvedValueOnce(true)
|
||||
.mockResolvedValueOnce({ ok: true }),
|
||||
};
|
||||
|
||||
const result = await sendWithFile(page, filePath, 'summarize this');
|
||||
|
||||
expect(result).toEqual({ ok: true }); expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[type="file"]');
|
||||
});
|
||||
|
||||
it('fails closed when upload preview appears but send button never enables', async () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
|
||||
tempDirs.push(dir);
|
||||
const filePath = path.join(dir, 'report.txt');
|
||||
fs.writeFileSync(filePath, 'hello');
|
||||
|
||||
const page = {
|
||||
setFileInput: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce(undefined) // sidebar collapse
|
||||
.mockResolvedValueOnce(true) // waitForFilePreview
|
||||
.mockResolvedValue(false), // send button never enables
|
||||
};
|
||||
|
||||
const result = await sendWithFile(page, filePath, 'summarize this');
|
||||
|
||||
expect(result).toEqual({ ok: false, reason: 'send button did not enable after upload' });
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(17);
|
||||
expect(result).toEqual({ ok: true });
|
||||
expect(page.setFileInput).toHaveBeenCalledWith([filePath], 'input[type="file"]');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -163,102 +142,4 @@ describe('deepseek selectModel', () => {
|
||||
expect(result).toEqual({ ok: false });
|
||||
expect(instantRadio.click).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('selects the correct radio for each model', async () => {
|
||||
const radios = [0, 1, 2].map(() => ({
|
||||
getAttribute: vi.fn(() => 'false'),
|
||||
click: vi.fn(),
|
||||
}));
|
||||
global.document = {
|
||||
querySelectorAll: vi.fn(() => radios),
|
||||
};
|
||||
const page = {
|
||||
evaluate: vi.fn(async (script) => eval(script)),
|
||||
};
|
||||
|
||||
await selectModel(page, 'instant');
|
||||
expect(radios[0].click).toHaveBeenCalled();
|
||||
expect(radios[1].click).not.toHaveBeenCalled();
|
||||
expect(radios[2].click).not.toHaveBeenCalled();
|
||||
|
||||
radios.forEach(r => r.click.mockClear());
|
||||
await selectModel(page, 'expert');
|
||||
expect(radios[1].click).toHaveBeenCalled();
|
||||
|
||||
radios.forEach(r => r.click.mockClear());
|
||||
await selectModel(page, 'vision');
|
||||
expect(radios[2].click).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('rejects unknown model names', async () => {
|
||||
const radios = [0, 1, 2].map(() => ({
|
||||
getAttribute: vi.fn(() => 'false'),
|
||||
click: vi.fn(),
|
||||
}));
|
||||
global.document = {
|
||||
querySelectorAll: vi.fn(() => radios),
|
||||
};
|
||||
const page = {
|
||||
evaluate: vi.fn(async (script) => eval(script)),
|
||||
};
|
||||
|
||||
const result = await selectModel(page, 'turbo');
|
||||
expect(result).toEqual({ ok: false });
|
||||
});
|
||||
});
|
||||
|
||||
describe('deepseek sendWithFile Not allowed fallback', () => {
|
||||
const tempDirs = [];
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
while (tempDirs.length) {
|
||||
fs.rmSync(tempDirs.pop(), { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('falls back to DataTransfer when setFileInput throws Not allowed', async () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
|
||||
tempDirs.push(dir);
|
||||
const filePath = path.join(dir, 'image.png');
|
||||
fs.writeFileSync(filePath, 'fake-png');
|
||||
|
||||
const page = {
|
||||
setFileInput: vi.fn().mockRejectedValue(new Error('Not allowed')),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce(undefined) // sidebar collapse
|
||||
.mockResolvedValueOnce({ ok: true }) // DataTransfer fallback
|
||||
.mockResolvedValueOnce(true) // waitForFilePreview
|
||||
.mockResolvedValueOnce(true) // send button enabled
|
||||
.mockResolvedValueOnce({ ok: true }),// sendMessage
|
||||
};
|
||||
|
||||
const result = await sendWithFile(page, filePath, 'describe');
|
||||
|
||||
expect(page.setFileInput).toHaveBeenCalled();
|
||||
expect(page.evaluate).toHaveBeenCalledTimes(5);
|
||||
expect(result).toEqual({ ok: true });
|
||||
});
|
||||
|
||||
it('does not treat send-button enablement alone as image upload proof', async () => {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'opencli-deepseek-'));
|
||||
tempDirs.push(dir);
|
||||
const filePath = path.join(dir, 'image.png');
|
||||
fs.writeFileSync(filePath, 'fake-png');
|
||||
|
||||
const page = {
|
||||
setFileInput: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce(undefined) // sidebar collapse
|
||||
.mockResolvedValue(false), // no filename / thumbnail preview
|
||||
};
|
||||
|
||||
const result = await sendWithFile(page, filePath, 'describe');
|
||||
|
||||
expect(result).toEqual({ ok: false, reason: 'file preview did not appear' });
|
||||
expect(page.evaluate.mock.calls[1][0]).toContain('img[src], canvas, video');
|
||||
expect(page.evaluate.mock.calls[1][0]).not.toContain("aria-disabled') === 'false'");
|
||||
});
|
||||
});
|
||||
|
||||
+11
-43
@@ -63,7 +63,7 @@ function getTranscriptLinesScript() {
|
||||
const stopLines = new Set([
|
||||
'豆包',
|
||||
'新对话',
|
||||
'内容由豆包 AI 生成,请仔细甄别',
|
||||
'内容由豆包 AI 生成',
|
||||
'AI 创作',
|
||||
'云盘',
|
||||
'更多',
|
||||
@@ -75,8 +75,6 @@ function getTranscriptLinesScript() {
|
||||
'PPT 生成',
|
||||
'图像生成',
|
||||
'帮我写作',
|
||||
'请仔细甄别',
|
||||
'下载电脑版',
|
||||
]);
|
||||
|
||||
const noisyPatterns = [
|
||||
@@ -90,7 +88,7 @@ function getTranscriptLinesScript() {
|
||||
|
||||
const transcriptText = clean(root.innerText || root.textContent || '')
|
||||
.replace(/新对话/g, '\\n')
|
||||
.replace(/内容由豆包 AI 生成,请仔细甄别/g, '\\n')
|
||||
.replace(/内容由豆包 AI 生成/g, '\\n')
|
||||
.replace(/在此处拖放文件/g, '\\n')
|
||||
.replace(/文件数量:[^\\n]*/g, '')
|
||||
.replace(/文件类型:[^\\n]*/g, '');
|
||||
@@ -146,20 +144,12 @@ function getTurnsScript() {
|
||||
if (
|
||||
root.matches('[data-testid="send_message"], [class*="send-message"]')
|
||||
|| root.querySelector('[data-testid="send_message"], [class*="send-message"]')
|
||||
|| root.matches('[class*="bg-g-send-msg-bubble"]')
|
||||
||
|
||||
root.querySelector('[class*="bg-g-send-msg-bubble"]')
|
||||
|| root.querySelector('[data-foundation-type="send-message-action-bar"]')
|
||||
) {
|
||||
return 'User';
|
||||
}
|
||||
if (
|
||||
root.matches('[data-testid="receive_message"], [data-testid*="receive_message"], [class*="receive-message"]')
|
||||
|| root.querySelector('[data-testid="receive_message"], [data-testid*="receive_message"], [class*="receive-message"]')
|
||||
|| root.matches('[class*="bg-g-receive-msg-bubble"]')
|
||||
||
|
||||
root.querySelector('[class*="bg-g-receive-msg-bubble"]')
|
||||
|| root.querySelector('[data-foundation-type="receive-message-action-bar"]')
|
||||
) {
|
||||
return 'Assistant';
|
||||
}
|
||||
@@ -173,10 +163,6 @@ function getTurnsScript() {
|
||||
'[data-testid*="message_content"]',
|
||||
'[class*="message-text"]',
|
||||
'[class*="message-content"]',
|
||||
'[class*="bg-g-send-msg-bubble"]',
|
||||
'[class*="bg-g-receive-msg-bubble"]',
|
||||
'.flow-markdown-body',
|
||||
'[class*="bubble"]',
|
||||
];
|
||||
const messageImageSelector = messageTextSelectors.map((s) => s + ' img').join(', ');
|
||||
|
||||
@@ -219,30 +205,14 @@ function getTurnsScript() {
|
||||
return text ? text + '\\n' + imageLines.join('\\n') : imageLines.join('\\n');
|
||||
};
|
||||
|
||||
const messageList = document.querySelector('[class*="message-list-S2Fv2S"], .container-PvPoAn, .scroll-view-OEiNXD, [data-testid="message-list"]');
|
||||
const messageList = document.querySelector('[data-testid="message-list"]');
|
||||
if (!messageList) return [];
|
||||
|
||||
const itemSelectors = [
|
||||
'[class*="item-kDun2N"]',
|
||||
'[data-testid="union_message"]',
|
||||
'[data-testid="message-block-container"]',
|
||||
'[data-message-id]',
|
||||
'[class*="bg-g-send-msg-bubble"]',
|
||||
'[class*="bg-g-receive-msg-bubble"]',
|
||||
];
|
||||
|
||||
const allRoots = [];
|
||||
const seen = new Set();
|
||||
for (const sel of itemSelectors) {
|
||||
messageList.querySelectorAll(sel).forEach((el) => {
|
||||
if (!seen.has(el)) {
|
||||
seen.add(el);
|
||||
allRoots.push(el);
|
||||
}
|
||||
});
|
||||
}
|
||||
const roots = allRoots
|
||||
.filter((el) => isVisible(el) && !el.closest('script, style, noscript'))
|
||||
const unionRoots = Array.from(messageList.querySelectorAll('[data-testid="union_message"]'))
|
||||
.filter((el) => isVisible(el));
|
||||
const blockRoots = Array.from(messageList.querySelectorAll('[data-testid="message-block-container"]'))
|
||||
.filter((el) => isVisible(el) && !el.closest('[data-testid="union_message"]'));
|
||||
const roots = (unionRoots.length > 0 ? unionRoots : blockRoots)
|
||||
.filter((el, index, items) => !items.some((other, otherIndex) => otherIndex !== index && other.contains(el)));
|
||||
|
||||
const turns = roots
|
||||
@@ -260,11 +230,11 @@ function getTurnsScript() {
|
||||
});
|
||||
|
||||
const deduped = [];
|
||||
const dedupedSeen = new Set();
|
||||
const seen = new Set();
|
||||
for (const turn of turns) {
|
||||
const key = turn.role + '::' + turn.text;
|
||||
if (dedupedSeen.has(key)) continue;
|
||||
dedupedSeen.add(key);
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
deduped.push({ Role: turn.role, Text: turn.text });
|
||||
}
|
||||
|
||||
@@ -1116,8 +1086,6 @@ export const __test__ = {
|
||||
clickSendButtonScript,
|
||||
composerStateScript,
|
||||
detectDoubaoVerificationScript,
|
||||
getTurnsScript,
|
||||
getTranscriptLinesScript,
|
||||
};
|
||||
export async function startNewDoubaoChat(page) {
|
||||
await ensureDoubaoChatPage(page);
|
||||
|
||||
@@ -144,25 +144,6 @@ describe('doubao send strategy', () => {
|
||||
await expect(sendDoubaoMessage(page, '你好')).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
describe('doubao receive strategy', () => {
|
||||
it('keeps both the new skin selectors and the older structural fallbacks in the turns script', () => {
|
||||
const turnsScript = __test__.getTurnsScript();
|
||||
expect(turnsScript).toContain('[class*="message-list-S2Fv2S"]');
|
||||
expect(turnsScript).toContain('.container-PvPoAn');
|
||||
expect(turnsScript).toContain('[data-testid="message-list"]');
|
||||
expect(turnsScript).toContain('[class*="bg-g-receive-msg-bubble"]');
|
||||
expect(turnsScript).toContain('[data-testid="receive_message"]');
|
||||
expect(turnsScript).toContain('[data-foundation-type="receive-message-action-bar"]');
|
||||
expect(turnsScript).toContain('[data-testid="union_message"]');
|
||||
expect(turnsScript).toContain('[data-testid="message-block-container"]');
|
||||
});
|
||||
|
||||
it('extends transcript-noise cleanup for the current zh-CN chrome copy', () => {
|
||||
const transcriptScript = __test__.getTranscriptLinesScript();
|
||||
expect(transcriptScript).toContain('请仔细甄别');
|
||||
expect(transcriptScript).toContain('下载电脑版');
|
||||
});
|
||||
});
|
||||
describe('collectDoubaoTranscriptAdditions', () => {
|
||||
it('ignores landing-page capability chips that are not assistant content', () => {
|
||||
const before = ['older'];
|
||||
|
||||
@@ -18,7 +18,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['time', 'code', 'name', 'title', 'category', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const market = String(args.market ?? 'SHA,SZA,BJA').trim() || 'SHA,SZA,BJA';
|
||||
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['rank', 'bondCode', 'bondName', 'bondPrice', 'bondChangePct', 'stockCode', 'stockName', 'stockPrice', 'stockChangePct', 'convPrice', 'convValue', 'convPremiumPct', 'remainingYears', 'ytm', 'listDate'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const sortKey = String(args.sort ?? 'turnover').toLowerCase();
|
||||
const sort = SORTS[sortKey];
|
||||
if (!sort) throw new CliError('INVALID_ARGUMENT', `Unknown sort "${sortKey}". Valid: ${Object.keys(SORTS).join(', ')}`);
|
||||
|
||||
@@ -26,7 +26,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'change', 'turnover', 'volume', 'turnoverRate'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const sortKey = String(args.sort ?? 'turnover').toLowerCase();
|
||||
const sort = SORTS[sortKey];
|
||||
if (!sort) throw new CliError('INVALID_ARGUMENT', `Unknown sort "${sortKey}". Valid: ${Object.keys(SORTS).join(', ')}`);
|
||||
|
||||
@@ -37,7 +37,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: '返回股东数(默认十大流通股东)' },
|
||||
],
|
||||
columns: ['rank', 'reportDate', 'name', 'holdNum', 'floatRatio', 'change'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
/** @type {string} */
|
||||
let secucode;
|
||||
try { secucode = toSecucode(args.symbol); }
|
||||
|
||||
@@ -47,7 +47,7 @@ cli({
|
||||
},
|
||||
],
|
||||
columns: ['code', 'name', 'price', 'changePercent', 'change', 'open', 'high', 'low', 'prevClose'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const group = String(args.group ?? 'main').toLowerCase();
|
||||
/** @type {[string,string][]} */
|
||||
let entries;
|
||||
|
||||
@@ -36,7 +36,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 30, help: '返回最近 N 根(末尾)' },
|
||||
],
|
||||
columns: ['date', 'open', 'close', 'high', 'low', 'volume', 'turnover', 'amplitude', 'changePercent', 'change', 'turnoverRate'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const secid = resolveSecid(args.symbol);
|
||||
const periodKey = String(args.period ?? 'day').toLowerCase();
|
||||
const klt = PERIOD_MAP[periodKey];
|
||||
|
||||
@@ -26,7 +26,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['time', 'title', 'summary', 'stocks'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const column = String(args.column ?? '102').trim();
|
||||
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['tradeDate', 'code', 'name', 'closePrice', 'changeRate', 'boardAmt', 'buyAmt', 'sellAmt', 'netAmt', 'turnover', 'dealRatio', 'market', 'reason'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const sinceDate = String(args.date || '').trim() || defaultTradeDate();
|
||||
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'mainNet', 'mainNetRatio', 'superNet', 'bigNet', 'mediumNet', 'smallNet'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const rangeKey = String(args.range ?? 'today').toLowerCase();
|
||||
const range = RANGES[rangeKey];
|
||||
if (!range) {
|
||||
|
||||
@@ -19,7 +19,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: '返回最近 N 分钟' },
|
||||
],
|
||||
columns: ['time', 'cumulativeNetYi', 'minuteNetYi', 'totalNetYi'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const dir = String(args.direction ?? 'north').toLowerCase();
|
||||
if (!['north', 'south', 'n', 's'].includes(dir)) {
|
||||
throw new CliError('INVALID_ARGUMENT', `Unknown direction "${dir}". Valid: north / south`);
|
||||
|
||||
@@ -57,7 +57,7 @@ cli({
|
||||
'turnoverRate', 'amplitude', 'peDynamic', 'priceBook',
|
||||
'marketCap', 'floatMarketCap',
|
||||
],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const raw = splitSymbols(args.symbols);
|
||||
if (raw.length === 0) {
|
||||
throw new CliError('INVALID_ARGUMENT', 'At least one symbol is required');
|
||||
|
||||
@@ -43,7 +43,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'change', 'turnover', 'volume', 'turnoverRate', 'peDynamic', 'marketCap'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const market = String(args.market ?? 'hs-a').toLowerCase();
|
||||
const sortKey = String(args.sort ?? 'change').toLowerCase();
|
||||
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 100));
|
||||
|
||||
@@ -33,7 +33,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回数量 (max 100)' },
|
||||
],
|
||||
columns: ['rank', 'code', 'name', 'price', 'changePercent', 'mainNet', 'leadStock', 'leadChangePercent', 'upCount', 'downCount'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const typeKey = String(args.type ?? 'industry').toLowerCase();
|
||||
const fs = SECTOR_TYPES[typeKey];
|
||||
if (!fs) throw new CliError('INVALID_ARGUMENT', `Unknown sector type "${typeKey}". Valid: ${Object.keys(SECTOR_TYPES).join(', ')}`);
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
function normalizeLimit(value) {
|
||||
const limit = Number(value ?? 20);
|
||||
if (!Number.isInteger(limit) || limit <= 0) {
|
||||
throw new ArgumentError('facebook marketplace-inbox --limit must be a positive integer');
|
||||
}
|
||||
return Math.min(limit, 100);
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'facebook',
|
||||
name: 'marketplace-inbox',
|
||||
description: 'List recent Facebook Marketplace buyer/seller conversations',
|
||||
domain: 'www.facebook.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of conversations to return' },
|
||||
],
|
||||
columns: ['index', 'buyer', 'listing', 'snippet', 'time', 'unread'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for facebook marketplace-inbox');
|
||||
const limit = normalizeLimit(args.limit);
|
||||
await page.goto('https://www.facebook.com/marketplace/inbox/');
|
||||
await page.wait(4);
|
||||
|
||||
const result = await page.evaluate(String.raw`(() => {
|
||||
const clean = (s) => String(s || '').replace(/[\u00a0\u202f]/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const timeRe = /^(?:\d{1,2}:\d{2}\s?(?:AM|PM|am|pm|上午|下午)?|Mon|Tue|Wed|Thu|Fri|Sat|Sun|Today|Yesterday|\d+[mhdw]|\d+\s*(?:min|h|d|w))$/;
|
||||
const text = document.body?.innerText || '';
|
||||
if (/log in|sign in/i.test(text) && !/Marketplace/i.test(text)) {
|
||||
return { authRequired: true, rows: [] };
|
||||
}
|
||||
|
||||
const lines = text.split(/\n+/).map(clean).filter(Boolean);
|
||||
const out = [];
|
||||
const seen = new Set();
|
||||
const skipBuyer = /^(Marketplace|Browse all|Notifications|Inbox|Marketplace access|Buying|Selling|Create new listing|Create multiple listings|Location|Categories|Vehicles|Property Rentals|All|Pending payment|Paid|To be shipped|Shipped|Cash on delivery|Completed|Filter by label)$/i;
|
||||
for (let i = 0; i < lines.length - 2; i += 1) {
|
||||
const buyer = lines[i];
|
||||
const meta = lines[i + 1];
|
||||
if (skipBuyer.test(buyer) || !/^·\s+/.test(meta)) continue;
|
||||
const listing = meta.replace(/^·\s*/, '');
|
||||
if (!listing || /^Within\b/i.test(listing)) continue;
|
||||
const snippet = lines[i + 2] || '';
|
||||
const time = timeRe.test(lines[i + 3] || '') ? lines[i + 3] : '';
|
||||
const key = buyer + '|' + listing;
|
||||
if (seen.has(key)) continue;
|
||||
seen.add(key);
|
||||
const nearby = lines.slice(Math.max(0, i - 2), i + 5).join(' ');
|
||||
out.push({
|
||||
buyer,
|
||||
listing,
|
||||
snippet,
|
||||
time,
|
||||
unread: /Unread/i.test(nearby),
|
||||
});
|
||||
}
|
||||
return { authRequired: false, rows: out };
|
||||
})()`);
|
||||
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError('facebook.com', 'Facebook Marketplace inbox requires an active signed-in Facebook session.');
|
||||
}
|
||||
const items = Array.isArray(result?.rows) ? result.rows : [];
|
||||
if (items.length === 0) {
|
||||
throw new EmptyResultError('facebook marketplace-inbox', 'No Marketplace inbox conversations were visible. Check that Marketplace inbox is available for this account.');
|
||||
}
|
||||
return items.slice(0, limit).map((item, index) => ({
|
||||
index: index + 1,
|
||||
buyer: item.buyer || '',
|
||||
listing: item.listing || '',
|
||||
snippet: item.snippet || '',
|
||||
time: item.time || '',
|
||||
unread: Boolean(item.unread),
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeLimit,
|
||||
};
|
||||
@@ -1,83 +0,0 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
|
||||
function normalizeLimit(value) {
|
||||
const limit = Number(value ?? 20);
|
||||
if (!Number.isInteger(limit) || limit <= 0) {
|
||||
throw new ArgumentError('facebook marketplace-listings --limit must be a positive integer');
|
||||
}
|
||||
return Math.min(limit, 100);
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'facebook',
|
||||
name: 'marketplace-listings',
|
||||
description: 'List your Facebook Marketplace seller listings',
|
||||
domain: 'www.facebook.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of listings to return' },
|
||||
],
|
||||
columns: ['index', 'title', 'price', 'status', 'listed', 'clicks', 'actions'],
|
||||
func: async (page, args) => {
|
||||
if (!page) throw new CommandExecutionError('Browser session required for facebook marketplace-listings');
|
||||
const limit = normalizeLimit(args.limit);
|
||||
await page.goto('https://www.facebook.com/marketplace/you/selling/');
|
||||
await page.wait(4);
|
||||
|
||||
const result = await page.evaluate(String.raw`(() => {
|
||||
const clean = (s) => String(s || '').replace(/\u00a0/g, ' ').replace(/\s+/g, ' ').trim();
|
||||
const allText = document.body?.innerText || '';
|
||||
if (/log in|sign in/i.test(allText) && !/Marketplace/i.test(allText)) {
|
||||
return { authRequired: true, rows: [] };
|
||||
}
|
||||
|
||||
const lines = allText.split(/\n+/).map(clean).filter(Boolean);
|
||||
const seen = new Set();
|
||||
const out = [];
|
||||
for (let i = 1; i < lines.length; i += 1) {
|
||||
if (!/^(?:CA\$|\$)\s*\d+/.test(lines[i])) continue;
|
||||
const title = lines[i - 1];
|
||||
if (!title || /^(Hide|All listings|Needs attention|Marketplace|Selling)$/i.test(title)) continue;
|
||||
if (seen.has(title)) continue;
|
||||
seen.add(title);
|
||||
const windowLines = lines.slice(i, i + 12);
|
||||
const status = windowLines.find((line) => /^(Active|Sold|Pending|Draft)$/i.test(line)) || '';
|
||||
const listed = windowLines.find((line) => /Listed on\b/i.test(line))?.replace(/^·\s*/, '') || '';
|
||||
const clickLine = windowLines.find((line) => /clicks? on listing/i.test(line)) || '';
|
||||
const clickMatch = clickLine.match(/([\d,.]+)\s+clicks? on listing/i);
|
||||
const actions = windowLines.filter((line) => /^(Mark as sold|Mark as available|Relist this item|Share|Boost listing)$/i.test(line));
|
||||
out.push({
|
||||
title,
|
||||
price: lines[i],
|
||||
status,
|
||||
listed,
|
||||
clicks: clickMatch ? clickMatch[1] : '',
|
||||
actions,
|
||||
});
|
||||
}
|
||||
return { authRequired: false, rows: out };
|
||||
})()`);
|
||||
|
||||
if (result?.authRequired) {
|
||||
throw new AuthRequiredError('facebook.com', 'Facebook Marketplace seller listings require an active signed-in Facebook session.');
|
||||
}
|
||||
const items = Array.isArray(result?.rows) ? result.rows : [];
|
||||
if (items.length === 0) {
|
||||
throw new EmptyResultError('facebook marketplace-listings', 'No seller listings were visible. Check that Marketplace selling is available for this account.');
|
||||
}
|
||||
return items.slice(0, limit).map((item, index) => ({
|
||||
index: index + 1,
|
||||
title: item.title || '',
|
||||
price: item.price || '',
|
||||
status: item.status || '',
|
||||
listed: item.listed || '',
|
||||
clicks: item.clicks || '',
|
||||
actions: Array.isArray(item.actions) ? item.actions.join(', ') : String(item.actions || ''),
|
||||
}));
|
||||
},
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
normalizeLimit,
|
||||
};
|
||||
@@ -1,91 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import './marketplace-listings.js';
|
||||
import './marketplace-inbox.js';
|
||||
|
||||
function makePage(overrides = {}) {
|
||||
return {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue([]),
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('facebook marketplace read commands', () => {
|
||||
it('marketplace-listings navigates to selling page and returns limited listing rows', async () => {
|
||||
const command = getRegistry().get('facebook/marketplace-listings');
|
||||
expect(command).toBeDefined();
|
||||
const page = makePage({
|
||||
evaluate: vi.fn().mockResolvedValue({
|
||||
authRequired: false,
|
||||
rows: [
|
||||
{ title: 'Black electric standing desk', price: 'CA$80', status: 'Active', listed: 'Listed on 4/26', clicks: '87', actions: ['Mark as sold', 'Share'] },
|
||||
{ title: 'Large gray corduroy beanbag chair', price: 'CA$30', status: 'Sold', listed: 'Listed on 4/26', clicks: '52', actions: ['Mark as available', 'Relist this item'] },
|
||||
],
|
||||
}),
|
||||
});
|
||||
|
||||
const rows = await command.func(page, { limit: 1 });
|
||||
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.facebook.com/marketplace/you/selling/');
|
||||
expect(page.wait).toHaveBeenCalledWith(4);
|
||||
expect(rows).toEqual([
|
||||
{
|
||||
index: 1,
|
||||
title: 'Black electric standing desk',
|
||||
price: 'CA$80',
|
||||
status: 'Active',
|
||||
listed: 'Listed on 4/26',
|
||||
clicks: '87',
|
||||
actions: 'Mark as sold, Share',
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('marketplace-inbox navigates to inbox and returns recent buyer conversations', async () => {
|
||||
const command = getRegistry().get('facebook/marketplace-inbox');
|
||||
expect(command).toBeDefined();
|
||||
const page = makePage({
|
||||
evaluate: vi.fn().mockResolvedValue({
|
||||
authRequired: false,
|
||||
rows: [
|
||||
{ buyer: 'Kulwant', listing: 'White 3-tier rolling utility cart', snippet: 'Can I pick up today?', time: '3:43 PM', unread: true },
|
||||
{ buyer: 'Gabriel', listing: 'Black electric standing desk', snippet: 'Yes, still available.', time: '12:17 PM', unread: false },
|
||||
],
|
||||
}),
|
||||
});
|
||||
|
||||
const rows = await command.func(page, { limit: 2 });
|
||||
|
||||
expect(page.goto).toHaveBeenCalledWith('https://www.facebook.com/marketplace/inbox/');
|
||||
expect(page.wait).toHaveBeenCalledWith(4);
|
||||
expect(rows).toEqual([
|
||||
{ index: 1, buyer: 'Kulwant', listing: 'White 3-tier rolling utility cart', snippet: 'Can I pick up today?', time: '3:43 PM', unread: true },
|
||||
{ index: 2, buyer: 'Gabriel', listing: 'Black electric standing desk', snippet: 'Yes, still available.', time: '12:17 PM', unread: false },
|
||||
]);
|
||||
});
|
||||
|
||||
it('throws EmptyResultError when Marketplace returns no inbox rows', async () => {
|
||||
const command = getRegistry().get('facebook/marketplace-inbox');
|
||||
const page = makePage({ evaluate: vi.fn().mockResolvedValue({ authRequired: false, rows: [] }) });
|
||||
|
||||
await expect(command.func(page, { limit: 5 })).rejects.toThrow(EmptyResultError);
|
||||
});
|
||||
|
||||
it('throws AuthRequiredError when Marketplace returns a login page', async () => {
|
||||
const command = getRegistry().get('facebook/marketplace-listings');
|
||||
const page = makePage({ evaluate: vi.fn().mockResolvedValue({ authRequired: true, rows: [] }) });
|
||||
|
||||
await expect(command.func(page, { limit: 5 })).rejects.toThrow(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('throws ArgumentError for invalid limits', async () => {
|
||||
const command = getRegistry().get('facebook/marketplace-listings');
|
||||
const page = makePage();
|
||||
|
||||
await expect(command.func(page, { limit: 0 })).rejects.toThrow(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -1,74 +0,0 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { requireNonEmptyQuery } from '../_shared/common.js';
|
||||
|
||||
cli({
|
||||
site: 'google-scholar',
|
||||
name: 'cite',
|
||||
description: 'Get citation for a Google Scholar paper',
|
||||
domain: 'scholar.google.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'query', positional: true, required: true, help: 'Paper title to search for' },
|
||||
{ name: 'style', default: 'bibtex', choices: ['bibtex', 'endnote', 'refman', 'refworks'], help: 'Citation format' },
|
||||
{ name: 'index', type: 'int', default: 1, help: 'Which search result to cite (1-based)' },
|
||||
],
|
||||
columns: ['title', 'format', 'citation'],
|
||||
navigateBefore: false,
|
||||
func: async (page, kwargs) => {
|
||||
const query = requireNonEmptyQuery(kwargs.query);
|
||||
const format = kwargs.style || 'bibtex';
|
||||
const index = Math.max(1, kwargs.index || 1) - 1;
|
||||
|
||||
await page.goto(`https://scholar.google.com/scholar?q=${encodeURIComponent(query)}&hl=en`);
|
||||
await page.wait(3);
|
||||
|
||||
const clicked = await page.evaluate(`(() => {
|
||||
var cites = document.querySelectorAll('a.gs_or_cit');
|
||||
if (cites.length <= ${index}) return { ok: false, reason: 'result not found at index ${index + 1}' };
|
||||
var titleEl = document.querySelectorAll('.gs_r.gs_or.gs_scl')[${index}];
|
||||
var title = '';
|
||||
if (titleEl) {
|
||||
var t = titleEl.querySelector('.gs_rt a, h3 a');
|
||||
title = t ? t.textContent.trim() : '';
|
||||
}
|
||||
cites[${index}].click();
|
||||
return { ok: true, title: title };
|
||||
})()`);
|
||||
|
||||
if (!clicked?.ok) {
|
||||
throw new CommandExecutionError(clicked?.reason || `Could not find search result at index ${index + 1}`);
|
||||
}
|
||||
|
||||
await page.wait(2);
|
||||
|
||||
const formatMap = { bibtex: 'BibTeX', endnote: 'EndNote', refman: 'RefMan', refworks: 'RefWorks' };
|
||||
const formatLabel = formatMap[format] || 'BibTeX';
|
||||
|
||||
const citeUrl = await page.evaluate(`(() => {
|
||||
var links = document.querySelectorAll('#gs_cit a.gs_citi');
|
||||
for (var i = 0; i < links.length; i++) {
|
||||
if (links[i].textContent.trim() === '${formatLabel}') return links[i].href;
|
||||
}
|
||||
return null;
|
||||
})()`);
|
||||
|
||||
if (!citeUrl) {
|
||||
throw new CommandExecutionError(`Could not find ${formatLabel} citation link for result ${index + 1}`);
|
||||
}
|
||||
|
||||
await page.goto(citeUrl);
|
||||
await page.wait(2);
|
||||
|
||||
const citation = await page.evaluate(`(() => {
|
||||
return (document.body.innerText || '').trim();
|
||||
})()`);
|
||||
|
||||
if (!citation) {
|
||||
throw new CommandExecutionError(`${formatLabel} citation page returned an empty response`);
|
||||
}
|
||||
|
||||
return [{ title: clicked.title, format: format, citation }];
|
||||
},
|
||||
});
|
||||
@@ -1,47 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './cite.js';
|
||||
|
||||
describe('google-scholar cite command', () => {
|
||||
const command = getRegistry().get('google-scholar/cite');
|
||||
|
||||
it('registers as a public browser command', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('google-scholar');
|
||||
expect(command.strategy).toBe('public');
|
||||
expect(command.browser).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects empty queries before browser navigation', async () => {
|
||||
const page = { goto: vi.fn() };
|
||||
await expect(command.func(page, { query: ' ' })).rejects.toMatchObject({
|
||||
name: 'ArgumentError',
|
||||
code: 'ARGUMENT',
|
||||
});
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws when the requested search result index does not exist', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValueOnce({ ok: false, reason: 'result not found at index 2' }),
|
||||
};
|
||||
await expect(command.func(page, { query: 'test', index: 2 })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('looks up the requested citation style instead of only locking BibTeX', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce({ ok: true, title: 'Paper Title' })
|
||||
.mockResolvedValueOnce('https://example.com/refworks')
|
||||
.mockResolvedValueOnce('RefWorks citation body'),
|
||||
};
|
||||
const result = await command.func(page, { query: 'test', style: 'refworks' });
|
||||
expect(result).toEqual([{ title: 'Paper Title', format: 'refworks', citation: 'RefWorks citation body' }]);
|
||||
expect(page.evaluate.mock.calls[1][0]).toContain('RefWorks');
|
||||
});
|
||||
});
|
||||
@@ -1,92 +0,0 @@
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { clampInt, requireNonEmptyQuery } from '../_shared/common.js';
|
||||
|
||||
cli({
|
||||
site: 'google-scholar',
|
||||
name: 'profile',
|
||||
description: 'View a Google Scholar author profile',
|
||||
domain: 'scholar.google.com',
|
||||
strategy: Strategy.PUBLIC,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'author', positional: true, required: true, help: 'Author name or Scholar user ID (e.g. JicYPdAAAAAJ)' },
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Max papers to show (max 20)' },
|
||||
],
|
||||
columns: ['rank', 'title', 'cited', 'year'],
|
||||
navigateBefore: false,
|
||||
func: async (page, kwargs) => {
|
||||
const author = requireNonEmptyQuery(kwargs.author, 'author');
|
||||
const limit = clampInt(kwargs.limit, 10, 1, 20);
|
||||
|
||||
const isUserId = /^[A-Za-z0-9_-]{12}$/.test(author);
|
||||
if (isUserId) {
|
||||
await page.goto(`https://scholar.google.com/citations?user=${author}&hl=en&sortby=citedby`);
|
||||
} else {
|
||||
await page.goto(`https://scholar.google.com/citations?view_op=search_authors&mauthors=${encodeURIComponent(author)}&hl=en`);
|
||||
await page.wait(3);
|
||||
|
||||
const profileClicked = await page.evaluate(`(() => {
|
||||
var link = document.querySelector('.gs_ai_pho, .gsc_oai_photo, a[href*="citations?user="]');
|
||||
if (link) { link.click(); return true; }
|
||||
return false;
|
||||
})()`);
|
||||
|
||||
if (!profileClicked) {
|
||||
throw new CommandExecutionError(`No profile found for: ${author}`);
|
||||
}
|
||||
}
|
||||
|
||||
await page.wait(3);
|
||||
|
||||
const data = await page.evaluate(`(() => {
|
||||
var name = (document.querySelector('#gsc_prf_in') || {}).textContent || '';
|
||||
var affiliation = (document.querySelector('.gsc_prf_il') || {}).textContent || '';
|
||||
|
||||
var stats = document.querySelectorAll('#gsc_rsb_st td.gsc_rsb_std');
|
||||
var citations = stats[0] ? stats[0].textContent.trim() : '';
|
||||
var hIndex = stats[2] ? stats[2].textContent.trim() : '';
|
||||
var i10Index = stats[4] ? stats[4].textContent.trim() : '';
|
||||
|
||||
var papers = [];
|
||||
var rows = document.querySelectorAll('#gsc_a_b .gsc_a_tr');
|
||||
for (var i = 0; i < rows.length && i < ${limit}; i++) {
|
||||
var titleEl = rows[i].querySelector('.gsc_a_at');
|
||||
var citedEl = rows[i].querySelector('.gsc_a_ac');
|
||||
var yearEl = rows[i].querySelector('.gsc_a_y span');
|
||||
if (titleEl) papers.push({
|
||||
rank: i + 1,
|
||||
title: titleEl.textContent.trim(),
|
||||
cited: citedEl ? citedEl.textContent.trim() : '0',
|
||||
year: yearEl ? yearEl.textContent.trim() : '',
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
name: name.trim(),
|
||||
affiliation: affiliation.trim(),
|
||||
citations: citations,
|
||||
hIndex: hIndex,
|
||||
i10Index: i10Index,
|
||||
papers: papers,
|
||||
};
|
||||
})()`);
|
||||
|
||||
if (!data?.name) {
|
||||
throw new CommandExecutionError(`Could not load Google Scholar profile for: ${author}`);
|
||||
}
|
||||
|
||||
if (!data.papers || data.papers.length === 0) {
|
||||
throw new CommandExecutionError(`No papers found for: ${data.name || author}`);
|
||||
}
|
||||
|
||||
const summary = {
|
||||
rank: 0,
|
||||
title: data.name + (data.affiliation ? ' (' + data.affiliation + ')' : ''),
|
||||
cited: 'h=' + data.hIndex + ' i10=' + data.i10Index + ' total=' + data.citations,
|
||||
year: '-',
|
||||
};
|
||||
|
||||
return [summary, ...data.papers];
|
||||
},
|
||||
});
|
||||
@@ -1,49 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import './profile.js';
|
||||
|
||||
describe('google-scholar profile command', () => {
|
||||
const command = getRegistry().get('google-scholar/profile');
|
||||
|
||||
it('registers as a public browser command', () => {
|
||||
expect(command).toBeDefined();
|
||||
expect(command.site).toBe('google-scholar');
|
||||
expect(command.strategy).toBe('public');
|
||||
expect(command.browser).toBe(true);
|
||||
});
|
||||
|
||||
it('rejects empty author before browser navigation', async () => {
|
||||
const page = { goto: vi.fn() };
|
||||
await expect(command.func(page, { author: ' ' })).rejects.toMatchObject({
|
||||
name: 'ArgumentError',
|
||||
code: 'ARGUMENT',
|
||||
});
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('throws when author search does not resolve to a profile', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValueOnce(false),
|
||||
};
|
||||
await expect(command.func(page, { author: 'missing author' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('throws when the loaded profile has no papers', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValueOnce({
|
||||
name: 'Author Name',
|
||||
affiliation: 'Org',
|
||||
citations: '0',
|
||||
hIndex: '0',
|
||||
i10Index: '0',
|
||||
papers: [],
|
||||
}),
|
||||
};
|
||||
await expect(command.func(page, { author: 'JicYPdAAAAAJ' })).rejects.toThrow(CommandExecutionError);
|
||||
});
|
||||
});
|
||||
@@ -23,7 +23,7 @@ cli({
|
||||
(() => {
|
||||
const normalize = v => (v || '').replace(/\\s+/g, ' ').trim();
|
||||
const results = [];
|
||||
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl')) {
|
||||
for (const el of document.querySelectorAll('.gs_r.gs_or.gs_scl, .gs_ri')) {
|
||||
const container = el.querySelector('.gs_ri') || el;
|
||||
const titleEl = container.querySelector('.gs_rt a, h3 a');
|
||||
const title = normalize(titleEl?.textContent);
|
||||
|
||||
@@ -20,19 +20,4 @@ describe('google-scholar search command', () => {
|
||||
});
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('locks dedup to outer Scholar result cards while preserving inner content extraction', async () => {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn().mockResolvedValue([]),
|
||||
};
|
||||
|
||||
await command.func(page, { query: 'transformer' });
|
||||
|
||||
const script = page.evaluate.mock.calls[0][0];
|
||||
expect(script).toContain("document.querySelectorAll('.gs_r.gs_or.gs_scl')");
|
||||
expect(script).not.toContain(".gs_r.gs_or.gs_scl, .gs_ri");
|
||||
expect(script).toContain("const container = el.querySelector('.gs_ri') || el");
|
||||
});
|
||||
});
|
||||
|
||||
+1
-1
@@ -18,7 +18,7 @@ cli({
|
||||
{ name: 'region', default: 'US', help: 'Region code (e.g. US, CN)' },
|
||||
],
|
||||
columns: ['title', 'source', 'date', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 100));
|
||||
const lang = encodeURIComponent(args.lang);
|
||||
const region = encodeURIComponent(args.region);
|
||||
|
||||
@@ -15,7 +15,7 @@ cli({
|
||||
{ name: 'lang', default: 'zh-CN', help: 'Language code' },
|
||||
],
|
||||
columns: ['suggestion'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const keyword = encodeURIComponent(args.keyword);
|
||||
const lang = encodeURIComponent(args.lang);
|
||||
const url = `https://suggestqueries.google.com/complete/search?client=firefox&q=${keyword}&hl=${lang}`;
|
||||
|
||||
@@ -16,7 +16,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Number of results' },
|
||||
],
|
||||
columns: ['title', 'traffic', 'date'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 100));
|
||||
const region = encodeURIComponent(args.region);
|
||||
const url = `https://trends.google.com/trending/rss?geo=${region}`;
|
||||
|
||||
+1
-1
@@ -50,7 +50,7 @@ cli({
|
||||
return getWeekRange();
|
||||
return kwargs.date ?? new Date().toISOString().slice(0, 10);
|
||||
},
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const period = String(kwargs.period ?? 'daily');
|
||||
const all = Boolean(kwargs.all);
|
||||
const endpoint = process.env.HF_ENDPOINT?.replace(/\/+$/, '') || 'https://huggingface.co';
|
||||
|
||||
+46
-678
@@ -5,545 +5,16 @@
|
||||
* 用法: opencli jd item 100291143898
|
||||
*/
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
|
||||
function normalizePositiveInt(value, fallback) {
|
||||
const n = Number(value);
|
||||
return Number.isFinite(n) && n >= 0 ? Math.floor(n) : fallback;
|
||||
}
|
||||
function normalizeJdSkuInput(input) {
|
||||
const text = String(input || '').trim();
|
||||
const itemMatch = text.match(/item\.jd\.com\/(\d+)\.html/i);
|
||||
if (itemMatch)
|
||||
return itemMatch[1];
|
||||
if (/^\d+$/.test(text))
|
||||
return text;
|
||||
const paramMatch = text.match(/(?:skuId|sku|wareId|productId)[=:](\d+)/i);
|
||||
if (paramMatch)
|
||||
return paramMatch[1];
|
||||
return text;
|
||||
}
|
||||
function normalizeJdImageUrl(rawUrl) {
|
||||
if (!rawUrl || typeof rawUrl !== 'string')
|
||||
return '';
|
||||
let url = rawUrl.trim();
|
||||
if (!url)
|
||||
return '';
|
||||
if (url.startsWith('//'))
|
||||
url = `https:${url}`;
|
||||
if (!/^https?:\/\//.test(url))
|
||||
return '';
|
||||
return url;
|
||||
}
|
||||
function normalizeJdImageSize(url) {
|
||||
return normalizeJdImageUrl(url)
|
||||
.replace(/\/pcpubliccms\/s\d+x\d+_jfs\//, '/pcpubliccms/jfs/')
|
||||
.replace(/\/(n\d+)\/s\d+x\d+_jfs\//, '/$1/jfs/')
|
||||
.replace(/\/s\d+x\d+_jfs\//, '/jfs/');
|
||||
}
|
||||
function isJdMainImage(url) {
|
||||
const normalized = normalizeJdImageSize(url);
|
||||
return /360buyimg\.com\/(?:pcpubliccms|n\d+)\/jfs\//.test(normalized) &&
|
||||
!/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_)/.test(normalized) &&
|
||||
!/\/(?:imgzone|sku|shaidan|popWaterMark|babel|jdcms|cms|ddimg|vc)\//.test(normalized);
|
||||
}
|
||||
function collectImageUrlsFrom(root, options = {}) {
|
||||
if (!root)
|
||||
return [];
|
||||
const urls = [];
|
||||
const ignoredContexts = '#spec-list, [class*="spec-list"], [class*="recommend"], [id*="recommend"], [class*="shaidan"], [id*="shaidan"], [class*="review"], [id*="review"], [class*="comment"], [id*="comment"], [class*="thumb"], [id*="thumb"], [class*="related"], [id*="related"]';
|
||||
const ignoreContexts = Boolean(options.ignoreContexts);
|
||||
const isIgnoredContext = (el) => {
|
||||
if (!ignoreContexts)
|
||||
return false;
|
||||
try {
|
||||
return !!el?.closest?.(ignoredContexts);
|
||||
}
|
||||
catch {
|
||||
return false;
|
||||
}
|
||||
};
|
||||
const pushUrlsFromText = (text) => {
|
||||
for (const match of String(text || '').matchAll(/url\(["']?([^"')]+360buyimg\.com[^"')]+)["']?\)/g)) {
|
||||
push(match[1]);
|
||||
}
|
||||
};
|
||||
const push = (value) => {
|
||||
const url = normalizeJdImageUrl(value);
|
||||
if (url && url.includes('360buyimg.com'))
|
||||
urls.push(url);
|
||||
};
|
||||
for (const img of root.querySelectorAll?.('img') || []) {
|
||||
if (isIgnoredContext(img))
|
||||
continue;
|
||||
push(img.currentSrc || img.src);
|
||||
push(img.getAttribute('data-src'));
|
||||
push(img.getAttribute('data-lazy-img'));
|
||||
push(img.getAttribute('data-lazyload'));
|
||||
push(img.getAttribute('data-original'));
|
||||
}
|
||||
for (const source of root.querySelectorAll?.('source') || []) {
|
||||
if (isIgnoredContext(source))
|
||||
continue;
|
||||
push(source.getAttribute('src'));
|
||||
push(source.getAttribute('srcset')?.split(/\s+/)[0]);
|
||||
push(source.getAttribute('data-src'));
|
||||
push(source.getAttribute('data-srcset')?.split(/\s+/)[0]);
|
||||
}
|
||||
for (const el of root.querySelectorAll?.('[style*="360buyimg.com"]') || []) {
|
||||
if (isIgnoredContext(el))
|
||||
continue;
|
||||
const style = el.getAttribute('style') || '';
|
||||
pushUrlsFromText(style);
|
||||
}
|
||||
if (typeof getComputedStyle === 'function') {
|
||||
const elements = [root, ...Array.from(root.querySelectorAll?.('*') || [])];
|
||||
for (const el of elements) {
|
||||
if (isIgnoredContext(el))
|
||||
continue;
|
||||
try {
|
||||
const style = getComputedStyle(el);
|
||||
pushUrlsFromText(style?.backgroundImage);
|
||||
pushUrlsFromText(style?.background);
|
||||
}
|
||||
catch {
|
||||
// ignore inaccessible/computed-style edge cases in the page context
|
||||
}
|
||||
}
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
function collectImageUrlsFromText(text) {
|
||||
const urls = [];
|
||||
const push = (value) => {
|
||||
const url = normalizeJdImageUrl(value);
|
||||
if (url && url.includes('360buyimg.com'))
|
||||
urls.push(url);
|
||||
};
|
||||
for (const match of String(text || '').matchAll(/(?:https?:)?\/\/[^"'`\s<>)\\]+360buyimg\.com[^"'`\s<>)\\]*/g)) {
|
||||
push(match[0]);
|
||||
}
|
||||
for (const match of String(text || '').matchAll(/url\(["']?([^"')]+360buyimg\.com[^"')]+)["']?\)/g)) {
|
||||
push(match[1]);
|
||||
}
|
||||
return urls;
|
||||
}
|
||||
function collectImageUrlsFromFramesAndScripts() {
|
||||
const urls = [];
|
||||
for (const script of document.scripts || []) {
|
||||
const text = script.textContent || '';
|
||||
if (!/360buyimg\.com/.test(text))
|
||||
continue;
|
||||
urls.push(...collectImageUrlsFromText(text));
|
||||
}
|
||||
for (const iframe of document.querySelectorAll('iframe')) {
|
||||
try {
|
||||
const frameDoc = iframe.contentDocument || iframe.contentWindow?.document;
|
||||
if (!frameDoc)
|
||||
continue;
|
||||
urls.push(...collectImageUrlsFrom(frameDoc.body || frameDoc.documentElement || frameDoc));
|
||||
for (const script of frameDoc.scripts || []) {
|
||||
const text = script.textContent || '';
|
||||
if (!/360buyimg\.com/.test(text))
|
||||
continue;
|
||||
urls.push(...collectImageUrlsFromText(text));
|
||||
}
|
||||
}
|
||||
catch {
|
||||
// ignore cross-origin or not-yet-loaded iframe content
|
||||
}
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
function collectImageUrlsFromPayload(payload, seen = new Set(), depth = 0) {
|
||||
if (payload == null || depth > 4)
|
||||
return [];
|
||||
const urls = [];
|
||||
const push = (value) => {
|
||||
const url = normalizeJdImageUrl(value);
|
||||
if (url && url.includes('360buyimg.com'))
|
||||
urls.push(url);
|
||||
};
|
||||
if (typeof payload === 'string') {
|
||||
urls.push(...collectImageUrlsFromText(payload));
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
if (typeof payload !== 'object')
|
||||
return [];
|
||||
if (seen.has(payload))
|
||||
return [];
|
||||
seen.add(payload);
|
||||
if (Array.isArray(payload)) {
|
||||
for (const item of payload)
|
||||
urls.push(...collectImageUrlsFromPayload(item, seen, depth + 1));
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
for (const value of Object.values(payload)) {
|
||||
if (typeof value === 'string') {
|
||||
push(value);
|
||||
urls.push(...collectImageUrlsFromText(value));
|
||||
continue;
|
||||
}
|
||||
urls.push(...collectImageUrlsFromPayload(value, seen, depth + 1));
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
function collectImageUrlsFromPageDataObjects() {
|
||||
const urls = [];
|
||||
const keys = ['__NEXT_DATA__', '__NUXT__', '__INITIAL_STATE__', '__INITIAL_DATA__', '__APOLLO_STATE__', '__INITIAL_PROPS__', '__REDUX_STATE__', '__PAGE_DATA__', 'pageData', '__data__', 'detailData'];
|
||||
for (const key of keys) {
|
||||
try {
|
||||
if (typeof globalThis !== 'undefined' && key in globalThis) {
|
||||
urls.push(...collectImageUrlsFromPayload(globalThis[key]));
|
||||
}
|
||||
}
|
||||
catch {
|
||||
// ignore inaccessible globals
|
||||
}
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
async function collectImageUrlsFromNetworkResources() {
|
||||
const urls = [];
|
||||
const entries = typeof performance !== 'undefined' && typeof performance.getEntriesByType === 'function'
|
||||
? performance.getEntriesByType('resource')
|
||||
: [];
|
||||
const candidates = [...new Set(entries
|
||||
.map((entry) => entry?.name)
|
||||
.filter((name) => typeof name === 'string' && /(?:jd\.com|360buyimg\.com)/.test(name) && /(?:detail|desc|ware|item|product|sku)/i.test(name) && !/pc_item_getWareGraphic/i.test(name)))].slice(0, 12);
|
||||
for (const url of candidates) {
|
||||
try {
|
||||
const resp = await fetch(url, { credentials: 'include' });
|
||||
if (!resp.ok)
|
||||
continue;
|
||||
const text = await resp.text();
|
||||
const contentType = resp.headers.get('content-type') || '';
|
||||
if (/json/i.test(contentType) || /^\s*[\[{]/.test(text)) {
|
||||
try {
|
||||
urls.push(...collectImageUrlsFromPayload(JSON.parse(text)));
|
||||
continue;
|
||||
}
|
||||
catch {
|
||||
// fall back to text scanning
|
||||
}
|
||||
}
|
||||
urls.push(...collectImageUrlsFromText(text));
|
||||
}
|
||||
catch {
|
||||
// ignore request failures and cross-origin oddities
|
||||
}
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
function collectImageUrlsFromWareGraphicText(text) {
|
||||
let graphicContent = '';
|
||||
try {
|
||||
const payload = JSON.parse(String(text || ''));
|
||||
graphicContent = payload?.data?.graphicContent || payload?.graphicContent || '';
|
||||
}
|
||||
catch {
|
||||
graphicContent = String(text || '');
|
||||
}
|
||||
return [...new Set(collectImageUrlsFromText(graphicContent))];
|
||||
}
|
||||
async function collectImageUrlsFromWareGraphicResources(sku) {
|
||||
const urls = [];
|
||||
const entries = typeof performance !== 'undefined' && typeof performance.getEntriesByType === 'function'
|
||||
? performance.getEntriesByType('resource')
|
||||
: [];
|
||||
const candidates = [...new Set(entries
|
||||
.map((entry) => entry?.name)
|
||||
.filter((name) => typeof name === 'string' && /pc_item_getWareGraphic/i.test(name)))];
|
||||
if (sku) {
|
||||
const body = encodeURIComponent(JSON.stringify({ skuId: String(sku) }));
|
||||
candidates.unshift(`https://api.m.jd.com/client.action?appid=item-v3&functionId=pc_item_getWareGraphic&client=pc&clientVersion=1.0.0&body=${body}`);
|
||||
}
|
||||
for (const url of candidates) {
|
||||
try {
|
||||
const resp = await fetch(url, { credentials: 'include' });
|
||||
if (!resp.ok)
|
||||
continue;
|
||||
urls.push(...collectImageUrlsFromWareGraphicText(await resp.text()));
|
||||
}
|
||||
catch {
|
||||
// ignore request failures and keep DOM/script fallbacks available
|
||||
}
|
||||
}
|
||||
return [...new Set(urls)];
|
||||
}
|
||||
async function collectImageUrlsFromFallbackSources() {
|
||||
return [
|
||||
...collectImageUrlsFromPageDataObjects(),
|
||||
...await collectImageUrlsFromNetworkResources(),
|
||||
];
|
||||
}
|
||||
function isJdDetailImage(url) {
|
||||
const normalized = normalizeJdImageSize(url);
|
||||
return /360buyimg\.com\/(?:imgzone|skuimg|babel|jdcms|cms|popWaterMark|vc|ddimg)\//.test(normalized) &&
|
||||
!/\/shaidan\//.test(normalized) &&
|
||||
!/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_|sku)\//.test(normalized);
|
||||
}
|
||||
function isJdWareGraphicDetailImage(url) {
|
||||
const raw = normalizeJdImageUrl(url);
|
||||
if (/\/(?:s\d+x\d+_|n\d\/s\d+x\d+_|sku\/s\d+x\d+_)jfs\//.test(raw))
|
||||
return false;
|
||||
const normalized = normalizeJdImageSize(url);
|
||||
return /360buyimg\.com\/sku\/jfs\//.test(normalized) &&
|
||||
!/\/shaidan\//.test(normalized);
|
||||
}
|
||||
function rankJdDetailImage(url) {
|
||||
const normalized = normalizeJdImageSize(url);
|
||||
if (/\.jpe?g(?:\.avif)?(?:$|[?#])/.test(normalized))
|
||||
return 0;
|
||||
if (/\.(?:png|webp)(?:\.avif)?(?:$|[?#])/.test(normalized))
|
||||
return 1;
|
||||
if (/\.gif(?:$|[?#])/.test(normalized))
|
||||
return 3;
|
||||
if (/\.avif(?:$|[?#])/.test(normalized))
|
||||
return 2;
|
||||
return 4;
|
||||
}
|
||||
function orderJdDetailImages(urls, options = {}) {
|
||||
const allowWareGraphicSku = Boolean(options.allowWareGraphicSku);
|
||||
return [...new Set(urls)]
|
||||
.filter((url) => isJdDetailImage(url) || (allowWareGraphicSku && isJdWareGraphicDetailImage(url)))
|
||||
.map((url, index) => ({ url, index, rank: rankJdDetailImage(url) }))
|
||||
.sort((a, b) => a.rank - b.rank || a.index - b.index)
|
||||
.map((item) => item.url);
|
||||
}
|
||||
function extractDetailImagesFromDom(maxImages) {
|
||||
const detailTitleParent = document.querySelector('#SPXQ-title')?.parentElement;
|
||||
const safeDetailTitleParent = detailTitleParent && detailTitleParent !== document.body && detailTitleParent !== document.documentElement
|
||||
? detailTitleParent
|
||||
: null;
|
||||
const selectorRoots = [
|
||||
'#J-detail',
|
||||
'#J-detail-content',
|
||||
'#detail',
|
||||
'.detail',
|
||||
'.detail-content',
|
||||
'.detail-content-wrap',
|
||||
'.ssd-module-wrap',
|
||||
'#SPXQ-title + *',
|
||||
];
|
||||
const scopedRoots = [
|
||||
safeDetailTitleParent,
|
||||
...selectorRoots.flatMap((selector) => Array.from(document.querySelectorAll(selector))),
|
||||
].filter((root) => root && root !== document.body && root !== document.documentElement);
|
||||
const scoped = [
|
||||
...scopedRoots.flatMap((root) => collectImageUrlsFrom(root, { ignoreContexts: true })),
|
||||
...collectImageUrlsFromFramesAndScripts(),
|
||||
];
|
||||
return orderJdDetailImages(scoped).slice(0, maxImages);
|
||||
}
|
||||
async function extractDetailImagesFromPage(maxImages, sku) {
|
||||
const scoped = extractDetailImagesFromDom(maxImages);
|
||||
const fallback = await collectImageUrlsFromFallbackSources();
|
||||
const wareGraphic = await collectImageUrlsFromWareGraphicResources(sku);
|
||||
const regularImages = orderJdDetailImages([...scoped, ...fallback]);
|
||||
const wareGraphicImages = orderJdDetailImages(wareGraphic, { allowWareGraphicSku: true });
|
||||
return orderJdDetailImages([...regularImages, ...wareGraphicImages], { allowWareGraphicSku: true }).slice(0, maxImages);
|
||||
}
|
||||
function getJdDetailScrollSnapshot(maxImages) {
|
||||
const doc = document.scrollingElement || document.documentElement || document.body;
|
||||
const scrollY = window.scrollY || window.pageYOffset || doc?.scrollTop || 0;
|
||||
const viewportHeight = window.innerHeight || document.documentElement?.clientHeight || 0;
|
||||
const scrollHeight = Math.max(doc?.scrollHeight || 0, document.documentElement?.scrollHeight || 0, document.body?.scrollHeight || 0);
|
||||
return {
|
||||
detailImageCount: extractDetailImagesFromDom(maxImages).length,
|
||||
scrollY,
|
||||
viewportHeight,
|
||||
scrollHeight,
|
||||
nearBottom: scrollY + viewportHeight >= scrollHeight - 120,
|
||||
};
|
||||
}
|
||||
function scrollJdDetailStep() {
|
||||
const step = Math.max(900, Math.floor((window.innerHeight || 900) * 0.9));
|
||||
window.scrollBy(0, step);
|
||||
return step;
|
||||
}
|
||||
async function scrollJdDetailIntoView() {
|
||||
const tab = document.querySelector('#SPXQ-tab-column');
|
||||
const title = document.querySelector('#SPXQ-title');
|
||||
if (tab)
|
||||
tab.click();
|
||||
title?.scrollIntoView({ block: 'start' });
|
||||
const step = Math.max(900, Math.floor((window.innerHeight || 900) * 0.9));
|
||||
for (let i = 0; i < 2; i++) {
|
||||
window.scrollBy(0, step);
|
||||
await new Promise((resolve) => setTimeout(resolve, 250));
|
||||
}
|
||||
}
|
||||
function extractMainImages(maxImages) {
|
||||
const roots = [
|
||||
document.querySelector('._gallery_116km_1'),
|
||||
...Array.from(document.querySelectorAll('[class*="_gallery_"]')),
|
||||
document.querySelector('.preview-wrap'),
|
||||
document.querySelector('#spec-img')?.parentElement,
|
||||
].filter(Boolean);
|
||||
const urls = roots.flatMap((root) => collectImageUrlsFrom(root).map(normalizeJdImageSize));
|
||||
return [...new Set(urls)]
|
||||
.filter(isJdMainImage)
|
||||
.slice(0, maxImages);
|
||||
}
|
||||
function extractAvifImages(imageUrls, maxImages) {
|
||||
const unique = [...new Set(imageUrls.map(normalizeJdImageSize).filter(Boolean))];
|
||||
const unique = [...new Set(imageUrls.filter(Boolean))];
|
||||
return unique
|
||||
.filter((url) => url.includes('.avif') && isJdDetailImage(url))
|
||||
.filter((url) => url.includes('.avif') && url.includes('pcpubliccms'))
|
||||
.slice(0, maxImages);
|
||||
}
|
||||
function extractPriceFromPayload(payload) {
|
||||
const items = Array.isArray(payload) ? payload : [];
|
||||
const item = items.find((entry) => entry && typeof entry === 'object');
|
||||
for (const key of ['p', 'op', 'm']) {
|
||||
const value = item?.[key];
|
||||
if (value && value !== '-1.00')
|
||||
return String(value);
|
||||
}
|
||||
return '';
|
||||
}
|
||||
function normalizePriceText(text) {
|
||||
const match = String(text || '').replace(/\s+/g, '').match(/(?:¥|¥)?(\d{2,7}(?:\.\d{1,2})?)/);
|
||||
return match ? match[1] : '';
|
||||
}
|
||||
function extractPriceFromDom(sku) {
|
||||
const selectors = [
|
||||
`.J-p-${sku}`,
|
||||
'[class*="price"] [class*="num"]',
|
||||
'[class*="price"]',
|
||||
'.p-price strong',
|
||||
'.price.jd-price',
|
||||
];
|
||||
for (const selector of selectors) {
|
||||
for (const el of document.querySelectorAll(selector)) {
|
||||
const price = normalizePriceText(el.textContent || '');
|
||||
if (price)
|
||||
return price;
|
||||
}
|
||||
}
|
||||
for (const el of document.querySelectorAll('span, strong, div')) {
|
||||
const text = (el.textContent || '').trim();
|
||||
if (!/预售价|到手价|秒杀价|京东价|¥|¥/.test(text))
|
||||
continue;
|
||||
const direct = normalizePriceText(text);
|
||||
if (direct)
|
||||
return direct;
|
||||
const parentPrice = normalizePriceText(el.parentElement?.textContent || '');
|
||||
if (parentPrice)
|
||||
return parentPrice;
|
||||
}
|
||||
return '';
|
||||
}
|
||||
async function fetchJdPrice(sku) {
|
||||
const controller = new AbortController();
|
||||
const timeout = setTimeout(() => controller.abort(), 3000);
|
||||
try {
|
||||
const resp = await fetch(`https://p.3.cn/prices/mgets?skuIds=J_${encodeURIComponent(sku)}&type=1`, {
|
||||
credentials: 'include',
|
||||
signal: controller.signal,
|
||||
});
|
||||
if (!resp.ok)
|
||||
return '';
|
||||
return extractPriceFromPayload(await resp.json());
|
||||
}
|
||||
catch {
|
||||
return '';
|
||||
}
|
||||
finally {
|
||||
clearTimeout(timeout);
|
||||
}
|
||||
}
|
||||
function extractSpecsFromText(text) {
|
||||
const specs = {};
|
||||
const lines = String(text || '').split('\n').map((line) => line.trim()).filter(Boolean);
|
||||
const allowedKeys = new Set(['品牌', '商品名称', '商品编号', '商品毛重', '商品产地', '货号', '类型', '能效等级', '洗涤容量', '烘干容量', '排水方式', '颜色', '型号', '系列', '系列品', '款式', '版本', '规格', '容量']);
|
||||
const setSpec = (key, val) => {
|
||||
const normalizedKey = String(key || '').trim().replace(/[::]+$/, '');
|
||||
const normalizedVal = String(val || '').trim();
|
||||
if (!allowedKeys.has(normalizedKey))
|
||||
return;
|
||||
if (!normalizedVal || normalizedVal.length > 120)
|
||||
return;
|
||||
if (/^(服务|支付定金|加入购物车|立即购买|首页|购物车|我的|客服|品牌闪购|以旧换新)$/.test(normalizedVal))
|
||||
return;
|
||||
if (!specs[normalizedKey])
|
||||
specs[normalizedKey] = normalizedVal;
|
||||
};
|
||||
for (const line of lines) {
|
||||
const compactMatch = line.match(/^([^::]{1,12})[::]\s*(.{1,120})$/);
|
||||
if (compactMatch) {
|
||||
setSpec(compactMatch[1], compactMatch[2]);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
for (let i = 0; i < lines.length - 1; i++) {
|
||||
const key = lines[i].replace(/[::]+$/, '');
|
||||
const val = lines[i + 1];
|
||||
if (allowedKeys.has(key) && !allowedKeys.has(val)) {
|
||||
setSpec(key, val);
|
||||
}
|
||||
}
|
||||
return specs;
|
||||
}
|
||||
function extractSpecs() {
|
||||
const specs = {};
|
||||
const setSpec = (label, value) => {
|
||||
const normalizedLabel = String(label || '').trim().replace(/[::]+$/, '');
|
||||
const normalizedValue = String(value || '').replace(/\s+/g, ' ').trim();
|
||||
if (!normalizedLabel || !normalizedValue)
|
||||
return;
|
||||
specs[normalizedLabel] = normalizedValue;
|
||||
};
|
||||
const selectedText = (root) => {
|
||||
const selected = root.querySelector('.specification-item-sku--selected, .specification-series-item--selected, .selected, [class*="selected"]');
|
||||
if (!selected)
|
||||
return '';
|
||||
return selected.querySelector('[class*="text"]')?.textContent?.trim() ||
|
||||
selected.textContent?.trim() ||
|
||||
selected.querySelector('img')?.getAttribute('alt')?.trim() ||
|
||||
'';
|
||||
};
|
||||
for (const el of document.querySelectorAll('.specification-series-layout')) {
|
||||
const label = el.querySelector('.layout-label')?.textContent?.trim();
|
||||
setSpec(label, selectedText(el));
|
||||
}
|
||||
for (const el of document.querySelectorAll('.specification-group')) {
|
||||
const label = el.querySelector('.specification-label, .specification-group-label, .label')?.textContent?.trim();
|
||||
setSpec(label, selectedText(el));
|
||||
}
|
||||
const attrsRoot = document.querySelector('#SPXQ-title')?.parentElement?.querySelector('.attrs') ||
|
||||
document.querySelector('#parameter2') ||
|
||||
document.querySelector('.Ptable');
|
||||
if (attrsRoot) {
|
||||
Object.assign(specs, extractSpecsFromText(attrsRoot.innerText || attrsRoot.textContent || ''));
|
||||
}
|
||||
return specs;
|
||||
}
|
||||
function detectJdPageState(expectedSku) {
|
||||
const href = location.href;
|
||||
const title = document.title || '';
|
||||
const bodyText = document.body?.innerText || document.body?.textContent || '';
|
||||
const hasProductMarker = Boolean(document.querySelector('.product-title, .sku-title, #spec-list, #J-detail, #SPXQ-title, [class*="_gallery_"]'));
|
||||
const text = `${title}\n${bodyText}`;
|
||||
const isLoginPage = /passport\.jd\.com|\/login\.aspx/.test(href) || /京东-欢迎登录|京东登录/.test(title);
|
||||
const hasSecurityChallenge = /risk_handler|安全验证|安全校验|完成安全验证|滑块|captcha|访问过于频繁|验证中心|京东验证/.test(`${href}\n${text}`);
|
||||
const loginOnlyWithoutProduct = /请登录|登录/.test(text) && !hasProductMarker;
|
||||
const looksBlocked = isLoginPage || hasSecurityChallenge || loginOnlyWithoutProduct;
|
||||
const onExpectedItemUrl = new RegExp(`item\.jd\.com/${expectedSku}\.html`).test(href);
|
||||
return {
|
||||
href,
|
||||
title,
|
||||
isProductPage: hasProductMarker && onExpectedItemUrl && !looksBlocked,
|
||||
hasProductMarker,
|
||||
onExpectedItemUrl,
|
||||
looksBlocked,
|
||||
isLoginPage,
|
||||
hasSecurityChallenge,
|
||||
};
|
||||
}
|
||||
cli({
|
||||
site: 'jd',
|
||||
name: 'item',
|
||||
description: '京东商品详情(价格、店铺、规格参数、主图、详情图)',
|
||||
description: '京东商品详情(价格、店铺、规格参数、AVIF 图片)',
|
||||
domain: 'item.jd.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
args: [
|
||||
@@ -556,174 +27,71 @@ cli({
|
||||
{
|
||||
name: 'images',
|
||||
type: 'int',
|
||||
default: 200,
|
||||
help: '图片数量上限(默认200)',
|
||||
default: 10,
|
||||
help: 'AVIF 图片数量上限(默认10)',
|
||||
},
|
||||
],
|
||||
columns: ['title', 'price', 'shop', 'specs', 'mainImages', 'detailImages'],
|
||||
columns: ['title', 'price', 'shop', 'specs', 'avifImages'],
|
||||
func: async (page, kwargs) => {
|
||||
const sku = normalizeJdSkuInput(kwargs.sku);
|
||||
const maxImages = normalizePositiveInt(kwargs.images, 200);
|
||||
const sku = kwargs.sku;
|
||||
const maxImages = kwargs.images;
|
||||
const url = `https://item.jd.com/${sku}.html`;
|
||||
const currentHref = await page.evaluate(`location.href`).catch(() => '');
|
||||
if (!currentHref.includes(`item.jd.com/${sku}.html`)) {
|
||||
await page.goto(url, { waitUntil: 'load' });
|
||||
await page.wait(2);
|
||||
}
|
||||
const initialState = await page.evaluate(`(() => {
|
||||
const detectJdPageState = ${detectJdPageState.toString()};
|
||||
return detectJdPageState(${JSON.stringify(sku)});
|
||||
})()`).catch(() => null);
|
||||
if (!initialState?.looksBlocked) {
|
||||
await page.evaluate(`
|
||||
(async () => {
|
||||
const scrollJdDetailIntoView = ${scrollJdDetailIntoView.toString()};
|
||||
return scrollJdDetailIntoView();
|
||||
})()
|
||||
`);
|
||||
await page.wait(1.5);
|
||||
let previousDetailImageCount = -1;
|
||||
let stableRounds = 0;
|
||||
for (let i = 0; i < 30; i++) {
|
||||
const snapshot = await page.evaluate(`(() => {
|
||||
const normalizeJdImageUrl = ${normalizeJdImageUrl.toString()};
|
||||
const normalizeJdImageSize = ${normalizeJdImageSize.toString()};
|
||||
const collectImageUrlsFrom = ${collectImageUrlsFrom.toString()};
|
||||
const collectImageUrlsFromText = ${collectImageUrlsFromText.toString()};
|
||||
const collectImageUrlsFromFramesAndScripts = ${collectImageUrlsFromFramesAndScripts.toString()};
|
||||
const collectImageUrlsFromPayload = ${collectImageUrlsFromPayload.toString()};
|
||||
const collectImageUrlsFromPageDataObjects = ${collectImageUrlsFromPageDataObjects.toString()};
|
||||
const collectImageUrlsFromNetworkResources = ${collectImageUrlsFromNetworkResources.toString()};
|
||||
const collectImageUrlsFromWareGraphicText = ${collectImageUrlsFromWareGraphicText.toString()};
|
||||
const collectImageUrlsFromWareGraphicResources = ${collectImageUrlsFromWareGraphicResources.toString()};
|
||||
const collectImageUrlsFromFallbackSources = ${collectImageUrlsFromFallbackSources.toString()};
|
||||
const isJdDetailImage = ${isJdDetailImage.toString()};
|
||||
const isJdWareGraphicDetailImage = ${isJdWareGraphicDetailImage.toString()};
|
||||
const rankJdDetailImage = ${rankJdDetailImage.toString()};
|
||||
const orderJdDetailImages = ${orderJdDetailImages.toString()};
|
||||
const extractDetailImagesFromDom = ${extractDetailImagesFromDom.toString()};
|
||||
const extractDetailImagesFromPage = ${extractDetailImagesFromPage.toString()};
|
||||
const getJdDetailScrollSnapshot = ${getJdDetailScrollSnapshot.toString()};
|
||||
const scrollJdDetailIntoView = ${scrollJdDetailIntoView.toString()};
|
||||
if (document.querySelector('#SPXQ-title') && window.scrollY < 1000) {
|
||||
return scrollJdDetailIntoView().then(() => getJdDetailScrollSnapshot(${maxImages}));
|
||||
}
|
||||
return getJdDetailScrollSnapshot(${maxImages});
|
||||
})()`).catch(() => null);
|
||||
if (!snapshot)
|
||||
break;
|
||||
stableRounds = snapshot.detailImageCount === previousDetailImageCount ? stableRounds + 1 : 0;
|
||||
previousDetailImageCount = snapshot.detailImageCount;
|
||||
if (snapshot.nearBottom && stableRounds >= 2)
|
||||
break;
|
||||
await page.evaluate(`(() => {
|
||||
const scrollJdDetailStep = ${scrollJdDetailStep.toString()};
|
||||
return scrollJdDetailStep();
|
||||
})()`);
|
||||
await page.wait(0.8);
|
||||
}
|
||||
await page.goto(url, { waitUntil: 'load' });
|
||||
await page.wait(2);
|
||||
// 滚动加载商品详情区域中的延迟图片
|
||||
for (let i = 0; i < 6; i++) {
|
||||
await page.evaluate(`window.scrollTo(0, ${i * 2500})`);
|
||||
await page.wait(1);
|
||||
}
|
||||
await page.evaluate(`window.scrollTo(0, document.body.scrollHeight)`);
|
||||
await page.wait(2);
|
||||
const data = await page.evaluate(`
|
||||
(async () => {
|
||||
(() => {
|
||||
const maxImg = ${maxImages};
|
||||
const normalizeJdImageUrl = ${normalizeJdImageUrl.toString()};
|
||||
const normalizeJdImageSize = ${normalizeJdImageSize.toString()};
|
||||
const isJdMainImage = ${isJdMainImage.toString()};
|
||||
const collectImageUrlsFrom = ${collectImageUrlsFrom.toString()};
|
||||
const collectImageUrlsFromText = ${collectImageUrlsFromText.toString()};
|
||||
const collectImageUrlsFromFramesAndScripts = ${collectImageUrlsFromFramesAndScripts.toString()};
|
||||
const collectImageUrlsFromPayload = ${collectImageUrlsFromPayload.toString()};
|
||||
const collectImageUrlsFromPageDataObjects = ${collectImageUrlsFromPageDataObjects.toString()};
|
||||
const collectImageUrlsFromNetworkResources = ${collectImageUrlsFromNetworkResources.toString()};
|
||||
const collectImageUrlsFromWareGraphicText = ${collectImageUrlsFromWareGraphicText.toString()};
|
||||
const collectImageUrlsFromWareGraphicResources = ${collectImageUrlsFromWareGraphicResources.toString()};
|
||||
const collectImageUrlsFromFallbackSources = ${collectImageUrlsFromFallbackSources.toString()};
|
||||
const isJdDetailImage = ${isJdDetailImage.toString()};
|
||||
const isJdWareGraphicDetailImage = ${isJdWareGraphicDetailImage.toString()};
|
||||
const rankJdDetailImage = ${rankJdDetailImage.toString()};
|
||||
const orderJdDetailImages = ${orderJdDetailImages.toString()};
|
||||
const extractMainImages = ${extractMainImages.toString()};
|
||||
const extractDetailImagesFromDom = ${extractDetailImagesFromDom.toString()};
|
||||
const extractDetailImagesFromPage = ${extractDetailImagesFromPage.toString()};
|
||||
const extractPriceFromPayload = ${extractPriceFromPayload.toString()};
|
||||
const fetchJdPrice = ${fetchJdPrice.toString()};
|
||||
const extractSpecsFromText = ${extractSpecsFromText.toString()};
|
||||
const extractSpecs = ${extractSpecs.toString()};
|
||||
const detectJdPageState = ${detectJdPageState.toString()};
|
||||
const pageState = detectJdPageState(${JSON.stringify(sku)});
|
||||
const normalizePriceText = ${normalizePriceText.toString()};
|
||||
const extractPriceFromDom = ${extractPriceFromDom.toString()};
|
||||
const apiPrice = await fetchJdPrice(${JSON.stringify(sku)});
|
||||
const domPrice = extractPriceFromDom(${JSON.stringify(sku)});
|
||||
const price = apiPrice || domPrice || 'not found';
|
||||
// 尝试多种价格选择器
|
||||
const skuMatch = location.pathname.match(/(\\d+)\\.html/);
|
||||
const sku = skuMatch ? skuMatch[1] : '';
|
||||
const priceEl = document.querySelector('.J-p-' + sku) ||
|
||||
document.querySelector('[class*="price"] [class*="num"]') ||
|
||||
document.querySelector('.p-price strong') ||
|
||||
document.querySelector('.price.jd-price');
|
||||
const price = priceEl?.textContent?.trim() || 'not found';
|
||||
|
||||
// 标题
|
||||
const title = document.querySelector('.product-title')?.textContent?.trim() ||
|
||||
document.querySelector('.sku-title')?.textContent?.trim() ||
|
||||
document.title.split('-')[0].trim();
|
||||
|
||||
const shop = document.querySelector('.J-shop-name')?.textContent?.trim() ||
|
||||
document.querySelector('.top-name')?.textContent?.trim() ||
|
||||
document.querySelector('[class*="shop"] [class*="name"]')?.textContent?.trim() ||
|
||||
'京东自营';
|
||||
// 店铺
|
||||
const shop = document.querySelector('.J-shop-name')?.textContent?.trim() || '京东自营';
|
||||
|
||||
// 所有图片
|
||||
const allImgs = Array.from(document.querySelectorAll('img[src*="360buyimg.com"]'));
|
||||
const srcs = allImgs.map(img => img.src).filter(Boolean);
|
||||
|
||||
const mainImages = extractMainImages(maxImg);
|
||||
const detailImages = await extractDetailImagesFromPage(maxImg, ${JSON.stringify(sku)});
|
||||
const specs = extractSpecs();
|
||||
// 所有 avif 图片(去重,只保留 pcpubliccms CDN)
|
||||
const avifImages = ${extractAvifImages.toString()}(srcs, maxImg);
|
||||
|
||||
const result = { title, price, shop, specs, mainImages, detailImages, totalImages: new Set(srcs).size, pageState };
|
||||
if (!pageState.isProductPage) {
|
||||
result.error = pageState.looksBlocked
|
||||
? 'JD page is blocked by login/security verification'
|
||||
: 'JD product page was not loaded';
|
||||
result.pageState = pageState;
|
||||
// 规格参数:从页面文本提取
|
||||
const text = document.body.innerText;
|
||||
const specMatch = text.match(/商品编号[\\s\\S]*?(?=包装清单|\\n\\n|$)/);
|
||||
let specs = {};
|
||||
if (specMatch) {
|
||||
const lines = specMatch[0].split('\\n').filter(l => l.trim());
|
||||
for (let i = 0; i < lines.length - 1; i += 2) {
|
||||
const key = lines[i].trim();
|
||||
const val = lines[i + 1]?.trim() || '';
|
||||
if (key && val && key !== '商品编号') {
|
||||
specs[key] = val;
|
||||
}
|
||||
}
|
||||
}
|
||||
return result;
|
||||
|
||||
return { title, price, shop, specs, avifImages, totalImages: new Set(srcs).size };
|
||||
})()
|
||||
`);
|
||||
if (data?.error) {
|
||||
if (data?.pageState?.looksBlocked) {
|
||||
throw new AuthRequiredError('item.jd.com', data.error);
|
||||
}
|
||||
throw new CommandExecutionError(data.error);
|
||||
}
|
||||
if (maxImages > 0 && data?.pageState?.isProductPage && (!Array.isArray(data.detailImages) || data.detailImages.length === 0)) {
|
||||
throw new CommandExecutionError('JD item detail images were not found', 'The product page loaded, but no detail images were detected from DOM, scripts, frames, page data, or WareGraphic fallback.');
|
||||
}
|
||||
return [data];
|
||||
},
|
||||
});
|
||||
export const __test__ = {
|
||||
normalizePositiveInt,
|
||||
normalizeJdSkuInput,
|
||||
normalizeJdImageUrl,
|
||||
normalizeJdImageSize,
|
||||
isJdMainImage,
|
||||
collectImageUrlsFrom,
|
||||
collectImageUrlsFromText,
|
||||
collectImageUrlsFromFramesAndScripts,
|
||||
collectImageUrlsFromPayload,
|
||||
collectImageUrlsFromPageDataObjects,
|
||||
collectImageUrlsFromNetworkResources,
|
||||
collectImageUrlsFromWareGraphicText,
|
||||
collectImageUrlsFromWareGraphicResources,
|
||||
collectImageUrlsFromFallbackSources,
|
||||
isJdDetailImage,
|
||||
isJdWareGraphicDetailImage,
|
||||
rankJdDetailImage,
|
||||
orderJdDetailImages,
|
||||
getJdDetailScrollSnapshot,
|
||||
scrollJdDetailStep,
|
||||
extractMainImages,
|
||||
extractDetailImagesFromDom,
|
||||
extractDetailImagesFromPage,
|
||||
extractAvifImages,
|
||||
extractPriceFromPayload,
|
||||
normalizePriceText,
|
||||
extractPriceFromDom,
|
||||
extractSpecsFromText,
|
||||
extractSpecs,
|
||||
detectJdPageState,
|
||||
};
|
||||
|
||||
+7
-318
@@ -1,17 +1,7 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { JSDOM } from 'jsdom';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { AuthRequiredError } from '@jackwener/opencli/errors';
|
||||
import { __test__ } from './item.js';
|
||||
import './item.js';
|
||||
const originalPerformance = globalThis.performance;
|
||||
const originalWindow = globalThis.window;
|
||||
const originalDocument = globalThis.document;
|
||||
function restoreGlobals() {
|
||||
globalThis.performance = originalPerformance;
|
||||
globalThis.window = originalWindow;
|
||||
globalThis.document = originalDocument;
|
||||
}
|
||||
describe('jd item adapter', () => {
|
||||
const command = getRegistry().get('jd/item');
|
||||
it('registers the command with correct shape', () => {
|
||||
@@ -28,79 +18,15 @@ describe('jd item adapter', () => {
|
||||
expect(skuArg.required).toBe(true);
|
||||
expect(skuArg.positional).toBe(true);
|
||||
});
|
||||
it('has images arg with default 200', () => {
|
||||
it('has images arg with default 10', () => {
|
||||
const imagesArg = command.args.find((a) => a.name === 'images');
|
||||
expect(imagesArg).toBeDefined();
|
||||
expect(imagesArg.default).toBe(200);
|
||||
});
|
||||
it('fails fast when JD blocks the item page', async () => {
|
||||
const page = {
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
|
||||
.mockResolvedValueOnce({ looksBlocked: true })
|
||||
.mockResolvedValueOnce({
|
||||
error: 'JD page is blocked by login/security verification',
|
||||
pageState: { looksBlocked: true },
|
||||
}),
|
||||
goto: vi.fn(),
|
||||
wait: vi.fn(),
|
||||
};
|
||||
await expect(command.func(page, { sku: '100328272886', images: 5 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
it('fails fast when a loaded product page has no detail images', async () => {
|
||||
const page = {
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
|
||||
.mockResolvedValueOnce({ looksBlocked: false })
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockResolvedValueOnce(null)
|
||||
.mockResolvedValueOnce({
|
||||
title: 'JD product',
|
||||
price: '100',
|
||||
shop: '京东自营',
|
||||
specs: {},
|
||||
mainImages: ['https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg'],
|
||||
detailImages: [],
|
||||
pageState: { isProductPage: true, looksBlocked: false },
|
||||
}),
|
||||
goto: vi.fn(),
|
||||
wait: vi.fn(),
|
||||
};
|
||||
await expect(command.func(page, { sku: '100328272886', images: 5 })).rejects.toThrow('JD item detail images were not found');
|
||||
});
|
||||
it('allows zero-image requests to skip detail image fail-fast', async () => {
|
||||
const data = {
|
||||
title: 'JD product',
|
||||
price: '100',
|
||||
shop: '京东自营',
|
||||
specs: {},
|
||||
mainImages: [],
|
||||
detailImages: [],
|
||||
pageState: { isProductPage: true, looksBlocked: false },
|
||||
};
|
||||
const page = {
|
||||
evaluate: vi.fn()
|
||||
.mockResolvedValueOnce('https://item.jd.com/100328272886.html')
|
||||
.mockResolvedValueOnce({ looksBlocked: false })
|
||||
.mockResolvedValueOnce(undefined)
|
||||
.mockResolvedValueOnce(null)
|
||||
.mockResolvedValueOnce(data),
|
||||
goto: vi.fn(),
|
||||
wait: vi.fn(),
|
||||
};
|
||||
await expect(command.func(page, { sku: '100328272886', images: 0 })).resolves.toEqual([data]);
|
||||
});
|
||||
it('normalizes JD item URL input to a SKU before building selectors', () => {
|
||||
expect(__test__.normalizeJdSkuInput('100328272886')).toBe('100328272886');
|
||||
expect(__test__.normalizeJdSkuInput('https://item.jd.com/100328272886.html?purchasetab=gfgm')).toBe('100328272886');
|
||||
expect(__test__.normalizeJdSkuInput('skuId=10218494560141')).toBe('10218494560141');
|
||||
expect(imagesArg.default).toBe(10);
|
||||
});
|
||||
it('includes expected columns', () => {
|
||||
expect(command.columns).toEqual(expect.arrayContaining(['title', 'price', 'shop', 'specs', 'mainImages', 'detailImages']));
|
||||
expect(command.columns).not.toContain('avifImages');
|
||||
expect(command.columns).toEqual(expect.arrayContaining(['title', 'price', 'shop', 'specs', 'avifImages']));
|
||||
});
|
||||
it('extracts only detail avif images and respects the limit', () => {
|
||||
it('extracts only pcpubliccms avif images and respects the limit', () => {
|
||||
const result = __test__.extractAvifImages([
|
||||
'https://img14.360buyimg.com/n1/jfs/t1/normal.jpg',
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.avif',
|
||||
@@ -110,245 +36,8 @@ describe('jd item adapter', () => {
|
||||
'https://example.com/not-jd.avif',
|
||||
], 2);
|
||||
expect(result).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.avif',
|
||||
'https://pcpubliccms.jd.com/image1.avif',
|
||||
'https://pcpubliccms.jd.com/image2.avif?x=1',
|
||||
]);
|
||||
});
|
||||
it('treats WareGraphic sku images as detail images only for WareGraphic fallback', () => {
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/sku/jfs/t1/color-option.gif')).toBe(false);
|
||||
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/jfs/t1/ware-graphic.jpg')).toBe(true);
|
||||
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/s228x228_jfs/t1/thumb.jpg')).toBe(false);
|
||||
});
|
||||
it('keeps main gallery images while ignoring unrelated contexts for detail images', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div class="_gallery_116km_1">
|
||||
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg" />
|
||||
<img data-src="//img10.360buyimg.com/n1/jfs/t1/main-b.jpg" />
|
||||
</div>
|
||||
<div class="recommend">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/recommend.jpg.avif" />
|
||||
</div>
|
||||
<div class="detail-content-wrap">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
try {
|
||||
expect(__test__.extractMainImages(10)).toEqual([
|
||||
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg',
|
||||
'https://img10.360buyimg.com/n1/jfs/t1/main-b.jpg',
|
||||
]);
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from computed background images', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div id="J-detail">
|
||||
<div class="ssd-module computed-bg"></div>
|
||||
<div class="ssd-module ignored-bg"></div>
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
const previousGetComputedStyle = globalThis.getComputedStyle;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.getComputedStyle = ((element) => ({
|
||||
background: '',
|
||||
backgroundImage: element.classList.contains('computed-bg')
|
||||
? 'url("//img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif")'
|
||||
: 'none',
|
||||
}));
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
globalThis.getComputedStyle = previousGetComputedStyle;
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from inline JSON-like script text', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<script>
|
||||
window.__DETAIL_DATA__ = {
|
||||
images: [
|
||||
"https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif",
|
||||
"//img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif"
|
||||
]
|
||||
};
|
||||
</script>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from same-origin iframe content', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<iframe id="detail-frame"></iframe>
|
||||
`, { url: 'https://item.jd.com/100328272886.html' });
|
||||
const frameDom = new JSDOM(`
|
||||
<div id="J-detail">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif" />
|
||||
<div style="background-image:url(//img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif)"></div>
|
||||
</div>
|
||||
`, { url: 'https://item.jd.com/detail-frame.html' });
|
||||
const iframe = dom.window.document.getElementById('detail-frame');
|
||||
Object.defineProperty(iframe, 'contentDocument', { value: frameDom.window.document, configurable: true });
|
||||
Object.defineProperty(iframe, 'contentWindow', { value: frameDom.window, configurable: true });
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from page data objects', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window;
|
||||
globalThis.__PAGE_DATA__ = {
|
||||
detail: {
|
||||
images: [
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
|
||||
{ src: '//img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp' },
|
||||
],
|
||||
},
|
||||
};
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
delete globalThis.__PAGE_DATA__;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from network resource text', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
const previousFetch = globalThis.fetch;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window;
|
||||
globalThis.performance = {
|
||||
getEntriesByType: () => [{ name: 'https://cdn.jd.com/detail/data.json' }],
|
||||
now: () => 0,
|
||||
};
|
||||
globalThis.fetch = (async () => ({
|
||||
ok: true,
|
||||
headers: { get: () => 'application/json' },
|
||||
text: async () => JSON.stringify({
|
||||
detail: {
|
||||
imgs: ['https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif'],
|
||||
},
|
||||
}),
|
||||
}));
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.fetch = previousFetch;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
it('collects JD detail images from WareGraphic graphicContent', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
const previousFetch = globalThis.fetch;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window;
|
||||
globalThis.performance = {
|
||||
getEntriesByType: () => [
|
||||
{ name: 'https://api.m.jd.com/client.action?functionId=pc_item_getWareGraphic&skuId=100328272886' },
|
||||
],
|
||||
now: () => 0,
|
||||
};
|
||||
globalThis.fetch = (async () => ({
|
||||
ok: true,
|
||||
headers: { get: () => 'application/json' },
|
||||
text: async () => JSON.stringify({
|
||||
data: {
|
||||
graphicContent: `
|
||||
<div style="background-image:url(//img30.360buyimg.com/sku/jfs/t1/ware-a.jpg)"></div>
|
||||
<img src="https://img30.360buyimg.com/sku/jfs/t1/ware-b.png" />
|
||||
<a href="https://item.jd.com/100328272886.html">not image</a>
|
||||
`,
|
||||
},
|
||||
}),
|
||||
}));
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img30.360buyimg.com/sku/jfs/t1/ware-a.jpg',
|
||||
'https://img30.360buyimg.com/sku/jfs/t1/ware-b.png',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.fetch = previousFetch;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
it('extracts selected specs from the newer JD spec-list DOM', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div id="spec-list" class="page-right-spec">
|
||||
<div class="horizontal-layout specification-series-layout">
|
||||
<div class="layout-label">系列品</div>
|
||||
<div class="layout-content">
|
||||
<div class="specification-series-item specification-series-item--selected">
|
||||
<span class="specification-series-item-text">【年度新品】玉兔3.0pro 12kg</span>
|
||||
</div>
|
||||
<div class="specification-series-item"><span class="specification-series-item-text">其他系列</span></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="specifications-panel-content">
|
||||
<div class="specification-group">
|
||||
<div class="specification-group-label">款式</div>
|
||||
<div class="specification-group-content">
|
||||
<div class="specification-item-sku has-image specification-item-sku--selected" title="">
|
||||
<img class="specification-item-sku-image" alt="洗烘套装" src="https://img13.360buyimg.com/pcpubliccms/s48x48_jfs/t1/spec.jpg.avif" />
|
||||
<span class="specification-item-sku-text">洗烘套装</span>
|
||||
</div>
|
||||
<div class="specification-item-sku has-image"><span class="specification-item-sku-text">滚筒单洗</span></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`);
|
||||
globalThis.document = dom.window.document;
|
||||
try {
|
||||
expect(__test__.extractSpecs()).toEqual({
|
||||
系列品: '【年度新品】玉兔3.0pro 12kg',
|
||||
款式: '洗烘套装',
|
||||
});
|
||||
}
|
||||
finally {
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,517 +0,0 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { JSDOM } from 'jsdom';
|
||||
import { __test__ } from './item.js';
|
||||
|
||||
const originalPerformance = globalThis.performance;
|
||||
const originalWindow = globalThis.window;
|
||||
const originalDocument = globalThis.document;
|
||||
|
||||
function restoreGlobals() {
|
||||
globalThis.performance = originalPerformance;
|
||||
globalThis.window = originalWindow;
|
||||
globalThis.document = originalDocument;
|
||||
}
|
||||
|
||||
describe('jd item image helpers', () => {
|
||||
it('normalizes JD item URL input to a SKU before building selectors', () => {
|
||||
expect(__test__.normalizeJdSkuInput('100328272886')).toBe('100328272886');
|
||||
expect(__test__.normalizeJdSkuInput('https://item.jd.com/100328272886.html?purchasetab=gfgm')).toBe('100328272886');
|
||||
expect(__test__.normalizeJdSkuInput('skuId=10218494560141')).toBe('10218494560141');
|
||||
});
|
||||
|
||||
it('normalizes protocol-relative and thumbnail-sized JD image URLs', () => {
|
||||
expect(__test__.normalizeJdImageUrl('//img10.360buyimg.com/imgzone/jfs/a.jpg.avif')).toBe('https://img10.360buyimg.com/imgzone/jfs/a.jpg.avif');
|
||||
expect(__test__.normalizeJdImageSize('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/a.jpg.avif')).toBe('https://img10.360buyimg.com/pcpubliccms/jfs/t1/a.jpg.avif');
|
||||
expect(__test__.normalizeJdImageSize('https://img10.360buyimg.com/n1/s450x450_jfs/t1/a.jpg.avif')).toBe('https://img10.360buyimg.com/n1/jfs/t1/a.jpg.avif');
|
||||
});
|
||||
|
||||
it('accepts only product main images for mainImages', () => {
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif')).toBe(true);
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/main.jpg.avif')).toBe(true);
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/n1/jfs/t1/main.jpg.avif')).toBe(true);
|
||||
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif')).toBe(false);
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif')).toBe(false);
|
||||
expect(__test__.isJdMainImage('https://img10.360buyimg.com/babel/jfs/t1/recommend.jpg.avif')).toBe(false);
|
||||
});
|
||||
|
||||
it('accepts only detail-area image CDN paths for detailImages', () => {
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif')).toBe(true);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/jdcms/jfs/t1/detail.jpg.avif')).toBe(true);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/babel/jfs/t1/detail.jpg.avif')).toBe(true);
|
||||
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif')).toBe(false);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/thumb.jpg.avif')).toBe(false);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/n1/jfs/t1/main.jpg.avif')).toBe(false);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/sku/jfs/t1/color-option.gif')).toBe(false);
|
||||
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/jfs/t1/ware-graphic.jpg')).toBe(true);
|
||||
expect(__test__.isJdWareGraphicDetailImage('https://img10.360buyimg.com/sku/s228x228_jfs/t1/thumb.jpg')).toBe(false);
|
||||
expect(__test__.isJdDetailImage('https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif')).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps legacy avifImages restricted to detail images only', () => {
|
||||
expect(__test__.extractAvifImages([
|
||||
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main.jpg.avif',
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif',
|
||||
'https://img10.360buyimg.com/shaidan/jfs/t1/user.jpg.avif',
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.gif',
|
||||
], 10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail.jpg.avif',
|
||||
]);
|
||||
});
|
||||
|
||||
it('prioritizes JPG detail images before PNG banners and GIFs when limiting detailImages', () => {
|
||||
expect(__test__.orderJdDetailImages([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/hero-a.gif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/banner.png.avif',
|
||||
'https://img12.360buyimg.com/imgzone/jfs/t1/326893/25/18592/117159/68c264f5F9a41addf/2c9ad60b7f390339.jpg.avif',
|
||||
'https://img12.360buyimg.com/imgzone/jfs/t1/330152/17/11906/130964/68c264f2Ffcf6e5c1/c2ccb28722dc47ce.jpg.avif',
|
||||
'https://img11.360buyimg.com/cms/jfs/t1/banner.gif',
|
||||
])).toEqual([
|
||||
'https://img12.360buyimg.com/imgzone/jfs/t1/326893/25/18592/117159/68c264f5F9a41addf/2c9ad60b7f390339.jpg.avif',
|
||||
'https://img12.360buyimg.com/imgzone/jfs/t1/330152/17/11906/130964/68c264f2Ffcf6e5c1/c2ccb28722dc47ce.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/banner.png.avif',
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/hero-a.gif',
|
||||
'https://img11.360buyimg.com/cms/jfs/t1/banner.gif',
|
||||
]);
|
||||
});
|
||||
|
||||
it('extracts valid JD price payload values', () => {
|
||||
expect(__test__.extractPriceFromPayload([{ id: 'J_100291143898', p: '6999.00' }])).toBe('6999.00');
|
||||
expect(__test__.extractPriceFromPayload([{ id: 'J_100291143898', p: '-1.00', op: '7299.00' }])).toBe('7299.00');
|
||||
expect(__test__.extractPriceFromPayload([])).toBe('');
|
||||
});
|
||||
|
||||
it('extracts visible JD price text from DOM', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div>
|
||||
<span>预售价</span>
|
||||
<span><span>¥</span><span>12221</span></span>
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.normalizePriceText('¥ 12221')).toBe('12221');
|
||||
expect(__test__.extractPriceFromDom('100291143898')).toBe('12221');
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('extracts only gallery main images and detail-container images from DOM', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div class="_gallery_116km_1">
|
||||
<img src="https://img10.360buyimg.com/pcpubliccms/s228x228_jfs/t1/main-a.jpg.avif" />
|
||||
<img data-src="//img10.360buyimg.com/n1/s450x450_jfs/t1/main-b.jpg.avif" />
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/wrong-detail-in-gallery.jpg.avif" />
|
||||
</div>
|
||||
<div class="recommend">
|
||||
<img src="https://img10.360buyimg.com/babel/jfs/t1/recommend.jpg.avif" />
|
||||
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/recommend-main-like.jpg.avif" />
|
||||
</div>
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<div class="detail-content-wrap">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-hero.gif" />
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
|
||||
<source srcset="https://img11.360buyimg.com/imgzone/jfs/t1/detail-source.jpg.avif 1x" />
|
||||
<img src="https://img10.360buyimg.com/pcpubliccms/jfs/t1/wrong-main-in-detail.jpg.avif" />
|
||||
<img src="https://img10.360buyimg.com/shaidan/jfs/t1/wrong-user.jpg.avif" />
|
||||
</div>
|
||||
<div id="spec-list">
|
||||
<img src="https://img10.360buyimg.com/pcpubliccms/s48x48_jfs/t1/wrong-sku-option.jpg.avif" />
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.extractMainImages(10)).toEqual([
|
||||
'https://img10.360buyimg.com/pcpubliccms/jfs/t1/main-a.jpg.avif',
|
||||
'https://img10.360buyimg.com/n1/jfs/t1/main-b.jpg.avif',
|
||||
]);
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/detail-source.jpg.avif',
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-hero.gif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from computed background images', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div id="J-detail">
|
||||
<div class="ssd-module computed-bg"></div>
|
||||
<div class="ssd-module ignored-bg"></div>
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
const previousGetComputedStyle = globalThis.getComputedStyle;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.getComputedStyle = ((element: Element) => ({
|
||||
background: '',
|
||||
backgroundImage: element.classList.contains('computed-bg')
|
||||
? 'url("//img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif")'
|
||||
: 'none',
|
||||
})) as typeof getComputedStyle;
|
||||
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/computed-detail.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
globalThis.getComputedStyle = previousGetComputedStyle;
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from inline JSON-like script text', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<script>
|
||||
window.__DETAIL_DATA__ = {
|
||||
images: [
|
||||
"https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif",
|
||||
"//img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif"
|
||||
]
|
||||
};
|
||||
</script>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/script-detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/script-detail-b.gif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from same-origin iframe content', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<iframe id="detail-frame"></iframe>
|
||||
`, { url: 'https://item.jd.com/100328272886.html' });
|
||||
const frameDom = new JSDOM(`
|
||||
<div id="J-detail">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif" />
|
||||
<div style="background-image:url(//img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif)"></div>
|
||||
</div>
|
||||
`, { url: 'https://item.jd.com/detail-frame.html' });
|
||||
const iframe = dom.window.document.getElementById('detail-frame') as HTMLIFrameElement;
|
||||
Object.defineProperty(iframe, 'contentDocument', { value: frameDom.window.document, configurable: true });
|
||||
Object.defineProperty(iframe, 'contentWindow', { value: frameDom.window, configurable: true });
|
||||
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/frame-detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/cms/jfs/t1/frame-detail-b.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from page data objects', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window as unknown as Window;
|
||||
(globalThis as typeof globalThis & { __PAGE_DATA__?: unknown }).__PAGE_DATA__ = {
|
||||
detail: {
|
||||
images: [
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
|
||||
{ src: '//img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp' },
|
||||
],
|
||||
},
|
||||
};
|
||||
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/page-data-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/page-data-b.webp',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
delete (globalThis as typeof globalThis & { __PAGE_DATA__?: unknown }).__PAGE_DATA__;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from network resource text', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
const previousFetch = globalThis.fetch;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window as unknown as Window;
|
||||
const fakePerformance = {
|
||||
getEntriesByType: () => [{ name: 'https://cdn.jd.com/detail/data.json' }],
|
||||
now: () => 0,
|
||||
} as Performance;
|
||||
globalThis.performance = fakePerformance;
|
||||
globalThis.fetch = (async () => ({
|
||||
ok: true,
|
||||
headers: { get: () => 'application/json' },
|
||||
text: async () => JSON.stringify({
|
||||
detail: {
|
||||
imgs: ['https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif'],
|
||||
},
|
||||
}),
|
||||
})) as typeof fetch;
|
||||
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/network-detail-a.jpg.avif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.fetch = previousFetch;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it('collects JD detail images from WareGraphic graphicContent', async () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
`);
|
||||
const previousFetch = globalThis.fetch;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window as unknown as Window;
|
||||
globalThis.performance = {
|
||||
getEntriesByType: () => [
|
||||
{ name: 'https://api.m.jd.com/client.action?functionId=pc_item_getWareGraphic&skuId=100328272886' },
|
||||
],
|
||||
now: () => 0,
|
||||
} as Performance;
|
||||
globalThis.fetch = (async () => ({
|
||||
ok: true,
|
||||
headers: { get: () => 'application/json' },
|
||||
text: async () => JSON.stringify({
|
||||
data: {
|
||||
graphicContent: `
|
||||
<div style="background-image:url(//img30.360buyimg.com/sku/jfs/t1/ware-a.jpg)"></div>
|
||||
<img src="https://img30.360buyimg.com/sku/jfs/t1/ware-b.png" />
|
||||
<a href="https://item.jd.com/100328272886.html">not image</a>
|
||||
`,
|
||||
},
|
||||
}),
|
||||
})) as typeof fetch;
|
||||
|
||||
try {
|
||||
await expect(__test__.extractDetailImagesFromPage(10)).resolves.toEqual([
|
||||
'https://img30.360buyimg.com/sku/jfs/t1/ware-a.jpg',
|
||||
'https://img30.360buyimg.com/sku/jfs/t1/ware-b.png',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.fetch = previousFetch;
|
||||
restoreGlobals();
|
||||
}
|
||||
});
|
||||
|
||||
it('collects images from every repeated JD detail module', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<div class="ssd-module-wrap">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
|
||||
</div>
|
||||
<div class="ssd-module-wrap">
|
||||
<img src="https://img11.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif" />
|
||||
</div>
|
||||
<div class="ssd-module-wrap">
|
||||
<img src="https://img12.360buyimg.com/imgzone/jfs/t1/detail-c.gif" />
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.extractDetailImagesFromDom(10)).toEqual([
|
||||
'https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif',
|
||||
'https://img11.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif',
|
||||
'https://img12.360buyimg.com/imgzone/jfs/t1/detail-c.gif',
|
||||
]);
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('reports detail scroll progress so lazy-loaded detail images can stabilize', () => {
|
||||
const dom = new JSDOM(`
|
||||
<h2 id="SPXQ-title">商品详情</h2>
|
||||
<div class="detail-content-wrap">
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-a.jpg.avif" />
|
||||
<img src="https://img10.360buyimg.com/imgzone/jfs/t1/detail-b.jpg.avif" />
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
const previousWindow = globalThis.window;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.window = dom.window as unknown as Window & typeof globalThis;
|
||||
Object.defineProperty(dom.window, 'scrollY', { value: 1800, configurable: true });
|
||||
Object.defineProperty(dom.window, 'innerHeight', { value: 900, configurable: true });
|
||||
Object.defineProperty(dom.window.document.documentElement, 'scrollHeight', { value: 2600, configurable: true });
|
||||
|
||||
try {
|
||||
expect(__test__.getJdDetailScrollSnapshot(10)).toMatchObject({
|
||||
detailImageCount: 2,
|
||||
scrollY: 1800,
|
||||
viewportHeight: 900,
|
||||
scrollHeight: 2600,
|
||||
nearBottom: true,
|
||||
});
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
globalThis.window = previousWindow;
|
||||
}
|
||||
});
|
||||
|
||||
it('parses structured specs without pairing unrelated body text', () => {
|
||||
expect(__test__.extractSpecsFromText([
|
||||
'品牌:美的(Midea)',
|
||||
'商品编号',
|
||||
'100291143898',
|
||||
'洗涤容量',
|
||||
'能效等级',
|
||||
'一级能效',
|
||||
'类型',
|
||||
'加入购物车',
|
||||
].join('\n'))).toEqual({
|
||||
品牌: '美的(Midea)',
|
||||
商品编号: '100291143898',
|
||||
能效等级: '一级能效',
|
||||
});
|
||||
});
|
||||
|
||||
it('extracts selected specs from the newer JD spec-list DOM', () => {
|
||||
const dom = new JSDOM(`
|
||||
<div id="spec-list" class="page-right-spec">
|
||||
<div class="horizontal-layout specification-series-layout">
|
||||
<div class="layout-label">系列品</div>
|
||||
<div class="layout-content">
|
||||
<div class="specification-series-item specification-series-item--selected">
|
||||
<span class="specification-series-item-text">【年度新品】玉兔3.0pro 12kg</span>
|
||||
</div>
|
||||
<div class="specification-series-item"><span class="specification-series-item-text">其他系列</span></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="specifications-panel-content">
|
||||
<div class="specification-group">
|
||||
<div class="specification-group-label">款式</div>
|
||||
<div class="specification-group-content">
|
||||
<div class="specification-item-sku has-image specification-item-sku--selected" title="">
|
||||
<img class="specification-item-sku-image" alt="洗烘套装" src="https://img13.360buyimg.com/pcpubliccms/s48x48_jfs/t1/spec.jpg.avif" />
|
||||
<span class="specification-item-sku-text">洗烘套装</span>
|
||||
</div>
|
||||
<div class="specification-item-sku has-image"><span class="specification-item-sku-text">滚筒单洗</span></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`);
|
||||
const previousDocument = globalThis.document;
|
||||
globalThis.document = dom.window.document;
|
||||
|
||||
try {
|
||||
expect(__test__.extractSpecs()).toEqual({
|
||||
系列品: '【年度新品】玉兔3.0pro 12kg',
|
||||
款式: '洗烘套装',
|
||||
});
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
}
|
||||
});
|
||||
|
||||
it('detects whether the loaded page is the expected JD product page', () => {
|
||||
const dom = new JSDOM(`
|
||||
<html>
|
||||
<head><title>京东</title></head>
|
||||
<body><div>请登录</div><div class="sku-title">商品标题</div><div id="spec-list"></div></body>
|
||||
</html>
|
||||
`, { url: 'https://item.jd.com/100291143898.html' });
|
||||
const previousDocument = globalThis.document;
|
||||
const previousLocation = globalThis.location;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.location = dom.window.location;
|
||||
|
||||
try {
|
||||
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
|
||||
isProductPage: true,
|
||||
hasProductMarker: true,
|
||||
onExpectedItemUrl: true,
|
||||
looksBlocked: false,
|
||||
isLoginPage: false,
|
||||
hasSecurityChallenge: false,
|
||||
});
|
||||
document.body.innerHTML = '<div>请登录后完成安全验证</div>';
|
||||
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
|
||||
isProductPage: false,
|
||||
looksBlocked: true,
|
||||
hasSecurityChallenge: true,
|
||||
});
|
||||
const riskDom = new JSDOM(`
|
||||
<html>
|
||||
<head><title>京东验证</title></head>
|
||||
<body><div>京东验证</div></body>
|
||||
</html>
|
||||
`, { url: 'https://cfe.m.jd.com/privatedomain/risk_handler/03101900/?returnurl=https%3A%2F%2Fitem.jd.com%2F100291143898.html' });
|
||||
globalThis.document = riskDom.window.document;
|
||||
globalThis.location = riskDom.window.location;
|
||||
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
|
||||
isProductPage: false,
|
||||
looksBlocked: true,
|
||||
hasSecurityChallenge: true,
|
||||
});
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
globalThis.location = previousLocation;
|
||||
}
|
||||
});
|
||||
|
||||
it('does not treat JD login page as a product page', () => {
|
||||
const dom = new JSDOM(`
|
||||
<html>
|
||||
<head><title>京东-欢迎登录</title></head>
|
||||
<body><img src="https://img10.360buyimg.com/img/jfs/login.png" /></body>
|
||||
</html>
|
||||
`, { url: 'https://passport.jd.com/new/login.aspx?ReturnUrl=https%3A%2F%2Fitem.jd.com%2F100291143898.html' });
|
||||
const previousDocument = globalThis.document;
|
||||
const previousLocation = globalThis.location;
|
||||
globalThis.document = dom.window.document;
|
||||
globalThis.location = dom.window.location;
|
||||
|
||||
try {
|
||||
expect(__test__.detectJdPageState('100291143898')).toMatchObject({
|
||||
isProductPage: false,
|
||||
hasProductMarker: false,
|
||||
onExpectedItemUrl: false,
|
||||
looksBlocked: true,
|
||||
isLoginPage: true,
|
||||
});
|
||||
}
|
||||
finally {
|
||||
globalThis.document = previousDocument;
|
||||
globalThis.location = previousLocation;
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -19,7 +19,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 5, help: 'Number of comments' },
|
||||
],
|
||||
columns: ['rank', 'score', 'author', 'text'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const postId = gqlEscape(parsePostId(String(kwargs['url-or-id'])));
|
||||
const limit = Number(kwargs.limit ?? 5);
|
||||
// Fetch post title and comments in parallel
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "curated", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "frontpage", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "new", limit: ${limit}}}) {
|
||||
|
||||
@@ -18,7 +18,7 @@ cli({
|
||||
},
|
||||
],
|
||||
columns: ['title', 'author', 'karma', 'comments', 'tags', 'content', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const postId = parsePostId(String(kwargs['url-or-id']));
|
||||
const query = `query PostsSingle {
|
||||
post(input: {selector: {documentId: "${gqlEscape(postId)}"}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query Sequences {
|
||||
sequences(input: {terms: {view: "communitySequences", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "shortform", limit: ${limit}}}) {
|
||||
|
||||
@@ -19,7 +19,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of results' },
|
||||
],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const tagInput = String(kwargs.tag);
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const tag = await resolveTagId(tagInput);
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 20, help: 'Number of results' }],
|
||||
columns: ['rank', 'name', 'posts'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 20);
|
||||
const query = `query Tags {
|
||||
tags(input: {terms: {view: "coreTags", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "top", after: "${daysAgo(30)}", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "top", after: "${daysAgo(7)}", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "top", after: "${daysAgo(365)}", limit: ${limit}}}) {
|
||||
|
||||
@@ -9,7 +9,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'limit', type: 'int', default: 10, help: 'Number of results' }],
|
||||
columns: ['rank', 'title', 'author', 'karma', 'comments', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const query = `query PostsList {
|
||||
posts(input: {terms: {view: "top", limit: ${limit}}}) {
|
||||
|
||||
@@ -18,7 +18,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of results' },
|
||||
],
|
||||
columns: ['rank', 'title', 'karma', 'comments', 'date', 'url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const username = String(kwargs.username);
|
||||
const limit = Number(kwargs.limit ?? 10);
|
||||
const user = await resolveUserId(username);
|
||||
|
||||
@@ -18,7 +18,7 @@ cli({
|
||||
},
|
||||
],
|
||||
columns: ['field', 'value'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const slug = gqlEscape(String(kwargs.username).toLowerCase());
|
||||
const query = `query UserProfile {
|
||||
user(input: {selector: {slug: "${slug}"}}) {
|
||||
|
||||
@@ -38,7 +38,7 @@ describe('paperreview submit command', () => {
|
||||
resolvedPath: '/tmp/paper.pdf',
|
||||
sizeBytes: 4096,
|
||||
});
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
pdf: './paper.pdf',
|
||||
email: 'wang2629651228@gmail.com',
|
||||
venue: 'RAL',
|
||||
@@ -80,7 +80,7 @@ describe('paperreview submit command', () => {
|
||||
message: 'Submission accepted',
|
||||
},
|
||||
});
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
pdf: './paper.pdf',
|
||||
email: 'wang2629651228@gmail.com',
|
||||
venue: 'RAL',
|
||||
@@ -112,7 +112,7 @@ describe('paperreview submit command', () => {
|
||||
s3_key: 'uploads/paper.pdf',
|
||||
},
|
||||
});
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
pdf: './paper.pdf',
|
||||
email: 'wang2629651228@gmail.com',
|
||||
venue: 'RAL',
|
||||
@@ -153,7 +153,7 @@ describe('paperreview submit command', () => {
|
||||
message: 'Submission accepted',
|
||||
},
|
||||
});
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
pdf: './paper.pdf',
|
||||
email: 'wang2629651228@gmail.com',
|
||||
venue: 'RAL',
|
||||
@@ -190,7 +190,7 @@ describe('paperreview review command', () => {
|
||||
response: { status: 202 },
|
||||
payload: { detail: 'Review is still processing.' },
|
||||
});
|
||||
const result = await cmd.func({ token: 'tok_123' });
|
||||
const result = await cmd.func(null, { token: 'tok_123' });
|
||||
expect(result).toMatchObject({
|
||||
status: 'processing',
|
||||
token: 'tok_123',
|
||||
@@ -214,7 +214,7 @@ describe('paperreview feedback command', () => {
|
||||
response: { ok: true, status: 200 },
|
||||
payload: { message: 'Thanks for the feedback.' },
|
||||
});
|
||||
const result = await cmd.func({
|
||||
const result = await cmd.func(null, {
|
||||
token: 'tok_123',
|
||||
helpfulness: 4,
|
||||
'critical-error': 'yes',
|
||||
|
||||
@@ -17,7 +17,7 @@ cli({
|
||||
{ name: 'additional-comments', help: 'Optional free-text feedback' },
|
||||
],
|
||||
columns: ['status', 'token', 'helpfulness', 'critical_error', 'actionable_suggestions', 'message'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const token = String(kwargs.token ?? '').trim();
|
||||
if (!token) {
|
||||
throw new CliError('ARGUMENT', 'A review token is required.');
|
||||
|
||||
@@ -13,7 +13,7 @@ cli({
|
||||
{ name: 'token', positional: true, required: true, help: 'Review token returned by paperreview.ai' },
|
||||
],
|
||||
columns: ['status', 'title', 'venue', 'numerical_score', 'has_feedback', 'review_url'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const token = String(kwargs.token ?? '').trim();
|
||||
if (!token) {
|
||||
throw new CliError('ARGUMENT', 'A review token is required.');
|
||||
|
||||
@@ -24,7 +24,7 @@ cli({
|
||||
return 'prepared only';
|
||||
return undefined;
|
||||
},
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const pdfFile = await readPdfFile(kwargs.pdf);
|
||||
const email = String(kwargs.email ?? '').trim();
|
||||
const venue = normalizeVenue(kwargs.venue);
|
||||
|
||||
@@ -19,7 +19,7 @@ cli({
|
||||
},
|
||||
],
|
||||
columns: ['rank', 'name', 'tagline', 'author', 'date', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const count = Math.min(Number(args.limit) || 20, 50);
|
||||
const category = String(args.category ?? '').trim() || undefined;
|
||||
const posts = await fetchFeed(category);
|
||||
|
||||
@@ -16,7 +16,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: 'Max results' },
|
||||
],
|
||||
columns: ['rank', 'name', 'tagline', 'author', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const count = Math.min(Number(args.limit) || 20, 50);
|
||||
const posts = await fetchFeed();
|
||||
if (posts.length === 0)
|
||||
|
||||
@@ -47,5 +47,5 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回的文章数量' },
|
||||
],
|
||||
columns: ['rank', 'title', 'author', 'date', 'description', 'url'],
|
||||
func: async (args) => searchSinaBlog(args.keyword, Math.max(1, Math.min(Number(args.limit) || 20, 50))),
|
||||
func: async (_page, args) => searchSinaBlog(args.keyword, Math.max(1, Math.min(Number(args.limit) || 20, 50))),
|
||||
});
|
||||
|
||||
@@ -34,7 +34,7 @@ cli({
|
||||
{ name: 'type', type: 'int', default: 0, help: 'News type: 0=全部 1=A股 2=宏观 3=公司 4=数据 5=市场 6=国际 7=观点 8=央行 9=其它' },
|
||||
],
|
||||
columns: ['id', 'time', 'content', 'views'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit), 50));
|
||||
const apiTag = TYPE_MAP[args.type] ?? 0;
|
||||
const params = new URLSearchParams({
|
||||
|
||||
@@ -62,7 +62,7 @@ cli({
|
||||
{ name: 'market', type: 'string', default: 'auto', help: 'Market: cn, hk, us, auto (default: auto searches cn → hk → us)' },
|
||||
],
|
||||
columns: ['Symbol', 'Name', 'Price', 'Change', 'ChangePercent', 'Open', 'High', 'Low', 'Volume', 'MarketCap'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const key = String(args.key);
|
||||
const market = String(args.market);
|
||||
const marketMap = {
|
||||
|
||||
@@ -28,7 +28,7 @@ describe('sinafinance stock command', () => {
|
||||
.mockResolvedValueOnce(textResponse('var hq_str_gb_AAPL="Apple Inc,189.98,1.23,0,1.56,0,188.50,180.00,195.00,175.00,1200000,0,3000000000000";'));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const result = await cmd.func({ key: 'AAPL', market: 'auto' });
|
||||
const result = await cmd.func(null, { key: 'AAPL', market: 'auto' });
|
||||
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(1, 'https://suggest3.sinajs.cn/suggest/type=11,31,41&key=AAPL', expect.any(Object));
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(2, 'https://hq.sinajs.cn/list=gb_AAPL', expect.any(Object));
|
||||
@@ -48,7 +48,7 @@ describe('sinafinance stock command', () => {
|
||||
.mockResolvedValueOnce(textResponse('var hq_str_gb_AAPL="苹果公司,189.98,1.23,0,1.56,0,188.50,180.00,195.00,175.00,1200000,0,3000000000000";'));
|
||||
vi.stubGlobal('fetch', fetchMock);
|
||||
|
||||
const result = await cmd.func({ key: '苹果', market: 'auto' });
|
||||
const result = await cmd.func(null, { key: '苹果', market: 'auto' });
|
||||
|
||||
expect(fetchMock).toHaveBeenNthCalledWith(2, 'https://hq.sinajs.cn/list=gb_AAPL', expect.any(Object));
|
||||
expect(result[0]).toMatchObject({
|
||||
|
||||
@@ -198,7 +198,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'query', type: 'str', default: '', positional: true, help: 'Track or artist to play (optional)' }],
|
||||
columns: ['track', 'artist', 'status'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
if (kwargs.query) {
|
||||
const { uri, name, artist } = await findTrackUri(kwargs.query);
|
||||
await api('PUT', '/me/player/play', { uris: [uri] });
|
||||
@@ -246,7 +246,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'level', type: 'int', default: 50, positional: true, required: true, help: 'Volume 0–100' }],
|
||||
columns: ['volume'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const level = Math.round(kwargs.level);
|
||||
if (level < 0 || level > 100)
|
||||
throw new CliError('INVALID_ARGS', 'Volume must be between 0 and 100');
|
||||
@@ -265,7 +265,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 10, help: 'Number of results (default: 10)' },
|
||||
],
|
||||
columns: ['track', 'artist', 'album', 'uri'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const limit = Math.min(50, Math.max(1, Math.round(kwargs.limit)));
|
||||
const data = await api('GET', `/search?q=${encodeURIComponent(kwargs.query)}&type=track&limit=${limit}`);
|
||||
const results = mapSpotifyTrackResults(data);
|
||||
@@ -282,7 +282,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'query', type: 'str', required: true, positional: true, help: 'Track to add to queue' }],
|
||||
columns: ['track', 'artist', 'status'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
const { uri, name, artist } = await findTrackUri(kwargs.query);
|
||||
await api('POST', `/me/player/queue?uri=${encodeURIComponent(uri)}`);
|
||||
return [{ track: name, artist, status: 'added to queue' }];
|
||||
@@ -296,7 +296,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'state', type: 'str', default: 'on', positional: true, choices: ['on', 'off'], help: 'on or off' }],
|
||||
columns: ['shuffle'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
await api('PUT', `/me/player/shuffle?state=${kwargs.state === 'on'}`);
|
||||
return [{ shuffle: kwargs.state }];
|
||||
},
|
||||
@@ -309,7 +309,7 @@ cli({
|
||||
browser: false,
|
||||
args: [{ name: 'mode', type: 'str', default: 'context', positional: true, choices: ['off', 'track', 'context'], help: 'off / track / context' }],
|
||||
columns: ['repeat'],
|
||||
func: async (kwargs) => {
|
||||
func: async (_page, kwargs) => {
|
||||
await api('PUT', `/me/player/repeat?state=${kwargs.mode}`);
|
||||
return [{ repeat: kwargs.mode }];
|
||||
},
|
||||
|
||||
@@ -69,7 +69,7 @@ cli({
|
||||
{ name: 'limit', type: 'int', default: 20, help: '返回结果数量' },
|
||||
],
|
||||
columns: ['rank', 'title', 'author', 'date', 'description', 'url'],
|
||||
func: async (args) => {
|
||||
func: async (_page, args) => {
|
||||
const limit = Math.max(1, Math.min(Number(args.limit) || 20, 50));
|
||||
return args.type === 'publications'
|
||||
? searchPublications(args.keyword, limit)
|
||||
|
||||
+74
-225
@@ -1,142 +1,11 @@
|
||||
import { AuthRequiredError, SelectorError } from '@jackwener/opencli/errors';
|
||||
import { cli, Strategy } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { resolveTwitterQueryId, sanitizeQueryId } from './shared.js';
|
||||
|
||||
const BEARER_TOKEN = 'AAAAAAAAAAAAAAAAAAAAANRILgAAAAAAnNwIzUejRCOuH5E6I8xnZz4puTs%3D1Zv7ttfk8LF81IUq16cHjhLTvJu4FA33AGWWjCpTnA';
|
||||
const FOLLOWING_QUERY_ID = 'zx6e-TLzRkeDO_a7p4b3JQ'; // Following fallback
|
||||
const USER_BY_SCREEN_NAME_QUERY_ID = 'qRednkZG-rn1P6b48NINmQ';
|
||||
|
||||
const FEATURES = {
|
||||
rweb_video_screen_enabled: false,
|
||||
profile_label_improvements_pcf_label_in_post_enabled: true,
|
||||
responsive_web_profile_redirect_enabled: false,
|
||||
rweb_tipjar_consumption_enabled: false,
|
||||
verified_phone_label_enabled: false,
|
||||
creator_subscriptions_tweet_preview_api_enabled: true,
|
||||
responsive_web_graphql_timeline_navigation_enabled: true,
|
||||
responsive_web_graphql_skip_user_profile_image_extensions_enabled: false,
|
||||
premium_content_api_read_enabled: false,
|
||||
communities_web_enable_tweet_community_results_fetch: true,
|
||||
c9s_tweet_anatomy_moderator_badge_enabled: true,
|
||||
responsive_web_grok_analyze_button_fetch_trends_enabled: false,
|
||||
responsive_web_grok_analyze_post_followups_enabled: true,
|
||||
responsive_web_jetfuel_frame: true,
|
||||
responsive_web_grok_share_attachment_enabled: true,
|
||||
responsive_web_grok_annotations_enabled: true,
|
||||
articles_preview_enabled: true,
|
||||
responsive_web_edit_tweet_api_enabled: true,
|
||||
graphql_is_translatable_rweb_tweet_is_translatable_enabled: true,
|
||||
view_counts_everywhere_api_enabled: true,
|
||||
longform_notetweets_consumption_enabled: true,
|
||||
responsive_web_twitter_article_tweet_consumption_enabled: true,
|
||||
tweet_awards_web_tipping_enabled: false,
|
||||
content_disclosure_indicator_enabled: true,
|
||||
content_disclosure_ai_generated_indicator_enabled: true,
|
||||
responsive_web_grok_show_grok_translated_post: false,
|
||||
responsive_web_grok_analysis_button_from_backend: true,
|
||||
post_ctas_fetch_enabled: false,
|
||||
freedom_of_speech_not_reach_fetch_enabled: true,
|
||||
standardized_nudges_misinfo: true,
|
||||
tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled: true,
|
||||
longform_notetweets_rich_text_read_enabled: true,
|
||||
longform_notetweets_inline_media_enabled: false,
|
||||
responsive_web_grok_image_annotation_enabled: true,
|
||||
responsive_web_grok_imagine_annotation_enabled: true,
|
||||
responsive_web_grok_community_note_auto_translation_is_enabled: false,
|
||||
responsive_web_enhance_cards_enabled: false,
|
||||
};
|
||||
|
||||
function buildFollowingUrl(queryId, userId, count, cursor) {
|
||||
const vars = {
|
||||
userId,
|
||||
count,
|
||||
includePromotedContent: false,
|
||||
withClientEventToken: false,
|
||||
withBirdwatchNotes: false,
|
||||
withVoice: true,
|
||||
withV2Timeline: true,
|
||||
};
|
||||
if (cursor)
|
||||
vars.cursor = cursor;
|
||||
return `/i/api/graphql/${queryId}/Following`
|
||||
+ `?variables=${encodeURIComponent(JSON.stringify(vars))}`
|
||||
+ `&features=${encodeURIComponent(JSON.stringify(FEATURES))}`;
|
||||
}
|
||||
|
||||
function buildUserByScreenNameUrl(queryId, screenName) {
|
||||
const vars = JSON.stringify({ screen_name: screenName, withSafetyModeUserFields: true });
|
||||
const feats = JSON.stringify({
|
||||
hidden_profile_subscriptions_enabled: true,
|
||||
rweb_tipjar_consumption_enabled: true,
|
||||
responsive_web_graphql_exclude_directive_enabled: true,
|
||||
verified_phone_label_enabled: false,
|
||||
subscriptions_verification_info_is_identity_verified_enabled: true,
|
||||
subscriptions_verification_info_verified_since_enabled: true,
|
||||
highlights_tweets_tab_ui_enabled: true,
|
||||
responsive_web_twitter_article_notes_tab_enabled: true,
|
||||
subscriptions_feature_can_gift_premium: true,
|
||||
creator_subscriptions_tweet_preview_api_enabled: true,
|
||||
responsive_web_graphql_skip_user_profile_image_extensions_enabled: false,
|
||||
responsive_web_graphql_timeline_navigation_enabled: true,
|
||||
});
|
||||
return `/i/api/graphql/${queryId}/UserByScreenName`
|
||||
+ `?variables=${encodeURIComponent(vars)}`
|
||||
+ `&features=${encodeURIComponent(feats)}`;
|
||||
}
|
||||
|
||||
function extractUser(result) {
|
||||
if (!result || result.__typename !== 'User')
|
||||
return null;
|
||||
const core = result.core || {};
|
||||
const legacy = result.legacy || {};
|
||||
return {
|
||||
screen_name: core.screen_name || legacy.screen_name || 'unknown',
|
||||
name: core.name || legacy.name || 'unknown',
|
||||
bio: legacy.description || result.profile_bio?.description || '',
|
||||
followers: legacy.followers_count || legacy.normal_followers_count || 0,
|
||||
};
|
||||
}
|
||||
|
||||
function parseFollowing(data) {
|
||||
const users = [];
|
||||
let nextCursor = null;
|
||||
const instructions = data?.data?.user?.result?.timeline_v2?.timeline?.instructions
|
||||
|| data?.data?.user?.result?.timeline?.timeline?.instructions
|
||||
|| [];
|
||||
for (const inst of instructions) {
|
||||
for (const entry of inst.entries || []) {
|
||||
const content = entry.content;
|
||||
// Extract cursor
|
||||
if (content?.entryType === 'TimelineTimelineCursor' || content?.__typename === 'TimelineTimelineCursor') {
|
||||
if (content.cursorType === 'Bottom' || content.cursorType === 'ShowMore')
|
||||
nextCursor = content.value;
|
||||
continue;
|
||||
}
|
||||
if (entry.entryId?.startsWith('cursor-bottom-') || entry.entryId?.startsWith('cursor-showMore-')) {
|
||||
nextCursor = content?.value || content?.itemContent?.value || nextCursor;
|
||||
continue;
|
||||
}
|
||||
// Extract user
|
||||
if (entry.entryId?.startsWith('user-')) {
|
||||
const user = extractUser(content?.itemContent?.user_results?.result);
|
||||
if (user)
|
||||
users.push(user);
|
||||
}
|
||||
}
|
||||
}
|
||||
return { users, nextCursor };
|
||||
}
|
||||
|
||||
function normalizeScreenName(value) {
|
||||
return String(value || '').trim().replace(/^\/+/, '').replace(/^@+/, '');
|
||||
}
|
||||
|
||||
cli({
|
||||
site: 'twitter',
|
||||
name: 'following',
|
||||
description: 'Get accounts a Twitter/X user is following',
|
||||
domain: 'x.com',
|
||||
strategy: Strategy.COOKIE,
|
||||
strategy: Strategy.INTERCEPT,
|
||||
browser: true,
|
||||
args: [
|
||||
{ name: 'user', positional: true, type: 'string', required: false },
|
||||
@@ -144,103 +13,83 @@ cli({
|
||||
],
|
||||
columns: ['screen_name', 'name', 'bio', 'followers'],
|
||||
func: async (page, kwargs) => {
|
||||
const limit = kwargs.limit === undefined || kwargs.limit === null ? 50 : Number(kwargs.limit);
|
||||
if (!Number.isInteger(limit) || limit <= 0) {
|
||||
throw new ArgumentError('twitter following --limit must be a positive integer', 'Example: opencli twitter following @elonmusk --limit 200');
|
||||
}
|
||||
let targetUser = normalizeScreenName(kwargs.user);
|
||||
|
||||
await page.goto('https://x.com');
|
||||
await page.wait(3);
|
||||
|
||||
const ct0 = await page.evaluate(`() => {
|
||||
return document.cookie.split(';').map(c => c.trim()).find(c => c.startsWith('ct0='))?.split('=')[1] || null;
|
||||
}`);
|
||||
if (!ct0)
|
||||
throw new AuthRequiredError('x.com', 'Not logged into x.com (no ct0 cookie)');
|
||||
|
||||
let targetUser = kwargs.user;
|
||||
// If no user is specified, figure out the logged-in user's handle
|
||||
if (!targetUser) {
|
||||
await page.goto('https://x.com/home');
|
||||
await page.wait({ selector: '[data-testid="primaryColumn"]' });
|
||||
const href = await page.evaluate(`() => {
|
||||
const link = document.querySelector('a[data-testid="AppTabBar_Profile_Link"]');
|
||||
return link ? link.getAttribute('href') : null;
|
||||
}`);
|
||||
if (!href)
|
||||
throw new AuthRequiredError('x.com', 'Could not detect logged-in user. Are you logged in?');
|
||||
targetUser = normalizeScreenName(href.replace('/', ''));
|
||||
}
|
||||
if (!targetUser) {
|
||||
throw new ArgumentError('twitter following user cannot be empty', 'Example: opencli twitter following @elonmusk --limit 200');
|
||||
}
|
||||
|
||||
const followingQueryId = await resolveTwitterQueryId(page, 'Following', FOLLOWING_QUERY_ID);
|
||||
const userByScreenNameQueryId = await resolveTwitterQueryId(page, 'UserByScreenName', USER_BY_SCREEN_NAME_QUERY_ID);
|
||||
const headers = JSON.stringify({
|
||||
'Authorization': `Bearer ${decodeURIComponent(BEARER_TOKEN)}`,
|
||||
'X-Csrf-Token': ct0,
|
||||
'X-Twitter-Auth-Type': 'OAuth2Session',
|
||||
'X-Twitter-Active-User': 'yes',
|
||||
});
|
||||
|
||||
// Get userId from screen_name
|
||||
const userLookup = await page.evaluate(`async () => {
|
||||
const url = ${JSON.stringify(buildUserByScreenNameUrl(userByScreenNameQueryId, targetUser))};
|
||||
const resp = await fetch(url, { headers: ${headers}, credentials: 'include' });
|
||||
if (!resp.ok) return { error: resp.status };
|
||||
const d = await resp.json();
|
||||
return { userId: d.data?.user?.result?.rest_id || null };
|
||||
const link = document.querySelector('a[data-testid="AppTabBar_Profile_Link"]');
|
||||
return link ? link.getAttribute('href') : null;
|
||||
}`);
|
||||
if (userLookup?.error === 401 || userLookup?.error === 403) {
|
||||
throw new AuthRequiredError('x.com', `Twitter user lookup failed (HTTP ${userLookup.error})`);
|
||||
}
|
||||
if (userLookup?.error) {
|
||||
throw new CommandExecutionError(`HTTP ${userLookup.error}: Failed to resolve Twitter user @${targetUser}`);
|
||||
}
|
||||
const userId = userLookup?.userId || null;
|
||||
if (!userId)
|
||||
throw new CommandExecutionError(`Could not find user @${targetUser}`);
|
||||
|
||||
const allUsers = [];
|
||||
const seen = new Set();
|
||||
let cursor = null;
|
||||
|
||||
const maxPages = Math.ceil(limit / 50) + 2;
|
||||
for (let i = 0; i < maxPages && allUsers.length < limit; i++) {
|
||||
const fetchCount = Math.min(50, limit - allUsers.length + 10);
|
||||
const apiUrl = buildFollowingUrl(followingQueryId, userId, fetchCount, cursor);
|
||||
const data = await page.evaluate(`async () => {
|
||||
const r = await fetch("${apiUrl}", { headers: ${headers}, credentials: 'include' });
|
||||
return r.ok ? await r.json() : { error: r.status };
|
||||
}`);
|
||||
if (data?.error) {
|
||||
if (data.error === 401 || data.error === 403)
|
||||
throw new AuthRequiredError('x.com', `Twitter following request failed (HTTP ${data.error})`);
|
||||
throw new CommandExecutionError(`HTTP ${data.error}: Failed to fetch following list. queryId may have expired.`);
|
||||
if (!href) {
|
||||
throw new AuthRequiredError('x.com', 'Could not find logged-in user profile link. Are you logged in?');
|
||||
}
|
||||
const { users, nextCursor } = parseFollowing(data);
|
||||
for (const u of users) {
|
||||
if (!seen.has(u.screen_name)) {
|
||||
seen.add(u.screen_name);
|
||||
allUsers.push(u);
|
||||
targetUser = href.replace('/', '');
|
||||
}
|
||||
// 1. Navigate to profile page
|
||||
await page.goto(`https://x.com/${targetUser}`);
|
||||
await page.wait(3);
|
||||
// 2. Install interceptor BEFORE SPA navigation.
|
||||
// goto() resets JS context, but SPA click preserves it.
|
||||
await page.installInterceptor('Following');
|
||||
// 3. Click the following link via SPA navigation (preserves interceptor)
|
||||
const safeUser = JSON.stringify(targetUser);
|
||||
const clicked = await page.evaluate(`() => {
|
||||
const target = ${safeUser};
|
||||
const link = document.querySelector('a[href="/' + target + '/following"]');
|
||||
if (link) { link.click(); return true; }
|
||||
return false;
|
||||
}`);
|
||||
if (!clicked) {
|
||||
throw new SelectorError('Twitter following link', 'Twitter may have changed the layout.');
|
||||
}
|
||||
await page.waitForCapture(5);
|
||||
// 4. Scroll to trigger pagination API calls
|
||||
await page.autoScroll({ times: Math.ceil(kwargs.limit / 20), delayMs: 2000 });
|
||||
// 5. Retrieve intercepted data
|
||||
const requests = await page.getInterceptedRequests();
|
||||
const requestList = Array.isArray(requests) ? requests : [];
|
||||
if (requestList.length === 0) {
|
||||
return [];
|
||||
}
|
||||
let results = [];
|
||||
for (const req of requestList) {
|
||||
try {
|
||||
// GraphQL response: { data: { user: { result: { timeline: ... } } } }
|
||||
let instructions = req.data?.user?.result?.timeline?.timeline?.instructions;
|
||||
if (!instructions)
|
||||
continue;
|
||||
let addEntries = instructions.find((i) => i.type === 'TimelineAddEntries');
|
||||
if (!addEntries) {
|
||||
addEntries = instructions.find((i) => i.entries && Array.isArray(i.entries));
|
||||
}
|
||||
if (!addEntries)
|
||||
continue;
|
||||
for (const entry of addEntries.entries) {
|
||||
if (!entry.entryId.startsWith('user-'))
|
||||
continue;
|
||||
const item = entry.content?.itemContent?.user_results?.result;
|
||||
if (!item || item.__typename !== 'User')
|
||||
continue;
|
||||
const core = item.core || {};
|
||||
const legacy = item.legacy || {};
|
||||
results.push({
|
||||
screen_name: core.screen_name || legacy.screen_name || 'unknown',
|
||||
name: core.name || legacy.name || 'unknown',
|
||||
bio: legacy.description || item.profile_bio?.description || '',
|
||||
followers: legacy.followers_count || legacy.normal_followers_count || 0
|
||||
});
|
||||
}
|
||||
}
|
||||
if (!nextCursor || nextCursor === cursor)
|
||||
break;
|
||||
cursor = nextCursor;
|
||||
catch (e) {
|
||||
// ignore parsing errors for individual payloads
|
||||
}
|
||||
}
|
||||
|
||||
if (allUsers.length === 0) {
|
||||
throw new EmptyResultError('twitter following', `No following accounts found for @${targetUser}`);
|
||||
}
|
||||
|
||||
return allUsers.slice(0, limit);
|
||||
},
|
||||
// Deduplicate by screen_name
|
||||
const unique = new Map();
|
||||
results.forEach(r => unique.set(r.screen_name, r));
|
||||
const deduplicatedResults = Array.from(unique.values());
|
||||
return deduplicatedResults.slice(0, kwargs.limit);
|
||||
}
|
||||
});
|
||||
|
||||
export const __test__ = {
|
||||
sanitizeQueryId,
|
||||
buildFollowingUrl,
|
||||
buildUserByScreenNameUrl,
|
||||
extractUser,
|
||||
normalizeScreenName,
|
||||
parseFollowing,
|
||||
};
|
||||
|
||||
@@ -1,277 +0,0 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { getRegistry } from '@jackwener/opencli/registry';
|
||||
import { ArgumentError, AuthRequiredError, CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
|
||||
import { __test__ } from './following.js';
|
||||
|
||||
describe('twitter following helpers', () => {
|
||||
it('falls back when queryId contains unsafe characters', () => {
|
||||
expect(__test__.sanitizeQueryId('safe_Query-123', 'fallback')).toBe('safe_Query-123');
|
||||
expect(__test__.sanitizeQueryId('bad"id', 'fallback')).toBe('fallback');
|
||||
expect(__test__.sanitizeQueryId('bad/id', 'fallback')).toBe('fallback');
|
||||
expect(__test__.sanitizeQueryId(null, 'fallback')).toBe('fallback');
|
||||
});
|
||||
|
||||
it('builds following url with cursor', () => {
|
||||
const url = __test__.buildFollowingUrl('query123', '42', 20, 'cursor-1');
|
||||
expect(url).toContain('/i/api/graphql/query123/Following');
|
||||
expect(decodeURIComponent(url)).toContain('"userId":"42"');
|
||||
expect(decodeURIComponent(url)).toContain('"count":20');
|
||||
expect(decodeURIComponent(url)).toContain('"cursor":"cursor-1"');
|
||||
});
|
||||
|
||||
it('builds following url without cursor', () => {
|
||||
const url = __test__.buildFollowingUrl('query123', '42', 20);
|
||||
expect(url).toContain('/i/api/graphql/query123/Following');
|
||||
expect(decodeURIComponent(url)).not.toContain('"cursor"');
|
||||
});
|
||||
|
||||
it('extracts user from result', () => {
|
||||
const user = __test__.extractUser({
|
||||
__typename: 'User',
|
||||
core: { screen_name: 'alice', name: 'Alice' },
|
||||
legacy: { description: 'bio text', followers_count: 100 },
|
||||
});
|
||||
expect(user).toMatchObject({
|
||||
screen_name: 'alice',
|
||||
name: 'Alice',
|
||||
bio: 'bio text',
|
||||
followers: 100,
|
||||
});
|
||||
});
|
||||
|
||||
it('returns null for non-User typename', () => {
|
||||
expect(__test__.extractUser({ __typename: 'Tweet' })).toBeNull();
|
||||
expect(__test__.extractUser(null)).toBeNull();
|
||||
expect(__test__.extractUser(undefined)).toBeNull();
|
||||
});
|
||||
|
||||
it('falls back to legacy screen_name if core is missing', () => {
|
||||
const user = __test__.extractUser({
|
||||
__typename: 'User',
|
||||
legacy: { screen_name: 'bob', name: 'Bob', description: '', followers_count: 0 },
|
||||
});
|
||||
expect(user?.screen_name).toBe('bob');
|
||||
});
|
||||
|
||||
it('parses following timeline with users and cursor', () => {
|
||||
const payload = {
|
||||
data: {
|
||||
user: {
|
||||
result: {
|
||||
timeline_v2: {
|
||||
timeline: {
|
||||
instructions: [{
|
||||
entries: [
|
||||
{
|
||||
entryId: 'user-1',
|
||||
content: {
|
||||
itemContent: {
|
||||
user_results: {
|
||||
result: {
|
||||
__typename: 'User',
|
||||
core: { screen_name: 'bob', name: 'Bob' },
|
||||
legacy: { description: 'hello', followers_count: 50 },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
entryId: 'user-2',
|
||||
content: {
|
||||
itemContent: {
|
||||
user_results: {
|
||||
result: {
|
||||
__typename: 'User',
|
||||
core: { screen_name: 'carol', name: 'Carol' },
|
||||
legacy: { description: 'world', followers_count: 200 },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
entryId: 'cursor-bottom-1',
|
||||
content: {
|
||||
entryType: 'TimelineTimelineCursor',
|
||||
cursorType: 'Bottom',
|
||||
value: 'next-cursor',
|
||||
},
|
||||
},
|
||||
],
|
||||
}],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = __test__.parseFollowing(payload);
|
||||
expect(result.users).toHaveLength(2);
|
||||
expect(result.users[0]).toMatchObject({ screen_name: 'bob', name: 'Bob', followers: 50 });
|
||||
expect(result.users[1]).toMatchObject({ screen_name: 'carol', name: 'Carol', followers: 200 });
|
||||
expect(result.nextCursor).toBe('next-cursor');
|
||||
});
|
||||
|
||||
it('handles cursor-bottom entryId pattern', () => {
|
||||
const payload = {
|
||||
data: {
|
||||
user: {
|
||||
result: {
|
||||
timeline: {
|
||||
timeline: {
|
||||
instructions: [{
|
||||
entries: [
|
||||
{
|
||||
entryId: 'cursor-bottom-0',
|
||||
content: {
|
||||
itemContent: { value: 'cursor-val' },
|
||||
},
|
||||
},
|
||||
],
|
||||
}],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
const result = __test__.parseFollowing(payload);
|
||||
expect(result.nextCursor).toBe('cursor-val');
|
||||
expect(result.users).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('returns empty users and null cursor for missing instructions', () => {
|
||||
const result = __test__.parseFollowing({ data: { user: { result: {} } } });
|
||||
expect(result.users).toHaveLength(0);
|
||||
expect(result.nextCursor).toBeNull();
|
||||
});
|
||||
|
||||
it('returns empty for completely empty payload', () => {
|
||||
const result = __test__.parseFollowing({});
|
||||
expect(result.users).toHaveLength(0);
|
||||
expect(result.nextCursor).toBeNull();
|
||||
});
|
||||
|
||||
it('normalizes screen names for CLI and profile-link inputs', () => {
|
||||
expect(__test__.normalizeScreenName('@elonmusk')).toBe('elonmusk');
|
||||
expect(__test__.normalizeScreenName('/elonmusk')).toBe('elonmusk');
|
||||
expect(__test__.normalizeScreenName(' @@alice ')).toBe('alice');
|
||||
});
|
||||
});
|
||||
|
||||
function followingPayload(users, cursor) {
|
||||
return {
|
||||
data: {
|
||||
user: {
|
||||
result: {
|
||||
timeline_v2: {
|
||||
timeline: {
|
||||
instructions: [{
|
||||
entries: [
|
||||
...users.map((name) => ({
|
||||
entryId: `user-${name}`,
|
||||
content: {
|
||||
itemContent: {
|
||||
user_results: {
|
||||
result: {
|
||||
__typename: 'User',
|
||||
core: { screen_name: name, name: name.toUpperCase() },
|
||||
legacy: { description: `${name} bio`, followers_count: 10 },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})),
|
||||
...(cursor ? [{
|
||||
entryId: `cursor-bottom-${cursor}`,
|
||||
content: {
|
||||
entryType: 'TimelineTimelineCursor',
|
||||
cursorType: 'Bottom',
|
||||
value: cursor,
|
||||
},
|
||||
}] : []),
|
||||
],
|
||||
}],
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function createFollowingPage(followingResponses, { ct0 = 'token', userLookup = { userId: '42' } } = {}) {
|
||||
const page = {
|
||||
goto: vi.fn().mockResolvedValue(undefined),
|
||||
wait: vi.fn().mockResolvedValue(undefined),
|
||||
evaluate: vi.fn(async (script) => {
|
||||
if (script.includes('document.cookie')) return ct0;
|
||||
if (script.includes('operationName')) return null;
|
||||
if (script.includes('/UserByScreenName')) return userLookup;
|
||||
if (script.includes('/Following')) return followingResponses.shift() || followingPayload([], null);
|
||||
if (script.includes('AppTabBar_Profile_Link')) return '/viewer';
|
||||
throw new Error(`Unexpected evaluate script: ${script.slice(0, 80)}`);
|
||||
}),
|
||||
};
|
||||
return page;
|
||||
}
|
||||
|
||||
describe('twitter following command', () => {
|
||||
it('paginates with cursor, deduplicates users, strips @, and respects limit', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([
|
||||
followingPayload(['alice', 'bob'], 'cursor-1'),
|
||||
followingPayload(['bob', 'carol', 'dave'], null),
|
||||
]);
|
||||
|
||||
const rows = await command.func(page, { user: '@elonmusk', limit: 3 });
|
||||
|
||||
expect(rows.map((row) => row.screen_name)).toEqual(['alice', 'bob', 'carol']);
|
||||
const userLookupScript = page.evaluate.mock.calls.find(([script]) => script.includes('/UserByScreenName'))?.[0] || '';
|
||||
expect(decodeURIComponent(userLookupScript)).toContain('"screen_name":"elonmusk"');
|
||||
expect(decodeURIComponent(userLookupScript)).not.toContain('"screen_name":"@elonmusk"');
|
||||
const followingCalls = page.evaluate.mock.calls.filter(([script]) => script.includes('/Following'));
|
||||
expect(followingCalls).toHaveLength(2);
|
||||
expect(decodeURIComponent(followingCalls[1][0])).toContain('"cursor":"cursor-1"');
|
||||
});
|
||||
|
||||
it('rejects invalid limits before navigating', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([]);
|
||||
|
||||
await expect(command.func(page, { user: 'elonmusk', limit: 0 })).rejects.toBeInstanceOf(ArgumentError);
|
||||
expect(page.goto).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('maps first-page auth failures to AuthRequiredError', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([{ error: 401 }]);
|
||||
|
||||
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('does not silently return partial rows when a later page fails', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([
|
||||
followingPayload(['alice'], 'cursor-1'),
|
||||
{ error: 429 },
|
||||
]);
|
||||
|
||||
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(CommandExecutionError);
|
||||
});
|
||||
|
||||
it('maps user lookup auth failures to AuthRequiredError', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([], { userLookup: { error: 403 } });
|
||||
|
||||
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(AuthRequiredError);
|
||||
});
|
||||
|
||||
it('fails fast when the following timeline is empty', async () => {
|
||||
const command = getRegistry().get('twitter/following');
|
||||
const page = createFollowingPage([followingPayload([], null)]);
|
||||
|
||||
await expect(command.func(page, { user: 'elonmusk', limit: 10 })).rejects.toBeInstanceOf(EmptyResultError);
|
||||
});
|
||||
});
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user