Compare commits

...

212 Commits

Author SHA1 Message Date
jackwener d51ca38cee refactor(cli): move external management under external 2026-05-02 01:35:25 +08:00
jakevin 5c871dd7a3 refactor(adapter): split browser command signatures (#1237) 2026-05-02 01:33:37 +08:00
jakevin b41ee2b671 feat(update-check): show extension update notice on exit (#1236)
* feat(update-check): show extension update notice on exit

The CLI exit hook already prints "Update available" when a newer @jackwener/opencli is on npm. Extension updates were only surfaced inside `opencli doctor`, so users running normal browser commands had no signal that the Chrome extension was out of date.

Solution piggybacks on the existing 24h background fetch:
- Daemon writes the live extensionVersion + lastSeenAt into the shared cache on every hello handshake (rare event, one fs.writeFileSync).
- CLI exit hook reads the cache it already loads and prints an extra extension notice when a newer release is available and the cache is fresh (<7d).
- writeCache becomes a read-merge-write so the daemon's currentExtensionVersion and the CLI's npm latestVersion don't clobber each other.

Net cost on the CLI hot path: zero new I/O, zero new daemon contact. The notice formatter is split into a pure helper (buildUpdateNotices) so the staleness window, equality, and combined-notice cases are unit-tested without touching disk or stderr.

* fix(update-check): tolerate partial cache when daemon writes first

Self-review caught a TypeError path: if the daemon's hello handler runs `recordExtensionVersion` before the CLI's npm fetch ever populated the cache, the resulting cache file has only `currentExtensionVersion` + `extensionLastSeenAt` and no `latestVersion`. The next CLI run then fed `undefined` into `isNewer`, which calls `.replace(...)` on it.

- Mark `lastCheck` and `latestVersion` optional in the cache schema (the merge pattern means either side may write first).
- Guard the CLI notice on `cache.latestVersion` being defined before comparing.
- Guard `checkForUpdateBackground`'s 24h short-circuit on `lastCheck` being defined.
- Add a test for the daemon-only cache case.
2026-05-02 01:03:37 +08:00
lakako 6c077237a8 feat(zhihu) add collection list and list collection content (#1234)
* feat(zhihu): add collection command to list favorite items

Add new 'opencli zhihu collection' command that:
- Lists items from a Zhihu collection (requires login)
- Supports pagination with --offset and --limit parameters
- Handles multiple content types: answer, article, pin
- Shows collection statistics: total count, total pages, current page

* feat(zhihu): split collection into collection and collections commands

- Rename zhihu collection list functionality to zhihu collections
- Keep zhihu collection for viewing specific collection contents by ID
- Convert collection.ts to collection.js so build-manifest picks it up
- Add tests for both commands
- Update cli-manifest.json

* fix(zhihu): harden collection read commands

---------

Co-authored-by: Developer <developer@example.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-02 00:59:36 +08:00
jakevin 8dd9d578d4 feat(browser): support multiple Chrome profiles (#1235)
* feat(browser): support multiple chrome profiles

* fix(browser): tighten profile popup context id

* fix(browser): harden profile routing edge cases

* refactor(browser): remove unnecessary profile id guard
2026-05-02 00:52:44 +08:00
hanzi d65cccd7d8 feat(facebook): add marketplace read commands (#1221)
* feat(facebook): add marketplace read commands

* feat(facebook): add marketplace reply draft command

* fix(facebook): parse narrow spaces in marketplace inbox

* fix(facebook): keep marketplace commands read-only

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-01 20:18:57 +08:00
AstroHan c0aea6c1ae fix(twitter): paginate following results
* fix(twitter/following): switch from INTERCEPT+autoScroll to COOKIE+cursor pagination

The previous INTERCEPT strategy relied on autoScroll to trigger Twitter's
pagination by scrolling document.body. Twitter's virtual list doesn't grow
document.body.scrollHeight, so scrolls stopped triggering API calls after
the first few pages, capping results at ~50 regardless of limit.

Now uses Strategy.COOKIE with explicit cursor-based GraphQL pagination
(same pattern as twitter/likes), which correctly fetches all pages.

Fixes #1230

* fix(twitter): harden following pagination

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-01 20:13:54 +08:00
huanghuoguoguo 349b4bab49 feat(boss): add --jobType filter, fix experience codes, surface bossOnline (#1231)
* feat(boss): add jobType filter and bossOnline output

Add --jobType param (全职/兼职/实习 = 1901/1902/1903) so callers can
exclude internships at the API layer instead of post-filtering by name
keywords. Without this, --experience 应届 returns a mix of 校招 and 实习
because BOSS bundles both under code 108.

Also surface bossOnline (Y/empty) in results so callers can prioritize
HRs currently online — this is the only activity signal exposed by the
web API; 'recently active' / 'newly posted' filters are mobile-only and
not accepted by /wapi/zpgeek/search/joblist.json.

* fix(boss): correct experience codes (应届=102, not 108)

The previous EXP_MAP was off by ~2 across the board. Verified each
code by clicking BOSS web's filter UI and reading the URL:

  108 = 在校生 (interns)         was: '在校/应届','应届' → 108 (wrong)
  102 = 应届生 (校招 full-time)   was: '1-3年' → 102      (wrong)
  101 = 经验不限                  was: '1年以内' → 101    (wrong)
  103 = 1年以内                   was missing
  104 = 1-3年                     was: '3-5年' → 103      (wrong)
  105 = 3-5年                     was: '5-10年' → 104     (wrong)
  106 = 5-10年                    was: '10年以上' → 105   (wrong)
  107 = 10年以上                  was missing

This is why --experience 应届 had been returning mostly 实习生 jobs:
it was secretly querying 在校生 (108). The fix makes 应届 actually
mean 应届生 (102 = 校招), and lets users pick 在校生 (108) explicitly
when they do want internships.

* fix(boss): validate job type filter

* fix(boss): keep legacy campus experience alias

---------

Co-authored-by: youhh <youhh@1051233107@qq.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-01 20:10:52 +08:00
Benjamin Liu 094ff0da80 feat(deepseek): add vision mode support
* feat(deepseek): add vision mode support

DeepSeek added a third model "识图模式" (Vision Mode) that accepts
image uploads for visual understanding. Add vision to the --model
choices, update selectModel to use explicit index mapping for all
three models, skip the search toggle in vision mode (not available),
and extend waitForFilePreview to detect image thumbnails via send
button state since vision mode shows a preview image instead of a
filename label.

Also catch "Not allowed" errors from setFileInput (Cloudflare may
block CDP file operations) so the DataTransfer fallback can run.

Closes #1215

* fix(deepseek): harden vision upload mode

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-01 20:03:01 +08:00
Benjamin Liu 25e86532a3 fix(chatgpt): fix image generation detection and output path (#1218)
* fix(chatgpt): fix image generation detection and output path

Three fixes for chatgpt image command:

1. Page navigation: ChatGPT redirects away from the conversation
   after sending. Poll for the /c/ URL after send, then periodically
   reload the conversation page during image wait to pick up
   asynchronously rendered images.

2. Composer selector: add fallback selectors for the chat input
   since ChatGPT uses different aria-labels across UI versions.

3. Output path: the default '~/Pictures/chatgpt' was passed as a
   literal string without tilde expansion, creating a directory
   named '~' in the working directory. Removed the string default
   and use os.homedir() fallback instead.

Fixes #1206

* fix(chatgpt): fail fast on image export failures

* fix(chatgpt): avoid reloads during image generation

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-05-01 20:01:34 +08:00
m72900024 8dee08bc4c fix(chatgpt-app): support Traditional Chinese UI labels
* fix(chatgpt-app): support Traditional Chinese UI labels

The send button and Options button matchers only included Simplified
Chinese ("发送", "选项"). On macOS systems with Traditional Chinese as
the system language, the ChatGPT desktop app exposes "傳送" and "選項"
via the Accessibility API, causing `chatgpt-app send` to fail with
"Could not find send button" and `chatgpt-app model` to fail with
"Could not find Options button" for zh-TW / zh-HK users.

Verified via AXUIElement walk on ChatGPT 1.2026.104 / macOS 26 with
system language set to Traditional Chinese.

The "Stop generating" detection at line 314 already handles Traditional
Chinese because 停止生成 uses identical glyphs in both writing systems.
"Legacy models" at line 261 still lacks any Chinese variant but is not
addressed here since the Traditional Chinese translation has not been
verified on a live UI.

* test(chatgpt-app): cover traditional chinese ax labels

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-30 12:20:39 +08:00
Benjamin Liu c86b6826a4 fix(zhihu): fix identity detection, comment, answer, and search (#1207)
* fix(zhihu): fix identity detection, comment, answer, and search

Identity detection: Zhihu removed __INITIAL_STATE__ and moved the
user avatar from a profile link into a button. Added fallback that
extracts the user slug from the header avatar alt text.

Comment and answer: Zhihu moved the comment editor into a Modal
and changed the submit button behavior, breaking the UI-based
write flow. Replaced with direct API calls (POST /api/v4/answers/
{id}/comments and POST /api/v4/questions/{id}/answers) which are
reliable and much simpler.

Search: Zhihu's search API now returns mixed result types (ads,
education, hot_timing) alongside search_result. Updated the filter
to select by object.type (answer/article/question) and increased
fetch size to compensate for non-content results.

Fixes #1198

* fix(zhihu): rewrite like, follow, favorite to use API

Same DOM breakage as comment/answer. Replaced UI-based click
flows with direct Zhihu API calls for all write commands.

* fix(zhihu): harden api write regressions

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-29 15:01:33 +08:00
Jean Zhang c264531586 feat(zlibrary): add search and info commands (#1211)
* feat(zlibrary): add search and info commands

Add Z-Library adapter with two browser-based commands:

- `search` — Search books by title, author, or ISBN.
  Navigates to /s/<query> and extracts results from
  <z-bookcard> shadow DOM custom elements.

- `info` — Get book details and available download formats
  from a book page URL.

Uses Strategy.COOKIE with browser automation to bypass
Cloudflare protection. The adapter reuses the user's existing
Z-Library login cookies from system Chrome.

Known limitation: actual file downloading requires Playwright's
download event handling (page.on('download')). OpenCLI's browser
automation does not currently intercept file downloads. Users
needing to download files should use Playwright to navigate to
the book URLs discovered by this adapter.

* fix(zlibrary): harden input and empty extraction

---------

Co-authored-by: jean <jean@jeandeMacBook-Pro.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-29 14:48:38 +08:00
jakevin 10baf02060 feat(web): make read render-aware (#1209)
* feat(web): make read render-aware

* fix(web): fail when networkidle readiness is unmet

* test(web): avoid unhandled networkidle rejection
2026-04-28 23:07:47 +08:00
jakevin dff3fd8950 feat(browser): manage owned workspaces as tab leases (#1204)
* feat(browser): manage owned workspaces as tab leases

* fix(browser): harden lease reconciliation paths
2026-04-28 21:05:33 +08:00
Xeron ff571fc965 fix(jd): separate main and detail image extraction (#1205)
* fix(jd): separate item image extraction

* chore: update CLI manifest

* test(jd): update item adapter expectations

* fix(jd): collect CSS detail images

* fix(jd): extract detail images from scripts and frames

* fix(jd): recover detail images and selected specs

* fix(jd): fail fast on blocked item extraction

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-28 21:05:18 +08:00
Benjamin Liu 08a2428306 fix(deepseek): fix send button detection and file upload (#1166)
* fix(deepseek): fix send button detection in sendMessage

The previous selector `btn.closest('div')?.querySelector('textarea')`
always returned null because the button itself is a div, so
closest('div') returns the button, which has no textarea inside.
This caused every send to fall through to the Enter key fallback.

Walk up from the textarea to find the input container, then select
the last enabled non-toggle button with an SVG icon (the send
button). Excludes `.ds-toggle-button` elements (DeepThink / Search
toggles) so only the actual send button is clicked.

* fix(deepseek): fail closed when upload never enables send

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-28 14:18:29 +08:00
dependabot[bot] 02b3033954 chore(deps): bump jsdom from 29.0.2 to 29.1.0 (#1199)
Bumps [jsdom](https://github.com/jsdom/jsdom) from 29.0.2 to 29.1.0.
- [Release notes](https://github.com/jsdom/jsdom/releases)
- [Commits](https://github.com/jsdom/jsdom/compare/v29.0.2...v29.1.0)

---
updated-dependencies:
- dependency-name: jsdom
  dependency-version: 29.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 14:10:13 +08:00
jakevin 141ec95c01 feat(browser): bind current tab to bound workspace (#1196)
* feat(browser): bind current tab to bound workspace

* docs(browser): document bound session idle semantics

* test(extension): cover bind-current owned-overwrite refusal

Adds regression for the second guard in handleBindCurrent that refuses
binding when the bound:* workspace already has an owned automation
window. Previously only the non-bound prefix path was tested.

* refactor(browser): rename bind command

* fix(browser): bind only current window tabs

* fix(browser): fail unbind when detach command fails
2026-04-27 17:35:37 +08:00
Benjamin Liu bc9ae39cfc feat(google-scholar): add cite and profile commands, fix search dedup (#1176)
* feat(google-scholar): add cite and profile commands, fix search dedup

- cite: get BibTeX/EndNote/RefMan/RefWorks citation for a paper.
  Clicks the cite button in search results and fetches the citation
  content from Google's citation endpoint.

- profile: view an author's Google Scholar profile (h-index,
  i10-index, citation count, top papers). Accepts author name
  or Scholar user ID.

- search: fix duplicate results caused by CSS selector matching
  both outer container (.gs_r.gs_or.gs_scl) and inner child
  (.gs_ri) for each paper.

Closes #1174, closes #1175

* fix(google-scholar): fail fast on cite and profile misses

* fix(google-scholar): document new commands and lock dedup test

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-27 15:36:49 +08:00
CissiBot 02dbbb1c18 fix(uiverse): harden navigation retries and preview lookup (#1171)
Pre-navigate Uiverse commands and retry detached browser bridge failures so code and preview flows stop falling back to about:blank. Broaden preview element matching for input-root components and cover the new navigation contract in tests.
2026-04-27 15:33:23 +08:00
yorick 07760d00ba fix: separate author name from date text in search results (#1173)
* separate author name from date text in search results

* fix(xiaohongshu): constrain author date stripping

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-27 15:28:03 +08:00
hanzi ac80c4962b Fix twitter post image uploads (#1180) 2026-04-27 15:25:42 +08:00
wjjsn e2b595272b fix(doubao): update message selectors for DOM restructure (#1190)
- Replace broken data-testid selectors with class-based selectors
- Message list: [class*="message-list-S2Fv2S"], .container-PvPoAn
- User messages: [class*="bg-g-send-msg-bubble"]
- Assistant messages: [class*="bg-g-receive-msg-bubble"]
- Add stopLines for UI noise: 请仔细甄别, 下载电脑版

Fixes #1183
2026-04-27 15:20:56 +08:00
darthjaja 23beb9508c fix(youtube): channel videos-tab fallback reads wrong tab from InnerTube response (#1164)
* fix(youtube): channel videos-tab fallback reads wrong tab from InnerTube response

After PR #1109, `opencli youtube channel <id>` still returns empty
`recent_videos` for channels whose Home tab is empty AND whose InnerTube
`/youtubei/v1/browse` response includes multiple tabs.

Root cause: the fallback fetch sends a browse request with the Videos
tab's `params`. The response, however, includes ALL tabs (Home, Videos,
Shorts, ...), with only the requested tab marked `selected: true`. The
existing code reads `tabs?.[0]?.tabRenderer?.content?.richGridRenderer?.contents`
— for multi-tab responses `tabs[0]` is Home (empty), so `richGrid` ends
up `[]` and `recentVideos` stays empty. PR #1109's test channels happened
to return single-tab lists with Videos at index 0, masking the bug.

Fix: find the tab with `selected: true` instead of assuming `tabs[0]`.

Reproducer: `opencli youtube channel UC44DSuDgw7_qccvZzIK3Jpg`
(杀鱼伟-Vi, ~3.1K subs, posts daily). Returns 0 videos pre-patch, 30+
videos post-patch.

`npm run typecheck` clean, `npm test` passes (1952/1952).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(youtube): preserve videos tab fallback

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-27 15:16:09 +08:00
wjjsn 9cd247d74d fix(doubao): use ID selector for send button (#1188)
* fix(doubao): use ID selector for send button

The clickSendButtonScript was searching for the send button by walking up
the DOM tree only 2 levels from the textarea, but the actual send button
#flow-end-msg-send is at level 5. This caused message sending to fail.

Fix by directly selecting the button via its ID.

* test(doubao): update send button selector assertions

* fix(doubao): keep send-button fallback contract

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-27 15:07:59 +08:00
sontjer f099e4cb3a fix(toutiao): fix NON_TITLE_LINES scope error in articles parser (#1179)
* fix(toutiao): move NON_TITLE_LINES inside function scope

NON_TITLE_LINES was defined outside parseToutiaoArticlesText() as a
module-level const. When the function is serialized via .toString()
and injected into browser evaluate context, outer scope variables
are not available, causing 'NON_TITLE_LINES is not defined' error.

Fix: move NON_TITLE_LINES inside the function so it's included in
the serialized string.

* test(toutiao): cover serialized articles parser

---------

Co-authored-by: sontjer <sontjer@github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-27 15:06:54 +08:00
jakevin a999dcec83 docs: update extension install to Chrome Web Store (#1194)
* docs: update extension install to Chrome Web Store link

Extension is now published on Chrome Web Store. Replace manual
download/unpack instructions with the store link across READMEs
and skill docs.

* docs: restore manual install as Option B alongside Chrome Web Store
2026-04-27 14:36:44 +08:00
jakevin ca8459c400 fix(browser): keep text/javascript API responses in network output 2026-04-27 14:23:01 +08:00
Benjamin Liu 54ffc88283 fix(web): preserve button text in web read output (#1185)
The shared article-download pipeline strips all <button> elements
via STRIPPED_TAGS, which is correct for article adapters (zhihu,
weixin) but causes web read to silently lose meaningful button
content like "Download All" on generic pages.

Override the button stripping in web read's configureTurndown
callback so button text is preserved as inline content.

Fixes #1184
2026-04-26 20:47:03 +08:00
jakevin d9c96f7e3b chore: bump version to 1.7.8 (#1178)
Release / release (push) Has been cancelled
2026-04-25 22:16:05 +08:00
jakevin 0e9e1ce953 chore(extension): restore pre-1.6.8 neon terminal icons (#1177)
Restore the original icons (commit b2fa7da) that were replaced by the
v1.6.8 "refresh icons" change in e9867dc. Per user feedback, the original
neon `>_` design read more clearly and was preferred over the abstract
arrow + dash variant.

Reverts only the four icon PNGs (16/32/48/128); manifest, popup, and
extension version stay where they are.
2026-04-25 21:20:14 +08:00
Ray的新范式 766677422d fix(chatgpt-app): use AX send flow and support zh-CN generating state (#1135)
* fix(chatgpt-app): use AX send flow and support zh-CN generating state

* fix(chatgpt-app): fail fast on stale AX send path

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-24 19:52:51 +08:00
Benjamin Liu c0a49e4b44 feat(weixin): add create-draft and drafts commands for Official Account (#1095)
* feat(weixin): add publish (create draft with cover) and drafts (list drafts)

Closes #441

* fix(weixin): rename publish to create-draft to match issue #441 proposal

* fix(weixin): fail fast on draft auth and empty states

* test(weixin): align adapter imports with repo style

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-24 19:39:34 +08:00
GanFanNewOrder 6827de4ab2 fix(amazon): fall back discussion to product page (#1154)
* fix(amazon): fall back discussion to product page

* fix(amazon): tighten sign-in fallback detection

---------

Co-authored-by: 泽加武 <zejiawu@zejiawudeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-23 17:25:46 +08:00
Aaron Chang 43873326c8 feat(toutiao): add articles adapter for 头条号 creator dashboard (#1148)
* feat(toutiao): add articles adapter for 头条号 creator dashboard

Add adapter to fetch article list and stats from 头条号 creator backend (mp.toutiao.com).
Supports pagination (1-4 pages) and returns title, date, status, views, reads, likes, comments.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(toutiao): preserve short article titles

---------

Co-authored-by: Aaron Chang <yugenchang@future.ov>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-23 17:24:42 +08:00
Benjamin Liu c8eedee760 fix(deepseek): fix history titles and resume conversation on ask (#1153)
* fix(deepseek): fix history titles and resume conversation on ask

- history: use link.innerText instead of link.querySelector('div') for
  title extraction. DeepSeek changed sidebar DOM; the first child div
  is now an empty ds-focus-ring element, causing all titles to show as
  (untitled).

- ask: when workspace is recycled (idle timeout) and --new is false
  (default), click the most recent sidebar conversation link to resume
  it instead of staying on the blank new-chat page. Skip model
  selection when inside an existing conversation since the selector is
  only rendered on the new-chat page.

- ensureOnDeepSeek: return boolean indicating whether navigation
  occurred, so callers can react to workspace recycling.

Closes #1149

* fix(deepseek): fail fast on explicit model resume

* fix(cli): expose only explicit option sources

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-23 17:10:23 +08:00
GanFanNewOrder 9870258075 feat(powerchina): add procurement search adapter (#1155)
* feat(powerchina): add procurement search adapter

* fix(powerchina): stabilize api detail urls

---------

Co-authored-by: 泽加武 <zejiawu@zejiawudeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-23 17:01:38 +08:00
Benjamin Liu a3d0185afa fix(sinafinance): match stock symbol in addition to name (#1158)
* fix(sinafinance): match stock symbol in addition to name

The scoring function only compared user input against the Chinese
display name (p[4] from suggest API), so searching "AAPL" matched
"AAPLU" (score 0.8) over Apple Inc. whose name field is "苹果"
(score 0). Check the symbol field first for exact and partial matches.

Fixes #1157

* docs(sinafinance): add missing commands to adapter index

The index table only listed `news` for sinafinance. Added the other
three commands (`rolling-news`, `stock`, `stock-rank`) and updated
the mode from Public to hybrid since rolling-news and stock-rank
require a browser.

Fixes #1156

* test(sinafinance): lock stock symbol matching

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-23 14:17:24 +08:00
jakevin 9c2eb07037 chore: bump version to 1.7.7 (#1152)
Release / release (push) Has been cancelled
2026-04-23 00:03:11 +08:00
jakevin 60114f99ba fix: quality audit bug fixes across core modules (#1151)
* fix: address quality audit bugs across core modules

- output.ts: fix elapsed=0 not showing (falsy check → undefined check)
- cdp.ts: log WebSocket parse errors and getResponseBody failures in verbose mode
- launcher.ts: replace sync execFileSync('sleep') with async setTimeout
- daemon.ts: add missing extensionCompatRange=null in error handler
- errors.ts: add recursion depth limit to serializeCause
- download/index.ts: remove Promise constructor anti-pattern (void async IIFE),
  use cookie.expirationDate instead of hardcoded 1-year expiry

* fix: log network interceptor parse failures, use correct exit codes

- captureNetworkItems: log JSON.parse failure in verbose mode instead of silent swallow
- emitNetworkError: use USAGE_ERROR only for invalid_args/filter/max_body,
  GENERIC_ERROR for runtime failures (capture_failed, cache_expired, etc.)

* test: add regression tests for elapsed=0 and deep cause chain truncation
2026-04-22 23:55:52 +08:00
jakevin f88b965dc5 fix(daemon): allow extension ping CORS (#1150) 2026-04-22 23:46:44 +08:00
jakevin 648390eacd feat(web,download): absorb #1048 — video/audio/iframe + --stdout (#1146)
* feat(web,download): absorb #1048 media + --stdout into web read

Distill the useful pieces of the abandoned PR #1048 (`web md`) into the
existing shared pipeline instead of introducing a parallel command:

- Turndown rules for <video> / <audio> / <iframe>. Video and audio are
  emitted as inline HTML so renderers that support it keep playback,
  and iframes degrade to markdown links (title + src) so embedded
  content (YouTube, CodePen, …) stays reachable. `iframe` moves out of
  STRIPPED_TAGS since it's now handled explicitly.
- `stdout` option on ArticleDownloadOptions: writes the full markdown
  to process.stdout, skips image download + mkdir + file write, and
  reports saved='-'. Remote image URLs stay intact so piped output is
  self-contained.
- `web read --stdout` wires the above through.
- Lazy-load src rewrite: the extractor now promotes data-src /
  data-original / data-lazy-src / data-srcset onto `src` before the
  HTML is frozen, so the markdown body and the image-download list
  reference the same URL (previously a page with placeholder.gif +
  data-src produced broken image links in the output).

Nothing in #1048 that overlapped with the already-merged #1143
hardening was kept — no new Readability wiring, no duplicate Turndown
config, no new command.

* fix(web): keep stdout streaming output clean

* fix(tests): update iframe e2e assertion and drop relative src import

- article-extract e2e fixture test: iframe now converts to a markdown
  link instead of being stripped, so assert the YouTube embed link
  survives rather than asserting its absence.
- clis/web/read.test.js: replace vi.importActual('../../src/registry.js')
  with a direct __test__.command export from read.js; the relative
  import into src/ tripped the package-exports adapter guardrail.
2026-04-22 18:42:38 +08:00
Kagura 733ac0747d fix(deepseek): separate thinking process from response in --think mode (#1142)
* fix(deepseek): separate thinking process from response in --think mode (#1124)

When --think is enabled, the response now includes separate fields:
- response: clean final answer only
- thinking: chain-of-thought reasoning content
- thinking_time: time spent thinking (e.g. '1')

Supports both English ('Thought for X seconds') and Chinese
('已思考(用时 X 秒)') thinking header patterns.

Fixes #1124

* chore: regenerate cli-manifest.json

* fix(deepseek): DOM-level think/response separation, dynamic columns

Blocker 1: Replace fragile split(/\n\n+/) heuristic in parseThinkingResponse()
with DOM-level extraction in waitForResponse(). The page evaluate now queries
distinct DOM nodes (.ds-markdown--think vs .ds-markdown) for thinking and
response content. The text-level parser falls back to treating everything
after the header as thinking (no split), avoiding silent corruption of
multi-paragraph content.

Blocker 2: Remove static columns declaration from askCommand. The renderer
infers columns from row keys, so non-think output only shows 'response'
while think output shows all three columns.

Tests added for multi-paragraph thinking, multi-paragraph answer, and
non-think column regression guard.

* chore: regenerate cli-manifest.json
2026-04-22 18:03:37 +08:00
jakevin e83148a2c1 feat(download): harden HTML→Markdown pipeline (#1143)
* feat(download): harden HTML→Markdown pipeline

Inspired by the MD-This-Page / markdown-viewer-extension analysis, tighten
the shared article→Markdown converter used by zhihu/weixin/web adapters:

- enable turndown-plugin-gfm (tables, strikethrough, task lists)
- strip script/style/noscript/iframe/canvas/form/button/dialog unconditionally
- strip SVG via a dedicated rule (not in HTMLElementTagNameMap)
- drop base64 data-URI images so they don't bloat .md output
- post-process: collapse NBSP, lone bullet/middle-dot residue,
  trailing whitespace, and 3+ blank lines
- frontmatter shape guarantees ≤2 consecutive newlines even when
  some metadata fields are absent

Adds a minimal local .d.ts for turndown-plugin-gfm and 6 new tests
covering GFM conversion, tag stripping, base64 drop, and whitespace cleanup.

* fix(download): emit canonical markdown strikethrough

* feat(download,browser): finish article pipeline polish

Per the follow-up from the MD-This-Page / markdown-viewer-extension
analysis, land the remaining items in the same PR instead of splitting:

article-download.ts
- extend STRIPPED_TAGS with header/footer/nav/aside (page chrome; the
  article's title/author/publishTime are supplied as separate fields on
  ArticleData, so duplicated DOM is redundant)
- new option ArticleDownloadOptions.cleanSelectors — per-adapter CSS
  selector list removed before conversion, applied as a Turndown rule
  via node.matches so invalid selectors fail silently

browser/article-extract.ts (new)
- generic Readability-based extraction that runs in-page via CDP
  evaluate (no jsdom in Node)
- short-circuits non-HTML documents (text/plain, JSON, XML) and the
  single-<pre> "browser rendering a plain text file" case
- clones the document before any mutation (preserves live page state
  for subsequent snapshot / click)
- isProbablyReaderable gate, Readability.parse on the clone, then a
  fallback chain main → [role="main"] → #main-content → … → body
- library sources are JSON-embedded and eval'd inside a Function scope
  so their backticks / module.exports guards don't collide with the
  surrounding IIFE

Tests
- article-download: page-chrome strip, cleanSelectors match + invalid
  selector silently ignored (2 new)
- article-extract: JS generation contents, default fallback chain,
  response normalization, null / malformed handling, and a Function()
  parse check to catch any template-literal break-out in the embedded
  Readability sources (8 new)

* fix(download): honor selector cleanup in fallback paths

* test(e2e): real-site regression for hardened article pipeline

Adds tests/e2e/article-download-pipeline.test.ts driving `opencli web read`
through 6 representative pages (example.com baseline, Wikipedia GFM tables,
MDN metadata, GitHub fenced code, Vercel SSR blog, Ruan Yifeng CJK+images)
and asserting the post-processing invariants: no base64/script/style leaks,
no blank-line runs, no residue, no trailing whitespace, no NBSP.

Graceful skip on bot detection / transient CDP errors, with a single retry.

All 6 sites pass locally (37s total).

* test(browser): add article extraction e2e fixtures
2026-04-22 14:49:35 +08:00
jakevin 3ec98b9405 feat(51job): comprehensive adapter (search / hot / detail / company) (#1132)
* feat(51job): add comprehensive 51job adapter (search / hot / detail / company)

Four adapters covering the main 51job surface:

- `51job search <keyword>` — keyword job search via we.51job.com/api/job/search-pc.
  Rich filters: --area (40+ city name/alias → 6-digit code), --salary, --experience,
  --degree, --companyType, --companySize, --sort, --page, --limit. Response already
  carries full jobDescribe + HR + company + encCoId, so most callers won't need detail.

- `51job hot` — same endpoint with empty keyword, returns 51job's recommendation feed.

- `51job detail <jobId>` — scrapes jobs.51job.com/x/<jobId>.html. Returns description,
  welfare tags, category, address, age requirement, company meta.

- `51job company <encCoId>` — scrapes jobs.51job.com/all/co<encCoId>.html. Job cards
  carry a `sensorsdata` JSON attribute, so we parse that instead of fragile DOM text.
  Company meta from `.c-info.ellipsis`, intro from `#companyIntroRef`.

All four are Strategy.COOKIE + browser:true + navigateBefore:false. 51job sits
behind Aliyun WAF — bare curl / Node-side fetch always hits the slider challenge
(tried copying acw_sc__v2 + ssxmod_itna cookies to Node, WAF also checks TLS
fingerprint and JS execution). Only reliable path is browser-context fetch via
`page.evaluate(fetch(url, {credentials:'include'}))`, so utils.js exports
`pageFetchJson` that wraps this pattern + detects WAF-served HTML.

Verify fixtures included (~/.opencli/sites/51job/verify/*.json) — four adapters
pass `opencli browser verify 51job/<cmd>` with rowCount / columns / types /
patterns / notEmpty checks. Eyeballed jobId 171699769 on jobs.51job.com/suzhou
matches adapter output.

* fix(51job): tighten city handling and docs

* chore: regenerate cli-manifest.json after 51job column cleanup
2026-04-22 13:16:23 +08:00
lwyang 00d54135ad feat(weread): add ai-outline command (#1141)
* feat(weread): add ai-outline command for AI-generated book outlines

Two-step API flow: fetch chapter UIDs via authenticated chapterInfos,
then retrieve hierarchical AI outline from public outline endpoint.

Supports --depth to control detail level (2=topics, 3=key points,
4=full details) and --raw for structured output (chapter/idx/level/text)
suitable for programmatic consumption.

Closes #1140

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(weread): tighten ai-outline auth contract

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-22 13:13:11 +08:00
lwyang 69d3a73390 fix(weread/book): add fallback selectors for reader page without cover (#1138)
* fix(weread/book): add fallback selectors for reader page without cover

When the private API session expires, `loadReaderFallbackResult` navigates
to the reader URL. The page now sometimes skips the cover/flyleaf and
renders reading content directly, causing the wait for cover/flyleaf title
selectors to time out.

- Add `.readerTopBar_title_link` to `page.wait` selector (always present)
- Use cascading `firstText()` for title: cover → flyleaf → outline → top bar
- Use cascading `firstText()` for author: cover → flyleaf → outline → document.title

Fixes #1137

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(weread/book): parse author from trailing title segments

* fix(weread): avoid author guess from document title

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-22 12:30:33 +08:00
Mike Jing 5460a18d71 fix(xiaoyuzhou): correct podcast-episodes API endpoint (#1129)
* fix(xiaoyuzhou): correct podcast-episodes API endpoint

The endpoint `/v1/podcast/listEpisode` returns 404. The correct
endpoint is `/v1/episode/list` (verified against Xiaoyuzhou iOS app
traffic; also matches the `episode-list` implementation in
ultrazg/xyz, a widely-used Xiaoyuzhou API wrapper).

Additionally, the server requires an `order` field in the request
body (returns 400 if omitted). Add `order: 'desc'` so callers get
the latest episodes first, matching typical UX for a podcast feed.

Before:  podcast-episodes -> HTTP 404 for every podcast
After:   podcast-episodes returns the N most recent episodes

Tested against real podcast 626b46ea9cbbf0451cf5a962
(张小珺|商业访谈录) — now returns 140 episodes correctly.

* test(xiaoyuzhou): lock podcast episodes endpoint

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-22 12:17:00 +08:00
jakevin dc724262f2 feat: agent-native retrospective — analyze / verify guards / fixture content checks (#1133)
* feat: agent-native retrospective — analyze / verify guards / fixture content checks

Post-mortem on slow 1point3acres + 51job adapter sessions, consolidated
into one PR. Scope is "reduce uncertainty and catch silent failures"
— the two things that sink agent success rate on first-time adapters.

Changes:
- `browser analyze <url>` — one command returns pattern (A/B/C/D),
  anti-bot vendor (Aliyun/Cloudflare/Akamai/Geetest), nearest adapter,
  and a single-sentence recommended_next_step. Replaces the three-step
  open/wait/network recon loop when it can reach a confident verdict.
- `browser wait xhr <regex>` — poll for a specific XHR URL instead of
  blind `wait time N`, so SPA data-arrival barriers are deterministic.
- Fixture `mustNotContain` / `mustBeTruthy` — catch two silent-failure
  modes `notEmpty` misses: content contamination (sibling DOM bleed)
  and `|| 0` / `|| false` fallbacks.
- `browser verify` post-success site-memory check + `--strict-memory`
  — verify-green no longer hides the case where `~/.opencli/sites/`
  was never written back. Memory only materializes if authors write it.
- CI: guard that committed `cli-manifest.json` matches a fresh build.
  Main was already drifted (#1118 left stale ordering + a missing arg);
  this PR regenerates the manifest and will catch the next drift.

Docs (opencli-adapter-author + opencli-autofix skills):
- `success-rate-pitfalls.md` — 10 concrete silent-failure scenarios
  seen in real adapter sessions, each with defense via fixture /
  adapter patterns.
- `autofix` gains discipline rule #6: verify pattern failure means
  tighten the adapter, never loosen the fixture.
- `site-recon.md` leads with `browser analyze`; `api-discovery.md`
  adds a §0 covering WAF vendor detection and cross-subdomain CORS
  (the two gotchas that burned the 51job session).
- `wait time 3` → `wait time 2`, with `wait xhr` as the robust choice.

* fix: make output-dir defaults host-independent in manifest

Three adapters (chatgpt/image, gemini/image, instagram/download) baked
`path.join(os.homedir(), ...)` into the `default` field of their args.
The committed manifest therefore carried my personal `/Users/jakevin/...`
paths — which agents running on a different host saw as surprising
defaults. The drift guard I just added to CI caught it on the first run.

Runtime behavior is unchanged: each adapter still falls back to
`path.join(os.homedir(), …)` inside `func` when the kwarg is absent.
Only the displayed / registered default becomes a tilde-path.

* fix(cli): enforce strict-memory without fixture

* fix(browser): harden analyze and xhr guards

* fix(browser): fallback to interceptor buffer
2026-04-22 01:59:19 +08:00
jakevin 5935191e04 feat(verify): fixture-based value validation + skill docs for COOKIE pitfalls (#1131)
* feat(verify): fixture-based value validation + skill docs for COOKIE pitfalls

`opencli browser verify` now loads `~/.opencli/sites/<site>/verify/<cmd>.json`
when present and validates row count / columns / types / patterns / notEmpty
against the live adapter output. Without a fixture, behavior is unchanged
(just runs the adapter and prints). New flags `--write-fixture`,
`--update-fixture`, `--no-fixture` seed / refresh / bypass the spec.

Motivation: previous verify only checked that the adapter exited 0 and
produced *something* — shape regressions (author name bleeding across rows,
a column silently becoming null after a site refresh, duplicated thread-level
time on every post) all passed "✓ Adapter works!" and shipped broken.

Skill doc updates (opencli-adapter-author):
- adapter-template.md: new "COOKIE adapter 骨架" section — HttpOnly +
  dual-domain cookie read via `page.getCookies`, Node-side fetch for HTML
  (explaining why `page.evaluate(fetch(...))` is the wrong tool when
  `navigateBefore: false` or the response is non-UTF-8), and empty-state
  sentinel row over `[]`
- api-discovery.md §4: note that BBS engines (Discuz/phpBB/vBulletin) set
  auth cookies on the root domain + HttpOnly, so single-domain `getCookies`
  calls silently miss them
- SKILL.md Step 10/12: make `--write-fixture` part of the runbook, forbid
  debug dumps outside `~/.opencli/sites/<site>/fixtures/` or `/tmp/`

* fix(verify): support positional argv in fixture args + site-memory docs

Reviewer feedback blocker: fixture.args was Record<string, unknown>,
expanded as --key value only, so positional-subject adapters
(<tid>/<url>/<query>) couldn't be verified. Repo convention is
"主语优先 positional".

- verify-fixture.ts: args now accepts Record<string, unknown> | unknown[].
  Object → --k v pairs; array → verbatim passthrough. New helper
  expandFixtureArgs() centralizes the branching.
- cli.ts verify action: swap inline expansion for expandFixtureArgs().
- verify-fixture.test.ts: 6 new cases covering array form, mixed
  positional+flag, empty shapes, passthrough stringification.
- site-memory.md: Layer 2 tree now lists verify/<cmd>.json; new schema
  block distinguishes it from fixtures/<cmd>-<ts>.json; runbook timing
  section gets a Step 10 verify-write row. Repo-tree debug-dump ban
  clarified.
- adapter-template.md: new "Verify fixture" section with named-flag and
  positional recipes, honest about --write-fixture only seeding named.

Smoke-tested 1point3acres/thread (positional <tid>): fixture round-trip
green (args=["1173710","--limit","2"]).
2026-04-22 00:08:19 +08:00
jakevin 0710678986 docs: fix stale references in READMEs and autofix skill doc (#1130)
- Add missing skills (opencli-browser, opencli-usage) to install list, table, and references
- Add missing browser commands (find, extract, frames)
- Update adapter command lists (twitter tweets, bilibili comments, xiaohongshu note+comments, xiaoyuzhou auth, amazon rankings, hackernews)
- Fix CLI Hub names: dingtalk→dws, wecom→wecom-cli
- Fix exit codes example: opencli github issues→opencli gh issue list
- Fix autofix skill doc: page.waitForSelector→page.wait({ selector })
2026-04-21 23:28:13 +08:00
Ray a6d1eca204 fix(bilibili): resolve full video URLs and preserve full description (#1118)
Two issues surfaced post-merge of #1110 by the Copilot reviewer:

1. Help text and docs advertise `video URL` as a valid input for
   `opencli bilibili video <bvid>`, but the original implementation
   delegated the whole input to `resolveBvid()` — which only recognises
   bare `BV...` IDs and `b23.tv` short codes. A canonical bilibili URL
   like `https://www.bilibili.com/video/BV.../` therefore got rewritten
   to `https://b23.tv/www.bilibili.com/video/BV.../` and failed before
   ever calling the view API.

   Fix: pre-extract the BV ID from `bilibili.com/video/<BV>...` and
   `bilibili.com/bangumi/play/<BV>...` URLs (www / m. / with or without
   query string) in `video.js`, and fall through to `resolveBvid()`
   only for bare BV IDs and `b23.tv` links.

2. `description` was being truncated to 200 chars with whitespace
   collapsed before being returned. JSON/YAML consumers silently lost
   the full `desc` value. Other bilibili adapters return raw fields.

   Fix: return the full `d.desc` verbatim and let consumers/display
   layers handle formatting.

Adds four regression tests for the URL paths (full URL, URL with query
string, m.bilibili.com mobile URL) and one for description integrity
(> 200 chars, preserved verbatim).

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 23:14:04 +08:00
Benjamin Liu 92efa38aba fix(deepseek): use position-based model selection instead of text matching (#1123)
* fix(deepseek): use position-based model selection instead of text matching

Fixes #1111

* fix(deepseek): preserve explicit instant model contract

* fix(deepseek): guard expert selector arity

---------

Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-21 23:10:17 +08:00
Dylanwoo 666a955fac feat(twitter): expose has_media and media_urls columns (#1115)
Adds two additive columns to the Twitter read commands (search, timeline,
tweets, thread, likes):

- has_media: boolean — true if the tweet contains any photo, video, or GIF
- media_urls: string[] — photo URLs and mp4 variant URLs for videos/GIFs,
  extracted from legacy.extended_entities.media (falls back to entities.media)

The INTERCEPT/COOKIE payloads already carry this data; this change only
extends the row-mapping layer, so no new network work is needed. Pattern
mirrors #465 (time column).

Shared extraction helper lives in clis/twitter/shared.js so all five
adapters stay consistent, with unit coverage for photo, video (mp4 variant
selection), animated_gif, entities.media fallback, and the empty case.

Closes #1107
2026-04-21 23:02:22 +08:00
jakevin 2ad417949a docs(skills): restore and rewrite opencli-usage as orientation skill (#1128)
* docs(skills): restore and rewrite opencli-usage as orientation skill

The original opencli-usage skill was deleted in PR #1094 as part of the
skill consolidation, but its role (top-level orientation to what opencli
is, how to discover adapters, what flags/env/formats are universal, and
which specialized skill to load next) was not covered elsewhere. Restore
it, but deliberately NOT as a verbatim copy:

- Drop the hand-maintained 100-adapter table. There are 100+ adapters
  and the list moves every week — `opencli list -f json` is the source
  of truth agents should call at the start of a task.
- Replace it with the meta-structure agents actually need: the three
  pillars (adapters / browser driving / external CLI passthrough), the
  strategy tags (PUBLIC | COOKIE | HEADER | INTERCEPT | UI | LOCAL)
  and what each implies for prerequisites, universal flags (-f, -v),
  output formats, env vars, self-repair hook, adapter authoring paths,
  plugins, external CLI passthrough.
- Explicitly list the commands PR #1094 removed (`explore`, `record`,
  `web` / `desktop` top-level groups) so agents don't attempt them.
- Cross-link to the four post-consolidation skills: opencli-browser
  (ad-hoc driving), opencli-adapter-author (writing adapters),
  opencli-autofix (repair flow), smart-search (search routing).

Adapter-author description updated to stop claiming it replaces
opencli-usage.

* docs(skills): tighten opencli-usage validate + doctor scope per review

- validate: describe as registry-level semantic check (description, domain,
  pipeline step names, func|pipeline|_lazy presence, arg duplicates), not
  YAML/TS syntax check — matches src/validate.ts
- doctor: narrow to browser-bridge diagnostic; PUBLIC/LOCAL adapters, list,
  validate, verify, plugins, and external-CLI passthrough do not need it
2026-04-21 22:04:09 +08:00
jakevin 9675f6262e docs: add CHANGELOG entry for 1.7.6 (#1127)
Backfill the 1.7.6 section that was missing from the release PR.
Covers window lifecycle flags, selector-first browser interactions,
agent-native payload, compound form fields, three new adapter commands,
four fixes, skill doc updates, and extension 1.0.2 body-truncation
contract unification.
2026-04-21 21:46:43 +08:00
jakevin dba333d1f9 chore: bump version to 1.7.6, extension to 1.0.2 (#1126)
Release / release (push) Has been cancelled
2026-04-21 21:42:46 +08:00
jakevin 7c35935861 docs: sync live and focus window docs (#1125) 2026-04-21 21:41:47 +08:00
jakevin d36bee04bb feat(cli): add --live and --focus flags for automation window lifecycle (#1122)
--live (OPENCLI_LIVE=1) keeps the automation window open after an adapter
command finishes, so agents or humans can inspect the page state. Default
behavior (immediate closeWindow) is unchanged.

--focus (OPENCLI_WINDOW_FOCUSED=1) surfaces the existing env-var toggle as a
CLI flag so users don't need to shell-export to see the window in foreground.

Both flags are parsed early in main.ts and stripped from argv, so they can be
placed anywhere on the command line and work on any subcommand (adapter or
browser).
2026-04-21 21:14:42 +08:00
jakevin 2f66d48a47 docs(skills): restore and upgrade opencli-browser skill (#1119)
Restore `skills/opencli-browser/SKILL.md`, deleted in #1094, rewritten for
the post-#1116 browser CLI surface: selector-first target contract,
`match_level { exact | stable | reidentified }`, compound fields for
date/time/select/file, structured error codes with `available` vs
`candidates`, new `find` / `extract` / `network --filter` commands,
html tree budgets, tabs/frames, cost guide, recipes, pitfalls.

Review tightened two contract-drift bugs before merge:
- `browser tab list` envelope field is `page`, not `targetId`
- `network --ttl` default is `24h`, not `~5min`

2 reviewers green (codex-mini1, First-principles-1); CI all-green.
2026-04-21 17:32:33 +08:00
jakevin 04a5a171d5 feat(browser): compound expansion + cascading stale-ref + bbox 0.99 dedup (#1116)
* feat(browser): compound expansion + cascading stale-ref + bbox 0.99 dedup

Three agent-native upgrades inspired by browser-use, landed as one PR
because they share the same target / snapshot / find surface.

  1. Compound expansion (compound.ts)
     Date/time/datetime-local/month/week, select, and file inputs now
     emit a `compound` JSON field on `browser find --css` entries with
     format, current value, min/max (date family), full options list
     + selected (select), accept / multiple / files[] (file). Kills
     the three biggest form-page failure modes (wrong date format,
     guessed options, re-uploaded files) without extra round-trips.

  2. Cascading stale-ref (target-resolver.ts)
     Numeric ref resolution now walks three tiers before giving up:
     exact → stable (tag + strong id match, soft signals drifted) →
     reidentified (original ref lost, fingerprint uniquely found a
     live element; re-tag + refresh identity). Every success envelope
     carries `match_level` so callers can tell which tier matched.
     SPA re-renders / i18n label swaps no longer stall agents.

  3. BBox 0.99 containment for interactive descendants (dom-snapshot.ts)
     Adds a second dedup tier on top of the existing 0.95 non-interactive
     one. When a parent is a propagator (tag a/button OR role button/
     link/menuitem/tab/option) and a child is interactive but
     undistinctive (no aria-label/id/testid/name/form-control), fold
     it into the parent — removes `[1]<button> [2]<svg> [3]<span>`
     noise on icon buttons.

Tests: 287/287 pass (src/browser + src/cli.test.ts). Typecheck clean.

* fix(browser): address reviewer blockers on PR #1116

- compound select: walk ALL options to collect selected labels, not just
  the first 50 we serialize. Fixes dropdowns where the selected entry
  sits past COMPOUND_SELECT_OPTIONS_CAP (e.g. country lists, timezones)
  reporting current: "" even though the user picked a valid option.
- match_level: propagate the cascading match tier
  (exact / stable / reidentified) through IPage.click/typeText/scrollTo,
  BasePage, and the cli command envelopes (click / type / select /
  get text|value|html|attributes). Agents now see in JSON that the
  resolver had to fall back, instead of the tier being swallowed.
- compound contract is now also emitted by `browser state`
  (per-ref compounds: sidecar) and by `browser get html --as json`
  (compound field on each node), not only by `browser find --css`.
  Closes the gap where agents using the default snapshot still
  round-tripped `find` for every date / select / file control.

Adds targeted regression tests for each blocker + updates cli.test.ts
mocks to the new envelope shape.
2026-04-21 17:02:01 +08:00
Chris Chen f7fd805ef8 fix(twitter): add 5s timeout to resolveTwitterQueryId to prevent hang (#1106)
The resolveTwitterQueryId() function in shared.js fetches an external JSON
file from GitHub without a timeout. If the network request stalls, the
function never resolves and the twitter article command hangs indefinitely.

Add a 5-second AbortController timeout so the fetch fails fast and falls
back to the local script-scanning strategy. This fixes the reported hang
when opencli twitter article loses network connectivity.
2026-04-21 15:58:40 +08:00
Ray b92755597c feat(bilibili): add video command (#1110)
* feat(bilibili): add video command

Add `opencli bilibili video <bvid|url|short-link>` to fetch one
video's metadata via the public /x/web-interface/view endpoint.

Returns title, author, category, publish time, duration, view /
danmaku / reply / like / coin / favorite / share counts, parts,
thumbnail, and description as a key/value table.

Reuses `resolveBvid` and `apiGet` from clis/bilibili/utils.js to
stay consistent with the existing bilibili adapters
(subtitle/search/etc. all follow the same navigate + apiGet
pattern). Non-zero API codes surface as CommandExecutionError.

Fills a visible gap: existing bilibili commands cover search,
hot, subtitle, ranking, user-videos etc., but nothing returned
metadata for a single video — `web read` only gets a DOM shell.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(manifest): register bilibili video command

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-21 15:58:32 +08:00
Benjamin Liu b9bb3020a4 feat(deepseek): add file upload support via --file flag (#1093)
* WIP: deepseek file upload (blocked by 30s idle timeout)

* feat(deepseek): add file upload support via --file flag

Closes #1092

* fix(deepseek): use native file input path for --file

---------

Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-21 15:49:37 +08:00
Kagura b3db955da3 fix(youtube): fall back to Videos tab when Home tab has no videos (#1109)
* fix(youtube): fall back to Videos tab when Home tab has no videos (#1108)

Some channels have no video shelves on their Home tab, causing
`opencli youtube channel <id>` to return an empty `recent_videos` list
even though the channel has videos visible in the browser.

When the Home tab extraction finds zero videos, the command now makes
a second InnerTube browse request to the Videos tab and extracts from
its richGridRenderer format.

* fix(youtube): make Videos tab fallback locale-safe

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-21 15:45:32 +08:00
jakevin 8a8f048a89 feat(browser): selector-first find + get/click/type/select (A2+A3) (#1112)
* feat(browser): selector-first find + get/click/type/select with JSON envelope

A2: new `browser find --css <sel>` — structured JSON (matches_n + entries[]) so
agents can go from semantic selector directly to a list of candidates without
parsing free-text snapshot output. Per-entry shape: nth/ref/tag/role/text/attrs/
visible. Attr whitelist kept small (11 high-signal fields), invisible elements
still returned so agents can reason about offscreen vs missing.

A3: get text/value/attributes now accept a selector-first <target> (numeric ref
OR CSS) and emit `{value, matches_n}`. Bonus scope (approved by reviewers):
click/type/select share the same contract with `--nth <n>`, emitting
`{clicked|typed|selected, target, matches_n, ...}` on success.

Unified structured error envelope across all selector-first commands:
  { error: { code, message, hint?, candidates?, matches_n? } }
with codes invalid_selector / selector_not_found / selector_ambiguous /
selector_nth_out_of_range (CSS) plus not_found / stale_ref (numeric ref).

Write commands reject multi-match CSS without `--nth` as selector_ambiguous;
reads default to "first match wins" but always expose matches_n so agents
notice ambiguity. `resolveTargetJs` is the single source of truth; click /
typeText / scrollTo share a `runResolve` helper in BasePage.

No back-compat shims per design directive.

125 targeted tests green; tsc clean.

* fix(browser): unify selector surface + allocate fresh refs in find

Two blockers from PR #1112 review:

1. First-principles-1 (blocker): `browser find --css` now allocates fresh
   numeric refs for untagged matches. It scans `window.__opencli_ref_identity`
   (and any stray `data-opencli-ref` attrs) for the current max, allocates
   `max+1` upward, writes `data-opencli-ref` on the element, and populates
   the identity map with the same fingerprint shape snapshot uses (tag,
   role, text, ariaLabel, id, testId). `find -> click <ref>` now works on
   fresh pages without requiring `browser state` first. Type changed from
   `ref: number | null` to `ref: number`.

2. codex-mini1 (blocker): removed the `isCssLike` regex
   (`^[a-zA-Z#.\[]`) in `resolveTargetJs`. Valid selectors like `:root`,
   `:has(...)`, `*` used to short-circuit to "Cannot parse target" before
   reaching `querySelectorAll`, so `find --css` accepted them but
   `get/click/type/select` did not. Now: numeric → ref path, everything
   else → querySelectorAll, and the browser parser decides. Same selector
   surface across all selector-first commands.

Tests added:
- target-resolver: pseudo-selectors flow into CSS branch (not rejection)
- find: ref allocation writes attribute + identity map; fingerprint shape matches resolver
- cli: find envelope now expects numeric refs

127 targeted tests green; tsc clean.
2026-04-21 13:47:19 +08:00
jakevin acb08a4050 feat(browser): agent-native payload — network bodies, html tree budgets, extract command (#1104)
* feat(browser): agent-native payload — network bodies, html tree budgets, extract command

Three fixes/additions driven by agent-usage gaps, as one complete change:

- network (P0 fix): lift silent 4000-char body truncation in CDP + extension
  paths to an 8MB memory-guard cap, and surface body_truncated / body_full_size
  / body_truncation_reason in the --detail envelope so the agent sees when a
  body was cut. List view also exposes body_truncated_count and per-entry flag.
  Adds --max-body flag for explicit caller-side capping.

- get html --as json (P1): add --depth / --children-max / --text-max budget
  knobs on the tree serializer, plus a truncated={depth,children_dropped,
  text_truncated} envelope that only appears when a budget is hit. Lets the
  agent narrow DOM output without walking away empty-handed.

- extract (P2 new command): agent-native article/content channel. Scope →
  denoise (strip nav/header/footer/scripts/forms/etc.) → HTML→markdown via
  existing htmlToMarkdown → paragraph-boundary-aware chunk with stateless
  next_start_char resume cursor. Agents no longer misuse `get html` to read.

* fix(browser): unify body-truncation signal contract across raw/detail/fallback

Addresses review blockers on #1104:

- NETWORK_INTERCEPTOR_JS fallback no longer silently drops bodies above the
  per-entry cap. Raised cap to 1 MiB (ring stays at 200 entries), and on
  overflow keeps the string prefix + sets `bodyTruncated` / `bodyFullSize`
  so `browser network` propagates the same agent-visible signal the CDP /
  extension paths emit.

- `CachedNetworkEntry` schema switches from internal camelCase
  `bodyTruncated` to the user-facing `body_truncated` / `body_full_size`
  fields. `--raw` emits cache entries verbatim, so this removes the
  snake_case/camelCase split across list / --detail / --raw.

- Adds a `--raw` truncation-contract test that also asserts the camelCase
  fields do not leak through.
2026-04-21 12:17:49 +08:00
jakevin 37020c4348 feat(browser): add network --filter <fields> for agent-native request discovery (#1103)
Agents often know what fields a target request's body should contain
but not which captured request carries it. --filter lets them declare
the field set and get back only matching entries.

Matching is "any-segment": a field matches when it equals any segment
name of any inferShape() path (ignoring root $, array indices, and
bracket-quoted key syntax). Multiple fields AND together. Case-sensitive.

- invalid_filter for empty / commas-only values
- invalid_args when combined with --detail (mutually exclusive)
- 0 matches is a valid empty result, not an error
- persisted cache stays unfiltered so later --detail lookups still resolve

Envelope gains `filter` (echo) and `filter_dropped` (count of entries
passing the static-resource filter but not --filter). Existing --raw
and --all compose normally.
2026-04-21 02:38:49 +08:00
jakevin 6cf5cb2f25 feat(browser): remove silent html truncation, add --as json (#1102)
* feat(browser): remove silent html truncation, add --as json tree output

`browser get html` had two agent-hostile defaults:

1. A silent 50000-char cap on the returned HTML — agents that got a
   truncated page had no signal they were looking at half the DOM.
2. Only raw HTML string output, forcing agents to re-parse for
   structured extraction.

Changes:

- Default output is now the full outerHTML, no truncation
- `--max <n>` opts in to a character cap; when the cap actually
  trips, the HTML is prepended with
  `<!-- opencli: truncated N of M chars; re-run without --max ... -->`
  so agents always see the signal
- `--as json` returns `{selector, matched, tree}` where `tree` is
  `{tag, attrs, text, children}` recursively. `matched` is the full
  count of selector matches so agents know when more elements exist
  beyond the first. `text` is the node's own direct text children,
  whitespace-collapsed; child elements live in `children`.
- `--selector` not matching any element now emits structured
  `{error:{code:"selector_not_found", ...}}` with a non-zero exit
  code, in both raw and json modes (was `(empty)` stdout previously,
  indistinguishable from empty element)
- Invalid `--as` / negative `--max` emit structured
  `invalid_format` / `invalid_max` error codes

Extracted the tree serializer as `src/browser/html-tree.ts` so the
JS expression can be unit-tested against a DOM stub.

* fix(browser get html): structured errors for invalid selector & strict --max

Both edges previously bypassed the structured-error contract introduced in
#1102, which agents rely on for branching:

- Invalid CSS selector: querySelector(All) would throw SyntaxError through
  page.evaluate into the generic exception path. Wrap the lookup in try/catch
  inside page context for both raw and --as json paths; surface as
  {error:{code:"invalid_selector", message}} + non-zero exit.

- --max validation: parseInt silently accepted "1.5" -> 1 and "10abc" -> 10.
  Switch to a strict /^\\d+$/ check so fractional, negative, and non-numeric
  values all return {error:{code:"invalid_max"}}; validation runs up front so
  bad values never reach the page.

Covered by new unit tests in cli.test.ts (fractional, non-numeric, invalid
selector on raw + json) and html-tree.test.ts (SyntaxError -> invalidSelector
envelope).

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>

---------

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
2026-04-21 02:16:44 +08:00
jakevin 7fd8bd6fdc feat(browser): rewrite network for agent-native discovery (#1100)
* feat(browser): rewrite network command for agent-native discovery

Replace the index-based list + pretty-printed --detail flow with a
structured JSON interface built around stable keys, body-shape previews,
and a persistent capture cache. Agents can now reference captured
requests by operationName (GraphQL) or `METHOD host+pathname` (REST)
instead of array indexes that shift on every rerun.

- `browser network` now emits JSON: `{workspace, captured_at, count,
  filtered_out, entries: [{key, method, status, url, ct, size, shape}],
  detail_hint}` — no body payloads by default
- Shape inference (src/browser/shape.ts) walks response JSON into a
  flat path -> descriptor map with depth cap 6 and a 2KB budget per
  entry, so agents see structure without paying body tokens
- Stable key generator (src/browser/network-key.ts) derives
  `operationName` from graphql URLs and `METHOD host+pathname`
  elsewhere, disambiguating collisions with `#N` suffixes
- Persistent cache (src/browser/network-cache.ts) snapshots every
  capture to `~/.opencli/cache/browser-network/<workspace>.json` with
  a 24h TTL, so `--detail <key>` survives later commands
- `--detail <key>` returns `{key, url, method, status, ct, size, shape,
  body}` with structured error codes (cache_missing / cache_expired /
  cache_corrupt / key_not_found, the latter including available_keys)
- Add `--raw` for agents that want every full body inline, `--ttl` for
  cache lookups
- Update opencli-adapter-author + opencli-autofix skill docs to
  reference `--detail <key>` and the shape-first discovery flow

Supersedes the cache prototype in #1051.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>

* fix(browser): structured errors for capture/save, shape budget guard

Self-review findings on the network refactor:

- captureNetworkItems throwing (browser crashed / CDP dropped) now emits
  `error.code: capture_failed` on stdout rather than leaking a bare
  stderr line from browserAction's generic handler — agents get a
  parseable JSON blob on every failure path, matching the design goal.
- saveNetworkCache throwing (disk full, read-only path) is a soft
  failure: the captured data is already in hand, so surface a
  `cache_warning` field in the envelope and keep going instead of
  aborting. `--detail` lookups on that run will miss the cache but the
  listing still reaches the agent.
- shape.ts: guard the sub-walk on `add()`'s return value so the
  "budget hits on the array/object descriptor itself" path can never
  emit a stray child without its parent marker.
- network-key.ts: document that `#N` suffixes start at `#2` — the first
  occurrence stays bare, there is no `#1`. Matches test + code.

Added regression tests: `capture_failed` on readNetworkCapture throw,
`cache_warning` on persistence failure, shape budget hit on array descriptor.

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>

---------

Co-authored-by: freemandealer <freeman.zhang1992@gmail.com>
2026-04-21 01:32:00 +08:00
jakevin 295c5237cb fix(jianyu): keep accessible detail urls in search (#1099) 2026-04-21 00:34:15 +08:00
jakevin 93395653f4 feat(twitter): add tweets command for fetching user's recent posts (#1098)
* feat(twitter): add tweets command for fetching a user's recent posts

Adds `opencli twitter tweets <username> [--limit N]` to pull a user's
most recent chronological tweets via the UserTweets GraphQL endpoint.
Long posts resolve via note_tweet, pinned entries are skipped, and
retweets are flagged. QueryIds resolve dynamically through
`resolveTwitterQueryId` with hardcoded fallbacks.

* fix(twitter): expose retweet flag in tweets output
2026-04-21 00:26:57 +08:00
jakevin 51e3ac4708 docs: add CHANGELOG entry for 1.7.5 (#1097)
Mirror GitHub Release notes for v1.7.5 (PR #1096, tag a0b2155).
2026-04-20 23:00:35 +08:00
GanFanNewOrder 4d25b2b99e fix(jianyu): block inaccessible detail links and verification pages (#918)
* fix(jianyu): filter blocked detail links

* fix(jianyu): keep recency filter opt-in

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-20 23:00:10 +08:00
jakevin a0b2155510 chore: bump version to 1.7.5, extension to 1.0.1 (#1096)
Release / release (push) Has been cancelled
2026-04-20 22:53:08 +08:00
jakevin afa5e6046c refactor: consolidate 6 skills into 3, remove mechanical commands (#1094)
* refactor: consolidate 6 skills into 3, remove mechanical commands

Replaces opencli-oneshot / opencli-explorer / opencli-browser /
opencli-usage with a single opencli-adapter-author skill that takes
the AI agent end-to-end: site recon, API discovery, field decoding,
adapter coding, and `opencli browser verify`.

Removes the mechanical commands (`explore`, `synthesize`, `generate`,
`cascade`, `record`) and their src/tests — they were codegen scaffolding
meant for agents, which the new skill handles more flexibly via
`opencli browser` primitives.

Skill highlights:
- Top-level decision tree + 12-step runbook
- 5 site patterns (SPA / SSR / JSONP / Token / Streaming)
- 5-layer API discovery (network → initial state → bundle → token → interceptor)
- Field decode playbook (self-explanatory → codes → sort-key comparison)
- Output design guide (columns, types, order, ≤15 per adapter)
- Two-layer site memory: in-repo seeds for eastmoney/xueqiu/bilibili/tonghuashun
  plus local `~/.opencli/sites/<site>/` runtime workspace

Kept skills: opencli-autofix (now points to adapter-author for rewrites),
smart-search. Kept primitives: `browser *`, `doctor`, `list`, `validate`,
`verify`, `<site> <cmd>`, `plugin *`, `completion`.

No backward compatibility shims. Full test suite (1605 tests) passes.

* review fixes: honest coverage, hard memory-hit path, typo, stale docs

- site-memory hit path no longer jumps to writing adapter; forces Step 5
  endpoint re-verification + Step 7 field check, and 30-day expiry
- site-memory.md now specifies exact schemas for endpoints.json /
  field-map.json / notes.md / fixtures + write-back timing rules
- coverage-matrix.md marks unverified patterns as 🟡 with an evidence
  section citing coingecko dry run + PR #1091 eastmoney + bilibili
- eastmoney seed typo: resolveSecids -> resolveSecid (and splitSymbols)
- docs/developer/ai-workflow.md rewritten to teach the adapter-author
  skill + opencli browser * primitives (dropped generate/synthesize/
  cascade/explore references)
- ts-adapter.md, getting-started.md, CHANGELOG.md:87 updated to point
  at opencli-adapter-author

* fix(ci): resync package-lock + drop stale built-in list reference

- Regenerate package-lock.json to restore @emnapi/core + @emnapi/runtime
  entries that got dropped during the rebase — `npm ci` was failing on all
  CI jobs (build / audit / docs-build / bun-test / unit-test)
- docs/guide/getting-started.md: built-in list dropped `explore`, now
  reads (list, validate, verify, browser, doctor, plugin...)

* fix(ci): restore package-lock.json from main (unrelated lockfile churn)
2026-04-20 22:00:17 +08:00
jakevin 0f903f544b chore(clis/eastmoney): mirror 13 adapters + _secid helper as Phase A oracle (#1091)
Mirror the remaining 13 read-oriented adapters and the shared _secid.js
helper from the author's local workspace into the repo, so that
clis/eastmoney/ becomes the full Phase A codegen regression oracle
described in OpenCLI Improvement Spec v1.1 §B.10.

Total repo oracle after this PR: 14 adapters under clis/eastmoney/
(hot-rank.js already exists; this PR adds the other 13) plus the
_secid.js normalize helper.

Covers the two schema-expressiveness gaps discovered during prep:
- CSV row_format: kline.js decodes "YYYYMMDD,open,close,..." strings
- :row_index source: convertible.js derives rank = i + 1

_secid.js is the canonical example of the v1.1 §B.7 helper contract
(pure normalize/derive function, serializable I/O, no env/fs/net/session
access, does not drive pagination/retry/fallback).

This PR is oracle-only, carries no framework changes. Phase A framework
PR depends on this merging first so the codegen diff target is stable.

Refs: task #177 / spec v1.1 §B.10
2026-04-20 18:42:46 +08:00
Benjamin Liu 163974652e feat(deepseek): add DeepSeek browser adapter with ask, new, status, read, history (#1088)
Closes #548
2026-04-20 16:27:30 +08:00
Benjamin Liu be2c1cd452 feat(download): show saved file path in web read and weixin download output (#1042)
* feat(download): show saved file path in web read and weixin download output

Closes #1038

* test(download): cover saved article path

---------

Co-authored-by: Benjamin Liu <beneecs@Benjamins-Mac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-20 13:25:02 +08:00
jakevin 1ecbf7a17c Revert "feat(output): optimize table formatting with width capping and key/value layout (#1081)" (#1085)
This reverts commit 3bbea014e5.
2026-04-19 21:47:35 +08:00
Benjamin Liu 3bbea014e5 feat(output): optimize table formatting with width capping and key/value layout (#1081)
* feat(output): optimize table formatting with column width capping and key/value layout

Closes #1017

* test(output): cover key-value and width-capped tables

* fix(output): truncate capped table cells

* test(output): make table assertions color-safe

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-19 21:39:29 +08:00
Eagle 6b2f4cdc31 feat(browser): add cross-origin iframe support via CDP execution contexts (#1084)
* feat(browser): add cross-origin iframe support via CDP execution contexts

Enable interaction with cross-origin iframes through CDP's execution
context mechanism, without requiring content scripts or all_frames.

- Track frame execution contexts via Runtime.executionContextCreated events
- Add 'frames' action to list all child frames (including cross-origin)
- Support frameIndex in 'exec' action to evaluate JS in specific frames
- Add Page.frames() and Page.evaluateInFrame() APIs for CLI consumers
- Tag cross-origin iframes with [F0]/[F1] indices in DOM snapshots
- Add Page.getFrameTree to CDP allowlist

Closes #1077

Change-Id: Id03361ddb616912dff3bfa8e59e8b68716de590b

* fix(browser): align cross-origin iframe routing contract

* fix(browser): unify iframe frame-index routing

---------

Co-authored-by: xuezhangying <xuezhangying@bytedance.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-19 20:41:59 +08:00
zhengyu fbdb1b24dc fix(browser): harden multi-tab routing and target isolation (#1072)
* fix(browser): harden multi-tab routing and target isolation

- make daemon command ids collision-resistant and retry duplicate pending ids\n- add validated tab list/new/select/close flows with persisted default targets\n- keep untargeted browser commands on the default tab unless tab select changes it\n- document tab targeting and add unit, extension, and e2e coverage for concurrent multi-tab execution

* fix(browser): keep default tab stable after tab new

* fix(browser): close remaining tab routing gates

* docs(browser): align target id wording

* docs(browser): refine target id examples

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-19 18:48:45 +08:00
jakevin fe59ee990c docs: rewrite browser sections — browser is for AI Agents, not manual use (#1080)
From first principles, `opencli browser` commands exist for AI Agents to
operate websites through the browser skill. Reframe both READMEs to reflect
this: show users how to install the skill into their AI agent and describe
tasks in natural language, rather than listing raw CLI commands.
2026-04-19 02:19:10 +08:00
Ocean bb21e7e831 feat(twitter): GraphQL-based lists + list-tweets + list-add/remove (#1076)
* feat(twitter): rewrite lists via GraphQL + add list-tweets

The DOM-scraping / detail-click approach in PR #1053 remained fragile
against X's frequent overview-page rendering changes and slow (N+1 page
loads per list). Rewrite `twitter lists` to call
`ListsManagementPageTimeline` GraphQL directly — one request returns all
owned + subscribed lists with id/name/member_count/subscriber_count/mode.

Also add `twitter list-tweets <listId>` for pulling the tweet stream from
a list, completing the read-side chain (lists → pick an id → list-tweets).

- lists: drop positional `user` arg (GraphQL returns only logged-in
  user's lists), add `id` column, change followers to exact integer from
  subscriber_count.
- list-tweets: same GraphQL pattern as bookmarks/likes (BEARER + ct0 +
  dynamic queryId with static fallback + cursor pagination).
- Delete obsolete lists-parser.js and lists.d.ts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(twitter): add list-add / list-remove with Save-button commit

Two new commands to toggle list membership. X's list dialog uses a
"click-to-stage, Save-to-commit" pattern — clicking a row only updates
optimistic UI; the actual POST fires when the user clicks the top-right
"Save" button. Pressing ESC or the close-X silently cancels the change.

Implementation:
- Resolve listId → name via ListsManagementPageTimeline GraphQL, so we
  match the dialog row by name (dialog rows have no data-testid listId).
- Open profile page → DOM click "…" menu → "Add/remove from Lists".
- Scroll dialog to locate target row (virtualized list).
- page.nativeClick on row — trusted CDP Input.dispatchMouseEvent fires
  React's onclick, flips aria-checked (.click() alone does not suffice;
  X ignores non-trusted events for list mutations).
- page.nativeClick on the Save button — commits to server.
- Verify by re-fetching ListsManagementPageTimeline and diffing
  member_count: success only if N→N±1. No silent successes.

This fixes the pattern where batch `list-add` calls returned success for
every user but committed zero to the server (optimistic UI lied).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix: stabilize twitter list manifest and query ids

* docs: add twitter list command discoverability

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-19 01:42:42 +08:00
Pandas886 b65df6b1e2 fix(zsxq): separate content field from title, remove title truncation (#1079)
* fix(zsxq): separate content field from title, remove title truncation

- Split getTopicText to return only title, add getTopicContent for body text
- Remove .slice(0, 120) that was truncating titles
- content field now contains full body text instead of duplicating title

* fix(zsxq): preserve title fallback for body-only topics

---------

Co-authored-by: huzekang <huzekang@opencode.ai>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-19 01:42:31 +08:00
Mu 0cd63562f2 feat: migrate academic and policy adapters (#243)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-17 15:46:32 +08:00
jakevin 02d637f3d2 fix(e2e): accept CONFIG_ERROR (exit 78) in xiaoyuzhou E2E guard (#1066)
PR #1059 migrated xiaoyuzhou from SSR scraping to authenticated API.
The E2E tests run without credentials, producing exit code 78
(CONFIG_ERROR). The existing `isExpectedChineseSiteRestriction` guard
only caught FETCH_ERROR, PARSE_ERROR, and NOT_FOUND — not config
errors from missing auth credentials.
2026-04-17 12:02:49 +08:00
jakevin 44d87879d8 refactor: clean up design debt — deprecated APIs, duplicated validation, dead plugin wrappers (#1065)
Three improvements from the design debt audit:

1. Remove deprecated `tabId` field and `getActiveTabId()` method
   - Delete `tabId` from DaemonCommand (daemon-client.ts) and Command (protocol.ts)
   - Delete `getActiveTabId()` from IPage interface (types.ts) and Page class (page.ts)
   - Update extension resolveCommandTabId() to remove legacy fallback
   - Update handleTabs select case to remove tabId check
   - The tab→page migration is now complete

2. Unify argument validation into single code path
   - Remove `normalizeArgValue()` from commanderAdapter.ts
   - Commander adapter now passes raw values to prepareCommandArgs()
   - All coercion (bool, int, number) and validation (required, choices)
     happens once in coerceAndValidateArgs() in execution.ts
   - Eliminates duplicated boolean normalization

3. Remove dead plugin filesystem wrappers
   - Delete `promoteDir()` — never called in production code
   - Delete `replaceDir()` — thin wrapper over beginReplaceDir, never called
   - Remove corresponding test-only exports and tests
   - Rename PromoteDirFsOps → ReplaceDirFsOps to match remaining usage
   - Transaction infrastructure (runTransaction, beginReplaceDir,
     beginReplaceSymlink) retained — used by publishStandalonePlugin
     and publishMonorepoPlugins for atomic multi-step operations
2026-04-17 10:52:57 +08:00
jakevin cb9521d52d fix(extension): per-workspace idle timeout for browser sessions (#1064)
* fix(extension): per-workspace idle timeout for browser sessions (#1058)

The global 30s WINDOW_IDLE_TIMEOUT was too aggressive for interactive
`opencli browser` commands where users type manually between invocations.

- browser:*/operate:* workspaces now default to 10 min idle timeout
- Adapter workspaces keep the existing 30s timeout
- Support custom timeout via OPENCLI_BROWSER_TIMEOUT env var (seconds)
  or command-level idleTimeout parameter
- Surface sessionExpired warning when a new window is created after
  the previous session timed out
- Fix stale comment (said 120s, actual was 30s)

Closes #1058

* fix: resolve sessionExpired double-delete race and timeout override lifecycle

Addresses @codex-coder review blockers:

1. sessionExpired flag was never set because getAutomationWindow()
   consumed expiredWorkspaces before handleCommand() could check it.
   Fix: use .has() in getAutomationWindow, only .delete() in handleCommand.

2. workspaceTimeoutOverrides was never cleaned up — once set, it
   persisted until extension restart. Fix: clear override on idle
   timeout expiry, explicit close-window, and borrowed-session detach.

Adds 5 tests covering:
- browser:* uses 10min timeout (not 30s)
- sessionExpired flag is set and consumed correctly
- workspaceTimeoutOverrides cleared on idle expiry
- workspaceTimeoutOverrides cleared on explicit close
- idleTimeout from command applies to workspace override

* refactor: remove sessionExpired warning per product decision

@WAWQAQ decided session-expired warning is not needed.
Remove expiredWorkspaces tracking, sessionExpired flag from protocol,
and related CLI-side warning code. Keep per-workspace timeout and
override lifecycle cleanup.

* fix: clean up workspaceTimeoutOverrides on user-initiated window close

The windows.onRemoved listener was missing workspaceTimeoutOverrides
cleanup, causing stale overrides to persist across sessions when users
manually close the automation window.
2026-04-17 10:51:39 +08:00
jakevin 025df31ce5 refactor(antigravity): keep timeout parsing local (#1063) 2026-04-17 10:11:46 +08:00
deepziyu 8a8f4a1778 fix(antigravity): implement configurable timeout and auto-reconnect for serve (#859)
* fix(antigravity): implement configurable timeout and auto-reconnect for serve

* fix(antigravity): avoid private runtime import

* docs(antigravity): document serve timeout options

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-17 10:04:08 +08:00
Kagura ab44d9f542 fix(xiaoyuzhou): migrate from broken SSR scraping to authenticated API (fixes #1023) (#1059)
* fix(xiaoyuzhou): migrate from broken SSR scraping to authenticated API (fixes #1023)

Xiaoyuzhou removed SSR rendering — /podcast/<id> and /episode/<id> pages
now return 404, breaking fetchPageProps() which scraped __NEXT_DATA__.

Migrate podcast, podcast-episodes, episode, and download commands to use
the existing authenticated API client (requestXiaoyuzhouJson) that
transcript.js already uses successfully.

Changes:
- podcast.js: use /v1/podcast/get API endpoint
- podcast-episodes.js: use /v1/podcast/listEpisode API endpoint
- episode.js: use /v1/episode/get API endpoint
- download.js: use /v1/episode/get API endpoint
- utils.js: remove unused fetchPageProps, keep format helpers
- Update all affected tests (download.test.js, utils.test.js)
- Change strategy from PUBLIC to LOCAL (requires credentials)

* fix(xiaoyuzhou): align local strategy contract

* fix(xiaoyuzhou): align local api metadata

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-16 23:15:40 +08:00
jakevin 4ebada6b99 docs: add adapter docs for eastmoney, tdx, ths (#1061)
#1025 merged hot-rank adapters for eastmoney/tdx/ths but the
accompanying docs were missing. This breaks the Doc Check CI
workflow on every PR ('--strict' mode, exits non-zero when
`scripts/check-doc-coverage.sh` finds adapters without docs),
blocking merges across the board.

Adds a doc page per adapter, registers them in the adapters
index table, and adds sidebar entries in the VitePress config.
2026-04-16 22:57:03 +08:00
AstroHan bc06d99c83 fix(xiaohongshu): detect current draft save success (#1060) 2026-04-16 22:55:29 +08:00
AstroHan 3738cd2595 fix(twitter): repair lists scraping from detail pages (#1053) 2026-04-16 14:33:18 +08:00
AstroHan 240dccd754 fix(xiaohongshu): verify title input sticks on publish (#1050) 2026-04-16 14:30:37 +08:00
Cosmostima 44b4107f36 feat(nowcoder): add 牛客网 adapter with 16 commands (#1036)
* feat(nowcoder): add 牛客网 adapter with 16 commands

Add adapters for Nowcoder (牛客网), China's leading tech job-seeking
and interview preparation community.

- 7 Public commands: hot, trending, topics, recommend, creators, companies, jobs
- 9 Cookie commands: search, suggest, experience, referral, salary, papers, practice, notifications, detail
- All post-list commands include id field for drill-down to detail
- Documentation: adapter page, index table, sidebar entry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(nowcoder): register adapter and document usage

---------

Co-authored-by: tima <tima@cosmos-macmini.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-15 22:51:55 +08:00
jakevin 3076c12d6c chore: bump version to 1.7.4 (#1045)
Release / release (push) Has been cancelled
2026-04-15 15:50:30 +08:00
Howard 44147e54c1 feat(youtube): add feed, history, watch-later, subscriptions, playlist, like, unlike, subscribe, unsubscribe (#1029)
* feat(youtube): add feed, history, watch-later, subscriptions, playlist, like, unlike, subscribe, unsubscribe

* fix(youtube): normalize subscriptions channel fields

* docs(skills): add youtube command coverage

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-15 12:43:27 +08:00
槑囿脑袋 677e37b7a4 feat(xiaoyuzhou): add episode download and transcript support (#1031)
* feat(xiaoyuzhou): add episode audio download

* feat(xiaoyuzhou): add transcript download support

* docs(xiaoyuzhou): clarify credential file requirement

* fix(xiaoyuzhou): remove env credential fallback
2026-04-15 12:35:27 +08:00
Harvey Yue d48c71b993 feat(binance): depth shows both bids and asks (#1019)
* feat(binance): depth shows both bids and asks

* test(pipeline): cover root data access after inline select

* fix(binance): preserve map select context and register manifest entries

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-15 12:30:31 +08:00
DavidDuang 6fbeda951e feat: add hot stock ranking adapters for eastmoney, tdx, ths (#1025)
* feat: add hot stock ranking adapters for eastmoney, tdx, ths

Add three new site adapters for Chinese stock hot rankings:
- eastmoney/hot-rank: 东方财富热股榜
- tdx/hot-rank: 通达信热搜榜
- ths/hot-rank: 同花顺热股榜

All use Strategy.COOKIE browser mode with page.evaluate() DOM scraping.
Each includes co-located tests (13 tests total, all passing).

* fix(tdx,ths): add symbol validation and deduplication in evaluate()

Add seen Set for deduplication and skip entries with empty symbol/name,
matching the pattern already used in eastmoney/hot-rank.js.

* fix: refine hot-rank selectors based on browser inspection

- eastmoney: use table.rank_table tbody tr with td index-based extraction,
  fix name from a[title] to avoid post content contamination
- tdx: use div.top-cell[data-code] data attributes for reliable extraction,
  add tags column from div.tips-item.gnbk
- ths: use card-based layout selectors, remove price column (not in UI),
  extract tags from div.tag.PFSC-R

* fix(hot-rank): align tdx and ths columns with actual output

* fix: register hot stock ranking adapters

---------

Co-authored-by: dengjingren <dengjingren@cn.wilmar-intl.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-15 12:24:41 +08:00
jakevin 9bcdaaa0be fix(external): use safe npm install for dws (#1033) 2026-04-15 12:12:13 +08:00
zhengyu db70a3aaf3 fix(deamon&extension): preserve network capture and surface extension mismatch diagnostics (#1030)
* fix: preserve network capture and surface extension mismatch diagnostics

Older Browser Bridge installs can still connect to the daemon while
missing two capabilities we now rely on: the network-capture actions
and the extension version handshake. That created three user-facing
failure modes with real impact:

1. `opencli explore ...` crashed with `Unknown action: network-capture-start`
   against an old extension, so exploration stopped before any site
   analysis finished.
2. `opencli doctor` and `opencli daemon status` could show a healthy
   connection even when the extension never reported a version, which
   hid the compatibility problem and sent users toward the wrong fix.
3. After reloading a new extension, `explore` could still report
   `Endpoints: 0 total, 0 API` because `handleNavigate()` detached the
   debugger before top-level navigation and cleared the active network
   capture state right before the page load we needed to observe.

Fix this in two layers:

- Teach `Page` to treat unsupported `network-capture-*` actions as an
  old-extension compatibility case. It now warns once, memoizes the
  unsupported state, and returns empty capture data instead of throwing.
- Teach `doctor` and `daemon status` to treat "connected but version
  unknown" as a warning instead of a healthy state, so version-handshake
  failures are visible immediately.
- Preserve the debugger attachment while network capture is armed, so
  the initial navigation keeps the capture state alive and the extension
  can record requests from the first page load.

Before:

- `opencli explore ...` -> `Error: Unknown action: network-capture-start`
- `opencli doctor` -> `[OK] Extension: connected` / `Everything looks good!`
- `opencli daemon status` -> `Extension: connected` even when the
  extension version was missing
- `opencli explore ...` after reloading the extension -> `Endpoints: 0 total, 0 API`

After:

- `opencli explore ...` on an old extension -> warns once and continues
- `opencli doctor` -> `[WARN] Extension: connected (version unknown)`
- `opencli daemon status` -> `Extension: connected (version unknown)`
- `opencli explore ...` on the reloaded extension keeps network capture
  armed across navigation instead of clearing it before the page load

* fix: reset network capture flags on closeWindow()

Prevents stale _networkCaptureUnsupported flag from persisting across
sessions when the user reinstalls or reloads the extension mid-session.

* fix: startNetworkCapture returns boolean to prevent false-positive on old extensions

When the extension doesn't support network-capture-*, startNetworkCapture()
now returns false instead of silently resolving. This ensures browser open/
network correctly falls back to the JS interceptor on old extensions.

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-15 12:07:31 +08:00
jakevin 0040081f2b fix: auto-restart stale daemon and improve connection error messages (#1028)
* fix: auto-restart stale daemon and improve connection error messages

When daemon is running but extension never connected (stale daemon started
before extension was installed), the CLI now auto-restarts the daemon to
give the extension a fresh WebSocket endpoint, instead of just waiting
and then telling the user to install the extension.

Also improves error messages across cli.ts, bridge.ts, and doctor.ts to
suggest "opencli daemon stop && opencli doctor" as the quick fix, since
that's what actually resolves the issue.

* fix: version-aware stale daemon detection and improved error messages

- Daemon /status now includes `daemonVersion` field
- bridge.ts: when daemon is running but extension not connected, checks
  daemonVersion vs CLI version. Only auto-restarts if version mismatch
  (stale daemon from older CLI). Same-version daemon shows improved error
  message with "opencli daemon stop && opencli doctor" hint.
- doctor.ts: explicitly identifies stale daemon (version mismatch) in
  diagnostics report, shows daemon version in status line
- cli.ts: error message changed to suggest "opencli daemon stop && opencli doctor"

* fix: treat missing daemonVersion as stale, verify shutdown before respawn

- Missing daemonVersion (pre-version daemon) is now treated as stale,
  covering the most common user scenario (old daemon without version field)
- After requestDaemonShutdown(), poll until daemon actually stops (port
  released) before spawning new one, with 3s timeout
- If shutdown request fails, log warning instead of silently proceeding
- doctor.ts also treats missing daemonVersion as stale with clear message

* fix: fail explicitly when stale daemon replacement fails

- If shutdown request fails or port isn't released within 3s, throw
  'Stale daemon could not be replaced' instead of blindly spawning on
  an occupied port
- Add tests for all three stale-daemon branches: same-version (no
  restart), missing daemonVersion (stale), mismatched version (stale)

* fix: use type-based error dispatch in browserAction instead of string matching

browserAction() now checks `instanceof BrowserConnectError` first and
renders both message and hint, instead of string-matching on message
content. This ensures stale daemon errors ("Stale daemon could not be
replaced") surface the actionable hint to the user.
2026-04-15 11:33:26 +08:00
AstroHan 16d597cfce fix(doubao): harden ask response parsing (#933)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-14 20:43:12 +08:00
flizzywine ba3a674d7b feat(grok): add image command for grok.com image generation (#906)
* feat(grok): add image command for grok.com image generation

Add `opencli grok image <prompt>` which submits a prompt via the existing
grok.com browser session and returns the generated image URLs from the
latest assistant bubble.

Because assets.grok.com URLs are gated by Cloudflare and cannot be
downloaded with a plain HTTP client, the --out flag triggers an in-page
fetch(credentials: 'include') so the browser session's cookies and
referer are attached, then writes the decoded blob to disk.

Flags:
- --new       start a fresh chat before sending
- --timeout   max seconds to wait for the image (default 240)
- --count     minimum number of images to wait for before returning
- --out       directory to save downloaded images

Ships with unit tests for the helpers (isOnGrok, normalizeBooleanFlag,
dedupeBySrc, imagesSignature, extFromContentType, buildFilename).

* fix(grok): harden image composer and bubble detection

* fix(grok): harden image flow and docs

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-14 20:42:43 +08:00
warkcod 0e38fd8c37 Feat/douban book subject (#993)
* chore: ignore local worktrees

* feat(douban): support book subject details
2026-04-14 20:41:14 +08:00
AstroHan cd48917a39 fix(xiaohongshu): require signed note URLs (#996)
* fix(xiaohongshu): require signed note urls

* chore: drop generated manifest from pr
2026-04-14 20:40:58 +08:00
CissiBot 45d6f5b09f feat(uiverse): add Uiverse code and preview adapters (#1000)
* feat(uiverse): add code and preview adapters

* fix(manifest): register uiverse commands

* docs(uiverse): add usage examples

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-14 20:38:38 +08:00
XavierCai 3ebc46f978 feat(bilibili): favorite command supports specifying fid (#1013)
* feat(bilibili): favorite command supports specifying fid

* fix(bilibili): sync favorite help and docs contract

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-14 20:37:54 +08:00
Benjamin Liu 6e29845dc3 fix(plugin): install monorepo sub-plugin dependencies when not hoisted (#1007)
Closes #722
2026-04-14 17:25:50 +08:00
mademing68092354-glitch 88bce1becf fix(chatgpt): support Chinese UI for model selector (#1006)
When ChatGPT macOS app is set to Chinese language, the "Options"
button label becomes "选项". This change checks for both English
and Chinese labels to find the button.

Co-authored-by: mad <mademing@maddeMac-mini.local>
Co-authored-by: Claude <noreply@anthropic.com>
2026-04-14 17:21:54 +08:00
jakevin ca68f3999b feat: Ref-Backed Locator for browser actions (#1016)
* feat: implement Ref-Backed Locator for browser actions

Introduces a unified target resolution system with fingerprint
verification and structured error diagnostics.

Snapshot phase:
- Each interactive element now gets a fingerprint (tag, role, text,
  ariaLabel, id, testId) stored in window.__opencli_ref_identity
- Zero overhead: metadata is already available during DOM walk

Resolution phase (new target-resolver.ts):
- Numeric input → ref path with fingerprint verification
- CSS-like input → querySelectorAll with uniqueness check
- No more silent first-match: ambiguous selectors are rejected

Error model (new target-errors.ts):
- stale_ref: element identity changed since snapshot
- ambiguous: CSS selector matched multiple elements (with candidates)
- not_found: element not in DOM or invalid input
- All errors include actionable hints for AI agents

base-page.ts:
- click() and typeText() now use two-phase resolve-then-act
- Existing CDP fallback for click preserved

* feat: migrate scrollTo to unified resolver pipeline

scrollTo now uses the same two-phase resolve-then-act pattern as
click and typeText, getting fingerprint verification and structured
error diagnostics (stale_ref/ambiguous/not_found) for free.

* fix: address review — stronger fingerprint verification & surface TargetError in CLI

1. Fingerprint verification now uses the full identity vector (tag, id,
   testId, ariaLabel, role, text) instead of just tag/role/text. Strong
   identifiers (id, testId) are decisive; remaining signals use majority
   voting. Fixes false negatives where same-tag elements swapped.

2. browserAction() now renders TargetError with code, hint, and
   candidates list instead of just the message string.

* fix: migrate get/select/type-autocomplete to unified resolver

- browser get text/value/attributes now resolve via resolveTargetJs
  instead of raw querySelector, getting fingerprint verification and
  structured errors for free
- browser select uses selectResolvedJs on __resolved element
- type command's autocomplete detection uses isAutocompleteResolvedJs
  on the already-resolved element
- Fix empty-string text prefix match: fp.text="Login" + text="" no
  longer falsely passes fingerprint check
2026-04-14 16:57:05 +08:00
jakevin 847c8317b6 fix(twitter): register lists command in manifest (#1011) 2026-04-14 10:37:34 +08:00
forvendettaw 741bcf9b6e Add bookmark_count field to bookmarks command (#1010)
* Add bookmark_count field to bookmarks command

Extract bookmark_count from legacy object in Twitter GraphQL
Bookmarks response. Add to returned tweet object and table columns.

* fix(manifest): sync twitter bookmarks columns

---------

Co-authored-by: Hermes Agent <hermes@lei.zong>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-14 10:27:20 +08:00
dependabot[bot] 44388d21fc chore(ci): bump softprops/action-gh-release from 2.6.1 to 3.0.0 (#1002)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2.6.1 to 3.0.0.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2.6.1...v3.0.0)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: 3.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 10:18:45 +08:00
dependabot[bot] 745ce459d1 chore(deps): bump undici from 8.0.2 to 8.1.0 (#1003)
Bumps [undici](https://github.com/nodejs/undici) from 8.0.2 to 8.1.0.
- [Release notes](https://github.com/nodejs/undici/releases)
- [Commits](https://github.com/nodejs/undici/compare/v8.0.2...v8.1.0)

---
updated-dependencies:
- dependency-name: undici
  dependency-version: 8.1.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 10:18:32 +08:00
dependabot[bot] a5cd0dc307 chore(deps): bump @types/node from 25.5.2 to 25.6.0 (#1004)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.5.2 to 25.6.0.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.6.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 10:18:24 +08:00
dependabot[bot] beabed4bad chore(deps): bump vitest from 4.1.2 to 4.1.4 (#1005)
Bumps [vitest](https://github.com/vitest-dev/vitest/tree/HEAD/packages/vitest) from 4.1.2 to 4.1.4.
- [Release notes](https://github.com/vitest-dev/vitest/releases)
- [Commits](https://github.com/vitest-dev/vitest/commits/v4.1.4/packages/vitest)

---
updated-dependencies:
- dependency-name: vitest
  dependency-version: 4.1.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-14 10:18:15 +08:00
jakevin fa208ec761 docs: sync Highlights cleanup across all doc surfaces (#1009)
- docs/index.md: update feature cards to match README Highlights
- docs/zh/index.md: sync Chinese feature cards
- docs/guide/getting-started.md: align Highlights section
- README.zh-CN.md: rename "为什么是 OpenCLI" to "亮点", align with EN
2026-04-14 09:24:33 +08:00
jakevin 56a727cc04 docs: remove empty Why OpenCLI section and clean up Highlights (#1008)
- Remove the empty "Why OpenCLI" heading
- Rename "CLI All Electron" to "Desktop App Control" for clarity
- Remove "Anti-detection built-in" (exposes implementation details)
- Remove "Broad coverage" (duplicates intro and Built-in Commands table)
- Merge "Self-healing setup" and "Dynamic Loader" out (minor features)
- Rename "External CLI Hub" to "CLI Hub" for brevity
2026-04-14 09:19:08 +08:00
jakevin feedaf93b4 fix: remove duplicate extension zip from releases (#1001)
* fix: remove duplicate extension zip from releases

The release and build-extension workflows were creating both
opencli-extension.zip and opencli-extension-v{version}.zip (identical
content), causing both to be uploaded. Keep only the versioned filename.

* docs: update extension zip filename to versioned format

Update all references from opencli-extension.zip to
opencli-extension-v{version}.zip to match the workflow change.
2026-04-13 23:47:58 +08:00
jakevin 9ebb921c89 chore: prune legacy config switches (#998) 2026-04-13 23:28:30 +08:00
jakevin 9ac2e1d8ef chore: bump version to 1.7.3 (#997)
Release / release (push) Has been cancelled
2026-04-13 23:12:50 +08:00
SherlockSalvatore 2aee4caa10 feat(mubu): add Mubu adapter with 5 commands (#964)
* feat(mubu): add mubu (mubu.com) adapter with 5 commands

Commands: doc, docs, notes, recent, search.

- Uses COOKIE strategy; API calls via in-page XHR with Jwt-Token
  from localStorage (matches the web app's own mechanism).
- Renders node trees to Markdown (default) or plain text;
  supports tables, tasks, images, emoji, mentions, strikethrough,
  underline, and nested structures.
- notes supports flexible time ranges: single day, month, year,
  or custom --from/--to spans, plus a --list overview mode.
- search returns full-text matches with hit count and snippets
  for both folders and documents.

* fix(manifest): register mubu commands in runtime manifest

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-13 16:50:32 +08:00
jakevin 323fe8857c refactor: unify OPENCLI_VERBOSE and DEBUG=opencli (#991)
* refactor: unify OPENCLI_VERBOSE and DEBUG=opencli into one mechanism

Three debug output levels (verbose/debug/diagnostic) was redundant.
Merge DEBUG=opencli into OPENCLI_VERBOSE so `-v` flag controls all
verbose/debug output through a single mechanism.

- log.verbose() now checks both OPENCLI_VERBOSE and DEBUG=opencli
- log.debug() becomes an alias for log.verbose() (backward compat)
- boss/utils.js verbose helper simplified to check OPENCLI_VERBOSE only
- DEBUG=opencli still works as fallback (no breaking change)

* fix(boss): preserve debug fallback for verbose logs
2026-04-13 16:48:06 +08:00
jakevin ff6563d12a Fix automation window not closing on command failure (#992)
The error path in executeCommand did not call page.closeWindow(),
leaving the automation window open until the extension's idle timer
fires. On Windows, MV3 service worker suspension makes this timer
unreliable, causing windows to linger indefinitely.

Now closeWindow is called after diagnostic collection but before
rethrowing, ensuring the window is closed on both success and failure.
2026-04-13 16:47:47 +08:00
jakevin c42b040af4 Rename chatgpt adapters: desktop → chatgpt-app, web → chatgpt (#989)
* Rename chatgpt adapters: desktop → chatgpt-app, web → chatgpt

Aligns with existing `-app` suffix convention (discord-app, doubao-app):
- clis/chatgpt/ (desktop, AppleScript) → clis/chatgpt-app/
- clis/chatgptweb/ (browser, chatgpt.com) → clis/chatgpt/
- electron-apps.ts: chatgpt → chatgpt-app
- Updated all docs and README references

Closes #283

* Fix review findings: update cli-manifest.json and skill docs

- cli-manifest.json: update site/modulePath/sourceFile from chatgpt to chatgpt-app
- skills/opencli-usage/desktop.md: update commands from chatgpt to chatgpt-app
2026-04-13 14:33:32 +08:00
jakevin 79a15e8353 Remove unused OPENCLI_SKIP_FETCH env var (#987)
The adapter sync already has version caching (skips if same version)
and makes no network requests, so this opt-out flag adds no value.
2026-04-13 14:09:10 +08:00
jakevin 6d769ff354 docs: document undocumented environment variables (#983)
Add missing env vars to both README and README.zh-CN:
- OPENCLI_SKIP_FETCH: skip adapter sync on global install
- OUTPUT: override output format (json/yaml/table)
- DEBUG=opencli: internal debug logging
- DEBUG_SNAPSHOT: DOM snapshot debug output
2026-04-13 14:01:46 +08:00
jakevin 988ed19223 fix: clean up stale .yaml adapter files from older versions (#953) (#986)
* fix: clean up stale .yaml adapter files from older versions (#953)

Users upgrading from v1.6.x retain .yaml adapter files in
~/.opencli/clis/ that trigger "Ignoring YAML adapter" warnings on
every run. The hash-based sync only tracks .js files, so these
legacy .yaml files are never cleaned up.

Add a cleanup step (3b) that removes .yaml/.yml files from user
adapter directories when the corresponding site exists in the
official package (i.e., the site has been migrated to .js).

* fix(fetch-adapters): narrow stale yaml cleanup
2026-04-13 13:17:27 +08:00
jakevin 51bc48ec61 feat: decouple extension version from CLI version (#985)
* feat: decouple extension version from CLI version

Extension and CLI had tightly coupled version numbers (both 1.7.2),
requiring manual sync across 3 files on every release. This decouples
them so each can release independently.

Changes:
- Extension version reset to 1.0.0 with independent versioning
- Extension sends compatRange (e.g. ">=1.7.0") in hello message
  so doctor can check CLI/extension compatibility
- Daemon stores and exposes extensionCompatRange via /status
- Doctor uses compatRange for compatibility checks (falls back to
  major-version check for older extensions without compatRange)
- Doctor shows extension update availability from cached GitHub
  Releases data
- release.yml always builds and attaches extension zip to every
  CLI release, so users always find both in the same release page
- build-extension.yml triggers on ext-v* tags (not v*) to avoid
  duplicate builds

* fix: version extension release assets
2026-04-13 12:43:34 +08:00
jakevin 72bc86cf41 fix: code audit round 2 — safety, hot-reload, error diagnostics (#982)
* fix: code audit round 2 — pruneEmptyDirs, evaluateWithArgs, hot-reload, error cause chain

1. pruneEmptyDirs: use path.relative() instead of startsWith() to prevent
   false boundary matches on overlapping directory names
2. evaluateWithArgs: add safe evaluate method that auto-serializes args via
   JSON.stringify, preventing injection by design
3. Hot-reload: detect mtime changes on user adapter files in daemon mode,
   invalidate module cache so edits take effect without restart
4. toEnvelope: preserve error cause chain in verbose mode for better
   production debugging

* fix: address review feedback on code audit round 2

- pruneEmptyDirs: resolve() paths before relative() check
- evaluateWithArgs: validate keys are valid JS identifiers
- hot-reload: only bust ESM cache on reload, not first load
- toEnvelope: move cause serialization into toEnvelope itself
  so all consumers (AI agents, MCP tools) get cause chain
2026-04-13 09:36:53 +08:00
jakevin ffb61c51ea fix: address code audit findings (C1-C4, I1, I4, I6) (#981)
* fix: address code audit findings (C1-C4, I1, I4, I6)

Security:
- C1: Fix page.evaluate injection in browser type/select commands and
  6 adapter files by using JSON.stringify for user input interpolation
- C2: Close WebSocket on CDP connect timeout to prevent resource leak
- C3: Reject CDP connect promise on Page.enable failure instead of
  silently swallowing the error

Reliability:
- C4: Guard against corrupted adapter-manifest.json hashes to prevent
  false-positive override deletion
- I1: Throw on pre-navigation failure instead of warn-and-continue
- I4: Use Map<string, Promise<void>> for lazy module loading to prevent
  concurrent double-imports of the same adapter

Performance:
- I6: Replace O(n) registry alias cleanup with O(k) direct deletion

* fix: address self-review findings on PR #981

- C1: add quotes around CSS selector attribute values in browser
  type/select to match other commands (get text/value/attributes)
- C2: clear this._ws in timeout handler to prevent race with open event
- C4: refine corruption guard — treat null/undefined hashes as empty,
  only skip sync for truly invalid types (string, number, array)
2026-04-13 09:24:01 +08:00
AstroHan 5dcbf92a59 fix(douban): classify tv search results correctly (#979) 2026-04-13 08:41:19 +08:00
AstroHan 83dce2430e fix(xiaohongshu): harden anti-detection flows (#980) 2026-04-13 08:40:48 +08:00
Tony Simons 4d1fa8a6e2 feat(clis/chatgptweb): add ChatGPT web image generation command (#973)
* feat(clis/chatgptweb): add ChatGPT web image generation command

Add `opencli chatgptweb image` command that generates images using
ChatGPT web (GPT-4o image generation) and saves them locally.

Features:
- Navigates to chatgpt.com/new with full page reload to ensure clean state
- Uses Playwright's page.type() for reliable text input in TipTap editor
- Closes sidebar if open (covers the chat composer on some layouts)
- Polls for response completion (handles thinking/throttling states)
- Extracts generated images from DOM (backend-api/estuary/content URLs)
- Downloads and saves as PNG/JPEG files to user-specified directory
- Supports --op for output directory and --sd to skip download

Files:
- clis/chatgptweb/image.js: CLI command definition
- clis/chatgptweb/utils.js: DOM helpers, send/wait/export functions

Works cross-platform (Linux/macOS/Windows) via OpenCLI browser automation.

* fix(chatgptweb): stabilize image generation flow

* docs(chatgptweb): add browser adapter guide

---------

Co-authored-by: Tony Simons <tony@tonysimons.dev>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-12 21:24:03 +08:00
Harvey Yue aa47de726d feat(bilibili): add feed-detail and enhance feed command (#974)
* feat(bilibili): add feed-detail and enhance feed command

* docs: add binance adapter documentation

* docs: add feed-detail command to bilibili docs

* docs: sync bilibili adapter contract for feed-detail

* docs: add ke adapter page for doc coverage

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-12 21:19:12 +08:00
runzhliu 232b6828be feat(ke): add Beike (贝壳找房) adapter with ershoufang, xiaoqu, zufang, chengjiao commands (#975)
Support browsing second-hand houses, neighborhoods, rentals, and
transaction records on ke.com with city/district/price filtering.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 16:21:59 +08:00
Ivan Xia 20e024a001 feat(maimai): add talent search with multi-dimensional filters (#977)
* feat(maimai): add talent search with multi-dimensional filters

Add maimai.cn talent search adapter with support for:
- Keyword search (query)
- Company filtering (multiple companies supported)
- School filtering (with 985/211 options)
- Location filtering (province/city)
- Work experience and education level filters
- Industry and position filters
- Direct chat availability
- Sort by relevance, activity, work years, or education

Features:
- Reuses Chrome login session for authentication
- Extracts candidate info: name, job title, company, work history
- Shows work years, education, age, active status
- Displays skill tags and mutual friends count

* fix docs and strategy for maimai adapter

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-12 16:16:55 +08:00
Alex Yang 01957bf654 feat(discord-app): add delete command to remove a message by ID (#976)
* feat(discord-app): add delete command to remove a message by ID

Adds a new `delete` command for the discord-app CLI that deletes a
message in the active channel by its snowflake ID. Uses the UI strategy
to hover the message, open the "More" menu, click "Delete Message", and
confirm the deletion dialog.

* docs: add binance adapter doc and update discord doc with delete command
2026-04-12 16:07:14 +08:00
jakevin 315cc59f8a chore: bump version to 1.7.2 (#972)
Build Chrome Extension / build (push) Has been cancelled
Release / release (push) Has been cancelled
2026-04-11 22:14:43 +08:00
jakevin ebe6be945e fix(zsxq): update topic test for group_id parameter added in #963 (#971)
The test mock was missing the evaluate call for getActiveGroupId,
which was added when #963 introduced the group_id parameter.
2026-04-11 22:12:26 +08:00
iiilin b0a019121e feat(weibo): support for-you and following feed types (#959)
* feat(weibo): support for-you and following feed types

* docs: clarify weibo feed types

---------

Co-authored-by: iiilin <19162130+iiilin@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-11 22:04:01 +08:00
Paul Zhu c63bad34b0 feat(twitter): add lists command to retrieve user lists (#958)
* feat(twitter): add lists command to retrieve user lists

Add twitter/lists command that fetches Twitter/X lists for a user.
Supports:
- Lists with member and follower counts
- Private/public mode detection
- Default to current user if no user specified
- Works for any Twitter user

* docs: add lists command to twitter commands in README

Add twitter lists command to Built-in Commands table in both
English and Chinese README files

* fix(twitter): parse lists from card DOM instead of locale-specific page text

---------

Co-authored-by: isanwenyu <isanwenyu@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-11 21:58:46 +08:00
Hoshea 1ffe12f85e fix(zsxq): accept topic_id as string in getTopicFromResponse (#963)
* fix(zsxq): accept topic_id as string in getTopicFromResponse

The ZSXQ API returns topic_id as a string, but getTopicFromResponse()
only checked for typeof === 'number', causing it to fall through and
return null. This made 'opencli zsxq topic <id>' fail with NOT_FOUND
for all valid topic IDs.

* fix(zsxq): use group-scoped topic endpoint instead of bare /v2/topics/{id}

The ZSXQ API requires topics to be fetched within their group context.
Change /v2/topics/{id} -> /v2/groups/{groupId}/topics/{id} for both
the detail and comments endpoints. Also adds optional --group_id arg.
2026-04-11 21:49:02 +08:00
jakevin 30f216b2c1 fix: include adapter tests in default npm test (#969)
* fix: include adapter tests in default npm test

`npm test` only ran unit + extension projects, so adapter tests
(clis/**/*.test.js) were never exercised by the default test command.
Add --project adapter so they run alongside unit and extension tests.

* test: include adapter project in default npm test
2026-04-11 21:33:50 +08:00
jakevin 3c088da53e refactor: smart sync adapters — hash-based diff instead of full copy (#966)
* refactor: smart sync adapters instead of full copy (#sparse-override)

Replace unconditional full-copy of all adapters to ~/.opencli/clis/ with
hash-based smart sync that only copies files whose content has changed.

Changes:
- fetch-adapters.js: use SHA-256 content hashes to skip unchanged files;
  store per-file hashes in adapter-manifest.json
- discovery.ts: simplify ensureUserAdapters() to only create the directory
  (no longer triggers full copy on first run)
- main.ts: fix fast completion to check manifest file existence instead of
  directory existence (sparse override may have empty user dir)
- cli.ts: add `opencli adapter eject/reset/status` commands for managing
  local adapter overrides
- engine.test.ts: add tests for empty user dir and ensureUserAdapters

* fix: address review blockers — site-level sync + reset --all

1. Fix `adapter reset --all`: change <site> from required to optional
   argument so --all can be used without specifying a site name.

2. Change smart sync from file-level to site-level granularity:
   if any file in a site has changed upstream, overwrite the entire
   site directory. This matches the agreed product semantics — local
   modifications to any file in a site are replaced when upstream
   updates that site.

* fix: delete old site dir before writing updated adapter files

When a site has upstream changes, delete the entire site directory
first, then write the new version. This prevents stale files from
older versions lingering in the user directory.

* fix: reset --all preserves custom sites, only removes official overrides

Blocker 3 fix: reset --all now checks BUILTIN_CLIS to identify official
sites and only deletes those, preserving user-created custom sites.

* refactor: sparse sync deletes local overrides instead of copying new versions

Changed fetch-adapters.js semantics per team agreement:
- When an official site has upstream changes, DELETE the local override
  instead of copying the new version into ~/.opencli/clis/
- Runtime automatically falls back to package baseline
- ~/.opencli/clis/ becomes a true sparse override layer

* fix: reset <site> rejects custom sites, only allows official overrides

Single-site reset now checks BUILTIN_CLIS before deleting, matching
the same protection that reset --all already has.

* fix: reset <site> allows custom sites per product decision

Per @WAWQAQ: explicit single-site reset should work on custom sites too.
Differentiate messaging: official sites say "using official baseline",
custom sites say "removed custom site".

reset --all still only removes official overrides (bulk safety).

* fix: reset --all deletes all local sites including custom per product decision

Per @WAWQAQ: --all should clear the entire local working cache,
including custom sites. Single-site reset already handles both types.
2026-04-11 21:28:40 +08:00
jakevin 00e200b0b7 migrate: move binance adapters from src/clis/ to clis/ (#967)
Binance was the only adapter left in src/clis/ after the TS→JS
migration (PR #928). Move all 11 adapters and the test file to
clis/binance/, strip TypeScript syntax from the test, and switch
the test import to the @jackwener/opencli/pipeline package export.
2026-04-11 21:21:52 +08:00
jakevin 1fd578c404 chore: bump version to 1.7.1 (#965)
Build Chrome Extension / build (push) Has been cancelled
Release / release (push) Has been cancelled
2026-04-11 20:28:23 +08:00
jakevin b0fd95279c docs: add CHANGELOG.md entry for v1.7.0 (#955)
Comprehensive release notes covering all changes since v1.6.1:
- Breaking changes: Node >= 21, YAML deprecated, .ts no longer loaded,
  error output as YAML envelope, tabId → targetId, operate → browser
- 10+ new adapters, 15+ adapter enhancements
- Major refactors: JS-first adapters, registry validation, strategy normalization
- Performance: P0 optimizations, fast-path completion, browser pipeline
- Upgrade guide with step-by-step migration instructions
2026-04-11 13:51:51 +08:00
jakevin 892ddcb19f docs: fix stale .ts adapter references in skills and guides (#954)
* docs: fix stale .ts adapter references in skills and guides

All adapters are now .js files. Update references in:
- opencli-oneshot SKILL.md (7 instances)
- opencli-autofix SKILL.md (1 instance)
- opencli-explorer references (15 instances)
- electron-app-cli guide (5 instances)

* docs: fix stale YAML/TS references in READMEs and zh docs

- Plugin type column: YAML/TS → JS (all 4 plugins have JS conversion PRs)
- synthesize command: "YAML adapters" / "TS adapters" → "JS adapters"
- zh index: remove "YAML 声明式" reference
- zh README: add missing vk plugin entry

* docs: fix remaining .ts references found in review

- electron-app-cli.md: "TypeScript desktop adapter" → "desktop adapter", file layout .ts → .js
- adapter-templates.md: section title "提取 utils.ts" → "提取 utils.js"
- opencli-explorer SKILL.md: "写 following.ts" → "写 following.js"
2026-04-11 13:13:27 +08:00
jakevin ee8d7cce77 docs: fix stale adapter counts and .ts reference (#950)
- README.md: "70+ pre-built adapters" → "87+"
- docs/comparison.md: "73+ sites" → "87+", ".ts adapter" → ".js adapter"
2026-04-11 13:13:21 +08:00
jakevin 420dc0f3c8 fix: DEBUG_SNAPSHOT should work without DEBUG=opencli (#952)
log.debug() requires DEBUG=opencli to output, which means
DEBUG_SNAPSHOT=1 alone no longer shows snapshot fallback diagnostics.
Use process.stderr.write directly since the DEBUG_SNAPSHOT guard
already controls when this diagnostic fires.
2026-04-11 13:13:14 +08:00
jakevin 0f90b42f71 fix: warn users when .ts adapters are found but not loaded (#951)
Users who created custom .ts adapters in ~/.opencli/clis/ will see
their commands silently disappear after upgrading to the JS-only
version. Add an explicit warning so they know to convert to .js.
2026-04-11 13:13:07 +08:00
jakevin 2469d12efd fix: resolve alias target correctly in validate command (#949)
The alias resolution logic checked `!registry.has(target)` before
calling `registry.get(target)`, which always returned undefined.
Moreover, aliases registered as `site/alias` keys meant `registry.has`
returned true, skipping the block entirely. The canonical name was
never resolved, so `validate site/alias` silently checked 0 commands.

Simplify to always resolve via `registry.get(target)` which handles
both canonical keys and alias keys correctly.
2026-04-11 13:13:02 +08:00
Harvey Yue 25014f1067 fix(bilibili): add missing domain for following cli (#947) 2026-04-11 12:36:41 +08:00
jakevin 63b7b291ab fix: clean up stale .ts adapter files during upgrade (#948)
Older versions (pre-1.7.1) shipped adapters as .ts files. When users
upgrade to a .js-only version, the old .ts files are left orphaned in
~/.opencli/clis/. Add a cleanup step that removes .ts files when a
corresponding .js official adapter exists.
2026-04-11 12:36:02 +08:00
jakevin 4a0b8054b2 fix: batch quality improvements — dedupe completion, unify logging, fix docs (#945)
* fix: batch quality improvements — dedupe completion, unify logging, fix docs

1. Extract shared completion code (BUILTIN_COMMANDS + shell scripts) into
   completion-shared.ts, eliminating duplication between completion.ts and
   completion-fast.ts.

2. Replace console.error/warn/log with log.* from logger.ts in:
   - daemon.ts (7 occurrences)
   - runtime.ts (1 occurrence)
   - cli.ts browserAction error handler (3 occurrences)
   - base-page.ts snapshot fallback (1 occurrence)
   - download/index.ts cookie warning (1 occurrence)
   - commands/daemon.ts (2 occurrences)

3. Fix Node version in build-extension.yml: 20 → 22 (matches package.json >=21)

4. Fix error handling consistency: tap.ts now throws CliError instead of bare Error

5. Remove 31 duplicate rows in docs/adapters/index.md (grok, gemini, yuanbao,
   notebooklm, doubao, weread + 25 more entries duplicated without .md suffix)

6. Update skill version: opencli-usage SKILL.md 1.6.9 → 1.7.0, adapter count 79 → 87

* fix: update daemon.test.ts to match logger migration

Tests now spy on process.stderr.write (used by log.*) instead of
console.log/console.error (no longer used by daemonStop).

* fix: address review feedback on PR #945

1. base-page.ts: restore DEBUG_SNAPSHOT env guard — log.debug uses a
   different env var (DEBUG=opencli), so keep the original gate to
   avoid breaking existing users.

2. daemon.ts: remove dead `prefix` variable left over from console.error
   migration.
2026-04-11 01:45:00 +08:00
jakevin 575986c656 perf: P0 performance optimizations (#944)
* perf: P0 performance optimizations — VM context reuse, startup parallelization, stealth caching

1. Reuse VM sandbox context in pipeline template engine instead of creating
   a new vm.createContext() on every expression evaluation. This eliminates
   ~0.3ms per call in map/filter loops over large arrays.

2. Cache sanitizeContext() results via WeakMap keyed by object reference.
   In pipeline loops, `args` and `data` are the same object across all
   iterations — the expensive JSON round-trip now runs only once per step.

3. Parallelize independent startup I/O: built-in CLI discovery now runs
   concurrently with ensureUserCliCompatShims and ensureUserAdapters,
   saving ~30-50ms on cold start.

4. Cache the stealth JS string (350 lines, pure static) after first
   generation — every subsequent goto() reuses the cached string.

* fix: address review feedback on P0 perf optimizations

1. sanitizeContext: cache JSON string instead of parsed object to prevent
   sandbox mutation from polluting subsequent calls
2. VM sandbox: clean non-whitelisted properties before each execution to
   prevent cross-expression state leakage
3. Startup parallelization: document registry overwrite semantics and
   confirm no shared-state race between parallel tasks
2026-04-11 01:31:34 +08:00
jakevin 110e047b3c refactor(validate): switch from YAML to registry-based validation (#943)
* refactor(validate): switch from YAML scanning to registry-based validation

The validate/verify commands only scanned YAML files, which are no
longer supported. Rewrite to validate commands from the in-memory
registry populated by discoverClis(), aligning with the JS-first
adapter architecture.

New checks: missing description, browser commands without domain,
pipeline step name typos, commands without func/pipeline, duplicate
arg names, and positional arg ordering.

* fix(validate): treat lazy-loaded commands as valid

Manifest-registered commands have _lazy=true and no func/pipeline
until execution time. Recognize this as a valid execution form.

* fix(validate): warn on empty registry, support alias targets

- Emit warning when registry is empty instead of silent PASS
- Resolve alias targets to canonical key before filtering
2026-04-11 01:17:14 +08:00
jakevin a9d21f3de0 fix: project hygiene — docs, lint, daemon restart (#942)
* fix: project hygiene — docs, lint, daemon restart, code fence

- Update Node version requirement from >= 20 to >= 21 in 7 doc files
  (README, README.zh-CN, installation guides, troubleshooting)
- Update adapter count from 79+ to 87+ in READMEs
- Remove duplicate `lint` script (identical to `typecheck`)
- Fix TESTING.md CI matrix: Node ['22'] instead of ['20', '22']
- Fix autofix SKILL.md code fence escaping (\``` → ~~~)
- Add daemon restart to postinstall so updated adapters are picked up
- Fix preuninstall to respect OPENCLI_DAEMON_PORT env var

* fix: align docs and skills with JS-first adapter contract

Adapters are now .js files (not .ts). Update all references across:
- README.md, README.zh-CN.md, CONTRIBUTING.md
- docs/guide/getting-started.md, docs/index.md
- skills/opencli-browser/SKILL.md, skills/opencli-explorer/SKILL.md

The runtime (discovery.ts) only loads .js from user clis/ directories,
and `opencli browser init` generates .js scaffolds. Documentation was
still teaching users to create .ts files.

* fix: update CI matrix to Node 22 only (drop Node 20)

package.json requires Node >= 21 (styleText dependency). The CI matrix
was still testing Node 20 which doesn't meet this requirement.

* fix: revert incorrect daemon restart from postinstall

The daemon (browser bridge) only handles CDP communication — it has no
knowledge of adapters. Adapter discovery, loading, and execution all
happen in the CLI process, which is fresh each invocation. The
_loadedModules cache in execution.ts is process-local and not a real
staleness concern. Remove the unnecessary restartDaemon() call.
2026-04-11 00:50:05 +08:00
jakevin 383d28fcf7 refactor: normalize strategy into runtime fields at registration time (#941)
Strategy is a 5-value enum (PUBLIC/COOKIE/HEADER/INTERCEPT/UI) that
the execution path was reading at two points — resolvePreNav() and
shouldUseBrowserSession() — to make decisions that are already fully
expressible by the existing `browser` and `navigateBefore` fields.

This commit introduces normalizeCommand() inside registerCommand(),
which expands strategy into concrete runtime fields at registration
time. After normalization, execution code never reads cmd.strategy.

normalizeCommand expansion rules:
  - strategy → browser: PUBLIC defaults to false, others to true.
    Explicit browser value always wins.
  - strategy + domain → navigateBefore:
    · COOKIE/HEADER + domain → 'https://{domain}' (pre-navigate)
    · Non-PUBLIC without domain → true (needs auth context, no URL)
    · PUBLIC → undefined (no auth needed)
    Explicit navigateBefore (false or string) always wins.

This matters because commands enter the registry from 4 sources
(cli(), manifest, generate-verified, tests), and previously only
cli() did strategy derivation. The other 3 constructed CliCommand
directly, leaving strategy as a runtime dependency. Now all sources
converge through registerCommand → normalizeCommand.

Changes:
  - registry.ts: add normalizeCommand(); simplify cli() to delegate
    all derivation to normalizeCommand via registerCommand()
  - execution.ts: resolvePreNav() no longer reads strategy; just
    reads the already-expanded navigateBefore field. Strategy import
    removed.
  - capabilityRouting.ts: shouldUseBrowserSession() checks
    cmd.navigateBefore (truthy = needs browser session) instead of
    cmd.strategy !== PUBLIC. Strategy import removed.
  - discovery.ts: manifest path no longer hardcodes browser default;
    delegates to normalizeCommand.
  - capabilityRouting.test.ts: test now reflects normalized command
    shape (navigateBefore: true for COOKIE without domain).

strategy is preserved as metadata on CliCommand — opencli list,
cascade probe, adapter generation, and documentation continue to
read it. Only the execution path stops consuming it.
2026-04-11 00:37:31 +08:00
jakevin f92f7571b4 chore: remove unused test-site.mjs script (#940)
Not referenced in package.json, CI, or documentation.
2026-04-11 00:19:02 +08:00
jakevin 93bd1eb88a docs: document autofix issue filing flow (#939) 2026-04-10 23:52:45 +08:00
jakevin 39a1d673ac feat(skill): add upstream issue filing step to opencli-autofix (#938)
Add Step 6 to the autofix skill: after a verified local fix, prepare a
GitHub issue draft and file it (with user confirmation) via `gh issue
create`. Pure skill/documentation approach — no new runtime code.

Closes the need addressed by #936 with zero code, zero tests to maintain.
2026-04-10 23:46:54 +08:00
jakevin cc18ed67b7 fix: sync package-lock.json to unblock CI (#937)
* fix: sync package-lock.json with package.json dependencies

package-lock.json was missing @emnapi/core@1.9.2 and
@emnapi/runtime@1.9.2 (transitive deps of @emnapi/wasi-threads),
causing `npm ci` to fail on all CI jobs.

* fix: resolve remaining CI failures after TS-to-JS adapter migration

- vitest.config.ts: update adapter project include/exclude from .test.ts
  to .test.{ts,js} to match converted adapter test files
- check-doc-coverage.sh: skip adapter directories containing only utility
  files (prefixed with _), fixing false positive for clis/slock/
- linux-do/topic-content.test.js: fix hardcoded reference to topic.ts
  (now topic.js after PR #928 migration)
2026-04-10 23:29:09 +08:00
jakevin 91c208c855 fix: address deep review findings (security, correctness, consistency) (#935)
* fix: address deep review findings (security, correctness, consistency)

1. Security: add path traversal guard for plugin manifest entry.path
2. Security: sanitize evaluate() index param via JSON.stringify
3. Correctness: fix startNetworkCapture idempotency (don't wipe entries on re-call)
4. Correctness: log pre-navigation failures instead of silently swallowing
5. Consistency: replace console.log/error with log module in commanderAdapter, external
6. Consistency: add PluginError class, convert user-facing plugin errors
7. Dedup: remove local isRecord() in plugin.ts, use shared utils.ts version
8. Clarify: document intentional double validateArgs call

* chore: remove unused chalk imports from external.ts and commanderAdapter.ts

* refactor: replace chalk with Node.js built-in util.styleText

- Remove chalk dependency, use `styleText` from `node:util` (stable in Node 21+)
- Bump engines to Node >= 21
- Update all 10 source files that used chalk
- Remove stale chalk mock from daemon.test.ts
- One fewer runtime dependency

* fix: tighten deep-review follow-up
2026-04-10 22:59:41 +08:00
jakevin 2288cf7149 fix: clean up legacy shim files and stale tmp files on upgrade (#934)
* fix: clean up legacy shim files and stale tmp files on upgrade

Add cleanup steps to fetch-adapters.js that run on every version upgrade:

1. Remove legacy compat shim files from ~/.opencli/ (registry.js,
   errors.js, utils.js, etc.) that were created by an older approach
   using file:// re-exports. Current approach uses node_modules symlink.
   Only deletes files containing "export * from 'file://" to avoid
   removing user-created files.

2. Remove legacy compat shim directories (browser/, download/, errors/,
   etc.) using the same safety check.

3. Clean up stale .plugins.lock.json.tmp-* files left behind by
   crashed processes. These accumulate over time (108 found on one
   machine) and clutter ~/.opencli/.

* fix: check every file in legacy shim directories before deleting

Instead of checking only the first file and deleting the entire
directory, now checks each file individually and only deletes files
matching the shim pattern. Directory is removed only if empty after
individual file cleanup.
2026-04-10 18:58:44 +08:00
jakevin 2457002167 chore: remove migration residuals (mapDistToSource, clean-yaml) (#931)
- Remove mapDistToSource() from diagnostic.ts — mapped dist/clis/
  paths back to clis/ but dist/clis/ no longer exists after JS-first
  migration. The function always returned null.
- Simplify resolveAdapterSourcePath() to check candidates directly
  without the dead dist→source mapping detour.
- Delete scripts/clean-yaml.cjs — walked dist/clis/ to delete YAML
  files, but dist/clis/ no longer exists.
- Remove clean-yaml script entry from package.json.
2026-04-10 16:41:44 +08:00
jakevin 714df646a5 fix(security): escape codegen strings and redact diagnostic body (#930)
1. candidateToJs: escape single quotes in site, name, domain, and arg
   name/type fields to prevent syntax errors in generated JS adapters.
   Previously only description and help fields were escaped.

2. diagnostic: pass network request body through redactText() to
   prevent sensitive data (JWT, bearer tokens) from leaking into
   repair context. responseBody/responsePreview already used
   sanitizeCapturedValue which calls redactText, but the body field
   only had truncation.
2026-04-10 15:29:32 +08:00
jakevin d2974a9ff6 refactor(adapters): convert adapter layer from TypeScript to JavaScript (#928)
* refactor(adapters): convert adapter layer from TypeScript to JavaScript

Core framework stays TypeScript; adapter layer moves to JS-first.
Adapters are essentially "executable config + browser scripts" that
barely use TS features — this simplifies the build/distribution pipeline
by removing the dist/clis/ intermediate compilation step.

Changes:
- Convert all 753 adapter files in clis/ from .ts to .js
- Update tsconfig to exclude clis/ from compilation
- Simplify build-manifest to scan clis/*.js directly (no dist/clis/)
- Update discovery, main, fetch-adapters to load JS adapters from clis/
- Update generate-verified to output .js artifacts
- Update package.json files field: dist/clis/ → clis/
- Fix all test files for the .ts → .js transition

* fix(main): use findPackageRoot for BUILTIN_CLIS path

The previous relative path (../../clis from __dirname) only worked for
dist/src/main.js but broke dev mode (tsx src/main.ts) where __dirname
is <repo>/src — resolving to /clis instead of <repo>/clis.

Use findPackageRoot() which works for both dev and prod paths.
2026-04-10 14:52:18 +08:00
jakevin b45a64d91d fix(build-manifest): import compiled JS from dist/clis/ instead of raw TS (#926)
* fix(build-manifest): import compiled JS from dist/clis/ instead of raw TS

Node's type stripping does not rewrite '.js' → '.ts' in import
specifiers, so dynamically importing .ts source files fails whenever
they contain relative imports like './utils.js'.

Switch to scanning dist/clis/ for compiled .js files after tsc runs.
This eliminates all 268 "Cannot find module" warnings and increases
manifest entries from 254 to 532 (previously half were silently skipped).

* fix: write manifest to dist/cli-manifest.json where runtime expects it

The runtime resolves BUILTIN_CLIS to dist/clis/ (relative to
dist/src/main.js), so discoverClis() looks for manifest at
dist/cli-manifest.json. Previously it was written to the package root
where the runtime never found it — manifest was effectively unused,
always falling through to filesystem scanning.
2026-04-10 12:58:34 +08:00
jakevin dbac7fc921 refactor(errors): unify error output as YAML envelope to stderr (#923)
* refactor(errors): unify error output as YAML envelope to stderr

Replace the 100+ line chalk renderError() switch-case with a single
YAML envelope output path. All errors now output a structured
{ok, error: {code, message, help, exitCode}} envelope to stderr,
regardless of TTY status.

This simplifies the error system from 5 mechanisms to 3:
1. Error Envelope (YAML → stderr) — unified error output
2. Exit codes (sysexits.h) — process exit semantics
3. Diagnostic (OPENCLI_DIAGNOSTIC=1) — autofix repair context

Removed: chalk error rendering, ERROR_ICONS map, classifyGenericError
regex classifier, BrowserConnectError-specific bridge status display.
Added: toEnvelope() utility, ErrorEnvelope type.

* refactor(errors): migrate adapters to throw CliError, update docs

- Migrate xueqiu adapters from return [{error,help}] to throw CliError
- xueqiu/utils.ts: fetchXueqiuJson now throws AuthRequiredError/
  CommandExecutionError instead of returning {error, help} objects
- Remove resolveColumns error fallback from output.ts (no longer needed)
- Add verbose stack trace support to error envelope
- Add ADAPTER_LOAD to AutoFix hint trigger codes
- Update skill docs (adapter-templates, explorer, oneshot, advanced-patterns)
  to recommend throw CliError pattern instead of return [{error, help}]

* fix: remove remaining dead error-forwarding in 4 xueqiu adapters + review fixes

- Remove `if ('error' in d) return [d]` from feed, hot, search, kline
  (fetchXueqiuJson now throws, so these were dead code)
- Add `stack?: string` to ErrorEnvelope interface (removes type cast hack)
- Fix adapter-templates.md: use AuthRequiredError instead of plain Error

* fix: migrate barchart/quote and yahoo-finance/quote to throw CliError

Last two adapters that silently returned [] on error instead of
throwing CommandExecutionError.

* fix: self-review fixes — doc evaluate crash, error messages, kline consistency

- adapter-templates.md: getServerContext was throwing AuthRequiredError
  inside a function serialized into page.evaluate() (browser has no
  CliError). Reverted to return {error} sentinel + func() body throw.
- yahoo-finance/quote, barchart/quote: include symbol in fallback error msg
- xueqiu/kline: throw EmptyResultError instead of returning [] for
  consistency with other xueqiu adapters
2026-04-10 03:20:53 +08:00
jakevin 309dadcf46 refactor(adapters): migrate pipeline adapters to func() + { error, help } pattern; docs: skill improvements (#922)
* docs(skills): add Tier 2.5 localStorage Bearer, SPA discovery, and test standards

From real-world experience building slock.ai CLI adapters:

- oneshot: add network-empty diagnosis, SPA baseURL bundle search, Tier 2.5
  localStorage Bearer template (with multi-tenant X-Server-Id pattern),
  updated auth quick-reference, file path note, opencli browser verify test flow
- explorer: add Tier 2.5 to decision tree and strategy table, update test section
  with opencli browser verify + Done standard, fix Step 5 path to ~/.opencli/clis/,
  add 4 new pitfall rows (SPA HTML, 400 context header, empty network, wrong dir)

* docs(skills): fix path conflict + add anti-change patterns from real adapters

Fix reviewer blocking issue:
- Remove the contradictory "~/.opencli/clis/" note that mixed user-local and
  repo-contributor workflows; replace with explicit two-scenario callout in
  Step 4, Step 5, pitfall table, and oneshot test section
- Template comments in oneshot restored to clis/<site>/<name>.ts (repo path)

Add "抗变更模式" section to explorer, based on opencli's own production code:
- Pattern 1: dynamic queryId discovery (twitter/shared.ts resolveTwitterQueryId)
  — scan loaded JS bundle by operationName (stable) to find queryId (unstable)
- Pattern 2: semantic DOM priority fallback (web/read.ts)
  — article > [role=main] > main > class-hint > body, pick largest text block
- Pattern 3: ordered selector array + timestamp comments (xiaohongshu/publish.ts)
  — first-match wins, comment records UI version and observed attribute values
- Pattern 4: nullish-coalescing field multi-path (xiaohongshu/user-helpers.ts)
  — covers camelCase/snake_case variants without assuming fixed key name

* docs(explorer): split SKILL.md into reference sub-documents

- Shrink main SKILL.md from 994 to 270 lines — core workflow only
- Extract all TS templates (Tier 1~4, pagination) to references/adapter-templates.md
- Add error handling standard: { error, remedy } pattern (remedy > hint)
- Add Tier 2.5 localStorage Bearer template with multi-tenant X-Server-Id example
- Extract cascading requests, tap debug, verbose mode, anti-change patterns to references/advanced-patterns.md
- Extract record workflow to references/record-workflow.md

* docs(skills): fix verify command — split by dev scenario

browser verify only reads ~/.opencli/clis/, not repo's clis/.
Split all verify instructions:
- Repo 贡献: npm run build + opencli <site> <cmd>
- 私人 adapter: opencli browser verify <site>/<name>

Fixes blocker in explorer:L209, L224 and oneshot:L286, L298

* docs(adapter-templates): add utils.ts extraction pattern for same-site adapters

* docs(skills): add decision matrix, stop conditions, sync comments

explorer: add path decision matrix before core workflow
oneshot: add explicit stop/switch conditions (when to escalate to explorer)
both: add keep-in-sync comment on the two-scenario verify block

* feat(slock): extract utils.ts + apply { error, help } pattern; docs: remedy→help

slock/utils.ts: new — getSlockContext(), resolveChannelId()
  - Shared token + workspace resolution, no more 4-line duplication
  - UUID regex (/^[0-9a-f]{8}-...$/) replaces fragile !includes('-')
  - Returns { error, help } instead of throwing

tasks.ts / members.ts / send.ts:
  - Import from utils.ts, remove all duplicated auth boilerplate
  - All errors return [{ error, help }], no more throw
  - members.ts: add limit arg (was unbounded before)

docs: rename remedy → help across all skill references

* refactor(adapters): migrate pipeline adapters to func() with { error, help } pattern

- slock: agents, channels, messages, servers now use getSlockContext/resolveChannelId
  from utils.ts; error handling uses { error, help } return instead of bare throws
- linux-do: export fetchLinuxDoJson from feed.ts; migrate search, topic, categories,
  tags, user-posts, user-topics from pipeline+throw to func() using fetchLinuxDoJson
- xueqiu: add utils.ts with fetchXueqiuJson helper; migrate hot, feed, search, stock,
  watchlist, hot-stock, groups, kline, earnings-date from pipeline+throw to func()

* fix(output): show error rows in table/csv/markdown when columns declared

When a command declares columns (e.g. ['rank', 'title', 'value']) but
returns an error row ({ error, help }), the declared columns would
render empty cells. Now resolveColumns detects the error key and falls
back to the row's actual keys, making diagnostics visible in all output
formats.

* chore: remove slock adapters from this PR

Slock adapters should be in a separate PR, not bundled with the
adapter refactor and skill docs improvements.
2026-04-10 02:29:35 +08:00
jakevin 56f371fbad docs(skills): improve oneshot & explorer with real-world SaaS patterns (#921)
* docs(skills): add Tier 2.5 localStorage Bearer, SPA discovery, and test standards

From real-world experience building slock.ai CLI adapters:

- oneshot: add network-empty diagnosis, SPA baseURL bundle search, Tier 2.5
  localStorage Bearer template (with multi-tenant X-Server-Id pattern),
  updated auth quick-reference, file path note, opencli browser verify test flow
- explorer: add Tier 2.5 to decision tree and strategy table, update test section
  with opencli browser verify + Done standard, fix Step 5 path to ~/.opencli/clis/,
  add 4 new pitfall rows (SPA HTML, 400 context header, empty network, wrong dir)

* docs(skills): fix path conflict + add anti-change patterns from real adapters

Fix reviewer blocking issue:
- Remove the contradictory "~/.opencli/clis/" note that mixed user-local and
  repo-contributor workflows; replace with explicit two-scenario callout in
  Step 4, Step 5, pitfall table, and oneshot test section
- Template comments in oneshot restored to clis/<site>/<name>.ts (repo path)

Add "抗变更模式" section to explorer, based on opencli's own production code:
- Pattern 1: dynamic queryId discovery (twitter/shared.ts resolveTwitterQueryId)
  — scan loaded JS bundle by operationName (stable) to find queryId (unstable)
- Pattern 2: semantic DOM priority fallback (web/read.ts)
  — article > [role=main] > main > class-hint > body, pick largest text block
- Pattern 3: ordered selector array + timestamp comments (xiaohongshu/publish.ts)
  — first-match wins, comment records UI version and observed attribute values
- Pattern 4: nullish-coalescing field multi-path (xiaohongshu/user-helpers.ts)
  — covers camelCase/snake_case variants without assuming fixed key name

* docs(explorer): split SKILL.md into reference sub-documents

- Shrink main SKILL.md from 994 to 270 lines — core workflow only
- Extract all TS templates (Tier 1~4, pagination) to references/adapter-templates.md
- Add error handling standard: { error, remedy } pattern (remedy > hint)
- Add Tier 2.5 localStorage Bearer template with multi-tenant X-Server-Id example
- Extract cascading requests, tap debug, verbose mode, anti-change patterns to references/advanced-patterns.md
- Extract record workflow to references/record-workflow.md

* docs(skills): fix verify command — split by dev scenario

browser verify only reads ~/.opencli/clis/, not repo's clis/.
Split all verify instructions:
- Repo 贡献: npm run build + opencli <site> <cmd>
- 私人 adapter: opencli browser verify <site>/<name>

Fixes blocker in explorer:L209, L224 and oneshot:L286, L298

* docs(adapter-templates): add utils.ts extraction pattern for same-site adapters

* docs(skills): add decision matrix, stop conditions, sync comments

explorer: add path decision matrix before core workflow
oneshot: add explicit stop/switch conditions (when to escalate to explorer)
both: add keep-in-sync comment on the two-scenario verify block
2026-04-10 01:47:37 +08:00
jakevin f6f7f04ff6 docs(skills): unify browser tool names to opencli browser commands (#920)
* docs(skills): unify browser tool names to `opencli browser` commands

Replace abstract MCP tool names (browser_navigate, browser_snapshot,
browser_network_requests, browser_click, browser_evaluate) with
concrete `opencli browser` CLI commands in explorer and oneshot skills.

This aligns all three browser-related skills into a clear hierarchy:
- opencli-browser: atomic command reference
- opencli-oneshot: 4-step quick generation workflow
- opencli-explorer: full site exploration workflow

* docs(skills): address review — demote explore, fix eval placeholder

1. Demote `opencli explore` from "recommended" to "supplementary helper"
   and make `opencli browser` the explicit primary path for API discovery.
2. Fix `url` undefined variable in eval example — use `<API URL>` placeholder.
2026-04-10 00:39:07 +08:00
jakevin 43f87d2ede fix: restore cross-platform entries in package-lock.json (#919)
Build Chrome Extension / build (push) Has been cancelled
Release / release (push) Has been cancelled
The lockfile generated on Node 25/darwin dropped optional+peer deps
(@emnapi/core, @emnapi/runtime) needed by CI on linux/x64, causing
npm ci to fail.
2026-04-09 23:07:31 +08:00
jakevin b87bbc7107 chore: bump version to 1.7.0 (#917)
Bump CLI, extension package.json, and extension manifest.json to 1.7.0.
Update package-lock.json.
2026-04-09 21:52:40 +08:00
jakevin da86659566 fix(jianyu): avoid early api bucket cutoff (#916) 2026-04-09 21:39:03 +08:00
GanFanNewOrder 606bc59f7b fix(jianyu): stabilize search and add detail extraction contract (#912)
* fix(jianyu): stabilize search and add detail extraction contract

* fix(jianyu): require query evidence for search results

---------

Co-authored-by: 泽加武 <zejiawu@zejiawudeMac-mini.local>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 21:28:20 +08:00
jakevin 2ddf571445 feat: auto-close adapter windows, add OPENCLI_WINDOW_FOCUSED, document config (#915)
* feat: auto-close adapter windows, add OPENCLI_WINDOW_FOCUSED, document config

1. Adapter commands now close the automation window immediately after
   completion instead of waiting for the 30s idle timeout.

2. OPENCLI_WINDOW_FOCUSED=1 opens automation windows in the foreground
   (useful for debugging). Default remains background.

3. Add Configuration section to README (EN/ZH) and opencli-usage skill
   listing all stable user-facing environment variables.

* Fix OPENCLI_WINDOW_FOCUSED to be per-request, not frozen at daemon startup

Move env var read from daemon (startup-time constant) to CLI side
(sendCommandRaw), so it works correctly with the persistent daemon model.
Each request now reads the env var fresh and includes windowFocused in
the command payload.
2026-04-09 21:27:02 +08:00
Clearner1 7f31df2912 fix(xiaoe): resolve missing episodes for long courses via auto-scroll (#904)
* fix(xiaoe): resolve missing episodes for long courses by handling lazy load

* fix(xiaoe): keep lazy-load scroll until inner list stabilizes

---------

Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 21:21:29 +08:00
jakevin b0c9966774 Remove daemon status/restart references from docs and READMEs (#914)
These commands were removed in the persistent daemon refactor.
Only `daemon stop` remains as a user-facing command.
2026-04-09 20:47:07 +08:00
jakevin 5f59f9b563 refactor: make daemon persistent, remove idle timeout (#913)
* refactor: make daemon persistent, remove idle timeout

- Remove IdleManager and 4-hour idle auto-exit
- Daemon now stays alive until explicit shutdown or uninstall
- Add preuninstall hook for best-effort daemon cleanup on npm uninstall
- Update docs to reflect persistent daemon model

* fix: remove stale idle timeout references from code and docs

* refactor: remove daemon status/restart commands and lastCliRequestTime

- Remove `daemon status` and `daemon restart` CLI commands (doctor covers diagnostics)
- Remove `lastCliRequestTime` tracking (no longer needed without idle timeout)
- Keep only `daemon stop` as the explicit shutdown command

* Add AbortSignal.timeout(3s) to preuninstall shutdown fetch

Prevents npm uninstall from hanging if the daemon port accepts
connections but never responds.
2026-04-09 20:37:04 +08:00
jakevin 45001ec025 chore: bump extension version to 1.6.10 (#911)
Build Chrome Extension / build (push) Has been cancelled
Release / release (push) Has been cancelled
2026-04-09 20:02:55 +08:00
jakevin 2ad95fca06 chore: bump version to 1.6.10 (#910) 2026-04-09 19:36:26 +08:00
jakevin bbd1163a01 refactor: unify browser error classification and deduplicate retry logic (#908)
* refactor: unify browser error classification and deduplicate retry logic

Replace two overlapping error classification systems with a single
classifyBrowserError() that returns retry advice (retryable + delayMs):

- Extension/daemon transient errors → retryable, 1500ms delay
- CDP target navigation errors → retryable, 200ms delay
- Non-transient errors → not retryable

Deduplicate sendCommand/sendCommandFull retry loop into sendCommandRaw,
making both public functions thin return-value wrappers.

* fix: add error kind to prevent page-level retry of extension errors

classifyBrowserError() now returns a `kind` field:
- extension-transient: retried by daemon-client only
- target-navigation: retried by page-level settle logic
- non-retryable: no retry

Page.goto() and Page.evaluate() now only settle-retry on
target-navigation, preventing extension/daemon errors from being
silently swallowed as settle noise.
2026-04-09 18:10:15 +08:00
jakevin 611b640458 docs: mention refreshing packaged skills on update (#902) 2026-04-09 12:36:52 +08:00
jakevin 555626f409 feat: replace tabId with targetId as cross-layer page identity (#899)
Use Chrome CDP targetId (UUID) as the canonical page identity across
all layers (extension → daemon → CLI), demoting tabId to an
extension-internal routing detail.

- Add extension/src/identity.ts: bidirectional targetId ↔ tabId mapping
  with lazy refresh via chrome.debugger.getTargets()
- Update protocol: Command.page and Result.page carry targetId
- Update background.ts: resolveCommandTabId() and pageScopedResult()
  helpers; all page-scoped handlers return targetId
- Add sendCommandFull() to daemon-client for responses with page identity
- Update Page class: _page stores targetId, goto/selectTab extract it
- Update record.ts: injectedPages tracks by targetId
- Add extension tests to vitest config and CI test scripts
2026-04-09 12:26:32 +08:00
luka2chat fb09f8565d docs: fix desktop adapter commands to match actual CLI output (#900)
Synced all desktop adapter command lists in desktop.md with
the actual `opencli <adapter> --help` output:

- cursor: remove non-existent status/new/dump/screenshot; add composer
- codex: remove non-existent status/new/dump/screenshot
- chatgpt: add missing model command
- chatwise: remove non-existent new/screenshot
- notion: update descriptions to match help text
- discord-app: update descriptions to match help text
- doubao-app: reorder to match help output
- antigravity: remove non-existent ask; add serve/status

Also moved `status` to the top of each adapter section where it exists.
2026-04-09 12:20:26 +08:00
AstroHan caebede7c2 fix: repair baseline main ci checks (#901) 2026-04-09 12:19:44 +08:00
Mu b09749576c feat(jd,taobao,cnki): revive shopping adapters on current layout (#248)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 02:44:46 +08:00
jakevin 93a650bcfe perf: fast-path completion/version/shell-scripts to bypass full discovery (#898)
* perf: fast-path completion, version, and shell scripts to bypass full discovery

Lightweight commands (--get-completions, --version, completion <shell>) now
resolve before any heavy module loading. Key changes:

- New completion-fast.ts: manifest-based completion + shell script generators
  with zero dependency on registry/discovery/cli modules
- main.ts: static imports replaced with dynamic import() for the full startup
  path so the fast path never pays the cost of loading discovery, registry,
  Commander, hooks, etc.
- USER_CLIS_DIR inlined to avoid importing the entire discovery module
- completion.ts: removed manifest functions (moved to completion-fast.ts),
  now only used as fallback when manifest is unavailable

* fix: address review blockers from codex-mini0

1. --version fast path: only match when argv[0] is --version/-V,
   not anywhere in argv. Prevents intercepting `opencli gh --version`
   which should pass through to the subcommand.

2. Completion fast path: require ALL manifests to exist (hasAllManifests),
   not just one. If user clis dir exists but has no manifest, fall back
   to full discovery so user adapters aren't silently dropped.
   If user clis dir doesn't exist at all, skip its manifest requirement
   since there are no user adapters to miss.
2026-04-09 02:20:14 +08:00
Luke 3cc60273a8 feat: add jimeng workspaces list command (#897)
Co-authored-by: root <root@example.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 02:10:43 +08:00
ziiiiiwang 2883620ba2 Feat : add Gitee adapters and docs (#845)
* feat: add gitee adapters and docs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: compact gitee trending table output

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add gitee adapter documentation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(gitee): avoid faking user index values

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 01:29:48 +08:00
Elias@Didoo 97d152ef39 fix: retry on No window with id CDP error (#892)
* fix: retry on No window with id CDP error

* test(browser): lock transient window-id retry behavior

---------

Co-authored-by: Yun Xiao <yunxiao@agents.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 01:23:22 +08:00
Luke 0b11a1ed62 feat(jimeng): add workspace create command (#895)
Co-authored-by: jackwener <jakevingoo@gmail.com>
2026-04-09 01:08:01 +08:00
jakevin a98cc8bdd3 docs(autofix): add "'Empty' ≠ 'Broken'" pre-check before repair loop (#896)
EMPTY_RESULT and structurally-valid SELECTOR failures are often not
adapter bugs — they're the platform shaping results under anti-scrape,
or a soft 404, or a legitimately empty search. Patching a working
adapter to chase a zero-result query breaks the next working path.

Add a pre-check section at the top of opencli-autofix listing four
rule-outs that must fail before a repair round is justified:

  1. Retry with an alternative query / entry point
  2. Spot-check the page in a normal Chrome tab
  3. Look for soft 404s (200 with empty payload)
  4. Remember that "0 results" from a search is a valid answer

Placed directly before "Step 1: Collect Diagnostic Context" so the
check runs at exactly the moment the agent would otherwise commit to
a repair round.

Audience/timing is the whole point: the skill is loaded precisely when
an error has occurred and the agent is deciding whether to repair, and
the pre-check intercepts that decision before it locks in.

11 lines of markdown, zero code, single file.

---

Inspired by https://github.com/eze-is/web-access by 一泽 Eze (MIT),
specifically the "平台返回的'内容不存在'不一定反映真实状态" mental model
from its SKILL.md. Adapted into concrete, actionable checks for
opencli's EMPTY_RESULT classification.

Note: an earlier version of this PR also added a tool-selection
decision table and a subagent-verb rule to opencli-usage. Both were
removed after review because opencli-usage only loads *after* an agent
has committed to using opencli — advice placed there arrives too late
to change tool selection, and is not seen by the main agent at
delegation time. The insights are still valid; they just don't have a
load-time match in the current skill system. This PR keeps only the
change where audience and timing actually line up.
2026-04-09 01:05:38 +08:00
jakevin aa83fb71b2 refactor(skills): unify command reference by site instead of technology (#894)
* refactor(skills): unify command reference by site instead of technology

- Merge Browser-based and Public API sections into single alphabetical
  table with type emoji tags (🌐//🖥️/🔧)
- Delete browser.md and public-api.md (replaced by unified SKILL.md table)
- Add GitHub/DevOps and collaboration rows to capability lookup
- Remove stale File column from capability table

* feat(skills): add commands.md with merged examples + 8 missing adapters

- Create commands.md: merge browser.md + public-api.md into single
  alphabetical-by-site reference with detailed usage examples
- Add 8 missing adapters: 1688, hupu, jianyu, lesswrong, quark,
  xianyu, xiaoe, yuanbao
- Bump skill version 1.6.3 → 1.6.9 to match package.json
2026-04-09 00:32:30 +08:00
jakevin 6b51a41372 feat(skills): add External CLI section to opencli-usage (#893)
- Add dedicated External CLI section listing all 7 registered CLIs
  (gh, obsidian, docker, lark-cli, dws, wecom-cli, vercel)
- Include install/register commands so AI agents know how to manage them
- Move gh from Desktop to External CLI section
- Update desktop.md to remove gh and reference External CLI section
2026-04-08 23:49:15 +08:00
jakevin 90722be08e refactor(skills): merge opencli-generate into opencli-explorer (#891)
* refactor(skills): merge opencli-generate into opencli-explorer

opencli-generate was a thin wrapper over generateVerifiedFromUrl,
essentially an internal pipeline orchestration. Merge its entry point
into opencli-explorer as the automated fast path, keeping one unified
skill for adapter creation.

- Delete skills/opencli-generate/SKILL.md
- Add automated generation tip to opencli-explorer SKILL.md
- Update README/README.zh-CN skill references
- Update skill-generate.ts comment

* fix(docs): fix dead link in yaml-adapter deprecation page

Change ../../CONTRIBUTING.md to ./contributing (VitePress internal link).
2026-04-08 23:38:16 +08:00
jakevin 810547c9a9 refactor: eliminate any types in core (non-clis) files (#886)
Replace explicit `any` with `unknown` + narrowing or concrete types across
all core src/ files (non-`clis/**`). Core drops from ~60 `any` occurrences
to a handful of documented, unavoidable boundaries.

Mechanical error-handler cleanup (uses getErrorMessage() from errors.ts):
  - cli.ts, cascade.ts, download/*.ts, external.ts, plugin.ts, doctor.ts
    — catch (err: any) → catch (err) + getErrorMessage(err)

Pipeline steps — typed params with per-step interfaces:
  - pipeline/steps/intercept.ts — InterceptParams, signature uses unknown
  - pipeline/steps/tap.ts       — TapParams
  - pipeline/steps/download.ts  — DownloadParams + DownloadedItem
    (ytdlp_args is now coerced via String(v) for defence-in-depth)

Probe / boundary typing:
  - cascade.ts — FetchProbeResponse interface; also fixes a latent bug
    where result.success could be assigned undefined (masked by any)
    by wrapping with !!(…)

Browser-side injected scripts — structural types at the TS boundary
(types stripped by tsc emit before .toString() runs, runtime unchanged):
  - scripts/store.ts     — PiniaStore / VuexModule / VueApp
  - scripts/framework.ts — VueAppEl / FrameworkWindow

Runtime detection:
  - runtime-detect.ts — BunGlobal interface; getRuntimeVersion reads Bun
    into a local to avoid non-null assertions.

Test files — precise structural casts replacing `as any`:
  - browser.test.ts             — `{ _state: string }` cast for private
                                   state; full DaemonStatus shape for
                                   the getDaemonHealth mock
  - browser/dom-helpers.test.ts — globalThis as Record<string, unknown>
  - browser/cdp.test.ts         — (...args: unknown[]) in mock handlers
  - runtime-detect.test.ts      — matches runtime-detect.ts BunGlobal
  - output.test.ts              — logSpy.mock.calls typed with unknown[]
  - engine.test.ts, snapshotFormatter.test.ts, pipeline/executor.test.ts
    — narrow structural casts / removed stale any casts

Verification:
  - npx tsc --noEmit: clean
  - npx vitest run (excluding e2e/smoke): 1415 passed, 1 skipped
2026-04-08 23:08:23 +08:00
jakevin 70b1145b5e refactor: migrate all CLI adapters from YAML to TypeScript (#887)
* refactor: remove version field from GenerateOutcome and EarlyHint

All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.

Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).

* refactor: migrate all 123 CLI adapters from YAML to TypeScript

Remove YAML as an adapter format entirely. All adapters now use
TypeScript with cli() from @jackwener/opencli/registry.

- Convert 123 YAML adapter files to TypeScript via batch script
- Remove YAML scanning from discovery.ts (registerYamlCli, yaml import)
- Remove scanYaml() and shouldReplaceManifestEntry() from build-manifest.ts
- Change synthesize.ts to output JSON candidates (internal format)
- Change generate-verified.ts to write .ts adapter files instead of .yaml
- Delete yaml-schema.ts (dead code) and scripts/yaml-to-ts.mjs (one-time tool)
- Update all tests to match new format

Closes discussion in #OpenCLI thread 47ddba82.

* fix: close YAML migration gaps in plugin scaffold, validation, and scan

- plugin-scaffold.ts: generate hello.ts (TS pipeline) instead of hello.yaml
- plugin.ts validatePluginStructure: no longer accept .yaml as valid command file
- plugin.ts scanPluginCommands: remove .yaml/.yml from scanned extensions
- discovery.ts: add explicit log.warn() when YAML files detected in clis/ or plugins/
- plugin.test.ts: update all test fixtures from .yaml to .js
- plugin-scaffold.test.ts: update hello.yaml references to hello.ts
- Delete dead src/yaml-schema.ts

Resolves PR #887 review blockers from @mbp-codex-pr0.

* refactor: complete YAML removal across docs, skills, record, and binance adapters

Code changes:
- record.ts: candidate output changed from .yaml (yaml.dump) to .json (JSON.stringify), removed js-yaml import
- src/clis/binance: convert all 11 YAML adapters to TypeScript cli() format
- binance/commands.test.ts: rewrite to use registry instead of yaml.load
- skill-generate.test.ts, diagnostic.test.ts: update mock paths from .yaml to .ts
- build-manifest.ts, synthesize.ts: update stale YAML comments

Documentation:
- README.md: remove .yaml from Dynamic Loader, fix plugin types, fix synthesize comment
- README.zh-CN.md: fix synthesize comment
- CONTRIBUTING.md: replace YAML Adapter section with Pipeline Adapter (TS), update arg examples
- docs/developer/yaml-adapter.md: replaced with deprecation redirect
- docs/developer/architecture.md: remove YAML pipeline references
- docs/developer/contributing.md: remove YAML adapter section
- docs/developer/ai-workflow.md: YAML → TS in synthesize description
- docs/guide/getting-started.md: remove .yaml from loader, update engine description
- docs/guide/plugins.md: remove YAML plugin option, update plugin types
- docs/index.md, docs/comparison.md: remove YAML adapter references
- docs/zh/guide/plugins.md: remove .yaml from scan description

Skills:
- opencli-explorer/SKILL.md: rewrite YAML vs TS decision tree to TS-only
- opencli-oneshot/SKILL.md: replace YAML templates with TS cli() templates
- opencli-generate/SKILL.md: YAML artifact path → TS artifact path
- opencli-usage/SKILL.md, plugins.md: update adapter format references

* fix: clean up remaining YAML adapter references in docs

- docs/zh/guide/plugins.md: replace YAML plugin example with TS pipeline
- docs/developer/testing.md: YAML Adapter heading → Adapter, remove validate line
- TESTING.md: same fix in root testing doc
- CONTRIBUTING.md: remove "YAML validation" comment
- docs/.vitepress/config.mts: mark YAML Adapter Guide as (Deprecated) in nav
- docs/advanced/download.md: remove "YAML Adapters" from pipeline step heading
2026-04-08 23:01:08 +08:00
jakevin f2de4ad63b docs: restructure README narrative (#885)
* docs: restructure readme narrative

* docs: clarify generate and agent entry points
2026-04-08 21:34:02 +08:00
jakevin ad9cce34d7 refactor: remove version field from GenerateOutcome and EarlyHint (#884)
All consumers are in the same repo and evolve together — version field
adds ceremony without practical value at this stage.

Keeps schema_version in VerifiedArtifactMetadata (sidecar file format).
2026-04-08 21:16:46 +08:00
jakevin 1662e9a73c refactor: rename operate to browser (#883)
* refactor: rename operate to browser

* fix: preserve browser rename compatibility

* fix: bump generate outcome schema version

* fix: keep generate outcome schema at v1
2026-04-08 21:03:57 +08:00
jakevin 991c8ce944 feat: P2 EarlyHint callback channel for cost gating (#882)
* fix: use Strategy.PUBLIC enum in skill-generate test to fix typecheck regression

* feat: add P2 EarlyHint callback channel to generateVerifiedFromUrl

Add optional onEarlyHint callback for internal cost gating before verify stage.

- EarlyHint type: version, stage, continue, reason, confidence, candidate?
- 3 emit points: explore (viable/not), synthesize (candidate/not), cascade (auth/ok)
- candidate only on synthesize/cascade + continue:true (not on stop or explore)
- unsupported-required-args goes directly to P1 terminal, no P2 hint emitted
- 6 new tests covering all hint paths + guardrails
2026-04-08 19:51:05 +08:00
jakevin 9365afc05f fix: use Strategy.PUBLIC enum in skill-generate test (#881) 2026-04-08 19:36:05 +08:00
1803 changed files with 123451 additions and 78559 deletions
+7 -8
View File
@@ -3,7 +3,7 @@ name: Build Chrome Extension
on:
push:
branches: [ "main" ]
tags: [ "v*.*.*" ]
tags: [ "ext-v*" ]
paths:
- 'extension/**'
- '.github/workflows/build-extension.yml'
@@ -26,7 +26,7 @@ jobs:
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: 20
node-version: 22
cache: 'npm'
cache-dependency-path: extension/package-lock.json
@@ -44,23 +44,22 @@ jobs:
- name: Create Extension ZIP
run: |
EXT_VERSION=$(node -p "require('./extension/package.json').version")
cd extension-package
zip -r ../opencli-extension.zip .
zip -r ../opencli-extension-v${EXT_VERSION}.zip .
- name: Upload Artifacts (Action Run)
uses: actions/upload-artifact@v7
with:
name: opencli-extension-build
path: |
opencli-extension.zip
path: opencli-extension-v*.zip
retention-days: 7
- name: Attach to GitHub Release
if: startsWith(github.ref, 'refs/tags/')
uses: softprops/action-gh-release@v2.6.1
uses: softprops/action-gh-release@v3.0.0
with:
files: |
opencli-extension.zip
files: opencli-extension-v*.zip
draft: false
prerelease: false
env:
+14 -6
View File
@@ -38,6 +38,18 @@ jobs:
- name: Build
run: npm run build
# Guard: committed cli-manifest.json must match the one build regenerates.
# Prevents silent drift where unrelated adapter entries vanish or change
# across PRs (agent hits unexpected manifest diff → surgical-merge churn).
- name: Check cli-manifest.json is up-to-date
if: runner.os == 'Linux'
shell: bash
run: |
if ! git diff --exit-code -- cli-manifest.json; then
echo "::error::cli-manifest.json is out of sync with the source. Run 'npm run build' and commit the result."
exit 1
fi
# ── Unit tests (vitest shard) ──
# PR: ubuntu + Node 22 only (fast feedback, 2 jobs).
# Push to main/dev: full matrix for cross-platform/cross-version coverage (12 jobs).
@@ -47,7 +59,7 @@ jobs:
fail-fast: false
matrix:
os: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && fromJSON('["ubuntu-latest","macos-latest","windows-latest"]') || fromJSON('["ubuntu-latest"]') }}
node-version: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && fromJSON('["20","22"]') || fromJSON('["22"]') }}
node-version: ${{ (github.event_name == 'push' || github.event_name == 'schedule' || github.event_name == 'workflow_dispatch') && fromJSON('["22"]') || fromJSON('["22"]') }}
shard: [1, 2]
steps:
- uses: actions/checkout@v6
@@ -61,7 +73,7 @@ jobs:
run: npm ci
- name: Run unit tests (Node ${{ matrix.node-version }}, shard ${{ matrix.shard }}/2)
run: npm test -- --reporter=verbose --shard=${{ matrix.shard }}/2
run: npx vitest run --project unit --project extension --reporter=verbose --shard=${{ matrix.shard }}/2
# ── Bun compatibility check ──
bun-test:
@@ -136,12 +148,8 @@ jobs:
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run tests/smoke/ --reporter=verbose
env:
OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
- name: Run smoke tests (macOS / Windows)
if: runner.os != 'Linux'
run: npx vitest run tests/smoke/ --reporter=verbose
env:
OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
timeout-minutes: 15
-4
View File
@@ -64,11 +64,7 @@ jobs:
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run tests/e2e/ --reporter=verbose
env:
OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
- name: Run E2E tests (macOS / Windows)
if: runner.os != 'Linux'
run: npx vitest run tests/e2e/ --reporter=verbose
env:
OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
+21 -1
View File
@@ -26,10 +26,30 @@ jobs:
- name: Type check
run: npx tsc --noEmit
- name: Install extension dependencies
run: npm ci
working-directory: extension
- name: Build extension
run: npm run build
working-directory: extension
- name: Package extension
run: npm run package:release -- --out ../extension-package
working-directory: extension
- name: Create extension ZIP
run: |
EXT_VERSION=$(node -p "require('./extension/package.json').version")
cd extension-package
zip -r ../opencli-extension-v${EXT_VERSION}.zip .
- name: Create GitHub Release
uses: softprops/action-gh-release@v2.6.1
uses: softprops/action-gh-release@v3.0.0
with:
generate_release_notes: true
files: |
opencli-extension-v*.zip
- name: Publish to npm
run: npm publish --provenance --access public
+1
View File
@@ -3,6 +3,7 @@ dist/
!extension/dist/
*.tsbuildinfo
.opencli/
.worktrees/
.mcp.json
*.log
.DS_Store
+215 -1
View File
@@ -1,5 +1,219 @@
# Changelog
## Unreleased
### Features
* **browser** — `bind` attaches `bound:*` workspaces to user-owned Chrome tabs without taking over window lifecycle; `sessions` reports `idleMsRemaining: null` for bound workspaces because they do not schedule idle close timers. ([#1169](https://github.com/jackwener/opencli/issues/1169), [#929](https://github.com/jackwener/opencli/issues/929))
* **browser lifecycle** — owned browser workspaces now lease tabs inside a shared dedicated automation container instead of owning one Chrome window per workspace; lease state is persisted for MV3 service-worker reconciliation and idle cleanup is backed by alarms.
* **web read** — make page extraction render-aware: same-origin iframe content is merged into the Markdown source, `--wait-for` can wait inside main/iframe documents, `--wait-until networkidle` waits for captured requests to settle, and `--diagnose` reports frames, empty containers, and API-like XHRs for shell/AJAX pages.
## [1.7.8](https://github.com/jackwener/opencli/compare/v1.7.7...v1.7.8) (2026-04-25)
### Features
* **powerchina** — procurement search adapter. ([#1155](https://github.com/jackwener/opencli/issues/1155))
* **toutiao** — `articles` adapter for 头条号 creator dashboard. ([#1148](https://github.com/jackwener/opencli/issues/1148))
* **weixin** — `create-draft` and `drafts` commands for Official Account. ([#1095](https://github.com/jackwener/opencli/issues/1095))
### Bug Fixes
* **chatgpt-app** — use AX send flow and support zh-CN generating state. ([#1135](https://github.com/jackwener/opencli/issues/1135))
* **deepseek** — fix history titles and resume conversation on `ask`. ([#1153](https://github.com/jackwener/opencli/issues/1153))
* **amazon** — fall back discussion to product page. ([#1154](https://github.com/jackwener/opencli/issues/1154))
* **sinafinance** — match stock symbol in addition to name. ([#1158](https://github.com/jackwener/opencli/issues/1158))
### Chores
* **extension** — restore pre-1.6.8 neon terminal icons. ([#1177](https://github.com/jackwener/opencli/issues/1177))
## [1.7.7](https://github.com/jackwener/opencli/compare/v1.7.6...v1.7.7) (2026-04-23)
### Features
* **51job** — comprehensive adapter: `search`, `hot`, `detail`, `company`. ([#1132](https://github.com/jackwener/opencli/issues/1132))
* **weread** — `ai-outline` command for AI-generated book outlines. ([#1141](https://github.com/jackwener/opencli/issues/1141))
* **web/download** — video/audio/iframe download + `--stdout` streaming. ([#1146](https://github.com/jackwener/opencli/issues/1146))
* **download** — hardened HTML→Markdown pipeline with better element handling. ([#1143](https://github.com/jackwener/opencli/issues/1143))
* **verify** — fixture-based value validation + skill docs for COOKIE pitfalls. ([#1131](https://github.com/jackwener/opencli/issues/1131))
* **agent-native retrospective** — analyze / verify guards / fixture content checks. ([#1133](https://github.com/jackwener/opencli/issues/1133))
* **twitter** — expose `has_media` and `media_urls` columns. ([#1115](https://github.com/jackwener/opencli/issues/1115))
### Bug Fixes
* **core** — quality audit fixes: elapsed=0 display, daemon error handler state reset, cause chain truncation guard, download cookie expiry, launcher async kill, verbose error logging. ([#1151](https://github.com/jackwener/opencli/issues/1151))
* **daemon** — allow extension ping CORS for reachability probing. ([#1150](https://github.com/jackwener/opencli/issues/1150))
* **deepseek** — separate thinking process from response in `--think` mode. ([#1142](https://github.com/jackwener/opencli/issues/1142))
* **deepseek** — use position-based model selection instead of text matching. ([#1123](https://github.com/jackwener/opencli/issues/1123))
* **weread/book** — add fallback selectors for reader page without cover. ([#1138](https://github.com/jackwener/opencli/issues/1138))
* **xiaoyuzhou** — correct podcast-episodes API endpoint. ([#1129](https://github.com/jackwener/opencli/issues/1129))
* **bilibili** — resolve full video URLs and preserve full description. ([#1118](https://github.com/jackwener/opencli/issues/1118))
### Docs
* Fix stale references in READMEs and autofix skill doc. ([#1130](https://github.com/jackwener/opencli/issues/1130))
* Restore and rewrite `opencli-usage` as orientation skill. ([#1128](https://github.com/jackwener/opencli/issues/1128))
## [1.7.6](https://github.com/jackwener/opencli/compare/v1.7.5...v1.7.6) (2026-04-21)
Extension bumped to 1.0.2 (body-truncation signal unified across raw / detail / fallback paths).
### Features
* **Window lifecycle flags** — `--live` (or `OPENCLI_LIVE=1`) keeps the automation window open after a command finishes; `--focus` (or `OPENCLI_WINDOW_FOCUSED=1`) brings the window to the foreground. Works on any subcommand. ([#1122](https://github.com/jackwener/opencli/issues/1122))
* **Selector-first browser interactions** — `find` / `get` / `click` / `type` / `select` accept CSS selectors in addition to numeric refs; `--nth` disambiguates multiple matches. ([#1112](https://github.com/jackwener/opencli/issues/1112))
* **Agent-native browser payload** — structured `network` bodies with truncation signal, `get html --as json` with `--depth` / `--children-max` / `--text-max` budgets, new `browser extract` command for long-form content with resume cursor. ([#1104](https://github.com/jackwener/opencli/issues/1104))
* **`network --filter <fields>`** — filter captured requests by body-shape path segments for quick API discovery. ([#1103](https://github.com/jackwener/opencli/issues/1103))
* **`get html --as json`** — structured HTML tree output; no more silent truncation on raw `--as html`. ([#1102](https://github.com/jackwener/opencli/issues/1102))
* **`browser network` rewrite** — agent-native discovery with cache keys and shape preview. ([#1100](https://github.com/jackwener/opencli/issues/1100))
* **Compound form fields** — date / select / file controls surface a `compound` envelope with format, options, `accept`. Cascading stale-ref recovery + bbox 0.99 dedup for tagged elements. ([#1116](https://github.com/jackwener/opencli/issues/1116))
* **twitter `tweets`** — fetch a user's recent posts. ([#1098](https://github.com/jackwener/opencli/issues/1098))
* **bilibili `video`** — new video command. ([#1110](https://github.com/jackwener/opencli/issues/1110))
* **deepseek `--file`** — file upload support on `ask`. ([#1093](https://github.com/jackwener/opencli/issues/1093))
### Bug Fixes
* **twitter** — 5s timeout on `resolveTwitterQueryId` to prevent hang. ([#1106](https://github.com/jackwener/opencli/issues/1106))
* **youtube** — fall back to Videos tab when Home has no videos. ([#1109](https://github.com/jackwener/opencli/issues/1109))
* **jianyu** — keep accessible detail urls in search. ([#1099](https://github.com/jackwener/opencli/issues/1099))
* **jianyu** — block inaccessible detail links and verification pages. ([#918](https://github.com/jackwener/opencli/issues/918))
### Docs
* **opencli-browser skill** — restored and upgraded for selector-first workflow. ([#1119](https://github.com/jackwener/opencli/issues/1119))
* **Window lifecycle** — sync README + skill docs with `--live` / `--focus` behavior. ([#1125](https://github.com/jackwener/opencli/issues/1125))
### Extension (1.0.2)
* Unify body-truncation contract across raw / detail / fallback network paths; surface `body_truncated` / `body_full_size` / `body_truncation_reason`. ([#1104](https://github.com/jackwener/opencli/issues/1104))
## [1.7.5](https://github.com/jackwener/opencli/compare/v1.7.4...v1.7.5) (2026-04-20)
Extension bumped to 1.0.1 (multi-tab routing + cross-origin iframe).
### Features
* **DeepSeek adapter** — browser-based `ask` / `history` / `new` / `read` / `status` ([#1088](https://github.com/jackwener/opencli/issues/1088))
* **Eastmoney adapters** — 13 finance adapters as Phase A oracle: `quote`, `rank`, `kline`, `sectors`, `etf`, `holders`, `money-flow`, `northbound`, `longhu`, `kuaixun`, `convertible`, `index-board`, `announcement` ([#1091](https://github.com/jackwener/opencli/issues/1091))
* **Twitter GraphQL lists** — `list-tweets`, `list-add`, `list-remove` ([#1076](https://github.com/jackwener/opencli/issues/1076))
* **nowcoder adapter** — 牛客网 with 16 commands ([#1036](https://github.com/jackwener/opencli/issues/1036))
* **Chinese academic & policy adapters** — `baidu-scholar`, `google-scholar`, `wanfang`, `gov-law`, `gov-policy` ([#243](https://github.com/jackwener/opencli/issues/243))
* **Download saved path** — `web read` and `weixin download` now show saved file location ([#1042](https://github.com/jackwener/opencli/issues/1042))
* **Cross-origin iframe support** — CDP execution context for iframed content ([#1084](https://github.com/jackwener/opencli/issues/1084))
### Improvements
* **Multi-tab routing** — hardened target isolation and tab routing ([#1072](https://github.com/jackwener/opencli/issues/1072))
* **Skill consolidation** — 6 skills merged into 3 (`opencli-adapter-author`, `opencli-autofix`, `smart-search`); removed mechanical commands `explore` / `synthesize` / `generate` / `cascade` / `record` ([#1094](https://github.com/jackwener/opencli/issues/1094))
* **Browser docs rewrite** — docs reoriented for AI Agent use case ([#1080](https://github.com/jackwener/opencli/issues/1080))
* **antigravity serve** — configurable timeout + auto-reconnect ([#859](https://github.com/jackwener/opencli/issues/859), [#1063](https://github.com/jackwener/opencli/issues/1063))
* **Design debt cleanup** — deprecated APIs, arg validation, dead plugin code ([#1065](https://github.com/jackwener/opencli/issues/1065))
### Bug Fixes
* **xiaoyuzhou** — migrate from broken SSR to authenticated API ([#1059](https://github.com/jackwener/opencli/issues/1059)); accept `CONFIG_ERROR` in E2E guard ([#1066](https://github.com/jackwener/opencli/issues/1066))
* **xiaohongshu** — detect draft save success ([#1060](https://github.com/jackwener/opencli/issues/1060)); verify title input sticks on publish ([#1050](https://github.com/jackwener/opencli/issues/1050))
* **twitter** — repair lists scraping from detail pages ([#1053](https://github.com/jackwener/opencli/issues/1053))
* **zsxq** — separate content from title, remove title truncation ([#1079](https://github.com/jackwener/opencli/issues/1079))
* **extension** — per-workspace idle timeout for browser sessions ([#1064](https://github.com/jackwener/opencli/issues/1064))
### Revert
* Undo output renderer table-formatting patch ([#1085](https://github.com/jackwener/opencli/issues/1085), reverts [#1081](https://github.com/jackwener/opencli/issues/1081))
### Extension (1.0.1)
* Multi-tab routing support ([#1072](https://github.com/jackwener/opencli/issues/1072))
* Cross-origin iframe CDP contexts ([#1084](https://github.com/jackwener/opencli/issues/1084))
## [1.7.0](https://github.com/jackwener/opencli/compare/v1.6.1...v1.7.0) (2026-04-11)
This is a major release with significant internal architecture changes.
Adapter code, validation, and error handling have been modernized.
### ⚠ BREAKING CHANGES
* **Node.js >= 21 required** — `import.meta.dirname` is used in core modules; Node 20 and below will fail at startup.
* **YAML adapters deprecated** — YAML-based `.yaml` adapters are no longer loaded. Existing YAML adapters must be converted to JS via `cli()` API. A deprecation warning is emitted if `.yaml` files are detected.
* **`.ts` adapters no longer loaded at runtime** — The runtime only discovers `.js` files. If you have `.ts` adapters in `~/.opencli/clis/`, compile them to `.js` or rewrite using plain JS. A warning is printed when `.ts` files without a matching `.js` are found.
* **Error output format changed** — All errors are now emitted as a structured YAML envelope to stderr. Scripts parsing stdout for `[{error, help}]` must switch to stderr / exit code. ([#923](https://github.com/jackwener/opencli/issues/923))
* **`tabId` replaced by `targetId`** — Cross-layer page identity now uses `targetId`. Extensions and plugins referencing `tabId` must update. ([#899](https://github.com/jackwener/opencli/issues/899))
* **`operate` renamed to `browser`** — All `opencli operate` commands are now `opencli browser`. ([#883](https://github.com/jackwener/opencli/issues/883))
### Features
* **auto-close adapter windows** — Browser tabs opened by adapters are automatically closed after execution; configurable via `OPENCLI_WINDOW_FOCUSED`. ([#915](https://github.com/jackwener/opencli/issues/915))
* **Self-Repair protocol** — Automatic adapter fixing when commands fail. ([#866](https://github.com/jackwener/opencli/issues/866))
* **EarlyHint callback** — Cost gating channel for generate pipeline. ([#882](https://github.com/jackwener/opencli/issues/882))
* **verified generate pipeline** — Structured contract for AI-driven adapter generation. ([#878](https://github.com/jackwener/opencli/issues/878))
* **structured diagnostic output** — AI-driven adapter repair gets structured diagnostics. ([#802](https://github.com/jackwener/opencli/issues/802))
* **auto-downgrade to YAML in non-TTY** — Machine-readable output when piped. ([#737](https://github.com/jackwener/opencli/issues/737))
* **Browser Use improvements** — Better click/type/state handling for browser automation. ([#707](https://github.com/jackwener/opencli/issues/707))
* **CDP session-level network capture** — Full network capture support for CDPPage. ([#815](https://github.com/jackwener/opencli/issues/815), [#816](https://github.com/jackwener/opencli/issues/816))
* **AutoResearch framework** — V2EX/Zhihu test suites (194 tasks). ([#731](https://github.com/jackwener/opencli/issues/731), [#717](https://github.com/jackwener/opencli/issues/717), [#741](https://github.com/jackwener/opencli/issues/741))
* **new adapters:** Gitee ([#845](https://github.com/jackwener/opencli/issues/845)), 闲鱼 ([#696](https://github.com/jackwener/opencli/issues/696)), 1688 ([#650](https://github.com/jackwener/opencli/issues/650), [#820](https://github.com/jackwener/opencli/issues/820)), LessWrong ([#773](https://github.com/jackwener/opencli/issues/773)), 虎扑 ([#751](https://github.com/jackwener/opencli/issues/751)), 小鹅通 ([#617](https://github.com/jackwener/opencli/issues/617)), 元宝 ([#693](https://github.com/jackwener/opencli/issues/693)), 即梦 ([#897](https://github.com/jackwener/opencli/issues/897), [#895](https://github.com/jackwener/opencli/issues/895)), Quark Drive ([#858](https://github.com/jackwener/opencli/issues/858)), GitHub Trending/Binance/Weather ([#214](https://github.com/jackwener/opencli/issues/214))
* **adapter enhancements:** Instagram post/reel/story/note ([#671](https://github.com/jackwener/opencli/issues/671)), Twitter image posts/replies ([#666](https://github.com/jackwener/opencli/issues/666), [#756](https://github.com/jackwener/opencli/issues/756)), 知乎 interactions ([#868](https://github.com/jackwener/opencli/issues/868)), Bilibili b23.tv short URL ([#740](https://github.com/jackwener/opencli/issues/740)), 雪球 kline/groups ([#809](https://github.com/jackwener/opencli/issues/809)), Amazon unified ranking ([#724](https://github.com/jackwener/opencli/issues/724)), Gemini deep-research ([#778](https://github.com/jackwener/opencli/issues/778)), 新浪财经热搜 ([#736](https://github.com/jackwener/opencli/issues/736)), linux-do topic split ([#821](https://github.com/jackwener/opencli/issues/821)), JD/淘宝/CNKI revived ([#248](https://github.com/jackwener/opencli/issues/248))
### Bug Fixes
* **security:** escape codegen strings and redact diagnostic body ([#930](https://github.com/jackwener/opencli/issues/930))
* **bilibili:** add missing domain for following cli ([#947](https://github.com/jackwener/opencli/issues/947))
* clean up stale `.ts` adapter files during upgrade ([#948](https://github.com/jackwener/opencli/issues/948))
* clean up legacy shim files and stale tmp files on upgrade ([#934](https://github.com/jackwener/opencli/issues/934))
* address deep review findings (security, correctness, consistency) ([#935](https://github.com/jackwener/opencli/issues/935))
* batch quality improvements — dedupe completion, unify logging, fix docs ([#945](https://github.com/jackwener/opencli/issues/945))
* graceful fallback when extension lacks network-capture support ([#865](https://github.com/jackwener/opencli/issues/865))
* handle missing electron executable gracefully ([#747](https://github.com/jackwener/opencli/issues/747))
* recover drifted tabs instead of abandoning them ([#715](https://github.com/jackwener/opencli/issues/715))
* retry on "No window with id" CDP error ([#892](https://github.com/jackwener/opencli/issues/892))
* **launcher:** graceful degradation and manual CDP override for Windows ([#744](https://github.com/jackwener/opencli/issues/744))
* **xiaohongshu:** scope note interaction selectors, replace blind retry with MutationObserver ([#839](https://github.com/jackwener/opencli/issues/839), [#730](https://github.com/jackwener/opencli/issues/730))
* **twitter:** relax reply composer timeout, use composer for text replies ([#862](https://github.com/jackwener/opencli/issues/862), [#860](https://github.com/jackwener/opencli/issues/860))
* **doubao:** preserve image URLs, connect to correct CDP target ([#708](https://github.com/jackwener/opencli/issues/708), [#674](https://github.com/jackwener/opencli/issues/674))
* **gemini:** stabilize ask reply state handling ([#735](https://github.com/jackwener/opencli/issues/735))
* **douban:** fix marks pagination and improve subject data extraction ([#752](https://github.com/jackwener/opencli/issues/752))
* **jianyu:** avoid early API bucket cutoff, stabilize search ([#916](https://github.com/jackwener/opencli/issues/916), [#912](https://github.com/jackwener/opencli/issues/912))
* **xiaoe:** resolve missing episodes for long courses via auto-scroll ([#904](https://github.com/jackwener/opencli/issues/904))
### Refactoring
* **adapters:** convert adapter layer from TypeScript to JavaScript ([#928](https://github.com/jackwener/opencli/issues/928))
* **adapters:** migrate all CLI adapters from YAML to TypeScript, then to JS ([#887](https://github.com/jackwener/opencli/issues/887), [#922](https://github.com/jackwener/opencli/issues/922))
* **validate:** switch from YAML-file scanning to registry-based validation ([#943](https://github.com/jackwener/opencli/issues/943))
* **strategy:** normalize strategy into runtime fields at registration time ([#941](https://github.com/jackwener/opencli/issues/941))
* **errors:** unify error output as YAML envelope to stderr ([#923](https://github.com/jackwener/opencli/issues/923))
* **daemon:** make daemon persistent, remove idle timeout ([#913](https://github.com/jackwener/opencli/issues/913))
* **browser:** unify browser error classification and deduplicate retry logic ([#908](https://github.com/jackwener/opencli/issues/908))
* **monorepo:** adapter separation — `clis/` at root ([#782](https://github.com/jackwener/opencli/issues/782))
* rename `operate` to `browser` ([#883](https://github.com/jackwener/opencli/issues/883))
* eliminate `any` types in core files ([#886](https://github.com/jackwener/opencli/issues/886))
* migrate adapter imports to package exports ([#795](https://github.com/jackwener/opencli/issues/795))
### Performance
* **P0 optimizations** — faster startup, reduced overhead ([#944](https://github.com/jackwener/opencli/issues/944))
* fast-path completion/version/shell-scripts to bypass full discovery ([#898](https://github.com/jackwener/opencli/issues/898))
* optimize browser pipeline — tab query dedup, parallel stealth, incremental snapshots ([#713](https://github.com/jackwener/opencli/issues/713))
* reduce round-trips in browser command hot path ([#712](https://github.com/jackwener/opencli/issues/712))
* skip blank page on first browser command ([#710](https://github.com/jackwener/opencli/issues/710))
### Documentation
* restructure README narrative ([#885](https://github.com/jackwener/opencli/issues/885))
* add Android Chrome usage guide ([#687](https://github.com/jackwener/opencli/issues/687))
* add Electron app CLI quickstart guide
* fix stale `.ts` references across skills and docs ([#954](https://github.com/jackwener/opencli/issues/954))
* unify skill command references and merge opencli-generate into opencli-explorer ([#891](https://github.com/jackwener/opencli/issues/891), [#894](https://github.com/jackwener/opencli/issues/894))
### Upgrade Guide
1. **Update Node.js** to v21 or later (v22 LTS recommended).
2. **Run `npm install -g @jackwener/opencli@latest`** — the preuninstall hook gracefully stops the old daemon; the first browser command after upgrade auto-restarts it.
3. **If you have custom `.ts` adapters** in `~/.opencli/clis/`, rename or compile them to `.js`. A warning will be printed on startup if stale `.ts` files are detected.
4. **If you have custom `.yaml` adapters**, convert them to JS using the `cli()` API (see `skills/opencli-adapter-author/references/adapter-template.md`).
5. **If you parse error output from stdout**, switch to stderr. Errors are now structured YAML envelopes with typed exit codes.
## [1.6.1](https://github.com/jackwener/opencli/compare/v1.6.0...v1.6.1) (2026-04-02)
@@ -13,7 +227,7 @@
### Features
* **opencli-operate:** add browser control commands for Claude Code skill ([#614](https://github.com/jackwener/opencli/issues/614))
* **opencli-browser:** add browser control commands for Claude Code skill ([#614](https://github.com/jackwener/opencli/issues/614))
* **docs:** add tab completion to getting started guides ([#658](https://github.com/jackwener/opencli/issues/658))
+44 -55
View File
@@ -18,7 +18,6 @@ npm run build
# 4. Run a few checks
npx tsc --noEmit
npm test
npm run test:adapter
# 5. Link globally (optional, for testing `opencli` command)
npm link
@@ -26,51 +25,45 @@ npm link
## Adding a New Site Adapter
This is the most common type of contribution. Start with YAML when possible, and use TypeScript only when you need browser-side logic or multi-step flows.
All adapters use TypeScript. Use the pipeline API for data-fetching commands, and `func()` for complex browser interactions.
### YAML Adapter (Recommended for data-fetching commands)
### Pipeline Adapter (Recommended for data-fetching commands)
Create a file like `clis/<site>/<command>.yaml`:
Create a file like `clis/<site>/<command>.js`:
```yaml
site: mysite
name: trending
description: Trending posts on MySite
domain: www.mysite.com
strategy: public # public | cookie | header
browser: false # true if browser session is needed
```typescript
import { cli, Strategy } from '@jackwener/opencli/registry';
args:
query:
positional: true
type: str
required: true
description: Search keyword
limit:
type: int
default: 20
description: Number of items
pipeline:
- fetch:
url: https://api.mysite.com/trending
- map:
rank: ${{ index + 1 }}
title: ${{ item.title }}
score: ${{ item.score }}
url: ${{ item.url }}
- limit: ${{ args.limit }}
columns: [rank, title, score, url]
cli({
site: 'mysite',
name: 'trending',
description: 'Trending posts on MySite',
domain: 'www.mysite.com',
strategy: Strategy.PUBLIC,
browser: false,
args: [
{ name: 'query', positional: true, required: true, help: 'Search keyword' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of items' },
],
columns: ['rank', 'title', 'score', 'url'],
pipeline: [
{ fetch: { url: 'https://api.mysite.com/trending' } },
{ map: {
rank: '${{ index + 1 }}',
title: '${{ item.title }}',
score: '${{ item.score }}',
url: '${{ item.url }}',
}},
{ limit: '${{ args.limit }}' },
],
});
```
See [`hackernews/top.yaml`](clis/hackernews/top.yaml) for a real example.
See [`hackernews/top.js`](clis/hackernews/top.js) for a real example.
### TypeScript Adapter (For complex browser interactions)
### func() Adapter (For complex browser interactions)
Create a file like `clis/<site>/<command>.ts`:
Create a file like `clis/<site>/<command>.js`:
```typescript
import { cli, Strategy } from '@jackwener/opencli/registry';
@@ -109,12 +102,12 @@ cli({
});
```
Use `opencli explore <url>` to discover APIs and see [opencli-explorer skill](./skills/opencli-explorer/SKILL.md) if you need the full adapter workflow.
Install the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md) if you need the full adapter workflow — recon → API discovery → field decoding → `opencli browser verify`.
### Validate Your Adapter
```bash
# Validate YAML syntax and schema
# Validate adapter
opencli validate
# Test your command
@@ -137,16 +130,12 @@ Use **positional** for the primary, required argument of a command (the "what"
Do **not** convert an argument to positional just because it appears first in the file. If the argument is optional, acts like a filter, or selects a mode/configuration, it should usually stay a named option.
YAML example:
```yaml
args:
query:
positional: true # ← primary arg, user types it directly
type: str
required: true
limit:
type: int # ← config arg, user types --limit 10
default: 20
Pipeline example:
```typescript
args: [
{ name: 'query', positional: true, required: true, help: 'Search query' }, // ← primary arg
{ name: 'limit', type: 'int', default: 20, help: 'Max results' }, // ← config arg
]
```
TS example:
@@ -162,8 +151,8 @@ args: [
See [TESTING.md](./TESTING.md) for the full guide and exact test locations.
```bash
npm test # Core unit tests (non-adapter)
npm run test:adapter # Focused adapter tests: zhihu/twitter/reddit/bilibili
npm test # Default local gate: unit + extension + adapter tests
npm run test:adapter # Adapter-only project (useful while iterating on adapters)
npx vitest run tests/e2e/ # E2E tests
npx vitest run # All tests
```
@@ -196,9 +185,9 @@ Common scopes: site name (`twitter`, `reddit`) or module name (`browser`, `pipel
3. Run the checks that apply:
```bash
npx tsc --noEmit # Type check
npm test # Core unit tests
npm run test:adapter # Focused adapter tests (if you touched adapter logic)
opencli validate # YAML validation (if applicable)
npm test # Default local gate: unit + extension + adapter
npm run test:adapter # Adapter-only project (optional while iterating on adapters)
opencli validate # Adapter validation
```
4. Commit using conventional commit format
5. Push and open a PR
+220 -95
View File
@@ -1,140 +1,249 @@
# OpenCLI
> **Make any website, Electron App, or Local Tool your CLI.**
> Zero risk · Reuse Chrome/Chromium login · AI-powered discovery · Universal CLI Hub
> **Turn websites, browser sessions, Electron apps, and local tools into deterministic interfaces for humans and AI agents.**
> Reuse your logged-in browser, automate live workflows, and crystallize repeated actions into reusable CLI commands.
[![中文文档](https://img.shields.io/badge/docs-%E4%B8%AD%E6%96%87-0F766E?style=flat-square)](./README.zh-CN.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
[![Node.js Version](https://img.shields.io/node/v/@jackwener/opencli?style=flat-square)](https://nodejs.org)
[![License](https://img.shields.io/npm/l/@jackwener/opencli?style=flat-square)](./LICENSE)
A CLI tool that turns **any website**, **Electron app**, or **local CLI tool** into a command-line interface — Bilibili, Zhihu, 小红书, Twitter/X, Reddit, YouTube, Antigravity, `gh`, `docker`, and [many more](#built-in-commands) — powered by browser session reuse and AI-native discovery.
OpenCLI gives you one surface for three different kinds of automation:
**Built for AI Agents** — Load the [`opencli-operate` skill](./skills/opencli-operate/SKILL.md) to give any AI agent (Claude Code, Cursor) direct browser control. Operate any website, then crystallize those interactions into reusable CLI commands. Configure `opencli list` in your `AGENT.md` or `.cursorrules` so the AI auto-discovers all available tools.
- **Use built-in adapters** for sites like Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, Twitter/X, and [many more](#built-in-commands).
- **Let AI Agents operate any website** — install the `opencli-adapter-author` skill in your AI agent (Claude Code, Cursor, etc.), and it can navigate, click, type, extract, and inspect any page through your logged-in browser via `opencli browser` primitives.
- **Write new adapters** end-to-end with `opencli browser` + the `opencli-adapter-author` skill, which guides from first recon through field decoding, code, and `opencli browser verify`.
**CLI Hub** — Register any local CLI (`opencli register mycli`) so AI agents can discover and call it alongside built-in commands. Auto-installs missing tools via your package manager (e.g. if `gh` isn't installed, `opencli gh ...` runs `brew install gh` first then re-executes seamlessly).
**CLI for Electron Apps** — Turn any Electron application into a CLI tool. Recombine, script, and extend apps like Antigravity Ultra from the terminal. AI agents can now control other AI apps natively.
---
It also works as a **CLI hub** for local tools such as `gh`, `docker`, and other binaries you register yourself, plus **desktop app adapters** for Electron apps like Cursor, Codex, Antigravity, ChatGPT, and Notion.
## Highlights
- **CLI All Electron** — CLI-ify apps like Antigravity Ultra! Now AI can control itself natively.
- **Browser Automation** — `operate` gives AI agents direct browser control: click, type, extract, screenshot — any interaction, fully scriptable.
- **Website → CLI** — Turn any website into a deterministic CLI: 70+ pre-built adapters, or crystallize your own with `opencli record`.
- **Desktop App Control** — Drive Electron apps (Cursor, Codex, ChatGPT, Notion, etc.) directly from the terminal via CDP.
- **Browser Automation for AI Agents** — Install the `opencli-adapter-author` skill, and your AI agent can operate any website: navigate, click, type, extract, screenshot — all through your logged-in Chrome session.
- **Multi-profile Browser Bridge** — Install the extension in each Chrome profile you want to use, then route commands with `--profile`, `OPENCLI_PROFILE`, or `opencli profile use`.
- **Website → CLI** — Turn any website into a deterministic CLI: 90+ pre-built adapters, or write your own with the `opencli-adapter-author` skill + `opencli browser verify`.
- **Account-safe** — Reuses Chrome/Chromium logged-in state; your credentials never leave the browser.
- **Anti-detection built-in** — Patches `navigator.webdriver`, stubs `window.chrome`, fakes plugin lists, cleans ChromeDriver/Playwright globals, and strips CDP frames from Error stack traces. Extensive anti-fingerprinting and risk-control evasion measures baked in at every layer.
- **AI Agent ready** — `explore` discovers APIs, `synthesize` generates adapters, `cascade` finds auth strategies, `operate` controls the browser directly.
- **External CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, obsidian, docker, etc). Zero setup.
- **Self-healing setup** — `opencli doctor` diagnoses and auto-starts the daemon, extension, and live browser connectivity.
- **Dynamic Loader** — Simply drop `.ts` or `.yaml` adapters into the `clis/` folder for auto-registration.
- **AI Agent ready** — One skill takes you from site recon through API discovery, field decoding, adapter writing, and verification.
- **CLI Hub** — Discover, auto-install, and passthrough commands to any external CLI (gh, docker, obsidian, etc).
- **Zero LLM cost** — No tokens consumed at runtime. Run 10,000 times and pay nothing.
- **Deterministic** — Same command, same output schema, every time. Pipeable, scriptable, CI-friendly.
- **Broad coverage** — 79+ sites across global and Chinese platforms (Bilibili, Zhihu, Xiaohongshu, Reddit, HackerNews, and more), plus desktop Electron apps via CDP.
---
## Quick Start
### 1. Install Browser Bridge Extension
> OpenCLI connects to your browser through a lightweight **Browser Bridge** Chrome/Chromium extension + micro-daemon (zero config, auto-start).
1. Go to the GitHub [Releases page](https://github.com/jackwener/opencli/releases) and download the latest `opencli-extension.zip`.
2. Unzip the file and open `chrome://extensions`, enable **Developer mode** (top-right toggle).
3. Click **Load unpacked** and select the unzipped folder.
### 2. Install OpenCLI
**Install via npm (recommended)**
### 1. Install OpenCLI
```bash
npm install -g @jackwener/opencli
```
# Install AI skills for Claude Code / Cursor
### 2. Install the Browser Bridge Extension
OpenCLI connects to Chrome/Chromium through a lightweight Browser Bridge extension plus a small local daemon. The daemon auto-starts when needed.
**Option A — Chrome Web Store (recommended):**
Install **OpenCLI** from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk).
**Option B — Manual install:**
1. Download the latest `opencli-extension-v{version}.zip` from the GitHub [Releases page](https://github.com/jackwener/opencli/releases).
2. Unzip it, open `chrome://extensions`, and enable **Developer mode**.
3. Click **Load unpacked** and select the unzipped folder.
### 3. Verify the setup
```bash
opencli doctor
```
### 4. Optional: name your Chrome profile
Each Chrome profile runs its own OpenCLI extension instance. If you use multiple Chrome profiles, list the connected profiles and assign local aliases:
```bash
opencli profile list
opencli profile rename <contextId> work
opencli profile use work
opencli --profile work browser state
```
With only one connected profile, OpenCLI uses it automatically. With multiple connected profiles and no default, OpenCLI asks you to choose instead of guessing.
### 5. Run your first commands
```bash
opencli list
opencli hackernews top --limit 5
opencli bilibili hot --limit 5
```
## For Humans
Use OpenCLI directly when you want a reliable command instead of a live browser session:
- `opencli list` shows every registered command.
- `opencli <site> <command>` runs a built-in or generated adapter.
- `opencli external register mycli` exposes a local CLI through the same discovery surface.
- `opencli doctor` helps diagnose browser connectivity.
## For AI Agents
OpenCLI's browser commands are designed to be used by AI Agents — not run manually. Install skills into your AI agent (Claude Code, Cursor, etc.), and the agent operates websites on your behalf using your logged-in Chrome session.
### Install skills
```bash
npx skills add jackwener/opencli
```
### 3. Verify & Try
Or install only what you need:
```bash
opencli doctor # Check extension + daemon connectivity
opencli daemon status # Check daemon state (PID, uptime, memory)
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
**Try it out:**
### Which skill to use
```bash
opencli list # See all commands
opencli hackernews top --limit 5 # Public API, no browser needed
opencli bilibili hot --limit 5 # Browser command (requires Extension)
```
| Skill | When to use | Example prompt to your AI agent |
|-------|------------|-------------------------------|
| **opencli-adapter-author** | Operate a site in real time, or write a reusable adapter for a new site | "Help me check my Xiaohongshu notifications" / "Write an adapter for douyin trending" / "Make a command that grabs the top posts from this page" |
| **opencli-autofix** | Repair a broken adapter when a built-in command fails | "`opencli zhihu hot` is returning empty — fix it" |
| **opencli-browser** | Browser automation reference for AI agents | "Use browser commands to scrape this page" |
| **opencli-usage** | Quick reference for all OpenCLI commands and sites | "What commands does OpenCLI have for Twitter?" |
| **smart-search** | Search across existing OpenCLI capabilities | "Find me a Bilibili trending adapter" |
### 4. Browser Automation — Make Websites Accessible for AI Agents
### How it works
Point your AI agent (Claude Code, Cursor) to [`skills/opencli-operate/SKILL.md`](./skills/opencli-operate/SKILL.md). It has everything needed — full command reference, examples, and workflow.
Once `opencli-adapter-author` is installed, your AI agent can:
Available commands: `open`, `state`, `click`, `type`, `select`, `keys`, `wait`, `get`, `screenshot`, `scroll`, `back`, `eval`, `network`, `init`, `verify`, `close`.
1. **Navigate** to any URL using your logged-in browser
2. **Read** page content via structured DOM snapshots (not screenshots)
3. **Interact** — click buttons, fill forms, select options, press keys
4. **Extract** data from the page or intercept network API responses
5. **Wait** for elements, text, or page transitions
### Update
The agent handles all the `opencli browser` commands internally — you just describe what you want done in natural language.
```bash
npm install -g @jackwener/opencli@latest
```
**Skill references:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — browser operation + adapter authoring, end-to-end
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — repair broken adapters
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — browser automation reference
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — command and site reference
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — capability search
### Install AI Skills
Available browser commands include `open`, `state`, `click`, `type`, `select`, `keys`, `wait`, `get`, `find`, `extract`, `frames`, `screenshot`, `scroll`, `back`, `eval`, `network`, `tab list`, `tab new`, `tab select`, `tab close`, `init`, `verify`, and `close`.
OpenCLI provides [skills](./skills/) for AI agents (Claude Code, etc.):
`opencli browser open <url>` and `opencli browser tab new [url]` both return a target ID. Use `opencli browser tab list` to inspect the target IDs of tabs that already exist, then pass `--tab <targetId>` to route a command to a specific tab. `tab new` creates a new tab without changing the default browser target; only `tab select <targetId>` promotes that tab to the default target for later untargeted `opencli browser ...` commands.
```bash
# Install all OpenCLI skills
npx skills add jackwener/opencli
## Core Concepts
# Or install specific skills
npx skills add jackwener/opencli --skill opencli-usage # Command reference
npx skills add jackwener/opencli --skill opencli-operate # Browser automation for AI agents
npx skills add jackwener/opencli --skill opencli-explorer # Adapter development guide
npx skills add jackwener/opencli --skill opencli-oneshot # Quick command reference
```
### `browser`: AI Agent browser control
---
`opencli browser` commands are the low-level primitives that AI Agents use to operate websites. You don't run these manually — instead, install the `opencli-adapter-author` skill into your AI agent, describe what you want in natural language, and the agent handles the browser operations.
### For Developers
For example, tell your agent: *"Help me check my Xiaohongshu notifications"* — the agent will use `opencli browser open`, `state`, `click`, etc. under the hood.
**Install from source**
### Built-in adapters: stable commands
```bash
git clone git@github.com:jackwener/opencli.git && cd opencli && npm install && npm run build && npm link
```
Use site-specific commands such as `opencli hackernews top` or `opencli reddit hot` when the capability already exists. These are deterministic and work without browser — ideal for both humans and AI agents.
**Load Source Browser Bridge Extension**
### Writing a new adapter
1. Open `chrome://extensions` and enable **Developer mode** (top-right toggle).
2. Click **Load unpacked** and select the `extension/` directory from this repository.
When the site you need is not yet covered, use the `opencli-adapter-author` skill. It takes the agent end-to-end:
---
1. Recon the site and classify its pattern (SPA / SSR / JSONP / Token / Streaming).
2. Discover the right endpoint — network inspection, initial state, bundle search, token trace, or interceptor fallback.
3. Decide the auth strategy — `PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`.
4. Decode response fields and design output columns.
5. `opencli browser init <site>/<name>` → write adapter → `opencli browser verify <site>/<name>`.
6. Persist site knowledge to `~/.opencli/sites/<site>/` so the next adapter for the same site is faster.
### CLI Hub and desktop adapters
OpenCLI is not only for websites. It can also:
- expose local binaries like `gh`, `docker`, `obsidian`, or custom tools through `opencli <tool> ...`
- control Electron desktop apps through dedicated adapters and CDP-backed integrations
## Prerequisites
- **Node.js**: >= 20.0.0 (or **Bun** >= 1.0)
- **Chrome or Chromium** running **and logged into the target site** (e.g. bilibili.com, zhihu.com, xiaohongshu.com, goofish.com).
- **Node.js**: >= 21.0.0 (or **Bun** >= 1.0)
- **Chrome or Chromium** running and logged into the target site for browser-backed commands
> **⚠️ Important**: Browser commands reuse your Chrome/Chromium login session. You must be logged into the target website in Chrome or Chromium before running commands. If you get empty data or errors, check your login status first.
> **Important**: Browser-backed commands reuse your Chrome/Chromium login session. If you get empty data or permission-like failures, first confirm the site is already open and authenticated in Chrome/Chromium.
## Configuration
| Variable | Default | Description |
|----------|---------|-------------|
| `OPENCLI_DAEMON_PORT` | `19825` | HTTP port for the daemon-extension bridge |
| `OPENCLI_PROFILE` | — | Browser Bridge profile alias/contextId to use when multiple Chrome profiles are connected |
| `OPENCLI_WINDOW_FOCUSED` | `false` | Set to `1` to open the automation container in the foreground (useful for debugging). The `--focus` flag sets this. |
| `OPENCLI_LIVE` | `false` | Set to `1` to keep the automation lease open after an adapter command finishes (useful for inspection). The `--live` flag sets this. |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | Seconds to wait for browser connection |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | Seconds to wait for a single browser command |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol endpoint for remote browser or Electron apps |
| `OPENCLI_CDP_TARGET` | — | Filter CDP targets by URL substring (e.g. `detail.1688.com`) |
| `OPENCLI_VERBOSE` | `false` | Enable verbose logging (`-v` flag also works) |
| `OPENCLI_DIAGNOSTIC` | `false` | Set to `1` to capture structured diagnostic context on failures |
| `DEBUG_SNAPSHOT` | — | Set to `1` for DOM snapshot debug output |
`--focus` works for both `opencli browser *` and browser-backed adapter commands. `--live` is mainly for adapter commands: browser subcommands already keep the automation lease open until you run `opencli browser close` or the idle timeout expires.
## Update
```bash
npm install -g @jackwener/opencli@latest
# If you use the packaged OpenCLI skills, refresh them too
npx skills add jackwener/opencli
```
Or refresh only the skills you actually use:
```bash
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
## For Developers
Install from source:
```bash
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link
```
To load the source Browser Bridge extension:
1. Open `chrome://extensions` and enable **Developer mode**.
2. Click **Load unpacked** and select this repository's `extension/` directory.
## Built-in Commands
| Site | Commands |
|------|----------|
| **xiaohongshu** | `search` `note` `comments` `feed` `user` `download` `publish` `notifications` `creator-notes` `creator-notes-summary` `creator-note-detail` `creator-profile` `creator-stats` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `user-videos` |
| **bilibili** | `hot` `search` `history` `feed` `ranking` `download` `comments` `dynamic` `favorite` `following` `me` `subtitle` `video` `user-videos` |
| **tieba** | `hot` `posts` `search` `read` |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` |
| **twitter** | `trending` `search` `timeline` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `post` `download` `profile` `article` `like` `likes` `notifications` `reply` `reply-dm` `thread` `follow` `unfollow` `followers` `following` `block` `unblock` `bookmark` `unbookmark` `delete` `hide-reply` `accept` |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `upvoted` `save` `saved` `comment` `subscribe` |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` |
| **1688** | `search` `item` `assets` `download` `store` |
| **gitee** | `trending` `search` `user` |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` |
| **yuanbao** | `new` `ask` |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` |
@@ -142,8 +251,19 @@ git clone git@github.com:jackwener/opencli.git && cd opencli && npm install && n
| **xianyu** | `search` `item` `chat` |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` |
| **uiverse** | `code` `preview` |
| **baidu-scholar** | `search` |
| **google-scholar** | `search` `cite` `profile` |
| **gov-law** | `search` `recent` |
| **gov-policy** | `search` `recent` |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` |
| **wanfang** | `search` |
| **hackernews** | `top` `new` `best` `ask` `show` `jobs` `search` `user` |
| **xiaoyuzhou** | `auth*` `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` |
79+ adapters in total — **[→ see all supported sites & commands](./docs/adapters/index.md)**
90+ adapters in total — **[→ see all supported sites & commands](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast`, `podcast-episodes`, `episode`, `download`, and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
## CLI Hub
@@ -155,14 +275,14 @@ OpenCLI acts as a universal hub for your existing command-line tools — unified
| **obsidian** | Obsidian vault management | `opencli obsidian search query="AI"` |
| **docker** | Docker | `opencli docker ps` |
| **lark-cli** | Lark/Feishu — messages, docs, calendar, tasks, 200+ commands | `opencli lark-cli calendar +agenda` |
| **dingtalk** | DingTalk — cross-platform CLI for DingTalk's full suite, designed for humans and AI agents | `opencli dingtalk msg send --to user "hello"` |
| **wecom** | WeCom/企业微信 — CLI for WeCom open platform, for humans and AI agents | `opencli wecom msg send --to user "hello"` |
| **dws** | DingTalk — cross-platform CLI for DingTalk's full suite, designed for humans and AI agents | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | WeCom/企业微信 — CLI for WeCom open platform, for humans and AI agents | `opencli wecom-cli msg send --to user "hello"` |
| **vercel** | Vercel — deploy projects, manage domains, env vars, logs | `opencli vercel deploy --prod` |
**Register your own** — add any local CLI so AI agents can discover it via `opencli list`:
```bash
opencli register mycli
opencli external register mycli
```
### Desktop App Adapters
@@ -174,7 +294,7 @@ Control Electron desktop apps directly from the terminal. Each adapter has its o
| **Cursor** | Control Cursor IDE — Composer, chat, code extraction | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | Drive OpenAI Codex CLI agent headlessly | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | Control Antigravity Ultra from terminal | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT** | Automate ChatGPT macOS desktop app | [Doc](./docs/adapters/desktop/chatgpt.md) |
| **ChatGPT App** | Automate ChatGPT macOS desktop app | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | Multi-LLM client (GPT-4, Claude, Gemini) | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | Search, read, write Notion pages | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord Desktop — messages, channels, servers | [Doc](./docs/adapters/desktop/discord.md) |
@@ -194,18 +314,24 @@ OpenCLI supports downloading images, videos, and articles from supported platfor
| **douban** | Images | Poster / still image lists |
| **pixiv** | Images | Original-quality illustrations, multi-page |
| **1688** | Images, Videos | Downloads page-visible product media from item pages |
| **xiaoyuzhou** | Audio, Transcript | Downloads episode audio and transcript JSON/text with local credentials |
| **zhihu** | Articles (Markdown) | Exports with optional image download |
| **weixin** | Articles (Markdown) | WeChat Official Account articles |
For video downloads, install `yt-dlp` first: `brew install yt-dlp`
```bash
opencli xiaohongshu download abc123 --output ./xhs
opencli xiaohongshu download "https://www.xiaohongshu.com/search_result/<id>?xsec_token=..." --output ./xhs
opencli xiaohongshu download "https://xhslink.com/..." --output ./xhs
opencli bilibili download BV1xxx --output ./bilibili
opencli twitter download elonmusk --limit 20 --output ./twitter
opencli 1688 download 841141931191 --output ./1688-downloads
opencli xiaoyuzhou download 69b3b675772ac2295bfc01d0 --output ./xiaoyuzhou
opencli xiaoyuzhou transcript 69dd0c98e2c8be31551f6a33 --output ./xiaoyuzhou-transcripts
```
`opencli xiaoyuzhou download` and `transcript` require local Xiaoyuzhou credentials in `~/.opencli/xiaoyuzhou.json`.
## Output Formats
All built-in commands support `--format` / `-f` with `table` (default), `json`, `yaml`, `md`, and `csv`.
@@ -234,8 +360,8 @@ opencli follows Unix `sysexits.h` conventions so it integrates naturally with sh
```bash
opencli spotify status || echo "exit $?" # 69 if browser not running
opencli github issues 2>/dev/null
[ $? -eq 77 ] && opencli github auth # auto-auth if not logged in
opencli gh issue list 2>/dev/null
[ $? -eq 77 ] && opencli gh auth login # auto-auth if not logged in
```
## Plugins
@@ -251,25 +377,24 @@ opencli plugin uninstall my-tool
| Plugin | Type | Description |
|--------|------|-------------|
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | YAML | GitHub Trending repositories |
| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | TS | Multi-platform trending aggregator |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | YAML | 稀土掘金 (Juejin) hot articles |
| [opencli-plugin-vk](https://github.com/flobo3/opencli-plugin-vk) | TS | VK (VKontakte) wall, feed, and search |
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | JS | GitHub Trending repositories |
| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | JS | Multi-platform trending aggregator |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | JS | 稀土掘金 (Juejin) hot articles |
| [opencli-plugin-vk](https://github.com/flobo3/opencli-plugin-vk) | JS | VK (VKontakte) wall, feed, and search |
See [Plugins Guide](./docs/guide/plugins.md) for creating your own plugin.
## For AI Agents (Developer Guide)
> **Quick mode**: To generate a single command for a specific page URL, see [opencli-oneshot skill](./skills/opencli-oneshot/SKILL.md) — just a URL + one-line goal, 4 steps done.
Before writing any adapter code, read the [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md). It takes you end-to-end:
> **Full mode**: Before writing any adapter code, read [opencli-explorer skill](./skills/opencli-explorer/SKILL.md). It contains the complete browser exploration workflow, the 5-tier authentication strategy decision tree, and debugging guide.
- Recon the site and pick a pattern (SPA / SSR / JSONP / Token / Streaming).
- Discover the right endpoint via `opencli browser network`, `eval`, or the interceptor fallback.
- Decide auth strategy (`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`).
- Decode response fields, design columns, scaffold with `opencli browser init`.
- Verify with `opencli browser verify <site>/<name>` before shipping.
```bash
opencli explore https://example.com --site mysite # Discover APIs + capabilities
opencli synthesize mysite # Generate YAML adapters
opencli generate https://example.com --goal "hot" # One-shot: explore → synthesize → register
opencli cascade https://api.example.com/data # Auto-probe: PUBLIC → COOKIE → HEADER
```
Adapters you write outside the repo live at `~/.opencli/clis/<site>/<name>.js`. Site knowledge (endpoints, field maps, fixtures) accumulates in `~/.opencli/sites/<site>/` so the next adapter for the same site starts from context instead of zero.
## Testing
@@ -277,10 +402,10 @@ See **[TESTING.md](./TESTING.md)** for how to run and write tests.
## Troubleshooting
- **"Extension not connected"** — Ensure the Browser Bridge extension is installed and **enabled** in `chrome://extensions` in Chrome or Chromium.
- **"Extension not connected"** — Ensure the Browser Bridge extension is installed from the [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) and **enabled** in `chrome://extensions`.
- **"attach failed: Cannot access a chrome-extension:// URL"** — Another extension may be interfering. Try disabling other extensions temporarily.
- **Empty data or 'Unauthorized' error** — Your Chrome/Chromium login session may have expired. Navigate to the target site and log in again.
- **Node API errors** — Ensure Node.js >= 20. Some dependencies require modern Node APIs.
- **Node API errors** — Ensure Node.js >= 21. Some features require `node:util` styleText (stable in Node 21+).
- **Daemon issues** — Check status: `curl localhost:19825/status` · View logs: `curl localhost:19825/logs`
## Star History
+223 -124
View File
@@ -1,140 +1,228 @@
# OpenCLI
> **把任何网站、本地工具、Electron 应用变成能够让 AI 调用的命令行!**
> 零风控 · 复用 Chrome/Chromium 登录 · AI 自动发现接口 · 全能 CLI 枢纽
> **把网站、浏览器会话、Electron 应用和本地工具,统一变成适合人类与 AI Agent 使用的确定性接口。**
> 复用浏览器登录态,先自动化真实操作,再把高频流程沉淀成可复用的 CLI 命令。
[![English](https://img.shields.io/badge/docs-English-1D4ED8?style=flat-square)](./README.md)
[![npm](https://img.shields.io/npm/v/@jackwener/opencli?style=flat-square)](https://www.npmjs.com/package/@jackwener/opencli)
[![Node.js Version](https://img.shields.io/node/v/@jackwener/opencli?style=flat-square)](https://nodejs.org)
[![License](https://img.shields.io/npm/l/@jackwener/opencli?style=flat-square)](./LICENSE)
OpenCLI 将任何网站、本地 CLI 或 Electron 应用(如 Antigravity)变成命令行工具 — B站、知乎、小红书、Twitter/X、Reddit、YouTube,以及 `gh``docker` 等[多种站点与工具](#内置命令) — 复用浏览器登录态,AI 驱动探索。
OpenCLI 可以用同一套 CLI 做三类事情:
**专为 AI Agent 打造**:加载 [`opencli-operate` skill](./skills/opencli-operate/SKILL.md),赋予 AI AgentClaude Code、Cursor 等)直接操控浏览器的能力——操作任意网站,并将这些交互沉淀为可复用的 CLI 命令。在 `AGENT.md``.cursorrules` 中配置 `opencli list`AI 即可自动发现并调用所有可用工具
- **直接使用现成适配器**:B站、知乎、小红书、Twitter/X、Reddit、HackerNews 等 [90+ 站点](#内置命令) 开箱即用
- **让 AI Agent 操作任意网站**:在你的 AI AgentClaude Code、Cursor 等)中安装 `opencli-adapter-author` skill,Agent 就能用你的已登录浏览器导航、点击、输入、提取任意网页内容。
- **把新网站写成 CLI**:用 `opencli browser` 原语 + `opencli-adapter-author` skill,从站点侦察、API 发现、字段解码到 `opencli browser verify` 一条龙。
**opencli 支持 CLI 化所有 electron 应用!最强大更新来袭!**
CLI all electron!现在支持把所有 electron 应用 CLI 化,从而组合出各种神奇的能力。
如果你在使用诸如 Antigravity Ultra 等工具时觉得不够灵活或难以扩展,现在通过 OpenCLI 把他 CLI 化,轻松打破界限。
现在,**AI 可以自己控制自己**!结合 cc/openclaw 就可以远程控制任何 electron 应用!无限玩法!!
---
除了网站能力,OpenCLI 还是一个 **CLI 枢纽**:你可以把 `gh``docker` 等本地工具统一注册到 `opencli` 下,也可以通过桌面端适配器控制 Cursor、Codex、Antigravity、ChatGPT、Notion 等 Electron 应用。
## 亮点
- **CLI All Electron** — 支持把所有 electron 应用(如 Antigravity UltraCLI 化,让 AI 控制自己!
- **浏览器自动化** — `operate` 赋予 AI Agent 直接操控浏览器的能力:点击、输入、提取、截图,任意交互皆可脚本化
- **网页转 CLI** — 将任意网站变成确定性命令行工具:79+ 置适配器,或用 `opencli record` 沉淀自己的操作
- **多站点覆盖** — 79+ 站点,横跨全球与中国平台(B站、知乎、小红书、Reddit、HackerNews 等),并支持通过 CDP 控制桌面 Electron 应用
- **零风控** — 复用 Chrome/Chromium 登录态,无需存储任何凭证
- **外部 CLI 枢纽** — 统一发现、自动安装、透传执行 `gh``docker` 等本地 CLI
- **自修复配置** — `opencli doctor` 自动启动 daemon,诊断扩展和浏览器连接状态
- **AI 原生** — `explore` 自动发现 API`synthesize` 生成适配器,`cascade` 探测认证策略,`operate` 直接控制浏览器
- **零 LLM 成本** — 运行时不消耗任何 token,跑一万次不花一分钱
- **确定性** — 同一命令永远返回同一结构,可管道化、可脚本化、CI 友好
## 前置要求
- **Node.js**: >= 20.0.0
- **Chrome 或 Chromium** 浏览器正在运行,且**已登录目标网站**(如 bilibili.com、zhihu.com、xiaohongshu.com、goofish.com
> **⚠️ 重要**:大多数命令复用你的 Chrome/Chromium 登录状态。运行命令前,你必须已在 Chrome 或 Chromium 中打开目标网站并完成登录。如果获取到空数据或报错,请先检查你的浏览器登录状态。
OpenCLI 通过轻量化的 **Browser Bridge** Chrome/Chromium 扩展 + 微型 daemon 与浏览器通信(零配置,自动启动)。
### Browser Bridge 扩展配置
你可以选择以下任一方式安装扩展:
**方式一:下载构建好的安装包(推荐)**
1. 到 GitHub [Releases 页面](https://github.com/jackwener/opencli/releases) 下载最新的 `opencli-extension.zip`
2. 解压后在 Chrome 或 Chromium 中打开 `chrome://extensions`,启用右上角的 **开发者模式**
3. 点击 **加载已解压的扩展程序**,选择解压后的文件夹。
**方式二:加载源码(针对开发者)**
1. 同样在 `chrome://extensions` 开启 **开发者模式**
2. 点击 **加载已解压的扩展程序**,选择本仓库代码树中的 `extension/` 文件夹。
完成!运行任何 opencli 浏览器命令时,后台微型 daemon 会自动启动与浏览器通信。无需配 API Token,零代码配置。
> **Tip**:后续诊断和 daemon 管理:
> ```bash
> opencli doctor # 检查扩展和 daemon 连通性
> opencli daemon status # 查看 daemon 状态
> opencli daemon stop # 停止 daemon
> ```
- **桌面应用控制** — 通过 CDP 直接在终端驱动 Electron 应用(Cursor、Codex、ChatGPT、Notion 等)。
- **AI Agent 浏览器自动化** — 安装 `opencli-adapter-author` skill,你的 AI Agent 就能操作任意网站:导航、点击、输入、提取、截图——全部通过你的已登录 Chrome 会话完成。
- **网站 → CLI** — 把任何网站变成确定性 CLI90+ 置适配器,或用 `opencli-adapter-author` skill + `opencli browser verify` 自己写。
- **账号安全** — 复用 Chrome/Chromium 登录态,凭证永远不会离开浏览器。
- **面向 AI Agent** — 一个 skill 带你走完站点侦察、API 发现、字段解码、适配器编写、验证的全流程。
- **CLI 枢纽** — 统一发现、自动安装、透传任何外部 CLIgh、docker、obsidian 等)。
- **零 LLM 成本** — 运行时不消耗模型 token,跑 10,000 次也不花一分钱。
- **确定性输出** — 相同命令,相同输出结构,每次一致。可管道、可脚本、CI 友好。
## 快速开始
### npm 全局安装(推荐)
### 1. 安装 OpenCLI
```bash
npm install -g @jackwener/opencli
```
# 安装 AI SkillsClaude Code / Cursor
### 2. 安装 Browser Bridge 扩展
OpenCLI 通过轻量 Browser Bridge 扩展和本地微型 daemon 与 Chrome/Chromium 通信。daemon 会按需自动启动。
**方式 A — Chrome Web Store(推荐):**
在 [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) 安装 **OpenCLI** 扩展。
**方式 B — 手动安装:**
1. 到 GitHub [Releases 页面](https://github.com/jackwener/opencli/releases) 下载最新的 `opencli-extension-v{version}.zip`
2. 解压后打开 `chrome://extensions`,启用 **开发者模式**
3. 点击 **加载已解压的扩展程序**,选择解压后的目录。
### 3. 验证环境
```bash
opencli doctor
```
### 4. 跑第一个命令
```bash
opencli list
opencli hackernews top --limit 5
opencli bilibili hot --limit 5
```
## 给人类用户
如果你只是想稳定地调用网站或桌面应用能力,主路径很简单:
- `opencli list` 查看当前所有命令
- `opencli <site> <command>` 调用内置或生成好的适配器
- `opencli register mycli` 把本地 CLI 接入同一发现入口
- `opencli doctor` 处理浏览器连通性问题
## 给 AI Agent
OpenCLI 的 browser 命令是给 AI Agent 用的——不是手动执行的。把 skill 安装到你的 AI AgentClaude Code、Cursor 等)中,Agent 就能用你的已登录 Chrome 会话替你操作网站。
### 安装 skill
```bash
npx skills add jackwener/opencli
```
直接使用
或只装需要的 skill
```bash
opencli list # 查看所有命令
opencli list -f yaml # 以 YAML 列出所有命令
opencli hackernews top --limit 5 # 公共 API,无需浏览器
opencli bilibili hot --limit 5 # 浏览器命令
opencli zhihu hot -f json # JSON 输出
opencli zhihu hot -f yaml # YAML 输出
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill opencli-browser
npx skills add jackwener/opencli --skill opencli-usage
npx skills add jackwener/opencli --skill smart-search
```
### 从源码安装(面向开发者)
### 选择哪个 skill
```bash
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link # 链接到全局环境
opencli list # 可以在任何地方使用了!
```
| Skill | 适用场景 | 你对 AI Agent 说的话 |
|-------|---------|-------------------|
| **opencli-adapter-author** | 实时操作任意网站,或为新站点写可复用适配器 | "帮我看看小红书的通知" / "帮我做一个抖音热门的适配器" / "帮我做一个抓取这个页面热帖的命令" |
| **opencli-autofix** | 内置命令失败时修复已有适配器 | "`opencli zhihu hot` 返回空了,修一下" |
| **opencli-browser** | 浏览器自动化参考文档 | "用浏览器命令抓取这个页面" |
| **opencli-usage** | 所有命令和站点的快速参考 | "OpenCLI 有哪些 Twitter 相关的命令?" |
| **smart-search** | 在现有 OpenCLI 能力里搜索 | "帮我找个 B 站热门相关的适配器" |
### 更新
### 工作原理
安装 `opencli-adapter-author` skill 后,你的 AI Agent 可以:
1. **导航**到任意 URL,使用你的已登录浏览器
2. **读取**页面内容——通过结构化 DOM 快照(不是截图)
3. **交互**——点击按钮、填写表单、选择选项、按键
4. **提取**页面数据或拦截网络 API 响应
5. **等待**元素、文本或页面跳转
Agent 在内部自动处理所有 `opencli browser` 命令——你只需用自然语言描述想做的事。
**Skill 参考文档:**
- [`skills/opencli-adapter-author/SKILL.md`](./skills/opencli-adapter-author/SKILL.md) — 浏览器操作 + 适配器编写,全流程
- [`skills/opencli-autofix/SKILL.md`](./skills/opencli-autofix/SKILL.md) — 修复已有适配器
- [`skills/opencli-browser/SKILL.md`](./skills/opencli-browser/SKILL.md) — 浏览器自动化参考
- [`skills/opencli-usage/SKILL.md`](./skills/opencli-usage/SKILL.md) — 命令和站点参考
- [`skills/smart-search/SKILL.md`](./skills/smart-search/SKILL.md) — 能力搜索
`browser` 可用命令包括:`open``state``click``type``select``keys``wait``get``find``extract``frames``screenshot``scroll``back``eval``network``tab list``tab new``tab select``tab close``init``verify``close`
`opencli browser open <url>``opencli browser tab new [url]` 都会返回 target ID。`opencli browser tab list` 用来查看当前已存在 tab 的 target ID,再通过 `--tab <targetId>` 把命令明确路由到某个 tab。`tab new` 只会新建 tab,不会改变默认浏览器目标;只有显式执行 `tab select <targetId>`,才会把该 tab 设为后续未指定 target 的 `opencli browser ...` 命令的默认目标。
## 核心概念
### `browser`AI Agent 的浏览器控制层
`opencli browser` 命令是 AI Agent 操作网站的底层原语。你不需要手动运行这些命令——把 `opencli-adapter-author` skill 安装到你的 AI Agent 中,用自然语言描述你想做的事,Agent 会自动处理浏览器操作。
比如你告诉 Agent:*"帮我看看小红书的通知"*——Agent 会在底层调用 `opencli browser open``state``click` 等命令。
### 内置适配器:稳定命令
当某个站点能力已经存在时,优先使用 `opencli hackernews top``opencli reddit hot` 这类稳定命令。这些命令是确定性的,无需浏览器——人类和 AI Agent 都可以直接使用。
### 为新站点写适配器
当你需要的网站还没覆盖时,用 `opencli-adapter-author` skill,它会把 Agent 带到闭环:
1. 侦察站点,分类 patternSPA / SSR / JSONP / Token / Streaming
2. 发现目标 endpoint——network 精读、initial state、bundle 搜索、token 溯源,或 interceptor 兜底
3. 定认证策略——`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
4. 字段解码 + 设计输出列
5. `opencli browser init <site>/<name>` → 写适配器 → `opencli browser verify <site>/<name>`
6. 把站点知识沉到 `~/.opencli/sites/<site>/`,下次写同站点的其他命令直接吃缓存
### CLI 枢纽与桌面端适配器
OpenCLI 不只是网站 CLI,还可以:
- 统一代理本地二进制工具,例如 `gh``docker``obsidian`
- 通过专门适配器和 CDP 集成控制 Electron 桌面应用
## 前置要求
- **Node.js**: >= 21.0.0
- 浏览器型命令需要 Chrome 或 Chromium 处于运行中,并已登录目标网站
> **重要**:浏览器型命令直接复用你的 Chrome/Chromium 登录态。如果拿到空数据或出现权限类失败,先确认目标站点已经在浏览器里打开并完成登录。
## 配置
| 变量 | 默认值 | 说明 |
|------|--------|------|
| `OPENCLI_DAEMON_PORT` | `19825` | daemon-extension 通信端口 |
| `OPENCLI_WINDOW_FOCUSED` | `false` | 设为 `1` 时 automation 窗口在前台打开(适合调试)。`--focus` 标志会设置此变量 |
| `OPENCLI_LIVE` | `false` | 设为 `1` 时 adapter 命令执行完后保留 automation 窗口不关闭(适合检查页面)。`--live` 标志会设置此变量 |
| `OPENCLI_BROWSER_CONNECT_TIMEOUT` | `30` | 浏览器连接超时(秒) |
| `OPENCLI_BROWSER_COMMAND_TIMEOUT` | `60` | 单个浏览器命令超时(秒) |
| `OPENCLI_CDP_ENDPOINT` | — | Chrome DevTools Protocol 端点,用于远程浏览器或 Electron 应用 |
| `OPENCLI_CDP_TARGET` | — | 按 URL 子串过滤 CDP target(如 `detail.1688.com` |
| `OPENCLI_VERBOSE` | `false` | 启用详细日志(`-v` 也可以) |
| `OPENCLI_DIAGNOSTIC` | `false` | 设为 `1` 时在失败时输出结构化诊断上下文 |
| `DEBUG_SNAPSHOT` | — | 设为 `1` 输出 DOM 快照调试信息 |
`--focus` 同时适用于 `opencli browser *` 和浏览器型 adapter 命令。`--live` 主要是给 adapter 命令用的:`browser` 子命令本来就会一直保留 automation window,直到你手动执行 `opencli browser close` 或等空闲超时。
## 更新
```bash
npm install -g @jackwener/opencli@latest
# 如果你在用打包发布的 OpenCLI skills,也一起刷新
npx skills add jackwener/opencli
```
### 浏览器自动化 — 让 AI Agent 直接控制浏览器
将 [`skills/opencli-operate/SKILL.md`](./skills/opencli-operate/SKILL.md) 指向你的 AI AgentClaude Code、Cursor),即可开箱即用,内含完整命令参考与使用示例。
可用命令:`open``state``click``type``select``keys``wait``get``screenshot``scroll``back``eval``network``init``verify``close`
### 安装 AI Skills
OpenCLI 提供 [skills](./skills/) 供 AI AgentClaude Code 等)使用:
如果你只装了部分 skill,也可以只刷新自己在用的:
```bash
# 安装所有 OpenCLI skills
npx skills add jackwener/opencli
# 或安装特定 skill
npx skills add jackwener/opencli --skill opencli-usage # 命令参考
npx skills add jackwener/opencli --skill opencli-operate # 浏览器自动化(AI Agent 专用)
npx skills add jackwener/opencli --skill opencli-explorer # 适配器开发指南
npx skills add jackwener/opencli --skill opencli-oneshot # 快速命令参考
npx skills add jackwener/opencli --skill opencli-adapter-author
npx skills add jackwener/opencli --skill opencli-autofix
npx skills add jackwener/opencli --skill smart-search
```
## 面向开发者
从源码安装:
```bash
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link
```
加载源码版 Browser Bridge 扩展:
1. 打开 `chrome://extensions` 并启用 **开发者模式**
2. 点击 **加载已解压的扩展程序**,选择本仓库里的 `extension/` 目录
## 内置命令
运行 `opencli list` 查看完整注册表。
| 站点 | 命令 | 模式 |
|------|------|------|
| **twitter** | `trending` `bookmarks` `profile` `search` `timeline` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` | 浏览器 |
| **twitter** | `trending` `search` `timeline` `tweets` `lists` `list-tweets` `list-add` `list-remove` `bookmarks` `profile` `thread` `following` `followers` `notifications` `post` `reply` `delete` `like` `likes` `article` `follow` `unfollow` `bookmark` `unbookmark` `download` `accept` `reply-dm` `block` `unblock` `hide-reply` | 浏览器 |
| **reddit** | `hot` `frontpage` `popular` `search` `subreddit` `read` `user` `user-posts` `user-comments` `upvote` `save` `comment` `subscribe` `saved` `upvoted` | 浏览器 |
| **tieba** | `hot` `posts` `search` `read` | 浏览器 |
| **hupu** | `hot` `search` `detail` `mentions` `reply` `like` `unlike` | 浏览器 |
| **cursor** | `status` `send` `read` `new` `dump` `composer` `model` `extract-code` `ask` `screenshot` `history` `export` | 桌面端 |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `dynamic` `ranking` `following` `user-videos` `download` | 浏览器 |
| **bilibili** | `hot` `search` `me` `favorite` `history` `feed` `subtitle` `video` `comments` `dynamic` `ranking` `following` `user-videos` `download` | 浏览器 |
| **codex** | `status` `send` `read` `new` `dump` `extract-diff` `model` `ask` `screenshot` `history` `export` | 桌面端 |
| **chatwise** | `status` `new` `send` `read` `ask` `model` `history` `export` `screenshot` | 桌面端 |
| **doubao** | `status` `new` `send` `read` `ask` `history` `detail` `meeting-summary` `meeting-transcript` | 浏览器 |
@@ -143,16 +231,23 @@ npx skills add jackwener/opencli --skill opencli-oneshot # 快速命令参
| **discord-app** | `status` `send` `read` `channels` `servers` `search` `members` | 桌面端 |
| **v2ex** | `hot` `latest` `topic` `node` `user` `member` `replies` `nodes` `daily` `me` `notifications` | 公开 / 浏览器 |
| **xueqiu** | `feed` `hot-stock` `hot` `search` `stock` `comments` `watchlist` `earnings-date` `fund-holdings` `fund-snapshot` | 浏览器 |
| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` | 桌面端 |
| **chatgpt** | `status` `new` `send` `read` `ask` `model` | 桌面端 |
| **xiaohongshu** | `search` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` | 浏览器 |
| **antigravity** | `status` `send` `read` `new` `dump` `extract-code` `model` `watch` `serve` | 桌面端 |
| **chatgpt-app** | `status` `new` `send` `read` `ask` `model` | 桌面端 |
| **xiaohongshu** | `search` `note` `comments` `notifications` `feed` `user` `download` `publish` `creator-notes` `creator-note-detail` `creator-notes-summary` `creator-profile` `creator-stats` | 浏览器 |
| **xiaoe** | `courses` `detail` `catalog` `play-url` `content` | 浏览器 |
| **quark** | `ls` `mkdir` `mv` `rename` `rm` `save` `share-tree` | 浏览器 |
| **uiverse** | `code` `preview` | 浏览器 |
| **apple-podcasts** | `search` `episodes` `top` | 公开 |
| **xiaoyuzhou** | `podcast` `podcast-episodes` `episode` | 公开 |
| **baidu-scholar** | `search` | 公开 |
| **google-scholar** | `search` `cite` `profile` | 公开 |
| **gov-law** | `search` `recent` | 公开 |
| **gov-policy** | `search` `recent` | 公开 |
| **nowcoder** | `hot` `trending` `topics` `recommend` `creators` `companies` `jobs` `search` `suggest` `experience` `referral` `salary` `papers` `practice` `notifications` `detail` | 公开 / 浏览器 |
| **wanfang** | `search` | 公开 |
| **xiaoyuzhou** | `podcast*` `podcast-episodes*` `episode*` `download*` `transcript*` `auth` | 本地凭证 |
| **zhihu** | `hot` `search` `question` `download` `follow` `like` `favorite` `comment` `answer` | 浏览器 |
| **weixin** | `download` | 浏览器 |
| **youtube** | `search` `video` `transcript` | 浏览器 |
| **youtube** | `search` `video` `transcript` `comments` `channel` `playlist` `feed` `history` `watch-later` `subscriptions` `like` `unlike` `subscribe` `unsubscribe` | 浏览器 |
| **boss** | `search` `detail` `recommend` `joblist` `greet` `batchgreet` `send` `chatlist` `chatmsg` `invite` `mark` `exchange` `resume` `stats` | 浏览器 |
| **coupang** | `search` `add-to-cart` | 浏览器 |
| **bbc** | `news` | 公共 API |
@@ -174,7 +269,7 @@ npx skills add jackwener/opencli --skill opencli-oneshot # 快速命令参
| **sinafinance** | `news` | 🌐 公开 |
| **barchart** | `quote` `options` `greeks` `flow` | 浏览器 |
| **chaoxing** | `assignments` `exams` | 浏览器 |
| **grok** | `ask` | 浏览器 |
| **grok** | `ask` `image` | 浏览器 |
| **hf** | `top` | 公开 |
| **jike** | `feed` `search` `create` `like` `comment` `repost` `notifications` `post` `topic` `user` | 浏览器 |
| **jimeng** | `generate` `history` | 浏览器 |
@@ -186,8 +281,9 @@ npx skills add jackwener/opencli --skill opencli-oneshot # 快速命令参
| **douban** | `search` `top250` `subject` `photos` `download` `marks` `reviews` `movie-hot` `book-hot` | 浏览器 |
| **facebook** | `feed` `profile` `search` `friends` `groups` `events` `notifications` `memories` `add-friend` `join-group` | 浏览器 |
| **google** | `news` `search` `suggest` `trends` | 公开 |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` | 浏览器 |
| **amazon** | `bestsellers` `search` `product` `offer` `discussion` `movers-shakers` `new-releases` `rankings` | 浏览器 |
| **1688** | `search` `item` `assets` `download` `store` | 浏览器 |
| **gitee** | `trending` `search` `user` | 公开 / 浏览器 |
| **gemini** | `new` `ask` `image` `deep-research` `deep-research-result` | 浏览器 |
| **spotify** | `auth` `status` `play` `pause` `next` `prev` `volume` `search` `queue` `shuffle` `repeat` | OAuth API |
| **notebooklm** | `status` `list` `open` `current` `get` `history` `summary` `note-list` `notes-get` `source-list` `source-get` `source-fulltext` `source-guide` | 浏览器 |
@@ -206,7 +302,9 @@ npx skills add jackwener/opencli --skill opencli-oneshot # 快速命令参
| **douyin** | `videos` `publish` `drafts` `draft` `delete` `stats` `profile` `update` `hashtag` `location` `activities` `collections` | 浏览器 |
| **yuanbao** | `new` `ask` | 浏览器 |
79+ 适配器 — **[→ 查看完整命令列表](./docs/adapters/index.md)**
90+ 适配器 — **[→ 查看完整命令列表](./docs/adapters/index.md)**
`*` `opencli xiaoyuzhou podcast``podcast-episodes``episode``download``transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`
### 外部 CLI 枢纽
@@ -218,8 +316,8 @@ OpenCLI 也可以作为你现有命令行工具的统一入口,负责发现、
| **obsidian** | Obsidian 仓库管理 | `opencli obsidian search query="AI"` |
| **docker** | Docker 命令行工具 | `opencli docker ps` |
| **lark-cli** | 飞书 CLI — 消息、文档、日历、任务,200+ 命令 | `opencli lark-cli calendar +agenda` |
| **dingtalk** | 钉钉 CLI — 钉钉全套产品能力的跨平台命令行工具,支持人类和 AI Agent 使用 | `opencli dingtalk msg send --to user "hello"` |
| **wecom** | 企业微信 CLI — 企业微信开放平台命令行工具,支持人类和 AI Agent 使用 | `opencli wecom msg send --to user "hello"` |
| **dws** | 钉钉 CLI — 钉钉全套产品能力的跨平台命令行工具,支持人类和 AI Agent 使用 | `opencli dws msg send --to user "hello"` |
| **wecom-cli** | 企业微信 CLI — 企业微信开放平台命令行工具,支持人类和 AI Agent 使用 | `opencli wecom-cli msg send --to user "hello"` |
| **vercel** | Vercel — 部署项目、管理域名、环境变量、日志 | `opencli vercel deploy --prod` |
**零配置透传**:OpenCLI 会把你的输入原样转发给底层二进制,保留原生 stdout / stderr 行为。
@@ -241,7 +339,7 @@ opencli register mycli
| **Cursor** | 控制 Cursor IDE — Composer、对话、代码提取等 | [Doc](./docs/adapters/desktop/cursor.md) |
| **Codex** | 在后台(无头)驱动 OpenAI Codex CLI Agent | [Doc](./docs/adapters/desktop/codex.md) |
| **Antigravity** | 在终端直接控制 Antigravity Ultra | [Doc](./docs/adapters/desktop/antigravity.md) |
| **ChatGPT** | 自动化操作 ChatGPT macOS 桌面客户端 | [Doc](./docs/adapters/desktop/chatgpt.md) |
| **ChatGPT App** | 自动化操作 ChatGPT macOS 桌面客户端 | [Doc](./docs/adapters/desktop/chatgpt-app.md) |
| **ChatWise** | 多 LLM 客户端(GPT-4、Claude、Gemini | [Doc](./docs/adapters/desktop/chatwise.md) |
| **Notion** | 搜索、读取、写入 Notion 页面 | [Doc](./docs/adapters/desktop/notion.md) |
| **Discord** | Discord 桌面版 — 消息、频道、服务器 | [Doc](./docs/adapters/desktop/discord.md) |
@@ -260,6 +358,7 @@ OpenCLI 支持从各平台下载图片、视频和文章。
| **Twitter/X** | 图片、视频 | 从用户媒体页或单条推文下载 |
| **Pixiv** | 图片 | 下载原始画质插画,支持多页作品 |
| **1688** | 图片、视频 | 下载商品页中可见的商品素材 |
| **小宇宙** | 音频、转录 | 使用本地凭证下载单集音频和转录 JSON / 文本 |
| **知乎** | 文章(Markdown) | 导出文章,可选下载图片到本地 |
| **微信公众号** | 文章(Markdown | 导出微信公众号文章为 Markdown |
| **豆瓣** | 图片 | 下载电影条目的海报 / 剧照图片 |
@@ -279,7 +378,8 @@ brew install yt-dlp
```bash
# 下载小红书笔记中的图片/视频
opencli xiaohongshu download abc123 --output ./xhs
opencli xiaohongshu download "https://www.xiaohongshu.com/search_result/<id>?xsec_token=..." --output ./xhs
opencli xiaohongshu download "https://xhslink.com/..." --output ./xhs
# 下载B站视频(需要 yt-dlp
opencli bilibili download BV1xxx --output ./bilibili
@@ -297,6 +397,12 @@ opencli douban download 30382501 --output ./douban
# 下载 1688 商品页中的图片 / 视频素材
opencli 1688 download 841141931191 --output ./1688-downloads
# 下载小宇宙单集音频
opencli xiaoyuzhou download 69b3b675772ac2295bfc01d0 --output ./xiaoyuzhou
# 下载小宇宙单集转录
opencli xiaoyuzhou transcript 69dd0c98e2c8be31551f6a33 --output ./xiaoyuzhou-transcripts
# 导出知乎文章为 Markdown
opencli zhihu download "https://zhuanlan.zhihu.com/p/xxx" --output ./zhihu
@@ -307,6 +413,8 @@ opencli zhihu download "https://zhuanlan.zhihu.com/p/xxx" --download-images
opencli weixin download --url "https://mp.weixin.qq.com/s/xxx" --output ./weixin
```
`opencli xiaoyuzhou download``transcript` 需要本地小宇宙凭证:`~/.opencli/xiaoyuzhou.json`
## 输出格式
@@ -351,7 +459,7 @@ esac
## 插件
通过社区贡献的插件扩展 OpenCLI。插件使用与内置命令相同的 YAML/TS 格式,启动时自动发现。
通过社区贡献的插件扩展 OpenCLI。插件使用与内置命令相同的 JS 格式,启动时自动发现。
```bash
opencli plugin install github:user/opencli-plugin-my-tool # 安装
@@ -365,9 +473,10 @@ opencli plugin uninstall my-tool # 卸载
| 插件 | 类型 | 描述 |
|------|------|------|
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | YAML | GitHub Trending 仓库 |
| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | TS | 多平台热榜聚合 |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | YAML | 稀土掘金热门文章 |
| [opencli-plugin-github-trending](https://github.com/ByteYue/opencli-plugin-github-trending) | JS | GitHub Trending 仓库 |
| [opencli-plugin-hot-digest](https://github.com/ByteYue/opencli-plugin-hot-digest) | JS | 多平台热榜聚合 |
| [opencli-plugin-juejin](https://github.com/Astro-Han/opencli-plugin-juejin) | JS | 稀土掘金热门文章 |
| [opencli-plugin-vk](https://github.com/flobo3/opencli-plugin-vk) | JS | VK (VKontakte) 动态、信息流和搜索 |
详见 [插件指南](./docs/zh/guide/plugins.md) 了解如何创建自己的插件。
@@ -375,36 +484,26 @@ opencli plugin uninstall my-tool # 卸载
如果你是一个被要求查阅代码并编写新 `opencli` 适配器的 AI,请遵守以下工作流。
> **快速模式**:只想为某个页面快速生成一个命令?看 [opencli-oneshot skill](./skills/opencli-oneshot/SKILL.md) — 给一个 URL + 一句话描述,4 步搞定。
在动代码前,先读 [`opencli-adapter-author` skill](./skills/opencli-adapter-author/SKILL.md)。它把整个流程串起来:
> **完整模式**:在编写任何新代码前,先阅读 [opencli-explorer skill](./skills/opencli-explorer/SKILL.md)。它包含完整的适配器探索开发指南、API 探测流程、5级认证策略以及常见陷阱。
- 侦察站点,选定 patternSPA / SSR / JSONP / Token / Streaming
-`opencli browser network``eval`、interceptor 等找到目标 endpoint
- 定认证策略(`PUBLIC` / `COOKIE` / `HEADER` / `INTERCEPT`
- 字段解码、设计 columns、`opencli browser init` 生成骨架
- 交付前用 `opencli browser verify <site>/<name>` 验证
```bash
# 1. Deep Explore — 网络拦截 → 响应分析 → 能力推理 → 框架检测
opencli explore https://example.com --site mysite
# 2. Synthesize — 从探索成果物生成 evaluate-based YAML 适配器
opencli synthesize mysite
# 3. Generate — 一键完成:探索 → 合成 → 注册
opencli generate https://example.com --goal "hot"
# 4. Strategy Cascade — 自动降级探测:PUBLIC → COOKIE → HEADER
opencli cascade https://api.example.com/data
```
探索结果输出到 `.opencli/explore/<site>/`
在仓库外写的私有适配器放到 `~/.opencli/clis/<site>/<name>.js`;每个站点的 endpoint、字段映射、抓包样本会累积在 `~/.opencli/sites/<site>/`,下次写同站点的其他命令可以直接复用。
## 常见问题排查
- **"Extension not connected" 报错**
- 确保你当前的 Chrome 或 Chromium 已安装且**开启了** opencli Browser Bridge 扩展`chrome://extensions`检查)
- 确保你已从 [Chrome Web Store](https://chromewebstore.google.com/detail/opencli/ildkmabpimmkaediidaifkhjpohdnifk) 安装 OpenCLI 扩展,且`chrome://extensions`**已启用**
- **"attach failed: Cannot access a chrome-extension:// URL" 报错**
- 其他 Chrome/Chromium 扩展(如 youmind、New Tab Override 或 AI 助手类扩展)可能产生冲突。请尝试**暂时禁用其他扩展**后重试。
- **返回空数据,或者报错 "Unauthorized"**
- Chrome/Chromium 里的登录态可能已经过期。请打开当前页面,在新标签页重新手工登录或刷新该页面。
- **Node API 错误 (如 parseArgs, fs 等)**
- 确保 Node.js 版本 `>= 20`
- 确保 Node.js 版本 `>= 21``node:util``styleText` 需要 Node 21+
- **Daemon 问题**
- 检查 daemon 状态:`curl localhost:19825/status`
- 查看扩展日志:`curl localhost:19825/logs`
+20 -20
View File
@@ -30,12 +30,15 @@ tests/
├── smoke/
│ └── api-health.test.ts # 外部 API、adapter 定义、命令注册健康检查
src/
── **/*.test.ts # 单元测试(当前 32 个文件
── **/*.test.ts # 单元测试(unit project
clis/
└── **/*.test.{ts,js} # adapter 测试(adapter project
```
| 层 | 位置 | 当前文件数 | 运行方式 | 用途 |
|---|---|---:|---|---|
| 单元测试 | `src/**/*.test.ts` | 32 | `npx vitest run src/` | 内部模块、pipeline、adapter 工具函数 |
| 单元测试 | `src/**/*.test.ts` | 32 | `npm test` | 内部模块、pipeline、runtime |
| Adapter 测试 | `clis/**/*.test.{ts,js}` | - | `npm test` / `npm run test:adapter` | adapter 命令与数据归一化 |
| E2E 测试 | `tests/e2e/*.test.ts` | 5 | `npx vitest run tests/e2e/` | 真实 CLI 命令执行 |
| 烟雾测试 | `tests/smoke/*.test.ts` | 1 | `npx vitest run tests/smoke/` | 外部 API 与注册完整性 |
@@ -43,7 +46,7 @@ src/
## 当前覆盖范围
### 单元测试32 个文件)
### 单元测试与 Adapter 测试
| 领域 | 文件 |
|---|---|
@@ -100,8 +103,11 @@ npm run build # 编译(E2E / smoke 测试需要 dist/src/main.js
### 运行命令
```bash
# 全部单元测试
npx vitest run src/
# 默认本地测试口径(unit + extension + adapter
npm test
# 只跑 adapter project
npm run test:adapter
# 全部 E2E 测试(会真实调用外部 API / 浏览器)
npx vitest run tests/e2e/
@@ -110,7 +116,7 @@ npx vitest run tests/e2e/
npx vitest run tests/smoke/
# 单个测试文件
npx vitest run clis/apple-podcasts/commands.test.ts
npm test -- --run clis/apple-podcasts/commands.test.ts
npx vitest run tests/e2e/management.test.ts
# 全部测试
@@ -132,10 +138,9 @@ npx vitest src/
## 如何添加新测试
### 新增 YAML Adapter(如 `clis/producthunt/trending.yaml`
### 新增 Adapter(如 `clis/producthunt/trending.ts`
1. `opencli validate` 的 E2E / smoke 测试会覆盖 adapter 结构校验
2. 根据 adapter 类型,在对应测试文件补一个 `it()` block
1. 根据 adapter 类型,在对应测试文件补一个 `it()` block
```typescript
// 如果 browser: false(公开 API)→ tests/e2e/public-commands.test.ts
@@ -193,7 +198,8 @@ it('producthunt me fails gracefully without login', async () => {
| Job | 触发条件 | 内容 |
|---|---|---|
| `build` | push/PR 到 `main`,`dev` | `tsc --noEmit` + `npm run build` |
| `unit-test` | push/PR 到 `main`,`dev` | Node `20``22` 双版本运行 `src/` 单元测试,按 `2` shard 并行 |
| `unit-test` | push/PR 到 `main`,`dev` | Node `22` 运行 `unit + extension`,按 `2` shard 并行 |
| `adapter-test` | push/PR 到 `main`,`dev` | Node `22` 单独运行 `adapter` project |
| `smoke-test` | `schedule``workflow_dispatch` | 安装真实 Chrome`xvfb-run` 执行 `tests/smoke/` |
### `e2e-headed.yml`
@@ -202,19 +208,18 @@ it('producthunt me fails gracefully without login', async () => {
|---|---|---|
| `e2e-headed` | push/PR 到 `main`,`dev`,或手动触发 | 安装真实 Chrome,`xvfb-run` 执行 `tests/e2e/` |
E2E 与 smoke 都使用 `./.github/actions/setup-chrome` 准备真实 Chrome,并通过 `OPENCLI_BROWSER_EXECUTABLE_PATH` 注入浏览器路径
E2E 与 smoke 都使用 `./.github/actions/setup-chrome` 准备真实 Chrome。
### Sharding
单元测试使用 vitest 内置 shard并在 Node `20` / `22` 两个版本上运行
CI 里的 `unit-test` job 使用 vitest shard只切 `unit + extension`,避免和独立的 `adapter-test` job 重复
```yaml
strategy:
matrix:
node-version: ['20', '22']
shard: [1, 2]
steps:
- run: npx vitest run src/ --reporter=verbose --shard=${{ matrix.shard }}/2
- run: npx vitest run --project unit --project extension --reporter=verbose --shard=${{ matrix.shard }}/2
```
---
@@ -228,12 +233,7 @@ opencli 通过 Browser Bridge 扩展连接浏览器:
| 扩展已安装 / 已连接 | Extension 模式 | 本地用户,连接已登录的 Chrome |
| 无扩展 token | CLI 自行拉起浏览器 | CI、无登录态或纯自动化场景 |
CI 中使用 `OPENCLI_BROWSER_EXECUTABLE_PATH` 指定真实 Chrome 路径:
```yaml
env:
OPENCLI_BROWSER_EXECUTABLE_PATH: ${{ steps.setup-chrome.outputs.chrome-path }}
```
CI 通过 `./.github/actions/setup-chrome` 准备真实 Chrome,再直接执行测试。
---
+152 -152
View File
@@ -2,8 +2,8 @@
{
"name": "extract-title-example",
"steps": [
"opencli operate open https://example.com",
"opencli operate eval \"document.title\""
"opencli browser open https://example.com",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "contains",
@@ -13,8 +13,8 @@
{
"name": "extract-title-iana",
"steps": [
"opencli operate open https://www.iana.org",
"opencli operate eval \"document.querySelector('h1')?.textContent || document.title || document.querySelector('title')?.textContent\""
"opencli browser open https://www.iana.org",
"opencli browser eval \"document.querySelector('h1')?.textContent || document.title || document.querySelector('title')?.textContent\""
],
"judge": {
"type": "nonEmpty"
@@ -23,8 +23,8 @@
{
"name": "extract-paragraph-wiki-js",
"steps": [
"opencli operate open https://en.wikipedia.org/wiki/JavaScript",
"opencli operate eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50 && !t.startsWith('{')) return t.slice(0,300); } return ''; })()\""
"opencli browser open https://en.wikipedia.org/wiki/JavaScript",
"opencli browser eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50 && !t.startsWith('{')) return t.slice(0,300); } return ''; })()\""
],
"judge": {
"type": "contains",
@@ -34,8 +34,8 @@
{
"name": "extract-paragraph-wiki-python",
"steps": [
"opencli operate open \"https://en.wikipedia.org/wiki/Python_(programming_language)\"",
"opencli operate eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50 && !t.startsWith('{')) return t.slice(0,300); } return ''; })()\""
"opencli browser open \"https://en.wikipedia.org/wiki/Python_(programming_language)\"",
"opencli browser eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50 && !t.startsWith('{')) return t.slice(0,300); } return ''; })()\""
],
"judge": {
"type": "contains",
@@ -45,8 +45,8 @@
{
"name": "extract-github-stars",
"steps": [
"opencli operate open https://github.com/browser-use/browser-use",
"opencli operate eval \"document.querySelector('#repo-stars-counter-star')?.textContent?.trim()\""
"opencli browser open https://github.com/browser-use/browser-use",
"opencli browser eval \"document.querySelector('#repo-stars-counter-star')?.textContent?.trim()\""
],
"judge": {
"type": "matchesPattern",
@@ -56,8 +56,8 @@
{
"name": "extract-github-description",
"steps": [
"opencli operate open https://github.com/anthropics/claude-code",
"opencli operate eval \"document.querySelector('p.f4, [data-testid=about-description], .f4.my-3, .BorderGrid-cell p')?.textContent?.trim()\""
"opencli browser open https://github.com/anthropics/claude-code",
"opencli browser eval \"document.querySelector('p.f4, [data-testid=about-description], .f4.my-3, .BorderGrid-cell p')?.textContent?.trim()\""
],
"judge": {
"type": "nonEmpty"
@@ -66,8 +66,8 @@
{
"name": "extract-github-readme-heading",
"steps": [
"opencli operate open https://github.com/vercel/next.js",
"opencli operate eval \"document.querySelector('[data-testid=readme] h1, [data-testid=readme] h2, #readme h1, #readme h2, article h1, article h2, .markdown-body h1, .markdown-body h2')?.textContent?.trim()\""
"opencli browser open https://github.com/vercel/next.js",
"opencli browser eval \"document.querySelector('[data-testid=readme] h1, [data-testid=readme] h2, #readme h1, #readme h2, article h1, article h2, .markdown-body h1, .markdown-body h2')?.textContent?.trim()\""
],
"judge": {
"type": "nonEmpty"
@@ -76,8 +76,8 @@
{
"name": "extract-npm-downloads",
"steps": [
"opencli operate open https://www.npmjs.com/package/zod",
"opencli operate eval \"document.querySelector('[data-nosnippet]')?.textContent?.trim() || document.querySelector('p.f2874b88')?.textContent?.trim()\""
"opencli browser open https://www.npmjs.com/package/zod",
"opencli browser eval \"document.querySelector('[data-nosnippet]')?.textContent?.trim() || document.querySelector('p.f2874b88')?.textContent?.trim()\""
],
"judge": {
"type": "matchesPattern",
@@ -87,9 +87,9 @@
{
"name": "extract-npm-description",
"steps": [
"opencli operate open https://www.npmjs.com/package/express",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var ps=document.querySelectorAll('p');for(var i=0;i<ps.length;i++){var t=ps[i].textContent.trim();if(t.length>10&&t.length<200)return t;}return '';})()\""
"opencli browser open https://www.npmjs.com/package/express",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var ps=document.querySelectorAll('p');for(var i=0;i<ps.length;i++){var t=ps[i].textContent.trim();if(t.length>10&&t.length<200)return t;}return '';})()\""
],
"judge": {
"type": "nonEmpty"
@@ -98,8 +98,8 @@
{
"name": "list-hn-top5",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.titleline > a')].slice(0,5).map(a=>({title:a.textContent,url:a.href})))\""
"opencli browser open https://news.ycombinator.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.titleline > a')].slice(0,5).map(a=>({title:a.textContent,url:a.href})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -109,8 +109,8 @@
{
"name": "list-hn-top10",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.athing')].slice(0,10).map(tr=>{const a=tr.querySelector('.titleline>a');const s=tr.nextElementSibling?.querySelector('.score');return{title:a?.textContent,score:parseInt(s?.textContent)||0}}))\""
"opencli browser open https://news.ycombinator.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.athing')].slice(0,10).map(tr=>{const a=tr.querySelector('.titleline>a');const s=tr.nextElementSibling?.querySelector('.score');return{title:a?.textContent,score:parseInt(s?.textContent)||0}}))\""
],
"judge": {
"type": "arrayMinLength",
@@ -120,8 +120,8 @@
{
"name": "list-books-5",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('article.product_pod')].slice(0,5).map(el=>({title:el.querySelector('h3 a')?.getAttribute('title'),price:el.querySelector('.price_color')?.textContent})))\""
"opencli browser open https://books.toscrape.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('article.product_pod')].slice(0,5).map(el=>({title:el.querySelector('h3 a')?.getAttribute('title'),price:el.querySelector('.price_color')?.textContent})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -131,8 +131,8 @@
{
"name": "list-books-10",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('article.product_pod')].slice(0,10).map(el=>({title:el.querySelector('h3 a')?.getAttribute('title'),price:el.querySelector('.price_color')?.textContent})))\""
"opencli browser open https://books.toscrape.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('article.product_pod')].slice(0,10).map(el=>({title:el.querySelector('h3 a')?.getAttribute('title'),price:el.querySelector('.price_color')?.textContent})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -142,8 +142,8 @@
{
"name": "list-quotes-3",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.quote, [class*=quote]')].slice(0,3).map(el=>({text:(el.querySelector('.text, [class*=text]')?.textContent)||(el.querySelector('span')?.textContent),author:(el.querySelector('.author, [class*=author]')?.textContent)||(el.querySelector('small')?.textContent)})))\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.quote, [class*=quote]')].slice(0,3).map(el=>({text:(el.querySelector('.text, [class*=text]')?.textContent)||(el.querySelector('span')?.textContent),author:(el.querySelector('.author, [class*=author]')?.textContent)||(el.querySelector('small')?.textContent)})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -153,8 +153,8 @@
{
"name": "list-quotes-tags",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.quote')].slice(0,5).map(el=>({text:el.querySelector('.text')?.textContent,author:el.querySelector('.author')?.textContent,tags:[...el.querySelectorAll('.tag')].map(t=>t.textContent)})))\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.quote')].slice(0,5).map(el=>({text:el.querySelector('.text')?.textContent,author:el.querySelector('.author')?.textContent,tags:[...el.querySelectorAll('.tag')].map(t=>t.textContent)})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -164,8 +164,8 @@
{
"name": "list-github-trending",
"steps": [
"opencli operate open https://github.com/trending",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('article.Box-row, article[class*=Box-row], [data-hpc] article, .Box article')].slice(0,3).map(el=>({name:(el.querySelector('h2 a, h1 a')?.textContent?.trim().replace(/\\\\s+/g,' '))||(el.querySelector('a[href^=\\\"/\\\"]')?.textContent?.trim()),desc:el.querySelector('p')?.textContent?.trim()})))\""
"opencli browser open https://github.com/trending",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('article.Box-row, article[class*=Box-row], [data-hpc] article, .Box article')].slice(0,3).map(el=>({name:(el.querySelector('h2 a, h1 a')?.textContent?.trim().replace(/\\\\s+/g,' '))||(el.querySelector('a[href^=\\\"/\\\"]')?.textContent?.trim()),desc:el.querySelector('p')?.textContent?.trim()})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -175,8 +175,8 @@
{
"name": "list-github-trending-lang",
"steps": [
"opencli operate open https://github.com/trending/python",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('article.Box-row, article[class*=Box-row], [data-hpc] article, .Box article')].slice(0,5).map(el=>({name:(el.querySelector('h2 a, h1 a')?.textContent?.trim().replace(/\\\\s+/g,' '))||(el.querySelector('a[href^=\\\"/\\\"]')?.textContent?.trim())})))\""
"opencli browser open https://github.com/trending/python",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('article.Box-row, article[class*=Box-row], [data-hpc] article, .Box article')].slice(0,5).map(el=>({name:(el.querySelector('h2 a, h1 a')?.textContent?.trim().replace(/\\\\s+/g,' '))||(el.querySelector('a[href^=\\\"/\\\"]')?.textContent?.trim())})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -186,8 +186,8 @@
{
"name": "list-jsonplaceholder-posts",
"steps": [
"opencli operate open https://jsonplaceholder.typicode.com/posts",
"opencli operate eval \"JSON.stringify(JSON.parse(document.body.innerText).slice(0,5).map(p=>({id:p.id,title:p.title})))\""
"opencli browser open https://jsonplaceholder.typicode.com/posts",
"opencli browser eval \"JSON.stringify(JSON.parse(document.body.innerText).slice(0,5).map(p=>({id:p.id,title:p.title})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -197,8 +197,8 @@
{
"name": "list-jsonplaceholder-users",
"steps": [
"opencli operate open https://jsonplaceholder.typicode.com/users",
"opencli operate eval \"JSON.stringify(JSON.parse(document.body.innerText).map(u=>({name:u.name,email:u.email})))\""
"opencli browser open https://jsonplaceholder.typicode.com/users",
"opencli browser eval \"JSON.stringify(JSON.parse(document.body.innerText).map(u=>({name:u.name,email:u.email})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -208,9 +208,9 @@
{
"name": "search-google",
"steps": [
"opencli operate open https://www.google.com/search?q=opencli+github",
"opencli operate wait time 3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('h3')].slice(0,3).map(h=>h.textContent))\""
"opencli browser open https://www.google.com/search?q=opencli+github",
"opencli browser wait time 3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('h3')].slice(0,3).map(h=>h.textContent))\""
],
"judge": {
"type": "arrayMinLength",
@@ -221,11 +221,11 @@
{
"name": "search-ddg",
"steps": [
"opencli operate open https://duckduckgo.com",
"opencli operate state",
"opencli operate type 1 \"weather beijing\"",
"opencli operate keys Enter",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a]')].slice(0,3).map(a=>a.textContent))\""
"opencli browser open https://duckduckgo.com",
"opencli browser state",
"opencli browser type 1 \"weather beijing\"",
"opencli browser keys Enter",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a]')].slice(0,3).map(a=>a.textContent))\""
],
"judge": {
"type": "nonEmpty"
@@ -235,10 +235,10 @@
{
"name": "search-ddg-tech",
"steps": [
"opencli operate open https://duckduckgo.com",
"opencli operate eval \"document.querySelector('input[name=q]').value='TypeScript tutorial';document.querySelector('form').submit();'submitted'\"",
"opencli operate wait time 3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a], .result__a')].slice(0,3).map(a=>({title:a.textContent,url:a.href})))\""
"opencli browser open https://duckduckgo.com",
"opencli browser eval \"document.querySelector('input[name=q]').value='TypeScript tutorial';document.querySelector('form').submit();'submitted'\"",
"opencli browser wait time 3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a], .result__a')].slice(0,3).map(a=>({title:a.textContent,url:a.href})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -249,9 +249,9 @@
{
"name": "search-wiki",
"steps": [
"opencli operate open \"https://en.wikipedia.org/w/index.php?search=Rust+programming+language&title=Special:Search&go=Go\"",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50) return t.slice(0,300); } return ''; })()\""
"opencli browser open \"https://en.wikipedia.org/w/index.php?search=Rust+programming+language&title=Special:Search&go=Go\"",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50) return t.slice(0,300); } return ''; })()\""
],
"judge": {
"type": "contains",
@@ -262,9 +262,9 @@
{
"name": "search-npm",
"steps": [
"opencli operate open https://www.npmjs.com/search?q=react",
"opencli operate wait time 3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('[data-testid=pkg-list-item] h3, section h3, .package-list-item h3, a[class*=package] h3')].slice(0,3).map(h=>h.textContent?.trim()))\""
"opencli browser open https://www.npmjs.com/search?q=react",
"opencli browser wait time 3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('[data-testid=pkg-list-item] h3, section h3, .package-list-item h3, a[class*=package] h3')].slice(0,3).map(h=>h.textContent?.trim()))\""
],
"judge": {
"type": "arrayMinLength",
@@ -275,9 +275,9 @@
{
"name": "search-github",
"steps": [
"opencli operate open https://github.com/search?q=browser+automation&type=repositories",
"opencli operate wait time 3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.search-title a, [data-testid=results-list] a.Link--primary')].slice(0,3).map(a=>a.textContent?.trim().replace(/\\\\s+/g,' ')))\""
"opencli browser open https://github.com/search?q=browser+automation&type=repositories",
"opencli browser wait time 3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.search-title a, [data-testid=results-list] a.Link--primary')].slice(0,3).map(a=>a.textContent?.trim().replace(/\\\\s+/g,' ')))\""
],
"judge": {
"type": "arrayMinLength",
@@ -288,10 +288,10 @@
{
"name": "nav-click-link-example",
"steps": [
"opencli operate open https://example.com",
"opencli operate eval \"document.querySelector('a')?.click();'clicked'\"",
"opencli operate wait time 2",
"opencli operate eval \"document.title + ' ' + location.href\""
"opencli browser open https://example.com",
"opencli browser eval \"document.querySelector('a')?.click();'clicked'\"",
"opencli browser wait time 2",
"opencli browser eval \"document.title + ' ' + location.href\""
],
"judge": {
"type": "contains",
@@ -301,10 +301,10 @@
{
"name": "nav-click-hn-first",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate eval \"document.querySelector('.titleline a')?.click();'clicked'\"",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://news.ycombinator.com",
"opencli browser eval \"document.querySelector('.titleline a')?.click();'clicked'\"",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -313,9 +313,9 @@
{
"name": "nav-click-hn-comments",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate eval \"document.querySelector('.subtext a:last-child')?.click(); 'clicked'\"",
"opencli operate eval \"document.title\""
"opencli browser open https://news.ycombinator.com",
"opencli browser eval \"document.querySelector('.subtext a:last-child')?.click(); 'clicked'\"",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -324,9 +324,9 @@
{
"name": "nav-click-wiki-link",
"steps": [
"opencli operate open https://en.wikipedia.org/wiki/JavaScript",
"opencli operate eval \"document.querySelector('.vector-toc-contents a[href*=History], #toc a[href*=History], .toc a[href*=History], [href=\\\"#History\\\"]')?.click(); 'clicked'\"",
"opencli operate eval \"document.querySelector('#History')?.textContent?.slice(0,100) || document.querySelector('[id*=History]')?.textContent?.slice(0,100)\""
"opencli browser open https://en.wikipedia.org/wiki/JavaScript",
"opencli browser eval \"document.querySelector('.vector-toc-contents a[href*=History], #toc a[href*=History], .toc a[href*=History], [href=\\\"#History\\\"]')?.click(); 'clicked'\"",
"opencli browser eval \"document.querySelector('#History')?.textContent?.slice(0,100) || document.querySelector('[id*=History]')?.textContent?.slice(0,100)\""
],
"judge": {
"type": "nonEmpty"
@@ -335,9 +335,9 @@
{
"name": "nav-click-github-tab",
"steps": [
"opencli operate open https://github.com/vercel/next.js",
"opencli operate eval \"document.querySelector('[data-tab-item=i1issues-tab] a, #issues-tab')?.click(); 'clicked'\"",
"opencli operate eval \"document.title\""
"opencli browser open https://github.com/vercel/next.js",
"opencli browser eval \"document.querySelector('[data-tab-item=i1issues-tab] a, #issues-tab')?.click(); 'clicked'\"",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -346,12 +346,12 @@
{
"name": "nav-go-back",
"steps": [
"opencli operate open https://example.com",
"opencli operate eval \"document.querySelector('a')?.click();'clicked'\"",
"opencli operate wait time 2",
"opencli operate back",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://example.com",
"opencli browser eval \"document.querySelector('a')?.click();'clicked'\"",
"opencli browser wait time 2",
"opencli browser back",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "contains",
@@ -361,9 +361,9 @@
{
"name": "nav-multi-step",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"document.querySelector('.next a')?.click(); 'clicked'\"",
"opencli operate eval \"document.querySelector('.quote .text')?.textContent\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"document.querySelector('.next a')?.click(); 'clicked'\"",
"opencli browser eval \"document.querySelector('.quote .text')?.textContent\""
],
"judge": {
"type": "nonEmpty"
@@ -372,10 +372,10 @@
{
"name": "scroll-footer-quotes",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate scroll down",
"opencli operate scroll down",
"opencli operate eval \"document.querySelector('footer, .footer, .tags-box')?.textContent?.trim().slice(0,100)\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser scroll down",
"opencli browser scroll down",
"opencli browser eval \"document.querySelector('footer, .footer, .tags-box')?.textContent?.trim().slice(0,100)\""
],
"judge": {
"type": "nonEmpty"
@@ -384,10 +384,10 @@
{
"name": "scroll-footer-books",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate scroll down",
"opencli operate scroll down",
"opencli operate eval \"document.querySelector('.pager .current')?.textContent?.trim()\""
"opencli browser open https://books.toscrape.com",
"opencli browser scroll down",
"opencli browser scroll down",
"opencli browser eval \"document.querySelector('.pager .current')?.textContent?.trim()\""
],
"judge": {
"type": "matchesPattern",
@@ -397,8 +397,8 @@
{
"name": "scroll-long-page",
"steps": [
"opencli operate open https://jsonplaceholder.typicode.com/posts",
"opencli operate eval \"JSON.parse(document.body.innerText).length\""
"opencli browser open https://jsonplaceholder.typicode.com/posts",
"opencli browser eval \"JSON.parse(document.body.innerText).length\""
],
"judge": {
"type": "matchesPattern",
@@ -408,8 +408,8 @@
{
"name": "scroll-find-element",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"document.querySelector('.next a')?.href\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"document.querySelector('.next a')?.href\""
],
"judge": {
"type": "nonEmpty"
@@ -418,8 +418,8 @@
{
"name": "scroll-lazy-load",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate eval \"document.querySelectorAll('article.product_pod').length\""
"opencli browser open https://books.toscrape.com",
"opencli browser eval \"document.querySelectorAll('article.product_pod').length\""
],
"judge": {
"type": "matchesPattern",
@@ -429,8 +429,8 @@
{
"name": "form-simple-name",
"steps": [
"opencli operate open https://httpbin.org/forms/post",
"opencli operate eval \"var el=document.querySelector('[name=custname]');el.value='OpenCLI Test';el.dispatchEvent(new Event('input',{bubbles:true}));el.value\""
"opencli browser open https://httpbin.org/forms/post",
"opencli browser eval \"var el=document.querySelector('[name=custname]');el.value='OpenCLI Test';el.dispatchEvent(new Event('input',{bubbles:true}));el.value\""
],
"judge": {
"type": "contains",
@@ -441,8 +441,8 @@
{
"name": "form-text-inputs",
"steps": [
"opencli operate open https://httpbin.org/forms/post",
"opencli operate eval \"var n=document.querySelector('[name=custname]');n.value='Alice';n.dispatchEvent(new Event('input',{bubbles:true}));var t=document.querySelector('[name=custtel]');t.value='555-1234';t.dispatchEvent(new Event('input',{bubbles:true}));n.value+'|'+t.value\""
"opencli browser open https://httpbin.org/forms/post",
"opencli browser eval \"var n=document.querySelector('[name=custname]');n.value='Alice';n.dispatchEvent(new Event('input',{bubbles:true}));var t=document.querySelector('[name=custtel]');t.value='555-1234';t.dispatchEvent(new Event('input',{bubbles:true}));n.value+'|'+t.value\""
],
"judge": {
"type": "contains",
@@ -453,8 +453,8 @@
{
"name": "form-radio-select",
"steps": [
"opencli operate open https://httpbin.org/forms/post",
"opencli operate eval \"document.querySelector('[value=medium]').checked=true;document.querySelector('[value=medium]').dispatchEvent(new Event('change',{bubbles:true}));document.querySelector('[value=medium]').checked\""
"opencli browser open https://httpbin.org/forms/post",
"opencli browser eval \"document.querySelector('[value=medium]').checked=true;document.querySelector('[value=medium]').dispatchEvent(new Event('change',{bubbles:true}));document.querySelector('[value=medium]').checked\""
],
"judge": {
"type": "contains",
@@ -464,8 +464,8 @@
{
"name": "form-checkbox",
"steps": [
"opencli operate open https://httpbin.org/forms/post",
"opencli operate eval \"var cb=document.querySelector('[value=cheese]');cb.checked=true;cb.dispatchEvent(new Event('change',{bubbles:true}));cb.checked\""
"opencli browser open https://httpbin.org/forms/post",
"opencli browser eval \"var cb=document.querySelector('[value=cheese]');cb.checked=true;cb.dispatchEvent(new Event('change',{bubbles:true}));cb.checked\""
],
"judge": {
"type": "contains",
@@ -475,8 +475,8 @@
{
"name": "form-textarea",
"steps": [
"opencli operate open https://httpbin.org/forms/post",
"opencli operate eval \"var ta=document.querySelector('textarea[name=comments], textarea[name=delivery], textarea');ta.value='AutoResearch test';ta.dispatchEvent(new Event('input',{bubbles:true}));ta.value\""
"opencli browser open https://httpbin.org/forms/post",
"opencli browser eval \"var ta=document.querySelector('textarea[name=comments], textarea[name=delivery], textarea');ta.value='AutoResearch test';ta.dispatchEvent(new Event('input',{bubbles:true}));ta.value\""
],
"judge": {
"type": "contains",
@@ -486,8 +486,8 @@
{
"name": "form-login-fake",
"steps": [
"opencli operate open https://the-internet.herokuapp.com/login",
"opencli operate eval \"var u=document.querySelector('#username');u.value='testuser';u.dispatchEvent(new Event('input',{bubbles:true}));var p=document.querySelector('#password');p.value='testpass';p.dispatchEvent(new Event('input',{bubbles:true}));u.value+'|'+p.value\""
"opencli browser open https://the-internet.herokuapp.com/login",
"opencli browser eval \"var u=document.querySelector('#username');u.value='testuser';u.dispatchEvent(new Event('input',{bubbles:true}));var p=document.querySelector('#password');p.value='testpass';p.dispatchEvent(new Event('input',{bubbles:true}));u.value+'|'+p.value\""
],
"judge": {
"type": "contains",
@@ -498,8 +498,8 @@
{
"name": "complex-wiki-toc",
"steps": [
"opencli operate open https://en.wikipedia.org/wiki/JavaScript",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.toc li a, #toc li a, .vector-toc-contents a')].slice(0,8).map(a=>a.textContent?.trim()))\""
"opencli browser open https://en.wikipedia.org/wiki/JavaScript",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.toc li a, #toc li a, .vector-toc-contents a')].slice(0,8).map(a=>a.textContent?.trim()))\""
],
"judge": {
"type": "arrayMinLength",
@@ -509,9 +509,9 @@
{
"name": "complex-books-detail",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate eval \"document.querySelector('article.product_pod h3 a')?.click();'clicked'\"",
"opencli operate eval \"JSON.stringify({title:document.querySelector('h1')?.textContent,price:document.querySelector('.price_color')?.textContent})\""
"opencli browser open https://books.toscrape.com",
"opencli browser eval \"document.querySelector('article.product_pod h3 a')?.click();'clicked'\"",
"opencli browser eval \"JSON.stringify({title:document.querySelector('h1')?.textContent,price:document.querySelector('.price_color')?.textContent})\""
],
"judge": {
"type": "nonEmpty"
@@ -520,9 +520,9 @@
{
"name": "complex-quotes-page2",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"document.querySelector('.next a')?.click();'clicked'\"",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.quote')].slice(0,3).map(el=>({text:el.querySelector('.text')?.textContent,author:el.querySelector('.author')?.textContent})))\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"document.querySelector('.next a')?.click();'clicked'\"",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.quote')].slice(0,3).map(el=>({text:el.querySelector('.text')?.textContent,author:el.querySelector('.author')?.textContent})))\""
],
"judge": {
"type": "arrayMinLength",
@@ -532,8 +532,8 @@
{
"name": "complex-github-repo-info",
"steps": [
"opencli operate open https://github.com/expressjs/express",
"opencli operate eval \"JSON.stringify({lang:document.querySelector('[itemprop=programmingLanguage]')?.textContent?.trim(),license:document.querySelector('[data-analytics-event*=license] span, .Layout-sidebar [href*=LICENSE]')?.textContent?.trim()})\""
"opencli browser open https://github.com/expressjs/express",
"opencli browser eval \"JSON.stringify({lang:document.querySelector('[itemprop=programmingLanguage]')?.textContent?.trim(),license:document.querySelector('[data-analytics-event*=license] span, .Layout-sidebar [href*=LICENSE]')?.textContent?.trim()})\""
],
"judge": {
"type": "nonEmpty"
@@ -542,9 +542,9 @@
{
"name": "complex-hn-story-comments",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate eval \"document.querySelector('.subtext a:last-child')?.click();'clicked'\"",
"opencli operate eval \"document.querySelector('.fatitem .titleline a')?.textContent\""
"opencli browser open https://news.ycombinator.com",
"opencli browser eval \"document.querySelector('.subtext a:last-child')?.click();'clicked'\"",
"opencli browser eval \"document.querySelector('.fatitem .titleline a')?.textContent\""
],
"judge": {
"type": "nonEmpty"
@@ -553,8 +553,8 @@
{
"name": "complex-multi-extract",
"steps": [
"opencli operate open https://en.wikipedia.org/wiki/TypeScript",
"opencli operate eval \"JSON.stringify({title:document.title,firstParagraph:document.querySelector('#mw-content-text p')?.textContent?.slice(0,150)})\""
"opencli browser open https://en.wikipedia.org/wiki/TypeScript",
"opencli browser eval \"JSON.stringify({title:document.title,firstParagraph:document.querySelector('#mw-content-text p')?.textContent?.slice(0,150)})\""
],
"judge": {
"type": "contains",
@@ -564,8 +564,8 @@
{
"name": "bench-reddit-top5",
"steps": [
"opencli operate open https://old.reddit.com",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('#siteTable .thing .title a.title')].slice(0,5).map(a=>a.textContent))\""
"opencli browser open https://old.reddit.com",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('#siteTable .thing .title a.title')].slice(0,5).map(a=>a.textContent))\""
],
"judge": {
"type": "arrayMinLength",
@@ -576,9 +576,9 @@
{
"name": "bench-imdb-matrix",
"steps": [
"opencli operate open https://www.imdb.com/title/tt0133093/",
"opencli operate wait time 3",
"opencli operate eval \"(function(){var title=document.querySelector('h1')?.textContent?.trim()||'';var year='';var links=document.querySelectorAll('a');for(var i=0;i<links.length;i++){if(links[i].textContent.trim()==='1999'){year='1999';break;}}var rating=document.querySelector('[data-testid=hero-rating-bar__aggregate-rating__score] span, .sc-bde20123-1')?.textContent?.trim()||'';return JSON.stringify({title:title,year:year,rating:rating});})()\""
"opencli browser open https://www.imdb.com/title/tt0133093/",
"opencli browser wait time 3",
"opencli browser eval \"(function(){var title=document.querySelector('h1')?.textContent?.trim()||'';var year='';var links=document.querySelectorAll('a');for(var i=0;i<links.length;i++){if(links[i].textContent.trim()==='1999'){year='1999';break;}}var rating=document.querySelector('[data-testid=hero-rating-bar__aggregate-rating__score] span, .sc-bde20123-1')?.textContent?.trim()||'';return JSON.stringify({title:title,year:year,rating:rating});})()\""
],
"judge": {
"type": "contains",
@@ -589,8 +589,8 @@
{
"name": "bench-npm-zod",
"steps": [
"opencli operate open https://www.npmjs.com/package/zod",
"opencli operate eval \"JSON.stringify({name:document.querySelector('h1 span, #top h2')?.textContent?.trim(),description:document.querySelector('[data-testid=package-description], p.package-description-redundant')?.textContent?.trim()})\""
"opencli browser open https://www.npmjs.com/package/zod",
"opencli browser eval \"JSON.stringify({name:document.querySelector('h1 span, #top h2')?.textContent?.trim(),description:document.querySelector('[data-testid=package-description], p.package-description-redundant')?.textContent?.trim()})\""
],
"judge": {
"type": "nonEmpty"
@@ -600,8 +600,8 @@
{
"name": "bench-wiki-search",
"steps": [
"opencli operate open https://en.wikipedia.org/wiki/Machine_learning",
"opencli operate eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50) return t.slice(0,300); } return ''; })()\""
"opencli browser open https://en.wikipedia.org/wiki/Machine_learning",
"opencli browser eval \"(() => { const ps = document.querySelectorAll('#mw-content-text .mw-parser-output > p'); for (const p of ps) { const t = p.textContent?.trim(); if (t && t.length > 50) return t.slice(0,300); } return ''; })()\""
],
"judge": {
"type": "contains",
@@ -612,8 +612,8 @@
{
"name": "bench-github-profile",
"steps": [
"opencli operate open https://github.com/torvalds",
"opencli operate eval \"JSON.stringify({name:document.querySelector('[itemprop=name]')?.textContent?.trim(),bio:document.querySelector('[data-bio-text]')?.textContent?.trim()||document.querySelector('.p-note')?.textContent?.trim()})\""
"opencli browser open https://github.com/torvalds",
"opencli browser eval \"JSON.stringify({name:document.querySelector('[itemprop=name]')?.textContent?.trim(),bio:document.querySelector('[data-bio-text]')?.textContent?.trim()||document.querySelector('.p-note')?.textContent?.trim()})\""
],
"judge": {
"type": "nonEmpty"
@@ -623,9 +623,9 @@
{
"name": "bench-books-category",
"steps": [
"opencli operate open https://books.toscrape.com",
"opencli operate eval \"document.querySelector('a[href*=science]')?.click();'clicked'\"",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('article.product_pod h3 a')].slice(0,3).map(a=>a.getAttribute('title')))\""
"opencli browser open https://books.toscrape.com",
"opencli browser eval \"document.querySelector('a[href*=science]')?.click();'clicked'\"",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('article.product_pod h3 a')].slice(0,3).map(a=>a.getAttribute('title')))\""
],
"judge": {
"type": "arrayMinLength",
@@ -636,9 +636,9 @@
{
"name": "bench-quotes-author",
"steps": [
"opencli operate open https://quotes.toscrape.com",
"opencli operate eval \"document.querySelector('.author + a, a[href*=author]')?.click();'clicked'\"",
"opencli operate eval \"document.querySelector('.author-description, .author-details p')?.textContent?.slice(0,100)\""
"opencli browser open https://quotes.toscrape.com",
"opencli browser eval \"document.querySelector('.author + a, a[href*=author]')?.click();'clicked'\"",
"opencli browser eval \"document.querySelector('.author-description, .author-details p')?.textContent?.slice(0,100)\""
],
"judge": {
"type": "nonEmpty"
@@ -648,10 +648,10 @@
{
"name": "bench-ddg-images",
"steps": [
"opencli operate open https://duckduckgo.com",
"opencli operate eval \"document.querySelector('input[name=q]').value='sunset';document.querySelector('form').submit();'submitted'\"",
"opencli operate wait time 3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a], .result__a')].slice(0,3).map(a=>a.textContent))\""
"opencli browser open https://duckduckgo.com",
"opencli browser eval \"document.querySelector('input[name=q]').value='sunset';document.querySelector('form').submit();'submitted'\"",
"opencli browser wait time 3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('[data-testid=result-title-a], .result__a')].slice(0,3).map(a=>a.textContent))\""
],
"judge": {
"type": "arrayMinLength",
@@ -663,8 +663,8 @@
{
"name": "bench-httpbin-headers",
"steps": [
"opencli operate open https://httpbin.org/headers",
"opencli operate eval \"JSON.parse(document.body.innerText).headers['User-Agent']\""
"opencli browser open https://httpbin.org/headers",
"opencli browser eval \"JSON.parse(document.body.innerText).headers['User-Agent']\""
],
"judge": {
"type": "nonEmpty"
@@ -674,8 +674,8 @@
{
"name": "bench-jsonapi-todo",
"steps": [
"opencli operate open https://jsonplaceholder.typicode.com/todos",
"opencli operate eval \"JSON.stringify(JSON.parse(document.body.innerText).slice(0,5).map(t=>({id:t.id,title:t.title,completed:t.completed})))\""
"opencli browser open https://jsonplaceholder.typicode.com/todos",
"opencli browser eval \"JSON.stringify(JSON.parse(document.body.innerText).slice(0,5).map(t=>({id:t.id,title:t.title,completed:t.completed})))\""
],
"judge": {
"type": "arrayMinLength",
+5 -5
View File
@@ -58,7 +58,7 @@ async function main() {
const tasks: BrowseTask[] = JSON.parse(readFileSync(TASKS_FILE, 'utf-8'));
// Show only failing tasks
for (const task of tasks) {
try { exec('opencli operate close'); } catch {}
try { exec('opencli browser close'); } catch {}
let lastOutput = '';
for (const step of task.steps) lastOutput = exec(step);
const passed = lastOutput.trim().length > 0; // simplified check
@@ -83,7 +83,7 @@ async function main() {
// Phase 1: Gather — run the task and capture output
console.log('Phase 1: Gathering symptoms...');
try { exec('opencli operate close'); } catch {}
try { exec('opencli browser close'); } catch {}
let lastOutput = '';
for (let i = 0; i < task.steps.length; i++) {
@@ -120,7 +120,7 @@ ${lastOutput.slice(0, 500)}
4. If CONFIRMED: describe the root cause and suggest a fix
5. Output format: one line "HYPOTHESIS: ...", one line "RESULT: CONFIRMED|DISPROVEN|INCONCLUSIVE — ..."
Do NOT fix the code — just diagnose. Use opencli operate commands to investigate.`;
Do NOT fix the code — just diagnose. Use opencli browser commands to investigate.`;
try {
const result = execSync(
@@ -152,11 +152,11 @@ Do NOT fix the code — just diagnose. Use opencli operate commands to investiga
}
// Re-run task for fresh output
try { exec('opencli operate close'); } catch {}
try { exec('opencli browser close'); } catch {}
for (const step of task.steps) lastOutput = exec(step);
}
try { exec('opencli operate close'); } catch {}
try { exec('opencli browser close'); } catch {}
console.log(`\nDebug log saved to: ${DEBUG_LOG}\n`);
}
+2 -2
View File
@@ -3,8 +3,8 @@
* /autoresearch — Main autonomous iteration loop.
*
* Usage:
* npx tsx autoresearch/commands/run.ts --preset operate-reliability
* npx tsx autoresearch/commands/run.ts --preset operate-reliability --iterations 5
* npx tsx autoresearch/commands/run.ts --preset browser-reliability
* npx tsx autoresearch/commands/run.ts --preset browser-reliability --iterations 5
* npx tsx autoresearch/commands/run.ts --goal "..." --scope "src/*.ts" --verify "..." --iterations 10
*
* The modify callback spawns Claude Code to make ONE atomic change per iteration.
+1 -1
View File
@@ -5,7 +5,7 @@
*/
export interface AutoResearchConfig {
/** Plain-language goal, e.g. "Increase operate pass rate to 59/59" */
/** Plain-language goal, e.g. "Increase browser pass rate to 59/59" */
goal: string;
/** Glob patterns for files the agent can modify */
scope: string[];
+1 -1
View File
@@ -134,7 +134,7 @@ export class Engine {
if (diff === '0') return null; // no changes
try {
execStrict(`git commit -m "experiment(operate): ${description.replace(/"/g, '\\"')}"`);
execStrict(`git commit -m "experiment(browser): ${description.replace(/"/g, '\\"')}"`);
return exec('git rev-parse --short HEAD');
} catch {
// Hook failure
+3 -3
View File
@@ -2,7 +2,7 @@
/**
* Layer 1: Deterministic Browse Command Testing
*
* Runs predefined opencli operate command sequences against real websites.
* Runs predefined opencli browser command sequences against real websites.
* No LLM involved — tests command reliability only.
*
* Usage:
@@ -137,12 +137,12 @@ function main() {
// Close browser between tasks for clean state
if (i < tasks.length - 1) {
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
}
}
// Final close
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
// Summary
const trainResults = results.filter(r => r.set === 'train');
+3 -3
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env npx tsx
/**
* Layer 5: Publish Testing — end-to-end content creation via operate commands
* Layer 5: Publish Testing — end-to-end content creation via browser commands
*
* Tests the full chain: read content → navigate to platform → fill title+body → (optionally) publish → verify → cleanup
*
@@ -179,12 +179,12 @@ function main() {
// Close browser between tasks for clean state
if (i < tasks.length - 1) {
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
}
}
// Final close
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
// Summary
const totalPassed = results.filter(r => r.passed).length;
+5 -5
View File
@@ -2,7 +2,7 @@
/**
* Layer 4: Save as CLI Testing — "Save as CLI" Pipeline
*
* Tests the full operate init → write adapter → operate verify flow.
* Tests the full browser init → write adapter → browser verify flow.
* Validates that browser exploration can be crystallized into reusable CLI adapters.
*
* Usage:
@@ -55,7 +55,7 @@ function judge(criteria: JudgeCriteria, output: string): boolean {
case 'contains':
return output.toLowerCase().includes(criteria.value.toLowerCase());
case 'arrayMinLength': {
// operate verify outputs table text; try JSON parse first, then count non-empty lines
// browser verify outputs table text; try JSON parse first, then count non-empty lines
try {
const arr = JSON.parse(output);
if (Array.isArray(arr)) return arr.length >= criteria.minLength;
@@ -121,7 +121,7 @@ function runTask(task: SaveTask): TaskResult {
try {
// Phase 1: init — create scaffold
const initOutput = runCommand(`opencli operate init ${site}/${command}`);
const initOutput = runCommand(`opencli browser init ${site}/${command}`);
if (!existsSync(adapterPath)) {
return {
name: task.name, phase: 'init', passed: false,
@@ -141,9 +141,9 @@ function runTask(task: SaveTask): TaskResult {
writeFileSync(adapterPath, task.adapter, 'utf-8');
}
// Phase 3: verify — run the adapter via operate verify
// Phase 3: verify — run the adapter via browser verify
const verifyOutput = runCommand(
`opencli operate verify ${site}/${command}`,
`opencli browser verify ${site}/${command}`,
45000, // longer timeout for network calls
);
+4 -4
View File
@@ -2,7 +2,7 @@
/**
* Layer 2: Claude Code Skill E2E Testing (LLM Judge)
*
* Spawns Claude Code with the opencli-operate skill. Claude Code
* Spawns Claude Code with the opencli-adapter-author skill. Claude Code
* completes the task using browse commands AND judges its own result.
*
* Task format: YAML with judge_context (multi-criteria, like Browser Use)
@@ -19,7 +19,7 @@ import { fileURLToPath } from 'node:url';
const __dirname = dirname(fileURLToPath(import.meta.url));
const RESULTS_DIR = join(__dirname, 'results');
const SKILL_PATH = join(__dirname, '..', 'skills', 'opencli-operate', 'SKILL.md');
const SKILL_PATH = join(__dirname, '..', 'skills', 'opencli-adapter-author', 'SKILL.md');
// ── Types ──────────────────────────────────────────────────────────
@@ -100,7 +100,7 @@ function runSkillTask(task: SkillTask): TaskResult {
const urlPart = task.url ? ` Start URL: ${task.url}` : '';
const criteria = task.judge_context.map((c, i) => `${i + 1}. ${c}`).join('\n');
const prompt = `Complete this browser task using opencli operate commands:
const prompt = `Complete this browser task using opencli browser commands:
TASK: ${task.task}${urlPart}
@@ -110,7 +110,7 @@ ${criteria}
At the very end of your response, output a JSON verdict on its own line:
{"success": true/false, "explanation": "brief explanation"}
Always close the browser with 'opencli operate close' when done.`;
Always close the browser with 'opencli browser close' when done.`;
try {
const output = execSync(
+2 -2
View File
@@ -168,12 +168,12 @@ function main() {
// Close browser between tasks for clean state
if (i < tasks.length - 1) {
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
}
}
// Final close
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
// Summary by layer
const layers = [...new Set(results.map(r => r.layer))].sort();
+2 -2
View File
@@ -178,12 +178,12 @@ function main() {
// Close browser between tasks for clean state
if (i < tasks.length - 1) {
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
}
}
// Final close
try { runCommand('opencli operate close'); } catch { /* ignore */ }
try { runCommand('opencli browser close'); } catch { /* ignore */ }
// Summary by layer
const layers = [...new Set(results.map(r => r.layer))].sort();
@@ -1,14 +1,14 @@
/**
* Preset: Operate Command Reliability
* Preset: Browser Command Reliability
*
* Optimizes opencli operate commands against the Layer 1 deterministic test suite.
* Optimizes opencli browser commands against the Layer 1 deterministic test suite.
* Metric: number of passing browse-tasks (out of 59).
*/
import type { AutoResearchConfig } from '../config.js';
export const operateReliability: AutoResearchConfig = {
goal: 'Increase operate command pass rate to 59/59 (100%)',
export const browserReliability: AutoResearchConfig = {
goal: 'Increase browser command pass rate to 59/59 (100%)',
scope: [
'src/browser/dom-snapshot.ts',
'src/browser/dom-helpers.ts',
+3 -3
View File
@@ -1,4 +1,4 @@
export { operateReliability } from './operate-reliability.js';
export { browserReliability } from './browser-reliability.js';
export { skillQuality } from './skill-quality.js';
export { v2exReliability } from './v2ex-reliability.js';
export { zhihuReliability } from './zhihu-reliability.js';
@@ -6,7 +6,7 @@ export { combinedReliability } from './combined-reliability.js';
export { saveReliability } from './save-reliability.js';
import type { AutoResearchConfig } from '../config.js';
import { operateReliability } from './operate-reliability.js';
import { browserReliability } from './browser-reliability.js';
import { skillQuality } from './skill-quality.js';
import { v2exReliability } from './v2ex-reliability.js';
import { zhihuReliability } from './zhihu-reliability.js';
@@ -14,7 +14,7 @@ import { combinedReliability } from './combined-reliability.js';
import { saveReliability } from './save-reliability.js';
export const PRESETS: Record<string, AutoResearchConfig> = {
'operate-reliability': operateReliability,
'browser-reliability': browserReliability,
'skill-quality': skillQuality,
'v2ex-reliability': v2exReliability,
'zhihu-reliability': zhihuReliability,
+3 -3
View File
@@ -1,7 +1,7 @@
/**
* Preset: Save as CLI Reliability
*
* Optimizes the "Save as CLI" pipeline: operate init → write adapter → run.
* Optimizes the "Save as CLI" pipeline: browser init → write adapter → run.
* Covers PUBLIC (no auth) and COOKIE (browser session) strategies.
* Metric: number of passing save-tasks.
*/
@@ -9,12 +9,12 @@
import type { AutoResearchConfig } from '../config.js';
export const saveReliability: AutoResearchConfig = {
goal: 'Increase "Save as CLI" pipeline pass rate to 100%. The flow is: operate init creates a scaffold, user writes adapter code, opencli discovers and runs it. Covers both PUBLIC (fetch API) and COOKIE (browser session) strategies. Focus on: init template correctness, user CLI discovery, adapter loading, verify command robustness, and browser session handling.',
goal: 'Increase "Save as CLI" pipeline pass rate to 100%. The flow is: browser init creates a scaffold, user writes adapter code, opencli discovers and runs it. Covers both PUBLIC (fetch API) and COOKIE (browser session) strategies. Focus on: init template correctness, user CLI discovery, adapter loading, verify command robustness, and browser session handling.',
scope: [
'src/cli.ts',
'src/discovery.ts',
'src/registry.ts',
'skills/opencli-operate/SKILL.md',
'skills/opencli-adapter-author/SKILL.md',
'autoresearch/save-tasks.json',
'autoresearch/save-adapters/*.ts',
],
+2 -2
View File
@@ -1,7 +1,7 @@
/**
* Preset: Skill E2E Quality
*
* Optimizes the opencli-operate SKILL.md against the Layer 2 LLM E2E test suite.
* Optimizes the opencli-adapter-author SKILL.md against the Layer 2 LLM E2E test suite.
* Metric: number of passing skill-tasks (out of 35).
*/
@@ -10,7 +10,7 @@ import type { AutoResearchConfig } from '../config.js';
export const skillQuality: AutoResearchConfig = {
goal: 'Increase skill E2E pass rate to 35/35 (100%)',
scope: [
'skills/opencli-operate/SKILL.md',
'skills/opencli-adapter-author/SKILL.md',
],
metric: 'pass_count',
direction: 'higher',
+2 -2
View File
@@ -1,14 +1,14 @@
/**
* Preset: V2EX Command Reliability
*
* Optimizes opencli operate commands against the V2EX-specific test suite.
* Optimizes opencli browser commands against the V2EX-specific test suite.
* 40 tasks across 5 difficulty layers (atomic → complex chain).
*/
import type { AutoResearchConfig } from '../config.js';
export const v2exReliability: AutoResearchConfig = {
goal: 'Increase V2EX operate command pass rate to 40/40 (100%)',
goal: 'Increase V2EX browser command pass rate to 40/40 (100%)',
scope: [
'src/browser/dom-snapshot.ts',
'src/browser/dom-helpers.ts',
+2 -2
View File
@@ -1,7 +1,7 @@
/**
* Preset: Zhihu Command Reliability
*
* Optimizes opencli operate commands against the Zhihu test suite.
* Optimizes opencli browser commands against the Zhihu test suite.
* 60 tasks across 8 difficulty layers (atomic → complex long chain).
* Zhihu is a React SPA with lazy loading, making it harder than V2EX.
*/
@@ -9,7 +9,7 @@
import type { AutoResearchConfig } from '../config.js';
export const zhihuReliability: AutoResearchConfig = {
goal: 'Increase Zhihu operate command pass rate to 60/60 (100%)',
goal: 'Increase Zhihu browser command pass rate to 60/60 (100%)',
scope: [
'src/browser/dom-snapshot.ts',
'src/browser/dom-helpers.ts',
+144 -144
View File
@@ -5,12 +5,12 @@
"type": "fill-only",
"description": "Navigate to tweet composer, fill in content (no publish)",
"steps": [
"opencli operate open https://x.com/compose/tweet",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] OpenCLI publish eval - fill only test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
"opencli browser open https://x.com/compose/tweet",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] OpenCLI publish eval - fill only test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -24,26 +24,26 @@
"type": "publish",
"description": "Post a tweet, verify success, then delete it",
"steps": [
"opencli operate open https://x.com/compose/tweet",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] OpenCLI publish eval ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const btn = document.querySelector('[data-testid=\\\"tweetButton\\\"]') || document.querySelector('[data-testid=\\\"tweetButtonInline\\\"]'); if (btn && !btn.disabled) { btn.click(); return 'clicked'; } return 'btn-not-ready'; })()\"",
"opencli operate wait time 4",
"opencli operate eval \"document.querySelector('[data-testid=\\\"toast\\\"]')?.textContent || document.title\""
"opencli browser open https://x.com/compose/tweet",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] OpenCLI publish eval ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const btn = document.querySelector('[data-testid=\\\"tweetButton\\\"]') || document.querySelector('[data-testid=\\\"tweetButtonInline\\\"]'); if (btn && !btn.disabled) { btn.click(); return 'clicked'; } return 'btn-not-ready'; })()\"",
"opencli browser wait time 4",
"opencli browser eval \"document.querySelector('[data-testid=\\\"toast\\\"]')?.textContent || document.title\""
],
"judge": {
"type": "matchesPattern",
"pattern": "post|sent|Your post|X"
},
"cleanup": [
"opencli operate open https://x.com/home",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); for (const t of tweets) { if (t.textContent?.includes('[AutoTest]')) { const more = t.querySelector('[data-testid=\\\"caret\\\"]'); if (more) { more.click(); return 'found-menu'; } } } return 'no-autotest-tweet'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Delete')) { item.click(); return 'clicked-delete'; } } return 'no-delete-option'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const confirm = document.querySelector('[data-testid=\\\"confirmationSheetConfirm\\\"]'); if (confirm) { confirm.click(); return 'deleted'; } return 'no-confirm'; })()\""
"opencli browser open https://x.com/home",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); for (const t of tweets) { if (t.textContent?.includes('[AutoTest]')) { const more = t.querySelector('[data-testid=\\\"caret\\\"]'); if (more) { more.click(); return 'found-menu'; } } } return 'no-autotest-tweet'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Delete')) { item.click(); return 'clicked-delete'; } } return 'no-delete-option'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const confirm = document.querySelector('[data-testid=\\\"confirmationSheetConfirm\\\"]'); if (confirm) { confirm.click(); return 'deleted'; } return 'no-confirm'; })()\""
],
"note": "6-step chain: open compose → paste text → click post → wait → verify toast → cleanup: find tweet → menu → delete → confirm"
},
@@ -53,29 +53,29 @@
"type": "publish",
"description": "Read HN top story title, compose a tweet about it, post, then delete",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('.titleline a')?.textContent?.trim() || 'no-title'\"",
"opencli operate open https://x.com/compose/tweet",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const title = document.title || 'HN Story'; const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Interesting from HN: ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const btn = document.querySelector('[data-testid=\\\"tweetButton\\\"]') || document.querySelector('[data-testid=\\\"tweetButtonInline\\\"]'); if (btn && !btn.disabled) { btn.click(); return 'clicked'; } return 'btn-not-ready'; })()\"",
"opencli operate wait time 4",
"opencli operate eval \"document.querySelector('[data-testid=\\\"toast\\\"]')?.textContent || document.title\""
"opencli browser open https://news.ycombinator.com",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('.titleline a')?.textContent?.trim() || 'no-title'\"",
"opencli browser open https://x.com/compose/tweet",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const title = document.title || 'HN Story'; const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Interesting from HN: ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const btn = document.querySelector('[data-testid=\\\"tweetButton\\\"]') || document.querySelector('[data-testid=\\\"tweetButtonInline\\\"]'); if (btn && !btn.disabled) { btn.click(); return 'clicked'; } return 'btn-not-ready'; })()\"",
"opencli browser wait time 4",
"opencli browser eval \"document.querySelector('[data-testid=\\\"toast\\\"]')?.textContent || document.title\""
],
"judge": {
"type": "matchesPattern",
"pattern": "post|sent|Your post|X"
},
"cleanup": [
"opencli operate open https://x.com/home",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); for (const t of tweets) { if (t.textContent?.includes('[AutoTest]')) { const more = t.querySelector('[data-testid=\\\"caret\\\"]'); if (more) { more.click(); return 'found-menu'; } } } return 'no-autotest-tweet'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Delete')) { item.click(); return 'clicked-delete'; } } return 'no-delete-option'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const confirm = document.querySelector('[data-testid=\\\"confirmationSheetConfirm\\\"]'); if (confirm) { confirm.click(); return 'deleted'; } return 'no-confirm'; })()\""
"opencli browser open https://x.com/home",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); for (const t of tweets) { if (t.textContent?.includes('[AutoTest]')) { const more = t.querySelector('[data-testid=\\\"caret\\\"]'); if (more) { more.click(); return 'found-menu'; } } } return 'no-autotest-tweet'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Delete')) { item.click(); return 'clicked-delete'; } } return 'no-delete-option'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const confirm = document.querySelector('[data-testid=\\\"confirmationSheetConfirm\\\"]'); if (confirm) { confirm.click(); return 'deleted'; } return 'no-confirm'; })()\""
],
"note": "9-step cross-site chain: read HN title → navigate to twitter compose → paste content → post → verify → cleanup delete"
},
@@ -85,13 +85,13 @@
"type": "fill-only",
"description": "Navigate to own profile, find latest tweet, open reply box, fill reply text",
"steps": [
"opencli operate open https://x.com/home",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const reply = tweet.querySelector('[data-testid=\\\"reply\\\"]'); if (reply) { reply.click(); return 'reply-clicked'; } return 'no-reply-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Reply test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
"opencli browser open https://x.com/home",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const reply = tweet.querySelector('[data-testid=\\\"reply\\\"]'); if (reply) { reply.click(); return 'reply-clicked'; } return 'no-reply-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Reply test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -105,13 +105,13 @@
"type": "fill-only",
"description": "Navigate to a popular question, open answer editor, fill in answer content (no publish)",
"steps": [
"opencli operate open https://www.zhihu.com/question/19550225",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const btn = document.querySelector('[data-zop-retarget=\\\"answer\\\"]') || Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 这是一个 OpenCLI 发文测试,时间戳: ' + Date.now() + '</p><p>这段内容用于验证 operate 命令链的完整性。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''\""
"opencli browser open https://www.zhihu.com/question/19550225",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const btn = document.querySelector('[data-zop-retarget=\\\"answer\\\"]') || Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 这是一个 OpenCLI 发文测试,时间戳: ' + Date.now() + '</p><p>这段内容用于验证 browser 命令链的完整性。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -125,13 +125,13 @@
"type": "fill-only",
"description": "Navigate to zhihu article editor (zhuanlan), fill title + body (no publish)",
"steps": [
"opencli operate open https://zhuanlan.zhihu.com/write",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const ta = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder]'); if (!ta) return 'no-title-input'; ta.focus(); var nativeSetter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set; nativeSetter.call(ta, '[AutoTest] OpenCLI 发文能力验证 ' + Date.now()); ta.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const editor = document.querySelector('[contenteditable=true]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>这是 OpenCLI autoresearch 发文测试集的一部分。</p><p>测试链路:导航 → 填写标题 → 填写正文 → 验证内容。</p><p>时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder]'))?.value || ''; const body = document.querySelector('[contenteditable=true]')?.textContent || ''; return JSON.stringify({ title: title.slice(0, 50), body: body.slice(0, 50) }); })()\""
"opencli browser open https://zhuanlan.zhihu.com/write",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const ta = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder]'); if (!ta) return 'no-title-input'; ta.focus(); var nativeSetter = Object.getOwnPropertyDescriptor(window.HTMLTextAreaElement.prototype, 'value').set; nativeSetter.call(ta, '[AutoTest] OpenCLI 发文能力验证 ' + Date.now()); ta.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const editor = document.querySelector('[contenteditable=true]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>这是 OpenCLI autoresearch 发文测试集的一部分。</p><p>测试链路:导航 → 填写标题 → 填写正文 → 验证内容。</p><p>时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder]'))?.value || ''; const body = document.querySelector('[contenteditable=true]')?.textContent || ''; return JSON.stringify({ title: title.slice(0, 50), body: body.slice(0, 50) }); })()\""
],
"judge": {
"type": "contains",
@@ -145,16 +145,16 @@
"type": "fill-only",
"description": "Read HN top story, then navigate to zhihu question and fill an answer about it",
"steps": [
"opencli operate open https://news.ycombinator.com",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const a = document.querySelector('.titleline a'); return a ? a.textContent?.trim() : 'no-title'; })()\"",
"opencli operate open https://www.zhihu.com/question/19550225",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const btn = Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')) || document.querySelector('[data-zop-retarget=\\\"answer\\\"]'); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=true]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 分享一个来自 Hacker News 的有趣内容</p><p>这是一个跨平台内容搬运测试,时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=true]'))?.textContent || ''\""
"opencli browser open https://news.ycombinator.com",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const a = document.querySelector('.titleline a'); return a ? a.textContent?.trim() : 'no-title'; })()\"",
"opencli browser open https://www.zhihu.com/question/19550225",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const btn = Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')) || document.querySelector('[data-zop-retarget=\\\"answer\\\"]'); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=true]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 分享一个来自 Hacker News 的有趣内容</p><p>这是一个跨平台内容搬运测试,时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=true]'))?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -168,16 +168,16 @@
"type": "fill-only",
"description": "Navigate to compose, type first tweet, add thread tweet, type second tweet, verify both",
"steps": [
"opencli operate open https://x.com/compose/tweet",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Thread tweet 1 - ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'first-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const addBtn = document.querySelector('[data-testid=\\\"addButton\\\"]') || document.querySelector('[aria-label=\\\"Add post\\\"]'); if (addBtn) { addBtn.click(); return 'thread-added'; } return 'no-add-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const boxes = document.querySelectorAll('[data-testid=\\\"tweetTextarea_0\\\"]'); const box = boxes[boxes.length - 1]; if (!box) return 'no-second-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Thread tweet 2 - continuation'); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'second-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const boxes = document.querySelectorAll('[data-testid=\\\"tweetTextarea_0\\\"]'); const t1 = boxes[0]?.textContent || ''; const t2 = boxes[boxes.length - 1]?.textContent || ''; return JSON.stringify({ tweet1: t1, tweet2: t2 }); })()\""
"opencli browser open https://x.com/compose/tweet",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Thread tweet 1 - ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'first-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const addBtn = document.querySelector('[data-testid=\\\"addButton\\\"]') || document.querySelector('[aria-label=\\\"Add post\\\"]'); if (addBtn) { addBtn.click(); return 'thread-added'; } return 'no-add-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const boxes = document.querySelectorAll('[data-testid=\\\"tweetTextarea_0\\\"]'); const box = boxes[boxes.length - 1]; if (!box) return 'no-second-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Thread tweet 2 - continuation'); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'second-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const boxes = document.querySelectorAll('[data-testid=\\\"tweetTextarea_0\\\"]'); const t1 = boxes[0]?.textContent || ''; const t2 = boxes[boxes.length - 1]?.textContent || ''; return JSON.stringify({ tweet1: t1, tweet2: t2 }); })()\""
],
"judge": {
"type": "contains",
@@ -191,15 +191,15 @@
"type": "fill-only",
"description": "Navigate to home, find first tweet, open retweet menu, select Quote, fill quote text, verify",
"steps": [
"opencli operate open https://x.com/home",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const retweet = tweet.querySelector('[data-testid=\\\"retweet\\\"]'); if (retweet) { retweet.click(); return 'retweet-menu-opened'; } return 'no-retweet-btn'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Quote') || item.textContent?.includes('引用')) { item.click(); return 'quote-selected'; } } return 'no-quote-option'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Quote retweet test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'quote-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
"opencli browser open https://x.com/home",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const retweet = tweet.querySelector('[data-testid=\\\"retweet\\\"]'); if (retweet) { retweet.click(); return 'retweet-menu-opened'; } return 'no-retweet-btn'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const items = document.querySelectorAll('[role=\\\"menuitem\\\"]'); for (const item of items) { if (item.textContent?.includes('Quote') || item.textContent?.includes('引用')) { item.click(); return 'quote-selected'; } } return 'no-quote-option'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Quote retweet test ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'quote-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -213,14 +213,14 @@
"type": "fill-only",
"description": "Search 'opencli' on twitter, find first result, click reply, fill reply text, verify",
"steps": [
"opencli operate open https://x.com/search?q=opencli&src=typed_query&f=live",
"opencli operate wait time 4",
"opencli operate eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); if (tweets.length === 0) return 'no-results'; return 'found-' + tweets.length + '-results'; })()\"",
"opencli operate eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const reply = tweet.querySelector('[data-testid=\\\"reply\\\"]'); if (reply) { reply.click(); return 'reply-clicked'; } return 'no-reply-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Reply from search result ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'reply-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
"opencli browser open https://x.com/search?q=opencli&src=typed_query&f=live",
"opencli browser wait time 4",
"opencli browser eval \"(() => { const tweets = document.querySelectorAll('[data-testid=\\\"tweet\\\"]'); if (tweets.length === 0) return 'no-results'; return 'found-' + tweets.length + '-results'; })()\"",
"opencli browser eval \"(() => { const tweet = document.querySelector('[data-testid=\\\"tweet\\\"]'); if (!tweet) return 'no-tweet'; const reply = tweet.querySelector('[data-testid=\\\"reply\\\"]'); if (reply) { reply.click(); return 'reply-clicked'; } return 'no-reply-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Reply from search result ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'reply-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -234,16 +234,16 @@
"type": "fill-only",
"description": "Search 'AI agent' on zhihu, click first question result, click 写回答, fill answer, verify",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=AI%20agent",
"opencli operate wait time 4",
"opencli operate eval \"(() => { const links = document.querySelectorAll('a[href*=\\\"/question/\\\"]'); if (links.length === 0) return 'no-question-links'; const link = links[0]; const href = link.getAttribute('href'); return 'found: ' + href; })()\"",
"opencli operate eval \"(() => { const links = document.querySelectorAll('a[href*=\\\"/question/\\\"]'); if (links.length === 0) return 'no-links'; const link = links[0]; const href = link.getAttribute('href'); const match = href.match(/\\\\/question\\\\/(\\\\d+)/); if (match) { window.location.href = 'https://www.zhihu.com/question/' + match[1]; return 'navigating-to-question'; } link.click(); return 'clicked-link'; })()\"",
"opencli operate wait time 4",
"opencli operate eval \"(() => { const btn = document.querySelector('[data-zop-retarget=\\\"answer\\\"]') || Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')) || Array.from(document.querySelectorAll('a')).find(a => a.textContent?.includes('写回答')); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] AI agent 搜索后回答测试 ' + Date.now() + '</p><p>这是通过搜索 → 进入问题 → 填写回答的完整链路测试。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''\""
"opencli browser open https://www.zhihu.com/search?type=content&q=AI%20agent",
"opencli browser wait time 4",
"opencli browser eval \"(() => { const links = document.querySelectorAll('a[href*=\\\"/question/\\\"]'); if (links.length === 0) return 'no-question-links'; const link = links[0]; const href = link.getAttribute('href'); return 'found: ' + href; })()\"",
"opencli browser eval \"(() => { const links = document.querySelectorAll('a[href*=\\\"/question/\\\"]'); if (links.length === 0) return 'no-links'; const link = links[0]; const href = link.getAttribute('href'); const match = href.match(/\\\\/question\\\\/(\\\\d+)/); if (match) { window.location.href = 'https://www.zhihu.com/question/' + match[1]; return 'navigating-to-question'; } link.click(); return 'clicked-link'; })()\"",
"opencli browser wait time 4",
"opencli browser eval \"(() => { const btn = document.querySelector('[data-zop-retarget=\\\"answer\\\"]') || Array.from(document.querySelectorAll('button')).find(b => b.textContent?.includes('写回答')) || Array.from(document.querySelectorAll('a')).find(a => a.textContent?.includes('写回答')); if (btn) { btn.click(); return 'editor-opened'; } return 'no-answer-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] AI agent 搜索后回答测试 ' + Date.now() + '</p><p>这是通过搜索 → 进入问题 → 填写回答的完整链路测试。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -257,15 +257,15 @@
"type": "fill-only",
"description": "Navigate to question page, scroll to first answer, click comment, fill comment text, verify",
"steps": [
"opencli operate open https://www.zhihu.com/question/19550225",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const answer = document.querySelector('[data-testid=\\\"answer\\\"]') || document.querySelector('.AnswerItem') || document.querySelector('.List-item'); if (answer) { answer.scrollIntoView({ behavior: 'smooth', block: 'center' }); return 'answer-scrolled'; } return 'no-answer'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const commentBtns = document.querySelectorAll('button'); for (const btn of commentBtns) { if (btn.textContent?.match(/评论|条评论|comment/i)) { btn.click(); return 'comment-opened: ' + btn.textContent.trim(); } } const commentIcons = document.querySelectorAll('[data-testid=\\\"comment\\\"]') || []; for (const icon of commentIcons) { icon.click(); return 'comment-icon-clicked'; } return 'no-comment-btn'; })()\"",
"opencli operate wait time 2",
"opencli operate eval \"(() => { const editor = document.querySelector('.CommentEditor textarea') || document.querySelector('[placeholder*=\\\"评论\\\"]') || document.querySelector('[placeholder*=\\\"comment\\\"]') || document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-comment-editor'; editor.focus(); if (editor.tagName === 'TEXTAREA' || editor.tagName === 'INPUT') { editor.value = '[AutoTest] 评论测试 ' + Date.now(); editor.dispatchEvent(new Event('input', { bubbles: true })); } else { editor.innerHTML = '<p>[AutoTest] 评论测试 ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); } return 'comment-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const editor = document.querySelector('.CommentEditor textarea') || document.querySelector('[placeholder*=\\\"评论\\\"]') || document.querySelector('[placeholder*=\\\"comment\\\"]') || document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; return editor.value || editor.textContent || ''; })()\""
"opencli browser open https://www.zhihu.com/question/19550225",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const answer = document.querySelector('[data-testid=\\\"answer\\\"]') || document.querySelector('.AnswerItem') || document.querySelector('.List-item'); if (answer) { answer.scrollIntoView({ behavior: 'smooth', block: 'center' }); return 'answer-scrolled'; } return 'no-answer'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const commentBtns = document.querySelectorAll('button'); for (const btn of commentBtns) { if (btn.textContent?.match(/评论|条评论|comment/i)) { btn.click(); return 'comment-opened: ' + btn.textContent.trim(); } } const commentIcons = document.querySelectorAll('[data-testid=\\\"comment\\\"]') || []; for (const icon of commentIcons) { icon.click(); return 'comment-icon-clicked'; } return 'no-comment-btn'; })()\"",
"opencli browser wait time 2",
"opencli browser eval \"(() => { const editor = document.querySelector('.CommentEditor textarea') || document.querySelector('[placeholder*=\\\"评论\\\"]') || document.querySelector('[placeholder*=\\\"comment\\\"]') || document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-comment-editor'; editor.focus(); if (editor.tagName === 'TEXTAREA' || editor.tagName === 'INPUT') { editor.value = '[AutoTest] 评论测试 ' + Date.now(); editor.dispatchEvent(new Event('input', { bubbles: true })); } else { editor.innerHTML = '<p>[AutoTest] 评论测试 ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); } return 'comment-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const editor = document.querySelector('.CommentEditor textarea') || document.querySelector('[placeholder*=\\\"评论\\\"]') || document.querySelector('[placeholder*=\\\"comment\\\"]') || document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; return editor.value || editor.textContent || ''; })()\""
],
"judge": {
"type": "contains",
@@ -279,15 +279,15 @@
"type": "fill-only",
"description": "Navigate to zhuanlan editor, fill title, fill body with multiple paragraphs and bold text, verify",
"steps": [
"opencli operate open https://zhuanlan.zhihu.com/write",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const titleInput = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'); if (!titleInput) return 'no-title-input'; titleInput.focus(); titleInput.value = '[AutoTest] 格式化文章测试 ' + Date.now(); titleInput.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>这是第一段:OpenCLI 格式化发文测试。</p><p><strong>[AutoTest-Bold] 这是加粗的第二段,用于验证富文本格式。</strong></p><p>这是第三段,包含普通文本内容,时间戳: ' + Date.now() + '。</p><p>这是第四段,测试多段落填充能力。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled-with-formatting'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; const hasBold = editor.querySelector('strong') || editor.querySelector('b'); const paragraphs = editor.querySelectorAll('p'); return JSON.stringify({ paragraphCount: paragraphs.length, hasBold: !!hasBold, preview: editor.textContent?.slice(0, 80) }); })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'))?.value || ''; const body = (document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''; return JSON.stringify({ title: title.slice(0, 60), bodyHasBold: body.includes('AutoTest-Bold'), bodyLength: body.length }); })()\""
"opencli browser open https://zhuanlan.zhihu.com/write",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const titleInput = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'); if (!titleInput) return 'no-title-input'; titleInput.focus(); titleInput.value = '[AutoTest] 格式化文章测试 ' + Date.now(); titleInput.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>这是第一段:OpenCLI 格式化发文测试。</p><p><strong>[AutoTest-Bold] 这是加粗的第二段,用于验证富文本格式。</strong></p><p>这是第三段,包含普通文本内容,时间戳: ' + Date.now() + '。</p><p>这是第四段,测试多段落填充能力。</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled-with-formatting'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; const hasBold = editor.querySelector('strong') || editor.querySelector('b'); const paragraphs = editor.querySelectorAll('p'); return JSON.stringify({ paragraphCount: paragraphs.length, hasBold: !!hasBold, preview: editor.textContent?.slice(0, 80) }); })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'))?.value || ''; const body = (document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''; return JSON.stringify({ title: title.slice(0, 60), bodyHasBold: body.includes('AutoTest-Bold'), bodyLength: body.length }); })()\""
],
"judge": {
"type": "contains",
@@ -301,16 +301,16 @@
"type": "fill-only",
"description": "Read zhihu hot topic title, navigate to twitter compose, fill tweet with zhihu content, verify",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const hotItem = document.querySelector('.HotItem-content a') || document.querySelector('.HotList-item a') || document.querySelector('[data-testid=\\\"hot-item\\\"] a') || document.querySelector('.HotItem a'); if (hotItem) return hotItem.textContent?.trim()?.slice(0, 60) || 'no-text'; const titles = document.querySelectorAll('h2'); for (const t of titles) { if (t.textContent?.trim().length > 5) return t.textContent.trim().slice(0, 60); } return 'no-hot-topic'; })()\"",
"opencli operate state save zhihu_hot_title",
"opencli operate open https://x.com/compose/tweet",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli operate eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Zhihu热榜话题搬运: 知乎上正在热议的话题 - ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'tweet-filled-with-zhihu'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const hotItem = document.querySelector('.HotItem-content a') || document.querySelector('.HotList-item a') || document.querySelector('[data-testid=\\\"hot-item\\\"] a') || document.querySelector('.HotItem a'); if (hotItem) return hotItem.textContent?.trim()?.slice(0, 60) || 'no-text'; const titles = document.querySelectorAll('h2'); for (const t of titles) { if (t.textContent?.trim().length > 5) return t.textContent.trim().slice(0, 60); } return 'no-hot-topic'; })()\"",
"opencli browser state save zhihu_hot_title",
"opencli browser open https://x.com/compose/tweet",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]') ? 'composer-ready' : 'not-found'\"",
"opencli browser eval \"(() => { const box = document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]'); if (!box) return 'no-box'; box.focus(); const dt = new DataTransfer(); dt.setData('text/plain', '[AutoTest] Zhihu热榜话题搬运: 知乎上正在热议的话题 - ' + Date.now()); box.dispatchEvent(new ClipboardEvent('paste', { clipboardData: dt, bubbles: true, cancelable: true })); return 'tweet-filled-with-zhihu'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelector('[data-testid=\\\"tweetTextarea_0\\\"]')?.textContent || ''\""
],
"judge": {
"type": "contains",
@@ -324,17 +324,17 @@
"type": "fill-only",
"description": "Read twitter trending/explore topic, navigate to zhihu zhuanlan editor, fill title and body, verify",
"steps": [
"opencli operate open https://x.com/explore/tabs/trending",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const trends = document.querySelectorAll('[data-testid=\\\"trend\\\"]'); if (trends.length > 0) { const first = trends[0]; return first.textContent?.trim()?.slice(0, 80) || 'no-text'; } const spans = document.querySelectorAll('span'); for (const s of spans) { if (s.textContent?.startsWith('#') || s.textContent?.includes('Trending')) { return s.textContent.trim().slice(0, 80); } } return 'no-trending-topic'; })()\"",
"opencli operate state save twitter_trending",
"opencli operate open https://zhuanlan.zhihu.com/write",
"opencli operate wait time 3",
"opencli operate eval \"(() => { const titleInput = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'); if (!titleInput) return 'no-title-input'; titleInput.focus(); titleInput.value = '[AutoTest] Twitter热点搬运: 来自推特的热门话题 ' + Date.now(); titleInput.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 这篇文章搬运自 Twitter 热门话题。</p><p>Twitter 上正在讨论的热门话题为大家带来了新的视角和思考。</p><p>时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled'; })()\"",
"opencli operate wait time 1",
"opencli operate eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'))?.value || ''; const body = (document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''; return JSON.stringify({ title: title.slice(0, 60), body: body.slice(0, 60) }); })()\""
"opencli browser open https://x.com/explore/tabs/trending",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const trends = document.querySelectorAll('[data-testid=\\\"trend\\\"]'); if (trends.length > 0) { const first = trends[0]; return first.textContent?.trim()?.slice(0, 80) || 'no-text'; } const spans = document.querySelectorAll('span'); for (const s of spans) { if (s.textContent?.startsWith('#') || s.textContent?.includes('Trending')) { return s.textContent.trim().slice(0, 80); } } return 'no-trending-topic'; })()\"",
"opencli browser state save twitter_trending",
"opencli browser open https://zhuanlan.zhihu.com/write",
"opencli browser wait time 3",
"opencli browser eval \"(() => { const titleInput = document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'); if (!titleInput) return 'no-title-input'; titleInput.focus(); titleInput.value = '[AutoTest] Twitter热点搬运: 来自推特的热门话题 ' + Date.now(); titleInput.dispatchEvent(new Event('input', { bubbles: true })); return 'title-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const editor = document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'); if (!editor) return 'no-editor'; editor.focus(); editor.innerHTML = '<p>[AutoTest] 这篇文章搬运自 Twitter 热门话题。</p><p>Twitter 上正在讨论的热门话题为大家带来了新的视角和思考。</p><p>时间戳: ' + Date.now() + '</p>'; editor.dispatchEvent(new Event('input', { bubbles: true })); return 'body-filled'; })()\"",
"opencli browser wait time 1",
"opencli browser eval \"(() => { const title = (document.querySelector('.WriteIndex-titleInput textarea') || document.querySelector('textarea[placeholder*=\\\"标题\\\"]'))?.value || ''; const body = (document.querySelector('.ql-editor') || document.querySelector('[contenteditable=\\\"true\\\"]'))?.textContent || ''; return JSON.stringify({ title: title.slice(0, 60), body: body.slice(0, 60) }); })()\""
],
"judge": {
"type": "contains",
+1 -1
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Layer 4: Save as CLI — test the full save pipeline
# Tests: operate init → write adapter → operate verify
# Tests: browser init → write adapter → browser verify
set -euo pipefail
cd "$(dirname "$0")/.."
+249 -249
View File
@@ -5,7 +5,7 @@
{
"name": "v2ex-open-home",
"steps": [
"opencli operate open https://v2ex.com/"
"opencli browser open https://v2ex.com/"
],
"judge": {
"type": "contains",
@@ -15,8 +15,8 @@
{
"name": "v2ex-state-home",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state"
"opencli browser open https://v2ex.com/",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
@@ -26,8 +26,8 @@
{
"name": "v2ex-get-title",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"document.title\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "contains",
@@ -37,8 +37,8 @@
{
"name": "v2ex-click-tab",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"(() => { const a = document.querySelector('a[href=\\\"/?tab=tech\\\"]'); if(a){a.click(); return 'clicked';} return 'not found'; })()\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"(() => { const a = document.querySelector('a[href=\\\"/?tab=tech\\\"]'); if(a){a.click(); return 'clicked';} return 'not found'; })()\""
],
"judge": {
"type": "contains",
@@ -48,9 +48,9 @@
{
"name": "v2ex-scroll-down",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate scroll down --amount 500",
"opencli operate eval \"window.scrollY > 100 ? 'scrolled' : 'not scrolled'\""
"opencli browser open https://v2ex.com/",
"opencli browser scroll down --amount 500",
"opencli browser eval \"window.scrollY > 100 ? 'scrolled' : 'not scrolled'\""
],
"judge": {
"type": "contains",
@@ -60,8 +60,8 @@
{
"name": "v2ex-get-first-topic-text",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"document.querySelector('a[href^=\\\"/t/\\\"]')?.textContent?.trim()\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"document.querySelector('a[href^=\\\"/t/\\\"]')?.textContent?.trim()\""
],
"judge": {
"type": "nonEmpty"
@@ -70,8 +70,8 @@
{
"name": "v2ex-eval-extract-titles",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -81,8 +81,8 @@
{
"name": "v2ex-get-url",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate get url"
"opencli browser open https://v2ex.com/",
"opencli browser get url"
],
"judge": {
"type": "contains",
@@ -92,10 +92,10 @@
{
"name": "v2ex-back-navigation",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate back",
"opencli operate get url"
"opencli browser open https://v2ex.com/",
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser back",
"opencli browser get url"
],
"judge": {
"type": "matchesPattern",
@@ -105,9 +105,9 @@
{
"name": "v2ex-wait-page-load",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate wait selector \"a[href^='/t/']\"",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length > 0 ? 'loaded' : 'empty'\""
"opencli browser open https://v2ex.com/",
"opencli browser wait selector \"a[href^='/t/']\"",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length > 0 ? 'loaded' : 'empty'\""
],
"judge": {
"type": "contains",
@@ -120,8 +120,8 @@
{
"name": "v2ex-hot-topics",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,10).map(a=>({title:a.textContent.trim(),url:a.href})).filter(t=>t.title.length>2))\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,10).map(a=>({title:a.textContent.trim(),url:a.href})).filter(t=>t.title.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -131,8 +131,8 @@
{
"name": "v2ex-node-list",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/go/\\\"]')].map(a=>a.textContent.trim()).filter(t=>t.length>0))\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/go/\\\"]')].map(a=>a.textContent.trim()).filter(t=>t.length>0))\""
],
"judge": {
"type": "arrayMinLength",
@@ -142,9 +142,9 @@
{
"name": "v2ex-topic-meta",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');const href=a?.href;return href||'';})()\"",
"opencli operate eval \"(()=>{const links=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')];const first=links[0];if(!first)return JSON.stringify({error:'no topic'});const title=first.textContent.trim();const row=first.closest('tr')||first.parentElement;const author=row?.querySelector('a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';return JSON.stringify({title,author});})()\" "
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');const href=a?.href;return href||'';})()\"",
"opencli browser eval \"(()=>{const links=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')];const first=links[0];if(!first)return JSON.stringify({error:'no topic'});const title=first.textContent.trim();const row=first.closest('tr')||first.parentElement;const author=row?.querySelector('a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';return JSON.stringify({title,author});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -154,8 +154,8 @@
{
"name": "v2ex-node-topics",
"steps": [
"opencli operate open https://v2ex.com/go/python",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/go/python",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -165,8 +165,8 @@
{
"name": "v2ex-node-pagination-info",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"(()=>{const pages=[...document.querySelectorAll('a[href*=\\\"?p=\\\"]')];if(pages.length===0)return'no pagination';const nums=pages.map(a=>{const m=a.href.match(/p=(\\d+)/);return m?parseInt(m[1]):0}).filter(n=>n>0);return JSON.stringify({pages:nums.length,max:Math.max(...nums)});})()\" "
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"(()=>{const pages=[...document.querySelectorAll('a[href*=\\\"?p=\\\"]')];if(pages.length===0)return'no pagination';const nums=pages.map(a=>{const m=a.href.match(/p=(\\d+)/);return m?parseInt(m[1]):0}).filter(n=>n>0);return JSON.stringify({pages:nums.length,max:Math.max(...nums)});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -176,8 +176,8 @@
{
"name": "v2ex-tab-content",
"steps": [
"opencli operate open https://v2ex.com/?tab=jobs",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/?tab=jobs",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -187,11 +187,11 @@
{
"name": "v2ex-topic-replies-extract",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"(()=>{const link=document.querySelector('a[href^=\\\"/t/\\\"]');if(!link)return'';return link.href;})()\"",
"opencli operate eval \"(()=>{const link=document.querySelector('a[href^=\\\"/t/\\\"]');if(link)window.location.href=link.href;return'navigating';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('.reply_content')].slice(0,5).map(el=>el.textContent.trim().slice(0,100)))\""
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"(()=>{const link=document.querySelector('a[href^=\\\"/t/\\\"]');if(!link)return'';return link.href;})()\"",
"opencli browser eval \"(()=>{const link=document.querySelector('a[href^=\\\"/t/\\\"]');if(link)window.location.href=link.href;return'navigating';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('.reply_content')].slice(0,5).map(el=>el.textContent.trim().slice(0,100)))\""
],
"judge": {
"type": "nonEmpty"
@@ -200,8 +200,8 @@
{
"name": "v2ex-topic-reply-count",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const counts=[...document.querySelectorAll('a[class*=\\\"count\\\"]')].map(a=>parseInt(a.textContent)).filter(n=>!isNaN(n));return JSON.stringify(counts.slice(0,10));})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const counts=[...document.querySelectorAll('a[class*=\\\"count\\\"]')].map(a=>parseInt(a.textContent)).filter(n=>!isNaN(n));return JSON.stringify(counts.slice(0,10));})()\" "
],
"judge": {
"type": "arrayMinLength",
@@ -211,8 +211,8 @@
{
"name": "v2ex-member-info",
"steps": [
"opencli operate open https://v2ex.com/member/Livid",
"opencli operate eval \"(()=>{const name=document.querySelector('h1')?.textContent?.trim();const bio=document.querySelector('.bigger')?.textContent?.trim()||'';return JSON.stringify({name,bio});})()\" "
"opencli browser open https://v2ex.com/member/Livid",
"opencli browser eval \"(()=>{const name=document.querySelector('h1')?.textContent?.trim();const bio=document.querySelector('.bigger')?.textContent?.trim()||'';return JSON.stringify({name,bio});})()\" "
],
"judge": {
"type": "contains",
@@ -222,8 +222,8 @@
{
"name": "v2ex-search-results",
"steps": [
"opencli operate open https://www.google.com/search?q=site:v2ex.com+TypeScript",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('h3')].slice(0,5).map(h=>h.textContent.trim()))\""
"opencli browser open https://www.google.com/search?q=site:v2ex.com+TypeScript",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('h3')].slice(0,5).map(h=>h.textContent.trim()))\""
],
"judge": {
"type": "arrayMinLength",
@@ -236,11 +236,11 @@
{
"name": "v2ex-click-topic-read",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate state",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked '+a.textContent.trim().slice(0,30);}return 'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,200)||document.title\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser state",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked '+a.textContent.trim().slice(0,30);}return 'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,200)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -249,10 +249,10 @@
{
"name": "v2ex-click-author-profile",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/member/\\\"]');if(a){const name=a.textContent.trim();a.click();return name;}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const h1=document.querySelector('h1');const joined=document.querySelector('.gray')?.textContent||'';return JSON.stringify({name:h1?.textContent?.trim(),info:joined.slice(0,100)});})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/member/\\\"]');if(a){const name=a.textContent.trim();a.click();return name;}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const h1=document.querySelector('h1');const joined=document.querySelector('.gray')?.textContent||'';return JSON.stringify({name:h1?.textContent?.trim(),info:joined.slice(0,100)});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -262,10 +262,10 @@
{
"name": "v2ex-navigate-node-from-home",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href=\\\"/go/programmer\\\"]')||document.querySelector('a[href^=\\\"/go/\\\"]');if(a){a.click();return 'clicked '+a.textContent.trim();}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href=\\\"/go/programmer\\\"]')||document.querySelector('a[href^=\\\"/go/\\\"]');if(a){a.click();return 'clicked '+a.textContent.trim();}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -275,10 +275,10 @@
{
"name": "v2ex-pagination-page2",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href*=\\\"?p=2\\\"]');if(a){a.click();return'navigating to page 2';}return'no page 2 link';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"JSON.stringify({url:location.href,topics:[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2)})\""
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href*=\\\"?p=2\\\"]');if(a){a.click();return'navigating to page 2';}return'no page 2 link';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"JSON.stringify({url:location.href,topics:[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,5).map(a=>a.textContent.trim()).filter(t=>t.length>2)})\""
],
"judge": {
"type": "matchesPattern",
@@ -288,12 +288,12 @@
{
"name": "v2ex-topic-and-back",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate back",
"opencli operate wait time 1",
"opencli operate get url"
"opencli browser open https://v2ex.com/",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser back",
"opencli browser wait time 1",
"opencli browser get url"
],
"judge": {
"type": "matchesPattern",
@@ -303,10 +303,10 @@
{
"name": "v2ex-tab-then-topic",
"steps": [
"opencli operate open https://v2ex.com/?tab=creative",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){const t=a.textContent.trim();a.click();return t;}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://v2ex.com/?tab=creative",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){const t=a.textContent.trim();a.click();return t;}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -315,11 +315,11 @@
{
"name": "v2ex-scroll-find-more",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli operate scroll down --amount 1000",
"opencli operate scroll down --amount 1000",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli browser scroll down --amount 1000",
"opencli browser scroll down --amount 1000",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
],
"judge": {
"type": "matchesPattern",
@@ -329,10 +329,10 @@
{
"name": "v2ex-node-to-topic-content",
"steps": [
"opencli operate open https://v2ex.com/go/python",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const content=document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,200)||'';return JSON.stringify({title,content});})()\" "
"opencli browser open https://v2ex.com/go/python",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const content=document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,200)||'';return JSON.stringify({title,content});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -342,10 +342,10 @@
{
"name": "v2ex-multi-tab-compare",
"steps": [
"opencli operate open https://v2ex.com/?tab=tech",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\"",
"opencli operate open https://v2ex.com/?tab=creative",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\""
"opencli browser open https://v2ex.com/?tab=tech",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\"",
"opencli browser open https://v2ex.com/?tab=creative",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\""
],
"judge": {
"type": "arrayMinLength",
@@ -355,12 +355,12 @@
{
"name": "v2ex-topic-reply-to-author",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const replies=document.querySelectorAll('.reply_content');const authors=[...document.querySelectorAll('a[href^=\\\"/member/\\\"]')];if(replies.length>0){const authorLink=document.querySelector('.cell a[href^=\\\"/member/\\\"]');if(authorLink){authorLink.click();return'clicked author';}};return'no replies found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('h1')?.textContent?.trim()||document.title\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const replies=document.querySelectorAll('.reply_content');const authors=[...document.querySelectorAll('a[href^=\\\"/member/\\\"]')];if(replies.length>0){const authorLink=document.querySelector('.cell a[href^=\\\"/member/\\\"]');if(authorLink){authorLink.click();return'clicked author';}};return'no replies found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('h1')?.textContent?.trim()||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -372,10 +372,10 @@
{
"name": "v2ex-reply-type-text",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const links=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')];const link=links.find(a=>a.closest('tr')?.querySelector('a[class*=\\\"count\\\"]'));if(link){link.click();return'clicked';}if(links[0]){links[0].click();return'clicked first';}return'no topic';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');if(ta){ta.focus();ta.value='AutoResearch test reply - please ignore';ta.dispatchEvent(new Event('input',{bubbles:true}));return ta.value;}return'no textarea';})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const links=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')];const link=links.find(a=>a.closest('tr')?.querySelector('a[class*=\\\"count\\\"]'));if(link){link.click();return'clicked';}if(links[0]){links[0].click();return'clicked first';}return'no topic';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');if(ta){ta.focus();ta.value='AutoResearch test reply - please ignore';ta.dispatchEvent(new Event('input',{bubbles:true}));return ta.value;}return'no textarea';})()\" "
],
"judge": {
"type": "contains",
@@ -386,10 +386,10 @@
{
"name": "v2ex-favorite-topic",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const favLink=[...document.querySelectorAll('a')].find(a=>a.textContent.includes('加入收藏')||a.textContent.includes('Favorite'));return favLink?favLink.href:'no fav link';})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const favLink=[...document.querySelectorAll('a')].find(a=>a.textContent.includes('加入收藏')||a.textContent.includes('Favorite'));return favLink?favLink.href:'no fav link';})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -400,10 +400,10 @@
{
"name": "v2ex-thank-reply-find",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const thankBtns=document.querySelectorAll('.thank_area,a[onclick*=\\\"thank\\\"],.thank');return JSON.stringify({found:thankBtns.length});})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const thankBtns=document.querySelectorAll('.thank_area,a[onclick*=\\\"thank\\\"],.thank');return JSON.stringify({found:thankBtns.length});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -414,10 +414,10 @@
{
"name": "v2ex-reply-form-detect",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');const btn=document.querySelector('input[type=\\\"submit\\\"],button[type=\\\"submit\\\"]');const once=document.querySelector('input[name=\\\"once\\\"]');return JSON.stringify({textarea:!!ta,submitBtn:!!btn,csrfToken:!!once});})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');const btn=document.querySelector('input[type=\\\"submit\\\"],button[type=\\\"submit\\\"]');const once=document.querySelector('input[name=\\\"once\\\"]');return JSON.stringify({textarea:!!ta,submitBtn:!!btn,csrfToken:!!once});})()\" "
],
"judge": {
"type": "contains",
@@ -427,9 +427,9 @@
{
"name": "v2ex-create-topic-form-detect",
"steps": [
"opencli operate open https://v2ex.com/new",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const title=document.querySelector('input[name=\\\"title\\\"],#topic_title');const content=document.querySelector('textarea[name=\\\"content\\\"],#topic_content,textarea#editor');const nodeSelect=document.querySelector('select[name=\\\"node_name\\\"],#node-select');return JSON.stringify({titleInput:!!title,contentArea:!!content,nodeSelect:!!nodeSelect,url:location.href});})()\" "
"opencli browser open https://v2ex.com/new",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const title=document.querySelector('input[name=\\\"title\\\"],#topic_title');const content=document.querySelector('textarea[name=\\\"content\\\"],#topic_content,textarea#editor');const nodeSelect=document.querySelector('select[name=\\\"node_name\\\"],#node-select');return JSON.stringify({titleInput:!!title,contentArea:!!content,nodeSelect:!!nodeSelect,url:location.href});})()\" "
],
"judge": {
"type": "nonEmpty"
@@ -442,8 +442,8 @@
{
"name": "v2ex-collect-hot-authors",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"JSON.stringify([...new Set([...document.querySelectorAll('a')].filter(a=>a.pathname&&a.pathname.startsWith('/member/')).map(a=>a.textContent.trim()).filter(n=>n.length>1))].slice(0,5))\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"JSON.stringify([...new Set([...document.querySelectorAll('a')].filter(a=>a.pathname&&a.pathname.startsWith('/member/')).map(a=>a.textContent.trim()).filter(n=>n.length>1))].slice(0,5))\""
],
"judge": {
"type": "arrayMinLength",
@@ -453,10 +453,10 @@
{
"name": "v2ex-multi-node-compare",
"steps": [
"opencli operate open https://v2ex.com/go/python",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\"",
"opencli operate open https://v2ex.com/go/go",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/go/python",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\"",
"opencli browser open https://v2ex.com/go/go",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -466,10 +466,10 @@
{
"name": "v2ex-topic-deep-read",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const author=document.querySelector('.header a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';const content=document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,300)||'';const replyCount=document.querySelectorAll('.reply_content').length;return JSON.stringify({title,author,content,replyCount});})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return'clicked';}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const author=document.querySelector('.header a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';const content=document.querySelector('.topic_content,.markdown_body')?.textContent?.trim()?.slice(0,300)||'';const replyCount=document.querySelectorAll('.reply_content').length;return JSON.stringify({title,author,content,replyCount});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -479,10 +479,10 @@
{
"name": "v2ex-cross-page-data-collect",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"(()=>{const titles=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim());window.__collected=titles;return JSON.stringify(titles);})()\"",
"opencli operate open https://v2ex.com/go/programmer?p=2",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"(()=>{const titles=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim());window.__collected=titles;return JSON.stringify(titles);})()\"",
"opencli browser open https://v2ex.com/go/programmer?p=2",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()).filter(t=>t.length>2))\""
],
"judge": {
"type": "arrayMinLength",
@@ -492,11 +492,11 @@
{
"name": "v2ex-full-workflow",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(()=>{const topics=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>({title:a.textContent.trim(),href:a.href}));return JSON.stringify(topics);})()\"",
"opencli operate eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked: '+a.textContent.trim().slice(0,30);}return'not found';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const replies=[...document.querySelectorAll('.reply_content')].slice(0,3).map(el=>el.textContent.trim().slice(0,80));const author=document.querySelector('.header a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';return JSON.stringify({title,author,replies,replyCount:replies.length});})()\" "
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(()=>{const topics=[...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>({title:a.textContent.trim(),href:a.href}));return JSON.stringify(topics);})()\"",
"opencli browser eval \"(()=>{const a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked: '+a.textContent.trim().slice(0,30);}return'not found';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(()=>{const title=document.querySelector('h1')?.textContent?.trim()||document.title;const replies=[...document.querySelectorAll('.reply_content')].slice(0,3).map(el=>el.textContent.trim().slice(0,80));const author=document.querySelector('.header a[href^=\\\"/member/\\\"]')?.textContent?.trim()||'';return JSON.stringify({title,author,replies,replyCount:replies.length});})()\" "
],
"judge": {
"type": "matchesPattern",
@@ -509,9 +509,9 @@
{
"name": "v2ex-state-click-topic",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate state",
"opencli operate click 1"
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser state",
"opencli browser click 1"
],
"judge": {
"type": "contains",
@@ -521,9 +521,9 @@
{
"name": "v2ex-state-click-tab-tech",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state",
"opencli operate eval \"(function(){var links=[...document.querySelectorAll('a')];var tab=links.find(a=>a.href&&a.href.includes('tab=tech'));if(tab){var ref=tab.getAttribute('data-opencli-ref');return ref||'no-ref';}return 'not-found';})()\""
"opencli browser open https://v2ex.com/",
"opencli browser state",
"opencli browser eval \"(function(){var links=[...document.querySelectorAll('a')];var tab=links.find(a=>a.href&&a.href.includes('tab=tech'));if(tab){var ref=tab.getAttribute('data-opencli-ref');return ref||'no-ref';}return 'not-found';})()\""
],
"judge": {
"type": "matchesPattern",
@@ -533,8 +533,8 @@
{
"name": "v2ex-state-count-interactive",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state"
"opencli browser open https://v2ex.com/",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
@@ -544,9 +544,9 @@
{
"name": "v2ex-state-scroll-state",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate scroll down --amount 500",
"opencli operate state"
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser scroll down --amount 500",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
@@ -556,9 +556,9 @@
{
"name": "v2ex-type-search-box",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state",
"opencli operate eval \"(function(){var input=document.querySelector('input[type=\\\"text\\\"]');if(input){input.focus();input.value='TypeScript';input.dispatchEvent(new Event('input',{bubbles:true}));return input.value;}return 'no-input';})()\""
"opencli browser open https://v2ex.com/",
"opencli browser state",
"opencli browser eval \"(function(){var input=document.querySelector('input[type=\\\"text\\\"]');if(input){input.focus();input.value='TypeScript';input.dispatchEvent(new Event('input',{bubbles:true}));return input.value;}return 'no-input';})()\""
],
"judge": {
"type": "contains",
@@ -568,10 +568,10 @@
{
"name": "v2ex-get-value-after-type",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a)a.click();return 'clicked';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');if(ta){ta.focus();ta.value='test message 12345';ta.dispatchEvent(new Event('input',{bubbles:true}));return ta.value;}return 'no-textarea';})()\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a)a.click();return 'clicked';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var ta=document.querySelector('textarea#reply_content,textarea[name=\\\"content\\\"]');if(ta){ta.focus();ta.value='test message 12345';ta.dispatchEvent(new Event('input',{bubbles:true}));return ta.value;}return 'no-textarea';})()\""
],
"judge": {
"type": "nonEmpty"
@@ -580,8 +580,8 @@
{
"name": "v2ex-screenshot-exists",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate screenshot /tmp/v2ex-test-screenshot.png"
"opencli browser open https://v2ex.com/",
"opencli browser screenshot /tmp/v2ex-test-screenshot.png"
],
"judge": {
"type": "nonEmpty"
@@ -590,8 +590,8 @@
{
"name": "v2ex-get-html-selector",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate get html --selector h1"
"opencli browser open https://v2ex.com/",
"opencli browser get html --selector h1"
],
"judge": {
"type": "nonEmpty"
@@ -600,8 +600,8 @@
{
"name": "v2ex-keys-escape",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate keys Escape"
"opencli browser open https://v2ex.com/",
"opencli browser keys Escape"
],
"judge": {
"type": "contains",
@@ -611,8 +611,8 @@
{
"name": "v2ex-wait-text",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate wait text V2EX"
"opencli browser open https://v2ex.com/",
"opencli browser wait text V2EX"
],
"judge": {
"type": "matchesPattern",
@@ -625,12 +625,12 @@
{
"name": "v2ex-chain-3-pages",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"document.title\"",
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"document.title\"",
"opencli operate open https://v2ex.com/go/python",
"opencli operate eval \"document.title\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"document.title\"",
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"document.title\"",
"opencli browser open https://v2ex.com/go/python",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -639,13 +639,13 @@
{
"name": "v2ex-chain-navigate-extract-back",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){var t=a.textContent.trim();a.click();return t;}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('h1')?.textContent?.trim()||document.title\"",
"opencli operate back",
"opencli operate wait time 1",
"opencli operate eval \"document.title\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){var t=a.textContent.trim();a.click();return t;}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('h1')?.textContent?.trim()||document.title\"",
"opencli browser back",
"opencli browser wait time 1",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "contains",
@@ -655,12 +655,12 @@
{
"name": "v2ex-chain-multi-node-scroll",
"steps": [
"opencli operate open https://v2ex.com/go/python",
"opencli operate scroll down --amount 500",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli operate open https://v2ex.com/go/go",
"opencli operate scroll down --amount 500",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
"opencli browser open https://v2ex.com/go/python",
"opencli browser scroll down --amount 500",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli browser open https://v2ex.com/go/go",
"opencli browser scroll down --amount 500",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
],
"judge": {
"type": "matchesPattern",
@@ -670,10 +670,10 @@
{
"name": "v2ex-chain-topic-replies-pagination",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(function(){var links=document.querySelectorAll('a[href^=\\\"/t/\\\"]');for(var i=0;i<links.length;i++){var row=links[i].closest('tr')||links[i].parentElement;var count=row?.querySelector('a[class*=\\\"count\\\"]');if(count&&parseInt(count.textContent)>5){links[i].click();return 'clicked topic with '+count.textContent+' replies';}}return 'no high-reply topic';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelectorAll('.reply_content').length\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(function(){var links=document.querySelectorAll('a[href^=\\\"/t/\\\"]');for(var i=0;i<links.length;i++){var row=links[i].closest('tr')||links[i].parentElement;var count=row?.querySelector('a[class*=\\\"count\\\"]');if(count&&parseInt(count.textContent)>5){links[i].click();return 'clicked topic with '+count.textContent+' replies';}}return 'no high-reply topic';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelectorAll('.reply_content').length\""
],
"judge": {
"type": "matchesPattern",
@@ -683,9 +683,9 @@
{
"name": "v2ex-chain-member-topics",
"steps": [
"opencli operate open https://v2ex.com/member/Livid",
"opencli operate eval \"document.querySelector('h1')?.textContent?.trim()\"",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\""
"opencli browser open https://v2ex.com/member/Livid",
"opencli browser eval \"document.querySelector('h1')?.textContent?.trim()\"",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,3).map(a=>a.textContent.trim()))\""
],
"judge": {
"type": "nonEmpty"
@@ -694,11 +694,11 @@
{
"name": "v2ex-chain-search-navigate-extract",
"steps": [
"opencli operate open https://www.google.com/search?q=site:v2ex.com+Python",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var links=[...document.querySelectorAll('a')];var v2exLink=links.find(a=>a.href&&a.href.includes('v2ex.com/t/'));if(v2exLink){v2exLink.click();return 'clicked';}return 'no v2ex link found';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.title\""
"opencli browser open https://www.google.com/search?q=site:v2ex.com+Python",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var links=[...document.querySelectorAll('a')];var v2exLink=links.find(a=>a.href&&a.href.includes('v2ex.com/t/'));if(v2exLink){v2exLink.click();return 'clicked';}return 'no v2ex link found';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -707,12 +707,12 @@
{
"name": "v2ex-chain-tab-topic-author",
"steps": [
"opencli operate open https://v2ex.com/?tab=tech",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var author=document.querySelector('.header a[href^=\\\"/member/\\\"]');if(author){var name=author.textContent.trim();author.click();return name;}return 'no author';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('h1')?.textContent?.trim()||'no h1'\""
"opencli browser open https://v2ex.com/?tab=tech",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var author=document.querySelector('.header a[href^=\\\"/member/\\\"]');if(author){var name=author.textContent.trim();author.click();return name;}return 'no author';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('h1')?.textContent?.trim()||'no h1'\""
],
"judge": {
"type": "nonEmpty"
@@ -721,12 +721,12 @@
{
"name": "v2ex-chain-node-page2-extract",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\"",
"opencli operate open https://v2ex.com/go/programmer?p=2",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\"",
"opencli operate open https://v2ex.com/go/programmer?p=3",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\""
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\"",
"opencli browser open https://v2ex.com/go/programmer?p=2",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\"",
"opencli browser open https://v2ex.com/go/programmer?p=3",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(0,2).map(a=>a.textContent.trim()))\""
],
"judge": {
"type": "arrayMinLength",
@@ -736,13 +736,13 @@
{
"name": "v2ex-chain-full-interaction",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate state",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate state",
"opencli operate eval \"(function(){var ta=document.querySelector('textarea#reply_content');if(ta)return 'reply form found';return 'no reply form';})()\"",
"opencli operate eval \"JSON.stringify({title:document.querySelector('h1')?.textContent?.trim()||document.title,replies:document.querySelectorAll('.reply_content').length})\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser state",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser state",
"opencli browser eval \"(function(){var ta=document.querySelector('textarea#reply_content');if(ta)return 'reply form found';return 'no reply form';})()\"",
"opencli browser eval \"JSON.stringify({title:document.querySelector('h1')?.textContent?.trim()||document.title,replies:document.querySelectorAll('.reply_content').length})\""
],
"judge": {
"type": "matchesPattern",
@@ -752,15 +752,15 @@
{
"name": "v2ex-chain-deep-5-step",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/go/\\\"]').length\"",
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelectorAll('.reply_content').length\"",
"opencli operate back",
"opencli operate eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/go/\\\"]').length\"",
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\"",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelectorAll('.reply_content').length\"",
"opencli browser back",
"opencli browser eval \"document.querySelectorAll('a[href^=\\\"/t/\\\"]').length\""
],
"judge": {
"type": "matchesPattern",
@@ -773,10 +773,10 @@
{
"name": "v2ex-rapid-navigate",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate open https://v2ex.com/go/python",
"opencli operate eval \"location.pathname\""
"opencli browser open https://v2ex.com/",
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser open https://v2ex.com/go/python",
"opencli browser eval \"location.pathname\""
],
"judge": {
"type": "contains",
@@ -786,10 +786,10 @@
{
"name": "v2ex-eval-after-click",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 1",
"opencli operate eval \"location.pathname.startsWith('/t/') ? 'on topic page' : 'wrong page: '+location.pathname\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 1",
"opencli browser eval \"location.pathname.startsWith('/t/') ? 'on topic page' : 'wrong page: '+location.pathname\""
],
"judge": {
"type": "contains",
@@ -799,10 +799,10 @@
{
"name": "v2ex-scroll-and-extract",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate scroll down --amount 2000",
"opencli operate wait time 1",
"opencli operate eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(-3).map(a=>a.textContent.trim().slice(0,30)))\""
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser scroll down --amount 2000",
"opencli browser wait time 1",
"opencli browser eval \"JSON.stringify([...document.querySelectorAll('a[href^=\\\"/t/\\\"]')].slice(-3).map(a=>a.textContent.trim().slice(0,30)))\""
],
"judge": {
"type": "arrayMinLength",
@@ -812,8 +812,8 @@
{
"name": "v2ex-concurrent-eval",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate eval \"JSON.stringify({title:document.title,url:location.href,links:document.querySelectorAll('a').length})\""
"opencli browser open https://v2ex.com/",
"opencli browser eval \"JSON.stringify({title:document.title,url:location.href,links:document.querySelectorAll('a').length})\""
],
"judge": {
"type": "matchesPattern",
@@ -823,8 +823,8 @@
{
"name": "v2ex-unicode-content",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');return a?a.textContent.trim():'none';})()\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser eval \"(function(){var a=document.querySelector('a[href^=\\\"/t/\\\"]');return a?a.textContent.trim():'none';})()\""
],
"judge": {
"type": "nonEmpty"
@@ -836,9 +836,9 @@
{
"name": "v2ex-agent-click-first-topic",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate state",
"opencli operate eval \"(function(){var links=document.querySelectorAll('[data-opencli-ref]');for(var i=0;i<links.length;i++){if(links[i].pathname&&links[i].pathname.startsWith('/t/'))return links[i].getAttribute('data-opencli-ref');}return 'none';})()\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser state",
"opencli browser eval \"(function(){var links=document.querySelectorAll('[data-opencli-ref]');for(var i=0;i<links.length;i++){if(links[i].pathname&&links[i].pathname.startsWith('/t/'))return links[i].getAttribute('data-opencli-ref');}return 'none';})()\""
],
"judge": {
"type": "matchesPattern",
@@ -849,10 +849,10 @@
{
"name": "v2ex-agent-type-search",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state",
"opencli operate type 3 TypeScript",
"opencli operate get value 3"
"opencli browser open https://v2ex.com/",
"opencli browser state",
"opencli browser type 3 TypeScript",
"opencli browser get value 3"
],
"judge": {
"type": "contains",
@@ -863,11 +863,11 @@
{
"name": "v2ex-agent-click-navigate-back",
"steps": [
"opencli operate open https://v2ex.com/?tab=hot",
"opencli operate state",
"opencli operate eval \"(function(){var links=document.querySelectorAll('[data-opencli-ref]');for(var i=0;i<links.length;i++){if(links[i].pathname&&links[i].pathname.startsWith('/t/')){var ref=links[i].getAttribute('data-opencli-ref');document.querySelector('[data-opencli-ref=\\\"'+ref+'\\\"]').click();return 'clicked '+ref;}}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://v2ex.com/?tab=hot",
"opencli browser state",
"opencli browser eval \"(function(){var links=document.querySelectorAll('[data-opencli-ref]');for(var i=0;i<links.length;i++){if(links[i].pathname&&links[i].pathname.startsWith('/t/')){var ref=links[i].getAttribute('data-opencli-ref');document.querySelector('[data-opencli-ref=\\\"'+ref+'\\\"]').click();return 'clicked '+ref;}}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -876,8 +876,8 @@
{
"name": "v2ex-agent-state-has-interactive",
"steps": [
"opencli operate open https://v2ex.com/",
"opencli operate state"
"opencli browser open https://v2ex.com/",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
@@ -887,9 +887,9 @@
{
"name": "v2ex-agent-state-after-scroll",
"steps": [
"opencli operate open https://v2ex.com/go/programmer",
"opencli operate scroll down --amount 800",
"opencli operate state"
"opencli browser open https://v2ex.com/go/programmer",
"opencli browser scroll down --amount 800",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
+247 -247
View File
@@ -5,7 +5,7 @@
{
"name": "zhihu-open-home",
"steps": [
"opencli operate open https://www.zhihu.com/"
"opencli browser open https://www.zhihu.com/"
],
"judge": {
"type": "contains",
@@ -15,8 +15,8 @@
{
"name": "zhihu-get-title",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"document.title\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "contains",
@@ -26,8 +26,8 @@
{
"name": "zhihu-state",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate state"
"opencli browser open https://www.zhihu.com/",
"opencli browser state"
],
"judge": {
"type": "matchesPattern",
@@ -37,8 +37,8 @@
{
"name": "zhihu-get-url",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate get url"
"opencli browser open https://www.zhihu.com/hot",
"opencli browser get url"
],
"judge": {
"type": "contains",
@@ -48,9 +48,9 @@
{
"name": "zhihu-scroll-down",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate scroll down --amount 500",
"opencli operate eval \"window.scrollY > 100 ? 'scrolled' : 'not scrolled'\""
"opencli browser open https://www.zhihu.com/",
"opencli browser scroll down --amount 500",
"opencli browser eval \"window.scrollY > 100 ? 'scrolled' : 'not scrolled'\""
],
"judge": {
"type": "contains",
@@ -60,8 +60,8 @@
{
"name": "zhihu-click-tab-hot",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var a=document.querySelector('nav a[href*=hot]');if(a){a.click();return 'clicked';}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var a=document.querySelector('nav a[href*=hot]');if(a){a.click();return 'clicked';}return 'not found';})()\""
],
"judge": {
"type": "contains",
@@ -71,10 +71,10 @@
{
"name": "zhihu-back-navigation",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate open https://www.zhihu.com/hot",
"opencli operate back",
"opencli operate get url"
"opencli browser open https://www.zhihu.com/",
"opencli browser open https://www.zhihu.com/hot",
"opencli browser back",
"opencli browser get url"
],
"judge": {
"type": "matchesPattern",
@@ -84,9 +84,9 @@
{
"name": "zhihu-wait-page-load",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate wait text 推荐",
"opencli operate eval \"document.querySelector('nav')?.textContent?.includes('推荐') ? 'loaded' : 'empty'\""
"opencli browser open https://www.zhihu.com/",
"opencli browser wait text 推荐",
"opencli browser eval \"document.querySelector('nav')?.textContent?.includes('推荐') ? 'loaded' : 'empty'\""
],
"judge": {
"type": "contains",
@@ -96,8 +96,8 @@
{
"name": "zhihu-keys-escape",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate keys Escape"
"opencli browser open https://www.zhihu.com/",
"opencli browser keys Escape"
],
"judge": {
"type": "matchesPattern",
@@ -107,8 +107,8 @@
{
"name": "zhihu-screenshot",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate screenshot /tmp/zhihu-test.png"
"opencli browser open https://www.zhihu.com/",
"opencli browser screenshot /tmp/zhihu-test.png"
],
"judge": {
"type": "nonEmpty"
@@ -120,8 +120,8 @@
{
"name": "zhihu-feed-titles",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var items=document.querySelectorAll('h2.ContentItem-title a, .ContentItem-title a');var r=[];for(var i=0;i<Math.min(items.length,5);i++){r.push(items[i].textContent.trim().slice(0,60));}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var items=document.querySelectorAll('h2.ContentItem-title a, .ContentItem-title a');var r=[];for(var i=0;i<Math.min(items.length,5);i++){r.push(items[i].textContent.trim().slice(0,60));}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -131,8 +131,8 @@
{
"name": "zhihu-hot-list",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,5);i++){r.push({title:items[i].textContent.trim().slice(0,50),href:items[i].pathname});}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,5);i++){r.push({title:items[i].textContent.trim().slice(0,50),href:items[i].pathname});}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -142,8 +142,8 @@
{
"name": "zhihu-hot-metrics",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var metrics=document.querySelectorAll('.HotItem-metrics');var r=[];for(var i=0;i<Math.min(metrics.length,5);i++){r.push(metrics[i].textContent.trim());}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var metrics=document.querySelectorAll('.HotItem-metrics');var r=[];for(var i=0;i<Math.min(metrics.length,5);i++){r.push(metrics[i].textContent.trim());}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -153,8 +153,8 @@
{
"name": "zhihu-nav-tabs",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var tabs=document.querySelectorAll('nav a');var r=[];for(var i=0;i<tabs.length;i++){r.push(tabs[i].textContent.trim());}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var tabs=document.querySelectorAll('nav a');var r=[];for(var i=0;i<tabs.length;i++){r.push(tabs[i].textContent.trim());}return JSON.stringify(r);})()\""
],
"judge": {
"type": "contains",
@@ -164,8 +164,8 @@
{
"name": "zhihu-feed-with-authors",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.ContentItem');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var title=items[i].querySelector('h2 a')?.textContent?.trim()||'';var author=items[i].querySelector('.AuthorInfo-name')?.textContent?.trim()||'';if(title)r.push({title:title.slice(0,40),author:author});}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.ContentItem');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var title=items[i].querySelector('h2 a')?.textContent?.trim()||'';var author=items[i].querySelector('.AuthorInfo-name')?.textContent?.trim()||'';if(title)r.push({title:title.slice(0,40),author:author});}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -175,8 +175,8 @@
{
"name": "zhihu-feed-types",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var links=document.querySelectorAll('h2.ContentItem-title a, .ContentItem-title a');var types={question:0,article:0,other:0};for(var i=0;i<links.length;i++){var h=links[i].pathname||'';if(h.includes('/question/'))types.question++;else if(h.includes('/p/'))types.article++;else types.other++;}return JSON.stringify(types);})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var links=document.querySelectorAll('h2.ContentItem-title a, .ContentItem-title a');var types={question:0,article:0,other:0};for(var i=0;i<links.length;i++){var h=links[i].pathname||'';if(h.includes('/question/'))types.question++;else if(h.includes('/p/'))types.article++;else types.other++;}return JSON.stringify(types);})()\""
],
"judge": {
"type": "matchesPattern",
@@ -186,8 +186,8 @@
{
"name": "zhihu-user-avatar",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var img=document.querySelector('img[alt*=\\\"头像\\\"],img[alt*=\\\"主页\\\"],img[class*=\\\"Avatar\\\"]');return img?img.src:'no avatar';})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var img=document.querySelector('img[alt*=\\\"头像\\\"],img[alt*=\\\"主页\\\"],img[class*=\\\"Avatar\\\"]');return img?img.src:'no avatar';})()\""
],
"judge": {
"type": "nonEmpty"
@@ -196,8 +196,8 @@
{
"name": "zhihu-search-input-exists",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var input=document.querySelector('input[role=combobox],input[type=search]');return input?'search found':'no search';})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var input=document.querySelector('input[role=combobox],input[type=search]');return input?'search found':'no search';})()\""
],
"judge": {
"type": "contains",
@@ -210,11 +210,11 @@
{
"name": "zhihu-question-title",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');return a?a.href:'none';})()\"",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');return a?a.href:'none';})()\"",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -223,10 +223,10 @@
{
"name": "zhihu-question-meta",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||'';var answerCount=document.querySelector('.List-headerText')?.textContent?.trim()||'';var followers=document.querySelector('[class*=FollowButton]')?.textContent?.trim()||'';return JSON.stringify({title:title.slice(0,60),answerCount:answerCount,followers:followers});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||'';var answerCount=document.querySelector('.List-headerText')?.textContent?.trim()||'';var followers=document.querySelector('[class*=FollowButton]')?.textContent?.trim()||'';return JSON.stringify({title:title.slice(0,60),answerCount:answerCount,followers:followers});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -236,10 +236,10 @@
{
"name": "zhihu-first-answer",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var content=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,200)||'';var author=document.querySelector('.AuthorInfo-name')?.textContent?.trim()||'';return JSON.stringify({author:author,content:content});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var content=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,200)||'';var author=document.querySelector('.AuthorInfo-name')?.textContent?.trim()||'';return JSON.stringify({author:author,content:content});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -249,10 +249,10 @@
{
"name": "zhihu-answer-votes",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button[class*=VoteButton]');var r=[];for(var i=0;i<Math.min(btns.length,6);i++){var label=btns[i].getAttribute('aria-label')||btns[i].textContent.trim();if(label)r.push(label.slice(0,30));}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button[class*=VoteButton]');var r=[];for(var i=0;i<Math.min(btns.length,6);i++){var label=btns[i].getAttribute('aria-label')||btns[i].textContent.trim();if(label)r.push(label.slice(0,30));}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -262,10 +262,10 @@
{
"name": "zhihu-question-buttons",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');var r=[];for(var i=0;i<btns.length;i++){var t=btns[i].textContent.trim();if(t.length>0&&t.length<25)r.push(t);}return JSON.stringify(r.slice(0,15));})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');var r=[];for(var i=0;i<btns.length;i++){var t=btns[i].textContent.trim();if(t.length>0&&t.length<25)r.push(t);}return JSON.stringify(r.slice(0,15));})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -275,10 +275,10 @@
{
"name": "zhihu-multiple-answers",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var answers=document.querySelectorAll('.List-item .RichContent-inner');var r=[];for(var i=0;i<Math.min(answers.length,3);i++){r.push(answers[i].textContent.trim().slice(0,80));}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var answers=document.querySelectorAll('.List-item .RichContent-inner');var r=[];for(var i=0;i<Math.min(answers.length,3);i++){r.push(answers[i].textContent.trim().slice(0,80));}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -288,10 +288,10 @@
{
"name": "zhihu-question-description",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var desc=document.querySelector('.QuestionRichText')?.textContent?.trim()?.slice(0,200)||'no description';return desc;})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var desc=document.querySelector('.QuestionRichText')?.textContent?.trim()?.slice(0,200)||'no description';return desc;})()\""
],
"judge": {
"type": "nonEmpty"
@@ -300,10 +300,10 @@
{
"name": "zhihu-answer-count-number",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var header=document.querySelector('.List-headerText');if(!header)return '0';var m=header.textContent.match(/\\\\d+/);return m?m[0]:'0';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var header=document.querySelector('.List-headerText');if(!header)return '0';var m=header.textContent.match(/\\\\d+/);return m?m[0]:'0';})()\""
],
"judge": {
"type": "matchesPattern",
@@ -316,10 +316,10 @@
{
"name": "zhihu-hot-to-question",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked: '+a.textContent.trim().slice(0,30);}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked: '+a.textContent.trim().slice(0,30);}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -328,10 +328,10 @@
{
"name": "zhihu-feed-to-question",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var a=document.querySelector('h2.ContentItem-title a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var a=document.querySelector('h2.ContentItem-title a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,80)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -340,12 +340,12 @@
{
"name": "zhihu-question-to-author",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var author=document.querySelector('.AuthorInfo-name a, .UserLink-link');if(author){var name=author.textContent.trim();window.location.href=author.href;return name;}return 'no author';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var author=document.querySelector('.AuthorInfo-name a, .UserLink-link');if(author){var name=author.textContent.trim();window.location.href=author.href;return name;}return 'no author';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -354,11 +354,11 @@
{
"name": "zhihu-search-navigate",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=TypeScript",
"opencli operate wait time 5",
"opencli operate scroll down --amount 300",
"opencli operate wait time 1",
"opencli operate eval \"(function(){var items=document.querySelectorAll('h2 a');var r=[];var seen={};for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>5&&t.length<100&&!seen[t]){seen[t]=1;r.push(t.slice(0,50));}}return JSON.stringify(r.slice(0,5));})()\""
"opencli browser open https://www.zhihu.com/search?type=content&q=TypeScript",
"opencli browser wait time 5",
"opencli browser scroll down --amount 300",
"opencli browser wait time 1",
"opencli browser eval \"(function(){var items=document.querySelectorAll('h2 a');var r=[];var seen={};for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>5&&t.length<100&&!seen[t]){seen[t]=1;r.push(t.slice(0,50));}}return JSON.stringify(r.slice(0,5));})()\""
],
"judge": {
"type": "nonEmpty"
@@ -367,9 +367,9 @@
{
"name": "zhihu-topic-page",
"steps": [
"opencli operate open https://www.zhihu.com/topic/19552832/hot",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var title=document.querySelector('.TopicName, .ContentItem-title, h1')?.textContent?.trim()||document.title;return title;})()\""
"opencli browser open https://www.zhihu.com/topic/19552832/hot",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var title=document.querySelector('.TopicName, .ContentItem-title, h1')?.textContent?.trim()||document.title;return title;})()\""
],
"judge": {
"type": "nonEmpty"
@@ -378,9 +378,9 @@
{
"name": "zhihu-user-profile",
"steps": [
"opencli operate open https://www.zhihu.com/people/excited-vczh",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var name=document.querySelector('.ProfileHeader-title .ProfileHeader-name')?.textContent?.trim()||document.querySelector('h1')?.textContent?.trim()||'';var bio=document.querySelector('.ProfileHeader-headline')?.textContent?.trim()||'';return JSON.stringify({name:name,bio:bio.slice(0,100)});})()\""
"opencli browser open https://www.zhihu.com/people/excited-vczh",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var name=document.querySelector('.ProfileHeader-title .ProfileHeader-name')?.textContent?.trim()||document.querySelector('h1')?.textContent?.trim()||'';var bio=document.querySelector('.ProfileHeader-headline')?.textContent?.trim()||'';return JSON.stringify({name:name,bio:bio.slice(0,100)});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -390,11 +390,11 @@
{
"name": "zhihu-question-and-back",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate open https://www.zhihu.com/hot",
"opencli operate get url"
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser open https://www.zhihu.com/hot",
"opencli browser get url"
],
"judge": {
"type": "contains",
@@ -404,11 +404,11 @@
{
"name": "zhihu-scroll-load-more",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"document.querySelectorAll('.HotItem-content').length\"",
"opencli operate scroll down --amount 2000",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelectorAll('.HotItem-content').length\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"document.querySelectorAll('.HotItem-content').length\"",
"opencli browser scroll down --amount 2000",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelectorAll('.HotItem-content').length\""
],
"judge": {
"type": "matchesPattern",
@@ -421,10 +421,10 @@
{
"name": "zhihu-upvote-button-find",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btn=document.querySelector('button[aria-label*=赞同]');return btn?JSON.stringify({text:btn.textContent.trim(),ariaLabel:btn.getAttribute('aria-label')}):'no upvote button';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btn=document.querySelector('button[aria-label*=赞同]');return btn?JSON.stringify({text:btn.textContent.trim(),ariaLabel:btn.getAttribute('aria-label')}):'no upvote button';})()\""
],
"judge": {
"type": "contains",
@@ -435,10 +435,10 @@
{
"name": "zhihu-follow-question-find",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('关注问题'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('关注问题'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
],
"judge": {
"type": "contains",
@@ -449,10 +449,10 @@
{
"name": "zhihu-comment-button-find",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('评论'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('评论'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
],
"judge": {
"type": "matchesPattern",
@@ -463,10 +463,10 @@
{
"name": "zhihu-bookmark-find",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){var t=btns[i].textContent.trim();if(t.includes('收藏')||t.includes('Bookmark'))return 'found: '+t;}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){var t=btns[i].textContent.trim();if(t.includes('收藏')||t.includes('Bookmark'))return 'found: '+t;}return 'not found';})()\""
],
"judge": {
"type": "matchesPattern",
@@ -477,10 +477,10 @@
{
"name": "zhihu-write-answer-btn",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('写回答'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('写回答'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
],
"judge": {
"type": "contains",
@@ -491,10 +491,10 @@
{
"name": "zhihu-share-find",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('分享'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var btns=document.querySelectorAll('button');for(var i=0;i<btns.length;i++){if(btns[i].textContent.includes('分享'))return 'found: '+btns[i].textContent.trim();}return 'not found';})()\""
],
"judge": {
"type": "contains",
@@ -508,10 +508,10 @@
{
"name": "zhihu-hot-read-answer-author",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var author=document.querySelector('.AuthorInfo-name')?.textContent?.trim()||'';var content=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,100)||'';return JSON.stringify({author:author,content:content});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var author=document.querySelector('.AuthorInfo-name')?.textContent?.trim()||'';var content=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,100)||'';return JSON.stringify({author:author,content:content});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -521,12 +521,12 @@
{
"name": "zhihu-hot-to-author-profile",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var link=document.querySelector('.AuthorInfo-name a, .UserLink-link');if(link){window.location.href=link.href;return 'going to author';}return 'no author link';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var name=document.querySelector('h1, .ProfileHeader-name')?.textContent?.trim()||document.title;return name;})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var link=document.querySelector('.AuthorInfo-name a, .UserLink-link');if(link){window.location.href=link.href;return 'going to author';}return 'no author link';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var name=document.querySelector('h1, .ProfileHeader-name')?.textContent?.trim()||document.title;return name;})()\""
],
"judge": {
"type": "nonEmpty"
@@ -535,8 +535,8 @@
{
"name": "zhihu-multi-hot-topics",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,40));}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,40));}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -546,11 +546,11 @@
{
"name": "zhihu-search-then-read",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=Python",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var a=document.querySelector('.ContentItem-title a, .SearchResult-Card h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('.QuestionHeader-title, h1')?.textContent?.trim()?.slice(0,80)||document.title\""
"opencli browser open https://www.zhihu.com/search?type=content&q=Python",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var a=document.querySelector('.ContentItem-title a, .SearchResult-Card h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('.QuestionHeader-title, h1')?.textContent?.trim()?.slice(0,80)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -559,12 +559,12 @@
{
"name": "zhihu-question-scroll-answers",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate scroll down --amount 1000",
"opencli operate wait time 1",
"opencli operate eval \"document.querySelectorAll('.RichContent-inner').length\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser scroll down --amount 1000",
"opencli browser wait time 1",
"opencli browser eval \"document.querySelectorAll('.RichContent-inner').length\""
],
"judge": {
"type": "matchesPattern",
@@ -574,10 +574,10 @@
{
"name": "zhihu-compare-tabs",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"(function(){var a=document.querySelector('h2.ContentItem-title a');return a?a.textContent.trim().slice(0,40):'none';})()\"",
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');return a?a.textContent.trim().slice(0,40):'none';})()\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"(function(){var a=document.querySelector('h2.ContentItem-title a');return a?a.textContent.trim().slice(0,40):'none';})()\"",
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');return a?a.textContent.trim().slice(0,40):'none';})()\""
],
"judge": {
"type": "nonEmpty"
@@ -586,9 +586,9 @@
{
"name": "zhihu-user-answers",
"steps": [
"opencli operate open https://www.zhihu.com/people/excited-vczh/answers",
"opencli operate wait time 4",
"opencli operate eval \"(function(){var items=document.querySelectorAll('h2 a, [class*=title] a, [class*=Title] a');var r=[];for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>10&&t.length<100)r.push(t.slice(0,50));}return JSON.stringify(r.slice(0,3));})()\""
"opencli browser open https://www.zhihu.com/people/excited-vczh/answers",
"opencli browser wait time 4",
"opencli browser eval \"(function(){var items=document.querySelectorAll('h2 a, [class*=title] a, [class*=Title] a');var r=[];for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>10&&t.length<100)r.push(t.slice(0,50));}return JSON.stringify(r.slice(0,3));})()\""
],
"judge": {
"type": "nonEmpty"
@@ -597,9 +597,9 @@
{
"name": "zhihu-topic-questions",
"steps": [
"opencli operate open https://www.zhihu.com/topic/19552832/hot",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.ContentItem-title a, h2 a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var t=items[i].textContent.trim();if(t.length>5)r.push(t.slice(0,50));}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/topic/19552832/hot",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.ContentItem-title a, h2 a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var t=items[i].textContent.trim();if(t.length>5)r.push(t.slice(0,50));}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -612,11 +612,11 @@
{
"name": "zhihu-search-basic",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=AI",
"opencli operate wait time 5",
"opencli operate scroll down --amount 300",
"opencli operate wait time 1",
"opencli operate eval \"(function(){var items=document.querySelectorAll('h2 a');var r=[];var seen={};for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>5&&t.length<100&&!seen[t]){seen[t]=1;r.push(t.slice(0,50));}}return JSON.stringify(r.slice(0,5));})()\""
"opencli browser open https://www.zhihu.com/search?type=content&q=AI",
"opencli browser wait time 5",
"opencli browser scroll down --amount 300",
"opencli browser wait time 1",
"opencli browser eval \"(function(){var items=document.querySelectorAll('h2 a');var r=[];var seen={};for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>5&&t.length<100&&!seen[t]){seen[t]=1;r.push(t.slice(0,50));}}return JSON.stringify(r.slice(0,5));})()\""
],
"judge": {
"type": "nonEmpty"
@@ -625,9 +625,9 @@
{
"name": "zhihu-search-people",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=people&q=Python",
"opencli operate wait time 3",
"opencli operate eval \"(function(){var items=document.querySelectorAll('[class*=UserItem] a, [class*=user] a, .List-item a');var r=[];for(var i=0;i<Math.min(items.length,10);i++){var t=items[i].textContent.trim();if(t.length>1&&t.length<30)r.push(t);}return JSON.stringify(r.slice(0,3));})()\""
"opencli browser open https://www.zhihu.com/search?type=people&q=Python",
"opencli browser wait time 3",
"opencli browser eval \"(function(){var items=document.querySelectorAll('[class*=UserItem] a, [class*=user] a, .List-item a');var r=[];for(var i=0;i<Math.min(items.length,10);i++){var t=items[i].textContent.trim();if(t.length>1&&t.length<30)r.push(t);}return JSON.stringify(r.slice(0,3));})()\""
],
"judge": {
"type": "nonEmpty"
@@ -636,9 +636,9 @@
{
"name": "zhihu-search-topic",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=topic&q=编程",
"opencli operate wait time 3",
"opencli operate eval \"(function(){var items=document.querySelectorAll('a');var r=[];for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>2&&t.length<30&&(t.includes('编程')||items[i].pathname?.includes('/topic/')))r.push(t);}return JSON.stringify(r.slice(0,3));})()\""
"opencli browser open https://www.zhihu.com/search?type=topic&q=编程",
"opencli browser wait time 3",
"opencli browser eval \"(function(){var items=document.querySelectorAll('a');var r=[];for(var i=0;i<items.length;i++){var t=items[i].textContent.trim();if(t.length>2&&t.length<30&&(t.includes('编程')||items[i].pathname?.includes('/topic/')))r.push(t);}return JSON.stringify(r.slice(0,3));})()\""
],
"judge": {
"type": "nonEmpty"
@@ -647,11 +647,11 @@
{
"name": "zhihu-search-click-result",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=Rust编程",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var a=document.querySelector('.ContentItem-title a, h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.title\""
"opencli browser open https://www.zhihu.com/search?type=content&q=Rust编程",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var a=document.querySelector('.ContentItem-title a, h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -660,9 +660,9 @@
{
"name": "zhihu-search-filter-answers",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=Docker",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.ContentItem');return JSON.stringify({total:items.length,hasAnswers:items.length>0});})()\""
"opencli browser open https://www.zhihu.com/search?type=content&q=Docker",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.ContentItem');return JSON.stringify({total:items.length,hasAnswers:items.length>0});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -672,13 +672,13 @@
{
"name": "zhihu-search-and-back",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=前端开发",
"opencli operate wait time 5",
"opencli operate eval \"(function(){var a=document.querySelector('h2 a, [class*=title] a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate open https://www.zhihu.com/search?type=content&q=前端开发",
"opencli operate wait time 3",
"opencli operate get url"
"opencli browser open https://www.zhihu.com/search?type=content&q=前端开发",
"opencli browser wait time 5",
"opencli browser eval \"(function(){var a=document.querySelector('h2 a, [class*=title] a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser open https://www.zhihu.com/search?type=content&q=前端开发",
"opencli browser wait time 3",
"opencli browser get url"
],
"judge": {
"type": "contains",
@@ -691,11 +691,11 @@
{
"name": "zhihu-full-browse-workflow",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,30));}return JSON.stringify(r);})()\"",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,60)||'';var answers=document.querySelectorAll('.RichContent-inner').length;return JSON.stringify({title:title,answers:answers});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,30));}return JSON.stringify(r);})()\"",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,60)||'';var answers=document.querySelectorAll('.RichContent-inner').length;return JSON.stringify({title:title,answers:answers});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -705,12 +705,12 @@
{
"name": "zhihu-deep-author-chain",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'step1';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var link=document.querySelector('.AuthorInfo-name a');if(link){var name=link.textContent.trim();window.location.href=link.href;return 'step2: '+name;}return 'no author';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var answers=document.querySelectorAll('.ContentItem-title a');var r=[];for(var i=0;i<Math.min(answers.length,2);i++){r.push(answers[i].textContent.trim().slice(0,40));}return JSON.stringify({profile:document.title,recentAnswers:r});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'step1';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var link=document.querySelector('.AuthorInfo-name a');if(link){var name=link.textContent.trim();window.location.href=link.href;return 'step2: '+name;}return 'no author';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var answers=document.querySelectorAll('.ContentItem-title a');var r=[];for(var i=0;i<Math.min(answers.length,2);i++){r.push(answers[i].textContent.trim().slice(0,40));}return JSON.stringify({profile:document.title,recentAnswers:r});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -720,11 +720,11 @@
{
"name": "zhihu-cross-question-compare",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');return items.length>=2?JSON.stringify([items[0].textContent.trim().slice(0,30),items[1].textContent.trim().slice(0,30)]):'not enough';})()\"",
"opencli operate eval \"(function(){var a=document.querySelectorAll('.HotItem-content a')[0];if(a){window.location.href=a.href;return 'q1';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){return JSON.stringify({q1_title:document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,40)||'',q1_answers:document.querySelectorAll('.RichContent-inner').length});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');return items.length>=2?JSON.stringify([items[0].textContent.trim().slice(0,30),items[1].textContent.trim().slice(0,30)]):'not enough';})()\"",
"opencli browser eval \"(function(){var a=document.querySelectorAll('.HotItem-content a')[0];if(a){window.location.href=a.href;return 'q1';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){return JSON.stringify({q1_title:document.querySelector('.QuestionHeader-title')?.textContent?.trim()?.slice(0,40)||'',q1_answers:document.querySelectorAll('.RichContent-inner').length});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -734,12 +734,12 @@
{
"name": "zhihu-search-read-chain",
"steps": [
"opencli operate open https://www.zhihu.com/search?type=content&q=Claude",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.ContentItem-title a, h2 a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,40));}return JSON.stringify(r);})()\"",
"opencli operate eval \"(function(){var a=document.querySelector('.ContentItem-title a, h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelector('.QuestionHeader-title, h1')?.textContent?.trim()?.slice(0,60)||document.title\""
"opencli browser open https://www.zhihu.com/search?type=content&q=Claude",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.ContentItem-title a, h2 a');var r=[];for(var i=0;i<Math.min(items.length,3);i++){r.push(items[i].textContent.trim().slice(0,40));}return JSON.stringify(r);})()\"",
"opencli browser eval \"(function(){var a=document.querySelector('.ContentItem-title a, h2 a');if(a){a.click();return 'clicked';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelector('.QuestionHeader-title, h1')?.textContent?.trim()?.slice(0,60)||document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -748,13 +748,13 @@
{
"name": "zhihu-3-page-chain",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate eval \"document.title\"",
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"document.querySelectorAll('.HotItem-content').length\"",
"opencli operate open https://www.zhihu.com/people/excited-vczh",
"opencli operate wait time 2",
"opencli operate eval \"document.title\""
"opencli browser open https://www.zhihu.com/",
"opencli browser eval \"document.title\"",
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"document.querySelectorAll('.HotItem-content').length\"",
"opencli browser open https://www.zhihu.com/people/excited-vczh",
"opencli browser wait time 2",
"opencli browser eval \"document.title\""
],
"judge": {
"type": "nonEmpty"
@@ -763,12 +763,12 @@
{
"name": "zhihu-hot-scroll-deep-read",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate scroll down --amount 1000",
"opencli operate eval \"document.querySelectorAll('.HotItem-content a').length\"",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var last=items[items.length-1];if(last){last.click();return 'clicked last';}return 'none';})()\"",
"opencli operate wait time 2",
"opencli operate eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||document.title;var firstAnswer=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,100)||'';return JSON.stringify({title:title.slice(0,60),firstAnswer:firstAnswer});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser scroll down --amount 1000",
"opencli browser eval \"document.querySelectorAll('.HotItem-content a').length\"",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content a');var last=items[items.length-1];if(last){last.click();return 'clicked last';}return 'none';})()\"",
"opencli browser wait time 2",
"opencli browser eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||document.title;var firstAnswer=document.querySelector('.RichContent-inner')?.textContent?.trim()?.slice(0,100)||'';return JSON.stringify({title:title.slice(0,60),firstAnswer:firstAnswer});})()\""
],
"judge": {
"type": "matchesPattern",
@@ -781,11 +781,11 @@
{
"name": "zhihu-rapid-navigate",
"steps": [
"opencli operate open https://www.zhihu.com/",
"opencli operate open https://www.zhihu.com/hot",
"opencli operate open https://www.zhihu.com/people/excited-vczh",
"opencli operate wait time 2",
"opencli operate eval \"location.pathname\""
"opencli browser open https://www.zhihu.com/",
"opencli browser open https://www.zhihu.com/hot",
"opencli browser open https://www.zhihu.com/people/excited-vczh",
"opencli browser wait time 2",
"opencli browser eval \"location.pathname\""
],
"judge": {
"type": "contains",
@@ -795,10 +795,10 @@
{
"name": "zhihu-hot-click-verify-url",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"location.pathname.startsWith('/question/') ? 'on question page' : 'wrong: '+location.pathname\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"location.pathname.startsWith('/question/') ? 'on question page' : 'wrong: '+location.pathname\""
],
"judge": {
"type": "contains",
@@ -808,13 +808,13 @@
{
"name": "zhihu-scroll-lazy-answers",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"document.querySelectorAll('.RichContent-inner').length\"",
"opencli operate scroll down --amount 2000",
"opencli operate wait time 2",
"opencli operate eval \"document.querySelectorAll('.RichContent-inner').length\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"document.querySelectorAll('.RichContent-inner').length\"",
"opencli browser scroll down --amount 2000",
"opencli browser wait time 2",
"opencli browser eval \"document.querySelectorAll('.RichContent-inner').length\""
],
"judge": {
"type": "matchesPattern",
@@ -824,8 +824,8 @@
{
"name": "zhihu-extract-structured",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var items=document.querySelectorAll('.HotItem-content');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var a=items[i].querySelector('a');var m=items[i].closest('[class*=HotItem]')?.querySelector('[class*=metrics]');r.push({title:(a?.textContent||'').trim().slice(0,40),heat:(m?.textContent||'').trim()});}return JSON.stringify(r);})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var items=document.querySelectorAll('.HotItem-content');var r=[];for(var i=0;i<Math.min(items.length,3);i++){var a=items[i].querySelector('a');var m=items[i].closest('[class*=HotItem]')?.querySelector('[class*=metrics]');r.push({title:(a?.textContent||'').trim().slice(0,40),heat:(m?.textContent||'').trim()});}return JSON.stringify(r);})()\""
],
"judge": {
"type": "arrayMinLength",
@@ -835,10 +835,10 @@
{
"name": "zhihu-question-answer-chain",
"steps": [
"opencli operate open https://www.zhihu.com/hot",
"opencli operate eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli operate wait time 3",
"opencli operate eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||'';var answers=document.querySelectorAll('.RichContent-inner');var first=answers[0]?.textContent?.trim()?.slice(0,100)||'';var count=answers.length;return JSON.stringify({title:title.slice(0,50),firstAnswer:first,answerCount:count});})()\""
"opencli browser open https://www.zhihu.com/hot",
"opencli browser eval \"(function(){var a=document.querySelector('.HotItem-content a');if(a){window.location.href=a.href;return 'navigating';}return 'none';})()\"",
"opencli browser wait time 3",
"opencli browser eval \"(function(){var title=document.querySelector('.QuestionHeader-title')?.textContent?.trim()||'';var answers=document.querySelectorAll('.RichContent-inner');var first=answers[0]?.textContent?.trim()?.slice(0,100)||'';var count=answers.length;return JSON.stringify({title:title.slice(0,50),firstAnswer:first,answerCount:count});})()\""
],
"judge": {
"type": "matchesPattern",
+21055
View File
File diff suppressed because it is too large Load Diff
+68 -121
View File
@@ -1,52 +1,7 @@
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
assertAuthenticatedState,
buildDetailUrl,
buildProvenance,
cleanText,
extractOfferId,
gotoAndReadState,
type MediaSource,
uniqueMediaSources,
} from './shared.js';
interface AssetBrowserPayload {
href?: string;
title?: string;
offerTitle?: string;
offerId?: string | number;
gallery?: {
mainImage?: string[];
offerImgList?: string[];
wlImageInfos?: Array<{ fullPathImageURI?: string }>;
[key: string]: unknown;
};
scannedAssets?: MediaSource[];
}
export interface Normalized1688Assets {
offer_id: string | null;
title: string | null;
item_url: string;
main_images: string[];
sku_images: string[];
detail_images: string[];
videos: string[];
other_images: string[];
raw_assets: MediaSource[];
source: string[];
main_count: number;
sku_count: number;
detail_count: number;
video_count: number;
source_url: string;
fetched_at: string;
strategy: string;
}
function scriptToReadAssets(): string {
return `
import { assertAuthenticatedState, buildDetailUrl, buildProvenance, cleanText, extractOfferId, gotoAndReadState, uniqueMediaSources, } from './shared.js';
function scriptToReadAssets() {
return `
(() => {
const root = window.context ?? {};
const model = root.result?.global?.globalData?.model ?? null;
@@ -174,84 +129,76 @@ function scriptToReadAssets(): string {
})()
`;
}
function normalizeAssets(payload: AssetBrowserPayload): Normalized1688Assets {
const offerId = cleanText(String(payload.offerId ?? '')) || extractOfferId(cleanText(payload.href)) || null;
const itemUrl = offerId ? buildDetailUrl(offerId) : cleanText(payload.href);
const seededAssets: MediaSource[] = [
...((payload.gallery?.mainImage ?? []).map((url) => ({ type: 'image' as const, group: 'main' as const, url, source: 'page_state:mainImage' }))),
...((payload.gallery?.offerImgList ?? []).map((url) => ({ type: 'image' as const, group: 'main' as const, url, source: 'page_state:offerImgList' }))),
...((payload.gallery?.wlImageInfos ?? []).map((item) => ({
type: 'image' as const,
group: 'main' as const,
url: item?.fullPathImageURI ?? '',
source: 'page_state:wlImageInfos',
}))),
];
const assets = uniqueMediaSources([...seededAssets, ...(payload.scannedAssets ?? [])]);
const mainImages = assets.filter((item) => item.type === 'image' && item.group === 'main').map((item) => item.url);
const skuImages = assets.filter((item) => item.type === 'image' && item.group === 'sku').map((item) => item.url);
const detailImages = assets.filter((item) => item.type === 'image' && item.group === 'detail').map((item) => item.url);
const videos = assets.filter((item) => item.type === 'video').map((item) => item.url);
const otherImages = assets
.filter((item) => item.type === 'image' && !['main', 'sku', 'detail'].includes(item.group))
.map((item) => item.url);
return {
offer_id: offerId,
title: cleanText(payload.offerTitle) || cleanText(payload.title) || null,
item_url: itemUrl,
main_images: mainImages,
sku_images: skuImages,
detail_images: detailImages,
videos,
other_images: otherImages,
raw_assets: assets,
source: [...new Set(assets.map((item) => cleanText(item.source)).filter(Boolean))],
main_count: mainImages.length,
sku_count: skuImages.length,
detail_count: detailImages.length,
video_count: videos.length,
...buildProvenance(cleanText(payload.href) || itemUrl),
};
function normalizeAssets(payload) {
const offerId = cleanText(String(payload.offerId ?? '')) || extractOfferId(cleanText(payload.href)) || null;
const itemUrl = offerId ? buildDetailUrl(offerId) : cleanText(payload.href);
const seededAssets = [
...((payload.gallery?.mainImage ?? []).map((url) => ({ type: 'image', group: 'main', url, source: 'page_state:mainImage' }))),
...((payload.gallery?.offerImgList ?? []).map((url) => ({ type: 'image', group: 'main', url, source: 'page_state:offerImgList' }))),
...((payload.gallery?.wlImageInfos ?? []).map((item) => ({
type: 'image',
group: 'main',
url: item?.fullPathImageURI ?? '',
source: 'page_state:wlImageInfos',
}))),
];
const assets = uniqueMediaSources([...seededAssets, ...(payload.scannedAssets ?? [])]);
const mainImages = assets.filter((item) => item.type === 'image' && item.group === 'main').map((item) => item.url);
const skuImages = assets.filter((item) => item.type === 'image' && item.group === 'sku').map((item) => item.url);
const detailImages = assets.filter((item) => item.type === 'image' && item.group === 'detail').map((item) => item.url);
const videos = assets.filter((item) => item.type === 'video').map((item) => item.url);
const otherImages = assets
.filter((item) => item.type === 'image' && !['main', 'sku', 'detail'].includes(item.group))
.map((item) => item.url);
return {
offer_id: offerId,
title: cleanText(payload.offerTitle) || cleanText(payload.title) || null,
item_url: itemUrl,
main_images: mainImages,
sku_images: skuImages,
detail_images: detailImages,
videos,
other_images: otherImages,
raw_assets: assets,
source: [...new Set(assets.map((item) => cleanText(item.source)).filter(Boolean))],
main_count: mainImages.length,
sku_count: skuImages.length,
detail_count: detailImages.length,
video_count: videos.length,
...buildProvenance(cleanText(payload.href) || itemUrl),
};
}
async function readAssetsPayload(page: IPage, itemUrl: string): Promise<AssetBrowserPayload> {
const state = await gotoAndReadState(page, itemUrl, 2500, 'assets');
assertAuthenticatedState(state, 'assets');
await page.autoScroll({ times: 3, delayMs: 400 });
await page.wait(1);
return await page.evaluate(scriptToReadAssets()) as AssetBrowserPayload;
async function readAssetsPayload(page, itemUrl) {
const state = await gotoAndReadState(page, itemUrl, 2500, 'assets');
assertAuthenticatedState(state, 'assets');
await page.autoScroll({ times: 3, delayMs: 400 });
await page.wait(1);
return await page.evaluate(scriptToReadAssets());
}
export async function extractAssetsForInput(page: IPage, input: string): Promise<Normalized1688Assets> {
const itemUrl = buildDetailUrl(String(input ?? ''));
const payload = await readAssetsPayload(page, itemUrl);
return normalizeAssets(payload);
export async function extractAssetsForInput(page, input) {
const itemUrl = buildDetailUrl(String(input ?? ''));
const payload = await readAssetsPayload(page, itemUrl);
return normalizeAssets(payload);
}
cli({
site: '1688',
name: 'assets',
description: '列出 1688 商品页可提取的图片/视频素材',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
site: '1688',
name: 'assets',
description: '列出 1688 商品页可提取的图片/视频素材',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
},
],
columns: ['offer_id', 'title', 'main_count', 'sku_count', 'detail_count', 'video_count'],
func: async (page, kwargs) => {
return [await extractAssetsForInput(page, String(kwargs.input ?? ''))];
},
],
columns: ['offer_id', 'title', 'main_count', 'sku_count', 'detail_count', 'video_count'],
func: async (page, kwargs) => {
return [await extractAssetsForInput(page, String(kwargs.input ?? ''))];
},
});
export const __test__ = {
normalizeAssets,
normalizeAssets,
};
+39
View File
@@ -0,0 +1,39 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './assets.js';
import { __test__ as sharedTest } from './shared.js';
describe('1688 assets normalization', () => {
it('normalizes gallery and scanned assets into grouped media lists', () => {
const result = __test__.normalizeAssets({
href: 'https://detail.1688.com/offer/887904326744.html',
title: '测试商品 - 阿里巴巴',
offerTitle: '测试商品',
offerId: 887904326744,
gallery: {
mainImage: ['//img.example.com/main-1.jpg'],
offerImgList: ['https://img.example.com/main-2.jpg'],
wlImageInfos: [{ fullPathImageURI: 'https://img.example.com/main-3.jpg' }],
},
scannedAssets: [
{ type: 'image', group: 'sku', url: 'https://img.example.com/sku-1.png', source: 'dom:.sku' },
{ type: 'image', group: 'detail', url: 'https://img.example.com/detail-1.jpg', source: 'dom:.detail' },
{ type: 'video', group: 'video', url: 'https://video.example.com/demo.mp4', source: 'script' },
{ type: 'image', group: 'detail', url: 'blob:https://detail.1688.com/1', source: 'ignore' },
],
});
expect(result.offer_id).toBe('887904326744');
expect(result.main_images).toEqual([
'https://img.example.com/main-1.jpg',
'https://img.example.com/main-2.jpg',
'https://img.example.com/main-3.jpg',
]);
expect(result.sku_images).toEqual(['https://img.example.com/sku-1.png']);
expect(result.detail_images).toEqual(['https://img.example.com/detail-1.jpg']);
expect(result.videos).toEqual(['https://video.example.com/demo.mp4']);
expect(result.main_count).toBe(3);
expect(result.video_count).toBe(1);
});
it('normalizes media urls from style syntax and protocol-relative URLs', () => {
expect(sharedTest.normalizeMediaUrl('url("//img.example.com/1.jpg")')).toBe('https://img.example.com/1.jpg');
expect(sharedTest.normalizeMediaUrl('blob:https://detail.1688.com/1')).toBe('');
});
});
-42
View File
@@ -1,42 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './assets.js';
import { __test__ as sharedTest } from './shared.js';
describe('1688 assets normalization', () => {
it('normalizes gallery and scanned assets into grouped media lists', () => {
const result = __test__.normalizeAssets({
href: 'https://detail.1688.com/offer/887904326744.html',
title: '测试商品 - 阿里巴巴',
offerTitle: '测试商品',
offerId: 887904326744,
gallery: {
mainImage: ['//img.example.com/main-1.jpg'],
offerImgList: ['https://img.example.com/main-2.jpg'],
wlImageInfos: [{ fullPathImageURI: 'https://img.example.com/main-3.jpg' }],
},
scannedAssets: [
{ type: 'image', group: 'sku', url: 'https://img.example.com/sku-1.png', source: 'dom:.sku' },
{ type: 'image', group: 'detail', url: 'https://img.example.com/detail-1.jpg', source: 'dom:.detail' },
{ type: 'video', group: 'video', url: 'https://video.example.com/demo.mp4', source: 'script' },
{ type: 'image', group: 'detail', url: 'blob:https://detail.1688.com/1', source: 'ignore' },
],
});
expect(result.offer_id).toBe('887904326744');
expect(result.main_images).toEqual([
'https://img.example.com/main-1.jpg',
'https://img.example.com/main-2.jpg',
'https://img.example.com/main-3.jpg',
]);
expect(result.sku_images).toEqual(['https://img.example.com/sku-1.png']);
expect(result.detail_images).toEqual(['https://img.example.com/detail-1.jpg']);
expect(result.videos).toEqual(['https://video.example.com/demo.mp4']);
expect(result.main_count).toBe(3);
expect(result.video_count).toBe(1);
});
it('normalizes media urls from style syntax and protocol-relative URLs', () => {
expect(sharedTest.normalizeMediaUrl('url("//img.example.com/1.jpg")')).toBe('https://img.example.com/1.jpg');
expect(sharedTest.normalizeMediaUrl('blob:https://detail.1688.com/1')).toBe('');
});
});
+76
View File
@@ -0,0 +1,76 @@
import * as path from 'node:path';
import { formatCookieHeader } from '@jackwener/opencli/download';
import { downloadMedia } from '@jackwener/opencli/download/media-download';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { cleanText } from './shared.js';
import { extractAssetsForInput } from './assets.js';
function extFromUrl(url, fallback) {
try {
const ext = path.extname(new URL(url).pathname).toLowerCase();
if (ext && ext.length <= 8)
return ext;
}
catch {
// ignore
}
return fallback;
}
function toDownloadItems(offerId, assets) {
const items = [];
const pushImages = (urls, prefix) => {
urls.forEach((url, index) => {
items.push({
type: 'image',
url,
filename: `${offerId}_${prefix}_${String(index + 1).padStart(2, '0')}${extFromUrl(url, '.jpg')}`,
});
});
};
pushImages(assets.main_images, 'main');
pushImages(assets.sku_images, 'sku');
pushImages(assets.detail_images, 'detail');
pushImages(assets.other_images, 'other');
assets.videos.forEach((url, index) => {
items.push({
type: 'video',
url,
filename: `${offerId}_video_${String(index + 1).padStart(2, '0')}${extFromUrl(url, '.mp4')}`,
});
});
return items;
}
cli({
site: '1688',
name: 'download',
description: '批量下载 1688 商品页可提取的图片和视频素材',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
},
{ name: 'output', default: './1688-downloads', help: '输出目录' },
],
columns: ['index', 'type', 'status', 'size'],
func: async (page, kwargs) => {
const assets = await extractAssetsForInput(page, String(kwargs.input ?? ''));
const offerId = cleanText(assets.offer_id) || '1688';
const items = toDownloadItems(offerId, assets);
const browserCookies = await page.getCookies({ domain: '1688.com' });
return downloadMedia(items, {
output: String(kwargs.output || './1688-downloads'),
subdir: offerId,
cookies: formatCookieHeader(browserCookies),
browserCookies,
filenamePrefix: offerId,
timeout: 60000,
});
},
});
export const __test__ = {
extFromUrl,
toDownloadItems,
};
+31
View File
@@ -0,0 +1,31 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './download.js';
describe('1688 download helpers', () => {
it('builds stable filenames for grouped assets', () => {
const items = __test__.toDownloadItems('887904326744', {
offer_id: '887904326744',
title: '测试商品',
item_url: 'https://detail.1688.com/offer/887904326744.html',
main_images: ['https://img.example.com/a.jpg'],
sku_images: ['https://img.example.com/b.png'],
detail_images: ['https://img.example.com/c.webp'],
videos: ['https://video.example.com/d.mp4'],
other_images: [],
raw_assets: [],
source: [],
main_count: 1,
sku_count: 1,
detail_count: 1,
video_count: 1,
source_url: 'https://detail.1688.com/offer/887904326744.html',
fetched_at: new Date().toISOString(),
strategy: 'cookie',
});
expect(items.map((item) => item.filename)).toEqual([
'887904326744_main_01.jpg',
'887904326744_sku_01.png',
'887904326744_detail_01.webp',
'887904326744_video_01.mp4',
]);
});
});
-33
View File
@@ -1,33 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './download.js';
describe('1688 download helpers', () => {
it('builds stable filenames for grouped assets', () => {
const items = __test__.toDownloadItems('887904326744', {
offer_id: '887904326744',
title: '测试商品',
item_url: 'https://detail.1688.com/offer/887904326744.html',
main_images: ['https://img.example.com/a.jpg'],
sku_images: ['https://img.example.com/b.png'],
detail_images: ['https://img.example.com/c.webp'],
videos: ['https://video.example.com/d.mp4'],
other_images: [],
raw_assets: [],
source: [],
main_count: 1,
sku_count: 1,
detail_count: 1,
video_count: 1,
source_url: 'https://detail.1688.com/offer/887904326744.html',
fetched_at: new Date().toISOString(),
strategy: 'cookie',
});
expect(items.map((item) => item.filename)).toEqual([
'887904326744_main_01.jpg',
'887904326744_sku_01.png',
'887904326744_detail_01.webp',
'887904326744_video_01.mp4',
]);
});
});
-83
View File
@@ -1,83 +0,0 @@
import * as path from 'node:path';
import { formatCookieHeader } from '@jackwener/opencli/download';
import { downloadMedia, type MediaItem } from '@jackwener/opencli/download/media-download';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { cleanText } from './shared.js';
import { extractAssetsForInput } from './assets.js';
function extFromUrl(url: string, fallback: string): string {
try {
const ext = path.extname(new URL(url).pathname).toLowerCase();
if (ext && ext.length <= 8) return ext;
} catch {
// ignore
}
return fallback;
}
function toDownloadItems(offerId: string, assets: Awaited<ReturnType<typeof extractAssetsForInput>>): MediaItem[] {
const items: MediaItem[] = [];
const pushImages = (urls: string[], prefix: string) => {
urls.forEach((url, index) => {
items.push({
type: 'image',
url,
filename: `${offerId}_${prefix}_${String(index + 1).padStart(2, '0')}${extFromUrl(url, '.jpg')}`,
});
});
};
pushImages(assets.main_images, 'main');
pushImages(assets.sku_images, 'sku');
pushImages(assets.detail_images, 'detail');
pushImages(assets.other_images, 'other');
assets.videos.forEach((url, index) => {
items.push({
type: 'video',
url,
filename: `${offerId}_video_${String(index + 1).padStart(2, '0')}${extFromUrl(url, '.mp4')}`,
});
});
return items;
}
cli({
site: '1688',
name: 'download',
description: '批量下载 1688 商品页可提取的图片和视频素材',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
},
{ name: 'output', default: './1688-downloads', help: '输出目录' },
],
columns: ['index', 'type', 'status', 'size'],
func: async (page, kwargs) => {
const assets = await extractAssetsForInput(page, String(kwargs.input ?? ''));
const offerId = cleanText(assets.offer_id) || '1688';
const items = toDownloadItems(offerId, assets);
const browserCookies = await page.getCookies({ domain: '1688.com' });
return downloadMedia(items, {
output: String(kwargs.output || './1688-downloads'),
subdir: offerId,
cookies: formatCookieHeader(browserCookies),
browserCookies,
filenamePrefix: offerId,
timeout: 60000,
});
},
});
export const __test__ = {
extFromUrl,
toDownloadItems,
};
+187
View File
@@ -0,0 +1,187 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { isRecord } from '@jackwener/opencli/utils';
import { assertAuthenticatedState, buildDetailUrl, buildProvenance, canonicalizeSellerUrl, cleanMultilineText, cleanText, extractLocation, extractMemberId, extractOfferId, extractShopId, gotoAndReadState, normalizePriceTiers, parseMoqText, parsePriceText, toNumber, uniqueNonEmpty, } from './shared.js';
function normalizeItemPayload(payload) {
const href = cleanText(payload.href);
const bodyText = cleanMultilineText(payload.bodyText);
const sellerName = cleanText(payload.seller?.companyName);
const sellerUrlRaw = cleanText(payload.seller?.winportUrl
?? payload.seller?.sellerWinportUrlMap?.defaultUrl
?? payload.seller?.sellerWinportUrlMap?.indexUrl);
const sellerUrl = canonicalizeSellerUrl(sellerUrlRaw);
const offerId = cleanText(String(payload.offerId ?? '')) || extractOfferId(href) || null;
const memberId = cleanText(payload.seller?.memberId) || extractMemberId(sellerUrlRaw || href) || null;
const shopId = extractShopId(sellerUrl ?? href);
const unit = cleanText(payload.trade?.unit);
const priceDisplay = cleanText(payload.trade?.priceDisplay);
const priceRange = parsePriceText(priceDisplay ? `¥${priceDisplay}` : bodyText);
const moqText = extractMoqText(bodyText, payload.trade?.beginAmount, unit);
const moq = parseMoqText(moqText);
const services = uniqueServices(payload);
const serviceBadges = uniqueNonEmpty(services.map((service) => cleanText(service.serviceName)));
const attributes = normalizeVisibleAttributes(payload.trade?.offerIDatacenterSellInfo);
const priceTiers = normalizePriceTiers(payload.trade?.offerPriceModel?.currentPrices ?? [], unit || null);
const images = uniqueNonEmpty([
...(payload.gallery?.mainImage ?? []),
...(payload.gallery?.offerImgList ?? []),
...((payload.gallery?.wlImageInfos ?? []).map((item) => item.fullPathImageURI ?? '')),
]);
const detailUrl = offerId ? buildDetailUrl(offerId) : href;
const provenance = buildProvenance(href || detailUrl);
return {
offer_id: offerId,
member_id: memberId,
shop_id: shopId,
title: cleanText(payload.offerTitle) || stripAlibabaSuffix(payload.title) || firstNonEmptyLine(bodyText) || null,
item_url: detailUrl,
main_images: images,
price_text: priceRange.price_text || null,
price_tiers: priceTiers,
currency: priceRange.currency,
moq_text: moq.moq_text || null,
moq_value: moq.moq_value,
seller_name: sellerName || null,
seller_url: sellerUrl,
shop_name: sellerName || null,
origin_place: extractLocation(bodyText),
delivery_days_text: extractDeliveryDaysText(bodyText, services, payload.shipping),
customization_text: extractKeywordLine(bodyText, ['来样定制', '来图定制', '支持定制', '可定制', '定制']),
private_label_text: extractKeywordLine(bodyText, ['贴牌', '贴标', '定制logo', '打logo', 'OEM', 'ODM']),
visible_attributes: attributes,
sales_text: extractSalesText(bodyText),
service_badges: serviceBadges,
stock_quantity: extractStockQuantity(bodyText),
...provenance,
};
}
function normalizeVisibleAttributes(raw) {
if (!isRecord(raw))
return [];
return Object.entries(raw)
.filter(([key, value]) => key !== 'sellPointModel' && cleanText(key) && cleanText(String(value)))
.map(([key, value]) => ({ key: cleanText(key), value: cleanText(String(value)) }));
}
function uniqueServices(payload) {
const combined = [
...(Array.isArray(payload.services) ? payload.services : []),
...(Array.isArray(payload.shipping?.protectionInfos) ? payload.shipping.protectionInfos : []),
...(Array.isArray(payload.shipping?.buyerProtectionModel) ? payload.shipping.buyerProtectionModel : []),
];
const seen = new Set();
const result = [];
for (const service of combined) {
const key = cleanText(service.serviceName);
if (!key || seen.has(key))
continue;
seen.add(key);
result.push(service);
}
return result;
}
function stripAlibabaSuffix(title) {
return cleanText(title).replace(/\s*-\s*阿里巴巴$/, '').trim();
}
function firstNonEmptyLine(text) {
return text.split('\n').map((line) => cleanText(line)).find(Boolean) ?? '';
}
function extractMoqText(bodyText, beginAmount, unit) {
const lineMatch = bodyText.match(/\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)\s*起批/);
if (lineMatch)
return lineMatch[0];
const moqValue = toNumber(beginAmount);
if (moqValue !== null) {
return `${moqValue}${unit || ''}起批`;
}
return '';
}
function extractDeliveryDaysText(bodyText, services, shipping) {
const shippingText = cleanText(shipping?.deliveryLimitText) || cleanText(shipping?.logisticsText);
if (shippingText)
return shippingText;
const textMatch = bodyText.match(/\d+\s*(?:小时|天)(?:内)?发货/);
if (textMatch)
return textMatch[0];
const hourMatch = services.find((service) => typeof service.agreeDeliveryHours === 'number');
if (hourMatch && typeof hourMatch.agreeDeliveryHours === 'number') {
return `${hourMatch.agreeDeliveryHours}小时内发货`;
}
return null;
}
function extractKeywordLine(bodyText, keywords) {
const lines = bodyText.split('\n').map((line) => cleanText(line)).filter(Boolean);
for (const line of lines) {
if (keywords.some((keyword) => line.includes(keyword))) {
return line;
}
}
return null;
}
function extractSalesText(bodyText) {
const match = bodyText.match(/(?:全网销量|已售)\s*\d+(?:\.\d+)?\+?\s*[件套个单]?/);
return match ? cleanText(match[0]) : null;
}
function extractStockQuantity(bodyText) {
const match = bodyText.match(/库存\s*(\d+)/);
return match ? Number.parseInt(match[1], 10) : null;
}
async function readItemPayload(page, itemUrl) {
const state = await gotoAndReadState(page, itemUrl, 2500, 'item');
assertAuthenticatedState(state, 'item');
const payload = await page.evaluate(`
(() => {
const root = window.context ?? {};
const model = root.result?.global?.globalData?.model ?? null;
const toJson = (value) => JSON.parse(JSON.stringify(value ?? null));
return {
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
offerTitle: model?.offerTitleModel?.subject ?? '',
offerId: model?.tradeModel?.offerId ?? '',
seller: toJson(model?.sellerModel),
trade: toJson(model?.tradeModel),
gallery: toJson(root.result?.data?.gallery?.fields ?? null),
shipping: toJson(root.result?.data?.shippingServices?.fields ?? null),
services: toJson(root.result?.data?.shippingServices?.fields?.protectionInfos ?? []),
};
})()
`);
const resolvedOfferId = cleanText(String(payload.offerId ?? '')) || extractOfferId(cleanText(payload.href));
if (!resolvedOfferId) {
throw new CommandExecutionError('1688 item page did not expose product context', '当前 tab 非商品详情上下文,请切到 detail.1688.com 商品页并重试');
}
return payload;
}
cli({
site: '1688',
name: 'item',
description: '1688 商品详情(公开商品字段、价格阶梯、卖家基础信息)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
},
],
columns: ['offer_id', 'title', 'price_text', 'moq_text', 'seller_name', 'origin_place'],
func: async (page, kwargs) => {
const itemUrl = buildDetailUrl(String(kwargs.input ?? ''));
const payload = await readItemPayload(page, itemUrl);
return [normalizeItemPayload(payload)];
},
});
export const __test__ = {
normalizeItemPayload,
normalizeVisibleAttributes,
stripAlibabaSuffix,
extractMoqText,
extractDeliveryDaysText,
extractKeywordLine,
extractSalesText,
extractStockQuantity,
};
+67
View File
@@ -0,0 +1,67 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './item.js';
describe('1688 item normalization', () => {
it('normalizes public item payload into contract fields', () => {
const result = __test__.normalizeItemPayload({
href: 'https://detail.1688.com/offer/887904326744.html',
title: '法式春季长袖开衫连衣裙女新款大码女装碎花吊带裙套装142077 - 阿里巴巴',
bodyText: `
青岛沁澜衣品服装有限公司
入驻13年
主营:大码女装
店铺回头率
87%
山东青岛
3套起批
已售1600+套
支持定制logo
`,
offerTitle: '法式春季长袖开衫连衣裙女新款大码女装碎花吊带裙套装142077',
offerId: 887904326744,
seller: {
companyName: '青岛沁澜衣品服装有限公司',
memberId: 'b2b-1641351767',
winportUrl: 'https://yinuoweierfushi.1688.com/page/index.html?spm=a1',
},
trade: {
beginAmount: 3,
priceDisplay: '96.00-98.00',
unit: '套',
saleCount: 1655,
offerIDatacenterSellInfo: {
面料名称: '莫代尔',
主面料成分: '莫代尔纤维',
sellPointModel: '{"ignore":true}',
},
offerPriceModel: {
currentPrices: [
{ beginAmount: 3, price: '98.00' },
{ beginAmount: 50, price: '97.00' },
],
},
},
gallery: {
mainImage: ['https://example.com/1.jpg'],
offerImgList: ['https://example.com/2.jpg'],
wlImageInfos: [{ fullPathImageURI: 'https://example.com/3.jpg' }],
},
services: [
{ serviceName: '延期必赔', agreeDeliveryHours: 360 },
{ serviceName: '品质保障' },
],
});
expect(result.offer_id).toBe('887904326744');
expect(result.member_id).toBe('b2b-1641351767');
expect(result.shop_id).toBe('yinuoweierfushi');
expect(result.seller_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.price_text).toBe('¥96.00-98.00');
expect(result.moq_text).toBe('3套起批');
expect(result.origin_place).toBe('山东青岛');
expect(result.delivery_days_text).toBe('360小时内发货');
expect(result.private_label_text).toBe('支持定制logo');
expect(result.visible_attributes).toEqual([
{ key: '面料名称', value: '莫代尔' },
{ key: '主面料成分', value: '莫代尔纤维' },
]);
});
});
-69
View File
@@ -1,69 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './item.js';
describe('1688 item normalization', () => {
it('normalizes public item payload into contract fields', () => {
const result = __test__.normalizeItemPayload({
href: 'https://detail.1688.com/offer/887904326744.html',
title: '法式春季长袖开衫连衣裙女新款大码女装碎花吊带裙套装142077 - 阿里巴巴',
bodyText: `
青岛沁澜衣品服装有限公司
入驻13年
主营:大码女装
店铺回头率
87%
山东青岛
3套起批
已售1600+套
支持定制logo
`,
offerTitle: '法式春季长袖开衫连衣裙女新款大码女装碎花吊带裙套装142077',
offerId: 887904326744,
seller: {
companyName: '青岛沁澜衣品服装有限公司',
memberId: 'b2b-1641351767',
winportUrl: 'https://yinuoweierfushi.1688.com/page/index.html?spm=a1',
},
trade: {
beginAmount: 3,
priceDisplay: '96.00-98.00',
unit: '套',
saleCount: 1655,
offerIDatacenterSellInfo: {
: '莫代尔',
: '莫代尔纤维',
sellPointModel: '{"ignore":true}',
},
offerPriceModel: {
currentPrices: [
{ beginAmount: 3, price: '98.00' },
{ beginAmount: 50, price: '97.00' },
],
},
},
gallery: {
mainImage: ['https://example.com/1.jpg'],
offerImgList: ['https://example.com/2.jpg'],
wlImageInfos: [{ fullPathImageURI: 'https://example.com/3.jpg' }],
},
services: [
{ serviceName: '延期必赔', agreeDeliveryHours: 360 },
{ serviceName: '品质保障' },
],
});
expect(result.offer_id).toBe('887904326744');
expect(result.member_id).toBe('b2b-1641351767');
expect(result.shop_id).toBe('yinuoweierfushi');
expect(result.seller_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.price_text).toBe('¥96.00-98.00');
expect(result.moq_text).toBe('3套起批');
expect(result.origin_place).toBe('山东青岛');
expect(result.delivery_days_text).toBe('360小时内发货');
expect(result.private_label_text).toBe('支持定制logo');
expect(result.visible_attributes).toEqual([
{ key: '面料名称', value: '莫代尔' },
{ key: '主面料成分', value: '莫代尔纤维' },
]);
});
});
-282
View File
@@ -1,282 +0,0 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import { isRecord } from '@jackwener/opencli/utils';
import {
assertAuthenticatedState,
buildDetailUrl,
buildProvenance,
canonicalizeSellerUrl,
cleanMultilineText,
cleanText,
extractLocation,
extractMemberId,
extractOfferId,
extractShopId,
gotoAndReadState,
normalizePriceTiers,
parseMoqText,
parsePriceText,
toNumber,
uniqueNonEmpty,
} from './shared.js';
interface BuyerProtectionModel {
serviceName?: string;
shortBuyerDesc?: string;
packageBuyerDesc?: string;
textDesc?: string;
agreeDeliveryHours?: number;
}
interface ItemBrowserPayload {
href?: string;
title?: string;
bodyText?: string;
offerTitle?: string;
offerId?: string | number;
seller?: {
companyName?: string;
memberId?: string;
winportUrl?: string;
sellerWinportUrlMap?: Record<string, string>;
};
trade?: {
beginAmount?: string | number;
priceDisplay?: string;
unit?: string;
saleCount?: string | number;
offerIDatacenterSellInfo?: Record<string, unknown>;
offerPriceModel?: {
currentPrices?: Array<{ beginAmount?: string | number; price?: string | number }>;
};
};
gallery?: {
mainImage?: string[];
offerImgList?: string[];
wlImageInfos?: Array<{ fullPathImageURI?: string }>;
};
shipping?: {
deliveryLimitText?: string;
logisticsText?: string;
protectionInfos?: BuyerProtectionModel[];
buyerProtectionModel?: BuyerProtectionModel[];
};
services?: BuyerProtectionModel[];
}
interface VisibleAttribute {
key: string;
value: string;
}
function normalizeItemPayload(payload: ItemBrowserPayload): Record<string, unknown> {
const href = cleanText(payload.href);
const bodyText = cleanMultilineText(payload.bodyText);
const sellerName = cleanText(payload.seller?.companyName);
const sellerUrlRaw = cleanText(
payload.seller?.winportUrl
?? payload.seller?.sellerWinportUrlMap?.defaultUrl
?? payload.seller?.sellerWinportUrlMap?.indexUrl,
);
const sellerUrl = canonicalizeSellerUrl(sellerUrlRaw);
const offerId = cleanText(String(payload.offerId ?? '')) || extractOfferId(href) || null;
const memberId = cleanText(payload.seller?.memberId) || extractMemberId(sellerUrlRaw || href) || null;
const shopId = extractShopId(sellerUrl ?? href);
const unit = cleanText(payload.trade?.unit);
const priceDisplay = cleanText(payload.trade?.priceDisplay);
const priceRange = parsePriceText(priceDisplay ? `¥${priceDisplay}` : bodyText);
const moqText = extractMoqText(bodyText, payload.trade?.beginAmount, unit);
const moq = parseMoqText(moqText);
const services = uniqueServices(payload);
const serviceBadges = uniqueNonEmpty(services.map((service) => cleanText(service.serviceName)));
const attributes = normalizeVisibleAttributes(payload.trade?.offerIDatacenterSellInfo);
const priceTiers = normalizePriceTiers(payload.trade?.offerPriceModel?.currentPrices ?? [], unit || null);
const images = uniqueNonEmpty([
...(payload.gallery?.mainImage ?? []),
...(payload.gallery?.offerImgList ?? []),
...((payload.gallery?.wlImageInfos ?? []).map((item) => item.fullPathImageURI ?? '')),
]);
const detailUrl = offerId ? buildDetailUrl(offerId) : href;
const provenance = buildProvenance(href || detailUrl);
return {
offer_id: offerId,
member_id: memberId,
shop_id: shopId,
title: cleanText(payload.offerTitle) || stripAlibabaSuffix(payload.title) || firstNonEmptyLine(bodyText) || null,
item_url: detailUrl,
main_images: images,
price_text: priceRange.price_text || null,
price_tiers: priceTiers,
currency: priceRange.currency,
moq_text: moq.moq_text || null,
moq_value: moq.moq_value,
seller_name: sellerName || null,
seller_url: sellerUrl,
shop_name: sellerName || null,
origin_place: extractLocation(bodyText),
delivery_days_text: extractDeliveryDaysText(bodyText, services, payload.shipping),
customization_text: extractKeywordLine(bodyText, ['来样定制', '来图定制', '支持定制', '可定制', '定制']),
private_label_text: extractKeywordLine(bodyText, ['贴牌', '贴标', '定制logo', '打logo', 'OEM', 'ODM']),
visible_attributes: attributes,
sales_text: extractSalesText(bodyText),
service_badges: serviceBadges,
stock_quantity: extractStockQuantity(bodyText),
...provenance,
};
}
function normalizeVisibleAttributes(raw: unknown): VisibleAttribute[] {
if (!isRecord(raw)) return [];
return Object.entries(raw)
.filter(([key, value]) => key !== 'sellPointModel' && cleanText(key) && cleanText(String(value)))
.map(([key, value]) => ({ key: cleanText(key), value: cleanText(String(value)) }));
}
function uniqueServices(payload: ItemBrowserPayload): BuyerProtectionModel[] {
const combined = [
...(Array.isArray(payload.services) ? payload.services : []),
...(Array.isArray(payload.shipping?.protectionInfos) ? payload.shipping.protectionInfos : []),
...(Array.isArray(payload.shipping?.buyerProtectionModel) ? payload.shipping.buyerProtectionModel : []),
];
const seen = new Set<string>();
const result: BuyerProtectionModel[] = [];
for (const service of combined) {
const key = cleanText(service.serviceName);
if (!key || seen.has(key)) continue;
seen.add(key);
result.push(service);
}
return result;
}
function stripAlibabaSuffix(title: string | undefined): string {
return cleanText(title).replace(/\s*-\s*阿里巴巴$/, '').trim();
}
function firstNonEmptyLine(text: string): string {
return text.split('\n').map((line) => cleanText(line)).find(Boolean) ?? '';
}
function extractMoqText(bodyText: string, beginAmount: string | number | undefined, unit: string): string {
const lineMatch = bodyText.match(/\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)\s*起批/);
if (lineMatch) return lineMatch[0];
const moqValue = toNumber(beginAmount);
if (moqValue !== null) {
return `${moqValue}${unit || ''}起批`;
}
return '';
}
function extractDeliveryDaysText(
bodyText: string,
services: BuyerProtectionModel[],
shipping: ItemBrowserPayload['shipping'],
): string | null {
const shippingText = cleanText(shipping?.deliveryLimitText) || cleanText(shipping?.logisticsText);
if (shippingText) return shippingText;
const textMatch = bodyText.match(/\d+\s*(?:小时|天)(?:内)?发货/);
if (textMatch) return textMatch[0];
const hourMatch = services.find((service) => typeof service.agreeDeliveryHours === 'number');
if (hourMatch && typeof hourMatch.agreeDeliveryHours === 'number') {
return `${hourMatch.agreeDeliveryHours}小时内发货`;
}
return null;
}
function extractKeywordLine(bodyText: string, keywords: string[]): string | null {
const lines = bodyText.split('\n').map((line) => cleanText(line)).filter(Boolean);
for (const line of lines) {
if (keywords.some((keyword) => line.includes(keyword))) {
return line;
}
}
return null;
}
function extractSalesText(bodyText: string): string | null {
const match = bodyText.match(/(?:全网销量|已售)\s*\d+(?:\.\d+)?\+?\s*[件套个单]?/);
return match ? cleanText(match[0]) : null;
}
function extractStockQuantity(bodyText: string): number | null {
const match = bodyText.match(/库存\s*(\d+)/);
return match ? Number.parseInt(match[1], 10) : null;
}
async function readItemPayload(page: IPage, itemUrl: string): Promise<ItemBrowserPayload> {
const state = await gotoAndReadState(page, itemUrl, 2500, 'item');
assertAuthenticatedState(state, 'item');
const payload = await page.evaluate(`
(() => {
const root = window.context ?? {};
const model = root.result?.global?.globalData?.model ?? null;
const toJson = (value) => JSON.parse(JSON.stringify(value ?? null));
return {
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
offerTitle: model?.offerTitleModel?.subject ?? '',
offerId: model?.tradeModel?.offerId ?? '',
seller: toJson(model?.sellerModel),
trade: toJson(model?.tradeModel),
gallery: toJson(root.result?.data?.gallery?.fields ?? null),
shipping: toJson(root.result?.data?.shippingServices?.fields ?? null),
services: toJson(root.result?.data?.shippingServices?.fields?.protectionInfos ?? []),
};
})()
`) as ItemBrowserPayload;
const resolvedOfferId = cleanText(String(payload.offerId ?? '')) || extractOfferId(cleanText(payload.href));
if (!resolvedOfferId) {
throw new CommandExecutionError(
'1688 item page did not expose product context',
'当前 tab 非商品详情上下文,请切到 detail.1688.com 商品页并重试',
);
}
return payload;
}
cli({
site: '1688',
name: 'item',
description: '1688 商品详情(公开商品字段、价格阶梯、卖家基础信息)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 商品 URL 或 offer ID(如 887904326744',
},
],
columns: ['offer_id', 'title', 'price_text', 'moq_text', 'seller_name', 'origin_place'],
func: async (page, kwargs) => {
const itemUrl = buildDetailUrl(String(kwargs.input ?? ''));
const payload = await readItemPayload(page, itemUrl);
return [normalizeItemPayload(payload)];
},
});
export const __test__ = {
normalizeItemPayload,
normalizeVisibleAttributes,
stripAlibabaSuffix,
extractMoqText,
extractDeliveryDaysText,
extractKeywordLine,
extractSalesText,
extractStockQuantity,
};
+309
View File
@@ -0,0 +1,309 @@
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { FACTORY_BADGE_PATTERNS, SERVICE_BADGE_PATTERNS, assertAuthenticatedState, buildProvenance, buildSearchUrl, canonicalizeItemUrl, canonicalizeSellerUrl, cleanText, extractBadges, extractLocation, extractMemberId, extractOfferId, extractShopId, gotoAndReadState, parseMoqText, parsePriceText, SEARCH_LIMIT_DEFAULT, SEARCH_LIMIT_MAX, parseSearchLimit, uniqueNonEmpty, } from './shared.js';
const SEARCH_ITEM_URL_PATTERNS = [
'detail.1688.com/offer/',
'detail.m.1688.com/page/index.html?offerId=',
];
const MAX_SEARCH_PAGES = 12;
function normalizeSearchCandidate(candidate, sourceUrl) {
const canonicalItemUrl = canonicalizeItemUrl(cleanText(candidate.item_url));
const containerText = cleanText(candidate.container_text);
const priceText = firstNonEmpty([
normalizeInlineText(candidate.price_text),
normalizeInlineText(extractPriceText(candidate.hover_price_text)),
]);
const priceRange = parsePriceText(priceText || containerText);
const moq = parseMoqText(firstNonEmpty([
normalizeInlineText(candidate.moq_text),
normalizeInlineText(extractMoqText(containerText)),
]));
const canonicalSellerUrl = canonicalizeSellerUrl(cleanText(candidate.seller_url));
const evidenceText = uniqueNonEmpty([
containerText,
...(candidate.desc_rows ?? []),
...(candidate.tag_items ?? []),
...(candidate.hover_items ?? []),
]).join('\n');
const badges = extractBadges(evidenceText, [...FACTORY_BADGE_PATTERNS, ...SERVICE_BADGE_PATTERNS]);
const salesText = firstNonEmpty([
extractSalesText(candidate.sales_text),
extractSalesText(containerText),
]);
const returnRateText = extractReturnRateText([...(candidate.tag_items ?? []), ...(candidate.hover_items ?? [])]);
const provenance = buildProvenance(sourceUrl);
return {
rank: 0,
offer_id: extractOfferId(canonicalItemUrl ?? '') ?? null,
member_id: extractMemberId(canonicalSellerUrl ?? '') ?? null,
shop_id: extractShopId(canonicalSellerUrl ?? '') ?? null,
title: cleanText(candidate.title) || firstWord(containerText) || null,
item_url: canonicalItemUrl,
seller_name: cleanText(candidate.seller_name) || null,
seller_url: canonicalSellerUrl,
price_text: priceRange.price_text || null,
price_min: priceRange.price_min,
price_max: priceRange.price_max,
currency: priceRange.currency,
moq_text: moq.moq_text || null,
moq_value: moq.moq_value,
location: extractLocation(containerText),
badges,
sales_text: salesText || null,
return_rate_text: returnRateText,
source_url: provenance.source_url,
fetched_at: provenance.fetched_at,
strategy: provenance.strategy,
};
}
function extractMoqText(text) {
const normalized = normalizeInlineText(text);
return normalized.match(/\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)\s*起批/i)?.[0]
?? normalized.match(/≥\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)?/i)?.[0]
?? normalized.match(/\d+(?:\.\d+)?\s*(?:~|-|至|到)\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)/i)?.[0]
?? '';
}
function extractPriceText(text) {
const normalized = normalizeInlineText(text);
return normalized.match(/[¥$€]\s*\d+(?:\.\d+)?/)?.[0] ?? '';
}
function extractSalesText(text) {
const normalized = normalizeInlineText(text);
if (!normalized)
return '';
if (/^\d+(?:\.\d+)?\+?\s*(件|套|个|单)$/.test(normalized)) {
return normalized;
}
const match = normalized.match(/(?:已售|销量|售)\s*\d+(?:\.\d+)?\+?\s*(件|套|个|单)?/);
return match ? cleanText(match[0]) : '';
}
function firstWord(text) {
return text.split(/\s+/).find(Boolean) ?? '';
}
function firstNonEmpty(values) {
return values.map((value) => cleanText(value)).find(Boolean) ?? '';
}
function normalizeInlineText(text) {
return cleanText(text)
.replace(/([¥$€])\s+(?=\d)/g, '$1')
.replace(/(\d)\s*\.\s*(\d)/g, '$1.$2')
.replace(/\s*([~-])\s*/g, '$1')
.trim();
}
function extractReturnRateText(values) {
return uniqueNonEmpty(values.map((value) => normalizeInlineText(value)))
.find((value) => /^回头率\s*\d+(?:\.\d+)?%$/.test(value))
?? null;
}
function buildDedupeKey(row) {
if (row.offer_id)
return `offer:${row.offer_id}`;
if (row.item_url)
return `url:${row.item_url}`;
return null;
}
async function readSearchPayload(page, url) {
const state = await gotoAndReadState(page, url, 2500, 'search');
assertAuthenticatedState(state, 'search');
const payload = await page.evaluate(`
(() => {
const normalizeText = (value) => (value || '').replace(/\\s+/g, ' ').trim();
const normalizeUrl = (href) => {
if (!href) return '';
try {
return new URL(href, window.location.href).toString();
} catch {
return '';
}
};
const isItemHref = (href) => ${JSON.stringify(SEARCH_ITEM_URL_PATTERNS)}
.some((pattern) => (href || '').includes(pattern));
const uniqueTexts = (values) => [...new Set(values.map((value) => normalizeText(value)).filter(Boolean))];
const collectTexts = (root, selector) => uniqueTexts(
Array.from(root.querySelectorAll(selector)).map((node) => node.innerText || node.textContent || ''),
);
const firstText = (root, selectors) => {
for (const selector of selectors) {
const node = root.querySelector(selector);
const value = normalizeText(node ? node.innerText || node.textContent || '' : '');
if (value) return value;
}
return '';
};
const findMoqText = (values, priceText) => {
const moqPattern = /(≥\\s*\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只)?)|(\\d+(?:\\.\\d+)?\\s*(?:~|-|至|到)\\s*\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只))|(\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只)\\s*起批)/i;
return values.find((value) => moqPattern.test(value))
|| normalizeText(priceText).match(moqPattern)?.[0]
|| '';
};
const isSellerHref = (href) => {
if (!href) return false;
try {
const url = new URL(href, window.location.href);
const host = url.hostname || '';
if (!host.endsWith('.1688.com')) return false;
if (
host === 's.1688.com'
|| host === 'r.1688.com'
|| host === 'air.1688.com'
|| host === 'detail.1688.com'
|| host === 'detail.m.1688.com'
|| host === 'dj.1688.com'
) {
return false;
}
return true;
} catch {
return false;
}
};
const pickContainer = (anchor) => {
let node = anchor;
while (node && node !== document.body) {
const text = normalizeText(node.innerText || node.textContent || '');
if (text.length >= 40 && text.length <= 2000) {
return node;
}
node = node.parentElement;
}
return anchor;
};
const collectCandidates = () => {
const anchors = Array.from(document.querySelectorAll('a')).filter((anchor) => isItemHref(anchor.href || ''));
const seen = new Set();
const items = [];
for (const anchor of anchors) {
const href = anchor.href || '';
if (!href || seen.has(href)) continue;
seen.add(href);
const container = pickContainer(anchor);
const tagItems = collectTexts(container, '.offer-tag-row .offer-desc-item');
const hoverItems = collectTexts(container, '.offer-hover-wrapper .offer-desc-item');
const sellerAnchor = Array.from(container.querySelectorAll('a'))
.find((link) => isSellerHref(link.href || ''));
const hoverPriceText = firstText(container, [
'.offer-hover-wrapper .hover-price-item',
'.offer-hover-wrapper .price-item',
]);
items.push({
item_url: href,
title: firstText(container, ['.offer-title-row .title-text', '.offer-title-row'])
|| normalizeText(anchor.innerText || anchor.textContent || ''),
container_text: normalizeText(container.innerText || container.textContent || ''),
desc_rows: collectTexts(container, '.offer-desc-row'),
price_text: firstText(container, ['.offer-price-row .price-item']),
sales_text: firstText(container, ['.offer-price-row .col-desc_after', '.offer-desc-row .col-desc_after']),
hover_price_text: hoverPriceText,
moq_text: findMoqText(hoverItems, hoverPriceText),
tag_items: tagItems,
hover_items: hoverItems,
seller_name: sellerAnchor ? normalizeText(sellerAnchor.innerText || sellerAnchor.textContent || '') : null,
seller_url: sellerAnchor ? sellerAnchor.href : null,
});
}
return items;
};
const findNextUrl = () => {
const selectors = [
'a.fui-next:not(.disabled)',
'a.next-pagination-item:not(.disabled)',
'a[rel="next"]:not(.disabled)',
'a[data-role="next"]:not(.disabled)',
];
for (const selector of selectors) {
const node = document.querySelector(selector);
if (!node) continue;
const href = normalizeUrl(node.getAttribute('href') || node.href || '');
if (href) return href;
}
const textBased = Array.from(document.querySelectorAll('a'))
.find((node) => /下一页|next/i.test(normalizeText(node.textContent || '')));
if (!textBased) return '';
return normalizeUrl(textBased.getAttribute('href') || textBased.href || '');
};
return {
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
next_url: findNextUrl(),
candidates: collectCandidates(),
};
})()
`);
if (!payload || typeof payload !== 'object') {
throw new CommandExecutionError('1688 search page did not return a readable payload', 'Open the same query in Chrome and verify the page is fully loaded before retrying.');
}
return payload;
}
async function collectSearchRows(page, query, limit) {
const rowsByKey = new Map();
const seenPages = new Set();
let nextUrl = buildSearchUrl(query);
let pageCount = 0;
while (nextUrl && rowsByKey.size < limit && pageCount < MAX_SEARCH_PAGES) {
if (seenPages.has(nextUrl))
break;
seenPages.add(nextUrl);
pageCount += 1;
const payload = await readSearchPayload(page, nextUrl);
const sourceUrl = cleanText(payload.href) || nextUrl;
const candidates = Array.isArray(payload.candidates) ? payload.candidates : [];
for (const candidate of candidates) {
const row = normalizeSearchCandidate(candidate, sourceUrl);
const dedupeKey = buildDedupeKey(row);
if (!dedupeKey || rowsByKey.has(dedupeKey))
continue;
rowsByKey.set(dedupeKey, row);
if (rowsByKey.size >= limit)
break;
}
const candidateNextUrl = cleanText(payload.next_url);
if (!candidateNextUrl || candidateNextUrl === sourceUrl)
break;
nextUrl = candidateNextUrl;
}
if (rowsByKey.size === 0) {
throw new EmptyResultError('1688 search', 'No visible results were extracted. Retry with a different query or open the same search page in Chrome first.');
}
return [...rowsByKey.values()]
.slice(0, limit)
.map((row, index) => ({ ...row, rank: index + 1 }));
}
cli({
site: '1688',
name: 'search',
description: '1688 商品搜索(结果候选、卖家链接、价格/MOQ/销量文本)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'query',
required: true,
positional: true,
help: '搜索关键词,如 "置物架"',
},
{
name: 'limit',
type: 'int',
default: SEARCH_LIMIT_DEFAULT,
help: `结果数量上限(默认 ${SEARCH_LIMIT_DEFAULT},最大 ${SEARCH_LIMIT_MAX}`,
},
],
columns: ['rank', 'title', 'price_text', 'moq_text', 'seller_name', 'location'],
func: async (page, kwargs) => {
const query = String(kwargs.query ?? '');
const limit = parseSearchLimit(kwargs.limit);
return collectSearchRows(page, query, limit);
},
});
export const __test__ = {
normalizeSearchCandidate,
extractMoqText,
extractSalesText,
firstWord,
buildDedupeKey,
};
+75
View File
@@ -0,0 +1,75 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './search.js';
describe('1688 search normalization', () => {
it('normalizes search candidates into structured result rows', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'https://detail.1688.com/offer/887904326744.html',
title: '宿舍置物架桌面加高架',
container_text: '宿舍置物架桌面加高架 ¥56.00 2套起批 山东青岛 已售300+套',
price_text: '¥ 56 .00',
sales_text: '300+套',
moq_text: '2套起批',
tag_items: ['退货包运费', '回头率52%'],
hover_items: ['验厂报告'],
seller_name: '青岛沁澜衣品服装有限公司',
seller_url: 'https://yinuoweierfushi.1688.com/page/index.html?spm=a123',
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=置物架');
expect(result.rank).toBe(0);
expect(result.offer_id).toBe('887904326744');
expect(result.shop_id).toBe('yinuoweierfushi');
expect(result.item_url).toBe('https://detail.1688.com/offer/887904326744.html');
expect(result.seller_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.price_text).toBe('¥56.00');
expect(result.price_min).toBe(56);
expect(result.price_max).toBe(56);
expect(result.moq_value).toBe(2);
expect(result.location).toBe('山东青岛');
expect(result.sales_text).toBe('300+套');
expect(result.badges).toEqual(expect.arrayContaining(['退货包运费', '验厂报告']));
expect(result.return_rate_text).toBe('回头率52%');
});
it('does not use hover_price_text as MOQ source', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'https://detail.1688.com/offer/887904326744.html',
title: 'test',
container_text: 'test ¥56.00',
price_text: '¥ 56 .00',
hover_price_text: '¥56.00 3件起批',
moq_text: null,
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=test');
// hover_price_text should not be used for MOQ extraction
expect(result.moq_text).toBeNull();
expect(result.moq_value).toBeNull();
});
it('extracts offer id from mobile detail search links', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'http://detail.m.1688.com/page/index.html?offerId=910933345396&sortType=&pageId=',
title: '',
container_text: '桌面书桌办公室工位收纳展示新中式博古架多层茶具厨房摆放置物架 ¥24.3 已售20+件',
price_text: '¥ 14 .28',
sales_text: '1500+件',
moq_text: '≥2个',
seller_name: '泰商国际贸易(宁阳)有限公司',
seller_url: 'http://tsgjmy.1688.com/',
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=桌面置物架');
expect(result.offer_id).toBe('910933345396');
expect(result.shop_id).toBe('tsgjmy');
expect(result.item_url).toBe('https://detail.1688.com/offer/910933345396.html');
expect(result.title).toContain('桌面书桌办公室工位收纳展示');
expect(result.price_text).toBe('¥14.28');
expect(result.sales_text).toBe('1500+件');
expect(result.moq_text).toBe('≥2个');
expect(result.moq_value).toBe(2);
});
it('prefers offer id and falls back to item url for dedupe key', () => {
expect(__test__.buildDedupeKey({
offer_id: '123456',
item_url: 'https://detail.1688.com/offer/123456.html',
})).toBe('offer:123456');
expect(__test__.buildDedupeKey({
offer_id: null,
item_url: 'https://detail.1688.com/offer/123456.html',
})).toBe('url:https://detail.1688.com/offer/123456.html');
expect(__test__.buildDedupeKey({ offer_id: null, item_url: null })).toBeNull();
});
});
-81
View File
@@ -1,81 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './search.js';
describe('1688 search normalization', () => {
it('normalizes search candidates into structured result rows', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'https://detail.1688.com/offer/887904326744.html',
title: '宿舍置物架桌面加高架',
container_text: '宿舍置物架桌面加高架 ¥56.00 2套起批 山东青岛 已售300+套',
price_text: '¥ 56 .00',
sales_text: '300+套',
moq_text: '2套起批',
tag_items: ['退货包运费', '回头率52%'],
hover_items: ['验厂报告'],
seller_name: '青岛沁澜衣品服装有限公司',
seller_url: 'https://yinuoweierfushi.1688.com/page/index.html?spm=a123',
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=置物架');
expect(result.rank).toBe(0);
expect(result.offer_id).toBe('887904326744');
expect(result.shop_id).toBe('yinuoweierfushi');
expect(result.item_url).toBe('https://detail.1688.com/offer/887904326744.html');
expect(result.seller_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.price_text).toBe('¥56.00');
expect(result.price_min).toBe(56);
expect(result.price_max).toBe(56);
expect(result.moq_value).toBe(2);
expect(result.location).toBe('山东青岛');
expect(result.sales_text).toBe('300+套');
expect(result.badges).toEqual(expect.arrayContaining(['退货包运费', '验厂报告']));
expect(result.return_rate_text).toBe('回头率52%');
});
it('does not use hover_price_text as MOQ source', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'https://detail.1688.com/offer/887904326744.html',
title: 'test',
container_text: 'test ¥56.00',
price_text: '¥ 56 .00',
hover_price_text: '¥56.00 3件起批',
moq_text: null,
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=test');
// hover_price_text should not be used for MOQ extraction
expect(result.moq_text).toBeNull();
expect(result.moq_value).toBeNull();
});
it('extracts offer id from mobile detail search links', () => {
const result = __test__.normalizeSearchCandidate({
item_url: 'http://detail.m.1688.com/page/index.html?offerId=910933345396&sortType=&pageId=',
title: '',
container_text: '桌面书桌办公室工位收纳展示新中式博古架多层茶具厨房摆放置物架 ¥24.3 已售20+件',
price_text: '¥ 14 .28',
sales_text: '1500+件',
moq_text: '≥2个',
seller_name: '泰商国际贸易(宁阳)有限公司',
seller_url: 'http://tsgjmy.1688.com/',
}, 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=桌面置物架');
expect(result.offer_id).toBe('910933345396');
expect(result.shop_id).toBe('tsgjmy');
expect(result.item_url).toBe('https://detail.1688.com/offer/910933345396.html');
expect(result.title).toContain('桌面书桌办公室工位收纳展示');
expect(result.price_text).toBe('¥14.28');
expect(result.sales_text).toBe('1500+件');
expect(result.moq_text).toBe('≥2个');
expect(result.moq_value).toBe(2);
});
it('prefers offer id and falls back to item url for dedupe key', () => {
expect(__test__.buildDedupeKey({
offer_id: '123456',
item_url: 'https://detail.1688.com/offer/123456.html',
})).toBe('offer:123456');
expect(__test__.buildDedupeKey({
offer_id: null,
item_url: 'https://detail.1688.com/offer/123456.html',
})).toBe('url:https://detail.1688.com/offer/123456.html');
expect(__test__.buildDedupeKey({ offer_id: null, item_url: null })).toBeNull();
});
});
-402
View File
@@ -1,402 +0,0 @@
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
FACTORY_BADGE_PATTERNS,
SERVICE_BADGE_PATTERNS,
assertAuthenticatedState,
buildProvenance,
buildSearchUrl,
canonicalizeItemUrl,
canonicalizeSellerUrl,
cleanText,
extractBadges,
extractLocation,
extractMemberId,
extractOfferId,
extractShopId,
gotoAndReadState,
parseMoqText,
parsePriceText,
SEARCH_LIMIT_DEFAULT,
SEARCH_LIMIT_MAX,
parseSearchLimit,
uniqueNonEmpty,
} from './shared.js';
interface SearchPayload {
href?: string;
title?: string;
bodyText?: string;
next_url?: string;
candidates?: Array<{
item_url?: string;
title?: string;
container_text?: string;
desc_rows?: string[];
price_text?: string | null;
sales_text?: string | null;
hover_price_text?: string | null;
moq_text?: string | null;
tag_items?: string[];
hover_items?: string[];
seller_name?: string | null;
seller_url?: string | null;
}>;
}
interface SearchRow {
rank: number;
offer_id: string | null;
member_id: string | null;
shop_id: string | null;
title: string | null;
item_url: string | null;
seller_name: string | null;
seller_url: string | null;
price_text: string | null;
price_min: number | null;
price_max: number | null;
currency: string | null;
moq_text: string | null;
moq_value: number | null;
location: string | null;
badges: string[];
sales_text: string | null;
return_rate_text: string | null;
source_url: string;
fetched_at: string;
strategy: string;
}
const SEARCH_ITEM_URL_PATTERNS = [
'detail.1688.com/offer/',
'detail.m.1688.com/page/index.html?offerId=',
];
const MAX_SEARCH_PAGES = 12;
function normalizeSearchCandidate(
candidate: NonNullable<SearchPayload['candidates']>[number],
sourceUrl: string,
): SearchRow {
const canonicalItemUrl = canonicalizeItemUrl(cleanText(candidate.item_url));
const containerText = cleanText(candidate.container_text);
const priceText = firstNonEmpty([
normalizeInlineText(candidate.price_text),
normalizeInlineText(extractPriceText(candidate.hover_price_text)),
]);
const priceRange = parsePriceText(priceText || containerText);
const moq = parseMoqText(firstNonEmpty([
normalizeInlineText(candidate.moq_text),
normalizeInlineText(extractMoqText(containerText)),
]));
const canonicalSellerUrl = canonicalizeSellerUrl(cleanText(candidate.seller_url));
const evidenceText = uniqueNonEmpty([
containerText,
...(candidate.desc_rows ?? []),
...(candidate.tag_items ?? []),
...(candidate.hover_items ?? []),
]).join('\n');
const badges = extractBadges(evidenceText, [...FACTORY_BADGE_PATTERNS, ...SERVICE_BADGE_PATTERNS]);
const salesText = firstNonEmpty([
extractSalesText(candidate.sales_text),
extractSalesText(containerText),
]);
const returnRateText = extractReturnRateText([...(candidate.tag_items ?? []), ...(candidate.hover_items ?? [])]);
const provenance = buildProvenance(sourceUrl);
return {
rank: 0,
offer_id: extractOfferId(canonicalItemUrl ?? '') ?? null,
member_id: extractMemberId(canonicalSellerUrl ?? '') ?? null,
shop_id: extractShopId(canonicalSellerUrl ?? '') ?? null,
title: cleanText(candidate.title) || firstWord(containerText) || null,
item_url: canonicalItemUrl,
seller_name: cleanText(candidate.seller_name) || null,
seller_url: canonicalSellerUrl,
price_text: priceRange.price_text || null,
price_min: priceRange.price_min,
price_max: priceRange.price_max,
currency: priceRange.currency,
moq_text: moq.moq_text || null,
moq_value: moq.moq_value,
location: extractLocation(containerText),
badges,
sales_text: salesText || null,
return_rate_text: returnRateText,
source_url: provenance.source_url,
fetched_at: provenance.fetched_at,
strategy: provenance.strategy,
};
}
function extractMoqText(text: string | null | undefined): string {
const normalized = normalizeInlineText(text);
return normalized.match(/\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)\s*起批/i)?.[0]
?? normalized.match(/≥\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)?/i)?.[0]
?? normalized.match(/\d+(?:\.\d+)?\s*(?:~|-|至|到)\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只)/i)?.[0]
?? '';
}
function extractPriceText(text: string | null | undefined): string {
const normalized = normalizeInlineText(text);
return normalized.match(/[¥$€]\s*\d+(?:\.\d+)?/)?.[0] ?? '';
}
function extractSalesText(text: string | null | undefined): string {
const normalized = normalizeInlineText(text);
if (!normalized) return '';
if (/^\d+(?:\.\d+)?\+?\s*(件|套|个|单)$/.test(normalized)) {
return normalized;
}
const match = normalized.match(/(?:已售|销量|售)\s*\d+(?:\.\d+)?\+?\s*(件|套|个|单)?/);
return match ? cleanText(match[0]) : '';
}
function firstWord(text: string): string {
return text.split(/\s+/).find(Boolean) ?? '';
}
function firstNonEmpty(values: Array<string | null | undefined>): string {
return values.map((value) => cleanText(value)).find(Boolean) ?? '';
}
function normalizeInlineText(text: string | null | undefined): string {
return cleanText(text)
.replace(/([¥$€])\s+(?=\d)/g, '$1')
.replace(/(\d)\s*\.\s*(\d)/g, '$1.$2')
.replace(/\s*([~-])\s*/g, '$1')
.trim();
}
function extractReturnRateText(values: string[]): string | null {
return uniqueNonEmpty(values.map((value) => normalizeInlineText(value)))
.find((value) => /^回头率\s*\d+(?:\.\d+)?%$/.test(value))
?? null;
}
function buildDedupeKey(row: Pick<SearchRow, 'offer_id' | 'item_url'>): string | null {
if (row.offer_id) return `offer:${row.offer_id}`;
if (row.item_url) return `url:${row.item_url}`;
return null;
}
async function readSearchPayload(page: IPage, url: string): Promise<SearchPayload> {
const state = await gotoAndReadState(page, url, 2500, 'search');
assertAuthenticatedState(state, 'search');
const payload = await page.evaluate(`
(() => {
const normalizeText = (value) => (value || '').replace(/\\s+/g, ' ').trim();
const normalizeUrl = (href) => {
if (!href) return '';
try {
return new URL(href, window.location.href).toString();
} catch {
return '';
}
};
const isItemHref = (href) => ${JSON.stringify(SEARCH_ITEM_URL_PATTERNS)}
.some((pattern) => (href || '').includes(pattern));
const uniqueTexts = (values) => [...new Set(values.map((value) => normalizeText(value)).filter(Boolean))];
const collectTexts = (root, selector) => uniqueTexts(
Array.from(root.querySelectorAll(selector)).map((node) => node.innerText || node.textContent || ''),
);
const firstText = (root, selectors) => {
for (const selector of selectors) {
const node = root.querySelector(selector);
const value = normalizeText(node ? node.innerText || node.textContent || '' : '');
if (value) return value;
}
return '';
};
const findMoqText = (values, priceText) => {
const moqPattern = /(≥\\s*\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只)?)|(\\d+(?:\\.\\d+)?\\s*(?:~|-|至|到)\\s*\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只))|(\\d+(?:\\.\\d+)?\\s*(件|个|套|箱|包|双|台|把|只)\\s*起批)/i;
return values.find((value) => moqPattern.test(value))
|| normalizeText(priceText).match(moqPattern)?.[0]
|| '';
};
const isSellerHref = (href) => {
if (!href) return false;
try {
const url = new URL(href, window.location.href);
const host = url.hostname || '';
if (!host.endsWith('.1688.com')) return false;
if (
host === 's.1688.com'
|| host === 'r.1688.com'
|| host === 'air.1688.com'
|| host === 'detail.1688.com'
|| host === 'detail.m.1688.com'
|| host === 'dj.1688.com'
) {
return false;
}
return true;
} catch {
return false;
}
};
const pickContainer = (anchor) => {
let node = anchor;
while (node && node !== document.body) {
const text = normalizeText(node.innerText || node.textContent || '');
if (text.length >= 40 && text.length <= 2000) {
return node;
}
node = node.parentElement;
}
return anchor;
};
const collectCandidates = () => {
const anchors = Array.from(document.querySelectorAll('a')).filter((anchor) => isItemHref(anchor.href || ''));
const seen = new Set();
const items = [];
for (const anchor of anchors) {
const href = anchor.href || '';
if (!href || seen.has(href)) continue;
seen.add(href);
const container = pickContainer(anchor);
const tagItems = collectTexts(container, '.offer-tag-row .offer-desc-item');
const hoverItems = collectTexts(container, '.offer-hover-wrapper .offer-desc-item');
const sellerAnchor = Array.from(container.querySelectorAll('a'))
.find((link) => isSellerHref(link.href || ''));
const hoverPriceText = firstText(container, [
'.offer-hover-wrapper .hover-price-item',
'.offer-hover-wrapper .price-item',
]);
items.push({
item_url: href,
title: firstText(container, ['.offer-title-row .title-text', '.offer-title-row'])
|| normalizeText(anchor.innerText || anchor.textContent || ''),
container_text: normalizeText(container.innerText || container.textContent || ''),
desc_rows: collectTexts(container, '.offer-desc-row'),
price_text: firstText(container, ['.offer-price-row .price-item']),
sales_text: firstText(container, ['.offer-price-row .col-desc_after', '.offer-desc-row .col-desc_after']),
hover_price_text: hoverPriceText,
moq_text: findMoqText(hoverItems, hoverPriceText),
tag_items: tagItems,
hover_items: hoverItems,
seller_name: sellerAnchor ? normalizeText(sellerAnchor.innerText || sellerAnchor.textContent || '') : null,
seller_url: sellerAnchor ? sellerAnchor.href : null,
});
}
return items;
};
const findNextUrl = () => {
const selectors = [
'a.fui-next:not(.disabled)',
'a.next-pagination-item:not(.disabled)',
'a[rel="next"]:not(.disabled)',
'a[data-role="next"]:not(.disabled)',
];
for (const selector of selectors) {
const node = document.querySelector(selector);
if (!node) continue;
const href = normalizeUrl(node.getAttribute('href') || node.href || '');
if (href) return href;
}
const textBased = Array.from(document.querySelectorAll('a'))
.find((node) => /下一页|next/i.test(normalizeText(node.textContent || '')));
if (!textBased) return '';
return normalizeUrl(textBased.getAttribute('href') || textBased.href || '');
};
return {
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
next_url: findNextUrl(),
candidates: collectCandidates(),
};
})()
`) as SearchPayload;
if (!payload || typeof payload !== 'object') {
throw new CommandExecutionError(
'1688 search page did not return a readable payload',
'Open the same query in Chrome and verify the page is fully loaded before retrying.',
);
}
return payload;
}
async function collectSearchRows(page: IPage, query: string, limit: number): Promise<SearchRow[]> {
const rowsByKey = new Map<string, SearchRow>();
const seenPages = new Set<string>();
let nextUrl = buildSearchUrl(query);
let pageCount = 0;
while (nextUrl && rowsByKey.size < limit && pageCount < MAX_SEARCH_PAGES) {
if (seenPages.has(nextUrl)) break;
seenPages.add(nextUrl);
pageCount += 1;
const payload = await readSearchPayload(page, nextUrl);
const sourceUrl = cleanText(payload.href) || nextUrl;
const candidates = Array.isArray(payload.candidates) ? payload.candidates : [];
for (const candidate of candidates) {
const row = normalizeSearchCandidate(candidate, sourceUrl);
const dedupeKey = buildDedupeKey(row);
if (!dedupeKey || rowsByKey.has(dedupeKey)) continue;
rowsByKey.set(dedupeKey, row);
if (rowsByKey.size >= limit) break;
}
const candidateNextUrl = cleanText(payload.next_url);
if (!candidateNextUrl || candidateNextUrl === sourceUrl) break;
nextUrl = candidateNextUrl;
}
if (rowsByKey.size === 0) {
throw new EmptyResultError(
'1688 search',
'No visible results were extracted. Retry with a different query or open the same search page in Chrome first.',
);
}
return [...rowsByKey.values()]
.slice(0, limit)
.map((row, index) => ({ ...row, rank: index + 1 }));
}
cli({
site: '1688',
name: 'search',
description: '1688 商品搜索(结果候选、卖家链接、价格/MOQ/销量文本)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'query',
required: true,
positional: true,
help: '搜索关键词,如 "置物架"',
},
{
name: 'limit',
type: 'int',
default: SEARCH_LIMIT_DEFAULT,
help: `结果数量上限(默认 ${SEARCH_LIMIT_DEFAULT},最大 ${SEARCH_LIMIT_MAX}`,
},
],
columns: ['rank', 'title', 'price_text', 'moq_text', 'seller_name', 'location'],
func: async (page, kwargs) => {
const query = String(kwargs.query ?? '');
const limit = parseSearchLimit(kwargs.limit);
return collectSearchRows(page, query, limit);
},
});
export const __test__ = {
normalizeSearchCandidate,
extractMoqText,
extractSalesText,
firstWord,
buildDedupeKey,
};
+557
View File
@@ -0,0 +1,557 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
export const SITE = '1688';
export const HOME_URL = 'https://www.1688.com/';
export const SEARCH_URL_PREFIX = 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=';
export const DETAIL_URL_PREFIX = 'https://detail.1688.com/offer/';
export const STORE_MOBILE_URL_PREFIX = 'https://winport.m.1688.com/page/index.html?memberId=';
export const STRATEGY = 'cookie';
export const SEARCH_LIMIT_DEFAULT = 20;
export const SEARCH_LIMIT_MAX = 100;
const STORE_GENERIC_HOSTS = new Set(['www', 'detail', 's', 'winport', 'work', 'air', 'dj']);
const TRACKING_QUERY_KEYS = new Set([
'spm',
'tracelog',
'clickid',
'source',
'scene',
'from',
'src',
'ns',
'cna',
'pvid',
]);
const CAPTCHA_URL_MARKER = '/_____tmd_____/punish';
const CAPTCHA_TEXT_PATTERNS = [
'请拖动下方滑块完成验证',
'请按住滑块,拖动到最右边',
'通过验证以确保正常访问',
'验证码拦截',
'访问验证',
'滑动验证',
];
const LOGIN_TEXT_PATTERNS = [
'请登录',
'登录后',
'账号登录',
'手机登录',
'立即登录',
'扫码登录',
'请先完成登录',
'请先登录后查看',
];
const LOGIN_URL_PATTERNS = ['/member/login', 'passport', 'login.taobao.com', 'account.1688.com'];
export const FACTORY_BADGE_PATTERNS = [
'源头工厂',
'深度验厂',
'实力工厂',
'工厂档案',
'加工专区',
'验厂报告',
'厂家直销',
'生产厂家',
'工厂直供',
];
export const SERVICE_BADGE_PATTERNS = [
'延期必赔',
'品质保障',
'破损包赔',
'退货包运费',
'晚发必赔',
'7*24小时响应',
'48小时发货',
'72小时发货',
'后天达',
'包邮',
'闪电拿样',
];
const CHINA_LOCATIONS = [
'北京',
'天津',
'上海',
'重庆',
'河北',
'山西',
'辽宁',
'吉林',
'黑龙江',
'江苏',
'浙江',
'安徽',
'福建',
'江西',
'山东',
'河南',
'湖北',
'湖南',
'广东',
'海南',
'四川',
'贵州',
'云南',
'陕西',
'甘肃',
'青海',
'台湾',
'内蒙古',
'广西',
'西藏',
'宁夏',
'新疆',
'香港',
'澳门',
];
export function cleanText(value) {
return typeof value === 'string'
? value.replace(/\u00a0/g, ' ').replace(/\s+/g, ' ').trim()
: '';
}
export function cleanMultilineText(value) {
return typeof value === 'string'
? value
.replace(/\u00a0/g, ' ')
.split('\n')
.map((line) => line.replace(/\s+/g, ' ').trim())
.filter(Boolean)
.join('\n')
: '';
}
export function uniqueNonEmpty(values) {
return [...new Set(values.map((value) => cleanText(value)).filter(Boolean))];
}
export function parseSearchLimit(input) {
const parsed = Number.parseInt(String(input ?? SEARCH_LIMIT_DEFAULT), 10);
if (!Number.isFinite(parsed) || parsed < 1) {
throw new ArgumentError('1688 search --limit must be a positive integer', 'Example: opencli 1688 search "桌面置物架" --limit 20');
}
return Math.min(SEARCH_LIMIT_MAX, parsed);
}
export function buildSearchUrl(query) {
const normalized = cleanText(query);
if (!normalized) {
throw new ArgumentError('1688 search query cannot be empty', 'Example: opencli 1688 search "桌面置物架" --limit 20');
}
return `${SEARCH_URL_PREFIX}${encodeURIComponent(normalized)}`;
}
export function buildDetailUrl(input) {
const offerId = extractOfferId(input);
if (!offerId) {
throw new ArgumentError('1688 item expects an offer URL or offer ID', 'Example: opencli 1688 item 887904326744');
}
return `${DETAIL_URL_PREFIX}${offerId}.html`;
}
export function resolveStoreUrl(input) {
const normalized = cleanText(input);
if (!normalized) {
throw new ArgumentError('1688 store expects a store URL or member ID', 'Example: opencli 1688 store https://yinuoweierfushi.1688.com/');
}
const memberId = extractMemberId(normalized);
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
if (/^https?:\/\//i.test(normalized)) {
return canonicalizeStoreUrl(normalized);
}
if (normalized.endsWith('.1688.com')) {
return canonicalizeStoreUrl(`https://${normalized}`);
}
if (/^[a-z0-9-]+$/i.test(normalized)) {
return canonicalizeStoreUrl(`https://${normalized}.1688.com`);
}
throw new ArgumentError('1688 store expects a store URL or member ID', 'Example: opencli 1688 store b2b-22154705262941f196');
}
export function canonicalizeStoreUrl(input) {
const url = parse1688Url(input);
const memberId = extractMemberId(url.toString());
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
const host = normalizeStoreHost(url.hostname);
if (!host) {
throw new ArgumentError('Invalid 1688 store URL', 'Example: opencli 1688 store https://yinuoweierfushi.1688.com/');
}
return `https://${host}`;
}
export function canonicalizeItemUrl(input) {
const offerId = extractOfferId(input);
if (offerId) {
return `${DETAIL_URL_PREFIX}${offerId}.html`;
}
const url = parse1688UrlOrNull(input);
if (!url)
return null;
stripTrackingParams(url);
url.hash = '';
return url.toString();
}
export function canonicalizeSellerUrl(input) {
const memberId = extractMemberId(input);
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
const url = parse1688UrlOrNull(input);
if (!url)
return null;
const host = normalizeStoreHost(url.hostname);
if (!host)
return null;
return `https://${host}`;
}
export function extractOfferId(input) {
const normalized = cleanText(input);
if (!normalized)
return null;
const directId = normalized.match(/^\d{6,}$/)?.[0];
if (directId)
return directId;
const detailMatch = normalized.match(/\/offer\/(\d{6,})\.html/i);
if (detailMatch)
return detailMatch[1];
const queryMatch = normalized.match(/[?&]offerId=(\d{6,})/i);
if (queryMatch)
return queryMatch[1];
return null;
}
export function extractMemberId(input) {
const normalized = cleanText(input);
if (!normalized)
return null;
const direct = normalized.match(/\bb2b-[a-z0-9]+\b/i)?.[0];
if (direct)
return direct;
const queryMatch = normalized.match(/[?&]memberId=(b2b-[a-z0-9]+)/i);
if (queryMatch)
return queryMatch[1];
const mobileMatch = normalized.match(/\/winport\/(b2b-[a-z0-9]+)\.html/i);
if (mobileMatch)
return mobileMatch[1];
return null;
}
export function extractShopId(input) {
const normalized = cleanText(input);
if (!normalized)
return null;
try {
const url = new URL(/^https?:\/\//i.test(normalized) ? normalized : `https://${normalized}`);
const host = normalizeStoreHost(url.hostname);
if (!host)
return null;
return host.split('.')[0] ?? null;
}
catch {
return /^[a-z0-9-]+$/i.test(normalized) ? normalized : null;
}
}
export function buildProvenance(sourceUrl) {
return {
source_url: sourceUrl,
fetched_at: new Date().toISOString(),
strategy: STRATEGY,
};
}
export function parsePriceText(text) {
const normalized = normalizeNumericText(cleanText(text));
const matches = normalized.match(/\d+(?:,\d{3})*(?:\.\d+)?/g) ?? [];
const values = matches
.map((value) => Number.parseFloat(value.replace(/,/g, '')))
.filter((value) => Number.isFinite(value));
if (values.length === 0) {
return {
price_text: normalized,
price_min: null,
price_max: null,
currency: null,
};
}
return {
price_text: normalized,
price_min: values[0] ?? null,
price_max: values[values.length - 1] ?? values[0] ?? null,
currency: normalized.includes('¥') || normalized.includes('元') ? 'CNY' : null,
};
}
export function normalizePriceTiers(rawTiers, unit) {
return rawTiers
.map((tier) => {
const quantityMin = toNumber(tier.beginAmount);
const priceText = cleanText(tier.price);
const price = toNumber(tier.price);
return {
quantity_text: quantityMin !== null ? `${quantityMin}${unit ?? ''}` : '',
quantity_min: quantityMin,
price_text: priceText,
price,
currency: priceText ? 'CNY' : null,
};
})
.filter((tier) => tier.price_text);
}
export function parseMoqText(text) {
const normalized = normalizeNumericText(cleanText(text));
const match = normalized.match(/(\d+(?:\.\d+)?)\s*(件|个|套|箱|包|双|台|把|只|pcs|piece|pieces)?\s*起批/i)
?? normalized.match(/≥\s*(\d+(?:\.\d+)?)/);
const rangeMatch = normalized.match(/(\d+(?:\.\d+)?)\s*(?:~|-|至|到)\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只|pcs|piece|pieces)/i);
if (!match && !rangeMatch) {
return {
moq_text: normalized,
moq_value: null,
};
}
return {
moq_text: normalized,
moq_value: Number.parseFloat((match ?? rangeMatch)[1]),
};
}
export function extractLocation(text) {
const normalized = cleanMultilineText(text);
const primaryRegion = normalized.split(/送至|发往/)[0] ?? normalized;
const lines = primaryRegion.split('\n');
for (const line of lines) {
const compact = cleanText(line);
if (!compact || compact.length > 16)
continue;
if (CHINA_LOCATIONS.some((location) => compact.startsWith(location))) {
return compact;
}
}
const locationPattern = new RegExp(`(${CHINA_LOCATIONS.join('|')})[\\u4e00-\\u9fa5]{0,8}`);
return primaryRegion.match(locationPattern)?.[0] ?? null;
}
export function extractAddress(text) {
const normalized = cleanMultilineText(text);
const lineMatch = normalized.match(/地址[:]\s*([^\n]+)/);
if (lineMatch)
return cleanText(lineMatch[1]);
return normalized
.split('\n')
.map((line) => cleanText(line))
.find((line) => line.includes('省') || line.includes('市') || line.includes('区') || line.includes('县'))
?? null;
}
export function extractMetric(text, label) {
const normalized = cleanMultilineText(text);
const direct = normalized.match(new RegExp(`(?:^|\\n)\\s*${escapeForRegex(label)}[:]?\\s*([^\\n]+)`));
if (direct)
return cleanText(direct[1]);
const lineBased = normalized.match(new RegExp(`(?:^|\\n)\\s*${escapeForRegex(label)}\\n([^\\n]+)`));
return lineBased ? cleanText(lineBased[1]) : null;
}
export function extractYearsOnPlatform(text) {
return text.match(/入驻\d+年/)?.[0] ?? null;
}
export function extractMainBusiness(text) {
const value = extractMetric(text, '主营');
return value ? value.replace(/^/, '').trim() : null;
}
export function extractBadges(text, candidates) {
return uniqueNonEmpty(candidates.filter((candidate) => cleanMultilineText(text).includes(candidate)));
}
export function guessTopCategories(text) {
const mainBusiness = extractMainBusiness(text);
if (!mainBusiness)
return [];
return uniqueNonEmpty(mainBusiness.split(/[、,/|]/).map((value) => value.trim()));
}
export function isCaptchaState(state) {
const href = cleanText(state.href).toLowerCase();
const title = cleanText(state.title);
const bodyText = cleanMultilineText(state.body_text);
if (href.includes(CAPTCHA_URL_MARKER))
return true;
return CAPTCHA_TEXT_PATTERNS.some((pattern) => title.includes(pattern) || bodyText.includes(pattern));
}
export function isLoginState(state) {
const href = cleanText(state.href).toLowerCase();
const title = cleanText(state.title);
const bodyText = cleanMultilineText(state.body_text);
if (LOGIN_URL_PATTERNS.some((pattern) => href.includes(pattern)))
return true;
return LOGIN_TEXT_PATTERNS.some((pattern) => title.includes(pattern) || bodyText.includes(pattern));
}
export function buildCaptchaHint(action) {
return [
`Open a clean 1688 ${action} page in the shared Chrome profile and finish any slider challenge first.`,
'If you run opencli via CDP, set OPENCLI_CDP_TARGET=1688.com or a more specific 1688 host before retrying.',
].join(' ');
}
export async function readPageState(page) {
const result = await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
body_text: document.body ? document.body.innerText || '' : '',
}))()
`);
return {
href: cleanText(result.href),
title: cleanText(result.title),
body_text: cleanMultilineText(result.body_text),
};
}
export async function gotoAndReadState(page, url, settleMs = 2500, action = 'page') {
try {
await page.goto(url, { settleMs });
await page.wait(1.5);
return readPageState(page);
}
catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (message.includes('Inspected target navigated or closed')
|| message.includes('Cannot find context with specified id')
|| message.includes('Target closed')) {
throw new CommandExecutionError(`1688 ${action} navigation lost the current browser target`, `${buildCaptchaHint(action)} If CDP is attached to a stale or blocked tab, open a fresh 1688 tab and point OPENCLI_CDP_TARGET at that tab.`);
}
throw error;
}
}
export async function ensure1688Session(page) {
const state = await gotoAndReadState(page, HOME_URL, 1500, 'homepage');
assertAuthenticatedState(state, 'homepage');
}
export function assertAuthenticatedState(state, action) {
if (!isCaptchaState(state) && !isLoginState(state))
return;
throw new AuthRequiredError('1688.com', `请先在共享 Chrome 完成 1688 登录/验证,再重试(${action}`);
}
export function assertNotCaptcha(state, action) {
assertAuthenticatedState(state, action);
}
export function toNumber(value) {
if (typeof value === 'number' && Number.isFinite(value)) {
return value;
}
if (typeof value === 'string') {
const normalized = value.replace(/,/g, '').trim();
if (!normalized)
return null;
const parsed = Number.parseFloat(normalized);
return Number.isFinite(parsed) ? parsed : null;
}
return null;
}
export function limitCandidates(values, limit) {
const normalizedLimit = Math.max(1, Math.trunc(limit) || 1);
return values.slice(0, normalizedLimit);
}
export function normalizeMediaUrl(input) {
const raw = cleanText(input);
if (!raw)
return '';
let value = raw
.replace(/^url\((.*)\)$/i, '$1')
.replace(/^['"]|['"]$/g, '')
.replace(/\\u002F/g, '/')
.replace(/&amp;/g, '&')
.trim();
if (!value || value.startsWith('data:') || value.startsWith('blob:'))
return '';
if (value.startsWith('//'))
value = `https:${value}`;
try {
const url = new URL(value);
return url.toString();
}
catch {
return '';
}
}
export function uniqueMediaSources(values) {
const seen = new Set();
const result = [];
for (const value of values) {
const url = normalizeMediaUrl(value.url);
if (!url)
continue;
const key = `${value.type}:${url}`;
if (seen.has(key))
continue;
seen.add(key);
result.push({
...value,
url,
source: cleanText(value.source) || undefined,
});
}
return result;
}
function normalizeNumericText(value) {
return value
.replace(/([¥$€])\s+(?=\d)/g, '$1')
.replace(/(\d)\s*\.\s*(\d)/g, '$1.$2')
.replace(/\s*([~-])\s*/g, '$1')
.trim();
}
function escapeForRegex(value) {
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
function parse1688Url(input) {
const normalized = cleanText(input);
try {
const url = new URL(normalized);
if (!url.hostname.endsWith('.1688.com') && url.hostname !== '1688.com' && url.hostname !== 'www.1688.com') {
throw new Error('invalid-host');
}
stripTrackingParams(url);
url.hash = '';
return url;
}
catch {
throw new ArgumentError('Invalid 1688 URL', 'Use a URL under 1688.com (for example: https://detail.1688.com/offer/887904326744.html)');
}
}
function parse1688UrlOrNull(input) {
try {
return parse1688Url(input);
}
catch {
return null;
}
}
function normalizeStoreHost(hostname) {
const lower = cleanText(hostname).toLowerCase();
if (!lower.endsWith('.1688.com'))
return null;
const [subdomain] = lower.split('.');
if (!subdomain || STORE_GENERIC_HOSTS.has(subdomain))
return null;
return lower;
}
function stripTrackingParams(url) {
const keys = [...url.searchParams.keys()];
for (const key of keys) {
if (TRACKING_QUERY_KEYS.has(key) || key.toLowerCase().startsWith('utm_')) {
url.searchParams.delete(key);
}
}
}
export const __test__ = {
SEARCH_LIMIT_DEFAULT,
SEARCH_LIMIT_MAX,
parseSearchLimit,
buildSearchUrl,
buildDetailUrl,
resolveStoreUrl,
canonicalizeStoreUrl,
canonicalizeItemUrl,
canonicalizeSellerUrl,
extractOfferId,
extractMemberId,
extractShopId,
parsePriceText,
normalizePriceTiers,
parseMoqText,
extractLocation,
extractAddress,
extractMetric,
extractYearsOnPlatform,
extractMainBusiness,
extractBadges,
guessTopCategories,
isCaptchaState,
isLoginState,
cleanText,
cleanMultilineText,
uniqueNonEmpty,
normalizeMediaUrl,
uniqueMediaSources,
limitCandidates,
};
+57
View File
@@ -0,0 +1,57 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './shared.js';
describe('1688 shared helpers', () => {
it('builds encoded search URLs and validates limit', () => {
expect(__test__.buildSearchUrl('置物架')).toBe('https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=%E7%BD%AE%E7%89%A9%E6%9E%B6');
expect(() => __test__.buildSearchUrl(' ')).toThrowError(/cannot be empty/i);
expect(__test__.parseSearchLimit(3)).toBe(3);
expect(__test__.parseSearchLimit('1000')).toBe(__test__.SEARCH_LIMIT_MAX);
expect(() => __test__.parseSearchLimit('0')).toThrowError(/positive integer/i);
});
it('extracts IDs and canonicalizes urls', () => {
expect(__test__.extractOfferId('887904326744')).toBe('887904326744');
expect(__test__.extractOfferId('https://detail.1688.com/offer/887904326744.html')).toBe('887904326744');
expect(__test__.extractMemberId('https://winport.m.1688.com/page/index.html?memberId=b2b-1641351767')).toBe('b2b-1641351767');
expect(__test__.extractMemberId('b2b-22154705262941f196')).toBe('b2b-22154705262941f196');
expect(__test__.resolveStoreUrl('b2b-22154705262941f196')).toBe('https://winport.m.1688.com/page/index.html?memberId=b2b-22154705262941f196');
expect(__test__.canonicalizeStoreUrl('https://yinuoweierfushi.1688.com/page/index.html?spm=foo')).toBe('https://yinuoweierfushi.1688.com');
expect(__test__.canonicalizeItemUrl('http://detail.m.1688.com/page/index.html?offerId=910933345396&spm=x')).toBe('https://detail.1688.com/offer/910933345396.html');
expect(__test__.canonicalizeSellerUrl('https://yinuoweierfushi.1688.com/page/contactinfo.html?tracelog=1')).toBe('https://yinuoweierfushi.1688.com');
expect(__test__.extractShopId('https://yinuoweierfushi.1688.com/page/index.html')).toBe('yinuoweierfushi');
});
it('parses price ranges and moq text', () => {
expect(__test__.parsePriceText('¥96.00-98.00')).toEqual({
price_text: '¥96.00-98.00',
price_min: 96,
price_max: 98,
currency: 'CNY',
});
expect(__test__.parsePriceText('¥ 14 .28')).toEqual({
price_text: '¥14.28',
price_min: 14.28,
price_max: 14.28,
currency: 'CNY',
});
expect(__test__.parseMoqText('3套起批')).toEqual({
moq_text: '3套起批',
moq_value: 3,
});
expect(__test__.parseMoqText('2~999个')).toEqual({
moq_text: '2~999个',
moq_value: 2,
});
});
it('detects captcha and login states', () => {
expect(__test__.extractLocation('山东青岛 送至 江苏苏州')).toBe('山东青岛');
expect(__test__.isCaptchaState({
href: 'https://s.1688.com/_____tmd_____/punish',
title: '验证码拦截',
body_text: '请拖动下方滑块完成验证',
})).toBe(true);
expect(__test__.isLoginState({
href: 'https://login.taobao.com/member/login.jhtml',
title: '账号登录',
body_text: '请登录后继续',
})).toBe(true);
});
});
-75
View File
@@ -1,75 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './shared.js';
describe('1688 shared helpers', () => {
it('builds encoded search URLs and validates limit', () => {
expect(__test__.buildSearchUrl('置物架')).toBe(
'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=%E7%BD%AE%E7%89%A9%E6%9E%B6',
);
expect(() => __test__.buildSearchUrl(' ')).toThrowError(/cannot be empty/i);
expect(__test__.parseSearchLimit(3)).toBe(3);
expect(__test__.parseSearchLimit('1000')).toBe(__test__.SEARCH_LIMIT_MAX);
expect(() => __test__.parseSearchLimit('0')).toThrowError(/positive integer/i);
});
it('extracts IDs and canonicalizes urls', () => {
expect(__test__.extractOfferId('887904326744')).toBe('887904326744');
expect(__test__.extractOfferId('https://detail.1688.com/offer/887904326744.html')).toBe('887904326744');
expect(__test__.extractMemberId('https://winport.m.1688.com/page/index.html?memberId=b2b-1641351767')).toBe('b2b-1641351767');
expect(__test__.extractMemberId('b2b-22154705262941f196')).toBe('b2b-22154705262941f196');
expect(__test__.resolveStoreUrl('b2b-22154705262941f196')).toBe(
'https://winport.m.1688.com/page/index.html?memberId=b2b-22154705262941f196',
);
expect(__test__.canonicalizeStoreUrl('https://yinuoweierfushi.1688.com/page/index.html?spm=foo')).toBe(
'https://yinuoweierfushi.1688.com',
);
expect(__test__.canonicalizeItemUrl('http://detail.m.1688.com/page/index.html?offerId=910933345396&spm=x')).toBe(
'https://detail.1688.com/offer/910933345396.html',
);
expect(__test__.canonicalizeSellerUrl('https://yinuoweierfushi.1688.com/page/contactinfo.html?tracelog=1')).toBe(
'https://yinuoweierfushi.1688.com',
);
expect(__test__.extractShopId('https://yinuoweierfushi.1688.com/page/index.html')).toBe('yinuoweierfushi');
});
it('parses price ranges and moq text', () => {
expect(__test__.parsePriceText('¥96.00-98.00')).toEqual({
price_text: '¥96.00-98.00',
price_min: 96,
price_max: 98,
currency: 'CNY',
});
expect(__test__.parsePriceText('¥ 14 .28')).toEqual({
price_text: '¥14.28',
price_min: 14.28,
price_max: 14.28,
currency: 'CNY',
});
expect(__test__.parseMoqText('3套起批')).toEqual({
moq_text: '3套起批',
moq_value: 3,
});
expect(__test__.parseMoqText('2~999个')).toEqual({
moq_text: '2~999个',
moq_value: 2,
});
});
it('detects captcha and login states', () => {
expect(__test__.extractLocation('山东青岛 送至 江苏苏州')).toBe('山东青岛');
expect(__test__.isCaptchaState({
href: 'https://s.1688.com/_____tmd_____/punish',
title: '验证码拦截',
body_text: '请拖动下方滑块完成验证',
})).toBe(true);
expect(__test__.isLoginState({
href: 'https://login.taobao.com/member/login.jhtml',
title: '账号登录',
body_text: '请登录后继续',
})).toBe(true);
});
});
-672
View File
@@ -1,672 +0,0 @@
import { ArgumentError, AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import type { IPage } from '@jackwener/opencli/types';
export const SITE = '1688';
export const HOME_URL = 'https://www.1688.com/';
export const SEARCH_URL_PREFIX = 'https://s.1688.com/selloffer/offer_search.htm?charset=utf8&keywords=';
export const DETAIL_URL_PREFIX = 'https://detail.1688.com/offer/';
export const STORE_MOBILE_URL_PREFIX = 'https://winport.m.1688.com/page/index.html?memberId=';
export const STRATEGY = 'cookie';
export const SEARCH_LIMIT_DEFAULT = 20;
export const SEARCH_LIMIT_MAX = 100;
const STORE_GENERIC_HOSTS = new Set(['www', 'detail', 's', 'winport', 'work', 'air', 'dj']);
const TRACKING_QUERY_KEYS = new Set([
'spm',
'tracelog',
'clickid',
'source',
'scene',
'from',
'src',
'ns',
'cna',
'pvid',
]);
const CAPTCHA_URL_MARKER = '/_____tmd_____/punish';
const CAPTCHA_TEXT_PATTERNS = [
'请拖动下方滑块完成验证',
'请按住滑块,拖动到最右边',
'通过验证以确保正常访问',
'验证码拦截',
'访问验证',
'滑动验证',
];
const LOGIN_TEXT_PATTERNS = [
'请登录',
'登录后',
'账号登录',
'手机登录',
'立即登录',
'扫码登录',
'请先完成登录',
'请先登录后查看',
];
const LOGIN_URL_PATTERNS = ['/member/login', 'passport', 'login.taobao.com', 'account.1688.com'];
export const FACTORY_BADGE_PATTERNS = [
'源头工厂',
'深度验厂',
'实力工厂',
'工厂档案',
'加工专区',
'验厂报告',
'厂家直销',
'生产厂家',
'工厂直供',
];
export const SERVICE_BADGE_PATTERNS = [
'延期必赔',
'品质保障',
'破损包赔',
'退货包运费',
'晚发必赔',
'7*24小时响应',
'48小时发货',
'72小时发货',
'后天达',
'包邮',
'闪电拿样',
];
const CHINA_LOCATIONS = [
'北京',
'天津',
'上海',
'重庆',
'河北',
'山西',
'辽宁',
'吉林',
'黑龙江',
'江苏',
'浙江',
'安徽',
'福建',
'江西',
'山东',
'河南',
'湖北',
'湖南',
'广东',
'海南',
'四川',
'贵州',
'云南',
'陕西',
'甘肃',
'青海',
'台湾',
'内蒙古',
'广西',
'西藏',
'宁夏',
'新疆',
'香港',
'澳门',
];
export interface ProvenanceFields {
source_url: string;
fetched_at: string;
strategy: string;
}
export interface PageState {
href: string;
title: string;
body_text: string;
}
export interface PriceRange {
price_text: string;
price_min: number | null;
price_max: number | null;
currency: string | null;
}
export interface MoqValue {
moq_text: string;
moq_value: number | null;
}
export interface PriceTier {
quantity_text: string;
quantity_min: number | null;
price_text: string;
price: number | null;
currency: string | null;
}
export interface SearchCandidate {
item_url: string;
title: string;
container_text: string;
seller_name: string | null;
seller_url: string | null;
}
export interface MediaSource {
type: 'image' | 'video';
group: 'main' | 'sku' | 'detail' | 'video' | 'unknown';
url: string;
source?: string;
}
export function cleanText(value: unknown): string {
return typeof value === 'string'
? value.replace(/\u00a0/g, ' ').replace(/\s+/g, ' ').trim()
: '';
}
export function cleanMultilineText(value: unknown): string {
return typeof value === 'string'
? value
.replace(/\u00a0/g, ' ')
.split('\n')
.map((line) => line.replace(/\s+/g, ' ').trim())
.filter(Boolean)
.join('\n')
: '';
}
export function uniqueNonEmpty(values: Array<string | null | undefined>): string[] {
return [...new Set(values.map((value) => cleanText(value)).filter(Boolean))];
}
export function parseSearchLimit(input: unknown): number {
const parsed = Number.parseInt(String(input ?? SEARCH_LIMIT_DEFAULT), 10);
if (!Number.isFinite(parsed) || parsed < 1) {
throw new ArgumentError(
'1688 search --limit must be a positive integer',
'Example: opencli 1688 search "桌面置物架" --limit 20',
);
}
return Math.min(SEARCH_LIMIT_MAX, parsed);
}
export function buildSearchUrl(query: string): string {
const normalized = cleanText(query);
if (!normalized) {
throw new ArgumentError(
'1688 search query cannot be empty',
'Example: opencli 1688 search "桌面置物架" --limit 20',
);
}
return `${SEARCH_URL_PREFIX}${encodeURIComponent(normalized)}`;
}
export function buildDetailUrl(input: string): string {
const offerId = extractOfferId(input);
if (!offerId) {
throw new ArgumentError(
'1688 item expects an offer URL or offer ID',
'Example: opencli 1688 item 887904326744',
);
}
return `${DETAIL_URL_PREFIX}${offerId}.html`;
}
export function resolveStoreUrl(input: string): string {
const normalized = cleanText(input);
if (!normalized) {
throw new ArgumentError(
'1688 store expects a store URL or member ID',
'Example: opencli 1688 store https://yinuoweierfushi.1688.com/',
);
}
const memberId = extractMemberId(normalized);
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
if (/^https?:\/\//i.test(normalized)) {
return canonicalizeStoreUrl(normalized);
}
if (normalized.endsWith('.1688.com')) {
return canonicalizeStoreUrl(`https://${normalized}`);
}
if (/^[a-z0-9-]+$/i.test(normalized)) {
return canonicalizeStoreUrl(`https://${normalized}.1688.com`);
}
throw new ArgumentError(
'1688 store expects a store URL or member ID',
'Example: opencli 1688 store b2b-22154705262941f196',
);
}
export function canonicalizeStoreUrl(input: string): string {
const url = parse1688Url(input);
const memberId = extractMemberId(url.toString());
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
const host = normalizeStoreHost(url.hostname);
if (!host) {
throw new ArgumentError(
'Invalid 1688 store URL',
'Example: opencli 1688 store https://yinuoweierfushi.1688.com/',
);
}
return `https://${host}`;
}
export function canonicalizeItemUrl(input: string): string | null {
const offerId = extractOfferId(input);
if (offerId) {
return `${DETAIL_URL_PREFIX}${offerId}.html`;
}
const url = parse1688UrlOrNull(input);
if (!url) return null;
stripTrackingParams(url);
url.hash = '';
return url.toString();
}
export function canonicalizeSellerUrl(input: string): string | null {
const memberId = extractMemberId(input);
if (memberId) {
return `${STORE_MOBILE_URL_PREFIX}${memberId}`;
}
const url = parse1688UrlOrNull(input);
if (!url) return null;
const host = normalizeStoreHost(url.hostname);
if (!host) return null;
return `https://${host}`;
}
export function extractOfferId(input: string): string | null {
const normalized = cleanText(input);
if (!normalized) return null;
const directId = normalized.match(/^\d{6,}$/)?.[0];
if (directId) return directId;
const detailMatch = normalized.match(/\/offer\/(\d{6,})\.html/i);
if (detailMatch) return detailMatch[1];
const queryMatch = normalized.match(/[?&]offerId=(\d{6,})/i);
if (queryMatch) return queryMatch[1];
return null;
}
export function extractMemberId(input: string): string | null {
const normalized = cleanText(input);
if (!normalized) return null;
const direct = normalized.match(/\bb2b-[a-z0-9]+\b/i)?.[0];
if (direct) return direct;
const queryMatch = normalized.match(/[?&]memberId=(b2b-[a-z0-9]+)/i);
if (queryMatch) return queryMatch[1];
const mobileMatch = normalized.match(/\/winport\/(b2b-[a-z0-9]+)\.html/i);
if (mobileMatch) return mobileMatch[1];
return null;
}
export function extractShopId(input: string): string | null {
const normalized = cleanText(input);
if (!normalized) return null;
try {
const url = new URL(/^https?:\/\//i.test(normalized) ? normalized : `https://${normalized}`);
const host = normalizeStoreHost(url.hostname);
if (!host) return null;
return host.split('.')[0] ?? null;
} catch {
return /^[a-z0-9-]+$/i.test(normalized) ? normalized : null;
}
}
export function buildProvenance(sourceUrl: string): ProvenanceFields {
return {
source_url: sourceUrl,
fetched_at: new Date().toISOString(),
strategy: STRATEGY,
};
}
export function parsePriceText(text: string): PriceRange {
const normalized = normalizeNumericText(cleanText(text));
const matches = normalized.match(/\d+(?:,\d{3})*(?:\.\d+)?/g) ?? [];
const values = matches
.map((value) => Number.parseFloat(value.replace(/,/g, '')))
.filter((value) => Number.isFinite(value));
if (values.length === 0) {
return {
price_text: normalized,
price_min: null,
price_max: null,
currency: null,
};
}
return {
price_text: normalized,
price_min: values[0] ?? null,
price_max: values[values.length - 1] ?? values[0] ?? null,
currency: normalized.includes('¥') || normalized.includes('元') ? 'CNY' : null,
};
}
export function normalizePriceTiers(
rawTiers: Array<{ beginAmount?: unknown; price?: unknown }>,
unit: string | null,
): PriceTier[] {
return rawTiers
.map((tier) => {
const quantityMin = toNumber(tier.beginAmount);
const priceText = cleanText(tier.price);
const price = toNumber(tier.price);
return {
quantity_text: quantityMin !== null ? `${quantityMin}${unit ?? ''}` : '',
quantity_min: quantityMin,
price_text: priceText,
price,
currency: priceText ? 'CNY' : null,
};
})
.filter((tier) => tier.price_text);
}
export function parseMoqText(text: string): MoqValue {
const normalized = normalizeNumericText(cleanText(text));
const match = normalized.match(/(\d+(?:\.\d+)?)\s*(件|个|套|箱|包|双|台|把|只|pcs|piece|pieces)?\s*起批/i)
?? normalized.match(/≥\s*(\d+(?:\.\d+)?)/);
const rangeMatch = normalized.match(
/(\d+(?:\.\d+)?)\s*(?:~|-|至|到)\s*\d+(?:\.\d+)?\s*(件|个|套|箱|包|双|台|把|只|pcs|piece|pieces)/i,
);
if (!match && !rangeMatch) {
return {
moq_text: normalized,
moq_value: null,
};
}
return {
moq_text: normalized,
moq_value: Number.parseFloat((match ?? rangeMatch)![1]),
};
}
export function extractLocation(text: string): string | null {
const normalized = cleanMultilineText(text);
const primaryRegion = normalized.split(/送至|发往/)[0] ?? normalized;
const lines = primaryRegion.split('\n');
for (const line of lines) {
const compact = cleanText(line);
if (!compact || compact.length > 16) continue;
if (CHINA_LOCATIONS.some((location) => compact.startsWith(location))) {
return compact;
}
}
const locationPattern = new RegExp(`(${CHINA_LOCATIONS.join('|')})[\\u4e00-\\u9fa5]{0,8}`);
return primaryRegion.match(locationPattern)?.[0] ?? null;
}
export function extractAddress(text: string): string | null {
const normalized = cleanMultilineText(text);
const lineMatch = normalized.match(/地址[:]\s*([^\n]+)/);
if (lineMatch) return cleanText(lineMatch[1]);
return normalized
.split('\n')
.map((line) => cleanText(line))
.find((line) => line.includes('省') || line.includes('市') || line.includes('区') || line.includes('县'))
?? null;
}
export function extractMetric(text: string, label: string): string | null {
const normalized = cleanMultilineText(text);
const direct = normalized.match(new RegExp(`(?:^|\\n)\\s*${escapeForRegex(label)}[:]?\\s*([^\\n]+)`));
if (direct) return cleanText(direct[1]);
const lineBased = normalized.match(new RegExp(`(?:^|\\n)\\s*${escapeForRegex(label)}\\n([^\\n]+)`));
return lineBased ? cleanText(lineBased[1]) : null;
}
export function extractYearsOnPlatform(text: string): string | null {
return text.match(/入驻\d+年/)?.[0] ?? null;
}
export function extractMainBusiness(text: string): string | null {
const value = extractMetric(text, '主营');
return value ? value.replace(/^/, '').trim() : null;
}
export function extractBadges(text: string, candidates: string[]): string[] {
return uniqueNonEmpty(candidates.filter((candidate) => cleanMultilineText(text).includes(candidate)));
}
export function guessTopCategories(text: string): string[] {
const mainBusiness = extractMainBusiness(text);
if (!mainBusiness) return [];
return uniqueNonEmpty(mainBusiness.split(/[、,/|]/).map((value) => value.trim()));
}
export function isCaptchaState(state: Partial<PageState>): boolean {
const href = cleanText(state.href).toLowerCase();
const title = cleanText(state.title);
const bodyText = cleanMultilineText(state.body_text);
if (href.includes(CAPTCHA_URL_MARKER)) return true;
return CAPTCHA_TEXT_PATTERNS.some((pattern) => title.includes(pattern) || bodyText.includes(pattern));
}
export function isLoginState(state: Partial<PageState>): boolean {
const href = cleanText(state.href).toLowerCase();
const title = cleanText(state.title);
const bodyText = cleanMultilineText(state.body_text);
if (LOGIN_URL_PATTERNS.some((pattern) => href.includes(pattern))) return true;
return LOGIN_TEXT_PATTERNS.some((pattern) => title.includes(pattern) || bodyText.includes(pattern));
}
export function buildCaptchaHint(action: string): string {
return [
`Open a clean 1688 ${action} page in the shared Chrome profile and finish any slider challenge first.`,
'If you run opencli via CDP, set OPENCLI_CDP_TARGET=1688.com or a more specific 1688 host before retrying.',
].join(' ');
}
export async function readPageState(page: IPage): Promise<PageState> {
const result = await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
body_text: document.body ? document.body.innerText || '' : '',
}))()
`) as Partial<PageState>;
return {
href: cleanText(result.href),
title: cleanText(result.title),
body_text: cleanMultilineText(result.body_text),
};
}
export async function gotoAndReadState(
page: IPage,
url: string,
settleMs: number = 2500,
action: string = 'page',
): Promise<PageState> {
try {
await page.goto(url, { settleMs });
await page.wait(1.5);
return readPageState(page);
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
if (
message.includes('Inspected target navigated or closed')
|| message.includes('Cannot find context with specified id')
|| message.includes('Target closed')
) {
throw new CommandExecutionError(
`1688 ${action} navigation lost the current browser target`,
`${buildCaptchaHint(action)} If CDP is attached to a stale or blocked tab, open a fresh 1688 tab and point OPENCLI_CDP_TARGET at that tab.`,
);
}
throw error;
}
}
export async function ensure1688Session(page: IPage): Promise<void> {
const state = await gotoAndReadState(page, HOME_URL, 1500, 'homepage');
assertAuthenticatedState(state, 'homepage');
}
export function assertAuthenticatedState(state: PageState, action: string): void {
if (!isCaptchaState(state) && !isLoginState(state)) return;
throw new AuthRequiredError('1688.com', `请先在共享 Chrome 完成 1688 登录/验证,再重试(${action}`);
}
export function assertNotCaptcha(state: PageState, action: string): void {
assertAuthenticatedState(state, action);
}
export function toNumber(value: unknown): number | null {
if (typeof value === 'number' && Number.isFinite(value)) {
return value;
}
if (typeof value === 'string') {
const normalized = value.replace(/,/g, '').trim();
if (!normalized) return null;
const parsed = Number.parseFloat(normalized);
return Number.isFinite(parsed) ? parsed : null;
}
return null;
}
export function limitCandidates<T>(values: T[], limit: number): T[] {
const normalizedLimit = Math.max(1, Math.trunc(limit) || 1);
return values.slice(0, normalizedLimit);
}
export function normalizeMediaUrl(input: unknown): string {
const raw = cleanText(input);
if (!raw) return '';
let value = raw
.replace(/^url\((.*)\)$/i, '$1')
.replace(/^['"]|['"]$/g, '')
.replace(/\\u002F/g, '/')
.replace(/&amp;/g, '&')
.trim();
if (!value || value.startsWith('data:') || value.startsWith('blob:')) return '';
if (value.startsWith('//')) value = `https:${value}`;
try {
const url = new URL(value);
return url.toString();
} catch {
return '';
}
}
export function uniqueMediaSources(values: MediaSource[]): MediaSource[] {
const seen = new Set<string>();
const result: MediaSource[] = [];
for (const value of values) {
const url = normalizeMediaUrl(value.url);
if (!url) continue;
const key = `${value.type}:${url}`;
if (seen.has(key)) continue;
seen.add(key);
result.push({
...value,
url,
source: cleanText(value.source) || undefined,
});
}
return result;
}
function normalizeNumericText(value: string): string {
return value
.replace(/([¥$€])\s+(?=\d)/g, '$1')
.replace(/(\d)\s*\.\s*(\d)/g, '$1.$2')
.replace(/\s*([~-])\s*/g, '$1')
.trim();
}
function escapeForRegex(value: string): string {
return value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
}
function parse1688Url(input: string): URL {
const normalized = cleanText(input);
try {
const url = new URL(normalized);
if (!url.hostname.endsWith('.1688.com') && url.hostname !== '1688.com' && url.hostname !== 'www.1688.com') {
throw new Error('invalid-host');
}
stripTrackingParams(url);
url.hash = '';
return url;
} catch {
throw new ArgumentError(
'Invalid 1688 URL',
'Use a URL under 1688.com (for example: https://detail.1688.com/offer/887904326744.html)',
);
}
}
function parse1688UrlOrNull(input: string): URL | null {
try {
return parse1688Url(input);
} catch {
return null;
}
}
function normalizeStoreHost(hostname: string): string | null {
const lower = cleanText(hostname).toLowerCase();
if (!lower.endsWith('.1688.com')) return null;
const [subdomain] = lower.split('.');
if (!subdomain || STORE_GENERIC_HOSTS.has(subdomain)) return null;
return lower;
}
function stripTrackingParams(url: URL): void {
const keys = [...url.searchParams.keys()];
for (const key of keys) {
if (TRACKING_QUERY_KEYS.has(key) || key.toLowerCase().startsWith('utm_')) {
url.searchParams.delete(key);
}
}
}
export const __test__ = {
SEARCH_LIMIT_DEFAULT,
SEARCH_LIMIT_MAX,
parseSearchLimit,
buildSearchUrl,
buildDetailUrl,
resolveStoreUrl,
canonicalizeStoreUrl,
canonicalizeItemUrl,
canonicalizeSellerUrl,
extractOfferId,
extractMemberId,
extractShopId,
parsePriceText,
normalizePriceTiers,
parseMoqText,
extractLocation,
extractAddress,
extractMetric,
extractYearsOnPlatform,
extractMainBusiness,
extractBadges,
guessTopCategories,
isCaptchaState,
isLoginState,
cleanText,
cleanMultilineText,
uniqueNonEmpty,
normalizeMediaUrl,
uniqueMediaSources,
limitCandidates,
};
+226
View File
@@ -0,0 +1,226 @@
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { FACTORY_BADGE_PATTERNS, SERVICE_BADGE_PATTERNS, assertAuthenticatedState, buildDetailUrl, buildProvenance, canonicalizeSellerUrl, canonicalizeStoreUrl, cleanMultilineText, cleanText, extractAddress, extractBadges, extractMemberId, extractMetric, extractOfferId, extractShopId, extractYearsOnPlatform, gotoAndReadState, guessTopCategories, resolveStoreUrl, uniqueNonEmpty, } from './shared.js';
function normalizeStorePayload(input) {
const storePayload = input.storePayload;
const contactPayload = input.contactPayload;
const seed = input.seed;
const contactText = cleanMultilineText(contactPayload?.bodyText);
const storeText = cleanMultilineText(storePayload?.bodyText);
const seedText = cleanMultilineText(seed?.bodyText);
const combinedText = [contactText, storeText, seedText].filter(Boolean).join('\n');
const sellerUrlRaw = cleanText(seed?.seller?.winportUrl
?? seed?.seller?.sellerWinportUrlMap?.defaultUrl
?? storePayload?.href
?? input.resolvedUrl);
const storeUrl = safeCanonicalStoreUrl(sellerUrlRaw || input.resolvedUrl) ?? input.resolvedUrl;
const sellerUrl = canonicalizeSellerUrl(sellerUrlRaw) ?? storeUrl;
const companyUrl = pickCompanyUrl(contactPayload?.href, storeUrl);
const memberId = cleanText(seed?.seller?.memberId)
|| input.explicitMemberId
|| extractMemberId(input.resolvedUrl)
|| extractMemberId(storePayload?.href ?? '')
|| null;
const shopId = extractShopId(sellerUrl) ?? extractShopId(storeUrl);
const companyName = cleanText(seed?.seller?.companyName)
|| firstNamedLine(contactText)
|| firstNamedLine(storeText)
|| null;
const serviceBadges = uniqueNonEmpty([
...extractBadges(combinedText, SERVICE_BADGE_PATTERNS),
...((seed?.services ?? []).map((service) => cleanText(service.serviceName))),
]);
const factoryBadges = extractBadges(combinedText, FACTORY_BADGE_PATTERNS);
return {
member_id: memberId,
shop_id: shopId,
store_name: companyName,
store_url: storeUrl,
company_name: companyName,
company_url: companyUrl,
business_model_text: firstMetric(combinedText, ['经营模式', '生产加工', '主营产品']),
years_on_platform_text: extractYearsOnPlatform(combinedText),
location: extractAddress(contactText) ?? extractAddress(storeText),
staff_size_text: firstMetric(combinedText, ['员工人数', '员工总数']),
factory_badges: factoryBadges,
service_badges: serviceBadges,
response_rate_text: firstMetric(combinedText, ['响应率', '回复率', '响应速度']),
return_rate_text: extractReturnRate(combinedText),
top_categories: guessTopCategories(combinedText),
phone_text: extractMetric(contactText, '电话'),
mobile_text: extractMetric(contactText, '手机'),
...buildProvenance(cleanText(contactPayload?.href) || cleanText(storePayload?.href) || input.resolvedUrl),
};
}
function safeCanonicalStoreUrl(url) {
try {
return canonicalizeStoreUrl(url);
}
catch {
return null;
}
}
function pickCompanyUrl(contactHref, storeUrl) {
const fromPage = cleanText(contactHref);
if (fromPage) {
const normalized = buildContactUrl(fromPage);
if (normalized)
return normalized;
}
return buildContactUrl(storeUrl);
}
function buildContactUrl(storeUrl) {
try {
const parsed = new URL(storeUrl);
if (!parsed.hostname.endsWith('.1688.com'))
return null;
return `${parsed.protocol}//${parsed.hostname}/page/contactinfo.html`;
}
catch {
return null;
}
}
function firstNamedLine(text) {
return text
.split('\n')
.map((line) => cleanText(line))
.find((line) => line.includes('有限公司') || line.includes('商行') || line.includes('工厂'))
?? null;
}
function firstMetric(text, labels) {
for (const label of labels) {
const value = extractMetric(text, label);
if (value)
return value;
}
return null;
}
function extractReturnRate(text) {
const inline = text.match(/回头率\s*([0-9.]+%)/);
if (inline)
return cleanText(inline[0]);
const multiline = text.match(/回头率\s*\n\s*([0-9.]+%)/);
if (!multiline)
return null;
return `回头率${cleanText(multiline[1])}`;
}
function firstOfferId(links) {
for (const link of links) {
const offerId = extractOfferId(link);
if (offerId)
return offerId;
}
return null;
}
function firstContactUrl(links) {
for (const link of links) {
const url = buildContactUrl(link);
if (url)
return url;
}
return null;
}
async function readStorePayload(page, url, action) {
const state = await gotoAndReadState(page, url, 2500, action);
assertAuthenticatedState(state, action);
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
offerLinks: Array.from(document.querySelectorAll('a[href*="detail.1688.com/offer/"], a[href*="offerId="]'))
.map((anchor) => anchor.href)
.filter(Boolean),
contactLinks: Array.from(document.querySelectorAll('a[href*="contactinfo"]'))
.map((anchor) => anchor.href)
.filter(Boolean),
}))()
`);
}
async function readItemSeed(page, offerId) {
const itemUrl = buildDetailUrl(offerId);
const state = await gotoAndReadState(page, itemUrl, 2500, 'store seed item');
assertAuthenticatedState(state, 'store seed item');
const seed = await page.evaluate(`
(() => {
const model = window.context?.result?.global?.globalData?.model ?? null;
const toJson = (value) => JSON.parse(JSON.stringify(value ?? null));
return {
href: window.location.href,
bodyText: document.body ? document.body.innerText || '' : '',
seller: toJson(model?.sellerModel),
services: toJson(model?.shippingServices?.fields?.buyerProtectionModel ?? []),
};
})()
`);
const hasSellerContext = !!cleanText(seed?.seller?.memberId) || !!cleanText(seed?.seller?.winportUrl);
if (!hasSellerContext) {
throw new CommandExecutionError('1688 store seed item did not expose seller context', '当前 tab 非商品详情上下文,请切到 detail.1688.com 商品页并重试');
}
return seed;
}
function hasAnyEvidence(storePayload, contactPayload, seed) {
return !!cleanText(storePayload?.bodyText)
|| !!cleanText(contactPayload?.bodyText)
|| !!cleanText(seed?.bodyText);
}
cli({
site: '1688',
name: 'store',
description: '1688 店铺/供应商公开信息(联系方式、主营、入驻年限、公开服务信号)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 店铺 URL 或 member ID(如 b2b-22154705262941f196',
},
],
columns: ['store_name', 'years_on_platform_text', 'location', 'return_rate_text'],
func: async (page, kwargs) => {
const rawInput = String(kwargs.input ?? '');
const resolvedUrl = resolveStoreUrl(rawInput);
const explicitMemberId = extractMemberId(rawInput);
const storePayload = await readStorePayload(page, resolvedUrl, 'store');
const contactUrl = firstContactUrl(storePayload.contactLinks ?? []) || buildContactUrl(storePayload.href || resolvedUrl);
const contactPayload = contactUrl ? await readStorePayload(page, contactUrl, 'store contact') : null;
const offerId = extractOfferId(rawInput)
|| firstOfferId(storePayload.offerLinks ?? [])
|| firstOfferId(contactPayload?.offerLinks ?? []);
let seed = null;
if (offerId) {
try {
seed = await readItemSeed(page, offerId);
}
catch (error) {
if (!(error instanceof CommandExecutionError))
throw error;
}
}
if (!hasAnyEvidence(storePayload, contactPayload, seed)) {
throw new EmptyResultError('1688 store', 'Store page is reachable but no visible fields were extracted. Open the store page in Chrome and retry.');
}
return [
normalizeStorePayload({
resolvedUrl,
storePayload,
contactPayload,
seed,
explicitMemberId,
}),
];
},
});
export const __test__ = {
normalizeStorePayload,
safeCanonicalStoreUrl,
buildContactUrl,
firstNamedLine,
firstMetric,
extractReturnRate,
firstOfferId,
firstContactUrl,
};
+62
View File
@@ -0,0 +1,62 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './store.js';
describe('1688 store normalization', () => {
it('merges store contact text with seller seed data', () => {
const result = __test__.normalizeStorePayload({
resolvedUrl: 'https://yinuoweierfushi.1688.com/?offerId=887904326744',
explicitMemberId: null,
storePayload: {
href: 'https://yinuoweierfushi.1688.com/page/index.html',
bodyText: `
青岛沁澜衣品服装有限公司
联系方式
地址:山东省青岛市即墨区环秀街道办事处湘江二路97号甲
`,
offerLinks: ['https://detail.1688.com/offer/887904326744.html'],
},
contactPayload: {
href: 'https://yinuoweierfushi.1688.com/page/contactinfo.html',
bodyText: `
青岛沁澜衣品服装有限公司
电话:86 0532 86655366
手机:15963238678
地址:山东省青岛市即墨区环秀街道办事处湘江二路97号甲
`,
},
seed: {
bodyText: `
入驻13年
主营:大码女装
店铺回头率
87%
延期必赔
品质保障
`,
seller: {
companyName: '青岛沁澜衣品服装有限公司',
memberId: 'b2b-1641351767',
winportUrl: 'https://yinuoweierfushi.1688.com/page/index.html?spm=abc',
},
services: [{ serviceName: '延期必赔' }, { serviceName: '品质保障' }],
},
});
expect(result.member_id).toBe('b2b-1641351767');
expect(result.store_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.company_url).toBe('https://yinuoweierfushi.1688.com/page/contactinfo.html');
expect(result.years_on_platform_text).toBe('入驻13年');
expect(result.location).toBe('山东省青岛市即墨区环秀街道办事处湘江二路97号甲');
expect(result.return_rate_text).toContain('87%');
expect(result.top_categories).toEqual(['大码女装']);
expect(result.service_badges).toEqual(['延期必赔', '品质保障']);
});
it('builds contact urls and extracts offer ids', () => {
expect(__test__.safeCanonicalStoreUrl('https://yinuoweierfushi.1688.com/page/index.html?spm=foo')).toBe('https://yinuoweierfushi.1688.com');
expect(__test__.buildContactUrl('https://yinuoweierfushi.1688.com')).toBe('https://yinuoweierfushi.1688.com/page/contactinfo.html');
expect(__test__.firstOfferId([
'https://detail.1688.com/offer/887904326744.html',
])).toBe('887904326744');
expect(__test__.firstContactUrl([
'https://yinuoweierfushi.1688.com/page/contactinfo.html?spm=1',
])).toBe('https://yinuoweierfushi.1688.com/page/contactinfo.html');
});
});
-69
View File
@@ -1,69 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './store.js';
describe('1688 store normalization', () => {
it('merges store contact text with seller seed data', () => {
const result = __test__.normalizeStorePayload({
resolvedUrl: 'https://yinuoweierfushi.1688.com/?offerId=887904326744',
explicitMemberId: null,
storePayload: {
href: 'https://yinuoweierfushi.1688.com/page/index.html',
bodyText: `
青岛沁澜衣品服装有限公司
联系方式
地址:山东省青岛市即墨区环秀街道办事处湘江二路97号甲
`,
offerLinks: ['https://detail.1688.com/offer/887904326744.html'],
},
contactPayload: {
href: 'https://yinuoweierfushi.1688.com/page/contactinfo.html',
bodyText: `
青岛沁澜衣品服装有限公司
电话:86 0532 86655366
手机:15963238678
地址:山东省青岛市即墨区环秀街道办事处湘江二路97号甲
`,
},
seed: {
bodyText: `
入驻13年
主营:大码女装
店铺回头率
87%
延期必赔
品质保障
`,
seller: {
companyName: '青岛沁澜衣品服装有限公司',
memberId: 'b2b-1641351767',
winportUrl: 'https://yinuoweierfushi.1688.com/page/index.html?spm=abc',
},
services: [{ serviceName: '延期必赔' }, { serviceName: '品质保障' }],
},
});
expect(result.member_id).toBe('b2b-1641351767');
expect(result.store_url).toBe('https://yinuoweierfushi.1688.com');
expect(result.company_url).toBe('https://yinuoweierfushi.1688.com/page/contactinfo.html');
expect(result.years_on_platform_text).toBe('入驻13年');
expect(result.location).toBe('山东省青岛市即墨区环秀街道办事处湘江二路97号甲');
expect(result.return_rate_text).toContain('87%');
expect(result.top_categories).toEqual(['大码女装']);
expect(result.service_badges).toEqual(['延期必赔', '品质保障']);
});
it('builds contact urls and extracts offer ids', () => {
expect(__test__.safeCanonicalStoreUrl('https://yinuoweierfushi.1688.com/page/index.html?spm=foo')).toBe(
'https://yinuoweierfushi.1688.com',
);
expect(__test__.buildContactUrl('https://yinuoweierfushi.1688.com')).toBe(
'https://yinuoweierfushi.1688.com/page/contactinfo.html',
);
expect(__test__.firstOfferId([
'https://detail.1688.com/offer/887904326744.html',
])).toBe('887904326744');
expect(__test__.firstContactUrl([
'https://yinuoweierfushi.1688.com/page/contactinfo.html?spm=1',
])).toBe('https://yinuoweierfushi.1688.com/page/contactinfo.html');
});
});
-300
View File
@@ -1,300 +0,0 @@
import { CommandExecutionError, EmptyResultError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
FACTORY_BADGE_PATTERNS,
SERVICE_BADGE_PATTERNS,
assertAuthenticatedState,
buildDetailUrl,
buildProvenance,
canonicalizeSellerUrl,
canonicalizeStoreUrl,
cleanMultilineText,
cleanText,
extractAddress,
extractBadges,
extractMemberId,
extractMetric,
extractOfferId,
extractShopId,
extractYearsOnPlatform,
gotoAndReadState,
guessTopCategories,
resolveStoreUrl,
uniqueNonEmpty,
} from './shared.js';
interface StoreBrowserPayload {
href?: string;
title?: string;
bodyText?: string;
offerLinks?: string[];
contactLinks?: string[];
}
interface StoreItemSeed {
href?: string;
bodyText?: string;
seller?: {
companyName?: string;
memberId?: string;
winportUrl?: string;
sellerWinportUrlMap?: Record<string, string>;
};
services?: Array<{ serviceName?: string }>;
}
function normalizeStorePayload(input: {
resolvedUrl: string;
storePayload: StoreBrowserPayload | null;
contactPayload: StoreBrowserPayload | null;
seed: StoreItemSeed | null;
explicitMemberId: string | null;
}): Record<string, unknown> {
const storePayload = input.storePayload;
const contactPayload = input.contactPayload;
const seed = input.seed;
const contactText = cleanMultilineText(contactPayload?.bodyText);
const storeText = cleanMultilineText(storePayload?.bodyText);
const seedText = cleanMultilineText(seed?.bodyText);
const combinedText = [contactText, storeText, seedText].filter(Boolean).join('\n');
const sellerUrlRaw = cleanText(
seed?.seller?.winportUrl
?? seed?.seller?.sellerWinportUrlMap?.defaultUrl
?? storePayload?.href
?? input.resolvedUrl,
);
const storeUrl = safeCanonicalStoreUrl(sellerUrlRaw || input.resolvedUrl) ?? input.resolvedUrl;
const sellerUrl = canonicalizeSellerUrl(sellerUrlRaw) ?? storeUrl;
const companyUrl = pickCompanyUrl(contactPayload?.href, storeUrl);
const memberId = cleanText(seed?.seller?.memberId)
|| input.explicitMemberId
|| extractMemberId(input.resolvedUrl)
|| extractMemberId(storePayload?.href ?? '')
|| null;
const shopId = extractShopId(sellerUrl) ?? extractShopId(storeUrl);
const companyName = cleanText(seed?.seller?.companyName)
|| firstNamedLine(contactText)
|| firstNamedLine(storeText)
|| null;
const serviceBadges = uniqueNonEmpty([
...extractBadges(combinedText, SERVICE_BADGE_PATTERNS),
...((seed?.services ?? []).map((service) => cleanText(service.serviceName))),
]);
const factoryBadges = extractBadges(combinedText, FACTORY_BADGE_PATTERNS);
return {
member_id: memberId,
shop_id: shopId,
store_name: companyName,
store_url: storeUrl,
company_name: companyName,
company_url: companyUrl,
business_model_text: firstMetric(combinedText, ['经营模式', '生产加工', '主营产品']),
years_on_platform_text: extractYearsOnPlatform(combinedText),
location: extractAddress(contactText) ?? extractAddress(storeText),
staff_size_text: firstMetric(combinedText, ['员工人数', '员工总数']),
factory_badges: factoryBadges,
service_badges: serviceBadges,
response_rate_text: firstMetric(combinedText, ['响应率', '回复率', '响应速度']),
return_rate_text: extractReturnRate(combinedText),
top_categories: guessTopCategories(combinedText),
phone_text: extractMetric(contactText, '电话'),
mobile_text: extractMetric(contactText, '手机'),
...buildProvenance(cleanText(contactPayload?.href) || cleanText(storePayload?.href) || input.resolvedUrl),
};
}
function safeCanonicalStoreUrl(url: string): string | null {
try {
return canonicalizeStoreUrl(url);
} catch {
return null;
}
}
function pickCompanyUrl(contactHref: string | undefined, storeUrl: string): string | null {
const fromPage = cleanText(contactHref);
if (fromPage) {
const normalized = buildContactUrl(fromPage);
if (normalized) return normalized;
}
return buildContactUrl(storeUrl);
}
function buildContactUrl(storeUrl: string): string | null {
try {
const parsed = new URL(storeUrl);
if (!parsed.hostname.endsWith('.1688.com')) return null;
return `${parsed.protocol}//${parsed.hostname}/page/contactinfo.html`;
} catch {
return null;
}
}
function firstNamedLine(text: string): string | null {
return text
.split('\n')
.map((line) => cleanText(line))
.find((line) => line.includes('有限公司') || line.includes('商行') || line.includes('工厂'))
?? null;
}
function firstMetric(text: string, labels: string[]): string | null {
for (const label of labels) {
const value = extractMetric(text, label);
if (value) return value;
}
return null;
}
function extractReturnRate(text: string): string | null {
const inline = text.match(/回头率\s*([0-9.]+%)/);
if (inline) return cleanText(inline[0]);
const multiline = text.match(/回头率\s*\n\s*([0-9.]+%)/);
if (!multiline) return null;
return `回头率${cleanText(multiline[1])}`;
}
function firstOfferId(links: string[]): string | null {
for (const link of links) {
const offerId = extractOfferId(link);
if (offerId) return offerId;
}
return null;
}
function firstContactUrl(links: string[]): string | null {
for (const link of links) {
const url = buildContactUrl(link);
if (url) return url;
}
return null;
}
async function readStorePayload(page: IPage, url: string, action: string): Promise<StoreBrowserPayload> {
const state = await gotoAndReadState(page, url, 2500, action);
assertAuthenticatedState(state, action);
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
bodyText: document.body ? document.body.innerText || '' : '',
offerLinks: Array.from(document.querySelectorAll('a[href*="detail.1688.com/offer/"], a[href*="offerId="]'))
.map((anchor) => anchor.href)
.filter(Boolean),
contactLinks: Array.from(document.querySelectorAll('a[href*="contactinfo"]'))
.map((anchor) => anchor.href)
.filter(Boolean),
}))()
`) as StoreBrowserPayload;
}
async function readItemSeed(page: IPage, offerId: string): Promise<StoreItemSeed> {
const itemUrl = buildDetailUrl(offerId);
const state = await gotoAndReadState(page, itemUrl, 2500, 'store seed item');
assertAuthenticatedState(state, 'store seed item');
const seed = await page.evaluate(`
(() => {
const model = window.context?.result?.global?.globalData?.model ?? null;
const toJson = (value) => JSON.parse(JSON.stringify(value ?? null));
return {
href: window.location.href,
bodyText: document.body ? document.body.innerText || '' : '',
seller: toJson(model?.sellerModel),
services: toJson(model?.shippingServices?.fields?.buyerProtectionModel ?? []),
};
})()
`) as StoreItemSeed;
const hasSellerContext = !!cleanText(seed?.seller?.memberId) || !!cleanText(seed?.seller?.winportUrl);
if (!hasSellerContext) {
throw new CommandExecutionError(
'1688 store seed item did not expose seller context',
'当前 tab 非商品详情上下文,请切到 detail.1688.com 商品页并重试',
);
}
return seed;
}
function hasAnyEvidence(
storePayload: StoreBrowserPayload | null,
contactPayload: StoreBrowserPayload | null,
seed: StoreItemSeed | null,
): boolean {
return !!cleanText(storePayload?.bodyText)
|| !!cleanText(contactPayload?.bodyText)
|| !!cleanText(seed?.bodyText);
}
cli({
site: '1688',
name: 'store',
description: '1688 店铺/供应商公开信息(联系方式、主营、入驻年限、公开服务信号)',
domain: 'www.1688.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: '1688 店铺 URL 或 member ID(如 b2b-22154705262941f196',
},
],
columns: ['store_name', 'years_on_platform_text', 'location', 'return_rate_text'],
func: async (page, kwargs) => {
const rawInput = String(kwargs.input ?? '');
const resolvedUrl = resolveStoreUrl(rawInput);
const explicitMemberId = extractMemberId(rawInput);
const storePayload = await readStorePayload(page, resolvedUrl, 'store');
const contactUrl = firstContactUrl(storePayload.contactLinks ?? []) || buildContactUrl(storePayload.href || resolvedUrl);
const contactPayload = contactUrl ? await readStorePayload(page, contactUrl, 'store contact') : null;
const offerId = extractOfferId(rawInput)
|| firstOfferId(storePayload.offerLinks ?? [])
|| firstOfferId(contactPayload?.offerLinks ?? []);
let seed: StoreItemSeed | null = null;
if (offerId) {
try {
seed = await readItemSeed(page, offerId);
} catch (error) {
if (!(error instanceof CommandExecutionError)) throw error;
}
}
if (!hasAnyEvidence(storePayload, contactPayload, seed)) {
throw new EmptyResultError(
'1688 store',
'Store page is reachable but no visible fields were extracted. Open the store page in Chrome and retry.',
);
}
return [
normalizeStorePayload({
resolvedUrl,
storePayload,
contactPayload,
seed,
explicitMemberId,
}),
];
},
});
export const __test__ = {
normalizeStorePayload,
safeCanonicalStoreUrl,
buildContactUrl,
firstNamedLine,
firstMetric,
extractReturnRate,
firstOfferId,
firstContactUrl,
};
+32 -39
View File
@@ -5,35 +5,30 @@
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import type { IPage } from '@jackwener/opencli/types';
/** Extract article ID from a full URL or a bare numeric ID string */
function parseArticleId(input: string): string {
const m = input.match(/\/p\/(\d+)/);
return m ? m[1] : input.replace(/\D/g, '');
function parseArticleId(input) {
const m = input.match(/\/p\/(\d+)/);
return m ? m[1] : input.replace(/\D/g, '');
}
cli({
site: '36kr',
name: 'article',
description: '获取36氪文章正文内容',
domain: 'www.36kr.com',
strategy: Strategy.INTERCEPT,
args: [
{ name: 'id', positional: true, required: true, help: 'Article ID or full 36kr article URL' },
],
columns: ['field', 'value'],
func: async (page: IPage, args) => {
const articleId = parseArticleId(String(args.id ?? ''));
if (!articleId) {
throw new CliError('INVALID_ARGUMENT', 'Invalid article ID or URL');
}
await page.installInterceptor('36kr.com/api');
await page.goto(`https://www.36kr.com/p/${articleId}`);
await page.wait(5);
const data: any = await page.evaluate(`
site: '36kr',
name: 'article',
description: '获取36氪文章正文内容',
domain: 'www.36kr.com',
strategy: Strategy.INTERCEPT,
args: [
{ name: 'id', positional: true, required: true, help: 'Article ID or full 36kr article URL' },
],
columns: ['field', 'value'],
func: async (page, args) => {
const articleId = parseArticleId(String(args.id ?? ''));
if (!articleId) {
throw new CliError('INVALID_ARGUMENT', 'Invalid article ID or URL');
}
await page.installInterceptor('36kr.com/api');
await page.goto(`https://www.36kr.com/p/${articleId}`);
await page.wait(5);
const data = await page.evaluate(`
(() => {
// Title: 36kr uses class "article-title" on h1
const title = document.querySelector('.article-title, h1')?.textContent?.trim() || '';
@@ -53,17 +48,15 @@ cli({
return { title, author, date, body };
})()
`);
if (!data?.title) {
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
}
return [
{ field: 'title', value: data.title },
{ field: 'author', value: data.author || '-' },
{ field: 'date', value: data.date || '-' },
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
{ field: 'body', value: data.body || '-' },
];
},
if (!data?.title) {
throw new CliError('NOT_FOUND', 'Article not found or failed to load', 'Check the article ID');
}
return [
{ field: 'title', value: data.title },
{ field: 'author', value: data.author || '-' },
{ field: 'date', value: data.date || '-' },
{ field: 'url', value: `https://36kr.com/p/${articleId}` },
{ field: 'body', value: data.body || '-' },
];
},
});
+86
View File
@@ -0,0 +1,86 @@
/**
* 36kr hot-list — DOM scraping.
*
* Navigates to the 36kr hot-list page and scrapes rendered article links.
* Supports category types: renqi (人气), zonghe (综合), shoucang (收藏), catalog (综合热门).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
const TYPE_MAP = {
renqi: '人气榜',
zonghe: '综合榜',
shoucang: '收藏榜',
catalog: '热门资讯',
};
function getShanghaiDate(date = new Date()) {
// Shanghai stays on UTC+8 year-round, so a fixed offset is sufficient here
// and avoids the slow Intl timezone path that timed out on Windows CI.
return new Date(date.getTime() + 8 * 60 * 60 * 1000).toISOString().slice(0, 10);
}
function buildHotListUrl(listType, date = new Date()) {
if (listType === 'catalog') {
return 'https://www.36kr.com/hot-list/catalog';
}
return `https://www.36kr.com/hot-list/${listType}/${getShanghaiDate(date)}/1`;
}
cli({
site: '36kr',
name: 'hot',
description: '36氪热榜 — trending articles (renqi/zonghe/shoucang/catalog)',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of items (max 50)' },
{
name: 'type',
type: 'string',
default: 'catalog',
help: 'List type: renqi (人气), zonghe (综合), shoucang (收藏), catalog (热门资讯)',
},
],
columns: ['rank', 'title', 'url'],
func: async (page, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const listType = String(args.type ?? 'catalog');
if (!TYPE_MAP[listType]) {
throw new CliError('INVALID_ARGUMENT', `Unknown type "${listType}". Valid types: ${Object.keys(TYPE_MAP).join(', ')}`);
}
const url = buildHotListUrl(listType);
await page.goto(url);
// Poll DOM until article links appear (36kr renders client-side)
const deadline = Date.now() + 5000;
while (Date.now() < deadline) {
if (await page.evaluate('document.querySelectorAll("a[href*=\\"/p/\\"]").length'))
break;
await new Promise(r => setTimeout(r, 300));
}
// Scrape rendered article links from DOM (deduplicated)
const domItems = await page.evaluate(`
(() => {
const seen = new Set();
const results = [];
const links = document.querySelectorAll('a[href*="/p/"]');
for (const el of links) {
const href = el.getAttribute('href') || '';
const title = el.textContent?.trim() || '';
if (!title || title.length < 5 || seen.has(href) || seen.has(title)) continue;
seen.add(href);
seen.add(title);
results.push({ title, url: href.startsWith('http') ? href : 'https://36kr.com' + href });
}
return results;
})()
`);
const items = Array.isArray(domItems) ? domItems : [];
if (items.length === 0) {
throw new CliError('NO_DATA', 'Could not retrieve 36kr hot list', '36kr may have changed its DOM structure');
}
return items.slice(0, count).map((item, i) => ({
rank: i + 1,
title: item.title,
url: item.url,
}));
},
});
export { buildHotListUrl, getShanghaiDate };
+15
View File
@@ -0,0 +1,15 @@
import { describe, expect, it } from 'vitest';
import { buildHotListUrl, getShanghaiDate } from './hot.js';
describe('36kr/hot date routing', () => {
it('formats dates in Asia/Shanghai instead of UTC', () => {
const date = new Date('2026-03-25T18:30:00.000Z');
expect(getShanghaiDate(date)).toBe('2026-03-26');
});
it('builds dated hot-list routes with Shanghai-local date', () => {
const date = new Date('2026-03-25T18:30:00.000Z');
expect(buildHotListUrl('renqi', date)).toBe('https://www.36kr.com/hot-list/renqi/2026-03-26/1');
});
it('keeps catalog on the static route', () => {
expect(buildHotListUrl('catalog')).toBe('https://www.36kr.com/hot-list/catalog');
});
});
-19
View File
@@ -1,19 +0,0 @@
import { describe, expect, it } from 'vitest';
import { buildHotListUrl, getShanghaiDate } from './hot.js';
describe('36kr/hot date routing', () => {
it('formats dates in Asia/Shanghai instead of UTC', () => {
const date = new Date('2026-03-25T18:30:00.000Z');
expect(getShanghaiDate(date)).toBe('2026-03-26');
});
it('builds dated hot-list routes with Shanghai-local date', () => {
const date = new Date('2026-03-25T18:30:00.000Z');
expect(buildHotListUrl('renqi', date)).toBe('https://www.36kr.com/hot-list/renqi/2026-03-26/1');
});
it('keeps catalog on the static route', () => {
expect(buildHotListUrl('catalog')).toBe('https://www.36kr.com/hot-list/catalog');
});
});
-105
View File
@@ -1,105 +0,0 @@
/**
* 36kr hot-list — DOM scraping.
*
* Navigates to the 36kr hot-list page and scrapes rendered article links.
* Supports category types: renqi (人气), zonghe (综合), shoucang (收藏), catalog (综合热门).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import type { IPage } from '@jackwener/opencli/types';
const TYPE_MAP: Record<string, string> = {
renqi: '人气榜',
zonghe: '综合榜',
shoucang: '收藏榜',
catalog: '热门资讯',
};
function getShanghaiDate(date = new Date()): string {
// Shanghai stays on UTC+8 year-round, so a fixed offset is sufficient here
// and avoids the slow Intl timezone path that timed out on Windows CI.
return new Date(date.getTime() + 8 * 60 * 60 * 1000).toISOString().slice(0, 10);
}
function buildHotListUrl(listType: string, date = new Date()): string {
if (listType === 'catalog') {
return 'https://www.36kr.com/hot-list/catalog';
}
return `https://www.36kr.com/hot-list/${listType}/${getShanghaiDate(date)}/1`;
}
cli({
site: '36kr',
name: 'hot',
description: '36氪热榜 — trending articles (renqi/zonghe/shoucang/catalog)',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of items (max 50)' },
{
name: 'type',
type: 'string',
default: 'catalog',
help: 'List type: renqi (人气), zonghe (综合), shoucang (收藏), catalog (热门资讯)',
},
],
columns: ['rank', 'title', 'url'],
func: async (page: IPage, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const listType = String(args.type ?? 'catalog');
if (!TYPE_MAP[listType]) {
throw new CliError(
'INVALID_ARGUMENT',
`Unknown type "${listType}". Valid types: ${Object.keys(TYPE_MAP).join(', ')}`,
);
}
const url = buildHotListUrl(listType);
await page.goto(url);
// Poll DOM until article links appear (36kr renders client-side)
const deadline = Date.now() + 5000;
while (Date.now() < deadline) {
if (await page.evaluate('document.querySelectorAll("a[href*=\\"/p/\\"]").length')) break;
await new Promise(r => setTimeout(r, 300));
}
// Scrape rendered article links from DOM (deduplicated)
const domItems: any = await page.evaluate(`
(() => {
const seen = new Set();
const results = [];
const links = document.querySelectorAll('a[href*="/p/"]');
for (const el of links) {
const href = el.getAttribute('href') || '';
const title = el.textContent?.trim() || '';
if (!title || title.length < 5 || seen.has(href) || seen.has(title)) continue;
seen.add(href);
seen.add(title);
results.push({ title, url: href.startsWith('http') ? href : 'https://36kr.com' + href });
}
return results;
})()
`);
const items = Array.isArray(domItems) ? (domItems as any[]) : [];
if (items.length === 0) {
throw new CliError(
'NO_DATA',
'Could not retrieve 36kr hot list',
'36kr may have changed its DOM structure',
);
}
return items.slice(0, count).map((item: any, i: number) => ({
rank: i + 1,
title: item.title,
url: item.url,
}));
},
});
export { buildHotListUrl, getShanghaiDate };
+51
View File
@@ -0,0 +1,51 @@
/**
* 36kr latest news — public RSS feed, no browser needed.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
cli({
site: '36kr',
name: 'news',
description: 'Latest tech/startup news from 36kr (36氪)',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles (max 50)' },
],
columns: ['rank', 'title', 'summary', 'date', 'url'],
func: async (kwargs) => {
const count = Math.min(kwargs.limit || 20, 50);
const resp = await fetch('https://www.36kr.com/feed', {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; opencli/1.0)' },
});
if (!resp.ok)
return [];
const xml = await resp.text();
const items = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < count) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url = block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ??
block.match(/<link>(.*?)<\/link>/)?.[1] ??
'';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
// Extract plain-text summary from HTML description (first ~120 chars)
const rawDesc = block.match(/<description><!\[CDATA\[([\s\S]*?)\]\]>/)?.[1] ?? '';
const summary = rawDesc
.replace(/<[^>]+>/g, ' ')
.replace(/&nbsp;/g, ' ')
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/\s+/g, ' ')
.trim()
.slice(0, 120);
if (title) {
items.push({ rank: items.length + 1, title, summary, date, url: url.trim() });
}
}
return items;
},
});
+85
View File
@@ -0,0 +1,85 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
const SAMPLE_RSS = `<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"><channel><title>36氪</title>
<item>
<title>红杉中国领投AI公司「示例」,金额近2亿元</title>
<link><![CDATA[https://36kr.com/p/1111111111111111?f=rss]]></link>
<pubDate>2026-03-26 10:00:00 +0800</pubDate>
</item>
<item>
<title>马斯克旗下xAI估值突破1000亿美元</title>
<link><![CDATA[https://36kr.com/p/2222222222222222?f=rss]]></link>
<pubDate>2026-03-26 09:00:00 +0800</pubDate>
</item>
<item>
<title>OpenAI发布GPT-5,多模态能力大幅提升</title>
<link><![CDATA[https://36kr.com/p/3333333333333333?f=rss]]></link>
<pubDate>2026-03-25 20:00:00 +0800</pubDate>
</item>
</channel></rss>`;
afterEach(() => {
vi.restoreAllMocks();
});
describe('36kr/news RSS parsing', () => {
it('parses RSS feed into ranked news items', async () => {
vi.spyOn(globalThis, 'fetch').mockResolvedValue({
ok: true,
text: async () => SAMPLE_RSS,
});
// Direct RSS parse test using the same regex logic as news.ts
const xml = SAMPLE_RSS;
const items = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < 10) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url = block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ??
block.match(/<link>(.*?)<\/link>/)?.[1] ??
'';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
if (title)
items.push({ rank: items.length + 1, title, date, url: url.trim() });
}
expect(items).toHaveLength(3);
expect(items[0].rank).toBe(1);
expect(items[0].title).toBe('红杉中国领投AI公司「示例」,金额近2亿元');
expect(items[0].date).toBe('2026-03-26');
expect(items[0].url).toBe('https://36kr.com/p/1111111111111111?f=rss');
});
it('respects limit — returns at most N items', async () => {
const xml = SAMPLE_RSS;
const limit = 2;
const items = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < limit) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url = block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ?? '';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
if (title)
items.push({ rank: items.length + 1, title, date, url: url.trim() });
}
expect(items).toHaveLength(2);
});
it('skips items with empty title', async () => {
const xml = `<rss><channel>
<item><title></title><link>https://36kr.com/p/0</link><pubDate>2026-01-01</pubDate></item>
<item><title>有标题的文章</title><link>https://36kr.com/p/1</link><pubDate>2026-01-01</pubDate></item>
</channel></rss>`;
const items = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml))) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
if (title)
items.push({ title });
}
expect(items).toHaveLength(1);
expect(items[0].title).toBe('有标题的文章');
});
});
-90
View File
@@ -1,90 +0,0 @@
import { describe, it, expect, vi, afterEach } from 'vitest';
const SAMPLE_RSS = `<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"><channel><title>36氪</title>
<item>
<title>红杉中国领投AI公司「示例」,金额近2亿元</title>
<link><![CDATA[https://36kr.com/p/1111111111111111?f=rss]]></link>
<pubDate>2026-03-26 10:00:00 +0800</pubDate>
</item>
<item>
<title>马斯克旗下xAI估值突破1000亿美元</title>
<link><![CDATA[https://36kr.com/p/2222222222222222?f=rss]]></link>
<pubDate>2026-03-26 09:00:00 +0800</pubDate>
</item>
<item>
<title>OpenAI发布GPT-5,多模态能力大幅提升</title>
<link><![CDATA[https://36kr.com/p/3333333333333333?f=rss]]></link>
<pubDate>2026-03-25 20:00:00 +0800</pubDate>
</item>
</channel></rss>`;
afterEach(() => {
vi.restoreAllMocks();
});
describe('36kr/news RSS parsing', () => {
it('parses RSS feed into ranked news items', async () => {
vi.spyOn(globalThis, 'fetch').mockResolvedValue({
ok: true,
text: async () => SAMPLE_RSS,
} as Response);
// Direct RSS parse test using the same regex logic as news.ts
const xml = SAMPLE_RSS;
const items: { rank: number; title: string; date: string; url: string }[] = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < 10) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url =
block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ??
block.match(/<link>(.*?)<\/link>/)?.[1] ??
'';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
if (title) items.push({ rank: items.length + 1, title, date, url: url.trim() });
}
expect(items).toHaveLength(3);
expect(items[0].rank).toBe(1);
expect(items[0].title).toBe('红杉中国领投AI公司「示例」,金额近2亿元');
expect(items[0].date).toBe('2026-03-26');
expect(items[0].url).toBe('https://36kr.com/p/1111111111111111?f=rss');
});
it('respects limit — returns at most N items', async () => {
const xml = SAMPLE_RSS;
const limit = 2;
const items: { rank: number; title: string; date: string; url: string }[] = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < limit) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url = block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ?? '';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
if (title) items.push({ rank: items.length + 1, title, date, url: url.trim() });
}
expect(items).toHaveLength(2);
});
it('skips items with empty title', async () => {
const xml = `<rss><channel>
<item><title></title><link>https://36kr.com/p/0</link><pubDate>2026-01-01</pubDate></item>
<item><title>有标题的文章</title><link>https://36kr.com/p/1</link><pubDate>2026-01-01</pubDate></item>
</channel></rss>`;
const items: any[] = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml))) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
if (title) items.push({ title });
}
expect(items).toHaveLength(1);
expect(items[0].title).toBe('有标题的文章');
});
});
-54
View File
@@ -1,54 +0,0 @@
/**
* 36kr latest news — public RSS feed, no browser needed.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
cli({
site: '36kr',
name: 'news',
description: 'Latest tech/startup news from 36kr (36氪)',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
args: [
{ name: 'limit', type: 'int', default: 20, help: 'Number of articles (max 50)' },
],
columns: ['rank', 'title', 'summary', 'date', 'url'],
func: async (_page, kwargs) => {
const count = Math.min(kwargs.limit || 20, 50);
const resp = await fetch('https://www.36kr.com/feed', {
headers: { 'User-Agent': 'Mozilla/5.0 (compatible; opencli/1.0)' },
});
if (!resp.ok) return [];
const xml = await resp.text();
const items: { rank: number; title: string; summary: string; date: string; url: string }[] = [];
const itemRegex = /<item>([\s\S]*?)<\/item>/g;
let match;
while ((match = itemRegex.exec(xml)) && items.length < count) {
const block = match[1];
const title = block.match(/<title>([\s\S]*?)<\/title>/)?.[1]?.trim() ?? '';
const url =
block.match(/<link><!\[CDATA\[(.*?)\]\]>/)?.[1] ??
block.match(/<link>(.*?)<\/link>/)?.[1] ??
'';
const pubDate = block.match(/<pubDate>(.*?)<\/pubDate>/)?.[1]?.trim() ?? '';
const date = pubDate.slice(0, 10);
// Extract plain-text summary from HTML description (first ~120 chars)
const rawDesc = block.match(/<description><!\[CDATA\[([\s\S]*?)\]\]>/)?.[1] ?? '';
const summary = rawDesc
.replace(/<[^>]+>/g, ' ')
.replace(/&nbsp;/g, ' ')
.replace(/&amp;/g, '&')
.replace(/&lt;/g, '<')
.replace(/&gt;/g, '>')
.replace(/\s+/g, ' ')
.trim()
.slice(0, 120);
if (title) {
items.push({ rank: items.length + 1, title, summary, date, url: url.trim() });
}
}
return items;
},
});
+34 -39
View File
@@ -5,33 +5,30 @@
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import type { IPage } from '@jackwener/opencli/types';
cli({
site: '36kr',
name: 'search',
description: '搜索36氪文章',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'query', positional: true, required: true, help: 'Search keyword (e.g. "AI", "OpenAI")' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of results (max 50)' },
],
columns: ['rank', 'title', 'date', 'url'],
func: async (page: IPage, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const query = encodeURIComponent(String(args.query ?? ''));
await page.goto(`https://www.36kr.com/search/articles/${query}`);
// Poll DOM until article links appear (36kr renders client-side)
const deadline = Date.now() + 5000;
while (Date.now() < deadline) {
if (await page.evaluate('document.querySelectorAll("a[href*=\\"/p/\\"]").length')) break;
await new Promise(r => setTimeout(r, 300));
}
const domItems: any = await page.evaluate(`
site: '36kr',
name: 'search',
description: '搜索36氪文章',
domain: 'www.36kr.com',
strategy: Strategy.PUBLIC,
browser: true,
args: [
{ name: 'query', positional: true, required: true, help: 'Search keyword (e.g. "AI", "OpenAI")' },
{ name: 'limit', type: 'int', default: 20, help: 'Number of results (max 50)' },
],
columns: ['rank', 'title', 'date', 'url'],
func: async (page, args) => {
const count = Math.min(Number(args.limit) || 20, 50);
const query = encodeURIComponent(String(args.query ?? ''));
await page.goto(`https://www.36kr.com/search/articles/${query}`);
// Poll DOM until article links appear (36kr renders client-side)
const deadline = Date.now() + 5000;
while (Date.now() < deadline) {
if (await page.evaluate('document.querySelectorAll("a[href*=\\"/p/\\"]").length'))
break;
await new Promise(r => setTimeout(r, 300));
}
const domItems = await page.evaluate(`
(() => {
const seen = new Set();
const results = [];
@@ -67,17 +64,15 @@ cli({
return results;
})()
`);
const items = Array.isArray(domItems) ? (domItems as any[]) : [];
if (items.length === 0) {
throw new CliError('NO_DATA', 'No results found', `Try a different query or check your keyword`);
}
return items.slice(0, count).map((item: any, i: number) => ({
rank: i + 1,
title: item.title,
date: item.date,
url: item.url,
}));
},
const items = Array.isArray(domItems) ? domItems : [];
if (items.length === 0) {
throw new CliError('NO_DATA', 'No results found', `Try a different query or check your keyword`);
}
return items.slice(0, count).map((item, i) => ({
rank: i + 1,
title: item.title,
date: item.date,
url: item.url,
}));
},
});
+125
View File
@@ -0,0 +1,125 @@
/**
* 51job company jobs + basic info by encCoId.
*
* Navigates to `jobs.51job.com/all/co<encCoId>.html`. Each job card is an
* `<a sensorsdata="…">` whose attribute is a JSON blob with jobId, title,
* salary, area, year, degree — so parsing is just JSON, not DOM-text fragile.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import { JOBS_ORIGIN, requirePage, navigateTo, parseCompanyJobCard } from './utils.js';
cli({
site: '51job',
name: 'company',
description: '51job 公司简介 + 在招职位(按 encCoId',
domain: 'jobs.51job.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'encCoId', type: 'string', required: true, positional: true, help: '加密公司 IDsearch 返回的 encCoId' },
{ name: 'limit', type: 'int', default: 20, help: '返回职位数(1-50' },
],
columns: [
'rank', 'jobId', 'title', 'salary', 'city', 'workYear', 'degree',
'funcType', 'issueDate', 'url',
'companyName', 'companyType', 'companySize', 'companyIndustry',
'companyIntro', 'companyUrl',
],
func: async (page, kwargs) => {
requirePage(page);
const encCoId = String(kwargs.encCoId ?? '').trim();
if (!encCoId) throw new CliError('INVALID_ARGUMENT', 'encCoId is required');
if (!/^[A-Za-z0-9_]+$/.test(encCoId)) {
throw new CliError('INVALID_ARGUMENT', `encCoId must be alphanumeric/underscore, got "${encCoId}"`);
}
const limit = Math.max(1, Math.min(Number(kwargs.limit) || 20, 50));
const url = `${JOBS_ORIGIN}/all/co${encCoId}.html`;
await navigateTo(page, url, 2);
const script = `(() => {
const sel = s => document.querySelector(s)?.innerText?.trim() || '';
const bodyText = (document.body.innerText || '').slice(0, 400);
if (/公司不存在|页面不存在|账号状态异常/.test(bodyText)) {
return { error: 'NOT_FOUND', bodyText };
}
const companyName = sel('h1') || sel('.cname');
// Company introduction block
const introEl = document.querySelector('#companyIntroRef, .c-intro');
const companyIntro = introEl ? (introEl.innerText || '').trim() : '';
// Info sidebar (type / size / industry) — labels sit in .com-info dl or .coinfo
const sidebarText = sel('.ci-content, .company-info, .coinfo, .com-info');
const links = [...document.querySelectorAll('a[sensorsdata]')]
.filter(a => /\\/\\d{6,}\\.html/.test(a.href || ''))
.slice(0, 60)
.map(a => {
return {
href: a.href,
sensorsdata: a.getAttribute('sensorsdata') || '',
text: (a.innerText || '').trim(),
};
});
// Company meta is three inline spans under .c-info.ellipsis
// (title/size/industry) — extract them by position.
const cInfo = document.querySelector('.c-info.ellipsis');
const cInfoParts = cInfo
? [...cInfo.querySelectorAll('span')].map(s => (s.innerText || '').trim()).filter(Boolean)
: [];
return {
companyName,
companyIntro,
links,
cInfoParts,
sidebarText: sidebarText.slice(0, 400),
};
})()`;
const data = await page.evaluate(script);
if (data.error === 'NOT_FOUND') {
throw new CliError('NO_DATA', `Company ${encCoId} not found`);
}
if (!data.companyName) {
throw new CliError('NO_DATA', `Could not parse company page ${encCoId}; layout may have changed`);
}
const companyUrl = url;
const [companyType = '', companySize = '', companyIndustry = ''] = data.cInfoParts || [];
const seen = new Set();
const rows = [];
for (const link of data.links || []) {
const job = parseCompanyJobCard(link);
if (!job) continue;
if (seen.has(job.jobId)) continue;
seen.add(job.jobId);
rows.push({
rank: rows.length + 1,
...job,
companyName: data.companyName,
companyType,
companySize,
companyIndustry,
companyIntro: data.companyIntro || '',
companyUrl,
});
if (rows.length >= limit) break;
}
if (rows.length === 0) {
// Still return a sentinel row with the company info so caller isn't left with [].
return [{
rank: 0,
jobId: '',
title: '(no active jobs)',
salary: '', city: '', workYear: '', degree: '',
funcType: '', issueDate: '', url: '',
companyName: data.companyName,
companyType, companySize, companyIndustry,
companyIntro: data.companyIntro || '',
companyUrl,
}];
}
return rows;
},
});
+108
View File
@@ -0,0 +1,108 @@
/**
* 51job job detail by jobId.
*
* Navigates to `jobs.51job.com/x/<jobId>.html` (SSR page — the generic `/x/`
* area slug always resolves) and scrapes the structured blocks. No API
* surface returns the full detail page, so DOM scraping is the only path.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import { JOBS_ORIGIN, requirePage, navigateTo } from './utils.js';
cli({
site: '51job',
name: 'detail',
description: '51job 职位详情(按 jobId',
domain: 'jobs.51job.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'jobId', type: 'string', required: true, positional: true, help: '职位 IDsearch 返回的 jobId' },
],
columns: [
'jobId', 'title', 'salary', 'location', 'workYear', 'degree',
'category', 'address', 'ageRequirement',
'description', 'welfare',
'company', 'companyType', 'companySize', 'companyIndustry',
'companyUrl', 'url',
],
func: async (page, kwargs) => {
requirePage(page);
const jobId = String(kwargs.jobId ?? '').trim();
if (!jobId) throw new CliError('INVALID_ARGUMENT', 'jobId is required');
if (!/^\d{6,12}$/.test(jobId)) throw new CliError('INVALID_ARGUMENT', `jobId must be a 6-12 digit number, got "${jobId}"`);
const url = `${JOBS_ORIGIN}/x/${jobId}.html`;
await navigateTo(page, url, 2);
const script = `(() => {
const sel = s => document.querySelector(s)?.innerText?.trim() || '';
const all = s => [...document.querySelectorAll(s)].map(e => e.innerText.trim()).filter(Boolean);
const finalUrl = window.location.href;
const bodyText = (document.body.innerText || '').slice(0, 400);
if (/职位已下线|该职位已删除|页面不存在/.test(bodyText)) {
return { error: 'EXPIRED', bodyText };
}
const companyA = document.querySelector('.cname a, .tCompany_sidebar .com_msg a');
const funcs = all('.bmsg .fp');
const pick = (prefix) => {
const row = funcs.find(f => f.startsWith(prefix));
return row ? row.slice(prefix.length).replace(/^[:\\s\\n]+/, '').trim() : '';
};
return {
finalUrl,
title: sel('h1') || sel('.cn .name'),
salary: sel('.cn strong') || sel('strong'),
meta: sel('.cn .msg.ltype') || sel('.msg.ltype'),
description: (() => {
const box = document.querySelector('.bmsg.job_msg') || document.querySelector('.job_msg');
if (!box) return '';
const clone = box.cloneNode(true);
clone.querySelectorAll('.fp, .mt10, script, style').forEach(n => n.remove());
return (clone.innerText || '').trim();
})(),
welfare: all('.t1 span, .jtag .t1 span'),
category: pick('职能类别'),
address: pick('上班地址'),
ageRequirement: pick('年龄要求'),
company: companyA?.innerText?.trim() || '',
companyUrl: companyA?.href || '',
companyTag: sel('.com_tag'),
};
})()`;
const data = await page.evaluate(script);
if (data.error === 'EXPIRED') {
throw new CliError('NO_DATA', `Job ${jobId} is offline or removed`);
}
if (!data.title) {
throw new CliError('NO_DATA', `Could not parse job detail for ${jobId}; page may have changed layout`);
}
// meta looks like "北京-丰台区 | 3年及以上 | 本科"
const [locRaw, workYear, degree] = (data.meta || '').split('|').map(s => s.trim());
// companyTag looks like "国企\n\n150-500人\n\n电子技术/半导体/集成电路"
const tagParts = (data.companyTag || '').split(/\n+/).map(s => s.trim()).filter(Boolean);
return [{
jobId,
title: data.title,
salary: data.salary || '',
location: locRaw || '',
workYear: workYear || '',
degree: degree || '',
category: data.category || '',
address: data.address || '',
ageRequirement: data.ageRequirement || '',
description: data.description || '',
welfare: (data.welfare || []).join(','),
company: data.company || '',
companyType: tagParts[0] || '',
companySize: tagParts[1] || '',
companyIndustry: tagParts.slice(2).join(' / '),
companyUrl: data.companyUrl || '',
url: data.finalUrl || url,
}];
},
});
+55
View File
@@ -0,0 +1,55 @@
/**
* 51job hot / recommended feed.
*
* Same endpoint as `search`, but with empty keyword — 51job returns its
* own ranked recommendation list (up to ~999 for most regions).
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import {
WE_ORIGIN, SEARCH_COLUMNS, SORT_CODES,
requirePage, navigateTo, pageFetchJson,
buildSearchUrl, mapJobItem, resolveCity, resolveCode,
} from './utils.js';
cli({
site: '51job',
name: 'hot',
description: '51job 推荐职位(按城市/行业/排序浏览)',
domain: 'we.51job.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'area', type: 'string', default: '全国', help: '城市名或 6 位城市码(默认 "全国"' },
{ name: 'sort', type: 'string', default: '综合', help: '排序:综合 / 最新 / 薪资 / 距离' },
{ name: 'page', type: 'int', default: 1, help: '页码(1-based' },
{ name: 'limit', type: 'int', default: 20, help: '返回条数(1-50' },
],
columns: SEARCH_COLUMNS,
func: async (page, kwargs) => {
requirePage(page);
const limit = Math.max(1, Math.min(Number(kwargs.limit) || 20, 50));
const pageNum = Math.max(1, Number(kwargs.page) || 1);
const jobArea = resolveCity(kwargs.area);
const sortType = resolveCode(kwargs.sort, SORT_CODES, '0');
const currentUrl = await page.evaluate(`(() => window.location.href)()`);
if (!String(currentUrl).startsWith(WE_ORIGIN)) {
await navigateTo(page, `${WE_ORIGIN}/pc/search?searchType=2`, 2);
}
const url = buildSearchUrl({
keyword: '', jobArea, sortType,
pageNum, pageSize: Math.min(limit, 50),
});
const data = await pageFetchJson(page, url);
if (data.status !== '1' && data.status !== 1) {
throw new CliError('API_ERROR', `51job hot failed: ${data.message ?? 'unknown'}`);
}
const items = data?.resultbody?.job?.items ?? [];
if (items.length === 0) throw new CliError('NO_DATA', 'No recommended jobs returned');
return items.slice(0, limit).map((it, i) => mapJobItem(it, (pageNum - 1) * limit + i + 1));
},
});
+79
View File
@@ -0,0 +1,79 @@
/**
* 51job keyword search.
*
* Backed by `we.51job.com/api/job/search-pc`, which returns a job list with
* the full `jobDescribe` embedded. Needs the browser session because the
* Aliyun WAF in front of `we.51job.com` challenges bare fetches; the
* `pageFetchJson` helper runs inside the page so the WAF sees a real browser.
*/
import { cli, Strategy } from '@jackwener/opencli/registry';
import { CliError } from '@jackwener/opencli/errors';
import {
WE_ORIGIN, SEARCH_COLUMNS,
SALARY_CODES, WORKYEAR_CODES, DEGREE_CODES,
COMPANY_TYPE_CODES, COMPANY_SIZE_CODES, SORT_CODES,
requirePage, navigateTo, pageFetchJson,
buildSearchUrl, mapJobItem, resolveCity, resolveCode,
} from './utils.js';
cli({
site: '51job',
name: 'search',
description: '51job 前程无忧关键词职位搜索',
domain: 'we.51job.com',
strategy: Strategy.COOKIE,
browser: true,
navigateBefore: false,
args: [
{ name: 'keyword', type: 'string', required: true, positional: true, help: '搜索关键词(岗位名 / 技能 / 公司)' },
{ name: 'area', type: 'string', default: '全国', help: '城市名或 6 位城市码(如 "杭州" / "080200" / "全国"' },
{ name: 'salary', type: 'string', default: '', help: '薪资区间(如 "10-15k" / "1-1.5万" / "20-30k"' },
{ name: 'experience', type: 'string', default: '', help: '工作年限(如 "应届" / "1-3年" / "3-5年" / "5-7年"' },
{ name: 'degree', type: 'string', default: '', help: '学历要求(如 "本科" / "大专" / "硕士"' },
{ name: 'companyType', type: 'string', default: '', help: '公司性质(如 "外资" / "国企" / "民营"' },
{ name: 'companySize', type: 'string', default: '', help: '公司规模(如 "50-150" / "1000-5000"' },
{ name: 'sort', type: 'string', default: '综合', help: '排序:综合 / 最新 / 薪资 / 距离' },
{ name: 'page', type: 'int', default: 1, help: '页码(1-based' },
{ name: 'limit', type: 'int', default: 20, help: '返回条数(1-50' },
],
columns: SEARCH_COLUMNS,
func: async (page, kwargs) => {
requirePage(page);
const keyword = String(kwargs.keyword ?? '').trim();
if (!keyword) throw new CliError('INVALID_ARGUMENT', 'keyword is required');
const limit = Math.max(1, Math.min(Number(kwargs.limit) || 20, 50));
const pageNum = Math.max(1, Number(kwargs.page) || 1);
const jobArea = resolveCity(kwargs.area);
const salary = resolveCode(kwargs.salary, SALARY_CODES);
const workYear = resolveCode(kwargs.experience, WORKYEAR_CODES);
const degree = resolveCode(kwargs.degree, DEGREE_CODES);
const companyType = resolveCode(kwargs.companyType, COMPANY_TYPE_CODES);
const companySize = resolveCode(kwargs.companySize, COMPANY_SIZE_CODES);
const sortType = resolveCode(kwargs.sort, SORT_CODES, '0');
// Establish WAF-clean origin. Reusing the same tab avoids the slider
// challenge fire every call.
const currentUrl = await page.evaluate(`(() => window.location.href)()`);
if (!String(currentUrl).startsWith(WE_ORIGIN)) {
await navigateTo(page, `${WE_ORIGIN}/pc/search?keyword=${encodeURIComponent(keyword)}&searchType=2`, 2);
}
const url = buildSearchUrl({
keyword, jobArea, salary, workYear, degree,
companyType, companySize, sortType,
pageNum, pageSize: Math.min(limit, 50),
});
const data = await pageFetchJson(page, url);
if (data.status !== '1' && data.status !== 1) {
throw new CliError('API_ERROR', `51job search failed: ${data.message ?? 'unknown'}`);
}
const items = data?.resultbody?.job?.items ?? [];
if (items.length === 0) {
throw new CliError('NO_DATA', `No jobs matched "${keyword}"`);
}
return items.slice(0, limit).map((it, i) => mapJobItem(it, (pageNum - 1) * limit + i + 1));
},
});
+302
View File
@@ -0,0 +1,302 @@
/**
* 51job shared utilities.
*
* Key design points:
* - we.51job.com is protected by Aliyun WAF — bare `curl` / Node-side fetch
* gets a slider CAPTCHA HTML page. Only browser-context fetch (page.evaluate)
* with the session's cookies survives the challenge.
* - `document.cookie` exposes the anti-bot cookies (`acw_sc__v2`, `ssxmod_itna`
* etc.) — no HttpOnly/login needed for public pages.
* - API (`we.51job.com/api/job/search-pc`) is same-origin when we've navigated
* to `https://we.51job.com/...`, so fetch inside page.evaluate works.
* - Detail / company pages live on `jobs.51job.com` and render data into the
* DOM (SSR), so adapters for those navigate and scrape.
*/
import { CliError } from '@jackwener/opencli/errors';
export const WE_ORIGIN = 'https://we.51job.com';
export const JOBS_ORIGIN = 'https://jobs.51job.com';
/**
* City name / alias → 6-digit jobArea code. `000000` is the national bucket.
* Covers the 40 largest cities the search UI surfaces. Unknown input passed
* as-is if it's already 6 digits; otherwise fall back to `000000` (all).
*/
export const CITY_CODES = {
'全国': '000000', 'all': '000000',
'北京': '010000', 'beijing': '010000',
'上海': '020000', 'shanghai': '020000',
'广州': '030200', 'guangzhou': '030200',
'深圳': '040000', 'shenzhen': '040000',
'武汉': '180200', 'wuhan': '180200',
'西安': '200200', "xi'an": '200200', 'xian': '200200',
'杭州': '080200', 'hangzhou': '080200',
'南京': '070200', 'nanjing': '070200',
'成都': '090200', 'chengdu': '090200',
'苏州': '070300', 'suzhou': '070300',
'重庆': '060000', 'chongqing': '060000',
'天津': '050000', 'tianjin': '050000',
'长沙': '190200', 'changsha': '190200',
'郑州': '170200', 'zhengzhou': '170200',
'青岛': '120300', 'qingdao': '120300',
'合肥': '150200', 'hefei': '150200',
'厦门': '110300', 'xiamen': '110300',
'无锡': '070400', 'wuxi': '070400',
'济南': '120200', 'jinan': '120200',
'佛山': '030700', 'foshan': '030700',
'东莞': '030800', 'dongguan': '030800',
'宁波': '080300', 'ningbo': '080300',
'福州': '110200', 'fuzhou': '110200',
'昆明': '250200', 'kunming': '250200',
'大连': '230300', 'dalian': '230300',
'沈阳': '230200', 'shenyang': '230200',
'哈尔滨': '220200', 'haerbin': '220200', 'harbin': '220200',
'石家庄': '160200', 'shijiazhuang': '160200',
'贵阳': '260200', 'guiyang': '260200',
'南宁': '100200', 'nanning': '100200',
'南昌': '130200', 'nanchang': '130200',
'长春': '240200', 'changchun': '240200',
'太原': '210200', 'taiyuan': '210200',
'兰州': '280200', 'lanzhou': '280200',
'乌鲁木齐': '310200', 'urumqi': '310200',
'海口': '270200', 'haikou': '270200',
'香港': '330000', 'hongkong': '330000', 'hk': '330000',
};
/** Salary bucket code (matches 51job's `salary` filter). */
export const SALARY_CODES = {
'不限': '',
'2千以下': '01', '2-3千': '02', '3-4.5千': '03',
'4.5-6千': '04', '6-8千': '05', '8k-1万': '06', '8-10k': '06',
'1-1.5万': '07', '10-15k': '07',
'1.5-2万': '08', '15-20k': '08',
'2-3万': '09', '20-30k': '09',
'3-5万': '10', '30-50k': '10',
'5万以上': '11', '50k以上': '11',
};
/** Work experience bucket. */
export const WORKYEAR_CODES = {
'不限': '',
'在校生': '01', '应届': '02', '1年以下': '03',
'1-3年': '04', '3-5年': '05', '5-7年': '06',
'7-10年': '07', '10年以上': '08',
};
/** Degree bucket. */
export const DEGREE_CODES = {
'不限': '',
'初中及以下': '01', '高中/中技/中专': '02', '高中': '02',
'大专': '03', '本科': '04', '硕士': '05', '博士': '06',
};
/** Company ownership type. */
export const COMPANY_TYPE_CODES = {
'不限': '',
'外资': '01', '欧美': '0101', '日韩': '0102',
'合资': '02', '国企': '03', '民营': '04',
'上市公司': '05', '创业公司': '06', '事业单位': '07',
'非营利': '08', '政府': '09',
};
/** Company headcount bucket. */
export const COMPANY_SIZE_CODES = {
'不限': '',
'少于50': '01', '50以下': '01',
'50-150': '02', '150-500': '03',
'500-1000': '04', '1000-5000': '05',
'5000-10000': '06', '10000以上': '07',
};
/** Sort strategy. */
export const SORT_CODES = {
'综合': '0', 'relevance': '0', 'default': '0',
'最新': '1', 'new': '1', 'newest': '1',
'薪资': '2', 'salary': '2', 'pay': '2',
'距离': '9', 'distance': '9',
};
export function resolveCity(input) {
if (!input) return '000000';
const s = String(input).trim();
if (!s || s === '全国' || s.toLowerCase() === 'all') return '000000';
if (/^\d{6}$/.test(s)) return s;
const key = s.toLowerCase();
if (CITY_CODES[s] !== undefined) return CITY_CODES[s];
if (CITY_CODES[key] !== undefined) return CITY_CODES[key];
for (const [name, code] of Object.entries(CITY_CODES)) {
if (typeof name === 'string' && name.includes(s)) return code;
}
throw new CliError('INVALID_ARGUMENT', `Unknown city/area "${s}"`, 'Use a supported city name like "杭州" or a 6-digit city code');
}
export function resolveCode(input, table, fallback = '') {
if (input === undefined || input === null || input === '') return fallback;
const s = String(input).trim();
if (table[s] !== undefined) return table[s];
const key = s.toLowerCase();
if (table[key] !== undefined) return table[key];
if (Object.values(table).includes(s)) return s;
for (const [k, v] of Object.entries(table)) {
if (typeof k === 'string' && k.includes(s)) return v;
}
return fallback;
}
export function requirePage(page) {
if (!page) throw new CliError('INTERNAL_ERROR', 'Browser page required (adapter must set browser: true)');
}
/**
* Navigate the page to a URL and give the SPA a moment to settle. Reuses
* existing session cookies — first call on a fresh browser may trigger the
* Aliyun WAF interstitial, which the headless Chromium solves automatically
* because the JS that sets `acw_sc__v2` runs in the page.
*/
export async function navigateTo(page, url, waitSeconds = 2) {
await page.goto(url);
await page.wait({ time: waitSeconds });
}
/**
* Browser-context fetch: execute `fetch(url, { credentials: 'include' })`
* inside the page so cookies apply and WAF sees a real browser. Returns
* parsed JSON; throws on network / parse / status failure.
*/
export async function pageFetchJson(page, url, opts = {}) {
const method = opts.method ?? 'GET';
const body = opts.body ?? null;
const timeout = opts.timeout ?? 15000;
const headers = opts.headers ?? {};
const script = `
async () => {
const ctrl = new AbortController();
const timer = setTimeout(() => ctrl.abort(), ${timeout});
try {
const resp = await fetch(${JSON.stringify(url)}, {
method: ${JSON.stringify(method)},
credentials: 'include',
headers: ${JSON.stringify({ Accept: 'application/json', ...headers })},
${body !== null ? `body: ${JSON.stringify(body)},` : ''}
signal: ctrl.signal,
});
const text = await resp.text();
return { ok: resp.ok, status: resp.status, text };
} catch (e) {
return { ok: false, status: 0, text: '', error: String(e && e.message || e) };
} finally {
clearTimeout(timer);
}
}
`;
const res = await page.evaluate(script);
if (res.error) throw new CliError('HTTP_ERROR', `51job fetch failed: ${res.error}`);
if (!res.ok) throw new CliError('HTTP_ERROR', `51job HTTP ${res.status}`);
if (res.text.trim().startsWith('<')) {
throw new CliError('ANTI_BOT', '51job returned HTML (likely Aliyun WAF slider). Refresh browser session.');
}
try {
return JSON.parse(res.text);
} catch (e) {
throw new CliError('API_ERROR', `51job invalid JSON: ${res.text.slice(0, 200)}`);
}
}
/**
* Build the canonical search-pc URL. All optional filters default to empty
* (no constraint). `scene=7` + `source=1` match what the real SPA sends.
*/
export function buildSearchUrl(params) {
const qs = new URLSearchParams();
qs.set('api_key', '51job');
qs.set('timestamp', String(Date.now()));
qs.set('keyword', params.keyword ?? '');
qs.set('searchType', '2');
qs.set('function', params.function ?? '');
qs.set('industry', params.industry ?? '');
qs.set('jobArea', params.jobArea ?? '000000');
qs.set('jobArea2', params.jobArea2 ?? '');
qs.set('landmark', params.landmark ?? '');
qs.set('metro', params.metro ?? '');
qs.set('salary', params.salary ?? '');
qs.set('workYear', params.workYear ?? '');
qs.set('degree', params.degree ?? '');
qs.set('companyType', params.companyType ?? '');
qs.set('companySize', params.companySize ?? '');
qs.set('jobType', params.jobType ?? '');
qs.set('issueDate', params.issueDate ?? '');
qs.set('sortType', params.sortType ?? '0');
qs.set('pageNum', String(params.pageNum ?? 1));
qs.set('pageSize', String(params.pageSize ?? 20));
qs.set('source', '1');
qs.set('scene', '7');
return `${WE_ORIGIN}/api/job/search-pc?${qs.toString()}`;
}
/**
* Map a raw search-pc `resultbody.job.items[i]` into the canonical row shape
* we expose to the user. Kept here so `search` and `hot` stay aligned.
*/
export function mapJobItem(it, rank) {
const area = it.jobAreaLevelDetail || {};
return {
rank,
jobId: String(it.jobId ?? ''),
title: it.jobName ?? '',
salary: it.provideSalaryString ?? '',
salaryMin: Number(it.jobSalaryMin ?? 0) || 0,
salaryMax: Number(it.jobSalaryMax ?? 0) || 0,
city: area.cityString ?? it.jobAreaString ?? '',
district: area.districtString ?? '',
workYear: it.workYearString ?? '',
degree: it.degreeString ?? '',
tags: Array.isArray(it.jobTags) ? it.jobTags.join(',') : '',
company: it.companyName ?? '',
companyFull: it.fullCompanyName ?? '',
companyType: it.companyTypeString ?? '',
companySize: it.companySizeString ?? '',
industry: it.industryType1Str ?? '',
hr: it.hrName ? `${it.hrName}·${it.hrPosition ?? ''}` : '',
issueDate: it.issueDateString ?? '',
url: it.jobHref ?? '',
companyUrl: it.companyHref ?? '',
encCoId: it.encCoId ?? '',
};
}
export const SEARCH_COLUMNS = [
'rank', 'jobId', 'title', 'salary', 'salaryMin', 'salaryMax',
'city', 'district', 'workYear', 'degree', 'tags',
'company', 'companyFull', 'companyType', 'companySize', 'industry',
'hr', 'issueDate', 'url', 'companyUrl', 'encCoId',
];
/**
* Parse a 51job company-page `<a sensorsdata="...">` payload into a stable
* row fragment. Returns null when the attribute is absent or malformed.
*/
export function parseCompanyJobCard(raw) {
if (!raw || typeof raw !== 'object') return null;
const href = typeof raw.href === 'string' ? raw.href : '';
const sensorsdata = typeof raw.sensorsdata === 'string' ? raw.sensorsdata : '';
if (!href || !sensorsdata) return null;
let data;
try {
data = JSON.parse(sensorsdata);
} catch {
return null;
}
if (!data || !data.jobId) return null;
return {
jobId: String(data.jobId),
title: data.jobTitle || '',
salary: data.jobSalary || '',
city: data.jobArea || '',
workYear: data.jobYear || '',
degree: data.jobDegree || '',
funcType: data.funcType || '',
issueDate: data.jobTime || '',
url: href,
};
}
+69
View File
@@ -0,0 +1,69 @@
import { describe, expect, it, vi } from 'vitest';
import { CliError } from '@jackwener/opencli/errors';
import { parseCompanyJobCard, pageFetchJson, resolveCity } from './utils.js';
describe('51job resolveCity', () => {
it('maps known city names and explicit national scope', () => {
expect(resolveCity('杭州')).toBe('080200');
expect(resolveCity('all')).toBe('000000');
expect(resolveCity('000000')).toBe('000000');
});
it('rejects unknown non-empty inputs instead of silently widening to 全国', () => {
expect(() => resolveCity('杭州z')).toThrowError(CliError);
expect(() => resolveCity('杭州z')).toThrow(/Unknown city\/area/);
});
});
describe('51job pageFetchJson', () => {
it('detects WAF challenge HTML and throws ANTI_BOT', async () => {
const page = {
evaluate: vi.fn().mockResolvedValue({
ok: true,
status: 200,
text: '<html><title>slider</title></html>',
}),
};
await expect(pageFetchJson(page, 'https://we.51job.com/api/job/search-pc')).rejects.toMatchObject({
code: 'ANTI_BOT',
});
});
});
describe('51job parseCompanyJobCard', () => {
it('parses sensorsdata JSON into a stable row fragment', () => {
const row = parseCompanyJobCard({
href: 'https://jobs.51job.com/shanghai/123456789.html',
sensorsdata: JSON.stringify({
jobId: '123456789',
jobTitle: 'Senior Engineer',
jobSalary: '20-30K',
jobArea: '上海',
jobYear: '3-5年',
jobDegree: '本科',
funcType: '后端开发',
jobTime: '04-22',
}),
});
expect(row).toEqual({
jobId: '123456789',
title: 'Senior Engineer',
salary: '20-30K',
city: '上海',
workYear: '3-5年',
degree: '本科',
funcType: '后端开发',
issueDate: '04-22',
url: 'https://jobs.51job.com/shanghai/123456789.html',
});
});
it('returns null on malformed sensorsdata', () => {
expect(parseCompanyJobCard({
href: 'https://jobs.51job.com/shanghai/123456789.html',
sensorsdata: '{bad json}',
})).toBeNull();
});
});
+32
View File
@@ -0,0 +1,32 @@
/**
* Shared utilities for CLI adapters.
*/
import { ArgumentError } from '@jackwener/opencli/errors';
/**
* Clamp a numeric value to [min, max].
* Matches the signature of lodash.clamp and Rust's clamp.
*/
export function clamp(value, min, max) {
return Math.max(min, Math.min(value, max));
}
export function clampInt(raw, fallback, min, max) {
const parsed = Number(raw);
if (!Number.isFinite(parsed)) {
return fallback;
}
return clamp(Math.floor(parsed), min, max);
}
export function normalizeNumericId(value, label, example) {
const normalized = String(value ?? '').trim();
if (!/^\d+$/.test(normalized)) {
throw new ArgumentError(`${label} must be a numeric ID`, `Pass a numeric ${label}, for example: ${example}`);
}
return normalized;
}
export function requireNonEmptyQuery(value, label = 'query') {
const normalized = String(value ?? '').trim();
if (!normalized) {
throw new ArgumentError(`${label} cannot be empty`);
}
return normalized;
}
-11
View File
@@ -1,11 +0,0 @@
/**
* Shared utilities for CLI adapters.
*/
/**
* Clamp a numeric value to [min, max].
* Matches the signature of lodash.clamp and Rust's clamp.
*/
export function clamp(value: number, min: number, max: number): number {
return Math.max(min, Math.min(value, max));
}
+108
View File
@@ -0,0 +1,108 @@
/**
* Shared command factories for Electron/desktop app adapters.
* Eliminates duplicate screenshot/status/new/dump implementations
* across cursor, codex, chatwise, etc.
*/
import * as fs from 'node:fs';
import { cli, Strategy } from '@jackwener/opencli/registry';
/**
* Factory: capture DOM HTML + accessibility snapshot.
*/
export function makeScreenshotCommand(site, displayName, extra = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'screenshot',
description: `Capture a snapshot of the current ${label} window (DOM + Accessibility tree)`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'output', required: false, help: `Output file path (default: /tmp/${site}-snapshot.txt)` },
],
columns: ['Status', 'File'],
func: async (page, kwargs) => {
const outputPath = kwargs.output || `/tmp/${site}-snapshot.txt`;
const snap = await page.snapshot({ compact: true });
const html = await page.evaluate('document.documentElement.outerHTML');
const htmlPath = outputPath.replace(/\.\w+$/, '') + '-dom.html';
const snapPath = outputPath.replace(/\.\w+$/, '') + '-a11y.txt';
fs.writeFileSync(htmlPath, html);
fs.writeFileSync(snapPath, typeof snap === 'string' ? snap : JSON.stringify(snap, null, 2));
return [
{ Status: 'Success', File: htmlPath },
{ Status: 'Success', File: snapPath },
];
},
});
}
/**
* Factory: check CDP connection status.
*/
export function makeStatusCommand(site, displayName, extra = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'status',
description: `Check active CDP connection to ${label}`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['Status', 'Url', 'Title'],
func: async (page) => {
const url = await page.evaluate('window.location.href');
const title = await page.evaluate('document.title');
return [{ Status: 'Connected', Url: url, Title: title }];
},
});
}
/**
* Factory: start a new session via Cmd/Ctrl+N.
*/
export function makeNewCommand(site, displayName, extra = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'new',
description: `Start a new ${label} session`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['Status'],
func: async (page) => {
const isMac = process.platform === 'darwin';
await page.pressKey(isMac ? 'Meta+N' : 'Control+N');
await page.wait(1);
return [{ Status: 'Success' }];
},
});
}
/**
* Factory: dump DOM + snapshot for reverse-engineering.
*/
export function makeDumpCommand(site) {
return cli({
site,
name: 'dump',
description: `Dump the DOM and Accessibility tree of ${site} for reverse-engineering`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['action', 'files'],
func: async (page) => {
const dom = await page.evaluate('document.body.innerHTML');
fs.writeFileSync(`/tmp/${site}-dom.html`, dom);
const snap = await page.snapshot({ interactive: false });
fs.writeFileSync(`/tmp/${site}-snapshot.json`, JSON.stringify(snap, null, 2));
return [
{
action: 'Dom extraction finished',
files: `/tmp/${site}-dom.html, /tmp/${site}-snapshot.json`,
},
];
},
});
}
-121
View File
@@ -1,121 +0,0 @@
/**
* Shared command factories for Electron/desktop app adapters.
* Eliminates duplicate screenshot/status/new/dump implementations
* across cursor, codex, chatwise, etc.
*/
import * as fs from 'node:fs';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import type { CliOptions } from '@jackwener/opencli/registry';
/**
* Factory: capture DOM HTML + accessibility snapshot.
*/
export function makeScreenshotCommand(site: string, displayName?: string, extra: Partial<CliOptions> = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'screenshot',
description: `Capture a snapshot of the current ${label} window (DOM + Accessibility tree)`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
args: [
{ name: 'output', required: false, help: `Output file path (default: /tmp/${site}-snapshot.txt)` },
],
columns: ['Status', 'File'],
func: async (page: IPage, kwargs: any) => {
const outputPath = (kwargs.output as string) || `/tmp/${site}-snapshot.txt`;
const snap = await page.snapshot({ compact: true });
const html = await page.evaluate('document.documentElement.outerHTML');
const htmlPath = outputPath.replace(/\.\w+$/, '') + '-dom.html';
const snapPath = outputPath.replace(/\.\w+$/, '') + '-a11y.txt';
fs.writeFileSync(htmlPath, html);
fs.writeFileSync(snapPath, typeof snap === 'string' ? snap : JSON.stringify(snap, null, 2));
return [
{ Status: 'Success', File: htmlPath },
{ Status: 'Success', File: snapPath },
];
},
});
}
/**
* Factory: check CDP connection status.
*/
export function makeStatusCommand(site: string, displayName?: string, extra: Partial<CliOptions> = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'status',
description: `Check active CDP connection to ${label}`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['Status', 'Url', 'Title'],
func: async (page: IPage) => {
const url = await page.evaluate('window.location.href');
const title = await page.evaluate('document.title');
return [{ Status: 'Connected', Url: url, Title: title }];
},
});
}
/**
* Factory: start a new session via Cmd/Ctrl+N.
*/
export function makeNewCommand(site: string, displayName?: string, extra: Partial<CliOptions> = {}) {
const label = displayName ?? site;
return cli({
...extra,
site,
name: 'new',
description: `Start a new ${label} session`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['Status'],
func: async (page: IPage) => {
const isMac = process.platform === 'darwin';
await page.pressKey(isMac ? 'Meta+N' : 'Control+N');
await page.wait(1);
return [{ Status: 'Success' }];
},
});
}
/**
* Factory: dump DOM + snapshot for reverse-engineering.
*/
export function makeDumpCommand(site: string) {
return cli({
site,
name: 'dump',
description: `Dump the DOM and Accessibility tree of ${site} for reverse-engineering`,
domain: 'localhost',
strategy: Strategy.UI,
browser: true,
columns: ['action', 'files'],
func: async (page: IPage) => {
const dom = await page.evaluate('document.body.innerHTML');
fs.writeFileSync(`/tmp/${site}-dom.html`, dom);
const snap = await page.snapshot({ interactive: false });
fs.writeFileSync(`/tmp/${site}-snapshot.json`, JSON.stringify(snap, null, 2));
return [
{
action: 'Dom extraction finished',
files: `/tmp/${site}-dom.html, /tmp/${site}-snapshot.json`,
},
];
},
});
}
@@ -1,8 +1,7 @@
import { cli } from '@jackwener/opencli/registry';
import { createRankingCliOptions } from './rankings.js';
cli(createRankingCliOptions({
commandName: 'new-releases',
listType: 'new_releases',
description: 'Amazon New Releases pages for early momentum discovery',
commandName: 'bestsellers',
listType: 'bestsellers',
description: 'Amazon Best Sellers pages for category candidate discovery',
}));
+29
View File
@@ -0,0 +1,29 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './rankings.js';
describe('amazon bestsellers normalization', () => {
it('normalizes bestseller cards and infers review counts from card text', () => {
const result = __test__.normalizeRankingCandidate({
asin: 'B0DR31GC3D',
title: '',
href: 'https://www.amazon.com/NUTIKAS-Shelves-Desktop-Orgnizer-Shlef/dp/B0DR31GC3D/ref=zg_bs',
price_text: '$25.92',
rating_text: '4.3 out of 5 stars',
review_count_text: '',
card_text: 'Desk Shelves Desktop Organizer Shlef\n4.3 out of 5 stars\n435\n$25.92',
}, {
listType: 'bestsellers',
rankFallback: 2,
listTitle: 'Amazon Best Sellers: Best Desktop & Off-Surface Shelves',
sourceUrl: 'https://www.amazon.com/example',
categoryTitle: null,
categoryUrl: 'https://www.amazon.com/example',
categoryPath: [],
visibleCategoryLinks: [],
});
expect(result.rank).toBe(2);
expect(result.asin).toBe('B0DR31GC3D');
expect(result.title).toBe('Desk Shelves Desktop Organizer Shlef');
expect(result.review_count).toBe(435);
expect(result.list_title).toBe('Amazon Best Sellers: Best Desktop & Off-Surface Shelves');
});
});
-31
View File
@@ -1,31 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './rankings.js';
describe('amazon bestsellers normalization', () => {
it('normalizes bestseller cards and infers review counts from card text', () => {
const result = __test__.normalizeRankingCandidate({
asin: 'B0DR31GC3D',
title: '',
href: 'https://www.amazon.com/NUTIKAS-Shelves-Desktop-Orgnizer-Shlef/dp/B0DR31GC3D/ref=zg_bs',
price_text: '$25.92',
rating_text: '4.3 out of 5 stars',
review_count_text: '',
card_text: 'Desk Shelves Desktop Organizer Shlef\n4.3 out of 5 stars\n435\n$25.92',
}, {
listType: 'bestsellers',
rankFallback: 2,
listTitle: 'Amazon Best Sellers: Best Desktop & Off-Surface Shelves',
sourceUrl: 'https://www.amazon.com/example',
categoryTitle: null,
categoryUrl: 'https://www.amazon.com/example',
categoryPath: [],
visibleCategoryLinks: [],
});
expect(result.rank).toBe(2);
expect(result.asin).toBe('B0DR31GC3D');
expect(result.title).toBe('Desk Shelves Desktop Organizer Shlef');
expect(result.review_count).toBe(435);
expect(result.list_title).toBe('Amazon Best Sellers: Best Desktop & Off-Surface Shelves');
});
});
+122
View File
@@ -0,0 +1,122 @@
import { AuthRequiredError, CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { buildProductUrl, buildDiscussionUrl, buildProvenance, cleanText, extractAsin, normalizeProductUrl, parseRatingValue, parseReviewCount, trimRatingPrefix, uniqueNonEmpty, assertUsableState, gotoAndReadState, } from './shared.js';
function normalizeDiscussionPayload(payload) {
const sourceUrl = cleanText(payload.href) || buildDiscussionUrl(payload.href ?? '');
const asin = extractAsin(payload.href ?? '') ?? null;
const averageRatingText = cleanText(payload.average_rating_text) || null;
const totalReviewCountText = cleanText(payload.total_review_count_text) || null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
product_url: asin ? normalizeProductUrl(asin) : null,
discussion_url: sourceUrl,
...provenance,
average_rating_text: averageRatingText,
average_rating_value: parseRatingValue(averageRatingText),
total_review_count_text: totalReviewCountText,
total_review_count: parseReviewCount(totalReviewCountText),
qa_urls: uniqueNonEmpty(payload.qa_links ?? []),
review_samples: (payload.review_samples ?? []).map((sample) => ({
title: trimRatingPrefix(sample.title) || null,
rating_text: cleanText(sample.rating_text) || null,
rating_value: parseRatingValue(sample.rating_text),
author: cleanText(sample.author) || null,
date_text: cleanText(sample.date_text) || null,
body: cleanText(sample.body) || null,
verified_purchase: sample.verified === true,
})),
};
}
function hasDiscussionSummary(payload) {
return Boolean(cleanText(payload.average_rating_text) || cleanText(payload.total_review_count_text));
}
function isSignInState(state) {
const href = cleanText(state.href).toLowerCase();
const title = cleanText(state.title).toLowerCase();
return href.includes('/ap/signin')
|| title.includes('amazon sign-in');
}
async function readCurrentDiscussionPayload(page, limit) {
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
average_rating_text: document.querySelector('[data-hook="rating-out-of-text"]')?.textContent || '',
total_review_count_text: document.querySelector('[data-hook="total-review-count"]')?.textContent || '',
qa_links: Array.from(document.querySelectorAll('a[href*="ask/questions"]')).map((anchor) => anchor.href || ''),
review_samples: Array.from(document.querySelectorAll('[data-hook="review"]')).slice(0, ${limit}).map((card) => ({
title: card.querySelector('[data-hook="review-title"]')?.textContent || '',
rating_text:
card.querySelector('[data-hook="review-star-rating"]')?.textContent
|| card.querySelector('[data-hook="cmps-review-star-rating"]')?.textContent
|| '',
author: card.querySelector('.a-profile-name')?.textContent || '',
date_text: card.querySelector('[data-hook="review-date"]')?.textContent || '',
body: card.querySelector('[data-hook="review-body"]')?.textContent || '',
verified: !!card.querySelector('[data-hook="avp-badge"]'),
})),
}))()
`);
}
async function readDiscussionPayload(page, input, limit) {
const reviewUrl = buildDiscussionUrl(input);
const reviewState = await gotoAndReadState(page, reviewUrl, 2500, 'discussion');
assertUsableState(reviewState, 'discussion');
const reviewPayload = await readCurrentDiscussionPayload(page, limit);
if (hasDiscussionSummary(reviewPayload)) {
return reviewPayload;
}
const productUrl = buildProductUrl(input);
const productState = await gotoAndReadState(page, productUrl, 2500, 'discussion');
assertUsableState(productState, 'discussion');
if (isSignInState(reviewState) && isSignInState(productState)) {
throw new AuthRequiredError('amazon.com', 'Amazon review discussion requires an active signed-in Amazon session in the shared Chrome profile.');
}
const productPayload = await readCurrentDiscussionPayload(page, limit);
if (hasDiscussionSummary(productPayload)) {
return productPayload;
}
if (isSignInState(reviewState)) {
throw new CommandExecutionError('amazon review page redirected to sign-in and product page fallback did not expose review summary', 'Open the product page in Chrome, verify reviews are visible, and retry.');
}
return reviewPayload;
}
cli({
site: 'amazon',
name: 'discussion',
description: 'Amazon review summary and sample customer discussion from product review pages',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
{
name: 'limit',
type: 'int',
default: 10,
help: 'Maximum number of review samples to return (default 10)',
},
],
columns: ['asin', 'average_rating_value', 'total_review_count'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const limit = Math.max(1, Number(kwargs.limit) || 10);
const payload = await readDiscussionPayload(page, input, limit);
const normalized = normalizeDiscussionPayload(payload);
if (!normalized.average_rating_text && !normalized.total_review_count_text) {
throw new CommandExecutionError('amazon discussion page did not expose review summary', 'The review page may have changed or hit a robot check. Open the review page in Chrome and retry.');
}
return [normalized];
},
});
export const __test__ = {
normalizeDiscussionPayload,
hasDiscussionSummary,
isSignInState,
};
+151
View File
@@ -0,0 +1,151 @@
import { describe, expect, it, vi } from 'vitest';
import { AuthRequiredError } from '@jackwener/opencli/errors';
import { getRegistry } from '@jackwener/opencli/registry';
import { __test__ } from './discussion.js';
import './discussion.js';
function createPageMock(evaluateResults) {
const evaluate = vi.fn();
for (const result of evaluateResults) {
evaluate.mockResolvedValueOnce(result);
}
return {
goto: vi.fn().mockResolvedValue(undefined),
wait: vi.fn().mockResolvedValue(undefined),
evaluate,
snapshot: vi.fn().mockResolvedValue(undefined),
click: vi.fn().mockResolvedValue(undefined),
typeText: vi.fn().mockResolvedValue(undefined),
pressKey: vi.fn().mockResolvedValue(undefined),
scrollTo: vi.fn().mockResolvedValue(undefined),
getFormState: vi.fn().mockResolvedValue({ forms: [], orphanFields: [] }),
tabs: vi.fn().mockResolvedValue([]),
selectTab: vi.fn().mockResolvedValue(undefined),
networkRequests: vi.fn().mockResolvedValue([]),
consoleMessages: vi.fn().mockResolvedValue([]),
scroll: vi.fn().mockResolvedValue(undefined),
autoScroll: vi.fn().mockResolvedValue(undefined),
installInterceptor: vi.fn().mockResolvedValue(undefined),
getInterceptedRequests: vi.fn().mockResolvedValue([]),
getCookies: vi.fn().mockResolvedValue([]),
screenshot: vi.fn().mockResolvedValue(''),
waitForCapture: vi.fn().mockResolvedValue(undefined),
};
}
describe('amazon discussion normalization', () => {
it('normalizes review summary and sample reviews', () => {
const result = __test__.normalizeDiscussionPayload({
href: 'https://www.amazon.com/product-reviews/B0FJS72893',
average_rating_text: '3.9 out of 5',
total_review_count_text: '27 global ratings',
qa_links: [],
review_samples: [
{
title: '5.0 out of 5 stars Great value and quality',
rating_text: '5.0 out of 5 stars',
author: 'GTreader2',
date_text: 'Reviewed in the United States on February 21, 2026',
body: 'Small but mighty.',
verified: true,
},
],
});
expect(result.asin).toBe('B0FJS72893');
expect(result.average_rating_value).toBe(3.9);
expect(result.total_review_count).toBe(27);
expect(result.review_samples).toEqual([
{
title: 'Great value and quality',
rating_text: '5.0 out of 5 stars',
rating_value: 5,
author: 'GTreader2',
date_text: 'Reviewed in the United States on February 21, 2026',
body: 'Small but mighty.',
verified_purchase: true,
},
]);
});
it('falls back to the product page when the review page redirects to sign-in', async () => {
const command = getRegistry().get('amazon/discussion');
const page = createPageMock([
{
href: 'https://www.amazon.com/ap/signin?openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fproduct-reviews%2FB09HKN2ZRT',
title: 'Amazon Sign-In',
body_text: 'Sign in Create account',
},
{
href: 'https://www.amazon.com/ap/signin?openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fproduct-reviews%2FB09HKN2ZRT',
average_rating_text: '',
total_review_count_text: '',
review_samples: [],
},
{
href: 'https://www.amazon.com/dp/B09HKN2ZRT',
title: 'Amazon.com: Example product',
body_text: 'Hello, zejia-wu Reviews',
},
{
href: 'https://www.amazon.com/dp/B09HKN2ZRT',
average_rating_text: '4.4 out of 5',
total_review_count_text: '349 global ratings',
review_samples: [
{
title: '5.0 out of 5 stars Perfect for the office',
rating_text: '5.0 out of 5 stars',
author: 'Ken',
date_text: 'Reviewed in the United States on March 19, 2026',
body: 'Good for the office, no complaints.',
verified: true,
},
],
},
]);
const result = await command.func(page, { input: 'B09HKN2ZRT', limit: 1 });
expect(page.goto.mock.calls.map((call) => call[0])).toEqual([
'https://www.amazon.com/product-reviews/B09HKN2ZRT',
'https://www.amazon.com/dp/B09HKN2ZRT',
]);
expect(result).toEqual([
expect.objectContaining({
asin: 'B09HKN2ZRT',
discussion_url: 'https://www.amazon.com/dp/B09HKN2ZRT',
average_rating_value: 4.4,
total_review_count: 349,
}),
]);
});
it('throws AuthRequiredError when both review and product pages are gated', async () => {
const command = getRegistry().get('amazon/discussion');
const authState = {
href: 'https://www.amazon.com/ap/signin?openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fproduct-reviews%2FB09HKN2ZRT',
title: 'Amazon Sign-In',
body_text: 'Sign in Create account',
};
const page = createPageMock([
authState,
{
href: authState.href,
average_rating_text: '',
total_review_count_text: '',
review_samples: [],
},
authState,
]);
await expect(command.func(page, { input: 'B09HKN2ZRT', limit: 1 })).rejects.toBeInstanceOf(AuthRequiredError);
});
it('does not treat a public product page with sign-in copy as a gated page', () => {
expect(__test__.isSignInState({
href: 'https://www.amazon.com/dp/B09HKN2ZRT',
title: 'Amazon.com: Example product',
body_text: 'Hello, sign in Account & Lists Create account',
})).toBe(false);
});
});
-38
View File
@@ -1,38 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './discussion.js';
describe('amazon discussion normalization', () => {
it('normalizes review summary and sample reviews', () => {
const result = __test__.normalizeDiscussionPayload({
href: 'https://www.amazon.com/product-reviews/B0FJS72893',
average_rating_text: '3.9 out of 5',
total_review_count_text: '27 global ratings',
qa_links: [],
review_samples: [
{
title: '5.0 out of 5 stars Great value and quality',
rating_text: '5.0 out of 5 stars',
author: 'GTreader2',
date_text: 'Reviewed in the United States on February 21, 2026',
body: 'Small but mighty.',
verified: true,
},
],
});
expect(result.asin).toBe('B0FJS72893');
expect(result.average_rating_value).toBe(3.9);
expect(result.total_review_count).toBe(27);
expect(result.review_samples).toEqual([
{
title: 'Great value and quality',
rating_text: '5.0 out of 5 stars',
rating_value: 5,
author: 'GTreader2',
date_text: 'Reviewed in the United States on February 21, 2026',
body: 'Small but mighty.',
verified_purchase: true,
},
]);
});
});
-131
View File
@@ -1,131 +0,0 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
buildDiscussionUrl,
buildProvenance,
cleanText,
extractAsin,
normalizeProductUrl,
parseRatingValue,
parseReviewCount,
trimRatingPrefix,
uniqueNonEmpty,
assertUsableState,
gotoAndReadState,
} from './shared.js';
interface DiscussionPayload {
href?: string;
title?: string;
average_rating_text?: string | null;
total_review_count_text?: string | null;
qa_links?: string[];
review_samples?: Array<{
title?: string | null;
rating_text?: string | null;
author?: string | null;
date_text?: string | null;
body?: string | null;
verified?: boolean;
}>;
}
function normalizeDiscussionPayload(payload: DiscussionPayload): Record<string, unknown> {
const sourceUrl = cleanText(payload.href) || buildDiscussionUrl(payload.href ?? '');
const asin = extractAsin(payload.href ?? '') ?? null;
const averageRatingText = cleanText(payload.average_rating_text) || null;
const totalReviewCountText = cleanText(payload.total_review_count_text) || null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
product_url: asin ? normalizeProductUrl(asin) : null,
discussion_url: sourceUrl,
...provenance,
average_rating_text: averageRatingText,
average_rating_value: parseRatingValue(averageRatingText),
total_review_count_text: totalReviewCountText,
total_review_count: parseReviewCount(totalReviewCountText),
qa_urls: uniqueNonEmpty(payload.qa_links ?? []),
review_samples: (payload.review_samples ?? []).map((sample) => ({
title: trimRatingPrefix(sample.title) || null,
rating_text: cleanText(sample.rating_text) || null,
rating_value: parseRatingValue(sample.rating_text),
author: cleanText(sample.author) || null,
date_text: cleanText(sample.date_text) || null,
body: cleanText(sample.body) || null,
verified_purchase: sample.verified === true,
})),
};
}
async function readDiscussionPayload(page: IPage, input: string, limit: number): Promise<DiscussionPayload> {
const url = buildDiscussionUrl(input);
const state = await gotoAndReadState(page, url, 2500, 'discussion');
assertUsableState(state, 'discussion');
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
average_rating_text: document.querySelector('[data-hook="rating-out-of-text"]')?.textContent || '',
total_review_count_text: document.querySelector('[data-hook="total-review-count"]')?.textContent || '',
qa_links: Array.from(document.querySelectorAll('a[href*="ask/questions"]')).map((anchor) => anchor.href || ''),
review_samples: Array.from(document.querySelectorAll('[data-hook="review"]')).slice(0, ${limit}).map((card) => ({
title: card.querySelector('[data-hook="review-title"]')?.textContent || '',
rating_text:
card.querySelector('[data-hook="review-star-rating"]')?.textContent
|| card.querySelector('[data-hook="cmps-review-star-rating"]')?.textContent
|| '',
author: card.querySelector('.a-profile-name')?.textContent || '',
date_text: card.querySelector('[data-hook="review-date"]')?.textContent || '',
body: card.querySelector('[data-hook="review-body"]')?.textContent || '',
verified: !!card.querySelector('[data-hook="avp-badge"]'),
})),
}))()
`) as DiscussionPayload;
}
cli({
site: 'amazon',
name: 'discussion',
description: 'Amazon review summary and sample customer discussion from product review pages',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
{
name: 'limit',
type: 'int',
default: 10,
help: 'Maximum number of review samples to return (default 10)',
},
],
columns: ['asin', 'average_rating_value', 'total_review_count'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const limit = Math.max(1, Number(kwargs.limit) || 10);
const payload = await readDiscussionPayload(page, input, limit);
const normalized = normalizeDiscussionPayload(payload);
if (!normalized.average_rating_text && !normalized.total_review_count_text) {
throw new CommandExecutionError(
'amazon discussion page did not expose review summary',
'The review page may have changed or hit a robot check. Open the review page in Chrome and retry.',
);
}
return [normalized];
},
});
export const __test__ = {
normalizeDiscussionPayload,
};
+7
View File
@@ -0,0 +1,7 @@
import { cli } from '@jackwener/opencli/registry';
import { createRankingCliOptions } from './rankings.js';
cli(createRankingCliOptions({
commandName: 'movers-shakers',
listType: 'movers_shakers',
description: 'Amazon Movers & Shakers pages for short-term growth signals',
}));
-8
View File
@@ -1,8 +0,0 @@
import { cli } from '@jackwener/opencli/registry';
import { createRankingCliOptions } from './rankings.js';
cli(createRankingCliOptions({
commandName: 'movers-shakers',
listType: 'movers_shakers',
description: 'Amazon Movers & Shakers pages for short-term growth signals',
}));
@@ -1,8 +1,7 @@
import { cli } from '@jackwener/opencli/registry';
import { createRankingCliOptions } from './rankings.js';
cli(createRankingCliOptions({
commandName: 'bestsellers',
listType: 'bestsellers',
description: 'Amazon Best Sellers pages for category candidate discovery',
commandName: 'new-releases',
listType: 'new_releases',
description: 'Amazon New Releases pages for early momentum discovery',
}));
+140
View File
@@ -0,0 +1,140 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { buildProductUrl, buildProvenance, cleanText, extractAsin, isAmazonEntity, normalizeProductUrl, PRIMARY_PRICE_SELECTORS, parsePriceText, assertUsableState, gotoAndReadState, } from './shared.js';
const OFFER_FACT_SELECTOR = [
'#sellerProfileTriggerId',
'#shipsFromSoldByInsideBuyBox_feature_div',
'#fulfillerInfoFeature_feature_div',
'#merchantInfoFeature_feature_div',
'#tabular-buybox-container',
'#merchant-info',
].join(', ');
function collapseAdjacentWords(text) {
const parts = cleanText(text).split(' ').filter(Boolean);
const deduped = [];
for (const part of parts) {
if (deduped[deduped.length - 1] === part)
continue;
deduped.push(part);
}
return deduped.join(' ');
}
function extractShipsFrom(text) {
const normalized = cleanText(text);
const match = normalized.match(/Ships from\s+(.+?)(?=Sold by|and Fulfilled by|$)/i);
return match ? collapseAdjacentWords(match[1].replace(/Ships from/ig, '')) : null;
}
function extractSoldBy(text) {
const normalized = cleanText(text);
const match = normalized.match(/Sold by\s+(.+?)(?=and Fulfilled by|Ships from|$)/i);
return match ? collapseAdjacentWords(match[1]) : null;
}
function isDeliveryLocationBlocked(text) {
const normalized = cleanText(text).toLowerCase();
return normalized.includes('cannot be shipped to your selected delivery location')
|| normalized.includes('similar items shipping to')
|| normalized.includes('deliver to hong kong');
}
function normalizeOfferPayload(payload) {
const asin = extractAsin(payload.href ?? '') ?? null;
const sourceUrl = cleanText(payload.href) || buildProductUrl(payload.href ?? '');
const price = parsePriceText(payload.price_text);
const merchantInfo = cleanText(payload.merchant_info) || null;
const soldBy = cleanText(payload.sold_by)
|| extractSoldBy(payload.ships_from_text ?? '')
|| extractSoldBy(merchantInfo ?? '')
|| null;
const shipsFrom = extractShipsFrom(payload.ships_from_text ?? '')
|| extractShipsFrom(merchantInfo ?? '')
|| cleanText(payload.ships_from_text)
|| null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
product_url: normalizeProductUrl(payload.href),
...provenance,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
merchant_info_text: merchantInfo,
sold_by: soldBy,
ships_from: shipsFrom,
offer_listing_url: cleanText(payload.offer_link) || null,
review_url: cleanText(payload.review_url) || null,
qa_url: cleanText(payload.qa_url) || null,
is_amazon_sold: isAmazonEntity(soldBy),
is_amazon_fulfilled: isAmazonEntity(shipsFrom) || /fulfilled by amazon/i.test(merchantInfo ?? ''),
};
}
async function readOfferPayload(page, input) {
const url = buildProductUrl(input);
const state = await gotoAndReadState(page, url, 2500, 'offer');
assertUsableState(state, 'offer');
// Reconnecting to an existing Amazon target can surface the product page
// before the buy-box / merchant blocks are reattached to the DOM.
await page.wait({ selector: OFFER_FACT_SELECTOR, timeout: 6 }).catch(() => { });
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
price_text: (() => {
const selectors = ${JSON.stringify(PRIMARY_PRICE_SELECTORS)};
for (const selector of selectors) {
const text = document.querySelector(selector)?.textContent || '';
if (text.trim()) return text;
}
return '';
})(),
merchant_info: document.querySelector('#merchant-info')?.textContent || '',
sold_by: document.querySelector('#sellerProfileTriggerId')?.textContent || '',
ships_from_text:
document.querySelector('#shipsFromSoldByInsideBuyBox_feature_div')?.textContent
|| document.querySelector('#fulfillerInfoFeature_feature_div')?.textContent
|| document.querySelector('#merchantInfoFeature_feature_div')?.textContent
|| document.querySelector('#tabular-buybox-container')?.textContent
|| '',
offer_link: document.querySelector('a[href*="/gp/offer-listing/"]')?.href || '',
review_url: document.querySelector('a[href*="#customerReviews"]')?.href || '',
qa_url: document.querySelector('a[href*="ask/questions"]')?.href || '',
buybox_text:
document.querySelector('#desktop_qualifiedBuyBox')?.textContent
|| document.querySelector('#buybox')?.textContent
|| '',
}))()
`);
}
cli({
site: 'amazon',
name: 'offer',
description: 'Amazon seller, buy box, and fulfillment facts from the product page',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
],
columns: ['asin', 'price_text', 'sold_by', 'ships_from', 'is_amazon_sold', 'is_amazon_fulfilled'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const payload = await readOfferPayload(page, input);
const normalized = normalizeOfferPayload(payload);
if (!normalized.sold_by && !normalized.ships_from && !normalized.merchant_info_text) {
if (isDeliveryLocationBlocked(payload.buybox_text)) {
throw new CommandExecutionError('amazon offer buy box is blocked by the current delivery location', 'The shared Chrome profile is not set to the target US delivery address. Switch Amazon delivery location to the requested US destination, reopen the product page, and retry.');
}
throw new CommandExecutionError('amazon offer surface did not expose seller or fulfillment facts', 'The product page may have changed. Open the product page in Chrome, make sure the buy box is visible, and retry.');
}
return [normalized];
},
});
export const __test__ = {
extractShipsFrom,
extractSoldBy,
isDeliveryLocationBlocked,
normalizeOfferPayload,
};
+29
View File
@@ -0,0 +1,29 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './offer.js';
describe('amazon offer normalization', () => {
it('extracts sold-by and fulfillment facts from product offer text', () => {
const result = __test__.normalizeOfferPayload({
href: 'https://www.amazon.com/dp/B0FJS72893',
price_text: '$15.99',
merchant_info: '',
sold_by: 'KUATUDIRECT',
ships_from_text: 'Ships from Amazon',
offer_link: null,
review_url: 'https://www.amazon.com/dp/B0FJS72893#customerReviews',
qa_url: null,
});
expect(result.asin).toBe('B0FJS72893');
expect(result.sold_by).toBe('KUATUDIRECT');
expect(result.ships_from).toBe('Amazon');
expect(result.is_amazon_sold).toBe(false);
expect(result.is_amazon_fulfilled).toBe(true);
});
it('parses merchant info fallback text', () => {
expect(__test__.extractSoldBy('Sold by Example Seller and Fulfilled by Amazon.')).toBe('Example Seller');
expect(__test__.extractShipsFrom('Ships from Amazon')).toBe('Amazon');
});
it('detects delivery-location blocking in the buy box text', () => {
expect(__test__.isDeliveryLocationBlocked('This item cannot be shipped to your selected delivery location. Similar items shipping to Hong Kong')).toBe(true);
expect(__test__.isDeliveryLocationBlocked('Ships from Amazon')).toBe(false);
});
});
-35
View File
@@ -1,35 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './offer.js';
describe('amazon offer normalization', () => {
it('extracts sold-by and fulfillment facts from product offer text', () => {
const result = __test__.normalizeOfferPayload({
href: 'https://www.amazon.com/dp/B0FJS72893',
price_text: '$15.99',
merchant_info: '',
sold_by: 'KUATUDIRECT',
ships_from_text: 'Ships from Amazon',
offer_link: null,
review_url: 'https://www.amazon.com/dp/B0FJS72893#customerReviews',
qa_url: null,
});
expect(result.asin).toBe('B0FJS72893');
expect(result.sold_by).toBe('KUATUDIRECT');
expect(result.ships_from).toBe('Amazon');
expect(result.is_amazon_sold).toBe(false);
expect(result.is_amazon_fulfilled).toBe(true);
});
it('parses merchant info fallback text', () => {
expect(__test__.extractSoldBy('Sold by Example Seller and Fulfilled by Amazon.')).toBe('Example Seller');
expect(__test__.extractShipsFrom('Ships from Amazon')).toBe('Amazon');
});
it('detects delivery-location blocking in the buy box text', () => {
expect(__test__.isDeliveryLocationBlocked(
'This item cannot be shipped to your selected delivery location. Similar items shipping to Hong Kong',
)).toBe(true);
expect(__test__.isDeliveryLocationBlocked('Ships from Amazon')).toBe(false);
});
});
-185
View File
@@ -1,185 +0,0 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
buildProductUrl,
buildProvenance,
cleanText,
extractAsin,
isAmazonEntity,
normalizeProductUrl,
PRIMARY_PRICE_SELECTORS,
parsePriceText,
assertUsableState,
gotoAndReadState,
} from './shared.js';
interface OfferPayload {
href?: string;
title?: string;
price_text?: string | null;
merchant_info?: string | null;
sold_by?: string | null;
ships_from_text?: string | null;
offer_link?: string | null;
review_url?: string | null;
qa_url?: string | null;
buybox_text?: string | null;
}
const OFFER_FACT_SELECTOR = [
'#sellerProfileTriggerId',
'#shipsFromSoldByInsideBuyBox_feature_div',
'#fulfillerInfoFeature_feature_div',
'#merchantInfoFeature_feature_div',
'#tabular-buybox-container',
'#merchant-info',
].join(', ');
function collapseAdjacentWords(text: string): string {
const parts = cleanText(text).split(' ').filter(Boolean);
const deduped: string[] = [];
for (const part of parts) {
if (deduped[deduped.length - 1] === part) continue;
deduped.push(part);
}
return deduped.join(' ');
}
function extractShipsFrom(text: string): string | null {
const normalized = cleanText(text);
const match = normalized.match(/Ships from\s+(.+?)(?=Sold by|and Fulfilled by|$)/i);
return match ? collapseAdjacentWords(match[1].replace(/Ships from/ig, '')) : null;
}
function extractSoldBy(text: string): string | null {
const normalized = cleanText(text);
const match = normalized.match(/Sold by\s+(.+?)(?=and Fulfilled by|Ships from|$)/i);
return match ? collapseAdjacentWords(match[1]) : null;
}
function isDeliveryLocationBlocked(text: string | null | undefined): boolean {
const normalized = cleanText(text).toLowerCase();
return normalized.includes('cannot be shipped to your selected delivery location')
|| normalized.includes('similar items shipping to')
|| normalized.includes('deliver to hong kong');
}
function normalizeOfferPayload(payload: OfferPayload): Record<string, unknown> {
const asin = extractAsin(payload.href ?? '') ?? null;
const sourceUrl = cleanText(payload.href) || buildProductUrl(payload.href ?? '');
const price = parsePriceText(payload.price_text);
const merchantInfo = cleanText(payload.merchant_info) || null;
const soldBy = cleanText(payload.sold_by)
|| extractSoldBy(payload.ships_from_text ?? '')
|| extractSoldBy(merchantInfo ?? '')
|| null;
const shipsFrom = extractShipsFrom(payload.ships_from_text ?? '')
|| extractShipsFrom(merchantInfo ?? '')
|| cleanText(payload.ships_from_text)
|| null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
product_url: normalizeProductUrl(payload.href),
...provenance,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
merchant_info_text: merchantInfo,
sold_by: soldBy,
ships_from: shipsFrom,
offer_listing_url: cleanText(payload.offer_link) || null,
review_url: cleanText(payload.review_url) || null,
qa_url: cleanText(payload.qa_url) || null,
is_amazon_sold: isAmazonEntity(soldBy),
is_amazon_fulfilled: isAmazonEntity(shipsFrom) || /fulfilled by amazon/i.test(merchantInfo ?? ''),
};
}
async function readOfferPayload(page: IPage, input: string): Promise<OfferPayload> {
const url = buildProductUrl(input);
const state = await gotoAndReadState(page, url, 2500, 'offer');
assertUsableState(state, 'offer');
// Reconnecting to an existing Amazon target can surface the product page
// before the buy-box / merchant blocks are reattached to the DOM.
await page.wait({ selector: OFFER_FACT_SELECTOR, timeout: 6 }).catch(() => {});
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
price_text: (() => {
const selectors = ${JSON.stringify(PRIMARY_PRICE_SELECTORS)};
for (const selector of selectors) {
const text = document.querySelector(selector)?.textContent || '';
if (text.trim()) return text;
}
return '';
})(),
merchant_info: document.querySelector('#merchant-info')?.textContent || '',
sold_by: document.querySelector('#sellerProfileTriggerId')?.textContent || '',
ships_from_text:
document.querySelector('#shipsFromSoldByInsideBuyBox_feature_div')?.textContent
|| document.querySelector('#fulfillerInfoFeature_feature_div')?.textContent
|| document.querySelector('#merchantInfoFeature_feature_div')?.textContent
|| document.querySelector('#tabular-buybox-container')?.textContent
|| '',
offer_link: document.querySelector('a[href*="/gp/offer-listing/"]')?.href || '',
review_url: document.querySelector('a[href*="#customerReviews"]')?.href || '',
qa_url: document.querySelector('a[href*="ask/questions"]')?.href || '',
buybox_text:
document.querySelector('#desktop_qualifiedBuyBox')?.textContent
|| document.querySelector('#buybox')?.textContent
|| '',
}))()
`) as OfferPayload;
}
cli({
site: 'amazon',
name: 'offer',
description: 'Amazon seller, buy box, and fulfillment facts from the product page',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
],
columns: ['asin', 'price_text', 'sold_by', 'ships_from', 'is_amazon_sold', 'is_amazon_fulfilled'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const payload = await readOfferPayload(page, input);
const normalized = normalizeOfferPayload(payload);
if (!normalized.sold_by && !normalized.ships_from && !normalized.merchant_info_text) {
if (isDeliveryLocationBlocked(payload.buybox_text)) {
throw new CommandExecutionError(
'amazon offer buy box is blocked by the current delivery location',
'The shared Chrome profile is not set to the target US delivery address. Switch Amazon delivery location to the requested US destination, reopen the product page, and retry.',
);
}
throw new CommandExecutionError(
'amazon offer surface did not expose seller or fulfillment facts',
'The product page may have changed. Open the product page in Chrome, make sure the buy box is visible, and retry.',
);
}
return [normalized];
},
});
export const __test__ = {
extractShipsFrom,
extractSoldBy,
isDeliveryLocationBlocked,
normalizeOfferPayload,
};
+92
View File
@@ -0,0 +1,92 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { buildProductUrl, buildProvenance, cleanText, extractAsin, PRIMARY_PRICE_SELECTORS, parsePriceText, parseRatingValue, parseReviewCount, normalizeProductUrl, uniqueNonEmpty, assertUsableState, gotoAndReadState, } from './shared.js';
const PRODUCT_TITLE_SELECTOR = '#productTitle, #title span, [data-feature-name="title"] h1 span';
const BYLINE_SELECTOR = '#bylineInfo, [data-feature-name="bylineInfo"] #bylineInfo';
function normalizeProductPayload(payload) {
const sourceUrl = cleanText(payload.href) || buildProductUrl(cleanText(payload.product_title) || cleanText(payload.href));
const asin = extractAsin(payload.href ?? '') ?? null;
const price = parsePriceText(payload.price_text);
const ratingText = cleanText(payload.rating_text) || null;
const reviewCountText = cleanText(payload.review_count_text) || null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
title: cleanText(payload.product_title) || cleanText(payload.title) || null,
product_url: normalizeProductUrl(payload.href),
...provenance,
brand_text: cleanText(payload.byline) || null,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
rating_text: ratingText,
rating_value: parseRatingValue(ratingText),
review_count_text: reviewCountText,
review_count: parseReviewCount(reviewCountText),
review_url: cleanText(payload.review_url) || null,
qa_url: cleanText(payload.qa_url) || null,
breadcrumbs: uniqueNonEmpty(payload.breadcrumbs ?? []),
bullet_points: uniqueNonEmpty(payload.bullets ?? []),
};
}
async function readProductPayload(page, input) {
const url = buildProductUrl(input);
const state = await gotoAndReadState(page, url, 2500, 'product');
assertUsableState(state, 'product');
// Amazon can report a "stable" DOM before the product title block hydrates,
// especially when reconnecting to an existing shared CDP target.
await page.wait({ selector: PRODUCT_TITLE_SELECTOR, timeout: 6 }).catch(() => { });
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
product_title: document.querySelector(${JSON.stringify(PRODUCT_TITLE_SELECTOR)})?.textContent || '',
byline: document.querySelector(${JSON.stringify(BYLINE_SELECTOR)})?.textContent || '',
price_text: (() => {
const selectors = ${JSON.stringify(PRIMARY_PRICE_SELECTORS)};
for (const selector of selectors) {
const text = document.querySelector(selector)?.textContent || '';
if (text.trim()) return text;
}
return '';
})(),
rating_text:
document.querySelector('#acrPopover')?.getAttribute('title')
|| document.querySelector('#acrPopover')?.textContent
|| '',
review_count_text: document.querySelector('#acrCustomerReviewText')?.textContent || '',
review_url: document.querySelector('a[href*="#customerReviews"]')?.href || '',
qa_url: document.querySelector('a[href*="ask/questions"]')?.href || '',
bullets: Array.from(document.querySelectorAll('#feature-bullets li .a-list-item')).map((node) => node.textContent || ''),
breadcrumbs: Array.from(document.querySelectorAll('#wayfinding-breadcrumbs_feature_div a')).map((node) => node.textContent || ''),
}))()
`);
}
cli({
site: 'amazon',
name: 'product',
description: 'Amazon product page facts for candidate validation',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
],
columns: ['asin', 'title', 'price_text', 'rating_value', 'review_count'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const payload = await readProductPayload(page, input);
if (!cleanText(payload.product_title)) {
throw new CommandExecutionError('amazon product page did not expose product content', 'The product page may have changed or hit a robot check. Open the product page in Chrome and retry.');
}
return [normalizeProductPayload(payload)];
},
});
export const __test__ = {
normalizeProductPayload,
};
+24
View File
@@ -0,0 +1,24 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './product.js';
describe('amazon product normalization', () => {
it('normalizes product facts from the product page', () => {
const result = __test__.normalizeProductPayload({
href: 'https://www.amazon.com/dp/B0FJS72893',
title: 'Amazon.com: KVTUKIAIT Desktop Shelf Organizer',
product_title: 'White Desktop Shelf Organizer for Top of Desk',
byline: 'Visit the KVTUKIAIT Store',
price_text: '$15.99',
rating_text: '3.9 out of 5 stars',
review_count_text: '27 ratings',
review_url: 'https://www.amazon.com/dp/B0FJS72893#customerReviews',
qa_url: null,
bullets: ['SPACE-SAVING DESK SHELF ORGANIZER', 'SMALL AND STYLISH AESTHETIC DECOR'],
breadcrumbs: ['Office Products', 'Desktop & Off-Surface Shelves'],
});
expect(result.asin).toBe('B0FJS72893');
expect(result.price_value).toBe(15.99);
expect(result.rating_value).toBe(3.9);
expect(result.review_count).toBe(27);
expect(result.breadcrumbs).toEqual(['Office Products', 'Desktop & Off-Surface Shelves']);
});
});
-26
View File
@@ -1,26 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './product.js';
describe('amazon product normalization', () => {
it('normalizes product facts from the product page', () => {
const result = __test__.normalizeProductPayload({
href: 'https://www.amazon.com/dp/B0FJS72893',
title: 'Amazon.com: KVTUKIAIT Desktop Shelf Organizer',
product_title: 'White Desktop Shelf Organizer for Top of Desk',
byline: 'Visit the KVTUKIAIT Store',
price_text: '$15.99',
rating_text: '3.9 out of 5 stars',
review_count_text: '27 ratings',
review_url: 'https://www.amazon.com/dp/B0FJS72893#customerReviews',
qa_url: null,
bullets: ['SPACE-SAVING DESK SHELF ORGANIZER', 'SMALL AND STYLISH AESTHETIC DECOR'],
breadcrumbs: ['Office Products', 'Desktop & Off-Surface Shelves'],
});
expect(result.asin).toBe('B0FJS72893');
expect(result.price_value).toBe(15.99);
expect(result.rating_value).toBe(3.9);
expect(result.review_count).toBe(27);
expect(result.breadcrumbs).toEqual(['Office Products', 'Desktop & Off-Surface Shelves']);
});
});
-131
View File
@@ -1,131 +0,0 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
buildProductUrl,
buildProvenance,
cleanText,
extractAsin,
PRIMARY_PRICE_SELECTORS,
parsePriceText,
parseRatingValue,
parseReviewCount,
normalizeProductUrl,
uniqueNonEmpty,
assertUsableState,
gotoAndReadState,
} from './shared.js';
interface ProductPayload {
href?: string;
title?: string;
product_title?: string | null;
byline?: string | null;
price_text?: string | null;
rating_text?: string | null;
review_count_text?: string | null;
review_url?: string | null;
qa_url?: string | null;
bullets?: string[];
breadcrumbs?: string[];
}
const PRODUCT_TITLE_SELECTOR = '#productTitle, #title span, [data-feature-name="title"] h1 span';
const BYLINE_SELECTOR = '#bylineInfo, [data-feature-name="bylineInfo"] #bylineInfo';
function normalizeProductPayload(payload: ProductPayload): Record<string, unknown> {
const sourceUrl = cleanText(payload.href) || buildProductUrl(cleanText(payload.product_title) || cleanText(payload.href));
const asin = extractAsin(payload.href ?? '') ?? null;
const price = parsePriceText(payload.price_text);
const ratingText = cleanText(payload.rating_text) || null;
const reviewCountText = cleanText(payload.review_count_text) || null;
const provenance = buildProvenance(sourceUrl);
return {
asin,
title: cleanText(payload.product_title) || cleanText(payload.title) || null,
product_url: normalizeProductUrl(payload.href),
...provenance,
brand_text: cleanText(payload.byline) || null,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
rating_text: ratingText,
rating_value: parseRatingValue(ratingText),
review_count_text: reviewCountText,
review_count: parseReviewCount(reviewCountText),
review_url: cleanText(payload.review_url) || null,
qa_url: cleanText(payload.qa_url) || null,
breadcrumbs: uniqueNonEmpty(payload.breadcrumbs ?? []),
bullet_points: uniqueNonEmpty(payload.bullets ?? []),
};
}
async function readProductPayload(page: IPage, input: string): Promise<ProductPayload> {
const url = buildProductUrl(input);
const state = await gotoAndReadState(page, url, 2500, 'product');
assertUsableState(state, 'product');
// Amazon can report a "stable" DOM before the product title block hydrates,
// especially when reconnecting to an existing shared CDP target.
await page.wait({ selector: PRODUCT_TITLE_SELECTOR, timeout: 6 }).catch(() => {});
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
product_title: document.querySelector(${JSON.stringify(PRODUCT_TITLE_SELECTOR)})?.textContent || '',
byline: document.querySelector(${JSON.stringify(BYLINE_SELECTOR)})?.textContent || '',
price_text: (() => {
const selectors = ${JSON.stringify(PRIMARY_PRICE_SELECTORS)};
for (const selector of selectors) {
const text = document.querySelector(selector)?.textContent || '';
if (text.trim()) return text;
}
return '';
})(),
rating_text:
document.querySelector('#acrPopover')?.getAttribute('title')
|| document.querySelector('#acrPopover')?.textContent
|| '',
review_count_text: document.querySelector('#acrCustomerReviewText')?.textContent || '',
review_url: document.querySelector('a[href*="#customerReviews"]')?.href || '',
qa_url: document.querySelector('a[href*="ask/questions"]')?.href || '',
bullets: Array.from(document.querySelectorAll('#feature-bullets li .a-list-item')).map((node) => node.textContent || ''),
breadcrumbs: Array.from(document.querySelectorAll('#wayfinding-breadcrumbs_feature_div a')).map((node) => node.textContent || ''),
}))()
`) as ProductPayload;
}
cli({
site: 'amazon',
name: 'product',
description: 'Amazon product page facts for candidate validation',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
required: true,
positional: true,
help: 'ASIN or product URL, for example B0FJS72893',
},
],
columns: ['asin', 'title', 'price_text', 'rating_value', 'review_count'],
func: async (page, kwargs) => {
const input = String(kwargs.input ?? '');
const payload = await readProductPayload(page, input);
if (!cleanText(payload.product_title)) {
throw new CommandExecutionError(
'amazon product page did not expose product content',
'The product page may have changed or hit a robot check. Open the product page in Chrome and retry.',
);
}
return [normalizeProductPayload(payload)];
},
});
export const __test__ = {
normalizeProductPayload,
};
+226
View File
@@ -0,0 +1,226 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { Strategy } from '@jackwener/opencli/registry';
import { assertUsableState, buildProvenance, cleanText, extractAsin, extractCategoryNodeId, extractReviewCountFromCardText, firstMeaningfulLine, gotoAndReadState, isRankingPaginationUrl, normalizeProductUrl, parsePriceText, parseRatingValue, parseReviewCount, resolveRankingUrl, toAbsoluteAmazonUrl, uniqueNonEmpty, } from './shared.js';
function parseRank(rawRank, fallback) {
const normalized = cleanText(rawRank);
const match = normalized.match(/(\d{1,4})/);
if (!match)
return fallback;
const parsed = Number.parseInt(match[1], 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
}
function normalizeVisibleCategoryLinks(links) {
const normalized = (links ?? [])
.map((entry) => ({
title: cleanText(entry?.title),
url: toAbsoluteAmazonUrl(entry?.url) ?? '',
node_id: cleanText(entry?.node_id) || extractCategoryNodeId(entry?.url) || null,
}))
.filter((entry) => Boolean(entry.title) && Boolean(entry.url));
const seen = new Set();
const deduped = [];
for (const entry of normalized) {
if (seen.has(entry.url))
continue;
seen.add(entry.url);
deduped.push(entry);
}
return deduped;
}
export function normalizeRankingCandidate(candidate, context) {
const productUrl = normalizeProductUrl(candidate.href);
const asin = extractAsin(candidate.asin ?? '') ?? extractAsin(productUrl ?? '') ?? null;
const title = cleanText(candidate.title) || firstMeaningfulLine(candidate.card_text);
const price = parsePriceText(cleanText(candidate.price_text) || candidate.card_text);
const ratingText = cleanText(candidate.rating_text) || null;
const reviewCountText = cleanText(candidate.review_count_text)
|| extractReviewCountFromCardText(candidate.card_text)
|| null;
const provenance = buildProvenance(context.sourceUrl);
const categoryUrl = context.categoryUrl || context.sourceUrl;
return {
list_type: context.listType,
rank: parseRank(candidate.rank_text, context.rankFallback),
asin,
title: title || null,
product_url: productUrl,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
rating_text: ratingText,
rating_value: parseRatingValue(ratingText),
review_count_text: reviewCountText,
review_count: parseReviewCount(reviewCountText),
list_title: context.listTitle,
category_title: context.categoryTitle,
category_url: categoryUrl,
category_node_id: extractCategoryNodeId(categoryUrl),
category_path: context.categoryPath,
visible_category_links: context.visibleCategoryLinks,
...provenance,
};
}
async function readRankingPage(page, listType, url) {
const state = await gotoAndReadState(page, url, 2500, listType);
assertUsableState(state, listType);
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
list_title:
document.querySelector('#zg_banner_text')?.textContent
|| document.querySelector('h1')?.textContent
|| '',
category_title:
document.querySelector('#zg_browseRoot .zg_selected')?.textContent
|| document.querySelector('#wayfinding-breadcrumbs_feature_div ul li:last-child')?.textContent
|| document.querySelector('#wayfinding-breadcrumbs_container ul li:last-child')?.textContent
|| '',
category_path: Array.from(document.querySelectorAll(
'#zg_browseRoot ul li a, #zg_browseRoot ul li span, ' +
'#wayfinding-breadcrumbs_feature_div ul li a, #wayfinding-breadcrumbs_feature_div ul li span.a-list-item, ' +
'#wayfinding-breadcrumbs_container ul li a, #wayfinding-breadcrumbs_container ul li span.a-list-item'
))
.map((entry) => (entry.textContent || '').trim())
.filter(Boolean),
cards: Array.from(document.querySelectorAll(
'.p13n-sc-uncoverable-faceout, .zg-grid-general-faceout, [data-asin][class*="p13n"]'
)).map((card) => ({
rank_text:
card.querySelector('.zg-bdg-text')?.textContent
|| card.querySelector('[class*="rank"]')?.textContent
|| '',
asin:
card.getAttribute('data-asin')
|| card.getAttribute('id')
|| '',
title:
card.querySelector('[class*="line-clamp"]')?.textContent
|| card.querySelector('img')?.getAttribute('alt')
|| card.querySelector('a[href*="/dp/"]')?.textContent
|| '',
href:
card.querySelector('a[href*="/dp/"], a[href*="/gp/product/"]')?.href
|| '',
price_text:
card.querySelector('.a-price .a-offscreen')?.textContent
|| card.querySelector('.a-color-price')?.textContent
|| '',
rating_text:
card.querySelector('[aria-label*="out of 5 stars"]')?.getAttribute('aria-label')
|| '',
review_count_text:
card.querySelector('a[href*="#customerReviews"]')?.textContent
|| card.querySelector('.a-size-small')?.textContent
|| '',
card_text: card.innerText || '',
})),
page_links: Array.from(document.querySelectorAll('.a-pagination a[href], li.a-normal a[href], li.a-selected a[href]'))
.map((anchor) => anchor.href || '')
.filter(Boolean),
visible_category_links: Array.from(document.querySelectorAll(
'#zg_browseRoot a[href], #zg-left-col a[href], [class*="zg-browse"] a[href]'
)).map((anchor) => ({
title: (anchor.textContent || '').trim(),
url: anchor.href || '',
node_id:
anchor.getAttribute('data-node-id')
|| anchor.dataset?.nodeid
|| '',
}))
.filter((entry) => entry.title && entry.url),
}))()
`);
}
function createEmptyResultHint(commandName) {
return [
`Open the same Amazon ${commandName} page in shared Chrome and verify ranked items are visible.`,
'If the page shows a robot check, clear it manually and retry.',
].join(' ');
}
export function createRankingCliOptions(definition) {
return {
site: 'amazon',
name: definition.commandName,
description: definition.description,
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
positional: true,
help: 'Ranking URL or supported Amazon path. Omit to use the list root.',
},
{
name: 'limit',
type: 'int',
default: 100,
help: 'Maximum number of ranked items to return (default 100)',
},
],
columns: ['list_type', 'rank', 'asin', 'title', 'price_text', 'rating_value', 'review_count'],
func: async (page, kwargs) => {
const limit = Math.max(1, Number(kwargs.limit) || 100);
const initialUrl = resolveRankingUrl(definition.listType, typeof kwargs.input === 'string' ? kwargs.input : undefined);
const queue = [initialUrl];
const visited = new Set();
const seenEntityKeys = new Set();
const results = [];
let listTitle = null;
while (queue.length > 0 && results.length < limit) {
const nextUrl = queue.shift();
if (visited.has(nextUrl))
continue;
visited.add(nextUrl);
const payload = await readRankingPage(page, definition.listType, nextUrl);
const sourceUrl = cleanText(payload.href) || nextUrl;
listTitle = cleanText(payload.list_title) || cleanText(payload.title) || listTitle;
const categoryPath = uniqueNonEmpty(payload.category_path ?? []);
const categoryTitle = cleanText(payload.category_title)
|| (categoryPath.length > 0 ? categoryPath[categoryPath.length - 1] : '');
const visibleCategoryLinks = normalizeVisibleCategoryLinks(payload.visible_category_links);
const cards = payload.cards ?? [];
for (const card of cards) {
const normalized = normalizeRankingCandidate(card, {
listType: definition.listType,
rankFallback: results.length + 1,
listTitle,
sourceUrl,
categoryTitle: categoryTitle || null,
categoryUrl: sourceUrl,
categoryPath,
visibleCategoryLinks,
});
const dedupeKey = cleanText(String(normalized.asin ?? ''))
|| cleanText(String(normalized.product_url ?? ''));
if (dedupeKey && seenEntityKeys.has(dedupeKey))
continue;
if (dedupeKey)
seenEntityKeys.add(dedupeKey);
results.push(normalized);
if (results.length >= limit)
break;
}
const pageLinks = uniqueNonEmpty(payload.page_links ?? []);
for (const href of pageLinks) {
const absolute = toAbsoluteAmazonUrl(href);
if (!absolute || !isRankingPaginationUrl(definition.listType, absolute))
continue;
if (!visited.has(absolute) && !queue.includes(absolute)) {
queue.push(absolute);
}
}
}
if (results.length === 0) {
throw new CommandExecutionError(`amazon ${definition.commandName} did not expose any ranked items`, createEmptyResultHint(definition.commandName));
}
return results.slice(0, limit);
},
};
}
export const __test__ = {
parseRank,
normalizeVisibleCategoryLinks,
normalizeRankingCandidate,
};
+41
View File
@@ -0,0 +1,41 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './rankings.js';
describe('amazon rankings helpers', () => {
it('normalizes ranking candidates with unified schema', () => {
const result = __test__.normalizeRankingCandidate({
rank_text: '#3',
asin: 'B0DR31GC3D',
title: 'Desk Shelves Desktop Organizer',
href: 'https://www.amazon.com/dp/B0DR31GC3D/ref=zg_bs',
price_text: '$25.92',
rating_text: '4.3 out of 5 stars',
review_count_text: '435',
}, {
listType: 'new_releases',
rankFallback: 3,
listTitle: 'Amazon New Releases',
sourceUrl: 'https://www.amazon.com/gp/new-releases',
categoryTitle: 'Home & Kitchen',
categoryUrl: 'https://www.amazon.com/gp/new-releases/home-garden',
categoryPath: ['Home & Kitchen'],
visibleCategoryLinks: [{ title: 'Storage', url: 'https://www.amazon.com/gp/new-releases/storage', node_id: null }],
});
expect(result.list_type).toBe('new_releases');
expect(result.rank).toBe(3);
expect(result.asin).toBe('B0DR31GC3D');
expect(result.product_url).toBe('https://www.amazon.com/dp/B0DR31GC3D');
expect(result.category_title).toBe('Home & Kitchen');
expect(result.visible_category_links).toEqual([
{ title: 'Storage', url: 'https://www.amazon.com/gp/new-releases/storage', node_id: null },
]);
});
it('deduplicates category links and parses rank fallback', () => {
const links = __test__.normalizeVisibleCategoryLinks([
{ title: 'Kitchen', url: '/gp/new-releases/home-garden' },
{ title: 'Kitchen', url: 'https://www.amazon.com/gp/new-releases/home-garden' },
{ title: 'Storage', url: '/gp/new-releases/storage', node_id: '1064954' },
]);
expect(links.length).toBe(2);
expect(__test__.parseRank('N/A', 8)).toBe(8);
});
});
-47
View File
@@ -1,47 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './rankings.js';
describe('amazon rankings helpers', () => {
it('normalizes ranking candidates with unified schema', () => {
const result = __test__.normalizeRankingCandidate(
{
rank_text: '#3',
asin: 'B0DR31GC3D',
title: 'Desk Shelves Desktop Organizer',
href: 'https://www.amazon.com/dp/B0DR31GC3D/ref=zg_bs',
price_text: '$25.92',
rating_text: '4.3 out of 5 stars',
review_count_text: '435',
},
{
listType: 'new_releases',
rankFallback: 3,
listTitle: 'Amazon New Releases',
sourceUrl: 'https://www.amazon.com/gp/new-releases',
categoryTitle: 'Home & Kitchen',
categoryUrl: 'https://www.amazon.com/gp/new-releases/home-garden',
categoryPath: ['Home & Kitchen'],
visibleCategoryLinks: [{ title: 'Storage', url: 'https://www.amazon.com/gp/new-releases/storage', node_id: null }],
},
);
expect(result.list_type).toBe('new_releases');
expect(result.rank).toBe(3);
expect(result.asin).toBe('B0DR31GC3D');
expect(result.product_url).toBe('https://www.amazon.com/dp/B0DR31GC3D');
expect(result.category_title).toBe('Home & Kitchen');
expect(result.visible_category_links).toEqual([
{ title: 'Storage', url: 'https://www.amazon.com/gp/new-releases/storage', node_id: null },
]);
});
it('deduplicates category links and parses rank fallback', () => {
const links = __test__.normalizeVisibleCategoryLinks([
{ title: 'Kitchen', url: '/gp/new-releases/home-garden' },
{ title: 'Kitchen', url: 'https://www.amazon.com/gp/new-releases/home-garden' },
{ title: 'Storage', url: '/gp/new-releases/storage', node_id: '1064954' },
]);
expect(links.length).toBe(2);
expect(__test__.parseRank('N/A', 8)).toBe(8);
});
});
-312
View File
@@ -1,312 +0,0 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { Strategy, type CliOptions } from '@jackwener/opencli/registry';
import type { IPage } from '@jackwener/opencli/types';
import {
assertUsableState,
buildProvenance,
cleanText,
extractAsin,
extractCategoryNodeId,
extractReviewCountFromCardText,
firstMeaningfulLine,
gotoAndReadState,
isRankingPaginationUrl,
normalizeProductUrl,
parsePriceText,
parseRatingValue,
parseReviewCount,
resolveRankingUrl,
toAbsoluteAmazonUrl,
uniqueNonEmpty,
type AmazonRankingListType,
} from './shared.js';
export interface RankingCardPayload {
rank_text?: string | null;
asin?: string | null;
title?: string | null;
href?: string | null;
price_text?: string | null;
rating_text?: string | null;
review_count_text?: string | null;
card_text?: string | null;
}
interface RankingPagePayload {
href?: string;
title?: string;
list_title?: string;
category_title?: string;
category_path?: string[];
cards?: RankingCardPayload[];
page_links?: string[];
visible_category_links?: Array<{
title?: string | null;
url?: string | null;
node_id?: string | null;
}>;
}
interface RankingCommandDefinition {
commandName: string;
listType: AmazonRankingListType;
description: string;
}
interface RankingNormalizeContext {
listType: AmazonRankingListType;
rankFallback: number;
listTitle: string | null;
sourceUrl: string;
categoryTitle: string | null;
categoryUrl: string | null;
categoryPath: string[];
visibleCategoryLinks: Array<{ title: string; url: string; node_id: string | null }>;
}
function parseRank(rawRank: string | null | undefined, fallback: number): number {
const normalized = cleanText(rawRank);
const match = normalized.match(/(\d{1,4})/);
if (!match) return fallback;
const parsed = Number.parseInt(match[1], 10);
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
}
function normalizeVisibleCategoryLinks(
links: RankingPagePayload['visible_category_links'],
): Array<{ title: string; url: string; node_id: string | null }> {
const normalized = (links ?? [])
.map((entry) => ({
title: cleanText(entry?.title),
url: toAbsoluteAmazonUrl(entry?.url) ?? '',
node_id: cleanText(entry?.node_id) || extractCategoryNodeId(entry?.url) || null,
}))
.filter((entry) => Boolean(entry.title) && Boolean(entry.url));
const seen = new Set<string>();
const deduped: Array<{ title: string; url: string; node_id: string | null }> = [];
for (const entry of normalized) {
if (seen.has(entry.url)) continue;
seen.add(entry.url);
deduped.push(entry);
}
return deduped;
}
export function normalizeRankingCandidate(
candidate: RankingCardPayload,
context: RankingNormalizeContext,
): Record<string, unknown> {
const productUrl = normalizeProductUrl(candidate.href);
const asin = extractAsin(candidate.asin ?? '') ?? extractAsin(productUrl ?? '') ?? null;
const title = cleanText(candidate.title) || firstMeaningfulLine(candidate.card_text);
const price = parsePriceText(cleanText(candidate.price_text) || candidate.card_text);
const ratingText = cleanText(candidate.rating_text) || null;
const reviewCountText = cleanText(candidate.review_count_text)
|| extractReviewCountFromCardText(candidate.card_text)
|| null;
const provenance = buildProvenance(context.sourceUrl);
const categoryUrl = context.categoryUrl || context.sourceUrl;
return {
list_type: context.listType,
rank: parseRank(candidate.rank_text, context.rankFallback),
asin,
title: title || null,
product_url: productUrl,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
rating_text: ratingText,
rating_value: parseRatingValue(ratingText),
review_count_text: reviewCountText,
review_count: parseReviewCount(reviewCountText),
list_title: context.listTitle,
category_title: context.categoryTitle,
category_url: categoryUrl,
category_node_id: extractCategoryNodeId(categoryUrl),
category_path: context.categoryPath,
visible_category_links: context.visibleCategoryLinks,
...provenance,
};
}
async function readRankingPage(
page: IPage,
listType: AmazonRankingListType,
url: string,
): Promise<RankingPagePayload> {
const state = await gotoAndReadState(page, url, 2500, listType);
assertUsableState(state, listType);
return await page.evaluate(`
(() => ({
href: window.location.href,
title: document.title || '',
list_title:
document.querySelector('#zg_banner_text')?.textContent
|| document.querySelector('h1')?.textContent
|| '',
category_title:
document.querySelector('#zg_browseRoot .zg_selected')?.textContent
|| document.querySelector('#wayfinding-breadcrumbs_feature_div ul li:last-child')?.textContent
|| document.querySelector('#wayfinding-breadcrumbs_container ul li:last-child')?.textContent
|| '',
category_path: Array.from(document.querySelectorAll(
'#zg_browseRoot ul li a, #zg_browseRoot ul li span, ' +
'#wayfinding-breadcrumbs_feature_div ul li a, #wayfinding-breadcrumbs_feature_div ul li span.a-list-item, ' +
'#wayfinding-breadcrumbs_container ul li a, #wayfinding-breadcrumbs_container ul li span.a-list-item'
))
.map((entry) => (entry.textContent || '').trim())
.filter(Boolean),
cards: Array.from(document.querySelectorAll(
'.p13n-sc-uncoverable-faceout, .zg-grid-general-faceout, [data-asin][class*="p13n"]'
)).map((card) => ({
rank_text:
card.querySelector('.zg-bdg-text')?.textContent
|| card.querySelector('[class*="rank"]')?.textContent
|| '',
asin:
card.getAttribute('data-asin')
|| card.getAttribute('id')
|| '',
title:
card.querySelector('[class*="line-clamp"]')?.textContent
|| card.querySelector('img')?.getAttribute('alt')
|| card.querySelector('a[href*="/dp/"]')?.textContent
|| '',
href:
card.querySelector('a[href*="/dp/"], a[href*="/gp/product/"]')?.href
|| '',
price_text:
card.querySelector('.a-price .a-offscreen')?.textContent
|| card.querySelector('.a-color-price')?.textContent
|| '',
rating_text:
card.querySelector('[aria-label*="out of 5 stars"]')?.getAttribute('aria-label')
|| '',
review_count_text:
card.querySelector('a[href*="#customerReviews"]')?.textContent
|| card.querySelector('.a-size-small')?.textContent
|| '',
card_text: card.innerText || '',
})),
page_links: Array.from(document.querySelectorAll('.a-pagination a[href], li.a-normal a[href], li.a-selected a[href]'))
.map((anchor) => anchor.href || '')
.filter(Boolean),
visible_category_links: Array.from(document.querySelectorAll(
'#zg_browseRoot a[href], #zg-left-col a[href], [class*="zg-browse"] a[href]'
)).map((anchor) => ({
title: (anchor.textContent || '').trim(),
url: anchor.href || '',
node_id:
anchor.getAttribute('data-node-id')
|| anchor.dataset?.nodeid
|| '',
}))
.filter((entry) => entry.title && entry.url),
}))()
`) as RankingPagePayload;
}
function createEmptyResultHint(commandName: string): string {
return [
`Open the same Amazon ${commandName} page in shared Chrome and verify ranked items are visible.`,
'If the page shows a robot check, clear it manually and retry.',
].join(' ');
}
export function createRankingCliOptions(definition: RankingCommandDefinition): CliOptions {
return {
site: 'amazon',
name: definition.commandName,
description: definition.description,
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'input',
positional: true,
help: 'Ranking URL or supported Amazon path. Omit to use the list root.',
},
{
name: 'limit',
type: 'int',
default: 100,
help: 'Maximum number of ranked items to return (default 100)',
},
],
columns: ['list_type', 'rank', 'asin', 'title', 'price_text', 'rating_value', 'review_count'],
func: async (page, kwargs) => {
const limit = Math.max(1, Number(kwargs.limit) || 100);
const initialUrl = resolveRankingUrl(definition.listType, typeof kwargs.input === 'string' ? kwargs.input : undefined);
const queue = [initialUrl];
const visited = new Set<string>();
const seenEntityKeys = new Set<string>();
const results: Record<string, unknown>[] = [];
let listTitle: string | null = null;
while (queue.length > 0 && results.length < limit) {
const nextUrl = queue.shift()!;
if (visited.has(nextUrl)) continue;
visited.add(nextUrl);
const payload = await readRankingPage(page, definition.listType, nextUrl);
const sourceUrl = cleanText(payload.href) || nextUrl;
listTitle = cleanText(payload.list_title) || cleanText(payload.title) || listTitle;
const categoryPath = uniqueNonEmpty(payload.category_path ?? []);
const categoryTitle = cleanText(payload.category_title)
|| (categoryPath.length > 0 ? categoryPath[categoryPath.length - 1] : '');
const visibleCategoryLinks = normalizeVisibleCategoryLinks(payload.visible_category_links);
const cards = payload.cards ?? [];
for (const card of cards) {
const normalized = normalizeRankingCandidate(card, {
listType: definition.listType,
rankFallback: results.length + 1,
listTitle,
sourceUrl,
categoryTitle: categoryTitle || null,
categoryUrl: sourceUrl,
categoryPath,
visibleCategoryLinks,
});
const dedupeKey = cleanText(String(normalized.asin ?? ''))
|| cleanText(String(normalized.product_url ?? ''));
if (dedupeKey && seenEntityKeys.has(dedupeKey)) continue;
if (dedupeKey) seenEntityKeys.add(dedupeKey);
results.push(normalized);
if (results.length >= limit) break;
}
const pageLinks = uniqueNonEmpty(payload.page_links ?? []);
for (const href of pageLinks) {
const absolute = toAbsoluteAmazonUrl(href);
if (!absolute || !isRankingPaginationUrl(definition.listType, absolute)) continue;
if (!visited.has(absolute) && !queue.includes(absolute)) {
queue.push(absolute);
}
}
}
if (results.length === 0) {
throw new CommandExecutionError(
`amazon ${definition.commandName} did not expose any ranked items`,
createEmptyResultHint(definition.commandName),
);
}
return results.slice(0, limit);
},
};
}
export const __test__ = {
parseRank,
normalizeVisibleCategoryLinks,
normalizeRankingCandidate,
};
+87
View File
@@ -0,0 +1,87 @@
import { CommandExecutionError } from '@jackwener/opencli/errors';
import { cli, Strategy } from '@jackwener/opencli/registry';
import { buildProvenance, buildSearchUrl, cleanText, extractAsin, normalizeProductUrl, parsePriceText, parseRatingValue, parseReviewCount, assertUsableState, gotoAndReadState, } from './shared.js';
function normalizeSearchCandidate(candidate, rank, sourceUrl) {
const productUrl = normalizeProductUrl(candidate.href);
const asin = extractAsin(candidate.asin ?? '') ?? extractAsin(productUrl ?? '') ?? null;
const price = parsePriceText(candidate.price_text);
const ratingText = cleanText(candidate.rating_text) || null;
const reviewCountText = cleanText(candidate.review_count_text) || null;
const provenance = buildProvenance(sourceUrl);
return {
rank,
asin,
title: cleanText(candidate.title) || null,
product_url: productUrl,
...provenance,
price_text: price.price_text,
price_value: price.price_value,
currency: price.currency,
rating_text: ratingText,
rating_value: parseRatingValue(ratingText),
review_count_text: reviewCountText,
review_count: parseReviewCount(reviewCountText),
is_sponsored: candidate.sponsored === true,
badges: (candidate.badge_texts ?? []).map((value) => cleanText(value)).filter(Boolean),
};
}
async function readSearchPayload(page, query) {
const url = buildSearchUrl(query);
const state = await gotoAndReadState(page, url, 2500, 'search');
assertUsableState(state, 'search');
return await page.evaluate(`
(() => ({
href: window.location.href,
cards: Array.from(document.querySelectorAll('[data-component-type="s-search-result"]'))
.map((card) => ({
asin: card.getAttribute('data-asin') || '',
title: card.querySelector('h2')?.textContent || '',
href: card.querySelector('a.a-link-normal[href*="/dp/"]')?.href || '',
price_text: card.querySelector('.a-price .a-offscreen')?.textContent || '',
rating_text: card.querySelector('[aria-label*="out of 5 stars"]')?.getAttribute('aria-label') || '',
review_count_text: card.querySelector('a[href*="#customerReviews"]')?.textContent || '',
sponsored: /sponsored/i.test(card.innerText || ''),
badge_texts: Array.from(card.querySelectorAll('.a-badge-text')).map((node) => node.textContent || ''),
})),
}))()
`);
}
cli({
site: 'amazon',
name: 'search',
description: 'Amazon search results for product discovery and coarse filtering',
domain: 'amazon.com',
strategy: Strategy.COOKIE,
navigateBefore: false,
args: [
{
name: 'query',
required: true,
positional: true,
help: 'Search query, for example "desk shelf organizer"',
},
{
name: 'limit',
type: 'int',
default: 20,
help: 'Maximum number of results to return (default 20)',
},
],
columns: ['rank', 'asin', 'title', 'price_text', 'rating_value', 'review_count'],
func: async (page, kwargs) => {
const query = String(kwargs.query ?? '');
const limit = Math.max(1, Number(kwargs.limit) || 20);
const payload = await readSearchPayload(page, query);
const sourceUrl = cleanText(payload.href) || buildSearchUrl(query);
const cards = (payload.cards ?? [])
.filter((card) => cleanText(card.asin) && cleanText(card.title))
.slice(0, limit);
if (cards.length === 0) {
throw new CommandExecutionError('amazon search did not expose any product cards', 'The search page may have changed or hit a robot check. Open the same query in Chrome, verify the page is visible, and retry.');
}
return cards.map((card, index) => normalizeSearchCandidate(card, index + 1, sourceUrl));
},
});
export const __test__ = {
normalizeSearchCandidate,
};
+22
View File
@@ -0,0 +1,22 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './search.js';
describe('amazon search normalization', () => {
it('normalizes search cards into research-friendly fields', () => {
const result = __test__.normalizeSearchCandidate({
asin: 'B0FJS72893',
title: 'White Desktop Shelf Organizer for Top of Desk',
href: 'https://www.amazon.com/KVTUKIAIT-White-Desktop-Shelf-Organizer/dp/B0FJS72893/ref=sr_1_1',
price_text: '$15.99',
rating_text: '3.9 out of 5 stars, rating details',
review_count_text: '(27)',
sponsored: false,
badge_texts: ['Limited time deal'],
}, 1, 'https://www.amazon.com/s?k=desk+shelf+organizer');
expect(result.asin).toBe('B0FJS72893');
expect(result.product_url).toBe('https://www.amazon.com/dp/B0FJS72893');
expect(result.price_value).toBe(15.99);
expect(result.rating_value).toBe(3.9);
expect(result.review_count).toBe(27);
expect(result.badges).toEqual(['Limited time deal']);
});
});
-24
View File
@@ -1,24 +0,0 @@
import { describe, expect, it } from 'vitest';
import { __test__ } from './search.js';
describe('amazon search normalization', () => {
it('normalizes search cards into research-friendly fields', () => {
const result = __test__.normalizeSearchCandidate({
asin: 'B0FJS72893',
title: 'White Desktop Shelf Organizer for Top of Desk',
href: 'https://www.amazon.com/KVTUKIAIT-White-Desktop-Shelf-Organizer/dp/B0FJS72893/ref=sr_1_1',
price_text: '$15.99',
rating_text: '3.9 out of 5 stars, rating details',
review_count_text: '(27)',
sponsored: false,
badge_texts: ['Limited time deal'],
}, 1, 'https://www.amazon.com/s?k=desk+shelf+organizer');
expect(result.asin).toBe('B0FJS72893');
expect(result.product_url).toBe('https://www.amazon.com/dp/B0FJS72893');
expect(result.price_value).toBe(15.99);
expect(result.rating_value).toBe(3.9);
expect(result.review_count).toBe(27);
expect(result.badges).toEqual(['Limited time deal']);
});
});

Some files were not shown because too many files have changed in this diff Show More