* feat: add download support for images, videos, and articles
Add comprehensive download functionality to OpenCLI with support for
multiple platforms and content types.
- Add `src/download/index.ts`: HTTP download with progress, yt-dlp
wrapper for video platforms, cookie export to Netscape format for
authenticated downloads
- Add `src/download/progress.ts`: Terminal progress bars, multi-file
download tracker with status summary
- Add `src/pipeline/steps/download.ts`: New `download` pipeline step
for declarative YAML pipelines
- Register `download` step in executor.ts
- Add template filters: `slugify`, `sanitize`, `ext`, `basename` for
filename templating
- `xiaohongshu download`: Download images and videos from notes
- `bilibili download`: Download videos using yt-dlp with cookie auth
- `twitter download`: Download media from user timeline or single tweet
- `zhihu download`: Export articles to Markdown with optional image
download
```yaml
pipeline:
- download:
url: ${{ item.imageUrl }}
dir: ./downloads
filename: ${{ item.title | sanitize }}.jpg
concurrency: 5
skip_existing: true
use_ytdlp: false
type: auto # auto|image|video|document
```
- Concurrent downloads with configurable parallelism
- Progress bars with file size display
- Skip existing files option
- Cookie forwarding for authenticated downloads
- yt-dlp integration for video platforms (YouTube, Bilibili, Twitter)
- HTML to Markdown conversion for article export
- yt-dlp: Required for video downloads from streaming platforms
- ffmpeg: Optional for video format conversion
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* docs: add download support documentation
- Add Download Support section to both README.md and README.zh-CN.md
- Document supported platforms: Xiaohongshu, Bilibili, Twitter, Zhihu
- Include prerequisites (yt-dlp installation)
- Add usage examples for all download commands
- Document the `download` pipeline step for YAML adapters
- Update built-in commands table with new `download` commands
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: preserve zhihu ordered list content
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: jackwener <jakevingoo@gmail.com>
OpenCLI
Make any website or Electron App your CLI.
Zero risk · Reuse Chrome login · AI-powered discovery · Browser + Desktop automation
A CLI tool that turns any website or Electron app into a command-line interface — Bilibili, Zhihu, 小红书, Twitter/X, Reddit, YouTube, Antigravity, and many more — powered by browser session reuse and AI-native discovery.
🔥 CLI All Electron Apps! The Most Powerful Update Has Arrived! 🔥 Turn ANY Electron application into a CLI tool! Recombine, script, and extend applications like Antigravity Ultra seamlessly. Now AI can control itself natively. Unlimited possibilities await!
Table of Contents
- Highlights
- Prerequisites
- Quick Start
- Built-in Commands
- Download Support
- Output Formats
- For AI Agents (Developer Guide)
- Remote Chrome (Server/Headless)
- Testing
- Troubleshooting
- Releasing New Versions
- License
Highlights
- CLI All Electron — CLI-ify apps like Antigravity Ultra! Now AI can control itself natively using cc/openclaw!
- Account-safe — Reuses Chrome's logged-in state; your credentials never leave the browser.
- AI Agent ready —
explorediscovers APIs,synthesizegenerates adapters,cascadefinds auth strategies. - Self-healing setup —
opencli setupauto-discovers tokens;opencli doctordiagnoses config across 10+ tools;--fixrepairs them all. - Dynamic Loader — Simply drop
.tsor.yamladapters into theclis/folder for auto-registration. - Dual-Engine Architecture — Supports both YAML declarative data pipelines and robust browser runtime TypeScript injections.
Prerequisites
- Node.js: >= 18.0.0
- Chrome running and logged into the target site (e.g. bilibili.com, zhihu.com, xiaohongshu.com).
⚠️ Important: Browser commands reuse your Chrome login session. You must be logged into the target website in Chrome before running commands. If you get empty data or errors, check your login status first.
OpenCLI connects to your browser through the Playwright MCP Bridge extension.
It prefers an existing local/global @playwright/mcp install and falls back to npx -y @playwright/mcp@latest automatically when no local MCP server is found.
Playwright MCP Bridge Extension Setup
- Install Playwright MCP Bridge extension in Chrome.
- Run
opencli setup— discovers the token, distributes it to your tools, and verifies connectivity:
opencli setup
The interactive TUI will:
- 🔍 Auto-discover
PLAYWRIGHT_MCP_EXTENSION_TOKENfrom Chrome (no manual copy needed) - ☑️ Show all detected tools (Codex, Cursor, Claude Code, Gemini CLI, etc.)
- ✏️ Update only the files you select (Space to toggle, Enter to confirm)
- 🔌 Auto-verify browser connectivity after writing configs
Tip
: Use
opencli doctorfor ongoing diagnosis and maintenance:opencli doctor # Read-only token & config diagnosis opencli doctor --live # Also test live browser connectivity opencli doctor --fix # Fix mismatched configs (interactive) opencli doctor --fix -y # Fix all configs non-interactively
Alternative: CDP Mode (For Servers/Headless) If you cannot install the browser extension (e.g. running OpenCLI on a remote headless server), you can connect OpenCLI to your local Chrome via CDP using SSH tunnels or reverse proxies. See the CDP Connection Guide for detailed instructions.
Manual setup (alternative)
Add token to your MCP client config (e.g. Claude/Cursor):
{
"mcpServers": {
"playwright": {
"command": "npx",
"args": ["-y", "@playwright/mcp@latest", "--extension"],
"env": {
"PLAYWRIGHT_MCP_EXTENSION_TOKEN": "<your-token-here>"
}
}
}
}
Export in shell (e.g. ~/.zshrc):
export PLAYWRIGHT_MCP_EXTENSION_TOKEN="<your-token-here>"
Quick Start
Install via npm (recommended)
npm install -g @jackwener/opencli
opencli setup # One-time: configure Playwright MCP token
Then use directly:
opencli list # See all commands
opencli list -f yaml # List commands as YAML
opencli hackernews top --limit 5 # Public API, no browser
opencli bilibili hot --limit 5 # Browser command
opencli zhihu hot -f json # JSON output
opencli zhihu hot -f yaml # YAML output
Install from source (for developers)
git clone git@github.com:jackwener/opencli.git
cd opencli
npm install
npm run build
npm link # Link binary globally
opencli list # Now you can use it anywhere!
Update
npm install -g @jackwener/opencli@latest
Built-in Commands
Run opencli list for the live registry.
| Site | Commands | Mode |
|---|---|---|
trending bookmarks profile search timeline thread following followers notifications post reply delete like article follow unfollow bookmark unbookmark download |
🔐 Browser | |
hot frontpage popular search subreddit read user user-posts user-comments upvote save comment subscribe saved upvoted |
🔐 Browser | |
| cursor | status send read new dump composer model extract-code ask screenshot history export |
🖥️ Desktop |
| bilibili | hot search me favorite history feed subtitle dynamic ranking following user-videos download |
🔐 Browser |
| codex | status send read new extract-diff model ask screenshot history export |
🖥️ Desktop |
| chatwise | status new send read ask model history export screenshot |
🖥️ Desktop |
| notion | status search read new write sidebar favorites export |
🖥️ Desktop |
| discord | status send read channels servers search members |
🖥️ Desktop |
| v2ex | hot latest topic daily me notifications |
🌐 / 🔐 |
| xueqiu | feed hot-stock hot search stock watchlist |
🔐 Browser |
| antigravity | status send read new evaluate |
🖥️ Desktop |
| chatgpt | status new send read ask |
🖥️ Desktop |
status send new search read |
🖥️ Desktop | |
| feishu | status send new search read |
🖥️ Desktop |
| xiaohongshu | search notifications feed me user download |
🔐 Browser |
| xiaoyuzhou | podcast podcast-episodes episode |
🌐 Public |
| zhihu | hot search question download |
🔐 Browser |
| youtube | search video transcript |
🔐 Browser |
| boss | search detail |
🔐 Browser |
| coupang | search add-to-cart |
🔐 Browser |
| bbc | news |
🌐 Public |
| ctrip | search |
🔐 Browser |
| github | search |
🌐 Public |
| hackernews | top |
🌐 Public |
search |
🔐 Browser | |
| reuters | search |
🔐 Browser |
| smzdm | search |
🔐 Browser |
hot |
🔐 Browser | |
| yahoo-finance | quote |
🔐 Browser |
Download Support
OpenCLI supports downloading images, videos, and articles from supported platforms.
Supported Platforms
| Platform | Content Types | Notes |
|---|---|---|
| xiaohongshu | Images, Videos | Downloads all media from a note |
| bilibili | Videos | Requires yt-dlp installed |
| Images, Videos | Downloads from user media tab or single tweet | |
| zhihu | Articles (Markdown) | Exports articles with optional image download |
Prerequisites
For video downloads from streaming platforms, you need to install yt-dlp:
# Install yt-dlp
pip install yt-dlp
# or
brew install yt-dlp
Usage Examples
# Download images/videos from Xiaohongshu note
opencli xiaohongshu download --note-id abc123 --output ./xhs
# Download Bilibili video (requires yt-dlp)
opencli bilibili download --bvid BV1xxx --output ./bilibili
opencli bilibili download --bvid BV1xxx --quality 1080p # Specify quality
# Download Twitter media from user
opencli twitter download --username elonmusk --limit 20 --output ./twitter
# Download single tweet media
opencli twitter download --tweet-url "https://x.com/user/status/123" --output ./twitter
# Export Zhihu article to Markdown
opencli zhihu download --url "https://zhuanlan.zhihu.com/p/xxx" --output ./zhihu
# Export with local images
opencli zhihu download --url "https://zhuanlan.zhihu.com/p/xxx" --download-images
Pipeline Step (for YAML adapters)
The download step can be used in YAML pipelines:
pipeline:
- fetch: https://api.example.com/media
- download:
url: ${{ item.imageUrl }}
dir: ./downloads
filename: ${{ item.title | sanitize }}.jpg
concurrency: 5
skip_existing: true
Output Formats
All built-in commands support --format / -f with table, json, yaml, md, and csv.
The list command supports the same format options, and keeps --json for backward compatibility.
opencli list -f yaml # Command registry as YAML
opencli bilibili hot -f table # Default: rich terminal table
opencli bilibili hot -f json # JSON (pipe to jq or LLMs)
opencli bilibili hot -f yaml # YAML (human-readable structured output)
opencli bilibili hot -f md # Markdown
opencli bilibili hot -f csv # CSV
opencli bilibili hot -v # Verbose: show pipeline debug steps
For AI Agents (Developer Guide)
If you are an AI assistant tasked with creating a new command adapter for opencli, please follow the AI Agent workflow below:
Quick mode: To generate a single command for a specific page URL, see CLI-ONESHOT.md — just a URL + one-line goal, 4 steps done.
Full mode: Before writing any adapter code, read CLI-EXPLORER.md. It contains the complete browser exploration workflow, the 5-tier authentication strategy decision tree, and debugging guide.
# 1. Deep Explore — discover APIs, infer capabilities, detect framework
opencli explore https://example.com --site mysite
# 2. Synthesize — generate YAML adapters from explore artifacts
opencli synthesize mysite
# 3. Generate — one-shot: explore → synthesize → register
opencli generate https://example.com --goal "hot"
# 4. Strategy Cascade — auto-probe: PUBLIC → COOKIE → HEADER
opencli cascade https://api.example.com/data
Explore outputs to .opencli/explore/<site>/ (manifest.json, endpoints.json, capabilities.json, auth.json).
Testing
See TESTING.md for the full testing guide, including:
- Current test coverage (unit + E2E tests across browser and desktop adapters)
- How to run tests locally
- How to add tests when creating new adapters
- CI/CD pipeline with sharding
- Headless browser mode (
OPENCLI_HEADLESS=1)
# Quick start
npm run build
npx vitest run # All tests
npx vitest run src/ # Unit tests only
npx vitest run tests/e2e/ # E2E tests
Troubleshooting
- "Failed to connect to Playwright MCP Bridge"
- Ensure the Playwright MCP extension is installed and enabled in your running Chrome.
- Restart the Chrome browser if you just installed the extension.
- Empty data returns or 'Unauthorized' error
- Your login session in Chrome might have expired. Open a normal Chrome tab, navigate to the target site, and log in or refresh the page to prove you are human.
- Node API errors
- Make sure you are using Node.js >= 18. Some dependencies require modern Node APIs.
- Token issues
- Run
opencli doctorto diagnose token configuration across all tools.
- Run
Releasing New Versions
npm version patch # 0.1.0 → 0.1.1
npm version minor # 0.1.0 → 0.2.0
git push --follow-tags
The CI will automatically build, create a GitHub release, and publish to npm.