github-featured/elizaos--eliza

Fork 0

Files

T

WeHub Mirror abf0e694a2

Hetzner Agent E2E / Provision + deploy + healthcheck (push) Has been cancelled

Hetzner Agent E2E / Teardown (push) Has been cancelled

Scenario PR E2E / Classify changed paths (push) Has been cancelled

Scenario PR E2E / Zero-Key unit + UI coverage (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser core (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser view lifecycle (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser all-pages (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser feature interactions (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser cloud keyless (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser ratcheted (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser auto-discovered specs (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser dashboard device matrix (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser Pixel-7 real-touch lane (push) Has been cancelled

Scenario PR E2E / Zero-Key app browser WebKit lane (push) Has been cancelled

Scenario PR E2E / Zero-Key accounts UI e2e (real API + pool + disk) (push) Has been cancelled

Scenario PR E2E / Zero-Key full-walkthrough (mock lane) (push) Has been cancelled

Scenario PR E2E / Zero-Key app diagnostics (push) Has been cancelled

Scenario PR E2E / Zero-Key scenario runner E2E (push) Has been cancelled

Scenario PR E2E / Zero-Key Deterministic E2E (push) Has been cancelled

Docker CI Smoke / Classify changed paths (push) Has been cancelled

Docker CI Smoke / Build production Docker image (+ smoke boot) (push) Has been cancelled

Build libelizainference (Android, FFI) / prepare-matrix (push) Has been cancelled

Chat shell gestures / Chat shell gesture + parity e2e (push) Has been cancelled

Cloud Gateway Discord / Test (push) Has been cancelled

Cloud Gateway Webhook / Test (push) Has been cancelled

Cloud Tests / lint-and-types (push) Has been cancelled

Cloud Tests / unit-tests (push) Has been cancelled

Build Agent Image / build-and-push (push) Has been cancelled

Electrobun Submodule Guard / electrobun gitlink is fetchable (push) Has been cancelled

gitleaks / gitleaks (push) Has been cancelled

Markdown Links / Relative Markdown Links (push) Has been cancelled

Orchestrator multi-account / multi-account selection e2e (push) Has been cancelled

Publish @elizaos/plugin-elizacloud / verify_version (push) Has been cancelled

Quality (Extended) / Homepage Build (PR smoke) (push) Has been cancelled

Quality (Extended) / Comment-only diff guard (push) Has been cancelled

Quality (Extended) / Format (push) Has been cancelled

Quality (Extended) / Develop Gate (secret scan + UI determinism) (push) Has been cancelled

Quality (Extended) / Develop Gate (lint) (push) Has been cancelled

Sandbox Live Smoke / Sandbox live smoke (push) Has been cancelled

Snap Build & Test / Build Snap (amd64) (push) Has been cancelled

Snap Build & Test / Build Snap (arm64) (push) Has been cancelled

Tests / Classify changed paths (push) Has been cancelled

UI Extended Fixture E2E / fixture-e2e (push) Has been cancelled

Cloud Tests / integration-tests (push) Has been cancelled

CUDA Coverage Continuity / Fail-closed manifest contracts (push) Has been cancelled

CUDA Coverage Continuity / Exact-head CUDA fixtures + runtime graph (push) Has been cancelled

Deploy Apps Worker (Product 2) / Determine environment (push) Has been cancelled

Deploy Eliza Provisioning Worker / Determine environment (push) Has been cancelled

Deploy Homepage / build-and-deploy (push) Has been cancelled

Dev Smoke / Classify changed paths (push) Has been cancelled

Tests / Remote Capability Provider Live E2E (push) Has been cancelled

Training Stack / CPU smoke (lint + import) (push) Has been cancelled

Training Stack / GPU build (QJL nvcc + Triton JIT) (push) Has been cancelled

UI Core Fixture E2E / ui-core-fixture-e2e (push) Has been cancelled

vault-ci / app-core wiring tests (push) Has been cancelled

verify-patches / verify patches/CHECKSUMS.sha256 (push) Has been cancelled

UI Story Gate / story-gate (push) Has been cancelled

vault-ci / test (macos-latest) (push) Has been cancelled

vault-ci / test (ubuntu-latest) (push) Has been cancelled

vault-ci / test (windows-latest) (push) Has been cancelled

Windows CI / windows ([bun run --cwd packages/app-core test bun run --cwd packages/elizaos test bun run --cwd packages/cloud/shared test], app-and-cli) (push) Has been cancelled

Windows CI / windows ([bun run --cwd packages/core typecheck bun run --cwd packages/shared typecheck bun run --cwd packages/cloud/shared typecheck bun run --cwd packages/core test bun run --cwd packages/shared test], core-runtime, 75) (push) Has been cancelled

Windows CI / windows ([bun run --cwd packages/scenario-runner test bun run --cwd packages/vault test bun run --cwd plugins/plugin-coding-tools test], framework-packages) (push) Has been cancelled

Windows CI / windows ([bun run --cwd plugins/plugin-coding-tools build node packages/scripts/run-turbo.mjs run build --filter=@elizaos/core --filter=@elizaos/shared --filter=@elizaos/agent --concurrency=1 node packages/scripts/run-python.mjs --version node packages… (push) Has been cancelled

Windows CI / windows ([bun run --cwd plugins/plugin-elizacloud test bun run --cwd plugins/plugin-discord test bun run --cwd plugins/plugin-anthropic test bun run --cwd plugins/plugin-openai test bun run --cwd plugins/plugin-app-control test bun run --cwd plugins/pl… (push) Has been cancelled

Cloud Tests / e2e-tests (push) Has been cancelled

Deploy Apps Worker (Product 2) / Deploy apps worker to apps-control host (${{ needs.determine-env.outputs.environment }}) (push) Has been cancelled

Deploy Eliza Provisioning Worker / Deploy worker to Hetzner host (${{ needs.determine-env.outputs.environment }} @ ${{ needs.determine-env.outputs.deployment_sha }}) (push) Has been cancelled

Dev Smoke / bun run dev onboarding chat (push) Has been cancelled

Dev Smoke / Vite HMR dependency-level smoke (push) Has been cancelled

Publish @elizaos/plugin-elizacloud / publish_npm (push) Has been cancelled

Tests / Test-runner vacuous-green guard (push) Has been cancelled

Tests / Server Tests (push) Has been cancelled

Tests / Client Tests (push) Has been cancelled

Tests / Plugin Tests (1/4) (push) Has been cancelled

Tests / Plugin Tests (2/4) (push) Has been cancelled

Tests / Plugin Tests (3/4) (push) Has been cancelled

Tests / Plugin Tests (4/4) (push) Has been cancelled

Tests / Plugin Tests (push) Has been cancelled

Tests / Integration Lane (personal-assistant) (push) Has been cancelled

Tests / Electrobun Desktop Contract (push) Has been cancelled

Tests / Zero-Key unit + UI coverage (push) Has been cancelled

Tests / Zero-Key app browser (push) Has been cancelled

Tests / Zero-Key diagnostics (push) Has been cancelled

Tests / Zero-Key scenario runner (push) Has been cancelled

Tests / Zero-Key harness E2E (push) Has been cancelled

Tests / Zero-Key Deterministic E2E (push) Has been cancelled

Build libelizainference (Android, FFI) / ${{ matrix.abi }} (push) Has been cancelled

Tests / Cloud Live E2E (Eliza Cloud) (push) Has been cancelled

Tests / Remote Capability GitHub Live Artifact Validator (push) Has been cancelled

Tests / Script Tests (Linux) (push) Has been cancelled

Tests / Merge Queue Quality Gate (push) Has been cancelled

Tests / ci-ok (push) Has been cancelled

WeHub snapshot of 9298cb46af86e1bd3a40a60e135ea855d70f229a

2026-08-07 16:49:07 +08:00

3.2 KiB

Raw Permalink Blame History

`@elizaos/plugin-computeruse`

Desktop automation plugin for elizaOS agents — screenshots, mouse / keyboard control, browser CDP automation, window management, clipboard, and the OCR provider registry that other plugins contribute to.

Ported from coasty-ai/open-computer-use (Apache 2.0).

Boundary with `@elizaos/plugin-vision`

This plugin owns the OS surfaces:

screen / display capture (src/platform/capture.ts, src/platform/displays.ts, ComputerUseService.captureScreen()),
input + windows + clipboard + accessibility,
the OCR provider registries — OcrProvider (line-level) and CoordOcrProvider (hierarchical with absolute coords), defined in src/mobile/ocr-provider.ts.

@elizaos/plugin-vision owns the camera pipeline, scene description via runtime.useModel(IMAGE_DESCRIPTION), the screen tiler, the detector pipeline (faces / people / objects), and the OCR implementations themselves. plugin-vision consumes capture from this plugin via runtime.getService("computeruse") and contributes the hierarchical OCR adapter into this plugin's registerCoordOcrProvider seam at boot.

Both seams are runtime feature-detected — neither package depends on the other.

Enabling

Config: features.computeruse: true
Env: COMPUTER_USE_ENABLED=1

Platform requirements

OS	Capture	Input
macOS	`screencapture` (built-in)	`cliclick` (`brew install cliclick`), AppleScript
Linux	`import` (ImageMagick) / `scrot`	`xdotool` (`sudo apt install xdotool`)
Windows	PowerShell + `System.Drawing`	PowerShell
Browser	—	`puppeteer-core` + Chrome / Edge / Brave

Surface

Actions — COMPUTER_USE (canonical screenshot / click / key / scroll / etc.), WINDOW (list / focus / arrange / move /...), and COMPUTER_USE_AGENT (high-level goal-driven autonomous desktop loop: Brain → Cascade → dispatch up to maxSteps iterations). Subactions of COMPUTER_USE and WINDOW are promoted to virtual top-level actions (e.g. COMPUTER_USE_CLICK, WINDOW_FOCUS) so the planner picks a specific verb directly from the catalogue.
Services — ComputerUseService (serviceType = "computeruse") and VisionContextProvider.
Providers — computerStateProvider, sceneProvider.
Routes — approval inbox + SSE stream + approval-mode toggle under /api/computer-use/....

File operations + shell

File operations live on the FILE action; shell / terminal access lives on the SHELL action. They are not exposed by this plugin.

3.2 KiB Raw Permalink Blame History

@elizaos/plugin-computeruse

Boundary with @elizaos/plugin-vision