github-featured/strukto-ai--mirage
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

refactor ts index cache (#73)

Browse Source 
* refactor ts index cache

* refactor(py): rename _index_ttl to index_ttl to align with ts indexTtl

* ci(ts): add redis service so redis index/file cache tests run, aligning with python

* ci: move redis examples into per-language test workflows with integ/ truth files

Reuse the integ/ + truth-file pattern: each example's expected output lives
in integ/truth_*.txt and integ/check_lines.sh asserts every truth line
appears in stdout (substring match, tolerant of volatile snapshot paths /
stat mtimes). Moves all 7 redis examples out of infra-example.yml's
examples-redis job into test_python.yml (general/index/cache/vfs) and
test_typescript.yml (general/index/cache), and deletes that job.

* ci: migrate all examples to integ/ truth-file checks; delete infra-example.yml

Move every example smoke test (ram, disk, pyodide python, custom_command,
redis, fs-shim) out of infra-example.yml into the per-language test
workflows, checked via integ/check_lines.sh against integ/truth_*.txt.
Python examples run in test_python.yml; TS examples in test_typescript.yml
(fs-shim gets its own node-24 job for --experimental-wasm-jspi). Harden the
checker to grep a temp file (handles large/binary example output). Deletes
infra-example.yml entirely.

* examples(ts): add disk_vfs.ts + redis_vfs.ts to match python VFS examples

Ports examples/python/disk/disk_vfs.py and example_redis_vfs.py to TS using
the patchNodeFs + require('fs') VFS idiom (mirrors ram_vfs.ts). Wires both
into test_typescript.yml with integ/truth_ts_disk_vfs.txt and
integ/truth_ts_redis_vfs.txt, closing the py/ts VFS coverage gap.

* examples(ts): rename python_ram.ts -> pyodide_ram.ts (clarify Pyodide-backed)

* integ: group truth files under integ/truth/<lang>/<name>.txt

* examples(ts): rename python_* pyodide examples to pyodide_* (basic/env/heredoc/script/vfs)

Consistent Pyodide naming for the TS python-execution examples and their
integ/truth/typescript/ files; updates test_typescript.yml refs and the
docs/typescript/python-fs.mdx links to pyodide_vfs.ts.

* examples(ts): move pyodide examples to examples/typescript/pyodide/

The python-execution examples (mirage TS python3 command, Pyodide-backed)
now live under examples/typescript/pyodide/ as basic/env/heredoc/script/ram/vfs.ts
instead of examples/typescript/python/pyodide_*.ts. Updates test_typescript.yml
and docs/typescript/python-fs.mdx refs.
This commit is contained in:
Zecheng Zhang
2026-05-21 04:16:53 -07:00
committed by GitHub
parent c5aa6de5ac
commit 91be6f6c53
95 changed files with 1125 additions and 606 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/infra-example.yml
-435
View File
@@ -1,435 +0,0 @@
name: Examples
"on":
push:
branches: [main]
paths:
- examples/**
- python/**
- typescript/**
- .github/workflows/infra-example.yml
pull_request:
branches: [main]
paths:
- examples/**
- python/**
- typescript/**
- .github/workflows/infra-example.yml
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
examples-ram:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install Python dependencies
working-directory: python
run: uv sync --all-extras --no-extra camel
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r build
- name: Run RAM example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/ram/ram.py 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -q "hello.txt" || (echo "FAIL: missing 'hello.txt'" && exit 1)
echo "$output" | grep -q '"alice"' || (echo "FAIL: missing jq output" && exit 1)
echo "$output" | grep -q "goodbye world" || (echo "FAIL: missing sed output" && exit 1)
- name: Run RAM example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx ram/ram.ts 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -q "hello.txt" || (echo "FAIL: missing 'hello.txt'" && exit 1)
echo "$output" | grep -q '"alice"' || (echo "FAIL: missing jq output" && exit 1)
echo "$output" | grep -q "goodbye world" || (echo "FAIL: missing sed output" && exit 1)
- name: Run RAM VFS example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/ram/ram_vfs.py 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -qi "hello.txt: true" || (echo "FAIL: missing exists check" && exit 1)
echo "$output" | grep -qi "nope.txt: false" || (echo "FAIL: missing not-exists check" && exit 1)
- name: Run RAM VFS example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx ram/ram_vfs.ts 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -qi "hello.txt: true" || (echo "FAIL: missing exists check" && exit 1)
echo "$output" | grep -qi "nope.txt: false" || (echo "FAIL: missing not-exists check" && exit 1)
examples-disk:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install Python dependencies
working-directory: python
run: uv sync --all-extras --no-extra camel
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r build
- name: Run Disk example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/disk/disk.py 2>&1)
echo "$output"
echo "$output" | grep -q "Strukto" || (echo "FAIL: missing 'Strukto' from JSON" && exit 1)
echo "$output" | grep -q "example.json" || (echo "FAIL: missing file listing" && exit 1)
- name: Run Disk example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx disk/disk.ts 2>&1)
echo "$output"
echo "$output" | grep -q "Strukto" || (echo "FAIL: missing 'Strukto' from JSON" && exit 1)
echo "$output" | grep -q "example.json" || (echo "FAIL: missing file listing" && exit 1)
- name: Run Disk VFS example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/disk/disk_vfs.py 2>&1)
echo "$output"
echo "$output" | grep -q "example.json" || (echo "FAIL: missing file listing" && exit 1)
examples-redis:
runs-on: ubuntu-latest
services:
redis:
image: redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install Python dependencies
working-directory: python
run: uv sync --all-extras --no-extra camel
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r build
- name: Run Redis example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/redis_resource/example_redis.py 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -q "hello.txt" || (echo "FAIL: missing 'hello.txt'" && exit 1)
echo "$output" | grep -q '"alice"' || (echo "FAIL: missing jq output" && exit 1)
echo "$output" | grep -q "goodbye world" || (echo "FAIL: missing sed output" && exit 1)
- name: Run Redis example (TypeScript)
env:
REDIS_URL: redis://localhost:6379/0
run: |
output=$(pnpm -C examples/typescript exec tsx redis/redis.ts 2>&1)
echo "$output"
echo "$output" | grep -q "hello world" || (echo "FAIL: missing 'hello world'" && exit 1)
echo "$output" | grep -q "hello.txt" || (echo "FAIL: missing 'hello.txt'" && exit 1)
echo "$output" | grep -q '"alice"' || (echo "FAIL: missing jq output" && exit 1)
echo "$output" | grep -q "goodbye world" || (echo "FAIL: missing sed output" && exit 1)
- name: Run Redis VFS example (Python)
run: ./python/.venv/bin/python examples/python/redis_resource/example_redis_vfs.py
examples-python-fs-shim:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node 24 (for --experimental-wasm-jspi)
uses: actions/setup-node@v6
with:
node-version: "24"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r --filter './packages/*' build
- name: Run Python FS shim example (no-creds smoke test)
working-directory: examples/typescript
run: |
output=$(node --experimental-wasm-jspi --import tsx/esm python/vfs.ts 2>&1)
echo "$output"
# Demos that always run (no creds needed): RAM read+write, lazy-on-miss, PIL save.
echo "$output" | grep -q "host sees: written from python" || (echo "FAIL: RAM write did not flush" && exit 1)
echo "$output" | grep -q "listdir: \['note.md'\]" || (echo "FAIL: lazy-on-miss listdir failed" && exit 1)
echo "$output" | grep -q "read: lazy demo" || (echo "FAIL: lazy-on-miss read failed" && exit 1)
echo "$output" | grep -q "PNG magic: 89 50 4e 47" || (echo "FAIL: PIL save did not produce a PNG" && exit 1)
examples-python:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v7
with:
enable-cache: true
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install Python dependencies
working-directory: python
run: uv sync --all-extras --no-extra camel
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r build
- name: Run python3 example (Python, RAM)
run: |
output=$(./python/.venv/bin/python examples/python/ram/ram_python.py 2>&1)
echo "$output"
# Regression guard: argv after -c must stay raw, not be path-resolved
echo "$output" | grep -q "argv after -c: \['alpha', 'beta'\]" || (echo "FAIL: argv after -c regression" && exit 1)
# Script + argv path
echo "$output" | grep -q "argv after script: \['one', 'two'\]" || (echo "FAIL: argv after script" && exit 1)
# Script dispatched through VFS
echo "$output" | grep -q "hello from vfs" || (echo "FAIL: script via dispatch" && exit 1)
# Session env passthrough
echo "$output" | grep -q "hello_mirage" || (echo "FAIL: env passthrough" && exit 1)
- name: Run python_basic example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx python/python_basic.ts 2>&1)
echo "$output"
# Regression guard: argv after -c
echo "$output" | grep -q "\['alpha', 'beta'\]" || (echo "FAIL: argv after -c regression" && exit 1)
# SystemExit honored
echo "$output" | grep -q "exit: 3 (expect 3)" || (echo "FAIL: SystemExit code" && exit 1)
# Traceback propagates to stderr
echo "$output" | grep -q "RuntimeError: boom" || (echo "FAIL: traceback on stderr" && exit 1)
- name: Run python_env example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx python/python_env.ts 2>&1)
echo "$output"
echo "$output" | grep -q "stdout: bar" || (echo "FAIL: env passthrough" && exit 1)
echo "$output" | grep -q "wsA: alice" || (echo "FAIL: cross-workspace isolation wsA" && exit 1)
echo "$output" | grep -q "wsB: bob" || (echo "FAIL: cross-workspace isolation wsB" && exit 1)
- name: Run python_heredoc example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx python/python_heredoc.ts 2>&1)
echo "$output"
# quoted heredoc: $X literal (fixed-string match to avoid regex anchor)
echo "$output" | grep -qF 'stdout: $X' || (echo "FAIL: quoted heredoc" && exit 1)
# unquoted heredoc: $X expanded
echo "$output" | grep -q "stdout: shellval" || (echo "FAIL: unquoted heredoc" && exit 1)
# dash-strip
echo "$output" | grep -q "item-0" || (echo "FAIL: dash-strip heredoc" && exit 1)
# heredoc + pipe
echo "$output" | grep -q "^keep 1$" || (echo "FAIL: heredoc + pipe" && exit 1)
# heredoc in for-loop
echo "$output" | grep -q "hello, alice!" || (echo "FAIL: heredoc in for-loop" && exit 1)
- name: Run python_script example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx python/python_script.ts 2>&1)
echo "$output"
echo "$output" | grep -q "count: 4" || (echo "FAIL: script + stdin pipeline" && exit 1)
echo "$output" | grep -q "alice -> login" || (echo "FAIL: inline -c pipeline" && exit 1)
- name: Run python_ram example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx python/python_ram.ts 2>&1)
echo "$output"
echo "$output" | grep -q "hello, world!" || (echo "FAIL: bare script run" && exit 1)
echo "$output" | grep -q "sum(1..10): 55" || (echo "FAIL: script computation" && exit 1)
# Regression guard: argv after script
echo "$output" | grep -q "args: \['alice', 'bob'\]" || (echo "FAIL: argv after script" && exit 1)
echo "$output" | grep -q "alice: 2" || (echo "FAIL: cat | python pipeline" && exit 1)
echo "$output" | grep -q "python3: /ram/missing.py: No such file" || (echo "FAIL: missing script error" && exit 1)
examples-custom-command:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"
- name: Install uv
uses: astral-sh/setup-uv@v7
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node
uses: actions/setup-node@v6
with:
node-version: "22"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install Python dependencies
working-directory: python
run: uv sync --all-extras --no-extra camel
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r build
- name: Run custom_command example (Python)
run: |
output=$(./python/.venv/bin/python examples/python/other/custom_command.py 2>&1)
echo "$output"
echo "$output" | grep -q "hello from RAMAccessor: /ram/note.txt" || (echo "FAIL: missing RAM greet output" && exit 1)
echo "$output" | grep -q "hello from DiskAccessor: /disk/note.txt" || (echo "FAIL: missing Disk greet output" && exit 1)
- name: Run custom_command example (TypeScript)
run: |
output=$(pnpm -C examples/typescript exec tsx other/custom_command.ts 2>&1)
echo "$output"
echo "$output" | grep -q "hello from RAMAccessor: /ram/note.txt" || (echo "FAIL: missing RAM greet output" && exit 1)
echo "$output" | grep -q "hello from DiskAccessor: /disk/note.txt" || (echo "FAIL: missing Disk greet output" && exit 1)
.github/workflows/test_python.yml
+20
View File
@@ -5,11 +5,15 @@ name: Test (Python)
branches: [main]
paths:
- python/**
- examples/python/**
- integ/**
- .github/workflows/test_python.yml
pull_request:
branches: [main]
paths:
- python/**
- examples/python/**
- integ/**
- .github/workflows/test_python.yml
workflow_dispatch:
@@ -58,6 +62,22 @@ jobs:
REDIS_URL: redis://localhost:6379/0
run: uv run pytest --ignore=tests/agents/camel
- name: Examples (output checked against integ/ truth files)
env:
REDIS_URL: redis://localhost:6379/0
run: |
run() { echo "## $1"; ./python/.venv/bin/python "$1" 2>&1 | bash integ/check_lines.sh "$2"; }
run examples/python/ram/ram.py integ/truth/python/ram.txt
run examples/python/ram/ram_vfs.py integ/truth/python/ram_vfs.txt
run examples/python/ram/ram_python.py integ/truth/python/ram_python.txt
run examples/python/disk/disk.py integ/truth/python/disk.txt
run examples/python/disk/disk_vfs.py integ/truth/python/disk_vfs.txt
run examples/python/other/custom_command.py integ/truth/python/custom_command.txt
run examples/python/redis_resource/example_redis.py integ/truth/python/redis.txt
run examples/python/redis_resource/example_redis_index.py integ/truth/python/redis_index.txt
run examples/python/redis_resource/example_redis_cache.py integ/truth/python/redis_cache.txt
run examples/python/redis_resource/example_redis_vfs.py integ/truth/python/redis_vfs.txt
- name: Sync Python dependencies for camel
working-directory: python
run: uv sync --extra camel
.github/workflows/test_typescript.yml
+70
View File
@@ -5,11 +5,15 @@ name: Test (TypeScript)
branches: [main]
paths:
- typescript/**
- examples/typescript/**
- integ/**
- .github/workflows/test_typescript.yml
pull_request:
branches: [main]
paths:
- typescript/**
- examples/typescript/**
- integ/**
- .github/workflows/test_typescript.yml
workflow_dispatch:
@@ -20,6 +24,16 @@ concurrency:
jobs:
test:
runs-on: ubuntu-latest
services:
redis:
image: redis:7
ports:
- 6379:6379
options: >-
--health-cmd "redis-cli ping"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v6
@@ -49,4 +63,60 @@ jobs:
- name: Run TypeScript tests
working-directory: typescript
env:
REDIS_URL: redis://localhost:6379/0
run: pnpm test
- name: Examples (output checked against integ/ truth files)
env:
REDIS_URL: redis://localhost:6379/0
run: |
run() { echo "## $1"; pnpm -C examples/typescript exec tsx "$1" 2>&1 | bash integ/check_lines.sh "$2"; }
run ram/ram.ts integ/truth/typescript/ram.txt
run ram/ram_vfs.ts integ/truth/typescript/ram_vfs.txt
run disk/disk.ts integ/truth/typescript/disk.txt
run disk/disk_vfs.ts integ/truth/typescript/disk_vfs.txt
run other/custom_command.ts integ/truth/typescript/custom_command.txt
run pyodide/basic.ts integ/truth/typescript/pyodide_basic.txt
run pyodide/env.ts integ/truth/typescript/pyodide_env.txt
run pyodide/heredoc.ts integ/truth/typescript/pyodide_heredoc.txt
run pyodide/script.ts integ/truth/typescript/pyodide_script.txt
run pyodide/ram.ts integ/truth/typescript/pyodide_ram.txt
run redis/redis.ts integ/truth/typescript/redis.txt
run redis/redis_index.ts integ/truth/typescript/redis_index.txt
run redis/redis_cache.ts integ/truth/typescript/redis_cache.txt
run redis/redis_vfs.ts integ/truth/typescript/redis_vfs.txt
python-fs-shim:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Install pnpm
uses: pnpm/action-setup@v6
with:
version: 10.32.1
- name: Set up Node 24 (for --experimental-wasm-jspi)
uses: actions/setup-node@v6
with:
node-version: "24"
cache: pnpm
cache-dependency-path: typescript/pnpm-lock.yaml
- name: Install node-gyp
run: npm install -g node-gyp
- name: Install TypeScript dependencies
working-directory: typescript
run: pnpm install --frozen-lockfile
- name: Build TypeScript packages
working-directory: typescript
run: pnpm -r --filter './packages/*' build
- name: Python FS shim example (output checked against integ/ truth file)
working-directory: examples/typescript
run: |
node --experimental-wasm-jspi --import tsx/esm pyodide/vfs.ts 2>&1 \
| bash ../../integ/check_lines.sh ../../integ/truth/typescript/pyodide_vfs.txt
docs/python/resource/disk.mdx
+1 -1
View File
@@ -49,7 +49,7 @@ to the root boundary.
## Cache
The Disk resource uses `IndexCacheStore` with `_index_ttl = 60` (1 minute).
The Disk resource uses `IndexCacheStore` with `index_ttl = 60` (1 minute).
Directory listings are cached for up to 60 seconds before being refreshed
from disk.
docs/python/resource/ram.mdx
+1 -1
View File
@@ -40,7 +40,7 @@ exists.
## Cache
The RAM resource uses `IndexCacheStore` with `_index_ttl = 0` (no
The RAM resource uses `IndexCacheStore` with `index_ttl = 0` (no
expiry). Since all data is in-memory, the index is always fresh,
no network calls or stale cache concerns.
docs/python/resource/s3.mdx
+1 -1
View File
@@ -66,7 +66,7 @@ Path mapping: virtual `/s3/data/file.txt` maps to S3 key `data/file.txt`.
## Cache
The S3 resource uses `IndexCacheStore` with `_index_ttl = 600` (10 minutes).
The S3 resource uses `IndexCacheStore` with `index_ttl = 600` (10 minutes).
Directory listings are cached for up to 600 seconds before being refreshed
from S3. This reduces API calls for repeated directory traversals.
docs/typescript/python-fs.mdx
+3 -3
View File
@@ -128,7 +128,7 @@ print(json.load(open("/ram/in.json"))["hello"])
console.log(r.stdoutText) // "world"
```
See the full demo at [`examples/typescript/python/vfs.ts`](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/python/vfs.ts).
See the full demo at [`examples/typescript/pyodide/vfs.ts`](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/pyodide/vfs.ts).
## Python packages (PIL, numpy, pandas, …)
@@ -167,7 +167,7 @@ Without JSPI, **reads** of preloaded files still work but **writes** throw `Runt
To run examples directly:
```bash
node --experimental-wasm-jspi --import tsx/esm examples/typescript/python/vfs.ts
node --experimental-wasm-jspi --import tsx/esm examples/typescript/pyodide/vfs.ts
```
## Errors you might see
@@ -182,4 +182,4 @@ node --experimental-wasm-jspi --import tsx/esm examples/typescript/python/vfs.ts
## See also
- [`python`](/typescript/python): broader `python3` builtin behavior (env, argv, stdin, exit codes)
- [`examples/typescript/python/vfs.ts`](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/python/vfs.ts): runnable demo across RAM, S3, GDocs, Linear
- [`examples/typescript/pyodide/vfs.ts`](https://github.com/strukto-ai/mirage/blob/main/examples/typescript/pyodide/vfs.ts): runnable demo across RAM, S3, GDocs, Linear
examples/python/redis_resource/example_redis_cache.py
+44
View File
@@ -0,0 +1,44 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
from mirage.cache.file.redis import RedisFileCacheStore
REDIS_URL = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
async def main() -> None:
# A Redis-backed file cache: file content is stored in Redis, so two
# Mirage processes sharing a key_prefix share one content cache.
key_prefix = "mirage:example:cache:"
cache = RedisFileCacheStore(url=REDIS_URL, key_prefix=key_prefix)
print("=== RedisFileCacheStore: FileCache backed by Redis ===")
await cache.set("/data/hello.txt", b"hello from redis cache")
got = await cache.get("/data/hello.txt")
print(f"cache.get: {got.decode() if got else '(none)'}")
# Another store with the same key_prefix sees the same data.
cache2 = RedisFileCacheStore(url=REDIS_URL, key_prefix=key_prefix)
got2 = await cache2.get("/data/hello.txt")
print(f"cache2.get: {got2.decode() if got2 else '(none)'}")
await cache.clear()
print("wiped cache keys from Redis")
if __name__ == "__main__":
asyncio.run(main())
examples/python/redis_resource/example_redis_index.py
+71
View File
@@ -0,0 +1,71 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import asyncio
import os
import time
from mirage import Workspace
from mirage.cache.index import (IndexEntry, RedisIndexCacheStore,
RedisIndexConfig)
from mirage.resource.ram import RAMResource
REDIS_URL = os.environ.get("REDIS_URL", "redis://localhost:6379/0")
def _file(name: str) -> IndexEntry:
return IndexEntry(id=name, name=name, resource_type="file")
async def main() -> None:
# A workspace-level ``index`` config points every mounted resource's
# index cache at the same Redis instance. Two separate Mirage processes
# that share a key_prefix then share one index -- the building block for
# running the same mounts locally and in a remote sandbox.
key_prefix = f"mirage:example:idx:{int(time.time() * 1000)}:"
index_config = RedisIndexConfig(url=REDIS_URL, key_prefix=key_prefix)
# Workspace A: a RAM mount whose INDEX is backed by Redis (not RAM).
ram_a = RAMResource()
Workspace({"/data": ram_a}, index=index_config)
print("index store A is redis-backed: "
f"{isinstance(ram_a.index, RedisIndexCacheStore)}")
# Populate the shared Redis index through workspace A.
await ram_a.index.put("/data/hello.txt", _file("hello.txt"))
await ram_a.index.set_dir(
"/data",
[("hello.txt", _file("hello.txt")), ("notes.md", _file("notes.md"))],
)
# Workspace B: a separate resource pointed at the same Redis index
# (same key_prefix). It sees what A cached without re-listing anything.
ram_b = RAMResource()
Workspace({"/data": ram_b}, index=index_config)
entry = await ram_b.index.get("/data/hello.txt")
name = entry.entry.name if entry.entry else "(none)"
print(f"shared index entry: {name}")
listing = await ram_b.index.list_dir("/data")
print(f"shared index listing: {', '.join(listing.entries or [])}")
await ram_a.index.clear()
await ram_a.index.close()
await ram_b.index.close()
print("wiped test keys from Redis")
if __name__ == "__main__":
asyncio.run(main())
examples/typescript/disk/disk_vfs.ts
+81
View File
@@ -0,0 +1,81 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { cpSync, mkdtempSync, rmSync } from 'node:fs'
import { createRequire } from 'node:module'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { DiskResource, MountMode, Workspace, patchNodeFs } from '@struktoai/mirage-node'
const require = createRequire(import.meta.url)
const fs = require('fs') as typeof import('fs')
const REPO_ROOT = new URL('../../..', import.meta.url).pathname
const DATA_DIR = join(REPO_ROOT, 'data')
const tmp = mkdtempSync(join(tmpdir(), 'mirage-disk-vfs-'))
const filesDir = join(tmp, 'files')
cpSync(DATA_DIR, filesDir, { recursive: true })
async function exists(p: string): Promise<boolean> {
try {
await fs.promises.stat(p)
return true
} catch {
return false
}
}
async function isDir(p: string): Promise<boolean> {
try {
return (await fs.promises.stat(p)).isDirectory()
} catch {
return false
}
}
async function main(): Promise<void> {
const ws = new Workspace(
{ '/data': new DiskResource({ root: filesDir }) },
{ mode: MountMode.READ },
)
patchNodeFs(ws)
console.log('=== VFS MODE (via require("fs")) ===\n')
console.log('--- fs.promises.readdir("/data") ---')
for (const e of (await fs.promises.readdir('/data')).sort()) console.log(` ${e}`)
console.log('\n--- fs.promises.readFile("/data/example.json", "utf-8") ---')
const json = await fs.promises.readFile('/data/example.json', 'utf-8')
console.log(json.trim())
console.log('\n--- fs.promises.stat() / exists() ---')
console.log(` example.json: ${String(await exists('/data/example.json'))}`)
console.log(` nope.txt: ${String(await exists('/data/nope.txt'))}`)
console.log('\n--- isDir("/data") ---')
console.log(` /data: ${String(await isDir('/data'))}`)
const total = ws.records.reduce((acc, r) => acc + r.bytes, 0)
console.log(`\nStats: ${String(ws.records.length)} ops, ${String(total)} bytes transferred`)
await ws.close()
rmSync(tmp, { recursive: true, force: true })
}
main().catch((err: unknown) => {
console.error(err)
process.exit(1)
})
examples/typescript/python/python_basic.ts → examples/typescript/pyodide/basic.ts
View File
examples/typescript/python/python_env.ts → examples/typescript/pyodide/env.ts
View File
examples/typescript/python/python_heredoc.ts → examples/typescript/pyodide/heredoc.ts
View File
examples/typescript/python/python_ram.ts → examples/typescript/pyodide/ram.ts
View File
examples/typescript/python/python_script.ts → examples/typescript/pyodide/script.ts
View File
examples/typescript/python/vfs.ts → examples/typescript/pyodide/vfs.ts
View File
examples/typescript/redis/redis_index.ts
+71
View File
@@ -0,0 +1,71 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
IndexEntry,
RAMResource,
RedisIndexCacheStore,
Workspace,
type RedisIndexConfig,
} from '@struktoai/mirage-node'
const REDIS_URL = process.env.REDIS_URL ?? 'redis://localhost:6379/0'
function file(name: string): IndexEntry {
return new IndexEntry({ id: name, name, resourceType: 'file' })
}
async function main(): Promise<void> {
// A workspace-level `index` config points every mounted resource's index
// cache at the same Redis instance. Two separate Mirage processes that
// share a keyPrefix then share one index -- the building block for running
// the same mounts locally and in a remote sandbox.
const keyPrefix = `mirage:example:idx:${String(Date.now())}:`
const indexConfig: RedisIndexConfig = { type: 'redis', url: REDIS_URL, keyPrefix }
// Workspace A: a RAM mount whose INDEX is backed by Redis (not RAM).
const ramA = new RAMResource()
const wsA = new Workspace({ '/data': ramA }, { index: indexConfig })
console.log(`index store A is redis-backed: ${ramA.index instanceof RedisIndexCacheStore}`)
// Populate the shared Redis index through workspace A.
await ramA.index.put('/data/hello.txt', file('hello.txt'))
await ramA.index.setDir('/data', [
['hello.txt', file('hello.txt')],
['notes.md', file('notes.md')],
])
// Workspace B: a *separate* resource pointed at the *same* Redis index
// (same keyPrefix). It sees what A cached without re-listing anything.
const ramB = new RAMResource()
const wsB = new Workspace({ '/data': ramB }, { index: indexConfig })
const entry = await ramB.index.get('/data/hello.txt')
console.log(`shared index entry: ${entry.entry?.name ?? '(none)'}`)
const listing = await ramB.index.listDir('/data')
console.log(`shared index listing: ${(listing.entries ?? []).join(', ')}`)
await ramA.index.clear()
if (ramA.index instanceof RedisIndexCacheStore) await ramA.index.close()
if (ramB.index instanceof RedisIndexCacheStore) await ramB.index.close()
await wsA.close()
await wsB.close()
console.log('wiped test keys from Redis')
}
main().catch((err: unknown) => {
console.error(err)
process.exit(1)
})
examples/typescript/redis/redis_vfs.ts
+83
View File
@@ -0,0 +1,83 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { createRequire } from 'node:module'
import { MountMode, RedisResource, Workspace, patchNodeFs } from '@struktoai/mirage-node'
const require = createRequire(import.meta.url)
const fs = require('fs') as typeof import('fs')
const REDIS_URL = process.env.REDIS_URL ?? 'redis://localhost:6379/0'
async function exists(p: string): Promise<boolean> {
try {
await fs.promises.stat(p)
return true
} catch {
return false
}
}
async function isDir(p: string): Promise<boolean> {
try {
return (await fs.promises.stat(p)).isDirectory()
} catch {
return false
}
}
async function main(): Promise<void> {
const seedWs = new Workspace(
{ '/data': new RedisResource({ url: REDIS_URL }) },
{ mode: MountMode.WRITE },
)
await seedWs.execute('echo "hello world" | tee /data/hello.txt')
await seedWs.execute('mkdir /data/sub')
await seedWs.execute('echo "nested" | tee /data/sub/nested.txt')
await seedWs.close()
const ws = new Workspace(
{ '/data': new RedisResource({ url: REDIS_URL }) },
{ mode: MountMode.WRITE },
)
patchNodeFs(ws)
console.log('=== VFS MODE (via require("fs")) ===\n')
console.log('--- fs.promises.readdir("/data") ---')
for (const e of (await fs.promises.readdir('/data')).sort()) console.log(` ${e}`)
console.log('\n--- fs.promises.readFile("/data/hello.txt", "utf-8") ---')
console.log(` ${(await fs.promises.readFile('/data/hello.txt', 'utf-8')).trim()}`)
console.log('\n--- fs.promises.stat() / exists() ---')
console.log(` hello.txt: ${String(await exists('/data/hello.txt'))}`)
console.log(` nope.txt: ${String(await exists('/data/nope.txt'))}`)
console.log('\n--- isDir("/data/sub") ---')
console.log(` /data/sub: ${String(await isDir('/data/sub'))}`)
console.log('\n--- fs.promises.readdir("/data/sub") ---')
for (const e of (await fs.promises.readdir('/data/sub')).sort()) console.log(` ${e}`)
const total = ws.records.reduce((acc, r) => acc + r.bytes, 0)
console.log(`\nStats: ${String(ws.records.length)} ops, ${String(total)} bytes transferred`)
await ws.close()
}
main().catch((err: unknown) => {
console.error(err)
process.exit(1)
})
integ/check_lines.sh
Executable
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
# Assert every expected line in a truth file appears in stdin.
#
# Usage: <command> 2>&1 | bash integ/check_lines.sh integ/truth_x.txt
#
# Truth lines are matched as fixed substrings (grep -aF), so the captured
# output may contain extra/volatile lines (snapshot temp paths, stat mtimes,
# aggregate counts) or even binary bytes without breaking the check. Blank
# lines and lines starting with # in the truth file are ignored.
set -euo pipefail
truth="$1"
tmp="$(mktemp)"
trap 'rm -f "$tmp"' EXIT
cat >"$tmp"
rc=0
matched=0
while IFS= read -r line || [ -n "$line" ]; do
[ -z "$line" ] && continue
case "$line" in \#*) continue ;; esac
if grep -aqF -- "$line" "$tmp"; then
matched=$((matched + 1))
else
echo "MISSING: $line" >&2
rc=1
fi
done <"$truth"
bytes="$(wc -c <"$tmp" | tr -d ' ')"
if [ "$rc" -ne 0 ]; then
echo "FAIL: $truth not satisfied (${bytes} bytes captured)" >&2
else
echo "OK: $truth (${matched} lines matched, ${bytes} bytes captured)"
fi
exit "$rc"
integ/truth/python/custom_command.txt
+2
View File
@@ -0,0 +1,2 @@
hello from RAMAccessor: /ram/note.txt
hello from DiskAccessor: /disk/note.txt
integ/truth/python/disk.txt
+2
View File
@@ -0,0 +1,2 @@
Strukto
example.json
integ/truth/python/disk_vfs.txt
+1
View File
@@ -0,0 +1 @@
example.json
integ/truth/python/ram.txt
+4
View File
@@ -0,0 +1,4 @@
hello world
hello.txt
"alice"
goodbye world
integ/truth/python/ram_python.txt
+4
View File
@@ -0,0 +1,4 @@
argv after -c: ['alpha', 'beta']
argv after script: ['one', 'two']
hello from vfs
hello_mirage
integ/truth/python/ram_vfs.txt
+3
View File
@@ -0,0 +1,3 @@
hello world
hello.txt: True
nope.txt: False
integ/truth/python/redis.txt
+15
View File
@@ -0,0 +1,15 @@
=== ls /data/ ===
hello world
revenue,100\nexpense,80
name=hello.txt size=12
"alice"
└── user.json
/data/hello.txt
HELLO WORLD
goodbye world
dlrow olleh
6f5902ac237024bdd0c176cb93063dc4
aGVsbG8gd29ybGQK
Precision.EXACT
deepcopy raises:
copy() mounts:
integ/truth/python/redis_cache.txt
+4
View File
@@ -0,0 +1,4 @@
=== RedisFileCacheStore: FileCache backed by Redis ===
cache.get: hello from redis cache
cache2.get: hello from redis cache
wiped cache keys from Redis
integ/truth/python/redis_index.txt
+4
View File
@@ -0,0 +1,4 @@
index store A is redis-backed: True
shared index entry: hello.txt
shared index listing: /data/hello.txt, /data/notes.md
wiped test keys from Redis
integ/truth/python/redis_vfs.txt
+6
View File
@@ -0,0 +1,6 @@
=== VFS MODE ===
hello world
hello.txt: True
nope.txt: False
/data/sub: True
nested.txt
integ/truth/typescript/custom_command.txt
+2
View File
@@ -0,0 +1,2 @@
hello from RAMAccessor: /ram/note.txt
hello from DiskAccessor: /disk/note.txt
integ/truth/typescript/disk.txt
+2
View File
@@ -0,0 +1,2 @@
Strukto
example.json
integ/truth/typescript/disk_vfs.txt
+6
View File
@@ -0,0 +1,6 @@
VFS MODE (via require
example.json
"company": "Strukto"
example.json: true
nope.txt: false
/data: true
integ/truth/typescript/pyodide_basic.txt
+3
View File
@@ -0,0 +1,3 @@
['alpha', 'beta']
exit: 3 (expect 3)
RuntimeError: boom
integ/truth/typescript/pyodide_env.txt
+3
View File
@@ -0,0 +1,3 @@
stdout: bar
wsA: alice
wsB: bob
integ/truth/typescript/pyodide_heredoc.txt
+5
View File
@@ -0,0 +1,5 @@
stdout: $X
stdout: shellval
item-0
keep 1
hello, alice!
integ/truth/typescript/pyodide_ram.txt
+5
View File
@@ -0,0 +1,5 @@
hello, world!
sum(1..10): 55
args: ['alice', 'bob']
alice: 2
python3: /ram/missing.py: No such file
integ/truth/typescript/pyodide_script.txt
+2
View File
@@ -0,0 +1,2 @@
count: 4
alice -> login
integ/truth/typescript/pyodide_vfs.txt
+4
View File
@@ -0,0 +1,4 @@
host sees: written from python
listdir: ['note.md']
read: lazy demo
PNG magic: 89 50 4e 47
integ/truth/typescript/ram.txt
+4
View File
@@ -0,0 +1,4 @@
hello world
hello.txt
"alice"
goodbye world
integ/truth/typescript/ram_vfs.txt
+3
View File
@@ -0,0 +1,3 @@
hello world
hello.txt: true
nope.txt: false
integ/truth/typescript/redis.txt
+15
View File
@@ -0,0 +1,15 @@
=== ls /data/ ===
hello world
revenue,100\nexpense,80
name=hello.txt size=12 modified=None type=text
"alice"
└── user.json
/data/hello.txt
HELLO WORLD
goodbye world
dlrow olleh
6f5902ac237024bdd0c176cb93063dc4
aGVsbG8gd29ybGQK
= 41 bytes (2 reads)
copy() sees original's writes: hello world
wiped test keys from Redis
integ/truth/typescript/redis_cache.txt
+10
View File
@@ -0,0 +1,10 @@
=== RedisFileCacheStore: FileCache backed by Redis ===
get: hello from redis cache
matches same fp: true
matches other fp: false
add existing: false
add new: true
immediate: true
after 1.1s: false
cache2.get: hello from redis cache
wiped cache keys from Redis
integ/truth/typescript/redis_index.txt
+4
View File
@@ -0,0 +1,4 @@
index store A is redis-backed: true
shared index entry: hello.txt
shared index listing: /data/hello.txt, /data/notes.md
wiped test keys from Redis
integ/truth/typescript/redis_vfs.txt
+6
View File
@@ -0,0 +1,6 @@
VFS MODE (via require
hello world
hello.txt: true
nope.txt: false
/data/sub: true
nested.txt
python/mirage/cache/index/config.py
Vendored
+1 -1
View File
@@ -58,4 +58,4 @@ class IndexConfig(BaseModel):
class RedisIndexConfig(IndexConfig):
type: IndexType = IndexType.REDIS
url: str = "redis://localhost:6379/0"
key_prefix: str = ""
key_prefix: str = "mirage:index:"
python/mirage/config.py
+36
View File
@@ -21,6 +21,7 @@ import yaml
from pydantic import BaseModel, ConfigDict, Field, field_validator
from mirage.cache.file.config import CacheConfig, RedisCacheConfig
from mirage.cache.index.config import IndexConfig, RedisIndexConfig
from mirage.resource.registry import build_resource
from mirage.types import ConsistencyPolicy, MountMode
@@ -124,6 +125,28 @@ CacheBlock = Annotated[
]
class RamIndexBlock(BaseModel):
model_config = ConfigDict(extra="forbid")
type: Literal["ram"] = "ram"
ttl: float = 600
class RedisIndexBlock(BaseModel):
model_config = ConfigDict(extra="forbid")
type: Literal["redis"]
ttl: float = 600
url: str = "redis://localhost:6379/0"
key_prefix: str = "mirage:index:"
IndexBlock = Annotated[
RamIndexBlock | RedisIndexBlock,
Field(discriminator="type"),
]
class MountBlock(BaseModel):
model_config = ConfigDict(extra="forbid")
@@ -152,6 +175,7 @@ class WorkspaceConfig(BaseModel):
history: int | None = 100
history_path: str | None = None
cache: CacheBlock | None = None
index: IndexBlock | None = None
@field_validator("mode", mode="before")
@classmethod
@@ -189,6 +213,8 @@ class WorkspaceConfig(BaseModel):
}
if self.cache is not None:
kwargs["cache"] = _build_cache_config(self.cache)
if self.index is not None:
kwargs["index"] = _build_index_config(self.index)
return kwargs
@@ -206,6 +232,16 @@ def _build_cache_config(block: RamCacheBlock | RedisCacheBlock) -> CacheConfig:
)
def _build_index_config(block: RamIndexBlock | RedisIndexBlock) -> IndexConfig:
if isinstance(block, RedisIndexBlock):
return RedisIndexConfig(
ttl=block.ttl,
url=block.url,
key_prefix=block.key_prefix,
)
return IndexConfig(ttl=block.ttl)
def load_config(source: str | Path | dict,
env: dict[str, str] | None = None) -> WorkspaceConfig:
"""Load a workspace config from a YAML / JSON file or a raw dict.
python/mirage/resource/base.py
+5 -2
View File
@@ -34,7 +34,7 @@ class BaseResource:
PROMPT: str = ""
WRITE_PROMPT: str = ""
_index_ttl: float = 600
index_ttl: float = 600
# Whether this resource carries enough version information for
# snapshot+replay drift detection. When True, the resource's stat()
@@ -53,7 +53,10 @@ class BaseResource:
super().__init__(**kwargs)
self._commands: list = []
self._ops_list: list = []
cfg = index or IndexConfig(ttl=self._index_ttl)
self.set_index(index)
def set_index(self, config: IndexConfig | None = None) -> None:
cfg = config or IndexConfig(ttl=self.index_ttl)
if isinstance(cfg, RedisIndexConfig):
if RedisIndexCacheStore is None:
raise ImportError(
python/mirage/resource/disk/disk.py
+1 -1
View File
@@ -58,7 +58,7 @@ _DISK_OPS = {
class DiskResource(BaseResource):
name: str = ResourceName.DISK
_index_ttl: float = 60
index_ttl: float = 60
_ops: dict = _DISK_OPS
PROMPT: str = PROMPT
python/mirage/resource/ram/ram.py
+1 -1
View File
@@ -65,7 +65,7 @@ _RAM_OPS = {
class RAMResource(BaseResource):
name: str = ResourceName.RAM
_index_ttl: float = 0
index_ttl: float = 0
_ops: dict = _RAM_OPS
PROMPT: str = PROMPT
python/mirage/resource/redis/redis.py
+1 -1
View File
@@ -71,7 +71,7 @@ _REDIS_OPS = {
class RedisResource(BaseResource):
name: str = ResourceName.REDIS
_index_ttl: float = 0
index_ttl: float = 0
_ops: dict = _REDIS_OPS
PROMPT: str = PROMPT
python/mirage/workspace/workspace.py
+4
View File
@@ -23,6 +23,7 @@ from typing import Any
from mirage.cache.file import io as cache_io
from mirage.cache.file.config import CacheConfig, RedisCacheConfig
from mirage.cache.file.ram import RAMFileCacheStore
from mirage.cache.index import IndexConfig
from mirage.commands.builtin.general import HISTORY_COMMANDS
try:
@@ -85,6 +86,7 @@ class Workspace:
resources: dict[str, BaseResource | tuple],
cache_limit: str | int = "512MB",
cache: CacheConfig | None = None,
index: IndexConfig | None = None,
mode: MountMode = MountMode.READ,
consistency: ConsistencyPolicy = ConsistencyPolicy.LAZY,
history: int | None = 100,
@@ -136,6 +138,8 @@ class Workspace:
else:
prov = value
mount_mode = mode
if index is not None:
prov.set_index(index)
self._registry.mount(prefix, prov, mount_mode)
self._fuse = FuseManager()
python/tests/resource/test_base.py
+35
View File
@@ -0,0 +1,35 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage.cache.index import (RAMIndexCacheStore, RedisIndexCacheStore,
RedisIndexConfig)
from mirage.resource.ram import RAMResource
def test_default_index_is_ram():
r = RAMResource()
assert isinstance(r.index, RAMIndexCacheStore)
def test_set_index_redis():
r = RAMResource()
r.set_index(RedisIndexConfig(url="redis://localhost:6379/0"))
assert isinstance(r.index, RedisIndexCacheStore)
def test_set_index_none_resets_to_ram():
r = RAMResource()
r.set_index(RedisIndexConfig(url="redis://localhost:6379/0"))
r.set_index(None)
assert isinstance(r.index, RAMIndexCacheStore)
python/tests/workspace/test_index_config.py
+46
View File
@@ -0,0 +1,46 @@
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
from mirage import Workspace
from mirage.cache.index import (RAMIndexCacheStore, RedisIndexCacheStore,
RedisIndexConfig)
from mirage.config import MountBlock, RedisIndexBlock, WorkspaceConfig
from mirage.resource.ram import RAMResource
def test_redis_index_config_default_key_prefix():
assert RedisIndexConfig().key_prefix == "mirage:index:"
def test_workspace_index_param_applies_to_mounts():
r = RAMResource()
Workspace({"/m": r},
index=RedisIndexConfig(url="redis://localhost:6379/0"))
assert isinstance(r.index, RedisIndexCacheStore)
def test_workspace_default_index_is_ram():
r = RAMResource()
Workspace({"/m": r})
assert isinstance(r.index, RAMIndexCacheStore)
def test_config_index_redis_block_builds_redis_config():
cfg = WorkspaceConfig(
mounts={"/m": MountBlock(resource="ram")},
index=RedisIndexBlock(type="redis"),
)
kwargs = cfg.to_workspace_kwargs()
assert isinstance(kwargs["index"], RedisIndexConfig)
assert kwargs["index"].key_prefix == "mirage:index:"
python/tests/workspace/test_index_integration.py
+4 -4
View File
@@ -44,7 +44,7 @@ def _stdout(io):
def test_ram_resource_has_index():
p = RAMResource()
assert p.index is not None
assert p._index_ttl == 0
assert p.index_ttl == 0
def test_s3_resource_has_index():
@@ -53,7 +53,7 @@ def test_s3_resource_has_index():
region_name="us-east-1").create_bucket(Bucket="test-idx")
p = S3Resource(S3Config(bucket="test-idx", region="us-east-1"))
assert p.index is not None
assert p._index_ttl == 600
assert p.index_ttl == 600
# ── RAM index integration ─────────────────────
@@ -106,7 +106,7 @@ def test_s3_resource_index_ttl():
region_name="us-east-1").create_bucket(Bucket="test-ttl")
prov = S3Resource(S3Config(bucket="test-ttl", region="us-east-1"))
assert prov.index is not None
assert prov._index_ttl == 600
assert prov.index_ttl == 600
def test_s3_index_can_store_entries():
@@ -152,7 +152,7 @@ def test_index_per_resource():
def test_ram_index_ttl_zero():
p = RAMResource()
assert p._index_ttl == 0
assert p.index_ttl == 0
def test_index_expired_refetches():
python/tests/workspace/test_index_writethrough.py
+1 -1
View File
@@ -21,7 +21,7 @@ from mirage.workspace import Workspace
class _FakeRemote(RAMResource):
is_remote = True
_index_ttl = 600
index_ttl = 600
def _seed_remote() -> _FakeRemote:
typescript/packages/core/package.json
+6 -1
View File
@@ -52,6 +52,7 @@
"@modelcontextprotocol/sdk": "^1.0.0",
"hyparquet-writer": "^0.13.0",
"pyodide": "^0.29.3",
"redis": "^5.0.0",
"tree-sitter-bash": "^0.25.1",
"tsup": "^8.5.0",
"typescript": "^6.0.0",
@@ -68,11 +69,15 @@
"web-tree-sitter": "^0.26.8"
},
"peerDependencies": {
"pyodide": "^0.29.0"
"pyodide": "^0.29.0",
"redis": "^5.0.0"
},
"peerDependenciesMeta": {
"pyodide": {
"optional": true
},
"redis": {
"optional": true
}
}
}
typescript/packages/core/src/cache/index/index.ts
Vendored
+1
View File
@@ -23,4 +23,5 @@ export {
type RedisIndexConfig,
} from './config.ts'
export { RAMIndexCacheStore } from './ram.ts'
export { RedisIndexCacheStore, type RedisClientLike, type RedisIndexCacheOptions } from './redis.ts'
export { IndexCacheStore } from './store.ts'
typescript/packages/node/src/cache/redis/index_cache.test.ts → typescript/packages/core/src/cache/index/redis.test.ts
Vendored
+10 -2
View File
@@ -12,9 +12,17 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { IndexEntry, LookupStatus } from '@struktoai/mirage-core'
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
import { RedisIndexCacheStore } from './index_cache.ts'
import { IndexEntry, LookupStatus } from './config.ts'
import { RedisIndexCacheStore } from './redis.ts'
describe('RedisIndexCacheStore default keyPrefix', () => {
it('namespaces keys under mirage:index: by default', () => {
const store = new RedisIndexCacheStore()
const prefix = (store as unknown as { entryPrefix: string }).entryPrefix
expect(prefix).toBe('mirage:index:mirage:idx:entry:')
})
})
const REDIS_URL = process.env.REDIS_URL
const skip = REDIS_URL === undefined
typescript/packages/node/src/cache/redis/index_cache.ts → typescript/packages/core/src/cache/index/redis.ts
Vendored
+40 -18
View File
@@ -12,41 +12,63 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { loadOptionalPeer } from '../../utils/optional_peer.ts'
import {
IndexCacheStore,
IndexEntry,
LookupStatus,
type IndexConfig,
type ListResult,
type LookupResult,
LookupStatus,
} from '@struktoai/mirage-core'
import type { RedisClientType } from 'redis'
import { loadOptionalPeer } from '../../optional_peer.ts'
} from './config.ts'
import { IndexCacheStore } from './store.ts'
const ENTRY_PREFIX = 'mirage:idx:entry:'
const CHILDREN_PREFIX = 'mirage:idx:children:'
const DEFAULT_KEY_PREFIX = 'mirage:index:'
interface RedisPipeline {
set: (key: string, value: string) => RedisPipeline
del: (key: string) => RedisPipeline
rPush: (key: string, values: string[]) => RedisPipeline
expire: (key: string, seconds: number) => RedisPipeline
exec: () => Promise<unknown>
}
export interface RedisClientLike {
connect: () => Promise<unknown>
get: (key: string) => Promise<string | null>
set: (key: string, value: string) => Promise<unknown>
exists: (key: string) => Promise<number>
ttl: (key: string) => Promise<number>
lRange: (key: string, start: number, stop: number) => Promise<string[]>
del: (key: string | string[]) => Promise<unknown>
multi: () => RedisPipeline
scanIterator: (options: { MATCH: string }) => AsyncIterable<string | string[]>
isOpen: boolean
quit: () => Promise<unknown>
}
export interface RedisIndexCacheOptions {
ttl?: number
url?: string
client?: RedisClientType
client?: RedisClientLike
keyPrefix?: string
}
export class RedisIndexCacheStore extends IndexCacheStore {
private readonly ttl: number
private readonly url: string
private readonly providedClient: RedisClientType | null
private readonly providedClient: RedisClientLike | null
private readonly entryPrefix: string
private readonly childrenPrefix: string
private clientPromise: Promise<RedisClientType> | null = null
private clientPromise: Promise<RedisClientLike> | null = null
constructor(options: RedisIndexCacheOptions = {}) {
super()
this.ttl = options.ttl ?? 600
this.url = options.url ?? 'redis://localhost:6379/0'
this.providedClient = options.client ?? null
const prefix = options.keyPrefix ?? ''
const prefix = options.keyPrefix ?? DEFAULT_KEY_PREFIX
this.entryPrefix = `${prefix}${ENTRY_PREFIX}`
this.childrenPrefix = `${prefix}${CHILDREN_PREFIX}`
}
@@ -66,20 +88,20 @@ export class RedisIndexCacheStore extends IndexCacheStore {
return `${this.childrenPrefix}${path}`
}
private client(): Promise<RedisClientType> {
private client(): Promise<RedisClientLike> {
if (this.providedClient !== null) return Promise.resolve(this.providedClient)
this.clientPromise ??= (async () => {
const mod = await loadOptionalPeer(
() =>
import('redis') as unknown as Promise<{
createClient: (o: { url: string }) => RedisClientType
}>,
{ feature: 'RedisIndexCacheStore', packageName: 'redis' },
)
const spec = 'redis'
const mod = (await loadOptionalPeer(() => import(/* @vite-ignore */ spec), {
feature: 'RedisIndexCacheStore',
packageName: 'redis',
})) as {
createClient: (o: { url: string; socket?: unknown }) => RedisClientLike
}
const c = mod.createClient({
url: this.url,
socket: { reconnectStrategy: false },
} as Parameters<typeof mod.createClient>[0])
})
await c.connect()
return c
})()
typescript/packages/core/src/index.ts
+7 -1
View File
@@ -33,7 +33,7 @@ export {
type FingerprintEntry,
liveOnlyMountPrefixes,
} from './workspace/snapshot/drift.ts'
export { type FindOptions, type Resource, throwUnsupported } from './resource/base.ts'
export { BaseResource, type FindOptions, type Resource, throwUnsupported } from './resource/base.ts'
export { RAMResource } from './resource/ram/ram.ts'
export { RAMStore } from './resource/ram/store.ts'
export { DevResource } from './resource/dev/dev.ts'
@@ -307,9 +307,15 @@ export {
type IndexConfig,
type ListResult,
type LookupResult,
type RedisIndexConfig,
} from './cache/index/config.ts'
export { IndexCacheStore } from './cache/index/store.ts'
export { RAMIndexCacheStore } from './cache/index/ram.ts'
export {
RedisIndexCacheStore,
type RedisClientLike,
type RedisIndexCacheOptions,
} from './cache/index/redis.ts'
export { ExecutionHistory, type ExecutionHistoryOptions } from './workspace/history.ts'
export {
ExecutionNode,
typescript/packages/core/src/resource/base.test.ts
+49
View File
@@ -0,0 +1,49 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it } from 'vitest'
import { IndexType, type RedisIndexConfig } from '../cache/index/config.ts'
import { RAMIndexCacheStore } from '../cache/index/ram.ts'
import { RedisIndexCacheStore } from '../cache/index/redis.ts'
import { BaseResource } from './base.ts'
class Probe extends BaseResource {
readonly indexTtl: number = 123
}
describe('BaseResource index', () => {
it('defaults to a RAM index using the resource indexTtl', () => {
const r = new Probe()
expect(r.index).toBeInstanceOf(RAMIndexCacheStore)
expect((r.index as unknown as { ttl: number }).ttl).toBe(123)
})
it('setIndex with a ram config rebuilds RAM with the config ttl', () => {
const r = new Probe()
r.setIndex({ type: IndexType.RAM, ttl: 5 })
expect(r.index).toBeInstanceOf(RAMIndexCacheStore)
expect((r.index as unknown as { ttl: number }).ttl).toBe(5)
})
it('setIndex with a redis config swaps in a RedisIndexCacheStore', () => {
const r = new Probe()
const cfg: RedisIndexConfig = {
type: IndexType.REDIS,
url: 'redis://localhost:6379/0',
keyPrefix: 'p:',
}
r.setIndex(cfg)
expect(r.index).toBeInstanceOf(RedisIndexCacheStore)
})
})
typescript/packages/core/src/resource/base.ts
+36 -1
View File
@@ -13,7 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import type { Accessor } from '../accessor/base.ts'
import type { IndexCacheStore } from '../cache/index/index.ts'
import { IndexType, type IndexConfig, type RedisIndexConfig } from '../cache/index/config.ts'
import { RAMIndexCacheStore } from '../cache/index/ram.ts'
import { RedisIndexCacheStore } from '../cache/index/redis.ts'
import type { IndexCacheStore } from '../cache/index/store.ts'
import type { RegisteredCommand } from '../commands/config.ts'
import type { RegisteredOp } from '../ops/registry.ts'
import type { FileStat, PathSpec } from '../types.ts'
@@ -52,6 +55,7 @@ export interface Resource {
readonly index?: IndexCacheStore
readonly accessor?: Accessor
readonly opsMap?: Record<string, unknown>
setIndex?(config?: IndexConfig): void
open(): Promise<void>
close(): Promise<void>
ops?(): readonly RegisteredOp[]
@@ -80,3 +84,34 @@ export interface Resource {
export function throwUnsupported(op: string): never {
throw new Error(`resource has no ${op} support`)
}
export abstract class BaseResource {
readonly indexTtl: number = 600
protected _index?: IndexCacheStore
get index(): IndexCacheStore {
let store = this._index
if (store === undefined) {
store = this.makeIndex()
this._index = store
}
return store
}
setIndex(config?: IndexConfig): void {
this._index = this.makeIndex(config)
}
private makeIndex(config?: IndexConfig): IndexCacheStore {
if (config?.type === IndexType.REDIS) {
const redis = config as RedisIndexConfig
return new RedisIndexCacheStore({
ttl: redis.ttl ?? 600,
...(redis.url !== undefined ? { url: redis.url } : {}),
...(redis.keyPrefix !== undefined ? { keyPrefix: redis.keyPrefix } : {}),
})
}
const ttl = config === undefined ? this.indexTtl : (config.ttl ?? 600)
return new RAMIndexCacheStore({ ttl })
}
}
typescript/packages/core/src/resource/ram/ram.ts
+2 -4
View File
@@ -33,11 +33,10 @@ import { truncate as truncateCore } from '../../core/ram/truncate.ts'
import { unlink as unlinkCore } from '../../core/ram/unlink.ts'
import { writeBytes as writeCore } from '../../core/ram/write.ts'
import { RAMAccessor } from '../../accessor/ram.ts'
import { type IndexCacheStore, RAMIndexCacheStore } from '../../cache/index/index.ts'
import { RAM_OPS } from '../../ops/ram/index.ts'
import type { RegisteredOp } from '../../ops/registry.ts'
import { PathSpec, ResourceName, type FileStat } from '../../types.ts'
import type { FindOptions, Resource } from '../base.ts'
import { BaseResource, type FindOptions, type Resource } from '../base.ts'
import { RAM_PROMPT } from './prompt.ts'
import { RAMStore } from './store.ts'
@@ -50,13 +49,12 @@ export interface RAMResourceState {
modified: Record<string, string>
}
export class RAMResource implements Resource {
export class RAMResource extends BaseResource implements Resource {
readonly kind = ResourceName.RAM
readonly isRemote: boolean = false
readonly indexTtl: number = 0
readonly store = new RAMStore()
readonly accessor = new RAMAccessor(this.store)
readonly index: IndexCacheStore = new RAMIndexCacheStore({ ttl: this.indexTtl })
readonly prompt = RAM_PROMPT
readonly opsMap: Record<string, unknown> = {
read_bytes: readCore,
typescript/packages/core/src/workspace/index_option.test.ts
+36
View File
@@ -0,0 +1,36 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { describe, expect, it } from 'vitest'
import { IndexType } from '../cache/index/config.ts'
import { RAMIndexCacheStore } from '../cache/index/ram.ts'
import { RAMResource } from '../resource/ram/ram.ts'
import { Workspace } from './workspace.ts'
describe('Workspace index option', () => {
it('applies the workspace index config to mounted resources', async () => {
const ram = new RAMResource()
const ws = new Workspace({ '/data': ram }, { index: { type: IndexType.RAM, ttl: 5 } })
expect(ram.index).toBeInstanceOf(RAMIndexCacheStore)
expect((ram.index as unknown as { ttl: number }).ttl).toBe(5)
await ws.close()
})
it('keeps the resource default index when no workspace index is given', async () => {
const ram = new RAMResource()
const ws = new Workspace({ '/data': ram }, {})
expect((ram.index as unknown as { ttl: number }).ttl).toBe(0)
await ws.close()
})
})
typescript/packages/core/src/workspace/workspace.ts
+7
View File
@@ -16,6 +16,7 @@ import { NOOPAccessor } from '../accessor/base.ts'
import { applyIo } from '../cache/file/io.ts'
import { CacheEntry } from '../cache/file/entry.ts'
import type { FileCache } from '../cache/file/mixin.ts'
import type { IndexConfig } from '../cache/index/config.ts'
import { RAMFileCacheStore } from '../cache/file/ram.ts'
import type { ByteSource } from '../io/types.ts'
import { IOResult, materialize } from '../io/types.ts'
@@ -148,6 +149,7 @@ export interface WorkspaceOptions {
sessionId?: string
cacheLimit?: string | number
cache?: FileCache & Resource
index?: IndexConfig
observerResource?: Resource
observerPrefix?: string
python?: {
@@ -268,6 +270,11 @@ export class Workspace {
...(options.modeOverrides ?? {}),
[observerPrefix]: MountMode.READ,
})
if (options.index !== undefined) {
for (const resource of Object.values(resources)) {
resource.setIndex?.(options.index)
}
}
this.sessionManager = new SessionManager(options.sessionId ?? 'default')
this.opsRegistry = options.ops ?? new OpsRegistry()
this.shellParser = options.shellParser ?? null
typescript/packages/node/src/index.ts
-1
View File
@@ -38,7 +38,6 @@ export {
type RedisResourceLike,
} from './commands/builtin/redis/provision.ts'
export { RedisFileCacheStore, type RedisFileCacheOptions } from './cache/redis/file.ts'
export { RedisIndexCacheStore, type RedisIndexCacheOptions } from './cache/redis/index_cache.ts'
export { FuseManager } from './workspace/fuse.ts'
export { MirageFS, type MirageFSOptions, type FuseAttr } from './fuse/fs.ts'
export {
typescript/packages/node/src/resource/box/box.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
BOX_COMMANDS,
BOX_PROMPT,
BOX_VFS_OPS,
BoxAccessor,
BoxTokenManager,
type FileStat,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,16 +39,16 @@ export interface BoxResourceState {
config: BoxConfigRedacted
}
export class BoxResource implements Resource {
export class BoxResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.BOX
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly prompt: string = BOX_PROMPT
readonly config: BoxConfig
readonly accessor: BoxAccessor
readonly index: IndexCacheStore
constructor(config: BoxConfig) {
super()
this.config = config
const tm = new BoxTokenManager({
...(config.clientId !== undefined ? { clientId: config.clientId } : {}),
@@ -62,7 +61,6 @@ export class BoxResource implements Resource {
: {}),
})
this.accessor = new BoxAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/discord/discord.ts
+3 -5
View File
@@ -13,6 +13,7 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
DISCORD_API,
DISCORD_COMMANDS,
DISCORD_PROMPT,
@@ -21,9 +22,7 @@ import {
DiscordAccessor,
type FileStat,
HttpDiscordTransport,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -54,7 +53,7 @@ export interface DiscordResourceState {
config: DiscordConfigRedacted
}
export class DiscordResource implements Resource {
export class DiscordResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.DISCORD
readonly isRemote: boolean = true
readonly indexTtl: number = 600
@@ -62,12 +61,11 @@ export class DiscordResource implements Resource {
readonly writePrompt: string = DISCORD_WRITE_PROMPT
readonly config: DiscordConfig
readonly accessor: DiscordAccessor
readonly index: IndexCacheStore
constructor(config: DiscordConfig) {
super()
this.config = config
this.accessor = new DiscordAccessor(new NodeDiscordTransport(config.token))
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/disk/disk.ts
+3 -5
View File
@@ -15,11 +15,10 @@
import { mkdir, readdir, readFile, writeFile } from 'node:fs/promises'
import path from 'node:path'
import {
BaseResource,
type FileStat,
type FindOptions,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -71,14 +70,13 @@ async function walkFiles(root: string, current: string, out: string[]): Promise<
}
}
export class DiskResource implements Resource {
export class DiskResource extends BaseResource implements Resource {
readonly kind = ResourceName.DISK
readonly isRemote: boolean = false
readonly indexTtl: number = 60
readonly prompt = DISK_PROMPT
readonly root: string
readonly accessor: DiskAccessor
readonly index: IndexCacheStore
readonly opsMap: Record<string, unknown> = {
read_bytes: readCoreFn,
write: writeCore,
@@ -101,9 +99,9 @@ export class DiskResource implements Resource {
}
constructor(options: DiskResourceOptions) {
super()
this.root = path.resolve(options.root)
this.accessor = new DiskAccessor(this.root)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
async open(): Promise<void> {
typescript/packages/node/src/resource/dropbox/dropbox.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
DROPBOX_COMMANDS,
DROPBOX_PROMPT,
DROPBOX_VFS_OPS,
DropboxAccessor,
DropboxTokenManager,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,16 +39,16 @@ export interface DropboxResourceState {
config: DropboxConfigRedacted
}
export class DropboxResource implements Resource {
export class DropboxResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.DROPBOX
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly prompt: string = DROPBOX_PROMPT
readonly config: DropboxConfig
readonly accessor: DropboxAccessor
readonly index: IndexCacheStore
constructor(config: DropboxConfig) {
super()
this.config = config
const tm = new DropboxTokenManager({
clientId: config.clientId,
@@ -58,7 +57,6 @@ export class DropboxResource implements Resource {
...(config.refreshFn !== undefined ? { refreshFn: config.refreshFn } : {}),
})
this.accessor = new DropboxAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/email/email.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
PathSpec,
RAMIndexCacheStore,
ResourceName,
type FileStat,
type IndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -39,19 +38,18 @@ export interface EmailResourceState {
config: EmailConfigRedacted
}
export class EmailResource implements Resource {
export class EmailResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.EMAIL
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly prompt: string = EMAIL_PROMPT
readonly config: EmailConfig
readonly accessor: EmailAccessor
readonly index: IndexCacheStore
constructor(config: EmailConfig) {
super()
this.config = config
this.accessor = new EmailAccessor(config)
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/gdocs/gdocs.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GDOCS_COMMANDS,
GDOCS_PROMPT,
GDOCS_VFS_OPS,
GDOCS_WRITE_PROMPT,
GDocsAccessor,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -41,7 +40,7 @@ export interface GDocsResourceState {
config: GDocsConfigRedacted
}
export class GDocsResource implements Resource {
export class GDocsResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GDOCS
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
@@ -49,9 +48,9 @@ export class GDocsResource implements Resource {
readonly writePrompt: string = GDOCS_WRITE_PROMPT
readonly config: GDocsConfig
readonly accessor: GDocsAccessor
readonly index: IndexCacheStore
constructor(config: GDocsConfig) {
super()
this.config = config
const tm = new TokenManager({
clientId: config.clientId,
@@ -59,7 +58,6 @@ export class GDocsResource implements Resource {
refreshToken: config.refreshToken,
})
this.accessor = new GDocsAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/gdrive/gdrive.ts
+3 -5
View File
@@ -13,14 +13,13 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GDRIVE_COMMANDS,
GDRIVE_PROMPT,
GDRIVE_VFS_OPS,
GDriveAccessor,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,16 +39,16 @@ export interface GDriveResourceState {
config: GDriveConfigRedacted
}
export class GDriveResource implements Resource {
export class GDriveResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GDRIVE
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly prompt: string = GDRIVE_PROMPT
readonly config: GDriveConfig
readonly accessor: GDriveAccessor
readonly index: IndexCacheStore
constructor(config: GDriveConfig) {
super()
this.config = config
const tm = new TokenManager({
clientId: config.clientId,
@@ -57,7 +56,6 @@ export class GDriveResource implements Resource {
refreshToken: config.refreshToken,
})
this.accessor = new GDriveAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/github/github.ts
+4 -3
View File
@@ -13,6 +13,7 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
fetchGitHubRepoInfo,
fetchGitHubTree,
type FileStat,
@@ -48,18 +49,18 @@ export interface GitHubResourceState {
truncated: boolean
}
export class GitHubResource implements Resource {
export class GitHubResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GITHUB
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly config: GitHubConfig
readonly accessor: GitHubAccessor
readonly index: IndexCacheStore
private constructor(config: GitHubConfig, accessor: GitHubAccessor, index: IndexCacheStore) {
super()
this.config = config
this.accessor = accessor
this.index = index
this._index = index
}
static async create(config: GitHubConfig): Promise<GitHubResource> {
typescript/packages/node/src/resource/github_ci/github_ci.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GITHUB_CI_COMMANDS,
GITHUB_CI_PROMPT,
GITHUB_CI_VFS_OPS,
GitHubCIAccessor,
HttpCITransport,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,16 +39,16 @@ export interface GitHubCIResourceState {
config: GitHubCIConfigRedacted
}
export class GitHubCIResource implements Resource {
export class GitHubCIResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GITHUB_CI
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
readonly prompt: string = GITHUB_CI_PROMPT
readonly config: GitHubCIConfig
readonly accessor: GitHubCIAccessor
readonly index: IndexCacheStore
constructor(config: GitHubCIConfig) {
super()
this.config = config
const transportOpts: { token: string; baseUrl?: string } = { token: config.token }
if (config.baseUrl !== undefined) transportOpts.baseUrl = config.baseUrl
@@ -60,7 +59,6 @@ export class GitHubCIResource implements Resource {
days: config.days ?? 30,
maxRuns: config.maxRuns ?? 300,
})
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/gmail/gmail.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GMAIL_COMMANDS,
GMAIL_PROMPT,
GMAIL_WRITE_PROMPT,
GMAIL_VFS_OPS,
GmailAccessor,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -41,7 +40,7 @@ export interface GmailResourceState {
config: GmailConfigRedacted
}
export class GmailResource implements Resource {
export class GmailResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GMAIL
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
@@ -49,9 +48,9 @@ export class GmailResource implements Resource {
readonly writePrompt: string = GMAIL_WRITE_PROMPT
readonly config: GmailConfig
readonly accessor: GmailAccessor
readonly index: IndexCacheStore
constructor(config: GmailConfig) {
super()
this.config = config
const tm = new TokenManager({
clientId: config.clientId,
@@ -59,7 +58,6 @@ export class GmailResource implements Resource {
refreshToken: config.refreshToken,
})
this.accessor = new GmailAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/gsheets/gsheets.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GSHEETS_COMMANDS,
GSHEETS_PROMPT,
GSHEETS_VFS_OPS,
GSHEETS_WRITE_PROMPT,
GSheetsAccessor,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -41,7 +40,7 @@ export interface GSheetsResourceState {
config: GSheetsConfigRedacted
}
export class GSheetsResource implements Resource {
export class GSheetsResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GSHEETS
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
@@ -49,9 +48,9 @@ export class GSheetsResource implements Resource {
readonly writePrompt: string = GSHEETS_WRITE_PROMPT
readonly config: GSheetsConfig
readonly accessor: GSheetsAccessor
readonly index: IndexCacheStore
constructor(config: GSheetsConfig) {
super()
this.config = config
const tm = new TokenManager({
clientId: config.clientId,
@@ -59,7 +58,6 @@ export class GSheetsResource implements Resource {
refreshToken: config.refreshToken,
})
this.accessor = new GSheetsAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/gslides/gslides.ts
+3 -5
View File
@@ -13,15 +13,14 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
GSLIDES_COMMANDS,
GSLIDES_PROMPT,
GSLIDES_VFS_OPS,
GSLIDES_WRITE_PROMPT,
GSlidesAccessor,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -41,7 +40,7 @@ export interface GSlidesResourceState {
config: GSlidesConfigRedacted
}
export class GSlidesResource implements Resource {
export class GSlidesResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.GSLIDES
readonly isRemote: boolean = true
readonly indexTtl: number = 86_400
@@ -49,9 +48,9 @@ export class GSlidesResource implements Resource {
readonly writePrompt: string = GSLIDES_WRITE_PROMPT
readonly config: GSlidesConfig
readonly accessor: GSlidesAccessor
readonly index: IndexCacheStore
constructor(config: GSlidesConfig) {
super()
this.config = config
const tm = new TokenManager({
clientId: config.clientId,
@@ -59,7 +58,6 @@ export class GSlidesResource implements Resource {
refreshToken: config.refreshToken,
})
this.accessor = new GSlidesAccessor({ tokenManager: tm })
this.index = new RAMIndexCacheStore({ ttl: 86_400 })
}
open(): Promise<void> {
typescript/packages/node/src/resource/langfuse/langfuse.ts
+3 -5
View File
@@ -13,9 +13,9 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpLangfuseTransport,
type IndexCacheStore,
LANGFUSE_COMMANDS,
LANGFUSE_PROMPT,
LANGFUSE_VFS_OPS,
@@ -24,7 +24,6 @@ import {
langfuseReaddir,
langfuseStat,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,16 +39,16 @@ export interface LangfuseResourceState {
config: LangfuseConfigRedacted
}
export class LangfuseResource implements Resource {
export class LangfuseResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.LANGFUSE
readonly isRemote: boolean = true
readonly indexTtl: number = 600
readonly prompt: string = LANGFUSE_PROMPT
readonly config: LangfuseConfig
readonly accessor: LangfuseAccessor
readonly index: IndexCacheStore
constructor(config: LangfuseConfig) {
super()
this.config = config
const transportOpts: { publicKey: string; secretKey: string; host?: string } = {
publicKey: config.publicKey,
@@ -71,7 +70,6 @@ export class LangfuseResource implements Resource {
accessorConfig.defaultFromTimestamp = config.defaultFromTimestamp
}
this.accessor = new LangfuseAccessor(new HttpLangfuseTransport(transportOpts), accessorConfig)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/linear/linear.ts
+3 -5
View File
@@ -13,9 +13,9 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpLinearTransport,
type IndexCacheStore,
LINEAR_COMMANDS,
LINEAR_PROMPT,
LINEAR_VFS_OPS,
@@ -26,7 +26,6 @@ import {
linearReaddir,
linearStat,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface LinearResourceState {
config: LinearConfigRedacted
}
export class LinearResource implements Resource {
export class LinearResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.LINEAR
readonly isRemote: boolean = true
readonly indexTtl: number = 600
@@ -50,14 +49,13 @@ export class LinearResource implements Resource {
readonly writePrompt: string = LINEAR_WRITE_PROMPT
readonly config: LinearConfig
readonly accessor: LinearAccessor
readonly index: IndexCacheStore
constructor(config: LinearConfig) {
super()
this.config = config
const transportOpts: { apiKey: string; baseUrl?: string } = { apiKey: config.apiKey }
if (config.baseUrl !== undefined) transportOpts.baseUrl = config.baseUrl
this.accessor = new LinearAccessor(new HttpLinearTransport(transportOpts))
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/mongodb/mongodb.ts
+3 -5
View File
@@ -13,9 +13,9 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
detectMongoScope,
type FileStat,
type IndexCacheStore,
MONGODB_COMMANDS,
MONGODB_OPS,
MONGODB_PROMPT,
@@ -26,7 +26,6 @@ import {
mongoReaddir,
mongoStat,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -43,7 +42,7 @@ export interface MongoDBResourceOptions {
prefix?: string
}
export class MongoDBResource implements Resource {
export class MongoDBResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.MONGODB
readonly isRemote: boolean = true
readonly indexTtl: number = 0
@@ -51,15 +50,14 @@ export class MongoDBResource implements Resource {
readonly config: MongoDBConfigResolved
readonly store: MongoDBStore
readonly accessor: MongoDBAccessor
readonly index: IndexCacheStore
constructor(options: MongoDBResourceOptions | MongoDBConfig) {
super()
const { config, prefix } =
'config' in options ? options : { config: options, prefix: undefined }
this.config = resolveMongoDBConfig(config)
this.store = new MongoDBStore(this.config.uri)
this.accessor = new MongoDBAccessor(this.store, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = MONGODB_PROMPT.replace('{prefix}', prefix ?? '')
}
typescript/packages/node/src/resource/postgres/postgres.ts
+3 -5
View File
@@ -13,8 +13,8 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
type IndexCacheStore,
PathSpec,
POSTGRES_COMMANDS,
POSTGRES_OPS,
@@ -25,7 +25,6 @@ import {
postgresRead,
postgresReaddir,
postgresStat,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -40,7 +39,7 @@ export interface PostgresResourceOptions {
prefix?: string
}
export class PostgresResource implements Resource {
export class PostgresResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.POSTGRES
readonly isRemote: boolean = true
readonly indexTtl: number = 0
@@ -48,15 +47,14 @@ export class PostgresResource implements Resource {
readonly config: PostgresConfigResolved
readonly store: PostgresStore
readonly accessor: PostgresAccessor
readonly index: IndexCacheStore
constructor(options: PostgresResourceOptions | PostgresConfig) {
super()
const { config, prefix } =
'config' in options ? options : { config: options, prefix: undefined }
this.config = resolvePostgresConfig(config)
this.store = new PostgresStore(this.config)
this.accessor = new PostgresAccessor(this.store, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = POSTGRES_PROMPT.replace('{prefix}', prefix ?? '')
}
typescript/packages/node/src/resource/posthog/posthog.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpPostHogDriver,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface PostHogResourceOptions {
driver?: PostHogDriver
}
export class PostHogResource implements Resource {
export class PostHogResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.POSTHOG
readonly isRemote: boolean = true
readonly indexTtl: number = 60
@@ -50,9 +49,9 @@ export class PostHogResource implements Resource {
readonly config: PostHogConfigResolved
readonly driver: PostHogDriver
readonly accessor: PostHogAccessor
readonly index: IndexCacheStore
constructor(options: PostHogResourceOptions | PostHogConfig = {}) {
super()
const opts: PostHogResourceOptions =
'config' in options || 'driver' in options || 'prefix' in options
? options
@@ -62,7 +61,6 @@ export class PostHogResource implements Resource {
opts.driver ??
new HttpPostHogDriver({ baseUrl: this.config.baseUrl, apiKey: this.config.apiKey })
this.accessor = new PostHogAccessor(this.driver, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = POSTHOG_PROMPT.replace('{prefix}', opts.prefix ?? '')
}
typescript/packages/node/src/resource/redis/redis.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
type FindOptions,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -67,7 +66,7 @@ export interface RedisResourceState {
dirs: string[]
}
export class RedisResource implements Resource {
export class RedisResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.REDIS
readonly isRemote: boolean = false
readonly indexTtl: number = 0
@@ -76,7 +75,6 @@ export class RedisResource implements Resource {
readonly keyPrefix: string
readonly store: RedisStore
readonly accessor: RedisAccessor
readonly index: IndexCacheStore
readonly opsMap: Record<string, unknown> = {
read_bytes: readCore,
@@ -100,11 +98,11 @@ export class RedisResource implements Resource {
}
constructor(options: RedisResourceOptions = {}) {
super()
this.url = options.url ?? 'redis://localhost:6379/0'
this.keyPrefix = options.keyPrefix ?? 'mirage:fs:'
this.store = new RedisStore({ url: this.url, keyPrefix: this.keyPrefix })
this.accessor = new RedisAccessor(this.store)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
async open(): Promise<void> {
typescript/packages/node/src/resource/s3/s3.ts
+3 -5
View File
@@ -13,6 +13,7 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
copy as copyCore,
create as createCore,
du as duCore,
@@ -21,11 +22,9 @@ import {
type FileStat,
type FindOptions,
find as findCore,
type IndexCacheStore,
mkdir as mkdirCore,
normalizeKeyPrefix,
PathSpec,
RAMIndexCacheStore,
rangeRead as rangeReadCore,
S3_COMMANDS,
read as readCore,
@@ -56,7 +55,7 @@ export interface S3ResourceState {
config: S3ConfigRedacted
}
export class S3Resource implements Resource {
export class S3Resource extends BaseResource implements Resource {
readonly kind: string = ResourceName.S3
readonly isRemote: boolean = true
readonly supportsSnapshot: boolean = true
@@ -64,7 +63,6 @@ export class S3Resource implements Resource {
readonly prompt: string = S3_PROMPT
readonly config: S3Config
readonly accessor: S3Accessor
readonly index: IndexCacheStore
readonly opsMap: Record<string, unknown> = {
read_bytes: readCore,
write: writeCore,
@@ -87,6 +85,7 @@ export class S3Resource implements Resource {
}
constructor(config: S3Config) {
super()
const normalized = normalizeKeyPrefix(config.keyPrefix)
const cfg: S3Config = { ...config }
if (normalized !== undefined) {
@@ -96,7 +95,6 @@ export class S3Resource implements Resource {
}
this.config = cfg
this.accessor = new S3Accessor(this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/slack/slack.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
type IndexCacheStore,
NodeSlackTransport,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -41,7 +40,7 @@ export interface SlackResourceState {
config: SlackConfigRedacted
}
export class SlackResource implements Resource {
export class SlackResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.SLACK
readonly isRemote: boolean = true
readonly indexTtl: number = 600
@@ -49,12 +48,11 @@ export class SlackResource implements Resource {
readonly writePrompt: string = SLACK_WRITE_PROMPT
readonly config: SlackConfig
readonly accessor: SlackAccessor
readonly index: IndexCacheStore
constructor(config: SlackConfig) {
super()
this.config = config
this.accessor = new SlackAccessor(new NodeSlackTransport(config.token, config.searchToken))
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/sscholar/author.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpSSCholarDriver,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface SSCholarAuthorResourceOptions {
driver?: SSCholarDriver
}
export class SSCholarAuthorResource implements Resource {
export class SSCholarAuthorResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.SSCHOLAR_AUTHOR
readonly isRemote: boolean = true
readonly indexTtl: number = 0
@@ -50,9 +49,9 @@ export class SSCholarAuthorResource implements Resource {
readonly config: SSCholarConfigResolved
readonly driver: SSCholarDriver
readonly accessor: SSCholarAccessor
readonly index: IndexCacheStore
constructor(options: SSCholarAuthorResourceOptions | SSCholarConfig = {}) {
super()
const opts: SSCholarAuthorResourceOptions =
'config' in options || 'driver' in options || 'prefix' in options
? options
@@ -62,7 +61,6 @@ export class SSCholarAuthorResource implements Resource {
opts.driver ??
new HttpSSCholarDriver({ baseUrl: this.config.baseUrl, apiKey: this.config.apiKey })
this.accessor = new SSCholarAccessor(this.driver, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = SSCHOLAR_AUTHOR_PROMPT.replace('{prefix}', opts.prefix ?? '')
}
typescript/packages/node/src/resource/sscholar/sscholar.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpSSCholarDriver,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface SSCholarPaperResourceOptions {
driver?: SSCholarDriver
}
export class SSCholarPaperResource implements Resource {
export class SSCholarPaperResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.SSCHOLAR_PAPER
readonly isRemote: boolean = true
readonly indexTtl: number = 0
@@ -50,9 +49,9 @@ export class SSCholarPaperResource implements Resource {
readonly config: SSCholarConfigResolved
readonly driver: SSCholarDriver
readonly accessor: SSCholarAccessor
readonly index: IndexCacheStore
constructor(options: SSCholarPaperResourceOptions | SSCholarConfig = {}) {
super()
const opts: SSCholarPaperResourceOptions =
'config' in options || 'driver' in options || 'prefix' in options
? options
@@ -62,7 +61,6 @@ export class SSCholarPaperResource implements Resource {
opts.driver ??
new HttpSSCholarDriver({ baseUrl: this.config.baseUrl, apiKey: this.config.apiKey })
this.accessor = new SSCholarAccessor(this.driver, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = SSCHOLAR_PAPER_PROMPT.replace('{prefix}', opts.prefix ?? '')
}
typescript/packages/node/src/resource/ssh/ssh.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
type FindOptions,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -54,14 +53,13 @@ export interface SSHResourceState {
config: SSHConfigRedacted
}
export class SSHResource implements Resource {
export class SSHResource extends BaseResource implements Resource {
readonly kind = ResourceName.SSH
readonly isRemote: boolean = true
readonly indexTtl: number = 60
readonly prompt = SSH_PROMPT
readonly config: SSHConfig
readonly accessor: SSHAccessor
readonly index: IndexCacheStore
readonly opsMap: Record<string, unknown> = {
read_bytes: readCoreFn,
write: writeCore,
@@ -85,9 +83,9 @@ export class SSHResource implements Resource {
}
constructor(config: SSHConfig) {
super()
this.config = config
this.accessor = new SSHAccessor(config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/trello/trello.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpTrelloTransport,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface TrelloResourceState {
config: TrelloConfigRedacted
}
export class TrelloResource implements Resource {
export class TrelloResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.TRELLO
readonly isRemote: boolean = true
readonly indexTtl: number = 600
@@ -50,9 +49,9 @@ export class TrelloResource implements Resource {
readonly writePrompt: string = TRELLO_WRITE_PROMPT
readonly config: TrelloConfig
readonly accessor: TrelloAccessor
readonly index: IndexCacheStore
constructor(config: TrelloConfig) {
super()
this.config = config
const transportOpts: { apiKey: string; apiToken: string; baseUrl?: string } = {
apiKey: config.apiKey,
@@ -60,7 +59,6 @@ export class TrelloResource implements Resource {
}
if (config.baseUrl !== undefined) transportOpts.baseUrl = config.baseUrl
this.accessor = new TrelloAccessor(new HttpTrelloTransport(transportOpts))
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
}
open(): Promise<void> {
typescript/packages/node/src/resource/vercel/vercel.ts
+3 -5
View File
@@ -13,11 +13,10 @@
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import {
BaseResource,
type FileStat,
HttpVercelDriver,
type IndexCacheStore,
PathSpec,
RAMIndexCacheStore,
type RegisteredCommand,
type RegisteredOp,
type Resource,
@@ -42,7 +41,7 @@ export interface VercelResourceOptions {
driver?: VercelDriver
}
export class VercelResource implements Resource {
export class VercelResource extends BaseResource implements Resource {
readonly kind: string = ResourceName.VERCEL
readonly isRemote: boolean = true
readonly indexTtl: number = 60
@@ -50,9 +49,9 @@ export class VercelResource implements Resource {
readonly config: VercelConfigResolved
readonly driver: VercelDriver
readonly accessor: VercelAccessor
readonly index: IndexCacheStore
constructor(options: VercelResourceOptions | VercelConfig = {}) {
super()
const opts: VercelResourceOptions =
'config' in options || 'driver' in options || 'prefix' in options
? options
@@ -66,7 +65,6 @@ export class VercelResource implements Resource {
teamId: this.config.teamId,
})
this.accessor = new VercelAccessor(this.driver, this.config)
this.index = new RAMIndexCacheStore({ ttl: this.indexTtl })
this.prompt = VERCEL_PROMPT.replace('{prefix}', opts.prefix ?? '')
}
typescript/packages/server/src/config.test.ts
+23
View File
@@ -12,6 +12,7 @@
// limitations under the License.
// ========= Copyright 2026 @ Strukto.AI All Rights Reserved. =========
import { RedisFileCacheStore } from '@struktoai/mirage-node'
import { describe, expect, it } from 'vitest'
import { interpolateEnv, loadWorkspaceConfig, configToWorkspaceArgs } from './config.ts'
@@ -67,4 +68,26 @@ describe('configToWorkspaceArgs', () => {
})
await expect(configToWorkspaceArgs(bad)).rejects.toThrow(/invalid mount mode/)
})
it('builds a redis index config from an index block', async () => {
const cfg = loadWorkspaceConfig({
mounts: { '/': { resource: 'ram' } },
index: { type: 'redis', url: 'redis://localhost:6379/0', keyPrefix: 'x:' },
})
const args = await configToWorkspaceArgs(cfg)
expect(args.options.index).toEqual({
type: 'redis',
url: 'redis://localhost:6379/0',
keyPrefix: 'x:',
})
})
it('builds a redis file cache from a cache block', async () => {
const cfg = loadWorkspaceConfig({
mounts: { '/': { resource: 'ram' } },
cache: { type: 'redis', keyPrefix: 'c:' },
})
const args = await configToWorkspaceArgs(cfg)
expect(args.options.cache).toBeInstanceOf(RedisFileCacheStore)
})
})
typescript/packages/server/src/config.ts
+78 -1
View File
@@ -14,7 +14,16 @@
import { readFileSync } from 'node:fs'
import { parse as parseYaml } from 'yaml'
import { buildResource, MountMode, type Resource } from '@struktoai/mirage-node'
import {
buildResource,
MountMode,
RAMFileCacheStore,
RedisFileCacheStore,
type FileCache,
type IndexConfig,
type RedisIndexConfig,
type Resource,
} from '@struktoai/mirage-node'
const VALID_MODES = new Set<string>([MountMode.READ, MountMode.WRITE, MountMode.EXEC])
@@ -67,6 +76,32 @@ export interface MountBlock {
config?: Record<string, unknown>
}
export interface RamCacheBlock {
type?: 'ram'
limit?: string | number
maxDrainBytes?: number | null
}
export interface RedisCacheBlock {
type: 'redis'
limit?: string | number
maxDrainBytes?: number | null
url?: string
keyPrefix?: string
}
export interface RamIndexBlock {
type?: 'ram'
ttl?: number
}
export interface RedisIndexBlock {
type: 'redis'
ttl?: number
url?: string
keyPrefix?: string
}
export interface WorkspaceConfigRaw {
mounts: Record<string, MountBlock>
mode?: string
@@ -75,6 +110,8 @@ export interface WorkspaceConfigRaw {
defaultAgentId?: string
history?: number | null
fuse?: boolean
cache?: RamCacheBlock | RedisCacheBlock | null
index?: RamIndexBlock | RedisIndexBlock | null
}
function readProcessEnv(): Record<string, string> {
@@ -120,9 +157,45 @@ export interface WorkspaceArgs {
mode: MountMode
sessionId: string
agentId: string
cache?: FileCache & Resource
index?: IndexConfig
}
}
function buildCache(
block: RamCacheBlock | RedisCacheBlock | null | undefined,
): (FileCache & Resource) | undefined {
if (block === null || block === undefined) return undefined
if (block.type === 'redis') {
return new RedisFileCacheStore({
...(block.limit !== undefined ? { cacheLimit: block.limit } : {}),
...(block.maxDrainBytes !== undefined ? { maxDrainBytes: block.maxDrainBytes } : {}),
...(block.url !== undefined ? { url: block.url } : {}),
...(block.keyPrefix !== undefined ? { keyPrefix: block.keyPrefix } : {}),
})
}
return new RAMFileCacheStore({
...(block.limit !== undefined ? { limit: block.limit } : {}),
...(block.maxDrainBytes !== undefined ? { maxDrainBytes: block.maxDrainBytes } : {}),
})
}
function buildIndex(
block: RamIndexBlock | RedisIndexBlock | null | undefined,
): IndexConfig | undefined {
if (block === null || block === undefined) return undefined
if (block.type === 'redis') {
const cfg: RedisIndexConfig = { type: 'redis' }
if (block.ttl !== undefined) cfg.ttl = block.ttl
if (block.url !== undefined) cfg.url = block.url
if (block.keyPrefix !== undefined) cfg.keyPrefix = block.keyPrefix
return cfg
}
const cfg: IndexConfig = { type: 'ram' }
if (block.ttl !== undefined) cfg.ttl = block.ttl
return cfg
}
export async function configToWorkspaceArgs(cfg: WorkspaceConfigRaw): Promise<WorkspaceArgs> {
const wsMode = coerceMountMode(cfg.mode, MountMode.WRITE)
const resources: Record<string, [Resource, MountMode]> = {}
@@ -131,12 +204,16 @@ export async function configToWorkspaceArgs(cfg: WorkspaceConfigRaw): Promise<Wo
const m = coerceMountMode(block.mode, wsMode)
resources[prefix] = [r, m]
}
const cache = buildCache(cfg.cache)
const index = buildIndex(cfg.index)
return {
resources,
options: {
mode: wsMode,
sessionId: cfg.defaultSessionId ?? 'default',
agentId: cfg.defaultAgentId ?? 'default',
...(cache !== undefined ? { cache } : {}),
...(index !== undefined ? { index } : {}),
},
}
}
typescript/packages/server/src/routers/workspaces.ts
+6 -1
View File
@@ -76,7 +76,12 @@ export function registerWorkspacesRoutes(app: FastifyInstance, deps: WorkspaceRo
resourceMap[prefix] = resource
modeOverrides[prefix] = mode
}
const ws = new Workspace(resourceMap, { mode: args.options.mode, modeOverrides })
const ws = new Workspace(resourceMap, {
mode: args.options.mode,
modeOverrides,
...(args.options.cache !== undefined ? { cache: args.options.cache } : {}),
...(args.options.index !== undefined ? { index: args.options.index } : {}),
})
let entry
try {
entry = deps.registry.add(ws, body.id)
typescript/pnpm-lock.yaml
Generated
+12 -1
View File
@@ -308,6 +308,9 @@ importers:
pyodide:
specifier: ^0.29.3
version: 0.29.3
redis:
specifier: ^5.0.0
version: 5.12.1(@opentelemetry/api@1.9.0)
tree-sitter-bash:
specifier: ^0.25.1
version: 0.25.1
@@ -7966,6 +7969,14 @@ snapshots:
optionalDependencies:
vite: 7.3.2(@types/node@22.19.17)(tsx@4.21.0)(yaml@2.8.3)
'@vitest/mocker@3.2.4(vite@7.3.2(@types/node@24.12.2)(tsx@4.21.0)(yaml@2.8.3))':
dependencies:
'@vitest/spy': 3.2.4
estree-walker: 3.0.3
magic-string: 0.30.21
optionalDependencies:
vite: 7.3.2(@types/node@24.12.2)(tsx@4.21.0)(yaml@2.8.3)
'@vitest/pretty-format@3.2.4':
dependencies:
tinyrainbow: 2.0.0
@@ -11069,7 +11080,7 @@ snapshots:
dependencies:
'@types/chai': 5.2.3
'@vitest/expect': 3.2.4
'@vitest/mocker': 3.2.4(vite@7.3.2(@types/node@22.19.17)(tsx@4.21.0)(yaml@2.8.3))
'@vitest/mocker': 3.2.4(vite@7.3.2(@types/node@24.12.2)(tsx@4.21.0)(yaml@2.8.3))
'@vitest/pretty-format': 3.2.4
'@vitest/runner': 3.2.4
'@vitest/snapshot': 3.2.4