fix(test): nest tool-call-policy under guardrails.policies

The config.yaml dir-bundle parser (omnigent.spec.parser) reads policies from guardrails.policies and ignores a top-level policies: block — so the converted fixture's DENY policy never loaded (spec.guardrails was None) and calculate ran (tool output '12') instead of being denied. The inline single-YAML form the test used before accepts top-level policies:, which masked the difference. Verified: parse() now loads deny_calculate_tool under guardrails, and the make_fixed_action_callable builtin denies tool_call:calculate with the sentinel (allows other tools/phases). Signed-off-by: dbczumar <corey.zumar@databricks.com>
tests: allowlist bundled-tool fixture agents in coverage-sync
2026-06-22 16:42:28 -07:00 · 2026-06-22 16:19:27 -07:00 · 2026-06-22 15:59:26 -07:00 · 2026-06-22 15:58:08 -07:00 · 2026-06-22 15:27:19 -07:00 · 2026-06-22 15:16:35 -07:00
26 changed files with 1325 additions and 395 deletions
@@ -0,0 +1,186 @@
+name: "Run e2e suite"
+description: >
+  Run the tests/e2e suite exactly as the e2e.yml gate does (mock LLM,
+  sharded). When `server_version` is set, the omnigent SERVER subprocess is
+  pinned to that released tag (built into an isolated venv) while the client,
+  runner, and tests stay on the checked-out ref — the server-version
+  backwards-compat configuration. Shared verbatim by e2e.yml (normal gate) and
+  server-compat.yml (backcompat jobs) so the two never drift. The caller is
+  responsible for the preceding `actions/checkout` (the checkout ref differs:
+  the gate tests refs/pull/N/merge; backcompat needs fetch-depth 0 for tags).
+
+inputs:
+  shard_id:
+    description: "pytest-shard shard index"
+    required: true
+  num_shards:
+    description: "pytest-shard shard count"
+    required: true
+  parallelism:
+    description: "pytest workers (-n)"
+    required: false
+    default: "2"
+  nightly_full:
+    description: "true = full pass (schedule/dispatch); false = exclude @nightly"
+    required: false
+    default: "false"
+  server_version:
+    description: >
+      Empty = run the checked-out server (normal gate). Set to a release tag
+      (e.g. v0.1.1) = build that old server into a venv and redirect the
+      server subprocess to it (backwards-compat run).
+    required: false
+    default: ""
+
+runs:
+  using: composite
+  steps:
+    - name: Configure environment
+      shell: bash
+      run: |
+        # Self-contained so the action behaves identically regardless of the
+        # caller's env. No ap-web SPA build during installs (this job never
+        # serves the bundle); blank provider keys so a spawned server can't
+        # pick up the runner's own credentials.
+        {
+          echo "OMNIGENT_SKIP_WEB_UI=true"
+          echo "ANTHROPIC_API_KEY="
+          echo "OPENAI_API_KEY="
+          echo "CODEX="
+          echo "CLAUDE_CODE="
+        } >> "$GITHUB_ENV"
+
+    - name: Set up Python
+      uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5
+      with:
+        python-version-file: ".python-version"
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a  # v4
+      with:
+        enable-cache: true
+
+    - name: Cache virtualenv
+      uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684  # v4
+      with:
+        path: .venv
+        key: venv-${{ runner.os }}-${{ hashFiles('.python-version') }}-${{ hashFiles('uv.lock') }}
+
+    - name: Install project and dev dependencies
+      shell: bash
+      run: uv sync --extra all --extra dev
+
+    - name: Install binary dependencies
+      # npm install against .github/ci-deps/package.json with --ignore-scripts
+      # to block postinstall on every package. The claude-code stub binary
+      # needs its install.cjs (audited: platform detect + same-tree hardlink,
+      # no network/exec) so we run that one explicitly; codex and pi have no
+      # install scripts and ship prebuilt CLIs. bubblewrap: the linux_bwrap
+      # sandbox backend fails loud if `bwrap` is missing, and the e2e runner
+      # runs real agents with os_env. The apparmor sysctl mirrors ci.yml
+      # (Ubuntu 24.04 blocks unprivileged user namespaces, which bwrap's
+      # unshare(CLONE_NEWUSER) needs).
+      working-directory: .github/ci-deps
+      shell: bash
+      run: |
+        sudo apt-get install -y tmux bubblewrap
+        sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+        npm install --ignore-scripts
+        node node_modules/@anthropic-ai/claude-code/install.cjs
+        echo "${GITHUB_WORKSPACE}/.github/ci-deps/node_modules/.bin" >> "$GITHUB_PATH"
+
+    - name: Build pinned old server (backwards-compat only)
+      # Only runs when server_version is set. Builds the released tag into an
+      # isolated venv (all three packages editable so the old ==<old> SDK
+      # cross-pins resolve without an index) and points the server subprocess
+      # at it via OMNIGENT_COMPAT_SERVER_PYTHON. The redirect also drops the
+      # worktree PYTHONPATH/CWD shadow (see tests/_helpers/compat.py) so the
+      # pinned install actually resolves. Requires fetch-depth 0 in the caller.
+      if: ${{ inputs.server_version != '' }}
+      shell: bash
+      env:
+        SERVER_VERSION_INPUT: ${{ inputs.server_version }}
+      run: |
+        tag="$SERVER_VERSION_INPUT"
+        if ! [[ "$tag" =~ ^v?[0-9]+\.[0-9]+(\.[0-9]+)?([a-z0-9.]*)?$ ]]; then
+          echo "Invalid server_version: '$tag'" >&2; exit 1
+        fi
+        src="$RUNNER_TEMP/server-src"
+        venv="$RUNNER_TEMP/server-env"
+        git worktree add --detach "$src" "$tag"
+        uv venv --python 3.12 "$venv"
+        uv pip install --python "$venv/bin/python" \
+          -e "$src" -e "$src/sdks/python-client" -e "$src/sdks/ui"
+        "$venv/bin/omnigent" --version
+        echo "OMNIGENT_COMPAT_SERVER_PYTHON=$venv/bin/python" >> "$GITHUB_ENV"
+        echo "OMNIGENT_COMPAT_SERVER_VERSION=${tag#v}" >> "$GITHUB_ENV"
+
+    - name: Run e2e tests
+      shell: bash
+      env:
+        PARALLELISM_INPUT: ${{ inputs.parallelism }}
+        SHARD_ID: ${{ inputs.shard_id }}
+        NUM_SHARDS: ${{ inputs.num_shards }}
+        NIGHTLY_FULL: ${{ inputs.nightly_full }}
+        E2E_TMP_BASE: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}
+        PYTEST_PROGRESS_LOG_DIR: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/progress
+        OMNIGENT_TOKEN_USAGE_JSON: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/tokens.json
+      run: |
+        # Validate parallelism (untrusted input -- bind to env, never
+        # interpolate a GitHub expression into the shell).
+        if ! [[ "$PARALLELISM_INPUT" =~ ^[1-9][0-9]?$ ]]; then
+          echo "Invalid parallelism input: $PARALLELISM_INPUT (expected 1-99)" >&2
+          exit 1
+        fi
+        WORKERS="$PARALLELISM_INPUT"
+        mkdir -p "$E2E_TMP_BASE"
+
+        EXTRA_ARGS=()
+        if [[ "$NIGHTLY_FULL" != "true" ]]; then
+          EXTRA_ARGS+=(-m "not nightly")
+        fi
+
+        # --junitxml emits per-test results eagerly so diagnostics survive a
+        # wall-clock overrun. --shard-id/--num-shards chunk the node IDs.
+        # --timeout=180 caps each test; --timeout-method=thread because our
+        # pty/subprocess children don't get SIGALRM. --max-worker-restart=0
+        # fails the shard fast instead of letting loadscope requeue deadlock
+        # the controller (the 2026-06-11 shard-2 wedge).
+        uv run pytest tests/e2e/ \
+          -n "$WORKERS" \
+          --dist=loadscope \
+          --max-worker-restart=0 \
+          --shard-id="$SHARD_ID" \
+          --num-shards="$NUM_SHARDS" \
+          --timeout=180 \
+          --timeout-method=thread \
+          --basetemp="$E2E_TMP_BASE" \
+          --junitxml="$E2E_TMP_BASE/junit.xml" \
+          -v --tb=long --showlocals --log-level=INFO -r a \
+          "${EXTRA_ARGS[@]}" \
+          || { rc=$?; [ "$rc" -eq 5 ] && echo "::notice::No tests collected in this shard; treating as a pass." || exit "$rc"; }
+
+    - name: Upload server logs on failure
+      # cancelled() too: failure() misses step timeouts (#426).
+      if: ${{ failure() || cancelled() }}
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
+      with:
+        name: e2e-server-logs-${{ github.run_id }}-shard${{ inputs.shard_id }}
+        path: |
+          /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/**/server.log
+          /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/**/runner.log
+          /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/**/.omnigent/logs/**/*.log
+          /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/junit.xml
+          /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/progress/progress-*.log
+        retention-days: 7
+        if-no-files-found: warn
+        include-hidden-files: true
+
+    - name: Upload token usage
+      if: ${{ always() }}
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
+      with:
+        name: e2e-tokens-${{ github.run_id }}-shard${{ inputs.shard_id }}
+        path: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ inputs.shard_id }}/tokens*.json
+        retention-days: 14
+        if-no-files-found: warn
@@ -0,0 +1,150 @@
+name: "Run integration suite"
+description: >
+  Run the tests/integration journey suite exactly as the integration.yml gate
+  does (mock LLM, one wrapped harness per invocation). When `server_version`
+  is set, the omnigent SERVER subprocess is pinned to that released tag while
+  the client, runner, and tests stay on the checked-out ref — the
+  server-version backwards-compat configuration. Shared verbatim by
+  integration.yml (normal gate) and server-compat.yml (backcompat jobs) so the
+  two never drift. The caller owns the preceding `actions/checkout` (backcompat
+  needs fetch-depth 0 for tags).
+
+inputs:
+  harness:
+    description: "Wrapped harness (claude-sdk | openai-agents | codex)"
+    required: true
+  model:
+    description: "Model name passed to --model"
+    required: true
+  workers:
+    description: "pytest workers (-n)"
+    required: true
+  server_version:
+    description: >
+      Empty = run the checked-out server (normal gate). Set to a release tag
+      = build that old server into a venv and redirect the server subprocess
+      to it (backwards-compat run).
+    required: false
+    default: ""
+
+runs:
+  using: composite
+  steps:
+    - name: Configure environment
+      shell: bash
+      run: |
+        {
+          echo "OMNIGENT_SKIP_WEB_UI=true"
+          echo "ANTHROPIC_API_KEY="
+          echo "OPENAI_API_KEY="
+          echo "CODEX="
+          echo "CLAUDE_CODE="
+        } >> "$GITHUB_ENV"
+
+    - name: Set up Python
+      uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5
+      with:
+        python-version-file: ".python-version"
+
+    - name: Install uv
+      uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a  # v4
+      with:
+        enable-cache: true
+
+    - name: Cache virtualenv
+      uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684  # v4
+      with:
+        path: .venv
+        key: venv-${{ runner.os }}-${{ hashFiles('.python-version') }}-${{ hashFiles('uv.lock') }}
+
+    - name: Install project and dev dependencies
+      shell: bash
+      run: uv sync --extra all --extra dev
+
+    - name: Install binary dependencies
+      # Mirrors e2e.yml. --ignore-scripts blocks npm postinstall hooks; we run
+      # claude-code's install.cjs explicitly (audited, no network). bubblewrap
+      # backs the linux_bwrap sandbox in tests/inner/*.
+      working-directory: .github/ci-deps
+      shell: bash
+      run: |
+        set -euo pipefail
+        sudo apt-get update
+        sudo apt-get install -y tmux ripgrep bubblewrap
+        sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
+        npm install --ignore-scripts
+        node node_modules/@anthropic-ai/claude-code/install.cjs
+        echo "${GITHUB_WORKSPACE}/.github/ci-deps/node_modules/.bin" >> "$GITHUB_PATH"
+
+    - name: Build pinned old server (backwards-compat only)
+      # See e2e-run for the full rationale. Requires fetch-depth 0 in the caller.
+      if: ${{ inputs.server_version != '' }}
+      shell: bash
+      env:
+        SERVER_VERSION_INPUT: ${{ inputs.server_version }}
+      run: |
+        tag="$SERVER_VERSION_INPUT"
+        if ! [[ "$tag" =~ ^v?[0-9]+\.[0-9]+(\.[0-9]+)?([a-z0-9.]*)?$ ]]; then
+          echo "Invalid server_version: '$tag'" >&2; exit 1
+        fi
+        src="$RUNNER_TEMP/server-src"
+        venv="$RUNNER_TEMP/server-env"
+        git worktree add --detach "$src" "$tag"
+        uv venv --python 3.12 "$venv"
+        uv pip install --python "$venv/bin/python" \
+          -e "$src" -e "$src/sdks/python-client" -e "$src/sdks/ui"
+        "$venv/bin/omnigent" --version
+        echo "OMNIGENT_COMPAT_SERVER_PYTHON=$venv/bin/python" >> "$GITHUB_ENV"
+        echo "OMNIGENT_COMPAT_SERVER_VERSION=${tag#v}" >> "$GITHUB_ENV"
+
+    - name: Run integration tests
+      shell: bash
+      env:
+        HARNESS: ${{ inputs.harness }}
+        MODEL: ${{ inputs.model }}
+        WORKERS: ${{ inputs.workers }}
+        INTEGRATION_TMP_BASE: /tmp/omnigent-integration-${{ github.run_id }}-${{ inputs.harness }}
+        CLAUDE_CODE_STREAM_CLOSE_TIMEOUT: "60000"
+        OMNIGENT_CLAUDE_SDK_NO_SANDBOX: ${{ inputs.harness == 'claude-sdk' && '1' || '' }}
+        PYTEST_PROGRESS_LOG_DIR: ${{ github.workspace }}/artifacts/progress-${{ inputs.harness }}
+        OMNIGENT_TOKEN_USAGE_JSON: ${{ github.workspace }}/artifacts/tokens-${{ inputs.harness }}.json
+        OMNIGENT_TEST_MODEL_SPREAD: "1"
+        OMNIGENT_TEST_MODEL_POOL_GPT: "databricks-gpt-5-5,databricks-gpt-5-4-mini"
+      run: |
+        set -euo pipefail
+        mkdir -p artifacts "$INTEGRATION_TMP_BASE"
+        # --capture=no + --log-cli-level=INFO stream live so progress shows
+        # even if the step hits its timeout before buffered output renders.
+        # --timeout=180 caps a single hung test (see e2e.yml).
+        env -u OPENAI_API_KEY -u ANTHROPIC_API_KEY -u DATABRICKS_TOKEN \
+          uv run pytest tests/integration/ \
+            --model "$MODEL" \
+            --harness "$HARNESS" \
+            -n "$WORKERS" \
+            --dist=loadscope \
+            --timeout=180 \
+            --timeout-method=thread \
+            --basetemp="$INTEGRATION_TMP_BASE" \
+            --junitxml="artifacts/integration-${HARNESS}.xml" \
+            --capture=no --log-cli-level=INFO \
+            -v --tb=long --showlocals --log-level=INFO -r a
+
+    - name: Upload server/runner logs on failure
+      if: ${{ failure() || cancelled() }}
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
+      with:
+        name: integration-server-logs-${{ inputs.harness }}-${{ github.run_id }}
+        path: |
+          /tmp/omnigent-integration-${{ github.run_id }}-${{ inputs.harness }}/**/server.log
+          /tmp/omnigent-integration-${{ github.run_id }}-${{ inputs.harness }}/**/runner.log
+        retention-days: 7
+        if-no-files-found: warn
+
+    - name: Upload junit + logs
+      if: ${{ always() }}
+      uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
+      with:
+        name: integration-${{ inputs.harness }}-${{ github.run_id }}
+        path: artifacts/
+        retention-days: 14
+        if-no-files-found: ignore
@@ -100,6 +100,9 @@ jobs:
    # (forks run via the fork-e2e/** mirror push), so no shard runs for them.
    needs: setup
    runs-on: ubuntu-latest
+    # Job-level cap (composite-action run steps can't set timeout-minutes):
+    # ~30 min of tests + setup, replacing the old per-step 30-min backstop.
+    timeout-minutes: 35
    strategy:
      # One red shard shouldn't cancel siblings -- we want every shard's signal.
      fail-fast: false
@@ -116,132 +119,16 @@ jobs:
          # Push (fork-e2e/**) and dispatch fall back to the branch / ref.
          ref: ${{ github.event.pull_request.number && format('refs/pull/{0}/merge', github.event.pull_request.number) || github.event.inputs.branch || github.ref }}

-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5
+      # Steps below are shared verbatim with server-compat.yml's backcompat-e2e
+      # job via the composite action, so the two never drift. server_version
+      # is omitted here -> normal gate (tests the checked-out server, mock LLM).
+      - name: Run e2e suite
+        uses: ./.github/actions/e2e-run
        with:
-          python-version-file: ".python-version"
-
-      - name: Install uv
-        uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a  # v4
-        with:
-          enable-cache: true
-
-
-
-
-      - name: Cache virtualenv
-        uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684  # v4
-        with:
-          path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('.python-version') }}-${{ hashFiles('uv.lock') }}
-      - name: Install project and dev dependencies
-        run: |
-          uv sync --extra all --extra dev
-
-      - name: Install binary dependencies
-        # npm install against .github/ci-deps/package.json with
-        # --ignore-scripts to block postinstall on every package. The
-        # claude-code stub binary needs its install.cjs (audited:
-        # platform detect + same-tree hardlink, no network/exec) so we run
-        # that one explicitly; codex and pi have no install scripts and
-        # ship prebuilt CLIs, so --ignore-scripts + the PATH line below
-        # make them runnable directly.
-        #
-        # bubblewrap: the linux_bwrap sandbox backend fails loud if `bwrap`
-        # is missing, and the e2e runner runs real agents with os_env. The
-        # apparmor sysctl mirrors ci.yml (Ubuntu 24.04 blocks unprivileged
-        # user namespaces, which bwrap's unshare(CLONE_NEWUSER) needs).
-        working-directory: .github/ci-deps
-        run: |
-          sudo apt-get install -y tmux bubblewrap
-          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
-          npm install --ignore-scripts
-          node node_modules/@anthropic-ai/claude-code/install.cjs
-          echo "${GITHUB_WORKSPACE}/.github/ci-deps/node_modules/.bin" >> "$GITHUB_PATH"
-
-
-
-      - name: Run e2e tests
-        timeout-minutes: 30
-        env:
-          # Cron fallback must match the workflow_dispatch default above.
-          PARALLELISM_INPUT: ${{ github.event.inputs.parallelism || '2' }}
-          SHARD_ID: ${{ matrix.shard_id }}
-          NUM_SHARDS: ${{ matrix.num_shards }}
-          # Schedule / dispatch are the full pass; PR and push skip @nightly.
-          NIGHTLY_FULL: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
-          # Stable per-shard prefix so the upload step finds the logs / junit.
-          E2E_TMP_BASE: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}
-          # Per-worker progress log (#426): fsynced START/END per test so we
-          # recover the last-started test when a runner wedges.
-          PYTEST_PROGRESS_LOG_DIR: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/progress
-          OMNIGENT_TOKEN_USAGE_JSON: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/tokens.json
-        run: |
-          # Validate parallelism (untrusted input -- bind to env, never
-          # interpolate a GitHub expression into the shell).
-          if ! [[ "$PARALLELISM_INPUT" =~ ^[1-9][0-9]?$ ]]; then
-            echo "Invalid parallelism input: $PARALLELISM_INPUT (expected 1-99)" >&2
-            exit 1
-          fi
-          WORKERS="$PARALLELISM_INPUT"
-          mkdir -p "$E2E_TMP_BASE"
-
-          EXTRA_ARGS=()
-          if [[ "$NIGHTLY_FULL" != "true" ]]; then
-            EXTRA_ARGS+=(-m "not nightly")
-          fi
-
-          # --junitxml emits per-test results eagerly so diagnostics survive
-          # a wall-clock overrun. --shard-id/--num-shards chunk the node IDs.
-          # --timeout=180 caps each test; --timeout-method=thread because our
-          # pty/subprocess children don't get SIGALRM. --max-worker-restart=0
-          # fails the shard fast instead of letting loadscope requeue deadlock
-          # the controller (the 2026-06-11 shard-2 wedge).
-          uv run pytest tests/e2e/ \
-            -n "$WORKERS" \
-            --dist=loadscope \
-            --max-worker-restart=0 \
-            --shard-id="$SHARD_ID" \
-            --num-shards="$NUM_SHARDS" \
-            --timeout=180 \
-            --timeout-method=thread \
-            --basetemp="$E2E_TMP_BASE" \
-            --junitxml="$E2E_TMP_BASE/junit.xml" \
-            -v --tb=long --showlocals --log-level=INFO -r a \
-            "${EXTRA_ARGS[@]}" \
-            || { rc=$?; [ "$rc" -eq 5 ] && echo "::notice::No tests collected in this shard; treating as a pass." || exit "$rc"; }
-
-      - name: Upload server logs on failure
-        # cancelled() too: failure() misses step timeouts (#426).
-        if: failure() || cancelled()
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
-        with:
-          # Per-shard name so parallel uploads don't collide.
-          name: e2e-server-logs-${{ github.run_id }}-shard${{ matrix.shard_id }}
-          # Whitelist diagnostic files (basetemp also holds large per-test
-          # DBs / tarballs). `warn` not `ignore` so a broken path is loud.
-          path: |
-            /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/**/server.log
-            /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/**/runner.log
-            /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/**/.omnigent/logs/**/*.log
-            /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/junit.xml
-            /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/progress/progress-*.log
-          retention-days: 7
-          if-no-files-found: warn
-          # Daemon logs live under hidden `.omnigent/` dirs, which v4 skips
-          # by default -- without this the `.omnigent/logs` glob matches nothing.
-          include-hidden-files: true
-
-      - name: Upload token usage
-        if: always()
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
-        with:
-          name: e2e-tokens-${{ github.run_id }}-shard${{ matrix.shard_id }}
-          path: /tmp/omnigent-e2e-${{ github.run_id }}-shard${{ matrix.shard_id }}/tokens*.json
-          retention-days: 14
-          # `warn` not `ignore`: every shard makes LLM calls, so a missing
-          # tokens file means the recorder broke.
-          if-no-files-found: warn
+          shard_id: ${{ matrix.shard_id }}
+          num_shards: ${{ matrix.num_shards }}
+          parallelism: ${{ github.event.inputs.parallelism || '2' }}
+          nightly_full: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}

  # Explicitly re-dispatch Merge Ready (same-repo PR or fork-e2e push): the
  # workflow_run hop is brittle and was dropped on #751/#792. No checkout,
@@ -102,104 +102,15 @@ jobs:
        with:
          ref: ${{ github.event.inputs.branch || github.ref }}

-      - name: Set up Python
-        uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065  # v5
+      # Shared verbatim with server-compat.yml's backcompat-integration job via
+      # the composite action, so the two never drift. server_version is omitted
+      # here -> normal gate (tests the checked-out server, mock LLM).
+      - name: Run integration suite
+        uses: ./.github/actions/integration-run
        with:
-          python-version-file: ".python-version"
-
-      - name: Install uv
-        uses: astral-sh/setup-uv@38f3f104447c67c051c4a08e39b64a148898af3a  # v4
-        with:
-          enable-cache: true
-
-
-
-
-      - name: Cache virtualenv
-        uses: actions/cache@5a3ec84eff668545956fd18022155c47e93e2684  # v4
-        with:
-          path: .venv
-          key: venv-${{ runner.os }}-${{ hashFiles('.python-version') }}-${{ hashFiles('uv.lock') }}
-      - name: Install project and dev dependencies
-        run: uv sync --extra all --extra dev
-
-      - name: Install binary dependencies
-        # Mirrors e2e.yml. `--ignore-scripts` blocks npm postinstall hooks;
-        # we run claude-code's install.cjs explicitly (audited, no network).
-        # `bubblewrap` backs the `linux_bwrap` sandbox in tests/inner/*.
-        working-directory: .github/ci-deps
-        run: |
-          set -euo pipefail
-          sudo apt-get update
-          sudo apt-get install -y tmux ripgrep bubblewrap
-          sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
-          npm install --ignore-scripts
-          node node_modules/@anthropic-ai/claude-code/install.cjs
-          echo "${GITHUB_WORKSPACE}/.github/ci-deps/node_modules/.bin" >> "$GITHUB_PATH"
-
-
-
-      - name: Run nightly tests
-        timeout-minutes: 25
-        env:
-          HARNESS: ${{ matrix.harness }}
-          MODEL: ${{ matrix.model }}
-          WORKERS: ${{ matrix.workers }}
-          # Stable basetemp so the failure-upload step can find the logs.
-          INTEGRATION_TMP_BASE: /tmp/omnigent-integration-${{ github.run_id }}-${{ matrix.harness }}
-          # SDK initialize control-request timeout (ms).
-          CLAUDE_CODE_STREAM_CLOSE_TIMEOUT: '60000'
-          # Diagnostic: bypass create_exec_launcher on claude-sdk to isolate
-          # whether the silent connect hang is sandbox-related.
-          OMNIGENT_CLAUDE_SDK_NO_SANDBOX: ${{ matrix.harness == 'claude-sdk' && '1' || '' }}
-          # Per-xdist-worker progress log (#426): recovers the last-started
-          # test when a runner wedges.
-          PYTEST_PROGRESS_LOG_DIR: ${{ github.workspace }}/artifacts/progress-${{ matrix.harness }}
-          # Per-model call/token tally (dev/aggregate_token_usage.py).
-          OMNIGENT_TOKEN_USAGE_JSON: ${{ github.workspace }}/artifacts/tokens-${{ matrix.harness }}.json
-          # Load-balance interchangeable gateway models (tests/_model_pools.py).
-          OMNIGENT_TEST_MODEL_SPREAD: '1'
-          # gpt-5-4 FMAPI quota is far below its pool neighbors; drain it
-          # until the tier is raised so 429s don't fail hashed-to-gpt-5-4 tests.
-          OMNIGENT_TEST_MODEL_POOL_GPT: 'databricks-gpt-5-5,databricks-gpt-5-4-mini'
-        run: |
-          set -euo pipefail
-          mkdir -p artifacts "$INTEGRATION_TMP_BASE"
-          # --capture=no + --log-cli-level=INFO stream live so progress shows
-          # even if the step hits its timeout before buffered output renders.
-          # --timeout=180 caps a single hung test (see e2e.yml).
-          env -u OPENAI_API_KEY -u ANTHROPIC_API_KEY -u DATABRICKS_TOKEN \
-            uv run pytest tests/integration/ \
-              --model "$MODEL" \
-              --harness "$HARNESS" \
-              -n "$WORKERS" \
-              --dist=loadscope \
-              --timeout=180 \
-              --timeout-method=thread \
-              --basetemp="$INTEGRATION_TMP_BASE" \
-              --junitxml="artifacts/integration-${HARNESS}.xml" \
-              --capture=no --log-cli-level=INFO \
-              -v --tb=long --showlocals --log-level=INFO -r a
-
-      - name: Upload server/runner logs on failure
-        if: failure() || cancelled()
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
-        with:
-          name: integration-server-logs-${{ matrix.harness }}-${{ github.run_id }}
-          path: |
-            /tmp/omnigent-integration-${{ github.run_id }}-${{ matrix.harness }}/**/server.log
-            /tmp/omnigent-integration-${{ github.run_id }}-${{ matrix.harness }}/**/runner.log
-          retention-days: 7
-          if-no-files-found: warn
-
-      - name: Upload junit + logs
-        if: always()
-        uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02  # v4
-        with:
-          name: integration-${{ matrix.harness }}-${{ github.run_id }}
-          path: artifacts/
-          retention-days: 14
-          if-no-files-found: ignore
+          harness: ${{ matrix.harness }}
+          model: ${{ matrix.model }}
+          workers: ${{ matrix.workers }}

  # Explicitly re-dispatch Merge Ready (same-repo PR or fork-e2e push): the
  # workflow_run hop is brittle and was dropped on #751/#792. No checkout,
@@ -0,0 +1,152 @@
+name: Server Backwards-Compat
+
+# Runs main's e2e + integration suites against a PINNED OLDER server, to catch
+# backwards-incompatible server changes before release. Only the server
+# subprocess is the old build; the client, runner, and tests are all the
+# checked-out ref. See docs/SERVER_VERSION_COMPAT_CI.md.
+#
+# The actual test runs are the SAME composite actions the normal gates use
+# (.github/actions/e2e-run, .github/actions/integration-run) — invoked here
+# with `server_version` set. So these backcompat jobs run the suites byte-for-
+# byte the way e2e.yml / integration.yml do (mock LLM), differing only in that
+# the server subprocess is redirected to the old build. No drift.
+#
+# Triggers:
+#   workflow_dispatch   manual; `server_version` input picks the old tag.
+#   schedule            nightly; pins the latest non-rc release tag.
+
+on:
+  workflow_dispatch:
+    inputs:
+      server_version:
+        description: "Old server tag to test against, e.g. v0.1.1. Empty = latest non-rc tag."
+        required: false
+        default: ""
+  schedule:
+    # Every 4 hours.
+    - cron: "0 */4 * * *"
+
+concurrency:
+  group: server-compat-${{ github.workflow }}-${{ github.event.pull_request.number || github.event.inputs.server_version || github.sha }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  # Compute the SAME matrices the gates use (e2e-shard-matrix.sh /
+  # integration-matrix.sh), so backcompat runs exactly the shards/legs the real
+  # gate runs for this event — no hardcoded list to drift. Notably integration
+  # is openai-agents only: claude-sdk/codex reject the mock LLM's "mock-model"
+  # so the gate excludes them (see integration-matrix.sh); backcompat must too.
+  setup:
+    name: setup
+    runs-on: ubuntu-latest
+    timeout-minutes: 5
+    outputs:
+      e2e_matrix: ${{ steps.e2e.outputs.matrix }}
+      integration_matrix: ${{ steps.integration.outputs.matrix }}
+    steps:
+      - name: Check out CI scripts
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          sparse-checkout: .github/scripts/ci
+          persist-credentials: false
+      - name: Compute e2e shard matrix
+        id: e2e
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          IS_DRAFT: ${{ github.event.pull_request.draft }}
+          IS_FORK: ${{ github.event.pull_request.head.repo.fork }}
+          NUM_SHARDS: "4"
+        run: bash .github/scripts/ci/e2e-shard-matrix.sh
+      - name: Compute integration matrix
+        id: integration
+        env:
+          EVENT_NAME: ${{ github.event_name }}
+          IS_DRAFT: ${{ github.event.pull_request.draft }}
+          IS_FORK: ${{ github.event.pull_request.head.repo.fork }}
+        run: bash .github/scripts/ci/integration-matrix.sh
+
+  # tests/e2e against the pinned old server — same shards as e2e.yml.
+  backcompat-e2e:
+    name: Backcompat e2e (server ${{ github.event.inputs.server_version || 'latest' }}, shard ${{ matrix.shard_id }}/${{ matrix.num_shards }})
+    needs: setup
+    runs-on: ubuntu-latest
+    # Job-level cap (composite run steps can't set timeout-minutes); mirrors
+    # e2e.yml's ~30-min test budget + setup + old-server build.
+    timeout-minutes: 40
+    strategy:
+      fail-fast: false
+      max-parallel: 4
+      matrix: ${{ fromJSON(needs.setup.outputs.e2e_matrix) }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          # Test the merge result on PRs (matches e2e.yml); fall back to the
+          # dispatched/triggering ref otherwise. fetch-depth 0 so the action
+          # can `git worktree add` the old release tag.
+          ref: ${{ github.event.pull_request.number && format('refs/pull/{0}/merge', github.event.pull_request.number) || github.event.inputs.branch || github.ref }}
+          fetch-depth: 0
+
+      - name: Resolve old server version
+        id: resolve
+        env:
+          SERVER_VERSION_INPUT: ${{ github.event.inputs.server_version }}
+        run: |
+          tag="$SERVER_VERSION_INPUT"
+          if [ -z "$tag" ]; then
+            tag="$(git tag --sort=-v:refname | grep -vi rc | head -1)"
+          fi
+          echo "Resolved old server tag: $tag"
+          echo "tag=$tag" >> "$GITHUB_OUTPUT"
+
+      - name: Run e2e suite against old server
+        uses: ./.github/actions/e2e-run
+        with:
+          server_version: ${{ steps.resolve.outputs.tag }}
+          shard_id: ${{ matrix.shard_id }}
+          num_shards: ${{ matrix.num_shards }}
+          parallelism: "2"
+          nightly_full: ${{ github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' }}
+
+  # tests/integration against the pinned old server — same harness matrix as
+  # integration.yml.
+  backcompat-integration:
+    name: Backcompat integration (server ${{ github.event.inputs.server_version || 'latest' }}, ${{ matrix.harness }})
+    needs: setup
+    runs-on: ubuntu-latest
+    # Job-level cap (composite run steps can't set timeout-minutes); mirrors
+    # integration.yml's 30-min budget + the old-server build.
+    timeout-minutes: 35
+    strategy:
+      fail-fast: false
+      # openai-agents only (per integration-matrix.sh) — same as the gate.
+      matrix: ${{ fromJSON(needs.setup.outputs.integration_matrix) }}
+    steps:
+      - name: Checkout
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0  # v7.0.0
+        with:
+          ref: ${{ github.event.pull_request.number && format('refs/pull/{0}/merge', github.event.pull_request.number) || github.event.inputs.branch || github.ref }}
+          fetch-depth: 0
+
+      - name: Resolve old server version
+        id: resolve
+        env:
+          SERVER_VERSION_INPUT: ${{ github.event.inputs.server_version }}
+        run: |
+          tag="$SERVER_VERSION_INPUT"
+          if [ -z "$tag" ]; then
+            tag="$(git tag --sort=-v:refname | grep -vi rc | head -1)"
+          fi
+          echo "Resolved old server tag: $tag"
+          echo "tag=$tag" >> "$GITHUB_OUTPUT"
+
+      - name: Run integration suite against old server
+        uses: ./.github/actions/integration-run
+        with:
+          server_version: ${{ steps.resolve.outputs.tag }}
+          harness: ${{ matrix.harness }}
+          model: ${{ matrix.model }}
+          workers: ${{ matrix.workers }}
@@ -302,6 +302,7 @@ markers = [
    "llm_flaky(reruns=2, reruns_delay=1): rerun a nondeterministic real-LLM test on failure, rotating to a different model each attempt (tests/_model_pools.py). Never apply to heavy e2e tests that can hit the CI --timeout=180 cap (rerun + loadscope can crash the shard).",
    "in_process_sessions: run a server integration test with the legacy in-process sessions path instead of sessions-native runner dispatch.",
    "nightly: runs only in the scheduled/dispatch pass of its suite; PR and push runs exclude it via -m 'not nightly'. Remove the marker to promote a burned-in test to the PR gate.",
+    "min_server_version(version): skip this test when the live server under test is older than `version` (PEP 440 release-tuple comparison, so a `.devN` of X satisfies X). Used by the server-version backwards-compat CI (docs/SERVER_VERSION_COMPAT_CI.md); inert in normal runs where the server is current.",
    "mock_only: tests/integration test that only works in mock-LLM mode (no --llm-api-key). Skipped by tests/integration/conftest.py when a real --llm-api-key is supplied (the real-LLM Integration jobs). Use for tests whose mock LLM is scripted with a fixed tool-call sequence — a real LLM cannot reproduce the scripted markers.",
    "visual: UI diff visual-regression snapshot (pytest-playwright-visual-snapshot). Runs only in the pinned-runner gate (.github/workflows/ui-snapshot.yml); the main e2e_ui suite excludes it via -m 'not visual' since it runs on the unpinned ubuntu-latest.",
 ]
@@ -0,0 +1,225 @@
+"""
+Helpers for the server-version backwards-compatibility harness.
+
+See ``docs/SERVER_VERSION_COMPAT_CI.md``. Two concerns:
+
+1. **Server redirect** — in compat mode the ``omnigent.cli server``
+   subprocess is launched from a *different* venv (one holding a pinned,
+   older ``omnigent`` build) than the test process. Driven by
+   ``OMNIGENT_COMPAT_SERVER_PYTHON``.
+2. **Version skip** — resolve the running server's version and enforce
+   ``@pytest.mark.min_server_version(...)`` so tests for features newer
+   than the server-under-test are skipped rather than failing.
+
+Outside compat mode (neither env var set) every function here is inert and
+the test harness behaves exactly as before.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+import tempfile
+
+import httpx
+from packaging.version import Version
+
+# Stable empty directory used as the server subprocess CWD in compat mode
+# (created lazily; see :func:`compat_server_cwd`).
+_compat_cwd: str | None = None
+
+# Interpreter for the SERVER subprocess. Set to a venv python holding the
+# pinned older build; unset in normal runs (use the test process's python).
+COMPAT_SERVER_PYTHON_ENV = "OMNIGENT_COMPAT_SERVER_PYTHON"
+# Version string the workflow pinned (e.g. "0.1.1"). Backstop / cross-check
+# for the skip logic — never used to launch anything.
+COMPAT_SERVER_VERSION_ENV = "OMNIGENT_COMPAT_SERVER_VERSION"
+
+
+# ── Server redirect ────────────────────────────────────────────────────
+
+
+def compat_server_python() -> str | None:
+    """
+    Interpreter the server subprocess should run under, or ``None``.
+
+    :returns: The value of ``OMNIGENT_COMPAT_SERVER_PYTHON`` (a venv python
+        path, e.g. ``"/tmp/server-env/bin/python"``) when compat mode is
+        active, else ``None``.
+    """
+    return os.environ.get(COMPAT_SERVER_PYTHON_ENV) or None
+
+
+def server_executable() -> str:
+    """
+    Interpreter to launch ``omnigent.cli server`` with.
+
+    :returns: The compat interpreter in compat mode, else ``sys.executable``
+        (the test process's own python).
+    """
+    return compat_server_python() or sys.executable
+
+
+def server_pythonpath(repo_root: str | os.PathLike[str]) -> str | None:
+    """
+    ``PYTHONPATH`` value for the server subprocess, or ``None`` to drop it.
+
+    Normally the worktree (*repo_root*) is prepended so the server imports
+    the branch's source rather than a stale installed copy. In compat mode
+    that prepend is **dropped** — otherwise the worktree would shadow the
+    pinned older ``omnigent`` in the compat venv, silently testing main
+    against main.
+
+    :param repo_root: Worktree root to prepend in normal mode, e.g.
+        ``Path("/Users/me/omnigent")``.
+    :returns: ``"<repo_root>:<existing PYTHONPATH>"`` in normal mode;
+        ``None`` in compat mode (caller should omit ``PYTHONPATH`` so the
+        compat venv's site-packages resolves ``omnigent``).
+    """
+    if compat_server_python() is not None:
+        return None
+    existing = os.environ.get("PYTHONPATH", "")
+    return f"{repo_root}{os.pathsep}{existing}"
+
+
+def compat_server_cwd() -> str | None:
+    """
+    Working directory for the server subprocess, or ``None`` to inherit.
+
+    In compat mode the subprocess must **not** run with the worktree as its
+    CWD: ``python -m omnigent.cli`` puts the CWD on ``sys.path[0]``, so the
+    worktree's ``omnigent/`` package would shadow the pinned older install
+    exactly like the ``PYTHONPATH`` prepend would — and CI runs from the repo
+    checkout root, which contains ``omnigent/``. Returning a stable empty
+    directory forces the compat venv's installed ``omnigent`` to resolve.
+
+    :returns: A stable empty directory path in compat mode; ``None`` outside
+        compat mode (inherit the parent's CWD — today's behavior).
+    """
+    global _compat_cwd
+    if compat_server_python() is None:
+        return None
+    if _compat_cwd is None:
+        _compat_cwd = tempfile.mkdtemp(prefix="omnigent-compat-cwd-")
+    return _compat_cwd
+
+
+def apply_server_env(env: dict[str, str], repo_root: str | os.PathLike[str]) -> dict[str, str]:
+    """
+    Set/drop ``PYTHONPATH`` on a server-subprocess env dict for the mode.
+
+    Mutates *env* in place (and returns it) so call sites can pass their
+    fully-built env straight to ``subprocess.Popen``.
+
+    :param env: The server subprocess environment being assembled.
+    :param repo_root: Worktree root (see :func:`server_pythonpath`).
+    :returns: The same dict, with ``PYTHONPATH`` set in normal mode or
+        removed in compat mode.
+    """
+    pythonpath = server_pythonpath(repo_root)
+    if pythonpath is None:
+        env.pop("PYTHONPATH", None)
+    else:
+        env["PYTHONPATH"] = pythonpath
+    return env
+
+
+# ── Version resolution + skip ──────────────────────────────────────────
+
+
+def release_tuple(version: str) -> tuple[int, ...]:
+    """
+    PEP 440 release tuple, ignoring ``.devN`` / ``rc`` / ``.postN`` suffixes.
+
+    Comparing on the release tuple lets a development version of ``X``
+    satisfy ``min_server_version("X")`` — main (e.g. ``0.1.2.dev0``) must
+    run its own just-landed features even though ``0.1.2.dev0 < 0.1.2``
+    under full PEP 440 ordering.
+
+    :param version: A version string, e.g. ``"0.1.2.dev0"`` or ``"0.1.1"``.
+    :returns: The release tuple, e.g. ``(0, 1, 2)`` or ``(0, 1, 1)``.
+    """
+    return Version(version).release
+
+
+def meets_min_server_version(server_version: str, required: str) -> bool:
+    """
+    Whether *server_version* is new enough to run a *required*-gated test.
+
+    :param server_version: The running server's version, e.g. ``"0.1.1"``.
+    :param required: The ``min_server_version`` marker argument, e.g.
+        ``"0.1.2"``.
+    :returns: ``True`` iff the server's release tuple is ``>=`` the
+        required release tuple.
+    """
+    return release_tuple(server_version) >= release_tuple(required)
+
+
+def reconcile_server_version(
+    reported: str | None,
+    override: str | None,
+    *,
+    source: str = "server",
+) -> str:
+    """
+    Combine the ``/api/version`` report with the env backstop into one version.
+
+    Pure decision logic (no I/O) so the precedence/fail-loud rules are
+    unit-testable. ``/api/version`` is the source of truth; the env backstop
+    is used only when the report is missing, and otherwise cross-checked.
+
+    :param reported: Version from ``GET /api/version``, or ``None`` if it
+        couldn't be read.
+    :param override: ``OMNIGENT_COMPAT_SERVER_VERSION`` value, or ``None``.
+    :param source: Base URL (for the error message), e.g.
+        ``"http://localhost:6767"``.
+    :returns: The reconciled server version string, e.g. ``"0.1.1"``.
+    :raises RuntimeError: If the report is missing and no backstop is set, or
+        if the report and backstop release tuples disagree.
+    """
+    if reported is None:
+        if override is not None:
+            return override
+        raise RuntimeError(
+            f"could not read {source}/api/version and no {COMPAT_SERVER_VERSION_ENV} "
+            f"backstop is set"
+        )
+    if override is not None and release_tuple(override) != release_tuple(reported):
+        raise RuntimeError(
+            f"server version mismatch: /api/version reports {reported!r} but "
+            f"{COMPAT_SERVER_VERSION_ENV}={override!r}. The pinned old server may be "
+            f"shadowed by the worktree via PYTHONPATH — see "
+            f"docs/SERVER_VERSION_COMPAT_CI.md."
+        )
+    return reported
+
+
+def _fetch_reported_version(base_url: str) -> str | None:
+    """
+    Read ``GET /api/version``, returning ``None`` if it can't be obtained.
+
+    :param base_url: Live server base URL, e.g. ``"http://localhost:6767"``.
+    :returns: The reported version string, or ``None`` on any HTTP/parse
+        error.
+    """
+    try:
+        return httpx.get(f"{base_url}/api/version", timeout=10).json()["version"]
+    except (httpx.HTTPError, KeyError, ValueError):
+        return None
+
+
+def resolve_server_version(base_url: str) -> str:
+    """
+    Resolve the running server's version (source of truth: ``GET /api/version``).
+
+    Thin I/O wrapper over :func:`reconcile_server_version`. The env backstop
+    ``OMNIGENT_COMPAT_SERVER_VERSION`` covers an unreadable endpoint and
+    cross-checks the report (mismatch → raise; the tripwire for the
+    PYTHONPATH-shadow regression).
+
+    :param base_url: Live server base URL, e.g. ``"http://localhost:6767"``.
+    :returns: The server version string, e.g. ``"0.1.1"``.
+    :raises RuntimeError: See :func:`reconcile_server_version`.
+    """
+    override = os.environ.get(COMPAT_SERVER_VERSION_ENV) or None
+    return reconcile_server_version(_fetch_reported_version(base_url), override, source=base_url)
@@ -42,7 +42,6 @@ import os
 import signal
 import socket
 import subprocess
-import sys
 import tarfile
 import tempfile
 import time
@@ -53,6 +52,8 @@ from pathlib import Path
 import httpx
 import pytest

+from tests._helpers.compat import apply_server_env, compat_server_cwd, server_executable
+
 # Project root — this file lives at tests/_helpers/live_server.py.
 _REPO_ROOT = Path(__file__).resolve().parents[2]

@@ -157,23 +158,21 @@ def start_live_server(
    """
    port = find_free_port()
    harness_env = _compute_harness_env(creds)
-    env = {
-        **os.environ,
-        **harness_env,
-        # Force the subprocess to import from the worktree, not
-        # whatever's installed in the venv. Otherwise a branch with
-        # schema/model changes runs against a stale installed copy
-        # and fails with cryptic "no such column" errors.
-        "PYTHONPATH": f"{_REPO_ROOT}{os.pathsep}{os.environ.get('PYTHONPATH', '')}",
-    }
+    env = {**os.environ, **harness_env}
+    # Force the subprocess to import from the worktree, not whatever's
+    # installed in the venv — otherwise a branch with schema/model changes
+    # runs against a stale installed copy and fails with cryptic "no such
+    # column" errors. In compat mode (OMNIGENT_COMPAT_SERVER_PYTHON set) this
+    # prepend is dropped so the pinned older build in the compat venv wins.
+    apply_server_env(env, _REPO_ROOT)
    log_handle = open(log_path, "w")  # noqa: SIM115 — handle lives for Popen lifetime
    proc = subprocess.Popen(
        [
-            # ``sys.executable`` so the subprocess uses the same
-            # interpreter pytest runs under. Bare ``"python"``
-            # resolves against the subprocess PATH which on macOS
-            # often picks up system Python 2.7 and SyntaxErrors.
-            sys.executable,
+            # The test process's own interpreter normally; in compat mode the
+            # pinned old server's venv python (server_executable()). Never bare
+            # "python" — that resolves against PATH and on macOS can pick up
+            # system Python 2.7 and SyntaxError.
+            server_executable(),
            "-m",
            "omnigent.cli",
            "server",
@@ -185,6 +184,9 @@ def start_live_server(
            str(artifact_dir),
        ],
        env=env,
+        # Compat mode: neutral CWD so the worktree's omnigent/ on sys.path[0]
+        # doesn't shadow the pinned old install. None (inherit) otherwise.
+        cwd=compat_server_cwd(),
        stdout=log_handle,
        stderr=subprocess.STDOUT,
    )
@@ -42,6 +42,13 @@ import pytest
 import yaml

 from omnigent.runner.identity import OMNIGENT_INTERNAL_WS_ORIGIN
+from tests._helpers.compat import (
+    apply_server_env,
+    compat_server_cwd,
+    meets_min_server_version,
+    resolve_server_version,
+    server_executable,
+)
 from tests._model_pools import current_attempt, resolve_model
 from tests.e2e._harness_probes import skip_if_harness_cli_missing
 from tests.e2e.helpers import HEALTH_TIMEOUT_S, POLL_INTERVAL_S, lookup_databricks_host
@@ -65,6 +72,56 @@ def _skip_when_harness_cli_missing(request: pytest.FixtureRequest) -> None:
        skip_if_harness_cli_missing(harness)


+@pytest.fixture(scope="session")
+def server_version(live_server: str) -> str:
+    """Version of the live server under test (source of truth: GET /api/version).
+
+    In the backwards-compat workflow the server is a pinned older build, so
+    this can differ from the installed (test-process) version. See
+    :func:`tests._helpers.compat.resolve_server_version` and
+    ``docs/SERVER_VERSION_COMPAT_CI.md``.
+
+    :param live_server: Base URL of the live server, e.g.
+        ``"http://localhost:54321"``.
+    :returns: The server version string, e.g. ``"0.1.1"``.
+    """
+    return resolve_server_version(live_server)
+
+
+@pytest.fixture(autouse=True)
+def _enforce_min_server_version(request: pytest.FixtureRequest) -> None:
+    """Skip tests marked ``@pytest.mark.min_server_version(X)`` on older servers.
+
+    Resolves :func:`server_version` (and thus requires a live server) when a
+    test carries the marker OR when a compat run is active
+    (``OMNIGENT_COMPAT_SERVER_VERSION`` set). The latter makes the
+    ``/api/version`` ↔ env cross-check (the PYTHONPATH/CWD-shadow tripwire in
+    :func:`resolve_server_version`) fire once per session even before any
+    feature has a marker. In normal runs with no marker, nothing is resolved,
+    so non-server tests are unaffected.
+
+    Comparison is on the PEP 440 release tuple, so a ``.devN`` of ``X``
+    satisfies ``min_server_version("X")``.
+
+    :param request: The pytest request, used to read the marker and lazily
+        resolve the ``server_version`` fixture.
+    """
+    marker = request.node.get_closest_marker("min_server_version")
+    compat_pinned = os.environ.get("OMNIGENT_COMPAT_SERVER_VERSION")
+    if marker is None and not compat_pinned:
+        return
+    # Resolving server_version cross-checks /api/version against the pinned
+    # version and fails loud on a shadow (server running the wrong code).
+    server_ver = request.getfixturevalue("server_version")
+    if marker is None:
+        return
+    if not marker.args:
+        raise pytest.UsageError("min_server_version marker requires a version argument")
+    required = marker.args[0]
+    if not meets_min_server_version(server_ver, required):
+        pytest.skip(f"requires server >= {required}; running {server_ver}")
+
+
 # Agent bundle directories relative to repo root.
 _REPO_ROOT = Path(__file__).resolve().parents[2]
 _CODER_DIR = _REPO_ROOT / "tests" / "resources" / "examples" / "coder"
@@ -537,9 +594,12 @@ def live_server(
    env = {
        **os.environ,
        "OPENAI_API_KEY": llm_api_key,
-        "PYTHONPATH": f"{_REPO_ROOT}{os.pathsep}{os.environ.get('PYTHONPATH', '')}",
        "OMNIGENT_BUILTIN_AGENT_DIRS": str(builtin_sdk_chat_spec),
    }
+    # Prepend the worktree so the server imports the branch's source (see
+    # comment above). Dropped in compat mode so the pinned older server in
+    # the compat venv resolves instead of being shadowed by main.
+    apply_server_env(env, _REPO_ROOT)
    if using_mock_llm and mock_llm_server_url is not None:
        # Mock mode: point all LLM calls at the mock server.
        # The OpenAI SDK appends /responses to the base URL, so
@@ -578,7 +638,10 @@ def live_server(
    # 401s under ``--profile``. Point it at the same gateway the
    # agent executors use so prompt-policy e2e tests can classify.
    server_args = [
-        sys.executable,
+        # Compat-aware: the test process's python normally, the pinned old
+        # server's venv python in compat mode. The runner below stays on
+        # sys.executable (it tracks the test process / client version).
+        server_executable(),
        "-m",
        "omnigent.cli",
        "server",
@@ -632,6 +695,9 @@ def live_server(
            **env,
            "OMNIGENT_RUNNER_TUNNEL_TOKEN": binding_token,
        },
+        # Compat mode: neutral CWD so the worktree omnigent/ doesn't shadow
+        # the pinned old install via sys.path[0]. None (inherit) otherwise.
+        cwd=compat_server_cwd(),
        stdout=log_handle,
        stderr=subprocess.STDOUT,
    )
@@ -894,6 +960,80 @@ def register_inline_agent(
    return name


+def register_dir_agent_with_mock_llm(
+    client: httpx.Client,
+    *,
+    agent_dir: Path,
+    name: str,
+    model: str,
+    mock_llm_base_url: str,
+) -> str:
+    """
+    Register a directory-bundle agent that ships its function tools as
+    Python source under ``tools/python/``, routed at a mock LLM.
+
+    Unlike :func:`register_inline_agent` (a single ``<name>.yaml`` whose
+    tool callables are dotted import paths), this tars *agent_dir* — whose
+    ``tools/python/*.py`` files the server loads by absolute file path from
+    the unpacked bundle (auto-discovered, like the ``archer`` fixture). So
+    the tools resolve on any server version without the server importing
+    the repo's ``tests/`` tree — the server-version backwards-compat failure
+    mode that dotted ``tests.*`` callables hit when the server is isolated.
+
+    The bundle's ``config.yaml`` is stamped per call: ``name`` and
+    ``executor.model`` are overridden and an ``executor.auth`` api-key block
+    is injected so the openai-agents harness hits the mock server.
+
+    :param client: HTTP client pointed at the server.
+    :param agent_dir: Fixture dir with ``config.yaml`` + ``tools/python/*.py``,
+        e.g. ``tests/resources/agents/decorator-tools``.
+    :param name: Agent name; suffixed per rerun attempt like
+        :func:`register_inline_agent` so llm_flaky rotation isn't defeated.
+    :param model: Mock model key (must match the ``configure_mock_llm`` key).
+    :param mock_llm_base_url: Mock server base URL including ``/v1``.
+    :returns: The registered agent name (use the return value, not *name*).
+    """
+    import json as _json
+
+    attempt = current_attempt()
+    if attempt > 0:
+        name = f"{name}-r{attempt}"
+
+    config = yaml.safe_load((agent_dir / "config.yaml").read_text())
+    config["name"] = name
+    executor = config.setdefault("executor", {})
+    executor["model"] = resolve_model(model)
+    executor["auth"] = {
+        "type": "api_key",
+        "api_key": "mock-key",
+        "base_url": mock_llm_base_url,
+    }
+
+    with io.BytesIO() as buf:
+        with tarfile.open(fileobj=buf, mode="w:gz") as tar:
+            cfg_bytes = yaml.dump(config).encode()
+            info = tarfile.TarInfo("config.yaml")
+            info.size = len(cfg_bytes)
+            tar.addfile(info, io.BytesIO(cfg_bytes))
+            # Ship the rest of the bundle (tools/python/*.py, etc.) verbatim;
+            # the stamped config.yaml above replaces the on-disk one.
+            for entry in sorted(agent_dir.rglob("*")):
+                if not entry.is_file() or entry.relative_to(agent_dir) == Path("config.yaml"):
+                    continue
+                tar.add(str(entry), arcname=str(entry.relative_to(agent_dir)))
+        bundle = buf.getvalue()
+
+    resp = client.post(
+        "/v1/sessions",
+        data={"metadata": _json.dumps({})},
+        files={"bundle": ("agent.tar.gz", bundle, "application/gzip")},
+        headers={"Origin": OMNIGENT_INTERNAL_WS_ORIGIN},
+    )
+    if resp.status_code not in (200, 201, 409):
+        raise RuntimeError(f"dir-agent register failed: {resp.status_code} {resp.text[:500]}")
+    return name
+
+
 def build_agent_bundle(
    agent_dir: Path,
    *,
@@ -1602,8 +1742,10 @@ def resume_test_server(
    env = {
        **os.environ,
        "OPENAI_API_KEY": llm_api_key,
-        "PYTHONPATH": f"{_REPO_ROOT}{os.pathsep}{os.environ.get('PYTHONPATH', '')}",
    }
+    # Worktree shadow in normal mode; dropped in compat mode (see the
+    # primary live_server fixture above).
+    apply_server_env(env, _REPO_ROOT)
    if databricks_workspace_host is not None:
        env["OPENAI_BASE_URL"] = f"{databricks_workspace_host}/serving-endpoints"
    # See docstring: an allow-list would reject the CLI's own runner.
@@ -1612,7 +1754,7 @@ def resume_test_server(
    log_handle = open(server_log, "w")  # noqa: SIM115 — lives for the Popen lifetime; closed in finally
    proc = subprocess.Popen(
        [
-            sys.executable,
+            server_executable(),
            "-m",
            "omnigent.cli",
            "server",
@@ -1624,6 +1766,8 @@ def resume_test_server(
            str(artifact_dir),
        ],
        env=env,
+        # Compat mode: neutral CWD (see the primary live_server fixture).
+        cwd=compat_server_cwd(),
        stdout=log_handle,
        stderr=subprocess.STDOUT,
    )
@@ -134,6 +134,13 @@ _ALT_COVERED: frozenset[str] = frozenset(
        #   test.
        "supervisor-terminal-test",
        "sys-terminal-test",
+        # Bundled @tool-source fixtures (config.yaml + tools/python/) loaded by
+        # register_dir_agent_with_mock_llm — covered by the shared tests
+        # test_decorated_tools_e2e.py / test_async_tools_e2e.py /
+        # test_tool_call_policy_e2e.py, not test_example_<name>.py files.
+        "decorator-tools",
+        "async-tools",
+        "tool-call-policy",
        # Skills-filter test fixtures under tests/resources/agents/.
        # Loaded by tests/e2e/test_codex_skills_filter_e2e.py,
        # test_pi_skills_filter_e2e.py, and
@@ -30,6 +30,7 @@ from __future__ import annotations

 import json
 import uuid
+from pathlib import Path
 from typing import Any

 import httpx
@@ -38,11 +39,17 @@ from tests.e2e.conftest import (
    configure_mock_llm,
    create_runner_bound_session,
    poll_session_until_terminal,
-    register_inline_agent,
+    register_dir_agent_with_mock_llm,
    reset_mock_llm,
    send_user_message_to_session,
 )

+# Fixture agent whose @tool functions ship as Python source under tools/python/
+# (auto-discovered, like the archer fixture), so the server loads them by file
+# path from the uploaded bundle on any version — no dependency on the repo's
+# tests/ tree being importable by the server.
+_ASYNC_TOOLS_DIR = Path(__file__).resolve().parents[1] / "resources" / "agents" / "async-tools"
+

 def _final_text(response_body: dict[str, Any]) -> str:
    """
@@ -89,42 +96,12 @@ def test_async_tool_real_llm_e2e(
    model = f"mock-async-single-{uuid.uuid4().hex[:6]}"
    reset_mock_llm(mock_llm_server_url)

-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_ASYNC_TOOLS_DIR,
        name=f"async-tools-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "You are the async-tools test fixture agent. Your only job is to call "
-            "the tools the user names, then report the literal result strings.\n\n"
-            "Tool routing: delayed_echo and boom_async are ASYNC — invoke them "
-            'via sys_call_async(tool="<name>", args="<json>"). count_chars is '
-            "SYNC — call it directly.\n\n"
-            "After an async dispatch, the real result auto-delivers as a system "
-            "message starting with [System: task ...]. Quote the BODY of that "
-            "message (the tool's return value) in your final reply."
-        ),
        mock_llm_base_url=f"{mock_llm_server_url}/v1",
-        extra_config={
-            "tools": {
-                "delayed_echo": {
-                    "type": "function",
-                    "description": "Sleep 2 seconds then echo the label.",
-                    "callable": "tests._fixtures.agents._async_tools.delayed_echo",
-                },
-                "boom_async": {
-                    "type": "function",
-                    "description": "Always raises.",
-                    "callable": "tests._fixtures.agents._async_tools.boom_async",
-                },
-                "count_chars": {
-                    "type": "function",
-                    "description": "Return the character count.",
-                    "callable": "tests._fixtures.agents._async_tools.count_chars",
-                },
-            }
-        },
    )

    # Mock queue: first response dispatches sys_call_async, second
@@ -205,31 +182,12 @@ def test_mixed_sync_and_async_tools_e2e(
    model = f"mock-async-mixed-{uuid.uuid4().hex[:6]}"
    reset_mock_llm(mock_llm_server_url)

-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_ASYNC_TOOLS_DIR,
        name=f"async-mixed-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "You call tools as instructed and report results verbatim.\n"
-            "ASYNC tools: use sys_call_async. SYNC tools: call directly."
-        ),
        mock_llm_base_url=f"{mock_llm_server_url}/v1",
-        extra_config={
-            "tools": {
-                "delayed_echo": {
-                    "type": "function",
-                    "description": "Sleep 2 seconds then echo the label.",
-                    "callable": "tests._fixtures.agents._async_tools.delayed_echo",
-                },
-                "count_chars": {
-                    "type": "function",
-                    "description": "Return the character count.",
-                    "callable": "tests._fixtures.agents._async_tools.count_chars",
-                },
-            }
-        },
    )

    # Turn 1: LLM calls count_chars sync AND sys_call_async for delayed_echo.
@@ -308,26 +266,12 @@ def test_async_tool_failure_surfaces_e2e(
    model = f"mock-async-fail-{uuid.uuid4().hex[:6]}"
    reset_mock_llm(mock_llm_server_url)

-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_ASYNC_TOOLS_DIR,
        name=f"async-fail-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "You call tools as instructed and report results verbatim.\n"
-            "ASYNC tools: use sys_call_async."
-        ),
        mock_llm_base_url=f"{mock_llm_server_url}/v1",
-        extra_config={
-            "tools": {
-                "boom_async": {
-                    "type": "function",
-                    "description": "Always raises.",
-                    "callable": "tests._fixtures.agents._async_tools.boom_async",
-                },
-            }
-        },
    )

    # Turn 1: dispatch boom_async via sys_call_async.
@@ -1,7 +1,8 @@
 """End-to-end tests for the @tool decorator (mock LLM).

 Verifies the full pipeline:
- Agent with ``callable``-style tool declarations is registered inline.
+- Agent ships its @tool functions as Python source in the uploaded bundle
+  (tools/python/, auto-discovered) — loaded by file path on any server.
 - Mock LLM emits tool_calls with the correct arguments.
 - Tools run in the server subprocess; results return through the runner.
 - Mock LLM's follow-up response references the literal output values.
@@ -15,6 +16,7 @@ from __future__ import annotations

 import json
 import uuid
+from pathlib import Path

 import httpx
 import pytest
@@ -23,12 +25,20 @@ from tests.e2e.conftest import (
    configure_mock_llm,
    create_runner_bound_session,
    poll_session_until_terminal,
-    register_inline_agent,
+    register_dir_agent_with_mock_llm,
    reset_mock_llm,
    send_user_message_to_session,
 )
 from tests.e2e.helpers import final_assistant_text

+# Fixture agent whose @tool functions ship as Python source under
+# tools/python/ (auto-discovered, like the archer fixture), so the server
+# loads them by file path from the uploaded bundle on any version — no
+# dependency on the repo's tests/ tree being importable by the server.
+_DECORATOR_TOOLS_DIR = (
+    Path(__file__).resolve().parents[1] / "resources" / "agents" / "decorator-tools"
+)
+

@pytest.mark.flaky(reruns=2, reruns_delay=5)
 def test_word_count_tool_e2e(
@@ -45,28 +55,12 @@ def test_word_count_tool_e2e(
    model = f"mock-wordcount-{uuid.uuid4().hex[:6]}"

    reset_mock_llm(mock_llm_server_url)
-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_DECORATOR_TOOLS_DIR,
        name=f"wordcount-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "You have a word_count tool. When asked to count words, "
-            "call the tool and report the result."
-        ),
        mock_llm_base_url=f"{mock_llm_server_url}/v1",
-        extra_config={
-            "tools": {
-                "word_count": {
-                    "type": "function",
-                    "description": "Count whitespace-delimited words in text.",
-                    "callable": (
-                        "tests.resources.examples.archer.tools.python.word_count.word_count"
-                    ),
-                },
-            },
-        },
    )

    # Turn 1: LLM calls word_count with a 7-word phrase.
@@ -130,38 +124,12 @@ def test_decorated_tools_varied_signatures_e2e(
    model = f"mock-decsig-{uuid.uuid4().hex[:6]}"

    reset_mock_llm(mock_llm_server_url)
-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_DECORATOR_TOOLS_DIR,
        name=f"decsig-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "You have three tools: greet, format_record, compute. "
-            "Call them as instructed and report their literal outputs."
-        ),
        mock_llm_base_url=f"{mock_llm_server_url}/v1",
-        extra_config={
-            "tools": {
-                "greet": {
-                    "type": "function",
-                    "description": "Return a greeting for the given name.",
-                    "callable": ("tests._fixtures.agents._decorator_signatures_tools.greet"),
-                },
-                "format_record": {
-                    "type": "function",
-                    "description": "Format a person record as a one-line string.",
-                    "callable": (
-                        "tests._fixtures.agents._decorator_signatures_tools.format_record"
-                    ),
-                },
-                "compute": {
-                    "type": "function",
-                    "description": ("Multiply value by multiplier and echo the optional note."),
-                    "callable": ("tests._fixtures.agents._decorator_signatures_tools.compute"),
-                },
-            },
-        },
    )

    # Mock queue:
@@ -21,6 +21,7 @@ tool registers, and the policy DENY surfaces as the tool output.
 from __future__ import annotations

 import uuid
+from pathlib import Path
 from typing import Any

 import httpx
@@ -29,13 +30,21 @@ from tests.e2e.conftest import (
    configure_mock_llm,
    create_runner_bound_session,
    poll_session_until_terminal,
-    register_inline_agent,
+    register_dir_agent_with_mock_llm,
    reset_mock_llm,
    send_user_message_to_session,
 )

-# Unique reason string — chosen so its presence proves OUR policy
-# fired, not an incidental denial from another path.
+# Fixture agent: the calculate tool ships as Python source under tools/python/
+# (auto-discovered) and the tool_call DENY policy lives in its config.yaml, so
+# both resolve from the uploaded bundle on any server version (no tests/ dep).
+_TOOL_CALL_POLICY_DIR = (
+    Path(__file__).resolve().parents[1] / "resources" / "agents" / "tool-call-policy"
+)
+
+# Unique reason string — chosen so its presence proves OUR policy fired, not an
+# incidental denial from another path. Must match the `reason:` in
+# tests/resources/agents/tool-call-policy/config.yaml.
 _DENY_REASON = "TOOL_BAN_TEST_SENTINEL_XYZQ"


@@ -79,42 +88,12 @@ def test_tool_call_deny_blocks_callable_tool(
    """
    model = f"mock-toolpolicy-{uuid.uuid4().hex[:6]}"
    reset_mock_llm(mock_llm_server_url)
-    agent_name = register_inline_agent(
+    agent_name = register_dir_agent_with_mock_llm(
        http_client,
+        agent_dir=_TOOL_CALL_POLICY_DIR,
        name=f"toolpolicy-{uuid.uuid4().hex[:6]}",
-        harness="openai-agents",
        model=model,
-        profile="",
-        prompt=(
-            "Use calculate to answer. If the tool output starts with "
-            "'[Denied by policy' or mentions a denial, reply that the "
-            "calculation was denied. Do not retry."
-        ),
-        mock_llm_base_url=(f"{mock_llm_server_url}/v1" if mock_llm_server_url else None),
-        extra_config={
-            "tools": {
-                "calculate": {
-                    "type": "function",
-                    "description": "Evaluate a math expression.",
-                    "callable": ("tests.resources.examples._shared.tool_functions.calculate"),
-                },
-            },
-            "policies": {
-                "deny_calculate_tool": {
-                    "type": "function",
-                    "on": ["tool_call:calculate"],
-                    "function": {
-                        "path": "omnigent.policies.function.make_fixed_action_callable",
-                        "arguments": {
-                            "action": "deny",
-                            "reason": _DENY_REASON,
-                            "on_phases": ["tool_call"],
-                            "on_tools": ["calculate"],
-                        },
-                    },
-                },
-            },
-        },
+        mock_llm_base_url=f"{mock_llm_server_url}/v1",
    )
    configure_mock_llm(
        mock_llm_server_url,
@@ -31,6 +31,7 @@ import pytest
 from tests import _model_pools
 from tests.e2e._harness_probes import skip_if_harness_cli_missing
 from tests.e2e.conftest import (  # noqa: F401  (re-exported pytest fixtures)
+    _enforce_min_server_version,
    create_runner_bound_session,
    databricks_workspace_host,
    http_client,
@@ -40,6 +41,7 @@ from tests.e2e.conftest import (  # noqa: F401  (re-exported pytest fixtures)
    mock_llm_server_url,
    register_inline_agent,
    reset_mock_llm,
+    server_version,
    using_mock_llm,
 )
 from tests.integration.model_selection import resolve_default_model
@@ -0,0 +1,32 @@
+spec_version: 1
+name: async-tools
+description: >
+  Fixture agent for the async-tool e2e suite. Its function tools ship as Python
+  source under tools/python/ (auto-discovered, like the archer fixture) so the
+  server loads them by file path from the unpacked bundle — they resolve on any
+  server version without the server importing the repo's tests/ tree.
+
+# executor.model and an executor.auth api-key block are stamped per-test by
+# register_dir_agent_with_mock_llm. config.harness selects the openai-agents
+# harness (required; without it the runner computes harness="omnigent").
+executor:
+  type: omnigent
+  model: mock-model
+  config:
+    harness: openai-agents
+
+prompt: |
+  You are the async-tools test fixture agent. Your only job is to call
+  the tools the user names, then report the literal result strings.
+
+  Tool routing: delayed_echo and boom_async are ASYNC — invoke them
+  via sys_call_async(tool="<name>", args="<json>"). count_chars is
+  SYNC — call it directly.
+
+  After an async dispatch, the real result auto-delivers as a system
+  message starting with [System: task ...]. Quote the BODY of that
+  message (the tool's return value) in your final reply.
+
+os_env:
+  type: caller_process
+  cwd: .
@@ -0,0 +1,16 @@
+"""boom_async fixture tool (always raises, to exercise the async failure path)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def boom_async() -> str:
+    """
+    Always raise so the failure path of the async pipeline is exercised.
+
+    :raises RuntimeError: Always, with message ``ASYNC_TOOL_BOOM_MARKER``.
+    :returns: Never returns normally.
+    """
+    raise RuntimeError("ASYNC_TOOL_BOOM_MARKER")
@@ -0,0 +1,16 @@
+"""count_chars fixture tool (sync; returns character count)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def count_chars(text: str) -> int:
+    """
+    Return the literal character count of ``text``.
+
+    :param text: Text to measure, e.g. ``"abc"``.
+    :returns: Length of ``text``, e.g. ``3``.
+    """
+    return len(text)
@@ -0,0 +1,23 @@
+"""delayed_echo fixture tool (slow; exercises async dispatch ↔ auto-delivery)."""
+
+from __future__ import annotations
+
+import time
+
+from omnigent_client.tools import tool
+
+
+@tool
+def delayed_echo(label: str) -> str:
+    """
+    Sleep 2s, then echo ``label`` inside an unambiguous marker.
+
+    The delay makes the dispatch -> auto-delivery sequence observable as
+    distinct events; the marker is a distinctive substring so e2e
+    assertions like ``"ECHO_FROM_ASYNC[..." in final_text`` are unambiguous.
+
+    :param label: Text to echo back, e.g. ``"alpha"``.
+    :returns: ``f"ECHO_FROM_ASYNC[{label}]"``.
+    """
+    time.sleep(2)
+    return f"ECHO_FROM_ASYNC[{label}]"
@@ -0,0 +1,26 @@
+spec_version: 1
+name: decorator-tools
+description: >
+  Fixture agent for the @tool-decorator e2e tests. Its function tools ship as
+  Python source under tools/python/ (auto-discovered, like the archer fixture),
+  so the server loads them by file path from the unpacked bundle — they resolve
+  on any server version without the server importing the repo's tests/ tree.
+
+# executor.model and an executor.auth api-key block are stamped per-test by
+# register_dir_agent_with_mock_llm so the openai-agents harness hits the mock
+# LLM server. config.harness is required: without it the runner computes
+# harness="omnigent" and produces no output (see compaction-test fixture).
+executor:
+  type: omnigent
+  model: mock-model
+  config:
+    harness: openai-agents
+
+prompt: |
+  You have these tools: word_count, greet, format_record, compute. When the
+  user asks you to call one of these tools, call the tool instead of answering
+  from memory, then report the tool's literal output.
+
+os_env:
+  type: caller_process
+  cwd: .
@@ -0,0 +1,19 @@
+"""compute tool (e2e fixture, multiple primitive args + default)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def compute(value: int, multiplier: int = 2, note: str = "") -> dict[str, int | str]:
+    """
+    Multiply ``value`` by ``multiplier`` and echo the optional note.
+
+    :param value: Base integer value, e.g. ``5``.
+    :param multiplier: Multiplier (defaults to ``2``).
+    :param note: Optional note to echo back, e.g. ``"hi"``.
+    :returns: ``{"product": value * multiplier, "note": note}``, e.g.
+        ``{"product": 10, "note": ""}``.
+    """
+    return {"product": value * multiplier, "note": note}
@@ -0,0 +1,30 @@
+"""format_record tool (e2e fixture, pydantic BaseModel arg with optional field)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+from pydantic import BaseModel
+
+
+class PersonRecord(BaseModel):
+    """A person record (test fixture)."""
+
+    name: str
+    age: int
+    email: str | None = None
+
+
+@tool
+def format_record(record: PersonRecord) -> str:
+    """
+    Format a person record as a one-line string.
+
+    :param record: The person record to format, e.g.
+        ``PersonRecord(name="Bob", age=42)``.
+    :returns: ``"Person(name=..., age=...[, email=...])"``, e.g.
+        ``"Person(name=Bob, age=42)"``.
+    """
+    parts = [f"name={record.name}", f"age={record.age}"]
+    if record.email is not None:
+        parts.append(f"email={record.email}")
+    return "Person(" + ", ".join(parts) + ")"
@@ -0,0 +1,16 @@
+"""Greet tool (e2e fixture, primitive str arg)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def greet(name: str) -> str:
+    """
+    Return a greeting for the given name.
+
+    :param name: The name to greet, e.g. ``"Alice"``.
+    :returns: ``f"Hello, {name}!"``, e.g. ``"Hello, Alice!"``.
+    """
+    return f"Hello, {name}!"
@@ -0,0 +1,16 @@
+"""Local word-count tool (e2e fixture, primitive str arg)."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def word_count(text: str) -> dict[str, int]:
+    """
+    Count whitespace-delimited words in ``text``.
+
+    :param text: Text to count, e.g. ``"one two three"``.
+    :returns: A JSON-serializable dict, e.g. ``{"word_count": 3}``.
+    """
+    return {"word_count": len(text.split())}
@@ -0,0 +1,44 @@
+spec_version: 1
+name: tool-call-policy
+description: >
+  Fixture agent for the tool_call DENY-policy e2e test. The calculate tool
+  ships as Python source under tools/python/ (auto-discovered) so the server
+  loads it by file path from the unpacked bundle on any server version — no
+  dependency on the repo's tests/ tree. The DENY policy below uses an omnigent
+  built-in policy callable (resolves on any server).
+
+# executor.model and an executor.auth api-key block are stamped per-test by
+# register_dir_agent_with_mock_llm. config.harness selects openai-agents.
+executor:
+  type: omnigent
+  model: mock-model
+  config:
+    harness: openai-agents
+
+prompt: |
+  Use calculate to answer. If the tool output starts with '[Denied by policy'
+  or mentions a denial, reply that the calculation was denied. Do not retry.
+
+# tool_call:calculate DENY. In the config.yaml dir-bundle form, policies live
+# under guardrails: — omnigent.spec.parser reads guardrails.policies and ignores
+# a top-level policies: block (unlike the single-YAML form). The reason sentinel
+# must stay in sync with _DENY_REASON in tests/e2e/test_tool_call_policy_e2e.py.
+guardrails:
+  policies:
+    deny_calculate_tool:
+      type: function
+      on:
+        - tool_call:calculate
+      function:
+        path: omnigent.policies.function.make_fixed_action_callable
+        arguments:
+          action: deny
+          reason: TOOL_BAN_TEST_SENTINEL_XYZQ
+          on_phases:
+            - tool_call
+          on_tools:
+            - calculate
+
+os_env:
+  type: caller_process
+  cwd: .
@@ -0,0 +1,28 @@
+"""calculate fixture tool — safe basic-arithmetic evaluation."""
+
+from __future__ import annotations
+
+from omnigent_client.tools import tool
+
+
+@tool
+def calculate(expression: str) -> str:
+    """
+    Safely evaluate a basic arithmetic expression and return the result.
+
+    Only supports basic arithmetic (digits and ``+-*/().%`` and spaces) for
+    safety; other characters are rejected.
+
+    :param expression: Arithmetic expression, e.g. ``"6 + 6"``.
+    :returns: The result as a string, e.g. ``"12"``, or an error string.
+    """
+    allowed = set("0123456789+-*/().% ")
+    if not all(c in allowed for c in expression):
+        return (
+            "Error: expression contains disallowed characters. Only basic arithmetic is supported."
+        )
+    try:
+        result = eval(expression, {"__builtins__": {}}, {})
+    except Exception as exc:
+        return f"Error evaluating '{expression}': {exc}"
+    return str(result)
@@ -0,0 +1,106 @@
+"""
+Unit tests for the server-version backwards-compat helpers
+(:mod:`tests._helpers.compat`). Pure logic only — no live server.
+
+See ``docs/SERVER_VERSION_COMPAT_CI.md``.
+"""
+
+from __future__ import annotations
+
+import os
+import sys
+
+import pytest
+
+from tests._helpers import compat
+
+
+@pytest.mark.parametrize(
+    ("version", "expected"),
+    [
+        ("0.1.1", (0, 1, 1)),
+        ("0.1.2.dev0", (0, 1, 2)),
+        ("0.1.2rc1", (0, 1, 2)),
+        ("1.2.3.post4", (1, 2, 3)),
+        ("2.0", (2, 0)),
+    ],
+)
+def test_release_tuple_ignores_suffixes(version: str, expected: tuple[int, ...]) -> None:
+    assert compat.release_tuple(version) == expected
+
+
+@pytest.mark.parametrize(
+    ("server", "required", "expected"),
+    [
+        # Dev version of X must satisfy a feature gated on X (the whole
+        # reason we compare release tuples, not full PEP 440 ordering).
+        ("0.1.2.dev0", "0.1.2", True),
+        # Equal releases.
+        ("0.1.2", "0.1.2", True),
+        # Newer server runs older-gated features.
+        ("0.2.0", "0.1.2", True),
+        # Old server skips a newer feature.
+        ("0.1.1", "0.1.2", False),
+        ("0.1.2.dev0", "0.1.3", False),
+    ],
+)
+def test_meets_min_server_version(server: str, required: str, expected: bool) -> None:
+    assert compat.meets_min_server_version(server, required) is expected
+
+
+@pytest.mark.parametrize(
+    ("reported", "override", "expected"),
+    [
+        # /api/version is source of truth.
+        ("0.1.1", None, "0.1.1"),
+        # Backstop used only when the report is missing.
+        (None, "0.1.1", "0.1.1"),
+        # Agreement (dev vs final of the same release counts as agreeing).
+        ("0.1.2.dev0", "0.1.2", "0.1.2.dev0"),
+        ("0.1.1", "0.1.1", "0.1.1"),
+    ],
+)
+def test_reconcile_server_version_ok(
+    reported: str | None, override: str | None, expected: str
+) -> None:
+    assert compat.reconcile_server_version(reported, override) == expected
+
+
+def test_reconcile_server_version_disagreement_raises() -> None:
+    # The PYTHONPATH-shadow tripwire: report and pinned version differ.
+    with pytest.raises(RuntimeError, match="version mismatch"):
+        compat.reconcile_server_version("0.1.2.dev0", "0.1.1")
+
+
+def test_reconcile_server_version_unreadable_without_backstop_raises() -> None:
+    with pytest.raises(RuntimeError, match="could not read"):
+        compat.reconcile_server_version(None, None, source="http://localhost:6767")
+
+
+def test_server_redirect_inert_without_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.delenv(compat.COMPAT_SERVER_PYTHON_ENV, raising=False)
+    monkeypatch.delenv("PYTHONPATH", raising=False)
+    assert compat.compat_server_python() is None
+    assert compat.server_executable() == sys.executable
+    # Inherit CWD (None) outside compat mode.
+    assert compat.compat_server_cwd() is None
+    # Normal mode prepends the worktree root to PYTHONPATH.
+    env: dict[str, str] = {}
+    compat.apply_server_env(env, "/repo/root")
+    assert env["PYTHONPATH"].startswith(f"/repo/root{os.pathsep}")
+
+
+def test_server_redirect_active_with_env(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.setenv(compat.COMPAT_SERVER_PYTHON_ENV, "/srv-venv/bin/python")
+    assert compat.compat_server_python() == "/srv-venv/bin/python"
+    assert compat.server_executable() == "/srv-venv/bin/python"
+    # Compat mode drops the worktree prepend so the pinned install resolves.
+    env = {"PYTHONPATH": "/repo/root:/preexisting"}
+    compat.apply_server_env(env, "/repo/root")
+    assert "PYTHONPATH" not in env
+    # Compat mode runs the server from a neutral dir that does NOT contain an
+    # omnigent/ package (else CWD on sys.path[0] would shadow the old install).
+    cwd = compat.compat_server_cwd()
+    assert cwd is not None
+    assert os.path.isdir(cwd)
+    assert not os.path.exists(os.path.join(cwd, "omnigent"))
Author	SHA1	Message	Date
dbczumar	566087451b	fix(test): nest tool-call-policy under guardrails.policies The config.yaml dir-bundle parser (omnigent.spec.parser) reads policies from guardrails.policies and ignores a top-level policies: block — so the converted fixture's DENY policy never loaded (spec.guardrails was None) and calculate ran (tool output '12') instead of being denied. The inline single-YAML form the test used before accepts top-level policies:, which masked the difference. Verified: parse() now loads deny_calculate_tool under guardrails, and the make_fixed_action_callable builtin denies tool_call:calculate with the sentinel (allows other tools/phases). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 16:42:28 -07:00
dbczumar	fa9a56084e	tests: allowlist bundled-tool fixture agents in coverage-sync The 3 new tests/resources/agents/ fixtures (decorator-tools, async-tools, tool-call-policy) are covered by shared e2e tests, not test_example_<name>.py, so add them to _ALT_COVERED (test_every_agent_has_a_dedicated_test_file). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 16:19:27 -07:00
dbczumar	de6b09dee3	Drop docs/SERVER_VERSION_COMPAT_CI.md from the PR Untracked (kept on disk) — not part of the merge per request. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 15:59:26 -07:00
dbczumar	1f1ac1b3bb	Pre-merge prep for server-compat: ruff format + dispatch/nightly-only triggers - ruff format the new test/fixture/helper code (ruff check passed locally but format was not run, so pre-commit's ruff-format reformatted them in CI). - server-compat.yml: drop the temporary pull_request trigger (validation done) and set the schedule to every 4 hours (cron 0 /4 * *). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 15:58:08 -07:00
dbczumar	d70040182e	e2e: ship async-tools + tool_call-policy tool source in the bundle, not tests/ callables Same backcompat fix as the decorated-tools tests: register_inline_agent declared function tools as dotted callables into the repo's tests/ tree, which 400 on the server-version-compat run (the isolated old server can't import tests/). - test_async_tools_e2e.py: new fixture tests/resources/agents/async-tools/ (config.yaml + tools/python/{delayed_echo,boom_async,count_chars}.py with @tool); all 3 register calls use register_dir_agent_with_mock_llm. - test_tool_call_policy_e2e.py: new fixture tests/resources/agents/tool-call-policy/ (config.yaml carries the tool_call:calculate DENY policy verbatim + tools/python/ calculate.py); register call uses register_dir_agent_with_mock_llm. tests/e2e/omnigent/test_run_omnigent_policy_enforcement.py is intentionally NOT converted: it runs 'omnigent run' in a subprocess with cwd=repo_root (so tests/ is importable) and never touches the compat-redirected live_server, so it does not 400 on backcompat. Verified against an isolated v0.1.1 server (cannot import tests/): both fixtures discover their tools and POST /v1/sessions -> 201 (was 400); the tool_call-policy bundle resolves both the calculate tool and the make_fixed_action_callable policy. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 15:27:19 -07:00
dbczumar	2beff228fe	e2e: ship decorated-tool source in the bundle (archer pattern), not tests/ callables test_decorated_tools_e2e registered agents whose function tools were dotted callables into the repo's tests/ tree (tests._fixtures... / tests.resources...). On the server-version-compat run the old server is isolated and can't import tests/, so bundle-load failed with HTTP 400 'function-type tool has no resolved callable'. That's a test shortcut, not a product break: a real agent ships its tool code IN the bundle. - New fixture tests/resources/agents/decorator-tools/ (config.yaml + tools/python/ {word_count,greet,format_record,compute}.py with @tool), mirroring the archer fixture: executor.type=omnigent + config.harness=openai-agents + os_env caller_process, tools auto-discovered and loaded by file path from the bundle. - New helper register_dir_agent_with_mock_llm: tars the dir, stamps name + executor.model + an executor.auth mock-LLM block, uploads. Keeps the openai-agents + mock-LLM flow and the mock scripting/assertions unchanged. - Both tests now load tools from the uploaded bundle, so they run on any server version with no tests/ dependency. Verified against an isolated v0.1.1 server (cannot import tests/): POST /v1/sessions -> 201 (was 400); the 4 tools discover and execute (greet->Hello Alice, compute(5)->product 10, word_count->3). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 15:16:35 -07:00
dbczumar	95a7b06ab8	Keep server-compat.yml PR trigger for backcompat triage on the PR Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 13:40:42 -07:00
dbczumar	def55e0206	Remove temporary PR trigger from server-compat.yml Backcompat validated on the PR; restore dispatch/nightly-only triggers. The jobs reuse the gates' composite actions + matrix scripts, so a manual dispatch (or the nightly schedule) runs them once this lands on main. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 13:31:40 -07:00
dbczumar	e786d56ead	Backcompat reuses the gates' matrix scripts (no hardcoded harness list) The backcompat-integration job hardcoded a stale 3-harness matrix (claude-sdk/openai-agents/codex) copied from the pre-mock workflow. But the real integration gate runs only openai-agents — claude-sdk/codex reject the mock LLM's 'mock-model' and were removed (see integration-matrix.sh). So the backcompat job ran two legs the gate never runs, failing on that known reason (noise, not a compat signal). Add a setup job that computes BOTH matrices from the same scripts the gates use (e2e-shard-matrix.sh / integration-matrix.sh); backcompat-e2e and backcompat-integration consume them. Now backcompat runs exactly the shards/legs the gate runs per event, with no hardcoded list to drift. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 13:18:35 -07:00
dbczumar	2000deb826	Parameterize e2e/integration run logic via composite actions; backcompat reuses them Root cause of the flaky maiden backcompat run: server-compat.yml mirrored the OLD real-LLM e2e.yml, but main migrated e2e/integration to the in-process mock LLM. Fix the drift at the source. - Add .github/actions/e2e-run and .github/actions/integration-run composite actions holding the exact run steps (mock LLM), with an optional server_version input that builds the pinned old server + redirects the server subprocess to it. - e2e.yml / integration.yml now call the actions (no server_version) — same steps, same job names (E2E Tests (shard ..) / Integration (..)) so the Merge Ready required gate is unaffected. Composite (not reusable workflow) to preserve those check names. - server-compat.yml: clearly-labeled backcompat-e2e + backcompat-integration jobs call the SAME actions with server_version set. Full matrix (mock LLM is free of gateway cost), no drift from the gates. - Move the per-step timeout to job level (composite steps can't set it). REVERT before merge: the temporary pull_request trigger on server-compat.yml (lets the backcompat jobs run on this PR; backcompat is dispatch/nightly only). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 13:02:59 -07:00
dbczumar	2bfea1ef63	TEMP: enable server-compat.yml on PR as a smoke (REVERT before merge) workflow_dispatch needs the file on the default branch, which it isn't until #896 merges. Add a pull_request trigger + trim to one e2e shard and one integration leg so the compat harness actually executes on Actions (build old server from tag -> redirect -> run suite). Reverted before merge. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 12:07:27 -07:00
dbczumar	0b161ef2dc	Merge origin/main into server-version-compat-ci Resolve conflicts: - Version files (pyproject x3, uv.lock): take main's 0.2.0.dev0. #896's 0.1.2.dev0 bump is obsolete — main already sorts ahead of release tags, which was the bump's only purpose. The min_server_version skip compares release tuples, so 0.2.0.dev0 works unchanged. - pyproject markers: keep min_server_version alongside main's new mock_only / visual markers. - tests/e2e/conftest.py: keep main's mock-LLM env branch AND the compat apply_server_env() call; both omnigent.cli server spawns use server_executable() + cwd=compat_server_cwd(); runner stays on sys.executable. - tests/integration/conftest.py: re-export server_version alongside main's reset_mock_llm / using_mock_llm. Brings the branch 567 commits current; fixes the stale-branch CodeQL 'Analyze (rust)' failure (branch predated the rust sidecar source). Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-22 12:02:54 -07:00
dbczumar	b69de69673	Add server-version backwards-compat CI harness Run main's network suites (e2e + integration) against a pinned older server to catch backwards-incompatible server changes. - Redirect the server subprocess to a pinned old build via OMNIGENT_COMPAT_SERVER_PYTHON: swap interpreter, drop the worktree PYTHONPATH prepend AND neutralize CWD (both shadow sys.path). Runner stays on main (tracks the client/test version). - min_server_version marker + server_version fixture/guard. /api/version is source of truth; OMNIGENT_COMPAT_SERVER_VERSION is a backstop and a shadow tripwire (fail loud on disagreement). Release-tuple comparison so a .devN of X satisfies min_server_version(X). - Bump dev version to 0.1.2.dev0 across the 3 packages + uv.lock so /api/version sorts ahead of released tags. - server-compat.yml workflow (compat-e2e sharded + compat-integration per-harness), building the old server from its git tag into a venv. - docs/SERVER_VERSION_COMPAT_CI.md spec; tests/test_server_compat.py. Signed-off-by: dbczumar <corey.zumar@databricks.com>	2026-06-20 16:00:12 -07:00