github-featured/chopratejas--headroom
2
0
Fork 0
Code Issues Pull Requests Actions 3 Deployments Models Agent Notes Packages Projects Releases Wiki Activity
2,630 Commits 405 Branches 169 Tags
fix/gitpython-3158
Commit Graph

71 Commits

Author SHA1 Message Date
Tejas Chopra cbb950a441 ci(governance): require a Conventional Commit PR title (#3063) 
## Description

The repo squash-merges, so the PR title — not the commits inside the PR
— becomes the commit subject on `main`. Nothing validated it.

`commitlint` (`ci.yml:429`) lints a PR's *commits* and therefore cannot
catch this by construction: a PR with clean conventional commits and a
prose title passes CI and then lands a prose subject on `main`.

That is how `31452426` landed:

```
Unify savings attribution across stats, perf, metrics, and dashboard (#2976)
```

release-please cannot parse it — `unexpected token ' ' at 1:6`, because
`Unify` is five characters and position six is a space where the parser
needs `(`, `!` or `:`. The change is silently dropped from the
changelog.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update

## Changes Made

- Added `COMMIT_TYPES` and `TITLE_RE` to `scripts/pr-governance.py`,
matching `.commitlintrc.json`'s `type-enum`.
- Added a title check to `validate_pull_request`, reported through the
existing governance comment.
- Added a test asserting `COMMIT_TYPES` equals `.commitlintrc.json`'s
`type-enum`, so the two gates cannot drift apart.
- Gave the `_event` test helper the `title` field a real `pull_request`
payload always carries.

## Testing

- [x] Unit tests pass
- [x] Linting passes (ruff check + format)
- [ ] Type checking passes — N/A (script + test only)
- [x] New tests added for new functionality

### Test Output

```text
$ .venv/bin/python -m pytest scripts/tests/test_pr_governance.py -q
12 passed in 0.02s
```

Against the parent commit:

```text
FAILED test_validate_pull_request_rejects_non_conventional_title
FAILED test_validate_pull_request_rejects_empty_and_typeless_titles
FAILED test_commit_types_match_commitlint_config
3 failed, 9 passed
```

## Real Behavior Proof

- Environment: macOS 15 (darwin 25.4.0), Python 3.12.13,
`scripts/pr-governance.py` loaded directly.
- Exact command / steps: ran `TITLE_RE` against the titles of **all 117
pull requests opened in this repository between 2026-08-10 and
2026-08-16**, pulled with `gh pr list --json title`.
- Observed result: exactly one title is flagged — `#2976`, `Unify
savings attribution across stats, perf, metrics, and dashboard`, the one
that jammed the release. Zero false positives across the other 116,
including every Dependabot `deps: bump ...` title, `chore: release
main`, and scoped forms like `fix(proxy/anthropic): ...`.
- Not tested: the check running inside a live `pull_request_target`
event on a GitHub runner.

## Runtime Rollout Safety

- Rollout-managed feature(s): none.
- Minimum rollout channel: N/A.
- Stable/default behavior changed: yes — a PR with a non-conventional
title now gets the `status: needs author action` label and a governance
comment.
- Kill switch / disable path: revert; the check is not independently
configurable.
- Unsafe override required: none.
- Qualification impact: none.
- Rollback path: revert this commit.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes

## Additional Notes

The check lives in `pr-governance.py` rather than `ci.yml` for two
reasons:

1. `ci.yml`'s `pull_request` trigger has no `edited` type, so a
corrected title would never be re-checked.
2. Its `paths-ignore` skips docs-only PRs, which still squash-merge a
subject onto `main`.

`pr-health.yml` already triggers on `edited` and reports through the
same governance comment the author is reading anyway.

Bot PRs keep their existing exemption — Dependabot and release-please
titles are already conventional, and the early return for `is_bot_pr` is
untouched.

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
 2026-08-16 19:05:36 -07:00
Abhay Singh ddd9f76729 fix(install): stop the PowerShell installer leaking temp dirs into the real user PATH (#2985) 
## Description

`scripts/install.ps1` persists the install directory to the user's PATH
through `Ensure-PathEntry`, which calls
`[Environment]::SetEnvironmentVariable('Path', ..., 'User')`. That value
lives in the `HKCU\Environment` registry key, so it is **not** scoped by
a `HOME` / `USERPROFILE` override.


`tests/test_install/test_native_installers.py::test_powershell_native_installer_supports_persistent_docker_lifecycle`
runs that real installer against a `tmp_path` fake home. Every run
therefore prepended the test's throwaway shim directory to the
developer's actual, persistent user PATH -- and it stayed there after
the test finished. The entries accumulate one per run, ahead of the real
install dir; and since the installer also drops
`headroom.ps1`/`headroom.cmd` into that dir, `headroom` in a fresh shell
could then resolve to a leftover wrapper from a deleted temp directory
(#2970).

## Fix

Make the persistence scope configurable via
`HEADROOM_INSTALL_PATH_SCOPE`, defaulting to `'User'` so production
behavior is unchanged:

```powershell
$scope = if ($env:HEADROOM_INSTALL_PATH_SCOPE) { $env:HEADROOM_INSTALL_PATH_SCOPE } else { 'User' }
$currentPath = [Environment]::GetEnvironmentVariable('Path', $scope)
...
[Environment]::SetEnvironmentVariable('Path', ($newPath -join ';'), $scope)
```

The installer tests (`_build_env`) set
`HEADROOM_INSTALL_PATH_SCOPE=Process`, so the PATH update stays in the
spawned PowerShell process (discarded when it exits) instead of writing
to the registry.

Fixes #2970

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature
- [ ] Breaking change
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `scripts/install.ps1` (`Ensure-PathEntry`): read/write the PATH via
`$env:HEADROOM_INSTALL_PATH_SCOPE` (default `'User'`).
- `tests/test_install/test_native_installers.py`: `_build_env` sets
`HEADROOM_INSTALL_PATH_SCOPE=Process` for every installer invocation;
add a Windows-only
`test_powershell_installer_does_not_leak_into_user_path` asserting the
real User PATH entry count is unchanged across an installer run.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] New test added

### Test Output

```text
tests/test_install/test_native_installers.py -k does_not_leak_into_user_path  1 passed
# uvx ruff@0.15.22 check tests/test_install/test_native_installers.py -> All checks passed!
```

## Real Behavior Proof

- Environment: Windows 11, Windows PowerShell 5.1, Python 3.12.11,
project venv, pytest 9.1.1, ruff 0.15.22 via uvx.
- Exact command / steps: recorded the real user PATH entry count
(`([Environment]::GetEnvironmentVariable('Path','User') -split
';').Count` = 27), ran the PowerShell installer test with the fix, then
re-read the count: still 27 -- no leak. The new
`test_powershell_installer_does_not_leak_into_user_path` formalizes this
(before == after).
- Observed result: running the installer test suite no longer mutates
the developer's persistent user PATH; production installs still persist
to `'User'` as before.
- Not tested: the sibling
`test_powershell_native_installer_supports_persistent_docker_lifecycle`
fails on my Windows host on an unrelated `trusted_cidrs`
dashboard-gateway assertion (it fails identically on `main` without this
change, and the whole PowerShell suite is skipped on the Linux CI
runners). This PR does not touch that path.

## Runtime Rollout Safety

- Rollout-managed feature(s): none. This is the native PowerShell
installer script, not a rollout-channel-gated runtime feature.
- Minimum rollout channel: N/A (no rollout-managed behavior).
- Stable/default behavior changed: no. Production installs still persist
PATH to the `User` scope exactly as before; the new
`HEADROOM_INSTALL_PATH_SCOPE` override defaults to `User` and is used
only by the test suite to avoid mutating the developer's persistent
PATH.
- Kill switch / disable path: leave `HEADROOM_INSTALL_PATH_SCOPE` unset
(the default) for the normal `User` behavior.
- Unsafe override required: no.
- Qualification impact: none. Installer-only; no proxy runtime path is
touched.
- Rollback path: revert this PR; the installer returns to writing the
`User` PATH unconditionally.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`: it is generated by
release-please from my Conventional Commit PR title

## Additional Notes

The scope override defaults to `'User'`, so nothing changes for real
installs. It doubles as an escape hatch for any environment (CI images,
ephemeral containers) that must not touch the persistent user PATH.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
 2026-08-16 15:05:04 -07:00
JD Davis 3077ac81e8 feat: add deterministic runtime rollout controls (#1490) 
## Description

Establish one centrally resolved, observable, deterministic, versioned
runtime rollout-control mechanism for Headroom. Runtime rollout controls
which behaviors an already-built artifact may expose; it does not select
or qualify a Headroom release/version.

## Type of Change

- [x] New feature (non-breaking change that adds functionality)
- [x] Bug fix (non-breaking change that fixes rollout enforcement
regressions)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `RolloutChannel`, `HEADROOM_ROLLOUT_CHANNEL`,
`--rollout-channel`, and a versioned immutable `RolloutSnapshot` shared
by Python configuration boundaries.
- Added schema/policy versions, canonical registry and snapshot SHA-256
identities, per-feature decision reasons, disable precedence, unsafe
qualification poisoning, strict CLI validation, and fail-closed
environment handling.
- Added `headroom rollout status --json`, Python `/stats.rollout`, and
Rust `/rollout/status` runtime provenance.
- Added equivalent Rust snapshot semantics and shared Python/Rust policy
vectors while retaining language-specific feature registries.
- Enforced rollout policy at alternate Python server composition roots
so `HEADROOM_READ_MATURATION=1` cannot bypass its beta gate.
- Preserved typed rollout snapshots across multi-worker serialization
with schema, policy, registry, snapshot-digest, type, and feature-name
validation.
- Made loopback runtime output-shaper updates replace the immutable
snapshot atomically for request readers, retain explicit request/disable
provenance, preserve channel and kill-switch precedence, invalidate
cached stats, and return the effective rollout decision.
- Made `headroom learn --verbosity --apply` report a channel-blocked
update instead of claiming the shaper is live.
- Made explicit CLI feature flags fail loudly when their current channel
blocks them.
- Made persistent interceptor installation select canary automatically,
or reject an explicitly insufficient channel unless the break-glass
override is set.
- Updated architecture, proxy, rollout, learn, and output-shaper
documentation with required channels and hot-reload semantics.

## Testing

- [x] Unit tests pass
- [x] Linting passes (`ruff check .` and `ruff format --check .`)
- [x] Type checking passes (`mypy headroom --ignore-missing-imports`)
- [x] New regression tests added for every corrected behavior
- [x] Rust tests and production-target Clippy pass
- [x] Documentation build passes

### Test Output

```text
Focused rollout coverage suite
57 passed; headroom.rollout + rollout CLI: 98% coverage

Affected proxy/rollout/transform/governance suites
222 passed; 0 failed

Final changed regression suites
100 passed; 0 failed

Cross-module hot-reload isolation regression
6 passed; 0 failed

cargo test -p headroom-core -p headroom-proxy --quiet
headroom-core: 924 passed; 1 ignored
headroom-proxy and integration suites: all passed

cargo clippy -p headroom-core -p headroom-proxy --lib --bins -- -D warnings
cargo fmt --all -- --check
ruff check .
ruff format --check .
mypy headroom --ignore-missing-imports
git diff --check
All passed

cd docs && npm run build
Compiled successfully; 164 static pages generated
```

The unsharded Windows-only CI selection exposed unrelated baseline
failures, principally the existing `sqlite:///C:\\...` URL parser
producing an invalid `\\C:\\...` path. At commit `8e793a80`, all 52
completed GitHub checks passed; the only other conclusions are expected
skips and superseded governance jobs.

## Real Behavior Proof

- **Environment:** Windows checkout on Python 3.13.3 and the current
Rust workspace, based on upstream `main` at `93f2d7a2`.
- **Exact command / steps:** Exercised canary and beta feature requests
through CLI status, Python `/stats.rollout`, Rust `/rollout/status`,
multi-worker payload round trips, loopback `/admin/runtime-env`, real
proxy request shaping before/after hot reload, installer manifest
generation, and shared Python/Rust policy vectors.
- **Observed result:** Stable blocks unstable requests; disable wins
over explicit/default/legacy/unsafe paths; unsafe state reports
`qualification_eligible=false`; worker handoff rejects tampering;
running output shaping changes only when the effective beta policy
permits it; explicit blocked flags fail with actionable diagnostics.
- **Not tested:** Live production traffic requiring provider
credentials, or future artifact qualification/promotion automation
(intentionally out of scope).

## Runtime Rollout Safety

- **Rollout-managed features:** Python `tool_result_interceptors`,
`proxy_output_shaper`, `read_maturation`; Rust `native_bedrock`,
`openai_responses_streaming`, `canary_probe`.
- **Minimum rollout channel:** Registry-defined per feature; process
default is `stable`.
- **Stable/default behavior changed:** No unstable feature becomes
enabled by default. Explicit blocked CLI flags now fail instead of
silently doing nothing.
- **Kill switch / disable path:**
`HEADROOM_DISABLE_FEATURES=<comma-separated feature names>`; explicit
disable has highest precedence, including over the unsafe override.
- **Unsafe override required:** No.
`HEADROOM_UNSAFE_ALLOW_UNSTABLE_FEATURES=1` is break-glass only and
makes qualification evidence ineligible.
- **Qualification impact:** Adds machine-readable policy/snapshot
identities and eligibility; does not implement qualification itself.
- **Rollback path:** Set the named disable list for operational
rollback, lower the channel, or revert this PR.

## Review Readiness

- [x] I have performed a full diff review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented hard-to-understand areas
- [x] I have made corresponding documentation changes
- [x] My changes generate no new warnings
- [x] I added tests that reproduce and prevent every regression fixed
during review
- [x] New and existing affected tests pass locally
- [x] I did **not** edit `CHANGELOG.md`; release-please generates it
from the Conventional Commit PR title

## Additional Notes

Out of scope: artifact candidates, benchmark orchestration,
qualification manifests/gates, promotion automation, release branches,
publication guards, and release-risk classification. Those workflows can
consume the rollout registry digest, runtime snapshot digest, decision
reasons, and qualification eligibility through supported black-box
interfaces.

---------

Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
Co-authored-by: JD Davis <jd@JDH-AIR-00.local>
 2026-08-12 23:16:54 -05:00
Abhay Singh fc5c4e239c fix(install): don't crash the PowerShell installer when $PROFILE is unset (#2469) 
## Description

The PowerShell installer (`scripts/install.ps1`) crashes at the very end
on any machine where PowerShell cannot resolve the current user's
profile path.

`Ensure-ProfileBlock` locates the profile with:

```powershell
$profileDir = Split-Path -Parent $PROFILE
```

`$PROFILE` is an empty string when PowerShell cannot compute the profile
path for the current user, which happens for a fresh account with no
Documents folder yet, a service or CI context, or a redirected profile.
`Split-Path -Parent ''` then throws:

```
Split-Path : Cannot bind argument to parameter 'Path' because it is an empty string.
```

Because the script runs under `$ErrorActionPreference = 'Stop'`, that
terminates the whole installer with a non-zero exit, even though it
happens after the `headroom` wrapper and the persistent User PATH entry
were already written. The user sees a scary Split-Path error and assumes
the install failed.

## Fix

Skip the profile convenience block when `$PROFILE` is empty and log why.
`Ensure-PathEntry` already persists the User PATH for new sessions, so
the only thing skipped is auto-refreshing PATH inside the current
profile file, which does not exist in that environment anyway.
Well-behaved environments with a real `$PROFILE` are unchanged.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `scripts/install.ps1`: early-return from `Ensure-ProfileBlock` with an
informational message when `$PROFILE` is null or empty, before the
`Split-Path` call.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
$ python -m pytest tests/test_install/test_native_installers.py -q
1 passed, 1 skipped

# The PowerShell lifecycle test was failing on main before this change and now passes:
$ python -m pytest "tests/test_install/test_native_installers.py::test_powershell_native_installer_supports_persistent_docker_lifecycle" -q
1 passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.12, Windows PowerShell 5.1, project
venv (`uv sync --extra proxy`), pytest in the venv.
- Exact command / steps: ran `install.ps1` under a temp `USERPROFILE`
with no Documents folder (the same setup the installer test uses).
Confirmed `$PROFILE` resolves to an empty string in that context and
that `Split-Path -Parent $PROFILE` throws there, then re-ran the
installer test with the fix.
- Observed result: before the fix the installer aborted with `Split-Path
: Cannot bind argument to parameter 'Path' because it is an empty
string` and exit code 1 (and the test failed); after the fix the
installer completes, writes the wrapper and PATH entry, logs that it
skipped the profile update, and the test passes. Ran against the actual
script.
- Not tested: a real end-user account whose Documents folder is
redirected to a network share.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
 2026-08-12 00:12:30 -05:00
Rod Boev 78591545ce fix: publish headroom-opencode in release workflow (#2372) 
## Description

`headroom-opencode` is documented as an npm package, but the release
workflow never published it, so installs failed with a registry 404 even
though the plugin source already lived under `plugins/opencode`. This
wires the existing package into the npm release path, keeps its version
synced with root releases, and adds release guards for the new package.
Closes #76.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- added `headroom-opencode` to the npm release workflow, including
release-version stamping and `headroom-ai` dependency rewrite before
publish
- added `plugins/opencode/package.json` to release-please and local
version-sync guards
- synced the source opencode package version to the current release line
and documented the new npm package in the release docs
- added focused release workflow and version-sync tests for the opencode
package
- aligned the two failing dashboard Playwright tests with the current
Session/Lifetime split and `/stats-lifetime` fixture contract

## Testing

- [x] Unit tests pass (`uv run pytest scripts/tests/test_version_sync.py
-q`, `uv run pytest tests/test_release_workflows.py -q -k
'publish_npm_rewrites_opencode_dependency_after_version_and_before_publish
or opencode_source_dependency_matches_lockfile_registry_range or
release_please_manifest_config_consistency'`)
- [x] Unit tests pass (`uv run pytest
tests/test_dashboard_cache_lifetime_playwright.py
tests/test_dashboard_cache_ttl_playwright.py -q`)
- [x] Linting passes (`uv run ruff check scripts/verify-versions.py
scripts/version-sync.py scripts/tests/test_version_sync.py
tests/test_release_workflows.py`)
- [ ] Type checking passes (`uv run mypy headroom`)
- [x] New tests added for new functionality when applicable
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest scripts/tests/test_version_sync.py -q
8 passed, 1 warning in 0.51s

$ uv run pytest tests/test_release_workflows.py -q -k 'publish_npm_rewrites_opencode_dependency_after_version_and_before_publish or opencode_source_dependency_matches_lockfile_registry_range or release_please_manifest_config_consistency'
2 passed, 38 deselected, 1 warning in 0.07s

$ uv run pytest tests/test_dashboard_cache_lifetime_playwright.py tests/test_dashboard_cache_ttl_playwright.py -q
4 passed, 1 warning in 4.04s

$ uv run ruff check scripts/verify-versions.py scripts/version-sync.py scripts/tests/test_version_sync.py tests/test_release_workflows.py
All checks passed!

$ npm ci && npm run build  (plugins/opencode)
Build success; dist/index.js, dist/entry.opencode.js, and DTS outputs emitted
```

## Real Behavior Proof

- Environment: Windows, Python 3.11.15, Node v24.15.0, npm 11.16.0
- Exact command / steps: inspected `.github/workflows/release.yml`,
updated the npm publish path for `plugins/opencode`, aligned the two
failing dashboard Playwright tests with the current Session/Lifetime
split, then ran the focused pytest commands above plus `npm ci && npm
run build` in `plugins/opencode`
- Observed result: the release workflow now versions and publishes
`headroom-opencode`, release-please and version-sync track
`plugins/opencode/package.json`, the dashboard tests now fetch durable
cache and setup-url data from the Lifetime view, and the opencode
package still builds locally from source
- Not tested: GitHub Package Registry publish

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Additional Notes

`CHANGELOG.md` is unchanged because release-please owns changelog
generation here.

---------

Co-authored-by: JD Davis <mxjerrett@gmail.com>
 2026-08-11 23:56:40 -05:00
Suliman Abdulrazzaq e044139001 fix(install): trust Docker bridge for dashboard metadata 
## Summary

Closes #2909.

The `persistent-docker` installer now discovers Docker's default bridge
gateway and passes the exact `/32` gateway CIDR to the proxy's dashboard
metadata allowlist when no explicit
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` value is configured.
This keeps the existing metadata gate intact while allowing the
first-party loopback-published container to see its own Recent Requests
and Per-Project Savings data. Explicit user configuration continues to
take precedence.

Both native wrappers (POSIX and PowerShell) use the same behavior, and
installer integration coverage verifies the generated Docker command.

## Validation

- `python -m pytest tests/test_install/test_native_installers.py -q -k
bash` (1 skipped on Windows because Bash is unavailable)
- PowerShell wrapper smoke test with the repository fake Docker shim:
verified `docker network inspect bridge` is called and
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS=172.17.0.1/32` is passed
to `docker run`
- Explicit allowlist smoke test: verified an existing
`HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` is preserved without
adding a discovered default
- `git diff --check`

## Real behavior proof

Setup tested: Windows 11 host, PowerShell wrapper, repository fake
Docker shim (Docker CLI is not installed in this environment).

Exact command: `headroom.ps1 install apply --profile smoke --port 18999
--image fake/headroom:test`.

Observed result: the generated Docker invocation included `docker
network inspect bridge --format ...` and `--env
HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS=172.17.0.1/32`, and the
installer completed successfully.

Not tested: a live Docker daemon/dashboard request on this host.

---------

Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
 2026-08-11 14:25:29 -07:00
Raúl 3f2ca99fe1 fix(ci): restrict Codecov shard uploads (#2745) 
## Description

Closes #2744

Restrict each Codecov Action v5 matrix upload to its declared
`coverage-${{ matrix.shard }}.xml` report. This prevents automatic
discovery
from uploading the unsharded `coverage.xml` alongside every shard.

## Type of Change

- [x] Bug fix (non-breaking change fixes an issue)

## Changes Made

- Set Codecov Action `disable_search: true` for Python shard uploads.
- Add a CI workflow contract test that protects the explicit-report-only
setup.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Type checking passes (`mypy headroom`) (not applicable:
workflow/test-only change)
- [x] New tests added new functionality
- [x] Manual testing performed (not applicable: GitHub Actions will
execute the workflow)

### Test Output

```text
$ uv run --with ruff ruff format --check scripts/tests/test_ci_workflow.py
1 file already formatted

$ uv run --with ruff ruff check scripts/tests/test_ci_workflow.py
All checks passed!

$ uv run --with pytest pytest scripts/tests/test_ci_workflow.py -q
2 passed
```

## Real Behavior Proof

- Environment: GitHub Actions Ubuntu runner using Python 3.12.13;
Codecov Action v5.
- Exact command / steps: Run the CI Python test matrix, which writes
`coverage-${{ matrix.shard }}.xml`, then runs the Codecov Action upload
step. Inspect the uploader's discovered/uploaded report list.
- Observed result: Before this change, raw CI logs showed the Action
explicitly uploading `coverage-2.xml` and additionally
discovering/uploading `coverage.xml`. This PR configures
`disable_search: true`; the workflow contract test confirms the explicit
report setting and search disablement. Runtime upload evidence will be
added from this draft PR's CI run.
- Not tested: Codecov's final cross-shard patch calculation; that
depends on Codecov processing the reports after CI completes.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR ready for human review

## Checklist

- [x] My code follows project's style guidelines
- [x] I have performed a self-review
- [x] I commented my code, particularly in hard-to-understand areas
- [x] I made corresponding changes documentation (not applicable)
- [x] My changes generate no new warnings
- [x] I added tests prove my fix is effective or feature works
- [x] New and existing unit tests pass locally changes
- [x] I did **not** edit `CHANGELOG.md` — generated by release-please
from Conventional Commit PR title (a CI guard enforces this)

## Additional Notes

This is intentionally limited to the Codecov upload configuration and
its
workflow contract test. It does not include the unrelated Copilot
Keychain fix.
 2026-08-03 14:20:17 -07:00
Tejas Chopra e0ce4b1d48 fix: remove rtk and lean-ctx CLI context tools (#2677) 
## Description

Removes both third-party CLI context tools — **rtk** and **lean-ctx** —
and with them the context-tool selector itself. Headroom no longer
downloads, installs or configures either one, and there is no
replacement.

The previous pass (#2344) gated only three entry points inside
`headroom/cli/wrap.py`. That left the feature reachable in practice:

| Gap | Effect |
|---|---|
| `scripts/install.sh:1544`, `install.ps1:1681` | Ran `rtk init --global
--auto-patch` from bash/PowerShell, **bypassing the Python gate
entirely** — `curl \| sh` still wrote a Claude Code `PreToolUse` hook
regardless of `HEADROOM_RTK` |
| `wrap.py` `_setup_context_tool_for_agent` | **`wrap openhands` was
broken by default**: `rtk_required=True` met a gate returning `None` →
`SystemExit(1)`. Invisible because all 8 openhands tests patched
`_ensure_rtk_binary` to a fake path |
| `proxy/helpers.py`, `subscription/tracker.py` | Proxy shelled out to
`rtk gain` from `/stats`, the dashboard and `headroom perf`; the tracker
polled it per contribution (`_RTK_WIRING_DEFAULT = "enabled"`) |
| No cleanup path | Nothing removed artifacts an earlier default had
installed, so a machine that once ran the old default kept rtk in the
loop forever (#1669, #1955) |

Also worth noting: the rtk binary download had **no SHA or signature
verification** — only `rtk --version` as a smoke test.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [x] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed** — `headroom/rtk/` and `headroom/lean_ctx/` packages,
`headroom/cli/wrap_rtk_metrics.py`, `_selected_context_tool` /
`_setup_context_tool_for_agent` / `_VALID_CONTEXT_TOOLS`, the `--rtk` /
`--no-rtk` / `--no-project-rtk` / `--keep-rtk` flags across all 18 wrap
subcommands, `HEADROOM_RTK*`, the proxy-side `rtk gain` polling, the
dashboard CLI-filtering panel (rows + all 8 `cliFiltering*` Alpine
getters), `paths.rtk_path()` / `lean_ctx_path()`, the SDK path helpers,
`benchmarks/rtk_loop_learn_eval.py`, and the `headroom/rtk/**` CI path
filters.

**Fails loudly, not silently** — `--context-tool` / `--no-context-tool`
/ `HEADROOM_CONTEXT_TOOL` are kept solely to error out. They live in
shell profiles, aliases and CI jobs, and accepting them as a no-op would
read as Headroom having quietly stopped working. The installers reject
them too, which matters more than it looks: their arg parsers forward
the first unknown flag **and everything after it** to the wrapped tool,
so a leftover `--no-rtk` would have silently swallowed a following
`--port` and then been ignored downstream.

**New `headroom/context_tool_cleanup.py`** — deleting the code cannot
help a machine that already ran the old default, since the hooks,
binaries and injected guidance are durable on disk.
`purge_context_tool_artifacts()` runs once per `wrap`/`unwrap` and
removes the registered hook entries, the generated hook scripts, the
Headroom-managed `~/.local/bin` symlinks, the vendored
`~/.headroom/bin/{rtk,lean-ctx}` binaries, the `lean-ctx` MCP server
entry and the marker-fenced instruction blocks. Deliberately
conservative: idempotent, **skips** a malformed config rather than
overwriting it, and only unlinks a symlink resolving inside Headroom's
own bin dir so a user's own build is untouched. It reports on
**stderr**, because `wrap/unwrap openclaw --prepare-only` emit
machine-readable JSON on stdout as their entire contract. Skipped for
`wrap selfheal` (runs from a SessionStart hook; must not race Claude
Code's writer for `~/.claude.json`) and for `--help`, which must stay
read-only.

**Client-config hardening** (discovered while investigating a "corrupted
Serena settings file" report) — `wrap.py` reset a settings file to `{}`
when an existing file would not parse, then wrote that back. One
hand-edited typo or a transient `EACCES`/`EINTR` on a valid file
destroyed the user's `permissions`, `env` and `hooks`, on **every
`headroom wrap claude`**. It now refuses to write. Separately,
`fsutil.write_text` is now atomic (temp file + `fsync` + `os.replace`),
fixing all 14 non-atomic client-config writes at once; it follows
symlinks rather than replacing them (dotfile managers) and preserves an
existing file's mode.

**Deliberately kept** — `rtk` stays in the wrapper-peel list in
`transforms/content_router.py`. It sits beside `sudo`/`env`/`timeout` as
shell-command grammar, so `rtk cat f` is still classified as a file read
for anyone running their own rtk install, which the purge intentionally
leaves alone.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ ruff check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
All checks passed!

$ ruff format --check headroom/ tests/ e2e/ --exclude headroom/dashboard/templates
1255 files already formatted

$ mypy headroom/
Success: no issues found in 508 source files

$ pytest tests/test_context_tool_cleanup.py -q
11 passed

$ pytest tests/test_fsutil.py -q
12 passed

$ pytest tests/test_cli/test_wrap_codex.py -q            # 89 tests
89 passed in 431.68s
$ pytest tests/test_cli/test_wrap_opencode.py -q
39 passed in 257.46s
$ pytest tests/test_cli/test_wrap_helpers.py -q
45 passed
$ pytest tests/test_paths.py -q
75 passed
$ pytest tests/test_cli/test_unwrap_claude.py -q
14 passed
$ pytest tests/test_proxy_savings_history.py -q
39 passed
$ pytest tests/test_cli/test_wrap_copilot.py -q
27 passed
$ pytest tests/test_cli/test_wrap_zcode.py -q
20 passed
$ pytest tests/test_subscription_tracker.py -q
9 passed
$ pytest tests/test_proxy_dashboard_stats_cache.py -q
5 passed, 1 skipped
```

Repo-wide grep for 14 removed symbols (`headroom.rtk`,
`headroom.lean_ctx`, `_ensure_rtk_binary`, `_selected_context_tool`,
`_get_context_tool_stats`, `rtk_path`, `lean_ctx_path`,
`wrap_rtk_metrics`, `HEADROOM_RTK`, `cli_tokens_avoided`,
`tokens_saved_rtk`, …) across `*.py`, `*.ts`, `*.sh`, `*.ps1`, `*.yml`,
`*.html`: **zero hits**.

Notable test changes: `test_wrap_openhands.py` no longer patches
`_ensure_rtk_binary` and asserts `wrap openhands --prepare-only` exits 0
unpatched — the regression that was previously masked.
`test_wrap_continue.py` and `test_wrap_hintfile_agents.py` were removed
(every test drove RTK instruction injection). A new
`test_subscription_tracker.py::test_load_state_written_before_cli_context_tools_were_removed`
proves a pre-removal `subscription_state.json` still loads.

## Real Behavior Proof

- **Environment:** macOS 15.4 (darwin 25.4.0), Python 3.12.6, Headroom @
this branch, real `~/.headroom` and `~/.claude` on the dev machine.
- **Exact command / steps and observed result:**

```text
# 1. Retired flag fails loudly instead of silently no-op'ing
$ headroom wrap codex --prepare-only --context-tool rtk
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: they
rewrote shell commands through a third-party binary Headroom no longer manages.
Drop --context-tool / --no-context-tool and unset HEADROOM_CONTEXT_TOOL;
`headroom wrap` uninstalls what they left behind on first run.

$ HEADROOM_CONTEXT_TOOL=lean-ctx headroom wrap codex --prepare-only
Error: CLI context tools (rtk, lean-ctx) have been removed from Headroom: ...

# 2. install.sh rejects the retired flags (extracted parse_wrap_args harness)
['--no-rtk', '--port', '9999']   rc=1  ERROR: CLI context tools ... Drop --no-rtk
['--context-tool=rtk']           rc=1  ERROR: CLI context tools ... Drop --context-tool
$ bash -n scripts/install.sh   # syntax OK

# 3. Purge ran against the real machine, which had all the orphaned artifacts
$ python -c "from headroom.context_tool_cleanup import purge_context_tool_artifacts; ..."
  removed ~/.headroom/bin/lean-ctx        (51 MB)
  removed ~/.headroom/bin/rtk             (7.7 MB)
  removed ~/.local/bin/rtk                (symlink into ~/.headroom/bin)
  removed ~/.claude/hooks/rtk-rewrite.sh
  removed 8 lean-ctx-* hook scripts
# ~/.claude.json afterwards: 90 top-level keys, 19 projects, mcpServers unchanged
# → ~59 MB reclaimed, no unrelated key touched

# 4. stdout stays machine-readable while the purge reports (planted a fake artifact)
$ headroom wrap openclaw --prepare-only --gateway-provider-id codex >out 2>err
$ cat out
{"enabled":true,"config":{"proxyPort":8787,...}}     # parses as JSON
$ cat err
Retired CLI context tool cleanup: removed /Users/tcms/.headroom/bin/rtk

# 5. --help is inert (planted artifact survives), a real run purges
$ headroom wrap codex --help   → artifact survived: CORRECT
$ headroom wrap openclaw --prepare-only → purged: CORRECT

# 6. MCP purge dry-run against a copy of the real 82 KB ~/.claude.json
top-level keys 90 -> 90;  projects 19 -> 19;  LOST keys: none
all content outside mcpServers byte-identical: True
```

Dashboard rendered via the Playwright test after the panel removal:
"Token Savings" shows only `Proxy 0 (0.0%)` / `Of total wire: 36.86%`,
and "Token Usage" reads Before Compression → Proxy Removed → After
Compression with no "Filtered (this session)" row. Nothing below the
removed panel broke.

- **Not tested:** Windows and Linux (macOS only) — `install.ps1` is
verified by brace-balance and inspection, not executed, since no `pwsh`
is available locally. The wrap e2e suite (`e2e/wrap/run.py`) was updated
but not run; it needs the Docker e2e image. `serena project index`
interaction is exercised in the stacked base PR.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md` — it is generated by
release-please from my Conventional Commit PR title (a CI guard enforces
this)

## Additional Notes

**Stacked on #2676** (`tejas/serena-config-bootstrap`) — please merge
that first; this PR's base should then be retargeted to `main`, or it
will read as containing that fix too.

**Breaking-change migration for users:**
- Drop `--rtk`, `--no-rtk`, `--no-project-rtk`, `--keep-rtk`,
`--context-tool`, `--no-context-tool` from any alias, script or CI job,
and unset `HEADROOM_RTK*` / `HEADROOM_CONTEXT_TOOL`. They now error
rather than being ignored, so the failure is immediate and
self-explaining.
- Previously-installed artifacts are purged automatically on the next
`wrap`/`unwrap`; no manual cleanup needed.
- `headroom perf --json` no longer carries a `cli_filtering` key, and
`/stats` no longer returns a `context_tool` section.

**Docs:** `docs/rtk-architecture.md` deleted; RTK/lean-ctx removed from
`README.md`,
`docs/content/docs/{configuration,opencode,grok-build,docker-install,filesystem-contract}.mdx`,
`docs/observability.md` and the matching `wiki/` pages.
`REALIGNMENT/09-phase-G-rtk-observability.md` is marked SUPERSEDED
rather than deleted, to keep the planning record.

**Follow-ups not in scope:** `_emit_wrap_interrupted` was deleted as
dead code — its only caller was the `except KeyboardInterrupt` guarding
the binary download, so with no download there is nothing slow left to
interrupt.
 2026-07-30 22:59:41 -07:00
Tejas Chopra 5383c6bf2f fix(release): sync generated version metadata on the release branch (#2659) 
## Description

The 0.33.0 release PR (#2339) has sat in `changes-requested` since
2026-07-17. Root cause: **release-please only rewrites `pyproject.toml`
and its configured `extra-files`**, but other tracked files also carry
the version — and `server.json` is asserted byte-for-byte against
`render_server_json()`, which derives its version from `pyproject.toml`.
So the bump alone fails
`tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder`
(the `test (2)` shard) on every regenerated release PR.

Nothing in the repo regenerated `server.json` at all, so it fell behind
every release.

Unblocks #2339.

### Why the release *build* passes but the release PR does not

`release.yml` already runs `scripts/version-sync.py` immediately before
its own `verify-versions.py` gate (lines 145 and 278). That is why
`build` and `build-wheels` are green on #2339 despite the drift — it
syncs in the workspace, uncommitted. The regular CI test job does
**not** sync, so the fix has to be committed to the branch.

This also explains why reviewers kept seeing `verify-versions.py` fail
locally while CI's build jobs passed: the verifier is never run
un-synced inside `release.yml`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **`scripts/version-sync.py`**: also write `server.json`. It was the
one version-carrying file with no writer anywhere. Values are rewritten
in place so key order and formatting keep matching the builder's
byte-for-byte output (verified: the file is pure ASCII and round-trips
exactly through `json.dumps(..., indent=2) + "\n"`).
- **`.github/workflows/release-metadata-sync.yml`** (new): on a push to
`release-please--branches--**`, run version-sync → gate on
verify-versions → commit if changed.
- **Keyed off the branch push** because release-please force-regenerates
that branch on every merge to main. That is precisely what wiped the
hand-pushed metadata fixes on #2339 (`2a86c8ff`, `d5ea4dc5`) — a push
trigger re-heals after every regeneration instead of being lost.
- **Uses the same PAT as `release-please.yml`**: a `GITHUB_TOKEN` push
does not trigger workflows, so the release PR's checks would never
re-run against the synced commit and would stay red.
- **Idempotent**: the self-triggered rerun finds no diff and exits
before pushing, so the loop terminates after one no-op run.
- **Corrected pre-existing drift on `main`**: the agent-hooks plugin
manifests, both marketplace manifests, and `.releasemetadata` were
stranded at **0.31.0** — never bumped for 0.32.0 either.
`verify-versions.py` now passes on `main`.

### Why not more `extra-files` entries

That would need ~13 jsonpath entries restating what `version-sync.py`
already knows, and a jsonpath that fails to match **fails silently** —
the same class of failure this PR removes, discoverable only after a
real release PR regenerates. There is also no precedent for nested
jsonpath (`$.packages[0].version`, `$.metadata.version`) in the config
today; both existing entries are plain `$.version`. Running the script
keeps one source of truth, and files added to it later are covered with
no change here.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`) — no `headroom/` sources
touched
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ python -m pytest scripts/tests/ tests/test_release_workflows.py tests/test_mcp_registry/ -q
207 passed in 2.69s

$ ruff check scripts/version-sync.py scripts/tests/test_version_sync.py tests/test_release_workflows.py
All checks passed!
$ ruff format --check <same>
3 files already formatted

$ actionlint .github/workflows/release-metadata-sync.yml
(clean)
```

New tests:
- `test_server_json_version_is_synchronized` — version-sync moves both
`server.json` version fields and preserves the other keys.
- `test_release_metadata_sync_runs_on_release_please_branch` — asserts
the trigger, the sync→verify→commit ordering, the no-op guard, and the
PAT.
- `test_version_sync_covers_every_file_the_verifier_gates` — guards
`version-sync.py` and `verify-versions.py` against drifting apart again,
which is the root cause here.

## Real Behavior Proof

- **Environment:** macOS (Darwin arm64), Python 3.12, repo venv.
- **Exact command / steps:** reproduced the CI failure locally by
simulating release-please's partial bump, then applying the fix.

**Reproducing the exact `test (2)` failure** — set `pyproject` to 0.33.0
while `server.json` stays at 0.32.0, as release-please leaves it:

```text
$ python -m pytest tests/test_mcp_registry/test_server_json.py -q
FAILED tests/test_mcp_registry/test_server_json.py::test_root_server_json_matches_builder
1 failed, 3 passed
```

**After `version-sync.py`:**

```text
$ python scripts/version-sync.py && python -m pytest tests/test_mcp_registry/test_server_json.py -q
4 passed
```

**Both gates green on a simulated 0.33.0 bump:**

```text
$ python scripts/version-sync.py --version 0.33.0
Version synchronized to 0.33.0
$ python scripts/verify-versions.py
All versions aligned at 0.33.0
$ python -m pytest tests/test_mcp_registry/test_server_json.py -q
4 passed
```

**Idempotency** (the property the workflow's loop-termination relies
on): re-running against an already-synced tree leaves `pyproject.toml`,
`server.json`, `openclaw`, and `sdk/typescript` untouched.

- **Not tested:** the workflow has not executed on a real release-please
branch regeneration — that can only be exercised once this is on `main`
and release-please next updates #2339. The PAT push path and the
self-trigger no-op are reasoned from `release-please.yml`'s existing
token comment and from local idempotency, not observed in CI.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

**Context on the v0.32.0 release failure, since it is easy to misread as
"images never build".** Every artifact built for v0.32.0 — all 5 wheel
platforms including Windows, all 16 Docker builds + 8 manifests +
`promote-latest`, npm, and GitHub Packages. Only `publish-pypi` failed
(PyPI attestations, already fixed by `f9cbdd6e` / #2405), and
`create-release` was skipped because it depends on it. That is why the
release looked like it produced nothing.

**Separate, approaching blocker — not addressed here.** PyPI is at
**9.69 GB of its 10 GB project cap (96.9%)**, leaving ~305 MB against
~68 MB per release, so roughly 4 more releases fit. The `0.21.x` series
alone holds **6.58 GB across 31 releases**, from the old
every-push-is-a-release era; pruning it would reclaim two thirds of the
quota. Worth a separate issue.

**`.releasemetadata` is written but never read** by anything outside
`version-sync.py` and its test. It is kept in sync here for internal
consistency, but it may be a deletion candidate.
 2026-07-29 15:12:04 -07:00
Ruben A. e530de5ad2 feat(rust): port CodeCompressor AST compressor to Rust (parity-only) (#1154) 
Adds crates/headroom-core/src/transforms/code_compressor.rs (1,882 lines): the
AST-aware CodeCompressor ported to Rust on tree-sitter, with grammars for
Python, JavaScript, TypeScript, Go, Rust, Java, C and C++.

Parity-only, like #1153. Nothing calls it: the only references outside the
module are the pub mod / pub use declarations in transforms/mod.rs, and
live_zone.rs still routes SourceCode to a no-op. The pyo3 bridge is untouched
and no Python source changes, so the engine is unreachable from the shipped
package. #1155 wires it into live-zone dispatch.

Every grammar is pinned with '=' to the exact version of the corresponding
Python tree-sitter-<lang> PyPI wheel. Same version on crates.io and PyPI means
the same grammar.js, hence the same generated parser.c, hence node-for-node
identical ASTs — the precondition for byte-parity. A canary over 9 samples x 8
languages confirmed identical node-type and line-span trees at these pins;
bumping any pin requires re-running it and re-recording the fixtures.

Ships 30 recorded parity fixtures, a CodeCompressorComparator in
headroom-parity, and scripts/record_code_compressor_fixtures.py.

Verified byte-identical to the recorded Python output:

  [code_aware_compressor] total=30 matched=30 skipped=0 diffed=0

Full harness on the merge result: 227 fixtures, 182 matched, 45 skipped
(cache_aligner + ccr stubs), 0 diffed, exit 0 — with kompress at 21/21 under
ONNX Runtime 1.24.4 (see #2591).

Also verified cargo check -p headroom-core --no-default-features passes, so the
static-musl path stays intact.
 2026-07-27 09:21:57 -07:00
Ruben A. 83e27e5036 feat(rust): port Kompress ML prose compressor to Rust (parity-only) (#1153) 
Adds crates/headroom-core/src/transforms/kompress.rs (672 lines): the Kompress
ML prose compressor ported to Rust, running the ModernBERT tokenizer plus the
kompress-v2-base ONNX model through ort with a cache-only loader that never
touches the network.

Parity-only. Nothing calls it: the only references outside the module are the
pub mod / pub use declarations in transforms/mod.rs. live_zone.rs still carries
TODO(PR-B4), so PlainText dispatch remains a no-op and Python continues to serve
prose compression. The pyo3 bridge is untouched and no Python source changes, so
the new engine is unreachable from the shipped package. #1155 wires it up.

Ships 21 recorded parity fixtures, a KompressComparator in headroom-parity, and
scripts/record_kompress_fixtures.py.

Verified byte-identical to the recorded Python output:

  [kompress] total=21 matched=21 skipped=0 diffed=0

That required ONNX Runtime >= 1.24 (see #2591) — below it ort deadlocks instead
of erroring, which is why these fixtures had never been run. In CI the model is
absent from the HF cache, so the comparator errors and the fixtures report
Skipped rather than hanging.

Also gates the module behind the ml feature, matching magika_detector: kompress.rs
uses ort, which is optional = true, so an unconditional pub mod broke
cargo check --no-default-features (the static-musl path). CI does not catch that
class of break because cargo test --workspace only builds default features.
 2026-07-27 08:17:53 -07:00
Floze 2bb14d1ab2 fix(ci): align Ruff tooling versions (#2406) 
## Description

Ruff currently has three independent versions: `uv.lock` resolves
`0.14.14`, pre-commit runs `0.9.4`, and CI installs `0.15.17`.
Contributors can therefore pass one formatter path and fail another.

Make the exact Ruff pin in `pyproject.toml` the source of truth, align
the lockfile and pre-commit hook to it, and make CI read that pin
through a deterministic consistency verifier instead of carrying another
hardcoded version.

Closes #2398

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that causes existing functionality
to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Pin the existing `[dev]` Ruff dependency to `0.15.17`, the formatter
baseline already used by CI.
- Refresh only Ruff in `uv.lock` with `uv 0.11.29`.
- Align `ruff-pre-commit` to `v0.15.17`.
- Add `scripts/verify-ruff-version.py` and run it from pre-commit and
CI.
- Make CI install the verified version read from `pyproject.toml` rather
than a separate literal.

## Testing

- [ ] Unit tests pass (`pytest`) — not run; no runtime source or test
behavior changed.
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New deterministic guard proves the configuration fix
- [x] Manual testing performed

### Test Output

```text
# Before: run the verifier with the patched pyproject pin but base-branch
# uv.lock, pre-commit config, and workflow.
Ruff version mismatch detected:
  uv.lock uses Ruff 0.14.14, expected 0.15.17
  .pre-commit-config.yaml uses Ruff 0.9.4, expected 0.15.17
  ci.yml does not run 'python scripts/verify-ruff-version.py --print-version'
  ci.yml does not install Ruff from 'steps.ruff-version.outputs.version'

$ python3 scripts/verify-ruff-version.py
Ruff versions aligned at 0.15.17

$ uvx uv@0.11.29 lock --check
Resolved 269 packages

$ uvx uv@0.11.29 tree --locked --package ruff
ruff v0.15.17

$ uvx ruff@0.15.17 check .
All checks passed!

$ uvx ruff@0.15.17 format --check .
1322 files already formatted

$ uvx mypy@1.20.2 headroom --ignore-missing-imports
Success: no issues found in 505 source files

$ uvx --with tomli mypy@1.20.2 scripts/verify-ruff-version.py --ignore-missing-imports
Success: no issues found in 1 source file

$ uvx pre-commit run ruff --all-files
Passed
$ uvx pre-commit run ruff-format --all-files
Passed
$ uvx pre-commit run verify-ruff-version --all-files
Passed
```

## Real Behavior Proof

- Environment: macOS 26.5, Python 3.14.6 locally; Python 3.10 fallback
also exercised; `uv 0.11.29`, Ruff `0.15.17`, mypy `1.20.2`.
- Exact command / steps: reproduced the mismatch using the base branch's
real `uv.lock`, `.pre-commit-config.yaml`, and
`.github/workflows/ci.yml`; then ran the verifier, locked Ruff tree,
full Ruff check/format, mypy, and actual pre-commit hooks after the
patch.
- Observed result: the base state fails with all four drift points
listed; the patched state reports one aligned Ruff version (`0.15.17`)
and every formatter path passes.
- Not tested: runtime proxy behavior and the pytest suite, because the
change is limited to development-tool configuration, lock metadata,
pre-commit, and CI wiring.

## Dependency / Supply-Chain Justification

- Ruff is an existing development-only formatter maintained by Astral;
this PR adds no new package.
- `0.15.17` is required to fix local/CI reproducibility and has already
been the repository's CI formatter baseline since #1295.
- Install surface is limited to the `[dev]` extra, lint CI job, and
pre-commit environment. Production/runtime dependencies are unchanged.
- The `uv.lock` refresh updates only Ruff; no unrelated dependency
upgrades are included.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented the non-obvious consistency checks
- [x] Documentation changes are N/A; contributor commands are unchanged
- [x] My changes generate no new warnings
- [x] The guard fails on the real base-state mismatch and passes after
the fix
- [ ] New and existing unit tests pass locally — not run; no runtime
code changed
- [x] I did not edit `CHANGELOG.md`; release-please will use the
conventional PR title

## Additional Notes

No formatter-driven source changes are included. AI assistance was used
to inspect configuration, implement the verifier, and run validation.
 2026-07-18 20:55:20 -07:00
ninosat00 5709291914 chore(release): harden local artifact smokes (#1824) 
## Description

Harden the release artifact workflow so maintainers can reproduce npm
and Python release checks locally and so OpenClaw packaging does not
depend on a not-yet-published SDK version. This follow-up also keeps the
OpenClaw loader export contract explicit and updates the PR after the
branch was merged with current `headroomlabs/main`.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [x] Documentation update

## Changes Made

- Added reusable local release smoke scripts for npm assets, Python
wheel/sdist artifacts, and the combined release gate.
- Switched the release workflow npm asset build to the reusable npm
builder.
- Made OpenClaw release asset building install against the just-built
local SDK tarball before rewriting packed metadata to the release
dependency range.
- Kept the OpenClaw source dependency registry-installable at
`headroom-ai@^0.22.3` while the packed artifact still ships
`^<release-version>`.
- Added `registerHeadroomPlugin` as a named export while preserving the
default `{ register }` OpenClaw loader contract.
- Added Windows development bootstrap docs/script and regression tests
for version sync, npm asset ordering, OpenClaw source installability,
and Python wheel smoke import isolation.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
uv run --with pytest python -m pytest tests/test_release_workflows.py scripts/tests/test_version_sync.py -q
44 passed, 1 warning

node --check scripts/build_npm_release_assets.mjs
node --check scripts/verify_npm_release_assets.mjs

python -m py_compile scripts/build_python_release_smoke.py scripts/release_smoke_all.py scripts/version-sync.py
python scripts/verify-versions.py
All versions aligned at 0.31.0

npm ci (plugins/openclaw)
added 88 packages, audited 89 packages, found 0 vulnerabilities

python scripts/release_smoke_all.py --out release-assets-local/all-pr1824-postmerge-fixed-20260710-104739
Verified npm release assets for 0.31.0
wheel metadata OK: headroom_ai-0.31.0-cp310-abi3-win_amd64.whl contains headroom/_core.pyd
sdist License-File metadata OK: ['LICENSE', 'NOTICE']
smoke-import OK: version=0.31.0 hello=headroom-core
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.14.3, Node v24.14.0, npm 11.9.0, uv
0.11.15, Rust/Cargo already installed in the local development
environment.
- Exact command / steps: Ran `python scripts/release_smoke_all.py --out
release-assets-local/all-pr1824-postmerge-fixed-20260710-104739` after
syncing this branch with current `headroomlabs/main`.
- Observed result: The command built `headroom-ai-0.31.0.tgz`,
`headroom-openclaw-0.31.0.tgz`,
`headroom_ai-0.31.0-cp310-abi3-win_amd64.whl`, and
`headroom_ai-0.31.0.tar.gz`; npm verifier passed; the wheel installed
into a fresh venv and imported `headroom._core`.
- Not tested: Full GitHub Actions release matrix and publish jobs with
real PyPI/npm/GitHub Packages credentials.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A.

## Additional Notes

The post-merge full smoke initially failed because the OpenClaw source
package depended on `headroom-ai@^0.31.0`, which is not available on
public npm yet. The builder now installs OpenClaw against the just-built
local SDK tarball before build/pack, and then rewrites packed metadata
to `^0.31.0`.

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
 2026-07-14 16:07:34 -04:00
Shubham Srivastava c3db8e47f8 fix(install): default docker image to headroomlabs-ai GHCR registry (#1867) (#2039) 
## Description

The GitHub repository was transferred from `chopratejas/headroom` to
`headroomlabs-ai/headroom`. GitHub 301-redirects transferred repos for
web and git operations, but **GitHub Container Registry (GHCR) does
not** — the old package `ghcr.io/chopratejas/headroom` is now orphaned
and frozen (its `latest` tag stopped advancing at `0.27.0`), while CI
publishes new images to `ghcr.io/headroomlabs-ai/headroom` (the workflow
derives the path from `${{ github.repository }}`).

Headroom's install tooling still defaulted to the dead path, so
`headroom install apply --preset persistent-docker`, `headroom init`,
and the standalone install scripts all pulled a stale `0.27.0` image
instead of the current release.

This changes every docker-image **default** to
`ghcr.io/headroomlabs-ai/headroom:latest`.

Closes #1867

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/install/models.py` — `InstallManifest.image` default.
- `headroom/cli/install.py` — `--image` Click option default.
- `headroom/cli/init.py` — two `InstallManifest(...)` image args.
- `scripts/install.sh` / `scripts/install.ps1` — `IMAGE_DEFAULT` /
`$ImageDefault` plus the `--image` help-text default.
- `docker/docker-compose.native.yml` — image default in both services.
- `tests/test_install/test_planner.py` (4) and
`tests/test_install/test_runtime.py` (5) — updated the assertions that
pinned the old image (including `assert "<image>" in command`), so they
now verify the corrected registry threads through the planner and docker
runtime command.

Scope note: `github.com/chopratejas/...` links and the plugin
marketplace slug are intentionally **not** changed — GitHub redirects
those, so they still work. Only the genuinely-dead GHCR image references
are touched. No new dependency, no new abstraction.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ uv run pytest tests/test_install/ -q
99 passed, 1 skipped in 48.34s

$ uv run ruff check headroom/install/models.py headroom/cli/install.py headroom/cli/init.py
All checks passed!

$ grep -rn "ghcr.io/chopratejas/headroom" headroom/ scripts/ docker/ tests/
# (no source matches — every default now points at headroomlabs-ai)
```

The install-test assertions pinned the old image, so they fail against
the old defaults and pass after the fix — they are the regression guard.

## Real Behavior Proof

- **Environment:** Windows 11, Python 3.13.5, headroom installed from
this branch (editable, via `uv`).
- **Exact command / steps:** The dead-registry claim is verifiable at
the registry level, independent of a release:
  ```text
docker pull ghcr.io/chopratejas/headroom:latest # old default → 0.27.0
(frozen / orphaned)
docker pull ghcr.io/headroomlabs-ai/headroom:latest # new default →
current release
  ```
And the install pipeline now emits the correct image (covered by
`tests/test_install/test_runtime.py`, which asserts the resolved docker
command contains `ghcr.io/headroomlabs-ai/headroom:latest`).
- **Observed result:** `grep` confirms no `ghcr.io/chopratejas/headroom`
default remains in code, scripts, compose, or tests;
`tests/test_install/` is green (99 passed) with the corrected image
asserted end-to-end through planner → runtime command.
- **Not tested:** A live `docker pull` of both tags on this specific
machine (no local Docker daemon guaranteed) — the registry difference is
reproducible by anyone running the two `docker pull` commands above; and
an end-to-end `headroom install apply --preset persistent-docker`
against a real Docker host.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [x] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A

## Additional Notes

- Documentation checklist item is N/A — no user-facing docs surface
beyond the CHANGELOG entry.
- `mypy` left unchecked — not run as part of this verification; the
change is a string-default swap with no type-level surface.
- The `--image` help text and `docker-compose.native.yml` were included
so every user-facing default is consistent; only the dead GHCR image
string was changed.

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Tejas Chopra <chopratejas@gmail.com>
 2026-07-13 14:01:28 -04:00
JerrettDavis 2d418335a1 ci: preserve merge labels while state is unknown 2026-07-09 19:51:41 -05:00
JerrettDavis 595b709a5b ci: keep ready label off changes-requested PRs 2026-07-09 19:49:55 -05:00
Sepuri Sai Krishna 772adc93b2 fix(scripts): rename .releaseetadata to .releasemetadata (#1246) 
## Description
  
Fixes a typo in the release metadata filename written by
`scripts/version-sync.py`.
The file was being created as `.releaseetadata` (double `e`) instead of
`.releasemetadata`.
Any downstream tooling or developer looking for the artifact by its
correct name would not find it.

  Closes #

  ## Type of Change

  - [x] Bug fix (non-breaking change that fixes an issue)
  - [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
  - [ ] Documentation update
  - [ ] Performance improvement 
  - [ ] Code refactoring (no functional changes)

  ## Changes Made

- `scripts/version-sync.py`: corrected the filename in
`write_release_metadata()` — both the docstring and the `metadata_path`
assignment.
- `scripts/tests/test_version_sync.py`: updated 3 test assertions to
reference `.releasemetadata`.

  ## Testing

  - [x] Unit tests pass (`pytest`)
  - [ ] Linting passes (`ruff check .`)
  - [ ] Type checking passes (`mypy headroom`)
  - [ ] New tests added for new functionality
  - [ ] Manual testing performed

  ### Test Output

  ```text   
  $ uv run python -m pytest scripts/tests/test_version_sync.py -q
============================= test session starts
==============================
  platform linux -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0
  rootdir: /home/sepurisaikrishna/Documents/calude-here/headroom
  configfile: pyproject.toml
  collected 6 items
  
scripts/tests/test_version_sync.py ...... [100%]

=============================== warnings summary
===============================
  PytestConfigWarning: Unknown config option: asyncio_mode

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========================= 6 passed, 1 warning in 1.00s
=========================

  $ git diff --check origin/main..HEAD
  # no output; command exited 0
 ```

  ## Real Behavior Proof

- Environment: Linux, Python 3.14.0, uv-managed .venv, branch based on
current origin/main.
- Exact command / steps: grep -r "releaseetadata" scripts/ before the
fix returns hits; after the fix returns nothing. Confirmed
.releasemetadata is written correctly by
  test_release_metadata_written.
- Observed result: all 6 test_version_sync.py tests pass with the
corrected filename.
- Not tested: full repository pytest, ruff, and mypy — this is a
one-line spelling fix with no logic changes.

  ## Review Readiness

  - [x] I have performed a self-review
  - [x] This PR is ready for human review
  
  ## Checklist

  - [x] My code follows the project's style guidelines
  - [x] I have performed a self-review of my code
- [ ] I have commented my code, particularly in hard-to-understand areas
  - [ ] I have made corresponding changes to the documentation
  - [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
  - [x] New and existing unit tests pass locally with my changes
  - [ ] I have updated the CHANGELOG.md if applicable

  ## Screenshots (if applicable)   

  N/A.

  ## Additional Notes

- The typo was consistent across implementation and tests, so all tests
passed before this fix with the wrong name. The fix corrects both the
code and the test expectations together.
- No production behaviour changes the file is written but not yet
consumed by any workflow step.
 2026-07-09 17:29:17 -05:00
Tejas Chopra a639540959 chore: remove committed node_modules + stray/internal markdown (repo hygiene) (#1528) 
## Description

Repo hygiene for a public OSS project: removes committed `node_modules`,
stray/internal/draft markdown, and commercial-surface references —
keeping every real doc (the published docs site, the wiki guides, and
all component READMEs) intact. Every file was content-audited before
removal, and load-bearing files were verified against the code/CI and
kept.

Net: **1,695 files changed, +23 / −266,409** (the deletions are
dominated by a committed `node_modules` tree).

Closes # (no tracking issue)

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [x] Code refactoring (no functional changes)

## Changes Made

**Removed (verified to have no code/CI dependencies):**
- `examples/vercel-ai-sdk-pr/` — 1,649 committed `node_modules` files
(zero example source); `node_modules/` added to `.gitignore`.
- `docs/spec/` (23 draft "Living Specification" files — orphaned,
`1.0.0-draft`, drifted from the code), `docs/superpowers/` (2 agent
plans), `docs/proposals/` (2 internal/commercial memos).
- 6 orphan `docs/*.md` (auth-modes, bedrock,
claude-code-vertex-headroom, cortex-code, output-token-reduction-guide,
rtk-loop-weighting).
- `PR.md` (committed PR draft), `ENTERPRISE.md`, `.github/FUNDING.yml`.

**Content scrubs:**
- Removed unreleased "Headroom Cloud" / `api.headroom.ai` / `hr_`
references from `configuration.mdx`, `wiki/configuration.md`,
`wiki/typescript-sdk.md`, `sdk/typescript/README.md` (reworded to
neutral, accurate phrasing).
- Dropped a stale "awaiting maintainer before merge" line from
`plugins/headroom-oauth2/SPEC.md`; tidied `.gitignore` comments (kept
the protective `headroom-managed/` ignore rule).
- Fixed the now-dangling links into removed files (README
nav/`output-token-reduction` link, `scripts/README`, `wiki/vertex`).

**Explicitly KEPT (load-bearing — would orphan in-code citations if
removed):**
- `.changelog.md` — consumed by `.github/workflows/release.yml` (read as
the release-notes file).
- `REALIGNMENT/`, `docs/observability.md`, `docs/rtk-architecture.md`,
`wiki/plans/`, `TESTING-copilot-subscription.md` — referenced by the
Rust core / Python / tests as design docs.

## Testing

- [ ] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [ ] Type checking passes (`mypy`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
# Docs/markdown + .gitignore only — no Python/Rust source changed, so the
# behavioral test suite is unaffected. Verified the cleanup did not orphan
# references or break the published docs site:

$ git ls-files 'docs/content/docs/*.mdx' | wc -l      # published site intact
42
$ # meta.json nav unchanged; no published page removed.

$ grep -rnI "Headroom Cloud|api.headroom.ai|'hr_" $(git ls-files '*.md' '*.mdx')
>>> none

$ # dangling refs to removed files (excl pre-existing P0/P2 spec stubs that
$ # never existed in git): none remaining.
```

## Real Behavior Proof

- Environment: macOS, local git clone of the repo (markdown/.gitignore
changes only — no runtime).
- Exact command / steps: 4 read-only content-audit agents classified
every `.md`/`.mdx` file; each removal candidate was cross-checked
against the codebase (`grep` for citations in `.rs`/`.py`/tests,
workflows, and configs); only files with no dependents were removed; the
tree was re-grepped after removal to confirm no new dangling references;
verified the published docs site page count (`git ls-files
'docs/content/docs/*.mdx' | wc -l` = 42, unchanged).
- Observed result: the 42-page published docs site and all wiki guides
are untouched; no source or workflow references a removed file;
`.changelog.md` (consumed by release.yml) and the code-cited design docs
were detected as dependencies and kept; the committed `node_modules`
tree is removed and `node_modules/` is gitignored so it can't be
re-committed; zero "Headroom Cloud"/`headroom.dev` references remain.
- Not tested: N/A — no executable code changed (only markdown, `.mdx`,
and `.gitignore`), so the behavioral test suite is unaffected.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

- This branch deletes `.github/FUNDING.yml` while PR #1526 edits it —
the two will be sequenced at merge (delete wins).
- A follow-up option (not in this PR): also remove the internal design
docs that are currently cited by the code (`REALIGNMENT/`,
`docs/observability.md`, `docs/rtk-architecture.md`, `wiki/plans/`) —
that requires scrubbing ~15–20 in-code citations so nothing dangles, so
it's deliberately deferred.
- Untracked local working files (`benchmarks/hf_pilot/`,
`tools/copilot-test/`) are intentionally left out of git (not
committed).
 2026-06-27 23:32:54 -07:00
JD Davis adb793bee1 ci: harden PR governance and model cache checks (#1401) 
## Description

Hardens two routine PR-review pain points from the recent open-PR sweep:

- PR Governance reruns could keep validating the stale
`pull_request_target` event body even after the live PR description had
been fixed.
- Main CI model-cache misses could surface as dozens of unrelated
memory-test failures instead of one clear cache-preflight failure.

This intentionally avoids PyPI/package-bloat and release/nightly
workflow changes so the PR stays scoped to review and CI stabilization.

## Type of Change

- [x] Bug fix
- [ ] New feature
- [ ] Documentation
- [x] Refactor
- [x] Tests only

## Changes Made

- Added `--body-file` support to `scripts/pr-governance.py` so workflows
can validate the current PR body rather than stale rerun payloads.
- Updated PR Governance to fetch the live PR body via the GitHub API
before validating template fields.
- Added a CI preflight script that loads the default
sentence-transformer model in offline mode and verifies the expected
embedding dimension.
- Wired that preflight into the sharded CI job before pytest starts,
turning missing/corrupt Hugging Face caches into one early, actionable
failure.
- Added workflow/script regression tests for the live-body override and
model-cache preflight placement.

## Testing

- [x] Unit tests
- [x] Lint/static checks
- [ ] Integration tests
- [ ] Manual testing

### Test Output

```text
uv run --with pytest --with pytest-asyncio python -m pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_workflow.py scripts/tests/test_ci_workflow.py -q
9 passed in 0.04s

uv run ruff check scripts/pr-governance.py scripts/ci/verify_hf_model_cache.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_workflow.py scripts/tests/test_ci_workflow.py
All checks passed!

python -m py_compile scripts\ci\verify_hf_model_cache.py scripts\pr-governance.py
# passed
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.3, isolated worktree
`C:\git\headroom\.worktrees\stabilization-hardening`.
- Exact command / steps: Ran the focused governance/workflow tests, ruff
on touched Python files, and `py_compile` for the executable scripts.
- Observed result: Governance tests prove a stale event body can be
overridden by the live PR body; workflow tests prove CI validates live
PR body and runs the Hugging Face offline-cache preflight before pytest
shards.
- Not tested: Full GitHub CI before PR creation; that will run on this
PR. The new Hugging Face preflight itself is intentionally not run
locally because it depends on the CI-warmed offline model cache.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
 2026-06-26 21:34:34 -07:00
Parideboy 5194388b66 fix(ci): normalize Windows CRLF line endings in PR governance script (#1012) 
## Description

The `CODE_BLOCK_RE` regex in `scripts/pr-governance.py` expects LF after
the opening fenced code block. PR bodies authored on Windows can arrive
with CRLF line endings, which leaves a `\r` before the `\n` and prevents
`has_test_output()` from detecting a valid Test Output block.

This normalizes CRLF to LF once when loading the pull request body,
before section extraction and code-block matching. A regression test now
verifies that a valid PR body with CRLF line endings still passes
governance.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [x] Tests only

## Changes Made

- Normalize Windows CRLF line endings in `scripts/pr-governance.py`
before regex-based validation runs.
- Added `test_validate_pull_request_accepts_crlf_test_output_code_block`
to prevent regressions.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check`)
- [x] Formatting passes (`ruff format --check`)
- [x] New tests added for new functionality

### Test Output

```text
python -m pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q
8 passed in 0.06s

ruff check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py
All checks passed!

ruff format --check scripts/pr-governance.py scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py
4 files already formatted
```

## Real Behavior Proof

- Environment: local Windows 11 checkout, Python 3.13.13.
- Exact command / steps: Converted the known-valid governance test body
to CRLF line endings and passed it through `validate_pull_request` in
the new regression test.
- Observed result: The report is valid with no problems, proving the
fenced Test Output block is recognized after normalization.
- Not tested: GitHub-hosted Windows PR authoring path end to end; the
unit test covers the exact CRLF body shape consumed by the validator.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
 2026-06-22 18:45:12 -05:00
Rudimar Ronsoni b4571cc346 feat: headroom wrap opencode / unwrap opencode CLI (#1105) 
## Summary

This PR implements transparent `headroom wrap opencode` support without
asking users to edit OpenCode provider URLs, choose an extra CLI flag,
or maintain a static provider list.

The wrapper now lives at the runtime transport boundary: OpenCode keeps
its user/provider config, while Headroom intercepts outbound provider
traffic in-process and routes it through the local Headroom proxy.

## What changed

### Transparent OpenCode wrapping

- `headroom wrap opencode` injects the `headroom-opencode` plugin
through `OPENCODE_CONFIG_CONTENT`.
- Existing OpenCode provider URLs are preserved. We do not rewrite user
config URLs to point at Headroom.
- Existing `OPENAI_BASE_URL` and `ANTHROPIC_BASE_URL` env vars are
preserved.
- Local OpenCode traffic, localhost traffic, and Headroom proxy traffic
bypass the shim to avoid loops.

### Runtime transport interception

- Added an OpenCode plugin transport shim that wraps:
  - `globalThis.fetch`
  - `http.request` / `http.get`
  - `https.request` / `https.get`
- External provider calls are routed to the local Headroom proxy.
- The original upstream origin is passed through `x-headroom-base-url`,
so the proxy can forward to the real provider without changing OpenCode
config.
- External `http2.connect` is blocked loudly instead of allowing direct
provider traffic to leak outside Headroom.

### Live provider additions

Provider coverage is no longer based on a static config scan. Because
routing happens at outbound request time, providers added mid-session
are routed through Headroom automatically as long as they use the
covered Node transport paths.

### Subagent and child-process coverage

- The parent OpenCode plugin sets a packaged Node preload shim through
`NODE_OPTIONS=--import=.../hook-shim/handler.js`.
- The transport shim patches `child_process.spawn`, `exec`, `execFile`,
and `fork` so child Node processes receive the Headroom preload even
when OpenCode passes a custom `env`.
- The child-process shim fails closed if it loads without
`HEADROOM_OPENCODE_TRANSPORT_PROXY_URL`.
- This closes the subagent leak path where a child Node process could
otherwise start without Headroom transport interception.

## Why this goes beyond PR #1089

PR #1089 improves OpenCode provider registration, but it still focuses
on provider config shape. This PR moves the enforcement boundary to
runtime transport interception.

This PR goes further because:

- No provider URL rewriting is required.
- New providers added mid-session are covered automatically.
- Subagents and child Node processes inherit the Headroom transport
shim.
- Direct external HTTP/2 paths fail loudly instead of leaking.
- The wrap remains transparent to the user's OpenCode provider config.
- The wrapper is fail-closed for unsupported child-process preload
state.

## Additional robustness fixes

While validating the change in Docker, the full Python suite exposed
unrelated Linux/container robustness issues. These are fixed in this PR
so the suite is green:

- Binary cache handling now treats cache paths under a non-writable
existing parent as unavailable, including when tests run as root in
Docker.
- `release_version.py` honors `MANUAL_VER` before git calls so direct
script execution works outside a `.git` checkout.
- Test logger isolation now resets relevant Headroom child loggers so
proxy logging setup cannot poison later `caplog` tests.
- The scanner missing-path test now uses a guaranteed missing `tmp_path`
child instead of relying on `/nonexistent/path`.

## Validation

All implementation validation was run inside Docker.

- Full Python suite from a fresh Docker copy: `6605 passed, 523
skipped`.
- Ruff on changed Python/OpenCode paths: passed.
- OpenCode plugin typecheck: passed.
- OpenCode plugin tests: `9 passed`.
- OpenCode plugin build: passed.
- Hook shim preload smoke test: passed.

## Notes

This PR intentionally does not add a CLI option. `headroom wrap
opencode` means full wrap. Either Headroom wraps OpenCode transparently,
or the path fails loudly instead of silently leaking provider traffic.

---------

Co-authored-by: Rudimar Ronsoni <6081613+rudironsoni@users.noreply.github.com>
 2026-06-22 11:07:12 -05:00
Tejas Chopra a99dc61424 feat: output-token reduction — verbosity shaper, per-user learning, counterfactual savings (#965) 
## Description

Adds the first levers that reduce the tokens the model **writes back**
(output), complementing Headroom's existing input compression. Output
costs 5× input on Opus-class models and is full of waste (ceremony,
restated code, deep "thinking" on routine steps). Two phases in one
self-contained PR off `main`: the request-side output shaper, then
per-user verbosity learning plus an honest counterfactual savings
estimator and dashboard surfacing.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [x] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- **Output shaper** (`output_shaper.py`, opt-in
`HEADROOM_OUTPUT_SHAPER=1`): cache-safe verbosity steering appended to
the system-prompt tail (5 levels); effort routing that lowers
`output_config.effort` on mechanical tool-result continuations; legacy
`thinking.budget_tokens` clamp. Never injects effort where absent, never
toggles `thinking.type`.
- **`headroom learn --verbosity`**: mines Claude Code transcripts for
behavioral signals (interrupts, length-adaptive fast-skips, echo ratio),
recommends a verbosity level (heuristic + optional `--llm-judge`), and
seeds the savings baseline.
- **Counterfactual estimator** (`output_savings.py`): per-stratum
synthetic-control (estimated) + A/B holdout (measured) with a propagated
95% CI; conversation-stable arm assignment for A/B validity and
prefix-cache safety.
- **AIMD verbosity controller** (`verbosity_controller.py`):
additive-increase / fast-back-off state machine; live signal emission
gated off by default.
- **Wiring + surfaces**: shaper resolves the learned level; recording
rides the existing `transforms_applied` channel through the outcome
funnel (no `RequestOutcome` changes); `headroom output-savings` CLI;
dashboard "Output Tokens Saved" card.
- **Docs**: simple-words user guide + design doc with the counterfactual
methodology.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ pytest tests/test_output_savings.py tests/test_output_savings_cli.py \
        tests/test_verbosity_learn.py tests/test_verbosity_controller.py \
        tests/test_output_shaper.py -q
94 passed in 0.54s

$ pytest tests/test_request_outcome.py tests/test_handler_outcome_tag_invariant.py \
        tests/test_proxy_dashboard_stats_cache.py -q
44 passed

$ ruff format --check .
831 files already formatted

$ mypy headroom --ignore-missing-imports
Success: no issues found in 361 source files
```

## Real Behavior Proof

- Environment: macOS, Python 3.12 (`.venv`), `anthropic` 0.76, live API
model `claude-opus-4-8`.
- Exact command / steps: `HEADROOM_OUTPUT_SHAPER=1`; `headroom learn
--verbosity --apply` (seeds level + baseline); `python
scripts/eval_output_shaper.py A` (live before/after); simulate holdout
traffic then `headroom output-savings`.
- Observed result: code-review ask — baseline 1,750 output tokens → L2
1,354 (−22.7%) → L3 599 (−65.8%), same bugs found. `learn --verbosity`
on 24 real sessions → 11% interrupt / 26% fast-skip → L3 (high
confidence). Measured A/B path → 31.7% reduction (95% CI 27.7%–35.7%).
94 new tests + 44 existing outcome/dashboard tests green; ruff + mypy
clean.
- Not tested: live streaming-path recording exercised only via unit
tests (the `transforms_applied` funnel is shared across paths); runtime
AIMD signal emission is gated off by default and not exercised live.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Additional Notes

Output savings are counterfactual (we never observe what the model
*would* have written), so the estimator separates **estimated** (vs a
learned baseline) from **measured** (A/B holdout via
`HEADROOM_OUTPUT_HOLDOUT`) and always reports a confidence band — never
a single made-up number. CHANGELOG left unchecked (release-please
manages it). Runtime AIMD self-tuning is intentionally a TODO
(controller built/tested; live signal emission gated behind
`HEADROOM_VERBOSITY_AUTOTUNE`).
 2026-06-16 21:06:43 -07:00
JD Davis ff221e6346 ci: scope PR workflow runs by changed paths (#1067) 
## Description

Makes PR workflow runs more selective by routing docs-only changes to
docs validation instead of the full CI workflow, while preserving
workflow validation and existing code/e2e/release gates for applicable
changes.

## Type of Change

- [ ] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [x] Code refactoring (no functional changes)

## Changes Made

- Added `pull_request.paths-ignore` to `.github/workflows/ci.yml` so
docs/wiki/markdown-only PRs do not queue the general CI workflow.
- Removed `.github/workflows/ci.yml` from the CI internal `code` path
filter so CI-only workflow edits can run workflow validation without
forcing Python/Rust code jobs.
- Added a docs PR validation job to `.github/workflows/docs.yml` for
`docs/**`, `wiki/**`, `mkdocs.yml`, and docs workflow changes.
- Reduced default docs workflow token permissions to `contents: read`,
with `contents: write` scoped only to the deploy job.
- Added docs workflow dry-runs to `scripts/validate-workflows.sh` so
local/CI workflow validation covers the new PR and manual docs paths.

## Testing

- [ ] Unit tests pass (`pytest`)
- [ ] Linting passes (`ruff check .`)
- [ ] Type checking passes (`mypy headroom`)
- [ ] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
$ actionlint .github/workflows/ci.yml .github/workflows/docs.yml
# no output

$ act pull_request -W .github/workflows/docs.yml -n
*DRYRUN* [Deploy Documentation/validate] 🏁  Job succeeded

$ act workflow_dispatch -W .github/workflows/docs.yml -n
*DRYRUN* [Deploy Documentation/deploy] 🏁  Job succeeded

$ act pull_request -W .github/workflows/ci.yml -n
*DRYRUN* [CI/changes] 🏁  Job succeeded
*DRYRUN* [CI/commitlint] 🏁  Job succeeded

$ python -m mkdocs build
INFO    -  Documentation built in 1.28 seconds

$ bash scripts/validate-workflows.sh
# completed successfully; act dry-runs passed. Some unsupported runner-platform matrix entries are skipped by local act, as before.

$ git diff --check
# no output
```

## Real Behavior Proof

- Environment: Windows local checkout, branch `smart-pr-runs`, `act`
0.2.87, temporary local `actionlint` installed via `go install`.
- Exact command / steps: Ran `actionlint` against changed workflows,
`act` dry-runs for docs PR/manual paths and CI PR path, actual `python
-m mkdocs build`, full `scripts/validate-workflows.sh`, and `git diff
--check`.
- Observed result: Changed workflows lint cleanly; docs PR and manual
docs workflow paths dry-run successfully; CI PR dry-run still covers
`changes` and `commitlint`; MkDocs builds; repository workflow
validation script completes with the new docs dry-runs included.
- Not tested: Full non-dry-run GitHub Actions execution on hosted
runners before PR creation.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable

## Screenshots (if applicable)

N/A.

## Additional Notes

- No issue is linked because this PR was not opened for a specific
tracked issue.
- `mkdocs build` reports existing docs/nav warnings but exits
successfully; strict mode currently fails on existing warnings, so the
PR validation uses the deploy-compatible non-strict build.
- Python unit/lint/type checks are not applicable to this workflow-only
change.
 2026-06-16 19:11:45 -07:00
JD Davis 74dff94fb8 fix(ci): make PR governance advisory (#1047) 
## Description

Make the PR Governance workflow advisory for incomplete pull request
bodies. The workflow still validates the template, writes the run
summary, comments on the PR, and syncs governance labels, but it no
longer marks the check red for expected author follow-up.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- Replaced the failing incomplete-template step with a reporting step
that exits successfully.
- Added a regression test that guards against reintroducing the hard
failure path.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] Manual testing performed

### Test Output

```text
pytest scripts/tests/test_pr_governance.py scripts/tests/test_pr_health_labels.py scripts/tests/test_pr_health_workflow.py -q
# 7 passed

act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-invalid.json -n
# Job succeeded

act pull_request_target -W .github/workflows/pr-health.yml -e .github/act/pr-governance-valid.json -n
# Job succeeded
```

## Real Behavior Proof

- Environment: Windows 11, Python 3.13.13, act 0.2.87, Docker Desktop
via npipe.
- Exact command / steps: Ran the focused governance/label tests and
`act` dry-runs for the valid and invalid PR governance payloads.
- Observed result: Tests passed, the invalid payload's reporting step
completed successfully, and both PR Governance dry-runs ended with job
success.
- Not tested: Full non-dry-run `act` execution against GitHub API
side-effect steps, to avoid mutating real labels/comments from a local
run.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
 2026-06-16 12:36:39 -05:00
RAPHAEL LUGO 99c874d423 fix(codex): PR health label check state (#986) 
## Description

Fix the PR health label job so `status: ci failing` reflects the latest
check attempt for each check, not historical failed or cancelled
attempts that still appear in `statusCheckRollup`.

This showed up on #984: the current checks were green, but the label job
kept `status: ci failing` because older failed template runs were still
present in the rollup payload.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- Added a small `.github/scripts/pr-health-labels.py` helper that groups
check-rollup entries by logical check name and evaluates only the newest
entry for each check.
- Updated the PR health workflow label job to call the helper instead of
treating any historical failing rollup entry as current failure.
- Added regression tests for historical failures followed by latest
passing attempts, plus current latest failure behavior.

## Testing

- [x] Unit tests pass (`pytest`)
- [x] New tests added for new functionality
- [x] Manual testing performed

### Test Output

```text
PYTEST_ADDOPTS='-p no:cacheprovider' pytest scripts/tests -q
47 passed, 1 warning in 0.39s

python .github/scripts/pr-health-labels.py --state-json '<payload with old FAILURE and latest SUCCESS>'
passing

data=$(gh pr view 984 --repo chopratejas/headroom --json statusCheckRollup)
python .github/scripts/pr-health-labels.py --state-json "$data"
passing
```

## Real Behavior Proof

- Environment: macOS local checkout, Python 3.11.7, live GitHub PR #984
check-rollup payload fetched with `gh pr view`.
- Exact command / steps: Added regression coverage for historical
failed/cancelled check runs followed by latest successful runs, ran the
scripts test suite, and evaluated live PR #984's `statusCheckRollup`
with the new helper.
- Observed result: The helper returns `passing` for #984's live payload
even though older failed/cancelled check runs are still present, while
still returning `failing` when the latest attempt for a check failed.
- Not tested: A full GitHub Actions run of the updated workflow on
upstream before merge; this PR should exercise the workflow on itself.

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review
 2026-06-15 16:52:26 -05:00
Copilot 96a7d7cbbe Fix CI lint failure by formatting PR governance scripts (#933) 
`CI / lint (pull_request)` failed because `ruff format --check` detected
formatting drift in the new PR governance script and its tests. This PR
aligns those files with repository formatting rules so the lint job can
pass.

- **Root cause**
  - `ruff format --check .` reported two files as non-canonical:
    - `scripts/pr-governance.py`
    - `scripts/tests/test_pr_governance.py`

- **Change set**
  - Applied `ruff` formatting to only the two flagged files.
- No behavioral or logic changes; edits are line-wrap/format
normalization only.

- **Representative update**
  ```python
  parser.add_argument(
"--event", type=Path, required=True, help="Path to the GitHub event
payload JSON."
  )
  ```

---------

Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-06-12 17:11:39 -05:00
Andrew Rich 93c69372e6 fix(proxy): lazy-import server to avoid fastapi crash (#442) 
## Summary

- Lazy-import `create_app`/`run_server` in `headroom/proxy/__init__.py`
via PEP 562 `__getattr__` to prevent CLI crash when `fastapi` is not
installed (i.e., installed without `[proxy]` extras)
- Fix `.pre-commit-config.yaml` to use `python3` instead of `python`
(unavailable on macOS Homebrew)
- Add graceful `ImportError` skip in `scripts/sync-plugin-versions.py`
for environments without dev dependencies

Fixes #441

## Test plan

- [x] `headroom --help` works without `[proxy]` extras installed
- [x] `headroom proxy --help` works with `[proxy]` extras installed
- [x] `headroom proxy --port 18787` starts and serves traffic
- [x] Lazy imports resolve correctly: `from headroom.proxy import
create_app, run_server`
- [x] `AttributeError` raised for invalid attributes on `headroom.proxy`
- [x] Pre-commit hooks pass (ruff, ruff-format, mypy,
sync-plugin-versions)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Code Bot <claude-code@smartwatermelon.github>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
 2026-06-10 12:44:23 -05:00
Tejas Chopra 74392b238e feat: switch Kompress default to kompress-v2-base with weight-only int8 ONNX (#799) 
## Summary

Replaces `chopratejas/kompress-base` with
**`chopratejas/kompress-v2-base`** as the default Kompress
text-compression model (the fallback for content not handled by
structured compressors), using a new **weight-only int8 ONNX** artifact
that is fp32-equivalent at 2.2x less memory.

## Why

v2 is the same dual-head ModernBERT (token classifier + span CNN),
LoRA-trained on the v2 dataset. The HF repo originally shipped PyTorch
weights only — pointing Headroom at it naively would have forced the
heavy `[ml]` (torch) path on every `[proxy]` install. We exported ONNX
artifacts reproducing the v1 loader contract (single `final_scores`
output) and published them to the HF repo.

## Eval (labeled dataset_v2 test split, n=500, threshold 0.5)

| artifact | size | f1 | must_keep_recall | keep_rate | fp32 agreement |
|---|---|---|---|---|---|
| fp32 (reference) | 601 MB | 0.9128 | 0.9770 | 0.8100 | 100% |
| **int8-wo (default)** | **261 MB** | **0.9130** | **0.9765** |
**0.8097** | **99.6%** |
| fp16 | 301 MB | 0.9129 | 0.9771 | 0.8099 | 99.9% |
| int4-wo | 230 MB | 0.9102 | 0.9825 | 0.8354 ⚠️ | 94.5% |
| int8-dynamic | 271 MB | 0.9041 | 0.9868 | 0.8857 ⚠️ | 90.5% |

Weight-only int8 (MatMulNBits) keeps activations fp32, avoiding the
upward score bias that makes dynamic int8 keep ~7% more tokens (≈40%
less compression savings). Quantized candidates were generated and
eval-gated by a Modal job in the kompress repo
(`modal_jobs/export_onnx_v2.py`) against the labeled test split.

## Changes

- Default model id → `chopratejas/kompress-v2-base`
- ONNX artifact resolution tries candidates in order (**int8-wo → fp32 →
v1 int8**), falling through on download miss **or session-load failure**
— onnxruntime builds without the MatMulNBits 8-bit kernel fall back to
fp32 instead of losing Kompress
- `HEADROOM_KOMPRESS_ONNX_FILENAME` env pins an exact artifact
- `scripts/export_kompress_v2_onnx.py`: reproducible fp32 export (loads
the merged v2 checkpoint, traces the `final_scores` contract, verifies
vs PyTorch)
- `.gitignore`: local `onnx/` artifacts dir; allowlist the export script

## Testing

- End-to-end: fresh `KompressCompressor().preload()` resolves int8-wo
from HF, loads on onnxruntime 1.23 (CPU, no torch), compresses with
error/traceback content preserved
- fp32 export verified lossless vs PyTorch (max |Δscore| = 0.0, 100%
keep agreement)
- ruff check + format clean, mypy clean, 63 targeted tests pass
 2026-06-09 23:28:40 -07:00
Ashish 3db6cd430f chore: wire pre-commit ruff hooks into make install-git-hooks (#786) 
## Problem

`.pre-commit-config.yaml` already has `ruff` + `ruff-format` configured,
and `pre-commit>=3.0.0` is already in `[dev]` deps — but `make
install-git-hooks` never called `pre-commit install`. Every
contributor's repo had the hook **config** but no running hook.

PR #772 merged with inline-comment spacing and import-order violations
that ruff would have caught automatically. The maintainer had to add a
separate fixup commit (`fix: format issue 728 regression test`) to clean
it up.

## Changes

**`scripts/install-git-hooks.sh`** — after installing the pre-push hook,
also run `pre-commit install`. Falls back to `.venv/bin/pre-commit` when
`pre-commit` is not on `PATH`, with a clear warning if neither is found:

```
✅ installed: .git/hooks/pre-push
   Runs 'make ci-precheck' before every git push.
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```

**`CONTRIBUTING.md`** — update PR workflow step 2 to mention `make
install-git-hooks` so contributors know to run it after `pip install`:

```
2. pip install -e ".[dev]" then make install-git-hooks — installs ruff on
   every commit and ci-precheck on every push.
```

## No behaviour change for existing code

Only the local dev setup script is touched. Nothing in the proxy, tests,
or CI pipeline changes.

## Real behavior proof

- **OS**: macOS darwin arm64
- **Steps**: ran `bash scripts/install-git-hooks.sh` with venv
available, then attempted a commit with a badly-formatted file
- **Result**: ruff caught and auto-fixed it before the commit landed

```
✅ installed: .git/hooks/pre-push
   Runs 'make ci-precheck' before every git push.
   Bypass (use sparingly): git push --no-verify
pre-commit installed at .git/hooks/pre-commit
✅ installed: .git/hooks/pre-commit (ruff lint + format via pre-commit)
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-09 23:09:22 -05:00
chopratejas 11f59aed16 ci(release): fix workflow-validation for new release trigger 
scripts/validate-workflows.sh exercises release.yml with an `act`
dry-run that posted a synthesized push-to-main event. After the
previous commit retired that trigger in favor of `release:
published`, the dry-run started failing in CI because release.yml
no longer responds to push events.

Simulate the new trigger instead: feed release.yml a
release-published event (.github/act/release-published.json) and
move the push-to-main dry-run onto release-please.yml — the
workflow that now owns that event.
 2026-05-25 18:36:12 -07:00
chopratejas 80f403db4a test(scripts): cover branch-aware _should_sync in sync-plugin-versions 
PR #484 added branch-awareness to ``scripts/sync-plugin-versions.py``
(no-op on feature branches unless ``HEADROOM_SYNC_VERSIONS=1``). The
existing ``test_main_runs_plugin_only_version_sync`` test broke
because it didn't account for the new ``_should_sync`` gate — on a
feature branch the main() function early-returns and the subprocess
mock was never invoked, but actually the test broke earlier because
``_current_branch`` calls ``subprocess.run(..., capture_output=True,
text=True, check=False)`` and the test's lambda only accepted
``(command, cwd, check)``.

Fix: force ``_should_sync`` True in the existing test so it locks
the run-path, then add 4 new tests covering the branch-aware logic
itself (env override, main vs feature, git-unavailable defensive
no-op).
 2026-05-19 12:49:21 -05:00
chopratejas a7b197c6ec refactor(proxy): MemoryRanker + ImageCompressionDecision + branch-aware version-sync 
Three independent contract-pattern follow-ons bundled into one PR.
Same frozen-dataclass + factory + apply_to_tags + Rust-portable
shape that PR #473 / #477 / #483 established.

## (1) MemoryRanker + RecencyBoostRanker

Pre-this-PR Headroom ranked memory candidates by pure cosine
similarity. Every other memory system we surveyed (Letta, Mem0,
Cognee, Supermemory) re-ranks beyond cosine.

* ``MemoryRanker`` Protocol — pluggable re-ranker; future PRs add
  source-weight + access-count rankers behind the same interface.
* ``RecencyBoostRanker`` — first concrete impl. Final score is
  ``cosine × exp(-age_days / decay_days)``. Default decay 30 days
  (half-life ~21 days; 60-day-old factor 0.135, 90-day-old 0.050).
* ``MemoryCandidate`` — backend-agnostic frozen value type that
  flows through the ranker. ``MemoryCandidate.from_backend_result``
  adapter converts the existing ``MemoryResult`` shape (with nested
  ``memory.created_at``) into the ranker's flatter form.
* Wired into ``memory_handler.search_and_format_context`` as an
  optional ``ranker=`` kwarg — backwards-compat: ``None`` (default)
  preserves the pure-cosine path identically.

Defensive:
* ``created_at=None`` → factor 1.0 (recency-neutral, back-compat with
  legacy rows / migrating backends)
* Negative age (clock skew) → clamped to factor 1.0 (a future-dated
  row can't outrank a real fresh memory)
* Sort is stable on ties — same input → same output every turn, so
  consecutive turns inject memories in the same order (prefix-cache
  friendly)

Performance: O(N) over candidates where N=top_k≈10. One ``math.exp``
per candidate. Sub-microsecond. Zero new I/O.

## (2) ImageCompressionDecision

Mirror of :class:`CompressionDecision` for image compression. Two
sites today (``openai.py:1203``, ``anthropic.py:868``) gate inline;
both already respect bypass (no Gemini-class drift bug like text
compression had), but consolidating into a value type:

* Locks bypass-respect via AST contract test — future sites can't
  drift on it
* Surfaces ``image_skip_reason`` in ``RequestOutcome.tags`` for
  dashboard slicing (same observability surface as
  ``passthrough_reason`` and ``memory_skip_reason``)
* Same Rust-port shape as the other decision types

Precedence: ``bypass_header`` > ``image_optimize_disabled`` >
``no_messages`` > ``should_compress=True``.

Anthropic's extra ``is_cache_mode`` check stays inline because it's
Anthropic-specific (openai/gemini don't have it). Documented in a
code comment.

## (3) Branch-aware sync-plugin-versions hook

Pre-this-fix the pre-commit ``sync-plugin-versions`` hook ran on
every commit and bumped manifests to the predicted-next-release
version. Every PR ended up carrying the prediction as collateral
("Why are we bumping ``.claude-plugin/marketplace.json`` — we
should not, right??" - user, on PR #483).

Fix: the hook is now a NO-OP unless EITHER:
* We're on the ``main`` branch, OR
* ``HEADROOM_SYNC_VERSIONS=1`` is set explicitly (release workflow)

On feature branches the hook prints a single line explaining the
skip and exits cleanly. The release workflow opts in via the env
var; behaviour on main / at release time is unchanged.

## Test coverage

* 16 new tests on ``MemoryRanker`` / ``RecencyBoostRanker``
  (frozen, equal cosine wins by recency, decay configurable, NULL
  timestamp neutral, no-mutation contract, Rust-port shape)
* 17 new tests on ``ImageCompressionDecision`` (frozen, all 3
  skip reasons, precedence, observability fields, apply_to_tags)
* 1 new AST invariant test (extends
  ``test_handler_outcome_tag_invariant.py``) — locks "no raw
  ``if self.config.image_optimize and messages and not _bypass:``
  conjunction in any handler"

All existing memory + cache-stability + handler tests pass (203 ✓).
``make ci-precheck`` clean.

## Rust portability

All three new value types port cleanly to frozen Rust structs +
pure functions. Same migration pattern as ``CompressionDecision``
(already locked in for the SmartCrusher Rust port).

## Zero-regression contract

* Default ``ranker=None`` → memory_handler behaves identically to
  pre-this-PR (pure cosine; no perf change)
* Image decision migration is identity at the bypass/optimize/messages
  gate — no behaviour change, just contract consolidation
* Hook fix is no-op on feature branches (less churn) and unchanged
  on main (release flow preserved)
 2026-05-19 12:13:09 -05:00
chopratejas e53276302e fix(tests): ship scripts/replay_codex_ws_load.py so CI can import it 
CI ran the Codex compression scheduler stress test on python 3.10/3.11/
3.12/3.13 and all four versions failed with:

    ModuleNotFoundError: No module named 'scripts.replay_codex_ws_load'

The test imports ``boot_proxy``, ``warmup``, ``replay_session``, ``Frame``,
and ``Scenario`` from the replay tool to drive 30 concurrent compression
calls and assert no p99 contention tail (the very regression this PR
fixes). The tool was untracked because ``scripts/*`` is gitignored by
allowlist — local-only tools work fine for measurement but CI cannot
import them.

Add the replay tool to the allowlist so it ships. Same pattern as
``scripts/smoke_issue_327.py`` and other single-purpose scripts already
on the allowlist. The tool is genuinely useful beyond this PR: it lets
any contributor reproduce the Codex slowness baseline numbers and
measure their own fix against the same workload shape.

Local re-run with the tool committed: 3 passed, 1 skipped — identical
to the pre-fix branch run.
 2026-05-14 13:44:41 -07:00
Gili Tzabari 4b061792b2 feat: add lean-ctx context tool support 2026-05-11 17:54:17 -04:00
Tejas Chopra ea1f608e79 fix: make proxy upgrades version-aware 
Derive source-tree versions from release history so headroom --version no longer reports stale project metadata.

Restart stale idle proxies after an upgrade, but leave active sessions running to avoid interrupting ongoing conversations.

Remove _version.py from version-sync ownership and make version verification catch package/plugin manifest drift.
 2026-05-09 15:58:27 -07:00
chopratejas 17d6207bf5 fix(crusher): shim __libc_single_threaded for glibc < 2.32 + extend audit 
PR #396's X2 dry-run caught a wheel-import failure on the manylinux_2_28
floor matrix entry (both x86_64 and aarch64). Same class as #355:

  ImportError: ... undefined symbol: __libc_single_threaded

`__libc_single_threaded` is a single-byte char added in glibc 2.32.
Newer libstdc++ (gcc 11+) reads it inside `__cxa_thread_atexit_impl`
to elide locking on the single-threaded fast path. ORT prebuilt static
archives compiled with gcc-14.2.1 against glibc-2.38+ headers bake in
the reference. Users with glibc < 2.32 hit ImportError on
`import headroom._core`.

Latent since the ORT artifact bump that started using gcc 14. X1 is
the gate that catches it at release time; X2 caught it at PR time —
exactly as designed.

Fix:
1. glibc_compat.c adds Section B: `char __libc_single_threaded = 0;`
   Setting to 0 (multi-threaded) is safe; libstdc++ takes the locked
   slow path. Setting to 1 would race in any multithreaded Rust wheel.
2. build.rs adds `-Wl,-u,__libc_single_threaded` so the shim's archive
   members are pulled regardless of scan order.
3. audit_wheel_glibc_symbols.py POST_FLOOR_SYMBOLS adds the new
   symbol — verified locally: the audit now rejects the failing
   PR #396 wheel with the right message.
 2026-05-05 13:59:21 -07:00
chopratejas e2146724af fix: ship glibc 2.38 compat shim + wheel symbol audit (closes #355) 
Issue #355: published headroom_ai-*-manylinux_2_28_*.whl fails to import on Ubuntu 22.04, Debian 11/12, Conda envs with libc < 2.38: 'ImportError: undefined symbol: __isoc23_strtoll'.

Root cause: ORT prebuilt artifacts (downloaded via fastembed's ort-download-binaries-rustls-tls feature) are compiled with gcc-14.2.1 on a glibc-2.38+ host and reference __isoc23_strtoll. Our manylinux build host has glibc 2.38 so the link succeeds; end users with older glibc don't.

Two-part fix: (1) crates/headroom-py/glibc_compat.c provides weak-alias definitions for __isoc23_strtol/strtoll/strtoul/strtoull delegating to the older strtol* family, compiled by build.rs on Linux/glibc only. The dynamic linker prefers glibc's strong symbol when present (>= 2.38) and falls back to ours when not (< 2.38). (2) scripts/audit_wheel_glibc_symbols.py is a release.yml gate that runs objdump -T on every Linux wheel and rejects any UND symbol whose required glibc version exceeds the wheel's manylinux floor.

Validated: the audit correctly rejects the actually-broken v0.20.26 wheel with a precise diagnostic. The shim itself is a tiny static link with zero runtime cost.

Regression tests in tests/test_release_workflows.py pin the shim's load-bearing pieces (.c file, build.rs trigger, [build-dependencies] cc dep) and the audit invocation in release.yml. Future drift fails at PR time, not in the next release.

This is the same bug class as PR #371 (rustls-everywhere). Each instance gets fixed; the audit gate now catches the *class* — any future static linkage that introduces a post-floor symbol gets blocked before publish.
 2026-05-04 21:31:19 -07:00
chopratejas 2a91cbb4b4 refactor: single-wheel maturin build backend (fixes #355) 
Eliminates the dual-package architecture that was the root cause of #355.
`pip install headroom-ai` now produces ONE wheel containing both the Python
source (headroom/*.py) and the compiled Rust extension (headroom/_core.so).
No more separate `headroom-core-py` package, no more chicken-and-egg with
PyPI publication, no more wheelhouse / PIP_FIND_LINKS / composite-action
plumbing in CI.

This is the canonical pattern used by cryptography, polars, ruff,
pydantic-core, and other Rust-as-core Python packages. Honors the
"Rust as core engine" direction.

## What changed

- pyproject.toml: `[build-system]` swapped from hatchling to maturin.
  `[tool.hatch.*]` deleted; `[tool.maturin]` added pointing at
  `crates/headroom-py/Cargo.toml` for the cdylib. `python-source = "."`
  picks up the root `headroom/` package directly (dashboard HTML
  templates and other non-Python files included automatically).
- crates/headroom-py/pyproject.toml: deleted. The crate is no longer a
  separate published package; its Cargo.toml stays as the cdylib build
  target invoked via `[tool.maturin] manifest-path`.
- crates/headroom-py/python/: deleted (placeholder layout for the old
  separate package).

## CI updates

- ci.yml: `test` / `test-extras` / `test-agno` jobs simplified — Rust
  toolchain set up before `pip install -e .` (which now invokes maturin
  via build-system). Removed the "build wheel + symlink .so" dance.
  `build` job swapped from `python -m build` (hatch) to
  `maturin build` + `maturin sdist`.
- release.yml: collapsed dual-package matrix into one. New `build-wheels`
  matrix produces cross-platform wheels for cp310/11/12/13 ×
  {linux x86_64, linux aarch64, macos x86_64, macos aarch64}. New
  `collect-dist` aggregator merges artifacts. publish-pypi consumes the
  merged dist.
- init-native-e2e.yml: dropped windows-latest from the matrix —
  upstream `esaxx-rs` (/MT) and `ort-sys` (/MD) link with conflicting
  MSVC C runtime libraries, so the Rust extension cannot build for
  win_amd64 today. Tracked as a follow-up; not a blocker for Linux+macOS.
- headroom-e2e-setup: composite action now sets up Rust toolchain +
  Swatinem/rust-cache before `pip install -e .[proxy]`.
- eval.yml, publish.yml, rust.yml: same pattern — rust toolchain before
  install. rust.yml's wheels job builds from root pyproject.toml (no
  more `-m crates/headroom-py/Cargo.toml`).
- e2e/init/Dockerfile, e2e/wrap/Dockerfile: install rust + maturin in
  the build stage; copy `crates/` + workspace `Cargo.toml/lock` so the
  install can build the extension. Dropped `HEADROOM_REQUIRE_RUST_CORE=false`
  from wrap-e2e — the image now ships the full Rust core.
- Dockerfile (main): simplified — no more Layer 2/3 dance with
  `headroom-core-py` install + symlink. Single `uv pip install` builds
  + installs everything.
- .devcontainer/Dockerfile: rust toolchain + libssl-dev + maturin
  added so `uv sync` builds the extension inside the devcontainer.

## Lockfile + script

- uv.lock: regenerated. No `headroom-core-py` entries remain.
- scripts/build_rust_extension.sh: simplified from a symlink-into-tree
  workaround to a thin wrapper around `pip install -e .`. The maturin
  build-backend handles placement automatically.

## Local validation (all green on macOS aarch64)

1. Clean venv `pip install -e .` → `from headroom._core import …` works.
2. `maturin build --release` → 13.8 MB wheel, 336 files including
   `headroom/_core.cpython-311-darwin.so` (32 MB cdylib) and
   `headroom/dashboard/templates/dashboard.html`.
3. `pip install <wheel>` in fresh venv → import works.
4. Wheel contents verified via `unzip -l`.
5. `pytest tests/test_transforms/test_diff_compressor.py` — 29 passed.
6. `pytest tests/test_relevance.py` — 30 passed.
7. `cargo build --workspace` + `cargo test --workspace` — all green.
8. `make ci-precheck` — 176 Python tests + Rust + commitlint green.

## Migration notes

Users on `pip install headroom-ai` get the Rust core automatically
(linux + macos wheels). sdist installs require rust toolchain available
locally — pip will build via maturin.

Closes #355
Supersedes #357 (workarounds-based fix abandoned in favor of
architectural fix)
 2026-05-03 13:16:41 -07:00
chopratejas 00ab1ea74d fix: A0 — fail-loud rust core deployment smoke test 
Production incident (Finding #2 of HEADROOM_PROXY_LOG_FINDINGS_2026_05_03.md):
on this customer's deployment the Rust extension `headroom._core` was
never installed into the runtime Docker image. Diff compression failed
54 times in a single day; "Optimization failed: ModuleNotFoundError" hit
379 times. The failure rate climbed every day and reached ~223/day on
2026-05-03 — effectively 100% of requests on the Rust path. Every Rust
PR we'd merged (MessageScorer, ICM, DiffCompressor, etc.) was providing
zero customer value because the module wasn't loadable at all.

Root cause: the Dockerfile builder stage installed Python deps and the
in-tree `headroom-ai` package but never ran `maturin build` for the
`headroom-py` crate, so the runtime image shipped without `_core.so`.
The Python proxy continued to start because the extension's absence is
caught and routed through Python-only fallbacks that either silently
no-op or raise per-request.

This change makes that mode impossible by default:

* `headroom.proxy.server._check_rust_core()` runs as the first step of
  the FastAPI lifespan. If the import fails it prints a structured
  diagnostic, logs `event=rust_core_missing`, and calls `sys.exit(78)`
  (sysexits.h `EX_CONFIG`). Process supervisors (systemd / k8s /
  docker) treat this as a deliberate config error and stop restart
  loops.
* `HEADROOM_REQUIRE_RUST_CORE=false` is the explicit opt-out for
  Python-only `pip install -e .` developer flows; lifespan logs
  `event=rust_core_disabled` and continues. Any other value (including
  unset) keeps the fail-loud default.
* `/health` now surfaces `rust_core: "loaded" | "disabled" | "missing"`
  (plus `rust_core_error` when non-loaded) so operators can alert on
  the degraded state rather than discovering it via a customer ticket.
* `scripts/build_rust_extension.sh` is the single dev-time path: build
  → install → import-verify with the same `hello()` marker the lifespan
  checks. Failures are loud at every step.
* `Makefile` exposes the script as `make verify-rust-core`.
* `Dockerfile` now installs `rustup` + `maturin`, builds the wheel from
  `crates/headroom-py`, force-installs it into site-packages, and runs
  the same `hello()` import-verify in the build image so a broken build
  fails the docker-build, not the next runtime restart.

Tests:
* `tests/test_rust_core_smoke.py` pins all four contracts:
  - `_core.hello()` returns `"headroom-core"`
  - missing extension + default env → `SystemExit(78)`
  - missing extension + opt-out env → lifespan starts, `/health`
    returns `rust_core: "disabled"` with the underlying error
  - present extension + default env → `("loaded", None)`

Per-finding-#2: ~/Desktop/HEADROOM_PROXY_LOG_FINDINGS_2026_05_03.md.
 2026-05-02 17:52:37 -07:00
chopratejas fa5fbfabf4 fix(rust): wire ICM compressor into Rust proxy on /v1/messages 
Adds an opt-in compression interceptor that buffers Anthropic
/v1/messages requests, runs IntelligentContextManager over the
messages array, and forwards the (possibly trimmed) body upstream.
All other paths, methods, and content-types stay on the original
streaming passthrough — so existing operators see zero change.

Behaviour gates ALL must be true to buffer + compress:
  - --compression flag (or HEADROOM_PROXY_COMPRESSION=1)
  - method == POST
  - path == /v1/messages
  - Content-Type: application/json
  - ICM constructed successfully at startup

Falls through to streaming on any failure: parse, missing fields,
unknown model, body-too-large. Compression must never break a
request — that's the safety contract.

Model context windows come from a vendored LiteLLM snapshot at
crates/headroom-proxy/data/model_prices_and_context_window.json
parsed once into an OnceLock<HashMap>. Refresh via
scripts/refresh_model_limits.sh. Rationale documented inline:
hardcoded tables silently rot; LiteLLM is the canonical source
the entire LLM-tooling ecosystem relies on.

New tests:
  - 16 unit tests across compression::{anthropic, icm, model_limits}
  - 5 integration tests: off-passthrough, on-short-passthrough,
    on-oversized-trim, on-non-json-skip, on-non-llm-path-skip

Verification:
  - cargo test --workspace -> 884 passed, 0 failed
  - cargo clippy --workspace -- -D warnings -> clean
  - cargo fmt --check -> clean
 2026-05-01 16:44:44 -07:00
chopratejas 35eaf8de7f fix(proxy): remove content-keyed TTL walker that conflated content with positional cache (#327) 
The Anthropic token-mode handler walked past prefix_tracker.frozen_message_count
whenever an upcoming tool_result's content-hash matched comp_cache._stable_hashes
or should_defer_compression returned True. That conflated content equality with
positional cache membership.

Anthropic's prefix cache is POSITIONAL: bytes 0..K cached, anything past K is
fresh. _stable_hashes is content-keyed and grows unbounded. In long Claude Code
sessions where tool_result content rhymes across turns (repeated system prompts,
repeated file reads, repeated tool descriptions), the walker advanced
frozen_message_count to len(messages) on every turn and the pipeline produced
transforms_applied=[] on 73% of requests in user SvenMeyer's reported session
(headroom-stats-2026-05-01.json: 74 of 101 eligible requests "prefix_frozen") —
even after the prior fix in 44944fb. The 15 requests that did compress averaged
21%, proving compression itself works when reached.

Fix: delete the walker. The freeze boundary is now

    frozen_message_count = min(
        prefix_tracker.frozen_message_count,    # positional ground truth
        comp_cache.compute_frozen_count(messages),  # local cache lower bound
    )

compute_frozen_count's use of _stable_hashes can only LOWER the freeze via the
min clamp, never raise it past prefix_tracker's value. For any position in the
gap [compute_frozen_count, prefix_tracker.frozen_count], recompressing produces
byte-stable output (compression is deterministic on input content), so
Anthropic's prefix cache stays valid.

Cross-handler verification:
* OpenAI handler (proxy/handlers/openai.py:358-382) does not have this walker
  — uses only compute_frozen_count. Codex routes through OpenAI handler. Both
  unaffected.
* Streaming and non-streaming both invoke anthropic_pipeline.apply() before the
  upstream call. One fix covers both paths.
* Cache mode (is_cache_mode) takes the _extract_cache_stable_delta path and is
  independent of the walker. Unaffected.

Tests: six new regression tests lock down the post-fix invariants — clamp to
min(prefix_tracker, compute_frozen_count); fresh tool_result whose hash matches
old _stable_hashes entry is not frozen; frozen prefix byte-stable across the
pipeline; 10-turn session produces non-empty compression suffix every turn;
streaming and non-streaming compute identical frozen_message_count; OpenAI
handler never calls the walker functions. Plus scripts/smoke_issue_327.py
(gated by RUN_LIVE_API=1) drives a 10-turn conversation against
api.anthropic.com in both shapes (string + list-of-blocks) and both modes
(streaming + non-streaming).

ci-precheck clean. 191 tests pass.

Follow-ups (separate PRs):
* Fix _cache perpetually empty (anthropic.py result.messages != working_messages
  comparison rarely fires in token mode).
* Cap _stable_hashes with bounded LRU + 1h TTL — hygiene only after the freeze
  gate is removed.
* List-shape tool_result content gates at content_router.py:1975 and
  intelligent_context.py:657 (cluster A from the audit).
 2026-05-01 12:04:28 -07:00
chopratejas d6a00ee89c ci: fix smart_crusher branch CI failures + add make ci-precheck pre-push gate 
Five gates broke on the 2026-04-27 push of the smart_crusher branch.
Each is fixed below; the second half adds a `make ci-precheck` target
(plus an installable git pre-push hook) so the same dance never happens
again.

Failures fixed:

1. cargo fmt — 22 files had formatting drift introduced over the
   stage 3c.1 work. `cargo fmt --all` reformatted them; no semantic
   changes. `cargo test --workspace` still green (388 + supporting).

2. wheels job (macOS x86_64) — `fastembed -> ort -> ort-sys` does not
   publish prebuilt ONNX Runtime binaries for `x86_64-apple-darwin`.
   Removed that target from `.github/workflows/rust.yml`'s wheels
   matrix. Apple Silicon (`aarch64-apple-darwin`) covers macOS
   distribution; Intel macOS users can build from source. The matrix
   now has 2 targets: linux x86_64 + macOS aarch64.

3. test-extras (relevance.py) — `tests/test_relevance.py::TestSmartCrusherIntegration`
   constructs a `SmartCrusher`, which hard-imports `headroom._core`
   since the python implementation was retired in stage 3c.1b. The
   test-extras job didn't build the rust extension. Added the same
   `maturin build + symlink` block the main `test` job uses.

4. smoke-test (eval.yml) — same root cause:
   `compression_only.evaluate_ccr_lossless` instantiates a SmartCrusher.
   Same fix: build the rust extension before the smoke test runs.

5. commitlint — three rules tripped:
   - `subject-case` rejects PascalCase identifiers in subjects, but
     the project deliberately names classes (SmartCrusher, HfTokenizer,
     ContentRouter, DiffCompressor) in commit subjects. Disabled.
   - `footer-leading-blank` is a warning that the wagoid action turns
     into a CI failure; lines like `Module: foo.rs` in our bodies
     match the conventional footer pattern and trip it. Disabled.
   - `type-enum` doesn't include `parity`, but the project ships
     parity-test infrastructure as its own concern (separate from
     `test:`); added `parity` to the allowed types.

Pre-push verification — the prevention half:

`make ci-precheck` runs all of the above CI gates locally:
- `ci-precheck-rust`: cargo fmt --check + clippy + test --workspace.
- `ci-precheck-python`: builds the rust extension via maturin, then
  runs the smart_crusher-affected python test files (185 tests across
  test_transforms/, test_relevance*, test_ccr, test_acceptance,
  test_critical_fixes, test_quality_retention).
- `ci-precheck-commitlint`: `npx commitlint --from origin/main --to
  HEAD` against the same config CI uses. Skipped silently if npx is
  not on PATH (install Node 18+ to enable).

`make install-git-hooks` (or `scripts/install-git-hooks.sh`) installs
a git pre-push hook that runs `make ci-precheck` automatically.
Bypass with `--no-verify` only when truly needed.

When new CI gates land in `.github/workflows/`, mirror them into a
`make ci-precheck-*` target. The Makefile is the local mirror of the
CI configuration; keeping them in sync is a load-bearing invariant.

Verification: `make ci-precheck` runs green on this commit.
 2026-04-27 11:13:47 -07:00
chopratejas f5f465418b feat(rust): retire python diff_compressor, ship rust-only via pyo3 
The python `DiffCompressor` is replaced by a thin pyo3-backed shim that
delegates to `headroom._core.DiffCompressor`. There is no python
implementation and no env-var fallback — the wheel is a hard import.

Why now: opt-in defaults don't drive python retirement. Byte-equal
parity was already proven across 27 fixtures (stage 3a); keeping a
shadow python impl behind a flag is a permanent maintenance cost with
no operational benefit. Stage 3b deletes ~700 lines of python parser /
scorer / formatter code; the rust crate has its own coverage.

Surface preserved:
- `headroom.transforms.diff_compressor.DiffCompressor` — same class
  name, same `__init__`, same `compress(content, context)` shape.
  Returns python `DiffCompressionResult` dataclasses so call sites that
  destructure with `asdict()` work unchanged.
- `DiffCompressorConfig` and `DiffCompressionResult` dataclasses kept.
- Sidecar `compress_with_stats(...)` exposes the rust-only
  `DiffCompressorStats` (per-file hunk drops, context lines trimmed,
  file_mode normalizations) for observability.

Removed:
- Python parser / scorer / formatter (~700 lines).
- Internal `DiffHunk` / `DiffFile` parser dataclasses (rust crate has
  parallel coverage).
- 12 tests that probed `_parse_diff` / `_score_hunks` or the deleted
  parser dataclasses. The 29 public-API tests in `test_diff_compressor.py`
  remain and now exercise the rust backend through the same import path.
- `HEADROOM_DIFF_COMPRESSOR_BACKEND` env var — no longer meaningful.

Build:
- `scripts/build_rust_extension.sh` runs `maturin develop` and symlinks
  the built `.so` into `headroom/` so `import headroom._core` resolves
  past the in-tree package shadowing the maturin overlay.
- `.gitignore` excludes the symlinks and allowlists the build script.

Tests:
- 544 transforms pass; 33 rust-vs-fixture parity tests guard the pyo3
  bridge against regressions (`tests/test_transforms/test_diff_compressor_rust_parity.py`).
- Mypy clean.
 2026-04-26 09:15:37 -07:00
chopratejas 4429a11166 Merge remote-tracking branch 'origin/main' into rust-rewrite 
# Conflicts:
#	headroom/proxy/server.py
 2026-04-25 13:01:37 -07:00
chopratejas 0414cb70e4 feat(rust): scaffold workspace + parity harness (phase-0) 
Bootstrap the Rust port of Headroom. Additive only — no existing Python
code modified. Ships the workshop, not the widgets.

Layout
  Cargo.toml (workspace) + rust-toolchain.toml
  crates/headroom-core    — transform library, stub only
  crates/headroom-proxy   — axum binary, /healthz only
  crates/headroom-py      — PyO3 cdylib, exposes headroom._core.hello()
  crates/headroom-parity  — Rust-vs-Python oracle harness + parity-run CLI

Tooling
  Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint
  .github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly
  deny.toml for cargo-deny

Parity corpus
  tests/parity/recorder.py + scripts/record_fixtures.py
  125 recorded fixtures across 5 leaf transforms (ccr, tokenizer,
  log_compressor, diff_compressor, cache_aligner)

Docs
  RUST_DEV.md — developer setup and workspace reference
  docs/spec/022-rust-migration.md — migration plan and stage breakdown

.gitignore: whitelist scripts/record_fixtures.py; ignore target/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 13:39:48 -07:00
Garm efd2ac1ca4 chore: renormalize line endings to LF 
`.gitattributes` declares `*.py text eol=lf` and `*.sh text eol=lf`, but
74 files (73 .py, 1 .sh) are stored in the index with CRLF line endings,
violating that contract. Every macOS/Linux clone reports these files as
"modified" on fresh checkout because git's diff engine sees the stored
bytes don't match the attribute contract, even though the working tree
and index match byte-for-byte.

Running `git add --renormalize .` rewrites each affected blob so the
stored form matches the attribute declaration. No semantic changes —
every affected file's diff is "N insertions, N deletions" with inserts
and deletes being the same lines modulo line endings.

Follow-up commit adds `.git-blame-ignore-revs` so `git blame` / GitHub
blame skip this mechanical commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 15:33:30 +02:00
JerrettDavis 297f499e37 ci: retry workflow validation dry-runs 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-23 13:20:13 -05:00
JerrettDavis 56f6307665 fix: support py310 version sync scripts 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 20:42:56 -05:00
JerrettDavis b852460af9 chore: normalize line endings in init diffs 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 20:17:14 -05:00
JerrettDavis a278a7b0ba test: cover init install flows end to end 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-21 20:15:11 -05:00