github-featured / chopratejas--headroom

发布

0 发布 170 Git 标签

RSS 订阅源
  • v0.31.0 55efb1c77d
    比较

    fix(proxy): keep OpenAI tool observations mutable in cache mode (#1884)

    frostbyte_neo 发布于 2026-07-09 14:51:01 +00:00

    Description

    Diagnoses and fixes the low-savings OpenAI-compatible cache-mode path
    reported in #1696.

    OpenAI-compatible tool-calling clients can end a turn with role: "tool" (or legacy role: "function") rather than role: "user". The
    OpenAI chat handler's cache-mode freeze boundary treated those tails as
    non-mutable, and because HeadroomProxy resolves
    _strict_previous_turn_frozen_count from the Anthropic mixin first, the
    OpenAI-specific helper was not used in production. That froze the entire
    conversation before ContentRouter ran, leaving no live tool
    observation to compress and producing near-pass-through savings on long
    coding sessions.

    This PR keeps final OpenAI tool/function observations mutable in cache
    mode, explicitly calls the OpenAI helper to avoid the mixin-name
    collision, and clamps negative token-savings artifacts at the
    metrics/cost aggregation boundary so stats cannot under-report actual
    forwarded savings.

    Closes #1696

    Type of Change

    • Bug fix (non-breaking change that fixes an issue)
    • New feature (non-breaking change that adds functionality)
    • Breaking change (fix or feature that would cause existing
      functionality to change)
    • Documentation update
    • Performance improvement
    • Code refactoring (no functional changes)

    Changes Made

    • Treat final OpenAI user, tool, and function messages as the
      mutable cache-mode live zone.
    • Route OpenAI cache-boundary calls through
      OpenAIHandlerMixin._strict_previous_turn_frozen_count explicitly so
      the Anthropic mixin method cannot shadow it in HeadroomProxy's MRO.
    • Preserve cache-mode live-tail boundaries even when compression-cache
      state would otherwise freeze the whole request.
    • Clamp negative tokens_saved artifacts in CostTracker.record_tokens
      and PrometheusMetrics.record_request.
    • Add regression coverage for OpenAI final tool/function tails,
      over-frozen tracker state, and non-negative savings aggregation.

    Testing

    • Unit tests pass (pytest)
    • Linting passes (ruff check .)
    • Type checking passes (mypy headroom)
    • New tests added for new functionality
    • Manual testing performed

    Test Output

    $ maturin build --profile ci --out dist --interpreter python
Built wheel for abi3 Python >= 3.10 to dist\headroom_ai-0.29.0-cp310-abi3-win_amd64.whl

$ python -m pytest tests\test_proxy_handler_helpers.py tests\test_proxy_openai_cache_stability.py tests\test_observability_metrics.py tests\test_cost_tracker_counterfactual.py
49 passed in 10.27s

$ python -m ruff check .
All checks passed!

$ python -m mypy headroom
Success: no issues found in 407 source files

$ python -m pytest
53 failed, 7703 passed, 488 skipped, 5893 warnings, 131 errors in 595.18s (0:09:55)

    Full-suite note: the full local pytest run was attempted on
    Windows/Python 3.13 after building headroom._core. It did not complete
    green due to broad pre-existing/local-environment failures outside this
    change area, dominated by SQLite/memory persistence permission/path
    errors plus unrelated adapter/cache/tool tests. The focused regression
    suite for this PR passes, and repo-level lint/type gates pass.

    Real Behavior Proof

    • Environment: Windows, Python 3.13.13, Rust/Cargo available, local
      headroom._core wheel built with maturin build --profile ci.
    • Exact command / steps: ran the OpenAI cache-stability tests with final
      role: "tool" and role: "function" chat tails.
    • Observed result:
      test_openai_cache_mode_keeps_final_tool_observation_mutable[tool] and
      [function] pass, proving the pipeline receives frozen_message_count == 2 for a 3-message request instead of freezing all 3 messages.
    • Not tested: live Lemonade/KiloCode upstream session; no local Lemonade
      Server was available.

    Review Readiness

    • I have performed a self-review
    • This PR is ready for human review

    Checklist

    • My code follows the project's style guidelines
    • I have performed a self-review of my code
    • I have commented my code, particularly in hard-to-understand areas
    • I have made corresponding changes to the documentation
    • My changes generate no new warnings
    • I have added tests that prove my fix is effective or that my
      feature works
    • New and existing unit tests pass locally with my changes
    • I have updated the CHANGELOG.md if applicable

    Screenshots (if applicable)

    N/A

    Additional Notes

    Docs and CHANGELOG are N/A for this narrow proxy bug fix. The broad
    local pytest checkbox is intentionally left unchecked because the full
    suite had unrelated local-environment failures; see the test output
    above. Focused regression tests, ruff check ., and mypy headroom are
    green.

    下载附件