fix(litellm): add async_post_call_success_hook to HeadroomCallback (#1322)
## Description Pointing a litellm proxy at the Headroom callback blows up on the post-call success path: ``` type object 'HeadroomCallback' has no attribute 'async_post_call_success_hook' ``` litellm's logging contract calls `async_post_call_success_hook` after a successful response, and `HeadroomCallback` simply doesn't have it. We implement `async_pre_call_hook`, `async_success_handler` and `async_failure_handler`, but not this one, so litellm hits an `AttributeError` instead of a no-op and the whole request fails. This adds the missing `async_post_call_success_hook(self, data, user_api_key_dict, response)` matching litellm's signature. It returns `response` unchanged, the token accounting already lives in `async_success_handler` so there's nothing to do here except not crash. A few notes: 1. I did not make `HeadroomCallback` inherit litellm's `CustomLogger`, on purpose. The class keeps litellm as an optional dependency, so it stays a plain class and just provides the hooks litellm looks up by name. 2. It's a pass-through, so it's safe regardless of what the response contains. Closes #1114 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/integrations/litellm_callback.py`: add `async_post_call_success_hook` to `HeadroomCallback`, returning the response unchanged; update the class docstring to list the full set of litellm hooks. - `tests/test_integrations/test_litellm_callback.py`: new tests that the method exists, is a coroutine, and returns the response untouched; build the module path with `pathlib` instead of a fragile `__file__.replace`. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ uv run --extra dev python -m pytest tests/test_integrations/test_litellm_callback.py -q 3 passed ruff: All checks passed! mypy: Success: no issues found ``` ## Real Behavior Proof - Environment: macOS, Python 3.13, this branch. - Exact command / steps: `uv run --extra dev python -m pytest tests/test_integrations/test_litellm_callback.py -q`. The tests import the callback module directly and resolve `async_post_call_success_hook` by name, the same way litellm does, then await it with a sentinel response. - Observed result: 3 passed. The hook exists, is a coroutine, and returns the exact response object it was given. Before the fix, resolving the attribute raised `AttributeError`. - Not tested: I did not stand up a full litellm proxy end to end. The fix is the missing hook method, which the unit tests cover. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes --------- Co-authored-by: JD Davis <mxjerrett@gmail.com>
This commit is contained in:
@@ -33,6 +33,10 @@ except ImportError: # litellm not installed — fall back to plain object
|
||||
class HeadroomCallback(_CustomLogger):
|
||||
"""LiteLLM callback that compresses messages before each API call.
|
||||
|
||||
Implements the LiteLLM callback hooks looked up by name:
|
||||
``async_pre_call_hook``, ``async_post_call_success_hook``,
|
||||
``async_success_handler`` and ``async_failure_handler``.
|
||||
|
||||
Inherits from litellm.integrations.custom_logger.CustomLogger so that
|
||||
any hook LiteLLM adds in future versions (e.g. async_post_call_success_hook
|
||||
added in 1.89.x) has a no-op default and won't raise AttributeError (#1114).
|
||||
@@ -192,6 +196,15 @@ class HeadroomCallback(_CustomLogger):
|
||||
result: dict[str, Any] = resp.json()
|
||||
return result
|
||||
|
||||
async def async_post_call_success_hook(
|
||||
self,
|
||||
data: dict[str, Any],
|
||||
user_api_key_dict: Any,
|
||||
response: Any,
|
||||
) -> Any:
|
||||
"""Called by the LiteLLM proxy after a successful call. Returns response unchanged."""
|
||||
return response
|
||||
|
||||
async def async_success_handler(
|
||||
self, kwargs: dict, response: Any, start_time: Any, end_time: Any
|
||||
) -> None:
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Tests for headroom.integrations.litellm_callback."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import inspect
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _import_callback() -> type:
|
||||
# Import the module directly to avoid triggering headroom/integrations/__init__.py
|
||||
# which pulls in langchain and the native .so extension.
|
||||
module_path = (
|
||||
Path(__file__).resolve().parents[2] / "headroom" / "integrations" / "litellm_callback.py"
|
||||
)
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"headroom.integrations.litellm_callback",
|
||||
module_path,
|
||||
)
|
||||
assert spec is not None and spec.loader is not None
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod) # type: ignore[union-attr]
|
||||
return mod.HeadroomCallback # type: ignore[attr-defined]
|
||||
|
||||
|
||||
HeadroomCallback = _import_callback()
|
||||
|
||||
|
||||
class TestHeadroomCallbackPostCallSuccessHook:
|
||||
"""async_post_call_success_hook must exist and return response unchanged."""
|
||||
|
||||
def test_method_exists(self) -> None:
|
||||
cb = HeadroomCallback()
|
||||
assert hasattr(cb, "async_post_call_success_hook"), (
|
||||
"HeadroomCallback must define async_post_call_success_hook "
|
||||
"for LiteLLM proxy compatibility"
|
||||
)
|
||||
|
||||
def test_method_is_coroutine(self) -> None:
|
||||
cb = HeadroomCallback()
|
||||
assert inspect.iscoroutinefunction(cb.async_post_call_success_hook)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_returns_response_unchanged(self) -> None:
|
||||
cb = HeadroomCallback()
|
||||
sentinel = object()
|
||||
result = await cb.async_post_call_success_hook(
|
||||
data={},
|
||||
user_api_key_dict=None,
|
||||
response=sentinel,
|
||||
)
|
||||
assert result is sentinel
|
||||
Reference in New Issue
Block a user