fix(providers/anthropic): don't crash token estimation on null tool_calls (#2472)
## Description
`AnthropicTokenCounter._count_message_estimated` (the
tiktoken-approximation fallback used when no Anthropic client is
available) counted OpenAI-format tool calls like this:
```python
if "tool_calls" in message:
for tool_call in message.get("tool_calls", []):
if isinstance(tool_call, dict):
func = tool_call.get("function", {})
...
```
The `if "tool_calls" in message` check only tests key presence, not the
value. OpenAI SDKs routinely include `"tool_calls": null` on an
assistant message with no tool calls, so `message.get("tool_calls", [])`
returned `None` (the default only applies when the key is absent) and
`for tool_call in None` raised `TypeError: 'NoneType' object is not
iterable`. That crashes token estimation for the entire request whenever
such a message is present. `tool_call.get("function", {})` had the same
gap for a `"function": null`.
## Fix
Iterate `message.get("tool_calls") or []` so a null or absent value
becomes an empty list, and read `function` with `or {}` for the same
reason. Valid tool calls are counted exactly as before.
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
- `headroom/providers/anthropic.py`: value-guard `tool_calls` and
`function` in `_count_message_estimated`.
- `tests/test_providers/test_anthropic.py`: regression counting a
message list that includes `tool_calls: null` and a tool call with
`function: null`.
## Testing
- [x] Unit tests pass (`pytest`)
- [x] Linting passes (`ruff check .`)
- [x] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed
### Test Output
```text
$ python -m pytest tests/test_providers/test_anthropic.py -q
17 passed
# with the fix reverted, the new test fails with
# TypeError: 'NoneType' object is not iterable
$ uvx ruff@0.15.17 check headroom/providers/anthropic.py tests/test_providers/test_anthropic.py
All checks passed!
$ uvx mypy@1.20.2 --ignore-missing-imports headroom/providers/anthropic.py
Success: no issues found in 1 source file
```
## Real Behavior Proof
- Environment: Windows 11, Python 3.12, project venv (`uv sync --extra
proxy`), `uvx ruff@0.15.17` / `uvx mypy@1.20.2`, pytest in the venv.
- Exact command / steps: built a real
`AnthropicTokenCounter('claude-3-5-sonnet-20241022')` and called
`count_messages` / `_count_message_estimated` with an assistant message
carrying `tool_calls: null` and one carrying `function: null`, plus a
valid tool call; then reverted `anthropic.py` and re-ran.
- Observed result: with the fix the null shapes count without error and
a valid tool call still adds its name/arguments tokens (5 -> 11 on the
sample); with the fix reverted the `tool_calls: null` message raises
`TypeError: 'NoneType' object is not iterable`. Ran against the actual
module.
- Not tested: a live request from an SDK that emits `tool_calls: null`,
end to end.
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md if applicable
This commit is contained in:
@@ -420,13 +420,14 @@ class AnthropicTokenCounter(TokenCounter):
|
||||
# str(block) catch-all would produce.
|
||||
tokens += count_content_blocks(content, self.count_text)
|
||||
|
||||
# OpenAI format tool calls
|
||||
if "tool_calls" in message:
|
||||
for tool_call in message.get("tool_calls", []):
|
||||
if isinstance(tool_call, dict):
|
||||
func = tool_call.get("function") or {}
|
||||
tokens += self.count_text(coerce_countable_text(func.get("name")))
|
||||
tokens += self.count_text(coerce_countable_text(func.get("arguments")))
|
||||
# OpenAI format tool calls. Guard the value, not just the key: an
|
||||
# OpenAI-format assistant message often carries `tool_calls: null` on a
|
||||
# no-tool turn, and `for ... in None` would raise TypeError.
|
||||
for tool_call in message.get("tool_calls") or []:
|
||||
if isinstance(tool_call, dict):
|
||||
func = tool_call.get("function") or {}
|
||||
tokens += self.count_text(coerce_countable_text(func.get("name")))
|
||||
tokens += self.count_text(coerce_countable_text(func.get("arguments")))
|
||||
|
||||
return tokens
|
||||
|
||||
|
||||
@@ -54,6 +54,18 @@ class TestAnthropicTokenCounting:
|
||||
count = counter.count_messages(messages)
|
||||
assert count > 0
|
||||
|
||||
def test_count_messages_tolerates_null_tool_calls(self, anthropic_provider):
|
||||
# OpenAI-format assistant messages routinely carry `tool_calls: null`
|
||||
# (and occasionally `function: null`) on a no-tool turn. The estimated
|
||||
# counter iterated the value after only a key-presence check, so it
|
||||
# raised `TypeError: 'NoneType' object is not iterable`.
|
||||
counter = anthropic_provider.get_token_counter("claude-3-5-sonnet-20241022")
|
||||
messages = [
|
||||
{"role": "assistant", "content": "hi", "tool_calls": None},
|
||||
{"role": "assistant", "content": "x", "tool_calls": [{"id": "a", "function": None}]},
|
||||
]
|
||||
assert counter.count_messages(messages) > 0
|
||||
|
||||
def test_count_text_allows_literal_special_tokens(self, anthropic_provider):
|
||||
counter = anthropic_provider.get_token_counter("claude-3-5-sonnet-20241022")
|
||||
count = counter.count_text("prefix <|fim_suffix|> suffix")
|
||||
|
||||
Reference in New Issue
Block a user