6e95517659
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) Rework the typing setup along the lines of the 'too many type checkers' approach: - Pyright (strict) is now the sole source-code type checker; mypy is removed from source and its [tool.mypy] block becomes a relaxed profile used only for tests/samples. - Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly, ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/ basic profile so authors aren't forced into over-annotation. - Add pyrightconfig.tests.json and bump sample pyright configs to basic. - Unify test/sample typing onto the same parallel fan-out used by source pyright via run_command_items in task_runner.py. - Make version-conditional imports symmetric: keep or drop the '# type: ignore' on both branches so results match across interpreter versions (local vs CI). - Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five gating checkers and pyright on source+tests+samples. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Fix merge regressions from main (typing + runtime) Merging main into the type-checker split branch surfaced regressions that the new five-checker test suite and unit tests caught: Runtime fixes: - anthropic: restore the dropped `cache_read_input_token_count` mapping in _parse_usage_from_anthropic (lost during merge conflict resolution). - gemini: _get_function_calling_mode test helper returned str(enum) ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO'). - openai: _response_id_from_token test helper was an infinite self-recursion; return token['response_id']. - orchestrations: reset output_events per approval iteration so the terminal output assertion counts only the final run. - core: drop a stale duplicate harness test whose message ('non-negative') contradicted the source ('positive'). - purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/ ExecutionMode used by the processor tests. Type-checker fixes (tests, relaxed profile): - core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP, observability and types tests. - anthropic/openai: route provider-namespaced UsageDetails keys through a dict cast (extra_items TypedDict unsupported by mypy/ty). - purview: typed model constructors and cache-mock casts. - ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test payloads, guard Optional forwarded_props, and ty-ignore intentional bad args. Source pyright (sole source checker) flagged unnecessary ignores newly introduced by merged code in core _tools.py and declarative _declarative_base.py. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Isolate per-package mypy cache in test-typing fan-out The parallel test-typing fan-out runs many mypy processes concurrently, all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on worker timing) -- which is why CI's Test Typing job failed on a shifting set of packages while a single-package run was fine. Give each mypy invocation an isolated cache dir keyed by its target paths so incremental caching still works per package without races. Other checkers (zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Make lab pyright-only on source (drop source mypy) Lab was the last package still running mypy on its source code, requiring mypy-only `# type: ignore` comments that pyright (the sole source checker everywhere else) flags as unnecessary. Align lab with the rest of the monorepo: - Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the now-dead strict [tool.mypy] config block. - Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only. Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly, ty, zuban, pyright over tests using the relaxed root config). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Fix test-typing regressions from latest main merge A fresh merge from main brought in new test code never run under the five-checker test-typing suite. Green up across the affected packages: - core: narrow Optional span.attributes with 'and' guards in span filters and assert+cast the json.loads(...attributes[...]) reads (test_observability); match the existing as_agent ignore on the protocol-typed fixture (test_clients). - openai: align new streaming tests with the established chat_options dict pattern (ChatOptions TypedDict isn't assignable to dict), route Optional .annotations[0] access through a small _first_annotation helper (mirrors the file's assert-not-None convention), and annotate a mapped ResponseStream. - foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {} (zuban needs the annotation). - foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly) and connections.get_default (zuban) SDK type gaps. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * updated pyright version * pyright fix * Python: Fix source typing for pyright 1.1.410 Pyright 1.1.410 tightened several checks. Apply the same source fixes as upstream PR #6275: - anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from the anthropic top-level package -> reportPrivateImportUsage). - core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType). - core _workflows/_events.py: annotate the @contextmanager helper as Generator[None] instead of Iterator[None] (reportDeprecated). - redis: build the combined filter expression with an explicit loop instead of reduce(and_, ...), which pyright could no longer fully type (drops the now unused functools.reduce / operator.and_ imports). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Python: Accept plain-text body in Azure Functions workflow/run endpoint The workflow_orchestrator already accepts plain strings as well as JSON objects via context.get_input(), but the start_workflow_orchestration HTTP handler only accepted JSON and returned 400 for any non-JSON body. This made the functions integration tests that POST text/plain to /api/workflow/run (e.g. test_09_workflow_shared_state) fail consistently with 400 != 202. Fall back to the raw request body (decoded as UTF-8) when the body is not JSON, rejecting only a truly empty body. The JSON path is unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
157 lines
6.0 KiB
Python
157 lines
6.0 KiB
Python
# Copyright (c) Microsoft. All rights reserved.
|
|
|
|
"""Tests for Agent Framework to ChatKit streaming utilities."""
|
|
|
|
from collections.abc import AsyncIterator
|
|
from unittest.mock import Mock
|
|
|
|
from agent_framework import AgentResponseUpdate, Content
|
|
from chatkit.types import (
|
|
AssistantMessageContent,
|
|
AssistantMessageContentPartTextDelta,
|
|
AssistantMessageItem,
|
|
ThreadItemAddedEvent,
|
|
ThreadItemDoneEvent,
|
|
ThreadItemUpdated,
|
|
)
|
|
|
|
from agent_framework_chatkit import stream_agent_response
|
|
|
|
|
|
class TestStreamAgentResponse:
|
|
"""Tests for stream_agent_response function."""
|
|
|
|
async def test_stream_empty_response(self):
|
|
"""Test streaming empty response."""
|
|
|
|
async def empty_stream() -> AsyncIterator[AgentResponseUpdate]:
|
|
return
|
|
yield # Make it a generator
|
|
|
|
events = []
|
|
async for event in stream_agent_response(empty_stream(), thread_id="test_thread"):
|
|
events.append(event)
|
|
|
|
assert len(events) == 0
|
|
|
|
async def test_stream_single_text_update(self):
|
|
"""Test streaming single text update."""
|
|
|
|
async def single_update_stream():
|
|
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello world")])
|
|
|
|
events = []
|
|
async for event in stream_agent_response(single_update_stream(), thread_id="test_thread"):
|
|
events.append(event)
|
|
|
|
# Should have: item_added, item_updated (delta), item_done
|
|
assert len(events) == 3
|
|
|
|
# Check event types
|
|
assert isinstance(events[0], ThreadItemAddedEvent)
|
|
assert isinstance(events[1], ThreadItemUpdated)
|
|
assert isinstance(events[2], ThreadItemDoneEvent)
|
|
|
|
# Check delta event
|
|
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
|
|
assert events[1].update.delta == "Hello world"
|
|
|
|
# Check final message content
|
|
assert isinstance(events[2].item, AssistantMessageItem)
|
|
assert len(events[2].item.content) == 1
|
|
assert isinstance(events[2].item.content[0], AssistantMessageContent)
|
|
assert events[2].item.content[0].text == "Hello world"
|
|
|
|
async def test_stream_multiple_text_updates(self):
|
|
"""Test streaming multiple text updates."""
|
|
|
|
async def multiple_updates_stream():
|
|
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Hello ")])
|
|
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="world!")])
|
|
|
|
events = []
|
|
async for event in stream_agent_response(multiple_updates_stream(), thread_id="test_thread"):
|
|
events.append(event)
|
|
|
|
# Should have: item_added, item_updated (delta 1), item_updated (delta 2), item_done
|
|
assert len(events) == 4
|
|
|
|
# Check event types
|
|
assert isinstance(events[0], ThreadItemAddedEvent)
|
|
assert isinstance(events[1], ThreadItemUpdated)
|
|
assert isinstance(events[2], ThreadItemUpdated)
|
|
assert isinstance(events[3], ThreadItemDoneEvent)
|
|
|
|
# Check delta events
|
|
assert isinstance(events[1].update, AssistantMessageContentPartTextDelta)
|
|
assert isinstance(events[2].update, AssistantMessageContentPartTextDelta)
|
|
assert events[1].update.delta == "Hello "
|
|
assert events[2].update.delta == "world!"
|
|
|
|
# Check final accumulated text
|
|
final_message_event = events[-1]
|
|
assert isinstance(final_message_event, ThreadItemDoneEvent)
|
|
assert isinstance(final_message_event.item, AssistantMessageItem)
|
|
assert isinstance(final_message_event.item.content[0], AssistantMessageContent)
|
|
assert final_message_event.item.content[0].text == "Hello world!"
|
|
|
|
async def test_stream_with_custom_id_generator(self):
|
|
"""Test streaming with custom ID generator."""
|
|
|
|
def custom_id_generator(item_type: str) -> str:
|
|
return f"custom_{item_type}_123"
|
|
|
|
async def single_update_stream():
|
|
yield AgentResponseUpdate(role="assistant", contents=[Content.from_text(text="Test")])
|
|
|
|
events = []
|
|
async for event in stream_agent_response(
|
|
single_update_stream(), thread_id="test_thread", generate_id=custom_id_generator
|
|
):
|
|
events.append(event)
|
|
|
|
# Check that custom IDs are used
|
|
message_added_event = events[0]
|
|
assert isinstance(message_added_event, ThreadItemAddedEvent)
|
|
assert message_added_event.item.id == "custom_msg_123"
|
|
|
|
async def test_stream_empty_content_updates(self):
|
|
"""Test streaming updates with empty content."""
|
|
|
|
async def empty_content_stream():
|
|
yield AgentResponseUpdate(role="assistant", contents=[])
|
|
yield AgentResponseUpdate(role="assistant", contents=None)
|
|
|
|
events = []
|
|
async for event in stream_agent_response(empty_content_stream(), thread_id="test_thread"):
|
|
events.append(event)
|
|
|
|
# Should have item_added and item_done
|
|
assert len(events) == 2
|
|
assert isinstance(events[0], ThreadItemAddedEvent)
|
|
assert isinstance(events[1], ThreadItemDoneEvent)
|
|
|
|
# Final message should have empty content
|
|
assert isinstance(events[1].item, AssistantMessageItem)
|
|
assert len(events[1].item.content) == 0
|
|
|
|
async def test_stream_non_text_content(self):
|
|
"""Test streaming updates with non-text content."""
|
|
# Mock a content object without text attribute
|
|
non_text_content = Mock(spec=Content)
|
|
non_text_content.type = "image"
|
|
# Don't set text attribute
|
|
non_text_content.text = None
|
|
|
|
async def non_text_stream():
|
|
yield AgentResponseUpdate(role="assistant", contents=[non_text_content])
|
|
|
|
events = []
|
|
async for event in stream_agent_response(non_text_stream(), thread_id="test_thread"):
|
|
events.append(event)
|
|
|
|
# Should have item_added and item_done, but no content since no text
|
|
assert len(events) == 2
|
|
assert isinstance(events[0], ThreadItemAddedEvent)
|
|
assert isinstance(events[1], ThreadItemDoneEvent)
|