Files
microsoft--agent-framework/python/packages/durabletask/tests/test_shim.py
Eduard van Valkenburg 6e95517659 Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) (#6443)
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)

Rework the typing setup along the lines of the 'too many type checkers'
approach:

- Pyright (strict) is now the sole source-code type checker; mypy is
  removed from source and its [tool.mypy] block becomes a relaxed profile
  used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
  ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
  basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
  pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
  '# type: ignore' on both branches so results match across interpreter
  versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
  gating checkers and pyright on source+tests+samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix merge regressions from main (typing + runtime)

Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:

Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
  _parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
  ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
  return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
  output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
  contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
  ExecutionMode used by the processor tests.

Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
  observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
  dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
  payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.

Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Isolate per-package mypy cache in test-typing fan-out

The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.

Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Make lab pyright-only on source (drop source mypy)

Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:

- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
  now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.

Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix test-typing regressions from latest main merge

A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:

- core: narrow Optional span.attributes with 'and' guards in span filters
  and assert+cast the json.loads(...attributes[...]) reads (test_observability);
  match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
  pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
  .annotations[0] access through a small _first_annotation helper (mirrors the
  file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
  (zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
  and connections.get_default (zuban) SDK type gaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated pyright version

* pyright fix

* Python: Fix source typing for pyright 1.1.410

Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:

- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
  AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
  the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
  Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
  reduce(and_, ...), which pyright could no longer fully type (drops the now
  unused functools.reduce / operator.and_ imports).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Accept plain-text body in Azure Functions workflow/run endpoint

The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.

Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 15:06:20 +00:00

229 lines
9.5 KiB
Python

# Copyright (c) Microsoft. All rights reserved.
"""Unit tests for DurableAIAgent shim and DurableAgentProvider.
Focuses on critical message normalization, delegation, and protocol compliance.
Run with: pytest tests/test_shim.py -v
"""
from typing import Any, cast
from unittest.mock import Mock
import pytest
from agent_framework import Message, SupportsAgentRun
from pydantic import BaseModel
from agent_framework_durabletask import DurableAgentSession
from agent_framework_durabletask._executors import DurableAgentExecutor
from agent_framework_durabletask._models import RunRequest
from agent_framework_durabletask._shim import DurableAgentProvider, DurableAIAgent
class ResponseFormatModel(BaseModel):
"""Test Pydantic model for response format testing."""
result: str
@pytest.fixture
def mock_executor() -> Mock:
"""Create a mock executor for testing."""
mock = Mock(spec=DurableAgentExecutor)
mock.run_durable_agent = Mock(return_value=None)
mock.get_new_session = Mock(return_value=DurableAgentSession())
# Mock get_run_request to create actual RunRequest objects
def create_run_request(
message: str,
options: dict[str, Any] | None = None,
) -> RunRequest:
import uuid
opts = dict(options) if options else {}
response_format = opts.pop("response_format", None)
enable_tool_calls = cast("bool", opts.pop("enable_tool_calls", True))
wait_for_response = cast("bool", opts.pop("wait_for_response", True))
return RunRequest(
message=message,
correlation_id=str(uuid.uuid4()),
response_format=response_format,
enable_tool_calls=enable_tool_calls,
wait_for_response=wait_for_response,
options=opts,
)
mock.get_run_request = Mock(side_effect=create_run_request)
return mock
@pytest.fixture
def test_agent(mock_executor: Mock) -> DurableAIAgent[Any]:
"""Create a test agent with mock executor."""
return DurableAIAgent(mock_executor, "test_agent")
class TestDurableAIAgentMessageNormalization:
"""Test that DurableAIAgent properly normalizes various message input types."""
def test_run_accepts_string_message(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run accepts and normalizes string messages."""
test_agent.run("Hello, world!")
mock_executor.run_durable_agent.assert_called_once()
# Verify agent_name and run_request were passed correctly as kwargs
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["agent_name"] == "test_agent"
assert kwargs["run_request"].message == "Hello, world!"
def test_run_accepts_chat_message(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run accepts and normalizes Message objects."""
chat_msg = Message(role="user", contents=["Test message"])
test_agent.run(chat_msg)
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].message == "Test message"
def test_run_accepts_list_of_strings(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run accepts and joins list of strings."""
test_agent.run(["First message", "Second message"])
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].message == "First message\nSecond message"
def test_run_accepts_list_of_chat_messages(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run accepts and joins list of Message objects."""
messages = [
Message(role="user", contents=["Message 1"]),
Message(role="assistant", contents=["Message 2"]),
]
test_agent.run(messages)
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].message == "Message 1\nMessage 2"
def test_run_handles_none_message(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run handles None message gracefully."""
test_agent.run(None)
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].message == ""
def test_run_handles_empty_list(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run handles empty list gracefully."""
test_agent.run([])
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].message == ""
class TestDurableAIAgentParameterFlow:
"""Test that parameters flow correctly through the shim to executor."""
def test_run_forwards_session_parameter(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run forwards session parameter to executor."""
session = DurableAgentSession(service_session_id="test-session")
test_agent.run("message", session=session)
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["session"] == session
def test_run_forwards_response_format(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify run forwards response_format parameter to executor."""
test_agent.run("message", options={"response_format": ResponseFormatModel})
mock_executor.run_durable_agent.assert_called_once()
_, kwargs = mock_executor.run_durable_agent.call_args
assert kwargs["run_request"].response_format == ResponseFormatModel
class TestDurableAISupportsAgentRunCompliance:
"""Test that DurableAIAgent implements SupportsAgentRun correctly."""
def test_agent_implements_protocol(self, test_agent: DurableAIAgent[Any]) -> None:
"""Verify DurableAIAgent implements SupportsAgentRun."""
assert isinstance(test_agent, SupportsAgentRun) # pyrefly: ignore[unsafe-overlap]
def test_agent_has_required_properties(self, test_agent: DurableAIAgent[Any]) -> None:
"""Verify DurableAIAgent has all required SupportsAgentRun properties."""
assert hasattr(test_agent, "id")
assert hasattr(test_agent, "name")
assert hasattr(test_agent, "display_name")
assert hasattr(test_agent, "description")
def test_agent_id_defaults_to_name(self, mock_executor: Mock) -> None:
"""Verify agent id defaults to name when not provided."""
agent: DurableAIAgent[Any] = DurableAIAgent(mock_executor, "my_agent")
assert agent.id == "my_agent"
assert agent.name == "my_agent"
def test_agent_id_can_be_customized(self, mock_executor: Mock) -> None:
"""Verify agent id can be set independently from name."""
agent: DurableAIAgent[Any] = DurableAIAgent(mock_executor, "my_agent", agent_id="custom-id")
assert agent.id == "custom-id"
assert agent.name == "my_agent"
class TestDurableAIAgentSessionManagement:
"""Test session creation and management."""
def test_create_session_delegates_to_executor(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify create_session delegates to executor."""
mock_session = DurableAgentSession()
mock_executor.get_new_session.return_value = mock_session
session = test_agent.create_session()
mock_executor.get_new_session.assert_called_once_with("test_agent")
assert session == mock_session
def test_get_session_forwards_service_session_id(
self, test_agent: DurableAIAgent[Any], mock_executor: Mock
) -> None:
"""Verify get_session forwards service_session_id and session_id to executor."""
mock_session = DurableAgentSession(service_session_id="svc-123")
mock_executor.get_new_session.return_value = mock_session
session = test_agent.get_session("svc-123", session_id="local-456")
mock_executor.get_new_session.assert_called_once_with(
"test_agent", service_session_id="svc-123", session_id="local-456"
)
assert session.service_session_id == "svc-123"
def test_get_session_without_session_id(self, test_agent: DurableAIAgent[Any], mock_executor: Mock) -> None:
"""Verify get_session works with only service_session_id (session_id defaults to None)."""
mock_session = DurableAgentSession(service_session_id="svc-789")
mock_executor.get_new_session.return_value = mock_session
session = test_agent.get_session("svc-789")
mock_executor.get_new_session.assert_called_once_with(
"test_agent", service_session_id="svc-789", session_id=None
)
assert session.service_session_id == "svc-789"
class TestDurableAgentProviderInterface:
"""Test that DurableAgentProvider defines the correct interface."""
def test_provider_cannot_be_instantiated(self) -> None:
"""Verify DurableAgentProvider is abstract and cannot be instantiated."""
with pytest.raises(TypeError):
DurableAgentProvider() # type: ignore[abstract]
def test_provider_defines_get_agent_method(self) -> None:
"""Verify DurableAgentProvider defines get_agent abstract method."""
assert hasattr(DurableAgentProvider, "get_agent")
if __name__ == "__main__":
pytest.main([__file__, "-v", "--tb=short"])