* Python: Preserve structured instructions when merging chat options
`instructions` is declared as `str` on `ChatOptions`, but chat clients may widen it
to a provider-native structured form. Three merge paths combined it with an f-string,
which coerced any non-string value to its `repr`, turning structured metadata into
literal text before any client could see it:
- `merge_chat_options` (`_types.py`)
- `_merge_options` (`_agents.py`, agent defaults + per-run options)
- provider-contributed instructions in `_prepare_session_and_messages` (`_agents.py`)
The last of these is the reported case: once any context provider (for example
`SkillsProvider`) contributes instructions, structured instructions were replaced by
their `repr`, so the model received Python dict syntax as its system prompt and
Anthropic prompt caching silently stopped working.
Add a shared `_append_instructions` helper that concatenates strings as before and
otherwise extends element-wise, always appending so the leading portion stays
unchanged for providers that treat it as a stable, structure-sensitive prefix. A lone
mapping is treated as a single element rather than iterated into its keys.
On the Anthropic side, `_extract_structured_instructions` now normalizes bare strings
into text blocks, since appended instructions arrive alongside caller-supplied blocks.
Fixes#7700
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f428b149-a306-484a-b423-4e9de69f0b90
* Python: address review feedback on structured instructions fix
Parameterize the Anthropic regression test over both the with- and
without-SkillsProvider configurations so the structure-preserving behavior
is asserted in the baseline case too.
Normalize structured instructions in `_get_instructions_from_options` so
telemetry records the instruction text for provider-native block shapes,
extracting only `text` values to keep provider metadata out of spans.
Use `cast` for the structured `default_options` in both regression tests so
the test type checkers resolve the client options type correctly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: f428b149-a306-484a-b423-4e9de69f0b90
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
Copilot-Session: f428b149-a306-484a-b423-4e9de69f0b90
* Python: fix Anthropic streaming double-counting token usage
* Python: address review on the Anthropic usage increment helper
- accumulate the emitted totals in a plain dict instead of string-cast
TypedDict views, so static checkers see real types throughout
- compute the increment through _types.add_usage_details with negated
emitted totals instead of a hand-rolled subtraction loop; keys absent
from a snapshot stay untouched, matching the partial-delta semantics
* Python: fix per-run additional_beta_flags leaking into Anthropic request kwargs
_prepare_options copied every key from the caller-supplied options dict
into run_options except "instructions" and "response_format". A
per-run additional_beta_flags value is correctly folded into the betas
set by _prepare_betas, but the raw key was never excluded, so it
survived into run_options and was forwarded straight through to
AsyncMessages.create(), which rejects it with TypeError: got an
unexpected keyword argument 'additional_beta_flags'. Add it to the
exclusion set alongside the other framework-level keys.
Fixes#5764
* Exclude additional_beta_flags from filtered_kwargs too
Copilot's review on the original fix pointed out the exclusion only
covered the options-dict copy, not kwargs passed directly to
_prepare_options — so additional_beta_flags supplied as a raw kwarg
would still leak through and reproduce the same TypeError. Add the
same exclusion to filtered_kwargs for consistency, with a regression
test covering the kwarg path.
---------
Co-authored-by: Chris Brown <albatrossflyon1@gmail.com>
Bug
---
`RawAnthropicClient._prepare_options` forwards `response_format` as the
**deprecated** beta parameter `output_format={"type": "json_schema", "schema":
{...}}` plus the beta flag `structured-outputs-2025-11-13`. When the same
request also includes `tools`, Claude emits concatenated / malformed JSON —
e.g. three copies of the schema's empty default like
`{"matches":[]}{"matches":[]}{"matches":[]}` — instead of populating the
schema. Anthropic's GA shape — `output_config={"format": {"type":
"json_schema", "schema": {...}}}` — works correctly with tools.
Verified empirically on `agent-framework-anthropic` against
`claude-sonnet-4-6` for a structured-output workload that combined
`response_format` with a tool (`run_shell`); the deprecated path produced
the malformed concatenated output, the GA path did not.
Changes
-------
- Move `response_format` into `run_options["output_config"]["format"]` and
stop adding the `structured-outputs-2025-11-13` beta flag (the GA path
doesn't need it).
- Merge the format into any caller-supplied `output_config` so e.g.
`output_config["effort"]` (adaptive-thinking effort level) survives the
transformation.
- Drop the now-unused `STRUCTURED_OUTPUTS_BETA_FLAG` constant (private to
this module — no external callers).
- `_prepare_response_format` keeps the same `{"type": "json_schema",
"schema": ...}` return shape; the docstring is updated to point at the
GA target.
Test plan
---------
- `uv run pytest packages/anthropic/tests` → 130 passed.
- New tests:
- `test_prepare_options_uses_output_config_for_response_format` — the
GA `output_config.format` shape is emitted, the deprecated
`output_format` key is not, and the `structured-outputs-2025-11-13`
beta flag is not added.
- `test_prepare_options_preserves_caller_supplied_output_config_effort`
— a caller-supplied `output_config["effort"]` survives the merge.
- `test_prepare_options_no_response_format_omits_output_config` — no
`output_config` is added implicitly when `response_format` is absent.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Fix auto function calling stripping explicit null arguments (fixes#5934)
* fix: re-role trailing assistant message to user for Anthropic (fixes#5008)
* fix: address Copilot review feedback (exclude_unset, test coverage, synthetic user turn)
* fix: update docstring and extend exclude_unset to auto_invoke_function
* revert: remove unrelated core _tools.py changes from Anthropic PR
The exclude_none/exclude_unset changes in the core package are out of scope
for this Anthropic-specific fix. This PR now only contains the Anthropic
chat client docstring fix and the synthetic user turn append.
* fix: avoid appending user turn after Anthropic tool use
* Fix Anthropic tool-use type narrowing
Use object-typed content narrowing before checking Anthropic tool-use block types so strict Pyright no longer treats dynamic message content as Unknown.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)
Rework the typing setup along the lines of the 'too many type checkers'
approach:
- Pyright (strict) is now the sole source-code type checker; mypy is
removed from source and its [tool.mypy] block becomes a relaxed profile
used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
'# type: ignore' on both branches so results match across interpreter
versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
gating checkers and pyright on source+tests+samples.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix merge regressions from main (typing + runtime)
Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:
Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
_parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
ExecutionMode used by the processor tests.
Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.
Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Isolate per-package mypy cache in test-typing fan-out
The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.
Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Make lab pyright-only on source (drop source mypy)
Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:
- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.
Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Fix test-typing regressions from latest main merge
A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:
- core: narrow Optional span.attributes with 'and' guards in span filters
and assert+cast the json.loads(...attributes[...]) reads (test_observability);
match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
.annotations[0] access through a small _first_annotation helper (mirrors the
file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
(zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
and connections.get_default (zuban) SDK type gaps.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated pyright version
* pyright fix
* Python: Fix source typing for pyright 1.1.410
Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:
- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
reduce(and_, ...), which pyright could no longer fully type (drops the now
unused functools.reduce / operator.and_ imports).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Accept plain-text body in Azure Functions workflow/run endpoint
The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.
Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(anthropic): add base_url parameter to AnthropicClient and RawAnthropicClient
Add base_url support to AnthropicSettings TypedDict, RawAnthropicClient,
and AnthropicClient so users can point the client at Foundry or other
Anthropic-compatible endpoints without having to construct AsyncAnthropic
manually.
- Add base_url field to AnthropicSettings (resolved from ANTHROPIC_BASE_URL env var)
- Add base_url parameter to RawAnthropicClient.__init__ and pass it to AsyncAnthropic
- Add base_url parameter to AnthropicClient.__init__ and forward to super
- Add unit tests for base_url on both client classes
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: Add `base_url` parameter to `AnthropicClient` and `RawAnthropicClient`
Fixes#5683
* test: add ANTHROPIC_BASE_URL env fallback tests for issue #5683
Add unit tests verifying that both AnthropicClient and RawAnthropicClient
pick up base_url from the ANTHROPIC_BASE_URL environment variable via
load_settings when base_url is not passed explicitly as a constructor arg.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test(anthropic): explicit base_url kwarg beats ANTHROPIC_BASE_URL env var (#5683)
Add regression tests asserting that when both ANTHROPIC_BASE_URL is set
in the environment *and* an explicit base_url kwarg is passed to
AnthropicClient / RawAnthropicClient, the explicit kwarg wins.
This closes the priority-ordering contract (explicit arg > env var) that
the existing tests left implicit.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The local MCP server can't be used for hosted tools tests because
Anthropic's backend needs to reach the MCP URL from their infrastructure
(not localhost on the CI runner). Revert to learn.microsoft.com/api/mcp
but catch BadRequestError, InternalServerError, APIConnectionError, and
APITimeoutError and pytest.skip so upstream outages don't block the
merge queue.
* Python: use local MCP server for hosted tools test and broaden image assertion
The hosted tools integration test was hitting rate limits on the external
learn.microsoft.com MCP server, causing persistent failures that retries
couldn't recover from. Switch to the local MCP server already spun up in
CI via LOCAL_MCP_URL, skipping when the env var isn't set.
Also broaden the image description assertion to accept common synonyms
(cottage, mansion, villa, etc.) instead of just "house", since the model
legitimately uses varied vocabulary for the same image.
* Address review feedback: validate LOCAL_MCP_URL scheme and use word boundaries
- Skip hosted tools test when LOCAL_MCP_URL lacks http/https scheme,
matching the pattern used in test_mcp.py.
- Use regex word boundaries for image assertion to avoid false matches
like "villain" matching "villa".
* Refactor Anthropic model option and provider clients
Rename the Anthropic client model option from model_id to model, add provider-specific Anthropic wrappers for Foundry, Bedrock, and Vertex, and expose them through the Anthropic, Foundry, Amazon, and Google namespaces. Update core option handling, docs, samples, and tests accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix Anthropic skills sample typing
Cast the Anthropic beta client to Any in the skills sample so the pre-commit sample pyright check no longer fails on beta skills and files endpoints that are not exposed by the current SDK stubs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* undo sample mypy
* Retry CI after transient external failures
Retrigger PR validation after an unrelated Copilot review workflow SAML failure and a transient external tau2 git fetch failure in the Windows Python test setup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback on model option merging
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address Anthropic compatibility review feedback
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* moved all to `model`
* fixes for azure ai search
* Python: standardize remaining sample env var names
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: fix foundry-local pyright compatibility
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated env vars in cicd
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* [BREAKING] Refactor middleware layering and raw clients
Reorder chat client layers so function invocation wraps chat middleware, and chat middleware stays outside telemetry while still running for each inner model call. Add middleware pipeline caching, refresh docs and samples, and split Anthropic into raw and public clients to match the standard layering model.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Tighten typing ignores in ancillary modules
Add targeted typing ignores in workflow visualization and lab modules so pyright stays clean alongside the middleware refactor work.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix categorize_middleware to unpack tuple/Sequence and use relative MRO assertions
- Broaden isinstance check in categorize_middleware from list to Sequence
so tuples and other Sequence types are properly unpacked instead of
being appended as a single item.
- Replace fragile hardcoded MRO index assertions in anthropic test with
relative ordering via mro.index().
- Add regression tests for categorize_middleware with tuple, list, and
None inputs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix middleware string decomposition, add middleware param to FunctionInvocationLayer, and add tests (#4710)
- Guard categorize_middleware Sequence check against str/bytes to prevent
character-by-character decomposition of accidentally passed strings
- Add explicit middleware parameter to FunctionInvocationLayer.get_response
and merge it into client_kwargs before categorization, fixing the
inconsistency where only OpenAIChatClient supported this parameter
- Add assertions that RawAnthropicClient does not inherit convenience layers
- Add chat middleware cache test with non-empty base middleware
- Add tests for single unwrapped middleware item and string input
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Apply pre-commit auto-fixes
* Apply pre-commit auto-fixes
* Address review feedback for #4710: review comment fixes
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
* feat(python): allow @tool functions to return rich content (images, audio)
Add support for tool functions to return Content objects that the model can perceive natively. Closes#4272
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Anthropic logging + mypy fix
* Address PR review: fix MCP ordering, fold helper into from_function_result, fix Chat client
- Preserve original content order in MCP tool results instead of text-first
- Move _build_function_result logic into Content.from_function_result()
- Chat Completions: inject user message for rich items (API only supports string tool content)
- Update tests for ordering and new from_function_result behavior
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Use native Responses API multi-part output, warn+omit for Chat client
- Responses client: put rich items directly in function_call_output's
output field as list (native API support) instead of user message injection
- Chat client: warn and omit rich items (API doesn't support multi-part
tool results), matching Ollama/Bedrock pattern
- Unify test image: use sample_image.jpg across all integration tests
- Add Azure OpenAI Responses integration test
- Assert model describes house image to verify perception
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix lint: remove print statement, wrap long line
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback: bug fixes, single-pass MCP, unit tests
- Add isinstance guard in from_function_result for non-Content lists
- Fix Anthropic empty tool_content fallback to string result
- Fix Content(type='text', text=None) edge case in parse_result
- Rewrite MCP _parse_tool_result_from_mcp as single-pass (no index counters)
- Add Anthropic unit tests: data image, uri image, unsupported media, all-unsupported
- Add OpenAI Chat unit test: rich items warning and omission
- Add OpenAI Responses unit tests: function_result with/without items
- Add test_types tests: only-rich-items list, non-Content list fallback
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix pyright errors: add type ignore comments for Any list iteration
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix mypy/pyright: ensure ToolExecutionException receives str
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix lint: remove duplicate test_prepare_options_excludes_conversation_id
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: unify all tool results into Content items
* addressed copilot comments
* pyright fix
* small fix
* comments
* fix: address Copilot review - warnings, blob safety, dedup
- Add warning logs when rich content is dropped in Claude agent and
MCP server handlers (matching Chat/Bedrock/Ollama pattern)
- Defensive blob URI construction: wrap plain base64 in data: prefix
- Simplify Chat client _prepare_content_for_openai to use content.result
- Simplify Responses client text-only path, remove redundant nesting
- Add test for plain base64 blob without data: prefix
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix token double-counting in compaction and address review comments
- Exclude items from _serialize_content() to prevent double-counting
tokens when items mirrors result in function_result content
- Add rich content warning in GitHub Copilot agent tool handler
- Replace raw Content debug log with concise item count/type summary
- Update stale test comments about FunctionTool.invoke return type
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Added shell tool
* Fixed CI error
* Add ShellTool support for OpenAI and Anthropic providers
- Add shell_tool_call, shell_tool_result, and shell_command_output content types
- Add ShellTool class and shell_tool decorator to core
- Add get_hosted_shell_tool() to OpenAI Responses client
- Handle shell_call and shell_call_output parsing in OpenAI (sync and streaming)
- Map ShellTool to Anthropic bash tool API format
- Parse bash_code_execution_tool_result as shell_tool_result in Anthropic
- Add unit tests for all new functionality
- Add sample scripts for hosted and local shell execution
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Addressed comments
* Reverted ruff change
* Fixed tests
* Addressed comments
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* updated integration tests and guidance
* fixed merge test
* updated integration tests
* fix: remove duplicate --dist loadfile flag from pytest-xdist config
Only one --dist mode can be active at a time; the second value silently
overrides the first. Keep --dist worksteal (dynamic load balancing) and
remove the redundant --dist loadfile from all workflow files and
pyproject.toml configs.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add keep-in-sync notes for merge and integration test workflows
Both python-merge-tests.yml and python-integration-tests.yml share the
same parallel job structure. Added sync reminders in workflow file
comments, the python-testing SKILL.md, and CODING_STANDARD.md.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: remove RUN_INTEGRATION_TESTS flag
Integration test gating now uses two mechanisms:
- `@pytest.mark.integration` for test selection via `-m` filtering
- `skip_if_*_disabled` for credential/service availability checks
The RUN_INTEGRATION_TESTS env var was redundant since the marker handles
selection and the skip decorators already check for actual credentials.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: sync missing env vars from merge-tests to integration-tests
Add OPENAI_EMBEDDINGS_MODEL_ID and AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME
to python-integration-tests.yml to match python-merge-tests.yml.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: remove remaining RUN_INTEGRATION_TESTS from embedding tests and docs
Missed test_openai_embedding_client.py and vector-stores README in the
earlier cleanup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* set functions tests to 3.10
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Python: improve .env precedence and observability samples
- Switch load_settings to explicit precedence: overrides -> explicit .env -> environment -> defaults\n- Raise when env_file_path is provided but missing\n- Update settings docs and tests for new behavior\n- Refresh observability samples and README guidance for env loading options\n\nCloses #3864\n\nCo-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fixed some imports
* Fix load_settings CI regressions
Allow explicit env_file_path values that exist but are not regular files (for example /dev/null) by checking path existence before dotenv parsing, and restore a dict accumulator with typed return cast to satisfy mypy.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Avoid implicit dotenv in observability
Only load dotenv in observability helpers when env_file_path is explicitly provided, and remove test os.devnull workarounds that are no longer necessary.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Replace Pydantic Settings with TypedDict + load_settings()
- Remove pydantic-settings dependency, add python-dotenv
- Delete _pydantic.py (AFBaseSettings, HTTPsUrl)
- Add _settings.py with generic load_settings() function, SecretString,
type coercion, and Required field validation (SettingNotFoundError)
- Convert all 13 settings classes from AFBaseSettings subclasses to
TypedDict definitions with load_settings() calls
- Update all consumers from attribute access to dict access
- Add 20 unit tests for load_settings() covering basic loading, dotenv,
SecretString, type coercion, and required field validation
- Update all existing tests for new settings patterns
* Fix mypy type errors from settings conversion
- Fix str | None attribute access in responses_client (walrus operator)
- Fix SecretString | None narrowing in bedrock (type: ignore after guard)
- Convert _context_provider.py attribute access to dict access (missed file)
- Fix endpoint type narrowing in search_provider and context_provider
- Fix purview: str | None .rstrip(), int | None defaults, urlparse bytes
* Address PR review: required_fields param, type validation, fixes
- Move required field validation from TypedDict annotations (Required)
to a required_fields parameter on load_settings(), enabling runtime
decisions about which fields are required
- Remove Required imports and restore from __future__ import annotations
in ollama and foundry_local
- Add _check_override_type() for deterministic ServiceInitializationError
on invalid override types (e.g. dict passed for str field)
- Fix all multi-exception test catches back to single exception type
- Fix Ollama host=None: use .get() so None is passed through to SDK default
- Fix Purview processor: use explicit is-None checks instead of or operator
- Remove unused BaseModel import from openai/_shared.py
- Add 4 new tests (24 total): required_fields param, type validation
* Fix type validation: allow int for float fields
_check_override_type now permits int values for float-typed fields,
matching Python's standard numeric promotion behavior.
* fix: wrap urlparse arg with str() to fix mypy bytes endswith error
* WIP
* big update to new ResponseStream model
* fixed tests and typing
* fixed tests and typing
* fixed tools typevar import
* fix
* mypy fix
* mypy fixes and some cleanup
* fix missing quoted names
* and client
* fix imports agui
* fix anthropic override
* fix agui
* fix ag ui
* fix import
* fix anthropic types
* fix mypy
* refactoring
* updated typing
* fix 3.11
* fixes
* redid layering of chat clients and agents
* redid layering of chat clients and agents
* Fix lint, type, and test issues after rebase
- Add @overload decorators to AgentProtocol.run() for type compatibility
- Add missing docstring params (middleware, function_invocation_configuration)
- Fix TODO format (TD002) by adding author tags
- Fix broken observability tests from upstream:
- Replace non-existent use_instrumentation with direct instantiation
- Replace non-existent use_agent_instrumentation with AgentTelemetryLayer mixin
- Fix get_streaming_response to use get_response(stream=True)
- Add AgentInitializationError import
- Update streaming exception tests to match actual behavior
* Fix AgentExecutionException import error in test_agents.py
- Replace non-existent AgentExecutionException with AgentRunException
* Fix test import and asyncio deprecation issues
- Add 'tests' to pythonpath in ag-ui pyproject.toml for utils_test_ag_ui import
- Replace deprecated asyncio.get_event_loop().run_until_complete with asyncio.run
* Fix azure-ai test failures
- Update _prepare_options patching to use correct class path
- Fix test_to_azure_ai_agent_tools_web_search_missing_connection to clear env vars
* Convert ag-ui utils_test_ag_ui.py to conftest.py
- Move test utilities to conftest.py for proper pytest discovery
- Update all test imports to use conftest instead of utils_test_ag_ui
- Remove old utils_test_ag_ui.py file
- Revert pythonpath change in pyproject.toml
* fix: use relative imports for ag-ui test utilities
* fix agui
* Rename Bare*Client to Raw*Client and BaseChatClient
- Renamed BareChatClient to BaseChatClient (abstract base class)
- Renamed BareOpenAIChatClient to RawOpenAIChatClient
- Renamed BareOpenAIResponsesClient to RawOpenAIResponsesClient
- Renamed BareAzureAIClient to RawAzureAIClient
- Added warning docstrings to Raw* classes about layer ordering
- Updated README in samples/getting_started/agents/custom with layer docs
- Added test for span ordering with function calling
* Fix layer ordering: FunctionInvocationLayer before ChatTelemetryLayer
This ensures each inner LLM call gets its own telemetry span, resulting in
the correct span sequence: chat -> execute_tool -> chat
Updated all production clients and test mocks to use correct ordering:
- ChatMiddlewareLayer (first)
- FunctionInvocationLayer (second)
- ChatTelemetryLayer (third)
- BaseChatClient/Raw...Client (fourth)
* Remove run_stream usage
* Fix conversation_id propagation
* Python: Add BaseAgent implementation for Claude Agent SDK (#3509)
* Added ClaudeAgent implementation
* Updated streaming logic
* Small updates
* Small update
* Fixes
* Small fix
* Naming improvements
* Updated imports
* Addressed comments
* Updated package versions
* Update Claude agent connector layering
* fix test and plugin
* Store function middleware in invocation layer
* Fix telemetry streaming and ag-ui tests
* Remove legacy ag-ui tests folder
* updates
* Remove terminate flag from FunctionInvocationContext, use MiddlewareTermination instead
- Remove terminate attribute from FunctionInvocationContext
- Add result attribute to MiddlewareTermination to carry function results
- FunctionMiddlewarePipeline.execute() now lets MiddlewareTermination propagate
- _auto_invoke_function captures context.result in exception before re-raising
- _try_execute_function_calls catches MiddlewareTermination and sets should_terminate
- Fix handoff middleware to append to chat_client.function_middleware directly
- Update tests to use raise MiddlewareTermination instead of context.terminate
- Add middleware flow documentation in samples/concepts/tools/README.md
- Fix ag-ui to use FunctionMiddlewarePipeline instead of removed create_function_middleware_pipeline
* fix: remove references to removed terminate flag in purview tests, add type ignore
* fix: move _test_utils.py from package to test folder
* fix: call get_final_response() to trigger context provider notification in streaming test
* fix: correct broken links in tools README
* docs: clarify default middleware behavior in summary table
* fix: ensure inner stream result hooks are called when using map()/from_awaitable()
* Fix mypy type errors
* Address PR review comments on observability.py
- Remove TODO comment about unconsumed streams, add explanatory note instead
- Remove redundant _close_span cleanup hook (already called in _finalize_stream)
- Clarify behavior: cleanup hooks run after stream iteration, if stream is not
consumed the span remains open until garbage collected
* Remove gen_ai.client.operation.duration from span attributes
Duration is a metrics-only attribute per OpenTelemetry semantic conventions.
It should be recorded to the histogram but not set as a span attribute.
* Remove duration from _get_response_attributes, pass directly to _capture_response
Duration is a metrics-only attribute. It's now passed directly to _capture_response
instead of being included in the attributes dict that gets set on the span.
* Remove redundant _close_span cleanup hook in AgentTelemetryLayer
_finalize_stream already calls _close_span() in its finally block,
so adding it as a separate cleanup hook is redundant.
* Use weakref.finalize to close span when stream is garbage collected
If a user creates a streaming response but never consumes it, the cleanup
hooks won't run. Now we register a weak reference finalizer that will close
the span when the stream object is garbage collected, ensuring spans don't
leak in this scenario.
* Fix _get_finalizers_from_stream to use _result_hooks attribute
Renamed function to _get_result_hooks_from_stream and fixed it to
look for the _result_hooks attribute which is the correct name in
ResponseStream class.
* Add missing asyncio import in test_request_info_mixin.py
* Fix leftover merge conflict marker in image_generation sample
* Update integration tests
* Fix integration tests: increase max_iterations from 1 to 2
Tests with tool_choice options require at least 2 iterations:
1. First iteration to get function call and execute the tool
2. Second iteration to get the final text response
With max_iterations=1, streaming tests would return early with only
the function call/result but no final text content.
* Fix duplicate function call error in conversation-based APIs
When using conversation_id (for Responses/Assistants APIs), the server
already has the function call message from the previous response. We
should only send the new function result message, not all messages
including the function call which would cause a duplicate ID error.
Fix: When conversation_id is set, only send the last message (the tool
result) instead of all response.messages.
* Add regression test for conversation_id propagation between tool iterations
Port test from PR #3664 with updates for new streaming API pattern.
Tests that conversation_id is properly updated in options dict during
function invocation loop iterations.
* Fix tool_choice=required to return after tool execution
When tool_choice is 'required', the user's intent is to force exactly one
tool call. After the tool executes, return immediately with the function
call and result - don't continue to call the model again.
This fixes integration tests that were failing with empty text responses
because with tool_choice=required, the model would keep returning function
calls instead of text.
Also adds regression tests for:
- conversation_id propagation between tool iterations (from PR #3664)
- tool_choice=required returns after tool execution
* Document tool_choice behavior in tools README
- Add table explaining tool_choice values (auto, none, required)
- Explain why tool_choice=required returns immediately after tool execution
- Add code example showing the difference between required and auto
- Update flow diagram to show the early return path for tool_choice=required
* Fix tool_choice=None behavior - don't default to 'auto'
Remove the hardcoded default of 'auto' for tool_choice in ChatAgent init.
When tool_choice is not specified (None), it will now not be sent to the
API, allowing the API's default behavior to be used.
Users who want tool_choice='auto' can still explicitly set it either in
default_options or at runtime.
Fixes#3585
* Fix tool_choice=none should not remove tools
In OpenAI Assistants client, tools were not being sent when
tool_choice='none'. This was incorrect - tool_choice='none' means
the model won't call tools, but tools should still be available
in the request (they may be used later in the conversation).
Fixes#3585
* Add test for tool_choice=none preserving tools
Adds a regression test to ensure that when tool_choice='none' is set but
tools are provided, the tools are still sent to the API. This verifies
the fix for #3585.
* Fix tool_choice=none should not remove tools in all clients
Apply the same fix to OpenAI Responses client and Azure AI client:
- OpenAI Responses: Remove else block that popped tool_choice/parallel_tool_calls
- Azure AI: Remove tool_choice != 'none' check when adding tools
When tool_choice='none', the model won't call tools, but tools should
still be sent to the API so they're available for future turns.
Also update README to clarify tool_choice=required supports multiple tools.
Fixes#3585
* Keep tool_choice even when tools is None
Move tool_choice processing outside of the 'if tools' block in OpenAI
Responses client so tool_choice is sent to the API even when no tools
are provided.
* Update test to match new parallel_tool_calls behavior
Changed test_prepare_options_removes_parallel_tool_calls_when_no_tools to
test_prepare_options_preserves_parallel_tool_calls_when_no_tools to reflect
that parallel_tool_calls is now preserved even when no tools are present,
consistent with the tool_choice behavior.
* Fix ChatMessage API and Role enum usage after rebase
- Update ChatMessage instantiation to use keyword args (role=, text=, contents=)
- Fix Role enum comparisons to use .value for string comparison
- Add created_at to AgentResponse in error handling
- Fix AgentResponse.from_updates -> from_agent_run_response_updates
- Fix DurableAgentStateMessage.from_chat_message to convert Role enum to string
- Add Role import where needed
* Fix additional ChatMessage API and method name changes
- Fix ChatMessage usage in workflow files (use text= instead of contents= for strings)
- Fix AgentResponse.from_updates -> from_agent_run_response_updates in workflow files
- Fix test files for ChatMessage and Role enum usage
* Fix remaining ChatMessage API usage in test files
* Fix more ChatMessage and Role API changes in source and test files
- Fix ChatMessage in _magentic.py replan method
- Fix Role enum comparison in test assertions
- Fix remaining test files with old ChatMessage syntax
* Fix ChatMessage and Role API changes across packages
- Add Role import where missing
- Fix ChatMessage signature: positional args to keyword args (role=, text=, contents=)
- Fix Role enum comparisons: .role.value instead of .role string
- Fix FinishReason enum usage in ag-ui event converters
- Rename AgentResponse.from_updates to from_agent_run_response_updates in ag-ui
Fixes API compatibility after Types API Review improvements merge
* Fix ChatMessage and Role API changes in github_copilot tests
* Fix ChatMessage and Role API changes in redis and github_copilot packages
- Fix redis provider: Role enum comparison using .value
- Fix redis tests: ChatMessage signature and Role comparisons
- Fix github_copilot tests: ChatMessage signature and Role comparisons
- Update docstring examples in redis chat message store
* Fix ChatMessage and Role API changes in devui package
- Fix executor: ChatMessage signature change
- Fix conversations: Role enum to string conversion in two places
- Fix tests: ChatMessage signatures and Role comparisons
* Fix ChatMessage and Role API changes in a2a and lab packages
- Fix a2a tests: Role comparisons and ChatMessage signatures
- Fix lab tau2 source: Role enum comparison in flip_messages, log_messages, sliding_window
- Fix lab tau2 tests: ChatMessage signatures and Role comparisons
* Remove duplicate test files from ag-ui/tests (tests are in ag_ui_tests)
* Fix ChatMessage and Role API changes across packages
After rebasing on upstream/main which merged PR #3647 (Types API Review
improvements), fix all packages to use the new API:
- ChatMessage: Use keyword args (role=, text=, contents=) instead of
positional args
- Role: Compare using .value attribute since it's now an enum
Packages fixed:
- ag-ui: Fixed Role value extraction bugs in _message_adapters.py
- anthropic: Fixed ChatMessage and Role comparisons in tests
- azure-ai: Fixed Role comparison in _client.py
- azure-ai-search: Fixed ChatMessage and Role in source/tests
- bedrock: Fixed ChatMessage signatures in tests
- chatkit: Fixed ChatMessage and Role in source/tests
- copilotstudio: Fixed ChatMessage and Role in tests
- declarative: Fixed ChatMessage in _executors_agents.py
- mem0: Fixed ChatMessage and Role in source/tests
- purview: Fixed ChatMessage in source/tests
* Fix mypy errors for ChatMessage and Role API changes
- durabletask: Use str() fallback in role value extraction
- core: Fix ChatMessage in _orchestrator_helpers.py to use keyword args
- core: Add type ignore for _conversation_state.py contents deserialization
- ag-ui: Fix type ignore comments (call-overload instead of arg-type)
- azure-ai-search: Fix get_role_value type hint to accept Any
- lab: Move get_role_value to module level with Any type hint
* Improve CI test timeout configuration
- Increase job timeout from 10 to 15 minutes
- Reduce per-test timeout to 60s (was 900s/300s)
- Add --timeout_method thread for better timeout handling
- Add --timeout-verbose to see which tests are slow
- Reduce retries from 3 to 2 and delay from 10s to 5s
This ensures individual test timeouts are shorter than the job
timeout, providing better visibility when tests hang.
With 60s timeout and 2 retries, worst case per test is ~180s.
* Fix ChatMessage API usage in docstrings and source
- Fix ChatMessage positional args in docstrings: _serialization.py, _threads.py, _middleware.py
- Fix ChatMessage in tau2 runner.py
- Fix role comparison in _orchestrator_helpers.py to use .value
- Fix role comparison in _group_chat.py docstring example
- Fix role assertions in test_durable_entities.py to use .value
* Revert tool_choice/parallel_tool_calls changes - must be removed when no tools
OpenAI API requires tool_choice and parallel_tool_calls to only be
present when tools are specified. Restored the logic that removes
these options when there are no tools.
- Restored check in _chat_client.py to remove tool_choice and
parallel_tool_calls when no tools present
- Restored same logic in _responses_client.py
- Reverted test to expect the correct behavior
* fixed issue in tests
* fix: resolve merge conflict markers in ag-ui tests
* fix: restructure ag-ui tests and fix Role/FinishReason to use string types
* fix: streaming function invocation and middleware termination
- Refactor streaming function invocation to use get_final_response() on inner streams
- Fix MiddlewareTermination to accept result parameter for passing results
- Fix _AutoHandoffMiddleware to use MiddlewareTermination instead of context.terminate
- Fix AgentMiddlewareLayer.run() to properly forward function/chat middleware
- Remove duplicate middleware registration in AgentMiddlewareLayer.__init__
- Fix exception handling in _auto_invoke_function to properly capture termination
- Fix mypy errors in core package
- Update tests to use stream=True parameter for unified run API
* fix all tests command
* Refactor integration tests to use pytest fixtures
- Merge testutils.py into conftest.py for azurefunctions integration tests
- Merge dt_testutils.py into conftest.py for durabletask integration tests
- Convert all integration tests to use fixtures instead of direct imports
(fixes ModuleNotFoundError with --import-mode=importlib)
- Add sample_helper fixture for azurefunctions tests
- Add agent_client_factory and orchestration_helper fixtures for durabletask
- Integration tests now skip with descriptive messages when services unavailable
- Restructure devui tests into tests/devui/ with proper conftest.py
- Add test organization guidelines to CODING_STANDARD.md
- Remove __init__.py from test directories per pytest best practices
* Fix pytest_collection_modifyitems to only skip integration tests
The hook was skipping all tests in the test session, not just
integration tests. Now it only skips items in the integration_tests
directory.
* Fix mem0 tests failing on Python 3.13
Use patch.object on the imported module instead of @patch with string
path to ensure the mock takes effect regardless of import timing.
* fix mem0
* another attempt for mem0
* fix for mem0
* fix mem0
* Increase worker initialization wait time in durabletask tests
Increase from 2 to 8 seconds to allow time for:
- Python startup and module imports
- Azure OpenAI client creation
- Agent registration with DTS worker
- Worker connection to DTS
This helps prevent test failures in CI where the first tests may run
before the worker is fully ready to process requests.
* Fix streaming test to use ResponseStream with finalizer
The _consume_stream method now expects a ResponseStream that can provide
a final AgentResponse via get_final_response(). Update the test to use
ResponseStream with AgentResponse.from_updates as the finalizer.
* Fix MockToolCallingAgent to use new ResponseStream API and update samples
* small updates to run_stream to run
* fix sub workflow
* temp fix for az func test
---------
Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
* Replace Role and FinishReason classes with NewType + Literal
- Remove EnumLike metaclass from _types.py
- Replace Role class with NewType('Role', str) + RoleLiteral
- Replace FinishReason class with NewType('FinishReason', str) + FinishReasonLiteral
- Update all usages across codebase to use string literals
- Remove .value access patterns (direct string comparison now works)
- Add backward compatibility for legacy dict serialization format
- Update tests to reflect new string-based types
Addresses #3591, #3615
* Simplify ChatResponse and AgentResponse type hints (#3592)
- Remove overloads from ChatResponse.__init__
- Remove text parameter from ChatResponse.__init__
- Remove | dict[str, Any] from finish_reason and usage_details params
- Remove **kwargs from AgentResponse.__init__
- Both now accept ChatMessage | Sequence[ChatMessage] | None for messages
- Update docstrings and examples to reflect changes
- Fix tests that were using removed kwargs
- Fix Role type hint usage in ag-ui utils
* Remove text parameter from ChatResponseUpdate and AgentResponseUpdate (#3597)
- Remove text parameter from ChatResponseUpdate.__init__
- Remove text parameter from AgentResponseUpdate.__init__
- Remove **kwargs from both update classes
- Simplify contents parameter type to Sequence[Content] | None
- Update all usages to use contents=[Content.from_text(...)] pattern
- Fix imports in test files
- Update docstrings and examples
* Rename from_chat_response_updates to from_updates (#3593)
- ChatResponse.from_chat_response_updates → ChatResponse.from_updates
- ChatResponse.from_chat_response_generator → ChatResponse.from_update_generator
- AgentResponse.from_agent_run_response_updates → AgentResponse.from_updates
* Remove try_parse_value method from ChatResponse and AgentResponse (#3595)
- Remove try_parse_value method from ChatResponse
- Remove try_parse_value method from AgentResponse
- Remove try_parse_value calls from from_updates and from_update_generator methods
- Update samples to use try/except with response.value instead
- Update tests to use response.value pattern
- Users should now use response.value with try/except for safe parsing
* Add agent_id to AgentResponse and clarify author_name documentation (#3596)
- Add agent_id parameter to AgentResponse class
- Document that author_name is on ChatMessage objects, not responses
- Update ChatResponse docstring with author_name note
- Update AgentResponse docstring with author_name note
* Simplify ChatMessage.__init__ signature (#3618)
- Make contents a positional argument accepting Sequence[Content | str]
- Auto-convert strings in contents to TextContent
- Remove overloads, keep text kwarg for backward compatibility with serialization
- Update _parse_content_list to handle string items
- Update all usages across codebase to use new format: ChatMessage("role", ["text"])
* Allow Content as input on run and get_response
- Update prepare_messages and normalize_messages to accept Content
- Update type signatures in _agents.py and _clients.py
- Add tests for Content input handling
* Fix ChatMessage usage across packages and samples
Update all remaining ChatMessage(role=..., text=...) to use new
ChatMessage('role', ['text']) signature.
* Fix Role string usage and response format parsing
- Fix redis provider: remove .value access on string literals
- Fix durabletask ensure_response_format: set _response_format before accessing .value
* Fix ollama .value and ai_model_id issues, handle None in content list
- Fix ollama _chat_client: remove .value on string literals
- Fix ollama _chat_client: rename ai_model_id to model_id
- Fix _parse_content_list: skip None values gracefully
* Fix A2AAgent type signature to include Content
* Fix Role/FinishReason NewType dict annotations and improve test coverage to 95%
* Fix mypy errors for Role/FinishReason NewType usage
* Fix Role.TOOL and Role.ASSISTANT usage in _orchestrator_helpers.py
* Fix Role NewType usage in durabletask _models.py
* changed AIFunction to FunctionTool and @ai_function to @tool
* test and mypy fixes
* mypy fix
* switch function tool to always_require
* fix noop
* fix github copilot imports
* test fixes
* fix ollama test
* fixes for tests
* fix tests
* reverted change to always_require and extended timeout
* fix test
* fix(anthropic): Add response_format support for structured outputs
* only use from options
* use native way of response format
* ruff lint fix
* address comment; handle dict
When processing `input_json_delta` events, the Anthropic client was
passing the tool name from the previous `tool_use` event. This caused
ag-ui's `_handle_function_call_content` to emit a `ToolCallStartEvent`
for every streaming chunk (since it triggers on `if content.name:`).
This fix changes the behavior to pass an empty string for `name` in
`input_json_delta` events, matching OpenAI's behavior where streaming
argument chunks have `name=""`. The initial `tool_use` event still
provides the tool name, so only one `ToolCallStartEvent` is emitted.
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
* refactoring and unifying naming schemes of internal methods of chat clients
* set tool_choice to auto
* fix for mypy
* added note on naming and fix#2951
* fix responses
* fixes in azure ai agents client
* initial version of anthropic connector
* updated implementation and added tests
* fix type and readme
* mypy fix and int tests enabled
* add integration test setup
* updated based on comments
* improved function result handling
* added extra unordered test
* updated from review
* fix tool choice handling
* same fix for chat client
Giles Odigwe