github-featured/microsoft--agent-framework
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity
2,350 Commits 301 Branches 183 Tags
fc3111c391afd5451c055a86d5cfd37ffab59617
Commit Graph

93 Commits

Author SHA1 Message Date
Eduard van Valkenburg fc3111c391 Python: Add FoundryAgent conversation session helper (#6623) 
* Add FoundryAgent conversation session helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Simplify Foundry conversation session helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Rename Foundry conversation helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* use named kw

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-22 08:41:15 +00:00
Ben Thomas 54a30571aa Dotnet - Add support for Foundry Adaptive evals (#6267) 
* .NET: feat(evals): RubricScore type + EvalScoreResult.Dimensions

Adds the core rubric-evaluator surface that mirrors the Python work in

PR #6101 (commit e45b934cc). Provider-agnostic types only — no Foundry

coupling. Subsequent commits will wire these into FoundryEvals.

- RubricScore: per-dimension score record (Id, Score?, Applicable, Weight, Reason).

- EvalScoreResult.Dimensions: optional init-only list of RubricScore.

  Null for non-rubric (built-in) evaluators.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(evals): GeneratedEvaluatorRef + assertion helpers

Adds the provider-agnostic surface for referencing a pre-existing rubric

evaluator and gating CI on per-item / per-dimension thresholds. Mirrors

Python PR #6101 commits e5830dd7f (ref type) and 4bc60462d (asserts).

- GeneratedEvaluatorRef: name + optional version/display-name, plus a

  Latest(name) factory for versionless refs (discouraged for CI; consumers

  should warn at run time).

- AgentEvaluationResults.AssertScoreAtLeast: walks DetailedItems[].Scores,

  optionally filtered by evaluator name, recurses into SubResults.

- AgentEvaluationResults.AssertDimensionScoreAtLeast: walks each score's

  Dimensions list, skips non-applicable dimensions by default, supports

  requireApplicable to flip that, recurses into SubResults.

- AgentEvaluationResults.AssertNoFailedItems: walks DetailedItems for

  fail/error statuses, recurses into SubResults.

All helpers throw InvalidOperationException (matches existing AssertAllPassed).

Truncates offender lists to the first 5 with a '+N more' suffix to keep

CI output readable, mirroring the Python helpers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(foundry-evals): accept GeneratedEvaluatorRef in evaluators=

Adds FoundryEvaluatorSpec, a readonly-struct union with implicit conversions
from both string and GeneratedEvaluatorRef so call sites can mix built-in
evaluator names with rubric evaluator references:

    var evals = new FoundryEvals(
        projectClient, model,
        new GeneratedEvaluatorRef("policy-rubric", "3"),
        FoundryEvals.Relevance,
        FoundryEvals.Coherence);

FoundryEvals constructors (3 overloads), EvaluateTracesAsync, and
EvaluateFoundryTargetAsync now take FoundryEvaluatorSpec[]/params instead of
string[]/params. Existing call sites using string literals or string[] keep
working unchanged via implicit conversion.

FoundryEvalConverter.BuildTestingCriteria emits the documented Foundry wire
format for rubric refs:
  {
    "type": "azure_ai_evaluator",
    "name": <DisplayName ?? Name>,
    "evaluator_name": <Name>,
    "evaluator_version": <Version>,   // omitted when null
    "initialization_parameters": { "deployment_name": <model> },
    "data_mapping": { conversation arrays, optional tool_definitions }
  }

WireTestingCriterion gains an optional EvaluatorVersion field. Rubric refs
are preserved through FilterToolEvaluators (tool-aware but not tool-required)
and ignored by FindMissingGroundTruthEvaluators. A versionless ref emits a
Trace.TraceWarning at criterion-build time so CI authors notice the floating
version (mirrors the Python warning).

Adds 6 new Foundry unit tests (3 BuildTestingCriteria rubric paths, 1
FindMissingGroundTruthEvaluators, 1 FilterToolEvaluators preservation, 1
mixed-order). 369/369 Foundry tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(foundry-evals): parse rubric dimension_scores into RubricScore

Adds FoundryEvals.ParseRubricScores, called per result inside ParseDetailedItem.
Each EvalScoreResult now populates Dimensions when the evaluator's sample carries
a rubric breakdown.

Accepts three shapes for forward compatibility with provider SDK iterations:

  1. sample.properties.dimension_scores  (canonical Foundry runtime shape)
  2. sample.properties.rubric_scores     (preview/legacy key)
  3. top-level sample.dimension_scores / sample.rubric_scores  (defensive fallback)

Entries missing 'id', 'weight', or 'applicable' are skipped without invalidating
well-formed siblings. Non-applicable dimensions may omit 'score' (parsed as null).

Adds 6 unit tests covering canonical and legacy keys, top-level fallback, no-match
returns null, malformed-entry skipping, and the non-applicable null-score path.
375/375 Foundry tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(samples): Evaluation_FoundryRubric end-to-end sample

Adds dotnet/samples/05-end-to-end/Evaluation/Evaluation_FoundryRubric mirroring
the Python evaluate_with_rubric_sample.py:

  - Fetches a pre-existing Foundry agent via AgentAdministrationClient
    (GetAgentAsync for latest, GetAgentVersionAsync when FOUNDRY_AGENT_VERSION
    is pinned).
  - References a rubric evaluator by GeneratedEvaluatorRef(name, version);
    falls back to GeneratedEvaluatorRef.Latest(name) with the documented
    floating-version warning.
  - Mixes the rubric with FoundryEvals.Relevance and FoundryEvals.Coherence
    in a single FoundryEvals run (implicit string-and-ref conversion).
  - Prints per-dimension breakdowns from EvalScoreResult.Dimensions for each
    item.
  - Demonstrates a CI quality gate with AssertDimensionScoreAtLeast("general_quality", 3.0).

Documents the FOUNDRY_PROJECT_ENDPOINT footgun (must be project-scoped URL
.../api/projects/<project>, not the bare Azure OpenAI endpoint) and the
Eval-Definition-vs-Rubric-Evaluator distinction in the README. Ships a
.env.example with the FOUNDRY_* variables.

Registers the project in agent-framework-dotnet.slnx and cross-links from
the sibling Evaluation_Multimodal / Evaluation_ExpectedOutputs READMEs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): harden FoundryEvals public surface for review

Address PR #6267 review comments on the .NET FoundryEvals integration:

- Add source-compat overloads accepting `string[] evaluators` for `FoundryEvals` ctor, `EvaluateTracesAsync`, and `EvaluateFoundryTargetAsync` so existing callers passing string arrays keep compiling unchanged. New overloads forward via a private `ToSpecs` helper that wraps each name through the implicit `string -> FoundryEvaluatorSpec` conversion.

- Guard against `default(FoundryEvaluatorSpec)` entries (both `BuiltinName` and `GeneratedRef` null) that would NRE the downstream converter. Adds `FoundryEvaluatorSpec.IsValid` / `EnsureValid` plus an internal `EnsureAllSpecsValid` helper, wired into the main ctor and both static evaluation entry points.

- Add 6 unit tests covering the new validation surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(sample): set ExitCode=1 when rubric dimension gate trips

PR #6267 review comment: the FoundryRubric sample swallowed the AssertDimensionScoreAtLeast failure, so a CI run that included it as a quality gate would still exit 0 even when the rubric regressed. Set `System.Environment.ExitCode = 1` in the catch so CI fails while still letting the rest of the sample's logging complete cleanly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): search typed Sample directly for rubric scores

PR #6267 review comment: `_extract_rubric_scores` only searched the `properties` dict when the sample exposed one. When the Azure AI Projects typed SDK returns a Sample object that puts `dimension_scores` / `rubric_scores` directly on the instance (no `properties` wrapper), we missed them and surfaced no per-dimension scores.

Add an `else: containers.append(sample)` branch so non-dict typed samples are also inspected for the score keys. Covered by two new tests: one with `dimension_scores` directly on a typed Sample without a `properties` wrapper, and one with the legacy `rubric_scores` key in the same shape.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(evals): cover assert_score_at_least and assert_no_failed_items

PR #6267 review comments: both assertion helpers shipped without unit tests. Add `TestAssertScoreAtLeast` (above threshold, below w/ offenders, evaluator filter, sub_results recursion) and `TestAssertNoFailedItems` (all passing, failed/errored statuses, sub_results recursion) with a shared `_score_results` fixture builder.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(samples): remove dead rubric-evaluator doc link from FoundryRubric sample

The Azure AI Foundry rubric evaluator concept doc page has not yet been published, so the link in the sample README and Program.cs comment 404s. Drop the references until the upstream doc is live.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Address PR 6267 review nits

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-06-18 20:50:28 +00:00
Eduard van Valkenburg 6e95517659 Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) (#6443) 
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)

Rework the typing setup along the lines of the 'too many type checkers'
approach:

- Pyright (strict) is now the sole source-code type checker; mypy is
  removed from source and its [tool.mypy] block becomes a relaxed profile
  used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
  ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
  basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
  pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
  '# type: ignore' on both branches so results match across interpreter
  versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
  gating checkers and pyright on source+tests+samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix merge regressions from main (typing + runtime)

Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:

Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
  _parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
  ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
  return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
  output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
  contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
  ExecutionMode used by the processor tests.

Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
  observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
  dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
  payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.

Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Isolate per-package mypy cache in test-typing fan-out

The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.

Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Make lab pyright-only on source (drop source mypy)

Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:

- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
  now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.

Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix test-typing regressions from latest main merge

A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:

- core: narrow Optional span.attributes with 'and' guards in span filters
  and assert+cast the json.loads(...attributes[...]) reads (test_observability);
  match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
  pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
  .annotations[0] access through a small _first_annotation helper (mirrors the
  file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
  (zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
  and connections.get_default (zuban) SDK type gaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated pyright version

* pyright fix

* Python: Fix source typing for pyright 1.1.410

Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:

- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
  AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
  the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
  Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
  reduce(and_, ...), which pyright could no longer fully type (drops the now
  unused functools.reduce / operator.and_ imports).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Accept plain-text body in Azure Functions workflow/run endpoint

The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.

Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 15:06:20 +00:00
Evan Mattson b55992bb67 Bump Python package versions for 1.9.0 release (#6583) 
Selective, CHANGELOG-driven version bumps for the 2026-06-18 release.

Released tier: agent-framework-core and the root agent-framework go to 1.9.0
(minor). Core ships new public APIs (agent-loop middleware, tool-approval
middleware and harness integration, shell-tool harness integration, AG-UI
thread snapshot persistence, context-provider telemetry) plus two behavioral
breaking changes on evolving surfaces: MCP sampling now denies server-initiated
requests by default, and the FileAccess tools were aligned with the .NET
implementation. These are treated as within-1.x changes because every package
caps core at <2; a major bump would require rewriting those caps. The foundry
and openai packages go to 1.8.2 (patch, bug fixes only). The root
agent-framework-core[all] pin was moved to 1.9.0 in lockstep with core.

Release-candidate tier: ag-ui to 1.0.0rc5 and declarative to 1.0.0rc2 for their
respective changes. orchestrations is promoted to stable 1.0.0; PACKAGE_STATUS
and the README install hint were updated accordingly.

Prerelease tier (new Pacific date stamp 260618): anthropic (beta),
azure-contentunderstanding (alpha) and foundry-hosting (alpha). No beta cohort
bump was applied; only packages with changes this cycle were stamped.

Dependency floors: following the established convention, the core floor was
raised to >=1.9.0 on every non-core package bumped this cycle, preserving the
existing <2 upper bound.

Also resolves two pre-existing failures in the dependency-bounds validator that
are unrelated to the version bumps. Hosted-environment detection now catches a
bare ImportError so optional Foundry hosting probing cannot crash user-agent
setup. The harness shell-tool integration, which lazily imports the separate
agent-framework-tools package to avoid a circular runtime dependency, is now
type-checked and tested in isolated environments via a core dev
dependency-group, with the shell-tool tests guarded to skip when that package
is absent.
 2026-06-18 18:01:17 +09:00
Eduard van Valkenburg d7e63d7d0e Fix Foundry aiohttp dependency (#6567) 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-18 02:03:16 +00:00
Eduard van Valkenburg 571cae426c Python: Fix Azure AI Search citation URLs (#6453) 
* Fix Azure AI Search citation URLs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Enrich MCP search citation metadata

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Azure AI Search citation enrichment follow-ups

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #6453 review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated filter for paths

* also updated python paths

* reverted dotnet-format change

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: westey <164392973+westey-m@users.noreply.github.com>
 2026-06-16 16:13:30 +01:00
Copilot 3daed114ee Python: bump package versions for 1.8.1 release (#6420) 
* Python: bump package versions for 1.8.1 release

* Python: bump agent-framework-foundry-hosting for 1.8.1 release

* Python: bump ag-ui and azurefunctions for 1.8.1 release

* Remove incorrect agent-framework-foundry changelog entry for #6259

* Add [1.8.1] changelog compare link and update [Unreleased] base

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-06-09 21:27:42 +00:00
Tao Chen 9cafd7e58b Python: Refactor workflow as agent pending request handling (#6259) 
* WIP: Refactor Workflow as agent pending request handling

* WIP: debugging empty message bug

* Working: Workflow as agent with function approval

* Address Copilot comments

* Fix mypy

* Address comments and fix pipeline

* Request info non function approval now becomes function call

* Revert uv.lock

* Fix mypy

* Bump min version of azure-ai-project

* Remove RequestInfoFunctionArgs

* fix tests

* Fix failing tests

* Fix sample
 2026-06-05 17:23:19 +00:00
Giles Odigwe 01fc518b29 Python: bump package versions for 1.8.0 release (#6351) 
- Released cohort (core, openai, foundry, root): 1.7.0 -> 1.8.0
- agent-framework-github-copilot: promote to RC (1.0.0rc1)
- agent-framework-orchestrations: rc2 -> rc3 (bug fix)
- Beta/alpha packages with changes: a2a, anthropic, azurefunctions, bedrock,
  foundry-hosting, mistral bumped to new date stamp (260604)
- Inter-package dependency bounds updated for changed packages
- CHANGELOG.md and PACKAGE_STATUS.md updated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-04 23:03:24 +00:00
Evan Mattson 6b94315161 Python: Add timeout parameter to FoundryAgent to fix ConnectTimeout on multi-turn conversations (#6263) 
* Python: fix ConnectTimeout on multi-turn FoundryAgent conversations (#6241)

Expose a `timeout` parameter on `RawFoundryAgentChatClient`,
`_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`, and
`RawOpenAIChatClient` so callers can override the HTTP timeout used by
the underlying AsyncOpenAI client.

Root cause: `RawFoundryAgentChatClient.__init__` called
`project_client.get_openai_client()` without configuring any timeout,
inheriting the OpenAI SDK default of `httpx.Timeout(connect=5.0)`.
When connections are recycled between turns under load, the 5 s connect
timeout fires and surfaces as `openai.APITimeoutError`.

Fix:
- `load_openai_service_settings` (`_shared.py`): accept `timeout` and
  include it in `client_args` for all three `AsyncOpenAI`/
  `AsyncAzureOpenAI` construction paths.
- `RawOpenAIChatClient.__init__` (`_chat_client.py`): accept `timeout`
  and forward to `load_openai_service_settings`.
- `RawFoundryAgentChatClient.__init__` (`_agent.py`): accept `timeout`
  and set `openai_client.timeout = timeout` on the client returned by
  `get_openai_client()` before passing it to the base class.
- `_FoundryAgentChatClient`, `RawFoundryAgent`, `FoundryAgent`: accept
  and propagate `timeout` through the construction chain.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add timeout parameter to FoundryAgent and RawOpenAIChatClient

Expose a timeout parameter on RawFoundryAgentChatClient,
_FoundryAgentChatClient, RawFoundryAgent, FoundryAgent, and
RawOpenAIChatClient. When provided, the value is applied to the
underlying AsyncOpenAI client so that connect timeouts under load
or after connection recycling can be tuned by callers.

Previously, get_openai_client() was called without any timeout
override, so the SDK default of httpx.Timeout(connect=5.0) was
inherited and could fire on multi-turn conversations where the
underlying connection is recycled between turns.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Add `timeout` parameter to `FoundryAgent` to fix `ConnectTimeout` on multi-turn conversations

Fixes #6241

* fix(foundry): use with_options to avoid mutating shared OpenAI client timeout (#6241)

Replace direct assignment  with
 in
RawFoundryAgentChatClient.__init__.

The Azure AI Projects SDK caches and returns a shared AsyncOpenAI client
per AIProjectClient. Mutating its .timeout attribute leaked the override
to all other code paths sharing that client (other agents, user code).
with_options() returns a new client instance with the override applied,
leaving the original shared client untouched.

Update tests to assert with_options is called with the correct timeout
and that the original shared client's timeout attribute is not mutated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry): assert with_options return value flows to instance.client (#6241)

The four timeout propagation tests verified that with_options was called
but did not confirm that the returned (timeout-configured) client was
actually stored on the instance. A silent discard of the return value
would have left the tests green while the timeout had no effect.

Each test now captures the constructed instance and asserts:
  assert <instance>.client is openai_client_mock.with_options.return_value

Affected tests:
- test_raw_foundry_agent_chat_client_init_applies_timeout_to_openai_client
- test_raw_foundry_agent_chat_client_init_applies_timeout_with_preview_enabled
- test_foundry_agent_chat_client_init_propagates_timeout
- test_foundry_agent_init_propagates_timeout_to_openai_client

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-04 18:25:18 +00:00
Benke Qu fa8cfb7567 Python: Fix FoundryAgent stripping model from PromptAgent requests (#5526) 
* Fix FoundryAgent stripping model from PromptAgent requests

Move run_options.pop('model', None) inside the _uses_foundry_agent_session()
conditional so that model is only stripped for hosted agent sessions (where
the server manages the model) and preserved for PromptAgent requests that
require it in the Responses API call.

Fixes #5525

* test: add coverage for resp_* continuation preserving model

Adds test_raw_foundry_agent_chat_client_prepare_options_preserves_model_for_resp_continuation
to explicitly verify that HostedAgent v1 / v2-no-session paths (where conversation_id
starts with resp_) preserve model and previous_response_id without triggering the
hosted-session gate.

---------

Co-authored-by: Benke Qu <bequ@microsoft.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
 2026-06-02 18:30:04 +00:00
Ben Thomas e0d0ad16a0 Python: feat(evals): Foundry Adaptive Evals integration (rubric-generation) (#6101) 
* Python: feat(evals): RubricScore type + EvalScoreResult.dimensions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(foundry-evals): RubricDimension + GeneratedEvaluatorRef + accept in evaluators=

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(evals): parse rubric_scores from output items + assertion helpers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(evals): BaseAgent.as_eval_source / Workflow.as_eval_source

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(foundry-evals): EvalGenerationSource + generate_rubric helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(foundry-evals): YAML config loader + sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix(evals): address PR review feedback

Addresses 4 Copilot review comments on PR #6101:

1. assert_dimension_score_at_least: drop the (not evaluator or found_any) guard so require_applicable=True correctly raises when the named evaluator produces no entries for the dimension. Adds TestRubricAssertions covering the regression.

2. GeneratedEvaluatorRef docstring: reword to describe actual behaviour (pinning recommended, not required) so it matches the dataclass default and FoundryEvals warning path.

3. _poll_generation_job: switch from asyncio.get_event_loop() to get_running_loop() and bound the per-iteration sleep by remaining time, matching _poll_eval_run.

4. generate_rubric: type category as Literal['quality','safety'] and validate at the entry point with a ValueError; drop the silent 'invalid -> quality' rewrite in _generation_job_to_ref. Adds a regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: feat(foundry-evals): hosted-agent-aware rubric generation

* Auto-detect hosted Foundry agents in agent_as_eval_source: when the
  agent's chat_client exposes a string agent_name (the convention used
  by RawFoundryAgentChatClient for PromptAgents/HostedAgents), emit a
  type='agent' EvalGenerationSource so the service fetches instructions
  and tools from the agent registry instead of relying on the local
  wrapper (which holds neither for hosted agents).
* Add hosted_agent_version kwarg and a new agent_version field on
  EvalGenerationSource so PromptAgent runs can pin to a specific hosted
  version for reproducible rubric generation.
* Add force_prompt_source escape hatch to bypass auto-detection and
  always emit a rendered prompt dossier - useful when the local wrapper
  carries overrides the service-side agent doesnt see.
* Fix _to_sdk_source for dataset sources: SDK ctor takes name=/version=,
  not dataset_name=/dataset_version=. The mismatch would raise TypeError
  against the real azure-ai-projects 2.3.0a* SDK; only unmocked
  integration paths were affected.

Tests cover: auto-detection happy path, versionless hosted agent,
explicit hosted_agent_version forwarding, force_prompt_source override,
non-string chat_client attrs (MagicMock test doubles) not mis-detected,
agent_version forwarded through _to_sdk_source, and the corrected
dataset SDK kwarg names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): accept canonical dimension_scores key per docs

The published Foundry rubric-evaluator output (Microsoft Learn 'Rubric evaluators' reference) places per-dimension breakdowns under properties.dimension_scores, not properties.rubric_scores. The parser now tries dimension_scores first and falls back to rubric_scores for preview-build compatibility, and tolerates non-list payloads (e.g. MagicMock auto-attrs) by trying the next candidate when parsing yields zero entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry-evals): add manual create_rubric_evaluator

Adds FoundryEvals.create_rubric_evaluator as the agent-framework surface over project_client.beta.evaluators.create_version. This is the manual counterpart to generate_rubric: callers supply RubricDimension instances (authored locally, ported from another framework, or hand-tuned) and we POST a RubricBasedEvaluatorDefinition. The service auto-attaches the non-editable residual dimension (general_quality for quality, general_policy_compliance for safety).

Per the Microsoft Learn 'Rubric evaluators' reference, the auto-generation path (create_generation_job) is primarily a portal/UI feature; external SDK clients with rich local agent context are better served by manual create_version. This keeps generate_rubric for users who want to round-trip through a Foundry-registered agent.

Validation up front: weight must be in [1,10], ids unique, descriptions non-empty, pass_threshold in [0,1]. The returned GeneratedEvaluatorRef is identical in shape to one obtained from generate_rubric, so downstream evaluators= lists work unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* samples(foundry-evals): manual rubric sample + namespace re-exports

Adds evaluate_with_manual_rubric_sample.py demonstrating the end-to-end dev scenario for FoundryEvals.create_rubric_evaluator: hand-author a list of RubricDimension, register via create_rubric_evaluator, then use the pinned GeneratedEvaluatorRef alongside built-in evaluators in an agent regression run.

Also re-exports RubricDimension, GeneratedEvaluatorRef, build_sources, and load_evals_config from agent_framework.foundry (both the lazy runtime shim and the type stub) so the rubric samples can import everything from a single namespace; the auto-generate sample was previously broken because the shim was missing build_sources / load_evals_config.

Updates the foundry-evals README with a chooser entry for the two rubric paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry-evals): remove rubric creation flows; keep consumption only

Reframes agent-framework as a pure consumer of Foundry rubric evaluators: scoring against rubrics that already exist (authored in the Foundry portal or via the dedicated SDK / REST surface) instead of creating them from the SDK.

Removed creation surface area:

- FoundryEvals.generate_rubric (auto-generate path) and create_rubric_evaluator (manual path), plus all _GenerationSdkTypes / _ManualRubricSdkTypes / _to_sdk_dimensions / _coalesce_generation_sources / _to_sdk_source / _poll_generation_job / _generation_job_to_ref / _evaluator_version_to_ref / _get_beta_evaluators / _import_*_sdk_types helpers.

- EvalGenerationSource (the input source discriminator), RubricDimension (the input dimension type), agent_as_eval_source / workflow_as_eval_source / _detect_hosted_foundry_agent helpers, and the YAML-config loader (_evals_config.py with RubricGenerationSpec / RubricSourceSpec / parse_evals_config / load_evals_config / build_sources).

- BaseAgent.as_eval_source / Workflow.as_eval_source plus the _render_agent_dossier / _render_workflow_dossier helpers in core. These existed only to feed the now-removed generation pipeline.

- Samples evaluate_with_generated_rubric_sample.py, evaluate_with_manual_rubric_sample.py, and evaluators.yaml. Replaced with a short README section showing how to reference an existing rubric evaluator via GeneratedEvaluatorRef.

Kept (consumption surface):

- GeneratedEvaluatorRef, slimmed to (name, version, display_name). Still accepted alongside built-in evaluator strings in FoundryEvals(evaluators=[...]). Versionless refs still warn.

- RubricScore on EvalScoreResult.dimensions plus EvalResults.assert_dimension_score_at_least for per-dimension CI gates.

- _parse_dimension_entries / _extract_rubric_scores output parsing (both canonical dimension_scores and the legacy rubric_scores key).

Tests: 160/160 foundry unit tests and 71/71 core local-eval tests pass; pyright is clean across changed files. The pre-existing tests/core/test_telemetry.py::test_detect_hosted_fallback_import_error failure is unrelated and reproduces on the prior commit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* samples(foundry-evals): add evaluate_with_rubric_sample

Adds a runnable end-to-end sample showing how to consume a pre-existing rubric evaluator created in Foundry: reference it with GeneratedEvaluatorRef(name, version), mix it with built-in evaluators in FoundryEvals, and gate CI with assert_dimension_score_at_least on a specific dimension.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): satisfy mypy on _fetch_output_items

mypy infers OutputItemListResponse.sample as dict[str, object] | None while pyright correctly infers the typed Sample model. Cast to Any so both type checkers accept the attribute access pattern, rename the local to avoid shadowing the inner-loop sample binding, and drop the now-stale pyright suppressions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry-evals): drop unpublished rubric-evaluators learn.microsoft.com link

The Adaptive Evals authoring docs are not yet published on Microsoft Learn, so the link 404s. Keep the descriptive text without the broken hyperlink; we can re-add it once the docs ship.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry-evals): hoist repeated local imports to module top

Per code review feedback (eavanvalkenburg): the test file repeated 'from agent_framework_foundry._foundry_evals import ...' inside 22 test bodies and 'from agent_framework_foundry import GeneratedEvaluatorRef' inside 8 more. Move all of them to the existing top-level imports; the symbols are the same across tests and the local imports were redundant.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-01 23:01:56 +00:00
Eduard van Valkenburg 8091d052d8 Python: refresh dev dependencies and validate runtime bounds (#6238) 
Updates third-party dev dependencies across the Python workspace and
validates that all runtime dependency bounds still hold at both ends.

Dev dependency bumps (root, lab, declarative, durabletask):
- uv 0.11.6 -> 0.11.17, ruff 0.15.8 -> 0.15.15,
  pytest-asyncio 1.3.0 -> 1.4.0, mcp 1.27.0 -> 1.27.2,
  azure-monitor-opentelemetry 1.8.7 -> 1.8.8,
  poethepoet 0.42.1 -> 0.46.0, prek 0.3.9 -> 0.4.3,
  types-python-dateutil and types-PyYaml stub bumps.
- Transitive Dependabot items swept via lock: idna 3.11 -> 3.17,
  pip 26.0.1 -> 26.1.2.

Deliberately excluded:
- opentelemetry-sdk stays 1.40.0: azure-monitor-opentelemetry (incl.
  1.8.8) hard-pins opentelemetry-sdk==1.40.
- mypy stays 1.20.0 and pyright stays 1.1.408: the 2.1.0 / 1.1.409
  bumps introduce new diagnostics that fail type checking and need
  dedicated PRs.
- rich kept as a range: agentlightning (lab[lightning]) forces
  rich==13.9.4.

Code/formatting changes driven by the ruff upgrade:
- devui lifespan now uses try/finally so shutdown cleanup always runs
  (ruff RUF075).
- Removed unused TYPE_CHECKING imports in core and foundry flagged by
  ruff 0.15.15.
- Reapplied ruff 0.15.15 formatting to the files it changed.

Validation: validate-dependency-bounds-test "*" passes (31/31 lower +
31/31 upper); typing 62/62; lint 31/31; devui tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-06-01 17:53:56 +00:00
Tao Chen 0578f4c910 Backfill chat span request model if it's unknown and response model is avaliable (#6160) 2026-05-28 20:03:46 +00:00
Evan Mattson a84ad42f6d Bump Python package versions for 1.7.0 release (#6142) 
Bumps the released 1.6.0 packages agent-framework, agent-framework-core, agent-framework-foundry, and agent-framework-openai to 1.7.0, with root continuing to exactly pin agent-framework-core[all]. Bumps the changed prerelease packages agent-framework-a2a, agent-framework-chatkit, agent-framework-declarative, agent-framework-devui, and agent-framework-foundry-hosting to the 260528 date stamp, raises core floors on the packages included in this release, raises Foundry's OpenAI floor alongside OpenAI, and raises ChatKit's openai-chatkit floor to the minimum version required by the current typed API usage. No beta cohort bump was applied; the absent mistal/mistral package was intentionally not bumped because no such package exists in this branch.
 2026-05-28 19:45:31 +09:00
Yufeng He 55dc3ce734 Python: fix: pass Foundry agent default headers (#6040) 
* fix: pass Foundry agent default headers

* test: loosen Foundry default header assertions
 2026-05-28 10:08:14 +00:00
Eduard van Valkenburg d5c07f2623 Python: feat(foundry): add to_prompt_agent / deploy_as_prompt_agent (experimental) (#5959) 
* feat(foundry): add experimental to_prompt_agent converter

Adds `to_prompt_agent(agent)`, an experimental converter
(`ExperimentalFeature.TO_PROMPT_AGENT`) that turns an Agent Framework
`Agent` into a Foundry `PromptAgentDefinition` ready to publish via
`AIProjectClient.agents.create_version(...)`.

Behaviour:

* `agent.client` must be a `FoundryChatClient` (or subclass); otherwise
  `TypeError` is raised. The model deployment name is lifted from the
  bound client so the same Agent definition used for local runs can be
  published as a hosted prompt agent without restating the model.
* Foundry SDK tool instances (from `FoundryChatClient.get_*_tool()`) are
  passed through unchanged. AF `FunctionTool`s (and `@tool`-decorated
  callables) are emitted as Foundry `FunctionTool` declarations.
* Local AF MCP tools cannot be expressed in a `PromptAgentDefinition`;
  the converter raises `ValueError` and points at
  `FoundryChatClient.get_mcp_tool()` for hosted MCP servers.
* The converter walks both `agent.default_options["tools"]` and
  `agent.mcp_tools` because `normalize_tools()` splits local MCP off
  into its own list.

Re-exported through the `agent_framework.foundry` lazy-loading namespace
(updates both `__init__.py` and the `__init__.pyi` type stub).

Adds a portable-agent sample showing the same `Agent` driven through
both `agent.run(...)` and `to_prompt_agent(agent)`, and a README section
covering the new converter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): remove snippet tags from portable agent sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): inline FoundryChatClient and enable prompt-agent publish

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): drop async credential context manager

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): trim README to_prompt_agent example to publish-only flow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): note FoundryAgent runs @tool callables for deployed prompt agents

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): address review comments on to_prompt_agent converter

* Construct `PromptAgentDefinition` `Tool` from a dict via `**tool_item`
  unpacking rather than the positional Mapping constructor \u2014 cleaner and
  matches the typical Pydantic / Azure SDK pattern.
* Drop the redundant `isinstance(mcp_tool, MCPTool)` guard in
  `_convert_tools`; the parameter is already typed `Iterable[MCPTool]` so
  the second `raise` was unreachable. The remaining single `raise`
  fires for every entry as intended.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): match Agent.__init__ model resolution in to_prompt_agent

* Read the model from `agent.default_options.get("model")` first,
  falling back to `agent.client.model`. This mirrors the order
  `Agent.__init__` uses (`_agents.py:740`) when assembling
  default_options, so the model the agent runs with is the same model
  the converter publishes \u2014 e.g. when the caller passes
  `default_options={"model": "..."}` to override the bound client.
* Updated the missing-model error message to point at both the client
  and the default_options paths.
* Added tests:
  * tool-only agent with no `instructions` produces a definition
    where `instructions` is `None` and is omitted from the dict
    payload (`Agent.__init__` strips None values from default_options
    before storing them).
  * `default_options['model']` wins over the bound client's model.
  * Fallback to client.model when default_options has no model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry): add deploy_as_prompt_agent helper + samples

Adds `deploy_as_prompt_agent(agent)`, a convenience wrapper around
`to_prompt_agent` that reuses the bound FoundryChatClient's project
client to call `project_client.agents.create_version(...)`. Defaults
`agent_name` / `description` from `agent.name` / `agent.description`
so the Agent stays the single source of truth.

* Exposed from `agent_framework_foundry` and the lazy-loading
  `agent_framework.foundry` namespace (including the .pyi stub).
* Marked experimental with the existing
  `ExperimentalFeature.TO_PROMPT_AGENT` tag.
* Tests cover the happy path, name/description defaulting, explicit
  override, no-name error, metadata + description forwarding, extra
  kwargs passthrough, and the experimental metadata.

Samples:
* Renamed the existing sample to `creating_prompt_agents.py`, drops
  'portable' wording, presents `deploy_as_prompt_agent` first as the
  recommended path and `to_prompt_agent` + `AIProjectClient` as the
  two-step alternative, and adds a cleanup step that deletes the
  published agent so re-runs stay idempotent.
* New `using_prompt_agents.py` shows the end-to-end loop: deploy the
  agent, connect to it with `FoundryAgent` passing the same local
  `@tool` callable, run a query against the deployed prompt agent,
  then clean up.

README updated to introduce `deploy_as_prompt_agent` as the
recommended path and link to both runnable samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): restore missing-model ValueError in to_prompt_agent

The check was accidentally dropped while reworking docstrings in the
previous commit. Test `test_to_prompt_agent_rejects_missing_model`
exercises this path and was failing on CI as a result.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): rename deploy_as_prompt_agent -> create_prompt_agent

Renames the helper across the foundry package, core lazy-loader stubs,
tests, README and samples. The new name better matches the action
performed (a prompt-agent definition is created in Foundry) and is
consistent with the surrounding ''create_*'' API surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): drop create_prompt_agent, enrich to_prompt_agent params

Remove the create_prompt_agent helper and consolidate on to_prompt_agent.
Expose every PromptAgentDefinition parameter that has either an Agent
Framework equivalent (sourced from default_options) or no equivalent
(accepted as a keyword argument).

* default_options-sourced (with kwarg overrides):
  temperature, top_p, string tool_choice
* kwarg-only Foundry knobs:
  reasoning, text, structured_inputs, rai_config, ToolChoiceParam tool_choice

Precedence is always: explicit keyword > default_options entry > unset.

Tests cover every path (defaults, default_options, kwargs, kwarg override).
Samples and README rewritten around the enriched to_prompt_agent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): single source of truth for prompt-agent options

Stop duplicating the generation-parameter surface between FoundryChatOptions
and to_prompt_agent. Translate every field with an Agent Framework equivalent
(temperature, top_p, tool_choice, reasoning, response_format/text/verbosity)
from agent.default_options via a new RawFoundryChatClient helper
_prepare_prompt_agent_options. Only Foundry-specific fields with no AF
equivalent — structured_inputs and rai_config — remain as keyword arguments
on to_prompt_agent.

- tool_choice is dropped when there are no tools (mirrors _prepare_options
  semantics and avoids polluting tool-less prompt agents with Agent.__init__'s
  'auto' default).
- response_format Pydantic models route through
  openai.lib._parsing._responses.type_to_text_format_param; dict shapes go
  through the existing _prepare_response_and_text_format helper.
- default_options is not mutated; text dict is defensively copied.

Tests, README, and creating_prompt_agents.py sample updated to reflect the
new single-source model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): consolidate prompt-agent sample

Drop creating_prompt_agents.py (the publish-only variant) and rename
using_prompt_agents.py to foundry_prompt_agents.py so the single sample
covers the full convert -> publish -> connect -> run loop. Update the
README link list accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): run local Agent + deployed agent in same sample

Add an agent.run() call against the local Agent before publishing, then run
the deployed prompt agent on the same query. Expand the docstring with a
compare-and-contrast covering runtime/latency, configurability, and
persistence/sharing differences between the two execution paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry): cover conflicting response_format + text.format in to_prompt_agent

Exercises the ValueError path when a Pydantic response_format would overwrite
an explicit text.format mapping with a different shape. Lifts _chat_client.py
coverage from 89% to 90%.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): move _prepare_prompt_agent_options into _to_prompt_agent

Lift the translation helper off RawFoundryChatClient and into the
_to_prompt_agent module as a module-private function that takes the client
as its first argument. The chat client no longer needs to carry a method
whose only consumer is the prompt-agent converter, while still serving as
the source of the request-path helper (_prepare_response_and_text_format)
that the converter reuses for dict-shaped response_format values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(python): codify GA terminology + post-run docs review

Add two pieces of guidance to python/AGENTS.md:

* Terminology - reserve 'GA' for hosted services; use 'released' or 'stable'
  for Agent Framework code/features to match the feature-lifecycle stages.
* Maintaining Documentation - review AGENTS.md and skills at the end of every
  run and update any guidance the conversation made stale; before adding a
  new principle, ask the user to confirm it should be captured.

Also pulls in a docstring fix in foundry_prompt_agents.py that swaps the
stray 'GA' for 'released', applying the new terminology rule.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* address PR review: strict=True default, Tool._deserialize dispatch, sample cleanup safety

- FunctionTool published as strict=True so the server-side schema validation
  matches what the local FoundryAgent(tools=[same_callable]) dispatcher
  enforces. AF FunctionTool has no 'strict' attribute, so the safer default
  is used uniformly instead of silently downgrading to a permissive contract.
- _validate_mapping_tool now dispatches through ProjectsTool._deserialize so
  dict-shaped tools rehydrate to the concrete subclass (FunctionTool,
  WebSearchTool, ...) via the 'type' discriminator instead of returning a
  generic Tool. Added a test that asserts isinstance(WebSearchTool) and a
  new test for the function-typed dict path.
- foundry_prompt_agents.py sample now wraps credential + project client in
  async with and the create_version / run flow in try/finally so a failure
  on connect or run still deletes the published prompt agent rather than
  leaving an orphaned, billable resource in the user's Foundry project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(ci): correct linkspector ignorePattern typo (./pulls -> ./pull)

GitHub PR URLs use the singular segment /pull/N (compare to /issues/N
for issues). The existing './pulls' ignore pattern never matched
anything as a result, so legitimately stale PR links (e.g. PRs deleted
from forks) surface as linkspector failures on unrelated PRs.

This is the same convention the './issues' rule above already follows.
Fixes the markdown-link-check failure on a dangling link in
dotnet/src/Microsoft.Agents.AI.DurableTask/CHANGELOG.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-27 13:31:21 +00:00
Eduard van Valkenburg 578416a379 Python: fix(core): point @experimental warnings at user code, not stdlib internals (#5996) 
* fix(core): point @experimental warnings at user code, not stdlib internals

Previously the wrappers installed by @experimental called warnings.warn
with a fixed stacklevel=3. ABCMeta inserts an extra abc.__new__ frame
when an experimental ABC is subclassed, so the warning landed inside
abc.py (or <frozen abc>:106 on modern CPython) instead of the user's
class Sub(...) line.

Resolve the user frame by walking inspect.currentframe(), skipping
frames whose module name is abc/functools/typing/contextlib (or
submodules), then emit via warnings.warn_explicit so the recorded
filename/lineno point at user code. Falls back to warnings.warn with
stacklevel=2 if no user frame is found. Module-name matching is used
because frozen stdlib modules report '<frozen abc>' as their filename.

Also install a one-line warnings.formatwarning specifically for
FeatureStageWarning so 'file:line: ExperimentalWarning: [ID] Name ...'
prints without the secondary source-snippet line. Other categories
delegate to the stdlib default formatter unchanged.

Added a regression test that subclasses an @experimental ABC inside
warnings.catch_warnings and asserts the recorded filename equals the
test file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(core): address review feedback on @experimental warning fix

- Make _install_feature_stage_formatter idempotent: tag the installed
  formatter with a marker attribute and short-circuit re-installation,
  so re-imports/reloads don't wrap the formatter on top of itself.
  Also expose the previous formatter via __wrapped__ for restoration.
- Avoid leaking frame references in _resolve_user_frame: capture data
  into plain locals inside try and del frame/candidate in finally,
  per CPython's guidance on inspect.currentframe usage.
- Drop redundant _WARNED_FEATURES.clear() in the new ABC subclass test
  (the autouse fixture already handles it).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* changed query for foundry web search test

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-22 12:07:10 +00:00
Giles Odigwe 950673ba47 Python: bump package versions for 1.6.0 release (#6017) 
* Python: bump package versions for 1.6.0 release

- Released cohort (agent-framework, core, openai, foundry): 1.5.0 -> 1.6.0
- Beta packages (21 packages): 1.0.0b260519 -> 1.0.0b260521
- Alpha packages (azure-contentunderstanding, foundry-hosting, gemini, monty): 1.0.0a260518/19 -> 1.0.0a260521
- ag-ui stays at 1.0.0rc2, orchestrations at 1.0.0rc1 (dependency bounds updated)
- Inter-package dependency lower bounds updated (>=1.5.0,<2 -> >=1.6.0,<2)
- Update CHANGELOG compare links
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: bump RC packages, add shell tool to changelog

- ag-ui: 1.0.0rc2 -> 1.0.0rc3
- orchestrations: 1.0.0rc1 -> 1.0.0rc2
- Add shell tool (#5664) to CHANGELOG
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-22 01:59:20 +00:00
Yufeng He 46ed66cfd5 Python: include tool definitions for Foundry agent evals (#5974) 2026-05-21 16:23:36 +00:00
Eduard van Valkenburg 47f5c3397f Python: feat(foundry): add experimental hosted tool factories on FoundryChatClient (#5958) 
* feat(foundry): add experimental hosted tool factories on FoundryChatClient

Adds eight new `@experimental` static factory methods on `FoundryChatClient`
covering Foundry-hosted tools that previously had no helper:

- get_azure_ai_search_tool
- get_sharepoint_tool
- get_fabric_tool
- get_memory_search_tool
- get_computer_use_tool
- get_browser_automation_tool
- get_bing_custom_search_tool
- get_a2a_tool

All factories are marked with the new `ExperimentalFeature.FOUNDRY_TOOLS` tag
and resolve the underlying `azure-ai-projects` preview classes lazily through
a `_require_sdk_class` helper so older SDK versions still import cleanly and
fail with a clear `ImportError` only on use.

Tests cover each factory's return type and field wiring, the experimental
metadata, and the missing-SDK-class fallback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry): address review comments on tool-factory tests

* Skip preview-tool tests gracefully (`_skip_if_sdk_class_missing`) when
  the installed `azure-ai-projects` does not expose the required preview
  class, matching the lazy-import guard in production code so the test
  suite stays green on older SDK installs.
* Add `filterwarnings("ignore::FutureWarning")` to each new tool-factory
  test (and the parametrized metadata test) so they remain stable under
  strict warning configurations \u2014 the global dedup in
  `_feature_stage._WARNED_FEATURES` makes `pytest.warns` brittle across
  ordered runs.
* Use `monkeypatch.setattr(..., None, raising=False)` instead of
  `delattr` in the missing-SDK-class test so it works for modules that
  implement PEP 562 `__getattr__`.
* Split the long `get_bing_custom_search_tool` return into two lines for
  readability.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): harden tool-factory kwargs against silent override

* Reorder the dict-literal kwargs assembly in get_azure_ai_search_tool,
  get_memory_search_tool, and get_bing_custom_search_tool so explicit
  parameters always take precedence over **kwargs (matching the safe
  pattern already used in get_a2a_tool). This prevents a caller
  passing `project_connection_id`, `index_name`, `memory_store_name`,
  `scope`, or `instance_name` through `**kwargs` from silently
  overriding the explicit security-sensitive arguments.
* Update the README experimental note to reflect once-per-feature-id
  dedup semantics of `_feature_stage._WARNED_FEATURES` rather than
  claiming a per-factory "first use" warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry): split FOUNDRY_TOOLS / FOUNDRY_PREVIEW_TOOLS, add bing-grounding

- Add ExperimentalFeature.FOUNDRY_PREVIEW_TOOLS to distinguish wrappers around
  preview Foundry SDK tool classes (Sharepoint/Fabric/Memory/ComputerUse/
  BrowserAutomation/BingCustomSearch/A2A) from FOUNDRY_TOOLS, which is for
  GA-SDK wrappers that are simply new in agent-framework-foundry
  (AzureAISearch, BingGrounding).
- Add get_bing_grounding_tool factory and a 'Choosing a web grounding tool'
  comparison block on get_web_search_tool / get_bing_grounding_tool /
  get_bing_custom_search_tool docstrings.
- Drop the _require_sdk_class lazy resolver: every guarded class is available
  at azure-ai-projects>=2.1.0 (the package floor), so import them eagerly.
  Concrete return types replace 'Any'.
- README: split the experimental factories into two tables, one per feature
  flag, with a note explaining the distinction.
- Tests: split into FOUNDRY_TOOLS / FOUNDRY_PREVIEW_TOOLS factory cases;
  drop the obsolete missing-SDK-class ImportError test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-21 08:39:08 +00:00
Tao Chen 72a6157c6a [BREAKING] Python: Enable instrumentation by default (#5865) 
* Enable instrumentation by default

* Update samples

* Optimization when span is not recording

* Address Copilot comments

* Revert uv.lock

* Add warning

* Formatting

* Fix mypy

* Add disable_instrumentation() with sticky user-intent semantics

Add a public disable_instrumentation() entry point so users can explicitly opt
out of Agent Framework telemetry, with a sticky-disable flag that makes the
user's intent "leading" — no framework code path (foundry's
configure_azure_monitor, configure_otel_providers, enable_instrumentation,
enable_sensitive_telemetry, or direct OBSERVABILITY_SETTINGS.enable_*
writes) can re-enable instrumentation until the user explicitly clears the
disable with enable_instrumentation(force=True) /
enable_sensitive_telemetry(force=True).

Also addresses the two remaining unresolved review threads on the PR:
1. test_observability_settings_defaults_instrumentation_true pins the new
   "ENABLE_INSTRUMENTATION defaults to True when env unset" behavior.
2. test_enable_instrumentation_reads_env_sensitive_data restores coverage
   for the post-import load_dotenv() fallback path.

Implementation:
- ObservabilitySettings.enable_instrumentation / enable_sensitive_data become
  properties backed by _enable_*. While _user_disabled is True, the getters
  return False and the setters drop True writes (defense in depth so third-
  party writes can't subvert the disable).
- Public is_user_disabled read-only property lets integrations (e.g. foundry's
  configure_azure_monitor) cheaply check the disable state without poking at
  privates.
- enable_instrumentation() and enable_sensitive_telemetry() short-circuit with
  an info log when disabled; gain a force=True kwarg that clears the disable.
- configure_otel_providers() still creates providers / exporters / views so a
  later force-enable can use them, but logs an info message when called while
  disabled.
- Foundry's FoundryChatClient.configure_azure_monitor and
  FoundryAgent.configure_azure_monitor early-return when the user has
  disabled, so Azure Monitor's global providers aren't installed unnecessarily.

Tests: 11 new tests covering default-on, env re-read at call time, sticky
behavior against each re-enable surface (enable_instrumentation,
enable_sensitive_telemetry, configure_otel_providers, direct attribute
writes), force=True override, re-arming the disable, and the __all__ export.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: document disable_instrumentation() and force=True paths

Add a "Disabling instrumentation" section to the observability sample README
that walks through:

- The distinction between the ENABLE_INSTRUMENTATION env var (initial,
  non-sticky) and disable_instrumentation() (process-wide, sticky).
- Why the sticky semantics matter: framework integrations like
  FoundryChatClient.configure_azure_monitor() can call
  enable_instrumentation() as part of their setup, and the user's opt-out
  needs to win.
- All five surfaces guarded by the sticky disable (property reads, public
  enable functions, configure_otel_providers, direct attribute writes,
  is_user_disabled-aware integrations).
- The force=True escape hatch on both enable_instrumentation() and
  enable_sensitive_telemetry().
- How third-party integrations should consult OBSERVABILITY_SETTINGS.is_user_disabled.
- The limits of the disable (does not tear down existing providers /
  in-flight spans / third-party instrumentation, does not persist across
  processes).

Cross-links the new section from the ENABLE_INSTRUMENTATION row in the env
vars table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: soften disable_instrumentation() overclaim about telemetry guarantees

Replace 'no telemetry will be emitted no matter what' (which is too strong,
since callers can still pass force=True or mutate private attributes) with
language framing the disable as a user-intent contract that library and
framework code is expected to honor: the framework actively short-circuits
the public enable paths, force=True and private-attribute writes are
acknowledged as out-of-contract escape hatches that integrations should
not use on the user's behalf.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: correct observability Dependencies section

- opentelemetry-sdk is no longer a hard dependency; it is lazily imported by
  create_resource(), create_metric_views(), and configure_otel_providers()
  with a clear ImportError when missing. Day-to-day instrumentation works
  with opentelemetry-api alone provided some other component configures the
  global OpenTelemetry providers (Azure Monitor, an APM agent, application
  bootstrap, etc.).
- opentelemetry-semantic-conventions-ai is no longer used anywhere in the
  source; remove it from the listed dependencies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: replace stale observability migration guide with current PR's only relevant migration

The old guide documented the move away from setup_observability(otlp_endpoint=...)
which was an earlier-release API change unrelated to this PR and stale enough that
it's more confusing than helpful at this point. Replace it with a short note on the
single migration this PR introduces: callers of
enable_instrumentation(enable_sensitive_data=True) should switch to
enable_sensitive_telemetry(). Cross-link to the Disabling instrumentation section
for the rare 'force on without enabling sensitive data' use case where
enable_instrumentation() still applies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-20 11:52:08 +00:00
Evan Mattson 4b0522d62d Python: Bump Python package versions for a release (#5964) 
* Bump Python package versions to 1.5.0 for a release

* Promote orchestrations to 1.0.0rc1

* ci(python-setup): merge dynamic exclude into existing workspace exclude

The python-setup action injected exclude = [...] verbatim into
[tool.uv.workspace], producing a duplicate 'exclude' key when the
section already had a static exclude. Scope the rewrite to the
[tool.uv.workspace] section and append the package to the existing
array when present; idempotent if the package is already excluded.

* Address Copilot review feedback: raise inter-package floors to 1.5.0

- foundry, foundry-local: agent-framework-openai >=1.4.0 -> >=1.5.0
- azure-contentunderstanding: agent-framework-foundry >=1.4.0 -> >=1.5.0
- azurefunctions: pin agent-framework-durabletask to >=1.0.0b260519,<2

Keeps lockstep cohort consistent and avoids mixed 1.4.x / 1.5.0 installs.

* Re-include azurefunctions and durabletask in the uv workspace

The pinned durabletask>=1.4.0 floor is enough to make resolution succeed;
the workspace exclude was over-correction and broke CI samples and pyright
type-checking (re-exports in agent_framework/azure/__init__.pyi plus
samples/04-hosting/{azure_functions,durabletask}/ could not resolve their
imports). Dropping them from agent-framework-core[all] still stands so the
metapackage does not pull them.

* Restore azurefunctions and durabletask in agent-framework-core[all]

The durabletask floor pin keeps users on the safe 1.4.0, so they are once
again included in the metapackage. Update CHANGELOG to reflect the pin
rather than an [all] removal.

* Raise uvicorn ceiling in ag-ui and devui to allow 0.42+

The root override-dependencies pins uvicorn[standard]>=0.34.0 (no upper)
and the workspace lock resolves to 0.47.0. The package ceiling <0.42.0
meant the workspace was no longer testing the declared supported range.
Bump to <1 so the lock fits within the declared bounds.

Also picked up by validate-dependency-bounds: refresh stale orchestrations
RC pin in devui dev deps.
 2026-05-20 09:20:53 +09:00
Tao Chen 1b6f7d80fd Python: Record actual served model from Azure OpenAI (#5910) 
* Record actual served model as response model for Azure OpenAI

* Formatting

* Fix tests

* Fix pipeline error

* Comments

* Address review: surface served model via ChatResponse.model

Apply blocking review feedback from PR #5910:

- Use ChatResponse.model / ChatResponseUpdate.model as the source of truth
  for the Azure x-ms-served-model header value, instead of stashing it in
  additional_properties and overriding it again in observability.
  Observability already reads response.model; the chat client now overwrites
  it post-parse when the served-model header is present. Empirically the
  Azure Responses API returns the deployment alias in body.model and the
  actual snapshot (e.g. gpt-5-nano-2025-08-07) in this header.

- Move the AZURE_OPENAI_SERVED_MODEL_HEADER constant out of observability.py
  and into RawOpenAIChatClient (as the SERVED_MODEL_HEADER ClassVar). The
  header is Azure-OpenAI-Responses-API-specific so observability does not
  need to know about it.

- Revert the streaming text_format path to client.responses.stream(...) and
  drop the _pydantic_model_to_text_format_param helper. That helper imported
  from openai.lib._parsing._responses (a private SDK path) and the swap to
  responses.create(stream=True) dropped client-side output_parsed for
  structured-output streaming. The streaming-with-text_format path is the
  only one that does not surface the served-model header - documented inline.

- Wrap the raw streaming responses in async with so the underlying socket
  closes deterministically (continuation_token retrieve + create paths).

- Fix the empty-string / whitespace-only header at the source by stripping
  in _extract_served_model and returning None when nothing remains.

- Revert unrelated formatting-only churn in _skills.py and test_mcp.py.

- Update unit tests to assert against chat_response.model / update.model
  and add an aggregated streaming assertion plus a pin that the
  streaming-with-text_format path does not get the header.

Verified end-to-end against Azure OpenAI Responses API: deployment alias
gpt-5-nano now reports gpt-5-nano-2025-08-07 as ChatResponse.model in both
the non-streaming and streaming paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: preserve streaming structured output finalization

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639

Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>

* refactor: name streaming response finalizer

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639

Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>

* fix: capture streaming response format after prepare

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639

Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>

* refactor: clarify streaming response format capture

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639

Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>

* test: use public API for streaming structured output

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f62076ef-558d-49e8-8fe2-f38d527c9639

Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>

* Inline the served-model header override at its two call sites

The `_apply_served_model_header` helper was a 1-line wrapper around
`_extract_served_model`. Inlining the `if served_model is not None: ...`
matches the pattern already used in the streaming paths and folds the
explanatory docstring onto `_extract_served_model` (which is now the
single place that knows about the header).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: eavanvalkenburg <13749212+eavanvalkenburg@users.noreply.github.com>
 2026-05-19 06:38:53 +00:00
Evan Mattson 3bbc81554b Python: Improve the handling of intermediate outputs for workflows and orchestrations (#5623) 
* Improve the handling of intermediate outputs for workflows and orchestrations

* Address PR review feedback on intermediate output forwarding

- Switch workflow.as_agent() forwarding to an explicit allowlist of {output,
  intermediate, data, request_info} so orchestration-internal events
  (group_chat, handoff_sent, magentic_orchestrator) stay inside the workflow
  instead of leaking into agent responses via str(data) coercion.
- Stop raising on intermediate AgentResponseUpdate in non-streaming run();
  surface the partial as a Message with text_reasoning content. The defensive
  raise still applies to terminal output events, where Update payloads would
  corrupt message ordering.
- Extend the DevUI workflow-event mapper so intermediate yields wrapping
  plain strings, Messages, and list[Message] render as visible output items
  instead of generic completed-trace events.
- Add orchestration coverage for GroupChat, Handoff, and Magentic builders
  (default vs intermediate_outputs=True; structural where end-to-end is heavy).

* Lift output-designation policy into a value type

Replace the ``Workflow._output_executors`` list and the
``RunnerContext.should_label_as_intermediate`` Protocol method with a single
immutable ``OutputDesignation`` value type owned by ``Workflow``. Thread the
designation as a parameter through the existing call chain (Runner ->
EdgeRunner -> Executor -> WorkflowContext) so ``yield_output`` consults the
threaded snapshot directly rather than calling back into the runner context.

Removes the ``InProcRunnerContext._workflow`` back-reference and the
``WorkflowBuilder.build()`` assignment that wired it up. Adds the public
predicate ``Workflow.is_terminal_executor(executor_id)`` for external
observers; ``OutputDesignation`` itself stays package-internal.

Key decisions
- ``OutputDesignation.designated`` is ``frozenset[str] | None`` -- ``None``
  preserves legacy "every yield is type='output'" behavior, any frozenset
  (including empty) opts into strict mode. The ``DeprecationWarning`` for
  legacy mode at build time is unchanged.
- ``output_designation`` is an optional parameter on ``Runner``,
  ``EdgeRunner.send_message``, ``EdgeRunner._execute_on_target``,
  ``Executor.execute``, ``Executor._create_context_for_handler``, and
  ``WorkflowContext.__init__``. Each defaults to legacy ``OutputDesignation()``
  so direct callers (Azure Functions ``CapturingRunnerContext``,
  ``test_runner`` recording fixtures) keep working without ceremony.
- The workflow-level filter in ``_run_core`` reads ``self._output_designation``
  live, preserving today's semantics where mutating the designation after
  build still affects subsequent runs (used by two existing tests).
- ``Workflow.to_dict()`` continues to emit ``"output_executors":
  list[str] | None`` (sorted from the frozenset). Checkpoint format unchanged.

Files changed
- _workflow.py: add ``OutputDesignation`` dataclass; replace
  ``_output_executors`` with ``_output_designation``; add
  ``is_terminal_executor``; delete ``_should_yield_output_event``.
- _runner_context.py: drop ``should_label_as_intermediate`` Protocol method
  and ``InProcRunnerContext`` impl; drop ``_workflow`` back-reference.
- _workflow_builder.py: remove ``context._workflow = workflow`` assignment.
- _runner.py, _edge_runner.py, _executor.py, _workflow_context.py: thread
  ``output_designation`` parameter through the call chain.
- tests/workflow/test_output_designation.py (new): three-state coverage of
  the value type plus the public predicate delegation.
- tests/workflow/test_workflow_builder.py, test_validation.py,
  test_workflow.py, test_runner.py and
  orchestrations/tests/test_orchestration_intermediate_vs_terminal.py:
  switch probes from ``_output_executors`` set checks to
  ``get_output_executors`` / ``is_terminal_executor``; update two
  post-build mutation tests to set ``_output_designation`` instead.

Verification
- core/tests/workflow/, orchestrations/tests/, azurefunctions/tests/:
  1119 passed, 42 skipped, 2 xfailed.
- ``uv run poe lint``: clean.
- ``uv run poe typing``: only the pre-existing
  ``_AGENT_FORWARDED_EVENT_TYPES`` pyright warning from 394bcd607 remains.

Notes for next iteration
- The builder's own ``_output_executors`` attribute (``list[Executor |
  SupportsAgentRun]``) is intentionally untouched; the issue scoped the
  rename to the workflow attribute.
- Adjacent review candidates (twin ``WorkflowAgent`` translators,
  ``_AGENT_FORWARDED_EVENT_TYPES`` kind classifier,
  ``_event_origin_context`` ContextVar removal, ``WorkflowEvent`` ADT
  split, legacy-mode removal) remain out of scope.

* Add explicit workflow output designation

Key decisions

- Extend the internal OutputDesignation value type from terminal-only membership to output/intermediate/hidden classification. Legacy mode remains outputs=None, so workflows built without output_executors or intermediate_executors still label every yield_output as type='output'.

- WorkflowBuilder now accepts intermediate_executors. Providing either designation enters explicit mode; output executors emit output, intermediate executors emit intermediate, and unlisted yield_output payloads are hidden from caller-facing events while remaining in executor_completed data.

- Empty explicit designation, duplicate entries, overlaps, unknown executors, and designated executors without workflow output annotations fail build validation. Existing orchestration builders pass intermediate-capable participants through intermediate_executors to preserve current intermediate_outputs behavior until participant-oriented designation lands.

Files changed

- packages/core/agent_framework/_workflows/_workflow.py, _workflow_builder.py, _workflow_context.py, _validation.py, _events.py

- packages/core/tests/workflow/test_output_designation.py, test_output_executors_contract.py, test_strict_mode_event_labeling.py, test_validation.py, test_workflow.py, test_workflow_agent_intermediate.py

- packages/orchestrations/agent_framework_orchestrations/_sequential.py, _concurrent.py, _group_chat.py, _magentic.py

- packages/core/AGENTS.md

Verification

- uv run pytest packages/core/tests/workflow packages/orchestrations/tests packages/devui/tests/devui/test_mapper.py -q

- uv run pytest packages/azurefunctions/tests -q

- uv run poe lint

- uv run poe typing fails only on pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

Notes for next iteration

- issues/03-core-workflow-explicit-designation.md was moved to issues/done but issues/ remains untracked and intentionally excluded from this commit.

- Slice 4 should tighten workflow.as_agent() mapping for hidden emissions and streaming-only update payloads; Slice 5 should replace orchestration intermediate_outputs with participant-oriented designation.

* Tighten workflow-as-agent output mapping

Key decisions

- Treat AgentResponseUpdate as a streaming-only payload across the workflow.as_agent() adapter, so non-streaming agent runs now reject both terminal output and intermediate workflow events carrying updates.
- Keep streaming classification behavior explicit: terminal update payloads remain normal text content, while intermediate update payloads are rewritten to text_reasoning content.
- Add explicit-mode coverage proving hidden yield_output emissions do not appear in non-streaming AgentResponse messages or streaming AgentResponseUpdate chunks.

Files changed

- packages/core/agent_framework/_workflows/_agent.py
- packages/core/tests/workflow/test_workflow_agent_intermediate.py

Verification

- uv run pytest packages/core/tests/workflow/test_workflow_agent_intermediate.py -q
- uv run pytest packages/core/tests/workflow/test_workflow_agent.py packages/core/tests/workflow/test_workflow_agent_intermediate.py -q
- uv run pytest packages/core/tests/workflow packages/orchestrations/tests packages/devui/tests/devui/test_mapper.py -q
- uv run poe lint
- uv run poe typing fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

Blockers or notes for next iteration

- issues/04-workflow-as-agent-output-mapping.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- Slice 5 should replace orchestration intermediate_outputs with participant-oriented designation.

* Add orchestration participant output designation

Key decisions

- Replace orchestration intermediate_outputs with participant-oriented output_participants and intermediate_participants across Sequential, Concurrent, GroupChat, Magentic, and Handoff builders.
- Keep synthetic final executors terminal by default for Concurrent, GroupChat, and Magentic; keep Sequential's final participant terminal by default; keep Handoff participants terminal by default.
- Centralize participant designation validation for empty explicit designation, duplicates, overlaps, and unknown participants, then map validated participants to workflow output/intermediate executors.

Files changed

- packages/orchestrations/agent_framework_orchestrations/_participant_designation.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- packages/orchestrations/agent_framework_orchestrations/_concurrent.py
- packages/orchestrations/agent_framework_orchestrations/_group_chat.py
- packages/orchestrations/agent_framework_orchestrations/_magentic.py
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/tests/test_orchestration_intermediate_vs_terminal.py
- packages/orchestrations/tests/test_magentic.py

Blockers or notes for next iteration

- issues/05-orchestration-participant-designation.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- Slice 7 should migrate samples and docs away from intermediate_outputs to the new participant designation API.
- uv run poe typing still fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

* Migrate samples to explicit output designation

Key decisions

- Replace sample usage of the removed orchestration intermediate_outputs boolean with participant-oriented intermediate_participants designation.
- Update raw workflow guidance to show output_executors together with intermediate_executors, and document that unlisted yields are hidden in explicit designation mode.
- Keep orchestration final outputs terminal while streaming designated participant responses as intermediate progress, including workflow.as_agent() samples where intermediates map to text_reasoning content.
- Refresh workflow and orchestration README guidance plus the changelog reference so public docs no longer point users at intermediate_outputs.

Files changed

- CHANGELOG.md
- packages/orchestrations/README.md
- samples/README.md
- samples/03-workflows/README.md
- samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py
- samples/03-workflows/orchestrations/README.md
- samples/03-workflows/orchestrations/group_chat_agent_manager.py
- samples/03-workflows/orchestrations/group_chat_philosophical_debate.py
- samples/03-workflows/orchestrations/group_chat_simple_selector.py
- samples/03-workflows/orchestrations/magentic.py
- samples/03-workflows/orchestrations/magentic_human_plan_review.py
- samples/03-workflows/orchestrations/sequential_chain_only_agent_responses.py
- samples/03-workflows/agents/group_chat_workflow_as_agent.py
- samples/03-workflows/agents/magentic_workflow_as_agent.py
- samples/03-workflows/agents/sequential_workflow_as_agent.py
- samples/semantic-kernel-migration/orchestrations/group_chat.py
- samples/semantic-kernel-migration/orchestrations/magentic.py

Blockers or notes for next iteration

- issues/07-samples-and-docs-explicit-output-designation.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- issues/06-devui-intermediate-event-rendering.md remains present and appears already satisfied by existing DevUI mapper/tests from the prior implementation slice.
- PRD-explicit-workflow-output-designation.md remains untracked and intentionally excluded from this commit.

* Render DevUI intermediate workflow outputs

Key decisions

- Preserve workflow output designation metadata on visible DevUI output messages and text deltas so intermediate/data emissions remain distinguishable from terminal output.
- Render intermediate workflow message items in the execution timeline using executor metadata, while excluding them from the final workflow result aggregation.
- Keep terminal output message rendering unchanged and retain legacy data events on the intermediate compatibility path.

Files changed

- packages/devui/agent_framework_devui/_mapper.py
- packages/devui/frontend/src/components/features/workflow/execution-timeline.tsx
- packages/devui/frontend/src/components/features/workflow/workflow-view.tsx
- packages/devui/frontend/src/types/openai.ts
- packages/devui/tests/devui/test_mapper.py

Blockers or notes for next iteration

- issues/06-devui-intermediate-event-rendering.md was moved to issues/done/ but issues/ remains untracked and intentionally excluded from this commit.
- PRD-explicit-workflow-output-designation.md remains untracked and intentionally excluded from this commit.
- uv run poe typing still fails only on the pre-existing packages/core/agent_framework/_workflows/_agent.py _AGENT_FORWARDED_EVENT_TYPES private-use pyright error.

* Fix mypy

* Clarify orchestration participant output config

* Rename participant output kwargs for clarity

output_participants -> final_output_from, intermediate_participants ->
intermediate_output_from. The old names read like categories of
participant; the new names make it clear the kwarg designates which
participants' outputs surface as final vs. intermediate events.

* Rename core workflow output kwargs with deprecation shim

Adds final_output_from / intermediate_output_from as canonical kwargs on
Workflow and WorkflowBuilder. Old output_executors / intermediate_executors
kwargs continue to work but emit DeprecationWarning via a shared coalesce
helper that also rejects supplying both. Wire-format keys in to_dict()
stay as output_executors / intermediate_executors so checkpoint
compatibility is preserved.

Internal call sites in orchestrations and samples updated to the new
names so users following sample code learn the canonical vocabulary;
legacy callers still work with a one-shot warning.

* Suppress pyright reportPrivateUsage on cross-module sentinel import

* Update docstrings

* Propagate sub-workflow intermediate outputs, fix handoff/sequential intermediate-only designation, and shore up tests, sample, and docstrings around the intermediate output contract.

* Add canonical workflow output_from selection

Key decisions:\n- Make output_from the canonical workflow-output allow-list and keep output_executors/final_output_from as deprecated compatibility aliases.\n- Treat empty output_from/intermediate_output_from lists as explicit selections and keep validation responsible for empty, duplicate, overlap, and unknown selections.\n- Remove the branch-only public intermediate_executors WorkflowBuilder kwarg while preserving legacy wire keys in to_dict().\n\nFiles changed:\n- packages/core/agent_framework/_workflows/_workflow.py\n- packages/core/agent_framework/_workflows/_workflow_builder.py\n- packages/core/agent_framework/_workflows/_workflow_context.py\n- packages/core/agent_framework/_workflows/_agent.py\n- packages/core/agent_framework/_workflows/_agent_executor.py\n- packages/core/tests/workflow/* output-selection coverage updates\n- packages/core/AGENTS.md\n- issues/done/001-canonical-list-based-output-selection.md\n\nBlockers/notes:\n- Orchestration builders still pass final_output_from internally; follow-up issue 004 should migrate them to output_from.\n- Legacy omitted-selection behavior and explicit all/all_other literals are left for issues 002 and 003.

* Add explicit all workflow output selection

Key decisions:
- Treat output_from='all' as an explicit workflow-output selection sentinel and expand it at build time to executors with declared workflow output types.
- Keep omitted output selections in legacy all-output mode with a deprecation warning that names output_from and intermediate_output_from and points to output_from='all'.
- Reject intermediate_output_from='all' at construction because the all-output literal is output-only for this issue.

Files changed:
- packages/core/agent_framework/_workflows/_workflow_builder.py
- packages/core/tests/workflow/test_output_executors_contract.py
- issues/done/002-explicit-all-output-and-legacy-migration.md

Blockers/notes:
- all_other intermediate-output selection remains for issue 003.
- Workflow-as-agent/orchestration parity remains for issue 004.

* Add all-other intermediate output selection

Key decisions:
- Treat intermediate_output_from='all_other' as an explicit intermediate-output selection sentinel and expand it at build time after the workflow graph is complete.
- Expand all_other to output-capable executors not selected by output_from; omitted or empty output_from selects no workflow outputs, while output_from='all' leaves an empty intermediate selection.
- Keep output_from='all_other' invalid so all_other remains intermediate-output-only and runtime classification still receives concrete executor-id sets.

Files changed:
- packages/core/agent_framework/_workflows/_workflow_builder.py
- packages/core/tests/workflow/test_output_executors_contract.py
- issues/done/003-all-other-intermediate-output-selection.md

Blockers/notes:
- Workflow-as-agent and orchestration parity remains for issue 004.
- Full documentation updates remain for issue 005.

* Add orchestration output selection parity

Key decisions:
- Expose output_from on sequential, concurrent, group chat, handoff, and magentic builders while keeping final_output_from as a deprecated compatibility alias.
- Resolve orchestration participant selections through the same explicit rules as workflows: output_from='all', intermediate_output_from='all_other', hidden unselected participant payloads, and overlap/duplicate/unknown/invalid-literal validation.
- Continue preserving documented orchestration defaults by always designating each pattern's terminal internal executor where applicable.

Files changed:
- packages/orchestrations/agent_framework_orchestrations/_participant_output_config.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- packages/orchestrations/agent_framework_orchestrations/_concurrent.py
- packages/orchestrations/agent_framework_orchestrations/_group_chat.py
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/agent_framework_orchestrations/_magentic.py
- packages/orchestrations/agent_framework_orchestrations/_orchestration_request_info.py
- packages/orchestrations/tests/test_orchestration_intermediate_vs_terminal.py
- issues/done/004-workflow-as-agent-and-orchestration-parity.md

Blockers/notes:
- Full documentation and sample migration wording remains for issue 005.
- Existing tests that intentionally use final_output_from now emit the new deprecation warning.

* Document workflow output selection contract

Key decisions:
- Use Workflow Output and Intermediate Output as the developer-facing terms for selected caller-facing emissions.
- Document output_from and intermediate_output_from as the canonical API, with output_from as an allow-list and unselected payloads hidden unless explicitly selected as intermediate.
- Add scenario and invalid-selection tables for workflow and orchestration docs, including legacy omission warnings, output_from='all', intermediate_output_from='all_other', list selections, invalid literals, overlap, duplicates, unknown selections, and empty explicit selections.
- Migrate samples away from final_output_from and output_executors except where compatibility aliases are explicitly documented.

Files changed:
- packages/core/AGENTS.md
- packages/orchestrations/README.md
- packages/orchestrations/agent_framework_orchestrations/_handoff.py
- packages/orchestrations/agent_framework_orchestrations/_sequential.py
- samples/03-workflows/README.md
- samples/03-workflows/control-flow/intermediate_vs_terminal_outputs.py
- samples/03-workflows/human-in-the-loop/agents_with_approval_requests.py
- samples/03-workflows/orchestrations/README.md
- samples/04-hosting/foundry-hosted-agents/responses/05_workflows/main.py
- scripts/sample_validation/create_dynamic_workflow_executor.py
- issues/done/005-document-output-selection-contract.md

Blockers/notes:
- Direct full Ruff on scripts/sample_validation/create_dynamic_workflow_executor.py still reports pre-existing docstring/print/line-length issues outside this docs migration; syntax-focused checks for changed files pass.
- No remaining AFK issue files are present under issues/.

* Latest updates

* Typing fixes

* Cleanup
 2026-05-19 00:15:25 +00:00
Giles Odigwe 47fa59f8e9 Python: bump package versions for 1.4.0 release (#5872) 
* fixes

* fixes

* Python: bump package versions for 1.4.0 release

Cuts the python-1.4.0 release. MINOR bump on the released cohort
(agent-framework, agent-framework-core, agent-framework-openai,
agent-framework-foundry: 1.3.0 -> 1.4.0), driven by breaking changes
in experimental skills API and new features. All 21 beta packages
stamp 1.0.0b260514, all 3 alpha packages stamp 1.0.0a260514, and
ag-ui remains at 1.0.0rc1 (freshly promoted). Date stamp reflects
2026-05-14 Pacific.

- Released cohort: 1.3.0 -> 1.4.0
- Beta packages (21): 1.0.0b260507 -> 1.0.0b260514
- Alpha packages (3): 1.0.0a260507 -> 1.0.0a260514
- ag-ui: stays at 1.0.0rc1 (dep bound updated only)
- Inter-package dependency lower bounds updated (>=1.3.0 -> >=1.4.0)
- Fix chatkit StructuredInputItem exhaustiveness for openai-chatkit 1.6.4
- Update CHANGELOG compare links
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-15 09:31:03 +09:00
Evan Mattson 09a3d0d307 Python: Strip server-issued response item IDs under storage (#3295) (#5690) 
Fixes microsoft/agent-framework#3295. When the OpenAI Responses chat
client sends a request that carries previous_response_id / conversation_id
/ conversation, the server already has the prior turn's response items
and rejects duplicates with "Duplicate item found with id fc_xxx". The
chat client was re-sending them inline whenever the input messages still
carried the items in additional_properties (workflow replay, history
providers, etc.), which broke any tool-using agent with persistent
history.

Decisions:
- Single chokepoint: _prepare_message_for_openai. When the resulting
  request uses service-side storage, drop function_call, reasoning,
  approval-request/response, and local-shell-call items from the wire
  input. Keep function_result with its call_id; the server pairs it to
  the prior function_call via that key.
- function_result is preserved unconditionally except for the local-shell
  variant, which carries its own server-issued item id.
- No public API change. Wire format change is subtractive and only on
  requests that would otherwise 400.
- Re-pointed the strict-xfail in test_full_conversation.py from #4047 to
  #3295. Kept xfail because the test asserts executor-level session-id
  clearing, which is the defense-in-depth half tracked by 3295-03; this
  slice closes the wire-level half.

Files:
- python/packages/openai/agent_framework_openai/_chat_client.py: strip
  rule applied alongside the existing reasoning-item branch.
- python/packages/openai/tests/openai/test_openai_chat_client.py: four
  new tests pin the contract (function_call, approval, local-shell-call
  stripped under storage; everything kept without storage). Updated
  pre-existing tests that exercised the storage-on path to either pass
  request_uses_service_side_storage=False explicitly or assert the new
  strip behavior.
- python/packages/foundry/tests/foundry/test_foundry_chat_client.py:
  same explicit storage-off opt-in for the inherited test.
- python/packages/core/tests/workflow/test_full_conversation.py:
  re-pointed xfail reason to #3295 and the executor-level follow-up.

Notes for next iteration:
- 3295-01 (HITL wire-format validation against live OpenAI/Foundry) was
  not run; it requires the user's API credentials. The PRD design is
  locked but the empirical confirmation is still pending. If script 3
  fails on either provider, this slice may need to be revisited.
- 3295-03 (clear service_session_id in AgentExecutor on full-history
  replay) remains open. After it lands the xfail in
  test_full_conversation.py can be removed.
- pytest was not run in this iteration because uv-based pytest commands
  required interactive approval. Validation rests on careful reading;
  next iteration should run the openai + core test suites.
 2026-05-13 22:09:04 +00:00
Giles Odigwe 0340b7596b Python: bump package versions for 1.3.0 release (#5706) 
* Python: bump package versions for 1.3.0 release

MINOR bump on the released cohort (agent-framework, agent-framework-core,
agent-framework-openai, agent-framework-foundry: 1.2.2 -> 1.3.0). All 22
beta packages stamp 1.0.0b260507 and all 3 alpha packages stamp
1.0.0a260507 per the lockstep convention. Date stamp reflects 2026-05-07
Pacific.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: bump foundry_local openai floor, fix devui orchestrations pin, clarify breaking scope

- foundry_local: bump agent-framework-openai lower bound from >=1.1.0 to >=1.3.0
- devui: update stale agent-framework-orchestrations dev pin from 1.0.0b260402 to 1.0.0b260507
- CHANGELOG: clarify [BREAKING] applies to experimental skills API only

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert devui orchestrations pin to 1.0.0b260402 to avoid breaking DevUI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-08 08:57:02 +09:00
Evan Mattson e56e6dad4d Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption (#5671) 
* Remove Foundry toolbox helpers; standardize on MCP for toolbox consumption

- Remove RawFoundryChatClient.get_toolbox() and its fetch_toolbox import
- Remove fetch_toolbox, select_toolbox_tools, get_toolbox_tool_name,
  get_toolbox_tool_type, FoundryHostedToolType, ToolboxToolSelectionInput
  from agent_framework_foundry._tools
- Remove ExperimentalFeature.TOOLBOXES from _feature_stage.py (no consumers)
- Drop toolbox re-exports from agent_framework_foundry/__init__.py and
  agent_framework.foundry namespace
- Update _sanitize_foundry_response_tool docstring to remove toolbox framing;
  sanitization logic itself is unchanged
- Update _agent.py docstring: 'toolbox-fetched MCP' → 'hosted MCP'
- Delete tests/test_toolbox.py (all tests covered removed helpers)
- Update test_foundry_chat_client.py: rename/redoc tests that mentioned
  toolbox but test sanitization that remains
- Delete foundry_chat_client_with_toolbox.py (bespoke toolbox API sample)
- Delete foundry_toolbox_context_provider.py (relied on select_toolbox_tools)
- Rename foundry_chat_client_with_toolbox_mcp.py →
  foundry_chat_client_with_toolbox.py (canonical MCP pattern)
- Rewrite 04_foundry_toolbox/main.py to use MCPStreamableHTTPTool
- Update provider/README, context_providers/README, 04_foundry_toolbox/README

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): update 06_files sample to consume toolbox via MCP (#5670)

Replace removed get_toolbox/select_toolbox_tools APIs with
MCPStreamableHTTPTool, using allowed_tools=["code_interpreter"] to
select only the code interpreter from the toolbox endpoint.

Update .env.example and README to use FOUNDRY_TOOLBOX_ENDPOINT
instead of TOOLBOX_NAME.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove non-existent toolbox helper APIs from README (#5670)

Remove the 'fetch, optionally filter, and pass tools directly' pattern
from the FoundryChatClient toolbox documentation, as select_toolbox_tools
and get_toolbox were removed. Only the MCP endpoint pattern is documented.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): remove residual toolbox docstring references and reproduction report

Remove REPRODUCTION_REPORT.md (workflow artifact that should not be committed),
and update two remaining docstring references that still said 'toolbox reads'
/'toolbox definition' after the toolbox helpers were removed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Remove bespoke Foundry toolbox helpers; standardize on MCP for toolbox consumption

Fixes #5670

* fix(#5670): resolve toolbox endpoint from TOOLBOX_NAME fallback; add namespace regression tests

- Add _resolve_toolbox_endpoint() helper in 04_foundry_toolbox/main.py and
  06_files/main.py that prefers FOUNDRY_TOOLBOX_ENDPOINT but falls back to
  deriving the MCP URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME — fixing
  the startup KeyError when agents are deployed via azd provision (which injects
  TOOLBOX_NAME, not FOUNDRY_TOOLBOX_ENDPOINT).
- Update 04_foundry_toolbox/.env.example to use FOUNDRY_TOOLBOX_ENDPOINT
  (consistent with 06_files).
- Add TOOLBOX_NAME env var to 06_files/agent.yaml so deployed agents have it
  available for the fallback derivation.
- Update both READMEs to document the two ways to supply the toolbox endpoint.
- Add test_foundry_namespace_no_longer_exposes_toolbox_helpers() with negative
  assertions for FoundryHostedToolType, get_toolbox_tool_name,
  get_toolbox_tool_type, and select_toolbox_tools — guarding against accidental
  re-introduction of removed symbols.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(samples): fail fast on empty FOUNDRY_TOOLBOX_ENDPOINT; add unit tests

Addresses review feedback for #5670:

- In _resolve_toolbox_endpoint() (04_foundry_toolbox/main.py and
  06_files/main.py) change the walrus-operator check from a truthy
  test to an explicit 'is not None' guard.  An explicitly set empty
  string now raises ValueError immediately with a clear message
  instead of silently falling through to the fallback URL
  construction.

- Add tests/samples/hosting/test_toolbox_endpoint.py covering both
  sample modules:
    (a) FOUNDRY_TOOLBOX_ENDPOINT set → returned as-is
    (b) FOUNDRY_TOOLBOX_ENDPOINT set to empty string → ValueError
    (c) fallback constructs URL from FOUNDRY_PROJECT_ENDPOINT + TOOLBOX_NAME,
        stripping trailing slashes
    (d) neither variable group set → KeyError

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: remove extraneous test and docstring content

- Remove test_foundry_namespace_no_longer_exposes_toolbox_helpers (no longer warranted)
- Remove docstring from _agent.py _prepare_tools_for_openai (extraneous)
- Trim _chat_client.py _prepare_tools_for_openai docstring to one-liner (toolbox references no longer relevant)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove remaining extraneous docstring from RawFoundryChatClient._prepare_tools_for_openai

Address review comment on PR #5671: reviewer noted the description
isn't warranted now that toolbox helpers have been removed. Matches
the pattern in RawFoundryAgentChatClient which has no docstring.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-06 23:56:16 +00:00
Evan Mattson f3db60fa65 Python: Support GPT-5 verbosity option and restore Foundry agent_reference (#5619) 
* Python: Support GPT-5 verbosity option and restore Foundry agent_reference

Adds verbosity as a typed Literal["low","medium","high"] field on
OpenAIChatOptions (Responses API) and OpenAIChatCompletionOptions (Chat
Completions API), set in the same way as the existing reasoning options.
For the Responses API, top-level verbosity is translated to the nested
text.verbosity shape the OpenAI service expects. The same field flows
through to FoundryChatClient via the existing FoundryChatOptions alias.

Also fixes #5582: PR #5447 removed the agent_reference injection from
RawFoundryAgentChatClient._prepare_options, so first-turn calls against
a Foundry Prompt Agent went out without model and without agent_reference
and were rejected by the Responses API with "Missing required parameter:
'model'". Restores the injection on the non-preview path
(allow_preview=False) and adds a guard test that asserts the preview
path does not inject agent_reference, since the preview SDK injects it
via project_client.get_openai_client(agent_name=...).

Closes #5516
Closes #5582

* Python: Address Copilot review on PR #5619

- Foundry verbosity sample docstring: replace the misleading "set deployment
  name on model=" instruction with the actual env-var pattern the sample relies
  on (FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL).
- _build_agent_reference docstring: clarify the helper is used for both
  Prompt Agents and HostedAgents on the non-preview path.
- Add a Responses API test that locks in the documented precedence rule:
  when both top-level verbosity and text["verbosity"] are supplied, the
  top-level value wins.

* Python: Drop redundant Foundry verbosity sample and list OpenAI sample in README

- Remove samples/02-agents/providers/foundry/foundry_chat_client_verbosity.py
  per review feedback. The verbosity functionality is identical across the
  OpenAI and Foundry clients (FoundryChatOptions is an alias of
  OpenAIChatOptions), so a single sample on the OpenAI side is sufficient.
- Add the new client_verbosity.py entry to the OpenAI samples README.
 2026-05-04 21:21:40 +00:00
Giles Odigwe 540193ccef Python: Reduce flaky integration tests and improve CI signal quality (#5454) 
* Enable Ollama integration tests in CI and rename report to Integration Test Report

- Install Ollama, cache models (qwen2.5:0.5b + nomic-embed-text), and start
  server in the Misc integration job for both workflow files
- Set OLLAMA_MODEL and OLLAMA_EMBEDDING_MODEL env vars so the 5 Ollama tests
  are no longer skipped
- Rename Flaky Test Report to Integration Test Report throughout (job names,
  artifact names, cache keys, file names, script titles/docstrings)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Bump Ollama model to qwen2.5:1.5b for better instruction following

The 0.5b model was too small to reliably follow simple prompts like
'Say Hello World', causing test assertion failures. The 1.5b model
follows instructions more reliably while still being small enough
for fast CI pulls (~1GB).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable reliable streaming integration tests

Remove the hard skip on test_03_reliable_streaming tests that was
temporarily disabled for instability investigation. CI infrastructure
(Azurite, DTS emulator, Redis, func CLI) is already in place.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable skipped Functions/DurableTask tests and bump timeout to 480s

- Remove hard skips from 4 tests in test_11_workflow_parallel.py
- Remove hard skip from test_conditional_branching in test_06_dt_multi_agent_orchestration_conditionals.py
- Increase pytest --timeout from 360 to 480 for Functions+DurableTask CI job
- Updated in both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip failing Functions/DurableTask tests with specific root causes

- test_11_workflow_parallel (4 tests): xdist worker crashes during execution
- test_conditional_branching: orchestration fails with RuntimeError, not a timeout
- Keep 480s timeout bump for remaining Functions tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix auth routing in samples 06/11: api_key -> credential for Azure OpenAI

Both samples passed a bearer token provider via api_key= which caused the
client to route to api.openai.com instead of Azure OpenAI, resulting in
401 Unauthorized. Changed to credential= which correctly triggers Azure
routing and picks up AZURE_OPENAI_ENDPOINT from the environment.

- samples/azure_functions/11_workflow_parallel/function_app.py: 1 fix
- samples/durabletask/06_multi_agent_orchestration_conditionals/worker.py: 2 fixes
- Re-enable 4 parallel workflow tests and 1 conditional branching test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-skip parallel workflow tests: xdist worker distribution issue

The 4 parallel workflow tests crash because xdist worksteal distributes
them across separate workers, each spawning its own func process against
shared emulators. Auth fix (api_key->credential) was valid and stays.
test_conditional_branching now passes with the auth fix.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix E501 line-too-long in azurefunctions parallel test skip reasons

Wrap skip reason strings to stay within 120 char line limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add retry logic and port-conflict fix for Ollama CI setup

- Kill any auto-started Ollama before launching serve (fixes port
  conflict: 'address already in use')
- Retry ollama pull up to 3 times with 15s backoff (fixes 429 rate
  limit failures)
- Applied to both python-merge-tests.yml and python-integration-tests.yml

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix flaky integration tests and re-enable skipped tests

- Foundry agent: add allow_preview=True to custom client test
- Foundry hosting: raise max_output_tokens 50->200, add temperature,
  relax assertion in test_temperature_and_max_tokens
- Foundry embedding: update skip reason with root cause (endpoint mismatch)
- OpenAI file search: fix vector store indexing race condition by polling
  file_counts before querying; fix get_streaming_response -> get_response(stream=True)
- Azure OpenAI file search: remove skip (transient 500 resolved)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove temperature from foundry hosting test (unsupported by CI model)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Stabilize Ollama tool call integration tests with no-arg function

Use a no-argument greet() function instead of hello_world(arg1) for
integration tests. The 1.5B model in CI is unreliable at generating
correct tool call arguments, causing 'Argument parsing failed' errors.
A no-arg function eliminates this flakiness entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Increase reliable streaming test timeouts from 30s to 60s

The LLM call through Azure OpenAI + Redis streaming pipeline can exceed
30s in CI due to cold starts or throttling. Raise to 60s to reduce
flaky timeouts while still bounded by pytest's 120s per-test limit.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Re-enable workflow parallel tests with xdist_group marker

The tests were skipped because xdist distributes module tests across
workers, each spawning their own func process (port conflicts). Adding
xdist_group forces all tests in this module onto a single worker so
the module-scoped function_app_for_test fixture works correctly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert "Re-enable workflow parallel tests with xdist_group marker"

This reverts commit 455c28da62.

* Rename flaky_report to integration_test_report and add try/finally cleanup

- Rename scripts/flaky_report/ to scripts/integration_test_report/ to
  reflect expanded scope beyond flaky-test detection
- Update workflow references in both CI files
- Wrap file search integration tests in try/finally to ensure vector
  store cleanup runs even on test failure or timeout

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix Ollama pull failure propagation and Azure OpenAI vector store readiness

- Ollama CI: fail the step immediately if model pull fails after 3
  retries instead of silently proceeding to tests
- Azure OpenAI file search: add the same vector-store readiness polling
  that was applied to the non-Azure OpenAI tests, preventing eventual
  consistency race conditions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* remove load_dotenv from test file

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-05-01 00:41:39 +00:00
Evan Mattson f5419b9f38 Python: bump package versions for 1.2.2 release (#5561) 
* Python: bump package versions for 1.2.2 release

PATCH bump (1.2.1 -> 1.2.2) for the released cohort. Five PRs land in this
window:

- agent-framework-openai: fix file_search citations breaking the assistant-
  message history roundtrip (#5557) — drives the released-tier PATCH
- agent-framework-orchestrations: [BREAKING] standardize orchestration
  terminal outputs as AgentResponse (#5301)
- agent-framework-core, agent-framework-declarative: preserve Workflow.run()
  shared state across calls, accept list[Message] in declarative start
  executor, and coerce Enum values when serializing PowerFx symbols (#5531)
- agent-framework-foundry-hosting: add hosted Durable Workflow support
  (#5531)
- agent-framework-azure-contentunderstanding: new alpha package — Azure AI
  Content Understanding context provider (#4829)
- dependencies: workspace package dependency refresh (#5555)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260429 and all 4
alpha packages (now including the new contentunderstanding) stamp
1.0.0a260429. Date stamp reflects 2026-04-29 Pacific. Every non-core package
floor on agent-framework-core is raised to >=1.2.2; the new
contentunderstanding package's stale >=1.0.0 floor is brought into line.

Two follow-on fixes bundled to keep validate-dependency-bounds-test green
at lowest-direct resolution:
- Bump agent-framework-azure-contentunderstanding's azure-ai-content
  understanding lower bound from >=1.0.0 to >=1.0.1 (1.0.0 ships without
  proper typing — pyright reports 65 unknown-type errors)
- Add pyright ignore comments to core/foundry/__init__.pyi for the new
  alpha package's type-stub imports, since alpha packages are not in
  core's [all] extra and therefore aren't installed at lowest-direct

* Python: add #5552 to 1.2.2 CHANGELOG

Add the streaming-span observability fix to the Fixed section. PR is on
upstream/main but not yet pulled into origin/main; the code itself will
land via the PR merge.

* Python: address PR #5561 review feedback on dependency bounds

Two packaging fixes flagged in review:

1. agent-framework-azure-contentunderstanding: add agent-framework-foundry
   as a runtime dependency. The package's README directs users to
   `pip install agent-framework-azure-contentunderstanding --pre` and the
   basic example imports `FoundryChatClient` from `agent_framework.foundry`,
   so the documented install path was failing with ImportError. Pulling
   agent-framework-foundry into deps makes the advertised entry path
   self-contained.

2. agent-framework-foundry: bump agent-framework-openai lower bound from
   >=1.1.0 to >=1.2.2,<2. Foundry imports private modules from
   agent_framework_openai (`_chat_client.py:22`, `_agent.py:34`), so
   resolvers were free to pair foundry==1.2.2 with older OpenAI versions
   that lack this release's coordinated Responses/history fix. Lockstep the
   floor with the released cohort to prevent mismatched installs.

Both changes pass `validate-dependency-bounds-test` lower + upper at
their respective packages.
 2026-04-29 17:51:48 +09:00
Evan Mattson 866a325b48 Python: [BREAKING] Standardize orchestration terminal outputs as AgentResponse (#5301) 
* Fix orchestration outputs so as_agent() returns the final answer only. Align other orchestration outputs

* Fix orchestration output issues from review comments

1. Sample cleanup: Remove commented-out FoundryChatClient block and update
   prerequisites to reference OPENAI_CHAT_MODEL_ID instead of FOUNDRY_* vars.

2. Sequential approval output: Change _EndWithConversation.end_with_agent_executor_response
   from a no-op sink to yield response.agent_response. When the last participant is
   AgentApprovalExecutor (via with_request_info), _EndWithConversation is the output
   executor so the yield produces the terminal answer. When the last participant is a
   regular AgentExecutor, _EndWithConversation is not in output_executors so the yield
   is silently filtered out.

3. Forward data events through WorkflowExecutor: _process_workflow_result now also
   forwards 'data' events from sub-workflows so that emit_intermediate_data=True on
   AgentExecutor works correctly when wrapped in AgentApprovalExecutor.

4. Concurrent docstring: Update _AggregateAgentConversations docstring to say
   'deterministic participant order' instead of 'completion order'.

5. Add test_concurrent_intermediate_outputs_emits_data_events verifying that
   ConcurrentBuilder(intermediate_outputs=True) emits per-participant data events
   alongside the single aggregated output event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add tests for sequential workflow with_request_info and intermediate_outputs (#5301)

Address PR review comments 2, 3, and 5:

- Add test_sequential_request_info_last_participant_emits_output:
  Verifies that when the last participant is wrapped via with_request_info()
  (AgentApprovalExecutor), the workflow still emits a terminal output after
  approval, exercising the _EndWithConversation.end_with_agent_executor_response
  fallback path.

- Add test_sequential_request_info_with_intermediate_outputs_emits_data_events:
  Verifies that emit_intermediate_data=True works correctly through
  AgentApprovalExecutor wrapping—WorkflowExecutor._process_result already
  forwards data events from sub-workflows, so intermediate agent responses
  surface as data events in the parent workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright type errors from AgentResponse output refactor (#5301)

Update cast() calls in _group_chat.py and _magentic.py to use
WorkflowContext[Never, AgentResponse] instead of the old
WorkflowContext[Never, list[Message]], matching the updated method
signatures in _base_group_chat_orchestrator.py.

Fix _sequential.py _EndWithConversation.end_with_agent_executor_response
to declare WorkflowContext[Any, AgentResponse] so yield_output accepts
AgentResponse[None].

Fix _workflow_executor.py data event forwarding to handle nullable
executor_id.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportUnknownVariableType in _agent.py (#5301)

Extract event.data into a typed local variable before the isinstance
check to avoid pyright narrowing it to AgentResponse[Unknown].

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportMissingImports for orjson in file history samples (#5301)

Add pyright: ignore[reportMissingImports] to orjson imports that are
already guarded by try/except ImportError, matching the existing pattern
used elsewhere in the samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5301: review comment fixes

* Address review feedback for #5301: review comment fixes

* Revert sequential_workflow_as_agent sample to FoundryChatClient

Reverts the mistaken switch from FoundryChatClient to OpenAIChatClient
in the sequential workflow as agent sample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address ultrareview feedback: emit_data_events rename + WorkflowAgent reasoning conversion

Layered on top of the prior review-feedback work in this branch.

Renames:
- AgentExecutor.emit_intermediate_data -> emit_data_events (mechanical
  rename; orchestration semantics live at the orchestration layer, not
  the general-purpose executor). Forwarded through MagenticAgentExecutor,
  AgentApprovalExecutor, and all orchestration call sites.
- HandoffAgentExecutor._check_terminate_and_yield -> _should_terminate
  (pure predicate; no longer yields anything). HandoffBuilder docstring
  rewritten to describe the new per-agent AgentResponse output contract.

WorkflowAgent reasoning-content conversion:
- Add _rewrite_text_to_reasoning(contents) and _msg_as_reasoning(msg)
  helpers; the as_agent() path now reframes text content from data events
  as text_reasoning Content blocks before merging into the AgentResponse.
- Consumers iterate msg.contents and branch on content.type — same path
  they already use for Claude thinking and OpenAI reasoning. No new
  field on Message/AgentResponse/WorkflowEvent.
- Streaming branch constructs fresh AgentResponseUpdate instances instead
  of mutating shared payloads (regression test added).
- Helper _msg_maybe_reasoning consolidates the conditional rewrite at
  three call sites in the non-streaming conversion.

Tests:
- TestWorkflowAgentReasoningHelpers + TestWorkflowAgentDataEventReasoningConversion
  add 9 new tests covering helpers, non-streaming, streaming, mixed content,
  already-reasoning passthrough, and mutation-safety regression.
- Updated test_sequential_as_agent_with_intermediate_outputs_includes_chain
  to assert text_reasoning content for intermediate agents.

* Fix pyright: widen event.data to Any to avoid partial-unknown narrowing

The streaming conversion path narrowed event.data via isinstance against
generic AgentResponse, producing AgentResponse[Unknown] and tripping
reportUnknownVariableType/reportUnknownMemberType. Binding data: Any
before the check keeps runtime behavior identical while restoring a fully
known type for downstream access.

* Clean up design

* Scope to agent output semantics only

* yield AgentResponseUpdate streaming, AgentResponse non-streaming

* Fix mypy/pyright: widen cast types at GroupChat callsites

Eight callsites in _group_chat.py still cast to WorkflowContext[Never,
AgentResponse] but the base orchestrator methods now accept the wider
WorkflowContext[Never, AgentResponse | AgentResponseUpdate] (mode-aware
yields). W_OutT is invariant, so the narrower cast is not assignable.
Magentic was widened in the same commit; this catches the GroupChat
callsites that were missed.

* Python: skip flaky Foundry / Foundry Hosting integration tests (#5553)

These two integration tests have been failing in the merge queue across
multiple unrelated PRs (5301, 5531). Both are marked `@pytest.mark.flaky`
with 3 retries, but all attempts fail back-to-back. Skipping both with a
reason pointing to #5553 so they can be fixed properly without continuing
to block unrelated merges.

- packages/foundry_hosting/tests/test_responses_int.py::TestOptions::test_temperature_and_max_tokens
- packages/foundry/tests/foundry/test_foundry_embedding_client.py::TestFoundryEmbeddingIntegration::test_text_embedding_live

Also includes a one-line uv.lock specifier-ordering normalization
auto-applied by the poe-check pre-commit hook.

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-29 00:35:36 +00:00
Evan Mattson 3a463b8bf6 Python: bump package versions for 1.2.1 release (#5536) 
* Python: bump package versions for 1.2.1 release

PATCH bump (1.2.0 -> 1.2.1) for the released cohort. The release window
covers two PRs, no new public APIs:

- agent-framework-core: prevent inner_exception from being lost in
  AgentFrameworkException (#5167)
- samples: add requirements.txt and .env.example to the a2a/ hosting
  sample for pip-based setup (#5510)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260428 and all
3 alpha packages stamp 1.0.0a260428, regardless of per-package code
churn. Every non-core package floor on agent-framework-core is raised to
>=1.2.1 to keep cohort signaling consistent. Date stamp reflects the
local (Asia) cut date 2026-04-28.

* Python: silence pyright unknown-type warnings in hosted-env detection

`azure.ai.agentserver.core` is probed at runtime via `importlib.util.find_spec`
and is not a declared dependency. The existing `# pyright: ignore[reportMissingImports]`
suppresses the missing-import warning, but at `lowest-direct` resolution pyright
still reports the imported symbol (`AgentConfig`) and its members (`from_env`,
`is_hosted`) as unknown, breaking `validate-dependency-bounds-test` for
`packages/core`.

Extend the existing ignore to cover `reportUnknownVariableType` on the import
and `reportUnknownMemberType` on the call site so the bounds check returns to
green. Behavior is unchanged.

Latent since #5455 (shipped in 1.2.0).

* Python: raise agent-framework-gemini lower bound to google-genai>=1.65.0

The Gemini chat client references several `google.genai.types` symbols
(`FileSearch`, `ThinkingLevel`, `SearchTypes`, `McpServer`,
`StreamableHttpTransport`, plus call-site keyword args `mcp_servers` and
`search_types`) that are not present at the lower bound of `google-genai>=1.0.0`.
At `lowest-direct` resolution this caused `validate-dependency-bounds-test` to
fail for `packages/gemini` with eleven `reportAttributeAccessIssue` /
`reportUnknownVariableType` errors.

Walking the upstream `google.genai.types` API:
- `GoogleMaps`, `AuthConfig`: present from 1.40.0
- `FileSearch`: introduced in 1.49.0
- `ThinkingLevel`: introduced in 1.55.0
- `SearchTypes`, `McpServer`, `StreamableHttpTransport`: introduced in 1.65.0

Bump the lower bound to 1.65.0 — the minimum version that exposes every symbol
the package actually uses. Keep the `<2.0.0` upper cap unchanged. With this
bump `validate-dependency-bounds-test` passes for both lower and upper
resolution scenarios across all 27 workspace packages.

Latent since #4847 (Gemini package introduction in 1.1.0); aggravated by
subsequent feature additions that pulled in newer `types.*` symbols.

* Python: add dependabot bumps to 1.2.1 CHANGELOG

Catalog the 15 dependabot dependency updates that merged on `upstream/main`
between python-1.2.0 and the 1.2.1 cut window under a new Changed section:

- Workspace dev/runtime deps: `rich`, `prek`, `python-multipart`, `pyasn1`,
  `pytest` (ag-ui, devui, lab), `uv` (lab)
- Frontend deps: `vite` (devui, chatkit), `postcss` (devui, chatkit, handoff),
  `picomatch` (devui, handoff)

CHANGELOG-only — no source or pyproject.toml changes. PRs themselves merged
upstream independently of this release branch and will be brought in via the
PR merge.
 2026-04-28 18:23:26 +09:00
Evan Mattson 0b69d7fd15 Python: Bump Python package versions for 1.2.0 release (#5468) 
* Bump Python package versions for 1.2.0 release

Released tier bumps 1.1.1 -> 1.2.0 (core, openai, foundry, root) to
reflect additive public APIs landed since 1.1.0: functional workflow API
(#4238) and FunctionTool SKIP_PARSING sentinel (#5424). All beta packages
stamped 1.0.0b260424, alpha packages 1.0.0a260424. All 26 non-core
agent-framework-core floors raised to >=1.2.0,<2. CHANGELOG consolidates
the never-tagged 1.1.1 entries with the post-merge additions into [1.2.0].

* Update CHANGELOG footer links for 1.2.0

Advance [Unreleased] comparison base from python-1.1.0 to python-1.2.0
and add a [1.2.0] reference link comparing python-1.1.0...python-1.2.0
so the heading links resolve correctly.

* Fix CHANGELOG: restore [1.1.1] section and add proper [1.2.0]

Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which
wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0.
This restores [1.1.1] to its origin/main content and adds a new [1.2.0]
section above containing only the commits in python-1.1.1..HEAD:

- #4238 functional workflow API
- #5142 GitHub Copilot OpenTelemetry
- #2403 A2A bridge support
- #5070 oauth_consent_request events in Foundry clients
- #5447 FoundryAgent hosted agent sessions
- #5459 hosting server dependency upgrade + types
- #5389 AG-UI reasoning/multimodal parsing fix
- #5440 stop [TOOLBOXES] warning spam
- #5455 user agent prefix fix

Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and
adds the missing [1.1.1] reference link.
 2026-04-24 19:54:59 +09:00
Giles Odigwe 7b70f80036 Python: Surface oauth_consent_request events from Responses API in Foundry clients (#5070) 
* Fix Foundry clients not surfacing oauth_consent_request events (#5054)

Override _parse_chunk_from_openai in both RawFoundryChatClient and
RawFoundryAgentChatClient to intercept response.output_item.added
events with item.type == 'oauth_consent_request'. The consent link
is validated (HTTPS required) and converted to
Content.from_oauth_consent_request, which the AG-UI layer already
knows how to emit as a CUSTOM event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review feedback for #5054 OAuth consent parsing

- Extract shared helper (try_parse_oauth_consent_event) to avoid
  duplicated logic between RawFoundryChatClient and
  RawFoundryAgentChatClient
- Use urllib.parse.urlparse() for HTTPS validation instead of
  case-sensitive startswith check
- Sanitize log messages to avoid leaking consent_link tokens;
  log only item id
- Add model=self.model to ChatResponseUpdate to match parent behavior
- Add assertions on role, raw_representation, and model in happy-path
  tests
- Add test for empty-string consent_link
- Add test verifying non-oauth events delegate to super()

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Handle response.oauth_consent_requested top-level event (#5054)

Add support for the top-level response.oauth_consent_requested stream
event in addition to the response.output_item.added variant. The
service may emit either form; handle both so the consent link is
reliably surfaced.

Extract _validate_consent_link helper within _oauth_helpers.py to
reduce nesting.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5054: Python: [Bug]: `FoundryAgent` (Responses API) Does Not Surface `oauth_consent_request` as a CUSTOM AG-UI Event

* Address review feedback: defensive getattr and dedicated helper tests (#5054)

- Use getattr(event, 'type', None) in try_parse_oauth_consent_event
  for defensive access against malformed events without a type attribute
- Add test_oauth_helpers.py with unit tests for _validate_consent_link
  and try_parse_oauth_consent_event covering edge cases:
  - HTTPS URL with empty netloc (https:///path)
  - Warning log messages for rejected consent links
  - Event objects missing 'type' attribute

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5054: Python: [Bug]: `FoundryAgent` (Responses API) Does Not Surface `oauth_consent_request` as a CUSTOM AG-UI Event

* Fix mypy: match _parse_chunk_from_openai signature with superclass

Add seen_reasoning_delta_item_ids parameter to _parse_chunk_from_openai
overrides in both RawFoundryChatClient and RawFoundryAgentChatClient to
match the updated superclass signature on main. Update super() calls and
test assertions accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
 2026-04-24 09:59:14 +00:00
Eduard van Valkenburg 62e02da698 Python: update FoundryAgent for hosted agent sessions (#5447) 
* fixes to FoundryAgent to connect to new hosted agents

Co-authored-by: Copilot <copilot@github.com>

* fix mypy

Co-authored-by: Copilot <copilot@github.com>

* Python: remove Foundry service session helpers

Remove the public hosted-agent service session CRUD helpers from FoundryAgent and drop the related feature-stage inventory entry.

Update the hosted-agent sample to create and delete service sessions directly through the preview AIProjectClient APIs, and tighten a few test harnesses surfaced by full workspace validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix from merge

* fix hosted env detection

Co-authored-by: Copilot <copilot@github.com>

* reverted sample update

* fix tests and code

Co-authored-by: Copilot <copilot@github.com>

* remove aenter

* skipping some tests

Co-authored-by: Copilot <copilot@github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-24 09:25:03 +00:00
Tao Chen 0989e68d1c Python: Fix user agent prefix (#5455) 
* Fix hosting user agent missing

* Fix other providers

* Add more tests

* comments

* Fix tests
 2026-04-23 23:40:38 +00:00
Evan Mattson b084d0461d Python: (foundry): stop emitting [TOOLBOXES] warning for every FoundryChatClient call (#5440) 
* Python: Foundry: make response tool sanitizer internal, drop TOOLBOXES warning

sanitize_foundry_response_tool runs on every tool passed to the Foundry
Responses API, so its @experimental(TOOLBOXES) decorator was emitting a
[TOOLBOXES] ExperimentalWarning for any FoundryChatClient call, even when
no toolbox was involved. The function isn't in __all__ and has no external
callers. Rename to _sanitize_foundry_response_tool and drop the decorator;
the actual toolbox-facing public helpers remain gated.

* Python: Foundry: silence pyright on intentional cross-module private import
 2026-04-23 22:19:37 +00:00
Evan Mattson acec9caa2f Python: Bump Python package versions for a release. (#5432) 
* Bump Python version for a release.

* Revert lockstep bumps on unchanged connectors

Per PR review: only connectors that changed (or whose published metadata
changed) should get new versions. Keeps released tier at 1.1.1, a2a/ag-ui
at 1.0.0b260422, foundry-hosting at 1.0.0a260422; reverts the 19 unchanged
betas and 2 unchanged alphas to 1.0.0b260421/1.0.0a260421. Reverts all 26
non-core agent-framework-core floors to >=1.1.0,<2 since no connector
actually depends on a 1.1.1 API or bug fix.

* Restore lockstep prerelease bumps and raise core floors to >=1.1.1

Reverses the lean-revert: all beta packages stamped 1.0.0b260423 and alpha
packages stamped 1.0.0a260423 (Asia date, matching release cut time). All
26 non-core packages raise agent-framework-core lower bound from >=1.1.0,<2
to >=1.1.1,<2 to signal the validated cohort for this release. CHANGELOG
date updated to 2026-04-23.
 2026-04-23 16:40:14 +09:00
Evan Mattson fffd0acb3e Python: fix(foundry): reconcile toolbox hosted-tool payloads with Responses API (#5414) 
* fix(foundry): reconcile toolbox hosted-tool payloads with Responses API

* docs(foundry): update create_sample_toolbox docstring to reflect all tools created
 2026-04-22 17:43:26 +00:00
chetantoshniwal aa582d021d Python: feat(evals): add ground_truth support for similarity evaluator (#5234) 
* feat(evals): add ground_truth support for similarity evaluator

- Include expected_output as ground_truth in Foundry JSONL dataset rows
- Add ground_truth to item schema and data mapping for similarity evaluator
- Add expected_output parameter to evaluate_workflow
- Add similarity Pattern 3 to evaluate_agent and evaluate_workflow samples
- Add tests for ground_truth in dataset, schema, and evaluate_workflow

* Apply suggestions from code review

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* fix: wrap long line to satisfy ruff E501

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
 2026-04-21 19:40:53 +00:00
Evan Mattson 2c8036779c Python: Bump versions for a release. Update CHANGELOG (#5385) 
* Bump versions for a release. Update CHANGELOG

* Bump devui
 2026-04-21 15:14:42 +09:00
Evan Mattson 04aaf0c1fe Python: Add support for Foundry Toolboxes (#5346) 
* Add support for the Foundry Toolbox in MAF

Introduces a Foundry Toolbox integration: FoundryChatClient gains a
get_toolbox() helper plus select_toolbox_tools(), normalize_tools in
the core package flattens tool-collection wrappers (ToolboxVersionObject
and generic iterables, while leaving Pydantic BaseModel instances
alone), and the new agent_framework.foundry namespace re-exports the
toolbox helpers. Ships with unit tests, a sample, and a design doc.

azure-ai-projects is pinned to the public >=2.0.0,<3.0 range and the
lockfile resolves from public PyPI. The toolbox test module skips when
Toolbox* types are unavailable so CI stays green until the public 2.1.0
SDK lands. OMC tooling directories (.omc/, .omx/) are gitignored.

* Update to latest azure ai projects package

* Improve sample

* Rename ADR to 0025

* Update ADR

* Apply suggestion from @alliscode

Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>

* Improve samples

* Update test

---------

Co-authored-by: Ben Thomas <ben.thomas@microsoft.com>
 2026-04-20 23:56:01 +00:00
Evan Mattson 4a36f10888 Python: Bump Python version to 1.0.1 for a release (#5196) 
* Bump Python version to 1.1.0 for a release

* Fix changelog

* 1.0.1 instead of 1.1.0

* Update CHANGELOG.md

* update version and changelog

* Bump lower bounds
 2026-04-10 12:23:21 +09:00
Matt Van Horn 18d1ba3624 Python: Strip tools from FoundryAgent request when agent_reference is present (#5101) 
_prepare_options() now removes tools, tool_choice, and parallel_tool_calls
from run_options after injecting agent_reference. The Foundry API rejects
requests containing both fields. FunctionTools are still invoked client-side
by the function invocation layer.

Fixes #5087

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
 2026-04-08 04:35:01 +00:00
Evan Mattson 36cafe4e5a Python: Raise clear handler registration error for unresolved TypeVar annotations (#4944) 
* Raise clear handler registration error for unresolved TypeVar (#4943)

Detect unresolved TypeVar in message parameter annotations during handler
registration in both _validate_handler_signature (Executor) and
_validate_function_signature (FunctionExecutor). Raises a ValueError with
an actionable message recommending @handler(input=..., output=...) or
@executor(input=..., output=...) instead of letting TypeVar leak through
to a confusing TypeCompatibilityError during workflow edge validation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #4943: reorder checks and harden function executor

- Move TypeVar check before validate_workflow_context_annotation in
  _executor.py so users see the more actionable error first
- Wrap get_type_hints in try/except in _function_executor.py matching
  the defensive pattern in _executor.py
- Repurpose duplicate test to cover bounded TypeVar rejection
- Add test_function_executor_allows_concrete_types for test symmetry

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Narrow get_type_hints except clause and add missing tests (#4943)

- Narrow `except Exception` to `except (NameError, AttributeError, RecursionError)`
  in both _executor.py and _function_executor.py so unexpected failures in
  get_type_hints are not silently swallowed.
- Add test_handler_unresolvable_annotation_raises to test_function_executor_future.py
  exercising the except branch of get_type_hints in the function executor path.
- Add test_function_executor_rejects_bounded_typevar_in_message_annotation to
  test_function_executor.py for parity with the Executor bounded TypeVar test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add error ordering test for TypeVar vs WorkflowContext priority (#4943)

Add test_handler_typevar_error_takes_priority_over_context_error to verify
that when a handler has both a TypeVar message and an unannotated ctx, the
TypeVar error is raised first (the more actionable issue).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix image content serialization sending null file_id to Foundry API

Omit file_id from input_image dict when not present instead of including
it as null, which Azure AI Foundry's stricter schema validation rejects.

* Python: Fix Foundry API rejecting rich content in function_call_output

Azure AI Foundry does not support list-format output in function_call_output
items. Add SUPPORTS_RICH_FUNCTION_OUTPUT flag (default True) to
RawOpenAIChatClient, set to False in RawFoundryChatClient so Foundry
falls back to string output for tool results with images/files.

Also omit file_id from input_image dicts when not set, since Foundry
rejects explicit nulls.

* Python: Surface rich tool content as user message when Foundry lacks support

When SUPPORTS_RICH_FUNCTION_OUTPUT is False, image/file items from tool
results are injected as a follow-up user message so the model can still
process the visual content via Foundry's supported user message format.

* Xfail Foundry image integration test for the meantime

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-04-07 16:59:17 +00:00
Eduard van Valkenburg 3446eb8d5d Python: [BREAKING] update to v1.0.0 (#5062) 
* updates to final deprecated pieces and versions

* fix mypy

* fix readme links
 2026-04-02 15:26:30 +00:00
Eduard van Valkenburg 95fd5ec658 Python: [BREAKING] Python: move Azure AI embeddings to Foundry (#5056) 
* renamed AzureAIINferenceEmbeddings and lazy load azure-cosmos and env var rename

* updated coverage

* fix readme
 2026-04-02 11:26:35 +00:00
Copilot 7607acd009 Python: Add experimental decorator to all Evals pieces (#5040) 
* Initial plan

* feat(python): add experimental decorator to all Evals pieces

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/99d71249-a5d6-4977-a5b5-6ffe0a3be2bc

Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: TaoChenOSU <12570346+TaoChenOSU@users.noreply.github.com>
 2026-04-02 09:18:35 +00:00