Compare commits

...

53 Commits

Author SHA1 Message Date
Tao Chen 9e4e4a8fab Fix typing 2026-06-22 08:42:23 -07:00
Tao Chen 7b5ef68abc Merge branch 'main' into local-branch-5559 2026-06-22 08:25:44 -07:00
Eduard van Valkenburg a7381d8bef Python: stabilize dependency maintenance final checks (#6662)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 14:23:29 +00:00
westey d108d4b549 Python: [BREAKING] Integrate looping into HarnessAgent (#6607)
* Integrate looping into harness

* Address PR comments

* Address PR comments.

* Fix typing error
2026-06-22 13:14:30 +00:00
Eduard van Valkenburg fd160a7782 Python: fix dependency maintenance cutoff (#6658)
* Python: fix dependency maintenance cutoff

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix Hyperlight output dir typing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 11:46:26 +00:00
Copilot ad3c1535c4 fix: propagate EnableSensitiveData to auto-wired inner OpenTelemetryChatClient (#6096)
When _autoWireChatClient=true and the caller sets EnableSensitiveData=true on
the outer OpenTelemetryAgent, the auto-wired inner OpenTelemetryChatClient now
also has EnableSensitiveData propagated, so the inner chat span correctly
captures message content (gen_ai.input.messages / gen_ai.output.messages).

Red test added first to reproduce the bug, then the fix applied (green).

Fixes #5873

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/fda69dd4-9576-4f3f-b954-514321652ea9

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: rogerbarreto <19890735+rogerbarreto@users.noreply.github.com>
2026-06-22 10:27:13 +00:00
安妮的心动录 10b7d08bff .NET: fix(hosting): emit url_citation annotation events from streamed AI Search responses (#6649)
* fix(hosting): emit url_citation annotation events from streamed AI Search responses

OutputConverter.ConvertUpdatesToEventsAsync accumulated text content deltas but
silently dropped CitationAnnotation metadata from TextContent.Annotations. As a
result, hosted agents that use CreateAzureAISearchTool emitted citation markers in
text (e.g. 【5:0†source】) but produced empty annotations arrays and no
response.output_text.annotation.added SSE events.

The fix accumulates UrlCitationBody SDK annotations across all TextContent updates
for a message and emits them via TextContentBuilder.EmitAnnotationAdded after
EmitTextDone (as required by the SDK lifecycle) and before EmitDone. Non-citation
and region-less annotations are silently skipped, matching the existing OpenAI
ChatCompletions path in AgentResponseExtensions.

Adds 7 unit tests (N-01–N-07) covering: basic emission, ordering constraints,
multiple annotations, multi-update accumulation, and skip conditions.

Fixes #6641

* test: convert annotation test comments to XmlDoc and group in region

* fix: remove redundant long casts on annotation region indices

* test: assert done events carry url_citation annotation metadata

---------

Co-authored-by: Roger Barreto <19890735+RogerBarreto@users.noreply.github.com>
2026-06-22 10:10:02 +00:00
Eduard van Valkenburg fc3111c391 Python: Add FoundryAgent conversation session helper (#6623)
* Add FoundryAgent conversation session helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Simplify Foundry conversation session helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Rename Foundry conversation helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* use named kw

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-22 08:41:15 +00:00
Roger Barreto 098e521586 .NET: Bring Hosted-Toolbox sample to parity with sibling hosting samples (#6633)
* Bring Hosted-Toolbox sample to parity with sibling hosting samples

Adds the standard scaffolding files (.env.example, agent.yaml, agent.manifest.yaml,
Dockerfile, Dockerfile.contributor) that every other 04-hosting Foundry sample ships
but Hosted-Toolbox lacked.

Fixes the toolbox name environment variable: reads TOOLBOX_NAME instead of the
platform reserved FOUNDRY_TOOLBOX_NAME so it survives agent create, and aligns the
default to my-toolset.

Rewrites the README to the standard section layout with PowerShell fenced commands,
and adds Using-Samples READMEs documenting why the client REPLs exist.

Renames Azure AI Foundry to Foundry across the 04-hosting sample READMEs and comments
for consistent product naming.

* Address PR review: accurate docs and TOOLBOX_NAME in ToolboxMcpSkills

- SimpleAgent README: correct the demo banner to the real per-agent URL the
  client prints (https scheme and the /api/projects/<project> segment).
- Hosted-Toolbox Program.cs: move FOUNDRY_MODEL out of the Required block into
  Optional since it has a gpt-4o default and an AZURE_AI_MODEL_DEPLOYMENT_NAME
  fallback.
- Hosted-ToolboxMcpSkills: switch the toolbox name from the reserved
  FOUNDRY_TOOLBOX_NAME to TOOLBOX_NAME across Program.cs, .env.example,
  agent.yaml, agent.manifest.yaml and README so it is deployable via the
  manifest, matching the other toolbox samples.
2026-06-20 09:31:19 +00:00
Eduard van Valkenburg 7435dd48d0 Python: harden Hyperlight output capture against symlinks (#6601)
* Python: harden Hyperlight output capture against symlinks

Mirror the input-staging symlink hardening on the output-capture path of
HyperlightExecuteCodeTool. Output discovery now walks via the symlink-safe
_iter_real_entries instead of rglob, per-file collection validates that no
path component is a symlink and the final entry is a regular file, and file
reads use os.O_NOFOLLOW. Adds regression tests for the output path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: reject traversal, fix listing test, harden read

- _is_safe_output_file now rejects '.'/'..' components (lexical relative_to
  could otherwise escape root without a symlink)
- _read_output_file_bytes adds a cross-platform TOCTOU guard (lstat/fstat
  st_dev+st_ino identity check) since O_NOFOLLOW is absent on Windows
- fix intermediate-dir-symlink test to use a relative listing path so it
  exercises normalization + validation; add a parent-traversal unit test

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 22:43:38 +00:00
Ahmed Muhsin 148f57020a Python: host MAF workflows on a standalone Durable Task worker (#6418)
* feat(durabletask): host MAF workflows on a standalone Durable Task worker

Add a host-agnostic workflow execution engine to agent-framework-durabletask so a MAF Workflow can run as a durable orchestration outside Azure Functions:

- WorkflowOrchestrationContext protocol + DurableTaskWorkflowContext adapter, the superstep orchestrator, serialization helpers, capturing runner context, and the shared non-agent activity body (including the yield-output classifier so intermediate executors are not surfaced as final outputs).

- DurableAIAgentWorker.configure_workflow auto-registers agent executors as entities, non-agent executors as activities, and the workflow orchestrator.

- plan_workflow_registration centralizes the 'what to register' decision so it can be shared across hosts.

- run_agent_coroutine runs all agent coroutines on one persistent event loop, fixing a cross-loop hang when shared chat clients/credentials bind their asyncio primitives to a dead loop.

- DurableWorkflowClient (start/await workflow + HITL discover/respond); DurableAIAgentClient stays agent-only.

* refactor(azurefunctions): delegate workflow execution to agent-framework-durabletask

AgentFunctionApp now reuses the shared orchestrator, activity body, and registration planner from agent_framework_durabletask instead of maintaining its own copies; _workflow.py becomes a thin host-specific adapter (AzureFunctionsWorkflowContext).

- Run agent entity coroutines on the shared persistent event loop, fixing the cross-loop hang.

- Relocate state-diff unit tests to the durabletask package; update entity loop tests.

* feat(core): expose durabletask workflow symbols via agent_framework.azure

Lazily re-export WORKFLOW_ORCHESTRATOR_NAME and DurableWorkflowClient from the agent_framework.azure namespace so standalone hosts can import them without depending on internal module paths.

* docs(samples): add standalone durabletask workflow and HITL samples

Add two samples under samples/04-hosting/durabletask demonstrating MAF workflows on a standalone Durable Task worker (no Azure Functions):

- 08_workflow: conditional spam-detection workflow started via DurableWorkflowClient.start_workflow / await_workflow_output.

- 09_workflow_hitl: content-moderation workflow that pauses with ctx.request_info and is resumed via DurableWorkflowClient.get_pending_hitl_requests / send_hitl_response.

Also add the durabletask workflow integration test (test_08_dt_workflow).

* fix: address PR review feedback

- Sanitize HITL external-event responses with strip_pickle_markers in the orchestrator (defense-in-depth for callers that bypass DurableWorkflowClient).

- Raise WorkflowConvergenceException when max_iterations is reached with pending messages, matching the core WorkflowRunner instead of silently returning partial output.

- Route falsy 'sent' messages (use 'is not None' instead of truthiness).

- Normalize None shared_state_snapshot/source_executor_ids in execute_workflow_activity.

- Cast Any returns in AzureFunctionsWorkflowContext to satisfy mypy/pyright.

- Fix sample docstrings to reference DurableWorkflowClient.

* fix: resolve pyright Package Checks errors

- Use typed locals instead of cast in AzureFunctionsWorkflowContext (mypy sees Any, pyright sees concrete types -> avoid reportUnnecessaryCast).

- Annotate shared_state_snapshot and cast partially-typed durabletask SDK returns / HITL custom-status parsing to satisfy reportUnknownVariableType/reportUnknownMemberType.

- Drop the dead deserialize/serialize re-export in _workflow.py and mark the intentional private _extract_message_content re-export.

* fix(durabletask): agent-executor identity and typed workflow input

Register each workflow agent entity under the executor id that the orchestrator dispatches to (instead of the agent name), so AgentExecutor(agent, id=...) works when the id differs from agent.name. The azure-functions host mirrors this.

Reconstruct the start executor declared input type from the workflow initial JSON payload in the shared engine (mirroring in-process delivery) instead of string-coercing it per host. Untrusted input is stripped of pickle markers before reconstruction to prevent deserialization RCE.

* fix(samples): type durable workflow start executors for reconstructed input

The HITL and parallel workflow samples no longer hand-parse a JSON string. Their start executors now declare their real input type (ContentSubmission / DocumentInput), which the durable engine reconstructs from the client payload before delivery.

* test(durabletask): unit coverage for registration, client, worker, and input coercion

Add unit tests for plan_workflow_registration, DurableWorkflowClient, the agent-executor identity registration (entity keyed by executor id), and the typed initial-input coercion including pickle-marker neutralization.

* test(durabletask): HITL and parallel durable workflow integration tests

Add an integration test for the standalone durabletask HITL workflow sample via a new workflow_client fixture. Re-enable the Azure Functions parallel workflow test, consolidated into one end-to-end case so the work-stealing xdist scheduler cannot spawn multiple func hosts for this sample.

* refactor(durabletask): group workflow modules into a _workflows subpackage

Move the eight workflow modules into a private _workflows/ subpackage and drop the redundant _workflow_ prefix (orchestrator.py, registration.py, activity.py, client.py, context.py, dt_context.py, runner_context.py, serialization.py). The public API and __all__ are unchanged; only direct internal-module imports were repointed (package __init__, the worker, the azure-functions shared shim, and the affected unit tests).

* fix(durabletask): harden workflow type resolution and HITL response handling

- resolve_type returns only real classes (avoids issubclass TypeError in reconstruct_to_type)

- re-wait on HITL responses rejected by pickle-marker sanitization instead of dropping the request and losing the run

- American spelling in strip_pickle_markers docstring

- unit tests for resolve_type

* fix(durabletask): treat async edge conditions as not-matched on the synchronous host

The durabletask orchestrator evaluates edge conditions synchronously and does not support async edge conditions. Such an edge is now treated as not matched (the edge is not traversed) rather than assuming a result. Adds unit coverage; full async-condition support will be handled separately.

* fix(durabletask): reconstruct typed workflow outputs at the host boundary

await_workflow_output and the Azure Functions status endpoint now decode the checkpoint-encoded outputs the shared activity produces, via a shared deserialize_workflow_output helper. The client returns the original objects; the AF endpoint emits clean domain JSON instead of checkpoint-marker dicts, keeping the two hosts consistent.

* fix(durabletask): address review findings on workflow hosting

- AF: register workflow agents through add_agent(entity_id=...) so they remain tracked in app.agents / get_agent() (restores documented behavior) while keying by the executor id the orchestrator dispatches to; mirrors DurableAIAgentWorker.add_agent.

- async bridge: treat the shared loop as reusable only while its backing thread is alive, so a dead loop thread is replaced instead of hanging future.result() forever.

- client: add get_runtime_status; the standalone HITL sample now stops polling and reports the real terminal state instead of a generic timeout.

- tests: guard send_hitl_response pickle-marker stripping and add get_runtime_status coverage.

* fix(durabletask): wait indefinitely for HITL responses, matching core

The durable workflow host previously raced HITL responses against a 72h timer and failed the orchestration on elapse. MAF core's request_info has no timeout concept (it waits for the response), and the .NET durable host waits too, so the durable Python host now does the same: it stays paused until a response arrives. Removes the hitl_timeout_hours parameter and DEFAULT_HITL_TIMEOUT_HOURS constant from both hosts. A configurable timeout can be added later once core defines the contract (what happens on elapse).

* feat(durabletask): typed workflow event streaming and async client API

Add a brokerless workflow event stream to the durable host. Each non-agent executor runs inside a durable activity that captures its real WorkflowEvents (with data payloads); the orchestrator replays them into the orchestration custom status after each superstep, and the client streams them back as typed WorkflowEvent objects with reconstructed data. Agent executors contribute synthesized invoked/completed lifecycle events.

Add async client methods run_workflow (start with optional wait) and stream_workflow (typed event iterator), plus is_replaying plumbing through the orchestration context protocol and both host adapters so live status is published only on non-replay execution.

* docs(samples): standalone durabletask workflow streaming sample

Add sample 10_workflow_streaming demonstrating the async DurableWorkflowClient API on a standalone Durable Task worker: run_workflow(wait=False) to start without blocking, then stream_workflow to consume typed WorkflowEvent objects as a WriterAgent -> ReviewerAgent -> publish pipeline runs.

* refactor(durabletask): internal-only checkpoint codec and host-scoped workflow event streaming

Two related hardening changes to the durable workflow hosting layer, plus a
rebase-restored improvement.

Internal-only serialization codec (MSRC follow-up):
- Rename serialize_value/deserialize_value -> _serialize_value/_deserialize_value
  in the shared durabletask serialization module and update all call sites, so the
  pickle-backed checkpoint codec is unambiguously framework-internal. Untrusted
  input is still neutralized with strip_pickle_markers at the HTTP boundary.
- Remove the duplicate agent_framework_azurefunctions._serialization module and
  import strip_pickle_markers from the shared durabletask module instead. Move its
  unique serialization/strip-marker tests into the durabletask test suite.

Scope workflow event streaming to hosts that can carry it:
- Add WorkflowOrchestrationContext.supports_event_streaming. The standalone
  DurableTask host returns True (no custom-status size cap, has a stream_workflow
  consumer); the Azure Functions host returns False.
- The orchestrator now accumulates and publishes the WorkflowEvent timeline to the
  orchestration custom status only when the host supports streaming. On Azure
  Functions the custom status returns to its pre-streaming shape
  ({state[, pending_requests]}), which fixes orchestrator failures with
  "The size of the JSON-serialized payload must not exceed 16 KB" and stops leaking
  pickle markers into the HTTP status response. The Azure Functions status endpoint
  never consumed the event stream.

Workflow start endpoint:
- Accept text/plain raw request bodies (fall back from get_json to the raw body),
  restoring an improvement from main that the rebase conflict resolution dropped.

* fix(azurefunctions): scope workflow status/respond endpoints to the workflow orchestrator

The workflow/status/{instanceId} and workflow/respond/{instanceId}/{requestId}
HTTP endpoints resolved durable instances by ID only. The durable client looks up
IDs across every orchestration in the task hub (agent entities, any
user-registered orchestrations, and other apps sharing the hub), so a caller
holding one instance ID could read another orchestration's status -- including
pending HITL request payloads -- or inject external events into it.

Add AgentFunctionApp._is_workflow_orchestration() and gate both endpoints on it:
an instance whose orchestration name is not WORKFLOW_ORCHESTRATOR_NAME now returns
404 instead of leaking state or accepting events. send_hitl_response now fetches
the orchestration status and validates ownership before raising the external
event. Legitimate workflow instances are unaffected.

Mirrors the .NET fix in PR #6608.

* fix(durabletask): resolve CI typing failures

- serialization: rename _serialize_value/_deserialize_value back to
  serialize_value/deserialize_value to follow the package convention for
  cross-module internal helpers (matches strip_pickle_markers, resolve_type).
  The leading underscore tripped pyright reportPrivateUsage on cross-module
  imports under the strict source gate; internal-only status is preserved by
  not exporting them from the public API.
- Remove type-ignore comments pyright flags as unnecessary
  (reportUnnecessaryTypeIgnoreComment) in _worker.py, orchestrator.py,
  serialization.py.
- test_08_dt_workflow: add AgentClientFactoryProtocol and annotate the
  agent_client_factory fixture as type[AgentClientFactoryProtocol] (matching
  test_01-07) so mypy/ty stop reporting "type has no attribute create".
- samples (08_workflow, 09_workflow_hitl): pass structured output via
  FoundryChatOptions[Any](response_format=...) instead of a plain dict so the
  samples pyright (basic) config accepts default_options.

---------

Co-authored-by: Gavin Aguiar <80794152+gavin-aguiar@users.noreply.github.com>
2026-06-19 22:21:08 +00:00
Ben Thomas 89d19a2370 .NET: Migrate 01-get-started samples to Foundry as canonical default (#6555)
* Migrate 01-get-started samples to Foundry as canonical default

Change canonical provider from Azure OpenAI to Microsoft Foundry Responses API:

Code changes:
- Updated all 01-get-started samples (01_hello_agent, 02_add_tools, 03_multi_turn,
  04_memory, 06_host_your_agent) to use FoundryAgent or AIProjectClient.AsAIAgent()
- Updated environment variables: AZURE_OPENAI_* → FOUNDRY_PROJECT_ENDPOINT/FOUNDRY_MODEL
- Updated .csproj files to reference Microsoft.Agents.AI.Foundry instead of Azure.AI.OpenAI
- Added warning comments about DefaultAzureCredential production usage
- 05_first_workflow unchanged (workflow pattern only, no AI model)

Documentation changes:
- Updated AGENTS.md Default provider section to reflect Foundry as canonical
- Updated code example to use FoundryAgent constructor pattern
- Updated env var documentation

Note: 04_memory (AIContextProvider sample) extracts IChatClient from FoundryAgent
to maintain the memory pattern while using Foundry backend.

All samples verified to build successfully.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR 6555 review feedback and format failures

- Add Microsoft.Agents.AI.Foundry using to AGENTS.md Foundry snippet
- Update verify-samples GetStarted env vars to FOUNDRY_PROJECT_ENDPOINT/FOUNDRY_MODEL
- Remove unnecessary usings flagged by dotnet format in 01_get_started samples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Switch 01-get-started samples from FoundryAgent to AIProjectClient.AsAIAgent()

Use AIProjectClient.AsAIAgent() as the canonical pattern for all 01-get-started
samples. Reserve FoundryAgent only for samples that specifically demonstrate the
Foundry-managed (prompt) agent — i.e. 02-agents/AgentsWithFoundry/.

Changes:
- 01_hello_agent, 02_add_tools, 03_multi_turn, 06_host_your_agent: swap
  FoundryAgent constructor for AIProjectClient.AsAIAgent(model, instructions)
- 04_memory: get IChatClient via AIProjectClient.AsAIAgent(options).GetService()
  instead of extracting from a throwaway FoundryAgent
- AGENTS.md: update default-provider snippet and note on when to use FoundryAgent

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 15:17:12 +00:00
Peter Ibekwe c815902344 .NET: InProcessRunnerContext bugfix for workflows (#6551)
* workfllow bugfix

* Update exception message

* Fix unit test.
2026-06-19 14:09:36 +00:00
Peter Ibekwe 074ac68a6c .NET: Harden fan-in barrier checkpoint state and extend resume coverage (#6574)
* Harden fan-in barrier checkpoint state and extend resume coverage

* Address PR comment
2026-06-19 13:33:26 +00:00
Eduard van Valkenburg d049d94b49 Python: consolidate dependency maintenance workflow (#6570)
* Python: consolidate dependency maintenance workflow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: delay dependency maintenance updates

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: track dependency bounds test failures

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: scope dependency maintenance token

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 09:41:35 +00:00
dependabot[bot] 41995e265d Build(deps): Bump anthropic from 0.80.0 to 0.107.1 in /python (#6396)
Bumps [anthropic](https://github.com/anthropics/anthropic-sdk-python) from 0.80.0 to 0.107.1.
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v0.80.0...v0.107.1)

---
updated-dependencies:
- dependency-name: anthropic
  dependency-version: 0.107.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-19 08:53:17 +00:00
dependabot[bot] 2adacb3034 Bump aiohttp from 3.13.4 to 3.14.1 in /python (#6395)
---
updated-dependencies:
- dependency-name: aiohttp
  dependency-version: 3.14.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-19 08:53:08 +00:00
dependabot[bot] 6e3836698b Bump Anthropic.Foundry from 0.5.0 to 0.6.0 (#6057)
---
updated-dependencies:
- dependency-name: Anthropic.Foundry
  dependency-version: 0.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-19 08:52:48 +00:00
dependabot[bot] 0d3e3504f0 Build(deps): Bump mistralai from 2.4.2 to 2.4.9 in /python (#6393)
Bumps [mistralai](https://github.com/mistralai/client-python) from 2.4.2 to 2.4.9.
- [Release notes](https://github.com/mistralai/client-python/releases)
- [Changelog](https://github.com/mistralai/client-python/blob/main/RELEASES.md)
- [Commits](https://github.com/mistralai/client-python/compare/v2.4.2...v2.4.9)

---
updated-dependencies:
- dependency-name: mistralai
  dependency-version: 2.4.9
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-19 08:45:29 +00:00
dependabot[bot] 2ba97ebcca Bump openai from 2.24.0 to 2.43.0 in /python (#6394)
Bumps [openai](https://github.com/openai/openai-python) from 2.24.0 to 2.43.0.
- [Release notes](https://github.com/openai/openai-python/releases)
- [Changelog](https://github.com/openai/openai-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/openai/openai-python/compare/v2.24.0...v2.43.0)

---
updated-dependencies:
- dependency-name: openai
  dependency-version: 2.41.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-19 08:45:17 +00:00
hanhan761 2d0555c537 Python: re-role trailing assistant message to user for Anthropic compatibility (fixes #5008) (#6207)
* Fix auto function calling stripping explicit null arguments (fixes #5934)

* fix: re-role trailing assistant message to user for Anthropic (fixes #5008)

* fix: address Copilot review feedback (exclude_unset, test coverage, synthetic user turn)

* fix: update docstring and extend exclude_unset to auto_invoke_function

* revert: remove unrelated core _tools.py changes from Anthropic PR

The exclude_none/exclude_unset changes in the core package are out of scope
for this Anthropic-specific fix. This PR now only contains the Anthropic
chat client docstring fix and the synthetic user turn append.

* fix: avoid appending user turn after Anthropic tool use

* Fix Anthropic tool-use type narrowing

Use object-typed content narrowing before checking Anthropic tool-use block types so strict Pyright no longer treats dynamic message content as Unknown.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Evan Mattson <evan.mattson@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 06:25:34 +00:00
Evan Mattson 5145d50be8 Python: Fix AG-UI tool history replay sanitization  (#6581)
* Python: Fix AG-UI tool history replay sanitization 

* Python: Address AG-UI replay review comments

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 15:01:01 +09:00
dependabot[bot] 7f7c88bfa5 Build(deps): Bump python-multipart from 0.0.26 to 0.0.32 in /python (#6406)
* Build(deps): Bump python-multipart from 0.0.26 to 0.0.27 in /python

Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.26 to 0.0.27.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/main/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.26...0.0.27)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.27
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

* Build(deps): Bump python-multipart to 0.0.32 via override-dependencies floor >=0.0.31

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-19 01:00:26 +00:00
Shyju Krishnankutty bcef77af6a .NET: (Durable): Scope workflow status/respond endpoints to route workflow name (#6608)
* Scope workflow status/respond endpoints to route workflow.

 Validate that the orchestration instance belongs to the workflow
 named in the route. Prevents cross-workflow access via runId.

* Add changelog.

* Address Copilot review feedback: fix duplicate XML doc, make IsOrchestrationOwnedByWorkflow non-throwing, drop misleading Async suffix in test name

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-19 00:01:15 +00:00
Ben Thomas 54a30571aa Dotnet - Add support for Foundry Adaptive evals (#6267)
* .NET: feat(evals): RubricScore type + EvalScoreResult.Dimensions

Adds the core rubric-evaluator surface that mirrors the Python work in

PR #6101 (commit e45b934cc). Provider-agnostic types only — no Foundry

coupling. Subsequent commits will wire these into FoundryEvals.

- RubricScore: per-dimension score record (Id, Score?, Applicable, Weight, Reason).

- EvalScoreResult.Dimensions: optional init-only list of RubricScore.

  Null for non-rubric (built-in) evaluators.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(evals): GeneratedEvaluatorRef + assertion helpers

Adds the provider-agnostic surface for referencing a pre-existing rubric

evaluator and gating CI on per-item / per-dimension thresholds. Mirrors

Python PR #6101 commits e5830dd7f (ref type) and 4bc60462d (asserts).

- GeneratedEvaluatorRef: name + optional version/display-name, plus a

  Latest(name) factory for versionless refs (discouraged for CI; consumers

  should warn at run time).

- AgentEvaluationResults.AssertScoreAtLeast: walks DetailedItems[].Scores,

  optionally filtered by evaluator name, recurses into SubResults.

- AgentEvaluationResults.AssertDimensionScoreAtLeast: walks each score's

  Dimensions list, skips non-applicable dimensions by default, supports

  requireApplicable to flip that, recurses into SubResults.

- AgentEvaluationResults.AssertNoFailedItems: walks DetailedItems for

  fail/error statuses, recurses into SubResults.

All helpers throw InvalidOperationException (matches existing AssertAllPassed).

Truncates offender lists to the first 5 with a '+N more' suffix to keep

CI output readable, mirroring the Python helpers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(foundry-evals): accept GeneratedEvaluatorRef in evaluators=

Adds FoundryEvaluatorSpec, a readonly-struct union with implicit conversions
from both string and GeneratedEvaluatorRef so call sites can mix built-in
evaluator names with rubric evaluator references:

    var evals = new FoundryEvals(
        projectClient, model,
        new GeneratedEvaluatorRef("policy-rubric", "3"),
        FoundryEvals.Relevance,
        FoundryEvals.Coherence);

FoundryEvals constructors (3 overloads), EvaluateTracesAsync, and
EvaluateFoundryTargetAsync now take FoundryEvaluatorSpec[]/params instead of
string[]/params. Existing call sites using string literals or string[] keep
working unchanged via implicit conversion.

FoundryEvalConverter.BuildTestingCriteria emits the documented Foundry wire
format for rubric refs:
  {
    "type": "azure_ai_evaluator",
    "name": <DisplayName ?? Name>,
    "evaluator_name": <Name>,
    "evaluator_version": <Version>,   // omitted when null
    "initialization_parameters": { "deployment_name": <model> },
    "data_mapping": { conversation arrays, optional tool_definitions }
  }

WireTestingCriterion gains an optional EvaluatorVersion field. Rubric refs
are preserved through FilterToolEvaluators (tool-aware but not tool-required)
and ignored by FindMissingGroundTruthEvaluators. A versionless ref emits a
Trace.TraceWarning at criterion-build time so CI authors notice the floating
version (mirrors the Python warning).

Adds 6 new Foundry unit tests (3 BuildTestingCriteria rubric paths, 1
FindMissingGroundTruthEvaluators, 1 FilterToolEvaluators preservation, 1
mixed-order). 369/369 Foundry tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(foundry-evals): parse rubric dimension_scores into RubricScore

Adds FoundryEvals.ParseRubricScores, called per result inside ParseDetailedItem.
Each EvalScoreResult now populates Dimensions when the evaluator's sample carries
a rubric breakdown.

Accepts three shapes for forward compatibility with provider SDK iterations:

  1. sample.properties.dimension_scores  (canonical Foundry runtime shape)
  2. sample.properties.rubric_scores     (preview/legacy key)
  3. top-level sample.dimension_scores / sample.rubric_scores  (defensive fallback)

Entries missing 'id', 'weight', or 'applicable' are skipped without invalidating
well-formed siblings. Non-applicable dimensions may omit 'score' (parsed as null).

Adds 6 unit tests covering canonical and legacy keys, top-level fallback, no-match
returns null, malformed-entry skipping, and the non-applicable null-score path.
375/375 Foundry tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: feat(samples): Evaluation_FoundryRubric end-to-end sample

Adds dotnet/samples/05-end-to-end/Evaluation/Evaluation_FoundryRubric mirroring
the Python evaluate_with_rubric_sample.py:

  - Fetches a pre-existing Foundry agent via AgentAdministrationClient
    (GetAgentAsync for latest, GetAgentVersionAsync when FOUNDRY_AGENT_VERSION
    is pinned).
  - References a rubric evaluator by GeneratedEvaluatorRef(name, version);
    falls back to GeneratedEvaluatorRef.Latest(name) with the documented
    floating-version warning.
  - Mixes the rubric with FoundryEvals.Relevance and FoundryEvals.Coherence
    in a single FoundryEvals run (implicit string-and-ref conversion).
  - Prints per-dimension breakdowns from EvalScoreResult.Dimensions for each
    item.
  - Demonstrates a CI quality gate with AssertDimensionScoreAtLeast("general_quality", 3.0).

Documents the FOUNDRY_PROJECT_ENDPOINT footgun (must be project-scoped URL
.../api/projects/<project>, not the bare Azure OpenAI endpoint) and the
Eval-Definition-vs-Rubric-Evaluator distinction in the README. Ships a
.env.example with the FOUNDRY_* variables.

Registers the project in agent-framework-dotnet.slnx and cross-links from
the sibling Evaluation_Multimodal / Evaluation_ExpectedOutputs READMEs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): harden FoundryEvals public surface for review

Address PR #6267 review comments on the .NET FoundryEvals integration:

- Add source-compat overloads accepting `string[] evaluators` for `FoundryEvals` ctor, `EvaluateTracesAsync`, and `EvaluateFoundryTargetAsync` so existing callers passing string arrays keep compiling unchanged. New overloads forward via a private `ToSpecs` helper that wraps each name through the implicit `string -> FoundryEvaluatorSpec` conversion.

- Guard against `default(FoundryEvaluatorSpec)` entries (both `BuiltinName` and `GeneratedRef` null) that would NRE the downstream converter. Adds `FoundryEvaluatorSpec.IsValid` / `EnsureValid` plus an internal `EnsureAllSpecsValid` helper, wired into the main ctor and both static evaluation entry points.

- Add 6 unit tests covering the new validation surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(sample): set ExitCode=1 when rubric dimension gate trips

PR #6267 review comment: the FoundryRubric sample swallowed the AssertDimensionScoreAtLeast failure, so a CI run that included it as a quality gate would still exit 0 even when the rubric regressed. Set `System.Environment.ExitCode = 1` in the catch so CI fails while still letting the rest of the sample's logging complete cleanly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry-evals): search typed Sample directly for rubric scores

PR #6267 review comment: `_extract_rubric_scores` only searched the `properties` dict when the sample exposed one. When the Azure AI Projects typed SDK returns a Sample object that puts `dimension_scores` / `rubric_scores` directly on the instance (no `properties` wrapper), we missed them and surfaced no per-dimension scores.

Add an `else: containers.append(sample)` branch so non-dict typed samples are also inspected for the score keys. Covered by two new tests: one with `dimension_scores` directly on a typed Sample without a `properties` wrapper, and one with the legacy `rubric_scores` key in the same shape.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(evals): cover assert_score_at_least and assert_no_failed_items

PR #6267 review comments: both assertion helpers shipped without unit tests. Add `TestAssertScoreAtLeast` (above threshold, below w/ offenders, evaluator filter, sub_results recursion) and `TestAssertNoFailedItems` (all passing, failed/errored statuses, sub_results recursion) with a shared `_score_results` fixture builder.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(samples): remove dead rubric-evaluator doc link from FoundryRubric sample

The Azure AI Foundry rubric evaluator concept doc page has not yet been published, so the link in the sample README and Program.cs comment 404s. Drop the references until the upstream doc is live.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Address PR 6267 review nits

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-06-18 20:50:28 +00:00
westey dc445592ed Python: [BREAKING] Port FileMemoryProvider and integrate FileMemoryProvider & FileAccess into the harness agent (#6547)
* Port FileMemoryProvider to python and integrate it and FileAccessProvider into the harness

* Address PR comments

* Address PR comments

* Create FileSystemAgentFileStore root lazily on first write

Construction no longer calls mkdir, so building a store (and therefore a
default create_harness_agent, which wires default file-memory and file-access
stores under the CWD) performs no filesystem writes and does not fail in
read-only working directories. The root directory is created on the first
write_file / create_directory call; all read/list/search operations already
tolerate a missing root. Updates docstrings and adds a regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix typing

* Fixing typing errors

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 20:15:46 +00:00
westey 92823e9e61 .NET: [BREAKING] Require approval for FileAccessProvider tools with auto-approval rules (#6521)
* Require approval for file-access

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Rename DisableToolApproval to DisableToolAutoApproval for clarity

* Fix broken suggestion.

* Address PR comments and fix build issue.

* Update dotnet/src/Microsoft.Agents.AI.Harness/HarnessAgentOptions.cs

Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Roger Barreto <19890735+rogerbarreto@users.noreply.github.com>
2026-06-18 18:55:22 +00:00
Eduard van Valkenburg 7a491f8e76 Python: Add hosting channel ADRs and spec (#6578)
* Add Python hosting channel ADRs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add Python hosting implementation spec

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 18:29:24 +00:00
Ben Thomas 015e3bcd3b .NET: Enabling sequential orchestration to pass entire conversation or only previous output. (#6554)
* Fix sequential workflow input forwarding

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Make sequential workflow context configurable

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Clarify sequential chain-only behavior

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Clarify sequential output messaging

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Ben Thomas <25218250+alliscode@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 17:53:47 +00:00
Eduard van Valkenburg 6e95517659 Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) (#6443)
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)

Rework the typing setup along the lines of the 'too many type checkers'
approach:

- Pyright (strict) is now the sole source-code type checker; mypy is
  removed from source and its [tool.mypy] block becomes a relaxed profile
  used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
  ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
  basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
  pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
  '# type: ignore' on both branches so results match across interpreter
  versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
  gating checkers and pyright on source+tests+samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix merge regressions from main (typing + runtime)

Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:

Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
  _parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
  ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
  return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
  output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
  contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
  ExecutionMode used by the processor tests.

Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
  observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
  dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
  payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.

Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Isolate per-package mypy cache in test-typing fan-out

The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.

Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Make lab pyright-only on source (drop source mypy)

Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:

- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
  now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.

Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix test-typing regressions from latest main merge

A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:

- core: narrow Optional span.attributes with 'and' guards in span filters
  and assert+cast the json.loads(...attributes[...]) reads (test_observability);
  match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
  pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
  .annotations[0] access through a small _first_annotation helper (mirrors the
  file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
  (zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
  and connections.get_default (zuban) SDK type gaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated pyright version

* pyright fix

* Python: Fix source typing for pyright 1.1.410

Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:

- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
  AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
  the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
  Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
  reduce(and_, ...), which pyright could no longer fully type (drops the now
  unused functools.reduce / operator.and_ imports).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Accept plain-text body in Azure Functions workflow/run endpoint

The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.

Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 15:06:20 +00:00
Evan Mattson 97bb1d588a Migrate to using issue type bug instead of label bug. (#6595) 2026-06-18 14:47:00 +00:00
Roger Barreto 1fc57c45ee .NET: Bump Azure.AI.Projects to 2.1.0-beta.3 (#6542)
* Bump Azure.AI.Projects to 2.1.0-beta.3

Updates Azure.AI.Projects from 2.1.0-beta.2 to 2.1.0-beta.3, together with the transitive Azure.Core (1.56.0 to 1.57.0) and System.ClientModel (1.12.0 to 1.13.0) pins that beta.3 requires (beta.3 forces System.ClientModel 1.13.0.0 via Azure.Core 1.57.0).

Migrates the affected samples and integration test to the beta.3 surface:
* MemorySearch sample: MemorySearchToolCallResponseItem renamed to MemorySearchToolCall, Results renamed to Memories, MemoryItem indirection removed.
* AgentSkills sample: skill provisioning/download API redesigned to a version based model (CreateSkillVersionFromFiles, GetSkillContent which now downloads and unzips), removing manual ZIP handling.
* Session files integration test: GetSessionFilesAsync now returns an async collection of SessionDirectoryEntry and renames the sessionId parameter to agentSessionId.

* Stream session file listing and short-circuit in integration test

Avoids materializing the entire session directory listing into a List. The test now streams GetSessionFilesAsync and breaks as soon as the expected entry is found, then asserts it was located.

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 13:58:22 +00:00
Evan Mattson b3f8aaa9d7 Python: adjust coverage report handoff (#6576)
* Adjust coverage report handoff

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Simplify coverage report handoff check

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 12:35:45 +00:00
dependabot[bot] c22fc8d653 Build(deps): Bump esbuild, @tailwindcss/vite, @vitejs/plugin-react and vite (#6503)
Removes [esbuild](https://github.com/evanw/esbuild). It's no longer used after updating ancestor dependencies [esbuild](https://github.com/evanw/esbuild), [@tailwindcss/vite](https://github.com/tailwindlabs/tailwindcss/tree/HEAD/packages/@tailwindcss-vite), [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/tree/HEAD/packages/plugin-react) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite). These dependencies need to be updated together.


Removes `esbuild`

Updates `@tailwindcss/vite` from 4.1.12 to 4.3.1
- [Release notes](https://github.com/tailwindlabs/tailwindcss/releases)
- [Changelog](https://github.com/tailwindlabs/tailwindcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/tailwindlabs/tailwindcss/commits/v4.3.1/packages/@tailwindcss-vite)

Updates `@vitejs/plugin-react` from 5.0.1 to 5.2.0
- [Release notes](https://github.com/vitejs/vite-plugin-react/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-react/blob/plugin-react@5.2.0/packages/plugin-react/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-react/commits/plugin-react@5.2.0/packages/plugin-react)

Updates `vite` from 7.3.2 to 8.0.16
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version:
  dependency-type: indirect
- dependency-name: "@tailwindcss/vite"
  dependency-version: 4.3.1
  dependency-type: direct:production
- dependency-name: "@vitejs/plugin-react"
  dependency-version: 5.2.0
  dependency-type: direct:development
- dependency-name: vite
  dependency-version: 8.0.16
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-18 11:35:48 +00:00
dependabot[bot] a3131b8130 Build(deps): Bump esbuild, @vitejs/plugin-react and vite (#6501)
Removes [esbuild](https://github.com/evanw/esbuild). It's no longer used after updating ancestor dependencies [esbuild](https://github.com/evanw/esbuild), [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/tree/HEAD/packages/plugin-react) and [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite). These dependencies need to be updated together.


Removes `esbuild`

Updates `@vitejs/plugin-react` from 4.7.0 to 6.0.2
- [Release notes](https://github.com/vitejs/vite-plugin-react/releases)
- [Changelog](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite-plugin-react/commits/plugin-react@6.0.2/packages/plugin-react)

Updates `vite` from 7.3.2 to 8.0.16
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v8.0.16/packages/vite)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version:
  dependency-type: indirect
- dependency-name: "@vitejs/plugin-react"
  dependency-version: 6.0.2
  dependency-type: direct:development
- dependency-name: vite
  dependency-version: 8.0.16
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-18 11:35:40 +00:00
dependabot[bot] 3d46595111 Python: Bump prek from 0.4.3 to 0.4.5 in /python (#6527)
* Bump prek from 0.4.3 to 0.4.5 in /python

Bumps [prek](https://github.com/j178/prek) from 0.4.3 to 0.4.5.
- [Release notes](https://github.com/j178/prek/releases)
- [Changelog](https://github.com/j178/prek/blob/master/CHANGELOG.md)
- [Commits](https://github.com/j178/prek/compare/v0.4.3...v0.4.5)

---
updated-dependencies:
- dependency-name: prek
  dependency-version: 0.4.5
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix python workspace prek pin mismatch

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-18 10:09:11 +00:00
dependabot[bot] 205f7bcca8 Python: Bump pytest from 9.0.3 to 9.1.0 across /python workspace (#6524)
* Bump pytest from 9.0.3 to 9.1.0 in /python

Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.3 to 9.1.0.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.3...9.1.0)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.1.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix Python workspace pytest pin mismatch

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-18 10:08:10 +00:00
dependabot[bot] 2048289fb0 Build(deps): Bump pydantic-monty from 0.0.17 to 0.0.18 in /python (#6392)
Bumps [pydantic-monty](https://github.com/pydantic/monty) from 0.0.17 to 0.0.18.
- [Release notes](https://github.com/pydantic/monty/releases)
- [Commits](https://github.com/pydantic/monty/compare/v0.0.17...v0.0.18)

---
updated-dependencies:
- dependency-name: pydantic-monty
  dependency-version: 0.0.18
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-18 10:05:13 +00:00
westey 699916d639 Python: Add WebSearchDisplayObserver to harness console (#6572)
* Adding an observer to the python harness for web search tools

* Escape dynamic strings with rich.markup.escape() in WebSearchDisplayObserver

Apply rich.markup.escape() to all user/tool-provided strings (queries, URLs,
titles, patterns) before interpolation into Rich-markup-enabled output. This
prevents characters like '['/']' from being interpreted as Rich markup tags.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 09:42:30 +00:00
Roger Barreto 1ba5cd3f44 .NET: Scope argument-based standing approvals correctly in ToolApprovalAgent (#6486) (#6487)
Ensure an argument-scoped standing approval (the "always approve with exact
arguments" path) records an empty argument set rather than null when the
approved call has no arguments, so it matches only future no-argument calls.
null remains reserved exclusively for tool-level approvals, keeping the two
scopes distinct. This aligns the .NET behavior with the existing Python harness.

Adds regression tests covering the no-argument standing-approval flow, the
MatchesRule argument-scoping semantics, and empty-arguments rule serialization.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 09:13:24 +00:00
Roger Barreto 1519e50f2f Harden archive extraction guard so path containment is statically recognized (#6564) (#6565)
The Hosted-AgentSkills sample and its mirrored unit-test helper gated ZIP
extraction on `StartsWith(destinationRoot)` OR `Equals(destinationRoot)`. The
second branch left an acceptance path not covered by the containment check, so
static analysis could not prove the extraction sink stays within the
destination. Make the single resolved-path StartsWith check the only gate to
extraction in both files and add a nested-entry regression test.

Closes #6564
2026-06-18 09:12:00 +00:00
Evan Mattson b55992bb67 Bump Python package versions for 1.9.0 release (#6583)
Selective, CHANGELOG-driven version bumps for the 2026-06-18 release.

Released tier: agent-framework-core and the root agent-framework go to 1.9.0
(minor). Core ships new public APIs (agent-loop middleware, tool-approval
middleware and harness integration, shell-tool harness integration, AG-UI
thread snapshot persistence, context-provider telemetry) plus two behavioral
breaking changes on evolving surfaces: MCP sampling now denies server-initiated
requests by default, and the FileAccess tools were aligned with the .NET
implementation. These are treated as within-1.x changes because every package
caps core at <2; a major bump would require rewriting those caps. The foundry
and openai packages go to 1.8.2 (patch, bug fixes only). The root
agent-framework-core[all] pin was moved to 1.9.0 in lockstep with core.

Release-candidate tier: ag-ui to 1.0.0rc5 and declarative to 1.0.0rc2 for their
respective changes. orchestrations is promoted to stable 1.0.0; PACKAGE_STATUS
and the README install hint were updated accordingly.

Prerelease tier (new Pacific date stamp 260618): anthropic (beta),
azure-contentunderstanding (alpha) and foundry-hosting (alpha). No beta cohort
bump was applied; only packages with changes this cycle were stamped.

Dependency floors: following the established convention, the core floor was
raised to >=1.9.0 on every non-core package bumped this cycle, preserving the
existing <2 upper bound.

Also resolves two pre-existing failures in the dependency-bounds validator that
are unrelated to the version bumps. Hosted-environment detection now catches a
bare ImportError so optional Foundry hosting probing cannot crash user-agent
setup. The harness shell-tool integration, which lazily imports the separate
agent-framework-tools package to avoid a circular runtime dependency, is now
type-checked and tested in isolated environments via a core dev
dependency-group, with the shell-tool tests guarded to skip when that package
is absent.
2026-06-18 18:01:17 +09:00
Evan Mattson e8cec71ed8 Use issue type for triage workflow (#6577)
* Use issue type for triage workflow

* Disable blank issue reports

* Revert "Disable blank issue reports"

This reverts commit 222c8444a7b3b5768e01b9d562195a27d1a29f1a.
2026-06-18 14:48:00 +09:00
Eduard van Valkenburg d7e63d7d0e Fix Foundry aiohttp dependency (#6567)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 02:03:16 +00:00
Changjian Wang f59d5c67d8 Python: Adopt azure-ai-contentunderstanding to_llm_input in CU context provider (#5796)
* Refactor DocumentEntry model and update result handling

- Changed the type of `result` in DocumentEntry from dict to str to store LLM-ready text.
- Introduced `search_payload` in DocumentEntry for optional alternate rendering.
- Updated FileSearchConfig to include `include_fields` option for vector store uploads.
- Modified tests to reflect changes in DocumentEntry and FileSearchConfig.
- Adjusted integration tests to validate new result structure and rendering.
- Removed legacy format_result tests as rendering is now handled by the SDK.

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Add test to ensure page markers are preserved in LLM input

Co-authored-by: Copilot <copilot@github.com>

* fix(cu-context-provider): scope LLMStats telemetry filter to rai_warnings block

Address PR #5796 review comment: the previous defensive scrubber ran a global regex substitution over the full rendered string, so any markdown body bullet shaped like '- LLMStats: ...' would also be silently deleted.

Add a _strip_rai_telemetry helper that confines the substitution to the front-matter rai_warnings: YAML sub-block, leaving the body verbatim. Cover the new behavior with three tests (scoped strip, body preservation, and no-op branches).

* Sync uv.lock with azure-ai-contentunderstanding>=1.2.0b1 dependency bump

* Python: Drop search_payload/include_fields, single to_llm_input rendering (CU context provider)

Address PR #5796 review: remove the redundant search_payload field and _render_search_payload helper, drop the include_fields opt-in (already covered by output_sections), rename _resolve_pending_tokens -> _resolve_pending_analysis, and have _upload_to_vector_store read entry['result'] directly.

* Python: Adopt SDK 1.2.0b2 LLMStats filtering, drop local workaround (CU context provider)

azure-ai-contentunderstanding 1.2.0b2 filters LLMStats telemetry from rai_warnings and emits InputPageNumber page markers in to_llm_input, so the provider's local defense is redundant.

- Bump dependency to azure-ai-contentunderstanding>=1.2.0b2 (re-lock uv.lock)

- Remove _strip_rai_telemetry and its two regexes; _render_for_llm now returns to_llm_input(...) directly

- Delete 4 workaround unit tests for the removed helper

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: changjian-wang <v-changjwang@microsoft.com>
Co-authored-by: aluneth <wangchangjian1130@163.com>
2026-06-18 01:57:41 +00:00
Shyju Krishnankutty 26a0a7e8be .NET: (Durable): bind MCP threadId to the current agent and guard cross-agent session dispatch (#6531)
* scope MCP threadId to the current agent

* Fix Async suffix on test methods and add CHANGELOG entries

- Rename three test methods to include Async suffix (IDE1006 fix)
- Add CHANGELOG entries for DurableTask and Hosting.AzureFunctions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 21:56:53 +00:00
Yufeng He 616315339e .NET: fix fan-in checkpoint edge state (#6491) 2026-06-17 20:17:53 +00:00
Eduard van Valkenburg fcc5576b04 .NET: feat(dotnet): Add LocalCodeAct package for local Python execution (#6105)
* feat(dotnet): Add LocalCodeAct package scaffold

Create Microsoft.Agents.AI.LocalCodeAct package with:
- Project file with embedded Python resources
- ExecutionMode enum (Subprocess only)
- ProcessExecutionLimits record
- FileMount record and FileMountMode enum
- README.md documentation
- Embedded Python runner and validator scripts

This is the .NET equivalent of the Python agent-framework-local-codeact
package. Next: Implement process bridge and tool integration.

* feat(dotnet): Add embedded Python runner and validator

Copy Python runner and validator scripts from the Python implementation
as embedded resources for the .NET package.

* feat(dotnet): Add CodeValidator wrapper

Implement CodeValidator.cs that:
- Extracts embedded Python validator script to temp file
- Invokes Python validator with JSON request
- Passes custom allow/block lists
- Throws CodeValidationException on failures
- Cleans up temp files

Uses the embedded Resources/validator.py for AST validation.

* feat(dotnet): Add LocalExecuteCodeFunction

Implement LocalExecuteCodeFunction as AIFunction:
- Accepts Python executable path (required)
- Registers host tools for code to call
- Validates code via CodeValidator if custom lists provided
- Executes via ProcessBridge
- Converts result dict to ChatMessage list
- Builds dynamic description including available tools

Matches Python LocalExecuteCodeTool functionality.

* feat(dotnet): Add LocalCodeActProvider

Implement AIContextProvider that:
- Injects execute_code tool into context
- Adds CodeAct instructions
- Enforces single-provider-per-agent via StateKeys
- Wraps LocalExecuteCodeFunction lifecycle

Minimal provider implementation matching Python LocalCodeActProvider.

* feat(dotnet): Add tests and sample for LocalCodeAct

Add unit tests:
- LocalExecuteCodeFunctionTests (4 tests)
- ProcessExecutionLimitsTests (2 tests)
- FileMountTests (2 tests)

Add sample:
- LocalCodeAct/Program.cs - Demonstrates provider and function usage
- LocalCodeAct/README.md - Documentation and safety warnings

Tests verify basic construction, metadata, and disposal.
Sample shows provider creation, function setup, and configuration.

Note: Build requires .NET 10 SDK per global.json.

* feat(dotnet): Add LocalCodeAct sample project

Add sample demonstrating:
- LocalCodeActProvider creation and configuration
- LocalExecuteCodeFunction direct usage
- Execution modes and file mount configuration
- Safety warnings and prerequisites

Includes project file and README with security guidance.

* feat(dotnet): Add file mount support and integration tests

- Added FileMountHelper.cs for file mount normalization, snapshot, and capture
- Updated LocalExecuteCodeFunction to support file mounts parameter
- Added file snapshot before/after execution with capture logic
- Updated LocalCodeActProvider to pass file mounts through
- Created comprehensive IntegrationTests.cs with 10 test cases:
  - Simple code execution
  - Timeout handling
  - Syntax error handling
  - Blocked import validation
  - Blocked builtin validation
  - Custom allowed imports
  - File mount read/write with capture
  - Stdout capture
  - Provider tool injection

All features from Python implementation now ported to .NET.

* Rewrite .NET LocalCodeAct to address all PR review comments

Complete rewrite that follows the Hyperlight package conventions
(see Microsoft.Agents.AI.Hyperlight) and addresses all 24 review
comments on PR #6105:

Architectural fixes:
* LocalCodeActProvider now uses options-class constructor pattern
  matching HyperlightCodeActProvider.
* Override of ProvideAIContextAsync uses the correct
  (InvokingContext, CancellationToken) signature returning
  ValueTask<AIContext>.
* ExecuteCodeFunction follows the AIFunction Name/Description/JsonSchema
  property pattern with InvokeCoreAsync override.
* Provider exposes AddTools/GetTools/RemoveTools/ClearTools and
  AddFileMounts/GetFileMounts/RemoveFileMounts/ClearFileMounts CRUD
  methods, with snapshot-at-invocation semantics under a lock.

Runtime/security fixes:
* Subprocess IPC uses JsonObject/JsonNode end-to-end (no
  Dictionary<string, object?> casts that broke under JsonElement
  deserialization).
* Validator runs in its own subprocess with a dedicated timeout
  (ProcessExecutionLimits.ValidationTimeoutSeconds), never reuses
  the runner script.
* Validation enabled by default; can be opt-ed out via
  ValidationEnabled = false.
* validator.py has a __main__ entrypoint that reads JSON from
  stdin and exits with structured errors.
* validator.py is now compatible with Python 3.9+ (Match nodes
  added conditionally).
* call_id parsed as long to match Python id(kwargs) range.

Other:
* README rewritten with valid C# syntax (options-class, FileMount
  constructor) and accurate descriptions of validator and file
  capture behavior.
* Added integration tests that exercise the real subprocess and
  validator (skipped gracefully when python3 is not on PATH).
* All 18 tests pass (15 unit + 3 integration) across net8/net9/net10.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Sync embedded validator.py with Python package allow-list enforcement

The embedded Python validator script used by the .NET LocalCodeAct
package now enforces the builtin allow-list, matching the latest
behavior of agent_framework_local_codeact._validator. Names that are
real Python builtins must appear in the allow-list, while unknown names
(user-defined functions, registered tools) remain allowed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add Hosted-LocalCodeAct foundry hosted-agent sample

Mirrors the Python foundry_hosted_agent.py sample for the local-codeact
package: registers compute and fetch_data as sandbox-only host tools on
LocalCodeActProvider so the model only sees execute_code and reaches them
via await call_tool(...). Includes the standard hosted-agent supporting
files (agent.yaml, agent.manifest.yaml, Dockerfile, Dockerfile.contributor,
.env.example, README.md) and installs python3 in the container images so
the embedded runner and validator can execute.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(local-codeact-dotnet): sync validator os.* allow-list with Python

Mirror the Python package change: the embedded validator.py invoked by the
.NET ProcessBridge replaces the os.* deny-list with an allow-list of
{environ, path}. Add allowed_os_attrs parameter to validate_code and
_CodeValidator, and surface it via the stdin JSON request schema so the
.NET host can opt in to a broader allow-list when needed.

Default behavior tightens to match the documented contract: any os.*
attribute outside {environ, path} (for example os.listdir, os.open,
os.getcwd) is rejected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(local-codeact-dotnet): address review + tighten validator

- validator.py: enforce os.* allow-list on `from os import X` so names like
  `system`, `getcwd` cannot bypass the visit_Attribute restriction.
- ProcessBridge.ConfigureEnvironment: document that null Environment inherits
  the parent env (matching real behavior) and update the public
  LocalCodeActProviderOptions.Environment doc to describe the explicit
  empty-dictionary opt-in for a scrubbed environment.
- Tests:
  * FileMountHelperTests covers per-file, per-mount, and total
    capture-limit branches that return TextContent omissions.
  * Integration tests cover unknown-tool dispatch error, tool throwing
    exception, and CodeValidator timeout that kills the process and
    raises CodeValidationException.
- Sample: drop unused `Microsoft.Agents.AI.Foundry` using in
  Hosted-LocalCodeAct/Program.cs to satisfy IDE0005 check-format.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(local-codeact-dotnet): remove stale orphan sample

The dotnet/samples/LocalCodeAct/ scaffolding sample referenced APIs
that don't exist in the current package (`ExecutionMode`, FileMount
object-initializer syntax, the old LocalExecuteCodeFunction
constructor signature, function.Metadata.*), produced a long list of
check-format violations (CHARSET, IMPORTS, IDE0073 header, IDE0005
unused using, IDE1006 Async suffix, RCS1037 trailing whitespace), and
did not match any of the documented sample layouts.

The hosted-agent example at
dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalCodeAct
is the supported entry-point sample for this package.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* style(local-codeact-dotnet): satisfy check-format rules

- Add UTF-8 BOM to source files (CHARSET)
- Remove unused using directives (IDE0005)
- Simplify type names (IDE0001/IDE0002/IDE0090)
- Rename static field JsonOptions -> s_jsonOptions (IDE1006)
- Rename static field SyncRoot -> s_syncRoot (IDE1006)
- Add missing this. qualifications in ProcessBridge (IDE0009)
- Remove unused _options field from LocalCodeActProvider (IDE0052)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(local-codeact-dotnet): wire hosted sample into solution

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(local-codeact-dotnet): sync embedded Python scripts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(local-codeact-dotnet): exercise Python integration on Windows

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Address LocalCodeAct API review feedback

Move the required Python executable path to LocalCodeAct constructors, invert the validation flag default, and apply small project/file mount cleanup suggestions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Address LocalCodeAct concurrency review

Surface unauthorized mount traversal errors and use concurrent provider registries for LocalCodeAct tool and file mount CRUD operations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Simplify LocalCodeAct function wrappers

Use AIFunctionFactory-created inner functions for LocalCodeAct execute_code wrappers and remove redundant script cache and JsonNode cloning logic.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* .NET: Update LocalCodeAct factory result tests

Handle JsonElement result values produced by AIFunctionFactory delegation in LocalCodeAct execute_code integration tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 15:30:52 +00:00
westey 6cc7ddb73e .NET: Integrate LoopAgent into HarnessAgent with TodoCompletionLoopEvaluator (#6544)
* Add LoopAgent to Harness with TodoEvaluator sample

* Address PR comments

* Fix build error
2026-06-17 19:03:16 +01:00
westey 39f4b5ec72 Align function tool names for BackgroundAgent and FileMemory between python and .net (#6550) 2026-06-17 17:14:12 +01:00
Tao Chen bd32e3142c Fix comments 2026-06-16 11:03:57 -07:00
Tao Chen 1a698f92ba Add tests 2026-06-16 10:52:03 -07:00
Tao Chen f70c58fa7c Make sure spans created inside sync ops in streaming path are correctly nested 2026-06-16 10:42:13 -07:00
638 changed files with 25504 additions and 11081 deletions
+1 -1
View File
@@ -1,7 +1,7 @@
name: .NET Bug Report
description: Report a bug in the Agent Framework .NET SDK
title: ".NET: [Bug]: "
labels: ["bug", ".NET"]
labels: [".NET"]
type: bug
body:
- type: textarea
+1 -1
View File
@@ -1,7 +1,7 @@
name: Python Bug Report
description: Report a bug in the Agent Framework Python SDK
title: "Python: [Bug]: "
labels: ["bug", "Python"]
labels: ["Python"]
type: bug
body:
- type: textarea
+5 -3
View File
@@ -24,12 +24,14 @@ updates:
- ".NET"
- "dependencies"
# Maintain dependencies for python
# Maintain dependencies for python.
# TODO: Remove these Python Dependabot entries after we have confidence in the
# Python dependency-maintenance workflow.
- package-ecosystem: "pip"
directory: "python/"
schedule:
interval: "weekly"
day: "monday"
day: "thursday"
labels:
- "python"
- "dependencies"
@@ -37,7 +39,7 @@ updates:
directory: "python/"
schedule:
interval: "weekly"
day: "monday"
day: "thursday"
labels:
- "python"
- "dependencies"
+3 -5
View File
@@ -2,7 +2,7 @@ name: Issue Triage
on:
issues:
types: [opened, labeled]
types: [opened, typed]
permissions:
contents: read
@@ -12,9 +12,7 @@ permissions:
concurrency:
group: >-
issue-triage-${{ github.repository }}-${{
((github.event.action == 'opened' && contains(github.event.issue.labels.*.name, 'bug'))
|| (github.event.action == 'labeled' && github.event.label.name == 'bug'))
&& github.event.issue.number
github.event.issue.type.name == 'Bug' && github.event.issue.number
|| github.run_id
}}
cancel-in-progress: true
@@ -28,7 +26,7 @@ env:
jobs:
team_check:
runs-on: ubuntu-latest
if: ${{ (github.event.action == 'opened' && contains(github.event.issue.labels.*.name, 'bug')) || (github.event.action == 'labeled' && github.event.label.name == 'bug') }}
if: ${{ github.event.issue.type.name == 'Bug' }}
outputs:
is_team_member: ${{ steps.check.outputs.is_team_member }}
issue_number: ${{ steps.issue.outputs.issue_number }}
+1 -3
View File
@@ -90,9 +90,7 @@ jobs:
// Check for issue type from issue form dropdown
const issueTypeField = getFormFieldValue(body, 'Type of Issue')
if (issueTypeField) {
if (issueTypeField === 'Bug') {
labels.push("bug")
} else if (issueTypeField === 'Feature Request') {
if (issueTypeField === 'Feature Request') {
labels.push("enhancement")
} else if (issueTypeField === 'Question') {
labels.push("question")
+4 -6
View File
@@ -113,8 +113,8 @@ jobs:
- name: Run markdown code lint
run: uv run poe markdown-code-lint
mypy:
name: Mypy Checks
test-typing:
name: Test Typing Checks
if: "!cancelled()"
strategy:
fail-fast: false
@@ -139,7 +139,5 @@ jobs:
os: ${{ runner.os }}
env:
UV_CACHE_DIR: /tmp/.uv-cache
- name: Run Mypy
env:
GITHUB_BASE_REF: ${{ github.event.pull_request.base.ref || github.base_ref || 'main' }}
run: uv run python scripts/workspace_poe_tasks.py ci-mypy
- name: Run tests/samples type checkers (mypy, pyrefly, ty)
run: uv run python scripts/workspace_poe_tasks.py ci-test-typing
@@ -0,0 +1,365 @@
name: Python - Dependency Maintenance
on:
workflow_dispatch:
schedule:
- cron: "0 4 * * 1"
permissions:
contents: write
issues: write
pull-requests: write
concurrency:
group: python-dependency-maintenance
cancel-in-progress: false
env:
UV_CACHE_DIR: /tmp/.uv-cache
jobs:
dependency-maintenance:
name: Dependency Maintenance
runs-on: ubuntu-latest
env:
# Match the existing Python dependency maintenance workflows. Reevaluate if package
# installability starts differing across supported Python versions.
UV_PYTHON: "3.13"
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 0
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
env:
UV_CACHE_DIR: /tmp/.uv-cache
- name: Set dependency release cutoff
run: |
cutoff="$(date -u -d '7 days ago' '+%Y-%m-%dT%H:%M:%SZ')"
echo "DEPENDENCY_RELEASE_CUTOFF=${cutoff}" >> "$GITHUB_ENV"
echo "Using dependency release cutoff: ${cutoff}"
- name: Repin dev dependency declarations
run: uv run poe upgrade-dev-dependency-pins
working-directory: ./python
- name: Refresh lockfile after dev pin updates
run: uv lock
working-directory: ./python
- name: Save dev dependency changes
run: |
DEV_PATCH="${RUNNER_TEMP}/python-dev-dependency-updates.patch"
git diff -- python/pyproject.toml "python/packages/*/pyproject.toml" python/uv.lock > "${DEV_PATCH}"
if [ -s "${DEV_PATCH}" ]; then
echo "has_dev_changes=true" >> "$GITHUB_OUTPUT"
else
echo "has_dev_changes=false" >> "$GITHUB_OUTPUT"
fi
echo "patch=${DEV_PATCH}" >> "$GITHUB_OUTPUT"
id: dev_changes
- name: Run dependency bounds test scenarios
id: validate_bounds_test
continue-on-error: true
run: uv run poe validate-dependency-bounds-test --package "*"
working-directory: ./python
- name: Run dependency upper-bound validation
id: validate_ranges
if: steps.validate_bounds_test.outcome == 'success'
continue-on-error: true
run: uv run poe validate-dependency-bounds-project --mode upper --package "*"
working-directory: ./python
- name: Upload dependency validation reports
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: dependency-maintenance-results
path: |
python/scripts/dependencies/dependency-bounds-test-results.json
python/scripts/dependencies/dependency-range-results.json
if-no-files-found: warn
- name: Create issue for failed dependency bounds test
if: steps.validate_bounds_test.outcome != 'success'
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require("fs")
const reportPath = "python/scripts/dependencies/dependency-bounds-test-results.json"
const owner = context.repo.owner
const repo = context.repo.repo
const openIssues = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: "open",
per_page: 100,
})
const openIssueTitles = new Set(
openIssues.filter((issue) => !issue.pull_request).map((issue) => issue.title)
)
const formatError = (message) => String(message || "No error output captured.").replace(/```/g, "'''")
const title = "Dependency bounds test failed"
if (openIssueTitles.has(title)) {
core.info(`Issue already exists: ${title}`)
return
}
const bodyLines = [
"Automated dependency bounds test mode failed before dependency upper-bound validation could run.",
"",
"The weekly dependency maintenance workflow kept only dev dependency updates for the generated PR, if any, and skipped dependency range updates for this run.",
"",
]
if (fs.existsSync(reportPath)) {
const report = JSON.parse(fs.readFileSync(reportPath, "utf8"))
const failedScenarios = (report.scenarios ?? []).filter((scenario) => scenario.status === "failed")
for (const scenario of failedScenarios) {
bodyLines.push(`### ${scenario.name} scenario (${scenario.resolution})`)
const failedPackages = (scenario.packages ?? []).filter((pkg) => pkg.status === "failed")
for (const pkg of failedPackages.slice(0, 10)) {
bodyLines.push(
"",
`- Package: \`${pkg.package_name}\``,
`- Project path: \`${pkg.project_path}\``,
"",
"```",
formatError(pkg.error).slice(0, 3500),
"```"
)
}
if (failedPackages.length > 10) {
bodyLines.push("", `_Additional failed packages omitted: ${failedPackages.length - 10}_`)
}
}
} else {
bodyLines.push(`No dependency bounds test report was found at \`${reportPath}\`.`)
}
bodyLines.push("", `Workflow run: ${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`)
await github.rest.issues.create({
owner,
repo,
title,
body: bodyLines.join("\n"),
})
core.info(`Created issue: ${title}`)
- name: Create issues for failed dependency candidates
if: always()
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
script: |
const fs = require("fs")
const reportPath = "python/scripts/dependencies/dependency-range-results.json"
if (!fs.existsSync(reportPath)) {
core.info(`No dependency range report found at ${reportPath}`)
return
}
const report = JSON.parse(fs.readFileSync(reportPath, "utf8"))
const dependencyFailures = []
for (const packageResult of report.packages ?? []) {
for (const dependency of packageResult.dependencies ?? []) {
const candidateVersions = new Set(dependency.candidate_versions ?? [])
const failedAttempts = (dependency.attempts ?? []).filter(
(attempt) => attempt.status === "failed" && candidateVersions.has(attempt.trial_upper)
)
if (!failedAttempts.length) {
continue
}
const failuresByVersion = new Map()
for (const attempt of failedAttempts) {
const version = attempt.trial_upper || "unknown"
if (!failuresByVersion.has(version)) {
failuresByVersion.set(version, attempt.error || "No error output captured.")
}
}
dependencyFailures.push({
packageName: packageResult.package_name,
projectPath: packageResult.project_path,
dependencyName: dependency.name,
originalRequirements: dependency.original_requirements ?? [],
finalRequirements: dependency.final_requirements ?? [],
failedVersions: [...failuresByVersion.entries()].map(([version, error]) => ({ version, error })),
})
}
}
if (!dependencyFailures.length) {
core.info("No failing dependency candidates found.")
return
}
const owner = context.repo.owner
const repo = context.repo.repo
const openIssues = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: "open",
per_page: 100,
})
const openIssueTitles = new Set(
openIssues.filter((issue) => !issue.pull_request).map((issue) => issue.title)
)
const formatError = (message) => String(message || "No error output captured.").replace(/```/g, "'''")
for (const failure of dependencyFailures) {
const title = `Dependency validation failed: ${failure.dependencyName} (${failure.packageName})`
if (openIssueTitles.has(title)) {
core.info(`Issue already exists: ${title}`)
continue
}
const visibleFailures = failure.failedVersions.slice(0, 5)
const omittedCount = failure.failedVersions.length - visibleFailures.length
const failureDetails = visibleFailures
.map(
(entry) =>
`- \`${entry.version}\`\n\n\`\`\`\n${formatError(entry.error).slice(0, 3500)}\n\`\`\``
)
.join("\n\n")
const body = [
"Automated dependency range validation found candidate versions that failed checks.",
"",
`- Package: \`${failure.packageName}\``,
`- Project path: \`${failure.projectPath}\``,
`- Dependency: \`${failure.dependencyName}\``,
`- Original requirements: ${
failure.originalRequirements.length
? failure.originalRequirements.map((value) => `\`${value}\``).join(", ")
: "_none_"
}`,
`- Final requirements after run: ${
failure.finalRequirements.length
? failure.finalRequirements.map((value) => `\`${value}\``).join(", ")
: "_none_"
}`,
"",
"### Failed versions and errors",
failureDetails,
omittedCount > 0 ? `\n_Additional failed versions omitted: ${omittedCount}_` : "",
"",
`Workflow run: ${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`,
].join("\n")
await github.rest.issues.create({
owner,
repo,
title,
body,
})
openIssueTitles.add(title)
core.info(`Created issue: ${title}`)
}
- name: Keep only dev updates when range validation fails
if: steps.validate_bounds_test.outcome != 'success' || steps.validate_ranges.outcome != 'success'
env:
DEV_PATCH: ${{ steps.dev_changes.outputs.patch }}
HAS_DEV_CHANGES: ${{ steps.dev_changes.outputs.has_dev_changes }}
run: |
git restore python/pyproject.toml python/packages/*/pyproject.toml python/uv.lock
if [ "${HAS_DEV_CHANGES}" = "true" ]; then
git apply "${DEV_PATCH}"
fi
- name: Refresh lockfile after dependency range updates
if: steps.validate_bounds_test.outcome == 'success' && steps.validate_ranges.outcome == 'success'
run: uv lock
working-directory: ./python
- name: Install final dependency set
run: uv run poe install
working-directory: ./python
- name: Run final checks
run: uv run poe check
working-directory: ./python
- name: Run final typing
run: uv run poe typing
working-directory: ./python
- name: Commit and push dependency updates
id: commit_updates
run: |
BRANCH="automation/python-dependency-maintenance"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -B "${BRANCH}"
git add python/pyproject.toml python/packages/*/pyproject.toml python/uv.lock
if git diff --cached --quiet; then
echo "has_changes=false" >> "$GITHUB_OUTPUT"
echo "No dependency updates to commit."
exit 0
fi
git commit -m "Python: chore: update dependencies"
git push --force-with-lease --set-upstream origin "${BRANCH}"
echo "has_changes=true" >> "$GITHUB_OUTPUT"
- name: Create or update pull request with GitHub CLI
if: steps.commit_updates.outputs.has_changes == 'true'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
BRANCH="automation/python-dependency-maintenance"
PR_TITLE="Python: chore: update dependencies"
PR_BODY_FILE="$(mktemp)"
cat > "${PR_BODY_FILE}" <<'EOF'
### Motivation & Context
This automated update keeps Python dependency metadata coherent across the uv workspace. Python dependencies can be declared in multiple `pyproject.toml` files, but the workspace has one shared `python/uv.lock`, so dependency maintenance should update and validate them together instead of through per-manifest Dependabot PRs.
### Description & Review Guide
- **What are the major changes?** Refresh Python dev dependency pins, update package dependency ranges when the bounds tooling succeeds, and refresh `python/uv.lock`.
- **What is the impact of these changes?** Keeps the Python workspace dependency set current while producing at most one dependency PR for the week. If dependency range validation fails, this PR contains only the dev dependency updates that still pass final validation, and separate issues track failed range candidates.
- **What do you want reviewers to focus on?** Review the generated dependency metadata changes and any dependency-range updates for package-specific compatibility concerns.
<!-- NOTE FOR AUTOMATED/AI REVIEWERS: the "What do you want reviewers to focus on?"
item above is intended for human reviewers only. Automated/AI reviewers should
ignore it and review the entire change rather than narrowing scope to it. -->
### Related Issue
No linked issue; this PR is generated by scheduled Python dependency maintenance.
### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] All unit tests pass, and I have added new tests where possible
- [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/agent-framework/blob/main/CONTRIBUTING.md)
- [ ] This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
- [x] **This is not a breaking change.** If it _is_ a breaking change, add the `breaking change` label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.
EOF
PR_NUMBER="$(gh pr list --head "${BRANCH}" --base main --state open --json number --jq '.[0].number')"
if [ -n "${PR_NUMBER}" ]; then
gh pr edit "${PR_NUMBER}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
else
gh pr create --base main --head "${BRANCH}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
fi
@@ -1,216 +0,0 @@
# Probe the highest allowed dependency versions, then open issues/PRs from the passing updates.
name: Python - Dependency Range Validation
on:
workflow_dispatch:
permissions:
contents: write
issues: write
pull-requests: write
env:
UV_CACHE_DIR: /tmp/.uv-cache
jobs:
dependency-range-validation:
name: Dependency Range Validation
runs-on: ubuntu-latest
env:
# For now only run 3.13, if we do encounter situations where there are mismatches between packages and python versions (other then 3.10 and 3.14 which are known to not be able to install everything)
# then we will have to reevaluate.
UV_PYTHON: "3.13"
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 0
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
env:
UV_CACHE_DIR: /tmp/.uv-cache
- name: Run dependency range validation
id: validate_ranges
# Keep workflow running so we can still publish diagnostics from this run.
continue-on-error: true
run: uv run poe validate-dependency-bounds-project --mode upper --package "*"
working-directory: ./python
- name: Upload dependency range report
# Always publish the report so failures are inspectable even when validation fails.
if: always()
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
with:
name: dependency-range-results
path: python/scripts/dependencies/dependency-range-results.json
if-no-files-found: warn
- name: Create issues for failed dependency candidates
# Always process the report so failed candidates create actionable tracking issues.
if: always()
uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8
with:
script: |
const fs = require("fs")
const reportPath = "python/scripts/dependencies/dependency-range-results.json"
if (!fs.existsSync(reportPath)) {
core.warning(`No dependency range report found at ${reportPath}`)
return
}
const report = JSON.parse(fs.readFileSync(reportPath, "utf8"))
const dependencyFailures = []
for (const packageResult of report.packages ?? []) {
for (const dependency of packageResult.dependencies ?? []) {
const candidateVersions = new Set(dependency.candidate_versions ?? [])
const failedAttempts = (dependency.attempts ?? []).filter(
(attempt) => attempt.status === "failed" && candidateVersions.has(attempt.trial_upper)
)
if (!failedAttempts.length) {
continue
}
const failuresByVersion = new Map()
for (const attempt of failedAttempts) {
const version = attempt.trial_upper || "unknown"
if (!failuresByVersion.has(version)) {
failuresByVersion.set(version, attempt.error || "No error output captured.")
}
}
dependencyFailures.push({
packageName: packageResult.package_name,
projectPath: packageResult.project_path,
dependencyName: dependency.name,
originalRequirements: dependency.original_requirements ?? [],
finalRequirements: dependency.final_requirements ?? [],
failedVersions: [...failuresByVersion.entries()].map(([version, error]) => ({ version, error })),
})
}
}
if (!dependencyFailures.length) {
core.info("No failing dependency candidates found.")
return
}
const owner = context.repo.owner
const repo = context.repo.repo
const openIssues = await github.paginate(github.rest.issues.listForRepo, {
owner,
repo,
state: "open",
per_page: 100,
})
const openIssueTitles = new Set(
openIssues.filter((issue) => !issue.pull_request).map((issue) => issue.title)
)
const formatError = (message) => String(message || "No error output captured.").replace(/```/g, "'''")
for (const failure of dependencyFailures) {
const title = `Dependency validation failed: ${failure.dependencyName} (${failure.packageName})`
if (openIssueTitles.has(title)) {
core.info(`Issue already exists: ${title}`)
continue
}
const visibleFailures = failure.failedVersions.slice(0, 5)
const omittedCount = failure.failedVersions.length - visibleFailures.length
const failureDetails = visibleFailures
.map(
(entry) =>
`- \`${entry.version}\`\n\n\`\`\`\n${formatError(entry.error).slice(0, 3500)}\n\`\`\``
)
.join("\n\n")
const body = [
"Automated dependency range validation found candidate versions that failed checks.",
"",
`- Package: \`${failure.packageName}\``,
`- Project path: \`${failure.projectPath}\``,
`- Dependency: \`${failure.dependencyName}\``,
`- Original requirements: ${
failure.originalRequirements.length
? failure.originalRequirements.map((value) => `\`${value}\``).join(", ")
: "_none_"
}`,
`- Final requirements after run: ${
failure.finalRequirements.length
? failure.finalRequirements.map((value) => `\`${value}\``).join(", ")
: "_none_"
}`,
"",
"### Failed versions and errors",
failureDetails,
omittedCount > 0 ? `\n_Additional failed versions omitted: ${omittedCount}_` : "",
"",
`Workflow run: ${context.serverUrl}/${owner}/${repo}/actions/runs/${context.runId}`,
].join("\n")
await github.rest.issues.create({
owner,
repo,
title,
body,
})
openIssueTitles.add(title)
core.info(`Created issue: ${title}`)
}
- name: Refresh lockfile
# Only refresh lockfile after a clean validation to avoid committing known-bad ranges.
if: steps.validate_ranges.outcome == 'success'
run: uv lock --upgrade
working-directory: ./python
- name: Commit and push dependency updates
id: commit_updates
if: steps.validate_ranges.outcome == 'success'
run: |
BRANCH="automation/python-dependency-range-updates"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -B "${BRANCH}"
git add python/packages/*/pyproject.toml python/uv.lock
if git diff --cached --quiet; then
echo "has_changes=false" >> "$GITHUB_OUTPUT"
echo "No dependency updates to commit."
exit 0
fi
git commit -m "chore: update dependency ranges"
git push --force-with-lease --set-upstream origin "${BRANCH}"
echo "has_changes=true" >> "$GITHUB_OUTPUT"
- name: Create or update pull request with GitHub CLI
# Only open/update PRs for validated updates to keep automation branches trustworthy.
if: steps.validate_ranges.outcome == 'success' && steps.commit_updates.outputs.has_changes == 'true'
run: |
BRANCH="automation/python-dependency-range-updates"
PR_TITLE="Python: chore: update dependency ranges"
PR_BODY_FILE="$(mktemp)"
cat > "${PR_BODY_FILE}" <<'EOF'
This PR was generated by the dependency range validation workflow.
- Ran `uv run poe validate-dependency-bounds-project --mode upper --package "*"`
- Updated package dependency bounds
- Refreshed `python/uv.lock` with `uv lock --upgrade`
EOF
PR_NUMBER="$(gh pr list --head "${BRANCH}" --base main --state open --json number --jq '.[0].number')"
if [ -n "${PR_NUMBER}" ]; then
gh pr edit "${PR_NUMBER}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
else
gh pr create --base main --head "${BRANCH}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
fi
@@ -1,91 +0,0 @@
name: Python - Dev Dependency Upgrade
on:
workflow_dispatch:
permissions:
contents: write
pull-requests: write
env:
UV_CACHE_DIR: /tmp/.uv-cache
jobs:
upgrade-dev-dependencies:
name: Upgrade Dev Dependencies
runs-on: ubuntu-latest
env:
UV_PYTHON: "3.13"
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
fetch-depth: 0
- name: Set up python and install the project
uses: ./.github/actions/python-setup
with:
python-version: ${{ env.UV_PYTHON }}
os: ${{ runner.os }}
env:
UV_CACHE_DIR: /tmp/.uv-cache
- name: Upgrade dev dependencies and validate workspace
run: uv run poe upgrade-dev-dependencies
working-directory: ./python
- name: Commit and push dev dependency updates
id: commit_updates
run: |
BRANCH="automation/python-dev-dependency-updates"
git config user.name "github-actions[bot]"
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
git checkout -B "${BRANCH}"
git add python/pyproject.toml python/packages/*/pyproject.toml python/uv.lock
if git diff --cached --quiet; then
echo "has_changes=false" >> "$GITHUB_OUTPUT"
echo "No dev dependency updates to commit."
exit 0
fi
git commit -F- <<'EOF'
Python: chore: upgrade dev dependencies
EOF
git push --force-with-lease --set-upstream origin "${BRANCH}"
echo "has_changes=true" >> "$GITHUB_OUTPUT"
- name: Create or update pull request with GitHub CLI
if: steps.commit_updates.outputs.has_changes == 'true'
run: |
BRANCH="automation/python-dev-dependency-updates"
PR_TITLE="Python: chore: upgrade dev dependencies"
PR_BODY_FILE="$(mktemp)"
cat > "${PR_BODY_FILE}" <<'EOF'
### Motivation and Context
This automated update refreshes Python dev dependency pins across the workspace and reruns the repo validation gates before opening a pull request.
### Description
- Ran `uv run poe upgrade-dev-dependencies`
- Refreshed dev dependency pins in workspace `pyproject.toml` files
- Refreshed `python/uv.lock` with `uv lock --upgrade`
- Reinstalled from the frozen lockfile and reran `check`, `typing`, and `test`
### Contribution Checklist
- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [Contribution Guidelines](https://github.com/microsoft/agent-framework/blob/main/CONTRIBUTING.md)
- [x] All unit tests pass, and I have added new tests where possible
- [ ] **Is this a breaking change?** If yes, add "[BREAKING]" prefix to the title of the PR.
EOF
PR_NUMBER="$(gh pr list --head "${BRANCH}" --base main --state open --json number --jq '.[0].number')"
if [ -n "${PR_NUMBER}" ]; then
gh pr edit "${PR_NUMBER}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
else
gh pr create --base main --head "${BRANCH}" --title "${PR_TITLE}" --body-file "${PR_BODY_FILE}"
fi
-3
View File
@@ -92,9 +92,6 @@ jobs:
- name: Run lab type checking
run: cd packages/lab && uv run poe pyright
- name: Run lab mypy
run: cd packages/lab && uv run poe mypy
# Surface failing tests
- name: Surface failing tests
if: always()
@@ -30,21 +30,31 @@ jobs:
merge-multiple: true
- name: Display structure of downloaded files
run: ls
- name: Read and set PR number
# Need to read the PR number from the file saved in the previous workflow
# because the workflow_run event does not have access to the PR number
# The PR number is needed to post the comment on the PR
- name: Read and validate PR number
# Keep the artifact handoff aligned with the workflow run that produced it.
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
RUN_HEAD_SHA: ${{ github.event.workflow_run.head_sha }}
run: |
if [ ! -s pr_number ]; then
echo "PR number file 'pr_number' is missing or empty"
exit 1
fi
PR_NUMBER=$(cat pr_number)
if ! [[ "$PR_NUMBER" =~ ^[0-9]+$ ]]; then
ARTIFACT_PR_NUMBER=$(cat pr_number)
if ! [[ "$ARTIFACT_PR_NUMBER" =~ ^[0-9]+$ ]]; then
echo "::error::PR number file contains invalid content"
exit 1
fi
echo "PR_NUMBER=$PR_NUMBER" >> "$GITHUB_ENV"
PR_HEAD_SHA=$(gh pr view "$ARTIFACT_PR_NUMBER" --repo "$REPO" --json headRefOid --jq '.headRefOid')
if [ "$PR_HEAD_SHA" != "$RUN_HEAD_SHA" ]; then
echo "::error::PR head SHA does not match the triggering workflow run"
exit 1
fi
echo "PR_NUMBER=$ARTIFACT_PR_NUMBER" >> "$GITHUB_ENV"
- name: Pytest coverage comment
id: coverageComment
uses: MishaKav/pytest-coverage-comment@26f986d2599c288bb62f623d29c2da98609e9cd4 # v1.6.0
+144
View File
@@ -0,0 +1,144 @@
---
status: accepted
contact: eavanvalkenburg
date: 2026-06-11
deciders: eavanvalkenburg
---
# Python minimal hosting core and pluggable channels
## Context and Problem Statement
Agent Framework has several protocol-specific hosting surfaces. App authors who want one agent or workflow on multiple protocols must compose servers, routes, middleware, session handling, and lifecycle code by hand.
We will introduce a small Python hosting core that owns the common server shape and leaves protocol details inside channel packages. The first public contract must be intentionally narrow so Python can ship a base contract before adding identity linking, proactive delivery, or multicast behavior. Other language implementations may reuse the same conceptual boundary, but this ADR records the Python decision.
## Decision Drivers
- Keep the first host easy to explain: one app, one hostable target, one or more channels.
- Reuse Agent Framework's existing agent, workflow, session, history, and checkpoint primitives.
- Let channel packages own protocol parsing, protocol responses, authentication details, and native command surfaces.
- Make session continuity explicit through a channel-supplied `ChannelSession(isolation_key=...)`.
- Avoid approving cross-channel identity and delivery semantics before their safety model is reviewed.
## Considered Options
1. Keep only protocol-specific hosts.
2. Ship a large hosting core with identity linking, authorization, background delivery, active-channel routing, and multicast in v1.
3. Ship a minimal host/channel core now and track linking/multicast as follow-up work.
### Keep only protocol-specific hosts
- Good: no new abstraction or package surface.
- Neutral: each protocol can continue evolving independently.
- Bad: every multi-channel app still has to compose servers, lifecycle, and session handling by hand.
### Ship the large cross-channel host in v1
- Good: the richest cross-channel scenarios are available immediately.
- Neutral: the host becomes the natural place to demonstrate identity and delivery policy.
- Bad: v1 becomes a security-sensitive identity and delivery system before the safety model is reviewed.
### Ship the minimal core now
- Good: the host/channel boundary can be implemented, tested, and explained without solving linking and durable delivery at the same time.
- Neutral: apps that need richer behavior must build it locally or wait for ADR-0028 follow-up work.
- Bad: proactive delivery and multicast scenarios are deliberately absent from v1.
## Decision Outcome
Chosen option: **minimal host/channel core now, follow-up enhancements later**.
`AgentFrameworkHost` owns:
- one application object,
- one hostable target (`SupportsAgentRun` agent-compatible object or a `Workflow`), and
- one or more channels.
Channels own:
- contributed routes, middleware, commands, and lifecycle callbacks,
- protocol-native request parsing into `ChannelRequest`,
- protocol-native rendering of the originating response, and
- any channel-specific authentication or signature validation.
The host owns:
- route/lifecycle aggregation,
- invocation of the target,
- `ChannelSession(isolation_key=...)` to `AgentSession` resolution and caching,
- `reset_session(isolation_key=...)`,
- host-level middleware, including Foundry isolation middleware only when the Foundry hosting environment flag is present,
- invocation of per-channel hooks (`ChannelRunHook`, `ChannelResponseHook`, `ChannelStreamUpdateHook`), and
- workflow checkpoint wiring through an explicit `checkpoint_location`.
`ChannelIdentity`, when present, is request metadata only. In v1 it is not a linking, authorization, or delivery key.
### Trust boundary for `isolation_key`
The host treats `ChannelSession.isolation_key` as a session partition key, not as proof of identity. Channels or host middleware must authenticate and authorize any externally supplied value before passing it to the host. For example, a Responses caller must not be allowed to choose an arbitrary `previous_response_id` or header-derived key unless the platform or middleware has already established that the caller owns that conversation. The host deliberately does not infer that trust from the string itself.
### Hook ownership
Channels provide hook configuration and protocol-native context. The host invokes those hooks as part of the common invocation pipeline:
- `ChannelRunHook` runs after channel parsing and before target invocation.
- `ChannelResponseHook` runs after target invocation and before the originating channel serializes its response.
- `ChannelStreamUpdateHook` is applied by the host while the channel consumes streamed updates because streaming serialization is protocol-specific.
`ChannelStreamUpdateHook` is an update hook, not a final-response sanitizer. Channels that use it for redaction or filtering must also apply equivalent policy to any final response they render. Channels choose whether the response is streaming before run hooks execute.
This keeps hook call conventions centralized while leaving protocol payload parsing and response formatting in channel packages.
### State owned by v1
`state_dir` is limited to host-owned local files for reset-session aliases and workflow checkpoint path derivation. It does not store linked identities, active-channel state, response-routing state, continuation records, durable runner queues, or delivery attempts. Those storage concerns belong to ADR-0028.
## Non-goals for v1
The following are deliberately **not** part of the v1 contract:
- cross-channel identity linking (`IdentityLinker`, `local_identity_link`, or `agent-framework-hosting-entra`),
- identity allowlists or authorization policy (`IdentityAllowlist`, `AuthPolicy`),
- response routing beyond the originating channel (`ResponseTarget`, active channel, specific linked channel, `all_linked`),
- push or payload codecs (`ChannelPush`, `ChannelPushCodec`),
- background/continuation delivery,
- durable task runners (`DurableTaskRunner`, `InProcessTaskRunner`),
- retry/replay policy (`RetryPolicy`),
- fan-out, multicast, or all-linked delivery,
- confidentiality tiers and `LinkPolicy`, and
- a host-level multi-agent router.
These areas are follow-up enhancements covered by [ADR-0028](0028-hosting-linking-multicast-enhancements.md). They are not prerequisites for shipping or using the v1 host.
## Consequences
Positive:
- The host/channel model can be implemented and tested without designing a security-sensitive identity graph.
- Existing and new channel packages can share one Starlette app, middleware stack, lifecycle, and target invocation path.
- Session continuity is explicit and debuggable: two channels share history only when they produce the same `isolation_key`.
- Hook invocation is centralized in the host, so channels do not each invent the call convention.
Negative:
- Apps that need OAuth linking, allowlists, proactive messages, or multicast must continue to implement those behaviors outside the v1 host.
- Some richer cross-channel scenarios from the original design move to a separate decision and validation cycle.
- The host must document `isolation_key` trust clearly because it now provides the shared session boundary.
## Validation Gates
Before this ADR is accepted:
- A sample can expose one target on multiple channels with one `AgentFrameworkHost` and no handwritten Starlette route composition.
- Built-in channel tests prove that routes, commands, startup, and shutdown callbacks are contributed by channels and aggregated by the host.
- Session tests prove that identical `ChannelSession.isolation_key` values resolve to the same cached `AgentSession`, and `reset_session` rotates that mapping.
- Channel tests prove that each channel renders only its own originating response; there is no host-level push, multicast, or active-channel delivery path.
- Workflow tests or samples use an explicit `checkpoint_location`.
- Foundry isolation middleware is documented and covered by integration or contract tests, including the non-Foundry case where raw isolation headers are ignored.
- The v1 API and packages do not expose the removed symbols or packages listed in [Non-goals for v1](#non-goals-for-v1).
- The Python spec is updated to match this simplified contract and uses "public", "stable", or "released" terminology for Agent Framework APIs.
## More Information
- Follow-up linking and multicast ADR: [ADR-0028](0028-hosting-linking-multicast-enhancements.md)
@@ -0,0 +1,132 @@
---
status: proposed
contact: eavanvalkenburg
date: 2026-06-11
deciders: eavanvalkenburg
---
# Hosting linking and multicast enhancements
## Context and Problem Statement
[ADR-0027](0027-hosting-channels.md) defines the minimal v1 hosting core: originating-channel responses, explicit `ChannelSession.isolation_key`, and no host-level identity linking, push, multicast, background delivery, or durable runners.
This ADR tracks the richer cross-channel behaviors that were removed from v1. These enhancements are **follow-up work** and are **not prerequisites** for shipping, using, or stabilizing the v1 host/channel core.
## Decision Drivers
- Cross-channel continuity must not create accidental cross-user, cross-tenant, or cross-channel data leaks.
- Non-originating delivery must be observable, idempotent, retryable, and supportable.
- Protocol payloads must remain channel-native while still being safe to persist and replay.
- App authors need opt-in policy controls, not hidden defaults.
- The enhancement stack should layer on top of the v1 host without reshaping the minimal channel contract.
## Enhancement Areas
The follow-up design should cover these capabilities together because they share identity, storage, delivery, and replay concerns:
- **Cross-channel identity linking** — a user can connect multiple `ChannelIdentity` values to one channel-neutral `isolation_key`.
- **Authorization and allowlist policy** — channels or hosts can require verified identity, allow specific native identities or claims, and deny unknown callers.
- **Non-originating response delivery** — a run can respond somewhere other than the request's originating protocol when explicitly configured.
- **Active-channel routing** — delivery can target the most recently observed linked channel for an `isolation_key`.
- **Multicast / all-linked delivery** — delivery can fan out to every linked channel or a selected set.
- **Background runs and continuation tokens** — long-running requests can return immediately and complete later, with a polling/status fallback.
- **Durable delivery runners** — delivery work can survive process restarts and support dead-letter handling.
- **Retry and replay semantics** — delivery attempts are bounded, deduplicated, and safe to replay.
- **Payload serialization** — channel-specific payloads can be persisted, redacted, versioned, and reconstructed without losing protocol fidelity.
Candidate API names from the broader design (`IdentityLinker`, `IdentityAllowlist`, `AuthPolicy`, `ResponseTarget`, `ChannelPush`, `ChannelPushCodec`, `DurableTaskRunner`, `InProcessTaskRunner`, `RetryPolicy`, `LinkPolicy`) remain design vocabulary for this ADR. They are not approved v1 APIs.
## Considered Options
### Option A — Leave all behavior to applications
Applications implement linking, authorization, push, retry, and serialization independently.
- Good: the hosting core stays very small.
- Neutral: advanced apps can still build what they need.
- Bad: every app must solve the same security and delivery problems, likely inconsistently.
### Option B — Add the full enhancement stack to v1
The first host release includes linking, authorization, active channel, multicast, background runs, durable runners, and codecs.
- Good: the original cross-channel experience is available immediately.
- Neutral: samples can demonstrate rich end-to-end flows.
- Bad: v1 becomes security-sensitive, storage-heavy, and harder to stabilize.
### Option C — Layer opt-in enhancement packages after v1
Ship the minimal host first, then add linking, authorization, and delivery packages behind explicit configuration.
- Good: v1 remains simple while leaving room for a reviewed, supportable enhancement stack.
- Neutral: apps that need advanced delivery wait for follow-up packages.
- Bad: the first release does not satisfy proactive or all-linked scenarios.
### Option D — Build only platform-specific integrations
Implement linking and proactive delivery separately in Telegram, Activity Protocol, Discord, and future channels.
- Good: each package can match its protocol exactly.
- Neutral: some shared abstractions may emerge later.
- Bad: cross-channel behavior becomes fragmented and hard to reason about.
## Decision Outcome
Proposed direction: **Option C — layered opt-in enhancement packages after v1**.
The minimal host remains the foundation. Follow-up packages may add linking, authorization, delivery, and durable execution, but must be explicitly enabled and must pass the validation gates below before becoming part of the public contract.
## Safety Requirements
### Threat model
The design must account for:
- spoofed channel-native identities,
- stolen or replayed link challenges,
- cross-tenant or cross-confidentiality data leakage,
- unsolicited proactive messages,
- malicious payloads persisted for replay,
- denial-of-service through fan-out or retry storms, and
- privacy leakage through logs, metrics, or support tooling.
Required mitigations include verified identity claims where available, signed and expiring link challenges, explicit user consent, per-channel capability checks, default-deny policy options, tenant partitioning, and uninformative denial messages on shared channels.
### Idempotency and replay
Exactly-once delivery is not a realistic guarantee. The design must provide:
- stable run, continuation, and delivery-attempt identifiers,
- channel-level idempotency keys where protocols support them,
- bounded retry with jitter and explicit terminal states,
- replay windows and expiration,
- duplicate suppression for persisted attempts, and
- clear semantics for "delivered", "accepted by platform", and "observed by user".
### Storage
Enhancement storage must stay distinct from v1 `AgentSession` history and workflow checkpoints unless an implementation deliberately backs them with the same physical store.
Stored data should be schema-versioned, minimized, encrypted or otherwise protected as appropriate, and partitioned by tenant/project. Link records, continuation records, active-channel state, delivery attempts, dead letters, and serialized payloads need independent TTL and deletion policies.
### Observability and support
The design must include structured logs, traces, and metrics for link attempts, authorization decisions, delivery scheduling, retries, replay, and dead-letter outcomes. Logs must avoid message content and sensitive identity claims by default. Operators need a way to inspect, revoke, replay, or purge stuck records safely.
## Validation Gates
Before these enhancements are accepted:
- A reviewed threat model covers identity linking, authorization, non-originating delivery, multicast, and replay.
- Cross-channel linking tests prove a verified identity can link two channels and that unlink/deny paths do not leak information.
- Authorization tests cover native-id allowlists, verified-claim allowlists, default-deny behavior, and misconfiguration failures.
- Delivery tests cover originating-only, specific-channel, active-channel, selected-channel, and all-linked routing.
- Background/continuation tests cover polling fallback, cancellation or expiration, process restart, retry, and dead-letter behavior.
- Codec tests prove payloads are versioned, redacted where needed, backward compatible, and rejected safely when unknown.
- Multicast tests prove fan-out is bounded, independently retried, and idempotent per destination.
- Observability tests or manual validation prove support operators can correlate a request to delivery attempts without exposing sensitive content.
## Relationship to ADR-0027
ADR-0027 remains valid without any of these enhancements. This ADR extends the hosting model only after the safety, storage, and support requirements above are satisfied.
+320
View File
@@ -0,0 +1,320 @@
---
status: proposed
contact: eavanvalkenburg
date: 2026-06-11
deciders: eavanvalkenburg
---
# Python hosting core and pluggable channels
## Scope
This specification is the Python implementation plan for [ADR-0027](../decisions/0027-hosting-channels.md). It documents the simplified v1 host/channel contract only.
The v1 contract is:
- `AgentFrameworkHost` owns one Starlette app, one hostable target, and one or more channels.
- A hostable target is either a `SupportsAgentRun`-compatible agent or a `Workflow`.
- Channels contribute routes, middleware, commands, and lifecycle callbacks.
- Channels parse protocol-native input into `ChannelRequest`.
- Channels render their own originating response.
- Session continuity is explicit: a channel supplies `ChannelSession(isolation_key=...)`, and the host resolves/caches an `AgentSession` for that key.
- The host invokes `ChannelRunHook` and `ChannelResponseHook`; channels provide hook configuration and protocol context.
The host does not link identities, route responses to other channels, run background continuations, or multicast in v1. Those enhancements are tracked in [ADR-0028](../decisions/0028-hosting-linking-multicast-enhancements.md).
## Goals
- Let an app expose one agent or workflow on multiple protocols without handwritten Starlette composition.
- Keep protocol parsing and response formatting inside channel packages.
- Provide one session-resolution path shared by all channels.
- Keep the channel authoring surface small enough for new channels to implement.
- Preserve full-fidelity agent and workflow results until a channel decides how to render them.
## Non-goals for v1
The following are removed from the v1 implementation pass:
- `IdentityLinker`, `IdentityAllowlist`, `AuthPolicy`, and `LinkPolicy`
- `ResponseTarget`, active-channel routing, `all_linked`, fan-out, and multicast
- `ChannelPush` and `ChannelPushCodec`
- `DurableTaskRunner`, `InProcessTaskRunner`, and `RetryPolicy`
- continuation tokens and background delivery
- confidentiality tiers
- `agent-framework-hosting-entra`
- `local_identity_link`
These are follow-up design topics, not hidden requirements of the v1 host.
## Packages
| Package | Import surface | Contents |
|---|---|---|
| `agent-framework-hosting` | `agent_framework_hosting` | `AgentFrameworkHost`, channel protocols, key request/result types, hooks, `reset_session`, state-path helpers. |
| `agent-framework-hosting-responses` | `agent_framework_hosting_responses` | `ResponsesChannel`. |
| `agent-framework-hosting-invocations` | `agent_framework_hosting_invocations` | `InvocationsChannel`. |
| `agent-framework-hosting-telegram` | `agent_framework_hosting_telegram` | `TelegramChannel` and Telegram command helpers. |
| `agent-framework-hosting-activity-protocol` | `agent_framework_hosting_activity_protocol` | `ActivityProtocolChannel` for Activity Protocol over Azure Bot Service. |
| `agent-framework-hosting-discord` | `agent_framework_hosting_discord` | `DiscordChannel` and Discord command/interaction helpers. |
| `agent-framework-foundry-hosting` | `agent_framework.foundry_hosting` | Foundry isolation middleware and Foundry-backed hosting helpers usable with the v1 host. |
Channel packages may depend on their native SDKs. The core hosting package should not depend on channel SDKs or on top-level legacy protocol hosts.
## Key Types
### `AgentFrameworkHost`
The host constructor accepts:
- `target`: one `SupportsAgentRun`-compatible object or one `Workflow`
- `channels`: one or more `Channel` instances
- optional Starlette middleware
- optional `state_dir`
- optional workflow `checkpoint_location`
The host exposes:
- `app`: the canonical Starlette ASGI application
- `serve(...)`: a convenience wrapper for local serving
- `reset_session(isolation_key: str)`: rotate the cached `AgentSession` for a host-tracked conversation
`state_dir` is narrowed to v1 host-owned local files only:
- session aliases (`isolation_key` to current `AgentSession` id), and
- workflow checkpoint paths when the app chooses the host-provided file layout.
It is not a store for identity links, continuations, active-channel state, delivery attempts, or multicast payloads.
Externally supplied isolation keys are trusted only after the channel or host middleware has authenticated and authorized the caller. The host uses `isolation_key` as a partition key; the string itself is not proof of identity or ownership.
### `Channel`
A channel implements a small protocol:
- declare a stable channel id/name,
- contribute routes, middleware, commands, and lifecycle callbacks,
- parse inbound protocol data into `ChannelRequest`,
- call the host through `ChannelContext.run(...)` or `ChannelContext.run_stream(...)`, and
- serialize the returned result to the originating protocol response.
Channels own protocol authentication, signature validation, native command registration, and protocol-specific error bodies.
### `ChannelContribution`
`ChannelContribution` is the channel's host-facing contribution:
- Starlette routes and optional middleware,
- native command descriptors,
- startup and shutdown callbacks, and
- any channel-local metadata needed by the package.
The host aggregates contributions but does not interpret protocol payloads.
### `ChannelRequest`
`ChannelRequest` is the host-neutral request envelope produced by a channel. It carries:
- target input,
- optional `ChannelSession`,
- optional `ChannelIdentity`,
- options and attributes produced by the channel, and
- request metadata useful to hooks and context providers.
The host may pass attributes through to context providers and middleware. Channels should treat attributes as a documented extension bag, not as a cross-channel delivery contract.
### `ChannelSession`
`ChannelSession(isolation_key=...)` is the only v1 session-continuity mechanism.
When a request contains an isolation key:
1. The host looks up or creates the cached `AgentSession` for that key.
2. The target runs with that `AgentSession` when the target is an agent.
3. `reset_session(isolation_key)` rotates the alias so the next request starts a new conversation.
If two channels produce the same isolation key on the same host, they share the same cached session. If they produce different keys, they do not share session state.
### `ChannelIdentity`
`ChannelIdentity` is optional request metadata such as channel id, native user id, tenant id, claims, or display attributes.
In v1, `ChannelIdentity` does not link channels, authorize callers, select delivery destinations, or imply that two identities should share an `AgentSession`. A channel that wants shared history must still produce the same `ChannelSession.isolation_key`.
### Hooks
Hooks are optional and channel-owned:
- `ChannelRunHook`: runs after channel parsing and before host invocation; returns the `ChannelRequest` to execute.
- `ChannelResponseHook`: runs after target completion and before the originating channel renders a one-shot response.
- `ChannelStreamUpdateHook`: the host applies it to streamed updates before the originating channel serializes the stream.
Common uses include adapting chat text into workflow inputs, enforcing deployment-specific options, flattening rich output for text-only protocols, or filtering streamed updates for a protocol. Stream update hooks are update-only; they do not automatically sanitize `get_final_response()` output. Channels choose their response transport from the parsed protocol request before invoking run hooks.
### `HostedRunResult`
`HostedRunResult[T]` wraps the target's full-fidelity result plus the resolved `AgentSession | None`.
- Agent targets produce `HostedRunResult[AgentResponse]`.
- Workflow targets produce `HostedRunResult[WorkflowRunResult]`.
The host does not flatten, filter, or translate the result. Each channel decides how much of the result its protocol can carry.
## Host Behavior
1. `AgentFrameworkHost` builds one Starlette app and asks each channel for its contribution.
2. A channel route receives a protocol-native request.
3. The channel validates/parses the native payload and creates `ChannelRequest`.
4. The channel passes the request, optional `ChannelRunHook`, and protocol-native context to the host.
5. The host invokes `ChannelRunHook`, if configured, and receives the prepared request.
6. The host resolves an `AgentSession` from `ChannelSession.isolation_key` when present.
7. The host invokes the agent or workflow target.
8. The host wraps the result in `HostedRunResult` or the streaming equivalent.
9. The host invokes `ChannelResponseHook`, if configured, for non-streaming/final response shaping.
10. The host applies stream update hooks while the channel consumes streams; the channel renders the originating protocol response.
There is no host-level route from one channel's request to another channel's response in v1.
## Workflow Checkpoints
Workflow checkpointing is explicit. Apps either configure checkpoint storage on the workflow itself or pass a `checkpoint_location` to the host so the workflow dispatch path can use the intended file location.
`state_dir` may provide a conventional location for workflow checkpoint files, but checkpointing is still opt-in and separate from agent session history. Checkpoints are workflow-runtime state, not channel state and not identity-link state.
## Foundry Isolation Middleware
V1 keeps Foundry isolation as middleware rather than as a channel-linking feature.
The middleware is installed only when the Foundry hosting environment flag is present. In that environment it reads Foundry-provided isolation values at the trusted hosting boundary, exposes them as read-only request context for Foundry-aware history or memory providers, and rejects unsafe session resumes when the live isolation context does not match persisted session context. Outside Foundry, raw isolation headers are ignored unless an app supplies its own trusted middleware.
This middleware does not create cross-channel identity links and does not authorize non-Foundry channels.
## Current Channels
### Responses
`ResponsesChannel` exposes the OpenAI-compatible Responses API shape. It maps request body fields such as input, options, and conversation identifiers into `ChannelRequest`, and it renders Responses-compatible one-shot or streaming responses.
Responses session continuity uses a channel-selected `isolation_key`, commonly derived from a response/conversation id, caller-provided session id, Foundry isolation context, or deployment-specific request metadata.
### Invocations
`InvocationsChannel` exposes an invocation endpoint for server-side callers and tools. It maps the request body into `ChannelRequest` and renders the invocation result on the same HTTP response.
Invocations is useful for typed workflow inputs because a `ChannelRunHook` can translate the request body into the workflow's expected input type.
### Telegram
`TelegramChannel` supports webhook or polling transport, native command registration, and message rendering back to the originating Telegram chat.
The channel chooses a default `isolation_key` from Telegram-native data such as chat id, user id, or a configured user/chat scope. A `/new` or equivalent command may call `reset_session` for that isolation key.
### Activity Protocol
`ActivityChannel` supports Activity Protocol requests, typically through Azure Bot Service for Teams, Web Chat, and other Bot Framework-fronted surfaces.
The channel maps incoming `Activity` objects to `ChannelRequest` and renders a reply activity to the originating conversation. Proactive Activity delivery, active-channel routing, and all-linked fan-out are not v1 host semantics.
### Discord
`DiscordChannel` supports Discord messages, slash commands, and interactions as channel-native input.
The channel maps Discord-native user, guild, channel, thread, and interaction data into `ChannelRequest` metadata and a configured `ChannelSession.isolation_key`. It renders the result to the originating Discord response path.
## High-level Samples
### One agent on Responses
```python
host = AgentFrameworkHost(
target=agent,
channels=[ResponsesChannel()],
)
app = host.app
```
### One agent on multiple channels
```python
host = AgentFrameworkHost(
target=agent,
channels=[
ResponsesChannel(),
InvocationsChannel(),
TelegramChannel(bot_token=os.environ["TELEGRAM_BOT_TOKEN"]),
],
)
host.serve(host="localhost", port=8000)
```
The host owns one Starlette app. Each channel contributes its own routes and renders its own response.
### Adapting a request before execution
```python
from dataclasses import replace
def enforce_options(request: ChannelRequest) -> ChannelRequest:
options = dict(request.options or {})
options["temperature"] = 0
return replace(request, options=options)
host = AgentFrameworkHost(
target=agent,
channels=[ResponsesChannel(run_hook=enforce_options)],
)
```
### Workflow with explicit checkpoints
```python
host = AgentFrameworkHost(
target=workflow,
channels=[InvocationsChannel(run_hook=adapt_to_workflow_input)],
checkpoint_location=Path("./.af-hosting/workflow_checkpoints"),
)
```
The hook adapts channel-native input to the workflow's typed input. Checkpoints use the explicit workflow checkpoint location, not identity-link or delivery storage.
### Message channel reset command
```python
async def new_chat(context):
if context.request.session is not None:
await context.host.reset_session(context.request.session.isolation_key)
await context.reply("Started a new conversation.")
```
Telegram, Activity Protocol, and Discord can expose equivalent native commands when their protocols support them.
## Follow-up Enhancements
See [ADR-0028](../decisions/0028-hosting-linking-multicast-enhancements.md) for the deferred design covering:
- cross-channel identity linking,
- authorization and allowlists,
- non-originating response delivery,
- active-channel routing,
- multicast and all-linked delivery,
- background runs and continuation tokens,
- durable delivery runners,
- retry/replay semantics, and
- payload serialization.
Those enhancements must layer on top of this v1 contract without requiring v1 users to adopt them.
## Validation Gates
The Python implementation should be considered complete when:
- a sample uses one `AgentFrameworkHost` with multiple channels and no manual Starlette route composition,
- each current channel has contract tests for route contribution, lifecycle, request parsing, hooks, and originating response rendering,
- session tests prove shared `isolation_key` values share an `AgentSession` and `reset_session` rotates it,
- workflow tests or samples use explicit `checkpoint_location`,
- Foundry isolation middleware is covered by integration or contract tests,
- no v1 package exposes the removed linking, multicast, durable-runner, or continuation APIs, and
- this spec and ADR-0027 remain aligned.
+4 -4
View File
@@ -12,7 +12,7 @@
<ItemGroup>
<!-- Aspire.* -->
<PackageVersion Include="Anthropic" Version="12.20.0" />
<PackageVersion Include="Anthropic.Foundry" Version="0.5.0" />
<PackageVersion Include="Anthropic.Foundry" Version="0.6.0" />
<PackageVersion Include="Aspire.Hosting" Version="$(AspireAppHostSdkVersion)" />
<PackageVersion Include="Aspire.Azure.AI.OpenAI" Version="13.0.0-preview.1.25560.3" />
<PackageVersion Include="Aspire.Azure.AI.Inference" Version="13.1.0-preview.1.25616.3" />
@@ -27,10 +27,10 @@
<PackageVersion Include="Azure.AI.AgentServer.Invocations" Version="1.0.0-beta.4" />
<PackageVersion Include="Azure.AI.AgentServer.Responses" Version="1.0.0-beta.5" />
<PackageVersion Include="Azure.Search.Documents" Version="12.0.0" />
<PackageVersion Include="Azure.AI.Projects" Version="2.1.0-beta.2" />
<PackageVersion Include="Azure.AI.Projects" Version="2.1.0-beta.3" />
<PackageVersion Include="Azure.AI.Agents.Persistent" Version="1.2.0-beta.10" />
<PackageVersion Include="Azure.AI.OpenAI" Version="2.9.0-beta.1" />
<PackageVersion Include="Azure.Core" Version="1.56.0" />
<PackageVersion Include="Azure.Core" Version="1.57.0" />
<PackageVersion Include="Azure.Identity" Version="1.21.0" />
<PackageVersion Include="DotNetEnv" Version="3.1.1" />
<PackageVersion Include="Azure.Monitor.OpenTelemetry.Exporter" Version="1.5.0" />
@@ -45,7 +45,7 @@
<PackageVersion Include="Microsoft.Bcl.AsyncInterfaces" Version="10.0.8" />
<PackageVersion Include="Microsoft.Bcl.HashCode" Version="6.0.0" />
<PackageVersion Include="Microsoft.Bcl.Memory" Version="10.0.5" />
<PackageVersion Include="System.ClientModel" Version="1.12.0" />
<PackageVersion Include="System.ClientModel" Version="1.13.0" />
<PackageVersion Include="System.CodeDom" Version="10.0.0" />
<PackageVersion Include="System.Collections.Immutable" Version="10.0.1" />
<PackageVersion Include="System.CommandLine" Version="2.0.0-rc.2.25502.107" />
+6
View File
@@ -331,6 +331,9 @@
<Folder Name="/Samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalTools/">
<Project Path="samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalTools/HostedLocalTools.csproj" />
</Folder>
<Folder Name="/Samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalCodeAct/">
<Project Path="samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalCodeAct/HostedLocalCodeAct.csproj" />
</Folder>
<Folder Name="/Samples/04-hosting/FoundryHostedAgents/responses/Hosted-McpTools/">
<Project Path="samples/04-hosting/FoundryHostedAgents/responses/Hosted-McpTools/HostedMcpTools.csproj" />
</Folder>
@@ -408,6 +411,7 @@
<Folder Name="/Samples/05-end-to-end/Evaluation/">
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_ConversationSplits/Evaluation_ConversationSplits.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_FoundryQuality/Evaluation_FoundryQuality.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_FoundryRubric/Evaluation_FoundryRubric.csproj" />
<Project Path="samples/05-end-to-end/Evaluation/Evaluation_MixedProviders/Evaluation_MixedProviders.csproj" />
</Folder>
<Folder Name="/Samples/05-end-to-end/A2AClientServer/">
@@ -616,6 +620,7 @@
<Project Path="src/Microsoft.Agents.AI.Hosting.AspNetCore/Microsoft.Agents.AI.Hosting.AspNetCore.csproj" />
<Project Path="src/Microsoft.Agents.AI.Hosting/Microsoft.Agents.AI.Hosting.csproj" />
<Project Path="src/Microsoft.Agents.AI.Hyperlight/Microsoft.Agents.AI.Hyperlight.csproj" />
<Project Path="src/Microsoft.Agents.AI.LocalCodeAct/Microsoft.Agents.AI.LocalCodeAct.csproj" />
<Project Path="src/Microsoft.Agents.AI.Mcp/Microsoft.Agents.AI.Mcp.csproj" />
<Project Path="src/Microsoft.Agents.AI.Mem0/Microsoft.Agents.AI.Mem0.csproj" />
<Project Path="src/Microsoft.Agents.AI.OpenAI/Microsoft.Agents.AI.OpenAI.csproj" />
@@ -671,6 +676,7 @@
<Project Path="tests/Microsoft.Agents.AI.Hosting.OpenAI.UnitTests/Microsoft.Agents.AI.Hosting.OpenAI.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.Hosting.UnitTests/Microsoft.Agents.AI.Hosting.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.Hyperlight.UnitTests/Microsoft.Agents.AI.Hyperlight.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.LocalCodeAct.UnitTests/Microsoft.Agents.AI.LocalCodeAct.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.Mcp.UnitTests/Microsoft.Agents.AI.Mcp.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.Mem0.UnitTests/Microsoft.Agents.AI.Mem0.UnitTests.csproj" />
<Project Path="tests/Microsoft.Agents.AI.OpenAI.UnitTests/Microsoft.Agents.AI.OpenAI.UnitTests.csproj" />
+10 -10
View File
@@ -26,8 +26,8 @@ internal static class GetStartedSamples
{
Name = "01_hello_agent",
ProjectPath = "samples/01-get-started/01_hello_agent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
RequiredEnvironmentVariables = ["FOUNDRY_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["FOUNDRY_MODEL"],
ExpectedOutputDescription =
[
"The output should contain a joke about a pirate.",
@@ -40,8 +40,8 @@ internal static class GetStartedSamples
{
Name = "02_add_tools",
ProjectPath = "samples/01-get-started/02_add_tools",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
RequiredEnvironmentVariables = ["FOUNDRY_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["FOUNDRY_MODEL"],
MustContain = [],
ExpectedOutputDescription =
[
@@ -56,8 +56,8 @@ internal static class GetStartedSamples
{
Name = "03_multi_turn",
ProjectPath = "samples/01-get-started/03_multi_turn",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
RequiredEnvironmentVariables = ["FOUNDRY_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["FOUNDRY_MODEL"],
ExpectedOutputDescription =
[
"The output should contain a joke about a pirate.",
@@ -71,8 +71,8 @@ internal static class GetStartedSamples
{
Name = "04_memory",
ProjectPath = "samples/01-get-started/04_memory",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
RequiredEnvironmentVariables = ["FOUNDRY_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["FOUNDRY_MODEL"],
MustContain =
[
">> Use session with blank memory",
@@ -97,8 +97,8 @@ internal static class GetStartedSamples
{
Name = "06_host_your_agent",
ProjectPath = "samples/01-get-started/06_host_your_agent",
RequiredEnvironmentVariables = ["AZURE_OPENAI_ENDPOINT"],
OptionalEnvironmentVariables = ["AZURE_OPENAI_DEPLOYMENT_NAME"],
RequiredEnvironmentVariables = ["FOUNDRY_PROJECT_ENDPOINT"],
OptionalEnvironmentVariables = ["FOUNDRY_MODEL"],
SkipReason = "Requires Azure Functions Core Tools runtime and starts a web server.",
},
];
@@ -9,13 +9,12 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -1,23 +1,19 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample shows how to create and use a simple AI agent with Azure OpenAI as the backend.
// This sample shows how to create and use a simple AI agent with AIProjectClient as the backend.
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using OpenAI.Chat;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(
new Uri(endpoint),
new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(instructions: "You are good at telling jokes.", name: "Joker");
AIAgent agent = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
.AsAIAgent(model: model, instructions: "You are good at telling jokes.", name: "Joker");
// Invoke the agent and output the text result.
Console.WriteLine(await agent.RunAsync("Tell me a joke about a pirate."));
@@ -9,13 +9,12 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -1,31 +1,27 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample demonstrates how to use a ChatClientAgent with function tools.
// It shows both non-streaming and streaming agent interactions using menu-related tools.
// This sample demonstrates how to use an AIProjectClient agent with function tools.
// It shows both non-streaming and streaming agent interactions using weather tools.
using System.ComponentModel;
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using OpenAI.Chat;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
[Description("Get the weather for a given location.")]
static string GetWeather([Description("The location to get the weather for.")] string location)
=> $"The weather in {location} is cloudy with a high of 15°C.";
// Create the chat client and agent, and provide the function tool to the agent.
// Create the agent and provide the function tool to the agent.
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(
new Uri(endpoint),
new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(instructions: "You are a helpful assistant", tools: [AIFunctionFactory.Create(GetWeather)]);
AIAgent agent = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
.AsAIAgent(model: model, instructions: "You are a helpful assistant", tools: [AIFunctionFactory.Create(GetWeather)]);
// Non-streaming agent interaction with function tools.
Console.WriteLine(await agent.RunAsync("What is the weather like in Amsterdam?"));
@@ -9,13 +9,12 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -2,22 +2,18 @@
// This sample shows how to create and use a simple AI agent with a multi-turn conversation.
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using OpenAI.Chat;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(
new Uri(endpoint),
new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(instructions: "You are good at telling jokes.", name: "Joker");
AIAgent agent = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
.AsAIAgent(model: model, instructions: "You are good at telling jokes.", name: "Joker");
// Invoke the agent with a multi-turn conversation, where the context is preserved in the session object.
AgentSession session = await agent.CreateSessionAsync();
@@ -9,13 +9,12 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="Microsoft.Extensions.AI.OpenAI" />
<PackageReference Include="Microsoft.Extensions.AI.Abstractions" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -8,23 +8,26 @@
using System.Text;
using System.Text.Json;
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Extensions.AI;
using OpenAI.Chat;
using SampleApp;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT") ?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT") ?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
ChatClient chatClient = new AzureOpenAIClient(
new Uri(endpoint),
new DefaultAzureCredential())
.GetChatClient(deploymentName);
var projectClient = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential());
// Get the underlying IChatClient to use for the memory component.
// The memory provider needs direct IChatClient access for structured extraction.
IChatClient chatClient = projectClient
.AsAIAgent(new ChatClientAgentOptions { ChatOptions = new() { ModelId = model } })
.GetService<IChatClient>()
?? throw new InvalidOperationException("Could not retrieve IChatClient from AIProjectClient agent.");
// Create the agent and provide a factory to add our custom memory component to
// all sessions created by the agent. Here each new memory component will have its own
@@ -36,7 +39,7 @@ ChatClient chatClient = new AzureOpenAIClient(
AIAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()
{
ChatOptions = new() { Instructions = "You are a friendly assistant. Always address the user by their name." },
AIContextProviders = [new UserInfoMemory(chatClient.AsIChatClient())]
AIContextProviders = [new UserInfoMemory(chatClient)]
});
// Create a new session for the conversation.
@@ -21,11 +21,10 @@
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" />
</ItemGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.OpenAI" />
<PackageReference Include="Azure.Identity" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Hosting.AzureFunctions\Microsoft.Agents.AI.Hosting.AzureFunctions.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.OpenAI\Microsoft.Agents.AI.OpenAI.csproj" />
<ProjectReference Include="..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -4,36 +4,32 @@
//
// Prerequisites:
// - Azure Functions Core Tools
// - Azure OpenAI resource
// - Foundry project endpoint and credentials
//
// Environment variables:
// AZURE_OPENAI_ENDPOINT
// AZURE_OPENAI_DEPLOYMENT_NAME (defaults to "gpt-5.4-mini")
// FOUNDRY_PROJECT_ENDPOINT
// FOUNDRY_MODEL (defaults to "gpt-5.4-mini")
//
// Run with: func start
// Then call: POST http://localhost:7071/api/agents/HostedAgent/run
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Hosting.AzureFunctions;
using Microsoft.Azure.Functions.Worker.Builder;
using Microsoft.Extensions.Hosting;
using OpenAI.Chat;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
// Set up an AI agent following the standard Microsoft Agent Framework pattern.
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(
instructions: "You are a helpful assistant hosted in Azure Functions.",
name: "HostedAgent");
AIAgent agent = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
.AsAIAgent(model: model, instructions: "You are a helpful assistant hosted in Azure Functions.", name: "HostedAgent");
// Configure the function app to host the AI agent.
// This will automatically generate HTTP API endpoints for the agent.
@@ -56,14 +56,13 @@ try
// Inspect memory search results if available in raw response items.
foreach (var message in response.Messages)
{
if (message.RawRepresentation is MemorySearchToolCallResponseItem memorySearchResult)
if (message.RawRepresentation is MemorySearchToolCall memorySearchResult)
{
Console.WriteLine($"Memory Search Status: {memorySearchResult.Status}");
Console.WriteLine($"Memory Search Results Count: {memorySearchResult.Results.Count}");
Console.WriteLine($"Memory Search Results Count: {memorySearchResult.Memories.Count}");
foreach (var result in memorySearchResult.Results)
foreach (var memoryItem in memorySearchResult.Memories)
{
var memoryItem = result.MemoryItem;
Console.WriteLine($" - Memory ID: {memoryItem.MemoryId}");
Console.WriteLine($" Scope: {memoryItem.Scope}");
Console.WriteLine($" Content: {memoryItem.Content}");
@@ -31,3 +31,4 @@ dotnet run --project .\Evaluation_ExpectedOutputs
- [Evaluation_SimpleEval](../Evaluation_SimpleEval/) — Simplest evaluation with built-in and custom checks
- [Evaluation_FoundryQuality](../../../05-end-to-end/Evaluation/Evaluation_FoundryQuality/) — Cloud-based quality evaluation with Foundry evaluators
- [Evaluation_FoundryRubric](../../../05-end-to-end/Evaluation/Evaluation_FoundryRubric/) — Rubric (adaptive) evaluators with per-dimension scores
@@ -26,4 +26,5 @@ dotnet run --project .\Evaluation_Multimodal
- [Evaluation_SimpleEval](../Evaluation_SimpleEval/) — Simplest evaluation with built-in checks and `agent.EvaluateAsync()`
- [Evaluation_FoundryQuality](../../../05-end-to-end/Evaluation/Evaluation_FoundryQuality/) — Cloud-based quality evaluation with Foundry evaluators
- [Evaluation_FoundryRubric](../../../05-end-to-end/Evaluation/Evaluation_FoundryRubric/) — Rubric (adaptive) evaluators with per-dimension scores
- [Evaluation_ConversationSplits](../../../05-end-to-end/Evaluation/Evaluation_ConversationSplits/) — Multi-turn conversation split strategies
@@ -6,22 +6,22 @@ using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>BackgroundAgents_*</c> tool calls with human-readable details
/// Formats <c>background_agents_*</c> tool calls with human-readable details
/// for task start, continue, wait, and result retrieval operations.
/// </summary>
public sealed class BackgroundAgentToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("BackgroundAgents_", StringComparison.Ordinal);
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("background_agents_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"BackgroundAgents_StartTask" => FormatStartBackgroundTask(call),
"BackgroundAgents_WaitForFirstCompletion" => FormatIdList(call, "taskIds", "Wait for"),
"BackgroundAgents_GetTaskResults" => FormatSingleId(call, "taskId"),
"BackgroundAgents_ContinueTask" => FormatContinueTask(call),
"BackgroundAgents_ClearCompletedTask" => FormatSingleId(call, "taskId"),
"background_agents_start_task" => FormatStartBackgroundTask(call),
"background_agents_wait_for_first_completion" => FormatIdList(call, "taskIds", "Wait for"),
"background_agents_get_task_results" => FormatSingleId(call, "taskId"),
"background_agents_continue_task" => FormatContinueTask(call),
"background_agents_clear_completed_task" => FormatSingleId(call, "taskId"),
_ => null,
};
@@ -5,21 +5,21 @@ using Microsoft.Extensions.AI;
namespace Harness.Shared.Console.ToolFormatters;
/// <summary>
/// Formats <c>FileMemory_*</c> tool calls, showing file names and search patterns
/// Formats <c>file_memory_*</c> tool calls, showing file names and search patterns
/// with tree-view corners for save operations.
/// </summary>
public sealed class FileMemoryToolFormatter : ToolCallFormatter
{
/// <inheritdoc/>
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("FileMemory_", StringComparison.Ordinal);
public override bool CanFormat(FunctionCallContent call) => call.Name.StartsWith("file_memory_", StringComparison.Ordinal);
/// <inheritdoc/>
public override string? FormatDetail(FunctionCallContent call) => call.Name switch
{
"FileMemory_SaveFile" => FormatSaveFile(call),
"FileMemory_ReadFile" => FormatStringArg(call, "fileName"),
"FileMemory_DeleteFile" => FormatStringArg(call, "fileName"),
"FileMemory_SearchFiles" => FormatSearchFiles(call),
"file_memory_save_file" => FormatSaveFile(call),
"file_memory_read_file" => FormatStringArg(call, "fileName"),
"file_memory_delete_file" => FormatStringArg(call, "fileName"),
"file_memory_search_files" => FormatSearchFiles(call),
_ => null,
};
@@ -89,6 +89,13 @@ AIAgent agent =
OpenTelemetrySourceName = TracingSourceName, // Use our custom source name so spans are captured by the TracerProvider above.
FileMemoryStore = new FileSystemAgentFileStore( // Configure the file memory provider to store files in a local folder called "agent-files".
Path.Combine(AppContext.BaseDirectory, "agent-files")),
// The built in ModeProvider has two default modes: "plan" and "execute".
// Adding a loop evaluator so that in "execute" mode, the harness keeps re-invoking itself until every todo item is complete.
LoopEvaluators =
[
new TodoCompletionLoopEvaluator(new TodoCompletionLoopEvaluatorOptions { Modes = ["execute"] }),
],
LoopAgentOptions = new LoopAgentOptions { MaxIterations = 10 }, // Safety cap on the number of autonomous passes per turn.
ChatOptions = new ChatOptions
{
Instructions = instructions,
@@ -9,6 +9,7 @@ Key features showcased:
- **Web Search** — the agent can search the web for current information via `ResponseTool.CreateWebSearchTool()`
- **TodoProvider** — the agent creates and manages a todo list to track research questions
- **AgentModeProvider** — the agent switches between "plan" mode (breaking down the topic) and "execute" mode (answering each research question)
- **TodoCompletionLoopEvaluator** — in "execute" mode the agent loops automatically, re-invoking itself until every todo item is complete (capped by `LoopAgentOptions.MaxIterations`). The loop is scoped to "execute" mode, so "plan" mode stays interactive. The `HarnessAgent` wraps itself in a `LoopAgent` automatically whenever `LoopEvaluators` is supplied.
- **Interactive conversation** — you can review the agent's plan, provide feedback, and approve before execution begins
- **Streaming output** — responses are streamed token-by-token for a natural experience
- **`/todos` command** — view the current todo list at any time without invoking the agent
@@ -47,7 +48,7 @@ The sample starts an interactive conversation loop. You can:
1. **Enter a research topic** — the agent will analyze it and create a plan with todos
2. **Review and adjust** — provide feedback on the plan, ask for changes, or approve it
3. **Type `/todos`** — to see the current todo list at any time
4. **Watch execution** — once approved, tell the agent to proceed and it will work through each todo
4. **Watch execution** — once approved, the agent will switch to "execute" mode and process each todo autonomously until the whole plan is complete
5. **Type `exit`** — to end the session
The prompt and agent output are colored by the current mode: **cyan** during planning, **green** during execution.
@@ -56,9 +56,9 @@ AIAgent webSearchAgent =
OpenTelemetrySourceName = TracingSourceName,
DisableTodoProvider = true,
DisableAgentModeProvider = true,
DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session
DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory
DisableToolApproval = true, // If enabled, this allows don't-ask-again approval functionality.
DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session
DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory
DisableToolAutoApproval = true, // If true, this disables the don't-ask-again approval functionality.
ChatOptions = new ChatOptions
{
Instructions = "You are a web search assistant. When asked to find information, use the web search tool to look it up and return a concise, factual answer.",
@@ -106,9 +106,9 @@ AIAgent parentAgent =
OpenTelemetrySourceName = TracingSourceName,
DisableTodoProvider = true,
DisableAgentModeProvider = true,
DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session
DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory
DisableToolApproval = true, // If enabled, this allows don't-ask-again approval functionality.
DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session
DisableFileAccess = true, // If enabled, this would allow the agent to read/write files in a working directory
DisableToolAutoApproval = true, // If true, this disables the don't-ask-again approval functionality.
DisableWebSearch = true,
BackgroundAgents = [webSearchAgent],
ChatOptions = new ChatOptions
@@ -9,16 +9,16 @@ A parent agent receives a list of stock tickers and uses a web-search background
### Architecture
```
┌────────────────────────────────────────┐
│ StockPriceResearcher │
│ (Parent Agent) │
│ │
│ BackgroundAgentsProvider │
│ ├─ BackgroundAgents_StartTask │
│ ├─ BackgroundAgents_WaitFor...
│ ├─ BackgroundAgents_GetTaskResults │
│ └─ ... │
└───────────────────────────────────────┘
┌──────────────────────────────────────────────────
│ StockPriceResearcher
│ (Parent Agent)
│ BackgroundAgentsProvider
│ ├─ background_agents_start_task
│ ├─ background_agents_wait_for_first_completion
│ ├─ background_agents_get_task_results
│ └─ ...
└─────────────┬────────────────────────────────────┘
│ delegates to
┌─────────────────────────────────┐
@@ -79,6 +79,13 @@ AIAgent agent =
Description = "A data analyst assistant that reads, analyzes, and processes data files.",
OpenTelemetrySourceName = TracingSourceName,
FileAccessStore = new FileSystemAgentFileStore(Path.Combine(AppContext.BaseDirectory, "working")),
ToolApprovalAgentOptions = new ToolApprovalAgentOptions()
{
// The HarnessAgent's FileAccessProvider requires approval for all file access operations.
// Add an auto-approval rule to skip prompts for specific operations (e.g., read-only access).
// You can also supply your own rule to implement custom approval logic.
AutoApprovalRules = [FileAccessProvider.ReadOnlyToolsAutoApprovalRule]
},
DisableTodoProvider = true,
DisableAgentModeProvider = true,
DisableFileMemory = true, // If enabled, this would allow the agent to store memories as files in a directory associated with the current session
@@ -116,7 +116,7 @@ async Task TodoLoopAsync()
{
var todoProvider = context.Agent.GetService<TodoProvider>()
?? throw new InvalidOperationException("The agent did not expose a TodoProvider.");
var remaining = await todoProvider.GetRemainingTodosAsync(context.Session).ConfigureAwait(false);
var remaining = await todoProvider.GetRemainingTodosAsync(context.Session, cancellationToken).ConfigureAwait(false);
return remaining.Count > 0
? LoopEvaluation.Continue($"Not all todos are complete yet ({remaining.Count} remaining). Please complete the remaining todo items.")
: LoopEvaluation.Stop();
@@ -27,7 +27,7 @@ public static class Program
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var client = new AzureOpenAIClient(new Uri(endpoint), new AzureCliCredential()).GetChatClient(deploymentName).AsIChatClient();
Console.Write("Choose workflow type ('sequential', 'concurrent', 'handoffs', 'groupchat'): ");
Console.Write("Choose workflow type ('sequential', 'sequential-chain-only', 'concurrent', 'handoffs', 'groupchat'): ");
switch (Console.ReadLine())
{
case "sequential":
@@ -36,6 +36,14 @@ public static class Program
[new(ChatRole.User, "Hello, world!")]);
break;
case "sequential-chain-only":
await RunWorkflowAsync(
AgentWorkflowBuilder.BuildSequential(
chainOnlyAgentResponses: true,
from lang in (string[])["French", "Spanish", "English"] select GetTranslationAgent(lang, client)),
[new(ChatRole.User, "Hello, world!")]);
break;
case "concurrent":
await RunWorkflowAsync(
AgentWorkflowBuilder.BuildConcurrent(from lang in (string[])["French", "Spanish", "English"] select GetTranslationAgent(lang, client)),
@@ -20,7 +20,6 @@
#pragma warning disable AAIP001 // ProjectAgentSkills is experimental
using System.ClientModel;
using System.IO.Compression;
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.Core;
@@ -121,8 +120,8 @@ app.Run();
// ── Helpers ──────────────────────────────────────────────────────────────────
// Downloads each named skill from Foundry and extracts the ZIP archive into a
// separate subdirectory under the target directory.
// Downloads each named skill from Foundry into a separate subdirectory under the target directory.
// GetSkillContentAsync downloads the skill package and unzips it into the destination directory.
static async Task DownloadSkillsAsync(ProjectAgentSkills skillsClient, string[] skillNames, string targetDir)
{
if (Directory.Exists(targetDir))
@@ -135,56 +134,16 @@ static async Task DownloadSkillsAsync(ProjectAgentSkills skillsClient, string[]
foreach (string name in skillNames)
{
Console.WriteLine($"Downloading skill '{name}' from Foundry...");
BinaryData zipData = await skillsClient.DownloadSkillAsync(name);
string skillDir = Path.Combine(targetDir, name);
Directory.CreateDirectory(skillDir);
using var zipStream = zipData.ToStream();
using var archive = new ZipArchive(zipStream, ZipArchiveMode.Read);
SafeExtractZip(archive, skillDir);
await skillsClient.GetSkillContentAsync(name, skillDir);
if (!File.Exists(Path.Combine(skillDir, "SKILL.md")))
{
throw new InvalidOperationException(
$"Downloaded archive for '{name}' did not contain a SKILL.md at the root.");
}
}
}
// Extracts a ZIP archive into a destination directory, rejecting entries that would
// escape the target path (zip-slip guard).
static void SafeExtractZip(ZipArchive archive, string destinationDir)
{
string destRoot = Path.GetFullPath(destinationDir);
string destRootWithSep = Path.EndsInDirectorySeparator(destRoot)
? destRoot
: destRoot + Path.DirectorySeparatorChar;
// Use ordinal comparison on Unix (case-sensitive FS) and ordinal-ignore-case on Windows.
var comparison = OperatingSystem.IsWindows()
? StringComparison.OrdinalIgnoreCase
: StringComparison.Ordinal;
foreach (ZipArchiveEntry entry in archive.Entries)
{
string entryPath = Path.GetFullPath(Path.Combine(destRoot, entry.FullName));
if (!entryPath.StartsWith(destRootWithSep, comparison)
&& !string.Equals(entryPath, destRoot, comparison))
{
throw new InvalidOperationException(
$"Refusing to extract unsafe path '{entry.FullName}' outside of '{destRoot}'.");
}
if (string.IsNullOrEmpty(entry.Name))
{
// Directory entry — ensure it exists.
Directory.CreateDirectory(entryPath);
}
else
{
Directory.CreateDirectory(Path.GetDirectoryName(entryPath)!);
entry.ExtractToFile(entryPath, overwrite: true);
$"Downloaded skill '{name}' did not contain a SKILL.md at the root.");
}
}
}
@@ -211,8 +170,8 @@ static async Task EnsureSkillsProvisionedAsync(ProjectAgentSkills skillsClient,
catch (ClientResultException ex) when (ex.Status == 404)
{
Console.WriteLine($"Provisioning skill '{name}' from {skillPath}...");
AgentsSkill imported = await skillsClient.CreateSkillFromPackageAsync(skillPath);
Console.WriteLine($" Imported skill '{imported.Name}' (id={imported.SkillId}, has_blob={imported.HasBlob}).");
AgentsSkill imported = (await skillsClient.CreateSkillVersionFromFilesAsync(name, skillPath)).Value;
Console.WriteLine($" Imported skill '{imported.Name}' (id={imported.Id}, version={imported.LatestVersion}).");
}
}
}
@@ -42,7 +42,7 @@ The agent is hosted using the [Agent Framework](https://github.com/microsoft/age
## Prerequisites
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
### Required RBAC
@@ -7,7 +7,7 @@ This sample is the Azure AI Search counterpart to `Hosted-TextRag`. Where `Hoste
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- An Azure AI Search service ([create one](https://learn.microsoft.com/azure/search/search-create-service-portal))
- **A pre-provisioned search index** with the schema and content described in the next section
- Azure CLI logged in (`az login`)
@@ -5,7 +5,7 @@ A simple general-purpose AI assistant hosted as a Foundry Hosted Agent using the
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -16,7 +16,7 @@ Copy the template and fill in your project endpoint:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -20,7 +20,7 @@
// indirect prompt injection in an uploaded file.
//
// Required environment variables:
// FOUNDRY_PROJECT_ENDPOINT - Azure AI Foundry project endpoint
// FOUNDRY_PROJECT_ENDPOINT - Foundry project endpoint
// FOUNDRY_MODEL - Model deployment name (default: gpt-4o)
//
// Optional:
@@ -43,7 +43,7 @@ The end-to-end alpha-SDK round trip (client uploads via `AgentSessionFiles.Uploa
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -7,7 +7,7 @@ This is the **Foundry hosting** pattern — the agent's behavior is configured i
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a **registered agent** (created via Foundry UI, CLI, or API)
- A Foundry project with a **registered agent** (created via Foundry UI, CLI, or API)
- Azure CLI logged in (`az login`)
## Configuration
@@ -18,7 +18,7 @@ Copy the template and fill in your project endpoint:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -0,0 +1,6 @@
FOUNDRY_PROJECT_ENDPOINT=<your-azure-ai-project-endpoint>
ASPNETCORE_URLS=http://+:8088
ASPNETCORE_ENVIRONMENT=Development
FOUNDRY_MODEL=gpt-4o
AZURE_BEARER_TOKEN=DefaultAzureCredential
LOCAL_CODEACT_PYTHON=python3
@@ -0,0 +1,23 @@
# Use the official .NET 10.0 ASP.NET runtime as a parent image
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base
WORKDIR /app
# Install Python 3 so LocalCodeAct can spawn the embedded runner / validator.
RUN apt-get update \
&& apt-get install -y --no-install-recommends python3 \
&& rm -rf /var/lib/apt/lists/*
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY . .
RUN dotnet restore
RUN dotnet publish -c Release -o /app/publish
# Final stage
FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENV LOCAL_CODEACT_PYTHON=python3
ENTRYPOINT ["dotnet", "HostedLocalCodeAct.dll"]
@@ -0,0 +1,24 @@
# Dockerfile for contributors building from the agent-framework repository source.
#
# This project uses ProjectReference to the local Microsoft.Agents.AI.Foundry and
# Microsoft.Agents.AI.LocalCodeAct sources, which means a standard multi-stage
# Docker build cannot resolve dependencies outside this folder. Instead, pre-publish
# the app targeting the container runtime and copy the output into the container:
#
# dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out
# docker build -f Dockerfile.contributor -t hosted-local-codeact .
# docker run --rm -p 8088:8088 -e AGENT_NAME=hosted-local-codeact -e AZURE_BEARER_TOKEN=$AZURE_BEARER_TOKEN --env-file .env hosted-local-codeact
#
# For end-users consuming the NuGet package (not ProjectReference), use the standard
# Dockerfile which performs a full dotnet restore + publish inside the container.
FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final
WORKDIR /app
# Install Python 3 so LocalCodeAct can spawn the embedded runner / validator.
RUN apk add --no-cache python3
COPY out/ .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENV LOCAL_CODEACT_PYTHON=python3
ENTRYPOINT ["dotnet", "HostedLocalCodeAct.dll"]
@@ -0,0 +1,33 @@
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<CentralPackageTransitivePinningEnabled>false</CentralPackageTransitivePinningEnabled>
<NoWarn>$(NoWarn);</NoWarn>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Azure.AI.Projects" />
<PackageReference Include="Azure.Identity" />
<PackageReference Include="DotNetEnv" />
</ItemGroup>
<!-- For contributors: uses ProjectReference to build against local source -->
<ItemGroup>
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.Foundry.Hosting\Microsoft.Agents.AI.Foundry.Hosting.csproj" />
<ProjectReference Include="..\..\..\..\..\src\Microsoft.Agents.AI.LocalCodeAct\Microsoft.Agents.AI.LocalCodeAct.csproj" />
<ProjectReference Include="..\Hosted_Shared_Contributor_Setup\Hosted_Shared_Contributor_Setup.csproj" />
</ItemGroup>
<!-- For end-users: uncomment the PackageReference below and remove the ProjectReference above
<ItemGroup>
<PackageReference Include="Microsoft.Agents.AI.Foundry" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.Foundry.Hosting" Version="1.6.1-preview.260514.1" />
<PackageReference Include="Microsoft.Agents.AI.LocalCodeAct" Version="1.6.1-preview.260514.1" />
</ItemGroup>
-->
</Project>
@@ -0,0 +1,120 @@
// Copyright (c) Microsoft. All rights reserved.
// Hosted Local CodeAct sample. Wires Microsoft.Agents.AI.LocalCodeAct into a
// Foundry hosted agent. The model only sees a single `execute_code` tool;
// `compute` and `fetch_data` are registered as sandbox-only host tools that
// generated Python reaches via `await call_tool(...)`. This mirrors the Python
// `foundry_hosted_agent.py` sample for the local-codeact package.
//
// SECURITY: LocalCodeAct executes LLM-generated Python in the agent process.
// Only deploy this sample to an externally sandboxed environment such as a
// Foundry hosted-agent container.
using System.ComponentModel;
using Azure.AI.Projects;
using Azure.Core;
using Azure.Identity;
using DotNetEnv;
using Hosted_Shared_Contributor_Setup;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry.Hosting;
using Microsoft.Agents.AI.LocalCodeAct;
using Microsoft.Extensions.AI;
// Load .env file if present (for local development)
Env.TraversePath().Load();
string endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
string deploymentName = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-4o";
string pythonExecutable = Environment.GetEnvironmentVariable("LOCAL_CODEACT_PYTHON")
?? (OperatingSystem.IsWindows() ? "python.exe" : "python3");
TokenCredential credential = new ChainedTokenCredential(
new DevTemporaryTokenCredential(),
new DefaultAzureCredential());
// ── Sandbox-only tools (model never sees these directly) ─────────────────────
[Description("Perform a math operation: add, subtract, multiply, or divide.")]
static double Compute(
[Description("Operation: add, subtract, multiply, or divide.")] string operation,
[Description("First numeric operand.")] double a,
[Description("Second numeric operand.")] double b) => operation switch
{
"add" => a + b,
"subtract" => a - b,
"multiply" => a * b,
"divide" => b == 0 ? double.PositiveInfinity : a / b,
_ => throw new ArgumentException($"Unknown operation '{operation}'.", nameof(operation)),
};
[Description("Fetch records from a named simulated table (users or products).")]
static IReadOnlyList<IReadOnlyDictionary<string, object>> FetchData(
[Description("Name of the simulated table to query.")] string table)
{
Dictionary<string, IReadOnlyList<IReadOnlyDictionary<string, object>>> data = new()
{
["users"] =
[
new Dictionary<string, object> { ["id"] = 1, ["name"] = "Alice", ["role"] = "admin" },
new Dictionary<string, object> { ["id"] = 2, ["name"] = "Bob", ["role"] = "user" },
new Dictionary<string, object> { ["id"] = 3, ["name"] = "Charlie", ["role"] = "admin" },
],
["products"] =
[
new Dictionary<string, object> { ["id"] = 101, ["name"] = "Widget", ["price"] = 9.99 },
new Dictionary<string, object> { ["id"] = 102, ["name"] = "Gadget", ["price"] = 19.99 },
],
};
return data.TryGetValue(table, out var rows) ? rows : [];
}
// ── LocalCodeAct provider with sandbox-only host tools ───────────────────────
var codeActOptions = new LocalCodeActProviderOptions
{
Tools =
[
AIFunctionFactory.Create(Compute, name: "compute"),
AIFunctionFactory.Create(FetchData, name: "fetch_data"),
],
ExecutionLimits = new ProcessExecutionLimits { TimeoutSeconds = 5 },
};
var codeAct = new LocalCodeActProvider(pythonExecutable, codeActOptions);
// ── Build the hosted agent ───────────────────────────────────────────────────
AIAgent agent = new AIProjectClient(new Uri(endpoint), credential)
.AsAIAgent(new ChatClientAgentOptions
{
Name = Environment.GetEnvironmentVariable("AGENT_NAME") ?? "hosted-local-codeact",
Description = "Hosted CodeAct agent with sandbox-only compute and fetch_data tools.",
ChatOptions = new ChatOptions
{
ModelId = deploymentName,
Instructions =
"""
You are a helpful assistant. Keep your answers brief. Prefer orchestrating your work
in a single `execute_code` block using `await call_tool(...)` over issuing many
direct tool calls. The sandbox exposes `compute` and `fetch_data` via `call_tool`.
""",
},
AIContextProviders = [codeAct],
});
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddFoundryResponses(agent);
builder.Services.AddDevTemporaryLocalContributorSetup(); // Local Docker debugging only - must not be used in production.
var app = builder.Build();
app.MapFoundryResponses();
// Contributor-only: in Development, also map the per-agent OpenAI route shape that live Foundry uses
// so a local REPL client can target this server via AIProjectClient.AsAIAgent(Uri agentEndpoint).
// Do not use this in production. Hosted Foundry agents only support the agent-endpoint path.
app.MapDevTemporaryLocalAgentEndpoint();
app.Run();
@@ -0,0 +1,159 @@
# Hosted-LocalCodeAct
A hosted agent that uses [`Microsoft.Agents.AI.LocalCodeAct`](../../../../../src/Microsoft.Agents.AI.LocalCodeAct/README.md)
to give the model a single `execute_code` tool. Two sandbox-only host tools,
`compute` and `fetch_data`, are registered on `LocalCodeActProvider` and are
reachable from inside generated Python via `await call_tool(...)` — never as
direct LLM tool calls.
This mirrors the Python
[`foundry_hosted_agent.py`](https://github.com/microsoft/agent-framework/blob/main/python/packages/local_codeact/samples/foundry_hosted_agent.py)
sample for the `agent-framework-local-codeact` package.
> **⚠️ Security:** LocalCodeAct executes LLM-generated Python in the agent
> process. The package is not a sandbox — it relies on the Foundry hosted-agent
> container (or another externally sandboxed environment) for process,
> filesystem, and network isolation. Do not run this outside of a sandbox.
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- Python 3 available on `PATH` (used by `LocalCodeActProvider` to execute the
embedded runner and validator). Override with the `LOCAL_CODEACT_PYTHON`
environment variable if you need a specific interpreter path.
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
Copy the template and fill in your project endpoint:
```bash
cp .env.example .env
```
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
ASPNETCORE_URLS=http://+:8088
ASPNETCORE_ENVIRONMENT=Development
FOUNDRY_MODEL=gpt-4o
LOCAL_CODEACT_PYTHON=python3
```
> **Note:** `.env` is gitignored. The `.env.example` template is checked in as a reference.
## Running directly (contributors)
This project uses `ProjectReference` to build against the local Agent Framework
source, including the `Microsoft.Agents.AI.LocalCodeAct` package.
```bash
cd dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalCodeAct
AGENT_NAME=hosted-local-codeact dotnet run
```
The agent will start on `http://localhost:8088`.
### Test it
Using the Azure Developer CLI:
```bash
azd ai agent invoke --local "Fetch all users, find the admins, multiply 7 by 6, and print the users, admins, and the multiplication result. Use execute_code with await call_tool(...)."
```
Or with curl:
```bash
curl -X POST http://localhost:8088/responses \
-H "Content-Type: application/json" \
-d '{"input": "Fetch all users, find the admins, multiply 7 by 6, and print the users, admins, and the multiplication result. Use execute_code with await call_tool(...).", "model": "hosted-local-codeact"}'
```
## Running with Docker
Since this project uses `ProjectReference`, use `Dockerfile.contributor` which
takes a pre-published output. The image installs Python 3 so the embedded
runner and validator scripts can execute.
### 1. Publish for the container runtime (Linux Alpine)
```bash
dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out
```
### 2. Build the Docker image
```bash
docker build -f Dockerfile.contributor -t hosted-local-codeact .
```
### 3. Run the container
Generate a bearer token on your host and pass it to the container:
```bash
# Generate token (expires in ~1 hour)
export AZURE_BEARER_TOKEN=$(az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv)
# Run with token
docker run --rm -p 8088:8088 \
-e AGENT_NAME=hosted-local-codeact \
-e AZURE_BEARER_TOKEN=$AZURE_BEARER_TOKEN \
--env-file .env \
hosted-local-codeact
```
### 4. Test it
```bash
azd ai agent invoke --local "Fetch all users and print the admins."
```
## How CodeAct works here
`LocalCodeActProvider` is registered as an `AIContextProvider`. On every run it
injects:
- A single `execute_code` tool that the model can call with a Python snippet.
- CodeAct instructions that teach the model to use `await call_tool(...)` for
the provider-owned host tools, rather than asking for direct tool calls.
The provider-owned host tools in this sample:
| Tool | Description |
|------|-------------|
| `compute(operation, a, b)` | Math operation: `add`, `subtract`, `multiply`, `divide`. |
| `fetch_data(table)` | Returns rows from a simulated `users` or `products` table. |
`execute_code` runs the generated Python in a separate Python process governed
by `ProcessExecutionLimits` (5 second timeout in this sample) and the
default-on AST allow-list validator that rejects disallowed imports, builtins,
and dynamic-eval constructs before execution.
## Deploying to Foundry (azd spec)
This sample includes an `azd` manifest (`agent.manifest.yaml`) and hosted agent
spec (`agent.yaml`) for deployment to Foundry.
Initialize an `azd` project from this sample's manifest:
```bash
mkdir hosted-local-codeact && cd hosted-local-codeact
azd ai agent init -m https://github.com/microsoft/agent-framework/blob/main/dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-LocalCodeAct/agent.manifest.yaml
```
Then deploy:
```bash
azd deploy
```
## NuGet package users
If you are consuming the Agent Framework as a NuGet package (not building from
source), use the standard `Dockerfile` instead of `Dockerfile.contributor`. See
the commented section in `HostedLocalCodeAct.csproj` for the `PackageReference`
alternative.
@@ -0,0 +1,30 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/AgentManifest.yaml
name: hosted-local-codeact
displayName: "Hosted Local CodeAct Agent"
description: >
A hosted agent that uses the CodeAct pattern via
Microsoft.Agents.AI.LocalCodeAct. The model only sees an `execute_code`
tool and orchestrates `compute` and `fetch_data` sandbox-only host tools
via `await call_tool(...)` from inside generated Python.
metadata:
tags:
- AI Agent Hosting
- Azure AI AgentServer
- Responses Protocol
- Local CodeAct
- Agent Framework
template:
name: hosted-local-codeact
kind: hosted
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.5"
memory: 1Gi
parameters:
properties: []
resources: []
@@ -0,0 +1,9 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml
kind: hosted
name: hosted-local-codeact
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.5"
memory: 1Gi
@@ -7,7 +7,7 @@ The agent specializes in finding hotels in Seattle, with a `GetAvailableHotels`
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -18,7 +18,7 @@ Copy the template and fill in your project endpoint:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -19,7 +19,7 @@ A hosted agent demonstrating **two layers of MCP (Model Context Protocol) tool i
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -17,7 +17,7 @@ This sample exists to demonstrate two things together:
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with at least one chat model deployment and one embedding model deployment
- A Foundry project with at least one chat model deployment and one embedding model deployment
- Azure CLI logged in (`az login`)
## Configuration
@@ -27,7 +27,7 @@ Foundry injects `APPLICATIONINSIGHTS_CONNECTION_STRING` when the agent runs in t
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -36,7 +36,7 @@ Foundry injects `APPLICATIONINSIGHTS_CONNECTION_STRING` when the agent runs in t
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -7,7 +7,7 @@ This sample demonstrates how to add knowledge grounding to a hosted agent withou
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-4o`)
- A Foundry project with a deployed model (e.g., `gpt-4o`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -18,7 +18,7 @@ Copy the template and fill in your project endpoint:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -1,4 +1,4 @@
# Azure AI Foundry project endpoint (auto-injected in hosted containers).
# Foundry project endpoint (auto-injected in hosted containers).
AZURE_AI_PROJECT_ENDPOINT=https://<your-foundry-account>.services.ai.azure.com/api/projects/<your-project>
# Model deployment name. Must exist in the Foundry project above.
@@ -11,7 +11,7 @@
//
// Required environment variables:
// AZURE_AI_PROJECT_ENDPOINT (local-dev) OR FOUNDRY_PROJECT_ENDPOINT (hosted runtime)
// - Azure AI Foundry project endpoint. The Foundry hosted
// - Foundry project endpoint. The Foundry hosted
// runtime auto-injects FOUNDRY_PROJECT_ENDPOINT; locally
// set AZURE_AI_PROJECT_ENDPOINT (the AF-repo convention).
// TOOLBOX_NAME - Name of the Foundry Toolbox to load
@@ -0,0 +1,16 @@
# Foundry project endpoint (auto-injected in hosted containers).
AZURE_AI_PROJECT_ENDPOINT=https://<your-foundry-account>.services.ai.azure.com/api/projects/<your-project>
# Model deployment name. Must exist in the Foundry project above.
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4o
# Name of the Foundry Toolbox you provisioned in the portal (see README.md).
TOOLBOX_NAME=my-toolset
# Agent name advertised over the wire. Must be unique if running side-by-side with
# other Hosted-* samples (e.g. Hosted-Toolbox-AuthPaths), otherwise the REPL client
# cannot disambiguate which agent to chat with.
AGENT_NAME=hosted-toolbox-agent
# Application Insights connection string (auto-injected in hosted containers; optional locally).
# APPLICATIONINSIGHTS_CONNECTION_STRING=InstrumentationKey=...
@@ -0,0 +1,17 @@
# Use the official .NET 10.0 ASP.NET runtime as a parent image
FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base
WORKDIR /app
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY . .
RUN dotnet restore
RUN dotnet publish -c Release -o /app/publish
# Final stage
FROM base AS final
WORKDIR /app
COPY --from=build /app/publish .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENTRYPOINT ["dotnet", "HostedToolbox.dll"]
@@ -0,0 +1,21 @@
# Dockerfile for contributors building from the agent-framework repository source.
#
# This project uses ProjectReference to the local source, which means a standard
# multi-stage Docker build cannot resolve dependencies outside this folder.
# Pre-publish the app targeting the container runtime and copy the output:
#
# dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out
# docker build -f Dockerfile.contributor -t hosted-toolbox .
# docker run --rm -p 8088:8088 \
# -e AGENT_NAME=hosted-toolbox-agent \
# -e AZURE_BEARER_TOKEN=$AZURE_BEARER_TOKEN \
# --env-file .env hosted-toolbox
#
# For end-users consuming the NuGet package (not ProjectReference), use the standard
# Dockerfile which performs a full dotnet restore + publish inside the container.
FROM mcr.microsoft.com/dotnet/aspnet:10.0-alpine AS final
WORKDIR /app
COPY out/ .
EXPOSE 8088
ENV ASPNETCORE_URLS=http://+:8088
ENTRYPOINT ["dotnet", "HostedToolbox.dll"]
@@ -7,20 +7,19 @@
//
// Required environment variables:
// FOUNDRY_PROJECT_ENDPOINT (hosted runtime) OR AZURE_AI_PROJECT_ENDPOINT (local-dev)
// - Azure AI Foundry project endpoint. The Foundry hosted
// - Foundry project endpoint. The Foundry hosted
// runtime auto-injects FOUNDRY_PROJECT_ENDPOINT; locally
// set AZURE_AI_PROJECT_ENDPOINT.
// FOUNDRY_MODEL - Model deployment name (default: gpt-4o)
//
// Optional:
// FOUNDRY_TOOLBOX_NAME - Name of the toolbox to load (default: my-toolset)
// FOUNDRY_AGENT_TOOLSET_ENDPOINT - Foundry Toolsets proxy base URL
// (injected automatically by Foundry platform at runtime)
// FOUNDRY_AGENT_NAME - Client name reported to MCP server (auto-injected in hosted runtime)
// FOUNDRY_AGENT_VERSION - Client version reported to MCP server (auto-injected in hosted runtime)
// FOUNDRY_AGENT_TOOLSET_FEATURES - Additional Foundry-Features header flags (the mandatory
// Toolboxes=V1Preview flag is always sent; this env var
// appends additional flags if present).
// FOUNDRY_MODEL (or AZURE_AI_MODEL_DEPLOYMENT_NAME)
// - Model deployment name (default: gpt-4o)
// TOOLBOX_NAME - Name of the toolbox to load (default: my-toolset).
// NOTE: All FOUNDRY_* and AGENT_* env-var prefixes (other
// than the platform-injected ones above) are reserved by the
// Foundry container platform and rejected at agent-create.
// Use TOOLBOX_NAME, not FOUNDRY_TOOLBOX_NAME, for the
// sample-owned toolbox name so it survives deployment.
//
// The Foundry.Hosting package builds the toolbox proxy URL from FOUNDRY_PROJECT_ENDPOINT
// per tools-integration-spec.md §2–§3.
@@ -43,8 +42,7 @@ string endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
"nor AZURE_AI_PROJECT_ENDPOINT (local-dev convention) is set.");
string deploymentName = Environment.GetEnvironmentVariable("FOUNDRY_MODEL")
?? Environment.GetEnvironmentVariable("AZURE_AI_MODEL_DEPLOYMENT_NAME") ?? "gpt-4o";
string toolboxName = Environment.GetEnvironmentVariable("FOUNDRY_TOOLBOX_NAME")
?? Environment.GetEnvironmentVariable("TOOLBOX_NAME") ?? "my-toolset";
string toolboxName = Environment.GetEnvironmentVariable("TOOLBOX_NAME") ?? "my-toolset";
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
@@ -1,27 +1,111 @@
# Hosted-Toolbox
A hosted Foundry agent that loads tools from a Foundry Toolbox via the AF Foundry hosting bridge.
A hosted Foundry agent that loads tools from a single Foundry Toolbox via the AF Foundry hosting bridge.
The agent declares one `FoundryAITool.CreateHostedMcpToolbox(name)` marker; `AddFoundryToolboxes(name)` registers a `FoundryToolboxService` that resolves the marker into the individual MCP tools the toolbox bundles, connecting to the Foundry Toolboxes MCP proxy at startup and discovering tools via `tools/list`.
`AddFoundryToolboxes(name)` registers a `FoundryToolboxService` that connects to the Foundry Toolboxes MCP proxy at startup, discovers the toolbox's bundled tools via `tools/list`, and makes them available to the agent on every request. The agent code does nothing per request; the toolbox is baked in on the server.
This is the minimal toolbox intro. For a richer walkthrough where a single toolbox bundles three MCP tools each authenticated differently, see [`Hosted-Toolbox-AuthPaths/`](../Hosted-Toolbox-AuthPaths/).
## Prerequisites
- A Microsoft Foundry project with a Toolbox configured.
- Azure CLI logged in (`az login`).
- Set environment variables:
- `AZURE_AI_PROJECT_ENDPOINT` (local-dev) or `FOUNDRY_PROJECT_ENDPOINT` (auto-injected in hosted containers)
- `AZURE_AI_MODEL_DEPLOYMENT_NAME` (default `gpt-4o`)
- `TOOLBOX_NAME` (default `my-toolbox`)
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- A Foundry project with a deployed model (e.g., `gpt-4o`) and a Toolbox configured
- Azure CLI logged in (`az login`)
## Configuration
Copy the template and fill in your values:
```powershell
Copy-Item .env.example .env
```
Edit `.env`:
```env
AZURE_AI_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
AZURE_AI_MODEL_DEPLOYMENT_NAME=gpt-4o
TOOLBOX_NAME=my-toolset
```
Configuration notes:
- `AZURE_AI_PROJECT_ENDPOINT` (local-dev) or `FOUNDRY_PROJECT_ENDPOINT` (auto-injected in hosted containers).
- `AZURE_AI_MODEL_DEPLOYMENT_NAME` (default `gpt-4o`).
- `TOOLBOX_NAME` (default `my-toolset`). Use `TOOLBOX_NAME`, not `FOUNDRY_TOOLBOX_NAME`: all `FOUNDRY_*` env-var names are reserved by the Foundry platform and rejected at agent-create, so a `FOUNDRY_*`-named value would not survive deployment.
The `Foundry.Hosting` package builds the toolbox proxy URL from `FOUNDRY_PROJECT_ENDPOINT` as `{FOUNDRY_PROJECT_ENDPOINT}/toolboxes/{TOOLBOX_NAME}/mcp?api-version=v1` per [`tools-integration-spec.md`](https://github.com/microsoft/AgentSchema/blob/main/specs/agents/hosted_agents/container-spec/docs/tools-integration-spec.md) §2–§3.
## Run
## Running directly (contributors)
```powershell
cd dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox
dotnet run --tl:off
```
### Test it
Using the Azure Developer CLI:
```powershell
azd ai agent invoke --local "What tools do you have available, and what can they do?"
```
## Running with Docker
### 1. Publish for the container runtime
```powershell
dotnet publish -c Debug -f net10.0 -r linux-musl-x64 --self-contained false -o out
```
### 2. Build and run
```powershell
docker build -f Dockerfile.contributor -t hosted-toolbox .
$env:AZURE_BEARER_TOKEN = (az account get-access-token --resource https://ai.azure.com --query accessToken -o tsv)
docker run --rm -p 8088:8088 `
-e AGENT_NAME=hosted-toolbox-agent `
-e AZURE_BEARER_TOKEN=$env:AZURE_BEARER_TOKEN `
--env-file .env `
hosted-toolbox
```
## Deploying to Foundry (azd spec)
This sample includes an `azd` manifest (`agent.manifest.yaml`) and hosted agent spec (`agent.yaml`) for deployment to Foundry.
Initialize an `azd` project from this sample's manifest:
```powershell
mkdir hosted-toolbox; cd hosted-toolbox
azd ai agent init -m https://github.com/microsoft/agent-framework/blob/main/dotnet/samples/04-hosting/FoundryHostedAgents/responses/Hosted-Toolbox/agent.manifest.yaml
```
Then deploy:
```powershell
azd deploy
```
If you need to override defaults, set deployment-time environment variables in the `azd` environment before deploying:
```powershell
azd env set AZURE_AI_MODEL_DEPLOYMENT_NAME gpt-4o
azd env set TOOLBOX_NAME my-toolset
```
For end-to-end hosted agent deployment guidance, see the [official deployment guide](https://learn.microsoft.com/en-us/azure/foundry/agents/how-to/deploy-hosted-agent).
---
## NuGet package users
Use the standard `Dockerfile` instead of `Dockerfile.contributor`. See the commented section in `HostedToolbox.csproj` for the `PackageReference` alternative.
## Related samples
- [`Hosted-Toolbox-AuthPaths/`](../Hosted-Toolbox-AuthPaths/) — extends this pattern with a three-tool toolbox demonstrating different MCP-tool authentication paths (key, Entra agent identity, inline `Authorization`), driven by the shared `Using-Samples/SimpleAgent/` REPL.
- [`Hosted-Toolbox-AuthPaths/`](../Hosted-Toolbox-AuthPaths/) — same hosting bones as this sample, but the toolbox bundles three MCP tools each authenticated differently (key, Entra agent identity, inline `Authorization`), driven by the shared `Using-Samples/SimpleAgent/` REPL.
- [`Hosted-McpTools/`](../Hosted-McpTools/) — contrasts client-side `McpClient` vs server-side `HostedMcpServerTool` for non-toolbox MCP servers.
@@ -0,0 +1,47 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/AgentManifest.yaml
name: hosted-toolbox
displayName: "Hosted Toolbox"
description: >
A hosted agent that loads its tools from a single Foundry Toolbox via the
AF Foundry hosting bridge. AddFoundryToolboxes(name) connects to the Foundry
Toolboxes MCP proxy at startup and exposes the toolbox's bundled tools to the
agent on every request. The toolbox itself is provisioned out of band; see this
sample's README for the portal walkthrough.
metadata:
tags:
- AI Agent Hosting
- Azure AI AgentServer
- Responses Protocol
- Agent Framework
- Foundry Toolbox
- MCP
template:
name: hosted-toolbox
kind: hosted
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.25"
memory: 0.5Gi
environment_variables:
- name: AZURE_AI_MODEL_DEPLOYMENT_NAME
value: "{{AZURE_AI_MODEL_DEPLOYMENT_NAME}}"
- name: TOOLBOX_NAME
value: "{{TOOLBOX_NAME}}"
parameters:
properties:
- name: TOOLBOX_NAME
type: string
default: "my-toolset"
description: "Name of the Foundry Toolbox to load at runtime."
resources:
- kind: model
id: gpt-4o
name: AZURE_AI_MODEL_DEPLOYMENT_NAME
- kind: toolbox
name: "{{TOOLBOX_NAME}}"
tools: []
@@ -0,0 +1,9 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/microsoft/AgentSchema/refs/heads/main/schemas/v1.0/ContainerAgent.yaml
kind: hosted
name: hosted-toolbox
protocols:
- protocol: responses
version: 1.0.0
resources:
cpu: "0.25"
memory: 0.5Gi
@@ -2,5 +2,5 @@ FOUNDRY_PROJECT_ENDPOINT=<your-azure-ai-project-endpoint>
ASPNETCORE_URLS=http://+:8088
ASPNETCORE_ENVIRONMENT=Development
FOUNDRY_MODEL=gpt-5
FOUNDRY_TOOLBOX_NAME=<your-toolbox-name>
TOOLBOX_NAME=<your-toolbox-name>
AZURE_BEARER_TOKEN=DefaultAzureCredential
@@ -7,9 +7,16 @@
// AgentSkillsProviderBuilder.UseMcpSkills().
//
// Required environment variables:
// FOUNDRY_PROJECT_ENDPOINT - Azure AI Foundry project endpoint
// FOUNDRY_TOOLBOX_NAME - Name of the Foundry Toolbox to connect to
// FOUNDRY_MODEL - Model deployment name (default: gpt-5)
// FOUNDRY_PROJECT_ENDPOINT - Foundry project endpoint
// TOOLBOX_NAME - Name of the Foundry Toolbox to connect to
//
// Optional:
// FOUNDRY_MODEL - Model deployment name (default: gpt-5)
//
// NOTE: All FOUNDRY_* and AGENT_* env-var prefixes (other than the platform-injected ones
// listed above) are reserved by the Foundry container platform and rejected at agent-create.
// Use TOOLBOX_NAME, not FOUNDRY_TOOLBOX_NAME, for the sample-owned toolbox name so it
// survives deployment.
using System.Net.Http.Headers;
using Azure.AI.Projects;
@@ -27,8 +34,8 @@ Env.TraversePath().Load();
var projectEndpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var deployment = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5";
var toolboxName = Environment.GetEnvironmentVariable("FOUNDRY_TOOLBOX_NAME")
?? throw new InvalidOperationException("FOUNDRY_TOOLBOX_NAME is not set.");
var toolboxName = Environment.GetEnvironmentVariable("TOOLBOX_NAME")
?? throw new InvalidOperationException("TOOLBOX_NAME is not set.");
// Build the Toolbox MCP URL from the project endpoint and toolbox name.
var toolboxMcpServerUrl = $"{projectEndpoint.TrimEnd('/')}/toolboxes/{toolboxName}/mcp?api-version=v1";
@@ -13,7 +13,7 @@ This way the full skill body and resources are only loaded when the agent actual
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `gpt-5`)
- A Foundry project with a deployed model (e.g., `gpt-5`)
- A Foundry Toolbox already configured with skills provisioned
- Azure CLI logged in (`az login`)
@@ -25,14 +25,14 @@ Copy the template and fill in your values:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint and toolbox name:
Edit `.env` and set your Foundry project endpoint and toolbox name:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
ASPNETCORE_URLS=http://+:8088
ASPNETCORE_ENVIRONMENT=Development
FOUNDRY_MODEL=gpt-5
FOUNDRY_TOOLBOX_NAME=my-toolbox
TOOLBOX_NAME=my-toolbox
```
> **Note:** `.env` is gitignored. The `.env.example` template is checked in as a reference.
@@ -30,11 +30,11 @@ template:
environment_variables:
- name: FOUNDRY_MODEL
value: "{{FOUNDRY_MODEL}}"
- name: FOUNDRY_TOOLBOX_NAME
value: "{{FOUNDRY_TOOLBOX_NAME}}"
- name: TOOLBOX_NAME
value: "{{TOOLBOX_NAME}}"
parameters:
properties:
- name: FOUNDRY_TOOLBOX_NAME
- name: TOOLBOX_NAME
secret: false
description: Name of the Foundry Toolbox to connect to for MCP skill discovery
resources:
@@ -10,5 +10,5 @@ resources:
environment_variables:
- name: FOUNDRY_MODEL
value: ${FOUNDRY_MODEL}
- name: FOUNDRY_TOOLBOX_NAME
value: ${FOUNDRY_TOOLBOX_NAME}
- name: TOOLBOX_NAME
value: ${TOOLBOX_NAME}
@@ -7,7 +7,7 @@ A hosted agent server demonstrating two patterns in a single app:
Both agents are served over the Responses protocol. The server also exposes interactive web demos at `/tool-demo` and `/workflow-demo`.
> Unlike the other samples in this folder, this one connects to an **Azure OpenAI** resource directly (not an Azure AI Foundry project endpoint).
> Unlike the other samples in this folder, this one connects to an **Azure OpenAI** resource directly (not a Foundry project endpoint).
## Prerequisites
@@ -5,7 +5,7 @@ A hosted agent that demonstrates **multi-agent workflow orchestration**. Three t
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- An Azure AI Foundry project with a deployed model (e.g., `hosted-workflow-simple`)
- A Foundry project with a deployed model (e.g., `hosted-workflow-simple`)
- Azure CLI logged in (`az login`)
## Configuration
@@ -16,7 +16,7 @@ Copy the template and fill in your project endpoint:
cp .env.example .env
```
Edit `.env` and set your Azure AI Foundry project endpoint:
Edit `.env` and set your Foundry project endpoint:
```env
FOUNDRY_PROJECT_ENDPOINT=https://<your-account>.services.ai.azure.com/api/projects/<your-project>
@@ -0,0 +1,51 @@
# Using-Samples — client REPLs for the hosted agents
This folder holds small **client** console apps that connect to the **server** samples in the
sibling `Hosted-*` folders. Each `Hosted-*` project is an agent you host (locally with
`dotnet run` or deployed to Foundry); the projects here are the thing that *talks* to them.
## Why these exist
A hosted Foundry agent is an HTTP server, not a chat UI. It exposes only the per-agent OpenAI
endpoint shape that the platform routes to:
```
{FOUNDRY_PROJECT_ENDPOINT}/agents/{AZURE_AI_AGENT_NAME}/endpoint/protocols/openai
```
There is no built-in console to poke it with. To actually exercise an agent — send a prompt,
watch it call its tools, read the streamed answer — you need a client that builds a
`FoundryAgent` against that endpoint and drives a conversation. That is all these REPLs do:
1. Read `FOUNDRY_PROJECT_ENDPOINT` + `AZURE_AI_AGENT_NAME` from the environment.
2. Derive the per-agent OpenAI endpoint URL.
3. `AIProjectClient(...).AsAIAgent(agentEndpoint)``FoundryAgent`.
4. Loop: read a line, `RunStreamingAsync`, print the streamed reply.
The client is deliberately dumb. It knows nothing about tools, files, toolboxes, or auth — all
of that is the hosted agent's concern on the server side. Swapping which agent you chat with is
just a matter of changing `AZURE_AI_AGENT_NAME`.
## Local HTTP dev
When the target is a local `http://localhost:8088` dev server, the REPLs install a small
`HttpSchemeRewritePolicy`: `AIProjectClient`/`BearerTokenPolicy` require HTTPS, so the client
presents the endpoint as `https://` to satisfy the TLS check, then rewrites the scheme back to
`http://` right before the request hits the wire. This is local-development only.
## The clients
| Client | What it targets | Notes |
|---|---|---|
| [`SimpleAgent/`](./SimpleAgent/) | Any hosted agent | Generic, agent-agnostic REPL. Point it at any `Hosted-*` server via `AZURE_AI_AGENT_NAME`. Used by `Hosted-Toolbox`, `Hosted-Toolbox-AuthPaths`, and `Hosted-McpTools`. |
| [`SessionFilesClient/`](./SessionFilesClient/) | [`Hosted-Files`](../Hosted-Files/) | Same shape as `SimpleAgent`, framed around the bundled-files demo. |
## Configuration (common to all clients)
```env
FOUNDRY_PROJECT_ENDPOINT=https://<host>/api/projects/<project>
AZURE_AI_AGENT_NAME=<registered-server-side-agent-name>
```
Both are required. Authenticate with `az login` before running. See each client's own README for
its end-to-end walkthrough.
@@ -0,0 +1,63 @@
# SimpleAgent
A generic, agent-agnostic chat REPL for any hosted Foundry agent. Point it at a running
`Hosted-*` agent via `AZURE_AI_AGENT_NAME`, and it builds a `FoundryAgent` against that agent's
per-agent OpenAI endpoint and streams replies. This is the shared client that `Hosted-Toolbox`,
`Hosted-Toolbox-AuthPaths`, and `Hosted-McpTools` reference for their end-to-end demos.
It knows nothing about the agent's tools, toolboxes, files, or auth — those are entirely the
server's concern. Changing which agent you chat with is just a different `AZURE_AI_AGENT_NAME`.
See [`../README.md`](../README.md) for why these client REPLs exist at all.
## Prerequisites
- [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0)
- A running hosted agent (any `Hosted-*` sample, locally via `dotnet run` or deployed to Foundry)
- Azure CLI logged in (`az login`)
## Configuration
```env
FOUNDRY_PROJECT_ENDPOINT=https://<host>/api/projects/<project>
AZURE_AI_AGENT_NAME=<registered-server-side-agent-name>
```
Both are required. `FOUNDRY_PROJECT_ENDPOINT` is the Foundry project endpoint URL and
`AZURE_AI_AGENT_NAME` is the registered server-side agent name. The sample builds the per-agent
OpenAI endpoint URL (`{FOUNDRY_PROJECT_ENDPOINT}/agents/{AZURE_AI_AGENT_NAME}/endpoint/protocols/openai`)
from these.
## Run
Against a local Hosted-Toolbox agent listening on `http://localhost:8088`:
```powershell
cd dotnet/samples/04-hosting/FoundryHostedAgents/responses/Using-Samples/SimpleAgent
$env:FOUNDRY_PROJECT_ENDPOINT = "http://localhost:8088/api/projects/local"
$env:AZURE_AI_AGENT_NAME = "hosted-toolbox-agent"
dotnet run
```
When the project endpoint is `http://`, the client presents it as `https://` to satisfy the
bearer-token TLS check, then rewrites the scheme back to `http://` right before transport
(local-development only).
## End-to-end demo
With a hosted agent running:
```text
══════════════════════════════════════════════════════════
Simple Agent Sample
Connected to: https://localhost:8088/api/projects/local/agents/hosted-toolbox-agent/endpoint/protocols/openai
Type a message or 'quit' to exit
══════════════════════════════════════════════════════════
You> What tools do you have available, and what can they do?
Agent> I have the following tools from the toolbox: ...
You> quit
Goodbye!
```
The client only sent a chat prompt; the agent resolved its toolbox tools server-side and answered.
@@ -0,0 +1,15 @@
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFrameworks>net10.0</TargetFrameworks>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<ProjectReference Include="..\..\..\..\src\Microsoft.Agents.AI.Foundry\Microsoft.Agents.AI.Foundry.csproj" />
</ItemGroup>
</Project>
@@ -0,0 +1,141 @@
// Copyright (c) Microsoft. All rights reserved.
// This sample evaluates a pre-existing Azure AI Foundry agent against a rubric evaluator
// that was authored in the Foundry portal.
//
// Rubric evaluators are LLM-as-judge evaluators with custom scoring dimensions you define
// for your domain. agent-framework consumes pre-existing rubric evaluators — they are
// authored in the Foundry portal (or via the dedicated SDK / REST surface) and referenced
// here by name and version.
//
// Prerequisites:
// - An Azure AI Foundry project with a deployed model.
// - A registered Foundry agent in that project (the rubric was created against this agent).
// - A rubric evaluator already created in the Foundry portal.
// - .env (or environment) populated with the FOUNDRY_* variables below.
//
// IMPORTANT: FOUNDRY_PROJECT_ENDPOINT must be the project-scoped URL
// https://<resource>.services.ai.azure.com/api/projects/<project>
// A bare Azure OpenAI endpoint silently fails eval submission with HTTP 500.
using Azure.AI.Projects;
using Azure.AI.Projects.Agents;
using Azure.Identity;
using Microsoft.Agents.AI;
using Microsoft.Agents.AI.Foundry;
using FoundryEvals = Microsoft.Agents.AI.Foundry.FoundryEvals;
string projectEndpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
string model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL")
?? throw new InvalidOperationException("FOUNDRY_MODEL is not set.");
string agentName = Environment.GetEnvironmentVariable("FOUNDRY_AGENT_NAME")
?? throw new InvalidOperationException("FOUNDRY_AGENT_NAME is not set.");
string? agentVersion = Environment.GetEnvironmentVariable("FOUNDRY_AGENT_VERSION");
string rubricName = Environment.GetEnvironmentVariable("FOUNDRY_RUBRIC_NAME")
?? throw new InvalidOperationException("FOUNDRY_RUBRIC_NAME is not set.");
string? rubricVersion = Environment.GetEnvironmentVariable("FOUNDRY_RUBRIC_VERSION");
// WARNING: DefaultAzureCredential is convenient for development but requires careful
// consideration in production. Prefer ManagedIdentityCredential (or a specific credential)
// to avoid latency, unintended credential probing, and fallback security risks.
AIProjectClient projectClient = new(new Uri(projectEndpoint), new DefaultAzureCredential());
// 1. Connect to the pre-existing Foundry agent the rubric was created against.
FoundryAgent agent;
if (agentVersion is null)
{
ProjectsAgentRecord agentRecord = await projectClient.AgentAdministrationClient.GetAgentAsync(agentName);
agent = projectClient.AsAIAgent(agentRecord);
}
else
{
ProjectsAgentVersion versionRecord = await projectClient.AgentAdministrationClient.GetAgentVersionAsync(agentName, agentVersion);
agent = projectClient.AsAIAgent(versionRecord);
}
// 2. Reference the pre-existing rubric evaluator by name + version.
// Always pin a version for reproducible CI runs; a versionless ref resolves to the
// current version at run time and emits a Trace.TraceWarning on each criterion build.
GeneratedEvaluatorRef rubric = rubricVersion is null
? GeneratedEvaluatorRef.Latest(rubricName)
: new GeneratedEvaluatorRef(rubricName, rubricVersion);
// 3. Mix the rubric with built-in evaluators in a single FoundryEvals config.
// The implicit conversion lets you pass strings and refs interchangeably.
FoundryEvals evals = new(
projectClient,
model,
rubric,
FoundryEvals.Relevance,
FoundryEvals.Coherence);
// 4. Run two example queries against the agent and evaluate the outputs in one call.
string[] queries =
[
"What's the weather like in Seattle?",
"Should I bring an umbrella to London tomorrow?",
];
Console.WriteLine(new string('=', 60));
Console.WriteLine($"Evaluating '{agent.Name}' with rubric '{rubricName}' (version {rubricVersion ?? "latest"})");
Console.WriteLine(new string('=', 60));
AgentEvaluationResults results = await agent.EvaluateAsync(queries, evals);
Console.WriteLine($"Status: {results.Status}");
Console.WriteLine($"Results: {results.Passed}/{results.Total} passed");
if (results.ReportUrl is not null)
{
Console.WriteLine($"Portal: {results.ReportUrl}");
}
Console.WriteLine(results.Passed == results.Total ? "[PASS] All passed" : $"[FAIL] {results.Failed} failed");
// 5. Print per-dimension breakdown for each evaluated item — this is the unique value
// of a rubric evaluator over the built-in numeric ones.
Console.WriteLine();
Console.WriteLine(new string('=', 60));
Console.WriteLine("Per-dimension scores");
Console.WriteLine(new string('=', 60));
if (results.DetailedItems is { Count: > 0 })
{
for (int i = 0; i < results.DetailedItems.Count; i++)
{
EvalItemResult item = results.DetailedItems[i];
Console.WriteLine($"Item {i + 1}{(i < queries.Length ? $" \"{queries[i]}\"" : string.Empty)}");
foreach (EvalScoreResult score in item.Scores)
{
Console.WriteLine($" {score.Name}: {score.Score:F1}{(score.Passed is bool p ? (p ? " (pass)" : " (fail)") : string.Empty)}");
if (score.Dimensions is { Count: > 0 } dims)
{
foreach (RubricScore d in dims)
{
string scoreStr = d.Score is int s ? s.ToString() : "n/a";
Console.WriteLine($" - {d.Id}: {scoreStr} (weight={d.Weight}, applicable={d.Applicable})");
}
}
}
Console.WriteLine();
}
}
// 6. CI quality gate — fail the build if a critical dimension drops below threshold.
// Replace "general_quality" with whatever dimension id your rubric actually defines.
Console.WriteLine(new string('=', 60));
Console.WriteLine("Per-dimension quality gate");
Console.WriteLine(new string('=', 60));
try
{
results.AssertDimensionScoreAtLeast("general_quality", minScore: 3.0, evaluator: rubricName, requireApplicable: true);
Console.WriteLine($"[PASS] {results.ProviderName}: general_quality >= 3 on every item");
}
catch (InvalidOperationException ex)
{
Console.WriteLine($"[FAIL] {results.ProviderName}: dimension gate tripped: {ex.Message}");
System.Environment.ExitCode = 1;
}
@@ -0,0 +1,55 @@
# Evaluation — Foundry Rubric
This sample evaluates a pre-existing Azure AI Foundry agent against a **rubric evaluator**
authored in the Foundry portal. Rubric evaluators are LLM-as-judge evaluators with custom
scoring dimensions you define for your domain; agent-framework references them by name and
version, mixes them with built-in evaluators, and exposes per-dimension scores you can gate
CI on.
## What this sample demonstrates
- Connecting to a pre-existing Foundry agent (`AgentAdministrationClient.GetAgentAsync`).
- Referencing a pre-existing rubric evaluator via `GeneratedEvaluatorRef(name, version)`.
- Mixing the rubric with built-in evaluators (`Relevance`, `Coherence`) in one
`FoundryEvals` run.
- Reading per-dimension breakdowns from `EvalScoreResult.Dimensions`.
- Gating CI on a per-dimension threshold via
`AgentEvaluationResults.AssertDimensionScoreAtLeast(...)`.
## Prerequisites
- .NET 10 SDK or later.
- Azure CLI installed and authenticated (`az login`).
- An Azure AI Foundry project with a deployed model.
- A registered Foundry agent in that project (the agent the rubric was created against).
- A rubric evaluator created in the Foundry portal. Creating rubrics through the portal
currently requires picking a Foundry agent as the generation context, so this
prerequisite is implied by having a rubric at all.
> [!IMPORTANT]
> `FOUNDRY_PROJECT_ENDPOINT` **must** be the project-scoped URL
> `https://<resource>.services.ai.azure.com/api/projects/<project>`. A bare Azure OpenAI
> endpoint silently fails eval submission with HTTP 500.
> [!NOTE]
> An **Eval Definition** (a saved bundle of testing_criteria with `"object": "eval"`) is
> not the same as a **Rubric Evaluator** (a standalone evaluator with dimensions, weights,
> and a version). `GeneratedEvaluatorRef` points at the latter.
## Environment variables
```powershell
$env:FOUNDRY_PROJECT_ENDPOINT="https://your-resource.services.ai.azure.com/api/projects/your-project"
$env:FOUNDRY_MODEL="gpt-4o-mini"
$env:FOUNDRY_AGENT_NAME="your-agent-name"
$env:FOUNDRY_AGENT_VERSION="1" # optional; omit for latest
$env:FOUNDRY_RUBRIC_NAME="your-rubric-name"
$env:FOUNDRY_RUBRIC_VERSION="1" # optional; omit for latest (CI: pin this)
```
## Run the sample
```powershell
cd dotnet/samples/05-end-to-end/Evaluation
dotnet run --project .\Evaluation_FoundryRubric
```
+14 -13
View File
@@ -80,33 +80,35 @@ dotnet/samples/
## Default provider
All canonical samples (01-get-started) use **Azure OpenAI** via `AzureOpenAIClient`
with `DefaultAzureCredential`:
All canonical samples (01-get-started) use **Microsoft Foundry** via `AIProjectClient.AsAIAgent()` with `DefaultAzureCredential`:
```csharp
using Azure.AI.OpenAI;
using Azure.AI.Projects;
using Azure.Identity;
using Microsoft.Agents.AI;
using OpenAI.Chat;
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT")
?? throw new InvalidOperationException("AZURE_OPENAI_ENDPOINT is not set.");
var deploymentName = Environment.GetEnvironmentVariable("AZURE_OPENAI_DEPLOYMENT_NAME") ?? "gpt-5.4-mini";
var endpoint = Environment.GetEnvironmentVariable("FOUNDRY_PROJECT_ENDPOINT")
?? throw new InvalidOperationException("FOUNDRY_PROJECT_ENDPOINT is not set.");
var model = Environment.GetEnvironmentVariable("FOUNDRY_MODEL") ?? "gpt-5.4-mini";
// WARNING: DefaultAzureCredential is convenient for development but requires careful consideration in production.
// In production, consider using a specific credential (e.g., ManagedIdentityCredential) to avoid
// latency issues, unintended credential probing, and potential security risks from fallback mechanisms.
AIAgent agent = new AzureOpenAIClient(new Uri(endpoint), new DefaultAzureCredential())
.GetChatClient(deploymentName)
.AsAIAgent(instructions: "...", name: "...");
AIAgent agent = new AIProjectClient(new Uri(endpoint), new DefaultAzureCredential())
.AsAIAgent(model: model, instructions: "...", name: "...");
```
Environment variables:
- `AZURE_OPENAI_ENDPOINT` — Your Azure OpenAI endpoint
- `AZURE_OPENAI_DEPLOYMENT_NAME` — Model deployment name (defaults to `gpt-5.4-mini`)
- `FOUNDRY_PROJECT_ENDPOINT` — Your Foundry project endpoint
- `FOUNDRY_MODEL` — Model name (defaults to `gpt-5.4-mini`)
For authentication, run `az login` before running samples.
**Note:** Use `FoundryAgent` only when demonstrating Foundry-managed (prompt) agents specifically — see `02-agents/AgentsWithFoundry/`. For all other samples, use `AIProjectClient.AsAIAgent()`.
**Note:** For samples demonstrating other providers (Azure OpenAI, OpenAI, Anthropic, etc.), see `02-agents/AgentProviders/`.
## Snippet tags for docs integration
Samples embed named snippet regions for future `:::code` integration:
@@ -135,4 +137,3 @@ dotnet run
- Azure Functions hosting uses `ConfigureDurableAgents(options => options.AddAIAgent(agent))`
- Workflows use `WorkflowBuilder` with `Executor<TIn, TOut>` and edge connections
@@ -2,6 +2,7 @@
## [Unreleased]
- Bind MCP threadId to the current agent and guard cross-agent session dispatch ([#6531](https://github.com/microsoft/agent-framework/pull/6531))
- Added support for durable workflows ([#4436](https://github.com/microsoft/agent-framework/pull/4436))
## v1.0.0-preview.260219.1
@@ -79,6 +79,15 @@ internal class DurableAIAgentProxy(string name, IDurableAgentClient agentClient)
RunRequest request = new([.. messages], responseFormat, enableToolCalls, enableToolNames);
AgentSessionId sessionId = durableSession.SessionId;
// The session must belong to this agent.
if (!string.Equals(sessionId.Name, this.Name, StringComparison.OrdinalIgnoreCase))
{
throw new ArgumentException(
$"The provided session belongs to agent '{sessionId.Name}' but was passed to agent '{this.Name}'. " +
"Sessions cannot be reused across agents.",
paramName: nameof(session));
}
AgentRunHandle agentRunHandle = await this._agentClient.RunAgentAsync(sessionId, request, cancellationToken);
if (isFireAndForget)
@@ -3,6 +3,7 @@
using System;
using System.Collections.Generic;
using System.Diagnostics.CodeAnalysis;
using System.Linq;
using System.Runtime.CompilerServices;
using System.Security.Cryptography;
using System.Text;
@@ -45,6 +46,7 @@ internal static class OutputConverter
OutputItemMessageBuilder? currentMessageBuilder = null;
TextContentBuilder? currentTextBuilder = null;
StringBuilder? accumulatedText = null;
List<Annotation>? accumulatedAnnotations = null;
string? previousMessageId = null;
bool hasTerminalEvent = false;
var executorItemIds = new Dictionary<string, string>();
@@ -60,7 +62,7 @@ internal static class OutputConverter
if (update.RawRepresentation is WorkflowEvent workflowEvent && update.Contents.Count == 0)
{
// Close any open message builder before emitting workflow items
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -68,6 +70,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
foreach (var evt in EmitWorkflowEvent(stream, workflowEvent, executorItemIds))
@@ -86,7 +89,7 @@ internal static class OutputConverter
{
if (!IsSameMessage(update.MessageId, previousMessageId) && currentMessageBuilder is not null)
{
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -94,6 +97,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
}
previousMessageId = update.MessageId;
@@ -115,6 +119,14 @@ internal static class OutputConverter
yield return currentTextBuilder!.EmitDelta(textContent.Text);
}
if (textContent.Annotations is { Count: > 0 })
{
foreach (var sdkAnnotation in ConvertToSdkAnnotations(textContent.Annotations))
{
(accumulatedAnnotations ??= []).Add(sdkAnnotation);
}
}
break;
}
@@ -125,7 +137,7 @@ internal static class OutputConverter
break;
}
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -133,6 +145,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
var arguments = functionCall.Arguments is not null
@@ -149,7 +162,7 @@ internal static class OutputConverter
case TextReasoningContent reasoningContent:
{
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -157,6 +170,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
var reasoningBuilder = stream.AddOutputItemReasoningItem();
@@ -176,7 +190,7 @@ internal static class OutputConverter
case ToolApprovalRequestContent approvalRequest when approvalRequest.ToolCall is FunctionCallContent approvalFunctionCall:
{
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -184,6 +198,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
// The Responses API only standardizes the MCP-flavored approval primitive.
@@ -237,7 +252,7 @@ internal static class OutputConverter
case ErrorContent errorContent:
{
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -245,6 +260,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
hasTerminalEvent = true;
@@ -269,7 +285,7 @@ internal static class OutputConverter
break;
}
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -277,6 +293,7 @@ internal static class OutputConverter
currentTextBuilder = null;
currentMessageBuilder = null;
accumulatedText = null;
accumulatedAnnotations = null;
previousMessageId = null;
var outputText = EncodeFunctionResultAsJsonStringPayload(functionResult.Result);
@@ -304,7 +321,7 @@ internal static class OutputConverter
}
// Close any remaining open message
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText))
foreach (var evt in CloseCurrentMessage(currentMessageBuilder, currentTextBuilder, accumulatedText, accumulatedAnnotations))
{
yield return evt;
}
@@ -318,7 +335,8 @@ internal static class OutputConverter
private static IEnumerable<ResponseStreamEvent> CloseCurrentMessage(
OutputItemMessageBuilder? messageBuilder,
TextContentBuilder? textBuilder,
StringBuilder? accumulatedText)
StringBuilder? accumulatedText,
List<Annotation>? annotations = null)
{
if (messageBuilder is null)
{
@@ -329,6 +347,16 @@ internal static class OutputConverter
{
var finalText = accumulatedText?.ToString() ?? string.Empty;
yield return textBuilder.EmitTextDone(finalText);
// Annotations must be emitted after EmitTextDone and before EmitDone.
if (annotations is not null)
{
foreach (var annotation in annotations)
{
yield return textBuilder.EmitAnnotationAdded(annotation);
}
}
yield return textBuilder.EmitDone();
}
@@ -338,6 +366,41 @@ internal static class OutputConverter
private static bool IsSameMessage(string? currentId, string? previousId) =>
currentId is not { Length: > 0 } || previousId is not { Length: > 0 } || currentId == previousId;
/// <summary>
/// Converts MEAI <see cref="AIAnnotation"/> instances to Responses SDK <see cref="Annotation"/> objects.
/// Only <see cref="CitationAnnotation"/> with a URL and at least one <see cref="TextSpanAnnotatedRegion"/>
/// with explicit start/end indices is converted; all other shapes are skipped.
/// </summary>
private static IEnumerable<Annotation> ConvertToSdkAnnotations(IList<AIAnnotation> annotations)
{
foreach (var ann in annotations)
{
if (ann is not CitationAnnotation citation || citation.Url is null)
{
continue;
}
var regions = citation.AnnotatedRegions?
.OfType<TextSpanAnnotatedRegion>()
.Where(r => r.StartIndex is not null && r.EndIndex is not null)
.ToList();
if (regions is not { Count: > 0 })
{
continue;
}
foreach (var region in regions)
{
yield return new UrlCitationBody(
citation.Url,
region.StartIndex!.Value,
region.EndIndex!.Value,
citation.Title ?? string.Empty);
}
}
}
private static ResponseUsage ConvertUsage(UsageDetails details, ResponseUsage? existing)
{
var inputTokens = details.InputTokenCount ?? 0;
@@ -148,19 +148,68 @@ internal static class FoundryEvalConverter
/// <summary>
/// Builds the <c>testing_criteria</c> array for <c>evals.create()</c>.
/// </summary>
/// <param name="evaluators">Evaluator names (short or fully-qualified).</param>
/// <param name="evaluators">
/// Evaluator specs — built-in evaluator names (short or fully-qualified) and/or
/// <see cref="GeneratedEvaluatorRef"/> instances for pre-existing rubric evaluators.
/// </param>
/// <param name="model">Model deployment name for the LLM judge.</param>
/// <param name="includeDataMapping">
/// Whether to include field-level data mapping (required for JSONL data source).
/// </param>
/// <param name="includeToolDefinitions">
/// Whether the mapped data items include tool definitions. Used to add a
/// <c>tool_definitions</c> mapping entry for rubric evaluators (built-in evaluators
/// derive this from their own <see cref="ToolEvaluators"/> membership).
/// </param>
internal static List<WireTestingCriterion> BuildTestingCriteria(
IEnumerable<string> evaluators,
IEnumerable<FoundryEvaluatorSpec> evaluators,
string model,
bool includeDataMapping = false)
bool includeDataMapping = false,
bool includeToolDefinitions = false)
{
var criteria = new List<WireTestingCriterion>();
foreach (var name in evaluators)
foreach (var spec in evaluators)
{
if (spec.IsRubric)
{
var @ref = spec.GeneratedRef!;
Dictionary<string, string>? refMapping = null;
if (includeDataMapping)
{
// Rubric evaluators accept conversation arrays like agent evaluators,
// plus tool_definitions when items are tool-aware.
refMapping = new Dictionary<string, string>
{
["query"] = "{{item.query_messages}}",
["response"] = "{{item.response_messages}}",
};
if (includeToolDefinitions)
{
refMapping["tool_definitions"] = "{{item.tool_definitions}}";
}
}
criteria.Add(new WireTestingCriterion
{
Name = @ref.DisplayName ?? @ref.Name,
EvaluatorName = @ref.Name,
EvaluatorVersion = @ref.Version,
InitializationParameters = new WireInitParams { DeploymentName = model },
DataMapping = refMapping,
});
if (@ref.Version is null)
{
System.Diagnostics.Trace.TraceWarning(
"GeneratedEvaluatorRef '{0}' has no pinned version; the eval run will resolve to whichever version is current at execution time. Pin the version for reproducible runs.",
@ref.Name);
}
continue;
}
var name = spec.BuiltinName!;
var qualified = ResolveEvaluator(name);
var shortName = name.StartsWith("builtin.", StringComparison.Ordinal)
? name.Substring("builtin.".Length)
@@ -248,8 +297,12 @@ internal static class FoundryEvalConverter
/// Returns the subset of <paramref name="evaluators"/> that require a ground-truth
/// (reference) value but cannot be evaluated because no item provided one.
/// </summary>
/// <remarks>
/// Rubric references (<see cref="GeneratedEvaluatorRef"/>) are skipped — they are not
/// ground-truthdependent on the wire.
/// </remarks>
internal static List<string> FindMissingGroundTruthEvaluators(
IEnumerable<string> evaluators,
IEnumerable<FoundryEvaluatorSpec> evaluators,
bool hasGroundTruth)
{
if (hasGroundTruth)
@@ -258,8 +311,14 @@ internal static class FoundryEvalConverter
}
var missing = new List<string>();
foreach (var name in evaluators)
foreach (var spec in evaluators)
{
if (spec.IsRubric)
{
continue;
}
var name = spec.BuiltinName!;
if (GroundTruthEvaluators.Contains(ResolveEvaluator(name)))
{
missing.Add(name);
@@ -137,6 +137,9 @@ internal sealed class WireTestingCriterion
[JsonPropertyName("evaluator_name")]
public required string EvaluatorName { get; init; }
[JsonPropertyName("evaluator_version")]
public string? EvaluatorVersion { get; init; }
[JsonPropertyName("initialization_parameters")]
public required WireInitParams InitializationParameters { get; init; }
@@ -43,7 +43,7 @@ public sealed class FoundryEvals : IAgentEvaluator
private readonly EvaluationClient _evaluationClient;
private readonly string _model;
private readonly string[] _evaluatorNames;
private readonly FoundryEvaluatorSpec[] _evaluators;
private readonly IConversationSplitter? _splitter;
private readonly double _pollIntervalSeconds = 5.0;
private readonly double _timeoutSeconds = 300.0;
@@ -58,17 +58,21 @@ public sealed class FoundryEvals : IAgentEvaluator
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="evaluators">
/// Names of evaluators to use (e.g., <see cref="Relevance"/>, <see cref="Coherence"/>).
/// When empty, defaults to relevance and coherence.
/// Evaluator specs to use. Each entry can be a built-in evaluator name (string, for example
/// <see cref="Relevance"/>) or a <see cref="GeneratedEvaluatorRef"/> for a rubric evaluator
/// already registered in the Foundry project. When empty, defaults to relevance, coherence,
/// and task adherence.
/// </param>
public FoundryEvals(AIProjectClient projectClient, string model, params string[] evaluators)
public FoundryEvals(AIProjectClient projectClient, string model, params FoundryEvaluatorSpec[] evaluators)
{
ArgumentNullException.ThrowIfNull(projectClient);
ArgumentException.ThrowIfNullOrWhiteSpace(model);
ArgumentNullException.ThrowIfNull(evaluators);
EnsureAllSpecsValid(evaluators, nameof(evaluators));
this._evaluationClient = projectClient.GetProjectOpenAIClient().GetEvaluationClient();
this._model = model;
this._evaluatorNames = evaluators.Length > 0
this._evaluators = evaluators.Length > 0
? evaluators
: [Relevance, Coherence, TaskAdherence];
}
@@ -84,14 +88,14 @@ public sealed class FoundryEvals : IAgentEvaluator
/// or a custom <see cref="IConversationSplitter"/> implementation.
/// </param>
/// <param name="evaluators">
/// Names of evaluators to use (e.g., <see cref="Relevance"/>, <see cref="Coherence"/>).
/// When empty, defaults to relevance and coherence.
/// Evaluator specs (built-in names and/or <see cref="GeneratedEvaluatorRef"/> instances).
/// When empty, defaults to relevance, coherence, and task adherence.
/// </param>
public FoundryEvals(
AIProjectClient projectClient,
string model,
IConversationSplitter? splitter,
params string[] evaluators)
params FoundryEvaluatorSpec[] evaluators)
: this(projectClient, model, evaluators)
{
this._splitter = splitter;
@@ -107,14 +111,16 @@ public sealed class FoundryEvals : IAgentEvaluator
/// </param>
/// <param name="pollIntervalSeconds">Seconds between status polls (default 5).</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion (default 300).</param>
/// <param name="evaluators">Evaluator names to use.</param>
/// <param name="evaluators">
/// Evaluator specs (built-in names and/or <see cref="GeneratedEvaluatorRef"/> instances).
/// </param>
public FoundryEvals(
AIProjectClient projectClient,
string model,
IConversationSplitter? splitter,
double pollIntervalSeconds,
double timeoutSeconds,
params string[] evaluators)
params FoundryEvaluatorSpec[] evaluators)
: this(projectClient, model, splitter, evaluators)
{
ArgumentOutOfRangeException.ThrowIfLessThanOrEqual(pollIntervalSeconds, 0);
@@ -123,6 +129,81 @@ public sealed class FoundryEvals : IAgentEvaluator
this._timeoutSeconds = timeoutSeconds;
}
// -----------------------------------------------------------------------
// string[] constructor overloads (source-compat with older API that took
// `params string[] evaluators` before FoundryEvaluatorSpec was introduced).
// `params` is intentionally omitted to avoid overload ambiguity with the
// spec-based ctors at zero-args; individual string literals still resolve
// through `params FoundryEvaluatorSpec[]` via implicit conversion.
// -----------------------------------------------------------------------
/// <summary>
/// Initializes a new instance of the <see cref="FoundryEvals"/> class using built-in evaluator
/// names. Preserves source compatibility for callers that pass a <see cref="string"/> array.
/// </summary>
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="evaluators">Built-in evaluator names (for example <see cref="Relevance"/>).</param>
public FoundryEvals(AIProjectClient projectClient, string model, string[] evaluators)
: this(projectClient, model, ToSpecs(evaluators))
{
}
/// <summary>
/// Initializes a new instance of the <see cref="FoundryEvals"/> class with a splitter and
/// built-in evaluator names.
/// </summary>
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="splitter">Default conversation splitter for multi-turn conversations.</param>
/// <param name="evaluators">Built-in evaluator names.</param>
public FoundryEvals(
AIProjectClient projectClient,
string model,
IConversationSplitter? splitter,
string[] evaluators)
: this(projectClient, model, splitter, ToSpecs(evaluators))
{
}
/// <summary>
/// Initializes a new instance of the <see cref="FoundryEvals"/> class with full configuration
/// and built-in evaluator names.
/// </summary>
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="splitter">Default conversation splitter for multi-turn conversations.</param>
/// <param name="pollIntervalSeconds">Seconds between status polls.</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion.</param>
/// <param name="evaluators">Built-in evaluator names.</param>
public FoundryEvals(
AIProjectClient projectClient,
string model,
IConversationSplitter? splitter,
double pollIntervalSeconds,
double timeoutSeconds,
string[] evaluators)
: this(projectClient, model, splitter, pollIntervalSeconds, timeoutSeconds, ToSpecs(evaluators))
{
}
private static FoundryEvaluatorSpec[] ToSpecs(string[]? evaluators)
{
if (evaluators is null || evaluators.Length == 0)
{
return [];
}
var specs = new FoundryEvaluatorSpec[evaluators.Length];
for (int i = 0; i < evaluators.Length; i++)
{
specs[i] = evaluators[i]
?? throw new ArgumentException($"Evaluator name at index {i} is null.", nameof(evaluators));
}
return specs;
}
// -----------------------------------------------------------------------
// IAgentEvaluator
// -----------------------------------------------------------------------
@@ -149,10 +230,10 @@ public sealed class FoundryEvals : IAgentEvaluator
bool allHaveGroundTruth = payloads.Count > 0 && payloads.All(p => p.GroundTruth is not null);
// Filter out tool evaluators if no items have tools; auto-add ToolCallAccuracy if tools present
var evaluators = FilterToolEvaluators(this._evaluatorNames, hasTools);
if (hasTools && !evaluators.Any(e => FoundryEvalConverter.ToolEvaluators.Contains(FoundryEvalConverter.ResolveEvaluator(e))))
var evaluators = FilterToolEvaluators(this._evaluators, hasTools);
if (hasTools && !HasToolEvaluator(evaluators))
{
evaluators = [.. evaluators, ToolCallAccuracy];
evaluators = [.. evaluators, (FoundryEvaluatorSpec)ToolCallAccuracy];
}
// Fail fast if a ground-truth evaluator (e.g. similarity) is requested but not
@@ -178,7 +259,7 @@ public sealed class FoundryEvals : IAgentEvaluator
ItemSchema = FoundryEvalConverter.BuildItemSchema(hasContext, hasTools, hasGroundTruth),
},
TestingCriteria = FoundryEvalConverter.BuildTestingCriteria(
evaluators, this._model, includeDataMapping: true),
evaluators, this._model, includeDataMapping: true, includeToolDefinitions: hasTools),
};
var createEvalJson = JsonSerializer.Serialize(createEvalPayload, s_jsonOptions);
@@ -270,6 +351,47 @@ public sealed class FoundryEvals : IAgentEvaluator
// Static evaluation methods (traces and targets)
// -----------------------------------------------------------------------
/// <summary>
/// Source-compat overload of <see cref="EvaluateTracesAsync(AIProjectClient, string, IEnumerable{string}, IEnumerable{string}, string, int, FoundryEvaluatorSpec[], string, double, double, CancellationToken)"/>
/// that accepts a <see cref="string"/> array of built-in evaluator names.
/// </summary>
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="responseIds">Evaluate specific Responses API response IDs.</param>
/// <param name="traceIds">Evaluate specific OTel trace IDs from App Insights.</param>
/// <param name="agentId">Filter traces by agent ID (used with <paramref name="lookbackHours"/>).</param>
/// <param name="lookbackHours">Hours of trace history to evaluate.</param>
/// <param name="evaluators">Built-in evaluator names. Each is wrapped via <see cref="FoundryEvaluatorSpec(string)"/>.</param>
/// <param name="evalName">Display name for the evaluation.</param>
/// <param name="pollIntervalSeconds">Seconds between status polls.</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>Evaluation results with status, report URL, and per-item details.</returns>
public static Task<AgentEvaluationResults> EvaluateTracesAsync(
AIProjectClient projectClient,
string model,
IEnumerable<string>? responseIds,
IEnumerable<string>? traceIds,
string? agentId,
int lookbackHours,
string[]? evaluators = null,
string evalName = "Agent Framework Trace Eval",
double pollIntervalSeconds = 5.0,
double timeoutSeconds = 300.0,
CancellationToken cancellationToken = default)
=> EvaluateTracesAsync(
projectClient,
model,
responseIds,
traceIds,
agentId,
lookbackHours,
ToSpecs(evaluators) is { Length: > 0 } specs ? specs : null,
evalName,
pollIntervalSeconds,
timeoutSeconds,
cancellationToken);
/// <summary>
/// Evaluates agent behavior from Responses API response IDs, OTel traces, or agent activity.
/// </summary>
@@ -287,7 +409,11 @@ public sealed class FoundryEvals : IAgentEvaluator
/// <param name="traceIds">Evaluate specific OTel trace IDs from App Insights.</param>
/// <param name="agentId">Filter traces by agent ID (used with <paramref name="lookbackHours"/>).</param>
/// <param name="lookbackHours">Hours of trace history to evaluate (default 24).</param>
/// <param name="evaluators">Evaluator names. Defaults to relevance, coherence, and task adherence.</param>
/// <param name="evaluators">
/// Evaluator specs. Each entry can be a built-in evaluator name (string) or a
/// <see cref="GeneratedEvaluatorRef"/> for a rubric evaluator. Defaults to relevance,
/// coherence, and task adherence.
/// </param>
/// <param name="evalName">Display name for the evaluation.</param>
/// <param name="pollIntervalSeconds">Seconds between status polls (default 5).</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion (default 300).</param>
@@ -300,7 +426,7 @@ public sealed class FoundryEvals : IAgentEvaluator
IEnumerable<string>? traceIds = null,
string? agentId = null,
int lookbackHours = 24,
string[]? evaluators = null,
FoundryEvaluatorSpec[]? evaluators = null,
string evalName = "Agent Framework Trace Eval",
double pollIntervalSeconds = 5.0,
double timeoutSeconds = 300.0,
@@ -320,9 +446,10 @@ public sealed class FoundryEvals : IAgentEvaluator
}
var evalClient = projectClient.GetProjectOpenAIClient().GetEvaluationClient();
var resolvedEvaluators = evaluators is { Length: > 0 }
FoundryEvaluatorSpec[] resolvedEvaluators = evaluators is { Length: > 0 }
? evaluators
: [Relevance, Coherence, TaskAdherence];
EnsureAllSpecsValid(resolvedEvaluators, nameof(evaluators));
// Create the evaluation definition with the appropriate data source scenario
object dataSourceConfig;
@@ -429,6 +556,41 @@ public sealed class FoundryEvals : IAgentEvaluator
};
}
/// <summary>
/// Source-compat overload of <see cref="EvaluateFoundryTargetAsync(AIProjectClient, string, IDictionary{string, object}, IEnumerable{string}, FoundryEvaluatorSpec[], string, double, double, CancellationToken)"/>
/// that accepts a <see cref="string"/> array of built-in evaluator names.
/// </summary>
/// <param name="projectClient">The Azure AI Foundry project client.</param>
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="target">Target configuration (must include a "type" key).</param>
/// <param name="testQueries">Queries for Foundry to send to the target.</param>
/// <param name="evaluators">Built-in evaluator names. Each is wrapped via <see cref="FoundryEvaluatorSpec(string)"/>.</param>
/// <param name="evalName">Display name for the evaluation.</param>
/// <param name="pollIntervalSeconds">Seconds between status polls.</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion.</param>
/// <param name="cancellationToken">Cancellation token.</param>
/// <returns>Evaluation results with status, report URL, and per-item details.</returns>
public static Task<AgentEvaluationResults> EvaluateFoundryTargetAsync(
AIProjectClient projectClient,
string model,
IDictionary<string, object> target,
IEnumerable<string> testQueries,
string[]? evaluators = null,
string evalName = "Agent Framework Target Eval",
double pollIntervalSeconds = 5.0,
double timeoutSeconds = 300.0,
CancellationToken cancellationToken = default)
=> EvaluateFoundryTargetAsync(
projectClient,
model,
target,
testQueries,
ToSpecs(evaluators) is { Length: > 0 } specs ? specs : null,
evalName,
pollIntervalSeconds,
timeoutSeconds,
cancellationToken);
/// <summary>
/// Evaluates a Foundry-registered agent or model deployment.
/// </summary>
@@ -440,7 +602,10 @@ public sealed class FoundryEvals : IAgentEvaluator
/// <param name="model">Model deployment name for the LLM judge evaluator.</param>
/// <param name="target">Target configuration (must include a "type" key, e.g. "azure_ai_agent").</param>
/// <param name="testQueries">Queries for Foundry to send to the target.</param>
/// <param name="evaluators">Evaluator names. Defaults to relevance, coherence, and task adherence.</param>
/// <param name="evaluators">
/// Evaluator specs (built-in names and/or <see cref="GeneratedEvaluatorRef"/> instances).
/// Defaults to relevance, coherence, and task adherence.
/// </param>
/// <param name="evalName">Display name for the evaluation.</param>
/// <param name="pollIntervalSeconds">Seconds between status polls (default 5).</param>
/// <param name="timeoutSeconds">Maximum seconds to wait for completion (default 300).</param>
@@ -451,7 +616,7 @@ public sealed class FoundryEvals : IAgentEvaluator
string model,
IDictionary<string, object> target,
IEnumerable<string> testQueries,
string[]? evaluators = null,
FoundryEvaluatorSpec[]? evaluators = null,
string evalName = "Agent Framework Target Eval",
double pollIntervalSeconds = 5.0,
double timeoutSeconds = 300.0,
@@ -473,9 +638,10 @@ public sealed class FoundryEvals : IAgentEvaluator
}
var evalClient = projectClient.GetProjectOpenAIClient().GetEvaluationClient();
var resolvedEvaluators = evaluators is { Length: > 0 }
FoundryEvaluatorSpec[] resolvedEvaluators = evaluators is { Length: > 0 }
? evaluators
: [Relevance, Coherence, TaskAdherence];
EnsureAllSpecsValid(resolvedEvaluators, nameof(evaluators));
var createEvalPayload = new WireCreateEvalRequest
{
@@ -831,7 +997,13 @@ public sealed class FoundryEvals : IAgentEvaluator
passed = pp.ValueKind == JsonValueKind.True;
}
scores.Add(new EvalScoreResult(name, score, passed));
IReadOnlyList<RubricScore>? dimensions = null;
if (r.TryGetProperty("sample", out var perResultSample))
{
dimensions = ParseRubricScores(perResultSample);
}
scores.Add(new EvalScoreResult(name, score, passed) { Dimensions = dimensions });
}
}
@@ -917,20 +1089,202 @@ public sealed class FoundryEvals : IAgentEvaluator
return result;
}
internal static string[] FilterToolEvaluators(string[] evaluators, bool hasTools)
private static readonly string[] s_rubricDimensionKeys = ["dimension_scores", "rubric_scores"];
/// <summary>
/// Extracts the per-dimension <see cref="RubricScore"/> list from a result-level <c>sample</c>
/// payload, when present. Accepts several legacy/canonical shapes for forward compatibility
/// with provider SDK changes:
/// </summary>
/// <remarks>
/// <list type="bullet">
/// <item>
/// <description><c>sample.properties.dimension_scores</c> (canonical Foundry shape).</description>
/// </item>
/// <item>
/// <description><c>sample.properties.rubric_scores</c> (preview / legacy key).</description>
/// </item>
/// <item>
/// <description>Top-level <c>sample.dimension_scores</c> / <c>sample.rubric_scores</c> as a
/// defensive fallback.</description>
/// </item>
/// </list>
/// Returns <see langword="null"/> when no rubric scores are present (the evaluator was not
/// a rubric evaluator). Malformed entries (missing <c>id</c>, <c>weight</c>, or <c>applicable</c>)
/// are skipped without failing the whole list.
/// </remarks>
internal static List<RubricScore>? ParseRubricScores(JsonElement sample)
{
if (sample.ValueKind != JsonValueKind.Object)
{
return null;
}
// Prefer sample.properties.<key> then fall back to top-level sample.<key>.
if (sample.TryGetProperty("properties", out var properties)
&& properties.ValueKind == JsonValueKind.Object)
{
foreach (var key in s_rubricDimensionKeys)
{
if (properties.TryGetProperty(key, out var raw))
{
var parsed = ParseDimensionEntries(raw);
if (parsed.Count > 0)
{
return parsed;
}
}
}
}
foreach (var key in s_rubricDimensionKeys)
{
if (sample.TryGetProperty(key, out var raw))
{
var parsed = ParseDimensionEntries(raw);
if (parsed.Count > 0)
{
return parsed;
}
}
}
return null;
}
private static List<RubricScore> ParseDimensionEntries(JsonElement raw)
{
var parsed = new List<RubricScore>();
if (raw.ValueKind != JsonValueKind.Array)
{
return parsed;
}
foreach (var entry in raw.EnumerateArray())
{
if (entry.ValueKind != JsonValueKind.Object)
{
continue;
}
if (!entry.TryGetProperty("id", out var idProp)
|| !entry.TryGetProperty("weight", out var weightProp)
|| !entry.TryGetProperty("applicable", out var applicableProp))
{
continue;
}
string? id = idProp.ValueKind switch
{
JsonValueKind.String => idProp.GetString(),
JsonValueKind.Number => idProp.GetRawText(),
_ => null,
};
if (string.IsNullOrEmpty(id))
{
continue;
}
if (weightProp.ValueKind != JsonValueKind.Number
|| !weightProp.TryGetInt32(out var weight))
{
continue;
}
if (applicableProp.ValueKind is not (JsonValueKind.True or JsonValueKind.False))
{
continue;
}
int? score = null;
if (entry.TryGetProperty("score", out var scoreProp)
&& scoreProp.ValueKind == JsonValueKind.Number)
{
if (scoreProp.TryGetInt32(out var intScore))
{
score = intScore;
}
else if (scoreProp.TryGetDouble(out var doubleScore))
{
score = (int)doubleScore;
}
}
string reason = entry.TryGetProperty("reason", out var reasonProp)
&& reasonProp.ValueKind == JsonValueKind.String
? reasonProp.GetString() ?? string.Empty
: string.Empty;
parsed.Add(new RubricScore(
Id: id!,
Score: score,
Applicable: applicableProp.ValueKind == JsonValueKind.True,
Weight: weight,
Reason: reason));
}
return parsed;
}
internal static FoundryEvaluatorSpec[] FilterToolEvaluators(FoundryEvaluatorSpec[] evaluators, bool hasTools)
{
if (hasTools)
{
return evaluators;
}
var filtered = Array.FindAll(evaluators, e =>
!FoundryEvalConverter.ToolEvaluators.Contains(FoundryEvalConverter.ResolveEvaluator(e)));
var filtered = Array.FindAll(evaluators, spec =>
{
if (spec.IsRubric)
{
// Rubric refs are tool-aware but not tool-required; preserve them.
return true;
}
return !FoundryEvalConverter.ToolEvaluators.Contains(FoundryEvalConverter.ResolveEvaluator(spec.BuiltinName!));
});
return filtered.Length > 0
? filtered
: throw new ArgumentException(
"All configured evaluators require tool definitions, but no tool calls were found in the eval items. "
+ $"Tool evaluators: {string.Join(", ", evaluators)}. Either add tool call content to your EvalItems or remove tool-type evaluators.");
+ $"Tool evaluators: {string.Join(", ", evaluators.Select(e => e.ToString()))}. Either add tool call content to your EvalItems or remove tool-type evaluators.");
}
/// <summary>
/// Validates every spec in <paramref name="evaluators"/> — defensively guards against
/// <c>default(FoundryEvaluatorSpec)</c> values that would otherwise NRE deep in the
/// dispatch pipeline (e.g. on <c>spec.BuiltinName!</c>).
/// </summary>
internal static void EnsureAllSpecsValid(FoundryEvaluatorSpec[] evaluators, string paramName)
{
for (int i = 0; i < evaluators.Length; i++)
{
if (!evaluators[i].IsValid)
{
throw new ArgumentException(
$"Invalid {nameof(FoundryEvaluatorSpec)} at index {i}: must be constructed with either a built-in " +
$"evaluator name or a {nameof(GeneratedEvaluatorRef)}. The default struct value is not a valid spec.",
paramName);
}
}
}
private static bool HasToolEvaluator(FoundryEvaluatorSpec[] evaluators)
{
foreach (var spec in evaluators)
{
if (spec.IsRubric)
{
continue;
}
if (FoundryEvalConverter.ToolEvaluators.Contains(FoundryEvalConverter.ResolveEvaluator(spec.BuiltinName!)))
{
return true;
}
}
return false;
}
}
@@ -0,0 +1,118 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
namespace Microsoft.Agents.AI.Foundry;
/// <summary>
/// Specifies a single evaluator for a <see cref="FoundryEvals"/> run — either a built-in
/// Foundry evaluator (referenced by short or fully-qualified name) or a pre-existing rubric
/// evaluator (referenced by <see cref="GeneratedEvaluatorRef"/>).
/// </summary>
/// <remarks>
/// <para>
/// Both <see cref="string"/> and <see cref="GeneratedEvaluatorRef"/> are implicitly convertible
/// to <see cref="FoundryEvaluatorSpec"/>, so call sites can mix the two:
/// </para>
/// <code>
/// var evals = new FoundryEvals(
/// projectClient,
/// "gpt-4o-mini",
/// new GeneratedEvaluatorRef("policy-rubric", Version: "3"),
/// FoundryEvals.Relevance,
/// FoundryEvals.Coherence);
/// </code>
/// </remarks>
public readonly struct FoundryEvaluatorSpec : IEquatable<FoundryEvaluatorSpec>
{
private FoundryEvaluatorSpec(string? builtinName, GeneratedEvaluatorRef? generatedRef)
{
this.BuiltinName = builtinName;
this.GeneratedRef = generatedRef;
}
/// <summary>
/// Initializes a new <see cref="FoundryEvaluatorSpec"/> for a built-in evaluator by name
/// (for example <c>"relevance"</c> or <c>"builtin.relevance"</c>).
/// </summary>
/// <param name="builtinName">Built-in evaluator name.</param>
public FoundryEvaluatorSpec(string builtinName)
: this(builtinName ?? throw new ArgumentNullException(nameof(builtinName)), null)
{
}
/// <summary>
/// Initializes a new <see cref="FoundryEvaluatorSpec"/> for a generated rubric evaluator
/// previously registered with the provider.
/// </summary>
/// <param name="generatedRef">Reference to the rubric evaluator.</param>
public FoundryEvaluatorSpec(GeneratedEvaluatorRef generatedRef)
: this(null, generatedRef ?? throw new ArgumentNullException(nameof(generatedRef)))
{
}
/// <summary>Gets the built-in evaluator name, or <see langword="null"/> when this is a rubric reference.</summary>
public string? BuiltinName { get; }
/// <summary>Gets the rubric reference, or <see langword="null"/> when this is a built-in evaluator.</summary>
public GeneratedEvaluatorRef? GeneratedRef { get; }
/// <summary>Gets whether this spec references a built-in evaluator.</summary>
public bool IsBuiltin => this.BuiltinName is not null;
/// <summary>Gets whether this spec references a generated rubric evaluator.</summary>
public bool IsRubric => this.GeneratedRef is not null;
/// <summary>Gets whether this spec is valid (i.e. references either a built-in or a rubric).</summary>
/// <remarks>
/// Because <see cref="FoundryEvaluatorSpec"/> is a struct, <c>default(FoundryEvaluatorSpec)</c>
/// is a syntactically-valid but semantically-invalid value (both <see cref="BuiltinName"/> and
/// <see cref="GeneratedRef"/> are <see langword="null"/>). Call <see cref="EnsureValid"/> at
/// API boundaries to fail fast instead of NRE-ing later.
/// </remarks>
public bool IsValid => this.BuiltinName is not null || this.GeneratedRef is not null;
/// <summary>Validates that this spec references either a built-in evaluator or a rubric.</summary>
/// <param name="paramName">Parameter name used in the thrown <see cref="ArgumentException"/>.</param>
/// <exception cref="ArgumentException">Thrown when neither <see cref="BuiltinName"/> nor <see cref="GeneratedRef"/> is set.</exception>
public void EnsureValid(string? paramName = null)
{
if (!this.IsValid)
{
throw new ArgumentException(
$"Invalid {nameof(FoundryEvaluatorSpec)}: must be constructed with either a built-in evaluator name " +
$"or a {nameof(GeneratedEvaluatorRef)}. The default struct value is not a valid spec.",
paramName);
}
}
/// <summary>Implicit conversion from a built-in evaluator name.</summary>
public static implicit operator FoundryEvaluatorSpec(string builtinName) => new(builtinName);
/// <summary>Implicit conversion from a <see cref="GeneratedEvaluatorRef"/>.</summary>
public static implicit operator FoundryEvaluatorSpec(GeneratedEvaluatorRef generatedRef) => new(generatedRef);
/// <inheritdoc/>
public bool Equals(FoundryEvaluatorSpec other)
=> this.BuiltinName == other.BuiltinName
&& Equals(this.GeneratedRef, other.GeneratedRef);
/// <inheritdoc/>
public override bool Equals(object? obj) => obj is FoundryEvaluatorSpec other && this.Equals(other);
/// <inheritdoc/>
public override int GetHashCode()
=> HashCode.Combine(this.BuiltinName, this.GeneratedRef);
/// <summary>Equality operator.</summary>
public static bool operator ==(FoundryEvaluatorSpec left, FoundryEvaluatorSpec right) => left.Equals(right);
/// <summary>Inequality operator.</summary>
public static bool operator !=(FoundryEvaluatorSpec left, FoundryEvaluatorSpec right) => !left.Equals(right);
/// <inheritdoc/>
public override string ToString()
=> this.IsRubric
? $"GeneratedEvaluatorRef({this.GeneratedRef!.Name}@{this.GeneratedRef.Version ?? "latest"})"
: this.BuiltinName ?? "<empty>";
}
@@ -57,8 +57,9 @@ namespace Microsoft.Agents.AI;
/// <para>
/// <strong>Agent decorators (each enabled by default, individually disableable):</strong>
/// <list type="bullet">
/// <item><description><see cref="ToolApprovalAgent"/> — "don't ask again" tool approval rules enabling safe unattended execution. Disable with <see cref="HarnessAgentOptions.DisableToolApproval"/>.</description></item>
/// <item><description><see cref="ToolApprovalAgent"/> — "don't ask again" tool approval rules enabling safe unattended execution. Disable with <see cref="HarnessAgentOptions.DisableToolAutoApproval"/>.</description></item>
/// <item><description><see cref="OpenTelemetryAgent"/> — OpenTelemetry instrumentation following semantic conventions for generative AI. Disable with <see cref="HarnessAgentOptions.DisableOpenTelemetry"/>.</description></item>
/// <item><description><see cref="LoopAgent"/> — re-invokes the agent until the configured evaluators are satisfied. Applied as the outermost decorator (so each iteration is a complete agent run) and only when <see cref="HarnessAgentOptions.LoopEvaluators"/> supplies at least one evaluator; otherwise omitted.</description></item>
/// </list>
/// </para>
/// <para>
@@ -142,7 +143,19 @@ public sealed class HarnessAgent : DelegatingAIAgent
AIAgentBuilder builder = innerAgent.AsBuilder();
if (options?.DisableToolApproval is not true)
// Register the loop decorator first so it ends up outermost (AIAgentBuilder applies factories in reverse): the
// loop drives complete agent runs, each independently tool-approved and OpenTelemetry-traced. Only added when at
// least one evaluator is supplied; otherwise the agent behaves as a single-shot agent.
if (options?.LoopEvaluators is IEnumerable<LoopEvaluator> loopEvaluators)
{
List<LoopEvaluator> evaluatorList = loopEvaluators.ToList();
if (evaluatorList.Count > 0)
{
builder.Use((inner, _) => new LoopAgent(inner, evaluatorList, options.LoopAgentOptions, loggerFactory));
}
}
if (options?.DisableToolAutoApproval is not true)
{
builder.UseToolApproval(options?.ToolApprovalAgentOptions);
}
@@ -146,6 +146,33 @@ public sealed class HarnessAgentOptions
/// </remarks>
public IEnumerable<AIContextProvider>? AIContextProviders { get; set; }
/// <summary>
/// Gets or sets the ordered collection of <see cref="LoopEvaluator"/> instances that, when supplied, cause the
/// <see cref="HarnessAgent"/> to be wrapped in a <see cref="LoopAgent"/> decorator.
/// </summary>
/// <remarks>
/// <para>
/// When this collection is non-<see langword="null"/> and contains at least one evaluator, the harness agent is
/// wrapped in a <see cref="LoopAgent"/> that re-invokes the agent until the evaluators are satisfied. The loop is
/// applied as the outermost decorator, so each iteration is a complete agent run (including tool approval and
/// OpenTelemetry instrumentation).
/// </para>
/// <para>
/// When <see langword="null"/> or empty (the default), no <see cref="LoopAgent"/> is added and the agent behaves
/// as a single-shot agent.
/// </para>
/// </remarks>
public IEnumerable<LoopEvaluator>? LoopEvaluators { get; set; }
/// <summary>
/// Gets or sets optional configuration for the <see cref="LoopAgent"/> created from <see cref="LoopEvaluators"/>.
/// </summary>
/// <remarks>
/// When <see langword="null"/>, the <see cref="LoopAgent"/> uses its default settings. This property is ignored
/// when <see cref="LoopEvaluators"/> is <see langword="null"/> or empty.
/// </remarks>
public LoopAgentOptions? LoopAgentOptions { get; set; }
/// <summary>
/// Gets or sets the maximum number of function-invocation loop iterations per request.
/// </summary>
@@ -156,20 +183,22 @@ public sealed class HarnessAgentOptions
public int? MaximumIterationsPerRequest { get; set; }
/// <summary>
/// Gets or sets a value indicating whether the <see cref="ToolApprovalAgent"/> wrapper is disabled.
/// Gets or sets a value indicating whether the <see cref="ToolApprovalAgent"/> auto-approval middleware is disabled.
/// </summary>
/// <remarks>
/// When <see langword="false"/> (the default), the agent is wrapped with tool approval middleware
/// that supports "don't ask again" auto-approval rules.
/// This disables the tool auto-approval functionality only, keeping the tool approval flow requiring approval (for example,
/// <see cref="ApprovalRequiredAIFunction"/> tools). This setting controls whether the agent is wrapped with the
/// <see cref="ToolApprovalAgent"/> middleware that supports "don't ask again" and auto-approval rules.
/// When <see langword="false"/> (the default), the middleware is added.
/// </remarks>
public bool DisableToolApproval { get; set; }
public bool DisableToolAutoApproval { get; set; }
/// <summary>
/// Gets or sets the options for the <see cref="ToolApprovalAgent"/> middleware.
/// </summary>
/// <remarks>
/// When <see langword="null"/>, the <see cref="ToolApprovalAgent"/> uses default settings.
/// This property has no effect when <see cref="DisableToolApproval"/> is <see langword="true"/>.
/// This property has no effect when <see cref="DisableToolAutoApproval"/> is <see langword="true"/>.
/// </remarks>
public ToolApprovalAgentOptions? ToolApprovalAgentOptions { get; set; }
@@ -20,6 +20,8 @@ internal static class BuiltInFunctions
{
internal const string HttpPrefix = "http-";
internal const string McpToolPrefix = "mcptool-";
internal const string StatusFunctionSuffix = "-status";
internal const string RespondFunctionSuffix = "-respond";
private const string WaitForResponseHeaderName = "x-ms-wait-for-response";
@@ -90,7 +92,7 @@ internal static class BuiltInFunctions
}
OrchestrationMetadata? metadata = await client.GetInstanceAsync(runId, getInputsAndOutputs: true);
if (metadata is null)
if (metadata is null || !IsOrchestrationOwnedByWorkflow(metadata.Name, context.FunctionDefinition.Name, StatusFunctionSuffix))
{
return await CreateErrorResponseAsync(req, context, HttpStatusCode.NotFound, $"Workflow run '{runId}' not found.");
}
@@ -146,7 +148,7 @@ internal static class BuiltInFunctions
// Verify the orchestration exists and is in a valid state
OrchestrationMetadata? metadata = await client.GetInstanceAsync(runId, getInputsAndOutputs: true);
if (metadata is null)
if (metadata is null || !IsOrchestrationOwnedByWorkflow(metadata.Name, context.FunctionDefinition.Name, RespondFunctionSuffix))
{
return await CreateErrorResponseAsync(req, context, HttpStatusCode.NotFound, $"Workflow run '{runId}' not found.");
}
@@ -363,9 +365,10 @@ internal static class BuiltInFunctions
string agentName = context.Name;
// Derive session id: try to parse provided threadId, otherwise create a new one.
// Bind the caller-supplied threadId as a session key under the current agent name,
// mirroring the behavior of RunAgentHttpAsync.
AgentSessionId sessionId = context.Arguments.TryGetValue("threadId", out object? threadObj) && threadObj is string threadId && !string.IsNullOrWhiteSpace(threadId)
? AgentSessionId.Parse(threadId)
? new AgentSessionId(agentName, threadId)
: new AgentSessionId(agentName, functionContext.InvocationId);
AIAgent agentProxy = client.AsDurableAgentProxy(functionContext, agentName);
@@ -654,6 +657,39 @@ internal static class BuiltInFunctions
return functionName[HttpPrefix.Length..];
}
/// <summary>
/// Extracts the workflow name from the function definition name by stripping the
/// <see cref="HttpPrefix"/> and the given suffix (e.g., "-status" or "-respond").
/// </summary>
internal static string GetWorkflowName(string functionName, string suffix)
{
if (!functionName.StartsWith(HttpPrefix, StringComparison.Ordinal) ||
!functionName.EndsWith(suffix, StringComparison.Ordinal))
{
throw new InvalidOperationException(
$"Built-in HTTP trigger function name '{functionName}' does not match the expected pattern '{HttpPrefix}<workflowName>{suffix}'.");
}
return functionName[HttpPrefix.Length..^suffix.Length];
}
/// <summary>
/// Returns true if the orchestration name matches the expected orchestration for the
/// workflow derived from the given function name and suffix.
/// </summary>
internal static bool IsOrchestrationOwnedByWorkflow(string orchestrationName, string functionName, string suffix)
{
if (!functionName.StartsWith(HttpPrefix, StringComparison.Ordinal) ||
!functionName.EndsWith(suffix, StringComparison.Ordinal))
{
return false;
}
string workflowName = GetWorkflowName(functionName, suffix);
string expectedOrchestrationName = WorkflowNamingHelper.ToOrchestrationFunctionName(workflowName);
return string.Equals(orchestrationName, expectedOrchestrationName, StringComparison.OrdinalIgnoreCase);
}
/// <summary>
/// Represents a request to run an agent.
/// </summary>
@@ -2,6 +2,8 @@
## [Unreleased]
- Scope workflow status/respond endpoints to the route workflow name ([#6608](https://github.com/microsoft/agent-framework/pull/6608))
- Bind MCP threadId to the current agent and guard cross-agent session dispatch ([#6531](https://github.com/microsoft/agent-framework/pull/6531))
- Support returning workflow results from HTTP trigger endpoint ([#5321](https://github.com/microsoft/agent-framework/pull/5321))
- Added MCP tool trigger support for durable workflows ([#4768](https://github.com/microsoft/agent-framework/pull/4768))
- Added Azure Functions hosting support for durable workflows ([#4436](https://github.com/microsoft/agent-framework/pull/4436))
@@ -89,7 +89,7 @@ internal sealed class DurableWorkflowsFunctionMetadataTransformer : IFunctionMet
// Register a status endpoint if opted in via AddWorkflow(exposeStatusEndpoint: true).
if (this._options.IsStatusEndpointEnabled(workflow.Key))
{
string statusFunctionName = $"{BuiltInFunctions.HttpPrefix}{workflow.Key}-status";
string statusFunctionName = $"{BuiltInFunctions.HttpPrefix}{workflow.Key}{BuiltInFunctions.StatusFunctionSuffix}";
if (registeredFunctions.Add(statusFunctionName))
{
this._logger.LogRegisteringWorkflowTrigger(workflow.Key, statusFunctionName, "http-status");
@@ -105,7 +105,7 @@ internal sealed class DurableWorkflowsFunctionMetadataTransformer : IFunctionMet
bool hasRequestPorts = workflow.Value.ReflectExecutors().Values.Any(b => b is RequestPortBinding);
if (hasRequestPorts)
{
string respondFunctionName = $"{BuiltInFunctions.HttpPrefix}{workflow.Key}-respond";
string respondFunctionName = $"{BuiltInFunctions.HttpPrefix}{workflow.Key}{BuiltInFunctions.RespondFunctionSuffix}";
if (registeredFunctions.Add(respondFunctionName))
{
this._logger.LogRegisteringWorkflowTrigger(workflow.Key, respondFunctionName, "http-respond");
@@ -0,0 +1,29 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
namespace Microsoft.Agents.AI.LocalCodeAct;
/// <summary>
/// Exception thrown when AST validation of generated Python code fails.
/// </summary>
public sealed class CodeValidationException : Exception
{
/// <summary>Initializes a new instance of the <see cref="CodeValidationException"/> class.</summary>
public CodeValidationException()
{
}
/// <summary>Initializes a new instance of the <see cref="CodeValidationException"/> class.</summary>
/// <param name="message">Validation error message.</param>
public CodeValidationException(string message) : base(message)
{
}
/// <summary>Initializes a new instance of the <see cref="CodeValidationException"/> class.</summary>
/// <param name="message">Validation error message.</param>
/// <param name="innerException">Underlying exception.</param>
public CodeValidationException(string message, Exception innerException) : base(message, innerException)
{
}
}
@@ -0,0 +1,64 @@
// Copyright (c) Microsoft. All rights reserved.
using Microsoft.Shared.Diagnostics;
namespace Microsoft.Agents.AI.LocalCodeAct;
/// <summary>
/// File mount access mode.
/// </summary>
public enum FileMountMode
{
/// <summary>Read-only access. Files are not scanned for capture after execution.</summary>
ReadOnly,
/// <summary>Read-write access. New or modified files are captured after execution.</summary>
ReadWrite,
}
/// <summary>
/// Represents a host directory exposed to locally executed code.
/// </summary>
/// <remarks>
/// <para>
/// Unlike a true sandbox, mounts in this package expose <see cref="HostPath"/>
/// directly to the subprocess. The <see cref="MountPath"/> is metadata used to
/// describe the mount to the model in the function description and to label
/// captured files. Real isolation must come from the surrounding sandbox
/// (container, VM, Foundry hosted agent, etc.).
/// </para>
/// </remarks>
public sealed class FileMount
{
/// <summary>
/// Initializes a new instance of the <see cref="FileMount"/> class.
/// </summary>
/// <param name="hostPath">Path on the host filesystem to expose to the subprocess. Must exist.</param>
/// <param name="mountPath">
/// Logical path used to describe the mount to the model (for example <c>"/input/data.csv"</c>).
/// </param>
/// <param name="mode">Access mode for the mount. Defaults to <see cref="FileMountMode.ReadWrite"/>.</param>
/// <param name="writeBytesLimit">
/// Optional per-mount write capture limit (in bytes). When <see langword="null"/>, the global
/// <see cref="ProcessExecutionLimits.MaxCapturedFileBytes"/> applies.
/// </param>
public FileMount(string hostPath, string mountPath, FileMountMode mode = FileMountMode.ReadWrite, long? writeBytesLimit = null)
{
this.HostPath = Throw.IfNullOrWhitespace(hostPath);
this.MountPath = Throw.IfNullOrWhitespace(mountPath);
this.Mode = mode;
this.WriteBytesLimit = writeBytesLimit;
}
/// <summary>Gets the host filesystem path exposed to the subprocess.</summary>
public string HostPath { get; }
/// <summary>Gets the logical mount path used to describe the mount to the model.</summary>
public string MountPath { get; }
/// <summary>Gets the access mode for the mount.</summary>
public FileMountMode Mode { get; }
/// <summary>Gets the optional per-mount write capture limit (in bytes).</summary>
public long? WriteBytesLimit { get; }
}
@@ -0,0 +1,107 @@
// Copyright (c) Microsoft. All rights reserved.
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.LocalCodeAct.Internal;
/// <summary>
/// Coordinates a single execution: optional validation, snapshot of writable mounts,
/// running the subprocess, capturing written files, and assembling the final content list.
/// </summary>
internal sealed class CodeExecutor
{
private readonly string _pythonExecutable;
private readonly string _runnerScript;
private readonly CodeValidator? _validator;
private readonly ProcessExecutionLimits _limits;
private readonly IReadOnlyDictionary<string, string>? _environment;
private readonly string? _workingDirectory;
public CodeExecutor(
string pythonExecutable,
string runnerScript,
CodeValidator? validator,
ProcessExecutionLimits limits,
IReadOnlyDictionary<string, string>? environment,
string? workingDirectory)
{
this._pythonExecutable = pythonExecutable;
this._runnerScript = runnerScript;
this._validator = validator;
this._limits = limits;
this._environment = environment;
this._workingDirectory = workingDirectory;
}
/// <summary>Immutable snapshot of provider state captured at the start of an invocation.</summary>
public sealed class RunSnapshot
{
public RunSnapshot(IReadOnlyList<AIFunction> tools, IReadOnlyList<FileMount> fileMounts)
{
this.Tools = tools;
this.FileMounts = fileMounts;
}
public IReadOnlyList<AIFunction> Tools { get; }
public IReadOnlyList<FileMount> FileMounts { get; }
}
public async Task<List<AIContent>> ExecuteAsync(RunSnapshot snapshot, string code, CancellationToken cancellationToken)
{
if (this._validator is not null)
{
await this._validator.ValidateAsync(code, cancellationToken).ConfigureAwait(false);
}
var preState = FileMountHelper.SnapshotWritableMounts(snapshot.FileMounts);
var bridge = new ProcessBridge(
this._pythonExecutable,
this._runnerScript,
snapshot.Tools,
this._limits,
this._environment,
this._workingDirectory);
var result = await bridge.RunAsync(code, cancellationToken).ConfigureAwait(false);
var captured = FileMountHelper.CaptureWrittenFiles(snapshot.FileMounts, preState, this._limits);
return BuildContents(result, captured);
}
private static List<AIContent> BuildContents(ProcessBridge.ExecutionResult result, List<AIContent> capturedFiles)
{
var contents = new List<AIContent>();
if (!string.IsNullOrEmpty(result.Stdout))
{
var stdoutText = result.StdoutTruncated ? result.Stdout + "\n[stdout truncated]" : result.Stdout;
contents.Add(new TextContent(stdoutText));
}
if (!string.IsNullOrEmpty(result.Stderr))
{
var stderrText = result.StderrTruncated ? result.Stderr + "\n[stderr truncated]" : result.Stderr;
contents.Add(new TextContent("stderr:\n" + stderrText));
}
if (result.OutputPresent && result.Output.HasValue)
{
contents.Add(new TextContent("result:\n" + result.Output.Value.GetRawText()));
}
contents.AddRange(capturedFiles);
if (contents.Count == 0)
{
contents.Add(new TextContent("Code executed successfully without output."));
}
return contents;
}
}
@@ -0,0 +1,178 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Text.Json;
using System.Text.Json.Nodes;
using System.Threading;
using System.Threading.Tasks;
namespace Microsoft.Agents.AI.LocalCodeAct.Internal;
/// <summary>
/// Runs the embedded Python AST validator in a child process with a strict timeout.
/// </summary>
internal sealed class CodeValidator
{
private readonly string _pythonExecutable;
private readonly string _validatorScript;
private readonly TimeSpan _timeout;
private readonly IReadOnlyList<string>? _allowedImports;
private readonly IReadOnlyList<string>? _blockedImports;
private readonly IReadOnlyList<string>? _allowedBuiltins;
private readonly IReadOnlyList<string>? _blockedBuiltins;
public CodeValidator(
string pythonExecutable,
string validatorScript,
TimeSpan timeout,
IReadOnlyList<string>? allowedImports,
IReadOnlyList<string>? blockedImports,
IReadOnlyList<string>? allowedBuiltins,
IReadOnlyList<string>? blockedBuiltins)
{
this._pythonExecutable = pythonExecutable;
this._validatorScript = validatorScript;
this._timeout = timeout;
this._allowedImports = allowedImports;
this._blockedImports = blockedImports;
this._allowedBuiltins = allowedBuiltins;
this._blockedBuiltins = blockedBuiltins;
}
/// <summary>Validates Python source code against the configured allow-lists.</summary>
/// <exception cref="CodeValidationException">Thrown when validation fails.</exception>
public async Task ValidateAsync(string code, CancellationToken cancellationToken)
{
var request = new JsonObject
{
["code"] = code,
};
AddList(request, "allowed_imports", this._allowedImports);
AddList(request, "blocked_imports", this._blockedImports);
AddList(request, "allowed_builtins", this._allowedBuiltins);
AddList(request, "blocked_builtins", this._blockedBuiltins);
var requestJson = request.ToJsonString();
var startInfo = new ProcessStartInfo
{
FileName = this._pythonExecutable,
UseShellExecute = false,
RedirectStandardInput = true,
RedirectStandardOutput = true,
RedirectStandardError = true,
CreateNoWindow = true,
};
startInfo.ArgumentList.Add("-I");
startInfo.ArgumentList.Add(this._validatorScript);
using var process = Process.Start(startInfo)
?? throw new InvalidOperationException("Failed to start Python validator process.");
using var timeoutCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
timeoutCts.CancelAfter(this._timeout);
try
{
await process.StandardInput.WriteLineAsync(requestJson.AsMemory(), timeoutCts.Token).ConfigureAwait(false);
await process.StandardInput.FlushAsync(timeoutCts.Token).ConfigureAwait(false);
process.StandardInput.Close();
var stdoutTask = process.StandardOutput.ReadToEndAsync(timeoutCts.Token);
var stderrTask = process.StandardError.ReadToEndAsync(timeoutCts.Token);
await process.WaitForExitAsync(timeoutCts.Token).ConfigureAwait(false);
var stdout = await stdoutTask.ConfigureAwait(false);
var stderr = await stderrTask.ConfigureAwait(false);
if (process.ExitCode == 0)
{
return;
}
throw new CodeValidationException(ExtractError(stdout, stderr));
}
catch (OperationCanceledException) when (timeoutCts.IsCancellationRequested && !cancellationToken.IsCancellationRequested)
{
TryKill(process);
throw new CodeValidationException($"Code validation exceeded {this._timeout.TotalSeconds:F0} seconds.");
}
catch
{
TryKill(process);
throw;
}
}
private static string ExtractError(string output, string errorOutput)
{
if (string.IsNullOrWhiteSpace(output))
{
return string.IsNullOrWhiteSpace(errorOutput) ? "Code validation failed." : errorOutput;
}
try
{
using var doc = JsonDocument.Parse(output);
if (doc.RootElement.TryGetProperty("errors", out var errors) && errors.ValueKind == JsonValueKind.Array)
{
var sb = new StringBuilder();
foreach (var err in errors.EnumerateArray())
{
if (sb.Length > 0)
{
sb.Append("; ");
}
sb.Append(err.ValueKind == JsonValueKind.String ? err.GetString() : err.ToString());
}
return sb.Length > 0 ? sb.ToString() : output;
}
if (doc.RootElement.TryGetProperty("message", out var message) && message.ValueKind == JsonValueKind.String)
{
return message.GetString() ?? output;
}
}
catch (JsonException)
{
// fall through
}
return output;
}
private static void TryKill(Process process)
{
try
{
if (!process.HasExited)
{
process.Kill(entireProcessTree: true);
}
}
catch
{
#pragma warning disable CA1031 // Do not catch general exception types
// best-effort cleanup
#pragma warning restore CA1031
}
}
private static void AddList(JsonObject obj, string key, IReadOnlyList<string>? values)
{
if (values is null)
{
return;
}
obj[key] = new JsonArray(values.Select(v => (JsonNode?)JsonValue.Create(v)).ToArray());
}
}
@@ -0,0 +1,60 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.IO;
namespace Microsoft.Agents.AI.LocalCodeAct.Internal;
/// <summary>
/// Extracts the embedded Python <c>runner.py</c> and <c>validator.py</c> scripts to a temporary
/// directory and caches their paths for the lifetime of the process.
/// </summary>
internal static class EmbeddedScripts
{
private static readonly object s_syncRoot = new();
private static string? s_runnerPath;
private static string? s_validatorPath;
/// <summary>Returns the path to the embedded <c>runner.py</c>, extracting it on first access.</summary>
public static string GetRunnerScriptPath() => GetOrExtract("runner.py", ref s_runnerPath);
/// <summary>Returns the path to the embedded <c>validator.py</c>, extracting it on first access.</summary>
public static string GetValidatorScriptPath() => GetOrExtract("validator.py", ref s_validatorPath);
private static string GetOrExtract(string fileName, ref string? cached)
{
if (cached is not null && File.Exists(cached))
{
return cached;
}
lock (s_syncRoot)
{
if (cached is not null && File.Exists(cached))
{
return cached;
}
var path = Extract(fileName);
cached = path;
return path;
}
}
private static string Extract(string fileName)
{
var assembly = typeof(EmbeddedScripts).Assembly;
var resourceName = $"Microsoft.Agents.AI.LocalCodeAct.Resources.{fileName}";
using var stream = assembly.GetManifestResourceStream(resourceName)
?? throw new InvalidOperationException($"Embedded resource '{resourceName}' not found.");
var dir = Path.Combine(Path.GetTempPath(), "agentframework-localcodeact-" + Guid.NewGuid().ToString("N"));
Directory.CreateDirectory(dir);
var path = Path.Combine(dir, fileName);
using var fileStream = File.Create(path);
stream.CopyTo(fileStream);
return path;
}
}
@@ -0,0 +1,51 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.ComponentModel;
using System.Text.Json;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.LocalCodeAct.Internal;
/// <summary>
/// Run-scoped <see cref="AIFunction"/> that exposes <c>execute_code</c> to the model.
/// </summary>
internal sealed class ExecuteCodeFunction : AIFunction
{
private const string ExecuteCodeName = "execute_code";
private readonly CodeExecutor _executor;
private readonly CodeExecutor.RunSnapshot _snapshot;
private readonly AIFunction _inner;
public ExecuteCodeFunction(CodeExecutor executor, CodeExecutor.RunSnapshot snapshot, string description)
{
this._executor = executor;
this._snapshot = snapshot;
this._inner = AIFunctionFactory.Create(
this.ExecuteCodeAsync,
new AIFunctionFactoryOptions
{
Name = ExecuteCodeName,
Description = description,
});
}
public override string Name => this._inner.Name;
public override string Description => this._inner.Description;
public override JsonElement JsonSchema => this._inner.JsonSchema;
protected override ValueTask<object?> InvokeCoreAsync(AIFunctionArguments arguments, CancellationToken cancellationToken) =>
this._inner.InvokeAsync(arguments, cancellationToken);
private async ValueTask<object?> ExecuteCodeAsync(
[Description("Python source code to execute locally in the agent environment.")] string code,
CancellationToken cancellationToken)
=> string.IsNullOrWhiteSpace(code)
? throw new ArgumentException("Parameter 'code' must not be empty.", nameof(code))
: await this._executor.ExecuteAsync(this._snapshot, code, cancellationToken).ConfigureAwait(false);
}
@@ -0,0 +1,270 @@
// Copyright (c) Microsoft. All rights reserved.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Microsoft.Extensions.AI;
namespace Microsoft.Agents.AI.LocalCodeAct.Internal;
/// <summary>
/// Filesystem helpers for read-write mount snapshotting and capture.
/// </summary>
internal static class FileMountHelper
{
/// <summary>Normalizes and validates a mount path (must be a clean absolute POSIX-style path).</summary>
public static string NormalizeMountPath(string mountPath)
{
if (string.IsNullOrWhiteSpace(mountPath))
{
throw new ArgumentException("Mount path must not be empty.", nameof(mountPath));
}
var raw = mountPath.Trim().Replace('\\', '/');
var parts = raw.Split('/', StringSplitOptions.RemoveEmptyEntries)
.Where(p => p != ".")
.ToList();
if (parts.Any(p => p == ".."))
{
throw new ArgumentException("Mount path must not contain '..' segments.", nameof(mountPath));
}
if (parts.Count == 0)
{
throw new ArgumentException("Mount path must point to a concrete absolute path.", nameof(mountPath));
}
return "/" + string.Join("/", parts);
}
/// <summary>
/// Validates a FileMount and returns a normalized copy (resolved host path, normalized mount path).
/// </summary>
public static FileMount Normalize(FileMount mount)
{
if (mount is null)
{
throw new ArgumentNullException(nameof(mount));
}
if (string.IsNullOrWhiteSpace(mount.HostPath))
{
throw new ArgumentException("HostPath must not be empty.", nameof(mount));
}
var fullHost = Path.GetFullPath(mount.HostPath);
if (!Directory.Exists(fullHost) && !File.Exists(fullHost))
{
throw new DirectoryNotFoundException($"FileMount host path '{mount.HostPath}' does not exist.");
}
if (mount.WriteBytesLimit.HasValue && mount.WriteBytesLimit.Value < 0)
{
throw new ArgumentException("WriteBytesLimit must be non-negative when set.", nameof(mount));
}
return new FileMount(fullHost, NormalizeMountPath(mount.MountPath), mount.Mode, mount.WriteBytesLimit);
}
/// <summary>Snapshot of (size, last-write-time ticks) per relative path under a writable mount.</summary>
public sealed class MountSnapshot
{
public MountSnapshot(IReadOnlyDictionary<string, (long Size, long Ticks)> files)
{
this.Files = files;
}
public IReadOnlyDictionary<string, (long Size, long Ticks)> Files { get; }
}
/// <summary>Captures the current file inventory of read-write mounts before execution.</summary>
public static Dictionary<string, MountSnapshot> SnapshotWritableMounts(IReadOnlyList<FileMount> mounts)
{
var snapshot = new Dictionary<string, MountSnapshot>(StringComparer.Ordinal);
foreach (var mount in mounts)
{
if (mount.Mode != FileMountMode.ReadWrite)
{
continue;
}
var root = new DirectoryInfo(mount.HostPath);
if (!root.Exists)
{
snapshot[mount.MountPath] = new MountSnapshot(new Dictionary<string, (long, long)>());
continue;
}
var files = new Dictionary<string, (long Size, long Ticks)>(StringComparer.Ordinal);
foreach (var file in EnumerateRealFiles(root))
{
var rel = MakeRelative(root.FullName, file.FullName);
files[rel] = (file.Length, file.LastWriteTimeUtc.Ticks);
}
snapshot[mount.MountPath] = new MountSnapshot(files);
}
return snapshot;
}
/// <summary>Captures files that were created or modified in read-write mounts since the snapshot was taken.</summary>
public static List<AIContent> CaptureWrittenFiles(
IReadOnlyList<FileMount> mounts,
IReadOnlyDictionary<string, MountSnapshot> preState,
ProcessExecutionLimits limits)
{
var captured = new List<AIContent>();
long totalBytes = 0;
foreach (var mount in mounts)
{
if (mount.Mode != FileMountMode.ReadWrite)
{
continue;
}
var root = new DirectoryInfo(mount.HostPath);
if (!root.Exists)
{
continue;
}
preState.TryGetValue(mount.MountPath, out var before);
var beforeFiles = before?.Files ?? new Dictionary<string, (long, long)>();
long mountBytes = 0;
var perMountLimit = mount.WriteBytesLimit ?? limits.MaxCapturedFileBytes;
foreach (var file in EnumerateRealFiles(root).OrderBy(f => f.FullName, StringComparer.Ordinal))
{
var rel = MakeRelative(root.FullName, file.FullName);
var current = (file.Length, file.LastWriteTimeUtc.Ticks);
if (beforeFiles.TryGetValue(rel, out var previous) && previous == current)
{
continue;
}
var sandboxPath = mount.MountPath.TrimEnd('/') + "/" + rel;
if (file.Length > limits.MaxCapturedFileBytes)
{
captured.Add(new TextContent($"[file {sandboxPath} omitted: exceeds per-file capture limit]"));
continue;
}
if (mountBytes + file.Length > perMountLimit)
{
captured.Add(new TextContent($"[file {sandboxPath} omitted: per-mount capture limit reached]"));
continue;
}
if (totalBytes + file.Length > limits.MaxTotalCapturedFileBytes)
{
captured.Add(new TextContent($"[file {sandboxPath} omitted: total capture limit reached]"));
continue;
}
byte[] data;
try
{
data = File.ReadAllBytes(file.FullName);
}
catch (IOException)
{
continue;
}
catch (UnauthorizedAccessException)
{
continue;
}
captured.Add(new DataContent(data, GuessMediaType(file.Name))
{
AdditionalProperties = new AdditionalPropertiesDictionary
{
["path"] = sandboxPath,
},
});
mountBytes += file.Length;
totalBytes += file.Length;
}
}
return captured;
}
private static string MakeRelative(string root, string full)
{
var rel = Path.GetRelativePath(root, full);
return rel.Replace(Path.DirectorySeparatorChar, '/');
}
private static IEnumerable<FileInfo> EnumerateRealFiles(DirectoryInfo root)
{
var stack = new Stack<DirectoryInfo>();
stack.Push(root);
while (stack.Count > 0)
{
var current = stack.Pop();
FileSystemInfo[] entries;
try
{
entries = current.GetFileSystemInfos();
}
catch (IOException)
{
continue;
}
foreach (var entry in entries)
{
if (entry.Attributes.HasFlag(FileAttributes.ReparsePoint))
{
continue;
}
if (entry is DirectoryInfo dir)
{
stack.Push(dir);
}
else if (entry is FileInfo file)
{
yield return file;
}
}
}
}
private static string GuessMediaType(string fileName)
{
#pragma warning disable CA1308 // Normalize strings to uppercase - file extensions are conventionally lowercase
var extension = Path.GetExtension(fileName).ToLowerInvariant();
#pragma warning restore CA1308
return extension switch
{
".txt" => "text/plain",
".json" => "application/json",
".xml" => "application/xml",
".html" => "text/html",
".css" => "text/css",
".js" => "application/javascript",
".png" => "image/png",
".jpg" or ".jpeg" => "image/jpeg",
".gif" => "image/gif",
".svg" => "image/svg+xml",
".pdf" => "application/pdf",
".zip" => "application/zip",
".csv" => "text/csv",
".md" => "text/markdown",
".py" => "text/x-python",
".cs" => "text/x-csharp",
_ => "application/octet-stream",
};
}
}

Some files were not shown because too many files have changed in this diff Show More