Commit Graph

270 Commits

Author SHA1 Message Date
Eduard van Valkenburg 54b390dd17 Python: add agent-framework-hosting-mcp channel (#6305)
* feat(python): add agent-framework-hosting-mcp channel

Add a hosting channel that exposes the host target (agent or workflow)
as a single Model Context Protocol tool over Streamable HTTP. The tool
invocation routes through the host pipeline (ChannelContext.run/
run_stream) so sessions, linking, and run/response hooks apply. Maps the
MCP request context to a ChannelSession isolation key and ChannelIdentity,
and forwards streaming output as MCP progress notifications.

Includes tests, README, and workspace registration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address MCP hosting channel review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:14:09 +02:00
Eduard van Valkenburg 81db307dad Python: add agent-framework-hosting-a2a channel (#6306)
* feat(python): add agent-framework-hosting-a2a channel

Add a hosting channel that exposes the host target (agent or workflow)
as a peer agent over the Agent-to-Agent (A2A) protocol (JSON-RPC plus a
served agent card). Requests are handled by a host-routed
HostAgentExecutor that drives the host pipeline (ChannelContext.run/
run_stream) instead of wrapping the target directly, so sessions,
linking, and run/response hooks apply. Maps the A2A conversation/context
id to a ChannelSession isolation key and the caller to a ChannelIdentity;
streaming emits incremental task artifacts.

Includes tests, README, and workspace registration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address A2A hosting channel review feedback

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:14:09 +02:00
Eduard van Valkenburg 109cff4d20 Simplify Python hosting core (#6492)
Remove linking, multicast, durable delivery, and host push machinery from the v1 hosting core. Keep those scenarios in a proposed follow-up ADR and update channel packages, samples, docs, tests, and workspace metadata around the smaller host/channel contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:14:09 +02:00
Eduard van Valkenburg db68bc65f3 Python: add agent-framework-hosting-discord channel (#6081)
* Add Discord hosting channel

Add an alpha agent-framework-hosting-discord package backed by Discord HTTP Interactions. The channel verifies signed slash-command requests, registers commands, runs hosted agents and ChannelCommand handlers, supports originating response hooks, streams by editing the original interaction response, and can push through Discord channel ids.

Factor standard channel response-hook context application into hosting core so both host fan-out and originating channel replies use one helper.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Discord review chunking feedback

Ensure Discord command replies are chunked and streaming preview edits stay under Discord's content limit while final streamed replies continue through the chunked reply path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* small fix in init

* updated lock

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:14:08 +02:00
Eduard van Valkenburg 3af116e485 Python: add agent-framework-hosting-entra identity-link helpers (#5644)
* feat(hosting-entra): add Entra (Azure AD) identity-linking channel

New ``agent-framework-hosting-entra`` package implementing a Microsoft
Entra OAuth-based identity-linking channel for the Hosting framework.
Mounts a small set of routes (``/entra/login``, ``/entra/callback``,
``/entra/whoami``) that walk a user through an Entra/Azure AD
authorization-code flow and stick the resulting verified identity
(``oid`` / ``email`` / ``tid``) onto the host's identity table so
later requests on any other channel (Responses, Telegram, …) can be
linked to the same user.

Surface (re-exported from ``agent_framework_hosting_entra``):

- ``EntraChannel`` -- concrete ``Channel`` implementation. Owns the
  three Starlette routes, signs/verifies short-lived ``state`` tokens
  to bind the round-trip to the originating channel, exchanges the
  authorization code for an ID token via MSAL, and writes the
  verified identity into the host's identity store via the standard
  ``ChannelIdentity`` plumbing so cross-channel push (e.g. send a
  Telegram message to the user who completed the link from
  Responses) works without the channels having to coordinate
  directly.
- 14 unit tests covering route wiring, ``state`` issue / verify,
  callback exchange happy + failure paths, and identity-store write.

Registers the package in ``python/pyproject.toml``
``[tool.uv.sources]`` and adds the matching pyright
``executionEnvironments`` entry. Stacks on PR-2 (Hosting core);
independent of PR-3 / PR-4 / PR-6.

The cross-channel sample (``local_identity_link/``) that demonstrates
this end-to-end alongside Responses + Telegram lands in PR-8 (samples).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-entra): close IDOR + reflected-XSS + open-redirect on the OAuth flow

Three SECURITY-CRITICAL fixes flagged in round-2 review.

1. IDOR on /auth/start (3198518308). Without authentication the
   endpoint accepted (channel, channel_id) from the query string and
   bound *whoever signed in* to that pair. An attacker could bind
   their own Entra oid to a victim's per-channel id (e.g.
   `telegram:<victim_chat_id>`), redirecting all of the victim's
   future inbound traffic to the attacker's isolation key.

   Fix: introduce link_token_secret + mint_start_url(channel, id, ...).
   When set, /auth/start requires `exp` + `sig` (HMAC-SHA256 over
   `channel|channel_id|expires_at`) before issuing the redirect.
   Channels that hand out start URLs (a Telegram /link command after
   verifying the inbound webhook signature) call mint_start_url so
   the token proves the (channel, id) pair was authorised by the
   channel that owns the surface. Unsigned mode is opt-in and logs a
   loud WARNING at startup *and* on every accepted request.

2. Reflected XSS on /auth/callback (3198520256, 3198527896). `error`,
   `error_description`, channel_key (from the unauthenticated /start
   query), and `upn` (from a Graph response) flowed straight into the
   text/html response body unescaped. With the IDOR above, an
   attacker could stash `<script>` payloads in `channel` or `id` and
   serve them from the auth host's origin (full XSS on the auth
   surface — cookies/storage of anything else mounted there).

   Fix: html.escape() every value before HTML output.

3. Open redirect on `return_to` (3198524746). Accepted any URL.

   Fix: `_validate_return_to` allows only relative paths starting
   with `/` (and not `//`) or absolute URLs whose host equals the
   configured `public_base_url` host. Validated at /start mint time
   AND defensively re-validated at /callback before redirect.

12 new tests cover signed-token rejection (missing/forged/expired),
mint helper requirements, startup warning visibility, XSS escaping
on both error and success paths, and the open-redirect allowlist
(external rejected, relative accepted, same-origin accepted,
protocol-relative `//evil.example/` rejected).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(hosting): drop redundant @pytest.mark.asyncio decorators

asyncio_mode = "auto" is configured in pyproject.toml across the
hosting packages, so individual @pytest.mark.asyncio decorators are
unnecessary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:13:41 +02:00
Eduard van Valkenburg eec0714d12 Python: add agent-framework-hosting-activity-protocol channel (#5641)
* feat(hosting-activity-protocol): rename Bot Framework channel to ActivityProtocolChannel

The existing Bot-Framework-via-Azure-Bot-Service channel was previously
shipped under the name ``hosting-teams`` / ``TeamsChannel``. That name
is misleading for what the channel actually does -- it speaks the Bot
Framework Activity Protocol against Azure Bot Service, which fans out
across MS Teams, Slack, Webex, Telegram-via-Bot-Service, etc., and does
not provide any Teams-specific affordances.

This PR renames the package atomically and frees the ``hosting-teams``
name for a future Teams-native channel built on
``microsoft-teams-apps`` (PR-5b, spec req #28).

Renames (all in one commit):

- Package: ``agent-framework-hosting-teams`` ->
  ``agent-framework-hosting-activity-protocol``
- Module: ``agent_framework_hosting_teams`` ->
  ``agent_framework_hosting_activity_protocol``
- Channel class: ``TeamsChannel`` -> ``ActivityProtocolChannel``
- Helper: ``teams_isolation_key`` -> ``activity_protocol_isolation_key``
  (isolation key prefix ``teams:`` -> ``activity:``)
- Channel name: ``"teams"`` -> ``"activity"``; default mount path
  ``/teams`` -> ``/activity``
- Internal helper: ``_parse_teams_activity`` -> ``_parse_activity``
- Worker task name + a couple of error strings updated for consistency

Updates README.md and the module docstring to call out:

- this is the channel-neutral Activity Protocol channel,
- it surfaces what every Bot-Service-connected channel has in common
  (text in / text out),
- a forthcoming ``agent-framework-hosting-teams`` package will layer
  Teams-specific affordances (adaptive cards, message extensions,
  dialogs, SSO, ...) on the same Bot Service transport.

Workspace: registers ``agent-framework-hosting-activity-protocol`` in
``python/pyproject.toml`` and adds the matching pyright
``executionEnvironments`` entry.

Behavior is unchanged. Pyright + mypy clean, 11 tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* review: address PR-5 round 2 feedback

- security (#3198327004): add `service_url_allowed_hosts` constructor
  option (default `botframework.com` + `smba.trafficmanager.net`) and
  reject inbound activities whose `serviceUrl` host falls outside it
  with HTTP 400 — without this gate a malicious caller could redirect
  outbound replies (and the attached bearer token) to an
  attacker-controlled host
- security (#3198324219): add `inbound_auth_validator` async callback;
  log a loud WARNING at startup when no validator AND no operator
  reverse-proxy is configured so the dev-mode bypass cannot
  accidentally ship to production. Document the contract: prototype
  intentionally does not ship JWT validation (out of scope); operators
  must plug a validator or terminate auth in front of the channel
- retry semantics (#3198328746): distinguish transient outbound
  failures (httpx network errors, non-2xx from Bot Service) — return
  502 so Bot Service retries — from deterministic agent failures —
  return 200 so Bot Service does not retry the same broken activity
  in a loop
- bug (#3198330424): fix the placeholder-failure deadlock. When
  `send_initial_placeholder` fails, `activity_id` stays `None`, the
  edit-worker loop exit condition (`accumulated == last_sent`) is
  unreachable while no PUT is possible, and the worker would deadlock
  on `wake.wait()` forever after `worker_done` is set. Now: skip the
  worker entirely on placeholder failure and POST a single final
  activity at the end with whatever accumulated
- tests (#3198334465, #3187178091, #3198336045): add coverage for
  - `_is_service_url_allowed` allow/deny matrix + webhook 400 on
    disallowed serviceUrl
  - `inbound_auth_validator` allow/deny/raises paths
  - outbound `Authorization: Bearer <token>` header presence in
    production mode and absence in dev mode
  - the streaming path (`_stream_to_conversation`): placeholder +
    final edit, placeholder-failure fallback (with timeout guard
    against deadlock regression), and empty-stream `(no response)`
    placeholder replacement
  - retry-signal differentiation: outbound `httpx.ConnectError` →
    502; deterministic `ValueError` from the agent → 200

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(hosting): drop redundant @pytest.mark.asyncio decorators

asyncio_mode = "auto" is configured in pyproject.toml across the
hosting packages, so individual @pytest.mark.asyncio decorators are
unnecessary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting-activity-protocol): add response hooks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting-activity-protocol): mark constructor keyword args

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:13:40 +02:00
Eduard van Valkenburg fbcf0619d5 Python: add agent-framework-hosting-telegram channel (#5643)
* feat(hosting-telegram): add Telegram channel package

New ``agent-framework-hosting-telegram`` package implementing the
Telegram Bot API channel for the Hosting framework. Mounts a webhook
endpoint (``POST /telegram/webhook``) and an in-process polling loop
onto an ``AgentFrameworkHost`` and translates Telegram ``Update``
payloads to/from the channel-neutral ``ChannelRequest`` /
``HostedRunResult`` plumbing.

Surface (re-exported from ``agent_framework_hosting_telegram``):

- ``TelegramChannel`` -- concrete ``Channel`` implementation. Owns the
  webhook route + an optional ``getUpdates`` long-polling lifespan,
  parses Telegram ``Update``s into ``ChannelRequest`` (text, photo,
  document, voice, callback_query, …), runs the optional
  ``ChannelRunHook``, calls back into the ``ChannelContext`` to invoke
  the agent target, and posts the response back via
  ``sendMessage`` / ``sendChatAction`` / ``answerCallbackQuery`` on the
  Telegram Bot API. Honours ``DeliveryReport.include_originating`` so
  cross-channel pushes can target the originating Telegram chat
  without double-acking.
- Native fields the channel doesn't lift onto ``ChannelRequest`` (e.g.
  ``chat.type``, ``message.message_id``, ``callback_query.data``) are
  attached to ``ChannelRequest.attributes`` so a ``ChannelRunHook``
  can pick them up via the standard ``protocol_request=`` kwarg.
- 13 unit tests covering route wiring, ``Update`` parsing across the
  common content shapes, hook composition, and originating vs
  non-originating delivery branches.

Registers the package in ``python/pyproject.toml``
``[tool.uv.sources]`` and adds the matching pyright
``executionEnvironments`` entry. Stacks on PR-2 (Hosting core);
independent of PR-3 / PR-4.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-telegram): preserve in-chat ordering, ack-before-run, drain shutdown

- Replace per-update task fan-out with per-chat asyncio.Queue + worker.
  Telegram only guarantees update ordering up to getUpdates; the
  previous code spawned one task per update, which broke ordering for
  adjacent updates in the same chat. Updates are now serialised per
  chat_id (so /start then "what's the weather" can't race) while
  different chats still process in parallel.

- Webhook handler now acks (200) immediately and runs the agent in
  the per-chat worker. Telegram redelivers any update the webhook
  doesn't 200 within ~60 seconds, so a streamed agent reply that runs
  longer than that previously triggered a retry storm and duplicate
  replies.

- _on_shutdown now drains everything: poll task → per-chat workers →
  webhook-spawned dispatcher tasks (the new ack-before-run path), then
  deletes the webhook + closes the HTTP client. Previously webhook
  tasks were not tracked at all, so an in-flight agent invocation
  could leak past app shutdown.

- _enqueue_update extracts chat_id from message / edited_message /
  callback_query; updates with no resolvable chat fall back to a
  one-shot dispatcher task that's still tracked in _update_tasks for
  shutdown.

- Webhook handler now also returns 400 on malformed JSON / non-object
  payloads instead of crashing the request.

4 new tests cover per-chat serial ordering, parallel-across-chats
isolation, ack-before-run latency, and shutdown drain.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(hosting): drop redundant @pytest.mark.asyncio decorators

asyncio_mode = "auto" is configured in pyproject.toml across the
hosting packages, so individual @pytest.mark.asyncio decorators are
unnecessary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-telegram): adapt push tests to hosted run result wrapper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting-telegram): add response hooks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:13:40 +02:00
Eduard van Valkenburg 07717d3124 Python: add agent-framework-hosting-invocations channel (#5640)
* feat(hosting-invocations): add Invocations channel package

New ``agent-framework-hosting-invocations`` package implementing the
"Invocations" HTTP channel for the Hosting framework -- a lightweight
JSON-over-HTTP shape (``POST /invocations``) for callers that want a
single request/response without committing to the full OpenAI Responses
envelope. Mounts onto an ``AgentFrameworkHost`` like any other channel.

Surface (re-exported from ``agent_framework_hosting_invocations``):

- ``InvocationsChannel`` -- concrete ``Channel`` implementation. Owns
  the Starlette route, parses inbound JSON into a ``ChannelRequest``
  (``input`` / ``session`` / ``metadata`` / ``options``), runs the
  optional ``ChannelRunHook``, calls back into the ``ChannelContext``
  to invoke the agent target, and returns a flat JSON envelope (or an
  SSE stream when ``stream=true``).
- 8 unit tests covering route wiring, isolation-key passthrough, hook
  composition, sync vs streaming paths, and ack-only behaviour for
  non-originating ``DeliveryReport``s.

Registers the package in ``python/pyproject.toml`` ``[tool.uv.sources]``
and adds the matching pyright ``executionEnvironments`` entry.

Independent of PR-3 (Responses); both depend only on PR-2 (Hosting
core).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* review: address PR-4 round 2 feedback

- expand `_stream` docstring to call out the HTTP-200 + `event: error`
  SSE contract (status committed before generator runs; hard failures
  surface as the first SSE frame, not an HTTP code)
- split chunked text on full-line terminators via `splitlines()` so
  embedded `\r` / `\r\n` no longer leak into `data:` framing on the
  wire, breaking EventSource consumers
- on `get_final_response()` failure, emit `event: error` instead of
  silently swallowing — finalize is what triggers
  history-provider persistence on the agent side, so a 5xx /
  disk-full / context-provider error must reach the client
- add tests covering `stream_transform_hook` (rewrite, drop, async),
  CRLF-in-chunk framing, and the finalize-error → no-`[DONE]` contract

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting-invocations): rename stale ChatMessage docstring reference to Message

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-invocations): adapt to hosted run result wrapper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting-invocations): add response hooks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:13:40 +02:00
Eduard van Valkenburg 7bd68f8897 Python: add agent-framework-hosting-responses channel (#5639)
* feat(hosting-responses): add OpenAI Responses-shaped channel package

New ``agent-framework-hosting-responses`` package implementing the
OpenAI Responses-shaped HTTP channel for the Hosting framework. Mounts
``POST /responses`` (and a ``/responses/{response_id}`` GET) onto an
``AgentFrameworkHost`` and translates the OpenAI Responses wire shape
to/from the channel-neutral ``ChannelRequest`` / ``HostedRunResult``
plumbing.

Surface (re-exported from ``agent_framework_hosting_responses``):

- ``ResponsesChannel`` -- concrete ``Channel`` implementation. Owns the
  Starlette route(s), parses inbound JSON into ``ChannelRequest``, runs
  the optional ``ChannelRunHook``, calls back into the
  ``ChannelContext`` to invoke the agent target, builds Responses
  envelopes (sync JSON or SSE), and respects
  ``DeliveryReport.include_originating`` so cross-channel push routes
  only ack to the originating Responses caller.
- The minted ``response_id`` is propagated via the host's ContextVar
  machinery so storage-side history providers (e.g.
  ``FoundryHostedAgentHistoryProvider``) persist envelopes against the
  same id the channel returns.
- 48 unit tests covering route wiring, parsing of each Responses input
  shape, hook composition, sync vs streaming paths, and originating
  vs non-originating delivery branches.

Registers the package in ``python/pyproject.toml`` ``[tool.uv.sources]``
and adds the matching pyright ``executionEnvironments`` entry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* review: address PR-3 round 2 feedback

- consume IsolationKeys.chat_key from the host-bound contextvar instead
  of the raw `x-agent-chat-isolation-key` header off the wire so the
  host's ASGI isolation middleware (or any operator-supplied
  replacement) is the authoritative point at which the caller is
  authenticated and the bucket key is established
- expand `response_id_factory` docstring to call out partition
  co-location vs. partition-ownership enforcement: the channel forwards
  `previous_response_id` as a hint to the factory; the storage layer
  validates the embedded partition against the bound user/chat
  isolation keys
- on mid-stream failure, call `deliver_response` with the accumulated
  text before emitting `response.failed` so host-side history /
  push-channel state stays consistent with the partial deltas the
  client already saw

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting-responses): fix quickstart to use current Agent API

ChatAgent was renamed to Agent and ChatMessage to Message. Update the
README quickstart to use client.as_agent(...) and refresh the stale
docstring reference in _channel.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-responses): adapt to hosted run result wrapper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting-responses): add response hooks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting-responses): keep instructions in chat options

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:13:40 +02:00
Eduard van Valkenburg a6027a1c91 Python: add agent-framework-hosting core package (#5638)
* feat(hosting): add agent-framework-hosting core package

New ``agent-framework-hosting`` package implementing ADR 0026 / SPEC-002:
the channel-neutral host that lets a single ``Agent`` (or ``Workflow``)
fan out across multiple wire protocols ("channels") behind one Starlette
ASGI app.

Surface (re-exported from ``agent_framework_hosting``):

- ``AgentFrameworkHost`` — wraps a hostable target, mounts channels onto
  an ASGI app, owns per-isolation-key ``AgentSession`` reuse, threads
  request context (``response_id`` / ``previous_response_id``) into
  context providers via an ``ExitStack`` of ``bind_request_context``
  calls, and exposes an opt-in Hypercorn ``serve()`` helper (extra
  ``[serve]``).
- ``Channel`` protocol + ``ChannelContribution`` — the surface a channel
  package implements (routes, lifespans, identity hooks, …).
- ``ChannelRequest`` / ``ChannelSession`` / ``ChannelIdentity`` /
  ``ChannelPush`` / ``ChannelCommand[Context]`` / ``ChannelRunHook`` /
  ``ChannelStreamTransformHook`` / ``DeliveryReport`` /
  ``HostedRunResult`` / ``ResponseTarget`` / ``ResponseTargetKind`` /
  ``apply_run_hook`` — channel-side dataclasses + helpers.
- ``IsolationKeys`` + ``ISOLATION_HEADER_USER`` / ``..._CHAT`` +
  ``get/set/reset_current_isolation_keys`` — the host's ASGI middleware
  reads the ``x-agent-{user,chat}-isolation-key`` headers off each
  inbound request and exposes them to the agent stack via a
  ``ContextVar`` so storage-side providers (e.g.
  ``FoundryHostedAgentHistoryProvider``) can apply per-tenant
  partitioning without channels having to forward anything.

Includes 45 unit tests covering the host, channel contributions,
isolation contextvar, and shared types. Registers the package in
``python/pyproject.toml`` ``[tool.uv.sources]`` and adds the matching
pyright ``executionEnvironments`` entry for tests.

Hypercorn is an optional dependency (``[serve]`` extra); the soft import
in ``serve()`` is annotated for pyright since it isn't on the default
install.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting): address PR-2 review comments

Source-code changes
- _suppress_already_consumed: narrow contract — RuntimeError now logs
  at WARNING with exc_info; non-RuntimeError still logs at exception().
  Docstring clarifies that any non-clean teardown is observable.
- _BoundResponseStream: add aclose() and route __await__ through
  get_final_response() so the binding is always released — fixes
  contextvar leak when channels abandon the stream or use the
  await-the-stream convenience.
- Lifespan: aggregate startup/shutdown callback errors; every callback
  runs, all failures are logged with their qualname, and the first
  error is re-raised so Starlette still aborts boot.
- _build_run_kwargs: switch session-cache write to dict.setdefault so
  concurrent racers cannot orphan a session if create_session ever
  yields.
- _deliver_response: introduce DeliveryReport.failed for push outages
  vs explicit "no link" drops; an outage no longer triggers an
  originating fallback so the channel can decide degraded behaviour.

Test additions
- tests/test_isolation.py (new): full coverage of IsolationKeys, the
  contextvar helpers, header constants, and end-to-end ASGI
  middleware lift / reset / passthrough.
- tests/test_host.py: TestBindRequestContext, TestBoundResponseStream
  (aclose / __await__ / __getattr__ forwarding / double-close
  idempotency), TestWrapInputListMessages (list[Message] LAST
  precedence), TestLifespanAggregation (startup + shutdown).
- tests/test_types.py: TestApplyRunHook (sync/async/None), and
  TestDeliveryReport (new failed field).
- Updated test_push_exception_marks_skipped ->
  test_push_exception_lands_in_failed_no_fallback to match the new
  delivery contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting): address PR-2 round-2 review comments

- Refactor workflow checkpoint restoration into shared helpers
  (_restore_workflow_checkpoint for blocking; the streaming sibling
  drains the rehydration stream) so the blocking and streaming paths
  rehydrate identically — clarifies the previously inline _maybe_restore
  by hoisting the pattern next to the blocking call site.
- Document that blocking workflow output is text-only by design;
  richer modalities ride the streaming AgentResponseUpdate channel,
  which preserves all content parts.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* review: address PR-4 _host.py round 2 feedback

These review comments were filed on PR-4 (#5640) but target lines that
live in the hosting-core package (PR-2 / #5638), so the fixes land here
and PR-4's stack will pick them up on rebase.

- _suppress_already_consumed: narrow the RuntimeError catch to the two
  documented benign messages (`Inner stream not available`, `Event loop
  is closed`); any other RuntimeError now logs at ERROR with a full
  traceback so executor bugs / runner-context state errors / checkpoint
  RuntimeErrors during the post-run flush no longer masquerade as
  benign cleanup noise. Still no propagation (we're in an
  async-generator finally during teardown) — see the docstring.
- _restore_workflow_checkpoint{,_streaming}: log a WARNING when a
  non-None latest checkpoint drains to zero events, so a stale or
  partially-written checkpoint_id surfaces as an operator signal
  instead of a silent state-loss.

(The `deliver_response` "no destinations resolvable" vs "every
destination errored" concern raised in 3198268038 is already addressed
by the existing `failed` vs `skipped` distinction surfaced through
`DeliveryReport.failed` — see lines 1080-1102 and the
`DeliveryReport` docstring.)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting): reject path-traversal patterns in checkpoint isolation_key

The host's `_resolve_checkpoint_storage` joined `request.session.isolation_key`
directly into the configured `checkpoint_location`. The key is caller-
controlled — sourced from inbound headers (`x-agent-{user,chat}-isolation-key`
injected by the Foundry runtime), from channel-supplied derivations such as
`telegram:<chat_id>` / `entra:<oid>`, or from values set by a channel
`run_hook`. A value like `../../../etc/foo` or an absolute path would let
the resulting checkpoint directory escape the configured root (CWE-22).
This matches the path-traversal class fixed upstream in #5851 for the
foundry_hosting checkpoint storage.

New `_checkpoint_path_for_isolation_key(root, isolation_key)` helper:

- Uses a denylist (not allowlist) so legitimate namespaced keys
  (`telegram:42`, `entra:abc-def`) continue to pass through unmodified.
- Rejects path separators (`/`, `\`), NUL, all-dot reductions (`.`, `..`,
  `...`, ...), absolute paths (`os.path.isabs`), and drive-letter prefixes
  (`os.path.splitdrive` plus an explicit `^[A-Za-z]:` check so payloads
  crafted on a POSIX host still fail closed if the resulting directory
  ever round-trips to Windows storage).
- After joining, resolves both sides and verifies
  `target.is_relative_to(root)` as defence-in-depth.

`_resolve_checkpoint_storage` now logs a WARNING and returns `None` for
invalid keys rather than crashing the request — checkpointing is best-
effort and we prefer dropping it to letting one malformed key abort an
otherwise valid agent run.

Tests:

- `TestCheckpointPathForIsolationKey` exercises the helper directly with
  legitimate keys (alphanumeric, `:`-namespaced, dotted, 200-char), all
  rejected traversal patterns from #5851's MSRC repro list, and
  non-string input.
- `TestHostWorkflowCheckpointingPathTraversal` verifies the end-to-end
  request path: a traversal key (`../escape`) and an in-key separator
  (`evil/sub`) both produce a successful agent response with no files
  written under `checkpoint_location`, and the traversal case logs a
  WARNING citing `isolation_key`.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting): address PR-2 round-3 review feedback + add response hooks

Round-3 review comment fixes:

- _types.py: drop the _EMPTY_MAPPING sentinel; ChannelIdentity.attributes
  uses plain dict() as the default — simpler, no extra symbol to track.
- _host.py: drop the local `import asyncio` + `from typing import cast as
  _cast` inside `serve()`; rely on the module-level imports.
- _host.py: switch `_log_incoming` to structured `extra={...}` payloads
  for both INFO and DEBUG so log aggregators get queryable fields.
- _host.py: delete `_flat_context_providers` and stop descending into a
  `.providers` attribute. Aggregator providers (AggregateContextProvider /
  ContextProviderBase) are responsible for forwarding `response_context`
  to their children themselves; the host treats whatever
  `agent.context_providers` exposes as the final, flat list.
- _host.py: stop collapsing agent / workflow output to text. `_invoke`
  forwards `AgentResponse.messages` (and `raw_response`) on the
  `HostedRunResult`. `_invoke_workflow` builds a per-event message list
  via a new `_workflow_output_to_messages` helper that preserves
  AgentResponse / AgentResponseUpdate / Message / Content branches and
  falls back to text only for arbitrary objects.
- _host.py: `_workflow_event_to_update` carries Content payloads through
  unchanged so multi-modal workflow outputs (images, function-call
  metadata, ...) survive into channels.

New features (per design discussion in the PR thread):

- HostedRunResult: rebuilt around `messages: list[Message]` with
  `.text` / `.contents` as projections, a `raw_response` slot for the
  underlying AgentResponse, and a `replace(messages=..., raw_response=...)`
  clone helper used by the delivery layer for per-destination isolation.
  The `HostedRunResult(text="...")` ctor is preserved as a back-compat
  shim that synthesises a single assistant text message.
- ResponseTarget: gain `echo_input: bool = False` (also exposed on
  `.channel(name, *, echo_input=...)` / `.channels([...], *, echo_input=...)`).
  When set, the host pushes the originating user message to each
  non-originating destination before the agent reply. Channels can
  filter or transform echoes via their response_hook.
- DeliveryReport: add `echoed` / `echo_failed` tuples to surface
  per-destination outcomes of the new echo phase. Echo failures do not
  abort the corresponding response push on the same destination.
- ChannelResponseHook + ChannelResponseContext + apply_response_hook:
  duck-typed `response_hook` attribute on channels for per-destination
  post-processing. Receives a clone of the HostedRunResult and a
  context carrying the request, channel name, destination identity,
  originating flag, and `is_echo` phase flag. Channels stay
  modality-aware (text-only wires flatten via the hook; card-capable
  channels render structured contents directly).
- _deliver_response: clone-before-hook fan-out so a hook mutating one
  channel's payload cannot leak into another destination's view.

Tests:

- Update _FakeAgentResponse to expose `.messages` (single assistant text
  message synthesised from `text`) so existing tests pass unchanged on
  the new multi-modal _invoke path.
- Replace the obsolete `test_bind_descends_one_level_into_providers_attribute`
  with a regression guard asserting the host does NOT descend into
  `.providers` (matches new contract).
- New tests for HostedRunResult multi-modal preservation, echo_input
  fan-out with success + failure, response_hook applied per destination,
  per-destination mutation isolation, and is_echo phase observability.

Docs:

- spec 002: rewrite Canonical flow with the new input → run_hook → host
  → target → wrap → per-destination clone → response_hook → push
  pipeline; document multi-modality contract and per-destination
  cloning; add `echo_input` row to ResponseTarget table; rewrite
  HostedRunResult/HostedStreamResult row; add ChannelResponseHook /
  ChannelResponseContext / apply_response_hook table; log decisions
  Q28 (no host-side text collapse), Q29 (duck-typed response_hook),
  Q30 (opt-in `echo_input` on ResponseTarget).
- ADR 0026: add ChannelResponseHook + multi-modality bullets;
  surface `echo_input` on the ResponseTarget bullet.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(hosting): drop HostedRunResult(text=...) back-compat shim; use from_text()

Pre-release cleanup — no released callers to break, so consolidate on one
canonical entry point plus a classmethod for the ergonomic
single-text-message case:

- HostedRunResult.__init__ takes ``messages`` positionally (required); no
  more ``text=`` kwarg overload, no more "synthesise an empty message
  when no args" path.
- New HostedRunResult.from_text(text, *, role="assistant", raw_response=None)
  classmethod for the common "wrap a single text content as one message"
  case (tests, channels emitting plain strings, the echo-input phase
  wrapping a user's text turn).
- ``_build_echo_payload`` uses ``HostedRunResult.from_text(raw, role="user")``
  for the ``str`` and fallback branches; the other branches use the plain
  ctor with explicit ``Message`` lists.
- Tests rewritten to use ``from_text("reply")`` everywhere
  ``HostedRunResult(text="reply")`` appeared. Added an explicit
  ``test_from_text_role_kwarg_overrides_default`` regression guard.
- spec 002: HostedRunResult row updated to describe the
  ``from_text(text, *, role="assistant")`` classmethod instead of the
  removed back-compat shim.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(hosting-core): reshape HostedRunResult into generic typed envelope

Replace the flattened multi-modal HostedRunResult (carrying
messages/raw_response/.text projections) with a typed generic
envelope around the target's full-fidelity output:

  class HostedRunResult(Generic[TResult]):
      result: TResult
      session: AgentSession | None

- Agent targets produce HostedRunResult[AgentResponse]; channels
  read result.messages, result.text, result.value, result.response_id,
  result.usage_details directly off the underlying response.
- Workflow targets produce HostedRunResult[WorkflowRunResult];
  channels iterate result.get_outputs() and inspect
  result.get_final_state() themselves (the host no longer collapses
  workflow outputs onto a synthesised message list).
- The echo-input phase synthesises a HostedRunResult[AgentResponse]
  wrapping the user's turn so the same per-destination delivery
  machinery applies.
- replace() is now {result, session} only; the host's clone is
  shallow — channels that need to mutate result itself are
  responsible for their own deep copy.

Rationale: the earlier shape pre-shaped target output (collapsing
workflows onto a Message list, losing per-executor outputs, final
state, and structured value affordances). Carrying the target output
unchanged keeps the host modality-agnostic, gives channel authors
static typing where they want it, and removes 30+ lines of
host-side projection helpers.

Also updates ADR 0026 + spec 002 (Q3, Q28, Q29 amended; new Q31
captures the generic-envelope decision and rationale).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting-core): document echo vs response distinction for push channels

The host already encodes the echo-vs-response phase via the
underlying Message.role on the pushed HostedRunResult:

- echo phase: payload.result.messages[*].role == "user"
- response phase: payload.result.messages[*].role == "assistant"

Both pushes go through the same ChannelPush.push(identity, payload)
entry point. Channels distinguish either by inspecting role (which
works for any push-capable channel) or — when a response_hook is
wired — by branching on ChannelResponseContext.is_echo directly.

Expand the ChannelPush Protocol docstring to make this discoverable
for channel implementers (esp. chat bots that cannot impersonate
the user on their wire and need to render echoes as quoted /
prefixed blocks rather than as bot replies).

Mirror the explanation into the spec's echo_input section.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting-core): fix quickstart to use current Agent API

ChatAgent was renamed to Agent and the preferred construction pattern
is client.as_agent(...). Also drop the sibling channel import so the
snippet imports only modules declared as dependencies of this package;
point readers at the sibling packages instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(hosting-core): drop redundant @pytest.mark.asyncio decorators

asyncio_mode = "auto" is configured in pyproject.toml, so individual
@pytest.mark.asyncio decorators are unnecessary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting): add authorization profiles + IdentityAllowlist seam to ADR/spec

Composes `require_link` + `allowlist` into three named profiles (open,
forced-link, allowlist) with the allowlist itself keyed on either the
channel-native id (pre-link) or a verified IdP claim (post-link), plus
`AnyOf`/`AllOf` combinators for mixed setups. Lifts the design into
an explicit host seam (`host.authorize(...)` → `AuthorizationOutcome`
of `Allowed` / `LinkRequired` / `Denied`) instead of leaving each
channel to roll its own.

Key contract bits:
- Tri-state `AllowlistDecision` (ALLOW / DENY / ABSTAIN) so claim-based
  lists can ABSTAIN until claims are available without composition
  silently flipping that into DENY.
- `AuthorizationContext` carries explicit `phase` + `claim_source`
  so allowlists can tell pre-link from post-link without overloading
  `verified_claims is None`.
- Channel-side `allowlist: ... | Literal["inherit"] | None` with an
  explicit inheritance sentinel, so the host-level `default_allowlist`
  is opt-out, not opt-in.
- Construction-time validator rejects silent-deny configurations
  (`LinkedClaimAllowlist` without a claim source) with a typed
  `ChannelConfigurationError`.
- Group-chat denial mirrors the existing `LinkChallenge` DM-redirect
  pattern; only the redacted `user_message` reaches the wire,
  structured `log_details` stay in telemetry.

Ships in two waves: the Protocol + `NativeIdAllowlist` + config
validator land with the next core PR ahead of the linker; the full
pipeline + `LinkedClaimAllowlist` enforcement land with the
`IdentityLinker` core PR.

Updates: ADR 0026 (summary bullet + conceptual-API table row + resolved
Q16), spec 002 (new req #22, renumbered v1 fast-follow #23..#29 and
stretch #30..#31, new "Authorization profiles and the IdentityAllowlist
seam" subsection, inbound-ownership row, resolved Q32, follow-up entry).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): add DurableTaskRunner seam + runtime_mode auto-detect

Introduces the explicit long-running vs ephemeral runtime distinction
and a generic DurableTaskRunner Protocol that owns non-originating
push dispatch — collapsing the previous deliveries[] per-destination
state machine, SupportsDeliveryTracking provider capability, and
Foundry update_item service ask down to a single immutable
intended_targets[] write on the message.

Spec / ADR:
- New §"Runtime modes" with auto-detect markers + defaults matrix.
- Rewrites §"Delivery tracking" → §"Intended targets + durable
  delivery": intent-only on the message, operational state lives in
  the runner.
- New §"Durable task runner" defining DurableTaskRunner / RetryPolicy
  / TaskHandle / TaskStatus.
- Drops §SupportsDeliveryTracking and §Foundry update_item gap.
- Resolved Qs: 12, 18, 21, 26 revised; new 17/18/19 (ADR) and
  33/34/35 (spec).

Code:
- New _runner.py with InProcessTaskRunner (asyncio + bounded retry,
  bounded terminal-status cache, register-after-start guard,
  shutdown drain).
- _host.py: runtime_mode + durable_task_runner ctor params;
  auto-detect via FOUNDRY_HOSTING_ENVIRONMENT /
  AZURE_FUNCTIONS_ENVIRONMENT / AWS_LAMBDA_FUNCTION_NAME;
  HOSTING_PUSH_TASK_NAME handler registered eagerly so
  _deliver_response can be called outside the lifespan;
  _handle_push_task does echo-then-response inline per destination;
  _deliver_response now schedules one task per destination via the
  runner (DeliveryReport.pushed = scheduled; .failed = schedule-time
  outage only).
- _types.py: new DurableTaskRunner Protocol + RetryPolicy /
  TaskHandle / TaskStatus; DeliveryReport drops echoed /
  echo_failed (echo outcome owned by the runner).
- __init__.py exports the new public surface.

Tests: 132 passing, 90% coverage. New test_runner.py covers
InProcessTaskRunner success/retry/terminal-failure/cancellation/
register-after-start, runtime-mode auto-detect with synthetic env,
and the warning-on-ephemeral-without-runner path. test_host.py
delivery tests use a sync runner fake for deterministic assertions
and validate the new "schedule succeeded vs runner backend
unreachable" semantics.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): rubber-duck round-5 — strict ephemeral, codec seam, allowlist Wave-1, drop DeliveryReport

Adopts the rubber-duck-approved package of changes from the round-5
review of PR #5638 (modulo DeliveryReport.failed — the value type is
removed entirely now that durable delivery covers the failure
surface, per user direction).

Code:
- Drop DeliveryReport value type; host-internal _deliver_response
  returns bool. Failure observability is now logs (in-process) /
  runner backend (durable adapters).
- Strict ephemeral default: ephemeral runtime_mode with the default
  in-process runner raises RuntimeError; opt-in via
  allow_in_process_runner=True (warns).
- ChannelPushCodec Protocol + DurableTaskPayloadMode enum +
  _validate_runner_codec_pairing so JSON-mode runners can be safely
  paired with channels via codecs; _handle_push_task accepts both
  object- and JSON-envelope shapes.
- ResponseTarget.identity(...) / .identities([...]) builders +
  IDENTITIES kind for explicit caller-supplied recipients; field
  rename identities → _target_identities (private) with a
  target_identities property to resolve the classmethod collision.
- Intent-only audit: _annotate_intended_targets writes
  hosting.intended_targets / skipped_targets / includes_originating /
  originating_channel onto assistant messages — single immutable
  write per the runner-owned operational-state model.
- InProcessTaskRunner: 2-phase drain on shutdown
  (shutdown_grace_seconds, default 5.0) so a clean shutdown does not
  abandon work mid-retry; payload_mode = OBJECT class-level.
- Echo idempotency: _handle_push_task tracks an echo_done cursor on
  runner-owned task state so a retry that fires after the echo
  phase succeeded does not double-echo.

Wave-1 authorization seam (full landing):
- New _authorization.py with AllowlistDecision tri-state,
  AuthorizationContext, IdentityAllowlist Protocol, AllowAll /
  NativeIdAllowlist (with async loader cache + channel-scope ABSTAIN) /
  LinkedClaimAllowlist (raise-until-Wave-2) / AnyOfAllowlists /
  AllOfAllowlists / CallableAllowlist built-ins, Allowed /
  LinkRequired / Denied outcomes, ChannelConfigurationError.
- Host(default_allowlist=..., identity_linker=...) + per-channel
  allowlist parameter with 'inherit' / None semantics.
- _validate_channel_authorization enforces all three rules at
  construction: claim-source requirement, linker presence for
  require_link=True (elevated from no-op — must not ship
  unenforced), and NativeIdAllowlist(channel=...) typo detection.
  Combinator-walking via _flatten_allowlists catches nested
  misconfigs.
- host.authorize(...) for the native-id pipeline: open path returns
  Allowed with auto-issued <channel>:<native_id> isolation key (or
  the existing key when the identity has been seen); ABSTAIN on a
  claim-required allowlist maps to
  Denied(reason_code='allowlist_requires_link') until Wave 2 wires
  the linker to convert it to LinkRequired.

Spec / ADR:
- docs/specs/002-python-hosting-channels.md: Wave-1 status updated
  to reflect the linker-presence rule elevation and the
  host.authorize landing; new sub-sections (codec contract, drain,
  echo cursor); Qs 18 / 21 DeliveryReport references purged; new
  resolved Qs 36–40 covering the strict-ephemeral default, codec
  contract, DeliveryReport removal, echo cursor, and drain.
- docs/decisions/0026-hosting-channels.md: Q12 DeliveryReport
  reference purged; Q16 updated to reflect Wave-1 landing; new
  resolved Qs 20 (codec contract) + 21 (strict ephemeral / drain /
  echo cursor).

Tests:
- New tests/test_authorization.py (35 cases) covering every Wave-1
  built-in, the three validator rules, combinator decision
  semantics, and host.authorize across open / allow / deny /
  abstain-with-claim-dep / abstain-without-claim-dep paths plus
  existing-key reuse and verified-claims propagation.
- tests/test_host.py: TestDeliverResponse rewritten for the bool
  return + runner.scheduled-count assertions; new tests for
  IDENTITIES variant + echo idempotency.
- tests/test_runner.py: strict-ephemeral now expects RuntimeError;
  allow_in_process_runner opt-in tests; shutdown drain test;
  payload_mode default test.
- tests/test_types.py: TestDeliveryReport removed; new
  TestDurableTaskPayloadMode + TestResponseTargetIdentities.

Validation: 178 tests pass, 91% coverage, fmt + lint + pyright +
mypy clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(hosting): add mermaid flow diagrams to ADR, spec, README

Insert the 10 hosting flow diagrams reviewed in
python/.user/hosting-diagrams.md into the public docs:

- README: runtime topology (1a) + cross-link to the spec for the
  richer set.
- ADR: runtime topology, channel contribution shape, and authorization
  decision (1a, 1b, 3) at the end of 'Conceptual API shape'.
- Spec: all 10 diagrams — 1a/1b at the top of API Surface, 2 in
  Canonical flow, 3 in Authorization profiles, 4-7 in Scenarios 6-8,
  8 in Codec contract, 9 in Echo idempotency, 10 in Scenario 9.

Doc-only; no API or behaviour change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): add opt-in disk persistence via state_dir

Long-running hosts (always-on container, single-VM bot, local dev) lose
state on every restart today. Add an opt-in disk persistence layer under
a new `state_dir` constructor parameter on `AgentFrameworkHost` that
survives process restarts without taking on a heavyweight database
dependency.

Backed by `diskcache` (installed via the new `[disk]` optional extra).
An OS-level advisory file lock guarantees single-owner semantics so two
hosts pointed at the same directory cannot double-execute scheduled
pushes.

What persists when `state_dir` is set:

- Pending durable-task records — scheduled-but-not-yet-completed pushes
  replay on the next host startup via `InProcessTaskRunner.resume()`.
  Records that crashed mid-attempt resume with the already-consumed
  retry budget (no full-budget re-grant).
- `_session_aliases` — per-isolation-key session-id rewrites.
- `_active` — most-recently-active channel per isolation key.
- `_identities` — `ChannelIdentity` rows for fan-out targeting,
  including nested mutations of the form
  `self._identities[ik][channel] = identity`.

The `state_dir` parameter accepts any of:

- `None` — today's purely in-memory behaviour.
- `str` / `PathLike` — single root; host auto-creates `runner/` and
  `sessions/` subfolders.
- `HostStatePaths` TypedDict / plain mapping — per-component overrides
  routed to different roots. Unknown keys raise `ValueError` to surface
  typos early.

Unpicklable push payloads raise `PushPayloadNotPicklable` eagerly from
`schedule()` so issues surface at the call site rather than on the
next restart. Corrupt on-disk records are quarantined-and-logged; the
runner never crashes on resume.

Live `AgentSession` objects stay in memory and are rehydrated lazily
by the history provider on the next turn.

- New modules: `_persistence.py` (lock + normalisation),
  `_state_store.py` (session-bookkeeping store).
- Runner rewrite: 4-state model (`pending` / `succeeded` / `failed`
  / `cancelled`); the transient `running` state was a bug that caused
  resume to skip records that crashed mid-handler.
- New tests: `test_runner_disk.py` (8 tests), `test_host_disk.py` (8
  tests). 194 passed total. pyright + mypy + ruff clean.
- README: new "Optional disk persistence" section with code samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): add checkpoints to state_dir + fix host docstring

Three related polish changes on top of the disk-persistence landing:

1. Extend `state_dir` to cover workflow checkpoints. Adds
   `checkpoints` as a third `HostStatePaths` key. Single-path form
   (`state_dir="/foo"`) now also auto-derives `/foo/checkpoints/`
   for workflow targets (equivalent to passing
   `checkpoint_location="/foo/checkpoints"`). The mapping form lets
   workflow callers opt out by omitting the key, or route checkpoints
   to a different volume.

   Conflict / precedence rules:
   * Explicit `checkpoint_location` always wins over the state_dir
     derived path; a warning surfaces the double-config.
   * Single-path `state_dir` + non-Workflow target → checkpoints path
     silently ignored (no eager directory creation either).
   * Mapping form with `checkpoints` + non-Workflow target → warn
     (almost certainly dead config).
   * Derived path with a workflow that already has its own
     `checkpoint_storage` → same `RuntimeError` as the explicit
     parameter triggers, so ownership stays unambiguous.

   Checkpoint persistence uses `FileCheckpointStorage` from the
   framework core — no extra dependency. Only `runner` and
   `sessions` require the `[disk]` extra.

2. Move `AgentFrameworkHost.__init__` parameter docs from `Args:` to
   `Keyword Args:` for every parameter after the `*`. Only `target`
   remains under `Args:`. Brings the docstring in line with the
   actual signature (the params have always been keyword-only).

3. `HostStatePaths` already existed as a TypedDict but did not cover
   `checkpoints`; updated to document the new key with the same
   per-attribute docstring style as `runner` / `sessions` so editors
   can surface help on the keys.

Validation: 201 tests pass (was 194; +7 checkpoint integration tests
in test_host_disk.py). pyright + mypy + ruff + bandit clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): add core IdentityLinker authorization seam

Fold the core IdentityLinker pieces into the hosting-core PR so the
authorization surface no longer has a deferred Wave-2 placeholder.
Provider-specific linkers (for example Entra OAuth helpers) can now plug
into core without core depending on an IdP SDK.

Core additions:
- Add LinkChallenge, LinkedIdentity, LinkResolution, and IdentityLinker.
  IdentityLinker.resolve(identity) is a single-call decision that returns
  either a linked identity with verified claims or a challenge the channel
  can render.
- Enable LinkedClaimAllowlist end-to-end. It now abstains pre-link and
  allows/denies post-link against verified claims, including multi-valued
  claims such as groups.
- Add AuthPolicy factories for common allowlist shapes.
- Extend Allowed with verified_claims and claim_source for audit/telemetry
  without requiring callers to re-derive how the decision was made.

Host behavior:
- identity_linker is now typed as IdentityLinker | None.
- authorize() supports open, native-id, forced-link, and linked-claim
  profiles end-to-end.
- require_link=True resolves via the linker and returns LinkRequired when
  the identity is not linked.
- claim-based allowlists use channel-emitted verified_claims when present,
  or linker-resolved claims otherwise.
- authorize() remains decision-only and does not mutate _identities/_active;
  identity registry writes remain on the actual request execution path.

Docs/tests:
- Remove Wave-1/Wave-2 language from core/spec/ADR surfaces touched here.
- Update the spec/ADR to describe the core linker seam and provider-specific
  linker packages.
- Add authorization tests for linker challenges, linked identities, linked
  claim allowlists, channel-emitted claims, AuthPolicy factories, and the
  no-mutation contract.

Validation: 214 tests pass, pyright/mypy/ruff clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(hosting): add link-store path to state_dir

Identity linking introduces host-adjacent state that needs the same state_dir treatment as runner, session, and checkpoint state. Add a links component to the host state paths so applications and linker packages have a typed, discoverable persistence location.

Changes:
- Extend HostStatePaths with links and include it in state_dir normalization (state_dir/links/ for the single-path form).
- Add SupportsLinkStorePath, an optional protocol for identity linkers that accept a host-provided link-store path.
- AgentFrameworkHost now offers state_dir links to compatible linkers, warns when an explicit links path is supplied without a linker, and warns when the configured linker manages persistence directly instead of implementing SupportsLinkStorePath.
- Update README and spec text to document the link-store component and clarify that concrete linkers still own the storage format.
- Add disk-state tests for compatible, missing, and non-configurable linkers.

Validation: 217 tests pass, pyright/mypy/ruff clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 17:12:30 +02:00
dependabot[bot] 02eb9435bf Bump litellm from 1.83.14 to 1.84.0 in /python (#6559)
Bumps [litellm](https://github.com/BerriAI/litellm) from 1.83.14 to 1.84.0.
- [Release notes](https://github.com/BerriAI/litellm/releases)
- [Commits](https://github.com/BerriAI/litellm/commits/v1.84.0)

---
updated-dependencies:
- dependency-name: litellm
  dependency-version: 1.84.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-17 12:56:50 +00:00
Copilot 3daed114ee Python: bump package versions for 1.8.1 release (#6420)
* Python: bump package versions for 1.8.1 release

* Python: bump agent-framework-foundry-hosting for 1.8.1 release

* Python: bump ag-ui and azurefunctions for 1.8.1 release

* Remove incorrect agent-framework-foundry changelog entry for #6259

* Add [1.8.1] changelog compare link and update [Unreleased] base

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-06-09 21:27:42 +00:00
Giles Odigwe e89e745bc0 Python: feat(claude): bump claude-agent-sdk to 0.2.87 (#6248)
* feat(claude): bump claude-agent-sdk to 0.2.87

Upgrade claude-agent-sdk dependency from >=0.1.36,<0.1.49 to >=0.2.87,<0.3.

Changes:
- Bump version pin in pyproject.toml
- Add 'xhigh' effort level to ClaudeAgentOptions (Opus 4.7 specific)
- Expose new upstream SDK options: skills, session_id, task_budget,
  include_hook_events, strict_mcp_config, continue_conversation,
  fork_session
- Add TaskBudget type import
- Update uv.lock

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: lower claude-agent-sdk floor to >=0.1.36

Keep the lower bound at 0.1.36 since the 0.1→0.2 transition was additive
and our code works on older versions as long as new options aren't used.
This avoids forcing unnecessary upgrades on existing users.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: replace TaskBudget import with inline type for SDK compat

TaskBudget was added in claude-agent-sdk 0.2.93 but does not exist in
0.2.87. Use dict[str, int] inline type instead so type checking passes
against 0.2.87. Lock file pinned to 0.2.87.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-09 06:01:55 +00:00
westey bad05a2bdc Python: Harness console for python (#6312)
* Add initial harness console for python

* Add textual to project

* Add planning and approval flows with list selector

* Address PR comments

* Fix list selection bug

* Fix PR #6312 round 2 review comments

- Escape untrusted agent text with rich.markup.escape() in observers
  (text_output, planning_output, reasoning_display) to prevent markup injection
- Remove non-functional 'Always approve' choices from tool_approval.py
  (framework lacks CreateAlwaysApproveToolResponse support)
- Remove textual from root pyproject.toml dev deps (sample-specific)
- Add PEP 723 inline script metadata to harness_research.py
- Narrow except Exception to except NoMatches in list_selection.py

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix build error

* Fix build errors

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-09 05:48:35 +00:00
Giles Odigwe 01fc518b29 Python: bump package versions for 1.8.0 release (#6351)
- Released cohort (core, openai, foundry, root): 1.7.0 -> 1.8.0
- agent-framework-github-copilot: promote to RC (1.0.0rc1)
- agent-framework-orchestrations: rc2 -> rc3 (bug fix)
- Beta/alpha packages with changes: a2a, anthropic, azurefunctions, bedrock,
  foundry-hosting, mistral bumped to new date stamp (260604)
- Inter-package dependency bounds updated for changed packages
- CHANGELOG.md and PACKAGE_STATUS.md updated

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-04 23:03:24 +00:00
Giles Odigwe fe08574a7c Python: [BREAKING] Upgrade github-copilot-sdk to v1.0.0 (stable) (#6292)
* Python: Upgrade github-copilot-sdk to v1.0.0 (stable)

Upgrade agent-framework-github-copilot from github-copilot-sdk 1.0.0b2 to the
stable 1.0.0 release, adapting to all breaking API changes.

Source changes (_agent.py):
- SubprocessConfig removed: use RuntimeConnection.for_stdio(path=...) +
  CopilotClient kwargs (connection, log_level, base_directory)
- Import paths: copilot.generated.session_events -> copilot.session_events
- Settings: copilot_home -> base_directory (env GITHUB_COPILOT_BASE_DIRECTORY)
- Default deny handler: PermissionDecisionUserNotAvailable() (from
  copilot.generated.rpc)

Test changes:
- Updated imports and client-construction assertions (kwargs-based)
- Permission handler tests use concrete decision types
  (PermissionDecisionApproveOnce, PermissionDecisionDeniedInteractivelyByUser)

Sample changes:
- Permission handlers use PermissionHandler.approve_all or sync
  approve_and_log pattern (v1.0.0 protocol v3 dispatch is incompatible
  with blocking input() in permission handlers)
- Function approval sample uses asyncio.to_thread for interactive prompts
- Simplified imports across all samples

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: scope permission handlers, widen type, add test

- Shell sample: only approve kind='shell', deny others
- URL sample: only approve kind='url', deny others
- Use getattr() for kind-specific attributes to satisfy pyright
- Widen PermissionHandlerType to accept async handlers (matches SDK)
- Add test for _deny_all_permissions return value

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix validation script and strengthen test assertion

- Update scripts/sample_validation/create_dynamic_workflow_executor.py to
  use copilot.session_events imports and PermissionHandler.approve_all
- Assert isinstance(result, PermissionDecisionUserNotAvailable) instead of
  stringly-typed kind check

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add integration tests for GitHubCopilotAgent

Add 6 integration tests mirroring .NET coverage:
- Basic non-streaming response
- Streaming response
- Function tool invocation
- Session context (multi-turn)
- Session resume by ID
- Shell command execution

Tests require COPILOT_GITHUB_TOKEN env var (skipped otherwise).
Each test cleans up its Copilot session via delete_session.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-04 08:42:35 +00:00
Eduard van Valkenburg 49a6e433a3 Python: progressive tool exposure via FunctionInvocationContext (#6233)
* Python: progressive tool exposure via FunctionInvocationContext

Add first-class progressive tool exposure to the Python core function-calling
loop. Tools can now add or remove real FunctionTool schemas at runtime via the
injected FunctionInvocationContext, taking effect on the next iteration of the
loop.

- FunctionInvocationContext gains a live `tools` list plus experimental
  `add_tools()` / `remove_tools()` helpers (feature: PROGRESSIVE_TOOLS).
- The function-calling loop establishes a run-local, normalized tools list and
  threads it into the context at both invocation paths so mutations propagate.
- Add a sample (dynamic_tool_exposure.py) and a tools samples README, including
  a note that CodeAct providers (Monty/Hyperlight) use their own provider-level
  tool management instead.

Supersedes #3877.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Validate non-negative input in dynamic_tool_exposure sample tools

Address review feedback: factorial and fibonacci now return an error
message for negative n instead of producing incorrect results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Make add_tools atomic and surface swallowed function errors

Address review feedback on progressive tool exposure:

- add_tools now validates the full batch against a throwaway copy before
  committing, so a duplicate-name clash partway through a sequence leaves
  the live tool list unchanged (all-or-nothing).
- _auto_invoke_function now logs a warning (with traceback) when a tool
  raises, so contract errors such as a duplicate-name ValueError from
  add_tools are debuggable without enabling include_detailed_errors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Avoid retaining tracebacks when logging swallowed function errors

Logging with exc_info=exc fed the exception traceback to the logging
machinery, whose frame references created reference cycles collected
lazily by the cyclic GC. On Windows that could drop a hyperlight
WasmSandbox on a non-owning thread ("unsendable, dropped on another
thread"), crashing the xdist worker. Log a pre-formatted message with
the exception repr instead, so no traceback object is retained.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* added missing decorator

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-03 09:01:07 +00:00
Peter Ibekwe 6086a74302 Python: Promote agent-framework-declarative package to RC (#6256)
* Promote agent-framework-declarative package to RC

* Update missed package status file.
2026-06-02 19:30:05 +00:00
Eduard van Valkenburg 8091d052d8 Python: refresh dev dependencies and validate runtime bounds (#6238)
Updates third-party dev dependencies across the Python workspace and
validates that all runtime dependency bounds still hold at both ends.

Dev dependency bumps (root, lab, declarative, durabletask):
- uv 0.11.6 -> 0.11.17, ruff 0.15.8 -> 0.15.15,
  pytest-asyncio 1.3.0 -> 1.4.0, mcp 1.27.0 -> 1.27.2,
  azure-monitor-opentelemetry 1.8.7 -> 1.8.8,
  poethepoet 0.42.1 -> 0.46.0, prek 0.3.9 -> 0.4.3,
  types-python-dateutil and types-PyYaml stub bumps.
- Transitive Dependabot items swept via lock: idna 3.11 -> 3.17,
  pip 26.0.1 -> 26.1.2.

Deliberately excluded:
- opentelemetry-sdk stays 1.40.0: azure-monitor-opentelemetry (incl.
  1.8.8) hard-pins opentelemetry-sdk==1.40.
- mypy stays 1.20.0 and pyright stays 1.1.408: the 2.1.0 / 1.1.409
  bumps introduce new diagnostics that fail type checking and need
  dedicated PRs.
- rich kept as a range: agentlightning (lab[lightning]) forces
  rich==13.9.4.

Code/formatting changes driven by the ruff upgrade:
- devui lifespan now uses try/finally so shutdown cleanup always runs
  (ruff RUF075).
- Removed unused TYPE_CHECKING imports in core and foundry flagged by
  ruff 0.15.15.
- Reapplied ruff 0.15.15 formatting to the files it changed.

Validation: validate-dependency-bounds-test "*" passes (31/31 lower +
31/31 upper); typing 62/62; lint 31/31; devui tests pass.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-01 17:53:56 +00:00
Daria Korenieva d2d5384f28 Python: Add Mistral AI embedding client package (#5480)
* Python: Add Mistral AI embedding client package

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Address review feedback: fix dimensions check, sort embeddings by index, align docs

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Address review feedback: downgrade to alpha, remove integration tests - Change version to 1.0.0a260505 (alpha) - Update classifier to Development Status :: 3 - Alpha - Update PACKAGE_STATUS.md to alpha - Remove Mistral from integration test workflows (no API keys yet)

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Add samples directory for alpha package compliance Per python-package-management skill: alpha packages must include samples inside the package directory.

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

* Fix ruff formatting in sample file

Signed-off-by: Daria Korenieva <daric2612@gmail.com>

---------

Signed-off-by: Daria Korenieva <daric2612@gmail.com>
2026-05-29 07:20:56 +00:00
Evan Mattson a84ad42f6d Bump Python package versions for 1.7.0 release (#6142)
Bumps the released 1.6.0 packages agent-framework, agent-framework-core, agent-framework-foundry, and agent-framework-openai to 1.7.0, with root continuing to exactly pin agent-framework-core[all]. Bumps the changed prerelease packages agent-framework-a2a, agent-framework-chatkit, agent-framework-declarative, agent-framework-devui, and agent-framework-foundry-hosting to the 260528 date stamp, raises core floors on the packages included in this release, raises Foundry's OpenAI floor alongside OpenAI, and raises ChatKit's openai-chatkit floor to the minimum version required by the current typed API usage. No beta cohort bump was applied; the absent mistal/mistral package was intentionally not bumped because no such package exists in this branch.
2026-05-28 19:45:31 +09:00
Eduard van Valkenburg d5c07f2623 Python: feat(foundry): add to_prompt_agent / deploy_as_prompt_agent (experimental) (#5959)
* feat(foundry): add experimental to_prompt_agent converter

Adds `to_prompt_agent(agent)`, an experimental converter
(`ExperimentalFeature.TO_PROMPT_AGENT`) that turns an Agent Framework
`Agent` into a Foundry `PromptAgentDefinition` ready to publish via
`AIProjectClient.agents.create_version(...)`.

Behaviour:

* `agent.client` must be a `FoundryChatClient` (or subclass); otherwise
  `TypeError` is raised. The model deployment name is lifted from the
  bound client so the same Agent definition used for local runs can be
  published as a hosted prompt agent without restating the model.
* Foundry SDK tool instances (from `FoundryChatClient.get_*_tool()`) are
  passed through unchanged. AF `FunctionTool`s (and `@tool`-decorated
  callables) are emitted as Foundry `FunctionTool` declarations.
* Local AF MCP tools cannot be expressed in a `PromptAgentDefinition`;
  the converter raises `ValueError` and points at
  `FoundryChatClient.get_mcp_tool()` for hosted MCP servers.
* The converter walks both `agent.default_options["tools"]` and
  `agent.mcp_tools` because `normalize_tools()` splits local MCP off
  into its own list.

Re-exported through the `agent_framework.foundry` lazy-loading namespace
(updates both `__init__.py` and the `__init__.pyi` type stub).

Adds a portable-agent sample showing the same `Agent` driven through
both `agent.run(...)` and `to_prompt_agent(agent)`, and a README section
covering the new converter.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): remove snippet tags from portable agent sample

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): inline FoundryChatClient and enable prompt-agent publish

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore(samples): drop async credential context manager

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): trim README to_prompt_agent example to publish-only flow

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): note FoundryAgent runs @tool callables for deployed prompt agents

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): address review comments on to_prompt_agent converter

* Construct `PromptAgentDefinition` `Tool` from a dict via `**tool_item`
  unpacking rather than the positional Mapping constructor \u2014 cleaner and
  matches the typical Pydantic / Azure SDK pattern.
* Drop the redundant `isinstance(mcp_tool, MCPTool)` guard in
  `_convert_tools`; the parameter is already typed `Iterable[MCPTool]` so
  the second `raise` was unreachable. The remaining single `raise`
  fires for every entry as intended.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): match Agent.__init__ model resolution in to_prompt_agent

* Read the model from `agent.default_options.get("model")` first,
  falling back to `agent.client.model`. This mirrors the order
  `Agent.__init__` uses (`_agents.py:740`) when assembling
  default_options, so the model the agent runs with is the same model
  the converter publishes \u2014 e.g. when the caller passes
  `default_options={"model": "..."}` to override the bound client.
* Updated the missing-model error message to point at both the client
  and the default_options paths.
* Added tests:
  * tool-only agent with no `instructions` produces a definition
    where `instructions` is `None` and is omitted from the dict
    payload (`Agent.__init__` strips None values from default_options
    before storing them).
  * `default_options['model']` wins over the bound client's model.
  * Fallback to client.model when default_options has no model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(foundry): add deploy_as_prompt_agent helper + samples

Adds `deploy_as_prompt_agent(agent)`, a convenience wrapper around
`to_prompt_agent` that reuses the bound FoundryChatClient's project
client to call `project_client.agents.create_version(...)`. Defaults
`agent_name` / `description` from `agent.name` / `agent.description`
so the Agent stays the single source of truth.

* Exposed from `agent_framework_foundry` and the lazy-loading
  `agent_framework.foundry` namespace (including the .pyi stub).
* Marked experimental with the existing
  `ExperimentalFeature.TO_PROMPT_AGENT` tag.
* Tests cover the happy path, name/description defaulting, explicit
  override, no-name error, metadata + description forwarding, extra
  kwargs passthrough, and the experimental metadata.

Samples:
* Renamed the existing sample to `creating_prompt_agents.py`, drops
  'portable' wording, presents `deploy_as_prompt_agent` first as the
  recommended path and `to_prompt_agent` + `AIProjectClient` as the
  two-step alternative, and adds a cleanup step that deletes the
  published agent so re-runs stay idempotent.
* New `using_prompt_agents.py` shows the end-to-end loop: deploy the
  agent, connect to it with `FoundryAgent` passing the same local
  `@tool` callable, run a query against the deployed prompt agent,
  then clean up.

README updated to introduce `deploy_as_prompt_agent` as the
recommended path and link to both runnable samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(foundry): restore missing-model ValueError in to_prompt_agent

The check was accidentally dropped while reworking docstrings in the
previous commit. Test `test_to_prompt_agent_rejects_missing_model`
exercises this path and was failing on CI as a result.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): rename deploy_as_prompt_agent -> create_prompt_agent

Renames the helper across the foundry package, core lazy-loader stubs,
tests, README and samples. The new name better matches the action
performed (a prompt-agent definition is created in Foundry) and is
consistent with the surrounding ''create_*'' API surface.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): drop create_prompt_agent, enrich to_prompt_agent params

Remove the create_prompt_agent helper and consolidate on to_prompt_agent.
Expose every PromptAgentDefinition parameter that has either an Agent
Framework equivalent (sourced from default_options) or no equivalent
(accepted as a keyword argument).

* default_options-sourced (with kwarg overrides):
  temperature, top_p, string tool_choice
* kwarg-only Foundry knobs:
  reasoning, text, structured_inputs, rai_config, ToolChoiceParam tool_choice

Precedence is always: explicit keyword > default_options entry > unset.

Tests cover every path (defaults, default_options, kwargs, kwarg override).
Samples and README rewritten around the enriched to_prompt_agent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): single source of truth for prompt-agent options

Stop duplicating the generation-parameter surface between FoundryChatOptions
and to_prompt_agent. Translate every field with an Agent Framework equivalent
(temperature, top_p, tool_choice, reasoning, response_format/text/verbosity)
from agent.default_options via a new RawFoundryChatClient helper
_prepare_prompt_agent_options. Only Foundry-specific fields with no AF
equivalent — structured_inputs and rai_config — remain as keyword arguments
on to_prompt_agent.

- tool_choice is dropped when there are no tools (mirrors _prepare_options
  semantics and avoids polluting tool-less prompt agents with Agent.__init__'s
  'auto' default).
- response_format Pydantic models route through
  openai.lib._parsing._responses.type_to_text_format_param; dict shapes go
  through the existing _prepare_response_and_text_format helper.
- default_options is not mutated; text dict is defensively copied.

Tests, README, and creating_prompt_agents.py sample updated to reflect the
new single-source model.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): consolidate prompt-agent sample

Drop creating_prompt_agents.py (the publish-only variant) and rename
using_prompt_agents.py to foundry_prompt_agents.py so the single sample
covers the full convert -> publish -> connect -> run loop. Update the
README link list accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(foundry): run local Agent + deployed agent in same sample

Add an agent.run() call against the local Agent before publishing, then run
the deployed prompt agent on the same query. Expand the docstring with a
compare-and-contrast covering runtime/latency, configurability, and
persistence/sharing differences between the two execution paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(foundry): cover conflicting response_format + text.format in to_prompt_agent

Exercises the ValueError path when a Pydantic response_format would overwrite
an explicit text.format mapping with a different shape. Lifts _chat_client.py
coverage from 89% to 90%.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(foundry): move _prepare_prompt_agent_options into _to_prompt_agent

Lift the translation helper off RawFoundryChatClient and into the
_to_prompt_agent module as a module-private function that takes the client
as its first argument. The chat client no longer needs to carry a method
whose only consumer is the prompt-agent converter, while still serving as
the source of the request-path helper (_prepare_response_and_text_format)
that the converter reuses for dict-shaped response_format values.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(python): codify GA terminology + post-run docs review

Add two pieces of guidance to python/AGENTS.md:

* Terminology - reserve 'GA' for hosted services; use 'released' or 'stable'
  for Agent Framework code/features to match the feature-lifecycle stages.
* Maintaining Documentation - review AGENTS.md and skills at the end of every
  run and update any guidance the conversation made stale; before adding a
  new principle, ask the user to confirm it should be captured.

Also pulls in a docstring fix in foundry_prompt_agents.py that swaps the
stray 'GA' for 'released', applying the new terminology rule.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* address PR review: strict=True default, Tool._deserialize dispatch, sample cleanup safety

- FunctionTool published as strict=True so the server-side schema validation
  matches what the local FoundryAgent(tools=[same_callable]) dispatcher
  enforces. AF FunctionTool has no 'strict' attribute, so the safer default
  is used uniformly instead of silently downgrading to a permissive contract.
- _validate_mapping_tool now dispatches through ProjectsTool._deserialize so
  dict-shaped tools rehydrate to the concrete subclass (FunctionTool,
  WebSearchTool, ...) via the 'type' discriminator instead of returning a
  generic Tool. Added a test that asserts isinstance(WebSearchTool) and a
  new test for the function-typed dict path.
- foundry_prompt_agents.py sample now wraps credential + project client in
  async with and the create_version / run flow in try/finally so a failure
  on connect or run still deletes the published prompt agent rather than
  leaving an orphaned, billable resource in the user's Foundry project.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(ci): correct linkspector ignorePattern typo (./pulls -> ./pull)

GitHub PR URLs use the singular segment /pull/N (compare to /issues/N
for issues). The existing './pulls' ignore pattern never matched
anything as a result, so legitimately stale PR links (e.g. PRs deleted
from forks) surface as linkspector failures on unrelated PRs.

This is the same convention the './issues' rule above already follows.
Fixes the markdown-link-check failure on a dangling link in
dotnet/src/Microsoft.Agents.AI.DurableTask/CHANGELOG.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-27 13:31:21 +00:00
Giles Odigwe 950673ba47 Python: bump package versions for 1.6.0 release (#6017)
* Python: bump package versions for 1.6.0 release

- Released cohort (agent-framework, core, openai, foundry): 1.5.0 -> 1.6.0
- Beta packages (21 packages): 1.0.0b260519 -> 1.0.0b260521
- Alpha packages (azure-contentunderstanding, foundry-hosting, gemini, monty): 1.0.0a260518/19 -> 1.0.0a260521
- ag-ui stays at 1.0.0rc2, orchestrations at 1.0.0rc1 (dependency bounds updated)
- Inter-package dependency lower bounds updated (>=1.5.0,<2 -> >=1.6.0,<2)
- Update CHANGELOG compare links
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: bump RC packages, add shell tool to changelog

- ag-ui: 1.0.0rc2 -> 1.0.0rc3
- orchestrations: 1.0.0rc1 -> 1.0.0rc2
- Add shell tool (#5664) to CHANGELOG
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-22 01:59:20 +00:00
Ben Thomas 8e54f0b0e7 Python: Shell tool with support for local and Docker (#5664)
* feat(tools): add cross-OS LocalShellTool in new agent-framework-tools package

Introduces a safe, cross-OS local shell tool as the first citizen of a new

agent-framework-tools workspace package. Supports persistent (default) and

stateless modes across pwsh/powershell.exe/bash/sh, with policy denylist,

allowlist, approval gating, process-tree kill on timeout, output truncation,

and audit hooks. Integrates with existing provider get_shell_tool(func=...)

factories via FunctionTool kind='shell'.

See docs/decisions/0026-builtin-tools-local-shell.md for the full design.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(tools): security hardening for LocalShellTool

Codifies what LocalShellTool does and does not defend against, and

delegates the security-relevant lifecycle primitive to a battle-tested

library instead of hand-rolled per-OS code.

Changes:

- Adopt psutil for cross-OS process-tree termination (executor + session).

  Replaces hand-rolled taskkill/killpg with one canonical implementation.

- Resolve taskkill.exe to absolute %SystemRoot%\System32 path so PATH

  poisoning cannot redirect us to an attacker-supplied binary.

- Reframe ShellPolicy docstring + ADR + README: denylist is a guardrail,

  not a security boundary.

- Require acknowledge_unsafe=True to set approval_mode='never_require',

  making the unsafe path explicitly opt-in with a self-documenting name.

- Add tests/test_security.py codifying named CVE-style cases. Defenses

  we DO claim are asserted; non-defenses (denylist bypasses via

  backslash insertion, variable expansion, interpreter escape, base64,

  alternative tools, PowerShell-native verbs) are documented as

  expected-to-pass tests so residual risk stays visible.

- Add Threat Model + Confidence Strategy sections to ADR 0026.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(tools): add DockerShellTool sandboxed shell tier

Adds a container-backed shell executor as the recommended pattern for untrusted-input shell workflows. The container provides the security boundary (--network none, non-root user, --read-only, --cap-drop ALL, no-new-privileges, memory/pids limits, tmpfs /tmp), so approval gating is optional unlike LocalShellTool.

Also introduces a ShellExecutor Protocol so callers can plug in custom backends (Firecracker, SSH, WASI) without forking the framework.

Removes the planned HyperlightShellExecutor follow-up from ADR 0026: Hyperlight is a WASM code sandbox with no kernel/userland/shell binary, so a Hyperlight-backed shell is not viable. Docker is the realistic sandbox tier for shell.

Tests: 11 unit tests for argv builders + lifecycle (no Docker daemon required); 3 integration tests gated on is_docker_available().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tools): backport shell-tool fixes from .NET parity review

Applies the applicable subset of bug fixes accumulated during the
.NET shell-tool PR review (microsoft/agent-framework#5604) to the
Python shell tool.

A1 - Quote workdir safely in _maybe_reanchor

  Previously _tool.py used double-quote interpolation when emitting
  the cd/Set-Location prefix, which expanded $VAR, $(), and backticks
  in the workdir path. A workdir containing shell metacharacters could
  trigger arbitrary command execution before the user command ran.

  Replaced with single-quote escaping helpers _quote_posix and
  _quote_powershell that emit literal-string forms safe for both
  hosts.

A5/A6 - Consolidate truncation to a single byte-aware helper

  Extracted a shared truncate_head_tail / truncate_text_head_tail
  helper in _truncate.py. The new implementation distributes odd
  caps so head receives floor(cap/2) and tail receives ceil(cap/2)
  bytes, matching the .NET round-9 fix and ensuring no input bytes
  are silently dropped on the boundary.

  _session.py previously truncated by Python str length while the
  caller passed _max_output_bytes - the unit mismatch is now gone:
  raw byte buffers go through truncate_head_tail and decoded text
  goes through truncate_text_head_tail.

Unit tests added for the truncate and quote helpers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(tools): tone down narrative and overconfident comments in shell tool

The shell tool's docstrings and comments contained two patterns that
the .NET review pushed back on:

- Narrative framing about implementation history ("hard-won",
  "we sidestep", "design inspiration: ...", competitor framework
  name-drops in module docstrings).
- Overstated security guarantees ("battle-tested",
  "reasonable for untrusted input", "recommended executor for any
  agent that runs commands from untrusted input",
  "destructive commands are blocked", "safe local shell tool",
  "blocks shell injection").

Rewrites the affected docstrings and comments to describe what the
code does in neutral terms. Behaviour is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(tools): add ShellEnvironmentProvider for the Python shell tool

Ports the .NET ShellEnvironmentProvider as a Python ContextProvider
so agents using LocalShellTool or DockerShellTool can be primed with
an accurate description of the shell they're talking to (family,
version, OS, working directory, and which CLIs are available).

The provider runs probes through any ShellExecutor, caches the
resulting snapshot, and on every before_run extends the session
instructions with a markdown block describing the shell idiom to
use. A failed first probe leaves the cache empty so the next call
retries (no permanent poisoning).

Probe failures from a narrow set of expected error types
(ShellCommandError, ShellExecutionError, ShellTimeoutError, and
asyncio.TimeoutError from the per-probe timeout) are recorded as
None fields in the snapshot. Other exceptions propagate. Tool
names are validated against ^[A-Za-z0-9._-]+$ before being
interpolated into a probe command.

Includes 12 unit tests covering happy path, stderr fallback,
timeout handling, expected/unexpected exception paths, malicious
tool name rejection, case-insensitive deduplication, retry after
failure, concurrent first-callers sharing one probe, and the
default and custom formatter paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(tools): document ShellEnvironmentProvider and finish comment cleanup

Add a README section introducing ShellEnvironmentProvider, soften two remaining overconfident security-boundary comments in _executor_base.py and the DockerShellTool class docstring, and add a sample (shell_with_environment_provider.py) that demonstrates the provider in stateless and persistent modes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* refactor(tools): move shell samples to python/samples/02-agents/tools

The repository convention is to host samples under python/samples/ rather than inside the package directory. Move the two net-new shell samples (allow-list and environment-provider) to python/samples/02-agents/tools/ and drop the in-package samples/ directory; the existing top-level providers/openai/client_with_local_shell.py already covers the basic LocalShellTool walkthrough.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(tools): cover confine_workdir default and ShellResult.format_for_model

Two new tests in test_local_shell_tool.py exercise the default confine_workdir=True behaviour on POSIX and PowerShell, asserting that 'cd' inside one persistent-mode call does not leak into the next. A new test_shell_result.py module provides direct unit coverage for every conditional branch of ShellResult.format_for_model (stdout, truncated, stderr, timed_out, exit_code) so regressions in the LLM-facing format are caught immediately.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tools): address PR #5664 review feedback

- _tool.py: detect PowerShell via is_powershell() helper instead of basename string match

- _environment.py: use public ContextProvider import (no private _ prefix)

- _session.py: trim _stdout_buf/_stderr_buf after copying to avoid unbounded retention across calls

- _docker.py: short-circuit start()/close() in stateless mode; add configurable shell kwarg (default bash, e.g. 'sh' for alpine)

- tests: parenthesized multi-line assert; alpine integration tests now pass shell='sh'

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tools): satisfy CI quality gates

- pyupgrade: drop quoted self-class refs in __aenter__/method annotations

- ruff format: reflow long lines per workspace style

- pyright: assert psutil non-None in optional-import branch; lowercase mutable module globals; annotate _approval_mode as Literal so tool() Literal-typed kwarg is accepted; add ... body to ShellExecutor.run protocol; remove unused deprecated _kill_tree wrapper

- tests: skip docker integration tests on win32 (Windows containers don't support --read-only / alpine images)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Remove DEFAULT_DENYLIST; document single-session ownership; fix bandit findings

Mirrors the .NET PR #5604 cleanup:

- Remove DEFAULT_DENYLIST from ShellPolicy. ShellPolicy() now ships with an empty deny-list; operators opt into site-specific patterns explicitly. No major agent framework uses regex matching as a primary security control; AutoGen v2 removed theirs. Approval gating + sandbox tier remain the real boundaries.

- Rewrite module / class docstrings to frame ShellPolicy as a UX pre-filter, not a security control.

- Add Single-session ownership paragraphs to ShellExecutor, ShellSession, LocalShellTool, and DockerShellTool: a persistent-mode tool is owned by exactly one conversation / agent session; do not share across users or concurrent conversations.

- Tests now supply explicit deny patterns instead of relying on a default.

- Address Pre-commit Hooks (bandit) CI failures: convert internal-invariant asserts to explicit RuntimeError, annotate intentional subprocess/shell usage with # nosec, document container-internal /tmp paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR #5664 round-2 review feedback

Deny-list documentation drift:

- README and the OpenAI/local-shell sample no longer claim a built-in deny-list of destructive commands. ShellPolicy is described as an optional, operator-supplied UX pre-filter; the real boundaries remain approval gating and the sandbox tier.

Behavioural fixes called out in review:

- ShellPolicy.evaluate() now denies empty / whitespace-only commands explicitly instead of returning allow with no rationale.

- truncate_head_tail() raises ValueError for cap <= 0 instead of silently returning the full input with truncated=False, which previously could defeat output-capping in callers that mis-configured the budget.

- LocalShellTool.as_function() / DockerShellTool.as_function() return the ShellCommandError text directly so the model sees a single, non-redundant 'Command rejected by policy: …' message instead of the prior duplicated 'Command blocked by policy: Command rejected …' wrapping.

- ShellSession POSIX sentinel trailer now snapshots and restores the prior errexit (set -e) state around the trailer, so a user 'set -e' in the persistent shell is no longer permanently disabled by the next run().

Tests:

- New test_shell_parse_rc.py covers the full _parse_rc() edge-case surface (zero, positive, negative, CRLF, no newline, missing prefix, empty input, non-digits, trailing garbage, partial digits).

- test_policy.py asserts the new empty-command deny.

- test_shell_truncate_and_quote.py asserts ValueError for cap=0 and cap<0.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review feedback for shell tool

- _resolve.py: reject empty/whitespace shell override string
- _tool.py / _docker.py: mode-aware default tool description (persistent vs stateless)
- _tool.py: fix misleading workdir docstring (re-anchor, not blocking)
- _types.py: emit stream-agnostic [output truncated] marker
- _policy.py: declare _denies/_allows as dataclass fields
- _environment.py: use $(pwd) instead of $PWD in POSIX probe

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review feedback: shell override flag + probe timeout safety

- _resolve.py: in stateless mode, ensure shell overrides end with -c/-Command so commands aren't misinterpreted as script-file paths.
- ShellExecutor.run / LocalShellTool.run / DockerShellTool.run now accept an optional 	imeout kwarg; ShellEnvironmentProvider drops the outer asyncio.wait_for and lets the executor enforce the probe timeout internally, so cancellation no longer risks leaving a hung subprocess or corrupted session.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback: docker isolation + lifecycle robustness

- pyproject.toml: bump agent-framework-core minimum from 1.2.0 to 1.2.2 to align with the rest of the workspace.
- _docker.py: validate extra_run_args at construction time and reject flags that would dismantle the isolation defaults (--privileged, --cap-add, --security-opt, --network/--net, -v/--volume/--mount, --device, --pid, --ipc, --userns, --user, --read-only, --tmpfs, --add-host, --gpus, --cgroupns, --device-cgroup-rule); also documented the warning on the docstring.
- _docker._stop_container: retry docker rm -f once and log a warning/error when it does not succeed, so operators can audit leaked containers instead of getting a silent success.
- _docker._run_stateless timeout path: fall back to docker rm -f when docker kill fails or times out (--rm only reaps on clean exit), and log instead of silently swallowing communicate() errors.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: alliscode <25218250+alliscode@users.noreply.github.com>
2026-05-22 00:29:59 +00:00
Tao Chen 72a6157c6a [BREAKING] Python: Enable instrumentation by default (#5865)
* Enable instrumentation by default

* Update samples

* Optimization when span is not recording

* Address Copilot comments

* Revert uv.lock

* Add warning

* Formatting

* Fix mypy

* Add disable_instrumentation() with sticky user-intent semantics

Add a public disable_instrumentation() entry point so users can explicitly opt
out of Agent Framework telemetry, with a sticky-disable flag that makes the
user's intent "leading" — no framework code path (foundry's
configure_azure_monitor, configure_otel_providers, enable_instrumentation,
enable_sensitive_telemetry, or direct OBSERVABILITY_SETTINGS.enable_*
writes) can re-enable instrumentation until the user explicitly clears the
disable with enable_instrumentation(force=True) /
enable_sensitive_telemetry(force=True).

Also addresses the two remaining unresolved review threads on the PR:
1. test_observability_settings_defaults_instrumentation_true pins the new
   "ENABLE_INSTRUMENTATION defaults to True when env unset" behavior.
2. test_enable_instrumentation_reads_env_sensitive_data restores coverage
   for the post-import load_dotenv() fallback path.

Implementation:
- ObservabilitySettings.enable_instrumentation / enable_sensitive_data become
  properties backed by _enable_*. While _user_disabled is True, the getters
  return False and the setters drop True writes (defense in depth so third-
  party writes can't subvert the disable).
- Public is_user_disabled read-only property lets integrations (e.g. foundry's
  configure_azure_monitor) cheaply check the disable state without poking at
  privates.
- enable_instrumentation() and enable_sensitive_telemetry() short-circuit with
  an info log when disabled; gain a force=True kwarg that clears the disable.
- configure_otel_providers() still creates providers / exporters / views so a
  later force-enable can use them, but logs an info message when called while
  disabled.
- Foundry's FoundryChatClient.configure_azure_monitor and
  FoundryAgent.configure_azure_monitor early-return when the user has
  disabled, so Azure Monitor's global providers aren't installed unnecessarily.

Tests: 11 new tests covering default-on, env re-read at call time, sticky
behavior against each re-enable surface (enable_instrumentation,
enable_sensitive_telemetry, configure_otel_providers, direct attribute
writes), force=True override, re-arming the disable, and the __all__ export.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: document disable_instrumentation() and force=True paths

Add a "Disabling instrumentation" section to the observability sample README
that walks through:

- The distinction between the ENABLE_INSTRUMENTATION env var (initial,
  non-sticky) and disable_instrumentation() (process-wide, sticky).
- Why the sticky semantics matter: framework integrations like
  FoundryChatClient.configure_azure_monitor() can call
  enable_instrumentation() as part of their setup, and the user's opt-out
  needs to win.
- All five surfaces guarded by the sticky disable (property reads, public
  enable functions, configure_otel_providers, direct attribute writes,
  is_user_disabled-aware integrations).
- The force=True escape hatch on both enable_instrumentation() and
  enable_sensitive_telemetry().
- How third-party integrations should consult OBSERVABILITY_SETTINGS.is_user_disabled.
- The limits of the disable (does not tear down existing providers /
  in-flight spans / third-party instrumentation, does not persist across
  processes).

Cross-links the new section from the ENABLE_INSTRUMENTATION row in the env
vars table.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: soften disable_instrumentation() overclaim about telemetry guarantees

Replace 'no telemetry will be emitted no matter what' (which is too strong,
since callers can still pass force=True or mutate private attributes) with
language framing the disable as a user-intent contract that library and
framework code is expected to honor: the framework actively short-circuits
the public enable paths, force=True and private-attribute writes are
acknowledged as out-of-contract escape hatches that integrations should
not use on the user's behalf.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: correct observability Dependencies section

- opentelemetry-sdk is no longer a hard dependency; it is lazily imported by
  create_resource(), create_metric_views(), and configure_otel_providers()
  with a clear ImportError when missing. Day-to-day instrumentation works
  with opentelemetry-api alone provided some other component configures the
  global OpenTelemetry providers (Azure Monitor, an APM agent, application
  bootstrap, etc.).
- opentelemetry-semantic-conventions-ai is no longer used anywhere in the
  source; remove it from the listed dependencies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: replace stale observability migration guide with current PR's only relevant migration

The old guide documented the move away from setup_observability(otlp_endpoint=...)
which was an earlier-release API change unrelated to this PR and stale enough that
it's more confusing than helpful at this point. Replace it with a short note on the
single migration this PR introduces: callers of
enable_instrumentation(enable_sensitive_data=True) should switch to
enable_sensitive_telemetry(). Cross-link to the Disabling instrumentation section
for the rare 'force on without enabling sensitive data' use case where
enable_instrumentation() still applies.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Eduard van Valkenburg <eavanvalkenburg@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-20 11:52:08 +00:00
Evan Mattson f390595188 Bump to 1.0.0rc2 for unique version (#5965) 2026-05-20 10:01:44 +09:00
Eduard van Valkenburg 4609535e22 Python: feat: add agent-framework-monty (Monty-backed CodeAct provider) (#5915)
* Python: feat: add agent-framework-monty (Monty-backed CodeAct)

New alpha package that wraps pydantic-monty (a Rust-based Python
interpreter) behind the same CodeAct API surface as
agent-framework-hyperlight, so users can swap providers with minimal
code change.

Public API (agent_framework_monty):
- MontyCodeActProvider — ContextProvider that injects a run-scoped
  execute_code tool plus dynamic CodeAct instructions.
- MontyExecuteCodeTool — standalone FunctionTool for mixed-tool agents
  or manual static wiring.
- FileMount / FileMountInput / MountMode — public types mirroring the
  Hyperlight names, with Monty's mode (read-only/read-write/overlay)
  and write_bytes_limit on FileMount.

Constructor kwargs (both classes) mirror Hyperlight where possible:
tools, approval_mode, workspace_root, file_mounts; plus a Monty-only
resource_limits forwarding ResourceLimits to Monty.start().

Filesystem flow:
- workspace_root auto-mounts at /input (read-write), matching Hyperlight.
- file_mounts accepts string shorthand, (host, mount) tuple, or
  FileMount with mode + write cap.
- Files written under read-write mounts are scanned post-execution and
  returned as Content.from_data items (mirrors Hyperlight /output).
- overlay mounts buffer writes in-memory; read-only mounts reject writes.

Internals:
- _monty_bridge.InlineCodeBridge ports the inline (non-durable) bridge
  from anthonychu/maf-codeact-monty-python; handles FunctionSnapshot /
  FutureSnapshot pause/resume, dispatches direct typed calls + the
  call_tool fallback, forwards mount/limits to Monty.start(...).
- generate_type_stubs emits per-tool stubs so Monty's `ty` type-checker
  rejects bad calls before any host tool runs.

Alpha-policy compliance (per python-package-management skill):
- Added agent-framework-monty = { workspace = true } to root
  pyproject.toml.
- Added row to python/PACKAGE_STATUS.md.
- Added monty entry under Experimental in python/AGENTS.md.
- NOT added to core[all]; NO agent_framework.monty lazy shim (deferred
  to beta promotion).

Samples (three sets, import from agent_framework_monty directly):
- samples/02-agents/context_providers/code_act/monty_code_act.py
  (provider pattern) + updated local README.
- samples/02-agents/tools/monty_code_interpreter/ (standalone +
  manual-wiring + README).
- samples/04-hosting/foundry-hosted-agents/responses/11_monty_codeact/
  (full hosted-agent layout with uv-based pyproject.toml + Dockerfile,
  Azure Monitor wiring via APPLICATIONINSIGHTS_CONNECTION_STRING +
  enable_instrumentation, ENABLE_INSTRUMENTATION and
  ENABLE_SENSITIVE_DATA env vars). The alpha wheel is vendored into
  ./wheels/ (gitignored) via vendor-wheel.sh; new row added to the
  parent Responses-API README.

Tests:
- 28 hermetic unit tests (stubbed pydantic_monty).
- 18 integration tests marked @pytest.mark.integration, auto-skipped
  when pydantic_monty is unimportable; exercise the real Monty
  runtime: print round-trip, last-expression value, direct typed
  tool dispatch, call_tool fallback, async tool, asyncio.gather
  parallelism, ty type-check rejection, OS blocked by default,
  workspace_root read+write capture, read-only / overlay mount
  semantics, resource_limits.max_duration_secs abort, approval
  gating end-to-end, full Agent run with a scripted chat client.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix: monty FileMount test compares against the normalized POSIX path

The shorthand string mount goes through _normalize_mount_path, which
rewrites Windows drive letters like 'C:\\Users\\...' into
'/C:/Users/...' (POSIX-style). The Windows CI runners surfaced this
because tmp_path resolves to a backslashed Windows path; the test was
comparing against the raw str(host_a) instead of the normalized form.

Compare against _normalize_mount_path(str(host_a)) so the assertion is
platform-independent.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix: address PR #5915 review feedback

- _execute_code_tool docstring: clarify that the Monty backend supports
  scoped filesystem access via workspace_root / file_mounts (blocked by
  default).
- _to_monty_mount: import pydantic_monty lazily through load_monty so
  missing-dependency errors surface as the same actionable RuntimeError
  the rest of the package raises (not a bare ImportError at module load).
  Renamed _load_monty -> load_monty for the same reason.
- _python_type_repr: emit None for type(None) instead of Any, and
  normalize both typing.Union[...] and PEP-604 X | Y to PEP-604 syntax
  so Optional[X] / Union[..., None] / -> None signatures round-trip
  correctly through ty validation. Added a regression test.
- _PrintCollector: track a running character count instead of
  recomputing sum(len(c) for c in self.chunks) per callback. Eliminates
  the O(n^2) cost on print-heavy code.
- Instructions: mention that the value of the final expression is also
  returned alongside captured stdout (matches actual behavior).
- 11_monty_codeact Dockerfile: pin ghcr.io/astral-sh/uv to 0.11.6
  instead of :latest for reproducible builds.
- 11_monty_codeact README: replace the bare "see parent README" pointer
  with sample-specific steps (./vendor-wheel.sh + uv sync + uv run),
  since the sample uses pyproject.toml + a vendored wheel rather than
  requirements.txt.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: sample: 11_monty_codeact installs agent-framework-monty from PyPI

Drop the vendored-wheel scaffolding now that agent-framework-monty is on
PyPI as an alpha (1.0.0a*) release:

- pyproject.toml: remove [tool.uv.sources] override; keep [tool.uv]
  prerelease = "allow" so uv pulls the alpha automatically.
- Dockerfile: drop the COPY wheels/ step.
- README: drop the ./vendor-wheel.sh setup step and the
  not-yet-on-PyPI warning.
- Delete vendor-wheel.sh and the gitignored wheels/ directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix(monty): harden post-execution file capture against symlink escape

Same class of issue as the MSRC-reported Hyperlight finding: the
post-execution capture walked workspace_root with Path.rglob() +
is_file() + read_bytes() - all of which follow symlinks. An attacker
who controls the workspace (cloned repo, extracted archive, shared
workspace) could pre-place `workspace/leak.txt -> /etc/passwd` or
`workspace/outside_dir -> /etc/` and have host files surface as
captured Content items.

Monty's mount layer already rejects symlink reads from inside the
sandbox across all three modes (verified empirically), so the runtime
path was safe. This commit closes the post-execution scan path.

Changes:
- New `_iter_real_files(root)` walker that uses iterdir() +
  is_symlink() to skip symlinks at every directory level and yields
  only real files. Replaces the previous `host_root.rglob("*")` calls
  in both `_snapshot_writable_mounts` and `_capture_written_files`.
- Use `Path.lstat()` instead of `Path.stat()` so size/mtime can never
  be taken from a symlink target.
- Three new integration tests reproducing the MSRC attack shape
  against the workspace_root flow: symlink-to-file outside workspace,
  symlink-to-directory outside workspace, and a guard ensuring
  legitimate sandbox writes are still captured when symlinks are
  present.

Per user request, hyperlight is untouched in this commit (separate fix).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix(monty): skip symlink regression tests when unsupported

Apply the same Windows-CI safety guard as the hyperlight fix in PR #5919:
the three symlink integration tests create symlinks via Path.symlink_to(),
which fails with OSError / NotImplementedError on unprivileged Windows
runners. Add a local _symlinks_supported helper (mirroring the one in
packages/core/tests/core/test_skills.py) and pytest.skip when symlinks
aren't available, so the tests no longer fail for environment reasons.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: fix(monty): address PR #5915 follow-up review feedback

- _invoke_tool: drop the inspect.iscoroutinefunction(...) branch and
  always `await self.tool_map[name](**kwargs)`. Every entry in
  tool_map is `partial(FunctionTool.invoke, skip_parsing=True)` and
  FunctionTool.invoke is `async def`, so the branching was dead code -
  and on Python versions affected by cpython#98590,
  iscoroutinefunction(partial(bound_async_method, ...)) returns False,
  causing the bridge to take the asyncio.to_thread path, return an
  unawaited coroutine, and surface it as a JSON-serialization failure
  for every tool call. Added a regression test
  test_invoke_tool_awaits_partial_wrapped_async_method.

- generate_type_stubs: skip tools whose name is not a valid Python
  identifier or is a Python keyword. FunctionTool.name has no upstream
  validation, so a name like "weird-name" produced a syntax error in
  the stubs and a name like "broken\n    pass\nasync def injected"
  would inject arbitrary stub source. Non-identifier names stay
  reachable via `call_tool("weird-name", ...)` at runtime; they just
  don't get type-checked stubs. Added regression test
  test_generate_type_stubs_skips_non_identifier_tool_names.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-20 00:35:23 +00:00
Evan Mattson 4b0522d62d Python: Bump Python package versions for a release (#5964)
* Bump Python package versions to 1.5.0 for a release

* Promote orchestrations to 1.0.0rc1

* ci(python-setup): merge dynamic exclude into existing workspace exclude

The python-setup action injected exclude = [...] verbatim into
[tool.uv.workspace], producing a duplicate 'exclude' key when the
section already had a static exclude. Scope the rewrite to the
[tool.uv.workspace] section and append the package to the existing
array when present; idempotent if the package is already excluded.

* Address Copilot review feedback: raise inter-package floors to 1.5.0

- foundry, foundry-local: agent-framework-openai >=1.4.0 -> >=1.5.0
- azure-contentunderstanding: agent-framework-foundry >=1.4.0 -> >=1.5.0
- azurefunctions: pin agent-framework-durabletask to >=1.0.0b260519,<2

Keeps lockstep cohort consistent and avoids mixed 1.4.x / 1.5.0 installs.

* Re-include azurefunctions and durabletask in the uv workspace

The pinned durabletask>=1.4.0 floor is enough to make resolution succeed;
the workspace exclude was over-correction and broke CI samples and pyright
type-checking (re-exports in agent_framework/azure/__init__.pyi plus
samples/04-hosting/{azure_functions,durabletask}/ could not resolve their
imports). Dropping them from agent-framework-core[all] still stands so the
metapackage does not pull them.

* Restore azurefunctions and durabletask in agent-framework-core[all]

The durabletask floor pin keeps users on the safe 1.4.0, so they are once
again included in the metapackage. Update CHANGELOG to reflect the pin
rather than an [all] removal.

* Raise uvicorn ceiling in ag-ui and devui to allow 0.42+

The root override-dependencies pins uvicorn[standard]>=0.34.0 (no upper)
and the workspace lock resolves to 0.47.0. The package ceiling <0.42.0
meant the workspace was no longer testing the declared supported range.
Bump to <1 so the lock fits within the declared bounds.

Also picked up by validate-dependency-bounds: refresh stale orchestrations
RC pin in devui dev deps.
2026-05-20 09:20:53 +09:00
Giles Odigwe 0d09d40f0f Python: Fix GitHubCopilotAgent to include tools added by ContextProvider.before_run in session creation (#5780)
* Fix GitHubCopilotAgent ignoring tools from context providers (#5736)

_create_session and _resume_session only forwarded self._tools (constructor
tools) to CopilotClient.create_session, dropping any tools contributed by
context providers via session_context.extend_tools() during before_run.

Merge provider-contributed tools into runtime_options in both _run_impl and
_stream_updates before session creation, mirroring how RawAgent handles the
merge at lines 1435-1440 in _agents.py. Update _create_session and
_resume_session to combine self._tools with the merged runtime tools.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix GitHubCopilotAgent to include tools added by ContextProvider.before_run in session creation

Fixes #5736

* Fix provider tool merge to avoid mutating caller's list

- Replace in-place .extend() with fresh list creation in both
  _run_impl and _stream_updates paths to prevent mutating the
  caller-provided options['tools'] list (shallow copy issue)
- Also handles immutable Sequence types (e.g. tuple) correctly
- Add test for provider tools forwarded via _resume_session path

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5736: review comment fixes

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-15 14:59:22 +00:00
Giles Odigwe 47fa59f8e9 Python: bump package versions for 1.4.0 release (#5872)
* fixes

* fixes

* Python: bump package versions for 1.4.0 release

Cuts the python-1.4.0 release. MINOR bump on the released cohort
(agent-framework, agent-framework-core, agent-framework-openai,
agent-framework-foundry: 1.3.0 -> 1.4.0), driven by breaking changes
in experimental skills API and new features. All 21 beta packages
stamp 1.0.0b260514, all 3 alpha packages stamp 1.0.0a260514, and
ag-ui remains at 1.0.0rc1 (freshly promoted). Date stamp reflects
2026-05-14 Pacific.

- Released cohort: 1.3.0 -> 1.4.0
- Beta packages (21): 1.0.0b260507 -> 1.0.0b260514
- Alpha packages (3): 1.0.0a260507 -> 1.0.0a260514
- ag-ui: stays at 1.0.0rc1 (dep bound updated only)
- Inter-package dependency lower bounds updated (>=1.3.0 -> >=1.4.0)
- Fix chatkit StructuredInputItem exhaustiveness for openai-chatkit 1.6.4
- Update CHANGELOG compare links
- uv.lock refreshed

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-15 09:31:03 +09:00
Evan Mattson ae666a4887 Python: Bump agent-framework-ag-ui to release candidate stage (#5844)
* Bump agent-framework-ag-ui to release candidate stage

* Mark agent-framework-ag-ui as rc in PACKAGE_STATUS
2026-05-14 14:56:34 +00:00
Evan Mattson 15a11a426a Python: add ag-ui tool result display channel (#5762)
* Python: add ag-ui tool result display channel

Key decisions:
- Add TOOL_RESULT_DISPLAY_KEY and make state_update accept optional state plus a tool_result display payload.
- Keep text as the LLM-bound tool result while using the display marker only for ToolCallResultEvent.content.
- Reuse one outer/inner Content additional_properties extraction helper for state and display markers, preserving fallback behavior when display is absent.

Files changed:
- python/packages/ag-ui/agent_framework_ag_ui/_state.py
- python/packages/ag-ui/agent_framework_ag_ui/_run_common.py
- python/packages/ag-ui/tests/ag_ui/test_run_common.py
- python/packages/ag-ui/tests/ag_ui/golden/test_scenario_deterministic_state.py
- python/issues/done/01-tool-result-display-channel.md

Blockers/notes:
- Slice 1 is complete and moved to issues/done.
- Slice 2 remains for docstring and README documentation.

* Python: document ag-ui tool result display channel

Key decisions:
- Document state_update as the single helper for LLM text, UI-only tool_result display content, and durable shared state.
- Keep the display guidance explicit that text remains LLM-bound while tool_result feeds ToolCallResultEvent.content.
- List both reserved additional_properties markers in the docstring return contract.

Files changed:
- python/packages/ag-ui/agent_framework_ag_ui/_state.py
- python/packages/ag-ui/README.md
- python/issues/done/02-docs-tool-result-display.md

Blockers/notes:
- Slice 2 is complete and moved to issues/done.
- Verification passed: uv run poe syntax -P ag-ui --check; uv run poe test -P ag-ui; uv run poe markdown-code-lint; uv run ruff check packages/ag-ui/agent_framework_ag_ui/_state.py.
- Commit hooks were skipped after poe-check repeatedly rewrote uv.lock ordering; the same checks were run manually and passed.

* Python: update gitignore
2026-05-12 22:12:04 +00:00
Giles Odigwe 4ad96b64e7 Python: [BREAKING] Migrate agent-framework-a2a to a2a-sdk v1.0 (#5752)
* Python: Migrate agent-framework-a2a to a2a-sdk v1.0

Upgrade the a2a-sdk dependency from v0.3.x to v1.0.0 and migrate all
source, tests, samples, and documentation to the v1.0 API.

Key changes:
- Dependency: a2a-sdk>=1.0.0,<2 (was >=0.3.5,<0.3.24)
- Types are now protobuf-based: Part replaces TextPart/FilePart/DataPart
- Enums use SCREAMING_SNAKE_CASE (e.g. TaskState.TASK_STATE_COMPLETED)
- Roles: Role.ROLE_AGENT, Role.ROLE_USER
- Client: SendMessageRequest wrapper, subscribe() replaces resubscribe()
- Server: A2AStarletteApplication replaced by Starlette + route factories
- DefaultRequestHandler now requires agent_card parameter
- TaskUpdater: final parameter removed, add_artifact gains last_chunk
- AgentCard.url removed; use supported_interfaces with AgentInterface
- Stream yields StreamResponse with WhichOneof('payload')

Closes #5661

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: validate fallback URL, remove unused task_id vars

- Raise ValueError with clear message when transport negotiation fails
  and no fallback URL is available (neither url arg nor supported_interfaces)
- Remove unused task_id local in status_update branch
- Inline artifact_event.task_id directly in artifact_update branch

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-11 22:46:12 +00:00
Giles Odigwe 0340b7596b Python: bump package versions for 1.3.0 release (#5706)
* Python: bump package versions for 1.3.0 release

MINOR bump on the released cohort (agent-framework, agent-framework-core,
agent-framework-openai, agent-framework-foundry: 1.2.2 -> 1.3.0). All 22
beta packages stamp 1.0.0b260507 and all 3 alpha packages stamp
1.0.0a260507 per the lockstep convention. Date stamp reflects 2026-05-07
Pacific.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review: bump foundry_local openai floor, fix devui orchestrations pin, clarify breaking scope

- foundry_local: bump agent-framework-openai lower bound from >=1.1.0 to >=1.3.0
- devui: update stale agent-framework-orchestrations dev pin from 1.0.0b260402 to 1.0.0b260507
- CHANGELOG: clarify [BREAKING] applies to experimental skills API only

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Revert devui orchestrations pin to 1.0.0b260402 to avoid breaking DevUI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-08 08:57:02 +09:00
Giles Odigwe 57fb32efc8 Python: Upgrade github-copilot-sdk to v1.0.0b2 with new features (#5665)
* Upgrade github-copilot-sdk to v1.0.0b1 and implement new features

- Bump github-copilot-sdk dependency from 0.2.1 to 1.0.0b1
- Fix breaking type renames: ErrorClass -> ToolExecutionCompleteError,
  Result -> ToolExecutionCompleteResult
- Add instruction_directories support in GitHubCopilotOptions (session-level)
- Add copilot_home support in GitHubCopilotSettings (client-level)
- Add sample: github_copilot_with_instruction_directories.py
- Update README with new env var and sample entry
- Add 8 new unit tests covering the new features (103 total, 96% coverage)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* mypy fix

* small fix

* Address PR feedback: fix resume path, remove copilot_home from Options, bump to beta.2

- Forward runtime_options through _resume_session (fixes silent drop of
  instruction_directories/model/etc on resumed sessions)
- Remove copilot_home from GitHubCopilotOptions (client-level setting only
  consumed at startup, not per-call)
- Bump github-copilot-sdk from 1.0.0b1 to 1.0.0b2
- Add test for instruction_directories override on resumed sessions
- Update existing resume test to match new _resume_session signature

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-07 21:43:47 +00:00
Eduard van Valkenburg 57c901a245 Python: Fix hyperlight WasmSandbox cross-thread Drop and harden hosted-agent sample (#5603)
* update hyperlight to beta and move samples, add hosted agent sample

* Python: Fix hyperlight WasmSandbox cross-thread Drop and harden sample

Root cause: when a worker-side closure raised, the exception's __traceback__
retained frame locals that included the partially constructed PyO3 sandbox.
Future.result() re-raised that exception on the caller thread, and when the
caller's exception was eventually GC'd the frame locals were released
off-thread, dec_ref'ing the unsendable sandbox from the wrong thread and
tripping the PyO3 panic
'_native_wasm::WasmSandbox is unsendable, but is being dropped on another thread'.

Fix:
* Add _SandboxWorker._run_on_worker which catches every exception on the
  worker, drops __traceback__ there, deletes the original exception, and
  re-raises a fresh instance on the caller thread. initialize and execute
  route through it; dispose keeps its bare-submit semantics.
* Add an opt-in diagnostic module _drop_diagnostic (no-op unless
  HYPERLIGHT_TRACE_DROPS=1) that installs a sys.unraisablehook and dumps
  owner-thread + per-thread stacks on any future cross-thread unsendable
  Drop. Useful for triaging similar PyO3 regressions.
* Tests: cross-thread invocation, traceback-leak isolation, _SandboxEntry
  attribute-shape check, and a stale-reference stress test driven through
  asyncio.to_thread.

Sample (samples/04-hosting/foundry-hosted-agents/responses/06_hyperlight_codeact):
* Dockerfile installs agent-framework-* from in-tree source with python/ as
  build context so unreleased fixes can be validated end-to-end.
* call_server.py pins the Responses API version.
* main.py enables include_detailed_errors=True so future tool failures
  surface the actual exception text instead of a bare 'Error: Function
  failed.' string.
* README.md documents the in-tree-package build and the Hyperlight
  hypervisor requirement (/dev/kvm on Linux, MSHV on Windows). Hosted
  environments without hypervisor passthrough surface 'No Hypervisor was
  found for Sandbox'; this is a hosting constraint, not a hyperlight bug.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: remove _drop_diagnostic from hyperlight package

The diagnostic module was useful while bisecting the cross-thread Drop bug,
but it is no longer needed now that _SandboxWorker._run_on_worker prevents
the panic at the source.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: address PR review feedback on hyperlight

- Use lazy agent_framework.hyperlight import in sample main.py.
- Env-driven endpoint (FOUNDRY_AGENT_ENDPOINT) in call_server.py; remove personal URLs.
- Align agent.yaml model deployment with manifest (gpt-4.1-mini).
- Tighten Dockerfile requirements guard; drop dangling deploy.ps1 reference.
- Preserve exception args when sanitizing tracebacks in _run_on_worker.
- Add public _SandboxWorker.is_alive(); update test to avoid private attr.
- Add namespace coverage tests for agent_framework.hyperlight lazy loader.
- Add prominent note: Foundry hosted-agent runtime does not yet support
  Hyperlight (no hypervisor exposed); container works locally with /dev/kvm.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: bump hyperlight-sandbox dependencies to 0.4.x

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: renumber hyperlight codeact sample to 08

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Coerce worker exception args to strings for cross-thread safety

Stringify exc.args on the worker thread before propagating, so any
PyO3 unsendable object captured in args (e.g. via a caller-supplied
callback or underlying SDK) cannot be Dropped on the calling thread.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* moved sample

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-05-05 10:06:16 +00:00
Peter Ibekwe bc42874690 Python: Add Python parity for HttpRequestAction in declarative workflow (#5599)
* Add Python parity for HttpRequestAction in declarative workflow

* Ran pyupgrade and pright to fix CI issues

* Fix conversation ID dot parsing for http executor

* Removed unnecessary export command
2026-05-01 23:04:07 +00:00
Giles Odigwe 570a4d54c2 Python: Support OpenAI and Gemini allowed_tools tool choice (#5322)
* Support OpenAI allowed_tools in ToolMode (#5309)

Add allowed_tools field to ToolMode TypedDict, enabling users to restrict
which tools the model may call via the OpenAI allowed_tools tool_choice
type. This preserves prompt caching by keeping all tools in the tools list
while limiting which ones the model can invoke.

- Add allowed_tools: list[str] to ToolMode TypedDict
- Add validation in validate_tool_mode() (only valid when mode == "auto")
- Convert to OpenAI API format in _prepare_options()
- Add tests for validation and API payload generation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Support OpenAI `allowed_tools` tool choice in Python SDK

Fixes #5309

* Fix #5309: Validate allowed_tools shape and add Chat Completions client support

- validate_tool_mode now checks allowed_tools is a non-string sequence of
  strings and normalizes to list[str], raising ContentError for invalid types
- Add missing allowed_tools branch in _chat_completion_client._prepare_options
  so allowed_tools is emitted as the OpenAI allowed_tools wire format instead
  of being silently dropped
- Add tests for invalid allowed_tools types (string, int, mixed), empty list,
  tuple normalization, and Chat Completions client payload generation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: support allowed_tools with mode 'required' in addition to 'auto'

OpenAI's allowed_tools tool_choice type supports both mode 'auto' and
'required'. Update validation, client conversion, and tests to allow
both modes instead of restricting to 'auto' only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: use Gemini VALIDATED mode for allowed_tools, warn in unsupported providers

- Use FunctionCallingConfigMode.VALIDATED instead of ANY when allowed_tools
  is set with auto mode in Gemini, preserving optional tool-call semantics.
- Handle allowed_tools in required mode with required_function_name precedence.
- Fix allowed_names guard to use identity check (is not None) so empty lists
  are preserved.
- Bump google-genai minimum to >=1.32.0 (VALIDATED added in that version).
- Add warnings in Anthropic and Bedrock when allowed_tools is set but not
  supported.
- Add Gemini unit tests for allowed_tools with auto, required, empty list,
  and required_function_name precedence scenarios.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: Chat Completions API does not support allowed_tools, add integration tests

- Chat Completions API (_chat_completion_client.py) now warns and falls
  back to plain mode when allowed_tools is set, since the /chat/completions
  endpoint does not support the allowed_tools type.
- Add allowed_tools integration test param to both OpenAIChatClient
  (Responses API) and OpenAIChatCompletionClient parametrized option tests.
- Update Chat Completions unit tests to reflect the warn-and-fallback
  behavior.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove unused walrus operator variable in chat completion client

Remove assigned-but-never-used variable 'allowed' flagged by ruff F841.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 17:43:47 +00:00
Evan Mattson f5419b9f38 Python: bump package versions for 1.2.2 release (#5561)
* Python: bump package versions for 1.2.2 release

PATCH bump (1.2.1 -> 1.2.2) for the released cohort. Five PRs land in this
window:

- agent-framework-openai: fix file_search citations breaking the assistant-
  message history roundtrip (#5557) — drives the released-tier PATCH
- agent-framework-orchestrations: [BREAKING] standardize orchestration
  terminal outputs as AgentResponse (#5301)
- agent-framework-core, agent-framework-declarative: preserve Workflow.run()
  shared state across calls, accept list[Message] in declarative start
  executor, and coerce Enum values when serializing PowerFx symbols (#5531)
- agent-framework-foundry-hosting: add hosted Durable Workflow support
  (#5531)
- agent-framework-azure-contentunderstanding: new alpha package — Azure AI
  Content Understanding context provider (#4829)
- dependencies: workspace package dependency refresh (#5555)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260429 and all 4
alpha packages (now including the new contentunderstanding) stamp
1.0.0a260429. Date stamp reflects 2026-04-29 Pacific. Every non-core package
floor on agent-framework-core is raised to >=1.2.2; the new
contentunderstanding package's stale >=1.0.0 floor is brought into line.

Two follow-on fixes bundled to keep validate-dependency-bounds-test green
at lowest-direct resolution:
- Bump agent-framework-azure-contentunderstanding's azure-ai-content
  understanding lower bound from >=1.0.0 to >=1.0.1 (1.0.0 ships without
  proper typing — pyright reports 65 unknown-type errors)
- Add pyright ignore comments to core/foundry/__init__.pyi for the new
  alpha package's type-stub imports, since alpha packages are not in
  core's [all] extra and therefore aren't installed at lowest-direct

* Python: add #5552 to 1.2.2 CHANGELOG

Add the streaming-span observability fix to the Fixed section. PR is on
upstream/main but not yet pulled into origin/main; the code itself will
land via the PR merge.

* Python: address PR #5561 review feedback on dependency bounds

Two packaging fixes flagged in review:

1. agent-framework-azure-contentunderstanding: add agent-framework-foundry
   as a runtime dependency. The package's README directs users to
   `pip install agent-framework-azure-contentunderstanding --pre` and the
   basic example imports `FoundryChatClient` from `agent_framework.foundry`,
   so the documented install path was failing with ImportError. Pulling
   agent-framework-foundry into deps makes the advertised entry path
   self-contained.

2. agent-framework-foundry: bump agent-framework-openai lower bound from
   >=1.1.0 to >=1.2.2,<2. Foundry imports private modules from
   agent_framework_openai (`_chat_client.py:22`, `_agent.py:34`), so
   resolvers were free to pair foundry==1.2.2 with older OpenAI versions
   that lack this release's coordinated Responses/history fix. Lockstep the
   floor with the released cohort to prevent mismatched installs.

Both changes pass `validate-dependency-bounds-test` lower + upper at
their respective packages.
2026-04-29 17:51:48 +09:00
Tao Chen 03e47b5232 Python: Fix spans not correctly nested when using streaming (#5552)
* Fix spans not correctly nested when using streaming

* fix pre commit

* Address comments
2026-04-29 08:21:28 +00:00
Evan Mattson 094f9903b3 Python: Update package dependencies (#5555)
* Update dependencies

* Preserve mcp[ws] and uvicorn[standard] extras in override-dependencies

Bare-package overrides on mcp and uvicorn dropped the [ws] and [standard]
extras (and their transitive deps like httptools, watchfiles) from the
generated lock. Re-add the extras to the overrides so the lock matches
what workspace packages actually request.
2026-04-29 06:18:03 +00:00
Ben Thomas 8b71f9459a Python: Feature/hosted dwf (#5531)
* Fix declarative Workflow.as_agent() by accepting list[Message] in start executor

The declarative start executor (JoinExecutor) only advertised dict and str
in its input_types, so WorkflowAgent.__init__ rejected it with
'Workflow's start executor cannot handle list[Message]'.

Add list[Message] to the JoinExecutor handler annotation and add a
matching branch in DeclarativeActionExecutor._ensure_state_initialized
that extracts the last user-message text and falls through to the
string-input initialization path, so =System.LastMessageText works
end-to-end via as_agent().

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Populate Conversation.messages from list[Message] trigger

When Workflow.as_agent() is invoked with a list[Message], the start executor now populates Conversation.messages / Conversation.history / System.conversations.{id}.messages with prior turns only (excluding the latest user message), and surfaces the latest user message via Inputs.input and System.LastMessage*. This matches InvokeAzureAgent's contract that the messages binding holds prior turns and the executor itself appends the new user input before invoking, avoiding double-append of the trailing user turn while preserving full history (incl. assistant/system/tool roles and multi-modal content) for downstream actions.

* Coerce Enum values when serializing PowerFx symbols

MessageRole and other str-subclass Enums passed isinstance(v, str) and were forwarded to pythonnet unchanged. pythonnet then raised 'MessageRole value cannot be converted to System.String' for every PowerFx primitive when ConditionGroup/Expr eval walked the symbol table containing Conversation.messages. Reduce Enum members to their underlying value before the primitive check so eval sees plain strings/ints.

* Foundry hosting: pass full conversation history to workflow agents

_handle_inner_workflow only forwarded the latest user turn to WorkflowAgent.run, even though _handle_inner_agent already prepends history fetched from Foundry storage to the messages it sends a regular agent. Declarative workflows reset Conversation.messages on every run (state.initialize), so checkpoint replay alone does not give them prior turns - the host has to pass them in, the same way it does for non-workflow agents. Mirror that contract: fetch context.get_history() and pass [*history, *input_messages] to the workflow agent.

* feat(workflows): support combined message + checkpoint_id for multi-turn continuation

Allow Workflow.run(message=..., checkpoint_id=...) so callers can restore
prior workflow state from a checkpoint AND deliver a new message to the
start executor in a single call. The existing reset_context logic
already preserves shared state when checkpoint_id is set, so this gives
us 'fresh start executor invocation with prior state intact' - exactly
what hosted multi-turn declarative workflows need.

- _workflow.py: drop the message+checkpoint_id mutual exclusion and
  update _execute_with_message_or_checkpoint to do both (restore then
  execute) when both are provided.
- _agent.py: in _run_core's checkpoint branch, also forward
  input_messages so WorkflowAgent.run(messages, checkpoint_id=...) works
  end-to-end. Falls back to the legacy 'restore only' behavior when
  messages are absent.
- _declarative_base.py: detect continuation in _ensure_state_initialized
  by checking whether DECLARATIVE_STATE_KEY already exists in shared
  state; if so, refresh inputs/LastMessage* and append non-user trigger
  messages instead of calling state.initialize() (which would wipe
  Conversation/Local/System).
- foundry_hosting/_responses.py: collapse the host's two-call pattern
  (restore-only, then fresh run) into a single combined call now that
  the underlying APIs support it.
- tests: drop the assertion that combined message+checkpoint_id raises.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Pivot: preserve workflow state across run() calls

Replace the prior 'combined message + checkpoint_id in one run()' approach
with a cleaner default: Workflow.run no longer wipes shared state or runner-
context messages between calls. Iteration counting and per-run kwargs still
reset on a fresh-message run; checkpoint and responses runs are continuations
that preserve everything.

This lets a WorkflowAgent be invoked repeatedly on the same instance and
maintain multi-turn context (e.g. accumulated Conversation.messages) without
asking developers to opt in. Hosted-agent multi-turn pattern becomes two
explicit calls: restore-from-checkpoint (drive to idle), then run-with-message.

Key changes:
- _workflow.py: drop _state.clear() and reset_for_new_run() from run().
  Reset iteration count and run kwargs on fresh-message runs only.
  Restore 'Cannot provide both message and checkpoint_id' validation.
  Add async guard: fresh-message run with un-drained pending executor
  messages from a prior run is invalid.
- _runner.py: clear _state before import_state in restore_from_checkpoint
  so restore is authoritative (import_state merges, not replaces).
- _agent.py: revert checkpoint branch to restore-only (no message forward).
- _responses.py (foundry_hosting): two-call host pattern - restore checkpoint
  silently, then run with new user input.
- tests: state-preservation is the new default; rebuild Workflow for clean slate.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix CI lint and mypy issues from prior pivot commit

- _workflow.py: collapse nested if (SIM102), drop redundant assignment (RET504)
- _declarative_base.py: remove unused last_user_msg = tail assignment
  whose Message | None type clashed with the prior Message-typed branch

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address PR review: fix Inputs.input update and checkpoint storage path

- _declarative_base.py: continuation branch was writing 'Inputs.input' via
  state.set, which routes to the Custom namespace and never updates the
  PowerFx-visible Workflow.Inputs.input. Update state_data['Inputs'] in
  place via get_state_data / set_state_data so =Workflow.Inputs.input and
  =inputs.input see the new turn's user text on continuation.
- _declarative_base.py: refresh docstring to clarify that on a list[Message]
  trigger, Conversation.messages excludes the current user message at the
  start of the turn (agent executors append it before invoking the inner
  agent).
- _responses.py: when previous_response_id is supplied (no conversation_id),
  the prior checkpoint lives under <storage>/<previous_response_id> but new
  checkpoints must land under <storage>/<current_response_id> for the next
  turn to find them. Hold onto restore_storage from the get_latest lookup
  and pass it to the restore-only run; pass write_storage (current id) to
  the message-delivery run and to checkpoint cleanup.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright errors in _declarative_base.py for CI

- Replace state._state.get(...) protected access with new public
  is_initialized() method on DeclarativeWorkflowState (also clearer intent
  for the continuation detection use case).
- Add narrow pyright ignores for the Any-typed trigger paths that pyright
  cannot fully narrow (the list[Message] isinstance loop and the
  fallback-DefaultTransform branch).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Copilot review batch: tests + Workflow.reset escape hatch

* Add Workflow.reset() public method as recovery escape hatch when an
  in-flight run aborted (e.g. WorkflowConvergenceException) and the
  workflow is not checkpointed. Update the in-flight messages guard's
  error message to point callers at it.

* Add test_workflow_run_inflight_messages_guard exercising both the
  guard (sync + streaming) and the reset() recovery path.
* Add test_workflow_reset_rejects_concurrent_runs to lock down the
  in-progress guard on reset.

* Add test_as_agent_continuation_preserves_prior_state covering the
  is_continuation branch in _ensure_state_initialized: stamps a marker
  between calls and asserts it survives, while Inputs.input and
  System.LastMessageText refresh to the new turn.

* Add test_powerfx_safe.py regression tests for the Enum branch in
  _make_powerfx_safe (str-subclass, int-subclass, plain Enum, and
  Enums nested in dict/list).

* Drop redundant @pytest.mark.asyncio on
  test_as_agent_round_trip_with_last_message_text (asyncio_mode='auto').

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Skip restore-only pre-pass when checkpoint has pending request_info

Address Copilot review on _responses.py: the restore-only checkpoint
replay populates self._agent.pending_requests for any request_info
events captured in the checkpoint. The follow-up run(input_messages)
call would then route through WorkflowAgent._process_pending_requests,
which expects function-response content and rejects plain text input
as 'unexpected content while awaiting request info responses'.

Workflows resumed from a checkpoint that was idle-with-pending-requests
would therefore fail every subsequent plain-text user turn. Inspect the
loaded checkpoint and skip the pre-pass when its
pending_request_info_events dict is non-empty. Workflows that don't use
request_info (the current sample set) are unaffected; workflows that do
will fall through to a fresh-message run rather than silently corrupting
the routing state.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Loosen azure-ai-agentserver-* pins to major version

The exact-version pins on azure-ai-agentserver-{core,responses,invocations}
forced foundry-hosting consumers to upgrade in lockstep with every beta
bump from upstream. Switch to '>=current,<next-major' so we pick up patch
and feature updates within the same major series without a coordinated
release.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Drop Workflow.reset(); checkpointing is the recovery path

The in-flight-messages guard prevented silent misbehavior, but the
companion Workflow.reset() escape hatch only cleared _messages while
leaving iteration count, executor-local state, and shared State
mutations in an indeterminate condition after a mid-run failure. That
gave a false sense of recovery.

Recovery from a mid-run failure is supported only via checkpoint
restoration. Keep the guard and reframe its error message accordingly;
remove reset() and its tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Tao's review on PR 5531

- Rename Workflow._run_workflow_with_tracing parameter
  is_fresh_message_run -> is_continuation (default False, inverted).
  Fresh-message turns reset per-run accounting; continuations
  (checkpoint restores, responses replays) preserve it.
- Simplify the in-flight-messages guard: _validate_run_params already
  enforces that 'message' is mutually exclusive with 'checkpoint_id'
  and 'responses', so the additional checks were dead code.
- foundry_hosting _responses: move the restore-only pre-pass above
  emit_created/emit_in_progress; restore is preparation, not run
  progress. Drop the skip-restore gate (state preservation requires
  unconditional restore) and instead clear agent.pending_requests
  after the restore-only call. Collapse over-conditioned check.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Don't clear pending_requests after restore-only pre-pass

Pending requests in the restored checkpoint represent genuinely
outstanding HITL requests. The next user input may carry function
responses (Responses API `function_call_output` items become
FunctionResultContent / FunctionApprovalResponseContent), which
`WorkflowAgent._process_pending_requests` correctly extracts and
matches against the populated `pending_requests`. Clearing them
after restore would silently drop that state and force the next turn
to be treated as a fresh input even when the caller is responding to
the outstanding requests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: alliscode <bentho@microsoft.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Evan Mattson <35585003+moonbox3@users.noreply.github.com>
2026-04-29 00:51:49 +00:00
Evan Mattson 866a325b48 Python: [BREAKING] Standardize orchestration terminal outputs as AgentResponse (#5301)
* Fix orchestration outputs so as_agent() returns the final answer only. Align other orchestration outputs

* Fix orchestration output issues from review comments

1. Sample cleanup: Remove commented-out FoundryChatClient block and update
   prerequisites to reference OPENAI_CHAT_MODEL_ID instead of FOUNDRY_* vars.

2. Sequential approval output: Change _EndWithConversation.end_with_agent_executor_response
   from a no-op sink to yield response.agent_response. When the last participant is
   AgentApprovalExecutor (via with_request_info), _EndWithConversation is the output
   executor so the yield produces the terminal answer. When the last participant is a
   regular AgentExecutor, _EndWithConversation is not in output_executors so the yield
   is silently filtered out.

3. Forward data events through WorkflowExecutor: _process_workflow_result now also
   forwards 'data' events from sub-workflows so that emit_intermediate_data=True on
   AgentExecutor works correctly when wrapped in AgentApprovalExecutor.

4. Concurrent docstring: Update _AggregateAgentConversations docstring to say
   'deterministic participant order' instead of 'completion order'.

5. Add test_concurrent_intermediate_outputs_emits_data_events verifying that
   ConcurrentBuilder(intermediate_outputs=True) emits per-participant data events
   alongside the single aggregated output event.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add tests for sequential workflow with_request_info and intermediate_outputs (#5301)

Address PR review comments 2, 3, and 5:

- Add test_sequential_request_info_last_participant_emits_output:
  Verifies that when the last participant is wrapped via with_request_info()
  (AgentApprovalExecutor), the workflow still emits a terminal output after
  approval, exercising the _EndWithConversation.end_with_agent_executor_response
  fallback path.

- Add test_sequential_request_info_with_intermediate_outputs_emits_data_events:
  Verifies that emit_intermediate_data=True works correctly through
  AgentApprovalExecutor wrapping—WorkflowExecutor._process_result already
  forwards data events from sub-workflows, so intermediate agent responses
  surface as data events in the parent workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright type errors from AgentResponse output refactor (#5301)

Update cast() calls in _group_chat.py and _magentic.py to use
WorkflowContext[Never, AgentResponse] instead of the old
WorkflowContext[Never, list[Message]], matching the updated method
signatures in _base_group_chat_orchestrator.py.

Fix _sequential.py _EndWithConversation.end_with_agent_executor_response
to declare WorkflowContext[Any, AgentResponse] so yield_output accepts
AgentResponse[None].

Fix _workflow_executor.py data event forwarding to handle nullable
executor_id.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportUnknownVariableType in _agent.py (#5301)

Extract event.data into a typed local variable before the isinstance
check to avoid pyright narrowing it to AgentResponse[Unknown].

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix pyright reportMissingImports for orjson in file history samples (#5301)

Add pyright: ignore[reportMissingImports] to orjson imports that are
already guarded by try/except ImportError, matching the existing pattern
used elsewhere in the samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback for #5301: review comment fixes

* Address review feedback for #5301: review comment fixes

* Revert sequential_workflow_as_agent sample to FoundryChatClient

Reverts the mistaken switch from FoundryChatClient to OpenAIChatClient
in the sequential workflow as agent sample.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address ultrareview feedback: emit_data_events rename + WorkflowAgent reasoning conversion

Layered on top of the prior review-feedback work in this branch.

Renames:
- AgentExecutor.emit_intermediate_data -> emit_data_events (mechanical
  rename; orchestration semantics live at the orchestration layer, not
  the general-purpose executor). Forwarded through MagenticAgentExecutor,
  AgentApprovalExecutor, and all orchestration call sites.
- HandoffAgentExecutor._check_terminate_and_yield -> _should_terminate
  (pure predicate; no longer yields anything). HandoffBuilder docstring
  rewritten to describe the new per-agent AgentResponse output contract.

WorkflowAgent reasoning-content conversion:
- Add _rewrite_text_to_reasoning(contents) and _msg_as_reasoning(msg)
  helpers; the as_agent() path now reframes text content from data events
  as text_reasoning Content blocks before merging into the AgentResponse.
- Consumers iterate msg.contents and branch on content.type — same path
  they already use for Claude thinking and OpenAI reasoning. No new
  field on Message/AgentResponse/WorkflowEvent.
- Streaming branch constructs fresh AgentResponseUpdate instances instead
  of mutating shared payloads (regression test added).
- Helper _msg_maybe_reasoning consolidates the conditional rewrite at
  three call sites in the non-streaming conversion.

Tests:
- TestWorkflowAgentReasoningHelpers + TestWorkflowAgentDataEventReasoningConversion
  add 9 new tests covering helpers, non-streaming, streaming, mixed content,
  already-reasoning passthrough, and mutation-safety regression.
- Updated test_sequential_as_agent_with_intermediate_outputs_includes_chain
  to assert text_reasoning content for intermediate agents.

* Fix pyright: widen event.data to Any to avoid partial-unknown narrowing

The streaming conversion path narrowed event.data via isinstance against
generic AgentResponse, producing AgentResponse[Unknown] and tripping
reportUnknownVariableType/reportUnknownMemberType. Binding data: Any
before the check keeps runtime behavior identical while restoring a fully
known type for downstream access.

* Clean up design

* Scope to agent output semantics only

* yield AgentResponseUpdate streaming, AgentResponse non-streaming

* Fix mypy/pyright: widen cast types at GroupChat callsites

Eight callsites in _group_chat.py still cast to WorkflowContext[Never,
AgentResponse] but the base orchestrator methods now accept the wider
WorkflowContext[Never, AgentResponse | AgentResponseUpdate] (mode-aware
yields). W_OutT is invariant, so the narrower cast is not assignable.
Magentic was widened in the same commit; this catches the GroupChat
callsites that were missed.

* Python: skip flaky Foundry / Foundry Hosting integration tests (#5553)

These two integration tests have been failing in the merge queue across
multiple unrelated PRs (5301, 5531). Both are marked `@pytest.mark.flaky`
with 3 retries, but all attempts fail back-to-back. Skipping both with a
reason pointing to #5553 so they can be fixed properly without continuing
to block unrelated merges.

- packages/foundry_hosting/tests/test_responses_int.py::TestOptions::test_temperature_and_max_tokens
- packages/foundry/tests/foundry/test_foundry_embedding_client.py::TestFoundryEmbeddingIntegration::test_text_embedding_live

Also includes a one-line uv.lock specifier-ordering normalization
auto-applied by the poe-check pre-commit hook.

---------

Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-29 00:35:36 +00:00
Yung-Shin Lin 1e1eda65ce [Python] Add agent-framework-azure-ai-contentunderstanding package (#4829)
* feat: add agent-framework-azure-contentunderstanding package

Add Azure Content Understanding integration as a context provider for the
Agent Framework. The package automatically analyzes file attachments
(documents, images, audio, video) using Azure CU and injects structured
results (markdown, fields) into the LLM context.

Key features:
- Multi-document session state with status tracking (pending/ready/failed)
- Configurable timeout with async background fallback for large files
- Output filtering via AnalysisSection enum
- Auto-registered list_documents() and get_analyzed_document() tools
- Supports all CU modalities: documents, images, audio, video
- Content limits enforcement (pages, file size, duration)
- Binary stripping of supported files from input messages

Public API:
- ContentUnderstandingContextProvider (main class)
- AnalysisSection (output section selector enum)
- ContentLimits (configurable limits dataclass)

Tests: 46 unit tests, 91% coverage, all linting and type checks pass.

* fix: update CU fixtures with real API data, fix test assertions

- Replace synthetic fixtures with real CU API responses (sanitized)
- Update test assertions to match real data (Contoso vs CONTOSO,
  TotalAmount vs InvoiceTotal, field values from real analysis)
- Add --pre install note in README (preview package)
- Document unenforced ContentLimits fields (max_pages, duration)

* chore: add connector .gitignore, update uv.lock

* refactor: rename to azure-ai-contentunderstanding, fix CI issues

Align naming with Azure SDK convention and AF pattern:
- Directory: azure-contentunderstanding -> azure-ai-contentunderstanding
- PyPI: agent-framework-azure-contentunderstanding -> agent-framework-azure-ai-contentunderstanding
- Module: agent_framework_azure_contentunderstanding -> agent_framework_azure_ai_contentunderstanding

CI fixes:
- Inline conftest helpers to avoid cross-package import collision in xdist
- Remove PyPI badge and dead API reference link from README (package not published yet)

* feat: add samples (document_qa, invoice_processing, multimodal_chat)

- document_qa.py: Single PDF upload, CU context provider, follow-up Q&A
- invoice_processing.py: Structured field extraction with prebuilt-invoice
- multimodal_chat.py: Multi-file session with status tracking
- Add ruff per-file-ignores for samples/ directory
- Update README with samples section, env vars, and run instructions

* feat: add remaining samples (devui_multimodal_agent, large_doc_file_search)

- S3: devui_multimodal_agent/ — DevUI web UI with CU-powered file analysis
- S4: large_doc_file_search.py — CU extraction + OpenAI vector store RAG
- Update README and samples/README.md with all 5 samples

* feat: add file_search integration for large document RAG

Add FileSearchConfig — when provided, CU-extracted markdown is automatically
uploaded to an OpenAI vector store and a file_search tool is registered on
the context. This enables token-efficient RAG retrieval for large documents
without users needing to manage vector stores manually.

- FileSearchConfig dataclass (openai_client, vector_store_name)
- Auto-create vector store, upload markdown, register file_search tool
- Auto-cleanup on close()
- When file_search is enabled, skip full content injection (use RAG instead)
- Update large_doc_file_search sample to use the integration
- 4 new tests (50 total, 90% coverage)

* fix: add key-based auth support to all samples

Follow established AF pattern: check for API key env var first,
fall back to AzureCliCredential. Supports AZURE_OPENAI_API_KEY and
AZURE_CONTENTUNDERSTANDING_API_KEY environment variables.

* FEATURE(python): add analyzer auto-detection, file_search RAG, and lazy init

_context_provider.py:
- Make analyzer_id optional (default None) with auto-detection by media
  type prefix: audio->audioSearch, video->videoSearch, else documentSearch
- Add _ensure_initialized() for lazy client creation in before_run()
- Add FileSearchConfig-based vector store upload
- Fix: background-completed docs in file_search mode now upload to vector
  store instead of injecting full markdown into context messages
- Add _pending_uploads queue for deferred vector store uploads

devui_file_search_agent/ (new sample):
- DevUI agent combining CU extraction + OpenAI file_search RAG

azure_responses_agent (existing sample fix):
- Add AzureCliCredential support and AZURE_AI_PROJECT_ENDPOINT fallback

Tests (19 new), Docs updated (AGENTS.md, README.md)

* feat(cu): MIME sniffing, media-aware formatting, unified timeout, vector store expiration

- Add three-layer MIME detection (fast path → filetype binary sniff → filename
  fallback) to handle unreliable upstream MIME types (e.g. mp4 sent as
  application/octet-stream). Adds filetype>=1.2,<2 dependency.
- Media-aware output formatting: video shows duration/resolution + all fields
  as JSON; audio promotes Summary as prose; document unchanged.
- Unified timeout for all media types (removed file_search special-case that
  waited indefinitely for video/audio). All files use max_wait with background
  polling fallback.
- Vector store created with expires_after=1 day as crash safety net.
- Add 8 MIME sniffing tests (TestMimeSniffing class).

* fix: merge all CU content segments for video/audio analysis

CU's prebuilt-videoSearch and prebuilt-audioSearch analyzers split long
media files into multiple `contents[]` segments. Previously,
`_extract_sections()` only read `contents[0]`, causing truncated
duration, missing transcript, and incomplete fields for any video/audio
longer than a single scene.

Now iterates all segments and merges:
- duration: global min(startTimeMs) → max(endTimeMs)
- markdown: concatenated with `---` separators
- fields: same-named fields collected into per-segment list
- metadata (kind, resolution): taken from first segment

Single-segment results (documents, short audio) are unaffected.

Update test fixture to realistic 3-segment video structure and expand
assertions to verify multi-segment merging. Add documentation for
multi-segment processing and speaker diarization limitation.

* refactor: improve CU context provider docs and remove ContentLimits

- Improve class docstring: clarify endpoint (Azure AI Foundry URL with
  example), credential (AzureKeyCredential vs Entra ID), and analyzer_id
  (prebuilt/custom with auto-selection behavior and reference links)
- Add SUPPORTED_MEDIA_TYPES comments explaining MIME-based matching
  behavior and add missing file types per CU service docs
- Use namespaced logger to align with other packages
- Remove ContentLimits and related code/tests
- Rename DEFAULT_MAX_WAIT to DEFAULT_MAX_WAIT_SECONDS for clarity

* feat: support user-provided vector store in FileSearchConfig

- Add vector_store_id field to FileSearchConfig (None = auto-create)
- Track _owns_vector_store to only delete auto-created stores on close()
- Remove vector_store_name; use internal _DEFAULT_VECTOR_STORE_NAME
- Add inline comments for private state fields
- Document output_sections default in docstring
- Update AGENTS.md, samples, and tests

* fix: remove ContentLimits from README code block

* refactor: create CU client in __init__ instead of __aenter__

Follow Azure AI Search provider pattern: create the client eagerly in
__init__, make __aenter__ a no-op. This ensures __aexit__/close() is
always safe to call and eliminates the _ensure_initialized() workaround.

* docs: add file_search param to class docstring

* feat: introduce FileSearchBackend abstraction for cross-client support

Replace direct OpenAI client usage with FileSearchBackend ABC:
- OpenAIFileSearchBackend: for OpenAIChatClient (Responses API)
- FoundryFileSearchBackend: for FoundryChatClient (Azure Foundry)
- Shared base _OpenAICompatBackend for common vector store CRUD

FileSearchConfig now takes a backend instead of openai_client.
Factory methods from_openai() and from_foundry() for convenience.

BREAKING: FileSearchConfig(openai_client=...) -> FileSearchConfig.from_openai(...)

* refactor: FileSearchBackend abstraction + caller-owned vector store

* fix: file_search reliability and sample improvements

- Poll vector store indexing (create_and_poll) to ensure file_search
  returns results immediately after upload
- Set status to failed when vector store upload fails
- Skip get_analyzed_document tool in file_search mode to prevent
  LLM from bypassing RAG
- Simplify sample auth: single credential, direct parameters
- Use from_foundry backend for Foundry project endpoints

* perf: set max_num_results=10 for file_search to reduce token usage

* fix: move import to top of file (E402 lint)

* chore: remove unused imports

* fix: align azure-ai-contentunderstanding with MAF coding conventions

- Add module-level docstrings to __init__.py and _context_provider.py
- Use Self return type for __aenter__ (with typing_extensions fallback)
- Use explicit typed params for __aexit__ signature
- Add sync TokenCredential to AzureCredentialTypes union
- Pass AGENT_FRAMEWORK_USER_AGENT to ContentUnderstandingClient
- Remove unused ContentLimits from public API and tests
- Fix FileSearchConfig tests to match refactored backend API
- Fix lifecycle tests to match eager client initialization

* refactor: improve CU context provider API surface and fix CI

- Refactor _analyze_file to return DocumentEntry instead of mutating dict
- Remove TokenCredential from AzureCredentialTypes (fixes mypy/pyright CI)
- Remove OpenAIFileSearchBackend/FoundryFileSearchBackend from public API
  (internal to FileSearchConfig factory methods)
- Remove DocumentStatus from public exports (implementation detail)
- Update file_search comments to reflect backend-agnostic design
- Add DocumentStatus enum, analysis/upload duration tracking
- Add combined timeout for CU analysis + vector store upload

* fix: improve file_search samples and move tool guidelines to context provider

- Delete redundant devui_file_search_agent sample (duplicate of azure_openai variant)
- Move tool usage guidelines from sample agent instructions into context provider
  (extend_instructions in step 6, applied automatically for all file_search users)
- Fix file_search purpose: use from_foundry() for Azure OpenAI (purpose="assistants")
- Add filename hint in upload instructions for targeted file_search queries
- Reduce max_num_results from 10 to 3 in both devui samples
- Simplify agent instructions in both samples (remove tool-specific guidance)

* feat: improve source_id, integration tests, and content assertions

- Rename DEFAULT_SOURCE_ID to "azure_ai_contentunderstanding" (matches
  azure_ai_search convention)
- Improve source_id docstring to describe default value
- Clarify _detect_and_strip_files docstring (CU-supported files)
- Add invoice.pdf test fixture from Azure CU samples repo
- Refactor integration tests to use invoice.pdf directly (assert instead
  of skip when fixture missing)
- Add URI content test (Content.from_uri with external URL)
- Add "CONTOSO LTD." content assertion to all integration tests
- Use max_wait=None in integration tests (wait until complete)

* feat: reject duplicate filenames, add integration tests and sample comments

- Reject duplicate document keys in before_run (skip + warn LLM to rename)
- Update _derive_doc_key docstring to document uniqueness constraint
- Add unit tests for duplicate filename rejection (cross-turn and same-turn)
- Add integration test for data URI content (from_uri with base64)
- Add integration test for background analysis (max_wait timeout + resolve)
- Add filename recommendation comments to all samples' Content.from_data()

* chore: improve doc key derivation, comments, and README

- Replace hash-based doc key with uuid4 for anonymous uploads (O(1), no payload traversal)
- Remove hashlib import (no longer needed)
- Add File Naming section to README (filename importance, duplicate rejection)
- Improve inline comments (_derive_doc_key, _extract_binary, URL parsing)

* test: strengthen _format_result assertions with exact expected strings

- Replace loose 'in' checks with exact 'assert formatted == expected'
  for both multi-segment and single-segment format tests
- Add object-type fields (ShippingAddress, Speakers) to test data
  to cover nested dict/list serialization
- Add position-based ordering assertions to verify structural
  correctness (header -> markdown -> fields across segments)

* refactor: move invoice.pdf to shared sample_assets directory

- Move invoice.pdf from tests/cu/test_data/ to
  python/samples/shared/sample_assets/ as single source of truth
- Add INVOICE_PDF_PATH constant in test_integration.py pointing
  to the shared location
- Update document_qa.py, invoice_processing.py, large_doc_file_search.py
  to use invoice.pdf instead of sample.pdf

* refactor: reorganize samples into numbered dirs and simplify auth

- Move script samples into 01-get-started/ with numbered prefixes
  (01_document_qa, 02_multimodal_chat, 03_invoice_processing,
   04_large_doc_file_search)
- Move devui samples into 02-devui/ with 01-multimodal_agent and
  02-file_search_agent/{azure_openai_backend,foundry_backend}
- Move invoice.pdf to CU package-local samples/shared/sample_assets/
- Replace kwargs dicts with direct constructor calls; support both
  API key (AZURE_OPENAI_API_KEY) and AzureCliCredential
- Update README sample table with new paths

* fix: resolve CI lint errors (D205, RUF001, E501)

- Fix D205: single-line docstring summary for _detect_and_strip_files
- Fix RUF001: replace EN DASH with HYPHEN-MINUS in segment headers
- Fix E501: wrap long assertion lines in tests
- Also includes samples reorg and auth simplification

* refactor: overhaul samples — FoundryChatClient, sessions, remove get_analyzed_document

Samples:
- Switch all samples from deprecated AzureOpenAIResponsesClient to FoundryChatClient
- Add 02_multi_turn_session.py showing AgentSession persistence across turns
- Rewrite 03_multimodal_chat.py with real PDF + audio + video (parallel
  analysis), per-modality follow-ups, cross-document question, elapsed
  time, user prompts, and input token counts
- Renumber: 02->03 multimodal, 03->04 invoice, 04->05 file_search

Context provider:
- Remove get_analyzed_document tool -- full content is in conversation
  history via InMemoryHistoryProvider, no retrieval tool needed
- Remove follow-up turn instructions about tools
- Only list_documents tool remains (for status queries)
- Update README to reflect tool removal

* feat: add 05_background_analysis sample and fix 04 session/max_wait

- Add 05_background_analysis.py demonstrating non-blocking CU analysis
  with max_wait=1s, status tracking via list_documents(), and automatic
  background task resolution on subsequent turns
- Fix 04_invoice_processing.py: add max_wait=None and AgentSession
- Rename 05→06 large_doc_file_search
- Update README sample table

* docs: update README and fix sample 06

README:
- Switch Quick Start from AzureOpenAIResponsesClient to FoundryChatClient
- Add AgentSession to Quick Start example
- Fix status values: pending -> analyzing/uploading/ready/failed
- Fix env var: AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME -> AZURE_OPENAI_DEPLOYMENT_NAME
- Update samples section with new paths, link to samples/README.md
- Update multi-segment description to reflect per-segment fields

Sample 06:
- Fix from_openai -> from_foundry for Azure endpoints
- Add AgentSession and max_wait=None

* docs: rewrite README — concise format, prerequisites, CU link

* fix: resolve pyright errors in _format_result segment cast

* docs: add numbered section comments and fresh sample output to all samples

- Add numbered section comments (# 1. ..., # 2. ...) per SAMPLE_GUIDELINES
- Re-run all 6 samples and update expected output with real results
- Fix duplicate sample output blocks in 04 and 05
- Update README code example to use public invoice URL

* feat: add load_settings support for env var configuration

- Make endpoint optional in constructor — auto-loads from
  AZURE_CONTENTUNDERSTANDING_ENDPOINT env var via load_settings()
- Add ContentUnderstandingSettings TypedDict
- Add env_file_path/env_file_encoding params for .env file support
- Add 4 unit tests: env var loading, explicit override, missing
  endpoint error, missing credential error
- Update README with env var auto-resolution docs
- Follows framework convention used by all other packages

* docs: polish README — fix duplicate env var, add Next steps, service limits link

* chore: trim invoice fixture from 199K to 33 lines

Keep only VendorName, InvoiceTotal, DueDate, InvoiceDate, InvoiceId
fields and first 500 chars of markdown. Strip spans/source/coordinates.
Reduces fixture from 6.6MB to 1.2KB.

* feat: per-file analyzer_id override via additional_properties

- Read analyzer_id from Content.additional_properties for per-file override
- Resolution order: per-file > provider-level > auto-detect by media type
- Update class docstring documenting filename and analyzer_id properties
- Update sample 04 to demonstrate per-file override (prebuilt-invoice)
- Add unit test for per-file analyzer override

* Trim PDF test fixture and clarify unique filename requirement

- Trim analyze_pdf_result.json from 4427 to 23 lines by removing
  pages, words, lines, paragraphs, sections, spans, and source
  fields that are not used by any unit test.
- Add docstring note that filename must be unique within a session;
  duplicate filenames are rejected and the file will not be analyzed.

* Update python/packages/azure-ai-contentunderstanding/agent_framework_azure_ai_contentunderstanding/_context_provider.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/agent_framework_azure_ai_contentunderstanding/_context_provider.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/02-devui/02-file_search_agent/azure_openai_backend/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/02-devui/01-multimodal_agent/agent.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update python/packages/azure-ai-contentunderstanding/samples/01-get-started/06_large_doc_file_search.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Fix AGENTS.md to match implementation; remove unused variable in test helper

AGENTS.md:
- Remove _ensure_initialized() reference (client is created in __init__)
- Fix multi-segment docs: segments kept as list, not merged into fields
- Remove get_analyzed_document() reference (only list_documents registered)
- Update sample names to match current directory structure

test_context_provider.py:
- Simplify _make_data_uri() — remove unused 'encoded' variable

* Fix premature file_search instruction for background-completed docs

- Change _resolve_pending_tasks() instruction from 'Use file_search'
  to 'being indexed' since the upload hasn't completed yet at that point.
- Add LLM instruction on upload failure in step 1b so the agent can
  inform the user the document isn't searchable.

* fix: wrap long line in devui agent instructions (E501)

* Fix Copilot review: unused logger, stray code in README, await cancelled tasks

- _file_search.py: Remove unused logger and logging import
- 01-multimodal_agent/README.md: Remove accidentally pasted Python script
- _context_provider.py close(): Await cancelled tasks before closing
  client to prevent 'Task destroyed but pending' warnings

* Sanitize doc keys and fix duplicate filename re-injection

- Add _sanitize_doc_key() to strip control characters, collapse
  whitespace, and cap length at 255 chars — prevents prompt injection
  via crafted filenames in extend_instructions() calls.
- Track accepted doc_keys in step 3 so step 5 only injects content
  for files actually analyzed this turn, not pre-existing duplicates.
- Soften duplicate upload instruction wording (remove IMPORTANT/caps).

* fix: add type annotation to tasks_to_cancel for pyright

* Move per-session mutable state to state dict for session isolation

Previously _pending_tasks, _pending_uploads, and _uploaded_file_ids
were stored on self, shared across all sessions. This caused
cross-session leakage: Session A's background task results could be
injected into Session B's context.

Now these are stored in the per-session state dict. Global copies
(_all_pending_tasks, _all_uploaded_file_ids) are kept on self only
for best-effort cleanup in close().

Add 2 new TestSessionIsolation tests verifying that background tasks
and resolved content stay within their originating session.

* Remove unused AnalysisSection enum values

Only MARKDOWN and FIELDS are handled by _extract_sections().
Remove FIELD_GROUNDING, TABLES, PARAGRAPHS, SECTIONS to avoid
exposing dead options to users.

* Recursively flatten object/array field values for cleaner LLM output

- Use SDK .value property with recursive extraction for object/array fields
- Object: AmountDue -> {Amount: 610, CurrencyCode: USD} (was raw SDK dict)
- Array: LineItems -> list of flattened items (was raw SDK list)
- Update invoice fixture with object/array fields from prebuilt-invoice
- Add 3 unit tests for object, array, and nested object field extraction

* Preserve sub-field confidence; compare full expected JSON in tests

* Remove incorrect MIME aliases (audio/mp4, video/x-matroska)

* feat: add AnalysisInput, content_range, warnings, and category support

- Use SDK AnalysisInput model instead of raw body dict for begin_analyze
- Forward content_range from additional_properties to CU (page/time ranges)
- Extract CU warnings with code/message/target (ODataV4Format) into output
- Include content-level category from classifier analyzers
- Add 5 new tests: warnings, category, content_range forwarding
- Fix pyright with explicit casts; fix en-dash lint (RUF002)

* fix: falsy-0 bug in duration calc; improve test coverage

- Fix start_time_ms=0 treated as falsy by 'or' short-circuit, use
  'is None' checks instead for duration and segment time extraction
- Update warnings test to use RAI ContentFiltered codes
- Enrich warnings extraction to include code/message/target (ODataV4Format)
- Add multi-segment video category test with per-segment assertions

* refactor: split _context_provider.py into focused modules

- Extract _constants.py: SUPPORTED_MEDIA_TYPES, MIME_ALIASES, analyzer maps
- Extract _detection.py: file detection, MIME sniffing, doc key derivation
- Extract _extraction.py: result extraction, field flattening, LLM formatting
- _context_provider.py delegates via thin wrappers (793 lines, was 1255)
- Update test imports to use _constants.py for SUPPORTED_MEDIA_TYPES

* docs: update AGENTS.md with DocumentStatus, FileSearchBackend, and _file_search.py

* refactor: replace AnalysisSection enum with Literal type for simpler DX

- Remove AnalysisSection(str, Enum) class, replace with Literal["markdown", "fields"] type alias
- Users can now pass plain strings: output_sections=["markdown"] — no extra import needed
- AnalysisSection type alias still exported for type annotation use
- Update all samples, tests, and internal code to use string literals
- Address PR review feedback (eavanvalkenburg)

* refactor: replace asyncio.Task with continuation tokens for serializable state

- Replace state["_pending_tasks"] (asyncio.Task — not serializable) with
  state["_pending_tokens"] (dict of continuation token strings) so the
  framework can persist session state to disk/storage
- Resume pending analyses via Azure SDK continuation_token mechanism
- Fix: resumed pollers have stale cached status (done() always False),
  use asyncio.wait_for(poller.result()) with 10s min timeout instead
- Remove _background_poll(), _all_pending_tasks, and task cancellation
- Address PR review feedback (eavanvalkenburg): state must be serializable

* fix: resolve CI lint (RUF052) and mypy (call-overload) errors

* feat: add structured output (Pydantic model) to invoice processing sample

- Use response_format=InvoiceResult for schema-constrained LLM output
- Use output_sections=["fields"] only (no markdown needed for structured output)
- Add LowConfidenceField model with confidence values
- Add comments about prebuilt-invoice extensive schema vs simplified model
- Address PR review feedback (eavanvalkenburg): use structured response

* fix: use FOUNDRY_PROJECT_ENDPOINT and FOUNDRY_MODEL env vars in all samples

Replace AZURE_AI_PROJECT_ENDPOINT → FOUNDRY_PROJECT_ENDPOINT and
AZURE_OPENAI_DEPLOYMENT_NAME → FOUNDRY_MODEL across all sample .py and
README.md files. Address PR review feedback (eavanvalkenburg).

* refactor: remove background_analysis sample, use FoundryChatClient in DevUI

- Remove 05_background_analysis.py (per reviewer feedback — discuss max_wait
  design separately from samples)
- Renumber 06_large_doc_file_search.py → 05_large_doc_file_search.py
- Replace AzureOpenAIResponsesClient with FoundryChatClient in all DevUI samples
- Replace client.as_agent() with Agent(client=client, ...) everywhere
- Add max_wait comments explaining interactive vs batch usage
- Update README.md and AGENTS.md
- Address PR review feedback (eavanvalkenburg)

* fix: vector_stores API moved from beta namespace in OpenAI SDK

* docs: add comments about multi-file support and CU service limits in file_search sample

* fix: broken markdown links after sample removal and renumbering

* fix: migrate BaseContextProvider to ContextProvider (non-deprecated)

* fix: Message(text=) -> Message(contents=[]) for API compatibility

* Inline _constants.py into consuming modules

Remove _constants.py and move constants to where they are used:
- SUPPORTED_MEDIA_TYPES, MIME_ALIASES → _detection.py
- MEDIA_TYPE_ANALYZER_MAP, DEFAULT_ANALYZER → _context_provider.py

Addresses review feedback to reduce file count.

* Mark package as alpha per package management skill

- Version: 1.0.0b260401 → 1.0.0a260401
- Classifier: Development Status 4 - Beta → 3 - Alpha
- Add to PACKAGE_STATUS.md as alpha

Follows the alpha package checklist from python-package-management skill.

* Replace extend_instructions with extend_messages for status notifications

Status/error/result notifications now use extend_messages (conversation
context) instead of extend_instructions (system prompt). This avoids
system prompt bloat and keeps behavioral directives separate from
event notifications.

- 11 extend_instructions calls → extend_messages (role='user')
- 1 extend_instructions retained: tool usage guidelines (behavioral)
- 6 test assertions updated to check context_messages

All 84 unit tests + 5 live integration tests pass.

* Fix lint: E402 import order, ISC004 implicit string concatenation

- Move constants after all imports to fix E402
- Wrap multi-line strings in parentheses inside contents=[] to fix ISC004

* Fix lint: remove unused json import in invoice sample

* Fix CI: apply ruff format + fix E501 line length after reformatting

ruff format expands Message() calls to multi-line, pushing string
indentation deeper. Break long strings to fit within 120 char limit
after formatting. Also removes unused json import in sample.

* Address review feedback: keyword-only args, accept pre-built client, remove wrappers

- All __init__ args now keyword-only (matches FoundryChatClient pattern)
- New 'client' param accepts pre-built ContentUnderstandingClient
- core dep bound: >=1.0.0rc5 → >=1.0.0,<2
- Self import moved after local imports
- Removed 9 static method wrappers; callsites use module functions directly
- Tests updated to import derive_doc_key and format_result directly

* fix: remove duplicate ContentUnderstandingClient instantiation

The client was being created twice — once inside the if/else block and
again unconditionally after it. The second instantiation overwrote the
pre-built client path and failed type checking when credential was None.

* rename: azure-ai-contentunderstanding → azure-contentunderstanding

Package: agent-framework-azure-ai-contentunderstanding → agent-framework-azure-contentunderstanding
Module: agent_framework_azure_ai_contentunderstanding → agent_framework_azure_contentunderstanding
Directory: packages/azure-ai-contentunderstanding → packages/azure-contentunderstanding

Per agreement with PM and MAF team to drop 'AI' from the package name.

* feat: add ContentUnderstanding re-export to agent_framework.foundry namespace

Enables: from agent_framework.foundry import ContentUnderstandingContextProvider

Exports: ContentUnderstandingContextProvider, FileSearchConfig,
FileSearchBackend, AnalysisSection, DocumentStatus

Updates all samples and README to use the foundry namespace import.

* fix: add missing copyright headers to standalone sample scripts

* chore: remove .vscode/settings.json and add to .gitignore

* refactor: reuse FoundryChatClient.client for vector store ops in file_search sample

Address review feedback from TaoChenOSU:
- 05_large_doc_file_search.py: use client.client instead of manually
  constructing AsyncAzureOpenAI; remove openai dependency
- azure_openai_backend/agent.py: import reorder only (AIProjectClient
  kept — required for sync vector store creation in DevUI)

* fix: skip closing client when caller passes pre-built client

When a ContentUnderstandingClient is passed via client=, the caller
owns its lifecycle. Added _owns_client flag so close() only closes
the client when we created it internally.

---------

Co-authored-by: yungshinlin <yungshin@msn.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-04-28 20:55:59 +00:00
Evan Mattson 3a463b8bf6 Python: bump package versions for 1.2.1 release (#5536)
* Python: bump package versions for 1.2.1 release

PATCH bump (1.2.0 -> 1.2.1) for the released cohort. The release window
covers two PRs, no new public APIs:

- agent-framework-core: prevent inner_exception from being lost in
  AgentFrameworkException (#5167)
- samples: add requirements.txt and .env.example to the a2a/ hosting
  sample for pip-based setup (#5510)

Per lockstep convention, all 21 beta packages stamp 1.0.0b260428 and all
3 alpha packages stamp 1.0.0a260428, regardless of per-package code
churn. Every non-core package floor on agent-framework-core is raised to
>=1.2.1 to keep cohort signaling consistent. Date stamp reflects the
local (Asia) cut date 2026-04-28.

* Python: silence pyright unknown-type warnings in hosted-env detection

`azure.ai.agentserver.core` is probed at runtime via `importlib.util.find_spec`
and is not a declared dependency. The existing `# pyright: ignore[reportMissingImports]`
suppresses the missing-import warning, but at `lowest-direct` resolution pyright
still reports the imported symbol (`AgentConfig`) and its members (`from_env`,
`is_hosted`) as unknown, breaking `validate-dependency-bounds-test` for
`packages/core`.

Extend the existing ignore to cover `reportUnknownVariableType` on the import
and `reportUnknownMemberType` on the call site so the bounds check returns to
green. Behavior is unchanged.

Latent since #5455 (shipped in 1.2.0).

* Python: raise agent-framework-gemini lower bound to google-genai>=1.65.0

The Gemini chat client references several `google.genai.types` symbols
(`FileSearch`, `ThinkingLevel`, `SearchTypes`, `McpServer`,
`StreamableHttpTransport`, plus call-site keyword args `mcp_servers` and
`search_types`) that are not present at the lower bound of `google-genai>=1.0.0`.
At `lowest-direct` resolution this caused `validate-dependency-bounds-test` to
fail for `packages/gemini` with eleven `reportAttributeAccessIssue` /
`reportUnknownVariableType` errors.

Walking the upstream `google.genai.types` API:
- `GoogleMaps`, `AuthConfig`: present from 1.40.0
- `FileSearch`: introduced in 1.49.0
- `ThinkingLevel`: introduced in 1.55.0
- `SearchTypes`, `McpServer`, `StreamableHttpTransport`: introduced in 1.65.0

Bump the lower bound to 1.65.0 — the minimum version that exposes every symbol
the package actually uses. Keep the `<2.0.0` upper cap unchanged. With this
bump `validate-dependency-bounds-test` passes for both lower and upper
resolution scenarios across all 27 workspace packages.

Latent since #4847 (Gemini package introduction in 1.1.0); aggravated by
subsequent feature additions that pulled in newer `types.*` symbols.

* Python: add dependabot bumps to 1.2.1 CHANGELOG

Catalog the 15 dependabot dependency updates that merged on `upstream/main`
between python-1.2.0 and the 1.2.1 cut window under a new Changed section:

- Workspace dev/runtime deps: `rich`, `prek`, `python-multipart`, `pyasn1`,
  `pytest` (ag-ui, devui, lab), `uv` (lab)
- Frontend deps: `vite` (devui, chatkit), `postcss` (devui, chatkit, handoff),
  `picomatch` (devui, handoff)

CHANGELOG-only — no source or pyproject.toml changes. PRs themselves merged
upstream independently of this release branch and will be brought in via the
PR merge.
2026-04-28 18:23:26 +09:00
dependabot[bot] 74a5ea8dca Python: Bump prek from 0.3.8 to 0.3.9 in /python (#5228)
* Bump prek from 0.3.8 to 0.3.9 in /python

Bumps [prek](https://github.com/j178/prek) from 0.3.8 to 0.3.9.
- [Release notes](https://github.com/j178/prek/releases)
- [Changelog](https://github.com/j178/prek/blob/master/CHANGELOG.md)
- [Commits](https://github.com/j178/prek/compare/v0.3.8...v0.3.9)

---
updated-dependencies:
- dependency-name: prek
  dependency-version: 0.3.9
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: bump prek to 0.3.9 in lab package and update uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/f17751e5-c5a8-4d42-9555-6bf708a2ef47

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 09:08:58 +00:00
dependabot[bot] 5ba7f8aa6f Bump python-multipart from 0.0.22 to 0.0.26 in /python (#5286)
Bumps [python-multipart](https://github.com/Kludex/python-multipart) from 0.0.22 to 0.0.26.
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.22...0.0.26)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.26
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-28 07:25:28 +00:00
dependabot[bot] 35a0b51523 Python: Bump uv from 0.11.3 to 0.11.6 in /python/packages/lab (#5469)
* Bump uv from 0.11.3 to 0.11.6 in /python/packages/lab

Bumps [uv](https://github.com/astral-sh/uv) from 0.11.3 to 0.11.6.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.11.3...0.11.6)

---
updated-dependencies:
- dependency-name: uv
  dependency-version: 0.11.6
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix CI: update uv from 0.11.3 to 0.11.6 in python/pyproject.toml and regenerate uv.lock

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/a1a7c648-b26f-44e7-bace-d56ed8489053

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

* Fix code quality CI: update uv-pre-commit rev from 0.11.3 to 0.11.6 in .pre-commit-config.yaml

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/cdfdd211-9f1e-4570-bc7c-86fd15240e91

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:24:43 +00:00
dependabot[bot] d28c841c50 Python: Bump pytest from 9.0.2 to 9.0.3 in /python/packages/lab (#5470)
* Bump pytest from 9.0.2 to 9.0.3 in /python/packages/lab

Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.2 to 9.0.3.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.2...9.0.3)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update pytest from 9.0.2 to 9.0.3 across all workspace packages

Fix dependency conflict: agent-framework workspace packages were pinning
pytest==9.0.2 while agent-framework-lab required pytest==9.0.3, causing
uv dependency resolution to fail. Updated all pyproject.toml files and
regenerated uv.lock to use pytest==9.0.3 consistently.

Agent-Logs-Url: https://github.com/microsoft/agent-framework/sessions/d274f7c5-b5ed-4b18-8eab-4db3cfd9d1bf

Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: moonbox3 <35585003+moonbox3@users.noreply.github.com>
2026-04-28 07:24:23 +00:00
Evan Mattson 0b69d7fd15 Python: Bump Python package versions for 1.2.0 release (#5468)
* Bump Python package versions for 1.2.0 release

Released tier bumps 1.1.1 -> 1.2.0 (core, openai, foundry, root) to
reflect additive public APIs landed since 1.1.0: functional workflow API
(#4238) and FunctionTool SKIP_PARSING sentinel (#5424). All beta packages
stamped 1.0.0b260424, alpha packages 1.0.0a260424. All 26 non-core
agent-framework-core floors raised to >=1.2.0,<2. CHANGELOG consolidates
the never-tagged 1.1.1 entries with the post-merge additions into [1.2.0].

* Update CHANGELOG footer links for 1.2.0

Advance [Unreleased] comparison base from python-1.1.0 to python-1.2.0
and add a [1.2.0] reference link comparing python-1.1.0...python-1.2.0
so the heading links resolve correctly.

* Fix CHANGELOG: restore [1.1.1] section and add proper [1.2.0]

Previous commit incorrectly renamed the [1.1.1] header to [1.2.0], which
wiped the historical 1.1.1 entries and wrongly attributed them to 1.2.0.
This restores [1.1.1] to its origin/main content and adds a new [1.2.0]
section above containing only the commits in python-1.1.1..HEAD:

- #4238 functional workflow API
- #5142 GitHub Copilot OpenTelemetry
- #2403 A2A bridge support
- #5070 oauth_consent_request events in Foundry clients
- #5447 FoundryAgent hosted agent sessions
- #5459 hosting server dependency upgrade + types
- #5389 AG-UI reasoning/multimodal parsing fix
- #5440 stop [TOOLBOXES] warning spam
- #5455 user agent prefix fix

Also corrects the [1.2.0] compare base to python-1.1.1 (not 1.1.0) and
adds the missing [1.1.1] reference link.
2026-04-24 19:54:59 +09:00