Files
Eduard van Valkenburg 6e95517659 Python: Split type checkers by target (pyright source, 5 checkers on tests/samples) (#6443)
* Python: Split type checkers by target (pyright source, 5 checkers on tests/samples)

Rework the typing setup along the lines of the 'too many type checkers'
approach:

- Pyright (strict) is now the sole source-code type checker; mypy is
  removed from source and its [tool.mypy] block becomes a relaxed profile
  used only for tests/samples.
- Tests are checked by all five checkers (pyright relaxed, mypy, pyrefly,
  ty, zuban); samples by pyright, pyrefly, and ty. All run in a relaxed/
  basic profile so authors aren't forced into over-annotation.
- Add pyrightconfig.tests.json and bump sample pyright configs to basic.
- Unify test/sample typing onto the same parallel fan-out used by source
  pyright via run_command_items in task_runner.py.
- Make version-conditional imports symmetric: keep or drop the
  '# type: ignore' on both branches so results match across interpreter
  versions (local vs CI).
- Update SKILL.md, DEV_SETUP.md, and CODING_STANDARD.md for the five
  gating checkers and pyright on source+tests+samples.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix merge regressions from main (typing + runtime)

Merging main into the type-checker split branch surfaced regressions that
the new five-checker test suite and unit tests caught:

Runtime fixes:
- anthropic: restore the dropped `cache_read_input_token_count` mapping in
  _parse_usage_from_anthropic (lost during merge conflict resolution).
- gemini: _get_function_calling_mode test helper returned str(enum)
  ('FunctionCallingConfigMode.AUTO') instead of the enum value ('AUTO').
- openai: _response_id_from_token test helper was an infinite self-recursion;
  return token['response_id'].
- orchestrations: reset output_events per approval iteration so the terminal
  output assertion counts only the final run.
- core: drop a stale duplicate harness test whose message ('non-negative')
  contradicted the source ('positive').
- purview: import PolicyLocation/PolicyScope/ProtectionScopeActivities/
  ExecutionMode used by the processor tests.

Type-checker fixes (tests, relaxed profile):
- core: pyright/mypy/pyrefly/ty/zuban green-ups across the harness, MCP,
  observability and types tests.
- anthropic/openai: route provider-namespaced UsageDetails keys through a
  dict cast (extra_items TypedDict unsupported by mypy/ty).
- purview: typed model constructors and cache-mock casts.
- ag-ui: annotate WorkflowContext[Any, Any] so yield_output accepts test
  payloads, guard Optional forwarded_props, and ty-ignore intentional bad args.

Source pyright (sole source checker) flagged unnecessary ignores newly
introduced by merged code in core _tools.py and declarative _declarative_base.py.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Isolate per-package mypy cache in test-typing fan-out

The parallel test-typing fan-out runs many mypy processes concurrently,
all defaulting to a single shared ./.mypy_cache. Concurrent writes corrupt
the cache and mypy aborts with INTERNAL ERROR (intermittently, depending on
worker timing) -- which is why CI's Test Typing job failed on a shifting set
of packages while a single-package run was fine.

Give each mypy invocation an isolated cache dir keyed by its target paths so
incremental caching still works per package without races. Other checkers
(zuban/pyrefly/ty/pyright) maintain their own caches and are unaffected.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Make lab pyright-only on source (drop source mypy)

Lab was the last package still running mypy on its source code, requiring
mypy-only `# type: ignore` comments that pyright (the sole source checker
everywhere else) flags as unnecessary. Align lab with the rest of the
monorepo:

- Remove the lab source mypy poe tasks (mypy-gaia/lightning/tau2) and the
  now-dead strict [tool.mypy] config block.
- Drop the 'Run lab mypy' CI step; lab source is type-checked by pyright only.

Lab tests remain covered by the workspace test-typing fan-out (mypy, pyrefly,
ty, zuban, pyright over tests using the relaxed root config).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Fix test-typing regressions from latest main merge

A fresh merge from main brought in new test code never run under the
five-checker test-typing suite. Green up across the affected packages:

- core: narrow Optional span.attributes with 'and' guards in span filters
  and assert+cast the json.loads(...attributes[...]) reads (test_observability);
  match the existing as_agent ignore on the protocol-typed fixture (test_clients).
- openai: align new streaming tests with the established chat_options dict
  pattern (ChatOptions TypedDict isn't assignable to dict), route Optional
  .annotations[0] access through a small _first_annotation helper (mirrors the
  file's assert-not-None convention), and annotate a mapped ResponseStream.
- foundry_hosting: annotate error: dict[str, Any] = body.get(...) or {}
  (zuban needs the annotation).
- foundry: narrow ignores for the live AIProjectClient credential arg (pyrefly)
  and connections.get_default (zuban) SDK type gaps.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* updated pyright version

* pyright fix

* Python: Fix source typing for pyright 1.1.410

Pyright 1.1.410 tightened several checks. Apply the same source fixes as
upstream PR #6275:

- anthropic: import AsyncAnthropicBedrock from anthropic.lib.bedrock and
  AsyncAnthropicVertex from anthropic.lib.vertex (no longer re-exported from
  the anthropic top-level package -> reportPrivateImportUsage).
- core _types.py: cast the transform-hook result to UpdateT (reportAssignmentType).
- core _workflows/_events.py: annotate the @contextmanager helper as
  Generator[None] instead of Iterator[None] (reportDeprecated).
- redis: build the combined filter expression with an explicit loop instead of
  reduce(and_, ...), which pyright could no longer fully type (drops the now
  unused functools.reduce / operator.and_ imports).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Python: Accept plain-text body in Azure Functions workflow/run endpoint

The workflow_orchestrator already accepts plain strings as well as JSON
objects via context.get_input(), but the start_workflow_orchestration HTTP
handler only accepted JSON and returned 400 for any non-JSON body. This made
the functions integration tests that POST text/plain to /api/workflow/run
(e.g. test_09_workflow_shared_state) fail consistently with 400 != 202.

Fall back to the raw request body (decoded as UTF-8) when the body is not
JSON, rejecting only a truly empty body. The JSON path is unchanged.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-18 15:06:20 +00:00
..
2025-10-16 21:46:04 +00:00

Microsoft Agent Framework Purview Integration (Python)

agent-framework-purview adds Microsoft Purview (Microsoft Graph dataSecurityAndGovernance) policy evaluation to the Microsoft Agent Framework. It lets you enforce data security / governance policies on both the prompt (user input + conversation history) and the model response before they proceed further in your workflow.

Status: Preview

Key Features

  • Middleware-based policy enforcement (agent-level and chat-client level)
  • Blocks or allows content at both ingress (prompt) and egress (response)
  • Works with any Agent / agent orchestration using the standard Agent Framework middleware pipeline
  • Supports both synchronous TokenCredential and AsyncTokenCredential from azure-identity
  • Configuration via PurviewSettings / PurviewAppLocation
  • Built-in caching with configurable TTL and size limits for protection scopes in PurviewSettings
  • Background processing for content activities and offline policy evaluation

When to Use

Add Purview when you need to:

  • Prevent sensitive data leaks: Inline blocking of sensitive content based on Data Loss Prevention (DLP) policies.
  • Enable governance: Log AI interactions in Purview for Audit, Communication Compliance, Insider Risk Management, eDiscovery, and Data Lifecycle Management.
  • Prevent sensitive or disallowed content from being sent to an LLM
  • Prevent model output containing disallowed data from leaving the system
  • Apply centrally managed policies without rewriting agent logic

Prerequisites

  • Microsoft Azure subscription with Microsoft Purview configured.
  • Microsoft 365 subscription with an E5 license and pay-as-you-go billing setup.

Authentication

PurviewClient uses the azure-identity library for token acquisition. You can use any TokenCredential or AsyncTokenCredential implementation.

Scopes

PurviewSettings.get_scopes() derives the Graph scope list (currently https://graph.microsoft.com/.default style).


Quick Start

import asyncio
from agent_framework import Agent, Message, Role
from agent_framework.openai import OpenAIChatCompletionClient
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
from azure.identity import InteractiveBrowserCredential

async def main():
	client = OpenAIChatCompletionClient()  # uses environment for endpoint + deployment

	purview_middleware = PurviewPolicyMiddleware(
		credential=InteractiveBrowserCredential(),
		settings=PurviewSettings(app_name="My Sample App")
	)

	agent = Agent(
		client=client,
		instructions="You are a helpful assistant.",
		middleware=[purview_middleware]
	)

	response = await agent.run(Message("user", ["Summarize zero trust in one sentence."]))
	print(response)

asyncio.run(main())

If a policy violation is detected on the prompt, the middleware terminates the run and substitutes a system message: "Prompt blocked by policy". If on the response, the result becomes "Response blocked by policy".


Configuration

PurviewSettings

PurviewSettings(
    app_name="My App",                         # Required: Display / logical name
    app_version=None,                          # Optional: Version string of the application
    tenant_id=None,                            # Optional: Tenant id (guid), used mainly for auth context
    purview_app_location=None,                 # Optional: PurviewAppLocation for scoping
    graph_base_uri="https://graph.microsoft.com/v1.0/",
    blocked_prompt_message="Prompt blocked by policy",      # Custom message for blocked prompts
    blocked_response_message="Response blocked by policy",  # Custom message for blocked responses
    ignore_exceptions=False,                   # If True, non-payment exceptions are logged but not thrown
    ignore_payment_required=False,             # If True, 402 payment required errors are logged but not thrown
    cache_ttl_seconds=14400,                   # Cache TTL in seconds (default 4 hours)
    max_cache_size_bytes=200 * 1024 * 1024     # Max cache size in bytes (default 200MB)
)

Caching

The Purview integration includes built-in caching for protection scopes responses to improve performance and reduce API calls:

  • Default TTL: 4 hours (14400 seconds)
  • Default Cache Size: 200MB
  • Cache Provider: InMemoryCacheProvider is used by default, but you can provide a custom implementation via the CacheProvider protocol
  • Cache Invalidation: Cache is automatically invalidated when protection scope state is modified
  • Exception Caching: 402 Payment Required errors are cached to avoid repeated failed API calls

You can customize caching behavior in PurviewSettings:

from agent_framework.microsoft import PurviewSettings

settings = PurviewSettings(
    app_name="My App",
    cache_ttl_seconds=14400,           # 4 hours
    max_cache_size_bytes=200 * 1024 * 1024  # 200MB
)

Or provide your own cache provider:

from typing import Any
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings, CacheProvider
from azure.identity import DefaultAzureCredential

class MyCustomCache(CacheProvider):
    async def get(self, key: str) -> Any | None:
        # Your implementation
        pass

    async def set(self, key: str, value: Any, ttl_seconds: int | None = None) -> None:
        # Your implementation
        pass

    async def remove(self, key: str) -> None:
        # Your implementation
        pass

credential = DefaultAzureCredential()
settings = PurviewSettings(app_name="MyApp")

middleware = PurviewPolicyMiddleware(
    credential=credential,
    settings=settings,
    cache_provider=MyCustomCache()
)

To scope evaluation by location (application, URL, or domain):

from agent_framework.microsoft import (
	PurviewAppLocation,
	PurviewLocationType,
	PurviewSettings,
)

settings = PurviewSettings(
	app_name="Contoso Support",
	purview_app_location=PurviewAppLocation(
		location_type=PurviewLocationType.APPLICATION,
		location_value="<app-client-id>"
	)
)

Customizing Blocked Messages

By default, when Purview blocks a prompt or response, the middleware returns a generic system message. You can customize these messages by providing your own text in the PurviewSettings:

from agent_framework.microsoft import PurviewSettings

settings = PurviewSettings(
	app_name="My App",
	blocked_prompt_message="Your request contains content that violates our policies. Please rephrase and try again.",
	blocked_response_message="The response was blocked due to policy restrictions. Please contact support if you need assistance."
)

Exception Handling Controls

The Purview integration provides fine-grained control over exception handling to support graceful degradation scenarios:

from agent_framework.microsoft import PurviewSettings

# Ignore all non-payment exceptions (continue execution even if policy check fails)
settings = PurviewSettings(
    app_name="My App",
    ignore_exceptions=True  # Log errors but don't throw
)

# Ignore only 402 Payment Required errors (useful for tenants without proper licensing)
settings = PurviewSettings(
    app_name="My App",
    ignore_payment_required=True  # Continue even without Purview Consumptive Billing Setup
)

# Both can be combined
settings = PurviewSettings(
    app_name="My App",
    ignore_exceptions=True,
    ignore_payment_required=True
)

Selecting Agent vs Chat Middleware

Use the agent middleware when you already have / want the full agent pipeline:

from agent_framework import Agent
from agent_framework.openai import OpenAIChatCompletionClient
from agent_framework.microsoft import PurviewPolicyMiddleware, PurviewSettings
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()
client = OpenAIChatCompletionClient()

agent = Agent(
	client=client,
	instructions="You are helpful.",
	middleware=[PurviewPolicyMiddleware(credential, PurviewSettings(app_name="My App"))]
)

Use the chat middleware when you attach directly to a chat client (e.g. minimal agent shell or custom orchestration):

import os
from agent_framework import Agent
from agent_framework.openai import OpenAIChatCompletionClient
from agent_framework.microsoft import PurviewChatPolicyMiddleware, PurviewSettings
from azure.identity import DefaultAzureCredential

credential = DefaultAzureCredential()

client = OpenAIChatCompletionClient(
	model=os.environ["AZURE_OPENAI_MODEL"],
	azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
	credential=credential,
	middleware=[
		PurviewChatPolicyMiddleware(credential, PurviewSettings(app_name="My App (Chat)"))
	],
)

agent = Agent(client=client, instructions="You are helpful.")

The policy logic is identical; the difference is only the hook point in the pipeline.


Middleware Lifecycle

  1. Before agent execution (prompt phase): all context.messages are evaluated.
    • If no valid user_id is found, processing is skipped (no policy evaluation)
    • Protection scopes are retrieved (with caching)
    • Applicable scopes are checked to determine execution mode
    • In inline mode: content is evaluated immediately
    • In offline mode: evaluation is queued in background
  2. If blocked: context.result is replaced with a system message and context.terminate = True.
  3. After successful agent execution (response phase): the produced messages are evaluated using the same user_id from the prompt phase.
  4. If blocked: result messages are replaced with a blocking notice.

The user identifier is discovered from Message.additional_properties['user_id'] during the prompt phase and reused for the response phase, ensuring both evaluations map consistently to the same user. If no user_id is present, policy evaluation is skipped entirely.

You can customize the blocking messages using the blocked_prompt_message and blocked_response_message fields in PurviewSettings. For more advanced scenarios, you can wrap the middleware or post-process context.result in later middleware.


Exceptions

Exception Scenario
PurviewPaymentRequiredError 402 Payment Required - tenant lacks proper Purview licensing or consumptive billing setup
PurviewAuthenticationError Token acquisition / validation issues
PurviewRateLimitError 429 responses from service
PurviewRequestError 4xx client errors (bad input, unauthorized, forbidden)
PurviewServiceError 5xx or unexpected service errors

Exception Handling

All exceptions inherit from PurviewServiceError. You can catch specific exceptions or use the base class:

from agent_framework.microsoft import (
    PurviewPaymentRequiredError,
    PurviewAuthenticationError,
    PurviewRateLimitError,
    PurviewRequestError,
    PurviewServiceError
)

try:
    # Your code here
    pass
except PurviewPaymentRequiredError as ex:
    # Handle licensing issues specifically
    print(f"Purview licensing required: {ex}")
except (PurviewAuthenticationError, PurviewRateLimitError, PurviewRequestError, PurviewServiceError) as ex:
    # Handle other errors
    print(f"Purview enforcement skipped: {ex}")

Notes

  • User Identification: Provide a user_id per request (e.g. in Message(..., additional_properties={"user_id": "<guid>"})) for per-user policy scoping. If no user_id is provided, policy evaluation is skipped entirely.
  • Blocking Messages: Can be customized via blocked_prompt_message and blocked_response_message in PurviewSettings. By default, they are "Prompt blocked by policy" and "Response blocked by policy" respectively.
  • Streaming Responses: Post-response policy evaluation presently applies only to non-streaming chat responses.
  • Error Handling: Use ignore_exceptions and ignore_payment_required settings for graceful degradation. When enabled, errors are logged but don't fail the request.
  • Caching: Protection scopes responses and 402 errors are cached by default with a 4-hour TTL. Cache is automatically invalidated when protection scope state changes.
  • Cold-cache parallelization: On a ProtectionScopes cache miss, scopes are refreshed in the background while ProcessContent runs in the foreground.
  • Background Processing: Content Activities and offline Process Content requests are handled asynchronously using background tasks to avoid blocking the main execution flow.