github-featured/microsoft--agent-framework
2
0
Fork 0
Code Issues Pull Requests Deployments Models Agent Notes Packages Projects Releases Wiki Activity

Rewording

Browse Source
This commit is contained in:
Tao Chen
2026-08-20 13:28:40 -07:00
parent dce1deb638
commit 5fcf7ea757
4 changed files with 118 additions and 115 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/packages/core/agent_framework/observability.py
+44 -42
View File
@@ -733,7 +733,8 @@ def create_metric_views() -> list[View]:
# Token recognized in the OTEL_SEMCONV_STABILITY_OPT_IN env var that opts into the GenAI
# conventions above the v1.36.0 stable release (collectively "experimental"; see
# conventions above the v1.36.0 baseline (referred to here as "latest", since even the
# baseline is not itself a stable release; see
# https://github.com/open-telemetry/semantic-conventions/blob/v1.37.0/docs/gen-ai).
GEN_AI_LATEST_EXPERIMENTAL_OPT_IN: Final[str] = "gen_ai_latest_experimental"
@@ -779,17 +780,18 @@ class ObservabilitySettings:
Can be set via environment variable ENABLE_SENSITIVE_DATA.
enable_console_exporters: Enable console exporters for traces, logs, and metrics.
Default is False. Can be set via environment variable ENABLE_CONSOLE_EXPORTERS.
enable_message_events: Emit the stable v1.36.0 GenAI message events (``gen_ai.system.message``,
``gen_ai.user.message``, ``gen_ai.assistant.message``, ``gen_ai.tool.message``, ``gen_ai.choice``).
Default is True. Can be set via environment variable ENABLE_MESSAGE_EVENTS. Only takes effect
when sensitive data capture is enabled.
enable_message_events: Emit the baseline v1.36.0 GenAI message events (``gen_ai.system.message``,
``gen_ai.user.message``, ``gen_ai.assistant.message``, ``gen_ai.tool.message``, ``gen_ai.choice``)
for model invocation. Default is True. Can be set via environment variable ENABLE_MESSAGE_EVENTS.
Only takes effect when sensitive data capture is enabled.
otel_semconv_stability_opt_in: A comma-separated list of category-specific values, following the
standard OpenTelemetry comma-separated opt-in list format, currently only containing a single
token ``"gen_ai_latest_experimental"``. v1.36.0 is the OTel-recommended stable release; every
version above it is collectively "experimental". The default, unlike upstream OpenTelemetry which
defaults to stable-only, ``"gen_ai_latest_experimental"`` selects the conventions above v1.36.0;
a list that omits that token (e.g. ``""``) selects the v1.36.0 conventions instead. Can be set via
environment variable OTEL_SEMCONV_STABILITY_OPT_IN.
token ``"gen_ai_latest_experimental"``. v1.36.0 is the OTel-recommended baseline; every
version above it is referred to here as "latest" (per OTel's own stability warning, even the
baseline is not a stable release of the GenAI conventions). The default, unlike upstream
OpenTelemetry which defaults to the baseline, ``"gen_ai_latest_experimental"`` selects the latest
conventions above v1.36.0; a list that omits that token (e.g. ``""``) selects the v1.36.0
conventions instead. Can be set via environment variable OTEL_SEMCONV_STABILITY_OPT_IN.
vs_code_extension_port: The port the AI Toolkit or Microsoft Foundry VS Code extensions are listening on.
Default is None.
Can be set via environment variable VS_CODE_EXTENSION_PORT.
@@ -891,15 +893,15 @@ class ObservabilitySettings:
@property
def use_latest_experimental_gen_ai_semconv(self) -> bool:
"""Whether to emit the GenAI semantic conventions above the v1.36.0 stable release.
"""Whether to emit the GenAI semantic conventions above the v1.36.0 baseline.
v1.36.0 is the OTel-recommended stable release; every version above it is collectively
"experimental".
v1.36.0 is the OTel-recommended baseline; every version above it is referred to here as
"latest".
Computed from ``otel_semconv_stability_opt_in`` (env var ``OTEL_SEMCONV_STABILITY_OPT_IN``), a
comma-separated opt-in list per the standard OpenTelemetry format. Agent Framework defaults this
to True (opted into the conventions above v1.36.0) when the setting is unset, which differs from
upstream OpenTelemetry's default of stable-only.
upstream OpenTelemetry's default of retaining the baseline conventions.
"""
if self.otel_semconv_stability_opt_in is None:
return True
@@ -1341,13 +1343,13 @@ def configure_otel_providers(
the environment variable ENABLE_SENSITIVE_DATA if set. Default is None.
enable_console_exporters: Enable console exporters for traces, logs, and metrics.
Overrides the environment variable ENABLE_CONSOLE_EXPORTERS if set. Default is None.
enable_message_events: Emit the stable v1.36.0 GenAI message events (``gen_ai.system.message``, etc.).
Overrides the environment variable ENABLE_MESSAGE_EVENTS if set. Default is None, which resolves
to True (events enabled).
enable_message_events: Emit the baseline v1.36.0 GenAI message events (``gen_ai.system.message``, etc.)
for model invocation. Overrides the environment variable ENABLE_MESSAGE_EVENTS if set. Default is
None, which resolves to True (events enabled).
otel_semconv_stability_opt_in: a comma-separated list of category-specific values (see
``ObservabilitySettings.otel_semconv_stability_opt_in`` for the full explanation). Overrides the
environment variable OTEL_SEMCONV_STABILITY_OPT_IN if set. Default is None, which resolves to the
conventions above the v1.36.0 stable release.
conventions above the v1.36.0 baseline.
exporters: A list of custom exporters for logs, metrics or spans, or any combination.
These will be added in addition to exporters configured via environment variables.
Default is None.
@@ -1652,7 +1654,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
system_instructions = _get_instructions_from_options(opts)
_capture_current_agent_system_instructions_experimental(
_capture_current_agent_system_instructions_latest_experimental(
agent_span,
span,
system_instructions,
@@ -1665,7 +1667,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
messages=messages,
system_instructions=system_instructions,
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=messages,
system_instructions=system_instructions,
@@ -1752,7 +1754,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
finish_reason=finish_reason,
output=True,
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=response.messages,
finish_reason=finish_reason,
@@ -1783,7 +1785,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
with _get_span(attributes=attributes, span_name_attribute=OtelAttr.REQUEST_MODEL) as span:
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
system_instructions = _get_instructions_from_options(opts)
_capture_current_agent_system_instructions_experimental(
_capture_current_agent_system_instructions_latest_experimental(
agent_span,
span,
system_instructions,
@@ -1793,7 +1795,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
messages=messages,
system_instructions=system_instructions,
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=messages,
system_instructions=system_instructions,
@@ -1834,7 +1836,7 @@ class ChatTelemetryLayer(Generic[OptionsCoT]):
finish_reason=finish_reason,
output=True,
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=response.messages,
finish_reason=finish_reason,
@@ -1993,7 +1995,7 @@ class AgentTelemetryLayer:
span = _start_streaming_span(attributes, OtelAttr.AGENT_NAME)
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=messages,
system_instructions=_get_instructions_from_options(dict(merged_options)),
@@ -2066,7 +2068,7 @@ class AgentTelemetryLayer:
and response.messages
and span.is_recording()
):
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=response.messages,
output=True,
@@ -2131,7 +2133,7 @@ class AgentTelemetryLayer:
with _get_span(attributes=attributes, span_name_attribute=OtelAttr.AGENT_NAME) as span:
try:
if OBSERVABILITY_SETTINGS.SENSITIVE_DATA_ENABLED and messages and span.is_recording():
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=messages,
system_instructions=_get_instructions_from_options(dict(merged_options)),
@@ -2163,7 +2165,7 @@ class AgentTelemetryLayer:
and response.messages
and span.is_recording()
):
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=response.messages,
output=True,
@@ -2789,7 +2791,7 @@ def capture_exception(span: trace.Span, exception: Exception, timestamp: int | N
span.set_status(status=trace.StatusCode.ERROR, description=repr(exception))
def _capture_system_instructions_experimental(span: trace.Span, system_instructions: str | list[str] | None) -> None:
def _capture_system_instructions_latest_experimental(span: trace.Span, system_instructions: str | list[str] | None) -> None:
"""Capture system instructions on a span."""
if not OBSERVABILITY_SETTINGS.use_latest_experimental_gen_ai_semconv or not system_instructions:
return
@@ -2802,7 +2804,7 @@ def _capture_system_instructions_experimental(span: trace.Span, system_instructi
)
def _capture_current_agent_system_instructions_experimental(
def _capture_current_agent_system_instructions_latest_experimental(
agent_span: trace.Span,
chat_span: trace.Span,
system_instructions: str | list[str] | None,
@@ -2834,7 +2836,7 @@ def _capture_current_agent_system_instructions_experimental(
):
return
_capture_system_instructions_experimental(agent_span, system_instructions)
_capture_system_instructions_latest_experimental(agent_span, system_instructions)
def _normalize_instructions(system_instructions: str | list[str]) -> list[str]:
@@ -2881,7 +2883,7 @@ def _capture_message_events_v1_36(
output: bool = False,
finish_reason: FinishReason | None = None,
) -> None:
"""Emit stable v1.36.0 GenAI events for a chat request or response."""
"""Emit baseline v1.36.0 GenAI events for a model invocation."""
if not OBSERVABILITY_SETTINGS.enable_message_events:
return
@@ -2931,7 +2933,7 @@ def _emit_otel_event_v1_36(
)
def _capture_message_span_attributes_experimental(
def _capture_message_span_attributes_latest_experimental(
span: trace.Span,
messages: AgentRunInputs,
*,
@@ -2939,24 +2941,24 @@ def _capture_message_span_attributes_experimental(
output: bool = False,
finish_reason: FinishReason | None = None,
) -> None:
"""Capture the latest-experimental GenAI message span attributes."""
"""Capture the latest (above-baseline) GenAI message span attributes."""
if not OBSERVABILITY_SETTINGS.use_latest_experimental_gen_ai_semconv:
return
from ._types import normalize_messages
otel_messages = [_to_otel_message_experimental(message) for message in normalize_messages(messages)]
otel_messages = [_to_otel_message_latest_experimental(message) for message in normalize_messages(messages)]
if finish_reason and otel_messages:
otel_messages[-1]["finish_reason"] = FINISH_REASON_MAP.get(finish_reason, finish_reason)
span.set_attribute(
OtelAttr.OUTPUT_MESSAGES if output else OtelAttr.INPUT_MESSAGES,
json.dumps(otel_messages, ensure_ascii=False),
)
_capture_system_instructions_experimental(span, system_instructions)
_capture_system_instructions_latest_experimental(span, system_instructions)
def _to_otel_input_events_v1_36(message: Message) -> list[tuple[OtelAttr, dict[str, Any]]]:
"""Create stable v1.36.0 event names and bodies for an input message."""
"""Create baseline v1.36.0 event names and bodies for an input message."""
event_name = ROLE_EVENT_MAP.get(message.role)
if event_name is None:
return []
@@ -2988,7 +2990,7 @@ def _to_otel_input_events_v1_36(message: Message) -> list[tuple[OtelAttr, dict[s
def _to_otel_choice_v1_36(message: Message, index: int, finish_reason: str) -> dict[str, Any]:
"""Create a stable v1.36.0 choice event body."""
"""Create a baseline v1.36.0 choice event body."""
choice_message: dict[str, Any] = {}
if message.text:
choice_message["content"] = message.text
@@ -3005,7 +3007,7 @@ def _to_otel_choice_v1_36(message: Message, index: int, finish_reason: str) -> d
def _to_otel_tool_calls_v1_36(message: Message) -> list[dict[str, Any]]:
"""Create stable v1.36.0 function-call structures for a message."""
"""Create baseline v1.36.0 function-call structures for a message."""
return [
{
"id": content.call_id,
@@ -3020,15 +3022,15 @@ def _to_otel_tool_calls_v1_36(message: Message) -> list[dict[str, Any]]:
]
def _to_otel_message_experimental(message: Message) -> dict[str, Any]:
def _to_otel_message_latest_experimental(message: Message) -> dict[str, Any]:
"""Create a otel representation of a message."""
return {
"role": message.role,
"parts": [_to_otel_part_experimental(content) for content in message.contents],
"parts": [_to_otel_part_latest_experimental(content) for content in message.contents],
}
def _to_otel_part_experimental(content: Content) -> dict[str, Any] | None:
def _to_otel_part_latest_experimental(content: Content) -> dict[str, Any] | None:
"""Create a otel representation of a Content."""
from ._types import _get_data_bytes_as_str # pyright: ignore[reportPrivateUsage]
python/packages/core/tests/core/test_observability.py
+63 -63
View File
@@ -36,7 +36,7 @@ from agent_framework.observability import (
MessageListTimestampFilter,
OtelAttr,
_capture_message_events_v1_36,
_capture_message_span_attributes_experimental,
_capture_message_span_attributes_latest_experimental,
_get_instructions_from_options,
_to_otel_choice_v1_36,
_to_otel_input_events_v1_36,
@@ -369,10 +369,10 @@ async def test_chat_client_observability_with_instructions(
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
async def test_chat_client_stable_semconv_omits_system_instructions(
async def test_chat_client_baseline_semconv_omits_system_instructions(
mock_chat_client, span_exporter: InMemorySpanExporter, enable_sensitive_data
):
"""Test that stable v1.36.0 telemetry omits the post-v1.36.0 system instructions attribute."""
"""Test that baseline v1.36.0 telemetry omits the post-v1.36.0 system instructions attribute."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
@@ -509,7 +509,7 @@ async def test_chat_client_streaming_input_events_correlated_to_chat_span(
) -> None:
"""Regression guard: streaming input events must carry the chat span's trace/span id.
``_capture_message_events_v1_36`` emits the stable v1.36.0 GenAI message events via the
``_capture_message_events_v1_36`` emits the baseline v1.36.0 GenAI message events via the
native OTel event logger, which derives trace/span correlation from whatever span is
current in the ambient context at emit time. In the streaming path the chat span is
started with ``_start_streaming_span`` (not attached as current), so those events must be
@@ -1950,7 +1950,7 @@ def test_enable_instrumentation_reads_env_sensitive_data(monkeypatch):
def test_semconv_defaults_to_latest_experimental_when_unset(monkeypatch):
"""OTEL_SEMCONV_STABILITY_OPT_IN unset → MAF defaults to the latest experimental conventions."""
"""OTEL_SEMCONV_STABILITY_OPT_IN unset → MAF defaults to the latest conventions."""
from agent_framework.observability import ObservabilitySettings
monkeypatch.delenv("OTEL_SEMCONV_STABILITY_OPT_IN", raising=False)
@@ -1960,8 +1960,8 @@ def test_semconv_defaults_to_latest_experimental_when_unset(monkeypatch):
assert settings.use_latest_experimental_gen_ai_semconv is True
def test_semconv_explicit_empty_opts_into_stable(monkeypatch):
"""Explicitly setting OTEL_SEMCONV_STABILITY_OPT_IN='' opts into the stable v1.36.0 conventions."""
def test_semconv_explicit_empty_opts_into_baseline(monkeypatch):
"""Explicitly setting OTEL_SEMCONV_STABILITY_OPT_IN='' opts into the baseline v1.36.0 conventions."""
from agent_framework.observability import ObservabilitySettings
monkeypatch.setenv("OTEL_SEMCONV_STABILITY_OPT_IN", "")
@@ -1971,7 +1971,7 @@ def test_semconv_explicit_empty_opts_into_stable(monkeypatch):
def test_semconv_explicit_token_opts_into_latest_experimental(monkeypatch):
"""Explicitly including 'gen_ai_latest_experimental' opts into the latest experimental conventions."""
"""Explicitly including 'gen_ai_latest_experimental' opts into the latest conventions."""
from agent_framework.observability import ObservabilitySettings
monkeypatch.setenv("OTEL_SEMCONV_STABILITY_OPT_IN", "gen_ai_latest_experimental")
@@ -1993,14 +1993,14 @@ def test_semconv_multi_value_list_checks_for_gen_ai_token(monkeypatch):
assert settings.use_latest_experimental_gen_ai_semconv is False
def test_stable_semconv_skips_current_agent_system_instruction_checks(span_exporter: InMemorySpanExporter):
"""Stable v1.36.0 returns before inspecting spans for the unsupported system instructions attribute."""
def test_baseline_semconv_skips_current_agent_system_instruction_checks(span_exporter: InMemorySpanExporter):
"""Baseline v1.36.0 returns before inspecting spans for the unsupported system instructions attribute."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
agent_span = Mock()
observability._capture_current_agent_system_instructions_experimental( # pyright: ignore[reportPrivateUsage]
observability._capture_current_agent_system_instructions_latest_experimental( # pyright: ignore[reportPrivateUsage]
agent_span,
Mock(),
"You are a helpful assistant.",
@@ -2020,7 +2020,7 @@ def test_enable_message_events_defaults_true(monkeypatch):
def test_enable_message_events_can_be_disabled(monkeypatch):
"""ENABLE_MESSAGE_EVENTS=false disables the stable v1.36.0 message events."""
"""ENABLE_MESSAGE_EVENTS=false disables the baseline v1.36.0 message events."""
from agent_framework.observability import ObservabilitySettings
monkeypatch.setenv("ENABLE_MESSAGE_EVENTS", "false")
@@ -2041,8 +2041,8 @@ def test_get_span_attributes_uses_provider_name_under_latest_semconv(span_export
@pytest.mark.parametrize("enable_sensitive_data", [False], indirect=True)
def test_get_span_attributes_uses_system_under_stable_semconv(span_exporter: InMemorySpanExporter):
"""Under the stable v1.36.0 semconv, the provider attribute reverts to gen_ai.system."""
def test_get_span_attributes_uses_system_under_baseline_semconv(span_exporter: InMemorySpanExporter):
"""Under the baseline v1.36.0 semconv, the provider attribute reverts to gen_ai.system."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
@@ -2053,10 +2053,10 @@ def test_get_span_attributes_uses_system_under_stable_semconv(span_exporter: InM
@pytest.mark.parametrize("enable_sensitive_data", [False], indirect=True)
def test_get_span_attributes_omits_post_v1_36_attributes_under_stable_semconv(
def test_get_span_attributes_omits_post_v1_36_attributes_under_baseline_semconv(
span_exporter: InMemorySpanExporter,
):
"""Stable v1.36.0 omits attributes introduced by later GenAI conventions."""
"""Baseline v1.36.0 omits attributes introduced by later GenAI conventions."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
@@ -2072,10 +2072,10 @@ def test_get_span_attributes_omits_post_v1_36_attributes_under_stable_semconv(
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
async def test_chat_client_observability_provider_name_under_stable_semconv(
async def test_chat_client_observability_provider_name_under_baseline_semconv(
mock_chat_client, span_exporter: InMemorySpanExporter
):
"""Chat spans report gen_ai.system (not gen_ai.provider.name) under the stable v1.36.0 semconv."""
"""Chat spans report gen_ai.system (not gen_ai.provider.name) under the baseline v1.36.0 semconv."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
@@ -2248,10 +2248,10 @@ def test_disable_instrumentation_in_all(monkeypatch):
def test_to_otel_part_text():
"""Test _to_otel_part with text content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="text", text="Hello world")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result == {"type": "text", "content": "Hello world"}
@@ -2259,10 +2259,10 @@ def test_to_otel_part_text():
def test_to_otel_part_text_reasoning():
"""Test _to_otel_part with text_reasoning content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="text_reasoning", text="Thinking about this...")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result == {"type": "reasoning", "content": "Thinking about this..."}
@@ -2270,10 +2270,10 @@ def test_to_otel_part_text_reasoning():
def test_to_otel_part_uri():
"""Test _to_otel_part with uri content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="uri", uri="https://example.com/image.png", media_type="image/png")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result == {
"type": "uri",
@@ -2286,10 +2286,10 @@ def test_to_otel_part_uri():
def test_to_otel_part_uri_no_media_type():
"""Test _to_otel_part with uri content without media_type."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="uri", uri="https://example.com/file")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result == {
"type": "uri",
@@ -2302,11 +2302,11 @@ def test_to_otel_part_uri_no_media_type():
def test_to_otel_part_data():
"""Test _to_otel_part with data content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
data = b"binary data"
content = Content.from_data(data=data, media_type="application/octet-stream")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result["type"] == "blob" # type: ignore[index] # pyrefly: ignore[unsupported-operation] # ty: ignore[not-subscriptable]
assert result["mime_type"] == "application/octet-stream" # type: ignore[index] # pyrefly: ignore[unsupported-operation] # ty: ignore[not-subscriptable]
@@ -2316,10 +2316,10 @@ def test_to_otel_part_data():
def test_to_otel_part_function_call():
"""Test _to_otel_part with function_call content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="function_call", call_id="call_123", name="test_function", arguments='{"arg1": "value1"}')
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result == {
"type": "tool_call",
@@ -2332,11 +2332,11 @@ def test_to_otel_part_function_call():
def test_to_otel_part_function_call_reuses_prepared_arguments():
"""Test _to_otel_part does not re-serialize function-call arguments in the observability hot path."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
arguments = {"payload": object()}
content = Content(type="function_call", call_id="call_789", name="handoff", arguments=arguments)
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result is not None
assert result["arguments"] is arguments
@@ -2391,27 +2391,27 @@ def test_make_json_safe_dict_with_non_string_keys():
def test_to_otel_part_function_result():
"""Test _to_otel_part with function_result content."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
content = Content(type="function_result", call_id="call_123", result="Success")
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
assert result["type"] == "tool_call_response" # type: ignore[index] # pyrefly: ignore[unsupported-operation] # ty: ignore[not-subscriptable]
assert result["id"] == "call_123" # type: ignore[index] # pyrefly: ignore[unsupported-operation] # ty: ignore[not-subscriptable]
# region Test stable v1.36.0 event conversion
# region Test baseline v1.36.0 event conversion
def test_to_otel_v1_36_user_message_body():
"""Stable user events use content rather than the experimental parts shape."""
"""Baseline user events use content rather than the experimental parts shape."""
events = _to_otel_input_events_v1_36(Message(role="user", contents=["Hello", "world"]))
assert events == [(OtelAttr.USER_MESSAGE, {"content": "Hello world"})]
def test_to_otel_v1_36_assistant_tool_call_body():
"""Stable assistant events use the v1.36 function-call nesting."""
"""Baseline assistant events use the v1.36 function-call nesting."""
message = Message(
role="assistant",
contents=[Content.from_function_call(call_id="call_123", name="get_weather", arguments='{"city":"Paris"}')],
@@ -2436,7 +2436,7 @@ def test_to_otel_v1_36_assistant_tool_call_body():
def test_to_otel_v1_36_tool_results_emit_one_event_per_call():
"""Each stable tool event has the one required tool-call id."""
"""Each baseline tool event has the one required tool-call id."""
message = Message(
role="tool",
contents=[
@@ -2454,14 +2454,14 @@ def test_to_otel_v1_36_tool_results_emit_one_event_per_call():
def test_to_otel_v1_36_tool_message_without_call_id_is_skipped():
"""A stable tool event is not emitted when its required call id is unavailable."""
"""A baseline tool event is not emitted when its required call id is unavailable."""
events = _to_otel_input_events_v1_36(Message(role="tool", contents=["Uncorrelated result"]))
assert events == []
def test_to_otel_v1_36_choice_body():
"""Stable choices contain index, finish reason, and a nested message."""
"""Baseline choices contain index, finish reason, and a nested message."""
body = _to_otel_choice_v1_36(
Message(role="assistant", contents=["Done"]),
index=1,
@@ -2825,8 +2825,8 @@ def test_get_response_attributes_with_additional_usage():
assert result[OtelAttr.REASONING_OUTPUT_TOKENS] == 30
def test_get_response_attributes_omits_post_v1_36_usage_under_stable_semconv(monkeypatch: pytest.MonkeyPatch):
"""Stable v1.36.0 keeps total usage while omitting newer token breakdowns."""
def test_get_response_attributes_omits_post_v1_36_usage_under_baseline_semconv(monkeypatch: pytest.MonkeyPatch):
"""Baseline v1.36.0 keeps total usage while omitting newer token breakdowns."""
from unittest.mock import Mock
import agent_framework.observability as observability
@@ -3011,11 +3011,11 @@ def test_observability_settings_configure_already_setup(monkeypatch):
def test_to_otel_part_generic():
"""Test _to_otel_part with unknown content type uses to_dict fallback."""
from agent_framework import Content
from agent_framework.observability import _to_otel_part_experimental
from agent_framework.observability import _to_otel_part_latest_experimental
# Create a content with type that falls to default case
content = Content(type="annotations", text="some text") # type: ignore[arg-type] # pyrefly: ignore[bad-argument-type] # ty: ignore[invalid-argument-type]
result = _to_otel_part_experimental(content)
result = _to_otel_part_latest_experimental(content)
# Should return result from to_dict
assert result is not None
@@ -3412,8 +3412,8 @@ async def test_capture_messages_with_finish_reason(mock_chat_client, span_export
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
def test_capture_messages_stable_semconv_emits_events_only(span_exporter: InMemorySpanExporter):
"""Stable v1.36.0 conventions (opt-in list without the experimental token): events only, no span attribute."""
def test_capture_messages_baseline_semconv_emits_events_only(span_exporter: InMemorySpanExporter):
"""Baseline v1.36.0 conventions (opt-in list without the experimental token): events only, no span attribute."""
from opentelemetry import trace
import agent_framework.observability as observability
@@ -3461,7 +3461,7 @@ def test_capture_messages_latest_experimental_emits_span_attribute_only_when_eve
patch("agent_framework.observability.otel_event_logger.emit") as mock_emit,
tracer.start_as_current_span("test_span") as span,
):
observability._capture_message_span_attributes_experimental( # type: ignore[reportPrivateUsage]
observability._capture_message_span_attributes_latest_experimental( # type: ignore[reportPrivateUsage]
span=span,
messages=[Message(role="user", contents=["Test"])],
)
@@ -3494,7 +3494,7 @@ def test_capture_messages_defaults_emit_both_events_and_span_attribute(span_expo
provider_name="test_provider",
messages=[Message(role="user", contents=["Test"])],
)
observability._capture_message_span_attributes_experimental( # type: ignore[reportPrivateUsage]
observability._capture_message_span_attributes_latest_experimental( # type: ignore[reportPrivateUsage]
span=span,
messages=[Message(role="user", contents=["Test"])],
)
@@ -3510,8 +3510,8 @@ def test_capture_messages_defaults_emit_both_events_and_span_attribute(span_expo
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
def test_capture_messages_stable_semconv_emits_one_event_per_choice(span_exporter: InMemorySpanExporter):
"""Stable v1.36 emits each model choice with its required index and finish reason."""
def test_capture_messages_baseline_semconv_emits_one_event_per_choice(span_exporter: InMemorySpanExporter):
"""Baseline v1.36 emits each model choice with its required index and finish reason."""
from opentelemetry import trace
import agent_framework.observability as observability
@@ -3542,7 +3542,7 @@ def test_capture_messages_stable_semconv_emits_one_event_per_choice(span_exporte
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
def test_capture_messages_preserves_custom_finish_reason(span_exporter: InMemorySpanExporter):
"""Custom finish reasons remain available to stable events and experimental attributes."""
"""Custom finish reasons remain available to baseline events and experimental attributes."""
import json
from opentelemetry import trace
@@ -3560,7 +3560,7 @@ def test_capture_messages_preserves_custom_finish_reason(span_exporter: InMemory
output=True,
finish_reason=cast(Any, "guardrail"),
)
observability._capture_message_span_attributes_experimental( # type: ignore[reportPrivateUsage]
observability._capture_message_span_attributes_latest_experimental( # type: ignore[reportPrivateUsage]
span=span,
messages=[Message(role="assistant", contents=["Done"])],
output=True,
@@ -4636,7 +4636,7 @@ async def test_system_instructions_preserves_non_ascii_characters(span_exporter:
span_exporter.clear()
with tracer.start_as_current_span("test_span") as span:
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=[Message(role="user", contents=["Test"])],
system_instructions=chinese_text,
@@ -4691,7 +4691,7 @@ def test_capture_messages_with_prepared_request_info_function_call_arguments(spa
span_exporter.clear()
tracer = trace.get_tracer("test")
with tracer.start_as_current_span("test_span") as span:
_capture_message_span_attributes_experimental(span=span, messages=[msg])
_capture_message_span_attributes_latest_experimental(span=span, messages=[msg])
spans = span_exporter.get_finished_spans()
span = spans[0] # type: ignore[assignment]
@@ -4704,7 +4704,7 @@ def test_capture_messages_with_prepared_request_info_function_call_arguments(spa
def test_capture_messages_emits_framework_instructions_separately_from_history(
span_exporter: InMemorySpanExporter,
):
"""Test separate framework instructions use their own stable event and experimental span attribute."""
"""Test separate framework instructions use their own baseline event and experimental span attribute."""
import json
from opentelemetry import trace
@@ -4721,7 +4721,7 @@ def test_capture_messages_emits_framework_instructions_separately_from_history(
messages=[Message(role="user", contents=["Test"])],
system_instructions="Framework system instruction",
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=[Message(role="user", contents=["Test"])],
system_instructions="Framework system instruction",
@@ -4745,7 +4745,7 @@ def test_capture_messages_emits_framework_instructions_separately_from_history(
def test_capture_messages_preserves_framework_instructions_and_system_history(
span_exporter: InMemorySpanExporter,
):
"""Test stable events preserve separate instructions and original system history."""
"""Test baseline events preserve separate instructions and original system history."""
import json
from opentelemetry import trace
@@ -4765,7 +4765,7 @@ def test_capture_messages_preserves_framework_instructions_and_system_history(
],
system_instructions="Framework system instruction",
)
_capture_message_span_attributes_experimental(
_capture_message_span_attributes_latest_experimental(
span=span,
messages=[
Message(role="system", contents=["Original system message"]),
@@ -6658,7 +6658,7 @@ async def test_chat_capture_messages_skipped_when_span_not_recording(
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
patch("agent_framework.observability._capture_message_events_v1_36") as mock_capture_v1_36,
patch(
"agent_framework.observability._capture_message_span_attributes_experimental"
"agent_framework.observability._capture_message_span_attributes_latest_experimental"
) as mock_capture_experimental,
patch("agent_framework.observability._capture_response") as mock_capture_response,
):
@@ -6687,7 +6687,7 @@ async def test_chat_streaming_capture_messages_skipped_when_span_not_recording(
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
patch("agent_framework.observability._capture_message_events_v1_36") as mock_capture_v1_36,
patch(
"agent_framework.observability._capture_message_span_attributes_experimental"
"agent_framework.observability._capture_message_span_attributes_latest_experimental"
) as mock_capture_experimental,
patch("agent_framework.observability._capture_response") as mock_capture_response,
):
@@ -6715,7 +6715,7 @@ async def test_agent_capture_messages_skipped_when_span_not_recording(
with (
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
patch("agent_framework.observability._capture_message_span_attributes_experimental") as mock_capture,
patch("agent_framework.observability._capture_message_span_attributes_latest_experimental") as mock_capture,
patch("agent_framework.observability._capture_response") as mock_capture_response,
):
response = await agent.run("Test message")
@@ -6737,7 +6737,7 @@ async def test_agent_streaming_capture_messages_skipped_when_span_not_recording(
with (
patch("agent_framework.observability.get_tracer", return_value=NoOpTracer()),
patch("agent_framework.observability._capture_message_span_attributes_experimental") as mock_capture,
patch("agent_framework.observability._capture_message_span_attributes_latest_experimental") as mock_capture,
patch("agent_framework.observability._capture_response") as mock_capture_response,
):
updates: list[Any] = []
@@ -6763,7 +6763,7 @@ async def test_chat_capture_messages_called_when_span_recording(
with (
patch("agent_framework.observability._capture_message_events_v1_36") as mock_capture_v1_36,
patch(
"agent_framework.observability._capture_message_span_attributes_experimental"
"agent_framework.observability._capture_message_span_attributes_latest_experimental"
) as mock_capture_experimental,
patch("agent_framework.observability._capture_response") as mock_capture_response,
):
python/packages/core/tests/core/test_tools.py
+2 -2
View File
@@ -653,8 +653,8 @@ async def test_tool_invoke_telemetry_sensitive_disabled(span_exporter: InMemoryS
@pytest.mark.parametrize("enable_sensitive_data", [True], indirect=True)
async def test_tool_invoke_telemetry_omits_tool_call_attrs_under_stable_semconv(span_exporter: InMemorySpanExporter):
"""gen_ai.tool.call.arguments/result were introduced above v1.36.0; omit them under the stable semconv."""
async def test_tool_invoke_telemetry_omits_tool_call_attrs_under_baseline_semconv(span_exporter: InMemorySpanExporter):
"""gen_ai.tool.call.arguments/result were introduced above v1.36.0; omit them under the baseline semconv."""
import agent_framework.observability as observability
observability.OBSERVABILITY_SETTINGS.otel_semconv_stability_opt_in = ""
python/samples/02-agents/observability/README.md
+9 -8
View File
@@ -199,8 +199,8 @@ Agent Framework reads the following environment variables:
| `ENABLE_INSTRUMENTATION` | `true` | Set to `false` to disable native instrumentation. See [Disabling instrumentation](#disabling-instrumentation) for the programmatic alternative with sticky semantics. |
| `ENABLE_SENSITIVE_DATA` | `false` | Set to `true` to emit sensitive data (prompts, responses, etc.). |
| `ENABLE_CONSOLE_EXPORTERS` | `false` | Set to `true` to add console exporters. Only used by `configure_otel_providers()`. |
| `ENABLE_MESSAGE_EVENTS` | `true` | Set to `false` to stop emitting the stable v1.36.0 GenAI message events (`gen_ai.system.message`, etc.). **Has no effect unless `ENABLE_SENSITIVE_DATA=true`.** See [GenAI semantic-conventions versioning](#genai-semantic-conventions-versioning). |
| `OTEL_SEMCONV_STABILITY_OPT_IN` | unset (conventions above v1.36.0) | A comma-separated list of category-specific values, following the standard OpenTelemetry comma-separated opt-in list format, currently only containing a single token ``"gen_ai_latest_experimental"``. v1.36.0 is the OTel-recommended stable release; every version above it is collectively "experimental". The default, unlike upstream OpenTelemetry which defaults to stable-only, ``"gen_ai_latest_experimental"`` selects the conventions above v1.36.0; a list that omits that token (e.g. ``""``) selects the v1.36.0 conventions instead. See [GenAI semantic-conventions versioning](#genai-semantic-conventions-versioning). |
| `ENABLE_MESSAGE_EVENTS` | `true` | Set to `false` to stop emitting the baseline v1.36.0 GenAI message events (`gen_ai.system.message`, etc.) for model invocation. **Has no effect unless `ENABLE_SENSITIVE_DATA=true`.** See [GenAI semantic-conventions versioning](#genai-semantic-conventions-versioning). |
| `OTEL_SEMCONV_STABILITY_OPT_IN` | unset (conventions above v1.36.0) | A comma-separated list of category-specific values, following the standard OpenTelemetry comma-separated opt-in list format, currently only containing a single token ``"gen_ai_latest_experimental"``. v1.36.0 is the OTel-recommended baseline; every version above it is referred to here as "latest" (even the baseline is an expeirmental release). The default, unlike upstream OpenTelemetry which retains the baseline conventions, ``"gen_ai_latest_experimental"`` selects the latest conventions above v1.36.0; a list that omits that token (e.g. ``""``) selects the v1.36.0 conventions instead. See [GenAI semantic-conventions versioning](#genai-semantic-conventions-versioning). |
| `VS_CODE_EXTENSION_PORT` | unset | Port used by the [AI Toolkit for VS Code](https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio#tracing) tracing integration. Only used by `configure_otel_providers()`. |
You can also call `enable_sensitive_telemetry()` from `agent_framework.observability` to opt in to sensitive-data capture programmatically.
@@ -209,9 +209,9 @@ You can also call `enable_sensitive_telemetry()` from `agent_framework.observabi
### GenAI semantic-conventions versioning
[v1.36.0](https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/gen-ai) is the OpenTelemetry-recommended **stable** release of the [GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/). Every release above it (v1.37.0 and later) is collectively **experimental** and, per OTel's own [stability warning](https://github.com/open-telemetry/semantic-conventions/blob/v1.37.0/docs/gen-ai), keeps changing in more than one way. `OTEL_SEMCONV_STABILITY_OPT_IN` is the OTel-standard switch between these two rule sets, and Agent Framework applies it consistently across every attribute/representation it knows differs between the two:
[v1.36.0](https://github.com/open-telemetry/semantic-conventions/blob/v1.36.0/docs/gen-ai) is the OpenTelemetry-recommended **baseline** for existing GenAI instrumentations. Releases above it (v1.37.0 and later) are referred to as **latest** and, per OTel's own [stability warning](https://github.com/open-telemetry/semantic-conventions/blob/v1.37.0/docs/gen-ai), keep changing in more than one way. `OTEL_SEMCONV_STABILITY_OPT_IN` is the OTel-standard switch between these two rule sets, and Agent Framework applies it consistently across every attribute/representation it knows differs between the two:
| Aspect | v1.36.0 (stable) | Above v1.36.0 (experimental, the default) |
| Aspect | v1.36.0 (baseline) | Above v1.36.0 (latest, the default) |
|--------|------------------|--------------------------------------------|
| Input/output message representation | Log-record **events** (`gen_ai.system.message`, `gen_ai.user.message`, `gen_ai.assistant.message`, `gen_ai.tool.message`, `gen_ai.choice`) | `gen_ai.input.messages`/`gen_ai.output.messages` **span attributes** |
| Provider-identifying attribute | `gen_ai.system` | `gen_ai.provider.name` |
@@ -221,17 +221,18 @@ You can also call `enable_sensitive_telemetry()` from `agent_framework.observabi
> **`ENABLE_SENSITIVE_DATA=true` is a prerequisite for the message-representation and tool-call-attribute rows above.** Chat content (prompts, responses, tool arguments/results) is only ever captured when sensitive-data capture is enabled (see [`ENABLE_SENSITIVE_DATA`](#environment-variables) above); the provider-attribute rename applies regardless, since `gen_ai.system`/`gen_ai.provider.name` is not sensitive data. If `ENABLE_SENSITIVE_DATA` is `false` (the default), `ENABLE_MESSAGE_EVENTS` has nothing to switch and is effectively ignored, and no `gen_ai.tool.call.*` attributes are emitted under either semconv version.
Agent Framework defaults to the conventions above v1.36.0 (unlike upstream OpenTelemetry, which defaults to stable-only) because most users already depend on them, and — to avoid a breaking change for anyone consuming the older message events — also keeps emitting those events by default via `ENABLE_MESSAGE_EVENTS`. `ENABLE_MESSAGE_EVENTS` is controlled independently of `OTEL_SEMCONV_STABILITY_OPT_IN`:
Agent Framework defaults to the conventions above v1.36.0 (unlike upstream OpenTelemetry, which retains the baseline conventions) because most users already depend on them, and — to avoid a breaking change for anyone consuming the older message events for modelinvocation — also keeps emitting those events by default via `ENABLE_MESSAGE_EVENTS`. `ENABLE_MESSAGE_EVENTS` is controlled independently of `OTEL_SEMCONV_STABILITY_OPT_IN`:
```bash
# Capture agent/chat client/tool input and output contents (default: false):
export ENABLE_SENSITIVE_DATA=true
# Opt into the stable v1.36.0 conventions only (default: "gen_ai_latest_experimental"):
# Opt into the baseline v1.36.0 conventions only (default: "gen_ai_latest_experimental"):
export OTEL_SEMCONV_STABILITY_OPT_IN=""
# Agent Framework still emits the stable v1.36.0 message events even when the semconv opt-in
# is set to experimental for compatibility reasons. To stop emitting those events (default: true):
# Agent Framework still emits the baseline v1.36.0 message events for model invocations even
# when the semconv opt-in is set to latest for compatibility reasons. To stop emitting those
# events (default: true):
export ENABLE_MESSAGE_EVENTS=false
```