* docs: ADR-0027 feature-usage bitmask in the User-Agent Add an ADR, design spec, and per-language bit registry for a lightweight feature-usage signal: a 64-bit mask, emitted as a `(feat=vN.<hex>)` User-Agent comment, stamped per request on first-party (Azure/Foundry) clients only. - docs/decisions/0027-feature-usage-bitmask-user-agent.md — ADR (options-first, with Limitations, Open Questions, and v1->v2 migration) - docs/specs/002-feature-usage-telemetry.md — design spec + implementation plan - docs/specs/feature-usage-bit-registry.md — per-language bit tables + governance Granularity is per package with core broken out per feature (each orchestration pattern and built-in context/history provider). Registries are per language (decoder selects by the language already in the UA). OpenTelemetry emission is deferred (privacy). Docs only; no code changes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: fix dead links to removed registry JSON in ADR-0027 The registry JSON was consolidated into feature-usage-bit-registry.md; point the ADR's two remaining links at the markdown instead of the deleted file (fixes markdown-link-check 404s). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: address review — drop JSON-parity wording, clarify per-language decode - ADR option J: the parity test compares the enum against the per-language table in the registry doc, not a (now-removed) JSON file. - Spec .NET mapping: the wire format is shared, but the mask is decoded per-language (select the table via the UA product token) — fixes the "decoded numbers mean the same thing in both SDKs" wording that conflicted with the per-language, non-synchronized bit indexes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: add dedicated mask-only opt-out env var (AGENT_FRAMEWORK_FEATURE_MASK_DISABLED) Re-introduce a dedicated opt-out that disables only the feature mask while keeping the base agent-framework-<lang>/{version} User-Agent, alongside the existing AGENT_FRAMEWORK_USER_AGENT_DISABLED (whole UA). Updates the spec accumulator gate, API surface, opt-out table and examples; the registry opt-out section; and the ADR (decision outcome, consequences, open questions -> decided). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: add prior-art comparison (AWS botocore m/, Stainless, Azure, etc.) Add a Prior art section to ADR-0027 surveying how comparable SDKs encode identity/usage in the User-Agent or sidecar headers, with citations: - AWS botocore `m/` feature-code list — the direct analog (per-request, usage-based feature flags in the UA); contrasts short-code set vs our hex bitmask. - OpenAI/Anthropic Stainless `X-Stainless-*` headers (static identity). - Azure azure-core UserAgentPolicy + AZURE_TELEMETRY_DISABLED. - Google x-goog-api-client; LangSmith version token + tracing opt-in. Also add an Open Question on honoring the cross-tool DO_NOT_TRACK convention. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: fold in botocore lessons; record accumulation-scope decision botocore's m/ feature list scopes features to a per-request contextvars set that resets between calls — clean per-call attribution, but it assumes every feature lives inside a service request. That holds for an SDK natively bound to its own services; it does not for us, where many features (agent/workflow/provider construction, session setup) are not bound to any request. - ADR: add Accumulation scope options — P (process-global monotonic, chosen) vs Q (botocore per-request set, rejected) with the request-binding rationale; reference P in the decision; reframe the "no per-call attribution" limitation as a deliberate scope choice. - ADR Prior art: bitmask gives bounded token size for free (vs botocore's 1024-byte cap + truncation); mechanism is private, wire format is the contract; fix a duplicated phrase. - Spec: note the mask is process-global, monotonic, never reset (intentional, lock/Interlocked.Or-safe), the token is safe-by-construction (no sanitization), and the helpers are private API. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: update feature mask ADR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: refresh feature usage telemetry design Rebase the proposal on current main, renumber it to ADR-0033/SPEC-004, and reconcile the registry and implementation notes with current Python and .NET surfaces. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: expand feature usage mask to 128 bits Repartition the v1 registries with additional skill categories, define the bit-allocation tenet, and document the two-lane .NET accumulator and 128-bit decoder contract. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: tighten feature telemetry activation and scoping Require approved pipeline and actual-origin classification, preserve OpenAI transport defaults, use activation-based marking, and move index ownership into packages with parity and no-overlap validation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: preserve SDK transport defaults for telemetry Record the transport-preservation requirement at the ADR decision level. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: split declarative agent and workflow usage Allocate separate adjacent v1 indexes for declarative agents and declarative workflows in Python and .NET, shifting later unreleased rows. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: accept feature usage telemetry ADR Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: record feature telemetry ADR participants Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: expand feature telemetry ADR consultation Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f * docs: clarify feature telemetry semantics Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 346bf168-b668-4c4a-a8db-a67282ee5e5f
38 KiB
status, contact, date, deciders, consulted, informed
| status | contact | date | deciders | consulted | informed |
|---|---|---|---|---|---|
| accepted | eavanvalkenburg | 2026-07-22 | eavanvalkenburg, chetantoshniwal | TaoChenOSU, moonbox3, peibekwe, rogerbarreto, westey-m |
Feature-usage bitmask in the User-Agent
Context and Problem Statement
We can see which Agent Framework packages are installed and that some framework
call happened (via the existing agent-framework-python/{version} User-Agent),
but we have no usage-based signal about which features are actually exercised
at runtime, nor which are used together (e.g. workflows + MCP + Foundry). How
can we collect a lightweight, privacy-respecting signal of feature usage for the
traffic we can actually read, without standing up new event pipelines?
The detailed mechanism is in SPEC-004; the per-language bit tables are in feature-usage-bit-registry.md.
Decision Drivers
- Transparency — openly documented, human-decodable, user-controllable. No hidden or obfuscated telemetry.
- First-party scope / no third-party leakage — emission requires both an explicitly approved client/pipeline family and an approved actual HTTPS origin on every request (including redirects). Credentials or an Azure setting alone never approve a custom gateway/origin.
- Live signal — read the process's observed-feature set so far at request send time, rather than freezing it at client construction.
- Low cost / few moving parts — reuse telemetry already in the request path; bounded fixed-width processing; as little machinery as the job needs.
- Privacy — encode only coarse "observed at least once" Boolean feature state, never counts; no identifiers, arguments, prompts, payloads, model/deployment names, endpoints, or customer-defined names.
- Use, not presence — package-level indexes mean a capability reached its first meaningful activation, not that a package was installed/imported or a DI container constructed an unused service.
- Versioning discipline — v1 is a point-in-time decision. Adding bits later is easier than removing or redefining them, so the initial table should lean toward fewer bits and avoid forcing v2 shortly after launch.
- Allocation discipline — each bit represents a stable framework-owned capability with a concrete product/support question and an actual-use mark point; implementation detail and speculative distinctions stay out.
Considered Options
The options below are grouped by the decisions that matter: the transport, the granularity, and the registry sharing model.
Transport
A. User-Agent token, first-party only, per request (chosen)
Stamp a (feat=...) comment onto the UA, but only on approved Azure/Foundry
client pipelines, and re-evaluate it per request.
- Good, reuses telemetry already sent to approved backends we can read.
- Good, request-time stamping reflects the live mask (not frozen at construction).
- Good, first-party scoping means no fingerprint leaks to third-party providers.
- Good, two-factor destination approval (pipeline + actual origin) denies custom
base_urlgateways and strips the token on unapproved redirect hops. - Good, maps onto .NET's existing per-request UA pipeline policies unchanged.
- Neutral, v1 stamps only pipelines the framework creates or can configure through supported public hooks. It does not mutate caller-owned clients or reach into private SDK pipelines.
- Bad, no signal for traffic that never hits a first-party endpoint (accepted — we couldn't read it anyway).
B. User-Agent token on all clients
- Good, simplest to wire (one static header).
- Bad, sends a deployment fingerprint to OpenAI/Anthropic/AWS/Google logs we cannot read — privacy leak for zero benefit.
- Bad, baked into static
default_headers, so it freezes at client construction and reports a near-empty mask.
C. OpenTelemetry span/resource attribute
- Good, precise per-call usage; no UA change.
- Bad (privacy — the main reason to hold it), a span attribute broadcasts the feature-combination fingerprint into the user's general telemetry pipeline, which is typically exported to third-party APM vendors (Datadog, Honeycomb, …). That re-introduces exactly the fingerprint leakage the first-party-only UA scoping (A) was chosen to avoid — just into a different set of third parties.
- Bad (secondary), also a cardinality footgun (a growing, combinatorial value must never become a metric dimension).
- Neutral, for the team's own goal it reaches us only if the user exports to Azure Monitor and we query it.
- Deferred, not rejected. The version prefix lets us add it later if the User-Agent path cannot answer a concrete query and there is an acceptable scoped/redacted variant.
D. Bespoke usage events
- Good, richest detail and flexibility.
- Bad, new data flow and cost; larger privacy surface; heavy to build and review; overkill for a coarse "which features" signal.
E. Install/import-time signal only (status quo-ish)
- Good, zero new runtime work.
- Bad, measures installation, not usage; cannot capture feature combinations — does not solve the problem.
Accumulation scope
S1. Process-global, monotonic mask (chosen)
A single mask per process; bits are OR-ed in as features are first used and never cleared. The token reflects "what this process has used so far."
- Binary interpretation: a set bit means the feature was observed at least once in this process before the request was sent. A bit repeated on later requests is the same Boolean observation, not another feature use. It cannot be summed into invocation, request, agent, user, or tenant counts.
- Good, fits our mixed feature lifecycle: many features are not bound to an outbound service request — an agent/workflow may first run or build, a context/history provider may first participate in a session, and a host may start serving before the request that later emits the token. A process-wide mask can carry those activations forward.
- Good, trivial and cheap: one OR under a lock (Python) / one atomic OR into one of two 64-bit lanes (.NET); no per-request state plumbing.
- Good, deliberately coarse for privacy: it avoids emitting a sequence of exact per-call feature combinations that could reconstruct a workload's behavioral trace.
- Neutral, coarser than per-call — early requests carry fewer bits than later ones, and the token says "this process used X", not "this call used X" or "X was used this many times."
For example, at time 1 Agent A can use MCP and a Foundry chat client. At time 2, Agent B in the same worker can make a normal Foundry chat call without MCP. The time-2 request still carries the MCP bit because MCP was previously observed in that process. It does not say Agent B used MCP, nor count a second MCP use.
S2. Per-request set, reset between calls (botocore's model — rejected)
AWS botocore scopes its m/ feature codes to a contextvars set that is reset
between requests, giving exact per-call attribution (and it deliberately no-ops
when called outside a request context to avoid features bleeding across requests).
See Prior art.
- Good, exact per-call attribution directly in the User-Agent.
- Bad, assumes every feature is exercised inside a single service request — true for botocore (an SDK natively bound to AWS service calls), but not for us. Our features split into request-scoped ones (a chat call, an MCP tool invocation) and decidedly non-request ones (workflow build/start, provider participation, hosting startup). The latter have no service request to attach to, so a per-request set would simply miss them.
- Bad, needs
contextvarspropagation through every async/threaded path and a reset discipline, plus enable/disable calls around every scoped operation; the bleed-guard botocore documents is the warning sign. - Bad, creates a more detailed per-call behavioral trace, increasing the privacy sensitivity and review burden compared with a coarse process-lifetime Boolean.
- Note, per-call attribution for the request-scoped subset is better served by the deferred OTel span path (option C) than by reshaping the UA token.
Granularity
The mechanism can support several granularities. The remaining decision before implementation is how detailed v1 should be. The estimates below are intentionally rough; v1 uses a fixed 128-bit bound to leave useful headroom without making the registry unbounded.
F0. Package-level bits
One bit per package, set on first use of a package-owned public API, client, provider, or tool. It is not set on install, import, or assembly load.
Examples that get bits:
agent-framework-corewhenAgent,AgentSession,Workflow, etc. is used.agent-framework-toolswhen aLocalShellToolorDockerShellToolfirst executes/probes its shell capability.agent-framework-foundrywhen aFoundryChatClient,FoundryAgent, etc. performs its first Foundry operation.agent-framework-openaiwhenOpenAIChatClient,OpenAIEmbeddingClient, etc. performs its first provider operation.agent-framework-azure-ai-searchwhenAzureAISearchContextProvideris used.agent-framework-azure-cosmoswhenCosmosHistoryProvideris used.agent-framework-rediswhenRedisContextProviderorRedisHistoryProvideris used.
Examples that do not get separate bits: merely installed dependencies;
imports or DI construction with no activation; Agent vs AgentSession vs
InMemoryHistoryProvider; FunctionTool vs MCPStdioTool vs LocalShellTool
vs DockerShellTool; FoundryChatClient vs FoundryAgent; OpenAIChatClient
vs OpenAIEmbeddingClient.
Rough estimate: Python ~25-35 bits; .NET ~15-25 bits.
- Good, lowest specificity and simplest registry.
- Good, clearly measures usage rather than dependency inventory if bits are set only at package-owned public API/client/provider/tool use sites.
- Bad, does not answer which major capability within a package is used.
F1. Package + major capability bits
Package bits plus selected major capabilities that are product-distinct and stable across implementations.
Examples that get bits:
agent-framework-coreplusAgent.AgentSessionplusInMemoryHistoryProvider/FileHistoryProvideras one history capability.Workflow/FunctionalWorkflowas one workflow capability.FunctionTool; MCP transports as one MCP capability; shell tools as one shell capability.- Skills provider plus stable source types: file, in-memory/programmatic, and MCP-backed skills (with .NET inline/class skill distinctions).
- Foundry chat/agent/embedding capabilities; OpenAI chat/embedding capabilities.
Examples that do not get separate bits: InMemoryHistoryProvider vs
FileHistoryProvider; WorkflowBuilder, AgentExecutor, FunctionExecutor, or
FanOutEdgeGroup; MCPStdioTool vs MCPStreamableHTTPTool vs
MCPWebsocketTool; LocalShellTool vs DockerShellTool vs
ShellEnvironmentProvider vs ShellPolicy; OpenAIChatClient vs
OpenAIChatCompletionClient; skill-source decorators such as caching, filtering,
deduplication, and aggregation.
Rough estimate: Python ~60-70 indexes; .NET ~45-55 indexes. The current candidate registry is at 63 Python / 52 .NET assigned indexes.
- Good, likely answers the first product adoption questions while staying compact.
- Good, fits comfortably within 128 bits while leaving room for additive package and feature growth.
- Neutral, some provider internals remain collapsed until a later additive bit is justified.
F2. Public construct / concrete type bits
One bit per public construct that users intentionally instantiate or configure.
Examples that get bits:
Agent,AgentSession,InMemoryHistoryProvider,FileHistoryProvider.Workflow,WorkflowBuilder,FunctionalWorkflow.FunctionTool,MCPStdioTool,MCPStreamableHTTPTool,MCPWebsocketTool.LocalShellTool,DockerShellTool,ShellEnvironmentProvider,ShellPolicy.FoundryChatClient,FoundryAgent,OpenAIChatClient,OpenAIChatCompletionClient,OpenAIEmbeddingClient.
Examples that do not get separate bits: Agent.run vs
Agent.run_streamed; workflow edge/executor internals such as AgentExecutor,
FunctionExecutor, or FanOutEdgeGroup; LocalShellTool persistent vs
stateless mode; ShellPolicy allowlist vs denylist configuration; FunctionTool
approval mode or result parser choices.
Rough estimate: Python ~70-100 bits; .NET ~55-80 bits.
- Good, concrete and directly tied to public API use.
- Neutral, fits within 128 bits at the current estimate, but consumes much of the deliberate growth reserve.
- Bad, adds many call sites and more fingerprint specificity for v1.
F3. Construct subtype / configuration bits
Split important constructs by mode, transport, storage, or workflow primitive when that distinction matters.
Examples that get bits:
InMemoryHistoryProviderandFileHistoryProviderseparately.FunctionalWorkflow,WorkflowBuilder,AgentExecutor,FunctionExecutor.FanOutEdgeGroup,FanInEdgeGroup,SwitchCaseEdgeGroup.LocalShellToolpersistent,LocalShellToolstateless,DockerShellTool.MCPStdioTool,MCPStreamableHTTPTool,MCPWebsocketTool;OpenAIChatClientvsOpenAIChatCompletionClient.
Examples that do not get separate bits: exact session id or persisted history file path; exact shell command, workdir, timeout, or output cap; exact MCP server command, URL, or tool names from the server; exact workflow graph shape or edge count; model/deployment names, prompts, tool arguments, payloads.
Rough estimate: Python ~110-150 bits; .NET ~85-125 bits.
- Good, useful where mode-level distinctions are decision-relevant.
- Bad, trades simplicity for precision, increases fingerprint specificity, and may exhaust or exceed 128 bits in Python.
F4. Option / behavior flag bits
The most detailed framework-owned option: bits for specific modes and behavior switches, still excluding customer/runtime values.
Examples that get bits:
- Agent streaming used vs non-streaming used.
FunctionToolapproval_mode="always_require"vs"never_require".FunctionToolSKIP_PARSING/ result-parser path used.- MCP sampling configured; MCP long-running task support used.
LocalShellToolclean_env/confine_workdir;DockerShellToolcontainer mode.
Examples that do not get separate bits: function names wrapped by
FunctionTool; approval rule arguments or approval decisions; MCP remote tool
names or schemas; shell command text or policy regex patterns; prompt/message
content, model names, URLs, tenant/user/session identifiers.
Rough estimate: Python 150+ bits; .NET 120+ bits.
- Good, maximum framework-owned detail.
- Bad, exceeds or nearly exhausts 128 bits and is too detailed for v1 without a concrete decision that requires it.
Registry sharing model
H. Per-language bit lists (chosen)
Each SDK owns an independent list; the decoder picks the list using the language already present in the UA product token.
- Good, no cross-language coordination: each SDK numbers and evolves its features independently; adding a Python feature never touches .NET numbering.
- Good, no null placeholders for one-SDK features, no "same bit, same meaning" rule, no SDK-aware decode caveats.
- Good, decoding is trivial: language (from UA) + version -> list -> AND.
- Neutral, two small lists to maintain instead of one (but they were going to diverge anyway — the packages differ).
I. Single shared cross-language registry
- Good, one list, one number space.
- Bad, forces synchronized numbering and null placeholders for features that exist in only one SDK, plus SDK-aware decode rules.
- Bad, the synchronization is pure accidental complexity — the language is already in the User-Agent, so sharing the number space buys nothing.
Registry maintenance
J. Package-local indexes + parity/no-overlap test (chosen)
- Good, each package owns private
FeatureIndexdeclarations only for its own rows; adding an optional-provider index does not require a core release after the marker API exists. - Good, one repository test compares the package-local declarations with the per-language table and rejects missing rows, wrong ids, out-of-range indexes, and any duplicate/overlapping index.
- Good, no build step, no generator to own.
K. Code-generate the enums from the registry
- Bad, a generator + drift test + schema test to maintain a short list of integer constants; likely justified only if v1 deliberately chooses the most detailed L3/L4 granularities.
Representation (how the mask is rendered as text)
All examples below encode the same mask — bits 0, 2, 32, 48, 56 set
(agent + workflow + sequential-orchestration + foundry.chat_client + openai, in
the Python v1 list) = decimal 72339073309605893.
L. Decimal — feat=v1.72339073309605893
- Good, human-familiar; trivial to parse.
- Neutral, no visual alignment to four-bit groups; slightly longer than hex for large masks. No advantage over hex.
M. Hex (chosen) — feat=v1.101000100000005
- Good, compact (≤32 chars for a 128-bit mask).
- Good, decodes with one stdlib call in every language (
int(x, 16)/ two 64-bit lane parses in .NET); each hex character corresponds to four consecutive bit positions. - Good, lowercase, no
0xprefix, no leading zeros — unambiguous and stable.
A grouped variant such as feat=v1.101.0001.0000.0005 was also considered.
Separators make the value longer and must be removed before int(x, 16) can
parse it, while the ordinary hex digits already preserve fixed four-bit groups.
N. Binary — feat=v1.100000001000000000000000100000000000000000000000000000101
- Good, directly shows every zero/one position.
- Bad, grows to 128 payload characters and is difficult to scan reliably.
O. Bit-list — feat=v1.0,2,32,48,56
- Good, most directly human-readable ("which bits").
- Bad, needs delimiter handling and grows with the number of set bits; a full 128-bit list is substantially larger than every fixed-width representation.
P. Alphabet / base-N (e.g. Crockford base32 feat=v1.208004000005, base62 feat=v1.5LJRx1i6xJ)
- Good, shortest representation.
- Bad, needs a custom alphabet + decode table on both ends; base62 is case-sensitive (fragile through case-normalizing intermediaries); not directly readable. Premature optimization for a value that is already ≤32 chars in hex.
All forms are ASCII. The table shows total bytes added to the existing
User-Agent, including the leading space and (feat=v1.) wrapper:
| Representation | Example (5 bits) | All current Python rows (63) | All current .NET rows (52) | Full 128-bit v1 |
|---|---|---|---|---|
| Hex | 26 | 34 | 30 | 43 |
| Grouped hex | 29 | 39 | 34 | 50 |
| Decimal | 28 | 38 | 34 | 50 |
| Binary | 68 | 100 | 86 | 139 |
| Bit-list | 23 | 189 | 156 | 412 |
| Crockford base32 | 23 | 29 | 26 | 37 |
| Base62 | 21 | 26 | 24 | 33 |
There is no defensible average before rollout, and the design does not depend on one: a process-global mask may eventually contain every assigned row. There is no smaller per-request bit budget because the bits are not request-scoped; the registry allocation tenet controls how many distinctions v1 assigns. Client processing is bounded by the fixed 128-bit width: marking performs one lock/atomic OR, and request-time stamping reads the mask, formats at most 32 hex characters, and replaces one User-Agent comment. It performs no registry scan, network call, or per-feature enable/disable bookkeeping.
Decision Outcome
Chosen: a request-time-stamped, first-party-only User-Agent (feat=...) token (A),
with a 128-bit process-global monotonic accumulator (S1), per-language bit lists
(H), package-local index enums kept honest by parity and no-overlap tests (J),
rendered as lowercase hex (M).
This is a bounded design with enough v1 headroom. A 128-bit
process-global, monotonic mask accumulates from universal
mark_feature_used() calls (so it spans build/start/participation activations
that aren't bound to any service request — the per-request set model (S2) can't);
the token is stamped per request only when both the client/pipeline and the
actual HTTPS origin are approved, so custom origins and cross-origin redirects
cannot inherit the fingerprint; each
SDK owns an independent bit list selected by the language already in the UA; the
mask is rendered as hex (feat=v1.101000100000005). The dedicated
AGENT_FRAMEWORK_FEATURE_MASK_DISABLED opt-out drops only the mask while
keeping the base SDK identity/version User-Agent. Python's existing
AGENT_FRAMEWORK_USER_AGENT_DISABLED continues to suppress its entire
contribution, including the mask; this decision does not introduce a matching
whole-User-Agent switch in .NET. OTel (C) is deferred — mainly because a
broadly-emitted span attribute would leak the fingerprint into the user's
general telemetry, against the first-party-only stance and would require
user-side OTel setup that may still not make the data available to us — but left
open behind the version prefix. Per-request scoping (S2), a shared registry (I),
codegen for the initial registry (K), and the decimal/grouped-hex/binary/bit-list/
base-N representations (L, M variant, N, O, P) are rejected as complexity or
length the problem does not require.
The remaining choice before implementation is the v1 granularity level among F0-F4. This is a point-in-time decision: adding new bits later is easier than removing or redefining them, because removals/redefinitions require a new registry version and historical decode tables. For v1, prefer the least detailed level that answers the known product/support questions so we do not force a v2 shortly after launch. The refreshed candidate registry uses 63 Python indexes and 52 .NET indexes, leaving 65 and 76 positions respectively. That headroom supports normal growth; it does not waive the registry's allocation tenet.
Consequences
- Good, adds a bounded-cost usage signal with no new data flow and few moving parts.
- Good, transparent (public registry, human-decodable token) and disabled by a
dedicated
AGENT_FRAMEWORK_FEATURE_MASK_DISABLEDmask-only opt-out. Python's existing whole-User-Agent opt-out also suppresses the mask. - Good, first-party-only + request-time stamping gives a live mask and no third-party fingerprint leak.
- Good, 128 bits leaves useful v1 headroom; .NET remains lock-free by storing two independently atomic 64-bit lanes; per-language lists remove all cross-language sync; package-local enums avoid both codegen and provider→core release coupling.
- Neutral, the token's reach equals eligible framework-configured first-party traffic; broader per-call signal (OTel) can be added later if needed.
- Neutral, every set bit is a repeated Boolean observation after first use; request rows carrying it are not feature invocation counts.
- Neutral, v1 granularity is intentionally a separate choice; the registry should start with fewer bits unless a more detailed bit answers a concrete question.
- Bad, each feature must add an activation mark, first-party clients need a per-request destination-aware hook, and the registry validator must scan all package-local index declarations.
Prior art
SDK telemetry-in-the-User-Agent is well-established; this design is closest to AWS's, and conventional in the rest. Summary of what comparable SDKs do:
| SDK | What's in the UA / headers | Usage-based? | Opt-out | Closest to ours? |
|---|---|---|---|---|
| AWS botocore | structured UA with an m/ token: a per-request set of short feature codes for features actually exercised (WAITER→B, PAGINATOR→C, retry mode, checksums, credential source, …) |
Yes — registered at call time via register_feature_id, contextvar-scoped per request |
AWS_SDK_UA_APP_ID sets app id (no opt-out for m/) |
Yes — direct analog |
| OpenAI / Anthropic (Stainless) | sidecar X-Stainless-* headers: lang, package version, OS, arch, runtime, runtime version; plus per-request x-stainless-retry-count, x-stainless-read-timeout |
Mostly static identity (retry/timeout are per-request) | none | No (static identity) |
Azure SDK (azure-core) |
User-Agent: azsdk-python-{pkg}/{ver} Python/{pyver} ({platform}) |
No | AZURE_TELEMETRY_DISABLED (tracing spans only, not the UA) |
No |
| Google API core | x-goog-api-client: gl-python/… grpc/… gax/… gapic/… |
No | none | No |
| LangSmith | User-Agent: langsmith-py/{ver}; usage lives in trace payloads |
No (header) | opt-in via LANGSMITH_TRACING_V2/LANGCHAIN_TRACING_V2; …HIDE_INPUTS/OUTPUTS |
No |
Takeaways that shaped (or validate) our choices:
- AWS
m/is the precedent for usage-based feature flags in a first-party User-Agent. It validates the core idea. Its key difference is the encoding: AWS uses a comma-separated set of 1–2 char short codes (open-ended, no bit coordination, but variable length), whereas we use a fixed-width hex bitmask (compact, bounded, decode-by-AND, but needs per-language bit allocation). We keep the bitmask for boundedness and trivial AND-decoding; AWS's short-code set is recorded as a viable alternative if bit-position coordination ever becomes painful (it would also drop the fixed 128-bit bound). - A fixed-width bitmask gives bounded token size for free. botocore must cap
the
m/component at 1024 bytes and truncate at delimiter boundaries (with a fallback log) precisely because its short-code set is unbounded. Our 128-bit hex is ≤32 chars by construction — no size cap, no truncation logic. - Scope is where we diverge most — and deliberately. botocore collects
features into a per-request
contextvarsset that is reset between requests, and no-ops outside a request context to prevent cross-request bleed. That works because every botocore feature is exercised inside an AWS service request. We are more general: some features are request-scoped (a chat call, an MCP tool invocation) but many are not bound to any request (workflow build/start, provider participation, hosting startup). So we use a process-global, monotonic mask (option S1), which is the only scope that can represent the non-request features. Our mask therefore intentionally "bleeds" (accumulates) for the life of the process — the opposite of botocore's reset — and that is the intended semantic, not the bug botocore guards against. - The mechanism is private; the wire format is the contract. botocore marks its whole user-agent module private and "subject to abrupt breaking changes." Same for us: the Python/.NET helpers are internal, and only the emitted token + the per-language registry tables are the stable, decodable contract.
- First-party-only emission is stricter than any of the above; the closest in spirit is Stainless headers, which only reach the owning API. We make the client/pipeline allowlist explicit (initially Foundry/Azure OpenAI) rather than attempting to infer safety from arbitrary request URLs. Other Azure clients join only after telemetry access is confirmed.
- Opt-out naming.
AZURE_TELEMETRY_DISABLEDis the family precedent for ourAGENT_FRAMEWORK_*_DISABLEDnames. Separately, the cross-toolDO_NOT_TRACKconvention (honored by e.g. HuggingFace Hub) is worth considering — see Open Questions.
Sources: botocore useragent.py
(_USERAGENT_FEATURE_MAPPINGS, register_feature_id, _build_feature_metadata);
openai-python _base_client.py platform_headers();
anthropic-sdk-python _base_client.py;
azure-core _universal.py UserAgentPolicy;
google-api-core client_info.py;
langsmith-sdk client.py /
utils.py;
huggingface_hub constants.py.
Registry versioning and migration (v1 → v2)
The token carries a per-language version (feat=v1.<hex>); a version bump is
independent for Python and .NET.
- Additive growth stays on v1 — no bump. Allocating a new feature to a reserved/unused bit is backward-compatible: an older decoder simply sees an unknown bit and ignores it. Normal package growth never needs a new version.
- A bump (v2) is required only for breaking changes: renumbering or re-partitioning existing bits, changing the meaning of an already-assigned index, or widening beyond 128-bit. Within a version an index is never reused or reassigned — that invariant is what lets old decoders stay correct.
- The draft 64→128 change is still v1. No v1 token or enum has shipped, so this pre-implementation repartition establishes the initial contract rather than migrating an existing one.
- Mixed-version coexistence is the norm. A fleet runs many SDK releases at
once, so
v1andv2tokens appear simultaneously for a long time (old SDKs keep emittingv1). The decoder keeps every published(language, version)table and selects by the token's version; thev1table is retained indefinitely for historical decode. - Unknown version → do not guess. A decoder without the
vNtable must record "unknown registry version" rather than decode against an older table — bit meanings may differ across versions, so mis-attribution is worse than no data. - Producing v2: publish the v2 table alongside v1, update the affected
package-local
FeatureIndexdeclarations and SDK version constant, and emitv2from the release that ships them. Prefer staying on v1 (additive) and reserving a clean v2 for an eventual deliberate re-partition.
Limitations
| Limitation | Caused by (choice) | Why we accepted it |
|---|---|---|
| No signal for self-hosted or third-party-only traffic. If a process never calls Azure/Foundry, we see nothing. | First-party-only emission (A) | We can't read third-party logs anyway, and must not leak a fingerprint into them. Reach traded for privacy. |
Not every first-party client is stampable. Caller-supplied AIProjectClient / OpenAI clients and toolkit-owned clients may not expose a supported per-request policy hook. |
Supported-hook-only emission (A) | V1 does not mutate caller-owned clients or private SDK pipelines. Those features may still appear on another eligible request from the same process-global mask. |
| Custom origins intentionally receive no feature token. A customer gateway may use Azure credentials or Azure-named settings but route to a non-approved origin. | Two-factor destination classification (A) | Credentials and configuration names are not proof of telemetry ownership. Unknown/custom origins and cross-origin redirects are denied by default. |
| No OTel / per-call signal in v1. | OTel deferred (C) — primarily on privacy and availability grounds | A broadly-emitted span attribute would push the fingerprint into the user's general telemetry / third-party APM vendors, undoing the first-party-only scoping. It also requires customer/user OTel setup, and even Foundry users may not export data where we can query it. Left open only if there is a compelling reason to add. |
| Mask reflects "usage so far," not the whole session. Early requests carry fewer bits than later ones. | Process-global accumulator + request-time stamping | Honest and still useful as a Boolean process-lifetime observation. Repeated request rows must not be summed as additional uses. Reading the mask at request time makes it grow rather than freeze. |
| No per-agent / per-call attribution. The mask is one process-wide value — "this process used X", not "this agent/call used X". | Process-global monotonic scope (S1) | A deliberate choice, not a transport limit: botocore does per-call attribution in the UA via a per-request contextvars set, but many AF activations (workflow build/start, provider participation, hosting startup) occur outside the service request that later emits the token. Per-call detail remains deferred to OTel. |
| Shared processes intentionally carry usage across agents and tenants. A request can include bits first set by another workload in the same worker. | Process-global monotonic scope (S1) | The token must be interpreted only as process-level "used so far," never as request/user/tenant attribution. Privacy review must explicitly accept this. |
| Bits are binary, sticky observations — not countable events. Once set, a bit appears on every later eligible request from that process, so raw request counts repeat the same observation and long-lived/high-traffic processes dominate. | Monotonic mask stamped at request time | The signal supports coarse observed-feature and co-occurrence questions only. It cannot provide first-use counts, unique-process counts, request attribution, or feature invocation frequency. |
| Granularity may be too coarse or too detailed. The chosen level may miss useful distinctions or create more specificity than needed. | v1 granularity choice (F0-F4) | This is the main remaining decision. Adding bits later is easier than removing/redefining them, so v1 should lean toward fewer bits that answer known questions. |
| .NET snapshots span two atomic lanes. A bit can be marked between the low/high reads, so one request may omit that just-added bit. | 128-bit width without a global lock | The mask is monotonic: the snapshot cannot invent or clear a bit, and the next request includes the addition. This matches the existing "usage so far" timing semantics. |
| Fingerprinting risk is reduced, not eliminated. A feature-combination mask is still a deployment signature, and it transits intermediaries (proxies/CDNs) even when first-party-scoped. | Emitting any feature-combination value | Scope + opt-out + coarse granularity mitigate it; v1 should avoid unnecessary detailed bits. |
Open Questions (for decider discussion)
These are unresolved and should be decided before implementation:
- Which v1 granularity level (F0-F4)? This is the primary remaining choice. Adding bits later is easier than removing or redefining bits, so v1 should choose the least detailed level that answers known questions and avoids a quick v2.
- Privacy approval for the v1 User-Agent signal. Before implementation, confirm that a transparent, opt-out, first-party-only feature-combination fingerprint is acceptable, including the exact client allowlist, retention, access, and permitted product queries. This is a rollout precondition.
- When (if ever) to add the OTel path? Held back mainly for privacy and data availability: a span attribute broadcasts the fingerprint into the user's general telemetry and onward to third-party APM vendors, contradicting the first-party-only stance, and it requires user-side OTel setup that may not make the data available to us even for Foundry users. It also carries a metric-cardinality hazard. Revisit only if the User-Agent path cannot answer a concrete question.
- Honor the cross-tool
DO_NOT_TRACKconvention? Several ecosystems treatDO_NOT_TRACK=1as a universal telemetry opt-out (HuggingFace Hub honors it; see Prior art). Should our mask opt-out also respectDO_NOT_TRACK(in addition toAGENT_FRAMEWORK_FEATURE_MASK_DISABLEDand Python's pre-existing whole-UA flag)? Cheap to add and community-friendly, but it widens the opt-out surface and needs a clear precedence rule. Recommend yes; confirm with the deciders.
Decided
- Dedicated opt-out flag — included. In addition to the existing
Python
AGENT_FRAMEWORK_USER_AGENT_DISABLED(drops the whole UA), v1 shipsAGENT_FRAMEWORK_FEATURE_MASK_DISABLED, which drops only the feature mask while keeping the base SDK identity/version User-Agent. This lets a privacy-conscious user withhold the usage signal without losing the support/compat value of the SDK-version header. .NET adopts the dedicated mask-only flag; adding a .NET whole-User-Agent switch is outside this decision. - Caller-owned clients are not modified. V1 stamps only framework-created clients or clients with a supported public policy/hook registration point. It does not patch private pipelines; injected clients are an explicit coverage limitation.
- Destination approval is explicit and redirect-aware. An eligible pipeline still emits only to a reviewed HTTPS origin. Custom origins are default-deny, and the token is removed on an unapproved redirect hop.
- Telemetry does not replace transport defaults. Framework-created OpenAI clients use the SDK's default async HTTP client with the request hook added, preserving redirect, timeout, connection-limit, and pooling behavior.
- Marking uses activation, not DI construction. Operational surfaces mark on first real use; a constructor marks only when construction itself exercises or registers the capability.
More Information
- Mechanism & API: SPEC-004
- Per-language bit tables, encoding, opt-out, governance: feature-usage-bit-registry.md
- Existing accumulator pattern:
python/packages/core/agent_framework/_telemetry.py - .NET emission policies:
dotnet/src/Microsoft.Agents.AI.Foundry/AgentFrameworkUserAgentPolicy.cs,dotnet/src/Microsoft.Agents.AI.Foundry.Hosting/HostedAgentUserAgentPolicy.cs