Files
cragwolfe 94e0204766 feat: add partition runtime telemetry (#4442)
## Summary

Adds one privacy-bounded, best-effort runtime telemetry attempt for each
outermost public partition invocation. A shared outer decorator covers
all 26 public entrypoints and uses context-local invocation state to
suppress automatic dispatch, format conversion, body/attachment, and
other nested calls.

Successful events report only fixed-enum processing characteristics and
aggregate counts over the final returned elements. Error events contain
only the required environment/package fields and a safely resolved
normalized document type when available. Processing results, ordering,
signatures, and original exceptions are preserved.

## Privacy and controls

Startup and runtime telemetry share `GET
https://packages.unstructured.io/python-telemetry`; the endpoint
supports both the existing per-application-startup query schema and the
new per-document runtime query schema. Runtime telemetry is default-on,
matching startup telemetry, and uses URL query parameters with no
request body. Either `DO_NOT_TRACK` or `SCARF_NO_ANALYTICS`, when
nonempty after trimming, disables collection before runtime argument
inspection or scheduling.

The event excludes content, filenames, paths, input URLs, raw MIME
values, exception details, credentials, proxy configuration, and
persistent identifiers. Delivery uses a fresh session that disables
environment/netrc trust, redirects, retries, and response-body
downloads.

## Reliability

Each eligible outer invocation makes one local delivery attempt. A
process-wide nonblocking single slot allows at most one daemon worker
and no queue. Partition processing never waits for the network; if the
worker is occupied or thread creation fails, the event is dropped. A
stalled DNS/proxy/socket operation can strand at most one daemon thread,
later events drop, and interpreter/process exit does not wait for it.
The `(0.5, 0.5)` connect/read timeout is an inactivity bound, not a
total network wall-clock deadline.

## Validation

- `uv run pytest -q test_unstructured/test_runtime_telemetry.py
test_unstructured/test_telemetry.py
test_unstructured/partition/test_text.py
test_unstructured/partition/test_api.py` — 133 passed, 6 skipped
- `uv run ruff check unstructured/telemetry.py
unstructured/partition/pdf.py
test_unstructured/test_runtime_telemetry.py`
- `uv run ruff format --check unstructured/telemetry.py
unstructured/partition/pdf.py
test_unstructured/test_runtime_telemetry.py`
- `uv run mypy unstructured/telemetry.py`
- Exact-head GitHub checks — 53 passed
- GPT-5.6 Sol Pro exact-head production review — `SAFE TO MERGE`

(authored by codex)
2026-08-21 11:09:56 -07:00
..