github-featured / unstructured-io--unstructured

发布

0 发布 249 Git 标签

RSS 订阅源
  • 0.27.0 94e0204766
    比较

    feat: add partition runtime telemetry (#4442)

    frostbyte_neo 发布于 2026-08-21 18:09:56 +00:00

    Summary

    Adds one privacy-bounded, best-effort runtime telemetry attempt for each
    outermost public partition invocation. A shared outer decorator covers
    all 26 public entrypoints and uses context-local invocation state to
    suppress automatic dispatch, format conversion, body/attachment, and
    other nested calls.

    Successful events report only fixed-enum processing characteristics and
    aggregate counts over the final returned elements. Error events contain
    only the required environment/package fields and a safely resolved
    normalized document type when available. Processing results, ordering,
    signatures, and original exceptions are preserved.

    Privacy and controls

    Startup and runtime telemetry share GET https://packages.unstructured.io/python-telemetry; the endpoint
    supports both the existing per-application-startup query schema and the
    new per-document runtime query schema. Runtime telemetry is default-on,
    matching startup telemetry, and uses URL query parameters with no
    request body. Either DO_NOT_TRACK or SCARF_NO_ANALYTICS, when
    nonempty after trimming, disables collection before runtime argument
    inspection or scheduling.

    The event excludes content, filenames, paths, input URLs, raw MIME
    values, exception details, credentials, proxy configuration, and
    persistent identifiers. Delivery uses a fresh session that disables
    environment/netrc trust, redirects, retries, and response-body
    downloads.

    Reliability

    Each eligible outer invocation makes one local delivery attempt. A
    process-wide nonblocking single slot allows at most one daemon worker
    and no queue. Partition processing never waits for the network; if the
    worker is occupied or thread creation fails, the event is dropped. A
    stalled DNS/proxy/socket operation can strand at most one daemon thread,
    later events drop, and interpreter/process exit does not wait for it.
    The (0.5, 0.5) connect/read timeout is an inactivity bound, not a
    total network wall-clock deadline.

    Validation

    • uv run pytest -q test_unstructured/test_runtime_telemetry.py test_unstructured/test_telemetry.py test_unstructured/partition/test_text.py test_unstructured/partition/test_api.py — 133 passed, 6 skipped
    • uv run ruff check unstructured/telemetry.py unstructured/partition/pdf.py test_unstructured/test_runtime_telemetry.py
    • uv run ruff format --check unstructured/telemetry.py unstructured/partition/pdf.py test_unstructured/test_runtime_telemetry.py
    • uv run mypy unstructured/telemetry.py
    • Exact-head GitHub checks — 53 passed
    • GPT-5.6 Sol Pro exact-head production review — SAFE TO MERGE

    (authored by codex)

    下载附件