发布

RSS 订阅源

1.10.57-4764 26ba36ff2c

比较
[NA] [SDK] CLI import/export improvements (#5616)

frostbyte_neo 发布于 2026-03-31 18:06:26 +00:00
- [SDK] Add opik import all and opik export all subcommands
Adds a new all subcommand to both the import and export CLI groups so
users can import or export everything in a workspace with a single command.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Preserve read-only fields in import metadata
Backend fields like created_at, created_by, last_updated_at, and
last_updated_by are read-only and ignored on the write path. Store them
under import* keys in trace/span/experiment metadata so the information
survives a round-trip and can be migrated once the backend accepts them
directly.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix PromptType case sensitivity during import
Exported prompt types are uppercased (e.g. "MUSTACHE") but the PromptType
enum uses lowercase values. Try lowercased value first, fall back to the
original, then default to MUSTACHE on unknown values.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix thread-safe project name cache in experiment export
Two concurrent threads could race on the same project_id cache miss and
both issue redundant API calls. Adds a threading.Lock with setdefault so
the first writer wins. Also accepts a pre-fetched experiment_obj to skip
the redundant get_experiment_by_id call when the caller already has it.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Add ExportManifest for resumable, incremental project exports
Introduces ExportManifest (export_manifest.db, SQLite/WAL) alongside each
project's trace directory to track download state across runs:
- Completed run: uses a created_at >= last_exported_at-5min filter so
  subsequent runs only fetch traces newer than the last export, reducing
  API pages from O(total traces) to O(new traces).
- Aborted run (status=in_progress): loads the already-downloaded set from
  the DB and resumes without re-downloading completed traces.
- No manifest yet: seeds from existing files on disk so pre-manifest
  downloads are not repeated.
- --force: resets the manifest and re-downloads everything.
- Format change (json<->csv): resets the manifest with a warning.
Also replaces per-trace Path.exists() calls with a single pre-scanned set
loaded once before the pagination loop.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Retry on 429 and honour Retry-After header in REST client
Adds 429 (Too Many Requests) to the retryable status codes and introduces
a custom wait function that reads the server's Retry-After header (capped
at 60 s) before falling back to exponential backoff.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix mypy type error in exports/all.py
prompt_obj was inferred as ChatPrompt | None from the first branch,
causing a type mismatch when the fallback assigned Prompt | None.
Explicitly annotate as Optional[Union[Prompt, ChatPrompt]].

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Skip redundant API calls for already-downloaded experiment traces
Phase 1 – filesystem scan: before submitting any work to the thread pool,
scan projects//trace_.{json,csv} to build a set of already-downloaded
trace IDs and pre-filter the pending list. Traces whose files exist never
trigger a get_trace_content() + search_spans() round-trip.

Phase 2 – per-experiment manifest: ExportManifest (filename param added)
is created per experiment under experiments/manifest_.db. After the
first successful export the full trace-ID list and downloaded set are
persisted. On re-run the manifest short-circuits get_items() pagination
entirely and filters the pending list from the DB instead of the filesystem.
manifest.complete() is only written when failed_count == 0 so interrupted
runs resume correctly.

ExportManifest gains store_all_trace_ids() / get_all_trace_ids() and an
optional filename constructor param to support per-experiment manifests
alongside the existing per-project ones.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix tuple unpack error in export all experiments
export_experiment_by_id returns a 3-tuple (stats, file_written, manifest)
but _export_all_experiments was only unpacking 2 values, causing the
"too many values to unpack" error on opik export all.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Add --filter to experiment and all export commands
Extends OQL trace filtering (already available on opik export project)
to opik export experiment and opik export all.

For experiment exports, filtering is applied client-side after each trace
is fetched by ID, using a new matches_trace_filter() helper in utils.py
that evaluates OQL expressions against a trace dict (handles date_time,
number, and string fields including nested key access).

For all exports, the filter is forwarded to both the projects phase
(server-side, unchanged) and the experiments phase (client-side via the
same helper).

Examples:
opik export ws experiment "my-exp" --filter 'created_at >= "2024-01-01T00:00:00Z"'
opik export ws all --filter 'created_at >= "2024-01-01T00:00:00Z"'

Also updates import_export_commands.mdx to document the broader --filter
support and the opik export all command options.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Address PR #5616 review comments in CLI export/import
- Wrap _fetch_experiments_page_raw HTTP call in try/except for
  httpx.ConnectError, httpx.TimeoutException, and HTTPStatusError;
  return empty page on transient errors instead of propagating the
  exception as a hard failure.
- Verify trace file exists on disk before treating a manifest entry as
  already-downloaded; stale entries (file deleted after manifest was
  written) now trigger a re-download instead of being silently skipped.
- Track had_errors in export_traces (API error break + write exceptions)
  and only call manifest.complete() when had_errors is False, so
  interrupted exports stay in_progress and resume correctly.
- Gate the export_experiment_by_id fast path on the experiment JSON file
  actually existing on disk; reset manifest and fall through to a full
  re-export when the file was deleted.
- Fall back to _scan_downloaded_trace_ids() in export_traces_by_ids when
  the manifest exists but load_downloaded_set() returns an empty set, so
  pre-existing trace files are detected before any API calls are made.
- Extract duplicate _validate_include logic into cli/utils.py
  validate_include(); both exports/all.py and imports/all.py now
  delegate to the shared helper.
- Extract duplicate "cap → print → export_traces_by_ids" sequence into
  _export_collected_trace_ids() in experiment.py; reuse it in all.py.
- Update test_cli_changes.py for the new (exported, skipped, had_errors)
  return signature of export_traces.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Skip full API scan when all project traces are already downloaded
Two improvements to avoid slow full-page scans on resume/re-run:
1. Fast-path early exit in export_traces: after fetching page 1, compare
  the API's total trace count against len(already_downloaded). If the
  API reports no more traces than we have locally, break immediately
  (1 API call) instead of scanning all remaining pages (O(total/100) calls).
2. After seeding a brand-new manifest from existing filesystem files, mark
  it completed immediately (using the newest file's mtime as the cutoff)
  so the incremental created_at filter kicks in for the current run, not
  just the next one.
Also fix the "No traces found" message to only print when both
exported_count and skipped_count are zero.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Parallelize experiment export with 10 workers and broaden manifest fast-path
- Replace sequential experiment loop in _export_all_experiments() with
  ThreadPoolExecutor(max_workers=10), matching how projects are exported.
  Trace IDs are collected from manifests in the main thread after each
  future completes, avoiding shared-set race conditions.
- Broaden the manifest fast-path in export_experiment_by_id() to fire
  whenever stored trace IDs exist + experiment file is on disk, not only
  when manifest status=completed. This also skips API calls for in_progress
  manifests (crashed runs that already fetched items).
- Set check_same_thread=False on ExportManifest SQLite connections so the
  main thread can safely read a manifest created by a worker thread after
  the worker has finished.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Retry on 429 in _fetch_spans and halve MAX_WORKERS
Parallel span fetches were hitting the server's rate limiter with no
retry logic, causing traces to be silently dropped. Now _fetch_spans
retries up to 5 times with exponential back-off (2s base) on HTTP 429,
and MAX_WORKERS is reduced from 20 to 10 to lower the burst rate.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Build trace-to-project map from filenames, not file contents
After importing 100k+ traces, the importer was silently stalling for
minutes opening and JSON-parsing every trace file just to extract the
trace ID for a project-name lookup. The ID is already encoded in the
filename (trace_.json), so parse it from the stem instead.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Address baz PR #5616 review comments (code quality + tests)
- Move matches_trace_filter to exports/trace_filter.py (avoid catch-all utils)
- Log warning on ValueError instead of silently returning True in matches_trace_filter
- Fix double-counting filtered traces in export_traces_by_ids progress bar
- Rename _export_collected_trace_ids → export_collected_trace_ids (public API)
- Move validate_include to cli/include_validation.py (avoid catch-all utils.py)
- Add threading.Semaphore backpressure to _export_all_experiments thread pool
- Extract extract_trace_id_from_filename helper; remove duplicated stem logic
- Rename TestBuildImportMetadata tests to test__case__expected convention
- Add tests for matches_trace_filter (trace_filter.py)
- Add tests for project inference from trace filenames in import
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Add unit tests for export_all, import_all, and manifests
57 tests covering ExportManifest/MigrationManifest lifecycle (not_started
→ in_progress → completed → reset), trace/file tracking with batch flush,
import_all manifest resume/incremental/force/dry-run behavior, export_all
phase filtering and stats aggregation, and _paginate pagination logic.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [Docs] Add intent/trigger prose and maintenance notes to CLI example snippets
PR review requested one-line intent/trigger context and either a canonical
source link or a hand-maintenance note for each new snippet block. Split the
monolithic export/import Examples code blocks into grouped sub-sections, each
preceded by a sentence explaining when to reach for that command. Added MDX
comment blocks recording that the snippets are hand-maintained, pointing to
sdks/python/src/opik/cli/ as the canonical source, and noting to re-verify
against --help when CLI options change.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Expand unit tests for matches_trace_filter to cover all branches
Adds 16 new tests covering number comparisons, is_empty/is_not_empty,
not_contains/starts_with/ends_with, dotted and key-based field access,
missing field → False, AND semantics across multiple expressions, naive
datetime treated as UTC, and unparseable date_time value → False.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Replace hand-rolled retry loop in _fetch_spans with opik_rest_retry
Remove the bespoke 429-only sleep/retry loop and dead raise RuntimeError("unreachable") in favour of the shared @opik_rest_retry tenacity decorator, which also handles 5xx and network errors, honours Retry-After headers, and re-raises the original exception on exhaustion.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Document parse-failure fallback in matches_trace_filter docstring
Clarify that a ValueError from OpikQueryLanguage causes the function to
return True (keep the trace) rather than False, explain why (avoid silent
data loss), and note that a warning is logged so callers can detect it.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix experiment export deadlock and add JSON-based fast-path
Deadlock: _export_all_experiments used a semaphore to cap concurrent
submissions, but released it in the as_completed loop (Loop 2) which
only runs after the submission loop (Loop 1) finishes. With more than
max_workers*2 (20) experiments the main thread blocks on acquire() in
Loop 1 forever, since Loop 2 never starts. Fixed by attaching a
done_callback to each future so the slot is released as soon as the
worker finishes, decoupling release from the collection loop.

Secondary fast-path: when an experiment JSON file already exists but
the manifest has no stored all_trace_ids (exported before the per-
experiment manifest feature, or an interrupted run), the export now
reads trace IDs directly from the JSON rather than calling
get_items() via the API. On the next run all previously-seen
experiments will skip get_items() entirely.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Address PR review comments on CLI export modules
- Use module-style import for retry_decorator in project.py (style guide)
- Narrow bare except Exception to (OSError, json.JSONDecodeError) in
  the JSON fast-path fallback so KeyboardInterrupt/SystemExit propagate
- Add module-level import json and remove redundant inline import
- Add unit tests for the JSON-based fast path in export_experiment_by_id:
  verifies get_items() is skipped and trace IDs are read from disk, and
  that a corrupt JSON file falls through to the full API path
- Clarify the semaphore done-callback comment in all.py (remove "Loop 2"
  reference, name the as_completed loop explicitly)
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Address remaining PR review comments: warning log and semaphore test
- experiment.py: upgrade JSON fast-path fallback from silent debug_print
  to always-visible console warning with stack trace in --debug mode
  (thread 2924741570)
- test_cli_changes.py: add TestExportAllExperimentsSemaphore unit test
  that submits N > max_workers*2 experiments to verify the done_callback
  releases the semaphore and prevents deadlock (thread 2924741577)
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Centralize retry and filter-validation helpers in project export
- Extract _fetch_traces_page() with @opik_rest_retry so pagination
  retries on 429/5xx instead of swallowing errors and breaking the loop
- Extract _validate_filter_syntax() + _print_oql_examples() to replace
  duplicate ~15-line validation blocks in export_traces and
  export_project_by_name
- Fix missing had_errors = True in the span-fetch future exception
  handler, which previously let the manifest be marked completed even
  when some traces failed to download
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix false-positive dataset import warning when manifest skips already-imported datasets
When running opik import all, Phase 1 imports datasets and the manifest
marks them completed. Phase 4 (experiments) calls the same importer and
the manifest skips them (datasets_skipped=1, datasets=0). The warning
"No datasets were imported" fired on datasets==0 without checking skipped,
producing a misleading message even though the dataset was already present.

Now the warning only fires when both imported==0 and skipped==0. The
status message also distinguishes between fresh imports and skipped ones.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Skip filter validation for whitespace-only filter strings
Guard both _validate_filter_syntax calls with filter_string.strip()
so that a whitespace-only --filter argument is treated as a no-op
instead of crashing the OQL parser with an IndexError.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Fix filter-skipped traces inflating failed_count in export_traces_by_ids
Filtered traces (dropped by client-side matches_trace_filter) were counted
as fetch failures in batch_fetch_failures, inflating failed_count and
preventing manifest.complete() from being called even when there were no
actual errors.

Track batch_filter_skipped separately and exclude those from the failure
count so manifests are correctly completed after filtered exports.

Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com
- [SDK] Address baz-reviewer open comments on CLI export
Fix five substantive issues flagged in the baz-reviewer review:
1. project.py fast-path guard — skip the api_total/already_downloaded
  short-circuit when a filter is active. previously, already_downloaded
  (built from an unfiltered run) could be larger than the filtered
  api_total, causing the loop to exit before downloading any new
  filtered traces.
2. experiment.py fetch errors — remove the if debug: gate so that
  exceptions from fetch_future.result() are always printed, not only
  in debug mode.
3. experiment.py None-result counting — _fetch_trace_data returns None
  for permanently-unresolvable traces (missing project_id, failed
  project lookup). these were counted as failures, blocking
  manifest.complete() forever. they are now counted as skips so the
  manifest can complete and incremental exports work correctly.
4. all.py error propagation — _export_all_projects and
  _export_all_experiments now return a had_errors bool as the last
  element. export_all accumulates the flag across phases and exits
  with code 1 when any project or experiment export fails, rather
  than silently exiting 0.
5. tests — update existing mocks to unpack the new 5-tuple from
  _export_all_experiments; fix the semaphore-deadlock test in
  test_cli_changes.py; add filename-based project-inference test for
  import_experiments_from_directory.
Co-Authored-By: Claude Sonnet 4.6 noreply@anthropic.com

Co-authored-by: Douglas Blank doug@comet.com
Co-authored-by: Claude Sonnet 4.6 noreply@anthropic.com
下载附件
- 源代码 (ZIP)
- 源代码 (TAR.GZ)

0 发布 8,340 Git 标签

[NA] [SDK] CLI import/export improvements (#5616)