-
[OPIK-4958] [SDK] feat: generate evaluation_suite.run() result report file (#5770)
frostbyte_neo 发布于
2026-03-24 16:06:53 +00:00 - [OPIK-4958] [SDK] feat: generate evaluation_suite.run() result report file
After evaluation suite run completes, automatically save a structured
JSON report with inputs, outputs, assertions, and pass/fail results
for every item and run. The report path is printed alongside the
experiment link. Controlled via generate_report param (default True).Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- docs(evaluation_suite): add generate_report params to internal docstring
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- refactor(result_file): use readable variable names
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- refactor(result_file): move suite_name and total_time into EvaluationSuiteResult
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- refactor(evaluation_suite): extract result class, clean up report file I/O
- Move EvaluationSuiteResult and ItemResult to evaluation_suite_result.py
- Rename result_file.py to report_file.py, split path resolution from writing
- Move report saving and display from evaluator into evaluation_suite
- Remove save_report method from result class
- Pass suite_name/total_time through build_suite_result constructor
- Add opik_evaluation_suite_reports/ to .gitignore
- Test only public APIs, no private attribute access
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- refactor(evaluation_suite): add report_processors namespace with displayer and file_writer
- Move display logic from evaluation/report.py to report_processors/displayer.py
- Move file writing from report_file.py to report_processors/file_writer.py
- Both modules accept EvaluationSuiteResult directly
- Simplify displayer signature: suite_name, total_time, experiment_url
are read from the result object
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- refactor(evaluation_suite): address PR review comments
- Extract shared is_score_passed() helper to eliminate duplicated pass logic
- Add configured_runs_per_item to ItemResult for accurate execution policy reporting
- Remove default param values from internal functions (displayer, evaluator)
- Fix docstring referencing opik_reports/ instead of opik_evaluation_suite_reports/
- Rename tests to follow test_WHAT__CASE__EXPECTED convention
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Fix lint errors
Co-authored-by: Claude Opus 4.6 noreply@anthropic.com
下载附件