github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.8.55-2754 29d77e7eff
    比较

    [OPIK-2439] [P SDK] Implement calculation of mean/min/max/std for each metric in experiment after the evaluation (#3357)

    frostbyte_neo 发布于 2025-09-24 11:32:13 +00:00

    • [OPIK-2439] [P SDK] Add aggregated score calculation support in an evaluation framework
    • Introduced EvaluationResultAggregatedScoresView to represent aggregated evaluation scores.
    • Added aggregate_evaluations_scores method in EvaluationResult for calculating and returning aggregated score views.
    • Refactored _calculate_aggregated_scores for broader reuse in score aggregation logic.
    • Updated relevant docstrings and method annotations for clarity.
    • [OPIK-2439] [P SDK] Fix typo in the aggregate_evaluation_scores method
    • Renamed aggregate_evaluations_scores to aggregate_evaluation_scores in EvaluationResult.
    • Added new test cases to cover aggregation scenarios, including handling of single, multiple, empty, and filtered-out scores.
    • [OPIK-2448] [P SDK] Add detailed evaluation statistics display for verbose mode
    • Introduced display_evaluation_scores_statistics function in report.py to show evaluation scores with mean, min, max, and std.
    • Updated evaluator.py to include detailed statistics when verbose mode is set to 2.
    • [OPIK-2439] [P SDK] Extract and modularize score statistics logic
    • Moved ScoreStatistics and _calculate_aggregated_scores functionalities to a new score_statistics.py module.
    • Updated evaluation_result.py to import and use the refactored score_statistics logic for calculating aggregated statistics.
    • Improved code structure and reusability with clear separation of concerns.
    • [OPIK-2439] [Docs] Add Python examples for analyzing evaluation results
    • Added usage examples demonstrating aggregate_evaluation_scores to the documentation.
    • Updated evaluation guides with Python snippets showcasing how to analyze and compare model performance.

    • [OPIK-2439] [Docs] Updated documentation

    • [OPIK-2439] [Docs] Fixed documentation_preview_link.yml workflow.

    下载附件