github-featured / comet-ml--opik

发布

0 发布 8,340 Git 标签

RSS 订阅源
  • 1.8.43-2585 7af704e373
    比较

    [OPIK-2228] [P SDK] LLM-a-a-J metrics improvements batch (#3157)

    frostbyte_neo 发布于 2025-09-05 12:17:11 +00:00

    • [OPIK-2228] Updated LiteLLMChatModel instantiation to replace name and base_url with model_name in custom metrics documentation.

    • [OPIK-2228] Enhanced documentation for custom metrics usage in evaluation.

    • Improved BaseMetric inheritance structure in examples (e.g., super().__init__() usage).
    • Updated score method signatures to include return type ScoreResult or List[ScoreResult].
    • Added an example for implementing a custom metric with usage in evaluate function.
    • Refactored code examples for better clarity and adherence to best practices.
    • [OPIK-2228] Improved G-Eval documentation and fixed minor grammar issue in metric docstring.
    • Updated G-Eval documentation to clarify score normalization and evaluation process.
    • Fixed a minor grammar issue in metric.py docstring for parent span/trace explanation.
    • [OPIK-2228] Added a temperature parameter to G-Eval metrics for model initialization.
    • Introduced a temperature parameter with default value 0.0 to G-Eval metric initialization.
    • Updated _init_model method to accept and apply temperature for model configuration.
    • Improved docstrings to reflect the new parameter.
    • [OPIK-2228] Added temperature parameter to evaluation metrics.
    • Introduced a temperature parameter (default 1e-8) for model initialization across Conversational Coherence, User Frustration, and Session Completeness metrics.
    • Updated _init_model method to support temperature configuration.
    • Enhanced docstrings to include the new parameter.
    • [OPIK-2228] Updated model_name to model for OpenAI Chat model initialization and adjusted test cases.
    • Replaced model_name parameter with model in ChatOpenAI instantiation across test and parser files.
    • Updated test input outputs to align with the parameter change.
    • Fixed logprobs dictionary access in parser.py for accurate result parsing.
    • [OPIK-2228] Refactored score parsing logic in G-Eval parser.
    • Introduced _extract_score_from_text_content function for cleaner text-based score extraction.
    • Enhanced log_probs handling with fallback logic for non-dict choices.
    • Improved robustness of score token extraction and response type checks.
    下载附件