发布

RSS 订阅源

1.10.24-4354 02940da29d

比较
[OPIK-4557] [BE] [SDK] Add evaluation_method field to experiments (#5385)

frostbyte_neo 发布于 2026-03-02 09:23:32 +00:00
- [OPIK-4557] [BE] [SDK] Add evaluation_method field to experiments
Add evaluation_method enum (dataset/evaluation_suite) to the experiments
table to distinguish how an experiment was created. The SDK now explicitly
passes evaluation_method="evaluation_suite" when calling evaluate_suite(),
while evaluate() defaults to "dataset".

Backend:
- New EvaluationMethod enum with ClickHouse migration
- Experiment DAO: INSERT, SELECT, and mapToDto support
- Field is immutable (set-once at creation, not updatable)
SDK:
- create_experiment() accepts evaluation_method param (default: "dataset")
- evaluate_suite() passes evaluation_method="evaluation_suite"
- Unit tests verify evaluate_suite sends correct method
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Revision 2: Address PR review comments — import style and test assertions
- Fix Python SDK import convention: from unittest import mock → import unittest.mock as mock
- Remove evaluationMethod from EXPERIMENT_IGNORED_FIELDS so the field gets validated
  via recursive comparison assertions (consistent with executionPolicy pattern)
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Revision 3: Bump migration prefix 000060 → 000061 to avoid conflict with main
Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com
- Revision 4: Use 'unknown' as DB default for evaluation_method
Existing experiments predate the evaluation_method column, and some may
have been created via evaluate_suite(). Using 'unknown' as the DB default
avoids incorrectly labeling them as 'dataset'. The Java enum does not
include 'unknown' — it maps to null on read.

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

Co-authored-by: Claude Opus 4.6 noreply@anthropic.com
下载附件
- 源代码 (ZIP)
- 源代码 (TAR.GZ)

0 发布 8,340 Git 标签

[OPIK-4557] [BE] [SDK] Add evaluation_method field to experiments (#5385)