-
[NA] [SDK] feat: rename evaluation_suite to test_suite, add Python SDK API improvements (#6223)
frostbyte_neo 发布于
2026-04-14 10:06:21 +00:00 - [OPIK-5797][OPIK-5798] [SDK] feat: rename evaluation_suite to test_suite, add SDK API improvements
Python SDK and TypeScript SDK changes for the test suite feature:
- Rename evaluation_suite to test_suite across both SDKs
- Add client-level test suite methods (create, get, delete, get_items, clear)
- Add TestSuiteVersion, converters module, suite result constructor
- Expose id/versioning, replace suite.run with opik.run_tests
- Fix pass_rate to exclude no-assertion items
- Isolate assertion text in LLM judge prompt
- Add comprehensive unit and e2e tests
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix: remove v1 compatibility test for test suites
Test suites are v2-only, no need for v1 compatibility tests.
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix: remove old evaluation_suite files replaced by test_suite
Delete leftover evaluation_suite source, tests, and examples from both
Python SDK and TypeScript SDK. These are fully replaced by the new
test_suite equivalents.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
-
Update gitignore
-
fix: use evaluation_suite API strings to match current Fern types
The Fern-generated code still uses "evaluation_suite" as the enum value
for dataset type and experiment evaluation method. Updated all SDK code
and tests to use "evaluation_suite" instead of "test_suite" when
communicating with the REST API.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
-
Skip GROQ test for langchain
-
fix: thread Opik client through get_test_suites and get_version_view
- Pass client through get_test_suites into Dataset and TestSuite
constructors so run_tests doesn't fall back to global client - Pass client in get_version_view when building DatasetVersion
- Fix evaluation_method type hint to use "evaluation_suite"
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix: update test expectation for LLM judge assertion delimiters
The prompt template now wraps assertions in ---BEGIN/END ASSERTIONS---
delimiters. Update the serialization test to match.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix: get_datasets() returns all datasets including test suites
Remove the evaluation_suite type filter from get_datasets() for backward
compatibility — users who created evaluation suites should still see
them as datasets. get_test_suites() continues to return only test suites.Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
- fix: e2e test fixes for get_items description key and get_datasets behavior
- Use .get("description") instead of ["description"] since the key is
only present when not None - Update get_datasets assertion to expect test suites included (backward
compat change)
Co-Authored-By: Claude Opus 4.6 (1M context) noreply@anthropic.com
Co-authored-by: Claude Opus 4.6 (1M context) noreply@anthropic.com
下载附件