-
[OPIK-3932] [FE][BE] Add CODE metric type for custom Python evaluation in Optimization Studio (#4817)
frostbyte_neo 发布于
2026-01-26 11:46:12 +00:00 - [NA] [FE][BE] Add CODE metric type for custom Python evaluation in Optimization Studio
- Add METRIC_TYPE.CODE to frontend types and constants
- Create CodeMetricConfigs.tsx component with CodeMirror for Python syntax highlighting
- Update MetricConfigs.tsx to render CODE metric configuration
- Add Zod schema validation for CODE metric parameters
- Implement MetricFactory.register('code') in Python backend to execute user-provided Python code
- Support both standalone functions and BaseMetric subclasses in code metric
- Add default code template for CODE metric
- Add e2e test for code metric optimization
- Revision 2: Add security validation for code metrics
- Add code safety validation against dangerous patterns (exec, eval, subprocess, etc.)
- Add module import allowlist (json, re, math, opik.* etc.)
- Add OPIK_CODE_METRIC_ENABLED env var to disable feature if needed
- Use restricted builtins namespace for code execution
- Add comprehensive audit logging for code metric creation
- Add 20+ unit tests for security validation
- Document that primary isolation is via subprocess execution
-
review fixes
-
Update Helm documentation
-
Fix baz review comments and remove the code safety checks
-
move e2e test
-
refactor code executor to use ProcessPoolExecutor
-
fix code evaluation implementation
-
fix tests
Co-authored-by: CometActions github-actions@comet.com
下载附件