-
[OPIK-2769] [Python BE] Add Optimization Studio infrastructure and LLM API keys support (#4290)
frostbyte_neo 发布于
2025-12-02 11:47:45 +00:00 - [OPIK-2769] [Python BE] Add Optimization Studio infrastructure and LLM API keys support
Add comprehensive Python backend support for Optimization Studio:
Core Infrastructure:
- RQ (Redis Queue) worker for background optimization jobs
- Integration with opik_optimizer library (GEPA, Evolutionary, Hierarchical Reflective)
- Job processing with automatic status management (INITIALIZED → RUNNING → COMPLETED/ERROR)
- Redis client with REDIS_URL support for flexible deployment
Optimizer Job Processing:
- Dataset validation (existence and non-empty checks)
- Prompt building from Studio configuration
- Metric factory supporting: equals, levenshtein_ratio, geval, json_schema_validator
- Optimizer factory for all supported algorithms
- Automatic result tracking and status updates
Code Refactoring (58% reduction):
- Extracted Optimization Studio code into dedicated studio/ module
- Created custom exceptions for better error handling
- Implemented status manager with automatic lifecycle management
- Added data classes (OptimizationJobContext, OptimizationConfig, OptimizationResult)
- Extracted helper functions for improved testability
- Implemented factory pattern for metrics and optimizers
- Added context manager for bulletproof status updates
Studio Module Structure:
- config.py - Configuration constants and environment variables
- metrics.py - Metric factory and builders
- optimizers.py - Optimizer factory
- types.py - Data classes with Pydantic validation
- exceptions.py - Custom exception hierarchy
- status_manager.py - Status management and lifecycle
- helpers.py - Reusable helper functions
Deployment Configuration:
- Docker Compose: PYTHON_BACKEND_PULL_POLICY environment variable
- Docker Compose: LLM API keys support (OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY)
- Helm: Optional secretRefs for Kubernetes secrets
- Helm: ExternalSecret support for AWS/GCP/Azure secret managers
LLM API Keys Configuration (3 methods):
- Local development: Environment variables (simple, quick)
- Docker Compose override: Persistent configuration file
- Kubernetes: ExternalSecret/manual secrets (production-ready)
Documentation:
- LLM_API_KEYS_CONFIGURATION.md - Complete guide for all deployment scenarios
- SECRETREFS_VALIDATION.md - Helm template validation guide
- REFACTORING_SUMMARY.md - Detailed refactoring documentation
Security:
- Secrets stored in external secret managers (AWS Secrets Manager, etc.)
- RBAC-controlled access
- Automatic secret rotation with ExternalSecret Operator
- No secrets in Helm values or Git
Impact:
- Reduced optimizer.py from 324 to 136 lines (58% reduction)
- Organized code into 8 focused modules
- All functions independently testable
- Easy to extend with new metrics/optimizers
This commit includes all Python backend changes for the Optimization Studio feature.
-
Update Helm documentation
-
Remove documentation files
Removing detailed documentation files as they are not needed for the PR review.
- Security: Remove specific API key logging
Do not log which specific API keys are available or their names.
Only log high-level information: whether at least one key is present.This prevents leaking information about configured LLM providers.
- Revision 4: Fix RQ contract test to use proper job message format
Updated test_rq_api_with_fakeredis_works to match the refactored OptimizationJobContext
message format with required fields: optimization_id, workspace_name, and config.-
addressing PR requests
-
Revision 6: Refactor factories to raise specific exceptions and cleanup helpers
Co-authored-by: CometActions github-actions@comet.com
下载附件