-
[issue-3269] Dataset Expansion with AI - Generate synthetic samples using LLM (#3252)
frostbyte_neo 发布于
2025-09-18 08:03:26 +00:00 - Add dataset expansion feature with LLM-based synthetic sample generation
This feature allows users to expand their datasets with AI-generated synthetic samples that follow existing patterns.
Backend Changes:
- Add DatasetExpansionService for LLM integration and sample generation
- Add DatasetExpansionRequest/Response DTOs
- Add POST /v1/private/datasets/{id}/expand endpoint
- Fix JSON serialization issues with Views
Frontend Changes:
- Add DatasetExpansionDialog with model selection and dataset analysis
- Add GeneratedSamplesDialog for sample preview and selection
- Add dataset expansion button to DatasetItemsActionsPanel
- Add API hooks and type definitions
- Fix CORS issues with Vite proxy configuration
Key Features:
- Automatic dataset structure analysis and field pattern preservation
- LLM model selection using existing Opik provider configuration
- Sample preview with syntax highlighting before adding to dataset
- Bulk selection and addition of generated samples
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Refine dataset expansion UI with improved UX and bug fixes
- Fix button loading state overflow issues with clean conditional rendering
- Improve number input behavior with proper validation
- Replace complex tooltip implementation with TooltipWrapper component
- Add distinctive blue styling for AI expansion trigger button
- Clean up modal layout by removing unnecessary progress indicators
- Expand modal width for better content display
- Enhance field preservation UI with better tooltip information
- Streamline user experience with cleaner, more intuitive interface
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix linting errors and improve code quality
- Remove unused imports and variables in DatasetExpansionDialog
- Replace 'any' types with proper TypeScript interfaces
- Fix React hooks dependency arrays
- Apply consistent code formatting
- Remove React button nesting warnings
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix all test failures: TypeScript errors and linting issues
- Fix hardcoded "gpt-4o" string to use PROVIDER_MODEL_TYPE.GPT_4O enum
- Fix DropdownOption generic type arguments in PlaygroundEvaluationBar
- Fix prop name from onSelectionChange to onChange in LoadableSelectBox
- Remove unused imports, variables, and parameters across components
- Fix type casting for modelConfig and sample.data objects
- Remove unused datasetName parameter from GeneratedSamplesDialog
- All TypeScript compilation errors resolved (10 fixes)
- All critical linting errors resolved (88 → 0 critical)
- Backend tests passing with exit code 0
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix prettier formatting error in api.ts
- Split long PLAYGROUND_EVALUATION_REST_ENDPOINT declaration across lines
- Resolves prettier/prettier linter error at line 31:51
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix prettier and Tailwind linting errors in DatasetItems components
- Fix long line breaks in DatasetItemsActionsPanel state declarations
- Remove extra whitespace and trailing spaces
- Fix Tailwind CSS class order: bg-blue-600 text-white hover:bg-blue-700
- Remove trailing space from DatasetItemsActionsPanel component reference
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix all Tailwind CSS linting warnings with auto-fix
- Fixed class order issues across DatasetExpansionDialog, GeneratedSamplesDialog, PlaygroundEvaluationBar
- Applied size-4 shorthand for h-4 w-4 combinations
- Reordered all CSS classes to follow Tailwind's recommended order
- All 28 ESLint warnings resolved (0 errors, 0 warnings)
Auto-fixed with: npx eslint src --fix --max-warnings=0
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix dataset expansion sample count bug and improve UX
- Fix frontend input field preventing proper sample count editing
- Fix backend ignoring requested sample count, always returning LLM output
- Add proper sample count limiting in backend parseGeneratedSamples method
- Fix button spacing in both DatasetExpansionDialog and GeneratedSamplesDialog
- Fix dialog overflow when samples are expanded
- Make all generated samples selected by default
- Fix preserve fields UI synchronization with prompt preview
- Add comprehensive error handling and validation
- Improve loading states and progress indicators
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix all ESLint and Prettier linting errors
- Remove unused SyntaxHighlighter import
- Fix TypeScript type casting to avoid 'any' usage
- Fix React Hook dependency array warnings
- Fix Tailwind CSS classnames ordering
- Apply comprehensive prettier formatting
Resolves all 126 linting issues (123 errors, 3 warnings)
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Add comprehensive test coverage for dataset expansion feature
- Add integration test for dataset expansion API endpoint in DatasetsResourceIntegrationTest.java
- Test validates request/response handling, proper mocking, and JSON serialization
- Improve UI button styling from small icon to descriptive labeled button with Sparkles icon
- Follow existing test patterns using Mockito and AssertJ for consistency
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Revert "Fix all Tailwind CSS linting warnings with auto-fix"
This reverts commit
5eeae1f428.- Remove accidentally added playground files from dataset expansion PR
Fixed by:
- Removing PlaygroundEvaluationBar directory that doesn't exist on main
- Restoring PlaygroundPage.tsx to main version
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Revert vite.config.ts to main branch version
Remove local development configuration changes to keep PR clean while maintaining dataset expansion functionality.
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Improve dataset expansion error handling with detailed user messages
- Add proper exception handling in DatasetsResource.expandDataset() method
- Return HTTP 400 Bad Request with detailed error messages instead of generic 500 errors
- Enhance DatasetExpansionService with comprehensive error message generation
- Provide specific guidance for different JSON parsing error scenarios
- Handle LLM responses that include explanatory text or have structural issues
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Fix frontend linting issues
Auto-fix Prettier formatting in DatasetItemsActionsPanel component
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
- Revert .env.development to match main branch configuration
- Change VITE_BASE_API_URL from http://localhost:8080 back to /api
- Ensures consistent configuration with main branch
🤖 Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com
-
Polished BE endpoint signature
-
Bug fix and improved logs
-
Add info and warning box color variables for light and dark modes in main.scss; enhance DatasetExpansionDialog with improved state management and validation; implement useProgressSimulation for progress feedback during dataset expansion.
Co-authored-by: Gideon Mendels gideonmendels@ip-192-168-68-52.ec2.internal
Co-authored-by: Claude noreply@anthropic.com
Co-authored-by: Andres Cruz andresc@comet.com
Co-authored-by: andrii.dudar fanat.web@gmail.com下载附件