-
[OPIK-2175] [BE] Add database migrations to dev-runner script (#3479)
frostbyte_neo 发布于
2025-10-01 06:44:27 +00:00 - [OPIK-2175] Add database migrations support to dev-runner script
- Add run_db_migrations() function that executes both MySQL and ClickHouse migrations
- Integrate migrations into start_backend() to run automatically before backend starts
- Add --migrate command line option for independent migration execution
- Update help documentation and restart step descriptions
- Ensure migrations run for all backend start paths (start, restart, default)
- Reuse existing JAR file discovery logic and error handling patterns
- Revision 2: Ensure infrastructure starts before running migrations
- Add start_infrastructure call to run_db_migrations function
- Prevents migration failures when --migrate is run independently
- Ensures MySQL and ClickHouse containers are running before migration attempts
- Fixes issue where migrations would fail without infrastructure running
- Revision 3: Fix ClickHouse migrations by setting ANALYTICS_DB_DATABASE_NAME
- Export ANALYTICS_DB_DATABASE_NAME=opik before running ClickHouse migrations
- Fixes Liquibase parameter substitution in migration files like 000014_add_thread_id_to_traces.sql
- Ensures ${ANALYTICS_DB_DATABASE_NAME} placeholder is properly replaced with 'opik'
- Resolves ClickHouse migration failures due to missing environment variable
- Revision 4: Fix MySQL migrations by setting STATE_DB_DATABASE_NAME
- Export STATE_DB_DATABASE_NAME=opik before running MySQL migrations
- Fixes Liquibase parameter substitution in MySQL migration files
- Resolves 'Access denied for user 'opik'@'%' to database '${STATE_DB_DATABASE_NAME}' error
- Ensures ${STATE_DB_DATABASE_NAME} placeholder is properly replaced with 'opik'
- Both MySQL and ClickHouse migrations now have proper environment variables set
- Revision 5: Fix ClickHouse database connection for migrations
- Export ANALYTICS_DB_MIGRATIONS_URL=jdbc:clickhouse://localhost:8123
- Resolves issue where migrations were trying to access wrong database
- Both database name parameter and connection URL are now properly configured
- Revision 6: Add robust error handling for database migrations
- Provide clear recovery instructions if all attempts fail
- Warn users about data loss when suggesting Docker volume cleanup
- Revision 7: Improve migration recovery message with current command context
- Capture original command in ORIGINAL_COMMAND variable at script start
- Update print_migrations_recovery_message() to show the exact command to retry
- Provide more contextual recovery instructions for users
- Makes it easier to retry the exact same operation after volume cleanup
Example output:
3. Continue your current flow: ./scripts/dev-runner.sh --migrate
3. Continue your current flow: ./scripts/dev-runner.sh --restart --debugThis helps users understand exactly what command to run after recovery.
- Revision 8: Fix Docker volume cleanup instructions for effective recovery
- Replace ineffective 'docker volume prune -f' with comprehensive cleanup
- Revision 9: Make recovery instructions dynamic and improve Docker cleanup
- Use $0 instead of hardcoded script path for stop command
- Makes recovery instructions work regardless of how script is invoked
- Use 'docker volume prune -a -f' for more aggressive volume cleanup
- The '-a' flag removes all unused volumes, not just anonymous ones
Benefits:
- Works when script is called from different directories
- Works when script is symlinked or renamed
- More effective volume cleanup with '-a' flag
- Consistent dynamic path resolution throughout recovery message
- Revision 10: Optimize Maven build for faster backend compilation
- Add parallel build with -T 1C (1 thread per CPU core)
- Skip JavaDoc generation with -Dmaven.javadoc.skip=true
- Skip source JAR generation with -Dmaven.source.skip=true
- Keep clean install and main JAR generation for proper builds
- Maintain test skipping for development speed
-
Manual revision
-
Update scripts/dev-runner.sh
Co-authored-by: Copilot 175728472+Copilot@users.noreply.github.com
- Update scripts/dev-runner.sh
Co-authored-by: Copilot 175728472+Copilot@users.noreply.github.com
- Fix recovery command argument preservation
-
Change ORIGINAL_COMMAND from '-- git commit -m "Fix recovery command argument preservation
-
Change ORIGINAL_COMMAND from '$0 $*' to '$0 "$@"'
-
Ensures proper argument boundary preservation when arguments contain spaces
-
Fixes recovery command functionality for complex invocations
Addresses PR comment #4"' to '-- "git commit -m "Fix recovery command argument preservation
- Change ORIGINAL_COMMAND from '$0 $*' to '$0 "$@"'
- Ensures proper argument boundary preservation when arguments contain spaces
- Fixes recovery command functionality for complex invocations
Addresses PR comment #4""'
- Ensures proper argument boundary preservation when arguments contain spaces
- Fixes recovery command functionality for complex invocations
Addresses PR comment #4
- Improve recovery command formatting
-
Change from '-- "git commit -m "Improve recovery command formatting
-
Change from '$0 "$@"' to '$0 $@' to avoid extra quotes in output
-
Preserves arguments without adding unnecessary quotation marks
-
Recovery command now shows: ./scripts/dev-runner.sh --migrate
-
Instead of: ./scripts/dev-runner.sh "--migrate"
Addresses PR comment #4 - improved formatting""' to '-- git commit -m "Improve recovery command formatting
- Change from '$0 "$@"' to '$0 $@' to avoid extra quotes in output
- Preserves arguments without adding unnecessary quotation marks
- Recovery command now shows: ./scripts/dev-runner.sh --migrate
- Instead of: ./scripts/dev-runner.sh "--migrate"
Addresses PR comment #4 - improved formatting"' to avoid extra quotes in output
- Preserves arguments without adding unnecessary quotation marks
- Recovery command now shows: ./scripts/dev-runner.sh --migrate
- Instead of: ./scripts/dev-runner.sh "--migrate"
Addresses PR comment #4 - improved formatting
Co-authored-by: Copilot 175728472+Copilot@users.noreply.github.com
下载附件