fix/docs-deploy-pythonpath
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Viren Baraiya
|
58b461bf34 | Agent docs (#1406) | ||
|
Kowser
|
4d941c7982 |
fix(persistence): baseline-0 on main flyway so boot survives scheduler-first migration
- mirrors postgres scheduler config (@Qualifier postgresRetryTemplate) - smoke-test fixture gets a mysqlRetryTemplate bean alongside postgresRetryTemplate - scheduler flyway shares the schema (own history table, baselineVersion 0); main flyway must tolerate a non-empty schema the same way or boot depends on bean-init order - postgres: baselineVersion 0 (default 1 would silently skip V1__initial_schema if second) - sqlite: baselineOnMigrate + baselineVersion 0 (had neither) - trade-off: legacy DBs w/ tables but no history now fail on V1 "already exists" instead of the baseline hint; same operator remedy Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Kowser
|
c08d60c8a4 |
fix(scheduler): return 404 from GET /schedules/{name} when the schedule does not exist (#1357)
Co-authored-by: Kowser <kowser.orkes@gmail.com> |
||
|
James Harris
|
8be8833342 |
Publish interfaces (#1107)
* add publisher to /api/metadata and Events * Add publisher to scheduler changes * remove unused code * Move MetadataChangeListener to org.conductoross namespace New listener code should live conductoross-side; the files only landed in com.netflix.conductor.core.listener because they sat next to the legacy Task/WorkflowStatusListener. Moves MetadataChangeListener and MetadataChangeListenerStub to org.conductoross.conductor.core.listener and fixes imports in ConductorCoreConfiguration, MetadataServiceImpl, and MetadataServiceTest. Part of #752. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Naomi Most <naomi.most@orkes.io> |
||
|
Andrew Lai (AJ)
|
7f9adbf8bb |
Fix test-harness failures (#1173)
* Try and fix test-harness failures * Run spotless |
||
|
Viren Baraiya
|
b8006bdf18 |
refactor(ai): move OkHttpClient from @Autowired field to constructor in provider configs (#1136)
* refactor(ai): move OkHttpClient from @Autowired field to constructor in provider configs Replace field-level @Autowired injection of conductorAiHttpClient with constructor-parameter wiring across all 11 AI provider configurations. Spring DI now flows through an explicit @Autowired setter that overrides Lombok's @Data-generated setter, keeping @ConfigurationProperties setter binding intact. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * scheduler consistency * refactor(ai): add setHttpClient to ModelConfiguration interface; rename field to httpClient - Add setHttpClient(OkHttpClient) to ModelConfiguration interface - All 11 provider configs implement @Override setHttpClient replacing the ad-hoc setConductorAiHttpClient setter - Rename field conductorAiHttpClient -> httpClient across all impls - BedrockConfiguration implements the no-op setHttpClient - Interface param name aligned to httpClient Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * formatting * Update AIReasoningEndToEndTest.java * fixes * refactor(ai): consolidate OkHttpClient construction into AIHttpClients factory Introduces AIHttpClients with build(props) and defaultClient() so timeouts, connection pooling, and the retry interceptor are configured in one place. Fallback clients in all providers, LLMHelper, GeminiVertex, and PineconeDB previously duplicated raw 60/600/60 second values and silently dropped the ConnectionPool and RetryInterceptor present in the Spring-managed bean. Also tightens SchedulerResource sort param parsing to guard against blank input and extra whitespace, matching the handling in WorkflowResource. Updates ModelConfiguration javadoc to reflect it is no longer a marker interface. --------- Co-authored-by: Shailesh Padave <shaileshpadave49@gmail.com> |
||
|
Shailesh Padave
|
396a09a4fd |
refactor(metrics): make Monitors self-contained; MetricsCollector becomes a registry wiring bean (#1059)
* refactor(metrics): Monitors owns its CompositeMeterRegistry; MetricsCollector wires registries in Previously Monitors pulled its registry from MetricsCollector, creating an awkward dependency from conductor-core → conductor-metrics. Now Monitors owns the CompositeMeterRegistry directly and exposes addMeterRegistry() / getRegistry(). MetricsCollector (contribs) becomes a thin Spring wiring component that calls Monitors.addMeterRegistry() on startup. - Removes conductor-core → conductor-metrics build dependency (cycle-free) - Adds conductor-metrics → conductor-core build dependency - Adds getGauge() / getDistributionSummary() aliases for callers using the 'get' naming convention - Deprecates MetricsCollector.getMeterRegistry() in favour of Monitors.getRegistry() * Applied spotless * test(metrics): add MonitorsTest covering registry ownership and meter APIs Verifies addMeterRegistry(), getRegistry(), counter/timer/gauge identity caching, getGauge/getDistributionSummary aliases, and tag isolation. * test(metrics): add Spring integration test verifying MetricsCollector wires registries into Monitors Boots a minimal Spring context with a SimpleMeterRegistry, confirms that counters/timers/gauges recorded via Monitors are visible in the Spring-wired registry after MetricsCollector initialises. * Applied spotless * refactor(metrics): retire conductor-metrics module, move classes to core/server MetricsCollector moves to core alongside Monitors — they are companion classes (Monitors owns the registry, MetricsCollector wires Spring-managed registries in). Registry-specific configs (Logging, CloudWatch, AzureMonitor) move to server where they belong as deployment-level concerns. No Java code imported from the old contribs.metrics package, so this is a pure relocation with no call-site changes. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * refactor(metrics): delete retired conductor-metrics module directory Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(metrics): add micrometer-registry-prometheus to server and server-lite Without this dependency Spring Boot cannot create PrometheusMeterRegistry, so /actuator/prometheus silently returns 404 even though conductor.metrics-prometheus.enabled=true is set in all default configs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: simplify metrics comment in server build files Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(build): remove stale :conductor-metrics dep from scheduler-core The scheduler module (merged from main via #1064) still referenced :conductor-metrics, which this branch retired. Monitors is already provided by :conductor-core. --------- |
||
|
Andrew Lai (AJ)
|
6d23b50251 |
Add playwright tests (#1082)
* Add playwright tests
* Update CI with playwright
* Fix types and add typecheck to pre-commit hook
* Fix tests
* - Fix double tooltip from appearing
- Update icon in ConductorInput to be vertically centered and fix the size and spacing of the icon
- Add snapshot tests
* Fix pnpm install error in CI
* Update CI so that if only ui-next files are changed, don't run backend e2e
* Update CI so that when
- ui-next files changed: ui-next-ci.yml (lint + mock e2e) and ui-next-integration-ci.yml (integration) are ran
- backend files changed: ui-next-integration-ci.yml (integration) and ci.yml (backend) are ran
- both ui-next and backend files changed: All tests run
* Improve CI caching
* Revert PostgresSchedulerConfiguration
* fix(scheduler): resolve RetryTemplate bean ambiguity in PostgresSchedulerConfiguration
Add @Qualifier("postgresRetryTemplate") to disambiguate RetryTemplate injection
and rename the test fixture bean to match, fixing the smoke test context failure.
* Add and improve tests
|
||
|
Viren Baraiya
|
31cee3cdf6 | Add scheduler to conductor oss from orkes (#1064) | ||
|
Naomi Most
|
25b7c8f1ea |
feat: add conductor-scheduler-postgres-persistence module (#885)
* feat: add conductor-scheduler module with Orkes-aligned DAO layer Introduces the conductor-scheduler Gradle module with a DAO architecture that mirrors Orkes Conductor's scheduler schema and interfaces, enabling a thin adapter in orkes-conductor to use this as a drop-in replacement. Key design decisions: - Table names match Orkes V117 schema: `scheduler` + `scheduler_execution` (+ archival `workflow_scheduled_executions`), no org_id (single-tenant) - Execution records stored as JSON blobs; `state` and `schedule_name` kept as queryable columns since OSS has no queue infrastructure - SchedulerDAO interface includes `findAllByNames(Set<String>)` matching Orkes' bulk-lookup method signature - New SchedulerCacheDAO interface mirrors Orkes' Redis cache layer pattern; RedisSchedulerDAO implements this (not the full SchedulerDAO), using the same key scheme: WORKFLOW_SCHEDULES hash + WORKFLOW_SCHEDULES_RUNTIME:<name> - WorkflowScheduleExecution gains workflowName, stackTrace, startWorkflowRequest fields to match WorkflowScheduleExecutionModel - WorkflowSchedule gains @JsonAnySetter/@JsonAnyGetter so Orkes-specific fields (e.g. tags) survive JSON round-trips without an OSS→Orkes type dep DAO implementations: - PostgresSchedulerDAO: ON CONFLICT upserts, ANY(?::text[]) for bulk lookup - MySQLSchedulerDAO: ON DUPLICATE KEY UPDATE, dynamic IN(?,?,...) for bulk - RedisSchedulerDAO: SchedulerCacheDAO only (cache layer, not authoritative) All implementations have Testcontainers integration tests. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: extract PostgresSchedulerDAO into scheduler-postgres-persistence module Refactor the conductor-scheduler module to be persistence-agnostic: - Remove PostgresSchedulerDAO, MySQLSchedulerDAO, RedisSchedulerDAO from the scheduler module (DAOs now live in dedicated persistence modules) - WorkflowSchedulerConfiguration no longer creates the SchedulerDAO bean or runs Flyway; it only wires SchedulerService + SchedulerResource via @ConditionalOnBean(SchedulerDAO.class) Add new conductor-scheduler-postgres-persistence module: - PostgresSchedulerDAO (moved + repackaged to ...scheduler.postgres.dao) - PostgresSchedulerConfiguration auto-configures the SchedulerDAO bean when conductor.db.type=postgres AND conductor.scheduler.enabled=true - Flyway migrations for scheduler tables moved here - PostgresSchedulerDAOTest updated to use @TestConfiguration directly Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add conductor-core dep and fix @ConditionalOnExpression for Postgres scheduler config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add scheduler persistence modules to server build + mc-loki deploy config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: disable external metric exporters in MC config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: server build only references scheduler persistence modules present on this branch * chore: remove mc-loki deploy config from scheduler PR branch Deployment config doesn't belong in a DAO module PR. Tracked separately outside the repo. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add AbstractSchedulerDAOTest contract suite (29 tests, all backends) Replaces per-module duplicate test classes with a shared abstract contract test in conductor-scheduler testFixtures. Each persistence module now extends AbstractSchedulerDAOTest and provides only its Spring wiring. New test coverage beyond the original happy-path tests: - JSON round-trip fidelity for all WorkflowSchedule fields (paused, pausedReason, scheduleStartTime/EndTime, runCatchupScheduleInstances, createdBy, updatedBy, description, nextRunTime) - @JsonAnySetter extension fields survive round-trip (Orkes compatibility) - WorkflowScheduleExecution all-field round-trip (workflowName, stackTrace, startWorkflowRequest, reason) - POLLED→EXECUTED/FAILED state transitions incl. FAILED with reason+stackTrace - saveExecutionRecord idempotency (double-save must not duplicate rows) - getPendingExecutionRecordIds drops EXECUTED records after transition - getExecutionRecords with reverse insertion order (verifies ORDER BY, not insertion order) - getExecutionRecords with limit=1 - deleteWorkflowSchedule cascade over multiple executions - deleteWorkflowSchedule on non-existent name does not throw - findAllByNames(null) returns empty map - updateSchedule resets next_run_time when nextRunTime is null (documents behavior) - Volume: 100-schedule getAllSchedules - Concurrency: 10 threads simultaneous upsert same name → exactly 1 row All 29 tests pass against PostgreSQL (Testcontainers). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add service integration + HTTP contract tests for scheduler - AbstractSchedulerServiceIntegrationTest (8 tests): wires real DAO + real SchedulerService + mocked WorkflowService; covers pruning, stale poll cleanup, next-run pointer advancement, timestamp contracts, and concurrent-poll double-fire documentation - PostgresSchedulerServiceIntegrationTest: concrete subclass wired with Testcontainers PostgreSQL - SchedulerResourceHttpTest (15 MockMvc tests): verifies HTTP status codes for all SchedulerResource endpoints using standalone MockMvc + local TestExceptionHandler (no database needed) - conductor-scheduler build.gradle: add conductor-core, mockito-core to testFixtures; add spring-boot-starter-web to testImplementation Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add auto-configuration smoke tests for scheduler persistence modules AbstractSchedulerAutoConfigurationSmokeTest (5 tests via ApplicationContextRunner): - testFullStack_registeredWhenBothPropertiesSet: verifies SchedulerDAO of the expected concrete type is registered, and that SchedulerService + SchedulerResource also appear via WorkflowSchedulerConfiguration - testNoBeansRegistered_whenSchedulerEnabledAbsent - testNoBeansRegistered_whenSchedulerEnabledFalse - testNoSchedulerDAO_whenDbTypeAbsent - testNoSchedulerDAO_whenDbTypeIsWrongBackend PostgresSchedulerAutoConfigurationSmokeTest: concrete subclass using Testcontainers PostgreSQL for the positive path. These tests catch bugs the DAO/service integration tests cannot: typos in @ConditionalOnExpression strings, missing AutoConfiguration.imports entries, and WorkflowSchedulerConfiguration failing to pick up the DAO. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: correct testFixtures deps and scope AbstractSchedulerAutoConfigurationSmokeTest - Add spring-boot-starter-test and jackson-databind to testFixturesImplementation so ApplicationContextRunner and AssertJ compile in the shared testFixtures source set - Remove SchedulerService/SchedulerResource assertions from smoke test (ConditionalOnBean ordering is non-trivial with ApplicationContextRunner; that wiring is covered by the service integration tests) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * docs: update scheduler test plan to reflect full test suite - Rename to "Scheduler Test Plan" (was "SchedulerDAO Contract Test Plan") - Add service integration tests section (8 tests per backend) - Add HTTP layer section (15 MockMvc tests, runs once) - Add auto-configuration smoke tests section (5 tests per backend) - Add case sensitivity + error conditions section (5 new DAO tests) - Update architecture diagram to show all three abstract base classes - Update results table: 226 total tests, all passing - Add "Relationship Between Test Layers" blind-spots matrix - Add "Adding New Tests" guidance for all four test layers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add conductor-scheduler-mysql-persistence module (#860) * docs: update scheduler test plan to reflect full test suite - Rename to "Scheduler Test Plan" (was "SchedulerDAO Contract Test Plan") - Add service integration tests section (8 tests per backend) - Add HTTP layer section (15 MockMvc tests, runs once) - Add auto-configuration smoke tests section (5 tests per backend) - Add case sensitivity + error conditions section (5 new DAO tests) - Update architecture diagram to show all three abstract base classes - Update results table: 226 total tests, all passing - Add "Relationship Between Test Layers" blind-spots matrix - Add "Adding New Tests" guidance for all four test layers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * feat: add conductor-scheduler-mysql-persistence module Add MySQL-backed SchedulerDAO implementation as a dedicated Gradle module, following the same pattern as conductor-scheduler-postgres-persistence. - MySQLSchedulerDAO implements SchedulerDAO using ON DUPLICATE KEY UPDATE and json_extract() for MySQL-compatible SQL - MySQLSchedulerConfiguration auto-configures when conductor.db.type=mysql AND conductor.scheduler.enabled=true, running Flyway migrations for scheduler tables - MySQLSchedulerDAOTest uses Testcontainers for full integration coverage Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: add conductor-core dep and fix @ConditionalOnExpression for MySQL scheduler config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: extend AbstractSchedulerDAOTest for MySQL — 29 tests, no logic duplication Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: expand AbstractSchedulerDAOTest to 34 tests + fix MySQL case sensitivity New test cases (apply to all three backends automatically): - testFindAllSchedules_caseSensitive: verifies workflow_name lookup is case-sensitive (requires utf8mb4_bin on MySQL) - testFindAllByNames_largeSet: 50 rows + 50 non-existent names; result contains exactly the 50 existing rows - testGetNextRunTime_nonExistentSchedule_returnsMinusOne - testSetNextRunTime_nonExistentSchedule_doesNotThrow (verifies UPDATE on missing row silently no-ops) - testGetExecutionRecords_nonExistentSchedule_returnsEmpty MySQL migration fix: add CHARACTER SET utf8mb4 COLLATE utf8mb4_bin to workflow_name column so case-sensitive lookups work the same as Postgres and SQLite. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add MySQLSchedulerServiceIntegrationTest Concrete subclass wired with Testcontainers MySQL 8.0. Inherits all 8 service integration tests from AbstractSchedulerServiceIntegrationTest. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * test: add MySQLSchedulerAutoConfigurationSmokeTest Concrete subclass using Testcontainers MySQL 8.0 for the positive path. Inherits all 5 smoke tests from AbstractSchedulerAutoConfigurationSmokeTest. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> * style: apply spotless formatting to conductor-scheduler module Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix remaining spotless violation in AbstractSchedulerDAOTest Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * style: fix spotless violations in mysql and postgres scheduler modules Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |