Commit Graph

10 Commits

Author SHA1 Message Date
Shailesh Padave 396a09a4fd refactor(metrics): make Monitors self-contained; MetricsCollector becomes a registry wiring bean (#1059)
* refactor(metrics): Monitors owns its CompositeMeterRegistry; MetricsCollector wires registries in

Previously Monitors pulled its registry from MetricsCollector, creating an
awkward dependency from conductor-core → conductor-metrics. Now Monitors
owns the CompositeMeterRegistry directly and exposes addMeterRegistry() /
getRegistry(). MetricsCollector (contribs) becomes a thin Spring wiring
component that calls Monitors.addMeterRegistry() on startup.

- Removes conductor-core → conductor-metrics build dependency (cycle-free)
- Adds conductor-metrics → conductor-core build dependency
- Adds getGauge() / getDistributionSummary() aliases for callers using the
  'get' naming convention
- Deprecates MetricsCollector.getMeterRegistry() in favour of
  Monitors.getRegistry()

* Applied spotless

* test(metrics): add MonitorsTest covering registry ownership and meter APIs

Verifies addMeterRegistry(), getRegistry(), counter/timer/gauge identity
caching, getGauge/getDistributionSummary aliases, and tag isolation.

* test(metrics): add Spring integration test verifying MetricsCollector wires registries into Monitors

Boots a minimal Spring context with a SimpleMeterRegistry, confirms that
counters/timers/gauges recorded via Monitors are visible in the
Spring-wired registry after MetricsCollector initialises.

* Applied spotless

* refactor(metrics): retire conductor-metrics module, move classes to core/server

MetricsCollector moves to core alongside Monitors — they are companion classes
(Monitors owns the registry, MetricsCollector wires Spring-managed registries in).
Registry-specific configs (Logging, CloudWatch, AzureMonitor) move to server where
they belong as deployment-level concerns. No Java code imported from the old
contribs.metrics package, so this is a pure relocation with no call-site changes.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(metrics): delete retired conductor-metrics module directory

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(metrics): add micrometer-registry-prometheus to server and server-lite

Without this dependency Spring Boot cannot create PrometheusMeterRegistry,
so /actuator/prometheus silently returns 404 even though
conductor.metrics-prometheus.enabled=true is set in all default configs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore: simplify metrics comment in server build files

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(build): remove stale :conductor-metrics dep from scheduler-core

The scheduler module (merged from main via #1064) still referenced
:conductor-metrics, which this branch retired. Monitors is already
provided by :conductor-core.

---------
2026-05-21 11:03:21 -07:00
Kowser daccc63552 Change Redis cluster port mapping to work on mac 2026-05-11 16:21:33 -07:00
Naomi Most 1ca7ba5c4e fix: slow GET /metadata/workflow — Redis cache + UI switch to names-and-versions (#844) (#974)
* fix: cache workflow defs in Redis to fix slow GET /metadata/workflow (#844)

getAllWorkflowDefs() previously issued N+1 serial Redis round-trips on
every request (1 SMEMBERS + 1 HGETALL per workflow name), returning full
definition bodies for every version. Under load with hundreds of workflow
definitions this caused multi-second response times; under concurrent
load, requests backed up into tens-of-seconds latency and ~30% HTTP 500s.

Reproduction: 2000 workflows x 5 versions on localhost Redis
- Before: ~3s serial, 5-70s under concurrent load, ~30% HTTP 500s
- After: sub-millisecond (cache hit)

Fix mirrors the existing taskDefCache pattern:
- Add in-memory workflowDefCache (volatile List<WorkflowDef>)
- getAllWorkflowDefs() returns a copy of the cache
- getAllWorkflowDefsLatestVersions() derives latest-per-name from cache
  (O(n) in-memory instead of N Redis round-trips per workflow name)
- Cache is refreshed synchronously after every create/update/remove so
  writes are immediately visible to subsequent reads
- Periodic background refresh uses existing metadataCacheRefreshInterval
  property (default 60s)
- Named executor threads + @PreDestroy shutdown for proper lifecycle
- Fixed pre-existing: taskDefCache was not declared volatile

Tests: two new tests verify the cache is updated after create/remove and
that getAllWorkflowDefsLatestVersions correctly picks highest version.

Closes #844

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* ui : using /metadata/workflow/names-and-versions instead of /metadata/workflow for better efficency

* test: add updateTime assertion and clean up dead instanceof Map branches

* fix: make workflow def cache opt-in, consolidate cache refresh executors

* fix: defensive copy in getAllWorkflowDefsLatestVersions, clean shutdown, remove dead Array.from branch

* style: spotless formatting fix

* fix: update Cypress intercept and fixture for names-and-versions endpoint

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: kousik.sv <kousikmahadeva.s@freshworks.com>
2026-04-14 14:49:14 -07:00
Uriel Nudelman 6a5a36e51d Add retries for standalone and sentinel Redis commands (#997) 2026-04-11 11:31:23 -07:00
Sofian Belahouel 17743b592b fix(redis): load Redis connection beans when Redis is used for queues 2026-04-10 10:10:34 +02:00
Rajeshwar Agrawal e467e266e3 Restore Redis Sentinel auth and support multiple sentinel addresses in redis-lock (#951)
* Restore Redis Sentinel auth configuration

* Support multiple sentinel addresses in redis-lock configuration

Split CONDUCTOR_REDIS_LOCK_SERVER_ADDRESS on semicolons so that
multiple sentinel endpoints can be provided, matching how the cluster
mode already splits on commas. This improves sentinel HA by allowing
the lock client to discover the master even if one sentinel is down.

Example: redis://sentinel-0:26379;redis://sentinel-1:26379

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Document semicolon-separated sentinel addresses for redis-lock

Add a note to the deployment guide explaining that multiple sentinel
endpoints can be provided using semicolons when serverType is SENTINEL,
improving high availability for the lock client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Fixes failing int test

* reverts test-harness fixes

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-07 15:50:11 -07:00
Miguel Prieto e7659e6ae4 feat: Workflow Message Queue (WMQ) — push messages into running workflows (#982)
CI / build (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / build-ui (push) Has been cancelled
2026-04-06 11:05:35 -07:00
Miguel Prieto f69354a3fc fix(redis-configuration): depend on conductor-redis-api instead of owning jedis interfaces 2026-04-01 12:27:30 -03:00
Miguel Prieto c64e1306b5 feat(redis-api): move JedisCommands and UnifiedJedisCommands to conductor-redis-api 2026-04-01 12:22:39 -03:00
Viren Baraiya 4cb030bf08 Refactor redis and upgrade jedis (#927)
CI / build (push) Has been cancelled
CI / test-harness (push) Has been cancelled
CI / build-ui (push) Has been cancelled
2026-03-28 13:12:39 -07:00