fix/docs-deploy-pythonpath
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Viren Baraiya
|
58b461bf34 | Agent docs (#1406) | ||
|
Viren Baraiya
|
b074897f39 | Remove sqlite vector (#1455) | ||
|
nicholascole
|
1e0072f97d | refactor(ai): consolidate agent integration configuration | ||
|
Viren Baraiya
|
0110401072 | Resolve #1286: Support conductor agents in the AGENT tasks (#1288) | ||
|
Viren Baraiya
|
cde7b8e934 |
feat: Durable A2A (Agent2Agent) protocol — client, server, observability (#1195)
* feat(a2a): durable A2A protocol — client, server, observability
Add Agent2Agent (A2A) protocol support to the AI module, in both directions.
Client: CALL_AGENT / GET_AGENT_CARD / CANCEL_AGENT_TASK system tasks call remote
A2A agents over JSON-RPC with poll, streaming (SSE), and push modes. Durable by
design — deterministic messageId, state in the execution (not a thread), liveness
guards, and a push backstop. SSRF-guarded (IPv6 ULA + cloud-metadata; redirects off).
Server: expose any workflow as an A2A agent (one agent per workflow), idempotent
message/send -> startWorkflow (RETURN_EXISTING), tasks/get / tasks/cancel, and
multi-turn resume (a follow-up message/send completes the paused HUMAN/WAIT task
instead of starting a duplicate).
Observability: Micrometer counters via the shared Monitors registry and MDC
correlation keys across the A2A code paths.
Gated by conductor.integrations.ai.enabled (client) and conductor.a2a.server.enabled
(server).
* test(a2a): unit, real-agent interop, and real-engine durability tests
- Unit/wire tests (MockWebServer), embedded-agent e2e, push callback, server
JSON-RPC dispatch + multi-turn resume, mapper/worker, and observability.
- A2ASdkInteropTest: drives the client against the official a2a-sdk reference
agent launched as a subprocess (discovery, send, poll, streaming, message-mode);
self-skips when no Python with a2a-sdk is available.
- A2ADurableEngineEndToEndTest (test-harness): CALL_AGENT through the real decider
+ AsyncSystemTaskExecutor + Redis, proving crash/restart resume from persistence.
* docs(a2a): integration guide, examples, UI task types, design notes
- docs/devguide/ai/a2a-integration.md (AI Cookbook nav) with worked examples:
call/expose, multi-turn resume request/response, and push end-to-end.
- ai/examples: call / get-card / server + streaming / push / multi-turn / cancel,
indexed in the examples README; runnable interop + durable demos under test
resources.
- ui-next: register CALL_AGENT / GET_AGENT_CARD / CANCEL_AGENT_TASK task types.
- design/a2a: protocol study, durability proposal, and server design notes.
Note: a CI step to install a2a-sdk (so A2ASdkInteropTest runs in the build job)
is left as a follow-up — it needs a token with the `workflow` scope to land.
* refactor(a2a): move inbound server auth to enterprise (OSS open by default)
The A2A server's optional shared-secret api-key (conductor.a2a.server.api-key)
is removed from OSS: the server is now open by default, matching OSS Conductor
REST. Inbound authentication (API keys, OAuth/OIDC, mTLS, per-skill scopes,
signed Agent Cards) belongs to the enterprise build; front OSS with a
gateway/firewall.
Unchanged and kept in OSS as safe-by-default guards: client SSRF protection,
push-callback token auth, cross-agent execution isolation, and client→remote
per-call auth headers.
Removes authorized()/apiKey + the two api-key tests; docs and design notes
updated.
* docs(a2a): error-handling/troubleshooting, multi-agent + client multi-turn + LLM-pick examples
Docs: add an error-handling & retries section (FAILED vs FAILED_WITH_TERMINAL_ERROR
mapping for HTTP/JSON-RPC/SSRF/liveness) + a troubleshooting table; flesh out the
client multi-turn section with a worked SWITCH-on-input-required snippet; add an
"orchestrating multiple agents" subsection.
Examples (validated by ExampleWorkflowValidationTest):
- 27-a2a-multi-agent: FORK_JOIN calling agents in parallel → JOIN.
- 28-a2a-llm-pick-skill: GET_AGENT_CARD → LLM_CHAT_COMPLETE → CALL_AGENT.
- 29-a2a-client-multi-turn: SWITCH on input-required, re-call with same context/taskId.
* refactor(a2a)!: rename CALL_AGENT→AGENT, CANCEL_AGENT_TASK→CANCEL_AGENT + add agentType
Generalize the agent task types ahead of multi-runtime support:
- CALL_AGENT → AGENT (class CallAgentTask → AgentTask), CANCEL_AGENT_TASK →
CANCEL_AGENT. GET_AGENT_CARD kept (discovery/"Agent Card" is A2A-specific).
- New input field `agentType` (default "a2a") on all three tasks — the extension
point for native runtimes (langgraph, openai, …). Unknown values are rejected
with a clear error; only "a2a" is implemented today.
Updated across enum, handlers, request models, mapper, callback, tests,
test-harness, examples, docs, design notes, and the ui-next task registration.
BREAKING CHANGE: workflows using type "CALL_AGENT"/"CANCEL_AGENT_TASK" must switch
to "AGENT"/"CANCEL_AGENT".
* docs(a2a): clean up post-rename residue (grammar, example names, phrasing)
Fix 'A AGENT'→'An AGENT' grammar (A2AMetrics, design), reword example descriptions and a doc sentence left awkward by the mechanical rename, and update legacy example workflow names (a2a_call_agent_* → a2a_agent_*).
* feat(a2a): server-side message/stream (SSE)
Exposed workflow-agents now support A2A message/stream. The endpoint returns an
SSE stream driven off a dedicated daemon pool: the initial Task, status-update
events as the workflow's A2A state changes, artifact-update events as output is
produced, and a final status-update at a terminal/input-required state (or when
the stream window elapses). The agent advertises capabilities.streaming=true.
A web-agnostic A2AStreamSink keeps SseEmitter out of A2AWorkflowAgent so the
stream driver stays unit-testable. Config: conductor.a2a.server.stream-poll-
interval-millis (500), stream-max-duration-seconds (300).
Tests: a mocked unit test asserting the task → artifact-update → final
status-update sequence, and a real-HTTP loopback (our client message/stream
against the server's SSE) aggregating to a completed task with artifacts.
Docs + design notes updated (streaming moves from follow-up to shipped).
* Update A2ADurableEngineEndToEndTest.java
* Update A2ADurableEngineEndToEndTest.java
* refactor(a2a): simplify client/server task code (no behavior change)
- AgentTask: replace hand-rolled isBlank/firstNonBlank/stripTrailingSlash/
asInt/asLong helpers with commons-lang3 StringUtils + Number-only reads;
reuse parseRequest().getPollIntervalSeconds() for pollInterval; dedup
stateOf via A2AResults; fix the stale {@link #isAsyncComplete} javadoc to
describe the real backstop-poll mechanism
- A2ACallbackResource: drop the deprecated query-param token path (untested,
no legacy clients on a feature branch); StringUtils for isBlank
- A2AServerResource/A2AWorkflowAgent: delete the duplicate basePath()/URL
building (delegate to the agent's agentUrl/new agentCardUrl); drop the
no-op `id == null ? null : id` ternaries
- remove dead fields A2AMessage.referenceTaskIds and PushNotificationConfig.id
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(a2a): add mermaid use-case diagrams (client & server directions)
Diagrams rendering both A2A directions — Conductor as client (workflow calls
remote agents via the AGENT task) and as server (a workflow exposed as an A2A
agent):
- docs/devguide/ai/a2a-integration.md: both-directions overview flowchart,
AGENT lifecycle sequence (poll/push/stream), FORK_JOIN→JOIN multi-agent
fan-out, and the server message/send→workflow→tasks/get sequence
- design/a2a/08: overview flowchart + a sequence per direction (A/B)
- design/a2a/10: server sequence emphasizing the streaming and multi-turn
resume paths
Rendered via pymdownx.superfences (mkdocs) and natively on GitHub.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(ai)!: rename package ai.models -> ai.model (singular)
Align the AI module with conductor-oss package conventions: every model
package in the codebase (com.netflix.conductor.common.model, core.model,
grpc/proto/model, ai.a2a.model) is singular `model` — `ai.models` was the
only plural `models` package in the entire repo, and sat one letter away
from the singular ai.a2a.model, which read as an inconsistency.
Pure mechanical rename: git mv of the package dir (history preserved) plus
package-decl/import/FQN updates across the module (main + test) and one doc
reference in ai/CONTRIBUTING.md. No type or behavior changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix /a2a endpoint not being reachable
* style: spotlessApply
* fix and upgrade version
* refactor for agentspan
* feat: add SQLite (sqlite-vec) vector database backend
Add an embedded, zero-infrastructure VectorDB backend implemented with the
sqlite-vec extension, alongside the existing pgvector/MongoDB/Pinecone backends.
- SqliteVectorDB stores embeddings in a vec0 virtual table and runs KNN via the
sqlite-vec MATCH operator (l2/cosine/l1).
- The native vec0 binaries (linux/macos/windows) are downloaded + checksum-verified
at build time (downloadSqliteVec) and bundled as jar resources; SqliteVecExtensions
extracts the right one at runtime.
- SqliteVectorDBAutoConfiguration registers a "default" instance automatically when
conductor.db.type=sqlite and conductor.integrations.ai.enabled=true, so the
all-in-one SQLite server gets a working vector store with no external dependency.
- Adds unit tests, a real end-to-end round-trip test, a RAG example
(30-rag-sqlite-vec.json), and documentation.
Note: a dedicated Linux e2e CI job (.github/workflows/ci.yml) is intentionally not
included here because the push token lacks the GitHub `workflow` scope; it must be
committed separately with a workflow-scoped token.
* test: enable workflow execution lock in integration profile to fix flaky specs
The integration-test profile disabled conductor.app.workflow-execution-lock-enabled
(since 2021), so acquireLock() was a no-op and the background WorkflowSweeper could
decide a workflow concurrently with specs' manual sweep()/asyncSystemTaskExecutor
calls. Two simultaneous decides could schedule duplicate tasks, intermittently
failing exact task-count assertions (FailureWorkflowSpec, and
HierarchicalForkJoinSubworkflowRetrySpec).
Enable the lock (matching production defaults; local_only lock is already configured)
so decides are serialized. Full :conductor-test-harness:test passes (252 tests).
* fix(build): wire downloadSqliteVec as a resource source so all consumers depend on it
The generated sqlite-vec resource directory is consumed by sourcesJar (and other
resource consumers), but only processResources depended on downloadSqliteVec. Gradle's
task-dependency validation (run as part of `build`) failed sourcesJar with an implicit
dependency error. Register the task provider itself as the resource srcDir so every
consumer depends on it automatically.
* fix(ai): harden sqlite-vec backend after review
Follow-up fixes from a deep review of the SQLite/sqlite-vec backend:
- SqliteVectorDB.upsertEmbeddings: restore autoCommit(true) in a finally block
so connections are never returned to the Hikari pool in manual-commit mode.
- SqliteVectorDB.getSqliteClient: validate the extension path against an
allowlist before interpolating it into load_extension(...).
- SqliteVectorDBAutoConfiguration.resolveDbPath: strip JDBC URL query params
(e.g. ?busy_timeout=15000&journal_mode=WAL) so the derived *_vectordb.db path
is a valid filename.
- SqliteVecExtensions: re-extract the bundled binary if the cached temp file was
deleted between calls (stale-cache guard).
- VectorDBProvider: warn when an auto-registered default is skipped because an
explicit instance with the same name already exists.
Adds SqliteVectorDBAutoConfigurationTest (resolveDbPath cases incl. query
params), SqliteVecExtensionsTest (platform detection), and a rollback/
autocommit-restore test in SqliteVectorDBTest.
* feat(ui-next): config forms for all AI / agentic task types
Wire up the workflow-editor task form dispatcher, Add-Task menu, icons, task
generators and JSON schemas for every AI task type exposed by the `ai/` module,
so each gets a real config form instead of the generic JSON editor.
- New bespoke forms: AGENT, GET_AGENT_CARD, CANCEL_AGENT, LIST_MCP_TOOLS,
CALL_MCP_TOOL, GENERATE_IMAGE, GENERATE_AUDIO, GENERATE_VIDEO, GENERATE_PDF,
LLM_SEARCH_EMBEDDINGS.
- Wire the existing-but-unwired LLM forms (chat/text complete, embeddings,
index text/document, search index, get document) into TaskFormContent.
- LLM_CHAT_COMPLETE: OSS plain "Instructions" textarea (enterprise prompt-name
picker remains pluggable via the plugin registry).
- A2A forms: agentType as a fixed A2A/Conductor radio; multi-row headers via the
shared HTTP headers editor; pushNotification is a boolean toggle.
- CALL_MCP_TOOL arguments accept inline JSON or a ${variable} reference.
- Media-gen + search-embeddings reuse the field-driven LLMFormFields for
provider/model/vectorDB/embedding selection (server-backed dropdowns).
- Full field coverage vs the backend input models; numeric fields use coerceTo
so the workflow JSON stores real numbers while still accepting ${variables}.
- A2A tasks added to the Agentic Orchestration quick-add section.
* fix(ui-next): address PR review on AI task forms
- Schemas.ts: add the new AI/agentic task types to genericSchema's type enum so
the code (JSON) editor no longer warns "type is not one of ..." for AGENT,
GENERATE_*, *_MCP_*, LLM_SEARCH_EMBEDDINGS, CHUNK_TEXT, LIST_FILES, PARSE_DOCUMENT.
- AgentTaskForm: fix spacing so the "Push backstop poll" field's floating label
no longer collides with the push-notification toggle (group switches, separate
the input row).
- LLMSearchEmbeddingsTaskForm: add spacing so the helper text no longer collides
with the Query field's floating label.
- QuickAddMenu: fill the two empty Agentic Orchestration slots (Call MCP Tool,
Generate Image) for a clean two-row grid.
* fix(ui-next): snug node height for AI/agentic task cards
AI task cards render header-only content (no custom body), but fell through to
the DEFAULT node height (100px), leaving empty space below the label on the
canvas. Size the AI/agentic header-only task family to 80px (matching the
header-only FORK_JOIN node) so the cards fit their content.
---------
Co-authored-by: Nicholas Cole <68611647+NicholasDCole@users.noreply.github.com>
Co-authored-by: nicholascole <nicholas.colesd@gmail.com>
|
||
|
Viren Baraiya
|
c195e9fe05 | Remove Spring AI provider SDKs and add native tools (#1046) | ||
|
Viren Baraiya
|
08cb977208 |
feat: Add GENERATE_PDF system task for markdown-to-PDF conversion (#818)
* feat: Add GENERATE_PDF system task for markdown-to-PDF conversion |
||
|
Viren Baraiya
|
bd4d7c757e |
Vector DB Fixes and Video Generation Support (#757)
Vector DB Fixes and Video Generation Support |