Andrew Lai (AJ)
9bfa635d4a
* Add frontend code coverage * Revert LLM task forms to correct version and add tests * Fix CI heap out of memory * Add OPENAI_API_KEY to env for test * - Update slack icon for coverage report - Make tests more stable * Set slack icon back to hidden based on flag * - Fix coverage report - Try and improve CI speed * Add better placeholder and helper text for Chat Complete task * Make instructions take priority over system messages if instructions are provided * Improve LLM chat complete UX so it's less confusing - Enterprise you either select a prompt OR provide instructions * Rename field to Prompt Template instead of AI Prompt and increase spacing from label
Conductor Docker Builds
Pre-built docker images
Conductor server with support for the following backend:
- Redis
- Postgres
- Mysql
- Cassandra
Docker File for Server and UI
Docker Image Source for Server with UI
Configuration Guide for Conductor Server
Conductor uses a persistent store for managing state.
The choice of backend is quite flexible and can be configured at runtime using conductor.db.type property.
Refer to the table below for various supported backend and required configurations to enable each of them.
Important
See config.properties for the required properties for each of the backends.
Backend Property postgres conductor.db.type=postgres redis conductor.db.type=redis_standalone mysql conductor.db.type=mysql cassandra conductor.db.type=cassandra
Conductor is using Elasticsearch or OpenSearch for indexing the workflow data. Currently, Elasticsearch 7 and OpenSearch 2.x/3.x are supported.
We welcome community contributions for other indexing backends.
Note: Docker images use Elasticsearch 7 by default. Elasticsearch 6 and OpenSearch 1.x are deprecated.
Helm Charts
TODO: Link to the helm charts
Run Docker Compose Locally
Use the docker-compose to bring up the local conductor server.
| Docker Compose | Description |
|---|---|
| docker-compose.yaml | Redis + Elasticsearch 7 |
| docker-compose-postgres.yaml | Postgres + Elasticsearch 7 |
| docker-compose-mysql.yaml | Mysql + Elasticsearch 7 |
| docker-compose-redis-os.yaml | Redis + OpenSearch 2.x (legacy - use os2) |
| docker-compose-redis-os2.yaml | Redis + OpenSearch 2.x |
| docker-compose-redis-os3.yaml | Redis + OpenSearch 3.x |
Network errors during UI build with yarn
It has been observed, that the UI build may fail with an error message like
> [linux/arm64 ui-builder 5/7] RUN yarn install && cp -r node_modules/monaco-editor public/ && yarn build:
269.9 at Object.onceWrapper (node:events:633:28)
269.9 at TLSSocket.emit (node:events:531:35)
269.9 at Socket._onTimeout (node:net:590:8)
269.9 at listOnTimeout (node:internal/timers:573:17)
269.9 at process.processTimers (node:internal/timers:514:7)
269.9 info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
281.2 info There appears to be trouble with your network connection. Retrying...
313.5 info There appears to be trouble with your network connection. Retrying...
920.3 info There appears to be trouble with your network connection. Retrying...
953.6 info There appears to be trouble with your network connection. Retrying...
This does not necessarily mean, that the network is unavailable, but can be caused by too high latency, as well. yarn accepts the option --network-timeout <#ms> to set a custom timeout in milliseconds.
For passing arguments to yarn, in this Dockerfile the optional build arg YARN_OPTS has been added. This argument will be added to each yarn call.
When using one of the docker-compose-* files, you can set this via the environment variable YARN_OPTS, e.g.:
YARN_OPTS='--network-timeout 10000000' docker compose -f docker-compose.yaml up
When building a Docker image using docker, you must call it like e.g.
docker build --build-arg='YARN_OPTS=--network-timeout 10000000' .. -f server/Dockerfile -t oss-conductor:v3.21.9