feat(startup): adopt docker-native installer flow and harden startup contracts

Move startup configuration to a Docker Compose installer entrypoint and remove wrapper-based ambiguity, while enforcing consistent secret validation and auth behavior across base, dev, and prod flows.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
0sm0s1z
2026-02-22 16:26:40 -08:00
parent 8d0c71dfd6
commit 620874feee
45 changed files with 1438 additions and 158 deletions
+1
View File
@@ -18,6 +18,7 @@ POSTGRES_PORT=5432
# UI auth settings (required secret)
NEXTAUTH_SECRET=change-me-long-random-nextauth-secret
NEXTAUTH_URL=http://localhost:3000
INITIAL_ADMIN_PASSWORD=change-me-strong-admin-password
# API and browser API URLs
SIRIUS_API_URL=http://sirius-api:9001
+4
View File
@@ -97,6 +97,9 @@ jobs:
- name: Validate Docker Compose Configurations
env:
SIRIUS_API_KEY: ci-placeholder-api-key
POSTGRES_PASSWORD: ci-postgres-password
NEXTAUTH_SECRET: ci-nextauth-secret
INITIAL_ADMIN_PASSWORD: ci-admin-password
run: |
echo "🔍 Validating Docker Compose configurations..."
docker compose config --quiet
@@ -247,6 +250,7 @@ jobs:
- SKIP_ENV_VALIDATION=1
- DATABASE_URL=postgresql://postgres:postgres@sirius-postgres:5432/sirius_test
- NEXTAUTH_SECRET=test-secret-key
- INITIAL_ADMIN_PASSWORD=test-admin-password
- NEXTAUTH_URL=http://localhost:3000
- SIRIUS_API_URL=http://sirius-api:9001
- NEXT_PUBLIC_SIRIUS_API_URL=http://localhost:9001
+1
View File
@@ -382,6 +382,7 @@ jobs:
- SKIP_ENV_VALIDATION=1
- DATABASE_URL=postgresql://postgres:postgres@sirius-postgres:5432/sirius_test
- NEXTAUTH_SECRET=test-secret
- INITIAL_ADMIN_PASSWORD=test-admin-password
- NEXTAUTH_URL=http://localhost:3000
- SIRIUS_API_URL=http://sirius-api:9001
- NEXT_PUBLIC_SIRIUS_API_URL=http://localhost:9001
+3
View File
@@ -48,10 +48,13 @@ build/
out/
# Environment files
.env
.env.local
.env.development.local
.env.test.local
.env.production.local
secrets/
*.secrets
# Database files
*.db
+29
View File
@@ -5,6 +5,35 @@ All notable changes to SiriusScan will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
### Added
- Installer-first startup workflow under `installer/` with interactive and non-interactive configuration generation.
- Optional deployment hardening overlays: `docker-compose.secrets.yaml` and `docker-stack.swarm.yaml`.
- Architecture decision record documenting stateless root key auth model.
### Changed
- Installer startup now uses `docker-compose.installer.yaml` as the canonical entrypoint.
- Root compose and overlays now require security-critical startup variables (`SIRIUS_API_KEY`, `POSTGRES_PASSWORD`, `NEXTAUTH_SECRET`, `INITIAL_ADMIN_PASSWORD`).
- UI startup scripts enforce required auth/seed variables before migrations and seeding.
- API middleware now marks explicit auth mode (`infra_env` vs `valkey`) and uses constant-time root key comparison.
- Documentation updated for installer-first setup and stateless root-key lifecycle operations.
### Fixed
- UI production image build compatibility with stricter secret validation during build stage.
- Integration test expectation for protected API error handling under API-key middleware.
### Migration Notes
- Replace manual `.env` copying with installer:
1. `docker compose -f docker-compose.installer.yaml run --rm sirius-installer`
2. `docker compose up -d` (or include prod/dev overlays)
- Ensure these variables are present for startup:
- `SIRIUS_API_KEY`
- `POSTGRES_PASSWORD`
- `NEXTAUTH_SECRET`
- `INITIAL_ADMIN_PASSWORD`
- Existing `.env` values are preserved by default; use `docker compose -f docker-compose.installer.yaml run --rm sirius-installer --force` when rotating/re-generating secrets.
## [1.0.0] - 2026-02-17
### Added
+89 -19
View File
@@ -15,17 +15,29 @@ Sirius is an open-source comprehensive vulnerability scanner that leverages comm
### ⚡ Quick Start (Current Runtime Requirements)
### ⚡ Startup Command Cheat Sheet
```bash
# 1) Generate/merge required runtime secrets (.env)
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# 2a) Start standard stack
docker compose up -d
# 2b) Start development overlay
docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up -d
# 2c) Optional hardened production overlay
docker compose -f docker-compose.yaml -f docker-compose.prod.yaml up -d
```
```bash
# Clone repository
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
# Create required environment file
cp .env.production.example .env
# Then edit .env and set at minimum:
# - SIRIUS_API_KEY
# - POSTGRES_PASSWORD
# - NEXTAUTH_SECRET
# Generate and validate startup secrets/config (installer-first)
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Start Sirius with release images
docker compose up -d
@@ -36,9 +48,59 @@ open http://localhost:3000
**Important**:
- `SIRIUS_API_KEY` is required for `sirius-ui`, `sirius-api`, and `sirius-engine`.
- For production overlay runs, `POSTGRES_PASSWORD` and `NEXTAUTH_SECRET` are also required.
- `POSTGRES_PASSWORD`, `NEXTAUTH_SECRET`, and `INITIAL_ADMIN_PASSWORD` are required.
- This repository does **not** include `docker-compose.user.yaml`; use `docker-compose.yaml`, `docker-compose.dev.yaml`, and `docker-compose.prod.yaml`.
### 🧭 Using the New Startup System
Sirius now uses an installer-first startup flow. This keeps secrets synchronized across services and removes insecure defaults.
#### 1) First-time local setup (interactive)
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose up -d
```
What happens:
- Reads `.env.production.example`
- Merges existing `.env` values if present
- Generates missing required values:
- `SIRIUS_API_KEY`
- `POSTGRES_PASSWORD`
- `NEXTAUTH_SECRET`
- `INITIAL_ADMIN_PASSWORD`
#### 2) Non-interactive setup (CI/Terraform/user-data)
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer --non-interactive --no-print-secrets
docker compose up -d
```
#### 3) Force secret rotation/regeneration
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer --force
```
#### 4) Development overlay startup
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up -d
```
#### 5) Verify configuration renders
```bash
SIRIUS_API_KEY=test-key \
POSTGRES_PASSWORD=test-pass \
NEXTAUTH_SECRET=test-secret \
INITIAL_ADMIN_PASSWORD=test-admin-pass \
docker compose config --quiet
```
## 🆕 What's New in v1.0.0
### System Monitoring & Observability
@@ -64,6 +126,7 @@ The default configuration provides a complete scanning environment:
```bash
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose up -d
```
@@ -74,24 +137,24 @@ Use live-reload/development mounts for active code work:
```bash
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose -f docker-compose.yaml -f docker-compose.dev.yaml up -d
```
#### Option 3: Production Overlay
Use production-oriented environment settings and validation:
Optional hardened production settings and validation overlay:
```bash
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
cp .env.production.example .env
# edit required values before starting
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose -f docker-compose.yaml -f docker-compose.prod.yaml up -d
```
##### Host Discovery Prerequisites (Prod Overlay)
##### Host Discovery Prerequisites (All Compose Modes)
- `sirius-engine` requires `NET_RAW` capability for ICMP-based fingerprint discovery.
- `sirius-engine` runs with `NET_RAW` capability in base/dev/prod compose configurations for ICMP-based fingerprint discovery.
- Keep `SIRIUS_API_URL` and `API_BASE_URL` pointing to `http://sirius-api:9001` for container-to-container API persistence.
- Use `NEXT_PUBLIC_SIRIUS_API_URL=http://localhost:9001` so browser calls hit the host-exposed API.
@@ -116,12 +179,13 @@ curl http://localhost:3000
curl http://localhost:9001/health
```
### 🔎 Host Discovery Validation (Prod Overlay)
### 🔎 Host Discovery Validation
```bash
# Confirm production overlay renders successfully and includes NET_RAW
# Confirm compose renders successfully and includes NET_RAW
SIRIUS_API_KEY=test-key POSTGRES_PASSWORD=test-pass NEXTAUTH_SECRET=test-secret \
docker compose -f docker-compose.yaml -f docker-compose.prod.yaml config | rg "NET_RAW"
INITIAL_ADMIN_PASSWORD=test-admin-pass \
docker compose -f docker-compose.yaml config | rg "NET_RAW"
# Confirm scanner system template is canonicalized on startup (quick includes fingerprint)
docker compose exec sirius-valkey valkey-cli GET template:quick | rg '"scan_types"'
@@ -275,6 +339,7 @@ Perfect for security professionals and penetration testers:
```bash
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
docker compose up -d
```
@@ -348,6 +413,7 @@ docker compose logs <service> # View service logs
docker system df # Check disk space
# Solutions
docker compose -f docker-compose.installer.yaml run --rm sirius-installer # Ensure required secrets exist in .env
docker compose down && docker compose up -d --build # Fresh restart
docker system prune -f # Clean up space
```
@@ -513,10 +579,14 @@ docker cp sirius-engine:/opt/sirius/ ./sirius-backup/
1. **Change Default Credentials**:
```bash
# Update in .env (used by docker-compose.prod.yaml)
POSTGRES_PASSWORD=your_secure_password
NEXTAUTH_SECRET=your_long_random_secret
SIRIUS_API_KEY=your_long_random_api_key
# Generate secure values with the installer
docker compose -f docker-compose.installer.yaml run --rm sirius-installer --force
# Or set explicit values in .env if needed
# POSTGRES_PASSWORD=your_secure_password
# NEXTAUTH_SECRET=your_long_random_secret
# SIRIUS_API_KEY=your_long_random_api_key
# INITIAL_ADMIN_PASSWORD=your_strong_admin_password
```
2. **Network Security**:
+5 -5
View File
@@ -46,7 +46,7 @@ services:
- NEXT_PUBLIC_SIRIUS_API_URL=http://localhost:9001
- CONTAINER_NAME=sirius-ui
# Coordinated dev API key shared across UI/API/Engine.
- SIRIUS_API_KEY=${SIRIUS_API_KEY:-sirius-dev-api-key-do-not-use-in-production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for development. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
# Performance: increase Node.js heap for faster webpack compilation
- NODE_OPTIONS=--max-old-space-size=4096
# Performance: enable filesystem polling for reliable hot reload on Docker bind mounts
@@ -88,7 +88,7 @@ services:
- API_PORT=9001
- POSTGRES_HOST=sirius-postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for development. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=sirius
- POSTGRES_PORT=5432
- VALKEY_HOST=sirius-valkey
@@ -97,7 +97,7 @@ services:
- LOG_LEVEL=info
- CONTAINER_NAME=sirius-api
# Coordinated dev API key shared across UI/API/Engine.
- SIRIUS_API_KEY=${SIRIUS_API_KEY:-sirius-dev-api-key-do-not-use-in-production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for development. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
sirius-engine:
build:
@@ -136,7 +136,7 @@ services:
- GRPC_AGENT_PORT=50051
- POSTGRES_HOST=sirius-postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for development. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=sirius
- POSTGRES_PORT=5432
- VALKEY_HOST=sirius-valkey
@@ -152,7 +152,7 @@ services:
- ENABLE_SCRIPTING=true
- CONTAINER_NAME=sirius-engine
# Coordinated dev API key shared across UI/API/Engine.
- SIRIUS_API_KEY=${SIRIUS_API_KEY:-sirius-dev-api-key-do-not-use-in-production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for development. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
networks:
default:
+11
View File
@@ -0,0 +1,11 @@
services:
sirius-installer:
image: sirius-installer:local
pull_policy: never
build:
context: .
dockerfile: installer/Dockerfile
working_dir: /workspace
volumes:
- ./:/workspace
command: ["--template", ".env.production.example", "--output", ".env"]
+8 -9
View File
@@ -5,18 +5,19 @@ services:
sirius-postgres:
environment:
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=${POSTGRES_DB:-sirius}
sirius-ui:
environment:
- NODE_ENV=production
- SKIP_ENV_VALIDATION=0
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET:?NEXTAUTH_SECRET is required for production}
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET:?NEXTAUTH_SECRET is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- INITIAL_ADMIN_PASSWORD=${INITIAL_ADMIN_PASSWORD:?INITIAL_ADMIN_PASSWORD is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- NEXTAUTH_URL=${NEXTAUTH_URL:-http://localhost:3000}
- SIRIUS_API_URL=${SIRIUS_API_URL:-http://sirius-api:9001}
- NEXT_PUBLIC_SIRIUS_API_URL=${NEXT_PUBLIC_SIRIUS_API_URL:-http://localhost:9001}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
restart: unless-stopped
sirius-api:
@@ -25,23 +26,21 @@ services:
- CORS_ALLOWED_ORIGINS=${CORS_ALLOWED_ORIGINS:-http://localhost:3000,https://sirius.local}
- POSTGRES_HOST=${POSTGRES_HOST:-sirius-postgres}
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=${POSTGRES_DB:-sirius}
- POSTGRES_PORT=${POSTGRES_PORT:-5432}
- VALKEY_HOST=${VALKEY_HOST:-sirius-valkey}
- VALKEY_PORT=${VALKEY_PORT:-6379}
- RABBITMQ_URL=${RABBITMQ_URL:-amqp://guest:guest@sirius-rabbitmq:5672/}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
restart: unless-stopped
sirius-engine:
cap_add:
- NET_RAW
environment:
- GO_ENV=production
- POSTGRES_HOST=${POSTGRES_HOST:-sirius-postgres}
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=${POSTGRES_DB:-sirius}
- POSTGRES_PORT=${POSTGRES_PORT:-5432}
- VALKEY_HOST=${VALKEY_HOST:-sirius-valkey}
@@ -49,5 +48,5 @@ services:
- RABBITMQ_URL=${RABBITMQ_URL:-amqp://guest:guest@sirius-rabbitmq:5672/}
- API_BASE_URL=${API_BASE_URL:-http://sirius-api:9001}
- SIRIUS_API_URL=${SIRIUS_API_URL:-http://sirius-api:9001}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for production. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
restart: unless-stopped
+30
View File
@@ -0,0 +1,30 @@
services:
sirius-postgres:
environment:
- POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password
secrets:
- postgres_password
sirius-ui:
secrets:
- sirius_api_key
- nextauth_secret
- initial_admin_password
sirius-api:
secrets:
- sirius_api_key
sirius-engine:
secrets:
- sirius_api_key
secrets:
postgres_password:
file: ./secrets/postgres_password.txt
sirius_api_key:
file: ./secrets/sirius_api_key.txt
nextauth_secret:
file: ./secrets/nextauth_secret.txt
initial_admin_password:
file: ./secrets/initial_admin_password.txt
+12 -8
View File
@@ -44,7 +44,7 @@ services:
hostname: sirius-postgres
environment:
POSTGRES_USER: ${POSTGRES_USER:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
POSTGRES_DB: ${POSTGRES_DB:-sirius}
VALKEY_HOST: ${VALKEY_HOST:-sirius-valkey}
VALKEY_PORT: ${VALKEY_PORT:-6379}
@@ -105,15 +105,17 @@ services:
environment:
- NODE_ENV=${NODE_ENV:-production}
- SKIP_ENV_VALIDATION=${SKIP_ENV_VALIDATION:-1}
- DATABASE_URL=${DATABASE_URL:-postgresql://postgres:postgres@sirius-postgres:5432/sirius}
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET:-change-this-secret-in-production-please}
- DATABASE_URL=${DATABASE_URL:-postgresql://postgres:${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}@sirius-postgres:5432/sirius}
# UI auth settings
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET:?NEXTAUTH_SECRET is required for sirius-ui. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- NEXTAUTH_URL=${NEXTAUTH_URL:-http://localhost:3000}
- INITIAL_ADMIN_PASSWORD=${INITIAL_ADMIN_PASSWORD:?INITIAL_ADMIN_PASSWORD is required for sirius-ui. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- SIRIUS_API_URL=${SIRIUS_API_URL:-http://sirius-api:9001}
- NEXT_PUBLIC_SIRIUS_API_URL=${NEXT_PUBLIC_SIRIUS_API_URL:-http://localhost:9001}
- DISCORD_CLIENT_ID=${DISCORD_CLIENT_ID:-dummy_client_id}
- DISCORD_CLIENT_SECRET=${DISCORD_CLIENT_SECRET:-dummy_client_secret}
# API key for authenticated requests to sirius-api (set via root key bootstrap or .env)
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-ui}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-ui. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
depends_on:
sirius-postgres:
condition: service_healthy
@@ -145,14 +147,14 @@ services:
- CORS_ALLOWED_ORIGINS=${CORS_ALLOWED_ORIGINS:-*}
- POSTGRES_HOST=${POSTGRES_HOST:-sirius-postgres}
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for sirius-api. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=${POSTGRES_DB:-sirius}
- POSTGRES_PORT=${POSTGRES_PORT:-5432}
- VALKEY_HOST=${VALKEY_HOST:-sirius-valkey}
- VALKEY_PORT=${VALKEY_PORT:-6379}
- RABBITMQ_URL=${RABBITMQ_URL:-amqp://guest:guest@sirius-rabbitmq:5672/}
- LOG_LEVEL=${LOG_LEVEL:-error}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-api}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-api. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
deploy:
resources:
limits:
@@ -180,6 +182,8 @@ services:
container_name: sirius-engine
hostname: sirius-engine
restart: unless-stopped
cap_add:
- NET_RAW
ports:
- "5174:5174"
- "50051:50051" # Agent gRPC
@@ -189,7 +193,7 @@ services:
- GRPC_AGENT_PORT=${GRPC_AGENT_PORT:-50051}
- POSTGRES_HOST=${POSTGRES_HOST:-sirius-postgres}
- POSTGRES_USER=${POSTGRES_USER:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:-postgres}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required for sirius-engine. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
- POSTGRES_DB=${POSTGRES_DB:-sirius}
- POSTGRES_PORT=${POSTGRES_PORT:-5432}
- VALKEY_HOST=${VALKEY_HOST:-sirius-valkey}
@@ -202,7 +206,7 @@ services:
- ENABLE_SCRIPTING=${ENABLE_SCRIPTING:-true}
- LOG_LEVEL=${LOG_LEVEL:-info}
# API key for authenticated requests to sirius-api
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-engine}
- SIRIUS_API_KEY=${SIRIUS_API_KEY:?SIRIUS_API_KEY is required for sirius-engine. Run installer first using docker compose -f docker-compose.installer.yaml run --rm sirius-installer}
depends_on:
sirius-rabbitmq:
condition: service_healthy
+63
View File
@@ -0,0 +1,63 @@
version: "3.9"
services:
sirius-postgres:
image: postgres:15-alpine
environment:
- POSTGRES_USER=postgres
- POSTGRES_DB=sirius
- POSTGRES_PASSWORD_FILE=/run/secrets/postgres_password
secrets:
- postgres_password
volumes:
- postgres_data:/var/lib/postgresql/data
networks:
- sirius
sirius-valkey:
image: valkey/valkey:latest
networks:
- sirius
sirius-rabbitmq:
image: rabbitmq:3-management
networks:
- sirius
sirius-api:
image: ghcr.io/siriusscan/sirius-api:${IMAGE_TAG:-latest}
environment:
- SIRIUS_API_KEY=${SIRIUS_API_KEY}
networks:
- sirius
sirius-ui:
image: ghcr.io/siriusscan/sirius-ui:${IMAGE_TAG:-latest}
environment:
- SIRIUS_API_KEY=${SIRIUS_API_KEY}
- NEXTAUTH_SECRET=${NEXTAUTH_SECRET}
- INITIAL_ADMIN_PASSWORD=${INITIAL_ADMIN_PASSWORD}
ports:
- "3000:3000"
networks:
- sirius
sirius-engine:
image: ghcr.io/siriusscan/sirius-engine:${IMAGE_TAG:-latest}
cap_add:
- NET_RAW
environment:
- SIRIUS_API_KEY=${SIRIUS_API_KEY}
networks:
- sirius
secrets:
postgres_password:
external: true
volumes:
postgres_data:
networks:
sirius:
driver: overlay
+3 -1
View File
@@ -3,7 +3,7 @@ title: "Documentation Index"
description: "Complete index of all documentation files in the Sirius project, organized by category and purpose"
template: "TEMPLATE.documentation-standard"
version: "1.0.0"
last_updated: "2025-01-03"
last_updated: "2026-02-22"
author: "Development Team"
tags: ["documentation", "index", "reference", "organization"]
categories: ["documentation", "reference"]
@@ -48,6 +48,7 @@ This document provides a complete index of all documentation files in the Sirius
- [README.cicd.md](dev/architecture/README.cicd.md) - CI/CD pipeline architecture and workflows
- [README.go-api-sdk.md](dev/architecture/README.go-api-sdk.md) - Go API SDK architecture, design patterns, and integration guide
- [README.auth-surface-matrix.md](dev/architecture/README.auth-surface-matrix.md) - Authentication and authorization policy matrix across API surfaces
- [ADR.startup-secrets-model.md](dev/architecture/ADR.startup-secrets-model.md) - Architectural decision record for installer-first startup and secrets model
- [ARCHITECTURE.nse-repository-management.md](dev/architecture/ARCHITECTURE.nse-repository-management.md) - NSE repository management architecture
- [README.docker-architecture.md](dev/architecture/README.docker-architecture.md) - Comprehensive Docker setup and container architecture
@@ -78,6 +79,7 @@ This document provides a complete index of all documentation files in the Sirius
- [README.new-project.md](dev/operations/README.new-project.md) - New project development workflow and structure
- [README.tasks.md](dev/operations/README.tasks.md) - Task management system and project tracking
- [startup-secrets-redesign-plan.md](dev-notes/startup-secrets-redesign-plan.md) - Detailed implementation plan for startup and secrets redesign
### Deployment
@@ -0,0 +1,114 @@
---
title: "Startup & Secrets Redesign - Project Plan"
description: "Detailed implementation strategy for installer-first startup, secure secret defaults, and stateless infrastructure API key validation."
template: "TEMPLATE.documentation-standard"
version: "1.0.0"
last_updated: "2026-02-22"
author: "Development Team"
tags: ["project-plan", "startup", "secrets", "installer", "auth", "docker"]
categories: ["development", "planning", "security", "operations"]
difficulty: "advanced"
prerequisites: ["docker", "docker-compose", "go", "nextauth", "prisma"]
related_docs:
- "README.tasks.md"
- "README.new-project.md"
- "README.api-key-operations.md"
- "README.auth-surface-matrix.md"
dependencies: []
llm_context: "high"
search_keywords:
[
"startup redesign",
"secrets management",
"installer",
"sirius_api_key",
"initial_admin_password",
"docker compose hardening",
]
---
# Startup & Secrets Redesign - Project Plan
## Project Overview
**Goal**: Deliver a secure-by-default and low-friction startup experience for Sirius using an installer-first flow, deterministic service key behavior, and strict runtime contracts.
**Scope**:
- Build a first-run installer workflow.
- Remove insecure secret defaults and weak fallbacks.
- Keep root service API key stateless from environment while preserving Valkey-backed user-generated keys.
- Update docs, tests, and CI to match new startup and security expectations.
## Key Outcomes
1. **Installer-first onboarding** for local and automation environments.
2. **No default admin password** in seed/startup workflows.
3. **Deterministic infra key auth** independent of Valkey bootstrap state.
4. **Updated deployment docs** for compose, Terraform, and secrets hardening options.
5. **Aligned validation pipeline** across local tests and CI.
## Technical Strategy
### 1) Installer Productization
- Create an installer module that loads `.env.production.example`, merges existing `.env`, and generates missing required secrets.
- Support interactive and non-interactive modes with output safety options for CI and production automation.
- Preserve backward compatibility by keeping `setup.sh` as a transition wrapper.
### 2) Runtime Contract Hardening
- Require critical auth and seed secrets in compose files.
- Remove fallback values that mask misconfiguration in production.
- Enforce fail-fast behavior in seed and UI runtime config when required secrets are missing.
### 3) Auth Model Clarification
- Validate infra requests statelessly using `SIRIUS_API_KEY` from environment.
- Retain Valkey-backed validation only for dynamic/user-generated API keys.
- Document this split clearly in runbooks and architecture docs.
### 4) Verification and Rollout
- Update tests and CI job environments to provide required vars.
- Add optional secrets overlays (`compose`/`swarm`) for hardened deployments.
- Publish migration notes for existing users.
## Milestones
### Milestone A: Foundations
- Add task tracker and this plan note.
- Record architecture decision updates.
### Milestone B: Installer + Compatibility
- Implement installer command and internals.
- Add compatibility wrapper behavior in `setup.sh`.
### Milestone C: Runtime + Auth Hardening
- Patch compose/env/auth/seed/script behavior.
- Validate stateless root-key and dynamic key paths.
### Milestone D: Docs + Validation Pipeline
- Rewrite onboarding/deployment/runbook docs.
- Update container tests and CI workflows.
### Milestone E: Optional Hardening + Release
- Add secrets overlay files.
- Execute release verification matrix and migration notes.
## Success Criteria
- [ ] Fresh install with Docker only can generate valid config and start successfully.
- [ ] Admin login uses installer-provided/generated password; no default password remains.
- [ ] Root API key rotation works via config change and restart without Valkey state repair.
- [ ] User-generated API keys continue to work for create/list/revoke.
- [ ] CI compose checks and security suites pass with strict required variables.
- [ ] Documentation reflects installer-first and stateless root-key architecture.
## Risks and Mitigations
- **Risk**: Startup regressions due to stricter required env vars.
- **Mitigation**: Provide explicit preflight checks and actionable error messages.
- **Risk**: Existing users may depend on old defaults.
- **Mitigation**: Add compatibility wrapper and migration notes.
- **Risk**: CI breakage from new required vars.
- **Mitigation**: Update CI and test scripts in same change set.
## Notes
This plan intentionally prioritizes secure defaults and deterministic behavior over permissive startup fallbacks. The migration path remains pragmatic by preserving compatibility entrypoints while moving users to the installer model.
+7
View File
@@ -27,6 +27,9 @@ search_keywords:
**New in v1.0.0**: Use the improved environment switching system for easier development.
```bash
# Generate/merge required .env values once (installer-first)
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Modern development workflow (recommended)
./scripts/switch-env.sh dev
@@ -54,6 +57,7 @@ search_keywords:
- No local repository setup required
- All services start with `go run` in development mode
- Live reloading for UI changes
- Uses secrets generated by `docker-compose.installer.yaml` (`.env`) for service auth/DB consistency
### 🔧 Extended Development
@@ -152,6 +156,9 @@ nano docker-compose.override.yaml
### Services Not Starting
```bash
# Ensure installer-generated .env exists and is up to date
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Check container status
./scripts/dev-setup.sh status
@@ -0,0 +1,49 @@
---
title: "ADR: Startup and Secrets Model"
description: "Architectural decision record for installer-first startup and stateless infrastructure API key validation."
template: "TEMPLATE.reference"
llm_context: "high"
categories: ["architecture", "security", "operations"]
tags: ["adr", "startup", "secrets", "apikey", "docker", "installer"]
related_docs:
- "README.auth-surface-matrix.md"
- "README.api-key-operations.md"
- "README.docker-container-deployment.md"
---
# ADR: Startup and Secrets Model
## Status
Accepted
## Context
Sirius startup historically depended on manual environment setup and a bootstrap pattern that could create drift between runtime configuration and persistent key state. The platform also tolerated insecure defaults in startup and seeding paths.
## Decision
1. **Installer-first startup**: a first-run installer is the canonical setup path for generating and merging required secrets/config.
2. **Stateless infrastructure key path**: `SIRIUS_API_KEY` from runtime environment is the authority for service-to-service root authentication.
3. **Dynamic key lifecycle in Valkey**: user-generated API keys continue to be managed in Valkey.
4. **Secure fail-fast runtime**: production startup and seeding flows must fail when required secrets are missing.
## Consequences
### Positive
- Deterministic behavior during restart and key rotation.
- Reduced operational complexity from bootstrap reconciliation state.
- Better first-time user experience with automated secret generation.
- Clear separation between infrastructure auth and user key lifecycle.
### Tradeoffs
- Stricter env requirements may break permissive legacy startup paths until migrated.
- CI/test and deployment scripts must be updated to provide required variables.
## Implementation Notes
- Use `docker-compose.installer.yaml` as the canonical installer entrypoint.
- Ensure compose contracts explicitly require auth-critical variables.
- Update runbooks and deployment documentation in lockstep with runtime changes.
@@ -23,6 +23,30 @@ This checklist is the canonical auth policy map for current Sirius architecture.
- Agent auth: gRPC token model for agent channel identity.
- Current platform model: single UI admin (no multi-user tenant isolation yet).
## Architecture Decision: Startup and Secret Strategy
### Decision Summary
- Startup onboarding is installer-first: generate/merge runtime config before compose startup.
- The infrastructure/root API key is validated statelessly from environment configuration.
- Valkey-backed key validation remains for dynamic, user-generated API keys only.
- Runtime auth and seed flows fail fast when required secrets are missing in production.
### Rationale
- Removes bootstrap drift between persistent key-value state and deployment configuration.
- Reduces first-run friction by automating secret generation and synchronization.
- Improves operational reliability for key rotation and service restarts.
- Aligns local and automated deployments with a single deterministic configuration model.
### Operational Implications
- Health endpoints remain public by explicit policy.
- Non-health API endpoints require `X-API-Key` and validate against either:
- environment root key (infrastructure path), or
- Valkey metadata (dynamic key path).
- Migration guidance is required for users moving from manual `.env` setup to installer flow.
## Policy Checklist
- [x] All sensitive tRPC procedures require `protectedProcedure`.
@@ -30,7 +54,7 @@ This checklist is the canonical auth policy map for current Sirius architecture.
- [x] UI -> Go API calls use shared authenticated client (`apiClient` / `apiFetch`).
- [x] Direct tRPC backends (Valkey/RabbitMQ) remain session-gated.
- [ ] Agent identity to HTTP/API actions is fully cryptographically bound.
- [ ] Production key lifecycle is fully deterministic for bootstrap/recovery/rotation.
- [x] Production key lifecycle is deterministic for root key validation and user-key lifecycle handling.
## tRPC Procedure Matrix (By Router)
@@ -57,6 +57,9 @@ This guide explains how to deploy Sirius using prebuilt container images from Gi
git clone https://github.com/SiriusScan/Sirius.git
cd Sirius
# Generate startup config and secrets
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Deploy with prebuilt images (default)
docker compose up -d
@@ -165,11 +168,12 @@ The `docker-compose.dev.yaml` file overrides the registry images with local buil
cd Sirius
```
2. **Configure environment** (optional):
2. **Configure environment**:
```bash
cp .env.example .env
# Edit .env with your configuration
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
# Optional: pass explicit values
# docker compose -f docker-compose.installer.yaml run --rm sirius-installer --non-interactive --no-print-secrets
```
3. **Deploy services**:
@@ -252,6 +256,25 @@ If GitHub Container Registry is unavailable or images fail to pull, you can fall
**Note**: Local builds take significantly longer (20-25 minutes vs 5-8 minutes) and require more system resources.
## Secrets Hardening Overlays
For hardened deployments, Sirius includes optional overlay manifests:
- `docker-compose.secrets.yaml` for Compose secrets mounted at `/run/secrets/*`
- `docker-stack.swarm.yaml` for Swarm stack deployments
Example:
```bash
mkdir -p secrets
printf '%s' "your-postgres-password" > secrets/postgres_password.txt
printf '%s' "your-service-key" > secrets/sirius_api_key.txt
printf '%s' "your-nextauth-secret" > secrets/nextauth_secret.txt
printf '%s' "your-admin-password" > secrets/initial_admin_password.txt
docker compose -f docker-compose.yaml -f docker-compose.secrets.yaml up -d
```
## Troubleshooting
### Images Not Pulling
@@ -1,6 +1,6 @@
---
title: "API Key Operations Runbook"
description: "Production runbook for service API key rotation, bootstrap recovery, and incident response."
description: "Production runbook for stateless service API key rotation, recovery, and incident response."
template: "TEMPLATE.guide"
llm_context: "high"
categories: ["operations", "security", "deployment"]
@@ -27,7 +27,8 @@ Applies to service-to-service credential `SIRIUS_API_KEY` used by:
- `SIRIUS_API_KEY` must be non-empty in production.
- Go API must reject requests without a valid `X-API-Key`.
- Only `/health` is public.
- Key bootstrap must reconcile metadata and bootstrap flag on startup.
- Root service key validation is stateless from runtime environment configuration.
- Valkey stores only dynamic/user-generated API key records.
## Rotation Procedure (No Downtime)
@@ -61,30 +62,28 @@ Applies to service-to-service credential `SIRIUS_API_KEY` used by:
## Incident: Valkey Data Loss
Symptoms:
- sudden 401s with previously valid key
- sudden 401s for user-generated keys
- API key list empty
- bootstrap state inconsistent
- dynamic key operations fail while root key requests still succeed
Recovery:
1. Restore Valkey from backup if available.
2. If no backup:
- ensure production `SIRIUS_API_KEY` is correctly set for all services.
- restart `sirius-api` first to reconcile key metadata + bootstrap state.
- restart `sirius-api` first; root key path remains stateless.
3. Restart `sirius-ui` and `sirius-engine`.
4. Validate end-to-end operations and security harness results.
## Incident: Bootstrap Mismatch (Flag/key metadata drift)
## Incident: Root Key Mismatch (Configuration Drift)
Symptoms:
- bootstrap flag exists but key lookups fail
- key exists but bootstrap flag absent
- services return 401 with old key after key rotation
- env values differ between `sirius-ui`, `sirius-api`, and `sirius-engine`
Recovery:
1. Confirm deployed `SIRIUS_API_KEY` value in runtime environment.
2. Restart `sirius-api` (reconciliation should repair state).
3. Re-check:
- key validation success for deployed key
- bootstrap complete marker present
2. Restart `sirius-api`, then `sirius-ui` and `sirius-engine`.
3. Re-check key validation success for deployed root key.
4. If still failing, capture API logs and run security harness `auth-surface` and `api` suites.
## Incident: Unauthorized Agent/Scanner API Writes
@@ -112,8 +111,8 @@ go run . --suite auth-surface
## Operational Checklist
- [ ] `SIRIUS_API_KEY` secret exists and is non-empty in production.
- [ ] API startup confirms key reconciliation succeeded.
- [ ] API middleware accepts configured root key on non-health routes.
- [ ] UI + engine are using the same active service key.
- [ ] Old keys are revoked only after rollout validation.
- [ ] Recovery procedure for Valkey/key mismatch is documented in incident ticket.
- [ ] Recovery procedure for config drift or Valkey loss is documented in incident ticket.
@@ -429,43 +429,10 @@ echo "Branch: ${sirius_branch}"
git clone --branch ${sirius_branch} ${sirius_repo_url} repo
cd repo
# Create environment configuration
# Create environment configuration using installer-first flow
echo "⚙️ Configuring environment..."
cat > .env << 'EOF'
# Environment Configuration
NODE_ENV=production
GO_ENV=production
# Database
POSTGRES_USER=postgres
POSTGRES_PASSWORD=change_me_in_production
POSTGRES_DB=sirius
POSTGRES_HOST=sirius-postgres
POSTGRES_PORT=5432
# API
API_PORT=9001
# UI
NEXTAUTH_SECRET=change_me_in_production
NEXTAUTH_URL=http://localhost:3000
# Redis/Valkey
VALKEY_HOST=sirius-valkey
VALKEY_PORT=6379
# RabbitMQ
RABBITMQ_URL=amqp://guest:guest@sirius-rabbitmq:5672/
# Engine
ENGINE_MAIN_PORT=5174
GRPC_AGENT_PORT=50051
# Logging
LOG_LEVEL=info
EOF
echo "✅ Environment configuration created"
docker compose -f docker-compose.installer.yaml run --rm sirius-installer --non-interactive --no-print-secrets
echo "✅ Environment configuration created with installer"
# Determine image tag from sirius_branch variable
if [[ "${sirius_branch}" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
+14
View File
@@ -0,0 +1,14 @@
FROM golang:1.24-alpine AS builder
WORKDIR /src
COPY installer/go.mod ./go.mod
COPY installer/cmd ./cmd
COPY installer/internal ./internal
RUN go build -o /out/sirius-installer ./cmd/sirius-installer
FROM alpine:3.20
WORKDIR /workspace
COPY --from=builder /out/sirius-installer /usr/local/bin/sirius-installer
ENTRYPOINT ["sirius-installer"]
+50
View File
@@ -0,0 +1,50 @@
# Sirius Installer
`sirius-installer` is the first-run setup utility for Sirius startup and secrets configuration.
## What it does
- Reads template values from `.env.production.example`
- Merges with existing `.env` values (idempotent by default)
- Generates missing required secrets:
- `SIRIUS_API_KEY`
- `POSTGRES_PASSWORD`
- `NEXTAUTH_SECRET`
- `INITIAL_ADMIN_PASSWORD`
- Supports interactive and non-interactive modes
- Supports `--force` regeneration and secret-safe output flags
## Recommended usage from repository root
Use the installer Compose entrypoint (preferred):
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
```
## Local usage
```bash
cd installer
go run ./cmd/sirius-installer --template ../.env.production.example --output ../.env
```
## Docker usage
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer
```
## Non-interactive usage
```bash
docker compose -f docker-compose.installer.yaml run --rm sirius-installer \
--non-interactive \
--no-print-secrets
```
## Useful flags
- `--force`: regenerate required secrets even if they already exist
- `--non-interactive`: never prompt; suitable for CI/user-data
- `--no-print-secrets`: suppress secret values in installer output
+184
View File
@@ -0,0 +1,184 @@
package main
import (
"bufio"
"errors"
"flag"
"fmt"
"os"
"path/filepath"
"strings"
"github.com/SiriusScan/sirius-installer/internal/config"
"github.com/SiriusScan/sirius-installer/internal/prompt"
)
type cliOptions struct {
TemplatePath string
OutputPath string
Force bool
NonInteractive bool
Quiet bool
PrintSecrets bool
NoPrintSecrets bool
AdminPassword string
NextAuthURL string
SiriusAPIURL string
NextPublicAPIURL string
CORSOrigins string
}
func main() {
opts := parseFlags()
if err := run(opts); err != nil {
fmt.Fprintf(os.Stderr, "sirius-installer error: %v\n", err)
os.Exit(1)
}
}
func parseFlags() cliOptions {
var opts cliOptions
flag.StringVar(&opts.TemplatePath, "template", ".env.production.example", "path to .env template")
flag.StringVar(&opts.OutputPath, "output", ".env", "path to generated .env output")
flag.BoolVar(&opts.Force, "force", false, "regenerate required secrets even when already set")
flag.BoolVar(&opts.NonInteractive, "non-interactive", false, "disable prompts and use generated/default values")
flag.BoolVar(&opts.Quiet, "quiet", false, "minimize output")
flag.BoolVar(&opts.PrintSecrets, "print-secrets", true, "print generated secrets to stdout")
flag.BoolVar(&opts.NoPrintSecrets, "no-print-secrets", false, "never print generated secrets")
flag.StringVar(&opts.AdminPassword, "admin-password", "", "explicit initial admin password")
flag.StringVar(&opts.NextAuthURL, "nextauth-url", "", "override NEXTAUTH_URL")
flag.StringVar(&opts.SiriusAPIURL, "sirius-api-url", "", "override SIRIUS_API_URL")
flag.StringVar(&opts.NextPublicAPIURL, "next-public-sirius-api-url", "", "override NEXT_PUBLIC_SIRIUS_API_URL")
flag.StringVar(&opts.CORSOrigins, "cors-origins", "", "override CORS_ALLOWED_ORIGINS")
flag.Parse()
return opts
}
func run(opts cliOptions) error {
if opts.NoPrintSecrets {
opts.PrintSecrets = false
}
templateFile, err := loadOrEmpty(opts.TemplatePath, false)
if err != nil {
return fmt.Errorf("failed to read template file %q: %w", opts.TemplatePath, err)
}
outputFile, err := loadOrEmpty(opts.OutputPath, true)
if err != nil {
return fmt.Errorf("failed to read output file %q: %w", opts.OutputPath, err)
}
cfgOpts := config.Options{
Force: opts.Force,
NonInteractive: opts.NonInteractive,
AdminPassword: opts.AdminPassword,
NextAuthURL: opts.NextAuthURL,
SiriusAPIURL: opts.SiriusAPIURL,
NextPublicAPIURL: opts.NextPublicAPIURL,
CORSAllowedOrigin: opts.CORSOrigins,
}
if !opts.NonInteractive {
cfgOpts, err = gatherInteractive(cfgOpts)
if err != nil {
return err
}
}
merged := config.Merge(templateFile.Values, outputFile.Values, cfgOpts)
finalVals, generated, err := config.EnsureRequired(merged, cfgOpts)
if err != nil {
return err
}
rendered := config.Render(templateFile, finalVals)
if err := writeSecure(opts.OutputPath, rendered); err != nil {
return err
}
if !opts.Quiet {
fmt.Printf("Sirius installer wrote %s\n", opts.OutputPath)
fmt.Println("Required startup secrets are configured.")
}
if opts.PrintSecrets && len(generated) > 0 {
fmt.Println("\nGenerated values (save securely):")
for _, key := range []string{
"SIRIUS_API_KEY",
"POSTGRES_PASSWORD",
"NEXTAUTH_SECRET",
"INITIAL_ADMIN_PASSWORD",
} {
if v, ok := generated[key]; ok {
fmt.Printf("- %s=%s\n", key, v)
}
}
}
if !opts.Quiet {
fmt.Println("\nNext step:")
fmt.Println("docker compose up -d")
}
return nil
}
func gatherInteractive(in config.Options) (config.Options, error) {
reader := bufio.NewReader(os.Stdin)
if strings.TrimSpace(in.AdminPassword) == "" {
pw, err := prompt.AskOptional(reader, "Initial admin password")
if err != nil {
return in, err
}
in.AdminPassword = pw
}
currentURL := fallback(in.NextAuthURL, "http://localhost:3000")
url, err := prompt.Ask(reader, "NEXTAUTH_URL", currentURL)
if err != nil {
return in, err
}
in.NextAuthURL = url
currentAPI := fallback(in.NextPublicAPIURL, "http://localhost:9001")
pubAPI, err := prompt.Ask(reader, "NEXT_PUBLIC_SIRIUS_API_URL", currentAPI)
if err != nil {
return in, err
}
in.NextPublicAPIURL = pubAPI
return in, nil
}
func loadOrEmpty(path string, allowMissing bool) (*config.EnvFile, error) {
f, err := config.ParseEnvFile(path)
if err == nil {
return f, nil
}
if allowMissing && errors.Is(err, os.ErrNotExist) {
return config.NewEmptyEnvFile(), nil
}
return nil, err
}
func writeSecure(path, content string) error {
abs, err := filepath.Abs(path)
if err == nil {
dir := filepath.Dir(abs)
if mkErr := os.MkdirAll(dir, 0o755); mkErr != nil {
return fmt.Errorf("failed to create output directory %q: %w", dir, mkErr)
}
}
if err := os.WriteFile(path, []byte(content), 0o600); err != nil {
return fmt.Errorf("failed writing %s: %w", path, err)
}
return nil
}
func fallback(v, d string) string {
if strings.TrimSpace(v) == "" {
return d
}
return v
}
+3
View File
@@ -0,0 +1,3 @@
module github.com/SiriusScan/sirius-installer
go 1.24
+129
View File
@@ -0,0 +1,129 @@
package config
import (
"bufio"
"os"
"sort"
"strings"
)
type LineKind string
const (
KindBlank LineKind = "blank"
KindComment LineKind = "comment"
KindKV LineKind = "kv"
)
type Line struct {
Kind LineKind
Raw string
Key string
Value string
}
type EnvFile struct {
Lines []Line
Values map[string]string
}
func ParseEnvFile(path string) (*EnvFile, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
out := &EnvFile{
Lines: make([]Line, 0, 64),
Values: make(map[string]string),
}
scanner := bufio.NewScanner(f)
for scanner.Scan() {
raw := scanner.Text()
trimmed := strings.TrimSpace(raw)
if trimmed == "" {
out.Lines = append(out.Lines, Line{Kind: KindBlank, Raw: raw})
continue
}
if strings.HasPrefix(trimmed, "#") {
out.Lines = append(out.Lines, Line{Kind: KindComment, Raw: raw})
continue
}
k, v, ok := splitEnvKV(raw)
if !ok {
// Preserve unknown rows as comments so rendering remains stable.
out.Lines = append(out.Lines, Line{Kind: KindComment, Raw: raw})
continue
}
out.Values[k] = v
out.Lines = append(out.Lines, Line{Kind: KindKV, Raw: raw, Key: k, Value: v})
}
if err := scanner.Err(); err != nil {
return nil, err
}
return out, nil
}
func NewEmptyEnvFile() *EnvFile {
return &EnvFile{
Lines: []Line{},
Values: map[string]string{},
}
}
func Render(base *EnvFile, values map[string]string) string {
if base == nil {
base = NewEmptyEnvFile()
}
seen := make(map[string]struct{}, len(values))
var b strings.Builder
for _, line := range base.Lines {
switch line.Kind {
case KindBlank, KindComment:
b.WriteString(line.Raw)
case KindKV:
v, ok := values[line.Key]
if !ok {
v = line.Value
}
b.WriteString(line.Key + "=" + v)
seen[line.Key] = struct{}{}
}
b.WriteString("\n")
}
extra := make([]string, 0, len(values))
for k := range values {
if _, ok := seen[k]; !ok {
extra = append(extra, k)
}
}
sort.Strings(extra)
if len(extra) > 0 {
b.WriteString("\n# Added by Sirius installer\n")
for _, k := range extra {
b.WriteString(k + "=" + values[k] + "\n")
}
}
return b.String()
}
func splitEnvKV(raw string) (string, string, bool) {
idx := strings.IndexRune(raw, '=')
if idx <= 0 {
return "", "", false
}
key := strings.TrimSpace(raw[:idx])
if key == "" {
return "", "", false
}
val := strings.TrimSpace(raw[idx+1:])
return key, val, true
}
+141
View File
@@ -0,0 +1,141 @@
package config
import (
"crypto/rand"
"encoding/base64"
"encoding/hex"
"fmt"
"strings"
)
type Options struct {
Force bool
NonInteractive bool
AdminPassword string
NextAuthURL string
SiriusAPIURL string
NextPublicAPIURL string
CORSAllowedOrigin string
}
func Merge(templateVals, existingVals map[string]string, opts Options) map[string]string {
out := make(map[string]string, len(templateVals)+len(existingVals)+8)
for k, v := range templateVals {
out[k] = v
}
for k, v := range existingVals {
if strings.TrimSpace(v) != "" {
out[k] = v
}
}
applyOpt(out, "INITIAL_ADMIN_PASSWORD", opts.AdminPassword)
applyOpt(out, "NEXTAUTH_URL", opts.NextAuthURL)
applyOpt(out, "SIRIUS_API_URL", opts.SiriusAPIURL)
applyOpt(out, "NEXT_PUBLIC_SIRIUS_API_URL", opts.NextPublicAPIURL)
applyOpt(out, "CORS_ALLOWED_ORIGINS", opts.CORSAllowedOrigin)
return out
}
func EnsureRequired(values map[string]string, opts Options) (map[string]string, map[string]string, error) {
generated := map[string]string{}
if shouldGenerate(values["SIRIUS_API_KEY"], opts.Force) {
s, err := randomHex(32)
if err != nil {
return nil, nil, err
}
values["SIRIUS_API_KEY"] = s
generated["SIRIUS_API_KEY"] = s
}
if shouldGenerate(values["POSTGRES_PASSWORD"], opts.Force) {
s, err := randomHex(16)
if err != nil {
return nil, nil, err
}
values["POSTGRES_PASSWORD"] = s
generated["POSTGRES_PASSWORD"] = s
}
if shouldGenerate(values["NEXTAUTH_SECRET"], opts.Force) {
s, err := randomHex(32)
if err != nil {
return nil, nil, err
}
values["NEXTAUTH_SECRET"] = s
generated["NEXTAUTH_SECRET"] = s
}
if shouldGenerate(values["INITIAL_ADMIN_PASSWORD"], opts.Force) {
if strings.TrimSpace(opts.AdminPassword) != "" {
values["INITIAL_ADMIN_PASSWORD"] = opts.AdminPassword
} else {
s, err := randomBase64(12)
if err != nil {
return nil, nil, err
}
values["INITIAL_ADMIN_PASSWORD"] = s
generated["INITIAL_ADMIN_PASSWORD"] = s
}
}
if strings.TrimSpace(values["INITIAL_ADMIN_PASSWORD"]) == "" {
return nil, nil, fmt.Errorf("INITIAL_ADMIN_PASSWORD is required")
}
return values, generated, nil
}
func IsPlaceholder(v string) bool {
s := strings.ToLower(strings.TrimSpace(v))
if s == "" {
return true
}
placeholders := []string{
"password",
"postgres",
"change-me",
"change_this",
"dummy",
"placeholder",
"your_",
"your-",
"test-secret",
"ci-placeholder-api-key",
}
for _, p := range placeholders {
if strings.Contains(s, p) {
return true
}
}
return false
}
func shouldGenerate(v string, force bool) bool {
return force || IsPlaceholder(v)
}
func applyOpt(values map[string]string, key, val string) {
if strings.TrimSpace(val) != "" {
values[key] = val
}
}
func randomHex(numBytes int) (string, error) {
buf := make([]byte, numBytes)
if _, err := rand.Read(buf); err != nil {
return "", fmt.Errorf("failed to read random bytes: %w", err)
}
return hex.EncodeToString(buf), nil
}
func randomBase64(numBytes int) (string, error) {
buf := make([]byte, numBytes)
if _, err := rand.Read(buf); err != nil {
return "", fmt.Errorf("failed to read random bytes: %w", err)
}
return base64.RawStdEncoding.EncodeToString(buf), nil
}
+34
View File
@@ -0,0 +1,34 @@
package prompt
import (
"bufio"
"fmt"
"os"
"strings"
)
func Ask(reader *bufio.Reader, message, fallback string) (string, error) {
if _, err := fmt.Fprintf(os.Stdout, "%s [%s]: ", message, fallback); err != nil {
return "", err
}
raw, err := reader.ReadString('\n')
if err != nil {
return "", err
}
val := strings.TrimSpace(raw)
if val == "" {
return fallback, nil
}
return val, nil
}
func AskOptional(reader *bufio.Reader, message string) (string, error) {
if _, err := fmt.Fprintf(os.Stdout, "%s (leave blank to auto-generate): ", message); err != nil {
return "", err
}
raw, err := reader.ReadString('\n')
if err != nil {
return "", err
}
return strings.TrimSpace(raw), nil
}
+32 -10
View File
@@ -9,6 +9,8 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
LOCAL_COMPOSE="$PROJECT_ROOT/docker-compose.local.yaml"
EXAMPLE_COMPOSE="$PROJECT_ROOT/docker-compose.local.example.yaml"
ENV_FILE="$PROJECT_ROOT/.env"
INSTALLER_COMPOSE_FILE="$PROJECT_ROOT/docker-compose.installer.yaml"
echo "🚀 Sirius Development Setup"
echo "================================"
@@ -75,11 +77,31 @@ init_dev() {
echo "🔧 For standard development (no local repos needed): '$0 start'"
}
# Ensure required env exists before startup.
ensure_env_ready() {
if [ -f "$ENV_FILE" ]; then
return 0
fi
echo "⚠️ Missing $ENV_FILE"
if [ "${SIRIUS_AUTO_SETUP:-0}" = "1" ]; then
echo "🔧 Running installer container in non-interactive mode..."
docker compose -f "$INSTALLER_COMPOSE_FILE" run --rm sirius-installer --non-interactive --no-print-secrets
return 0
fi
echo "Run installer first:"
echo " docker compose -f docker-compose.installer.yaml run --rm sirius-installer"
echo "Or rerun with SIRIUS_AUTO_SETUP=1 for automated setup."
return 1
}
# Function to start development environment
start_dev() {
local mode=$1
cd "$PROJECT_ROOT"
ensure_env_ready || return 1
if [ "$mode" = "extended" ]; then
echo "🔧 Starting extended development environment..."
@@ -87,10 +109,10 @@ start_dev() {
return 1
fi
echo "📁 Using local repository mounts from docker-compose.local.yaml"
docker-compose -f docker-compose.yaml -f docker-compose.override.yaml -f docker-compose.local.yaml up -d
docker compose -f docker-compose.yaml -f docker-compose.override.yaml -f docker-compose.local.yaml up -d
else
echo "🔧 Starting standard development environment..."
docker-compose up -d
docker compose up -d
fi
echo ""
@@ -99,10 +121,10 @@ start_dev() {
echo "🔧 API: http://localhost:9001"
echo "📊 RabbitMQ Management: http://localhost:15672 (guest/guest)"
if command -v docker-compose &> /dev/null; then
if command -v docker &> /dev/null; then
echo ""
echo "📋 Container Status:"
docker-compose ps
docker compose ps
fi
}
@@ -110,7 +132,7 @@ start_dev() {
stop_dev() {
echo "🛑 Stopping development environment..."
cd "$PROJECT_ROOT"
docker-compose down
docker compose down
echo "✅ Development environment stopped"
}
@@ -126,7 +148,7 @@ clean_dev() {
return 0
fi
docker-compose down -v --remove-orphans
docker compose down -v --remove-orphans
docker system prune -f
echo "✅ Development environment cleaned"
}
@@ -135,7 +157,7 @@ clean_dev() {
show_status() {
echo "📋 Container Status:"
cd "$PROJECT_ROOT"
docker-compose ps
docker compose ps
}
# Function to show logs
@@ -145,10 +167,10 @@ show_logs() {
if [ -n "$service" ]; then
echo "📜 Showing logs for $service..."
docker-compose logs -f "$service"
docker compose logs -f "$service"
else
echo "📜 Showing logs for all services..."
docker-compose logs -f
docker compose logs -f
fi
}
@@ -164,7 +186,7 @@ open_shell() {
cd "$PROJECT_ROOT"
echo "🐚 Opening shell in $service..."
docker-compose exec "$service" /bin/bash || docker-compose exec "$service" /bin/sh
docker compose exec "$service" /bin/bash || docker compose exec "$service" /bin/sh
}
# Main command handling
+23
View File
@@ -7,6 +7,26 @@ set -e
ENV=${1:-"base"}
PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
INSTALLER_COMPOSE_FILE="$PROJECT_ROOT/docker-compose.installer.yaml"
ENV_FILE="$PROJECT_ROOT/.env"
require_env_file() {
if [ -f "$ENV_FILE" ]; then
return 0
fi
echo "⚠️ Missing $ENV_FILE"
if [ "${SIRIUS_AUTO_SETUP:-0}" = "1" ]; then
echo "🔧 SIRIUS_AUTO_SETUP=1 detected; running installer container..."
docker compose -f "$INSTALLER_COMPOSE_FILE" run --rm sirius-installer --non-interactive --no-print-secrets
return 0
fi
echo "Run installer first:"
echo " docker compose -f docker-compose.installer.yaml run --rm sirius-installer"
echo "Or rerun with SIRIUS_AUTO_SETUP=1 to auto-generate missing values."
exit 1
}
echo "🔄 Switching Sirius environment to: $ENV"
@@ -58,14 +78,17 @@ cd "$PROJECT_ROOT"
case $ENV in
"dev"|"development")
require_env_file
cleanup
start_dev
;;
"prod"|"production")
require_env_file
cleanup
start_prod
;;
"base"|"default")
require_env_file
cleanup
start_base
;;
+3
View File
@@ -31,6 +31,9 @@ func (h *APIKeyHandler) CreateKey(c *fiber.Ctx) error {
// Determine creator from the API key metadata (set by middleware).
createdBy := "system"
if label, ok := c.Locals("apikey_label").(string); ok && label != "" {
createdBy = label
}
if meta, ok := c.Locals("apikey_meta").(store.APIKeyMeta); ok {
createdBy = meta.Label
}
+3 -32
View File
@@ -1,7 +1,6 @@
package main
import (
"context"
"fmt"
"log/slog"
"net"
@@ -107,35 +106,13 @@ func runMigrations() error {
return nil
}
// bootstrapAPIKeys reconciles the configured service API key with Valkey state.
// It repairs partial states by ensuring both key metadata and bootstrap flag
// exist on every startup.
func bootstrapAPIKeys(kvStore store.KVStore, rawKey string) error {
ctx := context.Background()
wasBootstrapped := store.IsBootstrapped(ctx, kvStore)
meta, err := store.EnsureAPIKey(ctx, kvStore, rawKey, "root", "system-bootstrap")
if err != nil {
return fmt.Errorf("failed to ensure root API key metadata: %w", err)
}
if err := store.MarkBootstrapped(ctx, kvStore); err != nil {
return fmt.Errorf("failed to mark bootstrap complete: %w", err)
}
if wasBootstrapped {
slog.Info("API key bootstrap reconciled", "key_prefix", meta.Prefix)
} else {
slog.Info("API key bootstrap completed", "key_prefix", meta.Prefix)
}
return nil
}
func main() {
// Initialize structured logging (reads LOG_LEVEL env var)
slogger.Init()
// Root service key is validated statelessly in middleware against env config.
// Valkey remains authoritative only for user-generated API keys.
serviceAPIKey := strings.TrimSpace(os.Getenv("SIRIUS_API_KEY"))
if serviceAPIKey == "" {
slog.Error("SIRIUS_API_KEY is required for sirius-api startup")
@@ -152,7 +129,7 @@ func main() {
os.Exit(1)
}
// Initialize Valkey store for API key management.
// Initialize Valkey store for dynamic API key management.
kvStore, err := store.NewValkeyStore()
if err != nil {
slog.Error("Failed to connect to Valkey", "error", err)
@@ -160,12 +137,6 @@ func main() {
}
defer kvStore.Close()
// Reconcile service API key metadata and bootstrap flag state.
if err := bootstrapAPIKeys(kvStore, serviceAPIKey); err != nil {
slog.Error("Failed to reconcile API key bootstrap state", "error", err)
os.Exit(1)
}
app := fiber.New()
// Add CORS middleware
@@ -2,6 +2,9 @@ package middleware
import (
"context"
"crypto/subtle"
"log/slog"
"os"
"strings"
"github.com/SiriusScan/go-api/sirius/store"
@@ -32,6 +35,15 @@ func APIKeyMiddleware(kvStore store.KVStore) fiber.Handler {
})
}
rootKey := strings.TrimSpace(os.Getenv("SIRIUS_API_KEY"))
if rootKey != "" && subtle.ConstantTimeCompare([]byte(apiKey), []byte(rootKey)) == 1 {
c.Locals("auth_mode", "infra_env")
c.Locals("apikey_label", "Infrastructure Key")
slog.Debug("request authenticated with environment infrastructure key")
return c.Next()
}
// Fallback: Check Valkey for dynamic, user-generated API keys
meta, err := store.ValidateAPIKey(context.Background(), kvStore, apiKey)
if err != nil {
return c.Status(fiber.StatusUnauthorized).JSON(fiber.Map{
@@ -40,6 +52,7 @@ func APIKeyMiddleware(kvStore store.KVStore) fiber.Handler {
}
// Store metadata in request locals for downstream handlers.
c.Locals("auth_mode", "valkey")
c.Locals("apikey_meta", meta)
return c.Next()
}
+2
View File
@@ -19,6 +19,8 @@ DATABASE_URL="file:./db.sqlite"
# https://next-auth.js.org/configuration/options#secret
# NEXTAUTH_SECRET=""
NEXTAUTH_URL="http://192.168.0.7:3000"
INITIAL_ADMIN_PASSWORD=""
SIRIUS_API_KEY=""
# Next Auth Discord Provider
DISCORD_CLIENT_ID=""
+1 -2
View File
@@ -74,7 +74,6 @@ FROM base AS builder
# Accept build args
ARG NEXT_PUBLIC_CLIENTVAR
ARG SIRIUS_API_KEY=build-time-placeholder
# Copy source code for building
COPY . .
@@ -91,7 +90,7 @@ RUN npx prisma generate
# Build the application
ENV NEXT_TELEMETRY_DISABLED=1
ENV NEXT_PUBLIC_CLIENTVAR=${NEXT_PUBLIC_CLIENTVAR}
ENV SIRIUS_API_KEY=${SIRIUS_API_KEY}
ENV SKIP_ENV_VALIDATION=1
# Create a temporary next.config.mjs with errors ignored for Docker building.
# Use a Node-based replacement for portability across sed variants.
+4 -4
View File
@@ -8,14 +8,14 @@
## Database Management
### Default Credentials
### Initial Credentials
- **Username**: `admin`
- **Password**: `password`
- **Password**: value from `INITIAL_ADMIN_PASSWORD`
### Resetting the Database
If you've modified the database during testing (e.g., changed the password) and need to reset it to default values:
If you've modified the database during testing and need to reset it:
**Inside the container:**
@@ -35,7 +35,7 @@ npm run seed
This will:
1. Reset the database schema
2. Recreate the default admin user with password: `password`
2. Recreate the admin user using `INITIAL_ADMIN_PASSWORD`
**Note:** Database files (`*.db`, `*.sqlite`) are gitignored to prevent committing test data. Each developer maintains their own local database state.
+6 -1
View File
@@ -3,8 +3,13 @@ import { hash } from 'bcrypt';
const prisma = new PrismaClient();
async function main() {
const rawPassword = process.env.INITIAL_ADMIN_PASSWORD;
if (!rawPassword) {
throw new Error('INITIAL_ADMIN_PASSWORD is required for database seed');
}
// Hash the password with bcrypt
const hashedPassword = await hash('password', 10);
const hashedPassword = await hash(rawPassword, 10);
try {
// First, try to get the existing user
+9 -2
View File
@@ -1,7 +1,7 @@
// Simple environment object - no validation needed for Docker compose setup
export const env = {
NODE_ENV: process.env.NODE_ENV || "production",
NEXTAUTH_SECRET: process.env.NEXTAUTH_SECRET || "change-this-secret-in-production-please",
NEXTAUTH_SECRET: process.env.NEXTAUTH_SECRET || "",
NEXTAUTH_URL: process.env.NEXTAUTH_URL || "http://localhost:3000",
DISCORD_CLIENT_ID: process.env.DISCORD_CLIENT_ID || "dummy_client_id",
DISCORD_CLIENT_SECRET: process.env.DISCORD_CLIENT_SECRET || "dummy_client_secret",
@@ -12,6 +12,13 @@ export const env = {
SIRIUS_API_KEY: process.env.SIRIUS_API_KEY || "",
};
if (env.NODE_ENV === "production" && !env.SIRIUS_API_KEY.trim()) {
const skipValidation = process.env.SKIP_ENV_VALIDATION === "1";
const isBuildPhase = process.env.NEXT_PHASE === "phase-production-build";
if (!skipValidation && !isBuildPhase && env.NODE_ENV === "production" && !env.SIRIUS_API_KEY.trim()) {
throw new Error("SIRIUS_API_KEY is required in production");
}
if (!skipValidation && !isBuildPhase && env.NODE_ENV === "production" && !env.NEXTAUTH_SECRET.trim()) {
throw new Error("NEXTAUTH_SECRET is required in production");
}
+1 -2
View File
@@ -64,8 +64,7 @@ const getNextAuthUrl = (req?: any): string => {
* @see https://next-auth.js.org/configuration/options
*/
export const authOptions: NextAuthOptions = {
secret:
process.env.NEXTAUTH_SECRET || "change-this-secret-in-production-please",
secret: env.NEXTAUTH_SECRET || undefined,
session: {
strategy: "jwt",
maxAge: 100 * 365 * 24 * 60 * 60, // 100 years in seconds - effectively indefinite
+13
View File
@@ -2,6 +2,19 @@
echo "🚀 Starting Sirius UI Development Server..."
require_env() {
VAR_NAME="$1"
eval "VAR_VALUE=\${$VAR_NAME}"
if [ -z "$VAR_VALUE" ]; then
echo "❌ Missing required environment variable: $VAR_NAME"
exit 1
fi
}
require_env "NEXTAUTH_SECRET"
require_env "INITIAL_ADMIN_PASSWORD"
require_env "SIRIUS_API_KEY"
# Keep container node_modules in sync with package manifests.
# Dev uses a persistent named volume for /app/node_modules, which can drift when deps change.
if [ -f "/app/package.json" ] && [ -f "/app/package-lock.json" ]; then
+13
View File
@@ -2,6 +2,19 @@
echo "🚀 Starting Sirius UI Production Server..."
require_env() {
VAR_NAME="$1"
eval "VAR_VALUE=\${$VAR_NAME}"
if [ -z "$VAR_VALUE" ]; then
echo "❌ Missing required environment variable: $VAR_NAME"
exit 1
fi
}
require_env "NEXTAUTH_SECRET"
require_env "INITIAL_ADMIN_PASSWORD"
require_env "SIRIUS_API_KEY"
# Start system monitor if available
if [ -f "/system-monitor/system-monitor" ] && [ -x "/system-monitor/system-monitor" ]; then
echo "📊 Starting system monitor..."
+237
View File
@@ -0,0 +1,237 @@
[
{
"id": "0",
"title": "PHASE 0: Tracking and Architecture Baseline",
"description": "Create project tracking artifacts and architecture decisions for startup and secrets redesign.",
"details": "Establish task tracking and explicit architecture guidance before implementation to keep rollout coherent and auditable.",
"status": "pending",
"priority": "high",
"dependencies": [],
"subtasks": [
{
"id": "0.1",
"title": "Create startup-secrets project plan note",
"description": "Add high-level planning document under documentation/dev-notes.",
"details": "Document goals, scope, migration strategy, and rollout phases aligned with installer-first setup and stateless root key auth.",
"status": "pending",
"priority": "high",
"dependencies": [],
"testStrategy": "Verify file exists, includes complete YAML front matter, and captures objectives/scope/risks."
},
{
"id": "0.2",
"title": "Create startup-secrets task tracker",
"description": "Add tasks JSON for phased execution.",
"details": "Include phases for installer, compose hardening, API/UI auth updates, docs, tests, CI, optional secrets overlays, and release verification.",
"status": "pending",
"priority": "high",
"dependencies": ["0.1"],
"testStrategy": "Validate JSON syntax and ensure dependencies/status values are valid."
},
{
"id": "0.3",
"title": "Record architecture decisions",
"description": "Capture installer-first and stateless root key model in architecture docs.",
"details": "Update auth surface and operations runbooks to remove bootstrap-centric assumptions and define deterministic key lifecycle behavior.",
"status": "pending",
"priority": "high",
"dependencies": ["0.1"],
"testStrategy": "Architecture docs explicitly describe root env key fast path and Valkey fallback for user keys."
}
]
},
{
"id": "1",
"title": "PHASE 1: Installer Implementation",
"description": "Build a Docker-friendly installer that generates and validates startup secrets/config.",
"details": "Implement installer module with template loading, idempotent merge, secure generation, interactive and non-interactive operation modes.",
"status": "pending",
"priority": "high",
"dependencies": ["0"],
"subtasks": [
{
"id": "1.1",
"title": "Scaffold installer module",
"description": "Create installer command, internal packages, Dockerfile, and README.",
"details": "Establish Go module and executable layout under installer/ with clear interfaces for env parsing and generation.",
"status": "pending",
"priority": "high",
"dependencies": ["0.2"],
"testStrategy": "Installer compiles and basic --help output works."
},
{
"id": "1.2",
"title": "Implement env template loader and serializer",
"description": "Load .env.production.example and preserve ordering/comments as practical.",
"details": "Provide robust parser for KEY=VALUE lines and rendering utility for deterministic output.",
"status": "pending",
"priority": "high",
"dependencies": ["1.1"],
"testStrategy": "Round-trip parse/render keeps expected key ordering and value integrity."
},
{
"id": "1.3",
"title": "Implement idempotent merge and secure generation",
"description": "Generate required secrets only when absent unless force regeneration is requested.",
"details": "Required keys: SIRIUS_API_KEY, POSTGRES_PASSWORD, NEXTAUTH_SECRET, INITIAL_ADMIN_PASSWORD.",
"status": "pending",
"priority": "high",
"dependencies": ["1.2"],
"testStrategy": "Repeated runs preserve existing values unless --force is set."
},
{
"id": "1.4",
"title": "Implement interactive and non-interactive modes",
"description": "Support prompts for local setup and strict no-prompt mode for automation.",
"details": "Add flags for quiet output and no secret printing; return non-zero on invalid non-interactive input state.",
"status": "pending",
"priority": "high",
"dependencies": ["1.3"],
"testStrategy": "Non-interactive mode runs cleanly in CI-style shell and prompts are skipped."
},
{
"id": "1.5",
"title": "Create setup.sh compatibility wrapper",
"description": "Preserve existing entrypoint while routing to installer workflow.",
"details": "Replace old generation script logic with a deprecation-safe shim that calls installer behavior.",
"status": "pending",
"priority": "medium",
"dependencies": ["1.4"],
"testStrategy": "Running setup.sh generates/updates .env via installer and prints migration guidance."
}
]
},
{
"id": "2",
"title": "PHASE 2: Compose and Runtime Hardening",
"description": "Enforce secure environment contracts and remove insecure defaults.",
"details": "Harden compose files, env templates, UI auth runtime config, and seed behavior.",
"status": "pending",
"priority": "high",
"dependencies": ["1"],
"subtasks": [
{
"id": "2.1",
"title": "Harden compose environment requirements",
"description": "Remove insecure defaults and require critical variables.",
"details": "Update base/dev/prod compose files to require auth secrets and root key consistently.",
"status": "pending",
"priority": "high",
"dependencies": ["1.5"],
"testStrategy": "docker compose config passes only when required vars are provided."
},
{
"id": "2.2",
"title": "Update .env templates and gitignore",
"description": "Align templates and ignore generated secret files.",
"details": "Add INITIAL_ADMIN_PASSWORD guidance and include root .env in gitignore.",
"status": "pending",
"priority": "high",
"dependencies": ["2.1"],
"testStrategy": "Templates include required keys and .env no longer appears as untracked by default."
},
{
"id": "2.3",
"title": "Refine API auth middleware",
"description": "Ensure stateless root key path is explicit and observable.",
"details": "Add clear request-local metadata/logging for env root key path versus Valkey user-key validation path.",
"status": "pending",
"priority": "high",
"dependencies": ["2.1"],
"testStrategy": "Requests with env key auth path succeed without Valkey key lookup."
},
{
"id": "2.4",
"title": "Harden UI auth and seed behavior",
"description": "Remove password fallback and enforce required secrets in production.",
"details": "Update seed.ts, env.mjs, auth.ts, and UI startup scripts for fail-fast behavior on missing secrets.",
"status": "pending",
"priority": "high",
"dependencies": ["2.2"],
"testStrategy": "Seed fails when INITIAL_ADMIN_PASSWORD is missing in production-mode startup."
}
]
},
{
"id": "3",
"title": "PHASE 3: Docs, Tests, CI, and Optional Hardening",
"description": "Update docs/runbooks and align validation pipelines with new startup contract.",
"details": "Rewrite onboarding and deployment docs, update test harnesses and CI pipelines, and add optional secrets overlays.",
"status": "pending",
"priority": "medium",
"dependencies": ["2"],
"subtasks": [
{
"id": "3.1",
"title": "Rewrite onboarding and operations docs",
"description": "Update README, UI README, API key operations, deployment guides.",
"details": "Replace manual secret-sync narrative with installer-first flow and stateless root-key model.",
"status": "pending",
"priority": "high",
"dependencies": ["2.4"],
"testStrategy": "Documentation references installer workflow and no longer recommends default passwords."
},
{
"id": "3.2",
"title": "Update container/security tests",
"description": "Supply required secrets in tests and align auth expectations.",
"details": "Patch container-testing scripts and security suite assumptions around bootstrap language.",
"status": "pending",
"priority": "high",
"dependencies": ["3.1"],
"testStrategy": "Container tests and security suites pass with required vars set."
},
{
"id": "3.3",
"title": "Update CI workflow environment contracts",
"description": "Set new required variables in compose validation and integration jobs.",
"details": "Update ci.yml and ci-modernized.yml so pipelines stay green under stricter runtime contracts.",
"status": "pending",
"priority": "high",
"dependencies": ["3.2"],
"testStrategy": "CI compose config checks and integration jobs pass in dry-run validation."
},
{
"id": "3.4",
"title": "Add optional secrets overlay files",
"description": "Provide compose/swarm secrets overlays for hardened deployments.",
"details": "Add docker-compose.secrets.yaml and docker-stack.swarm.yaml plus docs for _FILE patterns.",
"status": "pending",
"priority": "medium",
"dependencies": ["3.1"],
"testStrategy": "Overlay files parse via docker compose config and stack file syntax is valid."
}
]
},
{
"id": "4",
"title": "PHASE 4: Release Verification and Migration Notes",
"description": "Run end-to-end verification and publish migration guidance.",
"details": "Validate startup, auth, and key lifecycle behavior and publish upgrade notes for existing users.",
"status": "pending",
"priority": "high",
"dependencies": ["3"],
"subtasks": [
{
"id": "4.1",
"title": "Execute release verification matrix",
"description": "Run compose, container, and security validation for redesigned startup path.",
"details": "Verify first-run installer behavior, login flow, root key rotation semantics, and dynamic key operations.",
"status": "pending",
"priority": "high",
"dependencies": ["3.3"],
"testStrategy": "All defined verification points pass and issues are documented/fixed."
},
{
"id": "4.2",
"title": "Publish migration notes",
"description": "Document upgrade path from manual .env setup to installer-driven setup.",
"details": "Include compatibility expectations, changed required variables, and safe rollback guidance.",
"status": "pending",
"priority": "medium",
"dependencies": ["4.1"],
"testStrategy": "Migration notes are present in changelog/docs and reviewed for accuracy."
}
]
}
]
+6
View File
@@ -24,6 +24,12 @@ LOG_DIR="$PROJECT_ROOT/testing/logs"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
LOG_FILE="$LOG_DIR/build_test_$TIMESTAMP.log"
# Required compose variables for strict startup contract.
export SIRIUS_API_KEY="${SIRIUS_API_KEY:-test-api-key}"
export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-test-postgres-password}"
export NEXTAUTH_SECRET="${NEXTAUTH_SECRET:-test-nextauth-secret}"
export INITIAL_ADMIN_PASSWORD="${INITIAL_ADMIN_PASSWORD:-test-admin-password}"
# Create logs directory
mkdir -p "$LOG_DIR"
+6
View File
@@ -24,6 +24,12 @@ LOG_DIR="$PROJECT_ROOT/testing/logs"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
LOG_FILE="$LOG_DIR/health_test_$TIMESTAMP.log"
# Required compose variables for strict startup contract.
export SIRIUS_API_KEY="${SIRIUS_API_KEY:-test-api-key}"
export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-test-postgres-password}"
export NEXTAUTH_SECRET="${NEXTAUTH_SECRET:-test-nextauth-secret}"
export INITIAL_ADMIN_PASSWORD="${INITIAL_ADMIN_PASSWORD:-test-admin-password}"
# Environment variables for configuration
TEST_TIMEOUT="${TEST_TIMEOUT:-60}" # Default 60 seconds
TEST_RETRIES="${TEST_RETRIES:-60}" # Default 60 retries (2 seconds each = 120 seconds total)
@@ -24,6 +24,12 @@ LOG_DIR="$PROJECT_ROOT/testing/logs"
TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
LOG_FILE="$LOG_DIR/integration_test_$TIMESTAMP.log"
# Required compose variables for strict startup contract.
export SIRIUS_API_KEY="${SIRIUS_API_KEY:-test-api-key}"
export POSTGRES_PASSWORD="${POSTGRES_PASSWORD:-test-postgres-password}"
export NEXTAUTH_SECRET="${NEXTAUTH_SECRET:-test-nextauth-secret}"
export INITIAL_ADMIN_PASSWORD="${INITIAL_ADMIN_PASSWORD:-test-admin-password}"
# Environment variables for configuration
TEST_TIMEOUT="${TEST_TIMEOUT:-60}" # Default 60 seconds
TEST_RETRIES="${TEST_RETRIES:-60}" # Default 60 retries (2 seconds each = 120 seconds total)
@@ -158,7 +164,7 @@ test_error_handling() {
# Test 404 handling
run_test "UI 404 Error Handling" "curl -s -I http://localhost:3000/nonexistent | grep -q '404'"
run_test "API 404 Error Handling" "curl -s -I http://localhost:9001/nonexistent | grep -q '404'"
run_test "API Error Handling" "curl -s -I http://localhost:9001/nonexistent | grep -Eq '401|404'"
# Test invalid API requests (using actual endpoints)
run_test "Invalid API Request Handling" "curl -s -X POST http://localhost:9001/host -H 'Content-Type: application/json' -d '{}' | grep -q 'error\\|invalid' || echo 'API error handling working'"
+8 -8
View File
@@ -26,11 +26,11 @@ func RunAPISuite(cfg *Config) SuiteResult {
// ── Phase 1: Obtain a valid API key for positive tests ──────────────
validKey := cfg.APIKey
if validKey == "" {
validKey = bootstrapAPIKey(cfg, &suite)
validKey = provisionAPIKeyForTests(cfg, &suite)
if validKey == "" {
// Cannot proceed without a valid key; all remaining tests skipped.
suite.Results = append(suite.Results, TestResult{
Name: "API key bootstrap",
Name: "API key provisioning",
Result: Skip,
Severity: SevCritical,
Detail: "Could not obtain a valid API key; remaining tests skipped",
@@ -307,10 +307,10 @@ func testHeaderCaseSensitivity(base, key string) TestResult {
// ────────────────── Bootstrap helper ──────────────────
// bootstrapAPIKey creates a test API key using the /api/v1/keys endpoint.
// provisionAPIKeyForTests creates a test API key using the /api/v1/keys endpoint.
// It first tries without auth (in case API_KEY_REQUIRED=false in dev mode).
// Falls back to checking if there's an existing root key reference.
func bootstrapAPIKey(cfg *Config, suite *SuiteResult) string {
func provisionAPIKeyForTests(cfg *Config, suite *SuiteResult) string {
base := cfg.APIURL
// Try creating a key without auth (works if API_KEY_REQUIRED=false).
@@ -321,7 +321,7 @@ func bootstrapAPIKey(cfg *Config, suite *SuiteResult) string {
}
if json.Unmarshal([]byte(body), &resp) == nil && resp.RawKey != "" {
suite.Results = append(suite.Results, TestResult{
Name: "API key bootstrap (no auth)", Result: Warn, Severity: SevHigh,
Name: "API key provisioning (no auth)", Result: Warn, Severity: SevHigh,
Detail: "Key creation succeeded without API key — API_KEY_REQUIRED may be false",
})
return resp.RawKey
@@ -331,7 +331,7 @@ func bootstrapAPIKey(cfg *Config, suite *SuiteResult) string {
// If we got 401, auth is enforced. We need a key from the environment.
if code == 401 {
suite.Results = append(suite.Results, TestResult{
Name: "API key bootstrap", Result: Info, Severity: SevInfo,
Name: "API key provisioning", Result: Info, Severity: SevInfo,
Detail: "API requires authentication — set SIRIUS_API_KEY env var with a valid key",
})
}
@@ -339,8 +339,8 @@ func bootstrapAPIKey(cfg *Config, suite *SuiteResult) string {
// Last resort: check if the API returns something useful at health that hints at setup.
if strings.Contains(body, "error") {
suite.Results = append(suite.Results, TestResult{
Name: "API key bootstrap", Result: Skip, Severity: SevInfo,
Detail: fmt.Sprintf("Cannot bootstrap key: %s", truncate(body, 120)),
Name: "API key provisioning", Result: Skip, Severity: SevInfo,
Detail: fmt.Sprintf("Cannot provision key: %s", truncate(body, 120)),
})
}