6bf8bebf51
CI / Test and Build (push) Failing after 1s
CI / Migrate Dev DB (push) Has been skipped
CI / Migrate DB (push) Has been skipped
CodeQL / Analyze actions (push) Has been cancelled
CodeQL / Analyze javascript-typescript (push) Has been cancelled
CI / Detect Version (push) Has been cancelled
CI / Detect Desktop Changes (push) Has been cancelled
CI / Build AMD64 (blacksmith-2vcpu-ubuntu-2404, ./docker/cron.Dockerfile, ubuntu-latest, ghcr.io/simstudioai/cron) (push) Has been cancelled
CI / Build AMD64 (blacksmith-2vcpu-ubuntu-2404, ./docker/db.Dockerfile, ECR_MIGRATIONS, ubuntu-latest, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build AMD64 (blacksmith-4vcpu-ubuntu-2404, ./docker/pii.Dockerfile, ECR_PII, ubuntu-latest, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build AMD64 (blacksmith-4vcpu-ubuntu-2404, ./docker/realtime.Dockerfile, ECR_REALTIME, ubuntu-latest, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build AMD64 (blacksmith-8vcpu-ubuntu-2404, ./docker/app.Dockerfile, ECR_APP, linux-x64-8-core, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (blacksmith-4vcpu-ubuntu-2404-arm, ./docker/cron.Dockerfile, ubuntu-24.04-arm, ghcr.io/simstudioai/cron) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (blacksmith-4vcpu-ubuntu-2404-arm, ./docker/db.Dockerfile, ubuntu-24.04-arm, ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (blacksmith-4vcpu-ubuntu-2404-arm, ./docker/pii.Dockerfile, ubuntu-24.04-arm, ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (blacksmith-4vcpu-ubuntu-2404-arm, ./docker/realtime.Dockerfile, ubuntu-24.04-arm, ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build ARM64 (GHCR Only) (blacksmith-8vcpu-ubuntu-2404-arm, ./docker/app.Dockerfile, linux-arm64-8-core, ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Check Docs Changes (push) Has been cancelled
Publish CLI Package / publish-npm (push) Has been cancelled
Publish Python SDK / publish-pypi (push) Has been cancelled
CI / Deploy Trigger.dev (Dev) (push) Has been cancelled
Helm Chart / Lint, test, and validate chart (push) Has been cancelled
Helm Chart / Chart version bumped (push) Has been cancelled
Publish TypeScript SDK / publish-npm (push) Has been cancelled
CI / Build Dev ECR (blacksmith-8vcpu-ubuntu-2404, ./docker/app.Dockerfile, ECR_APP, linux-x64-8-core) (push) Has been cancelled
CI / Promote Images (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/cron) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/migrations) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/pii) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/realtime) (push) Has been cancelled
CI / Build Dev ECR (blacksmith-2vcpu-ubuntu-2404, ./docker/db.Dockerfile, ECR_MIGRATIONS, ubuntu-latest) (push) Has been cancelled
CI / Build Dev ECR (blacksmith-4vcpu-ubuntu-2404, ./docker/pii.Dockerfile, ECR_PII, ubuntu-latest) (push) Has been cancelled
CI / Build Dev ECR (blacksmith-4vcpu-ubuntu-2404, ./docker/realtime.Dockerfile, ECR_REALTIME, ubuntu-latest) (push) Has been cancelled
CI / Create GHCR Manifests (ghcr.io/simstudioai/simstudio) (push) Has been cancelled
CI / Process Docs (push) Has been cancelled
CI / Create GitHub Release (push) Has been cancelled
CI / Check Desktop Signing Secrets (push) Has been cancelled
CI / Desktop Release (push) Has been cancelled
CI / Create Desktop Prerelease (push) Has been cancelled
CI / Desktop Prerelease Build (push) Has been cancelled
CI / Publish Desktop Prerelease (push) Has been cancelled
CI / Prune Desktop Prereleases (push) Has been cancelled
Helm Chart / Install on kind and run helm test (push) Has been cancelled
70 lines
3.3 KiB
Docker
70 lines
3.3 KiB
Docker
# ========================================
|
||
# Combined Presidio service (analyzer + anonymizer) on a single port (5001).
|
||
# CPU spaCy NER + regex/checksum pattern recognizers.
|
||
#
|
||
# Source files are COPY'd last so code edits never re-download deps or models.
|
||
# ========================================
|
||
FROM python:3.12-slim-bookworm AS base
|
||
|
||
WORKDIR /app
|
||
|
||
# build-essential for any sdist that compiles native deps (e.g. blis/thinc).
|
||
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
|
||
--mount=type=cache,target=/var/lib/apt,sharing=locked \
|
||
apt-get update && apt-get install -y --no-install-recommends \
|
||
build-essential curl ca-certificates \
|
||
&& rm -rf /var/lib/apt/lists/*
|
||
|
||
# Pinned Python deps. Separate layer so source edits don't reinstall them.
|
||
COPY apps/pii/requirements.txt ./requirements.txt
|
||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||
pip install -r requirements.txt
|
||
|
||
# Pinned spaCy models (en + es/it/pl/fi, ~2.2GB total). Downloaded with
|
||
# retries/resume — the large wheels truncate on flaky networks if pip fetches
|
||
# the URLs directly.
|
||
ARG SPACY_MODELS="en_core_web_lg-3.8.0 es_core_news_lg-3.8.0 it_core_news_lg-3.8.0 pl_core_news_lg-3.8.0 fi_core_news_lg-3.8.0"
|
||
RUN --mount=type=cache,target=/root/.cache/pip \
|
||
for model in ${SPACY_MODELS}; do \
|
||
whl="${model}-py3-none-any.whl"; \
|
||
curl -fL --retry 5 --retry-delay 5 --retry-all-errors -C - \
|
||
-o "/tmp/${whl}" \
|
||
"https://github.com/explosion/spacy-models/releases/download/${model}/${whl}" || exit 1; \
|
||
done && \
|
||
pip install /tmp/*.whl && \
|
||
rm /tmp/*.whl
|
||
|
||
RUN groupadd -g 1001 pii && \
|
||
useradd -u 1001 -g pii pii && \
|
||
chown -R pii:pii /app
|
||
|
||
COPY --chown=pii:pii apps/pii/server.py ./
|
||
|
||
USER pii
|
||
|
||
# Listen on 5001. Runs as its own ECS service (separate task), reached via PII_URL;
|
||
# 5001 avoids colliding with the app's 3000 in local/compose runs on one host.
|
||
EXPOSE 5001
|
||
|
||
# Per-pattern regex match timeout (Presidio's `regex`-module `finditer(timeout=...)`),
|
||
# an interactive backstop against a catastrophic user-supplied custom regex. Presidio
|
||
# logs and skips a pattern that exceeds it — the request stays up rather than hanging.
|
||
# Set well below the 60s library default so a pathological pattern can't stall a worker.
|
||
# MUST be an integer — Presidio parses it with `int()`, so a float (e.g. 1.5) crashes
|
||
# the service at import.
|
||
ENV REGEX_TIMEOUT_SECONDS=2
|
||
|
||
# start-period covers the model cold start. With PII_WORKERS>1 each worker loads
|
||
# the five spaCy models independently and in parallel, so allow generous headroom
|
||
# (memory-bandwidth contention stretches the wall-time beyond the single-worker case).
|
||
HEALTHCHECK --interval=30s --timeout=5s --start-period=300s --retries=3 \
|
||
CMD curl -fsS http://localhost:5001/health || exit 1
|
||
|
||
# Worker count is env-driven so ONE image scales per task size: set PII_WORKERS to
|
||
# the task's vCPU count (each worker loads the models independently, ~3 GB each, so
|
||
# size task memory ≈ PII_WORKERS × 3 GB + overhead). Defaults to 1 for local/small.
|
||
# `sh -c exec` expands the env var while keeping uvicorn as PID 1 for clean SIGTERM.
|
||
# Quote the expansion so a malformed PII_WORKERS fails uvicorn arg-parsing rather
|
||
# than being interpreted by the shell.
|
||
CMD ["sh", "-c", "exec uvicorn server:app --host 0.0.0.0 --port 5001 --workers \"${PII_WORKERS:-1}\""]
|