Commit fbe1742d by PLN (Algolia)

docs(douanier): capture the two erable deploy gotchas + go-live (#42)

First real deploy of douanier:latest to erable went green, but only after
diagnosing two host-specific traps that DEPLOY.md now records so the next
deploy is one shot:

1. clone3 vs old seccomp — the container booted uvicorn then segfaulted
   (exit 139) / aborted with "OpenBLAS blas_thread_init: pthread_create failed
   … Operation not permitted". Root cause: Docker 19.03 on kernel 4.9's default
   seccomp profile rejects the clone3 syscall that python:3.12-slim's glibc 2.36
   uses for pthread_create. Fix: run with --security-opt seccomp=unconfined
   (safe — the container is loopback-only behind the gateway).
2. BLAS thread pool on a small shared box — pinned OPENBLAS/OMP/NUMEXPR/MKL
   _NUM_THREADS=1 in the env file: belt-and-braces with the seccomp fix on the
   old kernel and right-sized for CPU-light work on 4 vCPU / ~2 GB.

Also: data volume is /home/pln/srv/douanier/data (no sudo for /srv; it's pure
transient cache so the path is immaterial). Verified end-to-end through the
gateway: healthz/openapi/docs all 200, authed routes 401 without a token.
parent 397670c1
...@@ -51,6 +51,11 @@ mkdir -p /srv/douanier/data # SRE-owned; cap at 2 GB (see §5) ...@@ -51,6 +51,11 @@ mkdir -p /srv/douanier/data # SRE-owned; cap at 2 GB (see §5)
# image default DOUANIER_EMOTION_ENGINE=light is correct for erable — leave unset # image default DOUANIER_EMOTION_ENGINE=light is correct for erable — leave unset
# image default DOUANIER_ROOT_PATH=/audio/v1 matches the gateway — leave unset # image default DOUANIER_ROOT_PATH=/audio/v1 matches the gateway — leave unset
DOUANIER_MAX_UPLOAD_MB=40 DOUANIER_MAX_UPLOAD_MB=40
# Single-thread BLAS on erable's old kernel (see §4 gotcha #2):
OPENBLAS_NUM_THREADS=1
OMP_NUM_THREADS=1
NUMEXPR_NUM_THREADS=1
MKL_NUM_THREADS=1
# DOUANIER_TENANT_HEADER / DOUANIER_SCOPES_HEADER default to X-Tenant / X-Scopes; # DOUANIER_TENANT_HEADER / DOUANIER_SCOPES_HEADER default to X-Tenant / X-Scopes;
# override only if the platform injects different header names. # override only if the platform injects different header names.
# Do NOT set DOUANIER_DEV_TOKEN in prod (it's a local-dev-only bearer fallback). # Do NOT set DOUANIER_DEV_TOKEN in prod (it's a local-dev-only bearer fallback).
...@@ -60,8 +65,9 @@ DOUANIER_MAX_UPLOAD_MB=40 ...@@ -60,8 +65,9 @@ DOUANIER_MAX_UPLOAD_MB=40
```bash ```bash
docker run -d --name douanier --restart unless-stopped \ docker run -d --name douanier --restart unless-stopped \
--security-opt seccomp=unconfined \
--env-file ~/.config/douanier/douanier.env \ --env-file ~/.config/douanier/douanier.env \
-v /srv/douanier/data:/data \ -v /home/pln/srv/douanier/data:/data \
-p 127.0.0.1:9780:9780 \ -p 127.0.0.1:9780:9780 \
douanier:latest douanier:latest
``` ```
...@@ -70,6 +76,23 @@ docker run -d --name douanier --restart unless-stopped \ ...@@ -70,6 +76,23 @@ docker run -d --name douanier --restart unless-stopped \
gateway is the *only* way in, so the injected `X-Tenant`/`X-Scopes` headers can be gateway is the *only* way in, so the injected `X-Tenant`/`X-Scopes` headers can be
trusted. SRE wraps this in a keep-alive systemd unit. trusted. SRE wraps this in a keep-alive systemd unit.
### Two erable gotchas (verified at first deploy, 2026-06-28)
1. **`--security-opt seccomp=unconfined` is REQUIRED.** erable runs Docker 19.03
on kernel 4.9 (Debian 9); its default seccomp profile rejects the `clone3`
syscall that the image's glibc 2.36 (python:3.12-slim) uses for
`pthread_create`. Without the flag the container starts uvicorn, then
segfaults (exit 139) / aborts with `OpenBLAS blas_thread_init: pthread_create
failed … Operation not permitted` the moment numpy/librosa touch threads.
Unconfined seccomp is safe here: the container is loopback-only behind the
gateway. (Real fix is a newer Docker; this is the pragmatic one.)
2. **Single-thread the BLAS stack** via the env file (`OPENBLAS_NUM_THREADS=1`,
`OMP_NUM_THREADS=1`, `NUMEXPR_NUM_THREADS=1`, `MKL_NUM_THREADS=1`). Belt-and-
braces with #1 on the old kernel, and right-sized for a shared 4-vCPU / ~2 GB
box doing CPU-light work — avoids OpenBLAS spawning a pool that fights for RAM.
3. **Data volume** is `/home/pln/srv/douanier/data` (no sudo for `/srv`); it's
pure transient cache, so the path is immaterial — just keep it capped (§5).
Verify on the box, then from the internet: Verify on the box, then from the internet:
```bash ```bash
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment