- 29 Jun, 2026 10 commits
-
-
Refreshed the onboarding template (rendered PDF is gitignored, carries live tokens): -
✨ NEW §3 "The YouTube → emotion pipeline": /sources fetch → /jobs poll → /artifacts download → /analyze/emotion, the showcase end-to-end flow (curl + the @nech/api TS shape), idempotent-on-repeat noted. - Credentials block now carries BOTH tokens: the api:* Bearer AND the npm read token for installing @nech/api from npm.nech.pl. - Endpoint table gains /sources, /jobs/{id} (+DELETE), /artifacts/{cid}/{name}. - §1 surfaces the live /docs + /openapi.json links and corrects "access": only /healthz is public now; docs/spec need api:docs (covered by api:*). - Sections renumbered (pipeline=3, client=4, good-to-know=5, support=6).PLN (Algolia) authored -
So the session task board survives a push to git.nech.pl (it is local harness state, not in git). Two cold-readable TODOs next to their code: - armada/api/TODO.md — Fourier audio-API remaining: heavy chain #30 /separate + #31 /loops/grade/correlate (need a GPU runner), optional #47 X-Accel + #46 Grafana, follow-ups (nech_api python client, hexa npm read token); EPIC #21 closes with the chain. - tools/foundry/TODO.md — #19 auto-tune loop (in-progress, harness validated +10.5%, exact resume steps) + #20 batch-explore corpus. Plus archived #44 (Verdaccio + @nech/api live) to completed-archive.md. Root TODO.md left untouched (it is the paused Pulsar livecoding-perf session).
PLN (Algolia) authored -
Verdaccio is live on npm.nech.pl and @nech/api@0.1.0 is published + install-verified (AudioApi/Configuration import clean as a consumer). So the hand-written zero-dep nech.ts drop-in is retired (git rm); @nech/api is the one true client. README + onboarding.html now show the @nech scope install with the read token, and the docs row reflects that /docs + /openapi.json are bearer-gated (api:docs) not public. VERDACCIO.md marked DEPLOYED with the three gotchas folded in (listen 0.0.0.0, chown 10001, TLSv1.2-only) + the auth-gated-reads note.
PLN (Algolia) authored -
Rich entry for the documentary: the anonymous-exposure finding, the cross-cutting api:docs design (read-specs decoupled from call-API), the lockstep drift-guard dance, and the shared-repo rebase + diff-before-tee discipline that kept SRE work intact.
PLN (Algolia) authored -
Mirrors the merged platform canonical: required_for() maps docs|openapi.json|redoc → {realm}:docs. Byte-identical to nechapi/_platform/scopes.py (drift guard green). Part of #48; deploy = nechapi-platform redeploy + nginx reload.PLN (Algolia) authored -
Rich entries for the documentary trail: the freebox-deferred simplification of #27, the open-url-scope-with-apikey-trust decision on #29, the SSRF block set, and how #25/#26 pre-built the seams (result_ref/gc, alias table, born-done path) that made both mostly wiring.
PLN (Algolia) authored -
Two links of the heavy chain, built on the #25 job backbone. Both ship the CODE now; they go live the moment erable ssh is back (yt-dlp install) — no GPU needed. #27 — artifact store (artifacts.py). Big binaries (fetched sources, later stems/ loops) live content-addressed on local disk under FOURIER_ARTIFACTS, tracked as ordinary cache rows (kind→result_ref), so the existing LRU cache.gc() already evicts the coldest under a cap. Freebox was the original SSOT plan — deferred per PLN; erable-local for now. Serving is zero-copy via nginx X-Accel-Redirect (FOURIER_X_ACCEL), FileResponse fallback in dev. resolve() refuses path traversal / anything outside the root. #29 — /sources (engines/sources.py + POST /sources). The worker shells out to yt-dlp → bestaudio → 44.1k WAV (analysis-ready for /separate, /loops, features), content-addresses it, stores it, and aliases url→content_id so a repeat URL is an idempotent born-done job (no re-download). Submit returns 202 {job_id}; poll /jobs/{id}. URL scope is OPEN (any http(s)) per PLN — gated by the platform apikey + nechapi monitoring — but we still hard-block the SSRF footguns (non-http(s), localhost, RFC1918/link-local/reserved IPs; incl. the 169.254.169.254 metadata classic). The fetch is a thin seam so tests mock the network with a synthetic WAV. Also: GET /artifacts/{cid}/{name} serving endpoint; healthz reports yt-dlp presence; _meter now counts by ROUTE TEMPLATE not concrete path (else /jobs/{id} & /artifacts/{id} would mint unbounded Prometheus series). OpenAPI refreshed 13→17 ops, @nech/api regenerated (submitSource/getArtifact/getJob/cancelJob). yt-dlp added to requirements-deploy; DEPLOY.md §5b documents the yt-dlp+ffmpeg install and the nginx internal location. 78→95 tests green.PLN (Algolia) authored -
The audio API was internally codenamed "Douanier" (Le Douanier Rousseau — a customs-officer pun on edge auth). Renamed to "Fourier": the FFT is literally the transform behind /spectrum and most of the feature stack, and central auth is the platform’s job now, not ours — so the name should point at the signal work, not the gate. Internal-codename-only: the public contract (/audio/v1, gateway headers) is untouched. Mechanical case-aware sweep across armada/api + the completed-archive, plus hand-rewritten prose (the Rousseau attribution → Joseph Fourier; dropped the customs-gate /
🛂 metaphors). Renames: douanier.py → fourier.py, deploy/douanier-worker.service → deploy/fourier-worker.service. Env prefix DOUANIER_* → FOURIER_* (config contract; safe — nech.pl is pre-users), dev venv ~/.virtualenvs/douanier → fourier (shebangs fixed). 78 tests green after.PLN (Algolia) authored -
PLN (Algolia) authored
-
The async spine for the heavy chain (sources/separate/loops): the API enqueues and returns a job_id; a worker runs it out-of-band so the API never blocks. Built to the four decisions taken with PLN: - HYBRID sync/async: the 11 cheap analyses stay synchronous (cache-backed); only heavy work becomes a job. The line is wall-clock cost, not endpoint kind. - PER-ENGINE submit + SHARED poll: engines return 202 {job_id} (wired in #29/#30/#31); GET /jobs/{id} polls, DELETE /jobs/{id} cancels. Typed JobStatus response_model so the generated client gets audio.getJob()/cancelJob(). - SINGLE serialized worker: one job at a time (erable is 2 GB / no GPU; two demucs runs would OOM). The atomic claim is still race-safe for N workers. - Identity = gateway tenant; a job is visible only to its owner (cross-tenant poll/cancel → 404, not 403, so existence doesn't leak). Pieces: - db.py: jobs table (status/priority/progress/result/attempts/webhook) + a claim-ordered index. - jobs.py: enqueue (incl. born-done idempotent fast path when the cache already has the result), atomic claim (candidate → guarded UPDATE WHERE status= 'pending'; rowcount-0 retry; WAL serializes writers so no double-claim), progress/finish/fail-with-requeue-under-cap, crash recovery (running→pending on boot), cooperative cancel + is_cancelled checkpoints. - worker.py: serialized loop — recover → claim → dispatch by type → finish/fail → optional webhook; SIGTERM-graceful; imports engine handler modules (none yet, idles politely); engines register via @jobs.handler. - deploy/douanier-worker.service: systemd --user unit (linger) — the durable run path (harness/nohup jobs die on teardown). Prod container-vs-host wiring is the paved-road call (SRE/#34). VALIDATION: tests/test_jobs.py — submit→claim→done; born-done fast path; finish/requeue-to-cap; crash recovery; NO double-claim across 6 threads × 25 jobs (each claimed exactly once); worker dispatch to done; missing-handler error; exception→requeue; cancel mid-flight; HTTP poll + ownership 404 + idempotent cancel. 11 new tests, full suite 67→78 green. OpenAPI snapshot refreshed (15 ops) + @nech/api regenerated (getJob/cancelJob); noImplicitAny relaxed for the 100%-generated client (typescript-fetch's camel/snake guard trips TS7053).PLN (Algolia) authored
-
- 28 Jun, 2026 10 commits
-
-
The hand-written nech.ts was always interim; this replaces it with a client GENERATED from the OpenAPI spec, so the SDK can never drift from the API. Source-of-truth fixes (the spec drives the ergonomics): - Clean operationIds on all 13 routes (operation_id="analyzeEmotion" etc.) so the generator emits `audio.analyzeEmotion({file})`, not the default `analyzeEmotionAnalyzeEmotionPost`. Also cleans /docs. - One router tag ["audio"] so the generated class is AudioApi, not DefaultApi. - refresh_openapi.py: the canonical snapshot dump. app.openapi() omits the public `servers` block (only injected when served behind root_path), and the old README recipe silently dropped it — test_openapi_snapshot guards it, so the refresh now injects https://api.nech.pl/audio/v1 itself. The package (@nech/api, clients/nech-api/): - typescript-fetch generator → src/audio/ (checked in; regenerate via codegen.sh), bundled with tsup to a single ESM file + .d.ts. - Umbrella package, per-domain SUBPATH exports: `import {AudioApi} from '@nech/api/audio'` — one install/version, tree-shakeable, geo/iris slot in as siblings later (codegen.sh + exports map have the stubs). - Build via tsup not bare tsc: the generated code uses extensionless relative imports (./runtime) that Node ESM can't resolve from plain tsc output; bundling sidesteps it entirely. Verified end-to-end: `@nech/api/audio` resolves through the exports map, 13/13 methods present. - publishConfig + .npmrc point the @nech scope at https://npm.nech.pl. VERDACCIO.md: copy-paste runbook to stand the private registry up on erable (container :4873, nginx TLS vhost for npm.nech.pl, seed publisher + lock signups, publish). NOT yet run — `ssh erable` failed with publickey from the build host; needs the key loaded. DNS for npm.nech.pl is already set. README reframed: @nech/api is the official client; nech.ts stays documented as the working drop-in until the registry is live, then it's retired. Scrubbed a $DOUANIER_TOKEN codename leak in the curl example. 67 API tests still green.PLN (Algolia) authored -
PLN (Algolia) authored
-
The platform moved under us between sessions. SRE.md now records the landscape the audio service actually deploys into: - nechapi got its own repo (git@git.nech.pl:pln/nechapi.git) with a golden base image (nechapi/py) + a `nechapi ship` CLI — the paved road. Onboarding is FROM nechapi/py:1 + NECHAPI_ROOT_PATH + `nechapi ship`, not the old hand-rolled docker save/load. - A sibling tenant (geo/v1, Verniquet) is already live through it. - Correction to the 2026-06-25 "CLAP is the wrong engine" line: CLAP is right, used the canonical way — embeddings precomputed OFFLINE, a compact ANN index shipped to erable, request-time = tiny text-embed + cosine. What's wrong is running CLAP at request time on a GPU-less 2GB host. This closes the ops task (#34): observability shipped on the platform side (per-call capture, /admin analytics, public uptime status page — nechapi f89d7f9, now rebased onto the paved road and pushed), and the systemd-restart / deploy-runbook half is subsumed by `nechapi ship` + the golden base, owned in the SRE repo. No bespoke systemd units to write here.
PLN (Algolia) authored -
The 67-test suite existed and was green, but only under a --system-site-packages venv — which made it quietly host-dependent. A system python bump to 3.14 dropped fastapi/httpx from that venv and the whole suite went to "no tests collected" (red, but for an env reason, not a code reason). CI that inherits the host's site-packages would hide exactly this class of breakage. Fix = make the suite hermetic and prove it: - requirements-test.txt pins the real test surface — fastapi/uvicorn/ multipart (via requirements.txt) + numpy + soundfile + librosa + pyloudnorm + httpx + pytest. NOT --system-site-packages. - The heavy ML stack (torch, laion_clap, demucs) is deliberately absent: those engines aren't exercised by the tests, so CI installs in seconds- to-a-minute instead of pulling multi-GB wheels. - Rehearsed in a clean throwaway venv: first run surfaced two masked deps the system venv had been silently supplying — librosa (lazy-imported by signal/feats/ears/grade) and pyloudnorm (the LUFS loudness engine). Pinned both; clean-venv run is now 67 passed. - .gitea/workflows/api-ci.yml runs it on git.plnech.fr for any push/PR touching armada/api/**. Dormant until an act_runner is registered for the repo; the workflow is correct and locally rehearsed. Closes the test-suite + CI task (#35): the suite is reproducible and the pipeline is declared.
PLN (Algolia) authored -
Rich entries for the api.nech.pl platform build-out: first erable deploy + clone3/seccomp gotcha, the realm:domain:path scope convention + central enforcement, Shipow onboarding, and the observability stack (capture + admin analytics + public uptime). Source material for the documentary.
PLN (Algolia) authored -
The /docs + /openapi.json title read "Douanier — the audio sub-API…" and /healthz returned service:"douanier" — internal codename leaking to consumers. - FastAPI title → "Nech.PL Audio Intelligence API"; description rewritten to describe the engines (no "customs gate" framing). - /healthz service → "nech-audio". - Regenerated clients/openapi.json snapshot (info.title now clean; 13 paths, servers=api.nech.pl/audio/v1) — feeds the generated-client pipeline (#44). Built + redeployed; verified at the edge: openapi.json info.title and healthz both clean. "Douanier" now survives only as the internal repo/metric name. 67 tests green.
PLN (Algolia) authored -
"Douanier" is the internal codename for the audio sub-API; it shouldn't be what a consumer imports. Renamed the client-facing surface to the platform brand (NechAPI), keeping "Douanier" only as the internal service/repo name. - clients/nech.ts (was douanier.ts) — ONE umbrella `NechAPI` client, namespaced per sub-API: `new NechAPI({token}).audio.emotion(clip)`. Future sub-APIs add `nech.geo.…` with no import change. Also exports the standalone `NechAudio` sub-client for the smaller-bundle path. DouanierError→NechError, DouanierOptions→NechOptions. Typechecks clean under tsc --strict. - response header X-Douanier-Cache → X-Nech-Cache (app.py + all tests + client + docs). Verified live end-to-end: miss→hit, old header gone. - clients/README + onboarding.html (the Shipow PDF source) updated to NechAPI / nech.audio. PDF re-rendered. Built + redeployed douanier:latest to erable; 67 tests green. NOTE (next iteration): OpenAPI info.title and /healthz `service` still say "douanier" — a cosmetic /docs leak, scrub on the next redeploy.PLN (Algolia) authored -
A shareable getting-started for the hydra-live-hexa Studio: what the Audio Intelligence API does, base URL + bearer auth, the full endpoint list, a curl quickstart and the zero-dep TS/Vercel snippet, plus caching/limits/errors and support. Branded to the Nech.PL APIs / Ship's Bridge look; A4, print-clean. onboarding.html is the committed template (token placeholder __NECHPL_TOKEN__); render a per-tenant PDF with chromium --headless --print-to-pdf after sed-filling the key. The rendered PDF carries a live token, so clients/*.pdf is gitignored — never commit it; deliver it to the tenant over a private channel.
PLN (Algolia) authored -
The audio API now speaks the platform scope convention (nechapi scopes.py): access is hierarchical realm:domain:path and the required scope is DERIVED FROM THE ROUTE, so it's maintenance-free — add an endpoint and its scope exists. - scopes.py — vendored byte-for-byte from nechapi/_platform/scopes.py; a drift-guard test (test_scopes.py) fails if the two ever diverge, so the gateway and this service can never disagree on who's allowed in. - app.py — replaced the per-route require_scope("emotion"|"features"|…) strings with ONE path-derived dependency: `require` computes api:audio:<path> from the request and checks it; `require_auth` covers /me (any identity). Also closed a footgun: a gateway-injected request with a MISSING X-Scopes header now defaults to NO scopes (was "*"). - auth.py — Principal.has_scope is now the hierarchical matcher (api:audio:* authorizes api:audio:analyze:emotion, etc.). - tests — gateway-header tests grant api:audio:*; the scope-enforcement tests now prove real path-derivation (a sibling grant like api:audio:features → 403 on /grade and /onsets). +test_scopes.py for the matcher + drift guard. 67 passing. - clients/README — scope table rewritten to the convention (api:audio:<path>, grant api:audio:* or api:* for breadth). Validated end-to-end through https://api.nech.pl/audio/v1 with a freshly minted api:* token: /me → scopes [api:*]; /features 200 (cache miss→hit); a sibling scope 403s; no-identity 401s. Built + redeployed douanier:latest to erable (seccomp=unconfined per DEPLOY.md).PLN (Algolia) authored -
First real deploy of douanier:latest to erable went green, but only after diagnosing two host-specific traps that DEPLOY.md now records so the next deploy is one shot: 1. clone3 vs old seccomp — the container booted uvicorn then segfaulted (exit 139) / aborted with "OpenBLAS blas_thread_init: pthread_create failed … Operation not permitted". Root cause: Docker 19.03 on kernel 4.9's default seccomp profile rejects the clone3 syscall that python:3.12-slim's glibc 2.36 uses for pthread_create. Fix: run with --security-opt seccomp=unconfined (safe — the container is loopback-only behind the gateway). 2. BLAS thread pool on a small shared box — pinned OPENBLAS/OMP/NUMEXPR/MKL _NUM_THREADS=1 in the env file: belt-and-braces with the seccomp fix on the old kernel and right-sized for CPU-light work on 4 vCPU / ~2 GB. Also: data volume is /home/pln/srv/douanier/data (no sudo for /srv; it's pure transient cache so the path is immaterial). Verified end-to-end through the gateway: healthz/openapi/docs all 200, authed routes 401 without a token.
PLN (Algolia) authored
-
- 25 Jun, 2026 18 commits
-
-
PLN (Algolia) authored
-
hexa can now call the audio API with types, not guesswork. - clients/openapi.json — checked-in OpenAPI 3.1 snapshot (servers pinned to the public https://api.nech.pl/audio/v1), covering all 13 routes. A snapshot-drift guard test asserts it stays in sync with the live app (add a route → refresh or CI fails). - clients/douanier.ts — a typed, ZERO-dependency client (global fetch/FormData/Blob; works in Node 18+, Vercel Functions, Edge). One method per engine (emotion, features, samples, grade, onsets, waveform, analyze, loudness, spectrum, naming), typed results, X-Douanier-Cache surfaced as result._cache, DouanierError on non-2xx. Typechecks clean under tsc --strict. - clients/README.md — base URL + bearer, a Vercel Function example, the endpoint/ scope table, curl, and the openapi-generator one-liner for full codegen. 60/60 tests (added the snapshot guard). The API is now self-describing (/audio/v1/docs + openapi.json) and has a drop-in client.
PLN (Algolia) authored -
Three more torch-free, cached building blocks → 11 engines total. - POST /loudness → BS.1770 integrated LUFS (pyloudnorm) + sample/true-peak (4× oversample) + crest + the gain to hit each delivery target (-14 streaming, -9 club, reference_postprod_master). The mastering numbers PLN + hexa gate on. engines/loudness.py; pyloudnorm added to requirements-deploy (pure-python, tiny). - POST /spectrum → FFT-AS-A-SERVICE (PLN's ask): a downsampled, render-ready spectrogram bands×frames, 0..1 normalized; mel (perceptual, for visuals) or log-linear; frames=1 collapses to a single averaged FFT spectrum. Bounded payload so it caches cheaply. signal.spectrum(). - POST /naming → convention-compliant sample name (NN_role_character) from the MEASURED role + character, never the file name. naming.py vendored from the Foundry (drift-guarded); character_of() derives the adjective from features. Validated in the torch-free container (LUFS -19.1 w/ gain +5.1/+10.1; mel band centers; "07_melodic_warm" lint-clean). 58/58 tests (test_extra.py: engines + endpoints + naming drift guard + character_of). Image now serves emotion/features/ samples/grade/onsets/waveform/analyze/loudness/spectrum/naming + separate-503.
PLN (Algolia) authored -
Three cheap, torch-free, cached building blocks — the no-model tier, plus a convenience composite. The image now serves 8 engines (emotion/features/samples/ grade/onsets/waveform/analyze + separate-503). - POST /onsets → onset hit times (s) + tempo + onset rate. Rhythmic hits for visual sync / slicing. scope `onsets`. - POST /waveform → render-ready waveform: per-bin [min,max] in [-1,1] + a 0..1 RMS energy envelope (?bins= ≤4000). For hexa's audio-reactive visuals. scope `waveform`. - POST /analyze → emotion + features + sample role in ONE cached call (fewer round-trips for hexa); each is the same engine the dedicated routes use. scope `analyze`. engines/signal.py is self-contained librosa (onsets uses feature.tempo, the 0.11-correct path). Validated in the torch-free container (/onsets tempo 107.7 n=41; /waveform bins honored, peaks in range; /analyze returns all three blocks). 50/50 tests (added test_signal.py: engine + endpoint + cache + scope-gate + torch-free).
PLN (Algolia) authored -
Exposes the Foundry's mechanical loop-quality grader (the "katana") on /audio/v1/grade: upload a loop/one-shot → composite 0..1 + S/A/B/C/D tier, per-rule sub-scores (seam click, zero-crossing cleanliness, DC, bar self-consistency, level, bass mono-compat) + human-readable flags. scope `grade`, cached, torch-free. Directly serves "iterate on sampling quality". engines/grade.py is a BYTE-FOR-BYTE vendored copy of tools/foundry/engine/grade.py (self-contained — numpy/soundfile/pydantic/librosa, no Foundry/torch — because the container holds only armada/api/). It's kept identical on purpose, and tests/test_grade_endpoint.py is a DRIFT GUARD: it imports the Foundry canonical standalone and asserts the vendored copy grades identically (grade/tier/sub + WEIGHTS/THRESH) on a synth signal — a future Foundry tweak that isn't re-vendored fails CI here, not silently in prod (parsers-over-copy, applied to a vendored copy). Validated in the torch-free container (healthz engines now emotion/features/ samples/grade/separate; a sine tone grades D with a correct seam-click flag) + 43/43 tests. Note: this is the CPU-doable slice of #31's /loops /grade /correlate — the finder (/loops) and corpus correlation (/correlate) remain.
PLN (Algolia) authored -
Two new CPU-native, torch-free, cached endpoints on the audio sub-API — the "interesting value points" for hexa beyond emotion, each the same shape/pattern as /analyze/emotion (upload → content-address → cache → compute). - POST /features → the ~35-dim audio feature stack (spectral moments, MFCCs, chroma/key, envelope/attack-decay, + rhythm/tempo). scope `features`. - POST /analyze/samples → per-sample EDA + role (percs|bass|melodic|tops|atmos) decided by the MEASURED spectrum (centroid + band energy), never the name; an optional ?name= only disambiguates breaks/drums (feedback_mastering_eda). scope `samples`. Engine: engines/feats.py is SELF-CONTAINED (vendored DSP), like ears_light — the deployed container holds only armada/api/, not armada/tide-table/, so it can NOT import sample_features/audio_lens at runtime. It mirrors their algorithms and fixes the librosa-0.11 tempo bug (feature.tempo, not the removed feature.rhythm.tempo that silently dropped tempo in the tide-table original). Wiring: a shared _cached() helper now backs all three analyze routes (DRY); healthz advertises the engines map; both routes are scope-gated via the gateway X-Scopes. Validated in the torch-free container: /features 200 (41 features, tempo 107.7, key=Amaj), /analyze/samples role-by-measurement, 403 scope gate, X-Douanier-Cache hit on repeat. 40/40 tests (added test_feats.py + endpoint cases).
PLN (Algolia) authored -
The SRE edge is live (api.nech.pl returns 503 warming_up); the only thing blocking a green healthz was a CPU-deployable image. This ships it, and folds in the SRE's platform reframe that landed in the same letter. Baseline image - Dockerfile: python:3.12-slim + ffmpeg/libsndfile, the LIGHT torch-free stack only (requirements-deploy.txt: fastapi/uvicorn/librosa/numpy/soundfile). Builds to ~1 GB, runs well under the 2 GB-RAM erable budget. Default engine = light. - Validated in-container end to end: healthz green, emotion miss->hit cache, 401/403/200 auth gating, /metrics, /separate 503. Platform contract (SRE update: api.nech.pl is a multi-API gateway; we're the `audio` sub-API) - Public path is /audio/v1/...; the gateway strips the prefix and proxies to us at root. Routes moved off the /v1 router to root; root_path=/audio/v1 so OpenAPI/docs advertise the real public paths (verified servers=[{/audio/v1}]). - Central auth: dropped our own bearer verification in the request path. We now trust the gateway-injected X-Tenant / X-Scopes (loopback-only bind = only the gateway can reach us). Local-dev keeps a DOUANIER_DEV_TOKEN bearer fallback. Supersedes #23/#24 (the SQLite token store + CLI remain for dev only). - /metrics: Prometheus text (douanier_up, requests_total{path,status}, cache_rows/hits{kind}) for the erable scraper. - /separate: deliberate 503 compute_unavailable (retriable) — no GPU path on erable; route exists so hexa can code against it now. GPU backend is env-selected later. Docs + tests - DEPLOY.md rewritten as the erable container contract (build, /data volume, env-file, loopback publish, cache cap, central-auth onboarding). README reframed to the platform shape. 32/32 tests pass (smoke retargeted to root paths + header auth; added /separate, /metrics, openapi-root-path coverage).PLN (Algolia) authored -
Rich archive entries (blog/video source material) for the day's shipped work: the Douanier Audio-Intelligence API's first endpoints (#22/#36 scaffold+emotion walking skeleton, #23 SQLite auth+CLI, #26 content-addressed cache with the measured 5663x repeat speedup) and the Foundry sampling-classics demo sources. Each entry stands alone for a cold reader — goal, what shipped (commit hashes), non-obvious learnings + numbers.
PLN (Algolia) authored -
PLN's closed feedback loop: generate candidate loops with a settings profile → grade them on the full quality rubric → score the profile → search for the settings that maximise quality. Auto-tune the composite weights instead of hand-guessing them. WHY: the finder/grader/correlator exist; the composite weights W were "provisional, calibrate with #7". This builds the machine that calibrates them against measured grade — and measures the lift honestly rather than asserting a tuning is better. WHAT: - engine/autotune.py — evaluate(stems, settings) runs the finder with a profile, grades every produced candidate on the FULL rubric (grade.py adds dc / level / bass-mono / bar-consistency — features the finder's own score does NOT use, so tuning pulls in signal beyond seam/zc), aggregates objective = mean_grade × coverage. search() sweeps random|grid and always includes the current defaults (loops.W) as a baseline row so the report shows lift, not just a number. - autotune.py — CLI (--stems glob --n --method --max-stems); PII-safe aggregated leaderboard (counts, never filenames). - engine/loops.py — analyze_stem now accepts an optional `weights` override (defaults unchanged); the grid is preloaded once per stem (weight-independent) so the sweep is cheap — a scarcity-minded speedup. VALIDATION: - 4/4 mocked harness unit tests (aggregation, empty-set, ranked+baseline-included, weights threaded through to the finder). - Real end-to-end run, 2 drums stems × 8 configs: baseline objective 0.690 → best 0.763 (+10.5%); the defaults ranked LAST of 8 — the search found real lift. NOT DONE ON PURPOSE: the finder defaults (loops.W) are UNCHANGED. The winning profile (seam 0.4, zc 0.05, bars=(4,8)) is from a 2-stem sample AND seam/zc are in both the finder score and the grade (a confound) — so it's suggestive, not a mandate. Remaining for #19: recall-vs-provenance-GT as an anti-gaming 2nd objective, per-stem-role tuning (vocals=chops want different weights than drums), a full-corpus + demo-corpus campaign, an extract-once/score-many speedup, THEN adopt a validated profile as the default.
PLN (Algolia) authored -
The SRE reply (armada/api/SRE.md) revealed the public host is erable — Debian 9, no GPU, ~2 GB RAM, ~3 GB disk free — where the ~4 GB CLAP/torch image can't fit. This unblocks deployment in principle: a CPU-native emotion path that needs no torch, behind a selector so the rich CLAP engine stays the dev/GPU default. WHY: the SRE is "blocking on a CPU-deployable image". The emotion read must work within ~2 GB RAM and sub-second, without torch — but the API response shape must not change, so hexa's integration and the cache are unaffected by the swap. WHAT: - engines/ears_light.py — heuristic valence/arousal from librosa features only (no torch): arousal from RMS energy + tempo + spectral brightness; valence from major/minor mode (Krumhansl key-profile correlation) + brightness. Mapped onto the SAME 12 emotion-ontology anchors by V/A distance → a CLAP-shaped {valence, arousal, top, dist, confidence} dict, tagged engine="light". Honest baseline; an Essentia/CLAP-grade precise tier lands later behind the same shape. - config.DOUANIER_EMOTION_ENGINE (clap|light); ears.emotion_read() dispatches; cache keys on the engine so clap/light reads don't collide; healthz reports the selected engine + availability. VALIDATION: - 27/27 tests green (+5): anchors drift-guarded == emotion_ontology.EMOTIONS; mode-valence major>minor; light read shape + V/A in range + dist sums to 1; arousal orders loud/bright/noisy above quiet/low; and the endpoint runs end-to-end via TestClient with engine=light — the torch-free erable path proven. REMAINING in #37 (queued, not rapid): Dockerfile (light image, no torch, fits ~3 GB / 2 GB RAM, bind 127.0.0.1:9780, --env-file); Essentia MusiCNN upgrade for the light tier; separation /v1/separate → 503 + env-selected GPU-runner dispatch seam (with #29/#30); /v1/metrics; 2 GB cache hard-cap. See SRE.md + memory project_douanier_api 'HOSTING REALITY'.PLN (Algolia) authored -
Add a curated list of 10 sampling classics as built-in demo sources — fuel for demoing the Foundry, precomputing a fixed validation corpus, and iterating on loop/chop quality (#19 autotune input). WHY: we need a stable, pedagogically-diverse set to validate the finder across material types instead of ad-hoc URLs. The set spans the axes that stress the finder differently: canonical drum BREAKS (Amen, Funky Drummer, Apache — the gold standards; if the rubric can't nail these it's wrong), a BASS-defining groove (Chic – Good Times), VOCAL/no-drums (Loituma), ORCHESTRAL/no-drums edge case (Mozart 40 — exercises the no-drums grid fallback), a full clean POP mix (Rickroll), hip-hop sample-collage (Grandmaster Flash Wheels of Steel, Humpty Dance), and a drum+vocal-stab combo (Lyn Collins – Think). WHAT: - demos.json — authored data (slug/title/year/source/why/expect/tags). Each entry flags drums-presence because the finder's grid-from-drums path depends on it (no-drums → bass/first-stem fallback, per the recall notes). - engine/demos.py — loader (load / by_slug / source_of). - foundry.py "demos" command — list, or precompute via --catch <slug> / --all (+ --sep to separate). This batch path is exactly what #19 autotune consumes. - server.py — GET /api/demos; /api/fetch now accepts "ytsearch1:" queries too. - ui/index.html — a 'sampling classics' quick-pick strip (click → fills the URL, tooltip shows why/expect). HONESTY ON LINKS: only Loituma (from the repo's own test fixture) and Rickroll (universally known id) ship as verified watch URLs (✓). The rest use "ytsearch1:" queries (≈) that resolve to the top hit at fetch time — so we never ship a guessed video id that silently breaks. yt-dlp accepts both forms. VALIDATION: foundry demos lists 10 (✓/≈ marked); /api/demos serves the JSON; UI strip renders; loader resolves source_of('amen-break').PLN (Algolia) authored -
Identical audio is analyzed ONCE and served forever. This is the economic core that makes cheap-analysis-at-scale viable, and it makes the live emotion endpoint feel instant on repeats. WHY: per the design, repeat calls must cost ~nothing — that's the margin story vs cloud egress. A re-analyzed track shouldn't pay the CLAP compute twice. WHAT: - cache.py — content_id = sha256 of DECODED PCM (+ samplerate) so re-encodes / re-uploads of the same sound dedupe; raw-bytes fallback for formats we can't decode here. params_hash folds engine settings (model, bars…) so different params cache separately. get/put (JSON inline or a result_ref path for big artifacts), yt-id/url → content_id aliases (#29 will use them), and an LRU/size-cap gc() that unlinks evicted artifact files (cache is transient on the freebox — rebuildable). - db.py — cache + aliases tables (WAL already on), LRU index on (kind, last_access). - app.py — /v1/analyze/emotion now content-addresses the upload, serves cached reads with X-Douanier-Cache: hit, computes+stores on miss. Engine call still behind engines.ears (unchanged contract). - douanier.py — cache stats / gc admin commands. VALIDATION: - 22/22 tests green (+7 cache: stable/decode-invariant content id, raw fallback, miss→put→hit, params separate entries, hit counter, alias resolve, LRU gc spares the recently-touched entry). - Real-CLAP end-to-end: same clip twice → call 1 miss 10.6s, call 2 hit 0.002s = 5663x speedup, identical V/A. The headline product benefit, measured.
PLN (Algolia) authored -
Replace the walking skeleton's hardcoded dev-token stub with a real, self-hosted token store — the customs gate's papers check — keeping the dev token as a documented local-only bootstrap. WHY: hexa needs a proper 31-day scoped token (#33), and every future engine endpoint needs per-engine authorization. Self-hosted SQLite (no vendor, same DB that'll hold jobs/cache/usage) matches the 'self-host all' decision. WHAT: - db.py — stdlib sqlite3, WAL mode so the API + worker daemon read/write concurrently. Tables: accounts, tokens (token_hash, prefix, scopes, quota_json, expires_at, revoked). The DB is the one thing worth backing up; the artifact cache is rebuildable (noted in SRE.md). - auth.py — mint() returns the plaintext ONCE and stores only sha256(token), so a DB leak isn't replayable. verify() is an indexed hash lookup → a Principal (account, scopes, expiry); checks revoked + expiry. has_scope() honors the '*' wildcard. Scope-gated dependency require_scope(scope) in app.py: 401 unknown/expired/revoked, 403 scoped-out; scope=None = any valid token. - douanier.py — admin CLI (token mint/list/revoke, db init); the only way tokens are created (no public signup). list masks to a 6-char prefix. - /v1/me echoes the caller's identity (onboarding smoke test); /v1/analyze/emotion now requires the 'emotion' scope. VALIDATION: - 15/15 tests green (8 auth: mint→verify, wildcard, unknown→None, revoke, expiry, never-expires, only-hash-stored, dev-bootstrap; 7 smoke incl. /me guarded). No CLAP/torch needed; DB points at a temp file via conftest. - CLI smoke end-to-end: mint prints secret once + masks in list, expiry lands exactly 31 days out (2026-07-26), revoke flips state to REVOKED.
PLN (Algolia) authored -
Stand up Douanier — the ParVagues Audio-Intelligence API — as a new self-hosted service in armada/api/, and prove the lean MVP vertical end-to-end before investing in the auth/quota/jobs/cache machinery it'll later sit on. WHY: hexa (Shipow's hydra-live-hexa Studio) wants to call our engines (stems, loops, emotion, samples, features) live from his Vercel world. Rather than build six foundational tasks before the first useful call, we ship a walking skeleton: scaffold + ONE real synchronous endpoint + a stub token, to de-risk the three seams that actually matter — engines import cleanly, the service runs, a client can auth & call. WHAT: - FastAPI app (app.py) versioned at /v1: /v1/healthz (status + engine availability) and POST /v1/analyze/emotion (upload clip -> valence/arousal + top emotions), guarded by a hardcoded dev bearer that FAILS CLOSED (503 if no token configured, never accidentally unauthenticated). - engines/ adapter layer with lazy heavy imports — ears.emotion_of() wraps the real tide-table seam: sample_semantics.embed_audios -> emotion_ontology.score. The route never touches the engine directly, so #28 can wrap it with cache+async later without changing the public shape. - config.py (env-overridable paths/token/limits); requirements.txt documents the --system-site-packages venv trick (reuse the box's multi-GB torch/CLAP/demucs stack, add only FastAPI on top — Arch PEP-668 blocks system pip). - SRE.md: a context letter to whoever maps the public api.nech.pl path — the 10 hosting decisions that are theirs (path vs host routing, systemd vs Docker, ports, TLS, GPU dispatch, freebox mount, secrets, observability, WAF, CORS). - README.md + smoke tests. VALIDATION: - 5/5 smoke tests green (healthz; auth fails-closed/opens; happy-path mocked; 413 oversize) with no CLAP/torch needed. - Real engine seam confirmed importable, and the FULL pipeline run end-to-end on a synthetic clip: real CLAP read, valence 0.154 / arousal 0.141, V/A in range, 34.4s cold (model load — which is precisely why #28 warms CLAP once in the worker rather than per request).
PLN (Algolia) authored -
Structured archive of the night's 11 completed tasks (#5-#13,#16-#18 + audio fix) — grader/tierlist/correlation/finder/naming/merge, with the load-bearing learnings and measured numbers (ρ=0.34, recall 0.27→0.34, the 56min→coarse-to-fine + silent-window ∞ + PLP-octave + HTTP/1.0-audio gotchas). Blog/video source material; the board now shows only live work (#19 auto-tune, #20 batch-explore).
PLN (Algolia) authored -
Two PLN UX fixes from auditioning the loops panel: 1. Chops were way too short (0.06 s slices = clicks, not cuts). analyze_chops now floors at min_len_s=0.25 and extends to onset→skip-two for phrase-length options; vocal chops now land 0.29–1.07 s (verified) — actual word/phrase chops. 2. A 10+ vertical list was impractical to navigate. Replaced it with a TIMELINE overview (the research §6 design): every candidate is placed on the track by its start/length, coloured by lead stem, height + opacity by score — so you SEE where the candidates are at a glance and click one to focus. Below sits a single detail panel (stems, waveforms, audition, keep-toggles, Forge) for the selected candidate, plus prev/next stepping and a "candidate N / M" counter. Scannable map + one focused editor instead of a scroll-wall. Verified in Chromium: 8 segments render, click-to-focus (candidate 1→4), prev/next, no console errors; stem durations now display (the HTTP/1.1 + preload audio fix shows 2:43). Works for loops and chops modes (header reads "N-bar" or "chop · Ns").
PLN (Algolia) authored -
PLN's "merge of stems": the best vocal isolation cascaded into the best 4-way split. There is no 4-stem Roformer, so MultiModelBackend orchestrates a cascade — 1. Roformer (audio-separator, py3.12 venv) isolates vocals + instrumental; 2. demucs runs on the INSTRUMENTAL (vocals already gone → cleaner) for drums/bass/other; 3. merge = Roformer vocals ⊕ demucs drums/bass/other in the standard 4-stem contract. Refactored separate.py for composite backends: a backend declares what it `produces` and (for composites) a `run_composite` hook; the subprocess runner is factored into _run_cmd so the cascade can drive several sub-separations. RoformerBackend is now wired (was a stub): audio-separator CLI, 2-stem (vocals/instrumental), normalize() globs the "(Vocals)"/"(Instrumental)" output names. The default normalizer maps by produces[]. Verified end to end on the Loituma source: merge → vocals (Roformer) + drums/bass/other (demucs-on-instrumental), 4 clean PCM_16/44.1k stems. All three backends report available on this box. GUI renders whatever stems a backend emits (demucs/merge 4, roformer 2 incl. instrumental), canonical roles first. 53 tests (added: registry has the three engines, roformer produces 2 + builds its cmd, merge is composite/4-stem). Keeps the 2-stem Roformer door open as its own backend (vocal isolation), and the merge as the flagship. Foundry's engine registry now showcases the pluggable design PLN wants to grow into a product/API/service.
PLN (Algolia) authored -
PLN couldn't hear the Loituma stems and wanted the original too. The bytes/format were fine (PCM_16/44.1k, audio/x-wav, 206 Range all verified) — the problems were presentation + protocol: - Both audio servers defaulted to HTTP/1.0, which closes the connection after each response. <audio> seeking fires many small Range requests, so HTTP/1.0 makes playback/seek flaky in real browsers. Set protocol_version="HTTP/1.1" on the Foundry Handler and armada RangeHandler (every response sets an exact Content-Length, so keep-alive is safe). Reliable streaming + seeking now. - The stem players used preload="none", so they showed a misleading "0:00 / 0:00" until play — easy to read as "broken". Now preload="metadata": durations load and display (verified 164s on all five). - The ORIGINAL is now offered on separated cards too (an "original" row), not just on raw catches — so PLN can A/B the source against each stem. Verified end to end in Chromium: original + all 4 stems load 164s duration and currentTime advances on play, no errors.
PLN (Algolia) authored
-
- 23 Jun, 2026 2 commits
-
-
Follows 720e14d0 (canonical roots + catalog regen). Sweeps the rename through the authored layer that the catalog generators don't own, and corrects the release artifacts: Authored text (hand-fixed): backlog.md, release_priority.md, manifeste/calendar.md (setlist mentions); master_edl_take89.json (boundary name + bleed reason — Take89 is the Montreuil set); boundaries_take89_validated.json (ear-validated cut-6 label). Regenerated from those: boundary_bleed_take89.json. Found+fixed a hardcoded SETLIST in punkachien/build_stemmap_html.py (the name was baked into the generator, not read from data) and regenerated stemmap.html. Release masters (#29 + #22): retagged all 15 FLACs in Prod/Montreuil26_master/tracks_bandcamp/ — album → "Montreuil Algorave V3 — Mai Floral" (the CANONICAL tracks.json title; the old tag said "…V3 — Live@Les Nouveaux Sauvages", a venue string not the set title). Track 08 title → "Mafia Sans Serif" and file renamed 08-Aria_Sans_Serif.flac → 08-Mafia_Sans_Serif.flac. (Prod is plain files, not git — FLACs are the artifact.) Deliberately NOT touched: sources/soundcloud.jsonl is an EXTERNAL MIRROR of the published SC upload (which itself still reads "Aria Sans Serif" + "Montreuil Algorave III") — editing it would desync it from reality; the real fix is publish-side (tracked as a follow-up). eda_report.json carries the name as a cosmetic label and is freebox-EDA-gated — it refreshes on the next EDA pass, not worth an expensive regen now. The hexa public-audio copy of the FLACs is a downstream sync (Shipow's repo) — flagged, not edited here.
PLN (Algolia) authored -
Two findings while fixing the #22 track-name typo, both surfaced by tracing where the name actually lives rather than sed-ing the blast radius: PARSER BUG — the catalog generators were reading a dead path. Commit 6e51fff hoisted the site app out of next/ to the repo root, but five tide-table modules still pointed LIVES at /Work/Web/www/next/content/lives — which is now EMPTY. The committed catalog/map JSONs were stale pre-hoist artifacts; any regen would have silently produced an empty catalog. Fixed the path in build_track_recording_map, build_catalog_view, pattern_ngrams, tide_eda, boundary_bleed (+ 4 docs/README/conftest that documented the dead path). This is exactly the "a parser miss must never masquerade as data" footgun — the path was wrong at the source while the output looked plausible. RENAME — the track was renamed Aria→Mafia Sans Serif long ago in the score, but the canonical site tracks.json still carried name "Aria Sans Serif" AND a dangling file ref (aria_sans_serif.tidal, which no longer exists; the real score is mafia_sans_serif.tidal). Fixed at the canonical root (tracks.json, committed separately in the www repo) and REGENERATED the catalog via `tide.py build` — not hand-edited — so catalog.generated / catalog_view / pattern_registry / track_recording_map all pick up the rename from source. Validation: regen diff is clean — only the Mafia rename plus one legitimate side-effect (the catalog re-read the *current* mafia_sans_serif.tidal source, which had resetCycles commented out since the last build; the old cached source was from the pre-rename file). No other track's data moved (verified by grepping the diff for non-Mafia id/title/track changes — none). Neighbor-list reordering in pattern_registry is benign. Test suite: 80 pass, 1 fail — the failure (38c3-toilet recovers 0 backlog tracks) is PRE-EXISTING (reproduced with these changes stashed), a real data gap belonging to #27/#66, not this change. Interim n=172 emotion calibration JSON left uncommitted (regenerates at 534).
PLN (Algolia) authored
-