Commit 3ed54057 by PLN (Algolia)

docs(tasks): archive #25 (async job backbone + worker)

parent 17d3f468
......@@ -350,3 +350,9 @@ memory `project_douanier_api`. Named after Le Douanier Rousseau (the customs gat
CI is dormant until an act_runner is registered for the repo; the workflow is correct + locally
rehearsed.
- **Deps:** part of EPIC #21; was a blocker on #21.
## #25 — Async job model + worker daemon (SQLite)
- **Description:** the async backbone so heavy work (sources/separate/loops) runs out-of-band and the API stays responsive.
- **Done:** committed 17d3f46. Four design decisions taken with PLN (AskUserQuestion w/ sketches): HYBRID sync/async (cheap analyses stay sync, heavy = jobs), PER-ENGINE submit (202 {job_id}) + SHARED poll (GET /jobs/{id}) + cancel (DELETE), SINGLE serialized worker, X-Accel-Redirect for artifacts. Built: jobs table (db.py), jobs.py (enqueue + born-done idempotent fast path, atomic claim, progress/finish/fail-requeue-under-cap, crash recovery, cooperative cancel), worker.py (serialized recover→claim→dispatch→webhook loop, SIGTERM-graceful, @jobs.handler registry), typed JobStatus endpoints, deploy/douanier-worker.service (systemd --user). 11 new tests (67→78), incl. a real 6-thread×25-job no-double-claim concurrency test. OpenAPI refreshed (15 ops) + @nech/api regenerated (getJob/cancelJob).
- **Learnings:** SQLite WAL serializes writers, so the claim race reduces to candidate-select → `UPDATE ... WHERE id=? AND status='pending'` + rowcount-0 retry — no locks needed, provably exactly-once across threads. typescript-fetch's generated instanceOf guard indexes a camel/snake key union that trips TS7053 under noImplicitAny → relaxed noImplicitAny for the 100%-generated package (emitted .d.ts stays fully typed). MID-BUILD: the btrfs disk hit ENOSPC at 98% (20G "free" but unallocated, metadata-full) — writes failed intermittently; cleared ~5.7G of regenerable caches (npm + ~/.cache browser/HF, user-approved) to continue. Disk still 97% — flagged for real cleanup.
- **Deps:** was blocked by scaffold + central auth (done); unblocks #29 → #30 → #31. Worker prod container-vs-host wiring is a paved-road residual (SRE/#34).
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment