batchCooking/docker-compose.yml
Nicolas 393300205d
Some checks failed
CI / build (push) Successful in 1m27s
CI / e2e (push) Successful in 6m39s
CI / lint (push) Successful in 1m41s
CI / intent-service-test (push) Successful in 3m4s
CI / test (push) Failing after 12m43s
perf(tech-steps): corrige la largeur du textcat, entrainement 32x plus rapide
Le service mettait ~930 s (15-16 min) a devenir `healthy`, ce que le README
presentait comme "un compromis assume, pas un defaut de configuration a
corriger". Mauvais diagnostic : le cout ne venait ni des iterations ni de la
taille du corpus, mais du parametre `length` de `spacy.TextCatBOW.v3`, laisse
a son defaut spaCy de 262144 faute d'etre passe explicitement.

Avec `no_output_layer: False` et 74 techniques, la couche de sortie faisait
19 398 730 parametres mis a jour par Adam a chaque pas de gradient, pour une
table de hachage vide a 99 % (le corpus tient dans quelques milliers de
tokens distincts).

Mesure contre le vrai corpus (fr, 25 iterations, jeu de dev = derniere
utterance de chaque technique tenue hors entrainement) :

  length=262144 ngram=1  dev 97.3%  conf 0.809  394.7 s   <- avant
  length=2048   ngram=2  dev 97.3%  conf 0.919   12.3 s   <- apres

Meme accuracy, meilleure confiance, 32x plus rapide. Les bigrammes profitent
des locutions du corpus ("faire fondre", "laisser mijoter") qu'un sac
d'unigrammes ne distingue pas de leurs constituants isoles.

PipelineRegistry.initialize() (fr+en, chargement + entrainement) mesure a
28,5 s contre ~930 s. Suite pytest complete : 46 passed en 40,6 s.

- locale_pipeline.py : `_TEXTCAT_MODEL_LENGTH` / `_TEXTCAT_NGRAM_SIZE`
  explicites et commentes avec les mesures ; historique de calibration de
  `_TRAINING_ITERATIONS` corrige (il attribuait le cout aux iterations)
- README : sections "Temps de demarrage" et "Limitations connues" reecrites
  (la couverture d'evaluation, 26/74 techniques, y devient la limitation
  bloquante ; ajout des collisions de classes observees)
- docker-compose.yml / ci.yml : commentaires citant ~930 s corriges,
  start_period et timeout 1200s -> 300s (plafond pour runner lent, pas cible)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-03 18:01:06 +02:00

141 lines
6.6 KiB
YAML

services:
postgres:
image: postgres:16-alpine
restart: unless-stopped
environment:
# No defaults on purpose: POSTGRES_USER/PASSWORD/DB must be set in your
# local, git-ignored .env (see .env.example). Compose fails loudly if
# they're missing instead of falling back to a guessable credential.
POSTGRES_USER: ${POSTGRES_USER:?set POSTGRES_USER in .env}
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}
POSTGRES_DB: ${POSTGRES_DB:?set POSTGRES_DB in .env}
ports:
- "${POSTGRES_PORT:-5432}:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U $$POSTGRES_USER"]
interval: 5s
timeout: 5s
retries: 5
# Single service serving both the API and the built frontend (see
# apps/api/Dockerfile) — no separate nginx/web container, no cross-origin
# CORS_ORIGIN to keep in sync between two ports.
app:
build:
context: .
dockerfile: apps/api/Dockerfile
restart: unless-stopped
environment:
NODE_ENV: production
PORT: 3000
# Uses the "postgres" service name, not localhost/POSTGRES_PORT —
# container-to-container traffic stays on the compose network and
# always targets Postgres's internal port (5432).
DATABASE_URL: "postgresql://${POSTGRES_USER:?set POSTGRES_USER in .env}:${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env}@postgres:5432/${POSTGRES_DB:?set POSTGRES_DB in .env}?schema=public"
JWT_SECRET: ${JWT_SECRET:?set JWT_SECRET in .env}
# Unset by default (falls back to NODE_ENV === "production", i.e.
# Secure cookie required) — set COOKIE_SECURE=false in .env only if
# this deployment is reachable over plain HTTP (no TLS in front of
# it yet), otherwise the session cookie never comes back and every
# authenticated request 401s despite login succeeding. See its doc
# comment in apps/api/src/config/env.ts.
COOKIE_SECURE: ${COOKIE_SECURE:-}
# Shared with the `tech-step-llm-worker` service below — see
# requireInternalWorker's doc comment
# (apps/api/src/middlewares/require-internal-worker.ts). Unset by
# default: `/internal/tech-steps/*` fails closed rather than open
# for a deployment that doesn't run the worker at all.
INTERNAL_WORKER_SECRET: ${INTERNAL_WORKER_SECRET:-}
# Admin surface (apps/web's /admin/* routes + the API's /admin/*).
# Unset by default: `requireAdmin` fails closed without
# ADMIN_JWT_SECRET, so a stack that doesn't need the admin app simply
# has every /admin/* route 401. Must be a *different* secret than
# JWT_SECRET.
ADMIN_JWT_SECRET: ${ADMIN_JWT_SECRET:-}
# Compose network service name, not localhost — same reasoning as
# DATABASE_URL above. Unlike INTERNAL_WORKER_SECRET, no `:-` fallback:
# tech-step-intent-service is a core dependency (see its own entry
# below), not an optional background job.
INTENT_SERVICE_BASE_URL: "http://tech-step-intent-service:8000"
INTENT_SERVICE_SECRET: ${INTENT_SERVICE_SECRET:?set INTENT_SERVICE_SECRET in .env}
ports:
- "${APP_PORT:-3000}:3000"
depends_on:
postgres:
condition: service_healthy
tech-step-intent-service:
condition: service_healthy
# spaCy-based NER + intent classification microservice
# (services/tech-step-intent-service) — `app` delegates all tech-step
# detection to it over HTTP (see `IntentServiceClient`,
# apps/api/src/lib/recipe-matching/intent-service-client.ts). Unlike
# `tech-step-llm-worker` below, **not optional**: without it, `app` can no
# longer detect any cooking technique in a recipe step at all. No exposed
# port — reachable only from `app` on the compose network, nothing ever
# calls into it from outside.
tech-step-intent-service:
build:
context: .
dockerfile: services/tech-step-intent-service/Dockerfile
restart: unless-stopped
environment:
INTENT_SERVICE_SECRET: ${INTENT_SERVICE_SECRET:?set INTENT_SERVICE_SECRET in .env}
healthcheck:
# No curl/wget in the python:3.12-slim base image — a one-line Python
# request is the healthcheck for a service that's already guaranteed
# to have Python (see this service's Dockerfile).
test:
[
"CMD",
"python",
"-c",
"import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=2)",
]
interval: 15s
timeout: 3s
retries: 5
# This service trains itself from scratch on every start (no model
# ever persisted to disk, see its own README) — `/health` only
# returns 200 once that's done, not just once the base spaCy models
# are loaded. Measured at ~30s combined (fr+en) against the current
# ~74-technique corpus, down from ~930s: the old figure was dominated
# by `spacy.TextCatBOW.v3`'s `length` parameter sitting at its 262144
# default, not by the corpus size (see `_TEXTCAT_MODEL_LENGTH` in
# `locale_pipeline.py` for the measurements).
#
# `start_period` stays deliberately far above that ~30s: it only
# governs how long failing checks are excused from counting against
# `retries` (which would otherwise flip this container to "unhealthy"
# mid-training and block `app`'s `depends_on: condition:
# service_healthy` indefinitely), so an over-generous value costs
# nothing while an under-tight one breaks startup on a slow runner.
start_period: 300s
# Deliberately its own image, not built into `app`'s (see
# services/tech-step-llm-worker/Dockerfile's own doc comment) — a
# long-lived process with no exposed port (nothing ever calls *into* it,
# it only ever calls out to `app`). Optional: an `INTERNAL_WORKER_SECRET`-
# less deployment can omit this service entirely and `app` still runs
# fine, just without the offline audit/feedback-loop jobs.
tech-step-llm-worker:
build:
context: .
dockerfile: services/tech-step-llm-worker/Dockerfile
restart: unless-stopped
depends_on:
- app
environment:
API_BASE_URL: "http://app:3000"
INTERNAL_WORKER_SECRET: ${INTERNAL_WORKER_SECRET:?set INTERNAL_WORKER_SECRET in .env to run this service}
TECH_STEP_WORKER_CRON: ${TECH_STEP_WORKER_CRON:-0 3 * * 0}
volumes:
# GGUF weights persist across restarts — see this service's own
# Dockerfile doc comment on its VOLUME declaration.
- tech_step_llm_worker_models:/worker/models
volumes:
postgres_data:
tech_step_llm_worker_models: