Le service mettait ~930 s (15-16 min) a devenir `healthy`, ce que le README
presentait comme "un compromis assume, pas un defaut de configuration a
corriger". Mauvais diagnostic : le cout ne venait ni des iterations ni de la
taille du corpus, mais du parametre `length` de `spacy.TextCatBOW.v3`, laisse
a son defaut spaCy de 262144 faute d'etre passe explicitement.
Avec `no_output_layer: False` et 74 techniques, la couche de sortie faisait
19 398 730 parametres mis a jour par Adam a chaque pas de gradient, pour une
table de hachage vide a 99 % (le corpus tient dans quelques milliers de
tokens distincts).
Mesure contre le vrai corpus (fr, 25 iterations, jeu de dev = derniere
utterance de chaque technique tenue hors entrainement) :
length=262144 ngram=1 dev 97.3% conf 0.809 394.7 s <- avant
length=2048 ngram=2 dev 97.3% conf 0.919 12.3 s <- apres
Meme accuracy, meilleure confiance, 32x plus rapide. Les bigrammes profitent
des locutions du corpus ("faire fondre", "laisser mijoter") qu'un sac
d'unigrammes ne distingue pas de leurs constituants isoles.
PipelineRegistry.initialize() (fr+en, chargement + entrainement) mesure a
28,5 s contre ~930 s. Suite pytest complete : 46 passed en 40,6 s.
- locale_pipeline.py : `_TEXTCAT_MODEL_LENGTH` / `_TEXTCAT_NGRAM_SIZE`
explicites et commentes avec les mesures ; historique de calibration de
`_TRAINING_ITERATIONS` corrige (il attribuait le cout aux iterations)
- README : sections "Temps de demarrage" et "Limitations connues" reecrites
(la couverture d'evaluation, 26/74 techniques, y devient la limitation
bloquante ; ajout des collisions de classes observees)
- docker-compose.yml / ci.yml : commentaires citant ~930 s corriges,
start_period et timeout 1200s -> 300s (plafond pour runner lent, pas cible)
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
205 lines
8.6 KiB
YAML
205 lines
8.6 KiB
YAML
name: CI
|
|
|
|
# Every push, on every branch — not just main — so build breakage shows up
|
|
# on the first commit of a branch, not only once a PR targets main. No
|
|
# separate pull_request trigger: for a PR from a branch on this same repo,
|
|
# push already fires on that branch, and GitHub attaches the run to the PR
|
|
# by commit SHA regardless of which event triggered it — adding
|
|
# pull_request too would just run every job twice per push. (Re-add it,
|
|
# scoped to forks, only if this repo starts accepting fork PRs — push
|
|
# events from a fork never reach here.)
|
|
on:
|
|
push:
|
|
|
|
env:
|
|
# Hostname `postgres` = the container the `test` job starts itself (see
|
|
# that job's "Start Postgres" step) — a plain `docker run`, not a
|
|
# `services:` block, for the same reason the intent-service below can't be
|
|
# one: this repo's Forgejo `act` runner does not keep a `services:`
|
|
# container as the *same* instance across a job's steps, so `migrate
|
|
# deploy` would land the schema in one Postgres and `pnpm --filter api
|
|
# test`'s `beforeAll` would then hit an empty one ("table … does not
|
|
# exist" right after "migrations successfully applied"). One long-lived
|
|
# container in the job, shared by every step, removes that failure mode.
|
|
DATABASE_URL: "postgresql://ci:ci@postgres:5432/batchcooking_ci?schema=public"
|
|
# Test-only secret, never used outside CI — real deployments must set their own.
|
|
JWT_SECRET: "ci-only-secret-not-used-anywhere-else-32chars+"
|
|
# Same reasoning as JWT_SECRET above — lets tech-step-worker.routes.test.ts
|
|
# exercise the success path (matching secret), not just the "unset"
|
|
# rejection every environment that doesn't set this gets by default.
|
|
INTERNAL_WORKER_SECRET: "ci-only-worker-secret-not-used-anywhere-else-32chars+"
|
|
# Same reasoning — lets admin-auth.test.ts exercise the admin login
|
|
# success path + the requireAdmin-guarded routes, not just the "no admin
|
|
# secret configured -> 401" path.
|
|
ADMIN_JWT_SECRET: "ci-only-admin-secret-not-used-anywhere-else-32chars+"
|
|
# Shared between the `test` job's own uvicorn step (below) and apps/api's
|
|
# IntentServiceClient — that step runs `uvicorn` as a plain background
|
|
# process (not a `services:` block or even a container) because CI can
|
|
# only pull a published image, not build
|
|
# services/tech-step-intent-service/Dockerfile from this checkout.
|
|
INTENT_SERVICE_BASE_URL: "http://localhost:8000"
|
|
INTENT_SERVICE_SECRET: "ci-only-intent-secret-not-used-anywhere-else-32chars+"
|
|
|
|
jobs:
|
|
# Five independent jobs, no needs: between them — each starts in parallel
|
|
# and reports as its own check, instead of the previous single chained
|
|
# "lint-and-test then e2e" pipeline.
|
|
lint:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm lint
|
|
|
|
test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Install curl and the docker CLI
|
|
run: |
|
|
apt-get update && apt-get install -y curl docker.io
|
|
|
|
# Postgres as a plain background container instead of a `services:`
|
|
# block — see this workflow's top-level `env.DATABASE_URL` comment for
|
|
# why. Reachable as `postgres` from this job (the runner mounts the
|
|
# docker socket and runs jobs on a shared user-defined network, so a
|
|
# `--name postgres` sibling resolves by name); the network is
|
|
# discovered from the job container itself rather than hardcoded.
|
|
- name: Start Postgres in the background
|
|
run: |
|
|
net="$(docker inspect "$(hostname)" \
|
|
--format '{{range $k, $_ := .NetworkSettings.Networks}}{{$k}} {{end}}' \
|
|
| awk '{print $1}')"
|
|
echo "job network: ${net:?could not determine the job container network}"
|
|
docker rm -f postgres >/dev/null 2>&1 || true
|
|
docker run -d --name postgres --network "$net" \
|
|
-e POSTGRES_USER=ci -e POSTGRES_PASSWORD=ci -e POSTGRES_DB=batchcooking_ci \
|
|
postgres:16-alpine
|
|
for _ in $(seq 1 60); do
|
|
docker exec postgres pg_isready -U ci -d batchcooking_ci && exit 0
|
|
sleep 2
|
|
done
|
|
echo "Postgres did not become ready in time" >&2
|
|
docker logs postgres || true
|
|
exit 1
|
|
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- uses: https://github.com/astral-sh/setup-uv@v5
|
|
with:
|
|
python-version: "3.12"
|
|
enable-cache: true
|
|
|
|
# A `services:` block can only pull an already-published image — it
|
|
# can't build services/tech-step-intent-service/Dockerfile from this
|
|
# checkout. Running `uvicorn` as a plain background step instead: it
|
|
# keeps running for the rest of this job (steps in one job share the
|
|
# same process tree), and `pnpm --filter api test` below needs a real
|
|
# instance to talk to per this repo's "never mock an internal service"
|
|
# test convention — same "real dependency, started by the job itself"
|
|
# posture as the Postgres step above, not a mock HTTP server.
|
|
- name: Install services/tech-step-intent-service
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv sync --frozen
|
|
- name: Start services/tech-step-intent-service in the background
|
|
working-directory: services/tech-step-intent-service
|
|
run: |
|
|
uv run uvicorn intent_service.main:app --host 0.0.0.0 --port 8000 &
|
|
# `/health` only returns 200 once this service has finished
|
|
# training itself from scratch (no model ever persisted to disk —
|
|
# see its own README) — measured at ~30s combined (fr+en) against
|
|
# the current ~74-technique corpus, down from ~930s once
|
|
# `spacy.TextCatBOW.v3`'s `length` stopped being left at its
|
|
# 262144 default (see `_TEXTCAT_MODEL_LENGTH` in
|
|
# `locale_pipeline.py`, and docker-compose.yml's healthcheck for
|
|
# the same reasoning). The timeout stays well above that: it's a
|
|
# ceiling for a slow/contended runner, not a target.
|
|
timeout 300 bash -c 'until curl -sf http://localhost:8000/health > /dev/null; do sleep 2; done'
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm --filter api exec prisma migrate deploy
|
|
- run: pnpm --filter api test
|
|
|
|
intent-service-test:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: https://github.com/astral-sh/setup-uv@v5
|
|
with:
|
|
python-version: "3.12"
|
|
enable-cache: true
|
|
|
|
- name: Install services/tech-step-intent-service
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv sync --frozen
|
|
- name: Run pytest
|
|
working-directory: services/tech-step-intent-service
|
|
run: uv run pytest -q
|
|
|
|
build:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
- run: pnpm build
|
|
|
|
e2e:
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- uses: https://github.com/actions/checkout@v4
|
|
|
|
- name: Install Xvfb and Cypress dependencies
|
|
run: |
|
|
apt-get update
|
|
apt-get install -y xvfb libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth dbus-x11
|
|
|
|
- uses: pnpm/action-setup@v4
|
|
|
|
- uses: https://github.com/actions/setup-node@v4
|
|
with:
|
|
node-version: 22
|
|
cache: pnpm
|
|
|
|
- name: Cache Cypress binary
|
|
uses: https://github.com/actions/cache@v4
|
|
with:
|
|
path: ~/.cache/Cypress
|
|
key: cypress-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
|
|
|
|
- run: pnpm install --frozen-lockfile
|
|
# pnpm install doesn't reliably trigger Cypress's postinstall binary
|
|
# download (see apps/web's cypress caveat in the README) — install it
|
|
# explicitly so `cypress run` finds it.
|
|
- run: pnpm --filter web exec cypress install
|
|
- name: Run E2E tests
|
|
env:
|
|
HOST: "0.0.0.0"
|
|
run: pnpm --filter web e2e
|
|
# No dev server needed here — Cypress spins up its own Vite dev
|
|
# server internally for component testing (see cypress.config.ts's
|
|
# `component.devServer`), unlike `e2e` above which needs the real app
|
|
# running first.
|
|
- run: pnpm --filter web cy:run:component
|