fix(tech-steps): calibre le seuil F1 sur une vraie execution et corrige un bug de comptage

Docker etant redevenu disponible dans cette session, j'ai pu lancer pour de
vrai la suite Mocha d'apps/api (334/334, y compris les tests Phase 1/2
qui n'avaient pu etre executes precedemment) ainsi que les scripts de la
Phase 5 contre une vraie base de test.

- tech-step-eval-dataset.ts : corrige un vrai bug d'auteur - "Take the
  plates..." collisionnait avec le synonyme anglais enregistre "plates"
  (technique plate), invalidant ce cas negatif. Remplace par "dishes".
- tech-step-eval-runner.ts : F1 reel mesure = 0.815 (33 TP / 9 FP / 6 FN).
  Documente ce chiffre et les vraies erreurs de classification decouvertes
  (ex: "Blanchissez les haricots verts..." classifie a tort comme "peel")
  - des faiblesses reelles du classifieur que ce harness est cense
  detecter, pas a masquer en ajustant le jeu de test.
- retrain-tech-steps.ts : le script loggait `appliedIds.length`/
  `rejectedIds.length` (ce qui a ete demande) au lieu du `count` reel
  retourne par `updateMany` (ce qui a vraiment ete modifie) - un id
  inexistant faisait afficher un faux succes. Decouvert en executant le
  script pour de vrai avec des ids partiellement invalides.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

apps/api/src/lib/recipe-matching/tech-step-eval-dataset.ts

@@ -19,22 +19,20 @@
  * real DB ids and back by `tech-step-eval-runner.ts`, this file only ever
  * deals in stable keys so it doesn't need DB access to author or read.
  *
- * Known limit of this dataset: most cases anchor on a technique's own
- * registered synonym (verb form), which `_classifyClause` always falls
- * back to labeling correctly even when the intent classifier itself isn't
- * confident (see `tech-step-matcher.ts`'s doc comment, point 3) — so this
- * dataset mainly measures precision (wrong/duplicate matches, false
- * positives from a synonym overlapping another technique's vocabulary) and
- * breadth of coverage across all ~26 techniques, not the classifier's
- * ability to recognize a technique described without ever naming it
- * (`tech-step-matcher.test.ts` already covers a few of those specific,
- * verified cases at the unit level — e.g. "jusqu'à ce que le beurre ait
- * disparu dans la poêle" for `melt`). Extending this dataset with more
- * paraphrase-only cases is valuable future work, but each one needs to be
- * verified against a real trained classifier before being added (a wrong
- * expected label here fails the regression gate for the wrong reason) —
- * see this feature's plan document for the current gap in this session's
- * ability to run the classifier locally.
+ * Known limit of this dataset, confirmed against a real run (see
+ * `MIN_OVERALL_F1`'s own doc comment, `tech-step-eval-runner.ts`): most
+ * cases anchor on a technique's own registered synonym, but `_classifyClause`
+ * only falls back to that anchor when the intent classifier's own score is
+ * *below* `CONFIDENCE_THRESHOLD` — a confidently *wrong* whole-clause
+ * classification (e.g. "Blanchissez les haricots verts..." scoring
+ * confidently as `peel` despite the correct `blanch` anchor) overrides the
+ * anchor just as readily as a confidently *right* one does, so this
+ * dataset genuinely does measure real classifier failures, not just a
+ * synthetic floor. A handful of such real mismatches are expected and
+ * intentionally left uncorrected here (see `MIN_OVERALL_F1`'s doc comment)
+ * — fixing the classifier's actual behavior on them is corpus work for a
+ * future change, not something to hide by loosening this dataset's own
+ * expectations to match whatever it currently outputs.
  */
 
 export interface TechStepEvalCase {
@@ -240,7 +238,7 @@ export const TECH_STEP_EVAL_DATASET: TechStepEvalCase[] = [
     expectedKeys: [],
   },
   {
-    description: "Take the plates and glasses out of the cupboard.",
+    description: "Take the dishes and glasses out of the cupboard.",
     locale: "en",
     expectedKeys: [],
   },

apps/api/src/lib/recipe-matching/tech-step-eval-runner.ts

@@ -8,12 +8,24 @@ import {
 import { techStepClassifier } from "./tech-step-matcher.js";
 
 /**
- * Provisional floor (not a target) both `runTechStepEvalSuite`'s
- * consumers gate on — see `test/recipe-matching/tech-step-eval.test.ts`'s
- * own doc comment for the full reasoning behind this specific value and
- * when to tighten it. Exported from here (not defined separately in each
- * consumer) so the CI regression gate and `scripts/retrain-tech-steps.ts`'s
+ * Regression floor (not a target) both `runTechStepEvalSuite`'s consumers
+ * gate on — exported from here (not defined separately in each consumer)
+ * so the CI regression gate and `scripts/retrain-tech-steps.ts`'s
  * pre-backfill gate can never silently drift to different thresholds.
+ *
+ * Calibrated against a real run: the classifier trained on the corpus as
+ * of this constant's introduction scored **0.815** aggregate F1
+ * (33 TP / 9 FP / 6 FN) against `TECH_STEP_EVAL_DATASET` — `0.8` leaves a
+ * small margin below that for run-to-run noise while still catching a
+ * real regression (not a floor picked blind before ever running this
+ * suite — see this feature's plan document for that earlier state). The
+ * mismatches this run surfaced (e.g. "Blanchissez les haricots verts..."
+ * misclassified as `peel`, a handful of anchor-less sentences expected to
+ * match nothing instead scoring confidently as some technique) are real,
+ * known classifier weaknesses — evidence this harness is doing its job,
+ * not something to quietly paper over by loosening the dataset's own
+ * expectations. Improving them is corpus work for a future change, gated
+ * by this same suite.
  */
 export const MIN_OVERALL_F1 = 0.8;
 

apps/api/src/scripts/retrain-tech-steps.ts

@@ -70,18 +70,22 @@ async function retrainTechSteps(): Promise<void> {
   console.info(`Backfilled ${changed}/${total} step(s).`);
 
   if (appliedIds.length > 0) {
-    await prisma.techStepTrainingSuggestion.updateMany({
+    // `updateMany`'s own `count` (rows actually matched/updated), not
+    // `appliedIds.length` (what was merely *asked for*) — an id that
+    // doesn't exist (typo, already-processed id) would otherwise log a
+    // success count that silently doesn't match what really changed.
+    const { count } = await prisma.techStepTrainingSuggestion.updateMany({
       where: { id: { in: appliedIds } },
       data: { status: "applied" },
     });
-    console.info(`Marked ${appliedIds.length} suggestion(s) as applied.`);
+    console.info(`Marked ${count}/${appliedIds.length} suggestion(s) as applied.`);
   }
   if (rejectedIds.length > 0) {
-    await prisma.techStepTrainingSuggestion.updateMany({
+    const { count } = await prisma.techStepTrainingSuggestion.updateMany({
       where: { id: { in: rejectedIds } },
       data: { status: "rejected" },
     });
-    console.info(`Marked ${rejectedIds.length} suggestion(s) as rejected.`);
+    console.info(`Marked ${count}/${rejectedIds.length} suggestion(s) as rejected.`);
   }
 }
 

apps/api/test/recipe-matching/tech-step-eval.test.ts

@@ -12,22 +12,9 @@ import { resetDatabase } from "../../test-support/reset-db.js";
  * `TechStepTrainingSuggestion`, see `scripts/retrain-tech-steps.ts`) must
  * keep this suite green. Runs {@link runTechStepEvalSuite} (the real
  * trained classifier against `tech-step-eval-dataset.ts`) and asserts the
- * aggregate F1 doesn't fall below {@link MIN_OVERALL_F1}.
- *
- * `MIN_OVERALL_F1` (`tech-step-eval-runner.ts`) is a provisional floor,
- * not a target: most of the dataset's cases are built around a
- * technique's own registered synonym, which `_classifyClause` always
- * resolves correctly via its NER-anchor fallback even when the intent
- * classifier itself scores under `CONFIDENCE_THRESHOLD` (see
- * `tech-step-matcher.ts`'s doc comment, point 3) — so a healthy run should
- * land well above this floor. It's set low enough to tolerate the residual
- * uncertainty in a dataset authored without being able to run it against a
- * live trained classifier first (no local Postgres was reachable in the
- * session that introduced this file — see this feature's plan document).
- * Once this suite has actually run once (locally or in CI) and produced
- * real numbers, tighten that constant to just below the observed F1, so a
- * real future regression still fails loudly instead of hiding under a
- * floor that's too forgiving.
+ * aggregate F1 doesn't fall below {@link MIN_OVERALL_F1} — see that
+ * constant's own doc comment (`tech-step-eval-runner.ts`) for the real run
+ * it was calibrated against.
  */
 
 describe("tech-step-eval", () => {