feat(recipes): associe ingredients, quantites et ustensiles aux techniques detectees
Etend le pipeline de detection de techniques (tech-step-matcher.ts) pour resoudre, par clause, les metadonnees qui accompagnent une technique detectee : - Ingredients : nouvelle fonction findIngredientMentions (ingredient-matcher.ts) qui scanne le texte d'une clause contre le catalogue Ingredient existant (reutilise INGREDIENT_LABELS_FR/EN deja utilise par matchIngredientName), avec extraction best-effort de la quantite+unite immediatement avant la mention. - Ustensiles : nouveau catalogue Utensil (Prisma) + second PhraseMatcher cote service Python (intent_service/utensil_vocabulary.py), independant du textcat des techniques (pas d'interpretation necessaire pour un ustensile). POST /v1/process distingue desormais chaque entite via un champ kind (technique|utensil). - Persistance : deux nouvelles tables StepTechStepIngredient/ StepTechStepUtensil, liees a StepTechStep par sa cle composite (stepId, order), peuplees au moment du matching (recipe.service.ts) et exposees via StepTechStepView (packages/shared). Aucune analyse syntaxique ajoutee (le parser spaCy reste exclu du pipeline) : l'association se fait par appartenance a la clause deja calculee par splitIntoClauses. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
parent
bf58834aa9
commit
4e0a9ce8d2
27 changed files with 1339 additions and 57 deletions
|
|
@ -0,0 +1,10 @@
|
|||
-- CreateTable
|
||||
CREATE TABLE "utensil" (
|
||||
"id" SERIAL NOT NULL,
|
||||
"key" TEXT NOT NULL,
|
||||
|
||||
CONSTRAINT "utensil_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateIndex
|
||||
CREATE UNIQUE INDEX "utensil_key_key" ON "utensil"("key");
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
-- CreateTable
|
||||
CREATE TABLE "step_tech_step_ingredient" (
|
||||
"id" SERIAL NOT NULL,
|
||||
"step_id" INTEGER NOT NULL,
|
||||
"tech_step_order" INTEGER NOT NULL,
|
||||
"ingredient_id" INTEGER NOT NULL,
|
||||
"quantity" DECIMAL(10,2),
|
||||
"unit_id" INTEGER,
|
||||
"start" INTEGER NOT NULL,
|
||||
"end" INTEGER NOT NULL,
|
||||
"source" TEXT NOT NULL DEFAULT 'auto',
|
||||
|
||||
CONSTRAINT "step_tech_step_ingredient_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- CreateTable
|
||||
CREATE TABLE "step_tech_step_utensil" (
|
||||
"id" SERIAL NOT NULL,
|
||||
"step_id" INTEGER NOT NULL,
|
||||
"tech_step_order" INTEGER NOT NULL,
|
||||
"utensil_id" INTEGER NOT NULL,
|
||||
"start" INTEGER NOT NULL,
|
||||
"end" INTEGER NOT NULL,
|
||||
"source" TEXT NOT NULL DEFAULT 'auto',
|
||||
|
||||
CONSTRAINT "step_tech_step_utensil_pkey" PRIMARY KEY ("id")
|
||||
);
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "step_tech_step_ingredient" ADD CONSTRAINT "step_tech_step_ingredient_step_id_tech_step_order_fkey" FOREIGN KEY ("step_id", "tech_step_order") REFERENCES "step_tech_step"("step_id", "order") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "step_tech_step_ingredient" ADD CONSTRAINT "step_tech_step_ingredient_ingredient_id_fkey" FOREIGN KEY ("ingredient_id") REFERENCES "ingredients"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "step_tech_step_ingredient" ADD CONSTRAINT "step_tech_step_ingredient_unit_id_fkey" FOREIGN KEY ("unit_id") REFERENCES "unit"("id") ON DELETE SET NULL ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "step_tech_step_utensil" ADD CONSTRAINT "step_tech_step_utensil_step_id_tech_step_order_fkey" FOREIGN KEY ("step_id", "tech_step_order") REFERENCES "step_tech_step"("step_id", "order") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
||||
-- AddForeignKey
|
||||
ALTER TABLE "step_tech_step_utensil" ADD CONSTRAINT "step_tech_step_utensil_utensil_id_fkey" FOREIGN KEY ("utensil_id") REFERENCES "utensil"("id") ON DELETE CASCADE ON UPDATE CASCADE;
|
||||
|
|
@ -515,12 +515,15 @@ model Ingredient {
|
|||
/// catalog's own search with this ingredient's name).
|
||||
reproducible Boolean @default(false)
|
||||
|
||||
recipes RecipeIngredient[]
|
||||
allergies IngredientAllergy[]
|
||||
recipes RecipeIngredient[]
|
||||
allergies IngredientAllergy[]
|
||||
/// Profiles that personally dislike this ingredient — see {@link UserProfileDislikedIngredient}.
|
||||
dislikedBy UserProfileDislikedIngredient[]
|
||||
dislikedBy UserProfileDislikedIngredient[]
|
||||
/// Diet regimes this ingredient is compatible with — see {@link IngredientDiet}.
|
||||
diets IngredientDiet[]
|
||||
diets IngredientDiet[]
|
||||
/// Mentions of this ingredient detected in a step's free text alongside a
|
||||
/// technique — see `StepTechStepIngredient`.
|
||||
stepTechSteps StepTechStepIngredient[]
|
||||
|
||||
@@map("ingredients")
|
||||
}
|
||||
|
|
@ -596,7 +599,13 @@ model Unit {
|
|||
type UnitType
|
||||
toBaseFactor Decimal @default(1) @map("to_base_factor") @db.Decimal(12, 4)
|
||||
|
||||
recipeIngredients RecipeIngredient[]
|
||||
recipeIngredients RecipeIngredient[]
|
||||
/// Ingredient mentions detected alongside a technique in a step's free
|
||||
/// text (e.g. "50g" resolved against this `Unit`) — see
|
||||
/// `StepTechStepIngredient`. Distinct from `recipeIngredients` above
|
||||
/// (the recipe's structured ingredient list): a step can mention a
|
||||
/// quantity+unit that was never itself an ingredient list line.
|
||||
stepTechStepIngredients StepTechStepIngredient[]
|
||||
|
||||
@@map("unit")
|
||||
}
|
||||
|
|
@ -652,6 +661,25 @@ model TechStep {
|
|||
@@map("tech_step")
|
||||
}
|
||||
|
||||
/// `key` is `@unique`, same bare id+key shape as `TechStep` — no
|
||||
/// categorization taxonomy like `Ingredient` needed yet, and no matching
|
||||
/// data of its own here either: unlike `TechStep` (whose matching synonyms
|
||||
/// used to live in TS and were moved into
|
||||
/// `services/tech-step-intent-service`'s `training_data.py`), this catalog
|
||||
/// was *born* owned by that service (`utensil_vocabulary.py`) since nothing
|
||||
/// pre-existing needed it — this row only exists to be a stable id/key
|
||||
/// `StepTechStepUtensil` references, and to carry a French label
|
||||
/// (`apps/web`'s `catalog.utensils.<key>`, see `reference-seed-data.ts`'s
|
||||
/// `UTENSILS`).
|
||||
model Utensil {
|
||||
id Int @id @default(autoincrement())
|
||||
key String @unique
|
||||
|
||||
steps StepTechStepUtensil[]
|
||||
|
||||
@@map("utensil")
|
||||
}
|
||||
|
||||
/// Modeled as one-to-many (a step belongs to exactly one recipe), not the
|
||||
/// many-to-many noted in the spec doc: `order` only makes sense scoped to a
|
||||
/// single recipe, which isn't reconcilable with steps being shared across
|
||||
|
|
@ -720,13 +748,72 @@ model StepTechStep {
|
|||
contextEnd Int? @map("context_end")
|
||||
source String @default("auto")
|
||||
|
||||
step Step @relation(fields: [stepId], references: [id], onDelete: Cascade)
|
||||
techStep TechStep @relation(fields: [techStepId], references: [id], onDelete: Cascade)
|
||||
step Step @relation(fields: [stepId], references: [id], onDelete: Cascade)
|
||||
techStep TechStep @relation(fields: [techStepId], references: [id], onDelete: Cascade)
|
||||
/// Ingredients mentioned in the same clause as this technique occurrence
|
||||
/// — see `StepTechStepIngredient`.
|
||||
ingredients StepTechStepIngredient[]
|
||||
/// Utensils mentioned in the same clause as this technique occurrence —
|
||||
/// see `StepTechStepUtensil`.
|
||||
utensils StepTechStepUtensil[]
|
||||
|
||||
@@id([stepId, order])
|
||||
@@map("step_tech_step")
|
||||
}
|
||||
|
||||
/// An ingredient mention found in the same *clause* as one `StepTechStep`
|
||||
/// occurrence (`tech-step-matcher.ts`'s `matchTechStepSpans` — clauses are
|
||||
/// already the unit a technique is judged on, see that file's doc comment,
|
||||
/// so "same clause" is the association rule, no dependency-parsing needed).
|
||||
/// `quantity`/`unitId` are best-effort, populated only when a leading
|
||||
/// numeric expression immediately preceding the ingredient mention resolved
|
||||
/// against the `Unit` catalog (`ingredient-matcher.ts`'s
|
||||
/// `findIngredientMentions`) — both `null` when the clause names the
|
||||
/// ingredient with no quantity ("ajouter le sel"). `start`/`end` are the
|
||||
/// ingredient mention's own span in `Step.description`, same `[start, end)`
|
||||
/// convention as `StepTechStep.start`/`end`. `source` mirrors
|
||||
/// `StepTechStep.source` (`"auto"` today, room for a future user
|
||||
/// correction without a shape change).
|
||||
model StepTechStepIngredient {
|
||||
id Int @id @default(autoincrement())
|
||||
stepId Int @map("step_id")
|
||||
techStepOrder Int @map("tech_step_order")
|
||||
ingredientId Int @map("ingredient_id")
|
||||
quantity Decimal? @db.Decimal(10, 2)
|
||||
unitId Int? @map("unit_id")
|
||||
start Int
|
||||
end Int
|
||||
source String @default("auto")
|
||||
|
||||
stepTechStep StepTechStep @relation(fields: [stepId, techStepOrder], references: [stepId, order], onDelete: Cascade)
|
||||
ingredient Ingredient @relation(fields: [ingredientId], references: [id], onDelete: Cascade)
|
||||
unit Unit? @relation(fields: [unitId], references: [id])
|
||||
|
||||
@@map("step_tech_step_ingredient")
|
||||
}
|
||||
|
||||
/// A utensil mention found in the same clause as one `StepTechStep`
|
||||
/// occurrence — same association rule as `StepTechStepIngredient` (see its
|
||||
/// doc comment). Detected by
|
||||
/// `services/tech-step-intent-service`'s own utensil `PhraseMatcher`
|
||||
/// (`intent_service/utensil_vocabulary.py`), returned alongside technique
|
||||
/// entities in `POST /v1/process` and filtered to this clause's span by
|
||||
/// `tech-step-matcher.ts`.
|
||||
model StepTechStepUtensil {
|
||||
id Int @id @default(autoincrement())
|
||||
stepId Int @map("step_id")
|
||||
techStepOrder Int @map("tech_step_order")
|
||||
utensilId Int @map("utensil_id")
|
||||
start Int
|
||||
end Int
|
||||
source String @default("auto")
|
||||
|
||||
stepTechStep StepTechStep @relation(fields: [stepId, techStepOrder], references: [stepId, order], onDelete: Cascade)
|
||||
utensil Utensil @relation(fields: [utensilId], references: [id], onDelete: Cascade)
|
||||
|
||||
@@map("step_tech_step_utensil")
|
||||
}
|
||||
|
||||
/// One user-submitted correction to a `Step`'s detected techniques —
|
||||
/// captures ADD (a missing technique the classifier didn't find),
|
||||
/// REMOVE (a wrong technique it did), or RELABEL (both) as a single shape:
|
||||
|
|
|
|||
|
|
@ -148,6 +148,49 @@ export const TECH_STEPS: string[] = [
|
|||
"zest",
|
||||
];
|
||||
|
||||
// Same authoring convention as `TECH_STEPS` right above (stable English
|
||||
// camelCase uid, French label in `apps/web`'s `locales/fr/translation.json`
|
||||
// under `catalog.utensils.<key>`) — but unlike `TECH_STEPS`, the matching
|
||||
// data (per-locale synonym lists a `PhraseMatcher` matches against) lives
|
||||
// in `services/tech-step-intent-service/intent_service/utensil_vocabulary.py`'s
|
||||
// `UTENSIL_VOCABULARY`, not `training_data.py`: no textcat/training
|
||||
// involved, a utensil mention doesn't need to be classified, only matched.
|
||||
// Every entry here must have a matching entry there. See
|
||||
// `StepTechStepUtensil` in schema.prisma for how a mention gets attached to
|
||||
// a detected technique.
|
||||
export const UTENSILS: string[] = [
|
||||
"pan",
|
||||
"saucepan",
|
||||
"pot",
|
||||
"knife",
|
||||
"whisk",
|
||||
"bowl",
|
||||
"bakingSheet",
|
||||
"mold",
|
||||
"colander",
|
||||
"cuttingBoard",
|
||||
"oven",
|
||||
"blender",
|
||||
"mixer",
|
||||
"spatula",
|
||||
"ladle",
|
||||
"grater",
|
||||
"rollingPin",
|
||||
"lid",
|
||||
"tongs",
|
||||
"peeler",
|
||||
"sieve",
|
||||
"foodProcessor",
|
||||
"steamerBasket",
|
||||
"skewer",
|
||||
"pastryBrush",
|
||||
"ramekin",
|
||||
"dish",
|
||||
"wok",
|
||||
"thermometer",
|
||||
"mandoline",
|
||||
];
|
||||
|
||||
// The 14 allergens EU Regulation 1169/2011 (Annex II) requires food
|
||||
// businesses to declare — a standard, defensible reference list rather than
|
||||
// an invented one. Split into ALLERGY (classic IgE-mediated immune
|
||||
|
|
@ -1251,6 +1294,12 @@ export async function seedReferenceData(prisma: PrismaClient): Promise<void> {
|
|||
await prisma.techStep.upsert({ where: { key }, update: {}, create: { key } });
|
||||
}
|
||||
|
||||
// Utensil: same idempotent bare id/key upsert as TechStep right above —
|
||||
// no matching data alongside it either (see `UTENSILS`' own comment).
|
||||
for (const key of UTENSILS) {
|
||||
await prisma.utensil.upsert({ where: { key }, update: {}, create: { key } });
|
||||
}
|
||||
|
||||
// `Allergy` itself carries no `key` — it's the selectable instance of a
|
||||
// keyed `Category` (see schema.prisma) — so seeding an allergen means one
|
||||
// Category (upserted by key) plus exactly one Allergy row under it,
|
||||
|
|
|
|||
|
|
@ -111,6 +111,171 @@ function containsSubsequence(haystack: string[], needle: string[]): boolean {
|
|||
return false;
|
||||
}
|
||||
|
||||
/** One stemmed word from {@link tokenizeWithOffsets}, alongside its `[start, end)` span in the *original* (un-normalized) text it came from. */
|
||||
interface OffsetToken {
|
||||
word: string;
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
/** Matches a run of letters (any script, diacritics included) — the same "word" unit {@link tokenize} splits `normalizeText`'d text on (`/[^a-z]+/`), applied here directly to the *original* text instead so each token keeps its real character offsets. Digits/punctuation are never part of a run, same separator role they play for `tokenize` (a leading quantity is `extractQuantity`'s job, not this module's word-tokenizer's). */
|
||||
const LETTER_RUN_PATTERN = /\p{L}+/gu;
|
||||
|
||||
/**
|
||||
* {@link tokenize}'s positional twin: same stemmed/normalized words, but
|
||||
* each one keeps the `[start, end)` span it occupies in `text` — needed by
|
||||
* {@link findIngredientMentions} to report *where* a mention is, not just
|
||||
* that the catalog has a matching label somewhere. Splitting the original
|
||||
* text into letter-runs first (rather than normalizing the whole string up
|
||||
* front, the way `tokenize` does, then losing track of offsets) works
|
||||
* safely here because `normalizeText` only ever rewrites a character's own
|
||||
* form (case/diacritics) — see `_DiacriticsNormalizer`'s doc comment on the
|
||||
* Python side, ported from the same guarantee — never merges or splits
|
||||
* words, so normalizing one already-isolated run in place can't shift its
|
||||
* boundaries relative to the un-normalized text.
|
||||
*/
|
||||
function tokenizeWithOffsets(text: string, locale = "en"): OffsetToken[] {
|
||||
const tokens: OffsetToken[] = [];
|
||||
for (const match of text.matchAll(LETTER_RUN_PATTERN)) {
|
||||
const raw = match[0];
|
||||
const start = match.index ?? 0;
|
||||
const word = stemWord(normalizeText(raw), locale);
|
||||
if (word.length === 0) continue;
|
||||
tokens.push({ word, start, end: start + raw.length });
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
/**
|
||||
* Matches a quantity (integer/decimal/fraction/mixed number, same shapes as
|
||||
* {@link extractQuantity}) immediately followed by an optional unit
|
||||
* word/phrase (up to three words, e.g. "cuillères à soupe") and an optional
|
||||
* connector ("de"/"d'"/"of"/"a"/"an"), anchored at the *end* of whatever
|
||||
* string it's tested against (`$`) rather than the start. Anchoring at the
|
||||
* end — not the start — is what lets {@link findQuantityBeforeIngredient}
|
||||
* test the *whole* text preceding a mention without first having to guess
|
||||
* where an unrelated preamble ("ajouter", "puis", an earlier sentence) ends
|
||||
* and the quantity phrase begins: whatever doesn't fit the pattern
|
||||
* immediately before the ingredient simply isn't part of the match, no
|
||||
* separate boundary-finding step needed.
|
||||
*/
|
||||
const QUANTITY_BEFORE_INGREDIENT_PATTERN =
|
||||
/(\d+\s+\d+\/\d+|\d+\/\d+|\d+(?:[.,]\d+)?)\s*((?:\p{L}+\s+){0,2}\p{L}*)\s*(?:de\s|d['’]|of\s|a\s|an\s)?$/u;
|
||||
|
||||
/**
|
||||
* Best-effort quantity+unit lookup for an ingredient mention {@link findIngredientMentions}
|
||||
* just found at `mentionStart` in `text` — looks *only* at what immediately
|
||||
* precedes the mention (see {@link QUANTITY_BEFORE_INGREDIENT_PATTERN}), the
|
||||
* dominant French/English recipe phrasing ("200g de beurre", "2 cuillères à
|
||||
* soupe d'huile", "3 œufs"). Both `null` when nothing recognizable precedes
|
||||
* it (no leading digit at all) — same "no match, not an error" posture as
|
||||
* {@link extractQuantity}. Doesn't detect a quantity that *follows* its
|
||||
* ingredient ("du beurre, 50g") — an accepted gap, same trade-off
|
||||
* {@link extractQuantity} already documents for the leading-only case it
|
||||
* was built for.
|
||||
*/
|
||||
function findQuantityBeforeIngredient(
|
||||
text: string,
|
||||
mentionStart: number,
|
||||
unitCatalog: UnitMatchEntry[],
|
||||
locale: string,
|
||||
): { quantity: number | null; unitId: number | null } {
|
||||
const match = QUANTITY_BEFORE_INGREDIENT_PATTERN.exec(text.slice(0, mentionStart));
|
||||
if (!match) return { quantity: null, unitId: null };
|
||||
const { quantity } = extractQuantity(match[1] ?? "");
|
||||
const unitId = matchUnit(match[2] ?? "", unitCatalog, locale);
|
||||
return { quantity, unitId };
|
||||
}
|
||||
|
||||
/** One ingredient mention {@link findIngredientMentions} found in a free-text clause, alongside its `[start, end)` span (same convention as `TechStepMatch`, `tech-step-matcher.ts`) and any quantity+unit resolved immediately before it (see {@link findQuantityBeforeIngredient}) — both `null` when the clause names the ingredient with no quantity ("ajouter le sel"). */
|
||||
export interface IngredientMention {
|
||||
ingredientId: number;
|
||||
start: number;
|
||||
end: number;
|
||||
quantity: number | null;
|
||||
unitId: number | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Scans `text` (typically one technique's clause, see `tech-step-matcher.ts`'s
|
||||
* `splitIntoClauses`) for every mention of a catalog ingredient, left to
|
||||
* right, non-overlapping — the free-text-*scanning* counterpart to
|
||||
* {@link matchIngredientName} (which resolves one *already-isolated*
|
||||
* ingredient-line string to a single winner, not several mentions spread
|
||||
* across a longer text). Same "longest catalog label wins" rule as
|
||||
* {@link matchIngredientName}, applied at every token position in turn: once
|
||||
* a mention is found, scanning resumes right after it rather than
|
||||
* considering a shorter label starting inside an already-matched longer one.
|
||||
*
|
||||
* `locale` must match whatever `ingredientCatalog`/`unitCatalog` were loaded
|
||||
* in (see {@link loadIngredientCatalog}/{@link loadUnitCatalog}) — defaults
|
||||
* to `"en"`, same as every other function in this module.
|
||||
*/
|
||||
export function findIngredientMentions(
|
||||
text: string,
|
||||
ingredientCatalog: IngredientMatchEntry[],
|
||||
unitCatalog: UnitMatchEntry[],
|
||||
locale = "en",
|
||||
): IngredientMention[] {
|
||||
const tokens = tokenizeWithOffsets(text, locale);
|
||||
if (tokens.length === 0) return [];
|
||||
|
||||
const candidates = ingredientCatalog
|
||||
.map((entry) => ({
|
||||
ingredientId: entry.ingredientId,
|
||||
labelTokens: tokenize(entry.label, locale),
|
||||
}))
|
||||
.filter((entry) => entry.labelTokens.length > 0);
|
||||
|
||||
const mentions: IngredientMention[] = [];
|
||||
let i = 0;
|
||||
while (i < tokens.length) {
|
||||
let best: { ingredientId: number; tokenCount: number } | null = null;
|
||||
for (const candidate of candidates) {
|
||||
const { labelTokens } = candidate;
|
||||
if (i + labelTokens.length > tokens.length) continue;
|
||||
const matches = labelTokens.every((word, offset) => tokens[i + offset]?.word === word);
|
||||
if (!matches) continue;
|
||||
if (
|
||||
best === null ||
|
||||
labelTokens.length > best.tokenCount ||
|
||||
(labelTokens.length === best.tokenCount && candidate.ingredientId < best.ingredientId)
|
||||
) {
|
||||
best = { ingredientId: candidate.ingredientId, tokenCount: labelTokens.length };
|
||||
}
|
||||
}
|
||||
|
||||
if (best === null) {
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
const startToken = tokens[i];
|
||||
const endToken = tokens[i + best.tokenCount - 1];
|
||||
if (startToken === undefined || endToken === undefined) {
|
||||
// Unreachable — `best` was only ever set above after confirming
|
||||
// `i + labelTokens.length <= tokens.length`, so both tokens exist.
|
||||
// Satisfies `noUncheckedIndexedAccess`.
|
||||
i += 1;
|
||||
continue;
|
||||
}
|
||||
const { quantity, unitId } = findQuantityBeforeIngredient(
|
||||
text,
|
||||
startToken.start,
|
||||
unitCatalog,
|
||||
locale,
|
||||
);
|
||||
mentions.push({
|
||||
ingredientId: best.ingredientId,
|
||||
start: startToken.start,
|
||||
end: endToken.end,
|
||||
quantity,
|
||||
unitId,
|
||||
});
|
||||
i += best.tokenCount;
|
||||
}
|
||||
return mentions;
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolves free-text `name` (a `ParsedRecipeIngredient.name`, e.g. `"large
|
||||
* diced yellow onions"`) to the best-matching `Ingredient` in `catalog`, or
|
||||
|
|
|
|||
|
|
@ -17,11 +17,12 @@ import { env } from "../../config/env.js";
|
|||
* shape.
|
||||
*/
|
||||
|
||||
/** One candidate technique mention the service's `PhraseMatcher` found — offsets `[start, end)`, same convention as `String.prototype.slice`. Mirrors `EntityPayload` (Python `schemas.py`). */
|
||||
/** One candidate mention one of the service's two `PhraseMatcher`s found — offsets `[start, end)`, same convention as `String.prototype.slice`. Mirrors `EntityPayload` (Python `schemas.py`). `kind` distinguishes a technique mention (`self._matcher`, the corpus-trained one) from a utensil mention (`self._utensil_matcher`, static — see `utensil_vocabulary.py`) — `tech-step-matcher.ts` resolves each against a different catalog (`TechStep`/`Utensil`). */
|
||||
export interface IntentServiceEntity {
|
||||
uid: string;
|
||||
start: number;
|
||||
end: number;
|
||||
kind: "technique" | "utensil";
|
||||
}
|
||||
|
||||
/** The full result of a `POST /v1/process` call — mirrors `ProcessResponse` (Python `schemas.py`). `intent` is `null` only when `locale` isn't one this service trains for, or `text` is blank; otherwise always a real `uid` (the Python service's `textcat` has no "None" sentinel, unlike node-nlp — see that service's README). */
|
||||
|
|
|
|||
|
|
@ -1,4 +1,10 @@
|
|||
import { prisma } from "../../db/prisma.js";
|
||||
import {
|
||||
findIngredientMentions,
|
||||
type IngredientMention,
|
||||
loadIngredientCatalog,
|
||||
loadUnitCatalog,
|
||||
} from "./ingredient-matcher.js";
|
||||
import { intentServiceClient } from "./intent-service-client.js";
|
||||
|
||||
/**
|
||||
|
|
@ -90,6 +96,15 @@ export function normalizeText(text: string): string {
|
|||
* Persisted as `StepTechStep.start`/`end`/`contextStart`/`contextEnd`
|
||||
* (`recipe.service.ts`) so the recipe detail view can highlight both spans,
|
||||
* not just know a technique was mentioned somewhere.
|
||||
*
|
||||
* `ingredients`/`utensils` are the metadata found in this match's own
|
||||
* *clause* (see this file's doc comment, point 2) — an ingredient/utensil
|
||||
* mentioned in a different clause of the same description belongs to
|
||||
* *that* clause's own match, never this one, the same "judged on its own
|
||||
* surrounding context" rule the technique itself is judged by. Always `[]`
|
||||
* rather than omitted when nothing was found, so every caller can iterate
|
||||
* unconditionally. Persisted as `StepTechStepIngredient`/`StepTechStepUtensil`
|
||||
* rows (`recipe.service.ts`).
|
||||
*/
|
||||
export interface TechStepMatch {
|
||||
techStepId: number;
|
||||
|
|
@ -97,6 +112,21 @@ export interface TechStepMatch {
|
|||
end: number;
|
||||
contextStart: number;
|
||||
contextEnd: number;
|
||||
ingredients: IngredientMention[];
|
||||
utensils: UtensilMention[];
|
||||
}
|
||||
|
||||
/**
|
||||
* A utensil mention found by the intent service's utensil `PhraseMatcher`
|
||||
* (`kind: "utensil"` entities in `IntentServiceProcessResult`, see
|
||||
* `intent-service-client.ts`), resolved to a local `Utensil.id` and
|
||||
* attributed to whichever clause its span falls inside — same
|
||||
* `[start, end)` convention as every other span in this file.
|
||||
*/
|
||||
export interface UtensilMention {
|
||||
utensilId: number;
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
/** A candidate technique mention found by NER — the raw material {@link splitIntoClauses} cuts a description around. */
|
||||
|
|
@ -310,6 +340,10 @@ export class TechStepClassifierService {
|
|||
private _techStepIdsLoaded: Promise<void> | undefined;
|
||||
private _techStepIdByUid: Map<string, number> | undefined;
|
||||
|
||||
/** Same memoized-lookup shape as {@link _techStepIdsLoaded}/{@link _techStepIdByUid}, for `Utensil.key -> id` instead — a `kind: "utensil"` entity from the intent service resolves through this map, never `_techStepIdByUid`. */
|
||||
private _utensilIdsLoaded: Promise<void> | undefined;
|
||||
private _utensilIdByUid: Map<string, number> | undefined;
|
||||
|
||||
/**
|
||||
* Forces the `TechStep.key -> id` lookup to load now, synchronously with
|
||||
* server startup (see `server.ts`, which also retries this against a
|
||||
|
|
@ -345,22 +379,35 @@ export class TechStepClassifierService {
|
|||
*/
|
||||
public async matchTechStepSpans(description: string, locale: string): Promise<TechStepMatch[]> {
|
||||
try {
|
||||
await this._ensureTechStepIdsLoaded();
|
||||
await Promise.all([this._ensureTechStepIdsLoaded(), this._ensureUtensilIdsLoaded()]);
|
||||
if (description.trim().length === 0) return [];
|
||||
|
||||
// The intent service only ever returns enum-style candidates (its own
|
||||
// `PhraseMatcher`, built solely from `TECH_STEP_TRAINING_DATA`'s
|
||||
// `synonyms`) — unlike node-nlp, it never mixes in built-in
|
||||
// numbers/durations/dates entities, so no `type === "enum"` filter is
|
||||
// needed here anymore. Its `start`/`end` are already `[start, end)`
|
||||
// Loaded fresh per call (once per step, see `recipe.service.ts`'s
|
||||
// `matchStepsTechSteps`) rather than memoized like the id lookups
|
||||
// above — same "cheap enough, and reference data can change between
|
||||
// calls without a restart" posture `loadIngredientCatalog`/
|
||||
// `loadUnitCatalog`'s own doc comments already describe for their
|
||||
// other callers (`ingredient-matcher.ts`, `sources.service.ts`).
|
||||
const [ingredientCatalog, unitCatalog] = await Promise.all([
|
||||
loadIngredientCatalog(locale),
|
||||
loadUnitCatalog(locale),
|
||||
]);
|
||||
|
||||
// The intent service returns two kinds of candidate (see `kind` on
|
||||
// `IntentServiceEntity`): technique mentions (its corpus-trained
|
||||
// `PhraseMatcher`) and utensil mentions (its static one, see
|
||||
// `utensil_vocabulary.py`). Only the former ever anchor a clause —
|
||||
// `splitIntoClauses` cuts a description around *techniques*, a
|
||||
// mentioned utensil doesn't introduce a clause boundary of its own,
|
||||
// it just gets attributed to whichever clause its span falls inside
|
||||
// (see the loop below). Its `start`/`end` are already `[start, end)`
|
||||
// (matching `String.prototype.slice`), unlike node-nlp's inclusive
|
||||
// `end` — no `+ 1` needed either.
|
||||
const nerResult = await intentServiceClient.process(locale, description);
|
||||
const candidates: TechniqueCandidate[] = nerResult.entities.map((entity) => ({
|
||||
uid: entity.uid,
|
||||
start: entity.start,
|
||||
end: entity.end,
|
||||
}));
|
||||
const candidates: TechniqueCandidate[] = nerResult.entities
|
||||
.filter((entity) => entity.kind === "technique")
|
||||
.map((entity) => ({ uid: entity.uid, start: entity.start, end: entity.end }));
|
||||
const utensilEntities = nerResult.entities.filter((entity) => entity.kind === "utensil");
|
||||
|
||||
const clauses = splitIntoClauses(description, candidates);
|
||||
const matches: TechStepMatch[] = [];
|
||||
|
|
@ -374,12 +421,35 @@ export class TechStepClassifierService {
|
|||
// persist a dangling id.
|
||||
if (techStepId === undefined) continue;
|
||||
const span = clause.anchor ?? { start: clause.start, end: clause.end };
|
||||
|
||||
const ingredients = findIngredientMentions(
|
||||
description.slice(clause.start, clause.end),
|
||||
ingredientCatalog,
|
||||
unitCatalog,
|
||||
locale,
|
||||
).map((mention) => ({
|
||||
...mention,
|
||||
start: mention.start + clause.start,
|
||||
end: mention.end + clause.start,
|
||||
}));
|
||||
|
||||
const utensils: UtensilMention[] = utensilEntities.flatMap((entity) => {
|
||||
if (entity.start < clause.start || entity.end > clause.end) return [];
|
||||
const utensilId = this._utensilIdByUid?.get(entity.uid);
|
||||
// Same drift guard as `techStepId` above.
|
||||
return utensilId === undefined
|
||||
? []
|
||||
: [{ utensilId, start: entity.start, end: entity.end }];
|
||||
});
|
||||
|
||||
matches.push({
|
||||
techStepId,
|
||||
start: span.start,
|
||||
end: span.end,
|
||||
contextStart: clause.start,
|
||||
contextEnd: clause.end,
|
||||
ingredients,
|
||||
utensils,
|
||||
});
|
||||
}
|
||||
|
||||
|
|
@ -413,11 +483,9 @@ export class TechStepClassifierService {
|
|||
if (description.trim().length === 0) return [];
|
||||
|
||||
const nerResult = await intentServiceClient.process(locale, description);
|
||||
const candidates: TechniqueCandidate[] = nerResult.entities.map((entity) => ({
|
||||
uid: entity.uid,
|
||||
start: entity.start,
|
||||
end: entity.end,
|
||||
}));
|
||||
const candidates: TechniqueCandidate[] = nerResult.entities
|
||||
.filter((entity) => entity.kind === "technique")
|
||||
.map((entity) => ({ uid: entity.uid, start: entity.start, end: entity.end }));
|
||||
|
||||
const clauses = splitIntoClauses(description, candidates);
|
||||
const results: TechStepClauseClassification[] = [];
|
||||
|
|
@ -522,6 +590,28 @@ export class TechStepClassifierService {
|
|||
throw err; // see matchTechStepSpans()'s catch comment above
|
||||
}
|
||||
}
|
||||
|
||||
/** `Utensil.key -> id` counterpart of {@link _ensureTechStepIdsLoaded} — same memoize-once-retry-on-failure shape. */
|
||||
private async _ensureUtensilIdsLoaded(): Promise<void> {
|
||||
if (this._utensilIdsLoaded === undefined) {
|
||||
this._utensilIdsLoaded = this._loadUtensilIds();
|
||||
}
|
||||
try {
|
||||
await this._utensilIdsLoaded;
|
||||
} catch (err) {
|
||||
this._utensilIdsLoaded = undefined;
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
private async _loadUtensilIds(): Promise<void> {
|
||||
try {
|
||||
const utensils = await prisma.utensil.findMany({ select: { id: true, key: true } });
|
||||
this._utensilIdByUid = new Map(utensils.map((utensil) => [utensil.key, utensil.id]));
|
||||
} catch (err) {
|
||||
throw err; // see matchTechStepSpans()'s catch comment above
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Single shared instance — every caller reuses the one memoized `TechStep.key -> id` lookup rather than re-querying the DB. The actual model training (expensive — a couple of minutes, both locales combined) happens entirely inside `services/tech-step-intent-service`'s own startup, not here — see that service's `_TRAINING_ITERATIONS`. */
|
||||
|
|
|
|||
|
|
@ -280,10 +280,28 @@ export async function submitTechStepCorrection(
|
|||
input.correctedTechStepId ?? null,
|
||||
);
|
||||
|
||||
// Same nested `ingredients`/`utensils` include as `recipe.service.ts`'s
|
||||
// `recipeInclude` — `toStepTechStepViews` (reused below) expects it,
|
||||
// so the fresh sequence read right after a manual correction resolves
|
||||
// exactly the same way a normal `GET /recipes/:id` would.
|
||||
const freshTechSteps = await tx.stepTechStep.findMany({
|
||||
where: { stepId: step.id },
|
||||
orderBy: { order: "asc" },
|
||||
include: { techStep: true },
|
||||
include: {
|
||||
techStep: true,
|
||||
ingredients: {
|
||||
include: {
|
||||
ingredient: {
|
||||
include: {
|
||||
allergies: { include: { allergy: { include: { category: true } } } },
|
||||
diets: { include: { diet: true } },
|
||||
},
|
||||
},
|
||||
unit: true,
|
||||
},
|
||||
},
|
||||
utensils: { include: { utensil: true } },
|
||||
},
|
||||
});
|
||||
|
||||
return { correction: createdCorrection, techSteps: freshTechSteps };
|
||||
|
|
|
|||
|
|
@ -45,7 +45,29 @@ function recipeInclude(viewerId: number) {
|
|||
steps: {
|
||||
orderBy: { order: "asc" },
|
||||
include: {
|
||||
techSteps: { orderBy: { order: "asc" }, include: { techStep: true } },
|
||||
techSteps: {
|
||||
orderBy: { order: "asc" },
|
||||
include: {
|
||||
techStep: true,
|
||||
// Same `allergies`/`diets` nesting as this function's own
|
||||
// top-level `ingredients` include above — reused by
|
||||
// `toIngredientView` so a mentioned ingredient resolves to the
|
||||
// exact same `IngredientView` shape as the recipe's main
|
||||
// ingredient list, not a second, thinner shape.
|
||||
ingredients: {
|
||||
include: {
|
||||
ingredient: {
|
||||
include: {
|
||||
allergies: { include: { allergy: { include: { category: true } } } },
|
||||
diets: { include: { diet: true } },
|
||||
},
|
||||
},
|
||||
unit: true,
|
||||
},
|
||||
},
|
||||
utensils: { include: { utensil: true } },
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
diets: { include: { diet: true } },
|
||||
|
|
@ -145,7 +167,8 @@ export function toStepTechStepViews(
|
|||
): StepTechStepView[] {
|
||||
const views: StepTechStepView[] = [];
|
||||
for (const stepTechStep of techSteps) {
|
||||
const { start, end, contextStart, contextEnd, techStep, source } = stepTechStep;
|
||||
const { start, end, contextStart, contextEnd, techStep, source, ingredients, utensils } =
|
||||
stepTechStep;
|
||||
if (start === null || end === null) continue;
|
||||
views.push({
|
||||
techStep: { id: techStep.id, key: techStep.key },
|
||||
|
|
@ -159,6 +182,19 @@ export function toStepTechStepViews(
|
|||
// `StepTechStepView.source` to the frontend.
|
||||
source: source === "manual" ? "manual" : "auto",
|
||||
...(contextStart !== null && contextEnd !== null ? { contextStart, contextEnd } : {}),
|
||||
ingredients: ingredients.map((stepTechStepIngredient) => ({
|
||||
ingredient: toIngredientView(stepTechStepIngredient.ingredient),
|
||||
quantity:
|
||||
stepTechStepIngredient.quantity === null ? null : Number(stepTechStepIngredient.quantity),
|
||||
unit: stepTechStepIngredient.unit === null ? null : toUnitView(stepTechStepIngredient.unit),
|
||||
start: stepTechStepIngredient.start,
|
||||
end: stepTechStepIngredient.end,
|
||||
})),
|
||||
utensils: utensils.map((stepTechStepUtensil) => ({
|
||||
utensil: { id: stepTechStepUtensil.utensil.id, key: stepTechStepUtensil.utensil.key },
|
||||
start: stepTechStepUtensil.start,
|
||||
end: stepTechStepUtensil.end,
|
||||
})),
|
||||
});
|
||||
}
|
||||
return views;
|
||||
|
|
@ -553,6 +589,22 @@ async function createRecipeInternal(
|
|||
contextStart: match.contextStart,
|
||||
contextEnd: match.contextEnd,
|
||||
order,
|
||||
ingredients: {
|
||||
create: match.ingredients.map((ingredient) => ({
|
||||
ingredientId: ingredient.ingredientId,
|
||||
quantity: ingredient.quantity,
|
||||
unitId: ingredient.unitId,
|
||||
start: ingredient.start,
|
||||
end: ingredient.end,
|
||||
})),
|
||||
},
|
||||
utensils: {
|
||||
create: match.utensils.map((utensil) => ({
|
||||
utensilId: utensil.utensilId,
|
||||
start: utensil.start,
|
||||
end: utensil.end,
|
||||
})),
|
||||
},
|
||||
})),
|
||||
},
|
||||
})),
|
||||
|
|
|
|||
|
|
@ -7,6 +7,7 @@ import {
|
|||
getSources,
|
||||
getTechSteps,
|
||||
getUnits,
|
||||
getUtensils,
|
||||
} from "./reference.service.js";
|
||||
|
||||
/**
|
||||
|
|
@ -55,6 +56,13 @@ referenceRouter.get(
|
|||
}),
|
||||
);
|
||||
|
||||
referenceRouter.get(
|
||||
"/utensils",
|
||||
wrapAsyncHandler(async (_req, res) => {
|
||||
res.status(200).json(await getUtensils());
|
||||
}),
|
||||
);
|
||||
|
||||
referenceRouter.get(
|
||||
"/sources",
|
||||
wrapAsyncHandler(async (_req, res) => {
|
||||
|
|
|
|||
|
|
@ -5,6 +5,7 @@ import type {
|
|||
SourceView,
|
||||
TechStepView,
|
||||
UnitView,
|
||||
UtensilView,
|
||||
} from "@batch-cooking/shared";
|
||||
import { prisma } from "../../db/prisma.js";
|
||||
|
||||
|
|
@ -85,6 +86,19 @@ export async function getTechSteps(): Promise<TechStepView[]> {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* All reference cooking utensils, ordered by key (see {@link getDiets} for
|
||||
* why) — small, static list (see `reference-seed-data.ts`'s `UTENSILS`),
|
||||
* same bare `id`/`key` shape as {@link getTechSteps}.
|
||||
*/
|
||||
export async function getUtensils(): Promise<UtensilView[]> {
|
||||
try {
|
||||
return await prisma.utensil.findMany({ orderBy: { key: "asc" } });
|
||||
} catch (err) {
|
||||
throw err; // see getDiets()'s catch comment above
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Every implemented recipe source, ordered by name (not `key` — unlike
|
||||
* every other reference catalog, `name` here *is* the display string a
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ import { RecipeSourceError } from "../../lib/recipe-sources/recipe-source-errors
|
|||
import { getRecipeSource } from "../../lib/recipe-sources/recipe-source-registry.js";
|
||||
import { getHouseSourceIds } from "../house/house.service.js";
|
||||
import { createImportedRecipe } from "../recipe/recipe.service.js";
|
||||
import { getIngredients, getUnits } from "../reference/reference.service.js";
|
||||
import { getIngredients, getUnits, getUtensils } from "../reference/reference.service.js";
|
||||
|
||||
/**
|
||||
* Browsing, previewing, and importing a household's *enabled* external
|
||||
|
|
@ -190,9 +190,14 @@ export async function previewSourceItem(
|
|||
unitCatalog,
|
||||
adapter.locale,
|
||||
);
|
||||
const [ingredientViews, unitViews] = await Promise.all([getIngredients(), getUnits()]);
|
||||
const [ingredientViews, unitViews, utensilViews] = await Promise.all([
|
||||
getIngredients(),
|
||||
getUnits(),
|
||||
getUtensils(),
|
||||
]);
|
||||
const ingredientById = new Map(ingredientViews.map((view) => [view.id, view]));
|
||||
const unitById = new Map(unitViews.map((view) => [view.id, view]));
|
||||
const utensilById = new Map(utensilViews.map((view) => [view.id, view]));
|
||||
|
||||
// A source's raw ingredient lines aren't deduplicated by the matcher —
|
||||
// two different lines (e.g. "Egg Yolks"/"Eggs") can resolve to the same
|
||||
|
|
@ -231,6 +236,26 @@ export async function previewSourceItem(
|
|||
// comment) — always the classifier's own live match,
|
||||
// never a correction, so always "auto".
|
||||
source: "auto",
|
||||
ingredients: match.ingredients.flatMap((mention) => {
|
||||
const ingredient = ingredientById.get(mention.ingredientId);
|
||||
// Same drift guard as `techStep` above — an ingredientId
|
||||
// the matcher resolved but that's since vanished from the
|
||||
// catalog is dropped rather than shown with a hole in it.
|
||||
if (!ingredient) return [];
|
||||
return [
|
||||
{
|
||||
ingredient,
|
||||
quantity: mention.quantity,
|
||||
unit: mention.unitId !== null ? (unitById.get(mention.unitId) ?? null) : null,
|
||||
start: mention.start,
|
||||
end: mention.end,
|
||||
},
|
||||
];
|
||||
}),
|
||||
utensils: match.utensils.flatMap((mention) => {
|
||||
const utensil = utensilById.get(mention.utensilId);
|
||||
return utensil ? [{ utensil, start: mention.start, end: mention.end }] : [];
|
||||
}),
|
||||
},
|
||||
]
|
||||
: [];
|
||||
|
|
|
|||
|
|
@ -3,6 +3,7 @@ import { expect } from "chai";
|
|||
import { prisma } from "../../src/db/prisma.js";
|
||||
import {
|
||||
extractQuantity,
|
||||
findIngredientMentions,
|
||||
type IngredientMatchEntry,
|
||||
loadIngredientCatalog,
|
||||
loadUnitCatalog,
|
||||
|
|
@ -266,6 +267,100 @@ describe("ingredient-matcher", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("findIngredientMentions", () => {
|
||||
const butter: IngredientMatchEntry = { ingredientId: 1, label: "Butter" };
|
||||
const flour: IngredientMatchEntry = { ingredientId: 2, label: "Flour" };
|
||||
const egg: IngredientMatchEntry = { ingredientId: 3, label: "Egg" };
|
||||
const catalog = [butter, flour, egg];
|
||||
const gram: UnitMatchEntry = { unitId: 1, synonyms: ["g", "gram", "grams"] };
|
||||
const unitCatalog = [gram];
|
||||
|
||||
it("finds a single mention with no quantity or unit", () => {
|
||||
const text = "melt the butter";
|
||||
const mentions = findIngredientMentions(text, catalog, unitCatalog);
|
||||
expect(mentions).to.have.length(1);
|
||||
const [mention] = mentions;
|
||||
expect(mention?.ingredientId).to.equal(butter.ingredientId);
|
||||
expect(text.slice(mention?.start, mention?.end)).to.equal("butter");
|
||||
expect(mention?.quantity).to.equal(null);
|
||||
expect(mention?.unitId).to.equal(null);
|
||||
});
|
||||
|
||||
it('resolves a quantity and unit glued directly to the ingredient ("200g butter")', () => {
|
||||
const text = "add 200g butter";
|
||||
const [mention] = findIngredientMentions(text, catalog, unitCatalog);
|
||||
expect(mention?.ingredientId).to.equal(butter.ingredientId);
|
||||
expect(mention?.quantity).to.equal(200);
|
||||
expect(mention?.unitId).to.equal(gram.unitId);
|
||||
});
|
||||
|
||||
it("finds several mentions in reading order, non-overlapping", () => {
|
||||
const text = "melt the butter then add the flour and an egg";
|
||||
const mentions = findIngredientMentions(text, catalog, unitCatalog);
|
||||
expect(mentions.map((mention) => mention.ingredientId)).to.deep.equal([
|
||||
butter.ingredientId,
|
||||
flour.ingredientId,
|
||||
egg.ingredientId,
|
||||
]);
|
||||
});
|
||||
|
||||
it("is case- and accent-insensitive", () => {
|
||||
const text = "MELT THE BUTTER";
|
||||
const [mention] = findIngredientMentions(text, catalog, unitCatalog);
|
||||
expect(mention?.ingredientId).to.equal(butter.ingredientId);
|
||||
});
|
||||
|
||||
it("ignores an unrelated number earlier in the text (e.g. an oven temperature)", () => {
|
||||
const text = "preheat to 180 degrees then add the egg";
|
||||
const [mention] = findIngredientMentions(text, catalog, unitCatalog);
|
||||
expect(mention?.ingredientId).to.equal(egg.ingredientId);
|
||||
expect(mention?.quantity).to.equal(null);
|
||||
});
|
||||
|
||||
it("returns an empty array when nothing in the catalog is mentioned", () => {
|
||||
expect(findIngredientMentions("stir well", catalog, unitCatalog)).to.deep.equal([]);
|
||||
});
|
||||
|
||||
it("returns an empty array for empty text", () => {
|
||||
expect(findIngredientMentions("", catalog, unitCatalog)).to.deep.equal([]);
|
||||
});
|
||||
|
||||
describe("locale: fr", () => {
|
||||
const beurre: IngredientMatchEntry = { ingredientId: 10, label: "Beurre" };
|
||||
const farine: IngredientMatchEntry = { ingredientId: 11, label: "Farine" };
|
||||
const frCatalog = [beurre, farine];
|
||||
const gramme: UnitMatchEntry = { unitId: 40, synonyms: ["g", "gr", "gramme", "grammes"] };
|
||||
const cuillereASoupe: UnitMatchEntry = {
|
||||
unitId: 41,
|
||||
synonyms: ["cuillère à soupe", "cuillères à soupe", "càs"],
|
||||
};
|
||||
const frUnitCatalog = [gramme, cuillereASoupe];
|
||||
|
||||
it("resolves a quantity and unit before the ingredient, connected by 'de'", () => {
|
||||
const text = "faire fondre 50g de beurre";
|
||||
const [mention] = findIngredientMentions(text, frCatalog, frUnitCatalog, "fr");
|
||||
expect(mention?.ingredientId).to.equal(beurre.ingredientId);
|
||||
expect(mention?.quantity).to.equal(50);
|
||||
expect(mention?.unitId).to.equal(gramme.unitId);
|
||||
expect(text.slice(mention?.start, mention?.end)).to.equal("beurre");
|
||||
});
|
||||
|
||||
it('resolves a multi-word unit connected by "d\'"', () => {
|
||||
const text = "ajouter 2 cuillères à soupe de farine";
|
||||
const [mention] = findIngredientMentions(text, frCatalog, frUnitCatalog, "fr");
|
||||
expect(mention?.ingredientId).to.equal(farine.ingredientId);
|
||||
expect(mention?.quantity).to.equal(2);
|
||||
expect(mention?.unitId).to.equal(cuillereASoupe.unitId);
|
||||
});
|
||||
|
||||
it("is accent-insensitive", () => {
|
||||
const text = "FAIRE FONDRE LE BEURRE";
|
||||
const [mention] = findIngredientMentions(text, frCatalog, frUnitCatalog, "fr");
|
||||
expect(mention?.ingredientId).to.equal(beurre.ingredientId);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("loadIngredientCatalog / loadUnitCatalog", () => {
|
||||
beforeEach(async () => {
|
||||
await resetDatabase();
|
||||
|
|
|
|||
|
|
@ -135,10 +135,34 @@ describe("tech-step-matcher", () => {
|
|||
let meltId: number;
|
||||
let boilId: number;
|
||||
let chopId: number;
|
||||
// Real seeded catalog entries that also happen to be mentioned by
|
||||
// several fixtures below now that `matchTechStepSpans` also resolves
|
||||
// ingredient/utensil metadata — see `matchTechStepSpans`'s own describe
|
||||
// block for where each of these gets used.
|
||||
let panId: number;
|
||||
let saucepanId: number;
|
||||
let butterId: number;
|
||||
let onionId: number;
|
||||
let walnutsId: number;
|
||||
let gramId: number;
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetDatabase();
|
||||
const [simmer, cook, bake, preheat, melt, boil, chop] = await Promise.all([
|
||||
const [
|
||||
simmer,
|
||||
cook,
|
||||
bake,
|
||||
preheat,
|
||||
melt,
|
||||
boil,
|
||||
chop,
|
||||
pan,
|
||||
saucepan,
|
||||
butter,
|
||||
onion,
|
||||
walnuts,
|
||||
gram,
|
||||
] = await Promise.all([
|
||||
prisma.techStep.findFirstOrThrow({ where: { key: "simmer" } }),
|
||||
prisma.techStep.findFirstOrThrow({ where: { key: "cook" } }),
|
||||
prisma.techStep.findFirstOrThrow({ where: { key: "bake" } }),
|
||||
|
|
@ -146,6 +170,17 @@ describe("tech-step-matcher", () => {
|
|||
prisma.techStep.findFirstOrThrow({ where: { key: "melt" } }),
|
||||
prisma.techStep.findFirstOrThrow({ where: { key: "boil" } }),
|
||||
prisma.techStep.findFirstOrThrow({ where: { key: "chop" } }),
|
||||
prisma.utensil.findFirstOrThrow({ where: { key: "pan" } }),
|
||||
prisma.utensil.findFirstOrThrow({ where: { key: "saucepan" } }),
|
||||
prisma.ingredient.findFirstOrThrow({ where: { key: "butter" } }),
|
||||
prisma.ingredient.findFirstOrThrow({ where: { key: "onion" } }),
|
||||
// "Noix" (walnuts) — turns out to also be a real seeded ingredient
|
||||
// label, and "noix" is literally the French word for "a pat of
|
||||
// butter" ("une noix de beurre") used in one of the fixtures
|
||||
// below, so it's a genuine (if slightly comical) second match
|
||||
// alongside "beurre" in that clause, not a fixture bug.
|
||||
prisma.ingredient.findFirstOrThrow({ where: { key: "walnuts" } }),
|
||||
prisma.unit.findFirstOrThrow({ where: { key: "gram" } }),
|
||||
]);
|
||||
simmerId = simmer.id;
|
||||
cookId = cook.id;
|
||||
|
|
@ -154,6 +189,12 @@ describe("tech-step-matcher", () => {
|
|||
meltId = melt.id;
|
||||
boilId = boil.id;
|
||||
chopId = chop.id;
|
||||
panId = pan.id;
|
||||
saucepanId = saucepan.id;
|
||||
butterId = butter.id;
|
||||
onionId = onion.id;
|
||||
walnutsId = walnuts.id;
|
||||
gramId = gram.id;
|
||||
});
|
||||
|
||||
after(async () => {
|
||||
|
|
@ -255,7 +296,15 @@ describe("tech-step-matcher", () => {
|
|||
const text = "Faire mijoter à feu doux";
|
||||
const result = await techStepClassifier.matchTechStepSpans(text, "fr");
|
||||
expect(result).to.deep.equal([
|
||||
{ techStepId: simmerId, start: 6, end: 13, contextStart: 0, contextEnd: text.length },
|
||||
{
|
||||
techStepId: simmerId,
|
||||
start: 6,
|
||||
end: 13,
|
||||
contextStart: 0,
|
||||
contextEnd: text.length,
|
||||
ingredients: [],
|
||||
utensils: [],
|
||||
},
|
||||
]);
|
||||
expect(text.slice(6, 13).toLowerCase()).to.equal("mijoter");
|
||||
});
|
||||
|
|
@ -305,6 +354,12 @@ describe("tech-step-matcher", () => {
|
|||
end: 21,
|
||||
contextStart: 0,
|
||||
contextEnd: 22,
|
||||
// "poêle" (the pan) sits inside this very clause — a separate
|
||||
// utensil mention from `preheat`'s own "poêle chaude" keyword
|
||||
// span above, found by the intent service's *other* PhraseMatcher
|
||||
// (see `IntentServiceEntity.kind`).
|
||||
ingredients: [],
|
||||
utensils: [{ utensilId: panId, start: 9, end: 14 }],
|
||||
});
|
||||
expect(result[1]).to.deep.equal({
|
||||
techStepId: meltId,
|
||||
|
|
@ -312,6 +367,15 @@ describe("tech-step-matcher", () => {
|
|||
end: 37,
|
||||
contextStart: 22,
|
||||
contextEnd: text.length,
|
||||
// Two mentions in this clause: "noix" (walnuts — also a real
|
||||
// seeded ingredient, and literally the French word this phrase
|
||||
// uses for "a pat of [butter]") *and* "beurre" itself, in
|
||||
// reading order.
|
||||
ingredients: [
|
||||
{ ingredientId: walnutsId, start: 42, end: 46, quantity: null, unitId: null },
|
||||
{ ingredientId: butterId, start: 50, end: 56, quantity: null, unitId: null },
|
||||
],
|
||||
utensils: [],
|
||||
});
|
||||
expect(text.slice(result[0].start, result[0].end)).to.equal("poêle chaude");
|
||||
expect(text.slice(result[0].contextStart, result[0].contextEnd)).to.equal(
|
||||
|
|
@ -333,6 +397,15 @@ describe("tech-step-matcher", () => {
|
|||
end: text.length,
|
||||
contextStart: 0,
|
||||
contextEnd: text.length,
|
||||
// "beurre" and "poêle" are both mentioned in this same
|
||||
// anchor-less clause (there's no literal `melt` keyword here at
|
||||
// all — the whole point of this test, see its own title) —
|
||||
// still resolved, since ingredient/utensil scanning doesn't
|
||||
// depend on the clause having a technique anchor of its own.
|
||||
ingredients: [
|
||||
{ ingredientId: butterId, start: 18, end: 24, quantity: null, unitId: null },
|
||||
],
|
||||
utensils: [{ utensilId: panId, start: 45, end: 50 }],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
|
@ -341,10 +414,42 @@ describe("tech-step-matcher", () => {
|
|||
const text = "Chop the onions finely";
|
||||
const result = await techStepClassifier.matchTechStepSpans(text, "en");
|
||||
expect(result).to.deep.equal([
|
||||
{ techStepId: chopId, start: 0, end: 4, contextStart: 0, contextEnd: text.length },
|
||||
{
|
||||
techStepId: chopId,
|
||||
start: 0,
|
||||
end: 4,
|
||||
contextStart: 0,
|
||||
contextEnd: text.length,
|
||||
ingredients: [
|
||||
{ ingredientId: onionId, start: 9, end: 15, quantity: null, unitId: null },
|
||||
],
|
||||
utensils: [],
|
||||
},
|
||||
]);
|
||||
expect(text.slice(0, 4)).to.equal("Chop");
|
||||
});
|
||||
|
||||
it("resolves a quantity+unit and a utensil alongside the technique, all from the same clause", async () => {
|
||||
const text = "faire fondre 50g de beurre dans une casserole";
|
||||
const result = await techStepClassifier.matchTechStepSpans(text, "fr");
|
||||
|
||||
expect(result).to.deep.equal([
|
||||
{
|
||||
techStepId: meltId,
|
||||
start: 0,
|
||||
end: 12,
|
||||
contextStart: 0,
|
||||
contextEnd: text.length,
|
||||
ingredients: [
|
||||
{ ingredientId: butterId, start: 20, end: 26, quantity: 50, unitId: gramId },
|
||||
],
|
||||
utensils: [{ utensilId: saucepanId, start: 36, end: 45 }],
|
||||
},
|
||||
]);
|
||||
expect(text.slice(0, 12)).to.equal("faire fondre");
|
||||
expect(text.slice(20, 26)).to.equal("beurre");
|
||||
expect(text.slice(36, 45)).to.equal("casserole");
|
||||
});
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
|
|
@ -3,7 +3,7 @@ import request from "supertest";
|
|||
import { createApp } from "../src/app.js";
|
||||
import { prisma } from "../src/db/prisma.js";
|
||||
import { syncRecipeSources } from "../src/db/recipe-source-sync.js";
|
||||
import { seedReferenceData, TECH_STEPS } from "../src/db/reference-seed-data.js";
|
||||
import { seedReferenceData, TECH_STEPS, UTENSILS } from "../src/db/reference-seed-data.js";
|
||||
import type { RecipeSourceAdapter } from "../src/lib/recipe-sources/recipe-source-adapter.js";
|
||||
import {
|
||||
clearRecipeSources,
|
||||
|
|
@ -161,6 +161,31 @@ describe("Reference data", () => {
|
|||
});
|
||||
});
|
||||
|
||||
describe("GET /reference/utensils", () => {
|
||||
it("returns the seeded utensils, no session required", async () => {
|
||||
const res = await request(app).get("/reference/utensils");
|
||||
|
||||
expect(res.status).to.equal(200);
|
||||
expect(res.body).to.have.length(UTENSILS.length);
|
||||
expect(res.body.map((u: { key: string }) => u.key)).to.include("pan");
|
||||
expect(res.body[0]).to.have.keys(["id", "key"]);
|
||||
});
|
||||
|
||||
it("orders utensils alphabetically by key", async () => {
|
||||
const res = await request(app).get("/reference/utensils");
|
||||
|
||||
const keys = res.body.map((u: { key: string }) => u.key);
|
||||
expect(keys).to.deep.equal([...keys].sort());
|
||||
});
|
||||
|
||||
it("reseeding is idempotent — no duplicate utensils", async () => {
|
||||
await seedReferenceData(prisma);
|
||||
|
||||
const res = await request(app).get("/reference/utensils");
|
||||
expect(res.body).to.have.length(UTENSILS.length);
|
||||
});
|
||||
});
|
||||
|
||||
describe("GET /reference/sources", () => {
|
||||
afterEach(() => {
|
||||
clearRecipeSources();
|
||||
|
|
|
|||
|
|
@ -475,6 +475,38 @@
|
|||
"toast": "Torréfier",
|
||||
"zest": "Zester"
|
||||
},
|
||||
"utensils": {
|
||||
"pan": "Poêle",
|
||||
"saucepan": "Casserole",
|
||||
"pot": "Marmite",
|
||||
"knife": "Couteau",
|
||||
"whisk": "Fouet",
|
||||
"bowl": "Saladier",
|
||||
"bakingSheet": "Plaque de cuisson",
|
||||
"mold": "Moule",
|
||||
"colander": "Passoire",
|
||||
"cuttingBoard": "Planche à découper",
|
||||
"oven": "Four",
|
||||
"blender": "Blender",
|
||||
"mixer": "Batteur",
|
||||
"spatula": "Spatule",
|
||||
"ladle": "Louche",
|
||||
"grater": "Râpe",
|
||||
"rollingPin": "Rouleau à pâtisserie",
|
||||
"lid": "Couvercle",
|
||||
"tongs": "Pince de cuisine",
|
||||
"peeler": "Économe",
|
||||
"sieve": "Tamis",
|
||||
"foodProcessor": "Robot ménager",
|
||||
"steamerBasket": "Panier vapeur",
|
||||
"skewer": "Brochette",
|
||||
"pastryBrush": "Pinceau de cuisine",
|
||||
"ramekin": "Ramequin",
|
||||
"dish": "Plat",
|
||||
"wok": "Wok",
|
||||
"thermometer": "Thermomètre",
|
||||
"mandoline": "Mandoline"
|
||||
},
|
||||
"allergens": {
|
||||
"gluten": "Gluten",
|
||||
"crustaceans": "Crustacés",
|
||||
|
|
|
|||
|
|
@ -1,4 +1,11 @@
|
|||
import type { AllergyView, DietView, IngredientView, TechStepView, UnitView } from "./reference.js";
|
||||
import type {
|
||||
AllergyView,
|
||||
DietView,
|
||||
IngredientView,
|
||||
TechStepView,
|
||||
UnitView,
|
||||
UtensilView,
|
||||
} from "./reference.js";
|
||||
|
||||
/**
|
||||
* Who can *read* a recipe — mirrors `RecipeVisibility` in schema.prisma.
|
||||
|
|
@ -49,6 +56,11 @@ export interface RecipeIngredientView {
|
|||
* immediately (`recipe-tech-step-correction.service.ts`'s
|
||||
* `applyManualCorrection`). `StepDescription.tsx` renders the two with a
|
||||
* different highlight color so a viewer can tell which is which.
|
||||
*
|
||||
* `ingredients`/`utensils` are the metadata found in this technique's own
|
||||
* clause (see `tech-step-matcher.ts`'s `TechStepMatch` — same source data,
|
||||
* just resolved to full reference views here instead of bare ids) — `[]`
|
||||
* when nothing was mentioned alongside this technique.
|
||||
*/
|
||||
export interface StepTechStepView {
|
||||
techStep: TechStepView;
|
||||
|
|
@ -57,6 +69,32 @@ export interface StepTechStepView {
|
|||
contextStart?: number;
|
||||
contextEnd?: number;
|
||||
source: "auto" | "manual";
|
||||
ingredients: StepTechStepIngredientView[];
|
||||
utensils: StepTechStepUtensilView[];
|
||||
}
|
||||
|
||||
/**
|
||||
* An ingredient mentioned in the same clause as a detected technique (see
|
||||
* {@link StepTechStepView.ingredients}) — `quantity`/`unit` are `null` when
|
||||
* none was recognized immediately before the mention (e.g. "ajouter le
|
||||
* sel"), same "best-effort, not always present" contract as
|
||||
* `tech-step-matcher.ts`'s `IngredientMention`. `start`/`end` are the
|
||||
* mention's own span in the step's `description`, same `[start, end)`
|
||||
* convention as {@link StepTechStepView.start}.
|
||||
*/
|
||||
export interface StepTechStepIngredientView {
|
||||
ingredient: IngredientView;
|
||||
quantity: number | null;
|
||||
unit: UnitView | null;
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
/** A utensil mentioned in the same clause as a detected technique (see {@link StepTechStepView.utensils}). */
|
||||
export interface StepTechStepUtensilView {
|
||||
utensil: UtensilView;
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
|
|||
|
|
@ -210,6 +210,24 @@ export interface TechStepView {
|
|||
key: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A cooking utensil, as returned by `GET /reference/utensils` — reference
|
||||
* data (`Utensil`, seeded via `reference-seed-data.ts`'s `UTENSILS`), same
|
||||
* bare `id`+`key` shape and static/non-administrable status as
|
||||
* {@link TechStepView}. Detected in a step's free text the same way
|
||||
* techniques are (see `StepTechStepUtensilView`), but via a static
|
||||
* `PhraseMatcher` rather than a trained classifier — see
|
||||
* `services/tech-step-intent-service`'s `utensil_vocabulary.py`.
|
||||
*
|
||||
* `key` is a stable English camelCase uid (e.g. `"pan"`), not a display
|
||||
* label — resolved via `t(\`catalog.utensils.${key}\`)`, same as
|
||||
* {@link TechStepView.key}.
|
||||
*/
|
||||
export interface UtensilView {
|
||||
id: number;
|
||||
key: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* An implemented recipe source, as returned by `GET /reference/sources` —
|
||||
* reference data (`Source`, kept in sync with the adapter registry by
|
||||
|
|
|
|||
|
|
@ -11,6 +11,12 @@ pipeline `node-nlp` qui vivait dans `apps/api`
|
|||
de la technique qu'une clause de texte *signifie*, entraîné sur les
|
||||
`utterances` de chaque technique (y compris des paraphrases n'utilisant
|
||||
jamais le mot-clé lui-même).
|
||||
3. **NER par phrases, ustensiles** (`spacy.matcher.PhraseMatcher`, second
|
||||
matcher indépendant) — trouve les mentions d'un ustensile de cuisine
|
||||
(`intent_service/utensil_vocabulary.py`, `UTENSIL_VOCABULARY`), sans
|
||||
`textcat` associé : contrairement à une technique, un ustensile mentionné
|
||||
n'a pas besoin d'être interprété selon le contexte. Renvoyé dans la même
|
||||
liste `entities` que les techniques, discriminé par `kind`.
|
||||
|
||||
Basé sur **spaCy** (`fr_core_news_md`/`en_core_web_md`) plutôt que node-nlp —
|
||||
écosystème NLP plus robuste/maintenu, avec l'ambition à terme (hors scope de
|
||||
|
|
@ -34,7 +40,10 @@ de `POST /v1/train`, supprimé).
|
|||
Workflow mainteneur pour changer le corpus :
|
||||
|
||||
1. Éditer `intent_service/training_data.py` à la main (informé par le
|
||||
rapport de `apps/api/src/scripts/list-pending-training-suggestions.ts`).
|
||||
rapport de `apps/api/src/scripts/list-pending-training-suggestions.ts`)
|
||||
pour une technique, ou `intent_service/utensil_vocabulary.py` pour un
|
||||
ustensile (pas de rapport équivalent pour ce dernier — pas de mécanisme
|
||||
de correction utilisateur sur les ustensiles aujourd'hui).
|
||||
2. **Redémarrer ce service** (`docker compose restart tech-step-intent-service`,
|
||||
ou simplement redéployer) — le nouveau corpus n'a d'effet qu'une fois
|
||||
réentraîné au démarrage, contrairement à l'ancienne version qui pouvait
|
||||
|
|
@ -61,7 +70,10 @@ Voir `intent_service/schemas.py` pour le détail exact. En résumé :
|
|||
entièrement prêt : modèles spaCy de base chargés **et** les deux locales
|
||||
entraînées (pas de lazy-load, voir `intent_service/main.py`) — voir
|
||||
"Temps de démarrage" plus bas pour ce que ça implique en pratique.
|
||||
- `POST /v1/process` — `{ locale, text }` → `{ entities: [{ uid, start, end }], intent, score }`.
|
||||
- `POST /v1/process` — `{ locale, text }` → `{ entities: [{ uid, start, end, kind }], intent, score }`,
|
||||
`kind` valant `"technique"` ou `"utensil"` selon le `PhraseMatcher` qui a
|
||||
trouvé la mention (voir point 3 ci-dessus). `apps/api`'s `tech-step-matcher.ts`
|
||||
filtre par `kind` pour savoir laquelle des deux résoudre (`TechStep`/`Utensil`).
|
||||
|
||||
`/v1/process` exige le header `X-Intent-Service-Secret` (voir
|
||||
`intent_service/security.py`), qui doit matcher `INTENT_SERVICE_SECRET`
|
||||
|
|
@ -96,7 +108,7 @@ et son output complets, `pipeline_registry.py` journalise le déroulement de
|
|||
l'entraînement au démarrage :
|
||||
|
||||
```json
|
||||
{"timestamp": "...", "level": "info", "message": "tech-step NLP process", "locale": "fr", "text": "faire fondre le beurre", "entities": [{"uid": "melt", "start": 6, "end": 13}], "intent": "melt", "score": 0.93}
|
||||
{"timestamp": "...", "level": "info", "message": "tech-step NLP process", "locale": "fr", "text": "faire fondre le beurre", "entities": [{"uid": "melt", "start": 6, "end": 13, "kind": "technique"}], "intent": "melt", "score": 0.93}
|
||||
```
|
||||
|
||||
Le chatter interne de spaCy (`"spacy"` logger — chargement de vocabulaire,
|
||||
|
|
@ -138,6 +150,9 @@ uv run pytest
|
|||
exacts et d'insensibilité accents/casse de
|
||||
`apps/api/test/recipe-matching/tech-step-matcher.test.ts` — le point de
|
||||
fidélité le plus critique de ce service (voir le plan de migration).
|
||||
`tests/test_utensil_matching.py` couvre le second `PhraseMatcher`
|
||||
(ustensiles) de la même façon, contre le vocabulaire réel (statique, pas
|
||||
besoin d'un jeu de test dédié comme pour les techniques).
|
||||
`tests/conftest.py`'s fixture `client` (scope "session") ne s'entraîne
|
||||
qu'une seule fois pour toute la suite — c'est *le vrai corpus complet*,
|
||||
pas un jeu jouet, donc la première utilisation de cette fixture prend le
|
||||
|
|
|
|||
|
|
@ -30,6 +30,7 @@ from spacy.tokens import Doc, Span
|
|||
from spacy.training import Example
|
||||
from spacy.util import filter_spans, fix_random_seed, minibatch
|
||||
|
||||
from . import utensil_vocabulary
|
||||
from .text_normalization import normalize_text
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
|
@ -167,12 +168,18 @@ class TrainEntry:
|
|||
|
||||
@dataclass
|
||||
class Entity:
|
||||
"""Une mention candidate trouvée par le `PhraseMatcher` — offsets
|
||||
caractère `[start, end)`, miroir de `EntityPayload` (`schemas.py`)."""
|
||||
"""Une mention candidate trouvée par un `PhraseMatcher` — offsets
|
||||
caractère `[start, end)`, miroir de `EntityPayload` (`schemas.py`).
|
||||
`kind` distingue de quel `PhraseMatcher` la mention vient (`"technique"`
|
||||
— `self._matcher`, entraîné depuis `training_data.py` — ou `"utensil"`
|
||||
— `self._utensil_matcher`, statique, voir `utensil_vocabulary.py`) :
|
||||
`apps/api`'s `tech-step-matcher.ts` a besoin de savoir laquelle des deux
|
||||
résoudre (`TechStep.key` vs `Utensil.key`)."""
|
||||
|
||||
uid: str
|
||||
start: int
|
||||
end: int
|
||||
kind: str = "technique"
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
@ -210,6 +217,11 @@ class LocalePipeline:
|
|||
# comportement testé côté `apps/api` pour "une locale jamais
|
||||
# entraînée".
|
||||
self._matcher: PhraseMatcher | None = None
|
||||
# Construit une seule fois par `preload()`, jamais par `train()` —
|
||||
# contrairement à `self._matcher`, ce vocabulaire est statique
|
||||
# (`utensil_vocabulary.py`), il n'a pas de contrepartie "corpus
|
||||
# poussé par un appelant" à reconstruire.
|
||||
self._utensil_matcher: PhraseMatcher | None = None
|
||||
self._trained = False
|
||||
|
||||
@property
|
||||
|
|
@ -217,10 +229,18 @@ class LocalePipeline:
|
|||
return self._trained
|
||||
|
||||
def preload(self) -> None:
|
||||
"""Charge le modèle spaCy de base (tokenizer + vecteurs) et le
|
||||
composant `diacritics_normalizer` — idempotent, sans effet si déjà
|
||||
chargé. Appelé au démarrage du process pour les deux locales
|
||||
connues (voir `main.py`), pas paresseusement au premier `train()`.
|
||||
"""Charge le modèle spaCy de base (tokenizer + vecteurs), le
|
||||
composant `diacritics_normalizer`, et construit le `PhraseMatcher`
|
||||
d'ustensiles — idempotent, sans effet si déjà chargé. Appelé au
|
||||
démarrage du process pour les deux locales connues (voir
|
||||
`main.py`), pas paresseusement au premier `train()`.
|
||||
|
||||
Le matcher d'ustensiles est construit ici, pas dans `train()` :
|
||||
contrairement au `PhraseMatcher` de techniques (reconstruit à
|
||||
chaque `train()` depuis les `entries` reçues), le vocabulaire
|
||||
d'ustensiles est statique (`utensil_vocabulary.py`) — rien ne le
|
||||
fait jamais varier d'un appel à l'autre, donc rien ne justifie de
|
||||
payer son coût de construction plus d'une fois par démarrage.
|
||||
"""
|
||||
if self._base_nlp is not None:
|
||||
return
|
||||
|
|
@ -228,6 +248,15 @@ class LocalePipeline:
|
|||
nlp.add_pipe("diacritics_normalizer", first=True)
|
||||
self._base_nlp = nlp
|
||||
|
||||
diacritics_normalizer = nlp.get_pipe("diacritics_normalizer")
|
||||
utensil_matcher = PhraseMatcher(nlp.vocab, attr="NORM")
|
||||
for uid, synonyms in utensil_vocabulary.synonyms_for_locale(self._locale).items():
|
||||
if not synonyms:
|
||||
continue
|
||||
patterns = [diacritics_normalizer(nlp.make_doc(synonym)) for synonym in synonyms]
|
||||
utensil_matcher.add(uid, patterns)
|
||||
self._utensil_matcher = utensil_matcher
|
||||
|
||||
def train(self, entries: list[TrainEntry]) -> tuple[int, int, int]:
|
||||
"""Reconstruit le `textcat` et le `PhraseMatcher` de ce pipeline à
|
||||
partir de `entries` (le tokenizer/les vecteurs restent ceux chargés
|
||||
|
|
@ -409,13 +438,41 @@ class LocalePipeline:
|
|||
matched_spans = [
|
||||
Span(doc, start, end, label=match_id) for match_id, start, end in self._matcher(doc)
|
||||
]
|
||||
entities = sorted(
|
||||
(
|
||||
Entity(uid=self._base_nlp.vocab.strings[span.label], start=span.start_char, end=span.end_char)
|
||||
for span in filter_spans(matched_spans)
|
||||
),
|
||||
key=lambda entity: entity.start,
|
||||
)
|
||||
technique_entities = [
|
||||
Entity(
|
||||
uid=self._base_nlp.vocab.strings[span.label],
|
||||
start=span.start_char,
|
||||
end=span.end_char,
|
||||
kind="technique",
|
||||
)
|
||||
for span in filter_spans(matched_spans)
|
||||
]
|
||||
|
||||
# Second, independent `PhraseMatcher` pass for ustensiles — run and
|
||||
# `filter_spans`-resolved *separately* from the technique pass
|
||||
# above: the two matchers' candidates never compete for the same
|
||||
# position (a longer utensil match must never swallow/be swallowed
|
||||
# by a technique match the way two overlapping technique synonyms
|
||||
# do), only overlaps *within* the same matcher are the known
|
||||
# problem `filter_spans` exists for (see the technique pass's own
|
||||
# comment above).
|
||||
utensil_entities: list[Entity] = []
|
||||
if self._utensil_matcher is not None:
|
||||
utensil_spans = [
|
||||
Span(doc, start, end, label=match_id)
|
||||
for match_id, start, end in self._utensil_matcher(doc)
|
||||
]
|
||||
utensil_entities = [
|
||||
Entity(
|
||||
uid=self._base_nlp.vocab.strings[span.label],
|
||||
start=span.start_char,
|
||||
end=span.end_char,
|
||||
kind="utensil",
|
||||
)
|
||||
for span in filter_spans(utensil_spans)
|
||||
]
|
||||
|
||||
entities = sorted(technique_entities + utensil_entities, key=lambda entity: entity.start)
|
||||
|
||||
cats = doc.cats
|
||||
if not cats:
|
||||
|
|
|
|||
|
|
@ -30,14 +30,20 @@ def process(request: ProcessRequest) -> ProcessResponse:
|
|||
extra={
|
||||
"locale": request.locale,
|
||||
"text": request.text,
|
||||
"entities": [{"uid": entity.uid, "start": entity.start, "end": entity.end} for entity in result.entities],
|
||||
"entities": [
|
||||
{"uid": entity.uid, "start": entity.start, "end": entity.end, "kind": entity.kind}
|
||||
for entity in result.entities
|
||||
],
|
||||
"intent": result.intent,
|
||||
"score": result.score,
|
||||
},
|
||||
)
|
||||
|
||||
return ProcessResponse(
|
||||
entities=[EntityPayload(uid=entity.uid, start=entity.start, end=entity.end) for entity in result.entities],
|
||||
entities=[
|
||||
EntityPayload(uid=entity.uid, start=entity.start, end=entity.end, kind=entity.kind)
|
||||
for entity in result.entities
|
||||
],
|
||||
intent=result.intent,
|
||||
score=result.score,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -7,6 +7,8 @@ depuis `training_data.py` (voir `pipeline_registry.py`/`main.py`), plus
|
|||
besoin d'un contrat HTTP pour ça.
|
||||
"""
|
||||
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
|
|
@ -20,14 +22,21 @@ class ProcessRequest(BaseModel):
|
|||
|
||||
|
||||
class EntityPayload(BaseModel):
|
||||
"""Une mention candidate d'une technique — offsets caractère `[start, end)`
|
||||
dans `text`, convention identique à `String.prototype.slice` côté
|
||||
`apps/api` (pas de décalage `+1` à appliquer côté Node, contrairement à
|
||||
l'ancien `NlpManager` de node-nlp)."""
|
||||
"""Une mention candidate — technique ou ustensile, voir `kind` — offsets
|
||||
caractère `[start, end)` dans `text`, convention identique à
|
||||
`String.prototype.slice` côté `apps/api` (pas de décalage `+1` à
|
||||
appliquer côté Node, contrairement à l'ancien `NlpManager` de
|
||||
node-nlp).
|
||||
|
||||
`kind` distingue de quel `PhraseMatcher` la mention vient (voir
|
||||
`locale_pipeline.py`'s `Entity`) — `apps/api`'s `tech-step-matcher.ts`
|
||||
en a besoin pour savoir laquelle des deux résoudre (`TechStep.key` vs
|
||||
`Utensil.key`)."""
|
||||
|
||||
uid: str
|
||||
start: int
|
||||
end: int
|
||||
kind: Literal["technique", "utensil"] = "technique"
|
||||
|
||||
|
||||
class ProcessResponse(BaseModel):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,197 @@
|
|||
"""Vocabulaire du `PhraseMatcher` d'ustensiles — contrairement à
|
||||
`training_data.py`, ce catalogue n'a jamais existé côté `apps/api` avant ce
|
||||
service : il est *né* ici, pas rapatrié depuis TypeScript. Chaque `uid`
|
||||
ci-dessous doit avoir une entrée `UTENSILS` correspondante
|
||||
(`reference-seed-data.ts` côté `apps/api`) et un libellé
|
||||
`catalog.utensils.<uid>` (`apps/web`'s `locales/fr/translation.json`).
|
||||
|
||||
Un seul type de contenu par ustensile/locale (contrairement à
|
||||
`TechStepTrainingEntry`'s `synonyms`/`utterances`) : un ustensile mentionné
|
||||
n'a pas besoin d'être *interprété* comme une technique peut l'être
|
||||
(`préchauffer` vs `chauffer` dépend du contexte ; `poêle` n'en dépend pas) —
|
||||
juste reconnu, comme les `synonyms` de `training_data.py` alimentent le
|
||||
`PhraseMatcher` de techniques. Pas de `textcat` équivalent ici, voir
|
||||
`LocalePipeline`'s propre commentaire sur `_utensil_matcher`.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UtensilLocaleVocabulary:
|
||||
synonyms: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class UtensilEntry:
|
||||
"""`uid` doit correspondre à un `Utensil.key`."""
|
||||
|
||||
uid: str
|
||||
fr: UtensilLocaleVocabulary
|
||||
en: UtensilLocaleVocabulary
|
||||
|
||||
|
||||
UTENSIL_VOCABULARY: list[UtensilEntry] = [
|
||||
UtensilEntry(
|
||||
uid="pan",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["poêle", "sauteuse", "poêle antiadhésive"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["pan", "frying pan", "skillet"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="saucepan",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["casserole", "petite casserole"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["saucepan", "sauce pan"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="pot",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["marmite", "faitout", "cocotte"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["pot", "stockpot", "dutch oven"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="knife",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["couteau", "couteau de cuisine", "couteau d'office"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["knife", "kitchen knife", "chef's knife"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="whisk",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["fouet"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["whisk"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="bowl",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["saladier", "bol", "cul-de-poule"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["bowl", "mixing bowl"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="bakingSheet",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["plaque de cuisson", "plaque à pâtisserie", "plaque du four"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["baking sheet", "baking tray", "sheet pan"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="mold",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["moule", "moule à gâteau", "moule à cake"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["mold", "mould", "baking pan"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="colander",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["passoire", "égouttoir"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["colander", "strainer"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="cuttingBoard",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["planche à découper"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["cutting board", "chopping board"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="oven",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["four"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["oven"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="blender",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["blender", "mixeur plongeant", "mixeur girafe"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["blender", "immersion blender"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="mixer",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["batteur", "batteur électrique", "robot pâtissier"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["mixer", "stand mixer", "hand mixer"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="spatula",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["spatule", "maryse"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["spatula"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="ladle",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["louche"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["ladle"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="grater",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["râpe"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["grater"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="rollingPin",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["rouleau à pâtisserie"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["rolling pin"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="lid",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["couvercle"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["lid"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="tongs",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["pince", "pince de cuisine"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["tongs"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="peeler",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["économe", "éplucheur"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["peeler", "vegetable peeler"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="sieve",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["tamis", "chinois"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["sieve"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="foodProcessor",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["robot ménager", "robot de cuisine", "robot culinaire"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["food processor"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="steamerBasket",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["panier vapeur", "cuit-vapeur"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["steamer basket", "steamer"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="skewer",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["brochette", "pique en bois"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["skewer"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="pastryBrush",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["pinceau de cuisine", "pinceau à pâtisserie"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["pastry brush", "basting brush"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="ramekin",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["ramequin"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["ramekin"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="dish",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["plat", "plat à gratin", "plat allant au four"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["dish", "baking dish", "gratin dish"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="wok",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["wok"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["wok"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="thermometer",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["thermomètre", "thermomètre de cuisson"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["thermometer"]),
|
||||
),
|
||||
UtensilEntry(
|
||||
uid="mandoline",
|
||||
fr=UtensilLocaleVocabulary(synonyms=["mandoline"]),
|
||||
en=UtensilLocaleVocabulary(synonyms=["mandoline"]),
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def synonyms_for_locale(locale: str) -> dict[str, list[str]]:
|
||||
"""Aplati {@link UTENSIL_VOCABULARY} en `{uid: synonyms}` pour une seule
|
||||
locale — la forme que `LocalePipeline.preload()` attend pour construire
|
||||
son `PhraseMatcher` d'ustensiles. Miroir de `training_data.entries_for_locale`,
|
||||
en plus simple (pas d'`utterances`, un seul champ à extraire)."""
|
||||
return {
|
||||
entry.uid: getattr(entry, locale).synonyms
|
||||
for entry in UTENSIL_VOCABULARY
|
||||
if hasattr(entry, locale)
|
||||
}
|
||||
|
|
@ -84,10 +84,16 @@ def test_untrained_locale_returns_empty_without_error():
|
|||
|
||||
|
||||
def test_detects_two_techniques_with_exact_tight_spans_reading_order(fr_pipeline: LocalePipeline):
|
||||
# This text's "poêle" is now *also* a real utensil match ("pan", see
|
||||
# `utensil_vocabulary.py`) — filtered out here by `kind` since this test
|
||||
# is specifically about technique-candidate ordering, not the full
|
||||
# mixed entity list (see `test_utensil_matching.py` for the utensil
|
||||
# matcher's own coverage).
|
||||
text = "Préchauffer la poêle, puis faire fondre le beurre"
|
||||
result = fr_pipeline.process(text)
|
||||
|
||||
uids_by_start = sorted(((entity.start, entity.uid) for entity in result.entities))
|
||||
technique_entities = [entity for entity in result.entities if entity.kind == "technique"]
|
||||
uids_by_start = sorted(((entity.start, entity.uid) for entity in technique_entities))
|
||||
assert [uid for _, uid in uids_by_start] == ["preheat", "melt"]
|
||||
|
||||
preheat_entity = next(e for e in result.entities if e.uid == "preheat")
|
||||
|
|
|
|||
|
|
@ -0,0 +1,78 @@
|
|||
"""Couvre `LocalePipeline`'s second `PhraseMatcher` (ustensiles,
|
||||
`utensil_vocabulary.py`) — même style que `test_locale_pipeline_entities.py`
|
||||
(offsets exacts, insensibilité accents/casse), mais contre le vocabulaire
|
||||
*réel* (`UTENSIL_VOCABULARY`, statique, construit par `preload()` — pas
|
||||
besoin d'un jeu de test dédié comme pour les techniques, voir
|
||||
`LocalePipeline.preload`'s own comment)."""
|
||||
|
||||
from intent_service.locale_pipeline import LocalePipeline, TrainEntry
|
||||
|
||||
# Un `train()` minimal suffit — le `PhraseMatcher` d'ustensiles est
|
||||
# construit par `preload()` (appelé par `train()`), indépendamment du
|
||||
# `TrainEntry` de techniques passé ici (voir `preload()`'s own comment sur
|
||||
# pourquoi les deux ne sont pas couplés).
|
||||
_MINIMAL_ENTRIES = [
|
||||
TrainEntry(uid="melt", synonyms=["fondre"], utterances=["faire fondre le beurre"]),
|
||||
TrainEntry(uid="simmer", synonyms=["mijoter"], utterances=["faire mijoter à feu doux"]),
|
||||
]
|
||||
|
||||
|
||||
def _fr_pipeline() -> LocalePipeline:
|
||||
pipeline = LocalePipeline("fr")
|
||||
pipeline.train(_MINIMAL_ENTRIES)
|
||||
return pipeline
|
||||
|
||||
|
||||
def test_matches_a_real_utensil_with_exact_span():
|
||||
pipeline = _fr_pipeline()
|
||||
text = "Dans une poêle chaude, faire fondre le beurre"
|
||||
result = pipeline.process(text)
|
||||
|
||||
pan_entities = [e for e in result.entities if e.uid == "pan"]
|
||||
assert len(pan_entities) == 1
|
||||
entity = pan_entities[0]
|
||||
assert entity.kind == "utensil"
|
||||
assert text[entity.start : entity.end] == "poêle"
|
||||
|
||||
|
||||
def test_is_case_and_accent_insensitive():
|
||||
pipeline = _fr_pipeline()
|
||||
result = pipeline.process("Verser dans la POÊLE")
|
||||
utensil_uids = [e.uid for e in result.entities if e.kind == "utensil"]
|
||||
assert utensil_uids == ["pan"]
|
||||
|
||||
|
||||
def test_matches_a_multi_word_synonym():
|
||||
pipeline = _fr_pipeline()
|
||||
text = "Découper les légumes sur la planche à découper"
|
||||
result = pipeline.process(text)
|
||||
board_entities = [e for e in result.entities if e.uid == "cuttingBoard"]
|
||||
assert len(board_entities) == 1
|
||||
assert text[board_entities[0].start : board_entities[0].end] == "planche à découper"
|
||||
|
||||
|
||||
def test_technique_and_utensil_are_both_returned_without_interfering():
|
||||
pipeline = _fr_pipeline()
|
||||
text = "Dans une casserole, faire mijoter à feu doux"
|
||||
result = pipeline.process(text)
|
||||
|
||||
kinds_by_uid = {e.uid: e.kind for e in result.entities}
|
||||
assert kinds_by_uid.get("simmer") == "technique"
|
||||
assert kinds_by_uid.get("saucepan") == "utensil"
|
||||
|
||||
|
||||
def test_returns_no_utensil_entities_when_none_are_mentioned():
|
||||
pipeline = _fr_pipeline()
|
||||
result = pipeline.process("Laisser reposer la pâte une heure")
|
||||
assert [e for e in result.entities if e.kind == "utensil"] == []
|
||||
|
||||
|
||||
def test_matches_english_utensils_too():
|
||||
pipeline = LocalePipeline("en")
|
||||
pipeline.train([TrainEntry(uid="chop", synonyms=["chop"], utterances=["chop the onions finely"])])
|
||||
text = "Heat the pan before adding the onions"
|
||||
result = pipeline.process(text)
|
||||
pan_entities = [e for e in result.entities if e.uid == "pan"]
|
||||
assert len(pan_entities) == 1
|
||||
assert pan_entities[0].kind == "utensil"
|
||||
assert text[pan_entities[0].start : pan_entities[0].end] == "pan"
|
||||
|
|
@ -555,6 +555,32 @@ combien de temps ça prend).
|
|||
("préchauffer") ne matche jamais sa forme normalisée dans le texte cible
|
||||
(voir le commentaire dans `locale_pipeline.py`'s `train()`).
|
||||
|
||||
**Métadonnées d'action — ingrédients, quantités, ustensiles.** Chaque
|
||||
occurrence de technique (`TechStepMatch`) porte aussi ce qui a été détecté
|
||||
dans sa propre *clause* (celle calculée à l'étape 2 ci-dessus) :
|
||||
- **Ingrédients** — `ingredient-matcher.ts`'s `findIngredientMentions` scanne
|
||||
le texte de la clause contre le catalogue `Ingredient` *existant*
|
||||
(`INGREDIENT_LABELS_FR`/`_EN`, `packages/shared` — le même que
|
||||
`matchIngredientName` utilise déjà pour les listes structurées), plutôt que
|
||||
de dupliquer ce catalogue côté service Python. Une quantité+unité
|
||||
immédiatement avant la mention est résolue au mieux (regex ancrée sur la
|
||||
*fin* du texte précédent, voir `QUANTITY_BEFORE_INGREDIENT_PATTERN`) —
|
||||
`null`/`null` sinon, jamais une erreur.
|
||||
- **Ustensiles** — contrairement aux ingrédients, ce catalogue n'existait
|
||||
nulle part avant cette fonctionnalité : il est né directement côté service
|
||||
Python (`intent_service/utensil_vocabulary.py`), via un second
|
||||
`PhraseMatcher` indépendant du premier (pas de `textcat` — un ustensile
|
||||
mentionné n'a pas besoin d'être interprété, contrairement à une technique).
|
||||
`POST /v1/process` renvoie donc deux types d'entité discriminés par
|
||||
`kind: "technique" | "utensil"` dans la même liste `entities`.
|
||||
|
||||
Dans les deux cas, l'association à une technique se fait par appartenance à
|
||||
la même clause — pas d'analyse syntaxique (le `parser` spaCy reste exclu du
|
||||
pipeline, voir `_EXCLUDED_COMPONENTS`), juste "cette mention tombe dans
|
||||
`[clause.start, clause.end)`". Persisté comme `StepTechStepIngredient`/
|
||||
`StepTechStepUtensil`, deux tables référençant `StepTechStep` par sa clé
|
||||
composite `(stepId, order)`.
|
||||
|
||||
### Résolution ingrédients/unités — `ingredient-matcher.ts`
|
||||
|
||||
**Anglais uniquement** aujourd'hui (commit "matching anglais pour les tech
|
||||
|
|
|
|||
|
|
@ -18,6 +18,9 @@ d'ingrédients/unités normalisé, techniques détectées, visibilité) :
|
|||
- **Planification** — `Planning`, `PlanningItem`
|
||||
- **Recettes** — `Recipe`, `RecipeIngredient`, `Step`, `TechStep`,
|
||||
`StepTechStep`, `RecipeDiet`, `RecipeFavorite`
|
||||
- **Métadonnées d'action** — `Utensil`, `StepTechStepIngredient`,
|
||||
`StepTechStepUtensil` (ingrédients/quantités/ustensiles associés à une
|
||||
technique détectée, voir plus bas)
|
||||
- **Sources externes** — `Source`, `HouseSource`
|
||||
- **Catalogue ingrédients/unités** — `Ingredient`, `Unit`, `IngredientDiet`,
|
||||
`IngredientAllergy`, `UserProfileDislikedIngredient`
|
||||
|
|
@ -371,6 +374,17 @@ surlignage tant que sa recette n'est pas resauvegardée) sont le span détecté
|
|||
dans `Step.description`, utilisé pour le surlignage côté web
|
||||
(`highlight-tech-steps.ts`).
|
||||
|
||||
Chaque `step_tech_step` porte en plus les métadonnées trouvées dans sa propre
|
||||
clause : `step_tech_step_ingredient` (ingrédient résolu contre le catalogue
|
||||
`ingredients` existant, `quantity`/`unit_id` optionnels quand une quantité a
|
||||
pu être extraite juste avant la mention) et `step_tech_step_utensil`
|
||||
(ustensile résolu contre un nouveau catalogue `utensil`, même forme
|
||||
minimale `id`/`key` que `tech_step` — voir
|
||||
[backend-architecture.md](./backend-architecture.md#détection-des-techniques--tech-step-matcherts)
|
||||
pour comment chacun est détecté). Les deux référencent `step_tech_step` par
|
||||
sa clé composite `(step_id, order)`, `onDelete: Cascade` comme le reste de
|
||||
cette chaîne.
|
||||
|
||||
---
|
||||
|
||||
## Relations
|
||||
|
|
|
|||
Loading…
Reference in a new issue