batchCooking/services/tech-step-llm-worker

tech-step-llm-worker

Standalone scheduled worker for the tech-step detection reliability feature (see the repo root's feature plan). Periodically:

1. audit-low-confidence — samples clauses apps/api's NLP classifier (tech-step-matcher.ts) itself scored below its own confidence threshold, asks a local LLM for a second opinion, and proposes a new training utterance whenever the LLM disagrees with what the NLP anchor already implied.
2. transform-corrections — drains user-submitted tech-step corrections (StepDescription.tsx's editable mode, apps/web) not yet processed, and asks the LLM to propose new synonyms/example utterances from each one.

Both jobs only ever propose TechStepTrainingSuggestion rows for a maintainer to review — nothing here edits tech-step-training-data.ts automatically. See apps/api/src/scripts/retrain-tech-steps.ts for the maintainer-driven step that actually applies reviewed suggestions.

Why this lives outside the pnpm workspace

Same reasoning as experiments/llm-tech-step-poc: node-llama-cpp's native binding must never end up compiled into apps/api's own install/Docker build. This package has its own package.json/lockfile-less install, entirely separate from pnpm-workspace.yaml (which only covers apps/*/packages/*).

It also has no Prisma client and no direct database access — every read/write goes through apps/api's /internal/tech-steps/* routes (api-client.ts), authenticated with a shared secret (INTERNAL_WORKER_SECRET, must match apps/api's own). This keeps apps/api the single owner of the schema, and keeps this worker a simple "read some text over HTTP, run local inference, POST a suggestion" process with nothing to keep in sync if the schema changes shape.

Setup

cd services/tech-step-llm-worker
pnpm install --ignore-workspace
cp .env.example .env
# edit .env: set INTERNAL_WORKER_SECRET to match apps/api's own
pnpm start          # runs the cron loop
# or:
RUN_ONCE=true pnpm start   # runs both jobs once and exits

The GGUF model (qwen2.5-1.5b by default, Q4_K_M, ~1GB) downloads on first run into ./models/ (gitignored) and is cached there for subsequent runs — expect the very first run to take noticeably longer than later ones. See src/config.ts for every environment variable this reads, including TECH_STEP_LLM_MODEL_PATH to point at an already-downloaded GGUF file instead (useful offline, or when a mid-deploy network download isn't wanted).

Running via Docker Compose

docker-compose.yml (repo root) defines a tech-step-llm-worker service alongside app/postgres — it's optional: set INTERNAL_WORKER_SECRET in the root .env to enable it, leave it unset and the service simply won't start (its environment: block fails loudly if referenced without a value, same posture as the other required secrets in that file).

Testing

pnpm test

Unit tests (test/jobs/*.test.ts) mock api-client.ts's HTTP calls and a fake TechStepLlmService-shaped object directly — no real network calls, no real model loaded, no real apps/api needed. There is currently no integration test exercising a real model against a real apps/api instance; that would need to be run manually (see "Setup" above) before merging any future change to the prompts/schemas in llm-verdict.ts.

Known limitations (first version of this feature)

• Scheduler cadence (TECH_STEP_WORKER_CRON, default weekly) is a provisional floor, not a calibrated value — see the feature's plan document for what it should be tuned against (recipe/correction volume, server resources).
• Sampling in audit-low-confidence only looks at the AUDIT_SAMPLE_SIZE (apps/api's tech-step-worker.service.ts) most-recently-created steps, not the whole recipe catalog — a smarter sampling strategy (e.g. weighted by how often a recipe is actually viewed/planned) is future work.
• No per-key technique definitions are sent to the LLM today — just the bare TechStep.key list (GET /reference/tech-steps, e.g. "panFry", "foldIn"). Adding a short human-readable gloss per technique (a new TechStepView.description field) would likely improve judgeClause's accuracy but is out of scope for this version.
• Locale is always assumed "fr" in transform-corrections — no recipe/step in the app carries its own locale field yet (see recipe.service.ts's DEFAULT_TECH_STEP_LOCALE comment on the API side).