openpencil/scripts/ab-corpus
Fini c835976479 feat(ab-corpus): per-domain cookbook filter (Phase 2 of token diet)
ab-v3 / ab-v4 showed Phase 1A (cookbook strip on obvious difficulty)
shaved ~4.4k tokens off T-obvious. Phase 2 adds a per-category gate
that strips cookbook recipes whose domain doesn't match the prompt's
category — mobile briefs don't see dashboard recipes, dashboard
briefs don't see mobile / landing recipes, etc.

Mechanism: HTML comment block markers in elements.md
(`<!-- @domain:dashboard --> ... <!-- /@domain -->`) plus a
stripNonMatchingDomains() pass in buildSystemPrompt that drops blocks
whose tag list doesn't include the active category. Untagged content
is "general" and stays in every variant — the safe default.

Tagged 7 single-domain cookbook recipes:
- dashboard: Team / members list, Audit / activity feed, Faceted
  search filter sidebar, Dashboard KPI strip
- landing: Pricing section
- mobile: Onboarding "How it works", Support chat thread

Cross-domain recipes (Login, Signup, Settings page, OTP, Empty
inbox) stay untagged so they load for every category. Decision tree
+ PREFER list also untagged today; the per-tool annotations there
would be a much larger judgment pass for marginal additional savings.

Token measurements (chars / 4 estimate):
                       full     mobile  dashboard  landing
- T + composite       19.0k    17.9k    18.4k     17.7k
                              (-1.1k)  (-0.6k)   (-1.3k)
- T + obvious         14.6k    13.5k    14.0k     13.3k
                              (-1.1k)  (-0.6k)   (-1.3k)

Modest absolute savings — Phase 2 only filters cookbook RECIPES (in
elements.md), and most cookbook content is in elements-cookbook.md
which Phase 1A already strips on obvious. To hit the 6-8k T target
we still need decision-tree compression or PREFER-list trim, but
both are lossier than this gate. Phase 3 candidates noted in the
ab-v4 results doc.

real-model.ts plumbs call.prompt.category through to buildSystemPrompt.
3766 vitest pass (+6 category filter tests including a 500-char
floor regression guard that the filter actually shaves bytes).
2026-04-29 09:49:49 +08:00
..
__tests__ feat(ab-corpus): per-domain cookbook filter (Phase 2 of token diet) 2026-04-29 09:49:49 +08:00
clients feat(ab-corpus): bump minimax max_tokens to 8192 (defensive) 2026-04-29 09:49:48 +08:00
apply.ts fix(ab-corpus): plumb multi-tool output end-to-end for composite 2026-04-29 08:35:00 +08:00
build-prompt.ts feat(ab-corpus): per-domain cookbook filter (Phase 2 of token diet) 2026-04-29 09:49:49 +08:00
measure-prompt-sizes.ts feat(ab-corpus): per-domain cookbook filter (Phase 2 of token diet) 2026-04-29 09:49:49 +08:00
real-model.ts feat(ab-corpus): per-domain cookbook filter (Phase 2 of token diet) 2026-04-29 09:49:49 +08:00
run.ts feat(ab-corpus): bootstrap ab-v3 with token cost + composite difficulty 2026-04-29 07:45:00 +08:00
stub-model.ts fix(ai-skills): drop invalid section_header subtitle from cookbook + stubs 2026-04-29 09:49:40 +08:00
write-report.ts feat(ab-corpus): bootstrap ab-v3 with token cost + composite difficulty 2026-04-29 07:45:00 +08:00