Commit graph

3 commits

Author SHA1 Message Date
Fini f6746eb91c fix(ab-corpus): forbid mixing batch_design with element tools in T
The earlier T instruction split tool selection into PRIMARY / COMPOSITE
(element tools) and FALLBACK (batch_design). FALLBACK was scoped
"when no element tool fits a given component shape" — implying you
could mix per-component. But output-parser.ts silently drops every
batch_design tag whenever any element call is also present (filter
ELEMENT_TOOL_NAME_RE then return). A mixed response

  <op_tool>{"name":"add_section_header_v0",...}</op_tool>
  <op_tool>{"name":"batch_design", ...scaffolding...}</op_tool>

would lose the batch_design half and only run the element call —
the brief is half-applied without anyone noticing. Codex stop-time
review caught the mismatch.

Reframes T as a binary choice:
- STRATEGY A: every component fits an add_*_v0 tool — emit one tag
  per component
- STRATEGY B: at least one component needs batch_design — emit a
  SINGLE batch_design covering the whole brief

Plus an explicit "Do not mix Strategy A and Strategy B" guard line.
Tests assert both strategy markers are present in every T variant
and the dropped per-component-fallback phrase is gone.

The elements.md cookbook still teaches batch_design + element tool
composition for real MCP multi-round usage; the corpus T arm is
single-shot so the trailing T_TOOL_CALL_INSTRUCTIONS overrides via
"last instruction wins".
2026-04-29 09:49:45 +08:00
Fini b7e6097da5 fix(ab-corpus): allow multi-tool output in T system prompt
Codex stop-time review caught a contradiction: elements.md taught
"emit one <op_tool> per component" while T_TOOL_CALL_INSTRUCTIONS
explicitly forbade it ("Respond with one tag, nothing else" + "Do
not combine multiple tags"). Live models were probably reading the
later, more authoritative trailing instructions and dropping back to
batch_design — which matches ab-v3's 0/25 composite-T multi-tool
runs even though the cookbook had recipes.

Rewrites the T output-format block to:
- declare "one or more <op_tool> tags" up front
- add an explicit COMPOSITE clause showing chained tags for
  multi-component briefs (settings panel / team list / audit feed /
  onboarding) with a 3-tag worked example
- keep the FALLBACK clause for batch_design when no element tool fits
- keep the "no prose between tags" rule

Tests now assert the multi-tool marker is present in every T variant
AND the forbidding phrases ("one tag, nothing else", "Do not combine
multiple tags") are GONE — the regression guard Codex effectively
asked for. Also pulls in an oxfmt auto-format on the measurement
helper that landed unformatted in a3c5bf2d.
2026-04-29 09:49:44 +08:00
Fini f385f433e1 feat(ab-corpus): gate elements-cookbook on T arm by difficulty
ab-v3 left T prompts at ~22k tokens (vs ~6-7k for B). The 18kb
elements-cookbook teaches per-tool arg shapes, which is what
composite multi-tool chains genuinely need; single-tool obvious
prompts can route correctly from just the decision tree + PREFER
list alone.

buildSystemPrompt now takes opts.difficulty. T+obvious strips the
cookbook (saves ~18kb on the 47/52 ab-v3 obvious prompts);
T+composite, T+optional, and undefined keep both halves. B variant
unchanged — still strips both, so the A/B comparison stays clean.

Verified by 8 new build-prompt.test.ts cases including a 10kb floor
on the obvious-vs-composite delta. ab-v4 will measure whether the
diet hurts arg compliance on weak models; per-domain split is the
Phase 2 fallback if obvious-T garbage rate creeps up.
2026-04-29 09:49:42 +08:00