0f22f2dd3e
Adds stub-model FIXTURES entries for all 5 ab-v3 composite prompts
so `--dry-run --corpus ab-v3` exercises the new tool_calls list +
apply loop end-to-end without burning live credits.
Each fixture is a multi-tag raw string:
- dashboard-settings-page-composite → 1× section_header + 4× setting_row
- dashboard-team-people-page-composite → 1× section_header + 5× member_row + 1× invite_row
- dashboard-search-filters-composite → 2× filter_group
- dashboard-audit-feed-composite → 1× section_header + 6× activity_log
- mobile-onboarding-flow-composite → 4× step_card
Verifies the parser/apply/aggregate pipeline end-to-end:
- parseModelOutput surfaces all element-tool tags in emit order
(e.g. 7 tags → 7 calls in tool_calls)
- apply.ts loops `handleElementToolCall` for every call
- countRoles in scoreRun finds every required role across the
multi-tool tree (so M3 min_roles passes)
- aggregate.byTool tallies per-name (12 add_activity_log + 10
add_member_row + 8 add_setting_row + 8 add_step_card +
4 add_filter_group + 4 add_section_header + 2 add_invite_row
in a 2-model dry-run)
Result on `--dry-run --corpus ab-v3`:
- Composite routing 100% multi-tool (was 100% fallback before
fixtures — stub default emitted batch_design)
- M3 T 9.6% (5 composite passes / 52 prompts) — was 0% before,
proving every composite-T row genuinely apply-passes M3
|
||
|---|---|---|
| .. | ||
| clients | ||
| apply.ts | ||
| build-prompt.ts | ||
| real-model.ts | ||
| run.ts | ||
| stub-model.ts | ||
| write-report.ts | ||