The Iconify catalog was 41% of the web wasm; ~91% of its bytes are 3700
simple-icons brand logos vs only 0.46 MB for lucide+feather. Split it: embed the
core UI sets, load brands at runtime — desktop embeds + serves them
(/assets/iconify-catalog-brands.json), web fetches that route at mount and
registers via set_brand_catalog. Web wasm 13.7 -> 8.2 MB raw, 4.17 -> 2.18 MB
gzip (-48% over the wire); desktop keeps all icons. Brands registered in main()
before every native render path (GUI / --render-shots / MCP).
Close the non-web-productionization gaps from the 2026-06-18 recheck:
- a11y (#67/#57): assemble each widget's access_node() into an
accesskit::TreeUpdate (op-editor-ui/accessibility.rs); publish on
desktop via accesskit_macos/_windows/_unix SubclassingAdapter off the
raw window handle (op-host-desktop/a11y.rs, NOT accesskit_winit — the
casement winit fork), and on web via a hidden ARIA DOM mirror
(op-host-web/a11y_dom.rs); native/web region enumeration + action
routing in op-host-{native,web}.
- single-instance (#51): fixed-loopback-port guard + second-launch file
forwarding to the running window (op-host-desktop/single_instance.rs).
- layer panel: drop-into-container inserts at index 0 (#12a); container
predicate widened to frame/group/rectangle/ref (#12b); Escape closes
the context menu on both hosts (#14).
- distribution: Homebrew cask fix + op formula + install-op.sh, README
CLI install, CI codesign/notarize/signtool scaffolding gated on secrets.
Codex-reviewed (1 BLOCKER + 3 CONCERNs resolved). main.rs, app_handler.rs,
canvaskit.rs and Cargo.lock are staged whole and carry some unrelated
in-progress WIP they're interleaved with.
Point the built-in GlmCoding preset at glm-5.2 (was glm-4.7) and add a glm-5.2 route to the ab-v9 matrix via Zhipu's official coding plan (GLM_BASE + GLM_KEY env). Keys stay env-only.
scripts/ab-v9/run_matrix.py runs the ab-v3 corpus (52 prompts) through
the full Rust orchestrator per provider (op-smoke headless,
OPENPENCIL_MANIFEST=1) and scores M3 expected-shape (required roles in
the saved .op tree) + M5 element selection, appending rows per cell for
crash-safe resume. Keys come from env only (MM_KEY/ARK_KEY/DS_KEY).
op-smoke grows OPENPENCIL_SMOKE_KEEP_THINKING=1 to keep MiniMax
reasoning ON: ab-v9 showed M3-nothink emits lazy minimal manifests
(17%, ~10s answers) while M3-with-thinking lands 60% with composite
tied-best at ~110s — the MiniMax production routing target.
14th pre-validation detector + a preventive skill rule.
User-reported 2026-05-10 "Bistro" mobile food app shipped with root
padding [0,16,0,16] AND a "Today's Specials" section padding [0,24].
Effective gutter = 40px on a 375px page → only 295px of usable
content width. Reads as "too much padding" / pinched.
Two pieces:
1. layout.md AESTHETIC HYGIENE block now teaches "page gutter goes
on ONE layer, not both" — pick root horizontal padding OR
per-section horizontal padding, not both. Default convention:
root carries the gutter, sections set vertical-only padding.
Hero / banner / image-bleed sections then sit edge-to-edge by
simply NOT adding horizontal padding (root's gutter shows
through). Preventive teaching at prompt time.
2. detectStackedHorizontalPadding (info-only, detect-only). Walks
every mobile-shaped root (width 320–480 + tall + multi-child),
compares root horizontal padding against each direct child's
horizontal padding; flags the section as the offender when both
are > 0. Page-shape filter mirrors detectEdgeSectionPadding so
the legitimate component-internal padding stacking pattern
(chip → badge → icon, etc.) doesn't trip it. Severity is INFO
because a section may legitimately want a deeper inset for
visual emphasis — let the user/agent decide via audit panel.
Side-quest: scripts/ab-corpus/check-stacked-padding.ts ships with
this commit so the next stacked-padding-style detector calibration
can survey corpus frequency without rebuilding the harness.
Codex stop-hook review caught: the detectTextBgContrast ancestor walk
treated any wrapper with a solid `fill` entry as the bg color, even
when the fill was effectively invisible. The classic miss case:
page { fill: cream }
└─ wrapper { fill: [{ type: 'solid', color: '#FFFFFF', opacity: 0 }] }
└─ text { fill: cream }
Without the guard, the detector picked the wrapper's white fill as bg
and reported a healthy contrast ratio against the cream text — masking
the real cream-on-cream failure that lives one level up.
firstSolidColor() now skips fills with `opacity === 0` and 8-hex colors
whose alpha byte is `00` (e.g. `#FFFFFF00`). Both produce no visible
color, so the ancestor walk continues past them to the real bg.
Semi-transparent fills (opacity 0.5, 8-hex alpha 80, etc.) are out of
scope — the detector still treats them as opaque rather than trying to
math the layered composite. Tests pin both: opacity=0.5 + alpha=80
stay treated as bg.
4 new test cases cover the fix plus the boundary (opacity=0.5, alpha=80
should NOT be skipped). Full corpus replay shows 14 hits unchanged on
the 470-row corpus — no false-positive regression introduced.
Two complement scripts that ride alongside replay-detectors.ts:
- inspect-shape.ts: bucket every applied row's root by (width-bucket /
height-bucket / aspect-ratio / child-count). Used 2026-05-10 to
diagnose why detectEdgeSectionPadding scored 0 hits on a 220-row
mobile subset — turned out 49% of mobile rows produce roots with a
string-typed width ("fill_container" / "fit_content") because the
element-tools path emits component fragments, not pages. The
detector predicate `typeof width === 'number'` correctly skips them
→ 0 hits is the EVAL HARNESS coverage gap, not a detector bug.
- inspect-issue-category.ts: print every issue in a chosen category
with row id + node id + reason. Generic version of inspect-contrast-
hits.ts. 2026-05-10 used it to validate
excessive-frame-effects (4/4 TP — blur 48 cards + OTP slot spread)
and the two borderline mixed-sibling hits (header padding outlier,
spacer cornerRadius outlier — known role-aware limitation, 0.2%
noise rate, accepted).
Together with replay-detectors.ts these three give a fast empirical
loop for tuning a detector against real corpus output without burning
fresh API tokens.
Replayed the 2026-05-08-rank4-gpt55 corpus (104 GPT-5.5 dashboard
outputs, 95 applied) through the new detectTextBgContrast and got
41 hits — 43% of designs flagged. Sampling showed almost all of them
were industry-standard Tailwind palettes used as intentional tertiary
text:
- #94A3B8 (slate-400) caption on #FFFFFF, ratio 2.56 ← Linear/Vercel/Notion
- #2563EB (blue-600) chip on #DBEAFE, ratio 4.24 ← shadcn/ui tag pattern
- #10B981 (emerald-500) delta on #FFFFFF, ratio 2.54 ← stat-positive pattern
- #64748B (slate-500) row text on #F1F5F9, ratio 4.34 ← muted-row pattern
WCAG-AA 4.5:1 is a compliance threshold, not a design-diagnosis
threshold. The user-reported pain point is "white-on-cream" (1.10:1)
and "white-on-white" (1.0:1) — disasters that read as obviously broken
to anyone, not borderline-WCAG cases that production designers ship
on purpose.
Drop default normalThreshold to 2.5 and largeThreshold to 2.0. Open
both as opts so callers needing a stricter audit (e.g. compliance
report) can bring back WCAG-AA without re-implementing the walk.
Replay confirms the new thresholds:
- 41 hits → 6 hits (signal-to-noise from 50% to 0% on the sample)
- All 6 remaining are true positives:
* 3 × slate-400 on slate-100 (caption color used on a non-white
bg — designer mis-paired the palette)
* 3 × white initial on amber-500 avatar (the readability gap the
industry routinely ignores; legitimately worth flagging)
Codex review (a47ef892f72a2d315) confirmed the direction, the
specific numeric pair (2.5 not 3.0 — 3.0 still hits slate-400 at 2.56),
parameterization over a mode-flag, and keeping severity at info-only.
Side-quest: scripts/ab-corpus/replay-detectors.ts +
inspect-contrast-hits.ts ship with this commit so the next detector
calibration doesn't have to rebuild the harness from scratch.
GAP-1 (fontWeight) + GAP-2 (empty-vars early-exit) are fixed in
52f9549b — replace stale knownGap() entries with positive assertions
that lock in the new resolver behavior. Smoke test is now full
regression coverage: 28/28 PASS, 0 known gaps.
Validates full pipeline: createEmptyDocument → applySemanticPalette →
buildHeadingV1/buildSettingRowV1/etc. (theme:'system') → resolveNodeForCanvas
Light/Dark. Surfaces 2 known gaps in the resolver (fontWeight not resolved;
DEFAULT_PALETTE_FALLBACK unreachable for empty-vars docs).
The 4 extra single-value tokens (color-accent-dark, color-info-surface,
color-warning-text-strong, color-danger-text-strong) introduced in P1.1.6
violated spec §3.1 / §7.4 — those hex were INTENDED to merge into existing
tokens with ≤ 5% accepted color drift, not become new tokens.
Replaced with MERGE_MAP in measure-v0-hex-coverage.ts that tracks the 4
near-shade redirections (#1D4ED8→color-accent, #EFF6FF→color-info-bg,
#B45309→color-warning-text, #B91C1C→color-danger-text). Cover rate
calculation now reports direct + merge breakdown.
Final palette token count: 56 (28 color + 18 type + 2 letterSpacing +
5 spacing + 3 radius). Cover rate: 28 direct + 4 merge = 32/32 = 100.0%.
Codex stop-time review caught the previous commit (113bd55a) message
overstating apply.ts's behavior — I claimed "scripts/ab-corpus/apply
both catch per-shape and keep running the remaining tags" but the
loop at line 55 had no inner try/catch. A single throw from any
handleElementToolCall (e.g. the heading invalid-level reject 113bd55a
just added) would bubble up through the outer try at line 40 and
return early, dropping every remaining tag in a composite batch on
the floor — gpt-5.4's 13-tag team-people-page response would lose
tags 12-13 instead of just tag 12.
Wraps each handleElementToolCall in its own try/catch + accumulates
failures into a per-shape list. ELEMENT_TOOL_NAMES miss is also a
push-and-continue (was a return). When `failures.length > 0` we
return ok:false with a message listing every failed tag, AND the
partial PenDocument that DID land — so M3 (role coverage) can still
score the 11 tags that worked. M1 stays strict (any failure → false).
Mirrors apps/web/src/services/ai/element-tools-dispatcher::
dispatchElementToolCalls's "collect-errors-keep-going" semantics —
production already worked this way; ab-corpus now does too.
3772 vitest pass, format clean, tsc silent. Existing dry-run + live
sweeps exercise the path; a focused apply.ts unit test would need
pen-mcp setup that the harness's existing build-prompt test sidesteps,
so leaving that as a followup.
Codex stop-time review caught that the Phase 2 domain filter
corrupted output when two adjacent kept blocks abutted: the regex's
trailing `\s*` consumed every character of whitespace BEFORE the next
match's leading `\s*`, so block N's body ended on its closing ``` and
block N+1's body started on its `### heading` with no separator at
all. Output: ` ```### Audit / activity feed` on a single line — fence
left unclosed, heading swallowed.
Replace the greedy `\s*` on both sides of each marker with a literal
`\n`, so the regex only consumes the single newline immediately
adjoining the comment. Surrounding blank lines stay in the
surrounding text where they belong, separating adjacent kept blocks
naturally.
Adds a regression guard test that searches the output for ` ```###`
(closing fence directly followed by a level-3 heading on the same
line). Codex effectively asked for it.
3767 vitest pass, format clean. Side effect: dropped blocks now
leave their surrounding blank lines intact, so dropped-block savings
shrink by ~3-5 chars per block; total Phase 2 savings stay within
the 500-char floor the existing test asserts.
ab-v3 / ab-v4 showed Phase 1A (cookbook strip on obvious difficulty)
shaved ~4.4k tokens off T-obvious. Phase 2 adds a per-category gate
that strips cookbook recipes whose domain doesn't match the prompt's
category — mobile briefs don't see dashboard recipes, dashboard
briefs don't see mobile / landing recipes, etc.
Mechanism: HTML comment block markers in elements.md
(`<!-- @domain:dashboard --> ... <!-- /@domain -->`) plus a
stripNonMatchingDomains() pass in buildSystemPrompt that drops blocks
whose tag list doesn't include the active category. Untagged content
is "general" and stays in every variant — the safe default.
Tagged 7 single-domain cookbook recipes:
- dashboard: Team / members list, Audit / activity feed, Faceted
search filter sidebar, Dashboard KPI strip
- landing: Pricing section
- mobile: Onboarding "How it works", Support chat thread
Cross-domain recipes (Login, Signup, Settings page, OTP, Empty
inbox) stay untagged so they load for every category. Decision tree
+ PREFER list also untagged today; the per-tool annotations there
would be a much larger judgment pass for marginal additional savings.
Token measurements (chars / 4 estimate):
full mobile dashboard landing
- T + composite 19.0k 17.9k 18.4k 17.7k
(-1.1k) (-0.6k) (-1.3k)
- T + obvious 14.6k 13.5k 14.0k 13.3k
(-1.1k) (-0.6k) (-1.3k)
Modest absolute savings — Phase 2 only filters cookbook RECIPES (in
elements.md), and most cookbook content is in elements-cookbook.md
which Phase 1A already strips on obvious. To hit the 6-8k T target
we still need decision-tree compression or PREFER-list trim, but
both are lossier than this gate. Phase 3 candidates noted in the
ab-v4 results doc.
real-model.ts plumbs call.prompt.category through to buildSystemPrompt.
3766 vitest pass (+6 category filter tests including a 500-char
floor regression guard that the filter actually shaves bytes).
ab-v4 raw output capture on dashboard-search-filters-composite shows
minimax-m2.7 emitting <think>...</think> + 4 op_tool tags that fit
inside the 4096 default — its measured completion-token average for
this run was 697, well under the cap. So thinking-budget truncation
is NOT the actual root cause of minimax's lower multi-tool hit rate
(25% vs gpt+deepseek 50%); the real issues are instruction-following
(mixed Strategy A + B despite the explicit forbidance, invented
"canvas" parent_id placeholder).
Still doubling the cap defensively: composite multi-tool outputs can
chain 12-13 op_tool tags + thinking, and "fit easy" today doesn't
mean "fits headroom-free on a longer brief tomorrow." The bump is
free on the happy path (provider stops generating when done, doesn't
bill unused headroom) and only ever helps when the model would
otherwise hit a real ceiling.
Real follow-up for minimax: instruction compliance — the no-mix rule
needs to land harder than a single trailing sentence. Probably wants
the rule moved to top-of-prompt + a few-shot bad-example contrast.
Out of scope here; tracked under Phase 2 prompt design.
ab-v4's dashboard-search-filters-composite garbaged on gpt-5.4 with
"codex timed out after 300000ms" — Codex's own agent framing
(~20k tokens) plus our 18.9k composite system prompt plus thinking
budget plus a 13-tool composite output is enough to blow the old
5-minute cap. The same model produced 13 chained op_tool tags on
team-people-page-composite within the window, so we know it's
generation latency under heavy briefs, not a hung CLI.
Doubles the default to 600000ms and adds AB_CORPUS_CODEX_TIMEOUT_MS
to override either way (lower it to surface slowness as a hard fail
when iterating, raise it for one-off long-form runs). Pairs with
the existing AB_CORPUS_CALL_TIMEOUT_MS for openai-compat clients.
Doesn't fix the deepseek empty-content failure on the same prompt
(already covered by retries=2 + exp backoff) or the minimax mix
+ invented "canvas" parent_id (model-side instruction skip; out of
scope for this commit).
The earlier T instruction split tool selection into PRIMARY / COMPOSITE
(element tools) and FALLBACK (batch_design). FALLBACK was scoped
"when no element tool fits a given component shape" — implying you
could mix per-component. But output-parser.ts silently drops every
batch_design tag whenever any element call is also present (filter
ELEMENT_TOOL_NAME_RE then return). A mixed response
<op_tool>{"name":"add_section_header_v0",...}</op_tool>
<op_tool>{"name":"batch_design", ...scaffolding...}</op_tool>
would lose the batch_design half and only run the element call —
the brief is half-applied without anyone noticing. Codex stop-time
review caught the mismatch.
Reframes T as a binary choice:
- STRATEGY A: every component fits an add_*_v0 tool — emit one tag
per component
- STRATEGY B: at least one component needs batch_design — emit a
SINGLE batch_design covering the whole brief
Plus an explicit "Do not mix Strategy A and Strategy B" guard line.
Tests assert both strategy markers are present in every T variant
and the dropped per-component-fallback phrase is gone.
The elements.md cookbook still teaches batch_design + element tool
composition for real MCP multi-round usage; the corpus T arm is
single-shot so the trailing T_TOOL_CALL_INSTRUCTIONS overrides via
"last instruction wins".
Codex stop-time review caught a contradiction: elements.md taught
"emit one <op_tool> per component" while T_TOOL_CALL_INSTRUCTIONS
explicitly forbade it ("Respond with one tag, nothing else" + "Do
not combine multiple tags"). Live models were probably reading the
later, more authoritative trailing instructions and dropping back to
batch_design — which matches ab-v3's 0/25 composite-T multi-tool
runs even though the cookbook had recipes.
Rewrites the T output-format block to:
- declare "one or more <op_tool> tags" up front
- add an explicit COMPOSITE clause showing chained tags for
multi-component briefs (settings panel / team list / audit feed /
onboarding) with a 3-tag worked example
- keep the FALLBACK clause for batch_design when no element tool fits
- keep the "no prose between tags" rule
Tests now assert the multi-tool marker is present in every T variant
AND the forbidding phrases ("one tag, nothing else", "Do not combine
multiple tags") are GONE — the regression guard Codex effectively
asked for. Also pulls in an oxfmt auto-format on the measurement
helper that landed unformatted in a3c5bf2d.
One-shot diagnostic that prints char + token-estimate sizes for every
(variant, difficulty) combination of buildSystemPrompt. Used to verify
the elements-cookbook diet is shaving the bytes the test floor
predicts (>10kb) before running an actual sweep. Confirms today's
numbers: B 15kb, T-obvious 57kb, T-composite 75kb — diet saves ~17kb
(~4.4k tokens) on every obvious prompt. Phase 2 (per-domain split)
still needed to hit the 6-8k target.
ab-v3 left T prompts at ~22k tokens (vs ~6-7k for B). The 18kb
elements-cookbook teaches per-tool arg shapes, which is what
composite multi-tool chains genuinely need; single-tool obvious
prompts can route correctly from just the decision tree + PREFER
list alone.
buildSystemPrompt now takes opts.difficulty. T+obvious strips the
cookbook (saves ~18kb on the 47/52 ab-v3 obvious prompts);
T+composite, T+optional, and undefined keep both halves. B variant
unchanged — still strips both, so the A/B comparison stays clean.
Verified by 8 new build-prompt.test.ts cases including a 10kb floor
on the obvious-vs-composite delta. ab-v4 will measure whether the
diet hurts arg compliance on weak models; per-domain split is the
Phase 2 fallback if obvious-T garbage rate creeps up.
ab-v3 left 36 ark empty + 15 timeout + 4 429 + 6 deepseek empty + 4
minimax timeout AFTER the existing retries=1 fired 88 times. Linear
backoff 250ms*(attempt+1) was too tight when stepping up to
retries=2 (500ms then 750ms isn't a typical Ark recovery window).
Switches to exponential 250ms*4^attempt — 250 / 1000 / 4000ms
spacing — and bumps ark + deepseek to retries=2. minimax opts in to
retries=1 (its 4 errors were wall-clock timeouts, not the model's
<think> truncation that retry can't fix anyway).
10 existing retry tests still pass; the retries=2 case now sleeps
1.25s instead of 0.75s, still well under vitest's default timeout.
Codex stop-time review flagged the new ab-v3 composite cookbook
recipes calling add_section_header_v0 with a `subtitle` arg the tool
doesn't accept (silently dropped at runtime today, but teaches live
models to emit invalid shapes). The same bug was in the dry-run stub
fixtures.
Split each header into add_section_header_v0(title) +
add_body_text_v0(content) — semantically what the ab-v3 briefs ask
for, and reinforces the multi-tool chaining the cookbook now teaches.
Also fills in the missing required `number` arg on the onboarding
recipe's final completed step card (schema requires it even when
completed=true renders a check instead of the number).
Codex stop-hook caught: ab-v3 introduced composite-difficulty prompts
that *expect* multi-tool emit (e.g. 5× member_row + 1× invite_row
for a team page), but `ParsedOutput.tool_call` was a single
{name, arguments} so the parser silently dropped every call after
the first. apply.ts only invoked one tool, M3 min_roles couldn't
pass on legitimately-routed multi-tool runs, and byTool stats
under-counted. The composite routing 'multi-tool' bucket was
correctly assigned in classifyRouting, but downstream the pipeline
behaved as if the model emitted a single call.
This commit replaces `kind: 'tool_call'` with
`kind: 'tool_calls'` (NON-EMPTY list) across every consumer:
- types.ts: ParsedOutput tagged union; new ParsedOpToolCall.
ScoreRow.toolName → toolNames: string[].
- output-parser.ts: collects ALL element-tool tags in emit order;
unknown-tool path also surfaces as single-element tool_calls so
routing keeps the same wrong-tool semantics.
- score-run.ts: classifyRouting uses Array.includes for obvious
prompts (right-tool when ANY emitted call matches expected_tool —
over-production isn't a routing miss). Composite stays multi-tool
on any non-empty list.
- aggregate.ts byTool: tallies EVERY name in toolNames, so a
composite row that emits 6× add_activity_log_v0 + 1×
add_section_header_v0 contributes 6+1 = 7 invocations across two
tools (with row-level m1_legal applied to both buckets — apply is
all-or-nothing).
- apply.ts: loops over parsed.calls and invokes
handleElementToolCall in emit order. Any single call failing
aborts the row (M1=false); we don't partial-apply.
- mock-llm.ts mockLlmParsed: collects all `<op_tool>` tags into the
list (composite-prompt mocks can carry multi-call raw strings).
- apps/web design-parser.tryParseElementToolOutput: maps tool_calls
→ its single-shape DesignOutputShape contract using the FIRST
call (the multi-tag path `tryParseAllElementToolOutputs` was
already correct).
Tests: 3746 → 3750 vitest. New cases:
- output-parser: surfaces ALL element-tool tags in emit order with
intermixed batch_design scaffolds dropped (3 element calls from
5 tags).
- score-run: right-tool when expected appears alongside extras;
composite multi-call captures every name in toolNames.
- aggregate: 6× activity_log + 1× section_header → byTool reports
6 and 1 invocations respectively.
dry-run on ab-v3 produces a 208-row report; tsc + format clean.
ab-v3 succeeds ab-v1 (frozen 2026-04-28). Carries forward all 40
v1 obvious yaml files unchanged so the v1↔v3 overlap stays
comparable, then layers in two new dimensions.
**1. Token cost.** All clients (openai-compat, ark, bailian,
deepseek, minimax, codex-cli, stub-model) now return a
`ChatCallResult { content, usage }` instead of bare string.
Provider usage stats (`prompt_tokens` / `completion_tokens`) plumb
through realModelCall → run.ts → scoreRun → ScoreRow.{prompt,completion}Tokens.
aggregate adds avgPromptTokens{Baseline,Treatment} +
avgCompletionTokens{Baseline,Treatment} per ModelSummary.
write-report emits a new "Token cost" table with Δ columns so
narrow-tools-saves-tokens (the ab-v2 hypothesis) is measurable.
avgUsage skips rows with 0/0 usage so codex-cli (CLI doesn't
surface tokens) and harness errors don't deflate the average to
near-zero — they show '—' instead.
**2. Composite difficulty.** New 'composite' value alongside
obvious / optional. Composite prompts express multi-tool intents
where no single expected_tool_if_any applies. classifyRouting
routes composite-treatment runs into multi-tool / fallback /
garbage (3-bucket sum to 1, distinct from obvious's 4-bucket
right/wrong/fallback/garbage). aggregate adds m6_multi_tool +
m6_fallback + m6_garbage; write-report emits a "Composite routing"
table that gracefully degrades to a placeholder when no composite
yaml exists yet.
Harness side: scripts/ab-corpus/run.ts accepts --corpus ab-v3
(enum + parseArgs guard); dry-run on the v1-mirror corpus produces
a 160-row report including populated token table.
Tests: 4 new aggregate cases (composite, token avg with skip-zero,
NaN-when-no-data) + 4 new score-run cases (composite routing
multi-tool/fallback/garbage/baseline-n/a) + 2 new score-run cases
(usage plumbing) + 2 new openai-compat cases (usage parsing,
missing-usage fallback). Existing 5 retry tests updated for new
return shape. 3727 → 3740 vitest tests, all green; tsc + format
clean.
Token-cost docs and composite docs go straight into types.ts /
score-run.ts / aggregate.ts JSDoc — keeps the contract close to
the code that owns it.
ab-v2 (2026-04-28) saw kimi-k2.6 garbage rate hit 17.5% — every
failure was Ark returning empty `choices[0].message.content` or
hitting the 120s wall clock, not a model-quality issue (the model
itself routed to the right element tool 80% of the time when it
did respond). Same pattern at 7.5% on deepseek-v4-pro.
Adds optional `retries` to `callOpenAICompat` with a transient-error
allowlist: empty content, abort/timeout, HTTP 5xx, HTTP 429. Linear
250ms × (attempt+1) backoff. HTTP 4xx other than 429 stays fatal so
auth/bad-request failures don't burn retry budget.
Wires `retries: 1` through clients/ark.ts and clients/deepseek.ts.
MiniMax + Bailian + Codex stay untouched — their ab-v2 failures
were model-quality (DSL escape errors, output truncation), where
retry wastes a call without changing the outcome.
Adds an 8-case fixture in scripts/ab-corpus/clients/__tests__/ that
mocks fetch to verify: first-try success, empty-then-success,
5xx-then-success, 429-then-success, 401 fatal, retries-default-zero,
retries exhausted, and retries=2 (3 attempts total). Extends the
apps/web vitest include glob to pick up scripts/**/__tests__/.
DeepSeek wasn't in the harness; A/B v2 needed it for the 5-model run.
api.deepseek.com is OpenAI-compatible, so the client mirrors the
minimax pattern (callOpenAICompat with DEEPSEEK_API_KEY env, override-
able DEEPSEEK_BASE_URL).
Router: /^deepseek/i routes to the new client. Aliases `deepseek` /
`deepseek-pro` resolve to `deepseek-v4-pro` (current flagship per
docs); `deepseek-v4-flash` and the deprecated `deepseek-chat` /
`deepseek-reasoner` ids pass through verbatim until the 2026-07-24
sunset upstream.
Verified live with mobile-bio-textarea smoke (B emits batch_design,
T emits add_textarea_v0). Full ab-v2 results across all 5 models in
docs/notes/2026-04-28-ab-v2-results.md.
Trimming the Minimal usage block out of elements.md (65c31832) lost
arg-shape templates that the A/B harness depends on. Text-only LLMs
in the treatment arm see only the markdown skill content — no MCP
tools/list, no published inputSchema — so without the per-tool
example payloads they have to guess argument names and break M1.
Restore the full block as a sibling skill `elements-cookbook` (same
hasMcpTools flag, slightly later priority so it loads alongside
elements). Wire it through buildFullPrompt + the
get_design_prompt(section='elements-cookbook') section map. Update
the A/B harness to also strip the cookbook body when building the
baseline prompt — leaving it in B would leak tool names + arg shapes
back into the no-tools variant and re-bias the comparison.
Both files now under the 800-line per-file ceiling.
origin's v0.8.0 had cherry-picks of the v0.7.5 deepseek/image-search
fixes (a727632a, a5952bc8) overlapping local 2073cf5b / 04f4fbc1, plus
new commits (model-selector ark-coding deepseek-v4-pro/flash IDs that
ARK rejects, fetch error.cause unwrap, CI agent-native build, op
export docs cleanup, main merge). Resolved the ark-coding list in
favor of HEAD's deepseek-v3.2 entry (only model ARK Coding Plan
actually supports — see openpencil-docs note).
The Zig NAPI provisioner had a silent failure mode that affected any
matrix entry without a matching ZSeven-W/agent prebuilt: the source-
build fallback dropped `agent_napi.node` at `zig-out/napi/...`, but
electron-builder only ships `packages/agent-native/napi/`. The addon
was therefore absent from the produced .exe / .dmg / .AppImage, and
every chat call died at the dynamic `@zseven-w/agent-native` import.
- Drop the prebuilt-download path; always build from source on the
runner (mlugg/setup-zig is already provisioned for every workflow)
- Always copy the built binary into `napi/agent_napi.node` so
electron-builder packages it
- Honor `ZIG_TARGET` to cross-compile (mac-x64 on arm64 runners now
produces an x86_64 binary instead of a wrong-arch arm64 one)
- Add `OPENPENCIL_REQUIRE_AGENT_NATIVE=1` strict mode plus a
dedicated "Verify agent-native binary" step in build-electron.yml
so missing binaries fail the workflow loudly
- Add `OPENPENCIL_SKIP_AGENT_NATIVE=1` for publish-cli.yml, which
never ships the addon and shouldn't pay for the build
`/models` now returns only deepseek-v4-pro and deepseek-v4-flash;
deepseek-chat / deepseek-reasoner sunset 2026-07-24 and the
deepseek-v3.2 hard-coded in the ark-coding fallback list never
existed. Both v4 models default to thinking enabled and the API
toggles via `{"thinking":{"type":"disabled"}}` — keep
`thinkingMode: 'disabled'` so the app's fast/non-thinking default
stays intact (server reasoning paths honor it; the Zig openai-compat
path doesn't emit the toggle yet, so calls through that path still
get provider-default thinking until it's wired). v4-pro promoted to
full tier; legacy aliases pinned to an exact RegExp so future
deepseek-* variants don't inherit a forced disabled mode.
Bandaid for the unwired toggle: v4-pro gets `timeoutMultiplier: 2`
because its default-on reasoning blows past the orchestrator's
planning timeout on long system prompts (observed in dev: planning
phase falls back, sub-agent then succeeds — UX degraded but
functional). Drop the multiplier once the Zig path actually sends
`thinking:{type:disabled}`.
Don't add a BUILTIN_MODEL_LISTS.deepseek entry — DeepSeek exposes
/v1/models, so let `fetchProviderModels` pull the live catalog
through `/api/ai/provider-models` instead of pinning a snapshot
(the ark-coding `deepseek-v3.2` ghost above shows what those
snapshots drift into).
