The Ark router regex was `/^kimi-k2\.6/i` — required the `k`
prefix. But mapKimiArkId's alias list accepted both `kimi-k2.6`
AND `kimi-2.6` (no-prefix form). Result: `kimi-2.6` failed the
Ark regex, fell through to the generic `/^kimi/i` branch, got
routed to Bailian — which doesn't host K2.6. Bailian would
return HTTP 400 "model not supported" with no hint that the id
belonged on Ark.
Fix: Ark router regex now `/^kimi-k?2\.6(-ark)?$/i` — optional
`k` prefix + optional `-ark` suffix, anchored at end to prevent
accidentally matching a hypothetical later version. mapKimiArkId
normalizes all four accepted aliases (kimi-k2.6, kimi-2.6,
kimi-k2.6-ark, kimi-2.6-ark) to the canonical on-Ark id.
Same latent bug fixed on the glm-5.1 route: regex `/^glm-5\.1/i`
would prefix-match a hypothetical `glm-5.10` and wrongly route it
to Ark. Tightened to `/^glm-5\.1(-coding|-ark)?$/i` with the same
anchored-end + suffix-allowlist pattern.
Caught by Codex stop-hook review during 2026-04-22 session.
Volcengine 方舟 (Ark) added GLM-5.1 and Kimi-K2.6 to its coding
plan on 2026-04-22 — single ARK_CODING_KEY covers both. Harness
now prefers this route over the previous paths:
- glm-5.1 was routed to clients/glm.ts (GLM official CP via
open.bigmodel.cn with GLM_OFFICIAL_CODING_KEY). Now routed to
new clients/ark.ts. The old glm.ts file is kept on disk for
historical comparison but not wired into the default router —
callers who want to A/B the old GLM-official path vs. new Ark
path can import callGlm directly.
- kimi-k2.6 is new — added as a dedicated router branch above
the kimi-k2.5 (bailian) branch so the version-specific match
lands on Ark.
Old kimi-k2.5 continues to route through clients/bailian.ts
(DashScope aggregator) for continuity with earlier A/B runs.
Key management (unchanged from the harness convention):
- ARK_CODING_KEY — Volcengine 方舟 CP UUID format key. Export
in shell before running --live; never committed.
- Existing MINIMAX_API_KEY / GLM_OFFICIAL_CODING_KEY /
DASHSCOPE_BAILIAN_CODING_KEY all still honored for their
respective routes.
Throw message updated so missing-key errors surface the correct
env var for each route.
User hit at dev-server startup:
Module "node:fs" has been externalized for browser compatibility.
Cannot access "node:fs.readdirSync" in client code.
Chain: apps/web's design-parser.ts imports `parseModelOutput` from
`@zseven-w/pen-ai-skills`; main barrel re-exports everything from
`./corpus`; `./corpus/index.ts` re-exports `loadCorpus` which imports
`node:fs`. Vite pulls the whole graph into the client bundle → crash
on the first browser-side module evaluation.
Fix: remove `loadCorpus` from `./corpus/index.ts`. The barrel now
only exposes pure-string helpers (parser, scorer, aggregator, types)
— all browser-safe. `loadCorpus` stays in `corpus-loader.ts` but
Node-only consumers (`scripts/ab-corpus/run.ts`) import it directly
via a relative path. Package.json only declares the main entry in
`exports`, so sub-path imports via the package name fail at runtime
(pkg runs under Bun for the harness) — relative file path avoids
that gate.
Verification:
- `bun scripts/ab-corpus/run.ts --only X --dry-run` still runs end
to end
- tsc --noEmit exit 0
- Full test suite 1866/1866
Browser-side verification (user): restart Vite dev server — the
design-parser import no longer pulls node:fs through the barrel.
Harness at scripts/ab-corpus/ wires the pen-ai-skills corpus evaluator to
real model endpoints and pen-mcp handlers:
run.ts — CLI entry (--dry-run / --live / --models A,B,C / --only ID)
apply.ts — ApplyFn impl dispatching tool_call → element handler
and batch_design DSL → handleBatchDesign, against a
fresh tmp .op per run (isolated, auto-cleanup)
build-prompt.ts — B variant strips elements.md + appends batch_design
<op_tool> format instruction; T keeps elements + adds
element-tool PRIMARY / batch_design FALLBACK
instruction. Uniform <op_tool> wrapper in both arms
isolates "tool set width" as the only A/B variable.
stub-model.ts — fixture-based offline model for --dry-run
real-model.ts — router by model id (minimax* / gpt-*/o* / glm-5.1 /
glm-* / kimi-*)
clients/
openai-compat.ts — generic chat/completions POST
minimax.ts — api.minimax.io/v1, MINIMAX_API_KEY
codex-cli.ts — spawns `codex exec` (GPT-5.4 via Codex Pro sub)
bailian.ts — coding.dashscope.aliyuncs.com/v1 CP,
DASHSCOPE_BAILIAN_CODING_KEY (hosts glm-4.7, kimi-k2.5)
glm.ts — open.bigmodel.cn/api/coding/paas/v4 official CP,
GLM_OFFICIAL_CODING_KEY
write-report.ts — Report → report.md + report.json in out dir;
4-way routing breakdown table per model
Kept entirely outside packages/ — scripts are a local dev tool, not part
of the published SDK. API keys never hit disk or git.
v1 run results logged separately in openpencil-docs
superpowers/notes/2026-04-20-ab-v1-results.md (5 models × 24 prompts).
