docs(kiwi): update package split status

This commit is contained in:
Danila Poyarkov 2026-06-06 13:57:46 +03:00
parent 7d27cac5ab
commit 427b445802
2 changed files with 134 additions and 131 deletions

View file

@ -1,85 +1,68 @@
# `@open-pencil/kiwi` extraction plan
`@open-pencil/kiwi` should be the first file-format package split after `@open-pencil/dom-css`. The first migration must keep `.fig` import/export behavior inside `@open-pencil/core` while moving only scene-graph-agnostic Kiwi runtime code behind a standalone package boundary.
`@open-pencil/kiwi` is the standalone package for scene-graph-agnostic Kiwi runtime code. It owns low-level Kiwi schema parsing, Figma Kiwi schema data, binary message encode/decode, FIG Kiwi container framing helpers, and raw `.fig` parse helpers that return structural `NodeChange` data.
## Scope boundary
`.fig` import/export policy still lives in `@open-pencil/core` until a future `@open-pencil/fig` split. Core owns SceneGraph conversion, raw metadata invalidation, component/instance interpretation, and app/CLI-facing document I/O.
### Move first
## Current package boundary
| Current path | Target path | Why it belongs in `@open-pencil/kiwi` | Preconditions |
|---|---|---|---|
| `packages/core/src/kiwi/schema-runtime/**` | `packages/kiwi/src/schema-runtime/**` | Pure Kiwi schema parsing, validation, binary encode/decode, byte-buffer utilities. Imports only local runtime modules. | Add package-local schema runtime tests and dist smoke. |
| `packages/core/src/kiwi/fig/codec/schema/fig.kiwi` | `packages/kiwi/src/fig/schema/fig.kiwi` | Static Figma Kiwi schema text. It is data, not SceneGraph policy. | Keep raw-md bundler support in the new package build. |
| `packages/core/src/kiwi/fig/codec/schema/index.ts` | `packages/kiwi/src/fig/schema.ts` | Parses and validates the static schema. Currently only depends on schema runtime plus raw schema text. | Rewrite imports to package-local runtime paths. |
| `packages/core/src/kiwi/fig/codec/protocol.ts` | `packages/kiwi/src/fig/protocol.ts` | Low-level Figma multiplayer/Kiwi container byte inspection. No SceneGraph dependency. | Add fixtures for message-type and compression detection. |
| `packages/core/src/kiwi/fig/codec/variable-bindings.ts` | `packages/kiwi/src/fig/variable-bindings.ts` | Low-level variable binding binary encode/decode helper. It should not need SceneGraph. | Replace `#core/bytes/hex` / `#core/types` imports with local helpers/types. |
| `packages/core/src/kiwi/fig/codec/index.ts` | `packages/kiwi/src/fig/codec.ts` | Encodes/decodes Figma Kiwi messages. Mostly schema/runtime/protocol logic. | Remove `#core/color` dependency or make color parsing caller-owned; replace core shared types with local exported structural types. |
### In `@open-pencil/kiwi`
### Keep in `@open-pencil/core` for now
| Current path | Reason it stays |
| Path | Scope |
|---|---|
| `packages/core/src/kiwi/fig/import.ts` | Creates `SceneGraph`, imports variables/components/pages, applies OpenPencil source metadata. |
| `packages/core/src/kiwi/fig/lazy-import.ts` | Stores lazy import context in `SceneGraph` weak maps. |
| `packages/core/src/kiwi/fig/node-change/**` | Converts between Figma `NodeChange` records and OpenPencil `SceneNode`, text, vector, paint, font, and layout types. This is `.fig` policy, not raw Kiwi. |
| `packages/core/src/kiwi/fig/instance-overrides/**` | Resolves Figma component/instance override semantics into `SceneGraph`. |
| `packages/core/src/kiwi/fig/parse/**` | `parse/core.ts` can move later to `@open-pencil/fig`; `transfer.ts` and `worker.ts` serialize `SceneGraph`, so they stay until `.fig` package extraction. |
| `packages/core/src/kiwi/fig/container/**` | `.fig`/Kiwi container framing policy. Move later with `@open-pencil/fig`, unless a pure byte-container helper is split out deliberately. |
| `packages/core/src/kiwi/fig/file.ts` | Re-export of core `.fig` I/O. |
| `packages/core/src/io/formats/fig/**` | App/CLI-facing `.fig` read/write policy and renderer-backed export behavior. |
| `packages/kiwi/src/schema-runtime/**` | Pure Kiwi schema parsing, validation, binary encode/decode, and byte-buffer utilities. |
| `packages/kiwi/src/fig/schema/fig.kiwi` | Static Figma Kiwi schema text. |
| `packages/kiwi/src/fig/schema.ts` | Parses and validates the bundled Figma Kiwi schema. |
| `packages/kiwi/src/fig/protocol.ts` | Low-level Figma multiplayer/Kiwi byte inspection and message-type helpers. |
| `packages/kiwi/src/fig/types.ts` | Minimal structural `GUID`, `Color`, `Vector`, and `Matrix` types used by low-level FIG helpers. |
| `packages/kiwi/src/fig/variable-bindings.ts` | Variable binding binary encode/decode helpers. |
| `packages/kiwi/src/fig/codec.ts` | Figma Kiwi message encode/decode and structural `NodeChange` helpers. |
| `packages/kiwi/src/fig/container.ts` | `fig-kiwi` byte container framing and compression helpers. |
| `packages/kiwi/src/fig/parse.ts` | `.fig` zip/canvas parsing into structural node changes, blobs, images, schema bytes, and container version. |
## Dependency blockers to remove before moving
- `fig/codec/index.ts` imports `parseColor` from `#core/color` for convenience factories. Options:
- remove string color parsing from the low-level package, or
- accept a caller-supplied `parseColor` option, or
- move only structural encode/decode first and leave convenience constructors in core.
- `fig/codec/index.ts` and `variable-bindings.ts` import `Color`, `GUID`, `Matrix`, and `Vector` from `#core/types`. The Kiwi package should define minimal structural types locally and let core adapt them.
- `variable-bindings.ts` imports `hexToBytes` from `#core/bytes/hex`. Copy a tiny local helper or expose it from the Kiwi package; do not depend on core.
- Raw `.kiwi` loading currently relies on the root raw markdown/schema tooling. The new package build must prove raw schema import works from `packages/kiwi` in build and smoke tests.
## Initial package shape
```text
packages/kiwi/
package.json
tsconfig.json
tsconfig.test.json
tsdown.config.ts
scripts/smoke-dist.ts
src/
index.ts
schema-runtime/
fig/
codec.ts
protocol.ts
schema.ts
schema/fig.kiwi
variable-bindings.ts
types.ts
tests/
schema-runtime.test.ts
fig-schema.test.ts
fig-codec.test.ts
protocol.test.ts
variable-bindings.test.ts
```
Public exports should stay narrow at first:
Public exports:
```json
{
".": "./dist/index.js",
"./schema-runtime": "./dist/schema-runtime/index.js",
"./fig": "./dist/fig/index.js"
"./schema-runtime": "./dist/schema-runtime.js",
"./fig": "./dist/fig.js",
"./fig/codec": "./dist/fig/codec.js",
"./fig/container": "./dist/fig/container.js",
"./fig/parse": "./dist/fig/parse.js"
}
```
`@open-pencil/core/kiwi` should keep compatibility re-exports during the first migration so app, CLI, MCP, and external consumers do not need to change immediately.
`@open-pencil/kiwi` must not import `@open-pencil/core`, `#core/*`, app code, Vue code, CLI code, or MCP code.
### Kept in `@open-pencil/core` for now
| Path | Reason it stays |
|---|---|
| `packages/core/src/kiwi/fig/import.ts` | Creates `SceneGraph`, imports variables/components/pages, and applies OpenPencil source metadata. |
| `packages/core/src/kiwi/fig/lazy-import.ts` | Stores lazy import context in `SceneGraph` weak maps. |
| `packages/core/src/kiwi/fig/node-change/**` | Converts between Figma `NodeChange` records and OpenPencil `SceneNode`, text, vector, paint, font, and layout types. |
| `packages/core/src/kiwi/fig/instance-overrides/**` | Resolves Figma component/instance override semantics into `SceneGraph`. |
| `packages/core/src/kiwi/fig/parse/transfer.ts` | Serializes `SceneGraph` data for worker transfer. |
| `packages/core/src/kiwi/fig/parse/worker.ts` | Worker glue: parses through `@open-pencil/kiwi/fig/parse`, then imports into `SceneGraph`. |
| `packages/core/src/kiwi/fig/file.ts` | Core `.fig` I/O re-export. |
| `packages/core/src/io/formats/fig/**` | App/CLI-facing `.fig` read/write policy and renderer-backed export behavior. |
`@open-pencil/core/kiwi` remains the compatibility barrel for existing public consumers, but deep core shims for moved Kiwi internals have been removed. Internal core code imports low-level helpers directly from `@open-pencil/kiwi`.
## Completed extraction sequence
1. Added `packages/kiwi` with schema runtime, Figma schema data, package-local tests, build, and dist smoke.
2. Moved low-level protocol/schema helpers into `@open-pencil/kiwi`.
3. Moved variable binding binary helpers into `@open-pencil/kiwi`.
4. Removed core color/type dependencies from the low-level FIG codec by making color normalization caller-owned and defining structural FIG types in Kiwi.
5. Moved Figma Kiwi codec, parse helpers, and container helpers into `@open-pencil/kiwi`.
6. Rewired core internals and tests to import moved helpers from `@open-pencil/kiwi` directly.
7. Removed redundant deep core re-export shims for moved codec/parse/container modules.
## Package-local test plan
Add these tests in `packages/kiwi/tests/**` before moving production callers:
`packages/kiwi/tests/**` should continue to cover:
1. **Schema runtime smoke**
- Parse a small inline Kiwi schema.
@ -88,57 +71,62 @@ Add these tests in `packages/kiwi/tests/**` before moving production callers:
2. **Bundled Figma schema guard**
- Validate the bundled `fig.kiwi` schema.
- Assert stable high-value facts already covered in `tests/engine/kiwi/schema-runtime.test.ts`:
- definition count is `605`
- `NodeChange` exists
- `InteractiveSlideElementChange` exists
- `Paint.colorVar` field number is `21`
- `NodeChange.pageType` field number is `397`
- `VariableField.OVERRIDDEN_SYMBOL_ID` enum value is `37`
- Assert stable high-value Figma schema facts.
3. **Protocol helpers**
- `isZstdCompressed()` returns true for Zstd magic bytes and false for Kiwi payload bytes.
- `getKiwiMessageType()` reads a message type from a minimal encoded payload.
- Malformed payloads fail predictably without throwing unexpected internal errors.
- Zstd detection.
- Kiwi message type reading.
- Varint parsing.
- `fig-wire` header handling.
4. **Variable binding binary helpers**
- Varint encode/decode round-trips representative values: `0`, `1`, `127`, `128`, `255`, `16384`.
- GUID hex/byte conversion round-trips a known Figma GUID.
- Color variable binding encode output stays byte-stable for a known fixture.
- Varint encode/decode coverage.
- Figma variable ID parsing.
- Paint/node-change variable binding byte injection.
5. **Figma message codec**
- `initCodec()` is idempotent.
- `getSchemaBytes()` returns non-empty stable schema bytes.
- A minimal `FigmaMessage` encode/decode round-trips.
- A `NodeChange` with variable bindings uses the custom node-change path and decodes back to equivalent data.
- `initCodec()` idempotency.
- Non-empty schema bytes.
- Minimal message encode/decode.
- Variable-bound paint message encoding.
6. **Dist smoke**
- Import `@open-pencil/kiwi` built output from `dist`.
- Validate schema, initialize codec, encode/decode one minimal message.
- Assert no import of `@open-pencil/core` is required by the package dist.
6. **Container and parse helpers**
- `buildFigKiwi()` / `parseFigKiwiChunks()` round-trip.
- Sync/async decompression.
- Invalid container rejection.
- Plugin data deduplication.
Repo-level tests under `tests/engine/io/fig/**`, `tests/engine/kiwi/serialize-fixes/**`, and Figma oracle fixtures should remain in place. Package-local tests prove package isolation; repo-level tests prove OpenPencil `.fig` behavior did not regress.
7. **Dist smoke**
- Import built package outputs from `dist`.
- Validate schema/runtime/codec/container/parse exports.
- Assert no `@open-pencil/core` dependency is required by package dist.
## Migration sequence
Repo-level tests under `tests/engine/io/fig/**`, `tests/engine/kiwi/**`, and Figma oracle fixtures remain in place. Package-local tests prove package isolation; repo-level tests prove OpenPencil `.fig` behavior did not regress.
1. Add `packages/kiwi` with copied runtime/schema files and package-local tests, but do not update production imports yet.
2. Make `@open-pencil/core/kiwi` re-export from `@open-pencil/kiwi` for schema runtime and codec-only APIs.
3. Update internal core imports that only need schema runtime or codec APIs to public `@open-pencil/kiwi` exports.
4. Keep `.fig` import/export, node-change conversion, and instance override policy in core until `@open-pencil/fig` exists.
5. Run:
## Next extraction candidates
Move only helpers that are proven scene-graph-agnostic and useful to a future `@open-pencil/fig` package. Good candidates may include small byte/GUID helpers that operate only on structural Kiwi data.
Do not move yet:
- `NodeChange``SceneNode` conversion.
- Instance override interpretation.
- Raw metadata preservation/invalidation policy.
- Renderer/editor fallback behavior.
- `.fig` import/export APIs that need core document policy.
## Validation commands
```sh
cd packages/kiwi && bun run check
bun test tests/engine/kiwi/schema-runtime.test.ts tests/engine/io/fig/roundtrip/basic.test.ts
cd ../..
bun run check
```
6. Only after the package is stable, start the `@open-pencil/fig` split for `.fig` container/read/write and SceneGraph conversion policy.
For targeted behavior checks, prefer focused repo tests such as:
## Do not do yet
```sh
bun test tests/engine/kiwi/schema-runtime.test.ts tests/engine/io/fig/roundtrip/basic.test.ts
```
- Do not move `NodeChange``SceneNode` conversion into `@open-pencil/kiwi`.
- Do not make `@open-pencil/kiwi` depend on `@open-pencil/core`.
- Do not widen app/CLI imports to private package source paths.
- Do not guess or rewrite Figma schema fields during the split.
- Do not remove `@open-pencil/core/kiwi` compatibility exports in the first migration.
The broader `material3.fig` fixture currently still exposes the known `BooleanOperation: EXCLUDE` codec enum issue and should not be treated as a regression from this package split.

View file

@ -1,6 +1,6 @@
# Package split plan
OpenPencil currently keeps file-format internals in `@open-pencil/core` while `@open-pencil/dom-css` is being split out as a standalone DOM/CSS compatibility layer. Future package splits should preserve the current app behavior first, then move stable surfaces into independently publishable packages.
OpenPencil is splitting stable compatibility layers into independently publishable packages while keeping editor/runtime behavior intact. Each package boundary should be narrow, covered by package-local checks, and consumed through public workspace exports.
## Goals
@ -11,48 +11,54 @@ OpenPencil currently keeps file-format internals in `@open-pencil/core` while `@
## Current package boundary
- `@open-pencil/core` owns scene graph, renderer, editor actions, Figma API compatibility, Kiwi codec internals, and format import/export.
- `@open-pencil/core` owns scene graph, renderer, editor actions, Figma API compatibility, `.fig` SceneGraph policy, and format import/export.
- `@open-pencil/dom-css` owns DesignDOM, CSS runtimes, HTML/JSX/Tailwind projection, and SceneGraph ⇄ DesignDOM conversion.
- `@open-pencil/kiwi` owns pure Kiwi schema/runtime code plus low-level Figma Kiwi codec, container, and parse helpers.
- App, CLI, MCP, and Vue SDK consume packages through public workspace exports only.
See [`kiwi-package-plan.md`](./kiwi-package-plan.md) for the detailed `@open-pencil/kiwi` inventory, dependency blockers, and package-local test plan.
See [`kiwi-package-plan.md`](./kiwi-package-plan.md) for the detailed `@open-pencil/kiwi` inventory, package-local test plan, and remaining split boundaries.
## Candidate packages
### `@open-pencil/kiwi`
Status: extracted.
Scope:
- Kiwi binary schema runtime.
- Schema-generated codec modules.
- Generic binary parse/serialize helpers.
- Low-level validation helpers that do not know about OpenPencil scene graph nodes.
- Figma Kiwi schema data and validation.
- Low-level Figma Kiwi protocol/message helpers.
- Structural `NodeChange` encode/decode.
- `fig-kiwi` container framing and raw parse helpers that do not create `SceneGraph` objects.
Should not include:
Does not include:
- Figma `.fig` container policy.
- SceneGraph conversion.
- Raw metadata invalidation policy.
- Component/instance override interpretation.
- Renderer/editor code.
Minimum exit criteria:
Minimum maintenance criteria:
- Package-local typecheck, unit tests, build, and dist smoke.
- Existing Kiwi serialize/parse tests passing through the package public API.
- No import cycles from `@open-pencil/core` back into the package.
- No `@open-pencil/core`, `#core/*`, app, CLI, MCP, or Vue imports.
- Existing repo-level Kiwi/FIG tests continue to pass through public package APIs.
### `@open-pencil/fig`
Scope:
- `.fig` container read/write.
- `.fig` document read/write policy.
- Figma node-change import/export.
- Raw metadata preservation and invalidation policy.
- Figma component/instance interpretation.
- Figma oracle fixtures and compatibility helpers.
Should depend on:
- `@open-pencil/core` scene graph types.
- `@open-pencil/kiwi` once the Kiwi split exists.
- `@open-pencil/core` scene graph types and conversion policy while those remain core-owned.
- `@open-pencil/kiwi` for low-level schema/runtime/container/codec helpers.
Should not include:
@ -66,45 +72,53 @@ Minimum exit criteria:
- Heavy Figma fixture coverage still available at repo level.
- Public API supports CLI/MCP/app document I/O without private path imports.
## Initial inventory
## Current inventory
Likely `@open-pencil/kiwi` candidates:
In `@open-pencil/kiwi`:
- `packages/core/src/kiwi/schema-runtime/**`
- `packages/core/src/kiwi/fig/codec/**`
- Generated Kiwi schema modules under the Figma codec directory, as long as they stay scene-graph agnostic
- `packages/kiwi/src/schema-runtime/**`
- `packages/kiwi/src/fig/schema/**`
- `packages/kiwi/src/fig/schema.ts`
- `packages/kiwi/src/fig/protocol.ts`
- `packages/kiwi/src/fig/types.ts`
- `packages/kiwi/src/fig/variable-bindings.ts`
- `packages/kiwi/src/fig/codec.ts`
- `packages/kiwi/src/fig/container.ts`
- `packages/kiwi/src/fig/parse.ts`
Likely `@open-pencil/fig` candidates:
- `packages/core/src/kiwi/fig/file.ts`
- `packages/core/src/kiwi/fig/container/**`
- `packages/core/src/kiwi/fig/parse/**`
- `packages/core/src/kiwi/fig/parse/transfer.ts`
- `packages/core/src/kiwi/fig/parse/worker.ts`
- `packages/core/src/kiwi/fig/import.ts`
- `packages/core/src/kiwi/fig/lazy-import.ts`
- `packages/core/src/kiwi/fig/node-change/**`
- `packages/core/src/kiwi/fig/instance-overrides/**`
- `packages/core/src/io/formats/fig/**`
Keep in `@open-pencil/core` unless proven otherwise:
- `SceneGraph` and node type definitions
- Renderer/editor fallback behavior
- Layout, text measurement, and canvas-specific code
- Generic IO registry contracts that other formats use
- `SceneGraph` and node type definitions.
- Renderer/editor fallback behavior.
- Layout, text measurement, and canvas-specific code.
- Generic IO registry contracts that other formats use.
## Migration checklist
1. Add package-local tests before moving files.
2. Confirm every moved module imports only allowed public package exports.
3. Preserve existing `@open-pencil/core/kiwi` re-exports during the first migration step.
4. Move one boundary at a time: schema runtime first, generated codec second, `.fig` policy last.
5. Keep fixture/oracle tests in the repo-level suite even after package-local tests exist.
6. Run package smoke checks from a temporary consumer project before publishing.
3. Prefer direct imports from the new package for internal code; avoid accumulating deep re-export shims.
4. Preserve only intentional public compatibility barrels, such as `@open-pencil/core/kiwi`, when external consumers need a deprecation window.
5. Move one boundary at a time: schema/runtime first, low-level codec/container/parse second, `.fig` policy last.
6. Keep fixture/oracle tests in the repo-level suite even after package-local tests exist.
7. Run package smoke checks from a temporary consumer project before publishing.
## Migration order
1. Keep `@open-pencil/dom-css` standalone and stabilize its browser/headless runtime split.
2. Extract pure Kiwi runtime/codecs behind `@open-pencil/kiwi` without moving `.fig` policy.
3. Move `.fig` container and node-change conversion into `@open-pencil/fig`.
2. Extract pure Kiwi runtime/codecs behind `@open-pencil/kiwi` without moving `.fig` SceneGraph policy.
3. Move `.fig` document policy and node-change conversion into `@open-pencil/fig` when the boundary is clear.
4. Update core/app/CLI/MCP imports to consume public package exports.
5. Keep compatibility re-exports in `@open-pencil/core` only if existing consumers need a deprecation window.
@ -113,3 +127,4 @@ Keep in `@open-pencil/core` unless proven otherwise:
- Do not guess Figma schema fields during the split.
- Do not move renderer-specific fallback behavior into file-format packages.
- Do not add browser DOM dependencies to core or file-format packages.
- Do not widen app/CLI imports to private package source paths.