diff --git a/packages/docs/development/kiwi-package-plan.md b/packages/docs/development/kiwi-package-plan.md index f29c9755d..8f7ea0fa4 100644 --- a/packages/docs/development/kiwi-package-plan.md +++ b/packages/docs/development/kiwi-package-plan.md @@ -1,85 +1,68 @@ # `@open-pencil/kiwi` extraction plan -`@open-pencil/kiwi` should be the first file-format package split after `@open-pencil/dom-css`. The first migration must keep `.fig` import/export behavior inside `@open-pencil/core` while moving only scene-graph-agnostic Kiwi runtime code behind a standalone package boundary. +`@open-pencil/kiwi` is the standalone package for scene-graph-agnostic Kiwi runtime code. It owns low-level Kiwi schema parsing, Figma Kiwi schema data, binary message encode/decode, FIG Kiwi container framing helpers, and raw `.fig` parse helpers that return structural `NodeChange` data. -## Scope boundary +`.fig` import/export policy still lives in `@open-pencil/core` until a future `@open-pencil/fig` split. Core owns SceneGraph conversion, raw metadata invalidation, component/instance interpretation, and app/CLI-facing document I/O. -### Move first +## Current package boundary -| Current path | Target path | Why it belongs in `@open-pencil/kiwi` | Preconditions | -|---|---|---|---| -| `packages/core/src/kiwi/schema-runtime/**` | `packages/kiwi/src/schema-runtime/**` | Pure Kiwi schema parsing, validation, binary encode/decode, byte-buffer utilities. Imports only local runtime modules. | Add package-local schema runtime tests and dist smoke. | -| `packages/core/src/kiwi/fig/codec/schema/fig.kiwi` | `packages/kiwi/src/fig/schema/fig.kiwi` | Static Figma Kiwi schema text. It is data, not SceneGraph policy. | Keep raw-md bundler support in the new package build. | -| `packages/core/src/kiwi/fig/codec/schema/index.ts` | `packages/kiwi/src/fig/schema.ts` | Parses and validates the static schema. Currently only depends on schema runtime plus raw schema text. | Rewrite imports to package-local runtime paths. | -| `packages/core/src/kiwi/fig/codec/protocol.ts` | `packages/kiwi/src/fig/protocol.ts` | Low-level Figma multiplayer/Kiwi container byte inspection. No SceneGraph dependency. | Add fixtures for message-type and compression detection. | -| `packages/core/src/kiwi/fig/codec/variable-bindings.ts` | `packages/kiwi/src/fig/variable-bindings.ts` | Low-level variable binding binary encode/decode helper. It should not need SceneGraph. | Replace `#core/bytes/hex` / `#core/types` imports with local helpers/types. | -| `packages/core/src/kiwi/fig/codec/index.ts` | `packages/kiwi/src/fig/codec.ts` | Encodes/decodes Figma Kiwi messages. Mostly schema/runtime/protocol logic. | Remove `#core/color` dependency or make color parsing caller-owned; replace core shared types with local exported structural types. | +### In `@open-pencil/kiwi` -### Keep in `@open-pencil/core` for now - -| Current path | Reason it stays | +| Path | Scope | |---|---| -| `packages/core/src/kiwi/fig/import.ts` | Creates `SceneGraph`, imports variables/components/pages, applies OpenPencil source metadata. | -| `packages/core/src/kiwi/fig/lazy-import.ts` | Stores lazy import context in `SceneGraph` weak maps. | -| `packages/core/src/kiwi/fig/node-change/**` | Converts between Figma `NodeChange` records and OpenPencil `SceneNode`, text, vector, paint, font, and layout types. This is `.fig` policy, not raw Kiwi. | -| `packages/core/src/kiwi/fig/instance-overrides/**` | Resolves Figma component/instance override semantics into `SceneGraph`. | -| `packages/core/src/kiwi/fig/parse/**` | `parse/core.ts` can move later to `@open-pencil/fig`; `transfer.ts` and `worker.ts` serialize `SceneGraph`, so they stay until `.fig` package extraction. | -| `packages/core/src/kiwi/fig/container/**` | `.fig`/Kiwi container framing policy. Move later with `@open-pencil/fig`, unless a pure byte-container helper is split out deliberately. | -| `packages/core/src/kiwi/fig/file.ts` | Re-export of core `.fig` I/O. | -| `packages/core/src/io/formats/fig/**` | App/CLI-facing `.fig` read/write policy and renderer-backed export behavior. | +| `packages/kiwi/src/schema-runtime/**` | Pure Kiwi schema parsing, validation, binary encode/decode, and byte-buffer utilities. | +| `packages/kiwi/src/fig/schema/fig.kiwi` | Static Figma Kiwi schema text. | +| `packages/kiwi/src/fig/schema.ts` | Parses and validates the bundled Figma Kiwi schema. | +| `packages/kiwi/src/fig/protocol.ts` | Low-level Figma multiplayer/Kiwi byte inspection and message-type helpers. | +| `packages/kiwi/src/fig/types.ts` | Minimal structural `GUID`, `Color`, `Vector`, and `Matrix` types used by low-level FIG helpers. | +| `packages/kiwi/src/fig/variable-bindings.ts` | Variable binding binary encode/decode helpers. | +| `packages/kiwi/src/fig/codec.ts` | Figma Kiwi message encode/decode and structural `NodeChange` helpers. | +| `packages/kiwi/src/fig/container.ts` | `fig-kiwi` byte container framing and compression helpers. | +| `packages/kiwi/src/fig/parse.ts` | `.fig` zip/canvas parsing into structural node changes, blobs, images, schema bytes, and container version. | -## Dependency blockers to remove before moving - -- `fig/codec/index.ts` imports `parseColor` from `#core/color` for convenience factories. Options: - - remove string color parsing from the low-level package, or - - accept a caller-supplied `parseColor` option, or - - move only structural encode/decode first and leave convenience constructors in core. -- `fig/codec/index.ts` and `variable-bindings.ts` import `Color`, `GUID`, `Matrix`, and `Vector` from `#core/types`. The Kiwi package should define minimal structural types locally and let core adapt them. -- `variable-bindings.ts` imports `hexToBytes` from `#core/bytes/hex`. Copy a tiny local helper or expose it from the Kiwi package; do not depend on core. -- Raw `.kiwi` loading currently relies on the root raw markdown/schema tooling. The new package build must prove raw schema import works from `packages/kiwi` in build and smoke tests. - -## Initial package shape - -```text -packages/kiwi/ - package.json - tsconfig.json - tsconfig.test.json - tsdown.config.ts - scripts/smoke-dist.ts - src/ - index.ts - schema-runtime/ - fig/ - codec.ts - protocol.ts - schema.ts - schema/fig.kiwi - variable-bindings.ts - types.ts - tests/ - schema-runtime.test.ts - fig-schema.test.ts - fig-codec.test.ts - protocol.test.ts - variable-bindings.test.ts -``` - -Public exports should stay narrow at first: +Public exports: ```json { ".": "./dist/index.js", - "./schema-runtime": "./dist/schema-runtime/index.js", - "./fig": "./dist/fig/index.js" + "./schema-runtime": "./dist/schema-runtime.js", + "./fig": "./dist/fig.js", + "./fig/codec": "./dist/fig/codec.js", + "./fig/container": "./dist/fig/container.js", + "./fig/parse": "./dist/fig/parse.js" } ``` -`@open-pencil/core/kiwi` should keep compatibility re-exports during the first migration so app, CLI, MCP, and external consumers do not need to change immediately. +`@open-pencil/kiwi` must not import `@open-pencil/core`, `#core/*`, app code, Vue code, CLI code, or MCP code. + +### Kept in `@open-pencil/core` for now + +| Path | Reason it stays | +|---|---| +| `packages/core/src/kiwi/fig/import.ts` | Creates `SceneGraph`, imports variables/components/pages, and applies OpenPencil source metadata. | +| `packages/core/src/kiwi/fig/lazy-import.ts` | Stores lazy import context in `SceneGraph` weak maps. | +| `packages/core/src/kiwi/fig/node-change/**` | Converts between Figma `NodeChange` records and OpenPencil `SceneNode`, text, vector, paint, font, and layout types. | +| `packages/core/src/kiwi/fig/instance-overrides/**` | Resolves Figma component/instance override semantics into `SceneGraph`. | +| `packages/core/src/kiwi/fig/parse/transfer.ts` | Serializes `SceneGraph` data for worker transfer. | +| `packages/core/src/kiwi/fig/parse/worker.ts` | Worker glue: parses through `@open-pencil/kiwi/fig/parse`, then imports into `SceneGraph`. | +| `packages/core/src/kiwi/fig/file.ts` | Core `.fig` I/O re-export. | +| `packages/core/src/io/formats/fig/**` | App/CLI-facing `.fig` read/write policy and renderer-backed export behavior. | + +`@open-pencil/core/kiwi` remains the compatibility barrel for existing public consumers, but deep core shims for moved Kiwi internals have been removed. Internal core code imports low-level helpers directly from `@open-pencil/kiwi`. + +## Completed extraction sequence + +1. Added `packages/kiwi` with schema runtime, Figma schema data, package-local tests, build, and dist smoke. +2. Moved low-level protocol/schema helpers into `@open-pencil/kiwi`. +3. Moved variable binding binary helpers into `@open-pencil/kiwi`. +4. Removed core color/type dependencies from the low-level FIG codec by making color normalization caller-owned and defining structural FIG types in Kiwi. +5. Moved Figma Kiwi codec, parse helpers, and container helpers into `@open-pencil/kiwi`. +6. Rewired core internals and tests to import moved helpers from `@open-pencil/kiwi` directly. +7. Removed redundant deep core re-export shims for moved codec/parse/container modules. ## Package-local test plan -Add these tests in `packages/kiwi/tests/**` before moving production callers: +`packages/kiwi/tests/**` should continue to cover: 1. **Schema runtime smoke** - Parse a small inline Kiwi schema. @@ -88,57 +71,62 @@ Add these tests in `packages/kiwi/tests/**` before moving production callers: 2. **Bundled Figma schema guard** - Validate the bundled `fig.kiwi` schema. - - Assert stable high-value facts already covered in `tests/engine/kiwi/schema-runtime.test.ts`: - - definition count is `605` - - `NodeChange` exists - - `InteractiveSlideElementChange` exists - - `Paint.colorVar` field number is `21` - - `NodeChange.pageType` field number is `397` - - `VariableField.OVERRIDDEN_SYMBOL_ID` enum value is `37` + - Assert stable high-value Figma schema facts. 3. **Protocol helpers** - - `isZstdCompressed()` returns true for Zstd magic bytes and false for Kiwi payload bytes. - - `getKiwiMessageType()` reads a message type from a minimal encoded payload. - - Malformed payloads fail predictably without throwing unexpected internal errors. + - Zstd detection. + - Kiwi message type reading. + - Varint parsing. + - `fig-wire` header handling. 4. **Variable binding binary helpers** - - Varint encode/decode round-trips representative values: `0`, `1`, `127`, `128`, `255`, `16384`. - - GUID hex/byte conversion round-trips a known Figma GUID. - - Color variable binding encode output stays byte-stable for a known fixture. + - Varint encode/decode coverage. + - Figma variable ID parsing. + - Paint/node-change variable binding byte injection. 5. **Figma message codec** - - `initCodec()` is idempotent. - - `getSchemaBytes()` returns non-empty stable schema bytes. - - A minimal `FigmaMessage` encode/decode round-trips. - - A `NodeChange` with variable bindings uses the custom node-change path and decodes back to equivalent data. + - `initCodec()` idempotency. + - Non-empty schema bytes. + - Minimal message encode/decode. + - Variable-bound paint message encoding. -6. **Dist smoke** - - Import `@open-pencil/kiwi` built output from `dist`. - - Validate schema, initialize codec, encode/decode one minimal message. - - Assert no import of `@open-pencil/core` is required by the package dist. +6. **Container and parse helpers** + - `buildFigKiwi()` / `parseFigKiwiChunks()` round-trip. + - Sync/async decompression. + - Invalid container rejection. + - Plugin data deduplication. -Repo-level tests under `tests/engine/io/fig/**`, `tests/engine/kiwi/serialize-fixes/**`, and Figma oracle fixtures should remain in place. Package-local tests prove package isolation; repo-level tests prove OpenPencil `.fig` behavior did not regress. +7. **Dist smoke** + - Import built package outputs from `dist`. + - Validate schema/runtime/codec/container/parse exports. + - Assert no `@open-pencil/core` dependency is required by package dist. -## Migration sequence +Repo-level tests under `tests/engine/io/fig/**`, `tests/engine/kiwi/**`, and Figma oracle fixtures remain in place. Package-local tests prove package isolation; repo-level tests prove OpenPencil `.fig` behavior did not regress. -1. Add `packages/kiwi` with copied runtime/schema files and package-local tests, but do not update production imports yet. -2. Make `@open-pencil/core/kiwi` re-export from `@open-pencil/kiwi` for schema runtime and codec-only APIs. -3. Update internal core imports that only need schema runtime or codec APIs to public `@open-pencil/kiwi` exports. -4. Keep `.fig` import/export, node-change conversion, and instance override policy in core until `@open-pencil/fig` exists. -5. Run: +## Next extraction candidates + +Move only helpers that are proven scene-graph-agnostic and useful to a future `@open-pencil/fig` package. Good candidates may include small byte/GUID helpers that operate only on structural Kiwi data. + +Do not move yet: + +- `NodeChange` ⇄ `SceneNode` conversion. +- Instance override interpretation. +- Raw metadata preservation/invalidation policy. +- Renderer/editor fallback behavior. +- `.fig` import/export APIs that need core document policy. + +## Validation commands ```sh cd packages/kiwi && bun run check -bun test tests/engine/kiwi/schema-runtime.test.ts tests/engine/io/fig/roundtrip/basic.test.ts +cd ../.. bun run check ``` -6. Only after the package is stable, start the `@open-pencil/fig` split for `.fig` container/read/write and SceneGraph conversion policy. +For targeted behavior checks, prefer focused repo tests such as: -## Do not do yet +```sh +bun test tests/engine/kiwi/schema-runtime.test.ts tests/engine/io/fig/roundtrip/basic.test.ts +``` -- Do not move `NodeChange` ⇄ `SceneNode` conversion into `@open-pencil/kiwi`. -- Do not make `@open-pencil/kiwi` depend on `@open-pencil/core`. -- Do not widen app/CLI imports to private package source paths. -- Do not guess or rewrite Figma schema fields during the split. -- Do not remove `@open-pencil/core/kiwi` compatibility exports in the first migration. +The broader `material3.fig` fixture currently still exposes the known `BooleanOperation: EXCLUDE` codec enum issue and should not be treated as a regression from this package split. diff --git a/packages/docs/development/package-split.md b/packages/docs/development/package-split.md index 0ae0c1c0b..3ac5029d9 100644 --- a/packages/docs/development/package-split.md +++ b/packages/docs/development/package-split.md @@ -1,6 +1,6 @@ # Package split plan -OpenPencil currently keeps file-format internals in `@open-pencil/core` while `@open-pencil/dom-css` is being split out as a standalone DOM/CSS compatibility layer. Future package splits should preserve the current app behavior first, then move stable surfaces into independently publishable packages. +OpenPencil is splitting stable compatibility layers into independently publishable packages while keeping editor/runtime behavior intact. Each package boundary should be narrow, covered by package-local checks, and consumed through public workspace exports. ## Goals @@ -11,48 +11,54 @@ OpenPencil currently keeps file-format internals in `@open-pencil/core` while `@ ## Current package boundary -- `@open-pencil/core` owns scene graph, renderer, editor actions, Figma API compatibility, Kiwi codec internals, and format import/export. +- `@open-pencil/core` owns scene graph, renderer, editor actions, Figma API compatibility, `.fig` SceneGraph policy, and format import/export. - `@open-pencil/dom-css` owns DesignDOM, CSS runtimes, HTML/JSX/Tailwind projection, and SceneGraph ⇄ DesignDOM conversion. +- `@open-pencil/kiwi` owns pure Kiwi schema/runtime code plus low-level Figma Kiwi codec, container, and parse helpers. - App, CLI, MCP, and Vue SDK consume packages through public workspace exports only. -See [`kiwi-package-plan.md`](./kiwi-package-plan.md) for the detailed `@open-pencil/kiwi` inventory, dependency blockers, and package-local test plan. +See [`kiwi-package-plan.md`](./kiwi-package-plan.md) for the detailed `@open-pencil/kiwi` inventory, package-local test plan, and remaining split boundaries. ## Candidate packages ### `@open-pencil/kiwi` +Status: extracted. + Scope: - Kiwi binary schema runtime. -- Schema-generated codec modules. -- Generic binary parse/serialize helpers. -- Low-level validation helpers that do not know about OpenPencil scene graph nodes. +- Figma Kiwi schema data and validation. +- Low-level Figma Kiwi protocol/message helpers. +- Structural `NodeChange` encode/decode. +- `fig-kiwi` container framing and raw parse helpers that do not create `SceneGraph` objects. -Should not include: +Does not include: -- Figma `.fig` container policy. - SceneGraph conversion. +- Raw metadata invalidation policy. +- Component/instance override interpretation. - Renderer/editor code. -Minimum exit criteria: +Minimum maintenance criteria: - Package-local typecheck, unit tests, build, and dist smoke. -- Existing Kiwi serialize/parse tests passing through the package public API. -- No import cycles from `@open-pencil/core` back into the package. +- No `@open-pencil/core`, `#core/*`, app, CLI, MCP, or Vue imports. +- Existing repo-level Kiwi/FIG tests continue to pass through public package APIs. ### `@open-pencil/fig` Scope: -- `.fig` container read/write. +- `.fig` document read/write policy. - Figma node-change import/export. - Raw metadata preservation and invalidation policy. +- Figma component/instance interpretation. - Figma oracle fixtures and compatibility helpers. Should depend on: -- `@open-pencil/core` scene graph types. -- `@open-pencil/kiwi` once the Kiwi split exists. +- `@open-pencil/core` scene graph types and conversion policy while those remain core-owned. +- `@open-pencil/kiwi` for low-level schema/runtime/container/codec helpers. Should not include: @@ -66,45 +72,53 @@ Minimum exit criteria: - Heavy Figma fixture coverage still available at repo level. - Public API supports CLI/MCP/app document I/O without private path imports. -## Initial inventory +## Current inventory -Likely `@open-pencil/kiwi` candidates: +In `@open-pencil/kiwi`: -- `packages/core/src/kiwi/schema-runtime/**` -- `packages/core/src/kiwi/fig/codec/**` -- Generated Kiwi schema modules under the Figma codec directory, as long as they stay scene-graph agnostic +- `packages/kiwi/src/schema-runtime/**` +- `packages/kiwi/src/fig/schema/**` +- `packages/kiwi/src/fig/schema.ts` +- `packages/kiwi/src/fig/protocol.ts` +- `packages/kiwi/src/fig/types.ts` +- `packages/kiwi/src/fig/variable-bindings.ts` +- `packages/kiwi/src/fig/codec.ts` +- `packages/kiwi/src/fig/container.ts` +- `packages/kiwi/src/fig/parse.ts` Likely `@open-pencil/fig` candidates: - `packages/core/src/kiwi/fig/file.ts` -- `packages/core/src/kiwi/fig/container/**` -- `packages/core/src/kiwi/fig/parse/**` +- `packages/core/src/kiwi/fig/parse/transfer.ts` +- `packages/core/src/kiwi/fig/parse/worker.ts` - `packages/core/src/kiwi/fig/import.ts` +- `packages/core/src/kiwi/fig/lazy-import.ts` - `packages/core/src/kiwi/fig/node-change/**` - `packages/core/src/kiwi/fig/instance-overrides/**` - `packages/core/src/io/formats/fig/**` Keep in `@open-pencil/core` unless proven otherwise: -- `SceneGraph` and node type definitions -- Renderer/editor fallback behavior -- Layout, text measurement, and canvas-specific code -- Generic IO registry contracts that other formats use +- `SceneGraph` and node type definitions. +- Renderer/editor fallback behavior. +- Layout, text measurement, and canvas-specific code. +- Generic IO registry contracts that other formats use. ## Migration checklist 1. Add package-local tests before moving files. 2. Confirm every moved module imports only allowed public package exports. -3. Preserve existing `@open-pencil/core/kiwi` re-exports during the first migration step. -4. Move one boundary at a time: schema runtime first, generated codec second, `.fig` policy last. -5. Keep fixture/oracle tests in the repo-level suite even after package-local tests exist. -6. Run package smoke checks from a temporary consumer project before publishing. +3. Prefer direct imports from the new package for internal code; avoid accumulating deep re-export shims. +4. Preserve only intentional public compatibility barrels, such as `@open-pencil/core/kiwi`, when external consumers need a deprecation window. +5. Move one boundary at a time: schema/runtime first, low-level codec/container/parse second, `.fig` policy last. +6. Keep fixture/oracle tests in the repo-level suite even after package-local tests exist. +7. Run package smoke checks from a temporary consumer project before publishing. ## Migration order 1. Keep `@open-pencil/dom-css` standalone and stabilize its browser/headless runtime split. -2. Extract pure Kiwi runtime/codecs behind `@open-pencil/kiwi` without moving `.fig` policy. -3. Move `.fig` container and node-change conversion into `@open-pencil/fig`. +2. Extract pure Kiwi runtime/codecs behind `@open-pencil/kiwi` without moving `.fig` SceneGraph policy. +3. Move `.fig` document policy and node-change conversion into `@open-pencil/fig` when the boundary is clear. 4. Update core/app/CLI/MCP imports to consume public package exports. 5. Keep compatibility re-exports in `@open-pencil/core` only if existing consumers need a deprecation window. @@ -113,3 +127,4 @@ Keep in `@open-pencil/core` unless proven otherwise: - Do not guess Figma schema fields during the split. - Do not move renderer-specific fallback behavior into file-format packages. - Do not add browser DOM dependencies to core or file-format packages. +- Do not widen app/CLI imports to private package source paths.