Danila Poyarkov e0716a3a38

feat: behaviours and preview mode (#893 )

* feat: author behaviours on main components

A main component or component set can behave as a Switch, Checkbox,
Slider, or Tabs, after Reka UI's primitives. The behaviour lives in
OpenPencil plugin data: boolean values bind to variant or boolean
properties with the values meaning on and off, a number keeps its own
range since Figma has no number property, and the control's
subcomponents bind to the component's slots. A Behaviour section in the
properties panel adds, binds, and removes it, each as one undo step,
and flags required bindings that are missing. The canvas-only layout's
pill becomes a component that preview will reuse.

* feat: preview instances with behaviours on the canvas

View > Preview (Cmd+Alt+Enter) puts the canvas in preview: a lone canvas
switches to the canvas-only layout with a Previewing pill, and a split
canvas previews on its own side. Clicking a Switch or Checkbox flips it,
dragging a Slider moves its thumb and range, and clicking a Tabs trigger
shows its panel. Preview keeps its state on copies of the instances it
touched, in a private graph with the document's ids, and the canvas
draws those copies in place of the originals, so the document, undo,
autosave, and collaborators never see it. Escape or the pill leaves
preview, Reset restores every control, and editing shortcuts, labels,
and outlines stay off while previewing.

* feat: translate behaviour and preview strings; cover preview with an e2e flow

* refactor(vue): reuse VariantDefinitionControl for behaviour property options

* refactor: split variant actions and preview interactions by domain

Variant authoring was one 706-line closure; it is now graph queries
(model), undo snapshots (history), property definition edits
(definitions), and the editor facade (index). Preview interactions move
into play/kinds, one module per control, registered by behaviour kind so
a new kind cannot ship without its contract and interaction. Behaviour
contracts are keyed by kind. In the Vue SDK, slot and variant authoring
controls get their own folders beside component-props and behaviour,
and the app's variant section joins slot/ and behaviour/.

* refactor: keep the behaviour model in scene-graph's plugin-data registry

Master now defines every OpenPencil plugin-data key in one typed registry
in scene-graph. The behaviour schema registers there as a field, and the
model and contracts move beside slots, exported from the package root;
the @open-pencil/core/behaviours subpath is gone.

* feat: interaction states and keyboard focus in preview

A behaviour can bind a variant property to the default, hover, pressed,
focus, and disabled states; binding it maps values named like those
states. Preview switches the instance's copy to the matching variant as
the pointer hovers, presses, and releases, keeps other values when the
set draws the combination and falls back to rest otherwise, and skips
disabled instances. Tab moves visible keyboard focus between controls,
Space, Enter, arrows, Home, and End use the focused one, and Escape
takes visible focus off before leaving preview. A Button kind covers
controls that only have states.

* feat: toggle, radio, group, progress, collapsible, and accordion behaviours

Radio group, toggle group, and accordion hold their items in a slot;
each item is an instance with its own behaviour, so a press inside the
slot goes to the group, which turns the pressed item on and the others
off through the item's own interaction. Progress shares the slider's
number handling through rangeControl, and a collapsible shows and hides
its content slot from its trigger, remembering its open state even
when no property draws it. Tabs and groups share arrow-key navigation.

* feat: text field, textarea, and number field behaviours

A behaviour value can now be text, bound to a text property, so
preview types into a copy of the field through the same property path
the editor uses. A bound Filled value switches to the placeholder
variant when the field empties. A number field keeps its own range,
shows its value through a text property, and steps from its increment
and decrement slots and the arrow keys. Text fields show focus from a
click, and the focused control receives every key; Option still types,
and only Cmd or Ctrl combinations stay shortcuts.

* fix: keep behaviour bindings when saving as .fig

Saving as .fig gives component properties new GUIDs, but behaviours
kept the old ids in their plugin data, so every binding read as missing
after reopening. The export now renames the ids behaviours bind with
the same GUIDs, on its own copy of the document.

* fix: let previewed controls resize layout imported from .fig

Layers from a .fig keep the sizes Figma computed, and auto layout
prefers them, so an opened collapsible or accordion item kept its
closed height in preview. When preview shows, hides, or retypes a
layer in a copy, it drops those sizes from the layer's copied ancestors
so auto layout sizes them again; untouched layers keep Figma's sizes.

* fix: publish behaviours and other plugin content with library assets

Every OpenPencil plugin-data field now declares its role: content that
exists only as plugin data (behaviours, OkHCL picks), format copies of
node fields written for files, or bookkeeping about where a document
or node came from. Library snapshots keep a node's content plugin data,
including other plugins' entries, and drop the rest; the asset hash
counts the same entries, so a behaviour-only change is offered as an
update while a .fig round trip still changes nothing.

* feat: name behaviour rows by meaning and create what they need

The Behaviour section named every main value "Value" under a "Values"
heading, and a component without matching properties left an empty
picker with no way forward. Rows are now named for the control (On,
Checked, Pressed, Text), rows the control needs or already uses come
first, and the optional rest folds under More options; a button keeps
its states in view. An empty row creates what it needs in one undo
step: a text layer and text property, Off and On variants on a set, or
a slot frame for a part. The missing chip names the row it means and
takes you there.

* fix(dom-css): position free layers, hug content, and round ellipses

HTML and Tailwind export stacked the layers of frames without auto
layout in block flow, wrote fixed pixel sizes for auto layout frames
set to Hug and for auto-sizing text, and drew ellipses as boxes. Layers
a parent does not lay out are now absolutely positioned at their
coordinates inside a relative frame, hugging axes are left to the
content, and ellipses get a 50% radius.

* feat: run preview as live Reka UI islands over the canvas

Preview simulated controls on the canvas: copies of instances, a
handler per kind, its own key routing, and append-only text. It now
runs them as real components. Each top-level layer that holds an
instance with a behaviour becomes an island: its layers are projected
to DOM through dom-css into a shadow root laid over the pane at its pan
and zoom, and each behaviour mounts its Reka UI primitives on its
layers, so text fields are real inputs and focus, keys, and layout are
the browser's. Core's resolvePlayState shows instances in a state on a
private graph, so the component's variants draw it, and controls are
keyed by layer path so a variant switch keeps their DOM. The canvas
leaves island layers to the islands, and the canvas play runtime and
its key routing are gone.

* fix: derive variant properties from Property=Value component names

figma.combineAsVariants and Combine as variants only derived variant
properties from slash-separated names, so components named as Figma
names variants, such as State=On, Size=Large, became a set with no
properties. Both now derive each named property and its values, after
the slash form.

* feat: script and tool access to behaviours by name

Behaviour contracts follow Reka UI's anatomy: tabs keep their triggers
in the list slot and their content panels in a panels slot, and a slot
of repeated parts names the Reka part of its children. A behaviour
spec names component properties and slots instead of ids and resolves
to the stored behaviour and back, with errors that list what the
component has.

Scripts get an `openpencil` global next to `figma`, in the Figma API's
style: setBehaviour, getBehaviour with bindValue, bindPart, states,
and missing, behaviourKinds, and createSlot. The eval tool, the CLI,
and app automation compile scripts through one compileScript, so the
CLI now returns the last expression as the others do. MCP and AI chat
get set_behaviour, get_behaviour, and create_slot.

* feat: write controls in design JSX with Reka UI's element names

`<Switch.Root modelValue="State">` renders a main component, or a set
when its children are variants, that behaves as a switch, and
`<Switch.Thumb>` the slot that draws its thumb, one slot across the
set's variants. Inputs become the text property of a field, tab
triggers and panels go in their List and Panels slots, and a group's
items are `<RadioGroup.Item of={…} />` instances in its Items slot.
JSX export writes components with behaviours the same way, so they
render back unchanged. The authoring reference documents controls, and
the codegen and chat prompts now include it verbatim instead of
dedenting its code examples.

* chore: format the CLI export test

* docs: document slots, behaviours, preview, and the openpencil API

The components guide covers slots, behaviours, and preview with its
shortcut; scripting covers the openpencil global and eval's last-
expression result; the MCP and AI chat pages list the new tools; the
features overview, README, and roadmap mention working controls. The
chat prompt says how to build a control, and the codegen prompt builds
components with behaviours on their Reka UI primitives.

* chore: format the eval CLI test

* docs: explain behaviours and preview islands, and guide the openpencil API

A development page explains the behaviour model, the four authoring
surfaces, how preview islands turn a control's state into live Reka UI
components, and how to add a kind; the architecture page links it. The
Core guide sets the rules for OpenPencilAPI: Figma-only `figma`,
OpenPencil features on `openpencil` in the same style, one
compileScript, names over ids, and docs with every member. Package
READMEs mention the openpencil global, PlayIslands, Reka-named JSX, and
the behaviour model. Design JSX's behaviour modules move into a
behaviours folder instead of a suffixed sibling.

* refactor: center pasted layers through translate

centerNodesAt repeated translate's loop, which test:dupes reports on
master too.

* fix: validate behaviour ranges and guess on and off by name

A number value now needs max above min and a positive step: the schema,
specs, and the panel reject a range a slider cannot step through. Binding
a variant property guesses on and off by value name, as specs do, and a
boolean property gets no on/off pair. Part bindings are read through
partBinding, a replaced document restarts preview from its designed
state, and the e2e preview shortcut uses ControlOrMeta.

* feat: make the Behaviour section say what to do next

A slider's range fields now carry inline Min, Max, Step, and Start labels.
States offers Add state variants, which adds a Default, Hover, Pressed,
Focus, and Disabled variant and binds them; Add Off and On variants and
Add state variants turn a lone main component into a component set first,
and a part's slot can be added to a set, in every variant under one slot
id. Rows that could do nothing are gone: no empty pickers and no hints to
combine variants by hand, and an unbound Disabled is left to the states.
A warning line names what is still needed and replaces the missing chip,
and the Switch's main value is called Checked.

* fix: keep each slot to one part and keep creating slots at hand

A slot draws one part, so the Behaviour section no longer offers a slot
another part uses, and specs (the openpencil API, tools, and JSX) reject
binding one slot to two parts. A part's picker keeps an action to add a
new slot in its footer, so adding the first slot no longer hides it for
the other parts.

2026-10-06 13:23:05 +00:00

13 KiB

Raw Blame History

Core

Renderer, layout, editor, Figma API, tools, clipboard, vector conversion, and document I/O. Depends on scene-graph, pen, kiwi, and fig. Framework-neutral: no Vue, no app imports, no browser DOM; guard browser globals explicitly.

Public surface is the compatibility barrel plus the subpaths listed in packages/core/package.json exports; add a subpath there rather than deep-importing.
CanvasKit runtime loading is centralized in @open-pencil/core/canvaskit. Headless raster export may dynamically load canvaskit-wasm/full; elsewhere import type and pass CanvasKit in.
Drawing and input share preview-aware geometry through @open-pencil/core/geometry, built on Scene Graph matrices. Use it for world/screen transforms, inverses, bounds, and handle placement.

Layout

@open-pencil/yoga-layout supplies both flexbox and CSS Grid.
Recompute layout after demo creation and for each materialized or imported page; scope computation to the affected page or subtree where possible.
The first Hug/Fill dimension mutation switches only that axis to Fixed; focus is non-destructive, and mode/value changes share one undo transaction.

Components and instances

Component types use #9747ff.
Component edits propagate through editor component sync in packages/core/src/editor/components/; never hand-copy properties in app UI. Use Scene Graph copy helpers for nested values.

Vector conversion

Bitmap-to-vector conversion lives in packages/core/src/vector/vectorize/; app provider clients, preferences, and credential resolution live under src/app/editor/vectorize/ in the app. Bound request and response sizes and validate provider-owned download URLs before importing returned SVG.

Tools (AI, MCP, CLI, WebMCP)

Operations are ToolDefs under packages/core/src/tools/**; schema.ts defines the contract and registries expose them. Each definition owns its native Valibot input, execution/mutation metadata, and optional per-interface exposure exclusions (mcp, ai, webmcp). Exposure defaults to inclusion; adapters use isToolExposed(), then apply execution support and user permissions independently.
Infer arguments from the schema; derive effects and default capabilities from execution metadata. Do not maintain parameter DSLs or tool-name lists. Add work to the nearest existing domain and the appropriate registry.
packages/core/src/tools/ai-adapter.ts converts ToolDefs for Vercel AI; the app binds them to the active editor's FigmaAPI in src/app/ai/tools/index.ts. MCP v2 registration uses Standard Schema with Valibot JSON Schema conversion; AI and WebMCP adapters share the same input contract.
packages/core/src/editor/history/atomic-tool.ts owns synchronous property/variable transactions; Scene Graph owns checkpoint recovery. AI, MCP, and WebMCP share this execution path. Async and structural tools cannot declare atomic property execution.
Shared scene-authoring guidance and tested examples live under packages/design-jsx/src/reference/; reference.ts combines them with renderer metadata. Prompts under packages/core/src/tools/prompts/ and the app chat/ACP prompt compose that reference rather than copying it. Run bun run generate:authoring-reference after changes; check:authoring-reference (part of check:docs) verifies the committed skill/docs copies. Do not edit generated reference files.
The installable agent skill is maintained in skills/open-pencil/. Changes to agent-facing APIs, CLI/MCP behavior, or design authoring must update affected skill examples, prompts, and public documentation in the same change. Keep examples valid in their actual execution environment; do not advertise library exports as scripting globals unless exposed there. Prefer runtime discovery and canonical references over duplicated API/tool inventories.
MCP-only tools and transports: packages/mcp/AGENTS.md. WebMCP registration and app completion: src/AGENTS.md.

Editor

createEditor() in packages/core/src/editor/create.ts assembles an EditorContext plus domain action modules for viewport, selection, pages, shapes, structure, components, clipboard, undo/history, text, variables, layout, color space, graph reads, and the tool registry. Editor is ReturnType<typeof createEditor>. Check the folder before adding behavior; keep new actions in the nearest domain module or folder instead of growing unrelated files. Modules share state through EditorContext, never through app code or Vue.

All selection mutations go through ctx.setSelectedIds() and all tool changes through ctx.setActiveTool() so events fire consistently. App code uses editor actions such as clearSelection(), select(), or setTool(), never direct state.selectedIds = or state.activeTool = assignments.
The editor exposes a typed nanoevents emitter. Event names and payloads live in EditorEvents in packages/core/src/editor/types.ts; graph events are bridged from SceneGraph by packages/core/src/editor/graph-events.ts. Subscribe with editor.onEditorEvent(event, handler); in Vue use useEditorEvent() from packages/vue/src/editor/events/use.ts. UI that only cares about graph data should use editor events for incremental surfaces such as the layer tree instead of watching repaint-only state.
Commands under packages/core/src/editor/structure/ (group, boolean, container wrap, flatten) and packages/core/src/editor/components/ are the canonical implementations of user actions. The Figma API, tools, and app call them or share their helpers; do not reimplement sizing, placement, or propagation elsewhere.
Live property controls use selected-node projections from packages/core/src/editor/selection-state/nodes.ts: shallow reactive copies that receive node:previewUpdated patches at property granularity. Never add preview invalidation to all useSceneComputed consumers; catalogs and unrelated controls must not refresh for geometry previews. Projection subscriptions belong to the consuming scope or session and must be disposed.
Numeric geometry edits own a beginNodePreview() handle with update, commit, and cancel. It captures all affected fields and layout children, publishes the complete delta once, and restores exact originals on cancellation. Selection, page, and graph changes and disposal cancel the old edit; trailing input must not target the new selection. Controls must close previews even when a gesture returns to its starting value.
Rotation previews change through setRotationPreview() and rotation:preview-changed; cancellation must close the owning gesture without deselecting or committing it.
Renderer interaction policy uses explicit beginInteractiveEdit() leases and isInteractiveEditing(), not undo batching. Release leases on every terminal path. Keep live queries callable across app facades that spread editor actions.
packages/core/src/editor/history/atomic-tool.ts owns synchronous property/variable transactions for AI, MCP, and WebMCP tools; see Tools above.

OpenPencil API

OpenPencilAPI (packages/core/src/openpencil-api/) is what OpenPencil adds to the Plugin API, exposed to scripts as the openpencil global next to figma.

Keep figma Figma-shaped and put OpenPencil-only features on openpencil, in the same style: methods and node-like handles with getters and setters, not parallel helper functions. Never add non-Figma members to FigmaAPI (compatibility.ts).
Script runners get both globals only through compileScript (packages/core/src/tools/analyze/eval/wrap.ts); the eval tool, openpencil eval, and app automation must not build their own AsyncFunction.
Members take names, not internal ids, and write through the same scene-graph functions the editor's actions use, such as behaviourFromSpec; errors name what exists. Tools for the same feature wrap the openpencil API rather than reimplementing it (packages/core/src/tools/create/behaviours.ts).
A new member updates packages/docs/programmable/cli/scripting.md and the agent skill in the same change; tests drive it as scripts do, through compileScript (packages/core/tests/openpencil-api/).

Renderer

Canvas is CanvasKit (Skia WASM) on a WebGL surface, not DOM.

Invalidation

renderVersion is a canvas repaint (pan, zoom, hover); sceneVersion is a scene-graph mutation. requestRender() bumps both; requestRepaint() bumps only renderVersion. UI that only cares about graph data must not watch repaint-only state.
renderNow() is only for surface recreation and font loading, where an immediate draw is required.
The resize observer uses a rAF throttle, not a debounce; debounce causes canvas skew.
Viewport culling skips off-screen nodes; unclipped parents are not culled because children may extend beyond bounds.
Overscan images accelerate navigation; settled scenes rasterize existing retained pictures at the live viewport size and origin. Pixel-grid alignment alone does not guarantee Skia anti-aliasing parity. Keep settlement pending until the viewport pass completes; do not add a second viewport image cache.

Paints

A gradient or image paint builds a Skia shader through applyGradientFill and applyImageFill (packages/core/src/canvas/fills.ts), which take the target Paint, so a stroke reuses them instead of a second shader path.
forVisibleStrokes (packages/core/src/canvas/scene.ts) is where a stroke's shader is set and cleared; stroke draw helpers take an already-configured strokePaint and must not reset its shader.

Caches

Bounded rendering caches share packages/core/src/cache/resource.ts for recency, count/weight accounting, and removal disposal. Domain adapters own keys, font/page/dependency invalidation, and sizing units; use non-touching peek() for FIFO or planning reads. Rejected insertions leave ownership with the caller.
Keep weak memos, async request registries, pools, and dependency-owned picture/path maps on their distinct lifetime policies.
Paragraph construction is typed against packages/core/src/canvas/text/paragraph-inputs.ts; the same inputs drive preparation-cache invalidation. Add a mutation case when extending that contract. Drawing borrows native paragraphs; the renderer owns their bounded cache and destruction.

Geometry and overlays

Use @open-pencil/core/geometry for world/screen transforms, inverses, bounds, and handle placement instead of interpreting ancestor rotations or reflections independently.
Label drawing and hit testing share packages/core/src/canvas/labels/{layout,transform,style}.ts, including paragraph measurements and unreflected label axes.
Selection border width is constant regardless of zoom: divide by scale. Section and frame title text never scales: render at a fixed font size and ellipsize to fit.
Rulers are rendered on the canvas with selection range badges that do not overlap tick numbers. Remote cursors are Figma-style colored arrows with a white border and name pill, rendered in screen space.

Visual coverage

Pixel-affecting features need committed visual coverage, not only mock or geometry assertions. Add or update a Playwright canvas snapshot for changes to fills, gradients, images, blend modes, masks, boolean geometry, corners, strokes, shadows, blur, text rendering, or demo showcase scenes. Use targeted updates such as bun run test tests/e2e/canvas/fill-modes-visual.spec.ts --update-snapshots, then rerun the same test without --update-snapshots.

Figma API

FigmaAPI mimics Figma's Plugin API over the SceneGraph. openpencil eval, the AI and MCP tools, and the app bind to it. Tests live under tests/engine/figma/api/ and mirror the API surface.

Shape follows Figma. Match @figma/plugin-typings at the version pinned in the root package.json. packages/core/src/figma-api/compatibility.ts type-checks the supported surface against PluginAPI; add every new member to SupportedPluginAPI there. Model unsupported members explicitly rather than approximating them.
Behavior follows Figma. Read the Plugin API documentation for the member, and for anything observable (geometry, ordering, defaults, errors) run the same script in live Figma and record the observed result in the test. figma-use and tests/figma/ are the live oracle; tests/fixtures/figma-oracles/ holds recorded values.
User actions are shared code. When an editor command does the same thing (group, ungroup, boolean, flatten, instance creation, component sync), call the implementation under packages/core/src/editor/ or its shared helper. Do not reimplement geometry, placement, or propagation here; new geometry helpers belong in Scene Graph.
Node proxies live in packages/core/src/figma-api/proxy.ts and packages/core/src/figma-api/accessors/; packages/core/src/figma-api/render-bounds.ts implements Figma's absoluteRenderBounds semantics and is the only geometry that is Figma-specific.
packages/core/src/figma-api/index.ts stays under the 600-line limit by moving whole domains into sibling modules such as components.ts and text.ts, not by extracting generic helpers into this folder.

13 KiB Raw Blame History