diff --git a/AGENTS.md b/AGENTS.md index f7d019718..795d67e16 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,7 +2,7 @@ Vue 3 + CanvasKit (Skia WASM) + Yoga WASM design editor. Tauri v2 desktop, also runs in browser. -**Roadmap:** `docs/development/variables-ui-roadmap.md` tracks remaining variable-system UI work. Current architecture and commands live in this file. +**Roadmap:** `packages/docs/development/variables-ui-roadmap.md` tracks remaining variable-system UI work. Current architecture and commands live in this file. ## Monorepo diff --git a/docs/eval-command.md b/docs/eval-command.md deleted file mode 100644 index c1ebc4f66..000000000 --- a/docs/eval-command.md +++ /dev/null @@ -1,432 +0,0 @@ -# `open-pencil eval` — Figma-like Plugin API for Headless Scripting - -## Overview - -`bun open-pencil eval --code ''` executes JavaScript against a `.fig` file with a Figma-compatible `figma` global object. This enables headless scripting, batch operations, AI tool execution, and testing — all without the GUI. - -The `figma` object mirrors Figma's Plugin API surface as closely as possible, so existing Figma plugin knowledge and code snippets transfer directly. - -```bash -# Create a frame, set auto-layout, add children -bun open-pencil eval design.fig --code ' - const frame = figma.createFrame() - frame.name = "Card" - frame.resize(300, 200) - frame.layoutMode = "VERTICAL" - frame.itemSpacing = 12 - frame.paddingTop = frame.paddingBottom = 16 - frame.paddingLeft = frame.paddingRight = 16 - frame.fills = [{ type: "SOLID", color: { r: 1, g: 1, b: 1 } }] - - const title = figma.createText() - title.characters = "Hello World" - title.fontSize = 24 - frame.appendChild(title) - - return { id: frame.id, name: frame.name } -' - -# Query nodes -bun open-pencil eval design.fig --code ' - const buttons = figma.currentPage.findAll(n => n.type === "FRAME" && n.name.includes("Button")) - return buttons.map(b => ({ id: b.id, name: b.name, w: b.width, h: b.height })) -' - -# Read from stdin (for multiline scripts / piping) -cat transform.js | bun open-pencil eval design.fig --stdin - -# Write changes back -bun open-pencil eval design.fig --code '...' --write -bun open-pencil eval design.fig --code '...' -o modified.fig -``` - -## Architecture - -``` -┌──────────────────────────────────────────────────────┐ -│ CLI: `open-pencil eval --code '...'` │ -│ ↓ │ -│ loadDocument(file) → SceneGraph │ -│ ↓ │ -│ FigmaAPI(sceneGraph) → `figma` proxy object │ -│ ↓ │ -│ AsyncFunction('figma', wrappedCode)(figmaProxy) │ -│ ↓ │ -│ print result as JSON / agentfmt │ -│ optionally: saveDocument(file) if --write │ -└──────────────────────────────────────────────────────┘ -``` - -### Key classes - -| Class | Location | Role | -|-------|----------|------| -| `FigmaAPI` | `packages/core/src/figma-api/` | Proxy object implementing `figma.*` methods against `SceneGraph` | -| `FigmaNode` | `packages/core/src/figma-api/` | Proxy wrapping `SceneNode` with Figma-style property access (`.fills`, `.resize()`, `.appendChild()`, etc.) | -| `eval` command | `packages/cli/src/commands/eval.ts` | CLI command that loads doc, creates API, executes code | - -### Why in `@open-pencil/core`? - -The `FigmaAPI` class lives in core (not CLI) because: -- **AI tools reuse it** — the chat panel's `render` tool can execute JSX through the same API -- **Test scripts** — unit tests can use the API to set up fixtures -- **No DOM deps** — runs headless in Bun, no browser APIs needed - -## `FigmaAPI` — Phased Implementation - -### Phase 1: Core (MVP for eval command) - -These cover ~80% of real plugin scripts: - -#### Document & Page - -| Figma API | Our implementation | Notes | -|-----------|--------------------|-------| -| `figma.root` | Getter → proxy for root node | `.children` returns page proxies | -| `figma.currentPage` | Getter/setter → first page by default | Settable to any page proxy | -| `figma.currentPage.selection` | Get/set → tracked selection array | | -| `figma.getNodeById(id)` | `graph.getNode(id)` wrapped in proxy | Sync, like Figma's deprecated version | - -#### Node Creation - -| Figma API | Maps to | -|-----------|---------| -| `figma.createFrame()` | `graph.createNode('FRAME', currentPageId)` | -| `figma.createRectangle()` | `graph.createNode('RECTANGLE', ...)` | -| `figma.createEllipse()` | `graph.createNode('ELLIPSE', ...)` | -| `figma.createText()` | `graph.createNode('TEXT', ...)` | -| `figma.createLine()` | `graph.createNode('LINE', ...)` | -| `figma.createPolygon()` | `graph.createNode('POLYGON', ...)` | -| `figma.createStar()` | `graph.createNode('STAR', ...)` | -| `figma.createComponent()` | `graph.createNode('COMPONENT', ...)` | -| `figma.createPage()` | `graph.addPage(name)` | -| `figma.createSection()` | `graph.createNode('SECTION', ...)` | - -#### Node Properties (via `FigmaNode` proxy) - -Read/write on any node proxy. Property access maps to `SceneNode` fields: - -```ts -// Geometry -node.x, node.y // direct -node.width, node.height // read-only, use node.resize(w, h) -node.rotation // direct -node.resize(w, h) // updates width + height -node.resizeWithoutConstraints(w, h) // same (no constraint engine yet) - -// Visual -node.fills // get/set Fill[] -node.strokes // get/set Stroke[] -node.effects // get/set Effect[] -node.opacity // get/set number -node.visible // get/set boolean -node.locked // get/set boolean -node.blendMode // get/set BlendMode -node.clipsContent // get/set boolean - -// Corner radius -node.cornerRadius // get/set (number or figma.mixed) -node.topLeftRadius // get/set -node.topRightRadius // get/set -node.bottomLeftRadius // get/set -node.bottomRightRadius // get/set -node.cornerSmoothing // get/set - -// Identity -node.id // read-only -node.name // get/set -node.type // read-only -node.parent // read-only → FigmaNode | null -node.removed // read-only boolean -``` - -#### Tree Operations - -```ts -node.children // read-only FigmaNode[] -node.appendChild(child) // reparent to end -node.insertChild(index, child) // reparent at index -node.remove() // graph.deleteNode(id) - -// Traversal -node.findAll(callback?) // recursive find -node.findOne(callback) // first match -node.findChild(callback) // direct children only -node.findChildren(callback?) // direct children only -``` - -#### Auto-layout - -```ts -node.layoutMode // 'NONE' | 'HORIZONTAL' | 'VERTICAL' -node.primaryAxisAlignItems // 'MIN' | 'CENTER' | 'MAX' | 'SPACE_BETWEEN' -node.counterAxisAlignItems // 'MIN' | 'CENTER' | 'MAX' | 'BASELINE' -node.itemSpacing // number -node.counterAxisSpacing // number | null -node.paddingTop / Right / Bottom / Left // number -node.layoutWrap // 'NO_WRAP' | 'WRAP' - -// Child sizing -node.layoutPositioning // 'AUTO' | 'ABSOLUTE' -node.layoutGrow // 0 | 1 -node.layoutSizingHorizontal // 'FIXED' | 'HUG' | 'FILL' -node.layoutSizingVertical // 'FIXED' | 'HUG' | 'FILL' -``` - -#### Text - -```ts -node.characters // get/set (maps to node.text) -node.fontSize // get/set -node.fontName // get/set { family, style } -node.fontWeight // get/set -node.textAlignHorizontal // get/set -node.textAlignVertical // get/set -node.textAutoResize // get/set -node.letterSpacing // get/set -node.lineHeight // get/set -node.maxLines // get/set -node.textCase // get/set -node.textDecoration // get/set -``` - -#### Stroke details - -```ts -node.strokeWeight // get/set (maps to strokes[0].weight) -node.strokeAlign // get/set (maps to strokes[0].align) -node.dashPattern // get/set -``` - -#### Misc - -```ts -figma.mixed // Symbol sentinel for mixed values -figma.group(nodes, parent) // creates GROUP with given children -figma.ungroup(node) // ungroups, reparents children -figma.flatten(nodes) // NOT IMPLEMENTED YET — returns first node -``` - -#### Export - -```ts -node.exportAsync(settings?) // only works if CanvasKit is loaded - // settings: { format: 'PNG'|'JPG'|'SVG', constraint? } -``` - -### Phase 2: Components & Instances - -| API | Maps to | -|-----|---------| -| `figma.createComponent()` | `graph.createNode('COMPONENT', ...)` | -| `figma.createComponentFromNode(node)` | Convert existing frame to component | -| `figma.combineAsVariants(components, parent)` | Create COMPONENT_SET | -| Node: `node.createInstance()` | `graph.createInstance(componentId, parentId)` | -| Node: `node.detachInstance()` | `graph.detachInstance(id)` | -| `figma.getNodeById(id).mainComponent` | `graph.getMainComponent(id)` | - -### Phase 3: Variables - -| API | Maps to | -|-----|---------| -| `figma.variables.getLocalVariables(type?)` | `graph.variables` filtered | -| `figma.variables.getLocalVariableCollections()` | `graph.variableCollections` | -| `figma.variables.createVariable(name, collection, type)` | `graph.addVariable(...)` | -| `figma.variables.createVariableCollection(name)` | `graph.addCollection(...)` | -| `figma.variables.getVariableById(id)` | `graph.variables.get(id)` | -| `node.setBoundVariable(field, variable)` | `graph.bindVariable(...)` | -| `node.boundVariables` | getter from SceneNode | - -### Phase 4: Styles & Advanced - -| API | Notes | -|-----|-------| -| `figma.createPaintStyle()` | Requires style storage in SceneGraph | -| `figma.createTextStyle()` | Requires style storage in SceneGraph | -| `figma.createEffectStyle()` | Requires style storage in SceneGraph | -| `figma.loadFontAsync(fontName)` | No-op (we don't have font loading constraints) | -| `figma.listAvailableFontsAsync()` | Return system fonts if available | -| Boolean operations (`union`, `subtract`, `intersect`, `exclude`) | Requires path boolean engine | -| `figma.createNodeFromJSXAsync(jsx)` | Port figma-use's JSX renderer | - -## `FigmaNode` Proxy Design - -The proxy wraps a `SceneNode` and translates Figma property names to our internal names. Key mappings: - -```ts -const PROPERTY_MAP: Record = { - // Figma name → SceneNode field name (only where they differ) - 'characters': 'text', - 'strokeWeight': → computed from strokes[0].weight, - 'strokeAlign': → computed from strokes[0].align, - 'fontName': → computed from { family: fontFamily, style: ... }, - 'primaryAxisAlignItems': 'primaryAxisAlign', - 'counterAxisAlignItems': 'counterAxisAlign', - 'primaryAxisSizingMode': 'primaryAxisSizing', // value mapping: 'AUTO' → 'HUG', 'FIXED' → 'FIXED' - 'counterAxisSizingMode': 'counterAxisSizing', - 'layoutSizingHorizontal': → computed from primaryAxisSizing / counterAxisSizing depending on layoutMode - 'layoutSizingVertical': → computed -} -``` - -Methods on the proxy: - -```ts -class FigmaNode { - // The proxy is created via: new Proxy(target, handler) - // where handler.get intercepts property reads and handler.set intercepts writes - - resize(width: number, height: number): void - resizeWithoutConstraints(width: number, height: number): void - remove(): void - appendChild(child: FigmaNode): void - insertChild(index: number, child: FigmaNode): void - findAll(callback?: (node: FigmaNode) => boolean): FigmaNode[] - findOne(callback: (node: FigmaNode) => boolean): FigmaNode | null - findChild(callback: (node: FigmaNode) => boolean): FigmaNode | null - findChildren(callback?: (node: FigmaNode) => boolean): FigmaNode[] - exportAsync(settings?: ExportSettings): Promise - - // Components (Phase 2) - createInstance(): FigmaNode - detachInstance(): void - get mainComponent(): FigmaNode | null -} -``` - -## CLI Command - -``` -bun open-pencil eval [options] - -Arguments: - file .fig file to operate on - -Options: - --code, -c JavaScript code to execute (has access to `figma` global) - --stdin Read code from stdin instead of --code - --write, -w Write changes back to the input file - -o, --output Write to a different file - --json Output result as JSON (default for non-TTY) - --quiet, -q Suppress output, only write file -``` - -### Execution model - -1. Load `.fig` → `SceneGraph` -2. Create `FigmaAPI(graph)` → `figma` proxy -3. Wrap user code in async function: `return (async () => { })()` -4. Execute with `figma` as sole argument -5. Print return value (JSON or agentfmt) -6. If `--write` or `-o`: serialize `SceneGraph` back to `.fig` - -### Return value formatting - -- `undefined` / `void` → no output -- Primitives → printed directly -- Objects/arrays → `JSON.stringify(result, null, 2)` or agentfmt tables -- `FigmaNode` → serialized as `{ id, type, name, x, y, width, height, fills, ... }` -- Arrays of `FigmaNode` → serialized as list - -## Shared with AI Tools - -The `FigmaAPI` class is the **same API surface** that AI tools use. App AI tools are wired from `src/app/ai/tools/index.ts` through the shared core tool definitions, so CLI scripts, MCP calls, and chat tool calls execute the same scene-graph operations: - -```ts -// Tool implementation using FigmaAPI -execute: async ({ type, x, y, width, height }) => { - const frame = figma.createFrame() - frame.resize(width, height) - frame.x = x - frame.y = y - return { id: frame.id } -} -``` - -This ensures CLI scripts and AI tools behave identically. - -## File Layout - -``` -packages/core/src/ - figma-api/ # FigmaAPI class + FigmaNode proxy - tools/ # Shared ToolDef operations used by AI, MCP, and eval helpers - -packages/cli/src/commands/ - eval.ts # CLI command - -src/app/ai/tools/ - index.ts # App chat wiring for shared core tool definitions -``` - -## Test Plan - -### Unit tests (`packages/core/src/figma-api.test.ts`) - -1. **Node creation** — each `createX()` creates correct type, added to current page -2. **Property access** — `.fills`, `.x`, `.width`, `.name`, `.characters` read/write correctly -3. **Resize** — `.resize(w, h)` updates width/height -4. **Tree operations** — `.appendChild()`, `.insertChild()`, `.remove()`, `.parent`, `.children` -5. **Traversal** — `.findAll()`, `.findOne()`, `.findChild()`, `.findChildren()` with callbacks -6. **Auto-layout** — `.layoutMode`, `.itemSpacing`, `.paddingTop`, etc. -7. **Text** — `.characters` maps to `.text`, `.fontName` maps to `{ family, style }` -8. **Mixed values** — `.cornerRadius` returns `figma.mixed` when corners differ -9. **Selection** — `figma.currentPage.selection` get/set -10. **Page switching** — `figma.currentPage = page2` works -11. **Group/ungroup** — `figma.group()` creates group, `figma.ungroup()` dissolves it -12. **Clone** — node creation produces independent copies - -### CLI integration tests (`packages/cli/src/commands/eval.test.ts`) - -1. **Basic eval** — `eval test.fig --code 'return figma.currentPage.name'` → page name -2. **Create + read** — create a frame, return its properties -3. **Query nodes** — `findAll` returns correct nodes -4. **Write back** — `--write` saves changes, reloading shows them -5. **Stdin** — `echo 'return 42' | bun open-pencil eval test.fig --stdin` → `42` -6. **JSON output** — `--json` returns valid JSON -7. **Error handling** — syntax errors, runtime errors reported cleanly - -## Implementation Order - -1. **`FigmaNode` proxy** — property mapping, `.resize()`, `.remove()`, tree methods -2. **`FigmaAPI` class** — `createFrame/Rectangle/...`, `.root`, `.currentPage`, `.getNodeById()`, `.mixed`, `.group()` -3. **CLI `eval` command** — argument parsing, code wrapping, output formatting -4. **Unit tests** — all 12 test groups above -5. **CLI integration tests** — all 7 test groups above -6. **Wire to AI tools** — expose shared core tool definitions through `src/app/ai/tools/index.ts` -7. **Phase 2** — components & instances -8. **Phase 3** — variables -9. **Phase 4** — styles, boolean ops, JSX renderer - -## Property Mapping Reference - -| Figma Property | SceneNode Field | Type | Notes | -|---------------|-----------------|------|-------| -| `characters` | `text` | `string` | | -| `fontName` | `fontFamily` + `fontWeight` + `italic` | `{ family, style }` | Computed: `style` = "Bold Italic" etc. | -| `strokeWeight` | `strokes[0].weight` | `number` | Computed | -| `strokeAlign` | `strokes[0].align` | `string` | Computed | -| `primaryAxisAlignItems` | `primaryAxisAlign` | `string` | | -| `counterAxisAlignItems` | `counterAxisAlign` | `string` | | -| `layoutSizingHorizontal` | `primaryAxisSizing` or `counterAxisSizing` | `string` | Depends on `layoutMode` | -| `layoutSizingVertical` | (opposite of horizontal) | `string` | | -| `absoluteTransform` | computed from `x`, `y`, `rotation` | `Transform` | Read-only | -| `absoluteBoundingBox` | `getAbsoluteBounds(id)` | `Rect` | Read-only | -| All others | Same name | Same type | Direct passthrough | - -## Open Questions - -1. **Font loading**: `figma.loadFontAsync()` — should it be a no-op (we don't have font gating) or should we track loaded fonts? - → **Decision: No-op that returns resolved Promise.** We don't gate text editing on font loading. - -2. **Export in headless mode**: `node.exportAsync()` requires CanvasKit. Should eval load CanvasKit? - → **Decision: Optional.** If CanvasKit is available (via `--with-canvaskit` flag or env), enable export. Otherwise, throw "Export requires CanvasKit" error. - -3. **`figma.mixed` symbol**: Should we use the actual Figma symbol or our own? - → **Decision: Our own `Symbol('mixed')`.** Exposed as `figma.mixed`. - -4. **Undo**: `figma.commitUndo()` / `figma.triggerUndo()` — relevant in headless? - → **Decision: No-op in CLI.** Undo only matters in the live editor. The AI tools can add undo support separately via EditorStore. - -5. **Write format**: Should `--write` produce `.fig` (Kiwi binary) or also support `.json`? - → **Decision: `.fig` only for now.** JSON export is a separate feature. diff --git a/docs/renderer-profiler-plan.md b/docs/renderer-profiler-plan.md deleted file mode 100644 index a3aaf3219..000000000 --- a/docs/renderer-profiler-plan.md +++ /dev/null @@ -1,352 +0,0 @@ -# Renderer Profiler — Design Plan - -Production-grade instrumentation for OpenPencil's CanvasKit/Skia renderer, inspired by game engine profiling tools (Unreal Insights, Unity Frame Timing Manager) and browser render engine internals (Chrome Compositor, WebKit/Skia tracing). - -## What Skia gives us - -### 1. Profiling build of CanvasKit - -`canvaskit-wasm` ships three bundles: -- **default** — stripped WASM names -- **full** — includes Skottie etc. -- **profiling** — same as full but with **full internal function names** preserved in WASM - -```ts -import InitCanvasKit from 'canvaskit-wasm/profiling' -``` - -This means Chrome DevTools Performance panel flame charts will show real Skia C++ function names (`GrDrawingManager::internalFlush`, `SkCanvas::drawRect`, etc.) instead of mangled `$func123`. We should use this build behind a `?profiling` URL param or a dev-mode toggle. - -### 2. SkPicture recording & replay - -Skia's `SkPictureRecorder` records draw commands into an immutable `SkPicture`. We already use this for caching (`recordScenePicture`). Key profiling insight from Skia's own benchmark tool: - -> `CanvasKit.flush()` returns after it has sent all instructions to the GPU, but we don't know the GPU is done until the **next frame is requested**. Thus, we need to keep track of time **between frames** to accurately calculate draw time. - -- `drawPicture + flush` duration tells us CPU-side WASM/JS time -- `total_frame_ms` (time between rAFs) tells us the real GPU-inclusive cost -- If `total_frame_ms ≈ with_flush_ms` → CPU-bound -- If `total_frame_ms >> with_flush_ms` → GPU-bound - -### 3. Skia Debugger (external tool) - -Skia has an online debugger at https://debugger.skia.org that can load `.skp` files and show: -- Draw command playback -- GPU op bounds visualization -- Overdraw visualization -- Clip/matrix state at any step -- GPU operation batching (colored op IDs) - -We could add an "Export .skp" debug feature to save the current `scenePicture` for analysis. - -### 4. EXT_disjoint_timer_query (GPU timing) - -The `EXT_disjoint_timer_query_webgl2` WebGL extension provides **actual GPU-side timing** in nanoseconds. Figma built [figma/webgl-profiler](https://github.com/figma/webgl-profiler) on top of this exact extension. - -**Limitations:** -- Only works in Desktop Chrome ≥ 70 -- `TIMESTAMP_EXT` was removed for security; only `TIME_ELAPSED_EXT` (begin/end query) works -- Results are asynchronous (available next frame or later) -- Non-draw commands often report 0ns (but totals are accurate) -- Inconsistent on non-Apple-Silicon Intel GPUs - -**Usage pattern:** -```ts -const ext = gl.getExtension('EXT_disjoint_timer_query_webgl2') -const query = gl.createQuery() -gl.beginQuery(ext.TIME_ELAPSED_EXT, query) -// ... draw calls ... -gl.endQuery(ext.TIME_ELAPSED_EXT) -// next frame: poll gl.getQueryParameter(query, gl.QUERY_RESULT_AVAILABLE) -``` - -### 5. Chrome Performance Panel extensibility - -Chrome now supports custom tracks via `performance.mark`/`performance.measure` with `detail.devtools` metadata, and the lower-overhead `console.timeStamp` API: - -```ts -performance.measure("Scene Render", { - start: startMark, - detail: { - devtools: { - dataType: "track-entry", - track: "OpenPencil Renderer", - trackGroup: "OpenPencil", - color: "primary", - properties: [["Nodes", "142"], ["Cached", "true"]], - tooltipText: "Full scene render with SkPicture cache hit" - } - } -}) -``` - -This shows up as a dedicated "OpenPencil Renderer" track in the DevTools Performance panel alongside browser internals. Extremely powerful for production debugging. - ---- - -## Architecture - -### Layers of profiling (from light to heavy) - -| Layer | Overhead | Always-on | What it measures | -|-------|----------|-----------|-----------------| -| **Frame Stats HUD** | ~0.1ms | Optional toggle | FPS, frame time, CPU/GPU split, node count, cache hits | -| **User Timing marks** | ~0.01ms each | Dev builds | Phase timings visible in DevTools Performance panel | -| **GPU Timer Queries** | ~0.2ms setup | On demand | Actual GPU execution time per frame (async readback) | -| **Detailed profiler** | ~1-3ms | Manual trigger | Per-node timings, draw call counts, overdraw heatmap | - -### 1. Frame Stats HUD (always-available, minimal overhead) - -An in-canvas overlay rendered by SkiaRenderer itself, togglable via keyboard shortcut (Shift+P or similar). Modeled after game engine stat overlays. - -**Metrics displayed:** -``` - FPS: 60 (16.7ms) ← smoothed rAF delta - CPU: 4.2ms ← JS/WASM time (start of render → after flush) - GPU: 8.1ms ← GPU time (async, from EXT_disjoint_timer_query or inferred) - ───────────────── - Nodes: 342 (47 culled) ← total vs viewport-culled count - Draw calls: 128 ← WebGL draw call count - Cache: SkPicture HIT ← scene picture reuse status - Textures: 12 (48MB) ← image cache stats - WASM heap: 64MB ← CanvasKit memory -``` - -**Frame time graph** — rolling 120-frame history bar chart (like Unreal's `stat unit`), color-coded: -- Green: within budget (16.7ms for 60fps) -- Yellow: over budget but < 2× (16.7–33.3ms) -- Red: severe (> 33.3ms) - -The graph draws CPU bars from bottom and GPU bars from top, making CPU/GPU overlap visible at a glance (exactly how Unity Frame Timing Manager visualizes it). - -### 2. Phase instrumentation (User Timing API) - -Instrument the `render()` method with `performance.mark`/`performance.measure` using the Chrome DevTools extensibility API to create a custom "OpenPencil" track group: - -**Phases to instrument:** - -| Phase | What | Where in code | -|-------|------|---------------| -| `frame` | Total rAF-to-rAF | `useCanvas.ts` rAF callback | -| `render:scene` | Scene drawing (picture replay OR live render) | `render()` | -| `render:recordPicture` | SkPicture recording | `recordScenePicture()` | -| `render:drawPicture` | SkPicture replay | `canvas.drawPicture()` | -| `render:volatile` | Live render (hover/drag/text edit) | volatile path | -| `render:sectionTitles` | Section title drawing | `drawSectionTitles()` | -| `render:componentLabels` | Component labels | `drawComponentLabels()` | -| `render:selection` | Selection borders/handles | `drawSelection()` | -| `render:rulers` | Ruler drawing | `drawRulers()` | -| `render:flush` | GPU command submission | `surface.flush()` | -| `layout:compute` | Yoga layout | `computeAllLayouts()` | - -Replace the existing `console.time`/`console.timeEnd` calls (which already exist in the render method) with the structured User Timing API: - -```ts -// Before (current) -console.time('render:flush') -this.surface.flush() -console.timeEnd('render:flush') - -// After (with DevTools custom tracks) -const start = performance.now() -this.surface.flush() -performance.measure('flush', { - start, - detail: { - devtools: { - dataType: 'track-entry', - track: 'Renderer', - trackGroup: 'OpenPencil', - color: 'tertiary' - } - } -}) -``` - -### 3. GPU Timer Queries - -Wrap the WebGL context to insert `EXT_disjoint_timer_query_webgl2` queries around the flush boundary: - -```ts -class GpuTimer { - private ext: EXT_disjoint_timer_query_webgl2 | null - private pending: WebGLQuery[] - private results: number[] // rolling buffer of GPU times in ms - - beginFrame(gl: WebGL2RenderingContext) { ... } - endFrame(gl: WebGL2RenderingContext) { ... } - pollResults(gl: WebGL2RenderingContext): number | null { ... } -} -``` - -Since results come back asynchronously (typically 1-4 frames later), the HUD shows the most recently completed GPU measurement, not the current frame's. This is the same approach Unity and Unreal use. - -For accurate CPU vs GPU bottleneck detection (Figma's approach): -- `perceived_fps` = 1000 / rAF delta -- `cpu_fps` = 1000 / (JS render end − JS render start) -- `gpu_fps` = 1000 / GPU timer query result - -If `gpu_fps ≈ perceived_fps` and `cpu_fps >> perceived_fps` → **GPU-bound** -If `cpu_fps ≈ perceived_fps` and `gpu_fps >> perceived_fps` → **CPU-bound** - -### 4. Detailed Profiler (on-demand, higher overhead) - -Triggered via debug menu or keyboard shortcut. Captures detailed per-node data for a single frame or short burst: - -**Per-node metrics:** -- Time spent in `renderNode()` (cumulative, including children) -- Self time (excluding children) -- Number of draw calls generated -- Whether the node was culled -- Fill/stroke/effect complexity (number of paints, blur radii, etc.) - -**Implementation:** A profiling wrapper around `renderNode()` that pushes/pops a stack: - -```ts -private renderNodeProfiled(canvas, graph, nodeId, ...) { - const entry = this.profilerStack.push(nodeId) - entry.startTime = performance.now() - this.renderNode(canvas, graph, nodeId, ...) - entry.endTime = performance.now() - this.profilerStack.pop() -} -``` - -**Overdraw visualization:** Skia has a built-in overdraw mode (`SkOverdrawCanvas` / `OverdrawColorFilter`). In CanvasKit, we can achieve this by rendering with `CanvasKit.MakeMatrix` color filter that maps draw count → heat colors. Alternative: render each node's bounds with additive blend to accumulate overdraw count as brightness. - -**Output formats:** -- **In-canvas heatmap** — overlay showing per-pixel render cost (warm = slow) -- **Speedscope-compatible JSON** — export frame profile for viewing in https://www.speedscope.app/ (same format Figma's webgl-profiler uses) -- **Node table** — sorted list of most expensive nodes with self-time - -### 5. Draw Call Counter - -Proxy the WebGL context to count actual GL draw calls per frame: - -```ts -function instrumentGL(gl: WebGL2RenderingContext) { - let drawCalls = 0 - const origDrawArrays = gl.drawArrays.bind(gl) - const origDrawElements = gl.drawElements.bind(gl) - gl.drawArrays = (...args) => { drawCalls++; return origDrawArrays(...args) } - gl.drawElements = (...args) => { drawCalls++; return origDrawElements(...args) } - return { - getAndReset() { const c = drawCalls; drawCalls = 0; return c } - } -} -``` - -This is the standard technique used by Spector.js, WebGL Inspector, and every game engine's stat display. - ---- - -## Implementation plan - -### Phase 1: Foundation (`packages/core/src/profiler.ts`) - -Create a `RenderProfiler` class that all layers use: - -```ts -export class RenderProfiler { - enabled = false - hudVisible = false - - // Frame stats (always tracked when enabled, near-zero cost) - readonly frameStats: FrameStats - - // Phase timing (User Timing API integration) - beginPhase(name: string): void - endPhase(name: string): void - - // GPU timer (optional, requires extension) - readonly gpuTimer: GpuTimer | null - - // Draw call counter - readonly drawCallCounter: DrawCallCounter | null - - // Detailed per-node profiling (on-demand) - beginDetailedCapture(): void - endDetailedCapture(): FrameCapture - - // HUD rendering (called by SkiaRenderer) - drawHUD(canvas: Canvas, ck: CanvasKit): void -} -``` - -### Phase 2: Integrate into renderer - -1. Add `profiler: RenderProfiler` to `SkiaRenderer` -2. Replace all `console.time`/`console.timeEnd` with `profiler.beginPhase`/`endPhase` -3. Add phase markers for every render stage -4. Wrap `surface.flush()` with GPU timing queries -5. Add HUD drawing at end of `render()` (in screen space, after everything else) - -### Phase 3: HUD overlay - -Render directly on the Skia canvas as the very last step (after rulers). Uses small monospace font, semi-transparent dark background. Positioned top-left or bottom-left, not interfering with rulers. - -### Phase 4: GPU timing & bottleneck detection - -1. Implement `GpuTimer` class with `EXT_disjoint_timer_query_webgl2` -2. Add CPU vs GPU bottleneck indicator to HUD -3. Handle graceful degradation when extension unavailable - -### Phase 5: Detailed profiler & export - -1. Per-node timing capture -2. Speedscope JSON export -3. Overdraw visualization mode -4. Node cost table (could display in dev panel or console) - ---- - -## File structure - -``` -packages/core/src/ - profiler/ - index.ts — re-exports - render-profiler.ts — main RenderProfiler class - frame-stats.ts — FPS/frame-time tracking with rolling averages - gpu-timer.ts — EXT_disjoint_timer_query wrapper - draw-call-counter.ts — WebGL proxy for draw call counting - phase-timer.ts — User Timing API / console.timeStamp integration - hud-renderer.ts — in-canvas HUD overlay drawing - frame-capture.ts — detailed per-node capture data structures - speedscope-export.ts — export to speedscope JSON format -``` - -## Activation - -| Mechanism | What activates | -|-----------|---------------| -| `?profiling` URL param | Use profiling CanvasKit build with full WASM names | -| `Shift+P` keyboard shortcut | Toggle HUD overlay | -| `Shift+Alt+P` | Start/stop detailed frame capture | -| Dev menu → "Export frame profile" | Save speedscope JSON | -| Dev menu → "Show overdraw" | Toggle overdraw heatmap | -| `store.debug.profiler` | Programmatic control from Vue devtools | - -## Key design principles - -1. **Zero cost when off** — no allocations, no timing calls, no proxy overhead when profiler is disabled -2. **Minimal cost when HUD-only** — < 0.2ms overhead for the basic stats display -3. **Render on canvas, not DOM** — HUD is drawn by Skia itself, no DOM overlays that could interfere with WebGL -4. **Async GPU results** — never stall the pipeline waiting for GPU query results; show N-2 frame's GPU time -5. **Production-safe** — the basic HUD and User Timing marks can ship in production behind a flag -6. **DevTools-native** — leverage Chrome's Performance panel extensibility for zero-overhead external profiling -7. **Export-friendly** — speedscope JSON format is the standard for flame chart sharing - -## References - -- [Skia Tracing](https://skia.org/docs/dev/tools/tracing) — Skia's internal `TRACE_EVENT` macros, Perfetto integration -- [Skia Debugger](https://debugger.skia.org) — SKP visual debugger with GPU op bounds -- [Skia Perf](https://skia.org/docs/dev/testing/skiaperf) — Skia's continuous perf monitoring with 400k+ metrics per commit -- [CanvasKit profiling build](https://www.npmjs.com/package/canvaskit-wasm) — `canvaskit-wasm/profiling` with full WASM function names -- [Figma webgl-profiler](https://github.com/figma/webgl-profiler) — Figma's GPU profiler using EXT_disjoint_timer_query, speedscope output -- [Skia render-skp.html](https://skia.googlesource.com/skia/+/052566d8ccb7/tools/perf-canvaskit-puppeteer/render-skp.html) — Skia's own CanvasKit benchmark: frame-to-frame timing methodology -- [EXT_disjoint_timer_query_webgl2 spec](https://registry.khronos.org/webgl/extensions/EXT_disjoint_timer_query_webgl2/) — WebGL GPU timing extension -- [Chrome Performance extensibility API](https://developer.chrome.com/docs/devtools/performance/extension) — Custom tracks via `performance.measure` detail.devtools -- [Unity Frame Timing Manager](https://docs.unity3d.com/2022.3/Documentation/Manual/frame-timing-manager.html) — CPU main thread / render thread / GPU split -- [Unreal Insights](https://dev.epicgames.com/documentation/en-us/unreal-engine/introduction-to-performance-profiling-and-configuration-in-unreal-engine) — Per-thread, per-GPU frame profiling -- Skia Milestone 132-133 release notes — `GrGLInterface` timer query support, `GpuStats.elapsedTime` for GPU time reporting diff --git a/packages/docs/.vitepress/sidebars.ts b/packages/docs/.vitepress/sidebars.ts index 1fea4c2b1..bca0e9fe6 100644 --- a/packages/docs/.vitepress/sidebars.ts +++ b/packages/docs/.vitepress/sidebars.ts @@ -92,6 +92,8 @@ export const developmentSidebar = (prefix: string, label: string): DefaultTheme. items: [ { text: 'Contributing', link: `${prefix}/development/contributing` }, { text: 'Testing', link: `${prefix}/development/testing` }, + { text: 'Renderer Profiler', link: `${prefix}/development/renderer-profiler` }, + { text: 'Variables UI Roadmap', link: `${prefix}/development/variables-ui-roadmap` }, { text: 'OpenSpec', link: `${prefix}/development/openspec` }, { text: 'Roadmap', link: `${prefix}/development/roadmap` }, ], diff --git a/packages/docs/development/renderer-profiler.md b/packages/docs/development/renderer-profiler.md new file mode 100644 index 000000000..51ab485f5 --- /dev/null +++ b/packages/docs/development/renderer-profiler.md @@ -0,0 +1,95 @@ +--- +title: Renderer Profiler +description: Use the CanvasKit renderer profiler HUD and frame capture tools to investigate rendering performance. +--- + +# Renderer Profiler + +OpenPencil includes a CanvasKit renderer profiler for debugging frame time, GPU timing, draw calls, cache behavior, and expensive render phases. + +## Enable the HUD + +In the browser app, open the menu and choose: + +```txt +View → Profiler +``` + +The app toggles `store.toggleProfiler()`, which maps to `editor.renderer.profiler.toggle()`. + +The HUD is drawn directly on the Skia canvas so it measures the same rendering path as the document. It is not a DOM overlay. + +## HUD metrics + +The profiler HUD shows: + +- **FPS / frame time** — smoothed frame cadence. +- **CPU** — JavaScript/WASM render time for the frame. +- **GPU** — latest available `EXT_disjoint_timer_query_webgl2` result when the browser exposes it. +- **Nodes / culled nodes** — total visible scene work and viewport culling count. +- **Draws** — WebGL draw calls counted through the instrumented context. +- **Cache** — whether the scene picture cache was reused. +- **Phases** — timings for renderer phases such as scene draw, picture replay/record, volatile overlays, section labels, selection, rulers, and flush. +- **Frame graph** — rolling frame history with 60 fps / 30 fps / slow thresholds and GPU bars when available. + +GPU timing is asynchronous. The value shown is the latest completed GPU query, not necessarily the current frame. + +## Implementation locations + +Core profiler code lives in: + +```txt +packages/core/src/profiler/ +``` + +Main entry points: + +- `render-profiler.ts` — `RenderProfiler` facade used by `SkiaRenderer`. +- `frame/stats.ts` — rolling frame statistics. +- `gpu-timer.ts` — WebGL timer query wrapper. +- `draw-call-counter.ts` — WebGL draw-call instrumentation. +- `phase-timer.ts` — phase timing and User Timing integration. +- `hud-renderer.ts` — canvas HUD rendering. +- `frame/capture.ts` and `speedscope-export.ts` — detailed capture and Speedscope export. + +Renderer integration lives under: + +```txt +packages/core/src/canvas/renderer*.ts +packages/core/src/canvas/renderer/ +``` + +App wiring lives in: + +```txt +src/app/editor/profiler/index.ts +src/app/shell/menu/schema.ts +src/app/shell/menu/app-menu.ts +``` + +## Programmatic use + +From app/editor code: + +```ts +store.toggleProfiler() +``` + +From a renderer instance: + +```ts +renderer.profiler.toggle() +renderer.profiler.beginCapture() +// render one or more frames +const capture = renderer.profiler.endCapture() +const speedscopeJson = renderer.profiler.exportSpeedscope() +renderer.profiler.downloadSpeedscope() +``` + +Detailed captures are for targeted debugging. Keep the normal HUD path lightweight and avoid enabling expensive capture work unless a user or developer explicitly asks for it. + +## Notes + +- The profiler is designed to be safe when disabled: no timing calls or allocations should be added to hot paths unless `profiler.enabled` / `profiler.capturing` is active. +- GPU timing depends on browser and hardware support for `EXT_disjoint_timer_query_webgl2`. +- If GPU timing is unavailable, the HUD still reports CPU time, draw calls, phases, node counts, and cache status. diff --git a/docs/development/variables-ui-roadmap.md b/packages/docs/development/variables-ui-roadmap.md similarity index 100% rename from docs/development/variables-ui-roadmap.md rename to packages/docs/development/variables-ui-roadmap.md diff --git a/packages/docs/programmable/cli/scripting.md b/packages/docs/programmable/cli/scripting.md index 8c668bc5f..6182bc97d 100644 --- a/packages/docs/programmable/cli/scripting.md +++ b/packages/docs/programmable/cli/scripting.md @@ -1,70 +1,179 @@ --- title: Scripting -description: Execute JavaScript with the Figma Plugin API — query nodes, batch-modify designs, create frames. +description: Execute JavaScript with a Figma-compatible Plugin API to query, batch-modify, and generate designs. --- # Scripting -`open-pencil eval` gives you the full Figma Plugin API in the terminal. Read nodes, modify properties, create shapes — then write changes back to the file. +`open-pencil eval` runs JavaScript against an OpenPencil document with a Figma-compatible `figma` global. Use it for headless batch edits, inspection, fixture setup, and automation without opening the editor UI. -## Basic Usage +## Basic usage ```sh -open-pencil eval design.fig -c "figma.currentPage.children.length" +open-pencil eval design.fig -c "return figma.currentPage.children.length" ``` -The `-c` flag takes JavaScript. The `figma` global works like the Figma Plugin API. - -## Query Nodes +The `-c` flag accepts JavaScript. If the code does not start with `return`, OpenPencil wraps it in an async function and returns the value from that function when present. ```sh open-pencil eval design.fig -c " - figma.currentPage.findAll(n => n.type === 'FRAME' && n.name.includes('Button')) - .map(b => ({ id: b.id, name: b.name, w: b.width, h: b.height })) + const frame = figma.createFrame() + frame.name = 'Card' + frame.resize(300, 200) + frame.layoutMode = 'VERTICAL' + frame.itemSpacing = 12 + return { id: frame.id, name: frame.name } " ``` -## Modify and Save +## Query nodes ```sh open-pencil eval design.fig -c " - figma.currentPage.children.forEach(n => n.opacity = 0.5) -" -w + return figma.currentPage + .findAll((node) => node.type === 'FRAME' && node.name.includes('Button')) + .map((button) => ({ + id: button.id, + name: button.name, + width: button.width, + height: button.height + })) +" ``` -`-w` writes changes back to the input file. Use `-o output.fig` to write to a different file instead. +## Modify and save -## Read from Stdin - -For longer scripts: +Use `--write` / `-w` to write changes back to the input file: ```sh -cat transform.js | open-pencil eval design.fig --stdin -w +open-pencil eval design.fig -c " + figma.currentPage.children.forEach((node) => { + node.opacity = 0.5 + }) +" --write ``` -## Live App Mode - -Omit the file to run against the running desktop app: +Use `--output` / `-o` to write to a new file: ```sh -open-pencil eval -c "figma.currentPage.name" +open-pencil eval design.fig -c "figma.currentPage.name = 'Updated'" -o updated.fig ``` -## Available API - -The `figma` object supports: - -- `figma.currentPage` — the active page -- `figma.root` — the document root -- `figma.createFrame()`, `figma.createRectangle()`, `figma.createEllipse()`, `figma.createText()`, etc. -- `.findAll()`, `.findOne()` — search descendants -- `.appendChild()`, `.insertChild()` — tree manipulation -- All property setters: `.fills`, `.strokes`, `.effects`, `.opacity`, `.cornerRadius`, `.layoutMode`, `.itemSpacing`, etc. - -This is the same API Figma plugins use, so existing knowledge and code snippets transfer directly. - -## JSON Output +## Read scripts from stdin ```sh -open-pencil eval design.fig -c "..." --json +cat transform.js | open-pencil eval design.fig --stdin --write ``` + +## Live app mode + +Omit the file path to run against the currently open document in the desktop app: + +```sh +open-pencil eval -c "return figma.currentPage.name" +``` + +The desktop app must be running with a document open. + +## Output + +By default, non-TTY output is JSON. Use `--json` to force JSON output: + +```sh +open-pencil eval design.fig -c "return figma.currentPage.children.map((n) => n.name)" --json +``` + +Use `--quiet` / `-q` to suppress output when only writing a file. + +## Supported API surface + +The API is intentionally close to Figma's Plugin API, but it maps to OpenPencil's scene graph and file format. + +### Document and pages + +- `figma.root` +- `figma.currentPage` +- `figma.currentPage.selection` +- `figma.getNodeById(id)` +- `figma.createPage()` + +### Node creation + +- `figma.createFrame()` +- `figma.createRectangle()` +- `figma.createEllipse()` +- `figma.createText()` +- `figma.createLine()` +- `figma.createPolygon()` +- `figma.createStar()` +- `figma.createVector()` +- `figma.createComponent()` +- `figma.createSection()` + +### Tree operations + +- `node.children` +- `node.parent` +- `node.appendChild(child)` +- `node.insertChild(index, child)` +- `node.clone()` +- `node.remove()` +- `node.findAll(callback?)` +- `node.findOne(callback)` +- `node.findChild(callback)` +- `node.findChildren(callback?)` +- `figma.group(nodes, parent)` +- `figma.ungroup(node)` + +### Components + +- `figma.createComponentFromNode(node)` +- `component.createInstance()` +- `instance.mainComponent` + +### Variables + +- `figma.getLocalVariables(type?)` +- `figma.getVariableById(id)` +- `figma.getLocalVariableCollections()` +- `figma.getVariableCollectionById(id)` +- `figma.createVariable(name, type, collectionId, value?)` +- `figma.setVariableValue(variableId, modeId, value)` +- `figma.deleteVariable(id)` +- `figma.createVariableCollection(name)` +- `figma.deleteVariableCollection(id)` +- `figma.bindVariable(nodeId, field, variableId)` +- `figma.unbindVariable(nodeId, field)` + +### Properties + +Common node properties are readable/writable through the proxy, including: + +- Geometry: `x`, `y`, `width`, `height`, `rotation`, `resize(width, height)` +- Appearance: `fills`, `strokes`, `effects`, `opacity`, `visible`, `locked`, `blendMode`, `clipsContent` +- Radius: `cornerRadius`, `topLeftRadius`, `topRightRadius`, `bottomRightRadius`, `bottomLeftRadius` +- Text: `characters`, `fontSize`, `fontName`, `fontWeight`, alignment, line height, letter spacing, style-run helpers +- Auto-layout: `layoutMode`, `primaryAxisAlignItems`, `counterAxisAlignItems`, `itemSpacing`, padding, sizing, and layout positioning fields +- Stroke helpers: `strokeWeight`, `strokeAlign`, `dashPattern` + +### Utilities + +- `figma.mixed` +- `figma.createImage(data)` +- `figma.loadFontAsync(fontName)` no-ops because OpenPencil does not gate text edits on plugin font loading +- `figma.listAvailableFontsAsync()` returns host-provided fonts when available +- `figma.notify(message)` logs a warning in headless mode +- `figma.viewport` + +## Not yet Figma-compatible + +These Figma APIs are not exposed as compatible helpers yet: + +- `node.exportAsync()` +- `node.setBoundVariable(field, variable)` +- `node.detachInstance()` +- `figma.combineAsVariants(components, parent)` +- Figma style APIs such as `figma.createPaintStyle()` / `figma.createTextStyle()` +- Full vector boolean operation parity + +Use OpenPencil CLI export commands, core tools, or direct scene-graph helpers where available.