docs: move development notes into docs package

- Move variables roadmap into the VitePress development docs
- Replace the stale renderer profiler plan with current usage notes
- Refresh eval scripting docs with the supported API surface
This commit is contained in:
Danila Poyarkov 2026-05-13 02:48:09 +03:00
parent 858a8a2566
commit bfd364909e
7 changed files with 243 additions and 821 deletions

View file

@ -2,7 +2,7 @@
Vue 3 + CanvasKit (Skia WASM) + Yoga WASM design editor. Tauri v2 desktop, also runs in browser.
**Roadmap:** `docs/development/variables-ui-roadmap.md` tracks remaining variable-system UI work. Current architecture and commands live in this file.
**Roadmap:** `packages/docs/development/variables-ui-roadmap.md` tracks remaining variable-system UI work. Current architecture and commands live in this file.
## Monorepo

View file

@ -1,432 +0,0 @@
# `open-pencil eval` — Figma-like Plugin API for Headless Scripting
## Overview
`bun open-pencil eval <file> --code '<js>'` executes JavaScript against a `.fig` file with a Figma-compatible `figma` global object. This enables headless scripting, batch operations, AI tool execution, and testing — all without the GUI.
The `figma` object mirrors Figma's Plugin API surface as closely as possible, so existing Figma plugin knowledge and code snippets transfer directly.
```bash
# Create a frame, set auto-layout, add children
bun open-pencil eval design.fig --code '
const frame = figma.createFrame()
frame.name = "Card"
frame.resize(300, 200)
frame.layoutMode = "VERTICAL"
frame.itemSpacing = 12
frame.paddingTop = frame.paddingBottom = 16
frame.paddingLeft = frame.paddingRight = 16
frame.fills = [{ type: "SOLID", color: { r: 1, g: 1, b: 1 } }]
const title = figma.createText()
title.characters = "Hello World"
title.fontSize = 24
frame.appendChild(title)
return { id: frame.id, name: frame.name }
'
# Query nodes
bun open-pencil eval design.fig --code '
const buttons = figma.currentPage.findAll(n => n.type === "FRAME" && n.name.includes("Button"))
return buttons.map(b => ({ id: b.id, name: b.name, w: b.width, h: b.height }))
'
# Read from stdin (for multiline scripts / piping)
cat transform.js | bun open-pencil eval design.fig --stdin
# Write changes back
bun open-pencil eval design.fig --code '...' --write
bun open-pencil eval design.fig --code '...' -o modified.fig
```
## Architecture
```
┌──────────────────────────────────────────────────────┐
│ CLI: `open-pencil eval <file> --code '...'`
│ ↓ │
│ loadDocument(file) → SceneGraph │
│ ↓ │
│ FigmaAPI(sceneGraph) → `figma` proxy object │
│ ↓ │
│ AsyncFunction('figma', wrappedCode)(figmaProxy) │
│ ↓ │
│ print result as JSON / agentfmt │
│ optionally: saveDocument(file) if --write │
└──────────────────────────────────────────────────────┘
```
### Key classes
| Class | Location | Role |
|-------|----------|------|
| `FigmaAPI` | `packages/core/src/figma-api/` | Proxy object implementing `figma.*` methods against `SceneGraph` |
| `FigmaNode` | `packages/core/src/figma-api/` | Proxy wrapping `SceneNode` with Figma-style property access (`.fills`, `.resize()`, `.appendChild()`, etc.) |
| `eval` command | `packages/cli/src/commands/eval.ts` | CLI command that loads doc, creates API, executes code |
### Why in `@open-pencil/core`?
The `FigmaAPI` class lives in core (not CLI) because:
- **AI tools reuse it** — the chat panel's `render` tool can execute JSX through the same API
- **Test scripts** — unit tests can use the API to set up fixtures
- **No DOM deps** — runs headless in Bun, no browser APIs needed
## `FigmaAPI` — Phased Implementation
### Phase 1: Core (MVP for eval command)
These cover ~80% of real plugin scripts:
#### Document & Page
| Figma API | Our implementation | Notes |
|-----------|--------------------|-------|
| `figma.root` | Getter → proxy for root node | `.children` returns page proxies |
| `figma.currentPage` | Getter/setter → first page by default | Settable to any page proxy |
| `figma.currentPage.selection` | Get/set → tracked selection array | |
| `figma.getNodeById(id)` | `graph.getNode(id)` wrapped in proxy | Sync, like Figma's deprecated version |
#### Node Creation
| Figma API | Maps to |
|-----------|---------|
| `figma.createFrame()` | `graph.createNode('FRAME', currentPageId)` |
| `figma.createRectangle()` | `graph.createNode('RECTANGLE', ...)` |
| `figma.createEllipse()` | `graph.createNode('ELLIPSE', ...)` |
| `figma.createText()` | `graph.createNode('TEXT', ...)` |
| `figma.createLine()` | `graph.createNode('LINE', ...)` |
| `figma.createPolygon()` | `graph.createNode('POLYGON', ...)` |
| `figma.createStar()` | `graph.createNode('STAR', ...)` |
| `figma.createComponent()` | `graph.createNode('COMPONENT', ...)` |
| `figma.createPage()` | `graph.addPage(name)` |
| `figma.createSection()` | `graph.createNode('SECTION', ...)` |
#### Node Properties (via `FigmaNode` proxy)
Read/write on any node proxy. Property access maps to `SceneNode` fields:
```ts
// Geometry
node.x, node.y // direct
node.width, node.height // read-only, use node.resize(w, h)
node.rotation // direct
node.resize(w, h) // updates width + height
node.resizeWithoutConstraints(w, h) // same (no constraint engine yet)
// Visual
node.fills // get/set Fill[]
node.strokes // get/set Stroke[]
node.effects // get/set Effect[]
node.opacity // get/set number
node.visible // get/set boolean
node.locked // get/set boolean
node.blendMode // get/set BlendMode
node.clipsContent // get/set boolean
// Corner radius
node.cornerRadius // get/set (number or figma.mixed)
node.topLeftRadius // get/set
node.topRightRadius // get/set
node.bottomLeftRadius // get/set
node.bottomRightRadius // get/set
node.cornerSmoothing // get/set
// Identity
node.id // read-only
node.name // get/set
node.type // read-only
node.parent // read-only → FigmaNode | null
node.removed // read-only boolean
```
#### Tree Operations
```ts
node.children // read-only FigmaNode[]
node.appendChild(child) // reparent to end
node.insertChild(index, child) // reparent at index
node.remove() // graph.deleteNode(id)
// Traversal
node.findAll(callback?) // recursive find
node.findOne(callback) // first match
node.findChild(callback) // direct children only
node.findChildren(callback?) // direct children only
```
#### Auto-layout
```ts
node.layoutMode // 'NONE' | 'HORIZONTAL' | 'VERTICAL'
node.primaryAxisAlignItems // 'MIN' | 'CENTER' | 'MAX' | 'SPACE_BETWEEN'
node.counterAxisAlignItems // 'MIN' | 'CENTER' | 'MAX' | 'BASELINE'
node.itemSpacing // number
node.counterAxisSpacing // number | null
node.paddingTop / Right / Bottom / Left // number
node.layoutWrap // 'NO_WRAP' | 'WRAP'
// Child sizing
node.layoutPositioning // 'AUTO' | 'ABSOLUTE'
node.layoutGrow // 0 | 1
node.layoutSizingHorizontal // 'FIXED' | 'HUG' | 'FILL'
node.layoutSizingVertical // 'FIXED' | 'HUG' | 'FILL'
```
#### Text
```ts
node.characters // get/set (maps to node.text)
node.fontSize // get/set
node.fontName // get/set { family, style }
node.fontWeight // get/set
node.textAlignHorizontal // get/set
node.textAlignVertical // get/set
node.textAutoResize // get/set
node.letterSpacing // get/set
node.lineHeight // get/set
node.maxLines // get/set
node.textCase // get/set
node.textDecoration // get/set
```
#### Stroke details
```ts
node.strokeWeight // get/set (maps to strokes[0].weight)
node.strokeAlign // get/set (maps to strokes[0].align)
node.dashPattern // get/set
```
#### Misc
```ts
figma.mixed // Symbol sentinel for mixed values
figma.group(nodes, parent) // creates GROUP with given children
figma.ungroup(node) // ungroups, reparents children
figma.flatten(nodes) // NOT IMPLEMENTED YET — returns first node
```
#### Export
```ts
node.exportAsync(settings?) // only works if CanvasKit is loaded
// settings: { format: 'PNG'|'JPG'|'SVG', constraint? }
```
### Phase 2: Components & Instances
| API | Maps to |
|-----|---------|
| `figma.createComponent()` | `graph.createNode('COMPONENT', ...)` |
| `figma.createComponentFromNode(node)` | Convert existing frame to component |
| `figma.combineAsVariants(components, parent)` | Create COMPONENT_SET |
| Node: `node.createInstance()` | `graph.createInstance(componentId, parentId)` |
| Node: `node.detachInstance()` | `graph.detachInstance(id)` |
| `figma.getNodeById(id).mainComponent` | `graph.getMainComponent(id)` |
### Phase 3: Variables
| API | Maps to |
|-----|---------|
| `figma.variables.getLocalVariables(type?)` | `graph.variables` filtered |
| `figma.variables.getLocalVariableCollections()` | `graph.variableCollections` |
| `figma.variables.createVariable(name, collection, type)` | `graph.addVariable(...)` |
| `figma.variables.createVariableCollection(name)` | `graph.addCollection(...)` |
| `figma.variables.getVariableById(id)` | `graph.variables.get(id)` |
| `node.setBoundVariable(field, variable)` | `graph.bindVariable(...)` |
| `node.boundVariables` | getter from SceneNode |
### Phase 4: Styles & Advanced
| API | Notes |
|-----|-------|
| `figma.createPaintStyle()` | Requires style storage in SceneGraph |
| `figma.createTextStyle()` | Requires style storage in SceneGraph |
| `figma.createEffectStyle()` | Requires style storage in SceneGraph |
| `figma.loadFontAsync(fontName)` | No-op (we don't have font loading constraints) |
| `figma.listAvailableFontsAsync()` | Return system fonts if available |
| Boolean operations (`union`, `subtract`, `intersect`, `exclude`) | Requires path boolean engine |
| `figma.createNodeFromJSXAsync(jsx)` | Port figma-use's JSX renderer |
## `FigmaNode` Proxy Design
The proxy wraps a `SceneNode` and translates Figma property names to our internal names. Key mappings:
```ts
const PROPERTY_MAP: Record<string, string> = {
// Figma name → SceneNode field name (only where they differ)
'characters': 'text',
'strokeWeight': → computed from strokes[0].weight,
'strokeAlign': → computed from strokes[0].align,
'fontName': → computed from { family: fontFamily, style: ... },
'primaryAxisAlignItems': 'primaryAxisAlign',
'counterAxisAlignItems': 'counterAxisAlign',
'primaryAxisSizingMode': 'primaryAxisSizing', // value mapping: 'AUTO' → 'HUG', 'FIXED' → 'FIXED'
'counterAxisSizingMode': 'counterAxisSizing',
'layoutSizingHorizontal': → computed from primaryAxisSizing / counterAxisSizing depending on layoutMode
'layoutSizingVertical': → computed
}
```
Methods on the proxy:
```ts
class FigmaNode {
// The proxy is created via: new Proxy(target, handler)
// where handler.get intercepts property reads and handler.set intercepts writes
resize(width: number, height: number): void
resizeWithoutConstraints(width: number, height: number): void
remove(): void
appendChild(child: FigmaNode): void
insertChild(index: number, child: FigmaNode): void
findAll(callback?: (node: FigmaNode) => boolean): FigmaNode[]
findOne(callback: (node: FigmaNode) => boolean): FigmaNode | null
findChild(callback: (node: FigmaNode) => boolean): FigmaNode | null
findChildren(callback?: (node: FigmaNode) => boolean): FigmaNode[]
exportAsync(settings?: ExportSettings): Promise<Uint8Array>
// Components (Phase 2)
createInstance(): FigmaNode
detachInstance(): void
get mainComponent(): FigmaNode | null
}
```
## CLI Command
```
bun open-pencil eval <file> [options]
Arguments:
file .fig file to operate on
Options:
--code, -c JavaScript code to execute (has access to `figma` global)
--stdin Read code from stdin instead of --code
--write, -w Write changes back to the input file
-o, --output Write to a different file
--json Output result as JSON (default for non-TTY)
--quiet, -q Suppress output, only write file
```
### Execution model
1. Load `.fig``SceneGraph`
2. Create `FigmaAPI(graph)``figma` proxy
3. Wrap user code in async function: `return (async () => { <code> })()`
4. Execute with `figma` as sole argument
5. Print return value (JSON or agentfmt)
6. If `--write` or `-o`: serialize `SceneGraph` back to `.fig`
### Return value formatting
- `undefined` / `void` → no output
- Primitives → printed directly
- Objects/arrays → `JSON.stringify(result, null, 2)` or agentfmt tables
- `FigmaNode` → serialized as `{ id, type, name, x, y, width, height, fills, ... }`
- Arrays of `FigmaNode` → serialized as list
## Shared with AI Tools
The `FigmaAPI` class is the **same API surface** that AI tools use. App AI tools are wired from `src/app/ai/tools/index.ts` through the shared core tool definitions, so CLI scripts, MCP calls, and chat tool calls execute the same scene-graph operations:
```ts
// Tool implementation using FigmaAPI
execute: async ({ type, x, y, width, height }) => {
const frame = figma.createFrame()
frame.resize(width, height)
frame.x = x
frame.y = y
return { id: frame.id }
}
```
This ensures CLI scripts and AI tools behave identically.
## File Layout
```
packages/core/src/
figma-api/ # FigmaAPI class + FigmaNode proxy
tools/ # Shared ToolDef operations used by AI, MCP, and eval helpers
packages/cli/src/commands/
eval.ts # CLI command
src/app/ai/tools/
index.ts # App chat wiring for shared core tool definitions
```
## Test Plan
### Unit tests (`packages/core/src/figma-api.test.ts`)
1. **Node creation** — each `createX()` creates correct type, added to current page
2. **Property access**`.fills`, `.x`, `.width`, `.name`, `.characters` read/write correctly
3. **Resize**`.resize(w, h)` updates width/height
4. **Tree operations**`.appendChild()`, `.insertChild()`, `.remove()`, `.parent`, `.children`
5. **Traversal**`.findAll()`, `.findOne()`, `.findChild()`, `.findChildren()` with callbacks
6. **Auto-layout**`.layoutMode`, `.itemSpacing`, `.paddingTop`, etc.
7. **Text**`.characters` maps to `.text`, `.fontName` maps to `{ family, style }`
8. **Mixed values**`.cornerRadius` returns `figma.mixed` when corners differ
9. **Selection**`figma.currentPage.selection` get/set
10. **Page switching**`figma.currentPage = page2` works
11. **Group/ungroup**`figma.group()` creates group, `figma.ungroup()` dissolves it
12. **Clone** — node creation produces independent copies
### CLI integration tests (`packages/cli/src/commands/eval.test.ts`)
1. **Basic eval**`eval test.fig --code 'return figma.currentPage.name'` → page name
2. **Create + read** — create a frame, return its properties
3. **Query nodes**`findAll` returns correct nodes
4. **Write back**`--write` saves changes, reloading shows them
5. **Stdin**`echo 'return 42' | bun open-pencil eval test.fig --stdin``42`
6. **JSON output**`--json` returns valid JSON
7. **Error handling** — syntax errors, runtime errors reported cleanly
## Implementation Order
1. **`FigmaNode` proxy** — property mapping, `.resize()`, `.remove()`, tree methods
2. **`FigmaAPI` class** — `createFrame/Rectangle/...`, `.root`, `.currentPage`, `.getNodeById()`, `.mixed`, `.group()`
3. **CLI `eval` command** — argument parsing, code wrapping, output formatting
4. **Unit tests** — all 12 test groups above
5. **CLI integration tests** — all 7 test groups above
6. **Wire to AI tools** — expose shared core tool definitions through `src/app/ai/tools/index.ts`
7. **Phase 2** — components & instances
8. **Phase 3** — variables
9. **Phase 4** — styles, boolean ops, JSX renderer
## Property Mapping Reference
| Figma Property | SceneNode Field | Type | Notes |
|---------------|-----------------|------|-------|
| `characters` | `text` | `string` | |
| `fontName` | `fontFamily` + `fontWeight` + `italic` | `{ family, style }` | Computed: `style` = "Bold Italic" etc. |
| `strokeWeight` | `strokes[0].weight` | `number` | Computed |
| `strokeAlign` | `strokes[0].align` | `string` | Computed |
| `primaryAxisAlignItems` | `primaryAxisAlign` | `string` | |
| `counterAxisAlignItems` | `counterAxisAlign` | `string` | |
| `layoutSizingHorizontal` | `primaryAxisSizing` or `counterAxisSizing` | `string` | Depends on `layoutMode` |
| `layoutSizingVertical` | (opposite of horizontal) | `string` | |
| `absoluteTransform` | computed from `x`, `y`, `rotation` | `Transform` | Read-only |
| `absoluteBoundingBox` | `getAbsoluteBounds(id)` | `Rect` | Read-only |
| All others | Same name | Same type | Direct passthrough |
## Open Questions
1. **Font loading**: `figma.loadFontAsync()` — should it be a no-op (we don't have font gating) or should we track loaded fonts?
**Decision: No-op that returns resolved Promise.** We don't gate text editing on font loading.
2. **Export in headless mode**: `node.exportAsync()` requires CanvasKit. Should eval load CanvasKit?
**Decision: Optional.** If CanvasKit is available (via `--with-canvaskit` flag or env), enable export. Otherwise, throw "Export requires CanvasKit" error.
3. **`figma.mixed` symbol**: Should we use the actual Figma symbol or our own?
**Decision: Our own `Symbol('mixed')`.** Exposed as `figma.mixed`.
4. **Undo**: `figma.commitUndo()` / `figma.triggerUndo()` — relevant in headless?
**Decision: No-op in CLI.** Undo only matters in the live editor. The AI tools can add undo support separately via EditorStore.
5. **Write format**: Should `--write` produce `.fig` (Kiwi binary) or also support `.json`?
**Decision: `.fig` only for now.** JSON export is a separate feature.

View file

@ -1,352 +0,0 @@
# Renderer Profiler — Design Plan
Production-grade instrumentation for OpenPencil's CanvasKit/Skia renderer, inspired by game engine profiling tools (Unreal Insights, Unity Frame Timing Manager) and browser render engine internals (Chrome Compositor, WebKit/Skia tracing).
## What Skia gives us
### 1. Profiling build of CanvasKit
`canvaskit-wasm` ships three bundles:
- **default** — stripped WASM names
- **full** — includes Skottie etc.
- **profiling** — same as full but with **full internal function names** preserved in WASM
```ts
import InitCanvasKit from 'canvaskit-wasm/profiling'
```
This means Chrome DevTools Performance panel flame charts will show real Skia C++ function names (`GrDrawingManager::internalFlush`, `SkCanvas::drawRect`, etc.) instead of mangled `$func123`. We should use this build behind a `?profiling` URL param or a dev-mode toggle.
### 2. SkPicture recording & replay
Skia's `SkPictureRecorder` records draw commands into an immutable `SkPicture`. We already use this for caching (`recordScenePicture`). Key profiling insight from Skia's own benchmark tool:
> `CanvasKit.flush()` returns after it has sent all instructions to the GPU, but we don't know the GPU is done until the **next frame is requested**. Thus, we need to keep track of time **between frames** to accurately calculate draw time.
- `drawPicture + flush` duration tells us CPU-side WASM/JS time
- `total_frame_ms` (time between rAFs) tells us the real GPU-inclusive cost
- If `total_frame_ms ≈ with_flush_ms` → CPU-bound
- If `total_frame_ms >> with_flush_ms` → GPU-bound
### 3. Skia Debugger (external tool)
Skia has an online debugger at https://debugger.skia.org that can load `.skp` files and show:
- Draw command playback
- GPU op bounds visualization
- Overdraw visualization
- Clip/matrix state at any step
- GPU operation batching (colored op IDs)
We could add an "Export .skp" debug feature to save the current `scenePicture` for analysis.
### 4. EXT_disjoint_timer_query (GPU timing)
The `EXT_disjoint_timer_query_webgl2` WebGL extension provides **actual GPU-side timing** in nanoseconds. Figma built [figma/webgl-profiler](https://github.com/figma/webgl-profiler) on top of this exact extension.
**Limitations:**
- Only works in Desktop Chrome ≥ 70
- `TIMESTAMP_EXT` was removed for security; only `TIME_ELAPSED_EXT` (begin/end query) works
- Results are asynchronous (available next frame or later)
- Non-draw commands often report 0ns (but totals are accurate)
- Inconsistent on non-Apple-Silicon Intel GPUs
**Usage pattern:**
```ts
const ext = gl.getExtension('EXT_disjoint_timer_query_webgl2')
const query = gl.createQuery()
gl.beginQuery(ext.TIME_ELAPSED_EXT, query)
// ... draw calls ...
gl.endQuery(ext.TIME_ELAPSED_EXT)
// next frame: poll gl.getQueryParameter(query, gl.QUERY_RESULT_AVAILABLE)
```
### 5. Chrome Performance Panel extensibility
Chrome now supports custom tracks via `performance.mark`/`performance.measure` with `detail.devtools` metadata, and the lower-overhead `console.timeStamp` API:
```ts
performance.measure("Scene Render", {
start: startMark,
detail: {
devtools: {
dataType: "track-entry",
track: "OpenPencil Renderer",
trackGroup: "OpenPencil",
color: "primary",
properties: [["Nodes", "142"], ["Cached", "true"]],
tooltipText: "Full scene render with SkPicture cache hit"
}
}
})
```
This shows up as a dedicated "OpenPencil Renderer" track in the DevTools Performance panel alongside browser internals. Extremely powerful for production debugging.
---
## Architecture
### Layers of profiling (from light to heavy)
| Layer | Overhead | Always-on | What it measures |
|-------|----------|-----------|-----------------|
| **Frame Stats HUD** | ~0.1ms | Optional toggle | FPS, frame time, CPU/GPU split, node count, cache hits |
| **User Timing marks** | ~0.01ms each | Dev builds | Phase timings visible in DevTools Performance panel |
| **GPU Timer Queries** | ~0.2ms setup | On demand | Actual GPU execution time per frame (async readback) |
| **Detailed profiler** | ~1-3ms | Manual trigger | Per-node timings, draw call counts, overdraw heatmap |
### 1. Frame Stats HUD (always-available, minimal overhead)
An in-canvas overlay rendered by SkiaRenderer itself, togglable via keyboard shortcut (Shift+P or similar). Modeled after game engine stat overlays.
**Metrics displayed:**
```
FPS: 60 (16.7ms) ← smoothed rAF delta
CPU: 4.2ms ← JS/WASM time (start of render → after flush)
GPU: 8.1ms ← GPU time (async, from EXT_disjoint_timer_query or inferred)
─────────────────
Nodes: 342 (47 culled) ← total vs viewport-culled count
Draw calls: 128 ← WebGL draw call count
Cache: SkPicture HIT ← scene picture reuse status
Textures: 12 (48MB) ← image cache stats
WASM heap: 64MB ← CanvasKit memory
```
**Frame time graph** — rolling 120-frame history bar chart (like Unreal's `stat unit`), color-coded:
- Green: within budget (16.7ms for 60fps)
- Yellow: over budget but < 2× (16.733.3ms)
- Red: severe (> 33.3ms)
The graph draws CPU bars from bottom and GPU bars from top, making CPU/GPU overlap visible at a glance (exactly how Unity Frame Timing Manager visualizes it).
### 2. Phase instrumentation (User Timing API)
Instrument the `render()` method with `performance.mark`/`performance.measure` using the Chrome DevTools extensibility API to create a custom "OpenPencil" track group:
**Phases to instrument:**
| Phase | What | Where in code |
|-------|------|---------------|
| `frame` | Total rAF-to-rAF | `useCanvas.ts` rAF callback |
| `render:scene` | Scene drawing (picture replay OR live render) | `render()` |
| `render:recordPicture` | SkPicture recording | `recordScenePicture()` |
| `render:drawPicture` | SkPicture replay | `canvas.drawPicture()` |
| `render:volatile` | Live render (hover/drag/text edit) | volatile path |
| `render:sectionTitles` | Section title drawing | `drawSectionTitles()` |
| `render:componentLabels` | Component labels | `drawComponentLabels()` |
| `render:selection` | Selection borders/handles | `drawSelection()` |
| `render:rulers` | Ruler drawing | `drawRulers()` |
| `render:flush` | GPU command submission | `surface.flush()` |
| `layout:compute` | Yoga layout | `computeAllLayouts()` |
Replace the existing `console.time`/`console.timeEnd` calls (which already exist in the render method) with the structured User Timing API:
```ts
// Before (current)
console.time('render:flush')
this.surface.flush()
console.timeEnd('render:flush')
// After (with DevTools custom tracks)
const start = performance.now()
this.surface.flush()
performance.measure('flush', {
start,
detail: {
devtools: {
dataType: 'track-entry',
track: 'Renderer',
trackGroup: 'OpenPencil',
color: 'tertiary'
}
}
})
```
### 3. GPU Timer Queries
Wrap the WebGL context to insert `EXT_disjoint_timer_query_webgl2` queries around the flush boundary:
```ts
class GpuTimer {
private ext: EXT_disjoint_timer_query_webgl2 | null
private pending: WebGLQuery[]
private results: number[] // rolling buffer of GPU times in ms
beginFrame(gl: WebGL2RenderingContext) { ... }
endFrame(gl: WebGL2RenderingContext) { ... }
pollResults(gl: WebGL2RenderingContext): number | null { ... }
}
```
Since results come back asynchronously (typically 1-4 frames later), the HUD shows the most recently completed GPU measurement, not the current frame's. This is the same approach Unity and Unreal use.
For accurate CPU vs GPU bottleneck detection (Figma's approach):
- `perceived_fps` = 1000 / rAF delta
- `cpu_fps` = 1000 / (JS render end JS render start)
- `gpu_fps` = 1000 / GPU timer query result
If `gpu_fps ≈ perceived_fps` and `cpu_fps >> perceived_fps`**GPU-bound**
If `cpu_fps ≈ perceived_fps` and `gpu_fps >> perceived_fps` → **CPU-bound**
### 4. Detailed Profiler (on-demand, higher overhead)
Triggered via debug menu or keyboard shortcut. Captures detailed per-node data for a single frame or short burst:
**Per-node metrics:**
- Time spent in `renderNode()` (cumulative, including children)
- Self time (excluding children)
- Number of draw calls generated
- Whether the node was culled
- Fill/stroke/effect complexity (number of paints, blur radii, etc.)
**Implementation:** A profiling wrapper around `renderNode()` that pushes/pops a stack:
```ts
private renderNodeProfiled(canvas, graph, nodeId, ...) {
const entry = this.profilerStack.push(nodeId)
entry.startTime = performance.now()
this.renderNode(canvas, graph, nodeId, ...)
entry.endTime = performance.now()
this.profilerStack.pop()
}
```
**Overdraw visualization:** Skia has a built-in overdraw mode (`SkOverdrawCanvas` / `OverdrawColorFilter`). In CanvasKit, we can achieve this by rendering with `CanvasKit.MakeMatrix` color filter that maps draw count → heat colors. Alternative: render each node's bounds with additive blend to accumulate overdraw count as brightness.
**Output formats:**
- **In-canvas heatmap** — overlay showing per-pixel render cost (warm = slow)
- **Speedscope-compatible JSON** — export frame profile for viewing in https://www.speedscope.app/ (same format Figma's webgl-profiler uses)
- **Node table** — sorted list of most expensive nodes with self-time
### 5. Draw Call Counter
Proxy the WebGL context to count actual GL draw calls per frame:
```ts
function instrumentGL(gl: WebGL2RenderingContext) {
let drawCalls = 0
const origDrawArrays = gl.drawArrays.bind(gl)
const origDrawElements = gl.drawElements.bind(gl)
gl.drawArrays = (...args) => { drawCalls++; return origDrawArrays(...args) }
gl.drawElements = (...args) => { drawCalls++; return origDrawElements(...args) }
return {
getAndReset() { const c = drawCalls; drawCalls = 0; return c }
}
}
```
This is the standard technique used by Spector.js, WebGL Inspector, and every game engine's stat display.
---
## Implementation plan
### Phase 1: Foundation (`packages/core/src/profiler.ts`)
Create a `RenderProfiler` class that all layers use:
```ts
export class RenderProfiler {
enabled = false
hudVisible = false
// Frame stats (always tracked when enabled, near-zero cost)
readonly frameStats: FrameStats
// Phase timing (User Timing API integration)
beginPhase(name: string): void
endPhase(name: string): void
// GPU timer (optional, requires extension)
readonly gpuTimer: GpuTimer | null
// Draw call counter
readonly drawCallCounter: DrawCallCounter | null
// Detailed per-node profiling (on-demand)
beginDetailedCapture(): void
endDetailedCapture(): FrameCapture
// HUD rendering (called by SkiaRenderer)
drawHUD(canvas: Canvas, ck: CanvasKit): void
}
```
### Phase 2: Integrate into renderer
1. Add `profiler: RenderProfiler` to `SkiaRenderer`
2. Replace all `console.time`/`console.timeEnd` with `profiler.beginPhase`/`endPhase`
3. Add phase markers for every render stage
4. Wrap `surface.flush()` with GPU timing queries
5. Add HUD drawing at end of `render()` (in screen space, after everything else)
### Phase 3: HUD overlay
Render directly on the Skia canvas as the very last step (after rulers). Uses small monospace font, semi-transparent dark background. Positioned top-left or bottom-left, not interfering with rulers.
### Phase 4: GPU timing & bottleneck detection
1. Implement `GpuTimer` class with `EXT_disjoint_timer_query_webgl2`
2. Add CPU vs GPU bottleneck indicator to HUD
3. Handle graceful degradation when extension unavailable
### Phase 5: Detailed profiler & export
1. Per-node timing capture
2. Speedscope JSON export
3. Overdraw visualization mode
4. Node cost table (could display in dev panel or console)
---
## File structure
```
packages/core/src/
profiler/
index.ts — re-exports
render-profiler.ts — main RenderProfiler class
frame-stats.ts — FPS/frame-time tracking with rolling averages
gpu-timer.ts — EXT_disjoint_timer_query wrapper
draw-call-counter.ts — WebGL proxy for draw call counting
phase-timer.ts — User Timing API / console.timeStamp integration
hud-renderer.ts — in-canvas HUD overlay drawing
frame-capture.ts — detailed per-node capture data structures
speedscope-export.ts — export to speedscope JSON format
```
## Activation
| Mechanism | What activates |
|-----------|---------------|
| `?profiling` URL param | Use profiling CanvasKit build with full WASM names |
| `Shift+P` keyboard shortcut | Toggle HUD overlay |
| `Shift+Alt+P` | Start/stop detailed frame capture |
| Dev menu → "Export frame profile" | Save speedscope JSON |
| Dev menu → "Show overdraw" | Toggle overdraw heatmap |
| `store.debug.profiler` | Programmatic control from Vue devtools |
## Key design principles
1. **Zero cost when off** — no allocations, no timing calls, no proxy overhead when profiler is disabled
2. **Minimal cost when HUD-only**< 0.2ms overhead for the basic stats display
3. **Render on canvas, not DOM** — HUD is drawn by Skia itself, no DOM overlays that could interfere with WebGL
4. **Async GPU results** — never stall the pipeline waiting for GPU query results; show N-2 frame's GPU time
5. **Production-safe** — the basic HUD and User Timing marks can ship in production behind a flag
6. **DevTools-native** — leverage Chrome's Performance panel extensibility for zero-overhead external profiling
7. **Export-friendly** — speedscope JSON format is the standard for flame chart sharing
## References
- [Skia Tracing](https://skia.org/docs/dev/tools/tracing) — Skia's internal `TRACE_EVENT` macros, Perfetto integration
- [Skia Debugger](https://debugger.skia.org) — SKP visual debugger with GPU op bounds
- [Skia Perf](https://skia.org/docs/dev/testing/skiaperf) — Skia's continuous perf monitoring with 400k+ metrics per commit
- [CanvasKit profiling build](https://www.npmjs.com/package/canvaskit-wasm) — `canvaskit-wasm/profiling` with full WASM function names
- [Figma webgl-profiler](https://github.com/figma/webgl-profiler) — Figma's GPU profiler using EXT_disjoint_timer_query, speedscope output
- [Skia render-skp.html](https://skia.googlesource.com/skia/+/052566d8ccb7/tools/perf-canvaskit-puppeteer/render-skp.html) — Skia's own CanvasKit benchmark: frame-to-frame timing methodology
- [EXT_disjoint_timer_query_webgl2 spec](https://registry.khronos.org/webgl/extensions/EXT_disjoint_timer_query_webgl2/) — WebGL GPU timing extension
- [Chrome Performance extensibility API](https://developer.chrome.com/docs/devtools/performance/extension) — Custom tracks via `performance.measure` detail.devtools
- [Unity Frame Timing Manager](https://docs.unity3d.com/2022.3/Documentation/Manual/frame-timing-manager.html) — CPU main thread / render thread / GPU split
- [Unreal Insights](https://dev.epicgames.com/documentation/en-us/unreal-engine/introduction-to-performance-profiling-and-configuration-in-unreal-engine) — Per-thread, per-GPU frame profiling
- Skia Milestone 132-133 release notes — `GrGLInterface` timer query support, `GpuStats.elapsedTime` for GPU time reporting

View file

@ -92,6 +92,8 @@ export const developmentSidebar = (prefix: string, label: string): DefaultTheme.
items: [
{ text: 'Contributing', link: `${prefix}/development/contributing` },
{ text: 'Testing', link: `${prefix}/development/testing` },
{ text: 'Renderer Profiler', link: `${prefix}/development/renderer-profiler` },
{ text: 'Variables UI Roadmap', link: `${prefix}/development/variables-ui-roadmap` },
{ text: 'OpenSpec', link: `${prefix}/development/openspec` },
{ text: 'Roadmap', link: `${prefix}/development/roadmap` },
],

View file

@ -0,0 +1,95 @@
---
title: Renderer Profiler
description: Use the CanvasKit renderer profiler HUD and frame capture tools to investigate rendering performance.
---
# Renderer Profiler
OpenPencil includes a CanvasKit renderer profiler for debugging frame time, GPU timing, draw calls, cache behavior, and expensive render phases.
## Enable the HUD
In the browser app, open the menu and choose:
```txt
View → Profiler
```
The app toggles `store.toggleProfiler()`, which maps to `editor.renderer.profiler.toggle()`.
The HUD is drawn directly on the Skia canvas so it measures the same rendering path as the document. It is not a DOM overlay.
## HUD metrics
The profiler HUD shows:
- **FPS / frame time** — smoothed frame cadence.
- **CPU** — JavaScript/WASM render time for the frame.
- **GPU** — latest available `EXT_disjoint_timer_query_webgl2` result when the browser exposes it.
- **Nodes / culled nodes** — total visible scene work and viewport culling count.
- **Draws** — WebGL draw calls counted through the instrumented context.
- **Cache** — whether the scene picture cache was reused.
- **Phases** — timings for renderer phases such as scene draw, picture replay/record, volatile overlays, section labels, selection, rulers, and flush.
- **Frame graph** — rolling frame history with 60 fps / 30 fps / slow thresholds and GPU bars when available.
GPU timing is asynchronous. The value shown is the latest completed GPU query, not necessarily the current frame.
## Implementation locations
Core profiler code lives in:
```txt
packages/core/src/profiler/
```
Main entry points:
- `render-profiler.ts``RenderProfiler` facade used by `SkiaRenderer`.
- `frame/stats.ts` — rolling frame statistics.
- `gpu-timer.ts` — WebGL timer query wrapper.
- `draw-call-counter.ts` — WebGL draw-call instrumentation.
- `phase-timer.ts` — phase timing and User Timing integration.
- `hud-renderer.ts` — canvas HUD rendering.
- `frame/capture.ts` and `speedscope-export.ts` — detailed capture and Speedscope export.
Renderer integration lives under:
```txt
packages/core/src/canvas/renderer*.ts
packages/core/src/canvas/renderer/
```
App wiring lives in:
```txt
src/app/editor/profiler/index.ts
src/app/shell/menu/schema.ts
src/app/shell/menu/app-menu.ts
```
## Programmatic use
From app/editor code:
```ts
store.toggleProfiler()
```
From a renderer instance:
```ts
renderer.profiler.toggle()
renderer.profiler.beginCapture()
// render one or more frames
const capture = renderer.profiler.endCapture()
const speedscopeJson = renderer.profiler.exportSpeedscope()
renderer.profiler.downloadSpeedscope()
```
Detailed captures are for targeted debugging. Keep the normal HUD path lightweight and avoid enabling expensive capture work unless a user or developer explicitly asks for it.
## Notes
- The profiler is designed to be safe when disabled: no timing calls or allocations should be added to hot paths unless `profiler.enabled` / `profiler.capturing` is active.
- GPU timing depends on browser and hardware support for `EXT_disjoint_timer_query_webgl2`.
- If GPU timing is unavailable, the HUD still reports CPU time, draw calls, phases, node counts, and cache status.

View file

@ -1,70 +1,179 @@
---
title: Scripting
description: Execute JavaScript with the Figma Plugin API — query nodes, batch-modify designs, create frames.
description: Execute JavaScript with a Figma-compatible Plugin API to query, batch-modify, and generate designs.
---
# Scripting
`open-pencil eval` gives you the full Figma Plugin API in the terminal. Read nodes, modify properties, create shapes — then write changes back to the file.
`open-pencil eval` runs JavaScript against an OpenPencil document with a Figma-compatible `figma` global. Use it for headless batch edits, inspection, fixture setup, and automation without opening the editor UI.
## Basic Usage
## Basic usage
```sh
open-pencil eval design.fig -c "figma.currentPage.children.length"
open-pencil eval design.fig -c "return figma.currentPage.children.length"
```
The `-c` flag takes JavaScript. The `figma` global works like the Figma Plugin API.
## Query Nodes
The `-c` flag accepts JavaScript. If the code does not start with `return`, OpenPencil wraps it in an async function and returns the value from that function when present.
```sh
open-pencil eval design.fig -c "
figma.currentPage.findAll(n => n.type === 'FRAME' && n.name.includes('Button'))
.map(b => ({ id: b.id, name: b.name, w: b.width, h: b.height }))
const frame = figma.createFrame()
frame.name = 'Card'
frame.resize(300, 200)
frame.layoutMode = 'VERTICAL'
frame.itemSpacing = 12
return { id: frame.id, name: frame.name }
"
```
## Modify and Save
## Query nodes
```sh
open-pencil eval design.fig -c "
figma.currentPage.children.forEach(n => n.opacity = 0.5)
" -w
return figma.currentPage
.findAll((node) => node.type === 'FRAME' && node.name.includes('Button'))
.map((button) => ({
id: button.id,
name: button.name,
width: button.width,
height: button.height
}))
"
```
`-w` writes changes back to the input file. Use `-o output.fig` to write to a different file instead.
## Modify and save
## Read from Stdin
For longer scripts:
Use `--write` / `-w` to write changes back to the input file:
```sh
cat transform.js | open-pencil eval design.fig --stdin -w
open-pencil eval design.fig -c "
figma.currentPage.children.forEach((node) => {
node.opacity = 0.5
})
" --write
```
## Live App Mode
Omit the file to run against the running desktop app:
Use `--output` / `-o` to write to a new file:
```sh
open-pencil eval -c "figma.currentPage.name"
open-pencil eval design.fig -c "figma.currentPage.name = 'Updated'" -o updated.fig
```
## Available API
The `figma` object supports:
- `figma.currentPage` — the active page
- `figma.root` — the document root
- `figma.createFrame()`, `figma.createRectangle()`, `figma.createEllipse()`, `figma.createText()`, etc.
- `.findAll()`, `.findOne()` — search descendants
- `.appendChild()`, `.insertChild()` — tree manipulation
- All property setters: `.fills`, `.strokes`, `.effects`, `.opacity`, `.cornerRadius`, `.layoutMode`, `.itemSpacing`, etc.
This is the same API Figma plugins use, so existing knowledge and code snippets transfer directly.
## JSON Output
## Read scripts from stdin
```sh
open-pencil eval design.fig -c "..." --json
cat transform.js | open-pencil eval design.fig --stdin --write
```
## Live app mode
Omit the file path to run against the currently open document in the desktop app:
```sh
open-pencil eval -c "return figma.currentPage.name"
```
The desktop app must be running with a document open.
## Output
By default, non-TTY output is JSON. Use `--json` to force JSON output:
```sh
open-pencil eval design.fig -c "return figma.currentPage.children.map((n) => n.name)" --json
```
Use `--quiet` / `-q` to suppress output when only writing a file.
## Supported API surface
The API is intentionally close to Figma's Plugin API, but it maps to OpenPencil's scene graph and file format.
### Document and pages
- `figma.root`
- `figma.currentPage`
- `figma.currentPage.selection`
- `figma.getNodeById(id)`
- `figma.createPage()`
### Node creation
- `figma.createFrame()`
- `figma.createRectangle()`
- `figma.createEllipse()`
- `figma.createText()`
- `figma.createLine()`
- `figma.createPolygon()`
- `figma.createStar()`
- `figma.createVector()`
- `figma.createComponent()`
- `figma.createSection()`
### Tree operations
- `node.children`
- `node.parent`
- `node.appendChild(child)`
- `node.insertChild(index, child)`
- `node.clone()`
- `node.remove()`
- `node.findAll(callback?)`
- `node.findOne(callback)`
- `node.findChild(callback)`
- `node.findChildren(callback?)`
- `figma.group(nodes, parent)`
- `figma.ungroup(node)`
### Components
- `figma.createComponentFromNode(node)`
- `component.createInstance()`
- `instance.mainComponent`
### Variables
- `figma.getLocalVariables(type?)`
- `figma.getVariableById(id)`
- `figma.getLocalVariableCollections()`
- `figma.getVariableCollectionById(id)`
- `figma.createVariable(name, type, collectionId, value?)`
- `figma.setVariableValue(variableId, modeId, value)`
- `figma.deleteVariable(id)`
- `figma.createVariableCollection(name)`
- `figma.deleteVariableCollection(id)`
- `figma.bindVariable(nodeId, field, variableId)`
- `figma.unbindVariable(nodeId, field)`
### Properties
Common node properties are readable/writable through the proxy, including:
- Geometry: `x`, `y`, `width`, `height`, `rotation`, `resize(width, height)`
- Appearance: `fills`, `strokes`, `effects`, `opacity`, `visible`, `locked`, `blendMode`, `clipsContent`
- Radius: `cornerRadius`, `topLeftRadius`, `topRightRadius`, `bottomRightRadius`, `bottomLeftRadius`
- Text: `characters`, `fontSize`, `fontName`, `fontWeight`, alignment, line height, letter spacing, style-run helpers
- Auto-layout: `layoutMode`, `primaryAxisAlignItems`, `counterAxisAlignItems`, `itemSpacing`, padding, sizing, and layout positioning fields
- Stroke helpers: `strokeWeight`, `strokeAlign`, `dashPattern`
### Utilities
- `figma.mixed`
- `figma.createImage(data)`
- `figma.loadFontAsync(fontName)` no-ops because OpenPencil does not gate text edits on plugin font loading
- `figma.listAvailableFontsAsync()` returns host-provided fonts when available
- `figma.notify(message)` logs a warning in headless mode
- `figma.viewport`
## Not yet Figma-compatible
These Figma APIs are not exposed as compatible helpers yet:
- `node.exportAsync()`
- `node.setBoundVariable(field, variable)`
- `node.detachInstance()`
- `figma.combineAsVariants(components, parent)`
- Figma style APIs such as `figma.createPaintStyle()` / `figma.createTextStyle()`
- Full vector boolean operation parity
Use OpenPencil CLI export commands, core tools, or direct scene-graph helpers where available.