openpencil/openspec/specs/testing/spec.md

# testing Specification

## Purpose
Test infrastructure. Playwright visual regression (E2E with Figma CDP reference), bun:test unit tests, no-chrome test mode, data-ready synchronization, and page reuse optimization.
## Requirements
### Requirement: Playwright visual regression testing
E2E tests SHALL use Playwright to create shapes and compare screenshots against baseline snapshots.

#### Scenario: Visual regression test pass
- **WHEN** `bun run test` is executed
- **THEN** Playwright tests create shapes, take screenshots, and compare against baseline PNGs

### Requirement: Figma CDP reference tests
A separate Playwright project SHALL connect to Figma via Chrome DevTools Protocol (CDP) to capture reference screenshots for pixel-perfect comparison.

#### Scenario: Figma reference capture
- **WHEN** `bun run test:figma` is executed with Figma running in debug mode
- **THEN** Playwright connects to Figma via CDP and captures reference screenshots

### Requirement: No-chrome test mode
The editor SHALL support a test mode (activated via URL param or environment) that hides Chrome/UI elements for clean screenshot capture.

#### Scenario: Test mode rendering
- **WHEN** the editor loads in test mode
- **THEN** only canvas content is rendered, without toolbar, panels, or other UI chrome

### Requirement: Data-ready optimization
The editor SHALL signal readiness via a `data-ready` attribute for E2E test synchronization, replacing unreliable timeouts.

#### Scenario: Wait for ready
- **WHEN** a Playwright test waits for the data-ready attribute
- **THEN** it proceeds only when the canvas is fully rendered

### Requirement: Page reuse for speed
E2E tests SHALL reuse the browser page across test cases to minimize overhead. Target: all E2E tests complete in <3s.

#### Scenario: Fast E2E execution
- **WHEN** the full E2E test suite runs
- **THEN** it completes in under 3 seconds by reusing the page

### Requirement: bun:test unit tests
Engine unit tests SHALL use bun:test and complete in <50ms.

#### Scenario: Unit test speed
- **WHEN** `bun test ./tests/engine` is run
- **THEN** all unit tests pass in under 50ms

### Requirement: Fig-import unit tests
Unit tests SHALL verify the .fig import pipeline: node type mapping, transform extraction, fill/stroke/effect import, gradient stops, image fills, arc data, stroke properties, and nested frame hierarchies.

#### Scenario: Run fig-import tests
- **WHEN** `bun test ./tests/engine/fig-import.test.ts` is executed
- **THEN** all import pipeline tests pass covering tier 1 rendering features

### Requirement: Layout unit tests
Unit tests SHALL verify Yoga auto-layout computation: direction, gap, padding, justify, align, child sizing (fixed/fill/hug), cross-axis sizing, wrap, and nested layouts.

#### Scenario: Run layout tests
- **WHEN** `bun test ./tests/engine/layout.test.ts` is executed
- **THEN** all layout computation tests pass

### Requirement: Layers panel E2E tests
E2E tests SHALL verify the layers panel: node visibility in tree, expand/collapse frames, selection sync between canvas and layers panel. Test fixtures match current demo shapes (Components, App Preview).

#### Scenario: Run layers panel E2E
- **WHEN** the layers panel E2E tests run
- **THEN** all tests pass verifying tree structure, visibility toggles, and selection sync with updated demo shape names

### Requirement: Component-instance sync unit tests
Unit tests SHALL cover the component-instance sync lifecycle: instance creation with `componentId` mapping on children, sync propagation of property changes, override preservation during sync, new child addition to instances, and detach breaking the link.

#### Scenario: Instance child mapping test
- **WHEN** `bun test ./tests/engine/scene-graph.test.ts` runs
- **THEN** the test verifies instance children have `componentId` pointing to component children

#### Scenario: Sync propagation test
- **WHEN** a component's text child changes and sync is triggered
- **THEN** the test verifies instance children receive the updated text and fontSize

#### Scenario: Override preservation test
- **WHEN** an instance child has an overridden text value
- **THEN** the test verifies the override is preserved while non-overridden properties sync

#### Scenario: New child addition test
- **WHEN** a new child is added to a component and sync runs
- **THEN** the test verifies existing instances gain the new child

#### Scenario: Detach test
- **WHEN** an instance is detached
- **THEN** the test verifies type becomes FRAME, componentId is null, and the component's instance list no longer includes it

### Requirement: Variable system unit tests
Unit tests SHALL cover the variables system: add and resolve color/number variables, alias chain resolution, circular alias detection (returns undefined), mode switching, bind/unbind to nodes, and variable removal cleanup. Total: 7 variable tests in a "Variables" describe block.

#### Scenario: Variable tests pass
- **WHEN** `bun test ./tests/engine` is run
- **THEN** all variable tests pass (add/resolve color, resolve number, alias chain, circular alias, mode switching, bind/unbind, removal cleanup)

### Requirement: .fig roundtrip tests
The test suite SHALL include roundtrip tests for real .fig files: parsing property invariants, encode/decode cycle fidelity. Test fixtures (material3.fig, nuxtui.fig) SHALL be tracked via Git LFS.

#### Scenario: Roundtrip encode/decode
- **WHEN** a .fig file is parsed and re-encoded
- **THEN** the re-encoded data decodes to the same node properties

#### Scenario: Property invariants
- **WHEN** a real .fig file is parsed
- **THEN** all nodes have valid types, dimensions ≥ 0, and required fields present

### Requirement: .fig import performance
The .fig import pipeline SHALL avoid O(n²) child resolution. A children index SHALL be built upfront for linear-time lookups.

#### Scenario: Large file import speed
- **WHEN** material3.fig (87K nodes) is imported
- **THEN** parsing completes in under 2 seconds (was 37s before fix)

### Requirement: JSX renderer tests
The test suite SHALL include tests for the JSX renderer covering all node types (Frame, Text, Rectangle, Ellipse, etc.), layout props, effects, and nesting. Currently 27 tests.

#### Scenario: Render Frame with children
- **WHEN** a JSX tree with Frame containing Rectangle and Text is rendered
- **THEN** the scene graph contains correct parent-child relationships and properties

### Requirement: AI chat Playwright tests
The test suite SHALL include Playwright e2e tests for the AI chat panel with mock transport (no real LLM calls). Tests cover: sending a message, receiving a response, tool call display. Real LLM tests available via TEST_REAL_LLM=1 flag.

#### Scenario: Chat with mock transport
- **WHEN** a Playwright test sends a message via the chat input
- **THEN** the mock transport returns a response and the message appears in the chat

### Requirement: JSX export tests
The test suite SHALL include tests for sceneNodeToJsx() covering shapes, text, layout props, effects, and multi-selection. Currently 14 tests.

#### Scenario: Export rectangle to JSX
- **WHEN** sceneNodeToJsx is called on a rectangle with blue fill
- **THEN** the output includes Rectangle component with bg prop


### Requirement: FigmaAPI unit tests

The project SHALL provide comprehensive unit tests for the Figma Plugin API in `tests/engine/figma-api.test.ts`.

#### Scenario: Node creation tests
- **WHEN** `bun test tests/engine/figma-api.test.ts` runs
- **THEN** tests verify `createFrame()`, `createRectangle()`, `createText()`, and other creation methods

#### Scenario: Property access tests
- **WHEN** tests read node properties (x, y, width, height, name, fills, strokes)
- **THEN** all getters return correct values from SceneNode

#### Scenario: Property setting tests
- **WHEN** tests set node properties (`node.name = "Test"`, `node.x = 100`)
- **THEN** SceneGraph is updated correctly

#### Scenario: Text property tests
- **WHEN** tests access `textNode.characters`, `fontSize`, `fontName`
- **THEN** text-specific properties work correctly

#### Scenario: Auto-layout tests
- **WHEN** tests set `layoutMode = "VERTICAL"`, `itemSpacing`, `paddingLeft`
- **THEN** auto-layout properties are applied

#### Scenario: Tree operation tests
- **WHEN** tests call `appendChild()`, `insertChild()`, `remove()`
- **THEN** scene graph tree is modified correctly

#### Scenario: Traversal tests
- **WHEN** tests call `findAll()`, `findOne()`, `findAllWithCriteria()`
- **THEN** correct nodes are returned

#### Scenario: Component tests
- **WHEN** tests create components and instances
- **THEN** component-instance relationships work

#### Scenario: Serialization tests
- **WHEN** tests call `figma.toJSON(node)`
- **THEN** JSON output includes all required properties

#### Scenario: Frozen array tests
- **WHEN** tests access `fills`, `strokes`, `children` and try to mutate
- **THEN** arrays are frozen and throw errors on mutation attempts

#### Scenario: 924 LOC of tests
- **WHEN** FigmaAPI test file is counted
- **THEN** it contains 924 lines covering 60+ test cases

### Requirement: Eval CLI integration tests

The project SHALL provide integration tests for eval command in `tests/engine/eval-cli.test.ts`.

#### Scenario: Inline code execution test
- **WHEN** test runs eval with `--code 'return figma.currentPage.id'`
- **THEN** output matches expected page ID

#### Scenario: Node creation test
- **WHEN** test runs eval creating a frame
- **THEN** document is modified and frame exists

#### Scenario: Write flag test
- **WHEN** test runs eval with `--write`
- **THEN** file is modified in-place

#### Scenario: Output file test
- **WHEN** test runs eval with `-o output.fig`
- **THEN** new file is created with modifications

#### Scenario: JSON output test
- **WHEN** test runs eval with `--json`
- **THEN** result is formatted as JSON

#### Scenario: Error handling test
- **WHEN** eval code throws error
- **THEN** stderr contains error message

#### Scenario: 202 LOC of integration tests
- **WHEN** eval CLI test file is counted
- **THEN** it contains 202 lines covering 17 integration scenarios

### Requirement: Tool schema unit tests

The project SHALL provide unit tests for unified tool definitions in `tests/engine/tools.test.ts`.

#### Scenario: Schema structure tests
- **WHEN** tests validate tool schemas
- **THEN** each tool has name, description, parameters, and handler

#### Scenario: Tool handler tests
- **WHEN** tests invoke tool handlers with valid params
- **THEN** handlers return expected results

#### Scenario: Parameter validation tests
- **WHEN** tests call handlers with invalid params
- **THEN** errors are raised or handled gracefully

#### Scenario: 390 LOC of tool tests
- **WHEN** tools test file is counted
- **THEN** it contains 390 lines

### Requirement: Tool adapter tests

The project SHALL provide adapter tests in `tests/engine/tools-ai-adapter.test.ts` (190 LOC) and `tests/engine/tools-cli.test.ts` (219 LOC).

#### Scenario: AI adapter format test
- **WHEN** AI adapter converts schema to LLM format
- **THEN** output matches expected structure

#### Scenario: CLI adapter test
- **WHEN** CLI commands use tool schema
- **THEN** parameters map correctly

### Requirement: App menu integration tests

The project SHALL provide integration tests for app menu in `tests/e2e/app-menu.spec.ts`.

#### Scenario: File menu test
- **WHEN** test clicks File menu items
- **THEN** actions (Open, Save, Export) are triggered

#### Scenario: Edit menu test
- **WHEN** test uses Edit menu (Undo, Redo, Copy, Paste)
- **THEN** operations execute correctly

#### Scenario: View menu test
- **WHEN** test uses View → Zoom in/out
- **THEN** viewport zoom changes

#### Scenario: Object menu test
- **WHEN** test uses Object → Group/Ungroup
- **THEN** nodes are grouped/ungrouped

#### Scenario: Text menu test
- **WHEN** test uses Text → Bold/Italic
- **THEN** text styles are applied

#### Scenario: Arrange menu test
- **WHEN** test uses Arrange → Bring to front
- **THEN** z-order changes

#### Scenario: 131 LOC of app menu tests
- **WHEN** app menu test file is counted
- **THEN** it contains 131 lines

### Requirement: Autosave integration tests

The project SHALL provide integration tests for autosave in `tests/e2e/autosave.spec.ts`.

#### Scenario: Autosave trigger test
- **WHEN** test modifies scene graph
- **THEN** autosave timer starts

#### Scenario: Debounce test
- **WHEN** test makes rapid changes
- **THEN** only one save occurs after 3 seconds

#### Scenario: Write test
- **WHEN** autosave timer expires
- **THEN** file is written

#### Scenario: Skip unchanged test
- **WHEN** sceneVersion matches savedVersion
- **THEN** write is skipped

#### Scenario: 113 LOC of autosave tests
- **WHEN** autosave test file is counted
- **THEN** it contains 113 lines

### Requirement: Test coverage expansion

The test suite SHALL expand from original coverage to include 2571 LOC of new tests across 7 files (figma-api.test.ts, eval-cli.test.ts, tools.test.ts, tools-ai-adapter.test.ts, tools-cli.test.ts, app-menu.spec.ts, autosave.spec.ts).

#### Scenario: Total new test lines
- **WHEN** new test files are counted
- **THEN** 2571 lines of tests are added

### Requirement: Canvas manipulation E2E tests
The E2E suite SHALL include tests for `tests/e2e/canvas-manipulation.spec.ts` covering: marquee selection, resize handle drag, rotation handle drag, Alt+drag duplicate, Shift+Arrow nudge (10px), and hover highlight (visual screenshot comparison).

#### Scenario: Canvas manipulation tests pass
- **WHEN** `bun run test` executes `canvas-manipulation.spec.ts`
- **THEN** all 6+ tests pass verifying selection, resize, rotation, duplicate, nudge, and hover

### Requirement: Toolbar E2E tests
The E2E suite SHALL include tests for `tests/e2e/toolbar.spec.ts` covering: shapes flyout chevron, Polygon tool, Star tool, Pen tool (corner, close path, Escape), and Frame flyout.

#### Scenario: Toolbar tests pass
- **WHEN** `bun run test` executes `toolbar.spec.ts`
- **THEN** all 7+ tests pass verifying flyout menus and shape creation

### Requirement: Properties panel E2E tests
The E2E suite SHALL include tests for `tests/e2e/properties-panel.spec.ts` covering: ScrubInput drag, corner radius (uniform + independent), fill gradient switch, variable bind badge, alignment buttons, flip horizontal, and clip content checkbox.

#### Scenario: Properties panel tests pass
- **WHEN** `bun run test` executes `properties-panel.spec.ts`
- **THEN** all 8+ tests pass verifying property changes via store assertions

### Requirement: Text formatting E2E tests
The E2E suite SHALL include tests for `tests/e2e/text-formatting.spec.ts` covering: cursor positioning, double-click word select, ⌘B bold, ⌘I italic, Alt+ArrowRight word navigation, and Bold button in typography section.

#### Scenario: Text formatting tests pass
- **WHEN** `bun run test` executes `text-formatting.spec.ts`
- **THEN** all 6+ tests pass verifying text edit interactions

### Requirement: Auto-layout E2E tests
The E2E suite SHALL include tests for `tests/e2e/auto-layout.spec.ts` covering: Shift+A on selection, direction toggle, gap ScrubInput, uniform padding, alignment grid, and remove auto-layout.

#### Scenario: Auto-layout tests pass
- **WHEN** `bun run test` executes `auto-layout.spec.ts`
- **THEN** all 6+ tests pass verifying auto-layout frame state via store

### Requirement: Snap guides E2E tests
The E2E suite SHALL include tests for `tests/e2e/snap-guides.spec.ts` covering: edge snap guide and center snap guide, both verified via screenshot comparison.

#### Scenario: Snap guide tests pass
- **WHEN** `bun run test` executes `snap-guides.spec.ts`
- **THEN** 2 screenshot comparison tests pass confirming guide line visibility

### Requirement: Panel resize E2E tests
The E2E suite SHALL include tests for `tests/e2e/panels.spec.ts` covering: left panel drag resize, width persistence after reload, and ⌘\\ UI toggle (hide + show).

#### Scenario: Panel tests pass
- **WHEN** `bun run test` executes `panels.spec.ts`
- **THEN** all 4+ tests pass verifying panel DOM dimensions and visibility

### Requirement: Variables dialog E2E tests
The E2E suite SHALL include tests for `tests/e2e/variables-dialog.spec.ts` covering: dialog open, search filter, cell edit, and color swatch picker.

#### Scenario: Variables dialog tests pass
- **WHEN** `bun run test` executes `variables-dialog.spec.ts`
- **THEN** all 4+ tests pass verifying dialog interactions

### Requirement: Export E2E tests
The E2E suite SHALL include tests for `tests/e2e/export.spec.ts` covering: format selector, add/remove rows, preview toggle, and SVG hiding scale input.

#### Scenario: Export tests pass
- **WHEN** `bun run test` executes `export.spec.ts`
- **THEN** all 5+ tests pass verifying export section UI state

### Requirement: CanvasHelper extended helpers
The `CanvasHelper` class SHALL provide `marquee()`, `hover()`, `dragScrubInput()`, and `shiftDrag()` helper methods usable across all spec files.

#### Scenario: Helpers available in all specs
- **WHEN** a spec file imports `CanvasHelper`
- **THEN** `canvas.marquee()`, `canvas.hover()`, `canvas.dragScrubInput()`, and `canvas.shiftDrag()` are callable without TypeScript errors