openpencil/packages/docs/programmable/ai-chat.md
Danila Poyarkov 39bc4d2eb1
feat(acp): add reusable MCP connections (#516)
* feat(acp): add reusable MCP connections

- Store named Streamable HTTP connections separately from model providers
- Keep bearer tokens in the credential manager and resolve them per ACP session
- Add localized settings, validation, documentation, and focused coverage

* test(acp): harden MCP connection workflow

- Label credential inputs and confirm destructive connection removal
- Exercise MCP server delivery through an in-memory ACP session
- Extend the browser smoke test to cover accessible input and confirmation flows

* fix(acp): validate MCP connection lifecycle

- Keep non-browser storage initialization in memory and restore the German model copy
- Reject persisted name collisions and invalid draft IDs
- Preserve connections on credential failures and require credentials before enablement
2026-08-14 20:08:45 +03:00

5.7 KiB

title description
AI Chat Built-in AI assistant with 90+ tools for creating and modifying designs.

AI Chat

Press J (Ctrl + J) to open the AI assistant. Describe what you want — it creates shapes, sets styles, manages layout, works with components, and analyzes your design.

Setup

  1. Open the AI chat panel (J)
  2. Click the settings icon
  3. Add a model and configure its provider, model ID, credentials, and capabilities
  4. Save the model and assign it to Design agent

You can configure multiple reusable models and separately assign models for design work, reviews, fast tasks, and image input. Models using the same provider connection reuse its stored credential.

Supported Providers

Provider Models Setup
OpenRouter Claude, GPT, Gemini, DeepSeek, Qwen, and others API key from openrouter.ai
Anthropic Claude Sonnet 4.6, Claude Opus 4.6 API key from console.anthropic.com
OpenAI GPT-5.3 Codex, GPT-4.1, o3, o4-mini API key from platform.openai.com
Google AI Gemini 3.1 Pro, Gemini 3 Flash API key from aistudio.google.dev
Z.ai GLM-5.1, GLM-5, GLM-4.7, GLM-4.5 family API key from docs.z.ai
MiniMax MiniMax M3, M2.7, M2.7-highspeed, M2.5, M2.1 API key from platform.minimax.io
OpenAI-compatible Any endpoint with OpenAI API format Custom base URL + key. Supports Completions and Responses API toggle.
Anthropic-compatible Any endpoint with Anthropic API format Custom base URL + key

No backend, no subscription — your key talks directly to the provider. Browser requests are subject to each provider's CORS policy, and model deployments vary in how reliably they stream tool calls. See BYOK provider and model compatibility for measured results and reproduction steps.

External MCP connections

Desktop ACP agents can also use trusted remote Model Context Protocol servers. In Settings → AI & agents → MCP connections, add a named Streamable HTTP endpoint, optionally save a bearer token, and enable the connection. OpenPencil stores the token in the configured credential backend rather than ordinary settings and resolves it only when starting the ACP session.

Remote servers must use HTTPS. Loopback HTTP endpoints are accepted for local development. Review and trust a server before enabling it: its tools may read external data or perform actions with the credentials you provide. OpenPencil's built-in design MCP server remains attached automatically and does not need to be added here.

What It Can Do

The assistant has 90+ tools across these categories:

  • Create — frames, shapes, text, components, pages. Renders JSX for complex layouts.
  • Style — fills, strokes, effects, opacity, corner radius, blend modes.
  • Layout — auto-layout, grid, alignment, spacing, sizing.
  • Components — create components, instances, component sets. Manage overrides.
  • Variables — create/edit variables, collections, modes. Bind to fills.
  • Query — find nodes, XPath selectors, read properties, list pages, fonts, selection.
  • Inspectget_jsx for JSX roundtrip view, diff_jsx for structural diffs, describe for semantic role and design issue detection.
  • Analyze — color palette, typography audit, spacing consistency, cluster detection.
  • Export — PNG, SVG, JSX with Tailwind classes. Vision-based verification via export_image.
  • Vector — boolean operations, path manipulation.

Visual Verification

The assistant can verify its work visually. After creating or modifying designs, it uses export_image to capture a screenshot and checks the result against the original request. This catches layout issues, missing elements, and color mismatches that text-only responses would miss.

Example Prompts

  • "Create a card with a title, description, and a blue button"
  • "Make all buttons on this page use the same border radius"
  • "What fonts are used in this file?"
  • "Change the background of the selected frame to a gradient from blue to purple"
  • "Export the selected frame as SVG"
  • "Find all text nodes with font size less than 12"
  • "Describe the selected component — what role does it look like?"
  • "Show me the JSX for this frame"

Tips

  • Select nodes before asking — the assistant knows what's selected.
  • Be specific about colors, sizes, and positions for precise results.
  • The assistant can modify multiple nodes in one message.
  • Use "undo" in the editor if you don't like the result — AI mutations support full undo.
  • All layout is recomputed automatically after each tool execution.