openpencil/packages/docs/programmable/collaboration.md
Victor Wads 690c1247e4
feat(collab): show MCP agents, follow streamed JSX, and follow your agents as they work (#725)
* feat(MCP): follow agent activity in canvas

* feat(collab): show MCP, ACP, and harness sessions as agents

MCP clients worked on the document unseen: only the built-in chat had a
presence, and following agent activity meant a separate setting that
moved the viewport after every MCP tool. Each MCP session now shows as
an agent with a callsign in its owner's color, like the chat: the MCP
server forwards the session and the client's name with each tool call,
the app's own ACP and Pi harness chats mark their sessions with a
header, and the agent points at the layers each call reads or changes
on their page. It rests after a quiet spell, leaves when its session
ends or the server disconnects, and collaborators see it through
awareness. Following it works like following anyone else, from the
avatars, so the default-on follow setting and its viewport fitting go.

Co-authored-by: Victor Wads <victor@wads.dev>

* feat(ai): move the chat's agent through JSX as it streams

While the built-in chat streamed a render call, its preview grew on the
canvas but the agent stood still until the tool finished. The preview
now reports, after each update, the element that appeared last and the
bounds of what the JSX builds; the agent's cursor follows that element
and its outline traces the preview, for collaborators too, until the
tool runs and the agent outlines the real layers. Peers' outlines are
validated and capped like their selections.

Co-authored-by: Victor Wads <victor@wads.dev>

* feat(collab): glide cursors and the followed view instead of jumping

People's and agents' cursors jumped to each new point, which with
throttled awareness and an agent streaming JSX made them stutter, and
following re-centered the view in one jump on every update. Cursors now
ease to each new point from wherever they are drawn, and following pans
and zooms the view there the same way, stopping in place when you take
over. With animations off or reduced motion, both move at once. Each
cursor carries an id so it keeps its glide between updates.

Co-authored-by: Victor Wads <victor@wads.dev>

* test(collab): cover MCP agents and the streaming agent in the browser

Test runs send MCP requests through the bridge's own command handler, so
a browser test can show an MCP session as an agent to the editor and to
a collaborator without a separate server. The streaming JSX test checks
that the chat's agent cursor and outline follow the newest element.

Co-authored-by: Victor Wads <victor@wads.dev>

* docs: describe agent sessions, streaming cursors, and gliding follow

Co-authored-by: Victor Wads <victor@wads.dev>

* fix(ai): keep the streaming agent's name off the text it writes

The chat's agent sat at the newest streamed element's top-left corner,
so its name label covered the text being written. It now sits at the
element's trailing corner, where content grows.

Co-authored-by: Victor Wads <victor@wads.dev>

* fix(collab): end only a closed connection's own MCP sessions

When the app's connection to the MCP server closed, every MCP session's
agent left, including sessions that never came over that connection,
such as a second editor's. The bridge now remembers the sessions each
connection carried and ends only those.

Co-authored-by: Victor Wads <victor@wads.dev>

* feat(ai): follow your agents automatically while they work

Agents spun up from the chat or an MCP client worked out of sight and
then went idle, so people had to find what changed, and the agent skill
told agents to move the user's view and selection after every edit. A
Follow agents toggle in the AI panel's header, on by default, now has
the view follow our agents from the start of each run: whichever starts
first, then the next one at work once the followed agent rests. Leaving
the page, moving the view, Escape, or Stop following leaves that agent
alone until it rests; people are never followed this way. The skill no
longer asks agents to select and zoom to their work.

Co-authored-by: Victor Wads <victor@wads.dev>

* fix(collab): keep an MCP agent when a restarted bridge carries its session

Restarting MCP disconnects the old bridge and opens a new one at once,
but the browser reports the old socket's close only after its close
handshake. A tool call that reached the new connection in between was
undone by that close, which ended the session and removed its agent.
Sessions now record every connection their calls came over, across
bridges, and end only when the last of them closes.

The docs no longer say that following an agent keeps it from editing
out of sight: following moves your view, and the stale variables and
follow bullets the changelog's union merge brought back are removed.

Co-authored-by: Victor Wads <victor@wads.dev>

* test(collab): record what the bridge sends instead of an empty fake method

Co-authored-by: Victor Wads <victor@wads.dev>

---------

Co-authored-by: Danila Poyarkov <dev@dannote.net>
2026-10-07 10:04:58 +00:00

5.2 KiB

title description
Collaboration Real-time collaborative editing via P2P WebRTC — no server, no account.

Collaboration

Edit designs together in real time. Peers connect directly — no server relays your data, no account required.

Sharing a Room

  1. Click the share button in the top-right corner
  2. Click Share this file — the link (app.openpencil.dev/share/<room-id>) is copied
  3. Send it to your collaborators

Only Share puts a document into a room: the tab you share from becomes the room's tab and stays tied to its file. Anyone with the link can join.

Joining a Room

Open the link, or paste it (or just the room ID) into Join in the share panel or Join room… on Home. The room opens in a tab of its own, so the documents you already have open are never changed. On a computer, a browser also offers Open in desktop app, which opens the room in OpenPencil through an openpencil://join link.

You join right away under a generated name such as Teal Fox; set your own in the share panel or in Settings, and it is used in every room.

Rooms are not stored on a server: the file lives on the devices of the people who have been in the room, so a room tab opens its document only while one of them is online. Until then it says it is waiting, explains why, and opens the file as soon as someone who has it joins. A room you have been in before opens from this device's copy right away and syncs your changes when others return.

Leave room in the share panel ends your part in the room. A tab that shared its document goes back to being that document; a tab that joined keeps the room's file as a local unsaved copy you can save. Each room tab keeps its own connection, so you can be in several rooms at once.

What Syncs

  • Document changes — every edit (shapes, text, properties, layout) syncs instantly
  • Cursors — see where each collaborator is pointing, with their name and color
  • Selections — highlighted selections are visible to everyone
  • Agents — the built-in AI chat, ACP and Pi harness chats, and every connected MCP client appear as cursors at the layers they read or edit, each outlined label showing a sparkle and a callsign such as Fern. While the chat streams JSX, its cursor moves through the elements as they appear and outlines them. The cursor and outline have the color of the person running the agent, so you can tell whose agent it is. Only its name, kind, model, status, page, position, and edited layers are shared, never prompts or replies.

Follow Mode

Click a collaborator's avatar in the top bar to follow their viewport. Your canvas pans and zooms to match their view, and a frame in their color with a “Following …” bar shows whom you follow. Click the avatar again, press Esc, or click, scroll, zoom, or switch pages yourself to stop.

Your own agents, the AI chat and MCP clients such as Claude Code or Cursor, are followed automatically while they work, so what they edit stays in view. Turn this off with the crosshair button at the top of the AI panel. If you stop following an agent while it works, it is left alone until it finishes and followed again on its next run.

An avatar counts the agents that person runs. Hover over it to see each agent, what it is doing, and on which page, and click Follow next to an agent to keep the page and layers it is editing in view; following continues between its replies and stops when it leaves. The button after the avatars lists everyone in the room with their agents, and works from the keyboard. Your own avatar lists your agents — click one to rename it — and has Leave room.

The share panel lists everyone in the room with the agents they run, what each agent is doing, and on which page. Follow an agent the same way to keep the page and layers it is editing in view; following continues between its replies and stops when it leaves. Double-click one of your own agents to rename it.

How It Works

Peers connect directly via WebRTC — your design data goes straight from browser to browser, never through a central server. The document state uses a CRDT (conflict-free replicated data type), so concurrent edits merge automatically without conflicts.

Moving and reordering layers merges too. Each layer remembers every parent it has been moved into and its position among its siblings, and every peer works out the same layer tree from that history (Evan Wallace's tree CRDT). Moves, reorders, and new layers from different people all apply; if two people move the same layer at once, one move wins on every peer. When moves made at the same time would put two layers inside each other, the later move is undone, and a layer whose new parent was deleted meanwhile returns to where it was.

Everyone in a room needs a version of OpenPencil that records the layer tree the same way; versions that record it differently do not see each other's rooms.

The room persists locally — if you refresh the page, you rejoin it automatically with the same state.

Tips

  • Works in the browser and the desktop app
  • Room IDs are cryptographically random — only people with the link can join
  • Stale cursors are cleaned up automatically when someone disconnects