Reusable React infinite canvas for AI-generated documents, media, websites, and visual artifacts.

This package starts from the useful Family.blog prototype ideas: a plain working surface, pan and zoom, JSON-driven frames, live-updatable state, and a small command surface an AI can drive. It borrows architectural lessons from tldraw without depending on it: shape records, an editor-like imperative API, snapshots, custom renderers, and structured summaries for model context.

npm install @agent-canvas/react

import { AgentCanvas, createCanvasDocument } from "@agent-canvas/react";
import "@agent-canvas/react/styles.css";

const document = createCanvasDocument({
  title: "Research board",
  nodes: [
    {
      id: "brief",
      type: "document",
      x: 120,
      y: 120,
      width: 520,
      height: 360,
      title: "Brief",
      content: { markdown: "Collect the important findings here." }
    }
  ]
});

export function App() {
  return (
    <AgentCanvas
      document={document}
      theme="system"
      snap={{
        enabled: true,
        grid: { enabled: true, size: 24 },
        alignment: { enabled: true, targets: ["edge", "center"] }
      }}
    />
  );
}

AgentCanvas exposes a small imperative handle:

• getSnapshot() returns the full document, viewport, selection, and visible node ids.
• applyOperations(operations) applies validated create/update/delete/select/focus operations.
• getAgentContext() returns compact structured context for an AI prompt.
• fitView() and focusNode(id) control the camera.

The pure helpers in src/lib/core can also run on a server before broadcasting updated snapshots to clients.

AgentCanvas supports theme="system", theme="light", and theme="dark". The default is system, which follows prefers-color-scheme.

Dragging an unselected node pans the canvas. Click a node first to select it, then drag the selected node to move it. A single selected, unlocked node shows corner resize handles.

Website preview iframes are inert until their node is selected, so canvas pan and wheel zoom still work when the cursor is over an embedded design. Select the website node before interacting with the preview itself.

Snapping is opt-in for package consumers:

<AgentCanvas
  document={document}
  snap={{
    enabled: true,
    thresholdPx: 8,
    showGuides: true,
    grid: { enabled: true, size: 24 },
    alignment: { enabled: true, targets: ["edge", "center"], includeSections: true }
  }}
  resize={{
    enabled: true,
    handles: ["nw", "ne", "sw", "se"],
    minWidth: 180,
    minHeight: 120
  }}
/>

Snapping and resizing do not change the document schema. Drag and resize interactions commit standard updateNode changes to x, y, width, and height, so agents and host apps can use the same operation API.

Use section nodes to organize the canvas. A section is a real node with its own position, size, renderer, selection, z-order, and resize handles. Child membership is stored on each child as parentId; there is no separate children list to keep synchronized.

{
  "type": "createSection",
  "section": {
    "id": "design-section",
    "type": "section",
    "x": 80,
    "y": 80,
    "width": 1200,
    "height": 760,
    "content": { "label": "Design screens" }
  }
}

{ "type": "setNodeParent", "id": "home-screen", "parentId": "design-section" }

Nodes inside a section keep local x and y coordinates relative to the section. Moving a section moves its children. layoutSection lays out direct children using local section coordinates. getAgentContext() includes sections with bounds, child ids, visible child ids, and child counts so agents can target sections without scanning the full document.

For the demo and local agent workflows, start the canvas and the control server in separate terminals:

npm run dev
npm run control

The control server listens on 127.0.0.1:8787. Agents can post the same CanvasOperation[] records used by React:

npm run agent -- create-text --id agent-note-1 --title "Agent note" --text "Added from the CLI" --avoid-overlap

When installed as a package, the same commands are available as agent-canvas-control and agent-canvas.

Or send raw operations:

npm run agent -- send '[{"type":"updateNode","id":"agent-note-1","patch":{"content":{"tone":"note","text":"Updated in realtime"}}}]'

The demo listens to GET /events with Server-Sent Events and applies POST /operations payloads immediately.

For AI clients that support MCP, Agent Canvas also ships a stdio MCP server. Start the browser control bridge:

npm run control

Then add this MCP server to ~/.codex/config.toml:

[mcp_servers.agent-canvas]
command = "node"
args = ["/Users/shaun/Developer/Projects/agent-canvas/scripts/mcp-server.mjs"]
env = { AGENT_CANVAS_CONTROL_URL = "http://127.0.0.1:8787" }

Restart Codex after editing the config. For an installed package, use:

[mcp_servers.agent-canvas]
command = "npx"
args = ["-y", "-p", "@agent-canvas/react", "agent-canvas-mcp"]
env = { AGENT_CANVAS_CONTROL_URL = "http://127.0.0.1:8787" }

The MCP server exposes tools for reading context, creating nodes and sections, updating nodes, streaming document content, reparenting nodes, focusing nodes, and applying layouts. See docs/MCP.md for setup details and the full tool list.

Use the native Agent Canvas document shape in the host app. Avoid long-lived adapters from older frames or whiteboard schemas unless you need a staged migration.

1. Build this package:

   npm run build

2. Install it into the host app:

   npm install /Users/shaun/Developer/Projects/agent-canvas

3. Import the component and CSS:

   import { AgentCanvas } from "@agent-canvas/react";
   import "@agent-canvas/react/styles.css";

4. Store canvas JSON like this:

   {
     "schemaVersion": 1,
     "id": "family-blog",
     "title": "Family.blog planning board",
     "width": 2800,
     "height": 3600,
     "nodes": []
   }

5. Replace old routes and scripts around frames with native node operations:

   • POST /api/nodes
   • PATCH /api/nodes/:id
   • DELETE /api/nodes/:id
   • POST /api/operations with createNode, createSection, updateNode, deleteNode, setNodeParent, layoutSection, bringToFront, sendToBack, select, focus, and setViewport

6. Give agents these rules:

   • Use native node types: document, text, image, video, website, file, section.
   • Put nodes inside sections with parentId; do not maintain a separate child list.
   • Put app-specific details in metadata, not in custom top-level fields.
   • Use custom React renderers for app-specific presentation while keeping the data schema portable.
   • Call getAgentContext() before deciding what to change.

See AGENTS.md for a more operational checklist.

tldraw is excellent and much deeper. This package is intentionally narrower: first-class AI artifacts, permissive package ownership, host-controlled persistence, and a smaller API surface that can be embedded in other agent tools. A future adapter could render Agent Canvas records inside tldraw, but the core schema here stays independent.