Skip to content

apps/ui: annotate toolbar for the site preview - #3247

Merged
youknowriad merged 2 commits into
trunkfrom
claude/brave-cartwright-bcbaeb
Apr 29, 2026
Merged

apps/ui: annotate toolbar for the site preview#3247
youknowriad merged 2 commits into
trunkfrom
claude/brave-cartwright-bcbaeb

Conversation

@youknowriad

@youknowriad youknowriad commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

Related issues

  • Builds on the design from Add /annotate skill with Studio inspector #3243 (the /annotate skill in apps/cli) and ports the same idea into the apps/ui site preview, so users can annotate elements directly from the in-app preview and feed them straight into the chat composer.

How AI was used in this PR

Heavy AI involvement — Claude Code drove the initial design + implementation across many iterations (iframe → <webview>WebContentsView), and I worked through several real bugs with it (popup-closing race, scroll freeze, navigation-policy block, native overlay border-radius). Codex then reviewed and hardened the preview/inspector architecture. Worth a careful human read of:

  • The native-view ↔ React renderer wiring in apps/studio/src/preview-view.ts + apps/ui/src/components/site-preview/index.tsx. Bounds tracking via ResizeObserver/scroll, view lifecycle, IPC routing of inspector events keyed by viewId, and replaying the latest requested path after async view creation.
  • The preview navigation policy in apps/studio/src/index.ts + apps/studio/src/preview-view.ts. Preview webContents are scoped to the owning local site origin; external navigations/popups/redirects are blocked from loading in the privileged preview view and opened externally only for HTTP(S) URLs.
  • The injected inspector runtime in apps/studio/src/preview-inspector-script.ts. Studio main owns this script; the renderer no longer passes executable code over IPC. The script is vanilla DOM in a Shadow DOM root and runs inside the WordPress page, so it needs to be defensive against theme CSS / global handlers.

Proposed Changes

Site preview rewrite (apps/ui + apps/studio)

  • Site preview now renders via Electron's WebContentsView instead of an <iframe> — needed for the annotate inspector to run inside the cross-origin WordPress page. The renderer renders a placeholder <div>; main process attaches the view to mainWindow.contentView and keeps its bounds in sync with the placeholder via IPC + ResizeObserver + scroll/resize listeners.
  • Native-view corner rounding via view.setBorderRadius so the preview corners match the React container's border-radius (the native overlay otherwise ignores parent CSS clipping).
  • New Connector.previewView slice (create / setBounds / navigate / destroy / onEvent) — connector-pattern friendly, with a plain <iframe> fallback for non-Electron connectors without inspector support.

Preview security/lifecycle hardening

  • createPreviewView now takes validated data (siteId, path, bounds, feature flags) instead of renderer-supplied executable script source.
  • Main process resolves preview URLs from SiteServer, validates paths against the local site origin/custom domain, and owns the inspector asset.
  • Preview IPC commands are tied to the creating renderer webContents.id; commands from other renderers are ignored.
  • Preview views are disposed when the owning renderer is destroyed, avoiding orphan native overlays after reloads/crashes/window teardown.
  • Preview navigations, popups, and redirects are constrained to the allowed site origin. External HTTP(S) URLs open outside Studio; non-web or malformed URLs are ignored.
  • Inspector injection checks the final committed preview URL before executing the script.

Inspector (main-owned injected page script)

  • Toolbar at the bottom-right of the preview (Annotate / count badge / Done), styled to match Add /annotate skill with Studio inspector #3243.
  • Annotate toggles picking. While picking is on, the inspector intercepts clicks (capture phase, preventDefault) and opens a comment popup near the clicked element. Saving stores the annotation, closes the popup, and stops picking (no auto-resume — auto-resume silently blocked link navigation in the preview).
  • Markers and highlight are positioned in document coordinates (position: absolute) so they scroll with the page with zero per-scroll JS — the earlier per-scroll rebuild approach froze the renderer on real WP pages.
  • One inspector → host event: { type: 'done', annotations } on Done click. The preload exposes only window.__studioInspector.send(payload), and main validates the event shape before forwarding it to the host renderer.

Composer integration (apps/ui/src/components/session-view)

  • Composer exposes an imperative appendDraft(text) via forwardRef + useImperativeHandle. SessionView holds a composerRef and calls composerRef.current.appendDraft(formatAnnotationsAsPrompt(annotations)) when a batch arrives. Not a controlled value prop — that re-rendered the entire SessionView (and the heavy Conversation tree) on every keystroke and was producing visible UI freezes.
  • formatAnnotationsAsPrompt groups annotations by page (preserving insertion order, merging consecutive same-page runs) so a batch from one page reads as one section, while navigate-annotate-navigate-annotate sequences keep each page's items under their own header. Each item carries the element tag + nearby text + CSS selector.

Preview empty state

  • Adds a Start-site button to the empty state (Start the site to see a live preview.) using useStartSite — saves the user a trip to the sidebar to start the site.

Files

  • New apps/studio/src/preview-view.tsPreviewView class wrapping WebContentsView with owner tracking, allowed-origin navigation policy, lifecycle registry, and inspector event forwarding.
  • New apps/studio/src/preview-preload.ts — small bridge exposing window.__studioInspector.send(payload) to the guest page; uses ipcRenderer.send with main-process validation and forwarding to the host renderer.
  • New apps/studio/src/preview-inspector-script.ts — main-owned vanilla DOM inspector runtime, moved out of apps/ui so renderer IPC no longer carries executable source.
  • New apps/ui/src/components/site-preview/types.ts — extracted Annotation interface.
  • Modified apps/studio/src/index.ts — preview-specific navigation/redirect/window-open policy.
  • Modified apps/studio/src/ipc-handlers.tscreatePreviewView / setPreviewViewBounds / navigatePreviewView / destroyPreviewView with site URL resolution and owner checks.
  • Modified apps/studio/electron.vite.config*.ts — bundle the new preview-preload.ts as a second preload entry point.
  • Modified apps/ui/src/components/session-view/composer/index.tsxforwardRef + appendDraft imperative API.
  • Modified apps/ui/src/components/session-view/index.tsx — wires composerRef and formatAnnotationsAsPrompt into the preview's onAnnotationsDone.

Testing Instructions

  1. npm install
  2. npm run start:new
  3. Open a session whose owner site is local + running (or start one from the new "Start site" button in the preview empty state).
  4. Toggle the site preview on (header drawer icon).
  5. Click Annotate in the bottom-right toolbar of the preview, click an element on the WordPress page, type a comment, click Save. Marker appears with a number badge.
  6. (Optional) Click Annotate again to chain another, possibly after navigating to another page in the preview by clicking a link.
  7. Click Done. The composer's draft is replaced/appended with a numbered list grouped by page, with element tag + nearby text + CSS selector for each annotation. Edit if needed and Cmd+Enter to send.
  8. Verify: scrolling the preview is smooth (markers move with the page natively); link navigation works; existing markers re-anchor sensibly after navigation.
  9. Verify navigation hardening: a same-site link should load in the preview; an external HTTP(S) link or popup should open in the system browser; non-web URLs should not load in the preview.

Pre-merge Checklist

  • No TypeScript / lint errors for touched workspaces (npm run typecheck -w apps/studio, npm run typecheck -w apps/ui, eslint --fix on touched files).
  • Production builds (vite build for @studio/ui; electron-vite build --config electron.vite.config.new-ui.ts).
  • Unit / E2E coverage — none added in this PR. The interaction surface lives inside an injected vanilla-DOM script + a native overlay, similar to Add /annotate skill with Studio inspector #3243; reviewers may want to push for Playwright coverage before landing.
  • Cross-platform (Windows/Linux) verification — only manually tested on macOS. The WebContentsView bounds-sync code uses getBoundingClientRect, which should match Electron's content coordinates on macOS but may need offset adjustments under Windows titleBarOverlay.
  • Have you checked for TypeScript, React or other console errors?

Notes for reviewers

  • The toolbar lives inside the injected page script (Shadow DOM) rather than as a React component. WebContentsView is a native overlay that always paints on top of HTML in the same window, so HTML overlays are no longer possible. If the toolbar should integrate with apps/ui's i18n / theme, the path forward is to render it in the preview header instead (above the view's bounds) or move the inspector runtime into the local WordPress site/iframe architecture.
  • About <webview>: the Electron docs at https://www.electronjs.org/docs/latest/api/webview-tag explicitly recommend against it (not formally deprecated, but discouraged). WebContentsView is one of the suggested alternatives.
  • An iframe-only implementation would require injecting the inspector runtime into the local WordPress page itself and communicating back with postMessage; that may be a cleaner long-term architecture, but it is a larger redesign than this PR.

🤖 Generated with Claude Code

Adds an annotate toolbar to the apps/ui site preview, modelled on
PR #3243. Lets the user click elements in the running WordPress site
inside the preview, attach comments, and ship the batch back to the
chat composer as a structured prompt.

The preview swaps from `<iframe>` (and a brief detour through
`<webview>`) to a native `WebContentsView` attached to the main
window's contentView. The renderer renders a placeholder div whose
bounds are reported to main via `ResizeObserver`, and the inspector
script — toolbar, picker, comment popup, marker rendering — is
injected into the guest page via `webContents.executeJavaScript` on
every successful navigation.

The site-preview's webContents is exempt from the global
`will-navigate` allow-origin policy in `index.ts` so it can load
arbitrary user-owned WordPress URLs; popups (`target="_blank"`) load
in-place via `setWindowOpenHandler`.

Also adds a Start-site button to the preview's empty state.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@youknowriad
youknowriad marked this pull request as ready for review April 26, 2026 11:43
@youknowriad youknowriad self-assigned this Apr 26, 2026
Comment thread apps/studio/src/index.ts
}
}

function openExternalWebUrl( url: string ): void {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this function needed don't we have similar things already?

@wpmobilebot

Copy link
Copy Markdown
Collaborator

📊 Performance Test Results

Comparing 6a31896 vs trunk

app-size

Metric trunk 6a31896 Diff Change
App Size (Mac) 1441.77 MB 1441.82 MB +0.05 MB ⚪ 0.0%

site-editor

Metric trunk 6a31896 Diff Change
load 1791 ms 1836 ms +45 ms ⚪ 0.0%

site-startup

Metric trunk 6a31896 Diff Change
siteCreation 8086 ms 8089 ms +3 ms ⚪ 0.0%
siteStartup 4954 ms 4958 ms +4 ms ⚪ 0.0%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)

@youknowriad
youknowriad requested a review from mtias April 26, 2026 14:40
@youknowriad
youknowriad merged commit 108ee66 into trunk Apr 29, 2026
13 checks passed
@youknowriad
youknowriad deleted the claude/brave-cartwright-bcbaeb branch April 29, 2026 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants