apps/ui: annotate toolbar for the site preview - #3247
Merged
Conversation
Adds an annotate toolbar to the apps/ui site preview, modelled on PR #3243. Lets the user click elements in the running WordPress site inside the preview, attach comments, and ship the batch back to the chat composer as a structured prompt. The preview swaps from `<iframe>` (and a brief detour through `<webview>`) to a native `WebContentsView` attached to the main window's contentView. The renderer renders a placeholder div whose bounds are reported to main via `ResizeObserver`, and the inspector script — toolbar, picker, comment popup, marker rendering — is injected into the guest page via `webContents.executeJavaScript` on every successful navigation. The site-preview's webContents is exempt from the global `will-navigate` allow-origin policy in `index.ts` so it can load arbitrary user-owned WordPress URLs; popups (`target="_blank"`) load in-place via `setWindowOpenHandler`. Also adds a Start-site button to the preview's empty state. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
youknowriad
marked this pull request as ready for review
April 26, 2026 11:43
youknowriad
commented
Apr 26, 2026
| } | ||
| } | ||
|
|
||
| function openExternalWebUrl( url: string ): void { |
Contributor
Author
There was a problem hiding this comment.
Is this function needed don't we have similar things already?
Collaborator
📊 Performance Test ResultsComparing 6a31896 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
/annotateskill in apps/cli) and ports the same idea into the apps/ui site preview, so users can annotate elements directly from the in-app preview and feed them straight into the chat composer.How AI was used in this PR
Heavy AI involvement — Claude Code drove the initial design + implementation across many iterations (iframe →
<webview>→WebContentsView), and I worked through several real bugs with it (popup-closing race, scroll freeze, navigation-policy block, native overlay border-radius). Codex then reviewed and hardened the preview/inspector architecture. Worth a careful human read of:apps/studio/src/preview-view.ts+apps/ui/src/components/site-preview/index.tsx. Bounds tracking viaResizeObserver/scroll, view lifecycle, IPC routing of inspector events keyed byviewId, and replaying the latest requested path after async view creation.apps/studio/src/index.ts+apps/studio/src/preview-view.ts. PreviewwebContentsare scoped to the owning local site origin; external navigations/popups/redirects are blocked from loading in the privileged preview view and opened externally only for HTTP(S) URLs.apps/studio/src/preview-inspector-script.ts. Studio main owns this script; the renderer no longer passes executable code over IPC. The script is vanilla DOM in a Shadow DOM root and runs inside the WordPress page, so it needs to be defensive against theme CSS / global handlers.Proposed Changes
Site preview rewrite (
apps/ui+apps/studio)WebContentsViewinstead of an<iframe>— needed for the annotate inspector to run inside the cross-origin WordPress page. The renderer renders a placeholder<div>; main process attaches the view tomainWindow.contentViewand keeps its bounds in sync with the placeholder via IPC +ResizeObserver+scroll/resizelisteners.view.setBorderRadiusso the preview corners match the React container'sborder-radius(the native overlay otherwise ignores parent CSS clipping).Connector.previewViewslice (create/setBounds/navigate/destroy/onEvent) — connector-pattern friendly, with a plain<iframe>fallback for non-Electron connectors without inspector support.Preview security/lifecycle hardening
createPreviewViewnow takes validated data (siteId,path, bounds, feature flags) instead of renderer-supplied executable script source.SiteServer, validates paths against the local site origin/custom domain, and owns the inspector asset.webContents.id; commands from other renderers are ignored.Inspector (main-owned injected page script)
Annotatetoggles picking. While picking is on, the inspector intercepts clicks (capture phase,preventDefault) and opens a comment popup near the clicked element. Saving stores the annotation, closes the popup, and stops picking (no auto-resume — auto-resume silently blocked link navigation in the preview).position: absolute) so they scroll with the page with zero per-scroll JS — the earlier per-scroll rebuild approach froze the renderer on real WP pages.{ type: 'done', annotations }on Done click. The preload exposes onlywindow.__studioInspector.send(payload), and main validates the event shape before forwarding it to the host renderer.Composer integration (
apps/ui/src/components/session-view)Composerexposes an imperativeappendDraft(text)viaforwardRef+useImperativeHandle. SessionView holds acomposerRefand callscomposerRef.current.appendDraft(formatAnnotationsAsPrompt(annotations))when a batch arrives. Not a controlledvalueprop — that re-rendered the entire SessionView (and the heavy Conversation tree) on every keystroke and was producing visible UI freezes.formatAnnotationsAsPromptgroups annotations by page (preserving insertion order, merging consecutive same-page runs) so a batch from one page reads as one section, while navigate-annotate-navigate-annotate sequences keep each page's items under their own header. Each item carries the element tag + nearby text + CSS selector.Preview empty state
Start the site to see a live preview.) usinguseStartSite— saves the user a trip to the sidebar to start the site.Files
apps/studio/src/preview-view.ts—PreviewViewclass wrappingWebContentsViewwith owner tracking, allowed-origin navigation policy, lifecycle registry, and inspector event forwarding.apps/studio/src/preview-preload.ts— small bridge exposingwindow.__studioInspector.send(payload)to the guest page; usesipcRenderer.sendwith main-process validation and forwarding to the host renderer.apps/studio/src/preview-inspector-script.ts— main-owned vanilla DOM inspector runtime, moved out of apps/ui so renderer IPC no longer carries executable source.apps/ui/src/components/site-preview/types.ts— extractedAnnotationinterface.apps/studio/src/index.ts— preview-specific navigation/redirect/window-open policy.apps/studio/src/ipc-handlers.ts—createPreviewView/setPreviewViewBounds/navigatePreviewView/destroyPreviewViewwith site URL resolution and owner checks.apps/studio/electron.vite.config*.ts— bundle the newpreview-preload.tsas a second preload entry point.apps/ui/src/components/session-view/composer/index.tsx—forwardRef+appendDraftimperative API.apps/ui/src/components/session-view/index.tsx— wirescomposerRefandformatAnnotationsAsPromptinto the preview'sonAnnotationsDone.Testing Instructions
npm installnpm run start:newAnnotatein the bottom-right toolbar of the preview, click an element on the WordPress page, type a comment, clickSave. Marker appears with a number badge.Annotateagain to chain another, possibly after navigating to another page in the preview by clicking a link.Done. The composer's draft is replaced/appended with a numbered list grouped by page, with element tag + nearby text + CSS selector for each annotation. Edit if needed and Cmd+Enter to send.Pre-merge Checklist
npm run typecheck -w apps/studio,npm run typecheck -w apps/ui,eslint --fixon touched files).vite buildfor@studio/ui;electron-vite build --config electron.vite.config.new-ui.ts).WebContentsViewbounds-sync code usesgetBoundingClientRect, which should match Electron's content coordinates on macOS but may need offset adjustments under WindowstitleBarOverlay.Notes for reviewers
WebContentsViewis a native overlay that always paints on top of HTML in the same window, so HTML overlays are no longer possible. If the toolbar should integrate with apps/ui's i18n / theme, the path forward is to render it in the preview header instead (above the view's bounds) or move the inspector runtime into the local WordPress site/iframe architecture.<webview>: the Electron docs at https://www.electronjs.org/docs/latest/api/webview-tag explicitly recommend against it (not formally deprecated, but discouraged).WebContentsViewis one of the suggested alternatives.postMessage; that may be a cleaner long-term architecture, but it is a larger redesign than this PR.🤖 Generated with Claude Code