Add /annotate skill with Studio inspector - #3243
Merged
Merged
Conversation
youknowriad
marked this pull request as ready for review
April 25, 2026 18:37
youknowriad
commented
Apr 25, 2026
| * | ||
| * Hand-off to the agent: clicking "Done" sets `window.__studioAnnotateDone`, | ||
| * which `waitForAnnotationsDone()` in inspector-inject.ts polls for. | ||
| */ |
Contributor
Author
There was a problem hiding this comment.
If this one becomes more complex we could consider extracting it to its own "react app" in the apps folder.
Contributor
Author
There was a problem hiding this comment.
Good shout — agreed. Right now the inspector is ~330 lines of vanilla DOM in a Shadow root, which is light enough to live as an injected string. If we start adding things like keyboard navigation, an annotations panel, or theme variants, hoisting it to a proper apps/inspector React app (built and shipped as a static bundle the CLI loads as a string) would be the natural next step.
This was referenced Apr 25, 2026
Drops the dependency on the agentation npm package + agentation-mcp server in favour of a vanilla-DOM inspector injected from the CLI. The original PR #3030 wired up the agentation library, but in practice a series of issues surfaced that all stem from depending on a third-party React component loaded from esm.sh: - Window-sizing mismatch hid the toolbar bottom-right corner on 13" MacBooks (1440×900 page viewport vs. window-with-chrome). - Closing the page didn't tear down chromium, leaving a phantom dock icon until force-quit. - The agentation toolbar showed layout-mode and settings buttons we don't need; hiding them required CSS :has() hacks against hash-suffixed class names. - Annotations only synced from page localStorage to the MCP server's SQLite DB on toolbar interaction, so the CLI saw "0 annotations" while the page badge showed a count. - There was no clear "I'm done" hand-off; the agent had to call agentation MCP tools and hope the server was caught up. The replacement is a single TypeScript file (page-script.ts) that exports a self-contained vanilla-DOM widget mounted into a Shadow DOM root. No React, no esm.sh, no external runtime deps. All page-CSS isolation is handled by the shadow root rather than CSS-modules tricks. UX changes vs. the agentation toolbar: - Two-button toolbar: "Annotate" and "Done", plus a count badge. Layout-mode, settings page, theme picker, color palette all gone. - Annotate is sticky — after Save, picking mode stays on so the user can chain annotations without re-clicking the button. - Done sends the batch to the CLI via window.__studioAnnotateDone, clears local state, and triggers an auto-close with a live 10s countdown toast so the user sees what's happening. - Picked elements get a blue outline; saved annotations get numbered markers; clicking a marker reopens the popup for edit/delete. - Esc cancels picking / closes the popup. CLI side (inspector-inject.ts): - Pinned 1280×800 window + viewport: null so the bottom-right toolbar is always inside the visible region regardless of host screen size. - bringToFront() so the macOS chromium window doesn't open behind the terminal. - page.on('close') and browser.on('disconnected') tear the whole chromium process tree down, plus an exit/SIGINT/SIGTERM/SIGHUP hook so quitting `studio ai` doesn't orphan a dock icon. - 10-second deferred close after Done so the page-side countdown has time to play out and the user can register the confirmation before the window vanishes. Skill flow (SKILL.md): - After wait_for_annotations returns, the agent is required to print a numbered summary in plain language and ask the user to confirm via AskUserQuestion BEFORE making any changes. - Each summary item identifies the element by tag + nearbyText (not selector), restates intent in the agent's own words, and quotes the user's verbatim comment. - For another round, the user re-runs `/annotate` rather than re-using the closed window. Removed: - agentation-mcp dev dependency - agentation MCP server entry from agent.ts - apps/cli/ai/agentation-inject.ts
youknowriad
force-pushed
the
studio-inspector-annotate-skill
branch
from
April 27, 2026 15:21
c41127b to
868bc16
Compare
Collaborator
📊 Performance Test ResultsComparing 99d7d56 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
borkweb
requested changes
Apr 27, 2026
- Exclude the inspector host element when counting same-tag siblings in buildSelector. Without this, our injected #__studio-inspector-host appears in parent.children and can cause an off-by-one nth-of-type index for top-level body div children. Conceptually wrong even when the index happens to align — we shouldn't count our own injected scaffolding as part of the user's site structure. Per @borkweb's review on PR #3243. - Constrain `wait_for_annotations` timeoutMinutes to 1..120 (integer). Previously z.number().optional() allowed 0, which would feed Playwright's `timeout: 0` (= no timeout, block forever) and lock the agent up. 120 minutes is generous enough for any realistic annotation session. Per @borkweb's review on PR #3243.
borkweb
approved these changes
Apr 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
trunkand replaces the third-party agentation toolbar with a self-contained Studio inspector. The first commit on this branch is the original @lezama commit, preserved for attribution; the second commit is the rewrite.How AI was used in this PR
Heavy AI involvement — Claude Code drove the rebase, the discovery of the issues with the agentation-based approach (window sizing, dock-icon leak, sync gaps, missing confirmation step), and the rewrite into a vanilla-DOM inspector. End-to-end behaviours were verified with Playwright scripts; screenshots from each iteration are in the session transcript.
Reviewers should focus on:
apps/cli/ai/inspector/page-script.ts— it runs in the host WordPress page so it needs to be defensive against any theme CSS / global handlers.apps/cli/ai/inspector/inspector-inject.ts— close-on-page-close + close-on-process-exit hooks are the bits I'd most want a second pair of eyes on.SKILL.md. The intent is that the agent never edits files based on annotations without an explicit yes from the user.Proposed Changes
This PR is a Proof-of-Concept-grade rework of the annotation skill. Marking as draft + Proof of Concept label per AGENTS.md guidance for substantial AI-generated features.
Removed
agentationnpm package dependency (loaded at runtime fromesm.shin Add visual annotation browser with Agentation #3030)agentation-mcpdev dependencyagentationMCP server entry inapps/cli/ai/agent.tsapps/cli/ai/agentation-inject.tsAdded —
apps/cli/ai/inspector/page-script.ts— self-contained vanilla-DOM inspector mounted into a Shadow DOM root. No React, noesm.sh, no external runtime deps.inspector-inject.ts— Playwright launch + inject +waitForAnnotationsDonehelper.Inspector UI (vs. agentation toolbar in #3030)
AnnotateandDone, plus a count badge. Layout-mode, settings page, theme picker, colour palette all gone.Annotateis sticky — afterSave, picking mode stays on so the user can chain annotations.Doneships the batch to the CLI viawindow.__studioAnnotateDone, clears local state, plays a live 10-second countdown toast, then the CLI auto-closes the chromium window.Esccancels picking / closes the popup.CLI lifecycle hardening
viewport: nullso the bottom-right toolbar is always inside the visible region regardless of host screen size.bringToFront()so the macOS window doesn't open behind the terminal.page.on('close')andbrowser.on('disconnected')tear down the whole chromium process tree.process.once('exit'|'SIGINT'|'SIGTERM'|'SIGHUP')hooks so quittingstudio aidoesn't orphan a dock icon.Doneso the user has time to register the confirmation before the window vanishes.Skill flow (
SKILL.md)wait_for_annotationsreturns, the agent must print a numbered summary in plain language and callAskUserQuestionto confirm before making any edits.nearbyText(not selector), restates the intent in the agent's own words, and quotes the user's verbatim comment./annotaterather than reusing the (now-closed) window.Testing Instructions
npm install && npm run cli:buildnode apps/cli/dist/cli/main.mjs ai/annotateagainst any local Studio site.Annotate/Donetoolbar appears in the headed chromium window. ClickAnnotate, click any element on the page, type feedback, clickSave. Picking mode should stay on — click another element and repeat without re-clickingAnnotate.Done. Confirm:10s … 1s … closing now…./annotateagain — a fresh window should open with no leftover annotations from the previous session.studio ai(Ctrl-C) while the inspector window is open — the chromium process should exit too.Pre-merge Checklist
npm run typecheck --workspace apps/cli,npx eslint --fixon the modified files).npm run cli:build).