Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions apps/cli/ai/skills/block-content/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ Use this skill before writing or editing page content, post content, templates,
- No custom class names on inner DOM elements. Put custom classes only on the outermost block wrapper via the block `className` attribute.
- Style buttons via `.wp-element-button` — the inner element WordPress applies the button's padding, background, and border to (shared by the button block and buttons from other blocks). A custom class on a button block sits on the `.wp-block-button` wrapper, so descend to `.your-class .wp-element-button`; never style the wrapper directly, or its padding stacks on top of the default and the button doubles in size.
- No inline `style` attributes or block `style` attributes for styling. Use `className` plus the theme's `style.css`.
- Prefer theme palette colors over hardcoded hex. Apply block colors with palette **slug** attributes — `{"backgroundColor":"accent-1","textColor":"base"}` — and in `style.css` reference palette colors as `var(--wp--preset--color--<slug>)`. Discover the available slugs from the active theme's `theme.json` `settings.color.palette` (for a theme you are building, the palette you defined there); when you want a color the palette lacks, prefer adding it to the palette and referencing its slug. Keeping colors on the palette keeps sections in sync with Global Styles, theme switching, and light/dark variations. A raw hex value is fine for a deliberate one-off, but it should be the exception, not the default.
- Use `core/spacer` for empty spacing elements, not empty `core/group` blocks.
- No emojis anywhere in generated content.

Expand All @@ -43,8 +44,10 @@ For `style.css`, start with custom properties and anchor comments only:

```css
:root {
--site-bg: #ffffff;
--site-text: #111111;
/* Map section variables onto the theme palette — reference preset slugs,
never hardcode hex here. The slugs come from theme.json's color palette. */
--site-bg: var( --wp--preset--color--base );
--site-text: var( --wp--preset--color--contrast );
}

/* === reset === */
Expand Down
2 changes: 1 addition & 1 deletion apps/cli/ai/skills/visual-design/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ Build working code that is:
Focus on:

- **Typography**: Choose fonts that suit the concept. Avoid defaulting to generic choices like Arial, Inter, Roboto, or system fonts unless restraint is clearly part of the brief. Pair display and body typography intentionally.
- **Color and theme**: Commit to a palette. Use dominant colors and sharp accents deliberately instead of timid, evenly distributed colors.
- **Color and theme**: Commit to a palette and define it once in the theme's `theme.json` (`settings.color.palette`), then drive every section, block, and CSS rule from those palette colors by slug. Use dominant colors and sharp accents deliberately instead of timid, evenly distributed colors. When redesigning or adding sections to a site that already has an active theme, inherit its existing palette rather than inventing new custom colors. Treat the palette as the single source of truth — do not scatter hardcoded hex values across block markup or CSS; introduce a custom color only when the concept genuinely needs one the palette lacks, and add it to the palette first. See the `block-content` skill for how to reference palette colors.
- **Motion**: Use animation and transitions when they serve the concept. Prefer CSS where possible. A few well-orchestrated moments are better than scattered effects.
- **Spatial composition**: Use asymmetry, overlap, diagonal flow, grid-breaking elements, generous negative space, or controlled density when appropriate to the concept.
- **Backgrounds and visual details**: Add atmosphere and depth with contextual textures, patterns, shadows, borders, transparency, or custom visual treatments when they reinforce the direction.
Expand Down
1 change: 1 addition & 0 deletions apps/cli/ai/system-prompt.ts
Original file line number Diff line number Diff line change
Expand Up @@ -199,6 +199,7 @@ const REMOTE_CONTENT_GUIDELINES = `## Block content guidelines

- Use only core WordPress blocks. No custom HTML blocks except for inline SVGs.
- No decorative HTML comments (e.g. \`<!-- Hero Section -->\`). Only block delimiter comments are allowed.
- Color content from the active theme's palette using block color-slug attributes (e.g. \`{"backgroundColor":"primary","textColor":"base"}\`) rather than hardcoded hex values; only introduce a custom color when the palette genuinely lacks one.
Comment thread
youknowriad marked this conversation as resolved.
- No emojis anywhere in generated content.`;

const REMOTE_DESIGN_GUIDELINES = `## Design capabilities by plan
Expand Down
1 change: 1 addition & 0 deletions scripts/eval/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ npm run eval:view
- **screenshot-all-timing** — Agent creates a minimal site and visually verifies the homepage on desktop and mobile. Asserts the agent uses one `take_screenshot` call with `viewport: "all"`, returns valid desktop/mobile PNG payloads, and keeps the screenshot tool under 15s.
- **single-page-build-turn-cadence** — Agent builds a simple one-page site. Asserts (a) every individual turn stays under 60s (wall-clock between successive assistant messages) and (b) no `wp_cli` call uses `--post_content-file=` (which silently fails inside PHP-WASM).
- **jetpack-catchall-slideshow** — Agent reaches for Jetpack on a slideshow request. Asserts the generated page content uses a `jetpack/*` block (i.e. the catch-all rule fired instead of the agent falling back to raw HTML).
- **section-uses-theme-palette** — Agent adds sections to a site that keeps its default theme. Asserts the section block markup colors are drawn from the theme palette (color-slug attributes like `{"backgroundColor":"accent-1"}` or `var(--wp--preset--color--*)` in CSS) rather than hardcoded hex values. `theme.json` is excluded from the hex check since a palette is legitimately defined there.

## Adding tests

Expand Down
74 changes: 74 additions & 0 deletions scripts/eval/promptfoo.config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -374,3 +374,77 @@ tests:
const n = (d.toolCalls || []).filter(t => t.name === 'wp_cli' && /wc\s+product\s+create/.test(t.input?.command || '')).length;
return { pass: n >= 1, score: n >= 1 ? 1 : 0, reason: n >= 1 ? `${n} product(s) created` : 'no `wc product create` calls' };
});

# When adding sections to a site that already has an active theme, the agent
# should color them from the theme's color palette (block color-slug
# attributes like {"backgroundColor":"accent-1"} or var(--wp--preset--color--*)
# in CSS) instead of hardcoding hex values. Hardcoded hex detaches the section
# from Global Styles / theme switching / dark variations and is the behavior
# this case guards against. theme.json is excluded from the hex check because
# the palette is legitimately *defined* there with hex.
- description: added sections use the theme color palette, not hardcoded hex
vars:
caseId: section-uses-theme-palette
timeoutMs: 600000
askUserPolicy: allow_all
prompt: |
First check if a site named "Eval Palette Site" exists using
site_list. If it does, delete it with site_delete so we start from a
clean slate. Do NOT touch any other site.

Then create a new site named "Eval Palette Site". Keep its current
default theme — do not create, scaffold, or switch themes.

Add two visually distinct sections to the site's homepage: a hero
section with a colored background and a heading, and a call-to-action
section with a contrasting colored background and a button. Make the
colors feel cohesive with the site's existing theme.
assert:
- type: javascript
value: |
return import('node:fs').then(({ readFileSync }) => {
const marker = output.split(/\r?\n/).map(l => l.trim()).find(l => l.startsWith('EVAL_RUNNER_RESULT_FILE='));
if (!marker) return { pass: false, score: 0, reason: `no result-file marker on stdout; got: ${output.slice(0, 200)}` };
const d = JSON.parse(readFileSync(marker.slice('EVAL_RUNNER_RESULT_FILE='.length), 'utf8'));
const calls = d.toolCalls || [];
const pathOf = (i) => String((i && (i.file_path || i.path || i.filePath)) || '').toLowerCase();
// Collect every place a color can land. Block markup and theme CSS
// reach the site through Write/Edit (a scratch .html, style.css, or
// raw block markup) OR through wp_cli — both as post_content and as
// CSS injected via `wp_cli eval '$css = "..."'`. theme.json is the one
// exclusion: a palette is legitimately defined there with hex.
const blobs = [];
for (const c of calls) {
const i = c.input || {};
if ((c.name === 'Write' || c.name === 'Edit') && !pathOf(i).endsWith('.json')) {
for (const k of ['content', 'new_string', 'replacement', 'text']) {
if (typeof i[k] === 'string') blobs.push(i[k]);
}
// The Edit tool applies changes as { edits: [{ oldText, newText }] }.
if (Array.isArray(i.edits)) {
for (const e of i.edits) if (e && typeof e.newText === 'string') blobs.push(e.newText);
}
}
if (c.name === 'wp_cli' && typeof i.command === 'string') {
blobs.push(i.command);
}
}
const markup = blobs.join('\n');
if (!/<!--\s*wp:/.test(markup)) {
return { pass: false, score: 0, reason: `no block markup found in Write/wp_cli calls; tool calls: ${JSON.stringify(calls.map(c => c.name))}` };
}
// Hardcoded hex colors in section markup / theme CSS = custom colors (the bug).
const hex = markup.match(/#[0-9a-fA-F]{6}\b|#[0-9a-fA-F]{3}\b/g) || [];
// Palette references: block color-slug attributes or preset CSS vars.
const slugAttr = markup.match(/"(?:backgroundColor|textColor|borderColor|overlayColor|gradient)"\s*:\s*"[a-z][a-z0-9-]*"/g) || [];
const presetVar = markup.match(/var\(\s*--wp--preset--color--[a-z0-9-]+/g) || [];
const paletteRefs = slugAttr.length + presetVar.length;
if (hex.length > 0) {
const uniq = [...new Set(hex)].slice(0, 8);
return { pass: false, score: 0, reason: `section markup/CSS hardcodes ${hex.length} custom hex color(s) (${uniq.join(', ')}) instead of theme palette colors; palette refs found: ${paletteRefs}` };
}
if (paletteRefs === 0) {
return { pass: false, score: 0, reason: `sections apply no theme-palette colors (no color-slug attributes or --wp--preset--color-- vars) — cannot confirm the section is colored from the palette` };
}
return { pass: true, score: 1, reason: `sections colored from the theme palette (${paletteRefs} palette color ref(s)) with no hardcoded hex` };
});
Loading