Add CLI integration test for creating a site from a custom Blueprint - #3961
Merged
Conversation
# Conflicts: # apps/cli/commands/site/tests/helpers/cli-e2e.ts
gcsecsey
marked this pull request as ready for review
June 26, 2026 16:45
epeicher
approved these changes
Jun 29, 2026
epeicher
left a comment
epeicher
left a comment
Contributor
There was a problem hiding this comment.
Thanks @gcsecsey for adding those tests! I have tested the changes locally, and they work as expected.
For the E2E test failures, I think these changes need the following PR to be merged: #3985. Once that PR is merged and rebased onto this one, if the E2E passes, these can be merged. I am accepting this to unblock you from that. Changes LGTM!
gavande1
approved these changes
Jun 29, 2026
gavande1
left a comment
gavande1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for working on this. LGTM.
Member
Author
gcsecsey
enabled auto-merge (squash)
June 29, 2026 11:11
Collaborator
📊 Performance Test ResultsComparing a0aa66d vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
1 task
gcsecsey
pushed a commit
that referenced
this pull request
Jul 1, 2026
…) (#4029) ## Related issues - Reverts #3961 - Reverts #4008 ## Proposed Changes Reverts two previously merged changes, combined into a single PR: - **#3961** — removes the CLI integration test for creating a site from a custom Blueprint, along with the appdata-isolation changes it made to the e2e harness. - **#4008** — re-enables the Windows Studio E2E Buildkite job, which had been temporarily disabled. The unrelated `STUDIO_PROCESS_MANAGER_HOME` env addition introduced by #3989 is preserved in the e2e helper. ## Testing Instructions - Confirm CI passes, including the re-enabled Windows E2E job. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?
This was referenced Jul 2, 2026
gcsecsey
added a commit
that referenced
this pull request
Jul 13, 2026
…nt (#4124) ## Related issues - Fixes STU-1871 - Re-adds #3961, reverted in #4029 ## How AI was used in this PR Claude investigated the revert history and the current e2e harness, restored the test from git history, adapted it to the updated harness conventions, and verified all suites locally. I reviewed all changes. ## Proposed Changes PR #3961 added CLI e2e coverage for `studio site create --blueprint`, but was reverted in #4029 as part of the AINFRA-2588 Windows E2E hang investigation. The actual causes were later found and fixed in #4082. The blueprint tests were not at fault, so this PR re-adds them, updated for the harness as it exists today: - The 4 original cases (installTheme, installPlugin, runPHP, wp-cli) now create sites with `--runtime sandbox`, matching the other CLI e2e suites: it keeps runs hermetic, since native PHP would download its binary into the isolated config dir on first run. - Applying a Blueprint boots the process-manager daemon even with `--no-start`, and since #3989 each test env gets an isolated daemon — so teardown now runs `site stop --all` before removing the env, like the start/stop and site-management suites, to avoid orphaning a daemon per case. - Re-adds #3961's harness fix: `runCli` isolates the legacy Electron appdata dir (`E2E` + `E2E_APP_DATA_PATH`). Without it, the Studio-compatibility startup migration finds a developer's real pre-split `appdata-v1.json` and exits 1, failing every CLI e2e suite locally (CI passes only because agents have no legacy config). This also stops e2e runs from bumping production stats. ## Testing Instructions 1. Build the CLI: `npm run cli:build`. 2. Run the new suite: `npm test -- apps/cli/commands/site/tests/create-blueprint.e2e.test.ts --tagsFilter='e2e'` Confirm all four tests pass. 3. Confirm the fast suite is unaffected: `npm test -- apps/cli/commands/site/tests --tagsFilter='!e2e'` — the e2e cases are skipped. 4. Regression check for the harness fix — the sibling suites still pass: `npm test -- apps/cli/commands/site/tests/create.e2e.test.ts --tagsFilter='e2e'` (on a machine with a leftover `~/Library/Application Support/Studio/appdata-v1.json`, this fails without the harness change and passes with it). ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
How AI was used in this PR
I used Claude to review the existing Playwright blueprint tests and the new CLI e2e harness from #3947, plan the migration, and to write the test. I reviewd and verified all changes.
Proposed Changes
Blueprint coverage currently lives only in
apps/studio/e2e/blueprints.test.ts, which boots the entire desktop app through Playwright to create a site from a custom Blueprint and then inspects wp-admin — slow, flaky, and coupled to the UI. This PR continues the "Migrate Studio E2E tests to CLI" effort by exercising the same capability directly through the CLI.studio site create --blueprint <file>against an isolated config directory and asserts on the file system.appdata-v1.jsonand abort the CLI on a developer's machine. The harness now also isolates that path, which fixes the siblingcreate.e2e.test.tslocally.Testing Instructions
npm run cli:build.npm test -- apps/cli/commands/site/tests/create-blueprint.e2e.test.ts --tagsFilter='e2e'Confirm all four tests pass.
npm test -- apps/cli/commands/site/tests/create-blueprint.e2e.test.ts --tagsFilter='!e2e'The four tests are skipped.
npm test -- apps/cli/commands/site/tests/create.e2e.test.ts --tagsFilter='e2e'Pre-merge Checklist