Add e2e coverage for the Sandbox (Playground) runtime - #3893
Merged
Conversation
bcotrim
marked this pull request as ready for review
June 18, 2026 19:58
fredrikekelund
approved these changes
Jun 22, 2026
fredrikekelund
left a comment
fredrikekelund
left a comment
Contributor
There was a problem hiding this comment.
Good idea 👍 I suggest adding two HTTP requests to the site home page (one for each runtime), but other than that, this LGTM
Comment on lines
+29
to
+36
| // The site boots under the Playground runtime. | ||
| const siteContent = new SiteContent( session.mainWindow, siteName ); | ||
| await expect( siteContent.runningButton ).toBeAttached( { timeout: 180_000 } ); | ||
| await expect( siteContent.siteNameHeading ).toHaveText( siteName ); | ||
|
|
||
| // The Settings tab reports the site as running on the Sandbox runtime. | ||
| const settingsTab = await siteContent.navigateToTab( 'Settings' ); | ||
| await expect( settingsTab.phpRuntimeDisplay ).toContainText( 'Sandbox' ); |
Contributor
There was a problem hiding this comment.
Requesting the site home page and looking for a 200 HTTP status code response seems like a nice addition here.
1 task
bcotrim
added a commit
that referenced
this pull request
Jun 22, 2026
## Related issues - Related to #3893 (introduced the failing test) ## How AI was used in this PR Diagnosed the CI break with Claude Code (Opus 4.8): traced the Buildkite **Lint** step's `npm run typecheck` failure to a cross-file overload mismatch in the newly-added Sandbox e2e test, then verified the fix with `tsc` and ESLint on the changed file. ## Proposed Changes PR #3893 added the first e2e test for the Sandbox (Playground) runtime, but it called `navigateToTab( 'Settings' )` with a capitalized label. The `SiteContent` page object only accepts the lowercase tab ids (`'settings'`, `'import-export'`, `'preview'`), so no overload matched and typecheck failed — turning the **Lint** CI step (which runs `npm run typecheck`) red on trunk: ``` sandbox.test.ts(35,56): error TS2769: No overload matches this call. sandbox.test.ts(36,29): error TS2339: Property 'phpRuntimeDisplay' does not exist on type 'never'. ``` The second error cascaded from the first (no matching overload → `never` return type). The capitalized value would also have failed at runtime, since the tab selector resolves to `[role="tab"][id$="-settings"]`. This restores a green build by using the correct lowercase `'settings'` tab id. No product code changes. ## Testing Instructions 1. `npm run typecheck` — passes (no `sandbox.test.ts` errors). 2. `npx eslint apps/studio/e2e/sandbox.test.ts` — clean. 3. CI **Lint** + **E2E Tests** steps go green on this branch. ## Pre-merge Checklist - [x] Have you checked for TypeScript, React or other console errors? — `tsc` and ESLint are clean on the changed file; this PR specifically fixes the TypeScript break.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issues
No linked issue. Addresses the gap where the Sandbox (WordPress Playground) runtime had no end-to-end coverage after Native PHP became the default site runtime in #3786.
How AI was used in this PR
Investigated the e2e harness, the Add Site modal UI, and the runtime plumbing, then wrote the test and page-object helpers with Claude Code (Opus 4.8). Selectors were checked against the real components (the
#php-runtime-selectdropdown and the Settings "PHP runtime" row), the additivecompleteOnboardingchange was confirmed backward-compatible with all existing callers, and ESLint +tscwere run clean on the changed files.Proposed Changes
Studio supports two per-site PHP runtimes: Native PHP (the default since #3786) and the Sandbox (WordPress Playground / PHP-WASM). Every existing e2e test runs against the default runtime, so the Sandbox path had zero end-to-end coverage — and site/blueprint creation on Playground lost the implicit coverage it had back when Playground was the default.
This adds a single e2e test that exercises the Sandbox runtime through the real UI:
It runs as part of the normal e2e suite on every platform/arch — no dedicated CI job and no gating. The supporting page-object additions (runtime selection, the runtime display row) are reusable for future runtime-specific tests. No product code is changed.
Testing Instructions
npx playwright test sandbox.test.tsagainst an existing build)Pre-merge Checklist
--fixandtscare clean for the changed e2e files. The test is statically verified but not yet run green; please run it locally before merging.