Skip to content

Fix flaky "below the fold" dev-toolbar perf audit e2e test#17192

Open
eldardada wants to merge 1 commit into
withastro:mainfrom
eldardada:fix/dev-toolbar-audit-e2e-flake
Open

Fix flaky "below the fold" dev-toolbar perf audit e2e test#17192
eldardada wants to merge 1 commit into
withastro:mainfrom
eldardada:fix/dev-toolbar-audit-e2e-flake

Conversation

@eldardada

Copy link
Copy Markdown

Changes

Hardens a flaky e2e test: Dev Toolbar - Audits › can warn about perf issue for below the fold image in absolute container.

The audit gates highlight creation on the image's load event (apps/audit/index.ts awaits load so the above/below-the-fold position checks have accurate layout). The test clicked the audit button and immediately asserted toHaveCount(1) with the default 6s expect timeout. When the optimized image loads slowly (e.g. a contended CI runner), the highlight is not created in time and the assertion intermittently sees 0 highlights, failing every retry.

The test now waits for the image to finish loading before triggering the audit, so the assertion no longer races the image load. No production code changes.

This flaked (all 3 retries) on unrelated CI, e.g. https://github.com/withastro/astro/actions/runs/28129089505/job/83301140796

Testing

  • pnpm --filter astro exec playwright test dev-toolbar-audits.test.ts runs 11 passing, executed twice for stability.
  • The targeted test passes locally. The change only adds a wait before the audit runs, so it cannot make the assertion less reliable.

Docs

No docs changes; this is test-only. No changeset, since there are no published package changes.

@changeset-bot

changeset-bot Bot commented Jun 24, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 93f6618

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions Bot added the pkg: astro Related to the core `astro` package (scope) label Jun 24, 2026
@eldardada

eldardada commented Jun 25, 2026

Copy link
Copy Markdown
Author

This is a test-only fix for a flaky e2e test (below the fold dev-toolbar perf audit) that fails on unrelated CI runs across the repo. Merging it should reduce spurious CI failures for everyone working on the project.

cc @ematipico - could you take a look when you have a moment?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pkg: astro Related to the core `astro` package (scope)

1 participant