[PHP] Add streaming tar.zst WordPress bundle extraction - #3925
Merged
Conversation
adamziel
force-pushed
the
adamziel/tar-zst-runtime-extraction
branch
from
July 7, 2026 11:48
f1cb4aa to
2d8a456
Compare
adamziel
marked this pull request as ready for review
July 7, 2026 11:48
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds runtime support for WordPress core bundles shipped as solid .tar.zst archives by sniffing magic bytes and extracting via a bounded-memory streaming TAR parser, while keeping existing ZIP extraction behavior and public API names intact.
Changes:
- Add streaming TAR parser + MEMFS extractor and a
zstddecfallback decoder (with nativeDecompressionStream('zstd')when available). - Route
unzipWordPress()to either.tar.zststreaming extraction or existing ZIP path based on magic bytes, with optional file-count parity checking. - Add focused unit tests for streaming parsing/extraction edge cases and wire expected bundle file counts through the remote worker + boot flows.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/playground/wordpress/vite.config.ts | Includes .tar.zst bundles as Vite assets for build/runtime availability. |
| packages/playground/wordpress/src/streaming-tar-extract.ts | Implements streaming tar parsing, safe path handling, zstd decoding, and MEMFS extraction. |
| packages/playground/wordpress/src/streaming-tar-extract.spec.ts | Adds unit tests for parser/extractor correctness and security edge cases. |
| packages/playground/wordpress/src/legacy-wp/legacy-boot.ts | Passes expected file-count parity option through legacy boot path. |
| packages/playground/wordpress/src/index.ts | Routes unzipWordPress() by magic bytes and exports new streaming utilities. |
| packages/playground/wordpress/src/boot.ts | Documents wordPressZip as bundle, adds wordPressBundleFileCount option, and passes it through. |
| packages/playground/remote/src/lib/playground-worker-endpoint-blueprints-v1.ts | Supplies expected file count for tar.zst descriptors and renames downloaded file to wp.bundle. |
| package.json | Adds zstddec dependency needed for zstd streaming decode fallback. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+197
to
+201
| if (this.zeroBlocks === 0) { | ||
| throw new Error( | ||
| 'Truncated tar stream: missing end-of-archive marker' | ||
| ); | ||
| } |
Comment on lines
+379
to
+380
| // @ts-ignore -- no bundled type resolution for the zstddec/stream subpath | ||
| const { ZSTDDecoder } = await import('zstddec/stream'); |
adamziel
force-pushed
the
adamziel/tar-zst-runtime-extraction
branch
3 times, most recently
from
July 7, 2026 12:29
629d334 to
fa60a18
Compare
adamziel
force-pushed
the
adamziel/tar-zst-runtime-extraction
branch
from
July 7, 2026 20:43
fa60a18 to
e68741d
Compare
adamziel
force-pushed
the
adamziel/tar-zst-runtime-extraction
branch
from
July 7, 2026 21:14
007c129 to
c458d81
Compare
adamziel
added a commit
that referenced
this pull request
Jul 7, 2026
## What it does Related to #3913. Replaces the committed minified WordPress core bundle artifacts from `wp-*.zip` to deterministic `wp-*.tar.zst` files. This PR builds on the already-merged runtime/parser work: #3926, #3927, and #3925. It does not change non-core ZIP flows. WordPress.org downloads, custom URLs, `trunk`/`nightly`, plugins, themes, GitHub artifacts, and Blueprint ZIP imports continue to use ZIP. ## Rationale After #3916 removed the duplicated embedded `wordpress-static.zip`, the remaining minified WordPress tree compresses better as a solid zstd-compressed tar than as ZIP. The runtime can now stream-decode zstd and parse TAR entries incrementally, so the committed core bundle can switch formats without changing the public boot API. Keeping remote and user-supplied archives as ZIP avoids expanding the scope of this change beyond the committed Playground-managed core bundles. ## Implementation Adds a deterministic core-bundle build path that: - re-containers the minified WordPress ZIP build output into `wp-<version>.tar.zst`, - writes `format`, `container`, `codec`, `size`, `sha256`, and `fileCount` descriptors, - verifies committed tar.zst artifacts against those descriptors, - fails loudly for unsafe source archive paths instead of silently dropping them, - removes committed `wp-*.zip` artifacts for minified core versions, and - teaches Vite/offline asset handling about `.tar.zst` and hashed `.zst` files. The generated descriptor still marks `trunk`/`nightly` as ZIP because those versions are fetched directly from GitHub as ZIP archives instead of using committed Playground core artifacts. ## Testing instructions ```bash node packages/playground/wordpress-builds/build/build-tar-zst.mjs --verify npx nx test playground-wordpress-builds npx nx test playground-wordpress npx nx test playground-remote npx nx typecheck playground-remote npx nx run-many -t lint -p playground-wordpress-builds,playground-remote,playground-wordpress ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What it does
Adds the runtime extraction path for
tar.zstWordPress core bundles while keeping the existingwordPressZip/unzipWordPress()API names.This PR does not switch committed WordPress core bundle artifacts to
tar.zst. ZIP inputs from wordpress.org, custom URLs, trunk/nightly downloads, plugins, themes, Blueprints, and GitHub artifacts stay on the existing ZIP path.Rationale
#3926 added the TAR parser and #3927 added the zstd decoder stream. The next step for #3913 is making WordPress boot able to consume a
tar.zstcore bundle before changing the produced bundle artifacts.Keeping runtime support separate from the artifact switch lets reviewers validate the extraction behavior, API compatibility, and ZIP fallback path independently.
Implementation
unzipWordPress()now sniffs the first four bytes of the suppliedFile:28 b5 2f fd) routes throughcreateDecodedTarStream(..., 'zstd').extractTarStreamToPhp()one entry at a time.unzipFile()/ PHPZipArchive.wordPressBundleFileCountenables a tar.zst-only file-count parity check when bundle metadata provides one.The extractor writes files under
/tmp/unzipped-wordpress, rejects unsafe TAR entries through the parser, preservesoverwriteFiles=false, and handlestargetRoot='/'as an absolute root instead of producing relative paths.Testing instructions
npx nx test playground-wordpress --testFile=streaming-tar-extract.spec.ts npx nx run playground-wordpress:lintCI should run the broader package, typecheck, and browser/CLI coverage for the rebased PR.