feat: replace WordPress core bundle with streaming tar.zst

[erseco]

🔗 Proposed upstream for discussion: WordPress/wordpress-playground#3913
— this is the reference implementation behind that RFC.

What

Replace the WordPress core boot bundle (wp-<version>.zip, extracted with PHP
ZipArchive) with a single solid tar.zst archive extracted by streaming zstd decode
plus incremental USTAR/GNU-longlink parsing into PHP-WASM MEMFS. The full uncompressed tar is
never materialized.

The core bundle no longer uses PHP ZipArchive for extraction. Plugin/theme ZIP installation,
the unzip Blueprint step, wordpress.org full releases (CLI + custom-version boots), nightly/
trunk GitHub master.zip, the SQLite integration plugin, and the wordpress-static.zip
static-asset backfill are all unchanged — unzipWordPress() sniffs the bundle's magic
bytes (zstd 28 B5 2F FD vs ZIP PK\x03\x04) and routes accordingly.

Design notes + methodology + full benchmark data:
packages/docs/site/docs/developers/23-architecture/20-wordpress-core-bundle-tar-zst-design.md
(rendered at /developers/architecture/wordpress-core-bundle-tar-zst-design).

Why

• Smaller downloads. Solid zstd deduplicates across files where per-entry ZIP DEFLATE
  cannot: −15…−16 % for modern versions, −40 % for 6.3.
• Faster core extraction. ~2.5–2.8× faster than PHP ZipArchive (measured in Node
  PHP-WASM; the PoC measured the same on Chrome/Firefox/Safari).
• Bounded peak JS memory. The ~36 MiB uncompressed tree is never materialized in JS.
• One core-bundle format, one core extraction path.

Prior art — already merged in sibling Playground forks

The same "streaming tar.zst core bundle" change is already merged in four sibling
WebAssembly-PHP Playground projects on the same @php-wasm stack (where the payoff is larger,
since those are mostly-uncompressed PHP trees):

• Moodle — ateeducacion/moodle-playground#176
• Omeka S — ateeducacion/omeka-s-playground#114
• Nextcloud — ateeducacion/nextcloud-playground#36
• FacturaScripts — erseco/facturascripts-playground#87

So the mechanism (deterministic USTAR+GNU-longlink, zstddec streaming decode, incremental
parser, MEMFS writes, file-count parity) is battle-tested across four codebases; this PR adapts
it to WordPress, where the gains are smaller but still real.

Benchmark summary

All numbers below are measured locally (Apple Silicon macOS, Node v26.4.0, cold, median of
5). Full methodology + data: the tar.zst design notes under packages/docs/.

Size + extraction (PHP-WASM MEMFS via @php-wasm/node, PHP 8.3)

WordPress	ZIP size	tar.zst size	Size Δ	ZIP extraction (PHP ZipArchive)	tar.zst extraction	Extraction Δ
6.3	3.43 MiB	2.07 MiB	−39.6 %	63 ms	22 ms	−65 %
6.8	23.61 MiB	19.88 MiB	−15.8 %	91 ms	33 ms	−64 %
6.9	23.64 MiB	19.87 MiB	−15.9 %	98 ms	39 ms	−60 %
7.0	26.61 MiB	22.58 MiB	−15.2 %	106 ms	39 ms	−63 %

Real browser measurements (Playwright: Chromium / Firefox / WebKit 26.5)

In-browser tar.zst extraction (zstddec streaming decode → StreamingTarParser → JS
writes; the tar.zst path is pure JS), median of 5:

WordPress	Chromium	Firefox	WebKit	Peak JS buffer
6.9	26 ms	116 ms	25 ms	18.0 MiB
7.0	28 ms	129 ms	25 ms	20.4 MiB

Firefox's JS decode is ~4.5× slower than Chromium/WebKit — the same engine gap the PoC measured
for PHP ZipArchive (Firefox 701 ms), so Firefox benefits most.

Real network-throttled download (Chromium DevTools Network.emulateNetworkConditions) —
the slow-link benefit the smaller bundle buys:

Link	WordPress	ZIP	tar.zst	Saved
40 Mbps (broadband)	6.9	4.99 s	4.20 s	−0.79 s (−15.8 %)
40 Mbps (broadband)	7.0	5.61 s	4.77 s	−0.84 s (−14.9 %)
8 Mbps (DSL / 4G)	6.9	24.85 s	20.92 s	−3.93 s (−15.8 %)
8 Mbps (DSL / 4G)	7.0	27.98 s	23.76 s	−4.22 s (−15.1 %)

On a real 8 Mbps link the smaller bundle alone saves ~4 s of download per cold boot, on
top of the faster extraction — the download saving dominates on slow links.

PoC browser extraction for 6.9 into a real PHP-WASM MEMFS (measured): Chrome 142→60 ms (~2.3×),
Firefox 701→262 ms (~2.6×), Safari/WebKit 142→55 ms (~2.5×).

windowLog 25 (32 MiB) was chosen: same compression as 27 for these ≤40 MiB bundles, but a
bounded multi-segment sliding window instead of the single-segment whole-content buffer that
windowLog ≥ 26 forces on the decoder. windowLog 24 costs +1.7 MiB (+8 %) for 6.9.

Still pending preview: the composite per-engine end-to-end app-ready time (download +
extract + WASM compile + WP install for the whole site) — both variable components (download,
extraction) are measured above; the composite awaits the Cloudflare Pages build.

PHP runtime packaging investigation

Traced end-to-end (not a guess). The PHP-WASM runtime is not a tar.zst candidate and is
left unchanged:

• It arrives as a single uncompressed .wasm fetched via
  WebAssembly.instantiateStreaming(fetch(url)) — the glue's import '…/php_8_3.wasm' is
  turned into a hashed asset URL by esbuild/vite's file loader; the module stream-compiles
  during download.
• There is no Emscripten MEMFS .data preload package for the runtime (find over
  web-builds/node-builds for *.data is empty; the PHP compile uses EXPORT_NAME with no
  --preload-file). ICU (icu.dat, ~30 MiB) is loaded only when intl is requested and staged
  manually into MEMFS.
• No in-repo precompression; the only serving hint (remote/.htaccess) just sets the wasm MIME
  type. Transparent HTTP compression is left to the server.

Verdict: wrapping the .wasm in tar.zst would defeat instantiateStreaming (stream
compile), touches 8 web + 8 node version packages and the Emscripten build, and yields little
over an already-stream-compiled binary — not local, not low-risk, not worth it here.
Recorded as Future work: rely on server-side Content-Encoding: br/zstd for the .wasm.

How

• Build (packages/playground/wordpress-builds/build/):
  • lib/tar-ustar.mjs — deterministic USTAR + GNU ././@LongLink (never PAX), byte-wise sort,
    mtime=0, files-only.
  • build-tar-zst.mjs — re-container each committed wp-<v>.zip → wp-<v>.tar.zst
    (node:zlib zstd level 19 + LDM, windowLog 25); prints {fileCount, size, sha256};
    --all / --version / --verify / --descriptor-only.
  • lib/generate-module-details.mjs — shared descriptor code-generator.
  • build.js now delegates the tar.zst + descriptor step (transient ZIP → tar.zst → descriptor).
• Manifest (get-wordpress-module-details.ts, auto-generated): { size, url } →
  { format, container, codec, url, size, sha256, fileCount }. Remote nightly/trunk stay
  format: 'zip' (GitHub master.zip).
• Runtime (packages/playground/wordpress/src/streaming-tar-extract.ts):
  createDecodedTarStream, StreamingTarParser, extractTarStreamToPhp, sanitizeTarPath.
  zstddec (WASM) for zstd; native DecompressionStream for gzip/deflate/brotli. File-count
  parity vs the descriptor (fail loud).
• Cache/offline: assetsInclude accepts *.tar.zst; the offline precache manifest excludes
  the hashed core-bundle .zst assets (cached on-demand). SW cache invalidation is unchanged
  (buildVersion = git HEAD).
• Deps: add zstddec@^0.2.0. fflate and ZIP paths are untouched.
• CI: nightly bundle build → Node 24 (native node:zlib zstd); node --check for the new
  build scripts.

Security

Streaming extractor fails closed and is TAR-slip safe (unit-tested):

• Rejects absolute paths and .. traversal (throws).
• Normalizes \→/ before validation (Windows-style traversal).
• Skips empty/. segments.
• Rejects symlinks and other exotic entry types.
• Fails loudly on malformed/truncated archives (half-read entry throws).
• File-count parity against descriptor.fileCount.

Verification

Commands run (locally, Node v26.4.0):

node packages/playground/wordpress-builds/build/build-tar-zst.mjs --all      # generate bundles + descriptor
node packages/playground/wordpress-builds/build/build-tar-zst.mjs --verify   # sha256/size parity
node packages/playground/wordpress-builds/build/benchmark-tar-zst.mjs --runs=5           # size + JS throughput
node packages/playground/wordpress-builds/build/benchmark-tar-zst-browser.mjs --runs=5   # Chromium/Firefox/WebKit + throttled download
node --check <every new build .mjs/.js>
npx tsc --noEmit -p packages/playground/wordpress/tsconfig.lib.json          # + tsconfig.spec.json
npx tsc --noEmit -p packages/playground/wordpress-builds/tsconfig.lib.json
npx nx typecheck playground-remote
npx nx lint playground-wordpress playground-wordpress-builds playground-remote
npx nx test playground-wordpress            # streaming-tar-extract.spec + boot suites
npx nx test playground-wordpress-builds
npx nx build playground-wordpress-builds    # emits hashed .tar.zst assets

Results:

• build-tar-zst.mjs --all / --verify: 9 bundles generated; every sha256/size matches the
  descriptor. OK.
• Sizes: −15.2…−15.9 % (6.4–7.0/beta), −39.6 % (6.3). Extraction (PHP-WASM, median 5):
  2.5–2.8× faster.
• node --check on all new .mjs/.js: OK. tsc (lib+spec, wordpress & wordpress-builds):
  exit 0. nx typecheck playground-remote: success.
• Lint (wordpress, wordpress-builds, remote): pass.
• nx test playground-wordpress: streaming-tar-extract.spec.ts 26/26; the existing boot
  suites (version-detect for WP 6.3–7.0/beta/trunk, error-logging, most of database) boot
  WordPress from streaming tar.zst and pass. One pre-existing failure in
  database.spec.ts afterAll (rmdirSync({recursive:true}) removed in Node ≥ 24) — unrelated
  to this change, present on trunk.
• nx test playground-wordpress-builds: pass (descriptor test updated for the tar.zst shape).
• nx build playground-wordpress-builds: built, emits wp-<v>.tar-<hash>.zst assets + dts.
• Browser (Playwright, benchmark-tar-zst-browser.mjs): tar.zst extraction Chromium 26 ms /
  Firefox 116 ms / WebKit 25 ms (6.9); throttled download saves −0.79 s @ 40 Mbps and
  −3.93 s @ 8 Mbps vs ZIP (6.9). See the Benchmark summary.

Notes for review

• Targets only erseco/wordpress-playground. Opened as draft for preview review.
• The old src/wordpress/wp-<v>.zip core bundles are removed (the core boots from tar.zst;
  they were only the re-container build source, regenerated by the Docker rebuild). Net binary
  change ≈ −50 MiB (−200 MiB zip, +150 MiB tar.zst).
• Only pending measurement: the composite per-engine end-to-end app-ready time (whole-site
  boot) — its variable components (download −15 %, extraction) are already measured in real
  browsers above; the composite awaits the Cloudflare Pages build.
• Non-goals / Future work: converting the wordpress-static.zip static-asset backfill to
  tar.zst; native tar.zst emission from the Docker build; server-side Content-Encoding for
  the .wasm runtime.
• No changes proposed upstream to WordPress/wordpress-playground.