Skip to content

Proposal: stream a solid tar.zst WordPress core bundle instead of per-entry ZIP #3913

Description

@erseco

Proposal for maintainer feedback before any PR — asking whether this direction is wanted
and how you'd want it done, not "please merge". A complete, tested reference implementation
with green CI exists as a draft PR on my fork:
erseco/wordpress-playground#3.
Happy to adapt it to whatever you prefer.

Summary

Today each minified WordPress version ships as a per-entry DEFLATE ZIP
(packages/playground/wordpress-builds/src/wordpress/wp-<version>.zip), fetched in the browser
and extracted into the PHP-WASM MEMFS by PHP ZipArchive
(unzipFile()unzipWordPress()).

The proposal is to re-container that core bundle as a single solid tar.zst and extract it
by streaming: zstddec (WASM) decode → incremental USTAR/GNU-longlink parser →
php.writeFile() into MEMFS, one entry at a time. The full uncompressed tar is never
materialized. unzipWordPress() sniffs the bundle's magic bytes (zstd vs PK), so every ZIP
path stays unchanged
(CLI wordpress.org releases, custom URLs, nightly/trunk master.zip,
plugin/theme installs, the SQLite plugin, the wordpress-static.zip backfill).

Why (measured, not theory)

All local, cold, median of 5 (Apple Silicon, Node 26; extraction into a real PHP-WASM MEMFS via
@php-wasm/node, PHP 8.3):

WordPress ZIP tar.zst Size Δ ZIP extract (ZipArchive) tar.zst extract Extract Δ
6.3 3.43 MiB 2.07 MiB −39.6 % 63 ms 22 ms −65 %
6.8 23.61 MiB 19.88 MiB −15.8 % 91 ms 33 ms −64 %
6.9 23.64 MiB 19.87 MiB −15.9 % 98 ms 39 ms −60 %
7.0 26.61 MiB 22.58 MiB −15.2 % 106 ms 39 ms −63 %

Real browsers (Playwright). In-browser tar.zst JS extraction, median 5:
Chromium 26 ms / Firefox 116 ms / WebKit 25 ms (6.9). Firefox's decode is ~4.5× slower than
the others — the same engine gap that makes ZipArchive 701 ms on Firefox today — so Firefox
benefits most.

Real network-throttled download (Chromium DevTools Network.emulateNetworkConditions),
ZIP vs tar.zst:

Link WordPress ZIP tar.zst Saved
40 Mbps 6.9 4.99 s 4.20 s −0.79 s (−15.8 %)
8 Mbps (DSL/4G) 6.9 24.85 s 20.92 s −3.93 s (−15.8 %)
8 Mbps (DSL/4G) 7.0 27.98 s 23.76 s −4.22 s (−15.1 %)

On a real 8 Mbps link the smaller bundle alone saves ~4 s of download per cold boot, on top
of the faster extraction — the download saving dominates on slow links.

Already adopted & merged in sibling Playground forks

This isn't a first attempt — the exact same "streaming tar.zst core bundle" change is
already merged in four sibling WebAssembly-PHP Playground projects built on the same
@php-wasm stack, where the payoff was larger (mostly-uncompressed PHP trees compress far
better than WordPress's already-compressed assets):

So the mechanism (deterministic USTAR + GNU longlink, zstddec streaming decode, incremental
parser, MEMFS writes, file-count parity) is battle-tested across four codebases; this proposal
is about whether it's worth bringing to WordPress specifically, where the gains are smaller but
still real.

Bonus finding (bigger lever, orthogonal)

The core bundle embeds an ~18 MiB wordpress-static.zip as its single largest file — it's
redundant with public/wp-<v>/wordpress-static.zip. Removing it from the core bundle would
roughly halve the core download
, independent of the tar.zst switch. It's also why the
streaming decoder's peak JS buffer is ~18 MiB (bounded by that one entry) rather than a few
hundred KB. Worth considering on its own or together with this.

Honest trade-offs / open questions for maintainers

  1. Committed binaries. The reference impl commits the wp-<v>.tar.zst (like the current
    .zip) because vite resolves them via ?url at build time. Would you prefer generating them
    in CI/Docker and not versioning them? (That needs a pre-build step so ?url resolves.)
  2. New dependency: zstddec (~76 KB WASM). No shipping browser exposes
    DecompressionStream("zstd").
  3. Ripple: assetsInclude, the offline precache exclusion, and the descriptor
    (getWordPressModuleDetails gains format/container/codec/sha256/fileCount).
    buildVersion-keyed SW caching handles invalidation.
  4. Scope: this converts only the core boot bundle. The wordpress-static.zip backfill and
    the PHP .wasm runtime are intentionally left alone (the runtime is fetched via
    instantiateStreaming and is not a tar.zst candidate — wrapping it would break
    stream-compilation; server-side Content-Encoding is the right lever there).

Honest verdict

Real and consistent — smaller and faster on all three engines — but modest in absolute
terms
for WordPress's small, fast-booting bundle (the sibling Moodle/Nextcloud forks saw
−50…−64 %; WordPress is ~−15 % because it's mostly already-compressed assets). The clearest wins
are slow networks (~4 s at 8 Mbps) and Firefox (~4.5× faster extraction). Combined with
dropping the embedded static zip, it becomes a compelling download reduction.

Reference implementation

A complete, tested working branch exists as a draft PR against my fork:
erseco/wordpress-playground#3
deterministic USTAR+GNU-longlink build, streaming extractor (path-traversal/symlink safe,
file-count parity), descriptor changes, assetsInclude/offline updates, unit tests, docs, and
the reproducible benchmark scripts behind the numbers above.

Would you welcome a PR along these lines, and if so, which of the open questions (esp. 1.,
committed vs generated binaries) should it settle first?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions