[PHP] Support custom PHP.wasm extensions - #3566
Conversation
d625a2c to
8dc2bd7
Compare
8dc2bd7 to
0a01f5d
Compare
## What it does
Adds a single `extensions` array to `loadNodeRuntime()` and
`loadWebRuntime()` for bundled and external PHP extensions.
```ts
await loadNodeRuntime('8.4', {
extensions: [
'intl',
'redis',
{ name: 'xdebug', options: { ideKey: 'PLAYGROUND' } },
{
source: {
format: 'manifest',
url: 'https://example.com/extensions/parser/manifest.json',
},
},
],
});
```
The previous loader flags remain valid as deprecated aliases:
```ts
await loadNodeRuntime('8.4', {
withIntl: true,
withRedis: true,
withXdebug: { ideKey: 'PLAYGROUND' },
});
await loadWebRuntime('8.4', { withIntl: true });
```
This PR is stacked on #3566 and keeps Blueprint/compiler changes out of
this layer.
## Rationale
The old `withIntl`, `withRedis`, `withMemcached`, and `withXdebug` flags
only model bundled extensions. External extensions need the same startup
path as bundled extensions: select the artifact for the running
PHP/async mode, stage the `.so`, write `.ini`, add sidecar files, and
set env before PHP starts.
Keeping the old flags avoids breaking existing Node/Web callers and CLI
behavior while giving new code one extensible API.
## Implementation
Adds `resolvePHPExtensionInstallPlan()` to `@php-wasm/universal` so
manifest/direct-source extensions can be resolved before a PHP instance
exists. `loadPHPExtension()` still uses the same resolver for
already-started runtimes.
Node and Web now have small runtime extension registries:
- Node built-ins: `intl`, `xdebug`, `redis`, `memcached`
- Web built-ins: `intl`
- External entries: any `LoadPHPExtensionOptions`-style object with
`source`
`applyPHPLoaderExtensions()` resolves all requested extensions with
`Promise.all()`, then applies the install plans in declaration order
through one `onRuntimeInitialized` wrapper.
Deprecated loader flags are normalized into the same `extensions` array
and deduped against explicit built-in entries. `@php-wasm/cli --xdebug`
still uses `makeXdebugConfig()` so path skipping behavior stays
unchanged.
## Testing instructions
```bash
npx nx typecheck php-wasm-universal
npx nx typecheck php-wasm-node
npx nx typecheck php-wasm-web
npx nx typecheck playground-cli
npx nx typecheck playground-remote
npx nx typecheck playground-client
npx nx typecheck php-wasm-cli
npx nx typecheck php-wasm-xdebug-bridge
npx nx test php-wasm-universal --testFiles=load-extension.spec.ts
cd packages/php-wasm/node
PHP=8.4 npx vitest run src/test/php-dynamic-loading.spec.ts src/test/php-worker.spec.ts --config vite.config.ts
```
Additional BC checks added in this PR:
```bash
npx nx typecheck php-wasm-cli
npx nx typecheck playground-cli
cd packages/php-wasm/node
PHP=8.4 npx vitest run src/test/php-dynamic-loading.spec.ts --config vite.config.ts
```
The focused web Playwright BC test could not run in my local container
because Chromium system libraries are missing; CI covers it.
| 'xdebug.idekey': `"${ideKey}"`, | ||
| // Path mapping is only available starting from Xdebug 3.5, | ||
| // which is used by PHP 8.5+. Previous versions ignore it. | ||
| 'xdebug.path_mapping': 'yes', |
There was a problem hiding this comment.
Is this used by us internally? Because xdebug.path_mapping. It seems that in ^3.5, path mapping is simply enabled.
There was a problem hiding this comment.
@mho22 would know. In any case, this is a direct carry-over from the previous version of this file, not something introduced in this PR.
There was a problem hiding this comment.
In xdebug.c there's this line :
STD_PHP_INI_ENTRY("xdebug.path_mapping", "0", PHP_INI_ALL, OnUpdateBool, settings.library.path_mapping, zend_xdebug_globals, xdebug_globals)
Which I assume sets path_mapping false by default. That is why I enabled it in our ini entries when we enable Xdebug for PHP 8.5.
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Unifies PHP.wasm extension loading behind a single startup-time pipeline by introducing an extensions array API (bundled + external sources) and wiring Node/Web/Playground/CLI call sites to use it while keeping deprecated with* options working.
Changes:
- Added a universal extension resolver/installer (
resolvePHPExtension,withResolvedPHPExtensions) that stages.so+ per-extension.ini+ sidecar files and updatesPHP_INI_SCAN_DIR. - Updated Node/Web runtime loaders and downstream consumers (Playground remote/client/CLI + tests) to use
extensionswith backward-compatible aliases. - Added Node+Web loaders’ async-mode detection (JSPI vs asyncify) and propagated it into runtime options for correct artifact selection.
Reviewed changes
Copilot reviewed 47 out of 49 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| packages/playground/remote/src/lib/playground-worker-endpoint.ts | Replaces withIntl boot option with extensions passthrough to Web loader. |
| packages/playground/remote/src/lib/playground-worker-endpoint-blueprints-v2.ts | Defaults blueprint v2 worker boot to extensions = []. |
| packages/playground/remote/src/lib/playground-worker-endpoint-blueprints-v1.ts | Defaults blueprint v1 worker boot to extensions = []. |
| packages/playground/remote/src/lib/playground-worker-endpoint-blueprints-v1.spec.ts | Adjusts test boot args after removing withIntl. |
| packages/playground/client/src/blueprints-v1-handler.ts | Converts intl flag into extensions: ['intl'] for remote worker. |
| packages/playground/cli/src/php-extensions.ts | New helper mapping CLI flags to Node runtime extensions requests. |
| packages/playground/cli/src/blueprints-v2/worker-thread-v2.ts | Replaces with* fields with extensions in worker args and runtime boot. |
| packages/playground/cli/src/blueprints-v2/blueprints-v2-handler.ts | Uses getRequestedPHPExtensions() to build extensions array from CLI args. |
| packages/playground/cli/src/blueprints-v1/worker-thread-v1.ts | Replaces with* fields with extensions in v1 worker boot options. |
| packages/playground/cli/src/blueprints-v1/blueprints-v1-handler.ts | Uses getRequestedPHPExtensions() for v1 CLI worker boot. |
| packages/php-wasm/xdebug-bridge/src/tests/xdebug-cdp-bridge.spec.ts | Migrates to extensions: ['xdebug'] in test runtime creation. |
| packages/php-wasm/web/vite.config.ts | Updates comment/reference after moving Intl loading logic to unified loader. |
| packages/php-wasm/web/src/test/playwright/globals.ts | Exposes PHPRequestHandler on window for new lifecycle tests. |
| packages/php-wasm/web/src/test/php-dynamic-loading.spec.ts | Migrates tests to extensions and adds lifecycle coverage (rotation/instance pools). |
| packages/php-wasm/web/src/lib/load-runtime.ts | Adds extensions + async-mode detection, maintains deprecated withIntl. |
| packages/php-wasm/web/src/lib/index.ts | Exports new Web extension types from loader-level API. |
| packages/php-wasm/web/src/lib/extensions/load-extensions.ts | New Web-side unified extension resolution (built-in intl + external sources). |
| packages/php-wasm/web/src/lib/extensions/intl/with-intl.ts | Removes old Intl-specific loader in favor of unified pipeline. |
| packages/php-wasm/web/package.json | Adds wasm-feature-detect dependency for JSPI detection. |
| packages/php-wasm/universal/src/lib/load-php-runtime.ts | Propagates phpWasmAsyncMode onto the instantiated runtime. |
| packages/php-wasm/universal/src/lib/load-extension.ts | New universal extension resolver/installer and manifest/url/sha handling. |
| packages/php-wasm/universal/src/lib/load-extension.spec.ts | Adds unit tests for universal extension resolution and manifest selection. |
| packages/php-wasm/universal/src/lib/index.ts | Exports new universal extension APIs/types. |
| packages/php-wasm/node/src/test/with-php-extensions.spec.ts | Adds Node test ensuring local manifest paths work without custom fetch. |
| packages/php-wasm/node/src/test/proxyfs-mmap.spec.ts | Migrates Intl enabling to extensions: ['intl']. |
| packages/php-wasm/node/src/test/php-worker.spec.ts | Adds lifecycle test ensuring extensions are loaded across worker instances. |
| packages/php-wasm/node/src/test/php-redis.spec.ts | Migrates Redis enabling to extensions: ['redis']. |
| packages/php-wasm/node/src/test/php-part-2.spec.ts | Migrates Xdebug option to extensions and adjusts skip logic. |
| packages/php-wasm/node/src/test/php-part-1.spec.ts | Migrates Xdebug option to extensions and adjusts skip logic. |
| packages/php-wasm/node/src/test/php-networking.spec.ts | Migrates Xdebug option to extensions. |
| packages/php-wasm/node/src/test/php-memcached.spec.ts | Migrates Memcached enabling to extensions: ['memcached']. |
| packages/php-wasm/node/src/test/php-image-extensions.spec.ts | Migrates Xdebug option to extensions. |
| packages/php-wasm/node/src/test/php-fsockopen.spec.ts | Migrates Xdebug option to extensions. |
| packages/php-wasm/node/src/test/php-fopen.spec.ts | Migrates Xdebug option to extensions. |
| packages/php-wasm/node/src/test/php-file-get-contents.spec.ts | Migrates Xdebug checks to extensions?.includes('xdebug'). |
| packages/php-wasm/node/src/test/php-dynamic-loading.spec.ts | Migrates to extensions and adds rotation/BC tests for deprecated with*. |
| packages/php-wasm/node/src/test/php-crash.spec.ts | Migrates to extensions: ['xdebug']. |
| packages/php-wasm/node/src/lib/load-runtime.ts | Adds extensions API + async-mode detection, keeps deprecated with* aliases. |
| packages/php-wasm/node/src/lib/index.ts | Exports new Node extension types from loader-level API. |
| packages/php-wasm/node/src/lib/extensions/xdebug/with-xdebug.ts | Deprecates withXdebug helper and switches to unified installer for file staging. |
| packages/php-wasm/node/src/lib/extensions/redis/with-redis.ts | Removes old Redis-specific loader in favor of unified pipeline. |
| packages/php-wasm/node/src/lib/extensions/node-extension-resources.ts | Adds Node normalization + file:-aware fetch for manifests/artifacts. |
| packages/php-wasm/node/src/lib/extensions/memcached/with-memcached.ts | Removes old Memcached-specific loader in favor of unified pipeline. |
| packages/php-wasm/node/src/lib/extensions/load-extensions.ts | New Node-side unified extension resolution (bundled + external sources). |
| packages/php-wasm/node/src/lib/extensions/intl/with-intl.ts | Removes old Intl-specific loader in favor of unified pipeline. |
| packages/php-wasm/node/package.json | Adds wasm-feature-detect dependency for JSPI detection. |
| packages/php-wasm/cli/src/main.ts | Switches CLI runtime boot from withXdebug to extensions array. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
346cd65 to
e052b33
Compare
9b2e10b to
e170be1
Compare
e170be1 to
fc69850
Compare
dbbcdc2 to
d168904
Compare
|
Let's do it! |
## What it does Limits external PHP extension loading to JSPI runtimes in the extension API added by #3566: ```json { "phpVersion": "8.4", "file": "extension-php8.4-jspi.so" } ``` Bundled extensions shipped with the PHP.wasm packages still support Asyncify. `intl`, `xdebug`, `redis`, and `memcached` continue to use the artifact selected by their versioned packages. ## Rationale External Asyncify side modules would require Playground to support and document an Asyncify artifact matrix for third-party extensions. That is a larger compatibility surface than #3566 needs. Bundled extensions are different because their Asyncify artifacts are built, shipped, and tested with this repo. ## Implementation `resolvePHPExtension()` no longer accepts `asyncMode`. External manifests select artifacts by `phpVersion` only and reject manifest entries that still include `asyncMode`. Node and Web extension wrappers reject external extension sources when the runtime is using Asyncify. Bundled extension paths still go through the same staging resolver after their versioned packages choose the matching JSPI or Asyncify `.so`. ## Testing instructions ```bash npm exec nx -- test php-wasm-universal --testFile=load-extension.spec.ts npm exec nx -- run php-wasm-node:test-group-4-asyncify --testFiles=with-php-extensions.spec.ts npm exec nx -- typecheck php-wasm-universal npm exec nx -- typecheck php-wasm-node npm exec nx -- typecheck php-wasm-web npm exec nx -- lint php-wasm-universal npm exec nx -- lint php-wasm-node npm exec nx -- lint php-wasm-web ```
## What it does Adds manifest-declared sidecar files to the PHP extension loading API introduced in #3566, and folds in #3590's generated AJV validator for extension manifests. A manifest can now describe files and directories that must be staged before PHP starts, alongside the selected `.so` artifact. Each node's final VFS path is `vfsRoot + vfsPath`: ```json { "name": "spx", "artifacts": [ { "phpVersion": "8.4", "sourcePath": "spx.so", "extraFiles": { "vfsRoot": "/internal/shared/spx", "nodes": [ { "vfsPath": "data", "type": "directory" }, { "vfsPath": "ui/index.html", "sourcePath": "ui/index.html" } ] } } ] } ``` ## Rationale #3255 needs SPX to load more than a WASM side module. The extension also expects a writable data directory and web UI assets such as HTML, CSS, and JS files. This PR keeps that work focused on the loader contract. It does **not** rebuild PHP 8.2, ship SPX artifacts, or claim that SPX loads today. A follow-up can use this manifest shape when adding or rebuilding SPX artifacts. ## Integration For most consumers nothing changes structurally — manifest sources slot into the same `extensions` array as bundled extension names: ```ts await loadNodeRuntime('8.4', { extensions: [ 'intl', 'xdebug', { source: { format: 'manifest', manifestUrl: 'https://example.com/spx/manifest.json' } }, ], }); ``` For hand-rolled Emscripten setups, `resolvePHPExtension` + `withResolvedPHPExtensions` in `@php-wasm/universal` are the lower-level pair. ## Implementation `PHPExtensionManifest` accepts `extraFiles` at two levels: - manifest-level `extraFiles` for files shared by every artifact - artifact-level `extraFiles` for PHP-version-specific files Each group has a `vfsRoot` (the absolute VFS prefix) and an array of `nodes` with a `vfsPath`, an optional `type: 'file' | 'directory'`, and a `sourcePath` (the URL to fetch, only required for files). The resolver joins `vfsRoot + vfsPath` into absolute VFS paths and returns: ```ts { files: { '/internal/shared/spx/ui/index.html': bytes }, directories: ['/internal/shared/spx/data'], } ``` The resolver validates untrusted manifest JSON with a standalone AJV validator generated from the public `PHPExtensionManifest` TypeScript type. Sidecar files are fetched relative to the manifest URL or inline manifest `baseUrl`, with manifest-level and selected artifact-level nodes fetched in parallel through a shared concurrency limit. Caller-supplied `extraFiles` on `PHPExtensionInstallOptions` use the same absolute-path shape (`{ files, directories? }`) — no `targetPath` indirection. External extension manifests remain JSPI-only. ## Breaking changes - Manifest field renames: `artifact.file` → `artifact.sourcePath`; `extraFiles.targetPath` → `vfsRoot`; `extraFiles.files`/`extraFiles.directories` → unified `nodes` array with `vfsPath`/`type`/`sourcePath`. - `PHPExtensionExtraFiles` (caller-supplied sidecar files) is now flat absolute VFS paths under `files` and `directories`. The previous `targetPath + relative paths` shape is gone. - `sha256` fields removed from sources and artifacts. - `ResolvePHPExtensionOptions` is no longer exported (use `PHPExtensionInstallOptions & { phpVersion: string }`). ## Testing instructions \`\`\`bash npm exec nx -- test php-wasm-universal --testFile=load-extension.spec.ts npm exec nx -- typecheck php-wasm-universal npm exec nx -- lint php-wasm-universal npm exec nx -- build php-wasm-universal cd packages/php-wasm/node && npx vitest run src/test/with-php-extensions.spec.ts --config vite.jspi.config.ts npm exec nx -- typecheck php-wasm-node npm exec nx -- lint php-wasm-node npm exec nx -- typecheck php-wasm-web \`\`\`
After PR #3566 moved the icu.dat dynamic import from src/lib/extensions/ intl/with-intl.ts (depth 3) to src/lib/extensions/load-extensions.ts (depth 2), the viteExternalDynamicImports transform's hard-coded '../../../' prefix produced 'import("../intl/shared/icu.dat")' in the dist bundle — a path that resolves outside the package to a non-existent @php-wasm/intl/shared/icu.dat. Downstream Vite consumers failed with "Failed to resolve import" against that path. Derive the output as './shared/<file>' from the specifier's tail so the transform stays correct regardless of where the import statement lives under src/. The companion regression test in test-built-npm-packages/{es-modules-and-vitest,commonjs-and-jest}/tests/ assets.spec.ts now passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
What it does
Adds one startup-time
extensionsarray for PHP.wasm runtimes. The same option loads bundled extensions and externally hosted.soartifacts:External extensions use the same path:
@php-wasm/nodesupports bundledintl,xdebug,redis, andmemcached.@php-wasm/websupports bundledintl. Both runtimes also accept external extension sources: inline bytes, direct artifact URLs, or manifests.This PR does not add the extension compiler package. It defines the loading API and consumes already-built
.soartifacts. Compiler tooling can stack on top of this.Rationale
Playground could already load bundled extensions, but every extension had its own staging path: copy the
.so, write an.ini, stage sidecar files, and set startup env vars. That made core extensions and extender-provided extensions two different systems.This PR makes extension loading a runtime startup concern with one API. An extender can publish a manifest and load their extension the same way Playground core loads
intlorredis.The API is an array because extension loading is a list of requests. An object works for bundled toggles, but becomes ambiguous for external extensions whose name may come from a manifest:
This is intentionally pre-startup only. PHP reads extension declarations from
.inifiles while starting.dl()would only cover some regular extensions after startup, does not cover Zend extensions such as Xdebug, and is easy to get wrong for extensions that need startup env vars or sidecar files.Implementation
Adds
packages/php-wasm/universal/src/lib/load-extension.tsas the shared resolver and installer:resolvePHPExtension()resolves inline bytes, direct URLs, or manifests into aResolvedPHPExtension.withResolvedPHPExtensions()augments Emscripten options, updatesPHP_INI_SCAN_DIR, and installs resolved extensions before PHP starts.installPHPExtensionFilesSync()writes the.so, generated.ini, and sidecar files through EmscriptenFSwhile the runtime is initializing.Manifests select the artifact matching the running PHP version and async mode:
{ "name": "wp_mysql_parser", "version": "0.1.0", "artifacts": [ { "phpVersion": "8.4", "asyncMode": "jspi", "file": "wp_mysql_parser-php8.4-jspi.so", "sha256": "..." } ] }fileis either an absolute artifact URL or a path relative to the manifest URL. If the manifest is loaded fromhttps://cdn.example.com/extensions/wp_mysql_parser/manifest.json, thenwp_mysql_parser-php8.4-jspi.soresolves next to that manifest. Inline manifest objects usebaseUrlfor the same resolution rule. In Node,manifestUrlcan also be a filesystem path orfile:URL, and local artifacts work without a customfetch.PHP.wasm writes a generated startup
.inifile per extension.loadWithIniDirectivecontrols the first line of that generated file:extension=/internal/shared/extensions/intl.soor:
iniEntries,extraFiles,env, andextensionDirdescribe files and settings PHP.wasm stages before startup. They are not post-startup edits to the mainphp.ini.The Node and Web loaders resolve all requested extensions in parallel before augmenting Emscripten options, so multiple manifests or artifact downloads do not serialize.
Backward compatibility:
loadNodeRuntime()still acceptswithIntl,withXdebug,withRedis, andwithMemcachedwith their previous option shapes.loadWebRuntime()still acceptswithIntl.extensionsarray internally.The compatibility guarantee is for the loader options above. The old helper modules such as
extensions/intl/with-intl,extensions/redis/with-redis,extensions/memcached/with-memcached, andextensions/xdebug/with-xdebugare removed. They were implementation details rather than package-root exports. If real downstream code depends on them, we can add a targeted bridge.Pitfalls resolved in this PR:
intlneeds ICU data andICU_DATAbefore PHP starts, so it now uses the same startup file staging as every other extension..inimust usezend_extension=, notextension=.xdebug.path_mapping=yesand stages the mapping files for PHP 8.5+.file:URLs without a customfetch.sha256when provided.Promise.all()before installing them.wasm-feature-detect. The Vite config now forceswasm-feature-detectinto its own chunk, and the worker entrypoints throw a clear error if a Comlink endpoint is exposed twice in the same worker global.browser-globals.tsfile instead of a runtimeglobals.tsplus a separateglobals.d.ts.Docs and READMEs now cover bundled extensions, external manifests, direct bytes/URLs, local Node paths, generated startup
.inifiles, sidecar files, dependency files, and the startup-only limit.Testing instructions
CI coverage added or updated in this PR verifies deprecated
with*loader options, runtime rotation, Node worker instance pools, WebPHPRequestHandlerinstance pools, local Node manifests without customfetch, manifest artifact selection,sha256verification, Redis and Memcached extension loading, and the WebKit worker bundling fix.