Add curl installer scripts for standalone CLI distribution - #3064
Conversation
📊 Performance Test ResultsComparing 15b1272 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
|
I already mentioned this on Slack, but for the record: I think we should look in to Node's native "single executable application" feature https://nodejs.org/docs/latest-v25.x/api/single-executable-applications.html. This would simplify the installer, and ideally, we could reuse this logic for the bundled CLI. |
…on, SHA-256 output
fredrikekelund
left a comment
There was a problem hiding this comment.
I like the overall approach here and I definitely think we're doing the right thing in aligning the CLI build between the app and the standalone CLI bundle. However, it seems like the "everything in main.mjs" approach I suggested has the caveat of increased baseline memory usage. A 20MB JS script is definitely pushing it in terms of filesize…
On trunk, the process manager daemon occupies ~30MB on my machine. On this branch, it's closer to 100MB. Not necessarily a deal breaker, although TBH, this is probably more of a drawback than reducing the number of separate assets is a benefit. How do you feel about reverting to your previous approach, @bcotrim?
We still need to implement some kind of self-updater in the CLI, and we need to figure out distribution (as you rightly point out, @bcotrim).
| // Single-file bundle so the SEA can embed one self-contained main.mjs | ||
| // without a chunk-name scavenger hunt at runtime. Requires a single | ||
| // entry — ok for prod since there's only `main`. | ||
| inlineDynamicImports: true, |
There was a problem hiding this comment.
I'd consider putting this in vite.config.base.ts to ensure we align behavior as closely as possible between all builds. I don't think we need to make an exception for npm, either…
There was a problem hiding this comment.
Except I ultimately landed on that we might want to revert this approach and go with what you originally had, @bcotrim 😅
| rollupOptions: { | ||
| external: ( id ) => { | ||
| if ( id.includes( 'blueprint-schema-validator' ) ) { | ||
| return false; | ||
| } | ||
| if ( isNodeBuiltin( id ) ) { | ||
| return true; | ||
| } | ||
| // Externalize every dependency — the end user installs them. | ||
| return packageJsonDependencies.some( | ||
| ( dep ) => id === dep || id.startsWith( dep + '/' ) | ||
| ); | ||
| }, | ||
| }, |
There was a problem hiding this comment.
Nice 👍 We should have done this before anyway.
There was a problem hiding this comment.
Do we even need this and studio-cli.bat now? Maybe that's fodder for a different PR, but it seems like an unnecessary intermediate layer now.
| // Stream the asset to tar's stdin and extract in-place (cwd = destDir). | ||
| // No paths in argv => no GNU-tar "Cannot connect to C:" failure. | ||
| try { | ||
| execSync( 'tar -xz', { |
There was a problem hiding this comment.
I see now that we get the assets with the sea.getAsset() API, so I guess the answer here is no.
| { | ||
| "main": "entry.cjs", | ||
| "output": "bundle.blob", | ||
| "disableExperimentalSEAWarning": true, |
There was a problem hiding this comment.
| "disableExperimentalSEAWarning": true, | |
| "disableExperimentalSEAWarning": true, | |
| "execArgv": ["--experimental-wasm-jspi"], |
I haven't verified, but per the docs, this should let us work around the JSPI problem.
|
A 400 MB reduction in installer size is incredibly impressive. That's a lot more than I would have expected. We're not actually shipping less code here, either… Is all of that attributable to the fact that we compress the CLI dependencies? |
…staller # Conflicts: # .buildkite/commands/run-e2e-tests.sh # apps/cli/lib/pull/migration-client.ts # apps/cli/vite.config.base.ts
|
@fredrikekelund I reverted back to the initial approach. Can you give the PR another review, please? |
fredrikekelund
left a comment
There was a problem hiding this comment.
I think this is a pragmatic strategy 👍 Without a fully standalone executable, I agree there's really no good reason to go through the SEA packaging/unpackaging hassle.
I recommend addressing the studio site stop --all thing before landing this PR. Other than that, this tests well on macOS. I'm about to test it on Windows, too. If it works as expected there, I'll approve and let you handle the studio site stop --all thing without waiting for another review from me, @bcotrim
| install_studio() { | ||
| BUNDLE_NAME="studio-cli-${PLATFORM}-${ARCH}.tar.gz" | ||
| BUNDLE_URL="${BASE_URL}/${BUNDLE_NAME}" | ||
|
|
||
| mkdir -p "$INSTALL_DIR" | ||
|
|
||
| # Download, verify, and extract in a staging dir on the same filesystem as | ||
| # the final paths. A failed or corrupt download never touches a previously | ||
| # working install. | ||
| STAGING_DIR="$(mktemp -d "${INSTALL_DIR}/.studio-install.XXXXXX")" | ||
| trap 'rm -rf "$STAGING_DIR"' EXIT | ||
|
|
||
| TMP_BUNDLE="$STAGING_DIR/$BUNDLE_NAME" | ||
|
|
||
| echo "Downloading Studio CLI..." | ||
| download "$BUNDLE_URL" "$TMP_BUNDLE" | ||
| download "${BUNDLE_URL}.sha256" "$TMP_BUNDLE.sha256" | ||
|
|
||
| echo "Verifying checksum..." | ||
| if ! verify_checksum "$TMP_BUNDLE" "$TMP_BUNDLE.sha256"; then | ||
| echo "Error: checksum verification failed. Aborting; existing install left untouched." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "Installing to $INSTALL_DIR..." | ||
| mkdir -p "$STAGING_DIR/extracted" | ||
| tar -xzf "$TMP_BUNDLE" -C "$STAGING_DIR/extracted" | ||
|
|
||
| # Replace only the runtime dirs. Config files in $INSTALL_DIR (shared.json, | ||
| # cli.json, app.json, …) are left untouched. | ||
| rm -rf "$INSTALL_DIR/cli" "$INSTALL_DIR/bin" | ||
| mv "$STAGING_DIR/extracted/cli" "$INSTALL_DIR/cli" | ||
| mv "$STAGING_DIR/extracted/bin" "$INSTALL_DIR/bin" | ||
|
|
||
| chmod +x "$INSTALL_DIR/bin/node" "$INSTALL_DIR/bin/studio" | ||
|
|
||
| # Symlink to PATH | ||
| mkdir -p "$BIN_DIR" | ||
| ln -sf "$INSTALL_DIR/bin/studio" "$BIN_DIR/studio" | ||
| } |
There was a problem hiding this comment.
This function doesn't account for cases where a standalone Studio CLI is already installed on the system. I suggest something like this: if ~/.studio/bin or ~/.studio/cli already exist, try running studio site stop --all before mv "$STAGING_DIR/extracted/cli" "$INSTALL_DIR/cli"
|
|
||
| # --- Install --- | ||
|
|
||
| function Install-StudioCli { |
There was a problem hiding this comment.
Same issue here as in install.sh: we aren't accounting for existing standalone Studio CLI installs.
On Posix, removing the directory will mess up running processes, whereas on Windows, the installer will fail (since the kernel will disallow removing ~/.studio/cli or ~/.studio/bin if any process has a handle on the files in that directory).
| * NOTE: The `cli:package` step mutates `apps/cli/node_modules`. If you need a | ||
| * clean tree afterwards, run `npm ci` from the repo root to reset it. |
There was a problem hiding this comment.
Not a show stopper, but we get around this problem when building the Electron app locally by copying the entire repo to a temporary build dir, running all the build commands there (including npm run cli:package) and then copying back the output.
Might be a nice touch for this script, too
| run( | ||
| `npx tsx scripts/download-node-binary.ts ${ platformArg } ${ archArg } "${ path.join( | ||
| stagingDir, | ||
| 'bin' | ||
| ) }"` | ||
| ); |
There was a problem hiding this comment.
Fully optional, but we could statically import main() from scripts/download-node-binary.ts instead of running this via npx tsx
There was a problem hiding this comment.
We can tackle this in a follow-up PR, but these changes will make it so the Studio CLI for terminal toggle in the app's settings modal is always enabled if the standalone CLI installed. If the user tries to uninstall it, it would silently fail.
I'd argue the ideal behavior would be to disable the UI toggle when the CLI is externally managed.
fredrikekelund
left a comment
There was a problem hiding this comment.
We need -ExecutionPolicy Bypass when running the PowerShell script. Other than that, this tests well on Windows 👍
Please note that even though I'm approving now, I still recommend we address the studio site stop --all thing I raised before landing this PR.
There was a problem hiding this comment.
I had to run this script like so to make it work:
powershell -ExecutionPolicy Bypass -File scripts\standalone\install.ps1
…atest alias (#3835) ## Related issues - Related to STU-1780 - Builds on #3064 (standalone bundling + curl installers) and Automattic/wpcom#221531 (adds the `WordPress.com Studio CLI` Apps CDN product) ## How AI was used in this PR AI helped map the existing Apps CDN / Fastlane / Buildkite release flow, design the CLI upload to mirror the established PHP CLI binary path, switch the installers to the CDN `latest` alias, and verify the work locally (Ruby + YAML syntax, the fastlane helper tests, a `studio_cli_cdn_builds`/upload dry-run harness, an end-to-end `install.sh` behavior harness, and a real throwaway upload against the production CDN). All changes were reviewed and validated before pushing. ## Proposed Changes Makes the standalone Studio CLI available to users without the desktop app, kept current automatically. - **Every Studio build now publishes the standalone CLI bundles to the Apps CDN** (under the `WordPress.com Studio CLI` product), for nightly/dev, beta, and stable alike. The bundles track the app build exactly — same version and visibility — so stable bundles become public at publish time just like the app, and there's no separate step to run. - **The curl installers download straight from the CDN's versionless `latest` alias**, so no bespoke redirect service is needed. `STUDIO_CLI_VERSION` pins a specific version, and `STUDIO_CLI_URL` is retained as a bypass for local builds and mirrors. Both paths rely on HTTPS plus a staged-extraction guard (a corrupt download fails at `tar -xzf` before touching an existing install); the bundle's SHA-256 is still recorded as CDN build metadata. - Bundles are published as `studio-cli-<platform>-<arch>.tgz` (gzipped tar) for all six platform/arch targets. Trade-off / remaining dependency: the `install.sh` / `install.ps1` scripts themselves still need public hosting for the `curl … | bash` / `irm … | iex` entry points (the CDN serves uploaded builds, not arbitrary scripts) — a separate WordPress.com-side task. Until that lands, the CLI is installable via the direct CDN URL or `STUDIO_CLI_URL`. ## Testing Instructions **Fastlane (dry run, no remote side effects):** ```sh npm run cli:bundle -- darwin arm64 # build a bundle (repeat per platform/arch) DRY_RUN=true bundle exec fastlane publish_studio_cli_binaries version:"v1.11.0" ``` Confirm it lists the six platform/arch uploads with the right product, platform labels, version, and sha. Existing helper tests still pass: `bash .buildkite/commands/run-fastlane-tests.sh`. **Real CDN upload (verified):** a throwaway `0.0.1` / Internal build was uploaded to the production `WordPress.com Studio CLI` product via the same `media/new` endpoint the lane posts to — confirming the upload path works and that the `.tgz` file type is accepted (see Notes). **Installer — `STUDIO_CLI_URL` bypass:** ```sh STUDIO_CLI_HOME="$(mktemp -d)" STUDIO_CLI_URL="$PWD/standalone-bundles" bash scripts/standalone/install.sh ``` **Installer — default CDN mode:** with bundles published, a fresh install resolves `https://appscdn.wordpress.com/downloads/wordpress-com-studio-cli/<slug>/latest/full-install` (e.g. `mac-silicon`), and `STUDIO_CLI_VERSION=v1.11.0` pins that version. **Windows:** `install.ps1` mirrors the same logic (win32 → `windows-x64` / `windows-arm64`). It was validated structurally but **needs a real Windows run to confirm** (no `pwsh` locally). ## Pre-merge Checklist - [ ] Have you checked for TypeScript, React or other console errors? — N/A (changes are Fastlane/Ruby, Buildkite YAML, PowerShell/shell installers, and docs; no app code). ### Notes for reviewers - **Bundles are `.tgz`, not `.tar.gz`.** The Apps CDN's upload validation (WordPress `wp_check_filetype_and_ext`) rejects the `.tar.gz` *double extension* — even though `gz` is allow-listed — while `.tgz` (same gzip bytes, single extension, already on the CDN's accepted-mime list) uploads cleanly. `tar -xzf` handles both identically, so the installers and extraction logic are unchanged. - **macOS quarantine:** `install.sh` clears `com.apple.quarantine` from the extracted files. The bundled `.node` modules are unsigned, so a *browser-downloaded* bundle would otherwise be Gatekeeper-blocked ("library load disallowed by system policy"). The `curl … | bash` path isn't quarantined, so this is defensive for manual / MDM-managed installs. (Signing + notarizing the macOS bundle is a possible future hardening, tracked separately.) - **Cross-arch bundles are safe within an OS** (so the existing CI matrix runners are fine): the CLI uses no-fsevents chokidar, `fs-ext-extra-prebuilt` ships both arches per-OS, and the target Node binary is downloaded per arch. Only cross-*OS* needs a native runner, which the matrix already provides. - Stable bundles flip to public automatically because their CDN post IDs ride the draft GitHub release through the existing `create_draft_github_release` → `make_cdn_builds_public` path — no change to that logic.
Related issues
How AI was used in this PR
AI helped draft and iterate on the standalone installer/bundling changes, reason through review feedback, and inspect CI logs. The generated changes were reviewed, adjusted, and verified locally before pushing.
Proposed Changes
This PR makes the Studio CLI available as a standalone terminal install, so users can run
studiowithout first installing or launching the desktop app. This supports headless workflows, CI use, remote development, scripting, and users who only need CLI access.The installers (
install.sh/install.ps1) download a platform tarball, verify its SHA-256 checksum before touching an existing install, extract it, and putstudioon the user's PATH. The bundle reuses the same runtime layout the desktop app has shipped for a long time — a real Node.js binary plus the packaged CLI — fronted by a smallstudiolauncher, nvm/rustup-style. This keeps the CLI's runtime behavior (child processes, JSPI, browser installs) identical to the desktop app and dev environment, and leaves both the desktop app's packaging and the CLI's build/runtime code unchanged — the diff is almost entirely the new bundling and installer layer, plus detection so desktop-managed CLI installs don't overwrite standalone ones.This PR does not publish the public install endpoints yet. Follow-up work still needs to host
install.sh/install.ps1and publish release bundles/checksums tohttps://wp.build/releases/.Trade-offs:
studioon PATH is a launcher script rather than a single binary (same pattern as nvm/rustup installs).Testing Instructions
Local standalone bundle smoke test
standalone-bundles/containsstudio-cli-<platform>-<arch>.tar.gzand its.sha256checksum.macOS / Linux installer
STUDIO_CLI_URL="$PWD/standalone-bundles" bash scripts/standalone/install.shls -la ~/.local/bin/studiostudio site create --path ~/studio-test-site --skip-browser studio site stop --allWindows installer
Desktop package integration
Desktop packaging is unchanged from trunk (
resources/bin/node+resources/cli/). To sanity check:Notes
curl -fsSL https://wp.build/install.sh | bashon macOS/Linux andirm https://wp.build/install.ps1 | iexon Windows after the release publishing follow-up lands.Pre-merge Checklist