pull: fix the flattened runtime root and delta re-pull after deferred files

[epeicher]

What it does

Two fixes to the composite pull command so it correctly runs, and re-runs, the full preflight -> files-pull -> db-pull -> db-apply -> flat-docroot -> apply-runtime pipeline from a single invocation:

1. pull --flatten-to=DIR --runtime=... now generates a runtime rooted at the flattened layout (DIR) instead of the raw download tree. The two options now compose: --flatten-to builds the flattened docroot and --runtime targets it.
2. A completed pull can be re-pulled after its deferred-files tail ran. Once skipped-earlier files have been fetched, the files sub-command records a truthful status=complete, so the next pull --filter=essential-files proceeds as a delta re-pull without weakening the mid-flight --filter guard.

Rationale

--flatten-to did not reach apply-runtime. pull forwards --flatten-to to the flat-docroot stage as flatten_to, but run_apply_runtime() reads a different key, flat_document_root, and otherwise falls back to fs_root + remote document_root (the raw tree). So a pull that both flattens and generates a runtime produced a runtime pointing at the wrong root: the flatten and runtime stages did not compose.

Delta re-pull was blocked after a deferred-files tail. pull supports delta re-pull: re-running a completed pull resets its own state via prepare_repull() and re-syncs. But after an --filter=essential-files pull fetched its deferred skipped-earlier files, the state still read filter=skipped-earlier, status=in_progress. The next pull --filter=essential-files loaded that stale state in ImportClient::run() and the mid-flight guard threw:

Cannot change --filter from 'skipped-earlier' to 'essential-files' while a sync is in progress.

That guard is correct for real in-progress syncs. The bug was the stale lifecycle state: a successful skipped-earlier tail had finished, but did not restore the files sub-command to status=complete before returning.

Implementation

validate_and_default_options() derives the missing runtime key:

if (!empty($options['flatten_to']) && empty($options['flat_document_root'])) {
    $options['flat_document_root'] = $options['flatten_to'];
}

run_files_sync() now restores status=complete after a successful deferred fetch-skipped tail. If the pipeline saves status=partial, it still returns early so the next invocation can resume.

$this->run_files_sync_pipeline();
if (($this->state["status"] ?? null) === "partial") {
    return;
}
$this->state["status"] = "complete";
$this->save_state($this->state);

The filter guard remains generic: it blocks real mid-flight filter changes, and completed skipped-file tails no longer look mid-flight.

Testing instructions

cd tests && ../vendor/bin/phpunit Import/PullFilterOptionTest.php Import/FilesSyncStateTest.php

Tests cover:

• testPullDerivesFlatDocumentRootFromFlattenTo: a pull with --flatten-to plus --runtime hands apply-runtime a matching flat_document_root.
• testSkippedEarlierTailRestoresCompletedStatus: a successful files-pull --filter=skipped-earlier tail restores status=complete and clears the skipped list.
• testRepullAfterSkippedEarlierTailUsesCompletedFilesPullState: a completed pull can delta re-pull after the skipped-files tail because the files sub-command state is complete.

Testing using Studio

• Use a remote site that has uploads.
• Using the trunk branch of reprint, build reprint and move it to Studio repo:

composer build:phar && cp reprint.phar <studio-repo>/apps/cli/dist/cli/wp-files/reprint/reprint.phar

• Open a terminal on the root of the Studio repo and run:

node apps/cli/dist/cli/main.mjs pull-reprint --url <remote-site-url> --name g1 --yes

• Run the command again so a delta-pull is done:

node apps/cli/dist/cli/main.mjs pull-reprint --url <remote-site-url> --name g1 --yes

• Check there is an error Failed to pull site: {"error":"Cannot change --filter from 'skipped-earlier' to 'essential-files' while a sync is in progress.
• Repeat the steps using this branch and verify the site can be re-pulled without errors.

[github-actions]

Pull pipeline performance — large-directory

Site: large-directory · 2,000+ plus targeted file-transfer scenarios files · 10,000 posts · 25,000 postmeta · PHP 8.5.7

Stage	PR	trunk	Δ	Status	Details
playground-sqlite-db-pull	8.74 s	8.66 s	⚪ +83 ms (+1.0%)	✓	condition=db-pull in PHP.wasm runtime=php.wasm 8.3 wp_mysql_parser=enabled mode=lexer native_lexer=verified native_token_stream=WP_MySQL_Native_Token_Stream native_token_count=18 native_parser=selected trunk: condition=db-pull in PHP.wasm runtime=php.wasm 8.3 wp_mysql_parser=enabled mode=lexer native_lexer=verified native_token_stream=WP_MySQL_Native_Token_Stream native_token_count=18 native_parser=selected
playground-sqlite-db-apply	3.48 s	3.47 s	⚪ +9 ms (+0.3%)	✓	condition=db-apply to SQLite in PHP.wasm runtime=php.wasm 8.3 wp_mysql_parser=enabled mode=parser native_lexer=verified native_token_stream=WP_MySQL_Native_Token_Stream native_token_count=18 native_parser=verified native_ast=WP_MySQL_Native_Parser_Node sqlite_driver_parser=verified trunk: condition=db-apply to SQLite in PHP.wasm runtime=php.wasm 8.3 wp_mysql_parser=enabled mode=parser native_lexer=verified native_token_stream=WP_MySQL_Native_Token_Stream native_token_count=18 native_parser=verified native_ast=WP_MySQL_Native_Parser_Node sqlite_driver_parser=verified
Total	12.22 s	12.13 s	⚪ +92 ms (+0.8%)

_{Numbers carry runner noise; treat single-run deltas as directional, not authoritative.}

_{📈 Trunk performance history — commit-by-commit timeline.}

[Copilot]

Pull request overview

This PR fixes two bugs in the composite pull command so that the full pipeline (preflight → files-pull → db-pull → db-apply → flat-docroot → apply-runtime) works correctly both on first run with flatten + runtime options and on delta re-pulls after deferred files have been fetched.

Changes:

• Derives flat_document_root from flatten_to in validate_and_default_options() so that pull --flatten-to=DIR --runtime=... generates a runtime rooted at the flattened layout rather than the raw download tree.
• Exempts completed pull re-pulls (pull.stage === "complete") from the mid-flight filter guard in ImportClient::run(), allowing prepare_repull() to clear the stale sub-command state before the guard would otherwise block the filter change.
• Adds two focused unit tests covering both fixes.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
packages/reprint-importer/src/lib/pull/class-pull.php	Bridges flatten_to → flat_document_root in option validation so apply-runtime targets the flattened directory
packages/reprint-importer/src/import.php	Adds $is_repull check to the filter guard to exempt completed pulls about to delta re-pull
tests/Import/PullFilterOptionTest.php	Adds PullBridgeFakeClient test helper and two new tests for the flatten bridge and the re-pull guard bypass

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.