Add selective sync to pull-reprint via reprint pull-files/pull-db - #3976
Conversation
49be035 to
b28b127
Compare
📊 Performance Test ResultsComparing d58bc78 vs trunk app-size
site-editor
site-startup
Results are median values from multiple test runs. Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff) |
b28b127 to
24f60e0
Compare
epeicher
changed the title
…ction-to-pull' into stu-1816-selective-sync-execution
…ction-to-pull' into stu-1816-selective-sync-execution
…tive-sync-and-interactive-file-selection-to-pull # Conflicts: # apps/cli/commands/pull-reprint.ts
…ction-to-pull' into stu-1816-selective-sync-execution # Conflicts: # apps/cli/commands/pull-reprint.ts # apps/cli/commands/tests/pull-reprint.test.ts
…ction-to-pull' into stu-1816-selective-sync-execution
This reverts commit 33ff691.
…ction-to-pull' into stu-1816-selective-sync-execution
…tive-sync-and-interactive-file-selection-to-pull # Conflicts: # apps/cli/commands/pull-reprint.ts # apps/cli/commands/tests/pull-reprint.test.ts # apps/cli/lib/pull/reprint-state.test.ts # apps/cli/lib/pull/reprint-state.ts
![github-advanced-security[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…ve-file-selection-to-pull
There was a problem hiding this comment.
Pull request overview
Adds selective sync capabilities to the experimental studio pull-reprint command by introducing a selection model (interactive tree or CLI flags) and splitting the Reprint execution into discrete steps that can be skipped/scoped while keeping the resulting site bootable.
Changes:
- Introduces a Reprint-index-backed wp-content selector (plus database toggle) and CLI flags (
--only,--skip-database,--skip-uploads) with a persistedselection.jsonfor resumability. - Replaces the single composite
reprint pullwith a staged pipeline:pull-files→pull-db(optional) →flat-docroot→apply-runtime, including extra fixes for scoped pulls (wp-config synthesis, symlink restoration, skipped-files tail). - Adds first-pull preservation logic to carry unselected local
wp-content(and optionally local DB/uploads) into the scratch before flattening, plus unit tests.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| apps/cli/lib/sync-selector.ts | Exports buildRootTree for reuse by the Reprint selector UI. |
| apps/cli/lib/pull/reprint-state.ts | Extends Reprint state accessors/mutators (core roots, ABSPATH, table prefix, skipped-files flag, sqlite runtime target, scratch reset helpers). |
| apps/cli/lib/pull/reprint-state.test.ts | Adds unit coverage for the new Reprint state accessors/mutators. |
| apps/cli/lib/pull/reprint-selector.ts | New selective-sync selector logic: build tree from Reprint remote index, map checked nodes/CLI flags to Reprint --only, track selected symlinks. |
| apps/cli/lib/pull/reprint-selector.test.ts | Unit tests for selector parsing/mapping and token/path handling. |
| apps/cli/lib/pull/preserve-local-content.ts | New first-pull preservation logic for unselected local wp-content entries. |
| apps/cli/lib/pull/preserve-local-content.test.ts | Unit tests for local-content preservation behavior. |
| apps/cli/commands/tests/pull-reprint.test.ts | Updates and expands command-level helper tests for the split pipeline, scoped pulls, and preservation behavior. |
| apps/cli/commands/pull-reprint.ts | Wires selection into the command, splits Reprint pipeline execution, and threads selection/scoped behavior through download tail and runtime prep. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if ( cliOnly.length > 0 ) { | ||
| const contentDir = getContentDirFromState( session.stateDirectory ) ?? ''; | ||
| const mapped = mapCliOnlyToReprint( cliOnly, contentDir ); | ||
| // Selected entries that are symlinks on the remote need their |
| const contentDir = getContentDirFromState( metadata.stateDirectory ); | ||
| const sqlitePath = contentDir | ||
| ? `${ metadata.rawDirectory }${ contentDir }/database/.ht.sqlite` | ||
| : `${ metadata.sitePath }/wp-content/database/.ht.sqlite`; |
| let raw: string; | ||
| try { | ||
| raw = fs.readFileSync( remoteIndexPath, 'utf-8' ); | ||
| } catch { | ||
| return { children: [], linkTargets: {} }; | ||
| } | ||
|
|
||
| const entryByPath = new Map< string, { type?: string; target?: string } >(); | ||
| const dirPrefixes = new Set< string >(); | ||
| const contentEntries: string[] = []; | ||
|
|
||
| for ( const line of raw.split( '\n' ) ) { |
…tive-sync-and-interactive-file-selection-to-pull
|
@epeicher, I tested this and I am still testing it, but I want to share some initial findings. I ran into a couple of issues. So, I connected the remote site to a local Studio site and started testing selective sync. First I installed a new theme on the remote site, then pulled the site into Studio. That worked as expected, and the theme showed up locally. Next I installed the Code Snippets plugin on the remote site. I then ran a scoped pull for plugins and the database. After that pull, only Code Snippets was present locally. All of the other plugins were gone, including the managed ones like Akismet and Jetpack. 
I then tried a full pull again to restore the complete site. It did not bring the missing plugins back. I repeated the full pull three times, and the site still did not restore fully on my local site. ~$ STUDIO_ENABLE_PULL_REPRINT=1 node apps/cli/dist/cli/main.mjs pull-reprint \
--path ~/Studio/reprint-test \
--url https://perfectmainframe00.wpcomstaging.com
✔ Site loaded
Updating "reprint-test" from https://perfectmainframe00.wpcomstaging.com/ (delta sync)
Technical directory: /Users/gavande/.studio/pulls/b4fb45d4-6323-4328-8627-535fa18fba01
Site directory: /Users/gavande/Studio/reprint-test
✔ Connected – WordPress 7.0.1, PHP 8.4.23
✔ Select what to pull from the remote site 51 items selected
✔ Site pulled
✔ Site "reprint-test" updated
✔ WordPress server already running
Site "reprint-test" is ready.
Site URL: http://localhost:8887/studio-auto-login
WP Admin: http://localhost:8887/studio-auto-login?redirect_to=http%3A%2F%2Flocalhost%3A8887%2Fwp-admin%2F
Site "reprint-test" pulled successfully.
Site URL: http://localhost:8887/studio-auto-login
WP Admin: http://localhost:8887/studio-auto-login?redirect_to=http%3A%2F%2Flocalhost%3A8887%2Fwp-admin%2FAnother nit: Please notice that, the auto login links are printed twice. Note: I already confirmed that the reprint.phar in use was built from local Reprint trunk, not the packaged one. |
gcsecsey
left a comment
gcsecsey
left a comment
There was a problem hiding this comment.
I tested selective pull with multiple local sites, selectively pulling either plugins, themes, or both, but I wasn't able to reproduce the above issue.
I could test full and partial pulls from multiple Atomic sites, and the changes makes sense to me. I think this is good to go, and we can fix the reported issue if we can successfully reproduce it later.
|
Thanks for testing and for the review @gavande1!
I have tested this again, I have been able to partially pull the plugins and the Database, and the rest of the plugins are there. That issue seems to be related to this bug WordPress/reprint#334 so maybe the
That's existing Reprint behavior, it seems to be displaying the links after the two steps, feel free to create a follow-up task as this is not related with these changes. |
|
Thanks, @epeicher, for taking a look. I tested it on a fresh site and it worked well, but I hit the same issue on trunk, and it is reproducible there too. It looks like something needs to be fixed in Reprint. I connected a WP Cloud site and tested selective sync. Most of it worked, but WooCommerce did not come through. Journey:
Expected: WooCommerce should be present and active locally, matching the remote. |
…ve-file-selection-to-pull
Related issues
How AI was used in this PR
AI (Claude Code) implemented the selector, the CLI flags, the split execution pipeline, the first-pull preservation of local wp-content, and the unit tests, and verified the behavior end-to-end against a live WP.com site (full first pull, partial first pull with local content preserved,
--skip-database, scoped incremental re-pull, deferred media tail). I iterated multiple times over different approaches and verified the functionality manually.Proposed Changes
Adds selective sync to the experimental
pull-reprintcommand: users pick which wp-content folders, the media library, and the database to pull, and only that is fetched. Anything unselected keeps its local contents — including on a site's very first pull.--only,--skip-database,--skip-uploadsflags non-interactively. The choice is captured in aselection.jsonsidecar so a resumed pull reuses the exact same selection.reprint pullis replaced bypull-files→pull-db(skipped when the database is excluded) →flat-docroot→apply-runtime, with--onlyscoping the file download. On a first pull the preflight-detected WordPress core roots are added to any partial selection so the site stays bootable, and the unselected local wp-content (and a kept database) is carried into the scratch before flattening.The bulk of the diff handles WP Cloud's layout so a scoped pull still produces a working site — recreating the per-plugin symlinks a scoped listing omits, wiring a kept local database into the runtime, synthesizing the document-root
wp-config.phpthat falls outside the selection, and keeping the selector's indexing side-effect free. See the inline comments for the specifics.Testing Instructions
WordPress/reprint#334 (the fix for the
--onlyincremental data-loss bug) is now merged to Reprint trunk but not yet in a tagged release, so the bundledreprint.phar(v0.9.0) is still affected. Until a release is cut, build the phar from Reprint trunk (composer build:pharin the reprint repo) and copyreprint.pharoverapps/cli/dist/cli/wp-files/reprint/reprint.pharafternpm run cli:build.studio createor use an existing Studio siteSTUDIO_ENABLE_PULL_REPRINT=true node apps/cli/dist/cli/main.mjs pull-reprint --path <site> --url <remote> --verbosepull-reprintcommand on the same site, optionally, add a post or a media file to the remote site so you can test that it works as expected. Run the same command:STUDIO_ENABLE_PULL_REPRINT=true node apps/cli/dist/cli/main.mjs pull-reprint --path <site> --url <remote> --verbosePre-merge Checklist
reprint.pharupdated to a Reprint release containing Never delete the --only roots themselves in the files-pull delete drains WordPress/reprint#334 (merged to trunk; awaiting a tagged release).