Skip to content

Studio: Use local tree for push - #1940

Merged
katinthehatsite merged 65 commits into
trunkfrom
fix/pull-folder-structure
Nov 5, 2025
Merged

Studio: Use local tree for push#1940
katinthehatsite merged 65 commits into
trunkfrom
fix/pull-folder-structure

Conversation

@katinthehatsite

@katinthehatsite katinthehatsite commented Oct 27, 2025

Copy link
Copy Markdown
Contributor

Related issues

Fixes STU-756

Proposed Changes

This PR uses local tree of files from Studio sites for push operations instead of using hard-coded folders. For the plugins and themes folders, the depth is limited to individual plugins and themes and the user can not go further in. For folders like uploads, the user can go further into separate folders and individual uploads. Some files also remain hidden from the user such as SQLite integration database plugin:

Screenshot 2025-10-29 at 11 33 04 AM

Testing Instructions

  • Pull the changes from this branch
  • Start the app with npm start
  • Navigate to the Sync tab and connect a WPCOM site
  • Click on the Push option
  • Observe that the file structure reflects the local files (besides the files that should be hidden like sqlite-integration plugin)
  • Confirm that in the plugins and themes, we can only go to the individual plugins and themes and can't get deeper
  • Select some individual files
  • Start the push process
  • Confirm that it pushes selected files only (you can do this, for example, by removing the code that cleans up the zip to inspect the zip file that gets produced)
  • Confirm that push succeeds

Pre-merge Checklist

  • Have you checked for TypeScript, React or other console errors?
@katinthehatsite katinthehatsite self-assigned this Oct 27, 2025
@katinthehatsite
katinthehatsite marked this pull request as draft October 27, 2025 15:11
@github-actions

github-actions Bot commented Oct 27, 2025

Copy link
Copy Markdown
Contributor

📊 Performance Test Results

Comparing 62442b0 vs trunk

site-editor

Metric trunk 62442b0 Diff Change
load 9747.00 ms 9209.00 ms -538.00 ms 🟢 -5.5%

site-startup

Metric trunk 62442b0 Diff Change
siteCreation 10966.00 ms 10986.00 ms +20.00 ms 🔴 0.2%
siteStartup 3930.00 ms 3931.00 ms +1.00 ms 🔴 0.0%

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the file tree handling for push operations to use local file tree data instead of fetching individual wp-content folders. The changes establish a unified approach for both push and pull operations by loading complete file trees upfront, improving consistency and performance.

Key changes:

  • Replaced individual wp-content folder fetching with a single local file tree loader
  • Introduced new utility functions for sync exclusions and depth limiting
  • Updated test infrastructure to work with the new path-based tree structure

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
src/stores/sync/sync-hooks.ts Added useLocalFileTree hook for fetching local file tree data
src/stores/sync/index.ts Exported the new useLocalFileTree hook
src/preload.ts Renamed IPC handler from listWpContentFolders to listLocalFileTree
src/modules/sync/tests/index.test.tsx Added mock implementation for listLocalFileTree
src/modules/sync/tests/convert-tree-to-options-to-sync.tsx Refactored tests to use path-based tree structure instead of hook-based approach
src/modules/sync/lib/tree-utils.ts Created utility functions for sync exclusions and depth limiting
src/modules/sync/lib/convert-tree-to-sync-options.ts Added path-based logic for determining sync options and extracting selections
src/modules/sync/constants.ts Expanded folder list and added sync exclusion constants
src/modules/sync/components/sync-dialog.tsx Integrated local file tree loading for push operations
src/ipc-handlers.ts Replaced listWpContentFolders with recursive listLocalFileTree implementation
src/hooks/use-content-folders.ts Removed entire file as functionality replaced by new approach

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/modules/sync/lib/tree-utils.ts Outdated
Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts Outdated
Comment thread src/ipc-handlers.ts Outdated
Comment thread src/ipc-handlers.ts Outdated
Comment thread src/modules/sync/components/sync-dialog.tsx Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/ipc-handlers.ts Outdated
Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts Outdated
Comment thread src/ipc-handlers.ts Outdated
Comment thread src/modules/sync/components/sync-dialog.tsx
Comment thread src/modules/sync/hooks/use-selected-items-push-size.ts
Comment thread src/ipc-handlers.ts Outdated
@katinthehatsite
katinthehatsite marked this pull request as ready for review October 29, 2025 11:09
@katinthehatsite

Copy link
Copy Markdown
Contributor Author

Forgot to add to testing instructions that you can also add the Restore logs here https://mc.a8c.com/rewind/logs.php after push

@katinthehatsite
katinthehatsite requested a review from a team October 29, 2025 12:16
@katinthehatsite

Copy link
Copy Markdown
Contributor Author

@sejas @fredrikekelund I made some more changes to the exporter 😅 so feel free to give it another test. I will work on fixing tests in the meantime

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (4)

src/modules/sync/tests/convert-tree-to-options-to-sync.tsx:152

  • Test expectation uses outdated API. Should be { paths: ['plugins/plugin1', 'plugins/plugin3'] } to match the refactored implementation.
				specificSelections: {
					plugins: [ 'plugin1', 'plugin3' ],
				},

src/modules/sync/tests/convert-tree-to-options-to-sync.tsx:185

  • Test expectation should be updated to { paths: ['themes/theme1', 'themes/theme3'] } to reflect the new path-based API.
				specificSelections: {
					themes: [ 'theme1', 'theme3' ],
				},

src/modules/sync/tests/convert-tree-to-options-to-sync.tsx:218

  • Test expectation needs to be updated to use the new format: { paths: ['uploads/upload1', 'uploads/upload2'] }.
				specificSelections: {
					uploads: [ 'upload1', 'upload2' ],
				},

src/modules/sync/tests/convert-tree-to-options-to-sync.tsx:261

  • Test expectation uses the old multi-category format. Should be consolidated to { paths: ['plugins/plugin1', 'themes/theme1', 'themes/theme3'] } per the new implementation.
				specificSelections: {
					plugins: [ 'plugin1' ],
					themes: [ 'theme1', 'theme3' ],
				},

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/modules/sync/lib/tree-utils.ts
Comment thread src/ipc-handlers.ts Outdated
Comment thread src/modules/sync/components/sync-dialog.tsx Outdated
Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts

@fredrikekelund fredrikekelund left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is shaping up nicely! 👍 I can confirm that root-level files in wp-content are included in the push (still), and the logic is easier to follow now.

I left some comments that would be nice to address before landing, but I think we're on the home stretch now.

Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts Outdated
Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts Outdated
Comment thread src/modules/sync/lib/convert-tree-to-sync-options.ts
Comment thread src/lib/import-export/export/exporters/default-exporter.ts
Comment thread src/lib/import-export/export/types.ts Outdated
Comment thread src/ipc-handlers.ts Outdated
Comment thread src/modules/sync/constants.ts Outdated
Comment thread src/modules/sync/constants.ts
Comment thread src/modules/sync/components/sync-dialog.tsx Outdated
Comment thread src/modules/sync/components/sync-dialog.tsx Outdated

@sejas sejas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work Kat!
I tested many cases and they work pretty well. The only issue I found is that selecting wp-content (or just mu-plugins) also includes the sqlite-database-integration, which should be excluded.

Screenshot 2025-11-05 at 00 14 07

And selecting all (folders and database) it correctly excludes the sqlite-database-integration, however doesn't include the extra folders or files like extra-folder or test-wp-content.php.

Screenshot 2025-11-05 at 00 13 33

We could create a follow-up issue for that, but we should probably fix it before the next release.

}
}
return entry;
} );

@sejas sejas Nov 5, 2025

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have the opportunity to unify this if-else logic related to full vs partial export.
For example, currently the push doesn't exclude sqlite-database-integration, and the export site doesn't include other extra folders or files inside the wp-content folder.
We can tackle that as a follow-up.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katinthehatsite , I'll create a follow-up PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For example, currently the push doesn't exclude sqlite-database-integration, and the export site doesn't include other extra folders or files inside the wp-content folder.

Yeah I saw the bit with export, we could unify it. I left it as is for now to preserve the existing functionality.

For the sqlite-database-integration, let's fix it in this PR. It must be something small because this was working before. I will take a look in a bit

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To close the loop: Since we discussed this on Slack, we will solve both issues in #2004 and proceed with merging this one.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sejas sejas mentioned this pull request Nov 5, 2025
1 task
@katinthehatsite
katinthehatsite merged commit 31530d5 into trunk Nov 5, 2025
11 checks passed
@katinthehatsite
katinthehatsite deleted the fix/pull-folder-structure branch November 5, 2025 08:26
@sejas sejas mentioned this pull request Nov 5, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

5 participants