Performance: Parallelize recursive directory copying without fs-copy

[ivan-ottinger]

Related issues

• Related to STU-980
• Related to Performance: Parallelize recursive directory copying #2409

Proposed Changes

• parallelize recursive directory copying

Related discussion: #2409 (comment)

Here are the performance test results - comparing current trunk with this branch:

  Site Creation Time on macOS (7 rounds)
  ┌────────┬─────────┬──────────────────┐
  │ Round  │  trunk  │ update/file-copy │
  ├────────┼─────────┼──────────────────┤
  │ 1      │ 7042 ms │ 6046 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 2      │ 7040 ms │ 7032 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 3      │ 7058 ms │ 6030 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 4      │ 7050 ms │ 7040 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 5      │ 8098 ms │ 7056 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 6      │ 7051 ms │ 6041 ms          │
  ├────────┼─────────┼──────────────────┤
  │ 7      │ 7037 ms │ 6047 ms          │
  ├────────┼─────────┼──────────────────┤
  │ Median │ 7050 ms │ 6047 ms          │
  └────────┴─────────┴──────────────────┘
That is a -1003 ms (14.2% faster) improvement over current trunk.


 Site Creation Time on Windows via Parallels (5 rounds - as it took quite a long time, I decided to skip two rounds)
  ┌────────┬───────────┬──────────────────┐
  │ Round  │   trunk   │ update/file-copy │
  ├────────┼───────────┼──────────────────┤
  │ 1      │ 34,618 ms │ 32,651 ms        │
  ├────────┼───────────┼──────────────────┤
  │ 2      │ 32,572 ms │ 30,629 ms        │
  ├────────┼───────────┼──────────────────┤
  │ 3      │ 31,623 ms │ 31,652 ms        │
  ├────────┼───────────┼──────────────────┤
  │ 4      │ 32,639 ms │ 31,537 ms        │
  ├────────┼───────────┼──────────────────┤
  │ 5      │ 31,518 ms │ 31,624 ms        │
  ├────────┼───────────┼──────────────────┤
  │ Median │ 32,572 ms │ 31,624 ms        │
  └────────┴───────────┴──────────────────┘
That is a -948 ms (2.9% faster) improvement over current trunk.

Testing Instructions

1. Check out the PR branch and build the app with npm install && npm start.
2. Create several new sites.
3. The process should work correctly, the sites should get created and run without any issues.

Pre-merge Checklist

• Have you checked for TypeScript, React or other console errors?

[epeicher]

Thanks @ivan-ottinger for following up on this! I tested it and found no issues. I also tested it on Windows and noticed slightly better performance. The changes look good to me!

[sejas]

Nice improvement. Step by step we can make Studio run faster.

[wojtekn]

@ivan-ottinger do you know what caused performance downgrade after merging previous PR?

[ivan-ottinger]

@ivan-ottinger do you know what caused performance downgrade after merging previous PR?

Hey Wojtek 👋🏼 You mean this one (#2409), right? It is not clear - because when we reverted the change in #2520, the tests did not observe any change in performance.

I think it could have been related to that specific performance tests run recorded for that PR. Longer time could have been due to some other factors. Because of that, I run multiple rounds which I shared in this PR description where the median shows clear improvement in the copy approach introduced in this PR.