[turbopack] add a new hasher implementation to eliminate allocations

[lukesandberg]

We only store hashes for TaskTypes in the TaskCache keyspace, so this introduces an allocation free path to compute it.

For writing out the TaskCache this wont make much of a difference since we were reusing a scratch buffer, but this should eliminate a source of allocations from the read path.

This does of course cause a small binary size regression 157M (160,952K) -> 158M (161,800K) = +1M (+848K)

[lukesandberg]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [turbopack] Add a simple tool to inspect serialized SST files and report on their contents #89310
• [turbopack] Make compression in SST files disableable #89309
• Add inline value support for small values (≤8 bytes) #89271
• Create a new compaction mechanism to ensure the TaskCache functions properly #89075
• Create a per family configuration that will allow us to optimize the file sizes for our different file types. #89058
• [turbopack] add a new hasher implementation to eliminate allocations #89059 👈 (View in Graphite)
• [turbopack] remove the _backend field from TaskGuardImpl #89015
• Avoid using TaskTypes as keys in storage instead use hashes #88904
• Turbopack: refactor data storage to avoid reverse task cache #88492 : 1 other dependent PR (#88889 )
• Turbopack: add caching to chunking #88561
• Turbopack: use selective reads for merged modules #88560
• Turbopack: selective read for module_chunk_groups in ChunkGroupInfo #88559
• Turbopack: MappedReadRef should be Send and Sync #88558
• canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 3 skipped benchmarks¹

---

_{Comparing persistent_hash (196ff15) with remove_backend_field (9824cff)}

Footnotes

1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[nextjs-bot]

Stats from current PR

✅ No significant changes detected

📊 All Metrics

📖 Metrics Glossary

Dev Server Metrics:

• Listen = TCP port starts accepting connections
• First Request = HTTP server returns successful response
• Cold = Fresh build (no cache)
• Warm = With cached build artifacts

Build Metrics:

• Fresh = Clean build (no .next directory)
• Cached = With existing .next directory

Change Thresholds:

• Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
• Size: Changes < 1KB AND < 1% are insignificant
• All other changes are flagged to catch regressions

⚡ Dev Server

Metric	Canary	PR	Change	Trend
Cold (Listen)	508ms	507ms	✓	▁▁█▁▁
Cold (Ready in log)	463ms	463ms	✓	██▆▆▇
Cold (First Request)	869ms	836ms	✓	▇▆▇▇▇
Warm (Listen)	507ms	509ms	✓	▁█▁▁█
Warm (Ready in log)	482ms	465ms	✓	▂▁▃▁▂
Warm (First Request)	367ms	356ms	✓	▅▁▅▁█

📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric	Canary	PR	Change	Trend
Cold (Listen)	455ms	456ms	✓	▁▁▁▁▁
Cold (Ready in log)	437ms	439ms	✓	▁▁▁▁▁
Cold (First Request)	1.770s	1.764s	✓	▁▁▁▁▁
Warm (Listen)	456ms	455ms	✓	▁▁▁▁▁
Warm (Ready in log)	438ms	437ms	✓	▁▁▁▁▁
Warm (First Request)	1.787s	1.775s	✓	▁▁▁▁▁

⚡ Production Builds

Metric	Canary	PR	Change	Trend
Fresh Build	4.209s	4.380s	✓	▂▁▁▁▁
Cached Build	4.341s	4.350s	✓	▁▁▁▁▁

📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric	Canary	PR	Change	Trend
Fresh Build	13.705s	13.767s	✓	▂▁▁▁▁
Cached Build	13.888s	13.855s	✓	▁▁▁▁▁
node_modules Size	463 MB	463 MB	✓	▁▁▁▁▁

📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **434 kB** → **434 kB** ✅ -2 B

81 files with content-based hashes (individual files not comparable between builds)

Server

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	765 B	765 B	✓
Total	765 B	765 B	✓

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	451 B	451 B	✓
Total	451 B	451 B	✓

📦 Webpack

Client

Main Bundles

	Canary	PR	Change
5528-HASH.js gzip	5.47 kB	N/A	-
6280-HASH.js gzip	54.4 kB	N/A	-
6335.HASH.js gzip	169 B	N/A	-
912-HASH.js gzip	4.53 kB	N/A	-
e8aec2e4-HASH.js gzip	62.5 kB	N/A	-
framework-HASH.js gzip	59.7 kB	59.7 kB	✓
main-app-HASH.js gzip	256 B	254 B	✓
main-HASH.js gzip	39 kB	39 kB	✓
webpack-HASH.js gzip	1.68 kB	1.68 kB	✓
262-HASH.js gzip	N/A	4.52 kB	-
2889.HASH.js gzip	N/A	169 B	-
5602-HASH.js gzip	N/A	5.48 kB	-
6948ada0-HASH.js gzip	N/A	62.5 kB	-
9544-HASH.js gzip	N/A	55.1 kB	-
Total	228 kB	228 kB	⚠️ +692 B

Polyfills

	Canary	PR	Change
polyfills-HASH.js gzip	39.4 kB	39.4 kB	✓
Total	39.4 kB	39.4 kB	✓

Pages

	Canary	PR	Change
_app-HASH.js gzip	194 B	194 B	✓
_error-HASH.js gzip	183 B	180 B	🟢 3 B (-2%)
css-HASH.js gzip	331 B	330 B	✓
dynamic-HASH.js gzip	1.81 kB	1.81 kB	✓
edge-ssr-HASH.js gzip	256 B	256 B	✓
head-HASH.js gzip	351 B	352 B	✓
hooks-HASH.js gzip	384 B	383 B	✓
image-HASH.js gzip	580 B	581 B	✓
index-HASH.js gzip	260 B	260 B	✓
link-HASH.js gzip	2.49 kB	2.49 kB	✓
routerDirect..HASH.js gzip	320 B	319 B	✓
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	315 B	315 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Total	7.97 kB	7.97 kB	✅ -1 B

Server

Edge SSR

	Canary	PR	Change
edge-ssr.js gzip	126 kB	126 kB	✓
page.js gzip	247 kB	248 kB	✓
Total	374 kB	374 kB	⚠️ +316 B

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	616 B	614 B	✓
middleware-r..fest.js gzip	156 B	155 B	✓
middleware.js gzip	33.1 kB	33 kB	✓
edge-runtime..pack.js gzip	842 B	842 B	✓
Total	34.7 kB	34.6 kB	✅ -66 B

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	732 B	736 B	✓
Total	732 B	736 B	⚠️ +4 B

Build Cache

	Canary	PR	Change
0.pack gzip	3.77 MB	3.77 MB	✓
index.pack gzip	102 kB	102 kB	✓
index.pack.old gzip	103 kB	102 kB	✓
Total	3.98 MB	3.98 MB	⚠️ +430 B

🔄 Shared (bundler-independent)

Runtimes

	Canary	PR	Change
app-page-exp...dev.js gzip	311 kB	311 kB	✓
app-page-exp..prod.js gzip	166 kB	166 kB	✓
app-page-tur...dev.js gzip	311 kB	311 kB	✓
app-page-tur..prod.js gzip	166 kB	166 kB	✓
app-page-tur...dev.js gzip	307 kB	307 kB	✓
app-page-tur..prod.js gzip	164 kB	164 kB	✓
app-page.run...dev.js gzip	307 kB	307 kB	✓
app-page.run..prod.js gzip	164 kB	164 kB	✓
app-route-ex...dev.js gzip	70.2 kB	70.2 kB	✓
app-route-ex..prod.js gzip	48.8 kB	48.8 kB	✓
app-route-tu...dev.js gzip	70.2 kB	70.2 kB	✓
app-route-tu..prod.js gzip	48.8 kB	48.8 kB	✓
app-route-tu...dev.js gzip	69.8 kB	69.8 kB	✓
app-route-tu..prod.js gzip	48.6 kB	48.6 kB	✓
app-route.ru...dev.js gzip	69.8 kB	69.8 kB	✓
app-route.ru..prod.js gzip	48.6 kB	48.6 kB	✓
dist_client_...dev.js gzip	324 B	324 B	✓
dist_client_...dev.js gzip	326 B	326 B	✓
dist_client_...dev.js gzip	318 B	318 B	✓
dist_client_...dev.js gzip	317 B	317 B	✓
pages-api-tu...dev.js gzip	43.1 kB	43.1 kB	✓
pages-api-tu..prod.js gzip	32.8 kB	32.8 kB	✓
pages-api.ru...dev.js gzip	43.1 kB	43.1 kB	✓
pages-api.ru..prod.js gzip	32.8 kB	32.8 kB	✓
pages-turbo....dev.js gzip	52.4 kB	52.4 kB	✓
pages-turbo...prod.js gzip	39.3 kB	39.3 kB	✓
pages.runtim...dev.js gzip	52.4 kB	52.4 kB	✓
pages.runtim..prod.js gzip	39.3 kB	39.3 kB	✓
server.runti..prod.js gzip	62.5 kB	62.5 kB	✓
Total	2.77 MB	2.77 MB	⚠️ +2 B

[nextjs-bot]

Tests Passed