Create a new compaction mechanism to ensure the TaskCache functions properly

[lukesandberg]

When compacting TaskCache data, dedup by key and value. Ensure SSTs break sorting ties by value so that merging works correctly.

[lukesandberg]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [turbopack] Add a simple tool to inspect serialized SST files and report on their contents #89310
• [turbopack] Make compression in SST files disableable #89309
• Add inline value support for small values (≤8 bytes) #89271
• Create a new compaction mechanism to ensure the TaskCache functions properly #89075 👈 (View in Graphite)
• Create a per family configuration that will allow us to optimize the file sizes for our different file types. #89058
• [turbopack] add a new hasher implementation to eliminate allocations #89059
• [turbopack] remove the _backend field from TaskGuardImpl #89015
• Avoid using TaskTypes as keys in storage instead use hashes #88904
• Turbopack: refactor data storage to avoid reverse task cache #88492 : 1 other dependent PR (#88889 )
• Turbopack: add caching to chunking #88561
• Turbopack: use selective reads for merged modules #88560
• Turbopack: selective read for module_chunk_groups in ChunkGroupInfo #88559
• Turbopack: MappedReadRef should be Send and Sync #88558
• canary

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

Merging this PR will not alter performance

✅ 17 untouched benchmarks
⏩ 3 skipped benchmarks¹

---

_{Comparing collision_tolerant_compaction (c34eeaa) with family_size_configuration (d482259)²}

Footnotes

1. 3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on family_size_configuration (d482259) during the generation of this report, so 9c6dae8 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[nextjs-bot]

Failing test suites

Commit: a0ba566 | About building and testing Next.js

pnpm test packages/next/src/server/app-render/action-handler.test.ts (job)

Expand output

● Test suite failed to run

Jest encountered an unexpected token

Jest failed to parse a file. This happens e.g. when your code or its dependencies use non-standard JavaScript syntax, or when Jest is not configured to support such syntax.

Out of the box Jest supports Babel, which will be used to transform your files into valid JS based on your Babel configuration.

By default "node_modules" folder is ignored by transformers.

Here's what you can do:
 • If you are trying to use ECMAScript Modules, see https://jestjs.io/docs/ecmascript-modules for how to enable it.
 • If you are trying to use TypeScript, see https://jestjs.io/docs/getting-started#using-typescript
 • To have some of your "node_modules" files transformed, you can specify a custom "transformIgnorePatterns" in your config.
 • If you need a custom transformation specify a "transform" option in your config.
 • If you simply want to mock your non-JS modules (e.g. binary assets) you can stub them out with the "moduleNameMapper" config option.

You'll find more details and examples of these config options in the docs:
https://jestjs.io/docs/configuration
For information about custom transformations, see:
https://jestjs.io/docs/code-transformation

Details:

C:\actions-runner\_work\next.js\next.js\packages\next\src\server\app-render\action-handler.ts:1
({"Object.<anonymous>":function(module,exports,require,__dirname,__filename,jest){import { RSC_HEADER, RSC_CONTENT_TYPE_HEADER, NEXT_ROUTER_STATE_TREE_HEADER, ACTION_HEADER, NEXT_ACTION_NOT_FOUND_HEADER, NEXT_ROUTER_PREFETCH_HEADER, NEXT_ROUTER_SEGMENT_PREFETCH_HEADER, NEXT_URL, NEXT_ACTION_REVALIDATED_HEADER } from '../../client/components/app-router-headers';
                                                                                  ^^^^^^

SyntaxError: Cannot use import statement outside a module

  3 | describe('parseHostHeader', () => {
  4 |   it('should return correct host', () => {
> 5 |     expect(parseHostHeader({})).toBe(undefined)
    |                        ^
  6 |
  7 |     expect(
  8 |       parseHostHeader({

  at Runtime.createScriptFromCode (../node_modules/.pnpm/jest-runtime@29.7.0/node_modules/jest-runtime/build/index.js:1505:14)
  at Object.<anonymous> (../packages/next/src/server/app-render/action-handler.test.ts:5:24)

[nextjs-bot]

Stats from current PR

✅ No significant changes detected

📊 All Metrics

📖 Metrics Glossary

Dev Server Metrics:

• Listen = TCP port starts accepting connections
• First Request = HTTP server returns successful response
• Cold = Fresh build (no cache)
• Warm = With cached build artifacts

Build Metrics:

• Fresh = Clean build (no .next directory)
• Cached = With existing .next directory

Change Thresholds:

• Time: Changes < 50ms AND < 10%, OR < 2% are insignificant
• Size: Changes < 1KB AND < 1% are insignificant
• All other changes are flagged to catch regressions

⚡ Dev Server

Metric	Canary	PR	Change	Trend
Cold (Listen)	506ms	505ms	✓	▁▁█▁▁
Cold (Ready in log)	466ms	466ms	✓	██▆▆▇
Cold (First Request)	830ms	838ms	✓	▇▆▇▇▇
Warm (Listen)	506ms	507ms	✓	▁█▁▁█
Warm (Ready in log)	464ms	467ms	✓	▂▁▃▁▂
Warm (First Request)	337ms	339ms	✓	▅▁▅▁█

📦 Dev Server (Webpack) (Legacy)

📦 Dev Server (Webpack)

Metric	Canary	PR	Change	Trend
Cold (Listen)	457ms	506ms	✓	▁▁▁▁▁
Cold (Ready in log)	460ms	462ms	✓	▁▁▁▁▁
Cold (First Request)	1.904s	1.872s	✓	▁▁▁▁▁
Warm (Listen)	507ms	506ms	✓	▁▁▁▁▁
Warm (Ready in log)	463ms	460ms	✓	▁▁▁▁▁
Warm (First Request)	1.862s	1.914s	✓	▁▁▁▁▁

⚡ Production Builds

Metric	Canary	PR	Change	Trend
Fresh Build	4.115s	4.121s	✓	▂▁▁▁▁
Cached Build	4.088s	4.097s	✓	▁▁▁▁▁

📦 Production Builds (Webpack) (Legacy)

📦 Production Builds (Webpack)

Metric	Canary	PR	Change	Trend
Fresh Build	14.237s	14.228s	✓	▂▁▁▁▁
Cached Build	14.314s	14.322s	✓	▁▁▁▁▁
node_modules Size	463 MB	463 MB	✓	▁▁▁▁▁

📦 Bundle Sizes

Bundle Sizes

⚡ Turbopack

Client

Main Bundles: **434 kB** → **434 kB** ✅ -2 B

81 files with content-based hashes (individual files not comparable between builds)

Server

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	765 B	765 B	✓
Total	765 B	765 B	✓

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	451 B	451 B	✓
Total	451 B	451 B	✓

📦 Webpack

Client

Main Bundles

	Canary	PR	Change
5528-HASH.js gzip	5.47 kB	N/A	-
6280-HASH.js gzip	54.4 kB	N/A	-
6335.HASH.js gzip	169 B	N/A	-
912-HASH.js gzip	4.53 kB	N/A	-
e8aec2e4-HASH.js gzip	62.5 kB	N/A	-
framework-HASH.js gzip	59.7 kB	59.7 kB	✓
main-app-HASH.js gzip	256 B	254 B	✓
main-HASH.js gzip	39 kB	39 kB	✓
webpack-HASH.js gzip	1.68 kB	1.68 kB	✓
262-HASH.js gzip	N/A	4.52 kB	-
2889.HASH.js gzip	N/A	169 B	-
5602-HASH.js gzip	N/A	5.48 kB	-
6948ada0-HASH.js gzip	N/A	62.5 kB	-
9544-HASH.js gzip	N/A	55.1 kB	-
Total	228 kB	228 kB	⚠️ +692 B

Polyfills

	Canary	PR	Change
polyfills-HASH.js gzip	39.4 kB	39.4 kB	✓
Total	39.4 kB	39.4 kB	✓

Pages

	Canary	PR	Change
_app-HASH.js gzip	194 B	194 B	✓
_error-HASH.js gzip	183 B	180 B	🟢 3 B (-2%)
css-HASH.js gzip	331 B	330 B	✓
dynamic-HASH.js gzip	1.81 kB	1.81 kB	✓
edge-ssr-HASH.js gzip	256 B	256 B	✓
head-HASH.js gzip	351 B	352 B	✓
hooks-HASH.js gzip	384 B	383 B	✓
image-HASH.js gzip	580 B	581 B	✓
index-HASH.js gzip	260 B	260 B	✓
link-HASH.js gzip	2.49 kB	2.49 kB	✓
routerDirect..HASH.js gzip	320 B	319 B	✓
script-HASH.js gzip	386 B	386 B	✓
withRouter-HASH.js gzip	315 B	315 B	✓
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Total	7.97 kB	7.97 kB	✅ -1 B

Server

Edge SSR

	Canary	PR	Change
edge-ssr.js gzip	126 kB	126 kB	✓
page.js gzip	247 kB	248 kB	✓
Total	374 kB	374 kB	⚠️ +323 B

Middleware

	Canary	PR	Change
middleware-b..fest.js gzip	616 B	615 B	✓
middleware-r..fest.js gzip	156 B	155 B	✓
middleware.js gzip	33.1 kB	33.1 kB	✓
edge-runtime..pack.js gzip	842 B	842 B	✓
Total	34.7 kB	34.8 kB	⚠️ +34 B

Build Details

Build Manifests

	Canary	PR	Change
_buildManifest.js gzip	732 B	736 B	✓
Total	732 B	736 B	⚠️ +4 B

Build Cache

	Canary	PR	Change
0.pack gzip	3.76 MB	3.77 MB	🔴 +10.5 kB (+0%)
index.pack gzip	101 kB	102 kB	✓
index.pack.old gzip	102 kB	102 kB	✓
Total	3.97 MB	3.98 MB	⚠️ +10.2 kB

🔄 Shared (bundler-independent)

Runtimes

	Canary	PR	Change
app-page-exp...dev.js gzip	311 kB	311 kB	✓
app-page-exp..prod.js gzip	166 kB	166 kB	✓
app-page-tur...dev.js gzip	311 kB	311 kB	✓
app-page-tur..prod.js gzip	166 kB	166 kB	✓
app-page-tur...dev.js gzip	307 kB	307 kB	✓
app-page-tur..prod.js gzip	164 kB	164 kB	✓
app-page.run...dev.js gzip	307 kB	307 kB	✓
app-page.run..prod.js gzip	164 kB	164 kB	✓
app-route-ex...dev.js gzip	70.2 kB	70.2 kB	✓
app-route-ex..prod.js gzip	48.8 kB	48.8 kB	✓
app-route-tu...dev.js gzip	70.2 kB	70.2 kB	✓
app-route-tu..prod.js gzip	48.8 kB	48.8 kB	✓
app-route-tu...dev.js gzip	69.8 kB	69.8 kB	✓
app-route-tu..prod.js gzip	48.6 kB	48.6 kB	✓
app-route.ru...dev.js gzip	69.8 kB	69.8 kB	✓
app-route.ru..prod.js gzip	48.6 kB	48.6 kB	✓
dist_client_...dev.js gzip	324 B	324 B	✓
dist_client_...dev.js gzip	326 B	326 B	✓
dist_client_...dev.js gzip	318 B	318 B	✓
dist_client_...dev.js gzip	317 B	317 B	✓
pages-api-tu...dev.js gzip	43.1 kB	43.1 kB	✓
pages-api-tu..prod.js gzip	32.8 kB	32.8 kB	✓
pages-api.ru...dev.js gzip	43.1 kB	43.1 kB	✓
pages-api.ru..prod.js gzip	32.8 kB	32.8 kB	✓
pages-turbo....dev.js gzip	52.4 kB	52.4 kB	✓
pages-turbo...prod.js gzip	39.3 kB	39.3 kB	✓
pages.runtim...dev.js gzip	52.4 kB	52.4 kB	✓
pages.runtim..prod.js gzip	39.3 kB	39.3 kB	✓
server.runti..prod.js gzip	62.5 kB	62.5 kB	✓
Total	2.77 MB	2.77 MB	⚠️ +2 B