Upload media: decode high-bit-depth AVIF via the browser - #78893
Upload media: decode high-bit-depth AVIF via the browser#78893adamsilverstein wants to merge 4 commits into
Conversation
The bundled wasm-vips builds libaom without high-bit-depth support (CONFIG_AV1_HIGHBITDEPTH=0), so it cannot decode 10/12-bit AVIF. Uploading such an image failed client-side sub-size generation, which then deleted the already-uploaded original and surfaced a generic 'File could not be uploaded' error. Browsers decode high-bit-depth AVIF natively. Detect >8-bit AVIF with a cheap container header parse (no decode) and generate its sub-sizes via the browser decoder plus OffscreenCanvas instead of vips. The original AVIF is preserved unchanged; sub-sizes fall back to JPEG because canvas cannot encode AVIF. See #78889.
adamsilverstein
added
[Type] Bug
|
Size Change: +383 B (0%) Total Size: 8.59 MB 📦 View Changed
|
|
Alternative approach in #78894 (high-bit-depth |
| * @param scaledSuffix Whether to add a `-scaled` suffix (for threshold resizing). | ||
| * @return Resized ImageFile (JPEG) with dimension metadata. | ||
| */ | ||
| export async function canvasResizeImage( |
There was a problem hiding this comment.
Hmm, we shouldn't need this. Instead we can follow the HEIC approach and use the standard pipeline to process the extracted jpeg.
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
The bundled wasm-vips cannot decode 10/12-bit AVIF, so generating its sub-sizes previously required a bespoke per-sub-size canvas resize (canvasResizeImage) routed from resizeCropItem. Instead, decode the AVIF to a JPEG once up front in prepareItem (the same approach already used for HEIC) and feed that JPEG to the standard vips thumbnail pipeline as the item's sourceFile. The original AVIF is still uploaded untouched, so the attachment keeps its full bit depth; only the generated sub-sizes are flattened to 8-bit JPEG. Sub-sizes derive their basename from the server attachment but take the source file's extension when the formats differ, so JPEG sub-sizes of an AVIF original are named .jpg rather than .avif. Same-format uploads are unaffected. This removes canvasResizeImage and its detection branch, a net reduction of ~190 lines, while preserving the behavior asserted by the e2e test.
|
Flaky tests detected in d3c69c3. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/27035538702
|
…wser-decode # Conflicts: # packages/upload-media/CHANGELOG.md
…wser-decode # Conflicts: # packages/upload-media/CHANGELOG.md # packages/upload-media/src/store/private-actions.ts # test/e2e/specs/editor/various/client-side-media-processing.spec.js
|
Superseded by #79179. This PR worked around |
1 participant
What & why
Fixes the client-side-media half of #78889.
Uploading a high-bit-depth (10/12-bit) AVIF with client-side media processing active failed with a generic "File could not be uploaded". The bundled
wasm-vipsbuilds libaom with-DCONFIG_AV1_HIGHBITDEPTH=0, so it cannot decode (or encode) any AV1 stream above 8-bit. Sub-size generation threwBitstream not supported by this decoder, every sub-size sideload failed, and the pipeline then deleted the already-uploaded original and surfaced the generic error.Browsers decode high-bit-depth AVIF natively (the block preview already renders it). This PR uses that, reusing the exact same flow we already use for HEIC: decode the source to a JPEG once up front, then run that JPEG through the standard vips thumbnail pipeline.
How it works
avif-utils.ts—getAvifBitDepth()/isHighBitDepthAvif()read the bit depth straight from the ISOBMFF container (pixi/av1Cboxes). No decoding; falls back to "8-bit" if it can't parse, so the common path never changes.prepareItem— when an upload is a >8-bit AVIF, decode it to a JPEG once via the existingcanvasConvertToJpeg()helper (the same one HEIC uses) and set that JPEG as the item'ssourceFile. The original AVIF is still uploaded as the main file withgenerate_sub_sizes: false, so the attachment keeps its full bit depth.generateThumbnails— unchanged pipeline. It generates every sub-size fromsourceFile(now the JPEG) via the standardvipsResizeImage(). Sub-sizes derive their basename from the server attachment but take the source file's extension when the formats differ, so the JPEG sub-sizes of an AVIF original are named.jpgrather than.avif. Same-format uploads are unaffected.The original AVIF is uploaded untouched, so its full bit depth is preserved. Only the generated sub-sizes are flattened to 8-bit JPEG.
There is no longer a bespoke per-sub-size canvas resize:
canvasResizeImage()is removed andresizeCropItemis back to plainvipsResizeImage(). Net change is ~190 fewer lines.Pros and cons
Pros
wasm-vipsbinary/bundle size.canvasConvertToJpegonce + standard pipeline) instead of a parallel resize path.Cons
OffscreenCanvas.convertToBlob()cannot emit AVIF, so sub-sizes are written as JPEG — an AVIF attachment ends up with JPEG sub-sizes (mixed formats; WordPress handles this, but it's not ideal).For an alternative that keeps sub-sizes as full-bit-depth AVIF (at the cost of a larger
wasm-vipsbuild), see the companion PR that uses a high-bit-depth libaom build.Testing
avif-utils) and for the new sub-size naming ingenerateThumbnails(AVIF original →.jpgsub-sizes), alongside the existingcanvasConvertToJpegcoverage.1920x1080_e2e_test_image_10bit_hdr.avif) and asserts the upload succeeds, the original staysimage/avifat 1920×1080, and sub-sizes are generated (as JPEG). Fails ontrunk(upload cancelled); passes with this change.See #78889 for the full root-cause analysis.