Cranking Brotli up to 11 with Cloudflare Pro and 11ty

As with the past few blog posts I've written, this post is about my migration from GitHub Pages and Jekyll to Cloudflare Pages and 11ty. Once the migration was completed, I decided to have a look at how I could optimise the web performance of my blog. As with all web performance projects I start, I begin with the basics. For me, that's the 3 C's:

For assets being delivered to your user, make sure they are properly:

• Cached
• Concatenated
• Compressed

Before

Upon looking at the DevTools Network Panel in Firefox (my main browser of choice ����), I could see that only my HTML document was being Brotli compressed:

Before I get into the details, let's discuss what the difference between Gzip and Brotli is. If you already know all this, feel free to skip this section.

What is Brotli compression?

Brotli is a "new" compression algorithm that was released by Google in September 2015 (hence the quotes around the new). Brotli came with the following key features:

• Lossless Compression — No data is lost during compression / decompression.
• High Compression Ratios — It achieves higher compression ratios compared to other popular compression algorithms used on the web, like gzip, and deflate.
• Optimal for Web Performance — Brotli is specifically optimised for HTTP compression, its predefined static dictionary is optimised for patterns often found in web content. It's high compression ratios, and fast decompression, make it a perfect choice for HTTP compression.
• Static and Dynamic Compression — Brotli supports both dynamic (real-time) and static (pre-compressed) data compression, offering a balance between server CPU usage and optimal file size. Dynamic compression suits real-time needs, while static compression, using the highest setting (11), maximises asset compression during the build process. Pre-compressed assets can then be delivered with minimal CPU overhead.

It's these key features among many others that have made Brotli one of the preferred compression algorithms on the modern web. According to Can I Use, as of today, 97.79% of browsers on the web support Brotli Accept-Encoding/Content-Encoding.

I, personally, love that Brotli has compression settings ranging from 0 to 11, where 0 is the fastest compression speed but the lowest compression ratio, and 11 has the highest compression ratio but requires more computational resources and time. I'd love to know if the Google developers got the idea for a compression level of 11 from the very famous scene from the film 1984 film "This Is Spinal Tap".

If you are interested in learning more about Brotli compression, here's some further reading for you:

• RFC 7932.
• Google's Brotli Repository.
• Brotli: The New Era of Data Compression on Dev.to.
• Better than Gzip Compression with Brotli.
• Brotli Compression: A Fast Alternative to GZIP Compression from Kinsta.

Brotli and Cloudflare

Cloudflare has been a strong advocate for Brotli for many years, introducing support for it on September 21, 2016. Remarkably, this was just over a year after Google open-sourced Brotli on September 22, 2015, and only a few months after its formal standardisation in RFC 7932 in July 2016. Demonstrating impressive speed, Cloudflare rolled out Brotli support in just over two months, significantly ahead of one of their main competitors, Akamai, which implemented support on March 19, 2018—almost 18 months later.

At first, they were very explicit with their support, offering their users the option to toggle it on and off via the "Optimization" menu in their site dashboard:

Since then, Brotli has become so ubiquitous for web performance on the web that this toggle was removed from the dashboard in May 2024, as announced on their community forum here. It is now enabled by default for the following file types. Unfortunately, when moving from beta to live, Cloudflare decided that Brotli compression wouldn't be the default compression used on the free plan. Although, saying that, the free plan is still incredible for web performance! This sadly means that if you want all your website assets dynamically compressed (on the fly), by Cloudflare, then you will need to be on the Pro plan.

Another point to know about the Pro plan and Brotli compression is that it uses dynamic "on the fly" compression. It therefore uses a reduced compression setting. A happy medium between compression size and CPU usage happens to be around the 3 or 4 setting (out of 11). Cloudflare has chosen 4 to be the setting they use for dynamic compression. Depending on the content, Brotli's compression ratio at level 4 is approximately 2x to 4x, which is considered balanced compression. This level provides a good balance of speed and compression.

But since we are controlling the compression level on our static blog, let's crank it all the way up to 11! This is defined as maximum compression, and offers a compression ratio of approximately 4x to 7x smaller than the original file size. The use case for this level of compression is it's often used for static assets that don't change regularly, such as font files or infrequently updated files. In my case, once written, I don't plan to update the minimal amount of JavaScript in use on my blog. Hence, the JavaScript files are the perfect candidate for extreme compression. So let's get started.

Compression

Installation

The simplest way to compress your assets is by using a command-line tool in the terminal. To get started, you'll first need to install the Brotli Command-line Interface (CLI). On macOS, you can do this by either:

Using Homebrew

You may or may not have Homebrew already installed. Here I assume you haven't. To install it, you'd run the following commands in the terminal.

Install:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Ensure Homebrew is up-to-date:

brew update

Install Brotli:

brew install brotli

Verify Installation:

brotli --version

This will return the installed Brotli version. You are now ready to compress your assets using Brotli.

Using MacPorts

First, follow the MacPorts installation here.

Then update and install:

sudo port selfupdate
sudo port install brotli

Verify Installation:

brotli --version

These are just 2 ways of installing Brotli on OSX, others include:

• Build from source (using Xcode).
• Install via Python's pip.
• Download Precompiled Binaries (via the Brotli releases page).
• Use Docker.

Each method has its benefits depending on your environment and preferences. Once installed, the usage is the same for all methods listed above.

Usage

To compress a file, it's as simple as:

brotli [filename-here]

Or if you want to control the compression settings:

brotli --quality=[0-11] [filename-here]

This will output a Brotli version of the file so file.txt will be duplicated, and a new file called file.txt.br will be created. A shortcut for --quality=11 is --best:

So these commands would produce the same output:

brotli --quality=11 file.txt
brotli --best file.txt

For lots more information on the command-line options available, use:

brotli -h

Compression script — a single file

Now you can either do this manually, which is long and laborious, or we can semi-automate the task by writing a shell script (Thanks to ozcoder for the idea, and initial code!).

compress.sh

#!/bin/bash

# Check if the user provided a file name
if [ -z "$1" ]; then
	echo "Usage: $0 <filename>"
	exit 1
fi

# Get the file name from the first argument
fname="$1"

# Compress the file using Brotli at quality 11
brotli --quality=11 --output="${fname}.br" "${fname}"

# Notify the user
if [ $? -eq 0 ]; then
	echo "File compressed successfully: ${fname}.br"
else
	echo "Error occurred during compression."
	exit 1
fi

Remember to make the script executable using chmod +x compress.sh. Then you can use it like so:

./compress.sh filename.js

A Brotli compressed file (filename.js.br) will now be located in the same directory.

Compression script — whole directory

If you find manually compressing each file using the compress.sh script above tedious, then we can modify the shell script to compress all JavaScript assets in a single directory (compress-directory.sh):

compress-directory.sh

#!/bin/bash

# Check if the user provided a directory
if [ -z "$1" ]; then
	echo "Usage: $0 <directory>"
	exit 1
fi

# Get the directory from the first argument
directory="$1"

# Check if the specified directory exists
if [ ! -d "$directory" ]; then
	echo "Error: Directory '$directory' does not exist."
	exit 1
fi

# Compress all .js files in the specified directory (modify for other file types)
for file in "$directory"/*.js; do
	# Check if there are .js files in the directory
	if [ ! -e "$file" ]; then
		echo "No .js files found in '$directory'."
		exit 0
	fi

	# Compress the file using Brotli
	brotli --quality=11 --output="${file}.br" "$file"

	# Notify the user of success or failure
	if [ $? -eq 0 ]; then
		echo "Compressed: $file -> ${file}.br"
	else
		echo "Error compressing: $file"
	fi
done

echo "Compression complete."

Once either of these scripts have completed you will have one or more Brotli encoded files e.g. filename.js AND filename.js.br. You can now delete the original JavaScript file and remove the .br extension from the newly compressed file. Assuming you are using Git (which, of course, everyone does, right?), you can always retrieve the original JS file from the Git history should you need the uncompressed version again, or you can use the brotli --decompress filename.js.br command. Remember that Brotli is a Lossless compression algorithm, there's no data loss during compression and decompression, so you can always decompress to get the original file back.

My scripts are broken on localhost

So now that your JavaScript has been Brotli compressed, it's no longer in a plaintext format. Brotli compressed files are actually in a binary format, so they aren't directly editable in your code editor. These errors are in the console because presently, the local browser is expecting to receive the JavaScript in plaintext, so when it doesn't, it will let you know via the console with an error that says something like Uncaught SyntaxError: illegal character U+FFFD. Now, this is a scary looking error, but there's an easy fix. We need to tell our browser that these JavaScript files are no longer plaintext, they are Brotli compressed. To achieve that, we are going to modify our 11ty config file (e.g. eleventy.config.js) and add in some node middleware that our local 11ty development server will use.

// The rest of your eleventy.config.js code above...
eleventyConfig.setServerOptions({
	middleware: [(req, res, next) => {
		if (req.url.endsWith('.js')) {
			res.setHeader('Access-Control-Allow-Origin', '*');
			res.setHeader('Content-Encoding', 'br');
			res.setHeader('Content-Type', 'application/javascript');
			res.setHeader('Vary', 'Accept-Encoding');
		}
		next();
	}]
});
// The rest of your eleventy.config.js code below...

The code above will hopefully be fairly explanatory. It's basically saying when you serve a JavaScript file from localhost, make sure to include the above response headers. The critical one is the 'Content-Encoding', 'br' response header, which tells the browser that the files are now Brotli compressed. It then recognises that the response body has been compressed using the Brotli algorithm. The browser will then decompress the Brotli-compressed data before interpreting or rendering the content as it would do with a plaintext version.

Cloudflare Compression rules setup (Pro account)

Now that the development environment has been fixed, it's time to dive into the Cloudflare dashboard and make sure we have everything setup in there!

If you have a pro account, you will see a "Compression Rules" menu item under the "Rules" section of the navigation.

Once inside this menu, you will see an option to create a new compression rule:

You will see I've already created a "Brotli Compression" rule in the image, and it is enabled.

Within this section, you will see a long page with 2 main sections:

I added a name and just left the option on the default content types to be compressed.

Next, we need to set the compression settings we want to apply to these files being served to our users:

You will see in my settings I've been very explicit in the order of compression I would like: "Brotli, Zstandard, Gzip, Auto". Zstandard (Zstd) is generally considered better than Gzip in terms of performance and compression ratio for most use cases. Refer to this page here for more information about Zstandard.

After

Once all the settings have been updated, and we have cleared the CDN cache! We can see the final result from the network panel below:

Looking at the far-right column from the network panel, we can see that all my assets are now Brotli compressed.

Although the panel doesn't show it, the assets are being compressed using a mixture of dynamic and static compression:

• HTML document — Dynamic (level 4) — by Cloudflare.
• CSS file — Dynamic (level 4) — by Cloudflare.
• JavaScript files — Static (level 11) — by me.
• Favicon — Dynamic (level 4) — by Cloudflare.

If I wanted to, I could statically compress the favicon.ico file too, since that's never going to change. But I'll leave that to you the reader to figure out how to do that, should you wish to!

Summary

It's a real shame that you need a Cloudflare Pro account to use Brotli compression for all static assets served by Cloudflare. But their free account does give you a tremendous number of features for web performance and security out of the box, including:

• HTTP/2, and HTTP/3 + QUIC.
• 0-RTT.
• Cloudflare Workers.
• DNSSEC.
• Excellent cacheability of static assets.
• Email security and forwarding.
• Free hosting on Cloudflare Pages.
• Content Delivery Network (CDN) with global caching.
• Page Rules for fine-grained cache control.
• Improved security.
• Image optimisation.
• Improved web performance.
• DDoS protection.
• Basic Web Application Firewall (WAF) rules.
• SSL/TLS encryption (Free Universal SSL).
• Bot mitigation.
• IP masking via reverse proxy.
• Free DNS hosting with fast resolution.
• plus many other features too…

I'm genuinely interested to know if this can all be achieved on the free plan?

For example, by compressing your assets with Brotli (11), then setting your Pages _headers file to serve the 'Content-Encoding', 'br' response header along with these compressed assets. If so, I probably wouldn't recommend it as it may be against one of the Cloudflare Website and Online Services Terms of Use. And sounds like a quick way to get your account deleted by Cloudflare! If anyone tries this please do let me know! I really wish Cloudflare had a referral program, I'd probably get my Pro account paid for in no time!

And there we have it, another blog post off the back of my migration to 11ty. As always, thanks for reading, and I hope you found it useful and informative! If you have any feedback or comments, please do let me know!

Post changelog:

• 05/01/25: Initial post published.
• 05/01/25: Thanks to Barry Pollard for pointing out that I'd broken the shell scripts by adding a comment before the shebang! Doh!