AMD’s GPUOpen team announced a new open source project: Brotli-G, Brotli-G is based on the complete Brotli lossless compression algorithm standard (RFC7932) maintained by IETF, but can use GPU to further enhance the performance and compression ratio of Brotli compression algorithm.

The Brotli compression algorithm was developed by Google Research and first released in 2013, and the open source specification was developed and promoted between 2013 and 2016.

Brotli is a combination of LZ77 compression algorithm and Huffman encoding (similar to GZIP and Deflate). As the open source successor of GZIP and Deflate algorithm, Brotli has similar performance but better compression ratio.

Currently Brotli is widely used in web applications and content for compressing web application assets such as fonts, javascript, images, etc.

• Today more than 95% of web browsers support Brotli decompression.
• Many web application frameworks support Brotli encoding (such as NGINX and ASP.NET).
• Some CDNs use Brotli encoding (such as Azure and ImageKit)

Brotli-G performance optimization

Brotli-G Modifications to the standard Brotli bitstream format focus on allowing efficient data-parallel decompression on GPUs and multi-threaded CPUs, while still maintaining relatively high compression ratios. The main changes can be summarized as:

• Limits the size of blocks (pages) to be compressed, providing the ability to use multiple compute units (CUs) on the GPU.
• Implements parallel Huffman subflows, providing the ability to use SIMD processing.
• Simplified bitstream format to maximize GPU performance.

Currently the Brotli-G decompression algorithm is available as a “shader-shaders” program (using Microsoft’s HLSL HHLwritten) implementation that will run on any DirectX-compatible graphics card (but doesn’t seem to support AMD’s Radeon graphics cards…).Implementing decompression in shaders takes advantage of the massive parallelism of the GPU hardware for fast decompression.

Open sourced by AMD is a Brotli-G SDK that contains source code for CPU compression and decompression on both CPU and GPU. The SDK also includes source code for a CLI application that can be used to measure the performance of the Brotli-G codec.

One thing to note is that content that has been compressed using Brotli cannot be decompressed using the Brotli-G decompressor. To take advantage of Brotli-G, the content must be recompressed using a Brotli-G compatible compressor (also provided in the SDK).

The SDK is currently open-sourced under the MIT-based BCP 78 license (which is the license for the standard Brotli compression algorithm) and can be found in its GitHub repository.