Based on data collection, this paper summarizes the basic ideas of several texture compression formats, hoping to be helpful for learning.

Why do we need texture compression formats?

For example, uncompressed image formats such as R5G6B5, A4R4G4B4, A1R5G5B5, R8G8B8 or A8R8G8B8 are native texture formats that can be directly read by the GPU. But in low-end hardware devices or mobile platforms, there are two problems that need to be solved.

one isMemoryFor example, in the A8R8G8B8 format, one pixel occupies 4 bytes. If it is 512×512 resolution, it will occupy 512x512x4 B=1048576 B=1 MB. This kind of memory consumption is simply unacceptable on low-end devices.

Another important thing is when the data is transmittedbandwidthBandwidth is the culprit of heat generation. When rendering 3D scenes, a large number of textures will be transferred to the GPU. If it is not limited, the bus bandwidth will soon become a bottleneck, and the mobile phone will warm up in seconds, which will seriously affect the rendering performance. .

So we need a format that has a small memory footprint and can be read by the GPU – a compressed texture format. The corresponding algorithm for texture compression is some form of fixed-rate lossy vector quantization (Lossy Vector Quantization) that encodes a fixed-size block of pixels into a fixed-size block of bytes.

lossyIndicates that lossy compression is acceptable for rendering, and a balance between texture quality and file size is generally required when choosing a compression format.

Fixed rate compressionWhat do you mean? Because the GPU needs to be able to efficiently access a pixel at random, this means that the decoding speed should not change much for any pixel. Therefore, our common texture compression algorithms are lossy compression. Conversely, zip, for example, is a variable rate compression.

Vector Quantization (Vector Quantization, VQ)is a quantization technique that divides a large set of points (vectors) into groups with approximately the same number of points closest to them. Each group is represented by its centroid point, so there is data error, suitable for lossy compression. Putting it into texture compression to understand, for example, the color of 4×4 block pixels is represented by 2 primary colors.

Encoding and decoding speed: Generally speaking, it doesn’t matter if the encoding speed is slow, because usually texture compression only needs to be performed once when the game is packaged, which has no effect on the user’s runtime experience at all. But the decoding speed must be fast enough, and basically can not affect the rendering performance.

compression ratio: Usually expressed in bit rate or the average number of bits per pixel (bits per pixel, bpp), commonly 2~8bpp. Generally, the pixel of RGB native texture refers to 24 bits, and 4bpp means that each pixel occupies 4 bits, so it can also be considered that 4bpp means that the compression ratio is 6:1.

By the way, in Unity, there is an import process for any image file format, and the imported file format is Texture2D. In the import setting options of Texture2D, you need to set the texture compression format for different platforms.

Why don’t we use common compression formats like png and jpg?

Although the compression rate of jpg and png is very high, they are not suitable for textures. The main problem is that random access to pixels is not supported, which is quite unfriendly to GPU. When GPU rendering, only the required texture part is used, and it is impossible for us to access A certain pixel to decode the entire texture, right? Without knowing the order, and not sure if adjacent triangles are also adjacent on texture sampling, it’s hard to optimize. This type of format is more suitable for download transfers and to reduce disk space.

Common texture compression formats
ETC
ETC (Ericsson Texture Compression) was originally developed for mobile devices, and now it is the standard compression scheme for Android. ETC1 is supported in both OpenGL and OpenGL ES.

The principle is simple, it is a method of encoding a 4×4 pixel block into two blocks of 2×4 or 4×2 pixels, each block specifies a primary color, and the color of each pixel is encoded as a grayscale offset relative to these primary colors. value is determined.

Specifically, ETC1 encodes each 4×4 pixel block into 64-bit byte data, and each pixel block is divided into two 2×4 sub-blocks (horizontal or vertical division controlled by a “flip” bit), each sub-block contains a A 3-bit modifier table index and a base color value, either 2*R4G4B4 or R5G5B5+R3G3B3 (which is controlled by a “diff” bit).

The 3-digit modifier table index corresponds to 8 modifier values:

Such a sub-block can determine 4 new color values ​​from 1 basic color value and 4 modifier values:

color0 = base_color + RGB(modifier0, modifier0, modifier0) 

color1 = base_color + RGB(modifier1, modifier1, modifier1) 

color2 = base_color + RGB(modifier2, modifier2, modifier2) 

color3 = base_color + RGB(modifier3, modifier3, modifier3) 

The final color is selected from these 4 color values. The principle is that the other 32-bit data contains 16 2-bit selector data, and the color of each pixel is selected from these 4 types according to the value of the 2-bit selector.

The intuitive diagram of the ETC1 encoding method is as follows:

Several ETC texture compression formats for Unity:

RGB ETC1 4 bit: 4 bits/pixel, RGB compression ratio 6:1, does not support Alpha, most Android devices support it.

RGB ETC2 4 bit: 4 bits/pixel, 6:1 compression ratio to RGB. Alpha is not supported, ETC2 is compatible with ETC1, and the compression quality may be higher, but the block error with large chromaticity changes is also larger, which requires OpenGL ES 3.0 and OpenGL 4.3 or later.

RGBA ETC2 8bit: 8 bits/pixel, with a compression ratio of 4:1 to RGBA. Supports full transparent channel, the version requirements are the same as above.

RGB +1bit Alpha ETC2 4bit: 4 bits/pixel. It supports 1bit Alpha channel, that is, only hollow image is supported. The image has only transparent and opaque parts, and there is no intermediate transparency.

EAC: The core principle is the same as ETC, but it is only used for single-channel or dual-channel data. Most devices after OpenGL ES 3.0 and OpenGL 4.3 support it, but due to the various compatibility of the Android platform, it is generally not recommended to use single and dual channel textures .

DXT
Formerly known as S3TC (S3 Texture Compression), a block-related lossy compression algorithm developed by S3 Graphics, also called DXTn or DXTC (DirectX Texture), is an improvement of Block Truncation Coding. Due to copyright patents are generally used on the Windows platform.

The principle is simple, a 4×4 block of RGB pixels is described by a pair of low-precision “primary colors”, and each pixel is allowed to specify an interpolation between these primary colors. There are several variants of S3TC, each designed for a specific type of image data, but they all convert 4×4 blocks of pixels to 64-bit or 128-bit data.

BC1 (Block Compression) is the smallest variant and the one with the highest conversion ratio, which can be used when neither high precision nor a value is required. It stores 64-bit data as a block of 4×4 pixels, including two 16-bit RGB values ​​(R5G6B5, the human eye is more sensitive to green) $color_0$ and $color_1$, and 16 2-bit selector values , the final color value of a pixel is determined by $color_0$, $color_1$ and the corresponding 2-bit selector value. The mixing formula is as follows:

A visual diagram of how it works is as follows:

PS: The information loss of BC1 is mainly concentrated on relatively thin boundaries, which can be solved by increasing the resolution. The height and width are enlarged by 41% (1.41*1.41=1.9881), so that the entire texture is almost 2 times that of the previous one, but the total compression rate Still 3:1. Another loss is the conversion of 24-bit RGB to 16-bit color, which doesn’t look much different, but there are some subtle differences for gradients.

PPS: In Unity 2017, an improved DXT algorithm based on the Crunch library has been added, which occupies less disk space and decompresses faster, but the corresponding image quality is not as good as DXT.

BC2 supports a value based on BC1. It stores a total of 16 pixels in 4×4 as 128-bit data, including 64-bit a channel (4 bits per pixel) and 64-bit color channel, the color is consistent with DXT1, that is, two 16-bit RGB values ​​and 16 2-bit index tables x. Its transparency is only 4 bits and a total of 16 values. Compared with DXT5, some functions are basically unused. The difference between DXT2 and DXT3 is whether the color is premultiplied or not.

BC3 improves the a-value algorithm on the basis of BC2. BC3 stores a total of 16 pixels in 4×4 as 128-bit data, including 64-bit a channel (two 8-bit a values ​​and a 4×4 3-bit index table) and 64-bit color channel, the color is the same as DXT1, that is, two 16 bit RGB value and 16 2-bit index table x. Since the a value uses a block compression algorithm similar to that of color, the range of values ​​is wider, and BC is more common.

BC4 and BC5 are available in D3D10 and can only store one/two color channels.

BC6H and BC7 are available in D3D11. BC6H is an HDR format without a value, which stores 16 pixels in 4×4 as 128-bit data, including two 48-bit RGB values ​​(16:16:16), each color component is a signed float. Point value (1 sign bit + 5 exponent bits + 10 mantissa bits), and 16 2-bit index tables.

BC7 is a bit special compared to others. Although it also stores a total of 16 pixels in 4×4 as 128-bit data, its least significant bit is the Mode bit, (the least significant bit is the lowest non-0 bit). According to different modes, the color value of The storage format is different, whether there is a value or the storage format of the a value is also different. It is a relatively flexible storage format, but it also means more consumption caused by decoding.

PVRTC
PVRTC (PowerVR Texture Compression) is designed by Imagination for the core of PowerVR graphics cards. Due to patent reasons, it is generally only used in Apple devices, and only supported by iPhone, iPad and some Android devices of PowerVR. This is probably the least publicized technique of the several compression formats.

Different from block-based algorithms such as DXT and ETC, PVRTC divides the entire texture into high-frequency signals and low-frequency signals. The low-frequency signals are represented by two low-resolution images A and B, which are in two dimensions. Both are scaled down by a factor of 4, and the high-frequency signal is a full-resolution but low-precision modulated image M, which records the weight of each pixel’s mixing. When decoding, the A and B images are bilinearly enlarged by 4 times in width and height, and then mixed with the weights on the M image.

In PVRTC 4-bpp mode, each 4×4 pixel occupies a 64-bit data block, and in 2-bpp mode, there is a 64-bit data block for each 8×4 pixel. The two are similar, we only talk about 4-bpp mode.

In 4-bpp mode, only one color value is saved in both A and B after being reduced. As shown in the figure below, A is 1 bit less than B, but both images can be stored in RGB or ARGB (the highest bit Decide which one to use), A color can be encoded in RGB554 or ARGB3443 format, and B color can be encoded in RGB555 or ARGB3444 format.

When decoding, in order to decode an arbitrary pixel, 4 adjacent PVRTC blocks must be read and used to decode a 5×5 block.

Bilinear filtering is used to expand the A and B pictures, and then the A and B pictures are mixed with the “Mode” bit according to the M picture. When the “Mode” bit here is 1, the 10-valued pixels in the M picture are regarded as When “punch-through alpha” is turned on, the alpha channel will be forced to zero. This magical operation is for compatibility with old applications, so I won’t go into details.

From the 4-bpp mode alone, PVRTC is very similar to BC and ETC, with two color values, but the basic idea is different.

Several PVRTC texture compression formats of Unity:

All Apple mobile devices support PVRTC.

PVRTC2 is of higher quality than PVRTC, and supports NPOT (non-power of 2 textures), which is an upgraded version of PVRTC.

Both PVRTC and PVRTC2 support 4-bpp and 2-bpp ARGB formats.

RGB PVRTC 4 bit: 4 bits/pixel, RGB compression ratio 6:1, Android devices need PowerVR Series 5 or above.

RGBA PVRTC 4 bit: 4 bits/pixel, 8:1 compression ratio for RGBA, 3-bit Alpha value, the device is the same as above.

RGB PVRTC 2 bit: 2 bits/pixel, RGB compression ratio 6:1, Android and iOS devices need PowerVR Series 5X or above.

RGBA PVRTC 2 bit: 2 bits/pixel, 8:1 compression ratio for RGBA, 3-bit Alpha value, the device is the same as above.

ASTC
ASTC (Adaptive Scalable Texture Compression), jointly developed by ARM and AMD and released in 2012, is a relatively new compression format and the only compression format that is not affected by patent rights. ASTC is quite good in terms of compression rate, image quality, and types, and is gradually replacing the first three. The biggest disadvantage may be that the compatibility is not perfect and the decoding time is long. However, according to the current development trend of mobile terminals, GPU Computing power is becoming more and more difficult to become a bottleneck, so it is very hopeful that it will become a unified compression format in the future.

ASTC is also a block-based lossy compression algorithm. It is very similar to BC7, except that the number of pixels in the block is variable. It has many characteristics:

• High flexibility: supports 1-4 component textures
• Compression ratio/quality is flexible and variable: algorithms with different compression ratio levels will be selected according to different pictures
• 2D/3D texture support
• Cross-platform: iOS, Android, PC
• Support both LDR and HDR: BC6H supports HDR but not alpha channel

A block in ASTC format is a fixed size of 128 bits. In 2D texture coding, it has from 4×4 to 12×12 pixels, and the corresponding compression ratio is from 3:1 to 27:1. All supported blocks and bitrates are as follows:

The Increment column represents the increment of the compression ratio, which also means that the bit rate of ASTC can be changed in decimals, a technique called BISE (Bounded Integer Sequence Encoding).

ASTC’s compression is very complex, with a lot of variable configuration data, and I don’t plan to study it further.

But note that although textures can be encoded as 1-4 channel images, the decoded values ​​are always output in RGBA format. In LDR sRGB mode, the color value is returned as an 8-bit integer, while in the case of HDR it will be returned as a 16-bit float.

“ASTC Texture Compression Format” has done a detailed test on ASTC, and it is highly recommended to take a look.

Summarize

I have sorted out the information on the Internet and looked at the mobile device support in the market in 2020:

• Android: There is no problem with ETC2; as for ASTC after Android 5.0/OpenGL ES 3.1, most models in the market support it (98.5%), you can consider choosing, but after all, it is Android, so be prepared to deal with compatibility .
• iOS: ASTC is supported on iPhone6 ​​and above (included), PVRTC2 can be selected below 6.

Finally, a brief description of all compression formats follows, with grey boxes denoting obsolete or basically useless.

Reference source:

TEXTURE COMPRESSION TECHNIQUES

Recommended, default, and supported texture compression formats, by platform

Another Milestone for ASTC Texture Compression

ASTC does it

“ASTC Texture Compression Format”

“Mobile Platform Texture Compression Format Selection”

Why use crunch compression?

Crunch compression of ETC textures

ETC1 Compressed Texture Image Formats

Texture compression on mobile demystified

Texture compression using low-frequency signal modulation

This is the 1187th article of Yuhu Technology, thanks to the author Reverie Reverie. Welcome to forward and share, please do not reprint without the author’s authorization. If you have any unique insights or discoveries, please contact us and discuss together. (QQ group: 793972859)

Author homepage: https://www.zhihu.com/people/hei-jing-79

Thanks again for the sharing of Reverie, if you have any unique insights or discoveries, please contact us to discuss together. (QQ group: 793972859)