Introduction to Encoding and Compression

作者：罗上文，微信：Loken1，公众号：FFmpeg弦外之音

As we learned earlier, YUV420 already uses half as much data as YUV444, while the difference is almost imperceptible. As shown below:

As you can see above, although the data has been reduced by half, a single 1080p image is still 3038 kb, or about 3 MB. Smooth animation requires playing 24 images per second, so the size of a one-hour movie is: $3 *24 * 60 * 60 = 259200$ Therefore, a one-hour YUV420 movie needs 253 GB of storage. As of 2022, an ordinary hard drive was only 500 GB, enough for just two movies. Thus, even though the HVS technique used by YUV420 cuts the data volume in half, it still cannot meet the storage and transmission requirements of current hardware, so further compression is needed.

In the audio and video field, compression is called encode (encoding), because the term compression is quite generic: text strings can be compressed, as can other binary files. Compression algorithms for audio and video have their own industry-specific characteristics, so they are called encoding.

Common compression standards include the following, although this list is not exhaustive:

1. The JPEG standard, used for compressing individual images. Standard document: ISO/IEC 10918-1
2. The H.264 standard, which was very widely used as of 2022
3. VP9, a video codec standard developed by Google
4. AVS, a Chinese video compression standard
5. MP3, an audio compression standard, whose full name is MPEG-1 Audio Layer 3
6. AAC, an audio compression standard, whose full name is Advanced Audio Codec
7. Opus, an audio compression standard that does not require patent royalties

Audio coding techniques are usually divided into three categories:

1. Waveform coding
2. Parametric coding
3. Hybrid coding

Some of the techniques used in video coding systems are:

1. Prediction
2. Transform
3. Quantization
4. Entropy coding
5. In-loop filtering

It is fine if these terms are unfamiliar for now. They are introduced only briefly here to give you a general impression. This book focuses on how to use the FFmpeg API.

For now, you only need to know that after data such as YUV420 enters the coding system, the data coming out is much smaller. The exact reduction depends on the codec; H.264 can achieve a compression ratio of 102:1. That 253 GB YUV420 movie mentioned above would need only 2.5 GB after compression. What an astonishing ratio; this is the appeal of a coding system.

FFmpeg beginners do not need to master the specific algorithms and optimization techniques of codecs. If you are new to audio and video development, the first thing to learn is how to call the FFmpeg library APIs, including the codec APIs.

You can study the specific H.264 and AAC coding algorithms in depth after you become comfortable using FFmpeg.