H.264 Basic Concepts Summary: Slice and NALU
- 16 Mar, 2022
VCL and NAL
VCL: Video Coding Layer.
NAL: Network Abstraction Layer.
In the H.264/AVC video coding standard, the entire system framework is divided into two layers: the Video Coding Layer (VCL) and the Network Abstraction Layer (NAL). The former is responsible for effectively representing the content of video data; the latter is responsible for formatting the data and providing header information to ensure the data is suitable for transmission over various channels and storage media.
- Simply put, VCL refers to the encoded image data generated after an image frame undergoes video encoding using the H.264 algorithm. Therefore, the data encapsulated within a subsequent NALU includes two types: VCL and non-VCL. VCL represents the image encoding data, while non-VCL represents non-image compressed data;
- NAL, on the other hand, involves the reasonable and standardized encapsulation of the byte stream produced by H.264 image encoding. This facilitates its storage on various media or transmission over networks, ensuring it can be correctly parsed, read, decoded, and displayed by the playback end.
Frame and Slice
Each Frame is divided into one or more Slices for encoding, and each Slice can contain multiple macroblocks. H.264 performs encoding on a macroblock basis.
Specifically, the relationship between Frames and Slices in practice can be:
- A Frame may contain only one Slice, with all macroblocks within that Frame encoded within the same Slice;
- Each Frame may contain N Slices, and each Slice contains a fixed number M of macroblocks, where M and N are integers. The size of the byte stream generated after encoding each Slice varies significantly based on the motion and detail represented in the image within that Slice.
- Each Frame may contain N Slices, with an unfixed number of macroblocks in each Slice, but the size of the encoded byte stream condensed from each Slice is generally consistent after encoding.
NALU
NAL: Network Abstraction Layer.
NALU: Network Abstraction Layer Unit.
A NALU is the basic unit for saving and transmitting data generated by H.264 encoding. Data produced after encoding each Slice is packed and encapsulated using NALUs. In fact, a .h264 file consists of a continuous sequence of many NALUs.
- Of course, not all NALUs encapsulate encoded Slice data.
A standard NALU always consists of three parts: a start code (3 or 4 bytes), a NALU header (1 byte), and a NALU payload (variable length).
Every NALU consistently begins with 0x00 0x00 0x01 or 0x00 0x00 0x00 0x01. This sequence, 0x00 0x00 0x01 or 0x00 0x00 0x00 0x01, is known as the start code.
Immediately following the start code at the beginning of each NALU is the NALU header. The NALU header contains three fields:
- forbidden_zero_bit (1bit): The H.264 specification requires this bit to be set to 0.
- nal_ref_idc (2bit): This indicates the importance of the NALU. A higher value signifies greater importance. When the decoder is overwhelmed, NALUs with an importance of 0 can be discarded.
- nal_unit_type (5bit): This indicates the type of data contained in the Payload portion of this NALU structure.
nal_unit_type=6 indicates that this NALU contains SEI information.
nal_unit_type=7 indicates that this NALU contains SPS information.
nal_unit_type=8 indicates that this NALU contains PPS information.
- Relevant information for the above three types is compiled in H.264 Basic Concepts Summary: SPS, PPS, SEI.
The image compressed data (which is essentially VCL data) generated by H.264 after encoding a frame is stored in NALUs of types 1-5.
- Specifically, nal_unit_type=5 indicates that the current NALU encapsulates data for an IDR frame.