forked from AOMediaCodec/av1-mpeg2-ts
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
328 lines (250 loc) · 21.6 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<title>
Carriage of AV1 in MPEG-2 TS
</title>
<script src='https://aomediacodec.github.io/respec/builds/respec-aom.js' async class='remove'></script>
<script class='remove'>
var respecConfig = {
specStatus: "WGD",
shortName: "av1-mpeg2-ts",
isPreview: true,
editors: [{
name: "Kieran Kunhya",
mailto: "kierank@videolan.org"
}, {
name: "Thibaud Biatek",
mailto: "t.biatek@ateme.com",
}],
publishDate: "2021-10-04",
github: "AOMedia/av1-mpeg2-ts",
localBiblio: {
"MPEG-2 TS": {
title: "Information technology — Generic coding of moving pictures and associated audio information — Part 1: Systems",
href: "https://www.iso.org/standard/83239.html",
status: "Standard",
publisher: "ISO",
},
"ETSI EN 300 468": {
title: "Digital Video Broadcasting (DVB); Specification for Service Information (SI) in DVB systems",
href: "https://www.etsi.org/deliver/etsi_en/300400_300499/300468/",
status: "Standard",
publisher: "ETSI",
},
"BT-1886": {
title: "Recommendation BT.1886 : Reference electro-optical transfer function for flat panel displays used in HDTV studio production",
href: "https://www.itu.int/rec/R-REC-BT.1886",
status: "Recommendation",
publisher: "ITU"
},
"BT-709": {
title: "Recommendation BT.709 : Parameter values for the HDTV standards for production and international programme exchange",
href: "https://www.itu.int/rec/R-REC-BT.709",
status: "Recommendation",
publisher: "ITU"
},
"BT-2020": {
title: "Recommendation BT.2020 : Parameter values for ultra-high definition television systems for production and international programme exchange",
href: "https://www.itu.int/rec/R-REC-BT.2020",
status: "Recommendation",
publisher: "ITU"
},
},
};
</script>
</head>
<body>
# Introduction
This document specifies how to carry AV1 video elementary streams ([[AV1]]) in the MPEG-2 Transport Stream format ([[MPEG-2 TS]]). It does not specify the presentation of AV1 streams in the context of a program stream.
This document defines the carriage of AV1 in a single PID, assuming buffer model info from the first operating point. It may not be optimal for layered streams or streams with multiple operating points. Future versions may incorporate this capability.
## Modal verbs terminology
In the present document "shall", "shall not", "should", "should not", "may", "need not", "will", "will not", "can" and "cannot" are to be interpreted as described in clause 3.2 of the ETSI Drafting Rules (Verbal forms for the expression of provisions).
## Definition of mnemonics and syntax function
In the present document the mnemonics, the syntax functions, and the syntax descriptors are to be interpreted as described in [[MPEG-2 TS]]. The **uimsbf** and **bslbf** mnemonics are defined in Section 2.2.6 of [[MPEG-2 TS]]. The **nextbits()** function is interpreted as in [[MPEG-2 TS]].
<section class="normative">
<h1>Identifying AV1 streams in MPEG-2 TS</h1>
## AV1 registration descriptor
The presence of a Registration Descriptor, as defined in [[!MPEG-2 TS]], is mandatory with the *format_identifier* field set to 'AV01' (A-V-0-1). The Registration Descriptor shall be the first in the PMT loop and included before the AV1 video descriptor.
### Syntax
| Syntax | No. Of bits | Mnemonic |
|:---------------------------------|:-----------:|:----------:|
| registration_descriptor() { | | |
| **descriptor_tag** | **8** | **uimsbf** |
| **descriptor_length** | **8** | **uimsbf** |
| **format_identifier** | **32** | **uimsbf** |
| } | | |
### Semantics
**descriptor_tag** - This value shall be set to 0x05.
**descriptor_length** - This value shall be set to 4.
**format_identifier** - This value shall be set to 'AV01' (A-V-0-1).
## AV1 video descriptor
The AV1 video descriptor provides basic information for identifying coding parameters, such as profile and level parameters of an AV1 video stream. The same data structure as **AV1CodecConfigurationRecord** in ISOBMFF is used to aid conversion between the two formats, EXCEPT that two of the reserved bits are used for HDR/WCG identification. The syntax and semantics for this descriptor appears in the table below and in the subsequent text.
If an AV1 video descriptor is associated with an AV1 video stream, then this descriptor shall be conveyed in the descriptor loop for the respective elementary stream entry in the program map table.
### Syntax
| Syntax | No. Of bits | Mnemonic |
|:---------------------------------|:-----------:|:----------:|
| AV1_video_descriptor() { | | |
| **descriptor_tag** | **8** | **uimsbf** |
| **descriptor_length** | **8** | **uimsbf** |
| **marker** | **1** | **bslbf** |
| **version** | **7** | **uimsbf** |
| **seq_profile** | **3** | **uimsbf** |
| **seq_level_idx_0** | **5** | **uimsbf** |
| **seq_tier_0** | **1** | **bslbf** |
| **high_bitdepth** | **1** | **bslbf** |
| **twelve_bit** | **1** | **bslbf** |
| **monochrome** | **1** | **bslbf** |
| **chroma_subsampling_x** | **1** | **bslbf** |
| **chroma_subsampling_y** | **1** | **bslbf** |
| **chroma_sample_position** | **2** | **uimsbf** |
| **hdr_wcg_idc** | **2** | **uimsbf** |
| **reserved_zeros** | **1** | **bslbf** |
| **initial_presentation_delay_present** | **1** | **bslbf** |
| if (initial_presentation_delay_present) { |
| **initial_presentation_delay_minus_one** | **4** | **uimsbf** |
| } else { | | |
| **reserved_zeros** | **4** | **uimsbf** |
| } | | |
| } | | |
### Semantics
**descriptor_tag** - This value shall be set to 0x80.
**descriptor_length** - This value shall be set to 4.
**marker** - This value shall be set to 1.
**version** - This field indicates the version of the AV1_video_descriptor. This value shall be set to 1.
**seq_profile**, **seq_level_idx_0** and **high_bitdepth** - These fields shall be coded according to the semantics defined in [[!AV1]]. If these fields are not coded in the Sequence Header OBU in the AV1 video stream, the inferred values are coded in the descriptor.
**seq_tier_0**, **twelve_bit**, **monochrome**, **chroma_subsampling_x**, **chroma_subsampling_y**, **chroma_sample_position** - These fields shall be coded according to the same semantics when they are present. If they are not present, they will be coded using the value inferred by the semantics.
**hdr_wcg_idc** - The value of this syntax element indicates the presence or absence of high dynamic range (HDR) and/or wide color gamut (WCG) video components in the associated PID according to the table below. HDR is defined to be video that has high dynamic range if the video stream EOTF is higher than the reference EOTF defined in [[!BT-1886]]. WCG is defined to be video that is coded using colour primaries with a colour gamut not contained within [[!BT-709]].
| **hdr_wcg_idc** | **Description** |
|:---------------:|:----------------|
| 0 | SDR, i.e., video is based on the reference EOTF defined in [[!BT-1886]] with a color gamut that is contained within [[!BT-709]] with a [[!BT-709]] container |
| 1 | WCG only, i.e., video color gamut in a [[!BT-2020]] container that exceeds [[!BT-709]] |
| 2 | Both HDR and WCG are to be indicated in the stream |
| 3 | No indication made regarding HDR/WCG or SDR characteristics of the stream |
**reserved_zeros** - Will be set to zeroes.
**initial_presentation_delay_present** - Indicates **initial_presentation_delay_minus_one** field is present.
**initial_presentation_delay_minus_one** - Ignored for [[MPEG-2 TS]] use, included only to aid conversion to/from ISOBMFF.
</section>
# Constraints on AV1 streams in MPEG-2 TS
## General constraints
For AV1 video streams, the following constraints apply:
* An AV1 video stream conforming to a profile defined in Annex A of [[!AV1]] shall be an element of an MPEG-2 program and the stream_type for this elementary stream shall be equal to 0x06 (MPEG-2 PES packets containing private data).
* An AV1 video stream shall have the low overhead byte stream format as defined in [[!AV1]].
* The sequence_header_obu as specified in [[!AV1]], that are necessary for decoding an AV1 video stream shall be present within the elementary stream carrying that AV1 video stream.
* An OBU may contain the *obu_size* field. For applications that need easy conversion to MP4, using the *obu_size* field is recommended.
* OBU trailing bits should be limited to byte alignment and should not be used for padding.
* Tile List OBUs shall not be used
* Temporal Delimiters may be removed
* Redundant Frame Headers and Padding OBUs may be used.
In addition, a start code insertion and emulation prevention process shall be performed on the AV1 Bitstream prior to its PES encapsulation. This process is described in section 3.2.
## Start-code based format
Prior to carriage into PES, the AV1 **open_bitstream_unit()** is encapsulated into **ts_open_bitstream_unit()**. This is required to provide direct access to OBU through a start-code mechanism inserted prior to each OBU. The following syntax describes how to retrieve the **open_bitstream_unit()** from the **ts_open_bitstream_unit()** (tsOBU).
| Syntax | No. Of bits | Mnemonic |
|:------------------------------------------------------------------|:-----------:|:----------:|
| ts_open_bitstream_unit(NumBytesInTsObu) { | | |
| obu_start_code /* equal to 0x01 */ | **24** | **uimsbf** |
| NumBytesInObu = 0 | | |
| for( i = 2; i < NumBytesInTsObu; i++ ) { | | |
| if( i + 2 < NumBytesInTsObu && nextbits(24) == 0x000003 ) { | | |
| open_bitstream_unit[NumBytesInObu++] | **8** | **uimsbf** |
| open_bitstream_unit[NumBytesInObu++] | **8** | **uimsbf** |
| i += 2 | | |
| emulation_prevention_three_byte /* equal to 0x03 */ | **8** | **uimsbf** |
| } else | | |
| open_bitstream_unit[NumBytesInObu++] | **8** | **uimsbf** |
| } | | |
**obu_start_code** - This value shall be set to 0x000001.
**open_bitstream_unit[i]** - i-th byte of the AV1 open bitstream unit (As defined in section 5.3 of [[!AV1]]).
It is the responsability of the TS muxer to prevent start code emulation by escaping all the forbidden three-byte sequences using the **emulation_prevention_three_byte** (always equal to 0x03). The forbidden sequences are defined below.
Within the **ts_open_bitstream_unit()** payload, the following three-byte sequences shall not occur at any byte-aligned position :
* 0x000000
* 0x000001
* 0x000002
Within the **ts_open_bitstream_unit()** payload, any four-byte sequence that starts with 0x000003 other than the following sequences shall not occur at any byte-aligned position :
* 0x00000300
* 0x00000301
* 0x00000302
* 0x00000303
## The AV1 Access Unit
An AV1 Access Unit consists of all OBUs, including headers, between the end of the last OBU associated
with the previous frame, and the end of the last OBU associated with the current frame. With this definition, an Access Unit sometimes maps with a Decodable Frame Group (DFG) as defined in Annex E of [[!AV1]] and some other times to a Temporal Unit (TU) as defined in [[!AV1]], or both, as illustrated in the figure below. An illustration is provided in the figure below for a group of pictures with frames predicted as follows :
<img src="AccessUnitSplit_Example.png" alt="Practical example of an AV1 Access Unit split" width="100%" />
<figure>
<figcaption>Practical example of an AV1 Access Unit split</figcaption>
</figure>
## Use of PES packets
AV1 video encapsulated as defined in clause 4.2 is carried in PES packets as PES_packet_data_bytes, using the stream_id 0xBD (private_stream_id_1).
A PES shall encapsulate one, and only one, AV1 access unit as defined in clause 4.3. All the PES shall have data_alignment_indicator set to 1. Usage of *data_stream_alignment_descriptor* is not specified and the only allowed *alignment_type* is 1 (Access unit level).
The highest level that may occur in an AV1 video stream, as well as a profile and tier that the entire stream conforms to, shall be signalled using the AV1 video descriptor.
## Assignment of DTS and PTS
For AV1 video stream multiplexed into [[!MPEG-2 TS]], the *decoder_model_info* may not be present. If the *decoder_model_info* is present, then the STD model shall match with the decoder model defined in Annex E of [[!AV1]].
For synchronization and STD management, PTSs and, when appropriate, DTSs are encoded in the header of the PES packet that carries the AV1 video stream data setting the PTS_DTS_flags to '01' or '11'. For PTS and DTS encoding, the constraints and semantics apply as defined in the PES Header and associated constraints on timestamp intervals.
There are cases in AV1 bitstreams where information about a frame is sent multiple times. For example, first to be decoded, and subsequently to be displayed. In the case of a frame being decoded but not displayed, it is desired to assign a valid DTS but without need for a PTS. However, the MPEG2-TS specification prevents a DTS from being transmitted without a PTS. Hence, a PTS is always assigned for AV1 access units and its value is not relevant for frames being decoded but not displayed.
To achieve consistency between the STD model and the buffer model defined in Annex E of [[!AV1]], the following PTS and DTS assignment rules shall be applied :
|show_existing_frame|show_frame|showable_frame | PTS | DTS |
|:-----------------:|:--------:|:-------------:|:-------------------------:|:-------------------------:|
| 0 | 0 | 0 |ScheduledRemovalTiming[dfg]|ScheduledRemovalTiming[dfg]|
| 0 | 0 | 1 |ScheduledRemovalTiming[dfg]|ScheduledRemovalTiming[dfg]|
| 0 | 1 | n/a |PresentationTime[frame] |ScheduledRemovalTiming[dfg]|
| 1 | n/a | n/a |PresentationTime[frame] |ScheduledRemovalTiming[dfg]|
Note : The ScheduleRemovalTiming[] and PresentationTime[] are defined in the Annex E of [[!AV1]].
## Buffer considerations
### Buffer pool management
Carriage of an AV1 video stream over [[!MPEG-2 TS]] does not impact the size of the Buffer Pool.
For decoding of an AV1 video stream in the STD, the size of the Buffer Pool is as defined in [[!AV1]]. The Buffer Pool shall be managed as specified in Annex E of [[!AV1]].
A decoded AV1 access unit enters the Buffer Pool instantaneously upon decoding the AV1 access unit, hence at the Scheduled Removal Timing of the AV1 access unit. A decoded AV1 access unit is presented at the Presentation Time.
If the AV1 video stream provides insufficient information to determine the Scheduled Removal Timing and the Presentation Time of AV1 access units, then these time instants shall be determined in the STD model from PTS and DTS timestamps as follows:
1. The Scheduled Removal Timing of AV1 access unit n is the instant in time indicated by DTS(n) where DTS(n) is the DTS value of AV1 access unit n.
2. The Presentation Time of AV1 access unit n is the instant in time indicated by PTS(n) where PTS(n) is the PTS value of AV1 access unit n.
### T-STD Extensions for AV1
When there is an AV1 video stream in an [[!MPEG-2 TS]] program, the T-STD model as described in the section "Transport stream system target decoder" is extended as specified below.
<img src="t-std.svg" alt="T-STD Extensions for AV1" width="100%" />
<figure>
<figcaption>T-STD Extensions for AV1</figcaption>
</figure>
#### TB<sub>n</sub>, MB<sub>n</sub>, EB<sub>n</sub> buffer management
The following additional notations are used to describe the T-STD extensions and are illustrated in the figure above.
| Notation | Definition |
|:--|:--|
| t(i) | indicates the time in seconds at which the i-th byte of the transport stream enters the system target decoder |
| TB<sub>n</sub> | is the transport buffer for elementary stream n |
| TBS | is the size of the transport buffer TBn, measured in bytes |
| MB<sub>n</sub> | is the multiplexing buffer for elementary stream n |
| MBS<sub>n</sub> | is the size of the multiplexing buffer MBn, measured in bytes |
| EB<sub>n</sub> | is the elementary stream buffer for the AV1 video stream |
| EBS<sub>n</sub> | is the size of the multiplexing buffer MBn, measured in bytes |
| j | is an index to the AV1 access unit of the AV1 video stream |
| A<sub>n</sub>(j) | is the j-th access unit of the AV1 video bitstream |
| td<sub>n</sub> (j) | is the decoding time of An(j), measured in seconds, in the system target decoder |
| Rx<sub>n</sub> | is the transfer rate from the transport buffer TBn to the multiplex buffer MBn as specified below. |
| Rbx<sub>n</sub> | is the transfer rate from the multiplex buffer MBn to the elementary stream buffer EBn as specified below |
The following apply:
* There is exactly one transport buffer TB<sub>n</sub> for the received AV1 video stream where the size TBS is fixed to 512 bytes.
* There is exactly one multiplexing buffer MB<sub>n</sub> for the AV1 video stream, where the size MBS<sub>n</sub> of the multiplexing buffer MB is constrained as follows:
MBS<sub>n</sub> = BS<sub>mux</sub> + BS<sub>oh</sub> + 0.1 x BufferSize
where BS<sub>oh</sub>, packet overhead buffering, is defined as:
BS<sub>oh</sub> = (1/750) seconds × max{ 1100 × BitRate, 2 000 000 bit/s}
and BS<sub>mux</sub>, additional mutliplex buffering, is defined as:
BS<sub>mux</sub> = 0.004 seconds ×max{ 1100 × BitRate, 2 000 000 bit/s}
BufferSize and BitRate are defined in Annex E of the [[!AV1]]
* There is exactly one elementary stream buffer EB<sub>n</sub> for all the elementary streams in the set of received elementary streams associated by hierarchy descriptors, with a total size EBS<sub>n</sub>:
EBS<sub>n</sub> = BufferSize
* Transfer from TB<sub>n</sub> to MB<sub>n</sub> is applied as follows:
When there is no data in TB<sub>n</sub> then Rx<sub>n</sub> is equal to zero. Otherwise:
Rx<sub>n</sub> = 1.1 x BitRate
* The leak method shall be used to transfer data from MB<sub>n</sub> to EB<sub>n</sub> as follows:
Rbx<sub>n</sub> = 1.1 × BitRate
* The removal of start-code and emulation prevention as defined in section 4.2 is instantaneously performed between MB<sub>n</sub> and EB<sub>n</sub>.
If there is PES packet payload data in MB<sub>n</sub>, and buffer EB<sub>n</sub> is not full, the PES packet payload is transferred from MB<sub>n</sub> to EB<sub>n</sub> at a rate equal to Rbx<sub>n</sub>. If EB<sub>n</sub> is full, data are not removed from MB<sub>n</sub>. When a byte of data is transferred from MB<sub>n</sub> to EB<sub>n</sub>, all PES packet header bytes that are in MB<sub>n</sub> and precede that byte are instantaneously removed and discarded. When there is no PES packet payload data present in MB<sub>n</sub>, no data is removed from MB<sub>n</sub>. All data that enters MB<sub>n</sub> leaves it. All PES packet payload data bytes enter EB<sub>n</sub> instantaneously upon leaving MB<sub>n</sub>.
#### STD delay
The STD delay of any AV1 video through the system target decoders buffers TB<sub>n</sub>, MB<sub>n</sub>, and EB<sub>n</sub> shall be constrained by td<sub>n</sub>(j) – t(i) ≤ 10 seconds for all j, and all bytes i in access unit A<sub>n</sub>(j).
#### Buffer management conditions
Transport streams shall be constructed so that the following conditions for buffer management are satisfied:
* Each TB<sub>n</sub> shall not overflow and shall be empty at least once every second.
* Each MB<sub>n</sub>, EB<sub>n</sub> and Buffer Pool shall not overflow.
* EB<sub>n</sub> shall not underflow, except when the Operating parameters info syntax has low_delay_mode_flag set to '1'. Underflow of EB<sub>n</sub> occurs for AV1 access unit A<sub>n</sub>(j) when one or more bytes of A<sub>n</sub>(j) are not present in EB<sub>n</sub> at the decoding time td<sub>n</sub>(j).
# Acknowledgements and previous authors
A previous draft of this specification has been produced by VideoLAN, with inputs from different authors (Jean Baptiste Kempf, Kieran Kunhya, Adrien Maglo, Christophe Massiot, Mathieu Monnier and Mickael Raulet) from the following companies: ATEME, OpenHeadend, Open Broadcast Systems, Videolabs under the direction of VideoLAN.
</body>
</html>