draft-ietf-payload-rtp-h265-04.txt   draft-ietf-payload-rtp-h265-05.txt 
Network Working Group Y.-K. Wang Network Working Group Y.-K. Wang
Internet Draft Qualcomm Internet Draft Qualcomm
Intended status: Standards track Y. Sanchez Intended status: Standards track Y. Sanchez
Expires: November 2014 T. Schierl Expires: February 2015 T. Schierl
Fraunhofer HHI Fraunhofer HHI
S. Wenger S. Wenger
Vidyo Vidyo
M. M. Hannuksela M. M. Hannuksela
Nokia Nokia
May 28, 2014 August 5, 2014
RTP Payload Format for High Efficiency Video Coding RTP Payload Format for High Efficiency Video Coding
draft-ietf-payload-rtp-h265-04.txt draft-ietf-payload-rtp-h265-05.txt
Abstract Abstract
This memo describes an RTP payload format for the video coding This memo describes an RTP payload format for the video coding
standard ITU-T Recommendation H.265 and ISO/IEC International standard ITU-T Recommendation H.265 and ISO/IEC International
Standard 23008-2, both also known as High Efficiency Video Coding Standard 23008-2, both also known as High Efficiency Video Coding
(HEVC) [HEVC] and developed by the Joint Collaborative Team on Video (HEVC) [HEVC] and developed by the Joint Collaborative Team on Video
Coding (JCT-VC). The RTP payload format allows for packetization of Coding (JCT-VC). The RTP payload format allows for packetization of
one or more Network Abstraction Layer (NAL) units in each RTP packet one or more Network Abstraction Layer (NAL) units in each RTP packet
payload, as well as fragmentation of a NAL unit into multiple RTP payload, as well as fragmentation of a NAL unit into multiple RTP
skipping to change at page 2, line 18 skipping to change at page 2, line 18
months and may be updated, replaced, or obsoleted by other documents months and may be updated, replaced, or obsoleted by other documents
at any time. It is inappropriate to use Internet-Drafts as at any time. It is inappropriate to use Internet-Drafts as
reference material or to cite them other than as "work in progress." reference material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on November 28, 2014. This Internet-Draft will expire on February 5, 2015.
Copyright and License Notice Copyright and License Notice
Copyright (c) 2014 IETF Trust and the persons identified as the Copyright (c) 2014 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 10 skipping to change at page 3, line 10
respect to this document. Code Components extracted from this respect to this document. Code Components extracted from this
document must include Simplified BSD License text as described in document must include Simplified BSD License text as described in
Section 4.e of the Trust Legal Provisions and are provided without Section 4.e of the Trust Legal Provisions and are provided without
warranty as described in the Simplified BSD License. warranty as described in the Simplified BSD License.
Table of Contents Table of Contents
Abstract..........................................................1 Abstract..........................................................1
Status of this Memo...............................................1 Status of this Memo...............................................1
Table of Contents.................................................3 Table of Contents.................................................3
1. Introduction...................................................5 1 Introduction....................................................5
1.1. Overview of the HEVC Codec................................5 1.1 Overview of the HEVC Codec.................................5
1.1.1 Coding-Tool Features..................................5 1.1.1 Coding-Tool Features..................................5
1.1.2 Systems and Transport Interfaces......................7 1.1.2 Systems and Transport Interfaces......................7
1.1.3 Parallel Processing Support..........................14 1.1.3 Parallel Processing Support..........................14
1.1.4 NAL Unit Header......................................16 1.1.4 NAL Unit Header......................................16
1.2. Overview of the Payload Format...........................17 1.2 Overview of the Payload Format............................17
2. Conventions...................................................18 2 Conventions....................................................18
3. Definitions and Abbreviations.................................18 3 Definitions and Abbreviations..................................18
3.1 Definitions...............................................18 3.1 Definitions...............................................18
3.1.1 Definitions from the HEVC Specification..............18 3.1.1 Definitions from the HEVC Specification..............18
3.1.2 Definitions Specific to This Memo....................20 3.1.2 Definitions Specific to This Memo....................20
3.2 Abbreviations.............................................22 3.2 Abbreviations.............................................22
4. RTP Payload Format............................................23 4 RTP Payload Format.............................................23
4.1 RTP Header Usage..........................................23 4.1 RTP Header Usage..........................................23
4.2 Payload Header Usage......................................26 4.2 Payload Header Usage......................................26
4.3 Payload Structures........................................26 4.3 Payload Structures........................................26
4.4 Transmission Modes........................................27 4.4 Transmission Modes........................................27
4.5 Decoding Order Number.....................................28 4.5 Decoding Order Number.....................................28
4.6 Single NAL Unit Packets...................................30 4.6 Single NAL Unit Packets...................................30
4.7 Aggregation Packets (APs).................................31 4.7 Aggregation Packets (APs).................................31
4.8 Fragmentation Units (FUs).................................35 4.8 Fragmentation Units (FUs).................................35
4.9 PACI packets..............................................38 4.9 PACI packets..............................................38
4.9.1 Reasons for the PACI rules (informative).............41 4.9.1 Reasons for the PACI rules (informative).............41
4.9.2 PACI extensions (Informative)........................41 4.9.2 PACI extensions (Informative)........................41
4.10 Temporal Scalability Control Information.................43 4.10 Temporal Scalability Control Information.................43
5. Packetization Rules...........................................45 5 Packetization Rules............................................45
6. De-packetization Process......................................45 6 De-packetization Process.......................................45
7. Payload Format Parameters.....................................48 7 Payload Format Parameters......................................48
7.1 Media Type Registration...................................48 7.1 Media Type Registration...................................48
7.2 SDP Parameters............................................73 7.2 SDP Parameters............................................73
7.2.1 Mapping of Payload Type Parameters to SDP............73 7.2.1 Mapping of Payload Type Parameters to SDP............73
7.2.2 Usage with SDP Offer/Answer Model....................74 7.2.2 Usage with SDP Offer/Answer Model....................74
7.2.3 Usage in Declarative Session Descriptions............83 7.2.3 Usage in Declarative Session Descriptions............83
7.2.4 Parameter Sets Considerations........................84 7.2.4 Parameter Sets Considerations........................84
7.2.5 Dependency Signaling in Multi-Stream Mode............84 7.2.5 Dependency Signaling in Multi-Stream Mode............85
8. Use with Feedback Messages....................................85 8 Use with Feedback Messages.....................................85
8.1 Picture Loss Indication (PLI).............................86 8.1 Picture Loss Indication (PLI).............................86
8.2 Slice Loss Indication.....................................86 8.2 Slice Loss Indication.....................................86
8.3 Use of HEVC with the RPSI Feedback Message................87 8.3 Use of HEVC with the RPSI Feedback Message................87
8.4 Full Intra Request (FIR)..................................88 8.4 Full Intra Request (FIR)..................................88
9. Security Considerations.......................................88 9 Security Considerations........................................88
10. Congestion Control...........................................90 10 Congestion Control............................................90
11. IANA Consideration...........................................91 11 IANA Consideration............................................91
12. Acknowledgements.............................................91 12 Acknowledgements..............................................91
13. References...................................................91 13 References....................................................91
13.1 Normative References.....................................91 13.1 Normative References.....................................91
13.2 Informative References...................................93 13.2 Informative References...................................93
14. Authors' Addresses...........................................95 14 Authors' Addresses............................................95
1. Introduction 1 Introduction
1.1. Overview of the HEVC Codec 1.1 Overview of the HEVC Codec
High Efficiency Video Coding [HEVC], formally known as ITU-T High Efficiency Video Coding [HEVC], formally known as ITU-T
Recommendation H.265 and ISO/IEC International Standard 23008-2 was Recommendation H.265 and ISO/IEC International Standard 23008-2 was
ratified by ITU-T in April 2013 and reportedly provides significant ratified by ITU-T in April 2013 and reportedly provides significant
coding efficiency gains over H.264 [H.264]. coding efficiency gains over H.264 [H.264].
As both H.264 [H.264] and its RTP payload format [RFC6184] are As both H.264 [H.264] and its RTP payload format [RFC6184] are
widely deployed and generally known in the relevant implementer widely deployed and generally known in the relevant implementer
communities, frequently only the differences between those two communities, frequently only the differences between those two
specifications are highlighted in non-normative, explanatory parts specifications are highlighted in non-normative, explanatory parts
skipping to change at page 12, line 22 skipping to change at page 12, line 22
inter prediction of the current picture and that may be used in inter prediction of the current picture and that may be used in
inter prediction of one or more of the pictures following the inter prediction of one or more of the pictures following the
current picture in decoding order. RefPicSetStFoll and current picture in decoding order. RefPicSetStFoll and
RefPicSetLtFoll consist of all reference pictures that are not used RefPicSetLtFoll consist of all reference pictures that are not used
in inter prediction of the current picture but may be used in inter in inter prediction of the current picture but may be used in inter
prediction of one or more of the pictures following the current prediction of one or more of the pictures following the current
picture in decoding order. RPS provides an "intra-coded" signaling picture in decoding order. RPS provides an "intra-coded" signaling
of the DPB status, instead of an "inter-coded" signaling, mainly for of the DPB status, instead of an "inter-coded" signaling, mainly for
improved error resilience. The RPLC process in HEVC is based on the improved error resilience. The RPLC process in HEVC is based on the
RPS, by signaling an index to an RPS subset for each reference RPS, by signaling an index to an RPS subset for each reference
index. The RPLC process has been simplified compared to that in index; this process is simpler than the RPLC process in H.264.
H.264, by removal of the reference picture list modification (also
referred to as reference picture list reordering) process.
Ultra low delay support Ultra low delay support
HEVC specifies a sub-picture-level HRD operation, for support of the HEVC specifies a sub-picture-level HRD operation, for support of the
so-called ultra-low delay. The mechanism specifies a standard- so-called ultra-low delay. The mechanism specifies a standard-
compliant way to enable delay reduction below one picture interval. compliant way to enable delay reduction below one picture interval.
Sub-picture-level coded picture buffer (CPB) and DPB parameters may Sub-picture-level coded picture buffer (CPB) and DPB parameters may
be signaled, and utilization of these information for the derivation be signaled, and utilization of these information for the derivation
of CPB timing (wherein the CPB removal time corresponds to decoding of CPB timing (wherein the CPB removal time corresponds to decoding
time) and DPB output timing (display time) is specified. Decoders time) and DPB output timing (display time) is specified. Decoders
skipping to change at page 16, line 37 skipping to change at page 16, line 33
+-------------+-----------------+ +-------------+-----------------+
Figure 1 The structure of HEVC NAL unit header Figure 1 The structure of HEVC NAL unit header
The semantics of the fields in the NAL unit header are as specified The semantics of the fields in the NAL unit header are as specified
in [HEVC] and described briefly below for convenience. In addition in [HEVC] and described briefly below for convenience. In addition
to the name and size of each field, the corresponding syntax element to the name and size of each field, the corresponding syntax element
name in [HEVC] is also provided. name in [HEVC] is also provided.
F: 1 bit F: 1 bit
forbidden_zero_bit. MUST be zero. HEVC declares a value of 1 as forbidden_zero_bit. Required to be zero in [HEVC]. HEVC
a syntax violation. Note that the inclusion of this bit in the declares a value of 1 as a syntax violation. Note that the
NAL unit header is to enable transport of HEVC video over MPEG-2 inclusion of this bit in the NAL unit header is to enable
transport systems (avoidance of start code emulations) [MPEG2S]. transport of HEVC video over MPEG-2 transport systems (avoidance
of start code emulations) [MPEG2S].
Type: 6 bits Type: 6 bits
nal_unit_type. This field specifies the NAL unit type as defined nal_unit_type. This field specifies the NAL unit type as defined
in Table 7-1 of [HEVC]. If the most significant bit of this in Table 7-1 of [HEVC]. If the most significant bit of this
field of a NAL unit is equal to 0 (i.e. the value of this field field of a NAL unit is equal to 0 (i.e. the value of this field
is less than 32), the NAL unit is a VCL NAL unit. Otherwise, the is less than 32), the NAL unit is a VCL NAL unit. Otherwise, the
NAL unit is a non-VCL NAL unit. For a reference of all currently NAL unit is a non-VCL NAL unit. For a reference of all currently
defined NAL unit types and their semantics, please refer to defined NAL unit types and their semantics, please refer to
Section 7.4.1 in [HEVC]. Section 7.4.1 in [HEVC].
LayerId: 6 bits LayerId: 6 bits
nuh_layer_id. MUST be equal to zero. It is anticipated that in nuh_layer_id. Required to be equal to zero in [HEVC]. It is
future scalable or 3D video coding extensions of this anticipated that in future scalable or 3D video coding extensions
specification, this syntax element will be used to identify of this specification, this syntax element will be used to
additional layers that may be present in the coded video identify additional layers that may be present in the coded video
sequence, wherein a layer may be, e.g. a spatial scalable layer, sequence, wherein a layer may be, e.g. a spatial scalable layer,
a quality scalable layer, a texture view, or a depth view. a quality scalable layer, a texture view, or a depth view.
TID: 3 bits TID: 3 bits
nuh_temporal_id_plus1. This field specifies the temporal nuh_temporal_id_plus1. This field specifies the temporal
identifier of the NAL unit plus 1. The value of TemporalId is identifier of the NAL unit plus 1. The value of TemporalId is
equal to TID minus 1. A TID value of 0 is illegal to ensure that equal to TID minus 1. A TID value of 0 is illegal to ensure that
there is at least one bit in the NAL unit header equal to 1, so there is at least one bit in the NAL unit header equal to 1, so
to enable independent considerations of start code emulations in to enable independent considerations of start code emulations in
the NAL unit header and in the NAL unit payload data. the NAL unit header and in the NAL unit payload data.
1.2. Overview of the Payload Format 1.2 Overview of the Payload Format
This payload format defines the following processes required for This payload format defines the following processes required for
transport of HEVC coded data over RTP [RFC3550]: transport of HEVC coded data over RTP [RFC3550]:
o Usage of RTP header with this payload format o Usage of RTP header with this payload format
o Packetization of HEVC coded NAL units into RTP packets using three o Packetization of HEVC coded NAL units into RTP packets using three
types of payload structures, namely single NAL unit packet, types of payload structures, namely single NAL unit packet,
aggregation packet, and fragment unit aggregation packet, and fragment unit
skipping to change at page 18, line 15 skipping to change at page 18, line 12
the transmission order of NAL units is different from their the transmission order of NAL units is different from their
decoding order) decoding order)
o Media type parameters to be used with the Session Description o Media type parameters to be used with the Session Description
Protocol (SDP) [RFC4566] Protocol (SDP) [RFC4566]
o A payload header extension mechanism and data structures for o A payload header extension mechanism and data structures for
enhanced support of temporal scalability based on that extension enhanced support of temporal scalability based on that extension
mechanism. mechanism.
2. Conventions 2 Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in BCP 14, RFC 2119 document are to be interpreted as described in BCP 14, RFC 2119
[RFC2119]. [RFC2119].
In this document, these key words will appear with that In this document, these key words will appear with that
interpretation only when in ALL CAPS. Lower case uses of these interpretation only when in ALL CAPS. Lower case uses of these
words are not to be interpreted as carrying the RFC 2119 words are not to be interpreted as carrying the RFC 2119
significance. significance.
This specification uses the notion of setting and clearing a bit This specification uses the notion of setting and clearing a bit
when bit fields are handled. Setting a bit is the same as assigning when bit fields are handled. Setting a bit is the same as assigning
that bit the value of 1 (On). Clearing a bit is the same as that bit the value of 1 (On). Clearing a bit is the same as
assigning that bit the value of 0 (Off). assigning that bit the value of 0 (Off).
3. Definitions and Abbreviations 3 Definitions and Abbreviations
3.1 Definitions 3.1 Definitions
This document uses the terms and definitions of [HEVC]. Section This document uses the terms and definitions of [HEVC]. Section
3.1.1 lists relevant definitions copied from [HEVC] for convenience. 3.1.1 lists relevant definitions copied from [HEVC] for convenience.
Section 3.1.2 provides definitions specific to this memo. Section 3.1.2 provides definitions specific to this memo.
3.1.1 Definitions from the HEVC Specification 3.1.1 Definitions from the HEVC Specification
access unit: A set of NAL units that are associated with each other access unit: A set of NAL units that are associated with each other
skipping to change at page 23, line 30 skipping to change at page 23, line 24
STSA Step-wise Temporal Sub-layer Access STSA Step-wise Temporal Sub-layer Access
TSA Temporal Sub-layer Access TSA Temporal Sub-layer Access
TCSI Temporal Scalability Control Information TCSI Temporal Scalability Control Information
VCL Video Coding Layer VCL Video Coding Layer
VPS Video Parameter Set VPS Video Parameter Set
4. RTP Payload Format 4 RTP Payload Format
4.1 RTP Header Usage 4.1 RTP Header Usage
The format of the RTP header is specified in [RFC3550] and reprinted The format of the RTP header is specified in [RFC3550] and reprinted
in Figure 2 for convenience. This payload format uses the fields of in Figure 2 for convenience. This payload format uses the fields of
the header in a manner consistent with that specification. the header in a manner consistent with that specification.
The RTP payload (and the settings for some RTP header bits) for The RTP payload (and the settings for some RTP header bits) for
aggregation packets and fragmentation units are specified in aggregation packets and fragmentation units are specified in
Sections 4.7 and 4.8, respectively. Sections 4.7 and 4.8, respectively.
skipping to change at page 35, line 32 skipping to change at page 35, line 32
| | | |
| . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | . . . +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding | | :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 8 An example of an AP containing two aggregation units with Figure 8 An example of an AP containing two aggregation units with
the DONL and DOND fields the DONL and DOND fields
4.8 Fragmentation Units (FUs) 4.8 Fragmentation Units (FUs)
Fragmentation units (FUs) are introduced to enable fragmenting a Fragmentation units (FUs) are introduced to enable fragmenting a single
single NAL unit into multiple RTP packets, possibly without NAL unit into multiple RTP packets, possibly without cooperation or
cooperation or knowledge of the HEVC encoder. A fragment of a NAL knowledge of the HEVC encoder. A fragment of a NAL unit consists of
unit consists of an integer number of consecutive octets of that NAL an integer number of consecutive octets of that NAL unit. Fragments
unit. Fragments of the same NAL unit MUST be sent in consecutive of the same NAL unit MUST be sent in consecutive order with ascending
order with ascending RTP sequence numbers (with no other RTP packets RTP sequence numbers (with no other RTP packets within the same RTP
within the same RTP stream being sent between the first and last stream being sent between the first and last fragment).
fragment).
When a NAL unit is fragmented and conveyed within FUs, it is When a NAL unit is fragmented and conveyed within FUs, it is
referred to as a fragmented NAL unit. APs MUST NOT be fragmented. referred to as a fragmented NAL unit. APs MUST NOT be fragmented.
FUs MUST NOT be nested; i.e. an FU MUST NOT contain a subset of FUs MUST NOT be nested; i.e. an FU MUST NOT contain a subset of
another FU. another FU.
The RTP timestamp of an RTP packet carrying an FU is set to the The RTP timestamp of an RTP packet carrying an FU is set to the
NALU-time of the fragmented NAL unit. NALU-time of the fragmented NAL unit.
An FU consists of a payload header (denoted as PayloadHdr), an FU An FU consists of a payload header (denoted as PayloadHdr), an FU
skipping to change at page 40, line 6 skipping to change at page 40, line 6
A: 1 bit A: 1 bit
Copy of the F bit of the PACI payload NAL unit or NAL-unit-like Copy of the F bit of the PACI payload NAL unit or NAL-unit-like
structure. structure.
cType: 6 bits cType: 6 bits
Copy of the Type field of the PACI payload NAL unit or NAL-unit- Copy of the Type field of the PACI payload NAL unit or NAL-unit-
like structure. like structure.
PHSsize: 5 bits PHSsize: 5 bits
Indicates the total length of the fields F[0..2], Y, and PHES. Indicates the length of the PHES field. The value is limited to
The value is limited to be less than or equal to 32 octets, to be less than or equal to 32 octets, to simplify encoder design
simplify encoder design for MTU size matching. for MTU size matching.
F0 F0
This field equal to 1 specifies the presence of a temporal This field equal to 1 specifies the presence of a temporal
scalability support extension in the PHES. scalability support extension in the PHES.
F1, F2 F1, F2
MUST be 0, available for future extensions, see section 4.9.2. MUST be 0, available for future extensions, see section 4.9.2.
Y: 1 bit Y: 1 bit
MUST be 0, available for future extensions, see section 4.9.2. MUST be 0, available for future extensions, see section 4.9.2.
PHES: variable number of octets PHES: variable number of octets
A variable number of octets as indicated by the value of PHSsize. A variable number of octets as indicated by the value of PHSsize.
PACI Payload PACI Payload
The NAL unit or NAL-unit-like structure (such as: FU or AP) to be The single NAL unit packet or NAL-unit-like structure (such as:
carried, not including the first two octets. FU or AP) to be carried, not including the first two octets.
Informative note: The first two octets of the NAL unit or NAL- Informative note: The first two octets of the NAL unit or NAL-
unit-like structure carried in the PACI payload are not unit-like structure carried in the PACI payload are not
included in the PACI payload. Rather, the respective values included in the PACI payload. Rather, the respective values
are copied in locations of the PayloadHdr of the RTP packet. are copied in locations of the PayloadHdr of the RTP packet.
This design offers two advantages: first, the overall This design offers two advantages: first, the overall
structure of the payload header is preserved, i.e. there is no structure of the payload header is preserved, i.e. there is no
special case of payload header structure that needs to be special case of payload header structure that needs to be
implemented for PACI. Second, no additional overhead is implemented for PACI. Second, no additional overhead is
introduced. introduced.
skipping to change at page 41, line 45 skipping to change at page 41, line 45
the MTU size) NAL units. In order to efficiently packetize such the MTU size) NAL units. In order to efficiently packetize such
small NAL units, AP were introduced. The benefits of APs are small NAL units, AP were introduced. The benefits of APs are
independent from the need for a payload header extension. independent from the need for a payload header extension.
Therefore, a sender may place an AP into a PACI, and a receiver must Therefore, a sender may place an AP into a PACI, and a receiver must
be able to handle such a PACI. be able to handle such a PACI.
4.9.2 PACI extensions (Informative) 4.9.2 PACI extensions (Informative)
This subsection includes recommendations for future specification This subsection includes recommendations for future specification
designers on how to extent the PACI syntax to accommodate future designers on how to extent the PACI syntax to accommodate future
extensions. Obviously, designers are free to specify whatever extensions. Obviously, designers are free to specify whatever appears
appears to be appropriate to them at the time of their design. to be appropriate to them at the time of their design. However, a lot
However, a lot of thought has been invested into the extension of thought has been invested into the extension mechanism described
mechanism described below, and we suggest that deviations from it below, and we suggest that deviations from it warrant a good
warrant a good explanation. explanation.
This memo defines only a single payload header extension (Temporal This memo defines only a single payload header extension (Temporal
Scalability Control Information, described below in section 4.10), Scalability Control Information, described below in section 4.10),
and, therefore, only the F0 bit carries semantics. F1 and F2 are and, therefore, only the F0 bit carries semantics. F1 and F2 are
already named (and not just marked as reserved, as a typical video already named (and not just marked as reserved, as a typical video
spec designer would do). They are intended to signal two additional spec designer would do). They are intended to signal two additional
extensions. The Y bit allows to, recursively, add further F and Y extensions. The Y bit allows to, recursively, add further F and Y
bits to extend the mechanism beyond 3 possible payload header bits to extend the mechanism beyond 3 possible payload header
extensions. It is suggested to define a new packet type (using a extensions. It is suggested to define a new packet type (using a
different value for Type) when assigning the F1, F2, or Y bits different value for Type) when assigning the F1, F2, or Y bits
skipping to change at page 42, line 35 skipping to change at page 42, line 35
the Fx bits in the first flag-extensions are numbered F3, F4, ..., the Fx bits in the first flag-extensions are numbered F3, F4, ...,
F9, the F bits in the second flag-extension are numbered F10, F11, F9, the F bits in the second flag-extension are numbered F10, F11,
..., F16, and so forth. As a result, at least 3 Fx bits are always ..., F16, and so forth. As a result, at least 3 Fx bits are always
in the PACI, but the number of Fx bits (and associated types of in the PACI, but the number of Fx bits (and associated types of
extensions), can be increased by setting the next Y bit and adding extensions), can be increased by setting the next Y bit and adding
an octet of flag-extensions, carrying 7 flags and another Y bit. an octet of flag-extensions, carrying 7 flags and another Y bit.
The size of this list of flags is subject to the limits specified in The size of this list of flags is subject to the limits specified in
section 4.9 (32 octets for all flag-extensions and the PHES section 4.9 (32 octets for all flag-extensions and the PHES
information combined). information combined).
Each of the F bits can indicate either the presence of information Each of the F bits can indicate either the presence of information in
in the Payload Header Extension Structure (PHES), described below, the Payload Header Extension Structure (PHES), described below, or a
or a given F bit can indicate a certain condition, without including given F bit can indicate a certain condition, without including
additional information in the PHES. additional information in the PHES.
When a spec developer devises a new syntax that takes advantage of When a spec developer devises a new syntax that takes advantage of the
the PACI extension mechanism, he/she must follow the constraints PACI extension mechanism, he/she must follow the constraints listed
listed below; otherwise the extension mechanism may break. below; otherwise the extension mechanism may break.
1) The fields added for a particular Fx bit MUST be fixed in 1) The fields added for a particular Fx bit MUST be fixed in length
length and not depend on what other Fx bits are set (no parsing and not depend on what other Fx bits are set (no parsing
dependency). dependency).
2) The Fx bits must be assigned in order. 2) The Fx bits must be assigned in order.
3) An implementation that supports the n-th Fn bit for any value 3) An implementation that supports the n-th Fn bit for any value of
of n must understand the syntax (though not necessarily the n must understand the syntax (though not necessarily the
semantics) of the fields Fk (with k < n), so to be able to semantics) of the fields Fk (with k < n), so to be able to either
either use those bits when present, or at least be able to skip use those bits when present, or at least be able to skip over
over them. them.
4.10 Temporal Scalability Control Information 4.10 Temporal Scalability Control Information
This section describes the single payload header extension defined This section describes the single payload header extension defined
in this specification, known as Temporal Scalability Control in this specification, known as Temporal Scalability Control
Information (TSCI). If, in the future, additional payload header Information (TSCI). If, in the future, additional payload header
extensions become necessary, they could be specified in this section extensions become necessary, they could be specified in this section
of an updated version of this document, or in their own documents. of an updated version of this document, or in their own documents.
When F0 is set to 1 in a PACI, this specifies that the PHES field When F0 is set to 1 in a PACI, this specifies that the PHES field
includes the TSCI fields TL0REFIDX, IrapPicID, S, and E as follows: includes the TSCI fields TL0PICIDX, IrapPicID, S, and E as follows:
0 1 2 3 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PayloadHdr (Type=50) |A| cType | PHSsize |F0..2|Y| | PayloadHdr (Type=50) |A| cType | PHSsize |F0..2|Y|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TL0REFIDX | IrapPicID |S|E|RES| | | TL0PICIDX | IrapPicID |S|E| RES | |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | |-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| .... | | .... |
| PACI payload: NAL unit | | PACI payload: NAL unit |
| | | |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| :...OPTIONAL RTP padding | | :...OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 12 The structure of a PACI with a PHES containing a TSCI Figure 12 The structure of a PACI with a PHES containing a TSCI
TL0PICIDX (8 bits) TL0PICIDX (8 bits)
When present, the TL0PICIDX field MUST be set to equal to When present, the TL0PICIDX field MUST be set to equal to
temporal_sub_layer_zero_idx as specified in Section D.3.32 of temporal_sub_layer_zero_idx as specified in Section D.3.22 of
[H.265] for the access unit containing the NAL unit in the PACI. [H.265] for the access unit containing the NAL unit in the PACI.
IrapPicID (8 bits) IrapPicID (8 bits)
When present, the IrapPicID field MUST be set to equal to When present, the IrapPicID field MUST be set to equal to
irap_pic_id as specified in Section D.3.22 of [H.265] for the irap_pic_id as specified in Section D.3.22 of [H.265] for the
access unit containing the NAL unit in the PACI. access unit containing the NAL unit in the PACI.
S (1 bit) S (1 bit)
The S bit MUST be set to 1 if any of the following conditions is The S bit MUST be set to 1 if any of the following conditions is
true and MUST be set to 0 otherwise: true and MUST be set to 0 otherwise:
skipping to change at page 44, line 41 skipping to change at page 44, line 41
. The NAL unit in the payload of the PACI is the last VCL NAL . The NAL unit in the payload of the PACI is the last VCL NAL
unit, in decoding order, of a picture. unit, in decoding order, of a picture.
. The NAL unit in the payload of the PACI is an AP and the NAL . The NAL unit in the payload of the PACI is an AP and the NAL
unit in the last contained aggregation unit is the last VCL NAL unit in the last contained aggregation unit is the last VCL NAL
unit, in decoding order, of a picture. unit, in decoding order, of a picture.
. The NAL unit in the payload of the PACI is an FU with its E bit . The NAL unit in the payload of the PACI is an FU with its E bit
equal to 1 and the FU payload containing a fragment of the last equal to 1 and the FU payload containing a fragment of the last
VCL NAL unit, in decoding order of a picture. VCL NAL unit, in decoding order of a picture.
RES (2 bits) RES (6 bits)
MUST be equal to 0. Reserved for future extensions. MUST be equal to 0. Reserved for future extensions.
The value of PHSsize MUST be set to 3. Receivers MUST allow other The value of PHSsize MUST be set to 3. Receivers MUST allow other
values of the fields F0, F1, F2, Y, and PHSsize, and MUST ignore any values of the fields F0, F1, F2, Y, and PHSsize, and MUST ignore any
additional fields, when present, than specified above in the PHES. additional fields, when present, than specified above in the PHES.
5. Packetization Rules 5 Packetization Rules
The following packetization rules apply: The following packetization rules apply:
o If tx-mode is equal to "MSM" or sprop-max-don-diff is greater o If tx-mode is equal to "MSM" or sprop-max-don-diff is greater than
than 0 for an RTP stream, the transmission order of NAL units 0 for an RTP stream, the transmission order of NAL units carried in
carried in the RTP stream MAY be different than the NAL unit the RTP stream MAY be different than the NAL unit decoding order.
decoding order. Otherwise (tx-mode is equal to "SSM" and sprop- Otherwise (tx-mode is equal to "SSM" and sprop-max-don-diff is equal
max-don-diff is equal to 0 for an RTP stream), the transmission to 0 for an RTP stream), the transmission order of NAL units carried
order of NAL units carried in the RTP stream MUST be the same as in the RTP stream MUST be the same as the NAL unit decoding order.
the NAL unit decoding order.
o A NAL unit of a small size SHOULD be encapsulated in an o A NAL unit of a small size SHOULD be encapsulated in an
aggregation packet together with one or more other NAL units in aggregation packet together with one or more other NAL units in
order to avoid the unnecessary packetization overhead for small order to avoid the unnecessary packetization overhead for small
NAL units. For example, non-VCL NAL units such as access unit NAL units. For example, non-VCL NAL units such as access unit
delimiters, parameter sets, or SEI NAL units are typically small delimiters, parameter sets, or SEI NAL units are typically small
and can often be aggregated with VCL NAL units without violating and can often be aggregated with VCL NAL units without violating
MTU size constraints. MTU size constraints.
o Each non-VCL NAL unit SHOULD, when possible from an MTU size o Each non-VCL NAL unit SHOULD, when possible from an MTU size
match viewpoint, be encapsulated in an aggregation packet match viewpoint, be encapsulated in an aggregation packet
together with its associated VCL NAL unit, as typically a non-VCL together with its associated VCL NAL unit, as typically a non-VCL
NAL unit would be meaningless without the associated VCL NAL unit NAL unit would be meaningless without the associated VCL NAL unit
being available. being available.
o For carrying exactly one NAL unit in an RTP packet, a single NAL o For carrying exactly one NAL unit in an RTP packet, a single NAL
unit packet MUST be used. unit packet MUST be used.
6. De-packetization Process 6 De-packetization Process
The general concept behind de-packetization is to get the NAL units The general concept behind de-packetization is to get the NAL units
out of the RTP packets in an RTP stream and all RTP streams the RTP out of the RTP packets in an RTP stream and all RTP streams the RTP
stream depends on, if any, and pass them to the decoder in the NAL stream depends on, if any, and pass them to the decoder in the NAL
unit decoding order. unit decoding order.
The de-packetization process is implementation dependent. The de-packetization process is implementation dependent.
Therefore, the following description should be seen as an example of Therefore, the following description should be seen as an example of
a suitable implementation. Other schemes may be used as well as a suitable implementation. Other schemes may be used as well as
long as the output for the same input is the same as the process long as the output for the same input is the same as the process
skipping to change at page 48, line 7 skipping to change at page 48, line 5
o The NAL unit in the de-packetization buffer with the smallest o The NAL unit in the de-packetization buffer with the smallest
value of AbsDon is removed from the de-packetization buffer and value of AbsDon is removed from the de-packetization buffer and
passed to the decoder. passed to the decoder.
When no more NAL units are flowing into the de-packetization buffer, When no more NAL units are flowing into the de-packetization buffer,
all NAL units remaining in the de-packetization buffer are removed all NAL units remaining in the de-packetization buffer are removed
from the buffer and passed to the decoder in the order of increasing from the buffer and passed to the decoder in the order of increasing
AbsDon values. AbsDon values.
7. Payload Format Parameters 7 Payload Format Parameters
This section specifies the parameters that MAY be used to select This section specifies the parameters that MAY be used to select
optional features of the payload format and certain features or optional features of the payload format and certain features or
properties of the bitstream or the RTP stream. The parameters are properties of the bitstream or the RTP stream. The parameters are
specified here as part of the media type registration for the HEVC specified here as part of the media type registration for the HEVC
codec. A mapping of the parameters into the Session Description codec. A mapping of the parameters into the Session Description
Protocol (SDP) [RFC4566] is also provided for applications that use Protocol (SDP) [RFC4566] is also provided for applications that use
SDP. Equivalent parameters could be defined elsewhere for use with SDP. Equivalent parameters could be defined elsewhere for use with
control protocols that do not use SDP. control protocols that do not use SDP.
skipping to change at page 56, line 12 skipping to change at page 56, line 12
This parameter indicates whether the transmission mode is SSM This parameter indicates whether the transmission mode is SSM
or MSM. or MSM.
The value of tx-mode MUST be equal to either "MSM" or "SSM". The value of tx-mode MUST be equal to either "MSM" or "SSM".
When not present, the value of tx-mode is inferred to be equal When not present, the value of tx-mode is inferred to be equal
to "SSM". to "SSM".
If the value is equal to "MSM", MSM MUST be in use. Otherwise If the value is equal to "MSM", MSM MUST be in use. Otherwise
(the value is equal to "SSM"), SSM MUST be in use. (the value is equal to "SSM"), SSM MUST be in use.
The value of tx-mode MUST be equal to "MSM" for all RTP The value of tx-mode MUST be equal to "MSM" for all RTP sessions
sessions in an MSM. in an MSM.
sprop-vps: sprop-vps:
This parameter MAY be used to convey any video parameter set This parameter MAY be used to convey any video parameter set
NAL unit of the bitstream for out-of-band transmission of NAL unit of the bitstream for out-of-band transmission of
video parameter sets. The parameter MAY also be used for video parameter sets. The parameter MAY also be used for
capability exchange and to indicate sub-stream characteristics capability exchange and to indicate sub-stream characteristics
(i.e. properties of sub-layer representations as defined in (i.e. properties of sub-layer representations as defined in
[HEVC]). The value of the parameter is a comma-separated [HEVC]). The value of the parameter is a comma-separated
(',') list of base64 [RFC4648] representations of the video (',') list of base64 [RFC4648] representations of the video
skipping to change at page 72, line 37 skipping to change at page 72, line 37
Additional information: None Additional information: None
File extensions: none File extensions: none
Macintosh file type code: none Macintosh file type code: none
Object identifier or OID: none Object identifier or OID: none
Person & email address to contact for further information: Person & email address to contact for further information:
Ye-Kui Wang (yekuiw@qti.qualcomm.com).
Intended usage: COMMON Intended usage: COMMON
Author: See Section 14 of RFC XXXX. Author: See Section 14 of RFC XXXX.
Change controller: Change controller:
IETF Audio/Video Transport Payloads working group delegated IETF Audio/Video Transport Payloads working group delegated
from the IESG. from the IESG.
7.2 SDP Parameters 7.2 SDP Parameters
skipping to change at page 76, line 12 skipping to change at page 76, line 12
of sub-layers are present in the first video parameter set of sub-layers are present in the first video parameter set
contained in sprop-vps. If the sprop-vps is provided in an contained in sprop-vps. If the sprop-vps is provided in an
offer, an answerer MAY select a particular operation point offer, an answerer MAY select a particular operation point
indicated in the first video parameter set contained in sprop- indicated in the first video parameter set contained in sprop-
vps. When the answer includes recv-sub-layer-id that is less vps. When the answer includes recv-sub-layer-id that is less
than sprop-sub-layer-id in the offer, all video parameter sets than sprop-sub-layer-id in the offer, all video parameter sets
contained in the sprop-vps parameter in the SDP answer and all contained in the sprop-vps parameter in the SDP answer and all
video parameter sets sent in-band for either the offerer-to- video parameter sets sent in-band for either the offerer-to-
answerer direction or the answerer-to-offerer direction MUST be answerer direction or the answerer-to-offerer direction MUST be
consistent with the first video parameter set in the sprop-vps consistent with the first video parameter set in the sprop-vps
parameter of the offer (see the semantics of sprop-vps on one parameter of the offer (see the semantics of sprop-vps in section
video parameter set being consistent with another video parameter 7.1 of this document on one video parameter set being consistent
set), and the bitstream sent in either direction MUST conform to with another video parameter set), and the bitstream sent in
the profile, tier, level, and constraints of the chosen sub-layer either direction MUST conform to the profile, tier, level, and
representation as indicated by the first profile_tier_level( ) constraints of the chosen sub-layer representation as indicated
syntax structure in the first video parameter set in the sprop- by the first profile_tier_level( ) syntax structure in the first
vps parameter of the offer. video parameter set in the sprop-vps parameter of the offer.
Informative note: When an offerer receives an answer that Informative note: When an offerer receives an answer that
does not include recv-sub-layer-id, it has to compare payload does not include recv-sub-layer-id, it has to compare payload
types not declared in the offer based on the media type (i.e. types not declared in the offer based on the media type (i.e.
video/H265) and the above media configuration parameters with video/H265) and the above media configuration parameters with
any payload types it has already declared. This will enable any payload types it has already declared. This will enable
it to determine whether the configuration in question is new it to determine whether the configuration in question is new
or if it is equivalent to configuration already offered, or if it is equivalent to configuration already offered,
since a different payload type number may be used in the since a different payload type number may be used in the
answer. The ability to perform operation point selection answer. The ability to perform operation point selection
skipping to change at page 78, line 7 skipping to change at page 78, line 7
buffer, sprop-depack-buf-bytes, as well as sprop-max-don-diff and buffer, sprop-depack-buf-bytes, as well as sprop-max-don-diff and
sprop-depack-buf-nalus, in the offer for an interleaved HEVC sprop-depack-buf-nalus, in the offer for an interleaved HEVC
bitstream or for the MSM transmission mode. To enable the bitstream or for the MSM transmission mode. To enable the
offerer and answerer to inform each other about their offerer and answerer to inform each other about their
capabilities for de-packetization buffering in receiving RTP capabilities for de-packetization buffering in receiving RTP
streams, both parties are RECOMMENDED to include depack-buf-cap. streams, both parties are RECOMMENDED to include depack-buf-cap.
For interleaved RTP streams or in MSM, it is also RECOMMENDED to For interleaved RTP streams or in MSM, it is also RECOMMENDED to
consider offering multiple payload types with different buffering consider offering multiple payload types with different buffering
requirements when the capabilities of the receiver are unknown. requirements when the capabilities of the receiver are unknown.
o The capability parameter include-dph MAY be used to declare the
capability to utilize decoded picture hash SEI messages and which
types of hashes in any HEVC RTP streams received by the offerer
or answerer.
o The sprop-vps, sprop-sps, or sprop-pps, when present (included in o The sprop-vps, sprop-sps, or sprop-pps, when present (included in
the "a=fmtp" line of SDP or conveyed using the "fmtp" source the "a=fmtp" line of SDP or conveyed using the "fmtp" source
attribute as specified in section 6.3 of [RFC5576]), are used for attribute as specified in section 6.3 of [RFC5576]), are used for
out-of-band transport of the parameter sets (VPS, SPS, or PPS out-of-band transport of the parameter sets (VPS, SPS, or PPS
respectively). respectively).
o The answerer MAY use either out-of-band or in-band transport of o The answerer MAY use either out-of-band or in-band transport of
parameter sets for the bitstream it is sending, regardless of parameter sets for the bitstream it is sending, regardless of
whether out-of-band parameter sets transport has been used in the whether out-of-band parameter sets transport has been used in the
offerer-to-answerer direction. Parameter sets included in an offerer-to-answerer direction. Parameter sets included in an
answer are independent of those parameter sets included in the answer are independent of those parameter sets included in the
offer, as they are used for decoding two different bitstreams, offer, as they are used for decoding two different bitstreams,
one from the answerer to the offerer and the other in the one from the answerer to the offerer and the other in the
opposite direction. opposite direction. In case some RTP stream(s) are sent before
SDP offer/answer settles down, in-band parameter sets MUST be
o The capability parameter include-dph MAY be used to declare the used for those RTP stream parts sent before the SDP offer/answer.
capability to utilize decoded picture hash SEI messages and which
types of hashes in any HEVC RTP streams received by the offerer
or answerer.
o The following rules apply to transport of parameter set in the o The following rules apply to transport of parameter set in the
offerer-to-answerer direction. offerer-to-answerer direction.
o An offer MAY include sprop-vps, sprop-sps, and/or sprop-pps. o An offer MAY include sprop-vps, sprop-sps, and/or sprop-pps.
If none of these parameters is present in the offer, then If none of these parameters is present in the offer, then
only in-band transport of parameter sets is used. only in-band transport of parameter sets is used.
o If the level to use in the offerer-to-answerer direction is o If the level to use in the offerer-to-answerer direction is
equal to the default level in the offer, the answerer MUST be equal to the default level in the offer, the answerer MUST be
skipping to change at page 85, line 9 skipping to change at page 85, line 15
7.2.5 Dependency Signaling in Multi-Stream Mode 7.2.5 Dependency Signaling in Multi-Stream Mode
If MSM is used, the rules on signaling media decoding dependency in If MSM is used, the rules on signaling media decoding dependency in
SDP as defined in [RFC5583] apply. The rules on "hierarchical or SDP as defined in [RFC5583] apply. The rules on "hierarchical or
layered encoding" with multicast in Section 5.7 of [RFC4566] do not layered encoding" with multicast in Section 5.7 of [RFC4566] do not
apply, i.e. the notation for Connection Data "c=" SHALL NOT be used apply, i.e. the notation for Connection Data "c=" SHALL NOT be used
with more than one address. The order of session dependency is with more than one address. The order of session dependency is
given from the RTP stream containing the lowest temporal sub-layer given from the RTP stream containing the lowest temporal sub-layer
to the RTP stream containing the highest temporal sub-layer. to the RTP stream containing the highest temporal sub-layer.
8. Use with Feedback Messages 8 Use with Feedback Messages
As specified in section 6.1 of RFC 4585 [RFC4585], payload Specific As specified in section 6.1 of RFC 4585 [RFC4585], payload Specific
Feedback messages are identified by the RTCP packet type value PSFB Feedback messages are identified by the RTCP packet type value PSFB
(206). AVPF [RFC4585] defines three payload-specific feedback (206). AVPF [RFC4585] defines three payload-specific feedback
messages and one application layer feedback message, and CCM messages and one application layer feedback message, and CCM
[RFC5104] specifies four payload-specific feedback messages. [RFC5104] specifies four payload-specific feedback messages.
These feedback messages are identified by means of the feedback These feedback messages are identified by means of the feedback
message type (FMT) parameter as follows: message type (FMT) parameter as follows:
skipping to change at page 86, line 8 skipping to change at page 86, line 11
0: unassigned 0: unassigned
8-14: unassigned 8-14: unassigned
16-30: unassigned 16-30: unassigned
The following subsections define the use of the PLI, SLI, RPSI, and The following subsections define the use of the PLI, SLI, RPSI, and
FIR feedback messages with HEVC. FIR feedback messages with HEVC.
8.1 Picture Loss Indication (PLI) 8.1 Picture Loss Indication (PLI)
As specified in RFC 4585 section 6.3.1, the reception of a picture As specified in RFC 4585 section 6.3.1, the reception of a picture
loss indication by a media sender indicates "the loss of an loss indication by a media sender indicates "the loss of an undefined
undefined amount of coded video data belonging to one or more amount of coded video data belonging to one or more pictures.".
pictures.". Without having any specific knowledge of the setup of Without having any specific knowledge of the setup of the bitstream
the bitstream (such as: use and location of in-band parameter sets, (such as: use and location of in-band parameter sets, non-IDR decoder
non-IDR decoder refresh points, picture structures, and so forth) a refresh points, picture structures, and so forth) a reaction to the
reaction to the reception of an PLI by an HEVC sender SHOULD be to reception of an PLI by an HEVC sender SHOULD be to send an IDR picture
send an IDR picture and relevant parameter sets; potentially with and relevant parameter sets; potentially with sufficient redundancy so
sufficient redundancy so to ensure correct reception. However, to ensure correct reception. However, sometimes information about the
sometimes information about the bitstream structure is known. For bitstream structure is known. For example, state could have been
example, state could have been established outside of the mechanisms established outside of the mechanisms defined in this document that
defined in this document that parameter sets are conveyed out of parameter sets are conveyed out of band only, and stay static for the
band only, and stay static for the duration of the session. In that duration of the session. In that case, it is obviously unnecessary to
case, it is obviously unnecessary to send them in-band as a result send them in-band as a result of the reception of a PLI. Other examples
of the reception of a PLI. Other examples could be devised based on could be devised based on a priori knowledge of different aspects of
a priori knowledge of different aspects of the bitstream structure. the bitstream structure. In all cases, the timing and congestion
In all cases, the timing and congestion control mechanisms of RFC control mechanisms of RFC 4585 MUST be observed.
4585 MUST be observed.
8.2 Slice Loss Indication 8.2 Slice Loss Indication
RFC 4585's Slice Loss Indication can be used to indicate, to a RFC 4585's Slice Loss Indication can be used to indicate, to a sender,
sender, the loss of a number of Coded Tree Blocks (CTBs) in CTB the loss of a number of Coded Tree Blocks (CTBs) in CTB raster scan
raster scan order of a picture. In the SLI's Feedback Control order of a picture. In the SLI's Feedback Control Indication (FCI)
Indication (FCI) field, the subfield "First" MUST be set to the CTB field, the subfield "First" MUST be set to the CTB address of the first
address of the first lost CTB. Note that the CTB address is in CTB lost CTB. Note that the CTB address is in CTB raster scan order of a
raster scan order of a picture. For the first CTB of a slice picture. For the first CTB of a slice segment, the CTB address is the
segment, the CTB address is the value of slice_segment_address when value of slice_segment_address when present; or 0 when
present; or 0 when first_slice_segement_in_pic_flag is equal to 1; first_slice_segement_in_pic_flag is equal to 1; both syntax elements
both syntax elements are in the slice segment header. The subfield are in the slice segment header. The subfield "Number" MUST be set to
"Number" MUST be set to the number of consecutive lost CTBs, again the number of consecutive lost CTBs, again in CTB raster scan order of
in CTB raster scan order of a picture. Note that due to both the a picture. Note that due to both the "First" and "Number" are counted
"First" and "Number" are counted in CTBs in CTB raster scan order, in CTBs in CTB raster scan order, of a picture, not in tile scan order
of a picture, not in tile scan order (which is the bitstream order (which is the bitstream order of CTBs), multiple SLI messages may be
of CTBs), multiple SLI messages may be needed to report the loss of needed to report the loss of one tile covering multiple CTB rows but
one tile covering multiple CTB rows but less wide than the picture. less wide than the picture.
The subfield "PictureID" MUST be set to the 6 least significant bits The subfield "PictureID" MUST be set to the 6 least significant bits
of a binary representation of the value of PicOrderCntVal, as of a binary representation of the value of PicOrderCntVal, as defined
defined in [HEVC], of the picture for which the lost CTBs are in [HEVC], of the picture for which the lost CTBs are indicated. Note
indicated. Note that for IDR pictures the syntax element that for IDR pictures the syntax element slice_pic_order_cnt_lsb is
slice_pic_order_cnt_lsb is not present, but then the value is not present, but then the value is inferred to be equal to 0.
inferred to be equal to 0.
As described in RFC 4585, an encoder in a media sender can use this As described in RFC 4585, an encoder in a media sender can use this
information to "clean up" the corrupted picture by sending intra information to "clean up" the corrupted picture by sending intra
information, while observing the constraints described in RFC4585, information, while observing the constraints described in RFC4585, for
for example with respect to congestion control. In many cases, example with respect to congestion control. In many cases, error
error tracking is required to identify the corrupted region in the tracking is required to identify the corrupted region in the receiver's
receiver's state (reference pictures) because of error import in state (reference pictures) because of error import in uncorrupted
uncorrupted regions of the picture through motion compensation. regions of the picture through motion compensation. Reference picture
Reference picture selection can also be used to "clean up" the selection can also be used to "clean up" the corrupted picture, which
corrupted picture, which is usually more efficient and less likely is usually more efficient and less likely to generate congestion than
to generate congestion than sending intra information. sending intra information.
In contrast to the video codecs contemplated in RFC 4585 and RFC In contrast to the video codecs contemplated in RFC 4585 and RFC 5104,
5104, in HEVC, the "macroblock size" is not fixed to 16x16 luma in HEVC, the "macroblock size" is not fixed to 16x16 luma samples, but
samples, but variable. That, however, does not create a conceptual variable. That, however, does not create a conceptual difficulty with
difficulty with SLI, because the setting of the CTB size is a SLI, because the setting of the CTB size is a sequence-level
sequence-level functionality, and using a slice loss indication functionality, and using a slice loss indication across coded video
across coded video sequence boundaries is meaningless as there is no sequence boundaries is meaningless as there is no prediction across
prediction across sequence boundaries. However, a proper use of SLI sequence boundaries. However, a proper use of SLI messages is not as
messages is not as straightforward as it was with older, fixed- straightforward as it was with older, fixed-macroblock-sized video
macroblock-sized video codecs, as the state of the sequence codecs, as the state of the sequence parameter set (where the CTB size
parameter set (where the CTB size is located) has to be taken into is located) has to be taken into account when interpreting the "First"
account when interpreting the "First" subfield in the FCI. subfield in the FCI.
8.3 Use of HEVC with the RPSI Feedback Message 8.3 Use of HEVC with the RPSI Feedback Message
Feedback based reference picture selection has been shown as a Feedback based reference picture selection has been shown as a
powerful tool to stop temporal error propagation for improved error powerful tool to stop temporal error propagation for improved error
resilience [Girod99][Wang05]. In one approach, the decoder side resilience [Girod99][Wang05]. In one approach, the decoder side
tracks errors in the decoded pictures and informs to the encoder tracks errors in the decoded pictures and informs to the encoder
side that a particular picture that has been decoded relatively side that a particular picture that has been decoded relatively
earlier is correct and still present in the decoded picture buffer earlier is correct and still present in the decoded picture buffer
and requests the encoder to use that correct picture for reference and requests the encoder to use that correct picture for reference
skipping to change at page 88, line 35 skipping to change at page 88, line 35
8.4 Full Intra Request (FIR) 8.4 Full Intra Request (FIR)
The purpose of the FIR message is to force an encoder to send an The purpose of the FIR message is to force an encoder to send an
independent decoder refresh point as soon as possible (observing, independent decoder refresh point as soon as possible (observing,
for example, the congestion control related constraints set out in for example, the congestion control related constraints set out in
RFC 5104). RFC 5104).
Upon reception of a FIR, a sender MUST send an IDR picture. Upon reception of a FIR, a sender MUST send an IDR picture.
Parameter sets MUST also be sent, except when there is a priori Parameter sets MUST also be sent, except when there is a priori
knowledge that the parameter sets have been correctly established. knowledge that the parameter sets have been correctly established.
(A typical example for that is an understanding between sender and A typical example for that is an understanding between sender and
receiver, established by means outside this document, that parameter receiver, established by means outside this document, that parameter
sets are exclusively sent out of band.) sets are exclusively sent out of band.
9. Security Considerations 9 Security Considerations
RTP packets using the payload format defined in this specification RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP are subject to the security considerations discussed in the RTP
specification [RFC3550], and in any applicable RTP profile such as specification [RFC3550], and in any applicable RTP profile such as
RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711] or RTP/AVP [RFC3551], RTP/AVPF [RFC4585], RTP/SAVP [RFC3711] or
RTP/SAVPF [RFC5124]. However, as "Securing the RTP Protocol RTP/SAVPF [RFC5124]. However, as "Securing the RTP Protocol
Framework: Why RTP Does Not Mandate a Single Media Security Framework: Why RTP Does Not Mandate a Single Media Security
Solution" [I-D.ietf-avt-srtp-not-mandatory] discusses it is not an Solution" [I-D.ietf-avt-srtp-not-mandatory] discusses it is not an
RTP payload format's responsibility to discuss or mandate what RTP payload format's responsibility to discuss or mandate what
solutions are used to meet the basic security goals like solutions are used to meet the basic security goals like
skipping to change at page 90, line 8 skipping to change at page 90, line 8
containing the bitstream. containing the bitstream.
End-to-end security with authentication, integrity, or End-to-end security with authentication, integrity, or
confidentiality protection will prevent a MANE from performing confidentiality protection will prevent a MANE from performing
media-aware operations other than discarding complete packets. In media-aware operations other than discarding complete packets. In
the case of confidentiality protection, it will even be prevented the case of confidentiality protection, it will even be prevented
from discarding packets in a media-aware way. To be allowed to from discarding packets in a media-aware way. To be allowed to
perform such operations, a MANE is required to be a trusted entity perform such operations, a MANE is required to be a trusted entity
that is included in the security context establishment. that is included in the security context establishment.
10. Congestion Control 10 Congestion Control
Congestion control for RTP SHALL be used in accordance with RTP Congestion control for RTP SHALL be used in accordance with RTP
[RFC3550] and with any applicable RTP profile, e.g. AVP [RFC 3551]. [RFC3550] and with any applicable RTP profile, e.g. AVP [RFC 3551].
If best-effort service is being used, an additional requirement is If best-effort service is being used, an additional requirement is
that users of this payload format MUST monitor packet loss to ensure that users of this payload format MUST monitor packet loss to ensure
that the packet loss rate is within an acceptable range. Packet that the packet loss rate is within an acceptable range. Packet
loss is considered acceptable if a TCP flow across the same network loss is considered acceptable if a TCP flow across the same network
path, and experiencing the same network conditions, would achieve an path, and experiencing the same network conditions, would achieve an
average throughput, measured on a reasonable timescale, that is not average throughput, measured on a reasonable timescale, that is not
less than all RTP streams combined is achieving. This condition can less than all RTP streams combined is achieving. This condition can
skipping to change at page 91, line 18 skipping to change at page 91, line 18
that RTP stream was damaged due to previous packet losses. This can that RTP stream was damaged due to previous packet losses. This can
help reduce the network load in certain special cases. For example, help reduce the network load in certain special cases. For example,
MANES can remove those FUs where the leading FUs belonging to the MANES can remove those FUs where the leading FUs belonging to the
same NAL unit have been lost or those dependent slice segments when same NAL unit have been lost or those dependent slice segments when
the leading slice segments belonging to the same slice have been the leading slice segments belonging to the same slice have been
lost, because the trailing FUs or dependent slice segments are lost, because the trailing FUs or dependent slice segments are
meaningless to most decoders. MANES can also remove higher temporal meaningless to most decoders. MANES can also remove higher temporal
scalable layers if the outbound transmission (from the MANE's scalable layers if the outbound transmission (from the MANE's
viewpoint) experiences congestion. viewpoint) experiences congestion.
11. IANA Consideration 11 IANA Consideration
A new media type, as specified in Section 7.1 of this memo, should A new media type, as specified in Section 7.1 of this memo, should
be registered with IANA. be registered with IANA.
12. Acknowledgements 12 Acknowledgements
Muhammed Coban and Marta Karczewicz are thanked for discussions on Muhammed Coban and Marta Karczewicz are thanked for discussions on
the specification of the use with feedback messages and other the specification of the use with feedback messages and other
aspects in this memo. Jonathan Lennox and Jill Boyce are thanked aspects in this memo. Jonathan Lennox and Jill Boyce are thanked
for their contributions to the PACI design included in this memo. for their contributions to the PACI design included in this memo.
Rickard Sjoberg, Arild Fuldseth, Bo Burman, Magnus Westerlund, and Rickard Sjoberg, Arild Fuldseth, Bo Burman, Magnus Westerlund, and
Tom Kristensen are thanked for their contributions to parallel Tom Kristensen are thanked for their contributions to parallel
processing related signalling. Magnus Westerlund, Jonathan Lennox, processing related signalling. Magnus Westerlund, Jonathan Lennox,
Bernard Aboba, Jonatan Samuelsson, Roni Even, Rickard Sjoberg, Bernard Aboba, Jonatan Samuelsson, Roni Even, Rickard Sjoberg,
Sachin Deshpande, Woo Johnman, Mo Zanaty, and Ross Finlayson made Sachin Deshpande, Woo Johnman, Mo Zanaty, Ross Finlayson, and Danny
valuable reviewing comments that led to improvements. Hong made valuable reviewing comments that led to improvements.
This document was prepared using 2-Word-v2.0.template.dot. This document was prepared using 2-Word-v2.0.template.dot.
13. References 13 References
13.1 Normative References 13.1 Normative References
[HEVC] ITU-T Recommendation H.265, "High efficiency video [HEVC] ITU-T Recommendation H.265, "High efficiency video
coding", April 2013. coding", April 2013.
[H.264] ITU-T Recommendation H.264, "Advanced video coding for [H.264] ITU-T Recommendation H.264, "Advanced video coding for
generic audiovisual services", April 2013. generic audiovisual services", April 2013.
[RFC5583] Schierl, T. and Wenger, S., "Signaling Media Decoding [RFC5583] Schierl, T. and Wenger, S., "Signaling Media Decoding
skipping to change at page 95, line 5 skipping to change at page 95, line 5
presentation description and segment formats", 2012. presentation description and segment formats", 2012.
[RFC5109] Li, A., "RTP Payload Format for Generic Forward Error [RFC5109] Li, A., "RTP Payload Format for Generic Forward Error
Correction", RFC 5109, December 2007. Correction", RFC 5109, December 2007.
[Wang05] Wang, Y.-K., Zhu, C., and Li, H., "Error resilient video [Wang05] Wang, Y.-K., Zhu, C., and Li, H., "Error resilient video
coding using flexible reference fames", Visual coding using flexible reference fames", Visual
Communications and Image Processing 2005 (VCIP 2005), July Communications and Image Processing 2005 (VCIP 2005), July
2005, Beijing, China. 2005, Beijing, China.
14. Authors' Addresses 14 Authors' Addresses
Ye-Kui Wang Ye-Kui Wang
Qualcomm Incorporated Qualcomm Incorporated
5775 Morehouse Drive 5775 Morehouse Drive
San Diego, CA 92121 San Diego, CA 92121, USA
USA
Phone: +1-858-651-8345 Phone: +1-858-651-8345
EMail: yekuiw@qti.qualcomm.com EMail: yekuiw@qti.qualcomm.com
Yago Sanchez Yago Sanchez
Fraunhofer HHI Fraunhofer HHI
Einsteinufer 37 Einsteinufer 37
D-10587 Berlin D-10587 Berlin, Germany
Germany
Phone: +49-30-31002-227 Phone: +49-30-31002-227
Email: yago.sanchez@hhi.fraunhofer.de Email: yago.sanchez@hhi.fraunhofer.de
Thomas Schierl Thomas Schierl
Fraunhofer HHI Fraunhofer HHI
Einsteinufer 37 Einsteinufer 37
D-10587 Berlin D-10587 Berlin, Germany
Germany
Phone: +49-30-31002-227 Phone: +49-30-31002-227
Email: ts@thomas-schierl.de Email: ts@thomas-schierl.de
Stephan Wenger Stephan Wenger
Vidyo, Inc. Vidyo, Inc.
433 Hackensack Ave., 7th floor 433 Hackensack Ave., 7th floor
Hackensack, N.J. 07601 Hackensack, N.J. 07601, USA
USA
Phone: +1-415-713-5473 Phone: +1-415-713-5473
EMail: stewe@stewe.org EMail: stewe@stewe.org
Miska M. Hannuksela Miska M. Hannuksela
Nokia Corporation Nokia Corporation
P.O. Box 1000 P.O. Box 1000
33721 Tampere 33721 Tampere, Finland
Finland
Phone: +358-7180-08000 Phone: +358-7180-08000
EMail: miska.hannuksela@nokia.com EMail: miska.hannuksela@nokia.com
 End of changes. 61 change blocks. 
175 lines changed or deleted 169 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/