One document matched: draft-uberti-payload-vp9-00.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
<!ENTITY rfc2119 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.2119.xml">
<!ENTITY rfc3550 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3550.xml">
<!ENTITY rfc3551 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3551.xml">
<!ENTITY rfc3711 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3711.xml">
<!ENTITY rfc3984 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.3984.xml">
<!ENTITY rfc4855 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.4855.xml">
<!ENTITY rfc4566 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.4566.xml">
<!ENTITY rfc4585 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.4585.xml">
<!ENTITY rfc6838 SYSTEM "http://xml.resource.org/public/rfc/bibxml/reference.RFC.6838.xml">
<!ENTITY vp9 SYSTEM "http://xml.resource.org/public/rfc/bibxml3/reference.I-D.grange-vp9-bitstream.xml">
]>
<rfc category="std" docName="draft-uberti-payload-vp9-00" ipr="trust200902">
<?rfc symrefs="yes" ?>
<?rfc sortrefs="yes" ?>
<!-- alphabetize the references -->
<?rfc comments="no"?>
<!-- show comments -->
<?rfc inline="yes" ?>
<!-- comments are inline -->
<?rfc toc="yes" ?>
<!-- generate table of contents -->
<front>
<title abbrev="RTP Payload Format for VP9">RTP Payload Format for VP9
Video</title>
<author fullname="Justin Uberti" initials="J." surname="Uberti">
<organization abbrev="Google">Google, Inc.</organization>
<address>
<postal>
<street>747 6th Street South</street>
<city>Kirkland</city>
<region>WA</region>
<code>98033</code>
<country>USA</country>
</postal>
<email>justin@uberti.name</email>
</address>
</author>
<author fullname="Stefan Holmer" initials="S." surname="Holmer">
<organization abbrev="Google">Google, Inc.</organization>
<address>
<postal>
<street>Kungsbron 2</street>
<code>111 22</code>
<city>Stockholm</city>
<country>Sweden</country>
</postal>
</address>
</author>
<author fullname="Magnus Flodman" initials="M." surname="Flodman">
<organization abbrev="Google">Google, Inc.</organization>
<address>
<postal>
<street>Kungsbron 2</street>
<code>111 22</code>
<city>Stockholm</city>
<country>Sweden</country>
</postal>
</address>
</author>
<author fullname="Jonathan Lennox" initials="J." surname="Lennox">
<organization abbrev="Vidyo">Vidyo, Inc.</organization>
<address>
<postal>
<street>433 Hackensack Avenue</street>
<street>Seventh Floor</street>
<city>Hackensack</city>
<region>NJ</region>
<code>07601</code>
<country>US</country>
</postal>
<email>jonathan@vidyo.com</email>
</address>
</author>
<date/>
<area>RAI</area>
<workgroup>Payload Working Group</workgroup>
<keyword>RFC</keyword>
<keyword>Request for Comments</keyword>
<keyword>RTP</keyword>
<keyword>VP9</keyword>
<keyword>WebM</keyword>
<abstract>
<t>This memo describes an RTP payload format for the VP9 video codec.
The payload format has wide applicability, as it supports applications
from low bit-rate peer-to-peer usage, to high bit-rate video
conferences. It includes provisions for temporal and spatial scalability.</t>
</abstract>
</front>
<middle>
<section anchor="intro" title="Introduction">
<t>This memo describes an RTP payload specification applicable to the
transmission of video streams encoded using the VP9 video codec <xref
target="I-D.grange-vp9-bitstream"/>. The format described in this document can be used
both in peer-to-peer and video conferencing applications.</t>
<t>TODO: VP9 description. Please see <xref
target="I-D.grange-vp9-bitstream"/>.</t>
</section>
<section anchor="conventions"
title="Conventions, Definitions and Acronyms">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119"/>.</t>
</section>
<section anchor="mediaFormatDescription" title="Media Format Description">
<t>The VP9 codec can maintain up to eight reference frames, of
which up to three can be referenced or updated by any new frame.</t>
<t>VP9 also allows a reference frame to be resampled and used as a
reference for another frame of a different resolution. This
allows internal resolution changes without requiring the use of
keyframes.</t>
<t>These features together enable an encoder to
implement various forms of coarse-grained scalability,
including temporal, spatial, and quality scalability modes, as
well as combinations of these, without the need for explicit
spatially scalabile encoding modes.</t>
<t>This payload format specification defines how such
scalability modes can be encoded and communicated. In this
payload, three separate types of layers are defined: temporal,
spatial, and quality.</t>
<t>Temporal layers define different frame rates of video;
spatial and quality layers define different, dependent
representations of a single picture. Spatial layers allow
a picture to be encoded at different resolutions, whereas
quality layers allow a picture to be encoded at the same
resolution but at different bitrates (and thus with different
amounts of coding error).</t>
<t>Layers are designed (and MUST be encoded) such that if
any layer, and all higher layers, are removed from the bitstream
along any of the three dimensions, the remaining bitstream is
still correctly decodable.</t>
<t>For terminology, this document uses the term "frame" to refer
to a single encoded VP9 image, and "picture" to refer to all the
representations of frames at a single instant in time. A
picture thus can consist of multiple frames, encoding different
spatial and/or quality layers.</t>
<t>[Editor's Note: Are separate spatial and quality layers
necessary and useful? We could simplify by only defining a
single sequence of frames within a picture.</t>
<t>Two modes of describing layer information are possible:
"non-flexible mode" and "flexible mode". An encoder can
freely switch between the two as appropriate.</t>
<t>In non-flexible mode, an SS message, which defines the
layer hierarchy, is sent in the beginning of the stream
together with the key frame. Each packet will have a picture
id and reference indices, which in conjunction with the SS and the
RTP sequence number can be used to determine if the packet is
decodable or not. An SU message can be sent by the sending
client, or an MCU, to notify the receiver about what subset of
the SS it will actually be receiving.</t>
<t>In the flexible mode each packet contains 1-4 reference
indices, which identifies all frames referenced by the frame
transmitted in the current packet. This enables a receiver to
identify if a frame is decodable or not and helps it
understand the layer structure so that it can drop packets as
it sees fit. Since this is signaled in each packet it makes it
possible to have more flexible layer hierarchies and patterns
which are changing dynamically.</t>
</section>
<section anchor="payloadFormat" title="Payload Format">
<t>This section describes how the encoded VP9 bitstream is encapsulated
in RTP. To handle network losses usage of RTP/AVPF <xref
target="RFC4585"/> is RECOMMENDED. All integer fields in the
specifications are encoded as unsigned integers in network octet
order.</t>
<section anchor="RTPHeaderUsage" title="RTP Header Usage">
<figure anchor="figureRTPHeader">
<preamble>The general RTP payload format for VP9 is depicted
below.</preamble>
<artwork><![CDATA[
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|V=2|P|X| CC |M| PT | sequence number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| timestamp |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| synchronization source (SSRC) identifier |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| contributing source (CSRC) identifiers |
| .... |
+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
| VP9 payload descriptor (integer #bytes) |
: :
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| : VP9 pyld hdr | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
| |
+ |
: Bytes 2..N of VP9 payload :
| |
| +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| : OPTIONAL RTP padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
<postamble>The VP9 payload descriptor and VP9 payload header will be
described in the next section. OPTIONAL RTP padding MUST NOT be included
unless the P bit is set.</postamble>
</figure>
<t><list style="hanging">
<t hangText="Marker bit (M):">MUST be set for the final packet
of each encoded frame. This enables a decoder to finish decoding the
frame, where it otherwise may need to wait for the next packet
to explicitly know that the frame is complete. Note that,
if spatial or quality scalability is in use, more frames from the
same picture may follow; see the description of the E bit below.</t>
<t hangText="Timestamp:">The RTP timestamp indicates the time when
the frame was sampled, at a clock rate of 90 kHz. If a
picture is encoded with multiple frames, all of the
frames of the picture have the same timestamp.</t>
<t hangText="Sequence number:">The sequence numbers are
monotonically increasing in order of the encoded bitstream.</t>
<t>The remaining RTP header fields are used as specified in <xref
target="RFC3550"/>.</t>
</list></t>
</section>
<section anchor="VP9payloadDescriptor" title="VP9 Payload Description">
<figure anchor="figureVP9payloadDescriptor">
<preamble>The first octets after the RTP header are the VP9 payload
descriptor, with the following structure.</preamble>
<artwork><![CDATA[
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|I|L|F|B|E|V|U|-| (REQUIRED)
+-+-+-+-+-+-+-+-+
I: |M|PICTURE ID | (RECOMMENDED)
+-+-+-+-+-+-+-+-+
M: | EXTENDED PID | (RECOMMENDED)
+-+-+-+-+-+-+-+-+
L: | T | S | Q | R | (CONDITIONALLY RECOMMENDED)
+-+-+-+-+-+-+-+-+ -\
F: | PID |X| RS| RQ| (OPTIONAL) .
+-+-+-+-+-+-+-+-+ . - R times
X: | EXTENDED PID | (OPTIONAL) .
+-+-+-+-+-+-+-+-+ -/
V: | SS |
| .. |
+-+-+-+-+-+-+-+-+
U: | SU |
| .. |
+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<t><list style="hanging">
<t hangText="I:">PictureID present. When set to one, the OPTIONAL
PictureID MUST be present after the mandatory first octet and
specified as below. Otherwise, PictureID MUST NOT be present.</t>
<t hangText="L:">Layer indices present. When set to one,
the octets following the first octet and the extended
Picture ID (if present) are as described by "Layer
indices" below.</t>
<t hangText="F:">Reference indices present. When set to one,
the octets following the first octet and the extended
Picture ID (if present) are as described by "Reference
indices" below. This MUST only be set if L is also
1; if L is 0 then this MUST be set to zero and
ignored by receivers.</t>
<t hangText="B:">Start of VP9 frame. MUST be set to 1 if
the first payload octet of the RTP packet is the beginning of a
new VP9 frame, and MUST NOT be 1 otherwise. Note that this
frame might not be the first frame of the picture.</t>
<t hangText="E:">End of picture. MUST be set to 1 for the final
RTP packet of a VP9 picture, and 0 otherwise. Unless
spatial or quality scalability is in use for this picture, this will have the same
value as the marker bit in the RTP header.</t>
<t hangText="V:">Scalability Structure (SS) present. When set
to one, the OPTIONAL Scalability Structure MUST be
present in the payload descriptor. Otherwise, the
Scalability Structure MUST NOT be present.</t>
<t hangText="U:">Scalability Structure Update (SU) present. When set
to one, the OPTIONAL Scalability Structure Update MUST be
present in the payload descriptor. Otherwise, the
Scalability Structure Update MUST NOT be present.</t>
<t hangText="-:">Bit reserved for future use. MUST be set to
zero and MUST be ignored by the receiver.</t>
</list></t>
<t>After the extension bit field follow the extension data fields that
are enabled. <list style="hanging">
<t hangText="M:">The most significant bit of the first octet is an
extension flag. The field MUST be present if the I bit is equal to
one. If set the PictureID field MUST contain 16 bits else it MUST
contain 8 bits including this MSB, see PictureID.</t>
<t hangText="PictureID:">8 or 16 bits
including the M bit. This is a running index of the frames. The
field MUST be present if the I bit is equal to one. The 7
following bits carry (parts of) the PictureID. If the extension
flag is one, the PictureID continues in the next octet forming a
15 bit index, where the 8 bits in the second octet are the least
significant bits of the PictureID. If the extension flag is zero,
there is no extension, and the PictureID is the 7 remaining bits
of the first (and only) octet. The sender may choose 7 or 15 bits
index. The PictureID SHOULD start on a random number, and MUST
wrap after reaching the maximum ID. The receiver MUST NOT assume
that the number of bits in PictureID stay the same through the
session.</t>
<t hangText="Layer indices:">This byte is optional, but
recommended whenever encoding with layers. T, S and Q
are 2-bit indices for temporal, spatial, and quality
layers, respectively. S and Q start at zero for each
picture, and increment consecutively (with Q
incrementing before S). These can help MCUs measure bitrates per
layer and can help them make a quick decision on whether
to relay a packet or not. They can also help receivers
determine what layers they are currently decoding. If "F" is set in the initial octet, R
is 2 bits representing the number of reference fields this frame
refers to. R MAY be zero, indicating a keyframe. The layer indices field will be followed by R
reference indices. If "F" is not set, R MUST be set to zero and ignored by receivers.</t>
<t hangText="Reference indices:">These bytes are optional, but recommended when encoding with layers in
the flexible mode. They are also recommended in the
non-flexible mode when sending frames which are out of sync
with the pattern signaled with the SS, for instance when
encoding a layer synchronization frame in response to a LIR.
<list>
<t hangText="PID:">The relative Picture ID referred to by this frame. I.e., PID=3
on a packet containing the frame with Picture ID 112 means
that the frame refers back to the frame with picture ID
109. This calculation is done modulo the size of the Picture
ID field, i.e. either 7 or 15 bits. For most layer
structures a 3-bit relative Picture ID will be enough;
however, the X bit can be used to refer to pictures with
Picture IDs more than 7 previously.</t>
<t hangText="RS and RQ:">The spatial and quality layer
IDs of the frame referred to by this frame, in the picture
identified by the relative Picture ID.</t>
<t hangText="X:">1 if this layer index has an extended relative Picture ID.</t>
</list>
These 1-2 bytes are repeated R times, defined by the two R bits in the
layer indices field.</t>
</list></t>
<section anchor="VP9payloadDescriptorSS" title="Scalability Structure (SS):">
<t>The Scalability Structure data describes
the pattern of scalable frames that will be used in a scalable
stream. If the VP9 payload header's "V" bit is set,
the scalability structure (SS) is present in the position
indicated in <xref target="figureVP9payloadDescriptor"/>.</t>
<figure anchor="figureVP9ScalabilityStructure">
<artwork><![CDATA[
+-+-+-+-+-+-+-+-+
V: | PATTERN LENGTH|
+-+-+-+-+-+-+-+-+ -\
| T | S | Q | R | (OPTIONAL) .
+-+-+-+-+-+-+-+-+ -\ .
| PID |X| RS| RQ| (OPTIONAL) . . - PAT. LEN. times
+-+-+-+-+-+-+-+-+ . - R times .
X: | EXTENDED PID | (OPTIONAL) . .
+-+-+-+-+-+-+-+-+ -/ -/
]]></artwork>
</figure>
<t>The scalability structure allows the structure of the
VP9 stream to be predeclared, rather than indicating it on
the fly with every frame as with the layer indices.</t>
<t>Its structure consists of a sequence of frames, encoded
as with the layer indices. It begins with PATTERN LENGTH,
indicating the number of frames in the pattern; it is then
followed by that many instances of data encoded using the
same semantics as the layer indices.</t>
<t>TODO: add frame resolution information.</t>
<t>In a scalable
stream sent with a fixed pattern, the scalability
structure SHOULD be included in the first packet of every
keyframe picture, and also in the first packet of the
first picture in which the scalability structure changes.
If a SS is included in a picture with TID not equal to 0,
it MUST also be repeated in the first packet the first
frame with a lower TID, until TID equals 0.</t>
<t>If PATTERN LENGTH is 0, it indicates that no fixed
scalability information is present going forward in the
bitstream. An SS with a PATTERN LENGTH of 0 allows a
bitstream to be changed from non-flexible to flexible
mode.</t>
</section>
<section anchor="VP9payloadDescriptorSU" title="Scalability Structure Update (SU):">
<t>TODO</t>
<t><vspace blankLines="100"/></t>
<!-- force a pagebreak-->
</section>
</section>
<section anchor="VP9payloadHeader" title="VP9 Payload Header">
<t>TODO: need to describe VP9 payload header.</t>
</section>
<section title="Frame Fragmentation">
<t>VP9 frames are fragmented into packets, in RTP sequence
number order, beginning with a
packet with the B bit set, and ending with a packet with the
RTP marker bit set. There is no mechanism for finer-grained
access to parts of a VP9 frame.</t>
</section>
<section title="Examples of VP9 RTP Stream">
<t>TODO</t>
</section>
</section>
<section anchor="RPSIandSLI" title="Using VP9 with RPSI and SLI Feedback">
<t>The VP9 payload descriptor defined in <xref
target="VP9payloadDescriptor"/> above contains an optional PictureID
parameter. One use of this parameter is included to enable use of reference
picture selection index (RPSI) and slice loss indication (SLI), both
defined in <xref target="RFC4585"/>.</t>
<section anchor="RPSI" title="RPSI">
<t>TODO: Update to indicate which frame within the picture.</t>
<t>The reference picture selection index is a payload-specific
feedback message defined within the RTCP-based feedback format. The
RPSI message is generated by a receiver and can be used in two ways.
Either it can signal a preferred reference picture when a loss has
been detected by the decoder -- preferably then a reference that the
decoder knows is perfect -- or, it can be used as positive feedback
information to acknowledge correct decoding of certain reference
pictures. The positive feedback method is useful for VP9 used as
unicast. The use of RPSI for VP9 is preferably combined with a special
update pattern of the codec's two special reference frames -- the
golden frame and the altref frame -- in which they are updated in an
alternating leapfrog fashion. When a receiver has received and
correctly decoded a golden or altref frame, and that frame had a
PictureID in the payload descriptor, the receiver can acknowledge this
simply by sending an RPSI message back to the sender. The message body
(i.e., the "native RPSI bit string" in <xref target="RFC4585"/>) is
simply the PictureID of the received frame.</t>
</section>
<section anchor="SLI" title="SLI">
<t>TODO: Update to indicate which frame within the picture.</t>
<t>The slice loss indication is another payload-specific feedback
message defined within the RTCP-based feedback format. The SLI message
is generated by the receiver when a loss or corruption is detected in
a frame. The format of the SLI message is as follows <xref
target="RFC4585"/>:</t>
<figure anchor="figureSLIHeader">
<artwork><![CDATA[
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| First | Number | PictureID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
]]></artwork>
</figure>
<t>Here, First is the macroblock address (in scan order) of the first
lost block and Number is the number of lost blocks. PictureID is the
six least significant bits of the codec-specific picture identifier in
which the loss or corruption has occurred. For VP9, this
codec-specific identifier is naturally the PictureID of the current
frame, as read from the payload descriptor. If the payload descriptor
of the current frame does not have a PictureID, the receiver MAY send
the last received PictureID+1 in the SLI message. The receiver MAY set
the First parameter to 0, and the Number parameter to the total number
of macroblocks per frame, even though only parts of the frame is
corrupted. When the sender receives an SLI message, it can make use of
the knowledge from the latest received RPSI message. Knowing that the
last golden or altref frame was successfully received, it can encode
the next frame with reference to that established reference.</t>
</section>
<section title="Example">
<t>TODO: this example is copied from the VP8 payload format specification,
and has not been updated for VP9. It may be incorrect.</t>
<t>The use of RPSI and SLI is best illustrated in an example. In this
example, the encoder may not update the altref frame until the last
sent golden frame has been acknowledged with an RPSI message. If an
update is not received within some time, a new golden frame update is
sent instead. Once the new golden frame is established and
acknowledged, the same rule applies when updating the altref
frame.</t>
<texttable anchor="table_example_timing"
title="Example signaling between sender and receiver">
<ttcol align="left">Event</ttcol>
<ttcol align="left">Sender</ttcol>
<ttcol align="left">Receiver</ttcol>
<ttcol align="left">Established reference</ttcol>
<c>1000</c>
<c>Send golden frame PictureID = 0</c>
<c/>
<c/>
<c/>
<c/>
<c>Receive and decode golden frame</c>
<c/>
<c>1001</c>
<c/>
<c>Send RPSI(0)</c>
<c/>
<c>1002</c>
<c>Receive RPSI(0)</c>
<c/>
<c>golden</c>
<c>...</c>
<c>(sending regular frames)</c>
<c/>
<c/>
<c>1100</c>
<c>Send altref frame PictureID = 100</c>
<c/>
<c/>
<c/>
<c/>
<c>Altref corrupted or lost</c>
<c>golden</c>
<c>1101</c>
<c/>
<c>Send SLI(100)</c>
<c>golden</c>
<c>1102</c>
<c>Receive SLI(100)</c>
<c/>
<c/>
<c>1103</c>
<c>Send frame with reference to golden</c>
<c/>
<c/>
<c/>
<c/>
<c>Receive and decode frame (decoder state restored)</c>
<c>golden</c>
<c>...</c>
<c>(sending regular frames)</c>
<c/>
<c/>
<c>1200</c>
<c>Send altref frame PictureID = 200</c>
<c/>
<c/>
<c/>
<c/>
<c>Receive and decode altref frame</c>
<c>golden</c>
<c>1201</c>
<c/>
<c>Send RPSI(200)</c>
<c/>
<c>1202</c>
<c>Receive RPSI(200)</c>
<c/>
<c>altref</c>
<c>...</c>
<c>(sending regular frames)</c>
<c/>
<c/>
<c>1300</c>
<c>Send golden frame PictureID = 300</c>
<c/>
<c/>
<c/>
<c/>
<c>Receive and decode golden frame</c>
<c>altref</c>
<c>1301</c>
<c/>
<c>Send RPSI(300)</c>
<c>altref</c>
<c>1302</c>
<c>RPSI lost</c>
<c/>
<c/>
<c>1400</c>
<c>Send golden frame PictureID = 400</c>
<c/>
<c/>
<c/>
<c/>
<c>Receive and decode golden frame</c>
<c>altref</c>
<c>1401</c>
<c/>
<c>Send RPSI(400)</c>
<c/>
<c>1402</c>
<c>Receive RPSI(400)</c>
<c/>
<c>golden</c>
</texttable>
<t>Note that the scheme is robust to loss of the feedback messages. If
the RPSI is lost, the sender will try to update the golden (or altref)
again after a while, without releasing the established reference.
Also, if an SLI is lost, the receiver can keep sending SLI messages at
any interval allowed by the RTCP sending timing restrictions as
specified in <xref target="RFC4585"/>, as long as the picture is
corrupted.</t>
</section>
</section>
<section anchor="layerIntraRequest" title="Layer Intra Request">
<t>Editor's Note: The message described in this section is
applicable to other codecs beyond just VP9. In the future it
will be likely be split out into another document.</t>
<t>TODO: details of how this is encoded in RTCP.</t>
<t>A synchronization frame can be requested by sending a LIR,
which is an RTCP feedback message asking the encoder to encode a frame
which makes it possible to upgrade to a higher layer. The LIR
message contains two tuples, {T1,S1,Q1} and {T2,S2,Q2}, where
the first tuple is the currently highest layer the decoder can
decode, while the second tuple is the layer the decoder wants to
upgrade to.</t>
<t>Identification of an upgrade frame can be derived from the
reference IDs of each frame by backtracking the dependency chain
until reaching a point where only decodable frames are being
referenced. Therefore it's recommended both for both the
flexible and the non-flexible mode that, when upgrade frames are
being encoded in response to a LIR, those packets should contain
layer indices and the reference fields so that the decoder or an
MCU can make this derivation.</t>
<t>Example:</t>
<t>LIR {1,1,0}, {1,2,1} is sent by an MCU when it is currently
relaying {1,1,0} to a receiver and which wants to upgrade to
{1,2,1}. In response the encoder should encode the next frames
in layers {1,1,1} and {1,2,1} by only referring to frames in
{1,1,0}, {1,0,0} or {0,0,0}.</t>
<t>In the non-flexible mode, periodic upgrade frames can be
defined by the layer structure of the SS, thus periodic upgrade
frames can be automatically identified by the picture ID.</t>
</section>
<section anchor="payloadFormatParameters"
title="Payload Format Parameters">
<t>This payload format has two required parameters.</t>
<section anchor="mediaTypeRegistration" title="Media Type Definition">
<t>This registration is done using the template defined in <xref
target="RFC6838"/> and following <xref target="RFC4855"/>. <list
style="hanging">
<t hangText="Type name:">video</t>
<t hangText="Subtype name:">VP9</t>
<t hangText="Required parameters:"><vspace blankLines="0"/> These
parameters MUST be used to signal the capabilities of a receiver
implementation. These parameters MUST NOT be used for any other
purpose. <list style="hanging">
<t hangText="max-fr:">The value of max-fr is an integer
indicating the maximum frame rate in units of frames per
second that the decoder is capable of decoding.</t>
<t hangText="max-fs:">The value of max-fs is an integer
indicating the maximum frame size in units of macroblocks that
the decoder is capable of decoding.</t>
<t>The decoder is capable of decoding this frame size as long
as the width and height of the frame in macroblocks are less
than int(sqrt(max-fs * 8)) - for instance, a max-fs of 1200
(capable of supporting 640x480 resolution) will support widths
and heights up to 1552 pixels (97 macroblocks).</t>
</list></t>
<t hangText="Optional parameters:">none</t>
<t hangText="Encoding considerations:"><vspace blankLines="0"/>
This media type is framed in RTP and contains binary data; see
Section 4.8 of <xref target="RFC6838"/>.</t>
<t hangText="Security considerations:">See <xref
target="securityConsiderations"/> of RFC xxxx. <vspace
blankLines="0"/> [RFC Editor: Upon publication as an RFC, please
replace "XXXX" with the number assigned to this document and
remove this note.]</t>
<t hangText="Interoperability considerations:">None.</t>
<t hangText="Published specification:">VP9 bitstream format <xref
target="I-D.grange-vp9-bitstream"/> and RFC XXXX. <vspace blankLines="0"/> [RFC
Editor: Upon publication as an RFC, please replace "XXXX" with the
number assigned to this document and remove this note.] <vspace
blankLines="0"/></t>
<t hangText="Applications which use this media type:"><vspace
blankLines="0"/> For example: Video over IP, video
conferencing.</t>
<t hangText="Additional information:">None.</t>
<t
hangText="Person & email address to contact for further information:"><vspace
blankLines="0"/> TODO [Pick a contact]</t>
<t hangText="Intended usage:">COMMON</t>
<t hangText="Restrictions on usage:"><vspace blankLines="0"/> This
media type depends on RTP framing, and hence is only defined for
transfer via RTP <xref target="RFC3550"/>.</t>
<t hangText="Author:">TODO [Pick a contact]</t>
<t hangText="Change controller:"><vspace blankLines="0"/> IETF
Payload Working Group delegated from the IESG.</t>
</list></t>
</section>
<section title="SDP Parameters">
<t>The receiver MUST ignore any fmtp parameter unspecified in this
memo.</t>
<section title="Mapping of Media Subtype Parameters to SDP">
<t>The media type video/VP9 string is mapped to fields in the
Session Description Protocol (SDP) <xref target="RFC4566"/> as
follows: <list style="symbols">
<t>The media name in the "m=" line of SDP MUST be video.</t>
<t>The encoding name in the "a=rtpmap" line of SDP MUST be VP9
(the media subtype).</t>
<t>The clock rate in the "a=rtpmap" line MUST be 90000.</t>
<t>The parameters "max-fs", and "max-fr", MUST be included in
the "a=fmtp" line of SDP. These parameters are expressed as a
media subtype string, in the form of a semicolon separated list
of parameter=value pairs.</t>
</list></t>
<section title="Example">
<t>An example of media representation in SDP is as follows:</t>
<t>m=video 49170 RTP/AVPF 98<vspace blankLines="0"/> a=rtpmap:98
VP9/90000<vspace blankLines="0"/> a=fmtp:98 max-fr=30;
max-fs=3600;<vspace blankLines="0"/></t>
</section>
</section>
<section title="Offer/Answer Considerations">
<t>TODO: Update this for VP9</t>
<!--
<t>The VP9 codec offers a decode complexity that is roughly linear
with the number of pixels encoded. The parameters "max-fr" and
"max-fs" are defined in <xref target="mediaTypeRegistration"/>,
where the macroblock size is 16x16 pixels as defined in <xref
target="RFC6386"/>, the max-fs and max-fr parameters MUST be used to
establish these limits.</t>
<t>NOTE IN DRAFT: If closer control of width and height is desired,
the mechanism described in
draft-nandakumar-payload-sdp-max-video-resolution is a possible
candidate for signalling, but since that document appears to be far
from finalization, this document does not make a reference to that
document. This note is only intended for facilitating WG discussion,
and should be deleted before publication of this document as an
RFC.</t> -->
</section>
</section>
</section>
<section anchor="securityConsiderations" title="Security Considerations">
<t>RTP packets using the payload format defined in this specification
are subject to the security considerations discussed in the RTP
specification <xref target="RFC3550"/>, and in any applicable RTP
profile. The main security considerations for the RTP packet carrying
the RTP payload format defined within this memo are confidentiality,
integrity and source authenticity. Confidentiality is achieved by
encryption of the RTP payload. Integrity of the RTP packets through
suitable cryptographic integrity protection mechanism. Cryptographic
system may also allow the authentication of the source of the payload. A
suitable security mechanism for this RTP payload format should provide
confidentiality, integrity protection and at least source authentication
capable of determining if an RTP packet is from a member of the RTP
session or not. Note that the appropriate mechanism to provide security
to RTP and payloads following this memo may vary. It is dependent on the
application, the transport, and the signaling protocol employed.
Therefore a single mechanism is not sufficient, although if suitable the
usage of SRTP <xref target="RFC3711"/> is recommended. This RTP payload
format and its media decoder do not exhibit any significant
non-uniformity in the receiver-side computational complexity for packet
processing, and thus are unlikely to pose a denial-of-service threat due
to the receipt of pathological data. Nor does the RTP payload format
contain any active content.</t>
</section>
<section anchor="congestionControl" title="Congestion Control">
<t>Congestion control for RTP SHALL be used in accordance with RFC 3550
<xref target="RFC3550"/>, and with any applicable RTP profile; e.g., RFC
3551 <xref target="RFC3551"/>. The congestion control mechanism can, in
a real-time encoding scenario, adapt the transmission rate by
instructing the encoder to encode at a certain target rate. Media aware
network elements MAY use the information in the VP9 payload descriptor
in <xref target="VP9payloadDescriptor"/> to identify non-reference
frames and discard them in order to reduce network congestion. Note that
discarding of non-reference frames cannot be done if the stream is
encrypted (because the non-reference marker is encrypted).</t>
</section>
<section anchor="IANAConsiderations" title="IANA Considerations">
<t>The IANA is requested to register the following values:<vspace
blankLines="0"/> - Media type registration as described in <xref
target="mediaTypeRegistration"/>.</t>
</section>
</middle>
<back>
<references>
&vp9;
&rfc2119;
&rfc4585;
&rfc3550;
&rfc3711;
&rfc4566;
&rfc6838;
&rfc4855;
&rfc3551;
</references>
</back>
</rfc>
<!-- LocalWords: PictureID DCT Hadamard WHT SSRC CSRC pyld hdr FI VER RPSI
-->
<!-- LocalWords: stPartitionSize SLI SDP AVPF SRTP IANA PID PICIDX TID
-->
| PAFTECH AB 2003-2026 | 2026-04-24 02:38:38 |