One document matched: draft-basso-avt-videoconreq-02.txt
Differences from draft-basso-avt-videoconreq-01.txt
Audio Video Transport Group
Internet Draft A. Basso
Document: draft-basso-avt-videoconreq-02.txt NMS Communications
O.Levin
Microsoft
N. Ismail
Cisco Systems
Expires: April 2005 October 2004
Requirements for transport of video control commands
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of RFC 3667 [2].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of RFC 3668.
Abstract
A variety of video communication services such as video conferencing
and video messaging rely on the capability of video encoders and
decoders to respond to control commands. This document outlines this
set of commands as well as the requirements for their transport.
basso Expires April 2005 [Page 1]
Codec Control Requirements October 2004
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [1].
Table of Contents
1. Introduction...................................................3
2. Background.....................................................3
3. Video coding...................................................3
4. Use Cases......................................................4
5. Codec Commands.................................................5
5.1 Decoder Control Commands...................................5
5.2 Encoder Control Commands...................................6
6. General requirements...........................................6
6.1 Reuse of Existing Protocols................................7
6.2 Maintain Existing Protocol Integrity.......................7
6.3 Avoid Duplicating Existing Protocols.......................7
6.4 Efficiency.................................................7
7. Codec Control Requirements.....................................7
7.1 Reliable and Unreliable Delivery...........................7
7.2 Transport alternatives.....................................8
7.3 Capability description.....................................8
7.4 Relation with media session................................8
7.5 Bidirectional transport....................................8
7.6 Extensibility..............................................8
7.7 Unicast and Multicast Support..............................8
7.8 Timely delivery............................................9
8. Security Considerations........................................9
9. IANA Considerations............................................9
10. Acknowledgments...............................................9
11. Copyright Notice..............................................9
12. Informative References.......................................10
13. IPR Notices..................................................10
Author's Addresses...............................................11
What is new from version 01
1. Document updated to be conformant to Guidelines to Authors of
Internet-Drafts
2. Section 5.2.4 RateNotify command removed.
3. Section 7.1 Clarified Reliable versus unreliable delivery.
4. Added Section 7.2 Transport alternatives
5. Section 7.4 Relation with signaling. Removed
6. Section 9.8 Interoperability with other protocols. Removed
7. Section 9.9 MUST has been changed to SHOULD.
Basso Expires April 2005 [Page 2]
Codec Control Requirements October 2004
8. Updated references
What is new from version 00
1. Added boilerplate text.
2. Sec. 3: Clarification of terminology.
3. Sec. 6 : clarification the reference to IETF protocols only.
4. Harmonization with H.241.
1. Introduction
A variety of video communication services such as video conferencing
and video messaging rely on the capability of video encoders and
decoders to respond to control commands. This document outlines a
generic set of commands applicable to a variety of video codecs as
well as the requirements for their transport.
2. Background
RTP [9] is the protocol of choice for the delivery of real time
media. RTCP, the companion control protocol, allows some form of
monitoring of the media delivery. An enhanced RTCP feedback scheme
enabling a generic decoder to provide hints to the corresponding
encoder in case of network losses has been described in [7]. Similar
solutions were provided for specific coding schemes such ad H.261 [3]
H.263 [4] and MPEG-4 [5].
Currently, there is no standard protocol support that allows a given
application to exchange control commands with a given codec.
3. Video coding
In current coding schemes such as H.261 [2], H.263 [3], MPEG-1, 2,4
[5], H.264 [6] pictures can be coded with various modalities i.e.
intra o predicted pictures. Furthermore pictures can be used as
references in the decoding process or not.
More precisely, intra pictures are pictures that can be decoded
without first decoding any other picture. Predicted (or non-intra)
pictures may require data from one or more previously decoded
pictures in order to be decoded. A reference picture is a picture
that is stored in the decoder for use as a reference in the decoding
process of some subsequent picture in the video bitstream. Finally a
non-reference picture is a picture that is not used as a reference
for the decoding process of any other picture in the bitstream. The
concepts of intra versus non-intra and reference versus non-reference
Basso Expires April 2005 [Page 3]
Codec Control Requirements October 2004
are independent. A particular picture can in general be any one of
the four types, intra reference, non-intra reference, intra
non-reference, non-intra non-reference.
Furthermore video pictures are not coded as a whole but are
partitioned in small blocks called macrobolocks (MB) and every MB is
individually coded. MBs are grouped together in sets of variable
size. Such sets are called, in dependence of the coding standard,
slices or Group of Blocks (GoBs).Such sets of MB can be scattered in
the picture.
4. Use Cases
This section describes use cases of codec control commands.
1. A use case includes an RTP video mixer composing multiple encoded
video sources into a single encoded video stream. Each time a video
source is to be added to the video composition, the RTP mixer needs
to request an encoded reference picture from the video source or a
specific area of the picture defined by one or more slices.
2. Another use case includes an RTP video mixer that receives
multiple encoded RTP video streams from conference participants and
dynamically selects one of the streams to be included in its output
RTP stream. For every new video stream selected, the mixer will
request a intra picture from the remote source in order for the
receiving endpoints to be able to decode and display the output
stream smoothly when the switch occurs. The video mixer in this
scenario will stop the delivery of the current RTP stream and it will
wait for the intra picture from the source before it switches to that
source.
3. Another use case includes a given application that needs to signal
to the remote encoder a request of change in the coding strategy
asking to deliver video pictures at a lower frame rate but with
better picture quality or vice versa. Such requests may be based on
input from the end user.
4. Another use case includes an application that has became aware of
packet losses and in order to mitigate their effect requests an intra
picture from the remote encoder. This will stop the spatial and
temporal propagation of coding errors inherent to commonly used
predictive video coding schemes. It is also possible to obtain random
access recovery without a fast update. This is sometimes called
"gradual decoder refresh". See for example the recovery point SEI
message in H.264/AVC [6].
5. Another use case includes a video mixer that switches its output
stream to a new video source. The video mixer will instruct the
Basso Expires April 2005 [Page 4]
Codec Control Requirements October 2004
receiving endpoints by means of a codec control command to complete
the decoding of the current picture and then wait for a new video
reference picture. Concurrently, the video mixer requests a reference
picture from the new video source and immediately switches to the new
source. Once the new source receives the request for the reference
picture and acts on it, the receiving endpoints will restart decoding
and displaying the new picture.
The main benefit of this method as opposed to the video mixer
stopping video transmission of the new source until it detects a new
reference picture, as in use case 2, is that the video mixer does not
have to discover the beginning of a reference picture. This can
simplify the video mixer task especially in the case in which the
picture has multiple reference pictures.
6. Another use case includes a video mixer that dynamically selects
one of the received video streams to be sent out to participants and
tries to provide the highest bit rate possible to all participants
while minimizing stream transrating. One way of achieving this is for
the mixer to setup sessions with endpoints using the maximum bit rate
accepted by that endpoint and by the call admission method used by
the mixer.
By means of commands that allow flow control, the mixer can then
reduce the maximum bit rate sent by endpoints to the lowest common
denominator of all received streams. As the lowest common denominator
changes due to endpoints joining or leaving, the mixer can adjust the
limits to which endpoints can send their streams to match the new
limit.
The mixer then would request a new maximum bit rate, which is equal
or less than the maximum bit-rate negotiated at session setup, for a
specific media stream, and the remote endpoint can respond with the
actual bit-rate that it can support.
5. Codec Commands
The ensemble of commands described in this section is divided into
two sets. The first set includes commands that are sent to decoders
typically to control the presentation of the content. The second set
includes commands that are sent to remote encoders.
5.1 Decoder Control Commands
1. VideoFreezePicture
It instructs the video decoder to complete the decoding of the
current video picture and subsequently display it until a timeout
period is elapsed or the receipt of a message that indicates the
release of the frozen picture and resume normal decoding and
presentation. Note that the freeze picture release command is part of
Basso Expires April 2005 [Page 5]
Codec Control Requirements October 2004
the H.261, H.262, H.263 and H.264 bitstreams. Coding schemes that
support picture freeze release in their bitstreams, MUST use freeze
release to signal the remote end to resume decoding.
H.264 specifies a timeout period of at least 6 seconds from the
receipt of the VideoFreezePicture. See use case 5 for an example of
how such command might be used.
5.2 Encoder Control Commands
1. videoFastUpdatePicture
A "fast update", also known as an "instantaneous decoder refresh",
involves sending an intra picture to a decoder and thereafter
refraining from using any picture sent prior to that intra picture as
a reference for the decoding process of any subsequent picture sent
in the stream.
The videoFastUpdatePicture command instructs the video encoder to
complete the encoding of the current video picture and to generate a
full intra picture at the earliest opportunity. The evaluation of
such opportunity includes the current encoder coding strategy and the
current available network resources. An H.264 encoder can react to a
VideoFastUpdatePicture command with an IDR procedure or a gradual
recovery procedure as specified in [10]
Intra pictures, independently from the instant in time when they are
encoded, are in general several times larger in size than predicted
pictures. Thus in scenarios in which the available bandwidth is
small the use of a intra picture implies a delay that is
significantly longer than the typical picture duration.
2. VideoTemporalSpatialTradeOff(index)
It instructs the video encoder to change its trade-off between
temporal and spatial resolution. Index assumes values from O to 31 to
indicate monotonically a desire for higher frame rate.
In general the encoder reaction time may be significantly longer than
the typical picture duration.
3. RateRequest(MaxBitrate)
It instructs the far-end encoder to change the maximum bit rate of
the given media stream being transmitted. MaxBitRate indicates, in
units of 100 bit/s, the new requested maximum bit rate for the
associated media stream. The new requested bit rate has to be equal
to or less than the bit rate negotiated during session setup.
6. General requirements
Basso Expires April 2005 [Page 6]
Codec Control Requirements October 2004
6.1 Reuse of Existing Protocols
The codec control messages should be transported using an already
existing transport protocol whenever possible. The transport protocol
should allow at a minimum the leveraging of its security elements.
6.2 Maintain Existing Protocol Integrity
In meeting the requirement of Section 7, the codec control transport
mechanism MUST NOT break existing protocols or cause backward
compatibility problems.
6.3 Avoid Duplicating Existing Protocols
The codec control mechanism SHOULD NOT duplicate the functionality of
existing IETF protocols. The focus of codec control is new
functionality not addressed by existing IETF protocols or extending
existing IETF protocols within the structures of the requirement in
Section 7. Where an existing IETF protocol can be gracefully
extended to support codec control requirements, such extensions are
acceptable alternatives for meeting the requirements.
6.4 Efficiency
The codec control transport mechanism SHOULD employ protocol elements
known to result in efficient operation. Techniques to be considered
include re-use of transport connections across sessions i.e. codec
control messages that controls different media sessions may be
aggregated on one codec control transport channel and piggybacking of
responses on requests in the reverse direction
7. Codec Control Requirements
7.1 Reliable and Unreliable Delivery.
The commands VideoPictureFreeze and VideoTemporalSpatialTradeOff and
the commands relative to flow control as RateRequest require a
reliable delivery.
The command videoFastUpdatePicture implies a specific modification of
the media, which is delivered in an unreliable fashion. Given that
the delivery of the media is unreliable, the sender cannot rely on
the fact that the request has been safely delivered but needs to
assure that the requested modification of the data (i.e., insertion
of a reference picture) is received before taking any action. Thus
the receiver has always to "observe" the incoming data for the
requested change independently of the method of delivery of the
videoFastUpdatePicture command. VideoFastUpdatePicture can be thus
Basso Expires April 2005 [Page 7]
Codec Control Requirements October 2004
delivered over an unreliable channel. If the expected change in the
media does not happen the command will be retransmitted.
7.2 Transport alternatives
Commands such VideoTemporalSpatialTradeOff and RateRequest relative
to flow control can be interpreted as changes of a given presentation
description and potentially carried via existing protocols such SDP.
This is not the case of the VideoFastUpdatePicture and
VideoPictureFreeze commands.
7.3 Capability description
The capability of codec control for each supported message should be
described and negotiated, for example using SDP offer/answer, for
both senders and receivers during session setup. The transport
protocol used for the delivery of codec control messages should also
be specified as of session setup.
7.4 Relation with media session
The delivery channel of the codec control messages must be associated
with the media session it controls. Using one codec control channel
per media session and associating the two channels during session
setup could achieve this purpose. Alternatively one media control
channel could be used for multiple media sessions. In this case the
controlled media session MUST be identified in each codec control
message.
The transport channel of the codec control messages should follow a
similar path to that of the media session it controls.
Inter-operability with other standards for codec control delivery
might cause a deviation from this requirement.
7.5 Bidirectional transport
Messages can be originated from receivers as well as a senders thus
the transport mechanism must allow bi-directional exchange of
messages.
7.6 Extensibility
Codec control message syntax should be extensible to easily support
the addition of new control messages.
7.7 Unicast and Multicast Support
Basso Expires April 2005 [Page 8]
Codec Control Requirements October 2004
The codec control transport MUST work and scale for media sessions
that use point-to-point unicast.
The codec control transport MUST work and scale for media sessions
that use SSM (Source Specific Multicast) and has a small to moderate
group size.
The codec control transport will not address ASM (Any Source
Multicast) media sessions in which media sources are not known until
they start transmission.
7.8 Timely delivery
For some video services the ability to transmit codec control
commands in a timely fashion is essential to the delivery of a high
quality user experience. The delay introduced by the transport
protocol SHOULD be negligible with respect of the time constants of
the delivered media stream.
8. Security Considerations
<TODO>
9. IANA Considerations
<TODO>
10. Acknowledgments
The authors would like to acknowledge the comments from around the
Community in helping refine this document. Particular recognition
goes to Roni Evens.
11. Copyright Notice
Copyright (C) The Internet Society (2004). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Basso Expires April 2005 [Page 9]
Codec Control Requirements October 2004
12. Informative References
1 S. Bradner, "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997
2 S. Bradner "IETF Rights in Contributions" RFC 3667 February 2004
3 ITU-T Recommendation H.261 (1993), Video codec for audiovisual
services at p . 64 kbit/s.
4 ITU-T Recommendation H.263 (1998), Video coding for low bit rate
communication.
5 ISO/IEC 14496-2:2001/Amd.1:2002, "Information technology -
Coding of audio-visual objects - Part2: Visual", 2001.
6 Joint Video Team of ITU-T and ISO/IEC JTC 1, Draft ITU-T
Recommendation and Final Draft International Standard of Joint
Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC),
Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-G050,
March 2003.
7 J. Ott et al. Extended RTP Profile for RTCP-based Feedback
(RTP/AVPF), draft-ietf-avt-rtcp-feedback-09.txt, August 2004, IETF
Draft. Work in progress.
8 T. Turletti and C. Huitema, "RTP Payload Format for H.261 Video
Streams, RFC 2032, October 1996.
9 H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: A
Transport Protocol for Real-time Applications", RFC3550 STD 64
July 2003.
10 ITU-T Recommendation H.241 (07/2003), Extended video procedures
and control signals for H.300-series terminals.
11 S. Bradner "Intellectual Property rights in IETF Technology"
RFC3668, February 2004
13. IPR Notices
The IETF takes no position regarding the validity or scope of any
Basso Expires April 2005 [Page 10]
Codec Control Requirements October 2004
Intellectual Property Rights or other rights that might be claimed
to pertain to the implementation or use of the technology
described in this document or the extent to which any license
under such rights might or might not be available; nor does it
represent that it has made any independent effort to identify any
such rights. Information on the procedures with respect to rights
in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use
of such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository
at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention
any copyrights, patents or patent applications, or other
proprietary rights that may cover technology that may be required
to implement this standard. Please address the information to the
IETF at ietf-ipr@ietf.org.
Author's Addresses
Andrea Basso
NMS Communications
200 Shultz Drive
Red Bank, NJ 07701 USA
Phone: (732) 936-2118
Email: andrea_basso@nmss.com
Orit Levin
Microsoft Corporation
One Microsoft Way
Redmond, WA 98052 USA
EMail: oritl@microsoft.com
Nermeen Ismail
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706, USA
Phone: +1 408 853 8714
Email: nismail@cisco.com
Basso Expires April 2005 [Page 11]
| PAFTECH AB 2003-2026 | 2026-04-23 10:57:54 |