One document matched: draft-basso-avt-videoconreq-01.txt
Differences from draft-basso-avt-videoconreq-00.txt
Audio Video Transport Group
Internet Draft A. Basso
Document: draft-basso-avt-videoconreq-01.txt NMS Communications
O.Levin
Microsoft
N. Ismail
Cisco Systems
Expires: January 2005 July 2004
Requirements for transport of video control commands
By submitting this Internet-Draft, I certify that any applicable
patent or other IPR claims of which I am aware have been
disclosed, or will be disclosed, and any of which I become aware
will be disclosed, in accordance with RFC 3668.
Status of this Memo
This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026 [1].
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet-
Drafts.
Internet-Drafts are draft documents valid for a maximum of six
months and may be updated, replaced, or obsoleted by other
documents at any time. It is inappropriate to use Internet-Drafts
as reference material or to cite them other than as "work in
progress."
The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
basso Expires January 2005 [Page 1]
Codec Control Requirements 4
WhatÆs new from version û00
1. Added correct boilerplate text.
2. Sec. 3: Clarification of terminology.
3. Sec. 6 : clarification the reference to IETF protocols only.
4. Harmonization with H.241.
Abstract
A variety of video communication services such as video
conferencing and video messaging rely on the capability of video
encoders and decoders to respond to control commands. This
document outlines this set of commands as well as the requirements
for their transport.
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in
RFC-2119 [2].
Table of Contents
1. Introduction...................................................3
2. Background.....................................................4
3. Video coding...................................................4
4. Use Cases......................................................4
5. Codec Commands.................................................5
5.1 Decoder Control Commands...................................6
5.2 Encoder Control Commands...................................6
6. General requirements...........................................7
6.1 Reuse of Existing Protocols................................7
6.2 Maintain Existing Protocol Integrity.......................8
6.3 Avoid Duplicating Existing Protocols.......................8
6.4 Efficiency.................................................8
7. Codec Control Requirements.....................................8
7.1 Reliable versus unreliable delivery........................8
7.2 Capability description.....................................8
7.3 Relation with media session................................9
7.4 Relation with signaling....................................9
7.5 Bidirectional transport....................................9
7.6 Extensibility..............................................9
basso Expires January 2005 [Page 2]
Codec Control Requirements 4
7.7 Unicast and Multicast Support..............................9
7.8 Interoperability with other protocols......................9
7.9 Timely delivery...........................................10
8. Security Considerations.......................................10
9. Acknowledgments...............................................10
10. Copyright Notice.............................................10
References.......................................................10
Author's Addresses...............................................11
1. Introduction
A variety of video communication services such as video
conferencing and video messaging rely on the capability of video
encoders and decoders to respond to control commands. This
document outlines a generic set of commands applicable to a
variety of video codecs as well as the requirements for the
transport of such commands.
2. Background
RTP [6] is the protocol of choice for the delivery of real time
media. RTCP, the companion control protocol, allows some form of
monitoring of the media delivery. An enhanced RTCP feedback scheme
enabling a generic decoder to provide hints to the corresponding
encoder in case of network losses has been described in [6].
Similar solutions were provided for specific coding schemes such
ad H.261 [3] H.263 [4] and MPEG-4 [5].
Currently, there is no standard IETF protocol support that allows a
given application to exchange control commands with a given codec.
3. Video coding
In current coding schemes such as H.261 [2], H.263 [3], MPEG-1,
2,4 [5], H.264 [6] pictures can be coded as intra o predicted pictures.
Furthermore pictures can be used as references in the decoding process or not.
More precisely, intra pictures are pictures that can be decoded
without first decoding any other picture. Predicted (or non-intra)
pictures may require data from one or more previously-decoded
pictures in order to be decoded. A reference picture is a picture
that is stored in the decoder for use as a reference in the
decoding process of some subsequent picture in the video
bitstream. Finally a non-reference picture is a picture that is
not used as a reference for the decoding process of any other
picture in the bitstream. The concepts of intra versus non-intra
and reference versus non-reference are independent. A particular
basso Expires January 2005 [Page 3]
Codec Control Requirements 4
picture can in general be any one of the four types, intra
reference, non-intra reference, intra non-reference, non-intra
non-reference.
Furthermore video pictures are not coded as a whole but are
partitioned in small blocks called macrobolocks (MB) and every MB
is individually coded. MBs are grouped together in sets of
variable size. Such sets are called, in dependence of the coding
standard, slices or Group of Blocks (GoBs).Such sets of MB can be
scattered in the picture.
4. Use Cases
This section describes use cases of codec control commands.
1. A use case includes an RTP video mixer composing multiple
encoded video sources into a single encoded video stream. Each
time a video source is to be added to the video composition, the
RTP mixer needs to request an encoded reference picture from the
video source or a specific area of the picture defined by one or
more slices.
2. Another use case includes an RTP video mixer that receives
multiple encoded RTP video streams from conference participants
and dynamically selects one of the streams to be included in its
output RTP stream. For every new video stream selected, the mixer
will request a intra picture from the remote source in order for
the receiving endpoints to be able to decode and display the
output stream smoothly when the switch occurs. The video mixer in
this scenario will stop the delivery of the current RTP stream and
it will wait for the intra picture from the source before it
switches to that source.
3. Another use case includes a given application that needs to
signal to the remote encoder a request of change in the coding
strategy asking to deliver video pictures at a lower frame rate
but with better picture quality or vice versa. Such requests may
be based on input from the end user.
4. Another use case includes an application that has became aware
of packet losses and in order to mitigate their effect requests an
intra picture from the remote encoder. This will stop the
spatial and temporal propagation of coding errors inherent to
commonly used predictive video coding schemes. It is also possible
to obtain random access recovery without a fast update. This is
basso Expires January 2005 [Page 4]
Codec Control Requirements 4
sometimes called "gradual decoder refresh". See for example the
recovery point SEI message in H.264/AVC [6].
5. Another use case includes a video mixer that switches its
output stream to a new video source. The video mixer will instruct
the receiving endpoints by means of a codec control command to
complete the decoding of the current picture and then wait for a
new video reference picture. Concurrently, the video mixer
requests a reference picture from the new video source and
immediately switches to the new source. Once the new source
receives the request for the reference picture and acts on it, the
receiving endpoints will restart decoding and displaying the new
picture.
The main benefit of this method as opposed to the video mixer
stopping video transmission of the new source until it detects a
new reference picture, as in use case 2, is that the video mixer
does not have to discover the beginning of a reference picture.
This can simplify the video mixer task especially in the case in
which the picture has multiple reference pictures.
6. Another use case includes a video mixer that dynamically
selects one of the received video streams to be sent out to
participants and tries to provide the highest bit rate possible to
all participants while minimizing stream transrating. One way of
achieving this is for the mixer to setup sessions with endpoints
using the maximum bit rate accepted by that endpoint and by the
call admission method used by the mixer.
By means of commands that allow flow control, the mixer can then
reduce the maximum bit rate sent by endpoints to the lowest common
denominator of all received streams. As the lowest common
denominator changes due to endpoints joining or leaving, the mixer
can adjust the limits to which endpoints can send their streams to
match the new limit.
The mixer then would request a new maximum bit rate, which is
equal or less than the maximum bit-rate negotiated at session
setup, for a specific media stream, and the remote endpoint can
respond with the actual bit-rate that it can support.
5. Codec Commands
5.1 Decoder Control Commands
1. VideoFreezePicture
It instructs the video decoder to complete the decoding of the
current video picture and subsequently display it until a timeout
period is elapsed or the receipt of a message that indicates the
release of the frozen picture and resume normal decoding and
basso Expires January 2005 [Page 5]
Codec Control Requirements 4
presentation. Note that the freeze picture release command is part
of the H.261, H.262, H.263 and H.264 bitstreams. Coding schemes
that support picture freeze release in their bitstreams, MUST use
freeze release to signal the remote end to resume decoding.
H.264 specifies a timeout period of at least 6 seconds from the
receipt of the VideoFreezePicture. See use case 5 for an example
of how such command might be used.
5.2 Encoder Control Commands
1. videoFastUpdatePicture
A "fast update", also known as an "instantaneous decoder refresh",
involves sending an intra picture to a decoder and thereafter
refraining from using any picture sent prior to that intra picture
as a reference for the decoding process of any subsequent picture
sent in the stream.
The videoFastUpdatePicture command instructs the video encoder to
complete the encoding of the current video picture and to generate
a full intra picture at the earliest opportunity. The evaluation
of such opportunity includes the current encoder coding strategy
and the current available network resources. An H.264 encoder can
react to a VideoFastUpdatePicture command with an IDR procedure or
a gradual recovery procedure as specified in [10]
Intra pictures, independently from the instant in time when they
are encoded, are in general several times larger in size than
predicted pictures. Thus in scenarios in which the available
bandwidth is small the use of a intra picture implies a delay that
is significantly longer than the typical picture duration.
2. VideoTemporalSpatialTradeOff(index)
It instructs the video encoder to change its trade-off between
temporal and spatial resolution. Index assumes values from O to 31
to indicate monotonically a desire for higher frame rate.
In general the encoder reaction time may be significantly longer
than the typical picture duration.
3. RateRequest(MaxBitrate)
It instructs the far-end encoder to change the maximum bit rate of
the given media stream being transmitted. MaxBitRate indicates, in
units of 100 bit/s, the new requested maximum bit rate for the
associated media stream. The new requested bit rate has to be
equal to or less than the bit rate negotiated during session
setup.
basso Expires January 2005 [Page 6]
Codec Control Requirements 4
4. RateNotify(MaximumBitRate)
This message is sent as a response of a RateRequest message.
MaximumBitRate indicates, in units of 100 bit/s, the maximum bit
rate for the media stream at which the terminal is going to encode
the media stream. Note that MaximumBitRate may differ from the
requested MaxBitrate.
6. General requirements
6.1 Reuse of Existing Protocols
The codec control messages should be transported using an already
existing transport protocol whenever possible. The transport
protocol should allow at a minimum the leveraging of its security
elements.
6.2 Maintain Existing Protocol Integrity
In meeting the requirement of Section 7, the codec control
transport mechanism MUST NOT break existing protocols or cause
backward compatibility problems.
6.3 Avoid Duplicating Existing Protocols
The codec control mechanism SHOULD NOT duplicate the functionality
of existing IETF protocols. The focus of codec control is new
functionality not addressed by existing IETF protocols or
extending existing IETF protocols within the structures of the
requirement in Section 7. Where an existing IETF protocol can be
gracefully extended to support codec control requirements, such
extensions are acceptable alternatives for meeting the
requirements.
6.4 Efficiency
The codec control transport mechanism SHOULD employ protocol
elements known to result in efficient operation. Techniques to be
considered include re-use of transport connections across sessions
i.e. codec control messages that controls different media sessions
may be aggregated on one codec control transport channel and
piggybacking of responses on requests in the reverse direction
basso Expires January 2005 [Page 7]
Codec Control Requirements 4
7. Codec Control Requirements
7.1 Reliable versus unreliable delivery
The commands VideoPictureFreeze and VideoTemporalSpatialTradeOff
and the commands relative to flow control RateRequest, RateNotify
require a reliable delivery.
The command videoFastUpdatePicture implies a specific modification
of the media, which is delivered in an unreliable fashion. Given
that the delivery of the media is unreliable the sender cannot
rely on the fact that the request has been safely delivered but
needs to assure that the requested modification of the data (i.e.,
insertion of a reference picture) is received.
7.2 Capability description
The capability of codec control for each supported message should
be described and negotiated, for example using SDP offer/answer,
for both senders and receivers during session setup. The transport
protocol used for the delivery of codec control messages should
also be specified as of session setup.
7.3 Relation with media session
The delivery channel of the codec control messages must be
associated with the media session it controls. Using one codec
control channel per media session and associating the two channels
during session setup could achieve this purpose. Alternatively one
media control channel could be used for multiple media sessions.
In this case the controlled media session MUST be identified in
each codec control message.
The transport channel of the codec control messages should follow
a similar path to that of the media session it controls.
Inter-operability with other standards for codec control delivery
might cause a deviation from this requirement.
7.4 Relation with signaling
The codec control transport protocol MUST be independent of the
signaling protocol used to setup the media.
7.5 Bidirectional transport
Messages can be originated from receivers as well as a senders
thus the transport mechanism must allow bi-directional exchange of
messages.
basso Expires January 2005 [Page 8]
Codec Control Requirements 4
7.6 Extensibility
Codec control message syntax should be extensible to easily
support the addition of new control messages.
7.7 Unicast and Multicast Support
The codec control transport MUST work and scale for media sessions
that use point-to-point unicast.
The codec control transport MUST work and scale for media sessions
that use SSM (Source Specific Multicast) and has a small to
moderate group size.
The codec control transport will not address ASM (Any Source
Multicast) media sessions in which media sources are not known
until they start transmission.
7.8 Interoperability with other protocols
The codec control transport protocol MUST allow inter-operability
with the most commonly deployed IP-based video communication
protocols, such as H.323, H.324 and H.324M.
7.9 Timely delivery
For some video services the ability to transmit codec control
commands in a timely fashion is essential to the delivery of a
high quality user experience. The delay introduced by transport
protocol MUST be negligible with respect of the time constants of
the delivered media stream.
8. Security Considerations
<TODO>
9. Acknowledgments
The authors would like to acknowledge the comments from around the
community in helping refine this document. Particular recognition
goes to Roni Even.
10. Copyright Notice
Copyright (C) The Internet Society (2004). This document is
subject to the rights, licenses and restrictions contained in BCP
78, and except as set forth therein, the authors retain all their
rights.
basso Expires January 2005 [Page 9]
Codec Control Requirements 4
This document and the information contained herein are provided on
an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE
REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT
THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR
ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A
PARTICULAR PURPOSE.
References
1 Bradner, S., "The Internet Standards Process -- Revision 3",
BCP 9, RFC 2026, October 1996.
2 Bradner, S., "Key words for use in RFCs to Indicate Requirement
Levels", BCP 14, RFC 2119, March 1997
3 ITU-T Recommendation H.261 (1993), Video codec for audiovisual
services at p . 64 kbit/s.
4 ITU-T Recommendation H.263 (1998), Video coding for low bit rate
communication.
5 ISO/IEC 14496-2:2001/Amd.1:2002, "Information technology -
Coding of audio-visual objects - Part2: Visual", 2001.
6 Joint Video Team of ITU-T and ISO/IEC JTC 1, ôDraft ITU-T
Recommendation and Final Draft International Standard of Joint
Video Specification (ITU-T Rec. H.264 | ISO/IEC 14496-10 AVC),ö
Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, JVT-
G050, March 2003.
7 J. Ott et al., Extended RTP Profile for RTCP-based Feedback
(RTP/AVPF), draft-ietf-avt-rtcp-feedback-04.txt, June 2002,
IETF Draft. Work in progress.
8 T. Turletti and C. Huitema, "RTP Payload Format for H.261 Video
Streams, RFC 2032, October 1996.
9 H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP û
A Transport Protocol for Real-time Applications", Internet
basso Expires January 2005 [Page 10]
Codec Control Requirements 4
Draft, draft-ietf-avt-rtp-new-11.txt, Work in Progress,
November 2001.
10 ITU-T Recommendation H.241 (07/2003), Extended video procedures
and control signals for H.300-series terminals
Author's Addresses
Andrea Basso
NMS Communications
200 Shultz Drive
Red Bank, NJ 07701, USA
Phone: (732) 936-2118
Email: andrea_basso@nmss.com
Orit Levin
Microsoft Corporation
One Microsoft Way
Redmond, WA 98052, USA
EMail: oritl@microsoft.com
Nermeen Ismail
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706, USA
Phone: +1 408 853 8714
Email: nismail@cisco.com
basso Expires January 2005 [Page 11]
| PAFTECH AB 2003-2026 | 2026-04-23 16:27:05 |