One document matched: draft-alvestrand-constraints-resolution-01.xml
<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="info" docName="draft-alvestrand-constraints-resolution-01"
ipr="trust200902">
<front>
<title abbrev="Resolution Constraints">Resolution Constraints in Web Real
Time Communications</title>
<author fullname="Harald Alvestrand" initials="H. T." surname="Alvestrand">
<organization>Google</organization>
<address>
<email>harald@alvestrand.no</email>
</address>
</author>
<date day="21" month="October" year="2012" />
<abstract>
<t>This document specifies the constraints necessary for a Javascript
application to successfully indicate to a browser that supports WebRTC
what resolutions it desires on a video stream.</t>
</abstract>
<note title="Requirements Language">
<t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in <xref
target="RFC2119">RFC 2119</xref>.</t>
</note>
</front>
<middle>
<section title="Introduction">
<t>There are a number of scenarios where it's useful for a WebRTC
application to indicate to the WebRTC implementation in the supported
browser what the desired characteristics of a video stream are. These
include, but are not limited to:</t>
<t><list style="symbols">
<t>Specifying a minimum desired resolution for a given application,
in order to control the user experience or resource tradeoffs made
by the browser to favour a particular stream</t>
<t>Specifying a maximum desired resolution for a given stream, in
order to save some resource (bandwidth, CPU....), possibly outside
of the browser where the browser can't tell that it's exceeding a
constraint</t>
<t>Specifying resolutions that are a reasonable fit for the current
usage of the video stream, for instance fitting with the number of
pixels available on the part of a device's display surface that is
devoted to displaying this video stream</t>
<t>Specifying the shape of a video stream, in order to fit the video
onto a display surface without the need for black bars or image
distortion</t>
</list>Similar considerations apply for framerate.</t>
<t></t>
<section title="Disposition of this text">
<t>This draft is written in order to get something specific out to
refer to during spec-writing and implementation. The text may
eventually get merged into the JSEP specification, <xref
target="I-D.ietf-rtcweb-jsep"></xref>.</t>
</section>
</section>
<section title="Usage Scenarios">
<t>These constraints are usable in several places:</t>
<t><list style="symbols">
<t>As constraints to the getUserMedia call <xref
target="W3C.WD-mediacapture-streams-20120628"></xref>, where they
serve to guide the configuration of the camera obtained, and may
influence the choice of camera.</t>
<t>As constraints to the addStream call on a PeerConnection <xref
target="W3C.WD-webrtc-20120821"></xref>, where they serve to guide
the configuration of the codec that encodes the video content for
transmission.</t>
<t>As constraints applied to an existing local video stream using
the "change constraints" API, where it may cause the video engine to
reconfigure the device or codec for that particular stream.</t>
<t>As constraints applied to an incoming video stream using the
"change constrains" API on a MediaStreamTrack, where it serves to
inform the video engine about the desirable properties of the video
track, which may lead to the video engine choosing to reencode the
video and/or signal a remote video source that it wishes certain
constraints to be put in place.</t>
</list>All of the constraints may be meaningful in both "mandatory"
and "optional" forms.</t>
<t>Consider the following (simplified) model of a video stream through a
WebRTC application:</t>
<figure>
<artwork><![CDATA[
|<-------------- Browser A -------------------->|
Camera ---> Mediastream A ---> Peerconnection A ------+
|<------- Application A ---------->| |
v ^ v
Signalling channel Internet (media)
v ^ |
|<------- Application B ---------->| |
<video> tag <-- Mediastream B <--- Peerconnection B --+
|<-------------Browser B ----------------------->|
]]></artwork>
</figure>
<t></t>
<t>Both applications are running in browsers, with Application A
connected to a camera that is able to deliver video streams up to HD
quality (1280x720).</t>
<t></t>
<section title="Scenario: Resolution change">
<t>At one particular moment in time, the <video> tag in
Application B is rendered as a thumbnail, and video is flowing to it
in a 160x100 resolution; there is no need to send any more data, since
no more pixels are available for its display anyway.</t>
<t>Then the user of Application B hits the "full-screen" button. There
are now 1600x1200 pixels available for display.</t>
<t>Initially, Application B will splay the 160x100 image across the
larger surface, because there is no other choice, but it will desire
to have as many pixels as possible available to provide a high quality
image.</t>
<t>The choices available to the writer of application B are:</t>
<t><list style="symbols">
<t>Signal (by non-standard means) to Application A that more
pixels are needed. Application A will then modify the constraints
on Mediastream A to say that the desired (not mandatory) min
resolution is 1600x1200; Browser A will then reconfigure the
camera to generate the closest available resolution, which is
1280x720.</t>
<t>Apply a new constraint set to Mediastream B's video track,
saying that the desired resolution is now 1600x1200. Browser B
will then have to figure out that this is an incoming track via
Peerconnection B, and that the resolution needs to be signalled;
it will then fire a NegotationNeeded event at Application B, which
will then renegotiate the desired resolutions using an SDP
exchange with Browser A; Browser A will then figure out from the
SDP that it's useful to generate a higher resolution video stream,
and reconfigure the camera as above.</t>
<t>Execute a renegotiation with Application A, adding
a=remote-ssrc: attributes as described in <xref
target="s-examples"></xref> by modifying the SDP generated by
CreateOffer, and triggering the behaviour in the previous
alternative inside Browser A. API-wise, this is perhaps the most
complex method.</t>
</list></t>
<t>The advantage of the first method is that it does not require any
SDP parsing or generation.</t>
<t>The advantage of the second method is that it will work when
appliation A and application B are different applications; there is no
need for them to have any private agreement on how to set bitrate.</t>
<t>In the opinion of the author, there are no obvious advantages to
the third method when the second method is available.</t>
</section>
<section title="Scenario: Constrained bandwidth">
<t>At one particular moment in time, the camera is generating
1280x720, resulting in a 2 Mbits/second data flow from A to B.
Congestion control signals that this data rate is no longer available;
rather than letting the browser reduce the bandwidth of some flow of
its choice, Application A decides that the high definition video is
the feature that is least valuable. It can then apply a new constraint
to Mediastream A, specifying that resolution should be at most
640x360; browser A is then responsible for making sure this decision
is communicated to browser B (if it needs to be).</t>
</section>
</section>
<section anchor="s-examples" title="Syntax and Mapping Examples">
<t>See <xref target="IANA"></xref> for the actual definition of the
constraints used here.</t>
<section title="Examples with GetUserMedia">
<t>A constraint saying that we absolutely must have a minimum
resolution of 1024x768:</t>
<figure>
<artwork><![CDATA[
getUserMedia({
video: { mandatory: { minWidth: 1024, minHeight: 768 } }
}, successCallback, errorCallback);
]]></artwork>
</figure>
<t>A constraint saying that we'd prefer 60 frames per second, if
available, and if we can get that, we'd like to limit the max
resolution, but in all cases, the screen must be clamped to a 4:3
aspect ratio - 16:9 or odd aspect ratios are not acceptable to this
application:</t>
<figure>
<artwork><![CDATA[
getUserMedia({
video: {
mandatory: { minAspectRatio: 1.333, maxAspectRatio: 1.334 },
optional [
{ minFrameRate: 60 },
{ maxWidth: 640 },
{ maxHeigth: 480 }
]
}
}, successCallback, errorCallback);
]]></artwork>
</figure>
<t></t>
</section>
<section title="SDP mappings">
<t>This document does not specify or constrain how constraints get
reflected into SDP (if they do); that's an implementor decision.</t>
<t>The examples below are thought exercises, based on <xref
target="I-D.lennox-mmusic-sdp-source-selection"></xref> and <xref
target="I-D.alvestrand-rtcweb-resolution"></xref>.</t>
<t>An optional constraint has been applied to an incoming stream where
both upper and lower are constrained to 320x200. The stream has been
assigned to a hardware video decoder that can decode most resolutions
up to 1024x768, in any aspect ratio, but only if all divisions are
divisible by 4. The incoming stream has SSRC 1234.</t>
<t>Escaped line breaks are added for readability.</t>
<figure>
<artwork><![CDATA[
m=video
a=remote-ssrc:1234 imageattr:* [x=320,y=200,q=1.0] \
[x=[120:4:1024],y=[100:4:768],q=0.2]
]]></artwork>
</figure>
<t></t>
</section>
</section>
<section anchor="IANA" title="IANA Considerations">
<t>This document requests IANA to register constraints in the "RTCWeb
Media Constraints" registry created by <xref
target="I-D.burnett-rtcweb-constraints-registry"></xref>. NOTE: The
registrations assume that this document is updated to no longer have
"video" as part of the name, but have "video" as a field-of-use in the
registration.</t>
<t>The definitions of width, height and aspect ratio are taken from
<xref target="RFC6236"></xref>.</t>
<t><list style="symbols">
<t>minWidth - valid for video. Corresponds to the "x" value (pixel
count) from RFC 6236. Only integer values are valid.</t>
<t>maxWidth - valid for video. Definition as for minWidth.</t>
<t>minHeight - valid for video. Corresponds to the "y" value (pixel
count) from RFC 6236. Only integer values are valid.</t>
<t>maxHeight - valid for video. Definition as for minHeight.</t>
<t>minAspectRatio - valid for video. Corresponds to the "par"
(picture aspect ratio), with "sar" set to 1.0. A 4:3 format display
corresponds to an AspectRatio of 1.3333. Floating point values are
valid.</t>
<t>maxAspectRatio - valid for video. Definition as for
minAspectRatio.</t>
<t>minFramerate - valid for video. Corresponds to the framerate
defined in <xref target="RFC4566"></xref>, the "a=framerate"
attribute.</t>
<t>maxFramerate - valid for video. Definition as for
minFramerate.</t>
</list></t>
<t>The contact person is Harald Alvestrand <hta@google.com>.</t>
<t>Change control for the registration is with the IETF, as designated
by the IESG.</t>
<t>Note that minFramerate defines a lower bound for the a=framerate
attribute, which is itself defined as an upper limit; this means that
even if a high framerate is negotiated, the actual framerate used may be
lower due to temporary considerations (for instance CPU or bandwidth, or
simply lack of movement in the picture).</t>
</section>
<section anchor="Security" title="Security Considerations">
<t>No security considerations particular to these specific constraints
have so far been identified.</t>
</section>
<section anchor="Acknowledgements" title="Acknowledgements">
<t>Special thanks are given to Dan Burnett, Cullen Jennings, the IETF
RTCWEB WG and the W3C WEBRTC WG for strongly influencing this memo, and
to Per Kjellander for being the first to implement the constraints in
getUserMedia.</t>
</section>
</middle>
<back>
<references title="Normative References">
<?rfc include="reference.RFC.2119"?>
<?rfc include='reference.RFC.4566'?>
<?rfc include='reference.RFC.6236'?>
<?rfc include='reference.I-D.burnett-rtcweb-constraints-registry'?>
</references>
<references title="Informative References">
<?rfc include='reference.I-D.lennox-mmusic-sdp-source-selection'?>
<?rfc include='reference.I-D.alvestrand-rtcweb-resolution'?>
<?rfc include='reference.I-D.ietf-rtcweb-jsep'?>
<?rfc include='reference.W3C.WD-mediacapture-streams-20120628'?>
<?rfc include='reference.W3C.WD-webrtc-20120821'?>
</references>
<section title="Changes from -00 to -01">
<t>Added the "Usage Scenarios" chapter.</t>
<t>Repointed the eventual target to be incorporation in the JSEP
draft.</t>
<t>Made sure the constraints are consistently spelled in camelCase, with
a small initial letter.</t>
</section>
</back>
</rfc>
| PAFTECH AB 2003-2026 | 2026-04-24 02:56:52 |