One document matched: draft-alvestrand-constraints-resolution-00.xml

<?xml version="1.0" encoding="US-ASCII"?>
<!DOCTYPE rfc SYSTEM "rfc2629.dtd">
<?rfc toc="yes"?>
<?rfc tocompact="yes"?>
<?rfc tocdepth="3"?>
<?rfc tocindent="yes"?>
<?rfc symrefs="yes"?>
<?rfc sortrefs="yes"?>
<?rfc comments="yes"?>
<?rfc inline="yes"?>
<?rfc compact="yes"?>
<?rfc subcompact="no"?>
<rfc category="info" docName="draft-alvestrand-constraints-resolution-00"
     ipr="trust200902">
  <front>
    <title abbrev="Resolution Constraints">Resolution Constraints in Web Real
    Time Communications</title>

    <author fullname="Harald Alvestrand" initials="H. T." surname="Alvestrand">
      <organization>Google</organization>

      <address>
        <email>harald@alvestrand.no</email>
      </address>
    </author>

    <date day="26" month="August" year="2012" />

    <abstract>
      <t>This document specifies the constraints necessary for a Javascript
      application to successfully indicate to a browser that supports WebRTC
      what resolutions it desires on a video stream.</t>
    </abstract>

    <note title="Requirements Language">
      <t>The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
      "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
      document are to be interpreted as described in <xref
      target="RFC2119">RFC 2119</xref>.</t>
    </note>
  </front>

  <middle>
    <section title="Introduction">
      <t>There are a number of scenarios where it's useful for a WebRTC
      application to indicate to the WebRTC implementation in the supported
      browser what the desired characteristics of a video stream are. These
      include, but are not limited to:</t>

      <t><list style="symbols">
          <t>Specifying a minimum desired resolution for a given application,
          in order to control the user experience or resource tradeoffs made
          by the browser to favour a particular stream</t>

          <t>Specifying a maximum desired resolution for a given stream, in
          order to save some resource (bandwidth, CPU....), possibly outside
          of the browser where the browser can't tell that it's exceeding a
          constraint</t>

          <t>Specifying resolutions that are a reasonable fit for the current
          usage of the video stream, for instance fitting with the number of
          pixels available on the part of a device's display surface that is
          devoted to displaying this video stream</t>

          <t>Specifying the shape of a video stream, in order to fit the video
          onto a display surface without the need for black bars or image
          distortion</t>
        </list>Similar considerations apply for framerate.</t>

      <t></t>

      <section title="Disposition of this text">
        <t>This draft is written in order to get something specific out to
        refer to during spec-writing and implementation. The text may
        eventually get merged into either the IETF document on SDP usage by
        RTCWEB, or the W3C WEBRTC document on PeerConnection.</t>
      </section>
    </section>

    <section title="Usage considerations">
      <t>These constraints are usable in several places:</t>

      <t><list style="symbols">
          <t>As constraints to the getUserMedia call <xref
          target="W3C.WD-mediacapture-streams-20120628"></xref>, where they
          serve to guide the configuration of the camera obtained, and may
          influence the choice of camera.</t>

          <t>As constraints to the addStream call on a PeerConnection <xref
          target="W3C.WD-webrtc-20120821"></xref>, where they serve to guide
          the configuration of the codec that encodes the video content for
          transmission.</t>

          <t>As constraints applied to an existing local video stream using
          the "change constraints" API, where it may cause the video engine to
          reconfigure the device or codec for that particular stream.</t>

          <t>As constraints applied to an incoming video stream using the
          "change constrains" API on a MediaStreamTrack, where it serves to
          inform the video engine about the desirable properties of the video
          track, which may lead to the video engine choosing to reencode the
          video and/or signal a remote video source that it wishes certain
          constraints to be put in place.</t>
        </list>All of the constraints may be meaningful in both "mandatory"
      and "optional" forms.</t>

      <t></t>
    </section>

    <section title="Usage examples">
      <t>See <xref target="IANA"></xref> for the actual definition of the
      constraints used here.</t>

      <section title="Examples with GetUserMedia">
        <t>A constraint saying that we absolutely must have a minimum
        resolution of 1024x768:</t>

        <figure>
          <artwork><![CDATA[
getUserMedia({
   video: { mandatory: { minWidth: 1024, minHeight: 768 } }
}, successCallback, errorCallback);


]]></artwork>
        </figure>

        <t>A constraint saying that we'd prefer 60 frames per second, if
        available, and if we can get that, we'd like to limit the max
        resolution, but in all cases, the screen must be clamped to a 4:3
        aspect ratio - 16:9 or odd aspect ratios are not acceptable to this
        application:</t>

        <figure>
          <artwork><![CDATA[
getUserMedia({
   video: {
     mandatory: { minAspectRatio: 1.333, maxAspectRatio: 1.334 },
     optional [
       { minFrameRate: 60 },
       { maxWidth: 640 },
       { maxHeigth: 480 }
     ]
   }
}, successCallback, errorCallback);
]]></artwork>
        </figure>

        <t></t>
      </section>

      <section title="Possible SDP mappings">
        <t>This document does not specify or constrain how constraints get
        reflected into SDP (if they do); that's an implementor decision.</t>

        <t>The examples below are thought exercises, based on <xref
        target="I-D.lennox-mmusic-sdp-source-selection"></xref> and <xref
        target="I-D.alvestrand-rtcweb-resolution"></xref>.</t>

        <t>An optional constraint has been applied to an incoming stream where
        both upper and lower are constrained to 320x200. The stream has been
        assigned to a hardware video decoder that can decode most resolutions
        up to 1024x768, in any aspect ratio, but only if all divisions are
        divisible by 4. The incoming stream has SSRC 1234.</t>

        <t>Line breaks are added for readability.</t>

        <figure>
          <artwork><![CDATA[
m=video
a=remote-ssrc:1234 imageattr:* [x=320,y=200,q=1.0] \
                  [x=[120:4:1024],y=[100:4:768],q=0.2]



]]></artwork>
        </figure>

        <t></t>
      </section>
    </section>

    <section anchor="IANA" title="IANA Considerations">
      <t>This document requests IANA to register constraints in the "RTCWeb
      Media Constraints" registry created by <xref
      target="I-D.burnett-rtcweb-constraints-registry"></xref>. NOTE: The
      registrations assume that this document is updated to no longer have
      "video" as part of the name, but have "video" as a field-of-use in the
      registration.</t>

      <t>The definitions of width, height and aspect ratio are taken from
      <xref target="RFC6236"></xref>.</t>

      <t><list style="symbols">
          <t>MinWidth - valid for video. Corresponds to the "x" value (pixel
          count) from RFC 6236. Only integer values are valid.</t>

          <t>MaxWidth - valid for video. Definition as for MinWidth.</t>

          <t>MinHeight - valid for video. Corresponds to the "y" value (pixel
          count) from RFC 6236. Only integer values are valid.</t>

          <t>MaxHeight - valid for video. Definition as for MinHeight.</t>

          <t>MinAspectRatio - valid for video. Corresponds to the "par"
          (picture aspect ratio), with "sar" set to 1.0. A 4:3 format display
          corresponds to an AspectRatio of 1.3333. Floating point values are
          valid.</t>

          <t>MaxAspectRatio - valid for video. Definition as for
          MinAspectRatio.</t>

          <t>MinFramerate - valid for video. Corresponds to the framerate
          defined in <xref target="RFC4566"></xref>, the "a=framerate"
          attribute.</t>

          <t>MaxFramerate - valid for video. Definition as for
          MinFramerate.</t>
        </list></t>

      <t>The contact person is Harald Alvestrand <hta@google.com>.</t>

      <t>Change control for the registration is with the IETF, as designated
      by the IESG.</t>

      <t>Note that MinFramerate defines a lower bound for the a=framerate
      attribute, which is itself defined as an upper limit; this means that
      even if a high framerate is negotiated, the actual framerate used may be
      lower due to temporary considerations (for instance CPU or bandwidth, or
      simply lack of movement in the picture).</t>
    </section>

    <section anchor="Security" title="Security Considerations">
      <t>No security considerations particular to these specific constraints
      have so far been identified.</t>
    </section>

    <section anchor="Acknowledgements" title="Acknowledgements">
      <t>Special thanks are given to Dan Burnett, Cullen Jennings, the IETF
      RTCWEB WG and the W3C WEBRTC WG for strongly influencing this memo.</t>
    </section>
  </middle>

  <back>
    <references title="Normative References">
      <?rfc include="reference.RFC.2119"?>

      <?rfc include='reference.RFC.4566'?>

      <?rfc include='reference.RFC.6236'?>

      <?rfc include='reference.I-D.burnett-rtcweb-constraints-registry'?>
    </references>

    <references title="Informative References">
      <?rfc include='reference.I-D.lennox-mmusic-sdp-source-selection'?>

      <?rfc include='reference.I-D.alvestrand-rtcweb-resolution'?>

      <?rfc include='reference.W3C.WD-mediacapture-streams-20120628'?>

      <?rfc include='reference.W3C.WD-webrtc-20120821'?>
    </references>
  </back>
</rfc>