One document matched: draft-thomson-postel-was-wrong-00.xml

<?xml version="1.0" encoding="UTF-8"?>
  <?xml-stylesheet type="text/xsl" href="rfc2629.xslt" ?>
  <!-- generated by https://github.com/cabo/kramdown-rfc2629 version 1.0.22 -->

<!DOCTYPE rfc SYSTEM "rfc2629.dtd" [
]>

<?rfc toc="yes"?>
<?rfc sortrefs="yes"?>
<?rfc symrefs="yes"?>

<rfc ipr="trust200902" docName="draft-thomson-postel-was-wrong-00" category="info">

  <front>
    <title abbrev="Elephants Out, Donkeys In">The Harmful Consequences of Postel's Maxim</title>

    <author initials="M." surname="Thomson" fullname="Martin Thomson">
      <organization>Mozilla</organization>
      <address>
        <email>martin.thomson@gmail.com</email>
      </address>
    </author>

    <date year="2015"/>

    
    
    

    <abstract>


<t>Jon Postel’s famous statement in RFC 1122 of “Be liberal in what you accept, and
conservative in what you send” – is a principle that has long guided the design
of Internet protocols and implementations of those protocols.  The posture this
statement advocates might promote interoperability in the short term, but that
short term advantage is outweighed by negative consequences that affect the long
term maintenance of a protocol and its ecosystem.</t>



    </abstract>


  </front>

  <middle>


<section anchor="introduction" title="Introduction">

<t>Of the great many contributions Jon Postel made to the Internet, his remarkable
technical achievements are often ignored in favor of the design and
implementation philosophy that he first captured in the original IPv4
specification <xref target="RFC0760"></xref>:</t>

<t><list style='empty'>
  <t>In general, an implementation should be conservative in its sending behavior,
  and liberal in its receiving behavior.</t>
</list></t>

<t>In comparison, his contributions to the underpinnings of the Internet, which are
in many respects more significant, enjoy less conscious recognition.  Postel’s
principle has been hugely influential in shaping the Internet and the systems
that use Internet protocols.  Many consider this principle to be instrumental in
the success of the Internet as well as the design of interoperable protocols in
general.</t>

<t>Over time, considerable changes have occurred in both the scale of the Internet
and the level of skill and experience available to protocol and software
designers.  Part of that experience is with protocols that were designed,
informed by Postel’s maxim, in the early phases of the Internet.</t>

<t>That experience shows that there are negative long-term consequences to
interoperability if an implementation applies Postel’s advice.  Correcting the
problems caused by divergent behavior in implementations can be difficult or
impossible.</t>

<t>It might be suggested that the posture Postel advocates was indeed necessary
during the formative years of the Internet, and even key to its success.  This
document takes no position on that claim.</t>

<t>This document instead describes the negative consequences of the application of
Postel’s principle to the modern Internet.  A replacement design principle is
suggested.</t>

<t>There is good evidence to suggest that designers of protocols in the IETF widely
understand the limitations of Postel’s principle.  This document serves
primarily as a record of the shortcomings of His principle for the wider
community.</t>

</section>
<section anchor="time" title="The Protocol Decay Hypothesis">

<t>Divergent implementations of a specification emerge over time.  When
variations occur in the interpretation or expression of semantic components,
implementations cease to be perfectly interoperable.</t>

<t>Implementation bugs are often identified as the cause of variation, though it is
often a combination of factors.  Application of a protocol to new and
unanticipated uses, and ambiguities or errors in the specification are often
confounding factors.</t>

<t>Of course, situations where two peers disagree are common, and should be
expected over the lifetime of a protocol.  Even with the best intentions, the
pressure to interoperate can be significant.  No implementation can hope to
avoid having to trade correctness for interoperability indefinitely.</t>

<t>An implementation that reacts to variations in the manner advised by Postel sets
up a feedback cycle:</t>

<t><list style="symbols">
  <t>Over time, implementations progressively add new code to constrain how data is
transmitted, or to permit variations what is received.</t>
  <t>Errors in implementations, or confusion about semantics can thereby be masked.</t>
  <t>As a result, errors can become entrenched, forcing other implementations to be
tolerant of those errors.</t>
</list></t>

<t>An entrenched flaw can become a de facto standard.  Any implementation of the
protocol is required to replicate the aberrant behavior, or it is not
interoperable.  This is both a consequence of applying Postel’s advice, and a
product of a natural reluctance to avoid fatal error conditions.  This is
colloquially referred to as being “bug for bug compatible”.</t>

<t>It is debatable as to whether such a process can be completely avoided, but
Postel’s maxim encourages a reaction that compounds this issue.</t>

</section>
<section anchor="the-long-term-costs" title="The Long Term Costs">

<t>Once deviations become entrenched, there is little that can be done to rectify
the situation.</t>

<t>For widely used protocols, the massive scale of the Internet makes large scale
interoperability testing infeasible for all a privileged few.  Without good
maintenance, new implementations can be restricted to niche uses, where the
prolems arising from interoperability issues can be more closely managed.</t>

<t>This has a negative impact on the ecosystem of a protocol.  New implementations
of a protocol are important in ensuring the continued viability of a protocol.
New protocol implementations are also more likely to be developed for new and
diverse use cases and often are the origin of features and capabilities that can
be of benefit to existing users.  These problems also reduce the ability of
established implementations to change.</t>

<t>Protocol maintenance can help by carefully documenting divergence and
recommending limits on what is both acceptable and interoperable.  The
time-consuming process of documenting the actual protocol - rather than the
protocol as it was originally conceived - can restore the ability to create and
maintain interoperable implementations.</t>

<t>Such a process was undertaken for HTTP/1.1 <xref target="RFC7230"></xref>. This this effort took more
than 6 years, it has been successful in documenting protocol variations and
describing what has over time become a far more complex protocol.</t>

</section>
<section anchor="a-new-design-principle" title="A New Design Principle">

<t>The following principle applies not just to the implementation of a protocol,
but to the design and specification of the protocol.</t>

<t><list style='empty'>
  <t>Protocol designs and implementations should be maximally strict.</t>
</list></t>

<t>Though less pithy than Postel’s formulation, this principle is based on the
lessons of protocol deployment.  The principle is also based on valuing early
feedback, a practice central to modern engineering discipline.</t>

<section anchor="fail-fast-and-hard" title="Fail Fast and Hard">

<t>Protocols need to include error reporting mechanisms that ensure errors are
surfaced in a visible and expedient fashion.</t>

<t>Generating fatal errors for what would otherwise be a minor or recoverable error
is preferred, especially if there is any risk that the error represents an
implementation flaw.  A fatal error provides excellent motivation for addressing
problems.</t>

<t>On the whole, implementations already have ample motivation to prefer
interoperability over correctness.  The primary function of a specification is
to proscribe behavior in the interest of interoperability.</t>

</section>
<section anchor="implementations-are-ultimately-responsible" title="Implementations Are Ultimately Responsible">

<t>Implementers are encouraged to expose errors immediately and prominently in
addition to what a specification mandates.</t>

<t>Exposing errors is particularly important for early implementations of a
protocol.  If preexisting implementations generate errors in response to
divergent behaviour, then new implementations will be able to detect and correct
flaws quickly.</t>

<t>An implementer that discovers a scenario that is not covered by the
specification does the community a greater service by generating a fatal error
than by attempted to interpret and adapt.  Hiding errors can cause long-term
problems.  Ideally, specification shortcomings are taken to protocol
maintainers.</t>

<t>Unreasoning strictness can be detrimental.  Protocol designers and implementers
expected to exercise judgment in determining what level of strictness is
ultimately appropriate.  In every case, documenting the decision to deviate from
what is specified can avoid later issues.</t>

</section>
<section anchor="protocol-maintenance-is-important" title="Protocol Maintenance is Important">

<t>Protocol designers are strongly encouraged to continue to maintain and evolve
protocols beyond their initial inception and definition.  If protocol
implementations are less tolerant of variation, protocol maintenance becomes
critical.  Good extensibility <xref target="RFC6709"></xref> can relieve some of the pressure on
maintenance.</t>

</section>
</section>
<section anchor="iana-considerations" title="IANA Considerations">

<t>This document has no IANA actions.</t>

</section>
<section anchor="security-considerations" title="Security Considerations">

<t>Sloppy implementations, lax interpretations of specifications, and uncoordinated
extrapolation of requirements to cover gaps in specification can result in
security problems.  Hiding the consequences of protocol variations encourages
the hiding of issues, which can conceal bugs and make them difficult to
discover.</t>

<t>Designers and implementers of security protocols generally understand these
concerns.  However, general-purpose protocols are not exempt from careful
consideration of security issues.  Furthermore, because general-purpose
protocols tend to deal with flaws or obsolescence in a less urgent fashion than
security protocols, there can be fewer opportunities to correct problems in
protocols that develop interoperability problems.</t>

</section>


  </middle>

  <back>


    <references title='Informative References'>





<reference anchor='RFC0760'>

<front>
<title abbrev='Internet Protocol'>DoD standard Internet Protocol</title>
<author initials='J.' surname='Postel' fullname='Jon Postel'>
<organization>University of Southern California (USC)/Information Sciences Institute</organization>
<address>
<postal>
<street>4676 Admiralty Way</street>
<city>Marina del Rey</city>
<region>CA</region>
<code>90291</code>
<country>US</country></postal></address></author>
<date year='1980' day='1' month='January' /></front>

<seriesInfo name='RFC' value='760' />
<format type='TXT' octets='81507' target='http://www.rfc-editor.org/rfc/rfc760.txt' />
</reference>



<reference anchor='RFC6709'>

<front>
<title>Design Considerations for Protocol Extensions</title>
<author initials='B.' surname='Carpenter' fullname='B. Carpenter'>
<organization /></author>
<author initials='B.' surname='Aboba' fullname='B. Aboba'>
<organization /></author>
<author initials='S.' surname='Cheshire' fullname='S. Cheshire'>
<organization /></author>
<date year='2012' month='September' />
<abstract>
<t>This document discusses architectural issues related to the extensibility of Internet protocols, with a focus on design considerations.  It is intended to assist designers of both base protocols and extensions.  Case studies are included.  A companion document, RFC 4775 (BCP 125), discusses procedures relating to the extensibility of IETF protocols.  This document is not an Internet Standards Track specification; it is published for informational purposes.</t></abstract></front>

<seriesInfo name='RFC' value='6709' />
<format type='TXT' octets='103913' target='http://www.rfc-editor.org/rfc/rfc6709.txt' />
</reference>



<reference anchor='RFC7230'>

<front>
<title>Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing</title>
<author initials='R.' surname='Fielding' fullname='R. Fielding'>
<organization /></author>
<author initials='J.' surname='Reschke' fullname='J. Reschke'>
<organization /></author>
<date year='2014' month='June' />
<abstract>
<t>The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems.  This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations.</t></abstract></front>

<seriesInfo name='RFC' value='7230' />
<format type='TXT' octets='205947' target='http://www.rfc-editor.org/rfc/rfc7230.txt' />
</reference>




    </references>



  </back>
</rfc>