One document matched: draft-allman-problem-wg-revcomm-00.txt

Internet Engineering Task Force                              Mark Allman
INTERNET DRAFT                                                      ICIR
File: draft-allman-problem-wg-revcomm-00.txt                 James Kempf
                                                         DoCoMo Labs USA
                                                           October, 2003
                                                    Expires: April, 2004
    
    
                 Using Working Group Review Committees
    

Status of this Memo

    This document is an Internet-Draft and is in full conformance with
    all provisions of Section 10 of [RFC2026].

    Internet-Drafts are working documents of the Internet Engineering
    Task Force (IETF), its areas, and its working groups.  Note that
    other groups may also distribute working documents as
    Internet-Drafts.

    Internet-Drafts are draft documents valid for a maximum of six
    months and may be updated, replaced, or obsoleted by other documents
    at any time.  It is inappropriate to use Internet-Drafts as
    reference material or to cite them other than as "work in progress."

    The list of current Internet-Drafts can be accessed at
    http://www.ietf.org/ietf/1id-abstracts.txt

    The list of Internet-Draft Shadow Directories can be accessed at
    http://www.ietf.org/shadow.html.

Abstract

    This document sketches a potential quality control mechanism for the
    IETF in the form of working group review committees.  The idea is to
    form a small committee per working group that will provide document
    review and advice throughout the working group's lifetime.  

1   Introduction

    In this document we sketch a possible quality control mechanism for
    the IETF that is based on forming a small review committee for each
    working group.  While the members of this committee would have no
    special privledges within the IETF process we believe that a small
    group may aid the WG in several ways:

      + Provide architectural advice from a broader vantage point than
        the WG's focused viewpoint to help the WG understand the
        interactions of their work with the entire system.

      + Provide a view point from a different area of the IETF (e.g., a
        security viewpoint in an applications working group).

      + Provide a place for vetting the WGs output to a broader group

Expires: April 2004                                             [Page 1]

draft-allman-problem-wg-revcomm-00.txt                      October 2003

        than the WG itself before the output is sent to the IESG.

    The review committee notion is in some ways a refinement of the
    "stuckee" notion [Har03].  While the stuckee notion is focused on
    (slightly) formalizing roles within a WG a review committee is
    mainly focused on obtaining "outside" input (important input that
    generally comes from people who would otherwise not be deeply
    involved with the WG).
    
    The ideas outlined in this document attempt to provide a similar
    kind of quality control provided by the SIRs proposal [CC03].  SIRs
    are logical candidates for the review committee, but the committee
    concept goes further in that the review committee may draw in people
    who are not traditionally active in IETF, like implementors,
    academics, or people active in other fora who may have an interest
    in the work and the experience to judge it.  Also, unlike the
    original SIRs proposal, which was that a SIR would do a one-off
    review, the intent of a review committee is that they make
    themselves periodically available for review during the lifetime of
    the WG (as long as they have time, of course).

    The notion of review committees also is an extension of the
    "technical advisor" that is sometimes appointed to various WGs now.

    In some WGs the notion of a "review committee" is already informally
    put to use.  For instance, when documents are at a critical stage a
    WG chair will ask several people from around the IETF to read and
    comment on the draft.  This document formalizes this notion so that
    the committee is setup apriori for the WG chair(s) to utilize.

2   Forming a Review Committee    
    
    Each WG would form a review committee whose size would be dictated
    by the WG's workload and breadth.  There are no formal rules about
    who can sit on a review committee.  Likewise, there is no explicit
    rule on the size of the review committee, but the size should be
    somewhat dependant on the number of work items a particular WG has
    (an initial touchstone would be 2 members per document).  The
    committee is chosen by the WG chair(s) and the area directors.
    While there are no hard and fast rules there are some guidelines in
    choosing committee members:

      + A security guru is a likely must for groups outside the security
        area.
      + If the WG is involved in writing MIBs a MIB doctor would be
        useful. 
      + Etc.

    Beyond obvious choices the WG chairs and ADs should pick several
    people from different areas of the IETF to ensure the documents
    within the group received a thorough review from a number of
    angles.  In addition, viewpoints from outside the IETF may also be
    useful to include (e.g., from operators that do not normally
    participate but have a particularly interesting vantage point on

Expires: April 2004                                             [Page 2]

draft-allman-problem-wg-revcomm-00.txt                      October 2003

    some work).
    
    A committee is formed by consultation between the WG chairs and
    shepherding AD.  The WG chairs generate a list of potential members,
    and discuss with the AD about who might be appropriate. After an
    agreed upon list of members has been generated, the WG chairs
    contact the committee members to assess their willingness to serve.
    
3   Role of the Review Committee

    The review committee looks at documents as asked by the WG
    chair(s).  All reviews are sent to the WG mailing list for public
    comment and archiving.  The comments from the review committee are
    to be treated no differently than reviews from WG members or any
    other member of the community.
    
    The review committee members are not expected to closely track the
    WG if they do not have the time or inclination.  Some members of the
    committee may want to engage the WG on the mailing list with regards
    to the work and their reviews, while other members may not.  Either
    of these is fine (although, the WG chair(s) and AD(s) may consider a
    potential member's stated participation plan when forming the
    committee).  Should a committee member not directly participate on
    the mailing list or in meetings they should be available for
    questions and clarifications about their reviews to the WG chair(s).

    Review committee members contributions are acknowledged in two ways:

      + By placing their names on a Working Group's Web page.

      + Within the Contributions section of the Working Group documents
        they have reviewed.

    In order for a review committee member to qualify for
    acknowledgement, they must generate at least one detailed review of
    a working group draft.  People who agree to particpate but don't
    contribute will not be listed as contributors.
    
4   Experience with Review Committees

    Review committees have been used in the Seamoby WG and SEND WG. In
    both cases, high quality, detailed reviews of protocol documents
    were generated by most of the review committee members. Only a few
    of the reviews were perfunctory. In the Seamboy case, two reviewers
    were selected for their extensive expertiese in implementing Mobile
    IP, though they did not typically participate in the IETF. Several
    of the reviewers on the Seamoby committee followed up with
    discussion on the mailing list, even though they did not typically
    participate in the WG. An attempt was made to limit the demands on
    the review committee members, so reviews were only solicited at the
    semi-final and final stages of document preparation. Since review
    committee members may not be full time WG members, gauging exactly
    how much to ask of them is critical to maintaining their committment
    and interest.  All in all, the experience with review committees in

Expires: April 2004                                             [Page 3]

draft-allman-problem-wg-revcomm-00.txt                      October 2003

    these two cases has been extremely positive. The amount and quality
    of the technical input obtained from the review committees far and
    away exceeded the amount of input typically generated when reviews
    are not directly solicited, such as by issuance of a WG last call.

    To be effective, reviews need to be accompanied by effective WG
    issue management. Each review should be used to generate a list of
    issues recorded, along with other issues raised by the WG, on an
    issues Web page.  Document editors, or, in the absence of strong
    editorship, the WG chairs, then become responsible for posting
    issues on the mailing list and monitoring discussion until consensus
    is achieved on the issues. The results of this discussion then need
    to be edited back into the WG documents.  Editorial issues, which
    are also raised by reviewers, typically don't need as strict
    discussion, though editors and WG chairs need to assess carefully
    whether an editorial issue will affect interpretation of the
    document, and discuss it with the WG if so.

Non-Normative References
    
    [CC03] Brian Carpenter, Dave Crocker.  Careful Additional Review of
        Documents (CARD) by Senior IETF Reviewers (SIRS).
        Internet-Draft draft-carpenter-solution-sirs-01.txt, June 2003.
        Work in progress.

    [Har03] Ted Hardie.  Working Groups and Their Stuckees.
        Internet-Draft draft-hardie-wg-stuckees-00.txt, February 2003.
        Work in progress.

Authors' Addresses:

    Mark Allman
    ICSI Center for Internet Research
    1947 Center Street, Suite 600
    Berkeley, CA 94704-1198
    Phone: 216-243-7361
    mallman@icir.org
    http://www.icir.org/mallman/

    James Kempf
    DoCoMo Labs USA
    180 Metro Drive, Suite 300 
    San Jose, CA 95430 
    Phone: +1 408 451 4711 
    Email: kempf@docomolabs-usa.com 










Expires: April 2004                                             [Page 4]