One document matched: draft-ietf-proto-wgchair-doc-shepherding-00.txt
Proto Team H. Levkowetz
Internet-Draft D. Meyer
Expires: January 10, 2005 July 12, 2004
Protocol Pilot: Workgroup Chair Followup of AD Evaluation Comments
<draft-ietf-proto-wgchair-doc-shepherding-00.txt>
Status of this Memo
By submitting this Internet-Draft, each author represents that any
applicable patent or other IPR claims of which he or she is aware
have been or will be disclosed, and any of which he or she becomes
aware will be disclosed, in accordance with Section 6 of RFC 3668.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as
Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress".
The list of current Internet-Drafts can be accessed at http://
www.ietf.org/ietf/1id-abstracts.txt .
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html .
This Internet-Draft will expire on January 10, 2005.
Copyright Notice
Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract
This document describes a pilot implementation of a protocol change
within the IETF. The essence of the change is to have workgroup
chairs more involved in the progress of the document after the
workgroup and document editor have handed over the document for
publication. The activities involved in this are: 1) providing a
writeup for the document, to accompany the request for publication
which is sent to the secretariat and the ADs (Area Directors); 2)
following up on AD Evaluation comments on a draft to the authors and
workgroup; and 3) following up on IESG comments (DISCUSS comments as
well as other) on the draft.
Levkowetz & Meyer Expires January 10, 2005 [Page 1]
Internet-Draft WG Chair followup of AD-Comments July 2004
1. Introduction
As part of the currently ongoing effort to improve the work flow
(particularly speed of approval) of documents, the PROTO team
[PROTO] is defining pilot projects to test possible protocol changes.
This document describes such a pilot.
The purpose of the pilot described here is to test offloading
follow-up work which an Area Director (AD) traditionally has done.
This includes both follow-up after the AD has read through and
evaluated a document submitted to the IESG for publication, and
follow-up on IESG comments (DISCUSS comments and others) on a
document. It is hoped that offloading this onto the chair (or one of
the chairs) of the workgroup which submitted the draft will increase
the speed of follow-up and the transparency of the process, and
reduce the workload of the AD to boot. The pilot does not cover
follow-up for drafts which do not originate in a workgroup.
For a discussion of the reasoning underlying piloting of process
changes, see [JULY14].
2. Pilot description
2.1 Participants
This pilot involves Area Directors of selected areas, and some or all
of the chairs for which the Area Director is Area Advisor.
2.2 Running time and pilot size
This pilot is to be run not less than 4 months, and not more than 8
months, unless early experience shows it to be clearly detrimental.
It is expected that it will be started shortly after the IETF 59
meeting, and completed in time for the results to be reported at the
IETF 60 meeting. The pilot should be run with no less than 2 and not
more than 6 ADs, and between 5 and 20 workgroups.
The running time should be chosen such that the participating ADs and
WG chairs have opportunity to get past the initial learning and
first-time execution barrier, and get some familiarity with the
process before the pilot is closed and evaluated.
2.3 Pilot Process Description
2.3.1 WG Chair Writeup
When a draft is ready to be submitted for publication, it is the task
of the shepherding workgroup chair to do a document writeup for the
Levkowetz & Meyer Expires January 10, 2005 [Page 2]
Internet-Draft WG Chair followup of AD-Comments July 2004
draft.
There are two parts to this task. First, answer questions 1-8 so
that your responsible Area Director can get insight into the working
group process as applied to this draft. These questions may appear
redundant in some cases but are written to elicit any tidbits of
information that the AD should be aware of. (Pointers to relevant
entries in the WG archive will be helpful.) The goal is to inform
your AD about any issues that may have come up in IETF meetings, on
the mailing list, or in private communication that they should be
aware of prior to taking this draft to IESG. Don't be surprised if
your AD has further questions. Any significant issues mentioned in
the questionnaire will probably lead to a followup discussion with
the AD.
The second part is to prepare the "Protocol Writeup" which is dually
used first as the ballot writeup for the IESG telechat and then the
IETF-wide protocol announcement. Item number 9 describes the
elements of the writeup. If you haven't done this before, try
looking at other protocol announcements (in the IETF Announce
archive) and expect some comments from your AD on the draft.
1. Have the chairs personally reviewed this version of the ID and
do they believe this ID is sufficiently baked to forward to the
IESG for publication?
2. Has the document had adequate review from both key WG members
and key non-WG members? Do you have any concerns about the
depth or breadth of the reviews that have been performed?
3. Do you have concerns that the document needs more review from a
particular (broader) perspective (e.g., security, operational
complexity, someone familiar with AAA, etc.)?
4. Do you have any specific concerns/issues with this document that
you believe the ADs and/or IESG should be aware of? For
example, perhaps you are uncomfortable with certain parts of the
document, or whether there really is a need for it, etc., but at
the same time these issues have been discussed in the WG and the
WG has indicated it wishes to advance the document anyway.
5. How solid is the WG consensus behind this document? Does it
represent the strong concurrence of a few individuals, with
others being silent, or does the WG as a whole understand and
agree with it?
6. Has anyone threatened an appeal or otherwise indicated extreme
discontent? If so, please summarise what are they upset about.
Levkowetz & Meyer Expires January 10, 2005 [Page 3]
Internet-Draft WG Chair followup of AD-Comments July 2004
7. Have the chairs verified that the document adheres to _all_ of
the ID nits? (see http://www.ietf.org/ID-Checklist.html).
8. Does the document a) split references into normative/
informative, and b) are there normative references to IDs, where
the IDs are not also ready for advancement or are otherwise in
an unclear state? (Note: the RFC editor will not publish an RFC
with normative references to IDs, it will delay publication
until all such IDs are also ready for publication as RFCs.)
9. For Standards Track and BCP documents, the IESG approval
announcement includes a writeup section with the following
sections:
* Technical Summary
* Working Group Summary
* Protocol Quality
10. Please provide such a writeup. (We will hopefully use it as is,
but may make some changes.) For recent examples, have a look at
the "protocol action" announcements for approved documents.
11. Note:
* When doing the technical summary, one would expect that the
relevant information is in the abstract and/or introduction
of the document. It turns out that the step of producing the
writeup sometimes points out deficiencies in the
introduction/abstract that are also worthy of rectifying.
* For the Working Group Summary, was there anything in WG
process that is worth noting? (E.g., controversy about
particular points, decisions where consensus was particularly
rough, etc.)
* For the protocol quality, useful information could include:
+ is the protocol already being implemented?
+ have a significant number of vendors indicated they plan
to implement the spec?
+ are there any reviewers (during the end stages) that merit
explicit mention as having done a thorough review that
resulted in important changes or a conclusion that the
document was fine (except for maybe some nits?)
Levkowetz & Meyer Expires January 10, 2005 [Page 4]
Internet-Draft WG Chair followup of AD-Comments July 2004
2.3.2 AD Review Shepherding
The steps for workgroup chair shepherding of AD reviews are as
follows:
1. If there is more than one chair, the chairs decide on which one
should be responsible for ensuring that the needed fixes are done
when the AD returns comments. This can for instance be done at
the time the publication request is sent. It is important that
this is an explicit agreement.
2. The AD reads, evaluates and writes comments pretty much as
before. However, note that since the communication between AD
and authors is not direct, the need for clear and
well-articulated review comments is somewhat larger.
3. Depending on the magnitude of the issues found (and other
considerations?), the AD returns the full review to the chairs,
and requests either:
3.a) that further workgroup work be undertaken to put the
document into shape to be published
3.b) that authors and workgroup are informed of the issues found
and resolve them in a revised draft
3.c) that the authors fix nits as needed.
As covered below, the comments will be posted to the workgroup
mailing list. The comments will normally also be posted by the
AD in the ID Tracker [IDTRACKER]. Working groups that use issue
tracking should also record the issues (and eventually their
resolution) in the issue tracker.
4. The chair responsible reads through the AD Evaluation comments,
making very certain that all comments are understood, so that it
is possible to follow up on them with the authors and workgroup.
If there is some uncertainty as to what is requested, this must
be resolved with the Area Director.
5. The responsible chair sends the comments to the author(s) and to
the workgroup mailing list, in order to have a permanent record
of the comments. It is recommended that the chair solicit from
the author(s) an estimate on when the fixes will be done - i.e.,
when the submission of a revised draft can be expected.
6. When incorporating the fixes in the new version of the draft, it
is strongly recommended that the revising editor keep a summary
Levkowetz & Meyer Expires January 10, 2005 [Page 5]
Internet-Draft WG Chair followup of AD-Comments July 2004
list showing how the issues were addressed issue by issue, and
showing what the revised text is. If such a list is forwarded to
the AD with the revised draft, it will make it possible for the
AD to verify the fixes very quickly.
7. The responsible chair follows-up, nudges and iterates until the
authors (and workgroup if required) has fixed the issues found,
and submitted an updated draft. At this point, the AD is
notified of the revised draft, and provided with the summary list
of issues and resulting text changes.
In the event that the working group disagrees with a comment
raised by the AD or has already considered the issue and
previously ruled it out, this must be discussed and resolved with
the AD before the new version of the draft is submitted.
8. The Area Director verifies that the issues he found during AD
Evaluation are resolved by the new version of the draft.
9. (Hopefully, that's it, but in the worst case this starts over at
1 again...)
2.3.3 IESG Discuss Shepherding
In this section we detail the steps that a shepherding WG chair will
take in resolving the DISCUSS items against a given ID. The steps are
given below, in the order that they are to be executed.
1. Immediately after the weekly IESG conference call, the
shepherding WG chair queries the ID tracker [IDTRACKER] to
collect any DISCUSS comments raised against the ID. In order to
accomplish this, the shepherding WG chair's email address must be
added to the "State Change Notice To:" field in the ID tracker.
This will result in an email to the shepherding WG chair when the
document is moved from the "IESG Evaluation" state to the "IESG
Evaluation/New ID Needed state", which occurs after the IESG
teleconference. This notification indicates to the the
shepherding WG chair that the DISCUSS comments have been
registered.
Note that there may be exceptional cases when DISCUSS comments
are registered after the IESG teleconference. In these cases,
the DISCUSSing AD should notify the shepherding WG chair that new
comments have been entered.
2. The shepherding WG chair analyses comments from the tracker, and
initialises contact with any AD's who have placed comments
Levkowetz & Meyer Expires January 10, 2005 [Page 6]
Internet-Draft WG Chair followup of AD-Comments July 2004
(blocking or non-blocking) on a draft, notifying them that the
shepherding WG chair is the current document shepherd and seeking
any additional clarification necessary to understand the comment.
Note that the responsible AD must copied on this correspondence.
+------+ Comments +--------+ Comments +-------+
| (i) |-------------> | (ii) | -------------> | (iii) |
+------+ Collected +--------+ Understood +-------+
/|\ |
| | Comments not fully understood
| | (Further AD/shepherding WG chair
| | Discussion Required)
+----+
3. The shepherding WG chair then coordinates DISCUSS comments, and
builds a a consistent interpretation of the comments. This step
may require iteration with step (2). above. That is:
+------+ Consistent +-------+
| (ii) |----------------> | (iii) |
+------+ Interpretation +-------+
/|\ |
| | Further AD/shepherding WG chair
| | Discussion Required
+--------------------------+
4. The DISCUSS comments are then communicated to the working group.
5. After the author(s) resolve the issues provided by the
shepherding WG chair (i.e., the distilled DISCUSS issues), the
shepherding WG chair reviews the updated document to ensure that
(in her/his option) the DISCUSS issues have been resolved.
Note that the shepherding WG chair may also propose resolutions
to these issues, file them in an issue tracker, or do other steps
to streamline the resolution of the comments.
6. The shepherding WG chair communicates the resolution-so-far to
the responsible AD and the DISCUSSing AD(s).
7. DISCUSSing AD removes DISCUSS comment, or tells the WG why the
comment is not resolved.
If the DISCUSS comment in question was not resolved to the
satisfaction of the DISCUSSing and responsible ADs, two
possibilities exist:
Levkowetz & Meyer Expires January 10, 2005 [Page 7]
Internet-Draft WG Chair followup of AD-Comments July 2004
1. (a). The process returns to step (3), or
2. (b). The working group can appeal in accordance with the
procedures described in RFC 2418 [RFC2418].
Otherwise, the process continues with step (8).
8. The responsible AD moves document to APPROVED state, or sends it
back to the IESG for re-review (if the changes are deemed
significant).
2.4 Wrap-up
At the end of the pilot lifetime, it is expected that an evaluation
of the experienced benefits is made, using input solicited from the
participating Area Directors and Workgroup Chairs by means of an
email questionnaire, web-page form or something similar. The
questions are given below, in Section 2.4.2. A per-review
questionnaire is also provided in Section 2.4.1.
2.4.1 Questionnaire to be done after each individual AD Review
To be done by both WG Chair and AD.
R1. I'm submitting this questionnaire as
1. Area Director
2. Workgroup Chair
R2. Document name:
R3. WG Chair shepherding of the AD evaluation comments for this
draft speeded up the procedure:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Levkowetz & Meyer Expires January 10, 2005 [Page 8]
Internet-Draft WG Chair followup of AD-Comments July 2004
R4. WG Chair shepherding of the AD evaluation comments for this
draft resulted in the comments being resolved in a satisfactory
manner:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R5. WG Chair shepherding of the AD evaluation comments for this
draft resulted in a more transparent process:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R6. WG Chair shepherding of the AD evaluation comments for this
draft resulted in a more well-documented process:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R7. The interaction with the document editors in resolving the
comments worked out well:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
R8. - Public Comments?
R9. - Comments to IESG and PROTO-Team only?
R10. WG Chair shepherding of the AD evaluation comments for this
draft worked out well, overall:
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Levkowetz & Meyer Expires January 10, 2005 [Page 9]
Internet-Draft WG Chair followup of AD-Comments July 2004
R11. - Public Comments?
R12. - Comments to IESG and PROTO-Team only?
2.4.2 Questionnaire for the Pilot as a Whole
To be done by both WG Chair and AD.
X1. Document name:
X2. I clearly understood what was expected of me in this pilot.
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Comments?
X3. What is your evaluation of the benefit of the procedure you've
tried out in this pilot?
1. Definitely harmful
2. Somewhat harmful
3. Mixed feelings
4. Somewhat beneficial
5. Definitely beneficial
Comments?
X4. What is your evaluation of the added effort required for the
procedure you've tried out in this pilot?
1. Major increased effort
2. Somewhat increased
3. No change
4. Somewhat decreased effort
5. Major decreased effort
Comments?
Levkowetz & Meyer Expires January 10, 2005 [Page 10]
Internet-Draft WG Chair followup of AD-Comments July 2004
X5. Considering all factors, this procedure should be made the
normal way of handling AD evaluation comments.
1. Strongly disagree
2. Disagree
3. Undecided
4. Agree
5. Strongly agree
Comments?
X6. What do you consider to be the major advantages of this
procedure change?
X7. What do you consider to be the major disadvantages of this
procedure change?
X8. How would you change the procedure to minimise the
disadvantages?
X9. Comments to the IESG and PROTO-Team only:
3. Security Considerations
This document specifies a pilot implementation of a change in IETF
procedures. It does not raise or consider any protocol-specific
security issues. When evaluating the result of the pilot, the IESG
should check if the changes has reduced the quality of security
review and consideration for protocols, and take this into
consideration when deciding whether the changes should be made
permanent.
4 Informative References
[RFC2026] Bradner, S., "The Internet Standards Process -- Revision
3", BCP 9, RFC 2026, October 1996.
[RFC2028] Hovey, R. and S. Bradner, "The Organizations Involved in
the IETF Standards Process", BCP 11, RFC 2028, October
1996.
[JULY14] Klensin, J. and S. Dawkins, "A model for IETF Process
Experiments", draft-klensin-process-july14-02 (work in
progress), April 2004.
[SIRS] Carpenter, B. and D. Crocker, "Careful Additional Review
of Documents (CARD)by Senior IETF Reviewers (SIRS)",
draft-carpenter-solution-sirs-01 (work in progress), June
Levkowetz & Meyer Expires January 10, 2005 [Page 11]
Internet-Draft WG Chair followup of AD-Comments July 2004
2003.
[IDTRACKER]
"The IETF Draft Tracker", Web Application: https://
datatracker.ietf.org/.
[PROTO] "The IESG Process and Tools (PROTO) Team", Web Page:
http://psg.com/~mrw/PROTO-Team.
Authors' Addresses
Henrik Levkowetz
Torsgatan 71
Stockholm S-113 37
SWEDEN
Phone: +46 708 32 16 08
EMail: henrik@levkowetz.com
David Meyer
EMail: dmm@1-4-5.net
Levkowetz & Meyer Expires January 10, 2005 [Page 12]
Internet-Draft WG Chair followup of AD-Comments July 2004
Intellectual Property Statement
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
Disclaimer of Validity
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET
ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE
INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Copyright Statement
Copyright (C) The Internet Society (2004). This document is subject
to the rights, licenses and restrictions contained in BCP 78, and
except as set forth therein, the authors retain all their rights.
Acknowledgment
Funding for the RFC Editor function is currently provided by the
Internet Society.
Levkowetz & Meyer Expires January 10, 2005 [Page 13]
| PAFTECH AB 2003-2026 | 2026-04-22 23:18:52 |