Subject: [Gen-art] Gen-ART review of draft-dusseault-impl-reports-02

From: "Vijay K. Gurbani" <[email-masked]>
Date: Wed, 17 Jun 2009 17:49:31 -0500

I have been selected as the General Area Review Team (Gen-ART)
reviewer for this draft (for background on Gen-ART, please see
http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).

Please wait for direction from your document shepherd
or AD before posting a new version of the draft.

Document: draft-dusseault-impl-reports-02
Reviewer: Vijay K. Gurbani
Review Date: Jun 17, 2009
IESG Telechat date: N/A

Summary: This draft is ready for publication as a BCP; some minor
issues and nits below.

The draft has 0 major issues, 3 minor issues, and 8 Nits/Editorial
comments.

A general question I had is whether this report guidance is
only for documents moving from Proposed to Draft standard
and not from Draft to Internet standard?  Certain passages
and text in the draft appear to reflect the former
scenario (i.e., this guidance is good only for moving
drafts from Proposed to Draft standard) however the title
of the draft does not seem to restrict it to only this
scenario.

Minor issues:
1) In S2, page five, first "Note", it says that "Independent
  implementations should be written by different people at
  different organizations using different code and
  protocol libraries."

  While all else is fine in the text above, why the inclusion
  of different "protocol libraries"?  Sometimes, a protocol
  library is as much critical to a code base as it is complex
  (think OpenSSL).  As such, if implementation A uses
  OpenSSL but its SIP transaction engine code is different than
  implementation B's, which also uses OpenSSL, then these
  two implementations are independent despite sharing a protocol
  library (OpenSSL).

  If we position the criteria as "Independent implementations
  should be written by different people at different
  organizations using genetically dissimilar code base", do we
  loose something in the process?

2) In S2, it is stated that each report "MUST identify the author
  of the report, and in S3 it says that "Titles of implementation
  reports are strongly RECOMMENDED to contain one or more RFC
  number ..."

  This is good advice, however, it is ignored when, in S3, a simple
  outline for a minimal implementation report is given.  I would
  suggest that the outline begin with:

  TITLE: Implementation report for RFC-XXXX
  AUTHOR: Joe Q. Public-Shepherd <[email-masked]>

3) S5.5 is interesting.  Sometimes, the test suites do not even
  appear in the protocol document itself.  Case in point is SIP.
  RFC3261 uses X.509 certificates in TLS, but the test cases for
  how these certificates are to be formed and verified are
  actually in other documents (see S8 and Appendix A and B of
  http://tools.ietf.org/html/draft-ietf-sipcore-sec-flows-00, for
  instance.)

  I do not know what to suggest here, except that you may
  consider using this as an additional example if you think it is
  appropriate.

Nits:

1) S1, consider the following text:

     Moving standards along the standards track can be an important
     signal to the user and implementor communities, and the
     process of submitting a standard for advancement can help
     improve the standard or the quality of implementations that
     participate.

  There is an overuse of the word "standard" above; I can discern
  at least three usages of the word.  The first is "standard" as in
  what people think an RFC is, and the second usage is the
  specific "standards track" that the RFC is on, and the third
  usage is "standard" as a synonym for quality.  Maybe re-writing
  the above to better outline the intent may not be a bad idea.
  Something like the following (please feel free to edit):

      Moving documents or RFCs along the standards track can be
      an important signal to the user and implementor communities,
      and the process of revising a RFC on the standards track
      for further advancement can help improve the quality of
      implementations that participate.

2) S1, third paragraph:
  s/This memo may help in/This memo helps in/

3) S1, page four, fifth line:
  s/problem features/features that proved to be problematic/

4) S2, page five, first "Note":
  s/many more implementations or the/many more implementations, or the/

5) In S2, page five, second "Note":
  s/In contrast,/By contrast,/

6) S5.1:
  s/test-lab results/results derived from laboratory testing/

7) S5.3:
  s/implemented and syntax/implemented and their syntax/

8) S5.4, second line:
  s/implementation, the result/implementation; the result/

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Robert Sparks <[email-masked]>
Date: Thu, 18 Jun 2009 16:44:10 -0500

Hi Vijay -

See inline:

> of the draft does not seem to restrict it to only this

This focus reflects what's in 2026 - it brings up interoperability  
requirements
in section 4.1.2 on Draft Standards. The next section, on Internet  
Standard, presumes
interoperability and focuses on "significant implementation and  
successful
operational experience".

I don't think the language focusing on the Proposed to Draft  
transition in this
draft counter-indicates using its recommendations for other purposes.

> loose something in the process?

Using your example, the point was to avoid having people claim
interoperable independent implementations of SSL by demonstrating
two applications written on top of OpenSSL.
There's a line below which the use of common library code has little  
impact
on the evidence that the protocol in question can be independently
implemented (your SIP transaction engine code implementations could
use the same or different implementations of malloc for example).   
This is
just one of the places the preparer of an implementation report will  
have to
use good judgement.

> AUTHOR: Joe Q. Public-Shepherd <[email-masked]>

OK

> consider using this as an additional example if you think it is

The existing text ("tests can at least be constructed from real-world
protocol or document examples") doesn't restrict the search for test
examples to just the protocol document.

I'd prefer not to shift the focus of this section too far away from  
where
it is now - the message that passing an existing automated test suite
may not be enough to demonstrate independent implementability needs
to continue to show.

I'll look at the Nits while integrating the item above.

Thanks!

RjS

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Thu, 18 Jun 2009 19:34:11 -0500

> Hi Vijay -

Hello Robert: Thanks for going through the comments.  A
couple of points inline.

> two applications written on top of OpenSSL. 

I think we may be cross-communicating (either that or we are
in agreement, I am not sure which.)  In my original
review of the text, what perked my intrigue was the inclusion
of different "protocol libraries" when demonstrating
interoperability.

And I was merely pointing out that in some cases, a protocol
engine may be built on top of a well known, best-in-class
library (think OpenSSL as an example.)  Now, when the preparer
of the implementation report is given two protocol engines that
use the same OpenSSL library, it is obvious to him or her that
what is undergoing interoperability here is not OpenSSL,
but the protocol engine on top of it.  In other words, it
is perfectly okay to have a common underlying library as long
as the protocol engine using that library comes from an
independent strain of source code.

So, I was looking to see if we would loose anything by
reformulating the sentence as "Independent implementations
should be written by different people at different
organizations using genetically dissimilar code base."
That is, stay away from mentioning that "protocol libraries"
should be different as well -- in some cases, implementors
do not have the luxury of a choice among underlying low-level
protocol libraries.

> I'd prefer not to shift the focus of this section too far away from where
> it is now [...]

Sure, no problem.

> I'll look at the Nits while integrating the item above.

OK.

Thanks,

- vijay