Subject: [newtrk] comments on draft-ietf-newtrk-interop-reports-00

From: Pekka Savola <[email-masked]>
Date: Sun, 6 Nov 2005 05:01:53 +0200 (EET)

Hi,

General problem with this approach: implementation reports would need to be
done in XML, which is presumably even more work for the implementation
reporters than using TXT is today.  We get few enough reports now that any
new requirements may be a prohibitive.

A partial migitation for this could be automatically generated (based on
PFS) web forms that could be filled out, and hence the XML would also get
automatically filled out without the implementation reporters having to
worry about it at all.

issues
------

    These structures can be used to enhance the IETF standards process in
    the following ways:
    o  Working groups (or individuals) preparing specifications for new
       protocols may also prepare the initial Protocol Feature Set. The
       IETF should publish these if they represent rough consensus.
    o  Working groups preparing specifications for updating existing
       protocols or adding new options of features to an existing
       protocol may prepare a proposed extension to an existing published
       Protocol Feature Set. Again, updated Protocol Feature Sets that
       represent community (rough) consensus should be published.

==> 'may' or 'must' ?  If the WGs or individuals don't do those,
who would?

    o  A feature description should indicate whether implementation is
       REQUIRED or optional for the protocol.

==> is there a particular reason why you use the terminology "REQUIRED" and
"optional" (or OPTIONAL) here?  Shouldn't this depend entirely on the
terminology used in the specification -- if the specification uses more
finer grained format (e.g., MUST, SHOULD, MAY), why shouldn't the interop
report use the same?

Protocol Feature Sets require
    rough consensus before they are published.  However, rough consensus
    may be judged by the willingness of implementors to prepare Protocol
    Implementation Reports using the Protocol Feature Set framework.

==> there is a recursion deadlock here.  PIRs can't be done until PFS is
done, yet the preparing PIRs could affect the consensus decision whether to
do PFS or not?  Did you mean that if implementors would promise before
starting on PFS that they'd fill it, once complete, or ...?

editorial
---------

    There is a great deal of judgement needed about how details to get in
    the protocol feature set.

==> this doesn't seem to parse.

    o  Identity of author, relationship to implementation, IPR.

==> does "IPR" refer to the author, implementation, or something in general?
reword to clarify, move to a new bullet if general.

    o  QUESTION: Is it a requirement to allow for anonomous

==> s/anono/anony/

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings
.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: "Larry Masinter" <[email-masked]>
Date: Mon, 7 Nov 2005 10:44:11 -0800

Thanks for your comments. Newtrk isn't meeting this week? (I'm not
in Vancouver this week, either).

> A partial mitigation for this could be automatically generated (based on

I think whether it is _required_ to submit the implementation report
in XML is still open. We could also accept plain text implementation
reports. The idea of an automated form is an interesting approach,
and I'm interested in experimenting with it.

> ==> 'may' or 'must' ?  If the WGs or individuals don't do those,

I was thinking of the (common) case that an industry or group
outside of the official IETF might sponsor an "interoperability event".

The preparation of the protocol feature set to be tested is part
of the interoperability event planning, and the protocol implementation
reports are prepared (and updated!) on the spot. At least in the
interops I've witnessed, the developers were debugging and fixing
things on the spot.  And while there was substantial participation
by working group members, the leadership and participation in the
interop event was mainly by people who didn't show up at WG meetings
or on the mailing list.

> finer grained format (e.g., MUST, SHOULD, MAY), why shouldn't the interop

I'm not certain about this, but my thought is that the MUST/SHOULD/MAY of
a technical specification doesn't quite match what we're looking for
in the PFS. I'm not even sure the PFS needs to include this information
at all -- you can just go read the spec itself. This was more as a way
of an aid when the technical spec itself was insufficient or ambiguous.
For the purpose of evaluating implementation and interoperability, do
you want to consider interoperability among incomplete implementations?
I think we could accept a PFS that didn't repeat the information, or
one that just identified those features that were mandatory without
getting into the subtleties of MAY vs. SHOULD.

> starting on PFS that they'd fill it, once complete, or ...?

Perhaps this doesn't belong in the document, or needs some explanation?
When evaluating "rough consensus and running code", I've felt that it
was useful to listen a little more carefully to those who had
implementations
and experience with them.  Similarly, when evaluating "rough consensus"
for PFS, I was thinking that the people who really matter are the ones
who will try to fill out the PIR.  So, if you think you have a draft
PFS, and the people testing interoperability complain that they can't fill
it out, you need to fix it. If they're happy with it, and someone else
complains on more theoretical grounds, perhaps you don't.

So, if you have a successful interop event with lots of participants
and they produce their PFS beforehand and a set of PIRs after, that should
be enough. You don't need to "last call" it on the mailing list.

> ==> this doesn't seem to parse.

Sorry, 'details' => 'detailed'.

> reword to clarify, move to a new bullet if general.

I was trying to get at the issue where the author of the implementation
report had some relationship to intellectual property claims for which
independent licensing was being claimed.  (I have a patent and claim
three independent interoperable implementations that have each separately
licensed my patent). Perhaps it's the relationship of the author to
this implementation, any other implementation, or claimed intellectual
property.

>     o  QUESTION: Is it a requirement to allow for anonomous

Yes, thanks.

.