Subject: RE: [newtrk] draft-ietf-newtrk-interop-reports-00.txt

From: "Larry Masinter" <[email-masked]>
Date: Wed, 19 Oct 2005 10:43:08 -0700

>>2. You don't tackle one of the thorny problems - how to
>>handle "single ended" specifications such as MIBs, which don't
>>in themselves define protocol exchanges.

Many specifications have multiple roles. For network protocols,
sometimes there are peers, sometimes there are clients and servers,
or proxies or other kinds of agents. For APIs, there are usually
systems that provide the API and other systems that consume it.
For MIME types and other data formats, there are "readers" and
"writers". For MIBs, it sounds like there are SNMP agents (which
provide the MIB) and SNMP management stations (which consume the MIB).

Certainly section 2 bullet 5 should be expanded to include
some guidance about identifying roles for different kinds
of specifications.

Perhaps this is a case where the consumer (the SNMP management
station) has a minimal implementation role, and interoperability
testing with multiple producers (SNMP agents) is a manual inspection
of the results. (This seems to be case A.1 in RFC 2438)?

Perhaps there might be a high-level description of the
nature of the interoperability testing for the implementation.

(FWIW, I'm puzzled by RFC 2438 about how little it actually says, as
far as giving guidelines as to what constitutes an adequate
interoperability report. Appendix A contains some suggestions,
without coming out and saying that they are adequate. There's
a guideline that "basic object comparison testing" is normally
included, and that you have to find at least one person
who is willing to argue that interoperability testing is adequate.)

Larry
-- 
http://larry.masinter.net

.
newtrk resources:_____________________________________________________
web user interface: http://darkwing.uoregon.edu/~llynch/newtrk.html
mhonarc archive: http://darkwing.uoregon.edu/~llynch/newtrk/index.html

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 20 Oct 2005 07:27:33 -0700 (PDT)

> of the results. (This seems to be case A.1 in RFC 2438)?

That indeed seems to be more-or-less what RFC 2438 assumes.  And it
is true for MIB modules that contain nothing but read-only objects.  
However, many MIB modules contain read-write and/or read-create
objects, and some contain notifications.  Basic object comparison as
specified in A.1 of RFC 2438 -- i.e., reading the results of a MIB
walk -- will not tell you whether the agent allows objects to be
written or created, nor will it tell you whether the agent implements
notifications.

> who is willing to argue that interoperability testing is adequate.)

If you peruse the MIB module implementation reports that are
available at http://www.ietf.org/IESG/implementation.html you will
find that they vary greatly.  Two of the most complete are the
implementation report for RFC 1757 (Remote Network Monitoring
Management Information Base) and the one for RFC 1514 (Host
Resources MIB).  The report for RFC 1757 includes not only vendor
reports of implemented objects and groups but also a manager/agent
interoperability matrix.  It does not, however, include any
references to basic object verification via MIB walks; those details
were left up to the vendor.  The report for RFC 1514 does include
that information.  On the other hand, the report for the IF-MIB
(http://www.ietf.org/IESG/Implementations/ifmib-implementation.txt)
just compiles information from vendor reports of groups/objects
implemented in their agents;  there was no information on
manager/agent interoperability, nor is there a statement that MIB
walks were perused to verify the reported information.  The
AToM MIB WG used the IF-MIB model when it compiled the
implementation reports for RFCs 2558 and 2493.

//cmh

.