Subject: (ngtrans) RFC 1886 Interop Tests & Results

From: Mohsen Souissi <[email-masked]>
Date: Sun, 14 Jul 2002 13:25:35 +0200

Hello,

Upon request from Alain Durand (Co-Chair of ngtrans wg) and Olafur
Gudmundsson (Co-chair of dnsext wg), 6WIND, AFNIC, France Telecom R&D
and IRISA (as members of G6) organized RFC1886 interop tests in last
June-July 2002.

Three well-known server implementations and 2 resolver implementations
were used.

The results are :
- 2 server implementations are fully interoperable as defined in RFC1886
(section 2 and section 3).
- The two resolvers have identical behaviors and are correct.
- A third server is partially in conformity with RFC 1886: it does not
perform correctly additional section processing (section 3 of RFC1886)
for MX, SRV, NS and SOA .
 
The description and results of the tests are now online:

IPv4 : http://www.6wind.com/RFC1886/testRFC1886.html
IPv6 : http://www.ipv6.6wind.com/standard/testRFC1886.html

All comments/questions (which may be sent to [email-masked]) will be welcome.

Cheers,

Mohsen (for RFC1886 Interop tests team).

P.S. I apologize beforehand for possible multiple copies.

---

From: [email-masked] (Jun-ichiro itojun Hagino)
Date: Mon, 15 Jul 2002 06:35:32 +0900 (JST)

> Three well-known server implementations and 2 resolver implementations

	the co-existence of ip6.int and ip6.arpa tree will require us to:
		query ip6.arpa;
		if (no record)
			query ip6.int;
	for backward compatibility.  was it taken into account, or did you
	test just "ip6.arpa" lookups?

itojun

---

From: Pekka Savola <[email-masked]>
Date: Mon, 15 Jul 2002 08:49:22 +0300 (EEST)

On Mon, 15 Jul 2002, Brad Knowles wrote:
> At 6:35 AM +0900 2002/07/15, Jun-ichiro itojun Hagino wrote:
> 
> 	I checked the source code for BIND 9.2.1, and IIRC it checks 
> ip6.int first and then ip6.arpa second.  This allows us to stand up 
> ip6.arpa whenever, and then once that is set, we can tear down 
> ip6.int.

FWIW, e.g. Linux glibc resolver only checks ip6.arpa now, so you'd better 
start standing up..

-- 
Pekka Savola                 "Tell me of difficulties surmounted,
Netcore Oy                   not those you stumble over and fall"
Systems. Networks. Security.  -- Robert Jordan: A Crown of Swords

---

From: Randy Bush <[email-masked]>
Date: Mon, 15 Jul 2002 14:58:09 +0900

[ namedroppers removed, this is not a dns protocol design issue
  ipng removed, this is not a v6 design issue
  ]

> FWIW, e.g. Linux glibc resolver only checks ip6.arpa now, so you'd better 

zone "frobnitz.ip6.arpa"     {type master; file "frobnitz";};
zone "frobnitz.ip6.int"      {type master; file "frobnitz";};

or, if you demand terminal dots, us sed/awk to dupe after changes

no big deal.  but the clock is running on ip6.int.

randy

---

From: David Terrell <[email-masked]>
Date: Sun, 14 Jul 2002 23:55:48 -0700

> FWIW, e.g. Linux glibc resolver only checks ip6.arpa now, so you'd better 

2.0.0.2.ip6.arpa:  NXDOMAIN
e.f.f.3.ip6.arpa:  NXDOMAIN

That's probably 70-80% of all IP6 deployments reachable via the
public ipv6 internet (granted, especially most 2002:: folks don't
have reverse DNS set up yet, but plenty do...).  It would be
reasonable for glibc to at least make fallback to ip6.int an option...

-- 
David Terrell             | "Any sufficiently advanced technology 
Prime Minister, Nebcorp   | is indistinguishable from a rigged demo."
[email-masked]              |  - Brian Swetland
http://wwn.nebcorp.com/

---

From: Ólafur Guðmundsson<[email-masked]>
Date: Mon, 15 Jul 2002 10:57:28 -0400

At 09:22 PM 7/14/2002, Danny Mayer wrote:
>At 07:25 AM 7/14/02, Mohsen Souissi wrote:
>>Hello,
>>
>Two comments:
>1) The data is most peculiar since nowhere that I could find are X, Y and Z
>identified.  If they are somewhere I couldn't find it. Is there a reason 
>to keep
>them obscured?

This is standard procedure in DNSEXT interop testing and reporting.
The vendors have access to the full reports and I have been in contact
with the vendor that "failed" additional section processing and they
will fix this in their next release.

         Olafur

---

From: Randy Bush <[email-masked]>
Date: Tue, 16 Jul 2002 08:08:27 +0900

>> to keep them obscured?

s/DNSEXT/IETF/

the point is that the vendors' products are not what is being tested.  what
we are testing is the *spec* by testing if multiple implementors interpreted
it sufficiently similarly to create interoperable implementations.

randy

---

From: Bill Manning <[email-masked]>
Date: Tue, 16 Jul 2002 16:55:16 -0700 (PDT)

% 	Moreover, if you are someone looking to actually use products 
% such as the ones under test, it would be a benefit to know which 
% products worked and which ones didn't, or which ones had what 
% problems in what environments, so that you would have a better idea 
% as to what products might be suitable for use in your own environment.
% 
% 
% 	IMO, tests like this without full disclosure are meaningless.
% 
% -- 
% Brad Knowles, <[email-masked]>
% 

	Brad, at this point most of this work is still
	shaking out specification ambiguities.  It is
	useful to know which developers are involved
	with debugging the spec but details on the specific
	code/thinking that will be revised may not be
	that useful.

	When products are available -and- the spec is firm,
	then having interop results, OF THE PRODUCTS, makes
	sense.

--bill

---

From: Robert Elz <[email-masked]>
Date: Wed, 17 Jul 2002 09:13:29 +0900

Date:        Tue, 16 Jul 2002 23:17:57 +0200
    From:        Brad Knowles <[email-masked]>
    Message-ID:  <a05111b0eb95a3d13cbc1@[10.9.8.228]>

  | At 12:23 PM -0700 2002/07/16, Ed Sawicki wrote:
  | 
  | >  Yes, but isn't there value in knowing who the implementors are so

Sometimes a list of who participated is published, without
identifying which implementation belongs to which implementor.

  | 	Moreover, if you are someone looking to actually use products 
  | such as the ones under test,

This is not the purpose of the tests - that's a worthy purpose, but it
isn't the IETF's role.   What is being tested, as Randy Bush keeps on
saying, by the IETF is the standard upon which the implementations are
based.

We want to know if the standard describes something that can be implemented
in an interoperable way.   When a failure occurs, then we need to know
whether of not that is because of a faulty standard, or just an implementation
bug ("the standard was clear, I just screwed up...").  Apart from that,
whether bugs exist or not is irrelevant - and the tests are not designed
to attempt to discover implementation bugs - so that bugs weren't found in
some implementation or other doesn't mean that there weren't any.

kre

---

From: David Conrad <[email-masked]>
Date: Tue, 16 Jul 2002 18:44:02 -0700

Ed,

On 7/16/02 12:23 PM, "Ed Sawicki" <[email-masked]> wrote:
> Yes, but isn't there value in knowing who the implementors are so
> we can gauge what skill levels are required to produce interoperable
> implementations?

How do you gauge skill level?  How does an external body know how many
resources were put onto the implementation of a new protocol?

> Besides, what's the reason for the secrecy?

Many reasons.  For one, a company may not want to advertise the fact that
they are working on a particular technology (for whatever reason).  For
another, people might not want an experiment to be associated with what they
do for their products.  I'm sure you can think of other reasons.

> I, for one, get suspicious when information is deliberately
> withheld.  It tells me that there's something to hide.

Well, yeah.

> This may seem like a small thing but I think it sets a
> dangerous precedent. The standards process should be completely
> open. 

The standards _are_ open.  The implementations of those standards don't have
to be (and typically aren't).

> If IETF working groups can't be completely open about
> important standards activity, they should be dissolved and new
> ones formed that have more respect for their public trust.

Interesting view.  Wrong, but interesting.  Of course, interoperability
testing isn't, strictly speaking, a function of the IETF working group (as
Randy Bush has pointed out).  The point of interoperability testing as it is
relevant to the IETF is to insure the protocol specifications are actually
implementable.  From that perspective, it doesn't matter who does the
implementation, rather that at least two different sets of people can
actually do them.

Rgds,
-drc

---

From: David Conrad <[email-masked]>
Date: Tue, 16 Jul 2002 18:50:51 -0700

Brad,

On 7/16/02 2:17 PM, "Brad Knowles" <[email-masked]> wrote:
> Moreover, if you are someone looking to actually use products
> such as the ones under test,

Who says the things under test are products?  The DS implementation Nominum
did was done on a snapshot of the ISC BIND tree that has not yet been
released, even in alpha.  It shouldn't be too much of a surprise that that
version of BINDv9 contains bugs.  The implementation was done to prove that
the DS spec was actually implementable and to provide input back to the
standardization proceess to tighten up the ambiguous bits.

> it would be a benefit to know which
> products worked and which ones didn't, or which ones had what
> problems in what environments, so that you would have a better idea
> as to what products might be suitable for use in your own environment.

When products are available, such testing would indeed be useful.  Of
course, this isn't particularly relevant to what the IETF is doing.

> IMO, tests like this without full disclosure are meaningless.

To be blunt, your opinion would be wrong.  Of course, there is meaning --
just the fact that multiple implementations of a protocol exist is useful
information (indeed, an IETF protocol cannot advance to full standard
without this fact being true).

Rgds,
-drc

---

From: David Conrad <[email-masked]>
Date: Wed, 17 Jul 2002 04:32:12 -0700

Hi,

On 7/17/02 2:42 AM, "J-F C. (Jefsey)  Morfin" <[email-masked]>
wrote:
> The frustration results from an uncompleted report.

It is complete as it needs to be to meet IETF requirements.

> The target is to 
> demonstrate that different thinking families can have the same reading of
> the specs and can develop from them compatible softwares.

Right.

> The  bugs may 
> results from unclear specs or from an early development phase. That we need
> to know.

What a working group needs to know is whether or not a specification has
sufficient detail for two independent developers to develop something that
interoperates.  If the implementations do not interoperate, then the testers
generally (albeit not necessarily) take on the responsibility of figuring
out why and proposing revisions to the spec so that interoperability can be
achieved (after all, why would they bother implementing if they didn't want
to interoperate?).  If interoperability is not achieved, the spec doesn't
move forward.

Rgds,
-drc

---

From: Ólafur Gudmundsson/DNSEXT co-chair <[email-masked]>
Date: Thu, 18 Jul 2002 11:04:52 -0400

At 05:42 AM 7/17/2002, J-F C. (Jefsey)  Morfin wrote:

>The frustration results from an uncompleted report. The target is to 
>the specs and can develop from them compatible softwares. The  bugs may 
>results from unclear specs or from an early development phase. That we 
>need to know.
>
>The name of the participants would only help knowing the X, Y, Z 
>architectures, used libraries, concept affiliations, "style", etc..  To 
>evaluate if the specs testing spectrum is significant enough. But the 
>better, easiest and common way would be that each participant describes 
>his approach in his own terms (so there is no confidentiality break). IMHO 
>this is basic to any hidden testing protocol.

Just for the record, THIS interoperabilty test was performed without
any vendor participation, thus it is real hard for the report to
talk about approaches or reasons why.

The real important thing here is there are three implementations
that work. This enables the DNSEXT working group to advance RFC1886bis to
Draft Standard.
The fact that one implementation forces services (mail, NS, ..) to
use IPv4 transport when IPv6 transport is available,
is not critical as the information is available via explicit AAAA query.

         Olafur

---

From: Randy Bush <[email-masked]>
Date: Fri, 19 Jul 2002 07:21:49 +0900

> the specs and can develop from them compatible softwares.

i missed the part about "different thinking families" in 2026.

in the ietf, we still occasionally try NOT to make things more complex
than they must be.

randy

---

From: JINMEI Tatuya / 神明達哉<[email-masked]>
Date: Tue, 23 Jul 2002 19:44:17 +0900

>>>>> On Mon, 15 Jul 2002 01:10:44 +0200, 
>>>>> Brad Knowles <[email-masked]> said:

> ip6.arpa whenever, and then once that is set, we can tear down 

What exactly do you mean by BIND 9.2.1?

1. the resolver library under lib/bind
2. the resolver routine in lwresd
3. both 1 and 2
4. others

In my understanding (I've quickly checked the code again, too), both 1
and 2 only tries ip6.arpa with bitstring labels.

					JINMEI, Tatuya
					Communication Platform Lab.
					Corporate R&D Center, Toshiba Corp.
					[email-masked]