Subject: Re: axfr-clarify's fraudulent claims of consensus

From: Dean Anderson <[email-masked]>
Date: Fri, 14 Feb 2003 17:25:40 -0500 (EST)

On Fri, 14 Feb 2003, Andreas Gustafsson wrote:
> Dean Anderson writes:
> > I don't think they can be dismissed so easilly. Indeed, I think the
> > proposed changes are rather "oddball", and are actually unncessary in
> > order to implement IXFR (which was the original claimed justification).
>
> This is not a "change".  There are no existing standards which mandate
> or even explicitly allow mixing of data between zones.  On the
> contrary, RFC1035 specifically suggests using separate data structures
> that ensure that no such mixing occurs:

The key word is "suggests", which has the meaning...

> Which one is more "oddball", a server which follows this suggestion or
> one which does not?

Neither. It doesn't matter what the implementation is. What matters is the
protocol. Despite the "suggestion", all implementations prior to Bind 9
made different (but ultimately identical) assumptions. I don't think the
section you quote is so compelling.

The "oddball" part is requiring a change to a irrelevant part of a
protocol.  This is nothing less than bad or lazy engineering.** As I
mentioned previously, IXFR is in principle no different from an Oracle
incremental replication. Clearly, one does not need to alter SQL to have
replicated databases.  One _could_ alter SQL, but it would be a "bad
idea".  The concepts of replicated databases, and different kinds of
replication is well researched, and well documented.  There isn't any
rocket science to IXFR, nor are there any theoretical papers to be written
on the subject. The AXFR protocol does not _need_ to be changed for IXFR.
IXFR can be made to work with AXFR the way it stands.

** a collection of good engineering principles:

A "good" engineer avoids making gratiutous changes.

A "good" engineer, thinking a significant change to a standard of
interoperability needs to be made, will seek consensus from the standards
body on the change before distributing it.

A "good" engineer, upon finding a language ambiguity or vagueness, would
investigate how others analyzed the language, what assumptions they made
about the ambiguity and follow accordingly, or discuss and resolve the
ambiguity if there are no concrete implementations.

A "good" engineer, realizing an incompatible change has to be made to a
standard of interoperabilty, would consult the standards body prior to
publishing a (non-interoperable) book on the subject.

A "good" manager, would ensure that engineers are making "good"
engineering decisions.

Pretty clearly, the Bind 9 group didn't exercise "good" engineering
principles, nor "good" engineering management.  Quite obviously, they made
a mistake altering and distributing AXFR prior to standardization, or by
exploiting the vagueness in the specification. Clearly, they knew about
the incompatibility issue.  One must wonder if anyone could be so arrogant
as to assume that they could push anything they pleased through the
standardization process.  We probably can't know the answer. I don't know
anything about their intentions and can only fairly judge the results of
their actions. However, there is much to criticize about the actions.  I
do not think I am being unfair in that criticism. I also realize that Bind
9 was a group effort, and there is blame to spread among many.

However, I do not want to pay for that mistake. Nor do others.

> > Indeed, the proposed changes are only necessary to justify BINDs' oddball
> > IXFR changes. They are only necessary to make BIND 9 compliant with the
> > RFC's.
>
> BIND 9 is compliant with the existing AXFR standard even without
> axfr-clarify.  It's not hard to comply since the existing the
> specification is almost nonexistent.

No. This isn't true either.  If we clarify AXFR to standardize the
_assumptions_ identically and concretely made by many implementors over a
long period of time, then BIND 9 will not be compliant**.  Bind 9 is
"compliant" only with the vague AXFR spec by invirtue of its vagueness.
AXFR should be clarified. I (and others) think it should be clarified to
follow the concrete assumptions made by many implementors over many years.
This will only make BIND 9 non-compliant.

Then 23% of the servers will have to change. 77% will be unaffected.
**Actually, it might not even be this bad, as I think there are some
"backwards compatibility" functionality that current Bind 9 users can
utilize. So perhaps no servers need changing if we standardize AXFR as I
propose. Except that Bind 9 will have to withdraw IXFR support until it
fixes it to work with AXFR as clarified, or make it synonomous with AXFR.

> > It seems quite odd that a "clarification" would put 77% of the existing
> > servers out of compliance,
>
> 77% of the existing servers do not reliably interoperate in IXFR (even
> with one another), and sometimes serve data inconsistent between the
> authoritative servers of a zone.  They *should* be declared out of
> compliance.

No, because IXFR support is essentially optional. An IXFR server can
return the whole zone (AXFR) if it doesn't support IXFR.  A server can be
compliant with RFC 1995 just by making IXFR a synonym for AXFR. That's
probably a one line change for most if not all implementations. Further,
most people are not using IXFR.

AXFR is not optional. Most people are using AXFR.

> > and only brings into compliance a currently
> > non-compliant implementation.
>
> Again, BIND 9 is currently compliant.

Bind 9 won't be compliant (modulo its "backward compatibilty") after
clarification of AXFR to specify the commonly assumed and commonly
implemented behavior for AXFR.

> > Consensus on the part of Bind proponents, yes. Consensus when the wider
> > community is counted, no.

> I have been reluctant to bring up this issue because because it should
> make no difference whatsoever - the draft should be judged solely on
> its technical merits, not based on the affiliations of its proponents,
> but the groundless accusations of misconduct and conspiracy theories
> by you and Dr. Bernstein have gone too far.

I have made no allegations. My position on the issue of "conspiracy and
misconduct" is that allegations made against Dr. Bernstein need to be
investigated, and the allegations made by Dr.  Bernstein also need to
investigated.  I think the allegations made by both sides are sufficiently
specific that a determination can be made as to their truth.

There are many on the djbdiscuss list who seem to think the issue deserves
investigation. (I will try to get back to that matter next week, and work
with others to come to a conclusion about which accusations were credible
and which accusations weren't.)  Clearly, some of the allegations made go
way too far.

		--Dean

--
to unsubscribe send a message to [email-masked] with
the word 'unsubscribe' in a single line as the message text body.
archive: <http://ops.ietf.org/lists/namedroppers/>

---

From: Kevin Darcy <[email-masked]>
Date: Fri, 14 Feb 2003 18:38:51 -0500

> IXFR can be made to work with AXFR the way it stands.

"Can be made to work"? You mean, through some protocol clarification? If
clarifying *either* standard can effect interoperability, why give preference to
clarifying IXFR over clarifying AXFR? AXFR/IXFR interoperability issues aside,
AXFR was so poorly specified in the first place it seems to me a far better
candidate for clarification.

> long period of time, then BIND 9 will not be compliant

Funny, I thought the job of the IETF was to *lead* the development and
establishment of Internet standards. You seem to see their job as merely that of
a historian/archivist of whatever _de_facto_ standards implementors throw out
there, no matter how inefficient/convoluted/non-interoperable those _de_facto_
standards happen to be.

> AXFR is not optional. Most people are using AXFR.

Speaking as the architect of a large DNS infrastructure, AXFR is becoming less
and less feasible for us as time goes by, because of scaling issues. We need
IXFR ASAP, but we also need good interoperability between AXFR and IXFR because
it's not like we can migrate from one to the other overnight. IXFR may
technically be "optional" to implement, but in the real world, IXFR or something
like it, is quickly becoming a practical necessity. This is because of the
increasing size of zones and the frequency with which those zones are being
updated.

Dean, if you're willing to pony up the thousands of dollars we waste every year
because of inefficient zone transfers, then fine, we'll stick with AXFR
indefinitely. I can give you the address to which to mail the checks. But I'll
be damned if we continue to throw money down the toilet just because you and djb
and a handful of others want to maintain compatibility with some nebulous
_de_facto_ standard for AXFR.

- Kevin

---

From: Rob Austein <[email-masked]>
Date: Fri, 14 Feb 2003 19:11:40 -0500

> section you quote is so compelling.

Sorry, wrong.  Both JEEVES and CHIVES had very strict notions of zone
boundaries, and both enforced those boundaries during zone operations.

In CHIVES's case the implementation exactly matched the suggestion
that Andreas quoted from RFC 1035 6.1.2.  I no longer remember the
precise implementation details in JEEVES's case, but since JEEVES was
written by the author of RFC 1035, I would be rather surprised if the
author had not followed his own suggestion.

---

From: [email-masked] (Andreas Gustafsson)
Date: Fri, 14 Feb 2003 17:57:33 -0800 (PST)

Dean Anderson writes:
> Despite the "suggestion", all implementations prior to Bind 9
> made different (but ultimately identical) assumptions.

Rob Austein already pointed out JEEVES and CHIVES as counterexamples,
and before I worked on BIND, I wrote an authoritative-only server
which also used separate data structures for each zone.  When I
started working on BIND 9, the BIND 9 team had already independently
made the same design decision.  To both me and the BIND 9 team, it was
the obvious design, and the BIND 8 behavior was an obvious
implementation bug.

> This is nothing less than bad or lazy engineering.** As I
> idea".

If a version of Oracle had a bug that caused tuples to "leak" between
tables under certain conditions and the bug was only noticed when it
broke incremental replication, would you fix the bug or try to
redesign the replication protocol around the bug?  Would you do
differently if 77% of deployed Oracle servers had the bug?

> [more gratuitous insults that I'm not going to respond to]

> So perhaps no servers need changing if we standardize AXFR as I
> propose.

No servers will need changing if we standardize AXFR as I propose,
either.  There is no AXFR interoperability issue either way.  The
only difference (other than IXFR working better) is that a different
set of servers is declared non-compliant.
-- 
Andreas Gustafsson, [email-masked]

---

From: Paul Vixie <[email-masked]>
Date: Sat, 15 Feb 2003 02:51:56 +0000

> ... made the same design decision.  To both me and the BIND 9 team, it
> was the obvious design, and the BIND 8 behavior was an obvious
> implementation bug.

i just want to say, bind8's treatment of zone boundaries is terrible,
even though it's better in a dozen small ways than bind4's from which
it came.  my advice is, if you use bind, upgrade to bind9.

> set of servers is declared non-compliant.

agreed.  i am in strong support of the gustafsson axfr-clarify draft.

---

From: "D. J. Bernstein" <[email-masked]>
Date: 15 Feb 2003 03:41:14 -0000

Andreas Gustafsson writes:
> To both me and the BIND 9 team, it was the obvious design, and the
> BIND 8 behavior was an obvious implementation bug.

To me, the djbdns cache structure is the obvious design, and BIND's
continuing tree-related cache performance problems---all of which are
visible to clients on the wire---are obvious implementation bugs.

Do you think I should write drafts specifying my cache structure,
falsely label those drafts as ``clarifications,'' and pay a bunch of
people to push those drafts through the IETF standardization process?

---D. J. Bernstein, Associate Professor, Department of Mathematics,
Statistics, and Computer Science, University of Illinois at Chicago

---

From: Robert Elz <[email-masked]>
Date: Sat, 15 Feb 2003 15:20:46 +0700

Date:        Fri, 14 Feb 2003 17:25:40 -0500 (EST)
    From:        Dean Anderson <[email-masked]>
    Message-ID:  <[email-masked]>

  | 
  | On Fri, 14 Feb 2003, Andreas Gustafsson wrote:
  | > RFC1035 specifically suggests using separate data structures
  | > that ensure that no such mixing occurs:
  | 
  | The key word is "suggests", which has the meaning...

exactly what you think it does - but note, the suggestion relates to
an implementation technique, which certainly isn't going to be required.
Implementors are free to implement however they desire, as long as they
achieve the objective.

Now ask yourself why 1035 would bother even suggesting an implementation
technique.

Don't you think it entirely possible that it is because the requirement
(that which was to be implemented) is so fundamental to the design of
the DNS, that some kind of suggestion of how to actually comply with the
design was worth making?

Implementation methods are often "suggested" in circumstances like that.
That is, when it seems plausible that implementors might miss an important
point and just do it the "easy way", it is common for the spec to indicate
how the implementation can achieve the desired result - this way implementors
either just blindly copy the suggestion, or at least should think about
why they're doing it a different way, and ask themselves if their results
will be the same as the suggested method.

  | No. This isn't true either.  If we clarify AXFR to standardize the
  | _assumptions_ identically and concretely made by many implementors over a
  | long period of time,

I can absolutely guarantee to you that that will never happen.  No-one is
going to standardise a protocol that essentially says "send whatever data
you feel like sending, and believe whatever the other end sends you
without question" which is exactly what the early implementations did.

When we standardise something, we want to first be as sure as we can be
that the protocol is correct for its purpose.  We aren't always very good
at that, but that should certainly be a major aim - documenting something
that doesn't work is a waste of everyone's time.   AXFR as it has been
commonly implemented simply doesn't work reliably.

kre

---

From: Josh Littlefield <[email-masked]>
Date: Tue, 18 Feb 2003 17:07:46 -0500

I'd like to make clear my support for draft-ietf-dnsext-axfr-clarify-05. 
With the exception of a single subtle enhancement (the specification of 
returning NOTAUTH to an AXFR request about a zone for which the server is 
not authoritative), I find that the specification correctly matches my own 
interpretation of RFC1034 and RFC1035 and first-hand interoperability 
experience.

Regarding zone integrity, I find the specification supports the clear 
requirements for the independence of zone data described in RFC1034 
(sections 4.1 and 4.2), along with the practical need to ensure that all 
authoritative servers for a zone will have an accurate copy of the data for 
that zone for a given serial number (RFC1034, Sect. 4.3.5).

As for the requirement to carry zone transfer RRs in the answer section, 
this is a completely obvious application of the rules governing all 
responses to requests with opcode QUERY (0).  RFC1034 clearly states that 
the answer section "carries RRs which directly answer the query."  I don't 
see how anyone could conclude that AXFR clients should look for zone RRs 
anywhere else.

I implemented AXFR in 1997 on a server architected to clearly differentiate 
each zone's data, as specified (according to my reading) in RFC1034.  The 
axfr-clarify draft is consistent with that implementation, whose basic 
architecture has not changed.  This made the implementation of RFC1995 
relatively trivial, because the consistency of the baseline AXFR allows the 
unambiguous communication of incremental changes to that baseline, even if 
the AXFR has been retrieved from one authority, and the IXFR from another, 
and regardless of what other zones those servers may know about.

Regards,
  Josh Littlefield
-- 
=====================================================================
Josh Littlefield                                  Cisco Systems, Inc.
[email-masked]                                      250 Apollo Drive
tel: 978-497-8378  fax: same               Chelmsford, MA  01824-3627

---

From: "D. J. Bernstein" <[email-masked]>
Date: 19 Feb 2003 06:58:22 -0000

Josh Littlefield writes:
> RFC1034 clearly states that the answer section "carries RRs which
> directly answer the query."  I don't see how anyone could conclude
> that AXFR clients should look for zone RRs anywhere else.

There is overwhelming consensus that (in the absence of extended
behavior requested by the client) the AXFR _server_ must leave the
additional section and authority section empty. It must put the data
into the answer section. All existing servers work this way---and must
do so for interoperability.

There are several perfectly valid parsing strategies for the client.
In particular, my software uses the simple strategy

   while there are records left
      read a record

while BIND 9 uses the slightly more complicated strategy

   while there are records left in the answer section
      read a record

Both strategies work, since servers put all records into the answer
section. Neither strategy creates any interoperability problems.

There is certainly _not_ consensus on telling the _client_ to use the
BIND 9 strategy. Furthermore, any such statement (whether ``MUST'' or
``SHOULD'') would blatantly violate RFC 2119, section 6.

This is about the tenth time that I have had to point out the blazingly
obvious fact that constraining _server_ behavior is not the same as
constraining _client_ behavior.