Subject: Re: [dnsext] Fwd: New Version Notification for draft-kerr-ixfr-only-01

From: Shane Kerr <[email-masked]>
Date: Tue, 02 Mar 2010 11:41:36 +0100

Ohta-san,

On Sat, 2010-02-27 at 07:37 +0900, Masataka Ohta wrote: 
> > we have incorporated all comments on -00.
> 
> As the client SHOULD choose a server with the newest serial number,
> there actually are not much room for server selection, which limits
> applicability of your proposal.

This is an interesting assertion. It's not a bad idea, but is this
written down anywhere?

Of course, there are other algorithms which may be employed. For example
a client could always try the server that sends a NOTIFY (which seems
quite reasonable), instead of querying every server in the NS RRSET.
This may actually speed up zone propagation in the average case, on the
modern Internet.

> >    If the IXFR server is is not able to reply with an IXFR it MUST NOT
> >    reply with an AXFR unless AXFR result is smaller than IXFR result.
> 
> Though RFC1995 is not well written, its behaviour here is subtly
> different between UDP and TCP and you should assume UDP.
> 
> Then, answer will unlikely be a entire zone content. You usually
> get an SOA or a difference sequence.

There is still no way for the client to know whether it will get
differences or the full zone contents when it asks any particular server
via TCP though, which is the problem.

> Moreover, regardless of transport, QTYPE in answer of entire
> zone is IXFR, not AXFR.

I think originally I wrote something like "AXFR-style", perhaps that is
more appropriate.

> >    Instead, it MUST reply with RCODE CannotIXFR. (!FIXME)
> 
> No SOA serial number in the reply, even though the server may
> have the newest?

I considered this, but I didn't see how the client would change its
behavior based on this information. If it will, please explain it.

> >    If the IXFR result is larger than an AXFR, then an IXFR server MAY
> >    reply with an AXFR result instead.  This is an optimization, and IXFR
> 
> The optimization is specified in RFC1995:
> 
>    Information about older versions should be purged if the total length
>    of an IXFR response would be longer than that of an AXFR response.
>    Given that the purpose of IXFR is to reduce AXFR overhead, this
>    strategy is quite reasonable.  The strategy assures that the amount
>    of storage required is at most twice that of the current zone
>    information.
> 
> in a more useful way to bound server storage without your confusion
> on QTYPE.

The intention is not to save server storage, but rather to minimize the
size of information transferred, saving bits on the wire and the time
needed to transfer them.

The optimization of deleting old versions usually makes sense, but not
always. It may fail in the presence of compressed IXFR. With compressed
IXFR, the size of the differences depends on which version of the zone
the client has. If one is using compressed IXFR, then a version covering
2 serial numbers may actually be smaller than a version for 1 serial.
(As a real-world example, a DHCP lease that is bound and then released
may result in a compressed IXFR that has *no* changes, since the records
are added then removed.)

I am not a mathematician, but I believe there is no general-case
assertion you can make in the presence of compressed IXFR about when you
will be able to delete an old version of a zone and always save space.
Hence the suggestion to make this determination an run-time.

--
Shane

---

From: Masataka Ohta <[email-masked]>
Date: Wed, 03 Mar 2010 06:19:08 +0900

> This is an interesting assertion. It's not a bad idea, but is this

It's your assumption that many servers should be tried, though
it is not required anywhere.

Then, there is no point to have XFR not from the server with
newest content.

> For example
> quite reasonable),

Then, you are not allowed to search a IXFR-capable server.

> instead of querying every server in the NS RRSET.

That's not a behaviour of XFR clients.

You should investigate realities of not only XFR protocols but
also XFR management before writing drafts on XFR. A lot is
already written in RFCs including 103[45].

> via TCP though, which is the problem.

As managers of the IXFR-capable XFR client should know which XFR
servers are IXFR capable and configure XFR sources appropriately,
there actually is little problem to worry about at the protocol
level.

> behavior based on this information. If it will, please explain it.

See the first paragraph of this mail.

> size of information transferred, saving bits on the wire and the time

Description in 1995 covers both, which is why your description is not
only redundant but also less useful.

> always.

I know. That's why it is optional.

							Masataka Ohta

---

From: Brian Dickson <[email-masked]>
Date: Tue, 2 Mar 2010 20:44:01 -0400

I think what we have here, is a failure to communicate (about,
ironically, failure to communicate). :-)

Specifically, there is more than one way to operate a set of DNS
servers, administratively speaking.

In some scenarios, it may be a design goal that some secondary servers
are not able to talk directly to the primary (master) server.

Not everyone agrees that this is desirable, but I think we can all
agree that it is a valid topology, one which in turn leads to the
desire to handle discrepancies between zone IXFR capabilities (or
results of IXFR attempts).

What is not (yet) documented in RFCs for DNS, at least that I'm aware
of, are specific strategies or requirements for which server(s) an XFR
client should query, to do an XFR.

IMHO, here's why I think not:

First, there is the data on nameservers present (at the delegation, or
at the apex) in the zone data itself - NS records.
This tells outside parties who is supposed to be authoritative for the
zone, i.e. what servers to query for the zone RRs.

Second, contrast this with zone operator configuration files, a sort
of out-of-band metadata, telling secondary servers where to get their
zone data, by way of XFR.
Note that this is operator/zone specific, not even
implementation-mandated, let alone protocol-dictated.

In any case, the language in RFC1995 only refers to "a primary or
secondary" server, which implicitly allows for XFR chains like
"primary->secondaryX->secondaryY->secondaryZ".
It also is silent on multiple such XFR paths from "primary" to
"secondaryZ", e.g. for redundancy.

So, basically, the choices of client -> server are out of scope for
protocol work, necessarily. All possibilities need to be considered as
requirements for the protocol.

Note also, that the unreachability (between some secondaries and the
primary) may not be by design, but could in fact be by accident, e.g.
natural disaster or other major failure.

On to document-specific issues (draft-kerr-ixfr-only and RFC1995):

The current UDP->TCP fallback for IXFR (because IXFR is possible, but
the IXFR won't fit in a single UDP packet) is indistinguishable from
UDP->TCP fallback from IXFR query to AXFR response (because no IXFR is
possible, and AXFR won't fit in a single UDP packet.)

This itself is probably enough justification for IXFR-ONLY, and enough
to dictate behavior.

NB - Focusing on the TCP issue may simplify the draft, and ignoring or
allowing UDP AXFR (optimization or otherwise necessary) i.e. by
removing the relevant paragraph (paragraph 2 of section 2.)

E.g. if an AXFR answer will fit in a single UDP packet, perhaps that
answer should be allowed as an answer to IXFR-ONLY, regardless of
*why* (no IXFR possible, or AXFR-size < IXFR-size).

However, for completeness, and in particular to handle environments
where UDP is not available, the requirement of IXFR-ONLY over TCP is
pretty clear. Doing an IXFR query over TCP is dangerous simply because
the AXFR fall-back is not under the client's control.

And yes, while RFC1995 does discuss the benefits of discarding older
IXFR data, it is not deterministic in methodology, and does not
require deterministic discard.
So, two secondary servers could both be discarding older records
associated with particular SOA serial numbers, but do so with
different intermediate serial number data being kept.

E.g. server A has serial numbers 3, 7, 11, 15, and 16, server B has
serials 1, 5, 9, 13, and 16, server C has 4, 8, 12, and 16, and server
D has serial numbers 2, 6, 10, 14, and 16.
Observer, that each of these 4 servers has much less data, the AXFR
might be larger than UDP size, and only one of the 4 servers is
capable of an IXFR of any serial between 1 and 15, to 16.
In all cases, a UDP IXFR to the other three servers will fail (because
it doesn't have the older SOA data), and the client would not be able
to know apriori that one of the 4 would be able to IXFR.
If the AXFR is much larger (i.e. orders of magnitude more), then there
is significant reason to be interested in avoiding AXFR.

And, out-of-band methods do not have the atomic benefit of IXFR-ONLY.
If IXFR-ONLY succeeds, the XFR has occurred. Any SOA query followed by
an IXFR, is still a race condition, which can still result in a
fall-back to AXFR.

My two cents...

Brian

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 04 Mar 2010 08:42:14 +0900

> ironically, failure to communicate). :-)

It's a failure of your side to know communication done long ago.

Most, if not all, of point you raised was considered and RFC1995
was written with wording consistent to RFCs103[45] still not to
be an obstacle for practical DNS use at that time (and today).

Worse, most of your points are irrelevent to ixfr-only proposal.

> different intermediate serial number data being kept.

In theory, yes. In practice, the possibility is negligible.

If IXFR is properly used, most clients should have recent zones
and most servers should have full history of recent updates.

> And, out-of-band methods do not have the atomic benefit of IXFR-ONLY.

Wrong. As the client should try all the servers using UDP, until
it find a server with newer and the newest zone (and ixfr-capable
with difference sequence to clients' zone, if you insist on
ixfr-only), there is no atomicity.

It is the assumption of ixfr-only proposal to search all the
servers until it find a server with a positive answer to ixfr-only.

Moreover, as the client periodically checks all the servers almost
always resulting in reply with no update, there is no atomicity.

That's the practice of zone transfer.

						Masataka Ohta

---

From: Brian Dickson <[email-masked]>
Date: Wed, 3 Mar 2010 22:07:31 -0400

> In theory, yes. In practice, the possibility is negligible.

It is not so much a question of theory vs practice, as it is one of
by-design vs by-accident.

A purpose-built (or -modified) server could very reasonably support
this kind of differential behavior.

The purpose would be to maximize the probability that some server out
of a set would have an IXFR-capable payload, for a larger range of SOA
serials.

Absent such a design, i.e. if every such server had identical zone
history, it would be all-or-nothing.

There are example zones where AXFR is always far from optimal. Most
TLDs, for example.

> ixfr-only), there is no atomicity.

Let me lay out the assumptions in the example above.
(1) In no cases, can an IXFR be done over UDP, as the size(IXFR) >> 512B
(2) Thus, in all cases, UDP will not give an answer beyond the SOA.

This means that an IXFR is only possible using TCP.

However, an IXFR query over TCP, if the server has SOA serial > client
SOA serial, will always give an XFR answer. It will either be IXFR or
AXFR.

The client can know (by doing UDP query) whether the server has a
newer zone (and even which of the servers has the newest zone).

The client cannot know if an IXFR is possible, without risking an AXFR.

A query for IXFR-only would not have that risk.

Consider zones where size(IXFR) is approximately 1/10000 of
size(AXFR), for short intervals (T < 4 hours, for instance).
Clearly, the benefits of IXFR are considerable, and the effort to
increase the probability that IXFR is possible pay large dividends.
This is only beneficial if the "correct" server can be used, if not
all servers have the same IXFR sets available.

> servers until it find a server with a positive answer to ixfr-only.

No, it *tries* to find such an answer, and if it fails, it falls back
to the standard IXFR (which itself may fall back to AXFR).

It, like IXFR before it, is an optimization, one which has benefit,
and for which there is no simple alternative.

(In order to achieve similar results, a TCP IXFR would need to be
attempted, but aborted if what was returned was an AXFR. Doing an
ABORT on TCP is, to put it mildly, rude.)

Brian

---

From: Masataka Ohta <[email-masked]>
Date: Thu, 04 Mar 2010 11:45:41 +0900

> by-design vs by-accident.

If you make a protocol more complex in order to make it behave
more efficiently for a rare accident, the complexity cause more
accidents.

That's all you must know about design and accidents.

						Masataka Ohta

---

From: Brian Dickson <[email-masked]>
Date: Wed, 3 Mar 2010 23:16:42 -0400

> That's all you must know about design and accidents.

You are correct. But I had the two backwards; the actual cases are
by-design, not by-accident. (I think you make my point for me. Sorry.)

Making a protocol more flexible to make it behave more efficiently for
known common cases, *prevents* accidents.

Accidents are caused by failure to handle boundary conditions, such as
race conditions, scaling issues, or cascading failures.

Failures in IXFR which cause fallbacks to AXFR, are boundary
conditions, cause race conditions, resulting in scaling issues, and
causing cascading failures, need to be prevented if possible.

The "if possible" is both a combination of operator action, and protocol action.

I support the draft-kerr-ixfr-only, because of these, and because it
does so simply and elegantly, and most importantly, with backward
compatibility in mind.

Brian

---

From: Ondřej Surý <[email-masked]>
Date: Thu, 11 Mar 2010 12:39:58 +0100

> There are example zones where AXFR is always far from optimal. Most

I would add for completeness that IXFR-ONLY proposal was made with TLDs 
zones in mind.  With DNSSEC it becomes even more needed to ensure that 
IXFR is used without a fallback.

To give a real example, those are numbers of 30 minutes difference in 
.CZ TLD zone, and we are not the exactly biggest zone in the world.

IXFR:
;; Query time: 135 msec
;; WHEN: Thu Mar 11 12:16:58 2010
;; XFR size: 8971 records (messages 21, bytes 1334266)

AXFR:
;; Query time: 17450 msec
;; WHEN: Thu Mar 11 12:14:31 2010
;; XFR size: 3049236 records (messages 3543, bytes 200405310)

1.3MB vs 191MB can be a big difference in case you are distribution the 
zone to f.e. 20+ servers.

Moreover we designed IXFR-ONLY with backwards compatibility in the mind. 
  I know this proposal is mostly beneficial to TLDs or other big zone 
holders, but it improves DNS protocol in a clean way, making the XFR 
more deterministic.

Masataka-san, I still don't understand what do you want to achieve by 
your critical comments?  Do you want this entire proposal to be trashed 
or do you want us to improve it somehow based on your comments or ...? 
Could you please clarify your intents?

Ondrej
-- 
  Ondřej Surý
  vedoucí výzkumu/R&D manager
  -------------------------------------------
  CZ.NIC, z.s.p.o.    --    Laboratoře CZ.NIC
  Americka 23, 120 00 Praha 2, Czech Republic
  mailto:[email-masked]    http://nic.cz/
  tel:+420.222745110       fax:+420.222745112
  -------------------------------------------

---

From: Ondřej Surý <[email-masked]>
Date: Tue, 23 Mar 2010 16:08:35 +0100

Hi,

please adopt this document as a WG item.

Thank you,
Ondrej

On 26.2.2010 15:14, Ondřej Surý wrote:
> Hi,
>
> we have incorporated all comments on -00. Thanks to Mark, Alfred,
> Wouter, Joao, Joe and Jelte for those comments.
>
> Ondrej
>
> -------- Original Message --------
> Subject: New Version Notification for draft-kerr-ixfr-only-01
> Date: Fri, 26 Feb 2010 05:37:37 -0800 (PST)
> From: IETF I-D Submission Tool <[email-masked]>
> To: [email-masked]
> CC: [email-masked]
>
>
> A new version of I-D, draft-kerr-ixfr-only-01.txt has been successfuly
> submitted by Ondrej Sury and posted to the IETF repository.
>
> Filename: draft-kerr-ixfr-only
> Revision: 01
> Title: IXFR-ONLY to Prevent IXFR Fallback to AXFR
> Creation_date: 2010-02-26
> WG ID: Independent Submission
> Number_of_pages: 6
>
> Abstract:
> This documents proposes a new QTYPE (Query pseudo RRtype) for the
> Domain Name System (DNS). IXFR-ONLY is a variant of IXFR (RFC 1995)
> that allows an authoritative server to incrementally update zone
> content from another (primary) server without falling back from IXFR
> to AXFR. This way, alternate peers can be contacted more quickly and
> convergence of zone content may be achieved much faster in important,
> resilient operational scenarios.