Subject: getaddrinfo DNS search order issues

From: [email-masked]
Date: Thu, 18 Jul 2002 14:35:24 +0100

Some time ago, I had an issue with a recently IPv6 capable telnet client on a
Linux box
being apparently unable to connect to local IPv4 only capable hosts during a
local
ISP Connectivity outage.

I can't quite decide whether the problem is the down to the IPv6 or DNSExt
Groups to solve,
since the issue relates to the interface between the Sockets API and the lower
level resolver libraries.

The combination of requirements for which the problem arose were:

non-Windows2000 client host
getaddrinfo capable client app.
Dual IPv4/IPv6 capable ( i.e. kernel and sockets/resolver library ) client host.
IPv4 interfaces only actually configured and up on client host.
IPv4 only visible server destination host.
non-dotted server name passed as parameter to client app.
Local ISP outage

Further investigation revealed the following explanation.
The telnet client was passed a non-FQDN ( or more accurately a non-dottted )
hostname to
which to connect.

Within the client app, getaddrinfo was called to resolve the non-dotted name.

For the purposes of the argument, assume a local company name - and hence
resolv.conf
search path of dwarves.com, and a hostname of grumpy. Also discount for the
present any
use of hosts/NIS in nsswitch.conf - which adds yet further complication to the
problem.

getaddrinfo() performs an outer loop across all known address families-  in this
 case IPv6 and
IPv4 in that order, passing each request down to the resolver library's lower
levels

Resolver library searches for the hostname across the resolver search path.

Because all Unix resolver libraries I've been able to test force a last-resort
search path of "."
on non-dotted names, this combination results in the following DNS queries to
the local DNS server.

grumpy.dwarves.com  AAAA returns NODATA/NOERROR
grumpy.        AAAA returns NXDOMAIN
grumpy.dwarves.com  A    returns address

As far as I can tell, on Windows boxes, performing a gethostbyname or similar on
 a non-dotted name
always looks along the explicitly declared search path - there is no implicit
"." search lookup
for non-dotted names.

The problem, of course, is that the "grumpy.  AAAA" lookup is forced to query
all the way up to the
root servers ( assuming no local cache has the answer ), and if your local ISP
has just pulled the plug,
the client app sulks for several tens of seconds before falling back to the
final successful
"grumpy.dwarves.com A" lookup.

As a nasty kludge fix for my local Linux boxes, where most of the kernels happen
 to have been built without IPv6
support whilst the sockets library was still  IPv6 capable, I was able to patch
fix the problem to an extent
by making the telnet client call getaddrinfo with AF_UNSPEC if and only if it
was capable of successfully
opening a test IPv6 socket - thereby proving the kernel to be IPv6 capable.

As far as I can tell, the preferred solution seems to be a redefinition of
getaddrinfo()/resolver-library internal
operations to perform the lookup across the DNS search path as the outer loop,
with the search across the
address family set as the inner loop, instead of the other way round as at
present. As everyone can readily
guess, this is a non-trivial problem....

Another possible workround is to enhance the API to include some form of ioctl()
 like call which returns the number
of interfaces of a given family configured on a host. The client app could then
opt to only use AF_UNSPEC
as a parameter to getaddrinfo if ioctl() returns a non-zero count for the IPv6
address family.

A side effect of this is that even during normal - fully Internet connected
times - local dual stack hosts will
be busy annoying the root servers will TLD level AAAA lookups which invariably
return NXDOMAIN. Fixing
this issue would presumably relieve all the root servers of quite a bit of
nonsense.

Of course, if someone happens to choose a local hostname which matches a TLD,
and a rogue TLD admin
were to create a suitable AAAA record and corresponding IPv6 host, a form of
man-in-the-middle attack might
also be possible. Consider a real local host of biz.dwarves.com, and a biz. AAAA
 record pointing
to a live rogue IPv6 host; any client connecting to biz from a dual stack host
might potentially connect to the rogue.

The above might even argue for a redefinition of DNS itself, banning A/AAAA/A6
records in a TLD, but implementing
such a monumental change is probably completely out of the question.

The other argument to be made is to ban the last resort lookup in the search
path for A/AAAA/A6 records. It seems
that Windows boxes manage completely happily without it. Given that there are
very few TLD level A/AAAA/A6 records anyway,
( cc. being a known exception ) leaving that implicit root lookup in place is
perhaps more trouble than it's worth.

Ted Rule,
Flextech Television.

***************************************************************************************************

This E-mail message, including any attachments, is intended only for the person
or entity to which it is addressed, and may contain confidential information.

If you are not the intended recipient, any review, retransmission, disclosure,
copying, modification or other use of this E-mail message or attachments is
strictly forbidden.

If you have received this E-mail message in error, please contact the author and
delete the message and any attachments from your computer.

You are also advised that the views and opinions expressed in this E-mail
message and any attachments are the author's own, and may not reflect the views
and opinions of FLEXTECH Television Limited.

***************************************************************************************************

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Robert Elz <[email-masked]>
Date: Thu, 18 Jul 2002 23:55:47 +0900

Date:        Thu, 18 Jul 2002 14:35:24 +0100
    From:        [email-masked]
    Message-ID:  <[email-masked]>

  | I can't quite decide whether the problem is the down to the IPv6 or DNSExt
  | Groups to solve,

Probably neither, most of what you're describing is just implementation bugs.
Talk to the implementors of the stack/library, and get it fixed.

  | getaddrinfo() performs an outer loop across all known address families-
  | in this case IPv6 and IPv4 in that order,

While this makes the implementation easy, it is almost certainly not the
right way - since (in this scenario) both AAAA and A records would be
wanted (otherwise it would only be doing one of the queries) it really
should be doing both at once.   Not all the AAAA and then all the A.
That just turns out to be harder with the internal library interfaces
as they currently exist that the current scheme.

  | Because all Unix resolver libraries I've been able to test force a
  | last-resort search path of "."

That's another bug - failing to add a domain should only ever be done
to a name that already has dots, a name like "grumpy" (that is, not "grumpy.")
should never be looked up by normal applications, some default domain
should always be appended.   (Diagnostic tools can behave differently
sometimes).

  | The problem, of course, is that the "grumpy.  AAAA" lookup is forced
  | to query all the way up to the root servers

Just sloppy implementation.   Nothing in any spec requires that.

  | As a nasty kludge fix for my local Linux boxes, where most of the kernels
  | happen to have been built without IPv6 support whilst the sockets
  | library was still  IPv6 capable, I was able to patch fix the problem
  | to an extent by making the telnet client call getaddrinfo with AF_UNSPEC
  | if and only if it was capable of successfully opening a test IPv6 socket

That's a high overhead way of achieving a simple result (high because it
will often require two additional system calls - one to open, another to
close again) just to find out if the stack is IPv6 (or 4 for that matter)
capable.  There should be an easier way in the API to achieve this.
That's something which might be an ipv6 working group issue.

  | Another possible workround is to enhance the API to include some form
  | of ioctl() like call which returns the number of interfaces of a given
  | family configured on a host.

That sounds like more info than is required.

  | A side effect of this is that even during normal - fully Internet connected
  | times - local dual stack hosts will
  | be busy annoying the root servers will TLD level AAAA lookups which
  | invariably return NXDOMAIN.

Lots of things annoy the root servers.   But the implementation bug should
be fixed.  This is not the forum to achieve that however.

For what it is worth, the same problem can easily occur (and does occur,
though there are less implementations that suffer as much these days as
there used to be) without any IPv6 anywhere.

  | The above might even argue for a redefinition of DNS itself,
  | banning A/AAAA/A6 records in a TLD,

No, we don't need that.

  | The other argument to be made is to ban the last resort lookup in the search
  | path for A/AAAA/A6 records. It seems that Windows boxes manage completely
  | happily without it.

Everyone should be able to manage without it if the initial name has
no dots at all.   Of course, if you had said "grumpy.subdomain" expecting
"grumpy.subdomain.dwarves.com" to be found, then a lookup for just
"grumpy.subdomain" is to be expected, and will often be attempted before
appending anything from the search path.

kre

---

From: [email-masked]
Date: Fri, 19 Jul 2002 01:26:31 +0900

>getaddrinfo() performs an outer loop across all known address families-  in
>this  case IPv6 and IPv4 in that order, passing each request down to the
>resolver library's lower levels

	this is not a protocol issue but purely an implementation issue.
	resolver in *BSD does not do the above.  quick-and-easy implementation
	that calls gethostbyname() from within getaddrinfo() would behave
	like you have described.

itojun

---

From: [email-masked]
Date: Fri, 19 Jul 2002 10:37:29 +1000

> as they currently exist that the current scheme.

	Robert, I mentioned that this was a issue several years ago
	(on namedroppers / bind-workers) with respect to MX vs A
	and said that the search should stop on a NODATA response.
	You were quite happy for MX and A to hit different fqdn at
	the time.  It looks like AAAA vs A has changed your opinion.

	It looks like we need a multi-type search algorithm which
	can optionally stop on match or continue to look for the
	rest of the types.
 
> should always be appended.   (Diagnostic tools can behave differently

	This is configurable in modern resolvers.

> Just sloppy implementation.   Nothing in any spec requires that.

	Agreed, however changing *default* resolver behaviour is
	difficult as it ends up breaking things.  "localhost"
	lookups is what tends to break if you disable this.
 
> That's something which might be an ipv6 working group issue.

	It's not only if the kernel is capable.  It's also if there
	are interfaces configured.  Interface scanning is a pain
	to do in a portable manner.  Multiple ioctl and structures
	with the same names but different results / layouts.
 
>   | banning A/AAAA/A6 records in a TLD,

	Agreed.
> appending anything from the search path.

	This is where configuring your servers as stealth root
	servers becomes a big win, especially when you loose
	connectivity to the real roots.  It also helps when you
	have resolvers that you can't turn off "try unqalified".

	If you do this however only have one nameserver at your
	site fetch the root zone and turn NOTIFY off (or make it
	explict).

	Now to just get the root zone's serial to reflect changes
	to the zones contents and not the time it was dumped from
	the whois database.

	You can also just configure a empty subdomain zone to
	intercept these queries.  Watch out for conflicts with real
	tlds.

	The other thing to do is just not use partially qualified
	names, especially in configuration files.  This is hard as
	it requires changing human behaviour.

	Mark
--
Mark Andrews, Internet Software Consortium
1 Seymour St., Dundas Valley, NSW 2117, Australia
PHONE: +61 2 9871 4742                 INTERNET: [email-masked]
--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: Robert Elz <[email-masked]>
Date: Fri, 19 Jul 2002 11:32:06 +0900

Date:        Fri, 19 Jul 2002 10:37:29 +1000
    From:        [email-masked]
    Message-ID:  <[email-masked]>

  | 	Robert, I mentioned that this was a issue several years ago
  | 	(on namedroppers / bind-workers) with respect to MX vs A
  | 	and said that the search should stop on a NODATA response.
  | 	You were quite happy for MX and A to hit different fqdn at
  | 	the time.  It looks like AAAA vs A has changed your opinion.

Sorry, I don't remember the context, but the two cases aren't the same
at all.   If I am looking for an MX record, I am looking for an MX
record.   And if I want the MX record for "foo" that's what I want to
find.   Only if there is no MX record for "foo" should a A lookup
even be attempted.

On the other hand, when I want "the address of foo" (which is what
getaddrinfo() without a specific address type hint does), then I am
expressly asking for either an A or an AAAA (or even an A6...)

  | 	Agreed, however changing *default* resolver behaviour is
  | 	difficult as it ends up breaking things.  "localhost"
  | 	lookups is what tends to break if you disable this.

Only for badly configured zones - the name "localhost" should be
configured in every zone that is in a search list (if it doesn't
exist in a zone that is on the search list means that there is no
"localhost" existing, and the lookup should fail, going on to,
or locally configuring "localhost." isn't what should be being done.

Because of the way the decision was made to ground FQDN's in a trailing dot,
but almost never actually represent it, there's no way for the PTR record
for 127.0.0.1 to actually return the FQDN "localhost." (or rather, that
could be done, but it never is).   What's always returned is the
unqualified name, "localhost"

  | 	It's not only if the kernel is capable.  It's also if there
  | 	are interfaces configured.

Sure, there should be a simple API for "should I attempt this AF?"

I'd also be somewhat cautious about advising people to run non-standard
DNS setups.   There are things that people who know what they're doing
can safely do - but which aren't really practical to advise for the
world to attempt to copy (and everyone snarfing the root zone probably
doesn't scale, even if it isn't one of the more dangerous things).

kre

---

From: [email-masked]
Date: Fri, 19 Jul 2002 12:52:58 +1000

> expressly asking for either an A or an AAAA (or even an A6...)

	Well given that if you are looking for a MX for "foo" you
	want to send mail to foo and the fallback if there isn't a
	MX for foo is to look for a address record I would say that
	it is a indentical senario.
 
> doesn't scale, even if it isn't one of the more dangerous things).

	And constantly querying the root servers does?
	
	It's quite possible to make zone transfers scale.  You get
	your copy from your ISP.  Your ISP gets a copy from the
	roots / dedictated "stealth" root server.

	Anyway this need to be analysed.

---

From: Robert Elz <[email-masked]>
Date: Fri, 19 Jul 2002 12:01:09 +0900

Date:        Fri, 19 Jul 2002 12:52:58 +1000
    From:        [email-masked]
    Message-ID:  <[email-masked]>

  | 	Well given that if you are looking for a MX for "foo" you
  | 	want to send mail to foo and the fallback if there isn't a
  | 	MX for foo is to look for a address record I would say that
  | 	it is a indentical senario.

No, one is explicitly looking for an MX, the other is looking for
any address.   It is only appropriate to look for the A record
corresponding to a name if the MX record is first found not to
exist.   Finding an A record that might apply to some other variant
of the name is irrelevant.   You'd have to complete the MX search anyway.

On the other hand, "find me any A or AAAA records" is satisfied by
either A, or AAAA or both.

kre

---

From: [email-masked]
Date: Fri, 19 Jul 2002 13:24:07 +1000

> either A, or AAAA or both.

	Well when I type "mail user@foo" I don't care if there is
	a MX record or not.  MX records are NOT manditory.  Until
	they are made manditory the search for MX and address records
	should be done in parallel.

	This is the same as the search for A / AAAA / A6 should be
	done in parallel even if you are *not* going to return one
	of the address families.

	Mark

---

From: [email-masked]
Date: Fri, 19 Jul 2002 10:37:29 +1000

Agreed.

	Mark