Subject: [Asrg] Implementing IPv6 DNSBLs

From: "John R. Levine" <[email-masked]>
Date: 27 Nov 2010 23:47:32 -0500

I have been pondering the swamp which is IPv6 DNS blacklists and 
whitelists.

My DNSBL RFC describes v6 DNSBLs as a straightforward analogy to v4 
DNSBLs.  For IPv6, take the address as a 16 digit hex number, reverse the 
digits as is done for ip6.arpa rDNS, and looks that up, e.g., if the 
address is 1111:2222:3333:4444:5555:6666:7777:8888 you look up 
8.8.8.8.7.7.7.7.6.6.6.6.5.5.5.5.4.4.4.4.3.3.3.3.2.2.2.2.1.1.1.dnsbl.example.

While that is OK as far as it goes, I'm worried about performance problems 
due to blowing out DNS caches.  The usual allocation for a single 
broadband customer would be a /64, which includes 2^64 addresses.  That is 
enough that a sender on a single host could easily use a different address 
for every message he ever sent.  That would mean that for each message 
each client would fetch a new set of DNSBL entries, thereby both drowning 
the DNSBL servers, and blowing out the caches which would be filled with 
useless DNSBL lookups that would never be reused.  DNS wildcards don't 
help here, since the cache doesn't know that an answer was expanded from a 
wild card (but see below.)

This same problem affects rDNS lookups, so I'm trying to figure out what 
ISPs are planning to do about them.  In the meantime, here are some DNSBL 
related thoughts.

Paul Vixie is working on a way to distribute lists of naughty domain or IP 
names as DNS zones, using IXFR and NOTIFY.  He claims he's tried it with a 
3 million entry zone at high update rates, and it works OK, but that still 
means that the central server needs to know all the clients to notify.

If a zone is signed with DNSSEC (which I admit would be quite a leap for 
something like rbldnsd), caches can do a lot of the wildcard expansion. 
That is, if the answer to a query was expanded from a wildcard, the DNSSEC 
stuff that comes along with it tells the cache what the wildcard was and 
what the range of names it covers.  That's enough for a cache to handle 
subsequent queries matched by the wildcard itself.  The DNSSEC specs say 
it would be "prudent" for the cache not to do that, but the arguments for 
the alleged prudence seem to me to be pretty weak.

Someone suggested that you could use DNAMEs to cover ranges of names, with 
16 DNAMES (1-F) at each level pointing at a common main entry.  This 
shrinks the 2^64 entries in a /64 to 256 DNAMES, but I haven't done any 
modelling to figure out whether that's really likely to help, e.g., if 
there's less than 256 queries to a given /64 before the TTL on the DNAMEs 
starts to time out, you haven't saved anything.

Finally, I was thinking about ways to make backward compatible changes to 
IPv6 DNSBLs to make them more cache friendly.  It occurs to me that in 
many cases, you know administratively what the granularity of the DNSBL 
is.  For example assume a hypothetical DNSBL lists at /64 or coarser, so 
you know all of the results in a given /64 are the same.  If you could 
publish the granularity for clients to use, they could truncate the query 
name to the granularity, e.g the query for 
1111:2222:3333:4444:5555:6666:7777:8888 would just use the the 
1111:2222:3333:4444 part of the address, so you'd look up 
4.4.4.4.3.3.3.3.2.2.2.2.1.1.1.dnsbl.example.
For backwards compatibility, longer names would still have to work but 
that's not hard to do.  I'd suggest publishing the granularity like this:

_granularity.dnsbl.example IN TXT "64"

That is, it's a TXT record at the reserved name _granularity.  If there's 
no record there, you fall back to full length queries.  If there's a 
record but it contains something other than a two or three digit number, 
you stop because it's not a DNSBL.  I'd require the granularity to be at 
least 20, so there are at least five components in the reversed address, 
to avoid confusion with four-component IPv4 reversed addresses.

My suggestion would be to do the granularity hack, which is basically 
free, and think about encouraging caches to do DNSSEC based wildcard 
expansion.  What do you think?

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

---

From: Steve Atkins <[email-masked]>
Date: Sat, 27 Nov 2010 20:59:31 -0800

On Nov 27, 2010, at 8:47 PM, John R. Levine wrote:

> I have been pondering the swamp which is IPv6 DNS blacklists and whitelists.

[...]

> What do you think?

I think you need to consider quitting using DNS. It's ill-suited to this application (IPv4 or IPv6, though some of the weaknesses become more pronounced in IPv6 space).

Something that looks suspiciously like DNS to communicate between, say, an MTA and a local blacklist daemon makes some sense, for backwards compatibility if nothing else, but distributing the data should be more pushing deltas to a local authoritative store (a-la BGP) than querying to a remote server (or some hybrid of course-grained local store and authoritative fine-grained queries to a remote store in the case of a local store hit).

If y'all haven't read the specs of the Google malware blacklist protocol, it's well worth a look. It's a hideously badly documented hack in some respects, but is a good example of an efficient way of distributing blacklist data to the edges.

http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec

Cheers,
  Steve

---

From: Seth <[email-masked]>
Date: Sun, 28 Nov 2010 00:22:14 -0500 (EST)

Why not use the return code to specify the granularity?

Seth

---

From: John Levine <[email-masked]>
Date: 28 Nov 2010 05:29:14 -0000

>Why not use the return code to specify the granularity?

I don't follow.  Example, please?

R's,
John

---

From: Seth <[email-masked]>
Date: Sun, 28 Nov 2010 01:01:51 -0500 (EST)

> I don't follow.  Example, please?

The return code is an ip6 address, so you have lots of bits to play
with.  It doesn't take very many to specify the netmask in binary
(that would even fit in an ip4 address with plenty of bits left over).

Seth

---

From: John Levine <[email-masked]>
Date: 28 Nov 2010 06:20:38 -0000

>(that would even fit in an ip4 address with plenty of bits left over).

Hm, no.  The return value from an ip6 DNSBL is an A or TXT record, the
same as for ip4 DNSBLs.

But more important, I want the client to truncate the query name to
make the cache work better.  By the time the client's done the DNSBL
lookup, it's too late, so I have this special _granularity lookup it
can do once when it starts up.

R's,
John

---

From: Daniel Feenberg <[email-masked]>
Date: Sun, 28 Nov 2010 06:56:34 -0500 (EST)

> I have been pondering the swamp which is IPv6 DNS blacklists and whitelists.

I think IPv6 blacklists are probably not ever going to be practical or 
desirable. An MTA with only an IPv6 address will not have reachability 
comparable to an IPv4 address for many decades, if ever. Certainly it will 
be far less useful than an MTA on a pool address is today. So I don't see 
actual mail servers sending desirable mail without IPv4. Since such a mail 
server would have poor connectivity, it is likely only an "owned" machine 
would attempt to send mail via IPv6, and one wouldn't want mail from such 
a machine.

I know that many legitimate MTAs already have both IPv4 and IPv6 
addresses, my argument is that they will always have at least IPv4.

There is the further point that while it is possible to maintain a 
"badness" list in a 32 bit space, it isn't really possible to maintain 
such a list in 128-bit (or even 64-bit) space, especially when the bad 
guys don't have to keep track of which addresses they have already used.

Accepting mail only from IPv4 will be the norm and will not result in any 
lost mail, even far into the future. Devices with only IPv6, such as smart 
phones and desktops will send mail through smarthosts with IPv4 
connectivity.

Daniel Feenberg

---

From: John Levine <[email-masked]>
Date: 28 Nov 2010 14:40:47 -0000

>phones and desktops will send mail through smarthosts with IPv4 

I wouldn't disagree, even though steam comes from the ears of the IETF
old guard when one points this out.

R's,
John

---

From: der Mouse <[email-masked]>
Date: Sun, 28 Nov 2010 10:37:51 -0500 (EST)

>> Accepting mail only from IPv4 will be the norm and will not result
>> in any lost mail, even far into the future.  Devices with only IPv6,
>> such as smart phones and desktops will send mail through smarthosts
>> with IPv4 connectivity.

Make up your mind: if a v6-only device smarthosts, the smarthost must
be accepting mail from v6.

Perhaps you meant accepting _arm's-length_ mail from v4 only?  It's an
important distinction, in that it means, for example, that leaving v6
support entirely out of genera-purpose SMTP listener implementations
would be a Bad Thing.

> I wouldn't disagree, even though steam comes from the ears of the
> IETF old guard when one points this out.

I think I at least partially disagree.

If the big receivers (gmail, hotmail, mail.com, etc) don't take mail on
v6, spamware is unlikely to bother; if this happens, it means that
coming in over v6 correlates negatively with being spam, leading to a
spam advantage for accepting v6 mail for small receivers.

And if the big receivers do take mail on v6, I don't think it will be
fair to say that accepting only over v4 is "the norm".

At the moment, we seem to be in the former of those situations.  Almost
all the v6-carried spam I get these days comes through mailing lists
hosted on dual-stack hosts.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Seth <[email-masked]>
Date: Sun, 28 Nov 2010 10:59:49 -0500 (EST)

> Hm, no.  The return value from an ip6 DNSBL is an A or TXT record, the

Still plenty of bits available.

> can do once when it starts up.

Your way, it asks for granularity once, and it's specified per DNSBL.

My way, it never asks for granularity.  When it asks for X, it gets a
return code that says "the surrounding /Y is listed" (Y in binary).
It can then cache one entry for the /Y, just as if granularity were
previously given as /Y; this requires no extra lookups, and allows
granularity to be specified by the DNSBL on a per-entry basis.

Seth

---

From: Matthias Leisi <[email-masked]>
Date: Sun, 28 Nov 2010 19:35:01 +0100

On Sun, Nov 28, 2010 at 12:56 PM, Daniel Feenberg <[email-masked]> wrote:

> There is the further point that while it is possible to maintain a "badness"
> list in a 32 bit space, it isn't really possible to maintain such a list in
> 128-bit (or even 64-bit) space, especially when the bad guys don't have to
> keep track of which addresses they have already used.

I very much subscribe to this point of view. The futility of
"enumerating badness" in 128-bit-space was what originally led me to
push whitelisting with dnswl.org (which we are only now slowly
adapting to IPv6, but that's another story).

> lost mail, even far into the future.

As much as I support the above, I have my doubts about this. Of
course, not having the toolset on IPv6 as you are used to on IPv4 will
slow down the adoption. IMO, it makes sense to define a proper
protocol now, while adoption is still low, and people will need to
upgrade their systems anyway.

>From my experience with dnswl.org I can tell that a proper way to
lookup range (as opposed to sending queries for consecutive IPs) would
make sense in IPv4 as well.

The protocol need not be based on DNS, although it has been proven to
be actually quite useful for the task.

-- Matthias

---

From: Matthias Leisi <[email-masked]>
Date: Sun, 28 Nov 2010 19:42:49 +0100

> granularity to be specified by the DNSBL on a per-entry basis.

IMO, this is a requirement: specify granularity on a per-entry basis.
On first glance, this could be done through something like

_64.4.4.4.4.3.3.3.3.2.2.2.2.1.1.1.dnsbl.example

The first token is interpreted as the IPv6 netmask if preceded by "_",
with "_128" implicit if omitted; if four "data tokens" are present, an
IPv4 address is assumed, with an implicit "_32" IPv4 mask of omitted.

Alternatively, require a 2002::/16 IPv6 address for IPv4 range lookups.

A DNSxL would still be free to return a "the surrounding /Y is listed" response.

-- Matthias

---

From: Daniel Feenberg <[email-masked]>
Date: Sun, 28 Nov 2010 15:35:43 -0500 (EST)

> support entirely out of genera-purpose SMTP listener implementations

Perhaps I should have said port 25, MSA traffic will of course come to the 
MTA over v6 for domains that have v6 hosts originating messages.

> spam advantage for accepting v6 mail for small receivers.

That isn't an equilibrium - the big receivers wouldn't maintain such a 
policy if it increased their spam load.

Daniel Feenberg

---

From: John Levine <[email-masked]>
Date: 28 Nov 2010 23:19:30 -0000

In case it's not clear, my related goals here are to limit the number
of entries in the DNS cache, and the number of queries to the DNSBL's
authoritative server.

>granularity to be specified by the DNSBL on a per-entry basis.

I have to ask, what's "it".  On my Unix-ish system, the only common
point for DNSBL queries is the local DNS cache.  If the cache is going
to implement ad-hoc wildcard expansion of your flavor, why not just
do DNSSEC based expansion of real wildcards?

With my scheme, since every DNSBL lookup needs the _granularity
record, it is likely cached locally, so that lookup is approximately
free.  Once you have that, then the client truncates its query to the
/64 or whatever the granularity is, so if someone's made another query
in the same /64, which would be likely if the bad guy is cycling
through addresses on small network, that's free, too.

This does require somewhat more work at the client, but has the advantage
that the DNS cache doesn't have to do anything special.

To whoever said you wanted each entry to have its own granularity,
that would be swell if you can figure out a way to do it within the
constraints of the way the DNS works now.  If you have to do major surgery
to the DNS cache, you might as well take Steve's advice and come up with
a new custom protocol.

R's,
John

---

From: John Levine <[email-masked]>
Date: 28 Nov 2010 23:20:30 -0000

>be accepting mail from v6.

I think he means submit server.

>Perhaps you meant accepting _arm's-length_ mail from v4 only? 

Yeah.  Publish MXes with A records, not AAAA.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Mon, 29 Nov 2010 10:46:51 -0800

> http://code.google.com/p/google-safe-browsing/wiki/Protocolv2Spec
IPv6 will make supporting reverse DNS and RBLs difficult.  However, the 
safe-browsing protocol has limited utility for mapping left-most nodes.  
It would not work for reverse IP address listings for IPv6, for 
example.  Perhaps a convention of including "*" as left most labels 
where nodes collapse into providing the same result could work.

-Doug

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 30 Nov 2010 12:52:33 +0000

--On 28 November 2010 15:35:43 -0500 Daniel Feenberg <[email-masked]> 
wrote:

> Perhaps I should have said port 25, MSA traffic will of course come to
> the MTA over v6 for domains that have v6 hosts originating messages.

Port 587 for MSA, please. Most mail clients default to port 587 for message 
submission these days.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/