Subject: Re: [Asrg] Implementing IPv6 DNSBLs

From: David Nicol <[email-masked]>
Date: Wed, 8 Dec 2010 12:16:42 -0600

On Sat, Nov 27, 2010 at 10:47 PM, John R. Levine <[email-masked]> wrote:

> Finally, I was thinking about ways to make backward compatible changes to
> IPv6 DNSBLs to make them more cache friendly.  It occurs to me that in many
> cases, you know administratively what the granularity of the DNSBL is.  For
> example assume a hypothetical DNSBL lists at /64 or coarser, so you know all
> of the results in a given /64 are the same.  If you could publish the
> granularity for clients to use, they could truncate the query name to the
> granularity, e.g the query for 1111:2222:3333:4444:5555:6666:7777:8888 would
> just use the the 1111:2222:3333:4444 part of the address, so you'd look up
> 4.4.4.4.3.3.3.3.2.2.2.2.1.1.1.dnsbl.example.
> For backwards compatibility, longer names would still have to work but
> that's not hard to do.  I'd suggest publishing the granularity like this:
>
> _granularity.dnsbl.example IN TXT "64"
>
> That is, it's a TXT record at the reserved name _granularity.  If there's no
> record there, you fall back to full length queries.  If there's a record but
> it contains something other than a two or three digit number, you stop
> because it's not a DNSBL.  I'd require the granularity to be at least 20, so
> there are at least five components in the reversed address, to avoid
> confusion with four-component IPv4 reversed addresses.
>
> My suggestion would be to do the granularity hack, which is basically free,
> and think about encouraging caches to do DNSSEC based wildcard expansion.
>  What do you think?

how about response codes to queries at the wrong grain size that
trigger client retries with fewer (or more) bits to get meaningful
responses? The client would cache the appropriate grain sizes for
whatever range it was querying in, and even Just Guessing it will take
on average fewer than two tries, instead of always two tries as with a
new kind of record. The EGRANULARITY message would include the correct
grain size in seven (of the 32 available) bits assigned to that
purpose. Or just three bits if we're splitting at octets.

An entirely new convention using both address and length, encoded in
something other than decimal octet expansion might be good too,
although it wouldn't delegate as gracefully as the multi-level current
practice does, if RBL operators actually delegate ranges to different
servers (I doubt they do. Hey, RBL operators, do you?)

This method -- meaningful answer only in response to question at
correct granularity, granularity correction message otherwise -- would
also work for ipv4 ranges, such as the dynamically allocated public
ranges that can be queried for with the RBLs that specialize in that.

---

From: John Levine <[email-masked]>
Date: 8 Dec 2010 20:04:08 -0000

>responses?

I don't see how to do that and stay compatible with existing DNS.  In
particular, there's no plausible way to add new error codes.

>something other than decimal octet expansion might be good too,

I don't see how to do this and make it work with existing caches and
clients.  If we design something incompatible, we can do it from
scratch and make it much simpler.

R's,
John

---

From: der Mouse <[email-masked]>
Date: Wed, 8 Dec 2010 15:41:46 -0500 (EST)

> I don't see how to do that and stay compatible with existing DNS.

"Response code" as used here does not necessarily have to be carried in
the rcode bits of the DNS packet.  For example, a lookup could return
an A record with a TTL of zero and an address of 0.0.x.y; this allows
for 65536 different response codes without touching the DNS
infrastructure at all.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: John Levine <[email-masked]>
Date: 9 Dec 2010 03:14:41 -0000

>"Response code" as used here does not necessarily have to be carried
>in the rcode bits of the DNS packet.  For example, a lookup could
>return an A record with a TTL of zero and an address of 0.0.x.y; this
>allows for 65536 different response codes without touching the DNS
>infrastructure at all.

Let's back up for a few milliseconds and consider what the goal is
here.  A hostile or inept mail sender can hop to a new IP within their
subnet on every message, which would provoke vast numbers of different
DNSBL lookups that would fill up DNS caches with useless entries,
forcing out the useful ones.  My goal is to improve the cache behavior
for clients that know how to check the granularity, preferably without
making things worse for clients that don't.

If you start encoding response codes in the A records, you'll break
lookups for older clients, and I don't see any advantage compared to
my hack.

The lookup for the _granularity record will be cached well on systems
like mine that don't keep state between SMTP sessions and have to
check it on each session. It's followed by the lookup for the
truncated reversed IP, which will also cache well, whether or not
there's a record, since it's the same lookup for all addreses within a
grain, and modern caches remember NXDOMAIN responses, too.

It would be nice to have variable granularity, i.e. CIDR ranges, but I
don't see any cache-able way to do that simpler than what DNSSEC does.

R's,
John

---

From: Joe Sniderman <[email-masked]>
Date: Wed, 08 Dec 2010 22:58:19 -0500

> lookups for older clients, and I don't see any advantage compared to

Any disadvantage compared to your hack?

> check it on each session.

Would that break lookups for older clients any less that encoding
granularity variables in A records would?  Either way a client that
doesnt know to look for it or expect it, likely wont.

> It's followed by the lookup for the

Unless the client ignores or doesnt check granularity and gets royally
foobar'd in the process.

> don't see any cache-able way to do that simpler than what DNSSEC does.

Right, unless caching can be bypassed altogether without bombarding the
authoritative servers, ie requiring clients to fetch zones via rsync and
authoritatively serve locally.

Both could be supported without breaking existing implementation too badly.

A client could have the option of using DNSSEC or fetching via rsync.
Doing neither, and using a non-validating resolver would result in the
cache blowups and authoritative server bombardment until access is revoked.

-- 
Joe Sniderman

---

From: Matthias Leisi <[email-masked]>
Date: Thu, 9 Dec 2010 12:19:59 +0100

> don't see any cache-able way to do that simpler than what DNSSEC does.

This is not a nice-to-have, but I consider it a MUST. For a DNSxL
operator, it must be possible to punch holes in larger ranges (IIRC
the Spamhaus PBL requires such holes).

There are multiple ways to achieve this, eg through a "hierarchy of
granularities". Possibly, it can be something like the following
(using IPv4 terms for brevity):

Lookup a.b.c.d/32 in zone dnsxl.example:
1) _granularity.dnsxl.example => 16 (may have long TTL)
2) b.a._granularity.dnsxl.example => 24 (may have long TTL)
3) c.b.a._granularity.dnsxl.example => 32 (may have long TTL)
4) d.c.b.a.dnsxl.example => result (TTL depending on type of data, eg
short for blacklist, long for whitelist)

A default granularity can be specified in the protocol (/32 for IPv4,
possibly /64 for IPv6?), which would remain backwards compatible, but
does only allow "byte-aligned" network mask lookups. An alternative
would be to specify the network mask in the query for the granularity,
eg as the left-most label:

Lookup a.b.c.d/32 in zone dnsxl.example:
1) 0._granularity.dnsxl.example => 16 (may have long TTL)
2) 16.b.a._granularity.dnsxl.example => 22 (may have long TTL)
3) 22.c.b.a._granularity.dnsxl.example => 32 (may have long TTL)
4) d.c.b.a.dnsxl.example => result (TTL depending on type of data, eg
short for blacklist, long for whitelist)

(Noting that a.b.c/22 is not in all cases the correct super-net for a.b.c.d/32)

The DNSxL operator can automatically create a "granularity tree" (of
either type) which will result in the least amount of lookups based on
his own policy and/or actually available data. It could even be fully
automated by an intelligent DNS server.

A typical IPv4-/32-only DNSxL could also specify
0._granularity.dnsxl.example => 32.

Should IPv4 and IPv6 be handled in a unified way (ie, IPv4 translated
into IPv6 net space), or have dedicated namespaces for specifying
granularity ("v4._granularity.dnsxl.example" and "v6....")?

Minimizing the number of lookups is not only in the interest of the
DNS cache operators, but also in the DNSxL operators own interest,
because it will also keep the load on the authoritative nameservers at
an acceptable level (or, along the same lines, the number of lookups
on private mirrors).

-- Matthias

---

From: David Nicol <[email-masked]>
Date: Thu, 9 Dec 2010 06:08:35 -0600

> lookups for older clients, and I don't see any advantage compared to

* no TXT records
* fewer lookups when the DNSBL client caches what to do with a range
(instead of using DNS for that)

> grain, and modern caches remember NXDOMAIN responses, too.

You want the DNSBL information consumer to not have to cache anything.
I understand your proposal better now.

---

From: John Levine <[email-masked]>
Date: 9 Dec 2010 21:44:31 -0000

>short for blacklist, long for whitelist)

Unless I'm missing something, every lookup for an IPv6 address that's
not in the BL is now 33 or 34 lookups.  That doesn't strike me as a
big win.

>an acceptable level (or, along the same lines, the number of lookups

Quite right, but you have to minimize ALL lookups, not just ones that
find an entry.

R's,
John

---

From: David Nicol <[email-masked]>
Date: Thu, 9 Dec 2010 17:27:55 -0600

As usual I'm just running my fingers here; disregard at will...

how about if the client starts with the widest granularity and narrows
down with long TTLd EGRAINSIZE responses (coded into A records)? They
would be cached locally. The EGRAINSIZE messages would go into a
currently meaningless range of dnsxl A record response codes, and such
dnsxl servers would be designed for access with new clients only.

So the lookup for 0.1.2.3.4.5.6.7.8.9.a.b.c.d.e.f.theory.example (a
full 128 bit 1pv6 address, by octets) would go like this, presuming
that we can safely assume that grain size will always be more than 24
bits

d.e.f.theory..?
EGRAINSIZE: variable (long TTL)

c.d.e.f.theory..?
EGRAINSIZE: variable (long TTL)

b.c.d.e.f.theory..?
EGRAINSIZE: variable (long TTL)

a.e.c.d.e.f.theory..?
EGRAINSIZE: variable (long TTL)

9.a.b.c.d.e.f.theory..?
EGRAINSIZE: 64 bits (long TTL)

this means that all distinctions smaller than that are at eight
octets, or standard 64 bit allocations.

8.9.a.b.c.d.e.f.theory..?
answer: not in list (medium TTL, or whatever the BCP is)

that way no queries that are too fine-grained ever get made, and
neither TXT records or distribution of wildcarding policies to caches
need be involved.

netmask length (aka grain size) not evenly divisible by 8 could get
represented by trivially extending this.

---

From: John Levine <[email-masked]>
Date: 10 Dec 2010 05:00:19 -0000

>EGRAINSIZE: variable (long TTL)

I think that if you're proposing any changes at all to the existing
DNS protocol, it'd be easier to invent something new that handles
the situation directly.

Also remember that the DNSSEC protocol, if not current DNSSEC
implementations, already do nearly all of what we want.  In
particular, if a response is synthesized from a wildcard, the response
includes what the wildcard was, and what range of queries that
wildcard covers, so the cache can answer subsequent queries in that
range without asking the master again.  And on an NXDOMAIN, it
includes* an analogous record that says what the empty range around
the nonexistent name is, again allowing the cache to generate its
own NXDOMAIN for other queries in that range.

If we can't fit stuff into the existing non-DNSSEC DNS, the next
question is to compare the benefits of inventing and implementing a
special-purpose hack, rather than adjusting our servers and clients to
take advantage of existing DNSSEC facilities.

R's,
John

* - unless the server uses the optional NSEC3 stuff to prevent walking
the zone, which would be counterproductive in a DNSBL

---

From: David Nicol <[email-masked]>
Date: Fri, 10 Dec 2010 12:48:46 -0600

On Thu, Dec 9, 2010 at 11:00 PM, John Levine <[email-masked]> wrote:
> implementations, already do nearly all of what we want.

I'm not doing that. I'm proposing an alteration of your granularity
proposal (as I understand it) that doesn't require either TXT records
or DNSSEC. You were proposing a change to the existing RxL protocol,
right? The flaw seemed to be that using your proposal (as I understood
it) each RxL provider had to provide a uniform granularity for the
universe, represented in the _granularity.provider.. TXT record.

If your proposal was a straw man to promote adoption of DNSSEC, well
then I'm resisting that.

-- 
“The aeroplane is fatally defective. It is merely a toy—a sporting
play-thing.  It can never become commercially practical." -- Nikola
Tesla

---

From: John Levine <[email-masked]>
Date: 10 Dec 2010 19:05:02 -0000

>> the situation directly. ...

>or DNSSEC.

But, unless I'm misunderstanding it, it does require new status codes
which would require upgrading every DNS cache, client library, and
middlebox on the net.  If that's not right, could you show some
examples of a query for an entry that's in the BL, and for an entry
that's not in the BL?

>If your proposal was a straw man to promote adoption of DNSSEC, well

I'm not thrilled about DNSSEC but at the moment it's looking like the
best of a bad lot.

R's,
John

---

From: Matthias Leisi <[email-masked]>
Date: Fri, 10 Dec 2010 20:28:45 +0100

> not in the BL is now 33 or 34 lookups.  That doesn't strike me as a

You are missing something. It is an attempt at fixing the limitation
of having a single granularity for a whole zone, which is not
sufficient.

The rough draft I put together and David's approach are similar, and
need more thought. But we need something that is more flexible than a
single granularity.

> Quite right, but you have to minimize ALL lookups, not just ones that

True.

-- Matthias

---

From: Matthias Leisi <[email-masked]>
Date: Fri, 10 Dec 2010 20:30:59 +0100

> DNS protocol, it'd be easier to invent something new that handles

| EGRAINSIZE responses (coded into A records)

No change to the DNS protocol here.

---

From: David Nicol <[email-masked]>
Date: Fri, 10 Dec 2010 14:32:57 -0600

> examples of a query for an entry that's in the BL, and for an entry

okay okay, I've finally gone and reviewed
http://tools.ietf.org/html/rfc5782 and I think its expansion of ipv6
by nybble is dumb, but that's outside the scope of this communication.

rfc3782 gives 24 bits to work with in every response, as the first
octet is 0xFF to avoid collateral damage. It recommends that those 24
bits are allocated from the right end to list answers that the server
knows about, allowing a server to simultaneously give 24 simultaneous
boolean answers.

I propose that an answer code is reserved to indicate "grain size
error", including other bits to indicate the correct granularity at
which questions can be answered for an address.

At nybble resolution with 128-bit addresses -- in a possible future
where there is at least one 1pv6 range that has allocated all the way
down, to different people -- and presuming that at least one nybble
will always be provided -- the second octet could be divided in two,
the first nybble being the coarsest range in the too-loose answer
range and the second being the finest range, giving 16 bits to use for
answers -- although if a bit is reserved for "this is an answer, not a
granularity correction" the answers and range advice don't have to
coexist, and a full (ha!) seven bits aligned at an octet boundary can
be given to both the low and high range. When they match, of course,
that's the granularity at which an asked question should receive a
response.

Also, Matthias Leisi is correct.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 10 Dec 2010 17:50:37 -0800

On 12/9/10 9:00 PM, John Levine wrote:
>  special-purpose hack, rather than adjusting our servers and clients
>  to take advantage of existing DNSSEC facilities.

IPv6 represents an incomprehensibly large space.  Just the global and 
subnet portion of v6 assigned to an individual ISP might span the entire 
v4 space.  The lower 64 bits, the v6 interface, is not likely assigned 
linearly, so CIDRs within v6 interfaces will be of little value.  The 
size of the v6 interface represents the v4 space times the v4 space, or 
v4^2 for 1.8 x 10^19 possible hosts on 7.2 x 10^16 networks.  Divided 
evenly, this could allow every person on the planet to control 14 
million networks, so why be frugal?

There will be any number of tunnels joining v6 elements.  Traversing 
reverse DNS as a service is not practical for v6 interface space.  It is 
also rather dubious to suggest services that source the entire v6 abuse 
space.  15kRPM SAS drives are able to hold ~600GB of data with an 
average 2MS latency.  At only one bit per address, this maps only 4.8 x 
10^12 elements.  To linearly map just the global and subnet v6 space, 
which is smaller than just one v6 interface, 15,000 of these drives 
would be needed.

It is ridiculous to suggest RBLs would be able to map out even one v6 
interface.  By excluding the v6 interface space, granularity becomes 
much less of a concern.  This suggests every MTA should have their own 
network, since they will likely suffer the fate of the entire v6 
interface, with possible exceptions granted by white-lists.

-Doug

---

From: John Levine <[email-masked]>
Date: 11 Dec 2010 02:51:58 -0000

>of having a single granularity for a whole zone, which is not

I'm sorry if this sounds rude, but both of you pretty clearly aren't
getting the DNS cache problem.

Every query from an end system goes to a local DNS cache.  If the
cache already has the answer for that exact query, the cache returns
it, otherwise it goes to the authoritative server.  When the
authoritative answer comes back, it's saved in the cache, usually
kicking out some other older entry.  If the answer is NXDOMAIN or
NODATA, that negative answer is also cached, and takes up a cache
entry.

To make DNSBLs work reasonably with caches, the end systems have to
repeat the identical queries.  That's the point of my _granularity
hack: the end system first asks for _granularity, then for the address
truncated to the granularity.  The queries for all addresses in a
grain will be identical, so the cache will be able to answer all of
them after the first query for any address in that grain.

Your schemes send a series of different queries for each address.  All
of those queries will hit the cache, and will displace other cache
entries.  Since most MTAs share the same cache for DNSBL queries and
everything else, the effect is that the cache will fill up with
useless DNSBL entries, forcing the cache to go back to the
authoritative servers for everything.  This would be a big problem;
DNS caches normally handle the vast majority of queries themselves and
the extra traffic would be a killer.

RFC 2308 describes negative caching and has some helpful examples of
the way that DNS caches work.

DNSSEC adds the new possibility for caches to be smarter, and to know
that a single cache entry can cover a range of queries.  This is what
you need to do DNSBL CIDR ranges, so any other CIDR hack has to have
no worse cache behavior than DNSSEC to be plausible.

R's,
John

---

From: Matthias Leisi <[email-masked]>
Date: Sat, 11 Dec 2010 10:21:57 +0100

On Sat, Dec 11, 2010 at 2:50 AM, Douglas Otis <[email-masked]> wrote:

> interface.  By excluding the v6 interface space, granularity becomes much

There are not only blacklists, where your assumption about mapping out
an interface may be relevant. For a whitelist, it may very well make
sense to list individual /128s. The protocol must support such varying
granularity, also within a single zone.

-- Matthias

---

From: John Levine <[email-masked]>
Date: 11 Dec 2010 15:10:27 -0000

>granularity, also within a single zone.

Quite right, but of course it must also cache well.

Could you show the sequence of queries for an address that is in the BL,
and one that's not in the BL, and a rough estimate of the cache behavior,
like I've done for the _granularity hack?

R's,
John

---

From: der Mouse <[email-masked]>
Date: Sat, 11 Dec 2010 10:19:52 -0500 (EST)

> IPv6 represents an incomprehensibly large space.  [...]  Divided

Because it's not about just the numbers.  If it were, there'd be no
IPv4 space crunch at all, rather than the minor one we have now.

Like v4, v6 space is managed hierarchically, both in assignment and, in
a different way, routing.  There is no surer way to run out of a finite
resource than to think of, and treat, it as infinite.  We did that with
v4.  We did that with our planet's ability to supply us with petroleum.
We did that with our planet's ability to absorb and detoxify the
various wastes we generate.  And, guess what?  We're coming up against
the finiteness of these "infinite" resources.  Treat v6 the same way
and the same thing will happen.

> The lower 64 bits, the v6 interface,

There really is no such thing.  One relatively common way to assign v6
addresses is to use a /64 with an EUI64 in the low 64 bits, but it's
hardly the only way, and there definitely is no reason to assume
everyone will do it.  (There are also reasons to not do it; it's
extremely annoying to swap network cards, or move the disk to another
machine, and have the machine silently change addresses on you.)

Of course, if a DNSL decides that it will list at /64 granularity,
that's its business.  Some v4 DNSLs are coarser than /32, too.  But
anything that has the assumption that every /64 can be treated as
homogenous wired into it will have a rather nasty awakening when it
runs up against reality.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Matthias Leisi <[email-masked]>
Date: Sun, 12 Dec 2010 20:52:27 +0100

> like I've done for the _granularity hack?

Starting at 0/0, walk down the _granularity sub-zone. At each step,
you'll get one of three responses (note that I'm not firm on IPv6
bitmask calculation, please correct where necessary):

* "Next Granularity is /X": Query the next granularity level at /X.
* "Everything below is granularity /X": Everything below this point is
granularity /X.
* NXDOMAIN in _granularity sub-zone would indicate that no data is
available in the "data sub-zone".

"Walking down": Query format is (masklen).(reverse
nibble-format)._granularity.(zone), starting at
0.0._granularity.dnsxl.example. If the response at 0/0 is "next
granularity is /64", the application will query
64.8.7.6.5.4.3.2.1._granularity.dnsxl.example. If the response at 0/0
is "everything below is granularity /64", the application will not
query the _granularity sub-zone any further, but directly
8.7.6.5.4.3.2.1.dnsxl.example (no masklen as the left-most nibble, ie
no change on the data zone required).

In many real-world scenarios granularities can be expected to be
somewhere around /48, /56, /64 and /128; it is safe to assume that two
to three lookups in the _granularity sub-zone would be required until
a "everything below is granularity /X"-response is reached, eg "0/0 >
/48 > /64 > /128". One step could be omitted by specifying a
meaningful minimum granularity (/32? /48?).

Two examples:

A list may chose to globally specify "everything below 0/0 is
granularity /64", which would be one additional lookup.

Another list may chose to specify at 0/0 that "next granularity is
/64"; this would result in one additional lookup at the /64
granularity, which may result in either an NXDOMAIN or a "next
granularity is /128"-response. This answer would be cached for the
whole /64, ie subsequent IPs from the same net would be answered from
cache.

I believe that the _granularity sub-zones would need to be present for
every DNSxL sub-zone, ie _granularity.foo.dnsxl.example,
_granularity.bar.dnsxl.example.

How to encode the "everything below is granularity /X" and "next
granularity is /X"? A records with values defined/to be reserved for
that purpose? TXT records?

The "granularity tree" can be created automatically from the zone data
(it is actually a kind of index into the zone data).

What happens if a resolver enumerates all /128 in a whole /64? There
is nothing an intermediary resolver / cache or an authoritative server
can do than to rate-limit a "dumb" resolver. The above draft (or a
useful / corrected variation thereof) will allow an "intelligent"
resolver to limit the number of requests without globally limiting
lookups to an arbitrary limit.

-- Matthias

---

From: David Nicol <[email-masked]>
Date: Sun, 12 Dec 2010 18:17:56 -0600

> lookups to an arbitrary limit.

I'm certainly not going to respond to an assertion that I don't
understand DNS caching.

It seems to me that an extended DNSxL protocol that addresses the
identified weakness in John Levine's RFC5782 concerning ranges:

2.1:    If a range of addresses is listed in the DNSxL, the DNSxL MUST
   contain an A record (or a pair of A and TXT records) for every
   address in the DNSxL.  Conversely, if an IP address is not listed in
   the DNSxL, there MUST NOT be any records for the address.

2.4: there is no way to specify the length of the name that a wildcard matches

is possible using A record responses only, with TXT reserved for
verbose, human-intended reason codes, as with the 5782 protocol.

Mr Levine recommends DNSSEC wildcarding as the answer. The required
semantics can however be provided within a framework of A records, as
there are plenty of bits. Including dealing with dumb/misconfigrued
clients that make requests at too fine of a granualrity.

An extended non-5782 DNSxL protocol could provide a mechanism for
specifying ranges, and would then of course no longer include the
requirement for full enumeration. The exactl syntax of this mechanism,
and if it includes TXT records for carrying the granularity
information or not, is othogonal to the point that it is possible,
using traditional simple DNS, to provide cacheable wildcard length
data.

I don't know if I included or implied this bit in what I wrote
earlier, but the way I see it, a too-fine query would get a corrective
response too, just like a too-coarse query, and after several too-fine
responses, a client would get their access denied at a firewall for a
while, like any other DOS attack. That would go for my A-records-only
protocol extension or another scheme involving TXT records, which are
of course attractive because the response doesn't have to be packed
into 24 bits (23, after the bit indicating that it's a grain size
error instead of a response) and could be made human-readable and
machine-parsable ASCII.

Furthermore, a non-DNSSEC approach could coexist with DNSSEC
wildcarding just fine, until such time as DNSSEC wildcarding support
is universal.

David Nicol, armchair engineer and at-large crank

---

From: Douglas Otis <[email-masked]>
Date: Mon, 13 Dec 2010 14:37:08 -0800

> and the same thing will happen.
Check out the phone app ByeBye v4.  It currently shows IANA exhaustion 
in less than 72 days of the >100 million remaining v4 addresses.

http://ipv6.he.net/bgpview/bgp.html
Offers a view of current v6 assignments.

There are 3,849 routes with thousands having 96 bit (/32) or more ranges 
and some having 109 bit (/19) of the 128 bit address space.  While this 
represents a very small fraction of the possible v6 assignments, 
ignoring the lower 64 bits still exposes 32 to 45 bits of the routed 
portions of this space.

Just the routed portion of v6 represents tens of thousands times the 
range represented by the entire v4 address space, where each network may 
contain the entire v4 address space squared.  This suggests tracking 
just the global and subnet portion of v6 will likely require thousands 
of times more resources than that used for the entire v4.  The 
expenditure of these resources must also consider the exponentially 
growing v6 assignments.

The way you view IP address space _must_ fundamentally change.  Blocking 
lists containing individual addresses will _never_ be viable.  Defending 
critical infrastructure will likely involve reassignment within private 
namespace, rather than dependence upon blocking lists.  Acceptance of a 
v6 exchange initially expected use of IPsec after all.

For SMTP to survive, SMTP must cryptographically authenticate the domain 
of the server publicly issuing the message.  This domain must also 
encompass the entity accepting responsibility for the message's 
destination.  Unfortunately, DKIM falls short in this regard, and should 
be considered a defense against false-positive detections of phishing 
attempts and not a basis for reputation.

Perhaps keyassure or StartTLS might offer a cryptographic basis for 
email acceptance, which then implies one must depend upon a vetted list 
of known good domains.
> homogenous wired into it will have a rather nasty awakening when it

Nearly all network segments in v6 will be on a /64 network boundary. 
Routers control the upper 64 bits of this space, where hosts use the 
lower 64 bits. The smallest v6 subnet address range is equal to the 
current Internet squared.  See RFC2462 StateLess Address 
Auto-Configuration (SLAAC), and Section 5.4 Duplicate Address Detection 
(DAD).

Unfortunately, anyone that assumes blocking lists will be able to 
selectively exclude sections of the v6 Interface, or that these 
addresses will be typically assigned manually is likely in for a very 
rude awakening.  No doubt while people whisper sweet v6 DNSBLs into 
their ear. :^)

-Doug

---

From: Matthias Leisi <[email-masked]>
Date: Tue, 14 Dec 2010 07:32:47 +0100

On Mon, Dec 13, 2010 at 11:37 PM, Douglas Otis <[email-masked]> wrote:

> For SMTP to survive, SMTP must cryptographically authenticate the domain of
> the server publicly issuing the message.  This domain must also encompass

Not necessarily. IP-based whitelisting may be Good Enough[tm] for that purpose.

> selectively exclude sections of the v6 Interface, or that these addresses
> will be typically assigned manually is likely in for a very rude awakening.
>  No doubt while people whisper sweet v6 DNSBLs into their ear. :^)

Do not think in DNSBL terms, but from a DNSWL angle. In whitelisting
scenarios, a sub-/64 assignments are both technically and
operationally feasible and can be useful in some scenarios:

* shared hosting environments (where I don't think each user will get
a full /64)
* E-mail service providers
* 4-to-6 NAT/PAT
* Likely some more...

-- Matthias

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Dec 2010 10:00:38 -0800

On 12/13/10 10:32 PM, Matthias Leisi wrote:
> > For SMTP to survive, SMTP must cryptographically authenticate the
> > domain of the server publicly issuing the message. This domain
> > must also encompass
>

v6 white-listing takes renumbering to a horrific place and might work 
only during an initial transition where the number of v6 only sources 
are few.  Over a period of a few years, these numbers are likely to 
explode into millions.

> > addresses will be typically assigned manually is likely in for a
> > very rude awakening. No doubt while people whisper sweet v6 DNSBLs
> > into their ear. :^)
>
>  * shared hosting environments (where I don't think each user will
>  get a full /64) * E-mail service providers * 4-to-6 NAT/PAT * Likely
>  some more...

Why not offer v6 to v4 tunneling where there is truly zero tolerance for 
unsolicited commercial email?  Dealing with individual v6 addresses will 
be difficult to vet compared to previously seen domains.  Using domains 
leaves a smaller number of legitimate new entries.  v6 will not allow 
even a few UCEs to be considered acceptable from source IP addresses.  
However, it would be safer to grant tolerance from specific domains when 
sending servers have been authenticated by domain.

-Doug

---

From: John Levine <[email-masked]>
Date: 14 Dec 2010 19:32:12 -0000

>are few.  Over a period of a few years, these numbers are likely to 

Why?  The estimates I seen say that there are about 100,000 legitimate
mail emitters.  I do agree that there is not an obvious way to use the
DNS to find 100,000 addresses within the v6 address space without
blowing away DNS caches.

>unsolicited commercial email?

I still don't see any practical reason to exchange mail over v6.  Even
assuming all the consumer ISPs move their customers to v6, those
customers will be talking to MTAs or web mail over v6, which can then
use v4 to talk to other MTAs.  Given the small number of legit mail
sources, I find it hard to believe that the cost of finding some v4
address space would be an insurmountable problem for anyone whose mail
I wanted to receive.

R's,
John

---

From: Matthias Leisi <[email-masked]>
Date: Tue, 14 Dec 2010 20:44:03 +0100

> DNS to find 100,000 addresses within the v6 address space without

Some background on the "100'000".

At dnswl.org, we currently have a bit over 100'000 IPv4 netranges
listed, of which 99.9% are /32s.

Further, we aggregate DNS logs to find all IPs that are queried in our
zone. At any point in time, this list contains some 100'000 additional
IPs. These IPs pass the following tests:

* Have rDNS
* Are not listed in a small number of DNSBLs at the time of being added
* Are seen more than a trivial number of times
* Are not in a small "veto list" which we maintain (eg RFC1918, some
often ill configured IP ranges like 3.0.0/24, IANA reserved space
etc).

It is safe to assume that there may be about 50k to 75k actually
legitimate IPs left which we do not (yet!) list.

We also measure the traffic per IP, approximated through the number of
queries we see in the DNS logs. While this is mathematically not
correct, we at least have a consistent error, in that we underestimate
the bigger senders (better caching = less hits in our logs).

On this traffic measure, the 100k IPs currently listed are about 90%
of the traffic we observe. Now this excludes traffic that is never
visible in our logs (eg large organisations that may have email
exchange over dedicated mailservers/IPs), so we do not claim to cover
90% of "all" or "worldwide" traffic.

> I still don't see any practical reason to exchange mail over v6.  Even

In some distant future, everything will be v6. *cough*

-- Matthias

---

From: John Leslie <[email-masked]>
Date: Tue, 14 Dec 2010 15:06:18 -0500

>> Not necessarily. IP-based whitelisting may be Good Enough[tm] for

   Indeed, it might (note the [tm] ;^)...

   ... if the number of IPv6 originators stays "small" (which as folks
have noted is entirely possible).

> are few.  Over a period of a few years, these numbers are likely to 

   "Millions" doesn't particularly bother me -- but there's nothing
except intertia holding it to "millions", or even "billions". :^(

   'Twould be better to discuss _how_ to maintain such a whitelist.
IMHO, sending SMTP clients would have to publicly declare their intent
and there would have to be sufficient volume to convince some vouching
and/or reputation service that they send a sufficient volume of ham.

   This is not entirely trivial.

   Doug Otis and I worked with Dave Crocker on the draft-ietf-marid-csv
series of I-Ds, and considered such maintenance issues -- for _both_
IPv4 and IPv6. (They may be a bit dated with regard to IPv6, having
expired in 2005, but IPv6 was considered and seemed OK at the time.)

   We were of the opinion that query by domain-name was more useful
than query by IP-address -- largely because a single (IPv4) address
might well be used by more than one sending domain, with different
policies. For IPv6, it _might_ be possible to enforce an idea that
the IPv6 address must vary if the email policies differ...

>>   get a full /64) * E-mail service providers * 4-to-6 NAT/PAT * Likely

   Agreed.

   (I'd be _very_ interested in how anyone plans to _maintain_ a
reputation service for > 2**64 different addresses.)

--
John Leslie <[email-masked]>

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Dec 2010 13:47:33 -0800

On 12/14/10 11:32 AM, John Levine wrote:
> > v6 white-listing takes renumbering to a horrific place and might
> > work only during an initial transition where the number of v6 only
> > sources are few. Over a period of a few years, these numbers are
> > likely to explode into millions.
>
>  Why? The estimates I seen say that there are about 100,000
>  legitimate mail emitters. I do agree that there is not an obvious
>  way to use the DNS to find 100,000 addresses within the v6 address
>  space without blowing away DNS caches.

Our estimates are considerably higher, where smaller entities are 
considered legitimate sources.  These sources do not need rDNS entries, 
but exchange valid messages from consistently identifiable sources,  and 
not necessarily just exchanges with large ESPs.  A more liberal criteria 
increases the overall legitimate sources significantly.

This should not be a surprise, since nearly every Unix includes MTA 
services that can be applied in SOHO environments having static 
addresses.  v6 makes obtaining a static address easy.

The dnswl.org subscription service claims services for 50,000 
organizations, but then only lists 100,000 valid sources.  The Spamhaus 
whitelist is another fee based service that excludes non-DKIM and 
non-transactional sources.  Neither appear to have the goal of including 
_all_ possible sources of legitimate email, where overall numbers are 
much much higher.  Email vetting should not be limited to just bulk 
senders. :^(

> > Why not offer v6 to v4 tunneling where there is truly zero
> > tolerance for unsolicited commercial email?
>
>  I still don't see any practical reason to exchange mail over v6.
>  Even assuming all the consumer ISPs move their customers to v6,
>  those customers will be talking to MTAs or web mail over v6, which
>  can then use v4 to talk to other MTAs. Given the small number of
>  legit mail sources, I find it hard to believe that the cost of
>  finding some v4 address space would be an insurmountable problem for
>  anyone whose mail I wanted to receive.

In your view, v6 must tunnel to v4 to exchange email.  This suggests a 
need for translation services to deal with those not prepared for v6. :^)

How would one vet email sources over v6 from other geographic regions?  
After all, malefactors can originate from virtually infinite numbers of 
v6 addresses.  Why should providers refuse legitimate sources of email 
that authenticate their domain?  Why should one expect v6 DNSRBL to be 
practical?

-Doug

---

From: John Levine <[email-masked]>
Date: 15 Dec 2010 01:58:17 -0000

>need for translation services to deal with those not prepared for v6. :^)

No tunels, gateway.  The MTA speaks v6 to the local SUBMIT and POP
users, and v4 to its SMTP peers.

>How would one vet email sources over v6 from other geographic regions?  

Personally, I wouldn't even try.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Tue, 14 Dec 2010 19:19:48 -0800

On 12/14/10 5:58 PM, John Levine wrote:
> No tunnels, gateway.  The MTA speaks v6 to the local SUBMIT and POP
Yes of course.  But a gateway will likely bridge between v6 and v4 
through a tunnel, which could be offered as a service.
> Personally, I wouldn't even try.
I'm surprised, because you're not a fuddy duddy.  v6 makes it easier for 
SOHO configurations to obtain unpolluted static addresses.  v6 also 
introduces an era demanding domain authentication, since block lists 
based upon v6 addresses will never be practical.  When v6 systems 
authenticate their domain, they'll provide better information than that 
offered from typical v4 sources, which might only expose an IP address.

Authenticated domains offer a basis for reputation that can be applied 
uniformly against either v4 or v6 sources. Authenticated domain 
information also reveals registrar and name server information, in 
addition to that gleaned from the IP address itself.  v6 is a Good Thing 
(tm), but you'll need to trade in your hammer for a wrench.

-Doug

---

From: John Levine <[email-masked]>
Date: 15 Dec 2010 04:35:51 -0000

>through a tunnel, which could be offered as a service.

No, a gateway will talk to a v4 network for the SMTP and a v6 network
for the POP and IMAP.

>SOHO configurations to obtain unpolluted static addresses.

Maybe I'm a mean old person, but I have no interest at all in making
it easier for people who don't know what they're doing to set up a
mail server.

>uniformly against either v4 or v6 sources.

Right, so we agree that v6 mail is useless.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Wed, 15 Dec 2010 09:39:09 -0800

> No, a gateway will talk to a v4 network for the SMTP and a v6 network
Believe it or not, there already is v6 SMTP.
> it easier for people who don't know what they're doing to set up a
The way v4 SMTP works today has been such a success it should be 
preserved in amber, where bulk email senders (who know what they are 
doing) happily mix good and bad sources.

v6 is likely the best thing that could happen to SMTP.  The sheer size 
of this space requires server authentication.  This welcome change will 
identify servers of good sources, rather than guessing which address is 
static and authorized by their provider.
> Right, so we agree that v6 mail is useless.
Change that to v4 and then we agree. :^)

-Doug

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 16 Dec 2010 16:12:15 +0000

> address space would be an insurmountable problem for anyone whose mail

I'm with you here. I think it'll be a long while before we start listening 
for IPv6 connections.

In fact, given the difficulty of implementing IPv6 based reputation 
services, it may be worth considering this as a matter of policy: If I 
can't verify your domain reputation (with the assistance of DKIM, SPF, 
etc), and I can't look up your IP reputation (eg, because it's an IPv6 
address) then I won't accept your email.

Given that we already don't accept IPv6 connections, I think this is an 
opportunity to draw a line in the sand. When we do eventually deploy IPv6, 
it'll be on much stricter terms than now.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: Matthias Leisi <[email-masked]>
Date: Thu, 16 Dec 2010 18:38:30 +0100

> it'll be on much stricter terms than now.

Yes, "default deny, allow from trusted/whitelisted/... sources"
instead of the current "default accept, deny from blacklisted, plus
some white/greylisting".

-- Matthias

---

From: David Nicol <[email-masked]>
Date: Thu, 16 Dec 2010 12:38:42 -0600

On Thu, Dec 16, 2010 at 9:43 AM, Matthias Leisi <[email-masked]> wrote:
[...]
>
> All subsequent lookups for IPs in the same /64 would only repeat these
> two (cached) queries/answers and not create new cache entries. Traffic
> between application and cache would run hot, though.

with actual use of a recursive focus protocol like the ones we're
discussing, a good query library would probably do its own DNS
resolution.
Here are the layers of the systems under discussion. As the protocol
is going to be tricky to get right, we expect a reusable library to do
the querying, just as is seen with SPF.

LIST SERVER <- dns protocol -> LOCAL CACHE <- DNS query library ->
LIST QUERY LIBRARY <- library API -> APPLICATION
With integration of the LC function into the LQL, that traffic won't
have to be as hot, and the LQL might even be able to make preemptive
guesses based on the specifics of additional data provided in
responses by the LS.
For instance in the focus-on-the-nebula example, the server could
provide the record indicating exactly which pixel has the 2 brightness
as an additional datum with its response to the f-04 query. A full
response set might look like

     2+++
     ++++
     ++++
     ++++

f-04: additional focus needed: 1 confidence: 0xFD data: 0x00

     2+++
     ++++

     ++++
     ++++

f0-05: same thing

(Arrgh! a one-bit request is for 0 or 8, not 0 or 1)
f8-05: no additional data available at higher focus, data 0x00

     2+  ++
     ++  ++

f0-06: same as f-04 or f0-05
f4-06: same as f8-05

     2  +
     +  +

f0-07 will still be 127.1.253.0
f2-07, f4-07 and f6-07 will all be 127.0.255.0

     2

and f0-08 is 127.0.255.2

That's eight records.  I don't know exactly how much additional
cacheable data can be provided along with a DNS answer, but providing
the next few interesting data in the area being queried about will
reduce traffic between the LC and the server.

---

From: David Nicol <[email-masked]>
Date: Thu, 16 Dec 2010 13:21:09 -0600

> f2-07, f4-07 and f6-07 will all be 127.0.255.0

sorry, f6-07 and f4-07 may either be 127.0.255.0 or NXDOMAIN. Either
way, neither should appear as additional records in a response that
includes an f4-06 indicating that no further information can be
obtained through further focus on that range.

---

From: Douglas Otis <[email-masked]>
Date: Thu, 16 Dec 2010 15:28:55 -0800

On 12/16/10 9:38 AM, Matthias Leisi wrote:
> On Thu, Dec 16, 2010 at 5:12 PM, Ian Eiloart<[email-masked]>  wrote:
>
Agreed, but the white-listing should be by authenticated servers and 
_not_ their IP address.  v6 provides significant advantages when moving 
services. If done right, there would be little effort in renumbering.  
Dealing with umpteen address based white-listing services will prove 
horrific when each service asserts different policies regarding updates 
and points of contact where timely coordination would be impossible.

Efforts related to the keyassure wg appears headed in the right 
direction, which might offer a DIY StartTLS solution for SMTP, for 
example.  Reputations could then accrue against certificate holders, 
rather than addresses.  The difference in the scale of acceptance data 
would likely be in the area of millions to 1 smaller.

-Doug

---

From: Matthias Leisi <[email-masked]>
Date: Fri, 17 Dec 2010 10:21:38 +0100

On Fri, Dec 17, 2010 at 12:28 AM, Douglas Otis <[email-masked]> wrote:

> Agreed, but the white-listing should be by authenticated servers and _not_
> their IP address.

I guess we can agree to disagree on this point. In the real world,
there will always be multiple approaches to handling reputation
information which may simply co-exist or be used in combination.

-- Matthias

---

From: David Nicol <[email-masked]>
Date: Fri, 17 Dec 2010 09:03:41 -0600

> Agreed, but the white-listing should be by authenticated servers and _not_

Brilliant! IP is about routing the packets, and nothing else. Tieing
authentication in with the implementation detasils of the transport
protocol is the root cause of major headaches. Plus plus.

---

From: John Leslie <[email-masked]>
Date: Fri, 17 Dec 2010 11:03:45 -0500

> protocol is the root cause of major headaches. Plus plus.

   Thank you!

   (It's the old story: "The light's so much better over here!" ;^)

   Doug isn't always the easiest person to understand; but I think I
understand him, having worked with him on the draft-marid-csv series
of I-Ds.

   There we set out to use the HELO string as declaring the identity
of the sending SMTP client, and work from that to authenticate that
the server actually was authorized by that domain. MARID died of the
politics of that time; and the drafts have sat in limbo since.

   Doug backed off, thinking "If you won't let me authenticate by
HELO, I'll look for something else to authenticate by." I, OTOH,
looked for other windmills to tilt...

   Obviously, it is possible to authenticate differently, and Doug
has wrapped his mind around several different mechanisms over the
last five years. None of them, IMHO, work very well in the time
domain (delivering results that may be outdated, and struggling to
mitigate replay problems). But they are certainly better than
relying on IP addresses alone.

   The principle in CSV was to have the sending SMTP client declare
its identity in HELO or EHLO, and have the receiving SMTP server
authenticate at SMTP time that the domain authorized that client
to send according to its policies. (Of course, those policies do
not necessarily match the policies of domains named in MAIL FROM or
any similar domains which will be visible to recipients.)

   CSV relied on a weak but timely authentication; we have since
seen alternatives that are stronger but less timely -- the principle
of authenticating the SMTP client talking to you is more important
than the exact balance between strength and timeliness.

   In principle, it may be possible to build a business model with
even-weaker authentication by IP-address and unknown timeliness;
but I wouldn't want to be the one trying to build it.

---

From: David Nicol <[email-masked]>
Date: Fri, 17 Dec 2010 16:10:27 -0600

okay here we go again.

This time the response format is, the 32 bits of the response divided into
16 bits of confidence, 16 bits of data, and if the query library doesn't get
confidence for what its looking for it always adds another nybble of search,
so the length can be implied by the length of the query.

The trade-offs are, we only support one bit of data for each listed range,
and we're no longer returning "safe" ip addresses within 127.0.0.0/8.

Querying for a:b::1, which is ON, and is the only thing that is on,
anywhere, followed by enumeration of additional records needed to verify
that a:b::2 through a:b::ffff are OFF.

the first two octets are confidence, the second two are data. The first bit
represents f and the last bit of the second octet of each pair represents 0,
so you can create the word by left-shifting.

bit, meaning in second octet:
  1           0
  2           1
  4           2
  8           3
 16           4
 32           5
 64           6
128           7

bit, meaning in first octet:
  1           8
  2           9
  4           a
  8           b
 16           c
 32           d
 64           e
128           f

0.list
251.255.0.0

0a.list
255.254.0.0

0a0.list
247.255.0.0

0a0b.list, 0a0b0.list,0a0b00.list, through
0a0b00000000000000000000000000.list
are all
 255.254.0.0
which represents confidence that nothing appears in any ranges except the
one at 0.

0123456789abcdef0123456789abcdef
0a0b000000000000000000000000000.list
255.255.0.2

that represents the hit on a:b::1
at this point, we have enough in the cache to require
no additional queries for all the non-hits

for use with ipv4 and ipv6 space, it would make sense to define a shorthand
for ::*w.x.y.z *as, say, lwwxxyyzz where ww through zz are the data in
hexadecimal, and l is a letter ell. (Instead of in-arpa syntax, which is
inappropriate because lists are not under distributed administration)

---

From: David Nicol <[email-masked]>
Date: Thu, 16 Dec 2010 12:02:42 -0600

On Thu, Dec 16, 2010 at 9:43 AM, Matthias Leisi <[email-masked]> (as
part of a length off-list discussion) wrote:

> For the examples below, I constructed a "granularity response" as an A
> record response with IPs in 254.0.0.0/8. The last nibble indicates the
> "next granularity" which would need to be queried. A response in
> 127.0.0.0/8 is interpreted as a "data response". Note that this is
> only illustrative at this point in time, maybe there are more
> intelligent ways to construct responses.

A totally general way to represent the data at the current level as well as
the availability or nonavailablity of data in finer levels is as follows:

Queries are of the form

      rangeprefix DASH length DOT listaddress
with no additional dots anywhere. A list is presumed to be under one
administrative domain. rangeprefix and length are both hexadecimal, length
is in bits, and is always two hexits. (if you serve responses for more than
256 bit queries, like if you're serving images from the Hubble or something,
you can make it longer.)

Responses are of the form

  127.A.B.C, where A, B and C represent the three octets we have to work
with.

 A indicates how many more bits finer we have to go to find out anything
more interesting. It is relative to the length specified inthe query. When
it is zero, any finer query may be NXDOMAIN. When it is non-zero, it
indicates how many more bits of rangeprefix are required to find out
anything interesting, that is, to flip a 0 confidence bit to a 1.

 B is confidence, and is a mask over the bits of C which are data, so this
protocol can support 8 bits of data for each range.

How the confidence and data interact:

CONDIFENCE  DATA    MEANING
1           1       1 for this and all finer ranges
1           0       0 for this and all finer ranges
0           1       generally 1, 0 points may exist
0           0       generally 0, 1 points may exist

how to serve an image with this protocol:

given an image, such as this red-shift adjusted radio-telescope snapshot of
the Levine nebula:

      0123456789abcdef
     0++++++++++++++++
     1++++++++++++++++
     2++++++++++++++++
     3+++252+++++252++
     4+++5f5+++++5f5++
     5+++252+++++252++
     6++++++++++++++++
     7++++++++++++++++
     8++++++++22++++++
     9+++++2555552++++
     a++++35fffff53+++
     b++++5f55555f5+++
     c++++252+++252+++
     d++++++++++++++++
     e++++++++++++++++
     f++++++++++++++++

where each pixel may an eight-bit brightness, but none of them are brighter
than 0x0f, and the zeroes are "+" to make it look better,

and a division of the field into two dimensions, with bits alternating from
each dimension -- odd bits vertical, even bits horizontal:

one bit:
     0
     1

two bits:

     00  01
     10  11

three bits:

     000  010
     001  001

     100  110
     101  111

four bits:

     0000  0001     0100  0101
     0010  0011     0110  0111

     1000  1001     1100  1101
     1010  1011     1110  1111

and so on, up to the eight bits of full resolution as available in the data;

the top level, or root, query for this range list server earns a response
indicating that there are no data points with brightness greater than 0x0f
anywhere, in general the majority of the remaining data are zeroes, and to
get a more definitive clue you'll have to use at least three bits:

 -00.sampleimage.example => 127.3.240.0

(I initially thought it was four, but on closer inspection I found that at
the third bit of focus one finds that there are some regions without any
0x08 bits.)

below here, to avoid doing binary/decimal conversion, I'm going to mostly
talk about additional bits required, confidence, and data, without
converting them into a dotted quad.

here is the picture grouped at three bits of resolution:

        0-03     2-03
      01234567 89abcdef
     0++++++++ ++++++++
     1++++++++ ++++++++
     2++++++++ ++++++++
     3+++252++ +++252++

   query 0-03 or 2-03 to get a response indicating:
  one more bit of focus for new information
  confidence: 0b11111000
  data:       0b00000000

        1-03     3-03
      01234567 89abcdef
     4+++5f5++ +++5f5++
     5+++252++ +++252++
     6++++++++ ++++++++
     7++++++++ ++++++++

        4-03     6-03
      01234567 89abcdef
     8++++++++ 22++++++
     9+++++255 5552++++
     a++++35ff fff53+++
     b++++5f55 555f5+++

query 1-03, 3-03, 4-03 or 6-03 to get a response indicating:
  one more bit of focus for new information
  confidence: 0b11110000 (there are still f points)
  data:       0b00000000

        5-03     7-03
      01234567 89abcdef
     c++++252+ ++252+++
     d++++++++ ++++++++
     e++++++++ ++++++++
     f++++++++ ++++++++

querying for either of these will give us confidence on a zero for the 0x8
bit in the data, and notification that we would find out something else with
one more bit of focus.

These are the blocks at four bits of resolution:

      0123 4567 89ab cdef
     0++++ ++++ ++++ ++++
     1++++ ++++ ++++ ++++
     2++++ ++++ ++++ ++++
     3+++2 52++ +++2 52++

     4+++5 f5++ +++5 f5++
     5+++2 52++ +++2 52++
     6++++ ++++ ++++ ++++
     7++++ ++++ ++++ ++++

     8++++ ++++ 22++ ++++
     9++++ +255 5552 ++++
     a++++ 35ff fff5 3+++
     b++++ 5f55 555f 5+++

     c++++ 252+ ++25 2+++
     d++++ ++++ ++++ ++++
     e++++ ++++ ++++ ++++
     f++++ ++++ ++++ ++++

at a four-bit granularity, we can get several different responses.

A query for the lower left-hand corner at four bits would get a reponse
stating that every point is zero and no further info is available:

 a-04.sampleimage.example => 127.0.255.0

Also for the one above that:

  8-04.sampleimage.example => 127.0.255.0

The one to the right of the lower left hand corner will differentiate at the
fifth bit, but no point in it is greater than 7, so we can get another
confidence bit.

The bottom right hand corner has only one bit set in its data, so

  f-04.sampleimage.example => 127.1.253.0

tells us that there is a 0x02 bit set somewhere in there, but mostly the
data in that range is zeroes.

And so on.

---

From: Douglas Otis <[email-masked]>
Date: Fri, 17 Dec 2010 16:45:49 -0800

> relying on IP addresses alone.
John,

DKIM has a limited basis for accruing reputations, since this protocol 
neglects confirmation of destinations, which would not be a problem with 
CSV.  As a result, DKIM domains can not be applied against undesired 
messages on the basis of destination alone.  DKIM however inhibits 
false-positives with anti-phishing filtering.
> any similar domains which will be visible to recipients.)
Now that v6 deployment methods have become apparent, using IP addresses 
to confirm EHLOs is likely to become increasingly error prone.  SPF will 
equally fail, even with its many transactions involved.  In addition, 
when SPF is used in conjunction with SMTP transactions, this will also 
represent a DoS concern, especially with DNSsec.

At the last MAAWG meeting, consensus was reached with a statement that 
SPF can not be used to abate spam.  SPF offers a method to determine 
which IP address block-lists affect a domain's delivery, and secondly as 
a method to qualify feedback streams.  Sender-ID was considered unusable 
and ineffective.
> than the exact balance between strength and timeliness.
Likely some extension of RFC4954 will be needed, where public keys are 
used to confirm hashed and signed challenge responses.  This might use 
DKIM's public key or whatever resource is created by the keyassure wg.  
There must also be conventions about the name determined.
> but I wouldn't want to be the one trying to build it.
Terabytes of logs processed daily for just v4 is already staggering.  
The same approach with v6 simply won't work.  SMTP needs simple and 
stable handles that don't involve transport addresses.  That is not to 
say there can't be a transition strategy using existing conventions as a 
means to confirm server/mail-domain names.

Being able to combine v6 and v4 into common reputation queries with a 
destination basis for undesired messaging would tremendously aide SMTP's 
transition into v6, while also improving the state of SMTP in general.

-Doug

---

From: John Leslie <[email-masked]>
Date: Fri, 17 Dec 2010 21:10:25 -0500

Douglas Otis <[email-masked]> wrote:
> DKIM has a limited basis for accruing reputations,

   One can accrue reputation for a DKIM signer, just not a reputation
for the particular SMTP client sending you the message.

   (Also, the reputation will necessarily be compromised by any
intervening sending SMTP client and/or any spammer that manages a
replay. Exactly what the reputation would mean is a bit obscure.)

> since this protocol neglects confirmation of destinations, which
> would not be a problem with CSV.

   DKIM, of course, means to authenticate the _message_ -- which is
a useful thing, but not particularly correlated with reputation.

>  As a result, DKIM domains can not be applied against undesired 
> Now that v6 deployment methods have become apparent, using IP
> addresses to confirm EHLOs is likely to become increasingly error
> prone.

   True... but not so much as to prevent me from trying it under v6.

> SPF will equally fail, even with its many transactions involved.

   I quite disagree: SPF will fail far more spectacularly. It will be
hard to come up with SPF RRs that do something useful and don't assist
DoS attacks.

> In addition, when SPF is used in conjunction with SMTP transactions,
> this will also represent a DoS concern, especially with DNSsec.

   I know nobody believes you -- under v4 this problem hasn't really
surfaced. I'm hoping SPF doesn't see enough v6 deployment for this to
become an issue.

> SPF can not be used to abate spam.

   SPF, BTW, could have been a useful tool if MARID had lasted long
enough to write up what particular SPF atoms _mean_ instead of just
specifying algirithms to evaluate them. Folks _have_ tried to use
SPF RRs in "non-standard" ways to glean the information they want,
but it really can't work very well.

> SPF offers a method to determine which IP address block-lists affect
> a domain's delivery, and secondly as a method to qualify feedback
> streams.

   The first can be done better by other means; the second is an
attempt to overlay a meaning after the fact. Myself, I wouldn't
use it.

> Sender-ID was considered unusable and ineffective.

   Alas, those folks meant well -- but the design was never fixed to
allow reasonable experimentation. Both SPF and Sender-ID believed
the most important thing was early adoption by senders; and IMHO
neither accomplished that (except for nearly-useless RRs published.)

> used to confirm hashed and signed challenge responses.

   That could work, but the key-infrastructure seems problematic. It's
also rather heavyweight and burdens the receiving SMTP server overmuch
IMHO. Something of that ilk would be more useful _after_ weak
authentication of the sending client indicates minimal ground for
trust. (Or, of course, a micro-payment system to cover costs...)

> This might use DKIM's public key or whatever resource is created by
> the keyassure wg.

   I don't believe a "keyassure" WG exists. There is a proposed "kidns"
WG which may be created soon. I've lost track of what the difference
is supposed to be -- for all I know, it may end up created under some
other name entirely...

   In any case, the kidns proposal calls for:

- Jun 2011  Protocol for using DNS to associate domain names with keys
            for TLS and DTLS to IESG
- Aug 2011  Protocols for using DNS to associate domain names with keys
            for IPsec to IESG

   We shall see...

> There must also be conventions about the name determined.

   I'm not sure where those might come from... :^(

> Terabytes of logs processed daily for just v4 is already staggering.  

   Indeed! Myself, I'd try to avoid that path.

> say there can't be a transition strategy using existing conventions as
> a means to confirm server/mail-domain names.

   I still see no reason to abandon the HELO string...

> Being able to combine v6 and v4 into common reputation queries with
> a destination basis for undesired messaging would tremendously aid
> SMTP's transition into v6, while also improving the state of SMTP
> in general.

   Clearly, spammers and other miscreants will continue to congregate
where the pickings are easy. I doubt they want to be bothered with IPv6.

---

From: David Nicol <[email-masked]>
Date: Sun, 19 Dec 2010 11:43:13 -0500

of course, NXDOMAIN is supposed to mask by the domain, which is the thing
that we get by adding the dots -- we don't have separation of administrative
control, but we can pretend we do, so the glue for each additional portion
-- with one nibble per section, as per the current RFC, that amounts to
masking as the four-bit search level, and the problem is that the
representation works great as long as the empty ranges don't give us any
information except "this is an empty range."

So.  A RFC-5782 compatible protocol that can tell us something more
interesting about the ranges without points can be created by having some
records at the top of the missing trees.

Given a list containing a:b::1 and examining searches for a:b::2 through
a:b::ffff, we would have something like the following, resolving a search
for a:b::beef.

We're looking for the status of

f.e.e.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example

and there is data for a:b::1, which is different from a:b::2 through
a:b::ffff which are all the same, and there is something we wish to
communicate about all those points, as a range.

we get a NXDOMAIN for the

b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example

domain, while acquiring glue to resolve the query, which stops the query.

Here's where we diverge:

instead of just giving up, the client library starts dropping portions from
the beginning of the query until it gets an A record:

f.e.e.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example  no
such domain

e.e.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example no
such domain
e.b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example no such
domain
b.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example no such
domain
0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.a.0.example response
indicating the general state of the range

the majority of the traffic is between the query library and the cache; if
the cache and the query library (a query library that is its own resolver)
would be able to do that all in-memory instead of as query/response
messages.

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 20 Dec 2010 14:32:40 +0100

> instead of the current "default accept, deny from blacklisted, plus

+1, although this might be one of those good things that we already
lost (like avoiding fallback-to-AAAA.)

I don't mean that IP address is a clever authentication method, but
currently it is the most widely available one.

BTW, http://blog.teamlazerbeez.com/2009/09/14/email-on-ipv6/ assert
that some 1.9% sampled domains have one or more MX records with at
least one host on IPv6.  Are there more recent stats?