Subject: Comments on draft-ietf-ipngwg-ipv6-anycast-analysis

From: Erik Nordmark <[email-masked]>
Date: Wed, 5 Jun 2002 18:45:49 +0200 (CEST)

The anycast-analysis has been sitting in front of Thomas and I for quite a long
time while we were trying to figure out if the IETF needs to do something more
significant in the anycast space (such as an anycast WG; such as a way to
resolve or at least document the differences between the IPv4 DNS anycast
usage and the IPv6 architecture). But it is time to finish this document since
it provides useful information.

So here are our comments on draft-ietf-ipgnwg-ipv6-anycast-analysis-00.txt

In general this is a quite useful document since it attempts to write 
things down that haven't been written down before. But it makes sense to
make the document more clear and succinct, and perhaps also to
add some missing pieces to give a more complete picture.

Note that there are duplicate comments from Erik and Thomas here - hopefully
they are consistent and not contradictory :-)

Comments from Erik
------------------

I completely fail to see what section 1.1 is trying to say
other than "there are different forms of anycast".

It assigns the tag "BGP" anycast to both the Hardie and Ohta-san document,
even though only the latter is an interdomain anycast scheme.
And contrasting this against RFC 1546 seems odd, since RFC 1546 doesn't
say anything whether it limits itself to intradomain anycast or not.

Section 1.1 doesn't mention (but section 2.1 does) that RFC 1546 and RFC 2373
are different with respect to being able to syntactically distinguish
between unicast and anycast, which seems to be an important technical 
difference.
What high-level thing is section 1.1 trying to say?

> RFC2373 IPv6 anycast is very similar to RFC1546 IPv4 anycast.  The

Given RFC 1546's suggestion to use a separate range of addresses
for anycast, this doesn't seem to be true.

> BGP anycast tries to replicate unicast servers in distant domains.  The
> distribution of servers is worldwide.  BGP anycast is being used for
> specific upper-layer protocols only, like DNS and HTTP.  There is no
> consideration given for the cases when a client contacts multiple
> servers by chance (transport layer protocol will get confused), since it
> is unlikely to see BGP route changes during the short lifetime of the
> transport layer protocols being used.

While typically http connections might be short, there is nothing in
the http protocol that mandates or assumes that they are short.

----
Section 2.2 says:
  After we
  have discovered the unicast address of the server, we should use the
  server's unicast address for the protocol exchange.
It would be useful to state something about this security implications
(or high-level requirements) when doing something like that in the
security considerations section.

Section 2.3 should at a minimum have a "not yet" added - the magma WG 
is chartered to extend MLD to allow hosts to join anycast groups.
But perhaps it can be written more in the form of
	When RFC 2373 was written there was no standard way for
	hosts to join anycast groups (short of having the hosts
	fully participate in the routing protocol which has trust
	issues). Therefore, ...

If this section talks about the intent to extend MLD to support anycast, it
would also be useful to add a sentence or two about the security issues
in this area in the security considerations section.

Section 2.4 is correct, but doesn't help people understand what
the technical issues are in having a single source address identify
more than one node. Since these implications are not written
down anywhere as far as I know, it would make sense to add
the currently known implications in this document.

The ones I'm aware of are:
 - Incorrect reassembly of fragmented packets due to multiple anycast
   members sending packets with the same fragment ID to the same destination
   at about the same time; the same the source IP address, destination IP 
   address, nextheader, and fragment ID numbers might be accidentally used
   at the same time by different senders.

 - Errors and other response packets might be delivered to
   a different anycast member than sent the packet. This might be
   very likely since asymmetric routing is rather prevalent on the Internet.

   Particular cases of such errors that are known to cause protocol
   problems are
    - ICMP packet too big making path MTU discovery impossible.
    - <others>
   The misdelivery of other errors might cause operational problems - making
   the network harder to trouble-shoot when anycast source addresses are used.

Section 2.5.
Aren't there also replay protection issues when manual keying is used
for anycast addresses? (Such issues exist for multicast and manual
keying, so I'd be surprised if they didn't exist for anycast.)

Section 3.1 last paragraph seems to say that anycast can never be
used interdomain due to route scaling issues. This is clearly the
case if there are lots of interdomain anycast addresses being used.
But having lots of global anycast addresses where the anycast members
are in the same topological region, would presumably automatically
be aggregated as part of the prefix for advertised by that topological
region. (Whether one would call this interdomain or intradomain
is an interesting terminology issue; the point is that the anycast
service provided by multiple nodes in a single domain is accessible
from outside the domain.)

Section 3.4 and 3.5 don't seem to be finished - they just contain
a "TBD".
Is there something which you intended to put in there?
If not, presumably the sections can be dropped. (The document
doesn't *have to* suggest improvement everywhere.)

Section 4.1 says:
> Note that, however, bad guys can still inject fabricated results to the
> client, even if the client checks the source address of the reply.  The
> check does not improve security of the exchange at all.
I think that DNS folks would disagree with that.
Having the resolver check the source address of replies
prevents, when ingress filtering is in use, off-path attackers
from spoofing responses. Thus it adds some non-zero amount of
protection to do this.
Same IMHO incorrect statement is repeated in this section:
> The source address check itself has no real protection

It might make sense, for DNS query/replies over UDP, to instead look
into relaxing the anycast source address restriction under
appropriate constraints (e.g. must not send larger than MIN MTU
since path MTU discovery is unlikely to work).

Another possibility, which you may or may not want to add to the document,
is to specify a protocol which can establish a binding between the
anycast address and the unicast address of one of its members
with as much security as the routing system provides for routing
packets to the anycast address in the first place. IMHO such a protocol
can be built by reusing pieces of MIPv6.

I think (but I haven't verified) that (many) tftp implementations verify
the source address. Perhaps I'm confused and it is only the server that does
this and not the client. But using TFTP as an example of how secure protocols
should be done is not a very strong argument to say the least.

NITS:

> destination node, out of a group of destination nodes.  IP datagram will
s/IP/The IP/

> If multiple packets carry an anycast address in IPv6 destionation
Spell check needed - "destionation"?

>   check source address, based on RFC2181 section 4.1.  TFTP
The "TFTP" looks like extra characters at the end of the line.

References:
The Hardie document is now RFC 3258.

Comments from Thomas:
--------------------

> destination node.  The destination node is identified by an unicast

s/an unicast/a unicast/

>destination node, out of a group of destination nodes.  IP datagram will

s/datagram/datagrams/ ? (or "An IP datagram")

> based on routing measure of distance.  The source node does not need to
s/on/on the/

> Today, there are so-called "anycast" operated mostly for critical
> servers including DNS servers and web servers (like those for Olympic
> games) [Hardie, 2001; Ohta, 2000] .  We call the technique "BGP anycast"

Sentence doesn't parse. s/"anycast"/"anycast" services/

Also, name "BGP anycast" is not really very
accurate. draft-ietf-dnsop-hardie-shared-root-server-03.txt doesn't
rely on BGP only. The technique works also within a single AS where no
BGP is used.

> o A provider-independent IPv4 address prefix is allocated from an RIR.

Not required that the address be PI.

> o The address prefix is configured at multiple distant locations on the
>   Internet.
> o BGP routes are advertised from all of the locations.

Not sure what this means, especially the first bullet.

> BGP anycast tries to replicate unicast servers in distant domains.  The

Not sure I agree. The replication is within a single AS. That may or
may not involve services is "distant domains". Maybe you want to say
something like:

  BGP anycast tries to replicate unicast servers in different
  topological locations.

> distribution of servers is worldwide.  BGP anycast is being used for
> specific upper-layer protocols only, like DNS and HTTP.  There is no

Where is it being used for HTTP? The documents you cite talk about DNS
and UDP exclusively.

> specific upper-layer protocols only, like DNS and HTTP.  There is no
> consideration given for the cases when a client contacts multiple
> servers by chance (transport layer protocol will get confused), since it
> is unlikely to see BGP route changes during the short lifetime of the
> transport layer protocols being used.

It is misleading to say this is all a BGP issue. The Hardie document
says that within an AS routes are advertised statically (even if a
particular server becomes unavailable) precisely so that routes stay
pinned and don't change. Note that this is within an AS, so  its not
just BGP.

> RFC2373 anycast is defined in more generic manner, and does not limit
> the routing infrastructure nor upper-layer protocol, Therefore, RFC2373

The first part of the sentence suggests that 2373 does NOT limit
upper-layer protocols. This would imply that the "BGP" approach
does. Actually, some might argue the opposite. 2373 says a source
address cannot be an anycast address. This implies that higher-layer
protocols that make use of anycast addresses need to be changed to
understand anycast addressing. This is in contrast with the BGP
approach.

> depending on stability of the routing table.  The property leads to a
> couple of interesting symptoms.

s/symptoms/observations/ ?

> protocol exchange.  If we use more than 2 packets, 1st and 2nd packet

s/exchange/exchanges/

Text should make clear that it assumes when server responds, source
address is unicast, not anycast. And point out that this causes
problems for some existing protocols.

---

--------------------------------------------------------------------
IETF IPng Working Group Mailing List
IPng Home Page:                      http://playground.sun.com/ipng
FTP archive:                      ftp://playground.sun.com/pub/ipng
Direct all administrative requests to [email-masked]
--------------------------------------------------------------------

---

From: [email-masked]
Date: Thu, 20 Jun 2002 19:09:40 +0900

i'm trying to catch up with my pile of email...

>other than "there are different forms of anycast".

	i'll try to rework on it.

>say anything whether it limits itself to intradomain anycast or not.

	as far as I remember from the past presentations, Hardie draft did
	propose interdomain anycast scheme.  the diagram in appendix A look
	like intradomain, however, section 1.4 and 1.5 does not have any
	restriction on intra/interdomain.  the diagram I saw at dnsop meeting
	was interdomain (single prefix advertised with a single AS #, from
	multiple different location).

>should be done is not a very strong argument to say the least.

	many of tftp implementation do check address pairs, however, the
	specification clearly indicates that the address should not be checked.

>Where is it being used for HTTP? The documents you cite talk about DNS

	webserver for past olympic games.  i don't have any citable document
	on it.

itojun

---

From: Erik Nordmark <[email-masked]>
Date: Fri, 21 Jun 2002 10:24:02 +0200 (CEST)

> 	multiple different location).

The latter sounds like the Ohta draft.

> 	webserver for past olympic games.  i don't have any citable document

Was this usage interdomain or not?
Perhaps there is some IBM reference (even if it is not stable) that can be
added. Independent of that some more explanation of how it's been used 
with http would be helpful.

  Erik

---

From: [email-masked]
Date: Fri, 21 Jun 2002 17:40:18 +0900

>The latter sounds like the Ohta draft.

	it seemed to me Ohta draft and Hardie draft were exactly the same,
	modulo the usage policy of AS #s.  also for me Hardie draft (now RFC)
	does not seem to put any restriction to be intradomain.  it just
	doesn't say anything about intra/interdomain.

>with http would be helpful.

	interdomain.  single prefix advertised with a single AS #, from
	multiple different location (route injected into multiple different
	ISPs).

itojun