Subject: [pcp] How to guarantee the same PCP/AFTR be selected in ip-in-ip anycast deployment

From: Qiong <[email-masked]>
Date: Fri, 15 Mar 2013 06:01:14 +0800

Hi all,

I'm aware there was discussion on whether to use native IPv6 or ip-in-ip
for PCP. Sorry to re-open the discussion since  we have encountered real
problems when deploying anycast-based DS-Lite.

In our DS-Lite deployment, different AFTRs will be configured with the same
address to support load-balancing and announced with the same metric into
IGP. PCP server is co-located with AFTR, and there are multiple layer-3
hops between B4 and AFTR.

Currently, most intermediate routers along the path use 5-tuple by default
(source address, destination address, source port, destination port and
protocol) as the hashing index for native IPv6 PCP requests to determine
which PCP server will be selected. However, since the following data
traffic is ip-in-ip, the intermediate routers can not see the encapsulated
port numbers and they can only use 3-tupe (source address, destination
address, protocol) as the hashing index. In this case, there is no
guarantee that the same PCP/AFTR will be selected as the hashing index is
different, and therefore, the mapping will not consistent with different
AFTRs.

I'm hoping to find solutions to address this problem. Should we use
consistent PCP transportation as the following data traffic, e.g. ip-in-ip
in DS-Lite ?

Thanks in advance !

Best wishes
-- 
==============================================
Qiong Sun
China Telecom Beijing Research Institude

Open source code:
lightweight 4over6: *http://sourceforge.net/projects/laft6/*
PCP-natcoord:* http://sourceforge.net/projects/pcpportsetdemo/ *
===============================================

---

From: Dan Wing <[email-masked]>
Date: Thu, 14 Mar 2013 18:23:48 -0400

On Mar 14, 2013, at 6:01 PM, Qiong <[email-masked]> wrote:

> Hi all,
> 
> I'm aware there was discussion on whether to use native IPv6 or ip-in-ip for PCP. Sorry to re-open the discussion since  we have encountered real problems when deploying anycast-based DS-Lite.
> 
> In our DS-Lite deployment, different AFTRs will be configured with the same address to support load-balancing and announced with the same metric into IGP. PCP server is co-located with AFTR, and there are multiple layer-3 hops between B4 and AFTR. 
> 
> Currently, most intermediate routers along the path use 5-tuple by default (source address, destination address, source port, destination port and protocol) as the hashing index for native IPv6 PCP requests to determine which PCP server will be selected. However, since the following data traffic is ip-in-ip, the intermediate routers can not see the encapsulated port numbers and they can only use 3-tupe (source address, destination address, protocol) as the hashing index. In this case, there is no guarantee that the same PCP/AFTR will be selected as the hashing index is different, and therefore, the mapping will not consistent with different AFTRs.
> 
> I'm hoping to find solutions to address this problem. Should we use consistent PCP transportation as the following data traffic, e.g. ip-in-ip in DS-Lite ?

Putting PCP aside for a moment, if the entire 5-tuple is hashed and decides which AFTR is used, that means traffic from the same host will be sent to different AFTRs, which means the traffic will have different public IP addresses (after being NATted by the AFTR).  This will break passive-mode FTP (which is the default in every popular FTP client), and will break many HTTP websites (which include the source IP address in their authentication cookies; often these are baking sites).  Seems this is a problem more significant than PCP.  I imagine there are other protocols that have implicit assumptions that all connections come from the same IP address, because that has been the standard function of hosts on the Internet for a long while.

Also, I don't see a problem to this, no matter if PCP uses an IANA-assigned anycast address, or if there is a DHCP-assigned address, or anything other address.  What might work is that every router along the path between the B4 and the AFTR be smart enough to apply their same hash to the PCP messages as to native traffic.  

Or, don't use anycast with a CGN.  I don't see how anycast can work well, especially for FTP and (certain) HTTP cookies or possibly other protocols.

-d

> lightweight 4over6: http://sourceforge.net/projects/laft6/
> PCP-natcoord: http://sourceforge.net/projects/pcpportsetdemo/ 
> _______________________________________________
> pcp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/pcp

---

From: Qiong <[email-masked]>
Date: Fri, 15 Mar 2013 06:59:41 +0800

Hi Dan,

Thanks for the reply. Please see inline.

On Fri, Mar 15, 2013 at 6:23 AM, Dan Wing <[email-masked]> wrote:

> In our DS-Lite deployment, different AFTRs will be configured with the
> same address to support load-balancing and announced with the same metric
> into IGP. PCP server is co-located with AFTR, and there are multiple
> layer-3 hops between B4 and AFTR.
>
> Putting PCP aside for a moment, if the entire 5-tuple is hashed and
> decides which AFTR is used, that means traffic from the same host will be
> sent to different AFTRs, which means the traffic will have different public
> IP addresses (after being NATted by the AFTR).  This will break
> passive-mode FTP (which is the default in every popular FTP client), and
> will break many HTTP websites (which include the source IP address in their
> authentication cookies; often these are baking sites).  Seems this is a
> problem more significant than PCP.  I imagine there are other protocols
> that have implicit assumptions that all connections come from the same IP
> address, because that has been the standard function of hosts on the
> Internet for a long while.
>
[Qiong] For DS-Lite itself, since the intermediate routers can not see the
inner port numbers, they can only use 3-tupe (source address, destination
address and protocol) to do the hashing. The destination address is always
the tunnel end-point address. So it can be guaranteed that the same AFTR
would be selected for the same user. Therefore, all the upstream traffic
will have the same public v4 address. So anycast for DS-Lite itself will
not encounter this problem.

>
> Also, I don't see a problem to this, no matter if PCP uses an
> IANA-assigned anycast address, or if there is a DHCP-assigned address, or
> anything other address.  What might work is that every router along the
> path between the B4 and the AFTR be smart enough to apply their same hash
> to the PCP messages as to native traffic.
>
[Qiong] To guarantee that every router applies the same hash is really
difficult in reality. Different devices (even different versions) will have
different default behavior on determine the hashing.

Best wishes
Qiong

>
> Or, don't use anycast with a CGN.  I don't see how anycast can work well,
> especially for FTP and (certain) HTTP cookies or possibly other protocols.
>
> -d
>
>
> Thanks in advance !
>
> Best wishes
> --

Open source code: