Subject: Re: [v6ops] New Version Notification for draft-palet-v6ops-464xlat-opt-cdn-caches-02.txt

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 07 Jul 2019 14:49:14 +0200

Hi Martin,

I've been thinking about this and I think you're right. We should consider a solution that doesn't break the reverse-proxy cases, even if this means that they aren't optimized.

The best approach to me seems to avoid creating an EAMT entry (and thus not optimizing those cases), in order to avoid breaking anything or creating other delays in case a client already started the communication with an existing EAMT entry and another client is then connecting to another service using the same reverse-proxy.

This is the case we have:
    1) Client asks for A record of a.example.com.
    2) CPE would also ask for AAAA.
    3) CPE receives:
    a.example.com        IN A		192.0.2.1
    a.example.com	IN AAAA	2001:db8::a
    4) CPE makes EAMT entry:
    192.0.2.1 <-> 2001:db8::a
    5) CPE sends A record to client.
    6) Client starts to communicate with 2001:db8::a without knowing it.
    7) Client (same or another) asks for A record of b.example.com.
    8) CPE would also ask for AAAA.
    9) CPE receives:
    b.example.com	IN A		192.0.2.1
    b.example.com	IN AAAA	2001:db8::b
    10) CPE does what with EAMT entry? There is already EAMT entry present for 192.0.2.1!

Let's see the options:
a) Each time the CPE uses an EAMT (already created) re-check with an A and AAAA query if it is consistent with the existing EAMT. In the case there is no match, it means it is a reverse-proxy and the communication is to a different server. We could drop the EAMT and create a new one, but then existing communication with previous clients is broken. It may be recovered by the application, but this is a recurrent query re-check and will happen every time in the same network multiple clients use different resources from the same reverse-proxy. I think is better don't risking for it.

b) Add a field in each EAMT entry, with the SNI that has been queried. One more field to mark that entry as "invalid" (not to be optimized). This is also useful if you want to manually use a GUI in the CPE (or by provisioning) to disallow optimization to some destinations.

If the CPE receives a query for 
    a.example.com        IN A		192.0.2.1
    a.example.com	IN AAAA	2001:db8::a
It creates the EAMT entry and marks it as valid.

If afterwards, it gets a query for 
    b.example.com	IN A		192.0.2.1
    b.example.com	IN AAAA	2001:db8::b

and the TTL is still fine, but the CPE realizes that the entry A/AAAA/FQDN (b.example.com 192.0.2.1 2001:db8::b) is conflicting with the existing one (a.example.com 192.0.2.1 2001:db8::a). The conflict is the existence of the same IPv4 address for different IPv6 ones with also have different FQDN. In other cases, there is no conflict. It means that there is a reverse-proxy (or equivalent) and the EAMT needs to be invalidated.

This allows the optimization only if one or several clients (on this CPE) are using a single resource of a specific reverse proxy. But disables optimization if multiple clients in the same CPE are using multiple resources from the same reverse-proxy (if they share the same IPv4 address, but different IPv6 ones).

Of course, the EAMT still expires with the TTL, so it allows to re-enable optimization if anything changed.

What do you think? Other options?

Regards,
Jordi
@jordipalet

El 25/6/19 1:46, "v6ops en nombre de Martin Hunek" <[email-masked] en nombre de [email-masked]> escribió:

    Hi Jordi,
    
    don worry I'll manage, I wrote that just that you know some clients (kmail in my case) have issue with this quotation style. :-)
    
    Anyway, see reply in-line (shortened to relevant parts).
    
    Regards,
    Martin
    
    Dne pondělí 24. června 2019 23:39:52 CEST, JORDI PALET MARTINEZ napsal(a):
    > Hi Martin,
    > 
    > I use Outlook and it doesn't allow (as I know) anything else than indentation. I will try to find a solution, I just need some time ... but is not top priority.
    > 
    > When I see my emails back in the list, it works for me, so it is dificult ...
    > 
    > See below, in-line.
    > 
    > Regards,
    > Jordi
    > @jordipalet
    >  
    >  
    > 
    > El 24/6/19 23:14, "v6ops en nombre de Martin Hunek" <[email-masked] en nombre de [email-masked]> escribió:
    >     > El 24/6/19 0:28, "v6ops en nombre de Martin Hunek" <[email-masked] en nombre de [email-masked]> escribió:
    >     >     How did you solve situation in which multiple IPv6 services would share single IPv4 to IPv6 proxy?
    >     > 
    >     > When you mean a proxy, what do you exactly mean? I'm confused here with your example. Do you refer to SNI ? If that's the case, I believe it is resolved already, because what it matters here is the IPv6 address (the AAAA).
    >     
    >     Sort of. I mean essentially IPv4 web proxy shared by multiple IPv6 only servers. Proxy would use SNI to distinguish between target server to which it would send payload to. Thing is that you won't be able to do concurrent bindings in EAMT table for this case.
    > 
    > -> We need to see if this is really common. My guess is that even if you use SNI, the IPv4 and the IPv6 page is still hosted by the same dual-stack server. In that case, I don't think we have the problem you mention. You are describing a case where the IPv4 and IPv6 content is hosted by a different server, not a dual-stack one, right? We really need to check if CDNs/caches do that.
    
    Actually hosted on single (IPv6-only) server, on IPv4 there would be only proxy. By Apache terminology "reverse proxy". For a moment forget CDNs/caches, those wouldn't probably need that as they do have different tools how to deal with IPv4 runout. Think more like VPS providers. This is likely to happen when operator is running IPv6-only infrastructure.
    
    >     > Remember that what it matters here is the contents in IPv6, because the EAMT entry will bring the SmartTV (for example) either to the AAAA for a.example.com or b.example.com. So, if those AAAA provide the right address for IPv6, for the right service, it should work correctly.
    >     
    >     No problem on IPv6 side, problem would be to which IPv6 destination should IPv4 destination get mapped to. As in that case you get two bindings in EAMT:
    >     192.0.2.1 <-> 2001:db8::a
    >     192.0.2.1 <-> 2001:db8::b
    >     Both would be valid but which should be chosen by NAT46?
    >     > 
    >     > Otherwise (according to your example), you get different results for an IPv4 only host and IPv6-only hosts. Normally this is not done in production services, and again, if this is done, the result of the optimization is like the IPv4-only SmartTV is actually an IPv6-enabled one, so it works again, as intended.
    >     
    >     Maybe it is bit "ad absurdum" case, certainly for CDNs. But I think that it is not discouraged/prohibited and I can see its use for VPS provider which would like to save some IPv4 addresses. By the way, it would be somehow similar to what you are proposing: get IPv6-only services accessible by IPv4-only hosts. Only tools would be different.
    > 
    > -> We need to see if this is a real and usual production case. IPv6-only services today can be deployed with SIIT-DC to allow dual-stack access, but this will not suffer the problem you mention.
    
    You are right, but SIIT-DC requires as many IPv4 addresses as IPv4-enabled services (n:n). It provides stateless translation. I'm talking one IPv4 address for multiple IPv4-services (1:n).
    >     > 
    >     > This means (and I think is the *normal* way) to do that, that the "server" is dual-stack and is the "apache" (or whatever), using the SNI the way that resolves the problem. Is that what you mean?
    >     
    >     No. I case I'm trying to present multiple servers would be IPv6-only. And A record would point to web proxy (dual-staked) which would proxy web requests according to SNI (or request in HTTP) to appropriate IPv6-only server. Hosting provider would save IPv4 addresses and IPv6-only web server would still be accessible by IPv4. Clearly win-win, except that it would be broken by this optimization.
    > 
    > -> Avoiding the EAMT entry will resolve the problem, but will not be optimized.
    
    Yes it would solve that problem, but as it is written right now, you have no way of knowing that for this specific address EAMT entry should be avoided.
    
    This is how it could get broken and how NAT46 device would be able to detect that:
    1) Client asks for A record of a.example.com.
    2) CPE would also ask for AAAA.
    3) CPE receives:
    a.example.com        IN A              192.0.2.1
    a.example.com		IN AAAA	2001:db8::a
    4) CPE makes EAMT entry:
    192.0.2.1 <-> 2001:db8::a
    5) CPE sends A record to client.
    6) Client starts to communicate with 2001:db8::a without knowing it.
    7) Client (same or another) asks for A record of b.example.com.
    8) CPE would also ask for AAAA.
    9) CPE receives:
    b.example.com        IN A              192.0.2.1
    b.example.com		IN AAAA	2001:db8::b
    10) CPE does what with EAMT entry? There is already EAMT entry present for 192.0.2.1!
    
    Option A: CPE would simply update EAMT entry:
    10a) EAMT entry gets updated to:
    192.0.2.1 <-> 2001:db8::b
    11a) CPE sends A record to client.
    12a) Client starts communication with 2001:db8::b but connection with 2001:db8::a is severed
    Result: a.example.com is not accessible until its TTL expire.
    
    Option B: CPE ignores new EAMT entry:
    10b) CPE checks if A record for a.example.com have been changed if yes than option A, if not than:
    11b) EAMT entry stays:
    192.0.2.1 <-> 2001:db8::a
    12b) CPE sends A record to client.
    13b) Client is unable to connect to b.example.com but connection to a.example.com is not interrupted.
    Result: b.example.com is not accessible until TTL of a.example.com expire.
    
    Option C: CPE excludes 192.0.2.1 from EAMT:
    10c) CPE checks if A record for a.example.com have been changed if yes than option A, if not than:
    11c) EAMT entry of 192.0.2.1 <-> 2001:db8::a gets deleted:
    12c) CPE sends A record to client.
    13c) Client is able to connect to b.example.com but connection to a.example.com is momentary interrupted.
    Result: Session made to a.example.com via IPv6 is severed and had to be reinitiated, but both are accessible.
    
    Option D: CPE handles additional entries for 192.0.2.1 as it would not have A record:
    10d) CPE checks if A record for a.example.com have been changed if yes than option A, if not than:
    11d) CPE generates pseudo IPv4 address 198.51.100.1 for 2001:db8::b
    12d) CPE makes EAMT entry:
    198.51.100.1 <-> 2001:db8::b
    13d) CPE send 198.51.100.1 to client.
    14d) Client communicate with 2001:db8::b, communication with 2001:db8::a is not interrupted.
    Result: If client is validating DNSSEC, then validation would fail. Same way as if service would be IPv6-only as A query would end up with NSEC(3) with NODATA response. If client attempts validation b.example.com would not be accessible.
    
    >     
    >     Just so we would be on same page, connections to such deployment would look like this:
    >     a) IPv4 Client <--- IPv4 ---> CLAT <--- IPv6 ---> NAT64 <--- IPv4 ---> Proxy <--- IPv6 ---> IPv6 Server
    >     b) IPv6 Client <--- IPv6 ---> IPv6 Server
    >     
    >     You are proposing:
    >     c) IPv4 Client <--- IPv4 ---> CLAT <--- IPv6 ---> IPv6 Server
    >     We can probably all agree that "b" is best and "c" is better than "a". Problem is that you have no way of knowing if you are talking to actual server or just proxy.
    >     
    >     Another example when this optimization would not work exactly as planted would be case in which service provider deployed IPv6 by NAT64 placed before its IPv4-only infrastructure. But in this case I would say that such operator would deserve higher load on its NAT64. :-)
    >     This would be case:
    >     d) IPv6 Client <--- IPv6 ---> NAT64 <--- IPv4 ---> IPv4 Server
    >     Optimization would result in:
    >     e) IPv4 Client <--- IPv4 ---> CLAT <--- IPv6 ---> NAT64 <--- IPv4 ---> IPv4 Server
    >     But in this case NAT64 would be in the servers' network rather than clients'. This is actually good as it is transferring load from network doing IPv6 right to network doing it wrong. :-)
    > 
    > -> I think d) is using "only" NAT64, but it is a broken model, because it doesn't work for IPv4-only hosts behind the CPE. So not in scope for our document. e) is the "normal" case for 464XLAT when we don't have the optimization described in this document.
    
    Never mind d) and e), this is not important as this would not be actually broken. Difference is between transporting IPv4 or IPv6 packets over internet. IPv4 without optimization, IPv6 with.
    > 
    >     > 
    >     >     Possible solution:
    >     >     Maybe this could be solved by some kind of "blacklist for shared IPv4 addresses with different IPv6 target" thing. This situation should be detectable if there would be EAMT record already present with different IPv6 address. Then if CPE would ask for DNS record which created EAMT record and DNS record has not changed it should delete EAMT and blacklist that IPv4 address so it would not make more EAMT records for it.
    >     > 
    >     > Right now, we have this in the ID:
    >     > TBD.  Should a way to manually exclude EAMT entries be considered?
    >     >    May be a manual config in the CPE and by means of operator config.
    >     >    This is way-out to ensure nothing is broken by surprise and is not
    >     >    solvable.
    >     > 
    >     > May be what you mention help to be able to also do it automated.
    >     
    >     Doing it manually would not quite cut it as you are proposing to done optimization on CPE and as it may be under customer control, you may not be able to provision it with updated EAMT excludes. It had to be somehow automated as it would be hard to instruct customer to modify EAMT by hand.
    > 
    > -> Agree that manually only makes sense if there is a strange case or the user for some reason doesn't want optimization. It is a special case 99% of users will not use it, but must be supported.
    >     
    >     However there is also trade-off with automated approach as it would have to either severe session for user which is already using present binding (when conflicting binding appears). As user starts to communicate over IPv4 (from servers' perspective) as it would get excluded from EAMT. Or you would have to forge fake IPv4 for conflicting binding and corresponding A record, which would break DNSSEC.
    > 
    > -> I can't parse the complete paragraph ...
    
    See Option C and Option D above. Thing is that you had to choose between terminating current connection by deleting EAMT entry or making fake address for new connections.
    >
    > _______________________________________________
    > v6ops mailing list
    > [email-masked]
    > https://www.ietf.org/mailman/listinfo/v6ops
    > 
    
    _______________________________________________
    v6ops mailing list
    [email-masked]
    https://www.ietf.org/mailman/listinfo/v6ops

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.

---

From: JORDI PALET MARTINEZ <[email-masked]>
Date: Sun, 07 Jul 2019 14:53:30 +0200

Disregard the previous one, small but important type to understand the option b correctly.

Hi Martin,

Let's see the options:

b) Add a field in each EAMT entry, with the FQDN that has been queried. One more field to mark that entry as "invalid" (not to be optimized). This is also useful if you want to manually use a GUI in the CPE (or by provisioning) to disallow optimization to some destinations.

and the TTL is still fine, but the CPE realizes that the entry to be created A/AAAA/FQDN (b.example.com 192.0.2.1 2001:db8::b) is conflicting with the existing one (a.example.com 192.0.2.1 2001:db8::a). The conflict is the existence of the same IPv4 address for different IPv6 ones with also have different FQDN. In other cases, there is no conflict. It means that there is a reverse-proxy (or equivalent) and the EAMT needs to be invalidated, so no more optimization will be performed for that IPv4 address.

Regards,
Jordi
@jordipalet

    Hi Jordi,

    
    Regards,
    Martin