Subject: Re: [Softwires] I-D Action:draft-lee-softwire-dslite-deployment-00.txt

From: Brian E Carpenter <[email-masked]>
Date: Wed, 27 Oct 2010 15:13:39 +1300

Hi,

> 2.  AFTR Deployment Considerations
...
>    IPv4.  Although an operator can configure a dual-stack interface for
>    both functions, we strongly recommended to configure two individual
>    interfaces (i.e. one dedicated for IPv4 and one dedicated for IPv6)
>    to segregate the functions.

Can you clarify whether this means two physical interfaces or
two logical software interfaces?

> 2.1.  MTU Considerations
...
>    For fragmentation problem shares among all the tunneling protocols,
>    this is not unique to DS-Lite.  The IPv4 packet isn't over-sized, it
>    is the v6 encapsulation that MAY cause the oversized issue.  

I must confess I am a bit confused. The IPv6 minimum MTU is so much larger
than 576 that I don't see how you can fail to support the IPv4 minimum
MTU.

OK, suspending my disbelief on that point:

>                                                                 So the
>    tunnel points are responsible to handle the fragmentation.  In
>    general, the Tunnel-Entry Point and Tunnel-Exist Point should
>    fragment and reassemble the oversize datagram.  

I understand that, if DF is not set, the tunnel entry point (or more
accurately, the IPv4 entity forwarding the packet into the tunnel
entry point) should fragment the packet. But unless you are re-inventing
SEAL, the tunnel exit point will just forward the fragments, won't it?

> 2.2.  Lawful Intercept Considerations

Given RFC 2804, I suggest deleting this section.

> 2.3.  Logging at the AFTR

The arguments in this section are pretty powerful, but I wonder
whether they belong in v6ops, rather than in a more general analysis
of CGN/LSN/NAT444 issues such as draft-ietf-intarea-shared-addressing-issues.

> 2.6.  Strategic Placement of AFTR
...
>    The AFTR architecture design, then, is mostly figuring out the
>    strategic placement of each AFTR to best use the capacity of each
>    public IPv4 address without oversubscribing the address or overtaxing
>    the AFTR itself.  Although only a few studies of per-user port usage
>    have been done, an AFTR should be able to support 3000 - 5000 users
>    per public IPv4 address.

I find this number astonishing. We know that 200 open ports per user has
been reported as typical in some ISP secnarios; that suggests that even as
many as 300 users per public adress is close to the limit.

Regards
   Brian Carpenter

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 07:21:36 +0000

Hi Brian,

Thanks for taking the time to review it. Comments inline.

Regards,
Yiu

>two logical software interfaces?

[YL] We recommend to use two physical interfaces. This is better for many
monitoring tools to collect the netflow information from the routers.

>SEAL, the tunnel exit point will just forward the fragments, won't it?

[YL] Ok. I think we need to rewrite it to make it clear. All we are saying
is the IPv4 host behind B4 doesn't know the IPv4-in-IPv6 tunnel. So it is
possible that the host sends a 1500-byte packet. When B4 receives the
packet, the packet will be over-size with the tunnel overhead. In this
case, B4 must encapsulate the IPv4 into an IPv6 package first, then
perform IPv6 fragmentation. B4 must not fragment the IPv4 packet before
the encapsulation.

>Given RFC 2804, I suggest deleting this section.

[YL] Ok. I will need to read RFC2804 before giving comments.

>of CGN/LSN/NAT444 issues such as
>draft-ietf-intarea-shared-addressing-issues.

[YL] Agreed. We will reference the shared-addressing draft. However, we
still want to give some considerations to the operators what they should
be aware. In this end, this is a deployment draft.

[YL] We will present this in softwire meeting. If the WG thinks this
should be settled in v6ops, I have no problem for it.

>many as 300 users per public adress is close to the limit.
[YL] When we said 3000 - 5000 users, we mean *all* users including active
and inactive users. We will rewrite this to make this clear.

---

From: Arnaud Ongenae <[email-masked]>
Date: Wed, 27 Oct 2010 10:24:54 +0200

comments inline too :)

On 10/27/2010 09:21 AM, Lee, Yiu wrote:
> On 10/26/10 10:13 PM, "Brian E Carpenter"<[email-masked]>
> wrote:
>
>> Hi,
>>

[AO] monitoring method depends on the vendor way of achieving it. I 
think that no recommendation is needed on the "standard"/"generic" point 
of view but rather by a vendor specific implementation.

> perform IPv6 fragmentation. B4 must not fragment the IPv4 packet before

[AO] Does the B4 shouldn't try to identify the Tunnel MTU and advertise 
a lower value to the hosts so they avoid to send too large packets and 
so the B4 doesn't have to fragment.
According to me the MTU problem is more on the out-to-in path.

> and inactive users. We will rewrite this to make this clear.

[AO] Once again, this is highly related to the vendor implementation, 
it's really difficult to define a "pertinent" strategic placement of the 
AFTR.
-You could have a static number of ports allocated per B4 subscriber 
That subscriber hide many hosts. Each host uses around 200 ports. Alone 
or not you don't have more or less resources. So even 300 B4 subscribers 
per IPv4 is too high.
-You could have dynamic per host port allocation and thousands of hosts 
behind one IPv4 is possible...
I believe that putting figures is always tricky in a generic reference.

Regards,

Arnaud Ongenae

---

From: Rémi Després <[email-masked]>
Date: Wed, 27 Oct 2010 14:08:50 +0200

Hi, Yiu.

I fid this draft useful. 
More comments below.

> perform IPv6 fragmentation. B4 must not fragment the IPv4 packet before

The following alternative should IMHO be at least permitted:
The IPv4 packet with DF not set is split into fragments small enough for each one to fit, with its encapsulation header, in the tunnel MTU.
Thus:
- Reassembly of IPv6 isn't needed at tunnel exit (simpler and causing less delays)
- the overhead of the IPv6 extension header that is needed for fragmentation is avoided.

>> ...
>>>   Although only a few studies of per-user port usage

> and inactive users. We will rewrite this to make this clear.

The maximum number of ports that customers will be authorized to use, both on the average and individually, should IMHO remain free from any recommended numbers.
Each provider will have to make its decision based on many parameters of its own.
In particular, as IPv6 takes momentum, less and less IPv4 ports per customer will be needed over time.

A proposal would be to delete the sentence as being out of scope. 
(No one who may have problems with a choice in the 3000-5000 range should be able to say that IETF is responsible, because of the "should".) 

Regards,
RD

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 14:24:17 +0000

Hi Arnaud,

That's fine and I agree that we should not specify any real number in the
draft. 

For the AFTR interface, after reading your and Brian's comments, I will
only make a recommendation that the v4 and v6 interfaces should be
separated (either logically or physically). It is possible that you can
configure a single I/F on the AFTR (assign both v4 and v6 addresses to the
same interface) so that the tunnel will be terminated on the same
interface which is used to send out the NAT-ed v4 packets. I think this is
a bad deployment because it is hard to troubleshoot (at least when I did
it in my lab testing). Anybody objects this?

Thanks,
Yiu

On 10/27/10 4:24 AM, "Arnaud Ongenae" <[email-masked]>
wrote:

>>>>                                                                  So
>>>>the
>_______________________________________________
>Softwires mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/softwires

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 14:30:31 +0000

>According to me the MTU problem is more on the out-to-in path.

[YL] Indeed yes. But there exists hosts that don¹t listen to ICMP packet
(personal fw, etc). This is a safe-guard mechanism to prevent legal MTU
sized packet being dropped.

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 14:36:54 +0000

Hi Remi,
>The IPv4 packet with DF not set is split into fragments small enough for
>each one to fit, with its encapsulation header, in the tunnel MTU.
>Thus:
[YL] The fragmentation happens because of a tunnel between the B4 and AFTR
in the access network. I think this is the B4 and AFTR who created the
tunnel are responsible for fixing the problem. If B4 fragmented the v4
packet, this is a possibility that the target server may drop the
fragmented v4 packet due to security concerns. This will affect the user
experience behind the b4. We did some scaling test on ISC AFTR
implementation and an open source B4 implementation. The fragmentation and
reassemble didn't create a lot of overhead especially the AFTR would
receive both v6 packets almost immediately. It doesn't require the AFTR to
buffer for too long.

[YL] Also, this will also require some logic in the B4 to check the DF bit
and make decision what to do.

>
>The maximum number of ports that customers will be authorized to use,
>both on the average and individually, should IMHO remain free from any
>recommended numbers.
>(No one who may have problems with a choice in the 3000-5000 range should
>be able to say that IETF is responsible, because of the "should".)
>
[YL] Agreed and will remove it from the next revision.

Thanks,
Yiu

---

From: Rémi Després <[email-masked]>
Date: Wed, 27 Oct 2010 16:47:10 +0200

Le 27 oct. 2010 à 16:36, Lee, Yiu a écrit :

> tunnel are responsible for fixing the problem.

> If B4 fragmented the v4
> fragmented v4 packet due to security concerns.

I miss the point.
I don't see why servers would refuse fragmented datagrams for security reasons.

> This will affect the user

The behavior you described clearly makes sense. 
The point is ONLY that an alternative behavior, which also makes sense, should be authorized (not objected to by a "should") 

> and make decision what to do.

Right.
That's indeed part of the alternative that IMHO should be authorized.

Thanks,
RD

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 15:50:06 +0000

Not the servers, but the firewall in front of the servers.....

On 10/27/10 10:47 AM, "Rémi Després" <[email-masked]> wrote:

>The point is ONLY that an alternative behavior, which also makes sense,
>should be authorized (not objected to by a "should")
>
>>> Each provider will have to make its decision based on many parameters
>>>of
>>> its own.

---

From: Rémi Després <[email-masked]>
Date: Wed, 27 Oct 2010 18:07:06 +0200

> Not the servers, but the firewall in front of the servers.....

Well, that's different.
But is it normal for a FW to refused fragmented IPv4 datagrams?
It seems to me that would seriously limit connectivity.

RD

---

From: Jacni Qin <[email-masked]>
Date: Thu, 28 Oct 2010 00:19:42 +0800

One more comment on Section 2.8,

"...It is recommended the AFTR to copy the DSCP value in the IPv4 header to
the IPv6 header after the encapsulation."

I guess a similar function is needed on B4 for the upstream traffic. Add it
somewhere in the B4 section? :-)

Cheers,
Jacni

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 16:32:51 +0000

Agreed. Thx.

From: Jacni Qin <[email-masked]<mailto:[email-masked]>>
Date: Thu, 28 Oct 2010 00:19:42 +0800
To: Brian E Carpenter <[email-masked]<mailto:[email-masked]>>
Cc: Softwires <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Softwires] I-D Action:draft-lee-softwire-dslite-deployment-00.txt

"...It is recommended the AFTR to copy the DSCP value in the IPv4 header to the IPv6 header after the encapsulation."

I guess a similar function is needed on B4 for the upstream traffic. Add it somewhere in the B4 section? :-)

Cheers,
Jacni

On Wed, Oct 27, 2010 at 10:13 AM, Brian E Carpenter <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

...

...

...

Regards
  Brian Carpenter
_______________________________________________
Softwires mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/softwires

_______________________________________________ Softwires mailing list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/softwires

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 16:38:12 +0000

There are many recorded attacks about IP fragmentation (search for Rose
Frag attack). Some firewalls are smart enough to identify whether it is a
real attack or not. Some simply drop any fragmentation packet. I am not
going to argue which is a better policy. My point is if we can limit the
fragmentation only happening between b4 and AFTR, this will save us a lot
of headache in future.

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 27 Oct 2010 13:43:02 -0700

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Lee, Yiu
> Sent: Wednesday, October 27, 2010 9:38 AM
> To: Rémi Després
> Cc: Softwires
> Subject: Re: [Softwires] 
> I-DAction:draft-lee-softwire-dslite-deployment-00.txt
> 

Hold on there. Fragmentation and reassembly is a fundamenal
function of the IP service model - has been for decades.
There are major Internet services that serve content while
setting DF=0. Isolated large packets that must be delivered
without being lost due to an MTU restriction use DF=0.
End-to-end fragemntation and reassembly between consenting
end systems is also available even if the network itself
does not fragment. Are you saying we should disable a
fundamental function of the IP service model? And trace
down all of the places where things would break as result?

Fred
[email-masked]

> >>> I don't see why servers would refuse fragmented datagrams 
> for security
> >>> reasons.
> >
> >
>

---

From: "Lee, Yiu" <[email-masked]>
Date: Wed, 27 Oct 2010 20:55:02 +0000

Hi Fred,

I am not saying to disable fragmentation. All we say is to ask b4 to
fragment the packet and AFTR to reassemble it. We are on the same page.

Yiu

---

From: "Templin, Fred L" <[email-masked]>
Date: Wed, 27 Oct 2010 14:18:14 -0700

> From: Lee, Yiu [mailto:[email-masked]] 
> Sent: Wednesday, October 27, 2010 1:55 PM
> To: Templin, Fred L; Rémi Després
> Cc: Softwires
> Subject: Re: 
> [Softwires]I-DAction:draft-lee-softwire-dslite-deployment-00.txt
> 
> Hi Fred,
> 

I am just dropping in and have not followed the discussion
closely, but if you are asking for reassembly by a network
middlebox element then I disagree. End system OSs are not
dumb and have had working IP reassembly implementations
for decades. AFAICT, they do not have to be coddled by
intelligent middleboxes that think they know better.

Destination end systems will benefit from receiving the
fragments since they will then have knowledge of when to
tell the source end systems that they are fragmenting. Also,
fragmentation and reassembly may be by a mutual agreement
between source and destination end systems which would be
disrupted by reassembling middleboxes. Plus, due to routing
changes a reassembling middlebox may have no way of knowing
whether it will see all of the fragments of the same datagram.
And for sure, a reassembling middlebox will have no way of
knowing whether further fragmentation will occur on the
path to the destination.

I have heard something about "virtual fragment reassembly"
(sic) and if I understand the idea correctly I'm not sure
I like it. Is this what you had in mind?

Fred

---

From: Brian E Carpenter <[email-masked]>
Date: Thu, 28 Oct 2010 11:41:41 +1300

You can refer to RFC 2983 "Differentiated Services and Tunnels."
(Informational RFC)

Regards
   Brian Carpenter

---

From: "Lee, Yiu" <[email-masked]>
Date: Thu, 28 Oct 2010 06:36:13 +0000

Thx Brian, will do.

On Oct 27, 2010, at 6:41 PM, "Brian E Carpenter" <[email-masked] 

>> To: Brian E Carpenter <[email-masked]<mailto:[email-masked] 
>> >>
>> "...It is recommended the AFTR to copy the DSCP value in the IPv4  
>> header to the IPv6 header after the encapsulation."
>>
>> I guess a similar function is needed on B4 for the upstream  
>> traffic. Add it somewhere in the B4 section? :-)
>>
>>
>> Cheers,
>> Jacni
>>
>>
>>
>> On Wed, Oct 27, 2010 at 10:13 AM, Brian E Carpenter <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>> Hi,
>>
>> _______________________________________________ Softwires mailing  
>> list [email-masked]<mailto:[email-masked]> https://www.ietf.org/mailman/listinfo/softwires
>>

---

From: Rémi Després <[email-masked]>
Date: Thu, 28 Oct 2010 10:31:08 +0200

Le 27 oct. 2010 à 18:38, Lee, Yiu a écrit :

> real attack or not.

> Some simply drop any fragmentation packet. I am not

For clarification: is this to say that not only you prefer IPv6 fragmentation to IPv4 fragmentation at tunnel entrance, but also that you prefer to document only the first alternative?

Note that the Rose attack is based on "the first fragment and the last fragment of "a very large packet" (64k) [to be sent], but not the middle fragments".
It therefore doesn't concern the tunnel entrance fragmentation because all created fragments are transmitted.

RD