Subject: [tcpm] Exceeding value in MSS option?

From: Martin Duke <[email-masked]>
Date: Mon, 19 Oct 2020 12:22:43 -0700

Hello tcpm,

Section 4.2.2.6 of RFC 1122
<https://datatracker.ietf.org/doc/html/rfc1122#page-85> is pretty clear
that the TCP sender MUST consider all IP and TCP options when sizing
payloads with respect to the advertised MSS option.

I'm reviewing a document
<https://datatracker.ietf.org/doc/html/draft-ietf-lwig-tcp-constrained-node-networks-11#section-4.1.1>
that advises that some endpoints may want to reduce their advertised MSS on
IPv6 connections in case the peer isn't respecting that guidance. Is
noncompliance with this provision a problem in the internet? Are there
middleboxes injecting options that cause PMTU drops or fragmentation?

I have not heard of such problems, but thought I'd check with the community
to see if this precaution makes any sense at all.

Thanks,
Martin

---

From: Joseph Touch <[email-masked]>
Date: Mon, 19 Oct 2020 12:37:08 -0700

See RFC 6691.

Only the fixed headers are considered, NOT the options.

Joe

> On Oct 19, 2020, at 12:22 PM, Martin Duke <[email-masked]> wrote:
> 
> Hello tcpm,
> 
> Section 4.2.2.6 of RFC 1122 <https://datatracker.ietf.org/doc/html/rfc1122#page-85> is pretty clear that the TCP sender MUST consider all IP and TCP options when sizing payloads with respect to the advertised MSS option.
> 
> I'm reviewing a document <https://datatracker.ietf.org/doc/html/draft-ietf-lwig-tcp-constrained-node-networks-11#section-4.1.1> that advises that some endpoints may want to reduce their advertised MSS on IPv6 connections in case the peer isn't respecting that guidance. Is noncompliance with this provision a problem in the internet? Are there middleboxes injecting options that cause PMTU drops or fragmentation?
> 
> I have not heard of such problems, but thought I'd check with the community to see if this precaution makes any sense at all.
> 
> Thanks,
> Martin
> 
> 
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: Joseph Touch <[email-masked]>
Date: Mon, 19 Oct 2020 12:44:29 -0700

To be more clear:

The MSS value in the TCP option reflects the FIXED headers only.

The sending side uses that value, together with its knowledge of options it adds, to reduce the size of the data sent to fit.

Second guessing that information is problematic; sure, when something gets through you can always try something smaller but that’s not specific to TCP or anything else.

Middleboxes should never make a packet larger under any circumstances.

Joe

> On Oct 19, 2020, at 12:37 PM, Joseph Touch <[email-masked]> wrote:
> 
> See RFC 6691.
> 
>> On Oct 19, 2020, at 12:22 PM, Martin Duke <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> Hello tcpm,
>> 
>> [email-masked] <mailto:[email-masked]>

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 19 Oct 2020 21:58:15 +0200

> On 19. Oct 2020, at 21:22, Martin Duke <[email-masked]> wrote:
> 
> Hello tcpm,
> 
> Section 4.2.2.6 of RFC 1122 is pretty clear that the TCP sender MUST consider all IP and TCP options when sizing payloads with respect to the advertised MSS option.
> 
> I'm reviewing a document that advises that some endpoints may want to reduce their advertised MSS on IPv6 connections in case the peer isn't respecting that guidance. Is noncompliance with this provision a problem in the internet? Are there middleboxes injecting options that cause PMTU drops or fragmentation?
Hi Martin,

the document you are reviewing says:

   An IPv6 datagram size exceeding 1280 bytes can be avoided by setting
   the TCP MSS not larger than 1220 bytes.  This assumes that the remote
   sender will use no TCP options, aside from possibly the MSS option,
   which is only used in the initial TCP SYN packet.

The first sentence is correct. The second is not. I would suggest to simply
remove it.

Then the text says:

   In order to accommodate unrequested TCP options that may be used by
   some TCP implementations, a constrained device may advertise an MSS
   smaller than 1220 bytes (e.g. not larger than 1200 bytes).  Note that
   it is advised for TCP implementations to consume payload space
   instead of increasing datagram size when including IP or TCP options
   in an IP packet to be sent [RFC6691].  Therefore, the suggestion of
   advertising an MSS smaller than 1220 bytes is likely to be
   overcautious and its suitability should be considered carefully.

I read this as "use a smaller MSS", but this is "likely to be
overcautious and its suitability should be considered carefully."

I think the careful consideration is to remove this paragraph.

Best regards
Michael

---

From: Martin Duke <[email-masked]>
Date: Mon, 19 Oct 2020 13:50:53 -0700

Hi Joe,

I may have phrased it ambiguously but I understand the intent of 1122/6691
as you describe it.

IOAM does add IPv6 options at times, although it should scrub them before
it leaves the domain.

Thanks
Martin

On Mon, Oct 19, 2020 at 12:44 PM Joseph Touch <[email-masked]> wrote:

> The sending side uses that value, together with its knowledge of options
> it adds, to reduce the size of the data sent to fit.
>
> Second guessing that information is problematic; sure, when something gets
> through you can always try something smaller but that’s not specific to TCP
> or anything else.
>
> I have not heard of such problems, but thought I'd check with the
> community to see if this precaution makes any sense at all.
>
> Thanks,
> Martin
>
>

---

From: Joseph Touch <[email-masked]>
Date: Mon, 19 Oct 2020 15:10:44 -0700

Hi, all,
I agree with Michael’s note below.

The key to understanding it is this:

	- the party that sends the MSS indicates the MSS with fixed IP and TCP headers
	- the party that receives the MSS and apples it would then subtract *the options it adds*

And no, nobody subtracts to account for Middleboxes adding options - because they shouldn’t be doing that and doing so would undermine nearly all of how path MTU and MSS works in the Internet.

Joe

> On Oct 19, 2020, at 12:58 PM, Michael Tuexen <[email-masked]> wrote:
> 
> 
>

---

From: Joseph Touch <[email-masked]>
Date: Mon, 19 Oct 2020 15:15:33 -0700

Hi, Martin,

> On Oct 19, 2020, at 1:50 PM, Martin Duke <[email-masked]> wrote:
> 
> Hi Joe,
> 
> I may have phrased it ambiguously but I understand the intent of 1122/6691 as you describe it.

RFC6691 was written to clear up the ambiguity and conflicting info in other docs.

> IOAM does add IPv6 options at times, although it should scrub them before it leaves the domain. 

That is NOT OK. 

I don’t know why its not very clear that this is incorrect, but just to be clear:

	No subdomain should increase the size of a packet UNLESS it can GUARANTEE that it supports *all types of* MTUs that are equivalently LARGER than the minimums required as well as those reported outside the domain.

I.e., let’s say the domain adds 20 bytes. Then it CANNOT use equipment that supports only 1280B path MTUs and 1500B reassembly MTUs; it needs to have equipment throughout that supports at least 1300B path MTUs and 1520B reassembly MTUs.

Unless the subdomain has those guarantees, it can’t increase packet sizes. Even then, it’s playing with space the endpoints “own” (e.g., of the limits of IPv4 or TCP option space), and NO subdomain mods can make the increased packet size work.

Overall, this is a VERY BAD IDEA and should NOT be endorsed by the IETF in any way, shape, or form - including experimentally.

Joe

> 
> Thanks
> Martin
> 
> On Mon, Oct 19, 2020 at 12:44 PM Joseph Touch <[email-masked] <mailto:[email-masked]>> wrote:
> To be more clear:
> 
>> On Oct 19, 2020, at 12:37 PM, Joseph Touch <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> See RFC 6691.
>> 
>>> https://www.ietf.org/mailman/listinfo/tcpm <https://www.ietf.org/mailman/listinfo/tcpm>
>> 
>

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 20 Oct 2020 10:10:45 +0100

On 19/10/2020 20:22, Martin Duke wrote:
> <https://datatracker.ietf.org/doc/html/rfc1122#page-85> is pretty 
> clear that the TCP sender MUST consider all IP and TCP options when 
> sizing payloads with respect to the advertised MSS option.
>
> I'm reviewing a document 
> that advises that some endpoints may want to reduce their advertised 
> MSS on IPv6 connections in case the peer isn't respecting that 
> guidance. Is noncompliance with this provision a problem in the 
> internet? Are there middleboxes injecting options that cause PMTU 
> drops or fragmentation?
>

Hi Martin,

While a server advertising a restricted MSS clearly reduces the TCP 
packet size, it seems to me to be a rather poor solution to the problem 
in general, and only works for TCP.

If  it wants to say some advices, I think the IETF needs to consider 
current practice and the implications of this. I suspect the practice of 
clamping the IPv6 MSS at the server is already quite common, especially 
in IPv6 servers offering web content. MSS Clamping on-path is also 
common, but anyway I see some servers advertise an MSS much lower than 
permitted by the MTU. For example, see the section on MSS handling here:

https://tma.ifip.org/2018/wp-content/uploads/sites/3/2018/06/tma2018_paper57.pdf

This might be because of the reasons for this are cited in RFC8900.

Gorry

---

From: Mirja Kuehlewind <[email-masked]>
Date: Tue, 20 Oct 2020 12:56:30 +0000

Hi Martin, hi Gorry,

I think Gorry’s paper does indicate that at least for MMS there are middleboxes that add this option. However, I don’t think there is evidence that there is a general problem about middlebox adding option (rather than removing them). I also reviewed the draft in tsv-art review and given that they say it’s actually not recommended and should only be done with care, I thought that text is okay. However, I guess there is room to improve the wording and explain a bit better, when and if this might be consider or not.

Mirja

From: tcpm <[email-masked]> on behalf of Gorry Fairhurst <[email-masked]>
Date: Tuesday, 20. October 2020 at 11:13
To: Martin Duke <[email-masked]>, "[email-masked] Extensions" <[email-masked]>
Subject: Re: [tcpm] Exceeding value in MSS option?

Section 4.2.2.6 of RFC 1122<https://datatracker.ietf.org/doc/html/rfc1122#page-85> is pretty clear that the TCP sender MUST consider all IP and TCP options when sizing payloads with respect to the advertised MSS option.

I'm reviewing a document<https://datatracker.ietf.org/doc/html/draft-ietf-lwig-tcp-constrained-node-networks-11#section-4.1.1> that advises that some endpoints may want to reduce their advertised MSS on IPv6 connections in case the peer isn't respecting that guidance. Is noncompliance with this provision a problem in the internet? Are there middleboxes injecting options that cause PMTU drops or fragmentation?

Thanks,
Martin

tcpm mailing list

[email-masked]<mailto:[email-masked]>

Hi Martin,

While a server advertising a restricted MSS clearly reduces the TCP packet size, it seems to me to be a rather poor solution to the problem in general, and only works for TCP.

If  it wants to say some advices, I think the IETF needs to consider current practice and the implications of this. I suspect the practice of clamping the IPv6 MSS at the server is already quite common, especially in IPv6 servers offering web content. MSS Clamping on-path is also common, but anyway I see some servers advertise an MSS much lower than permitted by the MTU. For example, see the section on MSS handling here:

https://tma.ifip.org/2018/wp-content/uploads/sites/3/2018/06/tma2018_paper57.pdf<https://protect2.fireeye.com/v1/url?k=c964a91f-97c41371-c964e984-866038973a15-4c538007322d8392&q=1&e=8b72cbc2-683a-493b-a075-c9df067b88f7&u=https%3A%2F%2Ftma.ifip.org%2F2018%2Fwp-content%2Fuploads%2Fsites%2F3%2F2018%2F06%2Ftma2018_paper57.pdf>

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 20 Oct 2020 16:02:39 +0100

See below...

> On 20 Oct 2020, at 13:56, Mirja Kuehlewind <[email-masked]> wrote:
> 
> 
> Hi Martin, hi Gorry,
>  
> I think Gorry’s paper does indicate that at least for MSS, there are middleboxes that add this option. However, I don’t think there is evidence that there is a general problem about middlebox adding option (rather than removing them). I also reviewed the draft in tsv-art review and given that they say it’s actually not recommended and should only be done with care, I thought that text is okay. However, I guess there is room to improve the wording and explain a bit better, when and if this might be consider or not.
>  
> Mirja
>  
Right. This approach is not great for evolution of the size in future, but this use of MSS to restrict packet size has not broken the Internet - far from it, it is probably the way connections currently avoid black holes. I suspect that any operator adding overhead will be aware of what they are doing, and that If they do MSS clamping they will adjust that if needed, or just configure a bigger MTU within their network segment.

Gorry

---

From: "Scharf, Michael" <[email-masked]>
Date: Tue, 20 Oct 2020 19:56:29 +0000

Hi all,

Sorry for the delayed reply. More inline...

> > Section 4.2.2.6 of RFC 1122 is pretty clear that the TCP sender MUST
> consider all IP and TCP options when sizing payloads with respect to the
> advertised MSS option.
> >
> > I'm reviewing a document that advises that some endpoints may want to
> reduce their advertised MSS on IPv6 connections in case the peer isn't
> respecting that guidance. Is noncompliance with this provision a problem in
> the internet? Are there middleboxes injecting options that cause PMTU
> drops or fragmentation?
> Hi Martin,
> 

The statements regarding the MSS were discussed and also rewritten many times. And, sorry, apparently I have not carefully read the second sentence (but, well, that sentence was also in the version of the draft that was last called in TCPM...)

Given this discussion now, I agree that the second sentence could probably be removed.
 
> I think the careful consideration is to remove this paragraph.

I am not sure if we should simply remove the whole paragraph, including e.g. the reference to RFC 6691. Readers of this document may not be aware of RFC 6691, so a pointer may be useful, no? Also, during the IETF last call there was a discussion on MSS values significantly smaller than 1220 byte. People apparently *do* think about smaller MSS values on constrained devices and this document is a place to provide guidance to that community (as far as possible). This specific wording came up in earlier reviews of the draft, too. Thus, the final proposal "likely to be overcautious and its suitability should be considered carefully" is a result of several past discussions. IMHO we should carefully consider changes to this statement...

BTW, personally, I really wonder if using an MSS of 1220 byte (for IPv6) indeed causes real-world problems on constrained devices, i.e., if in year 2020 there is any real-world benefit of an MSS smaller than 1220 byte. But, unfortunately, I don't have measurement data that would back a different statement in the draft.

Michael

---

From: "Scharf, Michael" <[email-masked]>
Date: Tue, 20 Oct 2020 19:58:05 +0000

Out of my head, most current router operating systems support MSS clamping, and albeit I have no measurement data, I’d be surprised if that router feature is not widely used.

If the network clamps the MSS, it may not really matter if a connection endpoint picks an MSS larger than the value enforced by the network. But I am not sure if draft-ietf-lwig-tcp-constrained-node-networks-11 should discuss MSS clamping as technique. It is not at all specific to constrained devices.

Michael

From: tcpm <[email-masked]> On Behalf Of Gorry Fairhurst
Sent: Tuesday, October 20, 2020 5:03 PM
To: Mirja Kuehlewind <[email-masked]>
Cc: [email-masked] Extensions <[email-masked]>
Subject: Re: [tcpm] Exceeding value in MSS option?

See below...

On 20 Oct 2020, at 13:56, Mirja Kuehlewind <[email-masked]<mailto:[email-masked]>> wrote:

Hi Martin, hi Gorry,

Mirja

Gorry

From: tcpm <[email-masked]<mailto:[email-masked]>> on behalf of Gorry Fairhurst <[email-masked]<mailto:[email-masked]>>
Date: Tuesday, 20. October 2020 at 11:13
To: Martin Duke <[email-masked]<mailto:[email-masked]>>, "[email-masked] Extensions<mailto:[email-masked]%20Extensions>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [tcpm] Exceeding value in MSS option?

Thanks,
Martin

tcpm mailing list

Hi Martin,

Gorry

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 20 Oct 2020 23:04:16 +0200

> On 20. Oct 2020, at 21:56, Scharf, Michael <[email-masked]> wrote:
> 
> Hi all,
> 
> I am not sure if we should simply remove the whole paragraph, including e.g. the reference to RFC 6691. Readers
The paragraph starts with:

In order to accommodate unrequested TCP options that may be used by
some TCP implementations, a constrained device may advertise an MSS
smaller than 1220 bytes (e.g. not larger than 1200 bytes).

I don't think the selection of the MSS should depend on TCP options.
A TCP stack may send an MSS option with a value lower then 1220, but
it should not do it due to any TCP options. That is the reason, why
I suggested to remove the paragraph.

Best regards
Michael

>  of this document may not be aware of RFC 6691, so a pointer may be useful, no? Also, during the IETF last call there was a discussion on MSS values significantly smaller than 1220 byte. People apparently *do* think about smaller MSS values on constrained devices and this document is a place to provide guidance to that community (as far as possible). This specific wording came up in earlier reviews of the draft, too. Thus, the final proposal "likely to be overcautious and its suitability should be considered carefully" is a result of several past discussions. IMHO we should carefully consider changes to this statement...
>

---

From: Joseph Touch <[email-masked]>
Date: Tue, 20 Oct 2020 14:18:22 -0700

> On Oct 20, 2020, at 2:04 PM, Michael Tuexen <[email-masked]> wrote:
> 
>> ...

What does “unrequested” mean? It legitimately means options users haven’t asked for but are enabled somehow (default, OS override, etc.).

But that’s already taken care of - the side sending MMS indicates the value without considering options *because* it is the other side that uses the MSS *and its knowledge of which options it is adding, both at IP and TCP* to determine how much data payload room is available.

(And no, I still don’t think this should include a guess as to “how much a middlebox will expand the packet”).

> I don't think the selection of the MSS should depend on TCP options.

RFC6691 is very clear - it definitely should not.

> it should not do it due to any TCP options.

Agreed.

Joe

---

From: "Scharf, Michael" <[email-masked]>
Date: Wed, 21 Oct 2020 07:31:50 +0000

> > The statements regarding the MSS were discussed and also rewritten many
> times. And, sorry, apparently I have not carefully read the second sentence
> (but, well, that sentence was also in the version of the draft that was last
> called in TCPM...)
> >

OK, thanks, now I actually get the point. Sorry, it was late yesterday. I agree. This is all what RFC 6691 is about.

But I still wonder whether we should keep a reference to RFC 6691, given that this topic has repeatedly caused confusion.

>  That is the reason, why

What about the following shorter paragraph with three sentences:

   An IPv6 datagram size exceeding 1280 bytes can be avoided by setting
   the TCP MSS not larger than 1220 bytes. Note that
   it is advised for TCP implementations to consume payload space
   instead of increasing datagram size when including IP or TCP options
   in an IP packet to be sent [RFC6691]. Therefore, it is not required to
   advertise an MSS smaller than 1220 bytes in order to accommodate
   TCP options. 

Would that be reasonable?

Michael

> 
> Best regards
> Michael
> 
> >  of this document may not be aware of RFC 6691, so a pointer may be
> useful, no? Also, during the IETF last call there was a discussion on MSS
> values significantly smaller than 1220 byte. People apparently *do* think
> about smaller MSS values on constrained devices and this document is a
> place to provide guidance to that community (as far as possible). This specific
> wording came up in earlier reviews of the draft, too. Thus, the final proposal
> "likely to be overcautious and its suitability should be considered carefully" is
> a result of several past discussions. IMHO we should carefully consider
> changes to this statement...
> >
> > BTW, personally, I really wonder if using an MSS of 1220 byte (for IPv6)
> indeed causes real-world problems on constrained devices, i.e., if in year
> 2020 there is any real-world benefit of an MSS smaller than 1220 byte. But,
> unfortunately, I don't have measurement data that would back a different
> statement in the draft.
> >
> > Michael
> >
> >> Best regards
> >> Michael
> >>
> >>
> >>
> >>
> >>>

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 21 Oct 2020 11:20:19 +0200

> On 21. Oct 2020, at 09:31, Scharf, Michael <[email-masked]> wrote:
> 
No problem.
> But I still wonder whether we should keep a reference to RFC 6691, given that this topic has repeatedly caused confusion.
That makes sense.
> Would that be reasonable?
Sounds reasonable. Much better than the original text.

Best regards
Michael

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 21 Oct 2020 10:28:41 +0100

> Sounds reasonable. Much better than the original text.

+1

Gorry

-- 
G. Fairhurst, School of Engineering

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 21 Oct 2020 10:28:41 +0100

+1

Gorry

---

From: Markku Kojo <[email-masked]>
Date: Wed, 21 Oct 2020 13:04:09 +0300 (EEST)

Hi,

see inline.

> Would that be reasonable?

That's much better but I would say it even more strongly because this is 
MUST in RFC 1122 as pointed out earlier:

" ...
   Note that
   it is advised for TCP implementations to consume payload space
   instead of increasing datagram size when including IP or TCP options
   in an IP packet to be sent [RFC6691].
   ..."

->

" ...
   Note that
   TCP implementations MUST consume payload space
   instead of increasing datagram size when including IP or TCP options
   in an IP packet to be sent [RFC1122, RFC6691].
   ..."

OR (if RFC 2119 language is not used in this doc, and currently it is not 
used although section 2 exists):

" ...
   Note that
   it is required for TCP implementations to consume payload space
   instead of increasing datagram size when including IP or TCP options
   in an IP packet to be sent [RFC1122, RFC6691].
   ..."

BR,

/Markku

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 21 Oct 2020 15:53:48 +0200

> On 21. Oct 2020, at 12:04, Markku Kojo <[email-masked]> wrote:
> 
> Hi,
> 
> see inline.
> 
> That's much better but I would say it even more strongly because this is MUST in RFC 1122 as pointed out earlier:
> 
> " ...
>  Note that
> OR (if RFC 2119 language is not used in this doc, and currently it is not used although section 2 exists):
> 
> " ...
>  Note that
I like these variants.

Best regards
Michael

---

From: Joseph Touch <[email-masked]>
Date: Wed, 21 Oct 2020 09:40:16 -0700

FWIW - when citing requirements established elsewhere, IMO it’s always better to NOT repeat it as a requirement elsewhere. It makes updates very difficult.

I.e., avoid 2119 language when referring to an existing requirement and clearly indicate that this is a summary (“already a requirement”), e.g.:

---
Note that it is already a requirement that TCP implementations consume payload space
instead of increasing datagram size when including IP or TCP options
in an IP packet to be sent [RFC6691].
--

FWIW - I would avoid citing 1122 because the entire issue is addressed definitely in 6691.

Joe

---

From: Martin Duke <[email-masked]>
Date: Wed, 21 Oct 2020 09:41:58 -0700

OTOH 1122 is a standard and 6691 is Informational.

On Wed, Oct 21, 2020 at 9:40 AM Joseph Touch <[email-masked]> wrote:

> FWIW - when citing requirements established elsewhere, IMO it’s always
> better to NOT repeat it as a requirement elsewhere. It makes updates very
> difficult.
>
> I.e., avoid 2119 language when referring to an existing requirement and
> clearly indicate that this is a summary (“already a requirement”), e.g.:
>
> ---

---

From: Joe Touch <[email-masked]>
Date: Wed, 21 Oct 2020 10:15:43 -0700

IMO let 6691 speak to that. Ie, as explained in 6691

> On Oct 21, 2020, at 9:42 AM, Martin Duke <[email-masked]> wrote:
> 
>

---

From: Frode Kileng <[email-masked]>
Date: Fri, 23 Oct 2020 10:37:44 +0200

Hi,

please see below

On 20 Oct 2020, at 13:56, Mirja Kuehlewind 
<[email-masked]> wrote:
> Hi Martin, hi Gorry,
>
> I think Gorry’s paper does indicate that at least for MSS, there are 
> middleboxes that add this option. However, I don’t think there is 
> evidence that there is a general problem about middlebox adding option 
> (rather than removing them). I also reviewed the draft in tsv-art 
> review and given that they say it’s actually not recommended and 
> should only be done with care, I thought that text is okay. However, I 
> guess there is room to improve the wording and explain a bit better, 
> when and if this might be consider or not.
>
Although maybe not  a "general problem",  a study of MSS middlebox 
manipluation in  23 European and Asian mobile networks did reveal some 
issues:

- 20 of 23 networks always inserts MSS. Value for IPv4 varies from 1360 
to 1420 and the value typically seems to reflect  vendor defaults

- The 3 network that didn't clamp MSS do reduce an end-point specified 
values  if it's higher than some network specific threshold

- 2 Networks overrode a end-point specified lower MSS value and changed 
it to the higher network specific default! I.e. a potential source of 
problems.

- 1 Network did not clamp if some other TCP options was used (MPTCP, 
TFO, etc)! Also a source of problems.

I'm afraid that the results from this study has not been published.

Best regards

Frode Kileng

---

From: Joseph Touch <[email-masked]>
Date: Fri, 23 Oct 2020 08:11:28 -0700

> On Oct 23, 2020, at 1:37 AM, Frode Kileng <[email-masked]> wrote:
> 
> Although maybe not  a "general problem",  a study of MSS middlebox manipluation in  23 European and Asian mobile networks did reveal some issues:
> 
> - 20 of 23 networks always inserts MSS.

That IS a general problem, FWIW. 

Besides being published, it should also be reported to the operators of those networks as violating TCP specs.

Joe

---

From: Frode Kileng <[email-masked]>
Date: Mon, 26 Oct 2020 08:52:48 +0100

On 23/10/2020 17:11, Joseph Touch wrote:
>> On Oct 23, 2020, at 1:37 AM, Frode Kileng <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> manipluation in  23 European and Asian mobile networks did reveal 
>> some issues:
>>
> Besides being published, it should also be reported to the operators 
> of those networks as violating TCP specs.

Do you mean as in "layering violation" or is it explicitly stated in an 
RFC that MSS clamping is not allowed.

Anyway, since stopping this practice is likely to cause extensive 
fragmentation of TCP segments, mobile networks will continue this 
despite "violating TCP specs". At least until the networks fully support 
jumbo frames.

(The motivation for MSS clamping is the tunneling overhead in the mobile 
core of the networks. Within the core, the MSS is clamped to take into 
consideration the size of the  GTP-U header and potential additional 
ipsec headers. The GTP-U header is variable size and is at least 40 
bytes for IPv4 and 60 for IPv6. In addition it may include vendor or 
mobile operator specific extension headers. I do not now why networks 
don't reduce the MTU size instead. My understanding of the 3GPP specs is 
that this is suggested but could not find out how this is provisioned to 
the terminals for IPv4. I've only seen one mobile network that 
provisioned the terminals with a MTU lower than 1500 bytes. But their 
MTU size of 1464 for for IPv4 and IPv6 did not seem reflect the GTP-U 
tunnel.)

Regards

Frode Kileng

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 26 Oct 2020 10:34:41 +0100

> On 26. Oct 2020, at 08:52, Frode Kileng <[email-masked]> wrote:
> 
> 
> 
>>> On Oct 23, 2020, at 1:37 AM, Frode Kileng <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
> Do you mean as in "layering violation" or is it explicitly stated in an RFC that MSS clamping is not allowed.
I think we should differentiate two issues:
(a) a middlebox which reduces the MSS already in an SYN/SYN-ACK segement.
(b) a middlebox which adds an MSS option, which wasn't there in the SYN/SYN-ACK segment.

(a) is the typical MSS clamping you are referring to and it is used to mitigate MTU issues.
(b) is a problem in my view and referred to in the statement 'always inserts'.

Best regards
Michael
> 
> Anyway, since stopping this practice is likely to cause extensive fragmentation of TCP segments, mobile networks will continue this despite "violating TCP specs". At least until the networks fully support jumbo frames.
> 
> (The motivation for MSS clamping is the tunneling overhead in the mobile core of the networks. Within the core, the MSS is clamped to take into consideration the size of the  GTP-U header and potential additional ipsec headers. The GTP-U header is variable size and is at least 40 bytes for IPv4 and 60 for IPv6. In addition it may include vendor or mobile operator specific extension headers. I do not now why networks don't reduce the MTU size instead. My understanding of the 3GPP specs is that this is suggested but could not find out how this is provisioned to the terminals for IPv4. I've only seen one mobile network that provisioned the terminals with a MTU lower than 1500 bytes. But their MTU size of 1464 for for IPv4 and IPv6 did not seem reflect the GTP-U tunnel.)
> 
> 
> Regards
> 
> Frode Kileng
> 
> 
> 
> 
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 26 Oct 2020 07:24:32 -0700

> On Oct 26, 2020, at 2:35 AM, Michael Tuexen <[email-masked]> wrote:
> 
> 
>> 

You can if you want; I do not. Both are violations of TCP processing, whose headers aren’t permitted to be altered in flight. Saying it’s for the good of the endpoints doesn’t change the issue that it should be reported and fixed as an error.

If TCP is given an MSS by the other end of a connection and packets don’t get through, it can always lower the segment sizes it sends - the is what PLPMTUD does. 

Joe

---

From: Frode Kileng <[email-masked]>
Date: Mon, 26 Oct 2020 16:45:27 +0100

> If TCP is given an MSS by the other end of a connection and packets don’t get through, it can always lower the segment sizes it sends - the is what PLPMTUD does.

I think there may be other more important fights out there than 
convincing ~90% of the world mobile network operators and their vendors 
that a practice solving potential black-holing and typically improves 
performance should stop because it's a layering violation. For example 
that 2/3rd of the mobile networks I tested have a TCP Proxy and over 70% 
of such networks breaks MPTCP, TFO or ECN.

Regards

Frode Kileng

---

From: Joe Touch <[email-masked]>
Date: Mon, 26 Oct 2020 09:55:32 -0700

Given how many of us are using VPNs, it’s a shame they don’t work without this.  I guess users of those nets have to shut down their Skypes and other services over VPNs, because, according to you, it doesn’t work. 

Joe

> On Oct 26, 2020, at 8:46 AM, Frode Kileng <[email-masked]> wrote:
> 
> I think there may be other more important fights out there than convincing ~90% of the world mobile network operators and their vendors that a practice solving potential black-holing and typically improves performance should stop because it's a layering violation. For example that 2/3rd of the mobile networks I tested have a TCP Proxy and over 70% of such networks breaks MPTCP, TFO or ECN.
> 
> 
> Regards
> 
> Frode Kileng
> 
>

---

From: Frode Kileng <[email-masked]>
Date: Tue, 27 Oct 2020 07:35:11 +0100

I do not defending this practice, just trying to communicate their 
rationale and that there may be other more important fights out there. 
But I do understand that operators are cautious as I've seen an example 
of how fragmentation triggered an anti-DDoS feature in a firewall 
causing traffic from a large number of mobiles to halt.

Frode

---

From: Joseph Touch <[email-masked]>
Date: Tue, 27 Oct 2020 08:25:19 -0700

So the real question is why tunneled packets didn’t trigger the same issue - isn’t it?

Or are you implying that tunneled traffic shouldn’t be supported?

There are deeper issues going on here. We should avoid being glib about how “best” to do things that shouldn’t be done in the first place.

Joe

> On Oct 26, 2020, at 11:35 PM, Frode Kileng <[email-masked]> wrote:
> 
> I do not defending this practice, just trying to communicate their rationale and that there may be other more important fights out there. But I do understand that operators are cautious as I've seen an example of how fragmentation triggered an anti-DDoS feature in a firewall causing traffic from a large number of mobiles to halt.
> 
> Frode
>

---

From: Steven Sommars <[email-masked]>
Date: Tue, 27 Oct 2020 16:51:31 -0500

Much of today's mobile traffic IP traffic is GTP (GPRS Tunneling protocol)
tunneled within the core network (per the 3GPP specifications).   There can
be one or two back-to-back  bidirectional  tunnels between the base station
and the PGW/GGSN (gateways to the Internet).   The 3GIP IP MTU requirements
weren't very clear at the time, but we interpreted the requirement to be a
1500-byte subscriber IP MTU.     For those portions of the core network
that used IP/ATM (AAL5), transport of a ~1540 byte tunneled packet (IP
MTU + about 40 bytes for GTP/IPv4 header) was  usually provided.   [I'm
using 40 bytes for simplicity, could be smaller or larger.]  Ethernet was
cheaper and more readily available, so transport of 1540 byte packets was
needed.  [I'll skip other headers that are often used: VLAN, MPLS, etc.]
   The cellular carriers often leased IP/Ethernet transport and at times
only ~1500 byte MTUs were available.

If the IP/Ethernet transport network supports sufficiently large IP MTUs,
life is simpler.   If not, how does one stuff 1540 bytes into a 1500 byte
packet?
- Avoid core network IP fragmentation.    I'll list some of the options &
their drawbacks.
        -   Resegment TCP (e.g., proxy).    ===  many drawbacks
        -   Fragment subscriber packet  (ignore DF bit)    ===   many
drawbacks
        -   Drop packets that would require fragmentation (PMTUD).  ===
PMTU black hole was an issue
        -   Signal to the subscriber device that the IP MTU should be
reduced.   ===   This was proposed in 3GPP starting in the late 1990's, but
I don't know what happened.
        -   Insert/Alter TCP MSS option during SYN/ACK.   ===   Not
strictly legal, but it typically worked well.
- Implement core network IP fragmentation/reassembly.   Done correctly this
is undetectable by the subscriber- or server-side.   However,
        -   Transport network bandwidth requirements increased slightly.
        -    IP reassembly can be resource intensive.    [I've seen
performance drop by over 100x when IP reassembly was needed on some older
hardware.]
        -    IP reassembly may fail.
        -    Fragmentation and reassembly may trigger missequencing.

Is TCPM the right forum for discussing these end-to-end principle related
items?   RFC3234 has a somewhat dated (2002) middlebox taxonomy.

On Tue, Oct 27, 2020 at 10:25 AM Joseph Touch <[email-masked]> wrote:

> There are deeper issues going on here. We should avoid being glib about
> how “best” to do things that shouldn’t be done in the first place.
>
> Joe
>
> rationale and that there may be other more important fights out there. But
> I do understand that operators are cautious as I've seen an example of how
> fragmentation triggered an anti-DDoS feature in a firewall causing traffic
> from a large number of mobiles to halt.
> >
> > Frode
> >
> >> Given how many of us are using VPNs, it’s a shame they don’t work
> without this.  I guess users of those nets have to shut down their Skypes
> and other services over VPNs, because, according to you, it doesn’t work.
> >>
> >> Joe
> >>
> >>>>>>>>> Although maybe not  a "general problem",  a study of MSS
> middlebox manipluation in  23 European and Asian mobile networks did reveal
> some issues:
> >>>>>>>>>
> >>>>>>> Besides being published, it should also be reported to the
> operators of those networks as violating TCP specs.
> >>>>>> Do you mean as in "layering violation" or is it explicitly stated
> in an RFC that MSS clamping is not allowed.
> >>>> You can if you want; I do not. Both are violations of TCP processing,
> whose headers aren’t permitted to be altered in flight. Saying it’s for the
> good of the endpoints doesn’t change the issue that it should be reported
> and fixed as an error.
> >>>>
> >>>> If TCP is given an MSS by the other end of a connection and packets
> don’t get through, it can always lower the segment sizes it sends - the is
> what PLPMTUD does.
> performance should stop because it's a layering violation. For example that
> 2/3rd of the mobile networks I tested have a TCP Proxy and over 70% of such
> networks breaks MPTCP, TFO or ECN.
> >>>
> >>>
> >>> Regards
> >>>
> >>> Frode Kileng
> >>>
> >>>
> >
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 27 Oct 2020 15:28:11 -0700

These issues have been addressed at length in intarea and tsvwg. It wouldn’t hurt if the end systems used PLPMTUD, but fragmentation support in GRE and other tunnels is also supported. 

Ultimately, if you tunnel and cannot guarantee end system cooperation, frag/reassembly is on you. 

Joe

> On Oct 27, 2020, at 2:52 PM, Steven Sommars <[email-masked]> wrote:
> 
> 
> Much of today's mobile traffic IP traffic is GTP (GPRS Tunneling protocol) tunneled within the core network (per the 3GPP specifications).   There can be one or two back-to-back  bidirectional  tunnels between the base station and the PGW/GGSN (gateways to the Internet).   The 3GIP IP MTU requirements weren't very clear at the time, but we interpreted the requirement to be a 1500-byte subscriber IP MTU.     For those portions of the core network that used IP/ATM (AAL5), transport of a ~1540 byte tunneled packet (IP MTU + about 40 bytes for GTP/IPv4 header) was  usually provided.   [I'm using 40 bytes for simplicity, could be smaller or larger.]  Ethernet was cheaper and more readily available, so transport of 1540 byte packets was needed.  [I'll skip other headers that are often used: VLAN, MPLS, etc.]       The cellular carriers often leased IP/Ethernet transport and at times only ~1500 byte MTUs were available.  
> 
> If the IP/Ethernet transport network supports sufficiently large IP MTUs, life is simpler.   If not, how does one stuff 1540 bytes into a 1500 byte packet?   
> - Avoid core network IP fragmentation.    I'll list some of the options & their drawbacks.
>         -   Fragment subscriber packet  (ignore DF bit)    ===   many drawbacks
>         -   Drop packets that would require fragmentation (PMTUD).  ===  PMTU black hole was an issue
>         -   Signal to the subscriber device that the IP MTU should be reduced.   ===   This was proposed in 3GPP starting in the late 1990's, but I don't know what happened.
>         -   Insert/Alter TCP MSS option during SYN/ACK.   ===   Not strictly legal, but it typically worked well.
> - Implement core network IP fragmentation/reassembly.   Done correctly this is undetectable by the subscriber- or server-side.   However,
>         -    IP reassembly can be resource intensive.    [I've seen performance drop by over 100x when IP reassembly was needed on some older hardware.]
> Is TCPM the right forum for discussing these end-to-end principle related items?   RFC3234 has a somewhat dated (2002) middlebox taxonomy.
> 
> 
> 
> 
>