Subject: Re: [Int-area] New Version Notification for draft-bonica-intarea-lossless-pmtud-00.txt

From: Manoj Nayak <[email-masked]>
Date: Thu, 14 Nov 2019 04:34:54 +0000

Hello Joe,

Please find my reply.

>>- why does this doc assume the max ICMP is 576?
>>       we?re still talking IPv4 here; it?s still 68 (that?s why only 64 bits of the orig payload are guaranteed)
>>       (yes, your note in the end of sec 1 is relevant, but given v4-in-v4 tunneling, it?s possible that 
>> paths might be smaller than the 576 assumption)
 
We use an unused field in first 8 bytes of ICMP error/reply message. How the idea would be
affected if minimum packet size is 68 bytes or 576 bytes. As per my understanding,
existing ICMP error/reply message works in v4-in-v4 tunnelling, so it would continue to 
work with the idea proposed in our draft. we won’t let the ICMP message exceed a reasonable size. 
in our implementation, that will be 576.

>>- why would this approach find the largest fragment through a system?
>>       rfc1812 talks about various strategies, one of which is ?equal sized?, which might never find 
>> the max the way you propose

As per section 4.2.2.7 from rfc 1812,
 
“There are several fragmentation techniques in common use in the
Internet.  One involves splitting the IP datagram into IP
fragments with the first being MTU sized, and the others being
approximately the same size, smaller than the MTU. “
 
In both of the above cases, idea in our draft works. In our implementation,
We can assume the first fragment to be largest fragment. This first fragment remains
Largest fragment unless until one more fragment is found to be greater than the first fragment.
 
For example:
 
While assembling the fragments, 
 
   I=0;
   Largest Fragment = packet-I;
   For (, I < n , ++I)
      If ( packet-I > Largest Fragment)
        Largest Fragment = packet-I;
 
Hopefully I did not miss anything.

Regards
Manoj Nayak
    
    ------------------------------
    
    Message: 3
    Date: Tue, 29 Oct 2019 21:43:33 -0700
    From: Joe Touch <[email-masked]>
    To: Ron Bonica <rbonica=[email-masked]>
    Cc: "[email-masked]" <[email-masked]>
    Subject: Re: [Int-area] New Version Notification for
    	draft-bonica-intarea-lossless-pmtud-00.txt
    Message-ID: <[email-masked]>
    Content-Type: text/plain;	charset=utf-8
    
    Hi, Ron,
    
    A few things come to mind. The first one, IMO, renders the rest somewhat less important.
    
    Joe
    
    -------------
    
    - this approach applies only to IPv4; not sure it?s worth trying to optimize for only that case
    	(it requires on-path fragmentation permitted)
    
    - this approach relies on ICMPs, so it?s as robust (or, more to the point, not) as PMTUD
    	if ICMPs can find the reverse path from the dest, why wouldn?t the routers?
    	i.e., isn?t the problem with ICMPs not just routers not sending them but firewalls BLOCKING them?
    	(i.e., if ICMPs would work here, PMTU would have worked, rendering this unnecessary)
    
    - why does this doc assume the max ICMP is 576?
    	we?re still talking IPv4 here; it?s still 68 (that?s why only 64 bits of the orig payload are guaranteed)
    	(yes, your note in the end of sec 1 is relevant, but given v4-in-v4 tunneling, it?s possible that paths might be smaller than the 576 assumption)
    
    - why would this approach find the largest fragment through a system?
    	rfc1812 talks about various strategies, one of which is ?equal sized?, which might never find the max the way you propose 

    > On Oct 29, 2019, at 9:53 AM, Ron Bonica <rbonica=[email-masked]> wrote:
    > 
    > Folks,
    > 
    > Please review and comment.
    > 
    >                        Ron
    > 
    > 
    > 
    > Juniper Business Use Only
    > 
    > -----Original Message-----
    > From: [email-masked] <[email-masked]> 
    > Sent: Tuesday, October 29, 2019 11:48 AM
    > To: Ron Bonica <[email-masked]>; Hakan Alpan <[email-masked]>; Radon Rosborough <[email-masked]>; Bradely Newton <[email-masked]>; Miles President <[email-masked]>; Manoj Nayak <[email-masked]>
    > Subject: New Version Notification for draft-bonica-intarea-lossless-pmtud-00.txt
    > 
    > 
    > A new version of I-D, draft-bonica-intarea-lossless-pmtud-00.txt
    > has been successfully submitted by Ron Bonica and posted to the IETF repository.
    > 
    > Name:		draft-bonica-intarea-lossless-pmtud
    > Revision:	00
    > Title:		Lossless Path MTU Discovery (PMTUD)
    > Document date:	2019-10-29
    > Group:		Individual Submission
    > Pages:		8
    > URL:            https://urldefense.com/v3/__https://tools.ietf.org/html/draft-bonica-intarea-lossless-pmtud-00__;!8WoA6RjC81c!XifH7EcqHRKXHyGSwB3ojXm6YmKn_vYWCjgM-VDTPTEzP-khJGlb9MqM8x-YTELKJ64$ 
    > 
    > 
    > 
    > Abstract:
    >   This document describes alternative IPv4 PMTUD procedures that do not
    >   prevent IP fragmentation and do no rely on the network's ability to
    >   deliver ICMP Destination Unreachable messages to the source node.
    >   This document also defines a new ICMP message.  IPv4 nodes emit this
    >   new message when they reassemble a fragmented packet.
    > 
    > 
    > 
    > 
    > Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org.
    > 
    > The IETF Secretariat
    > _______________________________________________
    > Int-area mailing list
    > [email-masked]
    > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/int-area__;!8WoA6RjC81c!XifH7EcqHRKXHyGSwB3ojXm6YmKn_vYWCjgM-VDTPTEzP-khJGlb9MqM8x-YOVqXLNE$

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Thu, 14 Nov 2019 14:45:09 +0000

Manoj,

> In both of the above cases, idea in our draft works.

No it doesn't work. The text you quoted says that some techniques can
cause all fragments to be "smaller than the MTU". It doesn't say how much
smaller, so we must assume that it could be significantly smaller such that
the maximum fragment size bears no relation to the path MTU. Therefore,
in the general sense, it just doesn't work.

We have been through this before. It is in my expired drafts.

Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Manoj Nayak
> Sent: Wednesday, November 13, 2019 8:35 PM
> To: [email-masked]
> Cc: [email-masked]
> Subject: Re: [Int-area] New Version Notification for draft-bonica-intarea-lossless-pmtud-00.txt
> 
> Hello Joe,
> 
> Please find my reply.
> 
>     > To: Ron Bonica <[email-masked]>; Hakan Alpan <[email-masked]>; Radon Rosborough <[email-masked]>; Bradely
> Newton <[email-masked]>; Miles President <[email-masked]>; Manoj Nayak <[email-masked]>
>     > URL:            https://urldefense.com/v3/__https://tools.ietf.org/html/draft-bonica-intarea-lossless-pmtud-
> 00__;!8WoA6RjC81c!XifH7EcqHRKXHyGSwB3ojXm6YmKn_vYWCjgM-VDTPTEzP-khJGlb9MqM8x-YTELKJ64$
>     >
>     >
>     >
>     > Abstract:
>     > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/int-
> area__;!8WoA6RjC81c!XifH7EcqHRKXHyGSwB3ojXm6YmKn_vYWCjgM-VDTPTEzP-khJGlb9MqM8x-YOVqXLNE$
> 
> 
> 
> 
> https://www.ietf.org/mailman/listinfo/int-area

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Nov 2019 06:50:10 -0800

> On Nov 13, 2019, at 8:34 PM, Manoj Nayak <[email-masked]> wrote:
> 
> Hello Joe,
> 
> Please find my reply.
> 
> We use an unused field in first 8 bytes of ICMP error/reply message.

Please explain. Most ICMP messages have 4 bytes of unused field, but not all (one has only 3).

> How the idea would be

Sending the largest response possible given an untunneled MTU size is an invitation to black-hole the response itself if (when) an IPv4-in-IPv4 tunnel is encountered.

In most situations, ICMP responses are received from small initial messages that don’t stress that limit. The use in this doc is the opposite - it relies on ongoing use of ICMP for max-sized packets and returns max-sized payloads. This isn’t helpful. It would be more useful to try to limit the size to the minimum expected to be useful and account for these other encapsulations.

> In both of the above cases, idea in our draft works.

The issue is further down in that section:

      One other fragmentation technique discussed was splitting the IP
      datagram into approximately equal sized IP fragments, with the
      size less than or equal to the next hop network's MTU. ...

In that case, none of the fragments gives you the path MTU.

Joe

---

From: Manoj Nayak <[email-masked]>
Date: Thu, 21 Nov 2019 04:14:54 +0000

Hello Fred,

Yes, it is possible that the largest fragment received is *much* smaller than the PMTU. However, a survey of 
popular operating systems reveals that the largest fragment does reflect the PMTU.

If we are really worried about this problem, the sender can ignore the ICMP message when the MTU is smaller than
 the “smallest believable value” (e.g., 1500 bytes).

There was another query, if it is worth doing Lossless PMTUD for ipv4.
IPv4 will be with us for a long time. Lossless PMTUD doesn’t take much effort. So why not ...

Regards
Manoj Nayak

On 14/11/19, 8:15 PM, "Templin (US), Fred L" <[email-masked]> wrote:

    Manoj,

    
    
    Fred
    
    > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/int-area__;!8WoA6RjC81c!Tp5Pvame6pgAtmlsi-9_todREdZAH9ervNS0qugjDf4SKKWoRMSMvJ_yFYKUwTwaXCI$

---

From: Joe Touch <[email-masked]>
Date: Wed, 20 Nov 2019 21:06:18 -0800

> On Nov 20, 2019, at 8:14 PM, Manoj Nayak <[email-masked]> wrote:
> 
> Hello Fred,
> 

That’s great if that’s the only place this ever runs...

> the “smallest believable value” (e.g., 1500 bytes).

I suppose that means you don’t “believe” in tunnels.

Seriously, all this work to get direct report from the endpoint of a packet size that works
- using a protocol that we don’t rely on because it’s generally blocked
- providing a number that might work, if you run on “popular” systems
- all to then throw it out because it isn’t what you “expect”

> IPv4 will be with us for a long time. Lossless PMTUD doesn’t take much effort. So why not …

I provided a partial list above.

Joe

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 22 Nov 2019 13:50:35 +0000

You forced me to go back to my old documents to see where this is discussed. The
idea of using IPv4 fragmentation as a lossless path MTU determination mechanism
appears in RFC5320. The SEAL header indeed includes an "R" bit telling the receiver
when a fragmentation report response is desired.

As an experimental, RFC5320 did not go into an implementation diversity survey;
only linux. Listening to hearsay that MTU-sized fragments cannot normally be
expected is one of the reasons we abandoned the SEAL approach.  But, if you
have done a survey and concluded that the vast majority of implementations
result in MTU-sized fragments then good for you.

I still think though that some kind of RF bit is needed. In the same way that
ICMP "fragmentation needed" is only sent when DF is 1, ICMP "fragmentation
report" should only be sent when RF is 1. Or, maybe you want to send ICMP
"fragmentation needed" regardless of the state of the DF bit?

I think applications that rely on fragmentation will not want to receive ICMP
fragmentation reports over links that might have long delays and/or low
data rate capacities. The application knows it is invoking fragmentation;
it does not want for the network to be sending it useless reports along
the reverse path. Hence the RF bit.

Thanks - Fred

> From: Manoj Nayak [mailto:[email-masked]]
> Sent: Wednesday, November 20, 2019 8:15 PM
> To: Templin (US), Fred L <[email-masked]>; [email-masked]
> Cc: [email-masked]
>     > https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/int-area__;!8WoA6RjC81c!Tp5Pvame6pgAtmlsi-
> 9_todREdZAH9ervNS0qugjDf4SKKWoRMSMvJ_yFYKUwTwaXCI$
>

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 22 Nov 2019 14:00:43 +0000

BTW, RFC5320 gives attribution for where the original "report fragmentation" concept
came from (Charles Lynn; 1987).

Fred

> From: Int-area [mailto:[email-masked]] On Behalf Of Templin (US), Fred L
> Sent: Friday, November 22, 2019 5:51 AM
> To: Manoj Nayak <[email-masked]>; [email-masked]
> Cc: [email-masked]
> >     >     > Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available
> at
> >     > tools.ietf.org.
> >     >     >
> >     >     > The IETF Secretariat

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 22 Nov 2019 14:28:12 +0000

And to highlight the idea so it isn't lost, why not just have the reassembler send
back an ordinary ICMPv4 "Fragmentation Needed" instead of inventing a new
ICMPv4 message type?

Fred

> Sent: Friday, November 22, 2019 6:01 AM
> > >     >     	(yes, your note in the end of sec 1 is relevant, but given v4-in-v4 tunneling, it?s possible that paths might be smaller
> than the
> > >     > 576 assumption)
> > >     >

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Fri, 22 Nov 2019 15:59:18 +0000

One last put, then I'll shut up. What it generation of fragmentation reports were made
the default behavior (reassembler sends ICMPv4 "fragmentation needed" by default)
and a socket option is provided so that specific applications can disable the default
behavior. That way, applications that rely on IPv4 fragmentation can use it without
worrying about receiving unwanted ICMPs, and no RF bit is needed in any protocol
header.

With these inputs, and assuming the implementation survey is sound, I believe
I can withdraw my objections.

Fred

> Sent: Friday, November 22, 2019 6:28 AM

---

From: Manoj Nayak <[email-masked]>
Date: Sat, 23 Nov 2019 14:23:54 +0000

Hello Joe,

Section 5 of the below RFC talks about the way to relay MTU size to sender if ipv4-to-ipv4 tunnel is
there in the path from sender to destination.

There are two cases here.

  1.  Destination is outside tunnel.
  2.  Destination is tunnel end point.

When destination is tunnel end point then MTU size is returned to sender using soft state maintained
at tunnel entry point.

https://tools.ietf.org/html/rfc2003

5<https://tools.ietf.org/html/rfc2003#section-5>. Tunnel Management

   Unfortunately, ICMP only requires IP routers to return 8 octets (64
   bits) of the datagram beyond the IP header.  This is not enough to
   include a copy of the encapsulated (inner) IP header, so it is not
   always possible for the encapsulator to relay the ICMP message from
   the interior of a tunnel back to the original sender.  However, by
   carefully maintaining "soft state" about tunnels into which it sends,
   the encapsulator can return accurate ICMP messages to the original
   sender in most cases.  The encapsulator SHOULD maintain at least the
   following soft state information about each tunnel:

    - MTU of the tunnel (Section 5.1<https://tools.ietf.org/html/rfc2003#section-5.1>)
    - TTL (path length) of the tunnel
    - Reachability of the end of the tunnel

Regards
Manoj Nayak

From: Joe Touch <[email-masked]>
Date: Friday, 15 November 2019 at 8:20 PM
To: Manoj Nayak <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [Int-area] New Version Notification for draft-bonica-intarea-lossless-pmtud-00.txt

On Nov 13, 2019, at 8:34 PM, Manoj Nayak <[email-masked]<mailto:[email-masked]>> wrote:

Hello Joe,

Please find my reply.

How the idea would be

Joe

---

From: Joe Touch <[email-masked]>
Date: Sat, 23 Nov 2019 08:02:19 -0800

You haven’t answered any of the questions I asked, including the one about tunnels.

The point about tunnels is that the numbers you’re assuming don’t account for the space needed for tunneling.

The portion of RFC2003 below acts like a router; it isn’t relevant to your mechanism. Sec 5.2 of draft-ietf-intarea-tunnels also points out several errors in that RFC, including those in how that RFC refers to and calculates MTUs.

Joe

> On Nov 23, 2019, at 6:23 AM, Manoj Nayak <[email-masked]> wrote:
> 
> Hello Joe,
>  
> Destination is outside tunnel.
> Destination is tunnel end point.
>  
> https://tools.ietf.org/html/rfc2003 <https://tools.ietf.org/html/rfc2003>
>  
>  <>
> 5 <https://tools.ietf.org/html/rfc2003#section-5>. Tunnel Management
>  
>  
>     - MTU of the tunnel (Section 5.1 <https://tools.ietf.org/html/rfc2003#section-5.1>)
> From: Joe Touch <[email-masked] <mailto:[email-masked]>>
> To: Manoj Nayak <[email-masked] <mailto:[email-masked]>>
> Cc: "[email-masked] <mailto:[email-masked]>" <[email-masked] <mailto:[email-masked]>>
>> On Nov 13, 2019, at 8:34 PM, Manoj Nayak <[email-masked] <mailto:[email-masked]>> wrote:
>>  
>> Hello Joe,
>> 
>> Please find my reply.
>> 
>>

---

From: Manoj Nayak <[email-masked]>
Date: Fri, 29 Nov 2019 10:03:35 +0000

Hello Joe,

Sorry for the delayed response.

>> Please explain. Most ICMP messages have 4 bytes of unused field, but not all (one has only 3).

      As per RFC 1191,  Destination Unreachable Message has 8 bytes unused field.

       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |   Type = 3    |   Code = 4    |           Checksum                                 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           unused = 0          |         Next-Hop MTU                                |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |      Internet Header + 64 bits of Original Datagram Data            |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  Most of ICMP reply/error message need type, code and checksum in first four bytes.
  So our new ICMP message for this draft has the above three fields in first four bytes.
  We have used 1 byte for length and two bytes for Largest fragment. Our ICMP
  Package looks as follows:

        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |     Type      |     Code      |          Checksum             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |     Unused    |    Length     |       Largest Fragment        |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                      Original Datagram                                                        |
       |                                                                                                              |
       |                           //                                                                                |
       |                                                                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  If the packet mentioned in RFC 1191 does not have any issue with tunnel
  then how tunnel can create problem for our packet.

>> Sending the largest response possible given an untunneled MTU size is an invitation
>> to black-hole the response itself if (when) an IPv4-in-IPv4 tunnel is encountered.
>> In most situations, ICMP responses are received from small initial messages that don’t
>> stress that limit. The use in this doc is the opposite - it relies on ongoing use of ICMP
>> for max-sized packets and returns max-sized payloads. This isn’t helpful. It would be more
>> useful to try to limit the size to the minimum expected to be useful and account for
>> these other encapsulations.

We have mentioned that Original Datagram will be there in the third row of our ICMP packet.
However Internet Header + 64 bits of Original Datagram Data is good enough for us
to link our ICMP packet to the original packet that has generated this ICMP packet
at the sender.

>> In that case, none of the fragments gives you the path MTU.

The first fragment is the largest fragment when all fragments are equal sized.

Regards
Manoj Nayak

Date: Saturday, 23 November 2019 at 9:32 PM

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 29 Nov 2019 09:01:35 -0800

> On Nov 29, 2019, at 2:03 AM, Manoj Nayak <[email-masked]> wrote:
> 
> Hello Joe,
>  

DU has only 4 bytes unused, per RFC792.

Destination Unreachable Message

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |     Code      |          Checksum             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                             unused                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Internet Header + 64 bits of Original Data Datagram      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

RFC1191 uses half that space:

   ...To support the Path MTU Discovery
   technique specified in this memo, the router MUST include the MTU of
   that next-hop network in the low-order 16 bits of the ICMP header
   field that is labelled "unused" in the ICMP specification [7 <https://tools.ietf.org/html/rfc1191#ref-7>].  The
   high-order 16 bits remain unused, and MUST be set to zero.

The diagram above  is from 1191 - there are 2 bytes unused.

>   Package looks as follows:

You’re not using 8 unused bytes. You are following RFC792 for the first three fields (first 4 bytes) and defining a new template for the next 4. 

>   then

It does because it doesn’t include the entire original datagram….

> how tunnel can create problem for our packet.
>     

IH is a min of 20 bytes (for IPv4). The next 8 bytes of the tunnel header might not be enough to determine what to do with the ICMP message.

> The first fragment is the largest fragment when all fragments are equal sized.

It is the largest - but not the path MTU.

You claim that this technique will find the path MTU, but it will not in that case - and you can’t know when case occurs.

Joe

---

From: Derek Fawcus <[email-masked]>
Date: Sat, 30 Nov 2019 22:00:27 +0000

> You’re not using 8 unused bytes. You are following RFC792 for the first three fields (first 4 bytes) and defining a new template for the next 4. 

I'd suggest they seem to be starting from RFC 4884 (although they do not reference it),
and then seeing what else they can do.

DF

---

From: Joe Touch <[email-masked]>
Date: Sat, 30 Nov 2019 16:29:22 -0800

> On Nov 30, 2019, at 2:00 PM, Derek Fawcus <[email-masked]> wrote:
> 

That may be where they’re getting the “as much of the datagram as possible” part in some cases, but not the 8 unused bytes.

But if so, they need to start with a citation.

Either way, we still end up where we started - we do PLPMTUD because ICMPs are too often blocked. Not just from routers.

Joe