Subject: Re: [Mip4] Negotiating maximum packet size (draft-vaarala-mip4-fragmtu-00)

From: Kent Leung <[email-masked]>
Date: Mon, 01 Mar 2004 21:26:12 -0800

Hi Sami.

Interesting problem of dropping packets with DF bit unset.  Haven't
experience that.  The ICMP filter is common.  Some questions below.

1) Why GRE encap not supported?  Seems like adding a few words would
     allow that since there's no technical hurdle to cover supported 
encapsulation
     modes.

2) Does it make sense for FA to advertise the Frag-MTU-Pref value so
     the MN can send that info to the HA in a protected manner?

3) Not clear how the MN's IP stack use this effective MTU?  Is the IP MTU
     set to this value to tune the TCP MSS negotiation between MN and CN?
     IMHO, this is the most important objective so Tunnel MTU Discovery is
     not needed for obtain optimal MTU size.

4) If effective MTU is not related to IP MTU to affect TCP MSS, then
     we will still have the black hole problem between CN and MN due
     to ICMP filters.

5) If effective MTU is used for TCP MSS, then does it make sense to
     re-negotiate on each registration?  Should this be accomplished
     only in the initial registration?  How would active TCP session
     deal with changing IP MTU?

6) Let's say that MN has 2 interfaces, one that use IP-in-IP and the
     other use UDP.  Initially, MN roams to a network that uses IP-in-IP.
     TCP sessions are started with CNs.  Then MN roams to network
     that requires UDP.  We would be back at not having optimum
     MSS.

7) Seems like the lowest common denominator is the effective MTU
    of the lowest MTU among UDP, IPinIP, GRE?  And if one enables
    IPSec too, the optimal MTU shrinks more.  Just wondering at the
    end of the day, it may be just easier to say regardless which
    tunneling method is used, go with MTU size 1400 for one size fits
    all?

8) Is it expected that the HA which receives the effective MTU will
     set the tunnel mtu soft state accordingly for tunnel mtu discovery?
     So if tunnel mtu discovery is enabled on the tunnel between HA
     and CoA, then soft state is set.  If tunnel mtu discovery is disabled,
     the effective mtu is not used for forwarding MN traffic toward CoA?
     General question is how is the effective MTU linked to tunnel MTU
     Discovery?

9) Though this draft seems to say that the HA processes the effective MTU
     by ensuring that all encapsulated traffic (original packet is 
fragmented if
     too big) are within the size of effective MTU, when the DF bit is unset.
     In this case, when DF bit is set, HA will send ICMP unreachable to
     source.  This message may be filtered.

Kent

At 09:44 PM 2/24/2004 +0200, [email-masked] wrote:
>Hi,
>
>Here's a short draft on how the MN and the HA can negotiate the
>maximum packet size used for IP-IP or IP-over-UDP encapsulation.
>
>  http://www.ietf.org/internet-drafts/draft-vaarala-mip4-fragmtu-00.txt
>
>Although RFC 3519 already specifies that you first fragment and
>then encapsulate, the maximum size of encapsulated UDP packets
>is implementation specific (most likely link MTU).
>
>We've run into some configurations where a box is configured to
>drop packets larger than a certain size even when DF is unset.
>Limiting packet size helped, but cannot be controlled using a
>policy in the HA (for instance).  The intent of the mechanism in
>the draft is to make this possible.
>
>Has anyone else run into similar issues in practice?  Any
>feedback is appreciated.
>
>Best,
>
>-Sami
>--
>Sami Vaarala
>http://www.hut.fi/~svaarala
>
>--
>Mip4 mailing list
>[email-masked]
>https://www.ietf.org/mailman/listinfo/mip4

-- 
Mip4 mailing list: [email-masked]
    Web interface: https://www.ietf.org/mailman/listinfo/mip4
     Charter page: http://www.ietf.org/html.charters/mip4-charter.html
Supplemental site: http://www.mip4.org/

---

From: [email-masked]
Date: Tue, 02 Mar 2004 13:44:04 +0200

Hi Kent, 

Thanks for the comments, some responses and questions below. 

>     encapsulation modes.

No particular reason - this draft just documents what we thought would
solve our most immediate problems.  GRE is no different, except that
you may run protocols over GRE that don't support the notion of
fragmentation.  (Of course IP over GRE will work just fine.) 

>     the MN can send that info to the HA in a protected manner?

I'm not sure what you mean here, can you elaborate?  The preference
is protected using MN-HA authenticator?  And the MN and HA are
configured with their respective values, so they don't need input
from network. 

>     not needed for obtain optimal MTU size.

Well, that's one thing you can do, and should do (if possible).  You can
also do MSS rewriting and other such things. 

>     we will still have the black hole problem between CN and MN due

That's true - but this would also be the case if the MN was at home
and trying to communicate with the MN.  The intent of the mechanism
in the draft is to solve the case where the MN is away, and the route
between the MN and the HA is poor - although a connection from the MN
home network would work just fine.  (Tweaking the TCP MSS for initiated
connections so that it will work when initiated from the home network is
outside the MIP4 scope - there isn't necessarily even a registration
involved if the MN boots up at home.) 

>     deal with changing IP MTU?

If DF was set, the MIP code could simply manufacture an ICMP back to the
stack with packet too big (i.e. a local ICMP).  There are other
possibilities, though. 

>     that requires UDP.  We would be back at not having optimum

Yes, that's true.  But if you do MSS tweaks (which are local matters
anyway), you may take the maximum packet size and subtract the maximum
overhead (UDP) when determining your offered MSS? 

>    tunneling method is used, go with MTU size 1400 for one size fits

That's what we've done in our previous product version, but actually
1400 is too large, at least for some specific networks (it's precisely
the value we ran into problems with :-).  Choosing a value that will
work for sure is black magic as the routers where this is a problem
are clearly broken in any event. 

>     General question is how is the effective MTU linked to tunnel MTU

The HA and the MN should do whatever they do with plain RFC 2003 /
RFC 3519.  There is no difference, except that the tunnel MTU may
occasionally change.  To overcome the IP-IP vs. IP-over-UDP
difference, the tunnel MTU can be taken to be the worst case, but
that's an implementation decision IMO.  Other than that, the tunnel
MTU should change extremely rarely (e.g. HA is reconfigured). 

>     source.  This message may be filtered.

Yes, if we have a filter between HA and CN we're in trouble, unless
we stop honoring the DF bit (which is tempting after dealing with a
number of black hole problems..:).  I don't think there is a way around
this problem.  However, we don't have to rely on ICMP between the MN and
the HA, which is a larger problem, because the MN moves around in whatever
networks that are out there. 

Best, 

 -Sami

---

From: Kent Leung <[email-masked]>
Date: Tue, 02 Mar 2004 22:53:55 -0800

Generally, I like the idea, and like to extend it a bit further to deal with
the broader problem of avoiding fragmentations which is introduced
by the MIP tunnel for roaming MNs.

More comments below.

At 01:44 PM 3/2/2004 +0200, [email-masked] wrote:
  GRE is no different, except that
>fragmentation.  (Of course IP over GRE will work just fine.)

Yes, I implied IP over GRE.

>configured with their respective values, so they don't need input

In this case, MN is using an FA.
When MN learns about an FA, the FA advertises an effective MTU
to use to connect to any HA.  MN adds that to the extension in the
RRQ.  This way, the info is protected by the MN-HA AE.  The current
method in draft specifies the extension is after MN-HA AE.  Just a
thought if it makes sense to follow the same model as GRE tunnel
support using G bit.

>also do MSS rewriting and other such things.

OK.  This would be very useful.

>involved if the MN boots up at home.)

But it's when MN is roaming which requires tunnels that introduces
the need to reduce the MTU size.  Here's where I see the differences.

Given all things equal, an IP payload size of 1500 can reach MN/CN
without fragmentation.

a) Let's say MN is at home.  MN has interface with MTU of 1500 and
uses that to negotiate MSS with CN.  There's no need for fragmentation.
b) MN is roaming.  MN has interface with MTU of 1500.  But MIP
introduces a tunnel in the path between MN and CN.  Packets with
1500B must be fragmented.  The draft specifies the maximum size
of the encapsulated packets.  But traffic between MN and CN will
still suffer from the issue of "path MTU discovery relies on a number
of assumptions regarding ICMP processing in the intervening routers,
and is often problematic in the real world", right?

My idea is to use this effective MTU to set the MN's IP MTU so
MN/CN communications do not need to be fragmented.  Currently,
there are CNs which will automatically set DF bit.  And there are
ICMP filters.  This results in the need to disable Tunnel MTU Discovery
on the HA since no one knows who the MN will be contacting.

>stack with packet too big (i.e. a local ICMP).  There are other

When DF bit set, ICMP to the source may be filtered.  For the local case,
won't be a problem.

>overhead (UDP) when determining your offered MSS?

Yes, I was pointing out that effective IP MTU may determined not just by
the encapsulation method of the registration.

>are clearly broken in any event.

Agree.  1300 instead? :)  Finding the optimal IP MTU without the
aid of ICMP is guesswork.  But that is what we face today if DF bit
is set in the packets.

>MTU should change extremely rarely (e.g. HA is reconfigured).

I see the "For instance" wording in determining effective MTU section. :)
I had taken the examples too literally in my previous reading.

>networks that are out there.

Actually, I see that there's a generic ICMP filter issue regardless if it's
between MN and HA or between MN and CN.  Solving problem just
for MN and HA still forces us to disable Tunnel MTU Discovery since there
may be an ICMP filter in the HA/CN section.  But if we can set the IP MTU
for MN properly, then there is no problem for MN/HA or MN/CN cases?

Kent

---

From: [email-masked]
Date: Wed, 03 Mar 2004 12:15:15 +0200

Kent, 

Comments below. 

> Generally, I like the idea, and like to extend it a bit further to deal 
> with the broader problem of avoiding fragmentations which is introduced

Sounds good.  The current "problem statement" in the draft is probably
a bit too focused.  Expanding the scope a bit and writing up a better
rationale would be a good thing. 

>> [...]

Ah, I understand.  So even though the UDP encapsulation would in this
case be between FA and HA, we could use the MN-HA authentication to
set things up.  Didn't think about that :) 

This leaves the case where the MN is unaware of MTU negotiation but FA
is not.  This could work out as follows:  FA advertises, MN ignores the
advert and sends a normal RRQ.  The FA checks the RRQ for an MTU extension,
and if none is found, adds one after the MN extension (i.e. falls back
to less secure operation). 

The security implications are not straightforward; off the top of
my head: 

 - The HA must be required to ignore any MTU extensions after MN-HA
   authenticator if one is found within the authenticated extensions.
   Similar requirement for the MN. 

 - The FA has no way of verifying the HA->MN MTU extension (even when
   it is within the MN-HA authenticated extensions) - the MN doesn't
   tell the FA if authentication fails. 

 - For MN->HA direction the FA may be able to trust the MN as it's on the
   local link (more or less); if the MN sends a deliberately broken
   authenticator, that too will be detected (in most cases). 

But the end result is probably better than no authentication at all. 

> and is often problematic in the real world", right?

I see your point, and I agree.  We have an ICMP black hole which is not
exposed when using MTU of 1500.  Any reduction in MTU, for whatever
reason (such as MIP tunneling), exposes the problem? 

> on the HA since no one knows who the MN will be contacting.

Ok, I see what you mean.  I guess the most difficult part of doing
that is the CN; if we don't use ICMP, we have no way of telling the CN
to lower its MTU - except transport specific mechanisms such as TCP MSS?
Even so, getting the TCP MSS right from the MN's side would go a long
way in improving practical robustness of the connection. 

> the encapsulation method of the registration.

I agree; as you point out below, it also depends on presence of IPsec
and other things. 

> aid of ICMP is guesswork.  But that is what we face today if DF bit

1300 seems to work, in this particular case :) 

If one were thinking primarily about getting packets through, the
most optimal solution is probably to use a conservative MTU for the
tunneled packets, use DF=0 in the encapsulated header, and ignore
incoming DF-bit.  If combined with TCP MSS and MN IP MTU tweaking
this doesn't result in too many unoptimally fragmented packets.  But
this is rather ugly and affects path MTU discovery of upper layers. 

> Actually, I see that there's a generic ICMP filter issue regardless if 
> it's between MN and HA or between MN and CN.  Solving problem just

I can see how this solves the MN->CN case, but what about CN->MN?
Were you primarily thinking of TCP? 

Best, 

 -Sami