Subject: [Fwd: fragmentation of IP encaps'd LSPs ....]

From: Antoni Przygienda <[email-masked]>
Date: Sat, 12 Dec 1998 15:33:23 -0500

Subject: Re: fragmentation of IP encaps'd LSPs ....
From: Antoni Przygienda <[email-masked]>
To: [email-masked]
Date: Sat, 12 Dec 1998 00:10:56 -0500

Danny McPherson wrote:
> 
> Antoni Przygienda wrote:
> >After some insightfull discussions with Paul T. & Tony L. I think that
> >the LSPs encapsed over IP should be allowed to be fragmented despite the
> >initial result of the workgroup discussion. I think that there was some
> >misunderstanding as of the fact whether the originatingLXLSPBufferSize
> >would be still honored or not. My assumption was that it would be
> >disregarded when running encapsulated mode & the LSPs could grow up to
> >maximum of the IP datagram size. However, this would cause problems when
> >running networks with mix of OSI and IP encapsulation, of course.  So I
> >think
> >that the following is the best solution:
> >
> >       x never construct anything larger than bufferSize (that will allow to
> >               keep the size small if
> >               necessary in mixed environments by configuration)
> >       x if bufferSize larger than dataLinkSize, you are allowed to build LSPs
> >               up to bufferSize but
> >               you SHOULD not fragment
> 
> Hrrmm.. If  bufferSize is larger than dataLinkSize and LSPs can be built
> up to bufferSize, and you SHOULD not fragment, then you suggest that
> you "SHOULD now fragment, but can if required"?  Seems a bit confusing
> to me...
> 

Danny, 

I think that I caused some confusion by not saying what is "fragment",
we have 
two semantics for the same word now & need more context. I aimed to say
"you SHOULD not 
cause IP fragmentation if possible" without being carefull enough. 

> >however, the hellos in 10589 today say that you should construct the
> >MAXIMUM of bufferSize and dataLinkSize. That will fail & we need to say
> >there that on IP encaps'd interface you only construct up to
> >dataLinkSize.
> 
> Is it even necessary to construct Hello to dataLinkSize if LSP can
> be fragmented .. except when mixing encapsualtions within a routing
> domain, which would likely only result from mixing dual and IP-only,
> wihich is ill-advised in RFC1195, I believe?
> 
> Also, by doing this are we suggesting that with IP it's more
> efficient to fragment LSPs only at max IP datagram size, rather than
> fragmenting at originatingLXLSPBufferSize and being able to only
> send "portions" of an LSP when only portions of the LSP change?
> I'd consider that a notable offshoot to this approach, while in
> the end I agree that your suggestion is probablt more efficient.
> 

0. We all agreed violently on a previous thread that relying primarily
on IP fragmentation 
is very troublesome & should be avoided. 

1. You are right for the SNPs, they should be Minimum of the MTU &
originatingLXLSPBufferSize 
	on IP encapsulated interfaces (to prevent IP fragmentation again), but
it's analog to OSI so that's why I didn't bother 
	to mention it (which was a mistake). 

I have some difficulties following the rest of the argument here, maybe
we should alwasy
speak clearly about "IP fragmentation" and "LSP fragmentation". Let me
rephrase the 
proposed solution for an IP encaps'd interface as precise as I can:

EO stands for encapsulation overhead size (e.g. IP header)

	x you MUST send IIHs of size (MTU-EO) (not of maximum of Buffer &
DataLinkSize
			like in OSI) with DF set
	x you MUST NOT send SNPs larger than the minimum of MTU-EO and
originatingLXLSPBufferSize
	x you CAN construct LSP fragments of size up to
originatingLXLSPBufferSize
	x you MUST NOT construct LSP fragments larger than
originatingLXLSPBufferSize
	x LSPs are sent without DF

That will mean that anytime you configure your network to have
originatingLXLSPBufferSize larger than any the interface 
MTUs, IP fragmentation can kick in.  There is the last rule that I
didn't spell out clearly and that is tricky & in my
opinion will not work well

	x you SHOULD NOT construct LSP fragments larger than the size of the
minimum of dataLinkBlockSize and  originatingLXLSPBufferSize 
		(alternately we could use a default value as the infamous 1492 (or
1472)).

This will probably misfire in many cases and it makes more sense IMHO to
keep the size controlled by the originatingLXLSPBufferSize only.

Uff, this is getting tough ...

		thank you

		-- tony

---

From: Rajesh Balay <[email-masked]>
Date: Mon, 14 Dec 1998 11:17:29 -0500

> cause IP fragmentation if possible" without being carefull enough.

You sure have confused me :) .. when i raised the concern about fragmentation
w.r.t. interoperability at Orlando (session-1). It took me a while to realize 
we were speaking on different terms - IP vs LSP fragmention.

I agree with what is proposed below, but think that there is still room
for some confusion. I think the advantages of using IPv4 are 
two fold:

A. The ability to use IP stack (instead of ISO)

B. Make use of the advantages due to use of IPv4. Here one primary issue 
we are touching is the IP fragmentation. This suggests some change in IS-IS
protocol; i think there will be more changes as we try to optimize the protocol
over Ipv4 (which could be addressed later).

I suggest the following keeping the above in perspective:

Add sections to describe the behavior of the ISs in a mixed-IP mode (ISs using 
IS-IS over IPv4 and those using the rfc1195 IP-mode in the same area/domain) 
vs pure IPv4 environment. For A above, we will have minumum or no change in how
we interpret parameters, etc. Maybe list some MUST DO clauses to have a
valid (or operational) mixed environment. For the "over-IPv4-only" case we could
specify the additional/changed protocol behavior to take advantage of IPv4
features. For example, the new scope of originatingLXLSPBufferSize (or LSP MTU)
is no longer limited to be the min MTU of the interfaces at the IS.

> Uff, this is getting tough ...

regards,
-- 
Rajesh Balay 
Torrent Networking Technologies
(919) 468-8466, Extension 222

---

From: Mike Shand <[email-masked]>
Date: Mon, 14 Dec 1998 11:17:12 +0000

>	x you MUST send IIHs of size (MTU-EO) (not of maximum of Buffer &

Do we still believe it is useful to pad the hellos to max size? Especially
if we are going to allow LSPs to be (IP) fragmented? Most implementation
allow this to be turned off anyway.

	Mike

---

From: Joseph Malcolm <[email-masked]>
Date: Mon, 14 Dec 1998 13:17:55 -0500 (EST)

>allow this to be turned off anyway.

From the operator perspective, I have found this very useful to detect
some otherwise non-obvious failures.

---

From: Tony Li <[email-masked]>
Date: Mon, 14 Dec 1998 10:45:59 -0800 (PST)

|  >	x you MUST send IIHs of size (MTU-EO) (not of maximum of Buffer &
|  >DataLinkSize
|  >			like in OSI) with DF set
|  
|  Do we still believe it is useful to pad the hellos to max size? Especially
|  if we are going to allow LSPs to be (IP) fragmented? Most implementation
|  allow this to be turned off anyway.

To be specific, the pragmatic point behind padding IIH's is to insure that
user data sees the same MTU that the network thinks is configured.  While
this is not a perfect check (e.g., the IS-IS path is not identical to the
transit path), it does catch obvious and blatant MTU mismatches.

Tony

---

From: Mike Shand <[email-masked]>
Date: Tue, 15 Dec 1998 09:52:13 +0000

>transit path), it does catch obvious and blatant MTU mismatches.
Yes. That's why we put it in there in the first place. (and to catch any
problems caused by length specific data link errors, and cases where the
MTU of a link changed by for example changes in FDDI/Ethernet bridge
configuirations.) But I've seen so many complaints from users about all
these 'big' packets, that I was wondering if it was actually meeting a real
(as opposed to perceived) need. Evidently it does. Good.

	Mike

---

From: Geert Jan de Groot <[email-masked]>
Date: Tue, 15 Dec 1998 14:20:45 +0100

> some otherwise non-obvious failures.

I think that the ability to detect MTU mismatches is indeed very useful,
but I think it's also very useful for non-IS-IS clouds.
That is, MTU mismatch detection is very desirable but I wonder if
doing this in a routing protocol (thereby ONLY giving the benefits
of this detection to the users of this protocol) is something we want;
I think MTU mismatch detection is also desirable for non-IS-IS users
so maybe it doesnt' belong in a routing protocol per se.

Even more so because I have seen a number of mismatches on customer
lines, where it's unlikely that one would run IS-IS over them, and
(because both ends of the line are managed by different organisations),
it's more likely to detect this kind of error.

Geert Jan

---

From: Tony Li <[email-masked]>
Date: Tue, 15 Dec 1998 09:25:27 -0800 (PST)

|  I think that the ability to detect MTU mismatches is indeed very useful,
|  but I think it's also very useful for non-IS-IS clouds.
|  That is, MTU mismatch detection is very desirable but I wonder if
|  doing this in a routing protocol (thereby ONLY giving the benefits
|  of this detection to the users of this protocol) is something we want;
|  I think MTU mismatch detection is also desirable for non-IS-IS users
|  so maybe it doesnt' belong in a routing protocol per se.
|  
|  Even more so because I have seen a number of mismatches on customer
|  lines, where it's unlikely that one would run IS-IS over them, and
|  (because both ends of the line are managed by different organisations),
|  it's more likely to detect this kind of error.

Reasonable.  This would seem more like a call for some OTHER mechanism to
do this.  If and when that mechanism existed, backing it out of IS-IS would
be reasonable.

Meanwhile, in this century....  ;-)

Tony

---

From: Joseph Malcolm <[email-masked]>
Date: Tue, 15 Dec 1998 12:49:14 -0500 (EST)

>it's more likely to detect this kind of error.

No argument, but let's not take away a tool that already works before
providing a better one.

---

From: Henk Smit <[email-masked]>
Date: Mon, 21 Dec 1998 18:48:37 +0100 (MET)

Look at what NLSP does. TLV 0xC5 in p2p IIHs.
 Local MTU of the interface over which the IIH is sent.

  We could do something similar in ISIS. This would allow routers to
 detect MTU mismatches, without the overhead of padding.

  A few possible problems are:
 1) it would not detect the corner case problems of transparent bridging.
    (like FDDI-ethernet-FDDI).
 2) corruption of large packets, but not small packets (sometimes possible
    over some WAN transmission media).
 3) ISIS MTU might not be the same as the IP (or CLNS) MTU.

  Anyway, I think such a new TLV will still be beneficial.
 How about just using the number and format from NLSP ?
 
       Henk.

---

From: Joseph Malcolm <[email-masked]>
Date: Mon, 21 Dec 1998 13:07:35 -0500 (EST)

>    (like FDDI-ethernet-FDDI).

Or interworking. (FR-ATM, for example.)

>    over some WAN transmission media).

I've actually observed this.

> 3) ISIS MTU might not be the same as the IP (or CLNS) MTU.

Hmm. I'm not sure why one would do this, but I suppose it is possible.

> How about just using the number and format from NLSP ?

Seems like it would be useful to me.

---

From: Tony Li <[email-masked]>
Date: Mon, 21 Dec 1998 19:04:37 GMT

|    Look at what NLSP does. TLV 0xC5 in p2p IIHs.
|   Local MTU of the interface over which the IIH is sent.
|  
|    We could do something similar in ISIS. This would allow routers to
|   detect MTU mismatches, without the overhead of padding.

True, however, this would detect the difference in the configuration.  This
may or may not be the same thing as the actual, delivered MTU used by the
lower layers of the system.  I'll just say 'FFB' and Henk will agree.  ;-)

The advantage to padding is that it is explicit, it is brain-dead simple,
and it's pretty much guaranteed to work.  If you're that concerned about
the overhead, you can not pad things once the adjacency is up on a ptp
link.  This has some other, subtle failure modes (e.g., SONET cutover), but
is worth the tradeoff.

Tony

---

From: Mike Shand <[email-masked]>
Date: Mon, 21 Dec 1998 21:09:30 +0000

>link.  This has some other, subtle failure modes (e.g., SONET cutover), but

Yes, I agree its a simple and effective way to do it if it is required (and from the responses it would appear that it is a real problem), and we should certainly retain the ability to do it. I'm just inclined to allow it to be turned off to satisfy those who complain about the overhead and don't care about detecting the problems (though part of me still retains the old 'nanny knows best' attitude of the original design). Incidentally, 10589 says to only pad the initial IIHs on pt-pt (as I recall - I haven't checked). As you say this doesn't catch the cases where the pt-pt link degrades (either by cutover, or some error condition). The original thinking was that the pt-pt link was a reliable service (e.g. HDLC or DDCMP). Hence, once you had tried to send a 'too big' packet, anything else would get queued up behind it and the link would eventually fail. I don't think it works for PPP.

	Mike

---

From: Tony Li <[email-masked]>
Date: Mon, 21 Dec 1998 21:24:48 GMT

|  Yes, I agree its a simple and effective way to do it if it is required
|  (and from the responses it would appear that it is a real problem), and
|  we should certainly retain the ability to do it. I'm just inclined to
|  allow it to be turned off to satisfy those who complain about the
|  overhead and don't care about detecting the problems (though part of me
|  still retains the old 'nanny knows best' attitude of the original
|  design). Incidentally, 10589 says to only pad the initial IIHs on pt-pt
|  (as I recall - I haven't checked). As you say this doesn't catch the
|  cases where the pt-pt link degrades (either by cutover, or some error
|  condition). The original thinking was that the pt-pt link was a reliable
|  service (e.g. HDLC or DDCMP). Hence, once you had tried to send a 'too
|  big' packet, anything else would get queued up behind it and the link
|  would eventually fail. I don't think it works for PPP.

Actually, it works fine, regardless of whether the link layer is reliable
or not.  If the IIH doesn't propagate, you don't get and adjacency.  If you
don't get an adjacency, you don't forward traffic through it.  If you don't
forward traffic through it: there's no problem.  ;-)

Tony

---

From: Mike Shand <[email-masked]>
Date: Tue, 22 Dec 1998 10:01:25 +0000

>forward traffic through it: there's no problem.  ;-)

No. If the IIH doesn't propagate, you send another when the timer goes off,
and that one isn't required to be padded, so you get the link up even
though the MTU is too small. With a reliable service you can't send (well
strictly, receive) the second one until the first one has been correctly
received.

BTW. my memory seems to be wrong, since 10589 seems to allow an IIH to be
sent when the circuit is first enabled without waiting for an ISH. This
would appear to completely circumvent the padding, since it only requires
padding of the one sent in response to an ISH. Looks like the text has got
thoroughly messed up here.

---

From: Mike Shand <[email-masked]>
Date: Tue, 22 Dec 1998 10:36:19 +0000

>thoroughly messed up here.

But V2 of 10589 seems to have corrected this. It now says (correctly IMHO)
that you send an ISH when the circuit is enabled, and only send an IIH when
you have seen an ISH (and on subsequent timer expiry). It still suffers
from the PPP problem I identified.

This ISH stuff is of course something of an anachronism. If we've got the
hood up on this we should probably get rid of that and devise an
initialization stategy that works for PPP (which may be as simple as
padding all IIHs). Of course it would have to be compatible with what we
already have (in all its variants? :-) )

	Mike

---

From: Henk Smit <[email-masked]>
Date: Tue, 22 Dec 1998 15:51:38 +0100 (MET)

> lower layers of the system.  I'll just say 'FFB' and Henk will agree.  ;-)

  Haven't dealt with that in detail, but yes, there are little issue
 like that. This is also related what I said about "different ISIS, IP
 and CLNS MTUs".

  I always believed that the padding helps prevent MTU mismatches, but
 does not totally prevent them. In cisco's IOS we do not check for
 the lenght of incoming IIHs. As far as I can see, nowhere in 10589
 it says one should check the lenght. So if there is a mismatch between
 MTUs, ISIS (and packet forwarding) might keep working as long as
 the difference is small. Or as long as the receiving router reserves
 buffers that are (by coincedence) large enough to fit the incoming IIHs
 or data packet.

> link.  This has some other, subtle failure modes (e.g., SONET cutover), but

  During SONET cutover from A-B to A-C the adjacency should bounce,
 shouldn't it ? Or are you talking about cutover from A-B to A-B
 where only the ring wraps ? In the latter case the MTU issue is related
 to the translational bridging/switching issue.

        Henk.

---

From: Tony Li <[email-masked]>
Date: Tue, 22 Dec 1998 18:27:32 GMT

|  >Actually, it works fine, regardless of whether the link layer is reliable
|  >or not.  If the IIH doesn't propagate, you don't get and adjacency.  If you
|  >don't get an adjacency, you don't forward traffic through it.  If you don't
|  >forward traffic through it: there's no problem.  ;-)
|  
|  No. If the IIH doesn't propagate, you send another when the timer goes off,
|  and that one isn't required to be padded, so you get the link up even
|  though the MTU is too small. With a reliable service you can't send (well
|  strictly, receive) the second one until the first one has been correctly
|  received.

[Sound of furious page turning.]

Ah, you're right.  8.2.3.c is quite clear.  This could perhaps be enhanced.

|  BTW. my memory seems to be wrong, since 10589 seems to allow an IIH to be
|  sent when the circuit is first enabled without waiting for an ISH. This
|  would appear to completely circumvent the padding, since it only requires
|  padding of the one sent in response to an ISH. Looks like the text has got
|  thoroughly messed up here.

I would agree.  I would tend to extend this text to have all IIH's be
padded until the adjacency is up.

Tony

---

From: Tony Li <[email-masked]>
Date: Tue, 22 Dec 1998 18:36:48 GMT

|  > The advantage to padding is that it is explicit, it is brain-dead simple,
|  > and it's pretty much guaranteed to work.  If you're that concerned about
|  > the overhead, you can not pad things once the adjacency is up on a ptp
|  > link.  This has some other, subtle failure modes (e.g., SONET cutover), but
|  > is worth the tradeoff.
|  
|    During SONET cutover from A-B to A-C the adjacency should bounce,
|   shouldn't it ? Or are you talking about cutover from A-B to A-B
|   where only the ring wraps ? In the latter case the MTU issue is related
|   to the translational bridging/switching issue.

I was considering the A-B to A-C cutover.  Ignoring any proprietary
mechanisms, there is at least one scenario that I'm told can happen that
may not cause the adjacency to drop.

Scenario: Suppose that there's a SONET ADM immediately adjacent to both B
and C and C is provisioned as the protect side of the circuit.  The
particular ADM replicates traffic from A to both B and C, but only accepts
bits from the active side of the circuit (initially B).  The A-B adjacency
comes up and operates normally.  

The A-C adjacency is in a very wierd state: A never sees C, but C sees all
of A's IIH's.  Because this is a PTP circuit, and even with the DKatz
extensions, C must bring his side of the adjacency up!  C cannot tell that
its IIH's aren't being seen by A.  If this were a LAN, then A's IIH should
include C's SNPA, but there's no such provision for PTP.

Now: consider what happens if an operator trips over the fiber between B
and the ADM.  The ADM fails over to C.  Both sides see an up adjacency and
nothing has flapped.

Note that if C's MTU is misconfigured AND we're no longer sending padded
IIH's, then you can now get an MTU mismatch with an up adjacency.

Sick and twisted?  Yes.  Low probability?  Yes.  Still could happen.

Tony