Subject: Re: [Int-area] ICMP-MPLS

From: Joe Touch <[email-masked]>
Date: Wed, 07 Sep 2005 13:50:57 -0700

I had a question about the current state of draft-ietf-mpls-icmp-03:

I'm concerned about the issue of backward compatibility, notably the
ways in which the use of the extensions proposed will cause existing
ICMP processing to break, notably binding the length of the final field
to 128 bytes, esp. considering RFC1812 recommends:

   Therefore, the ICMP
   datagram SHOULD contain as much of the original datagram as possible
   without the length of the ICMP datagram exceeding 576 bytes

Given that, it seems that this new variant of ICMP message (which
includes headers before IP, rather than IP and thereafter - which is
curious enough in itself) ought to demand a new message type, which
necessatates use of the "Parameter Problem" code, or the definition of
other new codes. Inside those, the new format that uses a list of
pointers to include MPLS information might be appropriate.

Use of the "unused" area, as Ron suggested, seems inappropriate because
those fields are not known to be 'cleared' by existing ICMP sources, so
their value cannot be reliably used for flags IMO.

Pekka wrote:
> So, publish the current draft as Informational with a suitable  note 
> saying that this is the currently deployed practise, stating  that 
> there are architectural problems in the way it is done, and  that
> the  practise should not be extended?  And at the same time,  try to
> do the  right thing for IPv6, and possible future IPv4  extensions?

Individual informational drafts describing existing practice seem OK,
but to do so within an IETF WG seems in appropriate when the practice is
in violation of existing specs, as this is. At the least, there needs to
be much more strong and clear wording that this technique is in
violation of those specs and will thus cause problems with compliant
applications.

That said, I'm still very unclear on why ICMP is the right protocol to
usurp for this purpose, as Pekka suggests later:

> My current rough approximation is that we should try to keep ICMP
> error messages for IP-related errors.  Hence, the question seems
> to boil down to what to do if the protocol beneath IP, such as
> MPLS, generates an IP-related error.
> 
> Now I must admit that I do not properly understand the
> interactions between MPLS and IP, but based on my casual
> reading of 2.4 of RFC3032 it seems to be that if MPLS TTL=0
> and IP TTL!=0, the right thing to do is NOT to send ICMP Time
> Exceeded but a new *informational* ICMP message stating that
> the MPLS TTL exceeded.  In my perhaps seriously flawed
> understanding, the case is equal to e.g. bridged Ethernet
> dropping a packet due a link layer error, and the right
> thing to me seems to handle it at the IP layer as a silently
> dropped packet.

I agree completely ;-)

Joe

_______________________________________________
Int-area mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/int-area

---

From: Joe Touch <[email-masked]>
Date: Wed, 07 Sep 2005 15:24:12 -0700

PS:

> pointers to include MPLS information might be appropriate.

One possible solution to the use of new codes, and the need for
compatibility with old programs:

	Send two ICMP error messages

I.e., for a particular message, send

	ICMP type 3 code 0 net unreachable
	with regular IP packet as payload

	ICMP type 16 (e.g., to indicate MPLS)
	with internal fields that equate to the
	conventional type (3) and code (0),
	with a payload that includes the MPLS
	header and pointer where the error occurred
	as well as the IP packet as payload

The general rules would be:

	a) any program that interprets MPLS-enabled messages
	   MUST NOT also act on or interpret the equivalent
	   non-MPLS messages

	b) routers MUST decrease the rate at which they send
	   errors, as an aggregate, by at least half, to
	   compensate for doubling the number of messages
	   i.e., an MPLS-enabled router MUST NOT send more of
	   the dual-message errors than an equivalent
	   non-MPLS router would send

Does that help?

Joe

---

From: Pekka Nikander <[email-masked]>
Date: Thu, 8 Sep 2005 09:34:49 +0200

Joe,

There are two separate issues here:

   - creating a document for the current practise, with strong
     enough disclaimers and at the same time trying to do it
     as "right" as possible

   - doing the right thing for IPv6

AFAIK, Ron is working on a new version of the current draft to
include comments from the discussion here, and to add a field
to the reserved field to make the extension a little bit better
in syntactical terms.  I've promised to review and help him once
he gets a reasonable version.

I am not aware of anyone working on the IPv6 issues.  I think
the MPLS WG should charter an item there is any real life
interest; otherwise it can wait, at least as far as I am
concerned.  Before that we just must make sure that the
current IPv4 practise is NOT adopted for IPv6.

Now, taking a technical step back, the issues is pretty
complicated since there are different uses of MPLS.  In some
cases MPLS is used more-or-less as a link layer protocol, and
the argumentation that appeared here and that you continue
holds pretty well.  However, in other cases MPLS is used more
as a source routing mechanism for IP.  That is, instead of
using the usual hop-by-hop IP routing based on routing tables
created by a routing protocol, the MPLS ingress router (Label
Edge Router) creates an MPLS "tunnel" (Label Switch Path) to
the MPLS egress router, using the information from the very
same routing tables.

Note that in the latter usage the route that the IP packets
take is exactly the same, using exactly the same routing
information.  Hence, there is no difference between plain
IP routing and MPLS-based IP routing.  The difference lies
in the forwarding plane, where in plain IP case each box in
the path examines the IP header and makes the forwarding
decision based on it while in the MPLS case the forwarding
decision is based on the MPLS label prepended to the IP
packet.  In other words, in the plain IP case the forwarding
decision is done individually at each router (hop-by-hop
forwarding decision) while in the MPLS case the forwarding
decision is effectively done at the ingress router, in a
sort-of source-routing fashion.

So, while in some cases the approach where MPLS is seen as
a link layer and we define new layer 2 specific ICMP messages
(or even a different protocol) seems to be right, in the other
cases it indeed appears to be the right thing to send an
ICMP time exceeded, for example.  The case really is that
there is a routing loop.

Unfortunately my current understanding stops here.  I still
don't know how to tell the difference between these two uses
of MPLS in practise, i.e., in the boxes in the running network.
And I still can't make my mind of what would really be the
right thing (for IPv6).

Now, some more detailed comments:

> I'm concerned about the issue of backward compatibility,
> notably the ways in which the use of the extensions
> proposed will cause existing ICMP processing to break,
> notably binding the length of the final field to 128
> bytes, esp. considering RFC1812 recommends:

IIRC, Ron provided us with some data indicating that in
*practise* not much is likely to fail.  Hence, while I
fully agree with  you that from the specs point of view,
the current MPLS practise a dubious one, it hasn't broken
that much in real life.

What Ron and I are currently planning to do is to use the
reserved field (in those ICMP packets that have it) to
indicate the new syntax.  The new syntax is not pretty
(and therefore should not be used for IPv6), but it is
compatible with the current usage.

> Given that, it seems that this new variant of ICMP
> message (which includes headers before IP, rather than
> IP and thereafter - which is curious enough in itself)
> ought to demand a new message type, which necessitates
> use of the "Parameter Problem" code, or the definition of
> other new codes. Inside those, the new format that uses a
> list of pointers to include MPLS information might be
> appropriate.

Yes, something like that should be done for IPv6.  It would not
be that good to specify it for IPv4, as people are not likely
to do that large changes to their existing code.

> Use of the "unused" area, as Ron suggested, seems
> inappropriate because those fields are not known to be
> 'cleared' by existing ICMP sources, so their value cannot
> be reliably used for flags IMO.

I'll let someone with more information on this to answer
that.  I have no real life data here.

> Individual informational drafts describing existing
> practice seem OK, but to do so within an IETF WG seems
> in appropriate when the practice is in violation of
> existing specs, as this is.

If violating the existing specs has proven not to cause
any real life problems (which I believe is the case here),
it may be better to change the specs than to expect the
real life to change.

> At the least, there needs to be much more strong and
> clear wording that this technique is in violation of
> those specs and will thus cause problems with compliant
> applications.

Yes, we fully agree with that and AFAIK Ron is working
to address that.

> That said, I'm still very unclear on why ICMP is the
> right protocol to usurp for this purpose

I think I addressed this already above.

What comes to your next message, I think they are good
starting points were we to start making a spec for
ICMPv6.

--Pekka

---

From: Joe Touch <[email-masked]>
Date: Thu, 08 Sep 2005 08:31:04 -0700

> sort-of source-routing fashion.

In this case MPLS is equivalent to a tunneling mechanism; whereas it's
OK to cache (i.e., bury) forwarding steps in a tunnel mechanism, it
necessarily hides it from the IP layer.

> ICMP time exceeded, for example.  The case really is that

At best, it would be appropriate for the tunnel system to signal the
error to the tunnel source (via its own mechanisms), which MAY use that
error to trigger sending an appropriate L3 ICMP message back to the
source, in the spirit of RFC2003 (IPIP tunneling).

> *practise* not much is likely to fail. 

That's a statistical argument based on what subset of applications were
examined; I don't care as much whether it DOES break as whether it CAN,
and when it does, whether the error is detectable. That latter point is
more critical; silently failing applications are a bad thing.

> Hence, while I

That sounds better - I did doublecheck RFC792, which does describe all
reserved fields as set to zero by senders and must be ignored by
receivers, which is a fine way to include new flags. Unfortunately,
there is still the issue of silent failure of legacy systems in those cases.
> to do that large changes to their existing code.

IMO, anyone who cares enough about handling MPLS signals ought to extend
the code to handle that case. Usurping exising code without changing the
semantics means that you're MORE likely to end up with a
silently-failing variant.

> that.  I have no real life data here.

See above, and thus scratch that statement ;-)

> it may be better to change the specs than to expect the

Agreed. But this document isn't described as a change to the specs; it's
described as an extension with backward compatibility.

> starting points were we to start making a spec for

AOK.

Thanks,

Joe

---

From: Pekka Nikander <[email-masked]>
Date: Wed, 14 Sep 2005 15:12:37 +0200

Joe,

>> complicated since there are different uses of MPLS.  In some cases  
>> MPLS is used more-or-less as a link layer protocol, and the  
>> argumentation that appeared here and that you continue holds  
>> pretty well.  However, in other cases MPLS is used more as a  
>> source routing mechanism for IP.  That is, instead of using the  
>> usual hop-by-hop IP routing based on routing tables created by a  
>> routing protocol, the MPLS ingress router (Label Edge Router)  
>> creates an MPLS "tunnel" (Label Switch Path) to the MPLS egress  
>> router, using the information from the very same routing tables.
>>
>> Note that in the latter usage the route that the IP packets take  
>> is exactly the same, using exactly the same routing information.   
>> Hence, there is no difference between plain IP routing and MPLS- 
>> based IP routing. The difference lies in the forwarding plane,  
>> where in plain IP case each box in the path examines the IP header  
>> and makes the forwarding decision based on it while in the MPLS  
>> case the forwarding decision is based on the MPLS label prepended  
>> to the IP packet.  In other words, in the plain IP case the  
>> forwarding decision is done individually at each router (hop-by- 
>> hop forwarding decision) while in the MPLS case the forwarding  
>> decision is effectively done at the ingress router, in a sort-of  
>> source-routing fashion.
>>
>
> In this case MPLS is equivalent to a tunneling mechanism; whereas  
> it's OK to cache (i.e., bury) forwarding steps in a tunnel  
> mechanism, it necessarily hides it from the IP layer.

Well, yes and no.  There are a few things that complicate the situation:

1. MPLS header doesn't have the equivalent of a source address

2. MPLS LSPs can be multipoint-to-point, meaning that later at the  
LSP it may be impossible to determine where the packet has entered  
the LSP.  That is, the packet may have entered at any of a number of  
points.

3. In the cases we consider, the (multipoint-to-point) LSP has been  
created based on the IP routing table information.

4. The MPLS header may be a *stack*, i.e., contain an number of label  
headers stacked.

5. The current MPLS TTL mechanism effectively allows the *IP* TTL to  
be decremented at each LSR.

Considering 2, 3, and 5, the situation resembles more plain IP  
routing than tunnelling.

It is an interesting question whether that has been an  
architecturally sound approach, but that is where we *are* now.

>> So, while in some cases the approach where MPLS is seen as a link  
>> layer and we define new layer 2 specific ICMP messages (or even a  
>> different protocol) seems to be right, in the other cases it  
>> indeed appears to be the right thing to send an ICMP time  
>> exceeded, for example.  The case really is that there is a routing  
>> loop.
>
> At best, it would be appropriate for the tunnel system to signal  
> the error to the tunnel source (via its own mechanisms), which MAY  
> use that error to trigger sending an appropriate L3 ICMP message  
> back to the source, in the spirit of RFC2003 (IPIP tunneling).

AFAICT, that won't work because of 1 and 2 above.  No source address,  
multiple entry points.  No way.

> IMO, anyone who cares enough about handling MPLS signals ought to  
> extend the code to handle that case. Usurping exising code without  
> changing the
> semantics means that you're MORE likely to end up with a silently- 
> failing variant.

Please wait for the new draft.  We are trying to address this, but  
maybe the text in the draft could be tightened up.

> Agreed. But this document isn't described as a change to the specs;  
> it's described as an extension with backward compatibility.

Note taken.  Please see if the new draft addresses this better.

--Pekka Nikander

---

From: Joe Touch <[email-masked]>
Date: Wed, 14 Sep 2005 08:13:45 -0700

Comments below...

Pekka Nikander wrote:
> Joe,
> 
>>> Now, taking a technical step back, the issues is pretty  complicated
>>> since there are different uses of MPLS.  In some cases  MPLS is used
>>> more-or-less as a link layer protocol, and the  argumentation that
>>> appeared here and that you continue holds  pretty well.  However, in
>>> other cases MPLS is used more as a  source routing mechanism for IP. 
>>> That is, instead of using the  usual hop-by-hop IP routing based on
>>> routing tables created by a  routing protocol, the MPLS ingress
>>> router (Label Edge Router)  creates an MPLS "tunnel" (Label Switch
>>> Path) to the MPLS egress  router, using the information from the very
>>> same routing tables.
>>>
>>> Note that in the latter usage the route that the IP packets take  is
>>> exactly the same, using exactly the same routing information.  
>>> based IP routing. The difference lies in the forwarding plane,  where
>>> in plain IP case each box in the path examines the IP header  and
>>> makes the forwarding decision based on it while in the MPLS  case the
>>> forwarding decision is based on the MPLS label prepended  to the IP
>>> decision is done individually at each router (hop-by- hop forwarding
>>> decision) while in the MPLS case the forwarding  decision is
>>> effectively done at the ingress router, in a sort-of  source-routing
>>> fashion.
>>>
>>
>> it's OK to cache (i.e., bury) forwarding steps in a tunnel  mechanism,
>> it necessarily hides it from the IP layer.
> 
> 
> 2. MPLS LSPs can be multipoint-to-point, meaning that later at the  LSP
> it may be impossible to determine where the packet has entered  the
> LSP.  That is, the packet may have entered at any of a number of  points.
> 
> 5. The current MPLS TTL mechanism effectively allows the *IP* TTL to  be
> decremented at each LSR.
> 
> Considering 2, 3, and 5, the situation resembles more plain IP  routing
> than tunnelling.

IMO, it resembles tunneling. Tunnels MAY decrement the TTLs of IP
packets, though most reasonable systems require the decrement (1 or
more) to occur in IP processing at the tunnel entrypoint, to avoid
exactly the kind of confusion that MPLS is causing here.

Most of the issues above, notably 1, 2, and 5, are not IP nor ICMP's
responsibility, but an inadequacy of MPLS.

> It is an interesting question whether that has been an  architecturally
> sound approach, but that is where we *are* now.

Agreed, but where we are now doesn't necessitate usurping ICMP's
functionality in the IP space to support MPLS's inadequacies, IMO.

>>> different protocol) seems to be right, in the other cases it  indeed
>>> appears to be the right thing to send an ICMP time  exceeded, for
>>> example.  The case really is that there is a routing  loop.
>>
>>
>> error to the tunnel source (via its own mechanisms), which MAY  use
>> that error to trigger sending an appropriate L3 ICMP message  back to
>> the source, in the spirit of RFC2003 (IPIP tunneling).
> 

AOK. Again, a failure of MPLS, though.

> Note taken.  Please see if the new draft addresses this better.

Will do - IMO, the approach suggested in my followup message is at least
tolerable, by really supporting backward compatibility while still
trying to retain new capabilities (the one that sends two ICMPs back).

I would still prefer that MPLS fix its own mess (make its own signalling
layer, and fix the source address issue) rather than (expletive
deleted)-ing on our working protocols in ways that break current specs.

Joe