Subject: Arch Draft 06, PHP & LSP [was: Re: [Fwd: Label-based forwarding at R[n]?]]

From: "PAUL NADJ" <[email-masked]>
Date: Fri, 10 Sep 1999 15:11:48 -0400

Okay that's my POV, too, but then I think the definition on an LSP
in the arch-06 doc is misleading/incorrect and needs to be fixed.
Assuming I'm not out to lunch, is that possible or have we already gone
to WG last call?

Quote from 3.16

[Re: PHP]

From an architectural perspective, this is perfectly appropriate.

The purpose of the level m label is to get the packet
to Rn.  Once

R[n-1] has decided to send the packet to Rn, the label
no longer has

any function, and need no longer be carried.

I'd say it's overly simplistic to state "The purpose of the level m
label is to get the packet to Rn".  The purpose of the level m label
is to convey the FEC of the packet to Rn.  This means that Rn may
want to use PHP & do an IP lookup, for example, but it sure isn't going
want that for applications where the label conveys the QOS ...

Quotes from section 3.15:

A)  A "Label Switched Path (LSP) of level m" for a particular packet
P is a sequence of routers, <R1, ..., Rn>

B)  4. For all i, 1<i<n: Ri transmits P to R[i+1] by means
of MPLS, i.e., by using the label at the top of the label stack (the
level m label) as an index into an ILM;

C)   In other words, we can speak of the level m LSP for Packet
P as the sequence of routers:

[...] 3. which ends (at an "LSP Egress") when a forwarding decision
is made by label Switching on a level m-k label, where k>0, or when a forwarding
decision is made by "ordinary", non-MPLS forwarding procedures.

In points A & B, the doc does not say anywhere that Rn may in fact
use the level m label to forward.  In fact, it implies that
it cannot, since R[n-1] is the last R to use the level m label.  (It's
like the doc is saying that PHP must be used).

In point C, the LSP Egress is defined to be the point where switching
is performed on the m-k label.  Therefore if Rn is using the
level m label as per Eric's comments, the LSP really ends on R[n+1].
Kind of weird to say the LSP terminates on an R totally uninvolved in the
setup of the LSP, dontcha think?

It seems as though PHP concerns from 3.16 have bled backwards into 3.15
and corrupted the definition of an LSP.  Comments???  Am I out
to lunch???

Regards,

-- Paul

------------------------------------------------------------

Subject: Re: Label-based forwarding at R[n]?

Date: Fri, 03 Sep 1999 17:07:01 -0400

From: Eric Gray <[email-masked]>

To: Scott Mallonee <[email-masked]>

CC: [email-masked]

References: <[email-masked]>

Scott,

Indirectly, this has already
been discussed.  Some

time back there was a discussion on aggregation of labels

in which at least one person pointed out that the egress

may want to use the same label for all packets that would

be forwarded using the same output interface.  Implied in

this is the notion that any label could be used to decide

an output interface at the egress LSR.

It is a small step from this
to use a label to index

into a table containing L3 and L2 forwarding information -

which would also include output interface - for egressing

labeled packets to L3.  Since the egress determines how its

labels are assigned and interpreted, this is relatively

straight forward - if you want to do it.

One thing that the implementer
must consider is the

possibility that a route would have changed (such that the

packet is now supposed to be output on another interface)

and a new label may not yet have been negotiated with the

upstream peer(s).  Implicit in this consideration is the

perturbation of LSPs - perhaps unnecessarily - as a result

of changing local output interfaces for one or more route

changes.

All of this is implementation
detail.  Use of PHP

allows for one potential optimization in implementation.

There will be others.

You wrote:

> Hi All,

> I am engaged in a discussion that perhaps someone might be kind enough
to comment upon.

> In [ARCH] 2.16 when PHP is discussed:

> > From an architectural perspective, this is perfectly appropriate.

> > The purpose of the level m label is to get the packet to Rn.
Once

> > R[n-1] has decided to send the packet to Rn, the label no longer
has

> > any function, and need no longer be carried.

> > If one does not do this [PHP], then when the LSP egress receives
a packet,

> > it first looks up the top label, and determines as a result of
that

> > lookup that it is indeed the LSP egress.  Then it must pop
the stack,

> > and examine what remains of the packet.  ...  If there
is

> > no other label on the stack, then the packet is forwarded according

> > to its network layer destination address.  Note that this
would

> > require the egress to do TWO lookups, either two label lookups
or a

> > label lookup followed by an address lookup.

> My question is this: if m=1, why couldn't R[n] forgo PHP and use
the layer m label to

> index non-MPLS forwarding instructions? Is this a valid implementation
option? Is there a

> protocol-based hard requirement that packet be cracked in this case
and conventional

> forwarding be performed?

> Best Regards,

> Scott

> --

> +--------------------------------------------------------------------+

> | Scott Mallonee
(540)822-4432 |

> | Newbridge Networks Incorporated
(703)736-5982 |

> +--------------------------------------------------------------------+

------------------------------------------------------------

Eric W Gray <[email-masked]>

Bell Laboratories, Lucent Technologies, Inc.

PacketStar IP Switch Development

Eric W Gray

Bell Laboratories, Lucent Technologies, Inc.  <[email-masked]>

PacketStar IP Switch Development

PO Box 0710
Work: 603 659 3386

Durham
Conference Software Address

New Hampshire

03824-0710

US

Additional Information:

Last Name     Gray

First Name    Eric W

Version       2.1

---

From: "Navin kumar Agrawal" <[email-masked]>
Date: Sat, 11 Sep 1999 13:23:38 +0530

Hi Paul,

Please find below my comments.

> Okay that's my POV, too, but then I think the definition on an LSP in
> the arch-06 doc is misleading/incorrect and needs to be fixed.
> Assuming I'm not out to lunch, is that possible or have we already
> gone to WG last call?
>
> Quote from 3.16
>
>    [Re: PHP]
>    The purpose of the level m label is to get the packet to Rn.  Once
> is to convey the FEC of the packet to Rn.  This means that Rn may want
> to use PHP & do an IP lookup, for example, but it sure isn't going

When you are implementing Traffic Control mechanism (e.g. CBQ) for
providing QoS to labeled packets, the mechanism uses label for
classification
of the **outgoing** labeled packets to different QoS queues, not for the

incoming labeled packets.  Infact, if you are putting your traffic
control
mechanism in kernel properly, it would not come in the receive path at
all.

So from the QoS provisioning point of view also, it would
make no difference if Rn node is receiving labeled packet or unlabeled
packet. So the recommendation to implement PHP is correct in all sense.

Please correct me in case of disagreement.

Regards,
Navin

> A)  A "Label Switched Path (LSP) of level m" for a particular packet P
> is a sequence of routers, <R1, ..., Rn>
>
> B)  4. For all i, 1<i<n: Ri transmits P to R[i+1] by means of MPLS,
> i.e., by using the label at the top of the label stack (the  level m
> label) as an index into an ILM;
>
> C)   In other words, we can speak of the level m LSP for Packet P as
> the sequence of routers:
> is made by label Switching on a level m-k label, where k>0, or when a
> forwarding decision is made by "ordinary", non-MPLS forwarding
> procedures.
>
> In points A & B, the doc does not say anywhere that Rn may in fact use
> the level m label to forward.  In fact, it implies that it cannot,
> since R[n-1] is the last R to use the level m label.  (It's like the
> doc is saying that PHP must be used).
>
> is performed on the m-k label.  Therefore if Rn is using the level m
> label as per Eric's comments, the LSP really ends on R[n+1].  Kind of
> weird to say the LSP terminates on an R totally uninvolved in the
> It seems as though PHP concerns from 3.16 have bled backwards into
> 3.15 and corrupted the definition of an LSP.  Comments???  Am I out to
> lunch???
>
> Regards,
> -- Paul
>
>
>
>>         Indirectly, this has already been discussed.  Some
>>         It is a small step from this to use a label to index
>>         One thing that the implementer must consider is the
>>         All of this is implementation detail.  Use of PHP
>> > I am engaged in a discussion that perhaps someone might be kind
>> enough to comment upon.
>> >
>> > > lookup that it is indeed the LSP egress.  Then it must pop the
>> stack,
>> > > and examine what remains of the packet.  ...  If there is
>> > > to its network layer destination address.  Note that this would
>> > index non-MPLS forwarding instructions? Is this a valid
>> implementation option? Is there a
>> > protocol-based hard requirement that packet be cracked in this
>> case and conventional
>> > forwarding be performed?
>> >
>> > Best Regards,
>> >
>> > Scott
>> > --
>> >
>>   PO Box 0710                                   Work: 603 659 3386
>>   Durham                                        Conference Software
>> Address
>>   New Hampshire
>>   03824-0710
>>   US
>>   Additional Information:
>>   Last Name     Gray
>>   First Name    Eric W
>>   Version       2.1
>

---

From: Eric Gray <[email-masked]>
Date: Sun, 12 Sep 1999 09:57:45 -0400

Paul,

	I believe that yours is one possible interpretation
of what is trying to be said in the specifications.  I also
suspect that you may have missed the point that the down-
stream LSR gets to 'decide' what labels it assigns.  This
extends to use of PHP since the downstream LSR must signal
that this is what it wants - hence the choice is not up to
the upstream LSR - explicitly via the label it assigns.  An
upstream LSR otherwise cannot assume that it is the PH and
decide to pop the remaining label on its own.

	The egress LSR signals the intent to have the PH do
PHP by sending the 'Implicit Null Label' value (3) in Label
Mappings.  Unless an LSR receives this label, it should not 
pop the label before forwarding packets downstream.  See the
draft 'draft-ietf-mpls-label-encaps-06.txt'.  It would, of
course be ridiculous to assert the the downstream LSR means
to use the 'Implicit Null Label' to select an output port;
for one thing, this label will not be used in encapsulating
MPLS packets and for another it would not be very distinct
if misused in this way.

	This is part of the issue I was getting at sometime 
back when I pointed out that [ENCAPS] was possibly not the 
best place to define the Implicit Null Label, since it is 
never actually used in encapsulation (which is explicitly 
stated in [ENCAPS]).  In fact, it is used only in signaling.
For example, see [LDP].

	I believe that there should be a reference to the 
definition of the reserved Label values defined in [ENCAPS] 
in [ARCH].  It might also be useful to include a clearer 
discussion of how PHP is negotiated in [ARCH] as well.

> want that for applications where the label conveys the QOS ...

The 'hidden assumption' is that - when PHP might be negotiated - the
purpose of the label would otherwise be just as the above indicates.
This would be clearer if the above were re-worded as:

  "From an architectural perspective, this is perfectly appropriate.
   If the purpose of the level m label is to get the packet to R[n], 
   then - once R[n-1] has decided to send the packet to R[n] - the
   label no longer has any function, and need no longer be carried."

I'm not sure, but this version may cause some document editors to
put yellow squiggly lines under the resulting moderately long line.

> doc is saying that PHP must be used).

See above.  In this case, if the egress wants to receive a label, it
simply gives the PH a label with a value other than 'Implicit Null'.

> setup of the LSP, dontcha think?

Another point made previously; when PHP is used, the so-called "Egress"
LSR is effectively not part of the LSP (see discussion some time back
between Bob Thomas, Yoshihiro Ohba and myself on the issue of sending
a 'Withdraw Label' for the 'Implicit Null Label' in [LDP]).  It is even
possible to put LDP signaling in a 'box' such that it is technically 
an 'LSR', even if it has absolutely no label switching hardware, by 
always sending the 'Implicit Null Label'.  This is the 'cheap box'
implementation I was afraid would be allowed to occur as a result of
PHP in the first place, and why I objected to PHP some two years ago.

Any time that an LSR terminates an LSP at an output port, it effectively
uses an analogue (possibly an exact analogue) of PHP procedures.  Fact
is, one simple way to deal with terminating at an output port is to act
es if you had received an 'Implicit Null Label' from the (possibly non-
existant) downstream LSR peer.

> 3.15 and corrupted the definition of an LSP.  Comments???  Am I out to

I think you've pointed out some additional areas in which it is not
currently clear (at least in [ARCH]) how PHP is supposed to work...

---

From: Eric Gray <[email-masked]>
Date: Sun, 12 Sep 1999 10:06:51 -0400

Navin,

	So you won't be offended, you did ask.

	You are correct that in practically all cases, QoS
will be a characteristic of an output queue.  However, as I
understand Paul's point, an LSR may want to use an incoming
label to select an output queue - even if it is the egress.
The fact that the output queue is not for MPLS is irrelevant.
Hence, it is just as important that it be allowed to use a
label to do this (which IS the case - as explained in other
response), since QoS 'assurance' is only as good as the QoS
that is assured at every node (including ingress and egress).

You wrote:
> > want to use PHP & do an IP lookup, for example, but it sure isn't
> > going want that for applications where the label conveys the QOS ...
> 
> >   [...] 3. which ends (at an "LSP Egress") when a forwarding
> > decision is made by label Switching on a level m-k label, where k>0,
> > or when a forwarding decision is made by "ordinary", non-MPLS
> > forwarding procedures.
> >
> > use the level m label to forward.  In fact, it implies that it
> > cannot, since R[n-1] is the last R to use the level m label.  (It's
> > In point C, the LSP Egress is defined to be the point where
> > switching is performed on the m-k label.  Therefore if Rn is using
> > the level m label as per Eric's comments, the LSP really ends on
> > R[n+1].  Kind of weird to say the LSP terminates on an R totally
> > uninvolved in the setup of the LSP, dontcha think?
> >
> > 3.15 and corrupted the definition of an LSP.  Comments???  Am I out
> > to lunch???
> >
> > Regards,
> > -- Paul
> >
> >
> >
> >> > > R[n-1] has decided to send the packet to Rn, the label no
> >> longer has
> >> > > If one does not do this [PHP], then when the LSP egress
> >> receives a packet,

---

From: "PAUL NADJ" <[email-masked]>
Date: Mon, 13 Sep 1999 12:21:37 -0400

Navin,

Eric's response below is pretty much what I'm getting at.  MPLS is akin to
switching.  Typically, one does not switch from a given QoS to another in the
core of a network.  The same can be said of the innards of an LSR.

One is going to want to send the packets into the switching core of a router
with the correct priority so that congestion in the core can be handled
intelligently.  Some architectures may want to derive the QoS of the incoming
packet from the the MPLS label, much like a FR DLC or an ATM VC will derive the
QoS from the incoming DLCI or VPI/VCI.

Yeah, one may perform some kind of label merge at the egress line card using a
QoS for transmission that is a property of the egress label.  Presumably,
however, this will be the same QoS as the packet had in your router's switching
core, on the ingress line card (if there was any buffering there), and in the
upstream routers.  If you're not guaranteeing QoS throughout the entire network
then there's not really much point to it.

Thus my beef with the description of what a label is for.  "The purpose of the
level m label is to get the packet to Rn" seems somewhat simplistic.  Since
labels are downstream assigned, the idea is not only to get the packet to the
downstream router -- one label per interface or one label per router would
suffice for that.  The idea is to convey some useful information about the
packet (i.e., the FEC) to the downstream router.   (The information being
'interpretable' only by the downstream router, of course).

Regards,
-- Paul

---

From: "PAUL NADJ" <[email-masked]>
Date: Mon, 13 Sep 1999 12:31:03 -0400

Please see short comment below,
Thanks,
-- Paul

> decide to pop the remaining label on its own.

Sure.  My beef is the definition of an LSP which says:

- The LSP includes routers R1 .. Rn (stated)
- R[n-1] is the last router to use the label to forward (very strongly
implied).

If PHP is not used, Rn may use the label to make a forwarding decision.  This
is why it looks like some of the concerns of PHP have bled backwards from
section 3.16 into section 3.15.

For me, this seems to confuse the definition of just where an LSP terminates
(Rn, or R[n-1]?)   The terms are kind of important when trying to discuss
behaviour at the boundary points.  That's actually what got me started on
this:  a bit of confusion with terminology when I was chatting with others.

> discussion of how PHP is negotiated in [ARCH] as well.

Agreed.

---

From: Eric Rosen <[email-masked]>
Date: Mon, 13 Sep 1999 14:03:41 -0400

Paul> The idea is to convey  some useful information about the packet (i.e.,
Paul> the   FEC)  to   the  downstream   router.   (The   information  being
Paul> 'interpretable' only by the downstream router, of course).

No, this is not the idea.

Paul> Since labels are downstream assigned, the  idea is not only to get the
Paul> packet  to the downstream  router --  one label  per interface  or one
Paul> label per router would suffice for that.

If all LSRs  know, for each packet, the identity of  the egress router, then
there is nothing to stop them from  using one label for all packets with the
same egress router.  See the section on egress-targeted label assignment. 

Paul> My beef is the definition of an LSP which says:

Paul> - The LSP includes routers R1 .. Rn (stated)
Paul> - R[n-1] is the last router to use the label to forward (very strongly
Paul>   implied).

Paul> If  PHP is  not  used,  Rn may  use  the label  to  make a  forwarding
Paul> decision. 

No it may  not.  More precisely, if  Rn uses the label to  make a forwarding
decision,  then  Rn is  not  the  egress router,  and  the  LSP includes  an
additional router after Rn. 

Paul> Therefore  if Rn  is using  the  level m  label as  per Eric  [Gray]'s
Paul> comments, the LSP really ends on R[n+1].

Correct. 

Paul> Kind of weird to say the  LSP terminates on an R totally uninvolved in
Paul> the setup of the LSP, dontcha think?

So your beef  is that the definitions seem "kind  of weird"?  The definition
has to do with  the forwarding plane not the control plane,  so the issue of
"setup" isn't really relevant.

Look at it this way.  With the existing definition, R[i] and R[i+1] are in a
particular packet's level m  LSP if and only if R[i] uses  the level m label
as the  basis for  forwarding the  packet to R[i+1].   That doesn't  seem so
weird, though of course it is not the only possible way the terms could have
been  defined.  Of  course,  however one  defines  the terms,  it is  always
possible to define them differently.

---

From: Eric Gray <[email-masked]>
Date: Mon, 13 Sep 1999 14:24:24 -0400

Paul,

	Two short responses...

> - R[n-1] is the last router to use the label to forward (very strongly

I agree; the next question is - what to do about it?

I believe that we do not need to do anything about the
'strength of implication'.  This is a good thing, as
it is hard to word-smith something to satisfy everyone
on such a hard to quantify and subjective evaluation.
Doing anything about this is unnecessary for the simple
reason that an implication holds no force - regardless
of its strength (try to sue someone for an implied
insult :-)...

If there are explicit statements - such as the one you
pointed out - that are wrong, then of course that is 
different.

> this:  a bit of confusion with terminology when I was chatting with others.

Again I agree that it confuses the definition of boundary
points.  Unfortunately, this is a very confusing definition
under the best of conditions.  I would think it better if
we try not to dwell on the necessity of defining boundary 
points in excruciating detail and to everyone's complete
satisfaction - and, instead, let individual vendors work
with individual customers to develop a working definition
of the real 'boxes' needed to deploy a solution in a real
network.

In this case, what we need to be able to understand is that
some boxes may want to have their predecessor in an LSP pop
a label.  This may allow them to perform operations on the
remaining packet in a perhaps more efficient way and costs
almost nothing at the predecessor.  Other boxes may not
want this.  Conversely, some boxes may not be willing and
able to pop the label in this fashion (at present, ATM and
Frame Relay boxes are assumed to be unable/unwilling while 
others are assumed to be able/willing).  Since the egress 
box is in control of the label distribution process with 
its upstream predecessor, it all works out somewhat nicely.

> > ...

---

From: Eric Gray <[email-masked]>
Date: Mon, 13 Sep 1999 14:38:37 -0400

Eric,

	Oh, dear - whatever am I to do?  It seems as if
I have to disagree with you again :-)

	The definition you give is just not as simple as
you make it out to be.  Sorry about that.

	The 'complexity' not addressed in this definition
was the point of the discussion (I mentioned earlier) in
LDP (between Bob Thomas, Yoshihiro Ohba and myself) - it
makes a difference whether R[i] and R[i+1] are in the
same MPLS domain (or whether or not R[i+1] is in any MPLS
domain at all when it comes to control plane operations.
Definitions that define an LSR as part of an LSP for data
plane operations and not for control plane operations are
'weird'.

---

From: "Navin kumar Agrawal" <[email-masked]>
Date: Tue, 14 Sep 1999 19:57:29 +0530

Hi,

>         See below.
> > Hi Eric,
> > Only, If I am proven wrong.
> >
> > Do you mean to say that at the egress node (which is also the destination
> > node) also there will be some kind of QoS assurance provisioning? I
> > don't agree with this. Please throw some more light in this regard before
> > I put thought  across.
>
> Only consider this: the reasons why output queuing
> makes sense in a 'boxed' system should apply to an
> 'out-of-box' system as well.  In other words, it
> might be an incorrect implementation which commits
> to delivering packets at - say - 100 Mbits/sec if
> it's queuing mechanism is at its output interface
> and it does not perform CAC for that interface.  A
> system that does not perform CAC at its output may
> result in packets being delivered all the way along
> an LSP only to be dropped at the egress.
>
> That strikes me as a useful definition of how to
> waste resources.  Suppose that I have an LSP which
> consists of - say - 5 nodes (ingress, intermediate
> 1, intermediate 2, intermediate 3 and egress).
> Assume I setup this LSP as described above - with a
> capacity of 100 Mbits/sec.  Now suppose that - for
> some non-zero length of time - the egress only has
> output queuing capacity for this LSP at the rate of
> 10 Mbits per second.  Indeed, if absolutely no CAC
> is run using the output interface, this might be all
> the capacity that the egress ever expects to obtain
> using this interface (if - for example - it is a 10
> Base-T interface).  In any case, for some period of
> time, packets may or may not be delivered along the
> LSP at the maximum rate of 100 Mbits/sec only to be
> dropped if in excess of 10 Mbits/sec at the egress.
> This means that 90 Mbits/sec of reservable resources
> are not available at the ingress, and 3 intermediate
> LSRs - either because they are in use, or because
> they cannot be reserved for another use.  If this is
> the general way in which people have implemented LSR
> boxes at the edge of the network, then there will be
> a general lack of reservable bandwidth in a highly
> under-utilized network.  I could imagine seeing 20
> percent utilization with as little as 8 percent of
> all reservation (bandwidth allocation) requests
> being satisfied under circumstances in which I would
> expect to see 100 percent utilization with 80 percent
> satisfaction.
>
> Not the way I would choose to spend my money.

When you are saying 'egress node'...Why there would be any output
queueing?...And when there is no queueing...there won't be any need
of resource reservation...agreed?

Regards,
Navin