Subject: RE: Alternative MPLS Label Stack Encoding

From: Bora Akyol <[email-masked]>
Date: Tue, 5 Oct 1999 12:48:07 -0700

Micheal

The changes you are proposing are difficult to implement for hardware based
wire-speed routers. I believe that we have a working (maybe inelegant ;-)
label encapsulation, I personally don't see the point of messing with it at
this time.

I believe that there is more important issues to resolve such as:

1) MPLS based Path Restoration instead of using APS.

2) Dynamic on-line traffic engineering instead of using ad hoc offline tools
(one can dream, can't he?)

3) A definition of QoS for MPLS that is a lot more clearer and implementable
across a wide network than what is pseudo-specified in the RSVP Tunnel or
CR-LDP drafts (here comes the flames !)

4) Extending MPLS signaling protocols to operate across administrative
domains, so that we don't have to strip packets and re-encapsulate them at
NAPs, etc.

Regards

Bora Akyol
http://www.pluris.com

Maybe we should spend our time on so

-----Original Message-----
From: Michael Stefan Cox [mailto:[email-masked]]
Sent: Wednesday, September 29, 1999 6:07 PM
To: [email-masked]
Cc: Peter Ashwood-Smith; MPLS Mailing List
Subject: Re: Alternative MPLS Label Stack Encoding

Eric,
	as you have noted, I do not seriousely expect people to immediately
scrap
the present encapsulation. Rather I set out to do exactly what you have
identified below: To propose a label encoding mechanism that can be made
fully compatible with the present encapsulation and to demonstrate that it
can be made much more elegant with a minor redesign. Of course this was done
with the ultimate intention that future definitions may migrate towards it.

BTW the floor plan may have changed, but the basement is the same!
Firmware/Software would require trivial changes to support the bits in
different locations and I have not proposed any changes to the upper most
label when it is mapped onto existing protocol stacks such as ATM. I would
be _very_ surprised if any vendor has irrevocably committed themselves (via
fixed hardware) to reading the MPLS label stack bits as currently defined.
The proposed alternative encoding would allow vendors to go to market with
extremely powerful differentiators that permit new IP-MPLS services beyond
mapping an IP forwarding equivalence and explicit route traffic engineering.

Eric Gray wrote:
> 
> Michael,
> 
>         It's always so amusing that someone is certain to
> come along with a new floor plan right after the basement
> has been poured. ;-)
> 
>         I believe the point is that you can do everything
> you propose with the current labeling scheme.  Perhaps it
> is not as elegant as it could be.  Perhaps it is not what
> many of us would have wanted had we set out to design it
> in a vacuum.  Certainly we will uncover shortcomings in
> using it.  And perhaps the next version will be more like
> you propose.
> 
>         But surely you don't expect people to seriously
> consider scrapping the present encapsulation at this late
> date, do you?
> 
> You wrote:
> >
> > Peter,
> >         I am glad to see you agree with my fundamental conclusions.
> >
> > The proposed alternative label stack encoding provides improved
semantics to
> > aid in traffic analysis, debugging and fault identification. It
maximises
> > the bit field utilisation and provides an optimal encoding in support of
> > Differentiated Services Codepoints and Layer-3 address extension. It
> > simultaneousely supports a path oriented system (with extended state
> > maintained in every intermediate node) and connectionless systems (state
> > required only at the edge nodes) with self describing headers for
multiple
> > CoS per LSP and VPNs. And all without suffering any of the limitations
of
> > ATM technologies oand avoiding enourmous complexities that arise when
trying
> > to map 8 bit DS codepoints into 3 bit EXP fields and vice-versa.
> >
> > Peter Ashwood-Smith wrote:
> > >
> > > > All,
> > > >         I have some concerns with the current label stack encoding
that I believe
> > > > are very significant. Although the modifications proposed within
this email
> > > > are relatively minor with respect to HW/SW implementations they
potentially
> > > > represent a structural change to the bit ordering. I believe these
> > > > modifications are necessary to facilitate several advanced services
and
> > > > capabilities that are primarily market oriented and are not easily
> > > > implemented using any other mechanism. MPLS is becoming a vital
strategic
> > > > technology in the general marketplace and as such it would be
devastating to
> > > > the industry to define a label encoding that falls short of its
unique
> > > > potential.
> > > .....
> > > > First Proposal:
> > > >         The proposal is to define a number of Label types which are
distinguished
> > > > by the coding of the EXP field. The proposed encodings are:
> > > >                 EXP = xx0, Label Type 1 : VPN Identifier.
> > >
> > >   Michael,
> > >
> > >   You have obviously given some serious thought to these issues but I
think you
> > > are missing one important piece of the picture. This may help.
> > >
> > >   One thing to keep in mind is that in a path oriented routing system
the headers
> > > are not self describing. That is, you cannot tell exactly what to do
with the packet
> > > simply from the header. There is context that is associated with the
connection that
> > > was created during the setup phase, which was remembered (hopefully
permanently ;)
> > > and which can be referenced during switching/forwarding operations.
> >
> > This has be catered for via the Type 2 label.
> >
> > I was aware of this fact.
> >
> > >   For example, if we establish an LSP using say VCI for the primary
label and
> > > then a shim for the second label, and if that second label is to
correspond to a
> > > VPN of some kind, then since WE allocated the second label for that
purpose, WE know it
> > > is to be used for a VPN, which VPN etc. So, when we terminate the AAL5
ATM part
> > > of the path, then do the second label lookup, the entry in our table
tells us all
> > > about the VPN, what we are to do with it etc. There is no ambiguity.
> >
> > This has be catered for via the Type 1 label.
> >
> > Alternatively a Type 4 label could be used if you desire the extended
state
> > requirement in the edge nodes. The use of a Type 2 or 3 label would be
> > sub-optimal as the TTL or DS-Codepoint bits have no relevance.
> >
> > >   The other issue is that the EXP bits are almost certainly going to
be used for
> > > some kind of mapping to allow different per hop behaviors for packets
in the
> > > same LSP. Something which is unfortunately lacking in ATM. I know this
goes against
> > > the idea of not being self describing but in this case the benefits of
not having
> > > to allocate multiple identically routed connections outweighs the
slight violation
> > > of the concept.
> >
> > This has been catered for by the Type 3 label.
> >
> > As you have correctly noted, I have defined the EXP bits to allow
> > overloading of the TTL field to represent a complete DS Codepoint field
for
> > implementing multiple CoS per LSP. This elegently side-steps the need to
> > define complex, and potentially incomplete mappings of 8 bit PHBs to 3
bit
> > PHBs and guarantees complete transparency and predictability when
> > implementing QoS on the Internet and within MPSL domains.
> >
> > >   Peter Ashwood-Smith
> >
> > The type 4 label is by far the most useful. This label enables advanced
> > solutions to several unresolved issues inherent within connectionless
> > networks.
> >
> > regards,
> > --
> > Michael Stefan Cox                              Switch: +61 2 9699 0044
> > Principal  Engineer               V            Direct:  +61 2 9690 5295
> > Networks and Systems        A L C A T E L     Fax:      +61 2 9690 5247
> > Design  and  Strategy                        Mobile:    +61 418 226 108
> > mailto:[email-masked]   surfto:<http://www.alcatel.com.au/>

-- 
Michael Stefan Cox                              Switch: +61 2 9699 0044
Principal  Engineer               V            Direct:  +61 2 9690 5295
Networks and Systems        A L C A T E L     Fax:      +61 2 9690 5247
Design  and  Strategy                        Mobile:    +61 418 226 108
mailto:[email-masked]   surfto:<http://www.alcatel.com.au/>

---

From: Michael Stefan Cox <[email-masked]>
Date: Mon, 18 Oct 1999 18:35:11 +1000

Bora,
	Comments are inline...

> label encapsulation, I personally don't see the point of messing with it at

Not at all. As detailed below only Type 2 (IP-FEC) and Type 3 (Diffserv
Traffic Engineering) labels were intended for hardware switching and the
additional complexity there is almost nil. In fact the Type 2 labels are
identical to your current encoding and the Type 3 are simpler (read-only as
opposed to read-modify-write). The Type 1 and 4 labels are intended
initially for software based complex edge services, however they are still
considered _part_ of the MPLS label stack. I expect the Type 5 labels may be
defined to implement SLA verification systems (and IPPM functions) which
require hardware support, however they likely will have fixed
interpretations making them easy to support as well.

> 1) MPLS based Path Restoration instead of using APS.

I agree. But a solution based on an in-elegant MPLS label stack is likely to
be even more in-elegant. In contrast a Type 3 based multi-path, traffic
engineering system provides an elegant solution for this.

> (one can dream, can't he?)

I also agree. This is a high class problem, but I believe dreaming is no
longer neccessary. The problem becomes more one of stability v's
sensitivity. Heuristic and logarithmic state dampening can work wonders here
and of course support for Diffserv codepoints is almost mandatory (Type 3).

> CR-LDP drafts (here comes the flames !)

I agree. Clear support for Diffserv codepoints was one of the crucial
requirements taken into account when proposing the modifications.

> domains, so that we don't have to strip packets and re-encapsulate them at

I agree that this will be required in the long term, however we can make do
in the short term.

> http://www.pluris.com

MPLS is much more than just IP over ATM.

Many consider MPLS to simply be a layer-2 only technology acting as a
mechanism to integrate IP and ATM in the short term and potentially
replacing ATM and Frame Relay in the longer term. In short a frame based
ATM. Others see MPLS as a mechanism for improving IP forwarding performance
by replacing multiple IP route entries with a single 20 bit FEC label. I see
MPLS as both and more!

MPLS is a stackable shim technology sitting between the framing services of
layer-2 and the end-point addressing services of layer-3. It is this
positioning of MPLS that allows it to simultaneously adopt many of the
traditional layer-2 and layer-3 services and offer both from the same
technology in a simple and elegant manner. This unique ability allows MPLS
to offer _new_ services to providers and customers that are either
impossible or difficult to offer today. As John W Stewart (Juniper) says :
"it (MPLS) needs to offer a significant advantage over what is available. So
we need to focus on very concrete things that MPLS does that are completely
new or that it does significantly better". i.e. If we really want to see
MPLS be successful in the market place we MUST offer new or improved
services, it is not justification enough to say it is closer to IP than ATM
for instance.

Illustration of proposed new architecture for MPLS: (Multiple Layer-2,
Layer-3 functionality):

Diagram 1:        ____________________________________
                 /          MPLS label stack          \
                /                                      \
+----+---------+-------+-------+--------+-------+-------+---//-----------+
| L1 | Framing | Label | Label |   ...  | Label | Label |  // IP Payload |
+----+---------+-------+-------+--------+-------+-------+-//-------------+
                 L1      L2              Ln-1    Ln

Irrespective of which underlying transport you intend to operate IP over
there will be a need to a framing technology. This can be provided via
technologies such as HDLC (for serial connections), PPP (PoS, ...), Ethernet
(LANs) and ATM for instance. MPLS is being defined to operate over each of
these respective framing technologies.

In diagram 1 above it can be seen that MPLS is defined to implement a stack
of labels in between the layer-2 framing as discussed above and (currently)
IP as the layer-3. It is not difficult to envision the leftmost labels of
the label stack as consisting of a number of switching labels that represent
an end-to-end (uni-directional) path across some MPLS domain (be it multiple
AS or single AS). It should also not be difficult to envision the rightmost
labels of the label stack as being capable of extending information in the
IP header. For instance carrying AS specific information (security,
clearance etc), traffic ingress identifiers, Dynamic Packet State
(draft-stoica-diffserv-dps-00.txt) information and many other possibilities.
Another obvious layer-3 extension advantage is the VPN identifier. It is
well understood that IPv4 lacks sufficient bits to address every individual
on the globe (approx 6 billion), however if we are able to prepend a
VPN_identifier of say 30 bits to an IP packet we can increase this
addressability to approx 4x10^18 (although in reality heirarchical
allocation of the VPN_id will be used, reducing this number). This not only
provides immediate IP-VPN support and scalability today, but it also
provides a potential migrationary path to IPv6 long term.

In the alternative MPLS label stack encoding I proposed, several label types
were defined via encoding of the EXP field. These have been repeated in this
email below for clarity. I have received feedback indicating that the label
types proposed, in particular the layer-3 extenders, were too complex to
implement in hardware. Whilst I understand this issue, I believe some points
have been missed. As indicated in diagram 2, a typical end-to-end scenario
will involve end users communicating via an edge service aggregation node
through a traffic engineered core to the far end edge and user. In this
context the Type 2 (IP-FEC) and/or Type 3 (ER-TE) labels are intended to be
the outermost labels of the MPLS label stack and can easily be switched in
hardware (equivalent to current label encodings). These labels are expected
to be pushed onto the stack by the edge nodes and used only by the core
nodes. The Type 1 VPN label is likely to only have significance between the
two edge nodes, it typically will be stacked closer to the IP header and
below the switching labels discussed above. This label can be processed via
software and/or hardware (can be viewed as creating an aggregated 62bit
VPN_IPv4 address for hardware forwarding lookups). Likewise the Type 4 label
was never intended to be 'switched' per se, rather it was seen as a
mechanism for encoding packet related information on per AS/path/flow basis
that may be operated on by hardware or software implementations at the edge
of the domain and ultimately by the users for future services.

Diagram 2:

   User_A --> Edge_1 --> CORE_1 // T.E. // CORE_2 --> Edge_2 --> User_B

Diagram 3:

A complete example MPLS label stack and its possible interpretation:

+--------------------+
|    PoS Framing     |   This layer provides the framing adaptation.
+--------------------+
|      TE Label      |   This label defines a traffic engineered path.
+--------------------+
| Edge to Edge Label |   This label defines an path to the BGP next hop.
+--------------------+
|   AS to AS Label   |   This label may define an AS-AS-AS switched path.
+--------------------+
|   Traffic Source   |   This label may identify the traffic source.
+--------------------+
|   VPN Identifier   |   This label extends the IP Src/Dst addresses.
+--------------------+
| Dynamic Flow State |   This label may be used to provide flow state.
+--------------------+
|    IP Src/Dst      |   IP source and destination end points.
+--------------------+
|      Payload       |   Application Data
|        ...         |
+--------------------+

---------------------------------------------------------------------------
Proposed label encoding.

Type 1 = VPN identifier:
        When the least significant bit of the EXP field is a 0 then the
label
encodes a Virtual Private Network Identifier (VPN_id). The most significant
22 bits (VPNid_high) and the least significant 8 bits (VPNid_low) are
encoded separately to define a 30 bit total VPN_id when concatenated
together.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |                 VPNid_high                |0|S|   VPNid_low   | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

Type 2 = IP-FEC label (original):
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |                Label                  |0 0 1|S|       TTL     | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

Type 3 = ER-TE label (diffserv):
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |                Label                  |0 1 1|S|  DS Codepoint | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

Type 4 = AS specific header extender:
        This label type can be used to arbitrarily colour, mark or
differentiate
packets on a per AS basis. It creates the capability to extend (as many
times as desired) the Layer-3 header in an arbitrary manner.
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |              AS specific              |1 0 1|S|  AS Specific  | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

Type 5 = Reserved:
        This label type is reserved for future definition.
   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Label
  |                reserved               |1 1 1|S|    reserved   | Stack
  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Entry

---------------------------------------------------------------------------

regards,

---

From: Grenville Armitage <[email-masked]>
Date: Mon, 18 Oct 1999 12:08:27 -0700

Michael Stefan Cox wrote:
	[..]
> MPLS is much more than just IP over ATM.
	[..]
> layer-2 and the end-point addressing services of layer-3.

Michael,

The fact that MPLS allows label entries to be stacked seems to
be a limited justification for adding pre-defined semantics
to particular 'types' of stack entries.  One of the values of MPLS
is to compress semantic content into a few label fields. The
signaling (or provisioning) process is intended to associate
things like VPN_IDs, etc to particular LSPs. What your scheme
does is to un-compress such information into an a-priori defined
in-band stack structure. I can't see how this is efficient.

cheers,
gja

---

From: Michael Stefan Cox <[email-masked]>
Date: Tue, 19 Oct 1999 09:40:04 +1000

Grenville,
	one of the well understood problems with dedicated layer-2 path descriptor
systems is that core nodes need to maintain additional state information
beyond simple label swap entries. Yes I have increased the complexity of the
MPLS label stack, however this was in order to reduce the amount of state
complexity in core nodes. This simple complexity shift will significantly
improve the scalability of MPLS long term, beyond competing technologies
such as ATM and FR.

Additionally the alternative label stack proposal gives vendors and
customers the choice of which mechanisms they wish use. They can choose to
compress the information into a small set of switching labels (storing the
additional state in the intermediate switching nodes) AND/OR they can also
choose to include the self describing layer-3 extender labels (which can
carry state with the PDU). 
Where and how these labels will be used is entirely flexible. Vendors and
providers can choose which label types they want supported in which network
nodes to provide which services. This allows diverse MPLS offerings from
vendors without sacrificing interoperability.

Differentiation with compatibility is a win-win situation for service
providers and vendors alike.

Likewise this will end up as a win-win for MPLS technology as well.

---

From: Jeremy Lawrence <[email-masked]>
Date: Tue, 19 Oct 1999 14:11:54 +1000

At 12:08 10/18/1999 -0700, Grenville Armitage wrote:
>[...]

Agreed. To give a different perspective:

All advantages of MPLS stem, at least in part, from its
identification and definition of a forwarding function which is
separate from, and independent of, network-layer protocols, routing
(i.e. route determination) and signalling.

The proposed scheme would introduce, in the forwarding mechanism,
semantics specific to particular set of routing & signalling
mechanisms required for a specific application. This would break
the separation that MPLS introduces, which would be a major
disadvantage. Since everything that this scheme achieves can be
achieved by other (fairly straightforward) means, this major
disadvantage is not justified.

Jeremy Lawrence

---

From: Grenville Armitage <[email-masked]>
Date: Mon, 18 Oct 1999 22:32:10 -0700

Michael Stefan Cox wrote:
> beyond simple label swap entries.

Okay, I'm probably being a little dense right now, but what's
the difference in 'state' that a core (or transit, in this case)
node would have to carry between existing MPLS and your scheme?

It strikes me that the _amount_ of state is roughly dependent
on the complexity of service, not the header formats. _Where_ the
state is held depends on the manner in which per-frame header
information is used to drive frame forwarding decisions. The
more connection-less approaches require more state to sit in
the 'fast path' since it is referenced per-frame. Connection-oriented
approaches can push more state into the control-plane storage,
since its only required during path establishment. Once the path
is established, less state is required in the frame processing
fast path. This last point is typically considered one of the
reasons MPLS is the way it is.

So my problem is that I dont see that the nett amount of state
is really changed by your scheme. You're simply migrating some
of it from the slow-path control plane back into the fast-path
frame processing plane. What am I missing here?

cheers,
gja