Subject: Is PWE a layer - thread summary

From: Mark Seery <[email-masked]>
Date: Thu, 29 May 2003 13:13:19 -0700 (PDT)

Felt it might be useful to summarize some of the
issues/thoughts raised on this thread.

Mark

The thread started with an observance that is easy for
a protocol to creep from an in-between layer to a full
layer, siting the history of MPLS that was first
proposed as a simple shim; it was further noted that
when this occurs then formal treatment to control and
management is required. The inference then is that we
as a group should be concerned about actions we take
which make lead us down the road of developing a new
layer.

Issue 1. Is it the intention that PWE be just an
adaptation function?

The suggestion was then made that if we stick to
network interworking any client OAM/PWE OAM can be
carried transparently. Further, it was suggested that
service interworking could be avoided by doing
protocol translations either side of the PWE core, and
thereby creating a like-to-like service, end-to-end.

Issue 2. Can service interworking be avoided by
translations either side of PWE?
Issue 3. Is service interworking in or out of scope?

The still debated OAM issues leaked in to this thread
(in fact were the original genesis of the thread) with
a number of people supporting the view that the server
layer (MPLS) requires OAM in order to support network
interworking. Some stated that in the absence of MPLS
OAM, PWE OAM would be required, but that it was not
the preferred path.

Issue 4. Is it essential for the IETF to provide the
kind of MPLS OAM that is desired (or can another
standards body provide this)?

The problem with adapting a common mechanism to do
different PSN types is that each PSN different
capabilities. So it has been observed that if neither
the client layer (Ethernet) or the server/PSN (IP or
MPLS) has OAM, then there is a problem. It has been
further observed that the correct fix is to fix each
layer independently. The obvious attraction of the PWE
OAM approach is quick time to market with one
mechanism. Various opinions have been expressed as to
the architectural downsides of this approach.

Issue 5. Is it conceivable to think of PWE OAM as an
extension to the client layer for the network
interworking case?
Issue 6. Is it conceivable to think of PWE OAM as a
not required extension for the cleint layer for the
service interworking case?

The issue of how many layes of the stack does OAM need
to be performed at arose. It was mentioned "..fault
management is most frugally performed at the transport
layer although it must also exist to some degree at
the PW layer. Performance management hits both as
well, network planners need the transport
measurements, and SLA reporting needs the PW
measurements. "

But in an MPLS network with "n" layers, which is the
layer that is considered transport, and which is the
layer that SLA needs to be performed on. For example,
take a network that has a label for restoration, a
label for traffic engineering, a label for PE-PE, a
label for each VPN, and PWE. Which of these "layers"
is OAM required on? This many layers above the
facility layer is a new capability in networks, so hwo
do we think about it? It was also mentioned that when
e2e mechanisms are overlayed, the problem becomes not
one of fault correction, but fault correlation - so
does fault correlation become too complex with this
many layers?

Issue 7. Is OAM required on every layer?
Issue 8: Are there different OAM requirements at
different layers - and if so, does this problem become
too complex?

Issue 9: Is it possible that PWE OAM is a good
SLA-level mechanism but other transport-level OAMs may
be needed/optional?

Some disagreement occurred over whether LSP ping was
too heavy weight to do periodic connectivity
verification, with the later suggestion that much of
the discussion on the list focused in connectivity
verification rather than alarm indication suppression.

Issue 10. Is PWE OAM too heavy weight for alarm
indication/suppression or connectivity verification;
and if so why, and is there any evidence of this?
A criticism of the PWE OAM approach is that it
generates messages regardless of whether a fault exist
or not, which some see as being at variance to
typical/desired OAM behavior.

Issue 11. Should PWE OAM have an option to suppress
outputs?

The thread is winding down with discussion on the
difficulties of a generalized approach as MPLS injects
link swapping fault modes into what otherwise might be
considered a CNLS network; whether a P2P overlay is
ineeded needed to achieve OAM (to overcome MP2P merge)
vs just using OAM messages that indicate source (and
it seems to me both LSP-PING and Y.1711 can).

Personal observations:

-With the exception that there was disagreement about
whether mode swaps would prevent clock recovery, there
was general agreement that PWE over both MPLS and IP
can work
-PWE OAM appears to me to be rooted in concepts of SLA
management (vis-a-vis FR OAM was used as a model).
-Other transport OAM mechanisms should be available to
operators as an option (potentially developed by other
standards bodies if the IETF wants to pass on this
work).
-The terms CNLS and CO can be limiting. For example,
LDP is clearly CNLS in many respects, but because of
the label swap forwarding elelement, it introduces
fault modes that would normally be associated with CO
networks, and because of the semantics of the user
plane (i.e. no source address) it has restrictions
that would normally be associated with a CO network.
Therefore, it is my conclusion that LDP-MPLS is a
hybrid CO-CNLS mode; and clarity in discussion can be
achieved by referring specifically to the
characteristic that is at issue, i.e. label swap fault
modes, rather than assuming people understand the
moniker "CO".
-Ethernet is a service that SPs have indicated to many
vendors that they want to roll out over a PSN, and
they want to do it quickly in some cases. Ethernet is
defficient in a numer of ways, not the least of which
is a standardized OAM (though since the EFM is working
on one, it may be forthcoming). If we can use PWE OAM
as a client-layer psuedo OAM, and at the same time
bound the problems this causes and comit not to being
here in three years discussing Diffserv for PWE ;-)
then it could be a win-win for many people.

Issues summary
============

Issue 1. Is it the intention that PWE be just an
adaptation function?
Issue 2. Can service interworking be avoided by
translations either side of PWE?
Issue 3. Is service interworking in or out of scope?
Issue 4. Is it essential for the IETF to provide the
kind of MPLS OAM that is desired (or can another
standards body provide this)?
Issue 5. Is it conceivable to think of PWE OAM as an
extension to the client layer for the network
interworking case?
Issue 6. Is it conceivable to think of PWE OAM as a
not required extension for the cleint layer for the
service interworking case?
Issue 7. Is OAM required on every layer?
Issue 8: Are there different OAM requirements at
different layers - and if so, does this problem become
too complex?
Issue 9: Is it possible that PWE OAM is a good
SLA-level mechanism but other transport-level OAMs may
be needed/optional?
Issue 10. Is PWE OAM too heavy weight for alarm
indication/suppression or connectivity verification;
and if so why, and is there any evidence of this?
Issue 11. Should PWE OAM have an option to suppress
message output?

---

From: "Busschbach, Peter B (Peter)" <[email-masked]>
Date: Thu, 29 May 2003 22:06:03 -0400

Mark,

Great summary. Thanks.

A couple of the issues that you raise deal with the questions "how much OAM is required and where?" (issues 7, 8 and 9), and "is it the IETF's business anyway?" (issue 4). This is indeed the problem and we don't seem to be moving towards consensus.

I think there is a big difference between an ISP offering VPLS-like services over an MPLS network and a more traditional Telco looking at MPLS as a way to evolve its Frame Relay and ATM networks. The ISP may be able to get away with very little OAM. The Telco will definitely require a much more robust solution.

In my view, the IETF has taken a big step by deciding to define encapsulations of ATM and Frame Relay. If these respective standards move forward and become proposed standards, there needs to be an companion document that specifies a sufficently robust OAM solution. Vice versa, if the IETF decides not to define OAM and relegates it to another standards body, I believe it should stop all work on ATM and Frame Relay encapsulation.

Further comments in-line.

Peter

> -----Original Message-----
> From: Mark Seery [mailto:[email-masked]]
> Sent: Thursday, May 29, 2003 4:13 PM
> To: [email-masked]
> Subject: [PWE3] Is PWE a layer - thread summary
> 
> 

[snipped]

> Issues summary

I think that PWE was supposed to be just an adaptation layer. I think there are two forces that are turning into something more than that:

a) Difficulties in fault detection in the MPLS layer due to ECMP, which leads to the need for fault detection per PW.

b) The possibility of threading PWs together though switching based on the PW label (i.e. the TTL>2 discussion)

I am not sure how we would benefit from agreeing on an answer to your question. For the purist, it may not matter much. If PW is just adaptation, you do OAM in the underlying MPLS layer. If PW is a layer in and of itself, you do OAM in both the PW layer and the underlying MPLS layer. However, you would use the same mechanisms on both layers and the overall solution would not necessarily be more complex.
 
> translations either side of PWE?

Yes, you can avoid service inteworking with the MPLS layer. E.g. to achieve FR-ATM Service Interworking, you can use FRF.8 somewhere in the network (possibly in the NSP function of the PEs) and carry either FR or ATM over PW.

However, the issue that Neil brought up was different. I thought that in his view, Service Interworking was the preferred way to carry ATM and FR services over an MPLS core. In other words, to do FR-ATM Service Interworking, it might be best to do FR-MPLS Service Interworking at the ingress of the MPLS core and MPLS - ATM Service Interworking at the egress of the MPLS core.

I actually think Issue 2 is irrelevant to PWE. PWE is a solution for Network Interworking. If you want to use PWE and you need Service Interworking, you need to do it outside the PW between the respecive client layers.

> Issue 3. Is service interworking in or out of scope?

Right now it is out of scope. Should that change? I think Service Interworking is important and needs to be standardized. However, I see little synergy between the solutions that have been developed for PWE and what you would need for Service Interworking. Hence, adding Service Interworking to the charter might open Pandora's box.

> extension to the client layer for the network

It is conceivable, but wrong :-) But seriously, you are thinking of using PW OAM to provide OAM for Ethernet services, right? I think we have to be very careful with that. The discussion on "VPLS model for L2VPN Framework" that is currently going on on the PPVPN mailing list shows how tricky it is to correctly map IEEE bridging models. I believe that Ethernet OAM should be defined by the IEEE. Making some short-cuts using PW OAM may lead to big problems later.

> service interworking case?

As I mentioned above, the current definition of PWE is not well suited for Service Interworking. I find it hard to parse your formulation of Issue 6, but by all means, let's not link PWE and Service Interworking.

> Issue 11. Should PWE OAM have an option to suppress

In your email you wrote >> A criticism of the PWE OAM approach is that it
generates messages regardless of whether a fault exist or not, which some see as being at variance to typical/desired OAM behavior. <<

I am not sure which PWE OAM approach you are referring to. Is your point: if a PW is down, should a PE suppress client layer traffic? I don't think so. Layers should be independent as much as possible. A PWE OAM  function needs to detect failures in the PW/MPLS layer and inform the client layer of the failure. From that point on it is up to the client layer to decide what should happen. 

> 
> _______________________________________________
> pwe3 mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/pwe3
>

---

From: Mark Seery <[email-masked]>
Date: Thu, 29 May 2003 21:38:24 -0700 (PDT)

Hi Peter,

--- "Busschbach, Peter B (Peter)"
> Great summary. Thanks.

thx.

> I think there is a big difference between an ISP
> offering VPLS-like services over an MPLS network and
> a more traditional Telco looking at MPLS as a way to
> evolve its Frame Relay and ATM networks. The ISP may
> be able to get away with very little OAM. The Telco
> will definitely require a much more robust solution.

There is no such thing as a free lunch - there is a
tradeoff for "robust"; but which SPs pursue what value
proposition with what solutions should not be bounded
by what type of SP they are - going forward, this is a
choice that each makes. 

In a sense, this and other comments you make, suggest
a larger issue - that various organizations have
traditionally been integral elements of various and
often competing ecosystems.

The real issue is that the IETF has traditionally
pursued a certain approach to networking that was
believed to deliver certain values: lower cost,
greater scalability, faster time to market, open ness,
the unix philosophy of many small tools rather than
monolithic architectures,.....

All great value propositions are a package of elements
- aimed and focused. It is as hard for the IETF to
change one element of the package and still assert its
value proposition is whole, as it would be for the
ITU. This is related to your comment in the sense that
the IETF has traditionally been closely aligned to the
ISP ecosystem, and the ITU (and various other bodies)
have traditionally been aligned to the carrier
ecosystem.

Then there is the vendor element. It is **probably**
easier to migrate an ATM switch along the Y.1711
approach than to migrate a router along that approach;
and because ATM vendors already understand the OAM
concepts being discussed, it is also easier for them
to understand Y.1711, how to implement it, and how to
use it to solve a customer requirement.

And there is the operator element. Because a carrier
already undertands how to organize an operation around
OAM, then it is easier to migrate their operation
around Y.1711. Because an ISP may not have organized
their operation around OAM, it is much harder, and
possibly expensive, to organize their operation around
OAM. And then there is the issue of a carrier doing a
greenfield deployment for a new service - should they
pursue a new path for the sake of leading the rest of
the SP?

The only real problem here is defining what problem we
are solving. Once that is done, there is no lack of
brain power on this list (with the exception of me of
course) to go solve that problem.

All that said, there should be no reason why we can
not work together to develop solutions for a variety
of problems, network evolution scenarios, and
operators, while also finding commonality that allows
us to build upon a common framework.

> In my view, the IETF has taken a big step by
> deciding to define encapsulations of ATM and Frame
> Relay. If these respective standards move forward
> and become proposed standards, there needs to be an
> companion document that specifies a sufficently
> robust OAM solution. Vice versa, if the IETF decides
> not to define OAM and relegates it to another
> standards body, I believe it should stop all work on
> ATM and Frame Relay encapsulation.

I don't. The LSP-PING / PWE OAM approaches are
consistent with an alternative approach to building
networks. If it remains consistent with a package of
elements, then that allows the market to provide
differentiation among service providers - this is in
my purists point of view what makes for a beautiful
"market".

The participants in this working group do need to make
a decision though about whether one solution will meet
the needs of all significant elements of the SP
industry. It is valid for them to make that choice.

I do believe however that if the participants of this
WG are unable to come to agreement on an
implementation for an additional robust transport
layer OAM, then there are implications for the use and
future of MPLS, and for vendors who do not pursue
additional elements through other organizations (that
is purely a personal opinion - there is no impirical
proof for this, especially given the current health of
the vendor community). At the very least, vendors wil
be laying down a bet, which may or may not pay off;
each vendor will have to decide if they want to make
that bet. Each operator will have to decide if they
want to pursue alternative approaches through
different organizations.

Now back to more OT issues ;-)

> I think that PWE was supposed to be just an
> adaptation layer. I think there are two forces that
> are turning into something more than that:
> 
> a) Difficulties in fault detection in the MPLS layer
> due to ECMP, which leads to the need for fault
> detection per PW.

Thanks Pete, I forgot about that. Additionally I need
to educate myself on a) the implications of ECMP on
OAM, and b) whether OAM upper layer OAM even makes any
sense in an ECMP environment.

> b) The possibility of threading PWs together though
> switching based on the PW label (i.e. the TTL>2
> discussion)

So if something is not a layer, how can you stitch it
together? Seems to me this is heading down the road
that some of us are worried about. 3 years from now we
will be here discussing TE and Diffserv for PW ;-)

> I am not sure how we would benefit from agreeing on
> an answer to your question. For the purist, it may
> not matter much. If PW is just adaptation, you do
> OAM in the underlying MPLS layer. If PW is a layer
> in and of itself, you do OAM in both the PW layer
> and the underlying MPLS layer. However, you would
> use the same mechanisms on both layers and the
> overall solution would not necessarily be more
> complex.

The importance of such questions is to test the
premise whether PW over both IP and MPLS makes sense.
What I suspect is that some PWE elements should only
be used when running over IP; and then, the issue
arises of whether the package of elements that is IP,
is consistent with that approach.

> Yes, you can avoid service inteworking with the MPLS
> layer. E.g. to achieve FR-ATM Service Interworking,
> you can use FRF.8 somewhere in the network (possibly
> in the NSP function of the PEs) and carry either FR
> or ATM over PW.

Sounds reasonable to me.

> However, the issue that Neil brought up was
> different. I thought that in his view, Service
> Interworking was the preferred way to carry ATM and
> FR services over an MPLS core. In other words, to do
> FR-ATM Service Interworking, it might be best to do
> FR-MPLS Service Interworking at the ingress of the
> MPLS core and MPLS - ATM Service Interworking at the
> egress of the MPLS core.
> 
> I actually think Issue 2 is irrelevant to PWE. PWE
> is a solution for Network Interworking. If you want
> to use PWE and you need Service Interworking, you
> need to do it outside the PW between the respecive
> client layers.

So the issue becomes can we get consensus that it is
OK to use PWE for network interworking, but it is OK
for PWE to just disappear when doing service
interworking; but if you of the mindset that PWE is
actually a layer, you are bound to be perpetually
confused about what choice to make when.

> Right now it is out of scope. Should that change? I
> think Service Interworking is important and needs to
> be standardized. However, I see little synergy
> between the solutions that have been developed for
> PWE and what you would need for Service
> Interworking. Hence, adding Service Interworking to
> the charter might open Pandora's box.

OK, thanks.

> It is conceivable, but wrong :-) But seriously, you
> are thinking of using PW OAM to provide OAM for
> Ethernet services, right?

That is a potential, yes. Other potentials include any
future client that does not have OAM; not such an
issue if we can get robust OAM in to all the server
layers (IP and MPLS), but it appears we are not there
yet.

> I think we have to be very
> careful with that. The discussion on "VPLS model for
> L2VPN Framework" that is currently going on on the
> PPVPN mailing list shows how tricky it is to
> correctly map IEEE bridging models.

Yes, has all the hallmarks of being as much fun as
GMPLS ;-)

> I believe that
> Ethernet OAM should be defined by the IEEE. 

EFM is, I have to investigate how generalized it is
though. However, there is a lot of Ethernet out there
already.

> Making
> some short-cuts using PW OAM may lead to big
> problems later.

Not making them may lead to some people's careers
being terminated sooner ;-) And I am not just talking
about at vendors, I am talking about operators as
well.

Seriously, I have not seen a comprehensive
conversation yet which follows the line of...this is
the short-term problem, this is the long-term problem,
if we do it this way, we can evolve in the following
way. 

May be it is not possible, and perhaps that is too
idealistic.

> generates messages regardless of whether a fault
> exist or not, which some see as being at variance to
> typical/desired OAM behavior. <<

I'll have to try and dig it up. One of the posts made
the comment that an OAM mechanism should only produce
events if it detects a fault, whereas LSP Ping does on
every occassion. Though not explicitly stated, I think
the essential inference was that there is a difference
between a break-fix diagnostic tool, and an integrated
OAM mechanism.

Mark

---

From: [email-masked]
Date: Fri, 30 May 2003 13:10:27 +0100

Hi Mark,  comments in-line.  regards, Neil

>Mark Seery wrote 29 May 2003 21:13
NH=> A good idea.
> a protocol
NH=> What do you mean by a 'protocol'?  There are 'protocols' that are layer
networks and there are those that are not.  Any 'protocol' that can be
switched/routed at the native protocol level must be a layer
network.......the converse also holds, ie a protocol that only exists at 2
source/sinks points (eg UDP, TCP, AAL, PW, GFP, etc) is not a layer network.
This has some very significant architectural ramifications wrt what is
right/wrong.

>  to creep from an in-between layer to a full
NH=> MPLS can be switched/routed and so must be a layer network...PWs can't
be switched/routed (AKAIK) and so are not a layer network.  Being a layer
network has consequences that may not be palatable but are nevertheless
true, eg MPLS requires its own access point addresing scheme.  It does
indeed get this (simply because it has to) but its *different* for each
signalling type used (which is really bad, since addresses belong to the
data-plane and not the protocols of the control-plane that distribute and
act on them....and they also should be signalling protocol agnostic).
Further, the MPLS access point addresses do *not* belong to the IP address
space (or any other client address space for that matter). There is more to
a layer network than just addressing of course.
> carried transparently.
NH=> This is not a suggestion, it is a requirement. Think about all the
layer networks to the duct that have a recusive clinet/server
relationship.......client transparency is invariably taken for granted.  If
one starts tinkering with the client (ie to save a bit of BW or whatever)
then beware.  And remember that quite often a given operating entity will
*not* own all the layers to the duct, and so may lease these.....in that
case the operating entity will be expecting transparency.

A further observation here is that amongst the 3 networking modes some
client/server relationships make more sense than others.  This is simply an
expectancy of the normal performance a given client layer would get if it's
nodes were interconnected by L1 co cct-sw trails.  Of course one can deviate
from this, but again one should be aware of the consequences to avoid doing
stupid things.....for example trying to carry an SDH STM-N signal over IP
does not seem very sensible to me.

> Further, it was suggested that
NH=> Service interworking *is* protocol translation....across all functions
of the data and control planes that require conversion.  So I am not sure I
grasp your point.  More generally however, PWs are an adaptation
function.....that is quite clear to me.  But what really fazes me is *why*
some want to pretend that IP=~MPLS?  If IP=MPLS then why bother with MPLS at
all?  I can only assume its based on a desire *not* to allow MPLS to be seen
as a layer network in its own right.  I can actually understand why some
might hold this view for commercial or political reasons, but doing so does
not 'make it right' or beneficial to our industry at large.  MPLS uses a co
pkt-sw fowarding mode that is patently clear.....and we ought to use that
behavior to advantage.  The spoilier here is LDP mp2p....this is actually
the cause of nearly all the problems simply because its a topological misuse
of a co pkt-sw forwarding mode.....if one wants any-any behaviour one should
use a proper cnls mode like IP (or maybe ethernet.....though this has its
own problems because it does not have proper addressing (semantic of
'where') and no proper routing).

> translations either side of PWE?
NH=> Not sure what you mean by this.  In any case *why* would one want to
avoid service-interworking given it's really important?  It makes a great
deal of sense to us to drive down opex/capex by requiring that technologies
which belong to the *same* network mode should have edge conversion to a
common target technology for that mode.  Is this not simply common-sense?
If IETF cannot see the logic in this for whatever reason then they really
ought to stop playing in the co pkt-sw and co cct-sw space.

> Issue 3. Is service interworking in or out of scope?
NH=> I can really understand why some people would regard this as 'out of
scope'.  This view must be held by those who wish to constrain MPLS to a
role of IP-helper only.  But this is not a logical/rational view.  If IETF
believe MPLS is nothing more that an IP-helper they should not sanction PWE3
activities for arbitrary non-IP clients nor should they sanction GMPLS
activities...seems some want to have to their cake and (mistr)eat it.
> OAM, PWE OAM would be required, but that it was not
NH=> I wish *all* engineers who participate in any stds work were forced to
take a course in layered networks and understood G.805....if they did so
much of the nonsense I read/see would be avoided.  MPLS is a layer network
(though its not well specified as such) and PWs are not a layer
network....though they are now being imbued with 'functions' to cater for
poor functional architecture in MPLS (specifically the effects of mp2p
LSPs).  However, this issue (ie layer network functional components, where
OAM is simply one of these) is not specific to MPLS.....it applies to all
technologies that aspire to be used as layer networks.  When a technology is
functionally deficient problems arise when that technology is used in a more
general layer network sense......just look at the issues ethernet is
generating.  Ethernet is functionally deficient in many aspects from a
general layer network perspective, but especially addressing and routing. 
> standards body provide this)?
NH=> No of course not.  But the more relevant question IMO is 'why are IETF
working on co pkt-sw and co cct-sw technologies when they don't understand
layered networks, won't use functional modelling techniques and refuse to
use protocols that they don't own/control (when some are clearly not best of
breed)'.  
> 
> The problem with adapting a common mechanism to 
> different PSN types is that each PSN has different
> capabilities.
NH=> Exactly so....each network mode (cnls, co pkt-sw and co cct-sw) has
specific behaviours that are really important.  We need them all.  We want
solutions that bring the modes together in the *right* way.  So MPLS is
different to IP.  SDH is different to MPLS and IP.  Those people who started
off pushing the GMPLS peer-model stuff have a lot to answer for IMO.  I am
amazed how many people were (are still?) taken-in by this nonsense.

> So it has been observed that if neither
> mechanism.
NH=> Or is this 'marry in haste and repent at leisure'?  Being quick to
market is 1 thing but this does not imply being stupid too......bad
solutions have a longer-term opex hit.  Once established, WAN technologiues
tend to have a 25 year lifetime

> Various opinions have been expressed as to
NH=> We don't mind people doing crazy things.  What we do object to however
is trying to force those same stupid things on the whole community and not
allowing an alternative set of solutions to be produced.
> extension to the client layer for the network
NH=> It's not sufficient.  It only exists at the 2 end points......you still
have to manage the real layer network inbetween. 

> service interworking case?
NH=> I don't understand this.  PWs can't exist in service
interworking.....the 2 normally disjoint points of the PW (in the
client/server case) are now effectively co-located at the point of
interworking, ie the PW vanishes!
> to be performed at arose.
NH=> Its not an issue......all leayers need their own OAM independently.
This is not something we should even be discussing, its an obvious
requirement.

> It was mentioned "..fault
NH=> So what?  If a node fails at layer N all lower layers *must* be
blissfully unaware of this.
> is OAM required on?
NH=> All of them independently.   Marteen Vissers and my colleague Alan
McGuire who are both well respected experts in functional modelling are
developing the right functional models for MPLS right now.  So this activity
is well in-hand.

> This many layers above the
> facility layer is a new capability in networks,
NH=> Dealing with multiple layered networks is not new (see how many there
are in SDH), the novel feature of MPLS is that is has no fixed
hierarchy....its a true digital wrapper.  But we do know how to model it and
the required work is already well underway as noted above.

> so hwo
> do we think about it?
NH=> I can recommend a good book written by my colleague Andy Reid.  The
answer lies in taking the right architectural approach to describing layered
networks.  This has proved immensely useful as a precursor activity to
ensuring smoother vendor interworking, developing management models and
stopping people doing stupid things.....open-minded IETFers should try and
embrace this stuff.  It was not done just for the sake of it...it has real
value.

> It was also mentioned that when
NH=> No, providing one uses the right approach quite the opposite is true.
It will become very complex however if you don't.
> Issue 7. Is OAM required on every layer?
NH=> Ideally yes.  But there are different levels of monitoring that can be
used.  Further the source generation and sink processing operations can be
decoupled (see Appendix I of Y.1711).

> different layers - and if so, does this problem become
NH=> I think the question is wrongly phrased...if you are discussing MPLS.
The right observation here is that the OAM for a cnls mode is different to
the OAM for a co pkt-sw mode and is different to the OAM for a co cct-sw
mode.  However, within *each* mode there is an optimal best-of-breed
solution that *all* technologies belonging to that mode should
use.....Y.1711 is an optimal (sparse/poweful) solution for all technologies
belonging to the co pkt-sw mode (when that mode is correctly used of
course).
> SLA-level mechanism but other transport-level OAMs may
NH=> So we are offering PWs as services are we?  In that case is anyone ever
going to define the currently open-ended issue of entry/exit defect criteria
for sequence number aberrations and the consequent actions then?
> verification rather than alarm indication suppression.
NH=> LSP-Ping is a necessary, albeit complex, fix for mp2p LSPs.  However,
its hardly a 'always-on' defect detection/handling tool......in fact no
defects have even been defined for mp2p constructs.  Might make sense to
start there I think (but I can sure understand why this has not been done)!
> indication/suppression or connectivity verification;
NH=> This is not the issue.  The simple fact it is unecessary and can't
exist anyway with service-interworking.

> typical/desired OAM behavior.
NH=> Not sure I understand.  You have to generate some form of 'hello' to
detect failures.  Sending notifications only post failure makes no
sense.....esp if the data-plane used to send such notifications is broken.
Further, how does one detect misconfiguration/misbranching detects?
> Issue 11. Should PWE OAM have an option to suppress
NH=> No.....it should not be *optional*.  Under certain failure cases the
traffic must be supressed.
> considered a CNLS network;
NH=> I don't agree MPLS gives a cnls network.  mp2p attempts to mimic the
cnls behaviour but can't, viz: cnls *multiplexes* pkts, mp2p *merges*
pkts....these actions are not the same, which is why one is forced to always
run a higher level of p2p LSP to identify the source uniquely for an
arbitrary client layer above MPLS.  A whole raft of problems and kludges
stem from this.

> whether a P2P overlay is
NH=> If we have to run p2p LSPs then use Y.1711 on these....why add yet
further functions higher up.  We can now do service-interworking (with ATM
say) very easily.  In fact its already defined in Y.17iw.
> modes, rather than assuming people understand the
NH=> If you use G.805 to model mp2p the problems become obvious......ideally
before one embarks on letting it loose on an unspecting public.  Water under
the bridge now.....but the thing is to learn from this.

> vendors that they want to roll out over a PSN,
NH=> Are you sure this is not the other way round, ie we get vendor after
vendor trying to flog us this stuff and we are not asking for it?  More
tellingly, I don't hear customers asking us to build ethernet WANs
either......they want to interface to our WANs with ethernet, and how we
deliver this over the WAN is a different issue.

> and

---

From: "Thomas D. Nadeau" <[email-masked]>
Date: Fri, 30 May 2003 09:06:26 -0400

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Mark Seery
> Sent: Friday, May 30, 2003 12:38 AM
> To: [email-masked]
> Cc: [email-masked]
> Subject: RE: [PWE3] Is PWE a layer - thread summary
> 
> 
> Hi Peter,
> 
> proposition with what solutions should not be bounded by what 
> type of SP they are - going forward, this is a choice that 
> each makes. 
> 
> around Y.1711. 

	Not necessarily. My customer feedback indicates
that this is certainly not the case. In fact, the majority
my customers have told me that they do not want Y.1711
at all because of the added cost associated both in 
terms of hardware and operational cost.

	--Tom

> Because an ISP may not have organized
> greenfield deployment for a new service - should they pursue 
> a new path for the sake of leading the rest of the SP?
> 
> differentiation among service providers - this is in my 
> purists point of view what makes for a beautiful "market".
>

---

From: "Mark Seery" <[email-masked]>
Date: Fri, 30 May 2003 06:28:47 -0700

Neil,

> thereby creating a like-to-like service, end-to-end.
NH=> Service interworking *is* protocol translation....

MS=> Of course. The suggestion was made, by Yaakov.S. I think, that the
service interworking could be done in existing equipment, say an ATM switch,
before entering the MPLS cloud, to match to what ever was needed coming out
the other side of the MPLS cloud, and therefore relax the need to define
service interworking within the MPLS cloud. Now you may argue this is not
sufficient, and does not meet your needs in the long term, ......but seems
like a pragmatic short-term approach for some carriers. For example, a
number of carriers in the U.S. **are** planning to do this already.

MS=> From a practical perspective, if you assert something is crazy it
becomes impossible to broker an evolution path between two competing
solutions. This ensures that vendors will have to implement two disparate
approaches, or not. Which means either a) every operator pays the cost of
both approaches or b) some operators will have more equipment choices than
others.

are in SDH).....

MS=> The way SONET/SDH do OAM is much different, and actually preferable in
my opinion. With this approach, vendors will come to you and say, my product
has this granularity and in this part of the network it does section, line,
path, whatever,....but there is no expectation that every piece of SONET/SDH
equipment can terminate every layer of OAM in every part of the network.
More over when a fault does occur, you simply flip some indicators in
existing overhead and the fault is inidcated, you are not creating extra
bandwidth/processing at exactly the point the network is experiencing
stress. To take the SONET/SDH approach of OAM on every layer and apply it to
packet networks is problematic for implementors unless you are willing to
specify a hierarchy so the implementors can bound the problem.

If the implementors can not bound the problem (and I am not the only one who
sees this problem) then they have to choose between a worst case
implementation, or make assumptions on the operators behalf (and many
operator are likely to have a different networks given such extraordinary
layering flexibility) which may or may not meet requirements.

There has to be a more sophisticated conversion between implementors and
operators than simply a stand-off between "you must do every layer at x/sec
to y/sec" and "this is too expensive to implement". At least with ATM the
problem is bounded, ATM only supports two layers. With MPLS the problem is
wide-open; and the more layers deep a network is, the more complex the
processing pipeline becomes - once again it would be very different if OAM
was a fixed element of every packet and the implementors could choose what
granularity (which layers) they were operating on.

Mark

---

From: "Mark Seery" <[email-masked]>
Date: Fri, 30 May 2003 06:38:55 -0700

MS=> Of course this becomes a matter of conjecture without any formal way to
recognize the expression of an operator's desire ;-)

Jack's and Neil's remarks are at least to dissenting opinions.

I suspect what you might be seeing is feedack related to a set of
expectations about service characteristics+the assumption that if the
underlying layers are robust there is less need for heavy-weight approaches
at the higher layers. Which is one way to look at the problem - clearly not
every one agrees.

Mark

---

From: "Jack Pugaczewski" <[email-masked]>
Date: Fri, 30 May 2003 08:57:04 -0500

I find it amazing that manufacturers are itemizing the cost of functionality
(i.e.,Y.1711).  The functionality of I.610 currently leveraged pays for
itself over the life of the product.  The ability to fault isolate issues in
an automated fashion using the combination of an EMS and the embedded
switch/router OAM functionality is essential.  This functionality allows
fault isolation to the location within the path as well as the protocol
layer.  In a PWE-3 network and the combination of Frame Relay, ATM, TDM,
etc. this will more important than ever.

PJMWBGJJREGTATALDBJHJHSLDHGSDWJAMY

> From: [email-masked] [mailto:[email-masked]]On Behalf Of Mark
> Seery
> Sent: Friday, May 30, 2003 8:39 AM
> To: [email-masked]
> Cc: [email-masked]

---

From: Mark Seery <[email-masked]>
Date: Fri, 30 May 2003 07:16:27 -0700 (PDT)

Jack,

I appreciate your point.

My point is simply that if the problem (read layers)
are not bounded via some implementation agreements,
then it is very hard for the implementor to know
exactly what the requirement is. As I stated before,
the problem is well-bounded in both SONET/SDH and ATM
networks. It is not well-bounded in MPLS networks.

Mark

---

From: "Jack Pugaczewski" <[email-masked]>
Date: Fri, 30 May 2003 09:37:06 -0500

Then this looks like a great forum to get ideas out and explore the proper
means for implementation aggrements.  Now is the time to define these
mechanism between Layer 2 (Frame Relay, ATM, Ethernet, etc.) and MPLS.  This
functionality will be essential in the deployment of a quality network that
can be properly and efficiently maintained.

> Sent: Friday, May 30, 2003 9:16 AM
> To: Jack Pugaczewski
> Cc: [email-masked]

---

From: [email-masked]
Date: Fri, 30 May 2003 15:42:31 +0100

Tom,
<snipped>
> terms of hardware and operational cost.
NH=> The whole point of having decent OAM is to try and reduce operational
cost (and the customer disatisfaction index) not increase it.  Why do you
think there are major efforts in IEEE, MEF and ITU to provide ethernet with
OAM.....and there is 1 very large operator who is driving this (at least in
ITU) and its not BT?  

regards, Neil

---

From: [email-masked]
Date: Fri, 30 May 2003 16:09:48 +0100

Absolutely right Jack.  I can't believe there are operators who don't have
an aim of driving down their opex costs (esp thesedays)....so decent OAM is
not a 'desirable function' its a critical component.  In BT we also have a
some stretching targets on driving down the customer disastisfaction
index....again this means automated OAM as we don't want the customer to be
telling us (or even another operator) there is a problem with out network.

Sure one has to balance doing the right thing with time to market.....but if
one pays attention to these matters *before* deploying kit (ie during stds
work) then that is the best....retrospective fitting is painful, costly and
not a good idea.

regards, Neil  

> From: Jack Pugaczewski [mailto:[email-masked]]
> Sent: 30 May 2003 14:57
> To: Mark Seery; [email-masked]
> Cc: [email-masked]

---

From: [email-masked]
Date: Fri, 30 May 2003 16:09:49 +0100

Mark....How many layers do you think would be required in MPLS?  My guess is
not that many.  The beauty of being able to create an ad hoc LSP between 2
points (as a server to some higher layer trail) from an operators point of
view is that we can monitor the partition very simply.  This type of
functionality is limited or complex when layers have a fixed hierarchy
(check out some of the tandem connection monitoring ideas for G.709 for
example).

Regards, Neil

> Sent: 30 May 2003 15:16

---

From: Mark Seery <[email-masked]>
Date: Fri, 30 May 2003 09:25:31 -0700 (PDT)

Neil,

--- [email-masked] wrote:
> not that many.  

If as an operator you make a decision that OAM is just
**one** aspect of a proper CO-PKT-SW network, and
therefore you will continue to run services, of a
particular type, over an ATM/MPLS network you are
probably not as likely to get "creative" with the use
of layers, and the vendors providing equipment for
such a network (more than likely the incumbent you
already have) can very comfortably support your
requirements.

There is a well-known operator in the U.S., running FR
over MPLS today, without the use of Y.1711, who has
stated in the past quite a large number of layers (I
won't get in to details because of confidentiality
issues of which I am unsure where I stand).

But as an example:

One layer for restoration (multiple LSPs restored by a
server layer LSP).

One layer for traffic engineering and/or per class QoS

One layer for the BGP network (BGP overlay over a TE
core)

One layer for the PE-PE base LSP

One layer for each VPN over the PE-PE LSP

And of course PWE (if that ends up being a layer)

These are the kinds of network architectures (and the
above is not even a worse case) that vendors who
develop IP/MPLS products get presented with.

One one hand saying each layer requires the same OAM
makes the problem easier, because the implementor does
not have to guess which of the myriad of layer options
might be used. On the other hand it imposes a certain
burden on each layer.

At any rate, it is not my opinion of how many layers
that are necessary that really counts, it is the
operators opinion, and as you can see from above, the
requirements vary greatly.

In all MPLS networks (running VPNs) there is going to
be at least 2 layers, we know this much from the VPN
standards that are being ratified. In many MPLS
networks there will be 3 layers, in a number I suspect
there will be 4 layers; and in some networks  - the
number of layers will be certifiably pathological ;-)
But hey, at least it is flexible ;-)

Seriously, I don't have a problem if an operator wants
to have lots of layers, that is one of the things I
really like about MPLS - I really believe it has the
power to create innovative architectures. It just
becomes pathological if a variety of requirements are
imposed on each layer. 

In addition, unless I misunderstand the behavior of
FDI, then I am concerned about the implications if
multiple layers injecting FDI concurrently, trying to
percolate fault information up multiple layers, and do
correlation. The potential for coupling problems and
non-linearalities seems to be there. But it is also
possible that I simply don't understand how it works -
so I will try to study it some more.

> The beauty of being able to create
> an ad hoc LSP between 2
> view is that we can monitor the partition very
> simply.  

There is a cost to flexibility, that has to be
understood. Bound the flexibility, and the cost can be
bounded. If the flexibility is not bounded by the
operators, then implementors will have to make
arbitary choices. You can take your chances with that,
or you can go lay down some money at Monte Carlo -
your choice. I would speculate that if SONET/SDH was
not as bounded as it is, that it would cost even more
than what it does (which BTW is less than packet
solutions last time I checked - but you tell me, you
would know best).

> This type of

I flicked through G.709 and it wasn't obvious to me
which section dealt with tandem connection monitoring,
but I will take a closer look.

Thanks for the pointer,
Mark