Subject: Re: composite link - candidate for respin, maybe

From: Curtis Villamizar <[email-masked]>
Date: Thu, 01 Apr 2010 03:07:13 -0400

In message <[email-masked]>
"Mcdysan, David E" writes:
>  
> I had a general statement about scaling in the wg slides, and I recall
> Alex objecting that in general the IETF avoids quantitative values. 
>  
> This is an important issue, and is one we need to address to better
> understand the cost/benefit of approaches that differ from just
> extending the set of advertised parameters and/or their useage in
> signaled LSPs.
>  
> I propose that we discuss a parameteric model, e.g., Number of nodes
> (N), total number of Links (L), some range of Component links per
> Composite (C(i)), frequency of latency changes (L), frequency of
> capacity changes, etc. and try to compare the "scalability" and
> "response time" in a parametric way. Then, outside of the IETF,
> quantitative numbers could be inserted.
>  
> Dave

Really thorough treatment of the topic could go to SigComm or any of a
number of places.  A quick analysis or a summary of deployment
experience can go to NANOG.

The model can be proposed outside IETF as well.  It makes no sense to
put it in the requirements document.  In most analysis of IGP
convergence or CSPF workload or convergence there have been
assumptions about average degree of a node, etc.  Models will vary.

Curtis

> > -----Original Message-----
> > From: [email-masked] [mailto:[email-masked]] 
> > On Behalf Of Tony Li
> > Sent: Tuesday, March 30, 2010 3:18 AM
> > To: [email-masked]; John E Drake
> > Cc: [email-masked]
> > Subject: Re: composite link - candidate for respin, maybe
> > 
> > 
> > 
> > 
> > >> A clear statement as to what we are trying to accomplish wrt IGP 
> > >> scaling by using bundled links would be good.
> > > 
> > > Please indicate what in the requirements we are discussing is not 
> > > clear rather than a blacket statement that the goals are not clear.
> > 
> > 
> > Once again: what are the scalability goals that we're trying 
> > to achieve, both on a per-system and network-wide basis?  
> > This should be a quantitative requirement.
> > 
> > Tony
> > 
> > 
> > _______________________________________________
> > rtgwg mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/rtgwg

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 01 Apr 2010 03:18:42 -0400

In message <[email-masked]>
"Mcdysan, David E" writes:
>  
> "Pinning" an LSP to a specific component link was an intended use case
> from the prior draft not in the current list.
>  
> Do need to think more about the means (configuration, signaling) of
> how this could be done. I will draft some text as to why an operator
> would want to do this.
>  
> Dave

I'd like to know why if an operator had N links with identical
characteristics that operator would need to pick a specific link.

If some of those links had different characteristics, then that is
another story.  They should be advertised as different.

The only thing I can think of is an XRO functionality (exclude route
object used to insure disjoint loose paths, see RFC4874).

Curtis

> > Sent: Tuesday, March 30, 2010 3:30 AM
> > Hi Curtis,
> > > One is the abilit to move LSP from one component to another.  In 
> > > Tony's view that would not be possible because the LSP is nailed to 
> > > one component, not a group with common attributes and a 
> > specification 
> > > of maximum dynamic.
> > 
> > 
> > Actually, I'm rather into flexibility.  Some LSPs might be 
> > pinned.  Some might not.  
> > 
> > 
> > Tony

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 01 Apr 2010 03:27:52 -0400

In message <[email-masked]>
"Mcdysan, David E" writes:
> > Sent: Tuesday, March 30, 2010 12:27 PM
> > To: John E Drake; [email-masked]
> > Cc: [email-masked]
> > Hi John,
> > > If we are placing flows on the component links of a 
> > heterogeneous link 
> > > bundle, we may have to look at additional fields within 
> > packets and/or 
> > > maintain additional state, relative to what is required to 
> > place flows 
> > > on the component links of a homogeneous link bundle.  The 
> > example that 
> > > has been given is flows that require low latency.
>  
> The current draft combines change frequency and low latency. I think we
> need to separate these out as separate characteristics.

I don't see where that comment is coming from.  There is nothing to
say that change frequency and low latency are combined in one
parameter.

Please indicate what text you would like to see changed.

> > If we go by the definition that we're working with, a "flow" 
> > is atomic and therefore should never be demuxed, just to 
> > ensure ordering.
> > 
> > 
> > > I was agreeing with Tony, so please direct this question to him.  
> > > (But, I don't see any real discussion of IGP scaling requirements.)
> > 
> > 
> > I think that this has to come from Dave.  If I get to set the 
> > requirements,
> > I promise a Very Low Bar.   ;-) ;-) ;-)
>  
> I agree that we need to have scaling requirements. Also, I think that
> convergence time (or response time to a transient) is something we need
> to specify. See my earlier note. 

We can't specify convergence time.  For one thing, what are we
specifying convergence of?  Is it convergence of a load balancing
operation?  If so, how fast are the offerred loads of the flows
changing?  Is it convergence of an entire network after a fault?  If
so what is the topology of the network that is converging and what
fault are we covering?  Do we try to add realistic control plane
processing times?  If so who's implementation?

Curtis

---

From: John E Drake <[email-masked]>
Date: Thu, 1 Apr 2010 07:26:03 -0700

> From: [email-masked] [mailto:[email-masked]]
> Sent: Wednesday, March 31, 2010 11:00 PM
> To: John E Drake
> Cc: [email-masked]; Tony Li; [email-masked]
> In message <[email-masked]>
> John E Drake writes:
> >
> > >
> > > You are missing a few things.
> > >
> > > one component, not a group with common attributes and a specification
> > > of maximum dynamic.
> > [JD]
> >
> > Moving a section of an LSP from one path to another does not require
> > link bundling.
> 
> 
> Yes entirely true but being able to move a pinned LSP and pin it
> somewhere else is not the only requirement.
[JD] 

Yes, but it was the requirement *you* used.

> 
> 
> 
> > > Another is the ability to allow an LSP to be load split across a set
> > > of components.  The top LSP may (if signaling indicates this is OK) be
> > > split among component links.  This is the only way to support LSP of
> > > greater than 100G in the near future.
> > [JD]
> >
> > The discussion was about IGP scaling.  Both examples you describe
> > deal with signaling and could be decoupled from any IGP
> > enhancements that are made.
> 
> 
> You've deleted enough context (the examples) that I'm not sure if I'm
> adequately addressing your point here.
> 
> In order for an ingress to specify that some sort of load split of an
> LSP cound occur it needs to know that that capability is available on
> that link.  So for example, if an LSP is 180G and there are two 100G
> interfaces in parallel, the ingress needs to know that this is a CL
> and that it can handle this LSP.  The IGP provides that information.
[JD] 

This could be done without implementing the IGP scaling aspects of link bundling/composite links.  So, again what are the requirements wrt IGP scaling?

> 
> 
> > > > > > If we don't put a bound on the number of sets, in the limit each
> > > > > > component could be advertised with the added semantic that there
> is
> > > a
> > > > > > sharing option.  In that case the LSP may need to specify the
> set of
> > > > > > "groups" (as I have it above in the absense of a term) that are
> > > > > > acceptable.  This could make for a very big ERO.  It should be
> > > > > > adequate to indicate in the RRO that the bundle is being used
> with a
> > > > > > field indicating which member was currently in use (if nailed in
> > > > > > place).  This is beyond requirements, so if we are OK with the
> > > stated
> > > > > > requirement, let defer the details to later.
> > > > >
> > > > > I'm fine with that, but we may also want to specify the component
> in
> > > the
> > > > > ERO.
> > > > [JD]
> > > >
> > > > See
> > > > http://datatracker.ietf.org/doc/draft-ietf-mpls-explicit-resource-
> > > control-bundle/
> > >
> > >
> > > The functionality I mentioned above is missing.
> > [JD]
> >
> > Then you need to re-phrase your requirement (or re-read the I-D)
> 
> 
> Again you deleted context.  By "re-phrase your requirement" I take it
> that you mean just the following requirement:
> 
>   6.  A means to support non-disruptive reallocation of an existing
>       LSP to another component link is needed.
> 
> There is nothing here that prohibits using stitching or any other
> method.  We are not defining *how* to do anything, just what the
> requirements are. 
[JD] 

Actually, I think the discussion was about allowing the ingress to pin an LSP to a particular component link.  You had stated that this was not possible with link bundling and I was pointing out that it is.

> 
> The very first paragraph reads:
> 
>   These requirements refer to link bundling solely to provide a frame
>   of reference.  This requirements document does not intend to
>   constrain a solution to build upon link bundling.  Meeting these
>   requirements useing extensions to link bundling is not precluded, if
>   doing so is determined by later IETF work to be the best solution.
> 
> What about "solely to provide a frame of reference" don't you
> understand?
> 
> I don't see any point in adding that "One of these requirements might
> be met by LSP stitching.  This requirements document does not intend
> to constrain a solution to build upon LSP stitching.  Meeting these
> requirements useing extensions to LSP stitching is not precluded, if
> doing so is determined by later IETF work to be the best solution."
> 
> Do you get it now?
[JD] 

See above.

> 
> 
> > > > > > OK.  So what is your objection?  Are you suggesting that we need
> to
> > > > > > restrict the load balance to look down N labels from the top and
> no
> > > > > > further?
> > > > >
> > > > > No, I'm suggesting that we need a bit in the RESV to say that this
> LSP
> > > > > should be demuxed by inner labels or kept atomic.
> > > > [JD]
> > > >
> > > > In current link bundling an LSP is either assigned to a specific
> > > > component link or split across all of the component links.  In
> > > > the latter case this effectively means all or a subset of the
> > > > component links.
> > > >
> > > > If an LSP is assigned to a specific component link, RFC 5150
> > > > (http://datatracker.ietf.org/doc/rfc5150/) (a one hop LSP on the
> > > > assigned component link) and MBB can be used to move the LSP
> > > > non-disruptively to another component link
> > >
> > >
> > > Creating every LSP by stitching one hop LSPs together is a very poor
> > > solution IMO.  The requirement though do not specify a solution.
> > [JD]
> >
> > It depends upon how many LSPs need this behavior.  LSP Hierarchy
> > would also work.  So the requirement needs to be more precise.
> 
> 
> I also do not see the point of adding "One of these requirements might
> be met by LSP hierarchy.  This requirements document does not intend
> to constrain a solution to build upon LSP hierarchy.  Meeting these
> requirements useing extensions to LSP hierarchy is not precluded, if
> doing so is determined by later IETF work to be the best solution."
> 
> We are not specifying solutions at this point.  We are specifying
> requirements.  Got it?
[JD] 

You misunderstood me.  All I was saying was that any information that would allow us to evaluate potential solutions should be included in the requirements.

---

From: John E Drake <[email-masked]>
Date: Thu, 1 Apr 2010 07:45:00 -0700

> Sent: Wednesday, March 31, 2010 11:56 PM
> To: John E Drake
> Cc: [email-masked]; Tony Li; Mcdysan, David E; [email-masked]
> In message <[email-masked]>
> John E Drake writes:
> >
> >
> > > John,
> > >
> > > When nothing is being moved nothing is reordered.  If a flow or many
> > > flows is moved from a link with longer delay to one with shorter
> > > delay, unless the flow is stopped and bufferered for at least the
> > > delay difference and then drained, then reordering will occur.
> > >
> > > So I agree with Tony.  It is not practical to avoid reordering when
> > > traffic is moved.  It is therefore minimally disruptive.  The products
> > > of your new employer do the same thing.
> > [JD]
> >
> > One last time: Flows may be re-ordered, but packets within flows are
> > not re-ordered.  This is no different than placing flows of a single
> > LSP on multiple component links, which is a required behavior.  The
> > only difference is that when an LSP is pinned to a specific component
> > link, this only occurs when the LSP is being pinned to a new component
> > link using MBB.
> 
> 
> 
> John,
> 
> You are making no sense at all.
> 
> Consider the case where only one flow exists.  Two path exist, A and
> B.  Path A is used.  It has a longer delay than B.  The difference in
> delay is N packet times.  When switching from A to B, the first packet
> to arrive on path B arrives N-1 packets before the last packet to go
> into link B.  Packets from that one flow are interleaved for about N
> packets.
> 
> The case that we are talking about is moving a flow or group of flows
> from one link to another.  It is minimally disruptive.  Do you have a
> problem with that wording?
[JD] 

Fine, in the limiting case of an LSP with a very small number of flows, there could be packet re-ordering.

---

From: John E Drake <[email-masked]>
Date: Thu, 1 Apr 2010 07:43:20 -0700

> Sent: Wednesday, March 31, 2010 11:48 PM
> To: John E Drake
> In message <[email-masked]>
> John E Drake writes:
> >
> > > Sent: Monday, March 29, 2010 9:49 PM
> > > To: John E Drake
> > > Cc: Tony Li; [email-masked]; [email-masked]
> > > In message <5E893DB832F57341992548CDBB3331639806A1B9EB@EMBX01-
> HQ.jnpr.net>
> > > John E Drake writes:
> > > >
> > > > Tony,
> > > >
> > > > Clearly there is extra overhead (roughly 2*(N - 1)* 40 bytes?) with
> N
> > > > homogeneous bundled links versus one heterogeneous bundled link.
> The
> > > > concern I have is that having heterogeneous bundled links may
> > > > complicate the data plane packet forwarding.
> > >
> > > How the control plane information is presented does not "complicate
> > > the data plane packet forwarding".
> > [JD]
> >
> > If we are placing flows on the component links of a heterogeneous link
> > bundle, we may have to look at additional fields within packets and/or
> > maintain additional state, relative to what is required to place flows
> > on the component links of a homogeneous link bundle.  The example that
> If the LSP requires low latency, then the LSP is signaled for low
> latency.  If it asked to be pinned, all the flows in that LSP are
> pinned to one component link.  That is easy, just map the label using
> the ILM to one link.
> 
> If the LSP requires a particular flavor of low latency, but it doesn't
> mind being load split then it is load split across only those
> components that meet that latency requirement.  The top label
> identifies the subset of the component links that are "fair game" and
> some algorithm that the LSP has signaled it is OK to use splits the
> flows among those links.
[JD] 

You are specifying a solution here.  You are saying that the specific set of component links to be used by a given LSP is identified during signaling, rather than by data plane packet classification.

You also seem to be saying that the LSP ingress specifies the load balancing algorithm.  If so, I don't think this is a good idea.

> 
> If you think about how this is implemented in routers today this is
> all doable now, just not with a very good load split.  If you don't
> have the slightest clue how it is done today RFC2991 and RFC2992 will
> provide the slightest clue but not much more.
> 
> > The requirements document needs to be clear as to which fields in the
> > packet header are to be used in packet classification.  E.g., do we
> > deal strictly with the MPLS header or can the packet header also be
> > used, and if so, do we have to deal with encrypted as well as
> > unencrypted packets?
> 
> No it doesn't!  We are specifying requirements, not solutions.  When
> we get around to specifying solutions then it does.
> 
> There is 10-15 years of history on this that says it is doable,
> because it is being done so we are well beyond the point where we need
> example to establish that a feasible technique exists.
[JD] 

I don't think everyone is on the same page regarding what fields in a packet are to be used in packet classification.  Some of the text you added to the MPLS-TP OAM framework I-D could usefully be added to the requirements I-D.

> OK.
> > > > John