Subject: Progressing MAR

From: "Ash, Gerald R (Jerry), ALABS" <[email-masked]>
Date: Fri, 7 Mar 2003 10:08:11 -0500

All,

A specification for the Maximum Allocation with Reservation (MAR) bandwidth constraint (BC) model is proposed in http://www.ietf.org/internet-drafts/draft-ash-mpls-dste-bcmodel-max-alloc-resv-01.txt.  

MAR is an extension of the Maximum Allocation Model (MAM).  MAR is similar to  MAM in that a maximum bandwidth allocation is given to each class type (CT).  However, through the use of bandwidth reservation and protection mechanisms, CTs are allowed to exceed their bandwidth allocations under conditions of no congestion but revert to their allocated bandwidths when overload and congestion occurs.  

Analysis is presented in an ANNEX which shows that the MAR model meets all the objectives for BC models, and that it simultaneously achieves bandwidth efficiency, bandwidth isolation, and protection against QoS degradation without preemption.  The analysis information provides critical guidance to users' implementation of the BC models and some important guidelines which users should be aware of before choosing a BC model for their network.

I would like to get a sense of the list for using this I-D as the basis for the MAR specification and for accepting this as a WG document.  

Comments are welcome.

Thanks,
Jerry

---

From: Dimitry Haskin <[email-masked]>
Date: Mon, 10 Mar 2003 11:11:52 -0500

To my understanding all currently specified BC models (RD, MAM, MAR) can be
currently only considered for Experimental or Informational status at most.

Just to make sure that we are on the same page...

Dimitry

> -----Original Message-----
> From: Ash, Gerald R (Jerry), ALABS [mailto:[email-masked]]
> Sent: Friday, March 07, 2003 10:08 AM
> To: [email-masked]
> Cc: Ash, Gerald R (Jerry), ALABS; Lai, Wai S (Waisum), ALABS; Francois
> Le Faucheur (flefauch); Jim Boyle; Ed Kern (ejk)
> Subject: Progressing MAR
> 
> 
> All,
> 
> A specification for the Maximum Allocation with Reservation 
> (MAR) bandwidth constraint (BC) model is proposed in 
http://www.ietf.org/internet-drafts/draft-ash-mpls-dste-bcmodel-max-alloc-re
sv-01.txt.  

MAR is an extension of the Maximum Allocation Model (MAM).  MAR is similar
to  MAM in that a maximum bandwidth allocation is given to each class type
(CT).  However, through the use of bandwidth reservation and protection
mechanisms, CTs are allowed to exceed their bandwidth allocations under
conditions of no congestion but revert to their allocated bandwidths when
overload and congestion occurs.  

Analysis is presented in an ANNEX which shows that the MAR model meets all
the objectives for BC models, and that it simultaneously achieves bandwidth
efficiency, bandwidth isolation, and protection against QoS degradation
without preemption.  The analysis information provides critical guidance to
users' implementation of the BC models and some important guidelines which
users should be aware of before choosing a BC model for their network.

I would like to get a sense of the list for using this I-D as the basis for
the MAR specification and for accepting this as a WG document.  

Comments are welcome.

Thanks,
Jerry

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Thu, 13 Mar 2003 14:43:10 -0000

Jerry,

I believe the conclusion of our long discussion on BC Model was that we
agreed to specify RDM and MAM and to investigate further other models
including MAR.
I don't believe there is an agreement (at this stage anyway) that the WG
needs to produce a spec for MAR. So my view is that it would be
premature to have a WG document for MAR.

When we discussed the previous version of your MAR spec, if I remember
correctly I think we ended up concluding that your claim that "it
simultaneously achieves bandwidth efficiency, bandwidth isolation, and
protection against QoS degradation without preemption" is ONLY true
under the assumptions that :
	(i) the bandwidth that will be actually reserved by each CT is
known fairly accurately ahead of time for every link (ie you need to be
sure that on every link a CT will not, or only very temporarily, exceed
its Bwalloc).
	(ii) you configure the Bwalloc of each CT based on the expected
(and accurate) demand.
Is this not the case?

Personnally, I see these assumptions as a serious problem. Basically it
seems to defeit the whole purpose of DSTE: With DSTE you want to
configure some limits for each CT based on resources available and
engineering rules and have traffic redistributed in accordance with
that. Its seems with -00.txt MAR description, you need to do the
opposite ie configure the Bwalloac based on what the load will actually
and you assume that every link is engneered according to the actual
demand without redistribution of load (which makes you wonder why you
need DSTE in the first place).

Has something changed between -00.txt and -01.txt with respect to the
above assumptions?

As discussed last time, MAR seems to contain a very good idea of
allowing CTs to borrow bandwdith but I think we need to somehow enhance
MAR to avoid the above assumptions/constraints (unless you've already
done that in -01). 

Thanks

Francois

>> Sent: 07 March 2003 16:08
>> To: [email-masked]
>> Cc: Ash, Gerald R (Jerry), ALABS; Lai, Wai S (Waisum), 
>> ALABS; Francois Le Faucheur (flefauch); Jim Boyle; Ed Kern (ejk)
>> Subject: Progressing MAR
>> 
>> 
>> All,
>> 
http://www.ietf.org/internet-drafts/draft-ash-mpls-dste-bcmodel-max-allo
c-resv-01.txt.  

MAR is an extension of the Maximum Allocation Model (MAM).  MAR is
similar to  MAM in that a maximum bandwidth allocation is given to each
class type (CT).  However, through the use of bandwidth reservation and
protection mechanisms, CTs are allowed to exceed their bandwidth
allocations under conditions of no congestion but revert to their
allocated bandwidths when overload and congestion occurs.  

Analysis is presented in an ANNEX which shows that the MAR model meets
all the objectives for BC models, and that it simultaneously achieves
bandwidth efficiency, bandwidth isolation, and protection against QoS
degradation without preemption.  The analysis information provides
critical guidance to users' implementation of the BC models and some
important guidelines which users should be aware of before choosing a BC
model for their network.

I would like to get a sense of the list for using this I-D as the basis
for the MAR specification and for accepting this as a WG document.  

Comments are welcome.

Thanks,
Jerry

---

From: "Ash, Gerald R (Jerry), ALABS" <[email-masked]>
Date: Thu, 13 Mar 2003 13:46:06 -0600

Francois,

> agreed to specify RDM and MAM and to investigate further other models

I don't see that conclusion written down anywhere.  In Atlanta, the WG debated 1 default  versus 2 default models, and no conclusion was reached.  As to what to specify in the way of BC models, that's the discussion we're having now on all 3 models, so no conclusion is yet reached.

Anyway, my views are as follows:

1. progress MAM.

2. hold off on progressing RDM until some major issues have been resolved:
a) further extensions/protections should be proposed to protect against poor performance when preemption is not used,
b) at a minimum, provide a warning that RDM may be 'hazardous to the health of your network if preemption not used',
c) provide pointers to and/or inclusion of performance analysis.

3. continue the discussion on MAR toward a consensus.

There are service providers who do not use preemption, and have major concerns with RDM and its dependence on preemption to work well, that it can perform poorly when preemption is not enabled.  Our preference would be to use models not dependent on preemption (MAM or MAR).  There was much support on the list that a BC model should not require preemption to work well.  I think you should take that into account in RDM, and provide extensions to protect against same.

> premature to have a WG document for MAR.

That is what we are discussing, so it is premature to draw a conclusion before the discussion, based only on your opinion.  We should continue the discussion on MAR toward a consensus.  As stated in the I-D: 
"MAR is an extension of MAM and like MAM assigns a maximum bandwidth allocation to each CT, but with bandwidth reservation mechanisms, allows CTs to exceed their bandwidth allocations under conditions of no congestion but revert to their allocated bandwidths when overload and congestion occurs.  Analysis shows that MAR meets all the objectives for BC models, and that it simultaneously achieves bandwidth efficiency, bandwidth isolation, and protection against QoS degradation without preemption."
These are good reasons to pursue MAR as an extension of MAM.

> known fairly accurately ahead of time for every link (i.e. you need to be
> its BWalloc).
> 	(ii) you configure the BWalloc of each CT based on the expected
> (and accurate) demand.
> Is this not the case?

No, see comments below.

> Personally, I see these assumptions as a serious problem.  Basically it
> seems to defeat the whole purpose of DSTE: With DSTE you want to
> opposite i.e. configure the BWalloc based on what the load will actually
> and you assume that every link is engineered according to the actual

Most service providers (SPs) don't operate their networks in the way you suggest, that is, SPs don't put capacity out there and then decide how to allocate it to whatever traffic might show up.  To the contrary, SPs engineer their networks based on traffic projections to determine needed capacity, topology, routing, etc.  Accordingly, CT bandwidth allocations are estimated.  More sophisticated methods (in actual use) make on-line measurements of bandwidth usage as a basis for the bandwidth allocations.  

Once bandwidth is allocated, there is *no assumption* that this will actually represent, or be close to, the actual traffic that arrives.  Extremely large swings and surges in traffic occur, massive overloads occur, failures occur.  MAR is expected to operate well under these conditions (and does in actual operation, see below).  All the BC models in fact, should operate well under these same conditions.  There should be no presumption of 'accuracy' in bandwidth allocations, etc. in any model, or that the actual load is necessarily always close to what you are allocating on all links.

Perhaps some SPs do operate in the way you describe (I know of none, so it is certainly not typical or even representative), but not having any knowledge/estimate of how much traffic you're allocating to each CT makes no sense.  You should also take into account the extensive, large-scale network experience with MAR-like mechanisms in many SP networks.  See the references in the I-D for a discussion of how MAR has been used to allocate bandwidth to 'class-types' for hundreds of millions of flows per day, with superb network performance and reliability.  In those applications, radical shifts in traffic loads occur regularly, overloads, surges, failures, etc. occur.  There is nothing akin to 'fairly-slow-changing, well-understood evolution' as you have characterized.

> allowing CTs to borrow bandwidth but I think we need to somehow enhance

Per the above comments, I don't see that you have identified a real issue or as yet proposed any extensions.  

Thanks,
Jerry

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Fri, 14 Mar 2003 14:23:20 -0000

Jerry,

>> That is what we are discussing, so it is premature to draw a 
>> conclusion before the discussion, based only on your 
>> opinion.  We should continue the discussion on MAR toward a 
>> consensus. 

I certainly agree with you we should continue the discussion on MAR,
which is what I was doing in the rest of the message.

But your initial message asked for feed-back about accepting your
document as a WG document:
"I would like to get a sense of the list for using this I-D as the basis
for the MAR specification and for accepting this as a WG document.  "

I am just saying that right now it seems premature to me to turn any MAR
document into a WG document.
I'm not saying we should never have a MAR WG document, and/or that your
document would not be a good base for that.

So, more accurately, I am saying:
- if your question was intended to mean ".. for accepting this as a WG
document in some future if/when the WG has made a decision to specify
MAR", then my personal answer is I don't know yet; I need to think more
about MAR since I may not fully understand it yet.
- if your question was intended to mean ".. for accepting this as a WG
document now or in SFO", then my personal answer is that it is
premature.

Thanks 

Francois

 As stated in the I-D: 
>> "MAR is an extension of MAM and like MAM assigns a maximum 
>> bandwidth allocation to each CT, but with bandwidth 
>> reservation mechanisms, allows CTs to exceed their bandwidth 
>> allocations under conditions of no congestion but revert to 
>> their allocated bandwidths when overload and congestion 
>> occurs.  Analysis shows that MAR meets all the objectives 
>> for BC models, and that it simultaneously achieves bandwidth 
>> efficiency, bandwidth isolation, and protection against QoS 
>> degradation without preemption."
>> Most service providers (SPs) don't operate their networks in 
>> the way you suggest, that is, SPs don't put capacity out 
>> there and then decide how to allocate it to whatever traffic 
>> might show up.  To the contrary, SPs engineer their networks 
>> based on traffic projections to determine needed capacity, 
>> topology, routing, etc.  Accordingly, CT bandwidth 
>> allocations are estimated.  More sophisticated methods (in 
>> actual use) make on-line measurements of bandwidth usage as 
>> a basis for the bandwidth allocations.  
>> 
>> Once bandwidth is allocated, there is *no assumption* that 
>> this will actually represent, or be close to, the actual 
>> traffic that arrives.  Extremely large swings and surges in 
>> traffic occur, massive overloads occur, failures occur.  MAR 
>> is expected to operate well under these conditions (and does 
>> in actual operation, see below).  All the BC models in fact, 
>> should operate well under these same conditions.  There 
>> should be no presumption of 'accuracy' in bandwidth 
>> allocations, etc. in any model, or that the actual load is 
>> necessarily always close to what you are allocating on all links.
>> 
>> Perhaps some SPs do operate in the way you describe (I know 
>> of none, so it is certainly not typical or even 
>> representative), but not having any knowledge/estimate of 
>> how much traffic you're allocating to each CT makes no 
>> sense.  You should also take into account the extensive, 
>> large-scale network experience with MAR-like mechanisms in 
>> many SP networks.  See the references in the I-D for a 
>> discussion of how MAR has been used to allocate bandwidth to 
>> 'class-types' for hundreds of millions of flows per day, 
>> with superb network performance and reliability.  In those 
>> applications, radical shifts in traffic loads occur 
>> regularly, overloads, surges, failures, etc. occur.  There 
>> is nothing akin to 'fairly-slow-changing, well-understood 
>> evolution' as you have characterized.
>> 
>> Per the above comments, I don't see that you have identified 
>> a real issue or as yet proposed any extensions.  
>> 
>> Thanks,
>> Jerry
>>

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Sat, 15 Mar 2003 12:56:39 -0000

Jerry,
 
>> necessarily always close to what you are allocating on all links.

I entirely agree with the above. 

That was precisely my point. I fail to see how MAR can "simultaneously
achieves bandwidth efficiency, bandwidth isolation, and protection
against QoS degradation without preemption" in situations where loads
for each CT are quite different from allocated loads:

Imagine that Bwalloc(CT0)=10, Bwalloc(CT1)=30, Bwalloc(CT2)=60
Imagine at a given time load(CT0)=100, load (CT1)=10, load(CT2)=10.
Then I understand that with MAR:
	- CT2 will be allowed to grab 10
	- CT1 will be allowed to grab 10
	- CT0 will be allowed to grab 80-delta= say 70
So far so good.
But then let's assume the demand for CT1 and CT2 starts increasing again
without CT0 decreasing its load (which we agreed above is very possible
since CT0 load can be differnet from its allocated bandwidth), so we
have:
	- load(CT0)=80, load(CT1)=30, load(CT2). 
then in the absence of preemption I understand we will end up with
something like:
	- CT0 keeps its 70
	- CT1 will increase to say 15
	- CT2 will increase to say 15.
Is this not what would happen, more or less, with MAR?
If yes, then CT0 is hogging 70 when its engineered for 10, while CT2
only has access to 15 when it is engineered for 60. This would not be a
great result in terms of class isolation.
Now you could make "delta" bigger, this would result in better
isolation, but would also result in loss of bandwidth efficiency. For
example if delta=30, then in above example, intially CT0 will be limited
to 50. And when CT1 and CT2 later increase their load, they will be able
to grab say 25 each. This is a little better in terms of isolation (but
still far from good). And we initially had to reject a load of 30 of CT0
so that the max efficiency was 70 out of 100.

Hence my perception that :
	- MAR can "simultaneously achieve bandwidth efficiency,
bandwidth isolation, and protection against QoS degradation without
preemption" IF AND ONLY IF we make some assumpitions on the load
patterns of each CT.
	- MAR can NOT ""simultaneously achieves bandwidth efficiency,
bandwidth isolation, and protection against QoS degradation without
preemption" if we do not make assumptions about load patterns of each
CT.

What am I missing?

Thanks for your help in understanding MAR properties.

Francois

---

From: "Ash, Gerald R (Jerry), ALABS" <[email-masked]>
Date: Sat, 15 Mar 2003 21:37:05 -0600

Francois,

> What am I missing?

We've been over such examples before.  Let's not go over the same ground again, please refer to the earlier threads/responses for answers to your examples.  Basically, as I've pointed out in the earlier threads, realistic voice/data traffic patterns and network engineering rules don't happen like that.

All BC models should be designed to operate under actual network conditions, traffic, and engineering.  Let's focus on realistic examples, with realistic traffic patterns and engineering rules.  For MAR, this is what is presented in the I-D, real examples based on real traffic and engineering design.  I suggest you understand the examples in the I-D, and point out where these are `incorrect`.  That would be a more worthwhile discussion I think.

MAR is based on *operational* models in use for many years, which of course must operate well under real traffic conditions and which are engineered/designed in the way SP's normally operate their networks.  I think that's one thing you're missing.

Jerry

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Mon, 17 Mar 2003 00:50:30 -0000

Jerry,

The bottom line is that MAR only achieves the stated objectives IF AND
ONLY IF a number of asumptions are made about the CT load distribution.
We have to be upfront about it rather than pretend it always work well.
The example provided earlier is a case where MAR simply can NOT provide
any isolation across CTs.

Then, we can debate more constructively on whether the CT load
distributions which break MAR are realistic for the packet networks
which are targetted by DS-TE. 
It is far from obvious to me that they are not.
For example, a Service Provider using DS-TE in his network may only have
DS-TE tunnels for data initially and may only start building voice DS-TE
tunnels in a second stage as the VoIP load gets to a significant level.
This may result in many CTO/CT1 tunnels being setup first while CT2
tunnels only get established after. In that scenario, and assuming high
load, MAR would just NOT provide isolation across CTs (unless preemption
is used) and voice would just not get its share. Just an example. There
are other scenarios where a CT may grab more bandwidth than its share
and then not release it freely.

We should also use the same set of assumptions when assessing properties
of other models (not just for MAR).
For MAR to work well you basically need to make some assumption that one
CT will never have the opportunity to hog significantly more bandwidth
than its share and not release it later. If you make that sort of
assumptions, then you may find that RDM also works well without
preemption. 
By definition preemption is only required with RDM (or probably any
model) if you make the opposite assumption that some CT will indeed hogg
bandwidth and never release it.

Also, it would be useful if you could provide some more information on
what sort of network you are refering as a refernce for MAR (is it
multiservice, what is the granularity of a routed element for voice: a
voice call, aggregate of calls from region to region, what is the
granularity for data,...). This is to relate it to packet environments
where DS-TE is intended.

In your draft, I only saw performance comparison between MAR and full
sharing across classes (which is the same thing as saying that you have
disabled DS-TE). So this seems to be illustrating that DS-TEwMAR is
better than no DS-TE at all. Which I guess we were kind of expecting.
It does not actually yet give indication on how MAR compares to other
models. Right?
Will you be adding comparison of DSTEwMAR to DSTEwMAM and to DS-TEwRDM?

Also for the performance analysis, it would again be useful if you could
clarify what you have taken as a "flow/LSP" for your simulation (ie what
sort of aggregate) to get an idea as to the analogy with DS-TE
environments.

Another useful element for the assement of MAR would be to indicate what
protocol extensions may be required to support it in distributed mode.
For example, I assume the IGP would have to advertise Bwalloc (which
could be advertised instead of the BCs) and would also have to advertise
RBW and BWIP for each CT. No?
Also, how do you signal in RSVP-TE the CAC priority you mention
(high/mid/low) for each CT: would you defined some extension to RSVP-TE
, or redefine the preemption pririty field?
This would help understand the impact on protocols when assessing MAR.

Thank you

Francois

>> Sent: 16 March 2003 04:37
>> To: Francois Le Faucheur (flefauch)
>> Cc: Ash, Gerald R (Jerry), ALABS; [email-masked]; Lai, 
>> Wai S (Waisum), ALABS; Jim Boyle; Ed Kern (ejk)
>> Subject: RE: Progressing MAR
>> 
>> 
>> Francois,
>> 
>> > What am I missing?
>> 
>> We've been over such examples before.  Let's not go over the 
>> same ground again, please refer to the earlier 
>> threads/responses for answers to your examples.
>>  Basically, 
>> as I've pointed out in the earlier threads, realistic 
>> voice/data traffic patterns and network engineering rules 
>> don't happen like that.
>> 
>> All BC models should be designed to operate under actual 
>> network conditions, traffic, and engineering.  Let's focus 
>> on realistic examples, with realistic traffic patterns and 
>> engineering rules.  For MAR, this is what is presented in 
>> the I-D, real examples based on real traffic and engineering 
>> design.  I suggest you understand the examples in the I-D, 
>> and point out where these are `incorrect`.  That would be a 
>> more worthwhile discussion I think.
>> 
>> MAR is based on *operational* models in use for many years, 
>> which of course must operate well under real traffic 
>> conditions and which are engineered/designed in the way SP's 
>> normally operate their networks.  I think that's one thing 
>> you're missing.
>> 
>> Jerry
>>

---

From: "Ash, Gerald R (Jerry), ALABS" <[email-masked]>
Date: Mon, 17 Mar 2003 00:11:02 -0600

Francois,

> a Service Provider using DS-TE in his network may only have
> tunnels only get established after.
... 
> There are other scenarios where a CT may grab more bandwidth than its share

Your above examples of CT bandwidth allocation don't seem to be based on CT traffic load, or if they are, what are the traffic load assumptions?  Traffic load models are basic to the analysis done on RDM, MAM, and MAR.  You seem to be presenting some very different set of assumptions.

> of other models (not just for MAR).

I definitely agree.  These should include traffic load and CT bandwidth allocation/engineering assumptions.  Such assumptions have been included already in the I-Ds presenting BA model analysis.

Thanks,
Jerry