Subject: [PCN] Feedback to draft-briscoe-tsvwg-cl-architecture-03.txt

From: Michael Menth <[email-masked]>
Date: Tue, 19 Sep 2006 12:21:05 +0200

Hi all,

I've read the draft
http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-cl-architecture-03.txt

I find the proposed concept pretty good as I consider it a simple
and technically feasible solution. Here are my comments regarding the draft.

1) Abstract: Although I am working on this field for years, I haven't
understood what the draft is about upon the first reading. Here is how
I'd summarize the presented mechanism.

This document describes a distributed, measurement-based, and resilient
admission control for application in large DiffServ-based networks.
All routers have a configured admission and pre-emption rate for their 
adjacent links.
They measure the admitted traffic they forward on each outgoing link and 
use a so-called
pre-congestion notification (PCN) mechanism to mark packets that exceed
the configured rates. The egress router measures the rates of non-labelled,
admission-marked, and preemption-marked packets from each ingress, and
notifies the respective ingress if it must not admit any further packets
or that it must even pre-empt a certain rate of admitted flows. Per-flow
admission states are kept at the edge routers only, while the PCN markers
that are required for all routers focus on the aggregate traffic. The
admission control is resilient with regard to the following two aspects:
(1) the configured admission and preemption rates can be
chosen in such a way that the reservations for admitted flows are
not pre-empted if a foreseeable failure occurs and (2) that reservations
can be torn down explicitly if a large failure occurs to preserve the
quality of service of the remaining flows.

2) The summary is very long. As the terminology is explained only in 1.2,
it is sometime hard to follow Section 1.1. It is sometimes unnecessarily
unprecise and, therefore, hard to follow. I think this could be rewritten
without increasing the length of Sections 1.1 and 1.2 such that they
it provide more clarity even at the first reading.

3) Terminology for "sustainable-aggregate-rate" is not clear. Suggestion:
rate of non-preemption marked packets measured by an egress gateway for a
specific ingress gateway; this is the supportable traffic rate between
these gateways if the corresponding path is congested.

4) Section 2.2: o Routing: a study regarding the accuracy of load
balancing schemes can be found in

[LoadBalancing-a] Ruediger Martin, Michael Menth, and Michael Hemmkeppler:
"Accuracy and Dynamics of Hash-Based Load Balancing Algorithms for
Multipath Internet Routing", IEEE Broadnets, San Jose, CA, USA,
October 2006
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth06p.pdf

[LoadBalancing-b] Ruediger Martin, Michael Menth, and Michael Hemmkeppler:
"Accuracy and Dynamics of Multi-Stage Load Balancing for Multipath
Internet Routing", currently under submission
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth07-Sub-6.pdf

Section 3 and 4: no comments, they look good and are well understandable.

5) Section 5.1:

o tunnel: what kind of tunnel? IP-in-IP tunneling from
ingress gateway to egress gateway with the effect that all flows are
forced by ECMP on the paths that is determined by the IPs of these
tunnel endpoints? This is how understood tunneling later in 5.7.1.

o probes: probe packets have the same address as the corresponding
flow. Do they finally leave the CL-domain and reach the destination?
This is unwanted extra traffic for this machine.

6) Section 5.2:

Another potential solution: smart marking

The meaning of preemption marked packets is: this is an excess packet
and the rate of the corresponding CL-region-aggregate will be reduced
in the future. Therefore, packets that are already preemption marked
should not be counted at another router for the metering process which
is done for preemption marking. This does not solve the problem in
Figure 4, but it alleviates the problem in other constellations:

                        IAR-b=1
                           @
             +-----+ L1 +-----+ L2
             |CPR=1|    |CPR=2|@@@@ SAR-b=2/3
IAR-a=2 >####| R1  |####| R2  |#### SAR-a=1/2*2/3=1/3
             +-----+    +-----+

These SARs lead in the end to a rate of 1 on link L2 although the
configured-admission-rate (CAR) is 2.
If already marked packets are not respected at R2 for the metering
process for preemption marking, we get SAR-b=1 and SAR-a=1/2 which
leads in the end to the fully used CAR on link L2.

After I wrote this, I read the draft about the markers:
http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-cl-phb-02.txt
The suggested algorithm implements the preemption marker
in fact in such a way that it respects only non-preemption-marked
packets for the measurement process. This is a requirement which
should also be mentioned in this draft.

7) Section 5.5

Another potential solution: add the declared rate to the SAR at the
ingress gateway as long as new flows send hardly any traffic. This
information must be obtained from the corresponding policers which
are also under the control of the ingress gateway. Or add it for an
initial period of e.g. 10 s?

8) Section 5.7.2
What does over-congested mean?

9) Section 5.7.4
Reminds me of RFC4127: Russian Dolls Bandwidth Constraints Model for
Diffserv-aware MPLS Traffic Engineering (and other models for that
objective)

10) Section 5.7.5
I haven't understood the last sentence of this section.
Can it be rephrased?

11) Section 6.8 *NEW*
Other Network Admission Control Approaches
Link admission control (LAC) describes how admission control (AC) can be
done on a single link and comprises, e.g., the calculation of
effective bandwidths which may be the base for a parameter-based AC.
In contrast, network AC (NAC) describes how AC can be done for a network
and focuses on the locations from which data is gathered for the admission
decision. Most approaches implement a link budget based NAC (LB NAC)
where each link has a certain AC-budget. RSVP works according to that
principle, but also the new concept admits additional flows as long as 
each link
on the new flow's path still has resources available. The border-to-border
budget based NAC (BBB NAC) pre-configures an AC budget for all
border-to-border relationships (=CL-region-aggregates) and if this
capacity budget is exhausted, new flows are rejected. The TCA-based
admission control which is associated with the DiffServ architecture
implements an ingress budget based NAC (IB NAC). These basically different
concepts have different flexibility and efficiency with regard to
the use of link bandwidths [NAC-a,NAC-b]. They can be made resilient
by choosing the budgets in such a way that the network will not be
congested after rerouting due to a failure. The efficiency of the
approaches is different with and without such resilient requirements.

[NAC-a] Michael Menth: "Efficient Admission Control and Routing in
Resilient Communication Networks", PhD thesis, July 2004,
http://opus.bibliothek.uni-wuerzburg.de/opus/volltexte/2004/994/pdf/Menth04.pdf

[NAC-b] Michael Menth, Stefan Kopf, Joachim Charzinski, and Karl
Schrodi: "Resilient Network Admission Control", currently under
submission.
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth07-Sub-3.pdf

12) Section 11.1
Paragraph "The ECN-Capable ... codepoint set." is hard to understand.
Two sentences more explanation could maybe help.

13) Section 11.3
By running the EWMA whenever a new packet for a specific
CL-region-aggregate arrives makes the aging process depending on the
arrival rate of the traffic. As a consequence, there is the problem
with the stale information. This can be done in a better way, e.g.
by using the time-exponentially weighted moving average (TEWMA) [TEWMA].

[TEWMA] Ruediger Martin and Michael Menth: "Improving the Timeliness
of Rate Measurements", in proceedings of GI/ITG Conference on Measuring,
Modelling and Evaluation of Computer and Communication Systems (MMB),
Dresden, Germany, September 2004
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth04h.pdf

If the TEWMA seems too complex (it requires the calculation of an
exponential function), a rate of the admission- or preemption-marked
packets can be calculated based on a, e.g., 20 ms interval and used
as input for a conventional EWMA.

14) Some places in the document say that admission and preemption marking
is done based on the rates of the packet AND some virtual queue size. This
is misleading from my perspective. The objective is
a) to admission-mark all packets of an CL-region-aggregate if the 
configured admission rate ist exceeded and
b) to preemption-mark only those packets exceeding the configured 
preemption rate.
The virtual queue is just a means to allow a tolerance against 
short-time rate fluctuations
like the "cell delay variation tolerance" or the "burst tolerance" in 
ATM. More on this topic
in an extra mail regarding draft-briscoe-tsvwg-cl-phb-02.txt

Best wishes,

    Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

_______________________________________________
PCN mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pcn

---

From: <[email-masked]>
Date: Tue, 19 Sep 2006 16:23:39 +0100

Michael,
Thanks very much for your very detailed comments. Some follow-ups
in-line.

I'm not quite sure how much the draft will change as a result of the new
PCN bof/wg docs - eg the problem statement will take out the
goals/assumptions section. Hopefully the overall effect will be to make
this draft smaller and more focussed. 

Best wishes, 
phil

-----Original Message-----
From: Michael Menth [mailto:[email-masked]] 
Sent: 19 September 2006 11:21
To: [email-masked]
Subject: [PCN] Feedback to draft-briscoe-tsvwg-cl-architecture-03.txt

Hi all,

I've read the draft
http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-cl-architecture-
03.txt

I find the proposed concept pretty good as I consider it a simple
and technically feasible solution. 

[phil] thanks!

Here are my comments regarding the draft.

use a so-called
the configured rates. The egress router measures the rates of
non-labelled,
admission states are kept at the edge routers only, while the PCN
markers

[phil] thanks, will try & make the next version more understandable. 

2) The summary is very long. As the terminology is explained only in
1.2,
it is sometime hard to follow Section 1.1. It is sometimes unnecessarily
unprecise and, therefore, hard to follow. I think this could be
rewritten
without increasing the length of Sections 1.1 and 1.2 such that they
it provide more clarity even at the first reading.

[phil] ok, will try & make the next version shorter & clearer. Think
this will change anyway, eg because of the set of docs for PCN BoF/WG.

3) Terminology for "sustainable-aggregate-rate" is not clear.
Suggestion:
rate of non-preemption marked packets measured by an egress gateway for
a
specific ingress gateway; this is the supportable traffic rate between
these gateways if the corresponding path is congested.

[phil] ok. I don't think the phrase " if the corresponding path is
congested" is needed?

[LoadBalancing-a] Ruediger Martin, Michael Menth, and Michael
Hemmkeppler:
"Accuracy and Dynamics of Hash-Based Load Balancing Algorithms for
Multipath Internet Routing", IEEE Broadnets, San Jose, CA, USA,
October 2006
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth06p.pdf

[LoadBalancing-b] Ruediger Martin, Michael Menth, and Michael
Hemmkeppler:
"Accuracy and Dynamics of Multi-Stage Load Balancing for Multipath
Internet Routing", currently under submission
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth07-Sub-
6.pdf

Section 3 and 4: no comments, they look good and are well
understandable.

[phil] will follow them up, thanks

5) Section 5.1:

[phil] yes, that was the idea

[phil] this is a good point. The context at this point of the doc is
'how to dael with ECMP?'. I think in the general case the answer to your
question is 'yes'. Another option is that the probe has a flag in the
header (eg specific DSCP), to indicate it's a probe & which alerts the
egress it's a probe - which it then consumes. But you have to be careful
that the ecmp algorithm doesn't include the 'probe flag' field in its
caslculation - eg I think some ecmp algorithms do include the dscp.

6) Section 5.2:

                        IAR-b=1
                           @
             +-----+ L1 +-----+ L2

[phil] ok.

7) Section 5.5

[phil] not sure about this. would the ingress do both this adm ctrl on
declared rate & adm ctrl based on the adm marking of
draft-briscoe-tsvwg-cl-architecture? How long would your "initial
period" be chosen? When it ends, what does the ingress gateway have to
do? [worried about it having to remember state, failure modes etc]

8) Section 5.7.2

[phil] typo for congested, I think

9) Section 5.7.4

[phil] yes, reminiscent. Also, the idea in [Songhurst] would mean that
the amount of CL traffic that could be admitted depends on the amount of
non-CL traffic present; this isn't true for [I think] for the Russian
dolls model, and not true for the mechanism discussed in
draft-briscoe-tsvwg-cl-architecture

10) Section 5.7.5

[phil] sure. The sentence is meant to provide guidance on what the adm
marking algorithm should be like; it's saying that in Fig 3 in
http://www.ietf.org/internet-drafts/draft-briscoe-tsvwg-cl-phb-02.txt
the slope should be steep. 

11) Section 6.8 *NEW*

[phil] (this is in Section 6, 'Relationship to other QoS mechanisms').
Yes, can add something like this - thanks. 

and focuses on the locations from which data is gathered for the
admission
on the new flow's path still has resources available. The
border-to-border
implements an ingress budget based NAC (IB NAC). These basically
different

[NAC-a] Michael Menth: "Efficient Admission Control and Routing in
Resilient Communication Networks", PhD thesis, July 2004,
http://opus.bibliothek.uni-wuerzburg.de/opus/volltexte/2004/994/pdf/Ment
h04.pdf

[NAC-b] Michael Menth, Stefan Kopf, Joachim Charzinski, and Karl
Schrodi: "Resilient Network Admission Control", currently under
submission.
http://www3.informatik.uni-wuerzburg.de/~menth/Publications/Menth07-Sub-
3.pdf

12) Section 11.1

[phil] ok

13) Section 11.3

[phil] will read. What's not clear to me at this stage is whether we'll
end up standardising a specific averaging algorithm, or saying something
like "must average, but choice of algorithm is left open", or something
else. 

14) Some places in the document say that admission and preemption
marking
is done based on the rates of the packet AND some virtual queue size.
This

[phil] yes, this is a bit misleading, please could you point out where
this is said. It shouldn't say ' marking is done based on rate of packet
AND virtual queue size'. I know sometimes instead of talking about the
virtual queue, the draft [especially early on] uses a phrase like
'marking depends on the amount of traffic' - this was an attempt not to
get in to a specific implementation like a virtual queue. 
yes, you're right about the reason for the virtual queue (it's also so
the operator is free to choose any configured-admission-rate they like).
Probably should point out more explicitly?
Look forward to your other email, thanks for the feedback!

Best wishes,

    Michael

---

From: Michael Menth <[email-masked]>
Date: Wed, 20 Sep 2006 09:14:08 +0200

Dear Philip,
> [phil] ok. I don't think the phrase " if the corresponding path is
[Michael] I think it is needed and also how it is measured to better 
understand what it means. This is useful as one looks it up in the 
terminology section to exactly know what it is.

> draft-briscoe-tsvwg-cl-architecture? 
[Michael] Yes, as long as the already admitted flows are not reflected 
in the measurements. Otherwise this would result in problems with flash 
crowds. I have detailed this issue including the solution in my response 
to Anna.
> How long would your "initial period" be chosen? 
[Michael] Good question. I think this leaves room for research to be done.
> When it ends, what does the ingress gateway have to
[Michael] When the initial period ends, the ingress router just doesn't 
take the declared rate of the respective flow anymore into account as 
its rate is now reflected by the measurements.
> like "must average, but choice of algorithm is left open", or something
[Michael] I think it is good to leave it open, but there should be also 
a recommendation how to do it in a network. From my perspective, this is 
an open issue. However, it is probably igood to use the same averaging 
and metering mechanism together with the same parameters in all network 
devices for the sake of comparability.

Best wishes,

    Michael