Subject: [PCN] Comments /questions on LC-PCN draft

From: <[email-masked]>
Date: Fri, 2 Nov 2007 15:57:39 -0000

I've re-started the email thread.

Thanks Georgios for your description of LC-PCN in recent email. I'm
going to try & re-phrase how it works, partly to make sure I've
understood & partly to try and re-use the PCN terminology (I admit to
finding the translations required hard work). Hence it slightly repeats
some of my earlier email, but some of its different where I now
understand your scheme better

First, let me summarise to make sure I understand it right [fingers
crossed I do!].

- two encodings are used. one ('PCN-marking') is used for both adm ctrl
& termination to indicate the excess traffic. If a PCN-interior-node is
PCN-marking some pkts, then it marks all other pkts with another
encoding ('affected marking')

- the 'PCN-marking' algorithm has several regimes:

[let us assume that none of the packets arriving at an interface are
PCN-marked]

* when traffic rate is below PCN-lower-rate, no pkts are marked 

* as the traffic rate climbs above the PCN-lower-rate, an increasing
number of pkts are marked (marking rate = excess rate divided by N).
note, this applies whatever the traffic rate (the same formula, excess
rate divided by N, is used when the traffic rate is above the
PCN-upper-rate

* Fiddle factor 1: [applies to marking when rate is below
PCN-lower-rate] if pkts arrive already PCN-marked, then the node
measures the rate of pkts already marked & works out what excess rate
this corresponds to, and only marks additional pkts if its own excess
rate is even higher.

* Fiddle factor 2: [applies to marking when rate is above
PCN-upper-rate] assume that any rate above PCN-upper-rate, as measured
in the previous measurement window period T, will be in the process of
being terminated. So can ignore this much excess (above PCN-upper-rate)
in this measurement period.

* the PCN-egress-node measures the rate of PCN-marked pkts and
determines whether the overloaded node is in adm ctrl state or
termination ctrl state

* PCN-egress-node calculates the % of PCN-bytes that are PCN-marked,
multiplying by the N factor (to compensate for the PCN-interior-nodes
only marking 1/N of their excess rate). If above a certain % then new
calls are blocked. 

* PCN-egress-node also calculates the amount (bit rate) of PCN-bytes
that are PCN-marked (again multiplying by the N factor). If this is
above the PCN-upper-rate then it terminates some flows

* for a variety of reasons the following 4 factors have to be the same
on every node in the PCN-domain: N, T (sliding window measurement
period), [PCN-upper-rate - PCN-lower-rate], [(policed rate for the PCN
class) - PCN-upper-rate].

Questions:

* imagine there are 2 independent ECMP paths for a particular
ingress-egress-aggregate, and that there's a PCN-interior-node on each
path that is somewhat pre-congested (traffic between PCN-lower-rate and
PCN-upper-rate - in 'adm ctrl state'). These add together at the
PCN-egress-node so it appears that some PCN-interior-node is highly
pre-congested (traffic above PCN-upper-rate). From your replies I
believe you agree. To try to get round this you've introduced a fiddle
factor, basically the PCN-egress-node has to see even more PCN-marks
before it really believes that flows need terminate. * the same
'affected marking' is used in both adm ctrl state & termination ctrl
state. Imagine that a node [node-1] on one ECMP path is in adm ctrl
state & a node [node-2] on another ECMP path is in termination ctrl
state. Therefore flows are terminated, and only flows that are being
PCN-marked or affected-marked are terminated. However this could lead to
flows being terminated that go through node-1; node-2 will still be just
as badly overloaded 

* one reading of these two bullets is that you haven't really solved
ECMP.

* there are a lot of parameters that need to be configured with the same
value everywhere in the PCN-domain. It would need careful thought about
what goes wrong if some of them are mis-configured on some PCN-nodes

* it's particularly foul if T (the measurement period) needs to be the
same on every PCN-node. I'm not sure this is really the case, but it
seemed to me to come out of "Fiddle factor 2"

* looking at the 2 rate parameters, [PCN-upper-rate - PCN-lower-rate],
[(policed rate for the PCN class) - PCN-upper-rate]. It seems quite
restrictive that these have to be the same on all PCN-nodes in the
PCN-domain. They might have very different capacities for instance. 

* the "fiddle factors" make the algorithm quite complicated I think for
PCN-interior-nodes. 

* "fiddle factor 2" seems the wrong approach to me - I think it would be
better to keep marking at the same rate and for the PCN-egress-node to
terminate more slowly (in order to avoid the problem of terminating too
much). At least 2 reasons: if marked pkts are lost, then at best calls
take longer to be terminated (another measurement period before get
PCN-marks again) & at worst it might be possible to devise scenarios
where termination never happens; secondly, I think it makes it easier
for the PCN-egress-node to take account of policy [operator experience
etc] to decide how fast to react to overload, whereas your approach
fixes the speed of reaction as a parameter of the PCN-interior-node.

That's it for now!

best wishes

phil/

_______________________________________________
PCN mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pcn

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Fri, 02 Nov 2007 17:14:06 +0000

Hi Phil

Thank you very much!
Please see in line!

>* when traffic rate is below PCN-lower-rate, no pkts are marked

Georgios: Yes, right
>rate divided by N, is used when the traffic rate is above the

Georgios: when the traffic rate is above the PCN-upper-rate then the
excess rate is calculated the same, but the sliding window mechanism is
used to avoid undershoot.

>this corresponds to, and only marks additional pkts if its own excess

Georgios: As I said earlier this is only partially true, see pseudo-code
on page 13, where the remarking depends on the incoming_PHR_marking_rate
but also on teh Admission_offset_rate.

>in this measurement period.

Georgios: As I mentioned earlier it is more complex than that, please see
pseudocode on page 20. The remarking also depends on the
incoming_PCN_marking_rate and on the Termination_offset_rate.

>determines whether the overloaded node is in adm ctrl state or

Georgios: Yes, right
>only marking 1/N of their excess rate). If above a certain % then new

Georgios:Yes, but when probing is used then this somehow different,
because the egress checks if the probe is marked or not.
>above the PCN-upper-rate then it terminates some flows.

Georgios: Yes, right.

>class) - PCN-upper-rate].

Georgios: Yes. Note however, that the last one that represents
Termination_offset_rate could be released from this constraint if each
PCN_interior_node is able to know the Termination_offset_rate used by
each of its neighbouring PCN_interior_nodes.

>believe you agree. 

Georgios: Yes, I agree, but do you agree that this is a corner case?

>To try to get round this you've introduced a fiddle
>before it really believes that flows need terminate. 

Georgios: Yes, you are right

* the same
>flows being terminated that go through node-1; node-2 will still be just

Georgios: Not really! Node 2 will not be just as badly overloaded. Flows
will be terminated that go through node-2 and only few flows that go
throgh node-1 could be terminated. Without using affected_marking the
severe congestion on node-2 will be in a smaller proportion solved.

Furthermore, note that affected marking in admission control state is
only required when probing is required, in order to accomplish the case
that the probe always get marked when it passes through a congested node.

>* one reading of these two bullets is that you haven't really solved

Georgios: Agree that when Affected_marking is used during flow
termination state and is also used during  admission control state, then
the ECMP solution used during flow termination state works less optimal
than the situation that the affected_marking is only used during flow
termination state and not in admission control state. However, it works
better than if you would not use Affected_marking at all.

Note that if probing is not used, then affected_marking is not used
during the admisison control state. In this way the admission control
state does not solve the ECMP case, but during flow termination, the
ECMP case can be competely solved.

>what goes wrong if some of them are mis-configured on some PCN-nodes

Georgios: Agree,
>PCN-domain. They might have very different capacities for instance.

Georgios: I think that the main problem is related to setting the
[(policed rate for the PCN class) - PCN-upper-rate] constant. Please see
above for the explanation.

>* the "fiddle factors" make the algorithm quite complicated I think for

georgios: Quite complicated compared to what?

>fixes the speed of reaction as a parameter of the PCN-interior-node.

Georgios: Please note that the PCN_egress_node changes state from
admission control to termination state if the value of the excess rate
exceeds the value of Admission_offset_rate +/- multicongestion_error.
By using worst case scenarios the value of the multicongestion_error can
be a priori estimated.

---

From: <[email-masked]>
Date: Mon, 5 Nov 2007 13:05:49 -0000

Georgios, thanks yet more in-line
phil

> -----Original Message-----
> From: Georgios Karagiannis [mailto:[email-masked]]
> Sent: 02 November 2007 17:14
> To: Eardley,PL,Philip,CXR9 R; [email-masked]
> Subject: Re: [PCN] Comments /questions on LC-PCN draft
> 
> Hi Phil
> 
> Thank you very much!
> Please see in line!
> 
> 

[phil] eh? The T factor appears whether you've above or below the
PCN-upper-rate.

> but also on teh Admission_offset_rate.

[phil] the appearance of the admission-offset-rate in page 13 pseudocode
seems to me to exactly fit the way I described fiddle factor 1. but I
think this is too detailed to be worth discussing - there are more
important points at issue.

> incoming_PCN_marking_rate and on the Termination_offset_rate.

[phil] ditto previous comment. This is probably my brain not being able
to cope with all the new terminology.

> because the egress checks if the probe is marked or not.

[phil] probing is a completely different case. 

> each of its neighbouring PCN_interior_nodes.

[phil] above answers => at least I've more or less understood it!!

> Georgios: Yes, I agree, but do you agree that this is a corner case?

[phil] not really. If the network is operating with quite a lot of
traffic so the adm ctrl mechanism is often blocking calls, it seems
likely to me that there'll be several links that are doing pcn-marking.
I don't know if there are any simulations that can confirm/deny this.
anyway, it seems very bad news if this does happen with your algorithm -
assuming that flow termination is a mechanism of last resort. 

> Georgios: Yes, you are right

[phil] which means, when there's one link in above PCN-upper-rate, that
you might not terminate flows. Which reduced the effectiveness of the
flow termination mechanism. 

> severe congestion on node-2 will be in a smaller proportion solved.

[phil] I missed a line of explanation; "However, imagine that unluckily
we happen to select for termination only flows that go through node-1
and none at all that go through node-2". Now you could say [1] this is
unlikely [2] even if it happens then in the next round there'll still be
marking and then flows through node-2 will be terminated. Two responses
[1] in the first round you've terminated flows through node-1 that don't
need to be terminated, which is annoying. [2] there may be nasty cases
where the second round you still don't terminate any flows through
node-2 - eg where all the flows are emergency calls. You keep
terminating flows that don't actually help solve the problem. 
Overall, to me this means that your solution to the ECMP problem isnt
that good. (NB I fully accept that the ECMP problem is hard to solve and
that we don't know of any really nice solutions (simple and effective).)
 
> ECMP case can be competely solved.

[phil] true. But then if the operator decides they do want to use
probing they have to change all the routers. Maybe this is ok, not sure.

> georgios: Quite complicated compared to what?

[phil] compared with what they do at the moment [eg they have a token
bucket]. 

> By using worst case scenarios the value of the multicongestion_error
can

[phil] maybe. But then in the case where only one link is pre-congested,
then the PCN mechanism is less effective because the pre-congestion has
to be higher before the mechanisms kick in.