Subject: Re: [PCN] draft-ietf-pcn-architecture-03

From: <[email-masked]>
Date: Wed, 5 Mar 2008 15:26:05 -0000

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf Of
Lars
> Eggert
> Sent: 27 February 2008 09:17
> To: ext Geib, Ruediger
> Cc: [email-masked]
> Subject: Re: [PCN] draft-ietf-pcn-architecture-03
> 
> Hi,
> 
> On 2008-2-27, at 10:45, ext Geib, Ruediger wrote:
> > [Lars] It is important to discuss somewhere that some choices here
> > are less
> >   optimal than others. A good decision-making location is where PCN
> >   information naturally accumulates, e.g. at the egress. Any
> > relocation
> >   of that decision-making requires additional communication, which
> >   increases the delay of the control loop and makes the system more
> >   brittle, because correct operation now requires dependable
> >   communication between the egress and the decision-making point.
I'd
> >   like to suggest that we attempt to come to consensus on whether we
> > can
> >   focus on a single decision point for PCN.
> >
> > [RG] The per flow signaling must reside at the ingress by our actual
> > specification (see section 1, first bullet point) and to make
> > admission
> > control work, the egress must signal to the ingress anyway. It is
> 
> (Sentence truncated?)
> 
> I see your point. Thinking about this some more, I believe my comment
> was based on a different understanding on something that the
> architecture document currently doesn't discuss much, but probably
> should: What information is signaled from the egress to the ingress?
> 
> We agree that the egress looks at the volume of PCN marks compared to
> the volume of total PCN traffic to determine the level of congestion
> on an ingress-egress pair. The ingress then admits or terminates flows
> in a way that is somehow based on that information. But what gets
> signaled from egress to ingress? Is it some sort of load indication?
> Or is it a desired behavior, i.e., "stop admit" or "start preempt"? Or
> something else?
> 

how about the following for admission ctrl:
- the egress signals to the ingress "admit" or "block" on a per flow
basis.
- the egress bases this decision on the level of congestion on the
ingress-egress-aggregate, using an EWMA or similar
- the egress has been told what the "threshold" level of congestion is
(ie threshold for admit vs block decision). This could be a signalling
msg from the ingress of some configuration parameter. 
- if there's been no traffic from this ingress recently, then the "level
of congestion" is based simply on whether this first pkt (which is the
flow adm request msg) is marked or not. (those who believe that the
marking of a single pkt contains as much info as the level of congestion
(= markings of lots of pkts) simply set their EWMA weight to 1.)
- the ingress abides by the egress signal to admit or block, unless it
has to over-ride for some good reason eg perhaps instructed by
management system to block all calls at the moment. 
- the above doesn't allow an approach where the egress signals to the
ingress "admit (/block) for x secs, unless I tell you otherwise".
(Michael has mentioned this approach, although for me it's rather
difference in terms of performance implications on system reaction time,
impact of lost msgs etc.)
- there's an implication in the above that we indeed have a msg from the
ingress to the egress. Is this reasonable assumption?

for termination we have 2 quite different approaches (measuring rates vs
EMFT), so convergence here is trickier. 
Thanks,
phil
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: Michael Menth <[email-masked]>
Date: Wed, 05 Mar 2008 17:33:58 +0100

Hi Phil and others,

> - the egress signals to the ingress "admit" or "block" on a per flow
Yes, good idea as it decouples operation of ingress and egress nodes.

> ingress-egress-aggregate, using an EWMA or similar
For an architecture draft: " on the marking ". Details can be described 
by a different document.

> msg from the ingress of some configuration parameter. 
Yes, CLE-based Admission Control requires two thresholds: An 
admission-continue threshold and an admission-stop threshold such that 
appropriate messages can be triggered towards the PCN ingress.

> (= markings of lots of pkts) simply set their EWMA weight to 1.)
Well, this is already a kind of probing that reliably works only with 
exhaustive (threshold) marking. With excess marking this probably is 
equivalent to assuming the admit mode for empty aggregates. At least 
it's a safe option since the aggregate can admit another flow on which 
further CLE can be based.

> impact of lost msgs etc.)
You probably mean observation-based AC: egress sends admission-stop msg 
as soon as it sees first marked packet; it sends an admission-continue 
msg when it hasn't seen a marked packet for a minimum block interval. I 
don't think it's a contradiction. In general, refresh messages or other 
stuff can be sent without changing the general behavior of the AC method.

> ingress to the egress. Is this reasonable assumption?
We also need a mechanism for bootstrapping which is equivalent to an 
empty system. However, this can be simple such as empty = admit; This is 
problematic when we have lots of small edge-to-edge aggregates that 
contribute to a large aggregate in the core such that the aggregation 
condition is met.

One important question is: should PCN work well for edge-to-edge 
aggregates of 1, 100, 10000 flows? I am talking here about edge-to-edge 
aggregation, not about the number of flows on a link which may be 
significantly higher.

Regards,

    Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

---

From: "Geib, Ruediger" <[email-masked]>
Date: Thu, 6 Mar 2008 09:41:10 +0100

Phil, Michael,

| -----Original Message-----
| From: Michael Menth [mailto:[email-masked]] 
| Sent: Wednesday, March 05, 2008 5:34 PM
| To: [email-masked]
| Cc: [email-masked]; Geib, Rüdiger; [email-masked]
| Subject: Re: [PCN] draft-ietf-pcn-architecture-03
| 
| Hi Phil and others,
| 
| 
| [email-masked] wrote:
| >   
| >> -----Original Message-----
| >> From: [email-masked] [mailto:[email-masked]] 
| On Behalf Of
| >>     
| > Lars
| >   
| >> Eggert
| >> Sent: 27 February 2008 09:17
| >> To: ext Geib, Ruediger
| >> Cc: [email-masked]
| >> Subject: Re: [PCN] draft-ietf-pcn-architecture-03
| >>
| >> Hi,
| >>
| >> On 2008-2-27, at 10:45, ext Geib, Ruediger wrote:
| >>     
| >>> [Lars] It is important to discuss somewhere that some 
| choices here 
| >>> are less
| >>>   optimal than others. A good decision-making location is 
| where PCN
| >>>   information naturally accumulates, e.g. at the egress. Any 
| >>> relocation
| >>>   of that decision-making requires additional communication, which
| >>>   increases the delay of the control loop and makes the 
| system more
| >>>   brittle, because correct operation now requires dependable
| >>>   communication between the egress and the decision-making point.
| >>>       
| > I'd
| >   
| >>>   like to suggest that we attempt to come to consensus on 
| whether we 
| >>> can
| >>>   focus on a single decision point for PCN.
| >>>
| >>> [RG] The per flow signaling must reside at the ingress by 
| our actual 
| >>> specification (see section 1, first bullet point) and to make 
| >>> admission control work, the egress must signal to the ingress 
| >>> anyway. It is
| >>>       
| >> (Sentence truncated?)
| >>
| >> I see your point. Thinking about this some more, I believe my comment 
| >> was based on a different understanding on something that the 
| >> architecture document currently doesn't discuss much, but probably
| >> should: What information is signaled from the egress to the ingress?
| >>
| >> We agree that the egress looks at the volume of PCN marks compared to 
| >> the volume of total PCN traffic to determine the level of congestion 
| >> on an ingress-egress pair. The ingress then admits or terminates 
| >> flows in a way that is somehow based on that information. But what 
| >> gets signaled from egress to ingress? Is it some sort of load indication?
| >> Or is it a desired behavior, i.e., "stop admit" or "start preempt"? 
| >> Or something else?
| >>
| >>     
| >
P:> how about the following for admission ctrl:
| > - the egress signals to the ingress "admit" or "block" on a per flow 
| > basis.
| >   
M:Yes, good idea as it decouples operation of ingress and egress nodes.

[RG]:This isn't in line with the current architecture as is (per flow state 
just in the ingress node and only ingress/egress aggregate measurements in 
the egress). Please clarify which new egress functionalities are required.

| 
| 
P:> - the egress bases this decision on the level of congestion on the 
| > ingress-egress-aggregate, using an EWMA or similar
| >   
M:For an architecture draft: " on the marking ". Details can be 
| described by a different document.

[RG] Yes.

| 
P:> - the egress has been told what the "threshold" level of congestion is 
| > (ie threshold for admit vs block decision). This could be a signalling 
| > msg from the ingress of some configuration parameter.
| >   
M:Yes, CLE-based Admission Control requires two thresholds: An 
| admission-continue threshold and an admission-stop threshold 
| such that appropriate messages can be triggered towards the 
| PCN ingress.

[RG] I'd favour configuration by management. Signaling from the ingress sounds
a bit strange to me. What's trhe rational behind your idea, Phil?

| 
| 
P:> - if there's been no traffic from this ingress recently, then the 
| > "level of congestion" is based simply on whether this first pkt (which 
| > is the flow adm request msg) is marked or not. (those who believe that 
| > the marking of a single pkt contains as much info as the level of 
| > congestion (= markings of lots of pkts) simply set their EWMA weight 
| > to 1.)
| >   
M:Well, this is already a kind of probing that reliably works 
| only with exhaustive (threshold) marking. With excess marking 
| this probably is equivalent to assuming the admit mode for 
| empty aggregates. At least it's a safe option since the 
| aggregate can admit another flow on which further CLE can be based.

[RG] For a flow with silence suppression or the like, it may not be a 
request message. No probing, at least not if just admission control is 
active. Just take the "safe option" just admit. I'm allright with limiting 
the number of admitted new flows to empty aggregates based on a timer 
or so however (see also below).

| 
P:> - the ingress abides by the egress signal to admit or block, unless it 
| > has to over-ride for some good reason eg perhaps instructed by 
| > management system to block all calls at the moment.
| > - the above doesn't allow an approach where the egress signals to the 
| > ingress "admit (/block) for x secs, unless I tell you otherwise".
| > (Michael has mentioned this approach, although for me it's rather 
| > difference in terms of performance implications on system reaction 
| > time, impact of lost msgs etc.)
| >   
| You probably mean observation-based AC: egress sends 
| admission-stop msg as soon as it sees first marked packet; it 
| sends an admission-continue msg when it hasn't seen a marked 
| packet for a minimum block interval. I don't think it's a 
| contradiction. In general, refresh messages or other stuff 
| can be sent without changing the general behavior of the AC method.

[RG] I like Michaels idea. Maybe the number of new flows admitted should 
be limited after the egress indicated the first received pre-congestion 
marked packet from the egress. Unless the egress sends a second message 
to keep up the pre-congestion status, there could be a time out in the 
ingress after which the ingress continues admission as usual.

| 
| 
P:> - there's an implication in the above that we indeed have a msg from 
| > the ingress to the egress. Is this reasonable assumption?
| >   
M:We also need a mechanism for bootstrapping which is 
| equivalent to an empty system. However, this can be simple 
| such as empty = admit; This is problematic when we have lots 
| of small edge-to-edge aggregates that contribute to a large 
| aggregate in the core such that the aggregation condition is met.
| 
| One important question is: should PCN work well for 
| edge-to-edge aggregates of 1, 100, 10000 flows? I am talking 
| here about edge-to-edge aggregation, not about the number of 
| flows on a link which may be significantly higher.

[RG] It is a measurement based admission control mechanism and as such, 
it requires traffic to work well. If the measurement interval is small like 
1 ms, you'll need a higher bit rate and a number of flows per aggregate to 
see a packet with some probability. If the measurement interval is 10 s, 
you have a fair chance to see a packet with fewer flows or lower bit rates. 

We should consider whether we provide informational operational hints or 
whether we say, engineering PCN is an operator issue.  

Regards, Rudiger

| 
| Regards,
| 
|     Michael
| 
| > for termination we have 2 quite different approaches 
| (measuring rates 
| > vs EMFT), so convergence here is trickier.
| > Thanks,
| > phil
| >   
| > _______________________________________________
| > PCN mailing list
| > [email-masked]
| > https://www.ietf.org/mailman/listinfo/pcn
| >   
| 
| --
| Dr. Michael Menth, Assistant Professor
| University of Wuerzburg, Institute of Computer Science Am 
| Hubland, D-97074 Wuerzburg, Germany, room B206
| phone: (+49)-931/888-6644, fax: (+49)-931/888-6632 
| mailto:[email-masked]
| http://www3.informatik.uni-wuerzburg.de/research/ngn
| 
| 
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: Michael Menth <[email-masked]>
Date: Thu, 06 Mar 2008 09:57:18 +0100

Hi Rüdiger,

> the egress). Please clarify which new egress functionalities are required.

Ooops, sorry! Yes, Ruediger, you are absolutely right. I mean that based 
on per aggregate evaluation, the egress node signals to the ingress 
"admit" or "block" such that the ingress node can update its state to 
admit or block further flows. That's what I mean, anything else is of 
course out of scope. Probably the same happened to Phil?

> a bit strange to me. What's trhe rational behind your idea, Phil?
Yes, that's also my view.

> you have a fair chance to see a packet with fewer flows or lower bit rates.
The duration of the measurement interval is critical as it delays 
potential blocking. This is not bad in a stationary scenario where only 
a few flow request are above the admissible rate. AC is most important 
and most likely to kick in in case of flash crowds. Therefore, it is 
crucial that the reaction time is below 500 ms. Other opinions, also on 
the parameter?

> whether we say, engineering PCN is an operator issue.  

For further performance studies, statements from operators are helpful 
for which aggregation level PCN AC should work. I remember a survey from 
Phil, but I don't know the details anymore. I believe that there are 
edge-to-edge aggregate with low aggregation we have to care for although 
the aggregation on core links is high. Is that correct? What were the 
numbers?

Regards,

    Michael

---

From: "Geib, Ruediger" <[email-masked]>
Date: Thu, 6 Mar 2008 11:18:27 +0100

Hi Michael,

architecture plays a role here too. The information 10 000 
loactions some bandwidths, some traffic matrix may be 
insufficient.

If I look at telephony, there are systems like SBCs
in our network and the other ingress/egress nodes are 
broadband access routers or interconnection routers, 
numbering say 400. Let's assume distributed video servers 
closer to the end user and connected similar as an SBC. Assume 
a fair share of the videos to be local/regional top down 
communication. 

Flows pass Access Router, SBC and Access Router. One of 
the routers could also be an interconnection router.

There's no per flow state in routers. SBCs and Video Servers
are flow aware. SBC's are fully meshed and some have links 
to interconnection points.

Bandwidths for real time services are in the order of 
500 Mbit/s for Voice and Video services per link. A voice 
call would be 100 kbit/s, a video stream no more than 10 Mbit/s.

Let's assume each SBC and Video Server to be connected by 1 GE 
or bigger, full capacity available for PCN traffic.

The SBC is a signaling and traffic relay for applications, the 
video server terminates signaling. Assume the video server to 
be part of the PCN domain, also in the case of a viewer downloading 
an exotic video from a remote site.

The network is redundant in all sections (including 
interconnection).

If you look for a network architecture, Deutsche Telekom publishes 
some information on that - but you can choose any other network 
for that too (eg Geant). I don't want to publish a link for one 
of our product servers here. 

I can't give you a traffic matrix. But there's more communication 
between economic centers for voice, I guess. For video streaming 
I have to pass. I don't know how many people are interested in the 
same movies at the same time. And the consumer preferences are 
different in different countries (languages, dubbing acceptance...).

Regards,

Rudiger

[snipped all other stuff]

[RG] We should consider whether we provide informational operational hints 
| > or whether we say, engineering PCN is an operator issue.
| >   
| 
M:For further performance studies, statements from operators 
| are helpful for which aggregation level PCN AC should work. I 
| remember a survey from Phil, but I don't know the details 
| anymore. I believe that there are edge-to-edge aggregate with 
| low aggregation we have to care for although the aggregation 
| on core links is high. Is that correct? What were the numbers?
| 
| Regards,
| 
|     Michael
| 
| > Regards, Rudiger
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: <[email-masked]>
Date: Thu, 6 Mar 2008 11:26:21 -0000

> the egress). Please clarify which new egress functionalities are
required.

I think this is in-line with the current architecture. The egress
doesn't need to remember the adm ctrl decision as it trusts the ingress
to do what it tells it. if that's the state you were thinking of?

> | aggregate can admit another flow on which further CLE can be based.

True, the accuracy of the adm decision depends on what sort of marking
the interior nodes use for adm ctrl (excess vs threshold) - although I
think the egress actually does the same thing for both (use CLE if have
it, otherwise look at marking on this flow adm request msg. In both
cases, with excess marking (Single Marking) on interior nodes the
reliability of the adm decision is worse - so the operator has to accept
this [my view at the moment is that it's too inaccurate].)

> request message. 
Don't understand what silence suppression has to do with request msg.

No probing, at least not if just admission control is
> | can be sent without changing the general behavior of the AC method.

can we discuss this at ietf? I'm not against this approach in principle,
I just worry whether we should focus more to at least get something done
that has well understood performance, and add other bits later. 

> ingress after which the ingress continues admission as usual.

This is true. Question is what are the risks for an operator, what
advice is there (again: I think this is a reasonable approach, but it's
a different one to get the IETF community to understand during pre-RFC
review stages. Should discuss whether this is realistic (I'm open to
being persuaded either way!) 

> you have a fair chance to see a packet with fewer flows or lower bit

Rudi - I don't quite agree. The most important thing is the number of
pkts on the interior router that's in danger of being congested. My
opinion is that it should work ok however many flows there are on the
particular ingress-egress-aggregate (if one imagines a core network with
say 100 PCN-boundary-nodes and a typical voice traffic matrix, then
there'll be lots of ingress-egress pairs with no voice flows at any one
instant). however, I don't think this implies we need to generate lots
of "probe traffic" - reacting to a single pkt (ie the signalling msg) is
ok (ben has convinced me of this, he's done analytical simulations,
although I think talking about them at this ietf will probably just
distract from the convergence Lars wants - hopefully he may be
persuadable to come to Dublin!)

---

From: "Geib, Ruediger" <[email-masked]>
Date: Thu, 6 Mar 2008 12:57:51 +0100

Hi Phil,

I decompose that a little. An appointment in 15 minutes leaves time for 
one response (in line).

Regards, Rudiger

| -----Original Message-----
| From: [email-masked] [mailto:[email-masked]] 
| Sent: Thursday, March 06, 2008 12:26 PM
| To: Geib, Rüdiger; [email-masked]
| Cc: [email-masked]
| Subject: RE: [PCN] draft-ietf-pcn-architecture-03
| 
| > P:> how about the following for admission ctrl:
| > | > - the egress signals to the ingress "admit" or "block" on a per flow
| > | > basis.
| > | >
| > M:Yes, good idea as it decouples operation of ingress and egress nodes.
| > 
| > [RG]:This isn't in line with the current architecture as is (per flow 
| > state just in the ingress node and only ingress/egress aggregate measurements in
| > the egress). Please clarify which new egress functionalities are required.
| 
P:I think this is in-line with the current architecture. The 
| egress doesn't need to remember the adm ctrl decision as it 
| trusts the ingress to do what it tells it. if that's the 
| state you were thinking of?

[RG] Your idea is that the egress measures active flows and the definition of active 
is "those for which packets were within the sliding window". That's ok in principle, 
but still, the egress now measures per flow, not per aggregate. Or is your suggestion 
to measure per aggregate, then detect marked packets, identify the flows they belong to
and then signal decisions on them to the ingress?

I don't understand how the per flow admission decision should work in the egress, 
please describe this in more detail.

---

From: <[email-masked]>
Date: Thu, 6 Mar 2008 13:07:51 -0000

> 
> [RG] Your idea is that the egress measures active flows and the
definition
> of active
> is "those for which packets were within the sliding window". That's ok
in
> principle,
> but still, the egress now measures per flow, not per aggregate. Or is
your
> suggestion
> to measure per aggregate, then detect marked packets, identify the
flows
> they belong to
> I don't understand how the per flow admission decision should work in
the
> egress,

egress gets an adm request msg (as per normal). Looks up what the
ingress is (eg if it's rsvp this is the PHOP).
Egress then looks what congestion info it has for this ingress:
If it has a CLE (because there's already data flowing from this ingress
on other sessions), base adm decision on what the CLE is. 
If it doesn't have a CLE, see whether this pkt (adm request msg) is
marked. If it's marked adm decision is Reject. If it isnt, adm decision
is Accept.
Inform ingress of adm decision. 

This is the same as CL documentation, except that:
- the egress reported the CLE to the ingress, which then made the
decision. No longer necessary as egress makes decision. Implication of
new proposal is that egress has to know what "threshold CLE" is
(above/below => block/admit); you suggest this is a configured
parameter, which seems ok to me.
- if there wasn't a CLE (no data flowing from this ingress) then it
asked the ingress to send "probes" ie send dummy data in order to
generate CLE. Implication is that egress bases decision on whether a
single pkt is marked (the adm request msg). 
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn

---

From: "Geib, Ruediger" <[email-masked]>
Date: Fri, 7 Mar 2008 09:05:08 +0100

Phil, 

| > [RG] Your idea is that the egress measures active flows and the definition
| > of active is "those for which packets were within the sliding window". 
| > That's ok in principle, but still, the egress now measures per flow,  
| > not per aggregate. Or is your suggestion to measure per aggregate, 
| > then detect marked packets, identify the flows they belong to
| > and then signal decisions on them to the ingress?
| > 
| > I don't understand how the per flow admission decision should work in
| > the egress, please describe this in more detail.
| > 

[Phil]
| egress gets an adm request msg (as per normal). Looks up what 
| the ingress is (eg if it's rsvp this is the PHOP).

[RG] Umm maybe I'm slow here but putting this all together means:
- the egress node supports RSVP signaling to the extent that it 
  is a PHOP, supports signaling and reads the flow identifiers.
- the egress node does however not support any of the per flow 
  conditioning features required by RSVP.

Is that it? If yes, it may be useful to emphasize the concept a 
little more in the architecture. I don't object to the idea.

Regards,

Rudiger

| Egress then looks what congestion info it has for this ingress:
| If it has a CLE (because there's already data flowing from 
| this ingress on other sessions), base adm decision on what 
| the CLE is. 
| If it doesn't have a CLE, see whether this pkt (adm request 
| msg) is marked. If it's marked adm decision is Reject. If it 
| isnt, adm decision is Accept.
| Inform ingress of adm decision. 
| 
| This is the same as CL documentation, except that:
| - the egress reported the CLE to the ingress, which then made 
| the decision. No longer necessary as egress makes decision. 
| Implication of new proposal is that egress has to know what 
| "threshold CLE" is (above/below => block/admit); you suggest 
| this is a configured parameter, which seems ok to me.
| - if there wasn't a CLE (no data flowing from this ingress) 
| then it asked the ingress to send "probes" ie send dummy data 
| in order to generate CLE. Implication is that egress bases 
| decision on whether a single pkt is marked (the adm request msg). 
| 
_______________________________________________
PCN mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/pcn