Subject: [re-ECN] implementations

From: Leslie Daigle <[email-masked]>
Date: Thu, 15 Oct 2009 18:21:44 -0400

Hi,

Following up an off-list suggestion -- we've heard passing references to 
re-ecn implementations, but it would probably be good to be able to be a 
bit more authoritative.

If you've done such implementation work -- I'm happy to collect 
references here and at least have a slide up in Hiroshima to list 
them/provide further pointers for people.

Thanks,
Leslie.

-- 

-------------------------------------------------------------------
"Reality:
      Yours to discover."
                                 -- ThinkingCat
Leslie Daigle
[email-masked]
-------------------------------------------------------------------

---

From: <[email-masked]>
Date: Fri, 16 Oct 2009 09:48:11 +0100

Hi Leslie,

I have a working ns2 implementation of re-ECN (as described in the
current protocol document). It differs from that specification in 2
aspects: firstly it ignores packet size when marking, secondly it lacks
the compatibility mode at present.

Toby

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Leslie Daigle
> Sent: 15 October 2009 23:22
> To: [email-masked]
> Subject: [re-ECN] implementations
> 
> 
> Hi,
> 
> _______________________________________________
> re-ECN mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/re-ecn

---

From: <[email-masked]>
Date: Fri, 16 Oct 2009 10:00:28 +0100

BT has produced an implementation on Linux. It has specifically been
built on kernel version 2.6.27.7, but would work on any version where
the patched files are not altered from this version. 

It has a simplified version of byte accounting i.e it attempts to
forward mark packets to an equivalent number of bytes to those that have
been congestion marked. It does this by keeping a record of the largest
segment transmitted and assuming that feedback of a congestion mark
relates to the largest packet transmitted so far. This is a conservative
approach, ensuring that the implementation does not under report
expected congestion.

Re-ECN interworking with ECN (Re-ECN co-mode) has also been implemented.

Availability is via email to me.

Alan

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf
Of Leslie Daigle
Sent: 15 October 2009 23:22
To: [email-masked]
Subject: [re-ECN] implementations

Hi,

bit more authoritative.

Thanks,
Leslie.

--

---

From: Mirja Kühlewind <[email-masked]>
Date: Fri, 16 Oct 2009 13:00:05 +0200

Hi,

I have an implementation on a Linux kernel 2.6.26. It implements the whole 
specification but with (just) packet counting.

I'm using the implementation with the Network Simulation Cradle (NSC) and our 
own simulation library.

Mirja

---

From: "Don Bowman" <[email-masked]>
Date: Sun, 18 Oct 2009 11:45:52 -0400

From: Leslie Daigle

> Following up an off-list suggestion -- we've heard passing references

> to

> re-ecn implementations, but it would probably be good to be able to be

> a

> them/provide further pointers for people.

Sandvine has an 'implementation in spirit' we call fairshare traffic
management.

We detect the congestion in the forward path (implicitly, not
explicitly),

then act to reduce the priority of the users or applications which are
disproportionately

causing it.

Although specific implementations @ customers & access technologies
vary, those

basic principles are always maintained. The most common implementation
is

to enforce a 'volume accounting fairness' per subscriber, enforced only

during times of congestion. Variation on that theme include
sub-prioritisation

by application (so that the overall envelope of each user is balanced

against each other, and then within that envelope their applications are
balanced).

The most common implementation is to reduce scheduling priority, without

placing any fixed cap or capacity reduction. This has the affect in a
network

which is not dramatically under-provisioned of shifting the 'cost' to
the heavier

users, where the 'cost' is related to jitter, latency, and loss.

This is deployed in a large (well, large to us) number of residential
consumer

Access networks world-wide.

I understand it differs from the proposal for 'ConEx', i think that the
explicit

Exposure of congestion will be beneficial for us.

I'd be happy to talk in much more detail about this in japan.

Apologies for the HTML email and the image, but the below is an example
of the impact. 

The image shows a cumulative distribution of the round-trip latency
through the

Access network for 2 classes of user. The 'PBE' is the
priority-best-effort, the

Users who are not assigned any 'cost' for causing congestion.

The 'BE' is the best-effort group, those who are assigned the cost of
congestion.

In this particular network, the packet loss of both groups was
negligible (although

We've found packet loss to be unreliable means of measuring congestion

due to TCP rate control).

As you can see from the chart, the PBE group enjoys a situation where
almost

100% of their flows have the same latency. The BE group has a situation

where there is more significant jitter and variation in latency.

---

From: Matt Mathis <[email-masked]>
Date: Fri, 23 Oct 2009 15:01:01 -0400

Please explain "We’ve found packet loss to be unreliable means of measuring
congestion due to TCP rate control."  How do you define and measure
congestion?

Thanks,
--MM--
-------------------------------------------
Matt Mathis      http://staff.psc.edu/mathis
Work:412.268.3319   Home/Cell:412.654.7529
-------------------------------------------
Evil is defined by mortals who think they know
"The Truth" and use force to apply it to others.

On Sun, Oct 18, 2009 at 11:45 AM, Don Bowman <[email-masked]> wrote:

> We detect the congestion in the forward path (implicitly, not explicitly),
>
> Although specific implementations @ customers & access technologies vary,
> those
>
> basic principles are always maintained. The most common implementation is
>
> which is not dramatically under-provisioned of shifting the 'cost' to the
> heavier
>
> I understand it differs from the proposal for ‘ConEx’, i think that the
> explicit
>
> Apologies for the HTML email and the image, but the below is an example of
> the impact.
>
> The image shows a cumulative distribution of the round-trip latency through
> the
>
> Access network for 2 classes of user. The ‘PBE’ is the
> Users who are not assigned any ‘cost’ for causing congestion.
>
> The ‘BE’ is the best-effort group, those who are assigned the cost of
> congestion.
>
> In this particular network, the packet loss of both groups was negligible
> (although
>
> We’ve found packet loss to be unreliable means of measuring congestion
>

---

From: "Don Bowman" <[email-masked]>
Date: Fri, 23 Oct 2009 15:41:24 -0400

From: Matt Mathis [mailto:[email-masked]] 

>Please explain "We've found packet loss to be unreliable means 
>of measuring congestion due to TCP rate control."  How do you define 
>and measure congestion?

For the definition:

Congestion is a function of the buffering in networking equipment. When there is contention for an output link (i.e. instantaneously more packets want to be transmitted than there is capacity for), the buffer starts to fill. This buffering increases and adds variability to latency, and thus can create quality problems for interactive applications. As the buffers increase in depth eventually they overflow and cause packet loss. Packet loss is a normal part of a network since it is the mechanism by which TCP governs its throughput. Thus for the loss part of congestion we would define it as the situation in which an increase in data transmissions results in a proportionally smaller or even a reduction in throughput. 
A non-congested network is one in which the latency (end-to-end delay) is relatively constant, and has little packet loss.
Congestion is also important to be framed by the user experience. A situation of congestion which causes an instant message to be delivered 500ms later is irrelevant to the average user. The same delay on a gaming packet or VoIP packet is perceived as a full loss by the consumer.

Sandvine would therefore define congestion on a per-application class basis as the variability in delay or packet loss beyond what the application can withstand without the user noticing. The Wikipedia article @ http://en.wikipedia.org/wiki/Congestion_collapse provides a good description based on the assumption that all applications perceive congestion the same.

So from this, congestion is the point @ which if an application attempts to go faster, it gets no real return for the attempt, and the quality delivered to the user suffers.

Now, for use of TCP packet loss.
Since there is more than one TCP stack implementation in use, and they vary in aggressiveness of ramp up/down, and since the network is shared by so many sessions, and since congestion comes at many hops along the way but TCP packet loss @ one location doesn't show where it occurred, you have a blended mix. The loss may be in the wifi in the home, in the home gateway, in the dsl modem, in the dslam, in the bras, atm aggregation, aggregation routers, core routers, peering, transit, etc. It may be due to the user's own traffic, due to unreliable RF signals, due to other users traffic, due to network switchover, etc.

So when we modelled TCP packet loss and correlated it against congestion measured as % of link utilisation @ each choke point, and eyeballed the quality of some key applications, we found the correlation to be not that great.

A related question relates to the use of policing. If i put a 10Mbps policer on a network that is capable of 100Mbps, I do not see 90Mbps of 'drops' from the policer, i typically see 0. Therefore i cannot infer what the bandwidth need would be if i remove the policer (it might still be 10, it might go to 100). Thus the packet loss cannot be used to infer the bandwidth desired.

--don

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 26 Oct 2009 02:54:53 +0000

Don,

One motivation I always highlight for using ECN rather than drop to 
indicate congestion is that it breaks the link between congestion and 
impairment.

Because ECN keeps queues short, it largely removes jitter & loss. It 
can then be used as an app-independent measure of *approaching* 
congestion. You then get a QoS meshanism for free, because you've got 
rid of delay & loss.

That's not to say there's not value in trying to manage traffic using 
contribution to loss rather than ECN. However, you have to be careful 
how you sell it - it makes sense to say you're capping contribution 
to loss, but if not carefully explained customers think it sounds 
perverse to pay for losses.

Queries/disagreements with stuff below (long); inline...

At 19:41 23/10/2009, Don Bowman wrote:
>Congestion is a function of the buffering in networking equipment. 
>When there is contention for an output link (i.e. instantaneously 
>more packets want to be transmitted than there is capacity for), the 
>buffer starts to fill. This buffering increases and adds variability 
>to latency, and thus can create quality problems for interactive 
>applications. As the buffers increase in depth eventually they 
>overflow and cause packet loss. Packet loss is a normal part of a 
>network since it is the mechanism by which TCP governs its 
>throughput. Thus for the loss part of congestion we would define it 
>as the situation in which an increase in data transmissions results 
>in a proportionally smaller or even a reduction in throughput.

[BB] Would it be precise to define loss-based congestion as the loss 
probability? That's what I would do.

>A non-congested network is one in which the latency (end-to-end 
>delay) is relatively constant, and has little packet loss.
>Congestion is also important to be framed by the user experience. A 
>situation of congestion which causes an instant message to be 
>delivered 500ms later is irrelevant to the average user. The same 
>delay on a gaming packet or VoIP packet is perceived as a full loss 
>by the consumer.
>
>Sandvine would therefore define congestion on a per-application 
>class basis as the variability in delay or packet loss beyond what 
>the application can withstand without the user noticing. The 
>Wikipedia article @ http://en.wikipedia.org/wiki/Congestion_collapse 
>provides a good description based on the assumption that all 
>applications perceive congestion the same.

[BB] We need to unpick two separate aspects: an app both i) suffers 
from the effects of congestion and ii) contributes to that congestion:

== i) Sensitivity to congestion ==
The different sensitivities of different apps to a common congestion 
metric are a property of the app, not the network - they are not 
reasons to re-define congestion itself. Once you have an objective 
congestion measure, the response to it can be down to each app (see 
below for the special case of an app-specific middlebox).

== ii) Contribution to congestion ==
Different apps cause different amounts of congestion per data sent, 
by responding more sensitively (LEDBAT) or less (unresponsive UDP), 
or somewhere in the middle (TCP). And apps on shorter RTTs cause less 
congestion for the same data, because they can get out the way more quickly.
All these differences are choices of the app, not anyone else. So 
it's right to define an objective measure of congestion independent 
of these things. If an app chooses to use a long RTT path thereby 
causing more congestion, making it accountable for its contribution 
to congestion is still the right thing to do.

=== A special case of i) with app-specific middleboxes ===
Admittedly, if a middlebox is trying to do the app's job, then the 
app-specific response will need to be in this middlebox. But within 
the box, the app-specific response should still be separated from the 
non-app-specific congestion metric. At least an objective congestion 
metric gives us an architecture where you don't /have/ to have 
anything app-specific in the network.

If you deploy ECN, even a traffic mgmt middlebox doesn't need an 
app-specific response to different impairments, because there are no 
impairments, only congestion signals. Nonetheless, there will need to 
be policy-induced impairment (policer drop) - if all a user's load is 
unresponsive to the congestion signals, or it's responsive but not 
responsive enough for the amount of it.

== Definitions of congestion ==
Congestion is shared by all packets arriving at a queue at a certain 
time, so it is simpler (& more correct IMHO) to define congestion as 
the instantaneous probability of impairment.

You're right that there are different sorts of impairment, so we need 
to define X-related congestion where
* X = loss for loss-related congestion,
* X = queuing delay for delay-related congestion and
* X = marking for ECN-related congestion (not actually an impairment).

In practice, if Reno is dominant, the prob of loss is closely related 
to the prob of extra delay (because delay precedes loss). But on 
faster links congestion delay is imperceptible before loss occurs.

However, modern end-system responses de-correlate the different 
measures. Delay-sensing congestion controls like LEDBAT or Vegas 
reduce the loss probability relative to the delay probability. And 
ECN reduces delay and loss probability to nearly nothing but of 
course doesn't reduce the ECN probability.

So, ECN is the target and its simple, reducing everything to just one 
dimension of congestion. But while we don't have ECN, it makes sense 
to just use loss as a metric and not worry about the differences 
between delay and loss (why perfect the interim solution?).

>So from this, congestion is the point @ which if an application 
>attempts to go faster, it gets no real return for the attempt, and 
>the quality delivered to the user suffers.

[BB] One could define some variant of congestion 
('Sandvine-congestion'?!) as relative to each app, but I don't think 
it's useful to do so.

>Since there is more than one TCP stack implementation in use, and 
>they vary in aggressiveness of ramp up/down, and since the network 
>is shared by so many sessions, and since congestion comes at many 
>hops along the way but TCP packet loss @ one location doesn't show 
>where it occurred, you have a blended mix. The loss may be in the 
>wifi in the home, in the home gateway, in the dsl modem, in the 
>dslam, in the bras, atm aggregation, aggregation routers, core 
>routers, peering, transit, etc. It may be due to the user's own 
>traffic, due to unreliable RF signals, due to other users traffic, 
>due to network switchover, etc.

[BB] Yup.

>So when we modelled TCP packet loss and correlated it against 
>congestion measured as % of link utilisation @ each choke point, and 
>eyeballed the quality of some key applications, we found the 
>correlation to be not that great.

[BB] I'm not exactly sure what the second correlation you mention is 
between. I'm assuming quality:loss is somehow compared to 
quality:utilisation.  This might means something if ':' means 
division, but I'm not sure what it means if ':' is the function 'to eyeball'.

Anyway, you could define 'Sandvine-congestion' like this, but it 
wouldn't be as useful as congestion = probability of impairment. Your 
approach assumes:
a) link utilisation is somehow a benchmark measure of 'congestion',
b) 'congestion' is meant to give a deterministic measure of app 
quality for all apps.

These both seem backwards:
b) I've already argued against an app-specific congestion metric at 
item (i) above.
a) Should congestion be defined at a human's timescale, or a 
computer's timescale? Link utilisation is measured over some 
averaging period (say 15mins or 1min or whatever). Congestion control 
algos can respond in msec. And scheduling algorithms can respond in 
nano or femto secs. If network operators are going to get their 
customers to co-operate towards beneficial ends for all, they need to 
treat their customers as machine-mediated customers.

It's most useful to define congestion at the fastest timescale 
possible (probability of impairment at queuing timescales). Then if 
there's a way for a computer to avoid congestion, the operator can 
give it an incentive to do so, by making it accountable for congestion.

For instance, there might be a machine on a short RTT path that can 
delay sending traffic at little detriment to itself (e.g. using 
LEDBAT). Or there might be a machine that can't use a short RTT, but 
it can use a lower Diffserv class instead. If no machine can avoid 
congestion, then that's fine. But if some machines have flexibility 
that can be harnessed, there's no point setting the bar so low that 
they don't use their flexibility.

Ideally, all the TCPs are trying to keep instantaneous link 
utilisation at 100% while there is some traffic to send and 0% 
otherwise. Humans understand utilisation as the average of this 
process - nominally the mark-space ratio between two states (but 
actually the sum of all the slow starts & AIMD sawteeth).

A source's contribution to loss (or marking) gives a much more useful 
instantaneous measure of congestion than some average of utilisation 
over time. And when accumulated over time the amount of loss (or 
marking) gives a useful average measure too.

>A related question relates to the use of policing. If i put a 10Mbps 
>policer on a network that is capable of 100Mbps, I do not see 90Mbps 
>of 'drops' from the policer, i typically see 0. Therefore i cannot 
>infer what the bandwidth need would be if i remove the policer (it 
>might still be 10, it might go to 100). Thus the packet loss cannot 
>be used to infer the bandwidth desired.

[BB] There's two separate issues I have with this:

1/ Whether loss is due to policy or to congestion, bandwidth desired 
isn't relevant to how much harm one user contributes to others. 
That's the bandwidth they actually make do with, weighted by the 
congestion when they use it (ie their contribution to congestion).

2/ If a policer is basing its decisions on contribution to 
congestion, it should have two separate internal processes:
- measuring the congestion a user's traffic causes
- dropping traffic when policy is exceeded

It shouldn't include policy drop in its measurements to decide how 
much policy drop. This is straightforward if it is not congested 
itself, but it's just measuring congestion everywhere else using 
(re-)ECN. But if it is itself the bottleneck, it needs to create a 
virtual bottleneck internally to do the measurement step, before 
introducing the policing drop.

That was a long set of responses, but I hope they are worth it.

Bob

>--don

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: "Don Bowman" <[email-masked]>
Date: Mon, 26 Oct 2009 09:21:43 -0400

From: Bob Briscoe [mailto:[email-masked]]
> >From: Matt Mathis [mailto:[email-masked]]

 ... snip ...

> to congestion is still the right thing to do.

I'm not sure i would define UDP as an application. An application
is e.g. video streaming. Some applications have a natural pacing
mechanism (e.g. video, voip, IM, email), others do not (e.g. file
transfer).

Applications have a user-expectation of them, and an ability to deal
with congestion. Some (e.g. the paced ones) deal with the jitter
aspect of congestion by increasing their pre-buffer. The affect
on the user experience (longer channel change time) is better than
the alternative (glitches or freezes). So although congestion
is application-independent, the affect of it is not.

> division, but I'm not sure what it means if ':' is the function 'to

well, eyeball may have been a somewhat imprecise term here :)

Our equipment is able to measure VoIP QoE (MOS). We do this by locking
a local clock to the RTP stream clock, and also looking @ the RTP
sequence counter for detecting loss. In this fashion we can
see latency/loss/jitter (latency comes from RTCP-XR from the endpoints).
As a correlation, we looked @ the correlation between TCP packet loss
and MOS for VoIP. The correlation was low (~0.1). This is in a heavily
mixed network w/ hundreds of thousands of consumer connections.
We inferred there was congestion @ some times because the MOS varied
as a function of the hour. We assumed the congestion was in the network
we were in because some of the VoIP providers were directly peered,
and we assumed they were uncongested (no proof of the assumption).

In a similar fashion, we correlated access round-trip-time by
looking @ the delta between SYN-ACK towards the subscriber, and ACK
returning. We used this time as a proxy for congestion since it
correlated
better (~0.7).

> congestion when they use it (ie their contribution to congestion).

But it does relate to today's problem of 'how much capacity is truly
desired'. And it remains one of the common myths in the minds of
our customers, and of the end users of networks. It's the Heisenberg
uncertainty principle applied to internet bandwidth :) There are some
marked non-linearities. Anecdotal evidence that i have... networks
that are too congested to support streaming have no streaming. When you
start to fix that congestion, the bandwidth demanded actually goes
up sharply as people start to be able to use apps they didn't before.
The most interesting cases are small island nations (malta, Caribbean,
...):
no local content, no peering, disproportionate cost of transit to
access network, ... great case studies, often fun to visit too :)

> - dropping traffic when policy is exceeded

i agree.

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 26 Oct 2009 14:36:47 +0000

Don,

>is application-independent, the affect of it is not.

[BB] I deliberately said 'unresponsive UDP' cos I wouldn't define UDP 
as an application either. I was just giving an example at the far end 
of the spectrum of responsiveness (actually I've seen apps that are 
inversely responsive - the more loss the more redundancy is sent).

>returning. We used this time as a proxy for congestion since it

[BB] Understood. But the point I was making is that we're not looking 
for some metric that correlates with each application's QoS. We're 
looking for a metric that represents potential harm by each user U on 
'the most sensitive but unknown' application X that might be sharing 
with U. If different apps are better at working round that harm, all 
power to them.

Contribution to congestion (volume of congested bytes) might look 
like a somewhat arbitrary choice, but it's very robust because it's 
proportionate to the two factors involved, taken over a brief period of time:
- Your volume: if you're sending twice as much as someone else, you 
get twice as much pressure to get out;
- Everyone else's volume: if the queue is being pushed twice as hard, 
it also doubles the pressure for you to get out of the way;

>access network, ... great case studies, often fun to visit too :)

[BB] The proposed mechanism allows each user to push for the 
bandwidth they want, if they are prepared to push others out of the 
way, drawing down their congestion allowance in the process. That 
can't signal demand for a new app that requires more bandwidth than 
physically exists. But in more typical incremental cases it signals 
to the operator that there is real demand (ie backed by cash).

I'm not claiming it's a panacea tho - it's not something that would 
substitute for sending out marketing people to find out if there's 
some cool new app that people would use and be willing to pay for 3x 
as much bandwidth for.

> > - dropping traffic when policy is exceeded

[BB] Cheers

Bob

---

From: John Leslie <[email-masked]>
Date: Mon, 26 Oct 2009 10:54:36 -0400

Don Bowman <[email-masked]> wrote:
>>... 

   I'm afraid we do not all mean the same thing by "congestion". :^(

   While I've long since abandoned hope of getting everyone on the same
page, I believe it would help if we could keep it clear what individual
posters mean when they use that term.

   "Congestion" to historic TCP is packets lost, period.

   "Congestion" to file-transfer is anything that slows the transfer.

   "Congestion" to video-streaming is how much delay you need to
compensate for jitter and retransmissions.

   "Congestion" to VoIP is how much degradation the listener gets.

   "Congestion" to Bob is -- well, I don't know exactly: perhaps Bob
could enlighten us...

> ... Some applications have a natural pacing mechanism (e.g. video,
> voip, IM, email), others do not (e.g. file transfer).
> 
> is application-independent, the effect of it is not.

   (It's important to note that adding delay to VoIP tends to degrade
perceived quality much faster than adding delay to streaming video.)

> Our equipment is able to measure VoIP QoE (MOS).

   Acronym-Alert!

> We do this by locking a local clock to the RTP stream clock, and
> also looking @ the RTP sequence counter for detecting loss. In
> this fashion we can see latency/loss/jitter (latency comes from
> RTCP-XR from the endpoints).

   I'll guess Don is talking about video streaming (which IMHO is
insensitive to latency below several seconds). Thus increases in
latency cover jitter nicely.

> As a correlation, we looked @ the correlation between TCP packet
> loss and MOS for VoIP.

   VoIP, of course, _is_ sensitive to latency. In my experience,
latency over half a second borders on unacceptable.

> The correlation was low (~0.1).

   Alas, I don't understand how Don measures "MOS". In VoIP, packet
loss is perceived as noise-like, and much of VoIP is pretty noise
tolerant.

> This is in a heavily mixed network w/ hundreds of thousands of
> consumer connections.

   I'm left to guess what Don refers to here...

> We inferred there was congestion @ some times because the MOS
> varied as a function of the hour.

   To be expected...

> We assumed the congestion was in the network we were in because
> some of the VoIP providers were directly peered, and we assumed
> they were uncongested (no proof of the assumption).

   Alas, such assurances, unless tested, may not be dependable...
(And I'm not the least bit sure what Don means by "congestion".)

> looking @ the delta between SYN-ACK towards the subscriber, and
> ACK returning. We used this time as a proxy for congestion since
> it correlated better (~0.7).

   A correlation of 0.7 may be the best we can hope for, but it's
really not good enough.

   Round-trip-time variations are probably correlated with some
definitions of "congestion", and absolute RTTs are probably
correlated with a number of things which can degrade the experience.
Equating RTT with congestion sounds really funny, though...

--
John Leslie <[email-masked]>

---

From: "Don Bowman" <[email-masked]>
Date: Mon, 26 Oct 2009 13:46:08 -0400

From: John Leslie [mailto:[email-masked]]
>    "Congestion" to Bob is -- well, I don't know exactly: perhaps Bob

I would define congestion as the point @ which trying harder achieves no
more,
diminishing returns.
I would define the then affect of that in turns of user expectation of
experience.

 ...

> > Our equipment is able to measure VoIP QoE (MOS).
VoIP == voice over IP
QoE == quality of experience
MOS == mean opinion score, an ITU standard (ITU == international telecom
union)

> latency cover jitter nicely.

I was actually talking about voice. Video streaming today is largely
moved to RTMP transport, and does not use RTP.

Latency to video streaming affects channel change time. Several seconds
is not great to add on to the video decode buffers (VBV). If the latency
is at all variable, some decoders are not good @ initial latency
being different than long term latency.

> latency over half a second borders on unacceptable.

The 3GPP standard for voice is 600ms of end to end delay as an absolute
maximum (cell to cell call). In practise, 300ms is a significant
irritant
to the average person since they tend to step on each other when
talking.

>    I'm left to guess what Don refers to here...

MOS is a standard (mean opinion score) used by the voice industry. There
are methods that model the affect of loss/latency/jitter/codec fidelity
on the end user. This is crunched into a number from 1 to 5. Some
operators express a SLA (service level agreement) in terms of MOS
(e.g. 95% of calls will have a MOS of 4.0 or better).

>    A correlation of 0.7 may be the best we can hope for, but it's

I dunno, i thought it was sufficiently close to 1.0 to say 'mission
accomplished' :)

> Equating RTT with congestion sounds really funny, though...

We correlated jitter in RTT, not absolute RTT.

--don

---

From: "Don Bowman" <[email-masked]>
Date: Mon, 26 Oct 2009 14:52:25 -0400

...

> proportionate to the two factors involved, taken over a brief period
of
> time:

so the rationale for the measurement was: "Assume VoIP quality is a good
proxy for overall quality. Prove that your <product X> increases the
overall achieved quality". This was based on both pragmatic (i had
the VoIP measure available) and rational (it seems likely VoIP is
fragile) means.

We then use 'volume of congested bytes' in the approximation i mentioned
earlier: if the link is congested (defined for the current iteration
as 'over 70% utilisation') (note: we calibrated this quite extensively
in a lab see attached, but this is still an empirical approximation),
then the
users disproportionately contributing to the congestion (volume)
are offered a lower priority. The question we were trying to answer
was: did this improve the situation? What happened to the overall
experience of each class of user?

Now, how could this be improved? 'is congested' could become explicit.
I believe you are familiar w/ the concept so i won't elaborate :)
i was not suggesting 'is congested' be application specific, but
was suggesting that the 'did we make the world better' test has to
be based on application specific measurement and expectations.

I attach for reference some information on our 'calibration by
empirical means of the congestion in a cable network'. i well understand
this is not terribly generalised, but its what i have.

--don

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 26 Oct 2009 20:45:52 +0000

Don,

> >    "Congestion" to VoIP is how much degradation the listener gets.

'congestion to an application' is not a definition of congestion, 
it's a definition of the effect of congestion on an app. Congestion 
is a property of a link (or a road, or a bus).

It makes no sense to say a road with the same number of cars on it is 
more congested if you have an important interview than if you don't.

> >    "Congestion" to Bob is -- well, I don't know exactly: perhaps Bob

I thought I had made it fairly clear, but here's more precise 
definitions (and totally precise ones are in the ref given below that).

* Link congestion is the instantaneous probability of a certain type 
of impairment at the link (loss or marking - see below).
* Path congestion is the combinatorial probability of a certain type 
of impairment at all the links on the path.

Strictly impairment is defined as "not being served to the packet's 
requirements". That doesn't mean the application's requirements, it 
means the requirements implied in the packet. I.e. a best efforts 
packet's requirement is to be delivered, and not being served to its 
requirements means being dropped. An ECN-capable packet's requirement 
is to be delivered without being marked (or dropped). A packet with a 
certain Diffserv codepoint is not served to its requirements is it's 
not delivered within the SLA for that codepoint.

For totaly precise definitions, see appendix A.1 of our original 
SIGCOMM paper on re-feedback. I must acknowledge my ex-colleague 
David Songhurst for these definitions (David started working on QoS 
in 1973 and led BT's Performance Engineering Group for many years). 
It took thought and discussion over many weeks to get these 
definitions both correct and useful in practice:

Briscoe, B., Jacquet, A., Cairano-Gilfedder, C.D., Salvatori, A., 
Soppera, A. & Koyabe, M., "Policing Congestion Response in an 
Internetwork Using Re-Feedback," Proc. ACM SIGCOMM'05, Computer 
Communication Review 35(4):277--288 ACM Press (August 2005)  DOI: 
http://doi.acm.org/10.1145/1080091.1080124

>I would define the then affect of that in turns of user expectation of

Again, diminishing returns and user expectation are relative to the 
app and the user. These are not objective measures of the congestion 
of a link, they are measures of the effect of congestion.

Bob