Subject: [re-ECN] CONEX Principles & Constraints

From: <[email-masked]>
Date: Thu, 29 Oct 2009 18:02:44 -0000

We wanted to start some discussion on "principles" and "constraints" ie
what we want to do in the proposed WG and how would its scope would be
restricted (building especially on emails from John (about "principles")
and Rich (about "requirements"). Eventually this would feed into the
Charter. This is just a first guess, so please bash. Something on these
lines would be in the "Constraints" part of the Agenda.  

1.	the purpose is to standardise the conex protocol - this exposes
congestion information along the forwarding path of the Internet
("rest-of-path congestion"). (This is in addition to the current
capability that exposes the congestion on the "path-so-far".). Note, the
purpose doesn't include studying how the conex information could be
used. Reason: there are lots of potential uses for the information. In
order to reduce the scope of the proposed WG, we concentrate on the
underlying information protocol.
2.	The protocol should be open about the responses it allows to the
conex information. For example, it should not make assumptions about the
behaviour of a particular application, and it should allow a diversity
of potential traffic management practices (of networks and end-hosts).
3.	the conex information needs to be easily visible to all network
elements, and be application-independent (including encrypted apps).
Reason: this ensures widest possible applicability -- it is visible to
both end-systems (to help them control their sending rate) and to
network managers (to help them manage a bottleneck, whether traffic
originates from their own users or from other networks).
4.	the conex information needs to be sufficiently up-to-date, for
example so that it can usefully assist congestion control by end
systems. Reason: stale information is little use. [Question: instead of
"sufficiently up-to-date", would "real-time" or "near real-time" be
better?]
5.	the conex information needs to be sufficiently granular, to
allow identification of those sending traffic to the bottleneck
[Comment: this is trying to get a point that John made about 'granular
enough for cost allocation' but in a more generic way. "Identification"
is a bad word as, for some readers, it might be imply more than intended
- but I can't think of a better one at the moment. I hope that the
following bullet about conex information in the IP header solves how to
do it.]
6.	the conex information is carried in the IP header. Note: it is
in scope to work out which IP header bits to use (and the implications
for other stuff that may already be using those bits). 
7.	a protocol will be developed for both IPv4 and IPv6. Reason:
both are needed for full applicability. (There was push-back on an
earlier suggestion of narrowing the scope to IPv4.)
8.	TCP options will be used to inform the sender about congestion
received at the receiver. (Note, current proposals require this
information transfer.) Reason: narrow the scope by looking only at TCP,
which is used by the vast majority of the traffic (~85-90%). 
9.	a baseline assumption is that routers are unmodified. Reason:
deployment viability
10.	a baseline assumption is that conex information is signalled
explicitly (through the IP header) and not implicitly signalled through
packet loss, delay or jitter. Reason: explicit signalling allows a
"better" reaction, for example faster, without impairment
11.	but we will also develop a solution where the conex information
is signalled through packet loss. Reason: ECN is not enabled in most
routers today; incremental deployment is critical
12.	(possibly) we will study an incremental deployment step where
only the sender is conex-enabled [Step suggested by Matt]
13.	(possibly) another incremental deployment step?
14.	we will describe threats, especially from falsifying or
suppressing the conex information - whether by ISPs or end-hosts. The
protocol needs to allow such threats to be dealt with (in as simple a
manner as possible), but the actual mechanisms to tackle those threats
(using policer boxes, for instance) are out of scope. Reason: such
mechanisms are about uses of conex, rather than exposing the
information.
15.	anything else?

Thanks

phil

---

From: <[email-masked]>
Date: Fri, 30 Oct 2009 12:04:46 -0000

In general I think these are a good starting point. Some comments
inline...

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of [email-masked]
> Sent: 29 October 2009 18:03
> To: [email-masked]
> Subject: [re-ECN] CONEX Principles & Constraints
> 
> restricted (building especially on emails from John (about
> "principles") and Rich (about "requirements"). Eventually this would
> feed into the Charter. This is just a first guess, so please bash.
> Something on these lines would be in the "Constraints" part of the
> Agenda.
> 
> capability that exposes the congestion on the "path-so-far".). Note,
> the purpose doesn't include studying how the conex information could
be
> underlying information protocol.

I might even go so far as saying "Potential uses for such information
are explicitly beyond the scope of this initial
{WG/BoF/protocol/whatever}"

> conex information. For example, it should not make assumptions about
> the behaviour of a particular application, and it should allow a
> diversity of potential traffic management practices (of networks and
> end-hosts).

I would either merge this with the later points about IP or at least
re-order the list so they occur in a row.

> "sufficiently up-to-date", would "real-time" or "near real-time" be

How about "Sufficiently up-to-date to enable near-real-time traffic
management". 

Incidentally this is nothing to do with congestion control at
end-systems - the end-system already sees congestion! So the only link
to congestion control is if this in turn allows the network to be more
lenient towards end-systems.

> 5. the conex information needs to be sufficiently granular, to allow
> identification of those sending traffic to the bottleneck [Comment:
> this is trying to get a point that John made about 'granular enough
for
> cost allocation' but in a more generic way. "Identification" is a bad
> word as, for some readers, it might be imply more than intended - but
I
> can't think of a better one at the moment. I hope that the following
> bullet about conex information in the IP header solves how to do it.]

The information needs to granular enough to hold the originator of the
traffic accountable?

> 6. the conex information is carried in the IP header. Note: it is in
> scope to work out which IP header bits to use (and the implications
for
> other stuff that may already be using those bits).

Better: "This may require defining new codepoints in the IP header or
possibly even the re-definition of existing codepoints. We will be
careful to consider the impact of our choices both on existing protocols
and on deployability.

> 7. a protocol will be developed for both IPv4 and IPv6. Reason: both
> are needed for full applicability. (There was push-back on an earlier
> suggestion of narrowing the scope to IPv4.)

I might consider phrasing this "although on the face of it an IPv6
protocol may be easier, we will also develop solutions for IPv4 as it is
still the dominant network-layer protocol.."

> which is used by the vast majority of the traffic (~85-90%).

Do we want to specify using TCP options at the moment? Option space is
limited and already used for lots of other things + MPTCP will be using
even more (and there is a very high chance of eventual overlap between
conex and MPTCP). I would prefer to leave this as "we will concentrate
on TCP as this provides a feedback path..."

> 9. a baseline assumption is that routers are unmodified. Reason:

"This MUST work with unmodified routers, even if they don't do ECN...
although modifying routers may allow additional refinements...

> "better" reaction, for example faster, without impairment

In theory implicit signalling already exists - if you observe a TCP
source halving its rate you can assume it was (probably) in response to
a loss event...

> 11. but we will also develop a solution where the conex information is
> signalled through packet loss. Reason: ECN is not enabled in most

No! - we may develop a solution that informs the network of congestion
the sender has learnt about both as a result of explicit notification
(ECN) and implicit notification (loss). However we DON'T use loss to
signal ConEx information (which is surely impossible anyway?)! 

> 12. (possibly) we will study an incremental deployment step where only
> the sender is conex-enabled [Step suggested by Matt]

We certainly need to leave this open as a possible step

> 13. (possibly) another incremental deployment step?

Proxy?

> 14. we will describe threats, especially from falsifying or
suppressing
> the conex information - whether by ISPs or end-hosts. The protocol
> needs to allow such threats to be dealt with (in as simple a manner as
> possible), but the actual mechanisms to tackle those threats (using
> policer boxes, for instance) are out of scope. Reason: such mechanisms
> are about uses of conex, rather than exposing the information.
> 15. anything else?

Probably :)

---

From: John Leslie <[email-masked]>
Date: Fri, 30 Oct 2009 12:33:24 -0400

[email-masked] <[email-masked]> wrote:
>> [email-masked]
>> 
>> 1. the purpose is to standardise the conex protocol...
> 

   "the planned WG".

>> diversity of potential traffic management practices (of networks and

   Diversity of traffic management practices is critical: I'm not sure
whether the rest of this paragraph helps or tends to confuse.

>> Reason: this ensures widest possible applicability -- it is visible
>> to both end-systems (to help them control their sending rate) and to

   I wouldn't recommend "merging" this with anything else.

>> systems. Reason: stale information is little use. [Question:
>> instead of "sufficiently up-to-date", would "real-time" or "near
>> real-time" be better?]
> 

   I think Toby captures the essential point: that this needs to be
used for real-time traffic management. I don't think "up-to-date"
helps.

> end-systems - the end-system already sees congestion! So the only
> link to congestion control is if this in turn allows the network
> to be more lenient towards end-systems.

   "Usefully assist congestion control by end systems" isn't really
the point. While we envision a system where ECN marking is "re-inserted
into the stream being sent," this is mostly unrelated to what the end
user application and/or stack does to control congestion.

   I'm not sure what we can say about how this benefits end users.
I do imagine some overlap with the Multipath TCP WG, but it's way too
early for any specifics.

   Our point really is that ConEx marking can enable network operators
to distinguish "heavy" traffic from "congestion-causing" traffic, thus
enabling experimentation with transport algorithms that do a better
job of filling the pipes without harming other uses.

>> for cost allocation' but in a more generic way.

   I guess I should comment on that.

   In hindsight, I was already too "generic" when I wrote that phrase.
I meant allocation within the accounting system of the network operator
that might perform the upgrade; but I wished to allow for policing
which either charged for congestion from a particular forwarder or
dropped / delayed / whatever packets that would "cause" more congestion
than "normal".

   The point I wanted to make is that upgrading a bottleneck link needs
cost justification, and we need to collect information that can justify
the cost.

>> "Identification" is a bad word as, for some readers, it might be
>> imply more than intended

   Agreed.

>> - but I can't think of a better one at the moment.

   Better to leave out anything that even resembles "identification"...
The point is strictly related to who _forwards_ packets, not who _sends_
them; and how that is determined is pretty strictly outside the scope
of ConEx.

   The point of "granularity" is that we can't afford to try to aggregate
to "flows" and then de-aggregate to portions of those "flows" that cause
congestion: we need to operate packet-by-packet, determining a probability
that the packet in question will "cause" congestion.

>> I hope that the following bullet about conex information in the IP
>> header solves how to do it.]
> 

   I hope not!

>> for other stuff that may already be using those bits).

   This is really weak on justification for the use of scarce bits...

> careful to consider the impact of our choices both on existing protocols

   I would try to avoid trespassing on that ground.

   While there is space in the IPv6 header, we want to experiment in
IPv4-land. Talking of "defining new codepoints" is likely to get our
scope limited to IPv6. Better to experiment in IPv4, and perhaps _then_
define IPv6 codepoints.

   BTW, I firmly believe the "granularity" issue is essential to justify
experimenting with IPv4 codepoints: if we don't get buy-in to the need
for granularity, the whole experiment is endangered.

>> suggestion of narrowing the scope to IPv4.)

   I agree, but justification is weak. Bob Briscoe should clarify how
we'd experiment in IPv6 space: I believe that would be very good;
however, I suspect we'll eventually find a better way to do congestion
exposure in IPv6, and we shouldn't limit ourselves to designing the
final IPv6 solution too early.

> still the dominant network-layer protocol.."

   As I've already said, we need to _experiment_ in IPv4 in order to
collect sufficient data and observe how congestion exposure information
is actually used. I would avoid calling the IPv4 stuff "solutions"

>> information transfer.) Reason: narrow the scope by looking only at
>> TCP, which is used by the vast majority of the traffic (~85-90%).

   I thorouhly disagree!

   Limiting the scope to TCP would be fatal. We need to collect data
on congestion caused by UDP packets; and any limitation to particular
transport(s) incents the use of other transports to "end-run" ConEx.

> on TCP as this provides a feedback path..."

   TCP option "space" is limited in that packets are limited to 44 bytes
of options per packet. We _would_ get burned by that limit. :^(

   More importantly, we _really_ don't want to wander into the weeds of
how to juggle TCP options when we're dealing with the forwarding of IP
packets. (Yes, I know we do have folks clever enough to pull it off
99.999% of the time; but the rest of us will quickly lose track of the
details, and sooner or later the folks that are clever enough will find
themselves on the "rough" end of "rough consensus.)

> although modifying routers may allow additional refinements...

   A reasonable way to say it...

>> explicitly (through the IP header) and not implicitly signalled
>> through packet loss, delay or jitter. Reason: explicit signalling
>> allows a "better" reaction, for example faster, without impairment

   Pretty good... but the justification needs to be stronger.

   First of all, explicit signaling can carry more-than-one bit,
instead of less-than-one (due to the multitude of other things that
can lead to packet loss); and can reach the actor who needs to react
significantly faster.

   Second, explicit signaling carries consistent information to every
router along the path, meaning that incentives to slow down a sending
rate can be applied "close enough" to the sender to be effective; and
if a packet actually does need to be dropped, it could be dropped
before it causes other congestion earlier in the forwarding path.

> source halving its rate you can assume it was (probably) in response to

   Hmm... that's getting "too clever" for this bear-of-small-brain...

>> routers today; incremental deployment is critical

   Am I missing how to "re-insert" that signal?

> signal ConEx information (which is surely impossible anyway?)! 

   I strongly recommend against any statement along those lines.

> We certainly need to leave this open as a possible step

   I don't see any reason to mention this. Clearly, a ConEx-aware sender
can re-insert a signal whether or not the end-receiver is ConEx aware.
And that signal can enable all the experimenting we need along the
forwarding path.

   We need only specify the _meaning_ of the ConEx signal the sender
re-inserts -- not any particular algorithm for generating it.

>> 13. (possibly) another incremental deployment step?

   Way premature, IMHO.

>> suppressing the conex information - whether by ISPs or end-hosts.
>> The protocol needs to allow such threats to be dealt with (in as
>> simple a manner as possible), but the actual mechanisms to tackle
>> those threats (using policer boxes, for instance) are out of scope.
>> Reason: such mechanisms are about uses of conex, rather than exposing
>> the information.

   I agree.

>> 15. anything else?

   Not at this time.

--
John Leslie <[email-masked]>