Subject: [AVT] draft-ietf-avt-ecn-for-rtp-02 T2/ ECN-IP-RTCP

From: Bob Briscoe <[email-masked]>
Date: Thu, 05 Aug 2010 23:39:34 +0100

Magnus, Ingemar, Colin, Piers, Ken,

Below are my main technical concerns with 
draft-ietf-avt-ecn-for-rtp-02 as promised.
I have used the line numbering here:
<http://tools.ietf.org/idnits?url=http://tools.ietf.org/id/draft-ietf-avt-ecn-for-rtp-02.txt>

T1/ Combining pkts
T2/ ECN-IP-RTCP                         <<<---THIS EMAIL
T3/ Anti-Cheating
T4/ Logging Not-ECT fall-back
T5/ Delivery %age ECT vs Not-ECT
T6/ Pkt Reordering around session start

T2/ ECN in the IP header of RTCP reports.
-------------------
"

510        RTCP traffic MUST NOT be ECT marked for the following reason.

"
"

1047       The sender SHALL NOT include ECT marks on outgoing RTCP packets, and
1048       SHOULD NOT include ECT marks on any other outgoing control messages
1049       (e.g.  STUN [RFC5389] packets, DTLS [RFC4347] handshake packets, or
1050       ZRTP [I-D.zimmermann-avt-zrtp] control packets) that are multiplexed
1051       on the same UDP port.

"
If congestion rises on the reverse path it /could/ be a sign that it 
is rising on the forward path. Ideally, we want it to be possible for 
receivers to mark RTCP reports as ECT so they are not more likely to 
be lost just when they are needed to report rising congestion levels. 
E.g. if the congestion level on the reverse path is say, 0.5%, then 1 
in 200 RTCP reports will be lost.

I promised to propose a way to have ECT on RTCP packets that would be 
robust to middleboxes dropping ECT packets following a re-route. The 
solution makes the following assumptions:
A1) that an ECN-Blocking Muddlebox drops any ECT packet, irrespective 
of whether it is RTP or RTCP at the app-layer.
A2) that an RTP source knows how often to expect RTCP reports from 
receivers, so it can count when RTCP reporting intervals have passed 
without an RTCP report.

Here's the proposal:* As per the I-D, the session starts without ECT 
on any packets.
* First sender(s) introduce ECT on RTP packets through the probing 
methods already described in the I-D.
* Consider the unicast case of a session betw A & B.
   - Once A has received 3 successive RTCP reports containing counts 
of ECT and CE, it can send its own RTCP reports as ECT.
   - If A subsequently notices that 3 RTCP reporting periods have 
passed without a report, or A receives no traffic at all for time 
[TBA], then A reverts to sending RTCP reports as Not-ECT.
   - B follows the same two rules as A.
   - If B is a pure RTP receiver in a unidirectional session, it will 
never get RTCP reports containing counts of ECT and CE, so it will 
never switch to sending its own RTCP reports as ECT. This is because 
it has no way to test if ECT packets are getting through.

   - Then if a re-route sends B>A traffic through an ECN-Blocking 
Middlebox, no B>A packets will get through. But after 3 RTCP 
reporting periods A tells B in a Not-ECT RTCP report and B reverts to Not-ECT.
   - If a re-route sends both B>A and A>B traffic through an 
ECN-Blocking Middlebox, the above behaviour will also recover the session.

* Multicast multi-source case, A -> B,C,D,E; B -> A,C,D,E; etc.
   - Once A has decided it can safely turn on ECT in RTP packets it 
sends, it watches for any RTCP reports that report reception of a 
positive Not-ECT count but zero ECT or CE count on packets 'A' sent. 
Once it has received none of these RTCP reports for three RTCP 
reporting rounds, 'A' can switch on ECT for its own RTCP reports. 'A' 
determines 3 reporting rounds by picking a couple of receivers and 
waiting until it receives 3 reports from each.
   - B,C,D,E follow the same rule.

   - A cannot test whether its own ECT RTCP reports continue to 
arrive at their destinations, because the destinations do not know to 
expect traffic from A. So if they are getting no traffic from A 
because of an ECN-Blocking Muddlebox, the transport will never know, 
although the application or human may miss the contribution from A 
and reset the transport to use Not-ECT on RTCP reports.
   - With receiver-initiated multicast, A also cannot test for all 
sorts of other blockages that might suddenly appear on paths to some 
destinations. Therefore, once an ECN-capable session has been 
established, it would be reasonable to take a leap-of-faith that 
re-routes are likely to continue to find ECN-capable paths.   - And 
if the leap-of-faith is not warranted, the session breaks until the 
application resets it to Not-ECT - but that's not surprising if there 
are broken muddleboxes on some paths.
* Multicast single-source case, A -> P,Q,R,S etc.
   - All follow the same rule as the multi-source case, with the same 
limitations.
   - As in the unicast and multi-source cases, if any hosts are pure 
RTP receivers (P,Q,R,S), they won't ever get an RTCP report about RTP 
packets they have sent, so they will never set ECT on RTCP reports.

Comparison between ECT-RTCP and Not-ECT-RTCP:
* Session recovery time following a re-route through an ECN-Blocking Middlebox:
   - ECT-RTCP:           3 RTCP reporting intervals
   - Not-ECT-RTCP:       1 RTCP reporting interval
* ECT-RTCP has more robust RTCP reporting during rising symmetric congestion
* Not-ECT-RTCP involves less implementation (unless using 
leap-of-faith, when they are both the same).

BTW, the assumption that an ECN-Muddlebox will drop any ECT packets, 
whether RTP or RTCP, can be relaxed if we add reporting of RTCP 
packets from A>B in reports from B>A. But it seems a reasonable 
assumption so reporting on RTCP reports should not be necessary.

Cheers

Bob

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: ken carlberg <[email-masked]>
Date: Tue, 10 Aug 2010 10:26:18 -0400

Bob,

first off, let me thank you immensely for the very detailed review of the document.  It will take a bit of time to carefully go over all of your emails, and the various authors will probably selectively pick out certain points you make and respond to them directly.  And since we're scattered all over the place, we'll probably do this in an ad-hoc fashion.

> If congestion rises on the reverse path it /could/ be a sign that it is rising on the forward path. Ideally, we want it to be possible for receivers to mark RTCP reports as ECT so they are not more likely to be lost just when they are needed to report rising congestion levels. E.g. if the congestion level on the reverse path is say, 0.5%, then 1 in 200 RTCP reports will be lost.
> I promised to propose a way to have ECT on RTCP packets that would be robust to middleboxes dropping ECT packets following a re-route. The solution makes the following assumptions:
<snip> 
<snip>

For this particular topic, I think your proposal could be accomplished.  However, I'm quite concerned about the added complexity for an outlier case.  As I understand it, your overall objective is to ensure that as many packets as possible are ECT marked.  In addition, you'd prefer not to "reward" (for lack of a better term) broken software that resides in some subset of boxes by precluding ECT marking in packets.  Both objectives are understandable and agreeable at face value.  But if we go down this path, we are mostly providing slightly more fidelity, which I don't think provides us with a substantial gain compared to the complexity of the solution.

From my perspective, a big advantage in adding in adding ECN support for UDP/RTP flows is the speed of notification of congestive conditions with minimal disruption (ie, lost packets) of the flow -- speed being the key word.  But in your proposal, we still need to wait to go through a discovery process to see if its ok to send ECT marked RTCP traffic, so the focus becomes additional exactness involving a comparatively small subset of packets.

Also, the notion that congestion in one direction "could" be a sign of congestion arising in the opposite direction is on shaky ground, at best.  

I like the proposal as definitely good food for thought, but I'd prefer not to add it to the design outlined in the draft because I don't think the gain out ways the pain.

cheers,

-ken

---

From: "Piers O'Hanlon" <[email-masked]>
Date: Tue, 17 Aug 2010 20:17:12 +0100

Hi Bob,

Thanks for the suggestion on ECN for RTCP - some comments in-line
where appropriate and the rest here.

One observation is that your mechanism generally relies on
bi-directional flows to achieve the signalling but when there are
bidirectional RTP flows then the ECN status of the path would be
provided already by the respective RTCP reports in each direction.

It's hard to quantify the added benefit of knowing that the reverse
path is getting congested and whether that is worth an extra
2xRTCP_reporting_period of loss.... I guess the probability of each
should hopefully be quite different but it is still a potentially
nasty hit to take if an ugly muddlebox falls in the way.

The other snag is with asymmetric paths, as the fact that the RTCP
path is congested may not be indicative of the forward [RTP] path.

On balance I agree with Ken that it probably isn't worth the extra
effort, though I can see the attraction.

Thanks,

Piers

On 5 August 2010 23:39, Bob Briscoe <[email-masked]> wrote:
> Below are my main technical concerns with draft-ietf-avt-ecn-for-rtp-02 as
> promised.
> If congestion rises on the reverse path it /could/ be a sign that it is
> rising on the forward path. Ideally, we want it to be possible for receivers
> to mark RTCP reports as ECT so they are not more likely to be lost just when
> they are needed to report rising congestion levels. E.g. if the congestion
> level on the reverse path is say, 0.5%, then 1 in 200 RTCP reports will be
> lost.
>
> I promised to propose a way to have ECT on RTCP packets that would be robust
> to middleboxes dropping ECT packets following a re-route. The solution makes
> the following assumptions:
> A1) that an ECN-Blocking Muddlebox drops any ECT packet, irrespective of
> whether it is RTP or RTCP at the app-layer.
> receivers, so it can count when RTCP reporting intervals have passed without
> an RTCP report.
>
> Here's the proposal:* As per the I-D, the session starts without ECT on any
> packets.
> * First sender(s) introduce ECT on RTP packets through the probing methods
> already described in the I-D.
>  - Once A has received 3 successive RTCP reports containing counts of ECT
> and CE, it can send its own RTCP reports as ECT.
>  - If A subsequently notices that 3 RTCP reporting periods have passed
> without a report, or A receives no traffic at all for time [TBA], then A
> reverts to sending RTCP reports as Not-ECT.
>  - If B is a pure RTP receiver in a unidirectional session, it will never
> get RTCP reports containing counts of ECT and CE, so it will never switch to
> sending its own RTCP reports as ECT. This is because it has no way to test
> if ECT packets are getting through.
>
A 'pure RTP receiver', which doesn't send RTCP isn't compliant with
the base RTP spec (rfc3550) - an RTP receiver [only] should still send
periodic RTCP Receiver Reports.  Though there are quite a few Voip
phones do 'forget' to send RTCP.

>  - Then if a re-route sends B>A traffic through an ECN-Blocking Middlebox,
> no B>A packets will get through. But after 3 RTCP reporting periods A tells
> B in a Not-ECT RTCP report and B reverts to Not-ECT.
>  - If a re-route sends both B>A and A>B traffic through an ECN-Blocking
> Middlebox, the above behaviour will also recover the session.
>
> - Once A has decided it can safely turn on ECT in RTP packets it sends, it
> watches for any RTCP reports that report reception of a positive Not-ECT
> count but zero ECT or CE count on packets 'A' sent. Once it has received
> none of these RTCP reports for three RTCP reporting rounds, 'A' can switch
> on ECT for its own RTCP reports. 'A' determines 3 reporting rounds by
> picking a couple of receivers and waiting until it receives 3 reports from
> each.
>  - A cannot test whether its own ECT RTCP reports continue to arrive at
> their destinations, because the destinations do not know to expect traffic
> from A. So if they are getting no traffic from A because of an
> ECN-Blocking  Muddlebox, the transport will never know, although
> the application or human may miss the contribution from A and reset
> the transport to use Not-ECT on RTCP reports.
>
Again the receivers should be sending periodic Receiver Reports so A
should be able to know the situation.

>  - With receiver-initiated multicast, A also cannot test for all sorts of
> other blockages that might suddenly appear on paths to some destinations.
> Therefore, once an ECN-capable session has been established, it would be
> reasonable to take a leap-of-faith that re-routes are likely to continue to
> find ECN-capable paths.   - And if the leap-of-faith is not warranted, the
> session breaks until the application resets it to Not-ECT - but that's not
> surprising if there are broken muddleboxes on some paths.
>  - As in the unicast and multi-source cases, if any hosts are pure RTP
> receivers (P,Q,R,S), they won't ever get an RTCP report about RTP packets
> they have sent, so they will never set ECT on RTCP reports.
>
There is the option for all parties exchange RTCP report when using
SSM by employing RFC5760.

> * Not-ECT-RTCP involves less implementation (unless using leap-of-faith,
> when they are both the same).
>
> BTW, the assumption that an ECN-Muddlebox will drop any ECT packets, whether
> RTP or RTCP, can be relaxed if we add reporting of RTCP packets from A>B in
> reports from B>A. But it seems a reasonable assumption so reporting on RTCP
> reports should not be necessary.
>
> Cheers
>
>
> Bob
>
>
>
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt
>

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 18 Aug 2010 17:20:01 +0100

Ken,

I agree the proposal adds more complexity to an already complex 
protocol. But I don't think the I-D needs to prohibit ECN on RTCP as 
a "SHALL NOT". It should be up to an implementer whether to trade 
lines of source code for extra fidelity.

Actually, the additional complexity is just the following two tests. 
Some logic  watching for correctness over 3 RTCP reporting periods is 
already required in the protocol, so I suspect some implementers 
might be willing to add these 2 extra tests.

Bob

At 15:26 10/08/2010, ken carlberg wrote:
>Bob,
>
>first off, let me thank you immensely for the very detailed review 
>of the document.  It will take a bit of time to carefully go over 
>all of your emails, and the various authors will probably 
>selectively pick out certain points you make and respond to them 
>directly.  And since we're scattered all over the place, we'll 
>probably do this in an ad-hoc fashion.
>
> > If congestion rises on the reverse path it /could/ be a sign that 
> it is rising on the forward path. Ideally, we want it to be 
> possible for receivers to mark RTCP reports as ECT so they are not 
> more likely to be lost just when they are needed to report rising 
> congestion levels. E.g. if the congestion level on the reverse path 
> is say, 0.5%, then 1 in 200 RTCP reports will be lost.
> > I promised to propose a way to have ECT on RTCP packets that 
> would be robust to middleboxes dropping ECT packets following a 
> re-route. The solution makes the following assumptions:
><snip>
><snip>
>
>For this particular topic, I think your proposal could be 
>accomplished.  However, I'm quite concerned about the added 
>complexity for an outlier case.  As I understand it, your overall 
>objective is to ensure that as many packets as possible are ECT 
>marked.  In addition, you'd prefer not to "reward" (for lack of a 
>better term) broken software that resides in some subset of boxes by 
>precluding ECT marking in packets.  Both objectives are 
>understandable and agreeable at face value.  But if we go down this 
>path, we are mostly providing slightly more fidelity, which I don't 
>think provides us with a substantial gain compared to the complexity 
>of the solution.
>
> >From my perspective, a big advantage in adding in adding ECN 
> support for UDP/RTP flows is the speed of notification of 
> congestive conditions with minimal disruption (ie, lost packets) of 
> the flow -- speed being the key word.  But in your proposal, we 
> still need to wait to go through a discovery process to see if its 
> ok to send ECT marked RTCP traffic, so the focus becomes additional 
> exactness involving a comparatively small subset of packets.
>
>Also, the notion that congestion in one direction "could" be a sign 
>of congestion arising in the opposite direction is on shaky ground, at best.
>
>I like the proposal as definitely good food for thought, but I'd 
>prefer not to add it to the design outlined in the draft because I 
>don't think the gain out ways the pain.
>
>cheers,
>
>-ken

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 18 Aug 2010 17:29:34 +0100

Piers,

At 20:17 17/08/2010, Piers O'Hanlon wrote:
>bi-directional flows to achieve the signalling

Looking at your inline comments, I think you've misinterpreted my 
attempt at careful wording. I said, "If B is a pure RTP receiver in a 
unidirectional session, it will never get RTCP reports containing 
counts of ECT and CE," which did not mean "it will never get RTCP 
reports". It meant "Any RTCP reports it gets will never contain 
counts of ECT or CE."

Nonetheless, I agree my proposed variation to the protocol would only 
ever turn on ECT for RTCP packets if there was some bidirectional RTP 
traffic. That's the best I could do without requiring RTCP reports to 
report on RTCP packets in the other direction. But that's better than nothing.

>but when there are

Exactly. That's what I'm proposing to use in order to turn on ECT on 
RTCP packets, which the current draft forbids.

>2xRTCP_reporting_period of loss....

You only get this amount of loss if there has been a re-route from a 
working ECN path to a non-compliant one. As stated elesewhere in my 
review, I think the draft is rather paranoid about this possibility.
1) Some people already question how much effort the IETF should put 
into making protocols robust to non-compliant muddleboxes
2) Once a compliant path has been found, would any reasonable person 
really expect us to worry about losses while timing out after being 
re-routed onto a another path that happens to be non-compliant? The 
conditional probability of being re-routed onto a non-compliant path 
having previously been on a compliant path must surely be low?

>I guess the probability of each

I agree all this is hard to quantify, but this double jeopardy is 
like the risk of a pimple growing on a spot.

>path is congested may not be indicative of the forward [RTP] path.

The proposed protocol doesn't require or assume symmetric paths.

I agree the main /motivation/ I gave for ECT on RTCP is not losing 
RTCP reports /if there is/ correlation between congestion in both 
directions. But the protocol doesn't need this correlation in order 
to work. And there are other motivations anyway.

>effort, though I can see the attraction.

There is little extra effort for the gain you get in typical cases 
(see response to Ken). We certainly should not so emphatically 
prohibit ECT on RTCP packets with the ""SHALL NOT" in the draft, 
given it can 'just work' in most deployment cases, with my proposed variation.

I think we have different assumptions about which obscure 
possibilities should be considered. For instance, I think a lot of 
the complexity dealing with re-routes onto non-compliant paths is overkill.

Personally I would leave these choices to implementers, which was the 
spirit behind my proposal to modularise the protocol spec more, so 
certain parts could be made optional.

Bob

> would be robust

---

From: ken carlberg <[email-masked]>
Date: Wed, 25 Aug 2010 17:08:12 -0400

Bob,

> Ken,
> 
> I agree the proposal adds more complexity to an already complex protocol. But I don't think the I-D needs to prohibit ECN on RTCP as a "SHALL NOT". It should be up to an implementer whether to trade lines of source code for extra fidelity.

ok, so the next level up is "MAY", but given a measure of complexity, how many vendors will punt when they see "MAY"? (I think quite a lot)

> Actually, the additional complexity is just the following two tests. Some logic  watching for correctness over 3 RTCP reporting periods is already required in the protocol, so I suspect some implementers might be willing to add these 2 extra tests.
> 

its the potential impact of the tests compared to the gains that I have concerns about.  The gain is really just a slightly higher measure of fidelity (<5%) in adding ECN to the infrequently sent RTCP packet.  But lets look at the other side.  There are a wide variety of interval times used by vendors today for sending RTCP packets.  And if you compound that with an aggregate of participants and trying to keep down the load to <=5%, then a normal RTCP packet could be in the order of minutes.  But in adding the test, we now multiply those minutes by 3.  Ok, maybe, for long lived flows, but voice calls (yes, just one example) still average just 4 minutes per duration.

also, my understanding is that we have NAT boxes that rely on RTCP packets to refresh state.  If we do the test, do we run the risk of breaking other components in the pipeline?  So, I'll stick with the notion that the minimal amount of gain doesn't outweigh the risk/pain.  Perhaps Colin/Magnus/Ingemar have different opinions.

-ken

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 26 Aug 2010 00:49:56 +0100

Ken,

At 22:08 25/08/2010, ken carlberg wrote:
> protocol. But I don't think the I-D needs to prohibit ECN on RTCP 
> as a "SHALL NOT". It should be up to an implementer whether to 
> trade lines of source code for extra fidelity.
>
>ok, so the next level up is "MAY", but given a measure of 
>complexity, how many vendors will punt when they see "MAY"? (I think 
>quite a lot)
>
> > Actually, the additional complexity is just the following two 
> tests. Some logic  watching for correctness over 3 RTCP reporting 
> periods is already required in the protocol, so I suspect some 
> implementers might be willing to add these 2 extra tests.
> >
>its the potential impact of the tests compared to the gains that I 
>have concerns about.  The gain is really just a slightly higher 
>measure of fidelity (<5%) in adding ECN to the infrequently sent 
>RTCP packet.  But lets look at the other side.  There are a wide 
>variety of interval times used by vendors today for sending RTCP 
>packets.  And if you compound that with an aggregate of participants 
>and trying to keep down the load to <=5%, then a normal RTCP packet 
>could be in the order of minutes.  But in adding the test, we now 
>multiply those minutes by 3.  Ok, maybe, for long lived flows, but 
>voice calls (yes, just one example) still average just 4 minutes per duration.

You're taking a worst case scenario, in which I wouldn't bother 
putting ECN on RTCP. But the IETF doesn't standardise protocols just 
for one use-case. These protocols get used for all sorts of things we 
might not imagine now.

I don't think it's tenable to ban everyone from putting ECN on RTCP 
messages just because there are use-cases where it doesn't make sense to.

There are equally many use-cases where it does make perfect sense - e.g.:
- in a future world where ECN deployment problems might be a thing of the past,
- or for unicast VoIP flows in the developing world trying to cope 
with high loss levels.
- or unicast VoIP designed to ride out high loss levels during disasters

(perhaps that last one was below the belt ;)

Bob

>also, my understanding is that we have NAT boxes that rely on RTCP 
>packets to refresh state.  If we do the test, do we run the risk of 
>breaking other components in the pipeline?  So, I'll stick with the 
>notion that the minimal amount of gain doesn't outweigh the 
>risk/pain.  Perhaps Colin/Magnus/Ingemar have different opinions.
>
>-ken
>
>
> >
> >
> > Bob
> >
> >> first off, let me thank you immensely for the very detailed 
> review of the document.  It will take a bit of time to carefully go 
> over all of your emails, and the various authors will probably 
> >> > If congestion rises on the reverse path it /could/ be a sign 
> that it is rising on the forward path. Ideally, we want it to be 
> better term) broken software that resides in some subset of boxes 
> by precluding ECT marking in packets.  Both objectives are 
> think provides us with a substantial gain compared to the 
> complexity of the solution.
> >>
> >> Also, the notion that congestion in one direction "could" be a 
> sign of congestion arising in the opposite direction is on shaky 
> ground, at best.
> >>

---

From: ken carlberg <[email-masked]>
Date: Thu, 26 Aug 2010 07:11:17 -0400

Bob,

> You're taking a worst case scenario, in which I wouldn't bother putting ECN on RTCP. But the IETF doesn't standardise protocols just for one use-case. These protocols get used for all sorts of things we might not imagine now.
> 
> I don't think it's tenable to ban everyone from putting ECN on RTCP messages just because there are use-cases where it doesn't make sense to.
> 
> - or for unicast VoIP flows in the developing world trying to cope with high loss levels.

yes, your points are valid, and recall in my first response that I too felt uncomfortable taking a position because we have some broken equipment out there.  But my concerns have been centered on the gain of a more complete encompassing design versus its potential negative impact.  and while I'm stubborn, I'm not intractable.  Let's wait to hear what Ingemar and Colin have to say in the days/weeks ahead.

> (perhaps that last one was below the belt ;)

quite alright, I simply did my best Jackie Chan imitation and sidestepped the blow ;-)

cheers,

-ken