Subject: Re: [tsvwg] Review of draft-ietf-tsvwg-rfc5405bis-02.txt

From: "Eggert, Lars" <[email-masked]>
Date: Mon, 1 Jun 2015 11:33:40 +0000

Hi,

sorry it took me so long to get back to this.

On 2015-4-23, at 14:31, Magnus Westerlund <[email-masked]> wrote:
> 1. Section 3.1.1:
> 
>   Applications that support an uncontrolled or unadaptive transmission
>   behavior SHOULD NOT do so by default and SHOULD instead require users
>   to explicitly enable this mode of operation.
> 
> I think on could add as an alternative to be integrated with resource reservation system so that this mode is only engaged after having been granted the resources.

That is what the text in the same paragraph before the lines you quoted is about. Maybe we could be clearer that the guideline above is for transmission over the wider Internet?

> 2. Section 3.1.2:
> 
>   Applications that at any time exchange only a few UDP datagrams with
>   a destination SHOULD still control their transmission behavior by not
>   sending on average more than one UDP datagram per round-trip time
>   (RTT) to a destination.
> 
> In the discussions around the RTP circuit breakers we have found that this basic rule has a quite significant corner case that we need to discuss. As RTT -> 0, the allowed rate goes to infinity. Now RTT will not go to infinity, but I think we have an potential issue already on a LAN. Assuming a RTT of 2 ms, you are allowed to send 500 packets a second. Just by a 1400 bytes MSS, that is 5.6 mbps. If you have a lan supporting 9k Ethernet payload, then your supported non-congestion controlled rate becomes 36 mbps. A number of instances of an application behaving like this will clearly cause issues in quite a lot of LANs.

Right, but is 5.6Mbps an issue on a LAN? Remember that if there is loss, the calculated RTT will increase, and the rate therefore decrease. When there is no loss and the RTT is 2ms, obviously the LAN can sustain that traffic. (TFTP is basically operating in this way, and doesn't seem to create huge issues on LANs.)

> So the issue is really that any application that send sufficient number of packets per seconds also needs to be congestion controlled, even if they are of a one packet per RTT type.

Right. There is some text in the beginning of 3.1 that tries to say so:

   It is important to stress that an
   application SHOULD perform congestion control over all UDP traffic it
   sends to a destination, independently from how it generates this
   traffic.  For example, an application that forks multiple worker
   processes or otherwise uses multiple sockets to generate UDP
   datagrams SHOULD perform congestion control over the aggregate
   traffic.

Maybe that's not clear enough?

> This should be clarified, and possibly some guidance on what to frequent really is. An application sending 10 packets per seconds to a destination, even if only one per RTT clearly can sample the path well enough to have a proper CC.

Fair point. Should we then introduce some limit on pps that is considered "low data rate" and for everything above that say "use CC"? What limit should we pick?

> 3. Section 3.1.3:
> 
>   Even low data-volume UDP flows may benefit from rate control, e.g.,
>   an application that sends three copies of a packet to improve
>   robustness to loss is RECOMMENDED to pace out those three packets
>   over several RTTs, to reduce the probability that all three packets
>   will be lost due to the same congestion event.
> 
> I would suggest that one changes "rate control" in the first line of the above paragraph to "packet pacing" as that is what being discussed. Usage of rate control is to easily misinterpreted to mean not the packet rate, but also the total bit-rate.

Done, thanks.

> 4. Section 3.1.5:
> 
>   Applications that support an
>   uncontrolled or unadaptive transmission behavior SHOULD NOT do so by
>   default and SHOULD instead require users to explicitly enable this
>   mode of operation.
> 
> Same comment as in comment 1 above.

Same answer :-) Should we maybe point to the text in 3.1 or repeat it here?

> 5. Section 3.4:
> 
>   Application developers SHOULD implement additional checks
>   where data integrity is important, e.g., through a Cyclic Redundancy
>   Check (CRC) included with the data to verify the integrity of an
>   entire object/file sent over the UDP service.
> 
> I wonder if we should clarify this recommendation on additional checks. I think CRCs are good ways to verify that non-malicous changes has occured to the payload, but CRC may not be appropriate to "entire object/file" where better detection mechanisms may be needed that supports longer object better. Unless people would use a really long CRC polynom, I think keyed or non-keyed cryptographic hashes are more appropriate recommendation for object level changes, and if it can be keyed also protect against malicious changes.

CRC is given as an example. I'm adding "e.g., through a Cyclic Redundancy Check (CRC) or keyed or non-keyed cryptographic hash included with the data to verify the integrity of an entire object/file sent over the UDP service."

> 6. Section 3.5:
> 
>    When an application is deployed in a
>   controlled network environment, the deployer SHOULD investigate
>   whether the target environment allows applications to use longer
>   intervals, or whether it offers mechanisms to explicitly control
>   middlebox state timeout durations, for example, using Middlebox
>   Communications (MIDCOM) [RFC3303], Next Steps in Signaling (NSIS)
>   [RFC5973], or Universal Plug and Play (UPnP) [UPnP].
> 
> Add Port Control Protocol to the list?

Done.

> 7. Section 4:
> 
>   The IETF has defined a reliable multicast framework [RFC3048] and
>   several building blocks to aid the designers of multicast
>   applications, such as [RFC3738] or [RFC4654].  Anycast senders must
>   be aware that successive messages sent to the same anycast IP address
>   may be delivered to different anycast nodes, i.e., arrive at
>   different locations in the topology.
> 
> I think this paragraph needs to be split into two parts. As the first sentence and the Anycast one appears quite unrelated.

Done.

> 8. Section 6:
> 
> The IETF standard for securing RTP [RFC3550]
>   communication sessions over UDP is the Secure Real-time Transport
>   Protocol (SRTP) [RFC3711].
> 
> I think this sentence needs to be changed and primarily reference RFC7201. I would rather propose something like:
> 
> There exist a number of security options for RTP [RFC3550] over UDP, especially to accomplish key-management, see [RFC7201]. These options covers many different usages, including point-to-point, centralized group communication as well as multicast.

Changed.

Thanks!

Lars

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 2 Jun 2015 13:44:23 +0200

Hi,

Please see inline. Snipped dealt with issues.

Eggert, Lars skrev den 2015-06-01 13:33:
>> Applications that support an uncontrolled or unadaptive
>> transmission behavior SHOULD NOT do so by default and SHOULD
>> instead require users to explicitly enable this mode of operation.
>>
>> I think on could add as an alternative to be integrated with
>> resource reservation system so that this mode is only engaged after
>> having been granted the resources.
>
> That is what the text in the same paragraph before the lines you
> quoted is about. Maybe we could be clearer that the guideline above
> is for transmission over the wider Internet?

My point is that one can reinforce that if one assumes reserved path 
capacity, one actually needs to ensure it. Just assuming without having 
methods for verifying it is not good.

>
>> 2. Section 3.1.2:
>>
>> Applications that at any time exchange only a few UDP datagrams
>> with a destination SHOULD still control their transmission behavior
>> by not sending on average more than one UDP datagram per round-trip
>> time (RTT) to a destination.
>>
>> In the discussions around the RTP circuit breakers we have found
>> that this basic rule has a quite significant corner case that we
>> need to discuss. As RTT -> 0, the allowed rate goes to infinity.
>> Now RTT will not go to infinity, but I think we have an potential
>> issue already on a LAN. Assuming a RTT of 2 ms, you are allowed to
>> send 500 packets a second. Just by a 1400 bytes MSS, that is 5.6
>> mbps. If you have a lan supporting 9k Ethernet payload, then your
>> supported non-congestion controlled rate becomes 36 mbps. A number
>> of instances of an application behaving like this will clearly
>> cause issues in quite a lot of LANs.
>
> Right, but is 5.6Mbps an issue on a LAN? Remember that if there is
> loss, the calculated RTT will increase, and the rate therefore
> decrease. When there is no loss and the RTT is 2ms, obviously the LAN
> can sustain that traffic. (TFTP is basically operating in this way,
> and doesn't seem to create huge issues on LANs.)

5,6 mbps can clearly be significantly above your fair share. I am also 
worried that you will shave of a factor of 20 from the assumed RTT and 
suddenly you saturate a 100 mbps lan with just that stream. Sure, loss 
equals delay in ack and thus reduced rate. But, please do remember that 
if the RTT and packet size and amount of data to deliver support a 
higher than available bandwidth this protocol will contribute 
substantially to persistent loss as it doesn't tune itself to how much 
it overshoot.

The reason we commented on this, was that in RTP and the circuit breaker 
we don't have the lock step mechanism of TFTP, instead  a loss is just a 
loss. And because the RTT is short the circuit breaker becomes disabled. 
So this is highly dependent on the context and what your bulk transfer 
application is doing.

>
>> So the issue is really that any application that send sufficient
>> number of packets per seconds also needs to be congestion
>> controlled, even if they are of a one packet per RTT type.
>
> It is important to stress that an application SHOULD perform
> congestion control over all UDP traffic it sends to a destination,
> independently from how it generates this traffic.  For example, an
> application that forks multiple worker processes or otherwise uses
> multiple sockets to generate UDP datagrams SHOULD perform congestion
> control over the aggregate traffic.
>

That is not what I am talking about. In my case we have a single process 
sending a single stream of packets. The issue is that one packet per RTT 
is still potentially to much.

>
>> This should be clarified, and possibly some guidance on what to
>> frequent really is. An application sending 10 packets per seconds
>> to a destination, even if only one per RTT clearly can sample the
>> path well enough to have a proper CC.
>
> Fair point. Should we then introduce some limit on pps that is
> considered "low data rate" and for everything above that say "use
> CC"? What limit should we pick?

Yes, this is the difficult, but I think above 50 pps (i.e. one per 20 
ms) is high enough rate that it appears reasonable to require CC.

>
>> 4. Section 3.1.5:
>>
>> Applications that support an uncontrolled or unadaptive
>> transmission behavior SHOULD NOT do so by default and SHOULD
>> instead require users to explicitly enable this mode of operation.
>>

I think it depends on how big the resulting text is. If small I think 
repeat makes sense, if not reference.

>
>> 5. Section 3.4:
>>
>> Application developers SHOULD implement additional checks where
>> data integrity is important, e.g., through a Cyclic Redundancy
>> I wonder if we should clarify this recommendation on additional
>> checks. I think CRCs are good ways to verify that non-malicous
>> changes has occured to the payload, but CRC may not be appropriate
>> to "entire object/file" where better detection mechanisms may be
>> needed that supports longer object better. Unless people would use
>> a really long CRC polynom, I think keyed or non-keyed cryptographic
>> hashes are more appropriate recommendation for object level
>> changes, and if it can be keyed also protect against malicious
>> changes.
>
> CRC is given as an example. I'm adding "e.g., through a Cyclic
> Redundancy Check (CRC) or keyed or non-keyed cryptographic hash
> included with the data to verify the integrity of an entire
> object/file sent over the UDP service."

Yes, that looks good.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Services, Media and Network features, Ericsson Research EAB/TXM
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Färögatan 6                 | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: "Eggert, Lars" <[email-masked]>
Date: Thu, 4 Jun 2015 09:54:15 +0000

Hi,

On 2015-6-2, at 14:44, Magnus Westerlund <[email-masked]> wrote:
> My point is that one can reinforce that if one assumes reserved path capacity, one actually needs to ensure it. Just assuming without having methods for verifying it is not good.

ah, got it. I added "and they SHOULD verify that sufficient path capacity has been reserved for them" to the end of the last sentence.

> 5,6 mbps can clearly be significantly above your fair share.

Sure, but then there would be loss, and the computed RTT would increase and the bandwidth would become reduced.

> I am also worried that you will shave of a factor of 20 from the assumed RTT and suddenly you saturate a 100 mbps lan with just that stream.

But only for at 1/20th RTT, because after that, you would start seeing loss and the rate would reduce.

> Sure, loss equals delay in ack and thus reduced rate. But, please do remember that if the RTT and packet size and amount of data to deliver support a higher than available bandwidth this protocol will contribute substantially to persistent loss as it doesn't tune itself to how much it overshoot.

Right. But having one message per RTT outstanding and adapting the RTT based on what is measured does tune the rate to the path characteristics. It's not great, but it should prevent causing persistent congestion.

> The reason we commented on this, was that in RTP and the circuit breaker we don't have the lock step mechanism of TFTP, instead  a loss is just a loss. And because the RTT is short the circuit breaker becomes disabled. So this is highly dependent on the context and what your bulk transfer application is doing.

I'm not sure I follow the example here?

> That is not what I am talking about. In my case we have a single process sending a single stream of packets. The issue is that one packet per RTT is still potentially to much.

I think this document (silently) assumes that "too much" means "packet loss is seen." It seems that in your case, you have a rate that you consider "too much", but there is no packet loss yet?

> Yes, this is the difficult, but I think above 50 pps (i.e. one per 20 ms) is high enough rate that it appears reasonable to require CC.

OK. I personally have no strong view here, if the WG has one. (I do have a mild displeasure at having to introduce some magic constant, but I can totally live with this if we agree on one.)

WG, what do you think?

Lars

---

From: Magnus Westerlund <[email-masked]>
Date: Thu, 11 Jun 2015 16:28:59 +0200

> ah, got it. I added "and they SHOULD verify that sufficient path
> capacity has been reserved for them" to the end of the last
> sentence.

Great!

> Sure, but then there would be loss, and the computed RTT would
increase and the bandwidth would become reduced.

Assuming that they either have lock step mechanism or samples frequently 
the RTT. If one has infrequent RTT sampling, then the sampled RTT may 
not reflect the congested state, and thus provide no help in detecting 
the congestion occurance, thus no back off.

>
>> I am also worried that you will shave of a factor of 20 from the
assumed RTT and suddenly you saturate a 100 mbps lan with just that stream.
>
> But only for at 1/20th RTT, because after that, you would start
> seeingloss and the rate would reduce.
>
>> Sure, loss equals delay in ack and thus reduced rate. But, please
>> doremember that if the RTT and packet size and amount of data to deliver
>> support a higher than available bandwidth this protocol will contribute
 >> substantially to persistent loss as it doesn't tune itself to how
 >> much it overshoot.
>
> Right. But having one message per RTT outstanding and adapting the
> RTT
based on what is measured does tune the rate to the path
characteristics. It's not great, but it should prevent causing
persistent congestion.

Sure, if you actually have a method that really ensures that it has one 
and only one packet outstanding and then an accurate enough RTT 
estimator to deal with any re-transmissions then it is self limiting.

>
>> The reason we commented on this, was that in RTP and the circuit
breaker we don't have the lock step mechanism of TFTP, instead a loss is
just a loss. And because the RTT is short the circuit breaker becomes
disabled. So this is highly dependent on the context and what your bulk
transfer application is doing.
> I'm not sure I follow the example here?

I think you need to look at Section 4.1 of 
https://datatracker.ietf.org/doc/draft-ietf-avtcore-rtp-circuit-breakers/

That draft derives from the UDP guidelines that we basically turn off 
the forward path circuit breaker if less than one RTP packet per RTT 
where sent. The RTT sampling is done using RTCP, so it may be as 
infrequent as on average every 5 seconds.

The intention with turning the circuit breaker off if so that when 
transmission of RTP packets are infrequent as single or very small 
number of packet losses should not trigger the circuit breaker.

So basically we have turned the one 1 per RTT rule into a fence that 
actually allows quite high rates. And due to how the measurements are 
done it is insensitive to congestion events.

What I think may be needed in the UDP guidelines is to clarify that if 
one applies the no more than one per RTT, that actually have to do 
frequent sampling, or be some lock step protocol which thus would be 
self limiting in the face of congestion.

It is not okay to apply once per RTT as a rule of thumb.

(Damm, I hate it when I make it more difficult for myself)

Cheers

Magnus Westerlund

---

From: [email-masked]
Date: Thu, 11 Jun 2015 16:42:22 +0100

See a thought at the end.

>> capacity has been reserved for them" to the end of the last
WFM also.
> (Damm, I hate it when I make it more difficult for myself)

My researchers have been experimenting the RTCP circuit breaker, and came
across a similar problem: I think the problem we saw (large allowed rates
for low RTT) was not a problem specifically with low RTT, it was more the
time to REACT to an overload can become much larger than predicted by the
actual path RTT.

When a flow measures loss (and RTT) less often than every RTT, the Circuit
Breaker can still predict a reasonable safe rate. However, the source
continues to send for (multiple times) the measurement period. So an RTCP
report every few seconds over a path with a few mS of delay actually
creates as much damage to other flows as a flow would with a RTT of
several seconds, because for this time it continues to ignore the
congestion it experienced.

To put it another way, I wonder whether the recommendation should be one
packet per Max (RTT, rate adjustment interval)l (may need a better term)
rather, than one packet per path RTT.  If a flow operates by checking for
loss (or whatever congestion metric) for each packet ack'ed then this is
of course the same as one packet per RTT.

Gorry