Subject: Re: [Tsvwg] [tsv-area] TSVWG last call on "UDP Usage Guidelines for Application Designers" to BCP

From: Lars Eggert <[email-masked]>
Date: Mon, 12 May 2008 17:57:34 +0300

Hi,

On 2008-4-21, at 23:14, ext Robert Hancock wrote:
> - this document restricts its scope to UDP, but is there any reason  
> why the
> same content would not apply to 'applications' using any other
> non-congestion-controlled protocol? (Is there some reason to  
> restrict to UDP
> in particular?)

are you referring to datagram-based protocols other than UDP, e.g.,  
stuff that uses a custom IP protocol number? If so, I agree that many  
of these guidelines would probably apply in spirit, but it depends a  
lot on the individual case. I'd assume that if something runs directly  
on top of IP, it (1) fulfills a very specific (narrow) need and (2) is  
designed by folks that should be ore clueful than the average UD-based  
application designer - you need a standards action or IESG approval to  
get an IP number assigned.

But if people think it would be worth mentioning - in an informal way  
- that the spirit of these guidelines is worth keeping in mind when  
designing such IP-based protocols, that'd be fine with me. But I'd  
like to see some WG support before I make this addition.

> - is it intended that adherence to these principles will be standard  
> IESG
> review criteria (e.g. will this start to get referenced from
> http://www.ietf.org/ID-Checklist.html?)

I don't think we want to make compliance with this BCP-to-be a strict  
requirement for new protocols. I'd argue that for every single one of  
the guidelines in this document, there exist valid reasons to _not_  
follow it when designing different kinds of application protocols.  
Generally, without any specific reason not to do otherwise, I'd expect  
that most application protocols would do well to more or less follow  
these guidelines, but that's about all that can be said, I think.

Do you think that needs to be clarified in the document?

> 1. Introduction
>
> Para 3: It might be worth adding some text to the effect that  
> congestion
> sensitivity is in the interests of the application itself, which  
> otherwise
> may not be obvious to some of the intended audience. (At the moment  
> it reads
> that congestion avoidance is purely a matter of good citizenship  
> rather than
> self interest.)

If added this: "In addition, congestion-controlled transmission is of  
benefit to an application itself, because it can reduce self-induced  
packet loss, minimize retransmissions and hence reduce delays."

> Final para: does this guidance apply to anycast also?

Many of the guidelines should apply. Anycast addresses have the  
feature that they are allowed to jump around in the topology from one  
packet to the next (although in practice, that almost doesn't seem to  
happen.) Since the document mostly says "adapt your transmission based  
on the path conditions", that should be fine.

I'm wondering though if we want to add a sentence to the programming  
guidelines section about not assuming that anycast packets always end  
up in the same location. Then again, applications that use anycast are  
hopefully designed by folks that realize this.

> 3. Usage
>
> Para 1: It might be worth noting that, if an application knows its  
> peer is
> on the local link (e.g. because of the destination address), then it  
> can
> make stronger assumptions about the path, and congestion control and  
> mtu
> issues should not have to be handled (at least at the IP layer).

I give you the MTU part, but just because you're on the same link with  
your destination doesn't mean you know anything about traffic loads on  
that link. And a link could be a tunnel. Finally, the document is  
supposed to give guidelines for Internet communication, and I don't  
want to put to much stuff that's only applicable in more restricted  
environments in it.

> Para 3 (end): In addition, TCP is probably more efficient at tuning  
> its RTO
> and spotting when a retransmission is actually needed than most home- 
> brew
> application-layer reliability schemes.

This paragraph now begins with: "If used correctly, these more fully- 
featured transport protocols are not as "heavyweight" as often  
claimed. For example, the TCP algorithms have been continuously  
improved over decades, and have reached a level of efficiency (not to  
forget correctness) that home-brew application-layer mechanisms will  
struggle to easily duplicate. In addition, many TCP implementations  
allow connections to be tuned by an application to its purposes. For  
example, TCP's "Nagle" algorithm..."

> 3.1.2 penultimate para "Note that the initial timeout...": I found  
> this
> confusing because there is no 'timeout' in such scenarios: what you  
> mean in
> general is that there is a natural timescale over which UDP  
> applications
> should send a single packet; this is either the RTT (if you can  
> measure that
> - not the timeout in this case either), or a fixed number if you  
> cannot.

Good catch. I rephrased this as: "Note that the sending rate in this  
case must be more conservative than in the two previous cases..."

> 3.1.3 bullet 2 (this may just be a typo)
> s/to be IP-based or is known to be IP-based, but not congestion- 
> controlled/
>  to be IP-based, or is known to be IP-based but not congestion- 
> controlled.

Fixed.

> 3.2 para 3:
> I found this very confusing. Surely you are saying that applications  
> SHOULD
> adapt their transmit message size; if they "choose not to" they may  
> generate
> UDP datagrams with precisely the properties you complain about. I  
> think the
> term 'effective PMTU' should be called out more formally (I am fine  
> with the
> definition, 576 magic number and all, but would find "... exceed the
> effective PMTU, defined as follows:") easier to follow.

That paragraph is supposed to discuss applications that chose (for  
whatever reason) to not do what the previous paragraph recommends (do  
PMTU discovery).

I changed this paragraph to begin like this in my working copy, please  
let me know if this is an improvement:

"Applications that do not follow this recommendation SHOULD still  
avoid sending UDP messages that would result in IP datagrams that  
exceed the path MTU. Because the actual path MTU is unknown, such  
applications SHOULD fall back to sending messages that are shorter  
that the default effective MTU for sending (EMTU_S in [RFC1122]).  
EMTU_S is the smaller of 576 bytes and the first-hop MTU for IPv4  
[RFC1122]> and 1280 bytes for IPv6 [RFC2460]>. (...)"

> Two issues aren't highlighted explicitly in this part of the text:
> - applications which do their own fragmentation to achieve reliable  
> delivery
> of a large message need to be encouraged to do efficient reassembly  
> (i.e.
> the ability to retransmit a single UDP datagram rather than the whole
> message); otherwise they are just repeating the inefficiency of the  
> IP layer
> but with better middlebox traversal properties.

I'm open for a suggestion for a specific text change, but we should  
keep this short - it's bordering on giving protocol design guidance,  
which is broader than the scope of this document.

> - applications which generate multiple UDP datagrams for a single
> application message need to be encouraged (do they?) to apply  
> congestion
> control to the individual UDP datagrams, rather than to the  
> application
> layer message. Otherwise they will hit the network with packet bursts.

The whole document is pretty careful to always talk about UDP packets  
rather than application-level messages when it comes to congestion  
control. Do you think we need to add an explicit statement? (Where &  
what should it say, specifically?)

> 3.5 penultimate para:
> - it might be a good idea to recommend some jitter here (for  
> synchronisation
> avoidance).

If added this as the last sentence of that paragraph: "It is  
RECOMMENDED that applications apply slight random variations  
("jitter") to the timing of keep-alive transmissions, in order to  
reduce the potential for persistent synchronization between keep-alive  
transmissions from different hosts."

Lars

PS: I'll go through the other LC comments we've received and respond  
to those emails one-by-one, before posting a diff between the then- 
current working copy of the draft and -06 for final review. (But not  
before tomorrow.)

---

From: Lars Eggert <[email-masked]>
Date: Tue, 13 May 2008 11:51:53 +0300

On 2008-5-12, at 17:57, ext Lars Eggert wrote:
>> - is it intended that adherence to these principles will be  
>> standard IESG
> I don't think we want to make compliance with this BCP-to-be a  
> strict requirement for new protocols. I'd argue that for every  
> single one of the guidelines in this document, there exist valid  
> reasons to _not_ follow it when designing different kinds of  
> application protocols. Generally, without any specific reason not to  
> do otherwise, I'd expect that most application protocols would do  
> well to more or less follow these guidelines, but that's about all  
> that can be said, I think.
>

I've added this paragraph to the introduction:

"It is worth noting that this document describes guidelines and  
recommendations, and does not make mandatory requirements or impose  
immutable rules. Although most UDP applications would generally do  
well to follow the guidelines in this document, for any given  
guideline, there do exist valid reasons to _not_ follow it when  
designing a specific kind of application. In such cases, it is  
recommended that the application designers document the rationale for  
their design choice in the technical specification of their  
application or protocol."

Lars

---

From: "Hancock, Robert" <[email-masked]>
Date: Tue, 13 May 2008 10:16:05 +0100

hi,

snipped all i agreed with, and ...

> -----Original Message-----
> From: Lars Eggert [mailto:[email-masked]] 
> Sent: 12 May 2008 15:58
> To: Hancock, Robert
> Cc: [email-masked]; [email-masked]
> Subject: Re: [tsv-area] TSVWG last call on "UDP Usage 
> Guidelines for Application Designers" to BCP
> 
> Hi,
> 

i think such a mention would be useful. 
(as you've just sent to the list.)
 
> standard  

indeed that is quite true. but isn't that precisely the point of
distinguishing MUST and SHOULD in the document (otherwise what is 
the point of using 2119 language in it)? Should all the MUSTs actually
be SHOULDs?
 
> [RFC1122]> and 1280 bytes for IPv6 [RFC2460]>. (...)"

ok, but i would
s/follow this recommendation/do PMTU discovery/
(just to be clear)

> which is broader than the scope of this document.

the distinction between the two is a bit subtle for me in this
case, but how about
[see below for terminology]
"Applications that fragment a message over multiple UDP datagrams
SHOULD carry out that fragmentation so each datagram can be 
received independently, and retransmitted independently if the
application implements its own reliability mechanisms."
(or just leave it out).

> what should it say, specifically?)

this may just be a terminology issue: the document uses both 'message'
and 'UDP message' and it wasn't clear to me that they were supposed
to be equivalent (indeed, in parts of 3.2 they seemed to be quite
different). personally i prefer 'UDP datagram', and leave message
to refer to some fluffy application layer thingy.

robert h.

---

From: Lars Eggert <[email-masked]>
Date: Tue, 13 May 2008 14:00:31 +0300

Hi,

> the point of using 2119 language in it)? Should all the MUSTs actually

all the BCP guidelines are SHOULDs (or should be). The MUSTs in the  
document come from other specifications (or should).

The one current exception I know of is the "MUST accommodate wide  
range of Internet path conditions". I'm borderline whether this should  
remain a MUST or become a SHOULD. (Consistency with other guidelines  
would argue for a SHOULD, but I can't think of a case where an  
Internet application wouldn't need to accommodate wide range of  
Internet path conditions.)

> s/follow this recommendation/do PMTU discovery/

Changed.

> application implements its own reliability mechanisms."

Added this (with some minor wording changes.)

> to refer to some fluffy application layer thingy.

OK. I've changed terms throughout so that the document now  
consistently talks about UDP datagrams, IP packets, and application- 
layer messages.

Lars

---

From: Joe Touch <[email-masked]>
Date: Thu, 15 May 2008 06:44:11 -0700

Lars Eggert wrote:
...
> The one current exception I know of is the "MUST accommodate wide range 
> of Internet path conditions". I'm borderline whether this should remain 
> a MUST or become a SHOULD. (Consistency with other guidelines would 
> argue for a SHOULD, but I can't think of a case where an Internet 
> application wouldn't need to accommodate wide range of Internet path 
> conditions.)

It depends on your definition of accommodate. I think this too is 
sufficient as a SHOULD.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 15 May 2008 13:45:21 -0700

Hi, Gorry,

Gorry Fairohurst wrote:
> Joe Touch wrote:
>>> range of Internet path conditions". I'm borderline whether this 
>>> should remain a MUST or become a SHOULD. (Consistency with other 
>>> guidelines would argue for a SHOULD, but I can't think of a case 
>>> where an Internet application wouldn't need to accommodate wide range 
>>> of Internet path conditions.)
>>
> I my less happy with "SHOULD" than "MUST" here. I can see we could 
> debate what accommodate means in this context, but basically I believe 
> we must have an expectation that the transport protocols work across all 
> parts of the Internet (i.e. for which we have an IP over Foo Spec). I'd 
> expect someone to say how much they accommodate variation, not that 
> simply this transport only works over "foo" because they only expect it to.

The question is the term "accommodate" - there's no need to adjust to 
variations (accommodate, specifically), vs. to tolerate variations. I'd 
require the latter, but the former - though desirable - may not be required.

joe

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 15 May 2008 20:48:03 +0100

Gorry

---

From: Lars Eggert <[email-masked]>
Date: Mon, 19 May 2008 12:12:47 +0300

On 2008-5-15, at 23:45, ext Joe Touch wrote:
>>>> where an Internet application wouldn't need to accommodate wide  
>>>> range of Internet path conditions.)
>>>
>> debate what accommodate means in this context, but basically I  
>> believe we must have an expectation that the transport protocols  
>> work across all parts of the Internet (i.e. for which we have an IP  
>> over Foo Spec). I'd expect someone to say how much they accommodate  
>> variation, not that simply this transport only works over "foo"  
>> because they only expect it to.
>
> The question is the term "accommodate" - there's no need to adjust  
> to variations (accommodate, specifically), vs. to tolerate  
> variations. I'd require the latter, but the former - though  
> desirable - may not be required.

I'm happy to change "accommodate" to "tolerate" if you think that's  
more clear, or maybe "adapt to", which may be even more clear.

Lars

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 May 2008 07:17:52 -0700

See below...

Lars Eggert wrote:
>>> believe we must have an expectation that the transport protocols work 
>>> across all parts of the Internet (i.e. for which we have an IP over 
>>> Foo Spec). I'd expect someone to say how much they accommodate 
>> variations (accommodate, specifically), vs. to tolerate variations. 
>> I'd require the latter, but the former - though desirable - may not be 
>> required.
> 
> I'm happy to change "accommodate" to "tolerate" if you think that's more 
> clear, or maybe "adapt to", which may be even more clear.
> 
> Lars

I think "tolerate" is better. "adapt to" implies the opposite of what I 
think MUST should require.

Joe

---

From: Lars Eggert <[email-masked]>
Date: Wed, 21 May 2008 12:01:38 +0300

On 2008-5-19, at 17:17, ext Joe Touch wrote:
> I think "tolerate" is better. "adapt to" implies the opposite of  
> what I think MUST should require.

I changed "accommodate" to "tolerate". This phrasing only occurred in  
the summary table at the end of the document. The text itself actually  
says:

"Internet paths can have widely varying characteristics, including  
transmission delays, available bandwidths, congestion levels,  
reordering probabilities, supported message sizes or loss rates.  
Furthermore, the same Internet path can have very different conditions  
over time. Consequently, applications that may be used on the Internet  
MUST NOT make assumptions about specific path characteristics. They  
MUST instead use mechanisms that let them operate safely under very  
different path conditions. Typically, this requires conservatively  
probing the current conditions of the Internet path they communicate  
over to establish a transmission behavior that it can sustain and that  
is reasonably fair to other traffic sharing the path."

Just to double-check, in light of the recent discussion, we are still  
OK with how this is phrased?

Lars

---

From: Joe Touch <[email-masked]>
Date: Wed, 21 May 2008 06:34:55 -0700

This all looks OK to me.

Joe

Lars Eggert wrote:
>> I think "tolerate" is better. "adapt to" implies the opposite of what 
>> I think MUST should require.
> 
> transmission delays, available bandwidths, congestion levels, reordering 
> probabilities, supported message sizes or loss rates. Furthermore, the 
> same Internet path can have very different conditions over time. 
> Consequently, applications that may be used on the Internet MUST NOT 
> make assumptions about specific path characteristics. They MUST instead 
> use mechanisms that let them operate safely under very different path 
> conditions. Typically, this requires conservatively probing the current 
> conditions of the Internet path they communicate over to establish a 
> transmission behavior that it can sustain and that is reasonably fair to 
> other traffic sharing the path."
> 
> Just to double-check, in light of the recent discussion, we are still OK 
> with how this is phrased?
> 
> Lars