Subject: RE: [Tsvwg] Re: Last Call: The UDP Lite Protocol to Proposed Standard

From: Phillip Conrad <[email-masked]>
Date: Mon, 06 May 2002 12:47:32 -0400

Pete:
    You have raised some good points (some of which were also raised 
independently by Michael Ramalho in this same thread):

* the formula PER=1-(1-BER)^n only applies to i.i.d. bit errors, which is not
   necessarily the kind of error patterns we would see in the environment for
   which UDP-Lite is targetted (e.g. 3GPP).

* the choice of n values for partial checksum vs. full checksum that
   I chose are not necessarily the right ones for that environment.

* IPv6 lacks a network layer checksum

To clarify: my "quick and dirty" analysis was intended mainly to argue that 
extra signalling to inform the upper layer of the BER of the channel (as 
suggested by others in this thread) is not necessarily needed.  Given that 
"purpose" I think the following statements may still "hold up":

(1) that the niche for UDP-Lite is where the BER falls into a "sweet spot" 
somewhere between

-- "too low", where there is no significant difference between using a 
partial checksum and a full checksum, because checksum errors are so rare 
anyway.

-- "too high" where very few packets pass even the UDP-lite partial 
checksum (which in light of new facts you present, is the IP-psuedoheader, 
the 8-byte UDP-lite header, and probably the RTP header as well.)

(2) That the sweet spot is where the difference between the PER for partial 
UDP-lite checksum and full UDP (or UDP-lite) checksum is significant enough 
to make a difference to the end user (in terms of, for example, improved 
audio, video, or image quality)

(3) That in principle, if we choose the right mathematical model for the 
channel (which may not be i.i.d bit errors, but some other model), we 
should be able to determine some reasonable target range of BERs that fall 
into the "sweet spot."

(4) That, given some reasonable assumptions about the channel, the observed 
PER (which can be determined from, e.g. looking at gaps in RTP sequence 
numbers) is a good predictor of at least the order of magnitude of an upper 
bound for BER that might be observed in the unprotected part of the payload.

(Note: "upper bound", because there will be "false positives" from packets 
that may be been dropped due to congestion rather than bit errors detected 
by intermediate links, or the final UDP-Lite partial checksum over the header.)

Keith Moore suggested that my analysis (or a similar discussion) be made 
part of the draft, perhaps as an appendix.     I'm grateful for the 
suggestion, but in light of some valid criticisms of the details of my 
analysis made by Pete and Michael, I'd suggest that IF anything goes into 
the draft on this point (and I'm not saying that anything necessarily 
should) it should be either:

(a) a new improved analysis that takes Pete's observations about 3GPP 
environments into account (which probably should be done by someone 
familiar with how to properly model the distribution bit errors in that 
environment, which is not me :-) ), or else,

(b) just a discussion of the general tradeoffs involved (similar to the 
points (1) to (4) above) leading to the idea that the PER that results from 
the partial checksum provides a natural upper and lower bound  on the BER 
for which UDP-lite is useful, and that this PER and BER may vary from 
network to network.

Thanks,
Phill Conrad

---
Phill Conrad -- Asst. Professor CIS Dept, Temple University
http://www.cis.temple.edu/~conrad http://netlab.cis.temple.edu

At 08:45 AM 5/6/2002 -0500, Peter Barany wrote:

>Hi,
>
>A typical application that might use UDP Lite would be, for example, 3GPP 
>UMTS packet-switched voice (via the RTP payload format for the AMR speech 
>codec).
>
>For such a scenario, it can be anticipated that the RTP/UDP or UDP 
>Lite/IPv6 header will be compressed over the air interface using a an 
>existing/modified ROHC profile (depending upon whether UDP or UDP Lite is 
>used as the transport layer protocol) (see IETF ROHC WG work on this). 
>Therefore, the header size you mention below will typically not be RTP (12 
>bytes) + UDP or UDP Lite (8 bytes) + IPv6 (40 bytes). NOTE: I use IPv6 
>because it is the network layer protocol mandated by 3GPP for use with 
>packet-switched multimedia services.
>
>Also, for the RTP payload format for the AMR speech codec, you might 
>expect an RTP payload size of around 32 bytes for the highest codec rate 
>of 12.2 kbps (assuming one speech frame per RTP packet).
>
>Finally, the formula you use below is accurate/correct only if you have 
>random errors (i.e., uncorrelated) such as would be the case, for example, 
>with a binomial pdf. But wireless channels cannot be modeled this way.
>
>Therefore, I am not convinced that your calculations below are meaningful 
>for an air interface like 3GPP UMTS (or any other cellular system using 
>header compression).
>
>Now, once the RTP/UDP or UDP Lite/IPv6 reach the landline network, it is 
>anticipated that the BER will drop dramatically due to the nature of the 
>landline network vs. a cellular network (also, the BER pdf will more 
>closely resemble the binomial distribution you use in your formula). But 
>the potential benefits of UDP Lite are not really targeted for landline 
>networks. So, we are really talking about the benefits of UDP Lite over an 
>air interface that exist in, for example, cellular networks.
>
>FYI. Also, for IPv6, there is no header checksum as there is with IPv4.
>
>Regards,
>
>Pete

---

From: Keith Moore <[email-masked]>
Date: Mon, 06 May 2002 13:45:24 -0400

> for which UDP-lite is useful, and that this PER and BER may vary from 

I'm not sure the analysis is actually needed.  What I want is a statement
that provides some guidance to application and link-layer implementors. 
I.e. a statement about the assumed applicability of UDP Lite - e.g. 

- that link layers that wish to treat UDP Lite specially should provide
  an error rate of X or better, and 

- that applications choosing to use UDP Lite should be reasonably sure of 
  operating effectively with that error rate

Keith 

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "La Monte Henry Piggy Yarroll" <[email-masked]>
Date: Tue, 07 May 2002 22:05:16 -0500

Keith Moore <[email-masked]> writes:
> [Phillip Conrad <[email-masked]> writes:]

As cool as I think Phil's analysis, I do not think that we need to
include it in the UDP Lite draft.  It MIGHT be of interest in an
applicability statement.

Link layer error rate analysis is completely out of scope, even for
UDP Lite.  This is NOT the ITU-T--we do not spec the thresholds of
every network element from layer 1 to 7.  The less the final RFC says
about specific error rates, the better.  It is very unlikely that a
designer will pick UDP Lite without first having some notion of an
error-tolerant application.

If you are seeking to design a codec with a hard roll-off at some
particular BER or PER, you are probably designing a codec which will
not perform well on the live Internet.  This is the reason that so
many codecs designed for cellular telephony do perform particularly
well on the Internet.  You need a good soft rolloff over a very broad
range of PER.

---

From: Lloyd Wood <[email-masked]>
Date: Wed, 8 May 2002 12:01:12 +0100 (BST)

On Tue, 7 May 2002, La Monte Henry Piggy Yarroll wrote:

> UDP Lite.

Unfortunately, error rate analysis is the only possible
justification for UDP Lite's existence.

UDP Lite over IPSec, IPComp or VPNs isn't worthwhile, is it? That
will discourage application takeup.

L.

<[email-masked]>PGP<http://www.ee.surrey.ac.uk/Personal/L.Wood/>

---

From: Keith Moore <[email-masked]>
Date: Tue, 07 May 2002 23:14:03 -0400

> error-tolerant application.

I disagree.  You seem to think that users of UDP Lite - both at
the application and at the link layer - will share this group's
assumptions about what error rates are reasonable to expect
or provide.  In my experience Internet users are quite diverse
and it's unreasonable to expect users or implementors of a protocol 
to share the same assumptions as the folks who designed the 
protocol.

I'm not saying that the document should contain an analysis -
though if analysis yielded some practical bounds on BER then
it might be sufficient by itself to limit expectations.
I am saying that it should provide users and link implementors
with some idea of what level of service to expect/provide.
"anything goes" is not sufficient to assure interoperability.

> well on the Internet.  You need a good soft rolloff over a very broad

I don't expect a hard roll-off in error rate.  But even voice codecs 
for cellular telephony have some expectations of the underlying medium -
they can't tolerate arbitrary amounts of error.

Keith

---

From: "La Monte Henry Piggy Yarroll" <[email-masked]>
Date: Sun, 12 May 2002 06:42:37 -0500

I see no point in delaying the core UDP-Lite draft to address these
issues.  If you are interested in proposing an applicability document
for UDP-Lite, I'd be happy to provide some editorial input.  Perhaps
we could rope in Phillip Conrad or Mike Remhallo?  Is there a UDP-Lite
author who would be interested in such a document?

> to share the same assumptions as the folks who designed the 

The best reference I've found yet on understanding error behaviours on
the Internet is Vern Paxon's PhD thesis.  The data is a bit old now,
but the core methods are still highly applicable.  The message for
Internet application designers seems to be "Prepare for anything.".
The Internet is very hard to predict.

> "anything goes" is not sufficient to assure interoperability.

"Anything goes" is what really happens.  There isn't anything we can
do about it.  Yet this does not force us out of discussing
interoperability.  Isn't that nice?

> they can't tolerate arbitrary amounts of error.

The phrase "they can't tolerate arbitrary amounts of error" is what I
mean by hard roll-off.  In cellular telephony we have some control
over the layer 2 BER by using power control.  These tools are NOT
available to the Internet application designer.

E.g. one proprietary high compression (1/6th rate) CODEC for CDMA
which I studied behaves perfectly up to 1% BER--the usual target BER
for 2G CDMA systems.  The degradation at 2% is perceptible to most
listeners.  By 4% BER the CODEC is generally below communication
grade.  This roll-off works great in highly managed CDMA networks and
even in end-to-end managed IP networks, but does not work well on the
live Internet.

By contrast, full-rate PCM can suffer over 10 times the damage before
dropping below communication grade.  We were unable to detect any knee
in the PCM data--roll-off is essentially linear.
-- 
La Monte H.P. Yarroll      Home:                [email-masked]
   If you remember nothing else:  [email-masked]         NIC Handle: LY
   GPL - "Just give source a chance."

---

From: Keith Moore <[email-masked]>
Date: Sun, 12 May 2002 10:36:13 -0400

> issues.  

The reason for doing so, of course, is to delay the document until it
can be clarified to the point that applications using UDP Lite - now 
and in the future - are likely to be able to interoperate when their
signal paths employ links that treat UDP Lite differently than other
IP traffic.

I don't think that the document in its current form meets the requirements 
for PS status - it seems to me that there are known problems with the 
document.  If published in its current form I think Experimental would 
be a more appropriate classification.

> The Internet is very hard to predict.

What UDP Lite is currently proposing is a different kind of "anything".
Internet application designers currently have to prepare for any amount
of bandwidth or delay - each can vary by something like 8-9 orders
of magnitude - and for occasional outages.  This is difficult enough 
to do already, and it really limits the kinds of applications we can run.
A lot of effort has been put into specifying intserv and diffserv but 
it doesn't seem like the situation has improved much for most applications.

It's one thing to say to the link layer "make a best effort to deliver 
the packets just like you would if this were an ordinary IP packet, but 
don't drop those for which the link-layer frame integrity check or the UDP 
header checksum fails."  But this document doesn't say that - it allows
the link layer to provide worse than "best effort" service, and that
is a significant departure from widely-understood practice.

"Best effort" can be interpreted very practically as "avoid wasting
bandwidth".  In traditional IP you want to optimize the number of 
packets that make it across the link *without error*, since those
that have errors are going to be thrown away, and wasted bandwidth
is useless.  So for media where the BER is not naturally low, the 
link-layer framing and/or physical-layer can be adjusted to make 
the PER low enough to maximize the effective bandwidth of the link.  

But UDP Lite over IP doesn't seem to have this rule - it allows the 
lower layers to provide a higher error rate for UDP Lite packets.
But there's no guidance as to how much lower.  To some degree lowering
the error rate could be an optimization - especially by avoiding
link-layer retransmits where the retransmission delay is long enough to 
make the packet unusable for UDP Lite's target applications anyway. 
But lowering power or omitting FEC or or omitting link-layer integrity 
checks might not be desirable.

I think the crucial thing that is missing is guidance to the link
layer about what layer of service, if not "best effort", is 
acceptable.  Offhand, I can think of three ways to specify this:

1.  link layers SHOULD make a best effort to deliver the packets 
intact just as if these were ordinary IP packets, however they 
SHOULD NOT don't discard packets for which the link-layer frame 
integrity check or the UDP header checksum fails as long as the 
IP and UDP headers are intact.

2.  pick a number.  say: "in order to be compatible with expected
target applications for UDP Lite, link and physical layers SHOULD 
attempt to provide an average BER of <= X over the payloads
of all UDP Lite packets transmitted during any T1 second period.
Link-layer retransmission SHOULD NOT be employed if on average
such retransmission will add more than T2 seconds delay.  If this 
is not feasible, a link MAY treat UDP Lite packets identically to 
ordinary IP packets except that it SHOULD NOT discard packets for 
which errors are detected and it SHOULD NOT retransmit packets with
errors if it will add more than T2 seconds delay."  There could 
be different error rates for different time periods - e.g. it 
could allow a BER of .1 over a .5 second period but a BER of
only .001 over a 10 second period.

The numbers don't need to have a lot of precision - they are just
a guide.  What approximate maximum BER will allow your codecs to 
perform acceptably? .001? .01? .1? .5?

Of course the application cares about an end-to-end BER, not about
link BER, so the chosen link BER needs to assume some worst-case
number of links that might degrade the signal.

3.  allow the app to specify the maximum desired error rate in the
UDP Lite packet, then the link layer and/or physical layer can 
adjust the signal to attempt to provide this error rate.  

I don't recommend this approach for the current document - I 
think it would take too long to get the details right.
It might be appropriate for future study as an option.

> do about it.  

I disagree.  For ordinary IP, "anything goes" was found to not be 
good enough in practice for some kinds of links.  They had to
implement various mechanisms to provide better service.  It's true
that the level of service expected from links in order to support 
IP was not specified in advance (at least, if it was, I don't 
know where it is specified), but it's also true that by the time
we started making significant use of wireless links for IP that 
we had a lot of expectations about what applications needed - such 
links had to allow most existing applications to work.

For UDP Lite, we have neither existing links nor a large body 
of deployed applications to set expectations.  Consequently each is 
left to guess as to what level of service UDP Lite should/will 
provide.  And unlike the original IP Internet, I don't expect that
UDP Lite will be initially deployed on a small-scale by a few
researchers that are constantly communicating with each other
and who can quickly adjust things based on early experience.

If UDP Lite works as currently specified, it will be an accident.

> in the PCM data--roll-off is essentially linear.

It sounds like you are agreeing with me in either case - for any 
particular codec there is some error rate at which performance
becomes unacceptable.  It might be .04 for one codec and .4 (!?) 
for another but there's always some limit.  

An application designer therefore needs to know what worst-case rate 
to expect.  A codec that can deal with a .4 BER will almost certainly
consume more bandwidth than one which can only deal with .04 - 
this translates to more cost and more delay.    How is the application
designer supposed to choose which codec to use without some guidance?

And if it's difficult to know what will work over the existing Internet,
how much more difficult will it be to know what will work over an
Internet whose links are allowed to degrade the signal's error rate to 
an arbitrary (i.e. unspecified) degree?

Keith

---

From: Allison Mankin <[email-masked]>
Date: Mon, 13 May 2002 09:36:36 -0400

> signal paths employ links that treat UDP Lite differently than other 

Keith,

Please explain what _protocol interopability_ issue you see with
UDP-Lite.  

Allison

---

From: Keith Moore <[email-masked]>
Date: Mon, 13 May 2002 10:09:53 -0400

> you see with UDP-Lite.

I don't see any problem with UDP Lite implementations interoperating
with other UDP Lite implementations, if that's what you mean.
The meanings of the fields in the packet header are clear.

I do see a likely problem with applications being able to use UDP
Lite if link layers are allowed to transmit UDP Lite packets
with worse signal quality (e.g. less power or less FEC or more error-
prone modulation) than is used for ordinary IP packets.    

The potential interoperability problem is between applications and
link layers - since there's no expectation set on the error rate
of UDP Lite, link layers could provide one level of service while 
applications expect something very different.    Depending on conditions,
UDP could be better or worse than UDP Lite even for an application
that would rather have packets with errors.

So I would like to see either an assurance that UDP Lite won't
produce worse service than UDP (in terms of # of useful bits 
transferred over time) - which would make the choice for 
applications simpler, or some expectation of the worst-case
error rate which links that treat UDP Lite specially should provide.

I have a difficult time understanding how a service that is allowed to
degrade the signal to an arbitrary degree - to the point that it is 
delivering random noise and still complies with the spec - can be 
useful to applications.

Keith