Subject: RE: Reliability in LAN

From: "Manfredi, Albert E" <[email-masked]>
Date: Thu, 17 Apr 2003 17:06:48 -0400

I agree. My impression, perhaps wrongly, is that it was being suggested here that RFCs 3450-3453 somehow invalidate what was described in RFCs 2357, 2887, and 3048. I don't think they do.

Let's look at DATA transfers only. For data transfers, I don't agree that a FEC, with the possible addition of data carousel or layered coding, is always a viable option.

One of the applications of data carousel is the broadcasting of data packets. Without a return path, you have to make do with repetition of the data, until all receivers will have a high likelihood of having received the complete data download correctly. This works great in a broadcast, works well but is somewhat  wasteful in multicast, and becomes onerous and wasteful in unicast. Data carousel has to be designed for worst case scenarios. It loads the network down more than a TCP/IP (unicast) session would, for the same level of reliability. Packets have to be repeated enough times to compensate for the worst case corruption levels, must be repeated to compensate for packet loss due to congestion, and these repeated UDP packets in turn create congestion in the network, because they are non responsive.

So in a unicast, these techniques end up requiring more bandwidth for similar robustness as TCP/IP. But in broadcast, congestion is a non-issue. The network is sized according to the needs of the source. Besides, with no return path, it's the only game in town.

RFC 3453 in fact does refer back to RFC 3048 techniques. As shwon here:

1.1.  Application of FEC codes

   For some reliable IP multicast protocols, FEC codes are used in
   conjunction with ARQ to provide reliability.  For example, a large
   object could be partitioned into a number of source blocks consisting
   of a small number of source symbols each, and in a first round of
   transmission all of the source symbols for all the source blocks
   could be transmitted.  Then, receivers could report back to the
   sender the number of source symbols they are missing from each source
   block. 

This technique requires a resturn path, which is available in IP Multicast but not in broadcast, and this approach can be far more network friendly than data carousel.

I can also see another problem here, which is my use of the word "reliability." In RFC 3453, for example, "reliable" means "error free." As I used it, "more reliable" or "add reliability" are synonymous. They do _not_ mean "make it error free." Going back through the thread, I can see that created a problem.

Bert

> -----Original Message-----
> From: Tom Pusateri [mailto:[email-masked]]
> Sent: Thursday, April 17, 2003 4:03 PM
> To: Manfredi, Albert E
> Cc: Lloyd Wood; [email-masked]; [email-masked]
> Subject: Re: Reliability in LAN 
> 
> 
> I think Mike's point is being missed though. The algorithms he
> cites from RMT are more than simple FEC. They do provide error free
> delivery over time. By simply continuting to listen to the stream,
> the original data can be reproduced exactly everytime. You'll need
> to read the papers to see how this works.
> 
> Tom
> 
> In message 
> <[email-masked]>
>  you write:
> >Guys,
> >
> >I think we are digressing into meaningless semantic 
> nit-picking. Adding "relia
> >bility" simply means that information is correctly received 
> MORE OFTEN. That c
> >ertainly fits the mission statement of any form of forward 
> error correction. T
> >he FEC can compensate for some degree of packet corruption, 
> so without having 
> >to add any protocol complexity, the data packets are 
> delivered error free more
> > often. Not always, just more often.
> >
> >Being error-free means that the information is "always 
> perfect," when received
> > by the application. If it's screwed up, one of the ISO 
> layers below the appli
> >cation layer fixes the problem.
> >
> >If the transfer occurs only once in a blue moon, then error 
> free is a must, or
> > you might simply miss the data packet entirely. If the 
> information is transfe
> >rred continuously and an occasional corrupted packet can be 
> ignored, then FEC 
> >alone with UDP/IP is acceptable.
> >
> >I originally used streaming media as AN EXAMPLE of this type 
> of transfer. Data
> > can also fit this model, depending how the system is designed.
> >
> >Thus, FEC certainly DOES increase reliability. It just 
> doesn't guarantee error
> >-free reception of every packet.
> >
> >Bert
> >
> >
> >> -----Original Message-----
> >> From: Michael Luby [mailto:[email-masked]]
> >> Sent: Thursday, April 17, 2003 12:45 AM
> >> To: Lloyd Wood; Manfredi, Albert E
> >> Cc: Karthikeyan Subramaniam; [email-masked]
> >> Subject: RE: Reliability in LAN
> >> 
> >> 
> >> > -----Original Message-----
> >> > From: Lloyd Wood [mailto:[email-masked]]
> >> > Sent: Wednesday, April 16, 2003 6:32 AM
> >> > To: Manfredi, Albert E
> >> > Cc: Michael Luby; Karthikeyan Subramaniam; 
> >> [email-masked]
> >> > Subject: RE: Reliability in LAN
> >> >
> >> >
> >> > On Thu, 10 Apr 2003, Manfredi, Albert E wrote:
> >> >
> >> > > True, but these all rely on forward error correction. FEC 
> >> is a good
> >> > > way to add reliability, but it doesn't create an "error-free"
> >> > > communications link, at least as "error-free" has been 
> defined in
> >> > > the past. To be "error-free," you need some way of 
> re-delivering
> >> > > packets which were not correctly received or not 
> received at all.
> >> >
> >> > Thus, FEC does NOT add reliability.
> >> 
> >> I'd be curious to understand how your definition of 
> >> reliability differs so
> >> drastically from what is defined in the RMT working group 
> >> charter and the
> >> general working group RFCs, and in particular from what is in 
> >> RFC 3453 and
> >> RFC 3452 which show how FEC can provide reliability for data 
> >> delivery.  If
> >> you get a chance, it would be interesting if you read those 
> >> 3-4 RFCs so that
> >> you can clearly differentiate what you have in your head in 
> >> terms of what
> >> reliability means versus what is in these docs.
> >> 
> >> >
> >> > Perhaps it adds robustness...
> >> >
> >> > L.
_______________________________________________
routing-discussion mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/routing-discussion

---

From: "Michael Luby" <[email-masked]>
Date: Thu, 17 Apr 2003 21:04:13 -0700

It still seems that you are missing the point, and perhaps you didn't read
the FEC RFCs in totality, did you read the section in RFC 3453 on Large
Expandable FEC codes?  If you did, you will note that from a piece of data
you can produce an unlimited number of unique FEC packets (and it is
practical), such that any receiver only needs to receive the length of the
original data in FEC packets to reproduce an exact copy of the original
data.  Thus, there is no carousel, no repeating transmission of already
transmitted packets, no repetition of packets, and each receive need to
receive the minimal number of packets possible to reconstruct the data.  If
you understand this concept, then you will see that what you assume to have
to be true in your description is not true for Large Expandable FEC codes.

> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Manfredi, Albert
> E
> Sent: Thursday, April 17, 2003 2:07 PM
> To: Tom Pusateri
> Cc: [email-masked]
> I agree. My impression, perhaps wrongly, is that it was being
> suggested here that RFCs 3450-3453 somehow invalidate what was
> described in RFCs 2357, 2887, and 3048. I don't think they do.
>
> Let's look at DATA transfers only. For data transfers, I don't
> agree that a FEC, with the possible addition of data carousel or
> layered coding, is always a viable option.
>
> One of the applications of data carousel is the broadcasting of
> data packets. Without a return path, you have to make do with
> repetition of the data, until all receivers will have a high
> likelihood of having received the complete data download
> correctly. This works great in a broadcast, works well but is
> somewhat  wasteful in multicast, and becomes onerous and wasteful
> in unicast. Data carousel has to be designed for worst case
> scenarios. It loads the network down more than a TCP/IP (unicast)
> session would, for the same level of reliability. Packets have to
> be repeated enough times to compensate for the worst case
> corruption levels, must be repeated to compensate for packet loss
> due to congestion, and these repeated UDP packets in turn create
> congestion in the network, because they are non responsive.
>
> So in a unicast, these techniques end up requiring more bandwidth
> for similar robustness as TCP/IP. But in broadcast, congestion is
> a non-issue. The network is sized according to the needs of the
> source. Besides, with no return path, it's the only game in town.
>
> This technique requires a resturn path, which is available in IP
> Multicast but not in broadcast, and this approach can be far more
> network friendly than data carousel.
>
> I can also see another problem here, which is my use of the word
> "reliability." In RFC 3453, for example, "reliable" means "error
> free." As I used it, "more reliable" or "add reliability" are
> synonymous. They do _not_ mean "make it error free." Going back
> through the thread, I can see that created a problem.
>
> Bert
>
>

---

From: "Michael Luby" <[email-masked]>
Date: Thu, 17 Apr 2003 21:04:13 -0700

---

From: Lloyd Wood <[email-masked]>
Date: Fri, 25 Apr 2003 15:22:26 +0100 (BST)

On Thu, 17 Apr 2003, Michael Luby wrote:

> the FEC RFCs in totality,

Perhaps he simply didn't believe them. (I know I don't.)

> did you read the section in RFC 3453 on Large Expandable FEC codes?

RFC 3453 is informational. Yet you seem to be citing your work as
gospel.

> If you did, you will note that from a piece of data
> data.

Uniqueness of FEC packets is a red herring. You're paying far too much
attention to coding and not enough attention to delivery; you don't
know (and can't bound) when the receiver has reconstructed the copy.

blah blah packet loss blah. Thus my ass.

L.

and what does this have to do with routing, anyway?

> Thus, there is no carousel, no repeating transmission of already

<http://www.ee.surrey.ac.uk/Personal/L.Wood/><[email-masked]>

---

From: Lloyd Wood <[email-masked]>
Date: Fri, 25 Apr 2003 15:22:26 +0100 (BST)

L.