Subject: [Tsvwg] Multiple reliable message streams (take 2)

From: Greg Woodhouse <[email-masked]>
Date: Wed, 18 Oct 2006 14:36:40 -0700 (PDT)

I think I originally sent this question to the wrong group, so I apologize in advance for the repetition.
 
Here is my dilemma: I want to be able to deliver messages (of potentially unknown size) reliably. But to complicate things, I have a kind of partial order on message sequence numbers. To make this more concrete, I envision a kind of signaling protocol that can be used to establish "flows" or "streams" that *could* potentially be long-lived and could have unknown periods of inactivitity or could have large amounts of data to send for a sustained period. I know that's a lot of unknowns (sorry). Anyway, an obvious solution would just be to open a TCP connection for each stream and leave it open for the lifetime of the stream. Another solution would be to divide each stream into modest size segments (perhaps a fgraghmernt of a message) and send them independently, using a single TCP or UDP port. It would then fall to the sender to demultiplex the streams and reconstruct the individual messages (using elements of an application level header, of course). I envision a type of SACK
 mechanism being used here. 
 
Now, the question is: Which approach is better for the network? Common sense suggests that relying on a modern TCP implementation to do most of this work would be preferable to trying to reimplement much of what really belongs in a transport protocol (IMO, anyway). But opening large numbers of TCP connections (indeed, a potentially unknown number) doesn't sound like the best of ideas, either, especially when those links can be lightly utilized or even fall idle. Traffic will be bidirectional, but it will also follow the request/reply model of HTTP, suggesting that the Nagle algorithm could be an issue (many packets in flight, no data to send in the reverse direction).
 
In many ways, this situation seems analogous to VoIP, but 1) delivery must be reliable, 2) the data will not be a series of small packets (as with RTP), and 3) precise timing is not an issue.
 
After all of that, I suppose I should ask a question. Are my concerns over using seperate TCP connections for each stream well founded? Is there really a need for a transfer protocol along the lines I sketched above?
 
=== 
Gregory Woodhouse 
 
"Mathematics is the science of patterns."
--Lynn Arthur Steen, 1988

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 19 Oct 2006 09:51:21 +0200

Hi Greg,

it sounds that SCTP might be a transport protocol well suited for  
your application.
Have you looked at it?

Best regards
Michael

On Oct 18, 2006, at 11:36 PM, Greg Woodhouse wrote:

> Here is my dilemma: I want to be able to deliver messages (of  
> potentially unknown size) reliably. But to complicate things, I  
> have a kind of partial order on message sequence numbers. To make  
> this more concrete, I envision a kind of signaling protocol that  
> can be used to establish "flows" or "streams" that *could*  
> potentially be long-lived and could have unknown periods of  
> inactivitity or could have large amounts of data to send for a  
> sustained period. I know that's a lot of unknowns (sorry). Anyway,  
> an obvious solution would just be to open a TCP connection for each  
> stream and leave it open for the lifetime of the stream. Another  
> solution would be to divide each stream into modest size segments  
> (perhaps a fgraghmernt of a message) and send them independently,  
> using a single TCP or UDP port. It would then fall to the sender to  
> demultiplex the streams and reconstruct the individual messages  
> (using elements of an application level header, of course). I  
> envision a type of SACK
> Now, the question is: Which approach is better for the network?  
> Common sense suggests that relying on a modern TCP implementation  
> to do most of this work would be preferable to trying to  
> reimplement much of what really belongs in a transport protocol  
> (IMO, anyway). But opening large numbers of TCP connections  
> (indeed, a potentially unknown number) doesn't sound like the best  
> of ideas, either, especially when those links can be lightly  
> utilized or even fall idle. Traffic will be bidirectional, but it  
> will also follow the request/reply model of HTTP, suggesting that  
> the Nagle algorithm could be an issue (many packets in flight, no  
> data to send in the reverse direction).
>
> In many ways, this situation seems analogous to VoIP, but 1)  
> delivery must be reliable, 2) the data will not be a series of  
> small packets (as with RTP), and 3) precise timing is not an issue.
>
> After all of that, I suppose I should ask a question. Are my  
> concerns over using seperate TCP connections for each stream well  
> founded? Is there really a need for a transfer protocol along the  
> lines I sketched above?
>
> ===
> Gregory Woodhouse
>

---

From: Gregory Woodhouse <[email-masked]>
Date: Thu, 19 Oct 2006 04:19:50 -0700

> it sounds that SCTP might be a transport protocol well suited for  

No, not in any detail. I suppose I've generally heard SCTP presented  
as a signaling transport for multimedia applications, and the  
application domain I have in mind (healthcare) is very different, at  
least on the surface. I didn't go into the rationale behind this  
rather odd sounding set of requirements, perhaps out of concern that  
it would not be considered appropriate here. in brief, many  
applications depend on strict  (global) message ordering, and I'm  
interested in relaxing this requirement, both  to avoid unnecessary  
head of line blocking, and because it doesn't seem particularly TCP- 
friendly.

Gregory Woodhouse
[email-masked]

"If everything seems under control,
you're just not going fast enough."
-- Mario Andretti

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 19 Oct 2006 16:57:11 +0200

Hi Greg,

see my comment in-line.

Best regards
Michael

On Oct 19, 2006, at 1:19 PM, Gregory Woodhouse wrote:

> No, not in any detail. I suppose I've generally heard SCTP  
> presented as a signaling transport for multimedia applications, and  
> the application domain I have in mind (healthcare) is very  
> different, at least on the surface. I didn't go into the rationale  
> behind this rather odd sounding set of requirements, perhaps out of  
> concern that it would not be considered appropriate here. in brief,  
> many applications depend on strict  (global) message ordering, and  
> I'm interested in relaxing this requirement, both  to avoid  
> unnecessary head of line blocking, and because it doesn't seem  
> particularly TCP-friendly.
>
This is exactly the reason why SCTP supports multiple streams within  
each association.
It sounds like SCTP would be a good choice for your application. You  
might want to read
http://www.ietf.org/rfc/rfc3286.txt
to see the basic features of SCTP.