Subject: [tsvwg] SCTP flow control per stream

From: Florian Niederbacher <[email-masked]>
Date: Wed, 03 Feb 2010 15:30:02 +0100

Hello,
I have a question concerning the SCTP flow control per stream. Following 
up discussions on different mailing lists (IETF and SCTP developers) I 
see that this topic was already extensively discussed in the IETF and 
also rejected. My problem in this context is that I can't figure out the 
main reasons for the rejection (standard modification, deadlock 
situations, ...). Therefore I would ask the members of this mailing list 
if someone could help me to understand the main IETF argumentation 
points for it. SS7 network developers as well network applications for 
parallel data transfers (as usual nowadays) would demand this 
functionality, but probably I am missing some important backgrounds. 
Thanks for your comments in advance.

Regards
Florian Niederbacher

---

From: Caitlin Bestler <[email-masked]>
Date: Wed, 3 Feb 2010 20:46:22 -0800

On Feb 3, 2010, at 6:30 AM, Florian Niederbacher wrote:

> Hello,
> I have a question concerning the SCTP flow control per stream. Following up discussions on different mailing lists (IETF and SCTP developers) I see that this topic was already extensively discussed in the IETF and also rejected. My problem in this context is that I can't figure out the main reasons for the rejection (standard modification, deadlock situations, ...). Therefore I would ask the members of this mailing list if someone could help me to understand the main IETF argumentation points for it. SS7 network developers as well network applications for parallel data transfers (as usual nowadays) would demand this functionality, but probably I am missing some important backgrounds. Thanks for your comments in advance.
> 
> Regards
> Florian Niederbacher

If flow control were done per stream then what would the Association be?

The point of multi-streaming is to eliminate the redundant flow control.

--
Caitlin Bestler
[email-masked]
http://www.asomi.com/CaitlinBestlerResume.html

---

From: Michael Tüxen <[email-masked]>
Date: Thu, 4 Feb 2010 09:25:05 +0100

> I have a question concerning the SCTP flow control per stream. Following up discussions on different mailing lists (IETF and SCTP developers) I see that this topic was already extensively discussed…
Hi Florian,

it you want to do flow control per stream, you need to extend the SACK chunks to be able to
carry a window for each stream (at least conceptually). You need to modify the zero window
probing appropriately and think about the rwnd of the whole association. This is not impossible
to do, but requires substantial protocol changes. Also what would be the stream receive buffer?
If it is a socket buffer, then you would use one socket buffer per stream. This also has substantial
impacts on the socket layer. So it is a pretty complex change?

What to you want to accomplish? If the receiver prefers one stream at a time and another later,
you can use a per stream control flow control. But you can get the same by using some stream
scheduling. This does not require substantial protocol changes. Robin is working on this stuff
currently. Or do you have another usage scenario in mind?

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 4 Feb 2010 03:21:40 -0700

Florian,

What makes you think SS7 network developers demand this
functionality?  All of the SIGTRAN protocols were designed
without it and I can't really see a use for it even there.

Intra-stream flow control can easily be accomplished with
applications protocols where necessary.  Inter-stream flow
control (managing interferency between shared streams using the
same send and receive buffers) can most easily accomplished by
prioritizing transmissions at the sender and simply clearing
receive buffers to application buffers promptly at the receiver
(they are really not so big nowadays).

--brian

Florian Niederbacher wrote:                 (Wed, 03 Feb 2010 15:30:02)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Florian Niederbacher <[email-masked]>
Date: Thu, 04 Feb 2010 11:41:56 +0100

> The point of multi-streaming is to eliminate the redundant flow control.
My opinion is that a redundant flow control is probably less important as a redundant congestion control. The point of multi-streaming in my eyes is the usage for parallel transfers with a single association. Thus the congestion control is shared by all data transfers on the same end-to-end path which should be shared by all applications, but using different streams. The intention is to use a single association instead of parallel connections/associations. The results is an improved data transmission behaviour, as intended in a similar (but more complex) manner by the congestion manager specified in RFC 3124.

> impacts on the socket layer. So it is a pretty complex change?
The described concept seems to me to be similar as the drafts which were 
presented long time ago to the IETF and a specification is also 
mentioned in the book of Gerhard Rufa: Developments in 
Telecommunications - With a Focus on SS7 Network Reliability.

I agree that such a modification has a big impact on the socket layer as 
well on the SCTP protocol specification. The intention of my question 
was simply to figure out if the idea to modify SCTP for a stream-based 
flow control was completely closed and if yes, what was the main 
argumentation. I was not sure about due to the various statements on the 
mailing lists.

> What to you want to accomplish? 
The idea is using a single end-to-end association shared from all 
services on both endpoints for doing data transmissions in parallel. 
Instead of using several TCP connections or SCTP associations a single 
SCTP association should be used in combination with multi-streaming. I 
think not only standard applications (e.g HTTP, FTP, ...) demand a new 
way for doing parallel data transmissions but grid\cloud services as 
well and the SCTP multi-streaming feature seems to be well designed.

> If the receiver prefers one stream at a time and another later,
The scenario is: using all streams at the same time, doing transfers in 
parallel (processes/threads) and the services are identified by streams 
(stream number - not changeable at runtime).

I think another solution for solving the flow control problem with 
streams could be a modified buffer implementation (again socket layer 
modification) and access as originally intended and illustrated in the 
SCTP reference guide (Figure on p. 130) each stream separately. An 
alternative to use the complex window calculation, could be using a 
simple Stop-and-Go algorithm (similar to the serial interface) for each 
stream. The stream buffers have to grow dynamically and if they exceed a 
threshold a message is sent back to signal an overload and stop the send 
calls for this stream. Again a signal is sent back after the buffered 
values reach a lower limit and sending on this stream is allowed.

I know this can be realized all in user space, but a performance loss is 
included thereby. I guess this is a protocol activity and should 
therefore find its place in the network stack (kernel space). However, I 
don't want to propose some special algorithms or changes. My intention 
was only to clarify pros and cons of the IETF members regarding a flow 
control per stream.    

> without it and I can't really see a use for it even there.
The comments on the mailing lists and the mentioned book above (Developments in Telecommunications). 

> (they are really not so big nowadays).
I agree, that its possible to manage this things on application layer (unfortunately with performance losses). But I guess this is a protocol property and should therefore be defined as an extension of the SCTP protocol. However, the main question was the reasoning of the IETF members about a stream-based flow control. It is simply to complex and not really demanded?

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 4 Feb 2010 05:12:37 -0700

Florian,

> (Developments in Telecommunications). 

Consider that SS7 itself does not provide flow control on specific
traffic flows at any level except SCCP protocol class 3 (which is
not used anywhere).  SS7 congestion control mechanisms work on
aggregated flows and use stoichastic procedures to throttle noisy
senders.  For SIGTRAN protocols, these mechanisms interwork at the
application level and cannot use per-stream mechanisms in SCTP
because there is simply no equivalent mechanism within SS7 to which
to interwork.

I am sorry, but I do not have access to this book.  Perhaps you could
quote the specific remarks so that they can be challenged openly.
I certainly do not see a need for any SS7 (SIGTRAN) applications.

> I agree, that its possible to manage this things on application layer 
> (unfortunately with performance losses). But I guess this is a protocol 
> property and should therefore be defined as an extension of the SCTP 
> protocol. However, the main question was the reasoning of the IETF members 
> about a stream-based flow control. It is simply to complex and not really 
> demanded?

What performance loss?

Also consider that partial-reliability has already addressed the
requirements of many applications that may have otherwise used
per-stream flow controls.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Florian Niederbacher <[email-masked]>
Date: Thu, 04 Feb 2010 18:30:03 +0100

> I certainly do not see a need for any SS7 (SIGTRAN) applications.
The essential part is also available on google books: search for - 
Gerhard Rufa - Developments in Telecommunications: With a Focus on SS7 
Network Reliability. Section 9.5 SS7 over IP and flow control. I must 
admit that I am not really familiar with SS7 protocol and the demand of 
a stream based flow control is a personal interpretation (maybe also 
wrong) of the book content and by the (very old) postings on mailing list.
>> about a stream-based flow control. It is simply to complex and not really 
Implementations work generally faster (assumed Linux/Unix) in kernel 
space as in user space, because of the expensive context switches. I 
have a user space implementation in mind (based on the Figure of the 
SCTP reference guide on page 130 ), where stream reordering queues are 
accessible by the user application. This is not yet possible with the 
drafted SCTP socket API and a stream queue has to be implemented for 
each used stream additionally, to avoid a blocking of other stream 
traffic. This implementation style is necessary for parallel data 
transfers, where a a single SCTP association is used shared and each 
thread/process uses a different stream for the transmission. Furthermore 
the reordering queues must grow dynamically, otherwise they can be 
filled up completely and again the transfer is stuck.

The idea is reading in continue the data from socket buffer (single SCTP 
receive buffer for all streams), writing in a corresponding stream 
buffer  (new buffer for each stream) then the working thread/process 
(scheduling dependent) reads from this new stream buffer and dispatches 
finally the data to the user application. This implies many context 
switches and thus performance losses, caused by the many data I/O 
operations. In such a case the usage of SCTP is problematic, because one 
improves the data transfer by the protocol usage, but unfortunately time 
is wasted by many memory copy operations.

Of course is this an alternative solution (and surely not the only one), 
but as I have read about stream based flow control I had the thought 
that this would solve also the problem (in an easier manner), but I was 
not sure if a stream based flow control will ever be specified and 
implemented. Therefore my question, to know in which direction I should 
work on. I will not say that a flow control per stream in SCTP is 
absolutely necessary, but there exists at least some useful cases.