Subject: [Tsvwg] SCTP stream EOF

From: Florian Niederbacher <[email-masked]>
Date: Wed, 17 Sep 2008 15:49:07 +0200

Hi,
I would ask what are you thinking about a definition for an EOF on 
streams in SCTP. My problem is that I would need a signaling method to 
report an EOF without closing the whole SCTP association. If I am right, 
socket connections get only an EOF if the connection will be closed (no 
sending of EOF on socktes in another way is possible). In 
draft-stewart-tsvwg-sctpstrrst-00 the possibility the reset of streams 
is discussed. This would also be a solution if you can check the 
numbering sequence in User Space.
A proposal would be to generate an SCTP_EVENT e.g. SCTP_STREAM_EOF to 
signal EOF on streams. In this way you can stop receiving and also reuse 
the stream or at least continue in multi-streamed network applications, 
if you use different threads or processes for the receiving procedure on 
streams. Maybe an other possibility exists to signal an EOF on SCTP 
streams, but at the moment I can't find any solution to deal with this 
problem. What are your proposals and comments?

Best regards
Florian Niederbacher

---

From: Randy Stewart <[email-masked]>
Date: Wed, 17 Sep 2008 10:40:16 -0400

Florian:

What do you mean by an EOF?

For messages there is a SCTP_EOR to tell you a "message" as ended.

If you were using stream reset, and the peer decides to "reset" a stream
and then use the stream for some other purpose (this is what I think you
are stating).... then the user would get a STREAM_RESET event for that
stream.

I don't see the difference (other than name) in stream-x has been reset
vs stream X receives a EOF. Note that an app MAY choose to use a RESET
as a EOF.. but it can also mean something different to an app.. I am  
done
sending this type of message, I will now send other types of messages.
Which is different than an EOF....

Note also that the socket-api section is still missing (if I remember  
right) from
the stream-reconfig draft... I do need to add this. But I really think
the Stream-Reset serves the purpose you are looking for.

R
On Sep 17, 2008, at 9:49 AM, Florian Niederbacher wrote:

> streams in SCTP. My problem is that I would need a signaling method  
> to report an EOF without closing the whole SCTP association. If I am  
> right, socket connections get only an EOF if the connection will be  
> closed (no sending of EOF on socktes in another way is possible). In  
> draft-stewart-tsvwg-sctpstrrst-00 the possibility the reset of  
> streams is discussed. This would also be a solution if you can check  
> the numbering sequence in User Space.
> A proposal would be to generate an SCTP_EVENT e.g. SCTP_STREAM_EOF  
> to signal EOF on streams. In this way you can stop receiving and  
> also reuse the stream or at least continue in multi-streamed network  
> applications, if you use different threads or processes for the  
> receiving procedure on streams. Maybe an other possibility exists to  
> signal an EOF on SCTP streams, but at the moment I can't find any  
> solution to deal with this problem. What are your proposals and  
> comments?
>
> Best regards
> Florian Niederbacher

-----
Randall Stewart
[email-masked]

---

From: Florian Niederbacher <[email-masked]>
Date: Wed, 17 Sep 2008 16:54:36 +0200

> What do you mean by an EOF?
Same as EOF for files. That blocking recv and recvmsg can be stopped.
> For messages there is a SCTP_EOR to tell you a "message" as ended.
Yes, but only for message you pick up from recv- buffer if I interpret 
it right from socket_api-spec.
E.g. transfer Hello in to messages hel - lo. After 'lo' I get an EOF.
SCTP_EOR does only indicate that the message can be read in full in one 
read/recv, or not? Because i get always SCTP_EOR in my app, only if the 
messagesize is bigger then the buffer for recv\recv_msg.
> as a EOF.. but it can also mean something different to an app.. I am done
Yes I agree. I didn't know that STREAM_RESET is handled as event. This 
would definitely solve my case. But is this already implemented in LKSCTP?

>> signal EOF on streams. In this way you can stop receiving and also 
>> reuse the stream or at least continue in multi-streamed network 

- Regards Florian

---

From: Michael Tüxen <[email-masked]>
Date: Wed, 17 Sep 2008 20:57:20 +0200

> Which is different than an EOF....
Just provide a notification that the stream is reset...
>
>
> Note also that the socket-api section is still missing (if I  
> remember right) from
No we haven't. And that should go into the stream reset ID.
> the Stream-Reset serves the purpose you are looking for.
ACK.
>> to report an EOF without closing the whole SCTP association. If I  
>> am right, socket connections get only an EOF if the connection will  
>> be closed (no sending of EOF on socktes in another way is  
>> possible). In draft-stewart-tsvwg-sctpstrrst-00 the possibility the  
>> reset of streams is discussed. This would also be a solution if you  
>> can check the numbering sequence in User Space.
>> also reuse the stream or at least continue in multi-streamed  
>> network applications, if you use different threads or processes for  
>> the receiving procedure on streams. Maybe an other possibility  
>> exists to signal an EOF on SCTP streams, but at the moment I can't  
>> find any solution to deal with this problem. What are your  
>> proposals and comments?
>>
>> Best regards
>> Florian Niederbacher
>
> -----
> Randall Stewart
> [email-masked]
>
>
>
>
>

---

From: Michael Tüxen <[email-masked]>
Date: Wed, 17 Sep 2008 20:56:14 +0200

> the numbering sequence in User Space.
Using the stream reset ID, you could this. There is currently no  
socket API defined for it
in an ID, but it seems to be a good idea to get a notification on the  
receiver side.
> solution to deal with this problem. What are your proposals and  
As said above, sctp reset with a notification should do it...

---

From: Randy Stewart <[email-masked]>
Date: Fri, 19 Sep 2008 13:38:39 -0400

On Sep 17, 2008, at 10:54 AM, Florian Niederbacher wrote:

> Yes, but only for message you pick up from recv- buffer if I  
> interpret it right from socket_api-spec.
> SCTP_EOR does only indicate that the message can be read in full in  
> one read/recv, or not? Because i get always SCTP_EOR in my app, only  
> if the messagesize is bigger then the buffer for recv\recv_msg.

If I have a recv buffer of 2 bytes and you send me "hello\n"
I would get
read (he - no eor)
read (ll - no eor)
read (o\n - EOR)

It depends on your read size...

> Yes I agree. I didn't know that STREAM_RESET is handled as event.  
> This would definitely solve my case. But is this already implemented  
> in LKSCTP?
>

We have it in FreeBSD but we as yet do not support adding more  
streams.. another
thing I need to add :)

I don't know about LKSCTP at the last interop they did not support it

R

>>> streams in SCTP. My problem is that I would need a signaling  
>>> method to report an EOF without closing the whole SCTP  
>>> association. If I am right, socket connections get only an EOF if  
>>> the connection will be closed (no sending of EOF on socktes in  
>>> another way is possible). In draft-stewart-tsvwg-sctpstrrst-00 the  
>>> possibility the reset of streams is discussed. This would also be  
>>> a solution if you can check the numbering sequence in User Space.
>>> network applications, if you use different threads or processes  
>>> for the receiving procedure on streams. Maybe an other possibility  

-----
Randall Stewart
[email-masked]

---

From: Florian Niederbacher <[email-masked]>
Date: Fri, 19 Sep 2008 20:35:50 +0200

> It depends on your read size...

Yes, and it works in the following way if I use SCTP_EXPLICIT_EOR with a 
buffer greater then sizeof("hello\n"), agree?

setsocketopt(SCTP_EXPLICIT_EOR)

send (he - no eor)  -> read (he - no eor)
send (ll - no eor) ->  read(ll - no eor)
send (o\n - no eor) -> read (o\n - no eor)
send(EOR)   -> read(EOR)

But in this way no transfer of other fragmented streams on the same 
association is possible, right? Therefore an application causes nearly a 
similar HOL in SCTP, as TCP transmission does if more files(file1, 
file2,..) over one TCP connection are send. It could be very problematic 
if the records are really big.
My scope was using the EOR for signaling the end of a data file 
transmission. But with multiple streams this results up in blocking 
other streams, because only one stream can have a fragmentation at time 
(if I interpret it right from SCTP RFC).
The solution would be to use your new proposal with STREAM_RESET. It 
does exactly what I would need. But also this solution has some 
disadvantages. If an application needs a transfer of many files on 
multiple streams and STREAM_RESET is used to signal a "READ_STOP" or 
lets say an EOF, it can end up in many STREAM_RESET chunks on the net. 
This causes a lot of traffic and slows down SCTP transfers. The 
conclusion is that STREAM_RESET can be used but it should be avoided in 
this case, because it can be handled in a better way.

That what I have in mind would be to use and define an additional new 
flag from the SCTP_DATA chunk. At the moment only UBE flags are 
used(fortunately some space left). If an additional flag would be 
introduced this could signal an EOF or "READ_STOP". It would help to 
reduce the excessive sending of STREAM_RESET chunks and doesn't blow up 
network traffic. I guess a lot of application in future would benefit 
from the possibility to use an EOF or EOT(end of transfer) on streams. 
TCP generates EOF only by closing the connection (close socket). This is 
the EOF or EOT signal to stop blocking reads. SCTP cannot do it in the 
same way if an application uses multistreaming. MSG_EOF and MSG_ABORT 
from SCTP can also not be used, because it results in starting up the 
initiation of the SHUTDOWN procedure.

These would be some the arguments for defining an additional flag in SCTP.

What are your thoughts and comments?

>>>> find any solution to deal with this problem. What are your 
Regards
Florian

---

From: Michael Tüxen <[email-masked]>
Date: Fri, 19 Sep 2008 21:21:32 +0200

On Sep 19, 2008, at 8:35 PM, Florian Niederbacher wrote:

>>> SCTP_EOR does only indicate that the message can be read in full  
>>> in one read/recv, or not? Because i get always SCTP_EOR in my app,  
>>> only if the messagesize is bigger then the buffer for recv\recv_msg.
>>
> Yes, and it works in the following way if I use SCTP_EXPLICIT_EOR  
> with a buffer greater then sizeof("hello\n"), agree?
>
You can not just send an EOR... An this requires that the recv buffer  
size is 2 byte.
So you must send the n\n with EOR.
> association is possible, right? Therefore an
Right.
> application causes nearly a similar HOL in SCTP, as TCP transmission  
> does if more files(file1, file2,..) over one TCP connection are  
> send. It could be very problematic if the records are really big.
SCTP is not optimized for large messages... It is originally designed  
for small messages.
> other streams, because only one stream can have a fragmentation at  
> time (if I interpret it right from SCTP RFC).
Correct.
Why are you not signalling the end of transmission is the application  
layer protocol?
> lets say an EOF, it can end up in many STREAM_RESET chunks on the  
> net. This causes a lot of traffic and slows down SCTP transfers. The  
> conclusion is that STREAM_RESET can be used but it should be avoided  
> in this case, because it can be handled in a better way.
If you want to transfer "multiple files", Why not signal the end of a  
file in the application layer.

For example you could send a message indicating the start of a  
transfer, then the is several messages
the file and finally an application layer message indicating the end  
of it. You could use, for example,
the PPID to distinguish between them or have a length indication in  
the start message or so.
Using "one message per file" is theoretically possible, but not very  
efficient...
>
>
> That what I have in mind would be to use and define an additional  
> new flag from the SCTP_DATA chunk. At the moment only UBE flags are  
> reduce the excessive sending of STREAM_RESET chunks and doesn't blow  
> up network traffic. I guess a lot of application in future would  
> benefit from the possibility to use an EOF or EOT(end of transfer)  
> on streams. TCP generates EOF only by closing the connection (close  
> socket). This is the EOF or EOT signal to stop blocking reads. SCTP  
> cannot do it in the same way if an application uses multistreaming.  
> MSG_EOF and MSG_ABORT from SCTP can also not be used, because it  
> results in starting up the initiation of the SHUTDOWN procedure.
Hmmm. I think your problem is that you are using a message for big  
things. You might rethink about this.

Please note that a STREAM RESET is not a READ_STOP which you can  
continue later, it is the end of
the file. So you can still not mix sending several files in parallel.

So signalling file boundaries by message boundaries should be avoided.
>>> This would definitely solve my case. But is this already  
>>> implemented in LKSCTP?
>>>
>>
>>
>>>>> I would ask what are you thinking about a definition for an EOF  
>>>>> on streams in SCTP. My problem is that I would need a signaling  
>>>>> association. If I am right, socket connections get only an EOF  
>>>>> if the connection will be closed (no sending of EOF on socktes  
>>>>> in another way is possible). In draft-stewart-tsvwg- 
>>>>> sctpstrrst-00 the possibility the reset of streams is discussed.  
>>>>> This would also be a solution if you can check the numbering  
>>>>> sequence in User Space.
>>>>> A proposal would be to generate an SCTP_EVENT e.g.  
>>>>> SCTP_STREAM_EOF to signal EOF on streams. In this way you can  
>>>>> stop receiving and also reuse the stream or at least continue in  
>>>>> multi-streamed network applications, if you use different  
>>>>> threads or processes for the receiving procedure on streams.  
>>>>> Maybe an other possibility exists to signal an EOF on SCTP  
>>>>> streams, but at the moment I can't find any solution to deal  
>>>>> with this problem. What are your proposals and comments?
>>>>>
>>>>> Best regards
>>>>> Florian Niederbacher
>>>>
>>>> -----
>>>> Randall Stewart
>>>> [email-masked]
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>> - Regards Florian
>>>
>>
>> -----
>> Randall Stewart
>> [email-masked]
>>
>>
>>
>>
>>
> Regards
> Florian
>