Subject: [Tsvwg] SCTP socket writeability on one-to-many sockets

From: Mark Butler <[email-masked]>
Date: Fri, 03 Mar 2006 00:47:16 -0700

Hello everyone,

Several have noticed that socket writeability for one to many sockets 
(as returned via poll(2) or select(2)) is a very tenuous concept.  If 
you have association level buffering,  the socket may be "writeable", 
but a non-blocking send on a specific association will fail because the 
association buffer is full.  The alternative, socket level buffering, 
has a worse problem - the shared buffer may fill up with large writes on 
slow associations, blocking writes on otherwise idle associations 
indefinitely.

One way of solving the problem would be to create a new type of 
notification event - one that is delivered when write of a specified 
size will no longer block on an association.  Then a non-blocking server 
could operate in typical request / response fashion, and when any send 
operation needs to return with error EWOULDBLOCK or EAGAIN,  the stack 
would schedule a notification event that is delivered when space becomes 
available such that the operation would succeed without blocking.

Then the server application would hold the response in a queue for 
delivery when the notification event arrives, which would contain 
information so that delayed messages could be easily identified by a 
combination of association id, stream, ppid, and context id.  Something 
like this:

struct sctp_send_ready {
    uint16_t ssr_type;
    uint16_t ssr_flags;
    uint32_t ssr_length;
    uint32_t ssr_ppid;
    uint32_t ssr_context;
    uint64_t ssr_size;
    uint64_t ssr_offset;
    sctp_assoc_t ssr_assoc_id;
    uint16_t ssr_stream;
    uint16_t ssr_reserved;
};

A send ready notification literally means "now ready to accept a 
non-blocking write for size bytes on a specified association (and 
stream)". However, the buffer space may be used up by the time the 
application processes the notification, so the process may have to repeat.

The other fields allow the notification to be interpreted as "now ready 
to send size bytes of the message identified by (ppid,context,...) 
[starting at offset]."   The offset field is for future use with partial 
message writes, something necessary to send large messages with a 
non-blocking server.  Normally ssr_offset would always be zero and 
ssr_size the length of the original write.

One convention that should be implemented is non-blocking notification 
reads with MSG_OOB, so that notifications can be processed ahead of 
other normal incoming messages. SO_OOBINLINE would have to be on by 
default for compatibility. 

Comments?

  - Mark

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg