Subject: [Tsvwg] I-D ACTION:draft-ietf-tsvwg-sctpsocket-11.txt

From: [email-masked]
Date: Mon, 12 Sep 2005 15:50:03 -0400

A New Internet-Draft is available from the on-line Internet-Drafts directories.
This draft is a work item of the Transport Area Working Group Working Group of the IETF.

	Title		: Sockets API Extensions for Stream Control 
                          Transmission Protocol (SCTP)
	Author(s)	: R. Stewart, et al.
	Filename	: draft-ietf-tsvwg-sctpsocket-11.txt
	Pages		: 90
	Date		: 2005-9-12
	
This document describes a mapping of the Stream Control Transmission
   Protocol SCTP RFC2960 [8] into a sockets API.  The benefits of this
   mapping include compatibility for TCP applications, access to new
   SCTP features and a consolidated error and event notification scheme.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-sctpsocket-11.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Internet-Drafts are also available by anonymous FTP. Login with the username
"anonymous" and a password of your e-mail address. After logging in,
type "cd internet-drafts" and then
	"get draft-ietf-tsvwg-sctpsocket-11.txt".

A list of Internet-Drafts directories can be found in
http://www.ietf.org/shadow.html 
or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

Internet-Drafts can also be obtained by e-mail.

Send a message to:
	[email-masked].
In the body type:
	"FILE /internet-drafts/draft-ietf-tsvwg-sctpsocket-11.txt".
	
NOTE:	The mail server at ietf.org can return the document in
	MIME-encoded form by using the "mpack" utility.  To use this
	feature, insert the command "ENCODING mime" before the "FILE"
	command.  To decode the response(s), you will need "munpack" or
	a MIME-compliant mail reader.  Different MIME-compliant mail readers
	exhibit different behavior, especially when dealing with
	"multipart" MIME messages (i.e. documents which have been split
	up into multiple messages), so check your local documentation on
	how to manipulate these messages.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.
_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Vlad Yasevich <[email-masked]>
Date: Tue, 13 Sep 2005 12:04:14 -0400

Hi

Draft 11 limits the SO_LINGER socket option to one-to-one style
socket:

>    An application using the one-to-one style socket can use this option
>    to perform the SCTP ABORT primitive.  The linger option structure is:

Can we remove this limitation so that this option can be used on
one-to-many sockets.  (I think the implementers for the sctp-impl list
agreed to this).

Also, there is currently no way to specify TOS on a per-association basis.

I would propose to define SCTP_TOS option as an SCTP level option.

7.1.25. SCTP_TOS

  Applications can use this socket option to examine and set the IPv4
  type of service (similar to IP_TOS) on a per association basis.  The
  sctp TOS structure is:

  struct sctp_tos {
	sctp_assoc_t stos_assoc_id;
	uint8_t	     stos_tos;
  };

    stos_assoc_id: (one-to-many style socket) When this is filled
		   in by the application, it identifies the association
		   for this query.  If this parameter is 0, the call is
		   considered invalid since the	application should
		   reallybe using the IP_TOS IP-level option.

    stos_tos: contains the TOS value to set.  This value is the same
	      that would be used with the IP_TOS IP-level option.

  When an applications uses this option to specify type of service, only
  the given association will use this value.  Other associations,
  belonging to the same socket, would use the value set with the IP_TOS
  IP-level option.  This option only affects the IPv4 Type of Service
  field.  To set the IPv6 traffic class, the application should use
  interfaces documented in [RFC 3542].

-vlad

---

From: Randall Stewart <[email-masked]>
Date: Tue, 13 Sep 2005 13:05:41 -0400

Vlad:

Thats a good catch.. and it should not
be there :-D

As to the TOS option.. looks good to me..

I will copy your text into draft-12 if I hear
no strong objections :-D

R

-- 
Randall Stewart
803-345-0369 <or> 815-342-5222(cell)

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 14 Sep 2005 09:43:39 +0200

Vlad, Randy,

I support both.

Best regards
Michael

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 14 Sep 2005 17:13:12 +0800

> I would propose to define SCTP_TOS option as an SCTP level option.

Using an SCTP level socket option to change a property in
the IP header?  I don't think we should mix up the different
name spaces.

-- 

						K. Poon.
						[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 14 Sep 2005 11:43:23 +0200

How can you then have one assoc using TOS1 and another one TOS2?

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 14 Sep 2005 04:13:58 -0600

Michael,

> How can you then have one assoc using TOS1 and another one TOS2?

	int tos = 0;

	setsockopt(s, SOL_IP, IP_TOS, &tos, sizeof(tos));

I think that Kacheong's point was that there is already a IP_TOS
in SOL_IP (this is what is used for TCP on some imps, by the
way), so there is no need for a SOL_SCTP, SCTP_TOS option.  SCTP
(as does TCP) can merely interpret the IP_TOS setting from
SOL_IP.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 14 Sep 2005 12:46:27 +0200

Brian,

your code works for a 1-1 socket, how does it work for 1-M?

Best regards
Michael

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 14 Sep 2005 05:36:37 -0600

Michael,

Works for UDP too. ;)

--brian

---

From: Randall Stewart <[email-masked]>
Date: Wed, 14 Sep 2005 08:08:03 -0400

Brian:

This is true.. but I think Michaels point is

How does one set multiple TOS's on different
associations that are all under a single
one-2-many socket..

This is the same issue, I think, that you face
with a UDP sender having two converstations... as
I mentioned to Kacheong in private...

If you do

Then

sctp_send(sd, data, len, sinfo_assoc_1, flags);
sctp_send(sd, data, len, sinfo_assoc_2, flags);

You get the value in tos sent on both association
since the tos field is at the socket level.. not
the association level..

The only way to use SOL_IP level item would be to do

setsockopt(s, SOL_IP, IP_TOS, &tos1, sizeof(tos));
sctp_send(sd, data, len, sinfo_assoc_1, flags);
setsockopt(s, SOL_IP, IP_TOS, &tos2, sizeof(tos));
sctp_send(sd, data, len, sinfo_assoc_2, flags);

Which is probably what the UDP world dos now.. but is
horribly in-efficent...

R

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 14 Sep 2005 14:08:05 +0200

Can you show me the code you would use for one 1-M socket fd
having two associations with associd1 and associd2 and having
TOS1 for assoc1 and TOS2 for assoc2?

Best regards
Michael

On Sep 14, 2005, at 13:36 Uhr, Brian F. G. Bidulock wrote:

>>>>>> Also, there is currently no way to specify TOS on a 
>>>>>> per-association
>>>>>> basis.

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 14 Sep 2005 11:03:50 -0400

> associations that are all under a single

Yep, that's exactly what one of our apps is trying to
do.

> Which is probably what the UDP world dos now.. but is

An alternative would be to define an ancillary data item
that an application can pass down with each send call.  That
may not be as bad, and would be somewhat equivalent to what IPv6
enabled apps would need to do.

That problem then becomes where to define this, since it might be
appropriate to make it an IP-level ancillary data :(

Another alternative would be to add a member to the sctp_assocparams
structure that would define the TOS level for a given association.
We don't need another socket option, but we would still have a slight
layer violation which I guess is what people are objecting to.

Are any of the alternatives more preferable?

Thanks
-vlad

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 14 Sep 2005 11:33:48 -0600

Vlad,

> appropriate to make it an IP-level ancillary data :(

IP_TOS is per-packet and can be used as ancilliary data on a call
to sendmsg(2).

For UDP/RAWIP you can set it on the socket and then forget about it
or include it with every message.  For a UDP-style SCTP socket, I
suggest the same.

--brian

---

From: Sridhar Samudrala <[email-masked]>
Date: Wed, 14 Sep 2005 10:34:48 -0700

If multiple associations on a 1-n style socket need to use a
different TOS, may be they could do a peeloff and use
different IP_TOS values on each socket.

Thanks
Sridhar

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 14 Sep 2005 15:04:33 -0400

Brian

> IP_TOS is per-packet and can be used as ancilliary data on a call

Can you point me to an implementation.  The systems I have access to
don't implement this as ancillary data.

> or include it with every message.  For a UDP-style SCTP socket, I

The problem is that that multiple associations on the same SCTP endpoint
can't have different TOS values.  They all end up using the same value
specified by the IP_TOS option.

What I'd love to be able to do is set the option on a per association
basis and forget about it.

I figured others might be interested in doing something similar.

-vlad

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 14 Sep 2005 15:11:21 -0400

That would work to a point.  I've known of applications that have
1000s of associations on the same endpoint and at least half of them
have diffrent TOS values.  Managing that many individiual socks would
be nightmarish.

-vlad

---

From: Randall Stewart <[email-masked]>
Date: Wed, 14 Sep 2005 16:38:06 -0400

> have diffrent TOS values.  Managing that many individiual socks would

Especially when a simple setsockopt that would
save it down on the association level (like you proposed)
would avoid the nightmare :-D

Now I don't know about other implementations.. having not dug
in to the linux side of things.. but I do know in BSD land we do:
**********************************************************
if ((stcb->asoc.ecn_allowed) && ecn_ok) {
	/* Enable ECN */
#if defined(__FreeBSD__) || defined(__APPLE__)
    ip->ip_tos = (u_char)((inp->ip_inp.inp.inp_ip_tos & 0x000000fc) | 	 

                        sctp_get_ect(stcb, chk));
#elif defined(__NetBSD__)
    ip->ip_tos = (u_char)((inp->ip_inp.inp.inp_ip.ip_tos & 0x000000fc) |
                        sctp_get_ect(stcb, chk));
#else
    ip->ip_tos = (u_char)((inp->inp_ip_tos & 0x000000fc) |
                        sctp_get_ect(stcb, chk));
#endif
} else {
    /* No ECN */
#if defined(__FreeBSD__) || defined(__APPLE__)
    ip->ip_tos = inp->ip_inp.inp.inp_ip_tos;
#elif defined(__NetBSD__)
    ip->ip_tos = inp->ip_inp.inp.inp_ip.ip_tos;
#else
    ip->ip_tos = inp->inp_ip_tos;
#endif
}
***********************************************

Which means we are already digging in and getting the
tos value to fill in.. we do this since we end up
composing all fields in the IP header for source
address selection etc...

Soo.. its just a matter of having another
place to store it on the association..

Quite simple to do... its just the layer issue that I think
folks have a problem with.. me, I don't think its so bad.
Strict layering often leads to a system that works but
often will not perform well.. this is one of those
cases that I think would be worth "violating" a
preceived layering issue for the sake of performance :-D

R

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 14 Sep 2005 16:52:59 -0400

Linux is similarly very easy to this in and I've had a prototype
put together for a while now while this idea has been brewing.

-vlad

---

From: Sridhar Samudrala <[email-masked]>
Date: Wed, 14 Sep 2005 16:43:40 -0700

I am OK with having a SCTP_TOS option if having a large no. of
associations on a 1-many style socket with different TOS values
is a common application.

Thanks
Sridhar

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 15 Sep 2005 15:41:41 +0800

> place to store it on the association..

It is just code after all.  But...

> preceived layering issue for the sake of performance :-D

It is more of a name space mix up.  An SCTP level
socket option named SCTP_TOS should probably mean
something which affects SCTP qualify of service at
the SCTP layer.  For example, it can mean some form
of load balancing among the addresses.  I suggest we
think more about it before writing it up.

-- 

						K. Poon.
						[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 15 Sep 2005 09:52:06 +0200

Dear all,

So by looking at the arguments I have seen, we can use
an IP level option for 1-1 only in an efficient way.
But the support of 1-M style sockets is important in
my opinion.

So I vote for an SCTP level option.

Best regards
Michael

---

From: Randall Stewart <[email-masked]>
Date: Thu, 15 Sep 2005 07:32:11 -0400

> associations on a 1-many style socket with different TOS values

Hmm.. wello lets go ahead and make that change for
the next draft... unless someone has a better suggestion
on how to address this of course... I don't see any
other easy way to meet the type of thing Vlad is asking
for.. Other suggestions would be most welcome... if there
are any.. ??

R

---

From: David Borman <[email-masked]>
Date: Thu, 15 Sep 2005 10:58:03 -0500

Would calling it SCTP_IP_TOS make more sense?  That should clarify  
the name space issue.

The reality is that IP_TOS is not an independent IP option, it is  
tied to a specific socket.  The fact that it is an IP level socket  
option has more to do with the fact that it is common functionality  
that can be used my any transport protocol that runs on top of IP,  
and hence can be implemented in one common place.  For SCTP, a finer  
granularity of control is needed in setting the IP tos value in the 1- 
to-many case than the common routine provides, so it makes perfect  
sense to have an SCTP specific option for setting the IP tos values  
for that connection.

             -David Borman

> Randall Stewart wrote:

...

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 16 Sep 2005 10:36:31 +0200

I'm happy with that... Let's name it SCTP_IP_TOS.

Best regards
Michael

On Sep 15, 2005, at 17:58 Uhr, David Borman wrote:

> Would calling it SCTP_IP_TOS make more sense?  That should clarify the 
> name space issue.
>
> The reality is that IP_TOS is not an independent IP option, it is tied 
> to a specific socket.  The fact that it is an IP level socket option 
> has more to do with the fact that it is common functionality that can 
> be used my any transport protocol that runs on top of IP, and hence 
> can be implemented in one common place.  For SCTP, a finer granularity 
> of control is needed in setting the IP tos value in the 1-to-many case 
> than the common routine provides, so it makes perfect sense to have an 
> SCTP specific option for setting the IP tos values for that 
> connection.
>
>             -David Borman
>

---

From: Randall Stewart <[email-masked]>
Date: Fri, 16 Sep 2005 07:01:15 -0400

David:

That works for me :-D

I think we should adopt it in the next round of
the draft .. very very helpful suggestion.. thanks!

R

David Borman wrote:
> be used my any transport protocol that runs on top of IP,  and hence can 
> be implemented in one common place.  For SCTP, a finer  granularity of 
> control is needed in setting the IP tos value in the 1- to-many case 
> SCTP specific option for setting the IP tos values  for that connection.
> 
>             -David Borman
>

---

From: Vlad Yasevich <[email-masked]>
Date: Mon, 19 Sep 2005 17:15:22 -0400

Some additional comment...

There are a few structures with rather glaring memory holes in them.  I
am wondering if this is something that should be corrected in the spec,
or should each implementation try to align data individually.

The following are the glaring holes I found:

   struct sctp_sndrcvinfo {
      uint16_t sinfo_stream;
      uint16_t sinfo_ssn;
      uint16_t sinfo_flags;
               <-------------------- hole
      uint32_t sinfo_ppid;
      uint32_t sinfo_context;
      uint32_t sinfo_timetolive;
      uint32_t sinfo_tsn;
      uint32_t sinfo_cumtsn;
      sctp_assoc_t sinfo_assoc_id;
   };

   struct sctp_remote_error {
       uint16_t sre_type;
       uint16_t sre_flags;
       uint32_t sre_length;
       uint16_t sre_error;
		<---------------- usually a hole [1]
       sctp_assoc_t sre_assoc_id;
       uint8_t sre_data[0];
   };

     struct sctp_paddrparams {
       sctp_assoc_t            spp_assoc_id;
	        <------------------------------- usually a hole [2]
       struct sockaddr_storage spp_address;
       uint32_t                spp_hbinterval;
       uint16_t                spp_pathmaxrxt;
	        <------------------------------- hole
       uint32_t                spp_pathmtu;
       uint32_t                spp_sackdelay;
       uint32_t                spp_flags;
   };

Additinally, the following stucture need to be update to reflect the
sign-ness and size of it's members:

       struct sctp_paddr_change {
          uint16_t spc_type;
          uint16_t spc_flags;
          uint32_t spc_length;
          struct sockaddr_storage spc_aaddr;
          int spc_state;		<--- these two should be int32_t
          int spc_error;                <----
          sctp_assoc_t spc_assoc_id;
        }

-vlad

[1] this is a hole unless sctp_assoc_t is somehow only 16 bits long.  In
most implemenations this is going to be a memory hole.

[2] this is a hole when sctp_assoc_t is not 64 bits.  The alignment
requirements for sockaddr_storage are the same as for sockaddr_in6 and
are 64 bits as recomended by rfc 3493.

---

From: Randall Stewart <[email-masked]>
Date: Mon, 19 Sep 2005 21:17:58 -0400

Vlad:

Good catch..

If no one else objects.. lets put

xxxx_reserved

fields in to pad these out for a "consistent" set
of alignments :-D

R

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 20 Sep 2005 08:56:05 +0200

Dear all,

why do we need to specify these 'padding fields' at all? Everything
should work as long as
- the user does not have to set the padding fields
- the user does not look at them

So I think it is important to draw the implementers attention to
the problem but I do not see a reason why to specify the padding stuff.
Do I miss something?

Best regards
Michael

---

From: Randall Stewart <[email-masked]>
Date: Tue, 20 Sep 2005 04:48:21 -0400

Well.. maybe not.. I suppose the
compiler will insert the proper padding
(or at least it should).. however I have
always tried to take advantage of "padding"
to add "reserve" fields.. and sure enough
lots of times they end up getting used for
something :-D

We don't have to add them... and I suppose
we won't need to add more fields to these
notifications ... but :-D

R

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 20 Sep 2005 22:57:10 +0800

> Would calling it SCTP_IP_TOS make more sense?  That should clarify  the 

This does makes it clearer.  But see below.

> SCTP specific option for setting the IP tos values  for that connection.

If we look at it from the above point of view, I guess a more
appropriate way to specify the IPv4 TOS bit is to use the
existing peer address parameter socket option.  The reason is
that an app may want to use different TOS bits in the IP header
when sending to different peer addresses.  If we just have a
SCTP_IP_TOS option, there is no way an app can change the TOS
bits when sending to different peer addresses.

I suggest adding "uint8_t spp_ipv4_tos" field to the
sctp_paddrparams struct.

-- 

						K. Poon.
						[email-masked]

---

From: Peter Lei <[email-masked]>
Date: Tue, 20 Sep 2005 10:22:27 -0500

Randy,

I'm not sure we really need to add the reserved fields...
the flip side is if a CPU architecture and compiler can
handle non-aligned data and can pack data structures, adding
xxxx_reserved may actually cause additional memory utilzation.

Since these structures are internal use only between a user
application and the sockets API, and the compiler will hide
these "holes", I don't see the necessity of changing this.
This is not a strong opinion though.

thanks,
--peter

---

From: Peter Lei <[email-masked]>
Date: Tue, 20 Sep 2005 10:35:22 -0500

I agree with both suggestions:

SCTP_IP_TOS for marking the IP TOS field on a per-association
basis (for one-to-one sockets, this is really the same as
setting IP_TOS on the socket).

SCTP_TOS for SCTP stack prioritizations, like what Kacheong
mentions or maybe for handling stream priorities within a
single association, etc.

--peter

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 20 Sep 2005 23:46:20 +0800

> or should each implementation try to align data individually.

I guess not.  If we change anything now, it may break
existing apps.  And I don't see any real need to change
them.

>           sctp_assoc_t spc_assoc_id;

Thanks for catching this.  They are defined as 32 bit
in the text and the struct should reflect this.

-- 

						K. Poon.
						[email-masked]

---

From: Sridhar Samudrala <[email-masked]>
Date: Tue, 20 Sep 2005 10:10:32 -0700

> existing apps.  And I don't see any real need to change

We noticed this problem when running 32-bit sctp apps on a 64-bit kernel.
On some dual-mode architectures(ppc64, x86-64 etc) linux support 32-bit
userspace on a 64-bit kernel.

In order to support this scenario, the sizes of the structures exchanged
between the userspace and the kernel need to be same or have some 
compatibility code in the kernel that handle 32-bit userspace
structures. Is this issue faced by other implementations? If so, how do
they handle this?

Thanks
Sridhar

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 20 Sep 2005 11:24:52 -0600

Peter,

It is not necessary to make any change for one-to-one associations.
IP_TOS works fine for one-to-one.  Even for per-packet if you add it to
the ancilliary data recognized by sendmsg.  It is already reportable in
ancilliary data received with recvmsg.

SCTP_IP_TOS is only necessary for one-to-many, because it is so foreign
to the socket model (multiple connections, one socket).

Perhaps what is really needed is a sctp_setassocopt and sctp_getassocopt
that takes a socket option and association id, instead of socket file
descriptor as does setsockopt and getsockopt.  Something like:

int sctp_setassocopt(int associd, int level, int optname,
		     const void *optval, socklen_t option);
int sctp_getassocopt(int associd, int level, int optname,
		     void *optval, socklen_t * optlen);

or

int sctp_setassocopt(int s, int associd, int level, int optname,
		     const void *optval, socklen_t option);
int sctp_getassocopt(int s, int associd, int level, int optname,
		     void *optval, socklen_t * optlen);

Then you could set things like IP_TOS (without modification) on an
association within a one-to-many socket.  They you wouldn't have to
invent so many non-standard option names that are just an association
id followed by the standard option.

In any case, please do not bleed these one-to-many options into the
one-to-one case.  Anything with an association id field has no place
in the one-to-one case.

--brian

---

From: Randall Stewart <[email-masked]>
Date: Tue, 20 Sep 2005 13:30:22 -0400

> id followed by the standard option.

Not saying that the above is not a good idea.. it might actually
be rather elegant..

But I am afraid that when you did a

sctp_setassocopt(sd, id, ..., IP_TOS...);

You might have a problem if it was sent to the IP level since
the IP layer would not really understand what an assoc is.

In such a case I don't know what the need is for a level
in this call...

> one-to-one case.  Anything with an association id field has no place

I disagree here.. it is nice to have an assoc id that can
be considered a unique identifier no matter if its in a 1-2-many or
a 1-2-1 socket... something outside the sd... since that does
not work on 1-2-many..

R

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 21 Sep 2005 17:05:24 +0800

> userspace on a 64-bit kernel.

Solaris supports running both 32 or 64 bit apps on a 64 bit
kernel.  And there is no problem with the SCTP structures.

> structures. Is this issue faced by other implementations? If so, how do

I guess this is an OS issue outside the scope of the API
draft.  And it is actually not your problem to solve.  I
suggest you to go ask the 64 bit porting team to get their
advice.  They can give you the answer.  Otherwise, how
can they support all those already compiled 32 bit binaries
out there in the 64 bit kernel?  Changing the struct is
the wrong answer to this.

-- 

						K. Poon.
						[email-masked]