Subject: [Tsvwg] sctp_connectx / sctp_bindx interface

From: Mark Butler <[email-masked]>
Date: Wed, 08 Feb 2006 21:27:44 -0700

Hello all,

  The sctp_bindx and sctp_connectx have an ABI stability problem.  The 
address information is passed as a string of sockaddr structures.  The 
problem is there is no sockaddr length information, so if the length of  
sockaddr_in or sockaddr_in6  changes in the future, these functions will 
break.   For example, sockaddr_in6 grew by four bytes when sin6_scope_id 
was added in 1999.

A second problem is that sockaddr_in is commonly 14 bytes, leading to 
32-bit alignment problems when you string them together.  A final 
problem is that placing addresses together is not very friendly from an 
interface point of view.

There should be functions like:

int sctp_connectv(int sd, struct addrvec *vec, int count);
int sctp_bindv(int sd, struct addrvec *vec, int count);

with:

struct addrvec {
      struct sockaddr *addr;
      size_t  addr_len;
};

where struct addrvec is designed to be an alias for struct iovec.

Comments?

 - Mark

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 8 Feb 2006 23:34:22 -0700

Mark,

Why not just use sockaddr_storage ?

--brian

Mark Butler wrote:                              (Wed, 08 Feb 2006 21:27:44)
>    problem  is  there is no sockaddr length information, so if the length
>    of    sockaddr_in  or  sockaddr_in6   changes  in  the  future,  these
>    functions  will  break.   For example, sockaddr_in6 grew by four bytes
>    when sin6_scope_id was added in 1999.
>    problem  is  that placing addresses together is not very friendly from
>    an interface point of view.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 9 Feb 2006 09:20:19 +0100

Hi Mark,

BSD has a sin_len or sin6_len field in the sockaddr-like structures.  
I think the problem
is not specific to SCTP but more related how sockaddr structures are  
handled by specific OSes.

Best regards
Michael

On Feb 9, 2006, at 5:27 AM, Mark Butler wrote:

> Hello all,
>
>   The sctp_bindx and sctp_connectx have an ABI stability problem.   
> The address information is passed as a string of sockaddr  
> structures.  The problem is there is no sockaddr length  
> information, so if the length of  sockaddr_in or sockaddr_in6   
> changes in the future, these functions will break.   For example,  
> sockaddr_in6 grew by four bytes when sin6_scope_id was added in 1999.
>
> A second problem is that sockaddr_in is commonly 14 bytes, leading  
> to 32-bit alignment problems when you string them together.  A  
> final problem is that placing addresses together is not very  
> friendly from an interface point of view.
>

---

From: "Henrik Jensen (TED)" <[email-masked]>
Date: Thu, 09 Feb 2006 16:59:41 +0100

Hi

The socket address array problem does also exist for the following 
functions (as specified by the draft-ietf-tsvwg-sctpsocket-11 draft):
   int sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr **addrs);
   int sctp_getladdrs(int sd, sctp_assoc_t id, struct sockaddr **ss);
   int sctp_sendx(int sd,...,struct sockaddr *addrs, int addrcnt,...);

An argument against the sockaddr_storage type is that it can be rather 
large (e.g. approximately 128 bytes when unix domain sockets are 
supported). The size of the sockaddr_in structure is 16 bytes, but 
actually only 8 bytes are in use. Thus apprx 16 times more memory is 
used than actually needed. In the future, if (when) a new and larger 
socket address type is introduced the overhead will grow worse.

Regarding the addrvec structure:  If the addr_len is needed, I think the 
correct type is socklen_t. But is it really needed? - the sockaddr 
itself has the sa_family entry specifying the address family, thus the 
actual sockaddr_xx type is indirectly given.
But the socket API has a tradition for explicit specification of the 
socket address length - and it is useful for the kernel (or IP stack) as 
a consistency check.
The addrvec structure is also useful for hiding one of the pointers in 
the sctp_getpaddrs() function - otherwise the prototype will be:  
int sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr ***addrs );
or more precise:
int sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr *(*addrs)[] );

Regards
Henrik Jensen

>>   where struct addrvec is designed to be an alias for struct iovec.

------- 
Henrik Jensen   
Ericsson Telebit A/S 
DK-8260 Viby J,  Denmark

---

From: Mark Butler <[email-masked]>
Date: Thu, 09 Feb 2006 13:18:43 -0700

Henrik Jensen (TED) wrote:

> Regarding the addrvec structure:  If the addr_len is needed, I think 
> the correct type is socklen_t.

The reason why it is size_t instead of socklen_t is so that it is 
compatible with struct iovec. That allows implementations to reuse the 
same code.   Also, if socklen_t were smaller than size_t (8 or 16 bits 
for example), the space required to align the next entry for a pointer 
would be used anyway.

> But is it really needed? - the sockaddr itself has the sa_family entry 
> specifying the address family, thus the actual sockaddr_xx type is 
> indirectly given.

That works as long as the size never changes, which it does 
occasionally,  sockaddr_in6 being the most recent example. I doubt it 
will happen anytime soon, but sockaddr_in could use a sin_scope_id for 
link local (169.254.x.x) addresses, just like the one for sockaddr_in6.

> socket address length - and it is useful for the kernel (or IP stack) 
> as a consistency check.
> the sctp_getpaddrs() function - otherwise the prototype will be:  int 
> sctp_getpaddrs(int sd, sctp_assoc_t id, struct sockaddr ***addrs );
> or more precise:
Yes, same problem.

 - Mark

---

From: Mark Butler <[email-masked]>
Date: Thu, 09 Feb 2006 13:32:54 -0700

>Why not just use sockaddr_storage ?
Changing to sockaddr_storage would be the simplest way to fix several of 
the existing functions, especially sctp_getladdrs, etc.  The general 
problem with sctp_connectx() / sctp_bindx() the problem is that the user 
has to allocate a memory buffer, copy in the sockaddr structures, zero 
pad them, and probably free the buffer on exit, where with an addrvec 
interface, he/she just supplies pointers to an arbitrary set of sockaddr 
structures, no assembly required.

 - Mark

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 9 Feb 2006 13:44:21 -0700

Mark,

If the application maintains addresses in a sockaddr_storage array,
there is no need to copy.  AFAIK the POSIX purpose of sockaddr_storage
structure was so that an application could be guaranteed that the
structure will hold any socket address, as well as providing proper
alignment, and is just good design (making the application independent
of the exact address format makes it more portable to other transport
protocols).

--brian

Mark Butler wrote:                                                    (Thu, 09 Feb 2006 13:32:54)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Thu, 9 Feb 2006 13:46:13 -0700

Henrik,

Henrik Jensen (TED) wrote:                                          (Thu, 09 Feb 2006 16:59:41)
> socket address type is introduced the overhead will grow worse.

I would not consider 128 bytes large.  It is a very litte cost to pay
for address portability.  (Consider also that host name is a valid SCTP
address.)

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Thu, 09 Feb 2006 13:47:04 -0700

> handled by specific OSes.

I think the idea of adding sin_len to sockaddr is a good one.  
Unfortunately, POSIX does not require it, and it cannot be done in 
general without severely breaking the ABI, which means it isn't likely 
to be implemented for other operating systems anytime soon.

Even if sin_len were widely available, the function prototypes for all 
the multiple address functions are technically incorrect. They are not 
an array of sockaddr structures, because struct sockaddr_in6 is larger 
than struct sockaddr.   The only way to fix the existing function 
prototypes to be consistent is to use struct sockaddr_storage instead.

 - Mark

---

From: "Henrik Jensen (TED)" <[email-masked]>
Date: Fri, 10 Feb 2006 00:02:45 +0100

Hi,

>for address portability.  (Consider also that host name is a valid SCTP
The current revision of the sctp socket draft 
(draft-ietf-tsvwg-sctpsocket-11.txt) does not give any specifications on 
how to bind to a host name - but using a sockaddr_hostname(?I don't know 
the correct name?) would be a nice solution.
But... in order for the sockaddr_storage to hold a domain name the size 
must be 260=(256+4) bytes due to the maximum length of a domain name. 
That doubles the cost.....

If adding a new member (e.g. sin_scope_id) to the sockaddr_in structure 
(causing the size of the structure to change) causes problems, why are 
we then not facing the same problems if the size of the sockaddr_storage 
is changed? - I have in mind the suggested array of sockaddr_storage 
elements for the sctp_connectx(int sd, struct sockaddr_storage addr[], 
int count).
- or, am I missing a point? -  Does the kernel/IP-stack use the 
(socklen_t) len to deduce which members of a sockaddr_xx structure the 
calling application knows about.  I.e. if a sockaddr_in6 is given 
together with len=24, then the application does not know abount the 
sin6_scope_id member. If len=28 then the application knows the 
sin6_scope_id. This means that the len parameter gives a (kind of) 
revision number of the layout/contents of the structure.
But is that useful for other things than letting old applications 
(compiled on old header files) execute on a newer kernel/socket-API ? - 
or is this binary compability the original problem (Mark did start the 
discussion with the words 'ABI stability problem')?

NB: Adding new members to the sockaddr_in structure is not a good 
example. Due to the sin_zero[8] member new members can be added without 
changing the size of the sockaddr_in.
 
/Henrik

-- 
Henrik Jensen          
Ericsson Telebit A/S 
DK-8260 Viby J,  Denmark

---

From: Mark Butler <[email-masked]>
Date: Thu, 09 Feb 2006 21:05:12 -0700

Henrik Jensen (TED) wrote:

>> (Thu, 09 Feb 2006 16:59:41)
>>  
>>
>>> An argument against the sockaddr_storage type is that it can be 
>>> rather large (e.g. approximately 128 bytes when unix domain sockets 
>>> are supported). The size of the sockaddr_in structure is 16 bytes, 
>>> but actually only 8 bytes are in use. Thus apprx 16 times more 
>>> memory is used than actually needed. In the future, if (when) a new 
>>> and larger socket address type is introduced the overhead will grow 
>>> worse.
>>>
>>>   
>>
>>
> (draft-ietf-tsvwg-sctpsocket-11.txt) does not give any specifications 
> on how to bind to a host name - but using a sockaddr_hostname(?I don't 
> know the correct name?) would be a nice solution.
> But... in order for the sockaddr_storage to hold a domain name the 
> size must be 260=(256+4) bytes due to the maximum length of a domain 
> name. That doubles the cost.....
>
> If adding a new member (e.g. sin_scope_id) to the sockaddr_in 
> structure (causing the size of the structure to change) causes 
> problems, why are we then not facing the same problems if the size of 
> the sockaddr_storage is changed? - I have in mind the suggested array 
> of sockaddr_storage elements for the sctp_connectx(int sd, struct 
> sockaddr_storage addr[], int count).

Practicability aside, sockaddr_dns or whatever would not need a fixed 
size for API purposes because a host name can translate to multiple 
addresses with multiple types.  So there would be no need to pass or 
receive an array of host names.  It is an indication that 
sockaddr_storage may have to be replaced (again) someday.

> (compiled on old header files) execute on a newer kernel/socket-API ? 
> - or is this binary compability the original problem (Mark did start 
> the discussion with the words 'ABI stability problem')?
>
> example. Due to the sin_zero[8] member new members can be added 
> without changing the size of the sockaddr_in.

I believe the original purpose of the pad bytes was to make sockaddr_in 
the same size as sockaddr, so sockaddr would  be the same size for all 
network protocols (aside from PF_UNIX).

However, making sockaddr_in larger than necessary turned out poorly in 
practice, because the full length is passed as a parameter even though 
the extra pad bytes are ignored (and are not required to be set to 
zero).  So if a new field is added inside the pad area, protocol 
implementations will not know whether the data is random garbage or not 
unless the total size is increased, and all newly compiled applications 
are changed to clear the structure before use.  POSIX now requires that 
sockaddr_in6 be cleared before use, but not sockaddr_in (yet).

In practice that means that there would have to be a sockaddr_in4 with a 
size larger (or at least different from) sockaddr_in to add any fields 
for IPv4. There is no end of programs that do not clear sockaddr_in 
before use and there is no reliable way to detect whether they do or not.

So one might have:

struct sockaddr_in4 {
     sa_family_t  sin4_family;
     in_port_t    sin4_port;
     in_addr_t    sin4_addr;
     uint32_t     sin4_scope_id;
     uint32_t     sin4_flowinfo;
     uint32_t     _reserved;
};

sin4_scope_id would have identical semantics to sin6_scope_id, 
sin4_flowinfo would have identical semantics to sin6_flowinfo (as 
applicable), and the reserved field would be there so that 
implemenations could detect that they were getting a sockaddr_in4 
instead of a sockaddr_in.  Like sockaddr_in6, sockaddr_in4 would be 
required to be cleared before use.  Not that this is particularly 
germane to the SCTP socket draft...

 - Mark

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 10 Feb 2006 09:29:57 +0100

Hi Henrik,

what is the intention of binding to a hostname? Normally you would  
call a
library function of the resolver to resolve the domain name and then use
the result.

Best regards
Michael

On Feb 10, 2006, at 12:02 AM, Henrik Jensen (TED) wrote:

>> Henrik Jensen (TED)  
>> wrote:                                          (Thu, 09 Feb 2006  
>> 16:59:41)
>>> rather large (e.g. approximately 128 bytes when unix domain  
>>> sockets are supported). The size of the sockaddr_in structure is  
>>> 16 bytes, but actually only 8 bytes are in use. Thus apprx 16  
>>> times more memory is used than actually needed. In the future, if  
>>> (when) a new and larger socket address type is introduced the  
>>> overhead will grow worse.
>>>
>>>
>>
> The current revision of the sctp socket draft (draft-ietf-tsvwg- 
> sctpsocket-11.txt) does not give any specifications on how to bind  
> to a host name - but using a sockaddr_hostname(?I don't know the  
> correct name?) would be a nice solution.
> size must be 260=(256+4) bytes due to the maximum length of a  
> domain name. That doubles the cost.....
>
> problems, why are we then not facing the same problems if the size  
> of the sockaddr_storage is changed? - I have in mind the suggested  
> array of sockaddr_storage elements for the sctp_connectx(int sd,  
> struct sockaddr_storage addr[], int count).
> (socklen_t) len to deduce which members of a sockaddr_xx structure  
> the calling application knows about.  I.e. if a sockaddr_in6 is  
> given together with len=24, then the application does not know  
> abount the sin6_scope_id member. If len=28 then the application  
> knows the sin6_scope_id. This means that the len parameter gives a  
> (kind of) revision number of the layout/contents of the structure.
> (compiled on old header files) execute on a newer kernel/socket- 
> API ? - or is this binary compability the original problem (Mark  
> did start the discussion with the words 'ABI stability problem')?
>
> Henrik Jensen          Ericsson Telebit A/S DK-8260 Viby J,  Denmark
>

---

From: "Henrik Jensen (TED)" <[email-masked]>
Date: Fri, 10 Feb 2006 12:38:50 +0100

Hi Michael

RFC2960 gives a little information on the ideas behind the usage of 
domain name in the SCTP INIT (and INIT_ACK) chunks.
A host in a NAT'ed network can identify itself in the INIT chunk by a 
domain name. The receiver of the INIT must then resolve the domain name 
into ip-addresses (typically via DNS). In this way the INIT chunk does 
not contain any IP-addresses and the NAT-gateway does not need to modify 
the contents of the SCTP INIT chunk (addresses in the ip-header must 
still be modified).

/Henrik

Michael Tuexen wrote:

>> (compiled on old header files) execute on a newer kernel/socket- API 
>> ? - or is this binary compability the original problem (Mark  did 
>> start the discussion with the words 'ABI stability problem')?
>>

-- 
Henrik Jensen          
Ericsson Telebit A/S      
DK-8260 Viby J,  Denmark

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 10 Feb 2006 14:21:10 +0100

Hi Hendrik,

I think the hostname parameter in RFC 2960 is problematic. I do
not see how one can implement it without having several security  
problems
and how it really helps in NATed environments. If you are interested in
SCTP/NAT there is an ID which describes how this works:
http://www.ietf.org/internet-drafts/draft-stewart-behave-sctpnat-01.txt

Since I do not think it is a good idea to use the hostname parameter in
SCTP, I think there is no need to provide a way in the API to access  
it...

Best regards
Michael

On Feb 10, 2006, at 12:38 PM, Henrik Jensen (TED) wrote:

> A host in a NAT'ed network can identify itself in the INIT chunk by  
> a domain name. The receiver of the INIT must then resolve the  
> domain name into ip-addresses (typically via DNS). In this way the  
> INIT chunk does not contain any IP-addresses and the NAT-gateway  
> does not need to modify the contents of the SCTP INIT chunk  
> (addresses in the ip-header must still be modified).
>
> /Henrik
>
>
> Michael Tuexen wrote:
>
>> Hi Henrik,
>>
>> what is the intention of binding to a hostname? Normally you  
>> would  call a
>>>> wrote:                                          (Thu, 09 Feb  
>>>> 2006  16:59:41)
>>>>> An argument against the sockaddr_storage type is that it can  
>>>>> be  rather large (e.g. approximately 128 bytes when unix  
>>>>> domain  sockets are supported). The size of the sockaddr_in  
>>>>> structure is  16 bytes, but actually only 8 bytes are in use.  
>>>>> Thus apprx 16  times more memory is used than actually needed.  
>>>>> In the future, if  (when) a new and larger socket address type  
>>>>> is introduced the  overhead will grow worse.
>>>>>
>>>>>
>>>>
>>> sctpsocket-11.txt) does not give any specifications on how to  
>>> bind  to a host name - but using a sockaddr_hostname(?I don't  
>>> But... in order for the sockaddr_storage to hold a domain name  
>>> the  size must be 260=(256+4) bytes due to the maximum length of  
>>> a  domain name. That doubles the cost.....
>>>
>>> problems, why are we then not facing the same problems if the  
>>> size  of the sockaddr_storage is changed? - I have in mind the  
>>> suggested  array of sockaddr_storage elements for the  
>>> sctp_connectx(int sd,  struct sockaddr_storage addr[], int count).
>>> (socklen_t) len to deduce which members of a sockaddr_xx  
>>> structure  the calling application knows about.  I.e. if a  
>>> sockaddr_in6 is  given together with len=24, then the application  
>>> does not know  abount the sin6_scope_id member. If len=28 then  
>>> the application  knows the sin6_scope_id. This means that the len  
>>> parameter gives a  (kind of) revision number of the layout/ 
>>> contents of the structure.
>>> But is that useful for other things than letting old  
>>> applications  (compiled on old header files) execute on a newer  
>>> kernel/socket- API ? - or is this binary compability the original  
>>> problem (Mark  did start the discussion with the words 'ABI  
>>> stability problem')?
>>>
>>> NB: Adding new members to the sockaddr_in structure is not a  
>>> good  example. Due to the sin_zero[8] member new members can be  
>>> added  without changing the size of the sockaddr_in.
>>> /Henrik
>>>
>>>
>>> --

---

From: Mark Butler <[email-masked]>
Date: Fri, 10 Feb 2006 23:31:26 -0700

> SCTP, I think there is no need to provide a way in the API to access  

The RFC2960 hostname and API support for hostnames are really distinct 
issues.  The wire level hostname causes more problems than it solves (as 
you say).  But adding a library function to connect by hostname isn't a 
bad idea, especially given the complexity of using getnameinfo() and 
connectx() to do it.  Something like:

int sctp_connect_by_name(int sd, const char *host_name);

where the function uses the local resolver configuration to look up 
multiple IPv4 and/or IPv6 addresses and uses them to connect to the 
target host.

As far as the NAT situation goes,  an initiator behind a split NAT could 
use an API function to tell the system which bind addresses are behind a 
NAT, and what the exterior addresses are for each one, so that the 
system could send the correct addresses in the INIT chunk.    There 
would also need to be a new chunk to initiate mapping on the other NAT 
boxes from the _inside_.  Something like:

(type=PROBE, flags=0, length=12, local verification tag, remote 
verification tag)

This would be sent with each HEARTBEAT after receiving an INIT ACK until 
the corresponding path was confirmed valid. It would be forwarded 
through one or more layers of initiator side NATs to establish the 
proper mapping on each one.   That would allow associations to be easily 
established from clients behind split NATs.   Servers behind split NATs 
are easy - just set up static port forwarding for each one.

Comments?

  - Mark

---

From: Michael Tuexen <[email-masked]>
Date: Sat, 11 Feb 2006 09:27:19 +0100

Hi Mark,

see my comments in-line.

Best regards
Michael

On Feb 11, 2006, at 7:31 AM, Mark Butler wrote:

>> SCTP, I think there is no need to provide a way in the API to  
>> access  it...
>
>
> The RFC2960 hostname and API support for hostnames are really  
> distinct issues.  The wire level hostname causes more problems than  
> it solves (as you say).  But adding a library function to connect  
> by hostname isn't a bad idea, especially given the complexity of  
> using getnameinfo() and connectx() to do it.  Something like:
>
The problem I see with this is, that you must make sure that all  
addresses you use with sctp_connect*
belong to the same host. If you use DNS to resolve a name to a list  
of IP addresses and get back
multiple addresses these can all belong to the same host (and could  
be used) or they do not
belong to the same host, but to a number of servers providing the  
same service.
I do not know of any way to get out of the DNS all addresses of one  
particular host.
> As far as the NAT situation goes,  an initiator behind a split NAT  
> could use an API function to tell the system which bind addresses  
> are behind a NAT, and what the exterior addresses are for each one,  
> so that the system could send the correct addresses in the INIT  
> chunk.    There would also need to be a new chunk to initiate  
> mapping on the other NAT boxes from the _inside_.  Something like:
>
> This would be sent with each HEARTBEAT after receiving an INIT ACK  
> until the corresponding path was confirmed valid. It would be  
> forwarded through one or more layers of initiator side NATs to  
> establish the proper mapping on each one.   That would allow  
> associations to be easily established from clients behind split  
> NATs.   Servers behind split NATs are easy - just set up static  
> port forwarding for each one.
We have a concept (which has to be written down as an ID) to make  
multihomed hosts behind da NAT
working by using ADD-IP. Some packets containing both V-Tags are also  
necessary to establish
the state on paths where the initial association setup did not  
happen. But getting ADDIP/SCTP-AUTH to
PS has currently higher priority.

---

From: Mark Butler <[email-masked]>
Date: Sat, 11 Feb 2006 15:39:11 -0700

Michael Tuexen wrote:

>> it solves (as you say).  But adding a library function to connect  by 
>> hostname isn't a bad idea, especially given the complexity of  using 
>> getnameinfo() and connectx() to do it.  Something like:
>>

That shouldn't be a problem - you can try IP addresses for different 
hosts in the same service group transparently because the final 
addresses will be set from the first returning INIT ACK and the others 
will be discarded.

- Mark

---

From: Michael Tuexen <[email-masked]>
Date: Sun, 12 Feb 2006 20:58:45 +0100

Hi Mark,

see my comments in-line.

Best regards
Michael

On Feb 11, 2006, at 11:39 PM, Mark Butler wrote:

>>> distinct issues.  The wire level hostname causes more problems  
>>> than  it solves (as you say).  But adding a library function to  
>>> connect  by hostname isn't a bad idea, especially given the  
>>> complexity of  using getnameinfo() and connectx() to do it.   
>>> Something like:
>>>
>>> where the function uses the local resolver configuration to look  
>>> up  multiple IPv4 and/or IPv6 addresses and uses them to connect  
>>> to the  target host.
>>
>> belong to the same host. If you use DNS to resolve a name to a  
>> list  of IP addresses and get back
>> multiple addresses these can all belong to the same host (and  
>> could  be used) or they do not
>> I do not know of any way to get out of the DNS all addresses of  
>> one  particular host.
>
>
> That shouldn't be a problem - you can try IP addresses for  
> different hosts in the same service group transparently because the  
> final addresses will be set from the first returning INIT ACK and  
> the others will be discarded.
But what happens if you get two addresses from DNS which belong to  
two different hosts and only one
host is listening on a particular port p. When you now call connectx  
and two INITs are sent
out, one INIT-ACK is received and one ABORT. If the ABORT is received  
after the INIT-ACK, the
association is killed.
This means that you can not talk to the host listening on the port p  
just because another host
is not.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sun, 12 Feb 2006 14:43:58 -0700

Michael,
> just because another host

Use a different verification tag for each INIT.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Michael Tuexen <[email-masked]>
Date: Sun, 12 Feb 2006 22:58:56 +0100

Brian,

see my comments in-line.

Best regards
Michael

> Use a different verification tag for each INIT.
Which is like setting up multiple associations simultaneously but  
using only the
first one and therefore would work.
But it is more than just an issue of the (socket)API. Using multiple  
V-tags
also lowers the security during the setup.

---

From: Mark Butler <[email-masked]>
Date: Sun, 12 Feb 2006 15:35:46 -0700

Michael Tuexen wrote:

>> addresses will be set from the first returning INIT ACK and  the 
>> others will be discarded.
>

Good point. The worst case assumption is that every address may be a 
different host (that may be partially operable or overloaded), so a 
simple implementation would just connect to each address in turn.  A 
better implementation would set a small initial timeout and cycle 
through addresses, increasing the timeout each cycle similar to the way 
it is done at the SCTP level, except not aborting until connect(2) on 
every address returns with either ECONNREFUSED or EHOSTUNREACH, or they 
all return ETIMEDOUT with the maximum timeout.

 - Mark

---

From: Mark Butler <[email-masked]>
Date: Sun, 12 Feb 2006 17:00:31 -0700

> also lowers the security during the setup.

I think used properly, multiple v-tags increases the security (and 
definitely the reliability) of the init process.  One must simply add 
the rule that fatal errors (including ABORTs) must occur on all 
addresses before giving up.  Otherwise after an an address 
reconfiguration,  a new host might occupy one of the vacated addresses 
before the new information is distributed and issue an ABORT causing the 
association to fail unnecessarily.    Such a change would also make 
connectx() work with multiple hosts without another level of timeout / 
retry.

 - Mark

---

From: Randall Stewart <[email-masked]>
Date: Mon, 13 Feb 2006 07:27:06 -0500

Michael:

Having read most of this thread... 'here are my
high order bits' (stealing from Mark Allman :-D)

1) The hostname address parameter was intended to go through
    NAT's... and IMO it is of no use for that... there are
    better ways, as you pointed out... and will hopefully be
    forthcoming in our copious free time :-D

2) I don't see an alignment issue with an array of
    sockaddr_in's or an array of sockaddr_in6's.. at least
    most compiles should take care of this for you automatically
    (if they can't, handling an array of 32 bit int's on a 64 bit
     processor must be really interesting :-D)

3) It seems to me that sockaddr_storage is a pig.. 128 bytes
    seems an awful lot to allocate for either a 16 byte sockaddr
    for sockaddr_in or a 28 byte sockadd_in6 (at least thats what
    a little test program tells me on BSD):

     bash-stewlap: ./test
     sockaddr_storage is 128 bytes long
     sockaddr_in is 16 bytes long
     sockaddr_in6 is 28 bytes long

    Note that both of these structures print out as 64 bit aligned.

4) Yes, its true, that BSD has a sin_len field.. which I think
    was rather silly of the other implementations not to adopt. It
    makes it real handy to find the length of the sockaddr * passed
    in. Of course one can always do the grungy:
    if(sa->sa_family == FOO)
        len = x;
    else if(....)

    To bad other OS's did not pick the simpler sa_len mechanism.

5) Both BSD and Linux have internally defined
union sctp_xxxxxxxx {
#ifdef AF_INET
	struct sockaddr_in  sin;
#endif
#ifdef AF_INET6
	struct sockaddr_in6 sin6;
#endif
	struct sockaddr     sa;
};
    I put xxxxxx here since I think they are not the same
    name, but both implementations have this....

6) It seems to me that an easy way to approach the
    implementations that accept an array of sockaddr * as
    an argument is to either:

    a) Insist in the document that they are the same type (this
       means for V6 with V4 you end up with the ugly mapped
       stuff which I have always thought silly...).

    <or>

    b) Change it to an array of sctp_xxxxxx's.. and come up
       with a common name for both linux/bsd.

    <or>

    c) Change it to an array of sockaddr_storage's as mentioned
       previously.

I like <b> best of all.. <a> as a second option that is less
distasteful to me.. and I like the sockaddr_storage least of
all due to the tremendous amount of extra memory you end up with ;-<

The other consideration (IMO) is what is defined in Solaris.. Kacheong
is less likely to get changes into sun so it is best to get something
with the least disruption to solaris (if it is possible and makes
sense). I know sun does NOT have all the extended api implemented
so this might not be so hard...

7) As to doing a connect*() and listing different addresses from
    a DNS query or something. This is IMO NOT the purpose of the
    interface. It was added to connect to a SINGLE host that had
    multiple addresses so that one could, at least in theory, get
    some fault tolerance during a connect(). To prevert it to
    connect to multiple hosts seems to me the wrong approach. Instead
    I would suggest that some OTHER command be added for that very
    purpose (the proponents of such an interface are welcome to
    suggest a name.. I am not a proponent so I won't suggest one).

R

Michael Tuexen wrote:
> Hi Mark,
> 
> BSD has a sin_len or sin6_len field in the sockaddr-like structures.  I 
> think the problem
>> structures.  The problem is there is no sockaddr length  information, 
>> so if the length of  sockaddr_in or sockaddr_in6   changes in the 
>> future, these functions will break.   For example,  sockaddr_in6 grew 
>> by four bytes when sin6_scope_id was added in 1999.
>>

-- 
Randall Stewart
803-345-0369 <or> 815-342-5222(cell)

---

From: Mark Butler <[email-masked]>
Date: Mon, 13 Feb 2006 15:06:13 -0700

>     processor must be really interesting :-D) 

I was mistaken about word alignment.   The real problem is sockaddr size 
dependency.  If the size of sockaddr_in or sockaddr_in6 changes, the ABI 
breaks. 

>    a little test program tells me on BSD):

...

>       stuff which I have always thought silly...).

This has the size dependency problem.

>       with a common name for both linux/bsd.

This would work if there was a reasonable amount of room for growth.

> all due to the tremendous amount of extra memory you end up with ;-<
May I suggest another option

d)  Add an address length argument

Since changing the existing ABI is basically unavoidable, adding an 
address length argument should be considered.  It is less intrusive, 
easier to retrofit, doesn't need a new type, matches existing 
interfaces, etc.

Something like:

     int sctp_bindx(int sd, const struct sockaddr *addrs, socklen_t 
addrlen, int addrcnt, int flags);
     int sctp_connectx(int sd, const struct sockaddr *addrs, socklen_t 
addrlen, int addrcnt) ;
     int sctp_getladdrs(int sd, sctp_assoc_t id, void **addrs, socklen_t 
*addrlen);
     int sctp_getpaddrs(int sd, sctp_assoc_t id, void **addrs, socklen_t 
*addrlen);

where address families could be mixed by using the highest length and 
setting the pad bytes to zero.

 - Mark

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 13 Feb 2006 15:44:46 -0700

Mark,

That would also permit an application to always just use sockaddr_storage
sized elements in the array if it prefers.  But, then, it would just move
responsibility to the application to handle any changes.

--brian

Mark Butler wrote:                         (Mon, 13 Feb 2006 15:06:13)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Mon, 13 Feb 2006 21:03:25 -0700

>responsibility to the application to handle any changes.
The existing POSIX interfaces allow sockaddr_storage to be used, but do 
not require it.    Adding a length parameter here has the same benefits, 
and makes it easy to fix existing code.  One can have pre-processor 
definitions that wrap the old signatures with no problem, for example.

 - Mark

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 13 Feb 2006 22:52:16 -0700

Mark,

Yes, I agree, it might be the best solution.

Mark Butler wrote:                                                    (Mon, 13 Feb 2006 21:03:25)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Randall Stewart <[email-masked]>
Date: Tue, 14 Feb 2006 11:34:26 -0500

Seems fine to me :-D

R

---

From: Mark Butler <[email-masked]>
Date: Tue, 14 Feb 2006 11:39:13 -0700

> R

You agree we should add a length parameter to all four functions?

Something like:

- Mark

>>> The existing POSIX interfaces allow sockaddr_storage to be used, but 
>>> do not require it.    Adding a length parameter here has the same 
>>> benefits, and makes it easy to fix existing code.  One can have 
>>> pre-processor definitions that wrap the old signatures with no 
>>> problem, for example.
>>>
>>> - Mark
>>
>>
>>
>
>

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 14 Feb 2006 20:55:44 +0100

Mark,
If I get your suggestion right, you want that all addresses use the  
same amount
of memory given by addrlen?

This would break the possibility of using a packed array which is  
currently supported.

Best regards
Michael

On Feb 14, 2006, at 7:39 PM, Mark Butler wrote:

>    int sctp_connectx(int sd, const struct sockaddr *addrs,  
> socklen_t addrlen, int addrcnt) ;
>    int sctp_getladdrs(int sd, sctp_assoc_t id, void **addrs,  
> socklen_t *addrlen);
>    int sctp_getpaddrs(int sd, sctp_assoc_t id, void **addrs,  
> socklen_t *addrlen);
>
> - Mark
>
>>> Mark Butler  
>>> wrote:                                                    (Mon,  
>>> 13 Feb 2006 21:03:25)
>>>> The existing POSIX interfaces allow sockaddr_storage to be used,  
>>>> but do not require it.    Adding a length parameter here has the  
>>>> same benefits, and makes it easy to fix existing code.  One can  
>>>> have pre-processor definitions that wrap the old signatures with  
>>>> no problem, for example.
>>>>
>>>> - Mark
>>>
>>>
>>>
>>
>>
>
>

---

From: Randall Stewart <[email-masked]>
Date: Tue, 14 Feb 2006 17:13:12 -0500

Michael:

Which functions has sun implemented... I can never
keep straight which ones they have and which ones they don't.

I would hate to change the interface if it caused Sun
major problems..

Oh, another thought.. what about the embedded:

struct sctp_sockaddr {
      u_int16 len;
      u_int16 pad;
      union  {
        struct sockadd_in sin;
      }so;
};

This could give you a len and allow you to
wander through the structure...

Just a thought.

R

Michael Tuexen wrote:
>>>> wrote:                                                    (Mon,  13 
>>>> Feb 2006 21:03:25)

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 14 Feb 2006 23:23:39 +0100

The file /usr/include/netinet/sctp.h from Solaris 10, 01/06 contains

extern int sctp_bindx(int, void *, int, int);
extern void sctp_freeladdrs(void *);
extern void sctp_freepaddrs(void *);
extern int sctp_getladdrs(int, sctp_assoc_t, void **);
extern int sctp_getpaddrs(int, sctp_assoc_t, void **);
extern int sctp_opt_info(int, sctp_assoc_t, int, void *, socklen_t *);
extern int sctp_peeloff(int, sctp_assoc_t);
extern ssize_t sctp_recvmsg(int, void *, size_t, struct sockaddr *,
     socklen_t *, struct sctp_sndrcvinfo *, int *msg_flags);
extern ssize_t sctp_send(int, const void *, size_t,
     const struct sctp_sndrcvinfo *, int);
extern ssize_t sctp_sendmsg(int, const void *, size_t, const struct 
sockaddr *,
     socklen_t, uint32_t, uint32_t, uint16_t, uint32_t, uint32_t);

I'm very hesitating to change the API... I understand Marks point, but
we went through a discussion how the addresses should be handled about
2 years ago, I think.

Best regards
Michael

---

From: Mark Butler <[email-masked]>
Date: Tue, 14 Feb 2006 15:31:52 -0700

> This would break the possibility of using a packed array which is  
There is no portable, stable way to use a packed string of mixed
sockaddr structures because the size of sockaddr_xx changes
occasionally.    That is why connect(2), bind(2), getpeername(2), and
getsockname(2) all have address length parameters.  Most platforms still
perform operations on sockaddr_in6's which do not have sin6_scope_id,
something that was added about seven years ago, and they use the address
length to tell the difference.

On more general principles, I think all of these functions (or their
equivalents) ought to deal with independent, non-concatenated sockaddr
structures as I suggested earlier.  Then there are no padding issues
with mixed binds, and no one has to allocate and assemble sockaddr
strings unnecessarily.

- Mark

---

From: Mark Butler <[email-masked]>
Date: Tue, 14 Feb 2006 16:24:36 -0700

Michael Tuexen wrote:

> ....

These functions are likely to all break the ABI in the future. And if a 
platform makes adaptations to preserve the existing ABI, source 
compatibility will break instead.   The only (practical) way to use them 
without breaking the ABI is to add a function that gets the runtime 
length of a sockaddr for a particular address family, and then pad the 
compile time length to the runtime length with zeroes.  Or worse, lop 
bytes off the end, if the interface sockaddr length is frozen to 
preserve the ABI used by existing code.

So I guess we should add:

    int sctp_getaddrlen(sa_family_t family);

where the function returns the appropriate size to use for sctp_bindx, 
sctp_connectx, sctp_getladdrs, and sctp_getpaddrs, by address family or 
zero if the address family is not supported.  Comments?

 - Mark