Subject: [Tsvwg] Re: ephemeral ports and bindx

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Apr 2007 13:57:02 +0200

Randy,

I'm not sure if it is a good idea to silently accept things
from the user (x,y) which are wrong without complaining, just
ignoring...

So what about:

1. allow bindx() after bind() or without bind().
2. if in bind() or bindx() any non-zero port number
    is used and checked. This allows explicit binding
    with bind() and bindx(). It required to use the
    same non-zero port number in all bindx() calls.
3. If a zero port number is used the kernel will
    chose an ephemeral one if the socket is unbound
    or will use the already known one if the socket
    is bound.

If something goes wrong, return an error.

Best regards
Michael

On Apr 4, 2007, at 12:42 PM, Randall Stewart wrote:

> Jan:
>
> Well I was wrong..
>
> BSD does NOT act the way you describe that linux does.. and in
> fact what it does seems to make more sense..
>
> What BSD does is explicitly IGNORE the port that is
> present if you are already bound and uses the one
> you are bound to.
>
> So if you send in
>
> bindx(
> addr1: port=0
> addr2: port=x
> addr3: port=y
> )
>
> And you are already bound to say port z, you will
> get addr1, addr2 and addr3 added to the endpoint.
>
> If you were NOT bound, you would get an ephemeral port
> z and be bound to it on addr1, followed by additional
> bindings of z to addr2 and addr3.. the port arguments
> x, y would be ignored.
>
> Hmm.. not sure which is the right way.. but this really
> should be discussed on the tsvwg list (as to text
> changes to the socket-api).
>
> R
>
>
> Randall Stewart wrote:
>> Jan Newmarch wrote:
>>> Hi Randall,
>>>
>>> Sorry, I thought I included a copy of my query - but I am currenty
>>> forced to use one of those web mail systems and it must be doing  
>>> funny
>>> things... Here is my question:
>> I can still subscribe you to the list.. But I will try to
>> answer the question below..
>>>
>>>
>>> I have a query about ephemeral ports and subsetting interfaces in a
>>> bindx call. I understand that
>>> if connect/listen/etc are called before a bind operation is done,  
>>> then
>>> a bind to an ephemeral
>>> port is done with all available interfaces added to the source  
>>> side of
>>> the association. I also
>>> understand that I can call bindx to add add/remove interfaces  
>>> from the
>>> source side of an
>>> association, but that the port in these calls must be the same  
>>> for all
>>> interfaces and be the
>>> same as any existing bound port for the association.
>> Right, and further more once you have bound all.. you really can't
>> unbind all (aka by using bindx.. at least I don't think you can ;D)
>>>
>>> My query concerns binding a subset of interfaces to an ephemeral  
>>> port.
>>> I can ask for ephemeral
>>> port assignment by specifying a port number of zero in a bind of a
>>> single interface - this does
>>> not seem to be documented in the RFCs but follows the TCP practise.
>> Yep that should work. The kernel will pick a random port for
>> you.
>>> The Linux kernel implementation
>>> behaves in thie way.
>> So does BSD.
>> But if I call bindx with more than one interface
>>> all with port set to zero,
>>> the implementation assigns an ephemeral port to the first interface
>>> and then returns an error
>>> condition as the socket is now bound to this ephemeral port and the
>>> later interfaces have port zero,
>>> a different value.
>> Ahh.. this sound like a bug.. and I bet that BSD has the same
>> one. Basically when you call bindx with multiple interfaces
>> it usually is a socket opt (setsockopt()) underneath. Then,
>> after you have bound the port, it calls successively to
>> bindx each additional port. So that what is happening
>> is it sees an error, in that you are using a different
>> port (or trying to).
>> I will check the BSD code.. I bet the same bug
>> exists.. and I will fix it.. Linux, if they have
>> this bug, should do the same..
>>>
>>> Why do this? e.g. laptop with ethernet, wifi and wimax. In Aus,  
>>> wimax
>>> is horribly expensive, so
>>> using this should be avoided by user applications where possible.  
>>> So a
>>> client would only expose
>>> the ethernet and wifi interfaces when connecting to a service if  
>>> possible.
>> Thats find.. it SHOULD work.. I think you have just
>> discovered a bug (and like I said I bet its in my implementation
>> too :-D)
>>>
>>> Workaround: call bind with a single interface to bind to an  
>>> ephemeral
>>> port and then bindx with the
>>> remaining interfaces all set to this port value.
>>>
>>> Specification change? Can I propose the following addition to the  
>>> spec:
>>>
>>> "If bindx is called and all ports are set to zero, then if the  
>>> socket
>>> is currently bound, the ports
>>> will be set to the bound port value. If the socket currently is
>>> unbound, then the socket will be
>>> bound to the same ephemeral value for all the socket addresses."
>> This seems reasonable.. and is a clarification of how it
>> SHOULD be working :-D
>>>
>>> Note 1: This will involve a change to the Linux implementation at
>>> least. But it is a small change.
>> and BSD too I think ..
>>>
>>> Note 2: I ignore the possibility of the bindx call containing a
>>> mixture of zero ports and a single
>>> non-zero value to bind to. This is possible, but just messy.
>> And may fail if the first port is 0, since then you pick
>> a different port than those in the list.
>> R
>>>
>>> Jan
>>> -- 
>>>
>>> On 4/3/07, Randall Stewart <[email-masked]> wrote:
>>>> Jan Newmarch wrote:
>>>> > Hi Randall
>>>> >
>>>> > I've been trynig and failing to join the sctp-impl mailing  
>>>> list - I
>>>> > just get no reply from [email-masked] and can't post. In the
>>>> > meantime, I have a query which maybe you could comment on or  
>>>> perhaps
>>>> > just forward it to the list.
>>>> >
>>>>
>>>>
>>>> Hmm thats strange..
>>>>
>>>> Let me join you.. which email do you want the to or cc?
>>>>
>>>> R
>>>> > Cheers
>>>> >
>>>> > Jan
>>>> > --
>>>>
>>>> Randall Stewart
>>>> NSSTG - Cisco Systems Inc.
>>>> 803-345-0369 <or> 803-317-4952 (cell)
>>>>
>>>
>
>
> -- 
> Randall Stewart
> NSSTG - Cisco Systems Inc.
> 803-345-0369 <or> 803-317-4952 (cell)
>

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 08:14:48 -0400

> from the user (x,y) which are wrong without complaining, just

Yes.. I thought about that.. and it of course
depends... I can see someone really specifying
multiple ports in a bindx and thinking they could
do this.

On the other hand, I can see people just forgetting
to 0 it out.. and leaving garbage in there..

Thus I think we did it that way in the "be liberal
in what you accept tradition".. but it may be wrong :-D
> 1. allow bindx() after bind() or without bind().
ok (we do this today as you know and I think linux does too
     from Kacheong's email Solaris does NOT).

>    same non-zero port number in all bindx() calls.
ok

>    or will use the already known one if the socket

Ok

Kacheong, what do you think?

R

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 09:18:13 -0400

>     from Kacheong's email Solaris does NOT).

Hmm..  well, linux doesn't do bindx without bind yet.  This was
kicked around a while ago on both lksctp and sctp-impl lists, and I
think the decision was to not allow that particular combination.

I do think that it would be a useful relaxation of the rule.

>>    same non-zero port number in all bindx() calls.

agree

>>    or will use the already known one if the socket

also agree.

-vlad

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 09:57:29 -0400

> I do think that it would be a useful relaxation of the rule.

Oops... I looked more closely, and it looks like this was added
after all.

I think some clarifications in the draft would be useful.

-vlad

---

From: Kacheong Poon <[email-masked]>
Date: Thu, 05 Apr 2007 00:55:35 +0800

> in what you accept tradition".. but it may be wrong :-D

Thinking as an app writer, I guess it is easier if a field
is not required, then it should be ignored.  What Randy
wrote originally is easier.

>     from Kacheong's email Solaris does NOT).

If this is allowed, then I guess the first element of
the sctp_bindx() address array becomes the significant
one.  This needs to be noted.

>>    same non-zero port number in all bindx() calls.

I assume the last sentence means that the caller can
just use 0 as the port number after the first address
element.  For example, the address array of sctp_bindx()
has two elements.  The first element has port number X.  It
means that the stack will try to bind to X.  The second
element can have a port number 0.  It is not required to
have X also.  This is probably implied in (3) below.

-- 

						K. Poon.
						[email-masked]

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 14:00:46 -0400

> have X also.  This is probably implied in (3) below.

Yep.. exactly..

R

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Apr 2007 20:55:33 +0200

Randy,

a comment in-line.

Best regards
Michael

>>> to 0 it out.. and leaving garbage in there..
which also applies to the family field or length field...
This might result in a bug which sometimes has effects and
sometimes not. So I would like to provide the app writer
the information that he did something wrong...
>>> in what you accept tradition".. but it may be wrong :-D
I like that tradition, but I think "we work around your bug
most of the time" is not that helpful.

If we would say, that we ignore the the port numbers in the
bindx() at all, that might be OK...

On the other hand people are used to use port number 0 as
chose what you want, port number != 0 as use this one.
That was the reasoning to suggest the three rules...
>> have X also.  This is probably implied in (3) below.
Yes. You can have (only the port numbers are written):
0, 0, 0, 0, 0, 0 -> OK.
p, 0, p, 0, p, 0 -> OK.
p, p, p, p, p, p -> OK.
p, q (with p > 0, p!=q) -> not OK.
I would always fail the call for
0, p (p > 0)
even if the kernel choses by random the port number p...

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 15:25:59 -0400

Michael:

I am fine with that.. as long as Kacheong and Vlad agree :-D

R

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 15:33:01 -0400

> I am fine with that.. as long as Kacheong and Vlad agree :-D

Yes, I agree as well.

-vlad

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 15:42:45 -0400

One other question..

Should we change it to use a true array
of

union struct sctp_sockstore's

Or whatever we call them?

I did teach a class on the socket api and this was
a complaint .. the students would rather have a
array of something like this union type..  Its smaller
and more compact...

Or do we just try to do something more like Kacheong is
proposing.. and get a generic bindx/connectx?

R

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Apr 2007 22:15:56 +0200

On Apr 4, 2007, at 9:57 PM, Vlad Yasevich wrote:

> I think changing this API is a bit late.  It would be really hard
> to update everyone in a backward compatible way.
>
> Also, I don't think the union is more compact.  For a purely
> IPv4 usage, it would waste space.  Having a union simply makes
> it easily addressable array vs. a list (really) built as contiguous
> memory array.
I agree. Currently we use something like a "packed array". Any
"non-packed" array will be larger...

I understand the students: they want to write a simple loop,
not looking at the size of each element...
If they want, they can this right now by using mapped IPv4 addresses.
>
> At this point, I think perusing a generic standard bindx() is a
> better option, but that will probably have to deal with some
> specific sockaddr structures as well.
So you are suggesting new (different names) bindx() and connectx()  
calls or
what do you mean by "generic standard"?

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 15:57:24 -0400

-vlad

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 16:22:09 -0400

Michael Tuexen wrote:
> So you are suggesting new (different names) bindx() and connectx() calls or

I think Kacheong's idea of proposing a bindx() and connectx() API
for other socket types, not just SCTP, has merit, and I would
support that work faster then any changes to the sctp_bindx() or
sctp_connectx() api.

-vlad

---

From: Vlad Yasevich <[email-masked]>
Date: Wed, 04 Apr 2007 16:43:27 -0400

Michael Tuexen wrote:
> Which protocol could use connnectx()? Doesn't it require
> a transport protocol supporting multihoming. This would
> limit it to SCTP. bindx() however, could be used also by
> TCP and/or UDP.

I think the idea is to move the connect loop from the user app
into the API.  You pass the connectx() api a list of addresses
and it tries to connect to all the addresses you passed in.

This is just one scenario I can thing off.

-vlad

> 
> Best regards
> Michael
> 
> On Apr 4, 2007, at 10:22 PM, Vlad Yasevich wrote:
> 
>> Michael Tuexen wrote:
>>>>

---

From: Michael Tuexen <[email-masked]>
Date: Wed, 4 Apr 2007 22:30:34 +0200

Best regards
Michael

---

From: David Borman <[email-masked]>
Date: Wed, 4 Apr 2007 16:41:37 -0500

It depends on what semantics are assigned to it.  For example, as was  
stated, it could be used for TCP to have a series of choices for how  
to connect to the remote host, rather than doing a loop in user space  
trying to connect to each address one at a time.  I could even see it  
being an optimization for connection time, where a SYN gets sent to  
all the addresses at the same time; the first SYN/ACK back wins, and  
a RST is sent in response to any additional SYN/ACKs that come back.  
And if the list is only 1 long, then it should be obvious what TCP  
should do.

connectx() is for connecting to a multi-homed host, and it can make  
sense for initiating a TCP connection, just not in the same way as SCTP.

			-David Borman

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 19:39:59 -0400

> support that work faster then any changes to the sctp_bindx() or

I agree other than the 3 point clarification Michael made earlier

R

---

From: Randall Stewart <[email-masked]>
Date: Wed, 04 Apr 2007 19:41:23 -0400

> limit it to SCTP. bindx() however, could be used also by

Actually TCP could use it.. imagine I get back
a list of IP addresses from DNS. I do a TCP
connectx().

Then when the SYN times out multiple times
I can try an alternate. I only keep the single
IP address that gets connected.. but it saves
some.

There was some talk about this on TSV a while ago.

R

---

From: Rick Jones <[email-masked]>
Date: Wed, 04 Apr 2007 16:44:34 -0700

> IP address that gets connected.. but it saves

A leeeetle bit of application programming, but not really all that much? 
And if the application wanted to re-establish a session after a TCP 
connection failure, it would still want that initial list of IP's no?

Heck, if you are going to go ahead and give connectx() a list of IPs for 
TCP to try, why not just give it the hostname...

rick jones
mpg member of the peanut gallery

---

From: Janardhan Iyengar <[email-masked]>
Date: Wed, 04 Apr 2007 21:13:11 -0400

bindx() seems to make sense in the TCP/UDP context. I am still not sure 
about connectx() for TCP, but I know DCCP supports multihoming to some 
extent. Can connectx() be used for DCCP?

- jana

-- 
Janardhan R. Iyengar
Visiting Assistant Professor
Connecticut College
http://cs.conncoll.edu/iyengar/

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 5 Apr 2007 09:52:12 +0200

On Apr 5, 2007, at 1:44 AM, Rick Jones wrote:

> A leeeetle bit of application programming, but not really all that  
> much? And if the application wanted to re-establish a session after  
> a TCP connection failure, it would still want that initial list of  
> IP's no?
>
> Heck, if you are going to go ahead and give connectx() a list of  
> IPs for TCP to try, why not just give it the hostname...
The problem with hostname is that you have no indication in the DNS  
whether the IP addresses
belong to the same host or not. For SCTP this is crucial, all  
addresses must belong to the
same host.

So one one has to restrict the list of IP-addresses to belong to the  
same host.
Assume that you send out two SYSs, one to address A, one to address  
B. Now you
get back a RST from A. Do you fail the connectx() call?

---

From: Randall Stewart <[email-masked]>
Date: Thu, 05 Apr 2007 05:47:11 -0400

Michael Tuexen wrote:
>> much? And if the application wanted to re-establish a session after a 
>> TCP connection failure, it would still want that initial list of IP's no?
>>
>> Heck, if you are going to go ahead and give connectx() a list of IPs 
>> for TCP to try, why not just give it the hostname...
> belong to the same host or not. For SCTP this is crucial, all addresses 
> must belong to the
> same host.
> 
> Assume that you send out two SYSs, one to address A, one to address B. 
> Now you
Another problem with host names is the same one that has
made it so no one implements the "host name address" option
in SCTP. It is a pain to get the kernel to do a DNS resolution
from in the kernel since you end up doing a down up down
sort of dependency.

DNS is better left out of things like this IMO :-0

R

---

From: Michael Tuexen <[email-masked]>
Date: Thu, 5 Apr 2007 13:35:47 +0200

Randy,

yes, you are right.

But we stil have a semantic question:

Do the addresses listed in a connectx() call belong to the same host?

If yes, how do you react to a received RST segment in the TCP case.
For SCTP, we fail the sctp_connectx() call on reception of an ABORT.
This is due to the fact that in the SCTP case the host wants to
use *all* addresses of the peer. For TCP this is not possible, it
wants to use *one* of the addresses. It could even continue trying
addresses after for one of the addresses a RST segment has been
received. Therefore the restriction of all addresses belong to one
host is also not necessary for TCP... Haven't thought about
security implications for the TCP case.

Best regards
Michael

On Apr 5, 2007, at 11:47 AM, Randall Stewart wrote:

>>> A leeeetle bit of application programming, but not really all  
>>> that much? And if the application wanted to re-establish a  
>>> session after a TCP connection failure, it would still want that  
>>> initial list of IP's no?
>>>
>> The problem with hostname is that you have no indication in the  
>> DNS whether the IP addresses
>> So one one has to restrict the list of IP-addresses to belong to  
>> the same host.
>> Assume that you send out two SYSs, one to address A, one to  
>> address B. Now you

---

From: Vlad Yasevich <[email-masked]>
Date: Thu, 05 Apr 2007 12:18:48 -0600

> I agree other than the 3 point clarification Michael made earlier

Yes, sorry.  The context I was referring to here was changing the API 
itself.  I agree that the clarifications that Michael proposed are 
needed, but that don't change the API.

-vlad

---

From: Janardhan Iyengar <[email-masked]>
Date: Thu, 05 Apr 2007 20:16:02 -0400

Michael,

> Do the addresses listed in a connectx() call belong to the same host? 

I'm starting to see a semantic inconsistency that makes me uncomfortable 
here.

In the TCP case, it seems that it does not matter whether the addresses 
belong to the same host or not. So, connectx() can, in the TCP world, 
connect to one of the IP addresses that responds, does not matter which.

But in SCTP, it does. So, during address confirmation, if one of the 
peer's IP address happens to be on another host, an ABORT will result 
from the host not in the association, and the association drops.

Here's another related inconsistency:

In TCP connectx(), suppose that the IPs belonged to different hosts, and 
one IP address responds with a SYN/ACK and the other responds with an 
RST. And suppose that the RST is dropped in the network. Should the 
sender still continue with a connection to the IP that responded 
disregarding the one that sent an RST, or should the sender wait for ALL 
IPs to respond first?

(a) Seems strange that the sender would wait for all IPs to respond, 
since ultimately only one peer IP is required for the connection. So, it 
makes sense for the sender to NOT wait.

(b) On the other hand, not waiting becomes another semantic 
inconsistency. In SCTP, confirmation is required for all peer IP 
addresses. So, if any of the IPs returned an ABORT during confirmation, 
the association is dropped.

These inconsistencies in interpreting the same call make me uncomfortable.

- jana

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 6 Apr 2007 13:12:26 +0200

Jana,

comments in-line.

Best regards
Michael

On Apr 6, 2007, at 2:16 AM, Janardhan Iyengar wrote:

> I'm starting to see a semantic inconsistency that makes me  
> uncomfortable here.
>
> In the TCP case, it seems that it does not matter whether the  
> addresses belong to the same host or not. So, connectx() can, in  
> the TCP world, connect to one of the IP addresses that responds,  
> does not matter which.
>
> But in SCTP, it does. So, during address confirmation, if one of  
> the peer's IP address happens to be on another host, an ABORT will  
> result from the host not in the association, and the association  
> drops.
Yes, this is what I pointed out earlier in an e-mail... Maybe it did  
not get through.
> In TCP connectx(), suppose that the IPs belonged to different  
> hosts, and one IP address responds with a SYN/ACK and the other  
> responds with an RST. And suppose that the RST is dropped in the  
> network. Should the sender still continue with a connection to the  
> IP that responded disregarding the one that sent an RST, or should  
> the sender wait for ALL IPs to respond first?
>
> (a) Seems strange that the sender would wait for all IPs to  
> respond, since ultimately only one peer IP is required for the  
> connection. So, it makes sense for the sender to NOT wait.
>
> addresses. So, if any of the IPs returned an ABORT during  
> confirmation, the association is dropped.
>
>
Yes, basically for TCP it is "connect to one of these addresses" and  
for SCTP
it is "connect to all of these addresses". The same applies to bindx 
()...

Kacheong, what do you think? Is it a good idea to have a generic call  
with these
kind of differences in the semantics? I'm not sure. But I'm not an  
app writer...

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 06 Apr 2007 09:37:56 -0400

Michael,

> Yes, basically for TCP it is "connect to one of these addresses" and for 
> SCTP
> it is "connect to all of these addresses". The same applies to bindx()...

It seems to me that bindx() would be different. bind() allows binding to 
all local interfaces (srcIP = *) or to one of them (srcIP = chosenIP). 
bindx() allows a subset.

I see that there is an inconsistency in that SCTP allows _all_ of these 
addresses to be in an association, but TCP will allow only one of these. 
But that seems to be somewhat more benign than the inconsistency with 
connectx() in that an app programmer will know what to expect of TCP or 
SCTP with bindx(). What do you think?

- jana

---

From: Michael Tuexen <[email-masked]>
Date: Fri, 6 Apr 2007 15:57:16 +0200

Jana,

comment in-line.

Best regards
Michael

On Apr 6, 2007, at 3:37 PM, Janardhan Iyengar wrote:

> Michael,
>
>> Yes, basically for TCP it is "connect to one of these addresses"  
>> and for SCTP
> It seems to me that bindx() would be different. bind() allows  
> binding to all local interfaces (srcIP = *) or to one of them  
> (srcIP = chosenIP). bindx() allows a subset.
But it is still a difference for the the connections/associations.  
For connections on of the
bound addresses is used, for associations most of the addresses are  
used.
>
> I see that there is an inconsistency in that SCTP allows _all_ of  
> these addresses to be in an association, but TCP will allow only  
> one of these. But that seems to be somewhat more benign than the  
> inconsistency with connectx() in that an app programmer will know  
> what to expect of TCP or SCTP with bindx(). What do you think?
bindx() for SCTP is more complicated. Think about the cases where you  
call bindx() with
two address sets, same port number, and the address sets are not  
disjunct... Something
like SO_REUSEADDR/SO_REUSPORT is more complicated for SCTP than for  
TCP... And not all
cases are clear...

---

From: Janardhan Iyengar <[email-masked]>
Date: Fri, 06 Apr 2007 10:25:01 -0400

> But it is still a difference for the the connections/associations. For 
> connections on of the
> bound addresses is used, for associations most of the addresses are used.

Yup, I agree that is an inconsistency.

> like SO_REUSEADDR/SO_REUSPORT is more complicated for SCTP than for 

I see. I am not clear also on how much complication something like 
bindx() will add to the TCP API, in terms of implementation.

It seems to me that there's enough concern (and not enough motivation??) 
to hold off on something like this. But that's my take from what I've 
gathered so far.

- jana

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 19:08:19 +0800

> even if the kernel choses by random the port number p...

The last one requires the stack to keep a state
and I don't see why this is necessary.  In fact,
this will break an existing app.  For example,
an app does a bind(0) and getsockname(), then use
the port obtained in sctp_bindx().  This is correct
using the current semantics.  But with the last rule
above, it will fail.  I don't think we need to have
that rule.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 19:10:13 +0800

> I am fine with that.. as long as Kacheong and Vlad agree :-D

Sorry for the delay in response, it was holiday time :-)
Except for the last rule I mentioned in a previous mail,
I think Michael's new rules are good.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 19:13:12 +0800

> array of something like this union type..  Its smaller

I don't think we should change the prototype.  The
current rules are just clarifications on the existing
prototype.  Changing the argument is not a good idea
at this point.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 19:52:07 +0800

David Borman wrote:
> stated, it could be used for TCP to have a series of choices for how to
> connect to the remote host, rather than doing a loop in user space
> being an optimization for connection time, where a SYN gets sent to all
> the addresses at the same time; the first SYN/ACK back wins, and a RST
> is sent in response to any additional SYN/ACKs that come back. And if
> the list is only 1 long, then it should be obvious what TCP should do.
> 

Another usage which people have talked about is in IPv4/IPv6
transitioning.  Suppose a DNS query of a peer results in both
IPv4 and IPv6 addresses.  An app prefers IPv6 but somehow the
path to the peer does not quite work with IPv6.  It will take
some time before TCP times out the connection attempt.  But
if the stack knows that there is an IPv4 alternative, it can
be smarter to try the IPv4 address after the first RTO.

I agree that the important question is what semantics bindx()
and connectx() have.  And as with all other socket calls, it
will be different depending on the socket's address family and
protocol used.

-- 

						K. Poon.
						[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 10 Apr 2007 13:55:15 +0200

Kacheong,

comments in-line.

Best regards
Michael

> above, it will fail.  I don't think we need to have
I was thinking about the port number sequence in one
bindx() call.
On multiple calls you are right, of course. But if you
use 0, p in one call, the user has no chance in getting
the port number chosen. So most likely it is a bug in the
application. But if you think we should succeed the call
if the user guesses the port number right in advance, we
can do that. Running the code a second time will demonstrate
the bug.

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 19:58:59 +0800

> TCP to try, why not just give it the hostname...

Possibly.  But I suspect that this is better done by a
higher level function.  I think except name/service
resolution functions, all other socket calls do not deal
with such a "high level" concept as a hostname.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 23:45:33 +0800

> belong to the same host or not. For SCTP this is crucial, all addresses

Hmm, I think the current SCTP socket API draft does not have the
above restriction.  From the -14 version of the draft.

    The way the SCTP stack uses the list of addresses to set up
    the association is implementation dependent.  This function
    only specifies that the stack will try to make use of all the
    addresses in the list when needed.

    Note that the list of addresses passed in is only used for
    setting up the association.  It does not necessarily equal the
    set of addresses the peer uses for the resulting association.

An implementation is free to choose the following semantics.
It tries the list of addresses one by one.  It waits for two
RTOs before moving to the next one.  And when it moves to the
next one, it will forget about the old one.  So when there is a
much delayed reply coming from the old one, it will treat it as
out of the blue.  Using this semantics, the list of addresses does
not need to correspond to the same host.  The above is just a
simple example, I am not advocating that it is a good implementation.

Why do we need such a restriction on sctp_connectx()?  SCTP itself
exchanges the IP addresses used by an association.  There is no
need to tell the stack what to expect from the peer.  If an app
requires this kind of restriction, it can always find out what
the list of peer addresses is after the association is set up and
terminate the association if the list does not match what it expects.

> get back a RST from A. Do you fail the connectx() call?

If the semantics of the connectx() call is to try the whole list
if necessary to establish a connection, the call will not fail
after getting a RST from A.  It can continue to try B.  Are you
worried about the case that both A and B belong to the same host?

Let's consider a example.  Suppose for a reason, say load
balancing purpose, a server app writer decides to have two processes,
one with a socket bound to A and the other with a socket bound to
B.  A client can connect to either A or B to get the service.  Using
connectx(), a client can have A and B on the list of addresses to
try.  Suppose somehow the process bound to A crashes and the client's
stack tries A first.  So the client gets a RST from A.  Then the
client's stack tries B after getting the RST.  Is there a problem
with this?

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Tue, 10 Apr 2007 23:55:21 +0800

Michael Tuexen wrote:

> use *all* addresses of the peer. 

No, I don't think sctp_connectx() provides this kind of
semantics to the caller.  In fact, how can an app know
about which addresses the peer wants to use for an
association?  Why is this restriction necessary for an
SCTP app?

> For TCP this is not possible, it

I suppose there is a security implication for SCTP if
sctp_connectx() does not have the above restriction.
What is the issue?  So suppose an app does a for loop
on the list of addresses doing connect(), is there a
problem also?

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 11 Apr 2007 00:05:35 +0800

> from the host not in the association, and the association drops.

Isn't address confirmation done in the SCTP level?  It
has nothing to do with the sctp_connectx() call.  It
seems to me that the above is a suggestion that the
list of addresses given to sctp_connectx() must be
the same list the peer uses.  Why is this necessary
for the call?

> makes sense for the sender to NOT wait.

No, I don't think the wait is necessary.  But I suspect
that it can be implementation dependent.  I don't see
why it is necessary to require an implementation not
to wait.

> the association is dropped.

Given the current semantics as described in the SCTP socket
draft, I don't think the above is a requirement.

-- 

						K. Poon.
						[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 10 Apr 2007 18:48:35 +0200

Hi Kacheong,

sctp_connectx() was invented to held in the SIGTRAN scenario, where
you typically know all addresses of your peers in advance via  
configuration.
Alternatively RSerPool also provides multiple addresses of one host.
Using connnect() you can not make use of the peers multihoming. Using
sctp_connectx() you can use this right from the beginning.
Not sure if this is written, the sender also considers the addresses
listed in the sctp_connectx() call as confirmed.

If you receive an ABORT, response to the sctp_connectx(), you fail
the association.

Best regards
Michael

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 11 Apr 2007 01:24:19 +0800

> sctp_connectx() you can use this right from the beginning.

I still don't understand why the above implies the
restriction that the peer must use the same set of
addresses provided to the sctp_connectx() call.

Consider this example, an app knows that the peer has
three IP addresses A, B and C.  Using the connect() call,
the app can try A first and then if that fails, it will
try B and then C.  Suppose the peer's interface A is
somehow broken and the peer knows about it.  When the peer
gets the INIT to B, it replies with an INIT-ACK without
the address A because A is broken.  The association is setup
correctly and everything is working.

So if we have the restriction on sctp_connectx() that
the peer must use A, B, and C in the association, the
call will fail in this case.  I don't see why this is
necessary for SIGTRAN.

> listed in the sctp_connectx() call as confirmed.

I don't think it is a good assumption.  An address should
be confirmed in the SCTP layer, not in the sctp_connectx()
call layer.

-- 

						K. Poon.
						[email-masked]

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 10 Apr 2007 21:13:40 +0200

> addresses provided to the sctp_connectx() call.
The reason is the following:
Assume I know that Host1 has IP11 and IP12 and Host2 has
IP21 and IP22.

When I call connectx(IP11,IP12) and the application is not
running on Host1 (due to a failure) I get back an ABORT and
know that it does not make sense to retry IP11 or IP12. I
can report the failure and try to connectx(IP21,IP22).

I can consider the reception of an ABORT as a failure of
the application and can skip all related IP addresses.
> correctly and everything is working.
That is OK.
> call will fail in this case.  I don't see why this is
It can us a subset. But the problem is the ABORT handling
described above. If I do not know which IP addresses belong
to the same host, I must try all of them. But in SIGTRAN
I either know the grouping (and can use connectx()) or I
do not know it in which case I just use connect().
> be confirmed in the SCTP layer, not in the sctp_connectx()
The upper layer always provides some addresses of the peer.
But one could make this stuff optional...

---

From: Randall Stewart <[email-masked]>
Date: Wed, 11 Apr 2007 06:31:38 -0400

Kacheong:

One little note that neither adds to nor takes away
your argument..

All addresses that are listed by connect/connectx are
considered "confirmed" by the stack. This does not change
your scenario's .. but it is something to consider :-D

R

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 11 Apr 2007 18:30:52 +0800

> the application and can skip all related IP addresses.

As in the example I sent out in my previous mail, getting
a RST or ABORT does not necessarily imply that the peer app
is not running.  One can only make the above assumption if
the app knows exactly what the peer app must do, which means
that the app must be accepting INIT for the list of addresses.
It does not seem to be a general usage of sctp_connectx() and
I don't think connectx() should have this restrictions by
default.  If this kind of behavior is preferred by a certain
set of apps, connectx() may take a protocol specific flag
which indicates certain behavior.

BTW, the current specification of sctp_connectx() does not
have this restriction.  And it seems to me that if an app
expects this behavior, it will not be portable.

> do not know it in which case I just use connect().

Actually, the restriction is more than the knowledge of
all the addresses belonging to the same host.  The peer app
must also be accepting INIT for the whole list of addresses.

> But one could make this stuff optional...

Yes, the upper layer provides some peer addresses.  But
it does not mean that those addresses are automatically
confirmed.  The confirmation must be done in the SCTP
layer.

-- 

						K. Poon.
						[email-masked]

---

From: Randall Stewart <[email-masked]>
Date: Wed, 11 Apr 2007 06:38:22 -0400

> confirmed.  The confirmation must be done in the SCTP

I think I agree with all you say (i.e. no such restriction
that the addresses all are on the same host is in
connectx).. however.. I disagree with the last statement.
If a user passes SCTP some addresses... saying in effect
I would like to talk to one or some set of these
addresses IP1, IP2, IP3.. then these addresses
are considered "good" addresses and not an attacker..

Now if the INIT-ACK comes back WITHOUT  IP3.. thats
ok.. the association will come up...  it just will
NOT have IP3 in it.. However.. both IP1 and IP2 should
be considered confirmed.. and not require a confirmation
HB..

R

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 11 Apr 2007 19:32:20 +0800

> are considered "good" addresses and not an attacker..

But "good" does not imply "confirmed."  "Confirmed"
is a concept in the SCTP protocol.  It is not "clean"
to add this semantics to the call, IMHO.

> be considered confirmed.. and not require a confirmation

Is the purpose of forcing a confirmation just saving
the HB?  If this is the reason, I'd say not to have
this additional meaning.  The HB exchange serves more
than confirming an address.  SCTP can get some info,
say RTT and reachability from the exchange.

-- 

						K. Poon.
						[email-masked]

---

From: Kacheong Poon <[email-masked]>
Date: Wed, 11 Apr 2007 19:57:01 +0800

Jan Newmarch wrote:

> I'm just coming at this from an apps programmer viewpoint, and would
> seek clarity about what the spec should tell me.
> 
> For connectx, I think the spec is clear enough  that all the addresses
> MUST belong to the same endpoint. If they do not, the application may
> behave in unpredictable ways. If I put in a bogus address then the app
> may work ok or may fail (and it's my fault).

Yes, I think this was the original idea.  Note that the
sctp_connectx() call itself does not and probably cannot
check whether the list of addresses actually belong to
the same host.  It may assume that.  But it seems that
people are saying that if there is no such assumption,
the connectx() (not sctp_connectx() which I assume is what
is referred above) can do more.  For example, if an app
only cares about getting some service from a list of servers,
giving the list of servers to connectx() is quite handy.

So the question here is whether the stack should assume the
list of addresses belong to the same peer or not.  I guess
a flag, as mentioned in a previous mail, may be used for
this.  And whether the stack makes use of this assumption
or not is probably implementation and protocol dependent.

> However, it goes on to say "Note that the list of addresses passed in
> is only used for setting up the association.  It does not necessarily
> equal the set of addresses  the peer uses for the resulting
> association.  If the caller wants to find out the set of peer
> addresses, it must use sctp_getpaddrs() to retrieve them after the
> association has been set up."
> 
> a) Could this be tightened up at all?
> b) Not the right forum, I know, but Linux getpaddrs() seems to have a bug
> 
> For (a) what would make sense to me would be that either
> - the set of peer addresses is the intersection of the set requested
> by connectx and the set offered by the peer (either by bind or bindx);

I don't understand this.  The set of peer addresses is
controlled by the peer.  What the app uses in the sctp_connectx()
call does not matter.  In fact, the peer does not know what
the app passes to sctp_connectx().  There is no way to have
this meaning.

> or
> - the set of peer addresses is the set offered by the peer,
> irrespective of the set requested by the client.

This is what is meant in the current draft.  The SCTP protocol
does not have the provision to force a peer to use certain set
of addresses.

> Whatever, I assume that there must be a non-zero intersection between
> the addresses in connectx and bindx. Must the set of connectx
> addresses be a subset of the addresses on the peer, or a subset of the
> addresses bound to by the peer? I'm sometimes seeing a connection to a
> peer address not in the connectx list.

I think the caller of sctp_connectx() can only assume that one
of the addresses passed in is also used by the peer for the
association, otherwise the association cannot be set up at
all :-)  This means that an app can only assume that the
intersection of the set passes to sctp_connectx() and the set
of peer addresses used for the association is not empty.

-- 

						K. Poon.
						[email-masked]

---

From: "Jan Newmarch" <[email-masked]>
Date: Wed, 11 Apr 2007 21:01:33 +1000

Hi

b) What I am also seeing from the Linux implementation is that
getpaddrs just returns the set I asked for in connectx irrespective of
what may have been set up (including any bogus addresses I put in
connectx).

Cheers

Jan
--

---

From: Randall Stewart <[email-masked]>
Date: Wed, 11 Apr 2007 12:42:44 -0400

> to add this semantics to the call, IMHO.

Confirmed means the peer is not listing addresses
in its INIT/INIT-ACK that it does not own. Are you saying
that what the user lists might be an attack address, an
attack on itself?

I don't get this..

> say RTT and reachability from the exchange.

There may be other addresses to confirm besides the ones
given by connectx() example:

connectx(IP1, IP2, IP3)--->
                        -------INIT(IPA)--> IP1
                        IPA <---INIT-ACK(IP1, IP2, IP4, IP5, IP6, IP7)

Now IP4 - 7 NEED to be confirmed BEFORE you can use them..

IP3 gets dropped.

Why should IP2 take up time in the confirmation process.

If who you connected to is "evil" then IP4 -7 may not
belong to them.. so it is best to HB them asap.. at only
Max.Hb.Burst set to 1.. it delays possibly by 1 second
the "true unknown" addresses that were NOT listed in connectx()

Yes, we can use the HB to measure the RTT.. and that is
good.. but this is the "normal" HB that can work just as
well..

R