Subject: RE: [Tsvwg] End point and association Address conflict

From: Tuexen Michael <[email-masked]>
Date: Mon, 29 Apr 2002 10:50:24 +0200

Dear all,

looking at the discussion it seems that we have only one restriction 
for add-IP:

Consider the set E of all SCTP end points at a host. Partition
E = E_1 union E_2 union ... E_65535
into disjoint sets E_P of all end points having port P.

When adding an IP-address to multiple end points, all of these
end points MUST belong to different E_P.

This means that an SCTP implementation has to choose from all end points
having the same port on end point to which the add IP procedure is applied.

Another point:
Consider two associations
A1: E1 <-> E2  (The SCTP implementation of E2 is AddIP capable.)
A2: E1 <-> E3  (The SCTP implementation of E3 is not AddIP capable.)

Now you add an IP address to E1 and get E1'. So A1 is now E1' <-> E2.
Then we consider A2 also as A2 : E1' <-> E3, but the stack will never
use the added address. Is this correct usage of terminology?

Best regards
Michael

Michael Tuexen    Tel.:   +49 89 722 47210
Siemens AG        Fax:    +49 89 722 48212
ICN WN CC SE 7    E-mail: [email-masked]

> -----Original Message-----
> From:	Qiaobing Xie [SMTP:[email-masked]]
> Sent:	Friday, April 26, 2002 2:45 AM
> To:	[email-masked]
> Cc:	[email-masked]; [email-masked]; Qiaobing Xie; [email-masked]; [email-masked]
> Subject:	Re: [Tsvwg] End point and association Address conflict
> 
> [email-masked] wrote:
> ....
> >         Now I have another doubt... Imagine that we have an endpoint 
> > E1 with addresses IPa, IPb and port 1. We have an association A1 
> > that only uses IPa, whose peer is IPz and port 65535. Thus the 
> > association A1 can be represented as [(IPa; Port1), (IPz; Port 65535)]. 
> > What happens now if we receive an INIT from IPz/65535 directed to IPb/1?
> > 
> >         As IPb is not being used in A1, we should answer with an 
> > INIT ACK as in a normal establishment of an association. However, 
> > when you receive an INIT chunk, you include all the addresses of the endpoint
>                                       ^^^^^^^^^^^^^
> Here your "..include all the addr..." directly violated the "under no
> circumstance can an SCTP transport address ever be present in more than
> one SCTP endpoint" rule (see last the paragraph, page 23 of the SCTP
> book).
> 
>         E1               E2
> A1 = [(IPa; Port1), (IPz; Port 65535)]
> 
>         E3                 E2
> A2 = [(IPa, IPb; Port1), (IPz; Port 65535)]
> 
> And you have IPa/Port1 present in both E1 and E3 (they are TWO different
> endpoints by definition!). This is prohibited by the SCTP association
> definition.
> 
> -Qiaobing
> 
> > in the INIT ACK chunk... But we have to be careful, because in
> > this case, when receiving the COOKIE ECHO, we have two options:
> 
> 
> _______________________________________________
> tsvwg mailing list
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/tsvwg

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 29 Apr 2002 12:40:11 -0500

Hi, Michael,

I think two simple rules will be sufficient. Let me use an example to
illustrate them.

Assume we have the following endpoints and associations in place:

on local host         on remote host(s)
A1: E1 <-----------------> E3 
A2: E1 <---------------------> E4
A3: E2 <-------------------------> E5

Rule 1: whenever adding an SCTP transport to a local endpoint, make sure
this same transport is NOT already present in other existing endpoints
(e.g., you don't want to add one of E1's transport to E2).

Rule 2: when adding a transport to a local endpoint that has
associations to multiple peers (ie, a fan-out config), one MUST send an
add-ip ASCONF to each of those peers. And the new transport can only be
used until ALL the peers ack successfully. If anyone of the peers denies
the add-ip then you can't add it. So, if you add a new transport to E1
above, you have to send ASCONF to both E3 and E4 and have to get a
positive ack from both of them.

-Qiaobing

---

From: Randall Stewart <[email-masked]>
Date: Mon, 29 Apr 2002 13:41:55 -0500

> positive ack from both of them.
Q:

I don't agree with this rule.. just because a peer in a UDP model
endpoint on one of its assocaitions is NOT capable of add-ip why
should the rest be stuck. Yes you have an abnormality here in
that that peer COULD send a INIT to the new address.. again I repeat:
1) This is such a corner case it is almost silly.
2) If it happens it is not that hard to handle... Looking at
   the code Peter and I wrote you would end up finding the endpoint
   (since the address WAS added there) and you would then LOOK at
   its associations. This would then appear to be a RESTART where the
   peer added addresses. You would then SEND IT an ABORT saying restart
   while adding addresses. This would leave the other association
un-touched
   and would elimiate the error on the peers part... 

R

-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 29 Apr 2002 14:54:46 -0500

Randall,

>    and would elimiate the error on the peers part... 

I disagree with Rule 2 as well, but for different reasons: there may be
other reasons for not wanting to add the IP to an existing association
with, say, E1.

--brian

-- 
Brian F. G. Bidulock    ¦ The reasonable man adapts himself to the ¦
[email-masked]    ¦ world; the unreasonable one persists in  ¦
http://www.openss7.org/ ¦ trying  to adapt the  world  to himself. ¦
                        ¦ Therefore  all  progress  depends on the ¦
                        ¦ unreasonable man. -- George Bernard Shaw ¦

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 29 Apr 2002 22:06:01 +0200

Hi Qiaobing,

see my comments below.

Best regards
Michael

> positive ack from both of them.
I do NOT like this rule. How do you handle the case if E3 and E4 support 
AddIP, but
E5 does not. Do you say that you can not add an address to E1 because 
one of the peers
does not support AddIP? for deployment this means that AddIP only be 
used if all
implementations support this...
Why not add it to E1, but it is not allowed to be used as a source 
address for A5.
>>> Cc:   [email-masked]; [email-masked]; Qiaobing 
>>> Xie; [email-masked]; [email-masked]
[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 29 Apr 2002 15:11:43 -0500

...
>    un-touched and would elimiate the error on the peers part...

Randy,

Let's walk through this example in detail:
Say, local endpoint E1 has two associations to different peers,

     E1             A1     E2
[IPA,IPB:Port1] <-------> [...] 
     E1             A2     E3
[IPA,IPB:Port1] <-------> [IPZ:Port2] 

We do add-ip (IPC) to E1. After sending ASCONF to both E2 and E3, we
succeeded with E2 but failed on E3.

     E1'               A1'     E2
[IPA,IPB,IPC:Port1] <-------> [...] 
     E1'               A2'     E3
[IPA,IPB,IPC:Port1] <-------> [IPZ:Port2]  /* E3 rejected add-ip and */
                                      /* hence knows nothg about IPC */

My understanding is that you would like to allow this. 

Now, several things could happen,

1) E3 may get a packet from E1' with src addr=IPC. This packet will
becomes an OOTB to E3 and E3 will send back an ABORT to kill the
ill-fated A2'. This is ok.

2) E3 may send an INIT to IPC/Port1. Is this the corner case you
described above? If so, I can see E1'1 will find the association A2' and
take this E3 INIT as a restart, but I don't see why E1' will consider it
as a "restart with added address" (E3 didn't add any address, it still
uses only IPZ).

-Qiaobing

---

From: Randall Stewart <[email-masked]>
Date: Mon, 29 Apr 2002 15:57:55 -0500

> ill-fated A2'. This is ok.

Why would E1' use IPC... in this case it knows that it is NOT
part of the association. So it MUST NEVER source things from
IPC as long as this holds true .. no problem.. only if
E1's implementation is so bad it sources from IPC that a
problem will occur.

> as a "restart with added address" (E3 didn't add any address, it still

Yes.. this is a very corner case.. 

 A) E3 must be very confused to want two associations.. so in
    my mind this is a corner case.

 B) We would need to expand the rules that you must also check
    the your addresses in a COOKIE-ECHO to see if you have
    hit such a collision.. if so send an ABORT... 

I don't see this is a problem.. ABORTING your funny association
with E3 should leave A2 alone and all should be well.

It seems to me we need to add this new rule for ADD-IP.

R

---

From: Randall Stewart <[email-masked]>
Date: Mon, 29 Apr 2002 19:22:20 -0500

Qiaobing Xie wrote:
> > A3: E1 <-------------------------> E5
> ...
> > AddIP, but E5 does not. Do you say that you can not add an address to E1
> > because one of the peers does not support AddIP? for deployment this means
> > that AddIP only be used if all implementations support this...
> 
> An alternative is to find a way to abort E5 and force it to restart with
> E1. Maybe we should introduce an ABORT(error code=please restart with
> me).
> 
> This will split E1 into two logical endpoints: E1-with-new-addr, and
> E1-without-new-addr. And the two logically different endpoints will end
> up sharing all original addresses. This breaks the basic RFC2309
> definition/rule that says:
>     "A transport address used by an SCTP endpoint must not be used
>      by another SCTP endpoint.  In other words, a transport address
>      is unique to an SCTP endpoint."

I think we are still out of wack on definition of an endpoint
here. I don't see how the associations an endpoint E1 has can
split E1 into two logical endpoints.. it is STILL E1 and it
just has an association amongst its three ones that can't use
one of its addresses. This is NO different than E1 having
bound ALL IPv6 addresses Link local, Site Local and Global and
then setting up an association with a peer that is NOT on its
Site and NOT on its link.. and at the same time setting up an
association with a peer on the local link that is ALSO in the
same site. So you end up with:

<Endpoint Port:2222 IPv6G, IPv6S, IPV6LL>

E1(Port:2222 IPv6G)<------A1------------>PeerEX(Port:3333 IPv6G)
and
E1(Port:2222 IPv6LL, IPv6S, IPv6G)<------A2------->PeerIN(Port:4444
                                                   IPv6G,IPv6S,IPvLL)

Is E1 split by this... NO!!! 

You just have the case that E1 has different addresses active on one
association versus another... this is no different than

<Endpoint IP-X:Port 2222>
E1(Port:2222 IP-X)<-------------A10---------------->Peer1(Port:3333
IP-A)
E1(Port:2222 IP-X)<-------------A11---------------->Peer2(Port:4444
IP-B)

Now if I do a Add of IP-Y to the endpoint and
then find that peer1 does asconf and peer2 does not I end up with:

we have

<Endpoint IP-X,IP-Y: Port 2222>
E1(Port:2222 IP-X,IP-Y)<-----------A10--------------->Peer1(Port:3333
IP-A)
E1(Port:2222 IP-X)<----------------A11--------------->Peer2(Port:4444
IP-B)

A11 just is using a subset of the addresses in the same way the 
IPv6 A1 did the same thing... 

This did NOT magically make E1 into E1', Endpoint 1 is still Endpoint 1
It owns the addresses it has bound .. period...

When you attempt to bind IP-Y to the endpoint for port 2222 you must
verify that NO OTHER endpoint has it bound.. as long as that is true..
No problem... You add that one to the endpoint... and if some
association can only use a subset of the addresses so be it...

Now as to the corner case.. I think I have shown that this is NOT
a problem... proper coding will take care of this issue..

Do we need some documentation somewhere? Maybe...

R

---

From: Qiaobing Xie <[email-masked]>
Date: Mon, 29 Apr 2002 19:03:04 -0500

Hi, Michael,

...

-Qiaobing

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 29 Apr 2002 19:50:39 -0500

Randall,

[snip]
> verify that NO OTHER endpoint has it bound.. as long as that is true..

There is no such restriction in TCP.  It is quite permissible (and
falls under the category of local port reuse) to have the following
binding:

    E1 [IPA/PortX] ---
    E2 [IPA/PortX] ---

In fact these may start off life as:

    E1 [IPA/Port0] --
    E2 [IPA/Port0] --

And when they connect, they autobind with local port reuse and are
permitted to become:

    E1 [IPA/PortX] ------------------  [IPB/PortY]
    E2 [IPA/PortX] ------------------  [IPB/PortZ]

(These ports were never listening.)

Why does SCTP require that this not happen when it is not only
permissible but a common port reuse strategy in TCP?

--brian

---

From: Randall Stewart <[email-masked]>
Date: Mon, 29 Apr 2002 20:59:17 -0500

> falls under the category of local port reuse) to have the following
Oh yes there is.. try doing this

1) open a socket(tcp)
2) bind IP-A:port 2222 to it
3) open a new socket(tcp)
4) bind IP-A:port 2222 to it.

It will not work.. sorry 

> And when they connect, they autobind with local port reuse and are

I am not talking about the ephemeral case here.. Yes I agree
a ENDPOINT will spawn off multiple connected sockets reusing the
port number and the single IP address it has (for TCP that is).

This is not a problem... And I don't see it is a problem with
SCTP either.. An accepted assocaition (on another socket) is
an association under that original listening endpoint. I think
this is what is confusing Qiaobing. I don't see the new socket
as being a seperate endpoint.. it is indirectly associated
with the listener... If another INIT arrives it is NOT
the connected socket that will get the INIT but the 
listner... the listner WILL have to look up and find the connected
socket ... for the collision handling.. but the endpoint that
handles the INIT will be the one listening..

> (These ports were never listening.)

By the way I don't see the above two as seperate endpoints. The
endpoint to me is represented by the listener.. these are
not endpoints but associations of that listening endpoint.

> permissible but a common port reuse strategy in TCP?

I don't see that it is wrong.. and I think you are misreading
me...

What must be prevented however is
 
E1 [IPA/PortX] ---------A1------------- [IPB/PortZ]
E2 [IPA/PortX] ---------A2------------- [IPB/PortZ]

So:
MachineA has [IPA/IPB]                         Machine B has [IPZ]
E1 [IPA/Port 1000] <---------------A1---------> [IPZ/Port 2000]
and
E2 [IPB/Port 1000] <---------------A2---------> [IPZ/Port 2000]

Now if I add IP-C to machine A

Only one of these guys can have IP-C added...

E1 or E2....<the listner and its spawned associations>

Pick whichever one you want..

Otherwise you end up with

E1 [IPA,IPC/Port 1000]<--------------A1----->[IPZ/Port 2000]
and
E2 [IPA,IPC/Port 1000]<--------------A2----->[IPZ/Port 2000]

and when a packet comes along to the kernel of machine a:
           <----[To:IPC port 1000 From IPZ port 2000]----
or
        ----[From IPC/Port 1000 To IPZ port 2000]--------->

You have NO way to distingush which of the associations the
packets belong to..

I guess it all comes down to the definition of endpoint:

To me an Endpoint is either the connect() or listening() socket
and any associations it spawns. In another view the one that
binds the port. 

If I just do a implicit socket/connect for example.. in the process
of doing the connect an implicit bind will happen of a ephemeral
port and at that point that socket becomes an SCTP endpoint
bound to that port. Any associations it spawns under that
by doing connect() or closing it and changing to a listen
are part of that endpoint... 

If I do a socket/bind/listen or a socket/<bind>/connect this
makes me an endpoint.

For the UDP model I do a socket/bind/listen as well or a 
socket/connect... same result... the socket becomes an endpoint.

At that point associations underneath the endpoint (ones
created by implicit connection setup/ connect() calls/ 
accept() calls or even sctp_peel_offs are STILL part of that
endpoint.. and can use all or a sub-set of the addresses
of the endpoint. When we add an address to a endpoint it
must be to the base endpoint... i.e. the one that
has the binding. This then authorizes all of the sub-servant 
associations of the endpoint to also do an ADD-IP if they wish.

Thats how I see it anyway...

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 29 Apr 2002 23:48:27 -0500

Randall,

Please see comments below...

> 4) bind IP-A:port 2222 to it.

Err, yes it does, if you set a standard socket option...

Compile and run the following:

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <errno.h>
#include <netinet/in.h>
#include <netdb.h>
#include <string.h>

main()
{
	int s1, s2, flag = 1;
	struct sockaddr_in addr1 = { AF_INET, 0, { 0 } };
	struct hostent *haddr;
	haddr = gethostbyname("127.0.0.1");
	addr1.sin_port = htons(10000);
	addr1.sin_addr.s_addr = *(uint32_t *)haddr->h_addr;
	if ((s1 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0 ) {
		perror("test);
		exit(2);
	}
	if ((s2 = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP)) < 0 ) {
		perror("test);
		exit(2);
	}
	if (setsockopt(s1, SOL_SOCKET, SO_REUSEADDR, &flag, sizeof(flag)) < 0 ) {
		perror("test);
		exit(2);
	}
	if (setsockopt(s2, SOL_SOCKET, SO_REUSEADDR, &flag, sizeof(flag)) < 0 ) {
		perror("test);
		exit(2);
	}
	if (bind(s1, (struct sockaddr *)&addr1, sizeof(addr1)) < 0) {
		perror("test);
		exit(2);
	}
	if (bind(s2, (struct sockaddr *)&addr1, sizeof(addr1)) < 0) {
		perror("test);
		exit(2);
	}
	printf("Q.E.D.\n");
}

Output:

#> gcc -o test test.c

#> ./test
Q.E.D.

Now, is there any reason why I should not be able to do the same
for SCTP?  Also, why should the system stop me from overlapping
address lists with the same port number?  TCP doesn't stop me...

It is only once the address is fully formed that this overlap is
checked for TCP.  If you attempt to connect the second socket to
the same destination transport address (IP and port) as the first
one has connected, it will give you an error.  Again, why should
SCTP be any different?

Therefore, it is quite rational to have for SCTP:

   E1 [IPB, IPA port 10000] ---
   E2 [IPC, IPA port 10000] ---

regardless of the statements on Page 13 of the RFC.  The "MUST NOT"
still stands in that these two endpoints cannot connect to the same
destination transport address because they share IPA port 10000.
But there is nothing to stop me from binding and defining an endpoint
in this way except a rather off-base statement that does not consider
the longstanding use of SO_REUSEADDR.

--brian

---

From: Randall Stewart <[email-masked]>
Date: Tue, 30 Apr 2002 07:07:15 -0500

Brian:

A comment for you and a question please see below:

> Err, yes it does, if you set a standard socket option...

....Code deleted....

> the longstanding use of SO_REUSEADDR.

Note I did NOT say I added any special socket option... If you
take out the additional option it will NOT suceed.

The reason this CAN work for TCP is that the peer connecting
is a single endpoint... Great care must be taken when using
this socket option... that is why it is an additional option
and NOT default behavior. TCP would get in the same trouble
if both sides of a connection did the above and both sides
connected to each other... you would end up with

[IP-A Port 1000]------TCP connection 1------[IP-B Port 2000]
[IP-A Port 1000]------TCP connection 2------[IP-B Port 2000]

Which is a fundamental violation of the definition of a TCP
connection. I bet this is why it is an option.. you have to
specifically set this up in order to have it occur (and
frankly I think it would confuse most stacks.. data would
not go to the correct places if things would function at all).
The base socket system if used without auxillary options will
NOT allow you to do the above...  Going back to my early
email..

...But fundamentally you have NOT answered my question below.
How will you tell which endpoint gets which packet? 

Please answer the question below...

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Apr 2002 13:16:18 -0500

Randall,

> [IP-A Port 1000]------TCP connection 2------[IP-B Port 2000]

The standard error EADDRINUSE would be returned by
second connect() call.   It takes no big effort: in
a NET4 implementation a check is made that the connection
does not already exist in the established hashes.

> How will you tell which endpoint gets which packet? 

These endpoints are not listening.  When they engage in fully
formed connections the endpoint is determined from the established
hashes which requires a match on vtag, source address, source port,
destination address and destination port.  As long as this set
uniquely identifies an endpoint (and listen() and connect() will
make sure that this is true) there is never any problem, just
as there is never any problem for TCP above.

--brian

---

From: Randall Stewart <[email-masked]>
Date: Tue, 30 Apr 2002 13:26:50 -0500

> as there is never any problem for TCP above.

Vtags have never been defined as part of the identification
of an SCTP association. They are used in place of the
pseudo header and as a defense against blind attacks.. not
as part of association discrimination... Sorry this won't work.
Two associations are not allowed in this case.. just like
your TCP description would return a INUSE error...

One can easily prevent the TCP case.. i.e. at the connect
check the hash.... I guess the ADD-IP case also can
be prevented when one tries to add the IP-C to the second
association you would simply get the same error :>

Thinking about it.. we do this now.. since when you attempt
the sctp_bindx you would get such an error :>

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 30 Apr 2002 15:19:26 -0500

Randall,

The mistake is returning the error on bind when SO_REUSEADDR is
set.  EADDRINUSE is returned on the connect() instead.  When
ADD-IP goes to add an IP address to an existing association, it
can check for address conflicts in the established hashes and

> as part of association discrimination... Sorry this won't work.

I use vtag for fast hash lookup to find the entries to make the
daddr/dport/saddr/sport match for established associations for
all chunks not containing a vtag (vtag == 0) or without the T-bit
set.  When the T-bit is set I use the peer tag on a separate peer
tag hash.  For INIT chunks I check the listening hash.

> your TCP description would return a INUSE error...

listen() and connect() ensure that there are no conflicts in the
hashes. (not bind()).  There never results two associations between
the same subset of destination transport addresses per the "MUST NOT"
on Page 13.  It is the statement related to bind (naming an endpoint)
which is incorrect.

> the sctp_bindx you would get such an error :>

This is the wrong place because of SO_REUSEADDR.  Bind should
permit reuse of local addresses.  It is connect() and listen()
that ensure that no conflicts occur.

--brian

---

From: Randall Stewart <[email-masked]>
Date: Tue, 30 Apr 2002 15:32:08 -0500

> tag hash.  For INIT chunks I check the listening hash.

Vtag may be used for fast lookup but it is NOT a association
discriminator. 

> on Page 13.  It is the statement related to bind (naming an endpoint)

Not true when the SO_REUSE flag is NOT set...

> that ensure that no conflicts occur.

Sorry you must use it to filter out such events if you
are going to use it to trigger the ASCONF.... If you want to
NOT use it to trigger the ASCONF and use something else.. then
fine.. you can use that "new-call()" that would generate the
ASCONF to return the error...

R