Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt

From: Michael Tüxen <[email-masked]>
Date: Fri, 8 Apr 2011 21:07:30 +0200

Hi Alan,

sorry for the late response...

See my comments in-line.

Best regards
Michael

On Mar 30, 2011, at 1:27 PM, Ford, Alan wrote:
> Hi Michael,
> 
> Many thanks for your review. Comments inline...
> 
>> -----Original Message-----
>> From: [email-masked] [mailto:multipathtcp-
>> [email-masked]] On Behalf Of Michael Tüxen
>> Sent: 30 March 2011 12:04
>> To: mptcp
>> Subject: Re: [multipathtcp] I-D Action:draft-ietf-mptcp-api-01.txt
>> 
>> Dear all,
>> 
>> I read the ID and have the following comments:
>> 
>> * I'm not sure why you need REQ4. In SCTP we do have this for
>>  1-to-many style sockets, but I do not see that you use this
>>  for MPTCP. On 1-to-1 style sockets, it is not needed for
>>  SCTP at least.
> 
> The logic for REQ4 (connection ID) is explained in the text below that list:
> 
>   Finally, it should be possible for the
>   application to retrieve a unique connection identifier (local to the
>   endpoint on which it is running) for the MPTCP connection.  This is
>   equivalent to using the (address, port) pair for a connection 
>   identifier in single-path TCP, which is no longer static in MPTCP.
> 
> Since there is no longer a single, static (address, port) pair for a connection, we should provide a static connection ID for MPTCP-aware applications to track its connections.
Hmm. I'm confused. On section 4.2.2 it is stated that for backwards compatibility
getsockname() and getpeername() return something unique for the MP-TCP connection.

So why don't you do this always?
> 
>> * Section 5.3.1: Why can't you use TCP_MUTLIPATH_REMOVE only after
>> connection setup?
> 
> Not sure there's much of a call for it, but I guess there's no reason to prevent its use in address list management before setting up a connection.
Assume I bind 3 addresses to a listening socket. Lateron I can remove one of
the addresses. If then a connection is accepted, only two addresses are
used. At least this works on SCTP....
> 
>> * Table 1: What is a "list of addresses"?
> 
> An abstract data structure consisting of a list of IP (v4 and/or v6) addresses (and optionally ports).
> 
> Do you consider it necessary to be more explicit? The data structures are explained in more detail in the following sections.
When I would implement it, I would need a bit more specific information.
Do you really mean a list (some strucuture with a next pointer)? Do
you mean an array of struct sockaddr_in structures (for an AF_INET socket),
what is the number of addresses? Do you means an array of sockaddr_in6
structures (for an AF_INET6) socket? What is the number of addresses?
Or do you use an array of sockaddr_storage structures? Is a mixture
of sockaddr_in and sockaddr_in6 structures allowed? What about mapped
addresses?
> 
>> * Table 1: What is a "list of pairs of addresses"?
> 
> See above, but this time with pairs within the list :)
Same problem as above?
> 
>> * Section 5.3.5: Why is TCP_MULTIPATH_CONNID needed?
> 
> See my response to REQ4 above.
> 
>> * Section 6.1: Are you suggesting that sctp_bindx(), sctp_connectx(),
>>  sctp_getladdrs(), sctp_getpaddrs() are also provided for MPTCP?
>>  This sounds strange.
>>  Maybe one can introduce bindx(), connectx(), getladdrs(), getpaddrs()
>>  which supports SCTP and MPTCP. Kacheong suggested that a while ago.
>>  Maybe it is time to do that.
>>  Please note that you missed sctp_freeladdrs() and sctp_freepaddrs().
> 
> I'll leave Michael to answer that one :)
> 
> Cheers,
> Alan
> 
>

---

From: "Ford, Alan" <[email-masked]>
Date: Sat, 9 Apr 2011 17:54:30 +0100

Hi Michael,

Responses inline...

> > Since there is no longer a single, static (address, port) pair for a
> connection, we should provide a static connection ID for MPTCP-aware
> applications to track its connections.
>

No, 4.2.2 doesn't say it is unique for a connection. Whilst you can ensure local uniqueness (getsockname()), you cannot guarantee that the remote (addr, port) will be unique for the lifetime of a connection.

The key point of the getsockname/getpeername debate was about whether an address remains valid as a connection identifier when the address is removed from the connection. The conclusion from the very extensive discussions here was that we simply don't know if it will cause problems for applications, so we will leave it open as to whether a connection can stay open after the loss of the first subflow, and await implementation lessons.

If it can, then the information returned by the non-MPTCP-aware functions of getsockname and getpeername may not be accurate at the given instant in time. We want to get people away from using these functions with inaccurate assumptions, and instead use the new API for address querying.

The other purpose of such functions today would be for unique connection identification. A local connection identifier, decoupled from the concept of addresses, seems ideal for this purpose - that's what REQ4 is all about.

> > Do you consider it necessary to be more explicit? The data structures are
> explained in more detail in the following sections.
>

Feedback during IETF79 was strongly against specifying to this level of detail (we had previously done so - see the pre-WG-adoption versions). The IETF is not in a position to specify concrete APIs, so we specified it to the level of abstract definition you see currently: this specifies the kinds of data that need to be transferred only.

The term "list" seemed (to me at least) to be sufficiently abstract to allow its actual implementation and API to be left to library authors, in whatever is consistent with the rest of their API.

I realise the concerns you raise are valid ones, but are they things that should be dealt with in this draft, or left to software authors? Our previous feedback has been very much towards the latter option.

Thanks,
Alan

---

From: Michael Tüxen <[email-masked]>
Date: Sat, 9 Apr 2011 19:18:31 +0200

> The other purpose of such functions today would be for unique connection identification. A local connection identifier, decoupled from the concept of addresses, seems ideal for this purpose - that'…
.. so you are saying that some applications might break on some
implementations of MP-TCP. Right?
> I realise the concerns you raise are valid ones, but are they things that should be dealt with in this draft, or left to software authors? Our previous feedback has been very much towards the latte…
OK. However, this means that you can not build programs which can be
ported from one MP-TCP implementation to another.

Maybe you can state that it is not the goal of the document to specify
an API which allow writing portable applications.

I'm not sure about the above raised point regarding IETF and APIs. At least
http://tools.ietf.org/html/draft-ietf-tsvwg-sctpsocket
gives some kind of specification (the intended status is
Informational) and it was very useful even as an ID, since most
kernel implementations provide that interface.

Best regards
Michael

---

From: "Ford, Alan" <[email-masked]>
Date: Sun, 10 Apr 2011 13:39:57 +0100

Hi Michael,

> > No, 4.2.2 doesn't say it is unique for a connection. Whilst you can ensure
> local uniqueness (getsockname()), you cannot guarantee that the remote (addr,
> port) will be unique for the lifetime of a connection.
> >
> > The key point of the getsockname/getpeername debate was about whether an
> address remains valid as a connection identifier when the address is removed
> from the connection. The conclusion from the very extensive discussions here
> was that we simply don't know if it will cause problems for applications, so
> we will leave it open as to whether a connection can stay open after the loss
> of the first subflow, and await implementation lessons.
> >
> > If it can, then the information returned by the non-MPTCP-aware functions
> of getsockname and getpeername may not be accurate at the given instant in
> time. We want to get people away from using these functions with inaccurate
> assumptions, and instead use the new API for address querying.
> >
> > The other purpose of such functions today would be for unique connection
> identification. A local connection identifier, decoupled from the concept of
> addresses, seems ideal for this purpose - that's what REQ4 is all about.
>

The short answer is, we don't know.

The long answer: it is conceivable that there could be applications that make assumptions about the answers returned by getsockname()/getpeername() and which under certain circumstances may break. We do not know, and require experimental feedback to learn whether this is a problem, and if so, what the best way of solving it is (whilst maintaining the widest access to MPTCP benefits). This is covered in RFC6182 as well as the API draft.

However, this is orthogonal to your original question, asking about REQ4 - the concise answer to which is that we are trying to get away from assumptions about IP addresses being static since that is no longer necessarily the case with MPTCP (while maintaining as much backwards compatibility as possible).

> > Feedback during IETF79 was strongly against specifying to this level of
> detail (we had previously done so - see the pre-WG-adoption versions). The
> IETF is not in a position to specify concrete APIs, so we specified it to the
> level of abstract definition you see currently: this specifies the kinds of
> data that need to be transferred only.
> >
> > The term "list" seemed (to me at least) to be sufficiently abstract to
> allow its actual implementation and API to be left to library authors, in
> whatever is consistent with the rest of their API.
> >
> > I realise the concerns you raise are valid ones, but are they things that
> should be dealt with in this draft, or left to software authors? Our previous
> feedback has been very much towards the latter option.
>

OK, so what do we do? We should certainly specify the scope at the start, and I would like somehow to cover a more concrete specification.

However, it was a condition of the document's WG adoption that we made it more abstract than the original specification (similar to what you are asking for). Dave Thaler and others spoke very strongly against the level of concrete spec that you are asking for during IETF79.

An alternative approach - would it be appropriate to include a more detailed spec with proposed code snippets in an appendix? The danger, of course, is that if the API is changed by other standards bodies/implementers the appendix becomes irrelevant or even misleading.

Regards,
Alan

---

From: Michael Tüxen <[email-masked]>
Date: Sun, 10 Apr 2011 22:06:02 +0200

> The long answer: it is conceivable that there could be applications that make assumptions about the answers returned by getsockname()/getpeername() and which under certain circumstances may break.…
OK, I see.
> However, this is orthogonal to your original question, asking about REQ4 - the concise answer to which is that we are trying to get away from assumptions about IP addresses being static since that…
OK.
> However, it was a condition of the document's WG adoption that we made it more abstract than the original specification (similar to what you are asking for). Dave Thaler and others spoke very stron…
I see. If it is the WG consensus, I'm fine with it.
> An alternative approach - would it be appropriate to include a more detailed spec with proposed code snippets in an appendix? The danger, of course, is that if the API is changed by other standards…
In the case of SCTP, developers from FreeBSD, Solaris and Linux are coauthors.
So what these implementation do is really described well in the ID. And the
experience is that having this document for years was very helpful.
I'm not sure which other standards bodies will start working on such things.
Maybe POSIX, I don't know.

It seams that you are aware of the points I wanted to raise and the WG
decided to do it differently. This is fine with me.

Best regards
Michael