Subject: Re: [multipathtcp] comments on draft-ietf-mptcp-multiaddressed-07

From: Alan Ford <[email-masked]>
Date: Fri, 25 May 2012 11:17:36 +0100

Hi Phil,

Many thanks for you comments. Anything I have not directly commented on
below was straightforwardly addressed in 08. If there is anything you feel
wasn¹t sufficiently addressed, please shout.

Regards,
Alan

On 26/04/2012 17:57, "[email-masked]" <[email-masked]> wrote:

> Hi Alan, Costin, Mark, Olivier,
>  
> Here are some comments. Sorry I finally got through it.
> The first lot are either significant or something worth thinking about. The
> second lot are just typos & re-phrasings  feel free to ignore if you don¹t
> like the suggestion
>  
> Thanks!
> phil
>  
> --
>  
> Think it would be good to have a sub-section in S2 on Fast Close
>  
> Also might be worth adding one on MP_FAIL ?
> 
> [Alan] I decided against doing this. S2 is meant to be a light overview of the
> protocol; it seems unnecessary and potentially confusing to burden readers
> with these little-used features at this stage.
>  
> S3.1 penultimate para, << If this
>    option is not present, the connection should fall back to regular
>    TCP , as documented in Section 3.6.>>
> SHOULD?
> also S3.6 para 1 is about this  but it doesn¹t have any capitalisation, which
> presumably it should
>  
> I thought that S3.2 could be clearer. Basically there are a lot of minor
> things (see below) that add up to making it quite hard work.
>  
> Fig 8  I think the first msg (SYN + MP_CAPABLE) should have (Key-A)
>  
> S3.3.0  do you say somewhere what to do if the checksum is present but it
> wasn¹t negotiated in MP-CAPABLE, or it isnt present but wasn¹t negotiated?
>  
> [Alan] We do now: ³If a checksum is present, but its use had not been
> negotiated in the MP_CAPABLE handshake, it SHOULD be ignored. If a checksum is
> not present when its use has been negotiated, the receiver SHOULD close the
> subflow with a RST as it is considered broken.²
> 
> P36 para 4 < If the Address ID is not in use on a
>    live subflow, but is stored by the receiver, a new ADD_ADDR SHOULD
>    take precedence and replace the stored address.>
> I found this sentence quite hard to parse. If I understand it right, this is
> advertising a new address by over-writing an old address - using the same
> Address ID. Rather than removing the old address and adding a new one with a
> different Address ID. Sounds dodgy? why doing this?
> 
> [Alan] You understood it right, however you¹re right is has a whiff of
> dodgyness to it. So I¹ve removed it  now a host that wants to replace an
> Address ID MUST remove it first.
>  
> 3.5  Fast Close sends the Receiver¹s Key in the clear. Does this open an
> attack?  
> 
> [Alan] Not any new risks. Once it¹s sent on Fast Close, the connection is
> dead, so no further signalling with that key would have any effect. And yes,
> anyone who had seen the key on the initial handshake could Fast Close the
> connection. But they can do anything anyway if they¹ve seen the key on the
> initial handshake. In Fast Close, usual TCP sequence number verification is
> required to ensure the signal is in-window (so an attacker would need to know
> both the key of the receiver and the sequence space of the sender).
> 
> There might be value in e.g. hashing the key with the sender¹s DSN for
> slightly more protection; I¹m not sure if that adds anything. In fact, it
> probably goes against the point of Fast Close that the sender actively DOES
> NOT CARE whether its last packets actually get to the end host.
> 
> anyway, it contradicts S3.1 & S5 which say it¹s only sent in the clear once,
> ie on MP-CAPABLE
> 
> [Alan] Fixed this.
>  
> 3.6  I had it in my brain that fallback meant Œfallback from mptcp to tcp¹.
> Here it means Œ.. or close a problematic subflow¹. I¹m not absolutely sure
> it¹s used consistently throughout the doc, but it¹s probably worth adding an
> intro para to 3.6 that says there are n ways of coping with some problem,
> close the subflow, close the connection (?), shift to tcp
> 
> [Alan] You¹re right, it has evolved from its original meaning. Now re-worded:
> ³Sometimes, middleboxes will exist on a path that could prevent the operation
> of MPTCP.  MPTCP has been designed in order to cope with many middlebox
> modifications (see Section 6), but there are still some cases where a subflow
> could fail to operate within the MPTCP requirements.  These cases are notably:
> the loss of TCP options on a path; and the modification of payload data.  If
> such an event occurs, it is necessary to "fall back" to the previous, safe
> operation.  This may either be falling back to regular TCP, or removing a
> problematic subflow.²
>  
> 3.6 para 1  needs to be a SHOULD somewhere in here
>  
> 3.6 para 1 COMPARE 3.8.3 para 2  3.6 suggests give up on mptcp after one
> MP_CAPABLE fails (& also 6 & 3.1); 3.8 suggests after several. Which is right?
> 
> [Alan] It¹s up to the implementation. The heuristics are our initial thoughts
> on this; implementations optimise this. The heuristics now say ³one or more²
> rather than ³several².
> 
> S4 Receive window last sentence, ³so a host must²  SHOULD? Or maybe Œneeds
> to¹?  Also, this point doesn¹t appear in S3.3.4
> 
> [Alan] ³needs to². Re S3.3.4, I think it is implied  there is talk there
> about normal subflow processing.
>  
> P48 penul para < If some subflow-level space is left unmapped,
>    however, the subflow is treated as broken and is closed, as discussed
>    in Section 3.3.  >
> do you mean S3.6? I couldn¹t see anything in s3.3
>  
> [but it¹s a big section, so a more accurate ref would be useful  same comment
> on p10 just above the SS pic & p40 para 3]
> 
> [Alan] The reference to S3.6 there are more a general pointer to procedure
> than a specific instruction  I¹ve clarified this.
>  
> P49 PEPs < MPTCP will
>       therefore fall back to single-path TCP (see Section 3.6).>
> I read S3.6 to say that you close the subflow (rather than fall back to tcp)
> 
> [Alan] S3.6 is intended to cover both, and hopefully this new intro should
> make this clearer.
>  
> S8  can be updated with Option kind number 30 ! also Wes¹s comment about the
> subtypes [ie invent a policy for managing them]
> 
> [Alan] It¹s now been rewritten in RFC5226 style.
>  
> Appendix C  should the Œsnd DATA_ACK¹ arrows from M_ESTAB & from M_FIN WAIT-1
> be snd DATA_ACK[DFIN] ?
>  
> --
>  
> Minor suggestions etc  no need to ack these individually [but shout if I
> reveal some misunderstanding!]
>  
> Throughout doc: 
>  
> Be consistent on ŒDATA_ACK¹ & ŒDATA_FIN¹  many times appear without Œ_¹ [I
> guess I prefer MP_DATA_ACK but that may be too much for your taste!]
> 
> [Alan] Remember those two are flags and not specific options, so it¹s
> definitely inappropriate to call them MP_... But they¹ve all grown an
> underscore for consistency.
>  
> I think it would be more logical if all the subtype names started MP_ [ie also
> DSS, ADD_ADDR, REMOVE_ADDR] [again, maybe too much for your taste]
> 
> [Alan] Given that these are symbols within a MPTCP-specific registry, I only
> see MP_ as taking up unnecessary space. Sorry ;)
> 
> The bare ŒID¹ appears quite often- think you should use ŒAddress ID¹
> throughout
> 
> [Alan] I found three instances, let me know if I¹ve missed any.
>  
> --
>  
> S1.1 <<The
>    congestion control algorithms as discussed in [5] ensure this does
>    not act detrimentally.>>
> The congestion control algorithm defined in [5] achieves this.
>  
> S1.3 a reference to S4 might be nice
>  
> S1.4 bullet 2, ³where a TCP connection is established²  should be MPTCP
> connection
>  
> 2.1 & 2.2  would it be more logical if ŒACK MP_CAPABLE¹ & ŒACK MP_JOIN¹
> didn¹t have ŒACK¹ since there¹s no special ACK subtype? But maybe this makes
> it harder to understand what¹s happening?
>  
> 2.4 para 2 <<   The "Data Sequence Signal" option which carries this "Data
> Sequence
>    Mapping", which consists of the subflow sequence number, data>>
>    The "Data Sequence Signal" option carries the "Data Sequence
>    Mapping". The Data Sequence Mapping consists of the subflow sequence
> number, data
>  
> 2.7 bullet 1 ³MP-ADD-ADDR² should be ADD_ADDR (unless you change subtype names
> to all start MP_ !)
>  
> 3.1 para 4, last sentence (The subflow handshake mechanismŠ) is quite hard to
> parse
>  
> 3.1, last para p14 << The leftmost bit - labeled C
>    - indicates "Checksum required", and SHOULD be set to 1 unless
>    specifically overridden (for example, if the system administrator has
>    decided that checksums are not required - see Section 3.3 for more
>    discussion)>>
> The leftmost bit - labelled C - SHOULD be set to 1 to indicate "Checksum
> required", unless the system administrator has
>    decided that checksums are not required - see Section 3.3 for more
>    discussion)
>  
> also, S3.3 doesn¹t discuss why someone might decide to use checksums, or
> decide not to. Might be nice to add a hint about this somewhere
>  
> also, when there are multiple crypo algos, do we want to point out that the
> same one MUST be used for everything (generating token, isdn etc)? maybe this
> is obvious and anyway would be better in any subsequent doc about alternative
> crypto
>  
> [Alan] That¹s a good point, however I think it is best to leave it to any new
> documents to decide whether they also want to tie a new crypto algorithm with
> tokens/IDSN generation.
> 
> p15  I thought the first sentence [³These bits..² was redundant
>  
> p15 para 2 << The initiator
>    creates a proposal setting a bit for each algorithm >>
> The initiator sets a bit for each algorithm
>  
> P16, last para of 3.1 << The initial Data Sequence Number (IDSN) is generated
> as a hash from
>    the Key, in the same way as the token, i.e.  IDSN-A = Hash(Key-A) and
>    IDSN-B = Hash(Key-B).>>
> The Initial Data Sequence Number (IDSN) is generated as a hash from
>    the Key, i.e.  IDSN-A = Hash(Key-A) and
>    IDSN-B = Hash(Key-B).
>  
> Middle p17  the Œthis¹ bonanza made it hard to understand. How about
> (includes a couple of other changes):-
> The MP_JOIN option includes an "Address ID".  This is an identifier
>    that only has significance within a single connection, where it
>    identifies the original source address of the packet, even if the IP header
> has been changed in transit by a middlebox.  This allows
>    address removal (Section 3.4.2) without needing to know what the source
> address at
>    the receiver is, and thus allows address removal through NATs. It also
> allows correlation between new subflow
>    setup attempts and address signaling (Section 3.4.1), to prevent
>    setting up duplicate subflows on the same path.
>  
> P17 last para<< The MP_JOIN option on SYNs >> add Œand SYN_ACKs¹
> 
> [Alan] ³Packets with the SYN flag set²
>  
> P18 first para under Fig 5 << Although cryptographic calculations are required
> in the
>    SYN/ACK, it is felt that the 32 bit token gives >>
> Although calculating a MAC requires cryptographic operations, it is believed
> that the 32 bit token in the MP_JOIN SYN gives
>  
> P18 2nd para, sentence 2 ³This is to allow..² takes some working out. Maybe:
> ³This allow Host B to use random data sent by Host A in the SYN, as part of
> the MAC algorithm; and similarly Host A to use random data sent by Host B in
> the SYN/ACK.
>  
> Last sentence of this para can be shortened:
> Due
>    to option space limitations, the MAC included in the SYN/ACK is
>    truncated to the leftmost 64 bits, but this is acceptable since
> an attacker has only one chance to guess the MAC correctly.
>  
> Next para  ³and this is² -> ³as²
>  
> Same para ³MUST trigger an ACK in response²  maybe you could stress that this
> is an ordinary data ACK
>  
> P19 ³The message in each case ³ -> The message for the MAC algorithm in each
> case 
>  
> S3.2, the Random Number == Nonce in S2. Point out, or change one of them.
>  
> S3.3 para 2 & 3 could be re-phrased slightly:-
>    During normal MPTCP operation, the Data Sequence Mapping defines how the
> sequence space on the subflow maps to
>    the connection level; and the Data ACK acknowledges receipt of
>    data at the connection level.  These functions are described in more
>    detail in the following two subsections.
> The Data Sequence Mapping and the Data ACK are signalled in the Data Sequence
> Signal (DSS). Either or both can be signalled in one DSS, dependent on the
> flags set.
>  
> P23 << The data sequence mapping also contains a checksum of the data that
>    this mapping covers.  >> - to check I understand, this means you need all
> the data before you start to transmit it, so you can calculate the checksum?
>  
> P24 first para under fig 10, you could delete ³. Furthermore,² and put the
> rest of the sentence in (..)
>  
> Last para 3.3.1 ³data-level length field to ³
> Data-level Length field of the DSS option to
>  
> P26 para 3, ³that it is used² can be deleted
>  
> P26 para 4 << An MPTCP sender MUST only free data from the send buffer when it
> has
>    been acknowledged by both a Data ACK received on any subflow and at
>    the subflow level by any subflows the data was sent on.  >>
> could be clearer
> An MPTCP sender MUST NOT free data from the send buffer until it has
>    been acknowledged by both a Data ACK received on any subflow and at
>    the subflow level by all subflows the data was sent on.
> [note, 2nd Œany¹ -> Œall¹]
>  
> Same para restransmited / re-transmitted
>  
> P27 para 4 < and length 11> and Data-level Length of 11
>  
> Same thing in next para
>  
> P27 penult para  I think ŒData Sequence Signal¹ would be more accurate here
> than ŒMapping¹. Also Length -> Data-level Length
>  
> P27 last line, think you mean ŒDATA_FIN¹ and not FIN
>  
> 3.3.4 para 4 << When deciding to accept packets at subflow level, normal TCP
> uses the
>    sequence number in the packet and checks it against the allowed
>    receive window.  >>
> When deciding to accept packets at the subflow level, normal TCP checks the
>    sequence number in the packet against the allowed
>    receive window.
>  
> P28 last line
> Could you spell out what rcv_next is [or at least reference B2.2.2, where it¹s
> called RCV.NXT]
>  
> 3.3.4 last para Œmaximum bandwidth-delay product of any of the paths¹ wonder
> if Œany one of the¹ would be clearer?
>  
> 3.3.6 para 1 Œthe best behaviour¹ -> Œsensible behaviour¹
>  
> P30 last line active ACK -> actively ACK
>  
> 3.3.6 last line - < For example, subflows that perform highly asymmetrically
> may be mis-
>    diagnosed as underperforming.>
> For example, a highly asymmetrically path may be mis-
>    diagnosed as underperforming.
>  
> P33 para 1 ŒThe signal applies to a single direction:
>    the sender of this option, however, may >>
> The signal applies to a single direction  and so the sender of this option
> may 
>  
> Next para < This
>    applies the given setting of B to all subflows that use the address
>    identified by the given Address ID.  >
> do you think it¹s worth saying that this means all subflows in this
> connection?
>  
> S3.4 para 2 < This design makes use of two methods of sharing such
> information,
>    used simultaneously.  >
> This design makes use of two methods of sharing such information; both can be
> used on a connection.
>  
> 3.4.1 understake -> undertake
>  
> P35 para above Fig 12 as does the ephemeral port at the client  explain.
>  
> Same para, ³signalling subflow¹  should this  be Œinitial subflow¹?
>  
> Fig 12 caption  can delete Œ(shown for IPv4)¹  as pic includes v6
>  
> P36 para 2 < This would be to ensure that this> This would ensure that
>  
> 3.4.1 last para < an MPTCP
>    implementation MUST NOT treat duplicate ACKs with any MPTCP option
>    apart from DSS as indications of congestion >>
> ³NOT Š any .. apart from² is ambiguous.
>  
> P38 last bullet < The number of
>       retransmissions should be limited >
> SHOULD?
>  
> S3.6  end of sentence 1  add Œfor instance they aren¹t blocked or mangled by
> a middlebox 
>  
> 3.6 para 4 < with the length value of 0 > with the Data-level Length set to 0
>  
> P40 para 1 I couldn¹t parse this:-
> (Note that these rules do not apply if an
>    infinite mapping is included from the start - in which case, each end
>    will send DSS options declaring the infinite mapping.)
>  
> Fig 15 Length may be 8 & DSN may be 4 octets
>  
> 3.6 penult para ³fallback mode²  what is this?
>  
> 3.8.3 para 2 < a host should fall back> SHOULD
>  
> S4, Duplicate ack. ³To avoid any² -> To limit
>  
> S4, Receive window penult line <also need to> also needs to
>  
> S5, big para on p46 < used on subflow setup that verify that > used on subflow
> setup, in order to verify that
>  
> P47 line 1, ³will allow² -> allows
>  
> S6 para 1 &   P54 para 2
> accomodate -> accommodate
>  
> S6 para 2 < Most middleboxes
>    should just forward packets with new options unchanged>
> Middleboxes should just forward packets with new options unchanged
> [I think  (all) middleboxes aren¹t supposed to change options; most don¹t]
>  
> P49 Traffic normalisers  should you also mention about re-transmitting on the
> same subflow?
>  
> 9.2  maybe worth adding rfc editor note that hoping this will become rfc at
> the same time
>  
> P54 para 2 <(10B)> 10 bytes
>  
> Appendix C  point out that [DFIN] is shorthand for [DATA_FIN]  (I think!)
>  
> That¹s it, phew.
> 
>  
>  
> From: [email-masked] [mailto:[email-masked]] On
> Behalf Of Yoshifumi Nishida
> Sent: 18 April 2012 07:04
> To: [email-masked]; multipathtcp
> Subject: Re: [multipathtcp] comments on draft-ietf-mptcp-multiaddressed-07
>  
> Hi authors,
>  
> Sorry. I would like to add one more thing.
>  
> This draft requires to create new registries for subtypes of mptcp option and
> cryptographic algorithms handshake in MP_CAPABLE.
> For this, you will need to add the description of the policy used for managing
> these registries in the IANA considerations section. Please check the section
> 4 of RFC5226 to see the available choices (Section 4.1) and update the texts
> based on the guideline (Section 4.2).
>  
> Thanks,
> --
> Yoshifumi Nishida
>  
>  
>  
> On Mon, Apr 16, 2012 at 11:53 PM, Yoshifumi Nishida <[email-masked]>
> wrote:
> Hello,
>  
> Sorry for the long delay.
> We believe it's good time to proceed to submit draft-ietf-mptcp-multiaddressed
> to IESG.
>  
> Here's my comments on the draft to prepare a write-up for the draft.
> Please check and update if you agree with them.
>  
> Thanks,
> --
> Yoshifumi
>  
>  
> Page 13:
>   3.1. Connection Initiation
>  
>     ->  In my understanding, the key MUST be unique for each mptcp connection.
>          If so, I think we had better articulate it here.
>  
> Page 17:
>    A host MUST store the Address IDs associated with all established subflows.
>  
>    -> Does this mean both local Address IDs and remote Address IDs advertized
> from the receiver?
>        It might be better to clarify this.
>  
> Page 18:
>    therefore receipt of this packet MUST trigger an ACK in response
>  
>    -> Since this is MUST, I think Figure 8 needs to include this ACK in the
> sequence.
>  
>  
>    the packet MUST be retransmitted if this ACK is not received.
>  
>    -> In case of MP_CAPABLE, the third packet is not retransmitted.
>        But, in case of MP_JOIN, it's retransmitted. Readers might want to know
> the rationale about this.
>   
> Page 27:
>  
>    Essentially, a host MUST NOT FIN all functioning subflows unless it is safe
> to do so
>    
>     -> MUST NOT close?
>  
> Page 28:
>  
>    Both hosts SHOULD send FINS, as a courtesy to allow middleboxes to clean up
> state even if the subflow is failed.
>    -> I'm not very sure which situations can be addressed by this..
>    ..
>    Note that a host may also send a FIN on an individual subflow to shut it
> down, but this impact is limited to the subflow in question.
>  
>    -> Sorry. I couldn't understand this sentences well..
>  
> Page 30:
>  
>     The sender also remembers the receive windows advertised by each subflow.
>    
>     -> Don't we need to put SHOULD or MUST here?
>  
>     The send buffer must be, at the minimum, as big as the receive buffer
>  
>     -> MUST?
>  
>     When doing this, a host must still retransmit the original data on the
> original subflow,
>  
>     -> MUST?
>  
> Page 37:
>  
>    if a response is received, the path is not removed.
>  
>    a subflow that is still functioning MUST be closed with a FIN exchange
>  
>     -> Aren't these sentences inconsistent?
>  
> Page 40:
>  
>    fallback to regular TCP can become necessary at any point during a
> connection if a non-MPTCP-aware middlebox changes the data stream.
>  
>     -> The paragraph below this suggest when multiple subflows are in use, the
> affected subflow must be terminated, but not suggesting fallback.
>          So, fallback seems not to be necessary?
>  
>    Therefore, it is not possible to recover the subflow, and the affected
> subflow must be immediately closed with an RST.
>  
>     -> MUST?
>  
>    Failed data will not be DATA_ACKed.
>  
>     -> MUST NOT be DATA_ACKed?
>  
>  
> Page 47:
>  
>    If this is dropped, MPTCP SHOULD fall back to regular TCP.
>  
>     -> may be MUST? since we cannot continue to use MPTCP .
>  
>    If packets with the MP_JOIN option are dropped, the paths will simply not
> be used.
>  
>     -> MUST NOT be used?
>  
>  
> Page 51:
>  
>    Should use RFC6234 instead of RFC4634
>  
> Page 52:
>  
>    Should use RFC5681 instead of RFC2581
>    Should use draft-ietf-tcpm-secury-03 instead of -02
>    Should use draft-ietf-mptcp-api-04 or 05? instead of -03
>  
>  
> There are some typos in the doc. Please update them by some tools.
>  
>

---

From: <[email-masked]>
Date: Tue, 29 May 2012 09:22:08 +0100

Alan,
Thanks - your changes look good to me.
phil

From: Alan Ford [mailto:[email-masked]]
Sent: 25 May 2012 11:18
To: Eardley,PL,Philip,DUB8 R; [email-masked]
Cc: [email-masked]
Subject: Re: [multipathtcp] comments on draft-ietf-mptcp-multiaddressed-07

Hi Phil,

Many thanks for you comments. Anything I have not directly commented on below was straightforwardly addressed in -08. If there is anything you feel wasn't sufficiently addressed, please shout.

Regards,
Alan

Here are some comments. Sorry I finally got through it.
The first lot are either significant or something worth thinking about. The  second lot are just typos & re-phrasings - feel free to ignore if you don't like the suggestion

Thanks!
phil

--

[Alan] I decided against doing this. S2 is meant to be a light overview of the protocol; it seems unnecessary and potentially confusing to burden readers with these little-used features at this stage.

S3.1 penultimate para, << If this
   option is not present, the connection should fall back to regular
   TCP , as documented in Section 3.6.>>
SHOULD?
also S3.6 para 1 is about this - but it doesn't have any capitalisation, which presumably it should

I thought that S3.2 could be clearer. Basically there are a lot of minor things (see below) that add up to making it quite hard work.

Fig 8 - I think the first msg (SYN + MP_CAPABLE) should have (Key-A)

S3.3.0 - do you say somewhere what to do if the checksum is present but it wasn't negotiated in MP-CAPABLE, or it isnt present but wasn't negotiated?

[Alan] We do now: "If a checksum is present, but its use had not been negotiated in the MP_CAPABLE handshake, it SHOULD be ignored. If a checksum is not present when its use has been negotiated, the receiver SHOULD close the subflow with a RST as it is considered broken."

P36 para 4 < If the Address ID is not in use on a
   live subflow, but is stored by the receiver, a new ADD_ADDR SHOULD
   take precedence and replace the stored address.>
I found this sentence quite hard to parse. If I understand it right, this is advertising a new address by over-writing an old address - using the same Address ID. Rather than removing the old address and adding a new one with a different Address ID. Sounds dodgy? why doing this?

[Alan] You understood it right, however you're right is has a whiff of dodgyness to it. So I've removed it - now a host that wants to replace an Address ID MUST remove it first.

3.5 - Fast Close sends the Receiver's Key in the clear. Does this open an attack?

[Alan] Not any new risks. Once it's sent on Fast Close, the connection is dead, so no further signalling with that key would have any effect. And yes, anyone who had seen the key on the initial handshake could Fast Close the connection. But they can do anything anyway if they've seen the key on the initial handshake. In Fast Close, usual TCP sequence number verification is required to ensure the signal is in-window (so an attacker would need to know both the key of the receiver and the sequence space of the sender).

There might be value in e.g. hashing the key with the sender's DSN for slightly more protection; I'm not sure if that adds anything. In fact, it probably goes against the point of Fast Close that the sender actively DOES NOT CARE whether its last packets actually get to the end host.

anyway, it contradicts S3.1 & S5 which say it's only sent in the clear once, ie on MP-CAPABLE

[Alan] Fixed this.

3.6 - I had it in my brain that fallback meant 'fallback from mptcp to tcp'. Here it means '.. or close a problematic subflow'. I'm not absolutely sure it's used consistently throughout the doc, but it's probably worth adding an intro para to 3.6 that says there are n ways of coping with some problem, close the subflow, close the connection (?), shift to tcp

[Alan] You're right, it has evolved from its original meaning. Now re-worded: "Sometimes, middleboxes will exist on a path that could prevent the operation of MPTCP.  MPTCP has been designed in order to cope with many middlebox modifications (see Section 6), but there are still some cases where a subflow could fail to operate within the MPTCP requirements.  These cases are notably: the loss of TCP options on a path; and the modification of payload data.  If such an event occurs, it is necessary to "fall back" to the previous, safe operation.  This may either be falling back to regular TCP, or removing a problematic subflow."

3.6 para 1 - needs to be a SHOULD somewhere in here

3.6 para 1 COMPARE 3.8.3 para 2 - 3.6 suggests give up on mptcp after one MP_CAPABLE fails (& also 6 & 3.1); 3.8 suggests after several. Which is right?

[Alan] It's up to the implementation. The heuristics are our initial thoughts on this; implementations optimise this. The heuristics now say "one or more" rather than "several".

S4 Receive window last sentence, "so a host must" - SHOULD? Or maybe 'needs to'?  Also, this point doesn't appear in S3.3.4

[Alan] "needs to". Re S3.3.4, I think it is implied - there is talk there about normal subflow processing.

P48 penul para < If some subflow-level space is left unmapped,
   however, the subflow is treated as broken and is closed, as discussed
   in Section 3.3.  >
do you mean S3.6? I couldn't see anything in s3.3

[but it's a big section, so a more accurate ref would be useful - same comment on p10 just above the SS pic & p40 para 3]

[Alan] The reference to S3.6 there are more a general pointer to procedure than a specific instruction - I've clarified this.

P49 PEPs < MPTCP will

[Alan] S3.6 is intended to cover both, and hopefully this new intro should make this clearer.

S8 - can be updated with Option kind number 30 ! also Wes's comment about the subtypes [ie invent a policy for managing them]

[Alan] It's now been rewritten in RFC5226 style.

Appendix C - should the 'snd DATA_ACK' arrows from M_ESTAB & from M_FIN WAIT-1  be snd DATA_ACK[DFIN] ?

--

Minor suggestions etc - no need to ack these individually [but shout if I reveal some misunderstanding!]

Throughout doc:

Be consistent on 'DATA_ACK' & 'DATA_FIN' - many times appear without '_' [I guess I prefer MP_DATA_ACK but that may be too much for your taste!]

[Alan] Remember those two are flags and not specific options, so it's definitely inappropriate to call them MP_... But they've all grown an underscore for consistency.

I think it would be more logical if all the subtype names started MP_ [ie also DSS, ADD_ADDR, REMOVE_ADDR] [again, maybe too much for your taste]

[Alan] Given that these are symbols within a MPTCP-specific registry, I only see MP_ as taking up unnecessary space. Sorry ;)

The bare 'ID' appears quite often- think you should use 'Address ID' throughout

[Alan] I found three instances, let me know if I've missed any.

--

S1.1 <<The

S1.4 bullet 2, "where a TCP connection is established" - should be MPTCP connection

2.1 & 2.2 - would it be more logical if 'ACK MP_CAPABLE' & 'ACK MP_JOIN' didn't have 'ACK' since there's no special ACK subtype? But maybe this makes it harder to understand what's happening?

2.4 para 2 <<   The "Data Sequence Signal" option which carries this "Data Sequence
   Mapping", which consists of the subflow sequence number, data>>
   The "Data Sequence Signal" option carries the "Data Sequence
   Mapping". The Data Sequence Mapping consists of the subflow sequence number, data

2.7 bullet 1 "MP-ADD-ADDR" should be ADD_ADDR (unless you change subtype names to all start MP_ !)

3.1 para 4, last sentence (The subflow handshake mechanism...) is quite hard to parse

The leftmost bit - labelled C - SHOULD be set to 1 to indicate "Checksum required", unless the system administrator has

also, S3.3 doesn't discuss why someone might decide to use checksums, or decide not to. Might be nice to add a hint about this somewhere

also, when there are multiple crypo algos, do we want to point out that the same one MUST be used for everything (generating token, isdn etc)? maybe this is obvious and anyway would be better in any subsequent doc about alternative crypto

[Alan] That's a good point, however I think it is best to leave it to any new documents to decide whether they also want to tie a new crypto algorithm with tokens/IDSN generation.

p15 - I thought the first sentence ["These bits.." was redundant

P16, last para of 3.1 << The initial Data Sequence Number (IDSN) is generated as a hash from
   the Key, in the same way as the token, i.e.  IDSN-A = Hash(Key-A) and
   IDSN-B = Hash(Key-B).>>
The Initial Data Sequence Number (IDSN) is generated as a hash from
   the Key, i.e.  IDSN-A = Hash(Key-A) and
   IDSN-B = Hash(Key-B).

Middle p17 - the 'this' bonanza made it hard to understand. How about (includes a couple of other changes):-
The MP_JOIN option includes an "Address ID".  This is an identifier
   that only has significance within a single connection, where it
   identifies the original source address of the packet, even if the IP header has been changed in transit by a middlebox.  This allows
   address removal (Section 3.4.2) without needing to know what the source address at
   the receiver is, and thus allows address removal through NATs. It also allows correlation between new subflow
   setup attempts and address signaling (Section 3.4.1), to prevent
   setting up duplicate subflows on the same path.

P17 last para<< The MP_JOIN option on SYNs >> add 'and SYN_ACKs'

[Alan] "Packets with the SYN flag set"

P18 first para under Fig 5 << Although cryptographic calculations are required in the
   SYN/ACK, it is felt that the 32 bit token gives >>
Although calculating a MAC requires cryptographic operations, it is believed that the 32 bit token in the MP_JOIN SYN gives

P18 2nd para, sentence 2 "This is to allow.." takes some working out. Maybe: "This allow Host B to use random data sent by Host A in the SYN, as part of the MAC algorithm; and similarly Host A to use random data sent by Host B in the SYN/ACK.

Next para - "and this is" -> "as"

Same para "MUST trigger an ACK in response" - maybe you could stress that this is an ordinary data ACK

P19 "The message in each case " -> The message for the MAC algorithm in each case

S3.3 para 2 & 3 could be re-phrased slightly:-
   During normal MPTCP operation, the Data Sequence Mapping defines how the sequence space on the subflow maps to
   the connection level; and the Data ACK acknowledges receipt of
   data at the connection level.  These functions are described in more
   detail in the following two subsections.
The Data Sequence Mapping and the Data ACK are signalled in the Data Sequence Signal (DSS). Either or both can be signalled in one DSS, dependent on the flags set.

P23 << The data sequence mapping also contains a checksum of the data that
   this mapping covers.  >> - to check I understand, this means you need all the data before you start to transmit it, so you can calculate the checksum?

P24 first para under fig 10, you could delete ". Furthermore," and put the rest of the sentence in (..)

Last para 3.3.1 "data-level length field to "
Data-level Length field of the DSS option to

P26 para 3, "that it is used" can be deleted

P26 para 4 << An MPTCP sender MUST only free data from the send buffer when it has
   been acknowledged by both a Data ACK received on any subflow and at
   the subflow level by any subflows the data was sent on.  >>
could be clearer
An MPTCP sender MUST NOT free data from the send buffer until it has
   been acknowledged by both a Data ACK received on any subflow and at
   the subflow level by all subflows the data was sent on.
[note, 2nd 'any' -> 'all']

Same thing in next para

P27 penult para - I think 'Data Sequence Signal' would be more accurate here than 'Mapping'. Also Length -> Data-level Length

P27 last line, think you mean 'DATA_FIN' and not FIN

3.3.4 para 4 << When deciding to accept packets at subflow level, normal TCP uses the
   sequence number in the packet and checks it against the allowed
   receive window.  >>
When deciding to accept packets at the subflow level, normal TCP checks the
   sequence number in the packet against the allowed
   receive window.

P28 last line
Could you spell out what rcv_next is [or at least reference B2.2.2, where it's called RCV.NXT]

3.3.4 last para 'maximum bandwidth-delay product of any of the paths' wonder if 'any one of the' would be clearer?

3.3.6 para 1 'the best behaviour' -> 'sensible behaviour'

3.3.6 last line - < For example, subflows that perform highly asymmetrically may be mis-
   diagnosed as underperforming.>
For example, a highly asymmetrically path may be mis-
   diagnosed as underperforming.

P33 para 1 'The signal applies to a single direction:
   the sender of this option, however, may >>
The signal applies to a single direction - and so the sender of this option may

Next para < This
   applies the given setting of B to all subflows that use the address
   identified by the given Address ID.  >
do you think it's worth saying that this means all subflows in this connection?

S3.4 para 2 < This design makes use of two methods of sharing such information,
   used simultaneously.  >
This design makes use of two methods of sharing such information; both can be used on a connection.

P35 para above Fig 12 as does the ephemeral port at the client - explain.

Same para, "signalling subflow' - should this  be 'initial subflow'?

Fig 12 caption - can delete '(shown for IPv4)' - as pic includes v6

3.4.1 last para < an MPTCP
   implementation MUST NOT treat duplicate ACKs with any MPTCP option
   apart from DSS as indications of congestion >>
"NOT ... any .. apart from" is ambiguous.

S3.6 - end of sentence 1 - add 'for instance they aren't blocked or mangled by a middlebox

P40 para 1 I couldn't parse this:-
(Note that these rules do not apply if an
   infinite mapping is included from the start - in which case, each end
   will send DSS options declaring the infinite mapping.)

3.6 penult para "fallback mode" - what is this?

S4, Duplicate ack. "To avoid any" -> To limit

S5, big para on p46 < used on subflow setup that verify that > used on subflow setup, in order to verify that

P47 line 1, "will allow" -> allows

S6 para 1 &   P54 para 2

S6 para 2 < Most middleboxes
   should just forward packets with new options unchanged>
Middleboxes should just forward packets with new options unchanged
[I think - (all) middleboxes aren't supposed to change options; most don't]

P49 Traffic normalisers - should you also mention about re-transmitting on the same subflow?

9.2 - maybe worth adding rfc editor note that hoping this will become rfc at the same time

Appendix C - point out that [DFIN] is shorthand for [DATA_FIN] - (I think!)

That's it, phew.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Yoshifumi Nishida
Sent: 18 April 2012 07:04
To: [email-masked]; multipathtcp
Subject: Re: [multipathtcp] comments on draft-ietf-mptcp-multiaddressed-07

Hi authors,

This draft requires to create new registries for subtypes of mptcp option and cryptographic algorithms handshake in MP_CAPABLE.
For this, you will need to add the description of the policy used for managing these registries in the IANA considerations section. Please check the section 4 of RFC5226 to see the available choices (Section 4.1) and update the texts based on the guideline (Section 4.2).

Thanks,
--
Yoshifumi Nishida

On Mon, Apr 16, 2012 at 11:53 PM, Yoshifumi Nishida <[email-masked]> wrote:
Hello,

Sorry for the long delay.
We believe it's good time to proceed to submit draft-ietf-mptcp-multiaddressed to IESG.

Thanks,
--
Yoshifumi

Page 13:

Page 17:

   -> Does this mean both local Address IDs and remote Address IDs advertized from the receiver?
       It might be better to clarify this.

Page 18:

   -> Since this is MUST, I think Figure 8 needs to include this ACK in the sequence.

   -> In case of MP_CAPABLE, the third packet is not retransmitted.
       But, in case of MP_JOIN, it's retransmitted. Readers might want to know the rationale about this.

Page 27:

   Essentially, a host MUST NOT FIN all functioning subflows unless it is safe to do so

    -> MUST NOT close?

Page 28:

   Both hosts SHOULD send FINS, as a courtesy to allow middleboxes to clean up state even if the subflow is failed.
   -> I'm not very sure which situations can be addressed by this..
   ..
   Note that a host may also send a FIN on an individual subflow to shut it down, but this impact is limited to the subflow in question.

Page 30:

    -> MUST?

    When doing this, a host must still retransmit the original data on the original subflow,

    -> MUST?

Page 37:

Page 40:

   fallback to regular TCP can become necessary at any point during a connection if a non-MPTCP-aware middlebox changes the data stream.

    -> The paragraph below this suggest when multiple subflows are in use, the affected subflow must be terminated, but not suggesting fallback.
         So, fallback seems not to be necessary?

   Therefore, it is not possible to recover the subflow, and the affected subflow must be immediately closed with an RST.

    -> MUST?

Page 47:

   If packets with the MP_JOIN option are dropped, the paths will simply not be used.

    -> MUST NOT be used?

Page 51:

Page 52:

---

From: <[email-masked]>
Date: Tue, 29 May 2012 09:34:56 +0100

All,
It's probably worth pointing out explicitly that Section 8 IANA Considerations has been updated.

-          With our option kind assignment (30)

-          RFCs will be used for future assignments of MPTCP option subtype values & flags for crypto algos

Alan,
In the crypto algos part of S8

<<   Note that the length of this field is not fixed; it is a definition
   of the meaning of each bit in this field (i.e. 0x2, 0x4, 0x8, 0x10,
   etc).  Future specifications may encroach on the non-cryptographic
   flags at the other end of this field so the number of flags available
   for cryptographic algorithm use may change.>>

I don't understand the "encroaching". S3.1 says the flag octet leftmost bit is for Checksum required and "The remaining bits are used for crypto algorithm negotiation."

Thanks

phil

Hi Phil,

Regards,
Alan

Thanks!
phil

--

[Alan] Fixed this.

P49 PEPs < MPTCP will

--

Throughout doc:

--

S1.1 <<The

Same thing in next para

P28 last line

Next para < This

S6 para 1 &   P54 para 2

That's it, phew.

Hi authors,

Thanks,
--
Yoshifumi Nishida

Thanks,
--
Yoshifumi

Page 13:

Page 17:

Page 18:

Page 27:

    -> MUST NOT close?

Page 28:

Page 30:

    -> MUST?

    -> MUST?

Page 37:

Page 40:

    -> MUST?

Page 47:

    -> MUST NOT be used?

Page 51:

Page 52:

---

From: Alan Ford <[email-masked]>
Date: Tue, 29 May 2012 11:14:38 +0100

Ah, I think that is a minor error. I mean encroaching on the cryptographic
bits.

This seems more appropriate: ³Future specifications may define additional
flags at the leftmost bits of this field, thus the number of bits available
for cryptographic negotiation may change.²

Regards,
Alan

On 29/05/2012 09:34, "[email-masked]" <[email-masked]> wrote:

> All,
> It¹s probably worth pointing out explicitly that Section 8 IANA Considerations
> has been updated.
> I don¹t understand the ³encroaching². S3.1 says the flag octet leftmost bit is
> for Checksum required and ³The remaining bits are used for crypto algorithm
> negotiation.²
>  
> Thanks
> phil
>  
>  
> 
> Many thanks for you comments. Anything I have not directly commented on below
> was straightforwardly addressed in 08. If there is anything you feel wasn¹t
> sufficiently addressed, please shout.
> 
> Regards,
> Alan
>