Subject: Re: [multipathtcp] Kernel changes need for payload encoding

From: Costin Raiciu <[email-masked]>
Date: Thu, 29 Jul 2010 10:38:32 +0200

> If we need protection against payload shrink/expansion this, I think copying the start/end/boundary bytes would be much less impactful, as that's a localized operation in a single segment, which has to be constructed by the CPU anyhow for the most part. 
> Note, however, that detection of that may be delayed by chunk bytes - what at the time of detection might already have been delivered to the app (and freed on the sender). Not doing so would cause a potential latency burden (and addressing this with smaller chunks would, in turn, result in much worse performance, potentially lower than std tcp...)
> 
> Thus the question is, what shall be done in that case?
> 
> As we don't want to prevent middleboxes from meddling with MPTCP for lawful purposes (ie. Prevention of JOIN which may circumvent a firewall), and also some meddling of the payload by middleboxes is of some merit (wifi providers run their business model on that :) ), just making sure that the bytes don't shift should suffice.
> 
> And I think that could be done by the boundary/start bytes getting copied (it would also detect repagaging, and fail if that happens, btw), in the same starting seg carrying the MPTCP options (of chunk size, DATA_SEG, ...)

I agree that only byte shifting needs to be detected. However, as Mark said, I don't think anything less than a CRC is robust.

For instance, what if there is a large region of 0 bytes in the payload, and a few bytes get inserted before that? It is likely that won't be detected by just copying the last byte/bytes in the chunk.

> BTW: Some middleboxes might not appear to do repackaging, but do chance Sequence numbers - thus a implicit mapping of DATA_SEG to the first payload byte might be better, than a explicit DATA_SEG to sub-flow SEG mapping...

The subflow sequence numbers in the data sequence mapping are relative to the subflow ISN because of this reason.

Cheers,
Costin

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Thu, 29 Jul 2010 09:47:39 +0100

> -----Original Message-----
> From: Costin Raiciu [mailto:[email-masked]]

> > 
> > And I think that could be done by the boundary/start bytes getting 
> > copied (it would also detect repagaging, and fail if that happens, 
> > btw), in the same starting seg carrying the MPTCP options (of chunk 
> > size, DATA_SEG, ...)
>
> I agree that only byte shifting needs to be detected. 
> However, as Mark said, I don't think anything less than a CRC is 
> robust.
>
> For instance, what if there is a large region of 0 bytes in the 
> payload, and a few bytes get inserted before that? It is likely that 
> won't be detected by just copying the last byte/bytes in the chunk.
> 

I agree with that statement. But the question to be answered, at least
as it appears to me, is to answer what is necessary for MPTCP to run
reliable. If an appliaction requires full data integrity, with no
middlebox allowed to meddle with it, this appears to me as being outside
the scope of the MPTCP protocol.

And from this high level view, I don't think that the answer to that
question is Yes.

What is in-scope, however, is to provide a way to reliably run MPTCP
over existing infrastructure.

The easiest way to make sure that MPTCP can properly run would be to NOT
allow any size changes in the payload (DATA). (Which can be verified by
the simple data start check suggestion; segment boundary stuff will
probably also be good enough in most cases).

But that would only be the easiest way to spec and implement it. For
some very good reasons (ie. Http/mptcp traffic), it might actually be
better to have protocol extensions, which allow the adjustment of chunk
sizes while the chunks are on their way; (The DATA could still be
adjusted on the receiver side, as long as a chunk has not been delivered
- even if other different chunks are already sitting in the receiver
buffer. Sending chunks simultaneously via two sub-flows would be an
interesting corner case though). Such information, however, will
probably need to be made available to both receiver (explicity by a
middlebox, setting some to-be-defined signal; implicit by the receiver
maintaining a "translation" table for DATA_ACK vs DATA_SEQ). 

If that needs to be in an initial protocol spec, needs to be thought
about.

To repeat, I'm against a MPTCP chunk level DATA CRC (or segment level
DATA CRC for MPTCP), mainly for implementation reasons.

Regards,
   Richard

---

From: "Ford, Alan" <[email-masked]>
Date: Thu, 29 Jul 2010 10:04:53 +0100

Hi all,

> > As we don't want to prevent middleboxes from meddling with MPTCP for
lawful
> > purposes (ie. Prevention of JOIN which may circumvent a firewall),
and also
> > some meddling of the payload by middleboxes is of some merit (wifi
providers
> > run their business model on that :) ), just making sure that the
bytes don't
> > shift should suffice.
copied
> > (it would also detect repagaging, and fail if that happens, btw), in
the same
> > starting seg carrying the MPTCP options (of chunk size, DATA_SEG,
...)
> 
> I agree that only byte shifting needs to be detected. However, as Mark
said,
> I don't think anything less than a CRC is robust.
> 
payload, and
> a few bytes get inserted before that? It is likely that won't be
detected by
> just copying the last byte/bytes in the chunk.

It may not be /immediately/ detectable, but it will be detectable soon
after. I wonder if at the cost of very small delay to delivering data to
the application, we can still get away with this. If, for some reason,
we do not detect the loss of sync at the end of the data segment, there
are other options:

* Data is shorter, in which case it won't be delivered to the
application until more data arrives. If we did a combination (e.g. xor)
of start and finish data, this may be detected after the full next
segment arrives (but this is too late for the application). But if the
data is a large region of 0 bytes, will it matter that the "wrong" data
is delivered the application, since it's the same as what it expects -
the error will be detected at the

* Data is longer, in which case we may receive data for which we do not
have a valid mapping (and thus the subflow is considered failed), or a
combination of first + last bytes will detect the error on the next
segment.

In these corner cases, an implementation could kill the whole
connection. Not ideal, but if it's not possible to recover from such a
failure it's best to kill the connection so the application re-tries.
Applications that care about resilience will be able to cope with
failures anyway. I'm concerned about whether damage can occur here, I'm
not sure either way yet.

It would be interesting to know more about the prevalence of middleboxes
may cause these kinds of problems. Remember we don't want to stop boxes
doing this, just stop them causing unintentional damage to the data
stream due to MPTCP.

Regards,
Alan